Pasar al contenido principal

INTELIGENCIA SOSTENIBLE EN EL BORDE-UPV

PID2023-146569NB-C21

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal para Impulsar la Investigación Científico-Técnica y su Transferencia
Subprograma Subprograma Estatal de Generación de Conocimiento
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2023
Plan Estatal 2021-2023 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSITAT POLITÈCNICA DE VALÈNCIA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 7
Encontrada(s) 1 página(s)

RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Sparse Matrix-Vector Product on RISC-V Processors with SIMD units

  • Tomás Domínguez, Andrés Enrique
  • Siwinska, Patricia
  • Castelló, Adrián|||0000-0002-8576-8451
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
  • Martínez-Pérez, Héctor
  • Catalán, Sandra
  • Casas, Marc
[EN] The sparse matrix-vector multiplication (SpMV) kernel is a key kernel in scientific and engineering applications, forming the core of many iterative solvers for linear systems and eigenvalue problems. Due to its low arithmetic intensity and irregular memory access patterns, SpMV remains memory-bound on modern architectures, making its efficient implementation particularly challenging. This paper presents vectorized SpMV routines for RISC-V processors with SIMD support, exploiting the RISC-V Vector Extension (RVV 1.0). We implement and evaluate three storage formats-CSR (Compressed Sparse Row), SELL-p (a vector-friendly variant of ELLPACK), and JDS (Jagged Diagonal Storage)-providing low-level implementations that leverage RVV intrinsics. Performance is assessed on two commercial RISC-V platforms (CanMV-K230 and BananaPi F3) with 128-bit and 256-bit vector registers, and on the EPAC research system featuring 16,384-bit vectors. Results show that the vectorized routines significantly outperform scalar baselines, achieving a variety of speed-ups depending on the format and architecture. These findings highlight the potential of open RISC-V architectures for high-performance sparse linear algebra and provide a foundation for future vector-aware sparse kernel optimizations., This work was supported by the research project PID2023-146569NB-C2 of MCIN/ AEI/10.13039/501100011033. S. Catalán is supported by the grant RYC2021-033,973-I, funded by MCIN/ AEI/10. 13,039/501,100,011,033 and the "NextGenerationEU"/ PRTR and UJI-2023-04, funded by Universitat Jaume I. A. Castelló is supported by the CIAPOS/2023/431 grant, and Fondo Social Europeo Plus 2021 2027 from the Generalitat Valenciana and PAID_06_24 from Universitat Politècnica de València. Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Enhancing Transformer Performance and Portability through Auto-tuning Frameworks

  • Siwinska, Patricia
  • Lei, Jie|||0000-0002-9998-9503
  • Castelló, Adrián|||0000-0002-8576-8451
  • Alonso-Jordá, Pedro|||0000-0002-6882-6592
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] Transformer-based models such as BERT and GPT2 have become the foundation of many modern applications, yet their execution requires substantial computational and memory resources. To address these challenges, recent advances in compiler technology and hardware accelerators have introduced new opportunities for performance portability. In this work, we evaluate JAX and TVM as high-level frameworks that combine a NumPy-like programming model with Just-In-Time (JIT) or Ahead-of-Time (AOT) code optimization and compilation, enabling efficient execution across CPUs or GPUs, and in the case of JAX, on TPUs as well. We present systematic implementations of the core Transformer encoder and decoder blocks in JAX and TVM and compare their automatically optimized code against NumPy and CuPy baselines. Our experimental study covers heterogeneous hardware platforms (AMD CPU, NVIDIA GPUs, and Google TPUs) and multiple arithmetic precisions (FP32, BF16, INT8, and INT32). Results show that JAX and TVM deliver significant performance improvements over standard libraries, while reducing the programming effort required to adapt to different hardware. These findings demonstrate the potential of JIT- and AOT-oriented frameworks to serve as a portable and efficient solution for deploying Transformer workloads in diverse computing environments., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work was supported by the research project PID2023-146569NB-C2, of MCIN/ AEI/10.13039/501100011033, and PROGRAMA PROMETEO 2023-CIPROM/2022/20 of the Generalitat Valenciana. A. Castelló is supported by PAID_06_24 from Universitat Politècnica de València.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

The Cambrian Explosion of Mixed-Precision Matrix Multiplication for Quantized Deep Learning Inference

  • Martínez, Héctor
  • Igual, Francisco D.
  • Castelló, Adrián|||0000-0002-8576-8451
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] Recent advances in deep learning (DL) have promoted to a shift from traditional 64-bit floating point (FP64) arithmetic for scientific computing toward reduced-precision formats-such as FP16, BF16, or even 8-bit integers-combined with mixed-precision arithmetic. This transition enhances computational throughput, reduces memory and bandwidth usage, and improves energy efficiency, offering significant advantages for resource-constrained edge devices. To support this shift, hardware architectures have evolved accordingly, now including adapted ISAs (Instruction Set Architectures) that expose mixed-precision vector units and matrix engines tailored for DL workloads. At the heart of many DL and scientific computing tasks is the general matrix-matrix multiplication (GEMM), a fundamental kernel historically optimized using fused multiply-add (FMA) vector instructions on SIMD (single instruction, multiple data) units. However, as hardware moves toward mixed-precision dot (or inner)-product-centric operations optimized for quantized inference, these legacy approaches are being phased out. In response to this, our paper revisits the conventional, high-performance implementation of GEMM and describes strategies for adapting it to mixed integer precision (MIP) arithmetic across modern ISAs, including x86_64, Arm, and RISC-V. Concretely, we illustrate novel micro-kernel designs and data layouts that better exploit today's specialized hardware and demonstrate significant performance gains from MIP arithmetic over floatingpoint implementations across three representative CPUs. These contributions highlight a new era of GEMM optimization-driven by the demands of DL inference on heterogeneous architectures, marking what we term as the "Cambrian period" for matrix multiplication., This work is supported by grants PID2021-126576NB-I00, PID2023-146569NB-C2 and PID2024-158311NB-I00, funded by MCIN/AEI/10.13039/5011000 11033, by "ERDF A way of making Europe", Programa PROMETEO2023 (CIPROM/2022/20) by Conselleria de Innovacio, Universidades, Ciencia y Sociedad Digital from the Generalitat Valenciana, and by Ayudaa Primeros Proyectosde Investigacion (PAID-06-24) , Vicerrectorado de Investigacion de la Universitat Politecnica de Valencia. A. Castello is supported by the CIAPOS/2023/431 grant, and Fondo Social Europeo Plus 2021-2027 from the Generalitat Valenciana.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Latency-Critical Quantized Inference With Transformer Decoders on ARM and RISC-V CPUs

  • Martínez, Héctor
  • Catalán, Sandra
  • Mestre, Jose I.
  • Castelló, Adrián|||0000-0002-8576-8451
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] Large language models are transforming industries but face challenges due to their high computational and energy demands. Model compression via quantization mitigates these barriers by reducing the bit precision of parameters and arithmetic operations, enabling deployment on resource-constrained devices like smartphones and edge platforms. This article focuses on quantization applied to transformer decoders, which are critical for tasks, such as text generation and conversational artificial intelligence. Unlike encoders, decoders are constrained by memory due to their sequential processing nature and low arithmetic intensity. We propose optimizations targeting inference on low-power CPUs, emphasizing efficient linear layers with quantized data/arithmetic and cache optimization. Using two representative ARM and RISC-V platforms, we present optimized mixed-precision implementations of the matrix multiplication that outperform the instance of that computational kernel in popular libraries, such as basic linear algebra subprograms infrastructure software, XNNPACK and ARMCL. This work thus advances the understanding of the impact of quantization on transformer decoder efficiency, energy consumption and precision in edge environments., This work was supported by the Research Project MCIN/AEI/10.13039/501100011033 under Grant PID2023-146569NB-C2. The work of Hector Martinez was supported by the Junta de Andalucia under Grant POSTDOC_21_00025. The work of Sandra Catalan was supported in part by MCIN/AEI/10.13039/501100011033 under Grant RYC2021-033973-I, and in part by the Universitat Jaume I through "NextGenerationEU"/PRTR under Grant UJI-2023-04. The work of Adrian Castello was supported in part by Generalitat Valenciana under Grant CIAPOS/2023/431, and in part by the Fondo Social Europeo Plus 2021-2027 from the Generalitat Valenciana and PAID_06_24 from the Universitat Politecnica de Valencia.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Contribución de congreso (ConferenceOutput)

Migration of Ginkgo s Jacobi-Preconditioned CG Solver to Vector RISC-V

  • Siwinska, Patricia
  • Castelló, Adrián|||0000-0002-8576-8451
  • Martínez-Pérez, Héctor
[EN] The solution of large, sparse linear systems of equations lies at the core of many scientific and engineering computations, particularly those derived from the discretization of partial differential equations. Iterative methods such as Conjugate Gradient (CG), GMRES, and BiCGSTAB, combined with suitable preconditioners, offer high scalability and memory efficiency, yet their performance on modern CPU architectures depends critically on efficient utilization of vector units. This paper presents the development of SIMD implementations for several key computational kernels in the Ginkgo linear algebra library, namely the sparse matrix¿vector product, Jacobi preconditioner application, and Level-1 BLAS operations, targeting RISC-V architectures that support the RISC-V Vector Extension (RVV) 1.0 specification. We integrate these vectorized routines into Ginkgo¿s Jacobi-preconditioned CG solver and evaluate the performance of the resulting solver on the SpaceMiT K1 processor, a commercial multicore RISC-V CPU embedded into the BananaPi F3 board. Our experimental results using matrices from the SuiteSparse Matrix Collection demonstrate both the benefits of vectorization and the presence of a strong memory bandwidth bottleneck on the BananaPi F3 board that currently limits attainable speed-ups. Our findings highlight the potential of RVV-based SIMD acceleration for iterative solvers and outline directions for further optimization on emerging RISC-V hardware., This work was supported by the research project PID2023- 146569NB-C2 of MCIN/AEI/10.13039/501100011033.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Acceleration of the the MVS Workflow using Graphics Processors

  • Díaz-Cano, Roberto
  • Folch, Francesc
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
  • Alonso-Jordá, Pedro|||0000-0002-6882-6592
[EN] Computer Vision (CV) leverages artificial intelligence to analyse digital images, offering insights for a wide range of different applications. While CV software often relies on open-source libraries such as OpenCV, it is probably more common for this software to use custom codes. Creating particular solutions stems from the very nature of the specific CV problems being addressed but, despite these particularities, there are common links at the core that are either not addressed by generic CV libraries or require significant customisation for specific applications. Understanding the nature of real problems faced by a digital image analysis use case can contribute as much as solving a generic CV problem, and this is the aim of this paper. This article addresses the problem of migrating to CUDA a part of Multiscan Vision System, a complex CV workflow utilised in a real-world, industrial use case. The primary challenge lies in minimising the overhead due to data transfers between the host and GPU (graphics processing unit), or even within the device's memory itself. While the speed-up achieved may not rival that of other applications more suitable to GPU architecture (in particular, massively data parallel applications), the algorithms and data distribution proposed in this study effectively offload a substantial portion of the workflow to the GPU in the context of low (integer) arithmetic intensity and real-time constraints. This frees the CPU to handle other workflow components and increases the capability to incorporate more cameras, significantly boosting productivity and economic performance., This work was supported by the research project PID2023-146569NB-C21 of MCIN/AEI/10.13039/501100011033 and has also been partially funded by PROGRAMA PROMETEO 2023-CIPROM/2022/20.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Characterization of quantized inference with transformer encoders on low power CPUs

  • Martínez, Héctor
  • Catalán, Sandra
  • Castelló, Adrián|||0000-0002-8576-8451
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] Transformers, particularly large language models (LLMs), are revolutionizing applications in natural language processing and computer vision but at a high cost in memory, energy, and computational resources. Quantization has emerged as an effective compression method to alleviate these demands, reducing the bitwidth of model data and arithmetic precision to enable efficient inference on resource-constrained devices. This paper focuses on optimizing inference with transformer encoders on low-power general-purpose CPUs, as those often found in edge devices. Our key contributions include exposing the critical role of linear layers within transformer encoders on CPUs with a limited number of cores; developing mixed integer precision matrix multiplication on ARM and RISC-V CPUs; and evaluating performance impact and energy savings of quantized inference. In summary, this work highlights the advantages of applying quantization to transformer encoders on current single-core and multi-core low power CPUs, offering insights for efficient LLM deployment on edge platforms., The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: PID2023-146569NB-C2 and PID2020-113656RB-C22 of MCIN/AEI/10.13039/501100011033 and ERDF/UE, POSTDOC_21_00025 of Junta de Andalucia, RYC2021-033973-I of MCIN/AEI/10.13039/501100011033 and the "NextGenerationEU"/PRTR, UJI-2023-04 of Universitat Jaume I, and CIAPOS/2023/431 of Fondo Social Europeo Plus 2021-2027 from the Generalitat Valenciana.