Pasar al contenido principal

Optimization of HPC and AI applications on emerging architectures

RYC2021-033973-I

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal para Desarrollar, Atraer y Retener Talento
Subprograma Subprograma Estatal de Incorporación
Convocatoria Ramón y Cajal
Año convocatoria 2021
Plan Estatal 2021-2023 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSITAT JAUME I DE CASTELLO

Resultados relacionados

Resultados totales (Incluyendo duplicados): 6
Encontrada(s) 1 página(s)

RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Cache-aware optimization of matrix multiplication and matrix factorizations on multicore processors

  • Martínez-Pérez, Héctor
  • Catalán, Sandra
  • Igual, Francisco D.
  • Herrero, José R.
  • Rodríguez-Sánchez, Rafael
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] This paper advocates for a careful customization of the special general matrix multiplication (GEMM) kernels that are invoked from blocked routines for several relevant matrix factorizations in LAPACK, in order to improve their performance on modern multicore processors with hierarchical cache memories. To achieve this, we leverage a refined analytical model to dynamically tune the cache configuration parameters of GEMM for these kernels, taking into account the matrix operands' dimensions, in order to improve cache occupation. In addition, toward the same goal, we accommodate a flexible development of architecture-specific micro-kernels for GEMM that allows us to select the option that, depending on the operands' dimensions, ameliorates cache utilization. Our experiments for the LU and QR factorizations on two platforms, equipped with ARM (NVIDIA Carmel) and x86 (AMD EPYC) multi-core processors, demonstrate the benefits of this approach in terms of a better cache utilization and, in general, higher performance. Moreover, they also reveal the delicate balance between optimizing for multi-threaded parallelism versus cache usage as well as the positive effects of software prefetching., This work was supported by grants PID2020-113656RB-C22, PID2019-107255GB, PID2021-126576NB-I00 and PID2021-123627OB-C52 of MCIN/AEI/10.13039/501100011033, by "ERDF A way of making Europe", and 2021-SGR-01007 of the Generalitat de Catalunya. Hector Martinez is a postdoctoral fellow supported by the Consejeria de Transformacion Economica, Industria, Conocimiento y Universidades de la Junta de Andalucia. Sandra Catalan was supported by the grant RYC2021-033973- I, funded by MCIN/AEI/10.13039/501100011033 and the European Union "NextGenerationEU"/PRTR.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Sparse Matrix-Vector Product on RISC-V Processors with SIMD units

  • Tomás Domínguez, Andrés Enrique
  • Siwinska, Patricia
  • Castelló, Adrián|||0000-0002-8576-8451
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
  • Martínez-Pérez, Héctor
  • Catalán, Sandra
  • Casas, Marc
[EN] The sparse matrix-vector multiplication (SpMV) kernel is a key kernel in scientific and engineering applications, forming the core of many iterative solvers for linear systems and eigenvalue problems. Due to its low arithmetic intensity and irregular memory access patterns, SpMV remains memory-bound on modern architectures, making its efficient implementation particularly challenging. This paper presents vectorized SpMV routines for RISC-V processors with SIMD support, exploiting the RISC-V Vector Extension (RVV 1.0). We implement and evaluate three storage formats-CSR (Compressed Sparse Row), SELL-p (a vector-friendly variant of ELLPACK), and JDS (Jagged Diagonal Storage)-providing low-level implementations that leverage RVV intrinsics. Performance is assessed on two commercial RISC-V platforms (CanMV-K230 and BananaPi F3) with 128-bit and 256-bit vector registers, and on the EPAC research system featuring 16,384-bit vectors. Results show that the vectorized routines significantly outperform scalar baselines, achieving a variety of speed-ups depending on the format and architecture. These findings highlight the potential of open RISC-V architectures for high-performance sparse linear algebra and provide a foundation for future vector-aware sparse kernel optimizations., This work was supported by the research project PID2023-146569NB-C2 of MCIN/ AEI/10.13039/501100011033. S. Catalán is supported by the grant RYC2021-033,973-I, funded by MCIN/ AEI/10. 13,039/501,100,011,033 and the "NextGenerationEU"/ PRTR and UJI-2023-04, funded by Universitat Jaume I. A. Castelló is supported by the CIAPOS/2023/431 grant, and Fondo Social Europeo Plus 2021 2027 from the Generalitat Valenciana and PAID_06_24 from Universitat Politècnica de València. Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Parallel gemm-based convolutions for deep learning on multicore ARM and RISC-V architectures

  • Martínez-Pérez, H.
  • Catalán, Sandra
  • Castelló, Adrián|||0000-0002-8576-8451
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] We present high performance, multi-threaded implementations of three GEMM-based convolution algorithms for multicore processors with ARM and RISC-V architectures. The codes are integrated into CONVLIB, a library that has the following unique features: (1) scripts to automatically generate a key component of GEMM, known as the micro-kernel, which is typically written in assembly language; (2) a modified analytical model to automatically tune the algorithms to the underlying cache architecture; (3) the ability to select four hyper-parameters: micro-kernel, cache parameters, parallel loop, and GEMM algorithm dynamically between calls to the library, without recompiling it; and (4) a driver to identify the best hyper-parameters. In addition, we provide a detailed performance evaluation of the convolution algorithms, on five ARM and RISC-V processors, and we publicly release the codes., This work was supported by the research project PID2020-113656RB-C22 of MCIN/AEI/10.13039/501100011033; and project PROMETEO 2023-CIPROM/2022/20 of the Generalitat Valenciana, Spain. Héctor Martínez is a POSTDOC_21_00025 fellow supported by Junta de Andalucía, Spain. Sandra Catalán was supported by the grant RYC2021-033973-I, funded by MCIN/AEI/10.13039/501100011033 and the European Union NextGenerationEU /PRTR, and UJI-2023-04, funded by Universitat Jaume I, Spain
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Latency-Critical Quantized Inference With Transformer Decoders on ARM and RISC-V CPUs

  • Martínez, Héctor
  • Catalán, Sandra
  • Mestre, Jose I.
  • Castelló, Adrián|||0000-0002-8576-8451
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] Large language models are transforming industries but face challenges due to their high computational and energy demands. Model compression via quantization mitigates these barriers by reducing the bit precision of parameters and arithmetic operations, enabling deployment on resource-constrained devices like smartphones and edge platforms. This article focuses on quantization applied to transformer decoders, which are critical for tasks, such as text generation and conversational artificial intelligence. Unlike encoders, decoders are constrained by memory due to their sequential processing nature and low arithmetic intensity. We propose optimizations targeting inference on low-power CPUs, emphasizing efficient linear layers with quantized data/arithmetic and cache optimization. Using two representative ARM and RISC-V platforms, we present optimized mixed-precision implementations of the matrix multiplication that outperform the instance of that computational kernel in popular libraries, such as basic linear algebra subprograms infrastructure software, XNNPACK and ARMCL. This work thus advances the understanding of the impact of quantization on transformer decoder efficiency, energy consumption and precision in edge environments., This work was supported by the Research Project MCIN/AEI/10.13039/501100011033 under Grant PID2023-146569NB-C2. The work of Hector Martinez was supported by the Junta de Andalucia under Grant POSTDOC_21_00025. The work of Sandra Catalan was supported in part by MCIN/AEI/10.13039/501100011033 under Grant RYC2021-033973-I, and in part by the Universitat Jaume I through "NextGenerationEU"/PRTR under Grant UJI-2023-04. The work of Adrian Castello was supported in part by Generalitat Valenciana under Grant CIAPOS/2023/431, and in part by the Fondo Social Europeo Plus 2021-2027 from the Generalitat Valenciana and PAID_06_24 from the Universitat Politecnica de Valencia.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Experience-guided, mixed-precision matrix multiplication with apache TVM for ARM processors

  • Castelló, Adrián|||0000-0002-8576-8451
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
  • Martínez, Héctor
  • Catalán, Sandra
  • Igual Peña, Francisco D.
[EN] Deep learning (DL) generates new computational tasks that are different from those encountered in classical scientific applications. In particular, DL training and inference require general matrix multiplications (gemm) with matrix operands that are far from large and square as in other scientific fields. In addition, DL models gain arithmetic/storage complexity, and as a result, reduced precision via quantization is now mainstream for inferring DL models in edge devices. Automatic code generation addresses these new types of gemm by (1) improving portability between different hardware with only one base code; (2) supporting mixed and reduced precision; and (3) enabling auto-tuning methods that, given a base operation, perform a (costly) optimization search for the best schedule. In this paper, we rely on Apache TVM to generate an experience-guided gemm that provides performance competitive with the TVM auto-scheduler, while reducing tuning time by a factor of 48x., This work received funding from projects PID2020-113656RB and PID2021-126576NB-I00 of MCIN/AEI/ 10.13039/501100011033; PROMETEO 2023-CI PROM/2022/20. H. Martinez is a POSTDOC_21_00025 postdoctoral fellow supported by Junta de Andalucia. S. Catalan is supported by the grant RYC2021-033973-I, funded by MCIN/AEI/10. 13039/501100011033 and the "NextGenerationEU"/ PRTR, and UJI-2023-04, funded by Universitat Jaume I.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Characterization of quantized inference with transformer encoders on low power CPUs

  • Martínez, Héctor
  • Catalán, Sandra
  • Castelló, Adrián|||0000-0002-8576-8451
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] Transformers, particularly large language models (LLMs), are revolutionizing applications in natural language processing and computer vision but at a high cost in memory, energy, and computational resources. Quantization has emerged as an effective compression method to alleviate these demands, reducing the bitwidth of model data and arithmetic precision to enable efficient inference on resource-constrained devices. This paper focuses on optimizing inference with transformer encoders on low-power general-purpose CPUs, as those often found in edge devices. Our key contributions include exposing the critical role of linear layers within transformer encoders on CPUs with a limited number of cores; developing mixed integer precision matrix multiplication on ARM and RISC-V CPUs; and evaluating performance impact and energy savings of quantized inference. In summary, this work highlights the advantages of applying quantization to transformer encoders on current single-core and multi-core low power CPUs, offering insights for efficient LLM deployment on edge platforms., The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: PID2023-146569NB-C2 and PID2020-113656RB-C22 of MCIN/AEI/10.13039/501100011033 and ERDF/UE, POSTDOC_21_00025 of Junta de Andalucia, RYC2021-033973-I of MCIN/AEI/10.13039/501100011033 and the "NextGenerationEU"/PRTR, UJI-2023-04 of Universitat Jaume I, and CIAPOS/2023/431 of Fondo Social Europeo Plus 2021-2027 from the Generalitat Valenciana.