Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 4
Encontrada(s) 1 página(s)

RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

The Cambrian Explosion of Mixed-Precision Matrix Multiplication for Quantized Deep Learning Inference

  • Martínez, Héctor
  • Igual, Francisco D.
  • Castelló, Adrián|||0000-0002-8576-8451
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] Recent advances in deep learning (DL) have promoted to a shift from traditional 64-bit floating point (FP64) arithmetic for scientific computing toward reduced-precision formats-such as FP16, BF16, or even 8-bit integers-combined with mixed-precision arithmetic. This transition enhances computational throughput, reduces memory and bandwidth usage, and improves energy efficiency, offering significant advantages for resource-constrained edge devices. To support this shift, hardware architectures have evolved accordingly, now including adapted ISAs (Instruction Set Architectures) that expose mixed-precision vector units and matrix engines tailored for DL workloads. At the heart of many DL and scientific computing tasks is the general matrix-matrix multiplication (GEMM), a fundamental kernel historically optimized using fused multiply-add (FMA) vector instructions on SIMD (single instruction, multiple data) units. However, as hardware moves toward mixed-precision dot (or inner)-product-centric operations optimized for quantized inference, these legacy approaches are being phased out. In response to this, our paper revisits the conventional, high-performance implementation of GEMM and describes strategies for adapting it to mixed integer precision (MIP) arithmetic across modern ISAs, including x86_64, Arm, and RISC-V. Concretely, we illustrate novel micro-kernel designs and data layouts that better exploit today's specialized hardware and demonstrate significant performance gains from MIP arithmetic over floatingpoint implementations across three representative CPUs. These contributions highlight a new era of GEMM optimization-driven by the demands of DL inference on heterogeneous architectures, marking what we term as the "Cambrian period" for matrix multiplication., This work is supported by grants PID2021-126576NB-I00, PID2023-146569NB-C2 and PID2024-158311NB-I00, funded by MCIN/AEI/10.13039/5011000 11033, by "ERDF A way of making Europe", Programa PROMETEO2023 (CIPROM/2022/20) by Conselleria de Innovacio, Universidades, Ciencia y Sociedad Digital from the Generalitat Valenciana, and by Ayudaa Primeros Proyectosde Investigacion (PAID-06-24) , Vicerrectorado de Investigacion de la Universitat Politecnica de Valencia. A. Castello is supported by the CIAPOS/2023/431 grant, and Fondo Social Europeo Plus 2021-2027 from the Generalitat Valenciana.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Parallel GEMM-based convolution for deep learning on multicore RISC-V processors

  • Ramírez-Betancourth, Cristian
  • Martínez, Héctor
  • Castelló, Adrián|||0000-0002-8576-8451
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] We address the efficient implementation of the convolution operator on the GAP8 parallel ultra-low power platform (PULP), a heterogeneous multi-core processor equipped with a fabric controller (FC); a cluster of eight compute cores; and a four-level memory hierarchy with scratchpads instead of conventional, hardware-assisted cache memories. Our solution for this platform transforms the convolution into a general matrix-matrix multiplication (gemm) via the lowering approach, demonstrating that it is possible to attain reasonable performance on the GAP8 by carefully adapting techniques such as tiling and loop parallelism, which are mainstream in the multi-threaded, cache-aware realization of gemm., This research was funded by projects PID2020-113656RB and TED2021-129334B-I00, supported by MCIN/AEI/10.13039/501100011033 and by the "European Union NextGenerationEU/PRTR". C. Ramirez is a "Santiago Grisolia" fellow supported by Generalitat Valenciana. A. Castello is a FJC2019-039222-I fellow supported by MCIN/AEI/10.13039/501100011033. H. Martinez is a POSTDOC_21_00025 postdoctoral fellow supported by Junta de Andalucia.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Communication-Avoiding Fusion of GEMM-Based Convolutions for Deep Learning in the RISC-V GAP8 MCU.

  • Ramírez-Betancourth, Cristian
  • Martínez-Pérez-Cejuela, Hector
  • Castelló, Adrián|||0000-0002-8576-8451
  • Quintana-Ortí, Enrique S.|||0000-0002-5454-165X
[EN] Incorporating deep learning (DL) technologies to the edge is crucial for improving the security, privacy, and energy efficiency of the Internet of Things (IoT). In this scenario, the limitations of edge devices in terms of power dissipation, memory capacity, and processing power require a careful selection and optimization of algorithms for IoT DL applications. In this line, our work focuses on the convolution operator, a key component in deep neural networks for signal processing and computer vision. Specifically, the work aims at the efficient implementation of the lowering-based implementation of this operator, on the GAP8 parallel ultra-low power platform (PULP), with the goal of mitigating the data transfer costs across the memory hierarchy. Our contributions include 1) an analytical model for estimating the parallel execution time, 2) the exploration of different configuration options, and 3) four variants of the algorithm that fuse several components to address memory bottlenecks in the method. Overall, our best-fused variant provides a speedup of up to 1.25× over the baseline algorithm when applied to infer MobileNet-v1+ImageNet and VGG9+CIFAR10 using 8 threads, and up to 1.34× for ResNet18+ImageNet., This work was supported in part by MCIN/AEI/10.13039/501100011033 under Project PID2020-113656RB and Project TED2021-129334B-I00, and in part by EU NextGenerationEU/PRTR. The work of Héctor Martínez was supported by the Junta de Andalucía (POSTDOC_21_00025).
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Optimising Convolutions for Deep Learning Inference on ARM Cortex-M Processors

  • Maciá, Antonio
  • Barrachina Mir, Sergio
  • Fabregat Llueca, Germán
  • Dolz, Manuel F.
We perform a series of optimisations on the convo lution operator within the ARM CMSIS-NN library to improve the performance of deep learning tasks on Arduino development boards equipped with ARM Cortex-M4 and M7 microcontrollers. To this end, we develop custom microkernels that efficiently handle the internal computations required by the convolution operator via the lowering approach and the direct method, and we design two techniques to avoid register spilling. We also take advantage of all the RAM on the Arduino boards by reusing it as a scratchpad for the convolution filters. The integration of these techniques into CMSIS-NN, when invoked by TensorFlow Lite for microcontrollers for quantised versions of VGG, SqueezeNet, ResNet, and MobileNet-like convolutional neural networks enhances the overall inference speed by a factor ranging from 1.13× to 1.50×.