Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 2
Encontrada(s) 1 página(s)

RUC. Repositorio da Universidade da Coruña
Publicación Contribución de congreso (ConferenceOutput)

Adapt-S: Effective DNN Pruning via Unified Accuracy and Performance Tuning

  • Castro, Roberto L.
  • Andrade, Diego
  • Fraguela, Basilio B.
Traballo presentado no: 2025 IEEE International Parallel and Distributed Processing Symposium (IPDPS), 03-07 June 2025, Milán, Italia © 2025 IEEE. This version of the paper has been accepted for publication. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The final published paper is available online at: https://doi.org/10.1109/10.1109/IPDPS64566.2025.00018, [Abstract]: Model sparsification has emerged as a promising approach to reducing model size with minimum impact on accuracy. This is achieved through the removal of some model parameters, a process also known as Deep Neural Network (DNN) pruning. The irregular nature of the generated sparse tensors poses a great challenge in the development of efficient GPU kernels optimized for these workloads. This challenge has been recently addressed through the use of hardware-aware semistructured sparsification methods designed to conform to specialized sparse formats and codesigned with template-based kernel implementations. These methods are commonly based on grouping the non-pruned values in blocks of a given size to generate regularity or on generating patterns that fit specialized hardware units. This pruning pattern-format-kernel triplet presents a high degree of tunability, both at the pruning and kernel sides, which can be used to fit certain accuracy-to-performance tradeoffs. On the pruning side, using larger blocks of consecutive non-pruned values favors performance over accuracy, as the weight selection for removal policy becomes less flexible. On the kernel side, recent studies have proven that the tuning of the configuration parameters of template-based multi-level tiling kernel implementations can yield an extra performance boost. This paper presents AdAPT-S, an autotuning system that generates DNN pruning recipes and optimized kernel configurations to fit an accuracy-to-performance specification. This is done through a cost model that integrates both aspects. AdAPT-S gets extra benefits from the exploitation of layer sensitivity by providing per-layer pruning recipes and kernel configurations. The results show that our approach can achieve superior accuracy-to-performance trade-offs and that this can be used to produce models that fit the user requirements., This research was supported by grants PID2019-104184RBI00 and PID2022-136435NB-I00, funded by MCIN/AEI/10.13039/501100011033, PID2022 also funded by “ERDF Away of making Europe”, EU, the predoctoral grant of Roberto L. Castro (FPU19/03974), and by the Xunta de Galicia cofounded by the European Regional Development Fund (ERDF) under the Consolidation Programme of Competitive Reference Groups (ED431C 2021/30). CITIC, as a center accredited for excellence within the Galician University System and a member of the CIGUS Network, receives subsidies from the Department of Education, Science, Universities, and Vocational Training of the Xunta de Galicia. Additionally, it is co-financed by the EU through the FEDER Galicia 2021-27 operational program (Ref. ED431G 2023/01) Innovation Study ESPLAG has received funding through the Inno4scale project, which is funded by the European High-Performance Computing Joint Undertaking (JU) under Grant Agreement No 101118139. The JU receives support from the EuropeanUnion’s Horizon Europe Programme., Xunta der Galicia; ED431C 2021/30, Xunta de Galicia; ED431G 2023/01
RUC. Repositorio da Universidade da Coruña
Publicación Contribución de congreso (ConferenceOutput)

VENOM: A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Cores

  • Castro, Roberto L.
  • Ivanov, Andrei
  • Andrade, Diego
  • Ben-Nun, Tal
  • Fraguela, Basilio B.
  • Hoefler, Torsten
© 2023 Autores | ACM. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in International Conference for High Performance Computing, Networking, Storage and Analysis, https://doi.org/10.1145/3581784.3607087, [Abstract]: The increasing success and scaling of Deep Learning models demands higher computational efficiency and power. Sparsification can lead to both smaller models as well as higher compute efficiency, and accelerated hardware is becoming available. However, exploiting it efficiently requires kernel implementations, pruning algorithms, and storage formats, to utilize hardware support of specialized sparse vector units. An example of those are the NVIDIA's Sparse Tensor Cores (SPTCs), which promise a 2× speedup. However, SPTCs only support the 2:4 format, limiting achievable sparsity ratios to 50%. We present the V:N:M format, which enables the execution of arbitrary N:M ratios on SPTCs. To efficiently exploit the resulting format, we propose Spatha, a high-performance sparse-library for DL routines. We show that Spatha achieves up to 37× speedup over cuBLAS. We also demonstrate a second-order pruning technique that enables sparsification to high sparsity ratios with V:N:M and little to no loss in accuracy in modern transformers., This research was supported by the Ministry of Science and Innovation of Spain (grants PID2019-104184RB-I00 and PID2022-136435NB-I00, funded by MCIN/AEI/ 10.13039/501100011033, PID2022 also funded by "ERDF A way of making Europe", EU), the Ministry of Education (predoctoral grant of Roberto L. Castro, FPU19/03974), by Xunta de Galicia under the Consolidation Program of Competitive Reference Groups (ED431C 2021/30), and ERC grant PSAP, no. 101002047. We also acknowledge the support from CITIC, funded by Xunta de Galicia and FEDER funds of the EU (Centro de Investigación de Galicia accreditation 2019-2022, ED431G 2019/01). Finally, we thank the Swiss National Supercomputing Center (CSCS) and the Centro de Supercomputación de Galicia (CESGA) for the use of their computers., Xunta de Galicia; ED431C 2021/30, Xunta de Galicia; ED431G 2019/01