Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 5
Encontrada(s) 1 página(s)

RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

OpenCNN: A Winograd Minimal Filtering Algorithm Implementation in CUDA

  • Castro, Roberto L.
  • Andrade, Diego
  • Fraguela, Basilio B.
[Abstract] Improving the performance of the convolution operation has become a key target for High Performance Computing (HPC) developers due to its prevalence in deep learning applied mainly to video processing. The improvement is being pushed by algorithmic and implementation innovations. Algorithmically, the convolution can be solved as it is mathematically enunciated, but other methods allow to transform it into a Fast Fourier Transform (FFT) or a GEneral Matrix Multiplication (GEMM). In this latter group, the Winograd algorithm is a state-of-the-art variant that is specially suitable for smaller convolutions. In this paper, we present openCNN, an optimized CUDA C++ implementation of the Winograd convolution algorithm. Our approach achieves speedups of up to 1.76× on Turing RTX 2080Ti and up to 1.85× on Ampere RTX 3090 with respect to Winograd convolution in cuDNN 8.2.0. OpenCNN is released as open-source software., This research was supported by the Ministry of Science and Innovation of Spain (PID2019-104184RB-I00, AEI/FEDER/EU, 10.13039/501100011033) and the predoctoral grant of Roberto L. Castro (FPU19/03974). and by the Xunta de Galicia co-founded by the European Regional Development Fund (ERDF) under the Consolidation Programme of Competitive Reference Groups (ED431C 2021/30). CITIC, Centro de Investigación de Galicia ref. ED431G 2019/01, receives financial support from Consellería de Educación, Universidade e Formación Profesional, Xunta de Galicia, through the ERDF (80%) and Secretaría Xeral de Universidades (20%), Xunta de Galicia; ED431C 2021/30, Xunta de Galicia; ED431G 2019/01
RUC. Repositorio da Universidade da Coruña
Publicación Contribución de congreso (ConferenceOutput)

Adapt-S: Effective DNN Pruning via Unified Accuracy and Performance Tuning

  • Castro, Roberto L.
  • Andrade, Diego
  • Fraguela, Basilio B.
Traballo presentado no: 2025 IEEE International Parallel and Distributed Processing Symposium (IPDPS), 03-07 June 2025, Milán, Italia © 2025 IEEE. This version of the paper has been accepted for publication. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The final published paper is available online at: https://doi.org/10.1109/10.1109/IPDPS64566.2025.00018, [Abstract]: Model sparsification has emerged as a promising approach to reducing model size with minimum impact on accuracy. This is achieved through the removal of some model parameters, a process also known as Deep Neural Network (DNN) pruning. The irregular nature of the generated sparse tensors poses a great challenge in the development of efficient GPU kernels optimized for these workloads. This challenge has been recently addressed through the use of hardware-aware semistructured sparsification methods designed to conform to specialized sparse formats and codesigned with template-based kernel implementations. These methods are commonly based on grouping the non-pruned values in blocks of a given size to generate regularity or on generating patterns that fit specialized hardware units. This pruning pattern-format-kernel triplet presents a high degree of tunability, both at the pruning and kernel sides, which can be used to fit certain accuracy-to-performance tradeoffs. On the pruning side, using larger blocks of consecutive non-pruned values favors performance over accuracy, as the weight selection for removal policy becomes less flexible. On the kernel side, recent studies have proven that the tuning of the configuration parameters of template-based multi-level tiling kernel implementations can yield an extra performance boost. This paper presents AdAPT-S, an autotuning system that generates DNN pruning recipes and optimized kernel configurations to fit an accuracy-to-performance specification. This is done through a cost model that integrates both aspects. AdAPT-S gets extra benefits from the exploitation of layer sensitivity by providing per-layer pruning recipes and kernel configurations. The results show that our approach can achieve superior accuracy-to-performance trade-offs and that this can be used to produce models that fit the user requirements., This research was supported by grants PID2019-104184RBI00 and PID2022-136435NB-I00, funded by MCIN/AEI/10.13039/501100011033, PID2022 also funded by “ERDF Away of making Europe”, EU, the predoctoral grant of Roberto L. Castro (FPU19/03974), and by the Xunta de Galicia cofounded by the European Regional Development Fund (ERDF) under the Consolidation Programme of Competitive Reference Groups (ED431C 2021/30). CITIC, as a center accredited for excellence within the Galician University System and a member of the CIGUS Network, receives subsidies from the Department of Education, Science, Universities, and Vocational Training of the Xunta de Galicia. Additionally, it is co-financed by the EU through the FEDER Galicia 2021-27 operational program (Ref. ED431G 2023/01) Innovation Study ESPLAG has received funding through the Inno4scale project, which is funded by the European High-Performance Computing Joint Undertaking (JU) under Grant Agreement No 101118139. The JU receives support from the EuropeanUnion’s Horizon Europe Programme., Xunta der Galicia; ED431C 2021/30, Xunta de Galicia; ED431G 2023/01
RUC. Repositorio da Universidade da Coruña
Publicación Contribución de congreso (ConferenceOutput)

VENOM: A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Cores

  • Castro, Roberto L.
  • Ivanov, Andrei
  • Andrade, Diego
  • Ben-Nun, Tal
  • Fraguela, Basilio B.
  • Hoefler, Torsten
© 2023 Autores | ACM. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in International Conference for High Performance Computing, Networking, Storage and Analysis, https://doi.org/10.1145/3581784.3607087, [Abstract]: The increasing success and scaling of Deep Learning models demands higher computational efficiency and power. Sparsification can lead to both smaller models as well as higher compute efficiency, and accelerated hardware is becoming available. However, exploiting it efficiently requires kernel implementations, pruning algorithms, and storage formats, to utilize hardware support of specialized sparse vector units. An example of those are the NVIDIA's Sparse Tensor Cores (SPTCs), which promise a 2× speedup. However, SPTCs only support the 2:4 format, limiting achievable sparsity ratios to 50%. We present the V:N:M format, which enables the execution of arbitrary N:M ratios on SPTCs. To efficiently exploit the resulting format, we propose Spatha, a high-performance sparse-library for DL routines. We show that Spatha achieves up to 37× speedup over cuBLAS. We also demonstrate a second-order pruning technique that enables sparsification to high sparsity ratios with V:N:M and little to no loss in accuracy in modern transformers., This research was supported by the Ministry of Science and Innovation of Spain (grants PID2019-104184RB-I00 and PID2022-136435NB-I00, funded by MCIN/AEI/ 10.13039/501100011033, PID2022 also funded by "ERDF A way of making Europe", EU), the Ministry of Education (predoctoral grant of Roberto L. Castro, FPU19/03974), by Xunta de Galicia under the Consolidation Program of Competitive Reference Groups (ED431C 2021/30), and ERC grant PSAP, no. 101002047. We also acknowledge the support from CITIC, funded by Xunta de Galicia and FEDER funds of the EU (Centro de Investigación de Galicia accreditation 2019-2022, ED431G 2019/01). Finally, we thank the Swiss National Supercomputing Center (CSCS) and the Centro de Supercomputación de Galicia (CESGA) for the use of their computers., Xunta de Galicia; ED431C 2021/30, Xunta de Galicia; ED431G 2019/01
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

STuning-DL: Model-Driven Autotuning of Sparse GPU Kernels for Deep Learning

  • Castro, Roberto L.
  • Andrade, Diego
  • Fraguela, Basilio B.
[Abstract]: The relentless growth of modern Machine Learning models has spurred the adoption of sparsification techniques to simplify their architectures and reduce the computational demands. Network pruning has demonstrated success in maintaining original network accuracy while shedding significant portions of the original weights. However, leveraging this sparsity efficiently remains challenging due to computational irregularities, particularly in GPU kernels. A new trend of template-based GPU kernels for semi-structured sparsity shows promise in efficiency but lacks autotuning capabilities to adapt to input dynamics, often underperforming in scenarios where they have not been meticulously hand-tuned. We present STuning-DL, the first pruning-aware autotuner for third-party template-based implementations enabling efficient optimization of sparse kernels for Deep Learning, spanning from high-level aspects (CUDA C++ level) down to GPU-native instructions specifics (assembly-level). STuning-DL tunes and optimizes at run-time sparse kernels’ performance for each input problem, yielding speedups of up to 5.42× on NVIDIA T4-16GB and up to 3.6× on NVIDIA A100-40GB GPU in sparse matrices from real world models compared to existing heuristics from sparse libraries like cuSparse and cuSparseLt., This work was supported by grant PID2022-136435NB-I00, funded by MCIN/AEI/10.13039/501100011033 and by ‘‘ERDF A way of making Europe’’, EU; also by Xunta de Galicia under the Consolidation Programme of Competitive Reference Groups, ref. ED431C 2021/30. The work of Roberto L. Castro was supported by a predoctoral grant from the Ministry of Science, Innovation and Universities, ref. FPU19/03974., Xunta de Galicia; ED431C 2021/30
RUC. Repositorio da Universidade da Coruña
Publicación Tesis doctoral (DoctoralThesis)

Development of Techniques to Optimize Automated Machine Learning Processes in High Performance Computing Systems

  • Castro, Roberto L.
[Resumo]: A Intelixencia Artificial (IA) emerxeu como un dos campos máis revolucionarios desta década. Parte do seu éxito débese aos avances na Computación de Altas Prestacións (HPC), que permitiron manexar modelos cada vez máis esixentes. Porén,a dependencia de infraestruturas computacionais a grande escala acentúa o desafío de optimizar a eficiencia computacional, xa que os custos enerxéticos e de recursos medran notablemente. Esta Tese aborda estes desafíos desenvolvendo novos mecanismos para acadar unha converxencia entre o Automated Machine Learning (AutoML) e o HPC. A súa motivación xorde de dúas observacións chave: a sobreparametrización dos modelos actuais e o baixo acoplamento entre a arquitectura destes modelos e o hardware no que se executan. Para abordar estes problemas, esta Tese explora a xeración automática de arquitecturas de redes simplificadas, pero precisas,que optimizan o uso de recursos. Concretamente, presenta novos algoritmos para apoda de arquitecturas de redes neurais e o desenvolvemento de rutinas de álxebra lineal altamente optimizadas, deseñadas para mellorar o seu rendemento en GPU. Para lograr isto, a nosa proposta define e emprega dous novos espazos de procura interrelacionados e altamente parametrizables, un centrado na precisión e o outro norendemento computacional. Facendo uso destes espazos de procura, desenvolvemos estratexias de autoaxuste unificado que permiten xerar automaticamente modelos cunha relación precisión-rendemento superior. A nosa avaliación demostra que é posible xerar modelos comprimidos cunha alta eficiencia computacional e cun impacto mínimo ou nulo na precisión., [Resumen]: La Inteligencia Artificial (IA) ha emergido como uno de los campos más revolucionarios de esta década. Parte de su éxito se debe a los avances en la Computación de Altas Prestaciones (HPC), que han permitido manejar modelos cada vez más exigentes. Sin embargo, la dependencia de infraestructuras computacionales a gran escala acentúa el desafío de optimizar la eficiencia computacional, ya que los costos energéticos y de recursos crecen significativamente. Esta Tesis aborda estos desafíos desarrollando nuevos mecanismos para lograr una convergencia entre el Automated Machine Learning (AutoML) y el HPC. Su motivación surge de dos observaciones clave: la sobreparametrización de los modelos actuales y el bajo acoplamiento entre la arquitectura de estos modelos y el hardware en el que se ejecutan. Para abordar estos problemas, esta Tesis explora la generación automática de arquitecturas de redes simplificadas, pero precisas, que optimizan el uso de recursos. Concretamente, presenta nuevos algoritmos para la poda de arquitecturas de redes neuronales e implementaciones de rutinas de álgebra lineal altamente optimizadas, diseñadas para mejorar su rendimiento en GPU. Para lograr esto, nuestra propuesta define dos nuevos espacios de búsqueda interrelacionados y altamente parametrizables, uno centrado en la precisión y el otro en el rendimiento computacional. Haciendo uso de estos espacios de búsqueda, desarrollamos estrategias de autoajuste unificado orientadas a generar automáticamente modelos con una relación precisión-rendimiento superior. Nuestra evaluación demuestra que es posible generar modelos comprimidos con una alta eficiencia computacional y un impacto mínimo o nulo en la precisión, [Abstract]: Artificial Intelligence (AI) has emerged as one of the most revolutionary fields ofthis decade. Part of its success is due to advancements in High Performance Computing(HPC), which have made it possible to handle increasingly demanding models.However, the reliance on large-scale computational infrastructures emphasizes the challenge of optimizing computational efficiency, as energy and resource costs growsignificantly. This Thesis tackles these challenges by developing new mechanisms to achieve a convergence between Automated Machine Learning (AutoML) and HPC. Its motivation arises from two key observations: the over-parameterization of current models and the limited coupling between the architecture of these models andthe hardware on which they are executed. To overcome these issues, this Thesis explores the automatic generation of simplified yet accurate network architectures that optimize resource usage. Specifically, it introduces novel algorithms for pruning neural network architectures and highly optimized implementations of linear algebraroutines designed to enhance their performance on GPU. To achieve this, ourproposal defines two new interrelated and highly parameterizable search spaces, one focused on accuracy and the other on computational performance. By leveragingthese search spaces, we develop new unified autotuning strategies that enable the automatic generation of models with optimized accuracy-to-performance tradeoffs.Our evaluation demonstrates that it is possible to generate compressed models with high computational efficiency while maintaining little to no impact on accuracy., Xunta de Galicia; ED431G/01, Xunta de Galicia; ED431C 2017/04, Xunta de Galicia; ED431G 2019/01, Xunta de Galicia; ED431C 2021/30, The means used to accomplish this Thesis have been the following: Working material, human, and financial support primarily by the Computer Architecture Group of the Universidade da Coruña, along with the Research Fellowship by the Ministry of Science, Innovation and Universities (FPU program, ref. FPU19/03974). • Additional funding: State funding by the Ministry of Economy and Competitiveness of Spain (project “Nuevos desafíos en computación de altas prestaciones: desde arquitecturas hasta aplicaciones (II)”, ref. TIN2016-75845-P), the Ministry of Science and Innovation of Spain (projects “Desafíos Actuales en HPC: Arquitecturas, Software y Aplicaciones”, ref. PID2019-104184RBI00 / AEI / 10.13039/501100011033, and “Arquitecturas, Frameworks y Aplicaciones de la Computación de Altas Prestaciones”, ref. PID2022- 136435NB-I00).