RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación
Artículo científico (JournalArticle)
Evaluating and accelerating vision transformers on GPU-based embedded edge AI systems
[EN] Many current embedded systems comprise heterogeneous computing components including quite powerful GPUs, which enables their application across diverse sectors. This study demonstrates the efficient execution of a medium-sized self-supervised audio spectrogram transformer (SSAST) model on a low-power system-on-chip (SoC). Through comprehensive evaluation, including real time inference scenarios, we show that GPUs outperform multi-core CPUs in inference processes. Optimization techniques such as adjusting batch size, model compilation with TensorRT, and reducing data precision significantly enhance inference time, energy consumption, and memory usage. In particular, negligible accuracy degradation is observed, with post-training quantization to 8-bit integers showing less than 1% loss. This research underscores the feasibility of deploying transformer neural networks on low-power embedded devices, ensuring efficiency in time, energy, and memory, while maintaining the accuracy of the results., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. This work has been supported by the Spanish Ministry of Science and Innovation under projects PID2020-113656RB-C21 and PID2022-137048OA-C43 funded by MCIN/AEI/10.13039/501100011033 and by "ERDF A way of making Europe" as well as TED2021-131401B-C21 and TED2021-131401A-C22.
Proyecto:
Agencia Estatal de Investigación / Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020 PID2020-113656RB-C21
,
Agencia Estatal de Investigación / Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023 PID2022-137048OA-C43
,
European Regional Development Fund TED2021-131401B-C21
,
European Regional Development Fund TED2021-131401A-C22