Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 44
Encontrada(s) 1 página(s)

UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

An energy-efficient near-data processing accelerator for DNNs to optimize memory accesses

  • Khabbazan, Bahareh|||0000-0001-6726-2804
  • Sabri Abrebekoh, Mohammad|||0000-0002-3113-5392
  • Riera Villanueva, Marc|||0000-0002-2768-5703
  • González Colás, Antonio María|||0000-0002-0009-0996
The constant growth of DNNs makes them challenging to implement and run efficiently on traditional computecentric architectures. Some accelerators have attempted to add more compute units and on-chip buffers to solve the memory wall problem without much success, and sometimes even worsening the issue since more compute units also require higher memory bandwidth. Prior works have proposed the design of memorycentric architectures based on the Near-Data Processing (NDP) paradigm. NDP seeks to break the memory wall by moving the computations closer to the memory hierarchy, reducing the data movements and their cost as much as possible. The 3D-stacked memory is especially appealing for DNN accelerators due to its high-density/low-energy storage and near-memory computation capabilities to perform the DNN operations massively in parallel. However, memory accesses remain as the main bottleneck for running modern DNNs efficiently. To improve the efficiency of DNN inference we present QeiHaN, a hardware accelerator that implements a 3D-stacked memory-centric weight storage scheme to take advantage of a logarithmic quantization of activations. In particular, since activations of FC and CONV layers of modern DNNs are commonly represented as powers of two with negative exponents, QeiHaN performs an implicit in-memory bit-shifting of the DNN weights to reduce memory activity. Only the meaningful bits of the weights required for the bit-shift operation are accessed. Overall, QeiHaN reduces memory accesses by 25% compared to a standard memory organization. We evaluate QeiHaN on a popular set of DNNs. On average, QeiHaN provides4.3¿speedup and3.5¿energy savings over a Neurocube-like accelerator., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI), Spain under grant PID2020-113172RB-I00, the Catalan Agency for University and Research (AGAUR), Spain under grant 2021 SGR 00383, and the ICREA Academia program, Spain., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

E-BATCH: Energy-efficient and high-throughput RNN batching

  • Silfa Feliz, Franyell Antonio|||0000-0002-1134-9908
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • González Colás, Antonio María|||0000-0002-0009-0996
Recurrent Neural Network (RNN) inference exhibits low hardware utilization due to the strict data dependencies across time-steps. Batching multiple requests can increase throughput. However, RNN batching requires a large amount of padding since the batched input sequences may vastly differ in length. Schemes that dynamically update the batch every few time-steps avoid padding. However, they require executing different RNN layers in a short time span, decreasing energy efficiency. Hence, we propose E-BATCH, a low-latency and energy-efficient batching scheme tailored to RNN accelerators. It consists of a runtime system and effective hardware support. The runtime concatenates multiple sequences to create large batches, resulting in substantial energy savings. Furthermore, the accelerator notifies it when the evaluation of an input sequence is done. Hence, a new input sequence can be immediately added to a batch, thus largely reducing the amount of padding. E-BATCH dynamically controls the number of time-steps evaluated per batch to achieve the best trade-off between latency and energy efficiency for the given hardware platform. We evaluate E-BATCH on top of E-PUR and TPU. E-BATCH improves throughput by 1.8× and energy efficiency by 3.6× in E-PUR, whereas in TPU, it improves throughput by 2.1× and energy efficiency by 1.6×, over the state-of-the-art., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, and the ICREA Academia program., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

ARAS: Adaptive low-cost ReRAM-based accelerator for DNNs

  • Sabri Abrebekoh, Mohammad|||0000-0002-3113-5392
  • Riera Villanueva, Marc|||0000-0002-2768-5703
  • González Colás, Antonio María|||0000-0002-0009-0996
Processing Using Memory (PUM) accelerators have the potential to perform Deep Neural Network (DNN) inference by using arrays of memory cells as computation engines. Among various memory technologies, ReRAM crossbars show promising performance in computing dot-product operations in the analog domain. Nevertheless, the expensive writing procedure of ReRAM cells has led researchers to design accelerators whose crossbars have enough capacity to store the full DNN. Given the tremendous and continuous increase in DNN model sizes, this approach is unfeasible for some networks, or inefficient due to the huge hardware requirements. Those accelerators lack the flexibility to adapt to any given DNN model, facing an adaptability challenge. To address this issue we introduce ARAS, a cost-effective ReRAM-based accelerator that employs an offline scheduler to adapt different DNNs to the resource-limited hardware. ARAS also overlaps the computation of a layer with the weight writing of several layers to mitigate the high writing latency of ReRAM. Furthermore, ARAS introduces three optimizations aimed at reducing the energy overheads of writing in ReRAM. Our key optimization capitalizes on the observation that DNN weights can be re-encoded to augment their similarity between layers, increasing the amount of bitwise values that are equal or similar when overwriting ReRAM cells and, hence, reducing the amount of energy required to update the cells. Overall, ARAS greatly reduces the ReRAM writing activity. We evaluate ARAS on a popular set of DNNs. ARAS provides up to 2. 2x speedup and 45% energy savings over a baseline PUM accelerator without any optimization. Compared to a TPU-like accelerator, ARAS provides up to 1. 5x speedup and 62% energy savings., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grants PID2020-113172RBI00 and PID2024-155476OB-I00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, the ICREA Academia program, and the FPI-UPC 2020 grant., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

LAWS: Locality-AWare Scheme for automatic speech recognition

  • Yazdani Aminabadi, Reza|||0000-0002-7949-6453
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • González Colás, Antonio María|||0000-0002-0009-0996
Automatic Speech Recognition (ASR) systems are changing the way people interact with different applications on mobile devices. Fulfilling such user-interactivity requires not only a highly accurate, large-vocabulary recognition system, but also a real-time, energy-efficient solution. However, these ASR systems need high memory bandwidth and power budget, which may be impractical for most of small form-factor battery-operated devices. In this article, we propose two combined techniques implemented on top of a state-of-the-art ASR accelerator in order to significantly reduce its energy consumption and memory requirements. First, by leveraging the locality among consecutive segments of the speech signal, we develop a Locality-AWare-Scheme (LAWS) which exploits the on-chip recently-explored data while removing most of the off-chip accesses during the ASR's decoding process. As a result, we remove up to 60 percent of ASR's workload. As the second step, we introduce an approach to improve LAWS's effectiveness by selectively adapting the amount of ASR's workload, based on run-time feedback. In particular, we exploit the fact that the confidence of the ASR system varies along the recognition process. When confidence is high, the ASR system can be more restrictive and reduce the amount of work. The end design including both techniques provides a saving of more than 87 percent in the memory requests and 2.3x reduction in energy consumption, and a speedup of 2.1x with respect to a state-of-the-art baseline design., This work has been supported in part by the CoCoUnit ERC Advanced Grant of the EUs Horizon 2020 program under Grant 833057, in part by the Spanish State Research Agency under Grant TIN2016-75344-R (AEI/FEDER, EU), and in part by the ICREA Academia program., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

SHARP: An adaptable, energy-efficient accelerator for recurrent neural networks

  • Yazdani Aminabadi, Reza|||0000-0002-7949-6453
  • Ruwase, Olatunji
  • Zhang, Minjia
  • He, Yuxiong
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • González Colás, Antonio María|||0000-0002-0009-0996
The effectiveness of Recurrent Neural Networks (RNNs) for tasks such as Automatic Speech Recognition has fostered interest in RNN inference acceleration. Due to the recurrent nature and data dependencies of RNN computations, prior work has designed customized architectures specifically tailored to the computation pattern of RNN, getting high computation efficiency for certain chosen model sizes. However, given that the dimensionality of RNNs varies a lot for different tasks, it is crucial to generalize this efficiency to diverse configurations. In this work, we identify adaptiveness as a key feature that is missing from today’s RNN accelerators. In particular, we first show the problem of low resource utilization and low adaptiveness for the state-of-the-art RNN implementations on GPU, FPGA, and ASIC architectures. To solve these issues, we propose an intelligent tiled-based dispatching mechanism for increasing the adaptiveness of RNN computation, in order to efficiently handle the data dependencies. To do so, we propose Sharp as a hardware accelerator, which pipelines RNN computation using an effective scheduling scheme to hide most of the dependent serialization. Furthermore, Sharp employs dynamic reconfigurable architecture to adapt to the model’s characteristics. Sharp achieves 2×, 2.8×, and 82× speedups on average, considering different RNN models and resource budgets, compared to the state-of-the-art ASIC, FPGA, and GPU implementations, respectively. Furthermore, we provide significant energy reduction with respect to the previous solutions, due to the low power dissipation of Sharp (321 GFLOPS/Watt)., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, and the ICREA Academia program., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

CGPA: Coarse-Grained Pruning of Activations for Energy-Efficient RNN Inference

  • Riera Villanueva, Marc|||0000-0002-2768-5703
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • González Colás, Antonio María|||0000-0002-0009-0996
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Recurrent neural networks (RNNs) perform element-wise multiplications across the activations of gates. We show that a significant percentage of activations are saturated and propose coarse-grained pruning of activations (CGPA) to avoid the computation of entire neurons, based on the activation values of the gates. We show that CGPA can be easily implemented on top of a TPU-like architecture with negligible area overhead, resulting in 12% speedup and 12% energy savings on average for a set of widely used RNNs., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Exploiting beam search confidence for energy-efficient speech recognition

  • Pinto Rivero, Dennis|||0000-0002-2175-9580
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • Riera Villanueva, Marc|||0000-0002-2768-5703
  • Cruz Díaz, Josep Llorenç|||0000-0001-5325-9153
  • González Colás, Antonio María|||0000-0002-0009-0996
With mobile and embedded devices getting more integrated in our daily lives, the focus is increasingly shifting toward human-friendly interfaces, making automatic speech recognition (ASR) a central player as the ideal means of interaction with machines. ASR is essential for many cognitive computing applications, such as speech-based assistants, dictation systems and real-time language translation. Consequently, interest in speech technology has grown in the last few years, with more systems being proposed and higher accuracy levels being achieved, even surpassing human accuracy. However, highly accurate ASR systems are computationally expensive, requiring on the order of billions of arithmetic operations to decode each second of audio, which conflicts with a growing interest in deploying ASR on edge devices. On these devices, efficient hardware acceleration is key for achieving acceptable performance. In this paper, we propose a technique to improve the energy efficiency and performance of ASR systems, focusing on low-power hardware for edge devices. We focus on optimizing the DNN-based acoustic model evaluation, as we have observed it to be the main bottleneck in popular ASR systems, by leveraging run-time information from the beam search. By doing so, we reduce energy and execution time of the acoustic model evaluation by 25.6  and 25.9 %, respectively, with negligible accuracy loss., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, the ICREA Academia program, and the Spanish MICINN Ministry under grant BES-2017-080605., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Omega-Test: A predictive early-Z culling to improve the graphics pipeline energy-efficiency

  • Corbalán Navarro, David
  • Aragón Alcaraz, Juan Luis
  • Anglada Sánchez, Martí|||0000-0002-1204-1841
  • Lucas Casamayor, Enrique de
  • Parcerisa Bundó, Joan Manuel|||0000-0001-5771-8118
  • González Colás, Antonio María|||0000-0002-0009-0996
The most common task of GPUs is to render images in real time. When rendering a 3D scene, a key step is to determine which parts of every object are visible in the final image. There are different approaches to solve the visibility problem, the Z-Test being the most common. A main factor that significantly penalizes the energy efficiency of a GPU, especially in the mobile arena, is the so-called overdraw, which happens when a portion of an object is shaded and rendered but finally occluded by another object. This useless work results in a waste of energy; however, a conventional Z-Test only avoids a fraction of it. In this paper we present a novel microarchitectural technique, the Omega-Test, to drastically reduce the overdraw on a Tile-Based Rendering (TBR) architecture. Graphics applications have a great degree of inter-frame coherence, which makes the output of a frame very similar to the previous one. The proposed approach leverages the frame-to-frame coherence by using the resulting information of the Z-Test for a tile (a buffer containing all the calculated pixel depths for a tile), which is discarded by nowadays GPUs, to predict the visibility of the same tile in the next frame. As a result, the Omega-Test early identifies occluded parts of the scene and avoids the rendering of non-visible surfaces eliminating costly computations and off-chip memory accesses. Our experimental evaluation shows average EDP savings in the overall GPU/Memory system of 26.4% and an average speedup of 16.3% for the evaluated benchmarks., This work has been supported by the the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency under grant TIN2016-75344-R (AEI/FEDER, EU) and the ICREA Academia program. D. Corbalan-Navarro has been supported by a PhD research fellowship from the University of Murcia., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Dynamic sampling rate: harnessing frame coherence in graphics applications for energy-efficient GPUs

  • Anglada Sánchez, Martí|||0000-0002-1204-1841
  • de Lucas Casamayor, Enrique
  • Parcerisa Bundó, Joan Manuel|||0000-0001-5771-8118
  • Aragón Alcaraz, Juan Luis
  • González Colás, Antonio María|||0000-0002-0009-0996
In real-time rendering, a 3D scene is modelled with meshes of triangles that the GPU projects to the screen. They are discretized by sampling each triangle at regular space intervals to generate fragments which are then added texture and lighting effects by a shader program. Realistic scenes require detailed geometric models, complex shaders, high-resolution displays and high screen refreshing rates, which all come at a great compute time and energy cost. This cost is often dominated by the fragment shader, which runs for each sampled fragment. Conventional GPUs sample the triangles once per pixel; however, there are many screen regions containing low variation that produce identical fragments and could be sampled at lower than pixel-rate with no loss in quality. Additionally, as temporal frame coherence makes consecutive frames very similar, such variations are usually maintained from frame to frame. This work proposes Dynamic Sampling Rate (DSR), a novel hardware mechanism to reduce redundancy and improve the energy efficiency in graphics applications. DSR analyzes the spatial frequencies of the scene once it has been rendered. Then, it leverages the temporal coherence in consecutive frames to decide, for each region of the screen, the lowest sampling rate to employ in the next frame that maintains image quality. We evaluate the performance of a state-of-the-art mobile GPU architecture extended with DSR for a wide variety of applications. Experimental results show that DSR is able to remove most of the redundancy inherent in the color computations at fragment granularity, which brings average speedups of 1.68x and energy savings of 40%., This work has been supported by the the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (Grant No. 833057), Spanish State Research Agency (MCIN/AEI) under Grant PID2020-113172RB-I00, the ICREA Academia program, and the Generalitat de Catalunya under Grant FI-DGR 2016. Funding was provided by Ministerio de Economía, Industria y Competitividad, Gobierno de España (Grant No. TIN2016-75344-R)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Dynamic selection and detection of spreading factors and channels for end-node devices of LoRa networks

  • Aliagas Castell, Carles|||0000-0003-2645-5444
  • Pueyo Centelles, Roger|||0000-0002-3133-0657
  • Meseguer Pallarès, Roc|||0000-0002-9414-646X
  • Millán Marco, Pere
  • Molina Clemente, Carlos Maria
LoRa and LoRaWAN facilitate effective long-range communication for IoT, with LoRa concentrating on transmission efficiency and low-power usage, while LoRaWAN addresses network architecture. Custom LoRa networks are ideal for small-scale applications due to their control and cost benefits. Nevertheless, large-scale deployments can experience message collisions, impacting efficiency and latency. LoRaWAN addresses this with Adaptive Data Rate (ADR), enhancing capacity and power efficiency. Our research introduced a novel strategy to improve LoRa network efficiency through a decentralized method. We employed multiple channels and spreading factors on client chips. This minimized contention and collisions. This approach allowed for dynamic adjustments, ensuring comprehensive communication control and enhanced performance in diverse environments. Our two-step mechanism, integrating heuristics and selection policies, provided flexible communication. We optimized parameters such as message size, transmission power, and bandwidth. This enhanced data rate, RSSI, and SNR, and reduced energy consumption. These results underscore the relevance of precise parameter tuning in achieving optimal LoRa performance., This work has been partially funded by the EU Horizon program under grant agreements No. 833057, No. 825184 and No. 101092644. It has also received support from the Ministry of Economic Affairs and Digital Transformation, together with the European Union-NextGenerationEU (within the PRTR and MRR frameworks), through the CLOUDLESS UNICO I+D CLOUD 2022. Addi- tional funding was provided by the Government of Spain under contracts PID2019-106774RB-C21, Electronics 2025, 14, 3341 34 of 36, PID2019-106774RB-C22, PID2020-113172RB-I00 and PID2024-155476OB-I00, and by the Government of Catalonia as Consolidated Research Groups 2021-SGR-383, 2021-SGR-109 and 2017-SGR-990., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

DNN pruning with principal component analysis and connection importance estimation

  • Riera Villanueva, Marc|||0000-0002-2768-5703
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • González Colás, Antonio María|||0000-0002-0009-0996
DNN pruning reduces memory footprint and computational work of DNN-based solutions to improve performance and energy-efficiency. An effective pruning scheme should be able to systematically remove connections and/or neurons that are unnecessary or redundant, reducing the DNN size without any loss in accuracy. In this paper we show that some of the most popular pruning schemes, such as the Near Zero Weights, require an extremely time-consuming iterative process that requires retraining the DNN many times to tune the pruning hyperparameters. Then, we propose a DNN pruning scheme based on Principal Component Analysis and relative importance of each neuron’s connection (PCA+DIRIE) that automatically finds the optimized DNN in one shot without requiring hand-tuning of multiple parameters. The experimental results show the effectiveness of our method on several benchmarks. Notably, on ImageNet, PCA+DIRIE can prune up to 60% of ResNet-50 with negligible impact on accuracy., This work has been supported by the CoCoUnit ERC Advanced Grantof the EU’s Horizon 2020 program (grant No 833057), the Spanish StateResearch Agency under grant PID2020-113172RB-I00 (MCIN/AEI), theICREA Academia program, and the Spanish Ministry of Education undergrant FPU15/02294., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Triangle Dropping: An occluded-geometry predictor for energy-efficient mobile GPUs

  • Corbalán Navarro, David
  • Aragon Alcaraz, Juan Luis
  • Anglada Sánchez, Martí|||0000-0002-1204-1841
  • Parcerisa Bundó, Joan Manuel|||0000-0001-5771-8118
  • González Colás, Antonio María|||0000-0002-0009-0996
This article proposes a novel micro-architecture approach for mobile GPUs aimed at early removing the occluded geometry in a scene by leveraging frame-to-frame coherence, thus reducing the overall energy consumption. Mobile GPUs commonly implement a Tile-Based Rendering (TBR) architecture that differentiates two main phases: the Geometry Pipeline, where all the geometry of a scene is processed; and the Raster Pipeline, where primitives are rendered in a framebuffer. After the Geometry Pipeline, only non-culled primitives inside the camera’s frustum are stored into the Parameter Buffer, a data structure stored in DRAM. However, among the non-culled primitives there is a significant amount that are rendered but non-visible at all, resulting in useless computations. On average, 60% of those primitives are completely occluded in our benchmarks. Despite TBR architectures use on-chip caches for the Parameter Buffer, about 46% of the DRAM traffic still comes from accesses to such buffer. The proposed Triangle Dropping technique leverages the visibility information computed along the Raster Pipeline to predict the primitives’ visibility in the next frame to early discard those that will be totally occluded, drastically reducing Parameter Buffer accesses. On average, our approach achieves overall 14.5% energy savings, 28.2% energy-delay product savings, and a speedup of 20.2%., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant no. 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00 (AEI/FEDER, EU), and the ICREA Academia program. D. Corbalán-Navarro has been also supported by a PhD research fellowship from the University of Murcia’s “Plan Propio de Investigación., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Towards efficient LUT-based PIM: A scalable and low-power approach for modern workloads

  • Khabbazan, Bahareh|||0000-0001-6726-2804
  • Riera Villanueva, Marc|||0000-0002-2768-5703
  • González Colás, Antonio María|||0000-0002-0009-0996
Data movement in memory-intensive workloads is several orders of magnitude more energy-consuming than computation. Frequent memory-processor transfers make this a critical bottleneck. Processing-using-memory (PuM) alleviates this by enabling in-memory computation and reducing data movement. In this paper, we propose Lama, a LUT-based PuM architecture that efficiently executes SIMD operations by supporting independent column accesses within each mat of a DRAM subarray. Lama minimizes the number of energy-intensive memory activation commands, which are the primary source of overhead in most PuM architectures. Besides, Lama supports up to 8-bit operand precision without decomposing computations, unlike prior PuM solutions, while incurring only a 2.47% area overhead. Our evaluation shows that Lama achieves an average performance improvement of 8.5× over state-of-the-art PuM architectures and a 3.8× improvement over CPU, along with energy efficiency gains of 6.9×/8×, respectively, for bulk 8-bit multiplication. We introduce LamaAccel, an HBM-based PuM accelerator that leverages Lama to accelerate LLM inference. By transforming MAC computations into energy-efficient addition and counting operations, LamaAccel achieves up to 9.3×/19.2× energy reduction and 4.8×/9.8× speedup over a TPU/GPU, as well as up to 5.8× lower energy consumption and 2.1× higher performance compared to a state-of-the-art PuM baseline., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grants PID2020-113172RB-I00 and PID2024-155476OBI00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, the ICREA Academia program, and the FPI-UPC 2020 grant., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

CREW: Computation reuse and efficient weight storage for hardware-accelerated MLPs and RNNs

  • Riera Villanueva, Marc|||0000-0002-2768-5703
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • González Colás, Antonio María|||0000-0002-0009-0996
Deep Neural Networks (DNNs) have achieved tremendous success for cognitive applications. The core operation in a DNN is the dot product between quantized inputs and weights. Prior works exploit the weight/input repetition that arises due to quantization to avoid redundant computations in Convolutional Neural Networks (CNNs). However, in this paper we show that their effectiveness is severely limited when applied to FullyConnected (FC) layers, which are commonly used in state-of-the-art DNNs, as it is the case of modern Recurrent Neural Networks (RNNs) and Transformer models. To improve energy-efficiency of FC computation we present CREW, a hardware accelerator that implements Computation Reuse and an Efficient Weight Storage mechanism to exploit the large number of repeated weights in FC layers. CREW first performs the multiplications of the unique weights by their respective inputs and stores the results in an on-chip buffer. The storage requirements are modest due to the small number of unique weights and the relatively small size of the input compared to convolutional layers. Next, CREW computes each output by fetching and adding its required products. To this end, each weight is replaced offline by an index in the buffer of unique products. Indices are typically smaller than the quantized weights, since the number of unique weights for each input tends to be much lower than the range of quantized weights, which reduces storage and memory bandwidth requirements. Overall, CREW greatly reduces the number of multiplications and provides significant savings in model memory footprint and memory bandwidth usage. We evaluate CREW on a diverse set of modern DNNs. On average, CREW provides 2.61x speedup and 2.42x energy savings over a TPU-like accelerator. Compared to UCNN, a state-of-art computation reuse technique, CREW achieves 2.10x speedup and 2.08x energy savings on average., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, the ICREA Academia program, and the Spanish Ministry of Education under grant FPU15/02294., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

A survey of near-data processing architectures for neural networks

  • Hassanpour, Mehdi|||0000-0002-1278-9382
  • Riera Villanueva, Marc|||0000-0002-2768-5703
  • González Colás, Antonio María|||0000-0002-0009-0996
Data-intensive workloads and applications, such as machine learning (ML), are fundamentally limited by traditional computing systems based on the von-Neumann architecture. As data movement operations and energy consumption become key bottlenecks in the design of computing systems, the interest in unconventional approaches such as Near-Data Processing (NDP), machine learning, and especially neural network (NN)-based accelerators has grown significantly. Emerging memory technologies, such as ReRAM and 3D-stacked, are promising for efficiently architecting NDP-based accelerators for NN due to their capabilities to work as both high-density/low-energy storage and in/near-memory computation/search engine. In this paper, we present a survey of techniques for designing NDP architectures for NN. By classifying the techniques based on the memory technology employed, we underscore their similarities and differences. Finally, we discuss open challenges and future perspectives that need to be explored in order to improve and extend the adoption of NDP architectures for future computing platforms. This paper will be valuable for computer architects, chip designers, and researchers in the area of machine learning., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, and the ICREA Academia program., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

SIMIL: SIMple Issue Logic for GPUs

  • Huerta Gañán, Rodrigo|||0000-0003-0052-7710
  • Cruz Díaz, Josep Llorenç|||0000-0001-5325-9153
  • Arnau Montañes, Jose-Maria
  • González Colás, Antonio María|||0000-0002-0009-0996
GPU architectures have become popular for executing general-purpose programs. In particular, they are some of the most efficient architectures for machine learning applications which are among the most trendy and demanding applications nowadays. This paper presents SIMIL (SIMple Issue Logic for GPUs), an architectural modification to the issue stage that replaces scoreboards with a Dependence Matrix to track dependencies among instructions and avoid data hazards. We show that a Dependence Matrix is more effective in the presence of repetitive use of source operands, which is common in many applications. Besides, a Dependence Matrix with minor extensions can also support a simplistic out-of-order issue. Evaluations on an NVIDIA Tesla V100-like GPU show that SIMIL provides a speed-up of up to 2.39 in some machine learning programs and 1.31 on average for various benchmarks, while it reduces energy consumption by 12.81%, with only 1.5% area overhead. We also show that SIMIL outperforms a recently proposed approach for out-of-order issue that uses register renaming., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, and the ICREA Academia program., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Irregular accesses reorder unit: improving GPGPU memory coalescing for graph-based workloads

  • Segura Salvador, Albert|||0000-0001-5084-9246
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • González Colás, Antonio María|||0000-0002-0009-0996
GPGPU architectures have become the dominant platform for massively parallel workloads, delivering high performance and energy efficiency for popular applications such as machine learning, computer vision or self-driving cars. However, irregular applications, such as graph processing, fail to fully exploit GPGPU resources due to their divergent memory accesses that saturate the memory hierarchy. To reduce the pressure on the memory subsystem for divergent memory-intensive applications, programmers must take into account SIMT execution model and memory coalescing in GPGPUs, devoting significant efforts in complex optimization techniques. Despite these efforts, we show that irregular graph processing still suffers from low GPGPU performance. We observe that in many irregular applications the mapping of data to threads can be safely changed. In other words, it is possible to relax the strict relationship between thread and data processed to reduce memory divergence. Based on this observation, we propose the Irregular accesses Reorder Unit (IRU), a novel hardware extension tightly integrated in the GPGPU pipeline. The IRU reorders data processed by the threads on irregular accesses to improve memory coalescing, i.e., it tries to assign data elements to threads as to produce coalesced accesses in SIMT groups. Furthermore, the IRU is capable of filtering and merging duplicated accesses, significantly reducing the workload. Programmers can easily utilize the IRU with a simple API, or let the compiler issue instructions from our extended ISA. We evaluate our proposal for state-of-the-art graph-based algorithms and a wide selection of applications. Results show that the IRU achieves a memory coalescing improvement of 1.32x and a 46% reduction in the overall traffic in the memory hierarchy, which results in 1.33x speedup and 13% energy savings on average, while incurring in a small 5.6% area overhead., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00 and the ICREA Academia program., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

A low-power, high-performance speech recognition accelerator

  • Yazdani, Reza
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • González Colás, Antonio María|||0000-0002-0009-0996
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Automatic Speech Recognition (ASR) is becoming increasingly ubiquitous, especially in the mobile segment. Fast and accurate ASR comes at high energy cost, not being affordable for the tiny power-budgeted mobile devices. Hardware acceleration reduces energy-consumption of ASR systems, while delivering high-performance. In this paper, we present an accelerator for largevocabulary, speaker-independent, continuous speech-recognition. It focuses on the Viterbi search algorithm representing the main bottleneck in an ASR system. The proposed design consists of innovative techniques to improve the memory subsystem, since memory is the main bottleneck for performance and power in these accelerators' design. It includes a prefetching scheme tailored to the needs of ASR systems that hides main memory latency for a large fraction of the memory accesses, negligibly impacting area. Additionally, we introduce a novel bandwidth-saving technique that removes off-chip memory accesses by 20 percent. Finally, we present a power saving technique that significantly reduces the leakage power of the accelerators scratchpad memories, providing between 8.5 and 29.2 percent reduction in entire power dissipation. Overall, the proposed design outperforms implementations running on the CPU by orders of magnitude, and achieves speedups between 1.7x and 5.9x for different speech decoders over a highly optimized CUDA implementation running on Geforce-GTX-980 GPU, while reducing the energy by 123-454x., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Mixture-of-Rookies: saving DNN computations by predicting ReLU outputs

  • Pinto Rivero, Dennis|||0000-0002-2175-9580
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • Riera Villanueva, Marc|||0000-0002-2768-5703
  • Cruz Díaz, Josep Llorenç|||0000-0001-5325-9153
  • González Colás, Antonio María|||0000-0002-0009-0996
Deep Neural Networks (DNNs) are widely used in many application domains. However, they require a vast amount of computations and memory accesses to deliver outstanding accuracy. In this paper, we propose a scheme to predict whether the output of each ReLu activated neuron will be a zero or a positive number in order to skip the computation of those neurons that will likely output a zero. Our predictor, named Mixture-of-Rookies, combines two inexpensive components. The first one exploits the high linear correlation between binarized (1-bit) and full-precision (8-bit) dot products, whereas the second component clusters together neurons that tend to output zero at the same time. We propose a novel clustering scheme based on analysis of angles, as the sign of the dot product of two vectors depends on the cosine of the angle between them. We implement our hybrid zero output predictor on top of a state-of-the-art DNN accelerator. Experimental results show that our scheme introduces a small area overhead of 5.3% while achieving a speedup of 1.2x and reducing energy consumption by 16.5% on average for a set of diverse DNNs., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RBI00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, the ICREA Academia program, and the Spanish MICINN Ministry under grant BES-2017-080605., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Hamun: an approximate computing method to prolong the lifespan of ReRAM-based accelerators

  • Sabri Abrebekoh, Mohammad|||0000-0002-3113-5392
  • Riera Villanueva, Marc|||0000-0002-2768-5703
  • González Colás, Antonio María|||0000-0002-0009-0996
ReRAM-based accelerators exhibit enormous potential to increase computational efficiency for DNN inference tasks, delivering significant performance and energy savings over traditional platforms. By incorporating adaptive scheduling, these accelerators dynamically adjust to DNN requirements, optimizing allocation of constrained hardware resources. However, ReRAM cells have limited endurance cycles due to wear-out from multiple updates for each inference execution, which shortens the lifespan of ReRAM-based accelerators and presents a practical challenge in positioning them as alternatives to conventional platforms like TPUs. Addressing these endurance limitations is essential for making ReRAM-based solutions viable for long-term, high-performance DNN inference. To address the lifespan limitations of ReRAM-based accelerators, we introduce Hamun, an approximate computing method designed to extend the lifespan of ReRAM-based accelerators through a range of optimizations. Hamun incorporates a novel mechanism that detects faulty cells due to wear-out and retires them, avoiding in this way their otherwise adverse impact on DNN accuracy. Moreover, Hamun extends the lifespan of ReRAM-based accelerators by adapting wear-leveling techniques across various abstraction levels of the accelerator and implementing a batch execution scheme to maximize ReRAM cell usage for multiple inferences. Additionally, Hamun introduces a new approximation method that leverages the fault tolerance characteristics of DNNs to delay the retirement of worn-out cells, reducing the performance penalty of retired cells and further extending the accelerator’s lifespan. On average, evaluated on a set of popular DNNs, Hamun demonstrates an improvement in lifespan of 13.2x over a state-of-the-art baseline. The main contributors to this improvement are the fault handling and batch execution schemes, which provide 4.6x and 2.6x lifespan improvements respectively., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RBI00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, and the ICREA Academia program., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

Design and evaluation of an ultra low-power human-quality speech recognition system

  • Pinto Rivero, Dennis|||0000-0002-2175-9580
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • González Colás, Antonio María|||0000-0002-0009-0996
Automatic Speech Recognition (ASR) has experienced a dramatic evolution since pioneer development of Bell Lab’s single-digit recognizer more than 50 years ago. Current ASR systems have taken advantage of the tremendous improvements in AI during the past decade by incorporating Deep Neural Networks into the system and pushing their accuracy to levels comparable to that of humans. This article describes and characterizes a representative ASR system with state-of-the-art accuracy and proposes a hardware platform capable of decoding speech in real-time with a power dissipation close to 1 Watt. The software is based on the so-called hybrid approach with a vocabulary of 200K words and RNN-based language model re-scoring, whereas the hardware consists of a commercially available low-power processor along with two accelerators used for the most compute-intensive tasks. The article shows that high performance can be obtained with very low power, enabling the deployment of these systems in extremely power-constrained environments such as mobile and IoT devices., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No. 833057), the Spanish State Research Agency under grant TIN2016-75344-R (AEI/FEDER, EU), the ICREA Academia program, and the Spanish MICINN Ministry under grant BES-2017-080605., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publication Artículo científico (JournalArticle)

LOCATOR: Low-power ORB accelerator for autonomous cars

  • Taranco Serna, Raúl|||0000-0002-1564-4365
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • González Colás, Antonio María|||0000-0002-0009-0996
Simultaneous Localization And Mapping (SLAM) is crucial for autonomous navigation. ORB-SLAM is a state-of-the-art Visual SLAM system based on cameras used for self-driving cars. In this paper, we propose a high-performance, energy-efficient, and functionally accurate hardware accelerator for ORB-SLAM, focusing on its most time-consuming stage: Oriented FAST and Rotated BRIEF (ORB) feature extraction. The Rotated BRIEF (rBRIEF) descriptor generation is the main bottleneck in ORB computation, as it exhibits highly irregular access patterns to local on-chip memories causing a high-performance penalty due to bank conflicts. We introduce a technique to find an optimal static pattern to perform parallel accesses to banks based on a genetic algorithm. Furthermore, we propose the combination of an rBRIEF pixel duplication cache, selective ports replication, and pipelining to reduce latency without compromising cost. The accelerator achieves a reduction in energy consumption of 14597× and 9609×, with respect to high-end CPU and GPU platforms, respectively., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020- 113172RB-I00, the ICREA Academia program and the FPU grant FPU18/04413, Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

A low-power, high-performance speech recognition accelerator

  • Yazdani, Reza
  • Arnau Montañés, José María
  • González Colás, Antonio María
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Automatic Speech Recognition (ASR) is becoming increasingly ubiquitous, especially in the mobile segment. Fast and accurate ASR comes at high energy cost, not being affordable for the tiny power-budgeted mobile devices. Hardware acceleration reduces energy-consumption of ASR systems, while delivering high-performance. In this paper, we present an accelerator for largevocabulary, speaker-independent, continuous speech-recognition. It focuses on the Viterbi search algorithm representing the main bottleneck in an ASR system. The proposed design consists of innovative techniques to improve the memory subsystem, since memory is the main bottleneck for performance and power in these accelerators' design. It includes a prefetching scheme tailored to the needs of ASR systems that hides main memory latency for a large fraction of the memory accesses, negligibly impacting area. Additionally, we introduce a novel bandwidth-saving technique that removes off-chip memory accesses by 20 percent. Finally, we present a power saving technique that significantly reduces the leakage power of the accelerators scratchpad memories, providing between 8.5 and 29.2 percent reduction in entire power dissipation. Overall, the proposed design outperforms implementations running on the CPU by orders of magnitude, and achieves speedups between 1.7x and 5.9x for different speech decoders over a highly optimized CUDA implementation running on Geforce-GTX-980 GPU, while reducing the energy by 123-454x., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

CREW: Computation reuse and efficient weight storage for hardware-accelerated MLPs and RNNs

  • Riera Villanueva, Marc
  • Arnau Montañés, José María
  • González Colás, Antonio María
Deep Neural Networks (DNNs) have achieved tremendous success for cognitive applications. The core operation in a DNN is the dot product between quantized inputs and weights. Prior works exploit the weight/input repetition that arises due to quantization to avoid redundant computations in Convolutional Neural Networks (CNNs). However, in this paper we show that their effectiveness is severely limited when applied to FullyConnected (FC) layers, which are commonly used in state-of-the-art DNNs, as it is the case of modern Recurrent Neural Networks (RNNs) and Transformer models. To improve energy-efficiency of FC computation we present CREW, a hardware accelerator that implements Computation Reuse and an Efficient Weight Storage mechanism to exploit the large number of repeated weights in FC layers. CREW first performs the multiplications of the unique weights by their respective inputs and stores the results in an on-chip buffer. The storage requirements are modest due to the small number of unique weights and the relatively small size of the input compared to convolutional layers. Next, CREW computes each output by fetching and adding its required products. To this end, each weight is replaced offline by an index in the buffer of unique products. Indices are typically smaller than the quantized weights, since the number of unique weights for each input tends to be much lower than the range of quantized weights, which reduces storage and memory bandwidth requirements. Overall, CREW greatly reduces the number of multiplications and provides significant savings in model memory footprint and memory bandwidth usage. We evaluate CREW on a diverse set of modern DNNs. On average, CREW provides 2.61x speedup and 2.42x energy savings over a TPU-like accelerator. Compared to UCNN, a state-of-art computation reuse technique, CREW achieves 2.10x speedup and 2.08x energy savings on average., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, the ICREA Academia program, and the Spanish Ministry of Education under grant FPU15/02294., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

LAWS: Locality-AWare Scheme for automatic speech recognition

  • Yazdani Aminabadi, Reza
  • Arnau Montañés, José María
  • González Colás, Antonio María
Automatic Speech Recognition (ASR) systems are changing the way people interact with different applications on mobile devices. Fulfilling such user-interactivity requires not only a highly accurate, large-vocabulary recognition system, but also a real-time, energy-efficient solution. However, these ASR systems need high memory bandwidth and power budget, which may be impractical for most of small form-factor battery-operated devices. In this article, we propose two combined techniques implemented on top of a state-of-the-art ASR accelerator in order to significantly reduce its energy consumption and memory requirements. First, by leveraging the locality among consecutive segments of the speech signal, we develop a Locality-AWare-Scheme (LAWS) which exploits the on-chip recently-explored data while removing most of the off-chip accesses during the ASR's decoding process. As a result, we remove up to 60 percent of ASR's workload. As the second step, we introduce an approach to improve LAWS's effectiveness by selectively adapting the amount of ASR's workload, based on run-time feedback. In particular, we exploit the fact that the confidence of the ASR system varies along the recognition process. When confidence is high, the ASR system can be more restrictive and reduce the amount of work. The end design including both techniques provides a saving of more than 87 percent in the memory requests and 2.3x reduction in energy consumption, and a speedup of 2.1x with respect to a state-of-the-art baseline design., This work has been supported in part by the CoCoUnit ERC Advanced Grant of the EUs Horizon 2020 program under Grant 833057, in part by the Spanish State Research Agency under Grant TIN2016-75344-R (AEI/FEDER, EU), and in part by the ICREA Academia program., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

E-BATCH: Energy-efficient and high-throughput RNN batching

  • Silfa Feliz, Franyell Antonio
  • Arnau Montañés, José María
  • González Colás, Antonio María
Recurrent Neural Network (RNN) inference exhibits low hardware utilization due to the strict data dependencies across time-steps. Batching multiple requests can increase throughput. However, RNN batching requires a large amount of padding since the batched input sequences may vastly differ in length. Schemes that dynamically update the batch every few time-steps avoid padding. However, they require executing different RNN layers in a short time span, decreasing energy efficiency. Hence, we propose E-BATCH, a low-latency and energy-efficient batching scheme tailored to RNN accelerators. It consists of a runtime system and effective hardware support. The runtime concatenates multiple sequences to create large batches, resulting in substantial energy savings. Furthermore, the accelerator notifies it when the evaluation of an input sequence is done. Hence, a new input sequence can be immediately added to a batch, thus largely reducing the amount of padding. E-BATCH dynamically controls the number of time-steps evaluated per batch to achieve the best trade-off between latency and energy efficiency for the given hardware platform. We evaluate E-BATCH on top of E-PUR and TPU. E-BATCH improves throughput by 1.8× and energy efficiency by 3.6× in E-PUR, whereas in TPU, it improves throughput by 2.1× and energy efficiency by 1.6×, over the state-of-the-art., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, and the ICREA Academia program., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Design and evaluation of an ultra low-power human-quality speech recognition system

  • Pinto Rivero, Dennis
  • Arnau Montañés, José María
  • González Colás, Antonio María
Automatic Speech Recognition (ASR) has experienced a dramatic evolution since pioneer development of Bell Lab’s single-digit recognizer more than 50 years ago. Current ASR systems have taken advantage of the tremendous improvements in AI during the past decade by incorporating Deep Neural Networks into the system and pushing their accuracy to levels comparable to that of humans. This article describes and characterizes a representative ASR system with state-of-the-art accuracy and proposes a hardware platform capable of decoding speech in real-time with a power dissipation close to 1 Watt. The software is based on the so-called hybrid approach with a vocabulary of 200K words and RNN-based language model re-scoring, whereas the hardware consists of a commercially available low-power processor along with two accelerators used for the most compute-intensive tasks. The article shows that high performance can be obtained with very low power, enabling the deployment of these systems in extremely power-constrained environments such as mobile and IoT devices., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No. 833057), the Spanish State Research Agency under grant TIN2016-75344-R (AEI/FEDER, EU), the ICREA Academia program, and the Spanish MICINN Ministry under grant BES-2017-080605., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

SIMIL: SIMple Issue Logic for GPUs

  • Huerta Gañán, Rodrigo
  • Cruz Díaz, Josep Llorenç
  • Arnau Montañes, Jose-Maria
  • González Colás, Antonio María
GPU architectures have become popular for executing general-purpose programs. In particular, they are some of the most efficient architectures for machine learning applications which are among the most trendy and demanding applications nowadays. This paper presents SIMIL (SIMple Issue Logic for GPUs), an architectural modification to the issue stage that replaces scoreboards with a Dependence Matrix to track dependencies among instructions and avoid data hazards. We show that a Dependence Matrix is more effective in the presence of repetitive use of source operands, which is common in many applications. Besides, a Dependence Matrix with minor extensions can also support a simplistic out-of-order issue. Evaluations on an NVIDIA Tesla V100-like GPU show that SIMIL provides a speed-up of up to 2.39 in some machine learning programs and 1.31 on average for various benchmarks, while it reduces energy consumption by 12.81%, with only 1.5% area overhead. We also show that SIMIL outperforms a recently proposed approach for out-of-order issue that uses register renaming., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, and the ICREA Academia program., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

CGPA: Coarse-Grained Pruning of Activations for Energy-Efficient RNN Inference

  • Riera Villanueva, Marc
  • Arnau Montañés, José María
  • González Colás, Antonio María
© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Recurrent neural networks (RNNs) perform element-wise multiplications across the activations of gates. We show that a significant percentage of activations are saturated and propose coarse-grained pruning of activations (CGPA) to avoid the computation of entire neurons, based on the activation values of the gates. We show that CGPA can be easily implemented on top of a TPU-like architecture with negligible area overhead, resulting in 12% speedup and 12% energy savings on average for a set of widely used RNNs., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Dynamic selection and detection of spreading factors and channels for end-node devices of LoRa networks

  • Aliagas Castell, Carles
  • Pueyo Centelles, Roger
  • Meseguer Pallarès, Roc
  • Millán Marco, Pere
  • Molina Clemente, Carlos Maria
LoRa and LoRaWAN facilitate effective long-range communication for IoT, with LoRa concentrating on transmission efficiency and low-power usage, while LoRaWAN addresses network architecture. Custom LoRa networks are ideal for small-scale applications due to their control and cost benefits. Nevertheless, large-scale deployments can experience message collisions, impacting efficiency and latency. LoRaWAN addresses this with Adaptive Data Rate (ADR), enhancing capacity and power efficiency. Our research introduced a novel strategy to improve LoRa network efficiency through a decentralized method. We employed multiple channels and spreading factors on client chips. This minimized contention and collisions. This approach allowed for dynamic adjustments, ensuring comprehensive communication control and enhanced performance in diverse environments. Our two-step mechanism, integrating heuristics and selection policies, provided flexible communication. We optimized parameters such as message size, transmission power, and bandwidth. This enhanced data rate, RSSI, and SNR, and reduced energy consumption. These results underscore the relevance of precise parameter tuning in achieving optimal LoRa performance., This work has been partially funded by the EU Horizon program under grant agreements No. 833057, No. 825184 and No. 101092644. It has also received support from the Ministry of Economic Affairs and Digital Transformation, together with the European Union-NextGenerationEU (within the PRTR and MRR frameworks), through the CLOUDLESS UNICO I+D CLOUD 2022. Addi- tional funding was provided by the Government of Spain under contracts PID2019-106774RB-C21, Electronics 2025, 14, 3341 34 of 36, PID2019-106774RB-C22, PID2020-113172RB-I00 and PID2024-155476OB-I00, and by the Government of Catalonia as Consolidated Research Groups 2021-SGR-383, 2021-SGR-109 and 2017-SGR-990., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

ARAS: Adaptive low-cost ReRAM-based accelerator for DNNs

  • Sabri Abrebekoh, Mohammad
  • Riera Villanueva, Marc
  • González Colás, Antonio María
Processing Using Memory (PUM) accelerators have the potential to perform Deep Neural Network (DNN) inference by using arrays of memory cells as computation engines. Among various memory technologies, ReRAM crossbars show promising performance in computing dot-product operations in the analog domain. Nevertheless, the expensive writing procedure of ReRAM cells has led researchers to design accelerators whose crossbars have enough capacity to store the full DNN. Given the tremendous and continuous increase in DNN model sizes, this approach is unfeasible for some networks, or inefficient due to the huge hardware requirements. Those accelerators lack the flexibility to adapt to any given DNN model, facing an adaptability challenge. To address this issue we introduce ARAS, a cost-effective ReRAM-based accelerator that employs an offline scheduler to adapt different DNNs to the resource-limited hardware. ARAS also overlaps the computation of a layer with the weight writing of several layers to mitigate the high writing latency of ReRAM. Furthermore, ARAS introduces three optimizations aimed at reducing the energy overheads of writing in ReRAM. Our key optimization capitalizes on the observation that DNN weights can be re-encoded to augment their similarity between layers, increasing the amount of bitwise values that are equal or similar when overwriting ReRAM cells and, hence, reducing the amount of energy required to update the cells. Overall, ARAS greatly reduces the ReRAM writing activity. We evaluate ARAS on a popular set of DNNs. ARAS provides up to 2. 2x speedup and 45% energy savings over a baseline PUM accelerator without any optimization. Compared to a TPU-like accelerator, ARAS provides up to 1. 5x speedup and 62% energy savings., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grants PID2020-113172RBI00 and PID2024-155476OB-I00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, the ICREA Academia program, and the FPI-UPC 2020 grant., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

SHARP: An adaptable, energy-efficient accelerator for recurrent neural networks

  • Yazdani Aminabadi, Reza
  • Ruwase, Olatunji
  • Zhang, Minjia
  • He, Yuxiong
  • Arnau Montañés, José María
  • González Colás, Antonio María
The effectiveness of Recurrent Neural Networks (RNNs) for tasks such as Automatic Speech Recognition has fostered interest in RNN inference acceleration. Due to the recurrent nature and data dependencies of RNN computations, prior work has designed customized architectures specifically tailored to the computation pattern of RNN, getting high computation efficiency for certain chosen model sizes. However, given that the dimensionality of RNNs varies a lot for different tasks, it is crucial to generalize this efficiency to diverse configurations. In this work, we identify adaptiveness as a key feature that is missing from today’s RNN accelerators. In particular, we first show the problem of low resource utilization and low adaptiveness for the state-of-the-art RNN implementations on GPU, FPGA, and ASIC architectures. To solve these issues, we propose an intelligent tiled-based dispatching mechanism for increasing the adaptiveness of RNN computation, in order to efficiently handle the data dependencies. To do so, we propose Sharp as a hardware accelerator, which pipelines RNN computation using an effective scheduling scheme to hide most of the dependent serialization. Furthermore, Sharp employs dynamic reconfigurable architecture to adapt to the model’s characteristics. Sharp achieves 2×, 2.8×, and 82× speedups on average, considering different RNN models and resource budgets, compared to the state-of-the-art ASIC, FPGA, and GPU implementations, respectively. Furthermore, we provide significant energy reduction with respect to the previous solutions, due to the low power dissipation of Sharp (321 GFLOPS/Watt)., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, and the ICREA Academia program., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

LOCATOR: Low-power ORB accelerator for autonomous cars

  • Taranco Serna, Raúl
  • Arnau Montañés, José María
  • González Colás, Antonio María
Simultaneous Localization And Mapping (SLAM) is crucial for autonomous navigation. ORB-SLAM is a state-of-the-art Visual SLAM system based on cameras used for self-driving cars. In this paper, we propose a high-performance, energy-efficient, and functionally accurate hardware accelerator for ORB-SLAM, focusing on its most time-consuming stage: Oriented FAST and Rotated BRIEF (ORB) feature extraction. The Rotated BRIEF (rBRIEF) descriptor generation is the main bottleneck in ORB computation, as it exhibits highly irregular access patterns to local on-chip memories causing a high-performance penalty due to bank conflicts. We introduce a technique to find an optimal static pattern to perform parallel accesses to banks based on a genetic algorithm. Furthermore, we propose the combination of an rBRIEF pixel duplication cache, selective ports replication, and pipelining to reduce latency without compromising cost. The accelerator achieves a reduction in energy consumption of 14597× and 9609×, with respect to high-end CPU and GPU platforms, respectively., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020- 113172RB-I00, the ICREA Academia program and the FPU grant FPU18/04413, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Exploiting beam search confidence for energy-efficient speech recognition

  • Pinto Rivero, Dennis
  • Arnau Montañés, José María
  • Riera Villanueva, Marc
  • Cruz Díaz, Josep Llorenç
  • González Colás, Antonio María
With mobile and embedded devices getting more integrated in our daily lives, the focus is increasingly shifting toward human-friendly interfaces, making automatic speech recognition (ASR) a central player as the ideal means of interaction with machines. ASR is essential for many cognitive computing applications, such as speech-based assistants, dictation systems and real-time language translation. Consequently, interest in speech technology has grown in the last few years, with more systems being proposed and higher accuracy levels being achieved, even surpassing human accuracy. However, highly accurate ASR systems are computationally expensive, requiring on the order of billions of arithmetic operations to decode each second of audio, which conflicts with a growing interest in deploying ASR on edge devices. On these devices, efficient hardware acceleration is key for achieving acceptable performance. In this paper, we propose a technique to improve the energy efficiency and performance of ASR systems, focusing on low-power hardware for edge devices. We focus on optimizing the DNN-based acoustic model evaluation, as we have observed it to be the main bottleneck in popular ASR systems, by leveraging run-time information from the beam search. By doing so, we reduce energy and execution time of the acoustic model evaluation by 25.6  and 25.9 %, respectively, with negligible accuracy loss., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, the ICREA Academia program, and the Spanish MICINN Ministry under grant BES-2017-080605., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Dynamic sampling rate: harnessing frame coherence in graphics applications for energy-efficient GPUs

  • Anglada Sánchez, Martí
  • de Lucas Casamayor, Enrique
  • Parcerisa Bundó, Joan Manuel
  • Aragón Alcaraz, Juan Luis
  • González Colás, Antonio María
In real-time rendering, a 3D scene is modelled with meshes of triangles that the GPU projects to the screen. They are discretized by sampling each triangle at regular space intervals to generate fragments which are then added texture and lighting effects by a shader program. Realistic scenes require detailed geometric models, complex shaders, high-resolution displays and high screen refreshing rates, which all come at a great compute time and energy cost. This cost is often dominated by the fragment shader, which runs for each sampled fragment. Conventional GPUs sample the triangles once per pixel; however, there are many screen regions containing low variation that produce identical fragments and could be sampled at lower than pixel-rate with no loss in quality. Additionally, as temporal frame coherence makes consecutive frames very similar, such variations are usually maintained from frame to frame. This work proposes Dynamic Sampling Rate (DSR), a novel hardware mechanism to reduce redundancy and improve the energy efficiency in graphics applications. DSR analyzes the spatial frequencies of the scene once it has been rendered. Then, it leverages the temporal coherence in consecutive frames to decide, for each region of the screen, the lowest sampling rate to employ in the next frame that maintains image quality. We evaluate the performance of a state-of-the-art mobile GPU architecture extended with DSR for a wide variety of applications. Experimental results show that DSR is able to remove most of the redundancy inherent in the color computations at fragment granularity, which brings average speedups of 1.68x and energy savings of 40%., This work has been supported by the the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (Grant No. 833057), Spanish State Research Agency (MCIN/AEI) under Grant PID2020-113172RB-I00, the ICREA Academia program, and the Generalitat de Catalunya under Grant FI-DGR 2016. Funding was provided by Ministerio de Economía, Industria y Competitividad, Gobierno de España (Grant No. TIN2016-75344-R)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Triangle Dropping: An occluded-geometry predictor for energy-efficient mobile GPUs

  • Corbalán Navarro, David
  • Aragon Alcaraz, Juan Luis
  • Anglada Sánchez, Martí
  • Parcerisa Bundó, Joan Manuel
  • González Colás, Antonio María
This article proposes a novel micro-architecture approach for mobile GPUs aimed at early removing the occluded geometry in a scene by leveraging frame-to-frame coherence, thus reducing the overall energy consumption. Mobile GPUs commonly implement a Tile-Based Rendering (TBR) architecture that differentiates two main phases: the Geometry Pipeline, where all the geometry of a scene is processed; and the Raster Pipeline, where primitives are rendered in a framebuffer. After the Geometry Pipeline, only non-culled primitives inside the camera’s frustum are stored into the Parameter Buffer, a data structure stored in DRAM. However, among the non-culled primitives there is a significant amount that are rendered but non-visible at all, resulting in useless computations. On average, 60% of those primitives are completely occluded in our benchmarks. Despite TBR architectures use on-chip caches for the Parameter Buffer, about 46% of the DRAM traffic still comes from accesses to such buffer. The proposed Triangle Dropping technique leverages the visibility information computed along the Raster Pipeline to predict the primitives’ visibility in the next frame to early discard those that will be totally occluded, drastically reducing Parameter Buffer accesses. On average, our approach achieves overall 14.5% energy savings, 28.2% energy-delay product savings, and a speedup of 20.2%., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant no. 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00 (AEI/FEDER, EU), and the ICREA Academia program. D. Corbalán-Navarro has been also supported by a PhD research fellowship from the University of Murcia’s “Plan Propio de Investigación., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Towards efficient LUT-based PIM: A scalable and low-power approach for modern workloads

  • Khabbazan, Bahareh
  • Riera Villanueva, Marc
  • González Colás, Antonio María
Data movement in memory-intensive workloads is several orders of magnitude more energy-consuming than computation. Frequent memory-processor transfers make this a critical bottleneck. Processing-using-memory (PuM) alleviates this by enabling in-memory computation and reducing data movement. In this paper, we propose Lama, a LUT-based PuM architecture that efficiently executes SIMD operations by supporting independent column accesses within each mat of a DRAM subarray. Lama minimizes the number of energy-intensive memory activation commands, which are the primary source of overhead in most PuM architectures. Besides, Lama supports up to 8-bit operand precision without decomposing computations, unlike prior PuM solutions, while incurring only a 2.47% area overhead. Our evaluation shows that Lama achieves an average performance improvement of 8.5× over state-of-the-art PuM architectures and a 3.8× improvement over CPU, along with energy efficiency gains of 6.9×/8×, respectively, for bulk 8-bit multiplication. We introduce LamaAccel, an HBM-based PuM accelerator that leverages Lama to accelerate LLM inference. By transforming MAC computations into energy-efficient addition and counting operations, LamaAccel achieves up to 9.3×/19.2× energy reduction and 4.8×/9.8× speedup over a TPU/GPU, as well as up to 5.8× lower energy consumption and 2.1× higher performance compared to a state-of-the-art PuM baseline., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grants PID2020-113172RB-I00 and PID2024-155476OBI00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, the ICREA Academia program, and the FPI-UPC 2020 grant., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

An energy-efficient near-data processing accelerator for DNNs to optimize memory accesses

  • Khabbazan, Bahareh
  • Sabri Abrebekoh, Mohammad
  • Riera Villanueva, Marc
  • González Colás, Antonio María
The constant growth of DNNs makes them challenging to implement and run efficiently on traditional computecentric architectures. Some accelerators have attempted to add more compute units and on-chip buffers to solve the memory wall problem without much success, and sometimes even worsening the issue since more compute units also require higher memory bandwidth. Prior works have proposed the design of memorycentric architectures based on the Near-Data Processing (NDP) paradigm. NDP seeks to break the memory wall by moving the computations closer to the memory hierarchy, reducing the data movements and their cost as much as possible. The 3D-stacked memory is especially appealing for DNN accelerators due to its high-density/low-energy storage and near-memory computation capabilities to perform the DNN operations massively in parallel. However, memory accesses remain as the main bottleneck for running modern DNNs efficiently. To improve the efficiency of DNN inference we present QeiHaN, a hardware accelerator that implements a 3D-stacked memory-centric weight storage scheme to take advantage of a logarithmic quantization of activations. In particular, since activations of FC and CONV layers of modern DNNs are commonly represented as powers of two with negative exponents, QeiHaN performs an implicit in-memory bit-shifting of the DNN weights to reduce memory activity. Only the meaningful bits of the weights required for the bit-shift operation are accessed. Overall, QeiHaN reduces memory accesses by 25% compared to a standard memory organization. We evaluate QeiHaN on a popular set of DNNs. On average, QeiHaN provides4.3¿speedup and3.5¿energy savings over a Neurocube-like accelerator., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI), Spain under grant PID2020-113172RB-I00, the Catalan Agency for University and Research (AGAUR), Spain under grant 2021 SGR 00383, and the ICREA Academia program, Spain., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Omega-Test: A predictive early-Z culling to improve the graphics pipeline energy-efficiency

  • Corbalán Navarro, David
  • Aragón Alcaraz, Juan Luis
  • Anglada Sánchez, Martí
  • Lucas Casamayor, Enrique de
  • Parcerisa Bundó, Joan Manuel
  • González Colás, Antonio María
The most common task of GPUs is to render images in real time. When rendering a 3D scene, a key step is to determine which parts of every object are visible in the final image. There are different approaches to solve the visibility problem, the Z-Test being the most common. A main factor that significantly penalizes the energy efficiency of a GPU, especially in the mobile arena, is the so-called overdraw, which happens when a portion of an object is shaded and rendered but finally occluded by another object. This useless work results in a waste of energy; however, a conventional Z-Test only avoids a fraction of it. In this paper we present a novel microarchitectural technique, the Omega-Test, to drastically reduce the overdraw on a Tile-Based Rendering (TBR) architecture. Graphics applications have a great degree of inter-frame coherence, which makes the output of a frame very similar to the previous one. The proposed approach leverages the frame-to-frame coherence by using the resulting information of the Z-Test for a tile (a buffer containing all the calculated pixel depths for a tile), which is discarded by nowadays GPUs, to predict the visibility of the same tile in the next frame. As a result, the Omega-Test early identifies occluded parts of the scene and avoids the rendering of non-visible surfaces eliminating costly computations and off-chip memory accesses. Our experimental evaluation shows average EDP savings in the overall GPU/Memory system of 26.4% and an average speedup of 16.3% for the evaluated benchmarks., This work has been supported by the the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency under grant TIN2016-75344-R (AEI/FEDER, EU) and the ICREA Academia program. D. Corbalan-Navarro has been supported by a PhD research fellowship from the University of Murcia., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Irregular accesses reorder unit: improving GPGPU memory coalescing for graph-based workloads

  • Segura Salvador, Albert
  • Arnau Montañés, José María
  • González Colás, Antonio María
GPGPU architectures have become the dominant platform for massively parallel workloads, delivering high performance and energy efficiency for popular applications such as machine learning, computer vision or self-driving cars. However, irregular applications, such as graph processing, fail to fully exploit GPGPU resources due to their divergent memory accesses that saturate the memory hierarchy. To reduce the pressure on the memory subsystem for divergent memory-intensive applications, programmers must take into account SIMT execution model and memory coalescing in GPGPUs, devoting significant efforts in complex optimization techniques. Despite these efforts, we show that irregular graph processing still suffers from low GPGPU performance. We observe that in many irregular applications the mapping of data to threads can be safely changed. In other words, it is possible to relax the strict relationship between thread and data processed to reduce memory divergence. Based on this observation, we propose the Irregular accesses Reorder Unit (IRU), a novel hardware extension tightly integrated in the GPGPU pipeline. The IRU reorders data processed by the threads on irregular accesses to improve memory coalescing, i.e., it tries to assign data elements to threads as to produce coalesced accesses in SIMT groups. Furthermore, the IRU is capable of filtering and merging duplicated accesses, significantly reducing the workload. Programmers can easily utilize the IRU with a simple API, or let the compiler issue instructions from our extended ISA. We evaluate our proposal for state-of-the-art graph-based algorithms and a wide selection of applications. Results show that the IRU achieves a memory coalescing improvement of 1.32x and a 46% reduction in the overall traffic in the memory hierarchy, which results in 1.33x speedup and 13% energy savings on average, while incurring in a small 5.6% area overhead., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00 and the ICREA Academia program., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Mixture-of-Rookies: saving DNN computations by predicting ReLU outputs

  • Pinto Rivero, Dennis
  • Arnau Montañés, José María
  • Riera Villanueva, Marc
  • Cruz Díaz, Josep Llorenç
  • González Colás, Antonio María
Deep Neural Networks (DNNs) are widely used in many application domains. However, they require a vast amount of computations and memory accesses to deliver outstanding accuracy. In this paper, we propose a scheme to predict whether the output of each ReLu activated neuron will be a zero or a positive number in order to skip the computation of those neurons that will likely output a zero. Our predictor, named Mixture-of-Rookies, combines two inexpensive components. The first one exploits the high linear correlation between binarized (1-bit) and full-precision (8-bit) dot products, whereas the second component clusters together neurons that tend to output zero at the same time. We propose a novel clustering scheme based on analysis of angles, as the sign of the dot product of two vectors depends on the cosine of the angle between them. We implement our hybrid zero output predictor on top of a state-of-the-art DNN accelerator. Experimental results show that our scheme introduces a small area overhead of 5.3% while achieving a speedup of 1.2x and reducing energy consumption by 16.5% on average for a set of diverse DNNs., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RBI00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, the ICREA Academia program, and the Spanish MICINN Ministry under grant BES-2017-080605., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

DNN pruning with principal component analysis and connection importance estimation

  • Riera Villanueva, Marc
  • Arnau Montañés, José María
  • González Colás, Antonio María
DNN pruning reduces memory footprint and computational work of DNN-based solutions to improve performance and energy-efficiency. An effective pruning scheme should be able to systematically remove connections and/or neurons that are unnecessary or redundant, reducing the DNN size without any loss in accuracy. In this paper we show that some of the most popular pruning schemes, such as the Near Zero Weights, require an extremely time-consuming iterative process that requires retraining the DNN many times to tune the pruning hyperparameters. Then, we propose a DNN pruning scheme based on Principal Component Analysis and relative importance of each neuron’s connection (PCA+DIRIE) that automatically finds the optimized DNN in one shot without requiring hand-tuning of multiple parameters. The experimental results show the effectiveness of our method on several benchmarks. Notably, on ImageNet, PCA+DIRIE can prune up to 60% of ResNet-50 with negligible impact on accuracy., This work has been supported by the CoCoUnit ERC Advanced Grantof the EU’s Horizon 2020 program (grant No 833057), the Spanish StateResearch Agency under grant PID2020-113172RB-I00 (MCIN/AEI), theICREA Academia program, and the Spanish Ministry of Education undergrant FPU15/02294., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

Hamun: an approximate computing method to prolong the lifespan of ReRAM-based accelerators

  • Sabri Abrebekoh, Mohammad
  • Riera Villanueva, Marc
  • González Colás, Antonio María
ReRAM-based accelerators exhibit enormous potential to increase computational efficiency for DNN inference tasks, delivering significant performance and energy savings over traditional platforms. By incorporating adaptive scheduling, these accelerators dynamically adjust to DNN requirements, optimizing allocation of constrained hardware resources. However, ReRAM cells have limited endurance cycles due to wear-out from multiple updates for each inference execution, which shortens the lifespan of ReRAM-based accelerators and presents a practical challenge in positioning them as alternatives to conventional platforms like TPUs. Addressing these endurance limitations is essential for making ReRAM-based solutions viable for long-term, high-performance DNN inference. To address the lifespan limitations of ReRAM-based accelerators, we introduce Hamun, an approximate computing method designed to extend the lifespan of ReRAM-based accelerators through a range of optimizations. Hamun incorporates a novel mechanism that detects faulty cells due to wear-out and retires them, avoiding in this way their otherwise adverse impact on DNN accuracy. Moreover, Hamun extends the lifespan of ReRAM-based accelerators by adapting wear-leveling techniques across various abstraction levels of the accelerator and implementing a batch execution scheme to maximize ReRAM cell usage for multiple inferences. Additionally, Hamun introduces a new approximation method that leverages the fault tolerance characteristics of DNNs to delay the retirement of worn-out cells, reducing the performance penalty of retired cells and further extending the accelerator’s lifespan. On average, evaluated on a set of popular DNNs, Hamun demonstrates an improvement in lifespan of 13.2x over a state-of-the-art baseline. The main contributors to this improvement are the fault handling and batch execution schemes, which provide 4.6x and 2.6x lifespan improvements respectively., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RBI00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, and the ICREA Academia program., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publication Artículo científico (JournalArticle)

A survey of near-data processing architectures for neural networks

  • Hassanpour, Mehdi
  • Riera Villanueva, Marc
  • González Colás, Antonio María
Data-intensive workloads and applications, such as machine learning (ML), are fundamentally limited by traditional computing systems based on the von-Neumann architecture. As data movement operations and energy consumption become key bottlenecks in the design of computing systems, the interest in unconventional approaches such as Near-Data Processing (NDP), machine learning, and especially neural network (NN)-based accelerators has grown significantly. Emerging memory technologies, such as ReRAM and 3D-stacked, are promising for efficiently architecting NDP-based accelerators for NN due to their capabilities to work as both high-density/low-energy storage and in/near-memory computation/search engine. In this paper, we present a survey of techniques for designing NDP architectures for NN. By classifying the techniques based on the memory technology employed, we underscore their similarities and differences. Finally, we discuss open challenges and future perspectives that need to be explored in order to improve and extend the adoption of NDP architectures for future computing platforms. This paper will be valuable for computer architects, chip designers, and researchers in the area of machine learning., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, and the ICREA Academia program., Peer Reviewed, Postprint (published version)