Pasar al contenido principal

ARQUITECTURAS DE DOMINIO ESPECIFICO PARA SISTEMAS DE COMPUTACION ENERGETICAMENTE EFICIENTES

PID2020-113172RB-I00

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2020
Plan Estatal 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSITAT POLITECNICA DE CATALUNYA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 35
Encontrada(s) 1 página(s)

UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

A low-cost and do-it-yourself device for pumping monitoring in deep aquifers

  • Aliagas, Carles
  • Pérez Foguet, Agustí|||0000-0002-2737-4710
  • Meseguer Pallarès, Roc|||0000-0002-9414-646X
  • Millán Marco, Pere
  • Molina Clemente, Carlos Maria
Water crises due to climate change, high population growth and increasing demands from industry and agriculture claim for increasing efficiency and universalizing water resources management strategies and techniques. Water monitoring helps providing necessary evidences for making sound decisions about managing water resources both now and in the future. In this work, a low cost and “do it yourself” communication device is proposed to record water production and energy consumption of electric pumpings from deep boreholes/wells, and to predict the impact of the ongoing and previous pumpings in the evolution of the water level in the aquifer. The proposal incorporates an edge-computing approach for the simulation of the aquifer response in real-time. Computation of results of interest is performed at the sensor, minimizing communication requirements and ensuring almost immediate results. An approximated solution to physically based modeling of aquifer response is computed thanks to the a priori expression of the water level time evolution in a reduced basis. The accuracy is enough to detect deviations from expected behaviour. The energy consumption of the device is very much reduced with respect to that of a full modelling, which can be computed off-line for calibrating reduced model parameters and perform detailed analyses. The device is tested in a real scenario, in a mountain subbasin of the Ebro river in Spain, obtaining a good trade-off between performance, price, and energy consumption., This research has been partly supported by EU under grant agreement N. 825184 and funded by the Government of Spain under contracts PID2019-106774RB-C21, PID2019-106774RB-C22, and PID2020-113172RB-I00 and by the Government of Catalonia as Consolidated Research Groups 2017-SGR-688 and 2017-SGR-990, and Pre-consolidated Research Group 2017-SGR-1496. The APC was funded by the Open program from Universitat Rovira i Virgili., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

An energy-efficient near-data processing accelerator for DNNs to optimize memory accesses

  • Khabbazan, Bahareh|||0000-0001-6726-2804
  • Sabri Abrebekoh, Mohammad|||0000-0002-3113-5392
  • Riera Villanueva, Marc|||0000-0002-2768-5703
  • González Colás, Antonio María|||0000-0002-0009-0996
The constant growth of DNNs makes them challenging to implement and run efficiently on traditional computecentric architectures. Some accelerators have attempted to add more compute units and on-chip buffers to solve the memory wall problem without much success, and sometimes even worsening the issue since more compute units also require higher memory bandwidth. Prior works have proposed the design of memorycentric architectures based on the Near-Data Processing (NDP) paradigm. NDP seeks to break the memory wall by moving the computations closer to the memory hierarchy, reducing the data movements and their cost as much as possible. The 3D-stacked memory is especially appealing for DNN accelerators due to its high-density/low-energy storage and near-memory computation capabilities to perform the DNN operations massively in parallel. However, memory accesses remain as the main bottleneck for running modern DNNs efficiently. To improve the efficiency of DNN inference we present QeiHaN, a hardware accelerator that implements a 3D-stacked memory-centric weight storage scheme to take advantage of a logarithmic quantization of activations. In particular, since activations of FC and CONV layers of modern DNNs are commonly represented as powers of two with negative exponents, QeiHaN performs an implicit in-memory bit-shifting of the DNN weights to reduce memory activity. Only the meaningful bits of the weights required for the bit-shift operation are accessed. Overall, QeiHaN reduces memory accesses by 25% compared to a standard memory organization. We evaluate QeiHaN on a popular set of DNNs. On average, QeiHaN provides4.3¿speedup and3.5¿energy savings over a Neurocube-like accelerator., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI), Spain under grant PID2020-113172RB-I00, the Catalan Agency for University and Research (AGAUR), Spain under grant 2021 SGR 00383, and the ICREA Academia program, Spain., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

E-BATCH: Energy-efficient and high-throughput RNN batching

  • Silfa Feliz, Franyell Antonio|||0000-0002-1134-9908
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • González Colás, Antonio María|||0000-0002-0009-0996
Recurrent Neural Network (RNN) inference exhibits low hardware utilization due to the strict data dependencies across time-steps. Batching multiple requests can increase throughput. However, RNN batching requires a large amount of padding since the batched input sequences may vastly differ in length. Schemes that dynamically update the batch every few time-steps avoid padding. However, they require executing different RNN layers in a short time span, decreasing energy efficiency. Hence, we propose E-BATCH, a low-latency and energy-efficient batching scheme tailored to RNN accelerators. It consists of a runtime system and effective hardware support. The runtime concatenates multiple sequences to create large batches, resulting in substantial energy savings. Furthermore, the accelerator notifies it when the evaluation of an input sequence is done. Hence, a new input sequence can be immediately added to a batch, thus largely reducing the amount of padding. E-BATCH dynamically controls the number of time-steps evaluated per batch to achieve the best trade-off between latency and energy efficiency for the given hardware platform. We evaluate E-BATCH on top of E-PUR and TPU. E-BATCH improves throughput by 1.8× and energy efficiency by 3.6× in E-PUR, whereas in TPU, it improves throughput by 2.1× and energy efficiency by 1.6×, over the state-of-the-art., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, and the ICREA Academia program., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

ARAS: Adaptive low-cost ReRAM-based accelerator for DNNs

  • Sabri Abrebekoh, Mohammad|||0000-0002-3113-5392
  • Riera Villanueva, Marc|||0000-0002-2768-5703
  • González Colás, Antonio María|||0000-0002-0009-0996
Processing Using Memory (PUM) accelerators have the potential to perform Deep Neural Network (DNN) inference by using arrays of memory cells as computation engines. Among various memory technologies, ReRAM crossbars show promising performance in computing dot-product operations in the analog domain. Nevertheless, the expensive writing procedure of ReRAM cells has led researchers to design accelerators whose crossbars have enough capacity to store the full DNN. Given the tremendous and continuous increase in DNN model sizes, this approach is unfeasible for some networks, or inefficient due to the huge hardware requirements. Those accelerators lack the flexibility to adapt to any given DNN model, facing an adaptability challenge. To address this issue we introduce ARAS, a cost-effective ReRAM-based accelerator that employs an offline scheduler to adapt different DNNs to the resource-limited hardware. ARAS also overlaps the computation of a layer with the weight writing of several layers to mitigate the high writing latency of ReRAM. Furthermore, ARAS introduces three optimizations aimed at reducing the energy overheads of writing in ReRAM. Our key optimization capitalizes on the observation that DNN weights can be re-encoded to augment their similarity between layers, increasing the amount of bitwise values that are equal or similar when overwriting ReRAM cells and, hence, reducing the amount of energy required to update the cells. Overall, ARAS greatly reduces the ReRAM writing activity. We evaluate ARAS on a popular set of DNNs. ARAS provides up to 2. 2x speedup and 45% energy savings over a baseline PUM accelerator without any optimization. Compared to a TPU-like accelerator, ARAS provides up to 1. 5x speedup and 62% energy savings., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grants PID2020-113172RBI00 and PID2024-155476OB-I00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, the ICREA Academia program, and the FPI-UPC 2020 grant., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

SHARP: An adaptable, energy-efficient accelerator for recurrent neural networks

  • Yazdani Aminabadi, Reza|||0000-0002-7949-6453
  • Ruwase, Olatunji
  • Zhang, Minjia
  • He, Yuxiong
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • González Colás, Antonio María|||0000-0002-0009-0996
The effectiveness of Recurrent Neural Networks (RNNs) for tasks such as Automatic Speech Recognition has fostered interest in RNN inference acceleration. Due to the recurrent nature and data dependencies of RNN computations, prior work has designed customized architectures specifically tailored to the computation pattern of RNN, getting high computation efficiency for certain chosen model sizes. However, given that the dimensionality of RNNs varies a lot for different tasks, it is crucial to generalize this efficiency to diverse configurations. In this work, we identify adaptiveness as a key feature that is missing from today’s RNN accelerators. In particular, we first show the problem of low resource utilization and low adaptiveness for the state-of-the-art RNN implementations on GPU, FPGA, and ASIC architectures. To solve these issues, we propose an intelligent tiled-based dispatching mechanism for increasing the adaptiveness of RNN computation, in order to efficiently handle the data dependencies. To do so, we propose Sharp as a hardware accelerator, which pipelines RNN computation using an effective scheduling scheme to hide most of the dependent serialization. Furthermore, Sharp employs dynamic reconfigurable architecture to adapt to the model’s characteristics. Sharp achieves 2×, 2.8×, and 82× speedups on average, considering different RNN models and resource budgets, compared to the state-of-the-art ASIC, FPGA, and GPU implementations, respectively. Furthermore, we provide significant energy reduction with respect to the previous solutions, due to the low power dissipation of Sharp (321 GFLOPS/Watt)., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, and the ICREA Academia program., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Exploiting beam search confidence for energy-efficient speech recognition

  • Pinto Rivero, Dennis|||0000-0002-2175-9580
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • Riera Villanueva, Marc|||0000-0002-2768-5703
  • Cruz Díaz, Josep Llorenç|||0000-0001-5325-9153
  • González Colás, Antonio María|||0000-0002-0009-0996
With mobile and embedded devices getting more integrated in our daily lives, the focus is increasingly shifting toward human-friendly interfaces, making automatic speech recognition (ASR) a central player as the ideal means of interaction with machines. ASR is essential for many cognitive computing applications, such as speech-based assistants, dictation systems and real-time language translation. Consequently, interest in speech technology has grown in the last few years, with more systems being proposed and higher accuracy levels being achieved, even surpassing human accuracy. However, highly accurate ASR systems are computationally expensive, requiring on the order of billions of arithmetic operations to decode each second of audio, which conflicts with a growing interest in deploying ASR on edge devices. On these devices, efficient hardware acceleration is key for achieving acceptable performance. In this paper, we propose a technique to improve the energy efficiency and performance of ASR systems, focusing on low-power hardware for edge devices. We focus on optimizing the DNN-based acoustic model evaluation, as we have observed it to be the main bottleneck in popular ASR systems, by leveraging run-time information from the beam search. By doing so, we reduce energy and execution time of the acoustic model evaluation by 25.6  and 25.9 %, respectively, with negligible accuracy loss., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, the ICREA Academia program, and the Spanish MICINN Ministry under grant BES-2017-080605., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Dynamic sampling rate: harnessing frame coherence in graphics applications for energy-efficient GPUs

  • Anglada Sánchez, Martí|||0000-0002-1204-1841
  • de Lucas Casamayor, Enrique
  • Parcerisa Bundó, Joan Manuel|||0000-0001-5771-8118
  • Aragón Alcaraz, Juan Luis
  • González Colás, Antonio María|||0000-0002-0009-0996
In real-time rendering, a 3D scene is modelled with meshes of triangles that the GPU projects to the screen. They are discretized by sampling each triangle at regular space intervals to generate fragments which are then added texture and lighting effects by a shader program. Realistic scenes require detailed geometric models, complex shaders, high-resolution displays and high screen refreshing rates, which all come at a great compute time and energy cost. This cost is often dominated by the fragment shader, which runs for each sampled fragment. Conventional GPUs sample the triangles once per pixel; however, there are many screen regions containing low variation that produce identical fragments and could be sampled at lower than pixel-rate with no loss in quality. Additionally, as temporal frame coherence makes consecutive frames very similar, such variations are usually maintained from frame to frame. This work proposes Dynamic Sampling Rate (DSR), a novel hardware mechanism to reduce redundancy and improve the energy efficiency in graphics applications. DSR analyzes the spatial frequencies of the scene once it has been rendered. Then, it leverages the temporal coherence in consecutive frames to decide, for each region of the screen, the lowest sampling rate to employ in the next frame that maintains image quality. We evaluate the performance of a state-of-the-art mobile GPU architecture extended with DSR for a wide variety of applications. Experimental results show that DSR is able to remove most of the redundancy inherent in the color computations at fragment granularity, which brings average speedups of 1.68x and energy savings of 40%., This work has been supported by the the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (Grant No. 833057), Spanish State Research Agency (MCIN/AEI) under Grant PID2020-113172RB-I00, the ICREA Academia program, and the Generalitat de Catalunya under Grant FI-DGR 2016. Funding was provided by Ministerio de Economía, Industria y Competitividad, Gobierno de España (Grant No. TIN2016-75344-R)., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

DNN pruning with principal component analysis and connection importance estimation

  • Riera Villanueva, Marc|||0000-0002-2768-5703
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • González Colás, Antonio María|||0000-0002-0009-0996
DNN pruning reduces memory footprint and computational work of DNN-based solutions to improve performance and energy-efficiency. An effective pruning scheme should be able to systematically remove connections and/or neurons that are unnecessary or redundant, reducing the DNN size without any loss in accuracy. In this paper we show that some of the most popular pruning schemes, such as the Near Zero Weights, require an extremely time-consuming iterative process that requires retraining the DNN many times to tune the pruning hyperparameters. Then, we propose a DNN pruning scheme based on Principal Component Analysis and relative importance of each neuron’s connection (PCA+DIRIE) that automatically finds the optimized DNN in one shot without requiring hand-tuning of multiple parameters. The experimental results show the effectiveness of our method on several benchmarks. Notably, on ImageNet, PCA+DIRIE can prune up to 60% of ResNet-50 with negligible impact on accuracy., This work has been supported by the CoCoUnit ERC Advanced Grantof the EU’s Horizon 2020 program (grant No 833057), the Spanish StateResearch Agency under grant PID2020-113172RB-I00 (MCIN/AEI), theICREA Academia program, and the Spanish Ministry of Education undergrant FPU15/02294., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Triangle Dropping: An occluded-geometry predictor for energy-efficient mobile GPUs

  • Corbalán Navarro, David
  • Aragon Alcaraz, Juan Luis
  • Anglada Sánchez, Martí|||0000-0002-1204-1841
  • Parcerisa Bundó, Joan Manuel|||0000-0001-5771-8118
  • González Colás, Antonio María|||0000-0002-0009-0996
This article proposes a novel micro-architecture approach for mobile GPUs aimed at early removing the occluded geometry in a scene by leveraging frame-to-frame coherence, thus reducing the overall energy consumption. Mobile GPUs commonly implement a Tile-Based Rendering (TBR) architecture that differentiates two main phases: the Geometry Pipeline, where all the geometry of a scene is processed; and the Raster Pipeline, where primitives are rendered in a framebuffer. After the Geometry Pipeline, only non-culled primitives inside the camera’s frustum are stored into the Parameter Buffer, a data structure stored in DRAM. However, among the non-culled primitives there is a significant amount that are rendered but non-visible at all, resulting in useless computations. On average, 60% of those primitives are completely occluded in our benchmarks. Despite TBR architectures use on-chip caches for the Parameter Buffer, about 46% of the DRAM traffic still comes from accesses to such buffer. The proposed Triangle Dropping technique leverages the visibility information computed along the Raster Pipeline to predict the primitives’ visibility in the next frame to early discard those that will be totally occluded, drastically reducing Parameter Buffer accesses. On average, our approach achieves overall 14.5% energy savings, 28.2% energy-delay product savings, and a speedup of 20.2%., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant no. 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00 (AEI/FEDER, EU), and the ICREA Academia program. D. Corbalán-Navarro has been also supported by a PhD research fellowship from the University of Murcia’s “Plan Propio de Investigación., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Towards efficient LUT-based PIM: A scalable and low-power approach for modern workloads

  • Khabbazan, Bahareh|||0000-0001-6726-2804
  • Riera Villanueva, Marc|||0000-0002-2768-5703
  • González Colás, Antonio María|||0000-0002-0009-0996
Data movement in memory-intensive workloads is several orders of magnitude more energy-consuming than computation. Frequent memory-processor transfers make this a critical bottleneck. Processing-using-memory (PuM) alleviates this by enabling in-memory computation and reducing data movement. In this paper, we propose Lama, a LUT-based PuM architecture that efficiently executes SIMD operations by supporting independent column accesses within each mat of a DRAM subarray. Lama minimizes the number of energy-intensive memory activation commands, which are the primary source of overhead in most PuM architectures. Besides, Lama supports up to 8-bit operand precision without decomposing computations, unlike prior PuM solutions, while incurring only a 2.47% area overhead. Our evaluation shows that Lama achieves an average performance improvement of 8.5× over state-of-the-art PuM architectures and a 3.8× improvement over CPU, along with energy efficiency gains of 6.9×/8×, respectively, for bulk 8-bit multiplication. We introduce LamaAccel, an HBM-based PuM accelerator that leverages Lama to accelerate LLM inference. By transforming MAC computations into energy-efficient addition and counting operations, LamaAccel achieves up to 9.3×/19.2× energy reduction and 4.8×/9.8× speedup over a TPU/GPU, as well as up to 5.8× lower energy consumption and 2.1× higher performance compared to a state-of-the-art PuM baseline., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grants PID2020-113172RB-I00 and PID2024-155476OBI00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, the ICREA Academia program, and the FPI-UPC 2020 grant., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

CREW: Computation reuse and efficient weight storage for hardware-accelerated MLPs and RNNs

  • Riera Villanueva, Marc|||0000-0002-2768-5703
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • González Colás, Antonio María|||0000-0002-0009-0996
Deep Neural Networks (DNNs) have achieved tremendous success for cognitive applications. The core operation in a DNN is the dot product between quantized inputs and weights. Prior works exploit the weight/input repetition that arises due to quantization to avoid redundant computations in Convolutional Neural Networks (CNNs). However, in this paper we show that their effectiveness is severely limited when applied to FullyConnected (FC) layers, which are commonly used in state-of-the-art DNNs, as it is the case of modern Recurrent Neural Networks (RNNs) and Transformer models. To improve energy-efficiency of FC computation we present CREW, a hardware accelerator that implements Computation Reuse and an Efficient Weight Storage mechanism to exploit the large number of repeated weights in FC layers. CREW first performs the multiplications of the unique weights by their respective inputs and stores the results in an on-chip buffer. The storage requirements are modest due to the small number of unique weights and the relatively small size of the input compared to convolutional layers. Next, CREW computes each output by fetching and adding its required products. To this end, each weight is replaced offline by an index in the buffer of unique products. Indices are typically smaller than the quantized weights, since the number of unique weights for each input tends to be much lower than the range of quantized weights, which reduces storage and memory bandwidth requirements. Overall, CREW greatly reduces the number of multiplications and provides significant savings in model memory footprint and memory bandwidth usage. We evaluate CREW on a diverse set of modern DNNs. On average, CREW provides 2.61x speedup and 2.42x energy savings over a TPU-like accelerator. Compared to UCNN, a state-of-art computation reuse technique, CREW achieves 2.10x speedup and 2.08x energy savings on average., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, the ICREA Academia program, and the Spanish Ministry of Education under grant FPU15/02294., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

A survey of near-data processing architectures for neural networks

  • Hassanpour, Mehdi|||0000-0002-1278-9382
  • Riera Villanueva, Marc|||0000-0002-2768-5703
  • González Colás, Antonio María|||0000-0002-0009-0996
Data-intensive workloads and applications, such as machine learning (ML), are fundamentally limited by traditional computing systems based on the von-Neumann architecture. As data movement operations and energy consumption become key bottlenecks in the design of computing systems, the interest in unconventional approaches such as Near-Data Processing (NDP), machine learning, and especially neural network (NN)-based accelerators has grown significantly. Emerging memory technologies, such as ReRAM and 3D-stacked, are promising for efficiently architecting NDP-based accelerators for NN due to their capabilities to work as both high-density/low-energy storage and in/near-memory computation/search engine. In this paper, we present a survey of techniques for designing NDP architectures for NN. By classifying the techniques based on the memory technology employed, we underscore their similarities and differences. Finally, we discuss open challenges and future perspectives that need to be explored in order to improve and extend the adoption of NDP architectures for future computing platforms. This paper will be valuable for computer architects, chip designers, and researchers in the area of machine learning., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, and the ICREA Academia program., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Enhancing BNN storage and performance requirements via efficient quantization and variable encoding

  • Ordóñez Redondo, Marc
  • Silfa Feliz, Franyell Antonio|||0000-0002-1134-9908
  • González Colás, Antonio María|||0000-0002-0009-0996
Binary Neural Networks (BNNs) perform competitively in practical image classification tasks. In particular, their accuracy is similar to that obtained by full-precision models tailored to edge devices. BNNs are amenable for devices with constrained resources, since they employ just one bit to store inputs and weights; hence, their storage requirements are low. Moreover, BNN computations utilize XNOR operations, which are implemented efficiently using simple hardware structures. However, supporting BNNs efficiently on edge devices is challenging since they perform many memory accesses to load weights, hindering their benefits. In BNNs, weights and inputs are stored using one bit, and several of them are packed together as a sequence of bits. In this work, the number of unique sequences representing a set of weights is observed to be low. In addition, a small group of unique sequences is seen to be used more frequently than others. Accordingly, this observation is exploited by using variable length encoding to represent the most commonly used sequences and decompressing them during inference. Commonly, BNNs employ floating-point values to store the weights for the output layer, aiming to obtain competitive accuracy. This further magnifies the storage requirements and significantly affects performance. In this work, it is observed that using traditional quantization schemes to quantize these weights negatively affects network accuracy. Then, it is shown that one can use a small number of bits to store most of the values in the output layer without negatively impacting accuracy by carefully quantizing the weights that fall outside a given range. Finally, an efficient quantization scheme for the output layer is proposed. A previously proposed FPGA accelerator is extended with the required hardware to implement the proposed schemes. The proposal is evaluated on a Pynq Z1 device. Our techniques reduce memory requirement by 1.5 × and improve performance by 2.2 × on average., This work has been supported by the Spanish State Research Agency (MCIU/AEI/10.13039/501100011033), FEDER/UE and the European Social Fund Plus (ESF+) under grants PID2020-113172RB-I00 and PID2024-155476OB-I00, and the ICREA Academia program., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

SIMIL: SIMple Issue Logic for GPUs

  • Huerta Gañán, Rodrigo|||0000-0003-0052-7710
  • Cruz Díaz, Josep Llorenç|||0000-0001-5325-9153
  • Arnau Montañes, Jose-Maria
  • González Colás, Antonio María|||0000-0002-0009-0996
GPU architectures have become popular for executing general-purpose programs. In particular, they are some of the most efficient architectures for machine learning applications which are among the most trendy and demanding applications nowadays. This paper presents SIMIL (SIMple Issue Logic for GPUs), an architectural modification to the issue stage that replaces scoreboards with a Dependence Matrix to track dependencies among instructions and avoid data hazards. We show that a Dependence Matrix is more effective in the presence of repetitive use of source operands, which is common in many applications. Besides, a Dependence Matrix with minor extensions can also support a simplistic out-of-order issue. Evaluations on an NVIDIA Tesla V100-like GPU show that SIMIL provides a speed-up of up to 2.39 in some machine learning programs and 1.31 on average for various benchmarks, while it reduces energy consumption by 12.81%, with only 1.5% area overhead. We also show that SIMIL outperforms a recently proposed approach for out-of-order issue that uses register renaming., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, and the ICREA Academia program., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Irregular accesses reorder unit: improving GPGPU memory coalescing for graph-based workloads

  • Segura Salvador, Albert|||0000-0001-5084-9246
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • González Colás, Antonio María|||0000-0002-0009-0996
GPGPU architectures have become the dominant platform for massively parallel workloads, delivering high performance and energy efficiency for popular applications such as machine learning, computer vision or self-driving cars. However, irregular applications, such as graph processing, fail to fully exploit GPGPU resources due to their divergent memory accesses that saturate the memory hierarchy. To reduce the pressure on the memory subsystem for divergent memory-intensive applications, programmers must take into account SIMT execution model and memory coalescing in GPGPUs, devoting significant efforts in complex optimization techniques. Despite these efforts, we show that irregular graph processing still suffers from low GPGPU performance. We observe that in many irregular applications the mapping of data to threads can be safely changed. In other words, it is possible to relax the strict relationship between thread and data processed to reduce memory divergence. Based on this observation, we propose the Irregular accesses Reorder Unit (IRU), a novel hardware extension tightly integrated in the GPGPU pipeline. The IRU reorders data processed by the threads on irregular accesses to improve memory coalescing, i.e., it tries to assign data elements to threads as to produce coalesced accesses in SIMT groups. Furthermore, the IRU is capable of filtering and merging duplicated accesses, significantly reducing the workload. Programmers can easily utilize the IRU with a simple API, or let the compiler issue instructions from our extended ISA. We evaluate our proposal for state-of-the-art graph-based algorithms and a wide selection of applications. Results show that the IRU achieves a memory coalescing improvement of 1.32x and a 46% reduction in the overall traffic in the memory hierarchy, which results in 1.33x speedup and 13% energy savings on average, while incurring in a small 5.6% area overhead., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00 and the ICREA Academia program., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Mixture-of-Rookies: saving DNN computations by predicting ReLU outputs

  • Pinto Rivero, Dennis|||0000-0002-2175-9580
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • Riera Villanueva, Marc|||0000-0002-2768-5703
  • Cruz Díaz, Josep Llorenç|||0000-0001-5325-9153
  • González Colás, Antonio María|||0000-0002-0009-0996
Deep Neural Networks (DNNs) are widely used in many application domains. However, they require a vast amount of computations and memory accesses to deliver outstanding accuracy. In this paper, we propose a scheme to predict whether the output of each ReLu activated neuron will be a zero or a positive number in order to skip the computation of those neurons that will likely output a zero. Our predictor, named Mixture-of-Rookies, combines two inexpensive components. The first one exploits the high linear correlation between binarized (1-bit) and full-precision (8-bit) dot products, whereas the second component clusters together neurons that tend to output zero at the same time. We propose a novel clustering scheme based on analysis of angles, as the sign of the dot product of two vectors depends on the cosine of the angle between them. We implement our hybrid zero output predictor on top of a state-of-the-art DNN accelerator. Experimental results show that our scheme introduces a small area overhead of 5.3% while achieving a speedup of 1.2x and reducing energy consumption by 16.5% on average for a set of diverse DNNs., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RBI00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, the ICREA Academia program, and the Spanish MICINN Ministry under grant BES-2017-080605., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Hamun: an approximate computing method to prolong the lifespan of ReRAM-based accelerators

  • Sabri Abrebekoh, Mohammad|||0000-0002-3113-5392
  • Riera Villanueva, Marc|||0000-0002-2768-5703
  • González Colás, Antonio María|||0000-0002-0009-0996
ReRAM-based accelerators exhibit enormous potential to increase computational efficiency for DNN inference tasks, delivering significant performance and energy savings over traditional platforms. By incorporating adaptive scheduling, these accelerators dynamically adjust to DNN requirements, optimizing allocation of constrained hardware resources. However, ReRAM cells have limited endurance cycles due to wear-out from multiple updates for each inference execution, which shortens the lifespan of ReRAM-based accelerators and presents a practical challenge in positioning them as alternatives to conventional platforms like TPUs. Addressing these endurance limitations is essential for making ReRAM-based solutions viable for long-term, high-performance DNN inference. To address the lifespan limitations of ReRAM-based accelerators, we introduce Hamun, an approximate computing method designed to extend the lifespan of ReRAM-based accelerators through a range of optimizations. Hamun incorporates a novel mechanism that detects faulty cells due to wear-out and retires them, avoiding in this way their otherwise adverse impact on DNN accuracy. Moreover, Hamun extends the lifespan of ReRAM-based accelerators by adapting wear-leveling techniques across various abstraction levels of the accelerator and implementing a batch execution scheme to maximize ReRAM cell usage for multiple inferences. Additionally, Hamun introduces a new approximation method that leverages the fault tolerance characteristics of DNNs to delay the retirement of worn-out cells, reducing the performance penalty of retired cells and further extending the accelerator’s lifespan. On average, evaluated on a set of popular DNNs, Hamun demonstrates an improvement in lifespan of 13.2x over a state-of-the-art baseline. The main contributors to this improvement are the fault handling and batch execution schemes, which provide 4.6x and 2.6x lifespan improvements respectively., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RBI00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, and the ICREA Academia program., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

LOCATOR: Low-power ORB accelerator for autonomous cars

  • Taranco Serna, Raúl|||0000-0002-1564-4365
  • Arnau Montañés, José María|||0000-0002-0336-9191
  • González Colás, Antonio María|||0000-0002-0009-0996
Simultaneous Localization And Mapping (SLAM) is crucial for autonomous navigation. ORB-SLAM is a state-of-the-art Visual SLAM system based on cameras used for self-driving cars. In this paper, we propose a high-performance, energy-efficient, and functionally accurate hardware accelerator for ORB-SLAM, focusing on its most time-consuming stage: Oriented FAST and Rotated BRIEF (ORB) feature extraction. The Rotated BRIEF (rBRIEF) descriptor generation is the main bottleneck in ORB computation, as it exhibits highly irregular access patterns to local on-chip memories causing a high-performance penalty due to bank conflicts. We introduce a technique to find an optimal static pattern to perform parallel accesses to banks based on a genetic algorithm. Furthermore, we propose the combination of an rBRIEF pixel duplication cache, selective ports replication, and pipelining to reduce latency without compromising cost. The accelerator achieves a reduction in energy consumption of 14597× and 9609×, with respect to high-end CPU and GPU platforms, respectively., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020- 113172RB-I00, the ICREA Academia program and the FPU grant FPU18/04413, Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

CREW: Computation reuse and efficient weight storage for hardware-accelerated MLPs and RNNs

  • Riera Villanueva, Marc
  • Arnau Montañés, José María
  • González Colás, Antonio María
Deep Neural Networks (DNNs) have achieved tremendous success for cognitive applications. The core operation in a DNN is the dot product between quantized inputs and weights. Prior works exploit the weight/input repetition that arises due to quantization to avoid redundant computations in Convolutional Neural Networks (CNNs). However, in this paper we show that their effectiveness is severely limited when applied to FullyConnected (FC) layers, which are commonly used in state-of-the-art DNNs, as it is the case of modern Recurrent Neural Networks (RNNs) and Transformer models. To improve energy-efficiency of FC computation we present CREW, a hardware accelerator that implements Computation Reuse and an Efficient Weight Storage mechanism to exploit the large number of repeated weights in FC layers. CREW first performs the multiplications of the unique weights by their respective inputs and stores the results in an on-chip buffer. The storage requirements are modest due to the small number of unique weights and the relatively small size of the input compared to convolutional layers. Next, CREW computes each output by fetching and adding its required products. To this end, each weight is replaced offline by an index in the buffer of unique products. Indices are typically smaller than the quantized weights, since the number of unique weights for each input tends to be much lower than the range of quantized weights, which reduces storage and memory bandwidth requirements. Overall, CREW greatly reduces the number of multiplications and provides significant savings in model memory footprint and memory bandwidth usage. We evaluate CREW on a diverse set of modern DNNs. On average, CREW provides 2.61x speedup and 2.42x energy savings over a TPU-like accelerator. Compared to UCNN, a state-of-art computation reuse technique, CREW achieves 2.10x speedup and 2.08x energy savings on average., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, the ICREA Academia program, and the Spanish Ministry of Education under grant FPU15/02294., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

E-BATCH: Energy-efficient and high-throughput RNN batching

  • Silfa Feliz, Franyell Antonio
  • Arnau Montañés, José María
  • González Colás, Antonio María
Recurrent Neural Network (RNN) inference exhibits low hardware utilization due to the strict data dependencies across time-steps. Batching multiple requests can increase throughput. However, RNN batching requires a large amount of padding since the batched input sequences may vastly differ in length. Schemes that dynamically update the batch every few time-steps avoid padding. However, they require executing different RNN layers in a short time span, decreasing energy efficiency. Hence, we propose E-BATCH, a low-latency and energy-efficient batching scheme tailored to RNN accelerators. It consists of a runtime system and effective hardware support. The runtime concatenates multiple sequences to create large batches, resulting in substantial energy savings. Furthermore, the accelerator notifies it when the evaluation of an input sequence is done. Hence, a new input sequence can be immediately added to a batch, thus largely reducing the amount of padding. E-BATCH dynamically controls the number of time-steps evaluated per batch to achieve the best trade-off between latency and energy efficiency for the given hardware platform. We evaluate E-BATCH on top of E-PUR and TPU. E-BATCH improves throughput by 1.8× and energy efficiency by 3.6× in E-PUR, whereas in TPU, it improves throughput by 2.1× and energy efficiency by 1.6×, over the state-of-the-art., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, and the ICREA Academia program., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

SIMIL: SIMple Issue Logic for GPUs

  • Huerta Gañán, Rodrigo
  • Cruz Díaz, Josep Llorenç
  • Arnau Montañes, Jose-Maria
  • González Colás, Antonio María
GPU architectures have become popular for executing general-purpose programs. In particular, they are some of the most efficient architectures for machine learning applications which are among the most trendy and demanding applications nowadays. This paper presents SIMIL (SIMple Issue Logic for GPUs), an architectural modification to the issue stage that replaces scoreboards with a Dependence Matrix to track dependencies among instructions and avoid data hazards. We show that a Dependence Matrix is more effective in the presence of repetitive use of source operands, which is common in many applications. Besides, a Dependence Matrix with minor extensions can also support a simplistic out-of-order issue. Evaluations on an NVIDIA Tesla V100-like GPU show that SIMIL provides a speed-up of up to 2.39 in some machine learning programs and 1.31 on average for various benchmarks, while it reduces energy consumption by 12.81%, with only 1.5% area overhead. We also show that SIMIL outperforms a recently proposed approach for out-of-order issue that uses register renaming., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, and the ICREA Academia program., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

A low-cost and do-it-yourself device for pumping monitoring in deep aquifers

  • Aliagas, Carles
  • Pérez Foguet, Agustí
  • Meseguer Pallarès, Roc
  • Millán Marco, Pere
  • Molina Clemente, Carlos Maria
Water crises due to climate change, high population growth and increasing demands from industry and agriculture claim for increasing efficiency and universalizing water resources management strategies and techniques. Water monitoring helps providing necessary evidences for making sound decisions about managing water resources both now and in the future. In this work, a low cost and “do it yourself” communication device is proposed to record water production and energy consumption of electric pumpings from deep boreholes/wells, and to predict the impact of the ongoing and previous pumpings in the evolution of the water level in the aquifer. The proposal incorporates an edge-computing approach for the simulation of the aquifer response in real-time. Computation of results of interest is performed at the sensor, minimizing communication requirements and ensuring almost immediate results. An approximated solution to physically based modeling of aquifer response is computed thanks to the a priori expression of the water level time evolution in a reduced basis. The accuracy is enough to detect deviations from expected behaviour. The energy consumption of the device is very much reduced with respect to that of a full modelling, which can be computed off-line for calibrating reduced model parameters and perform detailed analyses. The device is tested in a real scenario, in a mountain subbasin of the Ebro river in Spain, obtaining a good trade-off between performance, price, and energy consumption., This research has been partly supported by EU under grant agreement N. 825184 and funded by the Government of Spain under contracts PID2019-106774RB-C21, PID2019-106774RB-C22, and PID2020-113172RB-I00 and by the Government of Catalonia as Consolidated Research Groups 2017-SGR-688 and 2017-SGR-990, and Pre-consolidated Research Group 2017-SGR-1496. The APC was funded by the Open program from Universitat Rovira i Virgili., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

ARAS: Adaptive low-cost ReRAM-based accelerator for DNNs

  • Sabri Abrebekoh, Mohammad
  • Riera Villanueva, Marc
  • González Colás, Antonio María
Processing Using Memory (PUM) accelerators have the potential to perform Deep Neural Network (DNN) inference by using arrays of memory cells as computation engines. Among various memory technologies, ReRAM crossbars show promising performance in computing dot-product operations in the analog domain. Nevertheless, the expensive writing procedure of ReRAM cells has led researchers to design accelerators whose crossbars have enough capacity to store the full DNN. Given the tremendous and continuous increase in DNN model sizes, this approach is unfeasible for some networks, or inefficient due to the huge hardware requirements. Those accelerators lack the flexibility to adapt to any given DNN model, facing an adaptability challenge. To address this issue we introduce ARAS, a cost-effective ReRAM-based accelerator that employs an offline scheduler to adapt different DNNs to the resource-limited hardware. ARAS also overlaps the computation of a layer with the weight writing of several layers to mitigate the high writing latency of ReRAM. Furthermore, ARAS introduces three optimizations aimed at reducing the energy overheads of writing in ReRAM. Our key optimization capitalizes on the observation that DNN weights can be re-encoded to augment their similarity between layers, increasing the amount of bitwise values that are equal or similar when overwriting ReRAM cells and, hence, reducing the amount of energy required to update the cells. Overall, ARAS greatly reduces the ReRAM writing activity. We evaluate ARAS on a popular set of DNNs. ARAS provides up to 2. 2x speedup and 45% energy savings over a baseline PUM accelerator without any optimization. Compared to a TPU-like accelerator, ARAS provides up to 1. 5x speedup and 62% energy savings., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grants PID2020-113172RBI00 and PID2024-155476OB-I00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, the ICREA Academia program, and the FPI-UPC 2020 grant., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

SHARP: An adaptable, energy-efficient accelerator for recurrent neural networks

  • Yazdani Aminabadi, Reza
  • Ruwase, Olatunji
  • Zhang, Minjia
  • He, Yuxiong
  • Arnau Montañés, José María
  • González Colás, Antonio María
The effectiveness of Recurrent Neural Networks (RNNs) for tasks such as Automatic Speech Recognition has fostered interest in RNN inference acceleration. Due to the recurrent nature and data dependencies of RNN computations, prior work has designed customized architectures specifically tailored to the computation pattern of RNN, getting high computation efficiency for certain chosen model sizes. However, given that the dimensionality of RNNs varies a lot for different tasks, it is crucial to generalize this efficiency to diverse configurations. In this work, we identify adaptiveness as a key feature that is missing from today’s RNN accelerators. In particular, we first show the problem of low resource utilization and low adaptiveness for the state-of-the-art RNN implementations on GPU, FPGA, and ASIC architectures. To solve these issues, we propose an intelligent tiled-based dispatching mechanism for increasing the adaptiveness of RNN computation, in order to efficiently handle the data dependencies. To do so, we propose Sharp as a hardware accelerator, which pipelines RNN computation using an effective scheduling scheme to hide most of the dependent serialization. Furthermore, Sharp employs dynamic reconfigurable architecture to adapt to the model’s characteristics. Sharp achieves 2×, 2.8×, and 82× speedups on average, considering different RNN models and resource budgets, compared to the state-of-the-art ASIC, FPGA, and GPU implementations, respectively. Furthermore, we provide significant energy reduction with respect to the previous solutions, due to the low power dissipation of Sharp (321 GFLOPS/Watt)., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, and the ICREA Academia program., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

LOCATOR: Low-power ORB accelerator for autonomous cars

  • Taranco Serna, Raúl
  • Arnau Montañés, José María
  • González Colás, Antonio María
Simultaneous Localization And Mapping (SLAM) is crucial for autonomous navigation. ORB-SLAM is a state-of-the-art Visual SLAM system based on cameras used for self-driving cars. In this paper, we propose a high-performance, energy-efficient, and functionally accurate hardware accelerator for ORB-SLAM, focusing on its most time-consuming stage: Oriented FAST and Rotated BRIEF (ORB) feature extraction. The Rotated BRIEF (rBRIEF) descriptor generation is the main bottleneck in ORB computation, as it exhibits highly irregular access patterns to local on-chip memories causing a high-performance penalty due to bank conflicts. We introduce a technique to find an optimal static pattern to perform parallel accesses to banks based on a genetic algorithm. Furthermore, we propose the combination of an rBRIEF pixel duplication cache, selective ports replication, and pipelining to reduce latency without compromising cost. The accelerator achieves a reduction in energy consumption of 14597× and 9609×, with respect to high-end CPU and GPU platforms, respectively., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020- 113172RB-I00, the ICREA Academia program and the FPU grant FPU18/04413, Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Exploiting beam search confidence for energy-efficient speech recognition

  • Pinto Rivero, Dennis
  • Arnau Montañés, José María
  • Riera Villanueva, Marc
  • Cruz Díaz, Josep Llorenç
  • González Colás, Antonio María
With mobile and embedded devices getting more integrated in our daily lives, the focus is increasingly shifting toward human-friendly interfaces, making automatic speech recognition (ASR) a central player as the ideal means of interaction with machines. ASR is essential for many cognitive computing applications, such as speech-based assistants, dictation systems and real-time language translation. Consequently, interest in speech technology has grown in the last few years, with more systems being proposed and higher accuracy levels being achieved, even surpassing human accuracy. However, highly accurate ASR systems are computationally expensive, requiring on the order of billions of arithmetic operations to decode each second of audio, which conflicts with a growing interest in deploying ASR on edge devices. On these devices, efficient hardware acceleration is key for achieving acceptable performance. In this paper, we propose a technique to improve the energy efficiency and performance of ASR systems, focusing on low-power hardware for edge devices. We focus on optimizing the DNN-based acoustic model evaluation, as we have observed it to be the main bottleneck in popular ASR systems, by leveraging run-time information from the beam search. By doing so, we reduce energy and execution time of the acoustic model evaluation by 25.6  and 25.9 %, respectively, with negligible accuracy loss., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, the ICREA Academia program, and the Spanish MICINN Ministry under grant BES-2017-080605., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Dynamic sampling rate: harnessing frame coherence in graphics applications for energy-efficient GPUs

  • Anglada Sánchez, Martí
  • de Lucas Casamayor, Enrique
  • Parcerisa Bundó, Joan Manuel
  • Aragón Alcaraz, Juan Luis
  • González Colás, Antonio María
In real-time rendering, a 3D scene is modelled with meshes of triangles that the GPU projects to the screen. They are discretized by sampling each triangle at regular space intervals to generate fragments which are then added texture and lighting effects by a shader program. Realistic scenes require detailed geometric models, complex shaders, high-resolution displays and high screen refreshing rates, which all come at a great compute time and energy cost. This cost is often dominated by the fragment shader, which runs for each sampled fragment. Conventional GPUs sample the triangles once per pixel; however, there are many screen regions containing low variation that produce identical fragments and could be sampled at lower than pixel-rate with no loss in quality. Additionally, as temporal frame coherence makes consecutive frames very similar, such variations are usually maintained from frame to frame. This work proposes Dynamic Sampling Rate (DSR), a novel hardware mechanism to reduce redundancy and improve the energy efficiency in graphics applications. DSR analyzes the spatial frequencies of the scene once it has been rendered. Then, it leverages the temporal coherence in consecutive frames to decide, for each region of the screen, the lowest sampling rate to employ in the next frame that maintains image quality. We evaluate the performance of a state-of-the-art mobile GPU architecture extended with DSR for a wide variety of applications. Experimental results show that DSR is able to remove most of the redundancy inherent in the color computations at fragment granularity, which brings average speedups of 1.68x and energy savings of 40%., This work has been supported by the the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (Grant No. 833057), Spanish State Research Agency (MCIN/AEI) under Grant PID2020-113172RB-I00, the ICREA Academia program, and the Generalitat de Catalunya under Grant FI-DGR 2016. Funding was provided by Ministerio de Economía, Industria y Competitividad, Gobierno de España (Grant No. TIN2016-75344-R)., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Triangle Dropping: An occluded-geometry predictor for energy-efficient mobile GPUs

  • Corbalán Navarro, David
  • Aragon Alcaraz, Juan Luis
  • Anglada Sánchez, Martí
  • Parcerisa Bundó, Joan Manuel
  • González Colás, Antonio María
This article proposes a novel micro-architecture approach for mobile GPUs aimed at early removing the occluded geometry in a scene by leveraging frame-to-frame coherence, thus reducing the overall energy consumption. Mobile GPUs commonly implement a Tile-Based Rendering (TBR) architecture that differentiates two main phases: the Geometry Pipeline, where all the geometry of a scene is processed; and the Raster Pipeline, where primitives are rendered in a framebuffer. After the Geometry Pipeline, only non-culled primitives inside the camera’s frustum are stored into the Parameter Buffer, a data structure stored in DRAM. However, among the non-culled primitives there is a significant amount that are rendered but non-visible at all, resulting in useless computations. On average, 60% of those primitives are completely occluded in our benchmarks. Despite TBR architectures use on-chip caches for the Parameter Buffer, about 46% of the DRAM traffic still comes from accesses to such buffer. The proposed Triangle Dropping technique leverages the visibility information computed along the Raster Pipeline to predict the primitives’ visibility in the next frame to early discard those that will be totally occluded, drastically reducing Parameter Buffer accesses. On average, our approach achieves overall 14.5% energy savings, 28.2% energy-delay product savings, and a speedup of 20.2%., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant no. 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00 (AEI/FEDER, EU), and the ICREA Academia program. D. Corbalán-Navarro has been also supported by a PhD research fellowship from the University of Murcia’s “Plan Propio de Investigación., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Towards efficient LUT-based PIM: A scalable and low-power approach for modern workloads

  • Khabbazan, Bahareh
  • Riera Villanueva, Marc
  • González Colás, Antonio María
Data movement in memory-intensive workloads is several orders of magnitude more energy-consuming than computation. Frequent memory-processor transfers make this a critical bottleneck. Processing-using-memory (PuM) alleviates this by enabling in-memory computation and reducing data movement. In this paper, we propose Lama, a LUT-based PuM architecture that efficiently executes SIMD operations by supporting independent column accesses within each mat of a DRAM subarray. Lama minimizes the number of energy-intensive memory activation commands, which are the primary source of overhead in most PuM architectures. Besides, Lama supports up to 8-bit operand precision without decomposing computations, unlike prior PuM solutions, while incurring only a 2.47% area overhead. Our evaluation shows that Lama achieves an average performance improvement of 8.5× over state-of-the-art PuM architectures and a 3.8× improvement over CPU, along with energy efficiency gains of 6.9×/8×, respectively, for bulk 8-bit multiplication. We introduce LamaAccel, an HBM-based PuM accelerator that leverages Lama to accelerate LLM inference. By transforming MAC computations into energy-efficient addition and counting operations, LamaAccel achieves up to 9.3×/19.2× energy reduction and 4.8×/9.8× speedup over a TPU/GPU, as well as up to 5.8× lower energy consumption and 2.1× higher performance compared to a state-of-the-art PuM baseline., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grants PID2020-113172RB-I00 and PID2024-155476OBI00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, the ICREA Academia program, and the FPI-UPC 2020 grant., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

An energy-efficient near-data processing accelerator for DNNs to optimize memory accesses

  • Khabbazan, Bahareh
  • Sabri Abrebekoh, Mohammad
  • Riera Villanueva, Marc
  • González Colás, Antonio María
The constant growth of DNNs makes them challenging to implement and run efficiently on traditional computecentric architectures. Some accelerators have attempted to add more compute units and on-chip buffers to solve the memory wall problem without much success, and sometimes even worsening the issue since more compute units also require higher memory bandwidth. Prior works have proposed the design of memorycentric architectures based on the Near-Data Processing (NDP) paradigm. NDP seeks to break the memory wall by moving the computations closer to the memory hierarchy, reducing the data movements and their cost as much as possible. The 3D-stacked memory is especially appealing for DNN accelerators due to its high-density/low-energy storage and near-memory computation capabilities to perform the DNN operations massively in parallel. However, memory accesses remain as the main bottleneck for running modern DNNs efficiently. To improve the efficiency of DNN inference we present QeiHaN, a hardware accelerator that implements a 3D-stacked memory-centric weight storage scheme to take advantage of a logarithmic quantization of activations. In particular, since activations of FC and CONV layers of modern DNNs are commonly represented as powers of two with negative exponents, QeiHaN performs an implicit in-memory bit-shifting of the DNN weights to reduce memory activity. Only the meaningful bits of the weights required for the bit-shift operation are accessed. Overall, QeiHaN reduces memory accesses by 25% compared to a standard memory organization. We evaluate QeiHaN on a popular set of DNNs. On average, QeiHaN provides4.3¿speedup and3.5¿energy savings over a Neurocube-like accelerator., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI), Spain under grant PID2020-113172RB-I00, the Catalan Agency for University and Research (AGAUR), Spain under grant 2021 SGR 00383, and the ICREA Academia program, Spain., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Irregular accesses reorder unit: improving GPGPU memory coalescing for graph-based workloads

  • Segura Salvador, Albert
  • Arnau Montañés, José María
  • González Colás, Antonio María
GPGPU architectures have become the dominant platform for massively parallel workloads, delivering high performance and energy efficiency for popular applications such as machine learning, computer vision or self-driving cars. However, irregular applications, such as graph processing, fail to fully exploit GPGPU resources due to their divergent memory accesses that saturate the memory hierarchy. To reduce the pressure on the memory subsystem for divergent memory-intensive applications, programmers must take into account SIMT execution model and memory coalescing in GPGPUs, devoting significant efforts in complex optimization techniques. Despite these efforts, we show that irregular graph processing still suffers from low GPGPU performance. We observe that in many irregular applications the mapping of data to threads can be safely changed. In other words, it is possible to relax the strict relationship between thread and data processed to reduce memory divergence. Based on this observation, we propose the Irregular accesses Reorder Unit (IRU), a novel hardware extension tightly integrated in the GPGPU pipeline. The IRU reorders data processed by the threads on irregular accesses to improve memory coalescing, i.e., it tries to assign data elements to threads as to produce coalesced accesses in SIMT groups. Furthermore, the IRU is capable of filtering and merging duplicated accesses, significantly reducing the workload. Programmers can easily utilize the IRU with a simple API, or let the compiler issue instructions from our extended ISA. We evaluate our proposal for state-of-the-art graph-based algorithms and a wide selection of applications. Results show that the IRU achieves a memory coalescing improvement of 1.32x and a 46% reduction in the overall traffic in the memory hierarchy, which results in 1.33x speedup and 13% energy savings on average, while incurring in a small 5.6% area overhead., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00 and the ICREA Academia program., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Mixture-of-Rookies: saving DNN computations by predicting ReLU outputs

  • Pinto Rivero, Dennis
  • Arnau Montañés, José María
  • Riera Villanueva, Marc
  • Cruz Díaz, Josep Llorenç
  • González Colás, Antonio María
Deep Neural Networks (DNNs) are widely used in many application domains. However, they require a vast amount of computations and memory accesses to deliver outstanding accuracy. In this paper, we propose a scheme to predict whether the output of each ReLu activated neuron will be a zero or a positive number in order to skip the computation of those neurons that will likely output a zero. Our predictor, named Mixture-of-Rookies, combines two inexpensive components. The first one exploits the high linear correlation between binarized (1-bit) and full-precision (8-bit) dot products, whereas the second component clusters together neurons that tend to output zero at the same time. We propose a novel clustering scheme based on analysis of angles, as the sign of the dot product of two vectors depends on the cosine of the angle between them. We implement our hybrid zero output predictor on top of a state-of-the-art DNN accelerator. Experimental results show that our scheme introduces a small area overhead of 5.3% while achieving a speedup of 1.2x and reducing energy consumption by 16.5% on average for a set of diverse DNNs., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RBI00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, the ICREA Academia program, and the Spanish MICINN Ministry under grant BES-2017-080605., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

DNN pruning with principal component analysis and connection importance estimation

  • Riera Villanueva, Marc
  • Arnau Montañés, José María
  • González Colás, Antonio María
DNN pruning reduces memory footprint and computational work of DNN-based solutions to improve performance and energy-efficiency. An effective pruning scheme should be able to systematically remove connections and/or neurons that are unnecessary or redundant, reducing the DNN size without any loss in accuracy. In this paper we show that some of the most popular pruning schemes, such as the Near Zero Weights, require an extremely time-consuming iterative process that requires retraining the DNN many times to tune the pruning hyperparameters. Then, we propose a DNN pruning scheme based on Principal Component Analysis and relative importance of each neuron’s connection (PCA+DIRIE) that automatically finds the optimized DNN in one shot without requiring hand-tuning of multiple parameters. The experimental results show the effectiveness of our method on several benchmarks. Notably, on ImageNet, PCA+DIRIE can prune up to 60% of ResNet-50 with negligible impact on accuracy., This work has been supported by the CoCoUnit ERC Advanced Grantof the EU’s Horizon 2020 program (grant No 833057), the Spanish StateResearch Agency under grant PID2020-113172RB-I00 (MCIN/AEI), theICREA Academia program, and the Spanish Ministry of Education undergrant FPU15/02294., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Hamun: an approximate computing method to prolong the lifespan of ReRAM-based accelerators

  • Sabri Abrebekoh, Mohammad
  • Riera Villanueva, Marc
  • González Colás, Antonio María
ReRAM-based accelerators exhibit enormous potential to increase computational efficiency for DNN inference tasks, delivering significant performance and energy savings over traditional platforms. By incorporating adaptive scheduling, these accelerators dynamically adjust to DNN requirements, optimizing allocation of constrained hardware resources. However, ReRAM cells have limited endurance cycles due to wear-out from multiple updates for each inference execution, which shortens the lifespan of ReRAM-based accelerators and presents a practical challenge in positioning them as alternatives to conventional platforms like TPUs. Addressing these endurance limitations is essential for making ReRAM-based solutions viable for long-term, high-performance DNN inference. To address the lifespan limitations of ReRAM-based accelerators, we introduce Hamun, an approximate computing method designed to extend the lifespan of ReRAM-based accelerators through a range of optimizations. Hamun incorporates a novel mechanism that detects faulty cells due to wear-out and retires them, avoiding in this way their otherwise adverse impact on DNN accuracy. Moreover, Hamun extends the lifespan of ReRAM-based accelerators by adapting wear-leveling techniques across various abstraction levels of the accelerator and implementing a batch execution scheme to maximize ReRAM cell usage for multiple inferences. Additionally, Hamun introduces a new approximation method that leverages the fault tolerance characteristics of DNNs to delay the retirement of worn-out cells, reducing the performance penalty of retired cells and further extending the accelerator’s lifespan. On average, evaluated on a set of popular DNNs, Hamun demonstrates an improvement in lifespan of 13.2x over a state-of-the-art baseline. The main contributors to this improvement are the fault handling and batch execution schemes, which provide 4.6x and 2.6x lifespan improvements respectively., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RBI00, the Catalan Agency for University and Research (AGAUR) under grant 2021SGR00383, and the ICREA Academia program., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

A survey of near-data processing architectures for neural networks

  • Hassanpour, Mehdi
  • Riera Villanueva, Marc
  • González Colás, Antonio María
Data-intensive workloads and applications, such as machine learning (ML), are fundamentally limited by traditional computing systems based on the von-Neumann architecture. As data movement operations and energy consumption become key bottlenecks in the design of computing systems, the interest in unconventional approaches such as Near-Data Processing (NDP), machine learning, and especially neural network (NN)-based accelerators has grown significantly. Emerging memory technologies, such as ReRAM and 3D-stacked, are promising for efficiently architecting NDP-based accelerators for NN due to their capabilities to work as both high-density/low-energy storage and in/near-memory computation/search engine. In this paper, we present a survey of techniques for designing NDP architectures for NN. By classifying the techniques based on the memory technology employed, we underscore their similarities and differences. Finally, we discuss open challenges and future perspectives that need to be explored in order to improve and extend the adoption of NDP architectures for future computing platforms. This paper will be valuable for computer architects, chip designers, and researchers in the area of machine learning., This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020-113172RB-I00, and the ICREA Academia program., Peer Reviewed, Postprint (published version)