Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 4
Encontrada(s) 1 página(s)

UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Towards a library of deep neural networks for experimenting with on-device training on microcontrollers

  • Aatab, Soufiane
  • Freitag, Fèlix|||0000-0001-5438-479X
Despite the limited computing capacity of microcontrollers, there is a growing interest in training neural networks on these devices for tiny machine learning applications. The traditional approach is to train models off-device on powerful machines, but this limits the model's ability on the microcontroller to adapt to changing data. A limitation for further research on on-device training is the lack of open source deep neural network implementations for the training on microcontroller boards. In this paper, we present the design and implementation of a library in C++ to enable neural network training and inference on microcontroller boards. We conduct a functional validation of the implementation by comparing a Convolutional Neural Network (CNN) and autoencoder model defined with our library with a reference implementation. We then integrate the library into a machine learning application and flash it on an Arduino Portenta board to demonstrate its usability for on-device training. The developed library is open source and is flexible for defining different neural network architectures. Researchers and developers can use the given implementation as a tool for building new trainable machine learning applications, but can also extend it with recent model optimization techniques for tiny nodes from the research in this field., This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 957258 — ASSIST-IoT, and was partially supported by the Spanish Government under contract PID2019-106774RB-C21, and the Generalitat de Catalunya as Consolidated Research Group 2021-SGR-01059., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Integrating a neural network library in embedded federated learning

  • Aatab, Soufiane
  • Freitag, Fèlix|||0000-0001-5438-479X
Federated learning has become a popular collaborative machine learning approach in which clients use their local data for training. This results in bandwidth savings since the training data does not have to be transferred to a central node, and also protects the privacy of these data. However, doing federated learning in tiny Internet of Things (IoT) devices is challenging due to the resource constraints of the devices and lack of experimental tools. In this paper, we develop a federated learning implementation that allows training of different machine learning models on embedded IoT devices. Our starting point is a library of neural network models which we have developed. We analyze the metadata to describe these models, design the interaction process between clients and server of a federated learning network, and implement the software. We validate the system by conducting experiments with our implementation deployed on real embedded boards. We observe in a non-IID experiment the capacity of federated learning to provide a trained model whose capacities go beyond those that the node could train by itself. Being heterogeneous data distribution a realistic scenario in distributed IoT devices, we believe that our implementation can be useful for the community as a tool to explore the training of neural networks on microcontrollers with the support of federated learning., This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 957258 — ASSIST-IoT, and was partially supported by the Spanish Government under contract PID2019-106774RB-C21, and the Generalitat de Catalunya as Consolidated Research Group 2021-SGR-01059., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Towards a library of deep neural networks for experimenting with on-device training on microcontrollers

  • Aatab, Soufiane
  • Freitag, Fèlix
Despite the limited computing capacity of microcontrollers, there is a growing interest in training neural networks on these devices for tiny machine learning applications. The traditional approach is to train models off-device on powerful machines, but this limits the model's ability on the microcontroller to adapt to changing data. A limitation for further research on on-device training is the lack of open source deep neural network implementations for the training on microcontroller boards. In this paper, we present the design and implementation of a library in C++ to enable neural network training and inference on microcontroller boards. We conduct a functional validation of the implementation by comparing a Convolutional Neural Network (CNN) and autoencoder model defined with our library with a reference implementation. We then integrate the library into a machine learning application and flash it on an Arduino Portenta board to demonstrate its usability for on-device training. The developed library is open source and is flexible for defining different neural network architectures. Researchers and developers can use the given implementation as a tool for building new trainable machine learning applications, but can also extend it with recent model optimization techniques for tiny nodes from the research in this field., This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 957258 — ASSIST-IoT, and was partially supported by the Spanish Government under contract PID2019-106774RB-C21, and the Generalitat de Catalunya as Consolidated Research Group 2021-SGR-01059., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Integrating a neural network library in embedded federated learning

  • Aatab, Soufiane
  • Freitag, Fèlix
Federated learning has become a popular collaborative machine learning approach in which clients use their local data for training. This results in bandwidth savings since the training data does not have to be transferred to a central node, and also protects the privacy of these data. However, doing federated learning in tiny Internet of Things (IoT) devices is challenging due to the resource constraints of the devices and lack of experimental tools. In this paper, we develop a federated learning implementation that allows training of different machine learning models on embedded IoT devices. Our starting point is a library of neural network models which we have developed. We analyze the metadata to describe these models, design the interaction process between clients and server of a federated learning network, and implement the software. We validate the system by conducting experiments with our implementation deployed on real embedded boards. We observe in a non-IID experiment the capacity of federated learning to provide a trained model whose capacities go beyond those that the node could train by itself. Being heterogeneous data distribution a realistic scenario in distributed IoT devices, we believe that our implementation can be useful for the community as a tool to explore the training of neural networks on microcontrollers with the support of federated learning., This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 957258 — ASSIST-IoT, and was partially supported by the Spanish Government under contract PID2019-106774RB-C21, and the Generalitat de Catalunya as Consolidated Research Group 2021-SGR-01059., Peer Reviewed, Postprint (author's final draft)