Pasar al contenido principal

TECNICAS BASADAS EN DATOS PARA MEJORAR LA CALIDAD DE LA INFORMACION EN REDES DE NODOS IOT

PID2022-138155OB-I00

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal para Impulsar la Investigación Científico-Técnica y su Transferencia
Subprograma Subprograma Estatal de Generación de Conocimiento
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2022
Plan Estatal 2021-2023 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSITAT POLITECNICA DE CATALUNYA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 25
Encontrada(s) 1 página(s)

UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

A neural network-based system for automated refrigeration alarm management in food retail

  • Ferrer Cid, Pau|||0000-0003-2112-8516
  • Albets Chico, Xavier
  • Barceló Ordinas, José María|||0000-0002-9738-2425
  • García Vidal, Jorge|||0000-0001-5969-1182
The increasing use of refrigeration monitoring platforms in the food retail sector is crucial for enhancing operational efficiency and mitigating food waste. These systems are essential for ensuring food safety and quality, as they alert personnel to abnormal temperature readings, which are often caused by equipment malfunctions or operational issues. However, the resulting high volume of alarms poses a significant challenge, requiring tailored management strategies and continuous supervision by expert staff. This article presents an intelligent alarm labeling system based on neural networks that evaluates various alarm dimensions. The proposed system effectively detects recurrent patterns and determines a device’s status based on its temperature measurements. The system was validated using a massive monitoring dataset from different supermarkets in Catalonia, Spain. The results show that the system can detect recurrent alarms with 98% accuracy and correctly identify device states with 93% accuracy, enabling more effective temperature alarm management., This work has been financed by the primary project AKO 4.4: Nueva plataforma de soluciones digitales de control, predicción y soporte a la decisión, para mejorar la eficiencia energética, garantizar la seguridad y evitar las mermas de alimentos en los activos de refrigeración de la cadena del frío [grant number PAG-020100-2023-104], under the Project grant PAG-010000-2023-9: modernización de los sistemas de trazabilidad, seguridad alimentaria y gestión de la calidad de la cadena de valor del sector porcino, vacuno y avícola. This Project grant has been awarded under the Proyecto Estratégico para la Recuperación y Transformación Económica Agroalimentario (PERTE Agroalimentario) by the Ministerio de Industria, Comercio y Turismo of Spain. This work is also supported by the National Spanish funding MCIN/AEI/ 10.13039/501100011033 and by “ERDF A way of making Europe” [grant number PID2022-138155OB-I00], and by regional project [grant number 2021SGR-01059]., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

A data-driven framework for air quality sensor networks

  • Ferrer Cid, Pau|||0000-0003-2112-8516
  • Paredes Ahumada, Juan Antonio
  • Allka, Xhensilda|||0000-0003-2187-9985
  • Guerrero Zapata, Manel|||0000-0002-7983-2525
  • Barceló Ordinas, José María|||0000-0002-9738-2425
  • García Vidal, Jorge|||0000-0001-5969-1182
In this article, we present our research vision of a framework for obtaining quality data in air quality monitoring networks using low-cost sensors (LCSs). The use of LCS networks is gaining increasing acceptance in many IoT air quality applications. However, data quality and reliability issues are a major barrier to widespread adoption, which means that the pre-processing tasks that are critical to achieving the required levels of data quality are crucial aspects of LCS network designs. The proposed framework takes advantage of a layered architecture, which has also proven useful in other fields, and from which we show the challenges and state-of-the-art techniques for obtaining quality data. In addition, we show its usefulness in application cases, including a real case with data measured by a LCS deployment measuring O 3 in the area of Barcelona, Spain., This work is supported by Grant PID2022-138155OB-I00 funded by MCIN/AEI/ 10.13039/501100011033 and by “ERDF A way of making Europe”, CDTI MIG20221061, by regional project 2021SGR-01059, and with the support of Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Evaluating missing value imputation strategies to enhance IoT data availability in the edge

  • Ferrer Cid, Pau|||0000-0003-2112-8516
  • Barceló Ordinas, José María|||0000-0002-9738-2425
  • García Vidal, Jorge|||0000-0001-5969-1182
  • Avila Torrado, Antonio
The rapid implementation of Internet of Things (IoT) technologies in industrial and air quality monitoring applications has resulted in large amounts of data being acquired. In addition, the integration of the artificial intelligence of things (AIoT) enables decision-making at edge nodes, involving the execution of data-driven models close to the data source. Data loss has become a limiting factor in AI-based applications. One way to address missing data is to leverage data from co-existing sensors, which are potentially correlated, to impute missing values. This article evaluates a set of missing value imputation (MVI) techniques in an edge node environment, where constraints on data availability and computational complexity must be met. Specifically, models such as multiple imputation by chained equations (MICE), k-nearest neighbors (KNN), and models using variational autoencoders (VAE) are evaluated. A comprehensive evaluation is presented covering different scenarios of missing data in terms of the percentage of missing values, bursts of missing values, and the size of the data windows stored on edge nodes. This evaluation uses real sensor data from an air quality monitoring network and an industrial sensor network. The results show that the VAE is able to obtain good imputation performances (R^2 greater than 0.90) while providing good uncertainty quantification (UQ) estimates with around 90%–95% of samples falling within the estimated confidence intervals. Moreover, a transfer learning-based VAE has been shown to adapt to the non-stationary nature of IoT sensing signals, and all methods have proven to be efficient in terms of execution time for the edge setting., This work was supported by Grant PID2022-138155OB-I00 funded by MCIN/AEI/10.13039/501100011033 and by ‘‘ERDF A way of making Europe’’, CDTI MIG-20221061, by regional project 2021 SGR 01059, and with the support of Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

W-UDDR: A unified framework for automated drift detection in IoT-based air quality monitoring systems

  • Allka, Xhensilda|||0000-0003-2187-9985
  • Ferrer Cid, Pau|||0000-0003-2112-8516
  • Barceló Ordinas, José María|||0000-0002-9738-2425
  • García Vidal, Jorge|||0000-0001-5969-1182
The use of low-cost sensors (LCS) in Internet of Things (IoT) networks offers a promising way to improve air quality monitoring. However, there is a major concern regarding their long-term accuracy due to continuous data drift, which requires frequent data recalibration. To address this, we present window-based uncertainty drift detection and recalibration (W-UDDR), a unified system that automates the entire process. Our system uses a Bayesian approach with Gaussian processes (GP) to automatically and accurately detect when sensor output needs to be corrected. To achieve this, the uncertainty of the estimates is quantified using predictive confidence intervals alongside a window system that detects the need for recalibration in real time. We validate our approach using an air quality real-world sensor deployment, systematically assessing key performance metrics such as the frequency of recalibration and the required sample size. Our results show that W-UDDR successfully triggers automatic events when it detects drifts, achieving significant long-term accuracy improvements ranging from 40% to 95%. Hence, this tool provides an automated real-time mechanism to facilitate long-term sensor deployment maintenance., This work is supported by the national Spanish Grant PID2022-138155OB-I00 funded by MCIN/AEI/ 10.13039/501100011033 and by “ERDF A way of making Europe”, by regional projects 2021 SGR-01059 and 2023 CLIMA 0097, and with the supportof Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Estimating black carbon levels using machine learning models in high-concentration regions

  • Gupta, Pratima
  • Ferrer Cid, Pau|||0000-0003-2112-8516
  • Barceló Ordinas, José María|||0000-0002-9738-2425
  • García Vidal, Jorge|||0000-0001-5969-1182
  • Soni, Vijay Kumar
  • Pöhlker, Mira L.
  • Ahlawat, Ajit
  • Viana Rodríguez, Mar
Black carbon (BC) is emitted into the atmosphere during combustion processes, often in conjunction with emissions such as nitrogen oxides (NOx) and ozone (O3), which are also by-products of combustion. In highly polluted regions, combustion processes are one of the main sources of aerosols and particulate matter (PM) concentrations, which affect the radiative budget. Despite the high relevance of this air pollution metric, BC monitoring is quite expensive in terms of instrumentation and of maintenance and servicing. With the aim to provide tools to estimate BC while minimising instrumentation costs, we use machine learning approaches to estimate BC from air pollution and meteorological parameters (NOx, O3, PM2.5, relative humidity (RH), and solar radiation (SR)) from currently available networks. We assess the effectiveness of various machine learning models, such as random forest (RF), support vector regression (SVR), and multilayer perceptron (MLP) artificial neural network, for predicting black carbon (BC) mass concentrations in areas with high BC levels such as Northern Indian cities (Delhi and Agra), across different seasons. The results demonstrate comparable effectiveness among the models, with the multilayer perceptron (MLP) showing the most promising results. In addition, the comparability between estimated and monitored BC concentrations was high. In Delhi, the MLP shows high correlations between measured and modelled concentrations during winter (R2: 0.85) and post-monsoon (R2: 0.83) seasons, and notable metrics in the pre-monsoon (R2: 0.72). The results from Agra are consistent with those from Delhi, highlighting the consistency of the neural network's performance. These results highlight the usefulness of machine learning, particularly MLP, as a valuable tool for predicting BC concentrations. This approach provides critical new opportunities for urban air quality management and mitigation strategies and may be especially valuable for megacities in medium- and low-income regions., This work was partly funded by project FIRE-RES (https://fire-res. eu/)- Horizon 2020- Research and Innovation Framework Programme under grant agreement No. 101037419; project AGAUR (project 2017 SGR41 and project 2021 SGR-01059); Grant PID2022-138155OB-I00 funded by MCIN/AEI/10.13039/501100011033 and by “ERDF A way of making Europe”., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Leveraging spatiotemporal correlations with recurrent autoencoders for sensor anomaly detection

  • Allka, Xhensilda|||0000-0003-2187-9985
  • Ferrer Cid, Pau|||0000-0003-2112-8516
  • Barceló Ordinas, José María|||0000-0002-9738-2425
  • García Vidal, Jorge|||0000-0001-5969-1182
The introduction of high- and low-cost Internet of Things (IoT) sensors in air quality monitoring networks, in addition to providing a cost-effective solution for monitoring pollutant levels, also brings with it the challenge of ensuring data reliability. These sensors can present anomalies in the data and identifying them is a challenging task. In this article, we propose a spatiotemporal correlation recurrent autoencoder anomaly detection (STC-RAAD) architecture that unlike the other existing architectures, in addition to the temporal correlation present in the sensor data, also involves information from the neighboring sensors in the monitoring network for better reconstruction. The performance of STC-RAAD is compared with the other methods on two different real data sets, for two categories of anomalies, outperforming the other existing approaches with an average increase of 38% for the detection rate and 36% for the precision., This work was supported in part by MCIN/AEI/ 10.13039/501100011033 under Grant PID2022-138155OB-I00; in part by the “ERDF A Way of Making Europe” under Grant CDTI MIG20221061; and in part by the Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu under Project 2021 SGR0 1059., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

A review of graph-powered data quality applications for IoT monitoring sensor networks

  • Ferrer Cid, Pau|||0000-0003-2112-8516
  • Barceló Ordinas, José María|||0000-0002-9738-2425
  • García Vidal, Jorge|||0000-0001-5969-1182
The development of Internet of Things (IoT) technologies has led to the widespread adoption of monitoring networks for a wide variety of applications, such as smart cities, environmental monitoring, and precision agriculture. A major research focus in recent years has been the development of graph-based techniques to improve the quality of data from sensor networks, a key aspect of the use of sensed data in decision-making processes, digital twins, and other applications. Emphasis has been placed on the development of machine learning (ML) and signal processing techniques over graphs, taking advantage of the benefits provided by the use of structured data through a graph topology. Many technologies such as graph signal processing (GSP) or the successful graph neural networks (GNNs) have been used for data quality enhancement tasks. This survey focuses on graph-based models for data quality control in monitoring sensor networks. In addition, it introduces the technical details that are commonly used to provide powerful graph-based solutions for data quality tasks in sensor networks, such as missing value imputation, outlier detection, or virtual sensing. To conclude, different challenges and emerging trends have been identified, e.g., graph-based models for digital twins or model transferability and generalization., This work is supported by Grant PID2022-138155OB-I00 funded by MCIN/AEI/10.13039/501100011033, Spain and by ‘‘ERDF A way of making Europe’’, CDTI MIG-20221061, regional project, Spain 2021SGR01059, and by AGAUR 2023 CLIMA 0097., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

ESOD: an Edge Streaming Data Outlier Detection framework for IoT platforms

  • Allka, Xhensilda|||0000-0003-2187-9985
  • Ferrer Cid, Pau|||0000-0003-2112-8516
  • Barceló Ordinas, José María|||0000-0002-9738-2425
  • García Vidal, Jorge|||0000-0001-5969-1182
  • Avila Torrado, Antonio
Intelligent computing at the IoT edge allows tasks that are normally performed in the cloud to be performed at the IoT node, enabling low-latency applications and more efficient control and management of services. Among the tasks that can be performed on the IoT edge is improving data quality, such as outlier detection (OD). This task is challenging because IoT nodes have fewer computational and storage resources than the cloud. In this article, we propose an OD framework adapted to IoT edge nodes that is lightweight, consumes few resources, adapts to changes in signal trends, and has the ability to provide real-time responses. The framework presented is based on the use of two windows. A sliding window for the current data, which captures the short-term changes in the signal, and a window that stores a summary of historical values, which captures whether the values are within the long-term range of the signal. We show how models using two windows reduce the number of false positives compared to models using only one window. A version for near real-time applications is also proposed, which improves detection by identifying trend changes in the signal at the cost of delaying the decision by a few samples., This work is supported by Grant PID2022-138155OB-I00 funded by MCIN/AEI/ 10.13039/501100011033 and by “ERDF A way of making Europe”, CDTI MIG-20221061, by regional projects 2021CDTI and 2023 CLIMA 0097, and with the support of Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Pattern-based attention recurrent autoencoder for anomaly detection in air quality sensor networks

  • Allka, Xhensilda|||0000-0003-2187-9985
  • Ferrer Cid, Pau|||0000-0003-2112-8516
  • Barceló Ordinas, José María|||0000-0002-9738-2425
  • García Vidal, Jorge|||0000-0001-5969-1182
Sensor networks play an essential role in today's air quality monitoring platforms. Nevertheless, sensors often malfunction, leading to data anomalies. In this paper, an unsupervised pattern-based attention recurrent autoencoder for anomaly detection (PARAAD) is proposed to detect and locate anomalies in a network of air quality sensors. The novelty of the proposal lies in the use of temporal patterns, i.e., blocks of data, instead of point values. By looking at temporal patterns and through an attention mechanism, the architecture captures data dependencies in the feature space and latent space, enhancing the model's ability to focus on the most relevant parts. Its performance is evaluated with two categories of anomalies, bias fault and drift anomalies, and compared with baseline models such as a feed-forward autoencoder and a transformer architecture, as well as with models not based on temporal patterns. The results show that PARAAD achieves anomalous sensor detection and localization rates higher than 80%, outperforming existing baseline models in air quality sensor networks for both bias and drift faults., This work was supported in part by MCIN AEI/10.13039/501100011033 under Grant PID2022-138155OB-I00, in part by “ERDF A way of making Europe”, CDTI MIG-20221061, through regional project 2021SGR-01059, and in part by Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Regional aerosol hygroscopicity influences radiative forcing globally

  • Deshmukh, Shravan|||0000-0001-5110-1969
  • Ferrer Cid, Pau|||0000-0003-2112-8516
  • Romshoo, Baseerat|||0009-0001-9330-5385
  • Poulain, Laurent|||0000-0002-9128-7881
  • Barceló Ordinas, José María|||0000-0002-9738-2425
  • García Vidal, Jorge|||0000-0001-5969-1182
  • Christodoulou, Aliki|||0000-0001-7748-4910
  • Bezantakos, Spyros
  • Denjean, Cyrielle
  • D’Anna, Barbara
Aerosol hygroscopicity is a critical parameter for predicting radiative forcing and climate sensitivity, particularly under sub-saturated regimes where it drives complex aerosol–water interactions. Here, we show that externally mixed aerosols exert a stronger influence on direct radiative forcing than is currently represented in models. Incorporating our findings into radiative forcing calculations indicates a stronger aerosol cooling effect, especially at suburban sites, highlighting the importance of representing regional differences in mixing state. The conventional bulk-chemistry approach, which assumes volume-based mixing with limited spatial variability, exhibits low predictive performance for aerosol hygroscopicity (R² ˜ 0.61) at urban and suburban sites. Using an interpretable machine learning framework trained on geographically diverse, region-specific datasets can capture this variability with higher accuracy (R² ˜ 0.97), identifying key chemical compositional and mixing-state drivers., PFC, JMB, and JGV work was funded by (grant nos. PID2022-138155OBI00 and MCIN/AEI/10.13039/501100011033 by ERDF A way of making Europe), Spain, and by (grant nos. 2021SGR-01059 and AGAUR 2023 CLIMA 0097) AGAUR regional projects, Spain. PK acknowledges support from the RECLAIM Network Plus (EP/W034034/1) project. PF and BD acknowledge the AEROCLO-sA project supported by the French National Research Agency (ANR) under (grant no. ANR-15-CE01-0014-01), the French national program LEFE/INSU, the Programme national de Télédetection Spatiale (PNTS, grant no. PNTS-2016-14), the French National Agency for Space Studies (CNES), and the South African National Research Foundation (NRF) under (grant UID 105958. PEGASUS receivesfunding as a national facility of the Institut National des Sciences de l’Univers du Centre national de la recherche scientifique (CNRS INSU) within the research infrastructure ACTRIS-FR. MDP acknowledges support from the U.S. Department of Energy, Office of Science, Biological and Environmental Research grant no. DE-SC0021074. A.C. and S.Bwork was supported by the European Union's Horizon 2020 research and innovation programme under grant agreement no.856612 (EMME-CARE) and by the STDF-AUF-IRD launched in 2016 (POLCAIR ‘Air Pollution in Cairo: sources and impact). The Deutsche Forschungsgemeinschaft (DFG) has supported this research (grant no. WE 2757/4-1). SD, MP, LP, BR, SH, and BW are funded and supported by the DFG. Managed by Leibniz Institute for Tropospheric Research (TROPOS)., Peer Reviewed, Article signat per 22 autors/es: Shravan Deshmukh 1, Pau Ferrer-Cid 2, Baseerat Romshoo 1, Laurent Poulain 1, Jose M. Barcelo-Ordinas 2, Jorge Garcia-Vidal 2, Aliki Christodoulou 3, 19, Spyros Bezantakos 3, Cyrielle Denjean 4, Barbara D’Anna 5, Paola Formenti 6, Subrata Mukherjee 7, Gazala Habib 8, Prashant Kumar 9, 10, Shan Huang 11, Zhijun Wu 12, Birgit Wehner 1, Silvia Henning 1, Mar Viana 13, 14, Markus D. Petters 15, 16, Ajit Ahlawat 1, 17 & Mira Pöhlker 1, 18 / 1 Leibniz Institute for Tropospheric Research, e.V. (TROPOS), Permoserstrasse 15, Leipzig, Germany. 2 Department of Computer Architecture, Universitat Politècnica de Catalunya (UPC), Barcelona, Spain. 3 Climate and Atmosphere Research Center (CARE-C), the Cyprus Institute, Nicosia, Cyprus. 4 Météo-France, CNRS, , University of Toulouse, CNRM, Toulouse, France. 5 Aix-Marseille Université, CNRS, LCE, Marseille, France. 6 Université Paris Cité and Univ Paris Est Créteil, CNRS, LISA, Paris, France. 7 Indian Institute of Tropical Meteorology, Pune, India. 8 Indian Institute for Technology New Delhi (IIT-D), New Delhi, India. 9 Global Centre for Clean Air Research (GCARE), School of Engineering, Civil and Environmental Engineering, Faculty of Engineering and Physical Sciences, University of Surrey, Guildford, UK. 10 Institute for Sustainability, University of Surrey, Guildford, Surrey, UK. 11 College of Environment and Climate, Jinan University Guangzhou, China. 12 State Key Joint Laboratory of Environmental Simulation and Pollution Control, College of Environmental Sciences and Engineering, Peking University Beijing, China. 13 Spanish Ministry for Ecological Transition, Madrid, Spain. 14 Institute of Environmental Assessment and Water Research, Spanish Research Council (IDAEA-CSIC), Barcelona, Spain. 15 Department of Chemical and Environmental Engineering, University of California Riverside, Riverside, CA, USA. 16 Center for Environmental Research and Technology (CE-CERT), University of California Riverside, Riverside, CA, USA. 17 Department of Geoscience and Remote Sensing, Delft University of Technology (TU Delft),Delft, The Netherlands. 18 Faculty of Physics and Earth Sciences, Leipzig Institute for Meteorology, Leipzig University, Leipzig, Germany. 19 Present address: PSI Center for Energy and Environmental Sciences, Paul Scherrer Institute, Villigen, Switzerland.
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Virtual sensor-based proxy for black carbon estimation in IoT platforms

  • Ferrer Cid, Pau|||0000-0003-2112-8516
  • Paredes Ahumada, Juan Antonio
  • Barceló Ordinas, José María|||0000-0002-9738-2425
  • García Vidal, Jorge|||0000-0001-5969-1182
Black carbon (BC) has been under the spotlight of research during the last few years due to its non-regulation, its role in air pollution, and its hazardous effects. Given the high cost of the instrumentation needed to measure BC concentrations, data-driven techniques have been adopted to implement proxies that provide BC measurements from other sensor measurements. These sensors may present data quality issues due to maintenance actions, loss of data, or relocation, among others. In this paper, we propose a data-driven proxy model for BC estimation that is powered by a hybrid sensor array, including physical and virtual sensors created from machine learning techniques and governmental air quality monitoring networks. Therefore, the proposed method provides an accurate alternative to traditional data-driven BC proxies in scenarios where some physical sensors are unavailable. The results show how a BC proxy can be partially implemented using virtual sensors, obtaining only an increase in the estimation error of around 4%, allowing the estimation of BC levels even when some physical sensors are absent., This work is supported by Grant PID2022-138155OB-I00 funded by MCIN/ AEI/10.13039/501100011033 and by ‘‘ERDF A way of making Europe’’, CDTI MIG-20221061, by AGAUR regional project 2021SGR-01059, and with the support of Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Convex relaxation method for sensor placement in multiclass monitoring networks

  • Paredes Ahumada, Juan Antonio
  • Ferrer Cid, Pau|||0000-0003-2112-8516
  • Barceló Ordinas, José María|||0000-0002-9738-2425
  • García Vidal, Jorge|||0000-0001-5969-1182
The design of a sensor network plays an important role in monitoring a physical field. This task becomes more difficult as the network consists of different classes of sensors with their corresponding uncertainties. In this article, the rankMax sensor placement method is proposed for scenarios where high-quality instruments are used along with low-cost sensors (LCSs) in a multiclass monitoring network. The novelty of the method relies on exploiting a closed form of the inverse of the Fisher information matrix (FIM) in terms of rank-1 matrices that allow expressing a convex relaxation of the sensor placement problem. The method performance is evaluated on an air pollution dataset for small and large monitoring networks and compared with a straightforward extension of the convex relaxation method proposed by Joshi and Boyd. The results show that the rankMax method outperforms the multiclass Joshi-Boyd (MCJB) method about 91% of the time, being particularly useful when there are not many reference instruments in the network., This work was supported in part by MCIN/AEI/10.13039/501100011033 under Grant PID2022-138155OB-I00; in part by the European Regional Development Fund (ERDF) A way of making Europe, Centro para el Desarrollo Tecnológico y la Innovación (CDTI) MIG-20221061, under Project 2021SGR-01059; and in part by Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Contribución de congreso (ConferenceOutput)

Edge-based missing value imputation for IoT platforms: Towards artificial intelligence of things

  • Cioca, Alexandru
  • Ferrer Cid, Pau|||0000-0003-2112-8516
  • Barceló Ordinas, José María|||0000-0002-9738-2425
  • García Vidal, Jorge|||0000-0001-5969-1182
Recently, the use of Internet of Things (IoT) technologies has been widely adopted, enabling large-scale monitoring and data acquisition. Together with the so-called artificial intelligence of things (AIoT), data-driven models are to be executed close to the data source. Nevertheless, one potential problem is the quality of the data used to feed these models. Data loss is a common issue in these systems and can hinder the use of monitoring data and the application of artificial intelligence (AI) models. This paper evaluates a set of missing value imputation techniques, emphasizing their application in an edge setting. Specifically, offline and sliding window variants are evaluated to meet data availability and computational complexity criteria. The benchmarking has been performed using the data of a real IoT air quality monitoring platform. The results demonstrate that the windowed variants can approach the offline performance using reduced sliding windows of 25 to 100 samples per sensor., This work is supported by the National Spanish funding PID2022-138155OB-I00, 2023 CLIMA 0097, by regional project 2021SGR-01059, and with the support of Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

A data-driven framework for air quality sensor networks

  • Ferrer Cid, Pau
  • Paredes Ahumada, Juan Antonio
  • Allka, Xhensilda
  • Guerrero Zapata, Manel
  • Barceló Ordinas, José María
  • García Vidal, Jorge
In this article, we present our research vision of a framework for obtaining quality data in air quality monitoring networks using low-cost sensors (LCSs). The use of LCS networks is gaining increasing acceptance in many IoT air quality applications. However, data quality and reliability issues are a major barrier to widespread adoption, which means that the pre-processing tasks that are critical to achieving the required levels of data quality are crucial aspects of LCS network designs. The proposed framework takes advantage of a layered architecture, which has also proven useful in other fields, and from which we show the challenges and state-of-the-art techniques for obtaining quality data. In addition, we show its usefulness in application cases, including a real case with data measured by a LCS deployment measuring O 3 in the area of Barcelona, Spain., This work is supported by Grant PID2022-138155OB-I00 funded by MCIN/AEI/ 10.13039/501100011033 and by “ERDF A way of making Europe”, CDTI MIG20221061, by regional project 2021SGR-01059, and with the support of Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Virtual sensor-based proxy for black carbon estimation in IoT platforms

  • Ferrer Cid, Pau
  • Paredes Ahumada, Juan Antonio
  • Barceló Ordinas, José María
  • García Vidal, Jorge
Black carbon (BC) has been under the spotlight of research during the last few years due to its non-regulation, its role in air pollution, and its hazardous effects. Given the high cost of the instrumentation needed to measure BC concentrations, data-driven techniques have been adopted to implement proxies that provide BC measurements from other sensor measurements. These sensors may present data quality issues due to maintenance actions, loss of data, or relocation, among others. In this paper, we propose a data-driven proxy model for BC estimation that is powered by a hybrid sensor array, including physical and virtual sensors created from machine learning techniques and governmental air quality monitoring networks. Therefore, the proposed method provides an accurate alternative to traditional data-driven BC proxies in scenarios where some physical sensors are unavailable. The results show how a BC proxy can be partially implemented using virtual sensors, obtaining only an increase in the estimation error of around 4%, allowing the estimation of BC levels even when some physical sensors are absent., This work is supported by Grant PID2022-138155OB-I00 funded by MCIN/ AEI/10.13039/501100011033 and by ‘‘ERDF A way of making Europe’’, CDTI MIG-20221061, by AGAUR regional project 2021SGR-01059, and with the support of Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

A review of graph-powered data quality applications for IoT monitoring sensor networks

  • Ferrer Cid, Pau
  • Barceló Ordinas, José María
  • García Vidal, Jorge
The development of Internet of Things (IoT) technologies has led to the widespread adoption of monitoring networks for a wide variety of applications, such as smart cities, environmental monitoring, and precision agriculture. A major research focus in recent years has been the development of graph-based techniques to improve the quality of data from sensor networks, a key aspect of the use of sensed data in decision-making processes, digital twins, and other applications. Emphasis has been placed on the development of machine learning (ML) and signal processing techniques over graphs, taking advantage of the benefits provided by the use of structured data through a graph topology. Many technologies such as graph signal processing (GSP) or the successful graph neural networks (GNNs) have been used for data quality enhancement tasks. This survey focuses on graph-based models for data quality control in monitoring sensor networks. In addition, it introduces the technical details that are commonly used to provide powerful graph-based solutions for data quality tasks in sensor networks, such as missing value imputation, outlier detection, or virtual sensing. To conclude, different challenges and emerging trends have been identified, e.g., graph-based models for digital twins or model transferability and generalization., This work is supported by Grant PID2022-138155OB-I00 funded by MCIN/AEI/10.13039/501100011033, Spain and by ‘‘ERDF A way of making Europe’’, CDTI MIG-20221061, regional project, Spain 2021SGR01059, and by AGAUR 2023 CLIMA 0097., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Convex relaxation method for sensor placement in multiclass monitoring networks

  • Paredes Ahumada, Juan Antonio
  • Ferrer Cid, Pau
  • Barceló Ordinas, José María
  • García Vidal, Jorge
The design of a sensor network plays an important role in monitoring a physical field. This task becomes more difficult as the network consists of different classes of sensors with their corresponding uncertainties. In this article, the rankMax sensor placement method is proposed for scenarios where high-quality instruments are used along with low-cost sensors (LCSs) in a multiclass monitoring network. The novelty of the method relies on exploiting a closed form of the inverse of the Fisher information matrix (FIM) in terms of rank-1 matrices that allow expressing a convex relaxation of the sensor placement problem. The method performance is evaluated on an air pollution dataset for small and large monitoring networks and compared with a straightforward extension of the convex relaxation method proposed by Joshi and Boyd. The results show that the rankMax method outperforms the multiclass Joshi-Boyd (MCJB) method about 91% of the time, being particularly useful when there are not many reference instruments in the network., This work was supported in part by MCIN/AEI/10.13039/501100011033 under Grant PID2022-138155OB-I00; in part by the European Regional Development Fund (ERDF) A way of making Europe, Centro para el Desarrollo Tecnológico y la Innovación (CDTI) MIG-20221061, under Project 2021SGR-01059; and in part by Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

A neural network-based system for automated refrigeration alarm management in food retail

  • Ferrer Cid, Pau
  • Albets Chico, Xavier
  • Barceló Ordinas, José María
  • García Vidal, Jorge
The increasing use of refrigeration monitoring platforms in the food retail sector is crucial for enhancing operational efficiency and mitigating food waste. These systems are essential for ensuring food safety and quality, as they alert personnel to abnormal temperature readings, which are often caused by equipment malfunctions or operational issues. However, the resulting high volume of alarms poses a significant challenge, requiring tailored management strategies and continuous supervision by expert staff. This article presents an intelligent alarm labeling system based on neural networks that evaluates various alarm dimensions. The proposed system effectively detects recurrent patterns and determines a device’s status based on its temperature measurements. The system was validated using a massive monitoring dataset from different supermarkets in Catalonia, Spain. The results show that the system can detect recurrent alarms with 98% accuracy and correctly identify device states with 93% accuracy, enabling more effective temperature alarm management., This work has been financed by the primary project AKO 4.4: Nueva plataforma de soluciones digitales de control, predicción y soporte a la decisión, para mejorar la eficiencia energética, garantizar la seguridad y evitar las mermas de alimentos en los activos de refrigeración de la cadena del frío [grant number PAG-020100-2023-104], under the Project grant PAG-010000-2023-9: modernización de los sistemas de trazabilidad, seguridad alimentaria y gestión de la calidad de la cadena de valor del sector porcino, vacuno y avícola. This Project grant has been awarded under the Proyecto Estratégico para la Recuperación y Transformación Económica Agroalimentario (PERTE Agroalimentario) by the Ministerio de Industria, Comercio y Turismo of Spain. This work is also supported by the National Spanish funding MCIN/AEI/ 10.13039/501100011033 and by “ERDF A way of making Europe” [grant number PID2022-138155OB-I00], and by regional project [grant number 2021SGR-01059]., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Edge-based missing value imputation for IoT platforms: Towards artificial intelligence of things

  • Cioca, Alexandru
  • Ferrer Cid, Pau
  • Barceló Ordinas, José María
  • García Vidal, Jorge
Recently, the use of Internet of Things (IoT) technologies has been widely adopted, enabling large-scale monitoring and data acquisition. Together with the so-called artificial intelligence of things (AIoT), data-driven models are to be executed close to the data source. Nevertheless, one potential problem is the quality of the data used to feed these models. Data loss is a common issue in these systems and can hinder the use of monitoring data and the application of artificial intelligence (AI) models. This paper evaluates a set of missing value imputation techniques, emphasizing their application in an edge setting. Specifically, offline and sliding window variants are evaluated to meet data availability and computational complexity criteria. The benchmarking has been performed using the data of a real IoT air quality monitoring platform. The results demonstrate that the windowed variants can approach the offline performance using reduced sliding windows of 25 to 100 samples per sensor., This work is supported by the National Spanish funding PID2022-138155OB-I00, 2023 CLIMA 0097, by regional project 2021SGR-01059, and with the support of Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Evaluating missing value imputation strategies to enhance IoT data availability in the edge

  • Ferrer Cid, Pau
  • Barceló Ordinas, José María
  • García Vidal, Jorge
  • Avila Torrado, Antonio
The rapid implementation of Internet of Things (IoT) technologies in industrial and air quality monitoring applications has resulted in large amounts of data being acquired. In addition, the integration of the artificial intelligence of things (AIoT) enables decision-making at edge nodes, involving the execution of data-driven models close to the data source. Data loss has become a limiting factor in AI-based applications. One way to address missing data is to leverage data from co-existing sensors, which are potentially correlated, to impute missing values. This article evaluates a set of missing value imputation (MVI) techniques in an edge node environment, where constraints on data availability and computational complexity must be met. Specifically, models such as multiple imputation by chained equations (MICE), k-nearest neighbors (KNN), and models using variational autoencoders (VAE) are evaluated. A comprehensive evaluation is presented covering different scenarios of missing data in terms of the percentage of missing values, bursts of missing values, and the size of the data windows stored on edge nodes. This evaluation uses real sensor data from an air quality monitoring network and an industrial sensor network. The results show that the VAE is able to obtain good imputation performances (R^2 greater than 0.90) while providing good uncertainty quantification (UQ) estimates with around 90%–95% of samples falling within the estimated confidence intervals. Moreover, a transfer learning-based VAE has been shown to adapt to the non-stationary nature of IoT sensing signals, and all methods have proven to be efficient in terms of execution time for the edge setting., This work was supported by Grant PID2022-138155OB-I00 funded by MCIN/AEI/10.13039/501100011033 and by ‘‘ERDF A way of making Europe’’, CDTI MIG-20221061, by regional project 2021 SGR 01059, and with the support of Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Leveraging spatiotemporal correlations with recurrent autoencoders for sensor anomaly detection

  • Allka, Xhensilda
  • Ferrer Cid, Pau
  • Barceló Ordinas, José María
  • García Vidal, Jorge
The introduction of high- and low-cost Internet of Things (IoT) sensors in air quality monitoring networks, in addition to providing a cost-effective solution for monitoring pollutant levels, also brings with it the challenge of ensuring data reliability. These sensors can present anomalies in the data and identifying them is a challenging task. In this article, we propose a spatiotemporal correlation recurrent autoencoder anomaly detection (STC-RAAD) architecture that unlike the other existing architectures, in addition to the temporal correlation present in the sensor data, also involves information from the neighboring sensors in the monitoring network for better reconstruction. The performance of STC-RAAD is compared with the other methods on two different real data sets, for two categories of anomalies, outperforming the other existing approaches with an average increase of 38% for the detection rate and 36% for the precision., This work was supported in part by MCIN/AEI/ 10.13039/501100011033 under Grant PID2022-138155OB-I00; in part by the “ERDF A Way of Making Europe” under Grant CDTI MIG20221061; and in part by the Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu under Project 2021 SGR0 1059., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Estimating black carbon levels using machine learning models in high-concentration regions

  • Gupta, Pratima
  • Ferrer Cid, Pau
  • Barceló Ordinas, José María
  • García Vidal, Jorge
  • Soni, Vijay Kumar
  • Pöhlker, Mira L.
  • Ahlawat, Ajit
  • Viana Rodríguez, Mar
Black carbon (BC) is emitted into the atmosphere during combustion processes, often in conjunction with emissions such as nitrogen oxides (NOx) and ozone (O3), which are also by-products of combustion. In highly polluted regions, combustion processes are one of the main sources of aerosols and particulate matter (PM) concentrations, which affect the radiative budget. Despite the high relevance of this air pollution metric, BC monitoring is quite expensive in terms of instrumentation and of maintenance and servicing. With the aim to provide tools to estimate BC while minimising instrumentation costs, we use machine learning approaches to estimate BC from air pollution and meteorological parameters (NOx, O3, PM2.5, relative humidity (RH), and solar radiation (SR)) from currently available networks. We assess the effectiveness of various machine learning models, such as random forest (RF), support vector regression (SVR), and multilayer perceptron (MLP) artificial neural network, for predicting black carbon (BC) mass concentrations in areas with high BC levels such as Northern Indian cities (Delhi and Agra), across different seasons. The results demonstrate comparable effectiveness among the models, with the multilayer perceptron (MLP) showing the most promising results. In addition, the comparability between estimated and monitored BC concentrations was high. In Delhi, the MLP shows high correlations between measured and modelled concentrations during winter (R2: 0.85) and post-monsoon (R2: 0.83) seasons, and notable metrics in the pre-monsoon (R2: 0.72). The results from Agra are consistent with those from Delhi, highlighting the consistency of the neural network's performance. These results highlight the usefulness of machine learning, particularly MLP, as a valuable tool for predicting BC concentrations. This approach provides critical new opportunities for urban air quality management and mitigation strategies and may be especially valuable for megacities in medium- and low-income regions., This work was partly funded by project FIRE-RES (https://fire-res. eu/)- Horizon 2020- Research and Innovation Framework Programme under grant agreement No. 101037419; project AGAUR (project 2017 SGR41 and project 2021 SGR-01059); Grant PID2022-138155OB-I00 funded by MCIN/AEI/10.13039/501100011033 and by “ERDF A way of making Europe”., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

W-UDDR: A unified framework for automated drift detection in IoT-based air quality monitoring systems

  • Allka, Xhensilda
  • Ferrer Cid, Pau
  • Barceló Ordinas, José María
  • García Vidal, Jorge
The use of low-cost sensors (LCS) in Internet of Things (IoT) networks offers a promising way to improve air quality monitoring. However, there is a major concern regarding their long-term accuracy due to continuous data drift, which requires frequent data recalibration. To address this, we present window-based uncertainty drift detection and recalibration (W-UDDR), a unified system that automates the entire process. Our system uses a Bayesian approach with Gaussian processes (GP) to automatically and accurately detect when sensor output needs to be corrected. To achieve this, the uncertainty of the estimates is quantified using predictive confidence intervals alongside a window system that detects the need for recalibration in real time. We validate our approach using an air quality real-world sensor deployment, systematically assessing key performance metrics such as the frequency of recalibration and the required sample size. Our results show that W-UDDR successfully triggers automatic events when it detects drifts, achieving significant long-term accuracy improvements ranging from 40% to 95%. Hence, this tool provides an automated real-time mechanism to facilitate long-term sensor deployment maintenance., This work is supported by the national Spanish Grant PID2022-138155OB-I00 funded by MCIN/AEI/ 10.13039/501100011033 and by “ERDF A way of making Europe”, by regional projects 2021 SGR-01059 and 2023 CLIMA 0097, and with the supportof Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Pattern-based attention recurrent autoencoder for anomaly detection in air quality sensor networks

  • Allka, Xhensilda
  • Ferrer Cid, Pau
  • Barceló Ordinas, José María
  • García Vidal, Jorge
Sensor networks play an essential role in today's air quality monitoring platforms. Nevertheless, sensors often malfunction, leading to data anomalies. In this paper, an unsupervised pattern-based attention recurrent autoencoder for anomaly detection (PARAAD) is proposed to detect and locate anomalies in a network of air quality sensors. The novelty of the proposal lies in the use of temporal patterns, i.e., blocks of data, instead of point values. By looking at temporal patterns and through an attention mechanism, the architecture captures data dependencies in the feature space and latent space, enhancing the model's ability to focus on the most relevant parts. Its performance is evaluated with two categories of anomalies, bias fault and drift anomalies, and compared with baseline models such as a feed-forward autoencoder and a transformer architecture, as well as with models not based on temporal patterns. The results show that PARAAD achieves anomalous sensor detection and localization rates higher than 80%, outperforming existing baseline models in air quality sensor networks for both bias and drift faults., This work was supported in part by MCIN AEI/10.13039/501100011033 under Grant PID2022-138155OB-I00, in part by “ERDF A way of making Europe”, CDTI MIG-20221061, through regional project 2021SGR-01059, and in part by Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu., Peer Reviewed, Postprint (author's final draft)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

ESOD: an Edge Streaming Data Outlier Detection framework for IoT platforms

  • Allka, Xhensilda
  • Ferrer Cid, Pau
  • Barceló Ordinas, José María
  • García Vidal, Jorge
  • Avila Torrado, Antonio
Intelligent computing at the IoT edge allows tasks that are normally performed in the cloud to be performed at the IoT node, enabling low-latency applications and more efficient control and management of services. Among the tasks that can be performed on the IoT edge is improving data quality, such as outlier detection (OD). This task is challenging because IoT nodes have fewer computational and storage resources than the cloud. In this article, we propose an OD framework adapted to IoT edge nodes that is lightweight, consumes few resources, adapts to changes in signal trends, and has the ability to provide real-time responses. The framework presented is based on the use of two windows. A sliding window for the current data, which captures the short-term changes in the signal, and a window that stores a summary of historical values, which captures whether the values are within the long-term range of the signal. We show how models using two windows reduce the number of false positives compared to models using only one window. A version for near real-time applications is also proposed, which improves detection by identifying trend changes in the signal at the cost of delaying the decision by a few samples., This work is supported by Grant PID2022-138155OB-I00 funded by MCIN/AEI/ 10.13039/501100011033 and by “ERDF A way of making Europe”, CDTI MIG-20221061, by regional projects 2021CDTI and 2023 CLIMA 0097, and with the support of Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu., Peer Reviewed, Postprint (author's final draft)