Pasar al contenido principal

Robust and Trustworthy AI Systems: Addressing Vulnerabilities and Points of Failure of AI Systems Deployed in High-Risk Applications

RYC2023-043607-I

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal para Desarrollar, Atraer y Retener Talento
Subprograma Subprograma Estatal de Incorporación
Convocatoria Ramón y Cajal
Año convocatoria 2023
Plan Estatal 2021-2023 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario AGENCIA ESTATAL CONSEJO SUPERIOR DE INVESTIGACIONES CIENTIFICAS (CSIC)

Resultados relacionados

Resultados totales (Incluyendo duplicados): 4
Encontrada(s) 1 página(s)

Repositori Obert UdL
Publicación Artículo científico (JournalArticle)

Certified Adversarial Robustness of End-to-End Malware Detectors via (De)Randomized Smoothing

  • Gibert, Daniel
  • Demetrio, Luca
  • Zizzo, Giulio
  • Le, Quan
  • Planes Cid, Jordi
  • Biggio, Battista
End-to-end machine learning malware detectors are vulnerable to adversarial EXEmples, carefully-crafted malicious programs that evade detection through minimal perturbations. Such attacks typically operate by either replacing unused content (patch attacks), or injecting new patterns (content-injection attacks). To counter these attacks, recent research has focused on certification methods for end-to-end models that aim to prove robustness guarantees within bounded perturbation sizes. However, existing approaches (i) are not robust against content-injection manipulations and (ii) provide only probabilistic guarantees for perturbations that are negligible relative to the overall program size. Hence, in this article we address these limitations through a novel deterministic certification schema based on (de)randomized smoothing. Our defense splits each executable into non-overlapping chunks and classifies them independently. The final decision is obtained via majority voting across all chunks, ensuring that localized modifications, such as injected or patched code, influence only a limited subset of chunks and have minimal impact in the overall classification. This design guarantees that each chunk either contains or does not contain an adversarial perturbation, enabling us to (i) handle manipulations occurring at arbitrary locations within the program, and (ii) compute deterministic estimates of the perturbation magnitude required to evade detection. We demonstrate the effectiveness of our certification schema through extensive experimental analysis, comparing our defense against a range of state-of-the-art attacks and defenses. The results show that our approach achieves unmatched robustness across all tested attack scenarios, substantially outperforming competing defenses., D. Gibert was supported by grant RYC2023-043607-I funded by MICIU/AEI/10.13039/501100011033 and FSE+. This project has received funding from the European Union’s Horizon 2020 Research and Innovation programme under the Marie Skodowska-Curie Grant Agreement”(as per Article 29.4 of the grant agreement). We would like to thank Cormac Doherty and UCD’s Centre for Cybersecurity and Cybercrime Investigation for their support. This project has also been partially support by Fondazione di Sardegna under the project “TrustML: Towards ML that Humans Can Trust’’, CUP: F73C22001320007; and European Union’s Horizon Europe research and innovation program under the project ELSA, grant agreement No 101070617; and European Union’s Horizon 2020 research and innovation programme under project TESTABLE, grant agreement No 101019206; and SERICS (PE00000014) under the MUR National Recovery and Resilience Plan funded by the European Union – NextGenerationEU, and by MCIN/AEI/10.13039/501100011033/FEDER, UE under the project PID2022-139835NB-C22. This research project was made possible through the access granted by the Galician Supercomputing Center (CESGA) to its supercomputing infrastructure. The supercomputer FinisTerrae III and its permanent data storage system have been funded by the NextGeneration EU 2021 Recovery, Transformation and Resilience Plan, ICT2021-006904, and also from the Pluriregional Operational Programme of Spain 2014-2020 of the European Regional Development Fund (ERDF), ICTS-2019-02-CESGA-3, and from the State Programme for the Promotion of Scientific and Technical Research of Excellence of the State Plan for Scientific and Technical Research and Innovation 2013-2016 State subprogramme for scientific and technical infrastructures and equipment of ERDF, CESG15-DE-3114
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

Bibliometric analysis of artificial intelligence applied to waste-to-syngas

  • Fornieles, Aleix
  • Etxegarai, Maddi
  • Gibert, Daniel
  • Planes, Jordi
The use of Artificial Intelligence techniques has grown rapidly in the field of waste-to- energy, with gasification processes benefiting from their ability to model and optimize syngas production. However, literature specifically focused on Artificial Intelligence applications in waste-to-syngas systems remains fragmented. This article presents a bibliometric analysis that maps the development of Artificial Intelligence driven research in this domain, focusing mainly on prediction tasks. The analysis is based on data extracted from Scopus and Web of Science, filtered through specific criteria to ensure relevance. Results show a clear rise in publications since 2020, with a dominant contribution from China and a strong focus on hydrogen prediction and neural network models. Despite the progress, this study highlights major gaps in real-time control strategies, data standardization, and industrial-scale implementation. These findings point to emerging opportunities in hybrid modeling, explainable Artificial Intelligence, and underexplored feedstocks should guide future research efforts., Clean Hydrogen Partnership provided financial support for this study through the Project 101137792 – HYEILD. Aleix Fornieles is a fellow of Eurecat’s “Vicente López” PhD grant program. D. Gibert was supported by grant RYC2023-043607-I funded by MICIU/AEI/10.13039/ 501100011033 and FSE+. This work was funded by the project PID2022-139835NB-C22 funded by MCIN/AEI/10.13039/501100011033/FEDER, UE ., Peer reviewed
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

DeepTrust: Multi-Step Classification through Dissimilar Adversarial Representations for Robust Android Malware Detection

  • Pulido-Cortázar, Daniel
  • Gibert, Daniel
  • Manyà, Felip
[Data Statement] The dataset used is provided by the ELSA Robust Android Malware Detection Competition, which is publicly available. The code will be released upon acceptance of the paper to ensure reproducibility., Over the last decade, machine learning has been extensively applied to identify malicious Android applications. However, such approaches remain vulnerable against adversarial examples, i.e., examples that are subtly manipulated to fool a machine learning model into making incorrect predictions. This research presents DeepTrust, a novel metaheuristic that arranges flexible classifiers, like deep neural networks, into an ordered sequence where the final decision is made by a single internal model based on conditions activated in cascade. In the Robust Android Malware Detection competition at the 2025 IEEE Conference SaTML, DeepTrust secured the first place and achieved state-of-the-art results, outperforming the next-best competitor by up to 266% under feature-space evasion attacks. This is accomplished while maintaining the highest detection rate on non-adversarial malware and a false positive rate below 1%. The method's efficacy stems from maximizing the divergence of the learned representations among the internal models. By using classifiers inducing fundamentally dissimilar embeddings of the data, the decision space becomes unpredictable for an attacker. This frustrates the iterative perturbation process inherent to evasion attacks, enhancing system robustness without compromising accuracy on clean examples., D. Pulido-Cortázar was supported by a grant JAE-ICU 2024 funded by IIIA-CSIC (reference 04264). D. Gibert was supported by grant RYC2023-043607-I funded by MICIU/AEI/10.13039/501100011033 and FSE+. F. Manyà was supported by grant PID2022-139835NB-C21 funded by MCIN/AEI/10.13039/501100011033 and by ERDF, EU. The computational resources used in this work were provided by the Drago high-performance computing cluster, operated by the Scientific Computing Area of SGAI-CSIC. This research project was made possible through the access granted by the Galician Supercomputing Center (CESGA) to its supercomputing infrastructure. The supercomputer FinisTerrae III and its permanent data storage system have been funded by the NextGeneration EU 2021 Recovery, Transformation and Resilience Plan, ICT2021-006904, and also from the Pluriregional Operational Programme of Spain 2014–2020 of the European Regional Development Fund (ERDF), ICTS-2019-02-CESGA-3, and from the State Programme for the Promotion of Scientific and Technical Research of Excellence of the State Plan for Scientific and Technical Research and Innovation 2013–2016 State subprogramme for scientific and technical infrastructures and equipment of ERDF, CESG15-DE-3114., Peer reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Certified Adversarial Robustness of End-to-End Malware Detectors via (De)Randomized Smoothing

  • Gibert, Daniel
  • Demetrio, Luca
  • Zizzo, Giulio
  • Le, Quan
  • Planes Cid, Jordi
  • Biggio, Battista
End-to-end machine learning malware detectors are vulnerable to adversarial EXEmples, carefully-crafted malicious programs that evade detection through minimal perturbations. Such attacks typically operate by either replacing unused content (patch attacks), or injecting new patterns (content-injection attacks). To counter these attacks, recent research has focused on certification methods for end-to-end models that aim to prove robustness guarantees within bounded perturbation sizes. However, existing approaches (i) are not robust against content-injection manipulations and (ii) provide only probabilistic guarantees for perturbations that are negligible relative to the overall program size. Hence, in this article we address these limitations through a novel deterministic certification schema based on (de)randomized smoothing. Our defense splits each executable into non-overlapping chunks and classifies them independently. The final decision is obtained via majority voting across all chunks, ensuring that localized modifications, such as injected or patched code, influence only a limited subset of chunks and have minimal impact in the overall classification. This design guarantees that each chunk either contains or does not contain an adversarial perturbation, enabling us to (i) handle manipulations occurring at arbitrary locations within the program, and (ii) compute deterministic estimates of the perturbation magnitude required to evade detection. We demonstrate the effectiveness of our certification schema through extensive experimental analysis, comparing our defense against a range of state-of-the-art attacks and defenses. The results show that our approach achieves unmatched robustness across all tested attack scenarios, substantially outperforming competing defenses., D. Gibert was supported by grant RYC2023-043607-I funded by MICIU/AEI/10.13039/501100011033 and FSE+. This project has received funding from the European Union’s Horizon 2020 Research and Innovation programme under the Marie Skodowska-Curie Grant Agreement”(as per Article 29.4 of the grant agreement). We would like to thank Cormac Doherty and UCD’s Centre for Cybersecurity and Cybercrime Investigation for their support. This project has also been partially support by Fondazione di Sardegna under the project “TrustML: Towards ML that Humans Can Trust’’, CUP: F73C22001320007; and European Union’s Horizon Europe research and innovation program under the project ELSA, grant agreement No 101070617; and European Union’s Horizon 2020 research and innovation programme under project TESTABLE, grant agreement No 101019206; and SERICS (PE00000014) under the MUR National Recovery and Resilience Plan funded by the European Union – NextGenerationEU, and by MCIN/AEI/10.13039/501100011033/FEDER, UE under the project PID2022-139835NB-C22. This research project was made possible through the access granted by the Galician Supercomputing Center (CESGA) to its supercomputing infrastructure. The supercomputer FinisTerrae III and its permanent data storage system have been funded by the NextGeneration EU 2021 Recovery, Transformation and Resilience Plan, ICT2021-006904, and also from the Pluriregional Operational Programme of Spain 2014-2020 of the European Regional Development Fund (ERDF), ICTS-2019-02-CESGA-3, and from the State Programme for the Promotion of Scientific and Technical Research of Excellence of the State Plan for Scientific and Technical Research and Innovation 2013-2016 State subprogramme for scientific and technical infrastructures and equipment of ERDF, CESG15-DE-3114