Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 38
Encontrada(s) 1 página(s)

Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

FRCSyn challenge at WACV 2024: Face recognition challenge in the era of synthetic data

  • Melzi, Pietro
  • Vera Rodríguez, Rubén
  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Ortega García, Javier
  • Tolosana Moranchel, Rubén
  • Andrés Tamé, Iván Ioel de
Artículo escrito por un elevado número de autores, solo se referencian el que aparece en primer lugar, el nombre del grupo de colaboración, si le hubiere, y los autores pertenecientes a la UAM, Despite the widespread adoption of face recognition technology around the world, and its remarkable performance on current benchmarks, there are still several challenges that must be covered in more detail. This paper offers an overview of the Face Recognition Challenge in the Era of Synthetic Data (FRCSyn) organized at WACV 2024. This is the first international challenge aiming to explore the use of synthetic data in face recognition to address existing limitations in the technology. Specifically, the FRCSyn Challenge targets concerns related to data privacy issues, demographic biases, generalization to unseen scenarios, and performance limitations in challenging scenarios, including significant age disparities between enrollment and testing, pose variations, and occlusions. The results achieved in the FRCSyn Challenge, together with the proposed benchmark, contribute significantly to the application of synthetic data to improve face recognition technology., This study has received funding from the European Union's Horizon 2020 TReSPAsS-ETN (No 860813) and is supported by INTER-ACTION (PID2021- 126521OB-100 MICINN/FEDER) and R&D Agreement DGGC/UAM/FUAM for Biometrics and Cybersecurity. It is also supported by the German Federal Ministry of Education and Research and the Hessian Ministry of Higher Education, Research, Science and the Arts within their joint support of the National Research Center for Applied Cybersecurity ATHENE. MeVer was supported by the EU Horizon Europe project MAMMOth (Grant Agreement 101070285)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Capítulo o parte de un libro (BookPart)

DeepFakes Detection Based on Heart Rate Estimation: Single- and Multi-frame

  • Hernández Ortega, Javier
  • Fiérrez Aguilar, Julián
  • Morales Moreno, Aythami
  • Tolosana Moranchel, Rubén
This chapter describes a DeepFake detection framework based on physiological measurement. In particular, we consider information related to the heart rate using remote photoplethysmography (rPPG). rPPG methods analyze video sequences looking for subtle color changes in the human skin, revealing the presence of human blood under the tissues. This chapter explores to what extent rPPG is useful for the detection of DeepFake videos. We analyze the recent fake detector named DeepFakesON-Phys that is based on a Convolutional Attention Network (CAN), which extracts spatial and temporal information from video frames, analyzing and combining both sources to better detect fake videos. DeepFakesON-Phys has been experimentally evaluated using the latest public databases in the field: Celeb-DF v2 and DFDC. The results achieved for DeepFake detection based on a single frame are over 98% AUC (Area Under the Curve) on both databases, proving the success of fake detectors based on physiological measurement to detect the latest DeepFake videos. In this chapter, we also propose and study heuristical and statistical approaches for performing continuous DeepFake detection by combining scores from consecutive frames with low latency and high accuracy (100% on the Celeb-DF v2 evaluation dataset). We show that combining scores extracted from short-time video sequences can improve the discrimination power of DeepFakesON-Phys, This work has been supported by projects: PRIMA (H2020-MSCA-ITN2019-860315), TRESPASS-ETN (H2020-MSCA-ITN-2019-860813), BIBECA (MINECO/FEDER RTI2018-101248-B-I00), and COST CA16101 (MULTI-FORESEE). J. H.-O. is supported by a PhD fellowship from UAM
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

Cancelable face biometrics with soft-biometric privacy enhancement

  • Melzi, Pietro
  • Otroshi Shahreza, Hatef
  • Rathgeb, Christian
  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Marcel, Sébastien
  • Busch, Christoph
  • Tolosana Moranchel, Rubén
The storage of biometric data has raised significant privacy concerns, necessitating robust measures for secure storage. While traditional Privacy-Enhancing Technologies (PETs), like Cancelable Biometric (CB) schemes, excel at creating protected templates that fulfill criteria such as irreversibility and unlinkability, they often fail to preserve the privacy of soft-biometric information. To address this issue, we propose a hybrid technology that combines PETs, leveraging their different properties to comprehensively address multiple privacy requirements and enhance overall protection for biometric templates. In our approach, we integrate Multi Incremental Variable Elimination (Multi-IVE), a recent technology designed to remove soft-biometric information from biometric templates, with conventional CB schemes. We apply our hybrid technology to facial templates and assess the properties of the resulting protected templates. In the event of stolen secrets, the combination of Multi-IVE with CB schemes helps decrease the accuracy of estimating soft-biometric attributes without affecting recognition performance, compared to CB schemes alone, This work was supported in part by European Union’s Horizon 2020 Research and Innovation Program under the Marie Skłodowska-Curie under Grant 860813—TReSPAsS-ETN; and in part by German Federal Ministry of Education and Research and the Hessian Ministry of Higher Education, Research, Science and the Arts within their joint support of the National Research Center for Applied Cybersecurity ATHENE. The work of Rubén Tolosana and Ruben Vera-Rodríguez was supported by INTERACTION under Grant PID2021-126521OB-I00 MICINN/FEDER. The work of Julián Fiérrez was supported by BBforTAI under Grant PID2021-127641OB-I00 MICINN/FEDER. UAM members are also supported by Cátedra ENIA UAM-VERIDAS en IA Responsable (NextGenerationEU PRTR TSI-100927-2023-2)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

SVC-onGoing: Signature verification competition

  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Morales Moreno, Aythami
  • Ortega García, Javier
  • Tolosana Moranchel, Rubén
  • Gonzalez-Garcia, Carlos
  • Carlos Ruiz-Garcia, Juan
  • Romero Tapiador, Sergio
  • Rengifo, Santiago
  • Caruana, Miguel
  • Jiang, Jiajia
  • Lai, Songxuan
  • Jin, Lianwen
  • Zhu, Yecheng
  • Galbally, Javier
  • Diaz, Moises
  • Angel Ferrer, Miguel
  • Gomez-Barrero, Marta
  • Hodashinsky, Ilya
  • Sarin, Konstantin
  • Slezkin, Artem
  • Bardamova, Marina
  • Svetlakov, Mikhail
  • Saleem, Mohammad
  • Lia Szcs, Cintia
  • Kovari, Bence
  • Pulsmeyer, Falk
  • Wehbi, Mohamad
  • Zanca, Dario
  • Ahmad, Sumaiya
  • Mishra, Sarthak
  • Jabin, Suraiya
This article presents SVC-onGoing1, an on-going competition for on-line signature verification where researchers can easily benchmark their systems against the state of the art in an open common platform using large-scale public databases, such as DeepSignDB2 and SVC2021_EvalDB3, and standard experimental protocols. SVC-onGoing is based on the ICDAR 2021 Competition on On-Line Signature Verification (SVC 2021), which has been extended to allow participants anytime. The goal of SVC-onGoing is to evaluate the limits of on-line signature verification systems on popular scenarios (office/mobile) and writing inputs (stylus/finger) through large-scale public databases. Three different tasks are considered in the competition, simulating realistic scenarios as both random and skilled forgeries are simultaneously considered on each task. The results obtained in SVC-onGoing prove the high potential of deep learning methods in comparison with traditional methods. In particular, the best signature verification system has obtained Equal Error Rate (EER) values of 3.33% (Task 1), 7.41% (Task 2), and 6.04% (Task 3). Future studies in the field should be oriented to improve the performance of signature verification systems on the challenging mobile scenarios of SVC-onGoing in which several mobile devices and the finger are used during the signature acquisition, This work has been supported by projects: PRIMA (H2020- MSCA-ITN-2019-860315), TRESPASS-ETN (H2020-MSCA-ITN-2019- 860813), INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER), Orange Labs, and by UAM-Cecabank
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

SensitiveNets: Learning agnostic representations with application to face images

  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Vera Rodríguez, Rubén
  • Tolosana Moranchel, Rubén
© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, This work proposes a novel privacy-preserving neural network feature representation to suppress the sensitive information of a learned space while maintaining the utility of the data. The new international regulation for personal data protection forces data controllers to guarantee privacy and avoid discriminative hazards while managing sensitive data of users. In our approach, privacy and discrimination are related to each other. Instead of existing approaches aimed directly at fairness improvement, the proposed feature representation enforces the privacy of selected attributes. This way fairness is not the objective, but the result of a privacy-preserving learning method. This approach guarantees that sensitive information cannot be exploited by any agent who process the output of the model, ensuring both privacy and equality of opportunity. Our method is based on an adversarial regularizer that introduces a sensitive information removal function in the learning objective. The method is evaluated on three different primary tasks (identity, attractiveness, and smiling) and three publicly available benchmarks. In addition, we present a new face annotation dataset with balanced distribution between genders and ethnic origins. The experiments demonstrate that it is possible to improve the privacy and equality of opportunity while retaining competitive performance independently of the task, This work was supported by projects: PRIMA (MSCA-ITN-2019-860315), TRESPASS-ETN (MSCA-ITN-2019-860813), and BIBECA (RTI2018-101248-B-I00 MINECO)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

OTB-morph: one-time biometrics via morphing

  • Ghafourian, Mahdi
  • Fiérrez Aguilar, Julián
  • Vera Rodríguez, Rubén
  • Morales Moreno, Aythami
  • Serna Cabello, José Ignacio de la
Cancelable biometrics are a group of techniques to transform the input biometric to an irreversible feature intentionally using a transformation function and usually a key in order to provide security and privacy in biometric recognition systems. This transformation is repeatable enabling subsequent biometric comparisons. This paper introduces a new idea to be exploited as a transformation function for cancelable biometrics aimed at protecting templates against iterative optimization attacks. Our proposed scheme is based on time-varying keys (random biometrics in our case) and morphing transformations. An experimental implementation of the proposed scheme is given for face biometrics. The results confirm that the proposed approach is able to withstand leakage attacks while improving the recognition performance, This work has been supported by projects: PRIMA (H2020-MSCA-ITN-2019- 860315), TRESPASS-ETN (H2020-MSCA-ITN-2019-860813), and BBforTAI (PID2021-127641OB-I00 MICINN/FEDER). M.G. is supported by PRIMA and I.S. is supported by an FPI fellowship from Univ. Autonoma de Madrid
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

SetMargin loss applied to deep keystroke biometrics with circle packing interpretation

  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Acién Ayala, Alejandro
  • Serna Cabello, José Ignacio de la
  • Tolosana Moranchel, Rubén
This work presents a new deep learning approach for keystroke biometrics based on a novel Distance Metric Learning method (DML). DML maps input data into a learned representation space that reveals a “semantic” structure based on distances. In this work, we propose a novel DML method specifically designed to address the challenges associated to free-text keystroke identification where the classes used in learning and inference are disjoint. The proposed SetMargin Loss (SM-L) extends traditional DML approaches with a learning process guided by pairs of sets instead of pairs of samples, as done traditionally. The proposed learning strategy allows to enlarge inter-class distances while maintaining the intra-class structure of keystroke dynamics. We analyze the resulting representation space using the mathematical problem known as Circle Packing, which provides neighbourhood structures with a theoretical maximum inter-class distance. We finally prove experimentally the effectiveness of the proposed approach on a challenging task: keystroke biometric identification over a large set of 78,000 subjects. Our method achieves state-of-the-art accuracy on a comparison performed with the best existing approaches, This work has been supported by projects: PRIMA ( MSCA-ITN- 2019-860315 ), TRESPASS-ETN (MSCA-ITN-2019-860813), BIBECA (RTI2018-101248-B-I00 MINECO), edBB (UAM), and Instituto de In- genieria del Conocimiento (IIC). A. Acien is supported by a FPI fel- lowship from the Spanish MINECO
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

Keystroke Biometrics in Response to Fake News Propagation in a Global Pandemic

  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Vera Rodríguez, Rubén
  • Ortega García, Javier
  • Tolosana Moranchel, Rubén
  • Acién Ayala, Alejandro
  • Monaco, John V.
© 2020 IEEE.  Personal use of this material is permitted.  Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, This work proposes and analyzes the use of keystroke biometrics for content de-anonymization. Fake news have become a powerful tool to manipulate public opinion, especially during major events. In particular, the massive spread of fake news during the COVID-19 pandemic has forced governments and companies to fight against missinformation. In this context, the ability to link multiple accounts or profiles that spread such malicious content on the Internet while hiding in anonymity would enable proactive identification and blacklisting. Behavioral biometrics can be powerful tools in this fight. In this work, we have analyzed how the latest advances in keystroke biometric recognition can help to link behavioral typing patterns in experiments involving 100,000 users and more than 1 million typed sequences. Our proposed system is based on Recurrent Neural Networks adapted to the context of content de-anonymization. Assuming the challenge to link the typed content of a target user in a pool of candidate profiles, our results show that keystroke recognition can be used to reduce the list of candidate profiles by more than 90%. In addition, when keystroke is combined with auxiliary data (such as location), our system achieves a Rank-1 identification performance equal to 52.6% and 10.9% for a background candidate list composed of 1K and 100K profiles, respectively, This work has been supported by projects: PRIMA (MSCA-ITN-2019-860315), TRESPASS (MSCA-ITN-2019-860813), BIBECA (RTI2018-101248-B-I00 MINECO), BioGuard (Ayudas Fundacion BBVA a Equipos de Investigacion Cientıfica 2017). A. Acien and R. Tolosana are supported by a FPI and postdoc fellowship from the Spanish MINECO
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

MEBAL: A multimodal database for eye blink detection and attention level estimation

  • Daza García, Roberto
  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Tolosana Moranchel, Rubén
© ACM 2020. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ICMI '20 Companion, October 25–29, 2020, Virtual Event, Netherlands, https://doi.org/10.1145/3395035.3425257, This work presents mEBAL, a multimodal database for eye blink detection and attention level estimation. The eye blink frequency is related to the cognitive activity and automatic detectors of eye blinks have been proposed for many tasks including attention level estimation, analysis of neuro-degenerative diseases, deception recognition, drive fatigue detection, or face anti-spoofing. However, most existing databases and algorithms in this area are limited to experiments involving only a few hundred samples and individual sensors like face cameras. The proposed mEBAL improves previous databases in terms of acquisition sensors and samples. In particular, three different sensors are simultaneously considered: Near Infrared (NIR) and RGB cameras to capture the face gestures and an Electroencephalography (EEG) band to capture the cognitive activity of the user and blinking events. Regarding the size of mEBAL, it comprises 6,000 samples and the corresponding attention level from 38 different students while conducting a number of e-learning tasks of varying difficulty. In addition to presenting mEBAL, we also include preliminary experiments on: i) eye blink detection using Convolutional Neural Networks (CNN) with the facial images, and ii) attention level estimation of the students based on their eye blink frequency, This work has been supported by projects: PRIMA (ITN-2019-860315), TRESPASS-ETN (ITN-2019-860813), IDEA-FAST (IMI2- 2018-15-two-stage-853981), BIBECA (RTI2018-101248-B-I00 MINECOFEDER), and edBB (Universidad Autonoma de Madrid). Ruben Tolosana and postdoc support from CAM/FEDER. Roberto Daza is supported by a PhD FPI fellowship from MINECO-FEDER
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

SaFL: Sybil-aware Federated Learning with application to face recognition

  • Ghafourian, Mahdi
  • Fiérrez Aguilar, Julián
  • Vera Rodríguez, Rubén
  • Morales Moreno, Aythami
  • Tolosana Moranchel, Rubén
© 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, Federated Learning (FL) is a machine learning paradigm to conduct collaborative learning among clients on a joint model. The primary goal is to share clients' local training parameters with an integrating server while preserving their privacy. This method permits to exploit the potential of massive mobile users' data for the benefit of machine learning models' performance while keeping sensitive data on local devices. On the downside, FL raises security and privacy concerns that have just started to be studied. To address some of the key threats in FL, researchers have proposed to use secure aggregation methods (e.g. homomorphic encryption, secure multiparty computation, etc.). These solutions improve some security and privacy metrics, but at the same time bring about other serious threats such as poisoning attacks, backdoor attacks, and free running attacks. This paper proposes a new defense method against poisoning attacks in FL called SaFL (Sybil-Aware Federated Learning) that minimizes the effect of sybils with a novel time-variant aggregation scheme, This work has been supported by projects: PRIMA (ITN-2019-860315), TRESPASS-ETN (ITN-2019-860813) and Cátedra ENIA UAM-VERIDAS en IA Responsable (NextGenerationEU PRTR TSI-100927-2023-2)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

Face image quality assessment: a literature survey

  • Schlett, Torsten
  • Rathgeb, Christian
  • Henninger, Olaf
  • Busch, Christoph
  • Galbally, Javier
  • Fiérrez Aguilar, Julián
The performance of face analysis and recognition systems depends on the quality of the acquired face data, which is influenced by numerous factors. Automatically assessing the quality of face data in terms of biometric utility can thus be useful to detect low-quality data and make decisions accordingly. This survey provides an overview of the face image quality assessment literature, which predominantly focuses on visible wavelength face image input. A trend towards deep learning-based methods is observed, including notable conceptual differences among the recent approaches, such as the integration of quality assessment into face recognition models. Besides image selection, face image quality assessment can also be used in a variety of other application scenarios, which are discussed herein. Open issues and challenges are pointed out, i.a., highlighting the importance of comparability for algorithm evaluations and the challenge for future work to create deep learning approaches that are interpretable in addition to providing accurate utility predictions, This research work has been funded by the German Federal Ministry of Education and Research and the Hessen State Ministry for Higher Education, Research and the Arts within their joint support of the National Research Center for Applied Cybersecurity ATHENE, project BIBECA (RTI2018-101248-B-I00 MINECO/FEDER), and project TReSPAsS-ETN (H2020- MSCA-ITN-2019-860813). This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 883356. This text reflects only the authors’ views and the Commission is not liable for any use that may be made of the information contained therein
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

GANprintR: Improved Fakes and Evaluation of the State of the Art in Face Manipulation Detection

  • Neves, João C.
  • Lopes, Vasco
  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Tolosana Moranchel, Rubén
  • Proença, Hugo
© 2020 IEEE.  Personal use of this material is permitted.  Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, The availability of large-scale facial databases, together with the remarkable progresses of deep learning technologies, in particular Generative Adversarial Networks (GANs), have led to the generation of extremely realistic fake facial content, raising obvious concerns about the potential for misuse. Such concerns have fostered the research on manipulation detection methods that, contrary to humans, have already achieved astonishing results in various scenarios. In this study, we focus on the synthesis of entire facial images, which is a specific type of facial manipulation. The main contributions of this study are four-fold: i) a novel strategy to remove GAN 'fingerprints' from synthetic fake images based on autoencoders is described, in order to spoof facial manipulation detection systems while keeping the visual quality of the resulting images; ii) an in-depth analysis of the recent literature in facial manipulation detection; iii) a complete experimental assessment of this type of facial manipulation, considering the state-of-the-art fake detection systems (based on holistic deep networks, steganalysis, and local artifacts), remarking how challenging is this task in unconstrained scenarios; and finally iv) we announce a novel public database, named iFakeFaceDB, yielding from the application of our proposed GAN-fingerprint Removal approach (GANprintR) to already very realistic synthetic fake images. The results obtained in our empirical evaluation show that additional efforts are required to develop robust facial manipulation detection systems against unseen conditions and spoof techniques, such as the one proposed in this study, This work has been supported by projects: PRIMA (H2020-MSCA-ITN-2019-860315), TRESPASS-ETN (H2020-MSCA-ITN2019-860813), BIBECA (RTI2018-101248-B-I00 MINECO/FEDER), BioGuard (Ayudas Fundación BBVA a Equipos de Investigación Cientíifica 2017), Accenture, by NOVA LINCS (UIDB/04516/2020) with the financial support of FCT - Fundação para a Ciência e a Tecnologia, through national funds, and by FCT/MCTES through national funds and co-funded by EU under the project UIDB/EEA/50008/2020
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

Sensitive loss: Improving accuracy and fairness of face representations with discrimination-aware deep learning

  • Serna Cabello, José Ignacio de la
  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Obradovich, Nick
We propose a discrimination-aware learning method to improve both the accuracy and fairness of biased face recognition algorithms. The most popular face recognition benchmarks assume a distribution of subjects without paying much attention to their demographic attributes. In this work, we perform a comprehensive discrimination-aware experimentation of deep learning-based face recognition. We also propose a notational framework for algorithmic discrimination with application to face biometrics. The experiments include three popular face recognition models and three public databases composed of 64,000 identities from different demographic groups characterized by sex and ethnicity. We experimentally show that learning processes based on the most used face databases have led to popular pre-trained deep face models that present evidence of strong algorithmic discrimination. Finally, we propose a discrimination-aware learning method, Sensitive Loss, based on the popular triplet loss function and a sensitive triplet generator. Our approach works as an add-on to pre-trained networks and is used to improve their performance in terms of average accuracy and fairness. The method shows results comparable to state-of-the-art de-biasing networks and represents a step forward to prevent discriminatory automatic systems., This work has been supported by projects: TRESPASS-ETN (MSCA-ITN-2019-860813), PRIMA (MSCA-ITN-2019-860315), BIBECA (RTI2018-101248-B-I00 MINECO/FEDER), and BBforTAI (PID2021-127641OB-I00 MICINN/FEDER). I. Serna is supported by a research fellowship from the Universidad Autónoma de Madrid (FPI-UAM-2020). A. Morales is supported by the Madrid Government (Comunidad de Madrid-Spain) under the Multiannual Agreement with Autonomous University of Madrid in the line Encouragement of the Research of Young Researchers, in the context of the V PRICIT (Regional Programme of Research and Technological Innovation)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

Deepfakes and beyond: A Survey of face manipulation and fake detection

  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Morales Moreno, Aythami
  • Ortega García, Javier
  • Tolosana Moranchel, Rubén
The free access to large-scale public databases, together with the fast progress of deep learning techniques, in particular Generative Adversarial Networks, have led to the generation of very realistic fake content with its corresponding implications towards society in this era of fake news. This survey provides a thorough review of techniques for manipulating face images including DeepFake methods, and methods to detect such manipulations. In particular, four types of facial manipulation are reviewed: i) entire face synthesis, ii) identity swap (DeepFakes), iii) attribute manipulation, and iv) expression swap. For each manipulation group, we provide details regarding manipulation techniques, existing public databases, and key benchmarks for technology evaluation of fake detection methods, including a summary of results from those evaluations. Among all the aspects discussed in the survey, we pay special attention to the latest generation of DeepFakes, highlighting its improvements and challenges for fake detection. In addition to the survey information, we also discuss open issues and future trends that should be considered to advance in the field, This work has been supported by projects: PRIMA (H2020-MSCAITN-2019-860315), TRESPASS-ETN (H2020-MSCA-ITN-2019-860813), BIBECA (MINECO/FEDER RTI2018-101248-B-I00), Bio-Guard (Ayudas Fundación BBVA a Equipos de Investigación Científica 2017), and Accenture. Ruben Tolosana is supported by Consejería de Educación, Juventud y Deporte de la Comunidad de Madrid y Fondo Social Europeo
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

A Comparative Evaluation of Heart Rate Estimation Methods using Face Videos

  • Hernández Ortega, Javier
  • Diaz, David
  • Fiérrez Aguilar, Julián
  • Morales Moreno, Aythami
© 2020 IEEE.  Personal use of this material is permitted.  Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, This paper presents a comparative evaluation of methods for remote heart rate estimation using face videos, i.e., given a video sequence of the face as input, methods to process it to obtain a robust estimation of the subject's heart rate at each moment. Four alternatives from the literature are tested, three based in hand-crafted approaches and one based on deep learning. The methods are compared using RGB videos from the COHFACE database. Experiments show that the learning-based method achieves much better accuracy than the hand-crafted ones. The low error rate achieved by the learning-based model makes possible its application in real scenarios, e.g. in medical or sports environments, This work has been supported by projects: IDEAFAST (IMI2-2018-15-two-stage-853981), PRIMA (ITN-2019-860315), TRESPASS-ETN (ITN-2019-860813), BIBECA (RTI2018- 101248-B-I00 MINECO/FEDER), and edBB (UAM). J.H.O. is supported by a PhD Scholarship from UAM. Portions of the research in this paper used the COHFACE Dataset made available by the Idiap Research Institute, Martigny, Switzerland
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

Assessing the quality of swipe interactions for mobile biometric systems

  • Santopietro, Marco
  • Guest, Richard
  • Acién Ayala, Alejandro
  • Vera Rodríguez, Rubén
  • Morales Moreno, Aythami
© 2020 IEEE.  Personal use of this material is permitted.  Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, Quality estimation is a key study in biometrics, allowing optimisation and improvement of existing authentication systems by giving a prediction on the model performance based on the goodness of the sample or the user. In this paper, we propose a quality metric for swipe gestures on mobile devices. We evaluate a quality score for subjects on enrollment and for swipe samples, we estimate three quality groups and explore the correlation between our quality score and a state-of-art biometric authentication classifier performance. A further analysis based on the combined effects of subject quality and the amount of enrollment samples is conducted, investigating if increasing or decreasing enrollment size affects the authentication performance for different quality groups. Results are shown for three different public datasets, highlighting how higher quality users score a lower equal error rate compared to medium and low quality users, while high quality samples get a higher similarity score from the classifier, Thanks to support by projects: PRIMA (H2020-MSCAITN-2019-860315), TRESPASS-ETN (H2020-MSCAITN-2019-860813), BIBECA (RTI2018-101248-B-I00 MINECO/FEDER), and BioGuard (Ayudas Fundacion BBVA a Equipos de Investigacion Científica 2017)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

DeepSign: Deep On-Line Signature Verification

  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Ortega García, Javier
  • Tolosana Moranchel, Rubén
Deep learning has become a breathtaking technology in the last years, overcoming traditional handcrafted approaches and even humans for many different tasks. However, in some tasks, such as the verification of handwritten signatures, the amount of publicly available data is scarce, what makes difficult to test the real limits of deep learning. In addition to the lack of public data, it is not easy to evaluate the improvements of novel proposed approaches as different databases and experimental protocols are usually considered. The main contributions of this study are: i) we provide an in-depth analysis of state-of-the-art deep learning approaches for on-line signature verification, ii) we present and describe the new DeepSignDB on-line handwritten signature biometric public database,1 iii) we propose a standard experimental protocol and benchmark to be used for the research community in order to perform a fair comparison of novel approaches with the state of the art, and iv) we adapt and evaluate our recent deep learning approach named Time-Aligned Recurrent Neural Networks (TA-RNNs)2. for the task of on-line handwritten signature verification. This approach combines the potential of Dynamic Time Warping and Recurrent Neural Networks to train more robust systems against forgeries. Our proposed TA-RNN system outperforms the state of the art, achieving results even below 2.0% EER when considering skilled forgery impostors and just one training signature per user.1https://github.com/BiDAlab/DeepSignDB 2Spanish Patent Application (P202030060), This work was supported in part by PRIMA under Grant H2020-MSCA-ITN-2019-860315; in part by TRESPASS-ETN under Grant H2020-MSCA-ITN-2019-860813; in part by BIBECA under Grant RTI2018-101248-B-I00 MINECO/FEDER; in part by Bio-Guard (Ayudas Fundación BBVA a Equipos de Investigación Científica 2017); and in part by UAM-Cecabank. The work of Ruben Tolosana was supported by the Comunidad de Madrid and Fondo Social Europeo. Spanish Patent Application (P202030060)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

SDFR: Synthetic Data for Face Recognition competition

  • Shahreza, Hatef Otroshi
  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Tolosana Moranchel, Rubén
  • Andrés Tamé, Iván Ioel de
Artículo escrito por un elevado número de autores, solo se referencian el que aparece en primer lugar, el nombre del grupo de colaboración, si le hubiere, y los autores pertenecientes a la UAM, Large-scale face recognition datasets are collected by crawling the Internet and without individuals' consent, raising legal, ethical, and privacy concerns. With the recent advances in generative models, recently several works proposed generating synthetic face recognition datasets to mitigate concerns in web-crawled face recognition datasets. This paper presents the summary of the Synthetic Data for Face Recognition (SDFR) Competition held in conjunction with the 18th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2024) and established to investigate the use of synthetic data for training face recognition models. The SDFR competition was split into two tasks, allowing participants to train face recognition systems using new synthetic datasets and/or existing ones. In the first task, the face recognition backbone was fixed and the dataset size was limited, while the second task provided almost complete freedom on the model backbone, the dataset, and the training pipeline. The submitted models were trained on existing and also new synthetic datasets and used clever methods to improve training with synthetic data. The submissions were evaluated and ranked on a diverse set of seven benchmarking datasets. The paper gives an overview of the submitted face recognition models and reports achieved performance compared to baseline models trained on real and synthetic datasets. Furthermore, the evaluation of submissions is extended to bias assessment across different demography groups. Lastly, an outlook on the current state of the research in training face recognition models using synthetic data is presented, and existing problems as well as potential future directions are also discussed, The organization of this competition was supported by the H2020 TReSPAsS-ETN Marie Skłodowska-Curie early training network (grant agreement 860813) as well as the Hasler foundation through the “Responsible Face Recognition” (SAFER) project. The work of BioLab team received funding from the European Union's Horizon 2020 research and innovation program under Grant Agreement No. 883356. The submitted solution by the IGD-IDiff-Face team has been funded by the German Federal Ministry of Education and Research and the Hessian Ministry of Higher Education, Research, Science and the Arts within their joint support of the National Research Center for Applied Cybersecurity ATHENE. The work of BiDA-PRA team was supported by INTER-ACTION (PID2021-1265210B-I00 MICINN/FEDER), Cátedra ENIA UAM-VERIDAS en IA Responsable (NextGenerationEU PRTR TSI-100927-2023-2), and R&D Agreement DGGC/UAM/FUAM for Biometrics and Cybersecurity. The work of BiDA-PRA team was also supported by the European Union - Next Generation EU through the Italian Ministry of University and Research (MUR) within the PRIN PNRR 2022 - BullyBuster 2 - the ongoing fight against bullying and cyberbullying with the help of artificial intelligence for the human wellbeing (CUP: F53D23009240001)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

Fishing gear classification from vessel trajectories and velocity profiles: database and benchmark

  • Melzi, Pietro
  • Rodriguez-Albala, Juan Manuel
  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Vera Rodríguez, Rubén
  • Tolosana Moranchel, Rubén
International Organizations demand to take care of our oceans and their ecosystems since they are of incalculable value to humanity. The illegal fishing activity does irreparable damage to these ecosystems and these organism are pushing to detect and combat illegal fishing activities. Fishing vessels are equipped with a radio frequency beacon that emits their GPS position and other information relevant to the Automatic Identification System (AIS). The GPS positions can be used to infer the vessel trajectories and detect illegal fishing activities. In this study we present a new database (https://github.com/BiDAlab/TrFGdb ) including trajectories representing 5 different fishing gears, and analyze them as in a problem of time sequence analysis. We extract global and local features from the trajectories of vessels, and propose several supervised learning algorithms to classify the kinematics of vessels according to different fishing gears. Compared to previous works, we highlight the importance of considering trajectories with sampling period in the order of minutes instead of hours, to detect activities carried out in a short time that could help to distinguish fishing gears. A considerable effort has been dedicated to pre-processing the real data at our disposal, to generate a quality dataset with highly reliable labels. The best classification accuracy obtained in this study is 90%. We expect to improve it if more trajectories describing the different fishing gears were available, We thank Tragsatec’s Management of Agricultural and Fisheries Information Systems and the General Secretariat of Fisheries of the Spanish Ministry of Agriculture, Fisheries and Food for the data and expertise provided to carry out the study. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sk lodowska-Curie grant agreement No 860813 - TReSPAsS-ETN. This study is also supported by the project INTER-ACTION (PID2021- 126521OB-I00 MICINN/FEDER)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

Second edition FRCSyn challenge at CVPR 2024: Face recognition challenge in the era of synthetic data

  • DeAndres-Tame, Ivan
  • Melzi, Pietro
  • Vera Rodríguez, Rubén
  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Ortega García, Javier
  • Tolosana Moranchel, Rubén
”Artículo escrito por un elevado número de autores, solo se referencian el que aparece en primer lugar, el nombre del grupo de colaboración, si le hubiere, y los autores pertenecientes a la UAM”, “© 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works”, Synthetic data is gaining increasing relevance for training machine learning models. This is mainly motivated due to several factors such as the lack of real data and intraclass variability, time and errors produced in manual labeling, and in some cases privacy concerns, among others. This paper presents an overview of the 2nd edition of the Face Recognition Challenge in the Era of Synthetic Data (FRCSyn) organized at CVPR 2024. FRCSyn aims to investigate the use of synthetic data in face recognition to address current technological limitations, including data privacy concerns, demographic biases, generalization to novel scenarios, and performance constraints in challenging situations such as aging, pose variations, and occlusions. Unlike the 1st edition, in which synthetic data from DCFace and GANDiffFace methods was only allowed to train face recognition systems, in this 2nd edition we propose new subtasks that allow participants to explore novel face generative methods. The outcomes of the 2nd FRCSyn Challenge, along with the proposed experimental protocol and benchmarking contribute significantly to the application of synthetic data to face recognition, This study has received funding from the European Union’s Horizon 2020 TReSPAsS-ETN (No 860813) and is supported by INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER), Catedra ENIA UAM-VERIDAS en IA Responsable (NextGenerationEU PRTR TSI-100927-2023-2) and R&D Agreement DGGC/ UAM/FUAM for Biometrics and Cybersecurity. It is also supported by the German Federal Ministry of Education and Research and the Hessian Ministry of Higher Education, Research, Science and the Arts within their joint support of the National Research Center for Applied Cybersecurity ATHENE. K-IBS-DS was supported by the Institute for Basic Science, Republic of Korea (IBS-R029-C2). UNICA-IGD-LSI was supported by the ARIS program P2-0250B
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

DeepFakes detection across generations: Analysis of facial regions, fusion, and performance evaluation

  • Romero Tapiador, Sergio
  • Gonzalez Sosa
  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Tolosana Moranchel, Rubén
Media forensics has attracted a tremendous attention in the last years in part due to the increasing concerns around DeepFakes. Since the release of the initial DeepFakes databases of the 1st generation such as UADFV and FaceForensics++ up to the latest databases of the 2nd generation such as Celeb-DF and DFDC, many visual improvements have been carried out, making fake videos almost indistinguishable to the human eye. This study provides an in-depth analysis of both 1st and 2nd DeepFakes generations in terms of fake detection performance. Two different methods are considered in our experimental framework: (i) the traditional one followed in the literature based on selecting the entire face as input to the fake detection system, and (ii) a novel approach based on the selection of specific facial regions as input to the fake detection system. Fusion techniques are applied both to the facial regions and also to three different state-of-the-art fake detection systems (Xception, Capsule Network, and DSP-FWA) in order to further increase the robustness of the detectors considered. Finally, experiments regarding intra- and inter-database scenarios are performed. Among all the findings resulting from our experiments, we highlight: (i) the very good results achieved using facial regions and fusion techniques with fake detection results above 99% Area Under the Curve (AUC) for UADFV, FaceForensics++, and Celeb-DF v2 databases, and (ii) the necessity to put more efforts on the analysis of inter-database scenarios to improve the ability of the fake detectors against attacks unseen during learning, This work has been supported by projects: PRIMA (H2020-MSCAITN-2019-860315), TRESPASS-ETN (H2020-MSCA-ITN-2019-860813), BIBECA, Spain (MINECO/FEDER RTI2018-101248-B-I00)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

PAD-Phys: Exploiting physiology for presentation attack detection in face biometrics

  • Gómez, Luis F.
  • Solano, Imanol
  • Garcia, Alejandro
  • Zamora-Martinez, Francisco
  • Fiérrez Aguilar, Julián
  • Morales Moreno, Aythami
  • Tolosana Moranchel, Rubén
  • Ghafourian, Mahdi
Presentation Attack Detection (PAD) is a crucial stage in facial recognition systems to avoid leakage of personal information or spoofing of identity to entities. Recently, pulse detection based on remote photoplethysmography (rPPG) has been shown to be effective in face presentation attack detection. This work presents three different approaches to the presentation attack detection based on rPPG: (i) The physiological domain, a domain using rPPG-based models, (ii) the Deepfakes domain, a domain where models were retrained from the physiological domain to specific Deepfakes detection tasks; and (iii) a new Presentation Attack domain was trained by applying transfer learning from the two previous domains to improve the capability to differentiate between bona-fides and attacks. The results show the efficiency of the rPPG-based models for presentation attack detection, evidencing a 21.70% decrease in average classification error rate (ACER) (from 41.03% to 19.32%) when the presentation attack domain is compared to the physiological and Deepfakes domains. Our experiments highlight the efficiency of transfer learning in rPPG-based models and perform well in presentation attack detection in instruments that do not allow copying of this physiological feature, This work has been supported by Veridas and by projects: TRESPASS-ETN (ITN-2019-860813), PRIMA (ITN2019-860315), and BBforTAI (PID2021-127641OB-I00 MICINN/FEDER)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

IRL-Net: inpainted region localization network via spatial attention

  • Daryani, Amir Etefaghi
  • Mirmahdi, Mahdieh
  • Hassanpour, Ahmad
  • Shahreza, Hatef Otroshi
  • Yang, Bian
  • Fiérrez Aguilar, Julián
Identifying manipulated regions in images is a challenging task due to the existence of very accurate image inpainting techniques leaving almost unnoticeable traces in tampered regions. These image inpainting methods can be used for multiple purposes (e.g., removing objects, reconstructing corrupted areas, eliminating various types of distortion, etc.) makes creating forensic detectors for image manipulation an extremely difficult and time-consuming procedure. The aim of this paper is to localize the tampered regions manipulated by image inpainting methods. To do this, we propose a novel CNN-based deep learning model called IRL-Net which includes three main modules: Enhancement, Encoder, and Decoder modules. To evaluate our method, three image inpainting methods have been used to reconstruct the missed regions in two face and scene image datasets. We perform both qualitative and quantitative evaluations on the generated datasets. Experimental results demonstrate that our method outperforms previous learning-based manipulated region detection methods and generates realistic and semantically plausible images. We also provide the implementation of the proposed approach to support reproducible research via https://github.com/amiretefaghi/IRL-Net, This work was supported by the Project Privacy Matters (PRIMA) under Grant H2020-MSCA-ITN-2019-860315. The work of Hatef Otroshi Shahreza was supported by the H2020 TReSPAsS-ETN Marie Skłodowska-Curie Early Training Network under Grant 860813. The work of Julian Fierrez was supported by the Project Biometrics and Behavior for Unbiased and Trusted AI with Applications (BBforTAI) under Grant PID2021-127641OB-I00 MICINN/FEDER
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

Learning emotional-blinded face representations

  • Peña Almansa, Alejandro
  • Lapedriza, Agata
  • Fiérrez Aguilar, Julián
  • Morales Moreno, Aythami
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, We propose two face representations that are blind to facial expressions associated to emotional responses. This work is in part motivated by new international regulations for personal data protection, which enforce data controllers to protect any kind of sensitive information involved in automatic processes. The advances in Affective Computing have contributed to improve human-machine interfaces but, at the same time, the capacity to monitorize emotional responses triggers potential risks for humans, both in terms of fairness and privacy. We propose two different methods to learn these expression-blinded facial features. We show that it is possible to eliminate information related to emotion recognition tasks, while the performance of subject verification, gender recognition, and ethnicity classification are just slightly affected. We also present an application to train fairer classifiers in a case study of attractiveness classification with respect to a protected facial expression attribute. The results demonstrate that it is possible to reduce emotional information in the face representation while retaining competitive performance in other face-based artificial intelligence tasks, This work has been supported by projects: PRIMA (H2020-MSCA-ITN-2019-860315), TRESPASS-ETN (H2020-MSCAITN-2019-860813), IDEA-FAST (IMI2-2018-15-853981), BIBECA (RTI2018-101248-B-I00 MINECO/FEDER), REAVIPERO (RED2018-102511-T), RTI2018-095232-B-C22 MINECO, and Accenture. A. Pena is supported by a research fellowship (PEJ2018-004094A) from the Spanish MINECO
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

Symbolic AI for XAI: Evaluating LFIT inductive programming for explaining biases in machine learning

  • Ortega de la Puente, Alfonso
  • Wang, Zilong
  • de la Cruz, Marina
  • Alonso, César Luis
  • Ribeiro, Tony
  • Fiérrez Aguilar, Julián
  • Morales Moreno, Aythami
Machine learning methods are growing in relevance for biometrics and personal information processing in domains such as forensics, e-health, recruitment, and e-learning. In these domains, white-box (human-readable) explanations of systems built on machine learning methods become crucial. Inductive logic programming (ILP) is a subfield of symbolic AI aimed to automatically learn declarative theories about the processing of data. Learning from interpretation transition (LFIT) is an ILP technique that can learn a propositional logic theory equivalent to a given black-box system (under certain conditions). The present work takes a first step to a general methodology to incorporate accurate declarative explanations to classic machine learning by checking the viability of LFIT in a specific AI application scenario: fair recruitment based on an automatic tool generated with machine learning methods for ranking Curricula Vitae that incorporates soft biometric information (gender and ethnicity). We show the expressiveness of LFIT for this specific problem and propose a scheme that can be applicable to other domains. In order to check the ability to cope with other domains no matter the machine learning paradigm used, we have done a preliminary test of the expressiveness of LFIT, feeding it with a real dataset about adult incomes taken from the US census, in which we consider the income level as a function of the rest of attributes to verify if LFIT can provide logical theory to support and explain to what extent higher incomes are biased by gender and ethnicity, This work was supported by projects: PRIMA (H2020-MSCA-ITN-2019-860315), TRESPASS-ETN(H2020-MSCA-ITN-2019-860813), IDEA-FAST (IMI2-2018-15-853981), BIBECA(RTI2018-101248-B-I00MINECO/FEDER), RTI2018-095232-B-C22MINECO, PLeNTaS project PID2019-111430RBI00MINECO; and also by Pays de la Loire Region through RFI Atlanstic 2020
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

Multi-IVE: privacy enhancement of multiple soft-biometrics in face embeddings

  • Melzi, Pietro
  • Shahreza, Hatef Otroshi
  • Rathgeb, Christian
  • Marcel, Sébastien
  • Busch, Christoph
  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Tolosana Moranchel, Rubén
This study focuses on the protection of soft-biometric at-tributes related to the demographic information of individ-uals that can be extracted from compact representations of face images, called embeddings. We consider a state-of-the-art technology for soft-biometric privacy enhancement, Incremental Variable Elimination (IVE), and propose Multi-IVE, a new method based on IVE to secure multiple soft-biometric attributes simultaneously. Several aspects of this technology are investigated, proposing different approaches to effectively identify and discard multiple soft-biometric at-tributes contained in face embeddings. In particular, we consider a domain transformation using Principle component Analysis (PCA), and apply IVE in the PCA domain. A complete analysis of the proposed Multi-IVE algorithm is carried out studying the embeddings generated by state-of-the-art face feature extractors, predicting soft-biometric attributes contained within them with multiple machine learning classifiers, and providing a cross-database evaluation. The results obtained show the possibility to simultane-ously secure multiple soft-biometric attributes and support the application of embedding domain transformations be-fore addressing the enhancement of soft-biometric privacy, This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 860813 - TReSPAsS-ETN and the German Federal Ministry of Education and Research and the Hessian Ministry of Higher Education, Research, Science and the Arts within their joint support of the National Research Center for Applied Cybersecurity ATHENE. R. Tolosana and R. Vera-Rodriguez are also supported by INTERACTION (PID2021-126521OB-I00 MICINN/FEDER)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

Spatio-temporal trajectory data modeling for fishing gear classification

  • Rodriguez‑Albala, Juan Manuel
  • Peña Almansa, Alejandro
  • Melzi, Pietro
  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Vera Rodríguez, Rubén
  • Ortega García, Javier
  • Tolosana Moranchel, Rubén
International Organizations urge the protection of our oceans and their ecosystems due to their immeasurable importance to humankind. Since illegal fshing activities, commonly known as IUU fshing, cause irreparable damage to these ecosystems, concerned organisms are pushing to detect and combat IUU fshing practices. The automatic identifcation system allows to locate the position and trajectory of fshing vessels. In this study we address the task of detecting vessels’ fshing gears based on the trajectory behavior defned by GPS position data, a useful task to prevent the proliferation of IUU fshing practices. We present a new database including trajectories that span 7 diferent fshing gears and analyze these as in a time sequence analysis problem. We leverage from feature extraction techniques from the online signature verifcation domain to model vessel trajectories, and extract relevant information in the form of both local and global feature sets. We show how, based on these sets of features, the kinematics of vessels according to diferent fshing gears can be efectively classifed using common supervised learning algorithms with accuracies up to 90%. Furthermore, motivated by the concerns raised by several organizations on the adverse impact of bottom trawling on marine biodiversity, we present a binary classifcation experiment in which we were able to distinguish this kind of fshing gear with an accuracy of 99%. We also illustrate in an ablation study the relevance of factors such as data availability and the sampling period to perform fshing gear classifcation. Compared to existing works, we highlight these factors, especially the importance of using sampling periods in the order of minutes instead of hours, This work has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 860813-TReSPAsS-ETN. This study is also supported by the project INTER-ACTION (PID2021- 126521OB-I00 MICINN/FEDER)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

Exploring transformers for on-line handwritten signature verification

  • Melzi, Pietro
  • Stragapede, Giuseppe
  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Ortega García, Javier
  • Tolosana Moranchel, Rubén
  • Delgado Santos, Paula
The application of mobile biometrics as a user-friendly authentication method has increased in the last years. Recent studies have proposed novel behavioral biometric recognition systems based on Transformers, which currently outperform the state of the art in several application scenarios. On-line handwritten signature verification aims to verify the identity of subjects, based on their biometric signatures acquired using electronic devices such as tablets or smartphones. This paper investigates the suitability of architectures based on recent Transformers for on-line signature verification. In particular, four different configurations are studied, two of them rely on the Vanilla Transformer encoder, and the two others have been successfully applied to the tasks of gait and activity recognition. We evaluate the four proposed configurations according to the experimental protocol proposed in the SVC-onGoing competition. The results obtained in our experiments are promising, and promote the use of Transformers for on-line signature verification, This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 860813 - TReSPAsSETN. With support also from projects INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER) and HumanCAIC (TED2021-131787B-I00 MICINN)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

BeCAPTCHA: Behavioral bot detection using touchscreen and mobile sensors benchmarked on HuMIdb

  • Acién Ayala, Alejandro
  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Vera Rodríguez, Rubén
  • Delgado Ben Mohatar, Óscar
In this paper we study the suitability of a new generation of CAPTCHA methods based on smartphone interactions. The heterogeneous flow of data generated during the interaction with the smartphones can be used to model human behavior when interacting with the technology and improve bot detection algorithms. For this, we propose BeCAPTCHA, a CAPTCHA method based on the analysis of the touchscreen information obtained during a single drag and drop task in combination with the accelerometer data. The goal of BeCAPTCHA is to determine whether the drag and drop task was realized by a human or a bot. We evaluate the method by generating fake samples synthesized with Generative Adversarial Neural Networks and handcrafted methods. Our results suggest the potential of mobile sensors to characterize the human behavior and develop a new generation of CAPTCHAs. The experiments are evaluated with HuMIdb1 (Human Mobile Interaction database), a novel multimodal mobile database that comprises 14 mobile sensors acquired from 600 users. HuMIdb is freely available to the research community, This work has been supported by projects: PRIMA, Spain (H2020-MSCA-ITN-2019-860315), TRESPASS-ETN, Spain (H2020-MSCA-ITN-2019-860813), BIBECA RTI2018-101248-B-I00 (MINECO/FEDER), and BioGuard, Spain (Ayudas Fundación BBVA a Equipos de Investigación Científica 2017). Spanish Patent Application P202030066
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

Blockchain in the Internet of Things: Architectures and Implementation

  • Delgado Ben Mohatar, Óscar
  • Fiérrez Aguilar, Julián
  • Morales Moreno, Aythami
  • Tolosana Moranchel, Rubén
© 2020 IEEE.  Personal use of this material is permitted.  Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., IEEE Annual International Computer Software and Applications Conference (COMPSAC) celebrado en 2020 en Madrid, The world is becoming more interconnected every day. With the high technological evolution and the increasing deployment of it in our society, scenarios based on the Internet of Things (IoT) can be considered a reality nowadays. However, and before some predictions become true (around 75 billion devices are expected to be interconnected in the next few years), many efforts must be carried out in terms of scalability and security. In this study we propose and evaluate a new approach based on the incorporation of Blockchain into current IoT scenarios. The main contributions of this study are as follows: i) an in depth analysis of the different possibilities for the integration of Blockchain into IoT scenarios, focusing on the limited processing capabilities and storage space of most IoT devices, and the eco nomic cost and performance of current Blockchain technologies; ii) a new method based on a novel module named BIoT Gateway that allows both unidirectional and bidirectional communications with IoT devices on real scenarios, allowing to exchange any kind of data; and iii) the proposed method has been fully implemented and validated on two different real-life IoT scenarios, extracting very interesting findings in terms of economic cost and execution time. The source code of our implementation is publicly available in the Ethereum testnet., This work has been supported by projects: PRIMA (H2020-MSCA-ITN-2019-860315), TRESPASS-ETN (H2020-MSCA-ITN-2019-860813), BIBECA (RTI2018- 101248-B-I00 MINECO/FEDER), and COPCIS (TIN2017- 84844-C2-1-R MINECO/FEDER). Ruben Tolosana is supported by Consejer´ıa de Educacion, Juventud y Deporte ´ de la Comunidad de Madrid y Fondo Social Europeo.
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Reseña (Review)

ECG biometric recognition: Review, system proposal, and benchmark evaluation

  • Melzi, Pietro
  • Vera Rodríguez, Rubén
  • Tolosana Moranchel, Rubén
ECGs have shown unique patterns to distinguish between different subjects and present important advantages compared to other biometric traits. However, the lack of public data and standard experimental protocols makes the evaluation and comparison of novel ECG methods difficult. In this study, we perform extensive analysis and comparison of different scenarios in ECG biometric recognition. We consider verification and identification tasks, single- and multi-session settings, and single- and multi-lead ECGs recorded with traditional and user-friendly devices. We also present ECGXtractor, a robust Deep Learning technology trained with an in-house large-scale database, and evaluate it with detailed experimental protocol and public databases. With the popular PTB database, we achieve Equal Error Rates of 0.14% and 2.06% in single- and multi-session verification. The results achieved prove the soundness of ECGXtractor across multiple scenarios and databases. We release the source code, experimental protocol details, and pre-trained models in GitHub to advance in the field, This work was supported in part by the European Union’s Horizon 2020 Research and Innovation Programme under the Marie Sklodowska-Curie Grant under Agreement 860813-TRESPASS-ETN and Agreement 860315-PRIMA-ITN; in part by the IDEA-FAST under Grant IMI2-2018-15-853981; and in part by INTER-ACTION, Ministerio de Ciencia e Innovación (MICINN)/Fondo Europeo de Desarrollo Regional (FEDER), under Grant PID2021-126521OB-I00
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

GANDiffFace: controllable generation of synthetic datasets for face recognition with realistic variations

  • Melzi, Pietro
  • Rathgeb, Christian
  • Lawatsch, Dominik
  • Domin, Florian
  • Schaubert, Maxim
  • Vera Rodríguez, Rubén
  • Tolosana Moranchel, Rubén
Face recognition systems have significantly advanced in recent years, driven by the availability of large-scale datasets. However, several issues have recently came up, including privacy concerns that have led to the discontinuation of well-established public datasets. Synthetic datasets have emerged as a solution, even though current synthesis methods present other drawbacks such as limited intraclass variations, lack of realism, and unfair representation of demographic groups. This study introduces GAN-DiffFace, a novel framework for the generation of synthetic datasets for face recognition that combines the power of Generative Adversarial Networks (GANs) and Diffusion models to overcome the limitations of existing synthetic datasets. In GANDiffFace, we first propose the use of GANs to synthesize highly realistic identities and meet target demographic distributions. Subsequently, we fine-tune Diffusion models with the images generated with GANs, synthesizing multiple images of the same identity with a variety of accessories, poses, expressions, and contexts. We generate multiple synthetic datasets by changing GANDiffFace settings, and compare their mated and non-mated score distributions with the distributions provided by popular real-world datasets for face recognition, i.e. VGG2 and IJB-C. Our results show the feasibility of the proposed GANDiffFace, in particular the use of Diffusion models to enhance the (limited) intra-class variations provided by GANs towards the level of real-world datasets, This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 860813 - TReSPAsS-ETN. This study is supported by the project INTER-ACTION (PID2021- 126521OB-I00 MICINN/FEDER). This research is based upon work supported by the Hessian Ministry of the Interior and Sport in the course of the Bio4ensics project and by the German Federal Ministry of Education and Research and the Hessian Ministry of Higher Education, Research, Science and the Arts within their joint support of the National Research Center for Applied Cybersecurity ATHENE
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

Synthetic data for the mitigation of demographic biases in face recognition

  • Melzi, Pietro
  • Rathgeb, Christian
  • Vera Rodríguez, Rubén
  • Morales Moreno, Aythami
  • Lawatsch, Dominik
  • Domin, Florian
  • Schaubert, Maxim
  • Tolosana Moranchel, Rubén
Los miembros de la UAM pertenecen al Biometrics and Data Pattern Analytics Laboratory, “© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works”, This study investigates the possibility of mitigating the demographic biases that affect face recognition technologies through the use of synthetic data. Demographic biases have the potential to impact individuals from specific demographic groups, and can be identified by observing disparate performance of face recognition systems across demographic groups. They primarily arise from the unequal representations of demographic groups in the training data. In recent times, synthetic data have emerged as a solution to some problems that affect face recognition systems. In particular, during the generation process it is possible to specify the desired demographic and facial attributes of images, in order to control the demographic distribution of the synthesized dataset, and fairly represent the different demographic groups. We propose to fine-tune with synthetic data existing face recognition systems that present some demographic biases. We use synthetic datasets generated with GANDiffFace, a novel framework able to synthesize datasets for face recognition with controllable demographic distribution and realistic intra-class variations. We consider multiple datasets representing different demographic groups for training and evaluation. Also, we fine-tune different face recognition systems, and evaluate their demographic fairness with different metrics. Our results support the proposed approach and the use of synthetic data to mitigate demographic biases in face recognition, This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 860813-TReSPAsS-ETN. This study is supported by the project INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER). This research is based upon work supported by the Hessian Ministry of the Interior and Sport in the course of the Bio4ensics project and by the German Federal Ministry of Education and Research and the Hessian Ministry of Higher Education, Research, Science and the Arts within their joint support of the National Research Center for Applied Cybersecurity ATHENE
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

TypeNet: Scaling up keystroke biometrics

  • Acién Ayala, Alejandro
  • Morales Moreno, Aythami
  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Monaco, John V.
© 2020 IEEE.  Personal use of this material is permitted.  Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, We study the suitability of keystroke dynamics to authenticate 100 K users typing free-text. For this, we first analyze to what extent our method based on a Siamese Recurrent Neural Network (RNN) is able to authenticate users when the amount of data per user is scarce, a common scenario in free-text keystroke authentication. With 1 K users for testing the network, a population size comparable to previous works, TypeNet obtains an equal error rate of 4.8% using only 5 enrollment sequences and 1 test sequence per user with 50 keystrokes per sequence. Using the same amount of data per user, as the number of test users is scaled up to 100K, the performance in comparison to 1 K decays relatively by less than 5%, demonstrating the potential of Type-Net to scale well at large scale number of users. Our experiments are conducted with the Aalto University keystroke database. To the best of our knowledge, this is the largest free-text keystroke database captured with more than 136M keystrokes from 168K users, This work has been supported by projects: PRIMA (MSCA-ITN-2019-860315), TRESPASS (MSCAITN-2019-860813), BIBECA (RTI2018-101248-B-I00 MINECO), and by edBB (UAM). A. Acien is supported by a FPI fellowship from the Spanish MINECO
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

An overview of privacy-enhancing technologies in biometric recognition

  • Melzi, Pietro
  • Rathgeb, Christian
  • Busch, Christoph
  • Tolosana Moranchel, Rubén
Privacy-enhancing technologies are technologies that implement fundamental data protection principles. With respect to biometric recognition, different types of privacy-enhancing technologies have been introduced for protecting stored biometric data, which are generally classified as sensitive. In this regard, various taxonomies and conceptual categorizations have been proposed and standardisation activities have been carried out. However, these efforts have mainly been devoted to certain sub-categories of privacy-enhancing technologies and therefore lack generalization. This work provides an overview of concepts of privacy-enhancing technologies for biometric recognition in a unified framework. Key properties and differences between existing concepts are highlighted in detail at each processing step. Fundamental characteristics and limitations of existing technologies are discussed and related to data protection techniques and principles. Moreover, scenarios and methods for the assessment of privacy-enhancing technologies for biometric recognition are presented. This article is meant as a point of entry to the field of data protection for biometric recognition applications and is directed toward experienced researchers as well as non-experts, This work has in part received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 860813 - TReSPAsS-ETN and the German Federal Ministry of Education and Research and the Hessen State Ministry for Higher Education, Research and the Arts within their joint support of the National Research Center for Applied Cybersecurity ATHENE. R. Tolosana and R. Vera-Rodriguez are also supported by INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER) and Cátedra ENIA UAM-VERIDAS en IA Responsable (NextGenerationEU PRTR
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

Prediction of atrial fibrillation from sinus-rhythm electrocardiograms based on deep neural networks: Analysis of time intervals and longitudinal study

  • Melzi, Pietro
  • Vera Rodríguez, Rubén
  • Sanz-García, Ancor
  • Cecconi, Alberto
  • Ortega, Guillermo J.
  • Jiménez-Borreguero, Luis Jesús
  • Tolosana Moranchel, Rubén
Objective: Artificial Intelligence (AI) in electrocardiogram (ECG) analysis helps to identify persons at risk of developing atrial fibrillation (AF) and reduces the risk for severe complications. Our aim is to investigate the performance of AI-based methods predicting future AF from sinus rhythm (SR) ECGs, according to different characteristics of patients, time intervals for prediction, and longitudinal measures. Methods: We designed a retrospective, prognostic study to predict AF occurrence in patients from 12-lead SR ECGs. We classified patients in two groups, according to their ECGs: 3,761 developed AF and 22,896 presented only SR ECGs. We assessed the impact of age on the overall performance of deep neural network (DNN)-based systems, which consist in a variation of Residual Networks for time series. Then, we analysed how much in advance our system can predict AF from SR ECGs and the performance for different categories of patients with AUC and other metrics. Results: After balancing the age distribution between the two groups of patients, our model achieves AUC of 0.79 (0.72-0.86) without additional constraints, 0.83 (0.76-0.89) for ECGs recorded in the last six months before AF, and 0.87 (0.81-0.93) for patients with stable AF risk measures over time, with sensitivity of 90.62% (80.70-96.48) and diagnostic odd ratio of 20.49 (8.56-49.09). Conclusion: This study shows the ability of DNNs to predict new onsets of AF from SR ECGs, with the best performance achieved for patients with stable AF risk score over time. The introduction of this time-based score opens new possibilities for AF prediction, thanks to the analysis of long-span time intervals and score stability, European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No860813 – TReSPAsS-ETNTRESPASS-ETN
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

OTB-morph: One-Time Biometrics via Morphing applied to Face Templates

  • Ghafourian, Mahdi
  • Serna Cabello, José Ignacio de la
  • Fiérrez Aguilar, Julián
  • Vera Rodríguez, Rubén
  • Morales Moreno, Aythami
© 2022 IEEE.  Personal use of this material is permitted.  Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, Cancelable biometrics refers to a group of techniques in which the biometric inputs are transformed intentionally using a key before processing or storage. This transformation is repeatable enabling subsequent biometric comparisons. This paper introduces a new scheme for cancelable biometrics aimed at protecting the templates against potential attacks, applicable to any biometric-based recognition system. Our proposed scheme is based on time-varying keys obtained from morphing random biometric information. An experimental implementation of the proposed scheme is given for face biometrics. The results confirm that the proposed approach is able to withstand against leakage attacks while improving the recognition performance, This work has been supported by projects: PRIMA (ITN-2019-860315), TRESPASS-ETN (ITN-2019-860813), and BIBECA (RTI2018-101248-B-I00 MINECO/FEDER). M.G. is supported by PRIMA and I.S. is supported by a FPI fellowship from Univ. Autonoma de Madrid.
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

FRCSyn-onGoing: Benchmarking and comprehensive evaluation of real and synthetic data to improve face recognition systems

  • Melzi, Pietro
  • Vera Rodríguez, Rubén
  • Andrés Tamé, Iván Ioel de
  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Ortega García, Javier
  • Tolosana Moranchel, Rubén
Artículo escrito por un elevado número de autores, solo se referencian el que aparece en primer lugar, el nombre del grupo de colaboración, si le hubiere, y los autores pertenecientes a la UAM, This article presents FRCSyn-onGoing, an ongoing challenge for face recognition where researchers can easily benchmark their systems against the state of the art in an open common platform using large-scale public databases and standard experimental protocols. FRCSyn-onGoing is based on the Face Recognition Challenge in the Era of Synthetic Data (FRCSyn) organized at WACV 2024. This is the first face recognition international challenge aiming to explore the use of real and synthetic data independently, and also their fusion, in order to address existing limitations in the technology. Specifically, FRCSyn-onGoing targets concerns related to data privacy issues, demographic biases, generalization to unseen scenarios, and performance limitations in challenging scenarios, including significant age disparities between enrollment and testing, pose variations, and occlusions. To enhance face recognition performance, FRCSyn-onGoing strongly advocates for information fusion at various levels, starting from the input data, where a mix of real and synthetic domains is proposed for specific tasks of the challenge. Additionally, participating teams are allowed to fuse diverse networks within their proposed systems to improve the performance. In this article, we provide a comprehensive evaluation of the face recognition systems and results achieved so far in FRCSyn-onGoing. The results obtained in FRCSynonGoing, together with the proposed public ongoing benchmark, contribute significantly to the application of synthetic data to improve face recognition technology, This study has received funding from the European Union’s Horizon 2020 TReSPAsS-ETN (No 860813) and is supported by INTER-ACTION, Spain (PID2021-126521OB-I00 MICINN/FEDER), Cátedra ENIA UAM-VERIDAS en IA Responsable (NextGeneration EU PRTR TSI-100927-2023-2), Spain R&D Agreement DGGC/UAM/FUAM for Biometrics and Cybersecurity, Spain, and Autonomous Community of Madrid (ELLIS Unit Madrid). MeVer was supported by the EU Horizon Europe project MAMMOth (Grant Agreement 101070285)