Skip to main content

Resultados relacionados

Resultados totales (Incluyendo duplicados): 35
Encontrada(s) 1 página(s)

Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Capítulo o parte de un libro (BookPart)

DeepFakes Detection Based on Heart Rate Estimation: Single- and Multi-frame

  • Hernández Ortega, Javier
  • Fiérrez Aguilar, Julián
  • Morales Moreno, Aythami
  • Tolosana Moranchel, Rubén
This chapter describes a DeepFake detection framework based on physiological measurement. In particular, we consider information related to the heart rate using remote photoplethysmography (rPPG). rPPG methods analyze video sequences looking for subtle color changes in the human skin, revealing the presence of human blood under the tissues. This chapter explores to what extent rPPG is useful for the detection of DeepFake videos. We analyze the recent fake detector named DeepFakesON-Phys that is based on a Convolutional Attention Network (CAN), which extracts spatial and temporal information from video frames, analyzing and combining both sources to better detect fake videos. DeepFakesON-Phys has been experimentally evaluated using the latest public databases in the field: Celeb-DF v2 and DFDC. The results achieved for DeepFake detection based on a single frame are over 98% AUC (Area Under the Curve) on both databases, proving the success of fake detectors based on physiological measurement to detect the latest DeepFake videos. In this chapter, we also propose and study heuristical and statistical approaches for performing continuous DeepFake detection by combining scores from consecutive frames with low latency and high accuracy (100% on the Celeb-DF v2 evaluation dataset). We show that combining scores extracted from short-time video sequences can improve the discrimination power of DeepFakesON-Phys, This work has been supported by projects: PRIMA (H2020-MSCA-ITN2019-860315), TRESPASS-ETN (H2020-MSCA-ITN-2019-860813), BIBECA (MINECO/FEDER RTI2018-101248-B-I00), and COST CA16101 (MULTI-FORESEE). J. H.-O. is supported by a PhD fellowship from UAM
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

SwipeFormer: Transformers for mobile touchscreen biometrics

  • Delgado Santos, Paula
  • Guest, Richard
  • Lamb, Parker
  • Khmelnitsky, Andrei
  • Coughlan, Colm
  • Fiérrez Aguilar, Julián
  • Tolosana Moranchel, Rubén
El autor de la UAM pertenecen al grupo Biometrics and Data Pattern Analytics Laboratory—BiDA Laboratory, The growing number of mobile devices over the past few years brings a large amount of personal information, which needs to be properly protected. As a result, several mobile authentication methods have been developed. In particular, behavioural biometrics has become one of the most relevant methods due to its ability to extract the uniqueness of each subject in a secure, non-intrusive, and continuous way. This article presents SwipeFormer, a novel Transformer-based system for mobile subject authentication by means of swipe gestures in an unconstrained scenario (i.e., subjects could use their personal devices freely, without restrictions on the direction of swipe gestures or the position of the device). Our proposed system contains two modules: (i) a Transformer-based feature extractor, and (ii) a similarity computation module. Mobile data from the touchscreen and different background sensors (accelerometer and gyroscope) have been studied, including in the analysis both Android and iOS operating systems. A complete analysis of SwipeFormer is carried out using an in-house large-scale database acquired in unconstrained scenarios. In these operational conditions, SwipeFormer achieves Equal Error Rate (EER) values of 6.6% and 3.6% on Android and iOS respectively, outperforming the state of the art. In addition, we evaluate SwipeFormer on the popular publicly available databases Frank DB and HuMIdb, achieving EER values of 11.0% and 5.0% respectively, outperforming previous approaches under the same experimental setup, This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 860315. With support also from projects INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER) and HumanCAIC (TED2021-131787B-I00 MICINN), and from the Autonomous Community of Madrid (ELLIS Unit Madrid)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

SVC-onGoing: Signature verification competition

  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Morales Moreno, Aythami
  • Ortega García, Javier
  • Tolosana Moranchel, Rubén
  • Gonzalez-Garcia, Carlos
  • Carlos Ruiz-Garcia, Juan
  • Romero Tapiador, Sergio
  • Rengifo, Santiago
  • Caruana, Miguel
  • Jiang, Jiajia
  • Lai, Songxuan
  • Jin, Lianwen
  • Zhu, Yecheng
  • Galbally, Javier
  • Diaz, Moises
  • Angel Ferrer, Miguel
  • Gomez-Barrero, Marta
  • Hodashinsky, Ilya
  • Sarin, Konstantin
  • Slezkin, Artem
  • Bardamova, Marina
  • Svetlakov, Mikhail
  • Saleem, Mohammad
  • Lia Szcs, Cintia
  • Kovari, Bence
  • Pulsmeyer, Falk
  • Wehbi, Mohamad
  • Zanca, Dario
  • Ahmad, Sumaiya
  • Mishra, Sarthak
  • Jabin, Suraiya
This article presents SVC-onGoing1, an on-going competition for on-line signature verification where researchers can easily benchmark their systems against the state of the art in an open common platform using large-scale public databases, such as DeepSignDB2 and SVC2021_EvalDB3, and standard experimental protocols. SVC-onGoing is based on the ICDAR 2021 Competition on On-Line Signature Verification (SVC 2021), which has been extended to allow participants anytime. The goal of SVC-onGoing is to evaluate the limits of on-line signature verification systems on popular scenarios (office/mobile) and writing inputs (stylus/finger) through large-scale public databases. Three different tasks are considered in the competition, simulating realistic scenarios as both random and skilled forgeries are simultaneously considered on each task. The results obtained in SVC-onGoing prove the high potential of deep learning methods in comparison with traditional methods. In particular, the best signature verification system has obtained Equal Error Rate (EER) values of 3.33% (Task 1), 7.41% (Task 2), and 6.04% (Task 3). Future studies in the field should be oriented to improve the performance of signature verification systems on the challenging mobile scenarios of SVC-onGoing in which several mobile devices and the finger are used during the signature acquisition, This work has been supported by projects: PRIMA (H2020- MSCA-ITN-2019-860315), TRESPASS-ETN (H2020-MSCA-ITN-2019- 860813), INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER), Orange Labs, and by UAM-Cecabank
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

SensitiveNets: Learning agnostic representations with application to face images

  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Vera Rodríguez, Rubén
  • Tolosana Moranchel, Rubén
© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, This work proposes a novel privacy-preserving neural network feature representation to suppress the sensitive information of a learned space while maintaining the utility of the data. The new international regulation for personal data protection forces data controllers to guarantee privacy and avoid discriminative hazards while managing sensitive data of users. In our approach, privacy and discrimination are related to each other. Instead of existing approaches aimed directly at fairness improvement, the proposed feature representation enforces the privacy of selected attributes. This way fairness is not the objective, but the result of a privacy-preserving learning method. This approach guarantees that sensitive information cannot be exploited by any agent who process the output of the model, ensuring both privacy and equality of opportunity. Our method is based on an adversarial regularizer that introduces a sensitive information removal function in the learning objective. The method is evaluated on three different primary tasks (identity, attractiveness, and smiling) and three publicly available benchmarks. In addition, we present a new face annotation dataset with balanced distribution between genders and ethnic origins. The experiments demonstrate that it is possible to improve the privacy and equality of opportunity while retaining competitive performance independently of the task, This work was supported by projects: PRIMA (MSCA-ITN-2019-860315), TRESPASS-ETN (MSCA-ITN-2019-860813), and BIBECA (RTI2018-101248-B-I00 MINECO)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

OTB-morph: one-time biometrics via morphing

  • Ghafourian, Mahdi
  • Fiérrez Aguilar, Julián
  • Vera Rodríguez, Rubén
  • Morales Moreno, Aythami
  • Serna Cabello, José Ignacio de la
Cancelable biometrics are a group of techniques to transform the input biometric to an irreversible feature intentionally using a transformation function and usually a key in order to provide security and privacy in biometric recognition systems. This transformation is repeatable enabling subsequent biometric comparisons. This paper introduces a new idea to be exploited as a transformation function for cancelable biometrics aimed at protecting templates against iterative optimization attacks. Our proposed scheme is based on time-varying keys (random biometrics in our case) and morphing transformations. An experimental implementation of the proposed scheme is given for face biometrics. The results confirm that the proposed approach is able to withstand leakage attacks while improving the recognition performance, This work has been supported by projects: PRIMA (H2020-MSCA-ITN-2019- 860315), TRESPASS-ETN (H2020-MSCA-ITN-2019-860813), and BBforTAI (PID2021-127641OB-I00 MICINN/FEDER). M.G. is supported by PRIMA and I.S. is supported by an FPI fellowship from Univ. Autonoma de Madrid
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

SetMargin loss applied to deep keystroke biometrics with circle packing interpretation

  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Acién Ayala, Alejandro
  • Serna Cabello, José Ignacio de la
  • Tolosana Moranchel, Rubén
This work presents a new deep learning approach for keystroke biometrics based on a novel Distance Metric Learning method (DML). DML maps input data into a learned representation space that reveals a “semantic” structure based on distances. In this work, we propose a novel DML method specifically designed to address the challenges associated to free-text keystroke identification where the classes used in learning and inference are disjoint. The proposed SetMargin Loss (SM-L) extends traditional DML approaches with a learning process guided by pairs of sets instead of pairs of samples, as done traditionally. The proposed learning strategy allows to enlarge inter-class distances while maintaining the intra-class structure of keystroke dynamics. We analyze the resulting representation space using the mathematical problem known as Circle Packing, which provides neighbourhood structures with a theoretical maximum inter-class distance. We finally prove experimentally the effectiveness of the proposed approach on a challenging task: keystroke biometric identification over a large set of 78,000 subjects. Our method achieves state-of-the-art accuracy on a comparison performed with the best existing approaches, This work has been supported by projects: PRIMA ( MSCA-ITN- 2019-860315 ), TRESPASS-ETN (MSCA-ITN-2019-860813), BIBECA (RTI2018-101248-B-I00 MINECO), edBB (UAM), and Instituto de In- genieria del Conocimiento (IIC). A. Acien is supported by a FPI fel- lowship from the Spanish MINECO
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

Keystroke Biometrics in Response to Fake News Propagation in a Global Pandemic

  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Vera Rodríguez, Rubén
  • Ortega García, Javier
  • Tolosana Moranchel, Rubén
  • Acién Ayala, Alejandro
  • Monaco, John V.
© 2020 IEEE.  Personal use of this material is permitted.  Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, This work proposes and analyzes the use of keystroke biometrics for content de-anonymization. Fake news have become a powerful tool to manipulate public opinion, especially during major events. In particular, the massive spread of fake news during the COVID-19 pandemic has forced governments and companies to fight against missinformation. In this context, the ability to link multiple accounts or profiles that spread such malicious content on the Internet while hiding in anonymity would enable proactive identification and blacklisting. Behavioral biometrics can be powerful tools in this fight. In this work, we have analyzed how the latest advances in keystroke biometric recognition can help to link behavioral typing patterns in experiments involving 100,000 users and more than 1 million typed sequences. Our proposed system is based on Recurrent Neural Networks adapted to the context of content de-anonymization. Assuming the challenge to link the typed content of a target user in a pool of candidate profiles, our results show that keystroke recognition can be used to reduce the list of candidate profiles by more than 90%. In addition, when keystroke is combined with auxiliary data (such as location), our system achieves a Rank-1 identification performance equal to 52.6% and 10.9% for a background candidate list composed of 1K and 100K profiles, respectively, This work has been supported by projects: PRIMA (MSCA-ITN-2019-860315), TRESPASS (MSCA-ITN-2019-860813), BIBECA (RTI2018-101248-B-I00 MINECO), BioGuard (Ayudas Fundacion BBVA a Equipos de Investigacion Cientıfica 2017). A. Acien and R. Tolosana are supported by a FPI and postdoc fellowship from the Spanish MINECO
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

MEBAL: A multimodal database for eye blink detection and attention level estimation

  • Daza García, Roberto
  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Tolosana Moranchel, Rubén
© ACM 2020. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ICMI '20 Companion, October 25–29, 2020, Virtual Event, Netherlands, https://doi.org/10.1145/3395035.3425257, This work presents mEBAL, a multimodal database for eye blink detection and attention level estimation. The eye blink frequency is related to the cognitive activity and automatic detectors of eye blinks have been proposed for many tasks including attention level estimation, analysis of neuro-degenerative diseases, deception recognition, drive fatigue detection, or face anti-spoofing. However, most existing databases and algorithms in this area are limited to experiments involving only a few hundred samples and individual sensors like face cameras. The proposed mEBAL improves previous databases in terms of acquisition sensors and samples. In particular, three different sensors are simultaneously considered: Near Infrared (NIR) and RGB cameras to capture the face gestures and an Electroencephalography (EEG) band to capture the cognitive activity of the user and blinking events. Regarding the size of mEBAL, it comprises 6,000 samples and the corresponding attention level from 38 different students while conducting a number of e-learning tasks of varying difficulty. In addition to presenting mEBAL, we also include preliminary experiments on: i) eye blink detection using Convolutional Neural Networks (CNN) with the facial images, and ii) attention level estimation of the students based on their eye blink frequency, This work has been supported by projects: PRIMA (ITN-2019-860315), TRESPASS-ETN (ITN-2019-860813), IDEA-FAST (IMI2- 2018-15-two-stage-853981), BIBECA (RTI2018-101248-B-I00 MINECOFEDER), and edBB (Universidad Autonoma de Madrid). Ruben Tolosana and postdoc support from CAM/FEDER. Roberto Daza is supported by a PhD FPI fellowship from MINECO-FEDER
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

SaFL: Sybil-aware Federated Learning with application to face recognition

  • Ghafourian, Mahdi
  • Fiérrez Aguilar, Julián
  • Vera Rodríguez, Rubén
  • Morales Moreno, Aythami
  • Tolosana Moranchel, Rubén
© 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, Federated Learning (FL) is a machine learning paradigm to conduct collaborative learning among clients on a joint model. The primary goal is to share clients' local training parameters with an integrating server while preserving their privacy. This method permits to exploit the potential of massive mobile users' data for the benefit of machine learning models' performance while keeping sensitive data on local devices. On the downside, FL raises security and privacy concerns that have just started to be studied. To address some of the key threats in FL, researchers have proposed to use secure aggregation methods (e.g. homomorphic encryption, secure multiparty computation, etc.). These solutions improve some security and privacy metrics, but at the same time bring about other serious threats such as poisoning attacks, backdoor attacks, and free running attacks. This paper proposes a new defense method against poisoning attacks in FL called SaFL (Sybil-Aware Federated Learning) that minimizes the effect of sybils with a novel time-variant aggregation scheme, This work has been supported by projects: PRIMA (ITN-2019-860315), TRESPASS-ETN (ITN-2019-860813) and Cátedra ENIA UAM-VERIDAS en IA Responsable (NextGenerationEU PRTR TSI-100927-2023-2)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

BehavePassDB: Public Database for Mobile Behavioral Biometrics and Benchmark Evaluation

  • Stragapede, Giuseppe
  • Vera Rodríguez, Rubén
  • Morales Moreno, Aythami
  • Tolosana Moranchel, Rubén
Mobile behavioral biometrics have become a popular topic of research, reaching promising results in terms of authentication, exploiting a multimodal combination of touchscreen and background sensor data. However, there is no way of knowing whether state-of-the-art classifiers in the literature can distinguish between the notion of user and device. In this article, we present a new database, BehavePassDB, structured into separate acquisition sessions and tasks to mimic the most common aspects of mobile Human-Computer Interaction (HCI). BehavePassDB is acquired through a dedicated mobile app installed on the subjects devices, also including the case of different users on the same device for evaluation. We propose a standard experimental protocol and benchmark for the research community to perform a fair comparison of novel approaches with the state of the art1. We propose and evaluate a system based on Long-Short Term Memory (LSTM) architecture with triplet loss and modality fusion at score level, This project has received funding from the European Unions Horizon 2020 research and innovation programme under the Marie Skodowska-Curie grant agreement no. 860315, and from Orange Labs. R. Tolosana and R. Vera-Rodriguez are also supported by INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

GANprintR: Improved Fakes and Evaluation of the State of the Art in Face Manipulation Detection

  • Neves, João C.
  • Lopes, Vasco
  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Tolosana Moranchel, Rubén
  • Proença, Hugo
© 2020 IEEE.  Personal use of this material is permitted.  Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, The availability of large-scale facial databases, together with the remarkable progresses of deep learning technologies, in particular Generative Adversarial Networks (GANs), have led to the generation of extremely realistic fake facial content, raising obvious concerns about the potential for misuse. Such concerns have fostered the research on manipulation detection methods that, contrary to humans, have already achieved astonishing results in various scenarios. In this study, we focus on the synthesis of entire facial images, which is a specific type of facial manipulation. The main contributions of this study are four-fold: i) a novel strategy to remove GAN 'fingerprints' from synthetic fake images based on autoencoders is described, in order to spoof facial manipulation detection systems while keeping the visual quality of the resulting images; ii) an in-depth analysis of the recent literature in facial manipulation detection; iii) a complete experimental assessment of this type of facial manipulation, considering the state-of-the-art fake detection systems (based on holistic deep networks, steganalysis, and local artifacts), remarking how challenging is this task in unconstrained scenarios; and finally iv) we announce a novel public database, named iFakeFaceDB, yielding from the application of our proposed GAN-fingerprint Removal approach (GANprintR) to already very realistic synthetic fake images. The results obtained in our empirical evaluation show that additional efforts are required to develop robust facial manipulation detection systems against unseen conditions and spoof techniques, such as the one proposed in this study, This work has been supported by projects: PRIMA (H2020-MSCA-ITN-2019-860315), TRESPASS-ETN (H2020-MSCA-ITN2019-860813), BIBECA (RTI2018-101248-B-I00 MINECO/FEDER), BioGuard (Ayudas Fundación BBVA a Equipos de Investigación Cientíifica 2017), Accenture, by NOVA LINCS (UIDB/04516/2020) with the financial support of FCT - Fundação para a Ciência e a Tecnologia, through national funds, and by FCT/MCTES through national funds and co-funded by EU under the project UIDB/EEA/50008/2020
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

Mobile passive authentication through touchscreen and background sensor data

  • Stragapede, Giuseppe
  • Acién Ayala, Alejandro
  • Vera Rodríguez, Rubén
  • Morales Moreno, Aythami
  • Tolosana Moranchel, Rubén
  • Lan, Gael Le
The security and usability shortcomings of current mobile user authentication systems based on PIN codes, fingerprint, and face recognition are well known. To overcome such limitations, the present work focuses on the comparative analysis of unimodal and multimodal behavioral biometric traits suitable for mobile passive authentication, such as touchscreen data during separate gestures (keystroke, scrolling, drawing a number, tapping on the screen), and background sensor data (accelerometer, gravity sensor, gyroscope, linear accelerometer, magnetometer).This paper carries out a performance evaluation over one of the most complete and challenging databases to date with mobile user interaction data, HuMIdb, with 600 subjects. For each individual modality, we propose a separate RNN (Recurrent Neural Network) trained with semi-hard triplet loss. In addition, we perform the fusion of the different modalities at score level. Our results show that the best performing tasks are keystroke and drawing a number, whereas the most discriminative background sensor is the magnetometer. Additionally, the fusion of modalities is very beneficial, consistently reducing the Equal Error Rates (EER) by half (ranging from 5% to 13% depending on the modality combination), This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 860315, and from Orange Labs. R. Tolosana and R. VeraRodriguez are also supported by INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

Deepfakes and beyond: A Survey of face manipulation and fake detection

  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Morales Moreno, Aythami
  • Ortega García, Javier
  • Tolosana Moranchel, Rubén
The free access to large-scale public databases, together with the fast progress of deep learning techniques, in particular Generative Adversarial Networks, have led to the generation of very realistic fake content with its corresponding implications towards society in this era of fake news. This survey provides a thorough review of techniques for manipulating face images including DeepFake methods, and methods to detect such manipulations. In particular, four types of facial manipulation are reviewed: i) entire face synthesis, ii) identity swap (DeepFakes), iii) attribute manipulation, and iv) expression swap. For each manipulation group, we provide details regarding manipulation techniques, existing public databases, and key benchmarks for technology evaluation of fake detection methods, including a summary of results from those evaluations. Among all the aspects discussed in the survey, we pay special attention to the latest generation of DeepFakes, highlighting its improvements and challenges for fake detection. In addition to the survey information, we also discuss open issues and future trends that should be considered to advance in the field, This work has been supported by projects: PRIMA (H2020-MSCAITN-2019-860315), TRESPASS-ETN (H2020-MSCA-ITN-2019-860813), BIBECA (MINECO/FEDER RTI2018-101248-B-I00), Bio-Guard (Ayudas Fundación BBVA a Equipos de Investigación Científica 2017), and Accenture. Ruben Tolosana is supported by Consejería de Educación, Juventud y Deporte de la Comunidad de Madrid y Fondo Social Europeo
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

A Comparative Evaluation of Heart Rate Estimation Methods using Face Videos

  • Hernández Ortega, Javier
  • Diaz, David
  • Fiérrez Aguilar, Julián
  • Morales Moreno, Aythami
© 2020 IEEE.  Personal use of this material is permitted.  Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, This paper presents a comparative evaluation of methods for remote heart rate estimation using face videos, i.e., given a video sequence of the face as input, methods to process it to obtain a robust estimation of the subject's heart rate at each moment. Four alternatives from the literature are tested, three based in hand-crafted approaches and one based on deep learning. The methods are compared using RGB videos from the COHFACE database. Experiments show that the learning-based method achieves much better accuracy than the hand-crafted ones. The low error rate achieved by the learning-based model makes possible its application in real scenarios, e.g. in medical or sports environments, This work has been supported by projects: IDEAFAST (IMI2-2018-15-two-stage-853981), PRIMA (ITN-2019-860315), TRESPASS-ETN (ITN-2019-860813), BIBECA (RTI2018- 101248-B-I00 MINECO/FEDER), and edBB (UAM). J.H.O. is supported by a PhD Scholarship from UAM. Portions of the research in this paper used the COHFACE Dataset made available by the Idiap Research Institute, Martigny, Switzerland
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

Assessing the quality of swipe interactions for mobile biometric systems

  • Santopietro, Marco
  • Guest, Richard
  • Acién Ayala, Alejandro
  • Vera Rodríguez, Rubén
  • Morales Moreno, Aythami
© 2020 IEEE.  Personal use of this material is permitted.  Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, Quality estimation is a key study in biometrics, allowing optimisation and improvement of existing authentication systems by giving a prediction on the model performance based on the goodness of the sample or the user. In this paper, we propose a quality metric for swipe gestures on mobile devices. We evaluate a quality score for subjects on enrollment and for swipe samples, we estimate three quality groups and explore the correlation between our quality score and a state-of-art biometric authentication classifier performance. A further analysis based on the combined effects of subject quality and the amount of enrollment samples is conducted, investigating if increasing or decreasing enrollment size affects the authentication performance for different quality groups. Results are shown for three different public datasets, highlighting how higher quality users score a lower equal error rate compared to medium and low quality users, while high quality samples get a higher similarity score from the classifier, Thanks to support by projects: PRIMA (H2020-MSCAITN-2019-860315), TRESPASS-ETN (H2020-MSCAITN-2019-860813), BIBECA (RTI2018-101248-B-I00 MINECO/FEDER), and BioGuard (Ayudas Fundacion BBVA a Equipos de Investigacion Científica 2017)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

DeepSign: Deep On-Line Signature Verification

  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Ortega García, Javier
  • Tolosana Moranchel, Rubén
Deep learning has become a breathtaking technology in the last years, overcoming traditional handcrafted approaches and even humans for many different tasks. However, in some tasks, such as the verification of handwritten signatures, the amount of publicly available data is scarce, what makes difficult to test the real limits of deep learning. In addition to the lack of public data, it is not easy to evaluate the improvements of novel proposed approaches as different databases and experimental protocols are usually considered. The main contributions of this study are: i) we provide an in-depth analysis of state-of-the-art deep learning approaches for on-line signature verification, ii) we present and describe the new DeepSignDB on-line handwritten signature biometric public database,1 iii) we propose a standard experimental protocol and benchmark to be used for the research community in order to perform a fair comparison of novel approaches with the state of the art, and iv) we adapt and evaluate our recent deep learning approach named Time-Aligned Recurrent Neural Networks (TA-RNNs)2. for the task of on-line handwritten signature verification. This approach combines the potential of Dynamic Time Warping and Recurrent Neural Networks to train more robust systems against forgeries. Our proposed TA-RNN system outperforms the state of the art, achieving results even below 2.0% EER when considering skilled forgery impostors and just one training signature per user.1https://github.com/BiDAlab/DeepSignDB 2Spanish Patent Application (P202030060), This work was supported in part by PRIMA under Grant H2020-MSCA-ITN-2019-860315; in part by TRESPASS-ETN under Grant H2020-MSCA-ITN-2019-860813; in part by BIBECA under Grant RTI2018-101248-B-I00 MINECO/FEDER; in part by Bio-Guard (Ayudas Fundación BBVA a Equipos de Investigación Científica 2017); and in part by UAM-Cecabank. The work of Ruben Tolosana was supported by the Comunidad de Madrid and Fondo Social Europeo. Spanish Patent Application (P202030060)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

GaitPrivacyON: Privacy-preserving mobile gait biometrics using unsupervised learning

  • Delgado Santos, Paula
  • Guest, Richard
  • Deravi, Farzin
  • Morales Moreno, Aythami
  • Vera Rodríguez, Rubén
  • Tolosana Moranchel, Rubén
Numerous studies in the literature have already shown the potential of biometrics on mobile devices for authentication purposes. However, it has been shown that, the learning processes associated to biometric systems might expose sensitive personal information about the subjects. This study proposes GaitPrivacyON, a novel mobile gait biometrics verification approach that provides accurate authentication results while preserving the sensitive information of the subject. It comprises two modules: i) two convolutional Autoencoders with shared weights that transform attributes of the biometric raw data, such as the gender or the activity being performed, into a new privacy-preserving representation; and ii) a mobile gait verification system based on the combination of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) with a Siamese architecture. The main advantage of GaitPrivacyON is that the first module (convolutional Autoencoders) is trained in an unsupervised way, without specifying the sensitive attributes of the subject to protect. Two experimental studies have been examinated: i) MotionSense and MobiAct databases; and ii) OU-ISIR database. The experimental results achieved suggest the potential of GaitPrivacyON to significantly improve the privacy of the subject while keeping user authentication results higher than 96.6% Area Under the Curve (AUC). To the best of our knowledge, this is the first mobile gait verification approach that considers privacy-preserving methods trained in an unsupervised way, This project has received funding from the European Unions Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement no 860315. R. Tolosana and R. Vera-Rodriguez are also supported by INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER )
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

DeepFakes detection across generations: Analysis of facial regions, fusion, and performance evaluation

  • Romero Tapiador, Sergio
  • Gonzalez Sosa
  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Tolosana Moranchel, Rubén
Media forensics has attracted a tremendous attention in the last years in part due to the increasing concerns around DeepFakes. Since the release of the initial DeepFakes databases of the 1st generation such as UADFV and FaceForensics++ up to the latest databases of the 2nd generation such as Celeb-DF and DFDC, many visual improvements have been carried out, making fake videos almost indistinguishable to the human eye. This study provides an in-depth analysis of both 1st and 2nd DeepFakes generations in terms of fake detection performance. Two different methods are considered in our experimental framework: (i) the traditional one followed in the literature based on selecting the entire face as input to the fake detection system, and (ii) a novel approach based on the selection of specific facial regions as input to the fake detection system. Fusion techniques are applied both to the facial regions and also to three different state-of-the-art fake detection systems (Xception, Capsule Network, and DSP-FWA) in order to further increase the robustness of the detectors considered. Finally, experiments regarding intra- and inter-database scenarios are performed. Among all the findings resulting from our experiments, we highlight: (i) the very good results achieved using facial regions and fusion techniques with fake detection results above 99% Area Under the Curve (AUC) for UADFV, FaceForensics++, and Celeb-DF v2 databases, and (ii) the necessity to put more efforts on the analysis of inter-database scenarios to improve the ability of the fake detectors against attacks unseen during learning, This work has been supported by projects: PRIMA (H2020-MSCAITN-2019-860315), TRESPASS-ETN (H2020-MSCA-ITN-2019-860813), BIBECA, Spain (MINECO/FEDER RTI2018-101248-B-I00)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

PAD-Phys: Exploiting physiology for presentation attack detection in face biometrics

  • Gómez, Luis F.
  • Solano, Imanol
  • Garcia, Alejandro
  • Zamora-Martinez, Francisco
  • Fiérrez Aguilar, Julián
  • Morales Moreno, Aythami
  • Tolosana Moranchel, Rubén
  • Ghafourian, Mahdi
Presentation Attack Detection (PAD) is a crucial stage in facial recognition systems to avoid leakage of personal information or spoofing of identity to entities. Recently, pulse detection based on remote photoplethysmography (rPPG) has been shown to be effective in face presentation attack detection. This work presents three different approaches to the presentation attack detection based on rPPG: (i) The physiological domain, a domain using rPPG-based models, (ii) the Deepfakes domain, a domain where models were retrained from the physiological domain to specific Deepfakes detection tasks; and (iii) a new Presentation Attack domain was trained by applying transfer learning from the two previous domains to improve the capability to differentiate between bona-fides and attacks. The results show the efficiency of the rPPG-based models for presentation attack detection, evidencing a 21.70% decrease in average classification error rate (ACER) (from 41.03% to 19.32%) when the presentation attack domain is compared to the physiological and Deepfakes domains. Our experiments highlight the efficiency of transfer learning in rPPG-based models and perform well in presentation attack detection in instruments that do not allow copying of this physiological feature, This work has been supported by Veridas and by projects: TRESPASS-ETN (ITN-2019-860813), PRIMA (ITN2019-860315), and BBforTAI (PID2021-127641OB-I00 MICINN/FEDER)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

IRL-Net: inpainted region localization network via spatial attention

  • Daryani, Amir Etefaghi
  • Mirmahdi, Mahdieh
  • Hassanpour, Ahmad
  • Shahreza, Hatef Otroshi
  • Yang, Bian
  • Fiérrez Aguilar, Julián
Identifying manipulated regions in images is a challenging task due to the existence of very accurate image inpainting techniques leaving almost unnoticeable traces in tampered regions. These image inpainting methods can be used for multiple purposes (e.g., removing objects, reconstructing corrupted areas, eliminating various types of distortion, etc.) makes creating forensic detectors for image manipulation an extremely difficult and time-consuming procedure. The aim of this paper is to localize the tampered regions manipulated by image inpainting methods. To do this, we propose a novel CNN-based deep learning model called IRL-Net which includes three main modules: Enhancement, Encoder, and Decoder modules. To evaluate our method, three image inpainting methods have been used to reconstruct the missed regions in two face and scene image datasets. We perform both qualitative and quantitative evaluations on the generated datasets. Experimental results demonstrate that our method outperforms previous learning-based manipulated region detection methods and generates realistic and semantically plausible images. We also provide the implementation of the proposed approach to support reproducible research via https://github.com/amiretefaghi/IRL-Net, This work was supported by the Project Privacy Matters (PRIMA) under Grant H2020-MSCA-ITN-2019-860315. The work of Hatef Otroshi Shahreza was supported by the H2020 TReSPAsS-ETN Marie Skłodowska-Curie Early Training Network under Grant 860813. The work of Julian Fierrez was supported by the Project Biometrics and Behavior for Unbiased and Trusted AI with Applications (BBforTAI) under Grant PID2021-127641OB-I00 MICINN/FEDER
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

Exploring transformers for behavioural biometrics: a case study in gait recognition

  • Delgado Santos, Paula
  • Guest, Richard
  • Deravi, Farzin
  • Vera Rodríguez, Rubén
  • Tolosana Moranchel, Rubén
Biometrics on mobile devices has attracted a lot of attention in recent years as it is considered a user-friendly authentication method. This interest has also been motivated by the success of Deep Learning (DL). Architectures based on Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) have established convenience for the task, improving the performance and robustness in comparison to traditional machine learning techniques. However, some aspects must still be revisited and improved. To the best of our knowledge, this is the first article that explores and proposes a novel gait biometric recognition systems based on Transformers, which currently obtain state-of-the-art performance in many applications. Several state-of-the-art architectures (Vanilla, Informer, Autoformer, Block-Recurrent Transformer, and THAT) are considered in the experimental framework. In addition, new Transformer configurations are proposed to further increase the performance. Experiments are carried out using the two popular public databases: whuGAIT and OU-ISIR. The results achieved prove the high ability of the proposed Transformer, outperforming state-of-the-art CNN and RNN architectures, This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement no. 860315. With support also from projects INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER) and HumanCAIC (TED2021-131787B-I00 MICINN), and from Comunidad de Madrid (ELLIS Unit Madrid)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

Learning emotional-blinded face representations

  • Peña Almansa, Alejandro
  • Lapedriza, Agata
  • Fiérrez Aguilar, Julián
  • Morales Moreno, Aythami
© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, We propose two face representations that are blind to facial expressions associated to emotional responses. This work is in part motivated by new international regulations for personal data protection, which enforce data controllers to protect any kind of sensitive information involved in automatic processes. The advances in Affective Computing have contributed to improve human-machine interfaces but, at the same time, the capacity to monitorize emotional responses triggers potential risks for humans, both in terms of fairness and privacy. We propose two different methods to learn these expression-blinded facial features. We show that it is possible to eliminate information related to emotion recognition tasks, while the performance of subject verification, gender recognition, and ethnicity classification are just slightly affected. We also present an application to train fairer classifiers in a case study of attractiveness classification with respect to a protected facial expression attribute. The results demonstrate that it is possible to reduce emotional information in the face representation while retaining competitive performance in other face-based artificial intelligence tasks, This work has been supported by projects: PRIMA (H2020-MSCA-ITN-2019-860315), TRESPASS-ETN (H2020-MSCAITN-2019-860813), IDEA-FAST (IMI2-2018-15-853981), BIBECA (RTI2018-101248-B-I00 MINECO/FEDER), REAVIPERO (RED2018-102511-T), RTI2018-095232-B-C22 MINECO, and Accenture. A. Pena is supported by a research fellowship (PEJ2018-004094A) from the Spanish MINECO
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

TypeFormer: Transformers for mobile keystroke biometrics

  • Stragapede, Giuseppe
  • Delgado Santos, Paula
  • Vera Rodríguez, Rubén
  • Guest, Richard
  • Morales Moreno, Aythami
  • Tolosana Moranchel, Rubén
The broad usage of mobile devices nowadays, the sensitiveness of the information contained in them, and the shortcomings of current mobile user authentication methods are calling for novel, secure, and unobtrusive solutions to verify the users’ identity. In this article, we propose TypeFormer, a novel transformer architecture to model free-text keystroke dynamics performed on mobile devices for the purpose of user authentication. The proposed model consists in temporal and channel modules enclosing two long short-term memory recurrent layers, Gaussian range encoding, a multi-head self-attention mechanism, and a block-recurrent transformer layer. Experimenting on one of the largest public databases to date, the Aalto mobile keystroke database, TypeFormer outperforms current state-of-the-art systems achieving equal error rate values of 3.25% using only five enrolment sessions of 50 keystrokes each. In such way, we contribute to reducing the traditional performance gap of the challenging mobile free-text scenario with respect to its desktop and fixed-text counterparts. To highlight the design rationale, an analysis of the experimental results of the different modules implemented in the development of TypeFormer is carried out. Additionally, we analyse the behaviour of the model with different experimental configurations such as the length of the keystroke sequences and the amount of enrolment sessions, showing margin for improvement, This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 860315. Moreover, it has been supported by INTER-ACTION (PID2021126521OB-I00 MICINN/FEDER) and Cátedra ENIA UAM-VERIDAS en IA Responsable (NextGenerationEU PRTR TSI-1009272023-2)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

Symbolic AI for XAI: Evaluating LFIT inductive programming for explaining biases in machine learning

  • Ortega de la Puente, Alfonso
  • Wang, Zilong
  • de la Cruz, Marina
  • Alonso, César Luis
  • Ribeiro, Tony
  • Fiérrez Aguilar, Julián
  • Morales Moreno, Aythami
Machine learning methods are growing in relevance for biometrics and personal information processing in domains such as forensics, e-health, recruitment, and e-learning. In these domains, white-box (human-readable) explanations of systems built on machine learning methods become crucial. Inductive logic programming (ILP) is a subfield of symbolic AI aimed to automatically learn declarative theories about the processing of data. Learning from interpretation transition (LFIT) is an ILP technique that can learn a propositional logic theory equivalent to a given black-box system (under certain conditions). The present work takes a first step to a general methodology to incorporate accurate declarative explanations to classic machine learning by checking the viability of LFIT in a specific AI application scenario: fair recruitment based on an automatic tool generated with machine learning methods for ranking Curricula Vitae that incorporates soft biometric information (gender and ethnicity). We show the expressiveness of LFIT for this specific problem and propose a scheme that can be applicable to other domains. In order to check the ability to cope with other domains no matter the machine learning paradigm used, we have done a preliminary test of the expressiveness of LFIT, feeding it with a real dataset about adult incomes taken from the US census, in which we consider the income level as a function of the rest of attributes to verify if LFIT can provide logical theory to support and explain to what extent higher incomes are biased by gender and ethnicity, This work was supported by projects: PRIMA (H2020-MSCA-ITN-2019-860315), TRESPASS-ETN(H2020-MSCA-ITN-2019-860813), IDEA-FAST (IMI2-2018-15-853981), BIBECA(RTI2018-101248-B-I00MINECO/FEDER), RTI2018-095232-B-C22MINECO, PLeNTaS project PID2019-111430RBI00MINECO; and also by Pays de la Loire Region through RFI Atlanstic 2020
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

M-GaitFormer: Mobile biometric gait verification using Transformers

  • Delgado Santos, Paula
  • Guest, Richard
  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Tolosana Moranchel, Rubén
Mobile devices such as smartphones and smartwatches are part of our everyday life, acquiring large amount of personal information that needs to be properly secured. Among the different authentication techniques, behavioural biometrics has become a very popular method as it allows authentication in a non-intrusive and continuous way. This study proposes M-GaitFormer, a novel mobile biometric gait verification system based on Transformer architectures. This biometric system only considers the accelerometer and gyroscope data acquired by the mobile device. A complete analysis of the proposed M-GaitFormer is carried out using the popular available databases whuGAIT and OU-ISIR. M-GaitFormer achieves Equal Error Rate (EER) values of 3.42% and 2.90% on whuGAIT and OU-ISIR, respectively, outperforming other state-of-the-art approaches based on popular Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 860315. With support also from projects INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER), HumanCAIC (TED2021-131787B-I00 MICINN), and Comunidad de Madrid (ELLIS Unit Madrid)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

E2F-Net: Eyes-to-face inpainting via StyleGAN latent space

  • Hassanpour, Ahmad
  • Jamalbafrani, Fatemeh
  • Yang, Bian
  • Raja, Kiran
  • Veldhuis, Raymond
  • Fiérrez Aguilar, Julián
Face inpainting, the technique of restoring missing or damaged regions in facial images, is pivotal for applications like face recognition in occluded scenarios and image analysis with poor-quality captures. This process not only needs to produce realistic visuals but also preserve individual identity characteristics. The aim of this paper is to inpaint a face given periocular region (eyes-to-face) through a proposed new Generative Adversarial Network (GAN)-based model called Eyes-to-Face Network (E2F-Net). The proposed approach extracts identity and non-identity features from the periocular region using two dedicated encoders have been used. The extracted features are then mapped to the latent space of a pre-trained StyleGAN generator to benefit from its state-of-the-art performance and its rich, diverse and expressive latent space without any additional training. We further improve the StyleGAN's output to find the optimal code in the latent space using a new optimization for GAN inversion technique. Our E2F-Net requires a minimum training process reducing the computational complexity as a secondary benefit. Through extensive experiments, we show that our method successfully reconstructs the whole face with high quality, surpassing current techniques, despite significantly less training and supervision efforts. We have generated seven eyes-to-face datasets based on well-known public face datasets for training and verifying our proposed methods. The code and datasets are publicly available, This work was supported by the Project Privacy Matters (PRIMA) under Grant H2020-MSCA-ITN-2019-860315. J.F. is funded by the ENIA Chair UAM-Veridas in Responsible AI (NextGenerationEU PRTR TSI-100927-2023-2) and project BBforTAI (PID2021-1276 41OB-I00 MICINN/FEDER)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

BeCAPTCHA: Behavioral bot detection using touchscreen and mobile sensors benchmarked on HuMIdb

  • Acién Ayala, Alejandro
  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Vera Rodríguez, Rubén
  • Delgado Ben Mohatar, Óscar
In this paper we study the suitability of a new generation of CAPTCHA methods based on smartphone interactions. The heterogeneous flow of data generated during the interaction with the smartphones can be used to model human behavior when interacting with the technology and improve bot detection algorithms. For this, we propose BeCAPTCHA, a CAPTCHA method based on the analysis of the touchscreen information obtained during a single drag and drop task in combination with the accelerometer data. The goal of BeCAPTCHA is to determine whether the drag and drop task was realized by a human or a bot. We evaluate the method by generating fake samples synthesized with Generative Adversarial Neural Networks and handcrafted methods. Our results suggest the potential of mobile sensors to characterize the human behavior and develop a new generation of CAPTCHAs. The experiments are evaluated with HuMIdb1 (Human Mobile Interaction database), a novel multimodal mobile database that comprises 14 mobile sensors acquired from 600 users. HuMIdb is freely available to the research community, This work has been supported by projects: PRIMA, Spain (H2020-MSCA-ITN-2019-860315), TRESPASS-ETN, Spain (H2020-MSCA-ITN-2019-860813), BIBECA RTI2018-101248-B-I00 (MINECO/FEDER), and BioGuard, Spain (Ayudas Fundación BBVA a Equipos de Investigación Científica 2017). Spanish Patent Application P202030066
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

Blockchain in the Internet of Things: Architectures and Implementation

  • Delgado Ben Mohatar, Óscar
  • Fiérrez Aguilar, Julián
  • Morales Moreno, Aythami
  • Tolosana Moranchel, Rubén
© 2020 IEEE.  Personal use of this material is permitted.  Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., IEEE Annual International Computer Software and Applications Conference (COMPSAC) celebrado en 2020 en Madrid, The world is becoming more interconnected every day. With the high technological evolution and the increasing deployment of it in our society, scenarios based on the Internet of Things (IoT) can be considered a reality nowadays. However, and before some predictions become true (around 75 billion devices are expected to be interconnected in the next few years), many efforts must be carried out in terms of scalability and security. In this study we propose and evaluate a new approach based on the incorporation of Blockchain into current IoT scenarios. The main contributions of this study are as follows: i) an in depth analysis of the different possibilities for the integration of Blockchain into IoT scenarios, focusing on the limited processing capabilities and storage space of most IoT devices, and the eco nomic cost and performance of current Blockchain technologies; ii) a new method based on a novel module named BIoT Gateway that allows both unidirectional and bidirectional communications with IoT devices on real scenarios, allowing to exchange any kind of data; and iii) the proposed method has been fully implemented and validated on two different real-life IoT scenarios, extracting very interesting findings in terms of economic cost and execution time. The source code of our implementation is publicly available in the Ethereum testnet., This work has been supported by projects: PRIMA (H2020-MSCA-ITN-2019-860315), TRESPASS-ETN (H2020-MSCA-ITN-2019-860813), BIBECA (RTI2018- 101248-B-I00 MINECO/FEDER), and COPCIS (TIN2017- 84844-C2-1-R MINECO/FEDER). Ruben Tolosana is supported by Consejer´ıa de Educacion, Juventud y Deporte ´ de la Comunidad de Madrid y Fondo Social Europeo.
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Reseña (Review)

ECG biometric recognition: Review, system proposal, and benchmark evaluation

  • Melzi, Pietro
  • Vera Rodríguez, Rubén
  • Tolosana Moranchel, Rubén
ECGs have shown unique patterns to distinguish between different subjects and present important advantages compared to other biometric traits. However, the lack of public data and standard experimental protocols makes the evaluation and comparison of novel ECG methods difficult. In this study, we perform extensive analysis and comparison of different scenarios in ECG biometric recognition. We consider verification and identification tasks, single- and multi-session settings, and single- and multi-lead ECGs recorded with traditional and user-friendly devices. We also present ECGXtractor, a robust Deep Learning technology trained with an in-house large-scale database, and evaluate it with detailed experimental protocol and public databases. With the popular PTB database, we achieve Equal Error Rates of 0.14% and 2.06% in single- and multi-session verification. The results achieved prove the soundness of ECGXtractor across multiple scenarios and databases. We release the source code, experimental protocol details, and pre-trained models in GitHub to advance in the field, This work was supported in part by the European Union’s Horizon 2020 Research and Innovation Programme under the Marie Sklodowska-Curie Grant under Agreement 860813-TRESPASS-ETN and Agreement 860315-PRIMA-ITN; in part by the IDEA-FAST under Grant IMI2-2018-15-853981; and in part by INTER-ACTION, Ministerio de Ciencia e Innovación (MICINN)/Fondo Europeo de Desarrollo Regional (FEDER), under Grant PID2021-126521OB-I00
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

Workshop on advances of mobile and wearable biometrics (WAMWB)

  • Stragapede, Giuseppe
  • Vera Rodríguez, Rubén
  • Tolosana Moranchel, Rubén
Biometrics is defined as the automated recognition of individuals based on their biological and behavioural characteristics. It represents a fundamental aspect of mobile Human-Computer Interaction (HCI), as mobile devices such as smartphones and wearables are designed to capture, process and transmit biometric data. While people benefit from the innumerable applications of biometric data in the context of HCI, new concerns have raised in relation with the performance, reliability, protection of privacy, bias, misuse, regulations, and their impact on society. The Workshop on Advances of Mobile and Wearable Biometrics (WAMWB) aims to highlight recent developments with respect to such challenges and risks, This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 860315. This workshop has also been supported by projects INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER) and HumanCAIC (TED2021-131787B-I00 MICINN)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

TypeNet: Scaling up keystroke biometrics

  • Acién Ayala, Alejandro
  • Morales Moreno, Aythami
  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Monaco, John V.
© 2020 IEEE.  Personal use of this material is permitted.  Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, We study the suitability of keystroke dynamics to authenticate 100 K users typing free-text. For this, we first analyze to what extent our method based on a Siamese Recurrent Neural Network (RNN) is able to authenticate users when the amount of data per user is scarce, a common scenario in free-text keystroke authentication. With 1 K users for testing the network, a population size comparable to previous works, TypeNet obtains an equal error rate of 4.8% using only 5 enrollment sequences and 1 test sequence per user with 50 keystrokes per sequence. Using the same amount of data per user, as the number of test users is scaled up to 100K, the performance in comparison to 1 K decays relatively by less than 5%, demonstrating the potential of Type-Net to scale well at large scale number of users. Our experiments are conducted with the Aalto University keystroke database. To the best of our knowledge, this is the largest free-text keystroke database captured with more than 136M keystrokes from 168K users, This work has been supported by projects: PRIMA (MSCA-ITN-2019-860315), TRESPASS (MSCAITN-2019-860813), BIBECA (RTI2018-101248-B-I00 MINECO), and by edBB (UAM). A. Acien is supported by a FPI fellowship from the Spanish MINECO
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Artículo científico (JournalArticle)

Keystroke Verification Challenge (KVC): ): Biometric and fairness benchmark evaluation

  • Stragapede, Giuseppe
  • Vera Rodríguez, Rubén
  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Ortega García, Javier
  • Tolosana Moranchel, Rubén
  • Damer, Naser
Los autores de la UAM pertenecen al grupo Biometrics and Data Pattern Analytics Laboratory—BiDA Laboratory, Analyzing keystroke dynamics (KD) for biometric verification has several advantages: it is among the most discriminative behavioral traits; keyboards are among the most common human-computer interfaces, being the primary means for users to enter textual data; its acquisition does not require additional hardware, and its processing is relatively lightweight; and it allows for transparently recognizing subjects. However, the heterogeneity of experimental protocols and metrics, and the limited size of the databases adopted in the literature impede direct comparisons between different systems, thus representing an obstacle in the advancement of keystroke biometrics. To alleviate this aspect, we present a new experimental framework to benchmark KD-based biometric verification performance and fairness based on tweet -long sequences of variable transcript text from over 185,000 subjects, acquired through desktop and mobile keyboards, extracted from the Aalto Keystroke Databases. The framework runs on CodaLab in the form of the Keystroke Verification Challenge (KVC). Moreover, we also introduce a novel fairness metric, the Skewed Impostor Ratio (SIR), to capture inter - and intra -demographic group bias patterns in the verification scores. We demonstrate the usefulness of the proposed framework by employing two state-of-the-art keystroke verification systems, TypeNet and TypeFormer, to compare different sets of input features, achieving a less privacy-invasive system, by discarding the analysis of text content (ASCII codes of the keys pressed) in favor of extended features in the time domain. Our experiments show that this approach allows to maintain satisfactory performance, This work was supported in part by the European Union’s Horizon 2020 Research and Innovation Program under the Marie Skłodowska-Curie under Grant 860315 (PriMa project), in part by INTER-ACTION Project under Grant PID2021-126521OB-I00 MICINN/FEDER, in part by the HumanCAIC Project under Grant TED2021-131787BI00 MICINN, and in part by the Autonomous Community of Madrid (European Laboratory for Learning and Intelligent Systems (ELLIS) Unit Madrid)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

OTB-morph: One-Time Biometrics via Morphing applied to Face Templates

  • Ghafourian, Mahdi
  • Serna Cabello, José Ignacio de la
  • Fiérrez Aguilar, Julián
  • Vera Rodríguez, Rubén
  • Morales Moreno, Aythami
© 2022 IEEE.  Personal use of this material is permitted.  Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, Cancelable biometrics refers to a group of techniques in which the biometric inputs are transformed intentionally using a key before processing or storage. This transformation is repeatable enabling subsequent biometric comparisons. This paper introduces a new scheme for cancelable biometrics aimed at protecting the templates against potential attacks, applicable to any biometric-based recognition system. Our proposed scheme is based on time-varying keys obtained from morphing random biometric information. An experimental implementation of the proposed scheme is given for face biometrics. The results confirm that the proposed approach is able to withstand against leakage attacks while improving the recognition performance, This work has been supported by projects: PRIMA (ITN-2019-860315), TRESPASS-ETN (ITN-2019-860813), and BIBECA (RTI2018-101248-B-I00 MINECO/FEDER). M.G. is supported by PRIMA and I.S. is supported by a FPI fellowship from Univ. Autonoma de Madrid.
Biblos-e Archivo. Repositorio Institucional de la UAM
Publication Contribución de congreso (ConferenceOutput)

IEEE BigData 2023 Keystroke Verification Challenge (KVC)

  • Stragapede, Giuseppe
  • Deandres-Tame, Ivan
  • Vera Rodríguez, Rubén
  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Ortega García, Javier
  • Tolosana Moranchel, Rubén
Artículo escrito por un elevado número de autores, solo se referencian el que aparece en primer lugar, el nombre del grupo de colaboración, si le hubiere, y los autores pertenecientes a la UAM, Institute, Warsaw, Poland This paper describes the results of the IEEE BigData 2023 Keystroke Verification Challenge 1 (KVC), that considers the biometric verification performance of Keystroke Dynamics (KD), captured as tweet-long sequences of variable transcript text from over 185,000 subjects. The data are obtained from two of the largest public databases of KD up to date, the Aalto Desktop and Mobile Keystroke Databases, guaranteeing a minimum amount of data per subject, age and gender annotations, absence of corrupted data, and avoiding excessively unbalanced subject distributions with respect to the considered demographic attributes. Several neural architectures were proposed by the participants, leading to global Equal Error Rates (EERs) as low as 3.33% and 3.61% achieved by the best team respectively in the desktop and mobile scenario, outperforming the current state of the art biometric verification performance for KD. Hosted on CodaLab2, the KVC will be made ongoing to represent a useful tool for the research community to compare different approaches under the same experimental conditions and to deepen the knowledge of the field, This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 860315, by project INTER-ACTION (PID2021-126521OBI00 MICINN/FEDER), and by project HumanCAIC (TED2021-131787BI00 MICINN)
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publication Artículo científico (JournalArticle)

A combination of ICT solutions with socio-ecological and economic indicators to evaluate the governance and improve the management of Posidonia oceanica in the Sierra Helada Marine Natural Park

  • Argente García, Jesús E.
  • Ramallo-González, Alfonso P.
  • Bernardeau, Jaime
  • Enguix-Egea, Ángel M.
  • Yago Martinez, José V.
  • Skarmeta Gómez, Antonio F.
Marine Protected Areas (MPAs) are essential tools in the protection and conservation of marine ecosystems. However, very few MPAs are effective and meet their conservation objectives mainly due to a lack of coordination, participation or resources. For instance, in the Mediterranean Sea there are about 1062 MPAs, but 72% of them do not have effective regulations to reduce human impacts on biodiversity and current efforts are insufficient in managing human uses for the conservation and protection of marine biodiversity. This makes necessary to seek new ways or instruments for their management. In this sense, digitalization and Information and Communication Technologies (ICT) can provide tools to facilitate the development of MPAs management in a more effective, participatory and efficient manner because automatic agents report their information and digest the data no matter the situation. In the specific case of Spain, one of the main problems identified for MPAs management is the overlapping of different kinds of protected areas in the same territory. In addition, these are managed by different administrations (Regional, National). An example of this situation is the Sierra Helada Marine Natural Park (SHMP) in Alicante. Within its territorial scope there are 3 MPAs with different protection figures that overlap each other, SHMP, the Natura 2000 site of Serra Gelada i la Marina Baixa and a fishing protected area. These MPAs are managed by different regional governmental bodies, and have a natural resources action plan in use, supported by a management collaboration agreement with the town councils in the area and the Altea Nautical Club. In the SHMP, Posidonia oceanica (PO) is one of the most important ecosystems and its presence prompted its creation and declaration. PO is a marine angiosperm endemic to the Mediterranean Sea that forms extensive meadows from the sea surface to 30–40 m depth and which provides essential goods and services: it improves water quality and protect beaches against erosion. PO meadows are recognized as biodiversity hotspots, and as major carbon sinks, but they are very sensitive to human activities. However, although many regulations protect PO, there are still significant threats to their conservation. In this regard, the research presented in this paper is the development of an ICT-based software tool, called Smart MPA, that includes an analysis framework based on management and governance indicators specific for PO and a database on PO that includes and combines different sources of information, including the results obtained of the evaluation of the analytical framework, input from information from citizens or MPA users, for example, by sending photos to report infractions in PO at specific points and from satellite images or underwater UAVs. The main objective of the Smart MPA tool is to offer services applied to PO management, either through the analysis of governance indicators or by facilitating the integration of citizen information to be used in management activities. The tool has been tested and used by the management staff of SHMP to be applied to an annual monitoring of management measures to develop an adaptive management of PO and the SHMP. After that, a semi-structured interview was carried out to collect information on the analytical framework co-developed and a survey was created where management staff of SHMP have given their perception on the usability of the tool and on the environmental problems affecting the conservation of PO in SHMP. According to the results obtained, the management of PO in SHMP is efficient, at a general level, with the lack of adequate funding being a limiting factor. The main threats are boat anchoring and organic pollution. In conclusion, the Smart MPA tool and the analytical framework are instruments that can facilitate the management and promote good environmental governance processes for the conservation of PO meadows in SHMP. At the same time, the importance of exploring mobile apps to increase citizen participation and mitigate threats and impacts on PO have been highlighted., This study forms part of: (1) ThinkInAzul project supported by Ministerio de Ciencia e Innovación with funding from European Union NextGenerationEU (PRTR-C17.I1) and by Comunidad Autónoma de la Región de Murcia-Fundación Seneca; (2) WATERMED4.0 project (Efficient use and management of conventional and non-conventional water resources through intelligent technologies applied to improve the quality and safety of Mediterranean agriculture in semi-arid areas) action 1821 of the PRIMA program of the European Union. The PRIMA program is an Art.185 initiative supported and funded by Horizon 2020 (https://www.watermed-project.eu); and (3) B-Blue (Building the Blue Biotechnology community in the Mediterranean) governance and innovation project co-financed by the European Regional Development Fund through the Interreg Med program (https://b-blue.interreg-med.eu), Peer reviewed