Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 20
Encontrada(s) 1 página(s)

Biblos-e Archivo. Repositorio Institucional de la UAM
Publicación Artículo científico (JournalArticle)

SwipeFormer: Transformers for mobile touchscreen biometrics

  • Delgado Santos, Paula
  • Guest, Richard
  • Lamb, Parker
  • Khmelnitsky, Andrei
  • Coughlan, Colm
  • Fiérrez Aguilar, Julián
  • Tolosana Moranchel, Rubén
El autor de la UAM pertenecen al grupo Biometrics and Data Pattern Analytics Laboratory—BiDA Laboratory, The growing number of mobile devices over the past few years brings a large amount of personal information, which needs to be properly protected. As a result, several mobile authentication methods have been developed. In particular, behavioural biometrics has become one of the most relevant methods due to its ability to extract the uniqueness of each subject in a secure, non-intrusive, and continuous way. This article presents SwipeFormer, a novel Transformer-based system for mobile subject authentication by means of swipe gestures in an unconstrained scenario (i.e., subjects could use their personal devices freely, without restrictions on the direction of swipe gestures or the position of the device). Our proposed system contains two modules: (i) a Transformer-based feature extractor, and (ii) a similarity computation module. Mobile data from the touchscreen and different background sensors (accelerometer and gyroscope) have been studied, including in the analysis both Android and iOS operating systems. A complete analysis of SwipeFormer is carried out using an in-house large-scale database acquired in unconstrained scenarios. In these operational conditions, SwipeFormer achieves Equal Error Rate (EER) values of 6.6% and 3.6% on Android and iOS respectively, outperforming the state of the art. In addition, we evaluate SwipeFormer on the popular publicly available databases Frank DB and HuMIdb, achieving EER values of 11.0% and 5.0% respectively, outperforming previous approaches under the same experimental setup, This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 860315. With support also from projects INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER) and HumanCAIC (TED2021-131787B-I00 MICINN), and from the Autonomous Community of Madrid (ELLIS Unit Madrid)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publicación Artículo científico (JournalArticle)

SVC-onGoing: Signature verification competition

  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Morales Moreno, Aythami
  • Ortega García, Javier
  • Tolosana Moranchel, Rubén
  • Gonzalez-Garcia, Carlos
  • Carlos Ruiz-Garcia, Juan
  • Romero Tapiador, Sergio
  • Rengifo, Santiago
  • Caruana, Miguel
  • Jiang, Jiajia
  • Lai, Songxuan
  • Jin, Lianwen
  • Zhu, Yecheng
  • Galbally, Javier
  • Diaz, Moises
  • Angel Ferrer, Miguel
  • Gomez-Barrero, Marta
  • Hodashinsky, Ilya
  • Sarin, Konstantin
  • Slezkin, Artem
  • Bardamova, Marina
  • Svetlakov, Mikhail
  • Saleem, Mohammad
  • Lia Szcs, Cintia
  • Kovari, Bence
  • Pulsmeyer, Falk
  • Wehbi, Mohamad
  • Zanca, Dario
  • Ahmad, Sumaiya
  • Mishra, Sarthak
  • Jabin, Suraiya
This article presents SVC-onGoing1, an on-going competition for on-line signature verification where researchers can easily benchmark their systems against the state of the art in an open common platform using large-scale public databases, such as DeepSignDB2 and SVC2021_EvalDB3, and standard experimental protocols. SVC-onGoing is based on the ICDAR 2021 Competition on On-Line Signature Verification (SVC 2021), which has been extended to allow participants anytime. The goal of SVC-onGoing is to evaluate the limits of on-line signature verification systems on popular scenarios (office/mobile) and writing inputs (stylus/finger) through large-scale public databases. Three different tasks are considered in the competition, simulating realistic scenarios as both random and skilled forgeries are simultaneously considered on each task. The results obtained in SVC-onGoing prove the high potential of deep learning methods in comparison with traditional methods. In particular, the best signature verification system has obtained Equal Error Rate (EER) values of 3.33% (Task 1), 7.41% (Task 2), and 6.04% (Task 3). Future studies in the field should be oriented to improve the performance of signature verification systems on the challenging mobile scenarios of SVC-onGoing in which several mobile devices and the finger are used during the signature acquisition, This work has been supported by projects: PRIMA (H2020- MSCA-ITN-2019-860315), TRESPASS-ETN (H2020-MSCA-ITN-2019- 860813), INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER), Orange Labs, and by UAM-Cecabank
Biblos-e Archivo. Repositorio Institucional de la UAM
Publicación Artículo científico (JournalArticle)

SensitiveNets: Learning agnostic representations with application to face images

  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Vera Rodríguez, Rubén
  • Tolosana Moranchel, Rubén
© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, This work proposes a novel privacy-preserving neural network feature representation to suppress the sensitive information of a learned space while maintaining the utility of the data. The new international regulation for personal data protection forces data controllers to guarantee privacy and avoid discriminative hazards while managing sensitive data of users. In our approach, privacy and discrimination are related to each other. Instead of existing approaches aimed directly at fairness improvement, the proposed feature representation enforces the privacy of selected attributes. This way fairness is not the objective, but the result of a privacy-preserving learning method. This approach guarantees that sensitive information cannot be exploited by any agent who process the output of the model, ensuring both privacy and equality of opportunity. Our method is based on an adversarial regularizer that introduces a sensitive information removal function in the learning objective. The method is evaluated on three different primary tasks (identity, attractiveness, and smiling) and three publicly available benchmarks. In addition, we present a new face annotation dataset with balanced distribution between genders and ethnic origins. The experiments demonstrate that it is possible to improve the privacy and equality of opportunity while retaining competitive performance independently of the task, This work was supported by projects: PRIMA (MSCA-ITN-2019-860315), TRESPASS-ETN (MSCA-ITN-2019-860813), and BIBECA (RTI2018-101248-B-I00 MINECO)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publicación Artículo científico (JournalArticle)

OTB-morph: one-time biometrics via morphing

  • Ghafourian, Mahdi
  • Fiérrez Aguilar, Julián
  • Vera Rodríguez, Rubén
  • Morales Moreno, Aythami
  • Serna Cabello, José Ignacio de la
Cancelable biometrics are a group of techniques to transform the input biometric to an irreversible feature intentionally using a transformation function and usually a key in order to provide security and privacy in biometric recognition systems. This transformation is repeatable enabling subsequent biometric comparisons. This paper introduces a new idea to be exploited as a transformation function for cancelable biometrics aimed at protecting templates against iterative optimization attacks. Our proposed scheme is based on time-varying keys (random biometrics in our case) and morphing transformations. An experimental implementation of the proposed scheme is given for face biometrics. The results confirm that the proposed approach is able to withstand leakage attacks while improving the recognition performance, This work has been supported by projects: PRIMA (H2020-MSCA-ITN-2019- 860315), TRESPASS-ETN (H2020-MSCA-ITN-2019-860813), and BBforTAI (PID2021-127641OB-I00 MICINN/FEDER). M.G. is supported by PRIMA and I.S. is supported by an FPI fellowship from Univ. Autonoma de Madrid
Biblos-e Archivo. Repositorio Institucional de la UAM
Publicación Artículo científico (JournalArticle)

SetMargin loss applied to deep keystroke biometrics with circle packing interpretation

  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Acién Ayala, Alejandro
  • Serna Cabello, José Ignacio de la
  • Tolosana Moranchel, Rubén
This work presents a new deep learning approach for keystroke biometrics based on a novel Distance Metric Learning method (DML). DML maps input data into a learned representation space that reveals a “semantic” structure based on distances. In this work, we propose a novel DML method specifically designed to address the challenges associated to free-text keystroke identification where the classes used in learning and inference are disjoint. The proposed SetMargin Loss (SM-L) extends traditional DML approaches with a learning process guided by pairs of sets instead of pairs of samples, as done traditionally. The proposed learning strategy allows to enlarge inter-class distances while maintaining the intra-class structure of keystroke dynamics. We analyze the resulting representation space using the mathematical problem known as Circle Packing, which provides neighbourhood structures with a theoretical maximum inter-class distance. We finally prove experimentally the effectiveness of the proposed approach on a challenging task: keystroke biometric identification over a large set of 78,000 subjects. Our method achieves state-of-the-art accuracy on a comparison performed with the best existing approaches, This work has been supported by projects: PRIMA ( MSCA-ITN- 2019-860315 ), TRESPASS-ETN (MSCA-ITN-2019-860813), BIBECA (RTI2018-101248-B-I00 MINECO), edBB (UAM), and Instituto de In- genieria del Conocimiento (IIC). A. Acien is supported by a FPI fel- lowship from the Spanish MINECO
Biblos-e Archivo. Repositorio Institucional de la UAM
Publicación Artículo científico (JournalArticle)

BehavePassDB: Public Database for Mobile Behavioral Biometrics and Benchmark Evaluation

  • Stragapede, Giuseppe
  • Vera Rodríguez, Rubén
  • Morales Moreno, Aythami
  • Tolosana Moranchel, Rubén
Mobile behavioral biometrics have become a popular topic of research, reaching promising results in terms of authentication, exploiting a multimodal combination of touchscreen and background sensor data. However, there is no way of knowing whether state-of-the-art classifiers in the literature can distinguish between the notion of user and device. In this article, we present a new database, BehavePassDB, structured into separate acquisition sessions and tasks to mimic the most common aspects of mobile Human-Computer Interaction (HCI). BehavePassDB is acquired through a dedicated mobile app installed on the subjects devices, also including the case of different users on the same device for evaluation. We propose a standard experimental protocol and benchmark for the research community to perform a fair comparison of novel approaches with the state of the art1. We propose and evaluate a system based on Long-Short Term Memory (LSTM) architecture with triplet loss and modality fusion at score level, This project has received funding from the European Unions Horizon 2020 research and innovation programme under the Marie Skodowska-Curie grant agreement no. 860315, and from Orange Labs. R. Tolosana and R. Vera-Rodriguez are also supported by INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publicación Artículo científico (JournalArticle)

GANprintR: Improved Fakes and Evaluation of the State of the Art in Face Manipulation Detection

  • Neves, João C.
  • Lopes, Vasco
  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Tolosana Moranchel, Rubén
  • Proença, Hugo
© 2020 IEEE.  Personal use of this material is permitted.  Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, The availability of large-scale facial databases, together with the remarkable progresses of deep learning technologies, in particular Generative Adversarial Networks (GANs), have led to the generation of extremely realistic fake facial content, raising obvious concerns about the potential for misuse. Such concerns have fostered the research on manipulation detection methods that, contrary to humans, have already achieved astonishing results in various scenarios. In this study, we focus on the synthesis of entire facial images, which is a specific type of facial manipulation. The main contributions of this study are four-fold: i) a novel strategy to remove GAN 'fingerprints' from synthetic fake images based on autoencoders is described, in order to spoof facial manipulation detection systems while keeping the visual quality of the resulting images; ii) an in-depth analysis of the recent literature in facial manipulation detection; iii) a complete experimental assessment of this type of facial manipulation, considering the state-of-the-art fake detection systems (based on holistic deep networks, steganalysis, and local artifacts), remarking how challenging is this task in unconstrained scenarios; and finally iv) we announce a novel public database, named iFakeFaceDB, yielding from the application of our proposed GAN-fingerprint Removal approach (GANprintR) to already very realistic synthetic fake images. The results obtained in our empirical evaluation show that additional efforts are required to develop robust facial manipulation detection systems against unseen conditions and spoof techniques, such as the one proposed in this study, This work has been supported by projects: PRIMA (H2020-MSCA-ITN-2019-860315), TRESPASS-ETN (H2020-MSCA-ITN2019-860813), BIBECA (RTI2018-101248-B-I00 MINECO/FEDER), BioGuard (Ayudas Fundación BBVA a Equipos de Investigación Cientíifica 2017), Accenture, by NOVA LINCS (UIDB/04516/2020) with the financial support of FCT - Fundação para a Ciência e a Tecnologia, through national funds, and by FCT/MCTES through national funds and co-funded by EU under the project UIDB/EEA/50008/2020
Biblos-e Archivo. Repositorio Institucional de la UAM
Publicación Artículo científico (JournalArticle)

Deepfakes and beyond: A Survey of face manipulation and fake detection

  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Morales Moreno, Aythami
  • Ortega García, Javier
  • Tolosana Moranchel, Rubén
The free access to large-scale public databases, together with the fast progress of deep learning techniques, in particular Generative Adversarial Networks, have led to the generation of very realistic fake content with its corresponding implications towards society in this era of fake news. This survey provides a thorough review of techniques for manipulating face images including DeepFake methods, and methods to detect such manipulations. In particular, four types of facial manipulation are reviewed: i) entire face synthesis, ii) identity swap (DeepFakes), iii) attribute manipulation, and iv) expression swap. For each manipulation group, we provide details regarding manipulation techniques, existing public databases, and key benchmarks for technology evaluation of fake detection methods, including a summary of results from those evaluations. Among all the aspects discussed in the survey, we pay special attention to the latest generation of DeepFakes, highlighting its improvements and challenges for fake detection. In addition to the survey information, we also discuss open issues and future trends that should be considered to advance in the field, This work has been supported by projects: PRIMA (H2020-MSCAITN-2019-860315), TRESPASS-ETN (H2020-MSCA-ITN-2019-860813), BIBECA (MINECO/FEDER RTI2018-101248-B-I00), Bio-Guard (Ayudas Fundación BBVA a Equipos de Investigación Científica 2017), and Accenture. Ruben Tolosana is supported by Consejería de Educación, Juventud y Deporte de la Comunidad de Madrid y Fondo Social Europeo
Biblos-e Archivo. Repositorio Institucional de la UAM
Publicación Artículo científico (JournalArticle)

DeepSign: Deep On-Line Signature Verification

  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Ortega García, Javier
  • Tolosana Moranchel, Rubén
Deep learning has become a breathtaking technology in the last years, overcoming traditional handcrafted approaches and even humans for many different tasks. However, in some tasks, such as the verification of handwritten signatures, the amount of publicly available data is scarce, what makes difficult to test the real limits of deep learning. In addition to the lack of public data, it is not easy to evaluate the improvements of novel proposed approaches as different databases and experimental protocols are usually considered. The main contributions of this study are: i) we provide an in-depth analysis of state-of-the-art deep learning approaches for on-line signature verification, ii) we present and describe the new DeepSignDB on-line handwritten signature biometric public database,1 iii) we propose a standard experimental protocol and benchmark to be used for the research community in order to perform a fair comparison of novel approaches with the state of the art, and iv) we adapt and evaluate our recent deep learning approach named Time-Aligned Recurrent Neural Networks (TA-RNNs)2. for the task of on-line handwritten signature verification. This approach combines the potential of Dynamic Time Warping and Recurrent Neural Networks to train more robust systems against forgeries. Our proposed TA-RNN system outperforms the state of the art, achieving results even below 2.0% EER when considering skilled forgery impostors and just one training signature per user.1https://github.com/BiDAlab/DeepSignDB 2Spanish Patent Application (P202030060), This work was supported in part by PRIMA under Grant H2020-MSCA-ITN-2019-860315; in part by TRESPASS-ETN under Grant H2020-MSCA-ITN-2019-860813; in part by BIBECA under Grant RTI2018-101248-B-I00 MINECO/FEDER; in part by Bio-Guard (Ayudas Fundación BBVA a Equipos de Investigación Científica 2017); and in part by UAM-Cecabank. The work of Ruben Tolosana was supported by the Comunidad de Madrid and Fondo Social Europeo. Spanish Patent Application (P202030060)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publicación Artículo científico (JournalArticle)

GaitPrivacyON: Privacy-preserving mobile gait biometrics using unsupervised learning

  • Delgado Santos, Paula
  • Guest, Richard
  • Deravi, Farzin
  • Morales Moreno, Aythami
  • Vera Rodríguez, Rubén
  • Tolosana Moranchel, Rubén
Numerous studies in the literature have already shown the potential of biometrics on mobile devices for authentication purposes. However, it has been shown that, the learning processes associated to biometric systems might expose sensitive personal information about the subjects. This study proposes GaitPrivacyON, a novel mobile gait biometrics verification approach that provides accurate authentication results while preserving the sensitive information of the subject. It comprises two modules: i) two convolutional Autoencoders with shared weights that transform attributes of the biometric raw data, such as the gender or the activity being performed, into a new privacy-preserving representation; and ii) a mobile gait verification system based on the combination of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) with a Siamese architecture. The main advantage of GaitPrivacyON is that the first module (convolutional Autoencoders) is trained in an unsupervised way, without specifying the sensitive attributes of the subject to protect. Two experimental studies have been examinated: i) MotionSense and MobiAct databases; and ii) OU-ISIR database. The experimental results achieved suggest the potential of GaitPrivacyON to significantly improve the privacy of the subject while keeping user authentication results higher than 96.6% Area Under the Curve (AUC). To the best of our knowledge, this is the first mobile gait verification approach that considers privacy-preserving methods trained in an unsupervised way, This project has received funding from the European Unions Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement no 860315. R. Tolosana and R. Vera-Rodriguez are also supported by INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER )
Biblos-e Archivo. Repositorio Institucional de la UAM
Publicación Artículo científico (JournalArticle)

DeepFakes detection across generations: Analysis of facial regions, fusion, and performance evaluation

  • Romero Tapiador, Sergio
  • Gonzalez Sosa
  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Tolosana Moranchel, Rubén
Media forensics has attracted a tremendous attention in the last years in part due to the increasing concerns around DeepFakes. Since the release of the initial DeepFakes databases of the 1st generation such as UADFV and FaceForensics++ up to the latest databases of the 2nd generation such as Celeb-DF and DFDC, many visual improvements have been carried out, making fake videos almost indistinguishable to the human eye. This study provides an in-depth analysis of both 1st and 2nd DeepFakes generations in terms of fake detection performance. Two different methods are considered in our experimental framework: (i) the traditional one followed in the literature based on selecting the entire face as input to the fake detection system, and (ii) a novel approach based on the selection of specific facial regions as input to the fake detection system. Fusion techniques are applied both to the facial regions and also to three different state-of-the-art fake detection systems (Xception, Capsule Network, and DSP-FWA) in order to further increase the robustness of the detectors considered. Finally, experiments regarding intra- and inter-database scenarios are performed. Among all the findings resulting from our experiments, we highlight: (i) the very good results achieved using facial regions and fusion techniques with fake detection results above 99% Area Under the Curve (AUC) for UADFV, FaceForensics++, and Celeb-DF v2 databases, and (ii) the necessity to put more efforts on the analysis of inter-database scenarios to improve the ability of the fake detectors against attacks unseen during learning, This work has been supported by projects: PRIMA (H2020-MSCAITN-2019-860315), TRESPASS-ETN (H2020-MSCA-ITN-2019-860813), BIBECA, Spain (MINECO/FEDER RTI2018-101248-B-I00)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publicación Artículo científico (JournalArticle)

IRL-Net: inpainted region localization network via spatial attention

  • Daryani, Amir Etefaghi
  • Mirmahdi, Mahdieh
  • Hassanpour, Ahmad
  • Shahreza, Hatef Otroshi
  • Yang, Bian
  • Fiérrez Aguilar, Julián
Identifying manipulated regions in images is a challenging task due to the existence of very accurate image inpainting techniques leaving almost unnoticeable traces in tampered regions. These image inpainting methods can be used for multiple purposes (e.g., removing objects, reconstructing corrupted areas, eliminating various types of distortion, etc.) makes creating forensic detectors for image manipulation an extremely difficult and time-consuming procedure. The aim of this paper is to localize the tampered regions manipulated by image inpainting methods. To do this, we propose a novel CNN-based deep learning model called IRL-Net which includes three main modules: Enhancement, Encoder, and Decoder modules. To evaluate our method, three image inpainting methods have been used to reconstruct the missed regions in two face and scene image datasets. We perform both qualitative and quantitative evaluations on the generated datasets. Experimental results demonstrate that our method outperforms previous learning-based manipulated region detection methods and generates realistic and semantically plausible images. We also provide the implementation of the proposed approach to support reproducible research via https://github.com/amiretefaghi/IRL-Net, This work was supported by the Project Privacy Matters (PRIMA) under Grant H2020-MSCA-ITN-2019-860315. The work of Hatef Otroshi Shahreza was supported by the H2020 TReSPAsS-ETN Marie Skłodowska-Curie Early Training Network under Grant 860813. The work of Julian Fierrez was supported by the Project Biometrics and Behavior for Unbiased and Trusted AI with Applications (BBforTAI) under Grant PID2021-127641OB-I00 MICINN/FEDER
Biblos-e Archivo. Repositorio Institucional de la UAM
Publicación Artículo científico (JournalArticle)

Exploring transformers for behavioural biometrics: a case study in gait recognition

  • Delgado Santos, Paula
  • Guest, Richard
  • Deravi, Farzin
  • Vera Rodríguez, Rubén
  • Tolosana Moranchel, Rubén
Biometrics on mobile devices has attracted a lot of attention in recent years as it is considered a user-friendly authentication method. This interest has also been motivated by the success of Deep Learning (DL). Architectures based on Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) have established convenience for the task, improving the performance and robustness in comparison to traditional machine learning techniques. However, some aspects must still be revisited and improved. To the best of our knowledge, this is the first article that explores and proposes a novel gait biometric recognition systems based on Transformers, which currently obtain state-of-the-art performance in many applications. Several state-of-the-art architectures (Vanilla, Informer, Autoformer, Block-Recurrent Transformer, and THAT) are considered in the experimental framework. In addition, new Transformer configurations are proposed to further increase the performance. Experiments are carried out using the two popular public databases: whuGAIT and OU-ISIR. The results achieved prove the high ability of the proposed Transformer, outperforming state-of-the-art CNN and RNN architectures, This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement no. 860315. With support also from projects INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER) and HumanCAIC (TED2021-131787B-I00 MICINN), and from Comunidad de Madrid (ELLIS Unit Madrid)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publicación Artículo científico (JournalArticle)

TypeFormer: Transformers for mobile keystroke biometrics

  • Stragapede, Giuseppe
  • Delgado Santos, Paula
  • Vera Rodríguez, Rubén
  • Guest, Richard
  • Morales Moreno, Aythami
  • Tolosana Moranchel, Rubén
The broad usage of mobile devices nowadays, the sensitiveness of the information contained in them, and the shortcomings of current mobile user authentication methods are calling for novel, secure, and unobtrusive solutions to verify the users’ identity. In this article, we propose TypeFormer, a novel transformer architecture to model free-text keystroke dynamics performed on mobile devices for the purpose of user authentication. The proposed model consists in temporal and channel modules enclosing two long short-term memory recurrent layers, Gaussian range encoding, a multi-head self-attention mechanism, and a block-recurrent transformer layer. Experimenting on one of the largest public databases to date, the Aalto mobile keystroke database, TypeFormer outperforms current state-of-the-art systems achieving equal error rate values of 3.25% using only five enrolment sessions of 50 keystrokes each. In such way, we contribute to reducing the traditional performance gap of the challenging mobile free-text scenario with respect to its desktop and fixed-text counterparts. To highlight the design rationale, an analysis of the experimental results of the different modules implemented in the development of TypeFormer is carried out. Additionally, we analyse the behaviour of the model with different experimental configurations such as the length of the keystroke sequences and the amount of enrolment sessions, showing margin for improvement, This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 860315. Moreover, it has been supported by INTER-ACTION (PID2021126521OB-I00 MICINN/FEDER) and Cátedra ENIA UAM-VERIDAS en IA Responsable (NextGenerationEU PRTR TSI-1009272023-2)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publicación Artículo científico (JournalArticle)

Symbolic AI for XAI: Evaluating LFIT inductive programming for explaining biases in machine learning

  • Ortega de la Puente, Alfonso
  • Wang, Zilong
  • de la Cruz, Marina
  • Alonso, César Luis
  • Ribeiro, Tony
  • Fiérrez Aguilar, Julián
  • Morales Moreno, Aythami
Machine learning methods are growing in relevance for biometrics and personal information processing in domains such as forensics, e-health, recruitment, and e-learning. In these domains, white-box (human-readable) explanations of systems built on machine learning methods become crucial. Inductive logic programming (ILP) is a subfield of symbolic AI aimed to automatically learn declarative theories about the processing of data. Learning from interpretation transition (LFIT) is an ILP technique that can learn a propositional logic theory equivalent to a given black-box system (under certain conditions). The present work takes a first step to a general methodology to incorporate accurate declarative explanations to classic machine learning by checking the viability of LFIT in a specific AI application scenario: fair recruitment based on an automatic tool generated with machine learning methods for ranking Curricula Vitae that incorporates soft biometric information (gender and ethnicity). We show the expressiveness of LFIT for this specific problem and propose a scheme that can be applicable to other domains. In order to check the ability to cope with other domains no matter the machine learning paradigm used, we have done a preliminary test of the expressiveness of LFIT, feeding it with a real dataset about adult incomes taken from the US census, in which we consider the income level as a function of the rest of attributes to verify if LFIT can provide logical theory to support and explain to what extent higher incomes are biased by gender and ethnicity, This work was supported by projects: PRIMA (H2020-MSCA-ITN-2019-860315), TRESPASS-ETN(H2020-MSCA-ITN-2019-860813), IDEA-FAST (IMI2-2018-15-853981), BIBECA(RTI2018-101248-B-I00MINECO/FEDER), RTI2018-095232-B-C22MINECO, PLeNTaS project PID2019-111430RBI00MINECO; and also by Pays de la Loire Region through RFI Atlanstic 2020
Biblos-e Archivo. Repositorio Institucional de la UAM
Publicación Artículo científico (JournalArticle)

M-GaitFormer: Mobile biometric gait verification using Transformers

  • Delgado Santos, Paula
  • Guest, Richard
  • Vera Rodríguez, Rubén
  • Fiérrez Aguilar, Julián
  • Tolosana Moranchel, Rubén
Mobile devices such as smartphones and smartwatches are part of our everyday life, acquiring large amount of personal information that needs to be properly secured. Among the different authentication techniques, behavioural biometrics has become a very popular method as it allows authentication in a non-intrusive and continuous way. This study proposes M-GaitFormer, a novel mobile biometric gait verification system based on Transformer architectures. This biometric system only considers the accelerometer and gyroscope data acquired by the mobile device. A complete analysis of the proposed M-GaitFormer is carried out using the popular available databases whuGAIT and OU-ISIR. M-GaitFormer achieves Equal Error Rate (EER) values of 3.42% and 2.90% on whuGAIT and OU-ISIR, respectively, outperforming other state-of-the-art approaches based on popular Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 860315. With support also from projects INTER-ACTION (PID2021-126521OB-I00 MICINN/FEDER), HumanCAIC (TED2021-131787B-I00 MICINN), and Comunidad de Madrid (ELLIS Unit Madrid)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publicación Artículo científico (JournalArticle)

E2F-Net: Eyes-to-face inpainting via StyleGAN latent space

  • Hassanpour, Ahmad
  • Jamalbafrani, Fatemeh
  • Yang, Bian
  • Raja, Kiran
  • Veldhuis, Raymond
  • Fiérrez Aguilar, Julián
Face inpainting, the technique of restoring missing or damaged regions in facial images, is pivotal for applications like face recognition in occluded scenarios and image analysis with poor-quality captures. This process not only needs to produce realistic visuals but also preserve individual identity characteristics. The aim of this paper is to inpaint a face given periocular region (eyes-to-face) through a proposed new Generative Adversarial Network (GAN)-based model called Eyes-to-Face Network (E2F-Net). The proposed approach extracts identity and non-identity features from the periocular region using two dedicated encoders have been used. The extracted features are then mapped to the latent space of a pre-trained StyleGAN generator to benefit from its state-of-the-art performance and its rich, diverse and expressive latent space without any additional training. We further improve the StyleGAN's output to find the optimal code in the latent space using a new optimization for GAN inversion technique. Our E2F-Net requires a minimum training process reducing the computational complexity as a secondary benefit. Through extensive experiments, we show that our method successfully reconstructs the whole face with high quality, surpassing current techniques, despite significantly less training and supervision efforts. We have generated seven eyes-to-face datasets based on well-known public face datasets for training and verifying our proposed methods. The code and datasets are publicly available, This work was supported by the Project Privacy Matters (PRIMA) under Grant H2020-MSCA-ITN-2019-860315. J.F. is funded by the ENIA Chair UAM-Veridas in Responsible AI (NextGenerationEU PRTR TSI-100927-2023-2) and project BBforTAI (PID2021-1276 41OB-I00 MICINN/FEDER)
Biblos-e Archivo. Repositorio Institucional de la UAM
Publicación Artículo científico (JournalArticle)

BeCAPTCHA: Behavioral bot detection using touchscreen and mobile sensors benchmarked on HuMIdb

  • Acién Ayala, Alejandro
  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Vera Rodríguez, Rubén
  • Delgado Ben Mohatar, Óscar
In this paper we study the suitability of a new generation of CAPTCHA methods based on smartphone interactions. The heterogeneous flow of data generated during the interaction with the smartphones can be used to model human behavior when interacting with the technology and improve bot detection algorithms. For this, we propose BeCAPTCHA, a CAPTCHA method based on the analysis of the touchscreen information obtained during a single drag and drop task in combination with the accelerometer data. The goal of BeCAPTCHA is to determine whether the drag and drop task was realized by a human or a bot. We evaluate the method by generating fake samples synthesized with Generative Adversarial Neural Networks and handcrafted methods. Our results suggest the potential of mobile sensors to characterize the human behavior and develop a new generation of CAPTCHAs. The experiments are evaluated with HuMIdb1 (Human Mobile Interaction database), a novel multimodal mobile database that comprises 14 mobile sensors acquired from 600 users. HuMIdb is freely available to the research community, This work has been supported by projects: PRIMA, Spain (H2020-MSCA-ITN-2019-860315), TRESPASS-ETN, Spain (H2020-MSCA-ITN-2019-860813), BIBECA RTI2018-101248-B-I00 (MINECO/FEDER), and BioGuard, Spain (Ayudas Fundación BBVA a Equipos de Investigación Científica 2017). Spanish Patent Application P202030066
Biblos-e Archivo. Repositorio Institucional de la UAM
Publicación Artículo científico (JournalArticle)

Keystroke Verification Challenge (KVC): ): Biometric and fairness benchmark evaluation

  • Stragapede, Giuseppe
  • Vera Rodríguez, Rubén
  • Morales Moreno, Aythami
  • Fiérrez Aguilar, Julián
  • Ortega García, Javier
  • Tolosana Moranchel, Rubén
  • Damer, Naser
Los autores de la UAM pertenecen al grupo Biometrics and Data Pattern Analytics Laboratory—BiDA Laboratory, Analyzing keystroke dynamics (KD) for biometric verification has several advantages: it is among the most discriminative behavioral traits; keyboards are among the most common human-computer interfaces, being the primary means for users to enter textual data; its acquisition does not require additional hardware, and its processing is relatively lightweight; and it allows for transparently recognizing subjects. However, the heterogeneity of experimental protocols and metrics, and the limited size of the databases adopted in the literature impede direct comparisons between different systems, thus representing an obstacle in the advancement of keystroke biometrics. To alleviate this aspect, we present a new experimental framework to benchmark KD-based biometric verification performance and fairness based on tweet -long sequences of variable transcript text from over 185,000 subjects, acquired through desktop and mobile keyboards, extracted from the Aalto Keystroke Databases. The framework runs on CodaLab in the form of the Keystroke Verification Challenge (KVC). Moreover, we also introduce a novel fairness metric, the Skewed Impostor Ratio (SIR), to capture inter - and intra -demographic group bias patterns in the verification scores. We demonstrate the usefulness of the proposed framework by employing two state-of-the-art keystroke verification systems, TypeNet and TypeFormer, to compare different sets of input features, achieving a less privacy-invasive system, by discarding the analysis of text content (ASCII codes of the keys pressed) in favor of extended features in the time domain. Our experiments show that this approach allows to maintain satisfactory performance, This work was supported in part by the European Union’s Horizon 2020 Research and Innovation Program under the Marie Skłodowska-Curie under Grant 860315 (PriMa project), in part by INTER-ACTION Project under Grant PID2021-126521OB-I00 MICINN/FEDER, in part by the HumanCAIC Project under Grant TED2021-131787BI00 MICINN, and in part by the Autonomous Community of Madrid (European Laboratory for Learning and Intelligent Systems (ELLIS) Unit Madrid)
DIGITAL.CSIC. Repositorio Institucional del CSIC
Publicación Artículo científico (JournalArticle)

A combination of ICT solutions with socio-ecological and economic indicators to evaluate the governance and improve the management of Posidonia oceanica in the Sierra Helada Marine Natural Park

  • Argente García, Jesús E.
  • Ramallo-González, Alfonso P.
  • Bernardeau, Jaime
  • Enguix-Egea, Ángel M.
  • Yago Martinez, José V.
  • Skarmeta Gómez, Antonio F.
Marine Protected Areas (MPAs) are essential tools in the protection and conservation of marine ecosystems. However, very few MPAs are effective and meet their conservation objectives mainly due to a lack of coordination, participation or resources. For instance, in the Mediterranean Sea there are about 1062 MPAs, but 72% of them do not have effective regulations to reduce human impacts on biodiversity and current efforts are insufficient in managing human uses for the conservation and protection of marine biodiversity. This makes necessary to seek new ways or instruments for their management. In this sense, digitalization and Information and Communication Technologies (ICT) can provide tools to facilitate the development of MPAs management in a more effective, participatory and efficient manner because automatic agents report their information and digest the data no matter the situation. In the specific case of Spain, one of the main problems identified for MPAs management is the overlapping of different kinds of protected areas in the same territory. In addition, these are managed by different administrations (Regional, National). An example of this situation is the Sierra Helada Marine Natural Park (SHMP) in Alicante. Within its territorial scope there are 3 MPAs with different protection figures that overlap each other, SHMP, the Natura 2000 site of Serra Gelada i la Marina Baixa and a fishing protected area. These MPAs are managed by different regional governmental bodies, and have a natural resources action plan in use, supported by a management collaboration agreement with the town councils in the area and the Altea Nautical Club. In the SHMP, Posidonia oceanica (PO) is one of the most important ecosystems and its presence prompted its creation and declaration. PO is a marine angiosperm endemic to the Mediterranean Sea that forms extensive meadows from the sea surface to 30–40 m depth and which provides essential goods and services: it improves water quality and protect beaches against erosion. PO meadows are recognized as biodiversity hotspots, and as major carbon sinks, but they are very sensitive to human activities. However, although many regulations protect PO, there are still significant threats to their conservation. In this regard, the research presented in this paper is the development of an ICT-based software tool, called Smart MPA, that includes an analysis framework based on management and governance indicators specific for PO and a database on PO that includes and combines different sources of information, including the results obtained of the evaluation of the analytical framework, input from information from citizens or MPA users, for example, by sending photos to report infractions in PO at specific points and from satellite images or underwater UAVs. The main objective of the Smart MPA tool is to offer services applied to PO management, either through the analysis of governance indicators or by facilitating the integration of citizen information to be used in management activities. The tool has been tested and used by the management staff of SHMP to be applied to an annual monitoring of management measures to develop an adaptive management of PO and the SHMP. After that, a semi-structured interview was carried out to collect information on the analytical framework co-developed and a survey was created where management staff of SHMP have given their perception on the usability of the tool and on the environmental problems affecting the conservation of PO in SHMP. According to the results obtained, the management of PO in SHMP is efficient, at a general level, with the lack of adequate funding being a limiting factor. The main threats are boat anchoring and organic pollution. In conclusion, the Smart MPA tool and the analytical framework are instruments that can facilitate the management and promote good environmental governance processes for the conservation of PO meadows in SHMP. At the same time, the importance of exploring mobile apps to increase citizen participation and mitigate threats and impacts on PO have been highlighted., This study forms part of: (1) ThinkInAzul project supported by Ministerio de Ciencia e Innovación with funding from European Union NextGenerationEU (PRTR-C17.I1) and by Comunidad Autónoma de la Región de Murcia-Fundación Seneca; (2) WATERMED4.0 project (Efficient use and management of conventional and non-conventional water resources through intelligent technologies applied to improve the quality and safety of Mediterranean agriculture in semi-arid areas) action 1821 of the PRIMA program of the European Union. The PRIMA program is an Art.185 initiative supported and funded by Horizon 2020 (https://www.watermed-project.eu); and (3) B-Blue (Building the Blue Biotechnology community in the Mediterranean) governance and innovation project co-financed by the European Regional Development Fund through the Interreg Med program (https://b-blue.interreg-med.eu), Peer reviewed