Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 6
Encontrada(s) 1 página(s)

Academica-e. Repositorio Institucional de la Universidad Pública de Navarra
Publicación Contribución de congreso (ConferenceOutput)

Diseño y captura de una base de datos para el reconocimiento de emociones minimizando sesgos

  • Jurío Munárriz, Aránzazu
  • Pascual Casas, Rubén
  • Domínguez Catena, Iris
  • Paternain Dallo, Daniel
  • Galar Idoate, Mikel
El reconocimiento de emociones a partir de expresiones faciales (FER) es un campo de investigación importante para la interacción persona-máquina. Sin embargo, los conjuntos de datos utilizados para entrenar modelos FER a menudo contienen sesgos demográficos que pueden conducir a la discriminación en el modelo final. En este trabajo, presentamos el diseño y la captura realizados para la creación de una nueva base de datos para FER, donde tratamos de minimizar los sesgos desde el propio diseño. La base de datos se ha creado utilizando diferentes métodos de captura. Para comprobar la reducción de los sesgos alcanzada, analizamos diferentes métricas de sesgo representacional y estereotípico sobre la base de datos generada y la comparamos frente a otras bases de datos estándar en la literatura de FER., Este trabajo ha sido financiado por el MICIU del Gobierno de España a través del proyecto PID2022-136627NBI00/ AEI/10.13039/501100011033 FEDER, UE y de una ayuda FPU (Rubén Pascual), por el Gobierno de Navarra (0011-1411-2020-000079 - Emotional Films), y por el Servicio de Investigación de la Universidad Pública de Navarra (contrato predoctoral de Iris Domínguez-Catena.)
Academica-e. Repositorio Institucional de la Universidad Pública de Navarra
Publicación Artículo científico (JournalArticle)

Metrics for dataset demographic bias: a case study on facial expression recognition

  • Domínguez Catena, Iris
  • Paternain Dallo, Daniel
  • Galar Idoate, Mikel
Demographic biases in source datasets have been shown as one of the causes of unfairness and discrimination in the predictions of Machine Learning models. One of the most prominent types of demographic bias are statistical imbalances in the representation of demographic groups in the datasets. In this paper, we study the measurement of these biases by reviewing the existing metrics, including those that can be borrowed from other disciplines. We develop a taxonomy for the classification of these metrics, providing a practical guide for the selection of appropriate metrics. To illustrate the utility of our framework, and to further understand the practical characteristics of the metrics, we conduct a case study of 20 datasets used in Facial Emotion Recognition (FER), analyzing the biases present in them. Our experimental results show that many metrics are redundant and that a reduced subset of metrics may be sufficient to measure the amount of demographic bias. The paper provides valuable insights for researchers in AI and related fields to mitigate dataset bias and improve the fairness and accuracy of AI models., This work was supported in part by a predoctoral fellowship from the Research Service of the Universidad Pública de Navarra through open access funding, in part by the Spanish MICIN under Grants PID2019-108392GB-I00, PID2020-118014RB-I00, and PID2022-136627NB-I00/AEI/10.13039/501100011033 FEDER, UE, and in part by the Government of Navarre under Grant 0011-1411-2020-000079 - Emotional Films.
Academica-e. Repositorio Institucional de la Universidad Pública de Navarra
Publicación Artículo científico (JournalArticle)

Less can be more: representational vs. stereotypical gender bias in facial expression recognition

  • Domínguez Catena, Iris
  • Paternain Dallo, Daniel
  • Jurío Munárriz, Aránzazu
  • Galar Idoate, Mikel
Machine learning models can inherit biases from their training data, leading to discriminatory or inaccurate predictions. This is particularly concerning with the increasing use of large, unsupervised datasets for training foundational models. Traditionally, demographic biases within these datasets have not been well-understood, limiting our ability to understand how they propagate to the models themselves. To address this issue, this paper investigates the propagation of demographic biases from datasets into machine learning models. We focus on the gender demographic component, analyzing two types of bias: representational and stereotypical. For our analysis, we consider the domain of facial expression recognition (FER), a field known to exhibit biases in most popular datasets. We use Affectnet, one of the largest FER datasets, as our baseline for carefully designing and generating subsets that incorporate varying strengths of both representational and stereotypical bias. Subsequently, we train several models on these biased subsets, evaluating their performance on a common test set to assess the propagation of bias into the models¿ predictions. Our results show that representational bias has a weaker impact than expected. Models exhibit a good generalization ability even in the absence of one gender in the training dataset. Conversely, stereotypical bias has a significantly stronger impact, primarily concentrated on the biased class, although it can also influence predictions for unbiased classes. These results highlight the need for a bias analysis that differentiates between types of bias, which is crucial for the development of effective bias mitigation strategies., This work was funded by a predoctoral fellowship and open access funding from the Research Service of the Universidad Publica de Navarra, the Spanish MICIN (PID2020-118014RB-I00 and PID2022-136627NB-I00, AEI/10.13039/501100011033 FEDER, UE), the Government of Navarre (0011-1411-2020-000079 - Emotional Films), and the support of the 2024 Leonardo Grant for Researchers and Cultural Creators from the BBVA Foundation.
Academica-e. Repositorio Institucional de la Universidad Pública de Navarra
Publicación Artículo científico (JournalArticle)

DSAP: analyzing bias through demographic comparison of datasets

  • Domínguez Catena, Iris
  • Paternain Dallo, Daniel
  • Galar Idoate, Mikel
In the last few years, Artificial Intelligence (AI) systems have become increasingly widespread. Unfortunately, these systems can share many biases with human decision-making, including demographic biases. Often, these biases can be traced back to the data used for training, where large uncurated datasets have become the norm. Despite our awareness of these biases, we still lack general tools to detect, quantify, and compare them across different datasets. In this work, we propose DSAP (Demographic Similarity from Auxiliary Profiles), a two-step methodology for comparing the demographic composition of datasets. First, DSAP uses existing demographic estimation models to extract a dataset's demographic profile. Second, it applies a similarity metric to compare the demographic profiles of different datasets. While these individual components are well-known, their joint use for demographic dataset comparison is novel and has not been previously addressed in the literature. This approach allows three key applications: the identification of demographic blind spots and bias issues across datasets, the measurement of demographic bias, and the assessment of demographic shifts over time. DSAP can be used on datasets with or without explicit demographic information, provided that demographic information can be derived from the samples using auxiliary models, such as those for image or voice datasets. To show the usefulness of the proposed methodology, we consider the Facial Expression Recognition task, where demographic bias has previously been found. The three applications are studied over a set of twenty datasets with varying properties. The code is available at https://github.com/irisdominguez/DSAP., This work was funded by a predoctoral fellowship from the Research Service of the Universidad Publica de Navarra, the Spanish MICIN (PID2020-118014RB-I00 and PID2022-136627NB-I00/AEI/10.13039/501100011033 FEDER, UE), and the Government of Navarre (0011-1411-2020-000079 - Emotional Films). Open access funding provided by Universidad Pública de Navarra.
Academica-e. Repositorio Institucional de la Universidad Pública de Navarra
Publicación Artículo científico (JournalArticle)

Biased heritage: how datasets shape models in facial expression recognition

  • Domínguez Catena, Iris
  • Paternain Dallo, Daniel
  • Galar Idoate, Mikel
  • Defrance, MaryBeth
  • Buyl, Maarten
  • De Bie, Tijl
In recent years, the rapid development of artificial intelligence (AI) systems has raised concerns about our ability to ensure their fairness, that is, how to avoid discrimination based on protected characteristics such as gender, race, or age. While algorithmic fairness is well-studied in simple binary classification tasks on tabular data, its application to complex, real-world scenarios¿such as Facial Expression Recognition (FER)-remains underexplored. FER presents unique challenges: it is inherently multiclass, and biases emerge across intersecting demographic variables, each potentially comprising multiple protected groups. We present a comprehensive framework to analyze bias propagation from datasets to trained models in image-based FER systems, while proposing new bias metrics for multiclass problems with multiple demographic groups necessary for this analysis. Our methodology studies bias propagation by (1) inducing controlled biases in FER datasets, (2) training models on these biased datasets, and (3) analyzing the correlation between dataset bias metrics and model fairness notions. Our findings reveal that stereotypical biases propagate more strongly to model predictions than representational biases, suggesting that preventing emotion-specific demographic patterns should be prioritized over general demographic balance in FER datasets. Additionally, we observe that biased datasets lead to reduced model accuracy, challenging the assumed fairness-accuracy trade-off in this setting., This work was funded by a predoctoral fellowship and open access funding from the Research Service of the Universidad Publica de Navarra, the Spanish MICIN (PID2022- 136627NB-I00/AEI/10.13039/501100011033 FEDER, UE), the Government of Navarre (0011-1411-2020-000079 – Emotional Films), and the support of the 2024 Leonardo Grant for Researchers and Cultural Creators from the BBVA Foundation. It was also supported by the Flemish Government under the "Onderzoeksprogramma Artificiele Intelligentie (AI) Vlaanderen" programme, and the European Union (ERC, VIGILIA, 101142229).
Academica-e. Repositorio Institucional de la Universidad Pública de Navarra
Publicación Contribución de congreso (ConferenceOutput)

Gender stereotyping impact in facial expression recognition

  • Domínguez Catena, Iris
  • Paternain Dallo, Daniel
  • Galar Idoate, Mikel
Facial Expression Recognition (FER) uses images of faces to identify the emotional state of users, allowing for a closer interaction between humans and autonomous systems. Unfortunately, as the images naturally integrate some demographic information, such as apparent age, gender, and race of the subject, these systems are prone to demographic bias issues. In recent years, machine learning-based models have become the most popular approach to FER. These models require training on large datasets of facial expression images, and their generalization capabilities are strongly related to the characteristics of the dataset. In publicly available FER datasets, apparent gender representation is usually mostly balanced, but their representation in the individual label is not, embedding social stereotypes into the datasets and generating a potential for harm. Although this type of bias has been overlooked so far, it is important to understand the impact it may have in the context of FER. To do so, we use a popular FER dataset, FER+, to generate derivative datasets with different amounts of stereotypical bias by altering the gender proportions of certain labels. We then proceed to measure the discrepancy between the performance of the models trained on these datasets for the apparent gender groups. We observe a discrepancy in the recognition of certain emotions between genders of up to 29 % under the worst bias conditions. Our results also suggest a safety range for stereotypical bias in a dataset that does not appear to produce stereotypical bias in the resulting model. Our findings support the need for a thorough bias analysis of public datasets in problems like FER, where a global balance of demographic representation can still hide other types of bias that harm certain demographic groups., This work was funded by a predoctoral fellowship of the Research Service of Universidad Publica de Navarra, the Spanish MICIN (PID2019-108392GB-I00 and PID2020-118014RB-I00 / AEI / 10.13039/501100011033), and the Government of Navarre (0011-1411-2020-000079 - Emotional Films).