Pasar al contenido principal

FAIRTRANSNLP-STEREOTYPES: IDENTIFICACION DE ESTEREOTIPOS Y PREJUICIOS Y DESARROLLO DE SISTEMAS EQUITATIVOS

PID2021-124361OB-C31

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal para Impulsar la Investigación Científico-Técnica y su Transferencia
Subprograma Subprograma Estatal de Generación de Conocimiento
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2021
Plan Estatal 2021-2023 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSITAT POLITÈCNICA DE VALÈNCIA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 14
Encontrada(s) 1 página(s)

RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Identification of Racial and Sexist Stereotypes in Spanish: A Learning with Disagreements Approach, Identificación de Estereotipos Raciales y Sexistas en Español: Un Enfoque Basado en Aprendizaje con Desacuerdos

  • Urios Alacreu, Elias
  • Rosso, Paolo|||0000-0002-8922-1242
[ES] El discurso del odio ha proliferado significativamente en los ¿ultimos a¿nos, en gran medida impulsado por la adopci¿on generalizada de plataformas de redes sociales. El discurso de odio a menudo opera de manera impl¿¿cita, aprovechando estereotipos sutiles para propagar pensamientos discriminatorios. Estos mecanismos encubiertos han permitido que el contenido odioso se oculte a si mismo, haciendo que su detecci¿on resulte cada vez m¿as compleja. Como resultado, la lucha contra los discursos de odio se ha convertido en una prioridad urgente, impulsando la adopci¿on generalizada de modelos de aprendizaje profundo para detectar y combatir contenidos nocivos. Dada la naturaleza inherentemente subjetiva de los discursos del odio, es necesario desarrollar modelos que sean lo m¿as generalizables posible. Esto ha llevado a la aparici¿on del paradigma de aprendizaje con desacuerdos, que tiene como objetivo introducir desacuerdos dentro de la propia tarea para mejorar la generalizabilidad del modelo. Este trabajo investiga este ¿ultimo paradigma a trav¿es de dos shared tasks. La primera tarea, DETEST-Dis, explora los estereotipos contra los inmigrantes en comentarios en l¿¿nea y fue organizada en IberLEF 2024. Nuestros resultados se encuentran entre los mejores de todos los equipos participantes, superando los enfoques tradicionales. La segunda tarea, EXIST, se centra en el sexismo en los memes y fue organizada en CLEF 2024. En este caso, nuestro rendimiento presenta una mejor¿¿a a¿nadiendo caracter¿¿sticas de un modelo externo as¿¿ como tambi¿en data augmentation. Nuestro c¿odigo fuente se puede encontrar en https://github.com/Buzzeitor30/DETESTS-DIS y https://github.com/Buzzeitor30/EXIST-TFM, [EN] Hate speech has proliferated significantly in recent years, largely driven by the widespread adoption of social media platforms. Hate speech often operates implicitly, leveraging subtle stereotypes to propagate discriminatory views. These covert mechanisms allow harmful content to disguise itself, making detection increasingly complex. As a result, tackling hate speech has become an urgent priority, driving the widespread adoption of deep learning models to detect and combat harmful content. Given the inherently subjective nature of hate speech and its nuanced manifestations, there is a need to develop models that are as generalizable as possible. This has led to the emergence of the learning with disagreements paradigm, which aims to introduce disagreements within the task itself to enhance model generalizability. This paper investigates the latter paradigm through two shared tasks. The first task, DETEST-Dis, explores stereotypes against immigrants in online comments and was organized at IberLEF 2024. Our results are among the best of all participating teams, surpassing traditional approaches. The second task, EXIST, focuses on sexism in memes and was organized at CLEF 2024. Here, our performance is enhanced by adding features from an external model as well as data augmentation. Our source code can be found on https://github.com/Buzzeitor30/DETESTS-DIS and https://github.com/Buzzeitor30/EXIST-TFM., This work has been developed under the project FAKEnHATE-PdC (PDC2022-133118-I00) with founding from European NextGenerationEU/PRTR. EUR and PR carried out part of thier work also in the framework of the following projects respectively: MARTINI (Grant PCI2022-135008-2) and FairTransNLP-Stereotypes (Grant PID2021-124361OB-C31) both funded by MCIN/AEI/10.13039/501100011033 and by European Union NextGenerationEU/PRTR.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Recognizing misogynous memes: Biased models and tricky archetypes

  • Rizzi, Giulia
  • Gasparini, Francesca
  • Saibene, Aurora
  • Fersini, Elisabetta
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] Misogyny is a form of hate against women and has been spreading exponentially through the Web, especially on social media platforms. Hateful content towards women can be conveyed not only by text but also using visual and/or audio sources or their combination, highlighting the necessity to address it from a multimodal perspective. One of the predominant forms of multimodal content against women is represented by memes, which are images characterized by pictorial content with an overlaying text introduced a posteriori. Its main aim is originally to be funny and/or ironic, making misogyny recognition in memes even more challenging. In this paper, we investigated 4 unimodal and 3 multimodal approaches to determine which source of information contributes more to the detection of misogynous memes. Moreover, a bias estimation technique is proposed to identify specific elements that compose a meme that could lead to unfair models, together with a bias mitigation strategy based on Bayesian Optimization. The proposed method is able to push the prediction probabilities towards the correct class for up to 61.43% of the cases. Finally, we identified the most challenging archetypes of memes that are still far to be properly recognized, highlighting the most relevant open research directions., The work of Paolo Rosso was done in the framework of the FairTransNLP-Stereotypes research project on Fairness and Transparency for equitable NLP applications in social media: Identifying stereotypes and prejudices and developing equitable systems (PID2021-124361OB-C31) funded by MCIN/AEI/10.13039/501100011033 and by ERDF, EU A way of making Europe. The work of Elisabetta Fersini and Francesca Gasparini has been partially funded by the European Union - NextGenerationEU under the National Research Centre For HPC, Big Data and Quantum Computing-Spoke 9-Digital Society and Smart Cities (PNRR-MUR), and by REGAINS - Excellence Department Research Project.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

On the Robustness of Transformer-Based Models to Different Linguistic Perturbations: A Case of Study in Irony Detection

  • Ortega-Bueno, Reynier
  • Fersini, Elisabetta
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] This study investigates the robustness of Transformer models in irony detection addressing various textual perturbations, revealing potential biases in training data concerning ironic and non-ironic classes. The perturbations involve three distinct approaches, each progressively increasing in complexity. The first approach is word masking, which employs wild-card characters or utilises BERT-specific masking through the mask token provided by BERT models. The second approach is word substitution, replacing the bias word with a contextually appropriate alternative. Lastly, paraphrasing generates a new phrase while preserving the original semantic meaning. We leverage Large Language Models (GPT 3.5 Turbo) and human inspection to ensure linguistic correctness and contextual coherence for word substitutions and paraphrasing. The results indicate that models are susceptible to these perturbations, and paraphrasing and word substitution demonstrate the most significant impact on model predictions. The irony class appears to be particularly challenging for models when subjected to these perturbations. The SHAP and LIME methods are used to correlate variations in attribution scores with prediction errors. A notable difference in the Total Variation of attribution scores is observed between original examples and cases involving bias word substitution or masking. Among the corpora used, TwSemEval2018 emerges as the most challenging. Regarding model performance, Transformer-based models such as RoBERTa and BERTweet demonstrate superior overall performance addressing these perturbations. This research contributes to understanding the robustness and limitations of irony detection models, highlighting areas for improvement in model design and training data curation., The work of Paolo Rosso was in the framework of the research project on Fairness and Transparency for equitable NLP applications in social media (grant PID2021-124361OB-C31 funded by MCIN/ AEI/10.13039/501100011033 and by ERDF/EU).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Computational Multimodal Analysis of Polarized Political Discourse After the DANA in Valencia

  • Aloy-Mayo, Maria
  • Rosso, Paolo|||0000-0002-8922-1242
  • Soler-Sanz, Guillem
[EN] This paper aims to analyze sentiment and emotions in news articles and user comments following the devastating floods that took place in Valencia on October 29, 2024, which caused 229 deaths, extensive damage, and significant public concern. The study focuses on news articles about two key politicians: Carlos Maz & oacute;n, leader of the Valencian Popular Party (PP) and President of the Generalitat Valenciana, and Teresa Ribera, Third Vice President of Spain and former Minister for the Ecological Transition (PSOE). Our corpora comprise news articles and their user comments published during the 15-day window following the disaster (2024-10-30 to 2024-11-13) across four Spanish newspapers with different political orientations: El Pa & iacute;s, elDiario.es, El Mundo, and El Confidencial. The news articles were automatically extracted and preprocessed to standardize their format and prepare them for computational analysis. The images of these news articles were also included for a preliminary multimodal analysis approach to explore the semantic relations between linguistic structures and images. Emotion analysis was carried out employing Artificial Intelligence models to quantify the emotional load of news articles and comments from online readers. Large Language Models were used to automatically detect emotions expressed through linguistic expressions targeting both politicians.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Exploring LLMs Applications in Law: A Literature Review on Current Legal NLP Approaches

  • Siino, Marco
  • Falco, Mariana
  • Croce, Daniele
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] Artificial Intelligence (AI) is reshaping the legal landscape, with software tools now impacting various aspects of legal work. The intersection of Natural Language Processing (NLP) and law holds potential to transform how legal professionals, including lawyers and judges, operate, resolve disputes, and retrieve case information to formulate their decisions. To identify the current state of the applications of Transformers (also known as Large Language Models or LLMs) in the legal domain, we analysed the existing literature from 2017 to 2023 through a database search and snowballing method. From 61 selected publications, we identified key application categories such as legal document analysis, case prediction, and contract review, along with their main characteristics. We observed a discernible upsurge in the volume of scholarly publications, a diversification of tasks undertaken (e.g., legal research, contract analysis, and regulatory compliance), and an increased range of languages considered. There has been a notable enhancement in the methodological sophistication employed by researchers in practical applications. The performance of models grounded in the Generative Pre-trained Transformer (GPT) architecture has consistently improved across various legal domains, including contract review, legal document summarization, and case outcome prediction. This paper makes several significant contributions to the field. Firstly, it identifies emerging trends in the application of LLMs within the legal domain, highlighting the growing interest and investment in this area. Secondly, it pinpoints methodological gaps in current research, suggesting areas where further development and refinement are needed. Lastly, it discusses the broader implications of these advancements for real-world legal tasks, offering insights into how LLM-based AI can enhance legal practice while addressing the associated challenges., This work was supported in part by the European Union through the Italian Ministero dell'Universita e della Ricerca (MUR) National Recovery and Resilience Plan (NRRP) of NextGenerationEU, partnership on "Telecommunications of the Future"(program "Research and Innovation on Future Telecommunications Systems and Networks, to make Italy more Smart (RESTART)"), within the project Net4Future under Grant PE0000001; in part by the research project Malicious Actors Profiling and Detection in Online Social Networks through Artificial Intelligence (MARTINI) funded by Ministerio de Ciencia e Innovacion (MCIN)/Agencia Estatal de Investigacion (AEI)/10.13039/501100011033 under Grant PCI2022-135008-2; in part by the European Union NextGenerationEU/Plan de Recuperacion, Transformacion y Resiliencia (PRTR) and FairTransNLP-Stereotypes-Fairness and transparency for equitable NLP applications in social media: Identifying stereotypes and prejudices and developing equitable systems funded by MCIN/AEI/10.13039/501100011033 under Grant PID2021-124361OB-C31; and in part by the European Regional Development Fund (ERDF), EU, A way of making Europe.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Multimodal OXYmorons: A Comprehensive Introduction and Computational Analysis Using a Dataset of Oxymoronic Memes in Italian and Spanish

  • Di Palma, Eliana
  • Rizzi, Giulia
  • Masini, Francesca
  • Fersini, Elisabetta
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] This article introduces the concept of multimodal oxymorons. Multimodal oxymorons extend the traditional oxymoron theory by constructing and communicating meaning through the interplay of multiple modalities (such as visual and textual) rather than relying solely on language. We argue that multimodal oxymorons are central mechanisms of meaning-making in contemporary communication, as evidenced by the use of memes as an example. While textual oxymorons have long been the subject of analysis in order to ascertain their role in shaping thought and meaning, multimodal oxymorons demonstrate how human cognitive process transcends linguistic boundaries, integrating different modalities (e.g., visual) in order to convey complex ideas. To encourage further study, we present a curated multilingual dataset of Multimodal OXYmoron (MOXY), which can be used as a foundation for further analysis and experimentation. Furthermore, we propose a methodical approach for the identification of multimodal oxymorons along with a pipeline for automated generation. Through illustrative examples and a detailed methodology, this work establishes a comprehensive framework for understanding, identifying, and generating multimodal oxymorons, paving the way for advancements in computational linguistics, artificial intelligence, and figurative language studies., We acknowledge the support of the PNRR ICSC National Research Centre for High Performance Computing, Big Data and Quantum Computing (CN00000013), under the NRRP MUR program funded by the NextGenerationEU. The work of Elisabetta Fersini has been partially funded by MUR under the grant ReGAInS, Dipartimenti di Eccellenza 2023-2027 of the Department of Informatics, Systems and Communication at the University of Milano-Bicocca. The work of Paolo Rosso was in the framework of the FairTransNLP-Stereotypes research project, grant PID2021-124361OB-C31 funded by MCIN/AEI/10.13039/501100011033 and by ERDF/EU.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Backtranslate what you are saying and I will tell who you are

  • Siino, Marco
  • Lomonaco, Francesco
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] With this work, we hypothesize that semantically enriching a user's text corpus using backtranslation and expansion modules can improve performance for author profiling tasks. To perform this textual enrichment, we translate an author's representative text. Translations are made from one language-the source language-into another-the target language-and then back to the original one. Finally, we expand an author's text by integrating the original version with the back-translated one. Our framework includes these backtranslation and expansion modules followed by a SOTA classifier successfully employed for text classification. The framework is tested on three author profiling datasets from the last three years' shared tasks on fake news, hate speech, irony and stereotypes detection hosted at the CLEF conference for the PAN Lab. This work is an extension of our previous one where we just presented our main idea. Here we improve our framework, and we also investigate more languages and more datasets. Finally, a qualitative analysis is provided. The results confirm that the backtranslation and expansion add-on modules improve model performance on all three datasets evaluated., European Union NextGenerationEU/PRTR, Grant/Award Number: PCI2022-135008-2; ERDF, Grant/Award Number:PID2021-124361OB-C31
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Systematic keyword and bias analyses in hate speech detection

  • Peña-Sarracén, Gretel Liz de la
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] Hate speech detection refers broadly to the automatic identification of language that may be considered discriminatory against certain groups of people. The goal is to help online platforms to identify and remove harmful content. Humans are usually capable of detecting hatred in critical cases, such as when the hatred is non-explicit, but how do computer models address this situation? In this work, we aim to contribute to the understanding of ethical issues related to hate speech by analysing two transformer-based models trained to detect hate speech. Our study focuses on analysing the relationship between these models and a set of hateful keywords extracted from the three well-known datasets. For the extraction of the keywords, we propose a metric that takes into account the division among classes to favour the most common words in hateful contexts. In our experiments, we first compared the overlap between the extracted keywords with the words to which the models pay the most attention in decision-making. On the other hand, we investigate the bias of the models towards the extracted keywords. For the bias analysis, we characterize and use two metrics and evaluate two strategies to try to mitigate the bias. Surprisingly, we show that over 50% of the salient words of the models are not hateful and that there is a higher number of hateful words among the extracted keywords. However, we show that the models appear to be biased towards the extracted keywords. Experimental results suggest that fitting models with hateful texts that do not contain any of the keywords can reduce bias and improve the performance of the models., This work was done in the framework of the research project on Fairness and Transparency for equitable NLP applications in social media, funded by MCIN/AEI/10.13039/501100011033 and by ERDF, EU A way of making EuropePI.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

The Expression of Happiness in Social Media of Individuals Reporting Depression

  • Bucur, Ana-Maria
  • Chulvi, Berta
  • Cosma, Adrian
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] Depression has long been studied in the NLP field, with most works focusing on individuals¿ negative emotions. People with depression experience happiness, but this has not been extensively studied. Previous works have shown that sentiment or emotion classification approaches are unsuitable for extracting happy moments because they may not be expressed only in positive words. In this work, we conduct a large-scale study of happy moments from social media texts of individuals mentioninga depression diagnosis. We develop an extensive deep learning-based framework to extract happy moments from text, and annotate them with semantic topics, gender labels, and agency and sociality measures. We analyze over 400,000 happy moments and show significant differences in topics, agency, and sociality of users in the depression and control groups, varying by gender. We found that male and female users in the depression group expressed more sociality in their happy moments than control users. Furthermore, male users¿ agency was not impaired in depression, while female users in the depression group expressed fewer happy moments with agency than the control group. Our research can inform psychology interventions, which can foster feelings of longer-lasting happiness and represent apromising path of collaboration between computational linguistics and psychology., The work of Paolo Rosso was supported by the framework of the FairTransNLP research project under Grant PID2021-124361OB-C31 and Grant MCIN/AEI/10.13039/501100011033 and by ERDF, EU A way of making EuropePI.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Emotions and News Structure: An Analysis of the Language of Fake News in Spanish

  • Togni, Benedetta
  • Coll, Mariona
  • Chulvi-Ferriols, María Alberta
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] Research has repeatedly demonstrated that fake news tend to appeal to the emotions, but it is less common to consider the presence of emotions in relation to the inverted pyramid, a key aspect of journalistic writing. In this work, we adopt an existing neural model for fake news detection, FakeFlow, one of the few approaches to consider the salience of emotions with respect to the structure of news articles, and adapt it for Spanish. We conduct our experiments on the Spanish Fake News Corpus, introduced at the Fake News Detection in Spanish shared task (FakeDeS), with the goal of gaining a better understanding of the characteristics underlying such texts. In our analyses, we show that both the distribution of affective features and the attention mechanism of the model validate the importance of considering the inverted pyramid structure for detecting fake news., This work was funded by the research project FAKEnHATE-PdC: FAKE news and HATE speech Proof of Concept, funded by MCIN/AEI/10.13039/501100011033 and by European Union NextGenerationEU/PRTR. The work of Paolo Rosso was in the framework of the Iberifier Plus project, co-funded by the European Commission under the Call DIGITAL-2023- DEPLOY-04, European Digital Media observatory (EDMO)-National and multinational hubs (reference 101158511), and the MARTINI project (Malicious Actors Profiling and Detection in Online Social Networks Through Arificial Intelligence), grant PCI2022-135008-2, funded by MCIN/AEI/10.13039/501100011033 and by European Union NextGenerationEU/PRTR. The work of Mariona Coll was in the framework of the research project FairTransNLP on Fairness and Transparency for equitable NLP applications in social media, grant PID2021-124361OB-C31, funded by MCIN/AEI/10.13039/501100011033 and by ERDF, EU A way of making Europe. The work of Berta Chulvi was conducted while at Universitat Politècnica de València.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Politicization of Immigration and Language Use in Political Elites: A Study of Spanish Parliamentary Speeches

  • Chulvi-Ferriols, María Alberta
  • Molpeceres, Mariangeles
  • Rodrigo, María
  • Toselli, Alejandro Héctor
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] This study uses natural language processing (NLP) tools to analyze how politicians rec reate the stereotype of immigrants in the Spanish Parliament. An interdisciplinary approach from computational linguistics and social psychology has been used to con struct a variety of indices about content and linguistic styles. The analysis of 2,516 par liamentary interventions about immigration delivered between 1996 and 2016 by representatives of the two political parties that alternated in power during that period (conservative Popular Party and Spanish Socialist Party) shows that both the rhetorical strategy to present immigrants as ¿victims¿ or as a ¿threat¿ and the language style that politicians use reveal an interaction between the ideology of the party and the party¿s political position in government or in the opposition. Results also suggest some changes over time in the polarization and politicization of the debate about immigration., Alejandro Toselli is supported by a María Zambrano grant from the Spanish Ministerio de Universidades and the European Union NextGenerationEU/PRT. Berta Chulvi and Paolo Rosso are supported by FairTransNLP-Stereotypes PID2021 124361OB-C31 funded by MCIN/AEI/10.13039/501100011033 and by ERDF, EU A way of making Europe
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Everybody Hurts, Sometimes Overview of HUrtful HUmour at IberLEF 2023: Detection of Humour Spreading Prejudice in Twitter, Todos herimos, a veces Resumen de HUHU en IberLEF 2023: Detección de Humor que Difunde Prejuicios en Twitter

  • Labadie Tamayo, Roberto
  • Chulvi-Ferriols, María Alberta
  • Rosso, Paolo|||0000-0002-8922-1242
[ES] El humor es una estrategia eficiente para propagar prejuicios porque, la mayoría de las veces, elude el juicio moral. Sin embargo, perpetúa estereotipos y, al hacerlo, justifica actos discriminatorios. En HUHU roponemos un marco para estudiar cómo el humor se utiliza para discriminar a las minorías y analizar su interacción con el grado de prejuicio expresado contra grupos específicos. Con este fin, proporcionamos un corpus de tweets prejuiciosos en español anotados en cuanto a la presencia de humor, su grado de prejuicio y los grupos de: mujeres y feministas, comunidad LGBTI+, inmigrantes y personas discriminadas racialmente, así como personas con sobrepeso. Este artículo analiza los resultados obtenidos por los 46 equipo que participaron en HUHU., [EN] Humour is an efficient strategy to spread prejudice because, most of the time, it evades moral judgement. However, it perpetuates stereotypes and doing so justifies discriminatory acts. At HUHU we propose a frame to study how humour is used to discriminate minorities and to analyse their interplay with the degree of prejudice expressed against specific groups. To this end, we provide a corpus of prejudiced tweets in Spanish annotated with the presence of humour, its prejudice degree and the targeted groups: women and feminists, the LGBTI+ community, immigrants and racially discriminated people, and over-weighted people. This paper analyses the results achieved by the 46 teams that participated in HUHU., This work has been partially developed with the support of valgrAI - Valencian Graduate School and Research Network of Artificial Intelligence and the Generalitat Valenciana, and co-funded by the European Union. The work of Berta Chulvi and Paolo Rosso is supported by FairTransNLP-Stereotypes PID2021-124361OB-C31 funded by MCIN/AEI/10.13039/501100011033 and by ERDF, EU A way of making Europe.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Towards Robust Models for Fake News Detection in Spanish

  • Gómez-González, Sergio|||0009-0006-0061-8056
  • Rosso, Paolo|||0000-0002-8922-1242
  • Coll-Ardanuy, María
[ES] En este artículo nos enfrentamos al reto de detectar noticias falsas exclusivamente en español, un campo en el cual no ha habido demasiado esfuerzo de investigación. Además, la temática de las noticias se mantiene en continuo cambio, por lo que los modelos que no pueden adaptarse acaban siendo poco efectivos a largo plazo. Es por ello que, en este campo, la robustez es imprescindible. En búsqueda de esa propiedad, hemos aplicado distintas técnicas de explotación y aumento de datos para mejorar los resultados de un clasificador simple basado en un transformer preentrenado. Adicionalmente, hemos incluido una comparativa con un gran modelo de lenguaje generativo. También, utilizamos dos particiones distintas de un mismo dataset para comparar su efectividad: una partición típica con conjuntos de entrenamiento y test parecidos y otra más realista (adversaria). Finalmente, analizamos qué aspectos ejercen mayor influencia sobre la robustez y efectividad de los modelos para la detección de noticias falsas., This work has been developed under the project FAKEnHATE-PdC (PDC2022- 133118-I00) with founding from European NextGenerationEU/PRTR. The work of Mariona Coll Ardanuy was done at the UPV under FairTransNLP Grant PID2021-124361OB-C31 funded by MCIN/AEI/10.13039/501100011033 and by ERDF/EU.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Overview of AuTexTification at IberLEF 2023: Detection and Attribution of Machine-Generated Text in Multiple Domains, Resumen de AuTexTification en IberLEF 2023: Detección y Atribución de Texto Generado Automáticamente en Múltiples Dominios

  • Sarvazyan, Areg Mikael
  • González, José Ángel
  • Franco-Salvador, Marc
  • Rangel, Francisco
  • Chulvi-Ferriols, María Alberta
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] This paper presents the overview of the AuTexTification shared task as part of the IberLEF 2023 Workshop in Iberian Languages Evaluation Forum, within the framework of the SEPLN 2023 conference. AuTexTification consists of two subtasks: for Subtask 1, participants had to determine whether a text is human-authored or has been generated by a large language model. For Subtask 2, participants had to attribute a machine-generated text to one of six different text generation models. Our AuTexTification 2023 dataset contains more than 160.000 texts across two languages (English and Spanish) and five domains (tweets, reviews, news, legal, and how-to articles). A total of 114 teams signed up to participate, of which 36 sent 175 runs, and 20 of them sent their working notes. In this overview, we present the AuTexTification dataset and task, the submitted participating systems, and the results., [ES] Este artículo presenta un resumen de la tarea AuTexTification como parte del workshop IberLEF 2023 sobre el Iberian Languages Evaluation Forum, en el marco de la conferencia SEPLN 2023. AuTexTification consta de dos subtareas: en la Subtarea 1, los participantes tuvieron que determinar si un texto fue escrito por un humano o generado por un modelo de lenguaje masivo. Para la Subtarea 2, los participantes debían atribuir un texto generado automáticamente a uno de seis modelos de generación de texto diferentes. El conjunto de datos AuTexTification contiene más de 160.000 textos en dos idiomas (inglés y español) y cinco dominios (tweets, reseñas, noticias, legislación y artículos instructivos). Un total de 114 equipos se inscribieron para participar, de los cuales 36 enviaron 175 resultados y 20 de ellos enviaron artículos. En este artículo, presentamos el conjunto de datos y la tarea AuTexTification, los sistemas enviados por los participantes y sus resultados., The work from Symanto has been partially funded by the Pro<SUP>2</SUP>Haters - Proactive Profiling of Hate Speech Spreaders (CDTi IDI-20210776), the XAI-DisInfodemics: eXplainable AI for disinformation and conspiracy detection during infodemics (MICIN PLEC2021-007681), the OBULEX - OBservatorio del Uso de Lenguage sEXista en la red (IVACE IMINOD/2022/106), and the ANDHI - ANomalous Diffusion of Harmful Information (CPP2021-008994) R&D grants. The work of Areg Mikael Sarvazyan has been partially developed with the support of valgrAI - Valencian Graduate School and Research Network of Artificial Intelligence and the Generalitat Valenciana, and co-founded by the European Union. The research at the Universitat Politecnica de Valencia was framed under the FairTransNLP research project, Grant PID2021-124361OB-C31 funded by MCIN/AEI/10.13039/501100011033 and by ERDF, EU A way of making Europe.