Pasar al contenido principal

FAIRTRANSNLP-STEREOTYPES: IDENTIFICACION DE ESTEREOTIPOS Y PREJUICIOS Y DESARROLLO DE SISTEMAS EQUITATIVOS

PID2021-124361OB-C31

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal para Impulsar la Investigación Científico-Técnica y su Transferencia
Subprograma Subprograma Estatal de Generación de Conocimiento
Convocatoria Proyectos de I+D+i (Generación de Conocimiento y Retos Investigación)
Año convocatoria 2021
Plan Estatal 2021-2023 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSITAT POLITÈCNICA DE VALÈNCIA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 22
Encontrada(s) 1 página(s)

RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Identification of Racial and Sexist Stereotypes in Spanish: A Learning with Disagreements Approach, Identificación de Estereotipos Raciales y Sexistas en Español: Un Enfoque Basado en Aprendizaje con Desacuerdos

  • Urios Alacreu, Elias
  • Rosso, Paolo|||0000-0002-8922-1242
[ES] El discurso del odio ha proliferado significativamente en los ¿ultimos a¿nos, en gran medida impulsado por la adopci¿on generalizada de plataformas de redes sociales. El discurso de odio a menudo opera de manera impl¿¿cita, aprovechando estereotipos sutiles para propagar pensamientos discriminatorios. Estos mecanismos encubiertos han permitido que el contenido odioso se oculte a si mismo, haciendo que su detecci¿on resulte cada vez m¿as compleja. Como resultado, la lucha contra los discursos de odio se ha convertido en una prioridad urgente, impulsando la adopci¿on generalizada de modelos de aprendizaje profundo para detectar y combatir contenidos nocivos. Dada la naturaleza inherentemente subjetiva de los discursos del odio, es necesario desarrollar modelos que sean lo m¿as generalizables posible. Esto ha llevado a la aparici¿on del paradigma de aprendizaje con desacuerdos, que tiene como objetivo introducir desacuerdos dentro de la propia tarea para mejorar la generalizabilidad del modelo. Este trabajo investiga este ¿ultimo paradigma a trav¿es de dos shared tasks. La primera tarea, DETEST-Dis, explora los estereotipos contra los inmigrantes en comentarios en l¿¿nea y fue organizada en IberLEF 2024. Nuestros resultados se encuentran entre los mejores de todos los equipos participantes, superando los enfoques tradicionales. La segunda tarea, EXIST, se centra en el sexismo en los memes y fue organizada en CLEF 2024. En este caso, nuestro rendimiento presenta una mejor¿¿a a¿nadiendo caracter¿¿sticas de un modelo externo as¿¿ como tambi¿en data augmentation. Nuestro c¿odigo fuente se puede encontrar en https://github.com/Buzzeitor30/DETESTS-DIS y https://github.com/Buzzeitor30/EXIST-TFM, [EN] Hate speech has proliferated significantly in recent years, largely driven by the widespread adoption of social media platforms. Hate speech often operates implicitly, leveraging subtle stereotypes to propagate discriminatory views. These covert mechanisms allow harmful content to disguise itself, making detection increasingly complex. As a result, tackling hate speech has become an urgent priority, driving the widespread adoption of deep learning models to detect and combat harmful content. Given the inherently subjective nature of hate speech and its nuanced manifestations, there is a need to develop models that are as generalizable as possible. This has led to the emergence of the learning with disagreements paradigm, which aims to introduce disagreements within the task itself to enhance model generalizability. This paper investigates the latter paradigm through two shared tasks. The first task, DETEST-Dis, explores stereotypes against immigrants in online comments and was organized at IberLEF 2024. Our results are among the best of all participating teams, surpassing traditional approaches. The second task, EXIST, focuses on sexism in memes and was organized at CLEF 2024. Here, our performance is enhanced by adding features from an external model as well as data augmentation. Our source code can be found on https://github.com/Buzzeitor30/DETESTS-DIS and https://github.com/Buzzeitor30/EXIST-TFM., This work has been developed under the project FAKEnHATE-PdC (PDC2022-133118-I00) with founding from European NextGenerationEU/PRTR. EUR and PR carried out part of thier work also in the framework of the following projects respectively: MARTINI (Grant PCI2022-135008-2) and FairTransNLP-Stereotypes (Grant PID2021-124361OB-C31) both funded by MCIN/AEI/10.13039/501100011033 and by European Union NextGenerationEU/PRTR.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Recognizing misogynous memes: Biased models and tricky archetypes

  • Rizzi, Giulia
  • Gasparini, Francesca
  • Saibene, Aurora
  • Fersini, Elisabetta
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] Misogyny is a form of hate against women and has been spreading exponentially through the Web, especially on social media platforms. Hateful content towards women can be conveyed not only by text but also using visual and/or audio sources or their combination, highlighting the necessity to address it from a multimodal perspective. One of the predominant forms of multimodal content against women is represented by memes, which are images characterized by pictorial content with an overlaying text introduced a posteriori. Its main aim is originally to be funny and/or ironic, making misogyny recognition in memes even more challenging. In this paper, we investigated 4 unimodal and 3 multimodal approaches to determine which source of information contributes more to the detection of misogynous memes. Moreover, a bias estimation technique is proposed to identify specific elements that compose a meme that could lead to unfair models, together with a bias mitigation strategy based on Bayesian Optimization. The proposed method is able to push the prediction probabilities towards the correct class for up to 61.43% of the cases. Finally, we identified the most challenging archetypes of memes that are still far to be properly recognized, highlighting the most relevant open research directions., The work of Paolo Rosso was done in the framework of the FairTransNLP-Stereotypes research project on Fairness and Transparency for equitable NLP applications in social media: Identifying stereotypes and prejudices and developing equitable systems (PID2021-124361OB-C31) funded by MCIN/AEI/10.13039/501100011033 and by ERDF, EU A way of making Europe. The work of Elisabetta Fersini and Francesca Gasparini has been partially funded by the European Union - NextGenerationEU under the National Research Centre For HPC, Big Data and Quantum Computing-Spoke 9-Digital Society and Smart Cities (PNRR-MUR), and by REGAINS - Excellence Department Research Project.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Tesis doctoral (DoctoralThesis)

On the Multi-lingual and Multi-epidemic Detection of Misinformation in Health

  • Baris Schlicht, Ipek
[ES] El acceso a información en línea sobre la atención sanitaria es una fuente primaria para muchas personas. Además de esto, los modelos de lenguaje de gran tamaño (LLMs, por sus siglas en inglés) han comenzado a utilizarse como proveedores de información. Sin embargo, las fuentes de información en línea agravan los desafíos existentes relacionados con la desinformación, particularmente en contextos multilingües donde la disponibilidad y la calidad de los datos varían significativamente. La falta de una regulación consistente entre plataformas contribuye aún más a la difusión de información inexacta o sesgada. Asimismo, se sabe que los LLMs pueden generar "alucinaciones" y pueden presentar sesgos ocultos en distintos dominios y culturas. Esta tesis examina la desinformación en salud desde una perspectiva computacional, clasificándola en desinformación causada por humanos y desinformación causada por LLMs. Las epidemias representan un contexto especialmente crítico para el estudio de la desinformación causada por humanos, ya que dan lugar a "infodemias", caracterizadas por la rápida propagación de información excesiva y, a menudo, poco fiable entre comunidades. Durante una infodemia, resulta difícil encontrar información creíble. Por esta razón, la tesis se centra en contextos multilingües y en múltiples epidemias. Aborda desafíos relacionados con el multilingüismo, la escasez de datos y las limitaciones de recursos. Propone enfoques eficientes en el uso de recursos basados en el aprendizaje por transferencia, como métodos basados en adaptadores y el uso de LLMs para anotaciones. Los resultados experimentales muestran que los métodos basados en adaptadores pueden ser eficaces para la tarea de detección multilingüe de verificabilidad, y a menudo superan a los principales enfoques multilingües. Los LLMs y los modelos transformadores entrenados con conjuntos de datos etiquetados por LLMs muestran un rendimiento comparable al de aquellos entrenados con etiquetas de expertos. En cuanto a la desinformación en salud causada por LLMs, la tesis investiga las disparidades interlingüísticas en Wikipedia y en las respuestas de los LLMs para preguntas y respuestas multilingües. Los hallazgos muestran que las respuestas generadas por los LLMs a menudo omiten algunos aspectos de la información, como detalles de un estudio mencionado o directrices clínicas, en comparación con las respuestas en inglés a las mismas preguntas. Además, la tesis examina cómo las diferencias en las fuentes de datos de preentrenamiento pueden contribuir a estas disparidades. Con este fin, se utilizan páginas de Wikipedia sobre atención sanitaria como un sustituto de las fuentes de preentrenamiento. Los resultados muestran disparidades sustanciales en términos de coincidencia de hechos, referencias y estructura de la información. Adicionalmente, las respuestas de los LLMs son más similares a las páginas correspondientes de Wikipedia en inglés que a las páginas de Wikipedia en otros idiomas, incluso cuando las preguntas se formulan en idiomas distintos del inglés. Por último, las contribuciones de esta tesis incluyen el desarrollo de nuevos conjuntos de datos y marcos de evaluación, así como métodos eficientes en el uso de recursos para entornos con pocos recursos. La tesis consiste en una recopilación de artículos publicados en conferencias y revistas de primer nivel., The work of IBS and LF was supported by DynSoDA (funded by BMBF) and the Lamarr Institute for Machine Learning and Artificial Intelligence. The work of PR was supported by the MARTINI project on Malicious Actors Profiling and Detection in Online Social Networks Through Artificial Intelligence funded by MCIN/AEI/ 10.13039/501100011033 and by EU NextGenerationEU/PRTR. Views and opinions expressed are however those of the author(s) only and do not reflect those of the funding organizations, EU or HaDEA. We also thank Benedetta Togni for the annotations. The work of IBS and LF was supported by DynSoDA (funded by BMBF) and the Lamarr Institute for Machine Learning and Artificial Intelligence. The work of PR was supported by MCIN/AEI and by ERDF/EU (Grant No. PID2021-124361OB-C31). BS acknowledges support from the Horizon Europe project TANGO (Grant Agreement no. 101120763). Funded by the European Union. Views and opinions expressed are however those of the author(s) only and do not reflect those of the funding organizations, EU or HaDEA.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

On the Robustness of Transformer-Based Models to Different Linguistic Perturbations: A Case of Study in Irony Detection

  • Ortega-Bueno, Reynier
  • Fersini, Elisabetta
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] This study investigates the robustness of Transformer models in irony detection addressing various textual perturbations, revealing potential biases in training data concerning ironic and non-ironic classes. The perturbations involve three distinct approaches, each progressively increasing in complexity. The first approach is word masking, which employs wild-card characters or utilises BERT-specific masking through the mask token provided by BERT models. The second approach is word substitution, replacing the bias word with a contextually appropriate alternative. Lastly, paraphrasing generates a new phrase while preserving the original semantic meaning. We leverage Large Language Models (GPT 3.5 Turbo) and human inspection to ensure linguistic correctness and contextual coherence for word substitutions and paraphrasing. The results indicate that models are susceptible to these perturbations, and paraphrasing and word substitution demonstrate the most significant impact on model predictions. The irony class appears to be particularly challenging for models when subjected to these perturbations. The SHAP and LIME methods are used to correlate variations in attribution scores with prediction errors. A notable difference in the Total Variation of attribution scores is observed between original examples and cases involving bias word substitution or masking. Among the corpora used, TwSemEval2018 emerges as the most challenging. Regarding model performance, Transformer-based models such as RoBERTa and BERTweet demonstrate superior overall performance addressing these perturbations. This research contributes to understanding the robustness and limitations of irony detection models, highlighting areas for improvement in model design and training data curation., The work of Paolo Rosso was in the framework of the research project on Fairness and Transparency for equitable NLP applications in social media (grant PID2021-124361OB-C31 funded by MCIN/ AEI/10.13039/501100011033 and by ERDF/EU).
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Tesis doctoral (DoctoralThesis)

New Avenues in Computational Irony Detection in Social Media

  • Ortega Bueno, Reynier
[ES] La presente tesis doctoral, presentada como compendio de artículos científicos, aborda el complejo problema de la detección de ironía. Esta figura del lenguaje es un fenómeno figurativo y pragmático que elude el análisis lingüístico del significado literal debido a su dependencia del contexto y la intencionalidad. La dificultad del problema se intensifica en contextos de redes sociales, donde la ironía se manifiesta en dimensiones multilingües y multimodales, complicando su modelación computacional. En este sentido, esta investigación está motivada en dos direcciones: primero, las aplicaciones potenciales de la detección de ironía más allá del análisis de sentimientos, especialmente en la mitigación de problemas de comunicación y desinformación; segundo, las limitaciones existentes de los métodos automáticos para la detección eficaz de la ironía. Esta tesis explora tres direcciones principales. Inicialmente, se analiza la ironía y la sátira a través de variantes del español, introduciendo un nuevo método que integra conocimiento lingüístico y representaciones profundas dentro de un modelo recurrente neuronal dotado con mecanismos de atención, superando el rendimiento del estado del arte. Posteriormente, se extiende la investigación al escenario multimodal, proponiendo una nueva arquitectura que combina transformers textuales con transformers visuales, mejorando la precisión en la detección de ironía en contenidos de redes sociales basados en texto e imagen. Finalmente, se investiga el sesgo temático en corpus de ironía y su impacto en la generalización de modelos, adoptando dos estrategias: la identificación y la mitigación del sesgo. El análisis realizado revela que los corpus de ironía exhiben sesgos que tienen un impacto negativo en la generalización de los modelos. Consecuentemente, la mitigación de dicho sesgo en los datos de entrenamiento produce mejoras de rendimiento en diversos dominios e idiomas. Además, el estudio examina la sensibilidad de los modelos al sesgo temático, cuando este es modificado mediante variaciones lingüísticas en los datos de evaluación, confirmando una significativa vulnerabilidad de los modelos analizados. Experimentos complementarios exploran las implicaciones sociales de la ironía y el humor, subrayando sus efectos negativos cuando se utilizan indebidamente para atacar a grupos minoritarios o individuos. Al abordar dimensiones computacionales, lingüísticas y sociales, esta tesis avanza en la detección computacional de la ironía y su implicación en la comunicación digital., [CA] La present tesi doctoral, presentada com a compendi d'articles científics, aborda el complex problema de la detecció de la ironia. Aquesta figura del llenguatge és un fenomen figuratiu i pragmàtic que eludeix l'anàlisi lingüística del significat literal a causa de la seua dependència del context i la intencionalitat. La dificultat del problema s'intensifica en contextos de xarxes socials, on la ironia es manifesta en dimensions multilingües i multimodals, complicant la seua modelització computacional. En aquest sentit, aquesta investigació està motivada en dues direccions: primer, les aplicacions potencials de la detecció de la ironia més enllà de l'anàlisi de sentiments, especialment en la mitigació de problemes de comunicació i desinformació; segon, les limitacions existents dels mètodes automàtics per a la detecció eficaç de la ironia. Aquesta tesi explora tres direccions principals. Inicialment, s'analitza la ironia i la sàtira a través de variants de l'espanyol, introduint un nou mètode que integra coneixement lingüístic i representacions profundes dins d'un model neuronal recurrent dotat amb mecanismes d'atenció, superant el rendiment de l'estat de l'art. Posteriorment, s'estén la investigació a l'escenari multimodal, proposant una nova arquitectura que combina transformers textuals amb transformers visuals, millorant la precisió en la detecció de la ironia en continguts de xarxes socials basats en text i imatge. Finalment, s'investiga el biaix temàtic en corpus d'ironia i el seu impacte en la generalització dels models, adoptant dues estratègies: la identificació i la mitigació del biaix. L'anàlisi realitzada revela que els corpus d'ironia exhibeixen biaixos que tenen un impacte negatiu en la generalització dels models. Conseqüentment, la mitigació d'aquest biaix en les dades d'entrenament produeix millores de rendiment en diversos dominis i idiomes. A més, l'estudi examina la sensibilitat dels models al biaix temàtic, quan aquest és modificat mitjançant variacions lingüístiques en les dades d'avaluació, confirmant una significativa vulnerabilitat dels models analitzats. Experiments complementaris exploren les implicacions socials de la ironia i l'humor, subratllant els seus efectes negatius quan s'utilitzen indegudament per a atacar grups minoritaris o individus. En abordar dimensions computacionals, lingüístiques i socials, aquesta tesi avança en la detecció computacional de la ironia i la seua implicació en la comunicació digital., [EN] This doctoral thesis, presented as a compendium of articles, addresses the complex challenge of irony detection. Irony is a figurative and pragmatic phenomenon that eludes straightforward linguistic analysis due to its reliance on context and intent. The difficulty intensifies in social media contexts, where irony manifests itself in multilingual and multimodal dimensions, complicating computational modelling. The motivation of this research is twofold: first, the potential applications of irony detection beyond sentiment analysis, notably in mitigating miscommunication and misinformation; second, the persistent computational gap in accurately identifying irony. The thesis explores three main directions. Initially, it analyses irony and satire across Spanish variants, introducing a novel method that integrates linguistic features and deep representations within an attentive LSTM model, surpassing state-of-the-art performance in irony and satire detection. Subsequently, it extends the investigation to multimodal scenarios, proposing an end-to-end architecture that fuses textual transformers with visual transformers, enhancing accuracy in detecting irony across text- and image-based social media content. Finally, it investigates topic bias in irony corpora and its impact on model generalisation, considering two perspectives: bias identification and mitigation. Analysis reveals that irony corpora exhibit bias, which impairs model generalisation; mitigating this bias in training data yields performance improvements across domains and languages. Furthermore, the study examines the model sensitivity to bias altered by linguistic perturbations in the evaluation data, confirming a significant vulnerability. Complementary experiments explore the social implications of irony and humour, underscoring their negative effects when misused to target minorities or individuals. By addressing linguistic, computational, and social dimensions, this thesis advances computational irony detection and its broader implications in digital communication., The work of the first two authors was in the framework of the research project MISMIS-FAKEnHATE on MISinformation and MIScommunication in social media: FAKE news and HATE speech (PGC2018-096212-B-C31), funded by Spanish Ministry of Science and Innovation, and DeepPattern (PROMETEO/2019/121), funded by the Generalitat Valenciana. The work of Paolo Rosso was in the framework of the research project on Fairness and Transparency for equitable NLP applications in social media (grant PID2021-124361OB-C31 funded by MCIN/AEI/10.13039/501100011033 and by ERDF/EU)
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Computational Multimodal Analysis of Polarized Political Discourse After the DANA in Valencia

  • Aloy-Mayo, Maria
  • Rosso, Paolo|||0000-0002-8922-1242
  • Soler-Sanz, Guillem
[EN] This paper aims to analyze sentiment and emotions in news articles and user comments following the devastating floods that took place in Valencia on October 29, 2024, which caused 229 deaths, extensive damage, and significant public concern. The study focuses on news articles about two key politicians: Carlos Maz & oacute;n, leader of the Valencian Popular Party (PP) and President of the Generalitat Valenciana, and Teresa Ribera, Third Vice President of Spain and former Minister for the Ecological Transition (PSOE). Our corpora comprise news articles and their user comments published during the 15-day window following the disaster (2024-10-30 to 2024-11-13) across four Spanish newspapers with different political orientations: El Pa & iacute;s, elDiario.es, El Mundo, and El Confidencial. The news articles were automatically extracted and preprocessed to standardize their format and prepare them for computational analysis. The images of these news articles were also included for a preliminary multimodal analysis approach to explore the semantic relations between linguistic structures and images. Emotion analysis was carried out employing Artificial Intelligence models to quantify the emotional load of news articles and comments from online readers. Large Language Models were used to automatically detect emotions expressed through linguistic expressions targeting both politicians.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Exploring LLMs Applications in Law: A Literature Review on Current Legal NLP Approaches

  • Siino, Marco
  • Falco, Mariana
  • Croce, Daniele
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] Artificial Intelligence (AI) is reshaping the legal landscape, with software tools now impacting various aspects of legal work. The intersection of Natural Language Processing (NLP) and law holds potential to transform how legal professionals, including lawyers and judges, operate, resolve disputes, and retrieve case information to formulate their decisions. To identify the current state of the applications of Transformers (also known as Large Language Models or LLMs) in the legal domain, we analysed the existing literature from 2017 to 2023 through a database search and snowballing method. From 61 selected publications, we identified key application categories such as legal document analysis, case prediction, and contract review, along with their main characteristics. We observed a discernible upsurge in the volume of scholarly publications, a diversification of tasks undertaken (e.g., legal research, contract analysis, and regulatory compliance), and an increased range of languages considered. There has been a notable enhancement in the methodological sophistication employed by researchers in practical applications. The performance of models grounded in the Generative Pre-trained Transformer (GPT) architecture has consistently improved across various legal domains, including contract review, legal document summarization, and case outcome prediction. This paper makes several significant contributions to the field. Firstly, it identifies emerging trends in the application of LLMs within the legal domain, highlighting the growing interest and investment in this area. Secondly, it pinpoints methodological gaps in current research, suggesting areas where further development and refinement are needed. Lastly, it discusses the broader implications of these advancements for real-world legal tasks, offering insights into how LLM-based AI can enhance legal practice while addressing the associated challenges., This work was supported in part by the European Union through the Italian Ministero dell'Universita e della Ricerca (MUR) National Recovery and Resilience Plan (NRRP) of NextGenerationEU, partnership on "Telecommunications of the Future"(program "Research and Innovation on Future Telecommunications Systems and Networks, to make Italy more Smart (RESTART)"), within the project Net4Future under Grant PE0000001; in part by the research project Malicious Actors Profiling and Detection in Online Social Networks through Artificial Intelligence (MARTINI) funded by Ministerio de Ciencia e Innovacion (MCIN)/Agencia Estatal de Investigacion (AEI)/10.13039/501100011033 under Grant PCI2022-135008-2; in part by the European Union NextGenerationEU/Plan de Recuperacion, Transformacion y Resiliencia (PRTR) and FairTransNLP-Stereotypes-Fairness and transparency for equitable NLP applications in social media: Identifying stereotypes and prejudices and developing equitable systems funded by MCIN/AEI/10.13039/501100011033 under Grant PID2021-124361OB-C31; and in part by the European Regional Development Fund (ERDF), EU, A way of making Europe.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Multimodal OXYmorons: A Comprehensive Introduction and Computational Analysis Using a Dataset of Oxymoronic Memes in Italian and Spanish

  • Di Palma, Eliana
  • Rizzi, Giulia
  • Masini, Francesca
  • Fersini, Elisabetta
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] This article introduces the concept of multimodal oxymorons. Multimodal oxymorons extend the traditional oxymoron theory by constructing and communicating meaning through the interplay of multiple modalities (such as visual and textual) rather than relying solely on language. We argue that multimodal oxymorons are central mechanisms of meaning-making in contemporary communication, as evidenced by the use of memes as an example. While textual oxymorons have long been the subject of analysis in order to ascertain their role in shaping thought and meaning, multimodal oxymorons demonstrate how human cognitive process transcends linguistic boundaries, integrating different modalities (e.g., visual) in order to convey complex ideas. To encourage further study, we present a curated multilingual dataset of Multimodal OXYmoron (MOXY), which can be used as a foundation for further analysis and experimentation. Furthermore, we propose a methodical approach for the identification of multimodal oxymorons along with a pipeline for automated generation. Through illustrative examples and a detailed methodology, this work establishes a comprehensive framework for understanding, identifying, and generating multimodal oxymorons, paving the way for advancements in computational linguistics, artificial intelligence, and figurative language studies., We acknowledge the support of the PNRR ICSC National Research Centre for High Performance Computing, Big Data and Quantum Computing (CN00000013), under the NRRP MUR program funded by the NextGenerationEU. The work of Elisabetta Fersini has been partially funded by MUR under the grant ReGAInS, Dipartimenti di Eccellenza 2023-2027 of the Department of Informatics, Systems and Communication at the University of Milano-Bicocca. The work of Paolo Rosso was in the framework of the FairTransNLP-Stereotypes research project, grant PID2021-124361OB-C31 funded by MCIN/AEI/10.13039/501100011033 and by ERDF/EU.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Backtranslate what you are saying and I will tell who you are

  • Siino, Marco
  • Lomonaco, Francesco
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] With this work, we hypothesize that semantically enriching a user's text corpus using backtranslation and expansion modules can improve performance for author profiling tasks. To perform this textual enrichment, we translate an author's representative text. Translations are made from one language-the source language-into another-the target language-and then back to the original one. Finally, we expand an author's text by integrating the original version with the back-translated one. Our framework includes these backtranslation and expansion modules followed by a SOTA classifier successfully employed for text classification. The framework is tested on three author profiling datasets from the last three years' shared tasks on fake news, hate speech, irony and stereotypes detection hosted at the CLEF conference for the PAN Lab. This work is an extension of our previous one where we just presented our main idea. Here we improve our framework, and we also investigate more languages and more datasets. Finally, a qualitative analysis is provided. The results confirm that the backtranslation and expansion add-on modules improve model performance on all three datasets evaluated., European Union NextGenerationEU/PRTR, Grant/Award Number: PCI2022-135008-2; ERDF, Grant/Award Number:PID2021-124361OB-C31
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Systematic keyword and bias analyses in hate speech detection

  • Peña-Sarracén, Gretel Liz de la
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] Hate speech detection refers broadly to the automatic identification of language that may be considered discriminatory against certain groups of people. The goal is to help online platforms to identify and remove harmful content. Humans are usually capable of detecting hatred in critical cases, such as when the hatred is non-explicit, but how do computer models address this situation? In this work, we aim to contribute to the understanding of ethical issues related to hate speech by analysing two transformer-based models trained to detect hate speech. Our study focuses on analysing the relationship between these models and a set of hateful keywords extracted from the three well-known datasets. For the extraction of the keywords, we propose a metric that takes into account the division among classes to favour the most common words in hateful contexts. In our experiments, we first compared the overlap between the extracted keywords with the words to which the models pay the most attention in decision-making. On the other hand, we investigate the bias of the models towards the extracted keywords. For the bias analysis, we characterize and use two metrics and evaluate two strategies to try to mitigate the bias. Surprisingly, we show that over 50% of the salient words of the models are not hateful and that there is a higher number of hateful words among the extracted keywords. However, we show that the models appear to be biased towards the extracted keywords. Experimental results suggest that fitting models with hateful texts that do not contain any of the keywords can reduce bias and improve the performance of the models., This work was done in the framework of the research project on Fairness and Transparency for equitable NLP applications in social media, funded by MCIN/AEI/10.13039/501100011033 and by ERDF, EU A way of making EuropePI.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

The Expression of Happiness in Social Media of Individuals Reporting Depression

  • Bucur, Ana-Maria
  • Chulvi, Berta
  • Cosma, Adrian
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] Depression has long been studied in the NLP field, with most works focusing on individuals¿ negative emotions. People with depression experience happiness, but this has not been extensively studied. Previous works have shown that sentiment or emotion classification approaches are unsuitable for extracting happy moments because they may not be expressed only in positive words. In this work, we conduct a large-scale study of happy moments from social media texts of individuals mentioninga depression diagnosis. We develop an extensive deep learning-based framework to extract happy moments from text, and annotate them with semantic topics, gender labels, and agency and sociality measures. We analyze over 400,000 happy moments and show significant differences in topics, agency, and sociality of users in the depression and control groups, varying by gender. We found that male and female users in the depression group expressed more sociality in their happy moments than control users. Furthermore, male users¿ agency was not impaired in depression, while female users in the depression group expressed fewer happy moments with agency than the control group. Our research can inform psychology interventions, which can foster feelings of longer-lasting happiness and represent apromising path of collaboration between computational linguistics and psychology., The work of Paolo Rosso was supported by the framework of the FairTransNLP research project under Grant PID2021-124361OB-C31 and Grant MCIN/AEI/10.13039/501100011033 and by ERDF, EU A way of making EuropePI.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Emotions and News Structure: An Analysis of the Language of Fake News in Spanish

  • Togni, Benedetta
  • Coll, Mariona
  • Chulvi-Ferriols, María Alberta
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] Research has repeatedly demonstrated that fake news tend to appeal to the emotions, but it is less common to consider the presence of emotions in relation to the inverted pyramid, a key aspect of journalistic writing. In this work, we adopt an existing neural model for fake news detection, FakeFlow, one of the few approaches to consider the salience of emotions with respect to the structure of news articles, and adapt it for Spanish. We conduct our experiments on the Spanish Fake News Corpus, introduced at the Fake News Detection in Spanish shared task (FakeDeS), with the goal of gaining a better understanding of the characteristics underlying such texts. In our analyses, we show that both the distribution of affective features and the attention mechanism of the model validate the importance of considering the inverted pyramid structure for detecting fake news., This work was funded by the research project FAKEnHATE-PdC: FAKE news and HATE speech Proof of Concept, funded by MCIN/AEI/10.13039/501100011033 and by European Union NextGenerationEU/PRTR. The work of Paolo Rosso was in the framework of the Iberifier Plus project, co-funded by the European Commission under the Call DIGITAL-2023- DEPLOY-04, European Digital Media observatory (EDMO)-National and multinational hubs (reference 101158511), and the MARTINI project (Malicious Actors Profiling and Detection in Online Social Networks Through Arificial Intelligence), grant PCI2022-135008-2, funded by MCIN/AEI/10.13039/501100011033 and by European Union NextGenerationEU/PRTR. The work of Mariona Coll was in the framework of the research project FairTransNLP on Fairness and Transparency for equitable NLP applications in social media, grant PID2021-124361OB-C31, funded by MCIN/AEI/10.13039/501100011033 and by ERDF, EU A way of making Europe. The work of Berta Chulvi was conducted while at Universitat Politècnica de València.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Politicization of Immigration and Language Use in Political Elites: A Study of Spanish Parliamentary Speeches

  • Chulvi-Ferriols, María Alberta
  • Molpeceres, Mariangeles
  • Rodrigo, María
  • Toselli, Alejandro Héctor
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] This study uses natural language processing (NLP) tools to analyze how politicians rec reate the stereotype of immigrants in the Spanish Parliament. An interdisciplinary approach from computational linguistics and social psychology has been used to con struct a variety of indices about content and linguistic styles. The analysis of 2,516 par liamentary interventions about immigration delivered between 1996 and 2016 by representatives of the two political parties that alternated in power during that period (conservative Popular Party and Spanish Socialist Party) shows that both the rhetorical strategy to present immigrants as ¿victims¿ or as a ¿threat¿ and the language style that politicians use reveal an interaction between the ideology of the party and the party¿s political position in government or in the opposition. Results also suggest some changes over time in the polarization and politicization of the debate about immigration., Alejandro Toselli is supported by a María Zambrano grant from the Spanish Ministerio de Universidades and the European Union NextGenerationEU/PRT. Berta Chulvi and Paolo Rosso are supported by FairTransNLP-Stereotypes PID2021 124361OB-C31 funded by MCIN/AEI/10.13039/501100011033 and by ERDF, EU A way of making Europe
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Everybody Hurts, Sometimes Overview of HUrtful HUmour at IberLEF 2023: Detection of Humour Spreading Prejudice in Twitter, Todos herimos, a veces Resumen de HUHU en IberLEF 2023: Detección de Humor que Difunde Prejuicios en Twitter

  • Labadie Tamayo, Roberto
  • Chulvi-Ferriols, María Alberta
  • Rosso, Paolo|||0000-0002-8922-1242
[ES] El humor es una estrategia eficiente para propagar prejuicios porque, la mayoría de las veces, elude el juicio moral. Sin embargo, perpetúa estereotipos y, al hacerlo, justifica actos discriminatorios. En HUHU roponemos un marco para estudiar cómo el humor se utiliza para discriminar a las minorías y analizar su interacción con el grado de prejuicio expresado contra grupos específicos. Con este fin, proporcionamos un corpus de tweets prejuiciosos en español anotados en cuanto a la presencia de humor, su grado de prejuicio y los grupos de: mujeres y feministas, comunidad LGBTI+, inmigrantes y personas discriminadas racialmente, así como personas con sobrepeso. Este artículo analiza los resultados obtenidos por los 46 equipo que participaron en HUHU., [EN] Humour is an efficient strategy to spread prejudice because, most of the time, it evades moral judgement. However, it perpetuates stereotypes and doing so justifies discriminatory acts. At HUHU we propose a frame to study how humour is used to discriminate minorities and to analyse their interplay with the degree of prejudice expressed against specific groups. To this end, we provide a corpus of prejudiced tweets in Spanish annotated with the presence of humour, its prejudice degree and the targeted groups: women and feminists, the LGBTI+ community, immigrants and racially discriminated people, and over-weighted people. This paper analyses the results achieved by the 46 teams that participated in HUHU., This work has been partially developed with the support of valgrAI - Valencian Graduate School and Research Network of Artificial Intelligence and the Generalitat Valenciana, and co-funded by the European Union. The work of Berta Chulvi and Paolo Rosso is supported by FairTransNLP-Stereotypes PID2021-124361OB-C31 funded by MCIN/AEI/10.13039/501100011033 and by ERDF, EU A way of making Europe.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Tesis doctoral (DoctoralThesis)

On the Keyword Extraction and Bias Analysis, Graph-based Exploration and Data Augmentation for Abusive Language Detection in Low-Resource Settings

  • Peña Sarracén, Gretel Liz de la
[ES] La detección del lenguaje abusivo es una tarea que se ha vuelto cada vez más importante en la era digital moderna, donde la comunicación se produce a través de diversas plataformas en línea. El aumento de las interacciones en estas plataformas ha provocado un aumento de la aparición del lenguaje abusivo. Abordar dicho contenido es crucial para mantener un entorno en línea seguro e inclusivo. Sin embargo, esta tarea enfrenta varios desafíos que la convierten en un área compleja y que demanda de continua investigación y desarrollo. En particular, detectar lenguaje abusivo en entornos con escasez de datos presenta desafíos adicionales debido a que el desarrollo de sistemas automáticos precisos a menudo requiere de grandes conjuntos de datos anotados. En esta tesis investigamos diferentes aspectos de la detección del lenguaje abusivo, prestando especial atención a entornos con datos limitados. Primero, estudiamos el sesgo hacia palabras clave abusivas en modelos entrenados para la detección del lenguaje abusivo. Con este propósito, proponemos dos métodos para extraer palabras clave potencialmente abusivas de colecciones de textos. Luego evaluamos el sesgo hacia las palabras clave extraídas y cómo se puede modificar este sesgo para influir en el rendimiento de la detección del lenguaje abusivo. El análisis y las conclusiones de este trabajo revelan evidencia de que es posible mitigar el sesgo y que dicha reducción puede afectar positivamente el desempeño de los modelos. Sin embargo, notamos que no es posible establecer una correspondencia similar entre la variación del sesgo y el desempeño de los modelos cuando hay escasez datos con las técnicas de reducción del sesgo estudiadas. En segundo lugar, investigamos el uso de redes neuronales basadas en grafos para detectar lenguaje abusivo. Por un lado, proponemos una estrategia de representación de textos diseñada con el objetivo de obtener un espacio de representación en el que los textos abusivos puedan distinguirse fácilmente de otros textos. Por otro lado, evaluamos la capacidad de redes neuronales convolucionales basadas en grafos para clasificar textos abusivos. La siguiente parte de nuestra investigación se centra en analizar cómo el aumento de datos puede influir en el rendimiento de la detección del lenguaje abusivo. Para ello, investigamos dos técnicas bien conocidas basadas en el principio de minimización del riesgo en la vecindad de instancias originales y proponemos una variante para una de ellas. Además, evaluamos técnicas simples basadas en el reemplazo de sinónimos, inserción aleatoria, intercambio aleatorio y eliminación aleatoria de palabras. Las contribuciones de esta tesis ponen de manifiesto el potencial de las redes neuronales basadas en grafos y de las técnicas de aumento de datos para mejorar la detección del lenguaje abusivo, especialmente cuando hay limitación de datos. Estas contribuciones han sido publicadas en conferencias y revistas internacionales., [CA] La detecció del llenguatge abusiu és una tasca que s'ha tornat cada vegada més important en l'era digital moderna, on la comunicació es produïx a través de diverses plataformes en línia. L'augment de les interaccions en estes plataformes ha provocat un augment de l'aparició de llenguatge abusiu. Abordar este contingut és crucial per a mantindre un entorn en línia segur i inclusiu. No obstant això, esta tasca enfronta diversos desafiaments que la convertixen en una àrea complexa i contínua de recerca i desenvolupament. En particular, detectar llenguatge abusiu en entorns amb escassetat de dades presenta desafiaments addicionals pel fet que el desenvolupament de sistemes automàtics precisos sovint requerix de grans conjunts de dades anotades. En esta tesi investiguem diferents aspectes de la detecció del llenguatge abusiu, prestant especial atenció a entorns amb dades limitades. Primer, estudiem el biaix cap a paraules clau abusives en models entrenats per a la detecció de llenguatge abusiu. Amb este propòsit, proposem dos mètodes per a extraure paraules clau potencialment abusives de col·leccions de textos. Després avaluem el biaix cap a les paraules clau extretes i com es pot modificar este biaix per a influir en el rendiment de la detecció de llenguatge abusiu. L'anàlisi i les conclusions d'este treball revelen evidència que és possible mitigar el biaix i que esta reducció pot afectar positivament l'acompliment dels models. No obstant això, notem que no és possible establir una correspondència similar entre la variació del biaix i l'acompliment dels models quan hi ha escassetat dades amb les tècniques de reducció del biaix estudiades. En segon lloc, investiguem l'ús de xarxes neuronals basades en grafs per a detectar llenguatge abusiu. D'una banda, proposem una estratègia de representació textual dissenyada amb l'objectiu d'obtindre un espai de representació en el qual els textos abusius puguen distingir-se fàcilment d'altres textos. D'altra banda, avaluem la capacitat de models basats en xarxes neuronals convolucionals basades en grafs per a classificar textos abusius. La següent part de la nostra investigació se centra en analitzar com l'augment de dades pot influir en el rendiment de la detecció del llenguatge abusiu. Per a això, investiguem dues tècniques ben conegudes basades en el principi de minimització del risc en el veïnatge d'instàncies originals i proposem una variant per a una d'elles. A més, avaluem tècniques simples basades en el reemplaçament de sinònims, inserció aleatòria, intercanvi aleatori i eliminació aleatòria de paraules. Les contribucions d'esta tesi destaquen el potencial de les xarxes neuronals basades en grafs i de les tècniques d'augment de dades per a millorar la detecció del llenguatge abusiu, especialment quan hi ha limitació de dades. Estes contribucions han sigut publicades en revistes i conferències internacionals., [EN] Abusive language detection is a task that has become increasingly important in the modern digital age, where communication takes place via various online platforms. The increase in online interactions has led to an increase in the occurrence of abusive language. Addressing such content is crucial to maintaining a safe and inclusive online environment. However, this task faces several challenges that make it a complex and ongoing area of research and development. In particular, detecting abusive language in environments with sparse data poses an additional challenge, since the development of accurate automated systems often requires large annotated datasets. In this thesis we investigate different aspects of abusive language detection, paying particular attention to environments with limited data. First, we study the bias toward abusive keywords in models trained for abusive language detection. To this end, we propose two methods for extracting potentially abusive keywords from datasets. We then evaluate the bias toward the extracted keywords and how this bias can be modified in order to influence abusive language detection performance. The analysis and conclusions of this work reveal evidence that it is possible to mitigate the bias and that such a reduction can positively affect the performance of the models. However, we notice that it is not possible to establish a similar correspondence between bias mitigation and model performance in low-resource settings with the studied bias mitigation techniques. Second, we investigate the use of models based on graph neural networks to detect abusive language. On the one hand, we propose a text representation framework designed with the aim of obtaining a representation space in which abusive texts can be easily distinguished from other texts. On the other hand, we evaluate the ability of models based on convolutional graph neural networks to classify abusive texts. The next part of our research focuses on analyzing how data augmentation can influence the performance of abusive language detection. To this end, we investigate two well-known techniques based on the principle of vicinal risk minimization and propose a variant for one of them. In addition, we evaluate simple techniques based on the operations of synonym replacement, random insertion, random swap, and random deletion. The contributions of this thesis highlight the potential of models based on graph neural networks and data augmentation techniques to improve abusive language detection, especially in low-resource settings. These contributions have been published in several international conferences and journals., This research work was partially funded by the Spanish Ministry of Science and Innovation under the research project MISMIS-FAKEnHATE on Misinformation and Miscommunication in social media: FAKE news and HATE speech (PGC2018-096212-B-C31). The authors thank also the EU-FEDER Comunitat Valenciana 2014-2020 grant IDIFEDER/2018/025. This work was done in the framework of the research project on Fairness and Transparency for equitable NLP applications in social media, funded by MCIN/AEI/10.13039/501100011033 and by ERDF, EU A way of making EuropePI. FairTransNLP research project (PID2021-124361OB-C31) funded by MCIN/AEI/10.13039/501100011033 and by ERDF, EU A way of making Europe. Part of the work presented in this article was performed during the first author’s research visit to the University of Mannheim, supported through a Contact Fellowship awarded by the DAAD scholarship program “STIBET Doktoranden”.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Towards Robust Models for Fake News Detection in Spanish

  • Gómez-González, Sergio|||0009-0006-0061-8056
  • Rosso, Paolo|||0000-0002-8922-1242
  • Coll-Ardanuy, María
[ES] En este artículo nos enfrentamos al reto de detectar noticias falsas exclusivamente en español, un campo en el cual no ha habido demasiado esfuerzo de investigación. Además, la temática de las noticias se mantiene en continuo cambio, por lo que los modelos que no pueden adaptarse acaban siendo poco efectivos a largo plazo. Es por ello que, en este campo, la robustez es imprescindible. En búsqueda de esa propiedad, hemos aplicado distintas técnicas de explotación y aumento de datos para mejorar los resultados de un clasificador simple basado en un transformer preentrenado. Adicionalmente, hemos incluido una comparativa con un gran modelo de lenguaje generativo. También, utilizamos dos particiones distintas de un mismo dataset para comparar su efectividad: una partición típica con conjuntos de entrenamiento y test parecidos y otra más realista (adversaria). Finalmente, analizamos qué aspectos ejercen mayor influencia sobre la robustez y efectividad de los modelos para la detección de noticias falsas., This work has been developed under the project FAKEnHATE-PdC (PDC2022- 133118-I00) with founding from European NextGenerationEU/PRTR. The work of Mariona Coll Ardanuy was done at the UPV under FairTransNLP Grant PID2021-124361OB-C31 funded by MCIN/AEI/10.13039/501100011033 and by ERDF/EU.
RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación Artículo científico (JournalArticle)

Overview of AuTexTification at IberLEF 2023: Detection and Attribution of Machine-Generated Text in Multiple Domains, Resumen de AuTexTification en IberLEF 2023: Detección y Atribución de Texto Generado Automáticamente en Múltiples Dominios

  • Sarvazyan, Areg Mikael
  • González, José Ángel
  • Franco-Salvador, Marc
  • Rangel, Francisco
  • Chulvi-Ferriols, María Alberta
  • Rosso, Paolo|||0000-0002-8922-1242
[EN] This paper presents the overview of the AuTexTification shared task as part of the IberLEF 2023 Workshop in Iberian Languages Evaluation Forum, within the framework of the SEPLN 2023 conference. AuTexTification consists of two subtasks: for Subtask 1, participants had to determine whether a text is human-authored or has been generated by a large language model. For Subtask 2, participants had to attribute a machine-generated text to one of six different text generation models. Our AuTexTification 2023 dataset contains more than 160.000 texts across two languages (English and Spanish) and five domains (tweets, reviews, news, legal, and how-to articles). A total of 114 teams signed up to participate, of which 36 sent 175 runs, and 20 of them sent their working notes. In this overview, we present the AuTexTification dataset and task, the submitted participating systems, and the results., [ES] Este artículo presenta un resumen de la tarea AuTexTification como parte del workshop IberLEF 2023 sobre el Iberian Languages Evaluation Forum, en el marco de la conferencia SEPLN 2023. AuTexTification consta de dos subtareas: en la Subtarea 1, los participantes tuvieron que determinar si un texto fue escrito por un humano o generado por un modelo de lenguaje masivo. Para la Subtarea 2, los participantes debían atribuir un texto generado automáticamente a uno de seis modelos de generación de texto diferentes. El conjunto de datos AuTexTification contiene más de 160.000 textos en dos idiomas (inglés y español) y cinco dominios (tweets, reseñas, noticias, legislación y artículos instructivos). Un total de 114 equipos se inscribieron para participar, de los cuales 36 enviaron 175 resultados y 20 de ellos enviaron artículos. En este artículo, presentamos el conjunto de datos y la tarea AuTexTification, los sistemas enviados por los participantes y sus resultados., The work from Symanto has been partially funded by the Pro<SUP>2</SUP>Haters - Proactive Profiling of Hate Speech Spreaders (CDTi IDI-20210776), the XAI-DisInfodemics: eXplainable AI for disinformation and conspiracy detection during infodemics (MICIN PLEC2021-007681), the OBULEX - OBservatorio del Uso de Lenguage sEXista en la red (IVACE IMINOD/2022/106), and the ANDHI - ANomalous Diffusion of Harmful Information (CPP2021-008994) R&D grants. The work of Areg Mikael Sarvazyan has been partially developed with the support of valgrAI - Valencian Graduate School and Research Network of Artificial Intelligence and the Generalitat Valenciana, and co-founded by the European Union. The research at the Universitat Politecnica de Valencia was framed under the FairTransNLP research project, Grant PID2021-124361OB-C31 funded by MCIN/AEI/10.13039/501100011033 and by ERDF, EU A way of making Europe.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Overview of EXIST 2022: sEXism Identification in Social neTworks, Overview de EXIST 2022: Identificación de Sexismo en Redes Sociales

  • Rodríguez-Sánchez, Francisco
  • Carrillo-de-Albornoz, Jorge
  • Plaza Morales, Laura
  • Mendieta-Aragón, Adrián
  • Marco Remón, Guillermo
  • Makeienko, Maryna
  • Plaza, María
  • Gonzalo Arroyo, Julio
  • Spina, Damiano
  • Rosso, Paolo
The paper describes the organization, goals, and results of the sEXism Identification in Social neTworks (EXIST)2022 challenge, a shared task proposed for the second year at IberLEF. EXIST 2022 consists of two challenges: sexism identification and sexism categorization of tweets and gabs, both in Spanish and English. We have received a total of 45 runs for the sexism identification task and 29 runs for the sexism categorization task, submitted by 19 different teams. In this paper, we present the dataset, the evaluation methodology, an overview of the proposed systems, and the results obtained. The final dataset consists of more than 12,000 annotated texts from two social networks (Twitter and Gab) labelled following two different procedures: external contributors and trained experts.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Overview of AuTexTification at IberLEF 2023: Detection and Attribution of Machine-Generated Text in Multiple Domains, Resumen de AuTexTification en IberLEF 2023: Detección y Atribución de Texto Generado Automáticamente en Múltiples Dominios

  • Sarvazyan, Areg Mikael
  • González, José Angel
  • Franco-Salvador, Marc
  • Rangel Pardo, Francisco Manuel
  • Chulvi, Berta
  • Rosso, Paolo
This paper presents the overview of the AuTexTification shared task as part of the IberLEF 2023 Workshop in Iberian Languages Evaluation Forum, within the framework of the SEPLN 2023 conference. AuTexTification consists of two subtasks: for Subtask 1, participants had to determine whether a text is human-authored or has been generated by a large language model. For Subtask 2, participants had to attribute a machine-generated text to one of six different text generation models. Our AuTexTification 2023 dataset contains more than 160.000 texts across two languages (English and Spanish) and five domains (tweets, reviews, news, legal, and how-to articles). A total of 114 teams signed up to participate, of which 36 sent 175 runs, and 20 of them sent their working notes. In this overview, we present the AuTexTification dataset and task, the submitted participating systems, and the results.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

When humour hurts: linguistic features to foster explainability, Cuando el humor duele: características lingüísticas para ganar en explicabilidad

  • Merlo, Lucía I.
  • Chulvi, Berta
  • Ortega, Reynier
  • Rosso, Paolo
The main objective of this research is to use different features for the textual representation of humorous texts and detect which are the characteristics that distinguish non-offensive jokes from the highly offensive ones. For this purpose, we use the data from the HaHackaton task in which jokes are annotated according to their degree of offensiveness. A new classification task is created by using two subsets of the jokes: the non-offensive ones and the highly offensive ones. The features with statistically significant differences in the two groups are used. By applying an ablation test, the most relevant features are used for a second classification task, showing that it is possible to obtain the same results with fewer computational resources.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

On the Poor Robustness of Transformer Models in Cross-Language Humor Recognition, Sobre la Poca Robustez de los Modelos Transformers en el Reconocimiento Translingüístico del Humor

  • Labadie Tamayo, Roberto
  • Ortega, Reynier
  • Rosso, Paolo
  • Rodriguez Cisneros, Mariano
Humor is a pervasive communicative device; nevertheless, its portability from one language to another remains challenging for computer machines and even humans. In this work, we investigate the problem of humor recognition from a cross-language and cross-domain perspective, focusing on English and Spanish languages. To this aim, we rely on two strategies: the first is based on multilingual transformer models for exploiting the cross-language knowledge distilled by them, and the second introduces machine translation to learn and make predictions in a single language. Experiments showed that models struggle in front of the humor complexity when it is translated, effectively tracking a degradation in the humor perception when messages flow from one language to another. However, when multilingual models face a cross-language scenario, exclusive between the fine-tuning and evaluation data languages, humor translation helps to align the knowledge learned in fine-tuning phase. According to this, a mean increase of 11% in F1 score was observed when classifying English-written texts with models fine-tuned with a Spanish dataset. These results are encouraging and constitute the first step towards a computationally cross-language analysis of humor.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Everybody Hurts, Sometimes. Overview of HUrtful HUmour at IberLEF 2023: Detection of Humour Spreading Prejudice in Twitter, Todos herimos, a veces. Resumen de HUHU en IberLEF 2023: Detección de Humor que Difunde Prejuicios en Twitter

  • Labadie Tamayo, Roberto
  • Chulvi, Berta
  • Rosso, Paolo
Humour is an efficient strategy to spread prejudice because, most of the time, it evades moral judgement. However, it perpetuates stereotypes and doing so justifies discriminatory acts. At HUHU we propose a frame to study how humour is used to discriminate minorities and to analyse their interplay with the degree of prejudice expressed against specific groups. To this end, we provide a corpus of prejudiced tweets in Spanish annotated with the presence of humour, its prejudice degree and the targeted groups: women and feminists, the LGBTI+ community, immigrants and racially discriminated people, and over-weighted people. This paper analyses the results achieved by the 46 teams that participated in HUHU.