Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 15
Encontrada(s) 1 página(s)

Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Democracy made easy: simplifying complex topics to enable democratic participation

  • Khallaf, Nouran
  • Bott, Stefan Markus
  • Eugeni, Carlo
  • O'Flaherty, John
  • Sharoff, Serge
  • Saggion, Horacio
Several groups of people are excluded from democratic deliberation because the language used in this context may be too difficult for them to understand. Our iDEM project aims to reduce existing linguistic barriers in deliberative processes by developing technology to facilitate the translation of complicated text into Easy-to-Read formats that are more suitable for many people. In this paper, we describe classification experiments for detecting different types of difficulties which should be amended in order to make texts easier to understand. We focus on a lexical simplification system that can achieve state-of-the-art results with the use of a free and open-weight large language model for the Romance Languages in our project. Moreover, a sentence segmentation system is introduced to produce text segmentation for long sentences based on training data. Finally, we describe the iDEM mobile app, which will make our technology available as a service for end-users of our target populations., This document is part of a project that has received funding from the European Union’s Horizon Europe research and innovation program under Grant Agreement No. 101132431 (iDEM Project). The views and opinions expressed in this document are solely those of the author(s) and do not necessarily reflect the views of the European Union. Neither the European Union nor the granting authority can be held responsible for them. The University of Leeds (UOL) was funded by UK Research and Innovation (UKRI) under the UK government’s Horizon Europe funding guarantee (Grant Agreement No. 10103529).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Can LLMs solve reading comprehension tests as second language learners?

  • Hayakawa, Akio
  • Saggion, Horacio
The manual evaluation of natural language processing systems is costly and time-consuming, especially when targeting people with specific attributes as evaluators. Current large language models (LLMs) are reported to outperform humans at various tasks, and recently have been used as substitutes for human evaluators. LLMs also have shown the ability to behave as specified in a prompt. This progress raises a fundamental question: can LLMs mimic the behavior of language learners? In this study, we intentionally weaken LLMs aiming to make them simulate language learners on multiple-choice reading comprehension tests. By comparing answer distributions from language learners and LLMs, we observe that prompts designed to weaken the LLMs indeed degrade their performance. However, this degration does not bridge the gap between the original LLMs and language learners, thereby hilighting a critical discrepancy between them., The authors acknowledge the support from Departament de Recerca i Universitats de la Generalitat de Catalunya (ajuts SGR-Cat 2021) and from Maria de Maeztu Units of Excellence Programme CEX2021-001195-M, funded by MCIN/AEI /10.13039/501100011033. This research is part of a project that has received funding from the European Union´s Horizon Europe research and innovation program under the Grant Agreement No. 101132431 (iDEM Project).
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

Artificial intelligence and natural language processing for easy-to-read texts

  • Saggion, Horacio
Access to information is a fundamental human right that contributes to freedom of expression and self-determination. However, information availability alone is not enough: the way in which information is expressed and presented on paper or in digital format can be extremely complicated to understand and act upon for a highly diverse set of people with varying ranges of reading, writing and understanding abilities. For a long time, artificial intelligence (AI) and natural language processing (NLP) have dedicated research efforts in the field of automatic text simplification to the development of methods to automate the production of easy-to-read texts. However, in spite of AI hype, the problem persists. Current NLP models such as large language models (LLMs) are still not well understood, and their use in the development of text simplification needs careful assessment. In this paper, having identified the need for easy-to-read, accessible texts, we provide a light overview of current methods in NLP and provide a lay explanation of several techniques applied to lexically and syntactically modified texts to make them simpler to read and understand. We conclude by raising awareness on the use of general tools to address this very challenging problem that can affect contemporary lives., We acknowledge funding from the European Union’s Horizon Europe research and innovation programme under Grant Agreement No. 101132431 (iDEM Project). However, the views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union. We also acknowledge support from the Ministry of Research and Universities of the Generalitat de Catalunya (grants SGR-Cat 2021).
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

Lexical complexity prediction and lexical simplification for catalan and spanish: resource creation, quality assessment, and ethical considerations

  • Saggion, Horacio
  • Bott, Stefan Markus
  • Szasz, Sandra
  • Pérez, Nelson
  • Calderón Ramírez, Saúl
  • Solís, Martín
Comunicació presentada al 3rd Workshop on Text Simplification, Accessibility and Readability (TSAR 2024), celebrat a Miami (EUA) el 15 de novembre de 2024., Automatic lexical simplification is a task to substitute lexical items that may be unfamiliar and difficult to understand with easier and more common words. This paper presents the description and analysis of two novel datasets for lexical simplification in Spanish and Catalan. This dataset represents the first of its kind in Catalan and a substantial addition to the sparse data on automatic lexical simplification which is available for Spanish. Specifically, it is the first dataset for Spanish which includes scalar ratings of the understanding difficulty of lexical items. In addition, we present a detailed analysis aiming at assessing the appropriateness and ethical dimensions of the data for the lexical simplification task.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

The BEA 2024 shared task on the Multilingual Lexical Simplification Pipeline

  • Shardlow, Matthew
  • Alva-Manchego, Fernando
  • Batista-Navarro, Riza
  • Bott, Stefan Markus
  • Calderón Ramírez, Saúl
  • Cardon, Rémi
  • Hayakawa, Akio
  • Saggion, Horacio
Comunicació presentada al 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), celebrat a Ciutat de Mèxic (Mèxic) el 20 de juny de 2024, We report the findings of the 2024 Multilingual Lexical Simplification Pipeline shared task. We released a new dataset comprising 5,927 instances of lexical complexity prediction and lexical simplification on common contexts across 10 languages, split into trial (300) and test (5,627). 10 teams participated across 2 tracks and 10 languages with 233 runs evaluated across all systems. Five teams participated in all languages for the lexical complexity prediction task and 4 teams participated in all languages for the lexical simplification task. Teams employed a range of strategies, making use of open and closed source large language models for lexical simplification, as well as feature-based approaches for lexical complexity prediction. The highest scoring team on the combined multilingual data was able to obtain a Pearson’s correlation of 0.6241 and an ACC@1@Top1 of 0.3772, both demonstrating that there is still room for improvement on two difficult sub-tasks of the lexical simplification pipeline.
Repositorio Digital de la UPF
Publicación Artículo científico (JournalArticle)

A novel Spanish dataset for financial education text simplification targeting visually impaired individuals

  • Pérez-Rojas, Nelson
  • Calderón Ramírez, Saúl
  • Solís, Martín
  • Romero-Sandoval, Mario Alberto
  • Arias-Monge, Monica
  • Saggion, Horacio
Automatic Text Simplification (ATS) is a crucial task in natural language processing, aimed at making texts more comprehensible, particularly for specific groups such as individuals with visual impairments. One of the primary challenges in developing models for ATS is the scarcity of data, especially in Spanish. This manuscript introduces a novel dataset tailored for Spanish speakers with visual impairments, consisting of 5,314 pairs of original and simplified sentences created using established simplification rules. Additionally, we evaluate the feasibility of augmenting this dataset using large language models such as Generative Pre-training Transformer (GPT)-3, TUNER, and Multilingual T5 (mT5). We compare the simplifications generated by these models with our dataset to assess their effectiveness in data augmentation. The characteristics of our dataset and the findings from these comparisons are discussed in detail. The dataset is publicly available on Hugging Face at https://huggingface.co/datasets/saul1917/FEINA., The work of Horacio Saggion was supported in part by the Maria de Maeztu Units of Excellence Program, funded by MCIN/AEI/10.13039/501100011033 under Grant CEX2021-001195-M; and in part by European Union’s Horizon Europe Research and Innovation Program through the iDEM Project under Grant 101132431.
Repositorio Digital de la UPF
Publicación Contribución de congreso (ConferenceOutput)

An extensible massively multilingual lexical simplification pipeline dataset using the MultiLS framework

  • Shardlow, Matthew
  • Bott, Stefan Markus
  • Hayakawa, Akio
  • Saggion, Horacio
Comunicació presentada a 3rd Workshop on Tools and Resources for People with REAding DIfficulties (READI) @ LREC-COLING 2024, celebrat el 20 de maig de 2024 a Torí, Itàlia., We present preliminary findings on the MultiLS dataset, developed in support of the 2024 Multilingual Lexical Simplification Pipeline (MLSP) Shared Task. This dataset currently comprises of 300 instances of lexical complexity prediction and lexical simplification across 10 languages. In this paper, we (1) describe the annotation protocol in support of the contribution of future datasets and (2) present summary statistics on the existing data that we have gathered. Multilingual lexical simplification can be used to support low-ability readers to engage with otherwise difficult texts in their native, often low-resourced, languages., Andrea Horbach is part of the research conducted at CATALPA – Center for Advanced Technology-Assisted Learning and Predictive Analytics of the FernUniversität in Hagen, Germany. Anna Hülsing is supported by the German Federal Ministry of Education and Research (grant no. FKZ 01JA23S03C). Joseph Imperial is supported by the National University Philippines (Project ID: 2023I-1T-05-MLA-CCIT-Computer Science) and the UKRI Centre for Doctoral Training in Accountable, Responsible and Transparent AI [EP/S023437/1] of the University of Bath. Horacio Saggion, Stefan Bott and Akio Hayakawa acknowledge funding from the European Union´s Horizon Europe research and innovation program under the Grant Agreement No. 101132431 (iDEM Project) – views and opinions expressed are however those of the author(s) only and do necessarily reflect those of the European Union.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Democracy made easy: simplifying complex topics to enable democratic participation

  • Khallaf, Nouran
  • Bott, Stefan Markus
  • Eugeni, Carlo
  • O'Flaherty, John
  • Sharoff, Serge
  • Saggion, Horacio
Several groups of people are excluded from democratic deliberation because the language used in this context may be too difficult for them to understand. Our iDEM project aims to reduce existing linguistic barriers in deliberative processes by developing technology to facilitate the translation of complicated text into Easy-to-Read formats that are more suitable for many people. In this paper, we describe classification experiments for detecting different types of difficulties which should be amended in order to make texts easier to understand. We focus on a lexical simplification system that can achieve state-of-the-art results with the use of a free and open-weight large language model for the Romance Languages in our project. Moreover, a sentence segmentation system is introduced to produce text segmentation for long sentences based on training data. Finally, we describe the iDEM mobile app, which will make our technology available as a service for end-users of our target populations., This document is part of a project that has received funding from the European Union’s Horizon Europe research and innovation program under Grant Agreement No. 101132431 (iDEM Project). The views and opinions expressed in this document are solely those of the author(s) and do not necessarily reflect the views of the European Union. Neither the European Union nor the granting authority can be held responsible for them. The University of Leeds (UOL) was funded by UK Research and Innovation (UKRI) under the UK government’s Horizon Europe funding guarantee (Grant Agreement No. 10103529).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Can LLMs solve reading comprehension tests as second language learners?

  • Hayakawa, Akio
  • Saggion, Horacio
The manual evaluation of natural language processing systems is costly and time-consuming, especially when targeting people with specific attributes as evaluators. Current large language models (LLMs) are reported to outperform humans at various tasks, and recently have been used as substitutes for human evaluators. LLMs also have shown the ability to behave as specified in a prompt. This progress raises a fundamental question: can LLMs mimic the behavior of language learners? In this study, we intentionally weaken LLMs aiming to make them simulate language learners on multiple-choice reading comprehension tests. By comparing answer distributions from language learners and LLMs, we observe that prompts designed to weaken the LLMs indeed degrade their performance. However, this degration does not bridge the gap between the original LLMs and language learners, thereby hilighting a critical discrepancy between them., The authors acknowledge the support from Departament de Recerca i Universitats de la Generalitat de Catalunya (ajuts SGR-Cat 2021) and from Maria de Maeztu Units of Excellence Programme CEX2021-001195-M, funded by MCIN/AEI /10.13039/501100011033. This research is part of a project that has received funding from the European Union´s Horizon Europe research and innovation program under the Grant Agreement No. 101132431 (iDEM Project).
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

An extensible massively multilingual lexical simplification pipeline dataset using the MultiLS framework

  • Shardlow, Matthew
  • Bott, Stefan Markus
  • Hayakawa, Akio
  • Saggion, Horacio
Comunicació presentada a 3rd Workshop on Tools and Resources for People with REAding DIfficulties (READI) @ LREC-COLING 2024, celebrat el 20 de maig de 2024 a Torí, Itàlia., We present preliminary findings on the MultiLS dataset, developed in support of the 2024 Multilingual Lexical Simplification Pipeline (MLSP) Shared Task. This dataset currently comprises of 300 instances of lexical complexity prediction and lexical simplification across 10 languages. In this paper, we (1) describe the annotation protocol in support of the contribution of future datasets and (2) present summary statistics on the existing data that we have gathered. Multilingual lexical simplification can be used to support low-ability readers to engage with otherwise difficult texts in their native, often low-resourced, languages., Andrea Horbach is part of the research conducted at CATALPA – Center for Advanced Technology-Assisted Learning and Predictive Analytics of the FernUniversität in Hagen, Germany. Anna Hülsing is supported by the German Federal Ministry of Education and Research (grant no. FKZ 01JA23S03C). Joseph Imperial is supported by the National University Philippines (Project ID: 2023I-1T-05-MLA-CCIT-Computer Science) and the UKRI Centre for Doctoral Training in Accountable, Responsible and Transparent AI [EP/S023437/1] of the University of Bath. Horacio Saggion, Stefan Bott and Akio Hayakawa acknowledge funding from the European Union´s Horizon Europe research and innovation program under the Grant Agreement No. 101132431 (iDEM Project) – views and opinions expressed are however those of the author(s) only and do necessarily reflect those of the European Union.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

A novel Spanish dataset for financial education text simplification targeting visually impaired individuals

  • Pérez-Rojas, Nelson
  • Calderón Ramírez, Saúl
  • Solís, Martín
  • Romero-Sandoval, Mario Alberto
  • Arias-Monge, Monica
  • Saggion, Horacio
Automatic Text Simplification (ATS) is a crucial task in natural language processing, aimed at making texts more comprehensible, particularly for specific groups such as individuals with visual impairments. One of the primary challenges in developing models for ATS is the scarcity of data, especially in Spanish. This manuscript introduces a novel dataset tailored for Spanish speakers with visual impairments, consisting of 5,314 pairs of original and simplified sentences created using established simplification rules. Additionally, we evaluate the feasibility of augmenting this dataset using large language models such as Generative Pre-training Transformer (GPT)-3, TUNER, and Multilingual T5 (mT5). We compare the simplifications generated by these models with our dataset to assess their effectiveness in data augmentation. The characteristics of our dataset and the findings from these comparisons are discussed in detail. The dataset is publicly available on Hugging Face at https://huggingface.co/datasets/saul1917/FEINA., The work of Horacio Saggion was supported in part by the Maria de Maeztu Units of Excellence Program, funded by MCIN/AEI/10.13039/501100011033 under Grant CEX2021-001195-M; and in part by European Union’s Horizon Europe Research and Innovation Program through the iDEM Project under Grant 101132431.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

Lexical complexity prediction and lexical simplification for catalan and spanish: resource creation, quality assessment, and ethical considerations

  • Saggion, Horacio
  • Bott, Stefan Markus
  • Szasz, Sandra
  • Pérez, Nelson
  • Calderón Ramírez, Saúl
  • Solís, Martín
Comunicació presentada al 3rd Workshop on Text Simplification, Accessibility and Readability (TSAR 2024), celebrat a Miami (EUA) el 15 de novembre de 2024., Automatic lexical simplification is a task to substitute lexical items that may be unfamiliar and difficult to understand with easier and more common words. This paper presents the description and analysis of two novel datasets for lexical simplification in Spanish and Catalan. This dataset represents the first of its kind in Catalan and a substantial addition to the sparse data on automatic lexical simplification which is available for Spanish. Specifically, it is the first dataset for Spanish which includes scalar ratings of the understanding difficulty of lexical items. In addition, we present a detailed analysis aiming at assessing the appropriateness and ethical dimensions of the data for the lexical simplification task.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Contribución de congreso (ConferenceOutput)

The BEA 2024 shared task on the Multilingual Lexical Simplification Pipeline

  • Shardlow, Matthew
  • Alva-Manchego, Fernando
  • Batista-Navarro, Riza
  • Bott, Stefan Markus
  • Calderón Ramírez, Saúl
  • Cardon, Rémi
  • Hayakawa, Akio
  • Saggion, Horacio
Comunicació presentada al 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), celebrat a Ciutat de Mèxic (Mèxic) el 20 de juny de 2024, We report the findings of the 2024 Multilingual Lexical Simplification Pipeline shared task. We released a new dataset comprising 5,927 instances of lexical complexity prediction and lexical simplification on common contexts across 10 languages, split into trial (300) and test (5,627). 10 teams participated across 2 tracks and 10 languages with 233 runs evaluated across all systems. Five teams participated in all languages for the lexical complexity prediction task and 4 teams participated in all languages for the lexical simplification task. Teams employed a range of strategies, making use of open and closed source large language models for lexical simplification, as well as feature-based approaches for lexical complexity prediction. The highest scoring team on the combined multilingual data was able to obtain a Pearson’s correlation of 0.6241 and an ACC@1@Top1 of 0.3772, both demonstrating that there is still room for improvement on two difficult sub-tasks of the lexical simplification pipeline.
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Artificial intelligence and natural language processing for easy-to-read texts

  • Saggion, Horacio
Access to information is a fundamental human right that contributes to freedom of expression and self-determination. However, information availability alone is not enough: the way in which information is expressed and presented on paper or in digital format can be extremely complicated to understand and act upon for a highly diverse set of people with varying ranges of reading, writing and understanding abilities. For a long time, artificial intelligence (AI) and natural language processing (NLP) have dedicated research efforts in the field of automatic text simplification to the development of methods to automate the production of easy-to-read texts. However, in spite of AI hype, the problem persists. Current NLP models such as large language models (LLMs) are still not well understood, and their use in the development of text simplification needs careful assessment. In this paper, having identified the need for easy-to-read, accessible texts, we provide a light overview of current methods in NLP and provide a lay explanation of several techniques applied to lexically and syntactically modified texts to make them simpler to read and understand. We conclude by raising awareness on the use of general tools to address this very challenging problem that can affect contemporary lives., We acknowledge funding from the European Union’s Horizon Europe research and innovation programme under Grant Agreement No. 101132431 (iDEM Project). However, the views and opinions expressed are those of the author(s) only and do not necessarily reflect those of the European Union. We also acknowledge support from the Ministry of Research and Universities of the Generalitat de Catalunya (grants SGR-Cat 2021).
Recercat. Dipósit de la Recerca de Catalunya
Publicación

Making democratic deliberation and participation more accessible: the iDEM project

  • Saggion, Horacio
  • O’Flaherty, John
  • Blanchet, Thomas
  • Sharoff, Serge
  • Sanfilippo, Silvia
  • Muñoz, Lian
  • Gollegger, Martin
  • Rascón, Almudena
  • Martí, José Luis
  • Szasz, Sandra
  • Bott, Stefan Markus
  • Sayman, Volkan
Comunicació presentada a: SEPLN – CEDI 2024 Seminar of the Spanish Society for Natural Language Processing - 7th Spanish Conference on Informatics; del 19 al 20 de juny 2024 a A Coruña, Espanya., Deliberative and participatory processes currently don’t have full legitimacy due to the exclusion and marginalisation of several vulnerable communities from democratic spaces. It is well documented that people with limited language skills, such as people with cognitive disabilities, struggle to participate in democratic processes. This happens in spite of the advocacy work of organizations which promote human rights. The iDEM project aims at addressing the barriers in deliberative and participatory democratic practices through a thorough intersectional analysis of conditions under which current structures and systems limit participation of marginalised and vulnerable communities. Specially those with limited skills in reading, writing or understanding a fairly complex language, which is often required for deliberative and participatory processes. iDEM will lay the theoretical foundations for the analysis of current marginalisation from deliberative processes of diverse under-represented groups due to a lack of language skills. It will adopt a user-centred approach for making participatory processes more accessible and inclusive, developing advanced natural language processing technologies (NLP) and artificial intelligence (AI) to empower under-represented groups, with tools to facilitate communication and dialog in democratic spaces. iDEM will co-create the next-generation multilingual models aimed at: (1) detecting possible sources of problems in understanding messages for several European languages and audiences, (2) automatically adapting texts in those languages to be accessible and unbiased for these audiences, (3) providing AI tools for enhancing the generation of appropriate messages and discourses. iDEM aims to create more accessible democratic spaces in Italy and Spain with customised technology enhancing the participation and representation of marginalised groups by providing unbiased and inclusive technology., This document is part of a project that has received funding from the European Union´s Horizon Europe research and innovation program under the Grant Agreement No. 101132431 (iDEM Project). Views and opinions expressed are however those of the author(s) only and do necessarily reflect those of the European Union. Neither the European Union nor the granting authority can be held responsible for them. UOL was funded by UK Research and Innovation (UKRI) under the UK government’s Horizon Europe funding guarantee (grant number 10103529).