Pasar al contenido principal

Big-eRisk: Predicción temprana de riesgos personales en conjuntos de datos masivos

PLEC2021-007662

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal de I+D+i Orientado a los Retos de la Sociedad
Convocatoria Proyectos de I+D+i en Líneas Estratégicas
Año convocatoria 2021
Plan Estatal 2017-2020 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSIDADE DA CORUÑA

Resultados relacionados

Resultados totales (Incluyendo duplicados): 27
Encontrada(s) 1 página(s)

Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
Publicación Artículo científico (JournalArticle)

DepreSym: A Depression Symptom Annotated Corpus and the Role of Large Language Models as Assessors of Psychological Markers

  • Pérez, Anxo
  • Fernández Pichel, Marcos
  • Parapar, Javier
  • Losada Carril, David Enrique
Computational methods for depression detection aim to mine traces of depression from online publications posted by Internet users. However, solutions trained on existing collections exhibit limited generalisation and interpretability. To tackle these issues, recent studies have shown that identifying specific depressive symptoms can lead to more robust and effective models. The eRisk initiative fosters research on this area and has recently proposed a new ranking task focused on developing search methods to find sentences related to depressive symptoms. This search challenge relies on the symptoms specified by the Beck Depression Inventory-II (BDI-II), a questionnaire widely used in clinical practice. It includes symptoms such as sadness, irritability or lack of sleep. Given the input submitted by systems participating in eRisk, we first apply top-k pooling over the systems’ relevance rankings, obtaining a diverse set of sentences. These sentences are judged for relevance, leading to DepreSym, a dataset consisting of 21,580 sentences annotated according to their relevance to the 21 BDI-II symptoms. This dataset serves as a valuable resource for advancing the development of models that monitor depression markers. Due to the complex nature of this relevance annotation, we designed a robust assessment methodology carried out by three expert assessors, including a trained psychologist. As part of this study, we explore the potential of recent Large Language Models (ChatGPT, GPT4 and Vicuna) as assessors in this complex task. We undertake a comprehensive examination of the LLMs’ performance, studying their main limitations and analysing their role as a complement or replacement for human annotators. Finally, we incorporate our dataset into the Benchmarking Information Retrieval (BEIR) framework for a thorough search evaluation. We use state-of-the-art retrieval systems, including lexical, sparse, dense and re-ranking architectures, to gain insights about the dataset’s complexity and identify potential avenues for improvement.
Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
Publicación Artículo científico (JournalArticle)

Evaluating search engines and large language models for answering health questions

  • Fernández Pichel, Marcos
  • Pichel Campos, Juan Carlos
  • Losada Carril, David Enrique
Search engines (SEs) have traditionally been primary tools for information seeking, but the new large language models (LLMs) are emerging as powerful alternatives, particularly for question-answering tasks. This study compares the performance of four popular SEs, seven LLMs, and retrieval-augmented (RAG) variants in answering 150 health-related questions from the TREC Health Misinformation (HM) Track. Results reveal SEs correctly answer 50–70% of questions, often hindered by many retrieval results not responding to the health question. LLMs deliver higher accuracy, correctly answering about 80% of questions, though their performance is sensitive to input prompts. RAG methods significantly enhance smaller LLMs’ effectiveness, improving accuracy by up to 30% by integrating retrieval evidence.
Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
Publicación Artículo científico (JournalArticle)

A multistage retrieval system for health-related misinformation detection

  • Fernández Pichel, Marcos
  • Losada Carril, David Enrique
  • Pichel Campos, Juan Carlos
Web search is widely used to find online medical advice. As such, health-related information access requires retrieval algorithms capable of promoting reliable documents and filtering out unreliable ones. To this end, different types of components, such as query-document matching features, passage relevance estimation and AI-based reliability estimators, need to be combined. In this paper, we propose an entire pipeline for misinformation detection, based on the fusion of multiple content-based features. We present experiments which study the influence of each pipeline stage for the target task. Our technological solution incorporates signals from technologies derived from diverse research fields, including search, deep learning for natural language processing, as well as advanced supervised and unsupervised learning. To combine evidence, different score fusion strategies are compared, including unsupervised rank fusion techniques and learning-to-rank methods. The reference framework for empirically validating our solution is the TREC Health Misinformation Track, which provides several challenging subtasks that foster research on the identification of reliable and correct information for health-related decision making tasks. More specifically, we address a total recall task, the goal of which is to identify all the documents conveying incorrect information for a specific set of topics, and an ad-hoc retrieval task, aiming to rank credible and correct information over incorrect information. All variants are evaluated with an assorted set of effectiveness metrics, which includes standard search measures, such as R-Precision, Average Precision or Normalised Discounted Cumulative Gain, and innovative metrics based on the compatibility between the ranked output and two reference rankings composed of helpful and harmful documents, respectively. Our experiments demonstrate the effectiveness of the proposed pipeline stages and indicate that sophisticated supervised fusion methods do not fare better than simpler fusion alternatives. Additionally, for reliability estimation, unsupervised textual similarity performs better than textual classification based on supervised learning. The results also show that the presented approach is highly competitive when compared with state-of-the-art solutions for the same problem.
Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
Publicación Artículo científico (JournalArticle)

OMP4Py: A pure Python implementation of openMP

  • Piñeiro Pomar, César Alfredo
  • Pichel Campos, Juan Carlos
Python demonstrates lower performance in comparison to traditional high performance computing (HPC) languages such as C, C++, and Fortran. This performance gap is largely due to Python’s interpreted nature and the Global Interpreter Lock (GIL), which hampers multithreading efficiency. However, the latest version of Python includes the necessary changes to make the interpreter thread-safe, allowing Python code to run without the GIL. This important update will enable users to fully exploit multithreading parallelism in Python. In order to facilitate that task, this paper introduces OMP4Py, the first pure Python implementation of OpenMP. We demonstrate that it is possible to bring OpenMP’s familiar directive-based parallelization paradigm to Python, allowing developers to write parallel code with the same level of control and flexibility as in C, C++, or Fortran. The experimental evaluation shows that OMP4Py significantly impacts the performance of various types of applications, although the current threading limitations of Python’s interpreter (v3.13) reduce its effectiveness for numerical
Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
Publicación Artículo científico (JournalArticle)

Towards universal MPI bindings for enhanced new language support

  • Piñeiro Pomar, César Alfredo
  • Vázquez Álvarez, Álvaro
  • Pichel Campos, Juan Carlos
In the field of High Performance Computing (HPC), Message Passing Interface (MPI) is the most widely used and prevalent programming model. Only the low-level programming languages C, C++, and Fortran have bindings available in the standard. Although there are attempts to provide MPI bindings for other programming languages, these may be limited, which could lead to incompatibilities, performance overhead, and functional gaps. To address those problems, we present MPI4All, a brand-new tool designed to make the process of developing effective MPI bindings for any programming language more straightforward. Support for additional languages can be added with little difficulty, and MPI4All is independent of the MPI implementation. Programming language binding generators for Go and Java are included in the most recent version of MPI4All. We demonstrate their good performance results with respect to other state-of-the-art approaches. This work is an extended version of the ICCS-2024 conference paper (Piñeiro et al., 2024).
Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
Publicación Artículo científico (JournalArticle)

Personality trait analysis during the COVID-19 pandemic: a comparative study on social media

  • Fernández Pichel, Marcos
  • Aragón Saenzpardo, Mario Ezra
  • Saborido Patiño, Julián
  • Losada Carril, David Enrique
The COVID-19 pandemic, a global contagion of coronavirus infection caused by Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), has triggered severe social and economic disruption around the world and provoked changes in people’s behavior. Given the extreme societal impact of COVID-19, it becomes crucial to understand the emotional response of the people and the impact of COVID-19 on personality traits and psychological dimensions. In this study, we contribute to this goal by thoroughly analyzing the evolution of personality and psychological aspects in a large-scale collection of tweets extracted during the COVID-19 pandemic. The objectives of this research are: i) to provide evidence that helps to understand the estimated impact of the pandemic on people’s temperament, ii) to find associations and trends between specific events (e.g., stages of harsh confinement) and people’s reactions, and iii) to study the evolution of multiple personality aspects, such as the degree of introversion or the level of neuroticism. We also examine the development of emotions, as a natural complement to the automatic analysis of the personality dimensions. To achieve our goals, we have created two large collections of tweets (geotagged in the United States and Spain, respectively), collected during the pandemic. Our work reveals interesting trends in personality dimensions, emotions, and events. For example, during the pandemic period, we found increasing traces of introversion and neuroticism. Another interesting insight from our study is that the most frequent signs of personality disorders are those related to depression, schizophrenia, and narcissism. We also found some peaks of negative/positive emotions related to specific events
Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
Publicación Artículo científico (JournalArticle)

BigSeqKit: a parallel Big Data toolkit to process FASTA and FASTQ files at scale

  • Piñeiro Pomar, César Alfredo
  • Pichel Campos, Juan Carlos
Background High-throughput sequencing technologies have led to an unprecedented explosion in the amounts of sequencing data available, which are typically stored using FASTA and FASTQ files. We can find in the literature several tools to process and manipulate those type of files with the aim of transforming sequence data into biological knowledge. However, none of them are well fitted for processing efficiently very large files, likely in the order of terabytes in the following years, since they are based on sequential processing. Only some routines of the well-known seqkit tool are partly parallelized. In any case, its scalability is limited to use few threads on a single computing node. Results Our approach, BigSeqKit, takes advantage of a high-performance computing–Big Data framework to parallelize and optimize the commands included in seqkit with the aim of speeding up the manipulation of FASTA/FASTQ files. In this way, in most cases, it is from tens to hundreds of times faster than several state-of-the-art tools. At the same time, our toolkit is easy to use and install on any kind of hardware platform (local server or cluster), and its routines can be used as a bioinformatics library or from the command line. Conclusions BigSeqKit is a very complete and ultra-fast toolkit to process and manipulate large FASTA and FASTQ files. It is publicly available at https://github.com/citiususc/BigSeqKit.
Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
Publicación Artículo científico (JournalArticle)

Efficient phylogenetic tree inference for massive taxonomic datasets: harnessing the power of a server to analyze 1 million taxa

  • Piñeiro Pomar, César Alfredo
  • Pichel Campos, Juan Carlos
Background Phylogenies play a crucial role in biological research. Unfortunately, the search for the optimal phylogenetic tree incurs significant computational costs, and most of the existing state-of-the-art tools cannot deal with extremely large datasets in reasonable times. Results In this work, we introduce the new VeryFastTree code (version 4.0), which is able to construct a tree on 1 server using single-precision arithmetic from a massive 1 million alignment dataset in only 36 hours, which is 3 times and 3.2 times faster than its previous version and FastTree-2, respectively. This new version further boosts performance by parallelizing all tree traversal operations during the tree construction process, including subtree pruning and regrafting moves. Additionally, it introduces significant new features such as support for new and compressed file formats, enhanced compatibility across a broader range of operating systems, and the integration of disk computing functionality. The latter feature is particularly advantageous for users without access to high-end servers, as it allows them to manage very large datasets, albeit with an increase in computing time. Conclusions Experimental results establish VeryFastTree as the fastest tool in the state-of-the-art for maximum likelihood phylogeny estimation. It is publicly available at https://github.com/citiususc/veryfasttree. In addition, VeryFastTree is included as a package in Bioconda, MacPorts, and all Debian-based Linux distributions.
Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela
Publicación Artículo científico (JournalArticle)

Semi-Supervised Learning in the Field of Conversational Agents and Motivational Interviewing, Aprendizaje Semisupervisado en el ´Ambito de los Agentes Conversacionales y la Entrevista Motivacional

  • Rosenova Tsakova, Gergana
  • Fernández Pichel, Marcos
  • Meyer, Selina
  • Losada Carril, David Enrique
The exploitation of Motivational Interviewing concepts for text analysis contributes to gaining valuable insights into individuals’ perspectives and attitudes towards behaviour change. The scarcity of labelled user data poses a persistent challenge and impedes technical advances in research under non-English language scenarios. To address the limitations of manual data labelling, we propose a semi-supervised learning method as a means to augment an existing training corpus. Our approach leverages machine-translated user-generated data sourced from social media communities and employs self-training techniques for annotation. To that end, we consider various source contexts and conduct an evaluation of multiple classifiers trained on various augmented datasets. The results indicate that this weak labelling approach does not yield improvements in the overall classification capabilities of the models. However, notable enhancements were observed for the minority classes. We conclude that several factors, including the quality of machine translation, can potentially bias the pseudo-labelling models and that the imbalanced nature of the data and the impact of a strict pre-filtering threshold need to be taken into account as inhibiting factors.
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

How does depression talk on social media? Modeling depression language with relevance-based statistical language models

  • Bao, Eliseo
  • Pérez, Anxo
  • Otero, David
  • Parapar, Javier
[Abstract]: Many individuals with mental health problems turn to the internet and social media for information and support. The text generated on these platforms serves as a valuable resource for identifying mental health risks, driving interdisciplinary research to develop models for mental health analysis and prediction. In this paper, we model depression-related language using relevance-based statistical language models to create lexicons that characterize linguistic patterns associated with depression. We also propose a ranking method that leverages these lexicons to prioritize users exhibiting stronger signs of depressive language on social media. Our models integrate clinical markers from established depression questionnaires, particularly the Beck Depression Inventory-II (BDI-II), enhancing explainability, generalization, and performance. Experiments across multiple social media datasets show that incorporating clinical knowledge improves user ranking and generalizes effectively across platforms. Additionally, we refine existing depression lexicons by applying weights estimated from our models, achieving better performance in generating depression-related queries. A comparative analysis of our models highlights differences in language use between control users and those with depression, aligning with prior psycholinguistic findings. This work advances the understanding of depression-related language through statistical modeling, paving the way for scalable social media interventions to identify at-risk individuals, This work has received support from projects: PLEC2021-007662 (MCIN/AEI/10.13039/501100011033 Ministerio de Ciencia e Innovación, European Union NextGeneration) and PID2022-137061OB-C21 (MCIN/AEI/10.13039/501100011033/, Ministerio de Ciencia e Innovación, by the European Union); Consellería de Educación, Universidade e Formación Profesional, Spain (grant number ED481A-2024-079 and accreditation 2019–2022 ED431G/01 and GRC ED431C 2025/49) and the European Regional Development Fund , which acknowledges the CITIC Research Center, Xunta de Galicia; ED481A-2024-079, Xunta de Galicia; ED431G/01, Xunta de Galicia; GRC ED431C 2025/49
RUC. Repositorio da Universidade da Coruña
Publicación Contribución de congreso (ConferenceOutput)

PsyProf: A Platform for Assisted Screening of Depression in Social Media

  • Pérez, Anxo
  • Piot, Paloma
  • Parapar, Javier
  • Barreiro, Álvaro
Presented at: 45th European Conference on Information Retrieval, ECIR 2023, Dublin, 2 April 2023 through 6 April 2023., This version of the article has been accepted for publication, after peer review (when applicable) and is subject to Springer Nature’s AM terms of use, but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: http://dx.doi.org/10.1007/978-3-031-28241-6_30, [Abstract]: Depression is one of the most prevalent mental disorders. For its effective treatment, patients need a quick and accurate diagnosis. Mental health professionals use self-report questionnaires to serve that purpose. These standardized questionnaires consider different depression symptoms in their evaluations. However, mental health stigmas heavily influence patients when filling out a questionnaire. In contrast, many people feel more at ease discussing their mental health issues on social media. This demo paper presents a platform for assisted examination and tracking of symptoms of depression for social media users. In order to bring a broader context, we have complemented our tool with user profiling. We show a platform that helps professionals with data labelling, relying on depression estimators and profiling models., This work has received support from projects: PLEC2021-007662 (MCIN/AEI/10.13039/501100011033, Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación, Unión Europea-Next GenerationEU, Spain); Consellería de Educación, Universidade e Formación Profesional, Spain (accreditation 2019-2022 ED431G/01 and GPC ED431B 2022/33) and the European Regional Development Fund, which acknowledges the CITIC Research Center, an ICT of the University of A Coruña as a Research Center of the Galician University System., Xunta de Galicia; ED431G/01, Xunta de Galicia; ED431B 2022/33
RUC. Repositorio da Universidade da Coruña
Publicación Contribución de congreso (ConferenceOutput)

eRisk 2023: Depression, Pathological Gambling, and Eating Disorder Challenges

  • Parapar, Javier
  • Martín-Rodilla, Patricia
  • Losada, David E.
  • Crestani, Fabio
Presented at: 45th European Conference on Information Retrieval, ECIR 2023, Dublin, 2 April 2023 through 6 April 2023., This version of the article has been accepted for publication, after peer review (when applicable) and is subject to Springer Nature’s AM terms of use, but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: https://doi.org/10.1007/978-3-031-28241-6_67, [Abstract]: In 2017, we launched eRisk as a CLEF Lab to encourage research on early risk detection on the Internet. Since then, thanks to the participants’ work, we have developed detection models and datasets for depression, anorexia, pathological gambling and self-harm. In 2023, it will be the seventh edition of the lab, where we will present a new type of task on sentence ranking for depression symptoms. This paper outlines the work that we have done to date, discusses key lessons learned in previous editions, and presents our plans for eRisk 2023., The first and second authors thank the financial support supplied by the Consellería de Cultura, Educación, Formación Profesional e Universidades (accreditation 2019–2022 ED431G/01, ED431B 2022/33) and the European Regional Development Fund, which acknowledges the CITIC Research Center in ICT of the University of A Coruña as a Research Center of the Galician University System. The third author thanks the financial support supplied by the Consellería de Cultura, Educación, Formación Profesional e Universidades (accreditation 2019–2022 ED431G-2019/04, ED431C 2022/19) and the European Regional Development Fund, which acknowledges the CiTIUS-Research Center in Intelligent Technologies of the University of Santiago de Compostela as a Research Center of the Galician University System. The first, second, and third author also thank the funding of project PLEC2021-007662 (MCIN/AEI/10.13039/501100011033, Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación, Plan de Recuperación, Transformación y Resiliencia, Unión Europea-Next Generation EU)., Xunta de Galicia; ED431G/01, Xunta de Galicia; ED431B 2022/33, Xunta de Galicia; ED431G-2019/04, Xunta de Galicia; ED431C 2022/19
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Enhancing Automatic Keyphrase Labelling with Text-to-Text Transfer Transformer (T5) Architecture: A Framework for Keyphrase Generation and Filtering

  • Gabín Brenlla, Jorge Juan
  • Ares Brea, Manuel Eduardo
  • Parapar, Javier
[Abstract]: Automatic keyphrase labelling stands for the ability of models to retrieve words or short phrases that adequately describe documents’ content. Previous work has put much effort into exploring extractive techniques to address this task; however, most of these methods cannot produce keyphrases not found in the text. Given this limitation, keyphrase generation approaches have arisen lately. This paper presents a keyphrase generation model based on the Text-to-Text Transfer Transformer (T5) architecture. Having a document’s title and abstract as input, we learn a T5 model to generate keyphrases which adequately define its content. We name this model docT5keywords. We not only perform the classic inference approach, where the output sequence is directly selected as the predicted values, but we also report results from a majority voting approach. In this approach, multiple sequences are generated, and the keyphrases are ranked based on their frequency of occurrence across these sequences. Along with this model, we present a novel keyphrase filtering technique based on the T5 architecture. We train a T5 model to learn whether a given keyphrase is relevant to a document. We devise two evaluation methodologies to prove our model’s capability to filter inadequate keyphrases. First, we perform a binary evaluation where our model has to predict if a keyphrase is relevant for a given document. Second, we filter the predicted keyphrases by keyphrase generation models and check if the evaluation scores are improved. Experimental results show that our best docT5keywords variant yields relative improvements over the strongest baselines ranging from −7.5% to +60.5% across datasets and evaluation metrics; by evaluation type the gains are: present exact-match +23.9% to +60.5%, absent exact-match −7.5% to +60.4%, and partial-match (absent) −0.9% to +10.4%. The proposed filtering technique demonstrates strong performance in eliminating false positives across all datasets, even though identifying true keyphrases remains more challenging., First and third authors acknowledge funding from the Ministry of Science, Innovation and Universities of the Government of Spain (projects PID2022-137061OB-C21, MCIN/AEI/10.13039/501100011033 and PID2025-167749OB-C22), as well as from the Department of Education, Science, Universities, and Vocational Training of the Xunta de Galicia (grant GRC ED431C 2025/49). CITIC, as a center accredited for excellence within the Galician University System and a member of the CIGUS Network, receives subsidies from the Department of Education, Science, Universities, and Vocational Training of the Xunta de Galicia. Additionally, it is co-financed by the EU through the FEDER Galicia 2021-27 operational program (Ref. ED431G 2023/01). Financial support was received from: Project PLEC2021-007662 (MCIN/AEI/10.13039/501100011033, Ministerio de Ciencia e Innovación, European Union NextGenerationEU/PRTR). Project PID2022-137061OB-C21 (MCIN/AEI/10.13039/501100011033, Ministerio de Ciencia e Innovación, ERDF “A way of making Europe” by the European Union). Consellería de Educación, Universidade e Formación Profesional, Spain (accreditation GRC ED431C 2025/49). Scholarship DIN2020-011582 financed by the MCIN/AEI/10.13039/501100011033 (acknowledged by the first author)., Xunta de Galicia; GRC ED431C 2025/49, Xunta de Galicia; ED431G 2023/01
RUC. Repositorio da Universidade da Coruña
Publicación Contribución de congreso (ConferenceOutput)

Decoding Hate: Exploring Language Models' Reactions to Hate Speech

  • Piot, Paloma
  • Parapar, Javier
[Abstract]: Hate speech is a harmful form of online expression, often manifesting as derogatory posts. It is a significant risk in digital environments. With the rise of Large Language Models (LLMs), there is concern about their potential to replicate hate speech patterns, given their training on vast amounts of unmoderated internet data. Understanding how LLMs respond to hate speech is crucial for their responsible deployment. However, the behaviour of LLMs towards hate speech has been limited compared. This paper investigates the reactions of seven state-of-the-art LLMs (LLaMA 2, Vicuna, LLaMA 3, Mistral, GPT-3.5, GPT-4, and Gemini Pro) to hate speech. Through qualitative analysis, we aim to reveal the spectrum of responses these models produce, highlighting their capacity to handle hate speech inputs. We also discuss strategies to mitigate hate speech generation by LLMs, particularly through fine-tuning and guideline guardrailing. Finally, we explore the models’ responses to hate speech framed in politically correct language., The authors thank the funding from the Horizon Europe research and innovation programme under the Marie Skłodowska-Curie Grant Agreement No. 101073351. The authors also thank the financial support supplied by the Consellería de Cultura, Educación, Formación Profesional e Universidades (accreditation 2019-2022 ED431G/01, ED431B 2022/33) and the European Regional Development Fund, which acknowledges the CITIC Research Center in ICT as a Research Center of the Galician University System and the project PID2022-137061OB-C21 (Ministerio de Ciencia e Innovación supported by the European Regional Development Fund). The authors also thank the funding of project PLEC2021-007662 (MCIN/AEI/10.13039/501100011033, Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación, Plan de Recuperación, Transformación y Resiliencia, Unión Europea-Next Generation EU)., Xunta de Galicia; 2019-2022 ED431G/01, Xunta de Galicia; ED431B 2022/33
RUC. Repositorio da Universidade da Coruña
Publicación Contribución de congreso (ConferenceOutput)

Towards Reliable Testing for Multiple Information Retrieval System Comparisons

  • Otero, David
  • Parapar, Javier
  • Barreiro, Álvaro
Presented at: Advances in Information Retrieval: 47th European Conference on Information Retrieval, ECIR 2025, Lucca, Italy, April 6–10, 2025, This version of the article has been accepted for publication, after peer review and is subject to Springer Nature’s AM terms of use, but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: http://dx.doi.org/10.1007/978-3-031-88711-6_27, [Abstract]: Null Hypothesis Significance Testing is the de facto tool for assessing effectiveness differences between Information Retrieval systems. Researchers use statistical tests to check whether those differences will generalise to online settings or are just due to the samples observed in the laboratory. Much work has been devoted to studying which test is the most reliable when comparing a pair of systems, but most of the IR real-world experiments involve more than two. In the multiple comparisons scenario, testing several systems simultaneously may inflate the errors committed by the tests. In this paper, we use a new approach to assess the reliability of multiple comparison procedures using simulated and real TREC data. Experiments show that Wilcoxon plus the Benjamini-Hochberg correction yields Type I error rates according to the significance level for typical sample sizes while being the best test in terms of statistical power., The authors thank the financial support supplied by the Consellería de Cultura, Educación, Formación Profesional e Universidades (accreditation 2019-2022 ED431G/01, ED431B 2022/33) and the European Regional Development Fund, which acknowledges the CITIC Research Center in ICT as a Research Center of the Galician University System and the project PID2022-137061OB-C21 (Ministerio de Ciencia e Innovación supported by the European Regional Development Fund). The authors also thank the funding of project PLEC2021-007662 (MCIN/AEI/10.13039/501100011033, Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación, Plan de Recuperación, Transformación y Resiliencia, Unión Europea-Next Generation EU)., Xunta de Galicia; ED431G/01, Xunta de Galicia; ED431B 2022/33
RUC. Repositorio da Universidade da Coruña
Publicación Contribución de congreso (ConferenceOutput)

Overview of eRisk 2024: Early Risk Prediction on the Internet (Extended Overview)

  • Parapar, Javier
  • Martín-Rodilla, Patricia
  • Losada, David E.
  • Crestani, Fabio
Included in: Working Notes of the Conference and Labs of the Evaluation Forum (CLEF 2024) Grenoble, France, 9-12 September, 2024., [Abstract]: This paper presents eRisk 2024, the eighth edition of the CLEF conference’s lab dedicated to early risk detection. Since its inception, the lab has been at the forefront of developing and refining evaluation methodologies, effectiveness metrics, and processes for early risk detection across various domains. These early alerting models hold significant value, particularly in sectors focused on health and safety, where timely intervention can be crucial. eRisk 2024 featured three main tasks designed to push the boundaries of early risk detection techniques. The first task challenged participants to rank sentences based on their relevance to standardized depression symptoms, a crucial step in identifying early signs of depression from textual data. The second task focused on the early detection of anorexia indicators, aiming to develop models that can recognize the subtle cues of this eating disorder before it becomes critical. The third task was centered around estimating responses to an eating disorders questionnaire by analyzing users’ social media posts. Participants had to leverage the rich, real-world textual data available on social media to gauge potential mental health risks. Through these tasks, eRisk 2024 continues to advance the field of early risk detection, fostering innovations that could lead to significant improvements in public health interventions., This work was supported by project PLEC2021-007662 (MCIN/AEI/10.13039/ 501100011033, Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación, Plan de Recuperación, Transformación y Resiliencia, Unión Europea-Next Generation EU). The first and second authors thank the financial support supplied by the Xunta de Galicia-Consellería de Cultura, Educación, Formación Profesional e Universidade (GPC ED431B 2022/33) and the European Regional Development Fund and project PID2022- 137061OB-C21 (MCIN/AEI/ 10.13039/501100011033, Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación, Proyectos de Generación de Conocimiento; supported by “ERDF A way of making Europe”, by the “European Union”). The CITIC, as a center accredited for excellence within the Galician University System and a member of the CIGUS Network, receives subsidies from the Department of Education, Science, Universities, and Vocational Training of the Xunta de Galicia. Additionally, CITIC is co-financed by the EU through the FEDER Galicia 2021-27 operational program (Ref. ED431G 2023/01). The third author thanks the financial support supplied by the Xunta de Galicia-Consellería de Cultura, Educación, Formación Profesional e Universidade (accreditation 2019-2022 ED431G-2019/04, ED431C 2022/19) and the European Regional Development Fund, which acknowledges the CiTIUS-Research Center in Intelligent Technologies of the University of Santiago de Compostela as a Research Center of the Galician University System. David E. Losada also thanks the financial support obtained from project SUBV23/00002 (Ministerio de Consumo, Subdirección General de Regulación del Juego) and project PID2022-137061OB-C22 (Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación, Proyectos de Generación de Conocimiento; supported by the European Regional Development Fund)., Xunta de Galicia; ED431B 2022/33, Xunta de Galicia; ED431G 2023/01, Xunta de Galicia; ED431C 2022/19, Xunta de Galicia; ED431G 2019/04
RUC. Repositorio da Universidade da Coruña
Publicación Contribución de congreso (ConferenceOutput)

MindWell: A Conversational Agent for Professional Depression Screening on Social Media

  • Bao, Eliseo
  • Pérez, Anxo
  • Parapar, Javier
Presented at: ECIR 2025, 47th European Conference on Information Retrieval. April 6–10, 2025, Lucca, Italy This version of the article has been accepted for publication, after peer review (when applicable) and is subject to Springer Nature’s AM terms of use, but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: https://doi.org/10.1007/978-3-031-88720-8_9 Part of the book series: Lecture Notes in Computer Science (LNCS,volume 15576), [Abstract]: Depression is among the most prevalent mental health conditions, with an early and accurate diagnosis being essential for mitigating its effects. Yet, stigma often prevents individuals from seeking professional help. In this context, social media offers a unique resource for depression screening, as users frequently share, comment, and disclose their daily struggles, providing key insights into their mental health through online activity. However, the immense volume of data generated on these platforms presents a significant challenge, requiring substantial time and effort for mental health professionals to analyze. This demo paper introduces MindWell, an open-source conversational agent designed to support clinicians in identifying symptoms and emotions relevant to clinical assessments. MindWell uses a Retrieval-Augmented Generation (RAG) framework, incorporating a Large Language Model (LLM) based on Llama 3.1 and fine-tuned specifically for depression screening based on clinical symptom criteria, particularly the Beck Depression Inventory-II (BDI-II). By leveraging users’ social media history as informed and reliable context, MindWell is designed to answer questions formulated by clinicians, facilitating the review process. We collaborated with a professional psychologist to assess MindWell’s responses in a clinical setting, finding that the system effectively captures users’ depressive signs and shows promise for mental health support applications., This work has received support from projects: PLEC2021-007662 (MCIN/AEI/10.13039/501100011033 Ministerio de Ciencia e Innovación, European Union NextGenerationEU/PRTR) and PID2022-137061OB-C21 (MCIN/AEI/10.13039 /501100011033/, Ministerio de Ciencia e Innovación, ERDF A way of making Europe, by the European Union); Consellería de Educación, Universidade e Formación Profesional, Spain (grant number ED481A-2024-079 and accreditations 2019–2022 ED431G/ 01 and GPC ED431B 2022/33) and the European Regional Development Fund, which acknowledges the CITIC Research Center., Xunta de Galicia; ED481A-2024-079, Xunta de Galicia; ED431G/ 01, Xunta de Galicia; ED431B 2022/33
RUC. Repositorio da Universidade da Coruña
Publicación Contribución de congreso (ConferenceOutput)

How Discriminative Are Your Qrels? How To Study the Statistical Significance of Document Adjudication Methods

  • Otero, David
  • Parapar, Javier
  • Ferro, Nicola
[Abstract]: Creating test collections for offline retrieval evaluation requires human effort to judge documents' relevance. This expensive activity motivated much work in developing methods for constructing benchmarks with fewer assessment costs. In this respect, adjudication methods actively decide both which documents and the order in which experts review them, in order to better exploit the assessment budget or to lower it. Researchers evaluate the quality of those methods by measuring the correlation between the known gold ranking of systems under the full collection and the observed ranking of systems under the lower-cost one. This traditional analysis ignores whether and how the low-cost judgements impact on the statistically significant differences among systems with respect to the full collection. We fill this void by proposing a novel methodology to evaluate how the low-cost adjudication methods preserve the pairwise significant differences between systems as the full collection. In other terms, while traditional approaches look for stability in answering the question "is system A better than system B?", our proposed approach looks for stability in answering the question "is system A significantly better than system B?", which is the ultimate questions researchers need to answer to guarantee the generalisability of their results. Among other results, we found that the best methods in terms of ranking of systems correlation do not always match those preserving statistical significance., This work has received support from: (i) project PLEC2021-007662 (MCIN/AEI/10.13039/501100011033, Ministerio de Ciencia e Inno-vación, Agencia Estatal de Investigación, Plan de Recuperación, Transformación y Resiliencia, Unión Europea-Next GenerationEU); (ii) Programa de Ayudas para la Formación de Profesorado Universitario, grant number FPU20/02659 (Ministerio de Universidades); (iii) project PID2022-137061OB-C21 (Proyectos de Generación de Conocimiento, MCIN); (iv) project ED431-B 2022/33 (Xunta de Galicia/ERDF); (v) CAMEO, PRIN 2022 n. 2022ZLL7MW., Xunta de Galicia; ED431-B 2022/33
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Comparison of Clustering Algorithms for Knowledge Discovery in Social Media Publications: A Case Study of Mental Health Analysis, Comparación de algoritmos de agrupamiento para el descubrimiento de conocimiento en publicaciones de redes sociales: un caso de estudio en salud mental

  • Couto, Manuel
  • Parapar, Javier
  • Losada, David E.
[Abstract]: In the age of social media, user-generated content is critical for detecting early signs of mental disorders. In this study, we use thematic clustering to analyze the content of the social media platform Reddit. Our primary goal is to use clustering techniques for comprehensive topic discovery, with a focus on identifying common themes among user groups suffering from mental illnesses such as depression, anorexia, gambling addiction, and self-harm. Our findings show that certain clusters are more cohesive, e.g., with a higher proportion of texts indicating depression. Furthermore, we discovered subreddits that are strongly linked to texts from the depressed user group. These findings shed light on how online interactions and subreddit themes may impact users’ mental health, paving the way for future research and more targeted interventions in the field of online mental health., [Resumen]: En la era de las redes sociales, el contenido generado por los usuarios es fundamental para detectar los primeros signos de trastornos mentales. En este estudio utilizamos el agrupamiento de publicaciones por tópicos para analizar el contenido de la plataforma Reddit. Nuestro objetivo primordial es utilizar técnicas de agrupamiento para descubrir temas centrales, con un enfoque en la identificación de temas comunes entre los grupos de usuarios que sufren enfermedades mentales como la depresión, la anorexia, la adicción a los juegos de azar y las autolesiones. Nuestros hallazgos muestran que ciertos clusters son más cohesivos, por ejemplo mostrando una mayor proporción de textos de personas con depresión. Además, hemos descubierto subreddits que estan fuertemente vinculados a textos escritos por usuarios deprimidos. Estos hallazgos arrojan luz sobre cómo las interacciones en línea y los temas que se tratan en los subreddits reflejan aspectos de salud mental, abriendo el camino para futuras investigaciones e intervenciones dirigidas a la prevención de trastornos., This work was supported by project PLEC2021-007662 (MCIN/AEI/10.13039/501100011033, Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación, Plan de Recuperación, Transformación y Resiliencia, Unión Europea-Next Generation EU). The first and third author thank the financial support supplied by the Xunta de Galicia-Consellería de Cultura, Educación, Formación Profesional e Universidade (ED431G 2023/04, ED431C 2022/19) and the European Regional Development Fund, which acknowledges the CiTIUS-Research Center in Intelligent Technologies of the University of Santiago de Compostela as a Research Center of the Galician University System. David E. Losada also thanks the financial support obtained from project SUBV23/00002 (Ministerio de Consumo, Subdirección General de Regulación del Juego) and project PID2022-137061OB-C22 (Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación, Proyectos de Generación de Conocimiento; supported by the European Regional Development Fund). The second author thanks the financial support supplied by the CITIC, as a center accredited for excellence within the Galician University System and a member of the CIGUS Network, funded by the Xunta de Galicia and the EU through the ERDF Galicia 2021-27 operational program (ref. ED431G 2023/01) and project PID2022-137061OB-C21 (MCIN/AEI/ 10.13039/501100011033, Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación, Proyectos de Generación de Conocimiento; supported by “ERDF A way of making Europe”, by the “European Union”)., Xunta de Galicia; ED431G 2023/04, Xunta de Galicia; ED431C 2022/19, Xunta de Galicia; ED431G 2023/01
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

DepreSym: A Depression Symptom Annotated Corpus and the Role of Large Language Models as Assessors of Psychological Markers

  • Pérez, Anxo
  • Fernández-Pichel; Marcos
  • Parapar, Javier
  • Losada, David E.
DepreSym can be obtained upon request, and we have adhered to strict ethical standards, particularly those about the ethical development of AI. Financiado para publicación en acceso aberto: CRUE/CSIC con Springer, [Abstract]: Computational methods for depression detection aim to mine traces of depression from online publications posted by Internet users. However, solutions trained on existing collections exhibit limited generalisation and interpretability. To tackle these issues, recent studies have shown that identifying specific depressive symptoms can lead to more robust and effective models. The eRisk initiative fosters research on this area and has recently proposed a new ranking task focused on developing search methods to find sentences related to depressive symptoms. This search challenge relies on the symptoms specified by the Beck Depression Inventory-II (BDI-II), a questionnaire widely used in clinical practice. It includes symptoms such as sadness, irritability or lack of sleep. Given the input submitted by systems participating in eRisk, we first apply top-k pooling over the systems’ relevance rankings, obtaining a diverse set of sentences. These sentences are judged for relevance, leading to DepreSym, a dataset consisting of 21,580 sentences annotated according to their relevance to the 21 BDI-II symptoms. This dataset serves as a valuable resource for advancing the development of models that monitor depression markers. Due to the complex nature of this relevance annotation, we designed a robust assessment methodology carried out by three expert assessors, including a trained psychologist. As part of this study, we explore the potential of recent Large Language Models (ChatGPT, GPT4 and Vicuna) as assessors in this complex task. We undertake a comprehensive examination of the LLMs’ performance, studying their main limitations and analysing their role as a complement or replacement for human annotators. Finally, we incorporate our dataset into the Benchmarking Information Retrieval (BEIR) framework for a thorough search evaluation. We use state-of-the-art retrieval systems, including lexical, sparse, dense and re-ranking architectures, to gain insights about the dataset’s complexity and identify potential avenues for improvement., Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature. The first and third authors has received support from projects: PLEC2021-007662 (MCIN/AEI/10.13039/501100011033 Ministerio de Ciencia e Innovación, European Union Next Generation EU/PRTR) and also PID2022-137061OB-C21 (MCIN/AEI/10.13039/501100011033, Ministerio de Ciencia e Innovación). We also thank the financial support supplied by the Consellería de Cultura, Educación, Formación Profesional e Universidades (accreditation 2019-2022 ED431G/01, ED431B 2022/33) and the European Regional Development Fund, which acknowledges the CITIC Research Center. The second and fourth authors thank the financial support supplied by MICIU/AEI/10.13039/501100011033 (PID2022-137061OB-C22, supported by ERDF) and Xunta de Galicia-Consellería de Cultura, Educación, Formación Profesional e Universidades (ED431G 2023/04, ED431C 2022/19, supported by ERDF)., Xunta de Galicia; ED431G/01 2019/2022, Xunta de Galicia; ED431B 2022/33, Xunta de Galicia; ED431G 2023/04, Xunta de Galicia; ED431C 2022/19
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Using machine learning techniques to predict adolescents’ involvement in family conflict

  • López-Larrosa, Silvia
  • Sánchez Souto, Vanesa
  • Losada, David E.
  • Parapar, Javier
  • Barreiro, Álvaro
  • Ha, Anh
  • Cummings, E. Mark
[Abstract] Many cases of violence against children occur in homes and other close environments. Machine leaning is a novel approach that addresses important gaps in ways of examining this socially significant issue, illustrating innovative and emerging approaches for the use of computers from a psychological perspective. In this paper, we aim to use machine learning techniques to predict adolescents’ involvement in family conflict in a sample of adolescents living with their families(community adolescents) and adolescents living in residential care centers, who are temporarilyseparated from their families because of adverse family conditions. Participants were 251 Spanish adolescents (Mage= 15.59), of whom 167 lived in residential care and 84 lived with their families.We measured perceived interparental and family conflict, adolescents’emotional security,emotional, cognitive, and behavioral immediate responses to analog interparental conflict (IPC),and adolescents’ sociodemographic variables (i.e., age, gender). With a prediction accuracy of 65%, our results show that adolescents in residential care are not at greater risk for involvement in family conflict compared to adolescents living with their families. Age and gender are not salient predictive variables. We could identify that responses to analog IPC, adolescents’emotiona lsecurity, triangulation in IPC, and the presence of insults or blame during family disputes predict adolescents’ involvement in family conflict. These results point to variables with a potential predictive capacity, which is relevant for research and intervention., The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was partially supported by projects PLEC2021-007,662 (MCIN/AEI/10.13,039/501100011033, Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación, Plan de Recuperación, Transformación y Resiliencia, Unión Europea-Next GenerationEU), RTI2018-093,336-B-C21 & RTI2018-093,336-B-C22 (Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación & ERDF). The fourth and fifth authors also thank the financial support supplied by the Consellería de Educación, Universidade e Formación Profesional (accreditation 2019–2022 ED431G-2019/01 and GPC ED431 B 2019/03) and the European Regional Development Fund, which acknowledges the CITIC Research Center in ICT of the University of A Coruña as a Research Center of the Galician University System. The third author also thanks the financial support supplied by the Consellería de Educación, Universidade e Formación Profesional (accreditation 2019–2022 ED431G-2019/04, ED431 C 2018/29) and the European Regional Development Fund, which acknowledges the CiTIUS-Research Center in Intelligent Technologies of the University of Santiago de Compostela as a Research Center of the Galician University System, Xunta de Galicia; ED431G-2019/01, Xunta de Galicia; ED431 B 2019/03, Xunta de Galicia; ED431G-2019/04, Xunta de Galicia; ED431 C 2018/29
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Relevance feedback for building pooled test collections

  • Otero, David
  • Parapar, Javier
  • Barreiro, Álvaro
This manuscript version of the article: Otero, D., Parapar, J., & Barreiro, Á. (2023). ‘Relevance feedback for building pooled test collections’ has been accepted for publication in Journal of Information Science, 2023, pp-1-18. DOI: https://doi.org/10.1177/0165551515598926., [Abstract]: Offline evaluation of information retrieval systems depends on test collections. These datasets provide the researchers with a corpus of documents, topics and relevance judgements indicating which documents are relevant for each topic. Gathering the latter is costly, requiring human assessors to judge the documents. Therefore, experts usually judge only a portion of the corpus. The most common approach for selecting that subset is pooling. By intelligently choosing which documents to assess, it is possible to optimise the number of positive labels for a given budget. For this reason, much work has focused on developing techniques to better select which documents from the corpus merit human assessments. In this article, we propose using relevance feedback to prioritise the documents when building new pooled test collections. We explore several state-of-the-art statistical feedback methods for prioritising the documents the algorithm presents to the assessors. A thorough comparison on eight Text Retrieval Conference (TREC) datasets against strong baselines shows that, among other results, our proposals improve in retrieving relevant documents with lower assessment effort than other state-of-the-art adjudicating methods without harming the reliability, fairness and reusability., The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: this work has received support from: (1) project PLEC2021-007662 (grant no. MCIN/AEI/10.13039/501100011033, Ministerio de Ciencia e Innovación (MCIN), Agencia Estatal de Investigación, Plan de Recuperación, Transformación y Resiliencia, Unión Europea-NextGenerationEU), (2) Programa de Ayudas para la Formación de Profesorado Universitario, grant number FPU20/02659 (Ministerio de Universidades), (3) project PID2022-137061OB-C21 (Proyectos de Generación de Conocimiento, MCIN), (4) Consellería de Educación, Universidade e Formación Profesional (accreditation 2019-2022 ED431G 2019/01) and the European Regional Development Fund, which acknowledges the Centro de Investigación en Tecnologías de la Información y la Comunicación (CITIC) Research Centre in Information and Communications Technology (ICT) of the University of A Coruña as a Research Centre of the Galician University System and (5) project ED431-B 2022/33 (Xunta de Galicia/European Regional Development Fund (ERDF)., Xunta de Galicia; ED431-B 2022/33, Xunta de Galicia; ED431G 2019/01
RUC. Repositorio da Universidade da Coruña
Publicación Contribución de congreso (ConferenceOutput)

Delving into the Depths: Evaluating Depression Severity through BDI-biased Summaries

  • Aragón, Mario Ezra
  • Parapar, Javier
  • Losada, David E.
[Abstract]: Depression is a global concern suffered by millions of people, significantly impacting their thoughts and behavior. Over the years, heightened awareness, spurred by health campaigns and other initiatives, has driven the study of this disorder using data collected from social media platforms. In our research, we aim to gauge the severity of symptoms related to depression among social media users. The ultimate goal is to estimate the user’s responses to a well-known standardized psychological questionnaire, the Beck Depression Inventory-II (BDI). This is a 21-question multiple-choice self-report inventory that covers multiple topics about how the subject has been feeling. Mining users’ social media interactions and understanding psychological states represents a challenging goal. To that end, we present here an approach based on search and summarization that extracts multiple BDI-biased summaries from the thread of users’ publications. We also leverage a robust large language model to estimate the potential answer for each BDI item. Our method involves several steps. First, we employ a search strategy based on sentence similarity to obtain pertinent extracts related to each topic in the BDI questionnaire. Next, we compile summaries of the content of these groups of extracts. Last, we exploit chatGPT to respond to the 21 BDI questions, using the summaries as contextual information in the prompt. Our model has undergone rigorous evaluation across various depression datasets, yielding encouraging results. The experimental report includes a comparison against an assessment done by expert humans and competes favorably with state-of-the-art methods., The second author thanks the financial support supplied by the Consellería de Cultura, Educación, Formación Profesional e Universidades (accreditation 2019-2022 ED431G/01, ED431B 2022/33) and the European Regional Development Fund, which acknowledges the CITIC Research Center in ICT of the University of A Coruña as a Research Center of the Galician University System and the project PID2022-137061OB-C21 (Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación, Proyectos de Generación de Conocimiento; supported by the European Regional Development Fund). The first and third authors thank: i) the financial support supplied by the Consellería de Cultura, Educación, Formación Profesional e Universidades (accreditation 2019-2022 ED431G-2019/04, ED431C 2022/19) and the European Regional Development Fund, which acknowledges the CiTIUS- Research Center in Intelligent Technologies of the University of Santiago de Compostela as a Research Center of the Galician University System, and ii) the financial support supplied by project PID2022-137061OB-C22 (Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación, Proyectos de Generación de Conocimiento; supported by the European Regional Development Fund). The third author thanks the financial support obtained from project SUBV23/00002 (Ministerio de Consumo, Subdirección General de Regulación del Juego). The authors also thank the funding of project PLEC2021-007662 (MCIN/AEI/10.13039/501100011033, Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación, Plan de Recuperación, Transformación y Resiliencia, Unión Europea-Next Generation EU)., Xunta de Galicia; ED431G/01, Xunta de Galicia; ED431B 2022/33, Xunta de Galicia; ED431G-2019/04, Xunta de Galicia; ED431C 2022/19
RUC. Repositorio da Universidade da Coruña
Publicación Contribución de congreso (ConferenceOutput)

Semantic Similarity Models for Depression Severity Estimation

  • Pérez, Anxo
  • Warikoo, Neha
  • Wang, Kexin
  • Parapar, Javier
  • Gurevych, Iryna
Presented at: EMNLP 2023 Conference on Empirical Methods in Natural Language Processing, Singapore, December 6th to December 10th., [Abstract]: Depressive disorders constitute a severe public health issue worldwide. However, public health systems have limited capacity for case detection and diagnosis. In this regard, the widespread use of social media has opened up a way to access public information on a large scale. Computational methods can serve as support tools for rapid screening by exploiting this user-generated social media content. This paper presents an efficient semantic pipeline to study depression severity in individuals based on their social media writings. We select test user sentences for producing semantic rankings over an index of representative training sentences corresponding to depressive symptoms and severity levels. Then, we use the sentences from those results as evidence for predicting symptoms severity. For that, we explore different aggregation methods to answer one of four Beck Depression Inventory (BDI-II) options per symptom. We evaluate our methods on two Reddit-based benchmarks, achieving improvement over state of the art in terms of measuring depression level., This work has received support from projects: PLEC2021-007662, PID2022-137061OB-C21 (MCIN/AEI/10.13039/501100011033, Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación, Proyectos de Generación de Conocimiento; suppported by the European Regional Development Fund); Consellería de Educación, Universidade e Formación Profesional, Spain (accreditation 2019–2022 ED431G/01 and GPC ED431B 2022/33) and the European Regional Development Fund, which acknowledges the CITIC Research Center, an ICT of the University of A Coruña as a Research Center of the Galician University System. The first author also acknowledges the predoctoral grant contract ref. ED481 A 2021/034 funded by Xunta de Galicia and the European Social Fund (ESF). From UKP Lab, this work has been funded by the LOEWE Distinguished Chair “Ubiquitous Knowledge Processing” (LOEWE initiative, Hesse, Germany)., Xunta de Galicia; ED481 A 2021/034, Xunta de Galicia; ED431G/01, Xunta de Galicia; ED431B 2022/33
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Explainable depression symptom detection in social media

  • Bao Souto, Eliseo
  • Pérez, Anxo
  • Parapar, Javier
[Abstract]: Users of social platforms often perceive these sites as supportive spaces to post about their mental health issues. Those conversations contain important traces about individuals’ health risks. Recently, researchers have exploited this online information to construct mental health detection models, which aim to identify users at risk on platforms like Twitter, Reddit or Facebook. Most of these models are focused on achieving good classification results, ignoring the explainability and interpretability of the decisions. Recent research has pointed out the importance of using clinical markers, such as the use of symptoms, to improve trust in the computational models by health professionals. In this paper, we introduce transformer-based architectures designed to detect and explain the appearance of depressive symptom markers in user-generated content from social media. We present two approaches: (i) train a model to classify, and another one to explain the classifier’s decision separately and (ii) unify the two tasks simultaneously within a single model. Additionally, for this latter manner, we also investigated the performance of recent conversational Large Language Models (LLMs) utilizing both in-context learning and finetuning. Our models provide natural language explanations, aligning with validated symptoms, thus enabling clinicians to interpret the decisions more effectively. We evaluate our approaches using recent symptom-focused datasets, using both offline metrics and expert-in-the-loop evaluations to assess the quality of our models’ explanations. Our findings demonstrate that it is possible to achieve good classification results while generating interpretable symptom-based explanations., This work has received support from projects: PLEC2021-007662 (MCIN/AEI/10.13039/501100011033, Ministerio de Ciencia e Innovación, European Union NextGenerationEU/PRTR) and PID2022-137061OB-C21 (MCIN/AEI/10.13039/501100011033/, Ministerio de Ciencia e Innovación, ERDF A way of making Europe, by the European Union); Consellería de Educación, Universidade e Formación Profesional, Spain (grant number ED481A-2024-079 and accreditations 2019–2022 ED431G/01 and GPC ED431B 2022/33) and the European Regional Development Fund, which acknowledges the CITIC Research Center. Funding for open access charge: Universidade da Coruña/CISUG. We would also like to thank Desireé Pombar, Silvia López-Larrosa and Laura Hermo for their efforts in evaluating the results. Their work allowed us to assess the practical utility and clinical relevance of the explanations generated., Xunta de Galicia; ED481A-2024-079, Xunta de Galicia; ED431G/01, Xunta de Galicia; ED431B 2022/33, Financiado para publicación en acceso aberto: Universidade da Coruña/CISUG
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Temporal Word Embeddings for Early Detection of Psychological Disorders on Social Media

  • Couto Pintos, Manuel
  • Pérez, Anxo
  • Parapar, Javier
  • Losada, David E.
[Abstract]: Mental health disorders represent a public health challenge, where early detection is critical to mitigating adverse outcomes for individuals and society. The study of language and behavior is a pivotal component in mental health research, and the content from social media platforms serves as a valuable tool for identifying signs of mental health risks. This paper presents a novel framework leveraging temporal word embeddings to capture linguistic changes over time. We specifically aim at at identifying emerging psychological concerns on social media. By adapting temporal word representations, our approach quantifies shifts in language use that may signal mental health risks. To that end, we implement two alternative temporal word embedding models to detect linguistic variations and exploit these variations to train early detection classifiers. Our experiments, conducted on 18 datasets from the eRisk initiative (covering signs of conditions such as depression, anorexia, and self-harm), show that simple models focusing exclusively on temporal word usage patterns achieve competitive performance compared to state-of-the-art systems. Additionally, we perform a word-level analysis to understand the evolution of key terms among positive and control users. These findings underscore the potential of time-sensitive word models in this domain, being a promising avenue for future research in mental health surveillance., The second and third authors thank the financial support supplied from projects: PLEC2021-007662 (MCIN/AEI/10.13039/501100011033 Ministerio de Ciencia e Innovación, European Union NextGenerationEU/PRTR) and PID2022-137061OB-C21 (MCIN/AEI/10.13039/501100011033/, Ministerio de Ciencia e Innovación, ERDF A way of making Europe, by the European Union); Consellería de Educación, Universidade e Formación Profesional, Spain (accreditations 2019-2022 ED431G/01 and GPC ED431B 2022/33) and the European Regional Development Fund, which acknowledges the CITIC Research Center. The first and fourth authors thank the financial support supplied by the Agencia Estatal de Investigación (Spain) (PID2022-137061OB-C22; PLEC2021-007662 MCIN/AEI/10.13039/501100011033, Plan de Recuperación, Transformación y Resiliencia, Unión Europea-Next Generation EU), Consellería de Cultura, Educación, Formación Profesional e Universidades (Centro de investigación de Galicia accreditation 2024-2027 ED431G-2023/04 and Reference Competitive Group accreditation 2022-2025, ED431C 2022/19) and the European Union (European Regional Development Fund - ERDF). The fourth author thanks the support obtained from project SUBV23/00002 (Ministerio de Consumo, Subdirección General de Regulación del Juego). Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature., Xunta de Galicia; ED431G/01, Xunta de Galicia; ED431B 2022/33, Xunta de Galicia; ED431G-2023/04, Xunta de Galicia; ED431C 2022/19
RUC. Repositorio da Universidade da Coruña
Publicación Contribución de congreso (ConferenceOutput)

Keyword Embeddings for Query Suggestion

  • Gabín, Jorge
  • Ares, M. Eduardo
  • Parapar, Javier
[Abstract]: Nowadays, search engine users commonly rely on query suggestions to improve their initial inputs. Current systems are very good at recommending lexical adaptations or spelling corrections to users’ queries. However, they often struggle to suggest semantically related keywords given a user’s query. The construction of a detailed query is crucial in some tasks, such as legal retrieval or academic search. In these scenarios, keyword suggestion methods are critical to guide the user during the query formulation. This paper proposes two novel models for the keyword suggestion task trained on scientific literature. Our techniques adapt the architecture of Word2Vec and FastText to generate keyword embeddings by leveraging documents’ keyword co-occurrence. Along with these models, we also present a specially tailored negative sampling approach that exploits how keywords appear in academic publications. We devise a ranking-based evaluation methodology following both known-item and ad-hoc search scenarios. Finally, we evaluate our proposals against the state-of-the-art word and sentence embedding models showing considerable improvements over the baselines for the tasks., Xunta de Galicia; ED431G/01, Xunta de Galicia; ED431B 2022/33, This work was supported by projects PLEC2021-007662 (MCIN/AEI/10.13039/501100011033, Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación, Plan de Recuperación, Transformación y Resiliencia, Unión Europea-Next Generation EU) and RTI2018-093336-B-C22 (Ministerio de Ciencia e Innovación, Agencia Estatal de Investigación). The first and third authors also thank the financial support supplied by the Consellería de Cultura, Educación e Universidade Consellería de Cultura, Educación, Formación Profesional e Universidades (accreditation 2019-2022 ED431G/01, ED431B 2022/33) and the European Regional Development Fund, which acknowledges the CITIC Research Center in ICT of the University of A Coruña as a Research Center of the Galician University System. The fist author also acknowledges the support of grant DIN2020-011582 financed by the MCIN/AEI/10.13039/501100011033.