Pasar al contenido principal

SAFEWORDS: LANGUAGE ANONYMIZATION WITH ETHICAL AND LEGAL SAFEGUARDS THROUGH NLP

AIA2025-163322-C63

•
Nombre agencia financiadora Agencia Estatal de Investigación
Acrónimo agencia financiadora AEI
Programa Programa Estatal para la Investigación y el Desarrollo Experimental
Subprograma Subprograma Estatal de Generación de Conocimiento Científico-Técnico y Desarrollo Experimental
Convocatoria Proyectos de Investigación en el ámbito de la Inteligencia Artificial
Año convocatoria 2025
Plan Estatal 2024-2027 Plan Estatal de Investigación Científica y Técnica y de Innovación
Centro beneficiario UNIVERSIDAD DE ALICANTE

Resultados relacionados

Resultados totales (Incluyendo duplicados): 10
Encontrada(s) 1 página(s)

RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Span-Reference and Grounding Reliability in Generative Spanish Clinical Named Entity Recognition: A Validation Study

  • Grande Ruiz, Eduardo
  • Muñoz, Rafael
  • Gutiérrez, Yoan
  • Saquete Boró, Estela
  • Grande Ruiz, Eduardo|||/items/d0bc54e8-b11a-41c0-9f6b-3889b4ed8179
  • Muñoz, Rafael|||/items/d6ef1286-fc50-4e8b-9d97-bcbc30e3d1bc
  • Gutiérrez, Yoan|||/items/8f7f89db-a015-4ed6-85b9-75bd041ee8bd
  • Saquete Boró, Estela|||/items/452393f2-fb8d-4840-b797-4b65a4bc9a87
Generative named entity recognition (NER) systems must identify clinical concepts and map them to exact source spans. We investigated how span-reference design affects this mapping and whether failures arise from recognition or source localisation. Four representation-and-grounding pipelines—inline XML, mention-list JSON, a tab-separated mention list, and direct-offset JSON—were compared for disease, procedure, and symptom recognition in a public Spanish clinical case-report collection. Two instruction-tuned 3-billion-parameter models were fine-tuned for each task, and outputs were evaluated for syntactic validity, parsing, grounding, and official strict-span performance. Inline XML, mention-list JSON, and the tab-separated format achieved F1 scores of 0.637–0.736, whereas direct-offset JSON achieved 0.001–0.005. With the same unadapted checkpoints, zero-shot F1 was at most 0.131 and three-shot F1 at most 0.386; three-shot prompting improved the mention-list outputs, while inline XML and direct offsets remained near zero. Direct-offset outputs usually contained relevant mention text but incorrect character positions. On a separate 75-document confirmation partition, grounding the emitted strings recovered F1 of 0.603–0.701, while replacing absolute offsets with mention-occurrence numbers achieved 0.652–0.740. Without further training, the external CARMEN-I evaluation on 458 hospital-record sections yielded F1 of 0.610–0.659 for string-grounded or occurrence-index outputs, compared with 0.113–0.177 for inline XML and at most 0.002 for direct offsets. These results show that output validity alone does not establish usable span extraction. Separating recognition from source localisation identifies where an otherwise valid generation fails, while occurrence-based references provide a more reliable alternative to absolute character offsets under exact matching in the tested settings.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

SAFEWORDs: un marco reproducible para anonimización conforme al RGPD y evaluación de generación en lenguas cooficiales

  • Muñoz, Rafael
  • Palomar, Manuel
  • Lloret, Elena
  • Fernández Pérez, Nuria
  • Muñoz, Rafael|||/items/d6ef1286-fc50-4e8b-9d97-bcbc30e3d1bc
  • Palomar, Manuel|||/items/71e8f2d8-8d56-4c8e-89df-d6ce2497d967
  • Lloret, Elena|||/items/4593ef69-8308-49d3-8f0f-17e6ee078be5
  • Fernández Pérez, Nuria|||/items/b19dd60d-215e-421d-a8f2-5ca2b51f6d1d
Los Grandes Modelos de Lenguaje (LLMs) abren oportunidades para el Procesamiento del Lenguaje Natural (PLN) en contextos institucionales, si bien plantean riesgos críticos en entornos regulados y multilingües, especialmente en lo relativo a protección de datos personales, trazabilidad de decisiones y equidad entre lenguas con distinta disponibilidad de recursos. Presentamos SAFEWORDs, proyecto que acaba de iniciarse en el marco del proyecto coordinado "HumanAIze" (Plan Nacional de Inteligencia Artificial 2025, España), que propone un marco reproducible de privacy-by-design y ethics-by-design para la evaluación y alineación de LLMs en las lenguas oficiales de la Península Ibérica (español, catalán, valenciano, gallego y euskera). El marco integra: (i) anonimización automática conforme al RGPD, con protocolos explícitos de detección de fuga residual y verificación adversarial; (ii) transformación orientada a la accesibilidad textual y al lenguaje claro; y (iii) evaluación en el dominio biomédico, donde la sensibilidad de los datos y la precisión terminológica exigen mecanismos adicionales de control generativo. Desde el punto de vista metodológico, se comparan configuraciones zero-shot y few-shot, y se documentan prompts, hiperparámetros y recursos para facilitar la replicabilidad y la gobernanza de recursos. Además de sintetizar resultados de referencia de la literatura para contextualizar métricas y órdenes de magnitud esperables, el trabajo discute implicaciones éticas y limitaciones del enfoque propuesto. La propuesta se alinea con las líneas de trabajo de SEPLN y con los objetivos de LANLP, al establecer protocolos transferibles para el desarrollo de tecnologías lingüísticas confiables en ecosistemas caracterizados por variación dialectal y lenguas infrarrepresentadas.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Exploring Cross-Lingual Transfer in Transformer-Based Fraud Detection Models

  • Martínez-Murillo, Iván
  • Sepúlveda-Torres, Robiert
  • Consuegra-Ayala, Juan Pablo
  • Martínez-Murillo, Iván|||/items/10185dd6-7baf-4c77-9241-9ca9c38f31a1
  • Sepúlveda-Torres, Robiert|||/items/60f24ee2-fe45-4b5c-a70c-c025ed1b5e4a
  • Consuegra-Ayala, Juan Pablo|||/items/5a98e4ca-607f-4aa0-91c2-a5d929bab52c
Digital communication channels have become major vectors for large-scale cyber fraud, including spam, phishing, and recruitment scams, causing significant financial losses and eroding user trust. While transformer-based models have improved text classification, most systems remain English-centric, limiting their effectiveness in multilingual environments and across diverse fraud typologies. In this work, we hypothesize that fraud detection models trained in a single language can exhibit cross-lingual generalization capabilities. We investigate the use of Natural Language Processing (NLP) for detecting multiple forms of textual fraud by developing and evaluating three transformer-based discriminative models targeting spam, phishing, and fake job postings. Specifically, we compare two architectures—MrBERT (308M parameters) and mROBERTa (283M parameters)—across English, Spanish, and Valencian data. Our results demonstrate that while models trained exclusively on English achieve near-perfect performance in-language (F1 ≈ 0.98), they exhibit limited cross-lingual generalization in other languages such as Spanish (F1 ≈ 0.44 for phishing detection, and F1 ≈ 0.85 for spam detection). However, incorporating multilingual training data dramatically improves performance in target languages (F1 ≈ 0.95–0.99) while maintaining or even enhancing English accuracy. We further observe that cross-lingual transfer is most effective when datasets are parallel or closely aligned. These findings underscore the critical importance of multilingual data inclusion for building robust, scalable cybersecurity frameworks in diverse linguistic settings.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Aproximación basada en inferencia lingüística y hechos estructurados para la detección de la alucinación en textos, An Approach Based on Linguistic Inference and Structured Facts for the Detection of Hallucinations in Texts

  • Madrigal Cutillas, Jairo
  • Lloret, Elena
  • Lloret, Elena|||/items/4593ef69-8308-49d3-8f0f-17e6ee078be5
El objetivo de este artículo es proponer y evaluar un enfoque para detectar alucinaciones factuales en textos en español, combinando análisis lingüístico con verificación de hechos extraídos de fuentes de conocimiento estructurado, en nuestro caso, Wikidata. Los resultados obtenidos demuestran que el enfoque propuesto permite identificar contradicciones y hechos inventados con una alta precisión (alrededor del 90 %), ofreciendo una herramienta pionera y competitiva con respecto a herramientas similares para la verificación factual en español.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

SmartMatch: Real-Time Semantic Retrieval for Translation Memory Systems

  • Estevanell-Valladares, Ernesto L.
  • Lamsiyah, Salima
  • Picazo-Izquierdo, Alicia
  • Ranasinghe, Tharindu
  • Mitkov, Ruslan
  • Muñoz, Rafael
  • Estevanell-Valladares, Ernesto L.|||/items/d8b1af9a-112f-48de-813f-b4d8f1df8bd1
  • Picazo-Izquierdo, Alicia|||/items/96eeb16f-1521-4e1f-a25b-77dad8099f55
  • Mitkov, Ruslan|||/items/30b15727-6f78-421c-ba49-df42c02f0e96
  • Muñoz, Rafael|||/items/d6ef1286-fc50-4e8b-9d97-bcbc30e3d1bc
Translation Memory (TM) systems are core components of commercial computer-aided translation (CAT) tools. However, traditional fuzzy matching methods often fail to retrieve semantically relevant content when surface similarity is low. We introduce SmartMatch, an open-source interactive demo and evaluation toolkit for TM retrieval that connects modern sentence encoders (including LLM-derived representations) and strong lexical/fuzzy baselines with a vector database, and exposes the end-to-end retrieval pipeline through a web-based UI for qualitative inspection and preference logging. The demo allows users to (i) enter a query segment, (ii) switch retrieval backends and embedding models, (iii) inspect top-k retrieved matches with similarity scores and qualitative cues, and (iv) observe end-to-end latency in real time. We provide a reproducible benchmark on multilingual TM data, reporting retrieval quality using reference-based MT metrics (COMET, BERTScore, METEOR, chrF) together with coverage and latency/throughput trade-offs relevant to real-time CAT workflows. On DGT-TM, encoder-based retrieval achieves full coverage (100%) with millisecond-level latency (p50/p90 ≤ 6–20 ms) and attains the strongest semantic-quality scores on the shared query set (e.g., BERTScore up to 0.91 at k=10), while BM25 remains a strong lightweight lexical baseline with very low latency. SmartMatch targets CAT researchers and tool builders and bridges recent advances in sentence encoders with the real-time constraints of translation memory retrieval.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Pragmatic Profiling of Textual Genres: Characterizing Subjectivity in Human and AI-generated Spanish Texts, Perfilado pragmático de géneros textuales: caracterización de la subjetividad en textos humanos y generados por IA en español

  • Pérez Montero, Alba
  • Pérez Montero, Alba|||/items/933a3a2a-82b5-4d6e-9337-2e478a6c3f42
  • Moreda, Paloma|||/items/aeb186ed-9034-4fdf-96fc-88941be830e7
  • Lloret, Elena|||/items/4593ef69-8308-49d3-8f0f-17e6ee078be5
The use of Generative AI for text creation has become a standard practice; however, a significant stylistic and pragmatic gap remains between human authorship and synthetic Spanish discourse. This study employs a multilevel analysis, based on morphosyntactic metrics and subjectivity markers, to compare scientific Abstracts, News, and Reviews produced by humans and models such as GPT-5, Llama-3.3, and Mistral 3. Results reveal that these models exhibit a hyper-nominal and rigid style, prioritizing informational density over the discursive flexibility of human writing. At the pragmatic level, a homogenization of subjectivity is observed: models fail to replicate the polarized extremes of human style, such as the technical specificity of Abstracts or the personal participation found in Reviews. Finally, cross-domain classification experiments confirm that this reduction in genre-specific subjective markers diminishes the stability of synthetic genres, evidencing a loss of linguistic identity in AI-generated texts.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Hybrid Annotation Framework for Spanish Subjectivity Detection, Marco de anotación híbrido para la detección de subjetividad en español

  • Pérez Montero, Alba
  • Lloret, Elena
  • Moreda, Paloma
  • Pérez Montero, Alba|||/items/933a3a2a-82b5-4d6e-9337-2e478a6c3f42
  • Lloret, Elena|||/items/4593ef69-8308-49d3-8f0f-17e6ee078be5
  • Moreda, Paloma|||/items/aeb186ed-9034-4fdf-96fc-88941be830e7
Subjectivity Detection (SD) is a fundamental NLP task that distinguishes between factual content and opinion. Its analysis provides profound linguistic knowledge, useful for tasks such as disinformation detection. Our objective is to develop and validate a hybrid annotation framework for SD in Spanish. Our methodology includes: (1) a pilot study comparing human annotation against Machine Learning (ML) and Large Language Models (LLMs); (2) an error analysis to refine the framework; and (3) the application of the consolidated framework. Results show ML achieving higher precision, but overlooking complex structures (e.g., subjunctive). The consolidated hybrid methodology is tested in three annotation setups (human [A1], unrevised automatic [A2], consolidated semi-automatic [A3]). A3 achieves the best balance between quality and time consumption (only 19% of the manual annotation effort).
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Pragmatic Profiling for Disinformation Detection: An Exploratory Analysis of Stylistic Features in Spanish News

  • Pérez Montero, Alba
  • Miró Maestre, María
  • Lloret, Elena
  • Moreda, Paloma
  • Pérez Montero, Alba|||/items/933a3a2a-82b5-4d6e-9337-2e478a6c3f42
  • Miró Maestre, María|||/items/54fede9e-4116-446d-b668-9871db0ac7f3
  • Lloret, Elena|||/items/4593ef69-8308-49d3-8f0f-17e6ee078be5
  • Moreda, Paloma|||/items/aeb186ed-9034-4fdf-96fc-88941be830e7
The spread of misleading and fabricated information has made automatic disinformation detection a central challenge for Natural Language Processing. While most approaches have focused on lexical, syntactic, or semantic cues, deceptive discourse is also shaped by pragmatic choices that reflect how information is framed, qualified, and directed toward readers. This paper investigates whether pragmatic information can improve disinformation detection in Spanish by explicitly incorporating two annotation layers: communicative intentions and subjectivity markers. We compare the impact of these pragmatic features across three modeling paradigms: traditional classifiers, an encoder-based transformer (RoBERTa), and generative language models (GPT-oss and Mistral-small). Our results show that pragmatic augmentation consistently improves over text-only baselines, with subjectivity markers displaying stronger discriminative power than intention labels. Statistical testing further confirms that the observed gains are robust for the generative models evaluated. These findings support the view that authorial stance and communicative purpose provide useful complementary evidence for veracity classification.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

Quality over Quantity: Ontology Dataset Selection for Efficient Triple Generation in LLMs, Calidad frente a cantidad: selecci´on de datasets ontol´ogicos para la generaci´on eficiente de triples en LLMs

  • Canal-Esteve, Miquel
  • Canal-Esteve, Miquel|||/items/f1b086bf-bce4-49ea-b765-73081f7a27f9
  • Abreu Salas, José Ignacio|||/items/99fd9ec6-0b97-4c02-a28e-98bc309cff48
  • Gutiérrez, Yoan|||/items/8f7f89db-a015-4ed6-85b9-75bd041ee8bd
  • Muñoz, Rafael|||/items/d6ef1286-fc50-4e8b-9d97-bcbc30e3d1bc
The integration of structured semantic knowledge into Large Language Models (LLMs) is increasingly relevant for knowledge-intensive Natural Language Processing tasks. Ontologies play a central role in organizing this knowledge, yet their manual construction and maintenance become inefficient as digital information grows. While LLMs are increasingly used for ontology learning, their ability to generate coherent structural knowledge remains limited, particularly in small open-source models needed for efficient and sustainable deployment. This work investigates a controlled ontology-to-ontology generation setup to evaluate LLMs’ structural generalization abilities and introduces a data-centric continual pretraining methodology based on ontology quality. Two open-source models, LLaMA 3.2-1B and GEMMA 3-1B, are trained and evaluated using structural and formatting criteria. Results show that selecting only high-quality ontology data substantially improves triple generation while reducing training time by an average of 67% for 1 epoch.
RUA. Repositorio Institucional de la Universidad de Alicante
Publicación Otros (Other)

El lenguaje figurado en los LLM: Evaluación y análisis, Figurative Language in LLMs: Evaluation and Analysis

  • Ramírez Trives, Marta
  • Lloret, Elena
  • Bonet-Jover, Alba
  • Lloret, Elena|||/items/4593ef69-8308-49d3-8f0f-17e6ee078be5
  • Bonet-Jover, Alba|||/items/6bd6e7a1-46a7-415c-bdde-2a1e56f97a2f
Pese a los avances en Procesamiento del Lenguaje Natural (PLN), el lenguaje figurado sigue siendo un reto para los grandes modelos de lenguaje (LLM). En este estudio evaluamos y comparamos el rendimiento de los LLM ChatGPT-4o, DeepSeek- V3, Gemini 2.5 Pro Experimental y Grok 3 Preview en tareas de procesamiento del lenguaje figurado. Basándonos en la teoría del lenguaje figurativo convencional (CFLT) de Dobrovol’skij y Piirainen (2021), compilamos CLUE, un conjunto de datos inglés-español de metáforas, modismos, símiles y proverbios. A partir de él, diseñamos y llevamos a cabo pruebas de detección, interpretación y generación de lenguaje figurado. Los modelos ofrecieron resultados excelentes en detección (0,94) e interpretación (0,91) en ambas lenguas. Sin embargo, exceptuando a Gemini, su rendimiento bajó notablemente en la prueba de generación (0,70), especialmente en español. Concluimos que los LLM son eficaces en tareas de comprensión del lenguaje figurado, pero su capacidad para producirlo es más limitada.