RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Publicación
Artículo científico (JournalArticle)
From formal to informal: how register shift confuses LLMs in Spanish dialect recognition
[EN] Language dialects and registers represent complex data shifts. The use of LLMs can take advantage of more abstract semantic features compared to traditional approaches based on word frequencies. We construct a register-aware benchmark spanning five countries (Argentina, Chile, Colombia, Mexico, Spain) and three registers (formal, informal, mixed). Four open small LLMs (LLaMA, Gemma, Mistral, DeepSeek) are fine-tuned for 5-way classification, evaluated with all train-test register pairings, and compared with an inexpensive classical TF-IDF + Na & iuml;ve Bayes setting. Several main findings emerge: (i) We find a specialization-generalization trade-off and asymmetric cross-register transfer: informal ->\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ ightarrow $$\end{document} formal degrades far less than formal ->\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$ ightarrow $$\end{document} informal; (ii) misclassifications are structured: confusions concentrate in linguistically proximate pairs (e.g., Mexico-Colombia) and align with a low-dimensional geometry from Jensen-Shannon distances; (iii) errors stem from two main sources: over-reliance on toponyms/proper nouns (often defaulting to Spain), and interference from shared slang/morphology., This work has been supported by Open Philanthropy Long-term Future Fund, CIPROM/2022/6 (FASSLOW), funded by Generalitat Valenciana, and Spanish grant PID2024-162030OB-100 (ROBIN), funded by MCIN/AEI/ 10.13039/501100011033 and ERDF A way of making Europe, Catedra ENIA-UPV in Sustainable AI Development, TSI-100930-2023-9, and INCIBE's Chair funded by the EU-NextGenerationEU through the Spanish government's Plan de Recuperacion, Transformacion y Resiliencia, and EUR2024-153548 (PREDAIT) "Towards Predictable AI" from "Spanish Europe Excelencia" 2024, and by the VANTAGE Project, Grant Agreement No. 101249800, funded by the EU through the ECCC under the Digital Europe Programme.