Pasar al contenido principal

Resultados relacionados

Resultados totales (Incluyendo duplicados): 24
Encontrada(s) 1 página(s)

RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

On the Use of Parsing for Named Entity Recognition

  • Alonso, Miguel A.
  • Gómez-Rodríguez, Carlos
  • Vilares, Jesús
[Abstract] Parsing is a core natural language processing technique that can be used to obtain the structure underlying sentences in human languages. Named entity recognition (NER) is the task of identifying the entities that appear in a text. NER is a challenging natural language processing task that is essential to extract knowledge from texts in multiple domains, ranging from financial to medical. It is intuitive that the structure of a text can be helpful to determine whether or not a certain portion of it is an entity and if so, to establish its concrete limits. However, parsing has been a relatively little-used technique in NER systems, since most of them have chosen to consider shallow approaches to deal with text. In this work, we study the characteristics of NER, a task that is far from being solved despite its long history; we analyze the latest advances in parsing that make its use advisable in NER settings; we review the different approaches to NER that make use of syntactic information; and we propose a new way of using parsing in NER based on casting parsing itself as a sequence labeling task., Xunta de Galicia; ED431C 2020/11, Xunta de Galicia; ED431G 2019/01, This work has been funded by MINECO, AEI and FEDER of UE through the ANSWER-ASAP project (TIN2017-85160-C2-1-R); and by Xunta de Galicia through a Competitive Reference Group grant (ED431C 2020/11). CITIC, as Research Center of the Galician University System, is funded by the Consellería de Educación, Universidade e Formación Profesional of the Xunta de Galicia through the European Regional Development Fund (ERDF/FEDER) with 80%, the Galicia ERDF 2014-20 Operational Programme, and the remaining 20% from the Secretaría Xeral de Universidades (Ref. ED431G 2019/01). Carlos Gómez-Rodríguez has also received funding from the European Research Council (ERC), under the European Union’s Horizon 2020 research and innovation programme (FASTPARSE, Grant No. 714150).
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Sentiment Analysis for Fake News Detection

  • Alonso, Miguel A.
  • Vilares, David
  • Gómez-Rodríguez, Carlos
  • Vilares, Jesús
[Abstract] In recent years, we have witnessed a rise in fake news, i.e., provably false pieces of information created with the intention of deception. The dissemination of this type of news poses a serious threat to cohesion and social well-being, since it fosters political polarization and the distrust of people with respect to their leaders. The huge amount of news that is disseminated through social media makes manual verification unfeasible, which has promoted the design and implementation of automatic systems for fake news detection. The creators of fake news use various stylistic tricks to promote the success of their creations, with one of them being to excite the sentiments of the recipients. This has led to sentiment analysis, the part of text analytics in charge of determining the polarity and strength of sentiments expressed in a text, to be used in fake news detection approaches, either as a basis of the system or as a complementary element. In this article, we study the different uses of sentiment analysis in the detection of fake news, with a discussion of the most relevant elements and shortcomings, and the requirements that should be met in the near future, such as multilingualism, explainability, mitigation of biases, or treatment of multimedia elements., Xunta de Galicia; ED431G 2019/01, Xunta de Galicia; ED431C 2020/11, This work has been funded by FEDER/Ministerio de Ciencia, Innovación y Universidades — Agencia Estatal de Investigación through the ANSWERASAP project (TIN2017-85160-C2-1-R); and by Xunta de Galicia through a Competitive Reference Group grant (ED431C 2020/11). CITIC, as Research Center of the Galician University System, is funded by the Consellería de Educación, Universidade e Formación Profesional of the Xunta de Galicia through the European Regional Development Fund (ERDF/FEDER) with 80%, the Galicia ERDF 2014-20 Operational Programme, and the remaining 20% from the Secretaría Xeral de Universidades (ref. ED431G 2019/01). David Vilares is also supported by a 2020 Leonardo Grant for Researchers and Cultural Creators from the BBVA Foundation. Carlos Gómez-Rodríguez has also received funding from the European Research Council (ERC), under the European Union’s Horizon 2020 research and innovation programme (FASTPARSE, grant No. 714150)
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Dependency parsing with bottom-up Hierarchical Pointer Networks

  • Fernández-González, Daniel
  • Gómez-Rodríguez, Carlos
Financiado para publicación en acceso aberto: Universidade da Coruña/CISUG, [Abstract] Dependency parsing is a crucial step towards deep language understanding and, therefore, widely demanded by numerous Natural Language Processing applications. In particular, left-to-right and top-down transition-based algorithms that rely on Pointer Networks are among the most accurate approaches for performing dependency parsing. Additionally, it has been observed for the top-down algorithm that Pointer Networks’ sequential decoding can be improved by implementing a hierarchical variant, more adequate to model dependency structures. Considering all this, we develop a bottom-up oriented Hierarchical Pointer Network for the left-to-right parser and propose two novel transition-based alternatives: an approach that parses a sentence in right-to-left order and a variant that does so from the outside in. We empirically test the proposed neural architecture with the different algorithms on a wide variety of languages, outperforming the original approach in practically all of them and setting new state-of-the-art results on the English and Chinese Penn Treebanks for non-contextualized and BERT-based embeddings., Ministerio de Ciencia e Innovación;PID2020-113230RB-C21, Xunta de Galicia;ED431C2020/11, European Regional Developmet Fund;ED431G 2019/01
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

New Treebank or Repurposed? On the Feasibility of Cross-Lingual Parsing of Romance Languages with Universal Dependencies

  • García, Marcos
  • Gómez-Rodríguez, Carlos
  • Alonso, Miguel A.
This is the final peer-reviewed manuscript that was accepted for publication in Natural Language Engineering. Changes resulting from the publishing process, such as editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document., [Abstract] This paper addresses the feasibility of cross-lingual parsing with Universal Dependencies (UD) between Romance languages, analyzing its performance when compared to the use of manually annotated resources of the target languages. Several experiments take into account factors such as the lexical distance between the source and target varieties, the impact of delexicalization, the combination of different source treebanks or the adaptation of resources to the target language, among others. The results of these evaluations show that the direct application of a parser from one Romance language to another reaches similar labeled attachment score (LAS) values to those obtained with a manual annotation of about 3,000 tokens in the target language, and unlabeled attachment score (UAS) results equivalent to the use of around 7,000 tokens, depending on the case. These numbers can noticeably increase by performing a focused selection of the source treebanks. Furthermore, the removal of the words in the training corpus (delexicalization) is not useful in most cases of cross-lingual parsing of Romance languages. The lessons learned with the performed experiments were used to build a new UD treebank for Galician, with 1,000 sentences manually corrected after an automatic cross-lingual annotation. Several evaluations in this new resource show that a cross-lingual parser built with the best combination and adaptation of the source treebanks performs better (77 percent LAS and 82 percent UAS) than using more than 16,000 (for LAS results) and more than 20,000 (UAS) manually labeled tokens of Galician., Ministerio de Economía y Competitividad; FJCI-2014-22853, Ministerio de Economía y Competitividad; FFI2014-51978-C2-1-R, Ministerio de Economía y Competitividad; FFI2014-51978-C2-2-R
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Bertinho: Galician BERT Representations, Bertinho: Representaciones BERT para el gallego

  • Vilares, David
  • García, Marcos
  • Gómez-Rodríguez, Carlos
[Abstract]: This paper presents a monolingual BERT model for Galician. We follow the recent trend that shows that it is feasible to build robust monolingual BERT models even for relatively low-resource languages, while performing better than the well-known official multilingual BERT (mBERT). More particularly, we release two monolingual Galician BERT models, built using 6 and 12 transformer layers, respectively; trained with limited resources (∼45 million tokens on a single GPU of 24GB). We then provide an exhaustive evaluation on a number of tasks such as POS-tagging, dependency parsing and named entity recognition. For this purpose, all these tasks are cast in a pure sequence labeling setup in order to run BERT without the need to include any additional layers on top of it (we only use an output classification layer to map the contextualized representations into the predicted label). The experiments show that our models, especially the 12-layer one, outperform the results of mBERT in most tasks., [Resumen]: Este artículo presenta un modelo BERT monolingüe para el gallego. Nos basamos en la tendencia actual que ha demostrado que es posible crear modelos BERT monolingües robustos incluso para aquellos idiomas para los que hay una relativa escasez de recursos, funcionando éstos mejor que el modelo BERT multilingüe oficial (mBERT). Concretamente, liberamos dos modelos monolingües para el gallego, creados con 6 y 12 capas de transformers, respectivamente, y entrenados con una limitada cantidad de recursos (~45 millones de palabras sobre una única GPU de 24GB.) Para evaluarlos realizamos un conjunto exhaustivo de experimentos en tareas como análisis morfosintáctico, análisis sintáctico de dependencias o reconocimiento de entidades. Para ello, abordamos estas tareas como etiquetado de secuencias, con el objetivo de ejecutar los modelos BERT sin la necesidad de incluir ninguna capa adicional (únicamente se a~nade la capa de salida encargada de transformar las representaciones contextualizadas en la etiqueta predicha). Los experimentos muestran que nuestros modelos, especialmente el de 12 capas, mejoran los resultados de mBERT en la mayor parte de las tareas., This work has received funding from the European Research Council (ERC), which has funded this research under the European Union's Horizon 2020 research and innovation programme (FASTPARSE, grant agreement No 714150), from MINECO (ANSWER-ASAP, TIN2017-85160-C2-1-R), from Xunta de Galicia (ED431C 2020/11), from Centro de Investigación de Galicia `CITIC', funded by Xunta de Galicia and the European Union (European Regional Development Fund- Galicia 2014-2020 Program), by grant ED431G 2019/01, and by Centro Singular de Investigación en Tecnoloxías Intelixentes (CiTIUS), ERDF 2014-2020: Call ED431G 2019/04. DV is supported by a 2020 Leonardo Grant for Researchers and Cultural Creators from the BBVA Foundation. MG is supported by a Ramón y Cajal grant (RYC2019-028473-I)., Xunta de Galicia; ED431C 2020/11, Xunta de Galicia; ED431G 2019/01, Xunta de Galicia; D431G 2019/04
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Towards Robust Word Embeddings for Noisy Texts

  • Doval, Yerai
  • Vilares, Jesús
  • Gómez-Rodríguez, Carlos
[Abstract] Research on word embeddings has mainly focused on improving their performance on standard corpora, disregarding the difficulties posed by noisy texts in the form of tweets and other types of non-standard writing from social media. In this work, we propose a simple extension to the skipgram model in which we introduce the concept of bridge-words, which are artificial words added to the model to strengthen the similarity between standard words and their noisy variants. Our new embeddings outperform baseline models on noisy texts on a wide range of evaluation tasks, both intrinsic and extrinsic, while retaining a good performance on standard texts. To the best of our knowledge, this is the first explicit approach at dealing with these types of noisy texts at the word embedding level that goes beyond the support for out-of-vocabulary words., Yerai Doval has been supported by the Spanish Ministry of Economy, Industry and Competitiveness (MINECO) through the ANSWER-ASAP project (TIN2017-85160-C2-2-R); by the Spanish State Secretariat for Research, Development and Innovation (which belongs to MINECO) and the European Social Fund (ESF) through a FPI fellowship (BES-2015-073768) associated with TELEPARES project (FFI2014-51978-C2-1-R); and by the Xunta de Galicia through TELGALICIA research network (ED431D 2017/12). The work of Jesús Vilares and Carlos Gómez-Rodríguez has also been funded by MINECO through the ANSWER-ASAP project (TIN2017-85160-C2-1-R in this case); and by Xunta de Galicia through a Group with Potential for Growth grant (ED431B 2017/01), a Competitive Reference Group grant (ED431C 2020/11), and a Remarkable Research Centre grant for the CITIC research centre (ED431G/01), the latter co-funded by EU with ERDF funding. Finally, Carlos Gómez-Rodríguez has also received funding from the European Research Council (ERC), under the European Union’s Horizon 2020 research and innovation programme (FASTPARSE, Grant No. 714150), Xunta de Galicia; ED431D 2017/12, Xunta de Galicia; ED431B 2017/01, Xunta de Galicia; ED431C 2020/11, Xunta de Galicia; ED431G/01
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Optimality of syntactic dependency distances

  • Ferrer-i-Cancho, Ramon
  • Gómez-Rodríguez, Carlos
  • Esteban, Juan Luis
  • Alemany-Puig, Lluís
[Abstract]: It is often stated that human languages, as other biological systems, are shaped by cost-cutting pressures but, to what extent? Attempts to quantify the degree of optimality of languages by means of an optimality score have been scarce and focused mostly on English. Here we recast the problem of the optimality of the word order of a sentence as an optimization problem on a spatial network where the vertices are words, arcs indicate syntactic dependencies, and the space is defined by the linear order of the words in the sentence. We introduce a score to quantify the cognitive pressure to reduce the distance between linked words in a sentence. The analysis of sentences from 93 languages representing 19 linguistic families reveals that half of languages are optimized to a 70 % or more. The score indicates that distances are not significantly reduced in a few languages and confirms two theoretical predictions: that longer sentences are more optimized and that distances are more likely to be longer than expected by chance in short sentences. We present a hierarchical ranking of languages by their degree of optimization. The score has implications for various fields of language research (dependency linguistics, typology, historical linguistics, clinical linguistics, and cognitive science). Finally, the principles behind the design of the score have implications for network science., We are grateful to C. Bentz, N. Català, M. H. Christiansen, M. Gustison, and A. Hernández-Fernández for helpful comments. R.F.C. and L.A.P. are supported by Grant No. TIN2017-89244-R from MINECO (Ministerio de Economía, Industria y Competitividad). R.F.C. is also supported by the recognition No. 2017SGR-856 (MACDA) from AGAUR (Generalitat de Catalunya). L.A.P. is also supported by Secretaria d’Universitats i Recerca de la Generalitat de Catalunya and the Social European Fund. C.G.R. is supported by the European Research Council (ERC), under the European Union’s Horizon 2020 research and innovation program (FASTPARSE, Grant No. 714150), the ANSWER-ASAP project (No. TIN2017-85160-C2-1-R) and SCANNER-UDC project (No. PID2020-113230RB-C21) from ERDF/MICINN-AEI, and Xunta de Galicia (ED431C 2020/11 and an Oportunius program grant to complement ERC grants); the CITIC research center is funded by ERDF and Xunta de Galicia (ERDFGalicia 2014-2020 program, Grant No. ED431G 2019/01). J.L.E. is funded by Grants No. TIN2016-76573-C2-1-P and No. PID2019-109137GB-C22 from MINECO. R.F.C. designed the research; R.F.C and C.G.R. wrote the paper; C.G.R. collected and preprocessed data; C.G.R. and R.F.C. analyzed the data; R.F.C, C.G.R., and J.L.E developed the mathematical arguments; C.G.R., J.L.E., and L.A.P. wrote the core code; all authors verified the mathematical arguments., Xunta de Galicia; ED431C 2020/11, Xunta de Galicia; ED431G 2019/01, Agència de Gestió d'Ajuts Universitaris i de Recerca; 2017SGR-856
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Comparing neural- and N-gram-based language models for word segmentation

  • Doval, Yerai
  • Gómez-Rodríguez, Carlos
[Abstract]: Word segmentation is the task of inserting or deleting word boundary characters in order to separate character sequences that correspond to words in some language. In this article we propose an approach based on a beam search algorithm and a language model working at the byte/character level, the latter component implemented either as an n-gram model or a recurrent neural network. The resulting system analyzes the text input with no word boundaries one token at a time, which can be a character or a byte, and uses the information gathered by the language model to determine if a boundary must be placed in the current position or not. Our aim is to use this system in a preprocessing step for a microtext normalization system. This means that it needs to effectively cope with the data sparsity present on this kind of texts. We also strove to surpass the performance of two readily available word segmentation systems: The well-known and accessible Word Breaker by Microsoft, and the Python module WordSegment by Grant Jenks. The results show that we have met our objectives, and we hope to continue to improve both the precision and the efficiency of our system in the future. © 2018 The Authors. Journal of the Association for Information Science and Technology published by Wiley Periodicals, Inc. on behalf of ASIS&T., This research received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program (grant agreement no. 714150-FASTPARSE). It was partially funded by the Spanish Ministry of Economy and Competitiveness (MINECO) through projects FFI2014-51978-C2-1-R and FFI2014-51978-C2-2-R, and by the Autonomous Government of Galicia through both the Galician Network for Lexicography-RELEX (ED431D R2016/046) and Grant ED431B-2017/01. Moreover, Yerai Doval is funded by the Spanish State Secretariat for Research, Development and Innovation (which belongs to MINECO) and by the European Social Fund (ESF) under an FPI fellowship (BES-2015-073768) associated with project FFI2014-51978-C2-1-R. We gratefully acknowledge NVIDIA Corporation for the donation of a GTX Titan X GPU used for this research., Xunta de Galicia; ED431D R2016/046, Xunta e Galicia; ED431B-2017/01
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

How important is syntactic parsing accuracy? An empirical evaluation on rule-based sentiment analysis

  • Gómez-Rodríguez, Carlos
  • Alonso-Alonso, Iago
  • Vilares, David
This version of the article has been accepted for publication, after peer review and is subject to Springer Nature’s AM terms of use, but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: https://doi.org/10.1007/s10462-017-9584-0, [Abstract]: Syntactic parsing, the process of obtaining the internal structure of sentences in natural languages, is a crucial task for artificial intelligence applications that need to extract meaning from natural language text or speech. Sentiment analysis is one example of application for which parsing has recently proven useful. In recent years, there have been significant advances in the accuracy of parsing algorithms. In this article, we perform an empirical, task-oriented evaluation to determine how parsing accuracy influences the performance of a state-of-the-art rule-based sentiment analysis system that determines the polarity of sentences from their parse trees. In particular, we evaluate the system using four well-known dependency parsers, including both current models with state-of-the-art accuracy and more innacurate models which, however, require less computational resources. The experiments show that all of the parsers produce similarly good results in the sentiment analysis task, without their accuracy having any relevant influence on the results. Since parsing is currently a task with a relatively high computational cost that varies strongly between algorithms, this suggests that sentiment analysis researchers and users should prioritize speed over accuracy when choosing a parser; and parsing researchers should investigate models that improve speed further, even at some cost to accuracy., Carlos Gómez-Rodríguez has received funding from the European Research Council (ERC), under the European Union’s Horizon 2020 research and innovation programme (FASTPARSE, Grant Agreement No 714150), Ministerio de Economía y Competitividad (FFI2014-51978-C2-2-R), and the Oportunius Program (Xunta de Galicia). Iago Alonso-Alonso was funded by an Oportunius Program Grant (Xunta de Galicia). David Vilares has received funding from the Ministerio de Educación, Cultura y Deporte (FPU13/01180) and Ministerio de Economía y Competitividad (FFI2014-51978-C2-2-R).
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Multitask Pointer Network for Multi-Representational Parsing

  • Fernández-González, Daniel
  • Gómez-Rodríguez, Carlos
Financiado para publicación en acceso aberto: Universidade da Coruña/CISUG, [Abstract] Dependency and constituent trees are widely used by many artificial intelligence applications for representing the syntactic structure of human languages. Typically, these structures are separately produced by either dependency or constituent parsers. In this article, we propose a transition-based approach that, by training a single model, can efficiently parse any input sentence with both constituent and dependency trees, supporting both continuous/projective and discontinuous/non-projective syntactic structures. To that end, we develop a Pointer Network architecture with two separate task-specific decoders and a common encoder, and follow a multitask learning strategy to jointly train them. The resulting quadratic system, not only becomes the first parser that can jointly produce both unrestricted constituent and dependency trees from a single model, but also proves that both syntactic formalisms can benefit from each other during training, achieving state-of-the-art accuracies in several widely-used benchmarks such as the continuous English and Chinese Penn Treebanks, as well as the discontinuous German NEGRA and TIGER datasets., We acknowledge the European Research Council (ERC), which has funded this research under the European Union’s Horizon 2020 research and innovation programme (FASTPARSE, grant agreement No 714150), ERDF/MICINN-AEI (ANSWER-ASAP, TIN2017-85160-C2-1-R; SCANNER-UDC, PID2020-113230RB-C21), Xunta de Galicia, Spain (ED431C 2020/11), and Centro de Investigación de Galicia “CITIC”, funded by Xunta de Galicia, Spain and the European Union (ERDF - Galicia 2014–2020 Program), by grant ED431G 2019/01. Funding for open access charge: Universidade da Coruña / CISUG, Xunta de Galicia; ED431C 2020/11, Xunta de Galicia; ED431G 2019/01
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Transition-based semantic role labeling with pointer networks

  • Fernández-González, Daniel
[Abstract] Semantic role labeling (SRL) focuses on recognizing the predicate–argument structure of a sentence and plays a critical role in many natural language processing tasks such as machine translation and question answering. Practically all available methods do not perform full SRL, since they rely on pre-identified predicates, and most of them follow a pipeline strategy, using specific models for undertaking one or several SRL subtasks. In addition, previous approaches have a strong dependence on syntactic information to achieve state-of-the-art performance, despite being syntactic trees equally hard to produce. These simplifications and requirements make the majority of SRL systems impractical for real-world applications. In this article, we propose the first transition-based SRL approach that is capable of completely processing an input sentence in a single left-to-right pass, with neither leveraging syntactic information nor resorting to additional modules. Thanks to our implementation based on Pointer Networks, full SRL can be accurately and efficiently done in , achieving the best performance to date on the majority of languages from the CoNLL-2009 shared task., Ministerio de Ciencia e Innovación ; SCANNER-UDC,PID2020-113230RB-C21), CITIC ; ED431G 2019/01., Xunta de Galicia ; ED431C 2020/11
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Natural Language Parsing : Progress and Challenges

  • Gómez-Rodríguez, Carlos
[Abstract] Natural language parsing is the task of automatically obtaining the syntactic structure of sentences written in a human language. Parsing is a crucial step for language processing systems that need to extract meaning from text or speech, and thus a key technology of artificial intelligence. This article presents an outline of the current state of the art in this field, as well as reflections on the main challenges that, in the author's opinion, it is currently facing: limitations in accuracy on especially difficult languages and domains, psycholinguistic adequacy, and speed., Xunta de Galicia; ED431B 2017/01, Ministerio de Economía y Competitividad; FFI2014-51978-C2-2-R, Ministerio de Economía y Competitividad; TIN2017-85160-C2-1-R
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Faster shift-reduce constituent parsing with a non-binary, bottom-up strategy

  • Fernández-González, Daniel
  • Gómez-Rodríguez, Carlos
© 2019. This is the final peer-reviewed manuscript that was accepted for publication at Artificial Intelligence and made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/ licenses/by-nc-nd/4.0/. This may not reflect subsequent changes resulting from the publishing process such as editing, formatting, pagination, and other quality control mechanisms. The final journal publication is available at https://doi.org/10.1016/j.artint.2019.07.006, [Absctract]: An increasingly wide range of artificial intelligence applications rely on syntactic information to process and extract meaning from natural language text or speech, with constituent trees being one of the most widely used syntactic formalisms. To produce these phrase-structure representations from sentences in natural language, shift-reduce constituent parsers have become one of the most efficient approaches. Increasing their accuracy and speed is still one of the main objectives pursued by the research community so that artificial intelligence applications that make use of parsing outputs, such as machine translation or voice assistant services, can improve their performance. With this goal in mind, we propose in this article a novel non-binary shift-reduce algorithm for constituent parsing. Our parser follows a classical bottom-up strategy but, unlike others, it straightforwardly creates non-binary branchings with just one transition, instead of requiring prior binarization or a sequence of binary transitions, allowing its direct application to any language without the need of further resources such as percolation tables. As a result, it uses fewer transitions per sentence than existing transition-based constituent parsers, becoming the fastest such system and, as a consequence, speeding up downstream applications. Using static oracle training and greedy search, the accuracy of this novel approach is on par with state-of-the-art transition-based constituent parsers and outperforms all top-down and bottom-up greedy shift-reduce systems on the Wall Street Journal section from the English Penn Treebank and the Penn Chinese Treebank. Additionally, we develop a dynamic oracle for training the proposed transition-based algorithm, achieving further improvements in both benchmarks and obtaining the best accuracy to date on the Penn Chinese Treebank among greedy shift-reduce parsers., This work has received funding from the European Research Council (ERC), under the European Union's Horizon 2020 research and innovation programme (FASTPARSE, grant agreement No 714150), from the ANSWER-ASAP project (TIN2017-85160-C2-1-R) from MINECO, and from Xunta de Galicia (ED431B 2017/01)., Xunta de Galicia; ED431B 2017/01
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

Discontinuous grammar as a foreign language

  • Fernández-González, Daniel
  • Gómez-Rodríguez, Carlos
[Abstract] In order to achieve deep natural language understanding, syntactic constituent parsing is a vital step, highly demanded by many artificial intelligence systems to process both text and speech. One of the most recent proposals is the use of standard sequence-to-sequence models to perform constituent parsing as a machine translation task, instead of applying task-specific parsers. While they show a competitive performance, these text-to-parse transducers are still lagging behind classic techniques in terms of accuracy, coverage and speed. To close the gap, we here extend the framework of sequence-to-sequence models for constituent parsing, not only by providing a more powerful neural architecture for improving their performance, but also by enlarging their coverage to handle the most complex syntactic phenomena: discontinuous structures. To that end, we design several novel linearizations that can fully produce discontinuities and, for the first time, we test a sequence-to-sequence model on the main discontinuous benchmarks, obtaining competitive results on par with task-specific discontinuous constituent parsers and achieving state-of-the-art scores on the (discontinuous) English Penn Treebank., Xunta de Galicia; ED431G 2019/01, Xunta de Galicia; ED431C 2020/11, We acknowledge the European Research Council (ERC), which has funded this research under the European Union’s Horizon 2020 research and innovation programme (FASTPARSE, grant agreement No 714150) and the Horizon Europe research and innovation programme (SALSA, grant agreement No 101100615), ERDF/ MICINN-AEI (SCANNER-UDC, PID2020-113230RB-C21), Xunta de Galicia (ED431C 2020/11), and Centro de Investigación de Galicia ‘‘CITIC”, funded by Xunta de Galicia and the European Union (ERDF - Galicia 2014–2020 Program), by grant ED431G 2019/01. Funding for open access charge: Universidade da Coruña/CISUG.
RUC. Repositorio da Universidade da Coruña
Publicación Artículo científico (JournalArticle)

The Impact of Edge Displacement Vaserstein Distance on UD Parsing Performance

  • Anderson, Mark
  • Gómez-Rodríguez, Carlos
[Abstract] We contribute to the discussion on parsing performance in NLP by introducing a measurement that evaluates the differences between the distributions of edge displacement (the directed distance of edges) seen in training and test data. We hypothesize that this measurement will be related to differences observed in parsing performance across treebanks. We motivate this by building upon previous work and then attempt to falsify this hypothesis by using a number of statistical methods. We establish that there is a statistical correlation between this measurement and parsing performance even when controlling for potential covariants. We then use this to establish a sampling technique that gives us an adversarial and complementary split. This gives an idea of the lower and upper bounds of parsing systems for a given treebank in lieu of freshly sampled data. In a broader sense, the methodology presented here can act as a reference for future correlation-based exploratory work in NLP., Ministerio de Ciencia e Innovación; TIN2017-85160-C2-1-R, Ministerio de Ciencia e Innovación; PID2020-113230RB-C21, Xunta de Galicia; ED431C 2020/11, Xunta de Galicia; ED431G 2019/01
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

Scarcity of crossing dependencies: A direct outcome of a specific constraint?

  • Gómez Rodríguez, Carlos
  • Ferrer Cancho, Ramon|||0000-0002-7820-923X
The structure of a sentence can be represented as a network where vertices are words and edges indicate syntactic dependencies. Interestingly, crossing syntactic dependencies have been observed to be infrequent in human languages. This leads to the question of whether the scarcity of crossings in languages arises from an independent and specific constraint on crossings. We provide statistical evidence suggesting that this is not the case, as the proportion of dependency crossings of sentences from a wide range of languages can be accurately estimated by a simple predictor based on a null hypothesis on the local probability that two dependencies cross given their lengths. The relative error of this predictor never exceeds 5 % on average, whereas the error of a baseline predictor assuming a random ordering of the words of a sentence is at least six times greater. Our results suggest that the low frequency of crossings in natural languages is neither originated by hidden knowledge of language nor by the undesirability of crossings per se, but as a mere side effect of the principle of dependency length minimization., Peer Reviewed
UPCommons. Portal del coneixement obert de la UPC
Publicación Artículo científico (JournalArticle)

A dependency look at the reality of constituency

  • Chen, Xinying
  • Gómez Rodríguez, Carlos
  • Ferrer Cancho, Ramon|||0000-0002-7820-923X
A comment on “Neurophysiological dynamics of phrase-structure building during sentence processing” by Nelson et al (2017), Proceedings of the National Academy of Sciences USA 114(18), E3669-E3678., Peer Reviewed
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

A dependency look at the reality of constituency

  • Chen, Xinying
  • Gómez Rodríguez, Carlos
  • Ferrer Cancho, Ramon
A comment on “Neurophysiological dynamics of phrase-structure building during sentence processing” by Nelson et al (2017), Proceedings of the National Academy of Sciences USA 114(18), E3669-E3678., Peer Reviewed, Postprint (published version)
Recercat. Dipósit de la Recerca de Catalunya
Publicación Artículo científico (JournalArticle)

Scarcity of crossing dependencies: A direct outcome of a specific constraint?

  • Gómez Rodríguez, Carlos
  • Ferrer Cancho, Ramon
The structure of a sentence can be represented as a network where vertices are words and edges indicate syntactic dependencies. Interestingly, crossing syntactic dependencies have been observed to be infrequent in human languages. This leads to the question of whether the scarcity of crossings in languages arises from an independent and specific constraint on crossings. We provide statistical evidence suggesting that this is not the case, as the proportion of dependency crossings of sentences from a wide range of languages can be accurately estimated by a simple predictor based on a null hypothesis on the local probability that two dependencies cross given their lengths. The relative error of this predictor never exceeds 5 % on average, whereas the error of a baseline predictor assuming a random ordering of the words of a sentence is at least six times greater. Our results suggest that the low frequency of crossings in natural languages is neither originated by hidden knowledge of language nor by the undesirability of crossings per se, but as a mere side effect of the principle of dependency length minimization., Peer Reviewed, Postprint (author's final draft)
Investigo. Repositorio Institucional de la Universidade de Vigo
Publicación Artículo científico (JournalArticle)

Faster shift-reduce constituent parsing with a non-binary, bottom-up strategy

  • Fernández González, Daniel
  • Gómez Rodríguez, Carlos
An increasingly wide range of artificial intelligence applications rely on syntactic information to process and extract meaning from natural language text or speech, with constituent trees being one of the most widely used syntactic formalisms. To produce these phrase-structure representations from sentences in natural language, shift-reduce constituent parsers have become one of the most efficient approaches. Increasing their accuracy and speed is still one of the main objectives pursued by the research community so that artificial intelligence applications that make use of parsing outputs, such as machine translation or voice assistant services, can improve their performance. With this goal in mind, we propose in this article a novel non-binary shift-reduce algorithm for constituent parsing. Our parser follows a classical bottom-up strategy but, unlike others, it straightforwardly creates non-binary branchings with just one transition, instead of requiring prior binarization or a sequence of binary transitions, allowing its direct application to any language without the need of further resources such as percolation tables. As a result, it uses fewer transitions per sentence than existing transition-based constituent parsers, becoming the fastest such system and, as a consequence, speeding up downstream applications. Using static oracle training and greedy search, the accuracy of this novel approach is on par with state-of-the-art transition-based constituent parsers and outperforms all top-down and bottom-up greedy shift-reduce systems on the Wall Street Journal section from the English Penn Treebank and the Penn Chinese Treebank. Additionally, we develop a dynamic oracle for training the proposed transition-based algorithm, achieving further improvements in both benchmarks and obtaining the best accuracy to date on the Penn Chinese Treebank among greedy shift-reduce parsers., Agencia Estatal de Investigación | Ref. TIN2017-85160-C2-1-R, Xunta de Galicia | Ref. ED431B 2017/01
Investigo. Repositorio Institucional de la Universidade de Vigo
Publicación Artículo científico (JournalArticle)

Transition-based semantic role labeling with pointer networks

  • Fernández González, Daniel
Semantic role labeling (SRL) focuses on recognizing the predicate–argument structure of a sentence and plays a critical role in many natural language processing tasks such as machine translation and question answering. Practically all available methods do not perform full SRL, since they rely on pre-identified predicates, and most of them follow a pipeline strategy, using specific models for undertaking one or several SRL subtasks. In addition, previous approaches have a strong dependence on syntactic information to achieve state-of-the-art performance, despite being syntactic trees equally hard to produce. These simplifications and requirements make the majority of SRL systems impractical for real-world applications. In this article, we propose the first transition-based SRL approach that is capable of completely processing an input sentence in a single left-to-right pass, with neither leveraging syntactic information nor resorting to additional modules. Thanks to our implementation based on Pointer Networks, full SRL can be accurately and efficiently done in , achieving the best performance to date on the majority of languages from the CoNLL-2009 shared task., Agencia Estatal de Investigación | Ref. PID2020-113230RB-C21, Xunta de Galicia | Ref. ED431C 2020/11, Xunta de Galicia | Ref. ED431G 2019/01
Investigo. Repositorio Institucional de la Universidade de Vigo
Publicación Artículo científico (JournalArticle)

Shift-reduce task-oriented semantic parsing with stack-transformers

  • Fernández González, Daniel
Intelligent voice assistants, such as Apple Siri and Amazon Alexa, are widely used nowadays. These task-oriented dialogue systems require a semantic parsing module in order to process user utterances and understand the action to be performed. This semantic parsing component was initially implemented by rule-based or statistical slot-filling approaches for processing simple queries; however, the appearance of more complex utterances demanded the application of shift-reduce parsers or sequence-to-sequence models. Although shift-reduce approaches were initially considered the most promising option, the emergence of sequence-to-sequence neural systems has propelled them to the forefront as the highest-performing method for this particular task. In this article, we advance the research on shift-reduce semantic parsing for task-oriented dialogue. We implement novel shift-reduce parsers that rely on Stack-Transformers. This framework allows to adequately model transition systems on the transformer neural architecture, notably boosting shift-reduce parsing performance. Furthermore, our approach goes beyond the conventional top-down algorithm: we incorporate alternative bottom-up and in-order transition systems derived from constituency parsing into the realm of task-oriented parsing. We extensively test our approach on multiple domains from the Facebook TOP benchmark, improving over existing shift-reduce parsers and state-of-the-art sequence-to-sequence models in both high-resource and low-resource settings. We also empirically prove that the in-order algorithm substantially outperforms the commonly used top-down strategy. Through the creation of innovative transition systems and harnessing the capabilities of a robust neural architecture, our study showcases the superiority of shift-reduce parsers over leading sequence-to-sequence methods on the main benchmark., Xunta de Galicia | Ref. ED431C 2020/11, Xunta de Galicia | Ref. ED431G 2019/01, Universidade de Vigo/CISUG, European Comission | Ref. 714150, Agencia Estatal de Investigación | Ref. PID2020-113230RB-C21
Investigo. Repositorio Institucional de la Universidade de Vigo
Publicación Artículo científico (JournalArticle)

Multitask pointer network for multi-representational parsing

  • Fernández González, Daniel
  • Gómez Rodríguez, Carlos
Dependency and constituent trees are widely used by many artificial intelligence applications for representing the syntactic structure of human languages. Typically, these structures are separately produced by either dependency or constituent parsers. In this article, we propose a transition-based approach that, by training a single model, can efficiently parse any input sentence with both constituent and dependency trees, supporting both continuous/projective and discontinuous/non-projective syntactic structures. To that end, we develop a Pointer Network architecture with two separate task-specific decoders and a common encoder, and follow a multitask learning strategy to jointly train them. The resulting quadratic system, not only becomes the first parser that can jointly produce both unrestricted constituent and dependency trees from a single model, but also proves that both syntactic formalisms can benefit from each other during training, achieving state-of-the-art accuracies in several widely-used benchmarks such as the continuous English and Chinese Penn Treebanks, as well as the discontinuous German NEGRA and TIGER datasets., Agencia Estatal de Investigación | Ref. TIN2017-85160-C2-1-R, Agencia Estatal de Investigación | Ref. PID2020-113230RB-C21, Xunta de Galicia | Ref. ED431C 2020/11, Xunta de Galicia | Ref. ED431G 2019/01, Universidade da Coruña/CISUG
Investigo. Repositorio Institucional de la Universidade de Vigo
Publicación Artículo científico (JournalArticle)

Dependency parsing with bottom-up Hierarchical Pointer Networks

  • Fernández González, Daniel
  • Gómez Rodríguez, Carlos
Dependency parsing is a crucial step towards deep language understanding and, therefore, widely demanded by numerous Natural Language Processing applications. In particular, left-to-right and top-down transition-based algorithms that rely on Pointer Networks are among the most accurate approaches for performing dependency parsing. Additionally, it has been observed for the top-down algorithm that Pointer Networks’ sequential decoding can be improved by implementing a hierarchical variant, more adequate to model dependency structures. Considering all this, we develop a bottom-up oriented Hierarchical Pointer Network for the left-to-right parser and propose two novel transition-based alternatives: an approach that parses a sentence in right-to-left order and a variant that does so from the outside in. We empirically test the proposed neural architecture with the different algorithms on a wide variety of languages, outperforming the original approach in practically all of them and setting new state-of-the-art results on the English and Chinese Penn Treebanks for non-contextualized and BERT-based embeddings., Agencia Estatal de Investigación | Ref. PID2020-113230RB-C21, Xunta de Galicia | Ref. ED431C 2020/11, Xunta de Galicia | Ref. ED431G 2019/01