3/11/2021

Carlos Gómez Rodríguez and Daniel Fernández González will attend the 2021 Conference on Empirical Methods in Natural Language Processing that will take place in Punta Cana (Dominican Republic) from 7 to 11 November 2021. They will present their developments at the research group:

  • Reducing Discontinuous to Continuous Parsing with Pointer Network Reordering

Discontinuous parsing is an especially challenging form of syntactic parsing which is needed to describe various syntactic phenomena, especially on free-word-order languages, but is often avoided because it makes parsing difficult. This paper presents an approach to reduce discontinuous to continuous (standard) parsing, in such a way that we can use any existing continuous parser to recognize those discontinuous phenomena, thus making their handling much easier. The approach is based on reordering sentences with a specific neural network called a Pointer Network before and after parsing.


23/01/2020

Carlos Gómez Rodríguez, Daniel Fernández González and David Vilares Calvo will attend the 34th AAAI Conference on Artificial Intelligence (AAAI-20) that will take place in New York (USA) from February 7th through February 12th, 2020. Below we include a small summary of the works that the researchers will presented at the conference:

  • Discontinuous Constituent Parsing with Pointer Networks

One of the most complex syntactic representations used in computational linguistics and NLP are discontinuous constituent trees, crucial for representing all grammatical phenomena of languages such as German. We propose a novel neural network architecture that, by means of Pointer Networks, is able to generate the most accurate discontinuous constituent representations to date. To do so, we internally model discontinuous constituent structures as augmented non-projective dependency structures. The proposed approach achieves state-of-the-art results on the two widely-used NEGRA and TIGER benchmarks, outperforming previous work by a wide margin.

  • Parsing as pretraining

Pretraining encoders are neural architectures that learn to generate continuous representations of words, i.e. word vectors, based on language modeling objectives (e.g. predicting the next word given the previous context). Recent analyses suggest that these encoders capture certain morpho-syntactic structure. However, probing frameworks for word vectors still do not report results on standard setups such as constituent and dependency parsing. This paper addresses this problem and does full parsing (on English) relying only on these pretraining architectures. To do so, we first cast constituent and dependency parsing as sequence tagging. We then use a single feed-forward layer to directly map word vectors to labels that encode a linearized tree. This is used to: (i) see how far we can reach on syntax modelling with just pretrained encoders, and (ii) shed some light about the syntax-sensitivity of different word vectors (by freezing the weights of the pretraining network during training).

17/10/2019

David Vilares Calvo and Michalina Strzyz will attend the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP2019) that will take place in Hong Kong (China) from November 3rd through November 7th, 2019. The researchers will present there their work:

    • Towards Making a Dependency Parser See

We explore whether it is possible to leverage eye-tracking data in an RNN dependency parser (for English) when such information is only available during training, i.e., no aggregated or token-level gaze features are used at inference time. To do so, we train a multitask learning model that parses sentences as sequence labeling and leverages gaze features as auxiliary tasks. Our method also learns to train from disjoint datasets, i.e. it can be used to test whether already collected gaze features are useful to improve the performance on new non-gazed annotated treebanks. Accuracy gains are modest but positive, showing the feasibility of the approach. It can serve as a first step towards architectures that can better leverage eye-tracking data or other complementary information available only for training sentences, possibly leading to improvements in syntactic parsing.

28/07/2019

Carlos Gómez-Rodríguez, David Vilares Calvo and Michalina Strzyz will attend the 2019 Annual Conference of the Association for Computational Linguistics (ACL 2019) that will take place in Florence (Italy) from July 28th to August 2nd, 2019. They will present their recent developments at the research group.

    • HEAD-QA: A Healthcare Dataset for Complex Reasoning

In recent years, neural networks have made significant progress in question answering, an area of natural language processing that tries to automatically find the answer to a given question. However, the configurations on which these experiments are performed are simple, and do not require reasoning, being often limited to factual questions (for example, in what year was Nikola Tesla born?). In this paper, we propose to use the questions of the *IR exams (used to access a resident position in the public healthcare system) as a more complex and also more real assessment environment. We then perform experiments in both Spanish, and a version automatically translated into English, using information recovery techniques and neural networks. The experiments demonstrate the utility to evaluate this type of models, which obtain positive, but modest scores and far from the results expected by human experts.

    • Sequence Labeling Parsing by Learning Across Representations

In linguistics, the terms constituents and dependency grammar are used to represent the syntactic structure of a sentence. In natural language processing, we talk about syntactic analyzers of constituents and dependencies to refer to computational models that automatically obtain these structures. However, most previous studies study these two problems separately, due to the differences between them. In this paper, we explore an extremely simple method to learn both representations at the same time. To do this, we obtain a linear representation of the representations of constituents and dependencies so that given a sentence of length n, these structures are represented by exactly n labels. Next, we train a single neural network for sequence tagging that solves both tasks simultaneously. The results show that our method is accurate and fast; and that the joint learning of both representations is even beneficial compared to learning each of them separately.

2/05/2019

Carlos Gómez-Rodríguez, Daniel Fernández-González, David Vilares Calvo and Michalina Strzyz will attend the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT 2019) that will take place in Minneapolis (EEUU) from June 2nd through June 7th, 2019. They will present their recent developments at the research group.

    • Left-to-Right Dependency Parsing with Pointer Networks

In this article, we propose a novel transition-based algorithm that straightforwardly parses sentences from left to right by building n attachments, with n being the length of the input sentence. Similarly to the recent stack-pointer parser by Ma et al. (2018), we use the pointer network framework that, given a word, can directly point to a position from the sentence. However, our left-to-right approach is simpler than the original top-down stack-pointer parser (not requiring a stack) and reduces transition sequence length in half, from 2n − 1 actions to n. This results in a quadratic non-projective parser that runs twice as fast as the original while achieving the best accuracy to date on the English PTB dataset (96.04% UAS, 94.43% LAS) among fully-supervised single- model dependency parsers, and improves over the former top-down transition system in the majority of languages tested.

    • Better, Faster, Stronger Sequence Tagging Constituent Parsers

Sequence tagging models for constituent parsing are faster, but less accurate than other types of parsers. In this work, we address the following weaknesses of such constituent parsers: (a) high error rates around closing brackets of long constituents, (b) large label sets, leading to sparsity, and (c) error propagation arising from greedy decoding. To effectively close brackets, we train a model that learns to switch between tagging schemes. To reduce sparsity, we decompose the label set and use multi-task learning to jointly learn to predict sublabels. Finally, we mitigate issues from greedy decoding through auxiliary losses and sentence-level fine-tuning with policy gradient. Combining these techniques, we clearly surpass the performance of sequence tagging constituent parsers on the English and Chinese Penn Treebanks, and reduce their parsing time even further. On the SPMRL datasets (morphologically rich languages), we observe even greater improvements across the board, specially for Basque, Hebrew, Polish and Swedish.

    • Harry Potter and the Action Prediction Challenge from Natural Language

This paper explores the challenge of action prediction from textual descriptions of scenes, a testbed to approximate whether text inference can be used to predict upcoming actions. As a case of study, we consider the world of the Harry Potter fantasy novels and inferring what spell will be cast next given a fragment of a story. Spells act as keywords that abstract actions (e.g. ‘Alohomora’ to open a door) and denote a response to the environment. This idea is used to automatically build HPAC, a corpus containing 82,836 samples and 85 actions. We then evaluate different baselines.

    • Viable Dependency Parsing as Sequence Labeling

We recast dependency parsing as a sequence labeling problem, exploring several encodings of dependency trees as labels. While dependency parsing by means of sequence labeling had been attempted in existing work, results suggested that the technique was impractical. We show instead that with a conventional BILSTM-based model it is possible to obtain fast and accurate parsers. These parsers are conceptually simple, not needing traditional parsing algorithms or auxiliary structures. However, experiments on the PTB and a sample of UD treebanks show that they provide a good speed-accuracy tradeoff, with results competitive with more complex approaches.

12/04/2019

Michalina Strzyz is attending the 20th International Conference on Computational Linguistics and Intelligent Text Processing (CICLing 2019). The researcher presents there her work:

    • Speeding up Natural Language Parsing by Reusing Partial Results

This paper proposes a novel technique that applies case-based reasoning in order to generate templates for reusable parse tree fragments, based on bigrams and trigrams of PoS tags that demonstrate low variability in their syntactic analyses from prior data. The aim of this approach is to improve the speed of dependency parsers by avoiding redundant calculations.

10/09/2018

Carlos Gómez-Rodríguez, Daniel Fernández-González and David Vilares Calvo will attend the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP2018) that will take place in Brussels (Belgium) from October 31st through November 4th, 2018. They will present their recent developments at the research group:

    • Dynamic Oracles for Top-Down and In-Order Shift-Reduce Constituent Parsing

We introduce novel dynamic oracles for training two of the most accurate known shift- reduce algorithms for constituent parsing: the top-down and in-order transition-based parsers. In both cases, the dynamic oracles manage to notably increase their accuracy, in comparison to that obtained by performing classic static training. In addition, by improving the performance of the state-of-the-art in-order shift-reduce parser, we achieve the best accuracy to date (92.0 F1) obtained by a fully- supervised single-model greedy shift-reduce constituent parser on the English Penn Treebank benchmark.

    • Constituent Parsing as Sequence Labeling

We introduce a method to reduce constituent parsing to sequence labeling. For each word, it generates a label that encodes: (1) the number of ancestors in the tree that the word and its next have in common, and (2) the nonterminal symbol at the lowest common ancestor. We first prove that the proposed encoding function is injective for any tree without unary branches. In practice, the approach is made extensible to all constituency trees by collapsing unary branches. We then use the PTB and CTB treebanks as testbeds and propose a set of fast baselines. We achieve 90% F-score on the PTB test set, outperforming the Vinyals et al. (2015) sequence-to-sequence parser. In addition, sacrificing some accuracy, our approach achieves the fastest constituent parsing speeds reported to date on PTB by a wide margin.

    • Transition-based Parsing with Lighter Feed-Forward Networks

We explore whether it is possible to build lighter parsers, that are statistically equivalent to their corresponding standard version, for a wide set of languages showing different structures and morphologies. As testbed, we use the Universal Dependencies and transition-based dependency parsers trained on feed-forward networks. For these, most existing research assumes de facto standard embedded features and relies on pre-computation tricks to obtain speed-ups. We explore how these features and their size can be reduced and whether this translates into speed-ups with a negligible impact on accuracy. The experiments show that grand-daughter features can be removed for the majority of treebanks without a significant (negative or positive) LAS difference. They also show how the size of the embeddings can be notably reduced.

10/09/2018

Mark Anderson and David Vilares have been invited to give a pair of talks at Natural Language Processing Copenhagen Meetup

For further information, please clic on the talk you are interested in

15/07/2018

Carlos Gómez-Rodríguez attended the 56th Annual Meeting of the Association for Computational Linguistics (ACL 2018) that took place in Melbourne (Australia) from July 15th through July 20th, 2018. He presented the paper:

    • Global Transition-based Non-projective Dependency Parsing

This paper presents the first practical implementations of natural language parsers that can deal with crossing dependencies and support both exact inference (via dynamic programming) and greedy inference. Thanks to the flexibility and learning capacity of minimal feature models when implemented with Bi-LSTM architectures, these algorithms, previously considered of theoretical interest only due to their prohibitive computational complexity, can now be implemented in practice with realistic runtimes, providing competitive accuracy with the state of the art.

1/06/2018

Carlos Gómez-Rodríguez, Daniel Fernández-González and David Vilares Calvo will attend the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL HLT 2018) that will take place in New Orleans (USA) from June 1st through june 6th, 2018. They will present their recent developments at the research group:

    • Improving Coverage and Runtime Complexity for Exact Inference in Non-Projective Transition-Based Dependency Parsers

This paper defines a new family of parsing algorithms supporting crossing dependencies, which have the flexibility of being compatible both with dynamic programming (an exact, but slow search method) and greedy transition-based parsing (an approximate, but fast search method). While there was one existing parser with the same flexibility, the proposed algorithms improve over it both in terms of efficiency and of coverage of the syntactic phenomena involving crossing dependencies in human languages.

    • A Dynamic Oracle for Linear-Time 2-Planar Dependency Parsing

In this article, we propose an efficient dynamic oracle for training the 2-Planar dependency parser, a well-known linear-time transition-based parser with over 99% coverage on non-projective syntactic corpora. This novel approach outperforms the traditional training strategy in the vast majority of languages tested and scored better on most datasets than the widely-used arc-hybrid parser enhanced with the SWAP transition, which can handle unrestricted non-projectivity.

    • Non-Projective Dependency Parsing with Non-Local Transitions

In this article, we present a novel transition system, based on the Covington non-projective dependency parser, introducing non-local transitions that can directly create arcs involving nodes to the left of the current focus positions. This avoids the need for long sequences of NO-ARCS transitions to create long-distance arcs, thus alleviating the main weakness of this kind of parsers: error propagation. The resulting parser outperforms the original version and achieves the best accuracy on the Penn Treebank among greedy transition-based parsers.

    • A Transition-based Algorithm for Unrestricted AMR Parsing

This paper addresses the problem of mapping the meaning of English sentences into abstract structures that are able to encode named-entities, word senses or semantic relationships between pairs of words, among other relevant information. Such structures are becoming of high interest in natural language processing, as they are of high utility by computers when dealing with tasks where semantics play a role, such as Information Extraction, Question Answering or Machine Translation.

In our work, we propose an algorithm that applies a set of transitions to transform a sequence of words into a directed, cycled, labeled graph, known as an AMR graph. The novelty of the approach comes from its simplicity to manage reentrancy and cycles, in comparison to other transition-based algorithms. The experiments have shown that this simplicity also translated into better results when identifying this particular type of edges, which are expected to happen when some of the words of the sentence are playing multiple semantic roles or when there is presence of co-reference.

For further information, please visit http://naacl2018.org/program.html

16/04/2018

Carlos Gómez-Rodriguez will attend as invited speaker the workshop THE ORIGINS AND EVOLUTION OF WORD ORDER at Evolang XII in Torun (Poland). The workshop will have three invited speakers who cover diverse areas of research on word order. Carlos will cover the topic computational linguistics:

In this talk, he will outline several ways in which the approaches used in computational linguistics to build efficient parsers for human languages are related with cognitive models of human language processing and its influence on the evolution of syntax. .

For further information, please visit https://sites.google.com/view/origins-evolution-word-order/inici

3/07/2017

Carlos Gómez-Rodríguez and Daniel Fernández-González will attend the 55th annual meeting of the Association for Computational Linguistics (ACL) that will take place in Vancouver (Canada) from July 30th through August 4th, 2017. They will present their recent developments at the research group:

    • A Full Non-Monotonic Transition System for Unrestricted Non-Projective Parsing

Efficient algorithms to parse the syntax of text in human languages typically proceed by reading sentences from left to right and building a syntactic tree at the same time, as humans are believed to do. However, this approach can run into errors when decisions are made with insufficient information, as when we have read "John bought an apple" and the next word is "tree" - only upon reading this last word, we realize that John did not buy a fruit. In this paper, we have developed an algorithm that can fix these errors by modifying previous decisions, the first of its kind that can deal with the crossing dependencies that arise in many human languages.

    • Generic Axiomatization of Families of Noncrossing Graphs in Dependency Parsing

This paper presents a novel encoding that can be used to represent various kinds of graphs used to describe the syntax and semantics of sentences in human languages, in such a way that a wide range of families of such graphs are described under the well-known framework of context-free languages. The new encoding is a theoretical development that can be exploited to define generic, efficient parsing algorithms that can be easily restricted to different families of syntactic or semantic graphs, allowing both for wide-coverage parsers and for more restricted parsers that sacrifice coverage to obtain greater efficiency.

For further information, please visit http://acl2017.org/

11/05/2017

Computer Science Department at the IT University of Copenhagen has invited Dr. Carlos Gómez-Rodríguez to give a pair of talks about Natural Language Processing. He will present an overview of the recent developments at the research group in the field of opinion mining, using natural language processing and machine learning techniques. These talks will take place:

For further information, please clic on the talk you are interested in