Masked language modeling and the distributional hypothesis: Order word matters pre-training for little
A possible explanation for the impressive performance of masked language model (MLM)
pre-training is that such models have learned to represent the syntactic structures prevalent …
pre-training is that such models have learned to represent the syntactic structures prevalent …
Natural language processing advancements by deep learning: A survey
Natural Language Processing (NLP) helps empower intelligent machines by enhancing a
better understanding of the human language for linguistic-based human-computer …
better understanding of the human language for linguistic-based human-computer …
Semantic probabilistic layers for neuro-symbolic learning
We design a predictive layer for structured-output prediction (SOP) that can be plugged into
any neural network guaranteeing its predictions are consistent with a set of predefined …
any neural network guaranteeing its predictions are consistent with a set of predefined …
How much do language models copy from their training data? evaluating linguistic novelty in text generation using raven
Current language models can generate high-quality text. Are they simply copying text they
have seen before, or have they learned generalizable linguistic abstractions? To tease apart …
have seen before, or have they learned generalizable linguistic abstractions? To tease apart …
Automated concatenation of embeddings for structured prediction
Pretrained contextualized embeddings are powerful word representations for structured
prediction tasks. Recent work found that better word representations can be obtained by …
prediction tasks. Recent work found that better word representations can be obtained by …
Disentangling syntax and semantics in the brain with deep networks
C Caucheteux, A Gramfort… - … conference on machine …, 2021 - proceedings.mlr.press
The activations of language transformers like GPT-2 have been shown to linearly map onto
brain activity during speech comprehension. However, the nature of these activations …
brain activity during speech comprehension. However, the nature of these activations …
SynGEC: Syntax-enhanced grammatical error correction with a tailored GEC-oriented parser
This work proposes a syntax-enhanced grammatical error correction (GEC) approach
named SynGEC that effectively incorporates dependency syntactic information into the …
named SynGEC that effectively incorporates dependency syntactic information into the …
Fast and accurate neural CRF constituency parsing
Estimating probability distribution is one of the core issues in the NLP field. However, in both
deep learning (DL) and pre-DL eras, unlike the vast applications of linear-chain CRF in …
deep learning (DL) and pre-DL eras, unlike the vast applications of linear-chain CRF in …
Nested named entity recognition as latent lexicalized constituency parsing
Nested named entity recognition (NER) has been receiving increasing attention.
Recently,(Fu et al, 2021) adapt a span-based constituency parser to tackle nested NER …
Recently,(Fu et al, 2021) adapt a span-based constituency parser to tackle nested NER …
Fusing heterogeneous factors with triaffine mechanism for nested named entity recognition
Nested entities are observed in many domains due to their compositionality, which cannot
be easily recognized by the widely-used sequence labeling framework. A natural solution is …
be easily recognized by the widely-used sequence labeling framework. A natural solution is …