In humans, Attention is a core property of all perceptual and cognitive operations. Given our
limited ability to process competing sources, attention mechanisms select, modulate, and …

Pretraining deep neural network architectures with a language modeling objective has
brought large improvements for many natural language processing tasks. Exemplified by …

Contextualised word representation models have been successfully used for capturing
different word usages and they may be an attractive alternative for representing idiomaticity …

Despite their success in a variety of NLP tasks, pre-trained language models, due to their
heavy reliance on compositionality, fail in effectively capturing the meanings of multiword …

The success of language models based on the Transformer architecture appears to be
inconsistent with observed anisotropic properties of representations learned by such …

State-of-the-art NLP systems represent inputs with word embeddings, but these are brittle
when faced with Out-of-Vocabulary (OOV) words. To address this issue, we follow the …

Accurate assessment of the ability of embedding models to capture idiomaticity may require
evaluation at token rather than type level, to account for degrees of idiomaticity and possible …

The out-of-vocabulary (OOV) words are difficult to represent while critical to the performance
of embedding-based downstream models. Prior OOV word embedding learning methods …

We present Variational Bayesian Network (VBN)-a novel Bayesian entity representation
learning model that utilizes hierarchical and relational side information and is particularly …

Word embeddings are powerful dictionaries, which may easily capture language variations.
However, these dictionaries fail to give sense to rare words, which are surprisingly often …