Bidirectional Encoder Representations from Transformers (BERT) has gained increasing
attention from researchers and practitioners as it has proven to be an invaluable technique …

Multilingual pre-trained language models (PLMs) have demonstrated impressive
performance on several downstream tasks for both high-resourced and low-resourced …

Guided by a corpus linguistics approach, in this article we present a comparative evaluation
of State-of-the-Art (SotA) models, with a special focus on Transformers, to address the task of …

Pre-trained transformers are now the de facto models in Natural Language Processing given
their state-of-the-art results in many tasks and languages. However, most of the current …

This paper introduces a Scandinavian benchmarking platform, ScandEval, which can
benchmark any pretrained model on four different tasks in the Scandinavian languages. The …

Prior work shows that it is possible to expand pretrained Masked Language Models (MLMs)
to new languages by learning a new set of embeddings, while keeping the transformer body …

Large scale adoption of large language models has introduced a new era of convenient
knowledge transfer for a slew of natural language processing tasks. However, these models …

In today's digital world, automated sentiment analysis from online reviews can contribute to a
wide variety of decision-making processes. One example is examining typical perceptions of …

Learning sparse representations using pretrained language models enhances the
monolingual ranking effectiveness. Such representations are sparse vectors in the …

For many tasks, state-of-the-art results have been achieved with Transformer-based
architectures, resulting in a paradigmatic shift in practices from the use of task-specific …