Pretraining large neural language models, such as BERT, has led to impressive gains on
many natural language processing (NLP) tasks. However, most pretraining efforts focus on …

A large array of pretrained models are available to the biomedical NLP (BioNLP) community.
Finding the best model for a particular task can be difficult and time-consuming. For many …

This work presents the first large-scale biomedical Spanish language models trained from
scratch, using large biomedical corpora consisting of a total of 1.1 B tokens and an EHR …

Language models pretrained on text from a wide variety of sources form the foundation of
today's NLP. In light of the success of these broad-coverage models, we investigate whether …

Abstract Domain knowledge is important for building Natural Language Processing (NLP)
systems for low-resource settings, such as in the clinical domain. In this paper, a novel joint …

Pretrained language models have served as important backbones for natural language
processing. Recently, in-domain pretraining has been shown to benefit various domain …

Inspired by the success of the General Language Understanding Evaluation benchmark, we
introduce the Biomedical Language Understanding Evaluation (BLUE) benchmark to …

Pretrained language models have shown success in many natural language processing
tasks. Many works explore incorporating knowledge into language models. In the …

Pretrained language models (PLMs) have demonstrated strong performance on many
natural language processing (NLP) tasks. Despite their great success, these PLMs are …

Pre-trained language models (PLMs) have been the de facto paradigm for most natural
language processing tasks. This also benefits the biomedical domain: researchers from …