This paper investigates how to effectively incorporate a pre-trained masked language model
(MLM), such as BERT, into an encoder-decoder (EncDec) model for grammatical error …

The success of bidirectional encoders using masked language models, such as BERT, on
numerous natural language processing tasks has prompted researchers to attempt to …

White blood cells (WBCs) are blood cells that fight infections and diseases as a part of the
immune system. They are also known as “defender cells.” But the imbalance in the number …

This paper presents a paradigm that adapts general large-scale pretrained models (PTMs)
to speech emotion recognition task. Although PTMs shed new light on artificial general …

In this paper, we present a substantial step in better understanding the SOTA sequence-to-
sequence (Seq2Seq) pretraining for neural machine translation~(NMT). We focus on …

在目前已发表的自然语言处理预训练技术综述中, 大多数文章仅介绍神经网络预训练技术或者极
简单介绍传统预训练技术, 存在人为割裂自然语言预训练发展历程. 为此, 以自然语言预训练发展 …

While large scale pre-trained language models such as BERT have achieved great success
on various natural language understanding tasks, how to efficiently and effectively …

Large-scale language models have achieved tremendous success across various natural
language processing (NLP) applications. Nevertheless, language models are vulnerable to …

Prompting has recently been shown as a promising approach for applying pre-trained
language models to perform downstream tasks. We present Multi-Stage Prompting (MSP), a …

Recent advances in natural language processing and machine learning have led to the
development of chatbot models, such as ChatGPT, that can engage in conversational …