[HTML][HTML] Pre-trained language models and their applications

H Wang, J Li, H Wu, E Hovy, Y Sun - Engineering, 2023 - Elsevier
Pre-trained language models have achieved striking success in natural language
processing (NLP), leading to a paradigm shift from supervised learning to pre-training …

Pre-trained language models for text generation: A survey

J Li, T Tang, WX Zhao, JY Nie, JR Wen - ACM Computing Surveys, 2024 - dl.acm.org
Text Generation aims to produce plausible and readable text in human language from input
data. The resurgence of deep learning has greatly advanced this field, in particular, with the …

[HTML][HTML] Pre-trained models: Past, present and future

X Han, Z Zhang, N Ding, Y Gu, X Liu, Y Huo, J Qiu… - AI Open, 2021 - Elsevier
Large-scale pre-trained models (PTMs) such as BERT and GPT have recently achieved
great success and become a milestone in the field of artificial intelligence (AI). Owing to …

Dense text retrieval based on pretrained language models: A survey

WX Zhao, J Liu, R Ren, JR Wen - ACM Transactions on Information …, 2024 - dl.acm.org
Text retrieval is a long-standing research topic on information seeking, where a system is
required to return relevant information resources to user's queries in natural language. From …

Curriculum learning: A survey

P Soviany, RT Ionescu, P Rota, N Sebe - International Journal of …, 2022 - Springer
Training machine learning models in a meaningful order, from the easy samples to the hard
ones, using curriculum learning can provide performance improvements over the standard …

Lift yourself up: Retrieval-augmented text generation with self-memory

X Cheng, D Luo, X Chen, L Liu… - Advances in Neural …, 2024 - proceedings.neurips.cc
With direct access to human-written reference as memory, retrieval-augmented generation
has achieved much progress in a wide range of text generation tasks. Since better memory …

InstructDial: Improving zero and few-shot generalization in dialogue through instruction tuning

P Gupta, C Jiao, YT Yeh, S Mehri, M Eskenazi… - arXiv preprint arXiv …, 2022 - arxiv.org
Instruction tuning is an emergent paradigm in NLP wherein natural language instructions
are leveraged with language models to induce zero-shot performance on unseen tasks …

Long time no see! open-domain conversation with long-term persona memory

X Xu, Z Gou, W Wu, ZY Niu, H Wu, H Wang… - arXiv preprint arXiv …, 2022 - arxiv.org
Most of the open-domain dialogue models tend to perform poorly in the setting of long-term
human-bot conversations. The possible reason is that they lack the capability of …

Revolutionizing generative pre-traineds: Insights and challenges in deploying ChatGPT and generative chatbots for FAQs

F Khennouche, Y Elmir, Y Himeur, N Djebari… - Expert Systems with …, 2024 - Elsevier
In the rapidly evolving domain of artificial intelligence, chatbots have emerged as a potent
tool for various applications ranging from e-commerce to healthcare. This research delves …

[HTML][HTML] A Structural equation modeling analysis of generative AI chatbots adoption among students and educators in higher education

A Saihi, M Ben-Daya, M Hariga, R As' ad - Computers and Education …, 2024 - Elsevier
In an era where artificial intelligence (AI) is reshaping educational paradigms, this study
explores AI-based chatbot adoption in higher education among students and educators …