Training language models with memory augmentation

Z Zhong, T Lei, D Chen - arXiv preprint arXiv:2205.12674, 2022 - arxiv.org
Recent work has improved language models (LMs) remarkably by equipping them with a
non-parametric memory component. However, most existing approaches only introduce …

A survey on retrieval-augmented text generation

H Li, Y Su, D Cai, Y Wang, L Liu - arXiv preprint arXiv:2202.01110, 2022 - arxiv.org
Recently, retrieval-augmented text generation attracted increasing attention of the
computational linguistics community. Compared with conventional generation models …

Contrastive learning-enhanced nearest neighbor mechanism for multi-label text classification

R Wang, X Dai - Proceedings of the 60th Annual Meeting of the …, 2022 - aclanthology.org
Abstract Multi-Label Text Classification (MLTC) is a fundamental and challenging task in
natural language processing. Previous studies mainly focus on learning text representation …

Domain adaptation and multi-domain adaptation for neural machine translation: A survey

D Saunders - Journal of Artificial Intelligence Research, 2022 - jair.org
The development of deep learning techniques has allowed Neural Machine Translation
(NMT) models to become extremely powerful, given sufficient training data and training time …

Retrieval-Augmented Generation for Natural Language Processing: A Survey

S Wu, Y Xiong, Y Cui, H Wu, C Chen, Y Yuan… - arXiv preprint arXiv …, 2024 - arxiv.org
Large language models (LLMs) have demonstrated great success in various fields,
benefiting from their huge amount of parameters that store knowledge. However, LLMs still …

Efficient cluster-based k-nearest-neighbor machine translation

D Wang, K Fan, B Chen, D Xiong - arXiv preprint arXiv:2204.06175, 2022 - arxiv.org
k-Nearest-Neighbor Machine Translation (kNN-MT) has been recently proposed as a non-
parametric solution for domain adaptation in neural machine translation (NMT). It aims to …

Improving few-shot performance of language models via nearest neighbor calibration

F Nie, M Chen, Z Zhang, X Cheng - arXiv preprint arXiv:2212.02216, 2022 - arxiv.org
Pre-trained language models (PLMs) have exhibited remarkable few-shot learning
capabilities when provided a few examples in a natural language prompt as demonstrations …

Efficient machine translation domain adaptation

PH Martins, Z Marinho, AFT Martins - arXiv preprint arXiv:2204.12608, 2022 - arxiv.org
Machine translation models struggle when translating out-of-domain text, which makes
domain adaptation a topic of critical importance. However, most domain adaptation methods …

Chunk-based nearest neighbor machine translation

PH Martins, Z Marinho, AFT Martins - arXiv preprint arXiv:2205.12230, 2022 - arxiv.org
Semi-parametric models, which augment generation with retrieval, have led to impressive
results in language modeling and machine translation, due to their ability to retrieve fine …

Towards robust k-nearest-neighbor machine translation

H Jiang, Z Lu, F Meng, C Zhou, J Zhou… - arXiv preprint arXiv …, 2022 - arxiv.org
k-Nearest-Neighbor Machine Translation (kNN-MT) becomes an important research
direction of NMT in recent years. Its main idea is to retrieve useful key-value pairs from an …