Pre-trained language models for text generation: A survey

J Li, T Tang, WX Zhao, JY Nie, JR Wen - ACM Computing Surveys, 2024 - dl.acm.org
Text Generation aims to produce plausible and readable text in human language from input
data. The resurgence of deep learning has greatly advanced this field, in particular, with the …

A survey of large language models

WX Zhao, K Zhou, J Li, T Tang, X Wang, Y Hou… - arXiv preprint arXiv …, 2023 - arxiv.org
Language is essentially a complex, intricate system of human expressions governed by
grammatical rules. It poses a significant challenge to develop capable AI algorithms for …

Unlocking efficiency in large language model inference: A comprehensive survey of speculative decoding

H Xia, Z Yang, Q Dong, P Wang, Y Li, T Ge… - arXiv preprint arXiv …, 2024 - arxiv.org
To mitigate the high inference latency stemming from autoregressive decoding in Large
Language Models (LLMs), Speculative Decoding has emerged as a novel decoding …

Text2motion: From natural language instructions to feasible plans

K Lin, C Agia, T Migimatsu, M Pavone, J Bohg - Autonomous Robots, 2023 - Springer
Abstract We propose Text2Motion, a language-based planning framework enabling robots
to solve sequential manipulation tasks that require long-horizon reasoning. Given a natural …

Grounding language models to images for multimodal inputs and outputs

JY Koh, R Salakhutdinov… - … Conference on Machine …, 2023 - proceedings.mlr.press
We propose an efficient method to ground pretrained text-only language models to the
visual domain, enabling them to process arbitrarily interleaved image-and-text data, and …

Mitigating object hallucinations in large vision-language models through visual contrastive decoding

S Leng, H Zhang, G Chen, X Li, S Lu… - Proceedings of the …, 2024 - openaccess.thecvf.com
Abstract Large Vision-Language Models (LVLMs) have advanced considerably intertwining
visual recognition and language understanding to generate content that is not only coherent …

Demonstrate-search-predict: Composing retrieval and language models for knowledge-intensive nlp

O Khattab, K Santhanam, XL Li, D Hall, P Liang… - arXiv preprint arXiv …, 2022 - arxiv.org
Retrieval-augmented in-context learning has emerged as a powerful approach for
addressing knowledge-intensive tasks using frozen language models (LM) and retrieval …

Survey on factuality in large language models: Knowledge, retrieval and domain-specificity

C Wang, X Liu, Y Yue, X Tang, T Zhang… - arXiv preprint arXiv …, 2023 - arxiv.org
This survey addresses the crucial issue of factuality in Large Language Models (LLMs). As
LLMs find applications across diverse domains, the reliability and accuracy of their outputs …

Trusting your evidence: Hallucinate less with context-aware decoding

W Shi, X Han, M Lewis, Y Tsvetkov… - arXiv preprint arXiv …, 2023 - arxiv.org
Language models (LMs) often struggle to pay enough attention to the input context, and
generate texts that are unfaithful or contain hallucinations. To mitigate this issue, we present …

[PDF][PDF] Scaling autoregressive multi-modal models: Pretraining and instruction tuning

L Yu, B Shi, R Pasunuru, B Muller… - arXiv preprint arXiv …, 2023 - aicommenter.com
We present CM3Leon (pronounced “Chameleon”), a retrieval-augmented, tokenbased,
decoder-only multi-modal language model capable of generating and infilling both text and …