Knowledge of knowledge: Exploring known-unknowns uncertainty with large language models

A Amayuelas, K Wong, L Pan, W Chen… - arXiv preprint arXiv …, 2023 - arxiv.org
This paper investigates the capabilities of Large Language Models (LLMs) in the context of
understanding their knowledge and uncertainty over questions. Specifically, we focus on …

Uncertainty in natural language generation: From theory to applications

J Baan, N Daheim, E Ilia, D Ulmer, HS Li… - arXiv preprint arXiv …, 2023 - arxiv.org
Recent advances of powerful Language Models have allowed Natural Language
Generation (NLG) to emerge as an important technology that can not only perform traditional …

Internal consistency and self-feedback in large language models: A survey

X Liang, S Song, Z Zheng, H Wang, Q Yu, X Li… - arXiv preprint arXiv …, 2024 - arxiv.org
Large language models (LLMs) often exhibit deficient reasoning or generate hallucinations.
To address these, studies prefixed with" Self-" such as Self-Consistency, Self-Improve, and …

Conformal prediction for natural language processing: A survey

M Campos, A Farinhas, C Zerva… - Transactions of the …, 2024 - direct.mit.edu
The rapid proliferation of large language models and natural language processing (NLP)
applications creates a crucial need for uncertainty quantification to mitigate risks such as …

Think twice before assure: Confidence estimation for large language models through reflection on multiple answers

M Li, W Wang, F Feng, F Zhu, Q Wang… - arXiv preprint arXiv …, 2024 - arxiv.org
Confidence estimation aiming to evaluate output trustability is crucial for the application of
large language models (LLM), especially the black-box ones. Existing confidence estimation …

Benchmarking llms via uncertainty quantification

F Ye, M Yang, J Pang, L Wang, DF Wong… - arXiv preprint arXiv …, 2024 - arxiv.org
The proliferation of open-source Large Language Models (LLMs) from various institutions
has highlighted the urgent need for comprehensive evaluation methods. However, current …

“Glossy green” banks: the disconnect between environmental disclosures and lending activities

M Giannetti, M Jasova, M Loumioti… - Banks: The Disconnect …, 2023 - papers.ssrn.com
Using confidential information on banks' portfolios, inaccessible to market participants, we
show that banks that emphasize the environment in their disclosures extend a higher …

[HTML][HTML] Managing the unknown in machine learning: Definitions, related areas, recent advances, and prospects

M Barcina-Blanco, JL Lobo, P Garcia-Bringas… - Neurocomputing, 2024 - Elsevier
In the rapidly evolving domain of machine learning, the ability to adapt to unforeseen
circumstances and novel data types is of paramount importance. The deployment of Artificial …

Copy suppression: Comprehensively understanding a motif in language model attention heads

C McDougall, A Conmy, C Rushing… - Proceedings of the …, 2024 - aclanthology.org
We present the copy suppression motif: an algorithm implemented by attention heads in
large language models that reduces loss. If i) language model components in earlier layers …

Linkner: Linking local named entity recognition models to large language models using uncertainty

Z Zhang, Y Zhao, H Gao, M Hu - Proceedings of the ACM on Web …, 2024 - dl.acm.org
Named Entity Recognition (NER) serves as a fundamental task in natural language
understanding, bearing direct implications for web content analysis, search engines, and …