We study how in-context learning (ICL) in language models is affected by semantic priors
versus input-label mappings. We investigate two setups-ICL with flipped labels and ICL with …

In-context learning has been recognized as a key factor in the success of Large Language
Models (LLMs). It refers to the model's ability to learn patterns on the fly from provided in …

Confidence estimation aiming to evaluate output trustability is crucial for the application of
large language models (LLM), especially the black-box ones. Existing confidence estimation …

The effectiveness of large language models (LLMs) is not only measured by their ability to
generate accurate outputs but also by their calibration-how well their confidence scores …

Abstract Self-detection for Large Language Models (LLMs) seeks to evaluate the
trustworthiness of the LLM's output by leveraging its own capabilities, thereby alleviating the …

Current question-answering benchmarks predominantly focus on accuracy in realizable
prediction tasks. Conditioned on a question and answer-key, does the most likely token …

Large language models (LLMs) are highly capable of many tasks but they can sometimes
generate unreliable or inaccurate outputs. To tackle this issue, this paper studies the …

The development of Large Language Models (LLMs) has notably transformed numerous
sectors, offering impressive text generation capabilities. Yet, the reliability and truthfulness of …

While in-context learning with large language models (LLMs) has shown impressive
performance, we have discovered a unique miscalibration behavior where both correct and …

Large language models are increasingly deployed for high-stakes decision making, for
example in financial and medical applications. In such applications, it is imperative that we …