A survey on evaluation of large language models

Y Chang, X Wang, J Wang, Y Wu, L Yang… - ACM Transactions on …, 2024 - dl.acm.org
Large language models (LLMs) are gaining increasing popularity in both academia and
industry, owing to their unprecedented performance in various applications. As LLMs …

Challenges and applications of large language models

J Kaddour, J Harris, M Mozes, H Bradley… - arXiv preprint arXiv …, 2023 - arxiv.org
Large Language Models (LLMs) went from non-existent to ubiquitous in the machine
learning discourse within a few years. Due to the fast pace of the field, it is difficult to identify …

Chatgpt-4 outperforms experts and crowd workers in annotating political twitter messages with zero-shot learning

P Törnberg - arXiv preprint arXiv:2304.06588, 2023 - arxiv.org
This paper assesses the accuracy, reliability and bias of the Large Language Model (LLM)
ChatGPT-4 on the text analysis task of classifying the political affiliation of a Twitter poster …

GPT is an effective tool for multilingual psychological text analysis

The social and behavioral sciences have been increasingly using automated text analysis to
measure psychological constructs in text. We explore whether GPT, the large-language …

Can we trust the evaluation on ChatGPT?

R Aiyappa, J An, H Kwak, YY Ahn - arXiv preprint arXiv:2303.12767, 2023 - arxiv.org
ChatGPT, the first large language model (LLM) with mass adoption, has demonstrated
remarkable performance in numerous natural language tasks. Despite its evident …

The next generation of machine learning for tracking adaptation texts

AJ Sietsma, JD Ford, JC Minx - Nature Climate Change, 2024 - nature.com
Abstract Machine learning presents opportunities for tracking evidence on climate change
adaptation, including text-based methods from natural language processing. In theory, such …

How to use llms for text analysis

P Törnberg - arXiv preprint arXiv:2307.13106, 2023 - arxiv.org
This guide introduces Large Language Models (LLM) as a highly versatile text analysis
method within the social sciences. As LLMs are easy-to-use, cheap, fast, and applicable on …

Large language models as a substitute for human experts in annotating political text

M Heseltine, B Clemm von Hohenberg - Research & Politics, 2024 - journals.sagepub.com
Large-scale text analysis has grown rapidly as a method in political science and beyond. To
date, text-as-data methods rely on large volumes of human-annotated training examples …

Synthetic replacements for human survey data? The perils of large language models

J Bisbee, JD Clinton, C Dorff, B Kenkel… - Political Analysis, 2023 - cambridge.org
Large language models (LLMs) offer new research possibilities for social scientists, but their
potential as “synthetic data” is still largely unknown. In this paper, we investigate how …

User modeling in the era of large language models: Current research and future directions

Z Tan, M Jiang - arXiv preprint arXiv:2312.11518, 2023 - arxiv.org
User modeling (UM) aims to discover patterns or learn representations from user data about
the characteristics of a specific user, such as profile, preference, and personality. The user …