Evaluating psychological safety of large language models

X Li, Y Li, L Qiu, S Joty, L Bing - Proceedings of the 2024 …, 2024 - aclanthology.org
In this work, we designed unbiased prompts to systematically evaluate the psychological
safety of large language models (LLMs). First, we tested five different LLMs by using two …

The better angels of machine personality: How personality relates to llm safety

J Zhang, D Liu, C Qian, Z Gan, Y Liu, Y Qiao… - arXiv preprint arXiv …, 2024 - arxiv.org
Personality psychologists have analyzed the relationship between personality and safety
behaviors in human society. Although Large Language Models (LLMs) demonstrate …

Shall we team up: Exploring spontaneous cooperation of competing llm agents

Z Wu, R Peng, S Zheng, Q Liu, X Han… - Findings of the …, 2024 - aclanthology.org
Abstract Large Language Models (LLMs) have increasingly been utilized in social
simulations, where they are often guided by carefully crafted instructions to stably exhibit …

Limited Ability of LLMs to Simulate Human Psychological Behaviours: a Psychometric Analysis

NB Petrov, G Serapio-García, J Rentfrow - arXiv preprint arXiv:2405.07248, 2024 - arxiv.org
The humanlike responses of large language models (LLMs) have prompted social scientists
to investigate whether LLMs can be used to simulate human participants in experiments …

Benchmarking Distributional Alignment of Large Language Models

N Meister, C Guestrin, T Hashimoto - arXiv preprint arXiv:2411.05403, 2024 - arxiv.org
Language models (LMs) are increasingly used as simulacra for people, yet their ability to
match the distribution of views of a specific demographic group and be\textit {distributionally …