How much are llms contaminated? a comprehensive survey and the llmsanitize library
With the rise of Large Language Models (LLMs) in recent years, new opportunities are
emerging, but also new challenges, and contamination is quickly becoming critical …
emerging, but also new challenges, and contamination is quickly becoming critical …
Large Language Model for Vulnerability Detection and Repair: Literature Review and Roadmap
The significant advancements in Large Language Models (LLMs) have resulted in their
widespread adoption across various tasks within Software Engineering (SE), including …
widespread adoption across various tasks within Software Engineering (SE), including …
Instructional fingerprinting of large language models
The exorbitant cost of training Large language models (LLMs) from scratch makes it
essential to fingerprint the models to protect intellectual property via ownership …
essential to fingerprint the models to protect intellectual property via ownership …
On catastrophic inheritance of large foundation models
Large foundation models (LFMs) are claiming incredible performances. Yet great concerns
have been raised about their mythic and uninterpreted potentials not only in machine …
have been raised about their mythic and uninterpreted potentials not only in machine …
If in a Crowdsourced Data Annotation Pipeline, a GPT-4
Recent studies indicated GPT-4 outperforms online crowd workers in data labeling
accuracy, notably workers from Amazon Mechanical Turk (MTurk). However, these studies …
accuracy, notably workers from Amazon Mechanical Turk (MTurk). However, these studies …
Beyond static AI evaluations: advancing human interaction evaluations for LLM harms and risks
Model evaluations are central to understanding the safety, risks, and societal impacts of AI
systems. While most real-world AI applications involve human-AI interaction, most current …
systems. While most real-world AI applications involve human-AI interaction, most current …
Language Models Trained to do Arithmetic Predict Human Risky and Intertemporal Choice
The observed similarities in the behavior of humans and Large Language Models (LLMs)
have prompted researchers to consider the potential of using LLMs as models of human …
have prompted researchers to consider the potential of using LLMs as models of human …
DARG: Dynamic Evaluation of Large Language Models via Adaptive Reasoning Graph
The current paradigm of evaluating Large Language Models (LLMs) through static
benchmarks comes with significant limitations, such as vulnerability to data contamination …
benchmarks comes with significant limitations, such as vulnerability to data contamination …
LMD3: Language Model Data Density Dependence
We develop a methodology for analyzing language model task performance at the individual
example level based on training data density estimation. Experiments with paraphrasing as …
example level based on training data density estimation. Experiments with paraphrasing as …
Questionable practices in machine learning
G Leech, JJ Vazquez, M Yagudin, N Kupper… - arXiv preprint arXiv …, 2024 - arxiv.org
Evaluating modern ML models is hard. The strong incentive for researchers and companies
to report a state-of-the-art result on some metric often leads to questionable research …
to report a state-of-the-art result on some metric often leads to questionable research …