Studying large language model generalization with influence functions

R Grosse, J Bae, C Anil, N Elhage, A Tamkin… - arXiv preprint arXiv …, 2023 - arxiv.org
When trying to gain better visibility into a machine learning model in order to understand and
mitigate the associated risks, a potentially valuable source of evidence is: which training …

If influence functions are the answer, then what is the question?

J Bae, N Ng, A Lo, M Ghassemi… - Advances in Neural …, 2022 - proceedings.neurips.cc
Influence functions efficiently estimate the effect of removing a single training data point on a
model's learned parameters. While influence estimates align well with leave-one-out …

Trak: Attributing model behavior at scale

SM Park, K Georgiev, A Ilyas, G Leclerc… - arXiv preprint arXiv …, 2023 - arxiv.org
The goal of data attribution is to trace model predictions back to training data. Despite a long
line of work towards this goal, existing approaches to data attribution tend to force users to …

Training data influence analysis and estimation: A survey

Z Hammoudeh, D Lowd - Machine Learning, 2024 - Springer
Good models require good training data. For overparameterized deep models, the causal
relationship between training data and model predictions is increasingly opaque and poorly …

Softpatch: Unsupervised anomaly detection with noisy data

X Jiang, J Liu, J Wang, Q Nie, K Wu… - Advances in …, 2022 - proceedings.neurips.cc
Although mainstream unsupervised anomaly detection (AD) algorithms perform well in
academic datasets, their performance is limited in practical application due to the ideal …

Datainf: Efficiently estimating data influence in lora-tuned llms and diffusion models

Y Kwon, E Wu, K Wu, J Zou - arXiv preprint arXiv:2310.00902, 2023 - arxiv.org
Quantifying the impact of training data points is crucial for understanding the outputs of
machine learning models and for improving the transparency of the AI pipeline. The …

Deta: Denoised task adaptation for few-shot learning

J Zhang, L Gao, X Luo, H Shen… - Proceedings of the …, 2023 - openaccess.thecvf.com
Test-time task adaptation in few-shot learning aims to adapt a pre-trained task-agnostic
model for capturing task-specific knowledge of the test task, rely only on few-labeled support …

{Meta-Sift}: How to Sift Out a Clean Subset in the Presence of Data Poisoning?

Y Zeng, M Pan, H Jahagirdar, M Jin, L Lyu… - 32nd USENIX Security …, 2023 - usenix.org
External data sources are increasingly being used to train machine learning (ML) models as
the data demand increases. However, the integration of external data into training poses …

Ease: Robust facial expression recognition via emotion ambiguity-sensitive cooperative networks

L Wang, G Jia, N Jiang, H Wu, J Yang - Proceedings of the 30th ACM …, 2022 - dl.acm.org
Facial Expression Recognition (FER) plays a crucial role in the real-world applications.
However, large-scale FER datasets collected in the wild usually contain noises. More …

Instance-dependent noisy label learning via graphical modelling

A Garg, C Nguyen, R Felix, TT Do… - Proceedings of the …, 2023 - openaccess.thecvf.com
Noisy labels are unavoidable yet troublesome in the ecosystem of deep learning because
models can easily overfit them. There are many types of label noise, such as symmetric …