Can Editing LLMs Inject Harm?

C Chen, B Huang, Z Li, Z Chen, S Lai, X Xu… - arXiv preprint arXiv …, 2024 - arxiv.org
Knowledge editing has been increasingly adopted to correct the false or outdated
knowledge in Large Language Models (LLMs). Meanwhile, one critical but under-explored …

Transformers to ssms: Distilling quadratic knowledge to subquadratic models

A Bick, KY Li, EP Xing, JZ Kolter, A Gu - arXiv preprint arXiv:2408.10189, 2024 - arxiv.org
Transformer architectures have become a dominant paradigm for domains like language
modeling but suffer in many inference settings due to their quadratic-time self-attention …

Can Knowledge Editing Really Correct Hallucinations?

B Huang, C Chen, X Xu, A Payani, K Shu - arXiv preprint arXiv …, 2024 - arxiv.org
Large Language Models (LLMs) suffer from hallucinations, referring to the non-factual
information in generated content, despite their superior capacities across tasks. Meanwhile …

Style-specific neurons for steering llms in text style transfer

W Lai, V Hangya, A Fraser - arXiv preprint arXiv:2410.00593, 2024 - arxiv.org
Text style transfer (TST) aims to modify the style of a text without altering its original
meaning. Large language models (LLMs) demonstrate superior performance across …

WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models

P Wang, Z Li, N Zhang, Z Xu, Y Yao, Y Jiang… - arXiv preprint arXiv …, 2024 - arxiv.org
Large language models (LLMs) need knowledge updates to meet the ever-growing world
facts and correct the hallucinated responses, facilitating the methods of lifelong model …

Monotonic representation of numeric properties in language models

B Heinzerling, K Inui - arXiv preprint arXiv:2403.10381, 2024 - arxiv.org
Language models (LMs) can express factual knowledge involving numeric properties such
as Karl Popper was born in 1902. However, how this information is encoded in the model's …

Monotonic Representation of Numeric Attributes in Language Models

B Heinzerling, K Inui - Proceedings of the 62nd Annual Meeting of …, 2024 - aclanthology.org
Abstract Language models (LMs) can express factual knowledge involving numeric
properties such as Karl Popper was born in 1902. However, how this information is encoded …

Generalisation first, memorisation second? Memorisation localisation for natural language classification tasks

V Dankers, I Titov - arXiv preprint arXiv:2408.04965, 2024 - arxiv.org
Memorisation is a natural part of learning from real-world data: neural models pick up on
atypical input-output combinations and store those training examples in their parameter …

Mitigating the Language Mismatch and Repetition Issues in LLM-based Machine Translation via Model Editing

W Wang, Z Li, D Lian, C Ma, L Song, Y Wei - arXiv preprint arXiv …, 2024 - arxiv.org
Large Language Models (LLMs) have recently revolutionized the NLP field, while they still
fall short in some specific down-stream tasks. In the work, we focus on utilizing LLMs to …

How Well Can Knowledge Edit Methods Edit Perplexing Knowledge?

H Ge, F Rudzicz, Z Zhu - arXiv preprint arXiv:2406.17253, 2024 - arxiv.org
As large language models (LLMs) are widely deployed, targeted editing of their knowledge
has become a critical challenge. Recently, advancements in model editing techniques, such …