Falsesum: Generating document-level NLI examples for recognizing factual inconsistency in summarization

PA Utama, J Bambrick, NS Moosavi… - arXiv preprint arXiv …, 2022 - arxiv.org
examples of generated NLI instances in Table 6. We also include cases where Falsesum
inadvertently generates factually … several examples of the formatted input and the generated

NonFactS: NonFactual summary generation for factuality evaluation in document summarization

A Soleimani, C Monz, M Worring - Findings of the Association for …, 2023 - aclanthology.org
… models often generate summaries that are inconsistent with … tion, a document-level Natural
Language Inference (NLI) … two standard benchmarks, FALSESUM and SUMMAC. Compared …

Fast and accurate factual inconsistency detection over long documents

BM Lattimer, P Chen, X Zhang, Y Yang - arXiv preprint arXiv:2310.13189, 2023 - arxiv.org
… of factual inconsistencies in generated sentences by identifying … For example, given an NLI
model M(p, h), source document … -edge NLI based factual inconsistency detection method …

On the Intractability to Synthesize Factual Inconsistencies in Summarization

G Luo, W Fan, M Li, Y He, Y Yang… - Findings of the …, 2024 - aclanthology.org
… • Finally, we present a document-level factuality classifier … train a factual consistency
classifier using an NLI model as … Realizing the diversity of inconsistencies and the challenges …

Fine-grained natural language inference based faithfulness evaluation for diverse summarisation tasks

H Zhang, Y Xu, L Perez-Beltrachini - arXiv preprint arXiv:2402.17630, 2024 - arxiv.org
… sentence is inconsistent with respect to the document. Entity Error (… The success of the
documentlevel approach lies on the fact that … We gratefully acknowledge the support of the UK …

Localizing Factual Inconsistencies in Attributable Text Generation

A Cattan, P Roit, S Zhang, D Wan, R Aharoni… - arXiv preprint arXiv …, 2024 - arxiv.org
… the task of localizing factual inconsistencies as identifying the set of “… Supervised We apply
two recent off-the-shelf NLI models to … For example, the sentences “John missed the train and …

Identifying Factual Inconsistencies in Summaries: Grounding LLM Inference via Task Taxonomy

L Xu, Z Su, M Yu, J Xu, JD Choi, J Zhou… - Findings of the …, 2024 - aclanthology.org
… derive stronger Natural Language Inference (NLI) models, we … of generating a binary decision
directly. A summary is thus … 50k examples from DocNLI and FalseSum respectively that do …

Falsesum: Generating Document-level NLI Examples for Recognizing Factual Inconsistency in Summarization

P Ajie Utama, J Bambrick, N Sadat Moosavi… - arXiv e …, 2022 - ui.adsabs.harvard.edu
examples that are diverse and inconsistent yet plausible. We show that models trained on a
Falsesum-augmented NLI … benchmarks for detecting factual inconsistency in summarization. …

A Survey of Factual Consistency in Summarization from 2021 to 2023

Y Li, X Hao, L Li - 2023 4th International Conference on …, 2023 - ieeexplore.ieee.org
… are many factual inconsistencies in the model-generated … levels of granularity, such as
document-level and sentence-level. [… Constructing training dataset for classification models in NLI-…

Amrfact: Enhancing summarization factuality evaluation with amr-driven training data generation

H Qiu, KH Huang, J Qu, N Peng - arXiv preprint arXiv:2311.09521, 2023 - arxiv.org
… framework that generates factually inconsistent summaries … for identifying the factual
inconsistencies within generated … to the data generated by FACTCC and FALSESUM and re-train …