Repairing the cracked foundation: A survey of obstacles in evaluation practices for generated text

S Gehrmann, E Clark, T Sellam - Journal of Artificial Intelligence Research, 2023 - jair.org
Abstract Evaluation practices in natural language generation (NLG) have many known flaws,
but improved evaluation approaches are rarely widely adopted. This issue has become …

A survey of evaluation metrics used for NLG systems

AB Sai, AK Mohankumar, MM Khapra - ACM Computing Surveys (CSUR …, 2022 - dl.acm.org
In the last few years, a large number of automatic evaluation metrics have been proposed for
evaluating Natural Language Generation (NLG) systems. The rapid development and …

Natural language processing: state of the art, current trends and challenges

D Khurana, A Koli, K Khatter, S Singh - Multimedia tools and applications, 2023 - Springer
Natural language processing (NLP) has recently gained much attention for representing and
analyzing human language computationally. It has spread its applications in various fields …

Handling divergent reference texts when evaluating table-to-text generation

B Dhingra, M Faruqui, A Parikh, MW Chang… - arXiv preprint arXiv …, 2019 - arxiv.org
Automatically constructed datasets for generating text from semi-structured data (tables),
such as WikiBio, often contain reference texts that diverge from the information in the …

[PDF][PDF] Minimum error rate training in statistical machine translation

FJ Och - Proceedings of the 41st annual meeting of the …, 2003 - aclanthology.org
Often, the training procedure for statistical machine translation models is based on maximum
likelihood or related criteria. A general problem of this approach is that there is only a loose …

Automatic summarization

I Mani - 2001 - torrossa.com
When I planned this book in 1999, it was intended to be a survey of current work in text
summarization. The survey would discuss various efforts in the context of a framework for …

Computational generation of referring expressions: A survey

E Krahmer, K Van Deemter - Computational Linguistics, 2012 - direct.mit.edu
This article offers a survey of computational research on referring expression generation
(REG). It introduces the REG problem and describes early work in this area, discussing what …

Global inference for sentence compression: An integer linear programming approach

J Clarke, M Lapata - Journal of Artificial Intelligence Research, 2008 - jair.org
Sentence compression holds promise for many applications ranging from summarization to
subtitle generation. Our work views sentence compression as an optimization problem and …

[PDF][PDF] Journalist versus news consumer: The perceived credibility of machine written news

HAJ Van der Kaa, EJ Krahmer - Computation+ …, 2014 - research.tilburguniversity.edu
This research aims to contribute to the unexplored field of audience studies with a focus on
the credibility of automated journalism. In this paper, we take a systematic look into the …

[PDF][PDF] Comparing automatic and human evaluation of NLG systems

A Belz, E Reiter - 11th conference of the european chapter of the …, 2006 - aclanthology.org
We consider the evaluation problem in Natural Language Generation (NLG) and present
results for evaluating several NLG systems with similar functionality, including a knowledge …