News summarization and evaluation in the era of gpt-3
The recent success of zero-and few-shot prompting with models like GPT-3 has led to a
paradigm shift in NLP research. In this paper, we study its impact on text summarization …
paradigm shift in NLP research. In this paper, we study its impact on text summarization …
Summeval: Re-evaluating summarization evaluation
The scarcity of comprehensive up-to-date studies on evaluation metrics for text
summarization and the lack of consensus regarding evaluation protocols continue to inhibit …
summarization and the lack of consensus regarding evaluation protocols continue to inhibit …
On faithfulness and factuality in abstractive summarization
It is well known that the standard likelihood training and approximate decoding objectives in
neural text generation models lead to less human-like responses for open-ended tasks such …
neural text generation models lead to less human-like responses for open-ended tasks such …
Revisiting the gold standard: Grounding summarization evaluation with robust human evaluation
Human evaluation is the foundation upon which the evaluation of both summarization
systems and automatic metrics rests. However, existing human evaluation studies for …
systems and automatic metrics rests. However, existing human evaluation studies for …
LongEval: Guidelines for human evaluation of faithfulness in long-form summarization
While human evaluation remains best practice for accurately judging the faithfulness of
automatically-generated summaries, few solutions exist to address the increased difficulty …
automatically-generated summaries, few solutions exist to address the increased difficulty …
Summary explorer: Visualizing the state of the art in text summarization
This paper introduces Summary Explorer, a new tool to support the manual inspection of text
summarization systems by compiling the outputs of 55~ state-of-the-art single document …
summarization systems by compiling the outputs of 55~ state-of-the-art single document …
Generating representative headlines for news stories
Millions of news articles are published online every day, which can be overwhelming for
readers to follow. Grouping articles that are reporting the same event into news stories is a …
readers to follow. Grouping articles that are reporting the same event into news stories is a …
An investigation of evaluation methods in automatic medical note generation
Recent studies on automatic note generation have shown that doctors can save significant
amounts of time when using automatic clinical note generation (Knoll et al., 2022) …
amounts of time when using automatic clinical note generation (Knoll et al., 2022) …
Ffci: A framework for interpretable automatic evaluation of summarization
In this paper, we propose FFCI, a framework for fine-grained summarization evaluation that
comprises four elements: faithfulness (degree of factual consistency with the source), focus …
comprises four elements: faithfulness (degree of factual consistency with the source), focus …
An investigation of evaluation metrics for automated medical note generation
Recent studies on automatic note generation have shown that doctors can save significant
amounts of time when using automatic clinical note generation (Knoll et al., 2022) …
amounts of time when using automatic clinical note generation (Knoll et al., 2022) …