This tutorial surveys neural approaches to conversational AI that were developed in the last
few years. We group conversational systems into three categories:(1) question answering …

We propose BERTScore, an automatic evaluation metric for text generation. Analogously to
common metrics, BERTScore computes a similarity score for each token in the candidate …

Automated evaluation metrics as a stand-in for manual evaluation are an essential part of
the development of text-generation tasks such as text summarization. However, while the …

Automatic metrics are fundamental for the development and evaluation of machine
translation systems. Judging whether, and to what extent, automatic metrics concur with the …

This paper presents the results of the WMT20 Metrics Shared Task. Participants were asked
to score the outputs of the translation systems competing in the WMT20 News Translation …

Automatically evaluating the quality of language generation is critical. Although recent
learned metrics show high correlation with human judgement, these metrics can not explain …

This paper presents the results of the WMT19 Metrics Shared Task. Participants were asked
to score the outputs of the translations systems competing in the WMT19 News Translation …

We report results from the SR'18 Shared Task, a new multilingual surface realisation task
organised as part of the ACL'18 Workshop on Multilingual Surface Realisation. As in its …

Automatically constructed datasets for generating text from semi-structured data (tables),
such as WikiBio, often contain reference texts that diverge from the information in the …

This paper presents the results of the WMT16 Metrics Shared Task. We asked participants of
this task to score the outputs of the MT systems involved in the WMT16 Shared Translation …