Human evaluation of modern high-quality machine translation systems is a difficult problem,
and there is increasing evidence that inadequate evaluation procedures can lead to …

Many research topics in natural language processing (NLP), such as explanation
generation, dialog modeling, or machine translation, require evaluation that goes beyond …

This work extends and evaluates a two-dimensional automatic evaluation metric for machine
translation, which is designed to operate at the sentence level. The metric is based on the …

This work proposes a new method for manual evaluation of Machine Translation (MT) output
based on marking actual issues in the translated text. The novelty is that the evaluators are …

Automatic machine translation (MT) metrics are widely used to distinguish the translation
qualities of machine translation systems across relatively large test sets (system-level …

Evaluating the semantic similarity of two sentences is a task central to automated
understanding of natural languages. We discuss the problem of semantic similarity and …

This work describes an analysis of inter-annotator disagreements in human evaluation of
machine translation output. The errors in the analysed texts were marked by multiple …

Generative large language models (LLMs) have demonstrated exceptional proficiency in
various natural language processing (NLP) tasks, including machine translation, question …

Automatic evaluation of machine translation (MT) systems is an important research topic for
the advancement of MT technology. Most automatic evaluation methods proposed to date …

Understanding speaker's feelings and producing appropriate responses with emotion
connection is a key communicative skill for empathetic dialogue systems. In this paper, we …