How to evaluate machine translation: A review of automated and human metrics

E Chatzikoumi - Natural Language Engineering, 2020 - cambridge.org
This article presents the most up-to-date, influential automated, semiautomated and human
metrics used to evaluate the quality of machine translation (MT) output and provides the …

A comprehensive survey on various fully automatic machine translation evaluation metrics

S Chauhan, P Daniel - Neural Processing Letters, 2023 - Springer
The fast advancement in machine translation models necessitates the development of
accurate evaluation metrics that would allow researchers to track the progress in text …

How to do human evaluation: A brief introduction to user studies in NLP

H Schuff, L Vanderlyn, H Adel, NT Vu - Natural Language …, 2023 - cambridge.org
Many research topics in natural language processing (NLP), such as explanation
generation, dialog modeling, or machine translation, require evaluation that goes beyond …

[HTML][HTML] Who evaluates the evaluators? On automatic metrics for assessing AI-based offensive code generators

P Liguori, C Improta, R Natella, B Cukic… - Expert Systems with …, 2023 - Elsevier
AI-based code generators are an emerging solution for automatically writing programs
starting from descriptions in natural language, by using deep neural networks (Neural …

Angler: Helping machine translation practitioners prioritize model improvements

S Robertson, ZJ Wang, D Moritz, MB Kery… - Proceedings of the 2023 …, 2023 - dl.acm.org
Machine learning (ML) models can fail in unexpected ways in the real world, but not all
model failures are equal. With finite time and resources, ML practitioners are forced to …

Arabic machine translation: A survey with challenges and future directions

J Zakraoui, M Saleh, S Al-Maadeed, JM Alja'am - IEEE Access, 2021 - ieeexplore.ieee.org
In recent years, computer language area has witnessed important evolvement with
applications in different domains. Machine Translation MT technology, considered as a …

Assessing the Role of Context in Chat Translation Evaluation: Is Context Helpful and Under What Conditions?

S Agrawal, A Farajian, P Fernandes, R Rei… - Transactions of the …, 2024 - direct.mit.edu
Despite the recent success of automatic metrics for assessing translation quality, their
application in evaluating the quality of machine-translated chats has been limited. Unlike …

Exploring the effectiveness of ChatGPT-based feedback compared with teacher feedback and self-feedback: Evidence from Chinese to English translation

S Cao, L Zhong - arXiv preprint arXiv:2309.01645, 2023 - arxiv.org
ChatGPT, a cutting-edge AI-powered Chatbot, can quickly generate responses on given
commands. While it was reported that ChatGPT had the capacity to deliver useful feedback …

Towards making the most of llm for translation quality estimation

H Huang, S Wu, X Liang, B Wang, Y Shi, P Wu… - … Conference on Natural …, 2023 - Springer
Abstract Machine Translation Quality Estimation (QE) aims to evaluate the quality of
machine translation without relying on references. Recently, Large-scale Language Model …

Large language models and control mechanisms improve text readability of biomedical abstracts

Z Li, S Belkadi, N Micheletti, L Han, M Shardlow… - arXiv preprint arXiv …, 2023 - arxiv.org
Biomedical literature often uses complex language and inaccessible professional
terminologies. That is why simplification plays an important role in improving public health …