The perils of using Mechanical Turk to evaluate open-ended text generation

M Karpinska, N Akoury, M Iyyer - arXiv preprint arXiv:2109.06835, 2021 - arxiv.org
Recent text generation research has increasingly focused on open-ended domains such as
story and poetry generation. Because models built for such tasks are difficult to evaluate …

Understanding game-playing agents with natural language annotations

N Tomlin, A He, D Klein - arXiv preprint arXiv:2204.07531, 2022 - arxiv.org
We present a new dataset containing 10K human-annotated games of Go and show how
these natural language annotations can be used as a tool for model interpretability. Given a …

Improving Chess Commentaries by Combining Language Models with Symbolic Reasoning Engines

A Lee, D Wu, E Dinan, M Lewis - arXiv preprint arXiv:2212.08195, 2022 - arxiv.org
Despite many recent advancements in language modeling, state-of-the-art language models
lack grounding in the real world and struggle with tasks involving complex reasoning …

Bridging the Gap between Expert and Language Models: Concept-guided Chess Commentary Generation and Evaluation

J Kim, J Goh, I Hwang, J Cho, J Ok - arXiv preprint arXiv:2410.20811, 2024 - arxiv.org
Deep learning-based expert models have reached superhuman performance in decision-
making domains such as chess and Go. However, it is under-explored to explain or …

Extreme Multi-Domain, Multi-Task Learning With Unified Text-to-Text Transfer Transformers

A Oshingbesan, C Ekoh, G Atakpa… - arXiv preprint arXiv …, 2022 - arxiv.org
Text-to-text transformers have shown remarkable success in the task of multi-task transfer
learning, especially in natural language processing (NLP). However, while there have been …

Interpretation Using Classified Gradient-Based Saliency Maps for Two-Player Board Games

G Nakasone, T Sugawara - 2023 IEEE Conference on Games …, 2023 - ieeexplore.ieee.org
In this study, we propose a gradient-based saliency map generation method to improve the
explainability of saliency maps in two-player board games, such as chess and shogi …

Rap-Style Commentary Generation in an Audience Participation Fighting Game

T Jumneanbun, M Wira, P Paliyawan… - 2022 IEEE Games …, 2022 - ieeexplore.ieee.org
This paper presents a rap-style commentary system for enhancing audience participation
games. In this work, an audience participation fighting game played by two AI players is …

Some chess-specific improvements for perturbation-based saliency maps

J Fritz, J Fürnkranz - 2021 IEEE Conference on Games (CoG), 2021 - ieeexplore.ieee.org
State-of-the-art game playing programs do not provide an explanation for their move choice
beyond a numerical score for the best and alternative moves. However, human players want …

Conceptual Representation and Evaluation of an FPS Game Commentary Generator

DLS Mamoru, AD Panditha, W Perera… - … on Image Processing …, 2022 - ieeexplore.ieee.org
Playing video games has been popular across all the age limits of modern society. In the
beginning, it was limited among the younger community and it was just a hobby limited to …

Automated commentary generation based on FPS gameplay analysis

DLS Mamoru, AD Panditha, WJ Perera… - 2022 7th …, 2022 - ieeexplore.ieee.org
Video games evolved into competitive sports within a short span of time. Commentary plays
a key role in any sport. Commentary is useful to understand the game as well as to capture …