The diffusestylegesture+ entry to the genea challenge 2023

S Yang, H Xue, Z Zhang, M Li, Z Wu, X Wu… - Proceedings of the 25th …, 2023 - dl.acm.org
In this paper, we introduce the DiffuseStyleGesture+, our solution for the Generation and
Evaluation of Non-verbal Behavior for Embodied Agents (GENEA) Challenge 2023, which …

Diffusion-based co-speech gesture generation using joint text and audio representation

A Deichler, S Mehta, S Alexanderson… - Proceedings of the 25th …, 2023 - dl.acm.org
This paper describes a system developed for the GENEA (Generation and Evaluation of Non-
verbal Behaviour for Embodied Agents) Challenge 2023. Our solution builds on an existing …

Freetalker: Controllable speech and text-driven gesture generation based on diffusion models for enhanced speaker naturalness

S Yang, Z Xu, H Xue, Y Cheng, S Huang… - ICASSP 2024-2024 …, 2024 - ieeexplore.ieee.org
Current talking avatars mostly generate co-speech gestures based on audio and text of the
utterance, without considering the non-speaking motion of the speaker. Furthermore …

Evaluating gesture generation in a large-scale open challenge: The GENEA Challenge 2022

T Kucherenko*, P Wolfert*, Y Yoon*, C Viegas… - ACM Transactions on …, 2024 - dl.acm.org
This article reports on the second GENEA Challenge to benchmark data-driven automatic co-
speech gesture generation. Participating teams used the same speech and motion dataset …

Co-Speech Gesture Video Generation via Motion-Decoupled Diffusion Model

X He, Q Huang, Z Zhang, Z Lin, Z Wu… - Proceedings of the …, 2024 - openaccess.thecvf.com
Co-speech gestures if presented in the lively form of videos can achieve superior visual
effects in human-machine interaction. While previous works mostly generate structural …

MotionScript: Natural Language Descriptions for Expressive 3D Human Motions

PJ Yazdian, E Liu, L Cheng, A Lim - arXiv preprint arXiv:2312.12634, 2023 - arxiv.org
This paper proposes MotionScript, a motion-to-text conversion algorithm and natural
language representation for human body motions. MotionScript aims to describe movements …

Diffugesture: Generating human gesture from two-person dialogue with diffusion models

W Zhao, L Hu, S Zhang - Companion Publication of the 25th International …, 2023 - dl.acm.org
This paper describes the DiffuGesture entry to the GENEA Challenge 2023. In this paper, we
utilize conditional diffusion models to formulate the gesture generation problem. The …

Unified speech and gesture synthesis using flow matching

S Mehta, R Tu, S Alexanderson… - ICASSP 2024-2024 …, 2024 - ieeexplore.ieee.org
As text-to-speech technologies achieve remarkable naturalness in read-aloud tasks, there is
growing interest in multimodal synthesis of verbal and non-verbal communicative behaviour …

Fake it to make it: Using synthetic data to remedy the data shortage in joint multimodal speech-and-gesture synthesis

S Mehta, A Deichler, J O'regan… - Proceedings of the …, 2024 - openaccess.thecvf.com
Although humans engaged in face-to-face conversation simultaneously communicate both
verbally and non-verbally methods for joint and unified synthesis of speech audio and co …

Past, Present, and Future: A Survey of The Evolution of Affective Robotics For Well-being

M Spitale, M Axelsson, S Jeong, P Tuttosı… - arXiv preprint arXiv …, 2024 - arxiv.org
Recent research in affective robots has recognized their potential in supporting human well-
being. Due to rapidly developing affective and artificial intelligence technologies, this field of …