A Comprehensive Review of Data‐Driven Co‐Speech Gesture Generation

S Nyatsanga, T Kucherenko, C Ahuja… - Computer Graphics …, 2023 - Wiley Online Library
Gestures that accompany speech are an essential part of natural and efficient embodied
human communication. The automatic generation of such co‐speech gestures is a long …

Listen, denoise, action! audio-driven motion synthesis with diffusion models

S Alexanderson, R Nagy, J Beskow… - ACM Transactions on …, 2023 - dl.acm.org
Diffusion models have experienced a surge of interest as highly expressive yet efficiently
trainable probabilistic models. We show that these models are an excellent fit for …

Beat: A large-scale semantic and emotional multi-modal dataset for conversational gestures synthesis

H Liu, Z Zhu, N Iwamoto, Y Peng, Z Li, Y Zhou… - European conference on …, 2022 - Springer
Achieving realistic, vivid, and human-like synthesized conversational gestures conditioned
on multi-modal data is still an unsolved problem due to the lack of available datasets …

Qpgesture: Quantization-based and phase-guided motion matching for natural speech-driven gesture generation

S Yang, Z Wu, M Li, Z Zhang, L Hao… - Proceedings of the …, 2023 - openaccess.thecvf.com
Speech-driven gesture generation is highly challenging due to the random jitters of human
motion. In addition, there is an inherent asynchronous relationship between human speech …

The IVI Lab entry to the GENEA Challenge 2022–A Tacotron2 based method for co-speech gesture generation with locality-constraint attention mechanism

CJ Chang, S Zhang, M Kapadia - Proceedings of the 2022 International …, 2022 - dl.acm.org
This paper describes the IVI Lab entry to the GENEA Challenge 2022. We formulate the
gesture generation problem as a sequence-to-sequence conversion task with text, audio …

Diff-TTSG: Denoising probabilistic integrated speech and gesture synthesis

S Mehta, S Wang, S Alexanderson, J Beskow… - arXiv preprint arXiv …, 2023 - arxiv.org
With read-aloud speech synthesis achieving high naturalness scores, there is a growing
research interest in synthesising spontaneous speech. However, human spontaneous face …

Unifiedgesture: A unified gesture synthesis model for multiple skeletons

S Yang, Z Wang, Z Wu, M Li, Z Zhang… - Proceedings of the 31st …, 2023 - dl.acm.org
The automatic co-speech gesture generation draws much attention in computer animation.
Previous works designed network structures on individual datasets, which resulted in a lack …

Human or Robot? Investigating voice, appearance and gesture motion realism of conversational social agents

Y Ferstl, S Thomas, C Guiard, C Ennis… - Proceedings of the 21st …, 2021 - dl.acm.org
Research on creation of virtual humans enables increasing automatization of their behavior,
including synthesis of verbal and nonverbal behavior. As the achievable realism of different …

Integrated speech and gesture synthesis

S Wang, S Alexanderson, J Gustafson… - Proceedings of the …, 2021 - dl.acm.org
Text-to-speech and co-speech gesture synthesis have until now been treated as separate
areas by two different research communities, and applications merely stack the two …

Semantically related gestures move alike: Towards a distributional semantics of gesture kinematics

W Pouw, J de Wit, S Bögels, M Rasenberg… - … Conference on Human …, 2021 - Springer
Most manual communicative gestures that humans produce cannot be looked up in a
dictionary, as these manual gestures inherit their meaning in large part from the …