Vasa-1: Lifelike audio-driven talking faces generated in real time

S Xu, G Chen, YX Guo, J Yang, C Li, Z Zang… - arXiv preprint arXiv …, 2024 - arxiv.org
We introduce VASA, a framework for generating lifelike talking faces with appealing visual
affective skills (VAS) given a single static image and a speech audio clip. Our premiere …

Media2face: Co-speech facial animation generation with multi-modality guidance

Q Zhao, P Long, Q Zhang, D Qin, H Liang… - ACM SIGGRAPH 2024 …, 2024 - dl.acm.org
The synthesis of 3D facial animations from speech has garnered considerable attention.
Due to the scarcity of high-quality 4D facial data and well-annotated abundant multi-modality …

Probabilistic Speech-Driven 3D Facial Motion Synthesis: New Benchmarks Methods and Applications

KD Yang, A Ranjan, JHR Chang… - Proceedings of the …, 2024 - openaccess.thecvf.com
We consider the task of animating 3D facial geometry from speech signal. Existing works are
primarily deterministic focusing on learning a one-to-one mapping from speech signal to 3D …

DiffSpeaker: Speech-Driven 3D Facial Animation with Diffusion Transformer

Z Ma, X Zhu, G Qi, C Qian, Z Zhang, Z Lei - arXiv preprint arXiv …, 2024 - arxiv.org
Speech-driven 3D facial animation is important for many multimedia applications. Recent
work has shown promise in using either Diffusion models or Transformer architectures for …

Learn2Talk: 3D Talking Face Learns from 2D Talking Face

Y Zhuang, B Cheng, Y Cheng, Y Jin, R Liu, C Li… - arXiv preprint arXiv …, 2024 - arxiv.org
Speech-driven facial animation methods usually contain two main classes, 3D and 2D
talking face, both of which attract considerable research attention in recent years. However …

3D facial modeling, animation, and rendering for digital humans: A survey

Y Zhang, R Su, J Yu, R Li - Neurocomputing, 2024 - Elsevier
With the continuous advancement of 3D and human–computer interaction technologies,
digital human systems have been widely implemented in our daily lives, such as 3D games …

AVI-Talking: Learning Audio-Visual Instructions for Expressive 3D Talking Face Generation

Y Sun, W Chu, H Zhou, K Wang, H Koike - IEEE Access, 2024 - ieeexplore.ieee.org
While considerable progress has been made in achieving accurate lip synchronization for
3D speech-driven talking face generation, the task of incorporating expressive facial detail …

TalkingStyle: Personalized Speech-Driven 3D Facial Animation with Style Preservation

W Song, X Wang, S Zheng, S Li… - IEEE Transactions on …, 2024 - ieeexplore.ieee.org
It is a challenging task to create realistic 3D avatars that accurately replicate individuals'
speech and unique talking styles for speech-driven facial animation. Existing techniques …

P‐2.3: Development Prospects and Current Status of Deep Learning Neural Network‐based Facial Capture in the Metaverse Field

H Qin, C Wang, Z Li, L Wang - SID Symposium Digest of …, 2024 - Wiley Online Library
The intricate physiological composition of the human face facilitates the manifestation of
diverse facial expressions, serving as a conduit for the conveyance of emotions, cognitive …