Although supervised deep learning has revolutionized speech and audio processing, it has
necessitated the building of specialist models for individual tasks and application scenarios …

Video recordings of speech contain correlated audio and visual information, providing a
strong signal for speech representation learning from the speaker's lip movements and the …

Although speech is a simple and effective way for humans to communicate with the outside
world, a more realistic speech interaction contains multimodal information, eg, vision, text …

As humans, we experience the world with all our senses or modalities (sound, sight, touch,
smell, and taste). We use these modalities, particularly sight and touch, to convey and …

Despite improvements to the generalization performance of automated speech recognition
(ASR) models, specializing ASR models for downstream tasks remains a challenging task …

Audio-Visual Speech Recognition (AVSR) is a promising approach to improving the
accuracy and robustness of speech recognition systems with the assistance of visual cues in …

Deep neural networks have largely demonstrated their ability to perform automated speech
recognition (ASR) by extracting meaningful features from input audio frames. Such features …

AV-HuBERT, a multi-modal self-supervised learning model, has been shown to be effective
for categorical problems such as automatic speech recognition and lip-reading. This …

Audio-visual speech recognition (AVSR) provides a promising solution to ameliorate the
noise-robustness of audio-only speech recognition with visual information. However, most …

In the realm of automated speech recognition (ASR), the robustness of systems operating
within noisy environments remains a pivotal challenge. This paper introduces an innovative …