Audiovisual synchronization and fusion using canonical correlation analysis

ME Sargin, Y Yemez, E Erzin… - IEEE transactions on …, 2007 - ieeexplore.ieee.org
It is well-known that early integration (also called data fusion) is effective when the
modalities are correlated, and late integration (also called decision or opinion fusion) is …

Modality combination techniques for continuous sign language recognition

J Forster, C Oberdörfer, O Koller, H Ney - … , June 5-7, 2013. Proceedings 6, 2013 - Springer
Sign languages comprise parallel aspects and use several modalities to form a sign but so
far it is not clear how to best combine these modalities in the context of statistical sign …

Lip shape and hand position fusion for automatic vowel recognition in cued speech for french

P Heracleous, N Aboutabit… - IEEE Signal Processing …, 2009 - ieeexplore.ieee.org
Cued speech is a visual mode of communication that uses handshapes and placements in
combination with the mouth movements of speech to make the phonemes of a spoken …

Cued speech automatic recognition in normal-hearing and deaf subjects

P Heracleous, D Beautemps, N Aboutabit - Speech Communication, 2010 - Elsevier
This article discusses the automatic recognition of Cued Speech in French based on hidden
Markov models (HMMs). Cued Speech is a visual mode which, by using hand shapes in …

Prosody based audiovisual coanalysis for coverbal gesture recognition

S Kettebekov, M Yeasin… - IEEE transactions on …, 2005 - ieeexplore.ieee.org
Despite recent advances in vision-based gesture recognition, its applications remain largely
limited to artificially defined and well-articulated gesture signs used for human-computer …

Analysis and recognition of NAM speech using HMM distances and visual information

P Heracleous, VA Tran, T Nagai… - IEEE transactions on …, 2009 - ieeexplore.ieee.org
Non-audible murmur (NAM) is an unvoiced speech signal that can be received through the
body tissue with the use of special acoustic sensors (ie, NAM microphones) attached behind …

Analysis of the visual Lombard effect and automatic recognition experiments

P Heracleous, CT Ishi, M Sato, H Ishiguro… - Computer Speech & …, 2013 - Elsevier
This study focuses on automatic visual speech recognition in the presence of noise. The
authors show that, when speech is produced in noisy environments, articulatory changes …

Visual-speech to text conversion applicable to telephone communication for deaf individuals

P Heracleous, H Ishiguro… - 2011 18th International …, 2011 - ieeexplore.ieee.org
The access to communication technologies has become essential for the handicapped
people. This study introduces the initial step of an automatic translation system able to …

Product HMMs for audio-visual continuous speech recognition using facial animation parameters

PS Aleksic, AK Katsaggelos - … on Multimedia and Expo. ICME'03 …, 2003 - ieeexplore.ieee.org
The use of visual information in addition to acoustic can improve automatic speech
recognition. In this paper we compare different approaches for audio-visual information …

A pilot study on augmented speech communication based on Electro-Magnetic Articulography

P Heracleous, P Badin, G Bailly, N Hagita - Pattern Recognition Letters, 2011 - Elsevier
Speech is the most natural form of communication for human beings. However, in situations
where audio speech is not available because of disability or adverse environmental …