查看文章

Abstract Building a robust Automatic Speech Recognition (ASR) system and improving recognition accuracy in adverse conditions is still a challenging task. One way to improve the robustness of an ASR system is combining information from multiple sources (streams). A multi-stream approach which handles the multiple inputs at the model level is the key contribution of our work. Standard mic (S m), Throat mic (T m), and Lip reading (L r) are the various source streams that have been used. This work explores a static weighted two stream HMM (TSH) and multi-stream HMM (MSH) model for the bimodal and multimodal systems. A syllabic units of the Hindi language database categorized into three categories–Vowel, Place of Articulation (POA), and Manner of Articulation (MOA) are used for training and testing. In this study, four types of TSH are proposed for the combination of bimodal ((S m+ T m),(T m+ L r),(S m+ L r …