Multimodal speech capture system for speech rehabilitation and learning

N Sebkhi, D Desai, M Islam, J Lu… - IEEE Transactions …, 2017 - ieeexplore.ieee.org
N Sebkhi, D Desai, M Islam, J Lu, K Wilson, M Ghovanloo
IEEE Transactions on Biomedical Engineering, 2017ieeexplore.ieee.org
Speech-language pathologists (SLPs) are trained to correct articulation of people diagnosed
with motor speech disorders by analyzing articulators' motion and assessing speech
outcome while patients speak. To assist SLPs in this task, we are presenting the multimodal
speech capture system (MSCS) that records and displays kinematics of key speech
articulators, the tongue and lips, along with voice, using unobtrusive methods. Collected
speech modalities, tongue motion, lips gestures, and voice are visualized not only in real …
Speech-language pathologists (SLPs) are trained to correct articulation of people diagnosed with motor speech disorders by analyzing articulators’ motion and assessing speech outcome while patients speak. To assist SLPs in this task, we are presenting the multimodal speech capture system (MSCS) that records and displays kinematics of key speech articulators, the tongue and lips, along with voice, using unobtrusive methods. Collected speech modalities, tongue motion, lips gestures, and voice are visualized not only in real-time to provide patients with instant feedback but also offline to allow SLPs to perform post-analysis of articulators’ motion, particularly the tongue, with its prominent but hardly visible role in articulation. We describe the MSCS hardware and software components, and demonstrate its basic visualization capabilities by a healthy individual repeating the words “Hello World.” A proof-of-concept prototype has been successfully developed for this purpose, and will be used in future clinical studies to evaluate its potential impact on accelerating speech rehabilitation by enabling patients to speak naturally. Pattern matching algorithms to be applied to the collected data can provide patients with quantitative and objective feedback on their speech performance, unlike current methods that are mostly subjective, and may vary from one SLP to another.
ieeexplore.ieee.org
以上显示的是最相近的搜索结果。 查看全部搜索结果