Voice restoration after laryngectomy based on magnetic sensing of articulator movement and statistical articulation-to-speech conversion

Gonzalez, Jose A.; Cheah, Lam A.; Gilbert, James M.; Bai, Jie; Ell, Stephen R.; Green, Phil D.; Moore, Roger K.

Abstract

© Springer International Publishing AG 2017. In this work, we present a silent speech system that is able to generate audible speech from captured movement of speech articulators. Our goal is to help laryngectomy patients, i.e. patients who have lost the ability to speak following surgical removal of the larynx most frequently due to cancer, to recover their voice. In our system, we use a magnetic sensing technique known as Permanent Magnet Articulography (PMA) to capture the movement of the lips and tongue by attaching small magnets to the articulators and monitoring the magnetic field changes with sensors close to the mouth. The captured sensor data is then transformed into a sequence of speech parameter vectors from which a time-domain speech signal is finally synthesised. The key component of our system is a parametric transformation which represents the PMA-tospeech mapping. Here, this transformation takes the form of a statistical model (a mixture of factor analysers, more specifically) whose parameters are learned from simultaneous recordings of PMA and speech signals acquired before laryngectomy. To evaluate the performance of our system on voice reconstruction, we recorded two PMA-and-speech databases with different phonetic complexity for several non-impaired subjects. Results show that our system is able to synthesise speech that sounds as the original voice of the subject and also is intelligible. However, more work still need to be done to achieve a consistent synthesis for phonetically-rich vocabularies.

Citation

Gonzalez, J. A., Cheah, L. A., Gilbert, J. M., Bai, J., Ell, S. R., Green, P. D., & Moore, R. K. Voice restoration after laryngectomy based on magnetic sensing of articulator movement and statistical articulation-to-speech conversion. Presented at Biomedical Engineering Systems and Technologies 9th International Joint Conference, BIOSTEC 2016, Rome, Italy

• Presentation Conference Type: Conference Paper (published)
• Conference Name: Biomedical Engineering Systems and Technologies 9th International Joint Conference, BIOSTEC 2016
• Acceptance Date: Apr 1, 2016
• Online Publication Date: Mar 4, 2017
• Publication Date: 2017
• Deposit Date: Apr 1, 2022
• Journal: Communications in Computer and Information Science
• Print ISSN: 1865-0929
• Publisher: Springer Verlag
• Peer Reviewed: Peer Reviewed
• Volume: 690
• Pages: 295-316
• ISBN: 9783319547169
• DOI: https://doi.org/10.1007/978-3-319-54717-6_17
• Keywords: Silent speech interfaces; Speech rehabilitation; Speech synthesis; Permanent magnet articulography
• Public URL: https://hull-repository.worktribe.com/output/3592127