Skip to main content

Research Repository

Advanced Search

A silent speech system based on permanent magnet articulography and direct synthesis

Gonzalez, Jose A.; Cheah, Lam A.; Gilbert, James M.; Bai, Jie; Ell, Stephen R.; Green, Phil D.; Moore, Roger K.

Authors

Jose A. Gonzalez

Lam A. Cheah

Jie Bai

Stephen R. Ell

Phil D. Green

Roger K. Moore



Abstract

In this paper we present a silent speech interface (SSI) system aimed at restoring speech communication for individuals who have lost their voice due to laryngectomy or diseases affecting the vocal folds. In the proposed system, articulatory data captured from the lips and tongue using permanent magnet articulography (PMA) are converted into audible speech using a speaker-dependent transformation learned from simultaneous recordings of PMA and audio signals acquired before laryngectomy. The transformation is represented using a mixture of factor analysers, which is a generative model that allows us to efficiently model non-linear behaviour and perform dimensionality reduction at the same time. The learned transformation is then deployed during normal usage of the SSI to restore the acoustic speech signal associated with the captured PMA data. The proposed system is evaluated using objective quality measures and listening tests on two databases containing PMA and audio recordings for normal speakers. Results show that it is possible to reconstruct speech from articulator movements captured by an unobtrusive technique without an intermediate recognition step. The SSI is capable of producing speech of sufficient intelligibility and naturalness that the speaker is clearly identifiable, but problems remain in scaling up the process to function consistently for phonetically rich vocabularies.

Citation

Gonzalez, J. A., Cheah, L. A., Gilbert, J. M., Bai, J., Ell, S. R., Green, P. D., & Moore, R. K. (2016). A silent speech system based on permanent magnet articulography and direct synthesis. Computer speech & language, 39, 67-87. https://doi.org/10.1016/j.csl.2016.02.002

Acceptance Date Feb 19, 2016
Online Publication Date Mar 14, 2016
Publication Date Sep 1, 2016
Deposit Date Feb 21, 2017
Publicly Available Date Mar 28, 2024
Journal Computer speech & language
Print ISSN 0885-2308
Electronic ISSN 1095-8363
Publisher Elsevier
Peer Reviewed Peer Reviewed
Volume 39
Pages 67-87
DOI https://doi.org/10.1016/j.csl.2016.02.002
Keywords Silent speech interfaces; Speech rehabilitation; Speech synthesis; Permanent magnet articulography; Augmentative and alternative communication
Public URL https://hull-repository.worktribe.com/output/448695
Publisher URL http://www.sciencedirect.com/science/article/pii/S0885230815300255
Additional Information Authors' accepted manuscript of article published in: Computer speech & language, 2016, v.39.

Files





You might also like



Downloadable Citations