Jose A. Gonzalez
A silent speech system based on permanent magnet articulography and direct synthesis
Gonzalez, Jose A.; Cheah, Lam A.; Gilbert, James M.; Bai, Jie; Ell, Stephen R.; Green, Phil D.; Moore, Roger K.
Authors
Lam A. Cheah
Professor James Gilbert J.M.Gilbert@hull.ac.uk
Professor of Engineering
Jie Bai
Stephen R. Ell
Phil D. Green
Roger K. Moore
Abstract
In this paper we present a silent speech interface (SSI) system aimed at restoring speech communication for individuals who have lost their voice due to laryngectomy or diseases affecting the vocal folds. In the proposed system, articulatory data captured from the lips and tongue using permanent magnet articulography (PMA) are converted into audible speech using a speaker-dependent transformation learned from simultaneous recordings of PMA and audio signals acquired before laryngectomy. The transformation is represented using a mixture of factor analysers, which is a generative model that allows us to efficiently model non-linear behaviour and perform dimensionality reduction at the same time. The learned transformation is then deployed during normal usage of the SSI to restore the acoustic speech signal associated with the captured PMA data. The proposed system is evaluated using objective quality measures and listening tests on two databases containing PMA and audio recordings for normal speakers. Results show that it is possible to reconstruct speech from articulator movements captured by an unobtrusive technique without an intermediate recognition step. The SSI is capable of producing speech of sufficient intelligibility and naturalness that the speaker is clearly identifiable, but problems remain in scaling up the process to function consistently for phonetically rich vocabularies.
Citation
Gonzalez, J. A., Cheah, L. A., Gilbert, J. M., Bai, J., Ell, S. R., Green, P. D., & Moore, R. K. (2016). A silent speech system based on permanent magnet articulography and direct synthesis. Computer speech & language, 39, 67-87. https://doi.org/10.1016/j.csl.2016.02.002
Acceptance Date | Feb 19, 2016 |
---|---|
Online Publication Date | Mar 14, 2016 |
Publication Date | Sep 1, 2016 |
Deposit Date | Feb 21, 2017 |
Publicly Available Date | Feb 21, 2017 |
Journal | Computer speech & language |
Print ISSN | 0885-2308 |
Publisher | Elsevier |
Peer Reviewed | Peer Reviewed |
Volume | 39 |
Pages | 67-87 |
DOI | https://doi.org/10.1016/j.csl.2016.02.002 |
Keywords | Silent speech interfaces; Speech rehabilitation; Speech synthesis; Permanent magnet articulography; Augmentative and alternative communication |
Public URL | https://hull-repository.worktribe.com/output/448695 |
Publisher URL | http://www.sciencedirect.com/science/article/pii/S0885230815300255 |
Additional Information | Authors' accepted manuscript of article published in: Computer speech & language, 2016, v.39. |
Contract Date | Feb 21, 2017 |
Files
Article.pdf
(7.9 Mb)
PDF
Copyright Statement
© 2017. This manuscript version is made available under the CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/
You might also like
Direct Speech Reconstruction from Articulatory Sensor Data by Machine Learning
(2017)
Journal Article
Voice restoration after laryngectomy based on magnetic sensing of articulator movement and statistical articulation-to-speech conversion
(2017)
Presentation / Conference Contribution
Towards an intraoral-based silent speech restoration system for post-laryngectomy voice replacement
(2017)
Presentation / Conference Contribution
Monitoring of Curing Process of Epoxy Resin by Long-Period Fiber Gratings
(2024)
Journal Article
Downloadable Citations
About Repository@Hull
Administrator e-mail: repository@hull.ac.uk
This application uses the following open-source libraries:
SheetJS Community Edition
Apache License Version 2.0 (http://www.apache.org/licenses/)
PDF.js
Apache License Version 2.0 (http://www.apache.org/licenses/)
Font Awesome
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search