Using Translation Models to Decode Overt Speech from BCI

Creating speech neuroprostheses, or devices that can help restore the ability to speak to people who have lost it due to neurological damage or disorder is extremely important because it can greatly improve the quality of life for these individuals. Despite this, there has not been a widely successful solution to this issue yet because creating a device that can effectively interface with the brain and restore speech is a technically challenging task that requires advances in neuroscience and engineering. Therefore there is very little publicly available data of people producing speech with implanted brain-computer interfaces. As a concequence, researchers struggle with creating generalizible and accurate algorithms for decoding speech. This works aims to re-use the models of natural language translation to inform the decoding model about language semantics. This can substantially decrease the amount of needed data for accurate predictions. Moreover, we aim to use transfer learning techniques to transfer knowledge between algorithms trained on different participants to address the issue of generalization.

Faculty Supervisor:

Bo Wang

Student:

Partner:

National University of Kyiv-Mohyla Academy

Discipline:

Computer science

Sector:

Health and Related Sciences & Technology; Biotechnology; Artificial Intelligence

University:

University of Toronto

Program: