Credit: CC0 Public Domain
In the first scientific field, Columbia's neuroscientists have created a system that translates the thought into understandable, recognizable speech. By observing somebody's brain activity, technology can reconstruct the words one can hear with unprecedented clarity. This breakthrough, which uses the power of speech synthesizers and artificial intelligence, can lead to new ways of direct communication between computers and the brain. It also sets the foundation for helping those who can not speak, such as those who live as amyotrophic lateral sclerosis (ALS) or recover from stroke, regain their ability to communicate with the outside world.
These findings were published today in Scientific Reports .
"Voices help us connect with our friends, family and the world around us, so losing the power of your voice due to injury or illness is so devastating," says Nima Messgarani, PhD, senior author and principal investigator at Mortimer B. Tukerman, Brain Brain Research Institute at Columbia University. "With today's research, we have a potential way to reinvigorate this power. We have shown that with the right technology the thoughts of these people can be decoded and understood by every listener. "Speak ̵
1; or even imagine that you are talking – in your brain, signals of activity appear different, but recognizable signal pattern too appears when we listen to someone to talk or imagine listening Experts trying to record and decode these models see a future in which thoughts should not remain hidden in the brain but instead can be translated into verbal speech by the way,
But he achieves The challenge of decoding brain signals by Dr. Messgani and others focused on simple computer models that analyzed spectrograms that are visual images of sound frequencies
But since this approach is not has succeeded in producing something similar to understandable speech, Dr. Messgani's team turned instead to a vocoder, a computer algorithm that can synthesize speech after being trained to record people who speak.
"This is the same technology used by Amazon Echo and Apple Siri to give our oral answers to our questions," said Dr. Messgani, an associate professor of electrical engineering at the Columbia School of Engineering and Applied Sciences.
In order to teach the vocoder to interpret brain activity, Dr. Messganani joined forces with Ashein Dinesh Mehta, Ph.D., a neurosurgeon at the Northwest Healthcare Institute and co-author of today's book. Dr. Mehta treats patients with epilepsy, some of which need to undergo regular surgery.
"Working with Dr. Mehta, we have asked patients with epilepsy who have already undergone brain surgery to listen to sentences spoken by different people as we measure patterns of brain activity," said Dr. Messgani. "These neural models train the vocoder."
The researchers then asked these patients to listen to speakers that recite numbers from 0 to 9 while recording brain signals that can then be passed through the vocoder, the sound produced by the vocoder in response to these signals, was analyzed and purified by neuronal nets, a type of artificial intelligence that mimics the structure of neurons in the biological brain
The end result was the robotic voice that recited a series of numbers. To check the accuracy of the recording, Dr. Messgani and his team instructed people to listen to the recording and report what they heard.
"We have found that people can understand and repeat the sounds about 75% of the time, which is above all previous experiments," said Dr. Messgarani. "Improving intelligibility is especially evident when comparing new records with a lower- the early, spectrogram-based experiments – Sensitive vocoder and powerful neural networks are the sounds the patients originally listened to with surprising accuracy
Dr. Mesgarani and his team plan to test more complex words and sentences later on want to run the same tests on m When they talk or imagine to talk, they hope that their system can be part of an implant similar to those carried by some patients with epilepsy that directly translates the user's thoughts.
"In this scenario, if the user thinks I need a glass of water," our system can take the brain signals generated by this thought and turn them into a synthesized, verbal speech, "says Dr. Messgani. "This will be a change in the game, which will give anyone who has lost the ability to talk, either through injury or illness, a renewed chance to connect with the world around them." from the human auditory crust.
The cognitive hearing aid filters the noise
Engineers translate brain signals directly into speech (2019, January 29)
drawn up on 29 January 2019
This document is subject to copyright. Apart from any honest work for the purposes of private research or research, no
may be reproduced without written permission. Content is provided for information purposes only.