A brain implant that uses artificial intelligence (AI) can almost instantaneously decode a person’s thoughts and stream them through a speaker, new research shows. This is the first time researchers have achieved near-synchronous brain-to-voice streaming.
The experimental mind-reading technology is designed to give a synthetic voice to people with severe paralysis who cannot speak. It works by putting electrodes onto the brain’s surface as part of an implant called a neuroprosthesis, which allows scientists to identify and interpret speech signals.
The brain-computer interface (BCI) uses AI to decode neural signals and can stream intended speech from the brain in close to real time, according to a statement released by the University of California (UC), Berkeley. The team previously unveiled an earlier version of the technology in 2023, but the new version is quicker and less robotic.
“Our streaming approach brings the same rapid speech decoding capacity of devices like Alexa and Siri to neuroprostheses,” study co-principal investigator Gopala Anumanchipalli, an assistant professor of electrical engineering and computer sciences at UC Berkeley, said in the statement. “Using a similar type of algorithm, we found that we could decode neural data and, for the first time, enable near-synchronous voice streaming.”
Anumanchipalli and his colleagues shared their findings in a study published Monday (March 31) in the journal Nature Neuroscience.
Related: AI analysis of 100 hours of real conversations — and the brain activity underpinning them — reveals how humans understand language
The first person to trial this technology, identified as Ann, suffered a stroke in 2005 that left her severely paralyzed and unable to speak. She has since allowed researchers to implant 253 electrodes onto her brain to monitor the part of our brains that controls speech — called the motor cortex — to help develop synthetic speech technologies.
“We are essentially intercepting signals where the thought is translated into articulation and in the middle of that motor control,” study co-lead author Cheol Jun Cho, a doctoral student in electrical engineering and computer sciences at UC Berkeley, said in the statement. “So what we’re decoding is after a thought has happened, after we’ve decided what to say, after we’ve decided what words to use and how to move our vocal-tract muscles.”
AI decodes data sampled by the implant to help convert neural activity into synthetic speech. The team trained their AI algorithm by having Ann silently attempt to speak sentences that appeared on a screen before her, and then by matching the neural activity to the words she wanted to say.
The system sampled brain signals every 80 milliseconds (0.08 seconds) and could detect words and convert them into speech with a delay of up to around 3 seconds, according to the study. That’s a little slow compared to normal conversation, but faster than the previous version, which had a delay of about 8 seconds and could only process whole sentences.
The new system benefits from converting shorter windows of neural activity than the old one, so it can continuously process individual words rather than waiting for a finished sentence. The researchers say the new study is a step toward achieving more natural-sounding synthetic speech with BCIs.
“This proof-of-concept framework is quite a breakthrough,” Cho said. “We are optimistic that we can now make advances at every level. On the engineering side, for example, we will continue to push the algorithm to see how we can generate speech better and faster.”
