A new brain-computer interface (BCI) developed at University of California (UC) Davis Health (Sacramento, USA) has allowed a man with severely impaired speech due to amyotrophic lateral sclerosis (ALS) to communicate by transforming his brain activity into text—which is then read aloud by a computer.
According to UC Davis Health, this novel device translates brain signals into speech with up to 97% accuracy, making it “the most accurate system of its kind”.
A study now published in the New England Journal of Medicine (NEJM) details how researchers implanted sensors into the patient’s brain, and he was able to communicate his intended speech “within minutes” of activating the system.
“Our BCI technology helped a man with paralysis to communicate with friends, families and caregivers,” said neurosurgeon David Brandman (UC Davis Health, Sacramento, USA), who is the co-principal investigator and co-senior author of this study. “Our paper demonstrates the most accurate speech neuroprosthesis [device] ever reported.”
Breaking the communication barrier
ALS—also known as Lou Gehrig’s disease—affects the nerve cells that control movement throughout the body. In addition to a gradual diminishing of the ability to stand, walk and use one’s hands, it can also cause a person to lose control of the muscles used to speak, leading to a loss of understandable speech.
BCI technologies are currently being developed to restore communication for people who are unable to speak due to paralysis or neurological conditions like ALS. Notable examples include an implantable device from Neuralink, founded by Elon Musk, and Synchron’s endovascular BCI technology—both of which are also being evaluated in early-stage human trials. The system introduced by UC Davis Health functions by interpreting brain signals when the user tries to speak and turning those signals into text that is subsequently read aloud by a computer.
To develop their system, the UC Davis Health team enrolled Casey Harrell, a 45-year-old man with ALS, in the BrainGate clinical trial. At the time of his enrolment, Harrell had weakness in his arms and legs, known as tetraparesis, and he had dysarthria, meaning his speech was very difficult to understand and required others to help interpret for him. In July 2023, Brandman implanted the investigational BCI device, placing four microelectrode arrays designed to record brain activity from 256 cortical electrodes into the left precentral gyrus—a brain region responsible for coordinating speech.
“We’re really detecting their attempt to move their muscles and talk,” explained neuroscientist Sergey Stavisky (UC Davis Health, Sacramento, USA), co-principal investigator of the study. “We are recording from the part of the brain that’s trying to send these commands to the muscles. We are basically listening into that, and we’re translating those patterns of brain activity into a phoneme—like a syllable, or the unit of speech—and then the words they’re trying to say.”
The benefits of rapid training
A UC Davis Health press release notes that—despite recent advances in BCI technology—efforts to enable communication have been slow and prone to errors, because the machine-learning programmes used to interpret brain signals have required a large amount of time and data to perform.
“Previous speech BCI systems had frequent word errors,” Brandman commented. “This made it difficult for the user to be understood consistently and was a barrier to communication. Our objective was to develop a system that empowered someone to be understood whenever they wanted to speak.”
ALS patient Harrell has used the system in both prompted and spontaneous conversational settings. In both cases, speech decoding occurred in real time, with continuous system updates ensuring that it kept working accurately. The decoded words were shown on a screen and even read aloud in a voice that sounded like Harrell’s before he had ALS, having been composed using software trained with existing audio samples of his pre-ALS voice.
At the first speech data training session, the system took 30 minutes to achieve a word accuracy rate of 99.6% across a 50-word vocabulary. In the second session, the size of the potential vocabulary increased to 125,000 words. With just an additional 1.4 hours of training data, the BCI achieved a word accuracy of 90.2% within this greatly expanded vocabulary. And, following continued data collection, the BCI has been able to maintain an accuracy of 97.5%.
“At this point, we can decode what Casey is trying to say correctly about 97% of the time, which is better than many commercially available smartphone applications that try to interpret a person’s voice,” Brandman said. “This technology is transformative because it provides hope for people who want to speak but can’t. I hope that technology like this speech BCI will help future patients speak with their family and friends.”
The NEJM study reports on 84 data collection sessions over a period of 32 weeks. In total, Harrell used the speech BCI in self-paced conversations for more than 248 hours, communicating in person and via video chat.
“Not being able to communicate is so frustrating and demoralising. It is like you are trapped,” Harrell said. “Something like this technology will help people back into life and society.”
“Casey and our other BrainGate participants are truly extraordinary. They deserve tremendous credit for joining these early clinical trials. They do this not because they’re hoping to gain any personal benefit, but to help us develop a system that will restore communication and mobility for other people with paralysis,” added neurologist Leigh Hochberg (Massachusetts General Hospital, Boston, USA/Brown University, Providence, USA), study co-author and BrainGate trial sponsor-investigator.
Brandman is the site-responsible principal investigator of the BrainGate2 pilot clinical study, and he and his colleagues are currently enrolling participants with the goal of obtaining preliminary safety and feasibility data on the BrainGate2 neural interface system—which is currently limited to investigational use by US federal law.
