Creating text from brain-computer interface

Scientists with the BrainGate research collaborative have, for the first time, used an implanted sensor to record the brain signals associated with handwriting and used those signals to create text on a computer in real time. In a study published in the journal Nature, a clinical trial participant with cervical spinal cord injury used the system to “type” words on a computer at a rate of 90 characters per minute, more than double the previous record for typing with a brain-computer interface. This was done by the participant merely thinking about the hand motions involved in creating written letters. “An important mission of our BrainGate consortium research is to restore rapid, intuitive communication for people with severe speech or motor impairments,” said Hochberg, who also directs the Center for Neurotechnology and Neurorecovery at Massachusetts General Hospital and the V.A. Rehabilitation Research and Development Center for Neurorestoration and Neurotechnology at the Veterans Affairs Providence Healthcare System. “Frank’s demonstration of fast, accurate neural decoding of handwriting marks an exciting new chapter in the development of clinically useful neurotechnologies.”

For this latest study, the team wanted to find out if asking a participant to think about motions involved in writing letters and words by hand would be faster. “We want to find new ways of letting people communicate faster,” Willett said. “This new system uses both the rich neural activity recorded by intracortical electrodes and the power of language models that, when applied to the neurally decoded letters, can create rapid and accurate text.” The trial participant, a 65-year-old man, was paralyzed from the neck down by a spinal cord injury. As part of the clinical trial, Henderson placed two tiny electrodes in a part of his brain associated with the movement of his right arm and hand. Using signals the sensors picked up from individual neurons when the man imagined writing, a machine learning algorithm recognized the patterns his brain produced with each letter. With this system, the man could copy sentences and answer questions at a rate similar to that of someone the same age typing on a smartphone. The system is so fast because each letter elicits a highly distinctive activity pattern, making it relatively easy for the algorithm to distinguish one from another, Willett says.

Source: Brown University news release