“For the first time, a man with locked-in syndrome was able to have him speak to other people in complete sentences – via brain interface.
Locked-in syndrome means that a person has lost all ability to communicate with the outside world. German researchers have now succeeded in breaking this condition. “It’s really remarkable when you can re-establish communication with a person who is in a state completely closed off from other people,” comments Jaimie Henderson, a neurosurgeon at Stanford University. “To me, this is a tremendous breakthrough and, of course, very significant for the study participant himself.”
ALS diagnosis eight years ago
The affected man, who lives in Germany, was diagnosed with amyotrophic lateral sclerosis (ALS) in August 2015 at the age of 30. ALS is a rare, usually worsening neurological disease that attacks the nerve cells involved in movement. As recently as the year of his diagnosis, the man was unable to speak or walk. Since July 2016, he has been dependent on a ventilator to breathe.
Since August 2016, he has used a so-called eye-tracking device to communicate. The technology detects eye movements and allows users to select letters on a computer screen. But a year later, his condition had deteriorated further and he was unable to even focus his eyes on a specific spot. The eye tracker became useless. His family members therefore began using a paper-based method in which one family member held up a grid of letters against a four-color background. Family members pointed to each color section and row and interpreted eye movements as “yes.”
The man and his family feared that he would eventually lose the ability to move his eyes at all. Therefore, they sought help from Niels Birbaumer, then working at the University of Tübingen. Further, Ujwal Chaudhary became involved with ALS Voice gGmbH, a nonprofit that provides brain interfaces, known as brain-computer interfaces (BCIs), and other novel technologies for people who otherwise cannot communicate.
Eye tracking no longer works
When Chaudhary met the man in February 2018, he first tried to improve the communication system the family was already using. The team combined its eye-tracking device with computer software that read aloud colors and series of numbers, allowing the man to select individual letters with his eye movements to spell words.
But as the man increasingly lost control of his eye movements, he became less able to communicate with this device as well. “So we suggested implanting an electrode,” Chaudhary says. Tiny electrodes can be implanted in the brain to directly record the electrical activity of nerve cells. The procedure, which usually involves drilling a hole in the skull and penetrating the individual layers of the brain, carries some risk of infection and brain damage. But it was a last resort, Birbaumer says. “If the noninvasive BCIs and eye trackers no longer work, there’s no other choice,” he says.
The man consented to the eye-tracking procedure, Chaudhary says. His wife and sister also gave their consent. By the time the procedure was approved by an ethics committee and the German Federal Institute for Drugs and Medical Devices in late 2019, the man had lost the ability to use the eye system. So in March 2019, surgeons implanted two tiny electrode grids, each 1.5 millimeters in diameter, into the man’s motor cortex – a region at the top of the brain responsible for controlling movement.
Converting signals into commands
The day after the electrodes were implanted, the team began helping the man communicate. First, the man was asked to visualize himself moving – this has helped other patients control prosthetic limbs and exoskeletons, and is also the approach Elon Musk’s company, Neuralink, wants to take. The idea is to get a reliable signal from the brain and translate it into commands.
But the team couldn’t get the BCI to work at first. After 12 weeks of trials, they discarded the previous idea and decided to try an approach called neurofeedback instead. Neurofeedback involves showing a person their brain activity in real time so they can learn to control it. In this case, a computer played a rising tone when the electrodes in the man’s brain registered increased activity. When brain activity decreased, a decreasing tone was played.
“Within two days, he was able to increase and decrease the frequency of a tone,” says Chaudhary, who says he visited the man in his home every weekday in 2019 until the COVID-19 pandemic began to occur. “It was just amazing.” The man eventually learned to control his brain activity so he could play a rising tone to signal “yes” and a falling tone to signal “no.”
Entire sentences communicable
The team then introduced software that mimicked the paper-based computer system the man had originally used to communicate with his family. For example, the man heard the word “yellow” or “blue” to select a block of letters from which he could choose. Then individual letters were played to him, which he could either select or discard with a rising or falling tone.
In this way, the man was soon able to speak complete sentences. “His family was so excited by what he had to say,” says Chaudhary, who with his colleagues published their findings last week in the journal Nature Communications. One of the first sentences the man spelled out translated as “Guys, it works so effortlessly.”
Communication was still slow – it takes about a minute to select each letter. But the researchers believe the device still greatly improved the man’s quality of life. He asked for certain meals and soups, instructed caregivers to move and massage his legs, and asked to watch movies with his young son, for example. At one point, he said, “I love my cool son,” the researchers said. “Often I was with him until midnight or after midnight,” Chaudhary said. “The last word was always ‘beer.’
“System to get faster
Chaudhary envisions developing a catalog of commonly used words that could eventually allow the software to automatically complete the man’s words as he spelled them. “There are many ways we could make the system faster,” he says.
No one knows how long the electrodes can remain in a man’s brain, but other studies have found that similar electrodes continue to work five years after they are implanted. But for a person with locked-in syndrome, “a single day can make a difference,” says Kianoush Nazarpour of the University of Edinburgh, who was not involved in the work. “This is a fundamental opportunity for them to take back control of their lives,” he says. A single day of increased quality of life as a result can be very important for this group of people, he adds.
Nazarpour believes the technology could be routinely offered to similarly diagnosed individuals in the next 10 to 15 years. “For a person who absolutely cannot communicate, even a “yes”/”no” is potentially life-changing,” he says.”
3 comments
Translation (1/2):
“For the first time, a man with locked-in syndrome was able to have him speak to other people in complete sentences – via brain interface.
Locked-in syndrome means that a person has lost all ability to communicate with the outside world. German researchers have now succeeded in breaking this condition. “It’s really remarkable when you can re-establish communication with a person who is in a state completely closed off from other people,” comments Jaimie Henderson, a neurosurgeon at Stanford University. “To me, this is a tremendous breakthrough and, of course, very significant for the study participant himself.”
ALS diagnosis eight years ago
The affected man, who lives in Germany, was diagnosed with amyotrophic lateral sclerosis (ALS) in August 2015 at the age of 30. ALS is a rare, usually worsening neurological disease that attacks the nerve cells involved in movement. As recently as the year of his diagnosis, the man was unable to speak or walk. Since July 2016, he has been dependent on a ventilator to breathe.
Since August 2016, he has used a so-called eye-tracking device to communicate. The technology detects eye movements and allows users to select letters on a computer screen. But a year later, his condition had deteriorated further and he was unable to even focus his eyes on a specific spot. The eye tracker became useless. His family members therefore began using a paper-based method in which one family member held up a grid of letters against a four-color background. Family members pointed to each color section and row and interpreted eye movements as “yes.”
The man and his family feared that he would eventually lose the ability to move his eyes at all. Therefore, they sought help from Niels Birbaumer, then working at the University of Tübingen. Further, Ujwal Chaudhary became involved with ALS Voice gGmbH, a nonprofit that provides brain interfaces, known as brain-computer interfaces (BCIs), and other novel technologies for people who otherwise cannot communicate.
Eye tracking no longer works
When Chaudhary met the man in February 2018, he first tried to improve the communication system the family was already using. The team combined its eye-tracking device with computer software that read aloud colors and series of numbers, allowing the man to select individual letters with his eye movements to spell words.
But as the man increasingly lost control of his eye movements, he became less able to communicate with this device as well. “So we suggested implanting an electrode,” Chaudhary says. Tiny electrodes can be implanted in the brain to directly record the electrical activity of nerve cells. The procedure, which usually involves drilling a hole in the skull and penetrating the individual layers of the brain, carries some risk of infection and brain damage. But it was a last resort, Birbaumer says. “If the noninvasive BCIs and eye trackers no longer work, there’s no other choice,” he says.
The man consented to the eye-tracking procedure, Chaudhary says. His wife and sister also gave their consent. By the time the procedure was approved by an ethics committee and the German Federal Institute for Drugs and Medical Devices in late 2019, the man had lost the ability to use the eye system. So in March 2019, surgeons implanted two tiny electrode grids, each 1.5 millimeters in diameter, into the man’s motor cortex – a region at the top of the brain responsible for controlling movement.
Converting signals into commands
The day after the electrodes were implanted, the team began helping the man communicate. First, the man was asked to visualize himself moving – this has helped other patients control prosthetic limbs and exoskeletons, and is also the approach Elon Musk’s company, Neuralink, wants to take. The idea is to get a reliable signal from the brain and translate it into commands.
But the team couldn’t get the BCI to work at first. After 12 weeks of trials, they discarded the previous idea and decided to try an approach called neurofeedback instead. Neurofeedback involves showing a person their brain activity in real time so they can learn to control it. In this case, a computer played a rising tone when the electrodes in the man’s brain registered increased activity. When brain activity decreased, a decreasing tone was played.
“Within two days, he was able to increase and decrease the frequency of a tone,” says Chaudhary, who says he visited the man in his home every weekday in 2019 until the COVID-19 pandemic began to occur. “It was just amazing.” The man eventually learned to control his brain activity so he could play a rising tone to signal “yes” and a falling tone to signal “no.”
Entire sentences communicable
The team then introduced software that mimicked the paper-based computer system the man had originally used to communicate with his family. For example, the man heard the word “yellow” or “blue” to select a block of letters from which he could choose. Then individual letters were played to him, which he could either select or discard with a rising or falling tone.
In this way, the man was soon able to speak complete sentences. “His family was so excited by what he had to say,” says Chaudhary, who with his colleagues published their findings last week in the journal Nature Communications. One of the first sentences the man spelled out translated as “Guys, it works so effortlessly.”
Communication was still slow – it takes about a minute to select each letter. But the researchers believe the device still greatly improved the man’s quality of life. He asked for certain meals and soups, instructed caregivers to move and massage his legs, and asked to watch movies with his young son, for example. At one point, he said, “I love my cool son,” the researchers said. “Often I was with him until midnight or after midnight,” Chaudhary said. “The last word was always ‘beer.’
“System to get faster
Chaudhary envisions developing a catalog of commonly used words that could eventually allow the software to automatically complete the man’s words as he spelled them. “There are many ways we could make the system faster,” he says.
No one knows how long the electrodes can remain in a man’s brain, but other studies have found that similar electrodes continue to work five years after they are implanted. But for a person with locked-in syndrome, “a single day can make a difference,” says Kianoush Nazarpour of the University of Edinburgh, who was not involved in the work. “This is a fundamental opportunity for them to take back control of their lives,” he says. A single day of increased quality of life as a result can be very important for this group of people, he adds.
Nazarpour believes the technology could be routinely offered to similarly diagnosed individuals in the next 10 to 15 years. “For a person who absolutely cannot communicate, even a “yes”/”no” is potentially life-changing,” he says.”
Here’s also a link to the original publishing:
https://www.nature.com/articles/s41467-022-28859-8
That’s huge! Imagine the military applications this technology, once mature, could have.
Of course I’m happy for the poor patient (but think of the implications, too).