People with paralysis control robotic arms using brain-computer interface

A new study in Nature reports that two people with tetraplegia were able to reach for and grasp objects in three-dimensional space using robotic arms that they controlled directly with brain activity. They used the BrainGate neural interface system, an investigational device currently being studied under an Investigational Device Exemption. One participant used the system to serve herself coffee for the first time since becoming paralyzed nearly 15 years ago.

One small step  

A 58-year-old woman, paralyzed by a stroke for almost 15 years, uses her thoughts to control a robotic arm, grasp a bottle of coffee, serve herself a drink, and return the bottle to the table.

PROVIDENCE, R.I. [Brown University] — On April 12, 2011, nearly 15 years after she became paralyzed and unable to speak, a woman controlled a robotic arm by thinking about moving her arm and hand to lift a bottle of coffee to her mouth and take a drink. That achievement is one of the advances in brain-computer interfaces, restorative neurotechnology, and assistive robot technology described in the May 17 edition of the journal Nature by the BrainGate2 collaboration of researchers at the Department of Veterans Affairs, Brown University, Massachusetts General Hospital, Harvard Medical School, and the German Aerospace Center (DLR).

A 58-year-old woman (“S3”) and a 66-year-old man (“T2”) participated in the study. They had each been paralyzed by a brainstem stroke years earlier which left them with no functional control of their limbs. In the research, the participants used neural activity to directly control two different robotic arms, one developed by the DLR Institute of Robotics and Mechatronics and the other by DEKA Research and Development Corp., to perform reaching and grasping tasks across a broad three-dimensional space. The BrainGate2 pilot clinical trial employs the investigational BrainGate system initially developed at Brown University, in which a baby aspirin-sized device with a grid of 96 tiny electrodes is implanted in the motor cortex — a part of the brain that is involved in voluntary movement. The electrodes are close enough to individual neurons to record the neural activity associated with intended movement. An external computer translates the pattern of impulses across a population of neurons into commands to operate assistive devices, such as the DLR and DEKA robot arms used in the study now reported in Nature.

BrainGate participants have previously demonstrated neurally based two-dimensional point-and-click control of a cursor on a computer screen and rudimentary control of simple robotic devices.

The study represents the first demonstration and the first peer-reviewed report of people with tetraplegia using brain signals to control a robotic arm in three-dimensional space to complete a task usually performed by their arm. Specifically, S3 and T2 controlled the arms to reach for and grasp foam targets that were placed in front of them using flexible supports. In addition, S3 used the DLR robot to pick up a bottle of coffee, bring it to her mouth, issue a command to tip it, drink through a straw, and return the bottle to the table. Her BrainGate-enabled, robotic-arm control during the drinking task required a combination of two-dimensional movements across a table top plus a “grasp” command to either grasp and lift or tilt the robotic hand.

“Our goal in this research is to develop technology that will restore independence and mobility for people with paralysis or limb loss,” said lead author Dr. Leigh Hochberg, a neuroengineer and critical care neurologist who holds appointments at the Department of Veterans Affairs, Brown University, Massachusetts General Hospital, and Harvard. He is the sponsor-investigator for the BrainGate2 pilot clinical trial. “We have much more work to do, but the encouraging progress of this research is demonstrated not only in the reach-and-grasp data, but even more so in S3’s smile when she served herself coffee of her own volition for the first time in almost 15 years.”

Leigh Hochberg
Even after nearly 15 years, a part of the brain essentially “disconnected” from its
original target by a brainstem stroke was still able to direct the complex,
multidimensional movement of an external arm.

Partial funding for this work comes from the VA, which is committed to improving the lives of injured veterans. “VA is honored to have played a role in this exciting and promising area of research,” said VA Secretary Eric Shinseki. “Today’s announcement represents a great step forward toward improving the quality of life for veterans and others who have either lost limbs or are paralyzed.”

Hochberg adds that even after nearly 15 years, a part of the brain essentially “disconnected” from its original target by a brainstem stroke was still able to direct the complex, multidimensional movement of an external arm — in this case, a robotic limb. The researchers also noted that S3 was able to perform the tasks more than five years after the investigational BrainGate electrode array was implanted. This sets a new benchmark for how long implanted brain-computer interface electrodes have remained viable and provided useful command signals.

John Donoghue, the VA and Brown neuroscientist who pioneered BrainGate more than a decade ago and who is co-senior author of the study, said the paper shows how far the field of brain-computer interfaces has come since the first demonstrations of computer control with BrainGate.

“This paper reports an important advance by rigorously demonstrating in more than one participant that precise three-dimensional neural control of robot arms is not only possible, but also repeatable,” said Donoghue, who directs the Brown Institute for Brain Science. “We’ve moved significantly closer to returning everyday functions, like serving yourself a sip of coffee, usually performed effortlessly by the arm and hand, for people who are unable to move their own limbs. We are also encouraged to see useful control more than five years after implant of the BrainGate array in one of our participants. This work is a critical step toward realizing the long-term goal of creating a neurotechnology that will restore movement, control, and independence to people with paralysis or limb loss.”

In the research, the robots acted as a substitute for each participant’s paralyzed arm. The robotic arms responded to the participants’ intent to move as they imagined reaching for each foam target. The robot hand grasped the target when the participants imagined a hand squeeze. Because the diameter of the targets was more than half the width of the robot hand openings, the task required the participants to exert precise control. (Videos of these actions are available on the Nature website.)

John Donoghue
“We’ve moved significantly closer to returning everyday functions, like serving yourself
a sip of coffee, usually performed effortlessly by the arm and hand, for people who are
unable to move their own limbs.”

In 158 trials over four days, S3 was able to touch the target within an allotted time in 48.8 percent of the cases using the DLR robotic arm and hand and 69.2 percent of the cases with the DEKA arm and hand, which has the wider grasp. In 45 trials using the DEKA arm, T2 touched the target 95.6 percent of the time. Of the successful touches, S3 grasped the target 43.6 percent of the time with the DLR arm and 66.7 percent of the time with the DEKA arm. T2’s grasp succeeded 62.2 percent of the time.

T2 performed the session in this study on his fourth day of interacting with the arm; the prior three sessions were focused on system development. Using his eyes to indicate each letter, he later described his control of the arm: “I just imagined moving my own arm and the [DEKA] arm moved where I wanted it to go.”

The study used two advanced robotic arms: the DLR Light-Weight Robot III with DLR five-fingered hand and the DEKA Arm System. The DLR LWR-III, which is designed to assist in recreating actions like the human arm and hand and to interact with human users, could be valuable as an assistive robotic device for people with various disabilities. Patrick van der Smagt, head of bionics and assistive robotics at DLR, director of biomimetic robotics and machine learning labs at DLR and the Technische Universität München, and a co-senior author on the paper said: “This is what we were hoping for with this arm. We wanted to create an arm that could be used intuitively by varying forms of control. The arm is already in use by numerous research labs around the world who use its unique interaction and safety capabilities. This is a compelling demonstration of the potential utility of the arm by a person with paralysis.”

DEKA Research and Development developed the DEKA Arm System for amputees, through funding from the United States Defense Advanced Research Projects Agency (DARPA). Dean Kamen, founder of DEKA said, “One of our dreams for the Luke Arm [as the DEKA Arm System is known informally] since its inception has been to provide a limb that could be operated not only by external sensors, but also by more directly thought-driven control. We’re pleased about these results and for the continued research being done by the group at the VA, Brown and MGH.” The research is aimed at learning how the DEKA arm might be controlled directly from the brain, potentially allowing amputees to more naturally control this prosthetic limb.

Over the last two years, VA has been conducting an optimization study of the DEKA prosthetic arm at several sites, with the cooperation of veterans and active duty service members who have lost an arm. Feedback from the study is helping DEKA engineers to refine the artificial arm’s design and function. “Brain-computer interfaces, such as BrainGate, have the potential to provide an unprecedented level of functional control over prosthetic arms of the future,” said Joel Kupersmith, M.D., VA chief research and development officer. “This innovation is an example of federal collaboration at its finest.”

The BrainGate2 Neural Interface System
An implanted microelectrode array, first used more than a decade ago,
detects brain signals which can be translated by a computer into machine
instructions, allowing control of robotic devices by thought.

Story Landis, director of the National Institute of Neurological Disorders and Stroke, which funded the work in part, noted: “This technology was made possible by decades of investment and research into how the brain controls movement. It’s been thrilling to see the technology evolve from studies of basic neurophysiology and move into clinical trials, where it is showing significant promise for people with brain injuries and disorders.”

In addition to Hochberg, Donoghue, and van der Smagt, other authors on the paper are Daniel Bacher, Beata Jarosiewicz, Nicolas Masse, John Simeral, Joern Vogel, Sami Haddadin, Jie Liu, and Sydney Cash.

Additional comments

Vincent Ng
Medical Center Director, Providence VA Medical Center
“The VA is on the forefront of translational research that’s improving the quality of life for our Veterans who have sacrificed so much for our Nation. We are proud to be a part of this exciting, collaborative research.”

U.S. Sen. Sheldon Whitehouse
“I congratulate Brown University and the Providence VA Medical Center for this ground-breaking project, which could help to significantly improve the quality of life of disabled and paralyzed Americans, including many veterans. The innovations produced in this new study highlight the value of federal support for basic scientific research.”

U.S. Rep. David Cicilline
“I congratulate the entire Brown University community on the progress it has made in this project. It is my hope that with continued success, this advancement will help improve the quality of life for individuals with disabilities, especially our men and women in uniform.”

Jennifer French
Executive Director, Neurotech Network
“This latest development in cortical control research has the potential to revolutionize the way we interact with technology. More specifically, the possibilities open a new level of independence for those living with severe paralysis. Simple tasks like drinking, eating or brushing your teeth are not possible for people living with severe paralysis. The ability to perform these every-day tasks can create a new world of independence for people with severe disabilities.”

R. John Davenport
Associate Director, Brown University Institute for Brain Science
“This exciting advance from the BrainGate team exemplifies the amazing science that can only result when researchers from disparate disciplines collaborate. The Institute works to link fundamental science, engineering, and medicine among our more than 100 faculty members.”
The BrainGate2 study continues to enroll participants to take part in this research and recently added Stanford University as a member of the collaboration and a clinical trial site.

About the BrainGate collaboration
This advance is the result of the ongoing collaborative BrainGate research at Brown University, Massachusetts General Hospital, Providence VA Medical Center; researchers at Stanford University have recently joined the collaboration as well. The BrainGate research team is focused on developing and testing neuroscientifically inspired technologies to improve the communication, mobility, and independence of people with neurologic disorders, injury, or limb loss.

Funding for the study and its projects comes from the Rehabilitation Research and Development Service, Office of Research and Development, U.S. Department of Veterans Affairs, the National Institutes of Health (some grants were funded all or in part through the American Recovery and Reinvestment Act), the Eunice Kennedy Shriver National Institute of Child Health and Human Development/National Center for Medical Rehabilitation Research (HD53403, HD100018, HD063931), the National Institute on Deafness and Other Communication Disorders, the National Institute of Neurological Disorders and Stroke (NS025074), the National Institute of Biomedical Imaging and Bioengineering (EB007401), the Doris Duke Charitable Foundation, the MGH-Deane Institute for Integrated Research on Atrial Fibrillation and Stroke, Katie Samson Foundation, and the Craig H. Neilsen Foundation. The contents do not represent the official views of the Department of Veterans Affairs or the United States Government.

The implanted microelectrode array and associated neural recording hardware used in the BrainGate research are manufactured by BlackRock Microsystems LLC (Salt Lake City, Utah). The research prototype Gen2 DEKAarm was provided by DEKA Integrated Solutions Inc, under contract from the Defense Advanced Research Project Agency (DARPA).

The BrainGate pilot clinical trial was previously directed by Cyberkinetics Neurotechnology Systems Inc. Foxborough, Mass., (CKI). CKI ceased operations in 2009, before the collection of data reported in the Nature manuscript. The clinical trials of the BrainGate2 Neural Interface System are now administered by Massachusetts General Hospital, Boston, Mass. Donoghue is a former chief scientific officer and a former director of CKI; he held stocks and received compensation. Hochberg received research support from Massachusetts General and Spaulding Rehabilitation Hospitals, which in turn received clinical trial support from Cyberkinetics.

CAUTION: Investigational Device. Limited by Federal Law to Investigational Use. The device is being studied under an IDE for the detection and transmission of neural signals from the cortex to externally powered communication systems, environmental control systems, and assistive devices by persons unable to use their hands due to physical impairment. The clinical trial is ongoing; results presented are thus preliminary. The safety and effectiveness of the device have not been established.

Press contacts
David Orenstein, Brown University, david_orenstein@brown.edu, 401-527-2525
Mark Ballesteros, U.S. Dept. of Veterans Affairs, Mark.Ballesteros@va.gov, 202-461-7559.
Michael Morrison, Massachusetts General Hospital, mdmorrison@partners.org, 617-724-6425

Read More

BrainGate neural interface system reaches 1,000-day performance milestone

An investigational implanted system being developed to translate brain signals toward control of assistive devices has allowed a woman with paralysis to accurately control a computer cursor at 2.7 years after implantation, providing a key demonstration that neural activity can be read out and converted into action for an unprecedented length of time.

PROVIDENCE, R.I. [Brown University] — Demonstrating an important milestone for the longevity and utility of implanted brain-computer interfaces, a woman with tetraplegia using the investigational BrainGate* system continued to control a computer cursor accurately through neural activity alone more than 1,000 days after receiving the BrainGate implant, according to a team of physicians, scientists, and engineers developing and testing the technology at Brown University, the Providence VA Medical Center, and Massachusetts General Hospital (MGH). Results from five consecutive days of device use surrounding her 1,000th day in the device trial appeared online March 24 in the Journal of Neural Engineering.

“This proof of concept — that after 1,000 days a woman who has no functional use of her limbs and is unable to speak can reliably control a cursor on a computer screen using only the intended movement of her hand — is an important step for the field,” said Dr. Leigh Hochberg, a Brown engineering associate professor, VA rehabilitation researcher, visiting associate professor of neurology at Harvard Medical School, and director of the BrainGate pilot clinical trial at MGH.

The woman, identified in the paper as S3, performed two “point-and-click” tasks each day by thinking about moving the cursor with her hand. In both tasks she averaged greater than 90 percent accuracy. Some on-screen targets were as small as the effective area of a Microsoft Word menu icon.

“Our objective with the neural interface is to reach the level of performance of a person without a disability using a mouse,” said report lead author John Simeral, a VA researcher and assistant professor of engineering (research) at Brown. “These results highlight the potential for an intracortical neural interface system to provide a person that has locked-in syndrome with reliable, continuous point-and-click control of a standard computer application.”

In each of S3’s two tasks, performed in 2008, she controlled the cursor movement and click selections continuously for 10 minutes. The first task was to move the cursor to targets arranged in a circle and in the center of the screen, clicking to select each one in turn. The second required her to follow and click on a target as it sequentially popped up with varying size at random points on the screen.

From fundamental neuroscience to clinical utility

Under development since 2002, the investigational BrainGate system is a combination of hardware and software that directly senses electrical signals produced by neurons in the brain that control movement. By decoding those signals and translating them into digital instructions, the system is being evaluated for its ability to give people with paralysis control of external devices such as computers, robotic assistive devices, or wheelchairs. The BrainGate team is also engaged in research toward control of advanced prosthetic limbs and toward direct intracortical control of functional electrical stimulation devices for people with spinal cord injury, in collaboration with researchers at the Cleveland FES Center.

The system is currently in pilot clinical trials, directed by Hochberg at MGH.

BrainGate uses a tiny (4x4 mm, about the size of a baby aspirin) silicon electrode array to read neural signals directly within brain tissue. Although external sensors placed on the brain or skull surface can also read neural activity, they are believed to be far less precise. In addition, many prototype brain implants have eventually failed because of moisture or other perils of the internal environment.

“Neuroengineers have often wondered whether useful signals could be recorded from inside the brain for an extended period of time,” Hochberg said. “This is the first demonstration that this microelectrode array technology can provide useful neuroprosthetic signals allowing a person with tetraplegia to control an external device for an extended period of time.”

Moving forward

Device performance was not the same at 2.7 years as it was earlier on, Hochberg added. At 33 months fewer electrodes were recording useful neural signals than after only six months. But John Donoghue — VA senior research career scientist, Henry Merritt Wriston Professor of Neuroscience, director of the Brown Institute for Brain Science, and original developer of the BrainGate system — said no evidence has emerged of any fundamental incompatibility between the sensor and the brain. Instead, it appears that decreased signal quality over time can largely be attributed to engineering, mechanical or procedural issues. Since S3’s sensor was built and implanted in 2005, the sensor’s manufacturer has reported continual quality improvements. The data from this study will be used to further understand and modify the procedures or device to further increase durability.

“None of us will be fully satisfied with an intracortical recording device until it provides decades of useful signals,” Hochberg said. “Nevertheless, I’m hopeful that the progress made in neural interface systems will someday be able to provide improved communication, mobility, and independence for people with locked-in syndrome or other forms of paralysis and eventually better control over prosthetic, robotic, or functional electrical stimulation systems [stimulating electrodes that have already returned limb function to people with cervical spinal cord injury], even while engineers continue to develop ever-better implantable sensors.”

In addition to demonstrating the very encouraging longevity of the BrainGate sensor, the paper also presents an advance in how the performance of a brain-computer interface can be measured, Simeral said. “As the field continues to evolve, we’ll eventually be able to compare and contrast technologies effectively.”

As for S3, who had a brainstem stroke in the mid-1990s and is now in her late 50s, she continues to participate in trials with the BrainGate system, which continues to record useful signals, Hochberg said. However, data beyond the 1000th day in 2008 has thus far only been presented at scientific meetings, and Hochberg can only comment on data that has already completed the scientific peer review process and appeared in publication.

In addition to Simeral, Hochberg, and Donoghue, other authors are Brown computer scientist Michael Black and former Brown computer scientist Sung-Phil Kim.

About the BrainGate collaboration

This advance is the result of the ongoing collaborative BrainGate research at Brown University, Massachusetts General Hospital, and Providence VA Medical Center. The BrainGate research team is focused on developing and testing neuroscientifically inspired technologies to improve the communication, mobility, and independence of people with neurologic disorders, injury, or limb loss.

For more information, visit www.braingate2.org.

The implanted microelectrode array and associated neural recording hardware used in the BrainGate research are manufactured by BlackRock Microsystems, LLC (Salt Lake City, UT).

This research was funded in part by the Rehabilitation Research and Development Service, Department of Veterans Affairs; The National Institutes of Health (NIH), including NICHD-NCMRR, NINDS/NICHD, NIDCD/ARRA, NIBIB, NINDS-Javits; the Doris Duke Charitable Foundation; MGH-Deane Institute for Integrated Research on Atrial Fibrillation and Stroke; and the Katie Samson Foundation.

The BrainGate pilot clinical trial was previously directed by Cyberkinetics Neurotechnology Systems, Inc., Foxborough, MA (CKI). CKI ceased operations in 2009. The clinical trials of the BrainGate2 Neural Interface System are now administered by Massachusetts General Hospital, Boston, Mass. Donoghue is a former chief scientific officer and a former director of CKI; he held stocks and received compensation. Hochberg received research support from Massachusetts General and Spaulding Rehabilitation Hospitals, which in turn received clinical trial support from Cyberkinetics. Simeral received compensation as a consultant to CKI.

* CAUTION: Investigational Device. Limited by Federal Law to Investigational Use.

Read More

BrainGate2 featured in article by the National Institute on Deafness and Other Communication Disorders

The National Institute on Deafness and Other Communication Disorders (NIDCD) is one of the Institutes that comprise the National Institutes of Health (NIH). NIH is the Federal government's focal point for the support of biomedical research.

"BrainGate2 is the brainchild of a group of neuroscientists and neuroengineers based at Brown University in Providence, R. I. NIDCD grantee Leigh Hochberg, M.D., Ph.D., affiliated with Brown and the VA Medical Center in Providence, and Massachusetts General Hospital and Harvard Medical School in Boston, leads the effort to make this investigational technology a communication tool for the locked-in. Without even moving an eyelid, someone with locked-in syndrome only has to imagine moving his or her hand on a computer mouse to make the cursor move on the computer screen."

Full feature story located here:
http://www.nidcd.nih.gov/health/inside/fall10/pg1.html

Read More

CIMIT awards BrainGate research team member, W. Malik

Wasim Malik, PhD, a member of the BrainGate research team (led by Professors Leigh Hochberg and John Donoghue - see www.braingate2.org for details) has been awarded the Center for Integration of Medicine and Innovative Technology (CIMIT) Miles and Eleanor Shore Fellowship for 2011. This is considered a career development award to enable interdisciplinary contributions by an individual who represents the CIMIT values of academic excellence. Career development awards recognize individuals who seek careers focused on near term impact on patient care, and whose philosophy embraces innovation and multidisciplinary collaboration.

Dr. Malik will receive up to $50,000 to undertake research in the highly interdisciplinary field of neural prosthetics which encompasses neuroscience, clinical research, electrical engineering, mathematics, and statistics. He will focus on designing a new generation of brain-machine interfaces to restore function in patients with limb loss or paralysis. He intends to develop robust neural prosthetics with considerably simpler signal processing which will have a high potential for transition from the laboratory to the clinic. Dr. Malik’s mentor, Emery Brown, will supervise his training and research activities in neuromotor disability.  To learn more about Dr. Malik's background, visit his website.

Read More