Computational NeuroEngineering Laboratory,
University of Florida
Our brains use our bodies to interact with the external world.
However, emerging technology is poised to challenge this state of
affairs and create direct brain machine interfaces (BMIs) to open
up a digital channel between the brain and the physical world. The
seamless integration of brain and body in the healthy human makes
us forget that under some circumstances, brains can be deprived
of their sensing abilities (for example, blindness or deafness)
or motor abilities (for example, paralysis). The general concept
is therefore to either create artificial sensory systems by delivering
the external world stimuli to the appropriate regions of the brain
(as seen in retinal and cochlear implants), or to allow the brain
to directly command and control external devices, such as computer
cursors or robotic prostheses.
Potentially, BMIs may enable a higher bandwidth interface between
the human brain and the digital computer, circumventing the need
for computer mice and keyboards, and rewriting our metaphors on
how humans interact with computers. BMIs can also augment humans’
natural abilities by providing mind-based control of engineered
devices that are much faster and more powerful than biological tissue.
1. Sensory BMIs
BMIs can be divided in two basic types, depending upon the application:
sensory or motor. Sensory BMIs stimulate sensory
areas of the brain with signals that are generated from external
physical stimuli. The most common sensory BMIs, with over 50,000
implants, are cochlear implants that allow deaf people to hear,
by translating sounds from wave pressure in the ear into spike firings
applied to the auditory nerve. The same basic concept is being developed
for a retinal prosthesis, which allows blind people to see outlines
of external objects by delivering the appropriate stimulation to
the visual cortex.
Motor BMIs, on the other hand, seek to translate electrical brain
activity that represents an intent to move into useful commands
to external devices; they are the ones emphasized here.
The two types of BMIs are very different. Sensory BMIs require
very accurate placement of a few tiny electrodes that stimulate
the appropriate site in the brain, and the device’s job is
to simulate the role of the appropriate sensory organ as accurately
as possible. In motor BMIs, the electrodes are placed “anywhere”
in the appropriate cortex area (such as the area that controls the
right hand), and their number is much higher. The decoding problem
for motor BMIs is much harder, both because there is little knowledge
of how the motor cortex encodes information, and because only a
small fraction of the cells is being probed.
2. Brain computer interfaces (BCIs)
There are two basic types of motor BMIs: non-invasive
and invasive. Research on non-invasive BMIs started in
the 1980s by measuring brain electrical activity over the scalp
(electroencephalogram (EEG)). Through training, subjects learn to
control their brain activity in a predetermined fashion that is
classified by a pattern recognition algorithm, and converted into
one of several discrete commands -- usually cursor actions (up/down,
left/right) on a computer display. The computer presents a set of
possibilities to the users, and they choose one of them through
these cursor actions, until a task is completed. This approach,
requiring only signal amplification and classification, is known
as a brain computer interface (BCI).
BCI classification algorithms combine machine learning techniques
with biomedical domain knowledge. There is now an annual competition
to evaluate the progress of these algorithms , where EEG data
sets are made publicly available, together with a performance measure.
Each data set has a labeled and an unlabeled part; contestants submit
estimated labels for the test data, which are then evaluated according
to the given performance measure. At present, the best algorithms
still have high error rates, as high as 20 percent for some tasks.
BCIs require lengthy subject training through biofeedback, and
they display a low bandwidth for effective communication (15 to
25 bits per minute) , hindering the speed at which tasks can
be accomplished, even with the most accurate classification algorithms.
However, BCIs have already been tested with success in paralyzed
patients. Several groups all over the world have demonstrated working
versions of BCIs , and a system software standard has been proposed
3. Control BMIs
In the last decade, emerging developments in microchip design,
signal processing algorithms, computers, sensors, and robotics are
coalescing into a new technology devoted to creating a different
type of BMI, which translates brain activity in a specific area
of the motor cortex into the corresponding movement of some device
in two-dimensional (2D) or three-dimensional (3D) space –
so-called trajectory control BMIs. For example, electrodes
placed in the part of the brain that controls the right hand can
provide real-time control of cursor movements on a computer screen,
as if a mouse is being used. These BMIs can be thought of as intelligent
agents that translate intention of movement from the biological
“wetware” to the firmware of a robotic actuator.
The technical problem with control BMIs cannot be solved by classifying
brain signals into a small set of discrete commands, as in BCIs.
Rather, the algorithms must translate spike firings in real time
into continuous output that represents motion. While the EEG signals
obtained noninvasively do not provide sufficient resolution for
trajectory control, implanted electrodes that directly sense the
brain’s neuronal firings or local field potentials make it
possible. Hence, control BMIs are invasive, probing hundreds of
neurons at once.
Figure 1 shows the architecture of a control BMI.
Figure 1: Schematic
of a control BMI
4. The future
BMIs are still in their
infancy, even sensory BMIs. For instance, the cochlea has more than
100,000 neuronal connections, but with cochlear implants we are
implanting just ten channels to tens of neurons! For most of us,
this would be awful resolution, but it’s amazing for someone
who was incapable of hearing before receiving an implant.
Unlike sensory BMIs, control BMIs are still at the proof-of-concept
stage; recently we have seen the first human BMI implants, after
successful experiments with primates . So far, this work was
driven by neuroscience research, and the role of computer scientists
has been modest. But now, to take BMIs to the next level, issues
of miniaturization of the electronics and algorithm accuracy and
scalability have become crucial. The challenges are many and difficult;
we identify four directions of computing research:
||The development of more accurate
data models that carry more spatio-temporal information from
the spikes in the motor cortex. The signals are non-gaussian
and nonstationary, so they are very difficult to model well
with present algorithms.
||The development of algorithmic paradigms that
scale up well for a variety of movement tasks. So far, control
BMIs have focused on cursor movements, but a mechanical hand
has much more freedom of motion than a mouse.
||Haptic interfaces in robotics. For successful
task completion, it will be necessary to provide feedback to
the user (aside from visual feedback), so the user can “feel”
the objects being touched by the mechanical hand.
||Intelligent system design. Where to place resources for better
overall performance is unknown, but the robot will probably
have partial autonomy, rather than be wholly subjugated to the
user’s control. For instance, if the mechanical hand is
close to a glass of water, the robotic interface may have to
grab and lift the glass autonomously.
Although the theoretical and technical problems are exceedingly
difficult, motor BMI research is at a very exciting phase, thanks
to the tight integration of research in computer science, engineering,
and neuroscience. There is optimism about impacting the daily lives
of paraplegics in the same way that sensory BMIs benefited hearing
There is also hope that, someday, field potentials collected at
the scalp will provide sufficient spatio-temporal resolution to
construct trajectory control BMIs noninvasively. The convenience
of less invasive BMIs would increase their applicability beyond
the restoration of lost movement in paraplegics, and would enable
normal individuals to have direct brain control of external devices
in their daily lives.
The technological explosion through the centuries is proof that
human society continuously seeks more sophisticated tools and faster
and more powerful forms of communication. BMIs are the enabling
medium that allows humans to extend the expression of their intent
far beyond what is provided by simple body motion or speech. As
mobile communications, personal computing, and the Internet become
more integrated into our homes, workplaces, and transportation,
it is foreseeable that we will naturally seek out BMIs that enable
seamless direct brain control of intelligent agents embedded in
our technology devices. Therefore, the impact of BMIs on our society
promises to surpass that of any earlier digital technology.
Created: Mar 11 2005
Last updated: Mar 11 2005
||"The problem of restoring
amputees to function, as well as the problem of restoring
to function people who have lost one or more of their
senses, [is] interesting ... as a method of exploration
[of] ... the nervo-muscular sensory reflex loop…
We are very far from doing it in a stable, viable way."
Wiener (1894-1964), writing in Introduction to
The Computational NeuroEngineering
Laboratory (CNEL): a lab at the University of Florida that
conducts research on adaptive information processing systems.
Lab: a group at Brown University that looks at how the brain
turns thought into action. The scientists at the lab are also
working on building prosthetic devices that provide an interface
between the brain and the external world for paralyzed people.
Laboratory of Miguel
A.L. Nicolelis: a group at Duke University Medical Center
that investigates computational principles underlying the dynamics
between cortical and subcortical neurons mediating tactile perception.
Transactions on Biomedical Engineering 51,
6 (2004). (Special issue on BCIs and BMIs)
Jonietz, E. Picking
your brain. Technology Review 107,
9 (2004), 74-75.
Moore, M.M.; Dua, U. A
galvanic skin response interface for people with severe motor
disabilities. In Proc. of the ACM SIGACCESS Conference
on Computers and Accessibility (ASSETS ’04) (Atlanta,
GA, Oct. 18-20, 2004), pp. 48-54.
Nicolelis, M.A.L.; Chapin, J.K. Controlling
robots with the mind. Scientific American October
Schalk, G.; McFarland, D.; Hinterberger, T.; Birbaumer, N.;
Wolpaw, J. BCI2000:
a general purpose brain computer interface. IEEE Transactions
on Biomedical Engineering 51, 6 (2004),
Schwartz, A.B.; Taylor, D.M.; Helms Tillery S.I. Extraction
algorithms for cortical control of arm prosthetics. Current
Opinion in Neurobiology 11, 6 (2001),
Wessberg, J.; Stambaugh, C.R.; Kralik, J.D.; Beck, P.D.; Laubach,
M.; Chapin, J.K.; Kim, J.; Biggs, S.J.; Srinivasan, M.A.; Nicolelis,
prediction of hand trajectory by ensembles of cortical neurons
in primates. Nature 408 (2000),
Wolpaw, J.R.; Birbaumer, N.; McFarland, D.J.; Pfurtscheller,
G.; Vaughan, T.M. Brain
computer interfaces for communication and control. Clinical
Neurophysiology 113 (2002), 767-791.
for neural ensemble recordings. Nicolelis M.A.L., 1998.
networks: a comprehensive foundation (2nd ed.). Haykin
exploring the neural code. Rieke F., Warland, D., de
Ruyter van Steveninck, R., Bialek, W., 1999.
Third BCI Competition: a competition for fostering research
interest in advanced signal processing and classification methods.
Processing Systems (NIPS) Conference 2004: an annual conference,
sponsored by the NIPS Foundation, that focuses on the biological,
technological, mathematical, and theoretical aspects of neural
information processing systems.
The 2nd International
IEEE/EMBS Conference on Neural Engineering: an upcoming
conference sponsored by the IEEE Engineering in Medicine and
Biology Society (EMBS) that will focus on the neural engineering
field, covering such areas as restoring lost sensory and motor
abilities, neurorobotics, and neuroelectronics.
A galvanic skin response interface for people with
severe motor disabilities. Moore
M., Dua U. ASSETS '04: 48-54, 2004
Human factors issues in the neural signals direct
brain-computer interfaces Moore M., Kennedy P. ASSETS '00: