Invited review

Brain–computer interfaces for communication and control

Jonathan R. Wolpaw a,b, *, Niels Birbaumer c,d
, Dennis J. McFarland a
,
Gert Pfurtscheller e
, Theresa M. Vaughan a
aLaboratory of Nervous System Disorders, Wadsworth Center, New York State
Department of Health, P.O. Box 509, Empire State Plaza,
Albany, NY 12201-0509, USA
bState University of New York, Albany, NY, USA
cInstitute of Medical Psychology and Behavioral Neurobiology, University of
Tuebingen, Tuebingen, Germany
dDepartment of Psychophysiology, University of Padova, Padova, Italy
eDepartment of Medical Informatics, Institute of Biomedical Engineering, Technical
University of Graz, Graz, Austria
Accepted 2 March 2002
Abstract
For many years people have speculated that electroencephalographic activity or
other electrophysiological measures of brain function
might provide a new non-muscular channel for sending messages and commands to the
external world – a brain–computer interface (BCI).
Over the past 15 years, productive BCI research programs have arisen. Encouraged by
new understanding of brain function, by the advent of
powerful low-cost computer equipment, and by growing recognition of the needs and
potentials of people with disabilities, these programs
concentrate on developing new augmentative communication and control technology for
those with severe neuromuscular disorders, such as
amyotrophic lateral sclerosis, brainstem stroke, and spinal cord injury. The
immediate goal is to provide these users, who may be completely
paralyzed, or ‘locked in’, with basic communication capabilities so that they can
express their wishes to caregivers or even operate word
processing programs or neuroprostheses. Present-day BCIs determine the intent of
the user from a variety of different electrophysiological
signals. These signals include slow cortical potentials, P300 potentials, and mu or
beta rhythms recorded from the scalp, and cortical neuronal
activity recorded by implanted electrodes. They are translated in real-time into
commands that operate a computer display or other device.
Successful operation requires that the user encode commands in these signals and
that the BCI derive the commands from the signals. Thus,
the user and the BCI system need to adapt to each other both initially and
continually so as to ensure stable performance. Current BCIs have
maximum information transfer rates up to 10–25 bits/min. This limited capacity can
be valuable for people whose severe disabilities prevent
them from using conventional augmentative communication methods. At the same time,
many possible applications of BCI technology, such
as neuroprosthesis control, may require higher information transfer rates. Future
progress will depend on: recognition that BCI research and
development is an interdisciplinary problem, involving neurobiology, psychology,
engineering, mathematics, and computer science; identi-
fication of those signals, whether evoked potentials, spontaneous rhythms, or
neuronal firing rates, that users are best able to control
independent of activity in conventional motor output pathways; development of
training methods for helping users to gain and maintain
that control; delineation of the best algorithms for translating these signals into
device commands; attention to the identification and
elimination of artifacts such as electromyographic and electro-oculographic
activity; adoption of precise and objective procedures for
evaluating BCI performance; recognition of the need for long-term as well as short-
term assessment of BCI performance; identification
of appropriate BCI applications and appropriate matching of applications and users;
and attention to factors that affect user acceptance of
augmentative technology, including ease of use, cosmesis, and provision of those
communication and control capacities that are most
important to the user. Development of BCI technology will also benefit from greater
emphasis on peer-reviewed research publications and
avoidance of the hyperbolic and often misleading media attention that tends to
generate unrealistic expectations in the public and skepticism
in other researchers. With adequate recognition and effective engagement of all
these issues, BCI systems could eventually provide an
important new communication and control option for those with motor disabilities
and might also give those without disabilities a supple-
mentary control channel or a control channel useful in special circumstances. q
2002 Elsevier Science Ireland Ltd. All rights reserved.
Keywords: Brain–computer interface; Electroencephalography; Augmentative
communication; Rehabilitation; Neuroprosthesis; Brain–machine interface
Clinical Neurophysiology 113 (2002) 767–791
1388-2457/02/$ - see front matter q 2002 Elsevier Science Ireland Ltd. All rights
reserved.
PII: S 1388-245 7(02)00057-3
www.elsevier.com/locate/clinph
CLINPH 2001764
* Corresponding author. Tel.: 11-518-473-3631; fax: 11-518-486-4910.
E-mail address: [email protected] (J.R. Wolpaw).
1. Introduction
1.1. Options for restoring function to those with motor
disabilities
Many different disorders can disrupt the neuromuscular
channels through which the brain communicates with and
controls its external environment. Amyotrophic lateral
sclerosis (ALS), brainstem stroke, brain or spinal cord
injury, cerebral palsy, muscular dystrophies, multiple
sclerosis, and numerous other diseases impair the neural
pathways that control muscles or impair the muscles them-
selves. They affect nearly two million people in the United
States alone, and far more around the world (Ficke, 1991;
NABMRR, 1992; Murray and Lopez, 1996; Carter, 1997).
Those most severely affected may lose all voluntary muscle
control, including eye movements and respiration, and may
be completely locked in to their bodies, unable to commu-
nicate in any way. Modern life-support technology can
allow most individuals, even those who are locked-in, to
live long lives, so that the personal, social, and economic
burdens of their disabilities are prolonged and severe.
In the absence of methods for repairing the damage done
by these disorders, there are 3 options for restoring function.
The first is to increase the capabilities of remaining pathways.
Muscles that remain under voluntary control can substitute
for paralyzed muscles. People largely paralyzed by massive
brainstem lesions can often use eye movements to answer
questions, give simple commands, or even operate a word
processing program; and severely dysarthric patients can use
hand movements to produce synthetic speech (e.g. Damper et
al., 1987; LaCourse and Hladik, 1990; Chen et al., 1999;
Kubota et al., 2000). The second option is to restore function
by detouring around breaks in the neural pathways that
control muscles. In patients with spinal cord injury, electro-
myographic (EMG) activity from muscles above the level of
the lesion can control direct electrical stimulation of paral-
yzed muscles, and thereby restore useful movement (Hoffer
et al., 1996; Kilgore et al., 1997; Ferguson et al., 1999).
The final option for restoring function to those with motor
impairments is to provide the brain with a new, non-muscular
communication and control channel, a direct brain–computer
interface (BCI) for conveying messages and commands to
the external world. A variety of methods for monitoring brain
activity might serve as a BCI. These include, besides electro-
encephalography (EEG) and more invasive electrophysiolo-
gical methods, magnetoencephalography (MEG), positron
emission tomography (PET), functional magnetic resonance
imaging (fMRI), and optical imaging. However, MEG, PET,
fMRI, and optical imaging are still technically demanding
and expensive. Furthermore, PET, fMRI, and optical
imaging, which depend on blood flow, have long time
constants and thus are less amenable to rapid communica-
tion. At present, only EEG and related methods, which have
relatively short time constants, can function in most environ-
ments, and require relatively simple and inexpensive equip-
ment, offer the possibility of a new non-muscular
communication and control channel, a practical BCI.
1.2. The fourth application of the EEG
In the 7 decades since Hans Berger’s original paper
(Berger, 1929), the EEG has been used mainly to evaluate
neurological disorders in the clinic and to investigate brain
function in the laboratory; and a few studies have explored
its therapeutic possibilities (e.g. Travis et al., 1975; Kuhl-
man, 1978; Elbert et al., 1980; Rockstroh et al., 1989; Rice
et al., 1993; Sterman, 2000). Over this time, people have
also speculated that the EEG could have a fourth applica-
tion, that it could be used to decipher thoughts, or intent, so
that a person could communicate with others or control
devices directly by means of brain activity, without using
the normal channels of peripheral nerves and muscles. This
idea has appeared often in popular fiction and fantasy (such
as the movie ‘Firefox’ in which an airplane is controlled in
part by the pilot’s EEG (Thomas, 1977)). However, EEG-
based communication attracted little serious scientific atten-
tion until recently, for at least 3 reasons.
First, while the EEG reflects brain activity, so that a
person’s intent could in theory be detected in it, the resolu-
tion and reliability of the information detectable in the spon-
taneous EEG is limited by the vast number of electrically
active neuronal elements, the complex electrical and spatial
geometry of the brain and head, and the disconcerting trial-
to-trial variability of brain function. The possibility of
recognizing a single message or command amidst this
complexity, distortion, and variability appeared to be extre-
mely remote. Second, EEG-based communication requires
the capacity to analyze the EEG in real-time, and until
recently the requisite technology either did not exist or
was extremely expensive. Third, there was in the past little
interest in the limited communication capacity that a first-
generation EEG-based BCI was likely to offer.
Recent scientific, technological, and societal events have
changed this situation. First, basic and clinical research has
yielded detailed knowledge of the signals that comprise the
EEG. For the major EEG rhythms and for a variety of
evoked potentials, their sites and mechanisms of origin
and their relationships with specific aspects of brain func-
tion, are no longer wholly obscure. Numerous studies have
demonstrated correlations between EEG signals and actual
or imagined movements and between EEG signals and
mental tasks (e.g. Keirn and Aunon, 1990; Lang et al.,
1996; Pfurtscheller et al., 1997; Anderson et al., 1998;
Altenmu ̈ller and Gerloff, 1999; McFarland et al., 2000a).
Thus, researchers are in a much better position to consider
which EEG signals might be used for communication and
control, and how they might best be used. Second, the extre-
mely rapid and continuing development of inexpensive
computer hardware and software supports sophisticated
online analyses of multichannel EEG. This digital revolu-
tion has also led to appreciation of the fact that simple
J.R. Wolpaw et al. / Clinical Neurophysiology 113 (2002) 767–791768