Electroencephalographic (Eeg) Control of Threedimensional Movement

NIH Public Access
Author Manuscript
J Neural Eng. Author manuscript; available in PMC 2011 June 1.
Published in final edited form as:
NIH-PA Author Manuscript
J Neural Eng. 2010 June ; 7(3): 036007. doi:10.1088/1741-2560/7/3/036007.
ELECTROENCEPHALOGRAPHIC (EEG) CONTROL OF THREE-

DIMENSIONAL MOVEMENT
Dennis J. McFarland1, William A. Sarnacki1, and Jonathan R. Wolpaw1

1Laboratory of Neural Injury and Repair, Wadsworth Center, New York State Department of
Health, Albany, New York 12201-0509
Abstract
Brain-computer interfaces (BCIs) can use brain signals from the scalp (EEG), the cortical surface
(ECoG), or within the cortex to restore movement control to people who are paralyzed. Like
muscle-based skills, BCI use requires activity-dependent adaptations in the brain that maintain
stable relationships between the person’s intent and the signals that convey it. This study shows
that humans can learn over a series of training sessions to use EEG for three-dimensional control.
The responsible EEG features are focused topographically on the scalp and spectrally in specific
frequency bands. People acquire simultaneous control of three independent signals (one for each
dimension) and reach targets in a virtual three-dimensional space. Such BCI control in humans has
not been reported previously. The results suggest that with further development noninvasive EEG-
based BCIs might control the complex movements of robotic arms or neuroprostheses.
1. Introduction
The adult CNS displays a large repertoire of adaptive behaviors acquired through practice,
usually referred to as skills. These skills are normally produced by muscles. In contrast,
brain-computer interfaces (BCIs) enable people to communicate or to control devices by
using brain signals rather than muscles. Thus, they can help people with devastating
neuromuscular disorders such as amyotrophic lateral sclerosis (ALS), brainstem stroke,
cerebral palsy, and spinal cord injury (Wolpaw and Birbaumer, 2006).
Studies to date show that humans and animals can learn to use electroencephalographic
activity (EEG) recorded from the scalp, electrocorticographic activity (ECoG) recorded from
the cortical surface, or signals recorded within the cortex (neuronal action potentials or local
field potentials (LFPs)) to control the movements of a cursor or other device in one or two
dimensions (Wolpaw et al., 1991, 2002; Chapin et al., 1999; Fetz, 1999; Serruya et al., 2002;
Taylor et al., 2002; Carmena et al., 2003; Andersen et al., 2004; Wolpaw and McFarland,
1994, 2004; Leuthardt et al., 2004; Hochberg et al., 2006; Schalk et al., 2008; Velliste et al.,
2008; Ganguly and Carmena, 2009). Three-dimensional (3-D) control has been reported
only for intracortical signals (i.e., neuronal action potentials) in monkeys (Taylor et al.,
2002; Velliste et al., 2008).
Both actual movement and movement imagery are accompanied by changes in the
amplitudes of certain EEG rhythms, specifically 8–12-Hz mu rhythms and 18–30-Hz beta
rhythms. These changes are focused over sensorimotor cortex (Pfurtscheller et al, 2008) in a
Proofs and correspondence: Dennis J. McFarland, Ph.D., Wadsworth Center, New York State Dept. of Health, P.O. Box 509, Empire
State Plaza, Albany, New York 12201-0509 USA, Voice: 518-473-4680, 518-486-2677, 518-473-3631, Fax: 518-486-4910,
[email protected].
McFarland et al. Page 2
manner consistent with the homuncular organization of this cortical region (Woolsey, 1958).
Thus, in our earlier demonstrations that people could learn to use EEG for two-dimensional
(2-D) movement control, we began the training process by using for control the mu- and/or
beta-rhythms changes normally associated with left-hand or right-hand movement imagery

(Wolpaw and McFarland, 1994 and 2004). In the present study, we extended this strategy to
3-D control by beginning the training process from the mu- and/or beta-rhythm changes
normally associated with left-hand, right-hand, or foot movement imagery (Morash et al,
2008; Pfurtscheller et al, 2008). The results show that EEG can support 3-D movement
control. Thus, they may accelerate the development of BCIs useful to people with severe
motor disabilities.
2. Methods
The methodology is summarized here. Additional detail is available elsewhere (Wolpaw and
McFarland, 2004; McFarland et al., 2006). The BCI users were four adults, one woman and
three men, aged 29–59. One man (User 2) had a spinal cord injury (T7) and was confined to
a wheelchair. Three had participated in earlier BCI studies (e.g., McFarland et al., 2008),
and one had no previous BCI experience. All gave informed consent for the study, which
was reviewed and approved by the New York State Department of Health Institutional
Review Board.
The BCI user sat in a reclining chair (or in his own wheelchair) facing a video screen and
remained motionless. BCI operation and data collection were supported by the general-
purpose BCI software platform BCI2000 (Schalk et al., 2004) in conjunction with a 64-
channel SA Instrumentation amplifier and a Data Translation DT-3003 64 channel A/D
board. EEG was recorded from 64 scalp locations (Sharbrough et al., 1991) by 9-mm tin
electrodes embedded in a cap (Electro-cap International) and referenced to an electrode on
the right ear, and was digitized at 160 Hz and stored for later analysis. Each user completed
2–3 sessions/wk. A session comprised eight 3-min runs separated by 1-min breaks and each
run averaged 14–25 trials.
For the first 1–4 sessions (i.e., 8–32 3-min runs totaling 24–96 min), the users practiced 1-D
control in each dimension of movement (i.e., vertical, horizontal, and depth dimensions; 2–3
runs/session for each dimension). For the next 10–12 sessions (i.e., 80–96 runs totaling 4–5
hr), they practiced 2-D control with the three possible pairings of the three dimensions (i.e.,
2–3 runs/session with each pairing). Finally, the users moved on to full 3-D control, and
each completed 21–42 sessions (i.e., 168–336 runs totaling 8–17 hr) on the standard 3-D
task (Fig. 1). Performance on the 3-D task improved steadily as each user gradually gained
better control over the EEG features that controlled cursor movement, and as the iterative
adaptive feature selection and weighting procedures (see below) progressively modified the
set of EEG features used and adjusted their weights so as to vest control of cursor movement
in those features that the user was best able to control. Training was continued until the
progressive improvement over sessions was no longer clearly apparent (i.e., until
performance began to asymptote).
To control each dimension of cursor movement (horizontal, vertical, and depth), the
digitized data from three or four electrodes over sensorimotor cortex of both hemispheres
were re-referenced according to a large Laplacian transform (McFarland et al., 1997). Every
50 ms, the frequency spectrum of the previous 400-ms segment from each electrode was
computed by a 16th-order autoregressive algorithm (McFarland and Wolpaw, 2008). The
logarithms of the amplitudes in specific 3-Hz-wide frequency bands (center frequencies 10–
31 Hz) were the EEG features that controlled cursor movements. One or more of these
features comprised the control signal (i.e., the independent variable) in a linear equation that

specified cursor movement in a particular dimension (McFarland et al., 2006). That is, if ΔV
was vertical cursor movement, Sv was the control signal for vertical movement, bv was the
gain, and av was the mean value of Sv for the user’s previous performance,
(1)
was the function that determined each vertical cursor movement. Similarly, if ΔH was the
horizontal cursor movement,
(2)
was the function that determined horizontal cursor movement. Finally, if ΔD was the cursor
movement in depth,
(3)
was the function that determined cursor movement in depth. Movements in each dimension
occurred 20 times/s.
Initial feature selection was based on a preliminary screening in which the user imagined
specific limb movements (Wolpaw and McFarland, 1994). The user was asked to imagine
left-hand, right-hand, or foot movement. The features selected initially and at each of the
periodic reevaluations came from electrodes located over sensorimotor cortex. Feature
selection was then periodically updated between sessions by a stepwise multiple regression
procedure (SAS Institute Inc) (Wolpaw and McFarland, 2004). Starting with no initial
model terms, the feature that most reduced the residual variance (i.e., the variance not
accounted for by target location), and did so with p<0.01, was added to the model.
Additional features were then added in the same fashion. After each new addition, a
backward stepwise regression removed any variables for which p was >0.01. This process
continued until no further features satisfied the addition/removal criteria.
The weights assigned to the selected features were determined by least-squares criteria
according to the equation:
(4)
where X was an m by n matrix formed from the n observations of m features and Y was the
vector of n values (i.e., target locations). Solving for b, the vector of feature weights, gave:
(5)
Following each of these periodic stepwise regression analyses, the features selected for each
of the three linear equations were weighted according to its results. Then, after each trial, the
weights assigned to these features were automatically adjusted by the LMS algorithm
(Haykin, 1996) so as to optimize for each dimension the correlation between target location
and cursor position. This continual automatic adaptation used past performance to optimize
the feature weights (Wolpaw and McFarland, 2004).
For the online update of feature weights, a prediction error was computed for each control
signal at the end of each trial:

(6)
where pt is the predicted target position on the current trial based on the current values of the
features and ot is the actual target position on the dimension in question. Then the weights
were updated according to:
(7)
where wit is the weight at the end of the current trial for the ith feature and r is a constant
that determines the rate of adaptation.
The objective of this feature selection and weighting process was to minimize, for each
dimension, the squared difference between the actual target position and the cursor position
predicted by the EEG control signals. Both the stepwise feature selection and online
adaptive algorithms used these same criteria. In summary, to the extent that past
performance predicted future performance, these procedures served to optimize for each
dimension the correlation between target location and cursor movement (Wolpaw and
McFarland, 2004).
3. Results
Table 1 shows, for each user’s final training sessions, the set of EEG features (specific
frequency bands from specific scalp electrodes) that comprised the independent variable for
each of the three linear equations that controlled the three movement dimensions. These
feature sets were the result of the iterative adaptive interactions over the course of training
between each user’s control capacities and the feature selection and weighting procedures
described In Methods.
As previously noted, each user completed 21–42 3-D sessions, and 3-D control gradually
improved over these sessions. Figure 2 shows this gradual improvement (as percent of trials
completed within 7 sec) for each user. The data from the first 21 sessions, for which there
were data from all four users, were evaluated by ANOVA. The effect of sessions was
significant (F= 3.84, p<0.0001), and thus confirmed that performance improved with
continued practice. Performance improved as each user gradually gained better control over
the EEG features that controlled cursor movement, and as the iterative adaptive feature
selection and weighting procedures (see Methods) periodically modified the set of EEG
features used and adjusted their weights so as to vest control of cursor movement in those
features that the user could best control. Figure 3 illustrates these progressive changes in
user control and in the features used for control with data from one user at early, middle, and
late stages of training.
Using three consecutive 3-D sessions at the end of training (336–608 trials from each user),
we assessed EEG control and the cursor movement control that it provided. We assessed
EEG control by spectral and topographical analyses of the correlations of the average values
for each trial of the vertical, horizontal, and depth control signals with the vertical,
horizontal, and depth locations of the target (Wolpaw and McFarland, 1994 and 2004).
Table 2 gives, for each user, the correlation of each dimension’s control signal with each of
the three dimensions of target position. As Table 2 shows, each control signal correlated
strongly with its own dimension of target location and showed no or much lower
correlations with the other two signals’ dimensions of target location. Thus, users developed
three independent control signals: one for each movement dimension.

Across the four users, performance did not correlate with amount of 3-D training. User 1,
who achieved the best control (Table 2), had the least number of 3-D training sessions (21
sessions, or 8.4 hrs). User 4, who achieved the least control, had 29 3-D sessions (11.6 hrs),
while Users 2 and 3 had 26 and 42 (10.4 and 16.8 hrs), respectively. (User 2 was the person
with a spinal cord injury.) Nor did performance correlate clearly with total amount of BCI
training (i.e., including participation in previous studies). Users 1–4 had total BCI
experience of 102, 367, 115, and 57 hrs., respectively. While the least successful user had
the least experience, the most successful user had less experience than Users 2 or 3. The
differences in performance evident in Table 2 may reflect inter-user differences in the
prominence of sensorimotor rhythms in the scalp-recorded EEG (i.e., differences in signal-
to-noise ratio) and/or in the effectiveness of the interactive adaptations in the algorithm and
the brain over the course of training. Further improvements of the user-specific adaptations
in the feature extraction and translation procedures might enable more users to achieve good
performance.
To determine whether the users were controlling the three dimensions simultaneously or
sequentially (e.g., moving up, then right, then forward), we evaluated for the first 0.5 s of
each trial the individual movements in each of the three dimensions (which occurred every
50 ms) to determine whether a correct movement (i.e., toward the target) in one dimension
affected the probability that the simultaneous movements in the other two dimensions were
correct. If the users were controlling the dimensions sequentially (i.e., one at a time), the
probability that a correct movement in one dimension was accompanied by correct

movements in the other two dimensions would be less than that predicted by simply
multiplying the fractions of all vertical, horizontal, and depth movements that were correct
together. This was not the case: the actual probabilities of simultaneous correct movements
were 145%, 164%, 151%, and 181% of their expected values for Users 1–4, respectively.
The fact that these probabilities were greater than 100% of expectation indicates that the
users were not simply controlling one dimension at a time; they were controlling the three
dimensions simultaneously.
Figure 4 shows, for each user, the topographies (nose at top) of the correlations for each of
the 64 electrodes between the spectral amplitude of the EEG and each dimension of target
location. For each dimension of target location, the topography is for the frequency band
that made the largest contribution to that dimension's online control signal. “X” indicates the
locations of the electrodes that provided the frequency-band amplitudes that were used
online. Table 1 gives the frequencies of the topographies. While the correlations are all
focused over sensorimotor cortex, they differ markedly across users as a result of the
iterative adaptive interactions during training between the brain and the feature selection and
weighting procedures.
Figure 5 shows User 1’s control in more detail. Figure 5A shows the topographies of the
correlations of 26-Hz activity (Table 1) with the three dimensions, with the electrode(s) that
controlled each dimension marked. Figure 5B shows the spectra for the correlations (as R2)
between activity at the electrode that made the largest (or only) contribution to the control
signal for that dimension and the target location in that dimension. Each feature correlates
strongly with its appropriate dimension of target location and not with the other dimensions.
Figure 5C shows single EEG traces from electrodes used in the vertical, horizontal, and
depth control signals for trials in which the target was at the top or bottom, right or left, or
back or front of the cube, respectively (Fig. 1). They illustrate the strong EEG feature
control the user employed to move the cursor to the target.
The EEG control summarized in Table 2 and illustrated in Figures 4 and 5 gave each user 3-
D movement control. Users 1–4 reached the target within the time allowed in 93%, 78%,

76%, and 56% of the trials, respectively, and their median movement times for these
completed trials were 1.6, 2.9, 3.2, and 4.9 s, respectively. Furthermore, the first of the eight
possible target locations reached by the cursor correlated strongly with the actual target
location (P<0.0001 by χ2 for each user), indicating that movement was not random.
Supplementary Videos 1 and 3 show the average trajectory to each target for Users 1 and 3,
respectively. Supplementary Videos 2 and 4 show real-time performance for Users 1 and 3,
respectively. While both users have 3-D control (i.e., Table 2), User 1’s performance is
clearly superior.
An ancillary study started the cursor from locations other than the center of the cube (i.e.,
near the corner opposite the target). In spite of the variable starting points and the greater
distance to the target, the percent of targets reached did not differ significantly from the
percent reached with the standard (center start) format (p>0.25 by ANOVA).
We also evaluated forearm and calf EMG activity during performance. EMG activity was
usually well below 10% of MVC throughout and uncorrelated with target location. In the
few instances in which EMG did correlate with target location in a particular dimension, the
correlation of the EEG control signal with target location remained significant after the
effect of the EMG correlation was removed. The results confirmed previous data (e.g.,
Vaughan et al., 1998; Wolpaw and McFarland, 2004) indicating that EEG-based cursor
control does not depend on covert contractions of muscle groups strongly represented in the
sensorimotor cortex areas that produce the EEG features used for control.
4. Discussion
The results show that people can learn to use scalp-recorded EEG activity to control three
movement dimensions simultaneously and independently (Table 2). This control develops
through training as the user gradually acquires better control of the EEG features that control
movement, and as the BCI system gradually focuses on those features that the user can best
control. Thus, 3-D control is basically a skill (i.e., an ability acquired through practice
(Brown, 1993)) that user and system master together. Users did not have greater difficulty
acquiring control of the second or third dimension than they did acquiring control of the first
dimension. They did need further practice to control three dimensions simultaneously. As
training proceeded and performance improved, users reported that the motor imagery they
initially employed became less important and cursor control became more automatic. In this
characteristic, the skill of EEG-based 3-D movement control resembles conventional motor
skills, in which training leads to automaticity and to performance that is less dependent on
attention (Moors and de Houwer, 2006).
Given this understanding of 3-D control as a skill acquired through practice, appropriate
modifications in the training protocol might be expected to facilitate learning and improve
final performance. For example, Table 2 indicates that several users were less successful in
controlling cursor movement in the depth dimension than in the horizontal and vertical
dimensions. This difference could be due in part to the fact that the depth dimension was
less salient in the display that we used (i.e., Fig. 1, Videos 2 and 4). Thus, adding binocular
cues for depth (e.g., through a stereoscopic display) might facilitate the user’s acquisition of
depth control.
This demonstration of EEG-based 3-D movement control has practical implications. It

suggests that, with further development, EEG-based BCI systems might prove capable of
restoring movement control to people who are paralyzed.

The factors important for movement control

The factors that determine the control possible with EEG, ECoG, and intracortical BCI
methods are as yet unknown. Many efforts to develop BCI multidimensional movement
control have started from the assumption that high-resolution signals are essential, and thus
they have used intracortical electrodes to record neuronal action potentials or local field
potentials (Serruya et al., 2002; Taylor et al., 2002; Carmena et al., 2003, Ganguly and
Carmena, 2009; Andersen et al., 2004; Hochberg et al., 2006; Velliste et al., 2008).
Intracortical methods provide the highest resolution signals. At the same time, they face
difficulties in ensuring stable long-term function (Otto et al., 2006), and the information
available from neurons may have inherent limitations (Rokni et al., 2007). Furthermore, by
demonstrating EEG-based 3-D control in humans, the present study adds to recent evidence
suggesting that signal resolution may not be the critical limiting factor.
Figure 6 shows the distributions of target-acquisition times for two studies of center-out 2-D
control in humans, one using a cortical neuron-based BCI (Hochberg et al., 2006) and one
using an EEG-based BCI (Wolpaw and McFarland, 2004). The figure also includes the
distribution of times for conventional muscle-based joystick control. The two BCIs studies
had similar protocols, and they yielded nearly identical distributions of target-acquisition
times. Both are slower and far less consistent than joystick control. Even after the
considerable BCI training provided in each study, performance remains inconsistent. Such
inconsistency is typical of BCI studies (e.g., compare supplementary videos 1 and 8 of
Hochberg et al. (2006)), including the present one. Indeed, the most striking feature of the
comparison in Figure 6 is the close similarity of the two BCI distributions. This similarity is
remarkable, given that one BCI used single-neuron activity recorded within the cortex while
the other used EEG recorded from the scalp. It suggests that their inconsistency was not
related to signal resolution (which was high for the neuronal BCI and low for the EEG BCI),
but rather reflects another factor that similarly limits both high-resolution and low-resolution
BCI methods.
Movement control has been traditionally viewed as highly localized (e.g., Woolsey, 1958).
However, recent work indicates that movements are controlled by distributed cortical
networks that include not only primary motor cortex, but other areas as well (e.g., premotor,
prefrontal) (Dum and Strick, 2002 and 2005; Aflalo and Graziano, 2006; Meier et al., 2008;
Ledberg et al., 2006). These networks appear to function through synchronous oscillations in
their constituent parts (Bullmore and Sporns, 2009; Salinas and Sejnowski, 2001; Sejnowski
and Paulsen, 2006; Zhang et al., 2008). This new understanding suggests that present-day
BCI movement control may be limited and inconsistent (e.g., Fig. 6) in large part because it
relies only on signals from a single cortical area. Neuron-based control has typically focused
on neurons from a few cubic mm of cortex, and has begun by using the neuronal activity
observed during actual movements (e.g., Andersen et al., 2004; Hochberg et al, 2006;
Velliste et al., 2008). Similarly, EEG-based control has focused on rhythms recorded over
sensorimotor cortex, and has begun by using the rhythm changes observed during movement
imagery (e.g., McFarland et al., 2008; Wolpaw and McFarland, 1994 and 2004).
This new understanding of the highly distributed nature of motor control suggests that the
performance of BCIs, whether they use EEG, ECoG, or intracortical signals, might possibly
be improved by extracting signal features from multiple cortical areas and using adaptive
algorithms similar to that of the present study to combine them to control movements.
Employing signals from multiple areas might allow BCI operation to more closely mimic
normal neuromuscular movement control. We speculate that, by eliminating the limit that
may be imposed on BCIs that use only one area, this approach might allow the control
capacities of the different signal types to be more fully realized and could produce more

reliable performance. Furthermore, identification of the combinations of areas that perform

best could illuminate the interactions most critical for movement control.
Development of BCIs to restore movement control

The primary goal of BCI research is to restore communication and control to people with
severe motor disabilities. This study advances the goal in two ways. First, its demonstration
of EEG-based 3-D control suggests that BCIs capable of controlling neuroprostheses,
robotic arms, or comparable devices might not require surgical implantation of recording
electrodes. This work complements and extends the recent demonstration that an EEG-based
BCI can control a sequential movement-selection (i.e., reach- and- grasp) action (McFarland
et al., 2008).
Second, the present demonstration that EEG can support 3-D movement control should
facilitate evaluation of the hypothesis that consistency can be improved by using signals
from multiple cortical areas. The inconsistency of the movement control provided by current
BCIs, regardless of which signals they use, is probably the single greatest impediment to
their practical use. Unless and until BCI-based movement control becomes consistent, it will
remain a laboratory curiosity, with little value to people who need to operate
neuroprostheses or robotic devices in their daily lives. Because EEG is noninvasive, BCI
research studies can readily record EEG from multiple areas and explore the usefulness of
their various combinations. Furthermore, by doing this they might also suggest promising
combinations of areas for exploration with high-resolution invasive methods. By

demonstrating the potential of EEG-based BCIs, the present results show that EEG is a valid
avenue for studying the value of combining signals from different areas, and they thereby
encourage such studies.
Efforts to realize the clinical potential of EEG-based movement control must also address
other important issues. These include the achievement of continuous sequential control, such
as the ability to move to a location, to stay there while performing another action, to
immediately move to another location, etc. Also important for practical applications will be
development of improved training methods and better adaptive algorithms to enable most
users to attain, and maintain, good control after relatively brief training. As hypothesized
above, algorithms that use features from additional brain areas and/or measures of cross-
channel relationships might possibly provide more reliable control (Varela et al., 2001).
Such expanded algorithms will need to incorporate better methods for identifying the most
useful signal features from among a large feature set on the basis of relatively limited bodies
of data (i.e., better regularization methods). In real-time BCI applications, the data most
useful for selecting signal features are those from very recent performance, and these data
are necessarily limited in amount. With such limited data, an algorithm that lacks effective
regularization tends to overfit the data from a user’s recent performance and thus provides
parameter values that are not appropriate for future performance (i.e., they do not generalize
well). While the present study used stepwise regression methods to select features,
alternative procedures (e.g., Tibshirani, 1996; Farquhar, 2009; van Gerven, 2009) might
prove more effective.
5. Conclusions
By demonstrating that an EEG-based BCI can support 3-D movement, this study shows that
high signal resolution is not essential for complex movement control and suggests that other
factors are more important. We hypothesize that combinations of signals from multiple
cortical areas might produce more consistent performance. With further development, it may
eventually become possible for people with severe neuromuscular disorders to operate

devices such as a robotic arm, a motorized wheelchair, or a neuroprosthesis with brain

signals recorded from the scalp.
Supplementary Material
Refer to Web version on PubMed Central for supplementary material.
Acknowledgments
This work was supported by grants from NIH (HD30146 (NCMRR, NICHD) and EB00856 (NIBIB & NINDS))
and the James S. McDonnell Foundation. We thank Theresa M. Vaughan and Gerwin Schalk for valuable advice
throughout this work and Jonathan S. Carp, Eric W. Sellers, Elizabeth Winter Wolpaw, Chadwick B. J. Boulay,
Brandon LaPallo, Scott Brainard, and Peter Brunner for their comments on the manuscript.
References
Aflalo TN, Graziano MS. Possible origins of the complex topographic organization of motor cortex:
reduction of a multidimensional space onto a two-dimensional array. Journal of Neuroscience
2006;26:6288–6297. [PubMed: 16763036]
Andersen RA, Musallam S, Pesaran B. Selecting signals for a brain-machine interface. Curr. Opin.
Neurobio 2004;14:720–726.
Brown, L., editor. The New Shorter Oxford English Dictionary. Vol. 2. Oxford: Clarenden Press;
1993. p. 2882
Bullmore E, Sporns O. Complex brain networks: graph theoretical analysis of structural and functional
systems. Nature Reviews: Neuroscience 2009;10:186–198.
Carmena JM, Lebedev MA, Crist RE, O’Doherty JE, Santucci DM, Dimitrov DF, Patil PG, Henriquez
CS, Nicolelis MAL. Learning to control a brain-machine interface for reaching and grasping by
primates. PloS Biol 2003;1:1–16.
Chapin JK, Moxon KA, Markowitz RS, Nicolelis MAL. Real-time control of a robot arm using
simultaneously recorded neurons in the motor cortex. Nat. Neurosci 1999;2:664–670. [PubMed:
10404201]
Dum RP, Strick PL. Motor areas in the frontal lobe of the primate. Physiology and Behavior
2002;77:677–682. [PubMed: 12527018]
Dum RP, Strick PL. Frontal lobe inputs to the digit representations of the motor areas on the lateral
surface of the hemisphere. Journal of Neuroscience 2005;25:1375–1386. [PubMed: 15703391]
Farquhar J. A linear feature space for simultaneously learning of spatio-spectral filters in BCI. Neural
Networks 2009;22:1278–1285. [PubMed: 19619982]
Fetz EE. Real-time control of a robotic arm. Nat. Neurosci 1999;2:583–584. [PubMed: 10404169]
Ganguly K, Carmena JM. Emergence of a stable cortical map for neuroprosthetic control. PLoS
Biology 2009;7:e1000153. [PubMed: 19621062]
Haykin, S. Adaptive Filter Theory. Upper Saddle River, NJ: Prentice-Hall; 1996.
Hochberg LR, Serruya MD, Friehs GM, Mukand JA, Saleh M, Caplan AH, Branner A, Penn DRD,
Donoghue JP. Neuronal ensemble control of prosthetic devices by a human with tetraplegia.
Nature 2006;442:164–171. [PubMed: 16838014]
Ledberg A, Bressler SL, Ding M, Coppola R, Nakamura R. Large-scale visuomotor integration in the
cerebral cortex. Cerebral Cortex 2007;17:44–62. [PubMed: 16452643]
Leuthardt EC, Schalk G, Wolpaw JR, Ojemann JG, Moran DW. A brain-computer interface using
electrocorticographic signals in humans. J. Neural. Eng 2004;1:63–71. [PubMed: 15876624]
McFarland DJ, Krusienski DJ, Sarnacki WA, Wolpaw JR. Emulation of computer mouse control with
a noninvasive brain-computer interface. J. Neural Eng 2008;5:101–110. [PubMed: 18367779]
McFarland DJ, Krusienski DJ, Wolpaw JR. Brain-computer interface signal processing at the
Wadsworth Center: mu and sensorimotor beta rhythms. Prog. Brain Res 2006;159:411–419.
[PubMed: 17071245]

McFarland DJ, McCane LM, David SV, Wolpaw JR. Spatial filter selection for EEG-based
communication. EEG Clin Neurophysiol 1997;103:386–394.
McFarland DJ, Wolpaw JR. Sensorimotor rhythm-based brain-computer interface (BCI): model order
selection for autoregressive spectral analysis. J. Neural Eng 2008;5:155–162. [PubMed:

18430974]
Meier JD, Afalo TN, Kastner S, Graziano MS. Complex organization of human primary motor cortex:
a high-resolution fMRI study. Journal of Neurophysiology 2008;100:1800–1812. [PubMed:
18684903]
Moors A, De Houwer J. Automaticity: A theoritical and conceptual analysis. Psychol. Bull
2006;132:297–302. [PubMed: 16536645]
Morash V, Bai O, Furlani S, Lin P, Hallett M. Classifying EEG signals preceding right hand,m left
hand, tongue, and right foot movements and motor imagery. Clin. Neurophysiol 2008;119:2570–
2578. [PubMed: 18845473]
Otto KJ, Johnson MD, Kipke DR. Voltage pulses change neural interface properties and improve unit
recordings with chronically implanted microelectrodes. IEEE Trans. Biomed. Eng 2008;53:333–
340. [PubMed: 16485763]
Pfurtscheller G, Scherer R, Muller-Putz GR, Lopes da Silva FH. Short-lived brain state after cued
motor imagery in naive subjects. Eur J Neurosci 2008;28:1419–1426. [PubMed: 18973568]
Rokni U, Richardson AG, Bizzi E, Seung HS. Motor learning with unstable neural representations.
Neuron 2007;54:653–666. [PubMed: 17521576]
Salinas E, Sejnowski TJ. Correlated neuronal activity and the flow of neural information. Nature
Reviews: Neuroscience 2001;2:539–550.

Schalk G, McFarland DJ, Hinterberger T, Birbaumer N, Wolpaw JR. BCI2000: A general-purpose
Brain-Computer interface (BCI) system. IEEE Trans. Biomed. Eng 2004;51:1034–1043.
[PubMed: 15188875]
Schalk G, Miller KJ, Anderson NR, Wilson JA, Smyth MD, Ojemann JG, Moran DW, Wolpaw JR,
Leuthardt EC. Two-dimensional movement control using electrocorticographic signals in humans.
J. Neural Eng 2008;5:75–84. [PubMed: 18310813]
Sejnowski TJ, Paulsen O. Network oscillations: emerging computational principals. Journal of
Neuroscience 2006;26:1673–1776. [PubMed: 16467514]
Serruya MD, Hatsopoulos NG, Paminski L, Fellows MR, Donoghue JP. Instant neural control of a
movement signal. Nature 2002;416:141–142. [PubMed: 11894084]
Sharbrough F, Chatrian CE, Lesser RP, Luders H, Nuwer M, Picton TW. American
Electroencephalographic Society guidelines for standard electrode position nomenclature. J. Clin.
Neurophysiol 1991;8:200–202. [PubMed: 2050819]
Taylor DA, Helms Tillery S, Schwartz AB. Direct cortical control of 3D neuroprosthetic devices.
Science 2002;296:1829–1832. [PubMed: 12052948]
Tibshirani R. Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical
Society, series B 1996;58:267–288.
van Gerven M, Hesse C, Jensen O, Heskes T. Interpreting single trial data using groupwise
regularization. Neuroimage 2009;46:665–676. [PubMed: 19285139]
Varela F, Lachaux J-P, Rodriguez E, Martinerie J. The brainweb: Phase synchronization and large-
scale integration. Nature Reviews: Neuroscience 2001;2:229–239.
Vaughan TM, Miner LA, McFarland DJ, Wolpaw JR. EEG-based communication: analysis of
concurrent EMG activity. Electroencephalogr. Clin. Neurophysiol 1998;107:428–433. [PubMed:
9922089]
Velliste M, Perel S, Spalding MC, Whitford AS, Schwartz AB. Cortical control of a prosthetic arm for
self-feeding. Nature 2008;453:1098–1101. [PubMed: 18509337]
Wolpaw, JR.; Birbaumer, N. Brain-Computer Interfaces for Communication and Control. In: Selzer,
ME.; Cohen, L.; Gage, FH.; Clarke, S.; Duncan, PW., editors. Textbook for Neural Repair and
Repair and Rehabilitation. Cambridge: Cambridge Univ. Press; 2006. p. 602-614.
Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain-computer interfaces
for communication and control. Clin. Neurophysiol 2002;113:767–791. [PubMed: 12048038]

Wolpaw JR, McFarland DJ. Multichannel EEG-based brain-computer communication.

Electroencephalogr. Clin. Neurophysiol 1994;90:444–449. [PubMed: 7515787]
Wolpaw JR, McFarland DJ, Neat GW, Forneris CA. Electroencephalogr. Clin. Neurophysiol
1991;78:252–259. [PubMed: 1707798]

Wolpaw JR, McFarland DJ. Control of a two-dimensional movement signal by a noninvasive brain-
computer interface in humans. Proc. Natl. Acad. Sci 2004;101:17849–17854. [PubMed:
15585584]
Wolpaw JR, McFarland DJ, Neat GW, Forneris CA. An EEG-based brain-computer interface for
cursor control. Electroencephalogr. Clin. Neurophysiol 1991;78:252–259. [PubMed: 1707798]
Woolsey, CN. Organization of somatic sensory and motor areas of the cerebral cortex. In: Harlow,
HF.; Woolsey, CN., editors. Biological and Biochemical Basis of Behavior. Madison: University
of Wisconsin Press; 1958. p. 63-81.
Zhang Y, Wang X, Bressler SL, Chen Y, Ding M. Prestimulus cortical activity is correlated with speed
of visuomotor processing. Journal of Cognitive Neuroscience 2008;20:1915–1925. [PubMed:
18370597]

Figure 1.
The 3-D movement control format. The large screen image on the left shows the virtual 3-D
cube with the eight possible targets in the corners and the cursor in the center. The smaller
screen images show the sequence of steps in one trial: (1) a target appears; (2) 1 s later the
cursor appears and moves in three dimensions controlled by the user’s EEG activity as
described in the text; (3) the cursor reaches the target; (4) the target turns yellow for 1.5 s;
(5) the screen is blank for 1 s and then the next trial begins. (Step 2 lasts up to 15 s. If the
cursor does not reach the target in this time, the screen goes blank for 1.5 s prior to step 5.)

Figure 2.
Percent of trials completed (i.e., target reached within 15 sec) for each user as a function of
sessions. User A is represented by the blue line, user B by the black line, user C by the
green, and user D by the red. Note each user’s gradual improvement over sessions.

Figure 3.
Topographies for User 1 at the beginning (sessions 1–3), middle (sessions 10–12), and end
(sessions 19–21) of 3-D training, for the correlations at each of the 64 electrodes between
the spectral amplitude of the EEG and each dimension of target location. For each
dimension of target location, the topography is for the 3-Hz frequency band centered at 26
Hz that provided that dimension's online control signal (i.e., Table 1). (The correlations are
shown as R rather than R2 to distinguish negative and positive correlations.) “X” indicates
the locations of the electrodes that provided the frequency-band amplitudes that were used
online. Note the changes over time in the topographies and magnitudes of control and in the
electrodes used for control. The progressive improvement in performance summarized in
Figure 2 is accounted for by the increases in the user’s control in the horizontal and depth
dimensions, together with the adaptive algorithm’s modifications in the electrodes used for
control in the vertical and depth dimensions.

Figure 4.
Topographies (nose at top) for each user (1–4) of the correlations for each of the 64
electrodes between the spectral amplitude of the EEG and each dimension of target location.
For each dimension of target location, the topography is for the frequency band that made
the largest contribution to that dimension's online control signal. (The correlations are shown
as R rather than R2 to distinguish negative and positive correlations.) “X” indicates the
locations of the electrodes that provided the frequency-band amplitudes that were used
online. The center frequencies of the 3-Hz frequency bands of each user’s topographies are
given in Table 1. While the correlations are all focused over sensorimotor cortex, they differ
markedly across users as a result of inter-user differences in the course of the iterative

adaptive interaction between user and system that occurs during training (see Methods). For
example, User 1 controlled the three movement dimensions with 26-Hz activity from three
different scalp electrodes, while User 3 controlled vertical movement with the left-right
difference in 10-Hz activity, horizontal movement with 19- and 31-Hz activity at the vertex,
and depth movement with 10-Hz activity on the left.

Figure 5.
Topographical and spectral properties of EEG control for User 1. In this user, movement in
each dimension was controlled by 26-Hz activity from specific scalp electrodes (Table 1).
A: Scalp topographies (nose at top) of the correlations of the 26-Hz frequency band with the
vertical, horizontal, and depth target locations, respectively. The electrode(s) that controlled
each dimension of movement are marked. (The correlations are shown as R rather than R2 in
order to distinguish negative and positive correlations.)
B: Spectra for the correlations (shown as R2) of the activity at the scalp electrode that made
the largest (or only) contribution to the control signal for each dimension of cursor
movement with the three dimensions of target location. The correlations with the vertical,

horizontal, and depth dimensions are red, blue, and black lines, respectively. It is clear that
activity at the electrode that provided each control signal correlated strongly with its
appropriate dimension of target location and did not correlate with the other dimensions.
Furthermore, the correlation was focused in the appropriate (i.e., in this case, 26-Hz)
frequency band.
C: Samples of EEG activity from single trials. The traces are single 400-msec epochs of
Laplacian-derived EEG from one electrode. On the left are traces from scalp electrode CPz
(the major source of the vertical control signal) for trials in which the target was at the top or
bottom of the cube. In the middle are traces from electrode C4 (the source of the horizontal
control signal) for trials in which the target was on the right or left side of the cube. On the
right are traces from electrode C3 (the source of the depth control signal) for trials in which
the target was at the front or back of the cube. They illustrate the strong 26-Hz control that
the user employed to move the cursor to the target.

Figure 6.
Distributions of target-acquisition times (i.e., time from target appearance to target hit) on a
2-D center-out cursor-movement task for joystick control (black), EEG-based BCI control
(blue), and cortical neuron-based BCI control (red). The EEG-based and neuron-based BCIs
perform similarly, and both are slower than and much less consistent than the joystick. For
both BCIs in a substantial number of trials, the target is not reached even in the 7 s allowed.
Such inconsistent performance is typical of movement control by present-day BCIs,

regardless of what brain signals they use. (The joystick data and neuron-based BCI data are
from Hochberg et al. (2006). The EEG-based BCI data are from Wolpaw and McFarland
(2004).)

Table 1
The EEG features (specific frequency bands at specific scalp electrodes) that controlled vertical, horizontal,
and depth cursor movements for each user. For each feature, the scalp electrode (Sharbrough et al., 1991) and
the center frequency of the 3-Hz wide frequency band (in parenthesis) are given.
ELECTRODE (FREQ)
USER
Vert Horiz Depth
1 Cz(26) C4(26) C3(26)

CPz(26)
2 Cz(24) C3(12) C4(12)

C4(12)
3 Cz(19) C3(10) C3(10)

Cz(31) C4(10)
4 Cz(20) C3(23) C3(20)

CPz(20) C4(23) C3(23)
CPz(29)

NIH-PA Author Manuscript NIH-PA Author Manuscript NIH-PA Author Manuscript
Table 2
Correlations (as R2) of the vertical, horizontal, and depth control signals (SV, SH, SD) with their appropriate (highlighted) and inappropriate dimensions of
target location (V, H, D). Each control signal is correlated exclusively or most strongly with its own dimension of target location.
Horizontal Signal
Vertical Signal (SV) Depth Signal (SD)
(SH)
McFarland et al.
User
SV-V SV-H SV-D SH-H SH-V SH-D SD-D SD-V SD-H
1 0.39 0.01 0.00 0.50 0.00 0.00 0.57 0.00 0.00

2 0.29 0.01 0.14 0.37 0.00 0.16 0.16 0.00 0.00
3 0.37 0.03 0.28 0.47 0.09 0.04 0.15 0.04 0.01
4 0.20 0.00 0.03 0.09 0.00 0.01 0.09 0.00 0.00

Page 21

Electroencephalographic (Eeg) Control of Threedimensional Movement

Uploaded by

Copyright:

Available Formats

Electroencephalographic (Eeg) Control of Threedimensional Movement

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Electroencephalographic (Eeg) Control of Threedimensional Movement

Uploaded by

Copyright:

Available Formats

NIH Public Access

J Neural Eng. 2010 June ; 7(3): 036007. doi:10.1088/1741-2560/7/3/036007.

ELECTROENCEPHALOGRAPHIC (EEG) CONTROL OF THREE-

Dennis J. McFarland1, William A. Sarnacki1, and Jonathan R. Wolpaw1

beta-rhythms changes normally associated with left-hand or right-hand movement imagery

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

probability that a correct movement in one dimension was accompanied by correct

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

This demonstration of EEG-based 3-D movement control has practical implications. It

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

The factors important for movement control

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

reliable performance. Furthermore, identification of the combinations of areas that perform

Development of BCIs to restore movement control

combinations of areas for exploration with high-resolution invasive methods. By

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

devices such as a robotic arm, a motorized wheelchair, or a neuroprosthesis with brain

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

selection for autoregressive spectral analysis. J. Neural Eng 2008;5:155–162. [PubMed:

Reviews: Neuroscience 2001;2:539–550.

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

Wolpaw JR, McFarland DJ. Multichannel EEG-based brain-computer communication.

1991;78:252–259. [PubMed: 1707798]

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

Such inconsistent performance is typical of movement control by present-day BCIs,

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

1 Cz(26) C4(26) C3(26)

2 Cz(24) C3(12) C4(12)

3 Cz(19) C3(10) C3(10)

4 Cz(20) C3(23) C3(20)

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

1 0.39 0.01 0.00 0.50 0.00 0.00 0.57 0.00 0.00

J Neural Eng. Author manuscript; available in PMC 2011 June 1.

You might also like