Classification of EEG Signals in a Brain-

Computer Interface System

Erik Andreas Larsen

Master of Science in Computer Science

Submission date: June 2011
Supervisor: Alf Inge Wang, IDI
Co-supervisor: Jørn Hokland, IDI

Norwegian University of Science and Technology

Department of Computer and Information Science
 Problem Description

This assignment should include both a theoretical and a practical part. In the
theoretical part, current Brain-Computer Interface systems and methods should be
researched, to create a state-of-the-art overview of, and to learn about the domain.
The focus should be on the emergent commercialization of EEG products on the
public market. In this project, a NeuroSky mindset (EEG equipment) is available
as a test device.
The practical part should implement a system consisting of three modules. The
first module should communicate with the NeuroSky mindset and get EEG signals,
the second module should process those signals and make input parameters for the
third module, which should be a game that have one or several features controlled
by the input parameters. The goal is 1) that users should be able to control a game
using their minds, and 2) investigate if low price EEG equipment enables that.
Assignment given: 17. January 2011
Supervisor: Alf Inge Wang, IDI
Co-supervisor: Jørn Hokland, IDI

i
ii
 Abstract

Electroencephalography (EEG) equipment are becoming more available on the

public market, which enables more diverse research in a currently narrow field.
The Brain-Computer Interface (BCI) community recognize the need for systems
that makes BCI more user-friendly, real-time, manageable and suited for people
that are not forced to use them, like clinical patients, and those who are disabled.
Thus, this project is an effort to seek such improvements, having a newly available
market product to experiment with: a single channel brain wave reader. However,
it is important to stress that this shift in BCI, from patients to healthy and ordinary
users, should ultimately be beneficial for those who really need it, indeed.
The main focus have been building a system which enables usage of the available
EEG device, and making a prototype that incorporates all parts of a functioning
BCI system. These parts are 1) acquiring the EEG signal 2) process and classify the
EEG signal and 3) use the signal classification to control a feature in a game. The
solution method in the project uses the NeuroSky mindset for part 1, the Fourier
transform and an artificial neural network for classifying brain wave patterns in
part 2, and a game of Snake uses the classification results to control the character
in part 3.
This report outlines the step-by-step implementation and testing for this system,
and the result is a functional prototype that can use user EEG to control the snake
in the game with over 90% accuracy. Two mental tasks have been used to separate
between turning the snake left or right, baseline (thinking nothing in particular)
and mental counting. The solution differentiates from other appliances of the Neu-
roSky mindset that it does not require any pre-training for the user, and it is only
partially real-time.

iii
iv
 Preface

This report is the final product to complete my Master of Science degree at the
Norwegian University of Science and Technology (NTNU). The project was con-
ducted in the period of January to June 2011, at the Department of Computer and
Information Science.
I would like to thank my supervisor, associate professor Alf Inge Wang, for valu-
able discussions and supporting follow-up throughout the project lifespan and for
providing me with the necessary equipment. Also, thanks to my co-supervisor,
associate professor Jørn Hokland for giving me this opportunity and for being un-
derstanding.
Finally, I would like to thank my fellow students Kjetil Aamodt and Kjetil Valle for
their general support, lectures on neural networks and for sharing their thoughts
when I asked questions, even though they were just as clueless to the answers as
me. And of course Bjørg Ane Sandve for being an enthusiastic test guinea pig.
Trondheim, June 2011,
Erik Andreas Larsen

v
vi
 Contents
1 Introduction : The Big Picture 1
1.1 Project Purpose: Why Take BCIs to a new Level? . . . . . . . . . . . 1
1.2 Project Strategy: How NeuroSky Could Level Up BCIs . . . . . . . . 2
1.3 Project Results: What a NeuroSky BCI System Could Become . . . . 2
1.4 Report Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Research Questions and Methods 5

2.1 Previous Work - How This Project Came Into Being . . . . . . . . . . 5
2.2 Project Focus Based on Experience and new Expectations . . . . . . 6

3 EEG Technology: Why and How it Works 9

3.1 EEG Background and Why Brain Activity Can Be Measured . . . . . 9
3.2 Identifying Different Brain Activity Patterns . . . . . . . . . . . . . . 10
3.3 Practical Application of EEG Technology . . . . . . . . . . . . . . . . 11

4 Brain-Computer Interfaces 15
4.1 What is a Brain-Computer Interface? . . . . . . . . . . . . . . . . . . 15
4.2 State of the Art BCI and Existing Systems . . . . . . . . . . . . . . . 16
4.3 BCI Development: EEG Analysis, FFT and Band Powers . . . . . . . 19
4.4 Five Different Approaches for Classifying EEG Signals . . . . . . . . 20

5 Solution Design: System Implementation 23

5.1 System Overview, Environment and Components . . . . . . . . . . . 23
5.2 Implementation Steps Overview and Test Procedure . . . . . . . . . 26

6 Step One: EEG Eye-Blink Classification 27

6.1 Method, System Setup and Neural Network Architecture . . . . . . 27
6.2 Test Results and Findings . . . . . . . . . . . . . . . . . . . . . . . . . 29
6.3 Discussion of Results and the Sensitive Electrode . . . . . . . . . . . 33

7 Step Two: EEG Mental Task Classification 35

7.1 Method, System Setup and Neural Network Architecture . . . . . . 35
7.2 Test Results and Findings . . . . . . . . . . . . . . . . . . . . . . . . . 37
7.3 Discussion of Results and How the Eye State is the Key . . . . . . . . 40

8 Step Three: Controlling a Game with EEG 43

8.1 Final System, Game Design and Test Conditions . . . . . . . . . . . . 43
8.2 Final Testing, Results and Findings . . . . . . . . . . . . . . . . . . . 46
8.3 Discussion of Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

9 Conclusion 51
9.1 Further Development and Ideas for the Future . . . . . . . . . . . . . 52

References 55

vii
viii
 List of Figures
1.1 The golden circle model (Sinek, 2009). . . . . . . . . . . . . . . . . . 1
1.2 Feedback using a blue bar graph. . . . . . . . . . . . . . . . . . . . . 2

2.1 System design of previous work - Brain Wave Controlled Playstation 5

2.2 Traditional EEG electrode cap vs. new type of headset . . . . . . . . 6

3.1 Artistic illustration of a single neuron and its synapses. . . . . . . . 9

3.2 The 1020 System - Standardized placement of electrodes on scalp. . 10
3.3 Pure gamma wave signal example . . . . . . . . . . . . . . . . . . . . 10
3.4 Pure beta wave signal example . . . . . . . . . . . . . . . . . . . . . . 10
3.5 Pure alpha wave signal example . . . . . . . . . . . . . . . . . . . . . 10
3.6 Pure theta wave signal example . . . . . . . . . . . . . . . . . . . . . 11
3.7 Pure delta wave signal example . . . . . . . . . . . . . . . . . . . . . 11
3.8 The 5 main frequency bands and their relation to each other. . . . . 11
3.9 Basic functional brain map (Universe-review.ca, n.d.). . . . . . . . . 12
3.10 The P300 EEG wave. . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.11 Neurofeedback principle. MR scanning is of the author’s own head. 14

4.1 Brain-Computer Interface process flow. . . . . . . . . . . . . . . . . . 15

4.2 What the EPOC neuroheadset looks like. . . . . . . . . . . . . . . . . 17
4.3 This is a neural network . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.4 Inside the artificial neuron in a feed-forward propatation network . 21

5.1 System design - Component overview . . . . . . . . . . . . . . . . . . 23

5.2 Screenshot of the graphical interface of BrainMonitor . . . . . . . . 25

6.1 Graphical sampling screen used to collect raw EEG samples . . . . . 28

6.2 Network architecture 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 28
6.3 Spatial baseline sample 1 . . . . . . . . . . . . . . . . . . . . . . . . . 29
6.4 Frequency spectrum of baseline sample 1 . . . . . . . . . . . . . . . 30
6.5 Spatial blink sample 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 30
6.6 Frequency spectrum of blink sample 1 . . . . . . . . . . . . . . . . . 30
6.7 Spatial blink samples . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
6.8 Plotting of attention and meditation levels during scenario 1 . . . . 32
6.9 Spatial sample of noise : Tapping on the forehead . . . . . . . . . . . 32
6.10 Frequency spectrum of noise : Tapping on the forehead . . . . . . . 32

7.1 Second graphical sampling screen used to collect raw EEG samples . 36
7.2 Comparing total band power from all 5 sets in session one . . . . . . 37
7.3 Comparing total band power from all 5 sets in the session two . . . 37
7.4 Spatial sample of baseline - Eyes closed . . . . . . . . . . . . . . . . . 39
7.5 Frequency spectrum of baseline - Eyes closed . . . . . . . . . . . . . 39
7.6 Spatial sample of visual count . . . . . . . . . . . . . . . . . . . . . . 39
7.7 Frequency spectrum of visual count sample . . . . . . . . . . . . . . 39

ix
8.1 Overview of final system flow . . . . . . . . . . . . . . . . . . . . . . 43
8.2 State machine of the game action control . . . . . . . . . . . . . . . . 44
8.3 Screenshot from a Snake game . . . . . . . . . . . . . . . . . . . . . . 46
8.4 Results from control testing, subject one . . . . . . . . . . . . . . . . 48
8.5 Results from control testing, subject two . . . . . . . . . . . . . . . . 48
8.6 Picture from testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

x
 List of Tables
2.1 Superior research questions . . . . . . . . . . . . . . . . . . . . . . . 7

5.1 NeuroSky output protocol . . . . . . . . . . . . . . . . . . . . . . . . 24

6.1 Band powers of baseline. Eyes open. . . . . . . . . . . . . . . . . . . 31

6.2 Band powers of blink . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
6.3 Classification result from neural network . . . . . . . . . . . . . . . 31
6.4 Single baseline sample (Scaled with respect to highest frequency) . . 34
6.5 Sum of band powers of 20 baseline samples . . . . . . . . . . . . . . 34
6.6 Sample set scaled to fit as neural network input . . . . . . . . . . . . 34

7.1 Classification result 1 from neural network . . . . . . . . . . . . . . 38

7.2 Band powers of baseline. Eyes closed. . . . . . . . . . . . . . . . . . . 38
7.3 Band powers of mental counting. Eyes open. . . . . . . . . . . . . . . 38
7.4 Classification result 2 from neural network . . . . . . . . . . . . . . 38

8.1 Test results game 1, subject one . . . . . . . . . . . . . . . . . . . . . 46

8.2 Test results game 2, subject one . . . . . . . . . . . . . . . . . . . . . 47
8.3 Test results game 1, subject two . . . . . . . . . . . . . . . . . . . . . 47
8.4 Test results game 2, subject two . . . . . . . . . . . . . . . . . . . . . 47

xi
xii
 1 Introduction : The Big Picture

This chapter introduces the project by elaborating its why, how and what, illus-
trated in figure 1.1. It presents the project motivation and context. Finally, an
overview of the report structure is stated.

Project Purpose: Why Take BCIs to a new Level?

1.1
The purpose of this project is to investigate and explore the
possibilities that lies within the domain of Brain-Computer
Interfaces, using consumer friendly equipment that have
recently become available on the public market. The field
of Brain-Computer Interfaces (BCI) is a driving force for
utilizing electroencephalography technology (EEG), which
is the process of recording brain activity from the scalp us-
ing electrodes (See chapter 3). In the past, the main fo-
cus have been about developing applications in a medical
context, helping paralyzed or disabled patients to interact Figure 1.1: The golden
with the external world (Plass-Oude Bos et al., 2010), by circle model (Sinek,
mapping brain signals to human cognitive and/or sensory- 2009).
motor functions.
However, BCI development is no longer constrained to just patients or for treat-
ment, there is a shift of focus towards people with ordinary health. Especially
gamers are becoming a target group that would likely to be adaptive to use EEG as
a new modality; giving them advantages or new experiences in gameplay. It is not
just treatment in mind, but entertainment also. This shift could benefit patients,
because when EEG technology becomes more available, and the powerful gaming
industry gets involved, they can become the same driver for improvements as they
are for all silicon-based technology: needing, and thus getting, faster processors
and graphic engines so they can create better games.
By taking BCI to the level of entertainment, the motivation for making more user-
friendly, faster, cheaper and public available systems will be totally different and
become of a much higher priority. The targeted group of users are not forced to
utilize BCI systems, and thus needs better reasons for wanting to, other than it is
cool to be able to control your computer with the mind. Current systems do not
meet such standards, which is further elaborated in the next section and chapter 3.
The motivating thought is that approaching this issue from an entertaining point
of view could help getting BCIs to such standards faster.

1
 Project Strategy: How NeuroSky Could Level Up BCIs
1.2
NeuroSky is a company with the slogan ”Bio Sensors for Everyone!”, and in late 2009
they released a EEG device named the NeuroSky mindset at a low cost, aimed at
the consumer market (Neurosky, n.d.). Shipped with software development tools,
people at home can explore the technology, and integrate it with their own design
and systems. What used to be reserved for those who had the big money to pay for
such equipment, are now very much available for anyone who are interested, either
as a user, a developer, or both. This is great news for accelerating BCI research
since more people can contribute. The strategy for this project is to utilize the
NeuroSky mindset technology and do just that. But, the mindset only provides
the brain wave signal, and to interpret and classify it, an artificial neural network
will be used. More on this in chapter 5.
To better understand current BCI, consider an example
from neurofeedback training used as a treatment alterna-
tive for persons with AD/HD. Neurofeedback, also called
biofeedback, is a treatment where EEG equipment is used
to present real-time records of brain activity. This enables
you to train yourself, for instance to become more concen-
trated or relaxed, by altering your brain wave patterns over
time. Feedback about your concentration level is for in- Figure 1.2: Feedback
stance given as a graph, illustrated in figure 1.2. using a blue bar graph.
The subjects task is usually to keep the bar above a certain threshold, indicated
here by the horizontal black line. Typically one training sessions like this last for
about 45-60 minutes and needs to be repeated to get the wanted effect, variat-
ing from 25-50 times depending on the patient (Heinrich, Gevensleven, & Strehl,
2007). This can easily get boring or frustrating for a patient with AD/HD, or any-
one else for that matter. Now it is possible to change that.

Project Results: What a NeuroSky BCI System Could Become

1.3
The goal of this project is to gain knowledge of the two domains, i.e Brain-Computer
Interfaces, especially methods for analyzing brain waves, and the NeuroSky EEG
equipment. From this research, a prototype software application should be imple-
mented that is able to read brain wave input from an EEG device, classify them,
and make them be part of the, or the only, user input to a game.
A simple example scenario is as follows: A user is wearing the NeuroSky mind-
set that forwards brain wave signals to the software application. In order to get
general information about the users brain wave pattern, a series of mental task
scenarios must be executed by the user. This information will then be used to
train a classification system so it can learn to recognize and thus map different
brain patterns to actions. The user can then start a game, and the classification
system will continuously analyze the incoming brain waves and map them to the
appropriate actions and thus control some feature(s) of the running game.

2
1.3.1 Medical Verification Disclaimer

This project is experimental. The primary objective is to achieve a proof of con-

cept by using the available technologies and following the strategy of the project.
Although the medical aspect and clinical value are discussed and argumented for,
it is for motivation only. It is outside the scope of this project to verify its medical
and clinical utility or its plausibility. However, the possibility for the project of
having such value needs to be put into consideration and kept in mind throughout
the duration of this project.

Report Outline
1.4
This section describes the remaining chapters and the report structure. The goals
and objectives of the project have been introduced in this chapter.

Chapter 2 describes the related work and background for this project and iden-
tifies the most important research questions and method of approach.

Chapter 3 explains how the EEG technology works, and the different brain activ-
ity patterns that are the foundation for EEG analysis.

Chapter 4 elaborates the Brain-Computer Interface technology, how far it has de-
veloped, what it is currently used for and a few underlying methods.

Chapter 5 introduces the main system design and equipment that was available
for realizing and implementing it. The different components and their relations to
each other are explained.

Chapter 6 reviews the result of using the implemented system for basic eye-blink
classification. It describes the method used, and analyses the results in a discus-
sion.

Chapter 7 reviews the result of using the implemented system for mental task
classification, which is counting. It describes the method used, and analyses the
results in a discussion.

Chapter 8 reviews the result of using the implemented system for controlling
a game (a classical Snake-game) according to the project goal. It describes the
method used, and analyses the results in a discussion.

Chapter 9 summarize and concludes the project in perspective of the big picture
and the research questions from chapter 2. It also makes suggestions for further
work.

3
4
 2 Research Questions and Methods

This chapter explains the background for this project, and identifies the main re-
search questions and methods to bring clarity and define the projects focus, based
on lessons learned from earlier efforts and new anticipations.

Previous Work - How This Project Came Into Being

2.1
This project builds upon two previous projects, described below. The latter is the
final work of the course TDT4500 - Intelligent Systems Specialization Project, ex-
ecuted autumn 2010, and is the first part of my Master of Science degree.

2.1.1 Playstation Controlled by Brain Waves

Figure 2.1: System Design: component overview. Sensors reads brain activity and sends
signal to the EEG amplifier equipment. The EEG signal is sent to the module, that over-
rides the Playstation controller. Feedback is given on the screen.

The goal of this project was to meet some of the challenges that people with Atten-
tion Deficit and Hyperactive Disorder (AD/HD) have regarding the use of neuro-
feedback as a treatment option (Pointed out in section 1.2). The electroencephalog-

5
raphy laboratory (EEGlab) at the Department of Psychology at NTNU, BUP (out-
patients clinic for children) and St.Olavs Hospital, have a co-operative venture
where neurofeedback is used when there is suspicion of such disorder or learning
disabilities in children. They were interested in exploring a different approach to
the usage of this technology; making neurofeedback training more attractive, re-
ducing the training and equipment expenses and making it more fun and available
to the public. The proposed method was to combine neurofeedback training with
playing video games on a Playstation.
The result was a system with the ability to use clinical approved EEG equipment to
control the acceleration of a racing car in a video game, both on PC and Playstation,
see figure 2.1. In conclusion, this made it possible to consider a more entertain-
ing way of performing neurotherapy and prompted for a field study to map the
potential for further investment (Larsen, 2010).

2.1.2 Helicopter Controlled by Brain Waves

The playstation project builds on work done by Andersen, Juvik, Kjellen, and
Storstein (2009), as part of a multidisciplinary course at NTNU. On initiative from
Egil Tjåland and in cooperation with Stig Hollup at Department of Psychology,
their assignment was to steer a radio controlled helicopter using the principles of
neurofeedback. With their solution they were able to adjust the speed of the rotor
using the level of concentration in a test person, and lift it off the ground. They
developed an application that interfaced EEG software with the radio controller
software of the helicopter.

Project Focus Based on Experience and new Expectations

2.2

Figure 2.2: Traditional cap with a grid of electrodes vs. new type, single electrode headset
(NeuroSky).

A major limitation in the playstation and the helicopter project was that the EEG
signal from the equipment was unavailable as raw and unaltered data. The only
information that was available was a modified number from the accompanying
software that represented the users concentration level, ranging from 1 to 15. Also,

6
the EEG equipment used, from the Russian company Mitsar (Mitsar, 2008), is ex-
pensive. But, the second major limitation is the electrodes needs to be manually
placed in a cap, similar to the ones to the left in figure 2.2, but with only two
electrodes. Conductive gel was need to be used between the electrodes and scalp
surface, and took some time to set up.
The NeuroSky mindset eliminates those limitations by making raw EEG data avail-
able and by using a dry sensor which comes pre-attached to the headset. Those are
major advantages regarding system development and usability, but there is also
drawbacks attached. There is only one electrode, that is statically placed on the
forehead. Observable at the tip of the arm to the right, in figure 2.2. Compar-
ing the traditional cap and the NeuroSky mindset, the traditional cap looks like it
gives a lot more accurate measurements.

2.2.1 Research Questions

Table 2.1 shows the few superior research questions pre to implementing and ex-
perimenting. They are kept open-wide as much details are unknown in advance,
but expected to emerge throughout the project.

Table 2.1: Superior research questions

Tag Description
RQ1: Can one static sensor on the forehead make up for a grid of sensors
placed across the scalp?
RQ2: What are the advantages and limitations of the NeuroSky mindset?
RQ3: What kind of classification of brain waves will it be possible to make
using a neural network? And if possible, how stable is this classification?
RQ4: How satisfying or useful will a resulting and working BCI system
be for the user?
RQ5: How little practice and training time is it possible to get away with
without affecting the performance of the BCI system?

2.2.2 Research Method

The approach will follow an iterative, empirical trial-and-error model. Uncer-

tainty about what that will work with regards to EEG classification, in a real-time
environment makes this reasonable. This also makes it difficulty to break down
the goal into easy-to-follow steps, pre-experimenting. The research questions will
be a priority, but any findings and interesting experiences should be investigated
further and reported. Thus, this research will be an floating process from start to
finish, and background information will be inserted were it is relevant and discov-
ered; not only in chapter 3 and 4. This is done to improve readability.

7
8
3 EEG Technology: Why and How it Works
This chapter explains how EEG technology works, and the different brain activity
patterns that are a foundation for EEG analysis. Followed by some words on the
practical applications of it. First, a quick distinction: EEG is about the brain waves
themselves, and BCI are more the entire system of interpreting these in a computer
fashion.

EEG Background and Why Brain Activity Can Be Measured

3.1

Figure 3.1: Artistic illustration of a single neuron and its synapses.

The brain have always fascinated humans, and particularly a German scientist
named Hans Berger, who discover electroencephalography (EEG) about 80 years
ago. After this, new methods for exploring it have been found and we can catego-
rize them into two main groups. Invasive and non-invasive. An invasive approach
requires physical implants of electrodes in humans or animals, making it possible
to measure single neurons or very local field potentials. A non-invasive approach
makes use of, for instance, magnetic resonance imaging (MRI) and EEG technol-
ogy to make measurements. Both gives different perspectives and enables us to
look inside the brain and to observe what happens (Kropotov, 2009). In EEG,
brain-related electrical potentials are recorded from the scalp. Pairs of conductive
electrodes (see figure 4.2) made of silver, for example, are used to read this elec-
tricity. The difference in voltage between the electrodes are measured, and since
the signal is weak (30-100µV) it has to be amplified. Current occurs when neurons
communicate. The simplest event is called action potential, and is a discharge
caused by fast opening and closing of N a+ and K + ion channels in the neuron
membrane. If the membrane depolarize to some threshold, the neuron will ”fire”.
Tracking these discharges over time reveals the brain activity.

9
Figure 3.2: The 1020 System - Standardized placement of electrodes on scalp for EEG
measurements (Immara, n.d).

Identifying Different Brain Activity Patterns

3.2
Returning to Berger, he found that different electrical frequencies could be linked
to actions and different stages of consciouses. This was done by observing subjects
performing different task, like solving mathematical problems, while recording
their EEG. Figure 3.8 shows the most used frequency bands, and their relations, of
the human brain wave activity.
Gamma waves are in the frequency range of 31Hz and up. It is
Figure 3.3: thought that it reflects the mechanism of consciousness. Beta and
Gamma gamma waves together have been associated with attention, percep-
wave. tion, and cognition (Rangaswamy et al., 2002).
Beta waves are in the frequency range of 12 and 30 Hz, but are of-
ten divided into β1 and β2 to get a more specific range. The waves
Figure 3.4: are small and fast, associated with focused concentration and best
Beta wave. defined in central and frontal areas. When resisting or suppress-
ing movement, or solving a math task, there is an increase of beta
activity (Y. Zhang, Chen, Bressler, & Ding, 2009). In one study by
Rangaswamy et al. (2002), significantly increased beta power was
found in all of the 307 alcohol-dependent subjects, measured across
the whole scalp. This leads to an hyperexcitable state which consump-
tion of alcohol temporarily alleviates.
Alpha waves, ranging from 7.5 to 12 Hz, are slower and associated
with relaxation and disengagement. Thinking of something peaceful
Figure 3.5: with eyes closed should give an increase of alpha activity. Most pro-
Alpha found in the back of the head (o1 and o2, figure 3.2) and in the frontal
wave. lobe. Several studies have found a significantly rise in alpha power af-
ter smoking marijuana (Lukas, Mendelson, & Benedikt, 1995).

10
 Theta waves, ranging from 3.5 to 7.5 Hz, are linked to inefficiency,
daydreaming, and the very lowest waves of theta represent the fine
Figure 3.6: line between being awake or in a sleep state. Theta arises from emo-
Theta
tional stress, especially frustration or disappointment (L. Zhang, He,
wave.
Miao, & Yang, 2005). It has also been associated with access to uncon-
scious material, creative inspiration and deep meditation. High levels
of theta are considered abnormal in adults, and is, for instance, much
related to AD/HD (Heinrich, Gevensleven, & Strehl, 2007).
Delta waves, ranging from 0.5 to 3.5 Hz, are the slowest waves and
occurs when sleeping (Hammond, 2006). If these waves occur in the
Figure 3.7: awake state, it thought to indicate physical defects in the brain. Move-
Delta wave. ment can make artificial delta waves, but with an instant analysis (just
observing raw EEG records), this can be verified or unconfirmed.
MU is associated with motor activities, and is also found in the alpha wave fre-
quency range, but where the maximum amplitude is recorded over motor cortex.
So it basically triggers when there is an actual movement or there is an intent to
move (Bernier, Dawson, Webb, & Murias, 2007).
All these wave-groups occur in different parts of the brain in varying degree.

Figure 3.8: The 5 main frequency bands and their relation to each other.

Practical Application of EEG Technology

3.3
The most used application of EEG is to observe and study records manually, to
search for, or to understand, brain damage and various disorders, like for in-
stance epilepsy (Sundaram, Sadler, Young, & Pillay, 1999). Empirical research and
case studies throughout the decades have led to functional brain maps, see fig-
ure 3.9, that combined with electrodes placed according to the 10-20 system (see

11
figure 3.2) makes activity in these areas observable. Indeed, EEG is a tool used in
hospitals for declaring patients brains to be dead, when no activity is monitored.

Figure 3.9: Basic functional brain map (Universe-review.ca, n.d.).

The study of brain waves and how they relate to different mental states, have led
to number of alternative methods and beliefs on how to manipulate these waves.
For instance, in order to become e.g more relaxed, focused and smarter, you can
buy music that plays in specific hertzes that promise to do just that (Brain Enter-
tainment, n.d.). A well known advice deduced from this is that you should let your
baby listen to Mozart while growing up, having the effects mentioned. Besides this
somewhat regarded pseudoscience, there have been a lot of interesting studies of
mental states and how they are effected, summarized for instance in Braverman
(1990). A area that have been researched in a great amount is the relation of drugs,
medication and EEG. And according to his article, an increase of alpha activity is
found when taking antidepressants, and addictive drugs like morphine, heroin
and marijuana. It has also been identified that drug users often lack a natural
amount of occurring alpha waves, and thus it can explain why they become ad-
dicts. Alcoholics have been found to have an excess of occurring beta waves, and
that this can inhibit their ability to relax. Alcohol research shows that its use in-
creases the amplitude of the slow waves frequencies and decrease the fast waves.
Thus, it suggests that alcohol becomes the quick remedy to become relaxed.

Further, Braverman talk about ”how brain waves symbolizes the various parts of
our consciousness, and that if we get the knowledge and treatment to change them,
we can get closer to get our very balanced brain waves, or happiness”. One way to
get knowledge is to use ERP, described below, and one way to conduct treatment
is a method called neurofeedback, further described in section 3.3.2.

12
3.3.1 Event Related Potentials (ERP)

Exposure to external stimuli, such as a tone or light flash, can generate responses in
the EEG wave, as shown in figure 3.10. Internal stimuli, like skipping an expected
stimulus can also generate a response. In both cases this is called Event Related
Potentials (ERP), or Evoked Potential (EP). What this means exactly is that there
is an observable amplitude peak (potential) that occurs at a definite latency time
after the specific stimuli. The most used is the P300, were the peak comes 300 ms
after the event. In chapter 4 we will see BCI systems applying this. In diagnostic
appliance, however, P300 is used to map the external stimuli to brain activity in
specific areas. For example, if exposed to a tone, there should be a reaction in the
EEG close to the auditory processing area (See figure 3.9).

Figure 3.10: ”Sample-evoked response potential in response to a target (70 dB. 2 kHz) and
nontarget (70 dB, 1 kHz) tone. The P300 wave is clearly seen only after presentation of the
target tone.” (Lukas et al., 1990).

3.3.2 Neurofeedback Training and how it Can be Used to Treat AD/HD

AD/HD is a common diagnostics that EEG helps to detect. Traditionally, an in-

crease of beta activity and a decrease of theta activity is wanted to reduce inatten-
tion (Kropotov et al., 2007). Studies of AD/HD treatment using neurofeedback
have been criticized for being small in scale, and for not providing enough statis-
tical power. But recently, in addition to these small scaled studies, larger studies
have been conducted that concludes in favor of such treatment (Gevensleben et al.,
2009) (Sherlin et al., 2010) (Heinrich et al., 2007) (Masterpasqua & Healey, 2003).
People with AD/HD tend to have more slow waves in general, and when they are
present in the frontal lobe, the forehead, it becomes difficult to manage attention,
behavior and impulse control.
The principle of altering brain waves is shown in figure 3.11, and is an example of
an BCI. Self-regulative abilities to do this comes by getting real-time visual and/or
auditory feedback, and is like operant conditioning. With frequent training, long
termed effect is possible (Hammond, 2006). Electrodes on the scalp picks up the
electrical activity, the signals are weak so they go into an amplifier that boosts them

13
before it is sent to a software on a computer. There the signal gets interpreted and
processed into feedback to the user. Dividing the signal into the frequency bands
can be done by applying a Fourier transform (further explained in section 4.3).
Their factors can then be used to calculate a ratio, for instance Beta/Theta (Leins,
Goth, Hinterberger, Klinger, & Strehl, 2007). This ratio can be used to determine
the hight of a bar graph (Figure 1.2) and displayed on a monitor. Feedback is
obtained.

Figure 3.11: Neurofeedback principle. MR scanning is of the author’s own head. Elec-
trodes are attached to the scalp. The amplifier reads the electricity registered, amplifies
this and issues a analog-to-digital converter which sends EEG data to the computer. The
computer process the data and display it for the patient. The feedback can be used to
verify correct behaviour.

3.3.3 Meditation - Selfconducting ”Neurofeedback” From Within

Not everyone seems to need visual feedback in order to alter their brain waves.
Studies of experienced mediators, using EEG, shows remarkable capabilities to af-
fect their frequency power while meditating, and keep a stable frequency for the
entire time (Stigsby, Rodenberg, & Moth, 1981). In one study, 20 normal peo-
ple showed increased fast Theta and slow Beta frequencies, predominantly in the
frontal area, by applying a Zen meditation task(Takahashia et al., 2005). It re-
quires no previous training because it focuses on breath control. This gives an
interesting perspective on neurofeedback and brain waves in general.

14
 4 Brain-Computer Interfaces

This chapter explains Brain-Computer Interfaces and reviews some of the avail-
able products and systems on the market today. Current methods and potentials
in the technology are identified.

What is a Brain-Computer Interface?

4.1

Wolpaw (2010) put it this way:

Brain-computer interface is a method of communication based on

neural activity generated by the brain and is independent of its normal
output pathways of peripheral nerves and muscles. The goal of BCI is
not to determine a person’s intent by eavesdropping on brain activity,
but rather to provide a new channel of output for the brain that requires
voluntary adaptive control by the user.

Further, it is identified four different application areas of BCI, some which have
been mentioned in chapter 3 already:

1. Bioengineering applications: Devices with assisting purposes for disabled

people.

2. Human subject monitoring: Research and detection of sleep disorders, neu-

rological diseases, attention monitoring, and/or overall ”mental state”.

3. Neuroscience research: real-time methods for correlating observable behav-

ior with recorded neural signals.

4. Human-Machine Interaction: Interface devices between humans, comput-

ers or machines.

In this project we are concerned with item 4, and all steps in the common BCI
process flow shown in figure 4.1.

Figure 4.1: Brain-Computer Interface process flow.

15
4.1.1 Five Central Factors of BCI Design

Lotte, Congedo, Lécuyer, Lamarche, and Arnaldi (2007) have identified a few fea-
tures that are common, and critical, to BCI design, that are listed here:
1. noise and outliers: BCI features are noisy or contain outliers because EEG
signals have a poor signal-to-noise ratio;
2. high dimensionality: in BCI systems, feature vectors are often of high di-
mensionality. Indeed, several features are generally extracted from sev-
eral channels and from several time segments before being concatenated
into a single feature vector;
3. time information: BCI features should contain time information as brain
activity patterns are generally related to specific time variations of EEG;
4. non-stationarity: BCI features are non-stationary since EEG signals may
rapidly vary over time and more especially over sessions;
5. small training sets: the training sets are relatively small, since the training
process is time consuming and demanding for the subjects.
Item 2 is highlighted because the NeuroSky EEG device used for this project is a
single channel device. This reduces complexity for processing, but it also reduces
signal accuracy, because there is only a single source of input. All of the referenced
studies that are presented in the next section used 4 electrodes and most of them
even more.

State of the Art BCI and Existing Systems

4.2
In chapter 1, it is mentioned that there is a focus shift from patients to gamers,
and that this focus could benefit patients also. This is further illustrated here
by using NASA and Sony as an example: NASA have been using neurofeedback
to train their pilots to be more alert and attentive during a flight (NASA patent,
dated 1994). Together with Sony, they used this technology to develop a special-
ized Playstation controller that would react to brain patterns. Correct brain pat-
terns were rewarded with a more responsive controller and vice versa. A study
was conducted were a group of children with ADHD was treated using neuro-
feedback. One half of the group trained using traditional neurofeedback, and the
other half used the Playstation video game. Both groups improved about equally,
but the group playing the video game enjoyed their training more, and their par-
ents generally reported to notice more improvement than the parents from the
other group (Reuderink, 2008a). Treatment can be made more appealing, fun and
still give the wanted results.

4.2.1 Available Products on the Market: Healty People

When describing some of the products on the market it gives a good insight to
the potential that lies in neurofeedback. The NASA and Sony feedback system

16
differs from the traditional neurofeedback in that the feedback and reward are not
explicit on the display, but implicit in the how the subject is able to control a task
using brain waves.
• ”Yt Mer” (Perform More): By expanding the target group, more efforts will
be made to make the technology available, user friendly, entertaining and
affordable. But this is only started. In 2010, the first ”BrainWave Optimiza-
tion” company was established in Norway (Berg, 2010). ”Yt Mer” is an or-
ganization that is offering neurofeedback training to customers so that they
can achieve higher performance, lower stress levels, better sleep patterns and
increased well-being. It is very expensive, thus it is mostly targeting larger
firms and enterprises.
• Emotiv: Their motto is ”You think, therefore you can” (Emotiv, n.d.). Emo-
tiv claims that you can use thoughts, feelings and emotions to control your
computer. The accompanying software comes with a machine learning al-
gorithm that learns how your brain visualizes, for example push and pull of
objects, from the EEG measurements. This can be mapped to computer con-
trols. Their neuroheadset consists of 14 sensors, includes a gyroscope and
is priced 299 US dollars. The headset is high-tech and a comparison with a
traditional EEG hair net is shown in figure 4.2.
However, the Emotiv EPOC Neuroheadset is only available
for purchase to US customers. This is because there is not
enough testing of the headset to achieve CE marking that
makes it possible to sell in Europe and other countries, in-
cluding Norway. Software Development Kits (SDKs) can be
bought however, it includes a beta-headset to use at ones
own risk and software to develop applications for it.

Figure 4.2: What the

EPOC neuroheadset
looks like.
4.2.2 Available Products on the Market: AD/HD Diagnose

• Play Attention: A company named Unique Logic and Technology (UL&T)

develops neurofeedback technology that targets children and people with
AD/HD and general attention problems. They focus on in-home training, us-
ing a computer and special made programs and games. Even though training
is done home, they have specialists and professionals available that ensures
that the system, and training, is correctly set up and executed by the user.
The EEG equipment is a helmet (looks like a bicycle helmet) that is based on
the NASA patent with sensors in it. Their system is not just for home use, it
is used in over 450 schools districts in the US, in hospitals and psychologist’s
offices also.
• SmartBrain: SmartBrain Technologies, in co-operation with NASA, have made
neurofeedback available for use with Playstation and Xbox (SmartBrain, n.d.).
This makes it possible to apply it to thousands of different games, though the
controlling feature is always the same. Their system includes a special made

17
 controller and does not require a computer to operate. The controller be-
comes more responsive when the user is concentrated and is basically the
market product of the NASA study mentioned in section 4.2. Two versions
are for sale: a personal and a professionally supervised. The personal edition
costs approximately 600 US dollars and comes with three training presets. In
general, they reward decrease of the slow brain patterns of alpha and theta
brain waves and increase in faster brain waves (beta). The professionally su-
pervised edition makes it possible to adjust the parameters individually to
fit the user needs.

4.2.3 Available Products on the Market: Other Dysfunction

• Spelling With the Brain: intendiX is a personal EEG-based spelling sys-

tem (G.tec, 2009), utilizing the P300 principle (see section 3.3.1). The user is
wearing a cap with several electrodes and presented with a matrix contain-
ing all the letters of the alphabet. The rows and columns are then flashed
quickly and randomly, and whenever the system flashes a row or column, it
registers your reaction according to P300. This is enough to pin down the
letter than you want to spell, when focusing on it. The system does this at
a fairly high speed, and to quote intendiX: ”This requires some training but
most subjects can use intendiX r after only 10 minutes with a reasonable per-
formance: A spelling rate of 5 to 10 characters per minute can be achieved
by the majority of healthy users at their first trial.” Still, as (Blankertz et al.,
2006) pointed out, it is strainful for the users of evoked potential BCIs, be-
cause their are continuously receiving stimuli. However, it really opens up a
way for patients to communicate, or issue commands on the computer.

• Mind Speech: Kellis et al. (2010) experimented with severely-epileptic pa-

tients that were going to have the seizure-stricken parts of their brain re-
moved. This operation requires that part of the skull is opened, and then
microelectrodes are installed on the surface to help narrow down the area
that needs to be removed. The researchers exploited this by putting micro-
electrodes directly on to the Face-Motor Cortex and Wernicke’s area (high-
lighted in figure 3.9) that are crucial for speech. Scientists matched the right
word, from a string of 10 words, to the corresponding EEG signal between 28
percent and 48 percent of the time, which is better than chance. This could
in the future make it possible for disabled people to communicate, using a
speech synthesizer to read the matched words out loud.

• Mind Cursor: A earlier study by Wolpaw and Mcfarland (1994), uses the
principle of MU (see section 3.2), the amplitude, or power levels for vertical
movement, and the difference between the power levels for horizontal move-
ment. This allowed the user to select from four icons, one at each corner
of the screen. Using MU for movement is common, because it maps well to
intentional movement.

18
4.2.4 Games on the Market and What all the Products Have in Common

The two game examples below is very common properties to apply in BCI games.
As with most games available today, they rely on attention and concentration val-
ues. It seems it is the most reliable or easiest property to calculate, especially when
readings from the forehead is used.
• Adventures of Neuroboy: NeuroSky have their own demo game that comes
with the mindset. You are a character that can move in a world and levitate,
push, pull and burn things with you mind. Attention and meditation values
calculated in the mindset decides how this happen. Increased meditation
value will lift objects, and increased attention will for instance burn things.
• Mindball: You wear a headband that reads brain waves, like the NeuroSky
mindset, from the forehead. Theta and alpha (drowsiness and relaxation)
band powers are used to control the speed of a ball. It is used to move a
ball from A to B in competition with an opponent. The one who is more
concentrated pushed the ball towards the other, and wins.
An example of a game that only uses the NeuroSky attention value is the Mind-
Blaster game (Peek, 2010a). Using a Nintendo Wiimote, targets on the screen are
locked in, and when maximum attention value is reach, the target explodes. Other
projects involve driving a remote controlled toy vehicle. These examples makes it
less interesting to apply the attention value from the mindset directly.
Another thing that these games have in common is that it reflects the brain waves
in almost real-time, so it is like neurofeedback. If you do not concentrate, the car
will not move. So it may require a lot of training to be able to drive smoothly, or
to levitate an object and so forth. This can be demotivating because one never gets
to the play part, but is stuck struggling with brain wave control. One could easily
get bored, or just giving up. Thus, this is a good reason for lowering the need for
training, and make games more easy to play.

BCI Development: EEG Analysis, FFT and Band Powers

4.3
The Fourier Transformation and extraction of band powers is by far the most ap-
plied method for signal processing and analysis (Lotte et al., 2007). The algorithm
is based on discrete Fourier transform (DFT) equation 4.1, and by applying that to
the EEG signal it makes it possible to separate the EEG rhythms (see section 3.2).
Definition of the DFT:
N −1
n
X
Xk = xn e−i2kπ N k = 0, ..., N − 1 (4.1)
n=0

and the inverse of it:

N −1
1 X n
xn = Xk ei2kπ N (4.2)
N
n=0

19
The performance of the DTF is O(N 2 ), but there is a more efficient algorithm called
fast Fourier Transform (FFT), that can compute the same result in only O(N logN ).
This is a great improvement and one of the reasons why FFT is the favorable
method of analyzing EEG signals, and other waves like sound.

Five Different Approaches for Classifying EEG Signals

4.4
There are five categories that covers the most used algorithms in BCI classifica-
tion systems, and they are: linear classifiers, nonlinear Bayesian classifiers, nearest
neighbor classifiers, neural networks, and a combination of classifiers (Lotte et al.,
2007). In all categories there have been achieved good BCI results, except for the
nearest neighbor classifiers, which seems to not handle dimensionality very well.
However, neural networks are the most popular in BCI research, and is the chosen
method to classify in this project also.

4.4.1 Neural Networks: Multilayer Perceptron

Multilayer perceptron means that the neural network consists of an input layer,
possibly and minimum one hidden layer, and one output layer, as shown in fig-
ure 4.3. The output of each node in one layer connects to the input of the next
layer, but not within the same layer.

Figure 4.3: Neural network: 2 input neurons, 3 hidden layer neurons and 2 output neu-
rons. Neurons are also referred to as nodes, and the lines between the nodes are called
synapses.

The inspiration comes from the network of neurons in the human brain, but very
much a simplified version of it. The great strength with neural networks is that
they are flexible and almost universal approximators, giving the right setup: Neu-
ral networks needs to be trained before it can be used. In case of supervised learn-
ing, training sets with sample input and the corresponding output are given to
the network, and then an algorithm adjust the weights of the synapses so that it
can map valid input to correct output. A very common algorithm for this use
is the backpropagation algorithm. In that case, the neurons look like the one in
figure 4.4.

20
Except from the input nodes, all other nodes will have their input determined
by the sum of all the incoming synapses. This becomes input to the activation
function in the nodes (corresponds to the action potential in biological neurons,
see section 3.1), which determine the nodes output: if it ”fires” nor not.

Figure 4.4: Neuron in a feed-forward propagation network. f is the activation function.

4.4.2 Backprogagation Algorithm

To understand neural networks better, and how the backpropagation works, the
algorithm is given in pseudocode below. For more details, please refer to Dario
and Mattiussi (2008) and NeuronDotNet (n.d.-a).

Initialize the weights in the network

while stopping criterion has reached do
for all example e ∈ training set do
O = actual, output(network, e); propagate forward
T = wanted output for e
Calculate error (T - O) at each neuron in the output layer
Compute Mean Squared Error value; propagate backward
Compute deltaweightupdate for all weights
Update all the weights in the network such that the sum-squared value of
error is minimized.
end for
end while

The Mean Squared Error (MSE) value is calculated using equation 4.3. This value
reflects the effectiveness of the training done so far. The stopping criterion could
either be when the MSE has reached an acceptable limit, or when the number of
training cycles is attained.
θ̂ = E[(θ̂(X) − θ)2 ] (4.3)

There are several types of activation function commonly used, these are:
• Sigmoid Activation Function: y = 1/(1 + Exp(−x))
• Linear Activation Function: y = x
• Logarithmic Activation Function: y = Log(1 + |x|)
• Sine Activation Function: y = Sin(x)
• Tanh Activation Function: y = T anh(x)

21
22
5 Solution Design: System Implementation

This chapter describes the overall final system implementation. Some details are
left for chapter 6, 7 and 8, as they are more relevant there.

System Overview, Environment and Components

5.1
Figure 5.1 shows an overview of the main components in the system design. The
system flow is described below, followed by a component specification. The entire
project is written in the Visual Studio 2010 environment, using C# as program-
ming language. A HP tx2020ed laptop with Windows XP SP3 have been used.

Figure 5.1: System Design: component overview. NeuroSky mindset and the computer
program main parts.

Comparing these components with the BCI process flow in figure 4.1, we see that
Mindset, Bluetooth-to-COM, and Mindset Interface, corresponds to ”brain sig-
nal input”; toolbox corresponds to ”EEG signal processing”; Neural Network cor-
responds to ”EEG signal classification” and Game corresponds to ”computer inter-
action”.

23
 NeuroSky mindset: The easy to apply EEG device, that is wear-
able like regular headphones. It has one dry sensor that can be
placed on the forehead, left side (approximately equal to Fp1 in
the 10-20 system, figure 3.2). And 3 dry sensors on the left ear,
for reference. It has a microchip which pre-process the EEG
signal, and transmits that data via bluetooth. The output data
is presented in table 5.1. The processing algorithms are not an
open protocol, but it does a FFT on the signal which gives the
band powers. However, these powers are scaled and filtered
and thus only relative to each other (NeuroSky, 2010). Also,
the mindset have speakers, like a headset, and a microphone,
so it can be used for multiple tasks.

Table 5.1: NeuroSky output protocol

Output Description
Raw Data Returns raw EEG data, sampled at 1HZ
Signal Returns poor signal level, 0 is good signal, 200 is off-head state.
Delta The ”delta band” of EEG (0.5 - 2.75Hz).
Theta The ”theta band” of EEG (3.5 - 6.75Hz).
Alpha 1 The ”low alpha” band of EEG (7.5 - 9.25Hz).
Alpha 2 The ”high alpha” band of EEG (10 - 11.75Hz).
Beta 1 The ”low beta” band of EEG (13 - 16.75Hz).
Beta 2 The ”high beta” band of EEG (18 - 29.75Hz).
Gamma 1 The ”low gamma” band of EEG (31 - 39.75Hz).
Gamma 2 The ”mid gamma” band of EEG (41 - 49.75Hz).
Attention Returns the eSense Attention integer value, between 0 and 100
Meditation Returns the eSense Meditation integer value, between 0 and 100
Blink Returns an integer value between 0-255, indicating the blink strength.

Note: Even though band power data are available from the mindset, having total
control of that process using only the raw data is part of understanding BCI.
Mindset interface: There are libraries available for almost ev-
ery platform and programming language, that interfaces the
bluetooth communication with the mindset and computer. In
this project, a .NET wrapper from Peek (2010b) was used.
It handles connection, disconnection, and receives a container
with all the output data in table 5.1 every 10 milliseconds (raw
data) and every second (all other output).
Bluetooth: A bluetooth device is included in the mindset pack-
age. The device software converts the bluetooth connection to
a COM-port, which makes application and device communica-
tion much easier.

24
 The Program is the main thread, event-handler and is named
BrainMonitor. It has a graphical user interface, viewable in
figure 5.2, that both displays and give users control of all the
implemented features.

Figure 5.2: Screenshot of BrainMonitor Graphical Interface. Upper left: Mindset Control
for connection, and mindset container view. At the top, middle screen: real-time raw EEG
record. Upper right: Sample view 27 spatial points and the Fourier Transformation and
band power. From bottom left to right: Scenario buttons, Neural Network training and
parameter setting, Neural Network testing, and Sample control buttons.

The ”Take sample” button in the sample control panel in figure 5.2 is particularly
useful for taking a live sample from the incoming EEG signal. Doing this, all data
about the sample is displayed in the sample view. It can be named, saved to disk
and retrieved later. Also, if a neural network is trained and ready to receive input,
the current live sample can be tested with that network. This is a great help to
quickly generate and test EEG samples.

The toolbox contains mainly the Fourier Transform algorithm.

The algorithm was verified with a 10Hz sine wave using the
function F(t) = sin(2π10t), giving a spectrum that only have a
power bar at 10Hz. Further, it contains the disk operator that
can read and write samples to disk so that sessions can be saved
and re-opened. Plotting of data in graphs for presentation is
also handled.

25
 The neural network is built on a solution available
at NeuronDotNet (n.d.-b), and is an implementation of the sort
explained in 4.4.1. A network setup screen was written to
enable easy modification of parameters, these are: Maximum
training cycles, learning rate, number of nodes in the hidden
layer, adjustment of input type to the network (details in chap-
ter 6,7,8), and the activation function in each layer; sigmoid
activation, linear activation, logarithmic activation, sine activa-
tion and tanh activation.
The game was one of the reasons for choosing Visual Studio
and C#, in addition to the available mindset library. The first
idea was to find an open source XNA game, preferably a role
playing game, to which the EEG could control some feature in
a dialog; feeling angry or tense could make the main character
only able to select bad options, for instance. However, this was
omitted later in the project, and a version of the classical game
Snake was implemented instead. This is covered in more detail
in chapter 8.

Implementation Steps Overview and Test Procedure

5.2
The following chapters, 6, 7 and 8, are a chronological approximation of the ongo-
ing implementation and testing done in the project. Basically, step 1 is were part
one, two and three in the BCI process flow (see figure 4.1) are realized and tested.
Step 2 is were part two and three are further improved to accomplish more com-
plex classification, and step 3 is the final implementation and realization of step 4
and the complete BCI system test.
Most testing is done by the author himself, unless specified otherwise. It was de-
cided to not test on many people, mainly to save time and to keep consistency since
experiments kept changing parameters. But, the interest for trying the EEG equip-
ment among fellow students was unquestionable. Several hundred test scenarios
were conducted in total, across several different experiments. Thus, only the most
representative test results are included here, to ensure fruitful discussions and
present the projects achievements.

26
6 Step One: EEG Eye-Blink Classification

Describes the first approach towards reaching the project goals. The method that
leads to the results is explained, the results presented, analyzed and followed by a
discussion about the methods benefits, disadvantages and lessons learned.

Method, System Setup and Neural Network Architecture

6.1
The first step was to 1) be able to connect to the NeuroSky mindset, 2) sample
the incoming raw EEG data, 3) use these samples to train a neural network, and
then 4) test the correctness. The game part was to be left out for later, as it would
depend on the neural network to be able to correctly classify brain waves. In this
section, each sub-step is described.

6.1.1 1: Connecting with the NeuroSky mindset

This worked flawlessly. The library that handles the communication with the blue-
tooth device did the job.

6.1.2 2: Sample the Incoming Raw EEG Data

The program is written to generate new sample objects that will add themselves
as observers of the mindset datastream (following the observer pattern), and fill
up an array of raw EEG data until it reaches the set sample size. This size was set
to 128 data points, so each sample takes 1280 milliseconds to complete (data is
updated every 10ms). There are two simple reasons for this: FFT analysis needs a
frame that is of a reasonable length, or else it will be difficult to do any accurate
analysis (Wolpaw, 2010). And, the FFT algorithm in the program needs an input
array were the size of it is a power of two.
By observing the real-time EEG record in the BrainMonitor program (see figure 5.1),
it was evident, but not surprisingly, that an eye blink gave a major fluctuation in
the EEG signal. Thus, it was decided to make this a first classification feature for
the neural network; correctness of the neural networks output would be easily ver-
ified, since analyzing the sample by observation would tell if it should be classified
as a blink or not. The other feature would be the baseline: thinking of nothing in
particular, keeping the eyes open. To gather a collection of EEG samples to be used
for network training, a scenario program was written that would do this automati-
cally while giving task instructions. Figure 6.1 shows the test scenario screen used.

27
Additionally, it was observed that the mindset was very sensitive to movement and
noise, and this is dealt with in section 6.2.2.

Figure 6.1: Sampling Screen: Middle: live instructions; bottom right: overall test instruc-
tions; bottom left: test control, session save directory and name. When the subject should
blink, the ”blink” string was flashed on the screen.
The test scenario was split in two parts, giving instructions to the user on the
screen. Part one gathered samples from the baseline task, 5 in total, and part two
gathered samples from the blink task, 5 in total. All the samples (with name,
number, raw EEG and FFT data) are saved to disk so they can be recalled later.
The collection of samples from each part in a scenario is called a set.

6.1.3 3: Neural Network Architecture and Training Setup

Figure 6.2 shows the architecture of the neural network. The input string to
the network was the five power bands of a sample: delta, theta, alpha, beta and
gamma. These were obtained by conducting FFT on the raw EEG data, and then
dividing them into buckets according to their frequency range. Each band value is
scaled with respect to the highest frequency in each individual sample, ensuring
that all values are in the interval between 0 and 1.

Figure 6.2: Neural network architecture 1: 5 input nodes, 3 hidden nodes, 1 output node.

28
Because there are only two features, 1 output node is sufficient, and all baseline
samples were trained to have 0 as the output value, and all blink samples were
trained to have 1 as the output value. The network was trained with 1 set of each
sample feature (5 blink samples, and 5 baseline samples).
Network parameters: learning rate was set to 0.25, learning cycles to 5000 and
number of hidden nodes to 3. Activation function for the input layer was linear,
and sigmoid was used for both hidden and output layer. Variations of these pa-
rameters were tested, but it did not have any great impact on the results.

6.1.4 4: Test of the Neural Network

The network can be tested using a single sample, or a collection of samples (which
was used in this experiment). In the latter case, the correctness of the set is cal-
culated with the basic average equation 6.1, which sums the result from each in-
dividual input sample in the set and divide it on the total number of samples.
An experiment consists of 5 scenarios taken in one single session (no headset re-
moval). Result values above 0.5 is regarded as a 1, and values below 0.5 is regarded
as a 0. N
1X
A= ai (6.1)
N
i=1

Test Results and Findings

6.2
This section show results from two test experiments, the first experiment is the one
described above in the previous section. Second experiment is a noise test scenario,
where movement of the head and touching is done and recorded, in order to get a
viewpoint on that. Noise is also referred to as artifacts, in the text.

6.2.1 Experiment One: Baseline vs. Blink

Figure 6.3, 6.4, 6.5, 6.6, and 6.7 are graphical presentation of input samples, from
both the baseline and blink task as specified.

Figure 6.3: Displays the raw EEG record of a baseline sample. Frequency spectrum of it is
showed in figure 6.4.

29
 Figure 6.4: Displays the frequency spectrum of baseline sample in figure 6.3.

Figure 6.5: Displays the raw EEG record of a blink sample. Frequency spectrum of it is
showed in figure 6.6.

Figure 6.6: Displays the frequency spectrum of blink sample in figure 6.5.

30
Figure 6.7: 4 of the 5 samples used to generate blink training input for the neural network.

Table 6.1 and 6.2 contains the baseline and blink input used, respectively, for the
neural network. Each set equals the sum of band powers for each sample in one
scenario run. Set 1 in each table was used to train the network. Table 6.3 is the
corresponding output, the classification results, from the neural network.

Table 6.1: Band powers of baseline samples. Eyes open.

Band Set 1 Set 2 Set 3 Set 4 Set 5 Total (Sum)

Delta 1.78 0.97 1.4 0.64 0.68 5.47
Theta 1.3 0.94 0.98 1.05 1.16 5.43
Alpha 1.42 1.49 1.28 1.1 1.33 6.62
Beta 4.28 4.84 4.62 4.69 5 23.43
Gamma 4.19 4.57 4.87 4.64 4.29 22.57
Sum 12.97 12.81 13.16 12.11 12.46 63.51

Table 6.2: Band powers of blink samples.

Band Set 1 Set 2 Set 3 Set 4 Set 5 Total (Sum)

Delta 4.94 4.39 4.69 4.74 3.54 22.3
Theta 2.88 3.59 3.98 3.26 3.5 17.21
Alpha 1.62 2.6 2.74 2.26 2.13 11.35
Beta 3.58 4.52 4.18 4.41 4.7 21.4
Gamma 2.83 3.33 4.23 3.67 3.46 17.52
Sum 15.85 18.43 19.82 18.34 17.34 89.77

Table 6.3: Classification result using data sets from table 6.1 and 6.2. Close to 1 = baseline,
and close to 0 = blink.

Scenario Task Set 1 Set 2 Set 3 Set 4 Set 5 Total (Average)

Baseline 0.82 0.96 0.94 0.97 0.96 0.93
(Eyes Open)
Blink 0.01 0.01 0 0.01 0.03 0.01

31
During each scenario, the attention and meditation values, that is calculated by
and received from the mindset, were plotten in a graph, and figure 6.8 is an exam-
ple of this. The plotting is from scenario 4, the first half (Time < 1600) is from the
baseline, and second half is the blink set (found in table 6.2 and 6.1).

Figure 6.8: Plotting of attention and meditation levels during scenario 1, a shift in both
values when going from baseline to blink task is clearly seen.

6.2.2 Experiment Two: Noise Sensitivity

Figure 6.9 shows a noise sample, with the respective frequency spectrum in fig-
ure 6.10. When compared to the figures 6.5 and 6.6 of the blink sample, they are
quite similar. This sample was taken when the forehead of the test person was
tapped with a finger. Other movements like jumping, turning the head, nodding
also gave similar noise samples. This particular sample was classified as a blink.

Figure 6.9: Spatial sample of noise : Tapping Figure 6.10: Frequency spectrum of noise :
on the forehead. Tapping on the forehead.

32
 Discussion of Results and the Sensitive Electrode
6.3

The result from the tests demonstrates that the implementation works as it should,
so far. It can connect and communicate with the mindset, sample EEG input, train
and test a neural network. Also, the system was able to classify blink correctly,
averaging at 99%.
Although noise was expected from the headset, it was still not anticipated that it
would be so sensitive to movements. Because of this, it was decided to conduct all
baseline experiments and sampling with eyes open, and without any movements
of head and body. The disadvantage of this is that it becomes very tiresome for the
eyes, on the other hand, the strict rule ease off the need to filter the EEG signal for
such artifacts. Indeed, this noise issue and the fact that the only electrode on the
mindset is placed in the forehead, brings research question 1 to the table: Does
blinking disturb the only input signal too much so that it makes it unreliable?
In figure 6.8, it is clearly seen that the NeuroSky algorithms that calculates atten-
tion and meditation are influenced by blinking. During the baseline part, the av-
erage attention and meditation level is approximately between 80-100 and 50-60,
respectively. When the blink part starts, both these levels drop drastically, espe-
cially attention. This is counter-intuitive, because waiting for the blink commands,
and respond to them appropriately, should bring the test person to a higher level
of alertness and focus, and not make them more inattentive. The blinks in this par-
ticular scenario may have been exaggerated and caused major disturbance, but the
tendency across all the experiments is that whenever there is an absence of blink,
the attention level goes to the top, and that the same level is very difficult to reach
even when blinking normally. But would this still be true if the electrode was
placed elsewhere on the scalp? This was tried, but it was unsuccessful to get a sig-
nal. And because the electrode is static, the options are limited without breaking
the mindset. In some studies, placement of frontal electrode is used to control that
the samples they take from other electrodes do not contain blink artifacts (Lukas
et al., 1995) (Wolpaw et al., 2000). However, the fact remains that the results in-
dicates that persons using the mindset should not blink during sampling. This is
not ideal when considering to make use of it in a real-time system and for longer
periods, like in gaming.

6.3.1 Simpler Blink Detection

In table 6.1 and 6.2, or when comparing the power spectrum of figure 6.4 and 6.6,
the difference between baseline and blink is clear. Blink is a simultaneously high
boost of delta (more than 4 times in average), theta and alpha amplitudes, and a
decrease in gamma. Blink detection could be done by monitoring, for instance,
delta, theta and gamma. If delta and theta values increased simultaneously, and
gamma decreased, that would be a blink signature. The amount of change from
pre-values could indicate strength. Like that of NeuroSky. No neural network is
needed. Finally, blinking could also be detected by using a camera, and it is a
muscle movement. Thus, it is not regarded as true brain communication as BCI

33
is defined in chapter 4. The system should be able to classify more than muscle
movement.

6.3.2 Critical Discovery: Making Better and More Stable Input Samples

This was actually discovered while trying to classify a mental task in step two. No
positive results were found when trying to classify input samples, so changes had
to be made. It is mention here, because it led to the fact that lot of test sessions
needed to be redone. Further, all results presented in this step, in step 2 and 3,
follows this way of generating network input.

Table 6.4 Table 6.5 Table 6.6

Band Level Band Level Band Level

Delta 1 Delta 19.38 Delta 0.923
Theta 0.024 Theta 6.54 Theta 0.311
Alpha 0.096 Alpha 3.95 Alpha 0.188
Beta 0.079 Beta 2.78 Beta 0.132
Gamma 0.677 Gamma 10.26 Gamma 0.489

Up to this point in the project, the neural network had been trained and tested with
single samples, representing 1280 milliseconds of EEG information each. This was
fine when only dealing with blink detection, because that is a very defined and
short event (Testing of blink was still redone to ensure consistency in all tests, in all
steps). However, when dealing with mental states, it is safe to assume that this is
not so, without long practice at least. The discovery was that the samples needed to
be reflecting a time period much greater than 1.3 seconds. This was done by taking
the average of all the band powers from all samples in a scenario set. The result
is shown above. Table 6.4 represent the band powers of a single baseline sample
(Scaled with respect to highest frequency). Table 6.5 represents the summation of
band powers of 20 baseline samples (taken from the same scenario), and finally,
table 6.6 is the average of table 6.5, and represents one sample set, usable for
training and testing. The difference between band powers of the single sample
and the set is evident.
The drawback is that it now takes approximately 20 seconds to generate input for
the network. In a game setting and real-time environment, this is a long time, and
it is a limitation.

34
7 Step Two: EEG Mental Task Classification

Method, System Setup and Neural Network Architecture

7.1
The classification of blink had demonstrated that the system worked and is able
to complete the three first step in the BCI process chain (see figure 4.1). However,
the system should be able to classify more than just a blink. This section describes
three approaches, called experiments, to find that which could be classified as a
mental state by the system. For all experiments, testing of the neural network are
the same as in step 1, explained in section 6.1.4.

7.1.1 Experiment One: Motion Task Attempt

It was decided that the mental state should be of a simple character, and the first
choice then became a motion task: thinking about and visualizing movement.
This is a mental task that is common to use in EEG experiments (Curran, 2003).
Collection of data sets was done by sampling baseline first, followed by visualizing
movement: Raising and lowering one arm, both arms, and legs. However, despite
all efforts and numerous trials and testing, there were no consistent findings that
proved to be classifiable. This led to the second experiment.

7.1.2 Experiment Two: Motion task and ERP Combined Attempt

The idea for this experiment was that visually invoked movement could help change
the mental state of the test person faster and more consistent. By combining mo-
tion intention and ERP (see section 3.3.1), it could be possible that the P300 effect
would have an impact on the band powers. A new test scenario screen was im-
plemented, see figure 7.1, that had a cartoon character doing arm movements:
moving an arm up and down repetitively towards the forehead. The test person
is instructed to imagine doing the same, and follow the rhythm in the motion;
receiving and responding to external stimuli.
Again, recording and sampling were done with eyes open during the whole ses-
sion. The scenario was split in two parts: sampling motion first and baseline sec-
ond. As input to the neural network, three types were tried: 5 band powers (delta,
theta, alpha, beta, gamma); 5 band powers and meditation and attention values,
making it a 7 input network; specific band powers (delta, theta, alpha1, alpha2,
beta1, beta2, gamma1, gamma2), making it a 8 input network.
Numerous trials and tests were conducted, but no results were found indicating
that this was a usable approach towards being able to classify a mental state. How-

35
ever, from the classification results it became clear that using meditation and at-
tention values as additional input to the neural network, and the specific band
powers, were worser options than only using the 5 band powers. This if further
discussed in section 7.3.

Figure 7.1: Sampling screen: Middle: live instructions; bottom right: overall test instruc-
tions; bottom left: test control, session save directory and options.

7.1.3 Experiment Three A: Visual Counting Task - Eyes Open

It was decided to try another mental task: visual counting, which have been used
successfully in, for instance, a study by Keirn and Aunon (1990). This also matches
the functionality of the frontal lobe, as it is related to mathematics and problem
solving. 1 scenario consisted of two parts: in the first part, 20 baseline samples
was recorded. In the second part, 20 visual counting samples was recorder. Eyes
were open during the entire scenario. The 20 samples from each part was used
to generate 1 training set per feature, averaged and scaled to 1, like discussed in
section 6.3.2.

7.1.4 Experiment Three B: Visual Counting Task - Eyes Closed

Experiment 3A gave results, but they were not satisfying. This led to a variant
where the baseline part of a scenario was conducted with eyes closed, in an attempt
to increase the difference in band powers. This should work because if one assumes
that when closing your eyes, the perception process is halted in the brain since
there are no visual input anymore. Regardless, in a study by L. Zhang et al. (2005)
it was recorded a major alpha waves increase in 20 EEG subjects, when their eyes
were closed vs. when their eyes were open.

36
7.1.5 Neural Network Architecture and Setup for Experiment Three

The neural network architecture are the same as in step 1 (see section 6.1.3). Re-
sults from experiment two indicated that the 5 input band power gave better per-
formance, so that was used in this setup as well. Network consisted of 1 input
layer: 5 input nodes, 1 hidden layer: 3 hidden nodes, 1 output layer, 1 node.
Network parameters setup: learning rate was set to 0.2, learning cycles set to 1000.
Activation function for the input layer was linear, sigmoid function for the hidden
and the output layer. Variations of these parameters were also tested, but did not
have any great impact on the results, like in step 1.

Test Results and Findings

7.2
Only results from experiment three are shown here, experiment one and two are
excluded due to the fact that they were unsuccessful attempts.

7.2.1 Results Experiment Three A: Eyes Open Only

Figure 7.2: Comparing total band power from all 5 sets in session one.

Figure 7.3: Comparing total band power from all 5 sets in the sessino two.

37
Figure 7.2 and 7.3 show results from two sessions of experiment three. The bars
show the total summation of power in each frequency band over all the sets from
5 scenarios. When comparing the two charts, they are the opposite of each other:
Blue dominates red in session one, and vice versa in session two.
The results in table 7.1 is the best among the sessions that were conducted, and
classified 4 of 5 sets correctly, both baseline and count. 3 correct classifications of
5 possible was the average. In one session, only 1 of 5 was correctly classified, and
that was when the neural network was tested with the same set that was trained
with.
Table 7.1: Classification result using data sets from figure 7.2

Task Set 1 Set 2 Set 3 Set 4 Set 5 Total (Average)

Baseline 0.93 0.95 0.95 0.8 0.87 0.9
(Eyes Open)
Blink 0.01 0.07 0.06 0.09 0.94 0.23

7.2.2 Results Experiment Three B: Eyes Closed and Eyes Open

Table 7.2: Band powers of baseline. Eyes closed.

Band Set 1 Set 2 Set 3 Set 4 Set 5 Total (Sum)

Delta 11.1 18.78 17.51 11.14 6.35 64.89
Theta 6.87 7.73 7.41 6.43 6.78 33.22
Alpha 15.64 10.08 14.03 17.47 20.16 77.38
Beta 8.53 6.52 6.22 11.08 8.29 40.64
Gamma 2.88 1.28 1.42 2.35 2.19 10.12
Sum 45.02 44.39 46.59 48.48 43.77 228.25

Table 7.3: Band powers of mental task: Count. Eyes open.

Band Set 1 Set 2 Set 3 Set 4 Set 5 Total (Sum)

Delta 19.02 19.26 18.32 17.37 14.15 88.12
Theta 10.51 9.23 11.97 11.53 13.88 57.11
Alpha 8.42 5.6 7.5 10.18 13.71 45.42
Beta 8.93 4.32 9.42 12.7 10.73 46.09
Gamma 4.36 1.85 4.88 7.85 5.82 24.76
Sum 51.24 40.25 52.08 59.64 58.3 261.51

Table 7.4: Classification result using data sets from table 7.2 and 7.3.

Task Set 1 Set 2 Set 3 Set 4 Set 5 Total (Average)

Baseline 0.93 0.74 0.93 0.95 0.98 0.91
Count 0.11 0.3 0.06 0.05 0.18 0.14

38
As seen in table 7.4, classification scores 5 of 5. Specific detail about the total band
power from each set used for baseline and counting, are found in 7.2 and 7.3,
respectively. The tables show an increase of alpha activity, and a decrease of all
the other waves, when eyes are closed.
Figure 7.4 through 7.7 are input samples used in the session, it illustrates the dif-
ficulty to interpret the waves by observation for the untrained eye.

Figure 7.4: Spatial sample of baseline - Eyes Figure 7.5: Frequency spectrum of baseline -
closed Eyes closed

Figure 7.7: Frequency spectrum of visual

Figure 7.6: Spatial Sample of visual count
count sample

39
 Discussion of Results and How the Eye State is the Key
7.3
Results from the experiments indicate the possibility to classify baseline and men-
tal count states, with a probability greater than 90%. This is if using a method that
combines sampling with eyes open and eyes closed. Experiments with eyes open
only, averaged with about 60% correct classifications. There were some exceptions
making only 20% in average, and one that made it above 90%. In summation,
these are unstable results. There can be a number of reasons for this, and they are
listed below and then discussed:
1. One electrode is not adequate to get the information needed to identify com-
plex though patterns in the mind.
2. The chosen tasks are not suitable with regards to the location of the electrode.
3. The chosen tasks are not suitable with regards to the test person.
4. Band frequencies are not the best choice for signal analysis in this context.
5. It is difficult to control that tasks are carried out correctly by the test person.
6. Neural Network is not the most suitable classification choice in this context.
As mentioned in section 4.1.1, most EEG studies uses 4 electrodes on more, placed
at some distance from the forehead (C, P, O groups in the 10-20 system, see fig-
ure 3.2). This suggest that perhaps one electrode is just not enough, since this is
not common practice. This aside, it can also mean that it is more difficult to find
a function or task that can be directed towards the electrode location. There is no
way to safeguard that one of several electrodes will record the essential informa-
tion. Still, results from experiment 3B accords with the conclusion from previous
research in L. Zhang et al. (2005). The biggest difference in total sum of band
power with eyes closed and eyes open is the alpha wave, as the study suggests.
However, in a game scenario, closing the eyes in long periods are not ideal for
gameplay, unless something is particularly designed for it. This is also the reason
why it took so long before experimenting with the eyes closed. The focus was what
would work in a game setting, and not in the system.
Visualizing movement is something that is very much used in EEG experiments.
Usually, it gives very good classification ratings in other studies (Lotte et al., 2007),
but no results were made when trying it in this project. Again, location of the elec-
trode can be assumed to be the reason for this. However, this can not be verified
since the mindset electrode is static. Moving it would require to break the device.
But there could also be other explanations. It is known that EEG patients and test
persons can learn to do a mental task, that is recognizable in the system, and then
forget how to do them some time later. Some are unable to re-learn it again. It
also happens that there are one or several persons in a healthy test population that
cannot be recognized for their mental efforts by the system, even though the rest of
the group can (Curran, 2003). The EEG of people is very different, so it is possible
that the test person in this experiment was not able to do the task, in a manner that
affected the EEG in any case. Also, it could be that other persons motion intentions

40
will be detected, but not their mental counting, or perhaps both. As an example
of not doing the task correctly, consider the baseline task. Could the instruction
to not think about anything in particular, have the opposite effect? That you start
to think about not thinking, or you focus to much about not thinking? This is
where practice comes into play, being able to regulate mental activity, an issue of
EEG that is discussed in section 3.3.2. However, this could explain the variations
of results in experiment 3A: High mental activity in both baseline and visualiza-
tion would give similar band powers. The test person may think he is doing two
different task, but in reality he is doing the same.
From this, some experiments were decided to be conducted with the attention and
meditation levels as extra input to the neural network. But the levels did not vary
enough, for the same reasons discussed in section 6.3: eyes open over time will
return a stable, high level attention value. Thus, this did not solve the problem.
Results were also made that suggests that EEG signals can vary even in the same
person, as the identifier in section 4.1.1 states: EEG signals are non-stationarity.
The results referred to are the two identical sessions with opposite input values in
experiment 3A, figure 7.2 and 7.3. The band powers in session one are generally
lower than baseline during mental count, but higher in session two. This indicates
that new neural networks should be trained with fresh input, for every session.

7.3.1 Baseline Comparison: Eyes open vs Eyes Closed

Having the baseline task done with eyes closed may not be ideal for gameplay, but
it gives a very good opportunity to rest them. It is stressed again here, that forcing
your eyes not to blink is disturbing in the long run. This is the most negative
when it comes to carry out these experiments, but as we have seen, it is the best
alternative if one wants to both view the screen and execute commands.

41
42
8 Step Three: Controlling a Game with EEG

Final System, Game Design and Test Conditions

8.1
With the results made in step 2 and experiment 3B, it was time for experimenting
with real-time classification while playing a game. A version of Snake was imple-
mented. First of all, because it is a very understandable and easy game to play
with few controls. Secondly, it is no doubt a classic game that is still popular to-
day, which leads to an assuming third point: that people find this game amusing.
Elaboration of the Snake implementation continues in section 8.1.2, explaining
the workings of the game loop. Finally, the last test of the entire BCI system is
explained, before the results are discussed.

8.1.1 Overview of the Main Parts of the Final System

Figure 8.1: Overview of final system flow

Figure 8.1 shows the two parts of the final system and the steps within them. In
order to play, the system needs to be initialized with a trained neural network.

43
8.1.2 Snake Gameplay, Controllable Features and Game Loop

This implemented version of Snake has the following gameplay. You start with
a single square that you control, and which represents the snake. It can move
in three directions: forward, left and right. The goal is to locate and eat apples
on a map, making the snake grow in length, and to become as large as possible.
Collision with the snake body will end the game, and avoiding this gets more
difficult as the snake grows, adding thrill. The map is borderless, meaning that if
the snake goes outside the screen, it will re-enter from the border opposite of it.
This adds an strategic element, and it can be used to escape from a tight spot, or
quickly navigate from one end of the screen, to the other.
The controllable features to choose from, are the three directions of movement.
For the sake of having speed in the game, moving forward should be automatic,
and whether to turn left or right should be controlled by the user via EEG input.
Section 6.3.2 discusses the need for many samples to be able to classify a mental
task, and as a result of that, the game can not be real-time; meaning that the game
should pause while collecting samples for classification. If it was to be real-time,
not only would the user need to know the next action 20 seconds in advance, but it
would only be possible to issue a command every 20 seconds. During that time, the
snake would continue forward uncontrolled. Unless the snake moves really slow.
To have EEG controlling all three directions would also be too slow, for the same
reasons. Alternatively, the forward direction could be controlled by the attention
level received every second from the mindset. But, for now, moving forward is
considered automatic. This is a fair setup to attain speed, since forward is the
most used direction in which the snake moves.
The system should have a mechanism to initialize the pause for sample collection.
This should be user controlled, and a blink input was chosen. User blink can
be detected almost real-time, and since the mindset gives information about the
blink strength, it is possible to set a a high threshold that allows regular blinking
for the user. This removes the need to play the entire game with open eyes. Also,
choosing blink eliminates the need to interact with a keyboard or mouse, making
the game totally controlled by EEG. Figure 8.2 is the resulting state diagram for the
game control. At game start, snake enters the move forward state. When blink is
detected, ”sampling mode” is entered. The system waits 1.5 seconds before doing
this. If not, the first sample taken would be contaminated with blink artifacts.

Figure 8.2: Snake control state diagram

44
Results from classification determines either a ”left turn” or a ”right turn” state,
before returning to the forward state again. Since the intended classification ex-
periment is 3B, the baseline task is conducted with eyes closed, so the user needs to
be informed when the sample collection is over. This is solved by playing a ”ding”
sound when sample mode is entered and exited.

8.1.3 Test Setup, Conditions and Statistics

The testing follows the system flow of figure 8.1. There is two types of tests, 1)
play and 2) control. Sessions in both test types repeats step 2 to 6. The headset is
not to be moved during a session. For step 2, 20 samples of baseline (eyes closed)
and 20 samples of mental counting (eyes open) were taken and used to train the
network. The network is then tested with the same sets, to ensure that they classify
correctly: Close to 1 represents baseline, close to 0 represents mental count. Same
network setup as in section 6.1.3 still applies.

NOTE: Because 20 seconds seemed like a long time, sample collection while play-
ing was reduced to 10 seconds, averaging 10 samples into a single input instead of
20.

1. Play Test: The user plays the game following the rules. Tests were done where
only step 3 to 6 was repeated, using old data to training the neural network. The
purpose is to see if those training sets are usable after disconnection and reloca-
tion of the headset. In that case, it could indicate that it is possible to make user
profiles, to quickly start a game and bypass the last setup steps.

Testing stops when the game is over, or if the user do not want to continue for
whatever reason. During the test, statistics of the following is taken:

• Number of correct classifications: result is like the test person intended.

• Number of wrong classifications: result is not like the test person intended.

• Number of correct blinks: System correctly enters sampling mode after user
gave a powerful blink to indicate that it should.

• Number of missed blinks: System incorrectly do not enter sampling mode

after the user gave a powerful blink to indicate that it should.

• Number of faulty blinks: System incorrectly enters sampling mode after the
user blinked normally, without intention of going into sampling mode.

• Total game time.

2. Control test: The user tries to classify a baseline task with eyes open, and a
counting task with eyes closed. This is interesting, because it can indicate whether
mental efforts affects the results at all, or if it is just controlled by eye states.

45
 Final Testing, Results and Findings
8.2

Figure 8.3: Screenshot from the snake-game. Middle top: Box that is colored red when
brain waves are NOT recorded, and green when it is recording. Center: Game display:
Red dots represents the snake body, green dot is the snake head, and blue dot is an apple.
Bottom left: current action.

8.2.1 Play Test Results

Test person one:

Game 1 lasted 7 minutes and 47 seconds, were 61% of that time was used to collect
samples. 4 apples were eaten before the game was over (crashed into the end of
the tail when growing). The neural network was trained, and had an accuracy of
0.98 when tested with the same sets it was trained with.
Table 8.1: Test results game 1, subject one

Feature Correct Miss Fault Accuracy

Blink 21 1 2 0.87
Classifications 21 2 - 0.91

46
Game 2 lasted approximately 6 minutes, were 51% of the time was used to collect
samples. 8 apples were eaten before it was game over (The 3 blink misses was the
cause of that). The neural network was trained, and had an accuracy of 0.95 when
tested with the same sets it was trained with.
Table 8.2: Test results game 2, subject one

Feature Correct Miss Fault Accuracy

Blink 15 3 1 0.8
Classifications 16 0 - 1.00

Test person two:

Table 8.3: Test results game 1, subject two

Feature Correct Miss Fault Accuracy

Blink 12 4 2 0.87
Classifications 10 2 - 0.83

Game 1 lasted 9 minutes and 13 seconds, were 34% of that time was used to collect
samples. 3 apples were eaten before it was game over. The neural network was
trained, and had an accuracy of 0.97 when tested with the same sets it was trained
with.
Table 8.4: Test results game 2, subject two

Feature Correct Miss Fault Accuracy

Blink 10 0 0 1.00
Classifications 10 0 - 1.00

Game 2 lasted approximately 5 minutes, were 38% of the time was used to collect
samples. 3 apples were eaten before it was game over. The neural network was
trained, and had an accuracy of 0.96 when tested with the same sets it was trained
with.
Reusing Training Sets: For both subjects, it was unsuccessful to reuse the training
sets. No correct classifications were achieved.

47
8.2.2 Control test

Test person one:

Figure 8.4: Results from control test 1.

In figure 8.4, each bar is the total average from 5 classification results in the control
test. With eyes closed, recognition certainty of the baseline task is close to 100%,
and gets reduced to about 50% when doing mental counting instead. With eyes
open, recognition certainty of mental counting is close to 80% (Close to 0 is classi-
fied as counting, thus the probability is calculated by subtracting 1), and reduced
to 60% when conducting the baseline task.

Test person two:

Figure 8.5: Results from control test 2.

In figure 8.5, each bar is the total average from 5 classification results in the con-
trol test. With eyes closed, recognition certainty of the baseline task is 91%, and
gets reduced to about 78% when doing mental counting instead. With eyes open,
recognition certainty of mental counting is close to 99% (Close to 0 is classified
as counting, thus the probability is calculated by subtracting 1), and reduced to
0.97% when conducting the baseline task.

48
 Discussion of Results
8.3
The results show a BCI system that enables users to play a game of Snake, control-
ling everything with EEG signals. Correct classification and eye-blink detection in
the play tests for the subjects are over 90%. Also, acquiring this accuracy required
no training. This is true for subject two, who had never used EEG equipment
before. The author of this project, who is subject one, have had two months of
experimenting with it, and can not be regarded as unfamiliar with the device and
the classification tasks. However, the results are acceptable as a proof-of-concept
for this project, and provides a good platform for which further improvements can
increase the accuracy.
Further, the control tests indicates that it is not just the two eye states, open and
closed, that affects the EEG signal. With eyes open, it was possible to successfully
classify a baseline task when attempted, and likewise, it was possible to identify
a count task when eyes were closed. In general, when not doing the mental tasks
correctly, the classification probability was lowered with 40% to 50%. This sug-
gests, that perhaps with more experience and practice it will be possible to obtain
faster and more accurate brain wave control and thus make the eye state redun-
dant. Still, these states are the key to correct classification in the current system,
at least for beginners. It is clear that subject one was more able to alter the waves
than subject two, implying that perhaps training made this effect. However, and
more importantly, these result indicates that the classification is not controlled by
muscle movement.

Figure 8.6: Testing - it is very relaxed.

49
Both users had good experiences with using EEG to control the snake. Compared
to the test experiments in step 2, the game testing was much more relaxed and less
strainful for the eyes. This has to do with the blink strength feature that allows
normal blinking to go undetected, and that closed eyes is a key feature for the
baseline task. This is a satisfying solution since the game needs to pause to collect
samples anyway, and the user is in control of when that will happen. Also, since
mouse and keyboard are not used, the possibilities of sitting position are many
more (Illustrated in figure 8.6).
Observations and notations from playing:
• If focused, it is easy to handle and control the snake.
• One time, when the snake was in a tight spot and it had to make a baseline
turn (left turn), the excitement led to too much mental activity, and it was
classified as a counting task, and the snake died.
• Since the snake rotates, it is not always so obvious what the left and the right
side of the snake is.
• There can be some delay from the physical blink to it is detected by the sys-
tem. Therefore, the blinking must be timed extra carefully.
• Being able to get the classification results that is intended gave a mastering
feeling.
• The borderless map was used a lot to make shortcuts and to escape. It re-
duces the necessity of having to turn very often. This also speeds up the
game.
• Time used to collect samples did not feel long.

50
 9 Conclusion

This report presents results that shows that is it possible to build a Brain-Computer
Interface system that allow users to play a game and controlling it with their brain
waves. This have been accomplished by using the NeuroSky mindset EEG equip-
ment featuring only one electrode on the forehead. EEG signals from the user are
sent to the computer via bluetooth. The signal is then processed and the waves
band power is calculated using the Fourier transform. This information is used
as input to a neural network that is trained to classify two different mental tasks.
Then this classification is used to control the movements (left and right turn) of a
character in a custom built PC game, successfully.
The biggest challenge and time-consumption in the project was not the implemen-
tation itself, but testing it with the use of real-time EEG input. Three major test
steps is outlined with results that have given insights in the workings of EEG and
difficulties with BCI systems. The first step was to implement all the components
of the system and make them communicate with each other, then enable the sys-
tem to classify blinks in the incoming EEG signal samples. This was done with
a success rate close to 99%. The solution worked, indeed, but was not adequate
to classify mental tasks. This was improved in step two, were a new method was
discovered that required ten times as many samples and thus effort from the user.
Still, it became possible to classify two mental states: baseline (relaxed, calm and
not thinking of anything in particular), and mental counting. The first results
only gave 60% probability of correct classification, but this was further increased
to greater than 90%. This high rate was only attainable when eye states (eyes open
and closed) were used actively by the user in addition to the tasks. However, this
enabled the realization of step 3, were a version of the game Snake was designed
and implemented to work with the EEG input. Results from user tests show that
the snakes movements were correctly controlled with an accuracy of 90 to 100%.
Regarding the research questions (see section 2.2.1), the following conclusions are
made from the project results and findings:
RQ1: That one electrode in the forehead can replace a grid of, or several, elec-
trodes is highly unlikely. No papers were found of previous studies, were only one
electrode have been used. The experience from this study, is that mental efforts
that are relatively safe to classify did not work. There can however be other rea-
sons for this than just the single electrode, but at least it suggest that one is not
adequate.
RQ2: Having the experience of using both the NeuroSky mindset and traditional
EEG equipment, NeuroSky fulfills the expectations as described in the early chap-
ters. More interestingly, the limitation that was most profound with the mindset
was the static electrode placement. It can be moved to a certain degree, but not

51
enough to get it away from the forehead. Also, one or two additional sensors would
have been great. The Emotive headset, featuring 14 electrodes (see section 4.2.1),
would be the recommended choice for this upgrade. It is only slightly more ex-
pensive. The connected limitation with placement is that the sensor is heavily
influenced by blink. However, the reason the electrode is placed on the forehead,
is because people do not have hair there, which is a requirement for this type of
dry sensor.

RQ3: The type of classification achieved using a neural network is that of band
power patterns. Where the difference between the power bands is of such degree
that they are distinguishable, and were the right mental tasks are chosen that will
promote this pattern in the users EEG.

RQ4: The resulting BCI system enables an entertaining way for training brain
wave control. Both test persons thought is was more fun to play the snake game,
than the NeuroBoy software that comes with the NeuroSky mindset. However,
more people should test the system to verify that this is truly so.

RQ5: It requires no training to use the resulting system, only 1-2 minutes of prepa-
ration time, were EEG samples needs to be taken for the neural network training.
With these samples the network learns the difference between the two mental task
patterns based on the users current EEG. It was assumed that some training would
be needed in order to learn how to do the tasks. This too should be verified by
testing the system on more people and get the necessary statistics.

Compared to the games that have used the NeuroSky mindset and the attention
value only, this game stand out as more easy to use. It is easier to control the
output of mental efforts and thus the accuracy is higher. In a context were the
output is binary, the accuracy is additionally higher. The classification (rather
than a threshold) takes longer time, so the game is slower. But is works, and it is
still fun. Also, there is no training needed in order to play, one just have to focus
on the task. In the other games, the parameters than controls the attention levels
are unknown. If it is raised by increased beta waves and lower theta waves, one
would have to figure out how to manipulate these waves in a trail-and-error style.
In neurofeedback therapy, this is perhaps wanted, but if one just want to play, the
snake game is much easier. Like a patient that is paralyzed and want to kill some
time, and not train brain wave control.

Further Development and Ideas for the Future

9.1

There are two main suggestion for continued work with the system presented in
this report. 1) Further development of the program, improving the game, signal
processing and classification procedures and feature extensions. 2) Use the cur-
rent program as a tool for larger scale tests with people, to do EEG surveys and
monitoring studies. Most of these suggestions comes from observations made, or
questions asked, in the discussion sections of chapter 6, 7 and 8.

52
1) Further development and improvements of BrainMonitor

• Implementation of automatic blink strength threshold adjustment: This

would minimize the blink fault error and increase playability. The strength
value varies a lot, depending on the electrode placement when the mindset
is put on. This can be automatically adjusted by collecting blink strength
samples, in an scenario run, from the user when blinking normally. The
average value of these samples would then become the threshold.
• Enable the possibility to classify several features: One option, using the
existing neural setup, is to have several neural networks trained for different
recognition task and test EEG samples from the user simultaneously on all of
them. Their results are then compared and the network with highest prob-
ability wins. Another option is to design a new neural network architecture
with all the features wanted in the system represented in the output layer.
One output node per feature.
• Add P300 recognition, to assist in classification verification: For example,
if the user blinks when playing the snake game to signal ”sample mode”,
detecting the P300 signal could verify that this is the wanted move. Instead
of looking at band powers to find the P300, the raw incoming EEG data could
be spatially analyzed by an algorithm that searches and verifies a peak in the
incoming sample.
• Interface the BrainMonitor program with existing games: There are many
open source games available, that could be interesting to connect with the
EEG equipment. The advantage is that a lot of work is already done, like the
graphical display and gameplay, so the focus can remain on what and how
the EEG function should be. This is further described in the ”ideas for the
future” section.
• Interface the BrainMonitor program with hardware: There exists a library
for simple communication with a microcontroller. The same microcontroller
that was used in the Playstation project described in section 2.1.1 could be
used to integrate it with hardware and make physical objects move, for ex-
ample.

2) BrainMonitor as a tool

Here are some suggestions of how to utilize the current system as it is:
• More verifications: Run tests that run over a period of time, to see if it is
possible to improve brain wave control skills using the system. The goal and
motivation should be to annihilate the need for the eye states, and rely on
mental efforts only.
• Explore classification possibilities: Define new experiments to find if there
are other mental tasks that are usable for classification in the system. This
could be rotation, mental speech or specific memory retrieval.

53
 • Explore placement possibilities: Disassemble, if possible, the arm on the
mindset that holds the electrode and find alternative placement for it. Con-
duct test to see if those placements are better suited for solving the classifi-
cation tasks at hand. The inconvenience is that there can be no hair between
the electrode and scalp.

9.1.1 Ideas for the Future - NeuroSky Entertainment

Instead of having the mind directly control features of a game, it could be used as
an unconscious influence in the gameplay. For example, in a role playing game,
you have a main character, and perhaps a party of followers or helpers. It is com-
mon, that throughout the game, your main character gains more experience and
get better in a set of skills that will help to conquer challenges later in the game.
EEG could be used to add a new dimension to that kind of character building. A
continuous analysis of the brain waves could form user models that will be used
for influence, with both short- and long-term effects. In the latter case, if you are
in general very relaxed or focused, your character might regenerate quicker (if
combat is involved), or have more endurance, or get access to special spells of the
defensive kind. The opposite would apply for those who are generally restless.
In the external game world, it the player gives signs of drowsiness, the game could
introduce unexpected happenings, like an suddenly ambush or develop a creepy
atmosphere using mediums like music and lights, whatever is most appropriate.
This would be a short-term effect. An other example could be that the dialog could
adjust to the users current mood, or how the party of followers engage and act
around you. This would create more variation in the game and give them more
personality. Additionally, the EEG classifications and algorithms that are to ana-
lyze the brain waves do not have to be that accurate. Incorrect results would not
hinder the gameplay. And thus it may be a good place to start applying EEG. At
least when applying one electrode equipment like the mindset and sampling takes
time, indirect influence may be the better way to go.

54
 References
Andersen, K., Juvik, E., Kjellen, E., & Storstein, K. I. B. (2009). Radio controlled
helicopter using neurofeedback. unpublished.
Andrew T Campbell, Tanzeem Choudhury, Shaohan Hu, Hong Lu, Mashfiqui
Rabbi, Rajeev D S Raizada, M. K. M. . (2010). NeuroPhone: Brain-Mobile
Phone Interface using a Wireless EEG Headset. MobiHeld 2, 3–8. Available
from http://www1.cs.dartmouth.edu/reports/TR2010-666.pdf
Berg, K. (2010). Balanser hjernen og yt mer. Retrieved from http://www.ytmer.no/.
Bernier, R., Dawson, G., Webb, S., & Murias, M. (2007, August). EEG mu rhythm
and imitation impairments in individuals with autism spectrum disorder.
Brain and cognition, 64(3), 228–37.
Blankertz, B., Dornhege, G., Krauledat, M., Müller, K.-R., Kunzmann, V., Losch,
F., et al. (2006, June). The Berlin Brain-Computer Interface: EEG-
based communication without subject training. IEEE transactions on neu-
ral systems and rehabilitation engineering : a publication of the IEEE En-
gineering in Medicine and Biology Society, 14(2), 147–52. Available from
http://www.ncbi.nlm.nih.gov/pubmed/16792281
Bos, O., & Reuderink, B. (2008). Brainbasher: a BCI game. Eindhoven University of
Technology, 36–39. Available from http://doc.utwente.nl/65050/
Brain Entertainment. (n.d.). Available from http://betabrainwave.com/
Braverman, E. (1990). Brain mapping: a short guide to interpretation, philosophy
and future. Journal of Orthomolecular Medicine, 5(4), 189–197. Available
from http://orthomolecular.org/library/jom/1990/pdf/1990-v05n04-
p189.pdf
Curran, E. (2003, April). Learning to control brain activity: A review of the pro-
duction and control of EEG components for driving brain–computer inter-
face (BCI) systems. Brain and Cognition, 51(3), 326–336. Available from
http://linkinghub.elsevier.com/retrieve/pii/S0278262603000368
Dario, F., & Mattiussi, C. (2008). Bio-Inspired Artificial Intelligence (R. C. Arkin,
Ed.). London, England: MIT press.
Emotiv. (n.d.). Brain Computer Interface Technology. Retrieved autumn 2010 from
http://www.emotiv.com/.
Fabiani, G. E., McFarland, D. J., Wolpaw, J. R., & Pfurtscheller, G. (2004,
September). Conversion of EEG activity into cursor movement by
a brain-computer interface (BCI). IEEE transactions on neural sys-
tems and rehabilitation engineering : a publication of the IEEE Engi-
neering in Medicine and Biology Society, 12(3), 331–8. Available from
http://www.ncbi.nlm.nih.gov/pubmed/15473195
Felzer, T., & Freisleben, B. (2003, December). Analyzing EEG signals using
the probability estimating guarded neural classifier. IEEE transactions on
neural systems and rehabilitation engineering : a publication of the IEEE En-
gineering in Medicine and Biology Society, 11(4), 361–71. Available from
http://www.ncbi.nlm.nih.gov/pubmed/14960111

55
Gevensleben, H., Holl, B., Albrecht, B., Vogel, C., Schlamp, D., Kratz, O., et al.
(2009). Is neurofeedback an efficacious treatment for ADHD? A randomised
controlled clinical trial. Journal of Child Psychology and Psychiatry, 50:7, 780–
789.
Gilden, L. (2009). The Role of Language in Human Social Conflict. Retrieved autumn
2010, from http://www.lifwynnfoundation.org/langrole.htm.
G.tec. (2009). intendiX - World ’ s first personal BCI speller (Vol. 2011) (No. i).
Available from http://www.intendix.com/
Hammond, D. C. (2006). What is Neurofeedback? Journal of Neurotherapy, 10:4,
11.
Heinrich, H., Gevensleven, H., & Strehl, U. (2007). Annotation: Neurofeedback -
train your brain to train behaviour. Journal of Child Psychology and Psychiatry,
48:1, 3–16.
Hiraiwa, A., Shimohara, K., & Tokunaga, Y. (2002). EEG topog-
raphy recognition by neural networks. Engineering in Medicine
and Biology Magazine, IEEE, 9(3), 39–42. Available from
http://ieeexplore.ieee.org/xpls/abs all.jsp?arnumber=59211
Hsu, W.-Y., & Sun, Y.-N. (2009, January). EEG-based motor imagery analysis using
weighted wavelet transform features. Journal of neuroscience methods, 176(2),
310–8. Available from http://www.ncbi.nlm.nih.gov/pubmed/18848844
Immrama. (n.d.). Available from http://www.immrama.org/eeg/electrode.html

Juri D. Kropotov. (2009). Quantitative EEG, Event-Related Potentials and Neurother-

apy.
Keirn, Z. a., & Aunon, J. I. (1990, December). A new mode of
communication between man and his surroundings. IEEE transac-
tions on bio-medical engineering, 37(12), 1209–14. Available from
http://www.ncbi.nlm.nih.gov/pubmed/2149711
Kellis, S., Miller, K., Thomson, K., Brown, R., House, P., & Greger, B. (2010,
September). Decoding spoken words using local field potentials recorded
from the cortical surface. Journal of neural engineering, 7(5), 10pp. Available
from http://www.ncbi.nlm.nih.gov/pubmed/20811093
Kropotov, J. D. (2009). Quantitative EEG, Event-Related Potentials And Neurother-
apy. 525 B Street, Suite 1900, San Diego, CA 92101-4495, USA: Elsevier Inc.
Kropotov, J. D., Yatsenko, V. A. G., Ponomarev, V. A., Chutko, L. S., Yakovenko,
E. A., & Nikishena, I. S. (2007). Changes in EEG Spectrograms, Event-Related
Potentials and Event-Related Desynchronization Induced by Relative Beta
Training in ADHD Children. Journal of Neurotherapy, 11:2, 3–11.
Larsen, E. A. (2010). Playstation Controlled by Brain Waves (Tech. Rep.). Trond-
heim.
Leins, U., Goth, G., Hinterberger, T., Klinger, C., & Strehl, U. (2007). Neuro-
feedback for Children with ADHD: A Comparison of SCP and Theta/Beta
Protocols. Appl Psychophysiol Biofeedback, 32, 73–88.
Leuthardt, E. C., Schalk, G., Wolpaw, J. R., Ojemann, J. G., & Moran, D. W.
(2004, June). A brain-computer interface using electrocorticographic sig-
nals in humans. Journal of neural engineering, 1(2), 63–71. Available from
http://www.ncbi.nlm.nih.gov/pubmed/15876624
Liu, M., Ji, H., & Zhao, C. (2008, May). Event Related Potentials Extraction from

56
 EEG Using Artificial Neural Network. 2008 Congress on Image and Signal
Processing, 213–215.
Lotte, F., Congedo, M., Lécuyer, a., Lamarche, F., & Arnaldi, B. (2007, June).
A review of classification algorithms for EEG-based brain-computer in-
terfaces. Journal of neural engineering, 4(2), R1–R13. Available from
http://www.ncbi.nlm.nih.gov/pubmed/17409472
Lukas, S. E., Mendelson, J. H., & Benedikt, R. (1995, February).
Electroencephalographic correlates of marihuana-induced eupho-
ria. Drug and alcohol dependence, 37(2), 131–40. Available from
http://www.ncbi.nlm.nih.gov/pubmed/7758402
Lukas, S. E., Mendelson, J. H., Kouri, E., Bolduc, M., & Amass, L. (1990). Ethanol-
induced alterations in EEG alpha activity and apparent source of the audi-
tory P300 evoked response potential. Alcohol (Fayetteville, N.Y.), 7(5), 471–7.
Available from http://www.ncbi.nlm.nih.gov/pubmed/2222851
Masterpasqua, F., & Healey, K. N. (2003). Neurofeedback in Psychological Practice.
Professional Psychology: Research and Practice, 34:6, 652–656.
Mitsar. (2008). BIOFEEDBACK TRAINER MITSAR-BFB. pdf available at
http://www.mitsar-medical.com/download/promo/Mitsar-BFB 2008.pdf.
Mitsar. (2010). Mitsar Co. Ltd. Medical equipment for functional diagnostics and
neurofeedback. Retrieved from http://www.mitsar-medical.com/.
Nawrath, M. (n.d.). Lab3 - Laboratory for Experimen-
tal Computer Science. Retrieved autumn 2010, from
http://interface.khm.de/index.php/lab/experiments/arduino-realtime-
audio-processing.
NeuronDotNet. (n.d.-a). Backpropagation Algorithm. Available from
http://neurondotnet.freehostia.com/manual/bpnalgo.html
NeuronDotNet. (n.d.-b). NeuronDotNet. Available from
http://neurondotnet.freehostia.com/download.html
Neurosky. (n.d.). Neurosky. Available from http://www.neurosky.com/
NeuroSky, I. A. (2010). ThinkGear Socket Protocol (Tech. Rep.). Available from
http://www.neurosky.com/
Peek, B. (2010a). MindBlaster. Available from
http://channel9.msdn.com/coding4fun/articles/MindBlaster
Peek, B. (2010b). ThinkGearNET 1.1. Available from
http://thinkgearnet.codeplex.com/
Plass-Oude Bos, D., Reuderink, B., Laar, B. L. A. van de, Gürkök, H.,
Mühl, C., Poel, M., et al. (2010, October). Human-Computer Inter-
action for BCI Games: Usability and User Experience. In A. Sourin
(Ed.), Proceedings of the international conference on cyberworlds 2010, sin-
gapore (pp. 277–281). Los Alamitos: IEEE Computer Society Press.
http://eprints.eemcs.utwente.nl/18057/.
PlayAttention. (n.d.). Play Attention. Retrieved in august 2010 from
http://www.playattention.com/.
Rangaswamy, M., Porjesz, B., Chorlian, D. B., Wang, K., Jones, K. a.,
Bauer, L. O., et al. (2002, October). Beta power in the EEG
of alcoholics. Biological psychiatry, 52(8), 831–42. Available from
http://www.ncbi.nlm.nih.gov/pubmed/12372655
Reuderink, B. (2008a). Games and Brain-Computer In-

57
 terfaces: The State of the Art. Internal Report.
http://www.borisreuderink.nl/files/publications/reuderink2008gbc.pdf.
Reuderink, B. (2008b). Games and brain-computer interfaces: The state of the
art. WP2 BrainGain Deliverable, HMI, University of Twente (September 2008),
1–11.
Schalk, G., McFarland, D. J., Hinterberger, T., Birbaumer, N., & Wol-
paw, J. R. (2004, June). BCI2000: a general-purpose brain-
computer interface (BCI) system. (Vol. 51) (No. 6). Available from
http://www.ncbi.nlm.nih.gov/pubmed/15188875
Sherlin, Leslie, Arns, Martijn, Lubar, Joel, et al. (2010). A Position Paper on Neuro-
feedback for the Treatment of ADHD. Journal of Neurotherapy, 14:2, 66–78.
Sinek, S. (2009). Start With Why. Retrieved autumn 2010 from
http://www.startwithwhy.com/.
SmartBrain. (n.d.). Biofeedback. Retrieved december 2010, from
http://www.smartbraintech.com/.
Stigsby, B., Rodenberg, J. C., & Moth, H. B. (1981). Electroencephalographic find-
ings during mantra meditation (Transcendental Meditation). A Controlled,
Quantitative Study of Experienced Mediators. Electroencephalography and
Chnical Neurophystology,, 51, 434–442.
Sundaram, M., Sadler, R. M., Young, G. B., & Pillay, N. (1999). REVIEW ARTICLE
EEG in Epilepsy : Current Perspectives. The Canadian Journal of Neurological
Sciences, 26(4), 255–262.
Takahashia, T., Murataa, T., Hamadab, T., Omoria, M., Kosakaa, H., Kikuchic, M.,
et al. (2005). Changes in EEG and autonomic nervous activity during med-
itation and their association with personality traits. International Journal of
Psychophysiology, 55, 199–207.
Universe-review.ca. (n.d.). A Review of the Universe. Available from
http://universe-review.ca/I10-80-prefrontal.jpg
Wolpaw, J. R. (2010, November). Brain-computer interface re-
search comes of age: traditional assumptions meet emerging re-
alities. Journal of motor behavior, 42(6), 351–3. Available from
http://www.ncbi.nlm.nih.gov/pubmed/21278858
Wolpaw, J. R., Editor, G., Birbaumer, N., Heetderks, W. J., Mcfarland, D. J., Peck-
ham, P. H., et al. (2000). Brain – Computer Interface Technology : A Review
of the First International Meeting. IEEE Trans. Rehabil., 8(2), 164–173.
Wolpaw, J. R., & Mcfarland, D. J. (1994). Multichannel EEG-based brain-computer
communication. Electroencephalography and clinical Neurophysiolog, 90, 444–
449.
Zhang, L., He, W., Miao, X., & Yang, J. (2005, January). Dynamic EEG Analy-
sis via the Variability of Band Relative Intensity Ratio: A Time-Frequency
Method. Conference proceedings : ... Annual International Conference of
the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in
Medicine and Biology Society. Conference, 3(1), 2664–7. Available from
http://www.ncbi.nlm.nih.gov/pubmed/17282787
Zhang, Y., Chen, Y., Bressler, S. L., & Ding, M. (2009). Response preparation and
inhibition: The role of the cortical sensorimotor beta rhythm. Neuroscience,
156:1, 238–246.

58