2016 IEEE 4th International Conference on MOOCs, Innovation and Technology in Education

Brain-Computer Interface Learning System for Quadriplegics

P. Sastha Kanagasabai¹, R. Gautam², Dr. G.N. Rathna³
Department of Electronics and Communication, Thiagarajar College of Engineering, Madurai¹
Department of Electronics and Communication, National Institute of Technology, Tiruchirapalli²
Department of Electrical Engineering, Indian Institute of Science, Bengaluru³
[email protected]¹, [email protected]²

Abstract—The proposed Brain-Computer Interface system generated by the brain. This neural activity can be
enables Quadriplegic patients, people with severe motor recorded using invasive and non-invasive methods.
disabilities to send commands to electronic devices and Electroencephalogram was found by Hans Berger in 1929.
communicate with ease. Interactive sessions are vital for Several progresses have been made in last 30 years
effective knowledge transfer in any learning eco-system. The improving the state of recording the brainwaves.
growth of Brain-Computer Interface (BCI) has led to rapid
development in 'Assistive Systems' for the disabled called
The General functional model of BCI System is discussed
'assistive domotics'. Brain-Computer-Interface is capable of below as shown in figure 1.
reading the brainwaves of an individual and analyse it to
obtain some meaningful data. This processed data can be
used to assist people having speech disorders and sometimes
people with limited locomotion to communicate. In this
Project, Emotiv EPOC Headset is used to obtain the
electroencephalogram (EEG). The obtained data is processed
to communicate pre-defined commands and queries for
interactive learning. EEG data can also be used to monitor
student's emotional behaviour and provide emotional
feedback to the students. Other Vital Information like the
heartbeat, blood pressure, ECG and temperature are
monitored and uploaded to the server. The Data is processed
in Intel Edison, system on chip (SoC). Patient metrics are
displayed via Intel IoT Analytics cloud service.

Keywords—Brain-Computer-Interface, Learning system, IoT

Analytics, Electroencephalogram

I. INTRODUCTION Figure 1: BCI-General functional model

Quadriplegic patients have partial or no control Brain Activity can be measured through various non-
over their limbs and torso. This might have occurred as a invasive methods like computerized tomography (CT),
result of spinal cord injury (SCI), strokes, cerebral palsy magnetic resonance imaging (MRI) and magneto
etc. According to the National Spinal Cord Injury encephalography (MEG). In our project,
Statistical Centre at University of Alabama, as of 2015, electroencephalography (EEG) is used to measure the
12500 new SCI occur every year. SCI primarily affects brain activity. EEG has high temporal resolution and
young adults. Nearly half of all injuries occurred between reasonable spatial resolution (256 electrode sites).
the ages of 16 and 30. For such a patient, alerts and International 10-20 system is followed in
external communication are of extreme importance. When electrode placement in Emotiv EPOC headset used for the
such patients are bed-ridden or under complete rest, it project as shown in figure 2. Signals recorded from EEG
becomes mandatory for them to convey certain things to can be split into 5 bands: delta band (0.5-3 Hz), theta band
their caretaker or nurse who might not be always next to (4-7 Hz), alpha band (8-15 Hz), beta band (16-31 Hz) and
them. Patients who also suffer from speech disorder are hit gamma band (31+ Hz).
badly in terms of communication. In BCI, the components produced by user must
Quadriplegics need special educational aids to be voluntary. Effectiveness of BCI is depends on
pursue their education. Interactive sessions are vital for conscious and consistent mental control over EEG. Motor
effective knowledge transfer in any learning eco-system. Imagery (MI) tasks are most widely used cognitive tasks.
As Quadriplegics can't interact like normal people, they Once the user is able to generate desired signals
need some assistance to communicate effectively. In this voluntarily, signal must be processed to extract features.
concern, we propose Brain-Computer Interface learning Relevant features must be chosen to get better
system which enables quadriplegics to communicate with characterization without ambiguity.
the external world effectively by displaying their ideas in a In this work, we have used the Intel IoT Analytics
graphical display. Brain-Computer Interface is capable of services with Intel Edison board. Intel provides a cloud-
reading the brainwaves of an individual and analyse it to based analytics system for the Internet-of-Things (IoT)
obtain some meaningful data. BCI can be used to control that includes resources for the collection and analysis of
exo-skeletons for locomotion. sensor data that the Intel® IoT Developer Kit provides.
Electroencephalography (EEG) data is used in Captured EEG data are processed in the cloud-based
Brain-Computer Interface for paralyzed (quadriplegic) analytics system.
patients, enabling them to communicate with others.
Brain-Computer Interface is based on the neural activity

978-1-5090-1062-2/16 $31.00 © 2016 IEEE 258

DOI 10.1109/MITE.2016.62
 classifiers, nearest neighbour classifiers and combinations
of classifiers are used commonly.
Ehsan Tarkesh Esfahani and V. Sundararajan
(2012) describe application based on BCI signal
classification. Visual mental imagery, auditory imagery
and kinaesthetic imagery – commonly called ‘motor
imagery’ – are main categories of mental imagery that can
be considered in brain–computer interfaces. They
achieved an average accuracy of about 44.6%.

III. METHODOLOGY

Our proposed BCI learning system contains

heart-rate, blood pressure, ECG and body temperature
monitoring in real-time. In addition to it, we have added
fire alarm system as a contingency measure. The Emotiv
EPOC headset is used to record EEG signals and classify
them based on their features. A pre-defined set of
commands/Instructions are available for the patient to
Figure 2: 10-20 system (source: wikimedia.org) communicate with.
The advancement in our system is to inculcate
II. LITERATURE SURVEY EEG signals over a voice based interactive control. The
primary reason behind this is to reduce the signal loss
Silent speech methodologies for disabled individuals occurring due to noise and reflections. Moreover, voice
utilize electromyography (EMG) recordings of vocal tract signal processing in a classroom like environment is really
movements. Electroencephalography (EEG) can be used tedious as there will be multitude of signals and even the
for paralyzed people to communicate. The advances in slightest mishap in the acquiring or processing part will
EEG techniques are as following. result in wrong inputs and finally wrong commands. On
Jonathan S. Brumberg et al (2010) reviews the the other hand, the interference in EEG signals due to
available silent speech technologies used for disabled external factors is almost null which proves a very good
individuals. Invasive methodologies like source for strong signals and proper data acquisition.
electrocorticogram (EcoG), local field potentials (LPF), Machine learning algorithms can be used to teach
single unit activity (SUA) and Non-invasive machines to recognise specific patterns.
methodologies like the electroencephalogram (EEG) and The entire system is connected to the same
magneto encephalogram (MEG) are used. internet connection enabling maximum utilization of IoT
Sumit Soman and B K Murthy's work (2015) concepts and minimizing the cost. The speed and
converts user eye-blinks to select text on screen into reliability of such IoT systems can benefit a large number
synthesized speech using a Text-to-Speech (TTS) system. of patients at the same time. Emotiv EPOC headset sends
The user’s eye-blinks are detected by processing EEG EEG waves to the firmware Emotiv SDK (software
signals using feature extraction and classification development kit) Community Edition installed in a
techniques. Computer. The EEG Wave data is transmitted using
Muhammad N. Fakhruzzaman et al's (2015) work Bluetooth 4.0 to the USB Receiver in the computer. After
has concluded that BCI can identify and distinguish classification, the Information is sent to Intel Edison board
activity by means of using intensively trained classifying using SSH (Secure shell) connection via Intel Arduino
algorithm with intended activity. The trained classifier is IDE (Integrated development environment). The Input
able to classify online EEG data into trained class from Emotiv SDK is decoded to execute user-defined
according to the pattern. commands.
EEG Signals are classified using various Heart rate sensor, blood pressure and temperature
algorithms. Pari Jahankhani et al (2006) discuss about sensors are interfaced with the Intel Edison. These sensors
EEG signal classification using wavelet transform (WT). provide real-time healthcare monitoring of the
The classification schemes explained are Multilayer quadriplegic. Fire sensor is used to implement a fire-alarm
Perceptron Network (MLP) and Radial Basis Function system with a buzzer Indication. The overall block
Network (RBF) scheme. diagram of project is shown below in figure 3.
Rishab Shah et al (2015) propose a smart home Intel Edison is connected to the Intel IoT cloud
controller using brainwaves for the paralyzed patients. via Wi-Fi in-built using the IoT Analytics. In the
They have used arduino development board interfaced dashboard of IoT Analytics, the logged data can be
with emotiv EPOC headset. accessed and required processing is done. Patient Metrics
F Lotte et al (2007) discuss the various can be accessed from any system logged-in to the IoT
classification algorithms and the feature properties used. Analytics account.
Linear classifiers, neural networks, nonlinear Bayesian

259
 combination of keystrokes. The desired expression is
mapped with a specific keystroke with key hold time and
trigger options. Sensitivity is set to a particular trigger
level. For example, the Imagery of a glass of water is
mapped to letter 'w', which is decoded as “I need a glass of
water” command. Figure 6 shows the EmoKey setup.

Figure 3: Overall Functional Block Diagram

The EmoKey software acts as a bridge connecting Emotiv Figure 6: EmoKey Creation
Control panel to the Intel Arduino IDE. Each Action
responsible for a particular command is mapped with any These EmoKey's along with Intel Edison allow a disabled
one character of the standard keyboard. After the optimal person to communicate their basic needs using pre-defined
connection of electrodes, EEG signals are read by the commands. These commands can also be used to control
headset. After feature extraction, the obtained variable is an application or hardware to serve their personal needs.
compared against the stored EmoKey variable set. In case
of a hit, the particular action is performed and then starts IV. EXPERIMENTAL RESULTS
to read EEG signal again.
Overall working of the Emotiv EPOC headset has IoT Kit agent runs a daemon on the device,
been analyzed and tested using the company firmware listening for simple messages from other processes and
Emotiv SDK community Edition as shown in figure 4. handling the necessary message formatting and security to
Facial Expressions, Mental commands were tested and send observations to the cloud. The putty terminal is used
trained using the Emotiv Control Panel and EmoBot to configure Intel Edison and establish connection with
applications as shown in figure 5. IoT analytics server. Iot kit-Admin provides utility
functions, such as testing the network, activating a device,
registering time series, registering components, providing
proxy information regarding the network, and sending test
observations.
In the cloud dashboard, components are added for
data access and analysis. A component is either a time
series which consists of sensor observations sent from the
device, or an actuator, which allows commands to be sent
to a device. In Figure 7, the Catalogue of devices
registered is shown.

Figure 4: Testing Emotiv EPOC headset

Figure 5: Training Commands via SDK

EmoKey links any detected EEG wave pattern into any Figure 7: dashboard – Catalogue containing components
registered

260
After proper logging of device and communication VI. FUTURE RESEARCH DIRECTIONS
channel, data obtained from the sensors are sent to the
cloud using 'Iotkit' library from Intel Arduino. Emotional Stability is considered as the degree to which a
Intel Edison is connected to Twilio by creation of person is well adjusted, calm and secure. Emotions
a Twilio account and installing Curl and Crypto libraries profoundly influence learning. Positive emotions are
needed for its functioning. The programming part for emotions that are experienced as pleasant and vary in
interfacing Edison with Twilio is done through C++. terms of the physiological and cognitive activation.
Positive emotions influence learning by affecting students’
Proper Data logging of the sensor data was attention, motivation, use of learning strategies and self-
achieved. These data can be visualized in form of graphs regulation of learning. Negative emotions draw students’
in the cloud. The user-defined commands controlled by attention away from learning and concern about
the EEG waves are displayed in the Arduino IDE Serial performance and its consequences, thereby reducing task-
Monitor. This system is capable of synthesizing voice focused attention. Negative emotions are a major factor
using a Text-to-speech (TTS) Converter. Using explaining why many students do not live up to their
EmoKey’s, one can emulate voice-synthesis for disabled potential and fail to pursue the educational career that
people by using an Interface enabling sentence formation would correspond to their abilities and interests.
and machine learning with Inputs from the EEG data. EEG data can be used to monitor student's
Figures 8 and 9 show the observations logged from the emotional behaviour and provide emotional feedback to
sensors. These collected data are used for further analytics the students. This feedback enables students to improve
and plotting. their emotional stability and learning process. Brainwave
analysis can also be extended to ascertain one's learning
style at an early age.
Study of an individual's response to various
visual, auditory and kinaesthetic stimuli can be used to
determine the most preferred learning style for effective
assimilation of knowledge.

VII. ACKNOWLEDGEMENT

We acknowledge Indian Institute of Science, Bangalore

and Indian Academy of Science, Bangalore for their
constant support for the successful completion of our
Figure 8: data logging done in the IoT Analytics cloud project.
VIII. REFERENCE

1. Muhammad N. Fakhruzzaman, Edwin Riksakomara,

Hatma Suryotrisongko, 'EEG Wave Identification in
Human Brain with Emotiv EPOC for Motor Imagery',
Procedia Computer Science 72, 269-276, ELSEVIER,
2015.
2. Rishab Shah, Harish N.S, Dr.G.N. Rathna, 'A SMART
HOME CONTROL AND MONITORING SYSTEM FOR
THE DISABLED',Volume 6, Issue 6, June (2015), pp. 26-
35, International Journal of Electronics and
Communication Engineering & Technology (IJECET),
2015.
Figure 9: Graphs depicting data recorded in the 3. Sumit Soman, B K Murthy, 'Using Brain Computer
IoTAnalytics cloud Interface for Synthesized Speech Communication for the
Physically Disabled', Procedia Computer Science 46, 292-
V. CONCLUSION 298, ELSEVIER, 2015.
4. Tyler C. Major, Dr. James M. Conrad, 'A Survey of
The designed system can recognize pre-defined Brain Computer Interfaces and Their Applications', IEEE,
basic commands from the quadriplegic, interact with 2014.
teacher and peers. The overall system helps quadriplegics 5. Ehsan Tarkesh Esfahani, V. Sundararajan,
to lead an Independent life and successful career. In 'Classification of primitive shapes using brain–computer
addition, the system periodically logs vital healthcare data interfaces', Computer-Aided Design Volume 44, Issue 10,
such as temperature, heart rate and blood pressure. Based October 2012.
on the data acquired from the sensors, alert messages are 6. Jonathan S. Brumberg, Alfonso Nieto-Castanon,
displayed in the Arduino IDE Serial monitor. This system Philip R. Kennedy, Frank H. Guenther, 'Brain-Computer
can be expanded by monitoring the emotional state of the interfaces for speech communication', Speech
patient and their sleep cycle. Analyzing emotional states Communication 52, 367–379 (ELSEVIER) 2010.
and sleep cycle gives a better perspective of patients 7. F Lotte, M Congedo, A Lecuyer, F Lamarche and B
mental well-being, thus a complete life-care system. Arnaldi, 'A review of classification algorithms for EEG-

261
based brain–computer interfaces',J. Neural Eng. 4, R1–
R13 JOURNAL OF NEURAL ENGINEERING, 2007.
8. Upkar Varshney, 'Pervasive Healthcare and Wireless
Health Monitoring',Mobile Netw Appl 12:113–127,
Springer Science + Business Media, 2007.
9. Pari Jahankhani, Vassilis Kodogiannis and Kenneth
Revett, 'EEG Signal Classification Using Wavelet Feature
Extraction and Neural Networks', IEEE John Vincent
Atanasoff 2006 International Symposium on Modern
Computing.
10. Gerwin Schalk, Dennis J. McFarland, Thilo
Hinterberger, Niels Birbaumer, and Jonathan R. Wolpaw,
'BCI2000: A General-Purpose Brain-Computer Interface
(BCI) System', IEEE TRANSACTIONS ON
BIOMEDICAL ENGINEERING, VOL. 51, NO. 6, JUNE
2004.
11. Kern SE, Jaron D, 'Healthcare technology, economics
and policy: an evolving balance', IEEE Eng Med Biol Mag
22:16–19, Jan–Feb, 2003.
12. http://www.emotiv.com, http://www.neurosky.com
and IoT Analytics guide from Intel
(https://software.intel.com/en-us/intel-iot-platforms-
getting-started-cloud-
analytics#Setting_up_your_IoT_Analytics_account)

262