2017 International Conference on Networks & Advances in Computational Technologies (NetACT) |20-22 July 2017| Trivandrum

Brain Computer Interface: Design and

Development of a Smart Robotic Gripper for
a Prosthesis Environment
* * 1
Prathibha R , Swetha L , Dr Shobha K R

*Student, Department of Telecommunication Engineering

*Equally Contributed Authors
1
Associate Professor, Department of Telecommunication Engineering
M S Ramaiah Institute of Technology
Bangalore, Karnataka, India

Abstract—Brain Computer Interface (BCI) technology is a perform control applications to interact with the world. The
tremendously growing research area having various application. main objective of BCI research [16] [17] is to provide
Its involvement in medical field lies with prevention and neuronal communication medium to help people who are severely
rehabilitation for severe injuries. BCI systems try to build a paralyzed or are suffering from Locked-In Syndrome(LIS),
pathway between the human brain and external devices,
Amyotrophic Lateral Sclerosis(ALS), or spinal cord injury.
providing the patients a media for communication with the world
Thus eliminating the scope of depending on others to a less Once the signal is acquired from the brain, BCI system gauges
degree. BCIs are often oriented at research, mapping, assisting, it and feature extraction is performed and it is made available
augmenting of human cognitive or sensory-motor functions. The to perform useful tasks.
growth of BCIs has increased in the recent years, paving way for
research and aiming to be more accessible for the people. Our human brain is highly complex and is made up of 100
billion neurons. There are many types of neurons in our brain
such as motor neurons, sensory neurons etc. These neurons get
This paper describes a method to control a prosthetic moving fired up while generating a response for some stimuli and
robotic gripper with the use of brain waves. Brain waves,
generate an electrical signal which is detected by the
detected using Neurosky Mindwave Mobile headset, are
processed, and sent to the microcontroller for controlling real-
electrodes and can be interfaced to control several devices.
time application. Thus, this system provides a method for the BCIs can be controlled through the direct use of brain signal,
Amyotrophic Laterals Sclerosis (ALS) patients, or people who bypassing the need for volitional muscle activity as a control
have little or no reliable muscle movement, to control the paradigm. Extraction of significant patterns from the
prosthetic gripper and lead a self- reliant life to some extent. ElectroEncephaloGraphy (EEG) signals is the main task with
BCI.
Keywords—BCI, EEG sensor, ALS, Bluetooth, Microcontroller,
robotic arm/gripper. Many developments are currently taking place in BCI
research to improve applications related to neuro-prosthetics
which basically aim at restoring movement [4][5]in patients
I. INTRODUCTION suffering from paralysis or develop devices to help them , such
In the recent years, tremendous progress has occurred in as computer interfaces, interacting devices or robotic arm
the field of cognitive neuroscience and human-computer [2][3]. BCIs are increasingly being integrated with other
commercial assistive technology on an experimental basis
interfaces, which has increased our competency to collaborate
with the human mind. People who suffer from motor neuron [18] and can form an interface that is based on brain signal
diseases face a lot of difficulty to interact with and incorporated within the framework of other existing
others around them. With the help of BCI [1], a assistive technology, increasing the accessibility of such
devices. [19]
straightforward link between human brain and other physical
device is achieved. There are two divisions of BCIs: those
which are implanted in the brain corresponds to invasive type Brain signals can be acquired in several forms, including
and the other is the non-invasive method. Both these methods electrical (e.g., EEG) or magnetic fields (e.g., Functional
have their own advantages and disadvantages and can be used Magnetic Resonance Imaging (FMRI)) or Functional Near
to acquire brain signals. Infrared Spectroscopy (FNIRS). BCI is developing
tremendously in the field of research contributing to many
applications. Recently , brain–computer interfaces using
A Brain-Computer interface [1] is a communication system
in which the activities of the cerebrum can alone be used to noninvasive methods has gained much interest. There has been

978-1-5090-6590-5/17/$31.00 ©2017 IEEE 278

a great progress in recording, processing and application of III. METHODOLOGY
EEG signals.
The main aim of this work is to develop a brainwave The most fundamental idea of BCI is to convert the brain
controlled prosthesis environment for the severely paralyzed patterns or the cerebral activity into respective scenarios
like the ALS patients or patients with motor disability. To which can be used for various control applications. In this
execute the prosthesis environment brain signals are utilized project, we are using a non-invasive environment to develop a
as the primary input. The suggested system has a single Brainwave Controlled Prosthesis gripper. Neurosky
channel MindWave Mobile sensor which is used to collect the Mindwave Mobile headset is used to detect the brain waves in
EEG signals. The process of feature extraction is performed to real time.
obtain two distinct values corresponding to blink strength and
attention level. These values are used to control the prosthetic i. Neurosky Mindwave mobile Headset
gripper in real time, to guide the gripper in right, left, forward, The Neurosky [7] Mindwave headset, as shown in Fig. 2, is
backward, up ,down, open and close [2][3]. This paper powered by AAA battery and communicates via Bluetooth. It
furnishes an overview of the methodology involved, design has a Think Gear chip [18] [19] inside and uses a TGAM
and experimental setup to achieve the same. module (Think Gear Application specific IC). It detects raw
brainwaves and outputs processed EEG signals consisting of
II. DESIGN Alpha, Beta, Delta, Theta and eSense parameters like
Attention, Meditation, Blink strength. The blink values range
Neurosky Mindwave Mobile headset, a dry and single from 0-255 and attention values ranges from 0-100. The
channel electrode is used to detect the brain waves in real frequency range of the headset is typically between frequency
time. These brain wave signals are used to develop a 2.42 – 2.472 GHz. Sampling rate of EEG signal is done at
Brainwave Controlled Prosthesis gripper. Neurosky 512Hz and its maximum power is found to be 50mw.
Mindwave Mobile headset uses Think Gear Connector
technology which communicates with Bluetooth enabled
devices that have the software installed. The block diagram of
the system is as shown in Fig 1.

The platform used to extract real-time data is Python. A

connection between Neurosky Headset and python is
established using Telnet Communication to collect and parse
the data. The collected data undergoes corresponding signal
processing, data extraction and feature recognition. The
processed data, in real time, is then forwarded to the Teensy
microcontroller. Using this, further data extraction is done to
drive the corresponding sensors. It is further coded depending Fig. 2. Neurosky Mindwave Mobile
on the blink values and the robotic gripper is thus controlled to
ii. Think Gear
pick and place objects by the user.
Think Gear technology [19] is a patented NeuroSky’s
technology which measures and analyses the EEG signals. The
TGAM (Think Gear) module consists of an inbuilt small chip
which helps in filtering the electrical noise. The headset can
measure the wearer's state of mind. The processed signal is
made available for applications. Further, mapping of different
brain signals is performed to achieve various control activities.

iii. Signal Detection

The different states of the brain corresponds to different
patterns of the cerebral activity. These patterns give rise to
waves which are distinguished with the help of amplitudes and
frequency pattern generated. The interaction occurring
between the neurons produce a microscopic electrical
discharge These patterns lead to waves characterized by
different amplitudes and frequencies. These processed signals
are then transmitted via Bluetooth communication to various
applications.

Fig.1. Block Diagram

279
 iv. Python interface- Telnet communication Fig. 3. Plot of real time brain waves

A python module establishes connection between

NeuroSky headset and Laptop using Telnet communication.
Telnet is a protocol used on local area networks to provide a
bidirectional communication facility. In general, the Telnet
protocol establishes a TCP connection using which a program
on one host (refers to Telnet client) may gain accessibility of
the resources of another host (refers to Telnet server).

v. Signal Acquisition
The headset captured the EEG signals and are transmitted
to the Laptop with the help of Bluetooth communication. The
headset performs detection, analysis and processing of the
signal and transmits the processed data to laptop.

vi. Pyserial
Pyserial module encapsulates the access for the serial port.
It provides an access to the communication port of Teensy
microcontroller. Different commands like read and write are
used to establish this communication.
The most common and widely used system is one
vii. Teensy consisting of multiple channels, which are located on the
Teensy 3.0 is a 32-bit ARM Cortex-M4 development board corresponding regions of the brain. these electrodes. In this
running at 48MHz. It has 34 Digital I/O Pins out of which 10 system we are adopting a single electrode to measure the EEG
pins are shared with analog and 3 UARTs (serial ports) and 10 signals. The channel A1 is related with sensor ear clip, FP1
PWM outputs. It has 128K Flash Memory, 16K RAM and 2K relates to the sensor placed on the forehead which taps the
EEPROM.. It supports a USB host mode, an I2S audio brain waves and T4 is where the reference point of the headset
interface (for high quality audio interface), SPI and I2C. The is located. This is shown in Fig. 2.
serial ports of Teensy is set to a baud rate in accordance with
the Pyserial command. Thus, data is transmitted from the
python module to Teensy using the digital pins, with the help
of the Arduino IDE (Teensyduino).

viii. Actuators
The data obtained from the Teensy microcontroller is used
to drive the motors using L293D motor drivers. These motors
perform action as per the data obtained and processed from
Neurosky and Python interface. Thus, real time brain waves
are used to control the prosthetic mobile robotic gripper.

IV. DETECTION OF EYE BLINK SIGNAL

Eye blink play a critical role to people suffering from

motor neuron diseases. It helps to control many applications.
[8] [10] [11]. Devices have been designed to understand the
blinks and help people interact with the external world more
effectively. Every eye blink has certain distinct feature such as
frequency of operation, amplitude and time elapsed between
closing and opening of the eyes. Attention values correspond
to increased concentration levels. We are using the blink
strength and attention values gathered from the processed
signal from the NeuroSky Mindset headset.
Fig. 4. Signal Display

280
 The brain signals detected from the headset is as shown in
Fig.3. The signals measured from the sensor are decomposed
and recorded. The blink spikes, shown in Fig 4, was plotted to
filter the noise in the blink signal using OpenVibe Platform.
The EEG signal was acquired in real time using an acquisition
server which connects with the Neurosky headset by enabling
the local host address. An OpenVibe design scenario was
created using a simple temporal filter of an order N. Thus,
distinct blink signals were captured and classified.

The blink strength values range from 0-255. A higher

number indicates a strong blink while a smaller number
indicates regular/lighter blink. The frequency of blinking is
often correlated with nervousness. Based on these values, it is
coded to perform the control applications as shown in Fig 5.

Fig 5. Blink and Attention status to perform control application

V. ARCHITECTURE OF THE SYSTEM

The architecture flow is as shown in Fig 6. The brain waves

detected and processed by the Neurosky EEG sensor, are Fig. 6. Flowchart of the system
transmitted via Bluetooth communication to Laptop. The
platform used to extract real-time data is Python. A Telnet
communication is established between Neurosky and Python Algorithm
to collect and parse the data. The values of the different mind-
states and mind-waves corresponding to a frequency and
activity are extracted. Step 1: Establish serial communication between the laptop and
microcontroller.
The signal flow of the system is as follows: if the Telnet Step 2: Establish telnet communication to Neurosky headset
communication isn’t established as shown in Fig 6, the (local host: 13854) and enable Think Gear Socket Protocol.
connection is retried. Once the connection is established the Step 3: Read and parse relevant eSense parameters like
brain waves are continuously collected. The parsed data are attention and blink strength values.
read and forwarded to the Teensy microcontroller which Step 4: Enable pyserial communication.
drives the actuators based on these values. Step 5: Transmit the parsed control parameters from the
program module to the microcontroller.
Step 6: Using the received parameters in microcontroller
perform the corresponding actions.

281
 VI. EXPERIMENTAL SETUP VII. CONCLUSION

The experimental setup, shown in Fig.7 and Fig. 8, consists The proposed system was achieved using a dry electrode
of a cylindrical robotic gripper, providing movement in which consisted of a single channel and was used to acquire
horizontal and vertical axes. To perform control action, the the EEG signals in real time. Based on the different brain
subject was asked to blink and the EEG headset picked up activities like attention and blink strength values, motion of
these blinks in real time. Training the subject for some time is the prosthetic gripper is controlled in real time. A major
necessary to synchronize the blink values in real-time. benefit of this pick and place system is consistency. It
provides assistance to the ALS patients and can also be used
A Telnet communication was established between the as a device to help people who have lost motor skills due to
headset and the python interface; such that data was read in stroke, lead a normal life to some extent. The robot can be
real time on the python shell. The extracted brain signals were easily programmed to provide multiple applications if
recorded and data was exported to python. These data were required. This system thus provides an assisted living method
further coded in python to drive the motors with the help of for the severely disabled or people with motor neuron
Teensy microcontroller and pyserial communication. diseases.
Depending on the presence of blink values detected, the
gripper was observed to perform actions such as picking and Further scope of improvement of this work involves
placing at the desired location, as guided by the subject using increasing the accuracy of data acquisition by using multiple
the headset. channel sensors, as accuracy plays a very important role in
controlling real-time applications. Further the entire system
Thus, with a single sensor EEG headset, a smart robotic can be made wireless to give more freedom of usage to the
gripper for a prosthesis environment was designed. The paralysed patients. An increase in training time could also be
motion of the gripper was controlled effectively. tested to provide the user a more precise way to control the
arm.

VIII. REFERENCES
[1] Erik Andreas Larsen, “Classification of EEG Signals in a Brain-
Computer Interface System”, NTNU
[2] Double Degree Program in Information and Communication
TechnologiesEngineering,“BrainComputerInterface
System”,Helwan University
[3] Dario Assante and Claudio Fornaro,” Involving graduating
engineers in applying a commercial brain-computer interface to
motorized wheelchair driving”, 2015 IEEE Global Engineering
Education Conference (EDUCON)
[4] B.JenitaAmaliRaniandA.Umamakeswari,”
Fig.7. Experimental setup showing the robot picking up the object from Electroencephalogram-based Brain Controlled Robotic
position A Wheelchair”, Indian Journal of Science and Technology, Vol
8(S9), 188–197, May 2015
[5] L. Ramya Stephygraph1, N. Arunkumar2 and V. Venkatraman3
,“Wireless Mobile Robot Control through Human Machine
Interface using Brain Signals”, 2015 International Conference on
Smart Technologies and Management for Computing,
Communication, Controls, Energy and Materials (ICSTM), Vel
Tech Rangarajan Dr. Sagunthala R&D Institute of Science and
Technology, Chennai, T.N., India. 6 - 8 May 2015. pp.596-603.
[6] Dany Bright, Amrita Nair, Devashish Salvekar, Prof. Swati
Bhisikar,” EEG-Based Brain Controlled Prosthetic Arm”, 2016
Conference on Advances in Signal Processing (CASP)Cummins
College of Engineering for Women, Pune. Jun 9-11, 2016.
[7] Jozsef Katona, Tibor Ujbanyi, Gergely Sziladi,” Speed control of
Festo Robotino mobile robot using NeuroSky MindWave EEG
headset based Brain-Computer Interface “7th IEEE International
Conference on Cognitive Info-communications (CogInfoCom
2016) Wrocław, Poland
[8] Sravanth Kumar, Vivek Kumary and Bharat Guptaz,” Feature
Extraction from EEG Signal through One Electrode Device for
Fig.8. Experimental setup showing the robotic gripper moving from Medical Application”,2015 1st International Conference on Next
position A and placing the object at position B. Generation Computing Technologies (NGCT-2015) Dehradun,
India, 4-5 September 2015

282
 [9] Sudarsanan.K, Dr. S.Sasipriya,” Controlling a Robot Using Brain [16] Studies in Computational Intelligence 647 Emerging Trends and
Waves”, 2014 IEEE International Conference on Computational Advanced Technologies for Computational Intelligence Edited by
Intelligence and Computing Research Liming Chen, Supriya Kapoor, Rahul Bhatia
[10] Wojciech SAŁABUN,West Pomeranian University of Technology, [17] Computational Intelligence in Data Mining—Volume 1
Szczecin, “Processing and spectral analysis of the raw EEG signal Proceedings of the International Conference on CIDM, 5-6
from the MindWave” December 2015 Editors: Behera, Himansu Sekhar, Mohapatra,
[11] Susmita Das, Sayan Kumar Swar, Shrisom Laha, Subhankar Durga Prasad (Eds.)
Mahindar, Suchetana Halder, KoushikHati, Sandipan Deb” Design [18] Zickler C, Riccio A, Leotta F, Hillian-Tress S, Halder S, Holz E, et
approach of Eye Tracking and Mind Operated Motorized System” al. A brain-computer interface as input channel for a standard
,International Journal of Innovative Research in Science assistive technology software. Clin EEG Neurosci. 2011
,Engineering and Technology (An ISO 3297: 2007 Certified Oct;42(4):236–44
Organization) Vol. 5, Issue 8, August 2016 [19] Thompson DE, Gruis KL, Huggins JE. A plug-and-play brain-
[12] Kamlesh H. Solanki1, Hemangi Pujara2,” Brainwave Controlled computer interface to operate commercial assistive technology.
Robot”, International Research Journal of Engineering and Disabil Rehabil Assist Technol. 2013 Apr 16
Technology (IRJET) [20] Neurosky Think Gear http://developer.neurosky.com/
[13] Cong Wang1, Bin Xia1,2, Jie Li2, Wenlu Yang1 Dianyun Xiao1, docs/doku.php? id=thinkgear_connector_tgc
Alejandro Cardona Velez1, HongYang1 “Motor Imagery BCI- [21] Brain wave signal of Neurosky.Inc. - http://sleepshepherd.com/wp-
based robot arm system”, 2011 Seventh International Conference content/uploads/2016/02/neurosky-vs-medical-eeg.pdf
on Natural Computation [22] Think Gear Socket Protocol http://developer.neurosky.com/
[14] Arulprasath.P, Harihara Ganesh.S, Hariharan.B, Hariharan.S, docs/lib/exe/fetch.php? media=thinkgear_socket_protocol.pdf
Mohankumar.R “Interaction of Human with Robotic Arm”, [23] Teensy 3.0 Datasheet, www.pjrc.com
International Journal of Innovative Research in Computer and [24] L293D Datasheet, http://www.ti.com/product/L293D
Communication Engineering (An ISO 3297: 2007 Certified
Organization) Vol. 4, Issue 3, March 2016
[15] Joy Bose,Amit Singhai,Anish Anil Patankar,Ankit Kumar,”
Attention sensitive Web Browsing”

283