See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/328686656

A New Design Approach for Gesture Controlled Smart Wheelchair Utilizing

Microcontroller

Conference Paper · October 2018

DOI: 10.1109/ICISET.2018.8745607

CITATIONS READS

16 2,667

4 authors, including:

Abu Tayab Noman Mohammad Emdadul Islam

International Islamic University Chittagong RPTU - Rheinland-Pfälzische Technische Universität Kaiserslautern Landau
18 PUBLICATIONS   85 CITATIONS    10 PUBLICATIONS   41 CITATIONS   

SEE PROFILE SEE PROFILE

Humayun Rashid
Revvity
43 PUBLICATIONS   244 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

DTMF Controlled Acoustic and Visual Detecting Surveillance Robot View project

A New Design Approach of Home Automation System for Patients with Physical Disability to Reduce Water Wastage and Power Consumption using Renewable Energy
View project

All content following this page was uploaded by Abu Tayab Noman on 02 July 2019.

The user has requested enhancement of the downloaded file.

2018 2nd Int. Conf. on Innovations in Science, Engineering and Technology (ICISET)
27-28 October 2018, Chittagong, Bangladesh

A New Design Approach for Gesture Controlled

Smart Wheelchair Utilizing Microcontroller
Abu Tayab Noman Md. Salman Khan Mohammad Emdadul Islam
Electrical and Electronic Electrical and Electronic Electrical and Electronic
Engineering (EEE), Engineering (EEE), Engineering (EEE),
International Islamic University International Islamic University University of Science and Technology
Chittagong (IIUC) Chittagong (IIUC) Chittagong (USTC)
Chittagong, Bangladesh Chittagong, Bangladesh Chittagong, Bangladesh
[email protected] [email protected]
Humayun Rashid
Electrical and Electronic
Engineering (EEE),
International Islamic University
Chittagong (IIUC)
Chittagong, Bangladesh

Abstract— Every year, a large number of people become Most of the users feel comfortable with the joystick
lame due to a road accident and unable to walk normally. controller to control an electric wheelchair. But the joystick
Wheelchair is the best assistive device used by the older and controlled wheelchair cannot be operated by the person
differently abled people who cannot walk normally. The whose upper half of the body is disabled [3]. In voice
driving and controlling of traditional manual wheelchair are
control system, there are problems in the noisy environment.
much harder task. The modern wheelchairs like joystick-
controlled and voice-controlled wheelchair is a bit difficult to The accelerometer sensor is the tilt detecting sensor used for
operate for certain people like the older and physically weak hand gesture recognition. The capacitive touch control
people. Moreover, this wheelchair is not cost effective. So the system allows the rider to control the chair by his finger.
aim of this paper is to make such a cost-effective electronic When the rider fails to control the chair, there needs a safety
gesture-based wheelchair which will be easy to operate rather system to avoid an unwanted collision. This paper is to
than the joystick input to control a wheelchair using in-built describe a smart wheelchair utilizing smartphone is
gesture function of a smartphone and touch sensor. introduced to control the navigation of wheelchair based on
ATMega328 used as the processor long with the L298N motor voice and gesture development [4].
driver, DC Gear Motor, Ultrasonic Sensor, TTP224 Capacitive
Touch Sensor, Bluetooth Module and IP Camera. Special
features of this wheelchair are that obstacles on the way of the Rakhi A. Kalantri and D. K. Chitre published a research
wheelchair can be detected which can avoid the collision proposal [5] that the wheelchair can be controlled in four
between the wheelchair and that detected obstacle. Another directions by just tilting acceleration sensor. The ultrasonic
feature of this chair is to use an IP camera that gives visual and sensors are used to control the wheelchair movement by
acoustic information to the guardian of the riding person. taking over some responsibility for steering and avoiding
collision with objects until the user is able to handle the
Keywords—Microcontroller, Gesture Control, Smart situation. By applying the simple formula, they calculated
Wheelchair, IP Camera
the amount of tilt and output of tilt will decide to move in
which direction.
I. INTRODUCTION
More than 100 million people around the world, with Prof. Vishal V. Pande et al. [6] developed a wheelchair
physical inabilities, require the help of a wheelchair control system for the handicapped persons which can be
however just a little level of them really possess or have the controlled with his hand movement or his hand gesture
freedom of getting one. Though manual wheelchairs have recognition using Acceleration technology. They employ a
turned out to be valuable for the debilitated however it has sensor ADXL202, which controls the wheelchair hand
just filled the need of individuals with minor inabilities [1]. gestures made by the user and interprets the motion intended
by the user and moves accordingly.
Currently, a lot of assistive and guidance systems
available out there in the market which makes more Gaurav Kumar Soni et al. published a paper on Hand
comfortable navigation for the physically disabled person. Gesture Recognition Based Wheel Chair Direction Control
The systems that are developed are highly competitive in Using AVR Microcontroller [7]. They designed and
replacing the old traditional systems [2]. developed a hand gestured based wheelchair that can be
easily controlled by the help of a gesture recognition system.
There are many assistive systems to control the They designed their wheelchair with the help of MEMS 9
wheelchair like Joystick, voice control and much more. accelerometer sensor.

978-1-5386-8524-2/18/$31.00 ©2018 IEEE

 In a developing country like Bangladesh, every year, movement of motors to avoid the collision to save the
most of the people become paralyzed by road accident. In wheelchair. Here power supply is a 12 volt dc battery.
2015, the number was 47,437 and in 2014, the number was
46, 558 [8]. Most of them are poor. So, a low cost, simple
and easily controllable device is needed to control the
wheelchair.

In our earlier research, we have designed several assistive

devices for physically disabled like visually impaired person
Figure 2: Functional block diagram of IP camera broadcasting.
[9]-[10] and health monitoring device [11]. In this paper, a
Gesture-controlled wheelchair is developed to mitigate the
drawbacks of handicapped person (like the lower part of the Figure 2 shows the functional block diagram of IP
body paralyzed person) for automatic navigation with a little camera broadcasting system. The IP camera is linked with a
bit sense of control their hand. Taking advantage of fixed IP and helps to observe the user by the guardian of the
technological evolution, an external IP camera is mounted to rider continuously from remote area and response for
observe the movement of the user on the wheelchair. emergency help and security purpose.

II. METHODOLOGY III. CIRCUIT DEVELOPEMENT

The goal of this paper is to assist lame peoples whose Figure 3 illustrates the circuit diagram of the proposed
lower part of the body is paralyzed to navigate without Gesture controlled wheelchair.
having help from others. An Ultrasonic Sensor is also
employed to detect obstacles. As designed, the robot can
also be controlled via a touch module along with gesture.
An IP camera is also used to monitor the surroundings of its
user from a remote area.

Figure 1: Block diagram of the control system.

Figure 1 demonstrates the functional block diagram of

the proposed system. The wheelchair can be control via
gesture and via touch module. The working function of this Figure 3: Circuit diagram of the implemented device.
wheelchair can be represented by this block diagram. The
smart phone of the rider is connected through Bluetooth Which is consists of a Arduino Uno (ATMega328),
with microcontroller. In build gesture function of the smart Ultrasonic Sensor, a pair of DC Gear Motor, a Motor Driver
phone give signals according to its tilting position and the Module L298, a Bluetooth Module HC-05 and a Capacitive
microcontroller gives prescribed output to the motor driver. Touch Module. Pin 11 and 12 of Arduino Uno is connected
to Left Motor through pin 1 and 2 of Motor Driver Module.
In addition, as an alternative control system, a 4 channel Pin 8 and 9 of Arduino Uno is connected to right motor
capacitive touch module is connected with the through pin 3 and 4 of Motor Driver Module. Trigger pin of
microcontroller. By touching per channel, it will give four Ultrasonic sensor is connected to pin 7 and Echo pin is
unique inputs to the microcontroller. Microcontroller gives connected to pin 6 of Arduino Uno. Output pins of capacitive
prescribed output to motor driver. touch module is connected to the pin 2,3,4 and 5 of the
Arduino Uno respectively. Tx and Rx pin of Bluetooth
By the rotation of the two motors, the device can be Module is connected to the pin 0 and 1 of Arduino Uno
easily controlled in four directions. The ultrasonic sensor respectively. A 12volt battery is connected as the power
will detect the obstacles in front of the chair and stop the supply unit.
 IV. OPERATRING PRINCIPLE Then it will be directed to obstacles searching. If
The user can control the wheelchair via his smart phone smartphone control status is “Yes” then the program will
or the touch pad. During the touch pad operation, there are search for Bluetooth. If the Bluetooth status is on then it will
four channels in the capacitive touch module. “1” is for go for obstacle searching. If any obstacle is found, the
forward, “2” is for backward, “3” is for left and “4” is for wheelchair will surpass the obstacle by changing the motors
right. The user just needs to touch his finger on the condition. If no obstacle is found then the program will go
particular channel to navigate the wheelchair. To control the for corresponding input pattern. If no input is available, the
wheelchair using in build gesture function of smart phone, program will stop executing.
the phone have to connect through Bluetooth using “BT
robot controller” smartphone software. After getting VI. RESULT ANALYSIS
connected through Bluetooth, he will be able to control the
wheelchair by just tilting his smartphone. Capacitive touch A. Hardware Prototype
pad or Smartphone gives corresponding signal to the
microcontroller. According to the program, it gives output to As the output of the device is motor control, no
the motor driver to control the rotation of two motors. The graphical representation is required. Figure 5(a) shows the
ultrasonic sensor will detect obstacles in front of the chair prototype of the smart wheelchair. The side view of the
and it will stop the motors as well as the wheelchair. The device containing Capacitive Touch Module is shown in
logic to control the motors are given in Table I. Figure 5(b).

TABLE I. CONDITIONS TO NAVIGATE THE WHEELCHAIR.

Position Arduino Pin No. Direction

#8 #9 #10 #11

Normal 0 0 0 0 STOP
Forward 1 0 1 0 FORWARD
Backward 0 1 0 1 BACK
Left 0 1 1 0 LEFT
Right 1 0 0 1 RIGHT

V. ALGORITHM DEVELOPEMENT
Figure 4 illustrates the flowchart of the proposed system.

Figure 5(a): The prototype of the Figure 5(b): The side view of the
smart wheelchair. smart wheelchair.

Figure 6: The main hardware connection.

Figure 4: The flowchart of the proposed system.
Figure 6 shows the main hardware connection. The
At very first, the program will check the smartphone system contains Arduino Uno, Motor Driver, Bluetooth
control is either enabled or not. If it is “No” then the Module, Power Supply and Motors. This hardware
program will check the capacitive touch module status. arrangement is placed under wheelchair. The circuit will
operate when it gets the signal from the smartphone through TABLE II. OUTPUT DIRECTION OF CAPACITIVE TOUCH MODULE.
Bluetooth or the capacitive touch module which is mounted Channel No. Direction
on the handle of the wheelchair.
Channel 1 Forward
Figure 7(a) shows the Ultrasonic sensor which is used to Channel 2 Backward
detect obstacles. Figure 7(b) shows the obstacle detection. Channel 3 Left
Channel 4 Right

C. Camera Performance

To minimize the cost, we have proposed a smart phone

to use as an IP camera. IP camera android application is
available in Google play store. By this application, one can
easily use his smartphone camera as an IP camera. Figure 9
shows the position of camera on the wheelchair.

(a) (b)
Figure 7: (a) Ultrasonic sensor to detect obstacles.
(b) Obstacle detection.

B. Controlling Unit

There are Two control unit is developed: a) Smartphone

Control, b) Touch Control.

a) Smartphone Control

In smartphone control, an android application is needed.

This android application named “BT Robot Controller” is
available in Google Play Store. Tilt movement angle can be
detected by the accelerometer of the smartphone, just need
to tilt smartphone in forward, backward, left and right to
Figure 9: Position of camera on the wheelchair.
control the direction of the chair. The application is
connected by HC-05 Bluetooth module with
microcontroller. Figure 8 shows the tilting position of the
smart phone. Figure 8 shows the tilting position of
smartphone

Figure 10(a): IP camera is capturing.

Figure 8: Tilting position of smartphone for navigation. Figure 10(a) shows the picture captured by the camera. It
can be easily transmitted to cloud streaming by mobile
b) Capacitive Touch Control internet. So, the cost of internet connection and other
arrangement to set up an IP camera in the chair will be
If the user is unable to move his hand, then he will be minimized. Figure 10(b) shows that the picture captured by
able to control the wheelchair by using Capacitive Touch the IP camera can be observed from anywhere. It ensures
Pad. Here a four channel capacitive touch module is used. the location and condition of the wheel chair rider to his
By touching each channel, the chair can be moved in four guardian. Moreover, the rider will be able to ask for any
directions. Table II shows the direction indicated by help to his guardian since, this camera is able to transmit
capacitive touch module. audio sound as well as.
 VIII. CONCLUSION
The project relates the smartphone and touch control
wheelchair. The ultimate goal of this paper is developing a
easily controllable, cost effective, low power consumed
assistive robotic wheelchair to assist physically challenged
persons. In this paper, a cost friendly, easily controllable
robotic wheelchair is developed with a built-in programmed
Arduino based on ATmega32 microcontroller. Besides
controlling wheelchair, there are some additional features is
added like obstacle detecting sensors to shut-down the entire
system in case of imminent collision, security camera.
However, there are still many improvements to be made:

For handicapped people who are not able to move their

hand, this wheelchair can be developed with the tongue
motion driver for the movement of wheelchair. Speed control
and Autonomous control feature can be added later. Self-
power generating from the rotation of the wheel can be done
in near future. And this device can be made more flexible by
placing solar power supply so the anxiety about charging the
battery can be reduced.
REFERENCES
Figure 10(b): Captured video can be accessed from anywhere. [1] R. Akmeliawati, F. S. Ba Tis and U. J. Wani, "Design and
development of a hand-glove controlled wheel chair", in 4th
VII. COMPARISON WITH PREVIOUS WORKS International Conference on Mechatronics (ICOM), 2011.
[2] G. Bourhis, K. Moumen, P. Pino, S. Rohmer and A. Pruski, "Assisted
Sharaddha Uddhay Khadilkar et al. [8] presented a paper navigation for a powered wheelchair", in Proceedings of IEEE
which can control by android phone using build in Systems Man and Cybernetics Conference - SMC, France, 1993, pp.
functions. The control system using voice, Gesture and 553-558.
Touch screen. But the limitation of the device is, a lot of [3] M. Shibata, C. Zhang, T. Ishimatsu, M. Tanaka and J. Palomino,
sensors took place in this system. So the controlling of the "Improvement of a Joystick Controller for Electric Wheelchair
User", Modern Mechanical Engineering, vol. 05, no. 04, pp. 132-138,
device is comparatively complex for a physically disabled 2015.
person. This limitation is reduced in this project by using a [4] S. Uddhav Khadilkar and N. Wagdarikar, "Android phone controlled
less number of sensors to cover the major goal of the voice, gesture and touch screen operated smart wheelchair", in
project. International Conference on Pervasive Computing (ICPC), 2015.
[5] R. A. Kalantri and D. Chitre, "Automatic Wheelchair using Gesture
Recognition", International Journal of Engineering and Innovative
Prof. Vishal V. Pande et al. designed a advanced Technology (IJEIT), vol. 2, no. 9, pp. 216-218, 2013.
wheelchair system for physically disabled person [6] that [6] P. Pande, N. Ubale, D. Masurkar, N. Ingole and P. Mane, "Hand
can be controlled by hand movement or hand gesture Gesture Based Wheelchair Movement Control for Disabled Person
recognition using accelerometer technology. But the major Using MEMS", International Journal of Engineering Research and
drawback of the project is this device cannot be adaptable Applications (IJERA), vol. 4, no. 4, pp. 152-158, 2014.
by all users due to its costly ADXL202 sensor and its [7] G. Soni, V. Poddar, Y. Sahu and P. Suryawanshi, "Hand Gesture
Recognition Based Wheel Chair Direction Control Using AVR
complex operation. This problem is mitigated in this project Microcontroller", International Journal of Advanced Research in
by its user friendly operation and its cost effectiveness. Computer and Communication Engineering, vol. 5, no. 3, pp. 344-
Power consumption of the device is also reduced in this 348, 2016.
project. [8] "National Orthopedic Hospital and Rehabilitation Center Report
2015", National Orthopedic Hospital and Rehabilitation Center
(Pangu Hospital), 2015.
Rini Akmeliawati et al. [1] designed a system named [9] Abu Tayab Noman, M A Mahmud Chowdhury, Humayun Rashid, S.
“Design and Development of a Hand-glove Controlled M. Saifur Rahman Faisal, Iftekhar Uddin Ahmed and S M Taslim
Wheelchair” in which an external controlling circuit is used Reza, "Design and implementation of microcontroller based assistive
to control the movement of the wheelchair. The control robot for person with blind autism and visual impairment", in 20th
International Conference of Computer and Information Technology
circuit is attached with a hand glove which can be (ICCIT), 2017, pp. 1-5.
uncomfortable for the user. And Power consumption of this [10] Humayun Rashid, A.S.M. Rabbi Al-Mamun, Mohammad Sijanur
device is also very high. In this project no external control Rahaman Robin, Miraz Ahasan and S M Taslim Reza, "Bilingual
circuit is needed, user can control the movement of the wearable assistive technology for visually impaired persons",
in International Conference on Medical Engineering, Health
wheelchair by just his smart phone. Less power consumed in Informatics and Technology (MediTec), 2016, pp. 1-6.
this proposed device due to its simple structure. [11] Iftekhar Uddin Ahmed, Nazia Hassan and Humayun Rashid, "Solar
powered smart wearable health monitoring and tracking device based
on GPS and GSM technology for children with autism", in 4th
International Conference on Advances in Electrical Engineering
(ICAEE), 2017, pp. 111-116.

View publication stats