Iccsce2017m F Ahmad
Iccsce2017m F Ahmad
net/publication/354253917
CITATIONS READS
12 118
5 authors, including:
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Mohamad Faiz Ahmad on 31 August 2021.
Abstract—Wheelchair user will face various challenges wheelchair. The weight of the luggage can roughen up the
during travelling from one destination to another. Their hands movement for the wheelchair user if the luggage is heavy.
will be required to maneuver the wheelchair around. It is Besides, the extra load of luggage will further increase the
difficult for a wheelchair user to identify the appropriate weight of the wheelchair and may cause extra strain on the
location to place their luggage within the wheelchair itself.
arm of the user that may lead to serious injury.
Therefore, luggage carrying following cart is an initiative that
could lighten the burden for wheelchair users. The method The objective is to design and develop a wheelchair
used is via colour tracking system using Pixy CMUcam5 following cart for wheelchair user using visual-based
sensor, Arduino MEGA microcontroller, ultrasonic sensors, tracking technique to carry their luggage. First, colour
servo motor, motor driver and transaxle motor. Pixy tracking technique must be fulfilled based on visual-based
CMUcam5 sensor is used to perform colour tracking on target sensor to identify the object in which the cart will follow.
of interest. Arduino MEGA is the main microcontroller that Second, a cart which uses transaxle motor as a drive and
controls all the input and output data based on the program. navigation system is developed for it to move. Finally, there
Servo motor is used to steer the direction of the front wheel must be an allowable distance between wheelchair and cart
while ultrasonic sensors are used to perform obstacles
to prevent collision.
avoidance task. The movement control of transaxle motor is
performed through motor driver. Colour tracking technique is This research covers the study of how an embedded
based on surface area of the target of interest and brightness system can apply in the application on wheelchair system. It
from the background. The field of view (FOV) of Pixy consists of hardware and software since the whole
CMUcam5 sensor is 73.98° when the distance between the application of this luggage following cart will depend on
sensor and the target is at 1.50m. In addition, the ultrasonic these two factors. The hardware consists of a camera for
sensors and servo motor are also discussed to identify the vision, a motor for the movement of the cart, servo motor
performance and its sensitivity respectively. The maximum for the steering of direction and sensors. On the other hand,
error on the ultrasonic sensors is 2.00%. The percentage error software part will depend on microcontroller used.
for the servo motor without a load is 1.58% while with a load is
15.74%. Lastly, an effective target tracking system is II. TYPE OF FOLLOWING TASKS
successfully designed for this project.
A. Object Following
Index Terms—Following task, Autonomous mobile robot, Object following requires image processing type of sensor
Visual based sensor, Colour tracking, Pixy CMUcam5, Cart to fulfill the task. Pixy CMUcam5 sensor is used as the
follower, Wheelchair. vision-based sensor to detect a specific pattern on the target
object, which in this case is the two colour patterns located
I. INTRODUCTION at the back of a wheelchair. Two colour patterns are chosen
Nowadays, the society has given privilege to those disable instead of one in order to increase the efficiency for pattern
people in many ways. For example, special toilets, lift and identification. The controller used is Altera DE0 Nano
many more have introduced to help them in our daily life. FPGA which uses programming software such as Quartus
Therefore, infrastructures and innovations are created to fit II, Qsys and Nios II eclipse [1].
the demand in helping those people with difficulties to
lessen their burden. Disabled people, especially wheelchair B. Leader Following
users, faced problems in handling challenges during The study involves several non-holonomic robots to achieve
travelling. the leader following formation without any communication
Wheelchair user will face various challenges during among each other. The leader following method is achieved
travelling from a destination to destination. Their hand will by using image sensor which is camera. The camera used is
be required to maneuver the wheelchair around. It did not pan-controlled camera which can find the specific pattern of
help when wheelchair user chose where to place their the leader robot easily by rotating the angle along the x-axis
luggage. Normal wheelchair does not have a space design to of the camera faced [2]. Another project that implement the
carry stuff on them. This will further burden them with the same leader following technique was introduced by
addition luggage when the luggage is carried along on the Mariottini [3]. A panoramic camera was equipped on a
C. Human Following
"ApriAttendaTM”, the name of the human following robot,
was created by Yoshida and his team. This human following
robot uses image processing to identify the individual’s
cloth colour and texture. The target of interest must be
within the range in the middle of the frame as it is normally
the location for the human clothing. Furthermore, it also Fig. 1 Dimension of the cart base in three views by using solidworks
program
uses ultrasonic sensor to avoid obstacles. When the robot
loses visual contact with the target, it will able to search and
reconnect with the target. The highlight of this human a) Object Identification
following robot is it has high robustness and was capable to Pixy CMUcam5 sensor is used as the camera for object
undergo the human following task smoothly [4]. identification. The sensor will then be connected to Arduino
MEGA microcontroller as the microcontroller will dictate
D. Vision-based sensor the next move for CMUcam5. Furthermore, two colour
Visual Servo Control was introduced by Chaumette and patterns are used as the target of interest and will be placed
Hutchinson [5] to compare the efficiency between image- behind the wheelchair. The colour pattern will be placed at
based and position-based servo control technique. However, the position 30cm above ground so that it is mostly at the
it is purely depends on the undergoing task. Stability is the same level with Pixy sensor.
main factor that can differentiate good technique.
Another research shows a low cost project using b) Cart Movement
CMUcam2, second generation of CMUcam, to measure the The task for the cart movement can be achieved through
performance and additional functionality compared to the transaxle motor. Pixy camera has an ability to detect the
CMUcam, first generation of CMUcam system [6]. target object while providing information for example width,
A PISALA project, which uses artificial vision embedded height and even the angle faced from the object towards the
method to detect floor line with CMUCAM3, is introduced camera sensor. The cart can be programmed such that when
by Girbes [7]. The main point of this project is to find the size of the target object is shrinking, which indicates the
Automated Guided Vehicle (AGV) a solution for line target object has moved away from sensor, the transaxle
following using vision-based method. motor will be triggered to move forward in a constant speed.
The latest model of CMUcam is CMUcam5. [8] came out On the other hand, when the size of the target object is
with overtaking asistant system (OAS) that help the driver expending, which indicates the target object has moved
to make decision on overtaking using a CMUcam5 camera toward the sensor, the transaxle motor will move backward
with color optical sensor meanwhile [9] used CMUcam5 for [11].
capturing image trajectory for air hockey system. Another
reseacher build a distance estimation based on colour block c) Cart Turning Direction
model with CMUcam5. By this they can get an information The cart turning direction can be help through the usage
on X and Y coordinate and block tracked [10]. of servo motor. The design for the steering mechanism will
be done base on the positioning of the servo motor.
Furthermore, the servo motor will be positioned in the
III. METHODOLOGY middle of the parallel linkage rode that connect with two
A. Luggage carrying following cart front caster wheels. Servo motor can also operate based on
1) Design of the cart the Pixy sensor programming to increase the efficiency.
The cart consists of two parts which are front half section
and bottom half section. The front half section of the cart
will contain all the electronic components while the bottom
half section of the cart will consist of mainly of battery and
space for luggage. Figure 1 shows the dimension for the cart
base in three views by using Solidworks program.
d) Obstacle Avoidance Other sensors used are Ultrasonic sensor, HC-sr04. It can
Obstacle avoidance task is considered as an important be used for remote measurement of physical quality via
aspect in this project. Ultrasonic sensor is used as it is the ultrasonic waves. This module has an ability to detect a
best option available in the market nowadays due to its distance from 2cm to 400cm in a correct matter. This sensor
robustness and low cost. In this project, three ultrasonic mainly used to detect the distance between the luggage cart
sensors will be used. One of the ultrasonic sensors will be and the wheelchair to prevent collision and for obstacles
mounted at the middle front section of the cart while the avoidance purpose.
other two ultrasonic sensors will also be placed at the front
section but in right and left of the side. C. Pixy CMUcam5 sensor field of view with Pan-tilt
When the ultrasonic sensors sense obstacle within the mechanism device
programmed distance, the cart will either stop or turn to the The combination of pan-tilt device with Pixy camera will
opposite direction based on which sensors it triggers. After improve the efficiency of tracking process. With the help of
entering a new path, the cart will continue follow the pan-tilt mechanism, the sensor can easily follow the target
wheelchair back to its original path as soon as there is no any of interest even when the cart is not facing the target of
obstacle detection. Moreover, since there is an extra sensor interest. The Pixy sensor can still able to detect the target of
which senses the obstacle in front of the cart, it can maintain interest without rotating the cart as the target of interest is
a safe distance between the cart and the wheelchair, thus acts within the maximum field of view for Pixy sensor
as an extra cushion of safety. completed with pan-tilt mechanism. Figure 4 shows the Pixy
CMUcam5 sensor field of view (FOV) from top view while
B. System sections Figure 5 shows the Pixy CMUcam5 FOV with pan-tilt
The block diagram for the overall process is illustrated in mechanism device.
Figure 3. The microcontroller used is Arduino MEGA The FOV for the Pixy sensor can be determined by using
which acts as the main component to control the overall derivation from the Pythagoras theorem as shown in
system. The driving force of the luggage cart comes from Equation 1.
Brushed Electric Scooter Geared DC Transaxle Motor. This
transaxle motor requires DC 24V with maximum supplied a = Distance between camera and tracked object
current of 3.0A to boost up. The maximum output power b = Wall Length
rate is 270W and the maximum speed that can be achieved b
by this motor is 4700rpm. The power supply for the c=
transaxle motor is from a 24V battery via motor driver 2
MD30B. This motor driver controlled the motor movement. FOV o
For the electromagnetic brake pin, another 24V battery is φ=
required to connect to its positive terminal and negative 2
terminal.
c
Pixy CMUcam5 camera is used as the main vision based tan φ =
sensor to perform the colour tracking technique. This sensor a
is chosen due to its ability to solve problems occurred when
image sensors are used [12]. It used colour-based filtering c
φ = tan −1
algorithm method to identify objects due to its robustness, a
faster speed and higher efficiency that it possesses.
The servo that was used is RC Servo motor with metal FOV o = 2φ = θ
gear. This servo motor has 17kg.cm holding torque at 6V
voltage supplies. However, the torque can be improved to c
20.45kg.cm if the voltage supplied is increase till 7.2V with FOV o = 2 tan −1 (1)
a
the maximum speed of rotation remain the same at 6V
voltage supplies (0.14sec / 60°). The maximum load that the
servo motor can lift is 10.225kg.
TABLE 5. SENSITIVITY OF A SERVO MOTOR WITH AND WITHOUT LOAD Furthermore, the efficiency of the colour signatures
Without Load With Load detected by the Pixy camera can be improved by setting
Angle Actual Percentage Actual Percentage more than two different colours coding to be tagged together
(°) Angle Error (%) Angle Error (%) through the usage of colour code (CC) function. Internet of
0 3 1.67 40 22.22
30 33 1.67 55 13.89
Things (IoT) related projects have bloom in recent years.
60 62 1.11 75 8.33 Therefore, IoT technology can be implemented in this
90 90 0.00 90 0.00 project, where wireless connection can be established for
120 116 2.22 105 8.33 indication purposes. This kind of improvement make easier
150 147 1.67 120 16.67 for the wheelchair user to control the cart via wireless
180 178 1.11 135 25.00 connection.
Average 1.58 Average 15.74
error error
ACKNOWLEDGEMENTS
(%) (%)
The authors would like to thank Universiti Sains
E. Obstacle avoidance Malaysia (USM) for the support of the research under grant
An ultrasonic wave will send out by the transmitter and then no 305/PELECT/6013112.
received by the receiver. By using Arduino programming
configuration, the data collected can be converted into REFERENCES
distance in the unit cm. TABLE IV presents the result on the [1] M.F. Ahmad, S.S.N. Alhady, S. Kaharuddin and W.A.F.W.
Othman, 2015, November. “Visual based sensor cart follower
sensitivity performance for the three ultrasonic sensors. A for wheelchair by using microcontroller,” In International
graph of Average Actual Distance (cm) versus Distance Control System, Computing and Engineering (ICCSCE), 2015
(cm) has been plotted in Figure 11. IEEE International Conference (pp. 123-128). IEEE.
[2] X. Chen and Y. Jia, 2015. “Adaptive leader-follower formation
control of non-holonomic mobile robots using active vision,” In
F. Rotation of a servo motor IET Control Theory & Applications, 9(8), pp.1302-1311.
Table V shows the comparison sensitivity of a servo motor [3] G.L. Mariottini, F. Morbidi, D. Prattichizzo, N.V. Valk, N.
with and without load for seven different angles taken from Michael, G. Pappas and K. Daniilidis, 2009. “Vision-based
the range of 0° to 180°. The percentage error for the servo localization for leader–follower formation control,” In IEEE
Transactions on Robotics, 25(6), pp.1431-1438.
motor without a load is 1.58% while with a load is 15.74%. [4] T. Yoshimi, M. Nishiyama, T. Sonoura, H. Nakamoto, S.
The increment of error happened when a load is exerting on Tokura, H. Sato, F. Ozaki, N. Matsuhira and H. Mizoguchi,
it. This is due to the movement limitation that both castor 2006, October. “Development of a person following robot with
wheels at the front of the cart that being attached to the vision based target detection,” In Intelligent Robots and
Systems, 2006 IEEE/RSJ International Conference (pp. 5286-
servo motor. It is impossible for the castor wheels to move 5291). IEEE.
flat 0° and 180° since it will make the cart unable to move. [5] F. Chaumette and S. Hutchinson, 2006. “Visual servo control. I.
Basic approaches,” In IEEE Robotics & Automation Magazine,
V. CONCLUSIONS 13(4), pp.82-90.
[6] A. Rowe, C. Rosenberg and I. Nourbakhsh, 2005, June. “A
The luggage carrying cart able to follow the wheelchair second generation low cost embedded color vision system. In
from behind without the help of human control. The Computer Vision and Pattern Recognition-Workshops,” CVPR
objective was achieved. Several hardware accompanied with Workshops. In IEEE Computer Society Conference (pp. 136-
its software are implemented in the development of the 136). IEEE.
[7] V. Girbés, L. Armesto and J. Tornero, 2010, June. “PISALA
luggage carrying cart. project. Intelligent Sensorization for Line tracking with
The hardware used are divided into 2 sections which are Artificial Vision,” In Robotics (ISR), 2010 41st International
sensor components and respondent components. Sensor Symposium on and 2010 6th German Conference on Robotics
components are used for object and target recognition and (ROBOTIK) (pp. 1-6). VDE.
[8] K. Anindyaguna, N.C. Basjaruddin and D. Saefudin, 2016,
obstacle avoidance purposes. The sensors used are Pixy October. “Overtaking assistant system (OAS) with fuzzy logic
CMUcam5 sensor and ultrasonic sensors HC-SR04 while method using camera sensor,” In Industrial, Mechanical,
the respondent components include servo motor and Electrical, and Chemical Engineering (ICIMECE), International
transaxle motor. Data and reading obtained from the sensor Conference of (pp. 89-94). IEEE.
[9] A.H. Moreira, V.A.B. Voni, A.D.A.L. Junior, E.S. Azuma, F.S.
components will dictate the functionality for respondent Madani and L.G. Trabasso, 2017, February. “A Low-cost Vision
components. Based Air Hockey System,” In Proceedings of the 3rd
Pixy CMUcam5 sensor has the field of view (FOV) of International Conference on Mechatronics and Robotics
73.98° at the distance of 1.50m. This shows that the greater Engineering (pp. 53-56). ACM.
[10] B. Rahmani, H. Aprilianto, H. Ismanto and H. Hamdani, 2017.
the distance between the target of interest and the sensor, the “Distance Estimation based on Color-Block: A Simple Big-O
higher the field of view (FOV). Furthermore, ultrasonic Analysis,” In International Journal of Electrical and Computer
sensors have a maximum error of 2.00% due to its Engineering (IJECE), 7(4), pp.2169-2175.
inconsistency. In addition, servo motor also has the [11] Kaandog, (2014). “Tutorial: Pixy (CMUcam5) | Physical
Computing,” Available at: http://courses.ideate.cmu.edu/
sensitivity error of 15.74% when a load was applied on it. physcomp/f14/16-223/tutorial-pixy-cmucam5/ [Accessed March
Some improvements can be made on the luggage 30, 2017].
carrying cart mainly on the hardware part. Extra ultrasonic [12] CMUcam5 Pixy, 2011 – “CMUcam: Open Source
sensors can be located at the back part of the following cart. Programmable Embedded Color Vision Sensors,” Available at:
http://cmucam.org/projects/cmucam5 [Accessed March 30,
The obstacle at the back can be avoided and ensure the 2017].
luggage carrying cart is not being damaged.