Vision and Infra-Red Sensor Based Fire Fighting

Robot
Shang Gao Zhiyang Zhang Zihan Zhao Mohsin M. Jamali
The University of Detroit The University of Detroit The Beijing University of The University of Texas of
Mercy Mercy Chemical Technology the Permian Basin
Detroit, USA Detroit, USA Beijing, P. R. China Odessa, USA
[email protected] [email protected] [email protected] [email protected]

Abstract—Fire-fighting robots are useful in situations where it equipment.

is too dangerous or has difficult access for fire fighters. Modular
design based on proximity, vision and IR sensors has been
II. SYSTEM ARCHITECTURE
incorporated into a fire-fighting model. The robot sprays water
from manipulator to extinguish flame. Software consists of Beijing Uptech-Robot Company “InnoStar” provides fire-
integrated tracking, obstacle avoidance, flame detection and extinguishing robot. It uses four-wheel structure, with its
motion algorithms. Through testing, it is possible to run in a superior load capacity and maneuverability, which can play an
realistic scene simulated in the lab and to detect and important role in firefighting operation [9]. This work mainly
extinguishment the flame. Robot can transfer video to remote introduces the experimental fire-extinguishing robot. The top
location. of the wheels is a four-axis manipulator for firefighting
operations. Modular firefighting robot has three features:
Keywords—robot; fire-fighting robot; sensor firstly, the use of drive-by-wire, simplifying the mechanical
structure; second, the functional modules are independent of
I. INTRODUCTION each other, leaving connectors for the installation of other
Robots are deployed in various industries and for consumer required equipment that can be simply replaced, repaired or
applications. The fire-fighting robot, as one of the autonomous modified. Thus, effectively reducing the cost and increasing its
mobile robots, has significant application in the future. They versatility. One can simply modify the structure, making the
can be deployed at close quarters and where it is too dangerous robot universal type, low maintenance costs, more suitable for
for humans or areas that are not easily accessible. Most of the actual industrial scene. Thirdly, the swinging of the
existing model fire-fighting robot are using fans to blow off the manipulator compensates the positioning error, and at the same
fire [1-8]. This fire-fighting strategy works well in some fire- time, it can cope with a wider range of fire conditions and
fighting robot applications. However, it is not only inefficient improve the efficiency and practicability of fire extinguishing.
in practical scenarios, but in some cases, it is possible to further In order to stabilize the robot and monitor the real-time
inflame the fire. The use of water or other flame retardant performance of the environment, the system needs higher level
liquid to extinguish the fire is used in real life scenario and of computational requirements and the processing capability of
have also been used in this work. The water spray firefighting the controller. This work uses MultiFLEX ™ 2-AVR controller
robot will require precise localization of the fire and placement with ATmega128 processor and the ATmega8 is the co-
of robotic arm. In order to compensate for the positioning error processor [10]. It can control R/C servo, robot bus servo, DC
caused by tracking and stopping, this work incorporates the motor servo drives. The main processor has real-time
arm above the smart car, winding the arm around the pipe, and processing capability, performing up to 16 MIPS when
installing the nozzle at the arm end, so as to expand the work operating at 16 MHz, with 128K bytes of system
area by swinging the arm during the firefighting operation. programmable Flash and 100,000 write / erase cycles of life. It
The purpose of this project is to design and develop an has high good reliability, low power consumption, and the
experimental modular intelligent fire extinguishing robot program is easily upgradable. Figure 1 shows its system
designed to run a patrol on a planned trajectory, detect the fire architecture.
autonomously, locate it and extinguish the flame by water In order to achieve the function of avoiding barrier, detect
spray. It simulates the scenes of warehouses, laboratories, flames and extinguish it utilizes appropriate sensors such as
storage rooms, cargo hold and cabin of aircraft. Making use of far-infrared flame sensor, greyscale sensor and IR approaching
the mobility of robots, it eliminates the deficiencies of sensor. The far-infrared flame sensor detects flame using ultra-
traditional fixed base fire detectors and fire extinguishing sensitive phototransistor probe, which provides analog or
digital output to the controller. The range is two meters for
This work is partially supported by College Student Science and
Technology Innovation Fund, Beijing University of Chemical Technology,
China Ministry of Education

978-1-5386-7392-8/18/$31.00 ©2018 IEEE 873

visible and infrared light. Sensors operate in linear fashion. In IR acts as an optoelectronic switch, which returns a high-
the mode of digital output, the high-voltage level means that level signal to the controller when the sensor is within a certain
there is fire. The analog mode provides an output ranging from distance of a barrier. E18-D90NK is used in this project, which
0 to 1023. Number indicates the degree of fire from low to could work at a range of 3 to 80 cm. Both sensors are placed in
high. front of the car, so that the robot can determine whether there is
an obstruction in front of it. It is also used to keep the robot at
an appropriate distance away from the flame during fire-
extinguishing procedure. Therefore, the robot can stop at an
optimum position to extinguish fire.
Good performance is required for the servo used in the
project. There are eight CDS5516 servos in this project, which
provide a torque of 16Kgf.cm and maximum speed of
0.16s/60°. It is waterproof and sealed to protect from ashes.
The wheels are designed as differential steering drive. The 4-
axis machine arm with tubes and spray on the upper level of
the robot. The arm is rotated to locate the flame during the fire
extinguishing procedure, and the water-spray area can be
increased by the rotating arm. This also compensates any error
in flame location, which may be induced by the flame sensor. It
Fig. 1. Hardware Architecture.
also allows and makes it easier for dealing with the large area
flame
In this robot, there is a 640x480 video camera on board,
fixed on a small servo that could rotate horizontally [11]. The
computer software via Wi-Fi controls it. Operators can monitor
the surrounding areas during navigation and fire extinguishing
operations. The video module is completely independent from
other parts, individually powered to prevent reducing the
cruising ability. Video helps the operators to estimate the scale
of the flame and the materials that are catching fire.
Fig. 2. Flame Sensor Circuit Diagram

Figure 2 shows flame sensor circuit, there are eight flame

sensors used in this design. Two flame sensors are in the front,
with a small angle in between. Other sensors are at 30° apart
and provide digital output. Figure 3 shows placement of flame
sensors.

Fig. 4. Camera Image

Spray module is the actuator of the fire-fighting robot. It

has a 1L tank with a 2W’s pump providing a flow rate of
180L/h. The relay controls the module. It is in between the
pump and isolates the power supply. The controller outputs
high (<GND+1.5V≈1.1V) and low (>VCC-1.5V≈4.7V) level
signal controls the on and off the relay essentially controlling
the pump’s operation. The use of the relay isolates the spray
Fig. 3. Flame Sensor Placement module from the main body, where shows the idea of modular
The robot patrols through the planned route along two design again.
white lines. To make sure the robot detects the line, couple of
greyscale sensors are placed in the front part of the car, and III. SOFTWARE DESIGN
1cm away from the ground. The greyscale sensor could sense
Software has been designed to track robot movement
and differentiate between white line and color areas. The
according to pre-planned path. It detects, locates and
lighter color will return higher value. In this project, the value
extinguishes the fire. Input sensors aid navigation and tracking.
for the white line is set at 850 and higher, which is measured
Sensors provides greyscale information at a periodic interval.
during the trial test.
When one of the two sensors detect the white line, the robot

874
will turn to the other side. The controller gathers data from all appropriate distance from the flame, and can start water
eight flame sensors and two greyscale sensors during each spraying. After completing the fire-tracking mode, the program
scanning circle. Following is the pseudo-code for the main will go to fire-extinguishing mode. The machine arm will move
program. Flowchart for fire localizing and extinguishing is from stand-by position to the firefight position, and then the
shown in Figures 5. relay will switch on by the high-level signal from the controller
and starts to spray with swing the manipulator. At the end of
/* each swing period, the front flame sensor will read the data.
System initialization, set manipulator to standby position The robot will spray three more round to make sure the flame
is off before stopping the pump, moves the arm upwards and
Read sensor values turn LEDs on to alert the operator. Water will flow back to the
If flame is detected tank.
Then execute fire-localization module IV. SIMULATION
Otherwise, execute traction algorithm and read sensor Due to limited testing environment, it is designed as Z-type
values again. route on the dark ground. The flame is simulated using five
Execute fire-extinguishing module fixed candles placed in transparent box and are 5 cm high from
the ground. Candles will be 3 cm above the box. The box is
/* placed at a random position 0.8-1.5 meter away from the route.
The robot is set to move at the speed of 15cm/s (robot might
off track on the corner or cannot stop soon enough because of
the inertia of the tank when it moves on higher speed). On the
corners, the outside wheels are set at 8.8cm/s and 4.4cm/s for
the inner. Table 1 shows test data. Four tests were performed.
Time in second for localizing the flame, time between the start
of the spray and extinguishing of five candles were recorded.

Fig. 6. Simulation testing mapping and trajectory

The Fig. 6 shows the arrangement of the simulation

playground, that the robot starts from one end. It moves alone
the planned route, within the boundary line. Once the robot hit
the boundary line, it will correct its heading. The red star point
indicates the position that one of the robot’s flame sensor
Fig. 5. Algorithm for Fire Location and Extinguishing Operation detects the flame. It is the point for starting time record. Then
the robot will localize the direction of the flame the move
Once the robot detects the flame, the program goes to the
forward to the flame, executing the fire-fighting operation. This
fire-tracking mode. The robot will stop moving, start to turn to
figure is not actual scaled. Also, in the actual test, the color is
the flame at the same position until both flame sensors detect
set as using white boundary line on the dark ground.
flames. In this way, it is easy for the robot to face the flame.
Then it moves slowly to the flame, stop when both IR sensors
transmit the signal, specifying that the robot be at an

875
 TABLE I. TEST DATA FOR FOUR DIFFERENT TESTS AND THEIR REFERENCES
DURATION (UNIT: SECOND)
[1] X.Y. Li, D.Y. Chen, X.Y. Ma, “Design and implementation of
Test 1 2 3 4 intelligent fire-fighting robot,” Electronic Design Engineering. China,
vol.18, pp. 51-54, No.3, March 2010.
Localize Flame 3.13 3.02 3.24 3.46
[2] Y.L. Zhu, The Design and implementation of ARM9-based fire-fighting
Start Spray 7.60 6.94 7.32 7.28 robot, Jiangsu University of Science and Technology. China, 2014.
Candle 1 8.50 9.11 9.85 14.76 [3] Y.M. Liang, L.H. Xu, “Research and design of multi-sensor ranging
Candle 2 14.00 10.93 13.48 29.03 system on mobile robot,” Computer Application. China, vol. 28, pp.
340-343, June 2008.
Candle 3 16.52 15.16 15.91 32.41 [4] X.Y. Chen, Y.H. Yan, “Design and implementation of firefighting robot
Candle 4 16.60 24.14 20.85 39.95 based on AVR single chip,” Computer & Information Technology,
China, vol. 07, pp. 29-31, 2008.
Candle 5 43.10 36.46 37.04 /
[5] Dearie S, Fisher K, Rajala B, et al. Design and construction of a fully
autonomous firefighting robot[C]// Electrical Insulation Conference and
Electrical Manufacturing & Coil Winding Conference, 2001.
After measurement, the average time to extinguish five Proceedings. IEEE, 2002:303-310.
candles is 38.87s. In the Test #4, the tank ran out of water [6] Rakib T, Sarkar M A R. Design and fabrication of an autonomous
before the fifth candle is put out so extinguishing time for the firefighting robot with multisensory fire detection using PID
fifth candle was not recorded. controller[C]// Intl. Conf. on Informatics, Electronics & Vision.
2016:909-914.
[7] Park S U, Park J H, Choi S, et al. Wireless image communication system
V. CONCLUSION for fire-fighting robots[C]// The, International Conference on Computer
The project was successfully designed manufacturing and and Automation Engineering. IEEE, 2010:254-256.
tested in simulating environment as a experimental modular [8] Chien T L, Guo H, Su K L, et al. Develop a Multiple Interface Based
firefight robot as expected. Future work includes Fire Fighting Robot[C]// Icm2007, IEEE International Conference on
Mechatronics. IEEE, 2010:1-6.
transformation of experimental robots into practical robots,
[9] Beijing UpTech. MultiFlex2-AVR Manual & Datasheet. 2010.
which needs improving the overall performance of robots,
[10] Atmel Corporation. ATmega128_datasheet [Z].
especially in controllability and load. Improve all parts of the
machine fire resistance, strength and resistance to fatigue. [11] Beijing UpTech. Servo CDS55xx Manual V1.2. 2010.
Various control algorithms as well as machine learning and
neural network may be integrated to improve the work
efficiency and stability.

876