IEEE - 49239

Vision-based Guidance and Navigation for

Autonomous MAV in Indoor Environment
Mahammad Irfan Sagar Dalai Kaushal Kishore
CSIR-Central Electronics Engineering CSIR-Central Electronics Engineering CSIR-Central Electronics Engineering
Research Institute Research Institute Research Institute
Rajasthan, India 333031 Rajasthan, India 333031 Rajasthan, India 333031
[email protected] [email protected] [email protected]

Samarth Singh S.A Akbar

CSIR-Central Electronics Engineering CSIR-Central Electronics Engineering
Research Institute Research Institute
Rajasthan, India 333031 Rajasthan, India 333031
[email protected] [email protected]

Abstract— The paper presents an autonomous vision-based ArUco markers. Finally, the simultaneous localization and
guidance and mapping algorithm for navigation of drones in a mapping (SLAM) based model using cartographer ROS has
GPS-denied environment. We propose a novel algorithm that been implemented for mapping 2D/3D surrounding. The test
accurately uses OpenCV ArUco markers as a reference for path is performed in a virtual environment created by gazebo and
detection and guidance using a stereo camera. It enables the Rviz by placing different dynamic ArUco markers for
drone to navigate and map an environment using vision-based different destinations on the map.
path planning. Special attention has been given towards the
robustness of guidance and controlling strategy, accuracy in the
vehicle pose estimation and real-time operation. The proposed
algorithm is evaluated in a 3D simulated environment using
ROS and Gazebo. The results have been presented for drone
navigation in a maze pattern indoor scenario. Evaluation of the
given guidance system in the simulated environment suggests
that the proposed system can be used for generating a 2D/3D
occupancy grid map autonomously without the use of high-level
algorithms and expensive sensors such as lidars.

Keywords—ROS, Gazebo, Pose estimation, Quadcopter,

Vision Guidance, SLAM.

I. INTRODUCTION
Micro Aerial Vehicles (MAVs) commonly known as
drones playing an important role in achieving various tasks in
both civilian and military areas because of their autonomy,
high versatility, low cost, extreme agility, easy to deploy, and
the capability for performing complex maneuvers in both
structured and unstructured environment [1]. However, a lot Figure 1. A vision-based navigation control law
of work has been progressed during the past recent years,
research on their deployment and control efforts are still This research aims to contribute to the emerging field of
focused on achieving fully autonomous drones that can research of MAVs in a wide range of different applications by
perform high-level missions particularly in GPS denied introducing a low-cost vision-based intelligent indoor
environment [2]. Estimating and mapping of the surrounding pathfinding and mapping technique. The novelty of the
3D environment is one of the key challenging task for fully developed algorithm relies on its simple yet accurate
autonomous MAVs to navigate safely and operate high-level autonomous real-time online navigation and mapping with no
tasks [3]. prior information from external infrastructure.
The proposed research work aims at developing an II. RELATED WORK
algorithm for vision-based path following and navigation
approach in indoor application. In this paper, we have The real-time vision-based navigation and mapping
deployed an online ArUco based OpenCV tracker, which approach is the research of interest in many areas from
helps the MAV to autonomously track, detect and find its augmented reality to intelligent robotics and autonomous
position with reference to the ArUco marker for the online aerial vehicle application. It collects the visual information
navigation application. Whenever an ArUco tracker is from vehicles using path trackers like ArUco [4], [5]. The
detected, the controller flag is set to high which results in the other type works without using such markers but consider the
position controller module to receive the sensor feedback data 3D surrounding information [6]. There are many techniques
in order to generate (x- y) relative positions of the MAV with that are already implemented to control the MAV
respect to the tracker (shown in Fig. 1). The reference value r attitude/altitude and position [7], which includes PID, LQR,
which is a relative distance to the marker of the controller is H∞ , and model predictive controllers. The ideal purpose of
set to zero. In order to achieve the proposed task, the such a controller is to maintain a suitable accurate position of
localization relies on the drone vision-based technique using the vehicle, however, the position control is used to drive the

11th ICCCNT 2020

July 1-3, 2020 - IIT - Kharagpur
Kharagpur,
Authorized licensed use limited to: HCT Musnaa. Downloaded on OctoberIndia
21,2021 at 04:58:32 UTC from IEEE Xplore. Restrictions apply.
 IEEE - 49239

drone to navigate a predefined indoor tracking path. To A. ArUco_ROS Trackers

enhance the performance of the controller, vehicle pose For autonomous target detection, pose estimation, and
feedback is an important parameter. GPS signals are widely path tracking purposes, we have used the ArUco_ROS
used for outdoor in the robotics control tasks because inertial
package from OpenCV library, which is commonly used in
measurement unit (IMU) sensors have accumulated errors.
robotics, augmented reality and camera calibration
Furthermore, in an indoor environment, GPS signals are
undetectable many times, which leads to new research scope applications.
to the design of autonomous MAV controllers [8]. The ArUco tracker consists of several binary bits having
The traditional monocular camera is popular because of its five bits in each row, while the second and fourth bits are
low-cost solution to determine the vehicle pose estimation in known as information bits and the rest three bits act as error
GPS denied environment. However, there a significant loss of control bits. There are different types of ArUco markers as
depth qualities as compared to binocular camera vision and shown in Fig.2. Each ArUco markers has its unique ID and
some additional features required to estimate the vehicle pose each marker ID’s are recognized by the information present in
[9]–[11]. Simultaneous localization and map (SLAM) bits. It consists of a total of 1024 different patterns. The
algorithm use depth information for MAV pose estimation proposed system uses 7x7 dictionary-based markers (7x7 bits)
[12] on the expense of large memory and high computation for better detection, resulting in reduced false-positive
resources. An OpenCV ArUco library initially developed for identification of the markers.
augmented reality but later used in many robotics applications
[4]. It also used as intelligent markers for MAV pose
estimation[13]. Google cartographers are used for mapping
purposes. SLAM provides autonomous vision-based guidance
and mapping work [14], [15].
In order to perform complex maneuver tasks in indoor
environment scenario, an autonomous MAV requires accurate
position estimation, localization and environment mapping.
Such system further requires high level processors as well as
sensors such as LIDAR’s, sonars, and lasers etc. Moreover Figure 2. ArUco square markers of 5x5, 6x6 and 7x7 dictionary size
these system leads to expensive in cost, high power respectively.
consumption, heavy weight, affecting the flight endurance
time and agility of MAV. For addressing this problem, we B. MAV pose estimation
present a high-level comprehensive vision-based position The vehicle pose can be estimated by using any calibrated
control algorithm in a complex indoor 3D environment. This camera which calculate minimum of four co-planer and non-
results in robust autonomous navigation in GNSS denied co-linear elements by the obtained image. Let us assume that
environment. An Intel realSense R200 camera equipped in the P (A, B, C) be the location of the camera in space. The camera
gazebo model-based quadcopter to observe ArUco trackers coordinates can be projected with 3D point in a condition
which are placed on the wall and estimate the pose of the such that translation, (t) and rotation, (R) values should be
MAV. To improve the accuracy and eliminate the error of known already. Given coordinate location of the point P(X,
pose estimation, an intelligence PID controller is implemented Y, Z) by the r200 camera can be determine using the obtained
for position control of the vehicle. A 3D ROS based Gazebo eqn. (1) and (2).
simulation platform has been created to verify the proposed 𝑋 𝐴
control algorithm. Different navigation trials have been [𝑌 ] = 𝑅 [𝐵 ] + 𝑡 (1)
performed and obtained simulated results are able to provide
𝑍 𝐶
an accurate vision-based algorithm for real-time MAV pose
calculation while simultaneously mapping the surrounding After expanding equation (1), we will get
seen at the same time. 𝑟00 𝑟01 𝑟02 𝑡𝑥 𝐴
𝑋
𝐵 (2)
[𝑌 ] = [𝑟10 𝑟11 𝑟12 𝑡𝑦 ] [ ]
III. VISION-BASED GUIDANCE ALGORITHM 𝐶
𝑍 𝑟20 𝑟21 𝑟22 𝑡𝑧
The proposed research work consists of three main 1
segments. Firstly, an OpenCV library ArUco marker-based Now, the value of real world coordinates (A B C) are obtained
technology for indoor navigation purposes. Second, a and points (X Y Z) are unknown. The (X, Y) point can be
stochastic method to track and generate a path to a goal calculated from eqn. (2).
location, taking into account for safe navigation. This method 𝑋 𝑓𝑥 0 𝑐𝑥 𝑋
is efficient because it only uses simple ArUco markers to find 𝑌
[ ] = 𝑠[0 𝑓𝑦 𝑐𝑦 ] [𝑌 ] (3)
its position for autonomous pre-defined path tracking and does 𝑍
𝑍
0 0 1
not require any high-level optimized method and sensors. 1
Lastly, a computationally efficient algorithm for creating the From the above obtained equation (3), 𝑓𝑥 and 𝑓𝑦 are known
map of its surrounding while navigating. All the above tasks as focal length of drone camera in x and y direction
can be achieved using a single RGB-D fixed non-tilting frontal respectively, whereas ‘s’ considered as unknown factor and
camera interfaced with the onboard processor. The proposed 𝑐𝑥 and 𝑐𝑦 are known as optical centers of drone camera.
vision-based guidance algorithm consists of several segments Whenever an image from 3D camera is projected towards a
of ROS compatible packages namely, ArUco_ROS package 2D image plane, it results loss of depth information, therefore
and MAV pose estimation method to determine the MAV scale factor ‘s’ can be obtain from X, Y, Z image. After
pose, design of intelligent PID controller for safe position
getting rotation and translational values, it is easy to
control of the vehicle and SLAM based mapping algorithm
determine the camera projected point as well as the image
using ROS_cartographer.

11th ICCCNT 2020

July 1-3, 2020 - IIT - Kharagpur
Kharagpur,
Authorized licensed use limited to: HCT Musnaa. Downloaded on OctoberIndia
21,2021 at 04:58:32 UTC from IEEE Xplore. Restrictions apply.
 IEEE - 49239

points (X, Y) in a camera matrix. Rotation and translation are ROS and Gazebo integrate with autopilot to communicate
arbitrarily selected and the obtained image point will be with the drone platform and receive camera output, IMU,
linked to projected X-Y. barometer, magnetometer and GPS sensor data with noise
from the simulated world and sends this data to generate
C. Intelligent controller design
motor/actuator thrust command values which are sent back to
In this section, a vision-based hybrid intelligent controller Gazebo physics simulator. This can also communicate with
is designed and developed to provide accurate position control Ground Control Station and Offboard API to guide telemetry
mechanism for MAV. The PID controller take difference data with simulated gazebo and accept commands. The
between pose feedback information and generated reference comprehensive configuration model of proposed SITL is
trajectory to calculate the vehicles 3D velocity.
shown in Fig. 4.
A set of PID controllers were implemented to control the
vehicle position by guiding thrust and attitude set point to the
flight controller. The controller system consists of three
parameters, taken as constant Kp, Ki, and Kd which are being
used respectively to tune the vehicle proportional, integral and
differential unit respectively. Output Control equation in
continuous form (3) and in discrete form (4) and it is
structured in Fig. 3. Figure 4. SITL configuration for MAV navigation and
visualization

The results provided by this paper were generated using

SITL simulations. The simulation model used here is an iris-
based quadcopter equipped with an IMU sensor for six
degrees of freedom position estimation, a barometer for
altitude control and a monocular RGBD camera Intel
realSense r200 for SLAM based mapping. The resulted image
topics published by ROS node is processed using ArUco
libraries for target path tracking. The propose testing
Figure 3. Block diagram for PID controller environment is shown in the Fig. 5.
PID is the most widely used feedback gain especially in
aerial robotics guidance and control application. The landing
phase of MAV is controlled by PID controller, with desired
acceleration, the control output u(t) is defined as:
𝑡 𝑑𝑒(𝑡)
𝑢(𝑡) = 𝐾𝑝 𝑒(𝑡) + 𝐾𝑖 ∫𝑜 𝑒(𝑡)𝑑𝑡 + 𝐾𝑑 𝑒(𝑡) (3)
𝑑𝑡
1 𝑁
𝑢(𝑠) = (𝐾𝑝 + 𝐾𝑖 + 𝐾𝑑 1 E(s) ) (4)
𝑠 1+𝑁
𝑠

D. SLAM Mapping using cartographer

In order to fly autonomously in an unknown environment,
a drone must localize itself and understand it’s surrounding,
this can be possible using sensors such as depth LIDAR’s and
camera etc. In this proposed research, sensors like RGB-D
camera and IMU (inertial measurement unit) are used. By
using generated ArUco pose and cartographer mapping Figure 5. Testing environment
algorithm the MAV can locate itself inside the map. SLAM
is based on extended kalman filter algorithm which helps B. Implementation
quadcopter to build a map, estimate the pose and localize This following section explains how the Algorithm 1
itself simultaneously [12]. Cartographer is a technology that works. ROS based Gazebo environment has been created,
provides real-time SLAM in 2D and 3D across multiple models such as ArUco trackers, IRIS drone equipped with real
platform and sensor configuration [14]. sense r200 camera is imported into the simulation platform.
ROS provides inter process communication mechanism, each
IV. EXPERIMENT running process in ROS is called as a node. Every node can
A. Software in the loop simulation (SITL) setup able to communicate with other nodes by
ROS (Robot Operating System) is a Linux based open publishing/subscribing message passing scheme. This allows
source robotics platform consisting of a set of tools and MAV software which composed of independent programs to
software libraries that help robotics researchers to develop and talk each other.
build robotic applications. ROS can be used with px4 and the The quadcopter flies using its fixed non-tilting frontal
Gazebo simulator [16]. SITL is useful to check new mission camera, searching navigation path identification only by the
performance or control algorithms before actually flying the intelligence ArUco path trackers placed inside the indoor
MAV and possibly damage it. environment. When visual tracker is identified, the vision
In order to communicate with flight controller PX4 based algorithm and image processing library computes the 6-
autopilot, MAVROS MAVLink protocol nodes are used. The DOF relative pose between the marker itself and the MAV.

11th ICCCNT 2020

July 1-3, 2020 - IIT - Kharagpur
Kharagpur,
Authorized licensed use limited to: HCT Musnaa. Downloaded on OctoberIndia
21,2021 at 04:58:32 UTC from IEEE Xplore. Restrictions apply.
 IEEE - 49239

The obtained 3D x y z pose is used to help the drone approach in ROS based Gazebo simulation by implementing proposed
towards trackers with the right orientation. An intelligence vision Algorithm 1. The obtained result explain that the
PID controller is designed to reduce the error on x-, y- and z- quadcopter recognize ArUco markers accurately and
axis along with yaw angle. The drone continuously fly and navigate in a 3D maze scenario from its initial take-off
keep tracking the marker provided at a given threshold value (position marked as green) to the final desired location
(distance of 50 cm and an angle of ±5°). Every tracker is (marked as red colour).
coded with some pre-defined task (perform the next
movement) done by the MAV, as it track the ArUco and
reaches near at a fixed distance, MAV read the coded
information and perform the given task to explore the pre-
defined path created in indoor environment by controlling the
drone position. While tracking the ArUco, MAV create a
SLAM based map of the surrounding environment in Rviz.

C. Simulation and results

Figure 7. 2D and 3D MAV trajectory
In this section, ROS based gazebo simulation experiment
has been performed to validate the efficacy of each proposed The Fig.8, shows the initial generation of the map using
research work and the developed SITL configuration which SLAM algorithm in Rviz, when launched. The vehicle moved
is constructed in Fig.4. The simulation model used here is an in a pre-defined path coded in the ArUco trackers and move
iris based MAV equipped with r200 camera as shown in the until a full map is created using the “auto_nav_indoor_pkg”
Fig.6 which describes an ArUco marker is detected by an iris package created in ROS platform, where the drone is
quadcopter and its 3D pose is obtained by using published controlled fully autonomously.
data from camera sensor node in order to track the desired
position.

Figure 8. Initial scan of the map in Rviz

The Fig 9. illustrate the final constructed octo map in the

Figure 6. ArUco marker detected by an iris quadcopter in Gazebo Rviz after MAV successfully navigates through the testing
simulation environment shown in Fig. 5 and reaches at the target
destination, similar to the created environment in the Gazebo.
The results as reported in Fig. 7. Showing 2D and 3D The point cloud information collected from R200 camera
trajectories of MAV during a complete experiment performed

11th ICCCNT 2020

July 1-3, 2020 - IIT - Kharagpur
Kharagpur,
Authorized licensed use limited to: HCT Musnaa. Downloaded on OctoberIndia
21,2021 at 04:58:32 UTC from IEEE Xplore. Restrictions apply.
 IEEE - 49239

which further processed to generate 3D octomap of the indoor REFERENCES

environment. [1] H. Shakhatreh et al., “Unmanned Aerial Vehicles (UAVs): A
Survey on Civil Applications and Key Research Challenges,”
IEEE Access, vol. 7, pp. 48572–48634, 2019.
[2] R. Shakeri et al., “Design Challenges of Multi-UAV Systems
in Cyber-Physical Applications: A Comprehensive Survey
and Future Directions,” IEEE Commun. Surv. Tutorials, vol.
21, no. 4, pp. 3340–3385, 2019.
[3] K. Yousif, A. Bab-Hadiashar, and R. Hoseinnezhad, “An
Overview to Visual Odometry and Visual SLAM:
Applications to Mobile Robotics,” Intell. Ind. Syst., vol. 1, no.
4, pp. 289–311, 2015.
[4] S. Garrido-Jurado, R. Muñoz-Salinas, F. J. Madrid-Cuevas,
and M. J. Marín-Jiménez, “Automatic generation and
detection of highly reliable fiducial markers under occlusion,”
Pattern Recognit., vol. 47, no. 6, pp. 2280–2292, 2014.
[5] E. Marchand, H. Uchiyama, and F. Spindler, “Pose Estimation
for Augmented Reality: A Hands-On Survey,” IEEE Trans.
Vis. Comput. Graph., vol. 22, no. 12, pp. 2633–2651, 2016.
Figure 9. Scan octo map of the indoor environment [6] F. Fraundorfer and D. Scaramuzza, “Visual odometry: Part II:
Matching, robustness, optimization, and applications,” IEEE
V. CONCLUSION Robot. Autom. Mag., vol. 19, no. 2, pp. 78–90, 2012.
We have designed and developed an autonomous MAV [7] F. Kendoul, “Survey of advances in guidance, navigation, and
online vision based path navigation algorithm using ArUco control of unmanned rotorcraft systems,” J. F. Robot., vol. 29,
tracker- a visual markers for self-localization technology in no. 2, pp. 315–378, Mar. 2012.
indoor scenario and mapping of 2D/3D surrounding [8] S. Ahrens, D. Levine, G. Andrews, and J. P. How, “Vision-
environment which is based on SLAM and cartographer based guidance and control of a hovering vehicle in unknown,
gps-denied environments,” Proc. - IEEE Int. Conf. Robot.
technology. We have used several open source known
Autom., pp. 2643–2648, 2009.
technologies by adapting these resources in our purposed
[9] E. Royer et al., “Monocular Vision for Mobile Robot
research work the system has been successfully validated on Localization and Autonomous Navigation To cite this
ROS based Gazebo simulator as well as Rviz and results were version : HAL Id : hal-01635679 Monocular vision for mobile
obtained. An intelligent PID controller has been integrated and robot localization and autonomous navigation,” 2017.
the effectiveness of controller performance shown through [10] J. Campbell, R. Sukthankar, I. Nourbakhsh, and A. Pahwa, “A
experimented simulation results. robust visual odometry and precipice detection system using
consumer-grade monocular vision,” Proc. - IEEE Int. Conf.
Simulation results show that, when MAV flying at a very Robot. Autom., vol. 2005, no. April, pp. 3421–3427, 2005.
high speed, the captured images may get blurred due to which [11] D. Eberli, D. Scaramuzza, S. Weiss, and R. Siegwart, “Vision
the system is limited in velocity. Furthermore, the size of the based position control for MAVs using one single circular
trackers also depends on the velocity of MAV. Moreover, the landmark,” J. Intell. Robot. Syst. Theory Appl., vol. 61, no. 1–
accuracy of developed algorithm can be improved by applying 4, pp. 495–512, 2011.
filtration process combine with advanced methods including [12] A. J. Davison, I. D. Reid, N. D. Molton, and O. Stasse,
kalman filter and other state estimation techniques. Future “MonoSLAM: Real-time single camera SLAM,” IEEE Trans.
lines involve the integration into real hardware platform as Pattern Anal. Mach. Intell., vol. 29, no. 6, pp. 1052–1067,
well as implementation with other vision based auto- 2007.
localization algorithms, control approaches and replacing [13] T. Collins and A. Bartoli, “Infinitesimal Plane-Based Pose
Estimation,” Int. J. Comput. Vis., vol. 109, no. 3, pp. 252–286,
them with other sensors without using the marker coded
Sep. 2014.
information at all. Furthermore, sensor data fusion could be
[14] W. Hess, D. Kohler, H. Rapp, and D. Andor, “Real-time loop
implemented in order to get exhaustive position control of closure in 2D LIDAR SLAM,” Proc. - IEEE Int. Conf. Robot.
MAV. Autom., vol. 2016-June, pp. 1271–1278, 2016.
[15] R. Yagfarov, M. Ivanou, and I. Afanasyev, “Map Comparison
ACKNOWLEDGMENT
of Lidar-based 2D SLAM Algorithms Using Precise Ground
The authors would like to thank the Department of Science Truth,” 2018 15th Int. Conf. Control. Autom. Robot. Vision,
and Technology (DST) Govt. of India as part of Indo-Korea ICARCV 2018, pp. 1979–1983, 2018.
Joint Network Centre on Robotics (IKJNCR) for its financial [16] L. Meier, D. Honegger, and M. Pollefeys, “PX4: A node-
support (Grant Number HRD/171030/2018, IIT Delhi) and based multithreaded open source robotics framework for
CSIR-Central Electronics Engineering Research Institute, deeply embedded platforms,” Proc. - IEEE Int. Conf. Robot.
India (GAP-3133). Autom., vol. 2015-June, pp. 6235–6240, 2015.

11th ICCCNT 2020

July 1-3, 2020 - IIT - Kharagpur
Kharagpur,
Authorized licensed use limited to: HCT Musnaa. Downloaded on OctoberIndia
21,2021 at 04:58:32 UTC from IEEE Xplore. Restrictions apply.