The 2019 International Workshop on Research, Education and Development on Unmanned Aerial Systems

(RED-UAS 2019)
Cranfield University, Cranfield, UK

Implementation of Ground Control System for Autonomous Multi-agents

using QGroundControl
Tristan Dardoize, Noé Ciochetto, Ju-Hyeon Hong and Hyo-Sang Shin

Abstract— This paper presents an integration approach to even use internet (additionally to the local network of UAVs)
deploy new mission planning methods for multi-vehicle systems to retrieve weather data [3].
within an open source ground control system (GCS), i.e.
Non-military GCS use MAVLink [4] to communicate
QGroundControl. As the autonomous mission planning features
of QGroundControl consist in pre-planning waypoints within with unmanned aircraft systems. This lightweight protocol is
a geofence, there is no way to deal with scenarios where the specially designed for micro-aerial vehicles and permits com-
agents of the swarm must counter moving targets of reduced munication of sensor data, commands and other information.
size, or more generally any scenario where an algorithm of Nonetheless, the protocol is not secured as its messages are
path planning is preferred instead of a manual planification.
not encrypted, therefore making UAVs vulnerable to ICMP
The proposed solution consists in extending the ”toolbox” of
the application (QGCToolbox) with a new tool (QGCTool), in flooding or packet injection [5]. This kind of attacks may
implementing a new controller and modifying views (Flight- lead to partial or complete failure of the system, putting at
DisplayViewMap, PlanView) while enabling information to flow risk the fulfilling of a current mission. Still, investigations
between components using Qt’s signals and slots. A test scenario are made to secure this protocol [6].
is presented to verify new mission planning interface and
path planning algorithm, that involves in this case eight UAVs GCS can also be decentralized [7] and use a single ground
operating in a zone to protect and an area to attack. Also a control system linked with multiple relays. Each relay is
workaround is proposed to increase the default limitation of composed of two levels: an executive level and an automatic
connections to QGroundControl. The proposed methodology is decision level. They permit a wider range of communication
tested and validated with software in the loop and flight test. since the relay distribution covers a larger area than a single
worksystem.
I. I NTRODUCTION
Usually, GCS provide:
For decades, remote controlled vehicles have been em-
ployed for military operations and more recently, for civil • Back and forth communication interface with paired
purposes. As democratisation of Unmanned Aircraft Vehi- UAVs, enabling reception and emission of messages.
cles (UAVs) widespread, prices decreased and open-source • Graphic User Interface (GUI) displaying vehicle posi-
ecosystems could be born. The first of these points led tions on 2D map, their states (GPS coordinates, heading
to whole new branch of applications: swarming UAVs. It and speed), their status (number of connected GPS satel-
consists in the art of coordinating multiple agents together lites, battery information, connection status) and error-
so they can fulfil mission statements more effectively than if related messages (connection issues, sensor calibration
they were uncoordinated or alone. Nevertheless, as conductor failures, processing errors).
are essential to orchestra, multi-agents systems need to be • High level control of UAVs. The first regular mode live
monitored and controlled by Ground Control system (GCS), and manual, which consists in overwriting manually
which are supreme instances providing high-level directives path and controls for a single agent. The second mode
and ensuring normal course of events happening in opera- is associated with mission pre-planning, that handles
tions. a priori coordination of multiple UAVs
In the literature [1], GCS are architectures comprising • Logging of flight information for data analysis or mis-
necessary hardware to operate communications or control sion replays
input to remote UAVs, and a software interface enabling • Configuration tool permitting airframe selection, sensor
pilots to operate out of line-of-sight. In more details, ENAC’s calibration and tuning
Paparazzi [2] describes basic GCS setup as a combination
of a standard laptop connected to a radio-modem which
maintains data link with remote UAVs, and of standard Some GCS handle event-based path planning [8] or ad-
RC transmitters for safety considerations. The software vanced ones [9] (area survey, rally-point and target follow-
interface can prepare and simulate missions, tune the UAVs, ing), but they do not cover the coordination or complex
monitor them and analyse their meta-data. Specific solutions strategies where swarm of UAVs are involved.
The missing element of existing solutions is that they do
Tristan Dardoize, Noé Ciochetto, Ju-Hyeon Hong and Hyo-Sang Shin not detail a general approach to implement new mission
are with the School of Aerospace, Transport and Manufacturing, Cranfield planning algorithms. Design of such specifications would
University, Cranfield MK43 0AL, UK (email: [email protected],
[email protected], [email protected], and require investigations on:
[email protected]

978-1-7281-6600-1/19/$31.00 ©2019 IEEE 24

Authorized licensed use limited to: Auckland University of Technology. Downloaded on May 28,2020 at 18:08:37 UTC from IEEE Xplore. Restrictions apply.
 • How to augment the GUI for allowing users to define they should have a communication system using the same
mission configuration and parameters values. protocol as the GCS, e.g. MAVLink. Also, they should have
• How to retrieve incoming communications to figure out a middleware enabling the transmission of information to the
the state of each vehicle, and out-of-the-box sensed right component of UAVs.
information (such as on-board computer vision outputs). The components of such UAVs are the servo-motors, the
• How to send commands in real-time to the UAVs of autopilot (flight controller), and additional systems such as
the system. cameras, gimbals and a companion computer.
With QGroundControl, the advised hardware and flight stack
As noticed previously, the open-source ecosystem asso- (middleware + autopilot) are respectively Pixhawk and PX4.
ciated to UAVs is not mature yet but many solution exist. C. QGroundControl specifications
Therefore, our work aims at upgrading one of the existing
This section aims at giving core concepts of QGroundCon-
solutions instead of developing one of our own. Hopefully,
trol architecture that are useful for building-up our solution.
this paper will provide keys to researchers and to developers
QGroundControl is a Qt application written in C ++ and its
willing to contribute in this domain.
user interface is described in QML.
II. M ETHODOLOGY The first element to emphasize is the workflow associated
to UAV communications (see Figure 1). Information flows
A. Ground Control System selection
from UAVs and is first pulled by Link objects. Link are
This section explains the comparison procedure that led threaded workers and each one receives bytes of information
to the selection of QGroundControl in a pool of many other from the associated UAV. An additional UAV is associated to
open-source GCS. a new Link, and the creation and destruction of these objects
There are four major desktop GCS: MAVProxy [10], are handled by a unique LinkManager.
QGroundControl [9], Mission Planner [8] and APM Planner From each Link, information flows to the unique
2 [11]. For mobile devices such as tablets or smartphones, MAVLinkProtocol object. It operates the deserialization of
there is Tower [12] and AndroPilot [13]. This study focuses the bytes into objects representing MAVLink messages,
on desktop GCS because of their maintainability. Preferably, and send this information to the MultiVehicleManager,
our choice would balance to an open-source solution that which handles construction and deletion of Vehicle and the
has an active developing community. Consequently, APM transmission of MAVLink message to the right Vehicle.
Planner 2 is discarded and that leaves three possibilities:
MAVProxy, QGroundControl and Mission Planner. Further- Vehicle are objects representing a particular UAV. They
more, the swarming capabilities of these softwares have been store both message data (in containers called FactGroup) and
considered: MAVProxy is meant to run only one vehicle UAV’s firmware specification (FirmwarePlugin) so messages
per instance, QGroundControl is perfectly adapted to multi- can be sent to the associated UAV by targeting the Link
agents systems and Mission Planner offers limited support in to which it is connected. Vehicle objects operate message
swarming. Given these selection criterias, QGroundControl processing with the help of their identifier, and of other
is the best candidate. Consequently, this study aims at features belonging to MAVLink protocol.
improving this particular GCS. Secondary, it is important to recall the high level design
pattern of the software, since it helps understanding how to
B. UAV specifications access to the desired data within the application. Whole solu-
This paper refers to real-time path planning for multi- tion is built around a unique application (QGCApplication).
agents, implying requirements on the target UAVs. This singleton possesses a multiple tools (QGCTool), among
For instance, UAVs must communicate with the GCS to which can be found the MultiVehicleManager explained
handle the mission commands, eventually to transmit sen- before. Each tool is contained in the toolbox (QGCToolbox)
sor information and to maintain the connection. Therefore, and keeps track of the other tools by storing a pointer to

Fig. 1: QGroundControl: UAV workflow

25

Authorized licensed use limited to: Auckland University of Technology. Downloaded on May 28,2020 at 18:08:37 UTC from IEEE Xplore. Restrictions apply.
the toolbox. Consequently, each tool can send MAVLink • Handling user interaction from the GUI: algorithm
commands to the connected UAVs using the FirmwarePlugin parameters (optional) and mission configuration (at
of the desired Vehicle. least start and stop the mission).
Last but not least, it is also important to recall the design
philosophy of the GUI where are involved the views, the The two first interfaces are already natively available for
application and the controllers. For instance, the application each tool of the application’s toolbox. Therefore, our archi-
singleton registers all the objects that will interact with tecture involves the implementation of a QGCTool running
QML views. By interacting, it means that the concerned the path planning algorithm.
objects read data from the views or update data into them. In Considering that algorithm parameters and mission config-
these registered objects can be found the controllers whose uration should be customisable graphically, PlanView must
goal are to handle user interaction (clicks, inputs, etc.). be modified because it gathers mission planning features.
In the context of this study, there are two noticeable top- If visual items should be displayed on the map then
level views: FlightDisplayView which displays the vehicles, FlightDisplayViewMap must be modified as well, otherwise
their status and their path and PlanView which concerns items shown on PlanView are not displayed in FlightDis-
the parametrisation of autonomous missions. These views playView. Since PlanView aims at configuring the mission
can be customised as wished so additional features can be and parametrising the algorithm, it should have a controller.
implemented. Also, as FlightDisplayViewMap is the view of reference
The last point to shed light on concerns the communication when running a mission, then at least items for starting
between registered QML components and other QGCTools. and stopping the mission should appear in the view, and a
The method is proper to Qt framework and is named after controller should handle related user interaction.
signal and slot idea. Basically, signal-slots are inter-thread As views do not share information without connections
communications that enable directed data flow between two between associated controllers, in the case where items
objects (e.g. between one QGCTool and another, between one need to be displayed on the map then there must be a
tool and a QML-registered object, etc.). Information flows information flow between the custom PlanView controller
from the object instance emitting the signal, to all objects and the FlightDisplayViewMap controller. Additionally, if
possessing a slot binded to this signal. the path-planning algorithm generates meta-data that must
It has also been remarked that QGroundControl does not be displayed, then bi-directional communication between the
support more than 7 vehicles connected at the time. tool and the controllers should be implemented.
In summary, UAVs are connected to QGroundControl
using Links objects that receive and send bytes of data from E. Implementation procedure
and to the associated aircraft systems. When data is received, The implementation procedure requires modifications of
it is parsed into MAVLink messages in the MAVLinkProtocol the the project directory tree. For convenience, new files are
tool. Messages are then fed to the right Vehicle object stored in a new folder: src/MultiAgentsPlanning. Seven new
contained in the MultiVehicleManager. Vehicle handle the files are required for a minimal example. And among them,
processing of incoming messages based on their unique Figure 2 explains the 5-steps procedure to take new files into
identifier and storing or updating the values in containers:
FactGroup. MAVLink commands can be sent to UAVs using
the FirmwarePlugin referred in every Vehicle instance.
The main element of the application is the toolbox. It
contains several objects that run different services and that
can access to other tools. User interface is made of cus-
tomisable views written in QML, that can interact with C ++
objects that are registered in the QML engine. C ++ objects
communicate together using signal-slot method.
Although the GCS is handling multi-vehicle support, the
number of simultaneous connections is limited to 7.

D. System architecture
This section describes various components that are re-
quired to implement a new mission planning method into
QGroundControl.
Multi-agents path planning integration requires to design
interfaces for:

• Receiving relevant data sent by the UAVs of the swarm. Fig. 2: Project structure changes to take into account upcom-
• Sending commands to the UAVs. ing modifications

26

Authorized licensed use limited to: Auckland University of Technology. Downloaded on May 28,2020 at 18:08:37 UTC from IEEE Xplore. Restrictions apply.
 Fig. 3: Structure of CustomEditor and of CustomVisuals
Fig. 5: MultiAgentPlanningTool implementation and integra-
tion
account at compilation and run time. The two steps of the
top refers to C ++ whereas the two others relate QM L files.
The algorithm of path planning to be implemented is to its variables and methods through Q PROPERTY and
assumed to require user inputs. Those are described in a Q INVOKABLE declarations. For enabling updates between
CustomEditor QML object. It is displayed on a map, and to views, a middleware called MultiAgentsPlanningTool is nec-
enable user interaction it requires a CustomController. Also, essary. The tool receives signals from one of the Cus-
as the editor can provide buttons to start or stop the mission tomController and transmit this information to the other
there should be this object on both plan and fly views. CustomController with another signal as shown Figure 4.
As users may need to define parameters associated to As different information can be updated at different times,
geographic coordinates (e.g. a zone to attack, an area to several different signals can be required for both controllers
defend), it is interesting to draw them on a map. For and tool.
that is used a CustomVisuals, corresponding to the visual It is important to connect each new controller’s signal
representation of the parameter to display. Different visuals (respectively slot) to the associated MultiAgentPlanning-
can be used for one controller. As they can be edited on Tool’s slot (respectively signal) to enable the information
the plan view but not on the fly one, CustomVisuals should to flow correctly. The details of implementation are given
know the view on which they are used. Figure 3 describes Figure 7: not only the CustomController needs to be im-
the basics of both editor and visuals: plemented, but also requires to be registered by the QM L
• Three properties (a CustomController, the map to sketch engine in QGCApplication.cc. For enabling the connec-
on and a boolean planView) tions of multiple UAVs to QGroundControl, the variable
• Other QML objects/items MAVLINK COMM NUM BUFFERS in QGCApplication.cc
These two QML objects retrieve readable Q PROPERTY needs to be at least equal to twice as much as the maximum
variables from the CustomController and can use the number of vehicles.
Q INVOKABLE methods of it. To take into account user To receive mission parameters the MultiAgentPlanning-
inputs, a CustomController needs to be implemented. It Tool implements slots binded to CustomController. Also, it
extends a public QObject, and provides a QML ”access” has a run method that implements the path planning algo-
rithm. For other considerations, the tool must be accessible
in QML and for that, modifications presented Figure 5 must
be done.
The last step of the integration process concerns the
display of editor and visuals on the application. That is
done by modifying both FlyDisplayViewMap and PlanView,
accordingly to Figure 6.

Fig. 4: Information flow: MultiAgentsPlanningTool and Cus-

tomControllers Fig. 6: CustomEditor and CustomVisuals integration

27

Authorized licensed use limited to: Auckland University of Technology. Downloaded on May 28,2020 at 18:08:37 UTC from IEEE Xplore. Restrictions apply.
 Fig. 7: Implementation and registration of CustomController, increase in the number of simultaneous connections

III. R ESULTS AND DISCUSSION B. Validation and verification

Our algorithm is integrated and information flows as
A. Requirement review expected: user interaction can be taken into account for
running or configuring the custom mission, UAVs receive
This part aims at giving a direct application of the their commands dynamically and autonomously and addi-
theoretical concepts that have been given above. For the tional items associated to mission geometry are displayed
experiment, a swarm of eight drones was studied. It had to on the map. Nevertheless, QGroundControl encounters can-
patrol in specific zones: four of them would be considered not natively connect to more than the maximum number
as defending an area of interest, while the other four are of MAVLink communication channels (7) defined within
considered as attacking an enemy position. The pattern MAVLink protocol itself. A workaround is found by over-
of patrolling would depend on time information, mission riding the corresponding variable with higher values.
geometry, and on the number of UAVs currently active. The Integration tests could be conducted using QGroundCon-
zones had pre-defined shapes (circle, rectangles), dimensions trol’s logging system and the MAVLink Inspector widget.
and relative positions. Three different situations (or rounds) The former permits to easily ensure that signal-slot com-
were considered, so as many geometry configurations. munications are working, that user interaction is taken into
Therefore, design requirements were to: account and that tools and controllers are initialised correctly.
The latter permits to analyse MAVLink messages associated
to each Vehicle, and therefore ensures that the algorithm
• Implement and integrate QML editor and visuals for generates appropriate commands to each UAV.
additional items: a menu for building the zones and a Also, our solution has been simulated in SITL (software in
drag and drop polygon representing the zone. the loop) using jMAVSim [14] to create UAV instances. The
• Implement and integrate a controller for PlanView and problem is that jMAVSim supports only one UAV instance at
FlightDisplayViewMap, enabling the parametrisation of the time, so multiple instances of jMAVSim should have been
the zones for each round. Then, it must have methods launched simultaneously. Overall, it could make us realise
related to the creation of the zones of interest, to display the first mistakes of implementations and lead us toward fly
the different areas of interest in the views and to start tests. An example of mission is displayed Figure 9.
or stop the custom mission. It also has three signals to The results show that a seven-step methodology permits
notify mission changes, and three slots to receive these the implementation and the integration of real-time mission
changes. planning features for multi-agents systems within QGround-
• Implement a QGCTool for the path planning algorithm Control.
that has one slot for running the algorithm and three
slots to receive the configuration for each round. It also C. Risk assessment
has three signals for transmitting each configuration to
the controller of FlightDisplayViewMap. What have not been discussed in this paper are the
• Implement the path planning algorithm. advantages and disadvantages of centralised methods for path
planning of swarms. In facts, GCS becomes a critical node
of the system and the whole mission is expected to fail if the
The resulting specifications of the user interface are shown GCS is damaged. The risk of failure can be worth taken in
Figure 8. some situation, but cannot be considered as a general truth.

28

Authorized licensed use limited to: Auckland University of Technology. Downloaded on May 28,2020 at 18:08:37 UTC from IEEE Xplore. Restrictions apply.
 (a) PlanView: extended mission configuration panel

Fig. 9: SITL Simulation using 7 jMAVSim instances. This

helped us (1) Determine expected performances of our
algorithm (2) Realise that QGroundControl natively support
no more than 7 connected vehicles at one time

the customised MAVLink library to be able to understand

(b) Round 1 Geometry (c) Round 2 Geometry uncommon messages.
IV. CONCLUSIONS
This paper proposed a technical design methodology for
implementing new mission planning features in QGround-
Control. The objective of our study was to facilitate the
work of developers and researchers by providing them
guidelines to prototype their work on a reliable GCS, and
to eventually contribute to UAV open-source community.
This study presents a technical insight on QGroundControl
(d) Round 3 Geometry
that has not been done before, since previous work [15]
discussing QGroundControl extension barely mentioned the
Fig. 8: Mission can be parametrised using the Zones control GCS’s implementation specifications and rather focussed
panel. It permits creation of mission areas with pre-defined on a functional aspect. Finally, the testing process is not
geometry (round 1, 2 and 3). Blue color correspond to the extensive but meant to be easy and quick to set-up. On
area to defend, the red is the one to attack and the light the other hand, test runs should at least be separated from
greyed zone is neutral zone defining the allowed zone of execution run. A reasonable and practical testing approach
operation. will be investigated as future work.
ACKNOWLEDGMENT
The authors would like to express our deep gratitude to
It was also seen that MAVLink is not completely mature
BAE Systems for their supports on the Swarm Challenge.
relatively to multi-agents systems, since the maximum num-
ber of vehicles that is handled by default is hardcoded and R EFERENCES
set to only seven. This protocol has also other weaknesses
[1] J. Zeng, “System framework and standards of ground control station
in the field of security, but it has not been discussed in this of unmanned aircraft system,” in Electrical Engineering and Control,
paper. When flight controllers communicate using MAVLink, M. Zhu, Ed. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011,
they speak in a particular dialect that can be standard or pp. 327–334.
[2] G. Hattenberger, M. Bronz, and M. Gorraz, “Using the paparazzi uav
customised. In the case where custom MAVLink messages system for scientific research,” in IMAV 2014, International Micro Air
were transmitted to the GCS and required by the algorithm, Vehicle Conference and Competition 2014, 2014, pp. 247–252.
appropriate message handlers should be implemented within [3] S. R. Haque, R. Kormokar, and A. U. Zaman, “Drone ground control
station with enhanced safety features,” in 2017 2nd International
the Vehicle class, with their corresponding FactGroup to store Conference for Convergence in Technology (I2CT). IEEE, 2017, pp.
related data. Furthermore, QGroundControl should possess 1207–1210.

29

Authorized licensed use limited to: Auckland University of Technology. Downloaded on May 28,2020 at 18:08:37 UTC from IEEE Xplore. Restrictions apply.
 [4] A. Kouba, A. Allouch, M. Alajlan, Y. Javed, A. Belghith, and
M. Khalgui, “Micro air vehicle link (mavlink) in a nutshell: A survey,”
IEEE Access, vol. 7, pp. 87 658–87 680, 2019.
[5] Y.-M. Kwon, J. Yu, B.-M. Cho, Y. Eun, and K.-J. Park, “Empirical
analysis of mavlink protocol vulnerability for attacking unmanned
aerial vehicles,” IEEE Access, vol. 6, pp. 43 203–43 212, 2018.
[6] N. Butcher, A. Stewart, and S. Biaz, “Securing the mavlink commu-
nication protocol for unmanned aircraft systems : Technical report
csse14-02,” Appalachian State University, Auburn University, USA,
2013.
[7] D. Perez, I. Maza, F. Caballero, D. Scarlatti, E. Casado, and A. Ollero,
“A ground control station for a multi-uav surveillance system,” Journal
of Intelligent & Robotic Systems, vol. 69, no. 1, pp. 119–130, Jan 2013.
[8] “Mission planner,” http://ardupilot.org/planner/, accessed: 2019-09-23.
[9] “Qgroundcontrol user guide,” https://docs.qgroundcontrol.com/en/, ac-
cessed: 2019-09-23.
[10] “Mavproxy,” http://ardupilot.github.io/MAVProxy/html/index.html, ac-
cessed: 2019-09-23.
[11] “Apm planner 2.0,” http://ardupilot.org/planner2/index.html, accessed:
2019-09-23.
[12] “Tower,” https://github.com/DroidPlanner/Tower, accessed: 2019-09-
23.
[13] “Andropilot,” https://github.com/geeksville/arduleader/tree/master/andropilot,
accessed: 2019-09-23.
[14] “jmavsim with sitl,” https://dev.px4.io/v1.9.0/en/simulation/jmavsim.html,
accessed: 2019-09-23.
[15] C. Ramirez-Atencia and D. Camacho, “Extending qgroundcontrol for
automated mission planning of uavs,” Sensors, vol. 18, p. 2339, 07
2018.

30

Authorized licensed use limited to: Auckland University of Technology. Downloaded on May 28,2020 at 18:08:37 UTC from IEEE Xplore. Restrictions apply.