PEOPLE’S DEMOCRATIC REPUBLIC OF ALGERIA

MINISTRY OF HIGHER EDUCATION AND SCIENTIFIC RESEARCH

University Of Science And Technology Houari Boumedienne Bab Ezzouar

Faculty of Computer Science

Department of IT Systems
Master's thesis

Field of study: Computer science

Major : Networks and Distributed Systems

Designing and simulating a routing protocol

for Flying ad-hoc networks based on
Software-Defined Networking

Suggested and supervised by :

Mrs BOUZIANE Nabila
Mrs DOUKHA Zouina

Presented on 29/06/2024 Presented by:

Djouama Mohamed Moundher

Kimri Faudel

In front of the committee members composed of:

Mr BERBAR Ahmed - President

Mrs GUEBLI Wassila - Member

Binomial N° 005/2024
Acknowledgements

First of all, we thank Allah, the Almighty and the Merciful, for giving us the health, the will
and the patience to complete our Master’s course.

This work would never have been undertaken or completed without the patient, assistance,
guidance and the meticulous controls and follow-ups by supervisors,

Mrs BOUZIANE Nabila

&
Mrs DOUKHA Zouina
We would like to express our gratitude and appreciation to them.

We would like to appreciate

Mr BERBAR Ahmed
&
Mrs GUEBLI Wassila
The professors of the USTHB for accepting the presidency of the jury, and to express our
respectful gratitude for their enlightened advice, which will enrich this study.

All those who have contributed in any way to the realization of this work are assured of
our deep consideration.

Without forgetting all the teams of the Faculty of Computer Science.

 Dedication
I dedicate this work to my parents:

May they find here the testimony of my deep gratitude and

acknowledgment

To my brothers and my sisters, my grandparents and my family

who give love and liveliness.

To all those who have helped me - directly or indirectly - and those

who shared with me the emotional moments during the
accomplishment of this work and who warmly supported and
encouraged throughout my journey.

To all my friends who have always encouraged me, and to whom I

wish more success.

Thanks!

Mohamed Moundher DJOUAMA

Faudel KIMRI
 Abstract
In today’s fast-evolving world of wireless networking, creating efficient
and reliable communication systems is essential. This study dives into Flying
Ad-Hoc Networks (FANETs), which are a type of Mobile Ad-Hoc Networks
(MANETs). These networks use Unmanned Aerial Vehicles (UAVs) that
communicate without any fixed infrastructure. We introduce a new routing
protocol designed specifically for FANETs that combines Software-Defined
Networking (SDN) and reinforcement learning, particularly Q-learning, to
enhance path selection and network performance. Our protocol tackles the
unique challenges of FANETs, such as finding efficient paths, managing UAVs
resources, and dealing with quickly changing topologies. Through detailed
simulations, we show that our protocol improves packet delivery rates, cuts
down transmission delays, and boosts overall network reliability. This re-
search helps push forward adaptive and smart networking solutions for aerial
communication systems.

Key words: FANET, Routing protocol, Q-learning, Ad-hoc, SDN.

 Résumé
Dans le monde en rapide évolution des réseaux sans fil, la conception de
systèmes de communication efficaces et fiables est essentielle. Cette étude se
concentre sur les réseaux ad hoc aérien (FANETs), un type de réseaux ad hoc
mobiles (MANET) utilisant des véhicules aériens sans pilote (UAV) qui com-
muniquent sans infrastructure fixe. Nous introduisons un nouveau protocole
de routage spécialement conçu pour les FANETs, combinant les principes
des réseaux définis par logiciel (SDN) et l’apprentissage par renforcement, en
particulier l’algorithme Q-learning, pour optimiser la sélection des chemins
et la performance du réseau. Notre protocole relève les défis uniques des FA-
NETs, tels que la détermination des chemins efficaces, gestion des ressources
des drones et les changements rapides de topologie. Grâce à des simulations
détaillées, nous montrons que notre protocole améliore les taux de livraison
des paquets, réduit les délais de transmission et augmente la fiabilité globale
du réseau. Cette recherche contribue à l’avancement de solutions de réseau
adaptatives et intelligentes dans le contexte des systèmes de communication
aérienne.
Mots clés :FANET, Protocole de routage, Q-learning, Ad-hoc, SDN.
Contents

List of Figures

List of Tables

General Introduction 1

1 Background Overview 3
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Ad Hoc Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2.1 Definition of Ad hoc Networks . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2.2 Types Of Ad Hoc Networks . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2.3 Applications Of Ad hoc Networks . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Flying Ad hoc Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3.1 Definition of FANETs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3.2 FANET characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.3.3 FANET Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.3.4 FANET Architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.4 Software Defined Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.4.1 Definition of SDN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.4.2 Architecture of SDN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.4.3 OpenFlow Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.4.4 SDN integration in FANETs . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.5 Overview of Artificial Intelligence . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1.5.1 Basics of artificial intelligence . . . . . . . . . . . . . . . . . . . . . . . . 20
1.5.2 Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
1.5.3 Q-Learning algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Contents

2 Related Work 26
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.2 Routing techniques in FANETs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.2.1 Store-Carry and Forward . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.2.2 Greedy Forwarding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.2.3 Predictive Forwarding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.2.4 Path discovery technique . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.2.5 Clustering technique . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.3 Routing Protocols in FANETs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.3.1 Traditional Routing protocols in FANETs . . . . . . . . . . . . . . . . . 31
2.3.2 AI-enabled routing protocols in FANETs . . . . . . . . . . . . . . . . . . 34
2.4 Routing Protocols in SD-FANETs . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2.4.1 Adaptive SDN-based routing framework for FANET . . . . . . . . . . . 35
2.4.2 SMURF: Reliable Multipath Routing in Flying Ad-Hoc Networks . . . 36
2.4.3 Routing Protocol for Heterogeneous FANETs with Mobility Prediction : 36
2.4.4 A Software-Defined Networking-based Routing Algorithm for Flying
Ad Hoc Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.5 Summary of SD-FANETs routing protocols . . . . . . . . . . . . . . . . . . . . . 38
2.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

3 Protocol Design 41
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.2 Problem Statement and Motivations . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.3 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.4 Overview of the protocol design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.5 Components of the protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.6 Type of messages and packets in the protocol . . . . . . . . . . . . . . . . . . . . 45
3.7 Virtual cell partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
3.8 General protocol design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.8.1 Gathering information process . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.8.2 Q-learning process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.8.3 Forward process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.8.4 Recovery mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

USTHB/Computer science: 2024

Contents

4 Protocol Implementation and Simulation Study 64

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.2 Simulation and measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.2.1 Material Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.2.2 Simulation Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
4.3 Simulation Design Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.3.1 Node Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.3.2 Mobility Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.3.3 Application Modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.4 Simulation measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.4.1 Simulation Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
4.4.2 Evaluation Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.5 Simulation performance evaluation and results analysis . . . . . . . . . . . . . . 71
4.5.1 Overloaded Network impact on performance . . . . . . . . . . . . . . . . 72
4.5.2 Hexagon cell size impact on performance . . . . . . . . . . . . . . . . . . 75
4.5.3 Initial angle impact on performance . . . . . . . . . . . . . . . . . . . . . 78
4.5.4 Period of learning impact on performance . . . . . . . . . . . . . . . . . . 81
4.5.5 Evaluating the performance for different speeds scenarios . . . . . . . . 83
4.5.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

General conclusion and perspectives 87

Bibliography 89

USTHB/Computer science: 2024

List of Figures

1.1 An Ad hoc Network of cellphones and laptops . . . . . . . . . . . . . . . . . . . 4

1.2 Different types of MANETs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Types of ad hoc networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.4 Flying UAVs Formation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.5 FANETs links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.6 Centralized Formation of UAVs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.7 Ad hoc UAV network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.8 Group formation of UAVs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.9 Layered architecture in FANETs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.10 Global SDN Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.11 OpenFlow Architecture [1]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.12 Reinforcement learning framework . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.1 SCF technique . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.2 Greedy technique . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.3 Predictive technique . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.4 Path discovery technique . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.5 Clustering formation in FANETs . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.6 General scheme for the existing traditional routing classification for FANETs . 30

3.1 General topology of the protocol. . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

3.2 Components of the protocol. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3.3 Virtual cell formation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
3.4 Hexagonal cell parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
3.5 General protocol design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3.6 An UAV link criterion example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.7 Center position criterion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
List of Figures

3.8 Example of Selecting candidates cells based on the initial angle. . . . . . . . . . 57

3.9 Example of increasing the angle to explore valid cells. . . . . . . . . . . . . . . . 58
3.10 Backtrack mechanism. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.11 The selected path by the exploration algorithm. . . . . . . . . . . . . . . . . . . 60
3.12 UAVs path selection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

4.1 Representation of Node modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

4.2 Representation of Deployed Application in the protocol . . . . . . . . . . . . . . 69
4.3 PDR measurement for different load scenarios. . . . . . . . . . . . . . . . . . . . 73
4.4 Average delay measurement for different load scenarios. . . . . . . . . . . . . . . 74
4.5 Re-routing ratio measurement for different load scenarios. . . . . . . . . . . . . 75
4.6 Pdr results with the cell size variation. . . . . . . . . . . . . . . . . . . . . . . . . 76
4.7 Average Delays results with various cell hexagon sizes. . . . . . . . . . . . . . . 77
4.8 Re-routing with cell size variation. . . . . . . . . . . . . . . . . . . . . . . . . . . 78
4.9 Pdr results with the initial angle variation. . . . . . . . . . . . . . . . . . . . . . 79
4.10 Average Delays results with intial angle variations. . . . . . . . . . . . . . . . . . 80
4.11 Re-routing varying initial angle. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.12 Pdr results with the learning period variation. . . . . . . . . . . . . . . . . . . . 82
4.13 Re-routing result by varying the learning interval. . . . . . . . . . . . . . . . . . 82
4.14 Pdr results under different speed scenarios. . . . . . . . . . . . . . . . . . . . . . 83
4.15 Average Delays results with speed variations. . . . . . . . . . . . . . . . . . . . . 84
4.16 Re-routing varying the speed value. . . . . . . . . . . . . . . . . . . . . . . . . . . 85

USTHB/Computer science: 2024

List of Tables

1.1 Comparative table between different types of MANETs [2] . . . . . . . . . . . . 9

2.1 Summary table of related works. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.1 Example of UAVs information table . . . . . . . . . . . . . . . . . . . . . . . . . . 49

3.2 Example of cell drones mapping table . . . . . . . . . . . . . . . . . . . . . . . . 49
3.3 Example of Q table entries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.4 Example of Flow table for a given UAV. . . . . . . . . . . . . . . . . . . . . . . . 62

4.1 Simulation Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

List of Abbreviations

AI: Artificial Intelligence

CH: Cluster Head
CSMA/CA: Carrier Sense Medium Access / Collision Avoidance
FANETs: Flying Adhoc Networks
GBS: Ground Base Station
GPS: Global Positioning System
GSM: Global System For Mobile Communication
IMU: Inertial Measurement Unit
LTE: Long Term Evolution
MANETs: Mobile Adhoc Networks
MBPs: MegaBytes Per Seconds
RL: Reinforcement Learning
RSSI: Received Signal Strength Indicator
SDN: Software Defined Networks
UAV: Unnamed Aerial vehicles
UDP: User Datagram Protocol
VANETs: Vehicular Adhoc Networks
WSN: Wireless Sensor Networks
WMN: Wireless Mesh Networks

USTHB/Computer science: 2024

General Introduction

In the rapidly evolving technological landscape, the networking field is not immune to
advancements. The network domain is the core of technology, and researchers and contributors
are continually seeking to develop this field to make technology usage easier and more
effective. Wireless networks are a contemporary technology widely used in various fields,
including military, medicine, commerce, and even personal use for messaging and entertainment
purposes.

Flying Ad-hoc Network (FANET) is a sub-field of wireless networking that consists of a

number of Unnamed Aerial vehicles (UAV) and adopts the concept of "No infrastructure",
such as routers and access points, where each node in the network participates in the
forwarding process. This approach presents unique challenges, such as determining an
efficient path from a source to a destination within a reasonable delay. Additionally, FANETs
require careful management of wireless nodes in terms of energy and capability to achieve
both efficient path selection and device longevity.

Several protocols have been proposed [3] [4] [5] in the Flying Ad-hoc Network (FANET)
domain, utilizing various techniques for efficient path calculation. However, these protocols
have faced challenges due to the dynamic nature of FANET networks and the lack of a
global view of the topology of UAVs. To address this issue, various architectures have been
proposed, including the integration of Software Defined Networks (SDN) into FANET to form
SDFANET networks. Although this integration provides a global view of the topology, it is
still challenging to develop a routing protocol that can effectively adapt to the unpredictable
and continuous movements of UAVs, leading to continuous topology changes. One promising
strategy to address this problem is the use of artificial intelligence, which can help in making
more informed and adaptive routing decisions.

Our work aims to propose a routing protocol based on field studies and existing previous

1
General Introduction

work, addressing the considerations mentioned above. To tackle the unique characteristics of
FANETs, various technologies, and techniques are employed, including the Software-defined
networking (SDN) approach, the well-known artificial intelligence algorithm "Q-learning",
and other proposed techniques.

The remainder of this paper is organized as follows:

Chapter 1 provides an overview of the networking field, introducing the reader to the various
technologies and aspects that are utilized in the proposed protocol.
Chapter 2 reviews the current state of the art in the domain, discussing existing protocols
and the strategies employed in routing for Software Defined Network Flying Ad-hoc Networks
(SDFANET).
Chapter 3 delves into the proposed routing protocol, detailing its different components and
features.
Finally, Chapter 4 presents the implementation and testing of the protocol under diverse
scenarios, analyzing the results gathered from these experiments.

USTHB/Computer science: 2024

Chapter 1

Background Overview

1.1 Introduction
In this chapter, we dive into the fundamental concepts of ad hoc networks in general,
with a specific exploration of the subclass of Flying Ad hoc Networks (FANETs) and how
they communicate with each other.
We also discuss the architectural and functional aspects of software-defined networks,
highlighting the key innovations that distinguish them from traditional network infrastructure
and their integration into FANETs.
Finally, the discussion is centered on artificial intelligence, specifically emphasizing
reinforcement learning.

1.2 Ad Hoc Networks

1.2.1 Definition of Ad hoc Networks

Ad hoc networks are considered a type of network where stations communicate directly
or indirectly [6]. As shown in Figure 1.1, multiple phones and laptops are connected without
the existence of a central infrastructure.
Each station is considered a host and a router at the same time, and it participates in
the routing activities to manage routes between stations. It can operate in wired or wireless
settings [6].
This type of network excels in its flexibility, cost-effectiveness, and resilience, making it
ideal for diverse scenarios. These networks can be swiftly deployed in challenging environments
like deserts, surviving natural disasters, and wartime conditions. Their adaptability suits

3
Background Overview

military operations, emergency responses, and disaster recoveries [6].

Figure 1.1: An Ad hoc Network of cellphones and laptops [7].

1.2.2 Types Of Ad Hoc Networks

Ad hoc networks can be classified into types based on their unique characteristics [8].
We present the three main types :

1.2.2.1 Mobile Ad hoc Networks:

Mobile Ad hoc Networks (MANETs) are a type of ad hoc network where mobile nodes move
independently and communicate over wireless links [9].
Nodes like in Figure 1.3.a communicate directly within their radio range and use intermediate
nodes for communication beyond that range, known as multi-hop communication [9].
Each node acts as an autonomous router, facilitating peer-to-peer communication and
independent data transmission, eliminating the need for dedicated routers. There are various
sub-types of MANETs presented below and illustrated in Figure 1.2, each adapted to address
specific networking requirements and environments.

1. Vehicular Ad hoc networks (VANETs) :

It’s a special type of MANETs where vehicles act as mobile nodes communicating
with each other and with roadside infrastructure to enable a wide range of applications.
It differs from MANETs in several features due to the unique environment of vehicular
communication [10].

USTHB/Computer Science : 2024 4

Background Overview

VANETs are characterized by high node mobility, rapid topology changes, on-board
sensors for routing, patterned mobility based on road types and driving behavior, highly
dynamic topology, unlimited battery power in vehicles, and frequently disconnected
networks [10].

2. Flying Ad hoc networks (FANETs):

Another special type of MANETs and VANETs consists of Unmanned Aerial
Vehicles (UAVs) operating in an aerial environment. These networks are designed to
facilitate communication between UAVs and potentially with ground control stations,
forming a dynamic and mobile network in the sky [11].
The unique characteristics of FANETs are discussed in more detail in the next
sections.

Figure 1.2: Different types of MANETs [12]

1.2.2.2 Wireless Sensor Networks (WSNs):

Wireless Sensor Networks are networks that use specialized sensors to monitor and
record physical conditions in specific environments [13]. These sensors as shown in Figure
1.3.b are arbitrarily deployed in an area to collect data on temperature, sound, pollution
levels, humidity, and more [13].
The sensors communicate wirelessly, each of them includes a radio transceiver, a
micro-controller, sensor interfaces, and an energy source [13].
WSNs are used in various applications, such as industrial monitoring, agriculture, and

USTHB/Computer Science : 2024 5

Background Overview

military surveillance. They are cost-effective, low-power solutions that enable real-time data
collection and analysis in diverse settings [13][14].

1.2.2.3 Wireless Mesh Networks (WMNs):

Wireless mesh networks are made up of radio nodes arranged in a mesh topology depicted
in Figure 1.3.c and each node acts as both client and router by forwarding data for other
nodes in the network [13][14].
WMNs have dynamic self-configuration and self-organization, with two types of terminals
within it: mesh routers and mesh clients. Power is provided by the static type of routers that
act as infrastructure for the network. Otherwise, clients link up with their devices, which
may be movable or stationary.
Mesh clients within a WMN allow multiple paths for data transmission between them,
thus enabling effective communication from source to destination [15].
Figure 1.3 represents an element that composes different types of wireless ad hoc networks.

Figure 1.3: Types of ad hoc networks [9] [16] [17].

1.2.3 Applications Of Ad hoc Networks

By leveraging Ad hoc features, ad hoc networks offer a flexible, reliable, and cost-effective
solution for a diverse array of applications.
We mention application like :

• Military Operation: Ad hoc networking lets the military use existing network systems,
allowing them to exchange information between vehicles, soldiers, and military bases,

USTHB/Computer Science : 2024 6

Background Overview

aiming also to provide reachability in known and unknown areas [18][19].

• Disaster Management: Rescue services use mobile ad hoc networks to communicate

during disasters. They can quickly set up these networks with inexpensive, portable
devices. This allows them to coordinate and share vital information in real-time [18][19].

• Education: The field of education benefits greatly from Mobile Ad Hoc Networks
(MANETs), which enhance teaching, learning, and communication. These networks
can quickly create temporary multimedia connections on devices, enabling the easy
sharing and exchange of information [18][19].

• Personal Area Network: A personal area network (PAN) connects nearby devices.
It allows mobile computers and cellphones to link through short-range networks like
Bluetooth [18][19].

1.3 Flying Ad hoc Networks

1.3.1 Definition of FANETs

FANET is a special type of ad hoc network that is made up of a collection of unmanned
aerial vehicles (UAVs) as shown in Figure 1.4, connected through wireless links, and work
together to coordinate and communicate to complete a mission without the need for human
intervention [20].

Figure 1.4: Flying UAVs Formation [11].

USTHB/Computer Science : 2024 7

Background Overview

1.3.2 FANET characteristics

FANETs have a unique characteristics that create challenges that need to be addressed.
The following are their main characteristics [21][22] :

• Mobility: The mobility in FANET is very high, the node’s speed varies from 6km/h
to 400km/h depending on the UAV’s type, with a random or predefined trajectories
movement in 2D or 3D pattern depending on the task [22].

• Density: FANETs density is low, unlike MANET or VANET, which sometimes makes
the communication range of an UAV node large to ensure a level of coverage in a specific
area [22].

• Line-of-sight: There is no interaction with physical obstacles. FANETs have LOS due
to the nature of their functionality in the air, where the obstructions are much less in
contrast to the ground [22].

• Energy issues: Nodes in FANET have no chance to be charged with power during
the task. Each node has its own consumption of energy based on its size, the hardware,
and sensors, the traveled distance, and more [22].

• Localisation: FANETs leverage GPS technology to track the locations of high-speed

UAVs, which is essential for managing the frequent topology changes in these networks
[22].

• Ground Base Station (GBS): A GBS is like a central operator for drones, handling
important tasks like sending and receiving flight data (speed, altitude, battery status),
communicating with all nearby drones, evaluating communication quality between
drones, and issuing commands such as altitude adjustments, processing instructions,
and speed settings [21].
Essentially, GBSs act as the control center for coordinating and managing drone operations
efficiently [22].

• Coverage: UAVs are capable of covering large areas, enabling their deployment for
a wide range of applications. Their ability to access difficult-to-reach locations and
provide aerial perspectives makes them valuable tools in various fields [21].

• Wireless technology: UAVs can support various types of wireless technologies, primarily
IEEE 802.11 and its variants such as 802.11ac, 802.11n, 802.11s, 802.11b, 802.11g,

USTHB/Computer Science : 2024 8

Background Overview

802.11a, and 802.11p. They also utilize IEEE 802.15.4 and infrared standards for
short-range communication [21].

Since FANET is considered a subset of VANETs and MANETs, we present a table for
comparison between different types of MANET networks [2].

Table 1.1: Comparative table between different types of MANETs [2]

Characteristics MANETs VANETs FANETs

Type of node Mobile, laptop Vehicles Drone , Aircraft , Copter
Environment Near to Earth Highways , Roads Aerial
Mobility Low High Very High
Mobility Model Random Depends on roads traffic Semi-Random or Predetermined
Trajectories
Density High High Low, Depends on the mission
Energy Low High High

1.3.3 FANET Communication

Communication between Unmanned Aerial Vehicles and centralized entities is facilitated
through three main types of links [21]: UAV to UAV communication, ground station to UAV
communication, and UAV to satellite communication, as shown in Figure 1.5 and discussed
below.

• UAV-UAV Communication : UAVs establish direct communication by exchanging

data packets among themselves [21], example in 1.5 (b).
However, given the limitations on transmission ranges, UAVs often engage in
multi-hop communication, where data is relayed through intermediate UAVs to reach
destinations beyond individual UAV ranges [21].
This multi-hop communication is essential for expanding the coverage of specific
areas of interest within the network, enabling UAVs to collaborate effectively and
maintain connectivity over larger distances [21].

• Ground-UAV Communication: In this type of communication, ground-based stations

(GBSs) serve as essential fixed infrastructures on the ground for exchanging crucial
control and data messages. This organization is represented in Figure 1.5 (a) [21].
The GBS plays a vital role in linking various groups of UAVs together, facilitating
communication and coordination within the network [21].

USTHB/Computer Science : 2024 9

Background Overview

Certain UAVs are designated to communicate directly with GBSs, aiming to

alleviate network congestion, boost throughput, and improve overall connectivity [21].

• Satellite-UAV Communication: Another form of centralized entity is satellite, as

illustrated in Figure 1.5 (c), which is essential for ensuring uninterrupted communication
where installing ground-based stations in challenging environments like oceans and
mountains is impractical [21].
Satellites emerge as a viable solution, acting as relays to centrally manage UAVs
and offering crucial Line-of-Sight (LoS) coverage, establishing satellite communication
(SATCOM) within the FANET architecture [21].

Figure 1.5: FANETs links [21].

1.3.4 FANET Architectures

Operating within dynamic environments, FANETs face distinctive challenges that necessitate
tailored architectural solutions. Examining the diverse network topologies, and communication
structures inherent to FANETs provides insight into the detailed design elements that enable
effective and dependable communication among UAV nodes. Here we present different
architecture used in literature [20][21] :

1.3.4.1 Centralized Architecture :

This basic topology features a central node, typically the ground stations shown in
Figure 1.6, directly connecting all UAVs [21] [20].

USTHB/Computer Science : 2024 10

Background Overview

Communication between UAVs is routed through this central node, which acts as a
router in traditional networks. However, this setup poses challenges such as increased load
on the ground station, which not only handles data and control processes for UAVs but also
manages communication between UAV nodes, potentially leading to delays in communication
[21] [20].
Additionally, maintaining connectivity between distant nodes and the ground station
requires costly wireless devices and power supplies. This architecture is suitable for small-scale
setups handling non-critical tasks [20].
Many technologies have been proposed to enable wireless links between UAVs and
the ground station. Each technology has different characteristics. For example, cellular
technologies (GSM, LTE, 5G) provide up to 10 Gbps and more in data rates and a large
coverage of up to 1 km. There are also Wi-Fi technologies (IEEE802.11 a,g,b,n,ac) that
provide from 54 Mbps to 866,7 Mbps in 30 meters and 100 meters ranges [21] [23].

Figure 1.6: Centralized Formation of UAVs [21].

1.3.4.2 Decentralized Architecture :

Communication occurs between UAVs without relying on a ground station. Packets

originating from one UAV and intended for another are either directly routed (if a direct
link exists) or relayed through intermediate nodes serving as relays. We distinguish three
varieties of this type of architecture [20] [21].

• UAV Ad hoc Network: All UAVs are interconnected in this architecture, either

USTHB/Computer Science : 2024 11

Background Overview

directly or indirectly, with each node participating in data forwarding.

Communication between UAVs and the ground station passes through a specialized
UAV called the backbone, which acts as a gateway, as illustrated in Figure 1.7. The
backbone is equipped with two wireless devices: one with long-range and high-power
capabilities for communicating with the ground station, and another with short-range
and low-power capabilities for communicating with other UAVs.

Figure 1.7: Ad hoc UAV network [21].

• Multi-Group Network: This architecture combines elements from the previous

designs by dividing drones into multiple groups, each group with its dedicated backbone
UAV linked to the ground station illustrated in Figure 1.8.
Intra-group communication operates in an ad hoc manner, with UAVs handling
routing internally and inter-group communication requiring routing through the ground
station, resulting in a semi-centralized structure. This setup is ideal for tasks involving
numerous UAVs with diverse flight requirements.

USTHB/Computer Science : 2024 12

Background Overview

Figure 1.8: Group formation of UAVs [21].

• Multi-Layer Network : This architecture divides communication into distinct layers.

At the base, UAVs within each group communicate in an ad hoc manner, either directly
or indirectly.
The communication in this architecture consists of two layers. The lower layer
(Layer 1) handles communication between individual UAVs within a group, while the
upper layer (Layer 2) facilitates communication between the backbone UAVs of all
groups and the ground station.
This multi-layered approach eliminates the need for all communication to pass through
the ground station, as shown in Figure 1.9, reducing the load on the central node and
mitigating the risk of a single point of failure.

USTHB/Computer Science : 2024 13

Background Overview

Figure 1.9: Layered architecture in FANETs [21].

1.4 Software Defined Networks

1.4.1 Definition of SDN

Software-defined Defined Networking (SDN) is an innovative networking architecture
designed to enhance flexibility, programmability, and control over network resources. It
achieves this by dissociating the data plane, which is responsible for forwarding network
traffic, from the control plane, which manages and configures the network operations [24].
This architectural separation as shown in Figure 1.10 is a fundamental concept in SDN, which
leads to the introduction of a component that acts as a middle communication interface and
facilitates the information exchange between the data plan and the control plan in both
directions [24]. A detailed explanation is provided in the next section.

1.4.2 Architecture of SDN

This section examines the basic structure of Software-Defined Networking. It looks at
the main parts that make up this network model and the key layers and interfaces that allow
SDN to separate its functionalities [25].

USTHB/Computer Science : 2024 14

Background Overview

1.4.2.1 SDN Layer Fundamentals:

Researchers have defined three separate layers that build the global paradigm of SDN: the
infrastructure layer, also called the data layer, the control layer, and finally the application
layer, which are interconnected by interfaces and coordinated by a central unit named the
controller [25], as shown in Figure 1.10.

■ Infrastructure Layer: This layer is composed of switching devices that are interconnected
with data channels to build a network. It includes the physical network elements like
Ethernet switches, routers, optical switches, virtual switches, and wireless access points
that form the data plane using different transmission media like copper wires, wireless
radio, and optical fiber. These physical forwarding devices are referred to as switches
and act as the forwarding hardware based on rules installed by the controller [26] [27].

■ Control Layer: The Control layer is a core layer that serves as an entrance between the
application and infrastructure layers. It is responsible for managing the data gathered
from the infrastructure layer along with establishing the rules and policies from the
application layer that are applied and transmitted as flow rules through control channels
to the physical devices [26] [27].

■ Application Layer: The primary characteristic of the application layer is the flexibility
to deploy different applications for networks according to business needs, for example,
routing, policy enforcement, load balancing, and firewalls, as it obtains a more abstracted
perspective of the network from the information and the statistics obtained from the
control layer. Another characteristic is the ability to implement strategies programmatically,
which eases manipulation and control over the physical infrastructure that lies beneath
the network [26] [27].

1.4.2.2 SDN Controller

The SDN controller is a centralized system that sits over the control layer, allowing it to
isolate different layers of the architecture and permit communication through interfaces [28].
Examples of these operations include receiving data and statistics from physical devices and
also pushing application-validated rules and policies to the devices [28].

USTHB/Computer Science : 2024 15

Background Overview

1.4.2.3 SDN Communication Interfaces

As mentioned earlier, the interfaces play a crucial role in allowing the exchange of
information between layers, making the controller adjust network operations in a real-time
manner, and offering a standardized manner to access network resources and expose applications
services to the infrastructure [29].
The following discussion provides a general view of the interfaces used in the architecture
[29]:

● Northbound Interfaces: The northbound interface acts as a common interface

between the controller and application plane. It serves to provide the underlying
network information for application development, making SDN control simple and
dynamic. It requires less standardization effort because it is involved in the software
ecosystem.

● Southbound Interfaces: The interface is used by the controller to communicate and

interact with the infrastructure layer. Its main objective is to monitor and gather
information about the state of the underlying elements of the network as well as
transmit the policies applied to the devices as flow tables, which are discussed in the
next section. In addition, the interface offers an open and standardized interface that
addresses the problem of heterogeneity in physical systems. Multiple protocols are
needed to establish this communication, for example, ForCes [30], OpenState [31].
However, OpenFlow [1] is the most common and used one.

Some works also consider the implementation of east-west bound interfaces to deploy in the
case of a multi-domain or distributed SDN environment [32] [33].
This communication between the controller and the SDN planes aims to get a synchronized
system as well as share the same global view of the network and also promote some benefits
like offering scalability and interoperability [33]. Figure 1.10 illustrates the main components
of the SDN architecture discussed above, which present the connection between the three
layers and also a simple demonstration of the east-west interfaces between controllers.

USTHB/Computer Science : 2024 16

Background Overview

Figure 1.10: Global SDN Architecture [34].

1.4.3 OpenFlow Protocol

1.4.3.1 Definition of OpenFlow :

OpenFlow [1] is a standardized protocol that enables communication between the control
plane and the data plane within SDN environments, thus enabling the controller to in turn
manipulate the forwarding operations of network hardware and retrieve statistics from them.

1.4.3.2 Process of OpenFlow Protocol:

The implementation of the OpenFlow protocol is centered around defining three key elements
that are included in the OpenFlow architecture: The data plane consists of OpenFlow

USTHB/Computer Science : 2024 17

Background Overview

switches, the control plane is made up of OpenFlow controllers, and a secure channel facilitates
the connection between switches and the controller [1].
– The controller function in this process consists of two major operations [35][36] :

✓ Translation of policies received from the application plan into commands known as flow
rules. This process involves converting higher-level policies into low-level instructions
that govern how network traffic should be handled.

✓ Install flow rules either before setting up the network or on demand from OpenFlow
switches.

✓ Gathering Topology information and statistics from the data plan.

– Regarding the OpenFlow Switch, it doesn’t have any decision-making functionalities. It

either processes the packet or drops it according to a valid flow rule. His behavior is
summarized in the following steps: [35][36] :

1. Push the received flow rules into flow tables.

2. Matching packets with one or more flow tables, each containing flow entries that specify
how packets should be handled.

3. If no matching flow entry is found in the flow tables, the switch contacts the controller
to request instructions on how to process the packet.

A demonstrative Figure 1.11 represents the components that enable the OpenFlow architecture.

USTHB/Computer Science : 2024 18

Background Overview

Figure 1.11: OpenFlow Architecture [1].

1.4.4 SDN integration in FANETs

The challenges posed by rapidly changing topologies in FANETs, such as high mobility
of nodes, dynamic topology, interference, and energy constraints, could be addressed by
leveraging the flexibility and programmability offered by SDN.
By defining a protocol stack in software for a centralized control and dynamic configuration of
network resources, UAVs can be programmed to adapt to different environments and network
conditions [37].
The following points outline key characteristics needed in FANETs networks and how
software-defined networking aims to address these essential aspects [37][38]:

✦ Management of Unreliable Links

The SDN can select the most effective link or channel, As its primary capability lies in
having a global view of the entire network through the OpenFlow protocol.

✦ Flexible Node’s Control and Mobility

Unlike decentralized formations, where UAVs operate based on their limited local view
and are constrained in terms of communication and resource utilization, SDN enables
effective centralized control of UAV behavior. The SDN, with its global perspective of
the network, can make better decisions to optimize the actions of individual UAVs.

USTHB/Computer Science : 2024 19

Background Overview

✦ Computational Complexity
SDN enables complex computations to be offloaded from resource-constrained UAVs to
the SDN controller, which has greater processing power and can make global decisions
based on a comprehensive view of the network.

1.5 Overview of Artificial Intelligence

In recent years, the field of Artificial Intelligence (AI) has undergone rapid and significant
advancements. To better understand these developments, we will first provide a basic
definition of AI and its sub-fields, with a particular focus on reinforcement learning and
the Q-learning algorithm that we have used in our protocol.

1.5.1 Basics of artificial intelligence

1.5.1.1 Definition of Artificial Intelligence:

Artificial Intelligence is a reference to computer systems capable of performing complex

tasks that historically only a human could do, such as reasoning, making decisions, or solving
problems [39].
It encompasses various sub-fields, including machine learning and deep learning, which
involve developing algorithms that can learn from data and make increasingly accurate
predictions over time [39].

1.5.1.2 Techniques of artificial intelligence

Artificial Intelligence employs a diverse array of strategies to enable machines to exhibit

intelligent behavior and perform tasks traditionally requiring human intelligence.
These strategies leverage mathematical, statistical, and computational methodologies
to analyze data, make decisions, and solve complex problems. Some of the key techniques in
AI are described below [39] :

➔ Machine Learning (ML)

Machine learning techniques enable computers to learn from data autonomously and
enhance their performance progressively through iterative processes based on the information
they acquire [40].
This technique is classified into three different sub-techniques [39]:

USTHB/Computer Science : 2024 20

Background Overview

1. Supervised Learning: This technique uses labeled data with known inputs and
outputs to teach algorithms to recognize patterns and make accurate predictions
or classifications [39][41].
2. Unsupervised Learning: It’s a type of machine learning where algorithms
analyze and process data without the need for pre-defined labeled examples. The
goal is to discover hidden patterns, relationships, and structures within the data
autonomously [39][41].
3. Reinforcement Learning(RL): It’s a framework where an agent learns to optimize
its behavior by interacting with its environment. RL involves learning from the
mapping of states to actions to maximize cumulative reward through trial and
error without prior knowledge of the correct actions [39][41].

➔ Deep Learning (DL): Deep learning, a branch of machine learning, means the use
of artificial neural networks with multiple layers that are called deep architectures to
capture complex data representations [39].

In this chapter, we introduce reinforcement learning, specifically Q-learning, as it has

been employed in our conceptual work.

1.5.2 Reinforcement Learning

Reinforcement learning is a type of machine learning where a system learns to make
decisions by receiving rewards for good actions and penalties for bad ones while interacting
with its environment [42][43].
We present below the fundamental Mechanisms of reinforcement learning as well as the
technique used in literature:

1.5.2.1 Learning components of reinforcement learning

Reinforcement learning is modeled as a Markov decision process (MDP) which is a

mathematical definition of the sequential decision-making problem [44].
To begin presenting the learning mechanisms of reinforcement learning, we should first
define the key components that constitute the reinforcement learning framework, as it is
formed basically of two principle entities [45] :

● Agent: It’s the decision-making entity that learns to behave optimally by interacting
with the environment, the system can either be a single agent or multiple agents [45].

USTHB/Computer Science : 2024 21

Background Overview

● Environment: It’s the world with which the agent interacts. It can be a simulation, a
game, or the real world [45]. It provides the agent with states, rewards, and transitions
to new states based on the agent’s actions [46].
The environment can be modeled as a Markov decision process (MDP) [46].

These two elements interact using key components such as policy, reward, value function,
and model, which are described below [46]:

● State: Current position of the agent in the environment.

● Action: Step taken by the agent in a particular state.

● Policy (π) : It consists of rules mapping states to actions.

● Reward: This function depends on state and action, reflecting the environment’s
response. The agent aims to maximize cumulative rewards to adjust its policy and
explore different actions.

● Value-Action function: This function evaluates the long-term benefit of actions. It

represents the cumulative rewards from a state, guiding the agent to choose actions
with the highest long-term value.

● Model: Describes the environment’s behavior, predicting future states and rewards
based on actions. RL methods are categorized into two types: model-based and
model-free. Model-based RL learns an explicit environment model, predicting future
states and rewards. This allows planning and simulation, making it more sample-efficient
but computationally intensive. Model-free RL directly learns a policy or value function
through trial-and-error interactions without modeling the environment, being simpler
and more robust but requiring more real-world interactions.

1.5.2.2 Learning process in reinforcement learning

The general process of learning is presented by a connection between agent and environment,
as illustrated in Figure 1.12.
Their interaction is described in the following steps:

1. An agent interacts with its environment using a trial-error technique by choosing

random or historically known actions [47].

USTHB/Computer Science : 2024 22

Background Overview

2. At each executed action, the agent receives an indication about the current state of the
environment noted as the observed state s(i+1) as shown in Figure 1.12 and a reward
r(i+1) revealing if the chosen action was good or bad [47].

The goal is to find a policy π that maximizes the total reward over time through trial and
error, guided by various algorithms [47].

Figure 1.12: Reinforcement learning framework [48].

The goal is to find a policy π that maximizes the total reward over time through trial
and error, guided by various algorithms [47]. These algorithms are used to guide the agent’s
behavior, explore the environment, evaluate the quality of his actions, and improve his
performances over successive iterations [46][49].
These algorithms are classified and compared in [46] and they involve: Dynamic
Programming [46], Monte Carlo approaches [46], State–action–reward–state–action (SARSA)
[46], Deep reinforcement learning (DRL) [46], and Q-learning [46], which are discussed with
details in the next section.

1.5.3 Q-Learning algorithm

Q-learning is a reinforcement learning algorithm where agents iteratively adjust their
actions based on the rewards received from the environment [50].
This process is modeled as MDP (Markov decision process), which consists of a set of states
(S), a set of actions (A), and a reward function (R). At each time step, the agent is
in a particular state (s) and takes an action (a) from the available actions in that state [44][50].

USTHB/Computer Science : 2024 23

Background Overview

Another key element of Q-learning is episodes, which refer to a complete sequence of interactions
between the agent and the environment. During an episode, the agent takes actions, receives
rewards or penalties based on those actions, and continues this cycle until it reaches a goal
state or the maximum number of steps is reached [44][51].
The core element of this algorithm is the Q-value denoted as Q(a,s), which represents the
cumulative reward for taking a specific action (a) in a current state (s) and then following
the optimal policy.
This Q-value value is updated using the bellman equation 1.1[44][50]:

Maximum predicted reward, given

z }| {
Q(s, ) = Q(s, ) + α [R(s, ) + γ mx Q′ (s′ , ′ ) −Q(s, )] (1.1)
| {z } | {z }
New Q-Value Reward
Learning rate
Discount rate

Where:

• α is the learning rate, which determines how much the new information overrides the
old Q-value.

• R is the immediate reward received for taking action  in state s.

• γ is the discount factor, which determines the importance of future rewards.

• max Q′ (s′ , ′ ) is the maximum Q-value for the next state s′ and all possible actions
′ .

There are also other key points of the Q learning algorithm [50]:

• Exploration and Experience: The agent must balance exploration (trying new
actions to discover better ones) and exploitation (using the currently known best
actions).
This is often achieved using an ε-greedy policy, where the agent selects the action
with the highest Q-value with probability (1-ε), and a random action with probability
ε [50].

• Convergence: As the agent continues to interact with the environment and update
the Q-values, the algorithm is guaranteed to converge to the optimal Q-values [50].

• Optimal Policy Once the Q-values have converged, the optimal policy can be derived
by always selecting the action with the highest Q-value in each state [50].

USTHB/Computer Science : 2024 24

Background Overview

1.6 Conclusion
In this chapter, we presented the generalities of flying ad hoc networks and software-defined
networks and their characteristics. We also introduced some aspects of artificial intelligence
that we judge are useful for the design of our work.
The next chapter discusses several works about routing approaches used in FANETs
and SDN-FANETs.

USTHB/Computer Science : 2024 25

Chapter 2

Related Work

2.1 Introduction
This chapter explores existing research on routing protocols in FANETs, aiming to offer a
comprehensive overview of the literature, research findings, and advancements in this critical
area of FANET communication.
The chapter begins by providing a comprehensive overview of the fundamental routing
techniques in FANETs. It then delves into the various routing protocols that have been
proposed, including classical routing protocols, AI-based routing protocols, and SDN-based
routing protocols. By reviewing the latest advances in the literature, we synthesize the key
findings, approaches, and limitations of recent research on FANET routing protocols in SDN
environments.

2.2 Routing techniques in FANETs

The routing technique in wireless networks, specifically FANETs, involves selecting the best
relay UAV to forward packets between UAVs based on various factors such as network
topology, UAV mobility, and resource availability to ensure effective communication and
data transfer within the network. In the next subsections, we present some of the most
commonly used techniques in the literature [21].

2.2.1 Store-Carry and Forward

This technique is based on the idea that the forwarder UAV may be unable to find
a potential next relay UAV in their communication range. Consequently, UAVs resort to

26
Related Work

carrying the packets themselves until they encounter another UAV or the intended target
destination. This method is commonly used in FANETs due to an environment with low
density [21]. In Figure 2.1, UAV S will move into the communication range of UAV D,
allowing it to forward the packet.

Figure 2.1: SCF technique [21].

2.2.2 Greedy Forwarding

This routing technique selects the next-hop UAV based on proximity to the destination.
This approach involves choosing the closest UAV geographically to the destination within a
reduced range, accounting for frequent neighborhood changes [21].
In Figure 2.2, UAVs that are about to transmit the packet choose the closest UAV colored
blue in its range to the destination and deny the farthest UAVs are aligned with red.

Figure 2.2: Greedy technique [21].

USTHB/Computer Science : 2024 27

Related Work

2.2.3 Predictive Forwarding

This technique involves estimating the speed and direction of each UAV to determine
its future position. This information is then used to select the most suitable UAV as the next
relay in the routing process. The approach begins by estimating the speed and direction of
each UAV, which is used to predict its future position. The future positions are predicted
and analyzed to identify the UAV closest to the destination, which is then chosen as the next
relay [21].
Figure 2.3 depicts the future positions of UAV 2 and UAV 3 relative to UAV 1, where the
estimated next position of UAV 2 is within the transmission range of UAV 1, but UAV 3’s
estimated next position exits its transmission range.

Figure 2.3: Predictive technique [21].

2.2.4 Path discovery technique

This technique is based on broadcasting a route request message denoted as rreq to
search for the destination in the network. If the destination exists, it replies with a route
reply message on a single reverse path [21]. Figure 2.4 presents a simple example of the path
discovery technique.

USTHB/Computer Science : 2024 28

Related Work

Figure 2.4: Path discovery technique [21].

2.2.5 Clustering technique

The technique involves the formation of multiple groups of UAVs, known as clusters, to
create the network [21]. Each cluster includes a cluster head (CH), gateways, and ordinary
UAVs. The cluster head manages communication within the cluster and with other clusters.
Gateways facilitate inter-cluster communication, while ordinary UAVs are part of the cluster
but do not participate in inter-cluster communication [21][52].
This technique optimizes resource management in networks and minimizes routing overhead
[52]. A simple representation of this structure is presented in figure 2.5, where green UAVs are
cluster heads that enable forwarding the packet from the source in cluster 2 to the destination
in cluster 1.

Figure 2.5: Clustering formation in FANETs [21].

USTHB/Computer Science : 2024 29

Related Work

2.3 Routing Protocols in FANETs

Routing protocols are a set of techniques and algorithms that manage communication
between nodes and determine the most efficient paths for transmitting data [53]. This
combination typically includes a set of rules, procedures, and mechanisms that allow nodes
to exchange information, make routing decisions, and adapt to changing network conditions
[53]. These techniques and algorithms vary depending on the protocol and may incorporate
additional features, for example, congestion control, security measures, or quality of service
guarantees [21].
The extensive range of classifications in [21], as depicted in Figure 2.6, has prompted us to
focus on presenting the most relevant ones that align with our research objectives in the fields
of artificial intelligence and software-defined networking. The wide variety of categories in
[21], illustrated in Figure 2.6, center on conventional routing protocols that do not employ
artificial intelligence (AI).

Figure 2.6: General scheme for the existing routing classification for FANETs [21].

USTHB/Computer Science : 2024 30

Related Work

2.3.1 Traditional Routing protocols in FANETs

a) Topology-based routing: These protocols establish routing paths using the information
from the link between the UAVs and determine the shortest path according to the hop
count [21]. They are classified into several categories:

• Static: This method is not well-suited for FANETs because of the dynamic nature
of UAVs, which are restricted to communicating with a predetermined set of UAVs
and base stations. The routing tables are fixed before deployment, and the UAVs
cannot recover from link failures because they cannot update their routing tables
in real time [21]. Some protocols are proposed in this category, such as LCAD
(Load Carry and Deliver) [54].
• Proactive: This category uses routing tables on each UAV to store information
about all newly established connections between each pair of UAVs. These tables
are updated whenever the network topology changes. This enables the efficient
identification of the shortest route between a source and a destination UAV [21].
For example, the protocols OLSR (Optimized Link State Routing) [55] and DSDV
(Destination Sequenced Vector) [56].
• Reactive: This category utilizes the path discovery approach, where the route
discovery process is activated only when a UAV needs to communicate. In contrast
to proactive protocols that maintain predetermined routes regardless of communication
needs, reactive on-demand routing initiates route discovery exclusively when a
connection is required. This process identifies the maximum number of available
routing paths as needed, reducing the overhead associated with maintaining outdated
routes [21].
For instance, protocols such as Dynamic Source Routing (DSR) [57], Ad-hoc
On-demand Distance Vector (AODV) [58], along with its extensions like TS-AODV,
used for collision control and bandwidth management, and M-AODV utilize multicast
communication.
• Hybrid: This approach combines proactive and reactive elements within a unified
process. For example, ZRP (zone routing protocol) [59].

b) Position-based routing: These protocols focus on routing based on the physical

locations of UAVs rather than traditional addressing methods. Using the spatial
coordinates of the UAVs, these protocols aim to route packets greedily towards their

USTHB/Computer Science : 2024 31

Related Work

destinations whenever possible. It is classified into three categories, such as reactive,

greedy, and predictive approaches [21].

• Reactive: This category employs the path discovery technique, where the route
discovery process is initiated only when a UAV needs to establish communication.
In on-demand reactive routing, when a UAV requires a connection, it triggers a
route discovery process to explore and define the maximum number of available
routing paths based on the positional information of UAVs. This approach ensures
that the routing process is only initiated when necessary, reducing the overhead
of maintaining stale routes and improving the efficiency of the network [21]. It
has protocols, for example, ARPAM (Ad hoc Routing Protocol for Aeronautical
MANETs) [60] and RGR (Reactive-Greedy Reactive) [61].
• Greedy: The greedy approach operates when the network is mostly connected
and the destination location is known. Data packets are transmitted to UAVs,
which route them to their final destination via the shortest route. [21]. The most
known protocol is GPSR (Greedy Perimeter Stateless Routing) [62].
• Predictive: Routing protocols could be designed to leverage the predictable
movements of UAVs, enabling them to anticipate the duration of wireless connections
between drones. This technique predicts a more accurate estimate of how long
these connections will last [21]. For example, we mention protocols like AeroRP
[63] and P-OLSR (Predictive Optimized Link State Routing Protocol) [64].

c) Energy-based routing: The main criteria for selecting the UAV on a candidate path
is to evaluate its remaining energy level, which consequently evaluates how long the
UAV will remain active in the communication phase [21]. Many protocols have been
designed, for example, CBLADSR [65] (Cluster-Based Location-Aided Dynamic Source
Routing). They combine three techniques, such as reactive and geographical routing
and cluster organization, where the energy level is one of the parameters used to select
the cluster head [21].

d) Secure-based routing: This set of routing protocols includes an additional layer to

ensure a certain level of confidentiality and privacy for communication security [21].
We identify some of the protocols, for example:

• SUANET (Secure UAV Ad-hoc Network Protocol) [66] uses an authentication

mechanism between source and destination to avoid any undesired malicious UAV

USTHB/Computer Science : 2024 32

Related Work

involving [21].
• Another method proposed in SUAP [67] (Secure UAV Ad hoc Routing Protocol)
is to authenticate messages and defend against wormhole attacks. Control packets
contain both fixed data, such as IP addresses, and variable data, like hop count.
The information is secured using digital signatures and hash chains.UAVs decrypt
these packets using the sender’s public key [21].

e) Hierarchical-based routing: Hierarchical protocols involve organizing a network

into different levels to efficiently manage and route data [21]. This method divides the
network into smaller and more manageable clusters. Each cluster has a cluster head,
chosen based on specific criteria. The other UAVs are assigned specific roles, improving
their efficiency in energy consumption, data aggregation, and routing decisions [21].
Here are a few examples of the latest works utilizing this technique: MPCA [68]
(Mobility prediction clustering algorithm for UAV networking) and EHSR (Extended
Hierarchical State Routing Protocol) [69].

f) Swarm-based routing: This type of protocol is inspired by the collective behavior

of natural swarms such as ants, bees, and birds [21]. They address many routing
issues, such as congestion and link failures, in APAR (an ant colony optimization-based
Polymorphism-Aware Routing algorithm) [70] and also in BeeAdhoc (a bee colony
algorithm for FANET routing) [71] for memory and bandwidth management.

g) Heterogeneous-based routing: FANETs are interoperable with a variety of ground

networks, including fixed, mobile, and vehicle networks. This allows drones and other
mobile devices to reliably share data for various purposes [21]. In the literature, we find
protocols like CRUV: connectivity-based traffic density-aware routing using UAVs for
VANETs [72] and Opportunistic routing for multi-flow video dissemination over Flying
Ad-Hoc Networks [73].

h) Delay-tolerant network-based routing: These protocols are used when networks

suffer from fragmentation, making them less connected to each other and worsening
the data delivery ratio [21]. Several categories are used to avoid UAVs using the SCF
technique. We mention three categories: deterministic, stochastic, and social network.

• Deterministic: This technique is used when the network formation is predictable

and the movement of the UAVs is not random [21].

USTHB/Computer Science : 2024 33

Related Work

• Stochastic: When UAVs move randomly, it is impossible to predict where they

will go next, making it hard to choose the next hop. Stochastic protocols send data
packets to all nearby UAVs to ensure gradual delivery to the target destination
[21].
• Social network: When UAV movement patterns and behaviors are known, these
protocols permit accurate predictions of their future positions and directions. This
information can then be used to select intermediate UAVs for forwarding data
packets. However, these routing protocols are new to FANET’s routing work [21].

2.3.2 AI-enabled routing protocols in FANETs

Researchers focus on advanced routing protocols that use artificial intelligence (AI) and
machine learning (ML) to choose the best routes more accurately based on factors like network
layout, signal strength, UAV behavior, and movement patterns. A simple classification covers
topology predictive protocols and self-learning algorithms for routing protocols [74].

a) Topology Predictive Protocols: Studies on these protocols rely on observing UAV

trajectories within the network topology for further analysis and optimization. By
tracking the movement patterns and interactions of the UAVs, researchers gain insight
into the dynamics of the network, enabling them to refine and improve the performance
of communications [74], for example:

• QGEO (Q-learning-based Geographic ad-hoc routing protocol) [75]:

which is a machine learning-based geographic routing protocol that reduces network
overhead in high-mobility scenarios [74]. UAVs make distributed routing decisions
using reinforcement learning without full topology knowledge. It includes location
estimation, neighbor table, and Q-learning modules to enable efficient routing in
dynamic environments [74].
• PARRoT (Predictive Ad-hoc Routing Fueled by Reinforcement Learning
and Trajectory Knowledge) [76]: Another ML-powered protocol has been
proposed that incorporates knowledge of the future movement of mobile UAVs
into the routing process by leveraging mobility control information [74]. Each
UAV predicts its future location using its current position and shares this estimate
with other UAVs in the network [74].

b) Self-Learning Protocols: Learning-based routing protocols often use reinforcement

learning (RL) to make adaptive routing decisions. They continuously learn from the

USTHB/Computer Science : 2024 34

Related Work

environment and the results of their decisions on key performance metrics such as delay,
throughput, and energy efficiency [74]. A significant benefit of RL-based algorithms
is their ability to learn from experience, allowing them to operate independently in
predicting network topology and estimating channel conditions[74]. For example, protocols
like:

– QMR (Q-learning-based Multi-Objective Optimization Routing Protocol)

[77]: Introduce a new Q-learning approach that adjusts learning parameters according
to network dynamics [74]. It explores new paths and re-evaluates neighbor relationships
to choose the most reliable next hop. Results show improved packet delivery, reduced
delays, and lower energy consumption compared to previous Q-learning-based methods
[74].

2.4 Routing Protocols in SD-FANETs

The existing literature indicates that the use of SDN-based routing protocols in FANETs
is uncommon. Most research on FANET routing protocols centers on conventional methods,
with minimal emphasis on SDN-based solutions. Although certain studies have delved into
SDN applications in the FANET environment, they do not directly tackle routing as the
primary issue. We present some of the selected works designed only for routing purposes.

2.4.1 Adaptive SDN-based routing framework for FANET

The authors in [78] propose an adaptive SDN-based routing framework for FANETs
called ASR-FANET. The framework uses an SDN controller fully connected with UAVs to
optimize routing in the network. On the other hand, each UAV has LTE and WiFi interfaces
for communication with SDN and another UAV node, respectively. The SDN controller
is responsible for managing routes between the UAV nodes, as it consists of three main
mechanisms described below that work together to accomplish the task of routing [78].

• Topology Discovery Mechanism: This mechanism is essential for collecting periodic

information about the network’s topology. This includes detailed data on nodes and
links within the FANET, enabling a clear understanding of the network structure and
dynamics.

USTHB/Computer Science : 2024 35

Related Work

• Statistics Gathering Mechanism: This mechanism collects vital information related

to wireless network connections and flight statistics. By monitoring the status of
connections and flight-related data, this mechanism provides valuable insights into
network performance and operational metrics.

• Route Computation Mechanism: It calculates the most efficient path within the
FANET using the gathered information.

2.4.2 SMURF: Reliable Multipath Routing in Flying Ad-Hoc Networks

SMURF [79] is a centralized and probabilistic multipath routing protocol for FANETs
comprising mobile UAVs and software-defined network controllers. The controller monitors
the probabilistic positions of the UAVs and calculates the probability of link existence to
create a routing metric. It calculates the most probable primary route from the source to the
destination, along with backup routes in case the primary route fails. Additionally, it sends
packets over multiple calculated routes instead of just one path. Some of the described steps
in [79] are presented below:

Step 1: The controller collects location updates from the UAVs using a long-range low-bitrate
technology like LoRaWAN. It tracks the UAV’s probabilistic positions rather than
knowing their exact locations.

Step 2: It forms a spanning tree to find the primary route from source to destination with
the highest probability of existence. This is done using an algorithm that calculates
link probabilities based on the joint position distributions.

Step 3: It computes a primary route and optimal backup routes for each link to maximize
reliability when the primary link is considered down.

Step 4: Route information is distributed to the UAVs using the SDN controller.

2.4.3 Routing Protocol for Heterogeneous FANETs with Mobility

Prediction :
The authors in [80] propose adopting the SDN architecture to address some challenges
in the heterogeneous environment of FANET. The FANETs architecture is based on clusters,
and it is divided into an upper-layer UAV called the Cluster Head UAV (CHU) and lower-layer
UAVs, which are members of the cluster called (CMU) and each UAV is equipped with a

USTHB/Computer Science : 2024 36

Related Work

global navigation satellite system (GNSS) for localization. The authors separated the routing
process into two categories :

• Inter-CHU routing (CHU-CHU): The SDN controller makes decisions by analyzing

the global network.

• Intra-CHU routing (CMU-CMU): Each CHU makes his decision based on the
proposed metric called routing estimated cost (REC).

The SDN controller in [80] is considered a centralized Swarm Intelligence (SI) entity. It
executes various tasks of mobility management and routing optimization in the heterogeneous
FANET environment. The authors proposed developing several key modules on the SDN
controller to enable these advanced functionalities:

1. Mobility Prediction Module: This module predicts UAV movement and optimizes
routing paths for CHU nodes using Extended Kalman Filters (EKF).

2. Transmission Rate Estimation and Route Cost Calculation Module: It calculates

the expected transmission rates for each link between CHU nodes using the estimated
mobility and the channel model. It also determines the estimated routing cost (REC)
for each potential path using transmission rates.

3. Routing Module: It models the routing task as a graph problem to minimize total
REC. It also applies the Directional Particle Swarm Optimization (DPSO) algorithm
to optimize path selection. ??? Chooses dynamically the optimal path for CHU nodes,
either (CHU-to-CHU or CHU-to-CMU) based on REC values. Finally, it re-estimates
nodes’s mobility and adjusts the routing using a timeout to handle network dynamics.

2.4.4 A Software-Defined Networking-based Routing Algorithm

for Flying Ad Hoc Networks
This protocol [81] introduces a routing algorithm for Flying Ad-Hoc Networks that
exploits software-defined networking technology to improve the efficiency of routing. The
authors have proposed a combination of proactive and reactive components to optimize
routing decisions based on the geographical positions of nodes. This approach integrates
several services that operate jointly, which are described on the following page.

• Data layer unit configuration service: This service is used to initialize nodes and
their timer periods in the FANETs environment.

USTHB/Computer Science : 2024 37

Related Work

• Topology mapping service: When UAV nodes send their periodic localization, the
controller makes use of this service to build an internal map of the topology to facilitate
upcoming tasks like routing.

• Table control service: Routing tables are set by the location of nodes. Any subsequent
adjustments are communicated directly to the concerned node.

• Proactive routing service: Routes are built using Dijkstra’s shortest path algorithm
and refreshed periodically based on the topology mapping service.

• Reactive routing service: In the case of the occurrence of a route request from a
node because it has not found a valid route, this service, with the coordination of the
topology mapping service, finds an updated route to the requester node.

• Timer management service: This service dynamically adjusts the time period of a
node to send its localization to the controller.

2.5 Summary of SD-FANETs routing protocols

The most recent SDN-based FANET routing protocols are shown in Table 2.1. It helps
in organizing information logically and enables grasping the essential points of studies.

USTHB/Computer Science : 2024 38

 Article AI usage Routing mode Strategy Criteria Simulator Weaknesses
[78] No Two modes: Optimal path Positional Python - Lack of full exploitation of SDN.
based on information software - The transition between centralized
information and statistics platform. and distributed mode may cause
• Centralized collected via (speed, GPS errors.
(SDN). continuous data, IMU
monitoring data).
• Distributed
Related Work

Performance
(Without
information
SDN).
(delay, RSSI,
bit error rate).

[79] No Centralized Route with Link existence MATLAB The process is based on the
(SDN) the highest probability. estimation of probability, which

USTHB/Computer Science : 2024

probability of may lead to incorrect results.
existence. Highest probability path may cause a
high delay and an increasing number
of hops, which leads to a high
possibility of packet loss.
[80] No Two separated Directional Transmission Mathematical The usage of UAVs CHU for
techniques: Particle Swarm rate. model routing-decision making increases
Optimization the load on the UAVs.
(DPSO) Mobility
• Centralized prediction
(SDN)
between CHU.

• Distributed
(CHU)
between CMU.

[81] No Centralized Proactive Shortest path Omnet++ Heavy communication between the
(SDN) controller and the UAVs (location
Reactive information + routing table entries
(Dijkstra) for each UAV) may lead to congestion
and protocol overhead.

39
The simulation results show high
protocol overhead for a small number
of UAVs.

Table 2.1: Summary table of related works.

Related Work

2.6 Conclusion
This chapter discusses the important research conducted in the field of routing in
FANETs. It also looks at protocols that have implemented software-defined networks in
FANETs environments. This exploration gives us a comprehensive understanding of how
SDN integrates with FANET environments for routing purposes.
The next chapter is dedicated to the design of our proposed protocol, which gives a
detailed explanation of every component and mechanism used.

USTHB/Computer Science : 2024 40

Chapter 3

Protocol Design

3.1 Introduction
In this chapter, we examine the various components of the proposed protocol and their
interactions with each other.
Additionally, we exhibit the functionality of each component and explain the reasoning
behind our choices. We begin by providing an overview of the protocol at a high level, then
gradually delve into the more detailed levels.

3.2 Problem Statement and Motivations

The unique characteristics of Flying Ad-Hoc Networks (FANET) make the routing task
challenging. The routing protocol must be adaptive to the high and unpredictable mobility of
the nodes. Additionally, the routing protocols should be efficient in terms of delay and packet
delivery ratio since FANET network tasks are often critical. On the other hand, the physical
limitations of the nodes, such as power storage and restricted processing capabilities, can be
a problem when implementing complex routing protocols capable of handling the FANET
environment.
Despite the pressing demands for efficient and reliable data transmission in Flying
Ad-hoc Networks (FANETs), existing solutions still fall short of meeting these requirements.
Even with the latest advancements in routing protocols, there remains a significant gap in
achieving the desired level of performance. In response to this need, we propose our protocol,
which aims to bridge this gap by addressing the key challenges and limitations in current
solutions. It involves a routing protocol implemented using the Software-Defined Networking

41
Protocol Design

(SDN) approach and the well-known reinforcement learning algorithm Q-Learning.

The choice of the SDN approach is necessary to minimize power consumption from the
UAVs’ perspective, allowing them to focus on their primary tasks while handling only the
forwarding task.
Q-Learning is a robust algorithm suitable for addressing problems that involve long-term
and dynamic decision-making processes. Its effectiveness shines when dealing with scenarios
where the state space and action space are relatively small [3]. Therefore, our approach
restricts the application of the algorithm to geographical zones within the network environment.
These zones represent virtual partitions of the operational area of the UAVs, with each zone
accommodating a specific number of UAVs.
By implementing the Q-Learning algorithm between those zones, we aim to gain a
broader perspective of the network topology. This approach allows us to assess various factors,
such as node density and mobility within each zone. Subsequently, this macroscopic view
aids in determining the most suitable zone when selecting routing paths, ensuring efficient
and adaptive routing decisions based on the network’s dynamics.

3.3 Assumptions
Assumptions taken into consideration during the design of our protocol are :

• At any moment, each drone has a direct link with the controller [23].

• Each drone connects to the global positioning system (GPS) to get its position.

• The position coordinates are reduced from 3D to 2D since UAVs can move on the same
latitude, and their positions are projected onto a 2D hexagonal cell.

3.4 Overview of the protocol design

Given the limitations of drone hardware [82] [83], including energy constraints, limited
processing power, in addition to difficulty in obtaining an overall view of the topology, our
proposed approach shifts the computational burden from the drones to a more powerful
controller. This allows the drones to focus on their primary tasks while the controller
handles the complex calculations and decision-making processes, thereby enhancing the
overall efficiency and scalability of the system.

USTHB/Computer Science : 2024 42

Protocol Design

As discussed in Chapter 1, Q-Learning is a well-suited algorithm for decision-making

[84]. However, its effectiveness decreases as the number of states increases [3]. To address
this limitation, our approach proposes applying Q-Learning at a higher level, focusing on
geographic zones that encompass multiple drones rather than individual drones as shown in
Figure 3.1. This strategy aims to reduce the number of states and improve the algorithm’s
performance, thereby increasing its effectiveness in the decision-making process. The objective
of conducting Q-learning across zones is to gain insight into the status of neighboring zones,
considering factors like node mobility and network congestion. This information is then
utilized to select the optimal paths between nodes, enhancing the efficiency and reliability of
the routing process.

In the context of the proposed protocol, it is assumed that the controller maintains
a direct communication link with each UAV, as discussed in [23]. Several technologies have
been proposed to facilitate this direct link, including satellite communication [23], LTE [23],
5G [23], WiMAX [23], and others. However, for the purpose of abstraction and simplification
during the conceptual and simulation phases, the specific technology used is not a primary
concern. The key assumption is that a direct link exists between the controller and each
UAV, enabling the necessary communication and coordination required for the protocol’s
operation.
By doing that, the number of states is significantly reduced, and an expanded view of the
topology is achieved.

USTHB/Computer Science : 2024 43

Protocol Design

Figure 3.1: General topology of the protocol.

3.5 Components of the protocol

To facilitate understanding of the protocol’s overall design, Figure 3.2 illustrates its
components and interactions with each other.

Figure 3.2: Components of the protocol.

Figure 3.2 illustrates the components of the proposed protocol and their interactions. The

USTHB/Computer Science : 2024 44

Protocol Design

UAVs periodically transmit their status information to the controller. The controller plays
a crucial role in initializing the virtual environment and partitioning the mission space area.
Additionally, the controller utilizes the Q-learning algorithm to determine the optimal path
for the UAVs and communicates this information to them. Each component is explained in
detail in the subsequent sections, providing a comprehensive understanding of the protocol’s
architecture and functionality.

3.6 Type of messages and packets in the protocol

In our protocol, communication between the controller and the UAVs is facilitated through
the OpenFlow protocol using dedicated control channels. To simulate the functionality of the
OpenFlow protocol, we utilize specific messages that enable the exchange of information and
instructions between the controller and the UAVs. These messages are designed to mimic the
behavior of the OpenFlow protocol :

● Hello message (HELLO): This message is sent by UAVs periodically to inform the
controller about their positions.

● Route Request message(RREQ): This message is used by UAVs to demand a route

to a specific destination or consult the controller for a new path in case of a route error.

● Route Reply message(RREP): It is a notification message by the controller to the

UAVs to inform them about their next hop for a given data packet.

● Data Packet(DATA): This packet is used by an UAV; it contains a payload to send

it to another potential UAV.

● Acknowledgement Packet(ACK ): This packet is used to inform the UAV about

the successful reception of a data packet. It is also used to determine if the packet
reached its next hop or not during the forwarding.

3.7 Virtual cell partitioning

In the proposed protocol, the surface is initially segmented into fixed adjacent virtual
regular hexagonal cells as shown in Figure 3.3. Each cell is characterized by its unique ID,
center position, and the coordinates of its six vertices. The controller constructs and stores

USTHB/Computer Science : 2024 45

Protocol Design

this geographical structure in a list for future computations and also maintains a mapping
of each cell and its neighboring cells, along with the direction of these neighbors.

Figure 3.3: Virtual cell formation.

The proposed protocol divides the area into multiple adjacent cells to perform the Q-learning
algorithm between these cells, rather than executing it directly between the UAVs. This
design choice is based on the fact that the Q-learning algorithm works more effectively
when the state space is reduced [3]. Additionally, executing the Q-learning algorithm and
maintaining a Q-table for each individual UAV would be resource-intensive and inefficient. By
shifting the Q-learning process to the cell level, the approach aims to optimize the utilization
of resources while still leveraging the powerful capabilities of the Q-learning algorithm to
make informed routing decisions.
While many existing studies have favored square-shaped zones [85] [86], our protocol adopts
hexagonal zones for several reasons:

• Hexagonal cells offer equal distances to all neighboring cells, simplifying geometric
calculations compared to rectangular zones with varying side lengths.

• Hexagonal zones can efficiently cover the entire space, providing better coverage compared
to circular zones, which may leave gaps or have overlapping areas.

USTHB/Computer Science : 2024 46

Protocol Design

• Hexagonal shapes are commonly used in networking applications.

In the proposed protocol, the dimensions of the hexagon are intricately linked to the
communication range. The size of the hexagon corresponds to the distance from the center of
the hexagon to one of its vertices. The formulas 3.2, 3.1, 3.3, and Figure 3.4 precisely delineate
the dimensions of the hexagon, establishing a clear relationship between the geometric
properties of the hexagon and the communication capabilities within the network.

HS : hexagon size

Hexagon height = 2 ∗ HS (3.1)

p
HW : Hexagon width = 3 ∗ HS = Distance between adjacent cells (3.2)

CR : Communication range = Hexagon height (3.3)

CR

HW HS

Figure 3.4: Hexagonal cell parameters.

Based on the formulas provided in equations 3.2, 3.1, and 3.3, we can conclude that the
communication range of each UAV is slightly larger than the distance between the centers of
two adjacent hexagonal cells. By designing the system in this manner, an UAV positioned
anywhere within a cell will have enough range to communicate with the UAVs positioned in
similar positions in neighboring cells at most.

3.8 General protocol design

The general design of the protocol is shown in Figure 3.5. The protocol consists of three
main processes: Gathering information process, Forwarding process, and Q-learning process.

USTHB/Computer Science : 2024 47

Protocol Design

Figure 3.5: General protocol design.

USTHB/Computer Science : 2024 48

Protocol Design

3.8.1 Gathering information process

In this process, each UAV must periodically transmit its position to the controller. This
information is received by the controller and stored in a database-like structure. The Q-Learning
process utilizes this information on-demand to update the Q-values for each pair (cell-action).
Additionally, the controller maintains a table of Cell-UAVs that associates each cell with the
UAVs currently located within it at a given moment. This table is continuously updated on
the basis of the position information received from the UAVs.

Table 3.1: Example of UAVs information table

Name Position
UAV[1] (110.45 , 155.20)
UAV[2] (213.11 , 105.32)
UAV[3] (50.09 , 56.20)

Table 3.2: Example of cell drones mapping table

Cell List of UAVs

1 uav[1],uav[4],uav[10]
2 uav[6],uav[5],uav[12]
3 uav[11],uav[25],uav[14]

3.8.2 Q-learning process

The learning process involves periodic evaluation of different cells based on information
received from the UAVs. The state space is represented by the cells, while the action
space corresponds to the six directions of a hexagon, representing the possible movements to
neighboring cells.
A simple structure of the Q table is presented in table 3.3

Cell 1 2 3 4 5 6
1 Q-value Q-value Q-value Q-value Q-value Q-value
2 ... ... ... ... ... ...

Table 3.3: Example of Q table entries.

In each episode, the algorithm randomly selects an initial state and then either takes
a random action to explore new options or chooses the action with the highest Q-value to

USTHB/Computer Science : 2024 49

Protocol Design

exploit existing knowledge, depending on the epsilon-greedy approach [45].

The global reward is calculated based on the condition of the new state, which then becomes
the current state, and the evaluation is repeated multiple times. The global reward is
estimated using the formula 3.4. In the following section, we explain rl, rc, and rd in
detail.

Gob Rerd = β1 ∗ r + β2 ∗ rc + β3 ∗ rd (3.4)

Where β1, β2, and β3 are weights, and their sum is equal to 1. This weighting
mechanism allows certain criteria to have a greater impact on the final reward calculation,
reflecting their relative importance in the decision-making process.

3.8.2.1 The UAVs link criterion

First, to determine the distance between two UAVs, denoted i and j, we use the formula :

q
d= ( − j )2 + (y − yj )2 (3.5)

Where  and y represent the coordinates of a drone.

For each drone in the current cell, we assess within the communication range R the relative
distance between it and each drone in the adjacent cell using the formula:

D=R−d (3.6)

Where D signifies the difference between the communication range R and the distance d
between nodes  and j. A negative value of D indicates that the UAV in the neighboring zone
is beyond the communication range, making it an unfavorable choice for the current drone.
Conversely, a positive value of D implies that the two nodes are within the communication
range. Next, we count the number of links that exist and the number of links that do not
exist between two nodes in different cells:
Next, we count the number of links that exist and the number of links that do not exist
between two nodes in different cells:

otRnge + +, if D < 0
(3.7)
nkEst + +, if D > 0

This assessment is performed for each UAV in the current cell with all nodes in the adjacent

USTHB/Computer Science : 2024 50

Protocol Design

cells like shown example of Figure 3.6 . Subsequently, we calculate the reward estimation for
this criterion as follows:

−1 , linkExist = 0
r = (3.8)
(W1 ∗ nkEst) − (W2 ∗ otRnge) , Otherwise

Note that W1 and W2 are weights to balance the reward.

A negative r value suggests that the nodes in the neighboring cell are not suitable for those
in the current cell, while a high positive value indicates that the nodes in the adjacent cell
are relatively reliable for the nodes in the current cell.

Figure 3.6: An UAV link criterion example.

3.8.2.2 Density criterion

The number of UAVs within a zone is a crucial factor in determining the suitability of a
candidate zone as the next hop. A higher number of drones in a zone increases the probability
that the zone is selected as the next hop, as it provides more potential options for forwarding
the data packets.
This criterion ensures that zones with a sufficient number of drones are prioritized. The
reward for this criterion is as follows:

−1 , Nbr of drones in cell = 0
rd = (3.9)
 Nbr of drones in cell , Otherwise
Total Nbr of drones

USTHB/Computer Science : 2024 51

Protocol Design

3.8.2.3 Center position criterion

Having nodes positioned closer to the center of the hexagon, is advantageous. This
arrangement increases the flexibility for selecting the next hop direction, as nodes located
near the middle have more options to choose from in terms of potential communication paths.
This strategic placement enhances the network’s connectivity and adaptability, allowing
for efficient data transmission and routing decisions, an example is shown in Figure 3.7. The
reward is calculated using the formula 3.10 as follows:

Nbr of drones close to center in the neighboring cell

rc = (3.10)
Total Nbr of drones in the network

Figure 3.7: Center position criterion.

3.8.2.4 Learning Phase

The learning phase is initiated periodically following the arrival of UAV information
packets. The steps involved in this phase are outlined as follows:

1. The algorithm randomly selects a cell as the initial state.

2. A random action is taken using the epsilon-greedy approach [45] where a choice is made
among the six neighboring cells to determine the next state. This step aims to assess
the next state relative to the current state.

USTHB/Computer Science : 2024 52

Protocol Design

3. If the chosen action is invalid, meaning the current state is a border cell and the selected
action leads to a non-existent cell, an immediate negative reward is assigned.

4. The reward is calculated based on the formula 3.4.

5. The Q-value for the current state-action pair is updated according to the formula 1.1
given in Chapter 1.

6. The new state transitions to become the current state, and this process is repeated
multiple times within one episode.

These steps are iterated multiple times, known as episodes. Upon completion of the learning
process, a comprehensive and up-to-date view of the network topology is attained, represented
in the Q-table.
In the Q-table, a negative value for a specific state-action pair indicates that the chosen
action is undesirable in that particular state.
Conversely, a positive value suggests that the action is a viable option and can be considered
when making decisions in that state.

It is crucial to note that the Q-value function is structured to assign a negative value
to actions that result in failure. Specifically, if the neighboring cell does not have any UAVs
or if there is no connection between any two UAVs, one in the current cell and one in the
neighboring cell, a negative Q-value is assigned.
This strategy is implemented to mitigate errors in the path selection process by discouraging
actions that may lead to unsuccessful outcomes.
This enables the controller to have a holistic understanding of each zone and its relevant
neighbors, facilitating informed decision-making in the routing process.

USTHB/Computer Science : 2024 53

Protocol Design

Algorithm 1 Q-learning algorithm.

Require: UAVsPositions
Ensure: Fresh_Q-table
1: i ← 0
2: j ← 0
3: while i≤ episodes do
4: currentState ← randomCell()
5: while j≤ steps do
6: action ← chooseAction()
7: if action is invalid then
8: Q(currentState,action) ← -1
9: break
10: end if
11: nextState ← getNextState(currentState,action)
12: reward ← calculateReward(currentState,nextState) using 3.4
13: update Q(currentState,action) using formula 1.1
14: j ← j+1
15: end while
16: i ← i+1
17: end while

3.8.3 Forward process

The forward process starts from the intention of an UAV to send a packet to another
UAV in the network.
The UAV sends a route request packet (RReq) to the controller, requesting a path for the
given destination. The controller in turn receives the RReq, performs the calculation, and
responds with a flow message to all chosen relay UAVs to inform them that a specific packet
is coming and forward it to the specific next hop.
The forwarding process is represented by the following sub-processes: Route Request sub-process,
Cell path calculation sub-process, UAVs path selection sub-process, Flow information sub-process,
and UAV to UAV forwarding sub-process. Each sub-process is described in detail in the
following sections:

3.8.3.1 Route Request sub-process

When the UAV decides to send a data packet to a destination UAV, it sends a route
request to the controller containing a unique identifier that represents this data packet and
the target destination. The controller receives the RReq and extracts the sender, the receiver,
and the packet identifiers.

USTHB/Computer Science : 2024 54

Protocol Design

3.8.3.2 Cell path calculation sub-process

The controller identifies the cells containing the source and destination UAVs. Subsequently,
an exploration algorithm is started to determine the optimal cell path for the UAV source.

The algorithm initiates by assessing neighboring cells aligned with the direction of
the destination cell, utilizing an angle parameter. The angle parameter serves as a filter
for the algorithm, ensuring that it selects only the next hop cell candidates that are in the
direction of the destination. A cell is considered a candidate for a source cell if the center
of that cell falls within the given angle of the source cell towards the destination cell. The
angle is constructed such that the center of the destination cell is equidistant from both sides
of the angle. This approach enables the algorithm to choose paths that minimize deviation,
reducing the computational time required for cell path calculation and improving the overall
performance by avoiding paths with excessive deviation.
The algorithm selects the cell with the highest Q-value associated with it. If there is no
neighboring cell that exhibits a positive Q-value, the angle is adjusted to explore other
potential neighbors. Algorithm 2 provides pseudo-code for the cell path calculation sub-process.

USTHB/Computer Science : 2024 55

Protocol Design

Algorithm 2 Cell path calculation

Require: sourceCell, destinationCell, initialAngle
Ensure: cell_Path
1: searchAngle ← initialAngle
2: offSet ← 20.0
3: currentCell ← sourceCell
4: excludedCells ← {}
5: cellPath ← {}
6: cellPath.add(sourceCell)
7: cellPath.add(-1)
8: while (cellPath.lastElement() == -1) && (cellPath.lastElement()-1 ̸ = sourceCell) do
9: cellPath ← {}
10: cellPath.add(sourceCell)
11: currentCell ← sourceCell
12: while (currentCell ̸= destinationCell) do
13: chosenCell ← -1
14: while (chosenCell == -1) && (searchAngle ≤360) do
15: candidateNeighbors ← getCandidatesNeighbors (currentCell,destinationCell,
searchAngle)
16: chosenCell ← chooseHighestCandidate(candidateNeighbors, excludedCells)
17: searchAngle ← searchAngle + offSet
18: end while
19: if (chosen_cell == -1) then
20: cellPath.add(-1)
21: excludedCells.add(currentCell)
22: break
23: else
24: cellPath.add(chosenCell)
25: end if
26: currentCell ← chosenCell
27: searchAngle ← initialAngle
28: end while
29: end while
30: if (cellPath.lastElement() == -1) then
31: no possible cell path !
32: else
33: return (cellPath)
34: end if

This process is iterated along the entire cell path. In cases where there is no viable
neighbor, the algorithm backtracks to the preceding cell, marking the current cell as an
excluded cell to prevent its selection in subsequent iterations. The search starts again from
the previous cell to explore alternative paths effectively.

USTHB/Computer Science : 2024 56

Protocol Design

Figures from 3.8 to 3.11 illustrate the cell path calculation sub-process in details:

Figure 3.8 illustrates the operation of the cell path composition. The blue cell represents
the source cell, while the pink cell denotes the destination.
The source cell has two potential neighboring cells within the specified angle range, labeled
A and B. Cell B has a negative Q-value, indicating an empty cell. The algorithm selects cell
A as the next cell in the path, as it has the highest positive Q-value among the available
options.

E F G

C D H I J

A
Ini
tia
an l
gle

Figure 3.8: Example of Selecting candidates cells based on the initial angle.

USTHB/Computer Science : 2024 57

Protocol Design

E F G

C D H I J

In
iti
al
an
gl
e
+
va
l
ue
A

Figure 3.9: Example of increasing the angle to explore valid cells.

Figure 3.9 shows that given the current angle, cell A does not have any viable neighboring
cells. To find an appropriate neighbor, the algorithm adjusts the angle. In this case, with
the widened angle, cell C becomes a potential neighbor. The algorithm selects cell C as the
next cell in the path, as it has the highest positive Q-value among the available options after
the angle adjustment.

USTHB/Computer Science : 2024 58

Protocol Design

E F G

C D H I J

Figure 3.10: Backtrack mechanism.

Figure 3.10 shows that the process continues iteratively until cell F is reached. Upon
reaching cell F, which has two potential neighbors, G and H, within the specified angle, F
selects G due to its higher Q-value. However, G does not have any feasible neighbors, as
the UAVs in cells H and I are too distant from G. Consequently, the algorithm backtracks
to the preceding cell, F, and adds cell G to the excluded cells list to prevent selecting it in
a possible future iteration within the same execution, and restarts the exploration process
from that point to identify alternative paths effectively.

USTHB/Computer Science : 2024 59

Protocol Design

E F G

C D H I J

Figure 3.11: The selected path by the exploration algorithm.

Finally, Figure 3.11 illustrates how the path is successfully constructed from cell F until
it reaches the destination cell, following the exploration and selection process outlined by the
algorithm.

3.8.3.3 UAVs path selection sub-process

After establishing the cell path, another algorithm is initiated to select the appropriate UAVs
along the chosen cell path. The strategy employed in this stage is to prioritize the farthest
UAV neighbor within the next cell that is closest to the two-hop neighbor cell as the relay
UAV. If no neighbor is available in the next cell, a local neighbor from the same cell is
chosen, provided that it has the capability to forward data to the next cell. This approach
aims to minimize the number of hops required for data transmission. The rationale behind
this strategy is that the cell path has already been determined by the exploration algorithm,
ensuring the existence of at least one viable UAV path. Additionally, the recovery mechanism,
which will be discussed later in one of the following sections, serves as a safeguard to correct
any potential failures that may occur in the selected relay UAVs. Figure 3.12 shows UAVs
path selection for the given example. Algorithm 3 provides a pseudo-code of the UAV path
selection sub-process.

USTHB/Computer Science : 2024 60

Protocol Design

E F G

C D H I J

Figure 3.12: UAVs path selection.

Algorithm 3 UAV path selection

Require: Cell_path, Src, Dest
Ensure: UAVs_Path
1: Previous_cell ← Cell_path.get(0)
2: Current_hop ← Src
3: UAVs_Path ← {}
4: UAVs_Path.add(Src)
5: for (i← 1 to Cell_path.size()) do
6: if (AreNeighbor(Current_hop,Dest)) then
7: UAVs_Path.add(Dest)
8: return(UAVs_Path)
9: end if
10: Farthest_UAV ← Get_farthest(Current_hop , Cell_path.get(i))
11: if (Farthest_UAV == NULL) then
12: Local_best ← Get_local_best(Current_hop, Previous_cell, Cell_path.get(i))
13: Current_hop ← Local_best
14: UAVs_Path.add(Current_hop)
15: Farthest_UAV ← Get_farthest(Current_hop, Cell_path.get(i))
16: Current_hop ← Farthest_UAV
17: UAVs_Path.add(Current_hop)

USTHB/Computer Science : 2024 61

Protocol Design

18: else
19: Current_hop ← Farthest_UAV
20: UAVs_Path.add(Current_hop)
21: end if
22: Previous_cell ← Cell_path.get(i)
23: end for
24: return (UAVs_path)

3.8.3.4 Flow information sub-process

Once the UAV trajectory is determined, the controller notifies all UAV relays about the
incoming data packet and instructs them on the next hop for forwarding.
It is crucial to note that communication between the controller and the UAVs,
involving messages such as RReq, UAV information, and flow information, occurs in a
dedicated channel separate from the channel used for communication between the UAVs
themselves.
This separation ensures secure and efficient communication between the controller and the
UAVs, maintaining the integrity and reliability of the data transmission process.

3.8.3.5 UAV to UAV forwarding

The UAVs receive flow information from the controller via Open-Flow. Then, they then
store in a flow table 3.4. This table serves as a reference point for the UAVs, allowing them
to consult it whenever a data packet arrives at the node. By referencing this table, the UAVs
can efficiently determine the next hop for forwarding the incoming data packet.

Table 3.4: Example of Flow table for a given UAV.

Packet id IP address next hop

25-1 11.0.0.20
11-4 11.0.0.9
3-3 11.0.0.14
... ...

Communication among UAVs incorporates an ACK (Acknowledgment) mechanism.

In essence, when a relay UAV receives a data packet from the preceding UAV, it must send
an ACK back to confirm the successful packet delivery. If no ACK is received, the recovery
mechanism is activated.

USTHB/Computer Science : 2024 62

Protocol Design

3.8.4 Recovery mechanism

It is crucial to address potential failures that may arise in one or more relay UAVs due
to factors such as congestion or movement of a relay UAV from its anticipated position. To
address these issues, a recovery mechanism is implemented as follows.

• After an UAV forwards a data packet to the next hop, it initiates a timer. Upon the
timer’s expiration, the UAV checks if it has received an ACK from the next hop.

• If an ACK is received, the transmission is deemed successful.

• If no ACK is received, the UAV assumes that congestion has prevented the packet from
reaching the next hop and retransmits the packet.

• This re-transmission mechanism is repeated up to three times. If no ACK is received

within this retransmission period, the UAV concludes that the next hop is no longer a
neighbor.

• In such cases, the UAV consults the controller to calculate a new path starting from
the point of failure.

3.9 Conclusion
The primary objective of our proposed protocol is to develop a system that can adapt
to changing network conditions, particularly the mobility of UAVs, and maintain reliable
data transmission by dynamically adjusting routing paths when necessary. This chapter
provided an overview of the protocol’s functionality, starting with virtual cell partitioning
and its parameters, which divide the area into adjacent cells. We then explained the messages
exchanged between the controller and UAVs to ensure smooth protocol operation. Additionally,
we discussed our usage of the Q-learning algorithm to maintain a global view of the topology
in the form of Q-values for each possible neighbor cell. The forwarding mechanism is also
detailed, starting from the route request from UAVs to the controller, passing through
constructing the cell path and selecting relay UAVs from the controller, and finally informing
those UAVs about incoming flows. Furthermore, we highlighted the recovery mechanism
introduced in the protocol, which is crucial in dynamic environments such as FANETs.
In the next section, we focus on the implementation of our solution and provide an
analysis of different scenarios.

USTHB/Computer Science : 2024 63

Chapter 4

Protocol Implementation and

Simulation Study

4.1 Introduction
After providing a detailed conceptual explanation of our protocol, we present the
implementation of the protocol, walking through the process from defining the necessary
work materials to setting up the simulation environment. We describe the technical details
and considerations involved in bringing the protocol to execution.
Finally, we conduct an in-depth analysis to observe and justify the results obtained
from our implementation. We examine the outcomes through multiple lenses, assessing their
performance and efficiency.

4.2 Simulation and measurement

4.2.1 Material Specification

We executed the simulation of the protocol using the following machine:

• Desktop machine :

– Processor: AMD Ryzen™ 5 5600X @3.7 GHz 4.6 GHz .

– RAM :16GB .
– Operating system: Windows 11 professional .

64
Protocol Implementation and Simulation Study

4.2.2 Simulation Environment

Using simulation tools in network protocol development is crucial in assessing and
evaluating protocols by varying parameters and observing their behavior. By doing this, we
measure performance metrics and become aware of their strengths and weaknesses.
Those tools allow researchers to model network environments, compare protocol performance,
analyze various scenarios, and validate results.
We have chosen to use Omnet++ [87] simulator along with the INET [88] framework.
Our reasons for choosing those specific tools are presented in the following points :

• OMNeT++ and INET are widely used for SDN-FANET simulation and seem to be
preferred by the research community.

• OMNeT++ is free to use and can be run on multiple platforms.

• OMNeT++ offers scalability in terms of complex simulations, which consist of many

nodes.

• OMNeT++ and INET Framework are known for their extensive documentation, providing
detailed resources.

• Examples provided within the INET Framework folder greatly facilitated our understanding
of how to efficiently combine modules. This knowledge enabled us to design and
implement our own custom modules for our specific research requirements.

Those tools are described below.

4.2.2.1 OMNeT++

OMNeT++ [87] is a discrete event simulation platform designed for constructing network
simulators. it provides a modular architecture where complex models can be built from
reusable components called module [87]. It can simulate a wide range of network scenarios
and protocols, including communication networks, multiprocessors, and distributed systems.
A module in OMNeT++ can represent any network component (application, node, network
topology). It is composed essentially of three files that make it operational, which include:

• File (.h): This header file is used for the general declarations of parameters, variables,
and functions used by the module.

• File (.cc): This file is used to define the behavior of the module by implementing the
declared functions.

USTHB/Computer Science : 2024 65

Protocol Implementation and Simulation Study

• File (.NED): This file is used to describe the node components, a node can also extend
functionalities from another node, this extension must occur in this file.

The network topology is defined using (.NED) file, and launched using the configuration file
(omnetpp.ini).

4.2.2.2 INET

INET [88] is an open-source model library for the OMNeT++ simulation environment
that provides protocols, agents, and other models to be used to simulate different network
scenarios.
Some of the key elements provided by INET support a wide range of network types, including
wired, wireless, mobile, ad-hoc, and sensor networks, providing models for the internet stack
like (TCP, UDP, IPv4, IPv6, OSPF, BGP) and for wired and wireless link layer protocols
for example, Ethernet, PPP, IEEE 802.11, etc.

4.3 Simulation Design Model

4.3.1 Node Module

The environment consists of two types of nodes, which are UAV and the controller
represented in the .ned files, for example, UAV.ned for their component description and
their behavior in UAV.cc.
Each element is composed of modules that work together to enable the behavior of a realistic
node. Each module is represented as a stack layer.
As shown in Figure 4.1, a UAV requires two wireless interfaces: one is used to communicate
with another UAV and has a short communication range, and the second is used to communicate
with the controller and has a larger communication range to be able to reach the controller.
The controller can reach any UAV using its wireless interface.
The CSMA/CA protocol is also adopted on the node module to prevent collisions
between packets transmitted by different nodes. This protocol ensures reliable and efficient
communication by leveraging carrier sensing and random backoff mechanisms to distribute
access to the shared medium.

USTHB/Computer Science : 2024 66

Protocol Implementation and Simulation Study

Figure 4.1: Representation of Node modules

To deploy our routing strategy, multiple connected applications must be setup. For
example, to implement the hello application, two sub-applications need to be connected: one
responsible for sending the hello packets and another for receiving them. This interconnected
deployment of applications is necessary for the proper functioning of the application layer,
more details are given in the application module section.

4.3.2 Mobility Module

In our implementation, two types of mobility models are used.
• The controller uses a stationary mobility model.

• UAV use RandomWaypointMobility model .

UAVs are positioned initially in random positions; they begin moving in straight
line segments, with each segment having a randomly selected destination position and
speed. Upon reaching the target position, the UAV pauses for a random wait time
before the algorithm calculates a new random destination, and the process repeats.

4.3.3 Application Modules

For better simplicity and management, each process is divided into a pair of a sub-app
module receiving and a sub-app module sending.

USTHB/Computer Science : 2024 67

Protocol Implementation and Simulation Study

Each pair communicates by sending the required message, as shown in Figure 4.2 and also
by providing information that helps to achieve a desired task, as explained in Chapter 3.
Our routing strategy is defined by the following applications :

• Hello Application: this operation is used by UAVs to inform the controller about its
state

– Hello Sending sub application: This sub-application is used by the UAVs to

form and send the hello packet to the controller.
– Hello Reception sub application: This sub-application is launched by the
controller and waits for any receiving hello packet from UAVs.

• Route Demand Application: This is a reactive application where an UAV demands

a route to a destination UAV from the controller.

– Route Request sub application: This sub-application is used by the UAV to

request a route to a random destination at any moment of the scenario.
– Route Reply sub application: When the controller finishes the calculation of
a route, it forms a RREP packet and sends it to all the intermediate UAV on the
calculated path to inform them about their next hop UAV based on the identity
of the packet.

• Rule Handling Application: When an intermediate node receives multiple RREP

(Route Reply) packets, it will add a flow rule entry to its flow table. This flow rule
entry will correspond to the packet ID and the associated next hop.

• Forwarding Application: This operation requires a forwarding application to manage

incoming data packets and send them to the next designated UAV using the corresponding
flow rules.
There is also an ACK application, which is used to notify the preceding UAV with
an ACK packet about the successful reception of the data packet, and it can also be
used to launch the recovery mechanism in case of not receiving any ACK packet after
a certain period of time.

• Broadcast Application: This simple broadcast application utilizes straightforward

broadcast packets. The purposes of employing this application include increasing
network traffic. For example, in real-world scenarios, additional applications like beaconing
or flooding or any application that only uses communication between flying nodes.

USTHB/Computer Science : 2024 68

Protocol Implementation and Simulation Study

Figure 4.2: Representation of Deployed Application in the protocol

4.4 Simulation measurements

To simulate our proposed protocol, various assessments are conducted. These assessments
require varying parameters to effectively evaluate our protocol in worst-case and best-case.
These cases also depend on the simulator’s limits and the realistic behaviour of the UAV.

4.4.1 Simulation Parameters

We have used different parameters to study and evaluate how they affect the performance
of our protocol, such as number of UAVs deployed, the search angle and also the number of
cells used.
The table 4.1 summarize the simulation parameters.

USTHB/Computer Science : 2024 69

Protocol Implementation and Simulation Study

Parameters Values
Simulation time 200 s
Simulation Surface 1000 m x 1000 m
Network density/Km2 20, 30, 40, 50, 60, 70, 80, 90, 100
Mobility model Random way point
Mobility wait time 1s
Speed Normal distribution (30,8)[89]
Hello Message interval 500 ms
Communication range UAV-UAV 200 m
Data packet size 1000 Bytes
Additional load No additional load, 1s broadcast, 0.5s broadcast
RREQ intervals [ 1.5s,2s ], [ 1s,1.5s ], [ 0.7s,1s ]

Table 4.1: Simulation Parameters

Normal distribution [89] is a distribution function that is used to generate random values
based on the given mean and standard deviation. Normal distribution(8,30) is equivalent to
attributing the speed value randomly each time by value from the range of [6,55] Mps to the
each UAV.

4.4.2 Evaluation Metrics

4.4.2.1 Packet Delivery Ration (PDR)

This performance metric is defined by the ratio of successfully delivered data packets at
a destination to the total number of packets transmitted a the source. It is calculated using
formula 4.1.

Number of Received Packets

 
PDR = × 100 (4.1)
Number of Sent Packets

4.4.2.2 Average Transmission Delay

The delay is the time required for a packet to reach the destination. To assess the delay
of all communication occurring in the network, we must calculate the average delay presented
in the equation 4.2.
PN
=1
(Time of Receiving − Time of Sending )
Average delay = (4.2)
N
Where N represents the number of packets received by the destination in the network.

USTHB/Computer Science : 2024 70

Protocol Implementation and Simulation Study

4.4.2.3 Re-routing Ratio

The re-routing ratio represents the proportion of errors that occur during the forwarding
process, necessitating the controller to be re-contacted for an alternative path. This re-routing
can arise from various factors, including the high mobility of relay UAVs, which may cause
them to change their positions, or congestion in specific areas that prevents UAVs from
receiving data packets or ACK packets. The re-reouting ration is calculated using formula
4.3.

Number of Re-routing attempts

Re-routing Ratio = × 100 (4.3)
Number of RREQ Packets

4.5 Simulation performance evaluation and results analysis

In the following section we will discuss the simulation results for the protocol. Different
scenarios are simulated to have an analytical view on the performance of the protocol.

In the simulation, we designed the work to make each UAV initiates a RReq to the
controller in each period of time. This period is represented by a value randomly chosen in a
given interval. Each UAV uses the same interval. It selects a random UAV destination and
sends a RREQ to the controller which calculates appropriate path to reach the destination.
Then, each UAV repeats the same operation until the end of the simulation. In our simulations,
we employed three distinct sending intervals for the RReq :

• Each UAV initiates RReq each time t in [1.5s,2s] .

• Each UAV initiates RReq each time t in [1s,1.5s] .

• Each UAV initiates RReq each time t in [0.7s,1s] .

In addition, to add more pressure to the network, we introduce additional load into the
system by simulating the broadcast messages between UAVs. These messages are periodic
broadcasts that flood the network, increasing the likelihood of congestion. By doing so, we
assess the protocol’s ability to handle a highly congested network.
The following are the different types of additional load used in the simulations:

• No additional load .

• Broadcast messages initiated by each UAV each 1s .

USTHB/Computer Science : 2024 71

Protocol Implementation and Simulation Study

• Broadcast messages initiated by each UAV each 0.5s .

4.5.1 Overloaded Network impact on performance

To investigate the impact of congestion on the network, we conduct a scenario where we
vary the Request request (RReq) interval for each UAV, which affects the network’s load.
Specifically, we use three different RReq intervals: [1.5,2] seconds, [1,1.5] seconds, and [0.7,1]
seconds.
Additionally, we introduce congestion by broadcasting messages between UAVs, which will
flood the network and test the resilience of the protocol. We also vary the number of UAVs,
maintain a fixed hexagon size of 100 meters, and have a fixed learning period of 0.5 seconds.

4.5.1.1 PDR

Figures 4.3a, 4.3b, and 4.3c represent the PDR simulation results for different RReq intervals,
with different broadcast scenarios. Figure 4.3d presents three distinct scenarios: one that
utilizes the recovery mechanism protocol, the second disables the re-consulting option of the
controller, and the third disables the retransmission option when no ACK is received. The
RReq interval was set to [1.5, 2] seconds, and no broadcast scenario was employed.
The retransmission mechanism is triggered when no acknowledgement (ACK) packet is
received from the next hop, requiring the UAV to resend the packet to the next hop. If
the retransmission mechanism is executed three times without receiving an ACK packet
from the next hop, the UAV will then execute the re-consulting mechanism. This involves
requesting the controller for a new path starting from the current UAV.
As depicted in Figures 4.3a, 4.3b, and 4.3c, the Packet Delivery Ratio (PDR) remains
consistently high, ranging from 92.5% to 95%, regardless of the network load and density.
This remarkable consistency is attributed to the learning algorithm’s ability to maintain
up-to-date information about the UAVs and the recovery mechanism, which prevents packet
loss due to network congestion.

USTHB/Computer Science : 2024 72

Protocol Implementation and Simulation Study

(a) No broadcast. (b) 1s broadcast.

(c) 0.5s broadcast. (d) Recovery mechanism.

Figure 4.3: PDR measurement for different load scenarios.

The graph in 4.3d illustrates a significant performance degradation when recovery measures
are not taken into account. This decline is attributed to the dynamic nature of FANETs,
which renders even ideal forwarding mechanisms insufficient without recovery mechanisms
that address the unpredictability of UAVs in the network.
As we can notice, the PDR significantly degrades when the re-routing mechanism is disabled,
dropping to 87%. This mechanism is executed after not receiving three successive ACK
packets from the next hop. Furthermore, when the retry mechanism is disabled, which is
executed when the first ACK packet is not received from the next hop, the PDR decreases
even further to 84%. These results could be even worse in more congested scenarios, such as
those with a shorter RReq interval of [0.7,1] seconds. From the above, it can be concluded that
the unpredictability of the FANET environment can lead to inappropriate routing decisions,
emphasizing the need for a recovery mechanism to ensure reliable communication.

USTHB/Computer Science : 2024 73

Protocol Implementation and Simulation Study

4.5.1.2 Average Delay

(a) No broadcast. (b) 1s broadcast.

(c) 0.5s broadcast.

Figure 4.4: Average delay measurement for different load scenarios.

Figures 4.4a, 4.4b, and 4.4c illustrate a progressive increase in delay up to 70 milliseconds.
This delay is attributed to the path selection mechanism, which chooses the farthest UAV
in each cell. This approach minimizes the number of hops while ensuring the presence of
sufficient UAVs in each selected cell.
To isolate the factors that may contribute to increased delay and effectively assess the
impact of parameters such as angle variation, speed variation, and cell size, the following
tests were conducted with a route request interval of 1.5 seconds and 2 seconds and without
the presence of broadcast packets.

USTHB/Computer Science : 2024 74

Protocol Implementation and Simulation Study

4.5.1.3 Re-routing ratio

(a) No broadcast. (b) 1s broadcast.

(c) 0.5s broadcast.

Figure 4.5: Re-routing ratio measurement for different load scenarios.

As depicted in Figures 4.5a, 4.5c, and 4.5b, the re-consulting ratio begins at 4.5% when the
number of UAVs is small and gradually increases with the number of UAVs, reaching up to
10%.
This trend indicates that as the number of UAVs in the network grows, the likelihood of
forwarding failures due to potential congestion increases. However, the re-routing ratio
remains below 10% due to the efficient route selection and retransmission mechanisms executed
before re-consulting the controller.

4.5.2 Hexagon cell size impact on performance

In this scenario, we maintain a fixed RReq interval of [1.5,2] seconds, a fixed learning period
of 0.5 seconds, and vary the hexagon cell size.
Changing the size of the cell will affect the number of cells in the network and the communication
range of the UAVs, as described by the formulas 3.3 and 3.1.
The variation in cell size will be within the limits of the simulation test space, ensuring that
the chosen values for hexagon size keep the communication viable within the given space of
the simulation (1000m*1000m).

USTHB/Computer Science : 2024 75

Protocol Implementation and Simulation Study

4.5.2.1 PDR

Figure 4.6: Pdr results with the cell size variation.

As depicted in Figure 4.6, the variation in cell size significantly affects PDR. The PDR
dropped from 95% to 89% when the hexagon size is 75 meters and to 70% when the hexagon
size is 50 meters. The underlying explanation for this phenomenon is that reducing the cell
size results in a greater number of partitions and cells in the given area, leading to multiple
empty cells. These empty cells, which surround different groups of crowded cells, can create
gaps between them, ultimately resulting in packet loss due to the absence of a viable path.
Therefore, to ensure efficient and consistent routing, it is crucial to carefully select the cell
size to match the number of UAVs and the space of the mission area.
This choice of the appropriate cell size mitigates the occurrence of empty cells and gaps,
thereby optimizing the protocol’s performance and maintaining a high PDR.

USTHB/Computer Science : 2024 76

Protocol Implementation and Simulation Study

4.5.2.2 Average Delay

Figure 4.7: Average Delays results with various cell hexagon sizes.

When the cell size is set to 75 or 50, the number of cells increases significantly, leading to
higher chances of reaching empty cells. This can result in frequent recalculation of paths and
increased delays, as shown in figure 4.7.
However, choosing a cell size similar to that used in the 100 cell hexagon size test can
mitigate the issue of empty cells, ensuring that each cell has a majority of its neighboring
cells containing UAVs.
In conclusion, empty cells in dynamic UAV networks may create difficulties with forwarding
packets, potentially resulting in more frequent route changes, which can lead to longer delays
when selecting a new path.

USTHB/Computer Science : 2024 77

Protocol Implementation and Simulation Study

4.5.2.3 Re-routing ratio

Figure 4.8: Re-routing with cell size variation.

Figure 4.8 demonstrates that as the number of cells increases, the re-routing ratio also
increases to an unacceptable level. This is attributed to the presence of numerous empty
cells between the cells that hold the UAVs, which limits the initial path selection options.
Furthermore, due to the mobility of UAVs, the path between the source and the destination
may necessitate being re-selected, facing limited options once again.
This result provides an explanation for the increase in the delay result mentioned previously
in the same scenario, as the limited path selection options and frequent re-routing lead to
increased latency in packet delivery.

4.5.3 Initial angle impact on performance

In this scenario, we maintain a fixed RReq interval of [0.7, 1] seconds and a fixed cell
size of 100 meters. We vary the initial angle from 50 to 360 degrees, where 360 degrees
represents no filtering at all. This scenario aims to demonstrate the importance of striking a
balance between choosing cells with high Q-values and selecting cells that lead towards the
destination while minimizing unnecessary turns.
By varying the initial angle, we can observe the impact of the angle filtering mechanism on
the protocol’s performance and its ability to guide the UAVs’ packets along efficient paths
towards their destinations.

USTHB/Computer Science : 2024 78

Protocol Implementation and Simulation Study

4.5.3.1 PDR

Figure 4.9: Pdr results with the initial angle variation.

As shown in Figure 4.9, the PDR shows a slight degradation when tested with 70 and 40
UAVs, as the network is not dense enough. However, with 100 UAVs, a significant degradation
in PDR is observed. This is because when using a large initial angle, the algorithm will choose
cells with high Q-values regardless of their direction relative to the destination; this may lead
to the selection of longer paths solely based on high Q-value, even if they move away from
the destination.
Choosing longer paths increases the likelihood of topology changes during forwarding, which
could result in packet loss. Therefore, the choice of initial angle should be related to the
desired level of exploration of alternative routes beyond the direct route. This point could
be a promising direction for future work, where deviated paths are considered for purposes
such as quality of service optimization.

USTHB/Computer Science : 2024 79

Protocol Implementation and Simulation Study

4.5.3.2 Average Delay

Figure 4.10: Average Delays results with intial angle variations.

The results in Figure 4.10 show that initial angles of 50, 100, and 150 degrees consistently
achieve stable delays between 40 and 60 milliseconds across all scenarios. In contrast,
angles between 200 and 360 degrees significantly increase average latency, reaching 80 to
120 milliseconds. This is because the protocol tends to select cells that are not directed
towards the intended destination. This results in packets being routed through distant UAVs
that are not closer to their target. It is important to note that the 360 degrees test case
represents a scenario where UAVs do not establish a direction, leading them to select from
any available cells the highest Q-value.
We conclude that the degradation of delays observed in larger angles is attributed to
the algorithm’s tendency to choose non-optimal paths, resulting in longer traversal times and
potentially unacceptable delays.

USTHB/Computer Science : 2024 80

Protocol Implementation and Simulation Study

4.5.3.3 Re-routing ratio

Figure 4.11: Re-routing varying initial angle.

Figure 4.11 presents the results for the re-routing ratio using different initial angles, ranging
from 50 to 360 degrees (equivalent to having no filter of direction at all).
The re-routing ratio starts at a low ratio under 10% when the initial angle does not exceed 200
degrees, as the algorithm initially selects cells in the direction of the destination, if possible.
However, the ratio increases when the angle exceeds 200 degrees because the algorithm may
choose cells that distance the source from the destination due to their high Q-values, rather
than selecting cells that approach the destination.

4.5.4 Period of learning impact on performance

In this scenario, we maintain a fixed RReq interval of [1.5, 2] seconds and a fixed cell size of
100 meters and use a normal distribution function for the speed of each UAV.
We vary the period at which the Q-learning algorithm updates its knowledge about the UAVs.
This analysis highlights the significance of providing the controller with fresh information
about the UAVs to maintain efficient and up-to-date routes.

USTHB/Computer Science : 2024 81

Protocol Implementation and Simulation Study

4.5.4.1 PDR

100

95

90
Pdr %

85 0.5s learning period

1s learning period
80
1.5s learning period
75

70
50 60 70 80 90 100
Number of UAVs

Figure 4.12: Pdr results with the learning period variation.

As shown in Figure 4.12, the choice of a learning period of 0.5 seconds produces improved
PDR results compared to the intervals of 1 second and 1.5 seconds.
This suggests that selecting a shorter learning period is generally beneficial. However, it
is essential to consider the network characteristics, such as the UAV’s ability to provide
information within the chosen interval. In addition, this parameter could be selected adaptively
to the dynamicity of the UAVs in the network. This adaptive approach could be a promising
direction for future research.

4.5.4.2 Re-routing ratio

25.0

20.0
Re-routing %

15.0

10.0

5.0

0.0
50 60 70 80 90 100
Number of UAVs

0.5s learning period 1s learning period 1.5s learning period

Figure 4.13: Re-routing result by varying the learning interval.

USTHB/Computer Science : 2024 82

Protocol Implementation and Simulation Study

Figure 4.13 illustrates the relationship between the learning period and re-routing ratio.
When the learning period is set to a low value, the re-routing ratio is low, indicating that
the algorithm is able to adapt quickly to changes in the network.
However, as the learning period increases, the re-routing ratio also increases due to the higher
chances of choosing outdated paths.
It is crucial to strike a balance between choosing a low learning period to avoid incorrect
paths and ensuring that the network can deliver UAV information to the controller within
that period.
One potential solution is to dynamically define this value based on the mobility of the UAVs,
allowing the algorithm to adapt to changing network conditions.

4.5.5 Evaluating the performance for different speeds scenarios

In this scenario, we maintain a fixed RReq interval of [1.5, 2] seconds and a fixed cell size of
100 meters. We test the protocol using different fixed speed values (10 Mps, 40 Mps, 70 Mps)
and also attribute random speed values to each UAV using a normal distribution function.
This variation allows us to assess the protocol’s stability and performance in terms of PDR
under high-dynamic scenarios, where UAVs have different and unpredictable speed profiles.

4.5.5.1 PDR

100
98
96
94
Pdr %

92
90
88
86
84
50 60 70 80 90 100
Number of UAVs

normal distribution 10 m/s 40 m/s 70 m/s

Figure 4.14: Pdr results under different speed scenarios.

As illustrated in Figure 4.14, the protocol demonstrates exceptional performance in low

dynamic environments, achieving a PDR up to 98%. At a speed of 40 Mps (144 Km/h), the

USTHB/Computer Science : 2024 83

Protocol Implementation and Simulation Study

protocol maintains a PDR of approximately 92%. Even in scenarios with high speeds of 70
Mps (252 Km/h), the protocol still exhibits a relatively acceptable PDR of 85%.
Notably, when using a normal distribution function to assign random speed values between
6 Mps and 55 Mps to each UAV, which closely mimics real-world scenarios, the protocol
achieves a PDR of up to 95%.
This remarkable performance is attributed to the controller’s rapid decision-making capabilities,
which are based on continuously updated Q-tables. When a request is received, the protocol
can swiftly extract the optimal path based on current information from the Q-table.
In addition, the recovery mechanism plays a crucial role in adapting to sudden changes in
UAV mobility, which can alter the network topology.
Furthermore, the mechanism of choosing the shortest possible path by adapting the angle
filtering idea significantly contributes to delivering packets as quickly as possible before
topology changes occur.

4.5.5.2 Average Delay

0.14

0.12
Average delay (s)

0.10
Normal distribution
0.08
10 m/s
0.06 40 m/s
70 m/s
0.04

0.02
50 60 70 80 90 100
Number of UAVs

Figure 4.15: Average Delays results with speed variations.

Figure 4.15 illustrates that UAVs experience lower delays, typically under 40 milliseconds,
when moving at a constant speed of 10m/s. In contrast, higher speeds (up to 40m/s and
70m/s) result in increased delays.
Another scenario where UAVs exhibit lower delays is when their speed distribution follows
realistic behavior, with UAVs changing their speed periodically. In this case, a stable delay
of 60 milliseconds is observed.
The primary cause of higher delays at high speeds is that the selected UAVs as next hops

USTHB/Computer Science : 2024 84

Protocol Implementation and Simulation Study

move extremely quickly, making it difficult for the UAV to forward packets to its next hop
since it is no longer within its communication range. As a result, UAVs that cannot forward
packets consult the controller for new paths, which takes time and contributes to the overall
delay.

4.5.5.3 Re-routing ratio

25.0

20.0
Re-routing %

15.0

10.0

5.0

0.0
50 60 70 80 90 100
Number of UAVs

Normal distribution 10m/s 40m/s 70m/s

Figure 4.16: Re-routing varying the speed value.

Figure 4.16 clearly demonstrates that a low speed significantly reduces the re-routing ratio.
In contrast, high speeds, such as 70 Mps (252 Km/h), lead to rapid topology changes,
rendering the validity of the path invalid quickly.
Although 25% re-routing ratio may not be a significant cost at high speeds, it is still important
to minimize the ratio as much as possible.
A potential future work idea is to implement a prediction mechanism based on the speed and
direction of the UAV in the path selection process. This could help reduce re-routing ratio
by anticipating and adapting to changes in the network topology.

4.5.6 Conclusion
As the simulation shows, the optimal performance of the routing protocol is contingent
upon the harmonious selection of environment parameters. Specifically, the number of cells
(or cell size) and the initial angle should be carefully chosen to be compatible with the number
of UAVs and the available space area. The learning period of the Q-learning algorithm is also
a crucial parameter that should be selected based on the mobility characteristics of the UAVs

USTHB/Computer Science : 2024 85

Protocol Implementation and Simulation Study

in the network. When this combination is properly selected, the routing process is optimized,
resulting in high packet delivery ratios, low average delays, and a minimal re-routing ratio.

USTHB/Computer Science : 2024 86

General conclusion and perspectives

Routing protocol in Flying ad hoc networks (FANETs) must address several unique
challenges and requirements to ensure efficient and reliable communication in these highly
dynamic environments.
Our study focused on the integration of software-defined networks to deal with the
limitations found in the existing routing protocols for FANETs.
Additionally, we explored the potential of the Q-learning algorithm, an artificial intelligence
technique, to optimize routing decisions in FANETs by integrating SDN’s centralized control
and programmability with Q-learning’s adaptive learning capabilities.
Specifically, we have developed a routing solution for SDFANET using the Q-learning
algorithm. The routing decision is centralized, as the controller learns and makes decisions,
while the UAVs forward packets based on the controller’s rules. To achieve scalability and
optimize the Q-learning table, we partitioned the space into adjacent hexagon cells and
applied the Q-learning algorithm between these cells. The angle filtering approach introduced
in the protocol has reduced the calculation cost in the decision-making process, optimizing
the path calculation and the routing performance. To fully make use of the SDN approach
and enhance performance, we employed a recovery mechanism to handle the dynamicity of
the network, which is the key feature of FANETs. In simulations, we tested the protocol
in various scenarios and measured its performance. The results showed an optimum Packet
Delivery Ratio (PDR) and lower average delay and re-routing ratio. This performance comes
from key features: the Q-learning algorithm adapts to changes, angle filtering reduces delays
and packet loss, hexagon partitioning aids scalability, and the recovery mechanism handles
dynamic networks.

As with any research endeavor, there are opportunities to enhance the proposed protocol
to address situations not explicitly covered in this document. Some potential areas for
improvement include:

87
General conclusion

• Handling scenarios where the controller becomes unavailable or experiences downtime.

• Addressing situations where no neighboring cells are accessible.

• Investigating the impact of alternative learning criteria on the protocol’s results. Experimenting
with different learning approaches or parameters may lead to enhanced outcomes,
potentially improving the protocol’s adaptability and decision-making capabilities.

• Adjusting dynamically the angle variation based on other factors such as congested
paths.

• Adjusting dynamically the cell size based on UAVs’ density.

By exploring these avenues for enhancement, future research can build upon the foundations
laid in this work, pushing the boundaries of FANET routing protocols and driving continuous
improvements in efficiency, reliability, and adaptability.

USTHB/Computer Science : 2024 88

Bibliography

[1] Nick McKeown, Thomas E. Anderson, H. Balakrishnan, Guru M. Parulkar, Larry L.

Peterson, Jennifer Rexford, Scott Shenker, and Jonathan S. Turner. Openflow: enabling
innovation in campus networks. Comput. Commun. Rev., 38:69–74, 2008.

[2] Antonio Guillen-Perez and Maria-Dolores Cano. Flying ad hoc networks: A new domain
for network communications. Sensors, 18:3571, 10 2018.

[3] Mehdi Hosseinzadeh, Saqib Ali, Liliana Ionescu-Feleaga, Bogdan-Stefan Ionescu,

Mohammad Sadegh Yousefpoor, Efat Yousefpoor, Omed Hassan Ahmed, Amir Masoud
Rahmani, and Asif Mehmood. A novel q-learning-based routing scheme using an
intelligent filtering algorithm for flying ad hoc networks (fanets). Journal of King
Saud University - Computer and Information Sciences, 35(10):101817, 2023.

[4] Muhammad Yeasir Arafat and Sangman Moh. A q-learning-based topology-aware

routing protocol for flying ad hoc networks. IEEE Internet of Things Journal,
9(3):1985–2000, 2022.

[5] Jianmin Liu, Qi Wang, ChenTao He, Katia Jaffrès-Runser, Yida Xu, Zhenyu Li, and
YongJun Xu. Qmr:q-learning based multi-objective optimization routing protocol for
flying ad hoc networks. Computer Communications, 150:304–316, 2020.

[6] S Sharmila and T Shanthi. A survey on wireless ad hoc network: Issues and
implementation. In 2016 International Conference on Emerging Trends in Engineering,
Technology and Science (ICETETS), pages 1–6, 2016.

[7] Vijaya M and Venkatachalam. Comparative analysis of qos routing protocols in manets:
Unicast multicast. International Journal of Emerging Technology and Advanced
Engineering, 2, 05 2012.

89
Bibliography

[8] Reeya Agrawal, Neetu Faujdar, Carlos Romero, Oshin Sharma, Ghaidaa Abdulsaheb,
Osamah Khalaf, Romany Mansoor, and Osama Ghoneim. Classification and
comparison of ad hoc networks: A review. Egyptian Informatics Journal, 24, 12 2022.

[9] Burhan Khan, Rashidah Olanrewaju, Farhat Anwar, Athaur Najeeb, and Mashkuri
Yaacob. A survey on manets: Architecture, evolution, applications, security issues
and solutions. Indonesian Journal of Electrical Engineering and Computer Science,
12:832–842, 11 2018.

[10] Ankur Bang. Wireless ad-hoc networks: Types, applications, security goals.
International Journal of Advent Research in Computer and Electronics, 03 2015.

[11] Ilker Bekmezci, Ozgur Koray Sahingoz, and Samil Temel. Flying ad-hoc networks
(fanets): A survey. Ad Hoc Networks, 11:1254–1270, 2013.

[12] Hassen Redwan, Sungchan Choi, Jong-Hong Park, and Jaeho Kim. Predictive
geographic multicast routing protocol in flying ad hoc networks. International Journal
of Distributed Sensor Networks, 15:155014771984387, 07 2019.

[13] Nancy Alshaer and El-Sayed El-Rabaie. A survey on ad hoc networks. 11 2016.

[14] Reeya Agrawal, Neetu Faujdar, Carlos Andres Tavera Romero, Oshin Sharma,
Ghadia Muttashar Abdulsahib, Osama Ibrahim Khalaf, Romany F Mansoor, and
Osama A. Ghoneim. Classification and comparison of ad hoc networks: A review.
Egyptian Informatics Journal, 24(1):1–25, 2023.

[15] J. Rejina Parvin. An overview of wireless mesh networks. Wireless Mesh

Networks-Security, Architectures and Protocols, 2019.

[16] Asside Djedouboum, Ado ARI, Abdelhak Gueroui, Alidou Mohamadou, and Zibouda
Aliouat. Big data collection in large-scale wireless sensor networks. Sensors, 18, 12
2018.

[17] J. Rejina Parvin. An overview of wireless mesh networks. Wireless Mesh

Networks-Security, Architectures and Protocols, 2019.

[18] Naeem Raza, Muhammad Umar Aftab, Muhammad Akbar, Omair Ashraf, and
Muhammad Irfan. Mobile ad-hoc networks applications and its challenges.
Communications and Network, 08:131–136, 01 2016.

USTHB/Computer Science : 2024 90

Bibliography

[19] Dinesh Ramphull, Avinash Mungur, Sheeba Armoogum, and Sameerchand Pudaruth.
A review of mobile ad hoc network (manet) protocols and their applications. 2021 5th
International Conference on Intelligent Computing and Control Systems (ICICCS),
pages 204–211, 2021.

[20] Muhammad Asghar Khan, Alamgir Safi, Ijaz Mansoor Qureshi, and Inam Ullah
Khan. Flying ad-hoc networks (fanets): A review of communication architectures,
and routing protocols. In 2017 First International Conference on Latest trends in
Electrical Engineering and Computing Technologies (INTELLECT), pages 1–9, 2017.

[21] Omar Sami Oubbati, Mohammed Atiquzzaman, Pascal Lorenz, Md. Hasan Tareque,
and Md. Shohrab Hossain. Routing in flying ad hoc networks: Survey, constraints, and
future challenge perspectives. volume 7, pages 81057–81105, 2019.

[22] Faezeh Pasandideh, Joao Paulo Javidi da Costa, Rafael Kunst, Nahina Islam, Wibowo
Hardjawana, and Edison Pignaton de Freitas. A review of flying ad hoc networks: Key
characteristics, applications, and wireless technologies. Remote Sensing, 14:4459, 09
2022.

[23] Lobna Krichen, Fourati Mohamed, and Lamia Fourati. Communication Architecture
for Unmanned Aerial Vehicle System: 17th International Conference on Ad Hoc
Networks and Wireless, ADHOC-NOW 2018, Saint-Malo, France, September 5-7,
2018. Proceedings, pages 213–225. 08 2018.

[24] Kamal Benzekki, Abdeslam El Fergougui, and Abdelbaki El Belrhiti El Alaoui.

Software-defined networking (sdn): A survey. Security and Communication Networks,
9, 02 2017.

[25] Evangelos Haleplidis, Kostas Pentikousis, Spyros G. Denazis, Jamal Hadi Salim, David
Meyer, and Odysseas G. Koufopavlou. Software-defined networking (sdn): Layers and
architecture terminology. RFC, 7426:1–35, 2015.

[26] Wenfeng Xia, Yonggang Wen, Chuan Heng Foh, Dusit Niyato, and Haiyong Xie. A
survey on software-defined networking. IEEE Communications Surveys and Tutorials,
17:27–51, 2015.

[27] Sanjeev Singh and Rakesh Kumar Jha. A survey on software defined networking:
Architecture for next generation network. Journal of Network and Systems
Management, 25:321–374, 4 2017.

USTHB/Computer Science : 2024 91

Bibliography

[28] Roaa Shubbar, Mohammad Alhisnawi, Aladdin Abdulhassan, and Mahmood Ahmadi.
A comprehensive survey on software-defined network controllers. pages 199–231, 01
2021.

[29] Diego Kreutz, Fernando M. V. Ramos, Paulo Esteves Veríssimo, Christian Esteve
Rothenberg, Siamak Azodolmolky, and Steve Uhlig. Software-defined networking: A
comprehensive survey. Proceedings of the IEEE, 103(1):14–76, 2015.

[30] Avri Doria, Jamal Hadi Salim, Robert Haas, Hormuzd M. Khosravi, Weiming Wang,
Ligang Dong, Ram Gopal, and Joel M. Halpern. Forwarding and control element
separation (forces) protocol specification. RFC, 5810:1–124, 2010.

[31] Giuseppe Bianchi, Marco Bonola, Antonio Capone, and Carmelo Cascone. Openstate:
Programming platform-independent stateful openflow applications inside the switch.
SIGCOMM Comput. Commun. Rev., 44(2):44–51, apr 2014.

[32] Fetia Bannour, Sami Souihi, and Abdelhamid Mellouk. Distributed sdn control: Survey,
taxonomy and challenges. IEEE Communications Surveys Tutorials, PP:1–1, 12 2017.

[33] Advait Dixit, Fang Hao, Sarit Mukherjee, T.V. Lakshman, and Ramana Kompella.
Towards an elastic distributed sdn controller. SIGCOMM Comput. Commun. Rev.,
43(4):7–12, aug 2013.

[34] Zohaib Latif, Kashif Sharif, Fan Li, Md. Monjurul Karim, and Yu Wang. A
comprehensive survey of interface protocols for software defined networks. ArXiv,
abs/1902.07913, 2019.

[35] Israat Tanzeena Haque and Nael Abu-Ghazaleh. Wireless software defined networking:
A survey and taxonomy. IEEE Communications Surveys Tutorials, 18(4):2713–2737,
2016.

[36] Yanbiao Li, Dafang Zhang, Javid Taheri, and Keqin Li. Sdn components and openflow.
Big Data Softw. Defin. Networks, 12:49–67, 2018.

[37] Lav Gupta, Raj Jain, and Gabor Vaszkun. Survey of important issues in uav
communication networks. IEEE Communications Surveys & Tutorials, 18:1123–1152,
2016.

USTHB/Computer Science : 2024 92

Bibliography

[38] Omar Sami Oubbati, Mohammed Atiquzzaman, Tariq Ahamed Ahanger, and Atef
Ibrahim. Softwarization of uav networks: A survey of applications and future trends.
IEEE Access, 8:98073–98125, 2020.

[39] Mohamed Khaleel and Abdullatif Jebrel. Artificial intelligence in computer science.
2:1–21, 03 2024.

[40] Thomas Rincy N and Roopam Gupta. A survey on machine learning approaches and its
techniques:. In 2020 IEEE International Students’ Conference on Electrical,Electronics
and Computer Science (SCEECS), pages 1–6, 2020.

[41] İbrahim Yazici, Ibraheem Shayea, and Jafri Din. A survey of applications of
artificial intelligence and machine learning in future mobile networks-enabled systems.
Engineering Science and Technology, an International Journal, 44:101455, 2023.

[42] Marc Ponsen, Matthew E Taylor, and Karl Tuyls. Abstraction and generalization in
reinforcement learning: A summary and framework. Adaptive and Learning Agents,
pages 1–32, 2010.

[43] Sifat Rezwan and Wooyeol Choi. A survey on applications of reinforcement learning in
flying ad-hoc networks. Electronics, 10(4), 2021.

[44] Beakcheol Jang, Myeonghwi Kim, Gaspard Harerimana, and Jong Kim. Q-learning
algorithms: A comprehensive classification and applications. IEEE Access, PP:1–1, 09
2019.

[45] K. Zhang, Zhuoran Yang, and Tamer Başar. Multi-agent reinforcement learning: A
selective overview of theories and algorithms. ArXiv, abs/1911.10635, 2019.

[46] Jan Lansky, Saqib Ali, Amir Rahmani, Mohammad Sadegh Yousefpoor, Efat
Yousefpoor, Faheem Khan, and Mehdi Hosseinzadeh. Reinforcement learning-based
routing protocols in flying ad hoc networks (fanet): A review. Mathematics, 10:3017,
08 2022.

[47] Leslie Pack Kaelbling, Michael L. Littman, and Andrew W. Moore. Reinforcement
learning: a survey. J. Artif. Int. Res., 4(1):237–285, may 1996.

[48] Hongbign Wang, Xin Chen, Qin Wu, Qi Yu, Xingguo Hu, Zibin Zheng, and Athman
Bouguettaya. Integrating reinforcement learning with multi-agent techniques for

USTHB/Computer Science : 2024 93

Bibliography

adaptive service composition. ACM Transactions on Autonomous and Adaptive

Systems, 12:1–42, 05 2017.

[49] Fadi AlMahamid and Katarina Grolinger. Reinforcement learning algorithms: An

overview and classification. 2021 IEEE Canadian Conference on Electrical and
Computer Engineering (CCECE), pages 1–7, 2021.

[50] Muhammad Morshed Alam and Sangman Moh. Survey on q-learning-based

position-aware routing protocols in flying ad hoc networks. Electronics, 11(7), 2022.

[51] Christopher Watkins and Peter Dayan. Technical note: Q-learning. Machine Learning,
8:279–292, 05 1992.

[52] Dali Wei and H. Anthony Chan. Clustering ad hoc networks: Schemes and
classifications. In 2006 3rd Annual IEEE Communications Society on Sensor and Ad
Hoc Communications and Networks, volume 3, pages 920–926, 2006.

[53] Deep Medhi and Karthik Ramasamy. Chapter 3 - routing protocols: Framework
and principles. In Deep Medhi and Karthik Ramasamy, editors, Network Routing
(Second Edition), The Morgan Kaufmann Series in Networking, pages 64–113. Morgan
Kaufmann, Boston, second edition edition, 2018.

[54] Chen-Mou Cheng, Pai-Hsiang Hsiao, H. T. Kung, and Dario Vlah. Maximizing
throughput of uav-relaying networks with the load-carry-and-deliver paradigm. In 2007
IEEE Wireless Communications and Networking Conference, pages 4417–4424, 2007.

[55] P. Jacquet, P. Muhlethaler, T. Clausen, A. Laouiti, A. Qayyum, and L. Viennot.

Optimized link state routing protocol for ad hoc networks. In Proceedings. IEEE
International Multi Topic Conference, 2001. IEEE INMIC 2001. Technology for the
21st Century., pages 62–68, 2001.

[56] Charles E. Perkins and Pravin Bhagwat. Highly dynamic destination-sequenced

distance-vector routing (dsdv) for mobile computers. In Proceedings of the Conference
on Communications Architectures, Protocols and Applications, SIGCOMM ’94, page
234–244, New York, NY, USA, 1994. Association for Computing Machinery.

[57] David B. Johnson, Yih-Chun Hu, and David A. Maltz. The dynamic source routing
protocol (dsr) for mobile ad hoc networks for ipv4. RFC, 4728:1–107, 2007.

USTHB/Computer Science : 2024 94

Bibliography

[58] C.E. Perkins and E.M. Royer. Ad-hoc on-demand distance vector routing. In
Proceedings WMCSA’99. Second IEEE Workshop on Mobile Computing Systems and
Applications, pages 90–100, 1999.

[59] Haas Zygmunt and Marc Pearlman. Zrp: a hybrid framework for routing in ad hoc
networks. Ad Hoc Networking, pages 221–253, 03 2001.

[60] Michael Iordanakis, Dimitrios Yannis, Kimon Karras, Georgios Bogdos, Georgios
Dilintas, Massimiliano Amirfeiz, Giorgio Colangelo, and Stefano Baiotti. Ad-hoc
routing protocol for aeronautical mobile ad-hoc networks. Fifth international
symposium on communication systems, networks and digital signal processing
(CSNDSP), 01 2006.

[61] Rostam Shirani. Reactive-greedy-reactive in unmanned aeronautical ad-hoc networks:

A combinational routing mechanism. 2011.

[62] Brad Karp and H. T. Kung. Gpsr: greedy perimeter stateless routing for wireless
networks. In Proceedings of the 6th Annual International Conference on Mobile
Computing and Networking, MobiCom ’00, page 243–254. Association for Computing
Machinery, 2000.

[63] Abdul Jabbar and James Sterbenz. Aerorp: A geolocation assisted aeronautical routing
protocol for highly dynamic telemetry environments. Proceedings of the International
Telemetering Conference, 45, 01 2009.

[64] Stefano Rosati, Karol Kruzelecki, Louis Traynard, and Bixio Rimoldi. Speed-aware
routing for uav ad-hoc networks. 2013 IEEE Globecom Workshops (GC Wkshps), pages
1367–1373, 2013.

[65] Nan Shi and Xiling Luo. A novel cluster-based location-aided routing protocol for
uav fleet networks. International Journal of Digital Content Technology and Its
Applications, 6:376–383, 2012.

[66] Jean-Aimé Maxa, Mohamed Slim Ben Mahmoud, and Nicolas Larrieu. Secure routing
protocol design for uav ad hoc networks. In 2015 IEEE/AIAA 34th Digital Avionics
Systems Conference (DASC), pages 4A5–1–4A5–15, 2015.

[67] Jean-Aimé Maxa, Mohamed Slim Ben Mahmoud, and Nicolas Larrieu. Joint
model-driven design and real experiment-based validation for a secure uav ad hoc

USTHB/Computer Science : 2024 95

Bibliography

network routing protocol. In 2016 Integrated Communications Navigation and

Surveillance (ICNS), pages 1E2–1–1E2–16, 2016.

[68] Chunhua Zang and Shouhong Zang. Mobility prediction clustering algorithm for uav
networking. In 2011 IEEE GLOBECOM Workshops (GC Wkshps), pages 1158–1161,
2011.

[69] D.L. Gu, Guangyu Pei, H. Ly, M. Gerla, and Xiaoyan Hong. Hierarchical routing
for multi-layer ad-hoc wireless networks with uavs. In MILCOM 2000 Proceedings.
21st Century Military Communications. Architectures and Technologies for Information
Superiority (Cat. No.00CH37155), volume 1, pages 310–314 vol.1, 2000.

[70] Yu-Ting Yu, Le Ru, Wensheng Chi, Yaqing Liu, Qiangqiang Yu, and Kun Fang.
Ant colony optimization based polymorphism-aware routing algorithm for ad hoc uav
network. Multimedia Tools and Applications, 75:14451 – 14476, 2016.

[71] Alexey V. Leonov. Application of bee colony algorithm for fanet routing. 2016 17th
International Conference of Young Specialists on Micro/Nanotechnologies and Electron
Devices (EDM), pages 124–132, 2016.

[72] Omar Sami Oubbati, Abderrahmane Lakas, Nasreddine Lagraa, and Mohamed Bachir
Yagoubi. Cruv: Connectivity-based traffic density aware routing using uavs for vanets.
In 2015 International Conference on Connected Vehicles and Expo (ICCVE), pages
68–73, 2015.

[73] Denis do Rosário, Zhongliang Zhao, Torsten Braun, Eduardo Cerqueira, Aldri Luiz dos
Santos, and Islam Alyafawi. Opportunistic routing for multi-flow video dissemination
over flying ad-hoc networks. Proceeding of IEEE International Symposium on a World
of Wireless, Mobile and Multimedia Networks 2014, pages 1–6, 2014.

[74] Arnau Rovira-Sugranes, Abolfazl Razi, Fatemeh Afghah, and Jacob Chakareski. A
review of ai-enabled routing protocols for uav networks: Trends, challenges, and future
outlook. Ad Hoc Networks, 130:102790, 2022.

[75] Woo-Sung Jung, Jinhyuk Yim, and Young-Bae Ko. Qgeo: Q-learning-based geographic
ad hoc routing protocol for unmanned robotic networks. IEEE Communications
Letters, 21(10):2258–2261, 2017.

USTHB/Computer Science : 2024 96

Bibliography

[76] Benjamin Sliwa, Cedrik Schüler, Manuel Patchou, and Christian Wietfeld. Parrot:
Predictive ad-hoc routing fueled by reinforcement learning and trajectory knowledge.
2021 IEEE 93rd Vehicular Technology Conference (VTC2021-Spring), pages 1–7, 2020.

[77] Jianmin Liu, Qi Wang, ChenTao He, Katia Jaffrès-Runser, Yida Xu, Zhenyu Li, and
Yong-Jun Xu. Qmr:q-learning based multi-objective optimization routing protocol for
flying ad hoc networks. Computer Communications, 150, 11 2019.

[78] Alaa Albu-Salih and Hayder Khudhair. Asr-fanet: An adaptive sdn-based routing
framework for fanet. International Journal of Electrical and Computer Engineering
(IJECE), 11:4403, 10 2021.

[79] Anay Ajit Deshpande, Federico Chiariotti, and Andrea Zanella. Smurf: Reliable
multipath routing in flying ad-hoc networks. In 2020 Mediterranean Communication
and Computer Networking Conference (MedComNet), pages 1–8, 2020.

[80] Qihui Wu. Routing protocol for heterogeneous fanets with mobility prediction. China
Communications, 19(1):186–201, 2022.

[81] Berat Erdemkılıç and Mehmet Akif Yazıcı. A software defined networking-based routing
algorithm for flying ad hoc networks. Sakarya University Journal of Computer and
Information Sciences, 5(2):246–256, 2022.

[82] Huilong Jin, Xiaozi Jin, Yucong Zhou, Pingkang Guo, Jie Ren, Jian Yao, and Shuang
Zhang. A survey of energy efficient methods for uav communication. Vehicular
Communications, 41:100594, 2023.

[83] Pedram Beigi, Mohammad Sadra Rajabi, and Sina Aghakhani. An overview of drone
energy consumption factors and models, 2023.

[84] Understanding q-learning and its connection to human learning. https://www.

linkedin.com/pulse/understanding-q-learning-its-connection-human-learning-elliott
Accessed: 2024-06-02.

[85] Chen-Pin Yang, Chin-En Yen, and Ing-Chau Chang. A software-defined directional
q-learning grid-based routing platform and its two-hop trajectory-based routing
algorithm for vehicular ad hoc networks. Sensors, 22(21), 2022.

USTHB/Computer Science : 2024 97

Bibliography

[86] Ruiling Li, Fan Li, Xin Li, and Yu Wang. Qgrid: Q-learning based routing protocol for
vehicular ad hoc networks. In 2014 IEEE 33rd International Performance Computing
and Communications Conference (IPCCC), pages 1–8, 2014.

[87] András Varga and Rudolf Hornig. An overview of the omnet++ simulation
environment. In International ICST Conference on Simulation Tools and Techniques,
2008.

[88] Levente Mészáros, Andras Varga, and Michael Kirsche. Inet framework. Recent
Advances in Network Simulation: TheOMNeT++ Environment and its Ecosystem,
pages 55–106, 05 2019.

[89] Normal distribution: What it is, uses, and formula. https://www.scribbr.com/

statistics/normal-distribution/. Accessed: 2024-06-07.

[90] Omer T. Abdulhae, Jit Singh Mandeep, and Mt Islam. Cluster-based routing protocols
for flying ad hoc networks (fanets). IEEE Access, 10:32981–33004, 2022.

[91] Ahmed M. Khedr, Ahmed Salim, Pravija Raj P V, and Walid Osamy. Mwcrsf:
Mobility-based weighted cluster routing scheme for fanets. Veh. Commun., 41(C),
jun 2023.

[92] Joel M. Halpern, Robert HAAS, avri doria, Ligang Dong, Weiming Wang, Hormuzd M.
Khosravi, Jamal Hadi Salim, and Ram Gopal. Forwarding and Control Element
Separation (ForCES) Protocol Specification. RFC 5810, March 2010.

[93] Wenjuan Li, Weizhi Meng, and Lam For Kwok. A survey on openflow-based software
defined networks: Security challenges and countermeasures. Journal of Network and
Computer Applications, 68:126–139, 2016.

[94] Arun G, A.V. Kumar, Priyanka Sharma, Indrarini Irawati, Chandrashekar V.,
I. Musirin, Hesham Mohammed Ali Abdullah, and Manjunatha L. Artificial intelligence
in computer science: An overview of current trends and future directions. pages 43–60,
09 2023.

[95] Muhammad Morshed Alam and Sangman Moh. Survey on q-learning-based

position-aware routing protocols in flying ad hoc networks. Electronics, 11(7), 2022.

USTHB/Computer Science : 2024 98

Bibliography

[96] Krishan Yadav, Amit Nain, and Sunil Maakar. Routing protocols in fanet:
Survey. National Conference on Innovative Trends in Computer Science Engineering
(ITCSE-2015), 04 2015.

[97] Hope Forsmann, Robert E. Hiromoto, and Jonathan Svoboda. A time-slotted

on-demand routing protocol for mobile ad hoc unmanned vehicle systems. In SPIE
Defense + Commercial Sensing, 2007.

[98] Charles E. Perkins Elizabeth M. Royer. Multicast ad hoc on-demand distance vector
(maodv) routing. Routing draft-ietf-manet-maodv-00.txt, 08 2000.

[99] Muhammad Fahad Khan, Kok-Lim Alvin Yau, Rafidah Md Noor, and Muhammad Ali
Imran. Routing schemes in fanets: A survey. Sensors, 20(1), 2020.

[100] Berat Erdemkiliç and Mehmet Akif Yazici. Sdn based routing protocol for fanets.
In 2022 30th Signal Processing and Communications Applications Conference (SIU),
pages 1–4, 2022.

[101] Panayiotis Kolios, Konstandinos Koumidis, Christos Panayiotou, and Georgios

Ellinas. Beaconing-based networking for localized information exchange in emergency
management. Ad Hoc Networks, 107:102225, 2020.

USTHB/Computer science: 2024 99

 Abstract
In today’s fast-evolving world of wireless networking, creating efficient
and reliable communication systems is essential. This study dives into Flying
Ad-Hoc Networks (FANETs), which are a type of Mobile Ad-Hoc Networks
(MANETs). These networks use Unmanned Aerial Vehicles (UAVs) that
communicate without any fixed infrastructure. We introduce a new routing
protocol designed specifically for FANETs that combines Software-Defined
Networking (SDN) and reinforcement learning, particularly Q-learning, to
enhance path selection and network performance. Our protocol tackles the
unique challenges of FANETs, such as finding efficient paths, managing UAVs
resources, and dealing with quickly changing topologies. Through detailed
simulations, we show that our protocol improves packet delivery rates, cuts
down transmission delays, and boosts overall network reliability. This re-
search helps push forward adaptive and smart networking solutions for aerial
communication systems.

Key words: FANET, Routing protocol, Q-learning, Ad-hoc, SDN.

 Résumé
Dans le monde en rapide évolution des réseaux sans fil, la conception de
systèmes de communication efficaces et fiables est essentielle. Cette étude se
concentre sur les réseaux ad hoc aérien (FANETs), un type de réseaux ad hoc
mobiles (MANET) utilisant des véhicules aériens sans pilote (UAV) qui com-
muniquent sans infrastructure fixe. Nous introduisons un nouveau protocole
de routage spécialement conçu pour les FANETs, combinant les principes
des réseaux définis par logiciel (SDN) et l’apprentissage par renforcement, en
particulier l’algorithme Q-learning, pour optimiser la sélection des chemins
et la performance du réseau. Notre protocole relève les défis uniques des FA-
NETs, tels que la détermination des chemins efficaces, gestion des ressources
des drones et les changements rapides de topologie. Grâce à des simulations
détaillées, nous montrons que notre protocole améliore les taux de livraison
des paquets, réduit les délais de transmission et augmente la fiabilité globale
du réseau. Cette recherche contribue à l’avancement de solutions de réseau
adaptatives et intelligentes dans le contexte des systèmes de communication
aérienne.
Mots clés :FANET, Protocole de routage, Q-learning, Ad-hoc, SDN.