This is an electronic reprint of the original article.

This reprint may differ from the original in pagination and typographic detail.

Toro Betancur, Veronica; Premsankar, Gopika; Liu, Chen-Feng; Slabicki, Mariusz; Bennis,
Mehdi; Di Francesco, Mario
Learning How to Configure LoRa Networks with No Regret: a Distributed Approach

Published in:
IEEE Transactions on Industrial Informatics

DOI:
10.1109/TII.2022.3187721

Published: 01/04/2023

Document Version
Peer-reviewed accepted author manuscript, also known as Final accepted manuscript or Post-print

Please cite the original version:

Toro Betancur, V., Premsankar, G., Liu, C.-F., Slabicki, M., Bennis, M., & Di Francesco, M. (2023). Learning
How to Configure LoRa Networks with No Regret: a Distributed Approach. IEEE Transactions on Industrial
Informatics, 19(4), 5633-5644. https://doi.org/10.1109/TII.2022.3187721

This material is protected by copyright and other intellectual property rights, and duplication or sale of all or
part of any of the repository collections is not permitted, except that material may be duplicated by you for
your research use or educational purposes in electronic or print form. You must obtain permission for any
other use. Electronic or print copies may not be offered, whether for sale or otherwise to anyone who is not
an authorised user.

Powered by TCPDF (www.tcpdf.org)

 1

Learning How to Configure LoRa Networks

with No Regret: a Distributed Approach
Verónica Toro-Betancur, Gopika Premsankar, Chen-Feng Liu,
Mariusz Słabicki, Mehdi Bennis, Mario Di Francesco

metering in particular. These applications demand low traffic

Abstract—LoRa is one of the most popular technologies for from each node, however, LoRa networks usually include
low-power wide area networks. It offers long-range communica- thousands of devices which makes reliable communication
tion with a low energy consumption, which makes it ideal for
many applications in the Internet of things. The performance very challenging [4]. Moreover, LoRa operates in the sub-GHz
of LoRa networks depends on the communication parameters Industrial, Scientific and Medical (ISM) bands; consequently,
used by individual nodes. Several works have proposed different the bandwidth is limited and sharing it becomes more difficult
solutions, typically running on a central network server, to as the number of devices increases.
select these parameters. However, existing approaches have not The performance of LoRa networks can be improved by
addressed the need to (re-)assign parameters when channel
conditions suddenly vary due to additional traffic, changes in optimizing the communication parameters of individual nodes
the weather or the presence of obstacles. Moreover, allocation – in particular, the spreading factor (SF) and transmission
strategies that require a central entity to decide communication power (TP). The SF trades-off communication range and
parameters do not scale due to the large number of configuration data rate [4]; a high SF results in a long communication
packets that must be sent to the nodes. To address these issues, range at a low data rate. The TP not only determines the
this work proposes NoReL, a distributed game-theoretic approach
that allows nodes to autonomously update their parameters coverage range of the node, but also affects the probability
and maximize their packet delivery ratio. NoReL is based on a of collision between two nodes [5], as a higher TP achieves
stochastic variant of no-regret learning, which is proven to reach a longer communication range. Furthermore, LoRa networks
an -coarse correlated equilibrium in LoRa networks. Extensive are subject to the capture effect [6]: depending on its receive
simulations show that NoReL achieves a higher delivery ratio than power, a packet can be successfully decoded by the gateway
the state of the art in both static and dynamic environments, with
an improvement up to 12%. even in presence of collisions. Therefore, a higher TP implies
a higher probability of success for packets that overlap in time
Index Terms—LoRa, no-regret learning, game theory, Internet
at the receiver, at the cost of higher energy consumption.
of Things, low power wide area networks
Different works in the literature have addressed the optimal
assignment of SFs and TPs to LoRa nodes with the goal
I. I NTRODUCTION of improving the overall network performance [7, 8]. These
Low-Power Wide Area Networks (LPWANs) offer long- approaches run at the network server and require the final
range wireless communication with low energy consumption parameters to be sent by the gateway to individual nodes as
at low data rates [1]. This makes them ideal in the In- downlink packets. As such, they target scenarios where the
ternet of Things (IoT), especially for scenarios where sen- nodes are configured once with the appropriate SF and TP, i.e.,
sors infrequently send small packets. Long Range (LoRa) is it is assumed that the network conditions will remain constant,
one of the most popular LPWAN technologies. It achieves therefore, the parameters do not need to be altered. However,
communication ranges of 3-5 km in urban areas and 10- recent empirical studies have shown that channel conditions
15 km in rural environments [2]. LoRa offers a scalable in LoRa vary over time due to the weather, the presence
network architecture [3] where nodes simply send packets to of obstacles, and in-band traffic [9, 10]. Such changes can
all gateways in range, which in turn forward them to a central severely impact the reliability of LoRa networks. To maintain
network server that filters out duplicates. For these reasons, an acceptable network performance under varying channel
LoRa has been used in several application scenarios, smart conditions, centralized optimization problems [7, 8] need to be
re-run periodically, resulting in an increased downlink traffic
This work was partially supported by: the Academy of Finland under grants
number 319710, 319758, 326346, and 338854; the Polish National Center for to communicate parameters, which further reduces network
Research and Development under grant POIR.04.01.04-00-1414/20. performance [11]. A few adaptive and distributed approaches
V. Toro-Betancur and M. Di Francesco are with the Depart- have also been proposed [12–14], wherein nodes can choose
ment of Computer Science, Aalto University in Espoo, Finland. E-
mail: {veronica.torobetancur, mario.di.francesco}@aalto.fi their SFs and TPs with some feedback from the gateway or
G. Premsankar is with the Department of Computer Science, University of the network server. However, some of these solutions require
Helsinki in Helsinki, Finland. E-mail: [email protected] all uplink packets to be acknowledged, which increases the
C.-F. Liu is with the Digital Science Research Center, Technology Inno-
vation Institute, 9639 Masdar City, Abu Dhabi, United Arab Emirates. E- overall traffic in the network, while others target scenarios
mail: [email protected] where network conditions do not frequently vary [14].
M. Słabicki is with the Institute of Theoretical and Applied Informatics, Pol- To address these issues, this work presents NoReL, a fully
ish Academy of Sciences in Gliwice, Poland. E-mail: [email protected]
M. Bennis is with the Centre of Wireless Communications, University of distributed approach based on no-regret learning [15] to as-
Oulu, 90014 Oulu, Finland. E-mail: [email protected] sign SFs and TPs in LoRa networks. With NoReL, nodes
©2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including
reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or
reuse of any copyrighted component of this work in other works. DOI: 10.1109/TII.2022.3187721
 2

autonomously select appropriate SFs and TPs with minimal as a leader and follower game [20]. Specifically, the authors
feedback from the gateway, thereby allowing nodes to adapt propose to allocate SFs to the nodes for a specific time
to dynamic changes in the environment. To this end, the duration. Their approach requires frequent downlink transmis-
problem is modeled as a stochastic game with incomplete sions, as the parameters need to be updated every time the
information where nodes act selfishly and aim at maximizing allocation time expires, which can occur every 20 s. Moreover,
their own packet delivery ratio [16, 17]. Here, nodes choose the solutions above assume perfect orthogonality between SFs
the action that minimizes the regret of not having played it and have not been evaluated in dynamic environments.
previously. By following such a procedure, nodes reach an - Adaptive assignment. Adaptive parameter assignment has
Coarse Correlated Equilibrium (-CCE) [18] – a generalization also been considered in the literature. The LoRaWAN speci-
of the well-known Nash equilibrium – in which no node can fications include a built-in Adaptive Data Rate (ADR) mech-
improve its payoff – namely, its delivery ratio – by unilaterally anism to assign SFs and TPs on a link basis and improve
changing its own communication parameters. scalability [21]. Słabicki et al. [22] have studied the perfor-
The main contributions of this work are the following.
mance of ADR; they have also proposed alternative approaches
First, it formulates a distributed game-theoretic approach that
to configure nodes and increase the delivery ratio. Moreover,
incurs limited communication overhead. Second, it proves
DyLoRa [12] has been designed as an adaptive mechanism that
that no-regret learning leads to an -CCE in LoRa networks.
assigns communication parameters to maximize the energy
Finally, extensive simulations show that NoReL achieves a
efficiency of the network. All these solutions allow both the
higher delivery ratio than the state of the art in both static
nodes and the network server to run independent algorithms
and dynamic environments, with an improvement up to 12%.
that converge to a stable configuration; in both cases, nodes
The rest of the article is organized as follows. Section II
can only increase their SF and TP when uplink connectivity is
reviews the related work. Section III introduces the system
lost. Instead, NoReL offers nodes more flexibility in choosing
model, then presents NoReL and proves its convergence. Sec-
their SFs and TPs, which allows them to quickly adapt to
tion IV evaluates NoReL and compares its performance against
highly varying channel conditions.
the state of the art. Last, Section V provides concluding
remarks as well as directions for future work. Machine learning-based. Machine learning techniques, such
as deep reinforcement learning [23] and multi-armed ban-
II. R ELATED WORK dits [13, 24], have also been employed in the literature to
This section classifies and discusses the state of the art on allocate SF and TP in LoRa networks. These approaches
configuration of LoRa networks. require downlink transmissions after each packet to either
acknowledge a received packet or notify nodes about updated
Centralized assignment. Many works have proposed central- communication parameters. As a consequence, they lower the
ized solutions to configure communication parameters in LoRa overall network performance since the probability of collisions
networks. One option is assigning a node the minimum SF increases with higher downlink traffic [11]. In contrast, NoReL
that allows it to reach at least one gateway; despite being very requires only sporadic feedback from the gateways.
simple, such a technique can already achieve a high delivery
ratio [19]. More sophisticated approaches have also been No-regret learning. No-regret learning has been widely stud-
proposed to further improve performance through optimization ied [15, 25] and applied to wireless networks [26]. However,
techniques leveraging linear programming [7] and sequential to the best of the authors’ knowledge, such a technique has
waterfilling [8]. Although these approaches obtain stable so- not been applied to LoRa networks. For instance, no-regret
lutions for SF and TP allocation, most do not define how to learning has been shown to converge to an -CCE in cellular
inform nodes about the parameters to be used. Moreover, these networks [18]. However, LoRa is inherently different from
solutions provide a static allocation of parameters, thus, they the cellular networks studied so far, as it uses an unslotted
are unsuitable for dynamic environments. In fact, they would ALOHA channel access model – transmissions are not aligned,
have to run periodically and send a large number of downlink which results in asynchronous operations.
packets with new configuration parameters. Instead, NoReL is
distributed; nodes can update their parameters at any time with III. D ISTRIBUTED A LLOCATION OF SF S AND TP S
only limited feedback from the gateway. This section describes the process of allocating transmission
parameters in LoRa with NoReL. It first introduces the system
Distributed assignment. Distributed approaches have primar-
model and formulates the problem of optimizing the delivery
ily leveraged game theory [14, 20] for nodes to autonomously
ratio in distributed settings. It then presents a non-cooperative
choose parameters, with appropriate feedback from the gate-
stochastic game and the proposed no-regret learning strategy.
ways or the network server. For instance, Tolio et al. [14]
present a two-player game to allocate SFs based on complete
information. However, that work requires each node to have A. System model
complete information about the SFs used by other nodes and The considered LoRa network comprises multiple gateways
assumes that all nodes transmit with the highest TP. Instead, and nodes denoted by M and N , respectively. In particular,
NoReL is modeled as an incomplete information game, wherein the reference architecture is the one in the LoRaWAN spec-
nodes do not require information about the actions taken by ifications [27]: nodes communicate to all gateways in range
other nodes. The allocation problem has also been modeled over the LoRa physical layer; the gateways forward received
 3

packets to a central network server over the backhaul network where Mn is the set of gateways that node n can reach.
(e.g., through wired or cellular links), where received packets Finally, the outage probability Pm
o is given by:
are filtered and finally sent to the relevant application server.   
d

Nodes are stationary and in the range of one or more gate- 1 Pr,s − pn + P L(d0 ) + 10γ log n,md 0
Pmo =
erfc  √  .
ways, without any restrictions on their actual location. Each 2 σn 2
node (n ∈ N ) uses a spreading factor (SF) sn and transmission (3)
power (TP) pn from the sets Sn and Pn , respectively. Sn and This probability expresses the chance that the receive power
Pn depend on the geographical region where the LoRa nodes of packets from node n at gateway m is below the sensitivity
operate [4], and on the node’s distance to the closest gateway. Pr,s of the receiver for SF s. Here, pn , P L(d0 ), γ, σn ,
Nodes are Class A devices and employ the ALOHA-based and dn,m are, respectively, the TP of node n, path loss at
medium access control protocol in LoRaWAN [27]. After distance d0 , path loss exponent, standard deviation of the
sending a packet, a node also remains active for two short time normal distribution for channel variations, and the distance
windows to receive downlink packets. Such a communication between node n and gateway m (see [5] for additional details).
protocol has a low implementation complexity and supports
energy-efficient operations. B. Problem description
The goal of this article is to maximize the delivery ratio
A distributed approach is introduced next for choosing SFs
of each node by choosing appropriate SFs and TPs. The
and TPs depending on external conditions. Specifically, every
delivery ratio of each node is described by using the model
node aims to individually maximize its own delivery ratio by
in [5], which is accurate enough to include the most im-
choosing an appropriate SF and TP pair based on observed
portant factors affecting real-world networks – including the
channel and traffic conditions. In this regard, a high standard
capture effect, channel variations, and duty-cycle restrictions.
deviation of the node’s channel (σn ) implies a higher outage
The capture effect states that one out of multiple packets
probability, according to Eq. (3). However, a higher SF lowers
simultaneously received at a destination can be successfully
the gateway sensitivity, resulting in a decreased probability
decoded – specifically, the packet whose receive power is
of outage (Pm o ). A node also has an incentive to use a high
higher than the others by the SF-dependent thresholds given
TP such that the probability of interference with other nodes,
in [6, Table II]. These thresholds show that different SFs are
i.e, P(Cjm ), is minimized [5, Eq. (16)]. Moreover, a higher
not fully orthogonal, namely, that packets transmitted with
average sending rate (λn ) entails a higher P(Ej0 ) in Eq. (2)
different SFs have a small probability of interfering with each
which, in turn, increases the probability of interference Pc .
other; for this reason, SFs are called quasi-orthogonal. The
Therefore, the delivery ratio Dn is maximized at all channel
packet generation of nodes follows a Poisson distribution as it
and traffic conditions by using the highest SF and TP available
represents the discrete and random nature of transmissions in
to the node – for instance, SF12 and TP14 in the European
LPWANs [3, 5]. Radio propagation follows the log-distance
region [4]. By this reasoning, all nodes will choose the same
path loss model. Accordingly, the delivery ratio of node n is
configuration. However, the increased time-on-air due to the
given by [5]:
higher SF [4, 8] increases the probability of interference with
Y
! other transmissions [5]. In other words, a node’s delivery ratio
m is decreased if other nodes in the network simultaneously use
Dn = (1 − Pc ) 1 − Po , (1)
m∈M high SFs and TPs [8]. Since the nodes operate individually,
each node is unaware of the SFs and TPs chosen by other
where Pc is the probability of interference between the packets nodes. Taking into account this lack of knowledge, the problem
sent by node n and the other nodes in the network, and Pm o to maximize the delivery ratio is formulated as follows.
is the outage probability of node n, i.e., the probability that
node n is unable to reach the gateway m. Following [5], Pc Problem 1 (Distributed optimization of the delivery ratio).
is defined as: The distributed maximization of delivery ratio is given by:
X X
|C|
    max Dn Pr(sn , pn |σn , λn ) Pr(σn , λn ),
Pr(sn ,pn |σn ,λn )
Y Y Y sn ,pn σn ,λn
1 − P(Ej0 )P(Cjm ) .


Pc = 1 −  1 − X X
i=1 k ∈C
Rn i
k
j∈Rn subject to Pr(sn , pn |σn , λn ) = 1,
(2) sn ∈Sn pn ∈Pn
Here, P(Ej0 ) is the probability of an interfering node j ∈ N 0 ≤ Pr(sn , pn |σn , λn ) ≤ 1, ∀sn ∈ Sn , pn ∈ Pn ,
transmitting simultaneously with node n, and P(Cjm ) is the
for each node n ∈ N . Here, Pr(sn , pn |σn , λn ) is the proba-
probability that packets from node j collide with those from
bility that node n chooses SF sn and TP pn given a standard
node n. To estimate the probability of node n’s transmission
deviation σn of the channel and an average sending rate λn .
not being received by any gateway in range, the set of
nodes (N ) is partitioned into sets Rnk , called regions, with Problem 1 incurs a configuration race among the nodes in
respect to each node n and set of gateways k. Each region Rnk the network, as previously mentioned. To tackle the compe-
contains the interferers of n that can reach the gateways in k. tition between nodes and the lack of information about other
C is the set of set covers of all regions, that is, if Ci ∈ C and nodes, a non-cooperative game [28] is leveraged to assign SFs
k
Ci = {Rnk1 , Rnk2 , . . . , Rnj }, then k1 ∪ k2 ∪ · · · ∪ kj = Mn , and TPs in a distributed manner, as described next.
 4

C. Non-cooperative stochastic game packets to a large number of nodes in the network whenever a
The competition among nodes to selfishly maximize their configuration change is required, e.g., every time the channel
individual delivery ratio under diverse external conditions conditions get worse. This would significantly reduce the
can be modeled as a non-cooperative stochastic game played capacity of the LoRa network [11]. Thus, a regret-based
among the set N of LoRa nodes, with |N
| = N . The method [18] is introduced for each node to learn an SF and
game is defined as G ≡ N , A, Ω, {un }n∈N , where A = TP allocation strategy that achieves an -CCE for each ωnj in
A1 × A2 × · · · × AN is the network-wide action space and a distributed fashion.
An = Sn ×Pn = {αn1 , αn2 , . . . , αnKn } denotes the action space
of player n with Kn = |Sn ||Pn | total number of actions. D. Distributed no-regret learning
Action αn = (sn , pn ) ∈ An consists of a tuple of SF and No-regret learning [18] is adopted to achieve an -CCE in
TP of player n with values sn and pn , respectively. That a distributed manner. Specifically, players minimize the regret
is, a node’s action is its choice of SF and TP. Moreover, of their actions: after carrying out a certain action that has
Ω = Ω1 × Ω2 × · · · × ΩN is the space of random events led to some utility, they evaluate their regret for not having
that characterizes the wireless channel and the network traffic. chosen a different action. Then, the probability of playing
In particular, Ωn is the set of all tuples ωn = (σn , λn ) the actions with the highest regret is increased accordingly.
that identify the random state of node n, where σn is the By doing so, nodes learn and maximize their utilities. This
standard deviation of the channel between n and the closest learning occurs in communication rounds as follows. Each
gateway; and, λn is the average sending rate of n. Nodes aim node sends c packets by autonomously choosing a particular
to maximize their individual delivery ratio, thus, each player’s action. Afterwards, the gateway calculates the observed utility
utility un is defined as in Eq. (1), i.e., un = Dn . Note that un of the node as the ratio between the successfully received and
depends on both its own action and random state, as well as sent packets during that round. Additionally, it determines the
the actions and random states of other nodes in the network. random state of the node (see Section IV-A) and then sends
Thus, node n’s utility un : A × Ω → R≥0 can be expressed the related information (i.e., the node’s utility and random
as un (αn , α−n , ωn , ω −n ), in which α−n and ω −n are the state) back to the node in a downlink packet. Note that the
vectors of all other nodes’ actions and random states1 . communication rounds are asynchronous – those of individual
The formulated non-cooperative stochastic game targets a nodes are independent and have different duration, based on
mixed strategy that describes a conditional probability distri- how often they send packets – therefore, the nodes change their
bution over all possible actions, as follows: actions at different times. Moreover, individual realizations of
π(αk , ω j ) = Pr(A = αk |Ω = ω j ), ∀ αk ∈ A, ω j ∈ Ω, the random state Ω in time – namely, each ω(t) ∈ Ω(t) – are
assumed to be i.i.d.
where αk and ω j represent a specific realization of the Given that all nodes in the network follow the strategies they
network-wide action and random state. One desirable solution used over the previous T rounds, the regret of a specific action
in the game is a game-theoretic equilibrium where all players αnk ∈ An , given a random state ωnj for node n, is defined as
are satisfied with the strategy. To quantify this, the Coarse
T
Correlated Equilibrium (CCE) and its more general form [18] 1 Xh i
rn (αnk , ωnj )= un (αnk , α−n (t), ωnj , ω j−n (t))−ũn (t) (5)
(-CCE) are considered next. T t=1
Definition 1 (-Coarse Correlated Equilibrium). Given a non- in which ũn (t) is the observed utility of node n at round
negative  ≥ 0, the mixed strategy π is an -CCE for some t. The regret is the normalized difference between the utility
ω j if it satisfies: that would have been obtained by playing action αnk and the
X actual utility given by the current action. A positive regret
un (αnk , α−n , ωnj , ω j−n )π(αnk , α−n , ωnj , ω j−n )
rn (αnk , ωnj ) > 0 signifies that node n can achieve a better
α−n ∈A−n
X utility by consistently playing action αnk , while the other nodes
− un (α, ωnj , ω j−n )π(α, ωnj , ω j−n ) ≤ , ∀n ∈ N , αnk ∈ An follow their original strategies. Then, node n with regret vector
α∈A r n = [rn (αnk , ωnj ) : αnk ∈ An , ωnj ∈ Ωn ] selects its action
In other words, the utility enhancement of a player (over based on the Boltzmann distribution that results from solving:
the utility achieved by following the -CCE strategy) who 
 X 
unilaterally deviates from the -CCE strategy, while other βn (r n , αn , ωnj ) = arg max πn (αnk , ωnj )rn (αnk , ωnj )
players follow the -CCE strategy, is bounded by . Note that j 
πn (αn ,ωn ) αk
n ∈An
a CCE is achieved if  = 0. Moreover, the equilibrium is #)
achieved for each ω j independently, i.e., there is an -CCE 1 k j k j
∀ωnj ∈ Ωn , (6)


− πn (αn , ωn ) ln πn (αn , ωn )
for each ω j . A straightforward way to implement the -CCE κn (ωnj )
strategy is with a central entity that instructs all nodes to
jointly play the action α according to the -CCE distribution where πn (αn , ωnj ) = Pr(αn |ωnj ). Such a distribution allows
π(α, ω). However, the centralized approach is not feasible to explore different actions [18]. In particular, the temperature
in the considered scenario, as it requires to send downlink parameter κn (ωnj ) > 0 controls the trade-off between the
optimality of the utility (i.e., exploitation) and maximizing the
1 The −n subscript denotes the set of all players excluding the n-th one. entropy of the strategy (i.e., exploration). Specifically, when
 5

P∞ 1
κn (ωnj ) is small, the entropy of the system is high and the Note that the p-series t=1 tp diverges when p ≤ 1
node explores actions more broadly. Conversely, the entropy and converges when p > 1. Hence, the learning rates
lowers when κn (ωnj ) increases and the node starts choosing the µn (ωnj , t), γn (ωnj , t) and νn (ωnj , t) can be selected as [26]:
actions with the highest probabilities. In contrast with [18, 26], 1
this work considers random states, therefore, the temperature µn (ωnj , t) = 1{ωn =ωnj } · , (11)
t pµ
parameter κn (ωnj ) is chosen to be dependent on ωnj to keep 1
the exploration and exploitation phases independent for each γn (ωnj , t) = 1{ωn =ωnj } · pγ , (12)
t
random state. The solution of Eq. (6) is given by [18]: 1
νn (ωnj , t) = 1{ωn =ωnj } · pν , (13)
exp κn (ωnj )rn+ (αnk , ωnj )

t
k j
βn (r n , αn , ωn ) = P  , with 0.5 < pµ , pγ , pν ≤ 1, so that the conditions in Eq. (10a)
exp κn (ωnj )rn+ (αni , ωnj ) and Eq. (10b) are satisfied. Furthermore, pµ > pγ > pν holds
αin ∈An
to meet the conditions in Eq. (10c) and (10d). The learning
rn+ (αnk , ωnj ) = max {0, rn (αnk , ωnj )}, rates µn , γn and νn are updated at every round, which in turn
affects the amount of new information adopted in updating the
∀ αnk ∈ An , ωnj ∈ Ωn . However, calculating the utility function
utilities, regrets, and probability distributions. These rates also
un (αnk , α−n (t), ωnj , ω j−n (t)) in Eq. (5) requires node n to
depend on ωnj to update the parameter for each random state
know all the previous actions and random states of all the other
independently. Finally, the temperature is updated as:
nodes in the network. That is, a perfect information game is
assumed. This is clearly impractical, as the information about κn (ωnj , t) = κn (ωnj , t − 1) + 1{ωn =ωnj } · ψ(t), (14)
the other nodes would need to be transmitted by frequent
downlink packets. To tackle this issue, NoReL is introduced with a non-decreasing function ψ(t). In this work, ψ(t) = t2 ,
next as an imperfect information game where nodes iteratively as it tends to infinity. As a consequence, the nodes explore
estimate their utility and regret of each action αnk ∈ An their actions when t is small and then exploit the best ones
to update their strategy at the beginning of each round t, as t increases. This, in turn, makes  smaller with time, thus
given their current random state ωnj . The detailed steps are asymptotically converging to a CCE [18]. Specifically, nodes
the following: are constantly participating in the game and the convergence
of the system is defined as follows.
ûn (αnk , ωnj , t) = ûn (αnk , ωnj , t − 1)

Definition 2 (Convergence time). The  convergence time t0 for
+νn (ωnj , t)1{αn =αkn } 1{ωn =ωnj } ũn (t) − ûn (αnk , ωnj , t−1) , (7)
a random state ω j = ω1j , · · · , ωN j
and some small δ > 0
r̂n (αnk , ωnj , t) = r̂n (αnk , ωnj , t − 1) is such that, for all t ≥ t0 and n ∈ N ,


+γn (ωnj , t) ûn (αnk , ωnj , t) − ũn (t) − r̂n (αnk , ωnj , t−1) , (8) P
πn (αnk , ωnj , t) − πn (αnk , ωnj , t − 1)
αk
n ∈An
πn (αnk , ωnj , t) = πn (αnk , ωnj , t − 1) ≤ δ.

Kn
+µn (ωnj , t) β(r̂ n (t), αnk , ωnj ) − πn (αnk , ωnj , t−1) , (9) NoReL is proven to converge to an -CCE [18] next.
∀ αnk ∈ An , ωnj +
∈ Ωn , t ∈ Z . Here, 1{x=y} is the indicator Definition 3 (Lipschitz function). An f function is said to
function that is equal to 1 if x = y or to 0 otherwise. be Lipschitz if |f (x) − f (y)| ≤ C|x − y|, for all x and y in the
The rationale behind these equations is the following. Pt Let us domain of f and for a real constant C independent of x and y.
consider the moving-average estimator â(t) = 1t τ =1 a(τ )
of the expectation E[a], which can be recursively written as Proposition 1. The learning in Eqs. (7)–(9) converge to an
â(t) = 1t a(t)+ t−1 1 -CCE strategy π ∗ (ω j ) = {π1∗ (ω1j ), · · · , πN
∗ j
(ωN )} of G for
t â(t−1) = â(t−1)+ t [a(t)−â(t−1)]. Here,
the difference between the latest realization and the previous all ω j if and only if the conditions in Eq. (10) are met, where
estimation – namely, a(t) − â(t − 1) – is used to calibrate the lim πn (ωnj , t) = πn∗ (ωnj ).
t→∞
estimation, and its impact decays in terms of 1/t. In contrast, Proof. For simplicity, the following considers a fixed ω j as the
the weights of the calibration terms in Eqs. (7)–(9) are in their derivation is the same for all ω j ∈ Ω. The learning in NoReL is
general form νn (ωnj , t), γn (ωnj , t), and µn (ωnj , t), which are a discrete-time process that can be expressed as the asymptotic
round-dependent learning rates and satisfy: trajectory of a flow [18], therefore, its convergence can be
T
X proven on the basis of [25, Proposition 4.1]. Accordingly, it
µn (ωnj , t) + γn (ωnj , t) = ∞, is enough to show that un and βn (r n , αn , ωnj ) are Lipschitz
 
lim (10a)
T →∞
t=1 functions, since the conditions in Eq. (10) hold by hypothesis.
T
X From Eq. (1), the observed utility of node n at a certain
µn (ωnj , t)2 + γn (ωnj , t)2 < ∞,
 
lim
T →∞
(10b) round t can be expressed as un = 1−x, wherein x = Pc +P∗o −
∗ ∗
Pc Po , with Po = m∈M Pm
Q
t=1
o , is a function of the network-
γn (ωnj , t) wide action and random state, i.e., x = x(αn , α−n , ωnj , ω j ).
lim = 0, (10c)
t→∞ νn (ωnj , t) Letting 1 ≥ x ≥ y ≥ 0 yields:
µn (ωnj , t) |un (x) − un (y)| |(1 − x) − (1 − y)| |y − x|
lim = 0. (10d) = = = 1.
t→∞ γn (ωnj , t) |x − y| |x − y| |x − y|
 6

TABLE I: Simulation parameters. in [5]. For comparison purposes, we also report the energy
Parameter Value consumption E calculated as:
Sending rate (λn ) 0.001 s−1 [5, 7] N
Duty cycle 0.01 [4]
X V (Itx (pn )Tair (sn ) + 2Irx Trx )
E= ,
Path loss L(d0 ) = 128.95 dBm, d0 = 1 km, n=1
Dn
η = 2.32, σ = {3.54, 7.08} [2]
Packet length 20 bytes [29] where V is the input voltage of the LoRa devices; Itx (pn ) is
Preamble length 8 bytes [4] the current consumed for a transmission with TP pn ; Tair (sn )
Frequency 868 MHz is the time on air of a packet transmitted with SF sn ; Irx is the
Bandwidth 125 kHz current consumed during the two receive windows, of duration
Coding rate 4/8
SFs {7, 8, 9, 10, 11, 12} Trx , that follow an uplink transmission; and Dn is as in Eq. (1).
TPs {2, 5, 8, 11, 14} dBm In this work, V = 3.3 V, Irx = 11 mA, Trx = 164 ms [31],
Receiver {7: -124, 8: -127, 9: -130, 10: -133, while Itx (pn ) and Tair (sn ) are taken from [30]. Therefore, E
sensitivity per SF 11: -135, 12: -137} dBm [30] is the energy needed by all nodes to transmit a packet that is
Supply current {2: 24, 5: 25, 8: 25, 11: 32, successfully received.
per TP 14: 44} mA [30]
Time on air per SF {7: 0.0780, 8: 0.1397, 9: 0.2467, For all the results presented next, each individual simulation
10: 0.4935, 11: 0.8560, 12: 1.7121} s run lasts for 15 days of simulated time, unless otherwise
stated; all experiments are repeated 30 times and the average
value is reported in the figures together with the corresponding
Hence, un is a Lipschitz function. Consider now the standard deviation. Table I shows the simulation parameters;
regret vectors x and y of node n; trivially, it is those for the path loss parameters were derived in [2] through
0 < |βn (x, αn , ωnj ) − βn (y, αn , ωnj )| < 1, therefore, extensive measurements on a real LoRa network deployed in
|βn (x, αn , ωnj ) − βn (y, αn , ωnj )| < C|x − y| ∀x, y and for a sub-urban environment. The packet length is set to 20 B,
some scalar C. Consequently, both un and βn (r n , αn , ωnj ) a typical value in LoRa networks as shown in [29]. The
are Lipschitz functions, which completes the proof. rest of the parameters were chosen according to those widely
employed in the state of the art [5, 7].
As explained in Section III-D, NoReL requires the gateways
IV. P ERFORMANCE EVALUATION to calculate, at every round, the random state of the nodes
This section evaluates NoReL against the state of the art by – namely, the observed standard deviation of the wireless
simulation. It first introduces the related setup and methodol- channel (σn ) and the average sending rate (λn ). To simplify the
ogy, it then discusses how to estimate the path loss parameters implementation, the channel conditions and sending rates are
based on observed channel conditions. It finally presents computed across the whole network rather than for individual
the results obtained for both static and dynamic scenarios, nodes. Therefore, the gateway only sends the network-wide
comprising networks with different layouts and characteristics. parameters (σ, λ) to all the nodes in range, instead of sending
individual values (σn , λn ) to each node. The ordinary least
squares method [32] is adopted to estimate the value of σ
A. Setup and methodology over a window of 80 packets as detailed in Section IV-B;
λ is calculated with the packets received over one-hour in-
A custom discrete-time Python simulator2 was developed tervals as λ = λN . Moreover, the random state space is
to determine the SF and TP assignments of nodes over time. discretized as follows: σ can belong to one of the inter-
The simulator incorporates – for all uplink and downlink vals [0, 5), [5, 10), [10, 15), [15, 20]; λ to one of the intervals
packets – the capture effect, quasi-orthogonal SFs, duty-cycle [0, 0.003), [0.003, 0.006]. Consequently, eight random states
restrictions for both LoRa nodes and gateways, and a collision are considered in total.
model that requires the last five preamble symbols to be Another critical step in the implementation of NoReL is the
correctly received to decode the whole packet [4]. Moreover, selection of the learning parameters and the duration of a
gateways are considered half-duplex, i.e., no packets are re- round, as they affect the convergence time and the performance
ceived while they are transmitting and vice versa. As discussed of the network. After extensive simulations (not included here
in Section III-D, nodes update their SF and TP at the beginning due to lack of space), the following parameters were chosen
of each round with NoReL. Accordingly, the utility and random as a trade-off between convergence time and delivery ratio:
state of each node are computed and Eqs. (7)–(9) are applied pν = 0.8, pγ = 0.9, pµ = 1 and the round size c = 10. That
to determine its SF and TP at each round. is, downlink packets are sent every 10 packets to each node
The performance is primarily evaluated in terms of con- similar to the protocol used in ADR [21], which is widely
vergence time with δ = 0.01 (according to Definition 2) and implemented in real-world LoRa networks.
the average delivery ratio in the network as the probability
(expressed as a percentage) that its packets are correctly
B. Estimation of path loss parameters
received, obtained with Eq. (1), i.e., by evaluating the model3
The gateways estimate the channel conditions and report
2 Available at: https://github.com/VeronicaToro/NoReL/ them to the nodes in downlink packets. To this end, the
3 Available at: https://github.com/VeronicaToro/LoRa-model/ ordinary least squares (OLS) [32] method is adopted, as it
 7

estimated estimated
135 real 2.8 5
real
2.6
130 4
2.4
PL(d0)

125 2.2 3

σ
γ
120 2.0
1.8 2
115
1.6 estimated
1
110 1.4 real

0 5 10 15 20 25 0 5 10 15 20 25 0 5 10 15 20 25
Time (h) Time (h) Time (h)

(a) (b) (c)

Fig. 1: Path loss parameters (a) P L(d0 ), (b) γ and (c) σ for random changes occurring every 5 hours in a day.

is easy to implement and it has been also used in real LoRa C. Comparison against state of the art
networks [33]. Here, the gateway estimates P L(d0 ), γ and σ The performance of NoReL is compared against the schemes
based on the receive and transmission power of packets as well discussed next. MinSF is a simple baseline that assigns each
as on its distance from the source node over a window of L node the lowest SF required to achieve connectivity to the
packets. For this purpose, the following matrix representation nearest gateway at the highest TP. This scheme requires infor-
is introduced. First, χ = [1 10 log(d/d0 )] is an L by 2 mation on the distance of each node to the nearest gateway and
matrix with 1 and d the column vectors of ones and distances, the path loss parameters. Thus, it runs at the central network
respectively. That is, d contains the distances between the server [19]. Tolio is the two-player game proposed in [14]
gateway and the source node for each of the L packets. to assign SFs to LoRa nodes, played between each node and
 T
Moreover, C = P L(d0 ) γ is a 2 by 1 matrix and the rest of the network. Each node is aware of the aggregate
Σ = [Xσ ] a column vector of size L containing the standard distribution of SFs in the network and chooses between
deviation of the channel for each of the L received packets, keeping the current SF or increasing it by one. All nodes
modeled by a normal distribution, i.e., Xσ ∼ N (0, σ). Then, use the highest TP. Adaptive Data Rate (ADR) assigns SFs
the vector of receive powers is given by: and TPs to each node such that the overall network capacity
y = χC + Σ. is increased and energy consumption is minimized [22]. It
comprises two algorithms that run independently at the node
This allows to estimate P L(d0 ) and γ, respectively, as the first and the network server. The algorithm running at the node is
and second entry of the following vector: defined by the LoRaWAN specifications [27]: it increases the

−1 T
Ĉ = χT χ χ y. SF and TP of a node whenever it cannot reach a gateway.
On the other hand, the algorithm running at the network
Finally, the variance σ 2 of Xσ is estimated as: server depends on the operator of the LoRa network. The
1  T   implementation here follows the one presented in [22]: the
σ2 = y − χĈ y − χĈ .
L−1 network server adjusts the SF and TP depending on the quality
To keep the estimated parameters smooth, a learning rate ζ is of a link estimated by observing the average signal-to-noise
used to satisfy: ratio (SNR) of the last 20 packets received from the node.
Unless otherwise stated, the device margin is set to 15 dB [21]
P
d L(d0 )t = ζ P
d L(d0 )t−1 + (1 − ζ)P
d L(d0 )t , and the remaining parameters are set to the default values
γ̂t = ζ γ̂t−1 + (1 − ζ)γ̂t , reported in [22, 27]. DyLoRa is the mechanism proposed in [12]
σ̂t = ζ σ̂t−1 + (1 − ζ)σ̂t , to assign SFs and TPs to nodes. Similar to ADR, there are
two components that run at the node and the network server.
where x̂t is the estimation of parameter x at time t. The value The algorithm at the node simply resets the SF and TP to
ζ = 0.3 was empirically found to achieve the best trade-off the highest values whenever uplink connectivity is lost (i.e.,
between the variance in the estimated data and the response 12 consecutive packets are dropped). On the other hand, the
delay after changes in the channel conditions. network server predicts the delivery ratio for each node for
Figure 1 characterizes the accuracy of path loss parameters all combinations of SFs and TPs based on the SNR of the 6
estimation for a case where P L(d0 ), γ and σ vary randomly previously received packets. It then chooses the configuration
every 5 hours during one day; the shaded areas in the plots that maximizes the energy efficiency of the node, i.e., the one
indicate the 95% confidence intervals. The figure clearly shows that achieves a high predicted delivery ratio with a low energy.
how the OLS-based estimation quickly obtains4 the correct 1) Static scenarios: First, the network performance is eval-
value upon changes, with only a small variation over time – uated in a static scenario wherein nodes are deployed uni-
namely, less than 5% in all cases. formly located within a radius of 2 km from a single gateway.
4 The convergence to the correct value could be shortened by decreasing ζ Different densities of the network are evaluated by varying
at the cost of a higher variance. the number of nodes. Figure 2a presents the average delivery
 8

(a) (b)
Fig. 2: (a) Average delivery ratio and (b) energy consumption as a function of the number of nodes in static scenarios.

ratio, showing similar values for MinSF, Tolio and NoReL in is the time on air of the packets determined by the SF that
networks with fewer than 800 nodes. Still, NoReL outperforms primarily affects the energy consumption. This can be clearly
other approaches in all network densities. Most nodes can use seen from the parameters Itx (pn ) and Tair (sn ) in Table I: while
the smallest available SF (i.e., SF7) to reach the gateway with a Itx (2) and Itx (14) differ by 45.4%, Tair (7) and Tair (12) differ
deployment radius of 2 km. Thus, MinSF assigns SF7 to around by 95.4%. Therefore, keeping a low SFs is the best option
75% of the nodes and SF8 to the rest of the nodes. On the other to save energy but not to achieve a high delivery ratio, as
hand, Tolio adds more SF diversity, with approximately 50% previously discussed. DyLoRa actually incurs in the highest
of the nodes using SF8. This results in a high delivery ratio energy consumption despite being designed to optimize energy
for networks with few nodes, e.g., with 200 nodes. However, efficiency because the related characterization [12, Eq. (3)]
as the density of nodes increases, using only SF7 and SF8 penalizes high TPs instead of high SFs.
increases the probability of collisions, even though the time- Different network layouts are evaluated next, including sce-
on-air of the packets is short. In such scenarios, NoReL assigns narios with multiple gateways and a non-uniform distribution
higher SFs to a few nodes (around 22% of the nodes use of nodes around a gateway. In both cases, the networks
SFs higher than 8) resulting in a better SF diversity, thereby comprise 1,000 nodes. Figures 3a and 3b show these layouts
reducing the probability of collisions between packets. For this where the gateways are represented by black triangles and the
reason, NoReL achieves the highest average delivery ratio in nodes by colored points whose colors show the SFs assigned
all considered densities. ADR achieves a slightly lower average by NoReL; Figures 3c and 3d, instead, show the SFs assignment
delivery ratio than MinSF because, while MinSF assigns TP14 of MinSF for comparison5 purposes. Figure 3a clearly shows
to all the nodes and NoReL does the same for approximately that NoReL mainly assigns SF7 to the nodes that are the
81% of the nodes, ADR assigns TPs lower than TP14 to 30% closest to the gateway and SF8 to the nodes in the outer
of the nodes. As discussed before, a high TP minimizes ring, similar to MinSF. However, there is some SF diversity
the probability of outage and collisions with other nodes. among all nodes. In contrast, Figure 3c highlights how SFs are
Finally, DyLoRa achieves the lowest average delivery ratio, assigned in rings by MinSF. ADR produces the same assignment
as it is designed to achieve a high energy efficiency through as the one obtained by MinSF, whereas Tolio and DyLoRa
lower SFs and TPs [12]. Specifically, DyLoRa assigns TP14 to exhibit a greater SF diversity, with SF8 and SF10 as the
only 5% of the nodes. Finally, additional simulations showed most used, respectively. For the non-uniform scenario, the
that in larger networks (with a 6 km deployment radius) the difference between NoReL (Figure 3b) and MinSF (Figure 3d) is
average delivery ratio achieved by all approaches is similar less visible, as both approaches primarily assign SFs in rings.
even with a high density of nodes (not included due to space This happens for NoReL because nodes are located far away
constraints). Indeed, the nodes are more constrained in their from the gateway, resulting in a few actions to choose from.
communication parameters, under larger deployment areas. Nevertheless, NoReL still assigns a high SF to a few nodes. The
For instance, the nodes farthest from the gateway must use assignments of Tolio and DyLoRa, instead, mostly rely on SF8
SF12 and TP14. Then, other nodes will avoid using SF12 to and SF10, respectively, similar to the previous scenario.
minimize collisions. Accordingly, the possibilities of changing Figures 3e and 3f show the delivery ratio of the different
SFs and TPs are limited, thus ADR, Tolio and NoReL converge schemes for the two considered static scenarios. These results
to an assignment similar to that of MinSF. clearly depend on the diversity of the assigned SFs. In fact,
Figure 2b depicts the energy consumption achieved by the NoReL and Tolio achieve the highest delivery ratio for the
different approaches in the considered static scenarios. The scenario with multiple gateways (Figure 3e) – in detail, NoReL
figure shows that MinSF, Tolio and ADR result in a similar achieves a delivery ratio 2.3% higher than ADR – as they exhibit
energy consumption at all network densities. This happens be- the highest TP and SF diversity. For the non-uniform scenario,
cause they assign low SFs, even though they also assign TP14 the difference between the average delivery ratios achieved
to most of the nodes. On the other hand, DyLoRa and NoReL 5 The rest of the discussion also refers to the assignments obtained by ADR,
consume higher energy as they assign higher SFs. In fact, it Tolio and DyLoRa even though they are not reported in figures for brevity.
 9

4.0 12 4.0 12
3.5 3.5
Distance (km)

Distance (km)
3.0 11 3.0 11
2.5 10 2.5 10

SF

SF
2.0 2.0
1.5 9 1.5 9
1.0 8 1.0 8
0.5 0.5
0.0 7 0.0 7
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
Distance (km) Distance (km)

(a) (b)

4.0 12 4.0 12
3.5 3.5
Distance (km)

Distance (km)
3.0 11 3.0 11
2.5 10 2.5 10
SF

SF
2.0 2.0
1.5 9 1.5 9
1.0 8 1.0 8
0.5 0.5
0.0 7 0.0 7
0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7
Distance (km) Distance (km)

(c) (d) (e) (f)

Fig. 3: Spreading factor assignment of: NoReL for the (a) 2-gateway and (b) non-uniform scenarios; MinSF for the (c) 2-gateway
and (d) non-uniform scenarios. Gateways are denoted as black triangles and nodes as colored circles in the figures. Average
delivery ratio of the considered approaches for the (e) 2-gateway and (f) non-uniform scenarios.

by MinSF, Tolio, ADR and NoReL is below 1% (Figure 3f). gateway and TP14. Figure 4a shows that NoReL achieves
The slightly better performance of NoReL is due to the few the highest delivery ratio. In fact, NoReL achieves a delivery
nodes that are assigned a high SF, as previously discussed. ratio 1.4% higher than ADR and 12% higher than DyLoRa, on
DyLoRa achieves a significantly lower delivery ratio than the average. Moreover, the convergence time of NoReL is low, as
other schemes in both scenarios due to the low TPs assigned it takes around 14,000 packets (approximately 7 hours) to
to the nodes while aiming to maximize energy efficiency. converge after the new nodes are added. This is due to the
2) Dynamic scenarios: The following three scenarios are learning rates that the other nodes had beforehand. That is, all
considered to evaluate how the different schemes adapt to the pre-existing nodes had already reached the steady state and
changes in network conditions, when: (i) new nodes are added only the new ones had to learn the best strategy. It is worth
to an existing network with a stable configuration; (ii) the observing that DyLoRa sets the new communication parameters
average sending rate changes after a certain time, for instance, more quickly for the recently-added nodes (they all reach their
an application may temporarily require more data during final parameters in around 4 hours), even though its delivery
certain time periods; (iii) the channel conditions vary due to ratio (around 76.4%) is lower than that of both ADR and NoReL.
changes in weather conditions [9], in-band traffic from other Figure 5 shows the SF and TP assignment over time of
networks or even the presence of obstacles (e.g., people) near the different methods for the considered scenario. It is clear
end devices [10]. All scenarios are evaluated in a network how NoReL tries different strategies before converging to the
with one gateway and 500 nodes uniformly deployed within final assignment, which is almost a uniform distribution of SFs
a radius of 4 km. MinSF and Tolio are no longer considered, and mainly high TPs (Figure 5a). The SF and TP assignment
as they are designed for static scenarios and do not define of ADR (Figure 5b) and DyLoRa (Figure 5c), instead, remains
means to identify when nodes are reconfigured or notified basically the same after the new nodes are added.
about the new configuration. In addition to the delivery ratio, Additional traffic. In the second scenario, the initial network
the results reported next provide the SF and TP assignment configuration is obtained with MinSF. Then, the average send-
obtained by the considered schemes in the different scenarios. ing rate per node is increased from λ = 0.001 to λ = 0.005
In all cases, values are shown as a function of the simulation after five days. The sending rate then goes back to its initial
time, expressed in terms of the number of packets sent in the value (λ = 0.001) after five more days. Figure 4b clearly
network, to abstract from the actual sending rate. The time at shows that the delivery ratio is affected by the increase of
which a change takes place is marked with a dotted vertical network traffic for all considered schemes (i.e., NoReL, ADR
line in all figures. In addition, Figure 4 shows the standard and DyLoRa). Specifically, the average delivery ratio of NoReL
deviations of the obtained values as a shaded area and the goes down and then up again while the network converges
convergence time for NoReL, marked as t0 . to the -CCE. This happens when the nodes change their
Additional nodes. In the first scenario (Figure 4a), the configuration from MinSF to NoReL (at the very beginning) and
network is initially configured with a stable configuration when λ is changed (after about 210,000 packets). Moreover,
achieved by the specific approach being evaluated (ADR, the nodes rapidly adopt the configuration they had already
DyLoRa or NoReL). After two days of simulated time (around learned at the very beginning after 10 days, when the initial
86,700 sent packets), 100 new nodes are added at random sending rate is used again. NoReL requires longer time than
locations within the deployment area. The newly-added nodes ADR and DyLoRa to converge to a stable configuration. This is
initially use the minimum SFs that allow them to reach the because of its completely distributed nature. In particular, the
 10

(a) (b) (c)

Fig. 4: Delivery ratio for scenarios where (a) 100 new nodes are added to the network, (b) the average sending rate is increased
by a factor of 5, and (c) the channel conditions alternate between two settings every 12 hours.

(a) (b) (c)

Fig. 5: SF and TP assignment over time for (a) NoReL, (b) ADR and (c) DyLoRa in the scenario with added nodes.

convergence time of NoReL at normal network traffic is around network traffic, except for small changes in the TP distribution
82,800 packets and 115,000 packets at high traffic, as shown (Figure 6b). On the other hand, DyLoRa quickly converges to a
by the black circles in Figure 4b. The convergence time is SF and TP assignment at each network traffic level (Figure 6c).
higher for the increased traffic as there is a higher probability
of packets being dropped, which results in longer time for the Changes in channel conditions. In the third scenario, the
nodes to complete their rounds. However, the transient only channel conditions vary following the empirical results pre-
takes place the first time a random state is met, after which sented in [9], i.e., the variance and mean of the received
the nodes learn the best configuration and use it every time power change periodically by approximately 10 dBm in the
they return to such a state. This explains why NoReL returns worst case [9, Figure 5]. Accordingly, the path loss parameters
almost instantaneously to the highest delivery ratio after the (γ, σ) were varied between the two settings of (2.32, 3.54) and
traffic goes back to the initial conditions. (3.32, 7.08). The device margin for ADR was set to σ + 8 dB
in the experiments, following a preliminary analysis showing
NoReL achieves the highest delivery ratio; compared to that such a margin had to be σ-dependent to avoid instability.
ADR, 3.6% higher with high network traffic and around 1%
Figure 4c shows that NoReL achieves the highest delivery
higher at normal traffic. Surprisingly, NoReL presents a peak
ratio throughout, about 7% higher than ADR and DyLoRa
shortly after the sending rate is increased, before converging
under the worst channel conditions. Note that the two channel
to a slightly lower delivery ratio. This happens because the
settings starting at 0 and 12 hours (every 21,500 packets)
overall performance of the configuration used at that peak
represent distinct random conditions that involve two indepen-
does not correspond to the equilibrium achieved by NoReL.
dent convergence periods. The figure shows that 12 hours are
In fact, converging to an -CCE does not imply that the
not enough for NoReL to converge in either random condition
network reaches its optimum configuration then. Instead, the
within a single time window. However, the nodes resume
convergence is in the -neighborhood of a CCE [15], i.e., the
learning when similar channel conditions occur again, starting
overall performance can deviate from the CCE by .
from the best configuration learned until then. Thus, the nodes
Figure 6 shows the SF and TP assignment over time in this do not encounter a drop in delivery ratio, for instance, towards
case. With NoReL, nodes again converge to their strategies in the end of the simulated time. Nevertheless, there is some
each random state and then quickly adopt the configuration transient for NoReL after each change is implemented, where
they previously had when a certain state is met again (Fig- the delivery ratio starts from a slightly lower value and quickly
ure 6a). Instead, ADR is mainly unaffected by the increased sets to the converged value. This is simply the time it takes
 11

(a) (b) (c)

Fig. 6: SF and TP assignment over time for (a) NoReL, (b) ADR and (c) DyLoRa in the scenario with increased traffic.

(a) (b) (c)

Fig. 7: SF and TP assignment over time for (a) NoReL, (b) ADR and (c) DyLoRa in the scenario with varying channel settings.

for all the nodes to receive information from the gateway TABLE II: Summary of results.
about the random condition. In contrast, nodes do not leverage Average delivery ratio (%)
previously-known best configurations with ADR or DyLoRa. Scenario
MinSF Tolio ADR DyLoRa NoReL
Finally, Figure 7 shows the SF and TP assignment over time 200 nodes 94.2 95.1 92.4 87.6 95.6
in the considered scenario. Interestingly, NoReL converges to 2,000 nodes 84.7 84.3 84.4 69.3 85.9
a stable assignment in two different random conditions (Fig- 2 gateways 94.6 95.2 93.1 82.9 95.2
ure 7a). It is also worth noting that lower SFs and higher TPs Non-uniform 88.3 88.2 88.5 73.3 88.7
are used in the state corresponding to (γ, σ) = (2.32, 3.54), 100 more nodes - - 86.8 76.4 88.2
while the opposite happens when the channel conditions λ = 0.001 - - 87.7 78.2 89.2
deteriorate. The assignment of ADR is again mostly the same λ = 0.005 - - 75.8 68.5 79.2
across all simulation time, except for small changes in the TPs γ = 2.32, σ = 3.54 - - 89.6 80.5 91.5
(Figure 7b). Finally, DyLoRa converges to a solution every γ = 3.32, σ = 7.08 - - 62.1 60.6 68.8
time the channel conditions change; however, DyLoRa tends
to assign higher SFs and TPs when the channel conditions
deteriorate (Figure 7c), in contrast to NoReL. obtained results establish that NoReL is flexible and suitable for
diverse, highly-dynamic environments.

D. Summary V. C ONCLUSION
Table II summarizes the obtained results. Clearly, NoReL This article has proposed NoReL, a game-theoretic approach
outperforms all other schemes for all considered scenarios. for setting SFs and TPs to improve communication reliability
This happens as it takes into consideration the delivery ratio of in LoRa networks. Specifically, a no-regret learning procedure
individual nodes to make decisions. Unsurprisingly, MinSF and was devised for nodes to independently learn the utilities of
Tolio perform well in static conditions, as they are specifically their actions and maximize their delivery ratio by choosing the
designed for such use cases; they are not suitable for the appropriate SF and TP. This procedure is shown to converge
dynamic environments of real-world deployments though. In to an equilibrium where nodes are satisfied with their utilities.
contrast, ADR and DyLoRa are adaptive, therefore, they allow Extensive simulations have shown that NoReL achieves a higher
nodes to set their parameters according to the actual envi- average delivery ratio than the state of the art. Moreover, it
ronmental conditions. However, they mostly rely on the SNR quickly obtains a new equilibrium in dynamic scenarios where
of packets sent by nodes, which does not account for all the the channel and network conditions suddenly change. The
factors that may arise in dynamic scenarios. In conclusion, the flexibility of the proposed solution makes it ideal for any type
 12

of LoRa network. The implementation of NoReL in real-world [19] D. Magrin, M. Centenaro, and L. Vangelista, “Performance
LoRa deployments is left as a promising future work. evaluation of LoRa networks in a smart city scenario,” in IEEE
ICC 2017.
[20] P. Kumari, H. P. Gupta, and T. Dutta, “An incentive mechanism-
ACKNOWLEDGMENTS based stackelberg game for scheduling of LoRa spreading
We would like to thank Yinghui Li for clarifying some factors,” IEEE Trans. Netw. Service Manag., vol. 17, no. 4, pp.
details of DyLoRa, Thomas Marchioro for providing the 2598–2609, 2020.
[21] The Things Network, “Adaptive Data Rate,” https://www.
source code used in [14], and the CSC – IT Center for Science thethingsnetwork.org/docs/lorawan/adaptive-data-rate/.
for provisioning the computational resources used in the study. [22] M. Słabicki, G. Premsankar, and M. Di Francesco, “Adaptive
configuration of LoRa networks for dense IoT deployments,” in
R EFERENCES IEEE/IFIP NOMS 2018.
[1] M. Centenaro, L. Vangelista, A. Zanella, and M. Zorzi, “Long- [23] I. Ilahi, M. Usama, M. O. Farooq, M. Umar Janjua, and J. Qadir,
range communications in unlicensed bands: the rising stars in “LoRaDRL: Deep reinforcement learning based adaptive PHY
the IoT and smart city scenarios,” IEEE Wireless Communica- layer transmission parameters selection for LoRaWAN,” in
tions, vol. 23, no. 5, pp. 60–67, 2016. IEEE LCN 2020, pp. 457–460.
[2] J. Petäjäjärvi, K. Mikhaylov, A. Roivainen, T. Hanninen, and [24] R. Kerkouche, R. Alami, R. Féraud, N. Varsier, and P. Maillé,
M. Pettissalo, “On the coverage of LPWANs: range evaluation “Node-based optimization of LoRa transmissions with multi-
and channel attenuation model for LoRa technology,” in 2015 armed bandit algorithms,” in 2018 25th International Confer-
14th International Conference on ITS Telecommunications. ence on Telecommunications (ICT), 2018, pp. 521–526.
[3] A. Mahmood, E. Sisinni, L. Guntupalli, R. Rondón, S. A. Has- [25] M. Benaı̈m, “Dynamics of stochastic approximation algo-
san, and M. Gidlund, “Scalability analysis of a LoRa network rithms,” in Séminaire de Probabilités XXXIII. Springer Berlin
under imperfect orthogonality,” IEEE Trans. Ind. Inform, 2018. Heidelberg, 1999, pp. 1–68.
[4] M. C. Bor, U. Roedig, T. Voigt, and J. M. Alonso, “Do LoRa [26] S. Batewela, C. Liu, M. Bennis, H. A. Suraweera, and C. S.
low-power wide-area networks scale?” in ACM MSWiM ‘16. Hong, “Risk-sensitive task fetching and offloading for vehicular
[5] V. Toro-Betancur, G. Premsankar, M. Słabicki, and M. Di edge computing,” IEEE Commun. Lett., 2020.
Francesco, “Modeling Communication Reliability in LoRa Net- [27] LoRa Alliance, “LoRaWAN Specification v1.1.”
works with Device-level Accuracy,” in INFOCOM 2021. [28] J. Nash, “Non-cooperative games,” Annals of mathematics, pp.
[6] D. Croce, M. Gucciardo, S. Mangione, G. Santaromita, and 286–295, 1951.
I. Tinnirello, “Impact of LoRa imperfect orthogonality: Analysis [29] N. Blenn and F. Kuipers, “LoRaWAN in the wild:
of link-level performance,” IEEE Commun. Lett., vol. 22, no. 4, Measurements from the things network,” arXiv preprint
pp. 796–799, 2018. arXiv:1706.03086, 2017.
[7] G. Premsankar, B. Ghaddar, M. Słabicki, and M. Di Francesco, [30] SX1272/73: Low Power Long Range Transceiver, Semtech,
“Optimal configuration of LoRa networks in smart cities,” IEEE March 2017, rev. 3.1.
Trans. Ind. Inform, vol. 16, no. 12, pp. 7243–7254, 2020. [31] Semtech, “An In-depth look at LoRaWAN Class A
[8] D. Garlisi, I. Tinnirello, G. Bianchi, and F. Cuomo, “Capture Devices,” https://lora-developers.semtech.com/documentation/
aware sequential waterfilling for LoRaWAN adaptive data rate,” tech-papers-and-guides/lorawan-class-a-devices.
IEEE Trans. Wireless Commun., 2021. [32] C. Gustafson, T. Abbas, D. Bolin, and F. Tufvesson, “Statistical
[9] T. Ameloot, P. Van Torre, and H. Rogier, “Variable link perfor- modeling and estimation of censored pathloss data,” IEEE
mance due to weather effects in a long-range, low-power LoRa Wireless Communications Letters, 2015.
sensor network,” Sensors, vol. 21, no. 9, 2021. [33] G. Callebaut and L. Van der Perre, “Characterization of LoRa
[10] ——, “A compact low-power LoRa IoT sensor node with point-to-point path loss: Measurement campaigns and modeling
extended dynamic range for channel measurements,” Sensors, considering censored data,” IEEE Internet of Things Journal,
vol. 18, no. 7, 2018. vol. 7, no. 3, pp. 1910–1918, 2020.
[11] M. Centenaro, L. Vangelista, and R. Kohno, “On the impact
of downlink feedback on LoRa performance,” in IEEE PIMRC
2017.
[12] Y. Li, J. Yang, and J. Wang, “DyLoRa: Towards energy efficient
dynamic LoRa transmission control,” in INFOCOM 2020.
[13] D.-T. Ta, K. Khawam, S. Lahoud, C. Adjih, and S. Martin,
“LoRa-MAB: A flexible simulator for decentralized learning
resource allocation in IoT networks,” in IFIP WMNC 2019.
[14] A. Tolio, D. Boem, T. Marchioro, and L. Badia, “Spreading
factor allocation in LoRa networks through a game theoretic
approach,” in IEEE ICC 2020, pp. 1–6.
[15] S. Hart and A. Mas-Colell, “A Simple Adaptive Procedure
Leading to Correlated Equilibrium,” Econometrica, 2000.
[16] J.-W. Lee, A. Tang, J. Huang, M. Chiang, and A. R. Calderbank,
“Reverse-engineering MAC: A non-cooperative game model,”
IEEE Journal on Selected Areas in Communications, vol. 25,
no. 6, pp. 1135–1147, 2007.
[17] M. A. Khan and Y. Sun, “Non-cooperative games with many
players,” ser. Handbook of Game Theory with Economic Ap-
plications. Elsevier, 2002, vol. 3, ch. 46, pp. 1761–1808.
[18] S. Samarakoon, M. Bennis, W. Saad, and M. Latva-aho,
“Backhaul-Aware Interference Management in the Uplink of
Wireless Small Cell Networks,” IEEE Trans. Wireless Commun.,
vol. 12, no. 11, pp. 5813–5825, 2013.