56 IEEE TRANSACTIONS ON SUSTAINABLE COMPUTING, VOL. 8, NO.

1, JANUARY-MARCH 2023

Clustering-Based Energy Efficient Task

Offloading for Sustainable Fog Computing
Anirudh Yadav , Prasanta K. Jana , Senior Member, IEEE, Shashank Tiwari, and Abhay Gaur

Abstract—Delay and energy efficient task offloading from device to fog nodes involves decision making challenges wherein an
integrated optimal scheme for preserving sustainability of the terminal nodes (TNs) and fog nodes (FNs) is extremely important. In this
paper, we propose a novel clustering based delay aware energy efficient task offloading scheme in a Software-Defined Networking
(SDN) based fog architecture. A bi-objective problem is formulated for optimum clustering of TNs with respect to FNs, selection of
offloading parameters and, joint delay and energy minimization. It is then tranformed to a scalarized single objective problem which has
a nested structure with the two problems: 1) optimal clustering and 2) optimal offloading for a given set of clusters. Based on this,
Optimal Clustering and Offloading Parameters (OCOP) algorithm is designed which has lesser time complexity than the usual
quadratic case. Through extensive simulations, we have shown that the use of explicit clustering in the proposed algorithm improves FN
participation and reduces activity time and energy levels thereby increasing sustainability of the FNs and TNs as compared with the
random case and a similar task offloading algorithm. Moreover, even cluster size distribution lowers our algorithm’s running time than
the quadratic case.

Index Terms—Fog computing, software defined network, task offloading, clustering, latency and energy minimization

1 INTRODUCTION The strategy of clustering TNs around FNs has been

opted by us because optimal clustering increases the fair uti-
a fog computing environment, fog nodes (FNs) are
I N
deployed to locally compute the tasks offloaded by the
end devices known as terminal nodes (TNs). This avoids
lization of the FNs for short bursts of time when they are
required so that sustainability is not affected. The purpose
of the SDN architecture in our proposed scheme is to enable
offloading of the tasks from the TNs to cloud and thus
concurrent virtual cluster formation and task offloading.
reduces cost in terms of latency and energy consumption.
The concurrency assists in the creation and maintenance of
However, sustainability of the FNs and TNs raises the fol-
a dictionary of these virtual clusters that can be accessed
lowing questions: 1) How to decide which TN task is to be
and utilized directly during real time offloading scenarios.
offloaded and to which FN? 2) What fraction of the TN task
The rationality behind the use of SDN based architecture is
is to be offloaded?, and 3) How much transmission power is
as follows. It can handle dynamic load on the network and
required by the TN to offload that fraction of the task? These
provides a global view of the network conditions. This
issues are challenging particularly when we require to
allows the centralized controller of the SDN to collect net-
maintain a trade off between maximal usage and sustain-
work information from heterogeneous wireless devices [4],
ability of the FNs and TNs.
[5], [20]. It also offers easy management of the network and
In this paper, we address all these issues and propose a
fog-based IoT infrastructures. All these advantages of the
delay aware, energy efficient task offloading scheme using
SDN are suitably exploited by our purposed scheme.
software-defined networking (SDN) in the fog layer. The
Although, there are some clustering based task offload-
scheme handles the first issue by clustering all the TNs sur-
ing algorithms but the idea of using explicit clustering for
rounding FNs optimally. Each cluster consists of a single
allocating FNs to TNs has been overlooked by all of them.
FN and multiple TNs (as shown in Fig. 1). The TNs can off-
They are thus deprived of the benefits as mentioned previ-
load their tasks to the FN in their respective clusters. We
ously. For example, the work proposed by Luo et al. [7] uses
refer to this as explicit clustering of TNs. The second and
a clustering approach for horizontal offloading among end
third issues are handled by optimizing the offloading
devices instead of vertical task offloading from the end
parameters, i.e., fraction of task size and transmission
device layer to the fog layer. And, the papers [8], [9], [10],
power.
[11] deal with either the intra-fog network/edge device
clustering or the clustering for fog to cloud scenario. To clar-

The authors are with the Department of Computer Science and Engineer- ify the positioning of this paper further, we present sum-
ing, Indian Institute of Technology (ISM) Dhanbad, Dhanbad, Jharkhand mary of comparisons in Table 1.
826004, India. E-mail: {yadavanirudh1997, shashanktwr007, btech. The main contributions of this paper are summarized as
abhaygaur}@gmail.com, [email protected].
follows:
Manuscript received 16 August 2021; revised 19 April 2022; accepted 22 June
2022. Date of publication 27 June 2022; date of current version 7 March 2023. 1) Formulation of multi-objective optimization problem: We
(Corresponding author: Anirudh Yadav.) formulate task offloading as a constrained non-linear
Recommended for acceptance by S. Misra.
Digital Object Identifier no. 10.1109/TSUSC.2022.3186585 optimization problem with respect to two proposed

2377-3782 © 2022 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See ht_tps://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY WARANGAL. Downloaded on January 02,2024 at 06:52:34 UTC from IEEE Xplore. Restrictions apply.
 4) Simulation results: We perform extensive simulations
and compare the results with the most closely
related algorithm and the random clustering case.
The proposed algorithm outperforms them in terms
of participation of the number of FNs, activity time
of the fog layer, and the distribution of energy con-
sumption among the FNs and TNs.
The organization of the paper is as follows. Section 2
briefly reviews the existing works, Section 3 presents the
system model and problem formulation. The proposed
algorithm is presented in Section 4. The simulation results
along with their analyses are described in Section 5 followed
by the conclusion in Section 6.

2 RELATED WORKS
A limited number of works have been carried out on SDN
platform. For example, Akbar et al. [13] applied machine
learning based multiobjective optimization in an SDN-
enabled multihop scenario for obtaining Pareto optimal
paths for task offloading. [14] and [15] proposed offloading
in SDN-enabled vehicular fog computing. Zhang et al. [12]
proposed a Fair and Energy-Minimized Task Offloading
(FEMTO) algorithm for heterogeneous fog enabled IoT net-
work. They achieved a trade-off between low energy con-
sumption and delay with fair selection among multiple FNs
for task offloading from a TN. Other related recent works
on SDN are [6] and [20]. Misra and Saha [20] presented a
greedy based heuristic for multi-hop dynamic task offload-
ing problem in the SDN fog architecture with access points.
Chen and Hao [6] proposed an architecture of software-
defined ultra dense network and then reported a task off-
loading scheme that minimizes the delay while saving the
battery life of user’s equipment.
Contemporary papers employing clustering strategy are
[7], [8], [9], [10], [11]. Luo et al. [7] proposed a coalition game
Fig. 1. Results before and after clustering. Dotted lines in (a) denote the
formulation which selects an optimal execution mode at
one hop reachability and solid lines in (b) indicate the offloading each end device resulting in the micro clusters for coopera-
connection. tive task execution via horizontal offloading among end
devices in such clusters. Wang et al. [8] developed a cluster-
ing strategy to classify mobile vehicles into multiple cooper-
novel parameters, clustering guess probability ative vehicular edge servers according to various deciding
parameter and offloading parameter. These two criteria. Dautov and Distefano [9] extended the Apache
parameters help to find the best clustering strategy, NiFi stream processing middleware with the help of run-
optimal offloading task size and transmission power time clustering of edge devices. Bozorgchenani et al. [11]
for jointly optimizing energy consumption and presented Energy-Aware Offloading Clustering Approach
delay. (EAOCA) entirely in the fog layer with heterogeneous FNs.
2) Derivation of a single objective function: By scalarizing, And lastly, Asensio et al. [10] solved the problem of fog-
we derive a single objective function of the above cloud clustering to find the optimal control architecture in
joint optimization problem. We find a nested struc- the fog-cloud scenario. However, all such works are differ-
ture in the objective function. The outer problem ent from our proposed scheme. They apply clustering
aims to find the best clustering and the inner, opti- approach for horizontal offloading among either the end
mal task size and the transmission power. devices or the FNs rather than vertical offloading from TNs
3) Proposal of a task offloading algorithm: Next, we present to FNs. The work which is closest to our approach is
the proposed algorithm called optimal clustering FEMTO [12]. Although this algorithm does not follow any
and offloading parameters (OCOP) algorithm. Clus- clustering approach, the result of fairly selecting the right
tering is made adaptive to the changes in the task FN for a TN with the best offloading parameters is in fact
sizes. The algorithm exhibits an improvement over one way of clustering TNs with respect to FNs just as we
the usual quadratic time complexity for clustering. have. A summary of the comparisons with the above related

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY WARANGAL. Downloaded on January 02,2024 at 06:52:34 UTC from IEEE Xplore. Restrictions apply.
 58 IEEE TRANSACTIONS ON SUSTAINABLE COMPUTING, VOL. 8, NO. 1, JANUARY-MARCH 2023

TABLE 1
Comparison Table of Literature on Clustering

References End device layer Delay Energy Consumption Explicit Clustering strategy Offloading paramters
Luo et al. [7] ✓ ✓ ✓ No None
Wang et al. [8] ✗ ✓ ✗ No None
Dautov and Distefano [9] ✗ ✓ ✗ No None
Asensio et al. [10] ✗ ✓ ✗ No Transmission power
Bozorgchenani et al. [11] ✗ ✗ ✓ No None
Zhang et al. [12] ✓ ✓ ✓ No Both1
Our proposed scheme ✓ ✓ ✓ Yes Both

works is also listed in Table 1. Thus it provides enough

TABLE 2
scope of novelty for clustering based delay aware energy Notations Used
efficient task offloading from device to fog in an SDN-fog
architecture for sustainable fog computing as proposed in Notation Description
this paper. fj symbolic identifier of the FN in the jth cluster
Various other task offloading algorithms [16], [17], eji symbolic identifier of the ith TN in the jth cluster
[18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], Nj Number of TNs in the jth cluster
[29], [30] have been reported for fog computing. Aazam aji Fraction of task offloaded by eji
et al. [16] evaluated three scenarios fog only, cloud only pji Transmission power used by eji
and collaborative fog-cloud, over real datasets for three- pmax Maximum allowed transmission power
tier fog-cloud IoT model and found the suitable task exe- pmin Minimum allowed transmission power
dji Maximum allowed delay for the task at eji
cution policy for each case. Chen et al. [17] designed a tj i Total task size in bits at eji
layer-aware fog computing architecture for privacy pre- n½eji  The processing frequency of eji in cycles/s
serving data collection and computation offloading. They n½fj  The processing frequency of fj in cycles/s
proposed sensing layer for data collection and encryp- ½eji  The energy consumption (Joules) at eji per cycle
tion, fog layer for delay efficient computation offloading ½fj  The energy consumption (Joules) at fj per cycle
and user layer for data decryption and reconstruction. s½eji  Number of cycles for processing one bit at eji
s½fj  Number of cycles for processing one bit at fj
Zhu et al. [18] jointly optimized the service latency and
g½eji ; fj  Channel gain between eji and fj
quality loss in the Vehicular Fog Computing (VFC) sce-
nario. Yang et al. [19] solved the multi-task multi-helper
problem using a game theory based algorithm called
Paired Offloading of Multiple Tasks (POMT). Zu et al. controller separated from the service plane. Since the algo-
[21] formulated a two phase Stable Matching Algorithm rithm runs as a software program in the virtual controller
for Energy-Minimized Task Offloading (SMETO) for ser- independent of the offloading which happens in the service
vice and energy efficiency. Papers [22], [23], [24], [25], plane, it allows preprocessing a set of virtual clusters and
[26] reported task scheduling algorithms in either homo- offloading parameters at the beginning and also concur-
geneous or heterogeneous fog networks. Zhu et al. [27] rently during task offloading. Therefore, this beneficial fea-
considered a novel stochastic program with delayed ban- ture of SDN based fog architecture makes our cluster
dit feedback under abrupt system parameter changes mapping adaptive to the generated instantaneous TN task
and solved it by a Bandit Learning based Offloading of size distribution vector, making our offloading scheme
Tasks (BLOT) algorithm. On the other hand works [28], unique.
[29], [30] dealt with the problem of resource allocation in We assume that each task is independent bit-wise which
fog and cloud networks. implies that it can be arbitrarily divided into two subtasks
as per the data-partition model [3]. We also assume that
each TN generates one task in real time. So, when a task is
3 SYSTEM MODEL AND PROBLEM FORMULATION
generated at a TN, it is divided into two parts, one part is
3.1 Model and Assumptions executed locally and the other part is offloaded to a nearby
We consider a fog network that consists of a set of M FNs, FN. We define a task to be the total number of bits say, t
ff1 ; f2 ; . . . ; fM g with heterogeneous computational resour- that are required to be processed for its complete execution.
ces and a set of N TNs, fe1 ; e2 ; . . . ; eN g, where N  M, Hence, with respect to task offloading a TN may transmit a
which are all deployed in some area as shown in Fig. 1a. All fraction of the total number of bits say, at; a 2 ½0; 1 to a
the TNs constitute the end device layer and all the FNs con- nearby FN and process the remaining part ð1  aÞt locally.
stitute the fog layer. The fraction a is chosen to fulfill our objective of reducing
We also assume that the interconnection network is energy consumption and delay.
based on SDN architecture that consists of a service plane,
comprising of all the fog and end devices and, a virtual
3.2 Definitions
1. ”Both” implies fraction of task offloading and transmission Let us now define all the required terminologies that will be
power. used later in the problem formulation and the proposed
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY WARANGAL. Downloaded on January 02,2024 at 06:52:34 UTC from IEEE Xplore. Restrictions apply.
 YADAV ET AL.: CLUSTERING-BASED ENERGY EFFICIENT TASK OFFLOADING FOR SUSTAINABLE FOG COMPUTING 59

TABLE 3
Definitions

Notation Description
Cðek Þ The choosing set of FNs for TN ek , Eq. (1)
 Cluster mapping of TNs to FNs, Eq. (3)
} Continuous cluster mapping of TNs to FNs, Eq. (4)
c Vector of offloading parameters, Eq. (10)
D Function for } 7! , Eq. (9)
Gðr=RÞ PDF of scaled FN distances r=R around a TN,
Eq. (5)
Dðek Þ Ascending ordered set of distances to reachable
FNs for TN ek
DTproc =DFproc Processing delay, Eq. (13)/(14)
Dtrans Transmission delay, Eq. (15)
Dwait Queue wait time at a FN, Eq. (17) Fig. 2. Illustration of the definition of choosing sets.
T F
Eproc =Eproc Processing energy consumption, Eq. (22)/(25)
T
Etrans Transmission energy consumption, Eq. (23)  ¼ ½f3 f2 f1 f2 f1 f2 f3 T ;
v Power supply indicator of a FN, Eq. (26)
e
E Historic energy consumption pattern of a FN, the clusters are ff1 ; e3 ; e5 g; ff2 ; e2 ; e4 ; e6 g; ff3 ; e1 ; e7 g
Eq. (27)
Since the number of TNs are generally quite larger than the
number of FNs i.e., N  M, the vector  will always have
multiple TNs mapped to the same FN forming a cluster con-
algorithm sections. We also provide a summary of com- taining those TNs and that FN. The search space of  of size
monly used notations in Table 2 and definitions in Table 3. N M is huge, so we create a continuous cluster mapping }
Definition 1: The choosing set Cðek Þ for each TN ek is defined as and map it to . We call } the clustering guess probability
the ordered set of all the FNs within its one hop reachability parameter and introduce a function DðÞ which transforms
where the order is the distance. } to . As the name suggests, } is a collection of guess
probabilities from all the TNs and is expressed as a vector
Mathematically the same can be expressed as
} ¼ ½}1 }2 . . . }N T s.t. 0 < }i < 1; 8ei (4)
Cðek Þ ¼ ffk1 ; fk2 ; . . . ; fkL g; 8ki 2 f1; 2; . . . ; Mg
To understand what the guess probabilities are, consider
s.t., distðek ; fk1 Þ < distðek ; fk2 Þ <    < distðek ; fkL Þ (1)
that for each TN a normal distribution is chosen over the
where, distðÞ is the distance function. ratio of a random variable r and the one hop radius R i.e.,2
r=R  N ð0; s 2 Þ, where r denotes the distances at which
Illustration 1: As an example, consider Fig. 2 which illustrates a FNs could possibly be present with respect to a TN. The
particular arrangement of three FNs ff1 ; f2 ; f3 g and seven guess probability for a TN then implies that it guesses the pres-
TNs fe1 ; e2 ; e3 ; e4 ; e5 ; e6 ; e7 g. An encircled shaded region indi- ence of an FN at a distance corresponding to that probability.
cates that the TNs and FNs in this region are within one hop Mathematically, let the kth component of the vector } be }k ,
reachability. The choosing sets for each TN are then then the TN ek guesses the presence of an FN at the distance b
such that the following equality holds for b and }k ,
Cðe4 Þ ¼ ff3 ; f2 ; f1 g  
Cðe5 Þ ¼ ff1 g; Cðe6 Þ ¼ ff2 g; Cðe7 Þ ¼ ff3 g b 1 
ðb=RÞ2
}k ¼ G ¼ pffiffiffiffiffiffi e 2s2 (5)
Cðe1 Þ ¼ ff3 ; f1 g; Cðe2 Þ ¼ ff2 ; f3 g; Cðe3 Þ ¼ ff2 ; f1 g (2) R s 2p
where GðÞ denotes the Gaussian probability distribution
function (PDF) of N ð0; s 2 Þ, the presence of FNs around TN
ek . In other words, the TN guesses distance,
3.2.1 Clustering Guess Probability Parameter
vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
!
Now we describe the method of cluster formation used in u
u 1
the proposed scheme. Using the choosing sets for each TN, 1  t
b ¼ RG ð}k Þ ¼ Rs 2 ln  pffiffiffiffiffiffi (6)
we construct an arbitrary vector  of dimension N which }k s 2p
maps each TN to an FN as follows:
According to the definition of the random variable r=R, the
mapped to ei presence of each FN can be anywhere in the open range ð0; RÞ.
z}|{ But for the implementation of the proposed algorithm, we
 ¼ ½fk1 fk2 . . . fki . . . fkN T
|{z} must find a closed range inside this open range. To do this, we
from Cðei Þ can assume without any loss of generality that each of the near-
where, fk1 2 Cðe1 Þ; fk2 2 Cðe2 Þ; . . . ; fkN 2 CðeN Þ (3) est FNs can neither be present farther than distance g nor can
the farthest FNs be farther than R  g for all TNs. Then we can
This vector  provides the clusters of the TNs surrounding
the FNs. For example, as per Fig. 2, for some clustering vec- 2. Z  N ð0; s 2 Þ means that the random variable Z follows a normal
tor say, distribution with mean 0 and variance s 2 .
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY WARANGAL. Downloaded on January 02,2024 at 06:52:34 UTC from IEEE Xplore. Restrictions apply.
 60 IEEE TRANSACTIONS ON SUSTAINABLE COMPUTING, VOL. 8, NO. 1, JANUARY-MARCH 2023

x 2 Dðei Þ, %1
i ½x ¼ Fi Cðei Þ, s.t.,
x ¼ distðei ; fm Þ; 8fm 2 Fi .
The DðÞ function discretizes } to  based on the afore-
said definitions as follows:
h i
j ¼ Dð} Þj ¼ %1 1 
j ½RghDðej Þ; RG ð}j Þi (9)
1

where, ½A1 operation selects one element form the set A

with equal probability 1=jAj. This equation, Eq. (9) defines
the D function. The function GðÞ can be a PDF which is not
Gaussian in nature and defined over the same random vari-
able, r=R 2 ð0; 1Þ until the condition: it is invertible in (0,1), is
satisfied.

3.2.3 Clustered Offloading Parameter

Note that we have to pay equal attention to the variables
Fig. 3. Case where the FN ft at distance d is selected by TN ek with that explicitly change both energy consumption and delay
choosing probability }k when r=R  N ð0; 0:5Þ.
referred as the offloading parameters. It is a combination of
1) the fraction of task size decided by each of the TN that
has to be offloaded and 2) the transmission power by which
close the range ð0; RÞ by defining r 2 ½g; R  g. Therefore, we the TNs transmit the same to their FNs of the cluster. We
must not let TNs waste their guesses in the regions ½0; gÞ and group the offloading parameter into a vector according to
ðR  g; R. Hence the bounds on } can be defined as follows: the clusters of the TNs and FNs as decided by the previous
parameter } . This is because energy consumption and
}min ¼ ½Gð1  g=RÞ . . . Gð1  g=RÞ . . . Gð1  g=RÞT delay give rise to this parameter and their efficiency is only
}max ¼ ½Gðg=RÞ . . . Gðg=RÞ . . . Gðg=RÞT meaningful if task offloading happens. And it happens only
after the clustering. So, the representation of the clustered
P ¼ f} : }min 3 }  }max g offloading parameter c for a given } or  is as follows:
 fraction of tasks to be offloaded
zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}|fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{
c ¼ a11 . . . a1N1 ; . . . ; aj1 . . . ajNj ; . . . ; aM1 . . .;
3.2.2 The D Function |fflfflfflfflfflfflffl{zfflfflfflfflfflfflffl}
for jth cluster TNs
In this subsection, we present the vector-valued DðÞ func-
transmission powers T
tion which transforms } to . We require the following zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}|fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{
definitions. p11 . . . p1N1 ; . . . ; pj1 . . . pjNj ; . . . ; pM1 . . . (10)
|fflfflfflfflfflfflffl{zfflfflfflfflfflfflffl}
for jth cluster TNs
Definition 2: An FN ft at distance d with probability of pres-
ence Gðd=RÞ is chosen by TN ek , if the guess }k by the TN ek
is just less than Gðd=RÞ. In other words, an FN ft at distance If there is no explicit restriction on the amount of offloaded
d is chosen by TN ek , if the guess distance RG1 ð}k Þ by the data by each TN, then all the aji are considered to be in the
TN ek is just greater than d. closed range [0,1]. If it is explicitly required, we can choose
Since, GðÞ is a decreasing function this is justified, the range from among ½amin ; 1, ½0; amax  or ½amin ; amax  as
per the restrictions. The operable power range is maintained
 
d between pmin and pmax . Therefore, the bounds on c can be

}k G ) RG1 ð}k Þ  d (8) summarized as follows:
R
This can be visualized as presented in Fig. 3. cmin ¼ ½0 . . . 0 . . . 0; pmin . . . pmin . . . pmin T
Definition 3: For an ascending value ordered set S ¼ cmax ¼ ½1 . . . 1 . . . 1; pmax . . . pmax . . . pmax T
fl1 ; l2 ; l3 ; . . . ; lK g and scalar l, we define the range function as C ¼ fc : cmin  c  cmax g (11)
RghS; li which finds ln just less than l i.e., ln l lnþ1
s:t:ln ; lnþ1 2 S.
The distance set Dðek Þ for each choosing set Cðek Þ can be 3.3 Problem Formulation
defined as
The basic objective of our proposed algorithm is to mini-
Definition 4: For a choosing set Cðei Þ ¼ ff1 ; . . . ; fm ; . . . ; fP g, mize energy consumption and delay. So, we first assume an
the distance set is Dðei Þ ¼ f%i ½f1 . . .; %i ½fm ; . . . ; %i ½fP g, arbitrary cluster mapping  ¼ Dð} Þ and consider the jth
where for each m, %i ½fm  ¼ distðei ; fm Þ and for some distance cluster ffj ; ej1 ; ej2 ; . . . ; ejNj g in it, where fj is the FN and eji
are the TNs surrounding fj . Similarly, we take an arbitrary
3 m  m ) mi mi ; 8i 2 f1; . . . ; Ng where m ¼ ½m1 ; . . . ; mN T and task offloading decision c. This gives us, the fraction aji of
m ¼ ½m1 ; . . . ; mN T the total task size tji to be offloaded from the TN eji to FN fj
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY WARANGAL. Downloaded on January 02,2024 at 06:52:34 UTC from IEEE Xplore. Restrictions apply.
 YADAV ET AL.: CLUSTERING-BASED ENERGY EFFICIENT TASK OFFLOADING FOR SUSTAINABLE FOG COMPUTING 61


at transmission power pji . The FN fj receives tasks from all
the TNs like eji through an MIMO antenna system and cre- Dji ¼ max DTproc ½eji ;Dtrans ½eji ; fj 
ates a queue of the same. In the queue, tasks with lesser
amount of permitted delay are kept more towards the front þ ðDwait ½eji  þ DFproc ½eji ; fj Þ ; (18)
of the queue so that they can be processed and completed
before their shorter deadlines. Hence, if dji is the maximum or,
allowed delay for the task from eji then the offloaded tasks

in the queue of the FN are ordered from front to back such
that, Dji ¼ max DTproc ½eji ; Dtrans ½eji ;fj þ

X
i
dj1 dj2 dj3 . . . djNj (12) DFproc ½ejs ; fj  : (19)
s¼1

Alternatively, if all the FNs are supposed to have some par-

allel processing capability then we can directly carry out the For the case when tasks are processed in parallel at fj
processing of all the tasks in parallel without forming any 
queue at the FN. Dji ¼ max DTproc ½eji ; Dtrans ½eji ; fj  þ DFproc ½eji ; fj  : (20)
Now let us go for the bi-objective problem formulation.
As supposed by [12] the output of processing and the prop-
agation delays are insignificant in terms of their transmis- Energy consumption at TN eji is the sum of the expendi-
sion time and energy consumption, hence they are ignored. ture in processing the fraction of task size present locally
So, the time DTproc in local processing of ð1  aji Þtji task at eji and in transmission of the fraction to be offloaded. So
and DFproc in processing aji tji at fj , [12] are,
E½eji  ¼ Eproc
T
½eji  þ Etrans
T
½eji ; fj ; (21)
s½eji 
DTproc ½eji  ¼ ð1  aji Þtji (13)
n½eji  where

and, T
Eproc ½eji  ¼ ð1  aji Þtji s½eji ½eji  (22)
T
Etrans ½eji ; fj  ¼ Dtrans ½eji ; fj pji : (23)
s½fj 
DFproc ½eji ; fj  ¼ aji tji (14)
n½fj  Hence

The transmission time incurred in sending aji tji fraction of X

Nj
X
Nj
EjT ¼ E½eji  ¼ T
Eproc ½eji  þ Etrans
T
½eji ; fj  ; (24)
task size from eji to fj denoted by Dtrans , [12] is, i¼1 i¼1

Dtrans ½eji ; fj  ¼ aji tji =r½eji ; fj ; (15) here EjT is the total energy consumption at the TNs of the
  jth cluster. Energy consumed in processing the offloaded
g½eji ; fj pji
r½eji ; fj  ¼ Blog 2 1 þ (16) tasks from all the TNs at fj denoted by Eproc F
½fj  is
Bn0 expressed as

where, B is the bandwidth, n0 is the noise power spectral X

Nj
density, g½eji ; fj  is the channel gain, defined as F
Eproc ½fj  ¼ aji tji s½fj ½fj : (25)
g 1 ðdistðeji ; fj ÞÞg 2 and distðÞ function measures the dis- i¼1

tance between eji and fj . g 1 ; g 2 are the path loss exponent We now present two definitions for problem formulation,
and the path constant respectively [31]. inspired from [12] and [25] as follows.
The delay Dwait ½eji  is the waiting time in the queue of fj
due to processing of all the tasks from TNs ejs ð8s < iÞ that Definition 5: The number v½fj  takes values in the set
come before eji in the order of processing tasks in the queue f1; 2; 3; 4g. An FN can have either an external power supply
of fj . It is expressed as, (EPS) or an internal power source (IPS), a battery for example.
Also it can be in two states, passive (when it is sleeping) or
X
i1 active (when it is not in sleep mode)
Dwait ½eji  ¼ DFproc ½ejs ; fj  (17) 8
s¼1 >
> 1 when FN has an EPS and is passive.
<
2 when FN has an EPS and is active.
v½fj  ¼ (26)
When FNs use serial processing, we add Dwait to DFproc to >
> 3 when FN has an IPS and is passive.
:
correct the processing time at the FNs. But when tasks are 4 when FN has an IPS and is active.
processed in parallel then Dwait ¼ 0 and we use the usual
DFproc to measure the processing delay. Hence the total time
for processing the task from eji including the offloading Definition 6: The historic time average energy consumption
latency is denoted by Dji which is the maximum of the local summarizes the pattern of average energy consumption of a
and offloading latency device with time, in a number with the dimensions of energy
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY WARANGAL. Downloaded on January 02,2024 at 06:52:34 UTC from IEEE Xplore. Restrictions apply.
 62 IEEE TRANSACTIONS ON SUSTAINABLE COMPUTING, VOL. 8, NO. 1, JANUARY-MARCH 2023

e fj  which is updated after a

i.e., Joules. That number here is E½t; 4.1 Single Objective Function
processing is done in FN as, To solve the bi-objective problem, we use the scalarization
procedure to form a linearly scalarized log-scaled single
e þ 1; fj  ¼ ð1  uÞE½t;
e fj  þ uE F ½fj ; objective function which is as follows:
E½t proc (27)
min fð; cÞ;  ¼ Dð} Þ (34)
F
where Eproc ½fj  is the energy consumption due to the process- } ;c
e fj  and E½t
ing at FN. E½t; e þ 1; fj  are the historical time aver- fð; cÞ ¼ wlog 10 ðfE Þ  ð1  wÞSlog 10 ðfD Þ; (35)
age energy consumption for FN fj at time steps t and t þ 1 s.t., w 2 ½0; 1; c 2 C; } 2 P; 
(36)
respectively and u is the forgetting factor, u  102 .
here, w is the weight of preference given to the energy con-
The two individual problems can now be stated as fol-
sumption over delay. The sign-log function Slog ðÞ used
lows:
in (35) is explained in Appendix A, which can be found
on the Computer Society Digital Library at http://doi.
Nj  
X
M X
Dj ieeecomputersociety.org/10.1109/TSUSC.2022.3186585.
PD : max fD ¼ max 1 i (28) Note that due to the inevitable clustered definition of c its
8} ;aji ;pji 8} ;aji ;pji j¼1 i¼1 dji
structure is dependent on the parameter } but not its val-
PE : min fE ues. Therefore, both the parameters are interdependent and
8} ;aji ;pji
we must separate the optimization of the single objective
M 
X
e fj E F ½fj  function with respect to each of them. This is done using the
¼ min ðEjT Þ2 þ v½fj E½t; proc (29)
8} ;aji ;pji j¼1 principle of conditional optimization [32].
Theorem ( Principle of Conditional Optimization 4)
s.t., } 2 P; 0 aji 1; pmin pji pmax ; (30)
( )
8i 2 f1; . . . ; Nj g; 8j 2 f1; . . . ; Mg (31)
opt Vðu; vÞ ¼ opt opt Vðu; vÞ :
u2U u2U v2V ðuÞ
v2V ðuÞ
The first problem PD simply aims to maximize the differ-
ence between the maximum allowed delay and the actual The set V ðuÞ is the restriction over V because of the value
delay. In fact, this is minimization of the actual delay. The u 2 U. If the sets U and V are independent then
difference is then scaled by dividing it with the maximum V ðuÞ V . u
t
delay to keep the values under representable range. The
first term in PE suggests that EjT is the energy expenditure By restriction, it is meant that the set can only take values
for all the TNs in a cluster multiplied with itself to equate in a subset of the range. Applying the aforesaid theorem,
the dimensions of the second term i.e., Joules2 . The second we obtain the two problems Pinner and Pouter as follows:
term in PE scales the energy consumption Eproc F
½fj  at the 8 9
<zfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}|fflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{ >
> Pinner
=
FN according to the external power supply and its regular
energy consumption pattern in time. So, the second problem min fðDð} Þ; cÞ ¼ min min fðDð} Þ; cÞ :
} 2P } 2P >:c2Cð} Þ >
;
formulation thereby minimizes the energy consumption c2Cð} Þ

over all the TNs in the clusters either in processing or trans- |fflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflfflffl}
Pouter
mitting and, the energy consumption at the FNs, depending
e
on their operational circumstances v½ , E½t; . The bi-objec- Since } does not impose any explicit range in the values
tive problem corresponding to Eqs. (28), (29), (30), and (31) taken by c it only decides its structure, we can write Cð} Þ
is as follows: as C noting that c’s structure is now defined according to
} or equivalently by  ¼ Dð} Þ.
minðfD ; fE Þ (32)
} ;c Pinner : finner ð} Þ ¼ min fðDð} Þ; cÞ ¼ min f ðcÞ
c2C c2C
s.t., } 2 P; c 2 C (33) Pouter : min finner ð} Þ: 
(37)
} 2P

where Eq. (33) is based on the bounds in Eqs. (7) and (11). Note that Pinner is nested in Pouter , and for some given } ,
fðDð} Þ; cÞ is rewritten as f ðcÞ.
4 PROPOSED SCHEME 1) Optimization of Pinner : For solving Pinner we can either
The basic idea of the proposed scheme is as follows. We first apply a non-gradient, heuristic based technique or a rigor-
transform the bi-objective problem (32) to a single objective ous Hessian based non-linear optimization technique. The
optimization problem and bifurcated it into two nested computations using first option would incur numerous
problems namely Pinner and Pouter . The problem Pinner errors due to the heuristic nature of the technique. While
defined over c finds the optimum fraction of task size to be the latter option would demand large computational
offloaded and transmission power for a given set of clusters. resources making the algorithm slower for online applica-
The problem Pouter defined over } finds the optimal choice tion. Therefore, the limited memory bound constrained
of FNs for all TNs forming best clusters of TNs surrounding Broyden Fletcher Goldfarb Shanno (L-BFGS-B) algorithm
FNs. Both the problems Pinner and Pouter are optimized with
the joint objective of delay and energy minimization. 4. Please refer to [32] for the proof of this theorem.
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY WARANGAL. Downloaded on January 02,2024 at 06:52:34 UTC from IEEE Xplore. Restrictions apply.
 YADAV ET AL.: CLUSTERING-BASED ENERGY EFFICIENT TASK OFFLOADING FOR SUSTAINABLE FOG COMPUTING 63

[33], [34] is used which is faster than the Hessian based tech- clusters in DðPi Þ witnessed by the ith particle so far and, P g ,
niques without involving too many approximations. It only the global best position yields the best set of clusters in
requires the function and its gradient at every point in the DðP g Þ that all the particles over all stages would have wit-
search space. A derivation of the gradient is provided in nessed until now. Therefore, as we move on to higher stages
Appendix B, available in the online supplemental material. we get better and better set of clusters in the swarm all of
2) Optimization of Pouter : Next for solving Pouter , we apply which are nearer to the most optimal set of clusters. After
particle swarm optimization (PSO). The motivation behind the PSO stages end we apply the D function to P g giving us
the use of a heuristic approach is that } depends exclusively the optimal choice of FNs for each TN or in other words, the
on  and its presence in a huge search space of size N M turns best clusters of TNs surrounding FNs.
the optimization problem into an NP-complete problem.
a) Fitness Function: The fitness function for the applica- Algorithm 1. OCOP Algorithm
tion of PSO over Pouter is finner ð} Þ (refer Eq. (37)).
Input: All constant value parameters in the Table-2 and the
b) Particle Representation: The particle representation for
global view at the SDN-controller in the fog layer. The
the ith particle Xi } . Its dimension is the same as that of time counter (TSC s) at the SDN-controller. And, a dictio-
} i.e., N. The rth component of the position of the ith parti- nary of previous clustering events with key G and value as
cle in the swarm at the sth stage i.e., Xir ðsÞ 2 ½Gð1  a pair ð ; c Þ.
g=RÞ; Gðg=RÞ ½0; 1 provides us the guess distance // Offloading Segment
RG1 ðXir ðsÞÞ (cf. Definition 2) made by the rth TN for the 1: From the real time generated tasks trt compute Grt .
sth stage while selecting a FN in the process of cluster for- 2: Perform task offloading on the clusters obtained at
mation. So, the initial position of the ith particle in this minG2½Grt h;Grt þh jGrt  Gj.
swarm Xi ð0Þ denotes one of the collection of initial guesses 3: if TSC 0 ðmodulo tÞ then
made by the TNs and DðXi ð0ÞÞ maps these guesses to an ini- 4: Perform Steps 7 through 11 with random t and parallely
tial cluster mapping creating one of the set of initial non- offload tasks as per the optimal pair found in step 2.
optimal clusters. Let us now illustrate the particle represen- // offloading and preprocessing clusters
tation by means of the same example as given in Illustration 5: end if
1 and explain how we can obtain the clusters. 6: Perform offloading as per the optimal pair found in
step 2, then Goto step 1.
Illustration 2: Suppose for the case of Fig. 2, with choosing sets // Clustering Segment
as in Eq. (2) 7: Initialize random t and find the load variable G.
8: Initialize Cðek Þ; Dðek Þ; 8ek using definitions-1, 4.
Xi ð0Þ ¼ ½0:27 0:48 0:35 0:29 0:26 0:25 0:3T : 9: Apply Global-best PSO algorithm to obtain  with appro-
priate values of v; b1 and b2 using Eqs. (39) & (40) at every
The guess distance array obtained using RG1 ðXir ð0ÞÞ, for
stage, where the particle representation is } and the fitness
each r, if R ¼ 10 m and s 2 ¼ 0:5 is, function is finner ð} Þ defined below.
10: For any } 2 P, perform the following steps to get finner ð} Þ:
guess distance ¼ ½8:58 4:02 6:9 8:15 8:8 9:02 7:94T : 1. Apply the D function with the help of Dðek Þ8ek to gen-
Lastly, if the distance sets corresponding to Eq. (2) are erate , as in Eq. (7).
2. Define f ðcÞ; rc f ðcÞ for some c. See Appendix B,
Dðe4 Þ ¼ f5; 7; 9g available in the online supplemental material.
Dðe5 Þ ¼ f3g; Dðe6 Þ ¼ f4g; Dðe7 Þ ¼ f5g 3. Apply L-BFGS-B with input f ðcÞ; rf ðcÞ, C, to obtain
finner ð} Þ ¼ minc f ðcÞ and c .
Dðe1 Þ ¼ f5; 7g; Dðe2 Þ ¼ f4; 8g; Dðe3 Þ ¼ f6; 9g; (38) EndFor
11: Store  ¼ Dð} Þ obtained from Global-best PSO and corre-
then using the definition of D function we can argue, since e1
sponding c from L-BFGS-B for key G.
guesses a distance of 8.58 which is greater than all the entries
in Dðe1 Þ, it has to choose from the whole set one FN with equal
probability, let’s say it chooses f3 at 5. Similarly for e2 the c) Movement of the particle: In order to solve Pouter , we
guess distance is just greater than 4 and less than 8 in Dðe2 Þ, apply the Global-best PSO variant [35], [36] which has faster
so it chooses f2 at 4. Continuing this forms the initial cluster convergence and works well for box-constrained problem
DðXi ð0ÞÞ, as considered in this paper. Following are the equations
governing the movement of the particle in the swarm,
DðXi ð0ÞÞ ¼ ½f3 f2 f2 f2 f1 f2 f3 T
Vij ðtÞ ¼ mVij ðt  1Þ þ k1 Z 1 ðPij  Xij ðt  1ÞÞ
ff1 ; e5 g; ff2 ; e2 ; e3 ; e4 ; e6 g; ff3 ; e1 ; e7 g:
þ k2 Z 2 ðPjg  Xij ðt  1ÞÞ (39)

Xij ðtÞ ¼ Xij ðt  1Þ þ Vij ðtÞ: (40)

In this way, the whole swarm of particles produces dif-
ferent variants of the cluster mapping creating a swarm of Since Xij ðtÞ for some i; j; t follow the definition of }j , Xij ðtÞ
distinct sets of initial clusters. After some finite number of is restricted to ½Gð1  g=RÞ; Gðg=RÞ ½0; 1. So, if Xij ðtÞ >
PSO stages say K the ith particle now has the position Gðg=RÞ or Xij ðtÞ < Gð1  g=RÞ for any i; j; t then it is set to
Xi ðKÞ where DðXi ðKÞÞ now yields a better cluster mapping Gðg=RÞ or Gð1  g=RÞ respectively. Velocities Vi ð0Þ are
or set of clusters compared to DðXi ð0ÞÞ. Pi , the personal best intialized with some appropriate values, m is the inertia con-
position for the ith particle yields the most optimal set of stant to avoid the particles from continuing in the same
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY WARANGAL. Downloaded on January 02,2024 at 06:52:34 UTC from IEEE Xplore. Restrictions apply.
 64 IEEE TRANSACTIONS ON SUSTAINABLE COMPUTING, VOL. 8, NO. 1, JANUARY-MARCH 2023

direction. k1 ; k2 are acceleration constants and Z 1 ; Z 2 are complexity T ðN; MÞ of the OCOP algorithm (Clustering
two random variables uniformly distributed in [0,1], used segment) has a lower bound of VðN lg M þ N 2log N M Þ (see
to tune the rate of convergence and steer the particle Appendix D, available in the online supplemental material)
towards the optima respectively. and an upper bound of OðN 2 Þ ) T ðN; MÞ ¼ QðN k Þ; 1
k < 2. Since in our algorithm N  M 1 ) 1  log N M
4.2 Algorithm Design of OCOP 0, so 1 2  log N M  2. Further the upper bound is
Here we present the proposed OCOP algorithm given as reached not when M  N but only when all TNs form just
Algorithm 1. This is an online offloading algorithm which one cluster which is impossible due to sparse spatial distri-
finds optimal clustering and offloading parameters. The bution of FNs and the limited reachability of the TNs, there-
algorithm is explained stepwise as follows. Step 1 obtains fore, 1 k < 2. Hence, our approach is better than the
the real time task distribution t from the global view of the usual OðNMÞ approach which in the worst case, when M 
SDN-controller P and finds the corresponding real time load N tends to OðN 2 Þ time complexity.
variable GðtÞ ¼ N i¼1 ti =N. The load variable is used as key
to store preprocessed optimal clustering and offloading pair 5 SIMULATION RESULTS
ð ; c Þ. This is because the optimal value fð ; c Þ as a func- Simulation Setup.
tion of t is monotonically increasing with respect to GðtÞ (see
Simulations were carried out over the Google co-labora-
Appendix C for proof, available in the online supplemental
tory Tesla K80 GPU. The OCOP algorithm was run for a
material). Step 2 finds the nearest key value in the prepro-
single instance of task distribution vector t generated ran-
cessed dictionary to the previously obtained real time load
domly. Python parallel global multi-objective optimization
variable. Steps 3-6 offload the tasks and in parallel prepro-
(pygmo-2.16.0) library coupled with Non-Linear optimiza-
cess more dictionary elements for later use. The condition tion (NLopt-2.6.2) were used for application of global best
for a preprocessing event is defined in multiples of t PSO and L-BFGS-B algorithms. The PSO was carried out for
8 ! 1000 iterations with 55 particles out of which 11 particles
1 X
Nj
>
> s½fj  were processed in parallel. The L-BFGS-B algorithm was
>
> max a tk ; case5  1:
< j2f1;...;Mg Nj k¼1 n½fj  k run in each PSO iteration for 1000 variants, each starting at
t¼ !! distinct initial points. They were evolved for some finite
>
> s½fj 
>
> ak tk ; case  2: number of iterations where a batch of 100 variants was
: max max
j2f1;...;Mg k2f1;...;Nj g n½fj  processed in parallel in each iteration. A virtual network of
a set of TNs and a set of FNs deployed randomly over an
area of 100  100 m2 was used with one hop reachability
As, in the case when TNs were allowed to process only radius of R ¼ 40 m for each device. A wireless transmission
locally they would on an average take t time for the com- link with 10 MHz bandwidth, noise power spectral density
plete execution of all their tasks. Steps 7-11 find the optimal n0 ¼ 173 dBm/Hz, path loss factor 38:46 þ 20log 10 ðdistð:ÞÞ
clustering and offloading parameters concurrent to offload- dB and MIMO antenna system was assumed. Gaussian
ing in step 4 as follows. Step 7 initializes the random task distribution (refer Section 3.1) was used for the presence
distribution vector and corresponding load variable. Step 8 of FNs under 40 m with respect to each TN, having
initializes each TN’s choosing set and corresponding dis- mean 0 and variance 0.5. Table 4 summarizes the values
tance set obtained from the SDN-controller’s global view. of the rest of the parameters in the simulation setting.
Steps 9 and 10 solve Pouter and Pinner as delinted in Sec- The OCOP algorithm for two types of processing at the
tion 4.1. Finally in step 11, we add the new key (random FNs, serial or parallel was compared with random clus-
load variable of step 7) and its pair (the obtained optimal tering and offloading parameters’ case with the same
parameters) to the dictionary. two processing scenarios and with, FEMTO [12] (cf. Sec-
tion 2) as the base algorithm.
4.3 Computational Complexity of OCOP Energy Distribution and Usage Efficiency. In order to mea-
The step 7 in Algorithm 1 takes OðNÞ time and the initiali- sure the efficiency of energy quantitatively, here we define
zation step 8 takes essentially constant time due to the pres- the energy distribution metric mDE (Joules/M-Bytes) as the
ence of a global view of the interconnection of the TNs and average amount of energy consumption in one of the partic-
the FNs in the SDN-controller. Step 9 runs for a constant ipant FN (or TN) per amount of data offloading to the FNs.
number of stages and the P updations for all the particles If E f (or E t ) is the total energy consumption at FNs (or TNs)
takes linear time of OðPNÞ. Then in the same stage, the fit- in Joules, P f M (or P t ¼ N) is the number of participant
ness function is computed for which step 10.1 takes FNs (or TNs) and T o is the average amount of data offload-
OðN lg MÞ (using binary search for Definition 3) time in the ing (in M-Bytes) then mDE is expressed as
worst case when all FNs are reachable. Computation of the    
i term for f ðcÞ in step 10.2 and the ji component of the
jth th Ef Et
mDE ¼ mFN
DE ¼ þ mTN ¼ : (41)
PNjsame step takes Oði  1Þ time, in total it P f :T o P t :T o
DE
gradientPin the
takes 2 M j¼1 i¼1 Oði  1Þ time. Lastly, step 10.3 has a run-
time of OðmNÞ per stage, if m is the length of the memory The metric mDE indicates the distribution of the energy
per stage in the L-BFGS-B method [34]. Hence, the time among the participant FNs or TNs that is required to carry
out the given average amount of data offloading. For sus-
5 case-1:serial processing in FNs, case-2: parallel processing in FNs tainability, our objective is to maintain higher amount of
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY WARANGAL. Downloaded on January 02,2024 at 06:52:34 UTC from IEEE Xplore. Restrictions apply.
 YADAV ET AL.: CLUSTERING-BASED ENERGY EFFICIENT TASK OFFLOADING FOR SUSTAINABLE FOG COMPUTING 65

Fig. 4. Plot of the metric mDE for four scenarios. Scenario I: N ¼ 500; M ¼ 100, Scenario II: N ¼ 750; M ¼ 150, Scenario III: N ¼ 1000; M ¼ 250, &
Scenario IV: N ¼ 1000; M ¼ 500.

TABLE 4 TABLE 5
Simulation Parameters Percentage Delay Violations

Parameter Value N M Random-s OCOP-s Random-p OCOP-p

Number of TNs (N) 500, 750, 1000 500 100 64.2% 51.6% 42% 15.2%
Number of FNs (M) 100, 150, 250, 500 750 150 64.8% 49.3% 43.067% 9.6%
Total TN task size (t ) 1  10 MB(bytes) 1000 250 59.1% 48.1% 42.5% 9.7%
Maximum allowable delay (d ) 10  15 s 1000 500 52.1% 31% 47% 12%
Operable power range (pmin  pmax ) 0:001mW  1W
Historic average energy consumption 60 J
e
(E½t; )
TN processing frequency (n½e ) 1  2 GHz this limit lower than its corresponding random case and the
FN processing frequency (n½f ) 1  15 GHz same can be observed for the serial processing FNs (OCOP-
TN energy consumption per cycle (½e ) 5  7 J/cyc s) case. However, for OCOP-s delay violations are accept-
FN energy consumption per cycle (½f ) 1  10 J/cyc able only for larger number of FNs and TNs.
Number of cycle per bit at TNs (s½e ) 500  5000 cyc/
Clustering Efficiency. By clustering efficiency we mean the
bit
Number of cycle per bit at FNs (s½f ) 200  2000 cyc/ degree of evenness in the formation of clusters. To measure
bit the clustering efficiency we calculate the mean absolute
Weight of preference (w) 0.4 deviation
PM h of the partitions Nj of N about N=M, h ¼
j¼1 jNj  N=Mj=M; 0 h < 2NðM  1Þ=M 2 . From the
plots in Fig. 5 we infer that as the number of TNs and FNs
offloading with low energy levels. This implies that mDE are scaled up the average distance at which a FN clusters
should take lower values for higher sustainability. Numeri- TN decreases. And from the values of h we infer that lower
cally averaging over all four scenarios a decrement of the ratio N=M for a given reachability radius R, lower is h
approximately 74.72% is obtained in the OCOP with serial and higher is the clustering efficiency. We must note that
processing FNs case and, 74.85% for OCOP with parallel for lower values of h (or higher clustering efficiency) the
processing FNs in Fig. 4c. The rationality behind the lower runtime is more towards its lower bound (see Appendix D,
values of the metric in our cases can be attributed to the available in the online supplemental material).
larger extent of participation by the FNs compared to that in Activity Time of the Fog Layer. The activity time of a FN is
FEMTO. The average FN participation in all four scenarios the cumulative time it spends in receiving and processing
and both cases (serial processing FNs and parallel process- all the TN tasks offloaded to it (in seconds). Mathematically
ing FNs) for OCOP algorithm was 95% compared to that of for an FN fj activity time T ½fj  is
5% for FEMTO averaged over all four scenarios. Comparing
with the random case we note that clustering alone would ( N Pi
not solve the the problem of sustainability and that it has to j
maxi¼1 ðDtrans ½eji ; fj  þ s¼1 Dproc ½eji ; fj Þ; case-1:
F
be optimized as is done here. T ½fj  ¼ Nj
maxi¼1 ðDtrans ½eji ; fj  þ Dproc ½eji ; fj Þ; case-2:
F
Delay Efficiency. For measuring delay efficiency, we sim-
ply calculate the percentage of delay violations occurring
per hundred tasks offloaded. By providing this metric, we where case-1 is when tasks are processed serially at the FNs
are numerically approximating the probability of delay vio- and case-2, when they are processed in parallel. The activity
lations for the tested scenario. Our goal is to have no delay time of the fog layer is defined as the maximum of the activity time
violations at all. However, it is unattainable because of the for each FN in the fog layer, TF ¼ maxfj T ½fj , because each FN
trade-off between the energy consumption and delay. The is assumed to work independently. Fig. 6 portrays the plots of
percentage of delay violation below 40% is acceptable for TF on the right axis and, log ðT ½fj Þ versus fj for four scenar-
the task size in the range of ½1; 10 MB as argued in [28]. ios. The average decrement of the fog layer activity time over
It can observed from Table 5 that for OCOP with parallel all the four scenarios for OCOP compared with FEMTO is
processing FNs (OCOP-p), the delay violations are within 97.54%. As explained before, this can be attributed to the
Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY WARANGAL. Downloaded on January 02,2024 at 06:52:34 UTC from IEEE Xplore. Restrictions apply.
 66 IEEE TRANSACTIONS ON SUSTAINABLE COMPUTING, VOL. 8, NO. 1, JANUARY-MARCH 2023

Fig. 5. Plot showing the radial distance at which TNs are present when all the FNs are fixed at the origin. All TNs marked with the same shape and
color are in the same cluster.

Fig. 6. The plot displays sorted log of FN activity time versus FN index fj . The colored lines in the legends indicate the log of FN activity time T ½fj .
Dots at the peak indicate the log of fog layer activity time TF . log ð0Þ is marked as 2 in these plots.

increased participation of the FNs marked by the flatness of REFERENCES

the curves in Fig. 6 for the cases of OCOP algorithm. [1] M. Mukherjee, L. Shu, and D. Wang, “Survey of fog computing:
Fundamental, network applications, and research challenges,”
IEEE Commun. Surv. Tut., vol. 20, no. 3, pp. 1826–1857, Jul.–Sep.
2018.
6 CONCLUSION [2] M. Aazam, S. Zeadally, and K. A. Harras, “Offloading in fog com-
puting for IoT: Review, enabling technologies, and research oppor-
In this paper, a delay aware energy efficient optimal clustering tunities,” Future Gener. Comput. Syst., vol. 87, pp. 278–289, 2018.
and offloading scheme called OCOP has been presented. We [3] Y. Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, “A survey on
have shown that the explicit clustering approach and simulta- mobile edge computing: The communication perspective,” IEEE
neous decision of the corresponding best offloading parame- Commun. Surv. Tut., vol. 19, no. 4, pp. 2322–2358, Oct.–Dec. 2017.
[4] K. Sood, S. Yu, and Y. Xiang, “Software-defined wireless network-
ters significantly reduce the fog layer activity time and the ing opportunities and challenges for Internet-of-Things: A review,”
average energy levels in the FNs and TNs. We have also shown IEEE Internet Things J., vol. 3, no. 4, pp. 453–463, Aug. 2016.
that its time complexity is much lesser than the general qua- [5] A. Hakiri, P. Berthou, A. Gokhale, and S. Abdellatif, “Publish/
dratic case. The results of extensive simulations have concurred subscribe-enabled software defined networking for efficient and
scalable IoT communications,” IEEE Commun. Mag., vol. 53, no. 9,
that there is a steep fall of about 97.5% and 74% on an average pp. 48–54, Sep. 2015.
in the activity time and combined energy levels respectively [6] M. Chen and Y. Hao, “Task offloading for mobile edge computing
with respect to the FEMTO algorithm. The proposed OCOP in software defined ultra-dense network,” IEEE J. Sel. Areas Com-
mun., vol. 36, no. 3, pp. 587–597, Mar. 2018.
algorithm has a drawback that it frequently violates the
[7] S. Luo, X. Chen, Z. Zhou, X. Chen, and W. Wu, “Incentive-aware
allowed delay in the case of serial processing of FNs. Our future micro computing cluster formation for cooperative fog computing,”
work will circumvent this issue by incorporating hybrid paral- IEEE Trans. Wireless Commun., vol. 19, no. 4, pp. 2643–2657, Apr. 2020.
lel and serial processing schemes among FNs such that time [8] J. Wang, K. Zhu, B. Chen, and Z. Han, “Distributed clustering-
based cooperative vehicular edge computing for real-time offload-
sensitive tasks are offloaded to parallel processing FNs in the ing requests,” IEEE Trans. Veh. Technol., vol. 71, no. 1, pp. 653–669,
fog layer and the rest to the serial processing ones. Jan. 2022.

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY WARANGAL. Downloaded on January 02,2024 at 06:52:34 UTC from IEEE Xplore. Restrictions apply.
 YADAV ET AL.: CLUSTERING-BASED ENERGY EFFICIENT TASK OFFLOADING FOR SUSTAINABLE FOG COMPUTING 67

[9] R. Dautov and S. Distefano, “Stream processing on clustered edge [31] G. Lee, W. Saad, and M. Bennis, “An online secretary framework
devices,” IEEE Trans. Cloud Comput., vol. 10, no. 2, pp. 885–898, for fog network formation with minimal latency,” in Proc. IEEE
Apr.–Jun. 2022. Int. Conf. Commun., 2017, pp. 1–6.
[10] A. Asensio et al. “Designing an efficient clustering strategy for [32] M. Sniedovich, Dynamic Programming: Foundations and Principles,
combined fog-to-cloud scenarios,” Future Gener. Comput. Syst., 2nd ed. Boca Raton, FL, USA: CRC Press, 2010. [Online]. Avail-
vol. 109, pp. 392–406, 2020. able: https://books.google.co.in/books?id¼KtSVtDs5yFYC
[11] A. Bozorgchenani, D. Tarchi, and G. E. Corazza, “An energy-aware [33] R. H. Byrd, P. Lu, J. Nocedal, and C. Zhu, “A limited memory
offloading clustering approach (EAOCA) in fog computing,” in Proc. algorithm for bound constrained optimization,” SIAM J. Sci. Com-
Int. Symp. Wireless Commun. Syst., 2017, pp. 390–395. put., vol. 16, no. 5, pp. 1190–1208, 1995.
[12] G. Zhang, F. Shen, Z. Liu, Y. Yang, K. Wang, and M. Zhou, [34] J. Nocedal and S. J. Wright, Numerical Optimization. New York,
“FEMTO: Fair and energy-minimized task offloading for fog- NY, USA: Springer, 2006.
enabled IoT networks,” IEEE Internet Things J., vol. 6, no. 3, [35] P. Kuila and P. K. Jana, “Energy efficient clustering and routing
pp. 4388–4400, Jun. 2019. algorithms for wireless sensor networks: Particle swarm optimi-
[13] A. Akbar, M. Ibrar, M. A. Jan, A. K. Bashir, and L. Wang, “SDN- zation approach,” Eng. Appl. Artif. Intell., vol. 33, pp. 127–140,
enabled adaptive and reliable communication in IoT-fog environ- 2014.
ment using machine learning and multiobjective optimization,” [36] Y. del Valle, G. K. Venayagamoorthy, S. Mohagheghi, J.-C. Her-
IEEE Internet Things J., vol. 8, no. 5, pp. 3057–3065, Mar. 2021. nandez, and R. G. Harley, “Particle swarm optimization: Basic
[14] Y. Zhai et al., “An energy aware offloading scheme for interdepen- concepts, variants and applications in power systems,” IEEE
dent applications in software-defined IoV with fog computing Trans. Evol. Comput., vol. 12, no. 2, pp. 171–195, Apr. 2008.
architecture,” IEEE Trans. Intell. Transp. Syst., vol. 22, no. 6,
pp. 3813–3823, Jun. 2021. Anirudh Yadav received the BTech degree in
[15] S. Misra and S. Bera, “Soft-VAN: Mobility-aware task offloading computer science and engineering from the Indian
in software-defined vehicular network,” IEEE Trans. Veh. Technol., Institute of Technology (Indian School of Mines)
vol. 69, no. 2, pp. 2071–2078, Feb. 2020. Dhanbad, India, in 2020. He is currently working
[16] M. Aazam, S. U. Islam, S. T. Lone, and A. Abbas, “Cloud of Things toward the MS degree in physics qunatum com-
(CoT): Cloud-fog-IoT task offloading for sustainable Internet of puting with the University of Wisconsin-Madison,
Things,” IEEE Trans. Sustain. Comput., vol. 7, no. 1, pp. 87–98, Madison, Wisconsin. His current research inter-
Jan.–Mar. 2022. ests include fog computing, quantum computing,
[17] S. Chen, X. Zhu, H. Zhang, C. Zhao, G. Yang, and K. Wang, and machine learning.
“Efficient privacy preserving data collection and computation off-
loading for fog-assisted IoT,” IEEE Trans. Sustain. Comput., vol. 5,
no. 4, pp. 526–540, Oct.–Dec. 2020.
[18] C. Zhu et al. “Folo: Latency and quality optimized task allocation
in vehicular fog computing,” IEEE Internet Things J., vol. 6, no. 3,
pp. 4150–4161, Jun. 2019. Prasanta K. Jana (Senior Member, IEEE) received
[19] Y. Yang, Z. Liu, X. Yang, K. Wang, X. Hong, and X. Ge, “POMT: the MTech degree in computer science from the Uni-
Paired offloading of multiple tasks in heterogeneous fog networks,” versity of Calcutta, Kolkata, India, in 1988, and the
IEEE Internet Things J., vol. 6, no. 5, pp. 8658–8669, Oct. 2019. PhD degree from Jadavpur University, Kolkata,
[20] S. Misra and N. Saha, “Detour: Dynamic task offloading in soft- India, in 2000. He is currently a professor with the
ware-defined fog for IoT applications,” IEEE J. Sel. Areas Commun., Department of Computer Science and Engineering,
vol. 37, no. 5, pp. 1159–1166, May 2019. Indian Institute of Technology (Indian School of
[21] Y. Zu, F. Shen, F. Yan, L. Shen, F. Qin, and R. Yang, “SMETO: Sta- Mines), Dhanbad, India. He has contributed 199
ble matching for energy-minimized task offloading in cloud-fog research publications, coauthored five books, three
networks,” in Proc. IEEE 90th Veh. Technol. Conf., 2019, pp. 1–5. book chapters and produced 15 PhDs. As a recogni-
[22] M. Mukherjee, M. Guo, J. Lloret, R. Iqbal, and Q. Zhang, tion of his outstanding research contributions, he
“Deadline-aware fair scheduling for offloaded tasks in fog com- has been awarded Canara Bank Research Publication Award in the year
puting with inter-fog dependency,” IEEE Commun. Lett., vol. 24, 2015 and 2017. His current research interests include wireless sensor net-
no. 2, pp. 307–311, Feb. 2020. works, cloud computing, fog computing, and machine learning.
[23] Z. Liu, X. Yang, Y. Yang, K. Wang, and G. Mao, “DATS: Disper-
sive stable task scheduling in heterogeneous fog networks,” IEEE
Internet Things J., vol. 6, no. 2, pp. 3423–3436, Apr. 2019. Shashank Tiwari received the BTech degree in
[24] Y. Yang, S. Zhao, W. Zhang, Y. Chen, X. Luo, and J. Wang, “DEBTS: computer science and engineering from the Indian
Delay energy balanced task scheduling in homogeneous fog Institute of Technology (Indian School of Mines),
networks,” IEEE Internet of Things J., vol. 5, no. 3, pp. 2094–2106, Dhanbad, India, in 2020. He is currently working as
Jun. 2018. a software development engineer (SDE) with Sam-
[25] G. Zhang, F. Shen, N. Chen, P. Zhu, X. Dai, and Y. Yang, “DOTS: sung Noida. His research interests include areas of
Delay-optimal task scheduling among voluntary nodes in fog fog computing, reinforcement learning, natural lan-
networks,” IEEE Internet of Things J., vol. 6, no. 2, pp. 3533–3544, guage processing, and deep learning.
Apr. 2019.
[26] Y. Yang, K. Wang, G. Zhang, X. Chen, X. Luo, and M.-T. Zhou,
“MEETS: Maximal energy efficient task scheduling in homogeneous
fog networks,” IEEE Internet Things J., vol. 5, no. 5, pp. 4076–4087,
Oct. 2018.
[27] Z. Zhu, T. Liu, Y. Yang, and X. Luo, “BLOT: Bandit learning-based Abhay Gaur ireceived the BTech and MTech
offloading of tasks in fog-enabled networks,” IEEE Trans. Parallel degrees in computer science and engineering
Distrib. Syst., vol. 30, no. 12, pp. 2636–2649, Dec. 2019. from the Indian Institute of Technology (Indian
[28] M. Mukherjee et al. “Task data offloading and resource allocation School of Mines) Dhanbad, India, in 2021. His
in fog computing with multi-task delay guarantee,” IEEE Access, current research interests include fog computing,
vol. 7, pp. 152 911–152 918, 2019. deep learning architectures, and reinforcement
[29] J. Du, L. Zhao, J. Feng, and X. Chu, “Computation offloading and learning.
resource allocation in mixed fog/cloud computing systems with
min-max fairness guarantee,” IEEE Trans. Commun., vol. 66, no. 4,
pp. 1594–1608, Apr. 2018.
[30] Y. Liu, F. R. Yu, X. Li, H. Ji, and V. C. M. Leung, “Distributed
resource allocation and computation offloading in fog and cloud " For more information on this or any other computing topic,
networks with non-orthogonal multiple access,” IEEE Trans. Veh.
please visit our Digital Library at www.computer.org/csdl.
Technol., vol. 67, no. 12, pp. 12 137–12 151, Dec. 2018.

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY WARANGAL. Downloaded on January 02,2024 at 06:52:34 UTC from IEEE Xplore. Restrictions apply.