IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING, VOL. 11, NO.

2, MARCH/APRIL 2024 2223

Towards Dynamic Request Updating With Elastic

Scheduling for Multi-Tenant Cloud-Based
Data Center Network
Shuaibing Lu , Member, IEEE, Jie Wu , Fellow, IEEE, Jiamei Shi, Juan Fang , Member, IEEE, Jiayue Zhang ,
and Haiming Liu

Abstract—Serving the ever-growing demand for computation,

storage, and networking resources for multi-tenant in cloud com-
puting is an important mission of Data Center Networks (DCNs).
In this paper, we study the dynamic request updating problem,
and our objective is to maximize the elasticity of cloud-based
DCNs while achieving rapid response to multi-tenants. We use
virtual clusters under the hose communication model to denote
requests. Instead of using heuristic algorithms as the existing work
does, this paper introduces a novel two-stage dynamic request
updating framework with elastic resource scheduling strategy. In
the first stage, we propose a multi-tenant fast initial provisioning
scheme to realize the real-time response and analyze its optimality
and complexity. Additionally, we provide a deep reinforcement
learning-based dynamic updating strategy to enhance the elasticity
of virtual clusters that are being used or scaling during the second Fig. 1. Illustration of the dynamic request updating in multi-tenant cloud-
stage. We train a fully connected neural network by creating a based DCN.
new feasible action set to realize the reduction, and it approxi-
mates the policy based on a proposed aggressive objective selection
method to improve training speed while avoiding high dimensions fundamental challenge of DCNs is to serve the varying needs of
caused by large scales of tenants and DCNs. Extensive evaluations multi-tenant without requiring frequent provisioning changes.
demonstrate that our scheme outperforms baselines in terms of This paper proposes an elastic resource provisioning scheme to
both elasticity and efficiency.
deal with the scaling without load redistribution during a run
Index Terms—Data center network, dynamic request updating, time. In order to simplify the description of the resource provi-
elastic scheduling, multi-tenant, resource provisioning. sioning problem, we use virtual clusters to denote the requests
of multi-tenant, and each virtual cluster is an abstraction of a set
I. INTRODUCTION of virtual machines (VMs), which has the requirements on both
computing and communication resources [3]. We use the notion
ITH the ever-increasing demand of cloud services, the
W data center network (DCN) has become an efficient and
promising data processing infrastructure for cloud computing.
of elasticity to measure the potential growth of multi-tenant in
terms of computing and communication resources at the same
time, which is defined as the degree of a system that is able to
As reported in the public data of Microsoft Azure [1], the adapt to the workload changes by provisioning and releasing
demand and deployment size of the tenant is very bursty and resources in an autonomic manner [5], [6], [8]. We consider
unpredictable in terms of memory, cores, and bandwidth. One the virtual clusters with hose model under constraints, and our
objective is to maximize the elasticity while achieving rapid
Manuscript received 22 May 2023; revised 21 October 2023; accepted 8 response to multi-tenants in the DCNs.
December 2023. Date of publication 12 December 2023; date of current version
23 February 2024. This work was supported in part by the Fundamental Research We give an example that illustrate our work in this paper, some
Funds for the Central Universities under Grant 2021RC258, in part by the China assumptions and notations are not explicitly stated and will be
Postdoctoral Science Foundation under Grant 2021M700366, and in part by the explained in a later section. As shown in Fig. 1, we consider a
National Natural Science Foundation under Grant 92267107. Recommended for
acceptance by Dr. H. Mohsenian-Rad. (Corresponding author: Haiming Liu.) multi-level tree-based network with m physical machines as the
Shuaibing Lu, Jiamei Shi, Juan Fang, and Jiayue Zhang are with the Fac- data center architecture. The capacity of each physical machine
ulty of Information Technology, Beijing University of Technology, Beijing is slotted, and each slot can only host one virtual machine. We
100124, China (e-mail: [email protected]; [email protected];
[email protected]; [email protected]). assume that several virtual clusters are running in the operating
Jie Wu is with the Department of Computer and Information Sciences, Temple data center, which are distributed on different servers resulting in
University, Philadelphia, PA 19122 USA (e-mail: [email protected]). variations in the remaining resources. We separate the scenarios
Haiming Liu is with the School of Software Engineering, Beijing Jiaotong
University, Beijing 100091, China (e-mail: [email protected]). of resource requesting of virtual clusters into two categories
Digital Object Identifier 10.1109/TNSE.2023.3341907 which are initial provisioning for new newcomers and dynamic
2327-4697 © 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: Sri Krishna College Of Engineering & Technology. Downloaded on May 21,2024 at 16:41:57 UTC from IEEE Xplore. Restrictions apply.
 2224 IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING, VOL. 11, NO. 2, MARCH/APRIL 2024

updating for those already in use. Here, we desire to find a r We conduct various evaluations with several state-of-the-
provisioning strategy for virtual clusters that achieving quick art algorithms under different topologies on the basic set-
response while also supporting maximum elasticity without ting that refers to the observations. The results are shown
resorting to reassignment. This problem is non-trivial due to the from different perspectives to provide conclusions.
following unique challenges: (i) It is nontrivial to implement fast The remainder of this paper is organized as follows. Section II
provisioning and offer a quick response to numerous customers surveys related works. Section III describes the model, problem
when several newly arrived virtual clusters arrive simultaneously formulation, and motivation. Section IV investigates the prob-
(# of n). (ii) The initial position is vital for the flexible growth lem by proposing an efficient framework. Section V presents the
of subsequent resources for virtual clusters. It is nontrivial evaluations. Section VI concludes the paper.
to improve their elasticity under the high dimensions caused
by the concurrent requests from enormous amount of virtual II. RELATED WORK
clusters and the huge volume of the cloud-based data center. (iii)
Additionally, due to fluctuations in demand, tenants who have Recently, the main research point of virtual cluster provision-
already provisioned resources in the data center may request ing in DCNs includes reliability, energy consumption, traffic
resource scaling during the operation. For instance, in Fig. 1, changing, and congestion control. However, with the explosion
tenant V1 initially requests 2 VMs, but when the demands varies of increasing types and scales of the requests by users, the
over the span of the operating period, the number scales to 4. It problem of elasticity in the data center has also been focused.
is challenging to deal with the dynamic scaling of requests that The solutions are mainly divided into three categories introduced
can realize adaptability. in the following.
In this paper, we introduce a novel dynamic updating frame-
work with elastic resource scheduling. Instead of using heuristic A. Elastic Scheduling by Extending Resources
algorithms or simple deep reinforcement learning as the existing With the explosion of increasing types and scales of requests
work does, we deal with the resource provisioning problem of by users, the problem of elasticity in the DCN has been focused.
multi-tenant and realize the dynamic adjustment for the elastic There have been a few recent work on elastic resource provision-
updating requests at the same time, which consists of two stages. ing by extending physical resources. Rui et al. [7] and Naskos
In the first stage, we try to find a feasible provisioning scheme et al. [9] showed a probability model and a cost-aware method
for the multi-tenant with a rapid response by relaxing partial to analyze the bottleneck in multi-layer cloud applications, and
constraints. In the second stage, we attempt to updating the they proposed a method to meet the elastic scaling of the data
virtual clusters of multi-tenant that have been provisioned by center. Farokhis et al. [10] designed a two-layer traffic-aware
using deep reinforcement learning, so as to improve the elastic- transmission algorithm, which can effectively solve the problem
ity. However, due to the large scale of the scenario, we need to of virtual machine placement and ensure the large-scale elastic
design an efficient solution that improving the training speed of scaling of potential user resources. Lin et al. [11] provided a
the neural network in high-dimensional space. Our contributions unified framework that integrates the representation of the logic
can be summarized as follows: graphs to maintain regular and reliable operation of data center
r We investigate the virtual clusters provisioning problem
networks and transmit data between servers. Fan et al. [12]
in multi-tenant cloud-based DCNs with hose model, and presented an adaptive path-finding algorithm for establishing
we propose to maximize the elasticity by considering the virtual links between any two nodes in the data center network.
limitation on computation and communication resources. Wang et al. [13] realized the elastic scaling of cloud-based
r We make a theoretical and experimental study of the com-
DCN by adjusting the size of CPU and memory. Chowdhury
monly used methods that are appropriate for provisioning et al. [14] analyzed the problem of elastic management based
of virtual clusters, and we analyze the insights that produce on virtual cluster service, separated resource allocation from
high complexity and slow convergence. service management, and provided the ability of elastic service
r We introduce a novel dynamic updating framework with
to adapt to dynamic workload changes. The above works realized
elastic scheduling that make it possible for multi-tenant the elastic resource provisioning by adding new instances (VMs,
cloud-based DCNs to provide scalable resources in two containers or application instance modules, etc.) or adjusting
stages. We construct a heuristic rapid provisioning scheme the size of the instance in itself during the runtime, which can
in the first stage to realize the real-time response to multi- solve the problem of the insufficient physical resources caused
tenant virtual clusters, and we prove the optimality under by the dynamic scaling. However, it is difficult to avoid the low
the single computation resource constraint. utilization and high cost caused by the uncertainty of requesting
r Based on that, we present an online dynamic updating
types and scales.
method based on deep reinforcement learning to enhance
the adaptability of virtual clusters that are running or scaled
during the second stage. In order to avoid the high dimen- B. Elastic Scheduling by Designing Heuristic Strategies
sions caused by the large scales of tenants and the DCNs, Quite a few works have been carried out on the elastic resource
we train a fully connected neural network by creating provisioning problem by designing heuristic strategies. Alfonso
a new feasible action set to realize the reduction and it et al. [15] proposed an open-source virtual cluster framework
approximates the policy based on a proposed aggressive based on the DCN, which analyzed the dynamic changes of
objective selection method to improve training speed. virtual clusters in the running process to minimize the cluster
Authorized licensed use limited to: Sri Krishna College Of Engineering & Technology. Downloaded on May 21,2024 at 16:41:57 UTC from IEEE Xplore. Restrictions apply.
 LU et al.: TOWARDS DYNAMIC REQUEST UPDATING WITH ELASTIC SCHEDULING 2225

consumption and meet the computing demands of users. Kholidy TABLE I

SYMBOLS AND DEFINITIONS
et al. [16] and Guerrero et al. [17] developed a prediction method
based on swarm intelligence, which realized the optimal alloca-
tion of multi-dimensional resources requested by tenants using
particle swarm optimization algorithm and genetic algorithm.
Qaddoum et al. [18] proposed an elastic resource scheduling
strategy based on load prediction, and solved the problem of
computing load fluctuation in Big Data streaming computing
platforms by designing a resource allocation method based on a
modified adaptive neural network. Wang et al. [19] presented an
elastic resource provisioning scheme to help service providers
pay less cost with users’ QoS guarantee in clouds. Gao et al. [30]
put forward a congestion-aware scheduling scheme to determine
the priority of flows based on the latest network congestion in
data center networks. Tran et al. [31] designed heuristic traffic-
aware virtual network function placement and migration algo- Ying et al. [27] proposed a scheduling program based on deep
rithms to minimize the total network traffic in policy-preserving reinforcement learning, which used a cross-entropy method to
data centers. Fei et al. [32] present a method of elastic resource train a fully connected neural network to realize VM migration
provisioning using data clustering in cloud service platform. before resource updating. Due to the complexities of applica-
Mina et al. [33] proposed a non dominated set construction tions and the scaling of DCNs may generate huge state and
algorithm based on task fluctuation to realize the multi-objective action spaces, applying deep reinforcement learning methods
dynamic scheduling strategy of elastic cloud resources. Most directly on the virtual cluster provisioning for multi-tenant will
of the above researches adopt centralized deployment, which result in a slow convergence rate or even be unable to converge.
prefers to use heuristic or meta-heuristic algorithms to construct
resource allocation architecture and realizes elastic allocation of III. MODEL AND PROBLEM FORMULATION
cloud data center resources by software architecture adjustment.
In this paper, we study the dynamic virtual cluster updat-
ing problem in multi-tenant cloud-based data center network
C. Elastic Scheduling Based on Reinforcement Learning through online scheduling, which attempts to realize rapid re-
Reinforcement learning (RL), as one of the paradigms and spond and high elasticity. In this section, we start with the
methodologies of machine learning, uses agent learning strate- descriptions of data center model and the virtual cluster model.
gies in the interaction of the environment to achieve specific The problem is also formulated, and the symbols used in this
goals [20], [29], [34], [35]. Deep reinforcement learning com- paper are summarized in Table I.
bines the perception ability of deep learning with the decision-
making ability of reinforcement learning. In recent years, deep A. Data Center Network Model
reinforcement learning has been successfully applied in the In this paper, the substrate topology of the data center network
game, robot control, simulation, optimization, and schedul- is defined as tree-structured, i.e. G = {C, L}. Let C denote
ing [21]. There are many applications of the resource provision- the set of physical machines, where C = {Ci }. We use |C| to
ing problem in the cloud data centers. Bitsakos et al. [22] used represent the total number of physical machines in G. Here,
the deep Q-network algorithm of deep reinforcement learning we choose Ci to denote the ith machine. The capacity of Ci is
to optimize the allocation of elastic resources in DCN with ci , and the remaining resources is ĉi . Based on our data center
current multidimensional states of virtual clusters as the input. topological structure, we use L to denote the set of physical links,
Liang et al. [23] proposed a method based on advantage actor where L = {Lij }. Here, Lij denotes the jth link on the ith level,
critical deep reinforcement learning, which effectively updates and the capacity of Lij is lij in G. For each link being used,
parameters by designing an adaptive scheduling algorithm to we use ˆlij to represent the remaining available communication
realize the resource provisioning in the cloud-based DCNs. Liu resources.
et al. [24] developed a reinforcement learning-based framework
that adopts neural networks with an overall consideration of
B. Virtual Clusters
data movement and analytical latency, and they trained with a
variant of q-learning to solve the data placement problem. Nouris We use virtual clusters to denote the demands of multi-tenant,
et al. [25] proposed and implemented a controller based on RL, which is denoted by V = {Vi }. Each virtual cluster represents
which can not only realize the rapid expansion of resources in the the requirements on both computing and communications re-
cloud data center, but also save costs by shutting down redundant source which consists of a set of virtual machines (VMs) and
servers. Chen et al. [26] developed a two-set deep reinforcement one virtual switch [3], where Vk = {vk(h) }. Let vk(h) denote
learning system to solve the problem of traffic elastic expansion the hth VM of virtual cluster Vk . We use |Vk | to denote the
in the cloud data center. By collecting flow information on the total number of VMs in Vk , and we suppose that each VM
terminal host, the data center traffic can be controlled online. has Bk Gbps unit bandwidth demand between VMs and the

Authorized licensed use limited to: Sri Krishna College Of Engineering & Technology. Downloaded on May 21,2024 at 16:41:57 UTC from IEEE Xplore. Restrictions apply.
 2226 IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING, VOL. 11, NO. 2, MARCH/APRIL 2024

virtual switch of virtual cluster Vk . The total  number of VMs

in set V is denoted as |V|, where |V| = nk=1 |Vk |. In this
paper, we consider the virtual clusters with hose model which
means that each customer specifies a set of endpoints to be
connected with a common endpoint-to-endpoint performance
guarantee [5]. Virtual clusters are independent from one another
and only interact with the inside VMs, there is no interaction
between external virtual clusters. Here, we use communication
demand to indicate the maximum bandwidth resources that each
VM may simultaneously communicate with the other virtual
machines in its own cluster. The communication demand of these Fig. 2. Illustration of a slow convergence condition based on a straightforward
components will be 0 Gbps since VMs that are housed on the deep reinforcement learning method.
same physical machine do not use link bandwidth resources
when communicate with one another. We use f (Vk , Ci ) to
indicate the communication demand of VMs provisioning on ĉi ≤ ci and ˆlij ≤ lij (4)
machine Ci for virtual cluster Vk which deployed across differ-  
ent machines. For each virtual cluster, we use a boolean ˆlij = f (Vk , Ci ) (5)
 variable
τvCk(h)
i
to indicate that whether vk(h) placed on Ci . h∈Vk τvCk(h)i
Ci ∈C(Lij )Vk ∈Ci
denotes the number of VMs belonging to Ci . Therefore, we have  
 
that the
total communication demand of Vk on Ci is f (Vk , Ci ) = f (Vk , Ci ) = min τvCk(h)
i
, |Vk |− τvCk(h)
i
· Bk
min{ h∈Vk τvCk(h) i
, |Vk | − h∈Vk τvCk(h)
i
} · Bk . Taking V3 as an h∈Vk h∈Vk
example shown in Fig. 1, which is made up of four VMs, each (6)
VM in V3 is able to communicate with the other three. We sup-
posed that each VM has a B3 = 1Gbps unit bandwidth demand
D. Motivation
of V3 , then f (V3 , Cm−1 ) = min{2, (4 − 2)} = 2 · B3 = 2 Gbps
is the result. 1) High Time Complexity Under DP: Based on the problem
formulation in (1) to (4), the numbers of variables and constraints
C. Problem Formulation are large which means that the virtual cluster provisioning
problem in multi-tenant DCNs cannot be efficiently solved by
This paper focuses on the virtual cluster provisioning problem the simplex or eclipse methods. A dynamic programming (DP)
in multi-tenant DCNs. We use E to denote the combinational scheme has been proposed in [3], and the time complexity is
elasticity of the DCN, which is expressed as a percentage and |V|
proved to be O(2h · j=1 (|Vj | + 1)h−j ), where h is the height
used to represent the usage of the minimum remaining resources, of the DCN. We take the data center of Alibaba [4] as an example,
which is shown in (1). We aim at minimizing the combinational which is constructed by 4034 physical machines. We suppose
elasticity by considering both computational and communica- that there are only two tenants with two virtual clusters, and
tion resources. Here we use Em to represent the elasticity of the each one only contains a single VM. According to the calculation
physical machines, which results from calculating the minimum method, the time complexity under DP will be higher than 24036 ,
scalable capacity of the physical machines. Similarly, we use El which is huge.
to represent the elasticity of physical links which is shown in (2). 2) Slow Convergence Under High Dimension: To avoid
In (3), we use λ and ϕ to represent coefficients reflecting different the high complexity brought by the DP method, we
computational and communication resource requirements. The consider to use deep reinforcement learning, such as deep
constraints during the virtual cluster provisioning are shown in Q-learning (DQN), to solve the virtual cluster provisioning
(4) and (5), which means that the total consumption of computa- problem in multi-tenant DCNs. We suppose the state
tion and communication resources cannot exceed the capacities space st=[ĉ1 , ĉ2 , . . ., ĉi , . . ., ĉm , |V̂1 |, |V̂2 |, . . ., |V̂j |, . . ., |V̂n |],
of physical machines and links. Here, (5) is the communication
 where ĉi denotes the remaining resource of ith physical
demands of virtual clusters on link lij , where h∈Vk τvCk(h) i
machine and |V̂j | denotes the number of remaining
denotes the number of VMs belonging to Ci . Equation (6) is VMs of jth virtual cluster. The action space is at =
the communication demand of the virtual cluster Vk on Ci . We [s1(1)| , . . ., s1(m) , s2(1) , . . ., s2(m) , . . ., sn(1) , . . ., sn(m) ], where
use the notion of elasticity to measure the potential growth of si(j) is a boolean variable that denotes whether the VM
multi-tenant in terms of both computing and communication provisioning on the jth physical machine.
resources [5], [6]. The problem formulation is shown as follows:  Based onthat, the
maximum times of calculation is C( nj=1 |V̂j |, m i=1 ĉi ).
maximize E = min{λEm , ϕEl } (1) The reward mechanism is defined in Algorithm 2, which is
  r = (E(St+1 ) − Ē)/(Ē + ξ). We take 6-layer topology as an
  ˆlij
ĉi example as shown in the sub-figure (a) of Fig. 2, and the capacity
s.t. Em = min 1 − , El = min 1 − (2) of the physical machine is defined as ci = 8. We suppose that
i ci i,j lij
there are only two virtual clusters V1 and V2 , where the number
0 ≤ λ ≤ 1 and 0 ≤ ϕ ≤ 1 (3) of VMs are |V1 | = 5 and |V2 | = 5, respectively. The result is

Authorized licensed use limited to: Sri Krishna College Of Engineering & Technology. Downloaded on May 21,2024 at 16:41:57 UTC from IEEE Xplore. Restrictions apply.
 LU et al.: TOWARDS DYNAMIC REQUEST UPDATING WITH ELASTIC SCHEDULING 2227

Fig. 3. Overview of DUES.

shown in Fig. 2(a). We can see that the elasticity of the DCN B. Stage 1: Fast Initial Provisioning
is close to convergence when the number of iterations reaches
In this subsection, we introduce a multi-tenant fast initial
104 . In addition, the range of the elasticity fluctuates greatly
provisioning scheme (MFIP) for virtual clusters which is shown
in the iterations from 6 × 103 to 104 . We enlarge the value
in Algorithm 1. The insight of our scheme is to identify the
of elasticity within 300 iterations in Fig. 2(b), which has no
partition for virtual clusters based on the computing resource
tendency of convergence and fluctuates sharply. Thus, we can
of the DCN. The input in Algorithm 1 is the set of virtual
see that the convergence time will be extremely slow if the deep
clusters V, and the output is the fast provisioning scheme Xf .
reinforcement learning method is used to directly search for In lines 1 and 2, we first check the feasibility of virtual clusters
and learn the best solution of the virtual cluster provisioning by comparing |V| with ĉ. Here, we use ĉ to represent the
problem for multi-tenant.
total remaining resources where ĉ = m i=1 ĉi . If the remain-
ing computing resources can accommodate the virtual clusters
IV. DYNAMIC UPDATING FRAMEWORK WITH ELASTIC of V, where |V| ≤ ĉ, Algorithm 1 continues. Otherwise, the
SCHEDULING requests of set V will be rejected. In lines 3 to 5, we start to
In this section, we show the detail of our novel online dynamic calculate the estimated number of accommodation based on the
updating framework with elastic scheduling (DUES) which computing capacities of each server in G. Here, we introduce a
constructed by two stages to realize the rapid response and high new definition of the estimated divided factor.
elasticity. Definition 1 (Estimated divided factor):  Let δi denote the
estimated divided factor of Ci and δi = ĉi / m i=1 ĉi , where ĉi
A. Overview denotes the rest available physical resources.
In line 3, we first initialize the group partition with estimated
The main idea of DUES is to realize real-time response to divided factors. Then we calculate the capacity of each group,
multi-tenant requests for provisioning and upgrading while max- which is the maximum amount of provisioning VMs on server
imizing the elasticity of the cloud-based DCN. The overview i, i.e, gi = δi · |V|. The value of gi is an integer that rounds
of DUES, which comprises of two stages, is shown in Fig. 3. down with gi = δi · |V| to avoid overflowing, which involves
In the first stage, we propose a heuristic scheme to realize the reducing to the nearest integer even if the fractional part is larger
fast provisioning for multi-tenant and analyze its optimality and than or equal to 0.5. We suppose that if an upward value or
complexity. We take the arriving requests of multi-tenant virtual rounding method is used to obtain the value of gi , it is possible
clusters as the input, and the output is the initial provisioning that the total available resources on the servers of the groups
scheme which also converts to the input of the second stage. will be higher than the total number of requests, resulting in
In the second stage, we propose a online dynamic updating an overflow error where the free position information exceeds
strategy based on deep reinforcement learning to improve the the total number of requests. Here, we highlight the potential
combinational elasticity of the cloud-based DCN. Since the main impact through using a straightforward example. We suppose
drawback of the simple deep reinforcement learning method is that there are only 3 available servers left in the data center,
that it will result in huge state and action spaces when applied and each has ĉ1 = 2, ĉ2 = 10, and ĉ3 = 8 remaining resources,
to the virtual cluster provisioning problem, we introduce a new respectively. We assume that the total amount of requests is
definition which is the feasible action set. Based on that, we train |V| = 15 altogether of virtual clusters at
a fully connected neural network to realize the reduction and it this time, we will
obtain g1 = 2, g2 = 8, and g3 = 6, where i=3 i=1 gi = 16 > |V|
approximates the policy based on a proposed aggressive objec- regardless of whether rounding up or normal rounding is used,
tive selection method to improve training speed. The detailed which results in overflow. In line 6, we update the number of
description are shown as follows.
Authorized licensed use limited to: Sri Krishna College Of Engineering & Technology. Downloaded on May 21,2024 at 16:41:57 UTC from IEEE Xplore. Restrictions apply.
 2228 IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING, VOL. 11, NO. 2, MARCH/APRIL 2024


VMs of multi-tenant virtual clusters |V| = |V| − m i=1 gi . In Algorithm 1: Multi-tenant Fast Initial Provisioning Scheme
lines 7 to 9, we resize some groups for the rest of the queries (MFIP).
in V. If |V| = 0, which means there are remaining VMs that
Input: Set of multi-tenant requests V;
cannot be covered, we choose the physical machine with the
Output: Fast initial provisioning scheme Xf ;
maximum available resources by arg maxCi ∈C ĉi and increase
1: if |V| < ĉ then
the estimated number of accommodations by gi = gi + 1 in
2: return False;
lines 8 and 9. Then, we update the total number of requests
3: for each server in G do
in line 10. In line 11, we update groups in set g with descending
4: Initialize group partition with the estimated divided
order based on their sizes, where g := descending(g). Then
factor δi ;
we start the provisioning process, which chooses request Vj
5: Calculate the capacity of each group gi ;
with maximum arg maxVj ∈V Nj and matches Vj into group
6: Update the virtual clusters |V| := |V| − m i=1 gi ;
gi . If the number of VMs |Vj | is over the estimated number
7: while |V| = 0 do
of accommodation in gi , we place a part of VMs according to
8: Choose the physical machine with arg maxCi ∈C ĉi ;
the size of gi . Then we update gi = 0 and remove gi from set
9: Update gi := gi + 1;
g. Otherwise, we update set g with gi = gi − |Vj |. After that,
10: Update |V| := |V| − 1;
we update g := descending(g) in line 19. Line 20 returns the
11: Update g := descending(g);
provisioning scheme Xf . The time complexity of Algorithm 1
12: for each group gi in G do
is O(m2 · |V|).
13: Choose request Vj with arg maxVj ∈V |Vj |;
Theorem 1: The total communication demand of virtual clus-
14: Matching Vj into group gi ;
ters V with MFIP is minimum in G under the single constraint
15: if |Vj | > gi then
ĉi ≤ ci .
16: |Vj | := |Vj | − gi ;
Proof: There are two steps in MFIP, which are initializing
17: Update gi := 0 and remove gi from set g;
the estimated groups of G and the identifying partition for
18: else
multiple virtual clusters. In the first step, the groups are estimated
19: Update set g with gi := gi − |Vj |;
by considering converting multiple virtual clusters to a single
20: Update g := descending(g);
one. The partition of each group is found by calculating the
21: return Initial Provisioning Xf ;
capacities based on the physical machines in G that is optimal,
which has been proved in [28]. Thus, we only need to prove
that the identifying partition for V obtained the minimum total
two cases (iii) (min{|Vi | − |gi |, |gi |} = |gi | under MFIP and
communication demand. We suppose that the set of estimated
min{|Vi | − |gj |, |gj |} = |gj |) and (iv) (min{|Vi | − |gi |, |gi |} =
groups g = {gi } has been updated with descending order. If the
|gi | under MFIP and min{|Vi | − |gj |, |gj |} = |Vi | − |gj |), the
demands of all virtual clusters in V with the same order are lower
proof process is the same as (a) and (b). In summary, the total
than groups in g, the total communication demand of virtual
communication demand of virtual clusters V with MFIP is
clusters V will be 0 which is minimum. If there is existing Vi ⊆
minimum under the single constraint ĉi ≤ ci . 
V larger than the size of group gi , where |Vi | > |gi |, Vi ⊆ V
will be divided into several parts. We discuss the case as follows.
C. Stage 2: Online Dynamic Updating Strategy
We suppose that the fast provisioning Xf is (Vi → gi )|Vi |>|gi | ,
the total communication demand is min{|Vi | − |gi |, |gi |}. As- In this subsection, we propose an online dynamic updating
suming that the provisioning with minimum communication de- strategy based on deep reinforcement learning. The main idea
mand (MCD) of Vi is group gj , then we have (Vi → gj )|Vj |>|gj | . is to realize the dynamic updating requests from multi-tenants
The total communication demand will be min{|Vi | − |gj |, |gj |}. inside each equal time slot after dividing continuous time into
Here, we prove by contradiction which assumes that |gi | > |gj |, equal slices. This process intends to identify the bottleneck of
then we have four possible scenarios. (i) min{|Vi | − |gi |, |gi |} = the DCN for each time slot in the context of the fast initial
|Vi | − |gi | under MFIP and min{|Vi | − |gj |, |gj |} = |Vi | − |gj | provisioning Xf and choose VMs to readjust based on an
under MCD. Since we suppose MCD has the minimum com- aggressive upgrading strategy for the objective selection. Here,
munication demand, then we have that |Vi | − |gj | < |Vi | − |gi |, it is worth noting that when a request is produced at a certain
i.e. |gi | > |gj |, which contradicts with the assumption |gj | < time slot, it is necessary to verify beforehand whether the tenant
|gi |. (ii) min{|Vi | − |gi |, |gi |} = |Vi | − |gi | under MFIP and is already located in the data center. If the tenant is already in
min{|Vi | − |gj |, |gj |} = |gj | under MCD. Suppose the group the data center, it goes directly to the second stage of dynamic
that provisioning the Vi \ gj request is gk , where the rest capacity online updating; if not, it must go through the first stage of fast
|Vi | − |gi | < |ĝk | < |gi |. The total communication demand will initial provisioning. We define the bottleneck as follows.
be 2(|Vi | − |gi |). Since we suppose MCD has the minimum com- Definition 2 (bottleneck): The bottleneck b∗ is a vector rep-
munication demand where |gj | < |Vi | − |gi |, then we have that resenting the location of physical machine or link with the
|Vi | − |gj | > gi . Since |ĝk | < |gi |, we have |ĝk | < |gi | < |Vi | − minimum elasticity of G.
|gj |. The MCD will be at least 2|gj | + 2|gk |, however, |Vi | − The core component of the agent is to design a policy where
|gi | < |ĝk |, then we have 2|gj | + 2|gk | > 2|gj | + 2(|Vi | − |gi |), it provides the probability distribution over the action space a
which contradict with the assumption. For the remaining and the state space s.

Authorized licensed use limited to: Sri Krishna College Of Engineering & Technology. Downloaded on May 21,2024 at 16:41:57 UTC from IEEE Xplore. Restrictions apply.
 LU et al.: TOWARDS DYNAMIC REQUEST UPDATING WITH ELASTIC SCHEDULING 2229

Algorithm 2: Reward Updating Mechanism. Algorithm 3: Feasible Actions Set Construction Function.
Input: Variable num; Input: Fast initial provisioning Xf ;
Output: Reward r(st , at ) and termination variable Γ; Output: Feasible actions set Â;
1: if E(st+1 ) ≤ E(st ) then 1: Calculate E under the fast initial provisioning Xf ;
2: num := num + 1; 2: Find the location of the bottleneck with minimum
3: if E(st+1 ) > E(st ) then elasticity;
4: r := (E(st+1 ) − Ē)/(Ē + ξ); 3: if E = Em Ci
then
5: else if E(st+1 ) = E(st ) then 4: Remove the bottleneck Ci from set  = {A/Ci };
6: r := E(st+1 ) − E(st ); 5: else
7: else 6: Remove the physical machines in set C(Lij ) under the
8: r := −1; bottleneck link Lij from set  = {A/C(Lij )};
9: if num ≥ Ψ then 7: return Feasible Actions Set Â;
10: Γ := ture;
11: else
12: Γ := f alse;
13: return Reward r(st , at ) and termination variable Γ; decreases after adjusting the VMs in one episode. The output of
this function is the reward after choosing action at and the value
of the termination variable Γ. In line 1, we first compare the value
1) Deep Reinforcement Learning Formulation: In order to of elasticity under the state st and st+1 . If E(st+1 ) ≤ E(st ), it
describe the environment of the DCN concisely and correctly means that the elasticity after choosing action at will decrease,
for the agent, the state space should include the knowledge of we will record it using num := num + 1. After that, we start to
the usage on the physical resources, the status of requests from define the reward updating mechanism under different cases. In
multiple tenants, and the information of the updating virtual lines 3 to 4, when there is an increment that E(st+1 ) > E(st ),
cluster. So the state is designed as follows. the reward will be defined as r := (E(st+1 ) − Ē)/(Ē + ξ),
Definition 3 (state): The state st is a vector consisting of where Ē is the baseline elasticity after the fast provisioning
st = [h1 , h2 , . . ., hm ]t , where hi denotes the provisioning list Xf . ξ is a factor that avoid the denominator obtaining zero,
on server ci at time slot t. The provisioning list hi records the where 0 < ξ ≤ 1. In lines 5 to 6, if the E(st+1 ) = E(st ), the
number of virtual clusters placed on this server, where hi = reward will be defined as r := 0. Otherwise, the reward will be
[v1 , v2 , vi , . . ., vk ]. defined as r := −1 under the E(st+1 ) > E(st ) case in lines
We consider realizing the dynamic updating by training the 7 to 8. Since the reward of one action cannot determine the
agent which needs to choose a destination physical machine for final result, the reward value r := −1 cannot represent that
each adjusting VM. The action at is designed as follows. the total provisioning order which is bad. However, if the bad
Definition 4 (action): The action space at = [C1 , C2 , . . . , cases continue to happen, which means that the agent always
Cm ]t is the updating action, where Ci = 0 or Ci = 1 means chooses the action with r := −1, this episode will be terminated
that the target location of adjustment is on Ci or not at time slot when num ≥ Ψ. Here, we use Ψ to denote a threshold that is
t. determined by the structure of the DCN, which is less than or
The objective of the agent is to find a provisioning scheme equal to the number of physical machines, i.e., Ψ ≤ |C|. Line
for multi-tenants that maximizes the elasticity of the DCN. For 13 returns the reward r(st , at ) and the termination variable Γ.
each episode, we decide the provisioning of VMs for the tenants 2) Construction of Feasible Actions Set: In order to reduce
by choosing an action. After that, the agent will get a reward the high dimensions caused by the large scales of tenants and
r(st , at ) at time slot t with state st after executing action at . the DCNs, we introduce a definition of the feasible action set.
In our problem, the value of this reward cannot determine the Definition 6 (Feasible Actions Set): Let  indicate the feasi-
final elasticity until all requests of tenants are provisioned. The ble action set of V, which only include the target VMs that have
reason is that the virtual cluster only communicate with VMs on positive effect on optimize the combinational elasticity of G.
their own, which means that although we make the adjustment Based on that, we further propose a method to construct a
decision for VMs one by one, the final elasticity is determined feasible action set Â. The main idea is to remove the invalid
until all virtual clusters finish provisioning. Here is the specific optional targets which exist under the bottleneck to improve the
definition. learning speed of the agent. The detail description is shown in
Definition 5 (Reward): The reward r is decided by the value Algorithm 3. In line 1, we first calculate the elasticity E under
of elasticity which defined in three cases, where the fast initial provisioning Xf , and we find the location of
⎧ bottleneck. If the bottleneck is located on the physical machine
⎪
⎨(E(st+1 ) − Ē)/(Ē + ξ) E(st+1 ) > E(st ) E = Em Ci
, where Em Ci Ci
is the elasticity of Ci , i.e., Em = 1 − cĉii .
r := E(st+1 ) − E(st ) E(st+1 ) = E(st ) (7) Then, we remove the optional target Ci from set A, where  =
⎪
⎩
−1 E(st+1 ) < E(st ) {A/Ci }. If the bottleneck is located on the physical link E =
ij ij ij l̂
The updating mechanism of the reward shown in Algorithm 2, ElL , where ElL is the elasticity of Lij , i.e., ElL = 1 − lij
ij
.
we initialize a variable num to record the times that the elasticity Then, we remove set CLij under the bottleneck link Lij from

Authorized licensed use limited to: Sri Krishna College Of Engineering & Technology. Downloaded on May 21,2024 at 16:41:57 UTC from IEEE Xplore. Restrictions apply.
 2230 IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING, VOL. 11, NO. 2, MARCH/APRIL 2024

Algorithm 4: Aggressive Objective Selection Algorithm. Algorithm 5: Dynamic Updating based on Deep Reinforce-
Input: State s under the fast initial provisioning Xf ; ment Learning (DU-DRL).
Output: VM that needs to be adjusted vk(h) ; Input: Set of updating requests V ;
1: Same as Algorithm 3 in lines 1 to 2; Output: Provisioning scheme X;
Ci
2: if E = Em then 1: Initialize D to N , Γ to f alse, Q with random weights θ,
3: Select VM vk with the maximum communication and Q̂ with weights θ− := θ;
demand on Ci to adjust; 2: for episode from 1 to κ do
4: else 3: Initialize sequence s based on Xf of Algorithm 1;
5: Choose the physical machine Cw in set C(Lij ) under 4: Preprocessed sequence φ1 = φ(sXf );
ij
bottleneck link ElL with minimum elasticity Em Cw
; 5: while Γ = f alse do
6: Select VM vk(h) with the maximum communication 6: Build feasible actions set  based on Algorithm 3;
demand on Cw to adjust; 7: With probability ε select a random action at ∈ Â;
7: return VM that needs to be adjusted vk(h) ; 8: Otherwise select at = argmaxa Q(φ(st ), a; θ);
9: Choose the adjusted objective based on Algorithm 4;
10: Execute action at in emulator and update r and Γ
based on Algorithm 2;
set  = {A/C(Lij )}. Here, we use C(Lij ) to denote the set
11: Set st+1 = st , at , xt+1 and preprocess
of physical machines under the physical link Lij . Line 7 returns
φt+1 = φ(St+1 ).
the feasible action set Â. 12: Store transition (φt , at , rt , φt+1 ) in D;
3) Aggressive Objective Selection: In a given episode, the 13: Sample random minibatch of transitions
agent choose an action from the set  that is detailed in (φj , aj , rj , φj+1 ) from D.
Algorithm 3. This action can only determine the destination 14: if episode terminates at step j + 1 then
that we can adjust the VMs, however, which VM is selected 15: Set yj = rj ;
to be adjusted cannot be determined by the action at . Here, 16: else
we design an aggressive adjusted objective selection algorithm 17: Set yj = rj + γmaxa Q̂(φj+1 , a ; θ− );
to recognize which VM is being adjusted based on its current 18: Perform a gradient descent step on
policy. Then the environment will return a reward rt to the agent (y − Q(φj , aj ; θ))2 with respect to the parameters
and transit to st+1 . In Algorithm 4, the input is the state s under θ.
the fast initial provisioning Xf and the output is the VM vk 19: Every C steps reset Q̂ = Q;
that is selected to be adjusted. In lines 1 to 2, we initialize 20: return Provisioning scheme X;
the elasticity E under the fast provisioning Xf and find the
bottleneck same as Algorithm 3. If the bottleneck is located
Ci
on Ci where E = Em , we select VM vk with the maximum
communication demand on Ci to adjust in line 3. Otherwise, if Q with random weight θ and the target action-value function Q̂
ij
the bottleneck is located on Lij where E = ElL , we choose the with weights θ− = θ. In lines 2 to 4, we start to train the agent by
physical machine Cw under bottleneck link Lij with minimum running a number of κ episodes with our environment. During
elasticity EmCw
in line 5. Based on that, we select VM vk with each episode, we initialize sequence S based on the fast initial
the maximum communication demand on Cw to adjust. Line 7 provisioning Xf and preprocess it with φ1 = φ(sXf ) in lines 3
returns the VM vk that needs to be adjusted. to 4. The training process starts from lines 5 to 20. The process of
the adjustment starts from choosing a physical machine from the
built feasible actions set  that is produced by Algorithm 3. In
D. Dynamic Updating Based on Deep Reinforcement
line 7, the agent selects a random action at ∈ Â with probability
Learning (DU-DRL) ε, otherwise, it will select at = argmaxa Q(φ(st ), a; θ) with the
The overview of the dynamic updating strategy based on deep maximum Q value in line 8. Since there is a queue of VMs from
reinforcement learning (DU-DRL) is shown in the right part of different virtual clusters provisioning on the chosen physical
Fig. 3. Algorithm 5 summarizes the specific steps. The main machine, the agent needs to choose the adjusted objective based
idea is to use a deep reinforcement learning agent to perform the on Algorithm 4 in line 9. At each time step, only one of the VMs
dynamic adjustment VM of virtual clusters to maximize the elas- in this queue is adjusted. Then the agent executes action at in
ticity of the DCN in each time slot. Before we conduct dynamic the emulator and updates r and Γ based on Algorithm 2. We
updates, we collect the status of multi-tenant virtual clusters V set st+1 = st , at , xt+1 , and preprocess φt+1 = φ(st+1 ), and
to construct the set of update requests V . We prioritize pro- we store the transition (φt , at , rt , φt+1 ) in the replay memory
cessing for tenants who requesting resource release V− . Then, D in line 12. After that, we sample a random minibatch of
we reinitialize the resources of cloud-based DCN and update transitions (φj , aj , rj , φj+1 ) from D. In lines 14 to 17, the agent
V = {V − V− } which serves as input of Algorithm 5. We first will calculate the reward after the termination of the episode.
initialize some preliminary parameters which include setting the The objective of our problem is to maximize the elasticity of the
replay memory D to capacity N and episode terminated variable DCN which is consistent with the cumulative reward received
Γ to f alse. Meanwhile, we initialize the action-value function by the agent. In line 18 to 19, the agent performs a gradient

Authorized licensed use limited to: Sri Krishna College Of Engineering & Technology. Downloaded on May 21,2024 at 16:41:57 UTC from IEEE Xplore. Restrictions apply.
 LU et al.: TOWARDS DYNAMIC REQUEST UPDATING WITH ELASTIC SCHEDULING 2231

TABLE II TABLE III

STEP-BY-STEP CALCULATION FOR THE EXAMPLE OF DUES HYPERPARAMETER SETTINGS

descent step on (y − Q(φj , aj ; θ))2 with respect to the network

parameters θ, and resets Q̂ = Q every C steps. The results are
returned in line 21. DUES fully considers the tight coupling
between virtual machine placement and bandwidth resource al- conditions in our basic setting consider the observations from
location for the requests of multi-tenant. When there are resource analyzing the public data of Azure [1]. The number of VMs
scaling from tenants running in the data center, our approach can in each virtual cluster ranges in [10, 20], and the core of VM
eliminate erroneous optional targets that seem plausible but are is set to be 1. We consider the number of users ranging from
prone to elastic bottlenecks by building up the set of feasible [60, 90]. The settings of hyper parameters are listed in Table III.
actions, which reduces the dimension of the search space and We test several groups of hyper parameters which include the
accelerates the learning rate of the agent. In addition, the strategy learning rate α = {0.0001, 0.0005, 0.001}, and the e-greedy
proposed in our framework enables the agents to accommodate = {0.95, 0.96, 0.97, 0.98} and so as to others. We choose the
workload fluctuations by recognizing which virtual machine is group of hyper parameters listed in Table III as the experimental
being adjusted based on its current policy, which results in the setting. In addition to the proposed scheduling algorithms, five
rapid allocation of resources under scaling. We use an example state-of-the-art algorithms are used, simple DQN [29], DU-DRL
to explain the details of DUES. We suppose that the topology with Random Provisioning (DU-RP), DU-DRL with Equally
of DCN is 3-layer with four physical machines. The remaining Distributed Provisioning (DU-EDP) [5], DU-DRL with Greedy
computation resource (C1 to C4 ) are ĉ1 = 2, ĉ2 = 5, ĉ3 = 7, and Provisioning (DU-GP), and DQN only with Stage 1 (DQN-S1).
ĉ4 = 10. The remaining communication resource of the physical r DU-RP: virtual clusters of multi-tenant are random dis-
links (L1,1 to L2,4 ) are ˆl1,1 = 15, ˆl1,2 = 15, ˆl2,1 = 5, ˆl2,2 = 10, tributed during the fast initial provisioning stage, and then
ˆl2,3 = 5, ˆl2,4 = 1. There are four tenants with virtual clusters V1 adaptively updated based on the DU-DRL.
to V4 , where |V1 | = 3, |V2 | = 2, |V3 | = 4, |V4 | = 1. According r DU-EDP: virtual clusters of multi-tenant are equally dis-
to the Algorithm 1, the fast provisioning Xf is shown in the first tributed based on the algorithm proposed in [5] during the
row of Table II, and the elasticity EXf = 0. The upper link of fast initial provisioning stage, and then adaptively updated
C4 is the bottleneck which is highlighted with underline. Then based on the DU-DRL.
we construct the feasible action set based on Algorithm 3, where r DU-GP: virtual clusters of multi-tenant are greedy provi-
 = C1 , C2 , C3 . We adjust the VMs based on Algorithm 5, and sioned based on the capacities of physical machines in the
the provisioning X is shown in the second row of Table II. DCN during the fast initial provisioning stage, and then
adaptively updated based on the DU-DRL.
V. EVALUATIONS r DQN-S1: virtual clusters of multi-tenant are provisioned
In this section, we conduct extensive simulations and ex- based on MFIP, and updated based on simple DQN.
periments to study the elastic virtual cluster provisioning in
multi-tenant DCNs. We develop a prototype of our algorithms B. Experiment Results of Baseline Algorithms
using python, which consists of the construction of the data
We first conducted a thorough analysis of the first stage and
center and the requests of multiple tenants. After presenting
verified its effectiveness in achieving optimal results within
the datasets and settings, the results are shown from different
its given constraints under the DCN with topologies k = 6,
perspectives to provide insightful conclusions.
k = 7, k = 8, and k = 9. The results are shown in Fig. 4. In the
first stage, we have the following observations: (i). The MFIP
A. Basic Setting algorithm has the greatest elasticity value under different topo-
We use python to build our prototype on workstation Precision logical configurations in the first stage. Among the comparison
T7910 with Intel Xeon(R) E5-2620 CPU, NVIDIA RTX5000 algorithms, the GP algorithm has the worst performance, mainly
GPU, 128Gb memory, and 2T b hard disk on Linux operating because the greedy algorithm selects one of the computational
system. We simulate a DCN of k-level binary tree topology, and communication resources in each decision process without
where k ∈ {6, 7, 8, 9}. Each physical machine resource is di- considering the bottlenecks and trade-offs. By comparison, the
vided into slots, and each slot can only accommodate one virtual results of EDP and Random algorithms are relatively better than
machine, which can be easily interpreted to a real configuration. GP, while the EDP relies on the distribution of the remaining
The physical machines have slot capacities ranging from 0 to resources of the underlying physical servers, and the Random
100, and each of them has a 1 Gbps link to connect with a algorithm depends on the placement position of the servers
higher layer switch. Every switch is connected by a higher selected by the randomized method each time slot. (ii). The
layer that has double the communication capacity of the current MFIP algorithm has the maximum elasticity value in the first
layer until it reaches the core. The deployments and workload stage for various sets of user requests with the same topology.

Authorized licensed use limited to: Sri Krishna College Of Engineering & Technology. Downloaded on May 21,2024 at 16:41:57 UTC from IEEE Xplore. Restrictions apply.
 2232 IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING, VOL. 11, NO. 2, MARCH/APRIL 2024

Fig. 4. Elasticities of different DCNs with tenants ranging from 60 to 90 (first-stage).

Fig. 5. Elasticities of 6-layer DCNs with tenants ranging from 60 to 90.

Since the physical resources are comparatively adequate when does not perform well in the first stage when k = 6, but as k
the scale of the data center is large (k = 9), different provisioning increases, the elasticity values show better results, and the trend
strategies will not have a significant impact on elasticity, and the in the second stage mirrors that of the first stage and relies on
advantage of MFIP on the elasticity is not particularly obvious the results. In addition, the DU-GP strategy could cause some
when the request size is modest (60, 70). However, the limited crucial resources (such as certain computing or communication
multi-dimensional resources (k = 6) might result in significant resources) to become scarce in the first phase. Such resource
differences in the elasticities across various solutions as the shortages could intensify in the second phase and lead to lower
number of user requests increases (60 to 90). elasticity values. Based on the dependence between the second
Based on the results of the first stage, we evaluate the elasticity stage and the first stage, we can clearly see that the performance
value of the second stage as follows. We deploy the algorithms of the second stage is closely related to the resource allocation
on tree topology with 6 to 9-layers under algorithms (DQN, strategy of the first stage. Efficient fast provisioning strategies
DU-EDP, DU-GP, DQN-S1, DUES) on each group of datasets for the multi-tenant will make great improvements to the final
and calculate the elasticities within 300 iterations. Among them, result, on the contrary, inefficient ones will bring bad effects.
DU-RP, DU-EDP, and DU-GP are based on the first-stage (iii). The algorithm DUES has the highest elasticity values
Random, EDP, and GP methods, respectively. We conducted among different topological configurations in the second stage.
experiments on the elasticity values of data centers with different Among the comparison algorithms, DU-EDP performs poorly
topologies (k = 6, k = 7, k = 8, and k = 9). The experiment when the topology is k = 6 and k = 7, but its performance
results are shown in Figs. 5 to 8, and we have the following improves when the topology is k = 8 and k = 9 as shown in
observations: (i). For the same group of users, the elasticity Figs. 7 and 8. This could be because the DU-EDP strategy can
values obtained by different algorithms are quite different. better balance resource allocation in larger topologies, thereby
Fig. 5(a) shows the elasticities under the five algorithms for avoiding resource bottlenecks that occur in smaller topologies.
the number of 60 tenants with the 6-layer DCN. The first three The algorithm DU-GP performs consistently poorly because
columns show the elasticities under the DQN, DU-EDP, and it prioritizes allocations based only on the elasticity values of
DU-GP, which are all negative values. It means that these three the computational resources in each decision process, without
algorithms do not give an appropriate solution for the virtual considering bottlenecks and tradeoffs. The performance of the
clusters of these 60 tenants. (ii). The choice of strategy used DU-RP algorithm varies significantly due to the randomness of
for the fast provisioning has an important influence on the final the algorithm, leading to inconsistent results. DU-DRL has an
elasticity value. As shown in Fig. 5(a), DU-EDP and DU-GP are efficient effect on improving the elasticity, especially when the
the strategies that adding EDP and GP fast provisioning based occupancy of physical resources is not close to saturation. Com-
on DQN. The final elasticity values under these two strategies paring sub-figures (a) and (d) of Fig. 5, the difference between
are lower than simple DQN. However, the elasticity under the the last two columns of 60 tenants is much higher than 90. The
DUES strategy is the highest. Therefore, we have that the choice average resource utilization of the DCNs with 60 and 90 tenants
of strategies using for fast provisioning is very important for is nearly 80% and 98% under the DUES. Then, the number
the elasticity of the DCNs. For example, the strategy DU-RP of tenants is large but not reaching saturation, and the impacts

Authorized licensed use limited to: Sri Krishna College Of Engineering & Technology. Downloaded on May 21,2024 at 16:41:57 UTC from IEEE Xplore. Restrictions apply.
 LU et al.: TOWARDS DYNAMIC REQUEST UPDATING WITH ELASTIC SCHEDULING 2233

Fig. 6. Elasticities of 7-layer DCNs with tenants ranging from 60 to 90.

Fig. 7. Elasticities of 8-layer DCNs with tenants ranging from 60 to 90.

Fig. 8. Elasticities of 9-layer DCNs with tenants ranging from 60 to 90.

of algorithms on the elasticities are higher. (iv). The trends of 1) Convergence: We investigate the convergence for the
a different group of multiple tenants with the same physical groups of tenants (60, 70, 80, and 90) under physical topologies
topology are various. Compared with sub-figures (a) to (d) of with different layers (7-layer, 8-layer, and 9-layer). The results
Figs. 5 to 8, we can see that a larger number of tenants in one are shown in Figs. 9 to 11. The gray parts are the ranges of
group will lead to lower elasticity for the same physical topology. the collection results, and the bright lines with different colors
The reason is that more virtual clusters demand more physical are the mean values. Additionally, we have the following obser-
resources, which will lead to an increase in the combinational vations: (i). The increasing number of tenants has an influence
utilization of the clusters and thus lower the elasticity of the on the convergence. As shown in Fig. 9, the elasticity of the
data centers. Thus, a good provisioning scheme can support 7-layer data center is scaling with the increasing number of
more virtual clusters of tenants in the larger data center net- iterations. For each group of tenants, the elasticity begins to
work. In summary, compared with DQN, DU-EDP, DU-GP, and converge between 50 and 100 iterations and keeps at a high
DQN-S1, DUES has better performance in elasticity across the level of around 300 iterations. The growth rate is slow when the
virtual cluster provisioning in multi-tenant DCNs. The average number of users is small, i.e. sub-figures (a) and (b) of Fig. 9,
elasticity can improve 1.91 times compared with DQN-S1 under while it is relatively fast when the number of users is scaling.
the ranges of tenants [60, 90]. The reason is that the higher number of the total virtual clusters
provisioning in the data center greatly decreases the rest of the
available physical resources, which leads to a reduction of the
searching space. (ii). The elasticity fluctuates within a relatively
C. Experiment Results Under Different DCNs fixed range. There are many different placement results in the
Based on the experiment results of baseline algorithms under learning process of DUES, and the elasticities generated by these
the 6-layer DCN, we conduct the experiments of DUES to results will fluctuate among several relatively fixed values in the
compare the elasticities under different topologies. For each convergence process. As shown in the sub-figures (a), (b), and
group of users, we collect five groups of results under the same (c) of Fig. 9, the fluctuation of elasticities is within the range
settings. between 0.1 and 0.2. Compared with 8-layer and 9-layer, the

Authorized licensed use limited to: Sri Krishna College Of Engineering & Technology. Downloaded on May 21,2024 at 16:41:57 UTC from IEEE Xplore. Restrictions apply.
 2234 IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING, VOL. 11, NO. 2, MARCH/APRIL 2024

Fig. 9. Elasticity under 7-layer topology.

Fig. 10. Elasticity under 8-layer topology.

Fig. 11. Elasticity under 9-layer topology.

Fig. 12. Elasticities under different DCNs.

fluctuation of elasticities is within the range of 0.1 and 0.12, assess the average of the highest elasticities among the five
respectively. This range is correlated to the topology of the DCN groups of results with (λ = 0.5, ϕ = 0.5) which are shown in
and the provisioning deviation of a few individual VMs. Fig. 12. Additionally, we have the following observations: (i).
2) Elasticity: According to the convergence of the elastic- The final elasticities decrease with the increasing number of
ities obtained under the topologies with different layers, we tenants under the same t opology. As shown in sub-figure (a)

Authorized licensed use limited to: Sri Krishna College Of Engineering & Technology. Downloaded on May 21,2024 at 16:41:57 UTC from IEEE Xplore. Restrictions apply.
 LU et al.: TOWARDS DYNAMIC REQUEST UPDATING WITH ELASTIC SCHEDULING 2235

Fig. 14. Elasticities of the different scaling requirements in extreme

scenarios.
Fig. 13. Average elasticity and descending rate (%) of DCNs.

TABLE IV the current number of tenants in the data center is 90, which
COEFFICIENT SETTING can be represented as (90, 1). The result is presented as fol-
lows. It is obvious that there are differences in the results of
elasticity at different coefficients, and the result is shown in
Fig. 14(a). Among them, the elasticity value is highest under
the coefficient (0.4, 0.6), which indicates that the expansion of
of Fig. 12, the increasing rates between the number of tenants this group of queries is focused on physical machines and the
and the elasticities are similar, which is not the case in the other subsequent scalability advantage is in communication resources.
three groups. Such as the 7-layer topology group, the gap of We then evaluate the scenario where only one virtual cluster has
the elasticity between 70 and 80 is not large, however, when a scaling requirement. Since only the internal resources within
the number of tenants increases to 90, the value of the elasticity one cluster have grown, the communication resources account
decreases sharply. Since the sizes of the requested virtual clusters for the main part where scaling in [45, 90] Gbps based on the
ranges in [10, 20], the total amount of resources requested of function f (Vk , Ci ). Here, we assume that the current cluster
the group with a lower number of tenants may be close to the resource growth requirement is 90, which can be represented as
higher one, which leads to a not large gap of these groups. (ii). (1, 90). The result is shown in Fig. 14(b). Among them, the elas-
With the scaling of the topologies of the DCNs, the impacts ticity value is the highest under the coefficient (0.3, 0.7), which
of algorithms on the elasticities are higher. However, when the indicates that the expansion of this group of queries is focused
resources are sufficient to a certain level, the effect of updating on physical links and the subsequent scalability advantage is in
strategy on elasticity may be small as shown in sub-figure (d) of computing resources. Based on the above result, we have that
9-layer topology. Based on the experiment results of benchmarks the values of coefficients can more accurately reflect the various
under the 6-layer DCN, the elasticity depends on the localities of requirements for different resource categories and optimize the
VMs requesting from different virtual clusters, which means a allocation by introducing coefficients to weigh the importance
provisioning scheme can support more virtual clusters in a larger of Em and El .
DCN. Compared with the same column on Y -axis with sub-
figures (a) to (d) of Fig. 12, the elasticities are increasing with
the scaling topologies of the DCNs, which means more available VI. CONCLUSION
resources can be supported by the providers. Additionally, we In this paper, we address the virtual cluster provisioning
estimate the descent rates of several DCNs and compare the problem in multi-tenant cloud data centers. We use elasticity
average elasticities; the results are shown in Fig. 13. We see that to measure the potential growth of multi-tenant in terms of
the fluctuations in elasticities decrease sharply when the scales of computing and communication resources. We aim to minimize
the topologies are 6 and 7 layers. However, the trend is flat when the elasticity by designing a two-stage framework DUES, which
the scales of the topologies increase to 8 and 9 layers. Thus, we consists of two stages. In the first stage, we first propose a fast
find that the fluctuation rates are inversely proportional to the initial provisioning MFIP scheme to realize the rapid response
scaling of the topologies. In summary, DUES in multi-tenant of multi-tenant, and we prove that MFIP is optimal under the
DCNs shows better performance in terms of elasticity at different single computation resource constrain. In the second stage, we
topologies. propose a dynamic updating strategy DU-DRL based on deep
3) Coefficient: In order to further analyze the impact of the reinforcement learning to further improve the elasticity of virtual
coefficient on the combinational elasticity, we experimented on clusters that are in use for scaling. Additionally, to avoid the
two distinct scenarios to confirm the efficacy of the introducing high dimensions caused by the large scales of tenants and the
coefficients, and we used λ and ϕ to represent coefficients DCN, we propose to train a fully connected neural network by
of physical machines and links, respectively. The setting of designing a new feasible action set to realize the reduction, and
coefficients is shown in Table IV. We first evaluate the scenario it approximates the policy based on the proposed aggressive
where only compute resources are increased and users in the objective selection method in DU-DRL. Finally, we conduct
data center request an increase of 1 virtual machine. Due to the extensive evaluations under various scenarios to demonstrate
scaling of virtual machines between different virtual clusters, that our scheme outperforms existing state-of-the-art methods
the communication between them is 0. Here, we assume that in terms of both elasticity and efficiency.

Authorized licensed use limited to: Sri Krishna College Of Engineering & Technology. Downloaded on May 21,2024 at 16:41:57 UTC from IEEE Xplore. Restrictions apply.
 2236 IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING, VOL. 11, NO. 2, MARCH/APRIL 2024

REFERENCES [25] S. M. R. Nouri, H. Li, S. Venugopal, W. Guo, M. He, and W. Tian,

“Autonomic decentralized elasticity based on a reinforcement learning
[1] O. Hadary et al., “Protean:VM allocation service at scale,” in Proc. 14th controller for cloud applications,” Future Gener. Comput. Syst., vol. 94,
USENIX Symp. Operating Syst. Des. Implementation, 2020, pp. 845–861. pp. 765–780, 2019.
[2] D. Yi, X. Zhou, Y. Wen, and R. Tan, “Efficient compute-intensive job [26] L. Chen, J. Lingys, K. Chen, and F. Liu, “Auto: Scaling deep reinforcement
allocation in data centers via deep reinforcement learning,” IEEE Trans. learning for datacenter-scale automatic traffic optimization,” in Proc. Conf.
Parallel Distrib. Syst., vol. 31, no. 6, pp. 1474–1485, Jun. 2020. ACM Special Int. Group Data Commun., 2018, pp. 191–205.
[3] S. Lu, J. Wu, and Z. Fang, “High-elasticity virtual cluster placement [27] C. Ying, B. Li, X. Ke, and L. Guo, “Raven: Scheduling virtual machine mi-
in multi-tenant cloud data centers,” in Proc. IEEE 21st Int. Conf. High gration during datacenter upgrades with reinforcement learning,” Mobile
Perform. Comput. Commun.; IEEE 17th Int. Conf. Smart City; IEEE 5th Netw. Appl., vol. 27, pp. 303–314, 2022.
Int. Conf. Data Sci. Syst., 2019, pp. 996–1003. [28] S. B. Lu, J. Wu, H. Y. Zheng, and Z. Y. Fang, “On maximum elastic
[4] H. Wu et al., “Aladdin: Optimized maximum flow management for shared scheduling in cloud-based data center networks for virtual machines with
production clusters,” in Proc. IEEE Int. Parallel Distrib. Process. Symp., the hose model,” J. Comput. Sci. Technol., vol. 34, no. 1, pp. 185–206,
2019, pp. 696–707. 2019.
[5] J. Wu, S. Lu, and H. Zheng, “On maximum elastic scheduling of virtual [29] V. Mnih et al., “Human-level control through deep reinforcement learning,”
machines for cloud-based data center networks,” in Proc. IEEE Int. Conf. Nature, vol. 518, no. 7540, pp. 529–533, 2015.
Commun., 2018, pp. 1–6. [30] C. Gao, S. Chu, H. Xu, M. Xu, K. Ye, and C. Z. Xu, “Flash: Joint flow
[6] D. C. Plummer et al., “Study: Five refining attributes of public and private scheduling and congestion control in data center networks,” IEEE Trans.
cloud computing,” Gartner, Stamford, CT, USA, Tech. Rep. 167182(5), Cloud Comput., vol. 11, no. 1, pp. 1038–1049, Jan.–Mar. 2023.
2009. [31] V. Tran, J. Sun, B. Tang, and D. Pan, “Traffic-optimal virtual network
[7] R. Han, M. M. Ghanem, L. Guo, Y. Guo, and M. Osmond, “Enabling function placement and migration in dynamic cloud data centers,” in Proc.
cost-aware and adaptive elasticity of multi-tier cloud applications,” Future IEEE Int. Parallel Distrib. Process. Symp., 2022, pp. 919–929.
Gener. Comput. Syst., vol. 32, pp. 82–98, 2014. [32] B. Fei, X. Zhu, D. Liu, J. Chen, W. Bao, and L. Liu, “Elastic resource
[8] K. T. Foerster and S. Schmid, “Survey of reconfigurable data center provisioning using data clustering in cloud service platform,” IEEE Trans.
networks: Enablers, algorithms, complexity,” ACM SIGACT News, vol. 50, Serv. Comput., vol. 15, no. 3, pp. 1578–1591, May/Jun. 2022.
no. 2, pp. 62–79, 2019. [33] M. Yazdanbakhsh, R. K. M. Isfahani, and M. Ramezanpour, “MODE: A
[9] A. Naskos et al., “Dependable horizontal scaling based on probabilistic multi-objective strategy for dynamic task scheduling through elastic cloud
model checking,” in Proc. IEEE/ACM 15th Int. Symp. Cluster, Cloud, Grid resources,” Majlesi J. Elect. Eng., vol. 14, no. 2, pp. 127–141, 2020.
Comput., 2015, pp. 31–40. [34] Y. Bao, Y. Peng, and C. Wu, “Deep learning-based job placement in
[10] S. Farokhi, P. Jamshidi, E. B. Lakew, I. Brandic, and E. Elmroth, “A distributed machine learning clusters,” in Proc. IEEE Conf. Comput.
hybrid cloud controller for vertical memory elasticity: A control-theoretic Commun., 2019, pp. 505–513.
approach,” Future Gener. Comput. Syst., vol. 65, pp. 57–72, 2016. [35] K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath,
[11] W. Lin, X. Y. Li, J. M. Chang, and X. Jia, “Constructing multiple CISTs on “Deep reinforcement learning: A brief survey,” IEEE Signal Process. Mag.,
BCube-Based data center networks in the occurrence of switch failures,” vol. 34, no. 6, pp. 26–38, Nov. 2017.
IEEE Trans. Comput., vol. 72, no. 7, pp. 1971–1984, Jul. 2023.
[12] W. Fan, F. Xiao, H. Cai, X. Chen, and S. Yu, “Disjoint paths construction Shuaibing Lu (Member, IEEE) received the Ph.D.
and fault-tolerant routing in BCube of data center networks,” IEEE Trans. degree in computer science and technology from
Comput., vol. 72, no. 9, pp. 2467–2481, Sep. 2023. Jilin University, Changchun, China, in 2019. She is
[13] Y. Wang, C. C. Tan, and N. Mi, “Using elasticity to improve inline currently a Lecturer with the Faculty of Information
data deduplication storage systems,” in Proc. IEEE 7th Int. Conf. Cloud Technology, Beijing University of Technology, Bei-
Comput., 2014, pp. 785–792. jing, China. She was supported by the China Schol-
[14] M. Chowdhury, Z. Liu, A. Ghodsi, and I. Stoica, “HUG: Multi-resource arship Council as a Visiting Scholar supervised by
fairness for correlated and elastic demands,” in Proc. 13th USENIX Symp. Prof. Jie Wu in the Department of Computer and In-
Netw. Syst. Des. Implementation, 2016, pp. 407–424. formation Sciences, Temple University, Philadelphia,
[15] C. de Alfonso, M. Caballer, A. Calatrava, G. Moltó, and I. Blanquer, PA, USA, from 2016 to 2018. Her research interests
“Multi-elastic datacenters: Auto-scaled virtual clusters on energy-aware include distributed computing, cloud computing, and
physical infrastructures,” J. Grid Comput., vol. 17, no. 1, pp. 191–204, fog computing.
2019.
[16] H. A. Kholidy, “An intelligent swarm based prediction approach for
predicting cloud computing user resource needs,” Comput. Commun.,
vol. 151, pp. 133–144, 2020. Jie Wu (Fellow, IEEE) is currently the Director with
[17] C. Guerrero, I. Lera, and C. Juiz, “Genetic algorithm for multi-objective the Center for Networked Computing and Laura H.
optimization of container allocation in cloud architecture,” J. Grid Com- Carnell Professor, Temple University, Philadelphia,
put., vol. 16, no. 1, pp. 113–135, 2018. PA, USA. He is also the Director of international
[18] K. S. Qaddoum, N. N. El Emam, and M. A. Abualhaj, “Elastic neural affairs with the College of Science and Technology.
network method for load prediction in cloud computing grid,” Int. J. Elect. He was the Chair of Department of Computer and
Comput. Eng., vol. 9, no. 2, 2019, Art. no. 1201. Information Sciences from the summer of 2009 to
[19] J. Wang, G. Zhao, H. Xu, Y. Zhao, X. Yang, and H. Huang, “TRUST: the summer of 2016 and Associate Vice Provost for
Real-time request updating with elastic resource provisioning in clouds,” International Affairs from the fall of 2015 to the
in Proc. IEEE Conf. Comput. Commun., 2022, pp. 620–629. summer of 2017. Prior to joining Temple University,
[20] P. Henderson, R. Islam, P. Bachman, J. Pineau, D. Precup, and D. Meger, he was a Program Director with the National Science
“Deep reinforcement learning that matters,” in Proc. AAAI Conf. Artif. Foundation and was a Distinguished Professor with Florida Atlantic University,
Intell., 2018, pp. 3207–3214. Boca Raton, FL, USA. His research interests include mobile computing and
[21] H. Mao, M. Alizadeh, I. Menache, and S. Kandula, “Resource management wireless networks, routing protocols, cloud and green computing, network trust
with deep reinforcement learning,” in Proc. 15th ACM Workshop Hot and security, and social network applications. He regularly publishes in scholarly
Topics Netw., 2016, pp. 50–56. journals, conference proceedings, and books. He serves on several editorial
[22] C. Bitsakos, I. Konstantinou, and N. Koziris, “DERP: A deep reinforce- boards, including IEEE Transactions on Service Computing and the Journal
ment learning cloud system for elastic resource provisioning,” in Proc. of Parallel and Distributed Computing. He was the general Co-chair of IEEE
IEEE Int. Conf. Cloud Comput. Technol. Sci., 2018, pp. 21–29. MASS 2006, IEEE IPDPS 2008, IEEE ICDCS 2013, ACM MobiHoc 2014, ICPP
[23] S. Liang, Z. Yang, F. Jin, and Y. Chen, “Data centers job scheduling with 2016, and IEEE CNS 2016, as well as program Co-chair of IEEE INFOCOM
deep reinforcement learning,” in Proc. Pacific-Asia Conf. Knowl. Discov. 2011 and CCF CNCC 2013. He was an IEEE Computer Society Distinguished
Data Mining, 2020, pp. 906–917. Visitor, ACM Distinguished Speaker, and the Chair of the IEEE Technical
[24] K. Liu et al., “A learning-based data placement framework for low latency Committee on Distributed Processing (TCDP). He is a CCF Distinguished
in data center networks,” IEEE Trans. Cloud Comput., vol. 10, no. 1, Speaker. He was the recipient of the 2011 China Computer Federation Overseas
pp. 146–157, Jan.–Mar. 2022. Outstanding Achievement Award.

Authorized licensed use limited to: Sri Krishna College Of Engineering & Technology. Downloaded on May 21,2024 at 16:41:57 UTC from IEEE Xplore. Restrictions apply.
 LU et al.: TOWARDS DYNAMIC REQUEST UPDATING WITH ELASTIC SCHEDULING 2237

Jiamei Shi received the B.Sc. degree in computer Jiayue Zhang received the Ph.D. degree in signal and
science and technology from Shanghai Maritime Uni- information processing from the Beijing University
versity, Shanghai, China. She is currently working of Posts and Telecommunications, Beijing, China, in
toward the M.Sc. degree on computer science with 2018. She is currently a Lecturer with the Faculty
Faculty of Information Technology, Beijing Univer- of Information Technology, Beijing University of
sity of Technology, Beijing, China. Her research in- Technology, Beijing. She was supported by the China
terests include cloud computing and edge computing. Scholarship Council as a Visiting Scholar supervised
by Prof. Jimmy Huang with the Department of Infor-
mation Technology, York University, Toronto, ON,
Canada from 2012 to 2014. She is a Member of China
Comupter Federation. Her research interests include
machine learning and data mining.

Haiming Liu received B.S. degree in Computer

Science and Technology from Jilin University,
Juan Fang (Member, IEEE) received the M.S. degree Changchun, in 2012, the M.S. degree in Com-
from the Jilin University of Technology, Changchun, puter Software and Theory from Jilin University,
China in 1997 and the Ph.D. degree from the College Changchun, in 2015, and the Ph.D. degree in com-
of Computer Science, Beijing University of Technol- puter science and technology (bioinformatics) from
ogy, Beijing, China, in 2005. In 1997, she was with Jilin University, Changchun, China, in 2019. He is
the College of Computer Science, Beijing University currently a Lecturer with the School of Software Engi-
of Technology. Since 2015, she has been with the neering, Beijing Jiaotong University, Beijing, China.
Professor of Beijing University of Technology. Her Before that, he received his. He is a Member of the
research interests include high performance comput- Chinese Association for Artificial Intelligence. His
ing, edge computing, and Big Data technology. research interests include the edge computing, data mining, and bioinformatics.

Authorized licensed use limited to: Sri Krishna College Of Engineering & Technology. Downloaded on May 21,2024 at 16:41:57 UTC from IEEE Xplore. Restrictions apply.