See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/375747972

Machine Learning Compliance-aware Dynamic Software Allocation for

Energy, Cost and Resource-efficient Cloud Environment

Article in Sustainable Computing Informatics and Systems · November 2023

DOI: 10.1016/j.suscom.2023.100938

CITATIONS READS

0 14

2 authors, including:

Leila Helali
Sousse University
7 PUBLICATIONS 70 CITATIONS

SEE PROFILE

All content following this page was uploaded by Leila Helali on 09 December 2023.

The user has requested enhancement of the downloaded file.

 Sustainable Computing: Informatics and Systems 41 (2024) 100938

Contents lists available at ScienceDirect

Sustainable Computing: Informatics and Systems

journal homepage: www.elsevier.com/locate/suscom

Machine learning compliance-aware dynamic software allocation for energy,

cost and resource-efficient cloud environment
Leila Helali ∗, Mohamed Nazih Omri
MARS Research Laboratory, University of Sousse, Tunisia

ARTICLE INFO ABSTRACT

Keywords: With the growing number of cloud services protected by licenses, compliance management and assurance
Compliance is becoming critical need to support the development of trustworthy cloud systems. In these systems, the
License optimization multiplication of services and the inefficient resource utilization incurred energy consumption and costs
Energy efficiency
increase despite the consolidation initiatives underway. Few works deal with resource allocation optimization
Resource management
at the SaaS level, which does not consider compliance aspects. Generally, the reported consolidation work
AI
Reinforcement learning
does not address license management in the cloud environment as a whole, particularly from a resource
management perspective, and the vast majority of consolidation work focuses on resource optimization at
the infrastructure level. Thus, we propose a software license consolidation scheme based on multi-objective
reinforcement learning that enables efficient use of resources and optimizes energy consumption, resource
wastage, and costs while ensuring compliance with the processor-based licensing model. The experimental
results show that our solution outperforms the baseline approaches in different scenarios with homogeneous
and heterogeneous resources under different data center scales.

1. Introduction and licenses should be scaled up and down rapidly according to real
needs, this method is unsuitable and costly. Moreover, software licens-
With the emergence of new paradigms like softwarization [1] and ing is considered as an obstacle for the cloud [6].1 Here, we consider
network function virtualization, the demand for services and resources processor-based licensing, which depends on the number of cores of
in cloud data centers (DCs) was accelerated. The SaaS market repre- the processor and a core processing license factor as in the Oracle
sents the most important evolution that remains in 2021, with $97
database case [7]. From a resource management perspective, energy
billion in global spending [2]. Proprietary solutions are particularly
consumption represents another important component of operational
underlined due to the current economic climate. Managing these ser-
vices protected by licenses represents a recent challenge where more expenditure in the cloud and can contribute to global warming due to
than 50% of the incurred costs within a DC are reserved for software the enormous emission of carbon and the released temperature. The
licensing, maintenance, and deployment [3]. Indeed, The software amount of energy is significant in the actual data centers delivering
license charges may be included in the virtual machine (VM) price cloud computing services that contain thousands of computational ma-
(License Charges Included) when VM-based virtualization technology chines [8]. Energy consumption is increased with increasing resource
is used or considered separately. In the latter case, the cloud user, utilization within a DC. The poor management of resources can cause
when deploying the software, considers his software license called exploitation of only 10% of about 30% cloud servers [1]. As a result,
Bring Your Own License (BYOL) [4]. Here, we consider the second up to 13% of global electricity will be used by data centers by the year
scenario. In addition to the generated costs, non-compliant situations 2030 [9]. To cope with the aforementioned problems, consolidation
can cause many potential risks which translate into economic losses.
is a suitable technique that allows us to run more workloads with
This notion of compliance appeals to the metric model used in software
lower costs and lowered power demands without adding new hardware
licensing which remains obscure in commercial cloud data centers. The
most popular licensing method uses declarative licenses, where the and resources [1]. Through migration and optimal allocation, software
business agreements established are periodically audited to check com- licenses are dynamically provisioned and released elastically according
pliance [5]. In highly dynamic cloud environments, where resources to the real use to prevent under/over utilization of resources.

∗ Corresponding author.
E-mail addresses: [email protected] (L. Helali), [email protected] (M.N. Omri).
1
Flexera 2020 State of the Cloud Report.

https://doi.org/10.1016/j.suscom.2023.100938
Received 14 February 2022; Received in revised form 20 January 2023; Accepted 18 November 2023
Available online 28 November 2023
2210-5379/© 2023 Elsevier Inc. All rights reserved.
L. Helali and M.N. Omri Sustainable Computing: Informatics and Systems 41 (2024) 100938

Fig. 1. An example of software license consolidation. Each software is associated with a utilization rate representing its resource requirements (CPU and memory) in its hosting
PM, and a cost of licenses, successively. For example, every instance of the software S1 needs 6% of the running PM CPU, 6% of that PM’s memory, and a software cost of 0.5.

Very few works address consolidation at the software level. To implicitly reduces the infrastructure costs. Furthermore, the costs re-
this end, we propose an intelligent scheme based on a reinforcement sulting from penalties caused by non-compliant situations are another
learning framework to migrate and dynamically allocate the software dimension of losses [10] that are treated and avoided in our work
licenses. Our approach tends to maximize the DC utilization by mini- by preserving compliance. This way allows a win–win strategy and
mizing the wasted resources at the physical machine (PM) and virtual improves the provider’s reputation. The following points summarize the
machine levels and reducing energy consumption along with costs key contributions to address the mentioned challenges:
including license and VM costs while being compliant. It is a proactive
approach that handles compliance and checks the number of installed • A pre-allocation approach that, combined with our Maximum Li-
instances of each software from the resource scalability step, instead cense Cost (MLC) migration policy [11], allows important savings
of reactively auditing the already installed software instances to deal via migration by saving the cost of the most expensive software
with compliance. Our proposed model was evaluated regarding homo- licenses, is proposed.
geneous and heterogeneous machines (PMs and VMs) with small and • An intelligent software license consolidation (SLC) scheme is
larger data center sizes. proposed which dynamically reallocates the SLs based on the
These DCs are virtualized according to the VM-container architec- reinforcement learning (RL) framework.
ture when software license (SL) applications are packaged in containers • Extensive simulations are conducted to evaluate our proposed
that run in VMs since software licenses require strict security mea- scheme considering different metrics and approaches which show
sures. This virtualization solution represents the best alternative until the superiority and the effectiveness of our approach.
now [1]. This is also encouraged by the quick deployment of new
SLs along with the agility and lightweight of containers to develop a The rest of this paper is structured as follows. In Section 2, we
dynamic consolidation solution in dynamic and highly changing cloud introduce the software license notions in the cloud environment and
environments. In this architecture, the resources used by the software provide the motivation of our work. In Section 3, we present the related
licenses (processes) are managed through control groups (cgroups) and work followed by the formulation of our proposed model in Section 4.
namespaces [1]. Optimizing operational expenditures, including energy Section 5 describes our proposed architecture and its building blocks,
consumption, represents a challenge for cloud providers (CP). Thus, and details our intelligent consolidation approach applying reinforce-
the classic consolidation works are generally performed from the CP ment learning. The simulation settings, results, and discussions are
perspective by grouping virtual machines in the minimum number detailed in Section 6, followed by a conclusion section.
of servers. Our approach represents another benefit by grouping the
software licenses in the minimum number of virtual machines. This
2. Preliminaries and motivations
allows for minimizing the license costs when many licenses are packed
in the same virtual machine and paid only once without escaping
compliance. This also allows for maximizing the client profit through This section is devoted to presenting an overview of the software
the minimization of the user-monetary costs by minimizing the number licenses in the cloud, the notion of licensing metrics, and the use case
of virtual machines, which offers more flexibility to further reduce considered in the rest of the article. Then, a motivating example of
the number of active physical machines in the data center, and thus, software license placement optimization will be presented.

2
L. Helali and M.N. Omri Sustainable Computing: Informatics and Systems 41 (2024) 100938

2.1. An overview of software licenses in the cloud In the other configuration (Placement B), only 10 VMs and 5 PM are
needed to run this workload.
Today, software services are externalized through the cloud and This encourages us to bundle multiple SLs into the same VM when-
their number is multiplied. Managing this software when they are ever possible instead of dedicating one VM to running each SL. This is
regulated by licenses represents a recent trend in the cloud computing enabled by the emergence of container-based clouds and containerized
environment virtualized to the extreme. workloads [1]. Thus, software licenses can be wrapped in containers
The license determines what a user can and cannot do with the for isolation and containerized SLs run in VMs to provide a high level
software. It represents the ‘‘legal rights to use the software in accor- of security [12]. As part of the work presented in [1], the benefits of
dance with terms and conditions specified by the software licensor’’.2 considering a VM container virtualization solution were discussed. We
Customers use a variety of software licenses example operating system believe that grouping several SLs in the same VM allows us to minimize
licenses (windows server, Red Hat, etc.), database licenses (SQL Server, the waste of resources and thus, to offer more possibilities to optimize
Oracle Database, etc.), Applications licenses (SAP ERP, IBM Websphere, energy consumption. This also helps to optimize virtual machine costs
etc.), and other third-party licenses. and licensing costs, as we will see later. This strategy, when applied to
The software products follow a whole life cycle from purchasing to the DCs of cloud giants like Google and Amazon, will certainly lead to
usage then decommissioning [10]. The rules to follow that organize the huge savings in terms of energy.
use of the product and that contain the information (for example the In our work, we address licensing and consolidation from a re-
region where the product can be installed) about contractual rights are source management perspective to economics, energy consumption
defined in the Product Use Rights (PURs). If the image of the product considerations, and compliance. Our goal is to place software licenses,
is non conform to the established rules or if the used licenses do not either initially or after migration, into the most appropriate machine
fit with the license stock, a penalty will be imposed by publishers. with respect to resource usage and inherent power consumption while
In addition to the usage rights defined in PURs that describe the minimizing costs and ensuring compliance. The originality of our work
conditions of the product use, possessing a sufficient number of licenses is the fact of considering the definition of the metric in an objective
that covers all products is an obligation to be compliant. Having enough of cost reduction, in parallel with the software consolidation which is
licenses is necessary but using them efficiently is more advantageous. little treated in the literature.
It is worth noting that to account for the number of licenses we need
for proprietary software, the concept of metric imposes, which is an 3. Related work
important PUR. In fact, for having the right to use a product, a given
user have to process multiple licenses and this number is computed This section presents software license consolidation-related works,
based on the metric. A single product may have several metrics (users, synthesized in Table 1. It is worth noting that the overwhelming
devices, etc.) with different prices per license depending on the metric majority of consolidation work was concentrated at the infrastructure
model. As an example, two metrics can be used to license the Oracle level and the workload consolidation at the software is still in its
Database namely: (i) Processor where ‘‘The number of required licenses early days [12]. Also, license management and optimization [10]
shall be determined by multiplying the total number of cores of the pro- and compliance aspects [13] are rarely addressed in the literature.
cessor by a core processor factor ’’ [7]. And (ii) Named User Plus (NUP) In [3], Sen et al. considered software license consolidation by placing
metric, which requires a minimum of 25 Named User Plus per Processor workloads with similar software stack on the same PM to reduce the
license or the total number of actual users, whichever is greater. When number of software instances, and thus costs. The authors consider a
licensing the Oracle Database by Named User Plus, all users who are typical CRM web application and they consolidate the software into
using the Oracle Database, as well as all non-human operated devices several layers. This work performs classical hardware consolidation and
that are accessing the Oracle Database must be licensed [7]. a per-server consolidation of software. The limits of this approach are
represented in isolation problems with the lack of dependencies and
2.2. Use case: Processor-based metric support to migrate hosted applications across servers. It also has a
security problem because of the fact of consolidating in the stack above
Counting users on the internet is difficult [7]. Thus, the NUP metric the VM without taking security measures, if the software (application,
is hard to apply. Moreover, in virtualized environments, there is an portal, middleware) is contaminated, it incurs serious security prob-
alternative model to the processor-based metric, based on the vCPU lems. In [14], the authors consider the cost of licensed components in
(virtual CPU). As the number of vCPU depends on the hypervisor, virtual machine consolidation. The objective is to select suitable VMs
it is hard to have the real accurate number of vCPU per machine. to place software components and suitable PMs to accommodate these
Thus, we consider a processor-based licensing use case that allows VMs, without considering the deployment or consolidation aspects of
us to determine accurately the number of cores per physical machine these components. In [15], a software consolidation approach was pro-
and thus the number of licenses in this machine. Furthermore, many posed to optimize energy consumption and costs that do not consider
users can run the same software through many instances in the same licenses. In [11] and [12], some heuristic-based consolidation solutions
provider. Thus, many instances may belong to the same software even for software licenses were proposed. The objective is to optimize costs
for different users. To simplify, each instance designates a license in the and energy consumption. This solution has shown motivating results.
rest of the manuscript. Some virtual machine consolidation work treated resource wastage like
the works proposed in [16] and in [17]. Regarding the technical aspect,
2.3. Motivations heuristics, metaheuristics, and reinforcement learning-based solutions
are the most adopted in cloud resource management and consolidation.
As shown in the example given by Fig. 1, we consider 7 software A comparison is given here [1].
types with 3 licenses of each software (the number of licenses by soft- Contrary to the approaches previously presented, in our work, the
ware can vary) colored with the same color, with their resource usage containers that package the software licenses and their dependencies
(CPU and memory) and fees. To place these SLs, two configurations resolve the isolation problem of Sen’s work [3], and the considered
are presented. In the first configuration (Placement A), each VM runs VM-container virtualization architecture allows to resolve the security
an SL. In this case, we need 21 VMs and 8 PMs to run the workload. risks [1]. We also perform consolidation at the software level and not
the VM as Mann [14] did. In [11] and [12], compliance and wastage
of resources have not been taken into account. In fact, none of the
2
ISO/IEC 19770-5, 3.41. literature work considers resource wastage and compliance aspects.

3
L. Helali and M.N. Omri Sustainable Computing: Informatics and Systems 41 (2024) 100938

Table 1
Summary of related works.
Work CL LC VM cost EC RW Comp
Mann [14] VM ✓ – – – –
Sen et al. [3] Hardware, software ✓ – – – –
Tchana et al.[15] Software, VM – ✓ ✓ – -
Helali et al. [11] Software license ✓ ✓ ✓ – –
Helali et al. [12] Software license ✓ ✓ ✓ – –
Our work Software license ✓ ✓ ✓ ✓ ✓

CL = Consolidation Level, Comp = Compliance, LC = License Cost, EC = Enegry Consumption, RW = Resource Wastage.

successively.
{
1 𝑆𝐿𝑟𝑘 is allocated to 𝑉 𝑀𝑗
𝛼𝑗𝑘 = (2)
0 else
Finally, we express the relationship between a software license
𝑆𝐿𝑟𝑘 and a physical machine 𝑃 𝑀𝑖 as 𝛼𝑗𝑘 𝛿𝑖𝑗 with 𝑖 ∈ [1..𝑁𝑃 𝑀 ], 𝑗 ∈
[1..𝑁𝑉 𝑀 ] and 𝑘 ∈ [1..𝑁𝑆 ].
Fig. 2. The set of PMs with their resource utilization.
{
1 𝑉 𝑀𝑗 is placed on 𝑃 𝑀𝑖
𝛿𝑖𝑗 = (3)
0 else
Our goal is to optimize the total cost (licenses+VMs) along with
the overall energy consumption while minimizing the wasted resources
within the cloud DC.

4.1. Processor-based metric and compliance models

Fig. 3. The set of VMs with their resource utilization.
Here, we consider a processor-based model use case as stated above.
Hence, for a given 𝑃 𝑀𝑖 , the number of required licenses 𝑁𝐿𝑖 for
an application licensed under the processor-based model is given by
Eq. (4).
𝑁𝐿𝑖 = 𝑁𝑐𝑖 ∗ 𝐶𝑓 (4)
Where 𝑁𝑐𝑖 is the total number of cores (see Eq. (5)), 𝐶𝑓 in the core
factor, 𝑁𝑝𝑖 is the number of processors per server, and 𝐶𝑝𝑖 is the number
of cores per processor. For example, to license a processor with 6 cores
and a core factor of 0.25, we need 2 licenses which is the aggregation
of 1.5 (6*0.25).
𝑁𝑐𝑖 = 𝑁𝑝𝑖 ∗ 𝐶𝑝𝑖 (5)
Fig. 4. The set of SLs with their resource utilization.
To ensure compliance with the considered license model, we pro-
pose Eq. (6) account for the number of licenses in each physical
machine 𝑃 𝑀𝑖 for each software 𝑆𝑘 .
Finally, we noticed that RL approaches surpass heuristic and meta- ∑
𝐿𝑘𝑃 𝑀 = 𝛼𝑗𝑘 𝛿𝑖𝑗 𝐿𝑘𝑉 𝑀
heuristic solutions [1]. The approaches proposed in [18] and in [19] 𝑖 𝑗
𝑗
reported promising results for virtual machine consolidation based on ∑∑ (6)
= 𝜏𝑘𝑟 𝛼𝑗𝑘 𝛿𝑖𝑗 𝐿𝑟
reinforcement learning.
𝑗 𝑟

4. System model and problem formulation 4.2. Resource utilization model

We define the normalized utilization vector of a 𝑃 𝑀𝑖 as:

The SLC is an NP-hard [15,20] problem modeled as a bin-packing.
𝑗 𝑞
We consider a set of physical servers 𝑃 𝑀 = {𝑃 𝑀𝑖 |𝑖 = 1, … , 𝑁𝑃 𝑀 } and ∑ 𝛿𝑖 𝑁𝑈𝑗
𝑁𝑈𝑖𝑞 =
a set of virtual machines 𝑉 𝑀 = {𝑉 𝑀𝑗 |𝑗 = 1, … , 𝑁𝑉 𝑀 } with associated 𝑗 𝐶𝑎𝑝𝑞𝑖
resource usage values to execute the SLs (see Fig. 2 and 3). 𝑗 𝑞
(7)
∑ ∑ 𝛼𝑗𝑘 𝛿𝑖 𝑁𝑈𝑘
A set of software 𝑆 = {𝑆𝑘 |𝑘 = 1, … , 𝑁𝑆 } offered by the SaaS =
provider with resource utilization (Fig. 4) and the associated licenses 𝑗 𝑘 𝐶𝑎𝑝𝑞𝑖
and fees. The set of 𝑁𝑙 licenses for each software 𝑆𝑘 is 𝐿 = {𝐿𝑟 |𝑟 = The normalized CPU utilization (𝑁𝑈𝑖𝑐 ) of a 𝑃 𝑀𝑖 is estimated by the
1, … , 𝑁𝑙 } (Eq. (4)). The set of software licenses is denoted 𝑆𝐿 = sum of the utilization of all software running in all VMs hosted in the
{𝑆𝐿𝑟𝑘 |𝑘 = 1, … , 𝑁𝑆 , 𝑟 = 1, … , 𝑁𝑙 }. PM and is given by Eq. (8). 𝑁𝑈𝑗𝑐 and 𝑁𝑈𝑘𝑐 are the normalized CPU
The relationship that defines if a license belongs to a given software utilization of a 𝑉 𝑀𝑗 and an 𝑆𝐿𝑘 successively, and 𝐶𝑎𝑝𝑐𝑖 is the CPU
is formally given by Eq. (1). capacity of the 𝑃 𝑀𝑖 .
{ 𝑗
1 𝑆𝑘 is licensed under 𝐿𝑟 ∑ 𝛿𝑖 𝑁𝑈𝑗𝑐
𝜏𝑘𝑟 = (1) 𝑁𝑈𝑖𝑐 =
0 else 𝑗
𝐶𝑎𝑝𝑐𝑖
𝑗
(8)
The relationship between a software license 𝑆𝐿𝑟𝑘 and a virtual ∑ ∑ 𝛼𝑗𝑘 𝛿𝑖 𝑁𝑈𝑘𝑐
=
machine 𝑉 𝑀𝑗 , a 𝑉 𝑀𝑗 and a 𝑃 𝑀𝑖 are given by Eqs. (2) and (3) 𝑗 𝑘
𝐶𝑎𝑝𝑐𝑖

4
L. Helali and M.N. Omri Sustainable Computing: Informatics and Systems 41 (2024) 100938

The normalized memory utilization of a 𝑃 𝑀𝑖 is given by Eq. (9). In the same way, we propose the Eq. (19) below to model the
Where 𝑁𝑈𝑗𝑚 , 𝑁𝑈𝑘𝑚 and 𝐶𝑎𝑝𝑚 𝑖 represent the normalized memory uti- resource wastage of a q-dimensional 𝑉 𝑀𝑗
lization of the 𝑉 𝑀𝑗 , that of the software license 𝑆𝐿𝑟𝑘 , and the memory ∑𝑞 | 𝑞 |
|𝑁𝑅𝑡𝑗 − 𝑚𝑖𝑛𝑜=1 (𝑁𝑅𝑜𝑗 )| + 𝜌
capacity of the 𝑃 𝑀𝑖 successively. 𝑡=1 | |
𝑅𝑊𝑗 = ∑𝑞 𝑡
(19)
𝑗 𝑁𝑈
∑ 𝛿𝑖 𝑁𝑈𝑗𝑚 𝑡=1 𝑗
𝑁𝑈𝑖𝑚 =
𝑗
𝐶𝑎𝑝𝑚
𝑖 𝑁𝑅𝑚𝑎𝑥
𝑗 = 𝑚𝑎𝑥(𝑁𝑅𝑐𝑗 , 𝑁𝑅𝑚
𝑗 ) (20)
(9)
∑∑ 𝛼𝑗𝑘 𝛿𝑖𝑗 𝑁𝑈𝑘𝑚
= 𝑁𝑅𝑚𝑖𝑛 = 𝑚𝑖𝑛(𝑁𝑅𝑐𝑗 , 𝑁𝑅𝑚 (21)
𝐶𝑎𝑝𝑚 𝑗 𝑗 )
𝑗 𝑘 𝑖
considering 𝑁𝑅𝑚𝑎𝑥
𝑗 and 𝑁𝑅𝑚𝑖𝑛𝑗 that denote the maximum (respec-
4.3. Power and energy models tively minimum) residual resources of memory and CPU utilization (see
Eq. (20) and (21)) of a given 𝑉 𝑀𝑗 , the Eq. (19) became:
The power consumption of a machine 𝑃 𝑀𝑖 is modeled as the sum
| |
of the static (𝑃 𝑠𝑡 ) and dynamic (𝑃 𝑑𝑦 ) parts as shown in Eq. (10). |𝑁𝑅𝑚𝑎𝑥 𝑗 || + 𝜌
− 𝑁𝑅𝑚𝑖𝑛
𝑅𝑊𝑗 = |
𝑗
(22)
𝑃𝑖 = 𝑃𝑖𝑠𝑡 + 𝑃𝑖𝑑𝑦 (10) 𝑁𝑈𝑗𝑐 + 𝑁𝑈𝑗𝑚

The former is defined by the power consumption of the host when 𝑁𝑅𝑡𝑗 is the normalized residual resource vector of the virtual machine
no VM is active and thus no SL runs on the machine. Otherwise, the 𝑉 𝑀𝑗 and is given by the Eq. (23).
dynamic part is added to obtain the total power of a server function
𝑁𝑅𝑡𝑗 = 1 − 𝑁𝑈𝑗𝑡 (23)
of the power consumption of a fully utilized machine (𝑃 𝑓 𝑢 ) which is
detailed in Eq. (11). 𝜌 and 𝜂 are small positive numbers, we assume that 𝜌 = 𝜂 = 0.0001 [16].
{ The total resource wastage in the DC is given by:
𝑐 𝑥 ∗ 𝑃𝑖𝑓 𝑢 + (1 − 𝑥) ∗ 𝑃𝑖𝑓 𝑢 ∗ 𝑁𝑈𝑖𝑐 (𝑡), 𝑁𝑈𝑖𝑐 (𝑡) > 0
𝑃𝑖 (𝑁𝑈𝑖 (𝑡)) = (11) 𝑁𝑃 𝑀
0, otherwise ∑ ∑
𝑅𝑊 = (𝑅𝑊𝑖 + 𝛿𝑖𝑗 𝑅𝑊𝑗 ) (24)
The static power consumption is a part of the dynamic consump- 𝑖=1 𝑗

tion [21] and is usually estimated as 70% of a fully utilized PM [1].

4.5. Cost model
The energy consumed by a server i is given by the Eq. (12).
𝑡1+𝛥𝑡
𝐸𝑖𝑠 = 𝑃𝑖 (𝑁𝑈𝑖𝑐 (𝑥))𝑑𝑥 (12) The cost of our model is formed by the fees paid by the user for
∫𝑡1 renting a VM (𝑅𝑓𝑉 𝑀𝑗 ) within an hour along with the license cost
This allows estimating the total energy consumed in a DC by (𝐿𝑓𝑉 𝑀𝑗 ) which is estimated by the sum of the license fees (𝐿𝑓𝑆𝐿𝑟 ) of
𝑘
Eq. (13). the software running in a given VM (Eq. (25)).
𝑁𝑃 𝑀
∑
∑ 𝐿𝑓𝑉 𝑀𝑗 = 𝜏𝑘𝑟 𝛼𝑗𝑘 𝐿𝑓𝑆𝐿𝑟 (25)
𝐸= 𝐸𝑖𝑠 (13) 𝑘
𝑘

𝑖=1
As we consider processor-based licensing, the license fee is paid
As discussed in [1], the energy consumption is composed of many once for at least an instance of software running on a given machine.
parts (network, cooling, etc.) rather than the server energy. But, this We notice that the number of licenses (the number of copies permitted
later represents the most important part. Thus, to simplify concerns, of the same software) can be controlled by the ID of containers packing
we only consider the energy consumption at the server level. them as Revenera do.3 The total cost of a VM is given by Eq. (26).

4.4. Resource wastage model 𝐶𝑗 = 𝑅𝑓𝑉 𝑀𝑗 + 𝐿𝑓𝑉 𝑀𝑗 (26)

In the same way, the cost of a 𝑃 𝑀𝑖 is estimated by the sum of the

Inspired by [16], we propose a generalized formula to estimate the
cost of all VMs running on it (Eq. (27)).
resource wastage for q-dimensional PM resources (see Eq. (14)).
∑ 𝑗
∑𝑞 | 𝑞 | 𝐶𝑖 = 𝛿𝑖 𝐶𝑗 (27)
|𝑁𝑅𝑡𝑖 − 𝑚𝑖𝑛𝑜=1 (𝑁𝑅𝑜𝑖 )| + 𝜂
𝑡=1 | | 𝑗
𝑅𝑊𝑖 = ∑𝑞 (14)
𝑡=1
𝑁𝑈𝑖𝑡 Finally, the total cost within a DC is given by Eq. (28).
As we consider the CPU and memory resources, this equation be- 𝑁𝑃 𝑀
∑
came (see Eq. (15)): 𝐶= 𝐶𝑖 (28)
| | 𝑖=1
|𝑁𝑅𝑚𝑎𝑥 𝑖 || + 𝜂
− 𝑁𝑅𝑚𝑖𝑛
𝑅𝑊𝑖 = |
𝑖
𝑐 𝑚 (15) 4.6. Problem formulation
𝑁𝑈𝑖 + 𝑁𝑈𝑖
𝑁𝑅𝑚𝑎𝑥
𝑖 denotes the maximum residual resources of memory and
The objective functions that reflect the minimization problems are
CPU utilization of the given 𝑃 𝑀𝑖 (see Eq. (16)).
given by Eq. (29).
𝑁𝑅𝑚𝑎𝑥 = 𝑚𝑎𝑥(𝑁𝑅𝑐𝑖 , 𝑁𝑅𝑚
𝑖 ) (16) ∑
𝑖 ⎧𝑚𝑖𝑛(𝐸) = 𝑁𝑃 𝑀 𝐸𝑖
⎪ ∑𝑁𝑖=1
𝑁𝑅𝑚𝑖𝑛
𝑖 denotes the minimum residual resources of memory and CPU 𝑓 = ⎨𝑚𝑖𝑛(𝐶) = 𝑖=1𝑃 𝑀 𝐶𝑖 (29)
utilization of that PM (see Eq. (17)). ⎪ ∑ 𝑁𝑃 𝑀 ∑ 𝑗
⎩ 𝑚𝑖𝑛(𝑅𝑊 ) = 𝑖=1
(𝑅𝑊 𝑖
+ 𝛿
𝑗 𝑖 𝑅𝑊 𝑗 )
𝑁𝑅𝑚𝑖𝑛
𝑖 = 𝑚𝑖𝑛(𝑁𝑅𝑐𝑖 , 𝑁𝑅𝑚
𝑖 ) (17) To transfer our optimization problem into a mono-objective prob-
𝑁𝑅𝑡𝑖 is the normalized residual resource vector of the host 𝑃 𝑀𝑖 and lem that combines the energy, resource wastage, and cost objective
is given by the Eq. (24).

𝑁𝑅𝑡𝑖 = 1 − 𝑁𝑈𝑖𝑡 (18) 3

https://www.revenera.com/resources.html?type=datasheet

5
L. Helali and M.N. Omri Sustainable Computing: Informatics and Systems 41 (2024) 100938

functions in one function 𝑓 , we adopt the most commonly used lin- resource usage and getting local resource states that will be used
ear weighted sum aggregation (Eq. (30)) [22]. The weights 𝑤𝑖 are by the launcher to trigger the consolidation process. Based on this
comprised in the interval [0,1] and verify the Eq. (31). information, the modeler estimates the cost, energy, and resource waste
∑
𝑛 in the DC. The selector, on the other hand, chooses the software license
𝑓= 𝑤𝑖 ∗ 𝑓 𝑖 (30) to be migrated according to an optimal strategy called MLC which will
𝑖=1
be detailed in the next section. Then, the smart allocator first selects
∑
𝑛
the list of candidate virtual machines for each migrated SL. Afterward,
𝑤𝑖 = 1 (31)
𝑖=1 it proceeds to the evaluation of the potential allocation to, finally,
decide on the most appropriate actions allowing to optimize the DC
In our case, n = 3 and 𝑓 became:
state according to the RL framework.
𝑚𝑖𝑛(𝑤1 ∗ 𝐸 + 𝑤2 ∗ 𝐶 + 𝑤3 ∗ 𝑅𝑊 ) (32) As a first step, the pre-allocator performs a pre-selection as de-
𝑤1 , 𝑤2 and 𝑤3 reflect the importance of each objective function in the scribed below. In the second step, the allocation decision is communi-
single-objective function f. Thus, the final result of the proposed model cated to the compliance checker to check whether any given allocation
depends on these three variables. We consider these three objectives of has escaped compliance proactively. This can be restricted by testing
a comparable magnitude and consider equivalent weights. whether or not the number of instances of the software, of the same
Furthermore, the following constraints are considered: license, exceeds the allowed amount based on the processor-based
metric. If this is the case, the compliance module triggers an alert
𝐿𝑘𝑃 𝑀 <= 𝑁𝑙𝑖 ∀𝑖 ∈ [1..𝑁𝑃 𝑀 ], 𝑘 ∈ [1..𝑁𝑁𝑆 ] (33) (see Fig. 9) and prevents the allocator to evaluating another allocation.
𝑖

The Eq. (33) denotes that the allowed software instances of a given This process is repeated until a conforming allocation is found. The
license type must not exceed the allowed number of that license type last decision of SL placement is then communicated to the activa-
following the processor-based metric. tor/deactivator module to activate new VMs/PMs if necessary and/or
∑ 𝑗 stop unused ones. Finally, the DC state is updated to dynamically start
𝛿𝑖 𝑁𝑈𝑗𝑐 < 𝐶𝑎𝑝𝑐𝑖 ∀𝑖 ∈ [1..𝑁𝑃 𝑀 ], 𝑗 ∈ [1..𝑁𝑉 𝑀 ] (34) a new consolidation after every 5-minute time interval.
𝑗

Eqs. (34) and (35) fix the constraints related to the memory and CPU
utilization limit of a PM which should not be exceeded by the CPU and 5.2. Compliance-aware RL-based dynamic software license placement
memory utilization of all hosted VMs.
∑ 𝑗 The software license consolidation follows a whole process illustrat-
𝛿𝑖 𝑁𝑈𝑗𝑚 < 𝐶𝑎𝑝𝑚 𝑖 ∀𝑖 ∈ [1..𝑁𝑃 𝑀 ], 𝑗 ∈ [1..𝑁𝑉 𝑀 ] (35)
𝑗 ing the main steps of the proposed intelligent scheme to perform the
DC optimization in terms of energy consumption, resource wastage,
Likewise, Eqs. (36) and (37) allow ensuring that the CPU and
and total costs. These steps, detailed below, are performed at runtime
memory utilization of software licenses that are co-located in a VM does
not exceed the limit of CPU and memory utilization of the VM hosting according to the utilization level of the PMs. The DC state is periodically
them. evaluated and the consolidation decision is made based on the resource
∑ state.
𝛼𝑗𝑘 𝑁𝑈𝑘𝑐 < 𝐶𝑎𝑝𝑐𝑗 ∀𝑗 ∈ [1..𝑁𝑉 𝑀 ], 𝑘 ∈ [1..𝑁𝑆𝐿 ] (36)
𝑘

𝐶𝑎𝑝𝑐𝑗 and 𝐶𝑎𝑝𝑚 5.2.1. State monitoring

𝑗 represent the resource capacity in terms of memory and
CPU of the 𝑉 𝑀𝑗 , successively. To identify the list of critical machines, we use the utilization rate
∑ of the physical machines. In the event that the processor utilization
𝛼𝑗𝑘 𝑁𝑈𝑘𝑚 < 𝐶𝑎𝑝𝑚𝑗 ∀𝑗 ∈ [1..𝑁𝑉 𝑀 ], 𝑘 ∈ [1..𝑁𝑆𝐿 ] (37) of a given machine exceeds an over-utilization threshold, the server
𝑘
will be considered overloaded (OL). Otherwise, it will be considered
Eq. (38) ensures that each software license is exactly mapped to one underloaded (UL). Thus, software licenses running on a UL machine
VM at a time. will be selected for migration and those hosts will likely be deactivated.
𝑁𝑉 𝑀
∑ In addition, some SLs will be selected for migration to resolve the OL
𝛼𝑗𝑘 = 1∀𝑘 ∈ [1..𝑁𝑆𝐿 ] (38) situation. Continuing with the example in Fig. 1, if we consider an
𝑗=1
underutilization and overutilization threshold of 20% and 80% [12]
In the same way, Eq. (39) ensures that each VM is hosted by one respectively, then the two hosts 𝑃 𝑀1 and 𝑃 𝑀4 are now underutilized,
PM at a time. in the first placement (A), with 18% and 17% CPU usage, respec-
𝑁𝑃 𝑀
∑ tively. In placement B, the two physical machines 𝑃 𝑀2 and 𝑃 𝑀5 are
𝛿𝑖𝑗 = 1∀𝑗 ∈ [1..𝑁𝑉 𝑀 ] (39) under-loaded, on the other hand, the machine 𝑃 𝑀3 is overloaded.
𝑖=1

5. Methodology
5.2.2. Migration
After triggering the consolidation process, some SLs have to be
5.1. System architecture
selected for migration (see Fig. 6).
In this section, we detail the proposed architecture for SL consoli- For this step, we use the Maximum License Cost (MLC) migration
dation shown in Fig. 5 and their building blocks. In the Entropy [23] strategy [11,12] that saves the cost of the most expensive SL that cre-
approach, where a central entity is responsible for all aspects of con- ates overload/underload situations. On a critical machine, the critical
solidation, an architecture is entirely centralized. In our approach, we SL(s) that have the maximum license fees will be given the highest
separate monitoring and compliance verification. These two parts are priority to be migrated and it is added to the list of migratable SLs. The
distributed and deployed in each software licensing compute host. This idea is that if we can migrate the software license with the maximum
distributed architecture, regarding resource monitoring and compliance cost in a VM that contains another instance of the same software,
checking, enables natural system scalability and avoids the single point we save the cost of the migrated license. This gives more chances to
of failure. The monitor is responsible for collecting information on optimize costs at the maximum [12].

6
L. Helali and M.N. Omri Sustainable Computing: Informatics and Systems 41 (2024) 100938

Fig. 5. System architecture.

Fig. 6. Migration strategy.

5.2.3. Pre-allocation Algorithm 1: Pre-selection of Promising Destination VMs

The main idea behind our solution is to select the list of candidate Input: SLToMigrateList, PM_List
VMs that contains at least one instance of the software in question Output: L_VM
belonging to the same license. In this case, no license fee would be // the list of preselected VMs
incurred with this mapping which makes those VMs very promising 1 Begin
for the current allocation. The preselection phase is detailed in the 2 SLToMigrateList.sortDecreasingUtilization()
Algorithm 1 below. Firstly, the list of migratable SLs is sorted (line 2) 3 for SL in SLToMigrateList do
according to the CPU utilization. Then, for each SL, the type of the 4 𝐿 ← 𝑔𝑒𝑡𝐿𝑖𝑐𝑒𝑛𝑠𝑒𝑇 𝑦𝑝𝑒(𝑆𝐿)
license for a given software is determined (line 3) to check all the VMs 5 for PM in PM_List do
in all the PMs within the DC who contains at least one SL of the same 6 for VM in PM do
license and add them to the list of promising VM destinations which 7 if there is a software instance in VM with license L
is augmented by some random VMs sorted in decreasing order of CPU then
utilization to extend the search space and promote the fullest ones to 8 add VM to L_VM
maximize resource utilization.
Going back to the example above (Placement B), the pre-selection 9 select VMs (randomVMs)
result is depicted in Fig. 7. From the PM3, the migration step allows 10 sortDesreasingOrderCPU(randomVMs)
us to select, first, the instance of the software S2 (the SL with the light 11 add randomVMs to L_VM
blue color), then the instance of the software S4 (the SL with the brown
12 return L_VM
color). Also, as stated before, all the SLs in the underloaded PMs (PM2
13 End
and PM5) are selected for migration. In the preselection step, the SL
with the brown color, if placed in PM1 or PM4, its cost will be saved.
Thus, the VMs running brown SLs in these two hosts are selected as
promising machines for allocating the migrated SLs. 𝑆𝑡𝑖 = {𝑆𝑡𝑠𝑡𝑎𝑡𝑢𝑠 , 𝑆𝑡
𝑡𝑜𝑡𝑐𝑝𝑢 𝑡𝑜𝑡𝑚𝑒𝑚
, 𝑆𝑡 ,
𝑖 𝑖 𝑖
𝑢𝑠𝑒𝑑 𝑢𝑠𝑒𝑑
𝑆𝑡 𝑐𝑝𝑢 , 𝑆𝑡 𝑚𝑒𝑚 }. 𝑆𝑡𝑠𝑡𝑎𝑡𝑢𝑠
∈ {𝑎𝑐𝑡𝑖𝑣𝑒, 𝑖𝑛𝑎𝑐𝑡𝑖𝑣𝑒} is the status of the server
5.2.4. Intelligent and dynamic placement 𝑖 𝑖 𝑖

An important step when applying reinforcement learning to a prac- 𝑃 𝑀𝑖 at the instant t. Allocating an 𝑆𝐿𝑘 to a 𝑉 𝑀𝑗 changes the state of
tical problem is the identification of the state space, action space, and this VM and thus the PM hosting it.
reinforcement signal (reward function). Action space: For each migrated SL, the action space is defined by
State space: The state space which is denoted by 𝑆𝑡 = {𝑆𝑡1 , 𝑆𝑡2 , … , a binary variable (PLACE/NOT) indicating if the SL will be placed into
𝑆𝑡𝑁 } is formed by the detailed information of the set of 𝑁𝑃 𝑀 a 𝑉 𝑀𝑗 for all the preselected VMs in the previous step if that VM has
𝑃𝑀
elements at time step t. Each element of the state space is given by enough resources to accommodate the SL. We consider all the VMs in

7
L. Helali and M.N. Omri Sustainable Computing: Informatics and Systems 41 (2024) 100938

Algorithm 2: Intelligent RL-based Allocator

Input: The state set: 𝑆 = 𝑠𝑡𝑖 , 𝑖 = 0, 1, ..., 𝑁𝑃 𝑀
The action set: 𝐴 = 𝑎𝑗 , 𝑗 = 0, 1, ..., 𝑁𝑉 𝑀
The reward, 𝑅 = 𝑓 (𝑆𝑖 , 𝑎𝑗 )
The action selection policy, 𝑄(𝜖 − 𝑔𝑟𝑒𝑒𝑑𝑦)
Output: Q_table
1 Begin
2 initQtable() ; // initialize the Q_table
3 for ep in episodes do
4 generateStates() ; // initialize the
environment
5 while not_final_state do
6 𝑠𝑡 =newState()
7 // the result of the
SList=getSelectedSLs() ;
monitoring step to select the SL to
migrate from UL/OL PMs
8 for SL in SList do
Fig. 7. An illustrative example of the pre-selection step. (For interpretation of the
9 VMList=getVMCandidates() ; // the list of
references to color in this figure legend, the reader is referred to the web version of
this article.)
preselected VMs
10 Choose a suitable action 𝑎𝑡 from A for the state
𝑠𝑡 ; // the action that verify the
resources and compliance
the DC to define the action space and, for a given SL, we evaluate only
constraints
the allocation to VMs from the preselected list. If a 𝑉 𝑀𝑗 is not hosted
11 Perform the action 𝑎𝑡
in a 𝑃 𝑀𝑖 , we consider the combination of the state 𝑆𝑡𝑖 and action 𝑎𝑗 as
12 Observe the resulted reward 𝑟𝑡 and the next state
impossible.
𝑠𝑡+1
Reward function: In each step, after executing an action a and
13 Update the Q_table
performing a new allocation, the agent receives a reward granted by 14 𝑄(𝑠𝑡 , 𝑎𝑡 ) = (1 − 𝛼)𝑄(𝑠𝑡 , 𝑎𝑡 ) + 𝛼(𝑟𝑡 + 𝛾𝑚𝑎𝑥𝑎 𝑄(𝑠𝑡+1 , 𝑎))
the environment at time t, which is given by: 𝑠𝑡 ← 𝑠𝑡+1
1 15 update the learning rate
𝑅= (40)
[ 13 (𝐸 + 𝐶 + 𝑅𝑊 )] 16 update the discount factor
The reward is expressed depending on the objective function seeking 17 return 𝑄_𝑡𝑎𝑏𝑙𝑒
to minimize energy consumption, costs, and resource wastage. 18 End
Q-learning based allocation algorithm: Our RL algorithm (see
Algorithm 2) is based on Q-learning [18] when the agent computes and
stores a Q-values Q(s,a) according to the Eq. (41) in a matrix called
the Q-table for each state s and action a in each step. 𝛾 and 𝛼 are and choose the action that maximizes the Q_value with a probability
positive values less than 1 that, successively, represent the discount 1 − 𝜖 (case 2).
factor determining the importance of the predicted future reward and Supposing 𝛼 = 0.3, 𝛾 = 1, 𝜖 = 1, and 𝑅𝑡 (0) = 0.3, 𝑅𝑡 (1) =
the learning rate. 0.8, 𝑎𝑛𝑑𝑅𝑡 (4) = 0.5 are the reward values after executing the three
corresponding actions (selection of the VM_0, VM_1 or VM_4 to place
𝑄(𝑠, 𝑎) = (1 − 𝛼)𝑄(𝑠, 𝑎) + 𝛼(𝑟 + 𝛾𝑚𝑎𝑥𝑎 𝑄(𝑠′ , 𝑎)) (41)
the migrated SL, respectively) in the actual state.
The Q-table is formed by the state space in the row and the action Case 1: a random action is chosen. If the agent chooses the action
space in the column. For each PM in the state space, all the VMs in that corresponds to VM_0, the suitable Q_value in the Q_table matrix
the DC are represented in a column and only the cases that reflect a will be updated using the Eq. (41) as follows:
potential allocation are evaluated. To explain more our strategy, the 𝑄(𝑠𝑡 , 0) = (1 − 0.3) ∗ 0.5 + 0.3 ∗ (0.3 + 1 ∗ 𝑚𝑎𝑥(0.5, 0.8, 0.4))
cases when a given VM is hosted by a PM are considered for potential 𝑄(𝑠𝑡 , 0) = 0.68.
allocations and the other cases in the Q-table are frozen. Also, only Case 2: the action that guarantees a maximum Q_value is chosen
the pre-selected VMs are considered and for the others, the given Q- that corresponds to action 1, that selects the 𝑉 𝑀_1. Thus, the suitable
value does not change. This allows important potential savings when Q_value is updated as folow:
considering only the VMs that run a software instance with the same 𝑄(𝑠𝑡 , 1) = (1 − 0.3) ∗ 0.8 + 0.3 ∗ (0.8 + 1 ∗ 𝑚𝑎𝑥(0.5, 0.8, 0.4))
license as the migrated software. Moreover, in this article, we use the
𝑄(𝑠𝑡 , 1) = 1.04.
𝜖-greedy policy to ensure the trade-off between the exploration when
the agent tries to explore new allocation possibilities and exploitation
when it uses the knowledge already acquired. 5.2.5. Compliance checking
Fig. 8 shows the mechanism of Q_table functioning and updating. In situations of non-compliance where the consolidator tends to
Supposing that we have a software license S1L1 to migrate. In the perform the allocation of an SL to a machine that reached the limit
example given by Fig. 8, if we consider that both VM1 and VM4 run of the allowed number of licenses of this migrated software, an alert
other instances of the software S1, these VMs, are selected as potential process is triggered according to Fig. 9. Once receiving this alert, the
destinations in the pre-selection step. After expanding the search space, consolidation agent chooses another allocation that preserves compli-
we suppose that VM0 is added to the pre-selection list. To decide the ance. This checking process allows us to claim the non-conformity
best destination for the migrated software license S1L1, depending on 𝜖, behavior to choose another compliant allocation, but also it allows
the RL agent will explore new possibilities and choose a random action to account for the number of non-compliant placements that helps to
with a probability 𝜖 (case 1), or exploit the already learned knowledge estimate the potential losses.

8
L. Helali and M.N. Omri Sustainable Computing: Informatics and Systems 41 (2024) 100938

Fig. 8. An example of Q_table functioning and updating. In this example, we have 3 PMs represented in rows of the Q_table and 7 VMs in columns. If a VM is not hosted by
a PM, the box at the intersection of the corresponding row and column is colored in black. Otherwise, this later represents the Q_value which is updated through time by the
intelligent agent.

Fig. 9. Compliance checking process.

6. Simulation and results analysis small scale, we consider a DC with 30 PMs, 100 VMs, and 300 SLs (30
software and 10 licenses). For the wide scenario, we consider 100 PMs,
In order to reach an optimal data center state in terms of energy 300 VMs, and 1000 SLs (100 software and 10 licenses). Moreover, the
consumption, overall cost, and resource utilization, we implemented overload and underload threshold values are 0.8 and 0.4 respectively.
the proposed scheme and evaluated its performances as will be de- For each PM, we assume a network bandwidth of 1 GB/s. Regarding
tailed below. Firstly, we present the research questions we tend to VM pricing, we use the popular Pay-as-you-go model. M1 medium
answer through extensive simulation, then we detail the setting of the instances VM types are used in the homogeneous scenario, which are
parameters before presenting the results and their analysis. charged at $0.120 per VM per hour. For the heterogeneous scenario,
The main research questions addressed in our work are: we used Azure VMs5 offering the flexibility to virtualize a wide range
of solutions supporting windows server, oracle, SAP, etc.
• How does the proposed intelligent approach perform compared
For software pricing, inspired by App Service pricing6 of Azure, we
to the baseline heuristics and the initial DC state while combined
generated the software license fees randomly in [0,1). Furthermore,
with the migration strategy?
we used the same method as in [12] to generate synthetic workload
• To what extent can our approach improve costs, resource utiliza-
instances. Through normalization, we set the total CPU and memory
tion and energy consumption in different cloud DC scales with
utilization of a PM as 1 and the total CPU and memory utilization
homogeneous and heterogeneous resources?
of a VM as 0.25. For each software, a number of licenses are created
• How does our consolidation solution ensure compliance with the
(initially 10), and their costs are also randomly generated as mentioned
processor-based licensing metric?
above. To simplify our model, let us assume that all software running
in the Dc is licensed. We generated 500 software and created variable
6.1. Simulation environment
licensed instances.
Finally, inspired by some best practices like [24] and [25], our
6.1.1. Setup
intelligent RL agent was trained using a learning rate of 0.3, a discount
The proposed solution was implemented in java language, and the
factor, and the exploration/exploitation factor of 1 that are decreased
simulations are carried out on Intel® CoreTM i3-8100 CPU @ 3.60 GHz,
by 99% during the learning process.
8 GB RAM, Windows 10. In our experiments, we consider homogeneous
and heterogeneous resource configurations as shown in Tables 2 and 3.
6.1.2. Compared algorithms
In the homogeneous scenario, all the servers are of type Haswell (Xeon
For the sake of comparison, regarding the SL selection for migration,
2695). The used VMs and their prices are presented in Table 3 and
we compared our MLC policy by:
here [12]. To take the values of power consumption of different types of
hosts used in our experiments at different utilization levels we used the 1. Maximum Usage (MU) [26]: this strategy tends to select the
SPECpower benchmark.4 For each scenario, to meet the real need for software license having the maximum utilization of CPU.
computing resources, we consider the DC size and experiment with our
proposed schemes in small and relatively large-scale resources. For the
5
https://azure.microsoft.com/en-us/pricing/details/virtualmachines/
windows/
4 6
https://www.spec.org/ https://azure.microsoft.com/en-us/pricing/details/app-service/windows/

9
L. Helali and M.N. Omri Sustainable Computing: Informatics and Systems 41 (2024) 100938

Table 2
Power consumption model for the used physical hosts.
Server type 𝑃 𝑠𝑡 (Wh) 𝑃 𝑓 𝑢 (Wh) Number of cores Number of CPU
Haswell (Xeon 2695) 70 120 14 2
Ivy Bridge (Xeon 2670) 65 115 10 2
Sandy Bridge (Xeon 2670) 55 105 8 2

Table 3
Virtual machines pricing.
Instance type Price ($/h)
DS11-1 v2 0.149
G1 0.49
E8a v4 0.504

2. Minimum Migration Time (MMT) [26]: this is also an adapted

version of the MMT policy to the software license consolida-
tion. This policy tends to select the SL that has the minimum
migration time, the ratio between the amount of memory of the
𝑆𝐿𝑘 (𝑁𝑈𝑘𝑚 ) and the bandwidth 𝐵, which is calculated using the
Eq. (42) below.
𝑁𝑈𝑘𝑚
𝑇𝑚𝑖𝑔 = (42)
𝐵
Regarding the destination selection step, we compared the perfor-
mance of our intelligent dynamic placement approach with the baseline
heuristic algorithms proposed here [12]. The proposed consolidation
solutions are compared to the initial DC state when no consolidation is
performed (NO).
Fig. 10. Energy consumption results.

6.1.3. Evaluation metrics

1. Energy consumption: This key metric given by Eq. (13) depends
on the number of active machines and the related utilization
of resources. It depends also on the power consumption of
homogeneous and heterogeneous machines as shown in Eq. (11).
2. Cost: We use this metric to estimate the total cost given by
Eq. (28), the total VM cost and total license cost to show the
effects of our MLC policy on these cost components.
3. Total resource wastage: This metric obtained by Eq. (24) esti-
mates the wasted resources to show the behavior of our proposed
scheme on the resource utilization maximization.
4. Number of active PMs: This metric allows us to show the impor-
tance of our consolidation scheme on the DC optimization by
reducing the number of used PMs.
5. The number of running VMs: This metric can impact the opera-
tional costs, especially the total VM costs.

6.2. Results and analysis

In the following, we present the outcomes of our scheme and the

analysis of the results to verify the assumed claims above. Fig. 11. Energy consumption RL training results.

6.2.1. Energy consumption results

Concerning the energy consumption experimental results (Figs. 10 gain varies between 62.4% in small-scale heterogeneous scenario and
and 11), in the majority of cases, the MU policy, combined with 71.38% in the large-scale homogeneous scenario when combined with
the License-adapted First Fit (LAF) and Cost-Power Efficient (CPE) MMT policy. For the heuristic solutions, the License Adapted Best Fit
heuristics and RL approach gave the best results in terms of total Decreasing (LAB) approach gave the worst results since it selects the
energy consumption. The intelligent approach gave the best savings machines which have the lower resource capacity of CPU, which limits
that reach 68.17% in the large-scale heterogeneous scenario. This is the consolidation possibilities and leads to the use of the maximum
due to the principle that this policy migrates the SLs with the maximum number of PMs (see Fig. 12).
utilization, and that allows more chance to maximize the utilization The CPE approach, based on LAB, surpasses this later since it selects
of active VMs. Especially in the heterogeneous scenario, if the target the most power-efficient PM destination. It realized 45.1% gains in
PMs have minimum power consumption values this leads to minimized the small-scale heterogeneous scenario and uses less number of hosts.
energy consumption. Furthermore, we noticed that our intelligent RL In homogeneous small scale, while combined with the MU policy,
approach gave the best results in terms of energy consumption than the LAF heuristic gave similar results or slightly surpasses the CPE
the heuristics in all the DC dimensions and resource scenarios. Their approach in terms of energy. This is because the CPE approach selects

10
L. Helali and M.N. Omri Sustainable Computing: Informatics and Systems 41 (2024) 100938

Fig. 12. Number of active PMs results. Fig. 13. Total cost results.

the most power-efficient PMs only from the machines with the lower
capacity of CPU resources. In the LAF approach, however, the first
available host that can accommodate the SL in question is selected,
and not necessarily the fullest one. Even if this led to executing more
migrations to reach an optimal state, this heuristic realized important
savings that reached 45.1% in the small-scale heterogeneous scenario.
As we can see in Fig. 11 in the RL training results, in the small-
scale DCs, the RL agent converges rapidly to the optimized results (at
around 2000 iterations). In large scenarios, however, it takes more
time (around 5000 iterations). This is due to the size of the DC when
the wide scenario needs more time to reach an optimal state. In the
homogeneous scenario, the MMT policy gave the best energy values as
it selects the SLs with minimum memory utilization, which is correlated
to CPU utilization. Thus, when combined with the RL approach using
the 𝜖-greedy policy that selects the best action with the maximum
reward when the exploitation rate is important gives the best results.
This can be seen in the training results of Fig. 11, especially, in the last
iterations when the exploration rate decreases and the agent exploit the
acquired competencies. Moreover, the results (Fig. 12) in terms of the
used number of PMs are equivalent to the energy consumption results
since the energy consumption is proportional to the CPU utilization and
Fig. 14. License cost results.
the amount of power consumed by these PMs.

6.2.2. Cost results

Regarding costs, it is worth noting that the total cost results (Fig. 13) The LAB heuristic showed the worst results in all cases because it
are similar to the license cost (Fig. 14), which represent the most uses the maximum amount of PMs and also VMs (Fig. 17), which
important part of the total cost. Furthermore, the MLC policy represents maximizes the total cost due to the inherent VM cost. Their best gains
the best results in terms of license cost and total cost while combined are estimated by 49.81% savings in terms of license costs and 45.49%
with all the allocation policies in all scenarios as shown in Fig. 13. in terms of total costs in the large-scale homogeneous scenario. LAF
This is due to the fact that it migrates the most expensive software and CPE heuristics showed similar behaviors in the majority of the
license which, once finding a destination machine that accommodates scenarios when the savings reached 57.71% and 53.9% in small-scale
at least one other instance of software with the same license, is paid heterogeneous scenario scenarios in terms of license costs (see Fig. 14)
once according to the processor-based licensing model adopted. This and total costs successively. Fig. 8 confirms the findings below and
allows saving the cost of the most expensive SLs independently of the shows that the MLC policy gave the best results during the training of
placement strategy. the RL agent with remarkable savings compared to the MMT and MU
The MLC policy, while combined with our RL approach (see Fig. 16), policies. These gains increase with the agent lifecycle. The total VM
allowed giving the best savings that vary from 62.58% savings, in costs (Fig. 15) is proportional to the number of running VMs as shown
small scale homogeneous scenario, to 73.49% savings, in the wide in Figs. 13 and 17, and the gains reached 68.66% in the large-scale
scenario, with homogeneous resources in terms of license cots, and homogeneous scenario with 68.66% savings in the number of running
from 61.64% savings to 71% savings in terms of total cost in small scale VMs by RL_MMT scheme. It is worth highlighting that the results in
heterogeneous and large scale homogeneous scenarios successively. terms of total VM costs and the number of used VMs are identical. As

11
L. Helali and M.N. Omri Sustainable Computing: Informatics and Systems 41 (2024) 100938

Fig. 17. Number of running VMs results.

Fig. 15. VM cost results.

Fig. 18. Total resource wastage RL training results.

Fig. 16. Total Cost RL training results.

the number of VMs decrease, the user monetary costs and the total cost minimum number of machines is used more efficiently, the amount
decrease, which guarantees a win–win strategy for both the client and of resource wastage will be minimized, which is the case with the RL
the provider. approach. The heuristic algorithms and RL approach showed similar
Our approach also allows cost minimization, not only in heteroge- behavior in terms of wasted resources. The LAB approach always gave
neous scenarios but also in homogeneous scenarios. This minimization slightly high values because it allows using the maximum number
is ensured by the fact that the total cost depends, mainly, on the cost of machines that are less utilized causing a less efficient use of the
of the licenses which vary according to the instance of the software resource. The best savings are noted in large-scale scenarios that prove
to which this license belongs and also on the model metric used to the scalability of our approaches and the efficient use of resources that
generate this license. increase with the size of the DC scenarios. The gains reached 77.45%
in the wide homogeneous DC.
6.2.3. Resource wastage results
From Fig. 18, which shows the training results of the RL agent, we 6.3. Compliance check results
notice that with some fluctuations at the beginning, the MU policy
converges to optimized results of resource wastage, especially with Fig. 19 shows the results of the compliance checking algorithm,
heterogeneous resources. This is due to the fact that migrating the which tends to verify if an allocation is compliant with the processor-
SLs with the highest CPU utilization allows better utilization of the based licensing model and report the non-compliant tentatives of place-
destination machines. With the heterogeneous scenario, some target ment. The checking results are transferred to the allocator that decides
machines could accommodate more SLs, which maximizes their uti- to evaluate another potential allocation in case of non-compliance.
lization and thus minimizes wasted resources. In addition, when the We can notice that the LAB heuristic showed the highest number of

12
L. Helali and M.N. Omri Sustainable Computing: Informatics and Systems 41 (2024) 100938

7. Conclusions and future scope

In this article, we have proposed a novel approach for compliance-

aware software license consolidation in the cloud. Our solution allows
efficient use of resources and optimizes energy consumption, costs, and
the waste of resources while guaranteeing compliance. The experimen-
tal results confirmed the effectiveness of our approach in many data
center scales with homogeneous and heterogeneous resources. In fact,
we recorded a savings rate of 71.38% in terms of energy consumption,
73.49% in terms of license cost, 71% in terms of total cost, 68.66% in
terms of VM costs and a rate of 77.45% in terms of wasted resources.
Looking at the other aspects related to the consolidation service,
such as migration costs, performance, affinity, and although our ap-
proach has shown its performance compared to literature work, never-
theless further improvements could be made to this version in order to
increase its performance and guarantee its robustness. Thus, we plan
to extend the current model by considering other licensing metrics
and models. Moreover, if the containerized SLs are co-located in the
same VM, they could pose many security risks in that VM as discussed
here [1]. Therefore, considering intra-VM security when managing elas-
ticity and optimizing software licenses is an important extension of the
present work. We also plan to implement and test other scalarization
functions in our future multi-objective RL work and implement a Pareto
Fig. 19. Compliance checking results. approach that fully meets all considered objectives. Finally, considering
other parameters like SLA violation remains an important extension in
order to study the effects of the energy consumption minimization on
non-compliant situations followed by the LAF heuristic, then the CPE, the performance of the system and seek for a tradeoff.
while the intelligent allocator reported the least number of allocation
tentatives that do not guarantee compliance. It is worth specifying that Declaration of competing interest
the software licenses and compliance management in the cloud are
in their early days. According to a survey published in the Flexera The authors declare that they have no known competing finan-
company report, 52% of organizations say that understanding the cial interests or personal relationships that could have appeared to
financial implications of software licensing is a challenge for software influence the work reported in this paper.
in the cloud. Also, there are no standards to model the software licenses
and compliance and to quantify the penalties and losses due to non- Data availability
compliant situations. Each company has its peculiarities regarding this
concern. Nevertheless, we are aware that compliance assurance from an Data will be made available on request.
early stage during resource optimization and scalability is an important
References
strategy to avoid the penalties of non-compliant situations and the
generated economic losses, which can also affect the reputation of
[1] L. Helali, M.N. Omri, A survey of data center consolidation in cloud computing
enterprises. systems, Comp. Sci. Rev. 39 (2021).
[2] A.L. Vion, Software Asset Management and Cloud Computing. (Gestion du pat-
rimoine logiciel et Cloud Computing) (Ph.D. thesis), Grenoble Alpes University,
6.4. Discussion France, 2018.
[3] A. Sen, A. Garg, A. Verma, T. Nayak, CloudBridge: On integrated
According to the results presented and analyzed previously, we can hardware-software consolidation, ACM SIGMETRICS Perform. Eval. Rev. 39
(2011).
say that the intelligent software license allocator offers the best results [4] M.K.M. Murthy, M.N. Ameen, H.A. Sanjay, P.M. Yasser, Software licensing mod-
in terms of energy, waste of resources, and costs when combined with els and benefits in cloud environment: A survey, in: Proceedings of International
the MLC migration strategy. In our approach, we considered actions Conference on Advances in Computing. Advances in Intelligent Systems and
related to pre-selected VMs in each allocation only, which saved time Computing. Vol. 174, Springer, New Delhi, 2013, pp. 645–650.
[5] F. Magnanini, L. Ferretti, M. Colajanni, Efficient license management based on
and memory. Similarly, determining a list of pre-selected VMs for smart contracts between software vendors and service providers, in: 2019 IEEE
each SL can be greedy, especially if the number of SLs is so large. In 18th International Symposium on Network Computing and Applications, NCA,
addition, in some cases, even if the available resources are sufficient IEEE, 2019, pp. 1–6.
to host the migrated SL, if the number of licenses authorized in a [6] D. Yimam, E.B. Fernandez, A survey of compliance issues in cloud computing,
J. Internet Serv. Appl. 7 (2016).
given machine is already used, it is not possible to place this SL in [7] Oracle, Database licensing, 2020, https://www.oracle.com/assets/
this machine. This conflicting constraint can affect the results and limit databaselicensing-070584.pdf.
the possibilities of consolidation. In addition, there are non-compliant [8] A. Yousefipour, A.M. Rahmani, M. Jahanshahi, Energy and cost-aware virtual
consolidation attempts (which do not respect compliance) that reach machine consolidation in cloud computing, Softw. - Pract. Exp. 48 (2018).
[9] A.S. Andrae, T. Edler, On global electricity usage of communication technology:
16147 in a small-scale heterogeneous scenario and 215570 in a large-
Trends to 2030, Challenges 6 (2015) 117–157.
scale homogeneous scenario by the LAB_MLC scheme. These attempts at [10] A. Chevalier, Optimization of Software License Placement in the Cloud for
consolidation, if effectively executed, will result in huge losses in terms Economical and Efficient Deployment (Ph.D. thesis), University of Lyon, France,
of the cost caused by non-compliance. Finally, the MLC policy assumes 2021.
[11] L. Helali, M.N. Omri, Heuristic-based approach for dynamic consolidation of
the existence of other instances of software with the same license in
software licenses in cloud data centers, Int. J. Intell. Syst. Appl.(IJISA) 13 (2021).
the destination VM. If this constraint is not met, the migration could [12] L. Helali, M.N. Omri, Software license consolidation and resource optimization
be more expensive due to the costs of the migrated SL. in container-based virtualized data centers, J. Grid Comput. 20 (2022).

13
L. Helali and M.N. Omri Sustainable Computing: Informatics and Systems 41 (2024) 100938

[13] M. Barati, O. Rana, Tracking GDPR compliance in cloud-based service delivery, [23] F. Hermenier, X. Lorca, J.M. Menaud, G. Muller, J. Lawall, Entropy: A consoli-
IEEE Trans. Serv. Comput. 15 (3) (2022) 1498–1511. dation manager for clusters, in: Proceedings of the 2009 ACM SIGPLAN/SIGOPS
[14] Z.A. Mann, Resource optimization across the cloud stack, IEEE Trans. Parallel International Conference on Virtual Execution Environments, ACM, 2009, pp.
Distrib. Syst. 29 (2018) 169–182. 41–50.
[15] A. Tchana, N.D. Palma, I. Safieddine, D. Hagimont, Software consolidation as an [24] M. Kim, J.-S. Kim, M.-S. Choi, J.-H. Park, Adaptive discount factor for deep
efficient energy and cost saving solution, Future Gener. Comput. Syst. 58 (2016) reinforcement learning in continuing tasks with uncertainty, Sensors 22 (2022).
1–12. [25] L. Zhang, L. Lengersdorff, N. Mikus, J. Gläscher, C. Lamm, Using reinforcement
[16] S. Azizi, M. Shojafar, J. Abawajy, R. Buyya, GRVMP: A greedy randomized learning models in social neuroscience: Frameworks, pitfalls and suggestions of
algorithm for virtual machine placement in cloud data centers, IEEE Syst. J. best practices, Soc. Cogn. Affect. Neurosci. 15 (6) (2020) 695–707.
15 (2) (2021) 2571–2582. [26] S.F. Piraghaj, Energy-Efficient Management of Resources in Container-based
[17] M. Hussein, M. Mousa, M. Alqarni, A placement architecture for a container as Clouds (Ph.D. thesis), The University of Melbourne, Australia, 2016.
a service (caas) in a cloud environment, J Cloud Comp 8 (2019).
[18] W. Ding, F. Luo, C. Gu, H. Lu, Q. Zhou, Performance-to-power ratio aware Leila Helali is a Lecturer in Computer Science at the University Of Sousse, Tunisia.
resource consolidation framework based on reinforcement learning in cloud data Her research interests include reinforcement learning; cloud computing, compliance
centers, IEEE Access 8 (2020) 15472–15483. and resource optimization. Moreover, she has enough knowledge of applied machine
[19] Y. Qin, H. Wang, S. Yi, X. Li, L. Zhai, Virtual machine placement based on learning, distributed systems and computer programming. She is a reviewer of interna-
multi-objective reinforcement learning, Appl. Intell. 50 (2020) 2370–2383. tional journals such as Computer Science Review, Journal of Grid Computing, Scientific
[20] Y. Li, X. Tang, W. Cai, On dynamic bin packing for resource allocation in the Reports and The Journal of Supercomputing.
cloud, in: Proceedings of the 26th ACM Symposium on Parallelism in Algorithms
and Architectures, 2014, pp. 2–11. Mohamed Nazih Omri received his Ph.D. in Computer Science from University of
[21] S.S. Masoumzadeh, Autonomic Management of Virtual Machines in Cloud Data Jussieu, Paris, France, in 1994. He is a Professor in computer science at the University
Centers Using Machine Learning (Ph.D. thesis), University of Vienna, Austria, Of Sousse, Tunisia. From January 2011, he is a member of MARS (Modeling of
2017. Automated Reasoning Systems) Research Laboratory. His group conducts research on
[22] J. García, R. Iglesias, M.A. Rodríguez, C.V. Regueiro, Directed exploration in Information Retrieval, Data Base, Knowledge Base, and Web Services. He supervised
black-box optimization for multi-objective reinforcement learning, Int. J. Inf. more than 20 Ph.D. and Msc students in different fields of computer science. He is a
Technol. Decis. Mak. (2019). reviewer of many international journals such as Information Fusion journal, Psihologija
Journal, and many International Conferences such as AMIA, ICNC-FSKD, AMAI, etc.

14

View publication stats