Received December 4, 2021, accepted February 4, 2022, date of publication February 9, 2022, date of current version February 17,

2022.
Digital Object Identifier 10.1109/ACCESS.2022.3149955

Multi-Objective Task Scheduling Optimization for

Load Balancing in Cloud Computing Environment
Using Hybrid Artificial Bee Colony Algorithm
With Reinforcement Learning
BOONHATAI KRUEKAEW AND WARANGKHANA KIMPAN , (Member, IEEE)
Department of Computer Science, School of Science, King Mongkut’s Institute of Technology Ladkrabang, Bangkok 10520, Thailand
Corresponding author: Warangkhana Kimpan ([email protected])
This work was supported by the School of Science, King Mongkut’s Institute Technology Ladkrabang.

ABSTRACT Workload balancing in cloud computing is still challenging problem, especially in Infras-
tructure as a Service (IaaS) in the cloud model. A problem that should not occur during cloud access is
a host or server being overloaded or underloaded, which may affect the processing time or may result in a
system crash. Therefore, to prevent these problems, an appropriate schedule of access should be considered
so that the system can distribute tasks across all available resources, which is called load balancing. The
load balancing technique should ensure that all Virtual Machines (VMs) are used appropriately. In this
paper, an independent task scheduling approach in cloud computing is proposed using a Multi-objective
task scheduling optimization based on the Artificial Bee Colony Algorithm (ABC) with a Q-learning
algorithm,which is a reinforcement learning technique that helps the ABC algorithm work faster, called the
MOABCQ method. The proposed method aims to optimize scheduling and resource utilization, maximize
VM throughput, and create load balancing between VMs based on makespan, cost, and resource utilization,
which are limitations of concurrent considerations. Performance analysis of the proposed method was
compared using CloudSim with the existing load balancing and scheduling algorithms: Max-Min, FCFS,
HABC_LJF, Q-learning, MOPSO, and MOCS algorithms in three datasets: Random, Google Cloud Jobs
(GoCJ), and Synthetic workload. The experimental results indicated that the algorithms used MOABCQ
approach outperformed the other algorithms in terms of reducing makespan, reducing cost, reducing degree
of imbalance, increasing throughput and average resource utilization.

INDEX TERMS Cloud computing, hybrid artificial bee colony algorithm, multi-objective task, Q-learning,
task scheduling.

I. INTRODUCTION it, users do not need to purchase any platform or software;

Recently, cloud computing has played an important role in they only have an internet connection to access and pay for
many organizations. Cloud computing started out to provide their usage.
of users requirements for accessing resource computing or Infrastructure as a Service (IaaS) is one of the technology
to enable users to purchase cloud services as required within models behind the management of servers, data centers, and
the concept of on-demand resource sharing through highly Virtual Machines (VMs). IaaS is a cloud service that provides
internet-based applications. Cloud computing can also pro- a cloud computer or server for processing and storing data in
cess many different services depending on the service and the cloud. Users can run any operating system or application
working platforms that are needed by the users [1]. Cloud on a rental server free of maintenance and operating costs of
computing is a combination of distributed and parallel com- hardware. Since the cloud infrastructure is run on VMs, IaaS
puting with the sharing of resources such as software and has another advantage, including providing the access of the
hardware that will be utilized as ‘‘pay-as-you-go’’ [2]. To use user to the servers in nearby locations. This service depends
on the needs of the user, called Quality of Service (QoS)
The associate editor coordinating the review of this manuscript and and Service Level Agreements (SLAs). The payment for the
approving it for publication was Gerard P. Parr. services depends on the agreement between the user and

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.

VOLUME 10, 2022 For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/ 17803
 B. Kruekaew, W. Kimpan: Multi-Objective Task Scheduling Optimization for Load Balancing in Cloud Computing Environment

The cloud resource management framework is shown in

Fig. 1. When a user submits a request into the system, the task
is passed to the cloud broker, which is the main objective that
researchers should focus on providing effective algorithms.
The proposed method should efficiently submit tasks to the
appropriate VM by the user depending on the required param-
eters, such as the deadline or other requirements. It must
ensure that the user submitted the requests that are fulfilled
in accordance with the requirements specified in the SLA
document. Users make their requests over the internet, and
the requests are stored in VMs. Then, the CSPs deliver
QoS-based queries to ensure that the requests of the user
can be processed as intended. This process depends on the
performance of the scheduling policy (Data Broker). In this
step, the workload balance must be adjusted between the
machine and the server. Cloud computing depends highly
on Virtual Machine Monitor (VMM) or hypervisors. Hyper-
visors help operate multiple VMs in the same hardware
layer [7]. VMware is one of the most famous software
services efficiently used for managing server resources in
organizations. Even though virtualization plays an important
role in cloud technology, problems still often arise, such
as improper scheduling or handing over tasks to VMs that
FIGURE 1. Cloud resource management framework. cannot meet the requests. To solve this problem, scheduling
and load balancing between nodes in cloud computing should
be implemented to optimize resource utilization. Scheduling
the Cloud Service Providers (CSPs) [3]. Moreover, Cloud and load balancing between nodes also resulted in a reduction
computing enables CSPs to provide data centers with high- in processing time and an imbalance in the system.
performance computing resources to support users accessing Task allocation to match the right resource, such as
cloud services. time, cost, and success rate, must be considered in cloud
A VM consumes the resources of the host machine: RAM, computing. Many studies have discussed scheduling or
hard disk, or CPU. The resources on the VM are required by optimizing resources in the cloud by single or bi-objective
the requested resource for operation. As a result, the cloud optimization of QoS parameters [8]–[10], and several arti-
network has an unequal resource distribution, and some VMs cles have discussed multi-objective optimization [11]. Exam-
may not access the resources they require because many ples of algorithms that were used to increase the efficiency
VMs have preemptive and non-preemptive connections to of on-demand tasks such as Particle Swarm Optimization
resources [4]. (PSO) [12], [13], Genetic Algorithm (GA) [14], niched Pareto
When a task is sent off to be processed in the cloud, genetic algorithm (NPGA) [15], and strength Pareto evolu-
the VM should be able to operate quickly to reduce wait- tionary algorithm (SPEA) [16].
ing time. However, tasks should be distributed among all The Artificial Bee Colony (ABC) algorithm is a meta-
VMs in parallel to balance the system and ensure efficient heuristic method used to solve problems and find solutions
use of available resources. For this reason, task scheduling for an optimization answer that is close to the appropriate
to be assigned and distributed across existing resources is value developed by Karaboga [17], [18]. The ABC algorithm
necessary. When multiple tasks are assigned to one VM or mimics the foraging behavior of bee colonies, which require
multiple VMs, the assigned tasks will run concurrently to adaptation to habitat and food environments. Several studies
complete the assignments. Therefore, the assignment must have demonstrated the effectiveness of the ABC algorithm in
be ensured that not all tasks are loaded on only a single solving problems such as traveling salesman problem [19],
VM, which will cause other VMs to become inaccessible and job shop scheduling problems [20]. The ABC algorithm
or imbalanced in the system. To avoid this, other conditions works in a strategy similar to Reinforcement Learning (RL)
must be considered in scheduling, such as makespan, cost, with environment-based learning, which derives from obser-
and resource utilization. Several researchers have proposed vation to predict or decide a good solution. Agents learn how
ideas to manage load balancing in both heterogeneous [5] and to behave in an environment by taking action and then observ-
homogeneous environments [6]. The main goal of allocating ing the results for themselves. In this research, we use the
tasksof a system in a load balance state is to optimize the Q-learning algorithm to help the system make predictions and
distribution of tasks across existing resources and to reduce decisions about choosing the appropriate schedule. There-
the system processing time. fore, we propose a multi-objective optimization-scheduling

17804 VOLUME 10, 2022

B. Kruekaew, W. Kimpan: Multi-Objective Task Scheduling Optimization for Load Balancing in Cloud Computing Environment

multi-objective Particle Swarm Optimization (MOPSO)

algorithm, and multi-objective Cuckoo Search (MOCS)
algorithm. The three datasets used in this study are
Random, Google Cloud Jobs (GoCJ), and Synthetic
workload to observe in terms of makespan, cost, Aver-
age Resource Utilization Ratio (ARUR), throughput,
and Degree of Imbalance (DI).
The rest of this article is structured as follows: Section II
discusses the literature review. In Section III, the problem
definition and formulation of the objective function will be
described. The hybrid ABC algorithm for load balancing is
presented in Section IV. In Section V, experimental evalua-
tion and discussion will be explained, and finally, Section VI
FIGURE 2. Q-learning algorithm process. presents conclusions and future work.
model using the ABC algorithm and Q-learning (MOABCQ) II. LITERATURE REVIEW
method for solving the scheduling problem in a cloud com- Optimizing task scheduling and load balancing in cloud com-
puting environment. The paper focuses on developing a puting environments is known as an NP-hard problem [21].
multi-objective scheduling approach to optimize job schedul- Many studies have proposed optimization algorithms in cloud
ing with the goal of minimizing makespan, cost, and simulta- environment in various aspects, such as resource allocation,
neous use of resources, also considering the load balance of scheduling and load balancing procedures to reduce uptime,
the system. and system power consumption. In terms of efficient use of
The main contributions of the proposed method can be existing resources or fast processing, we need to consider an
described as follows: idea in scheduling multiple objectives or cloud environments.
• Formulate mathematical models for calculating Therefore, many ideas have been proposed using heuris-
resource utilization, makespan and cost for scheduling tic algorithms [22]–[29], meta-heuristic algorithm [30]–[37],
problems in cloud computing. hybrid meta-heuristic algorithm [14], [38], or even machine
• Propose a multi-objective optimization scheduling learning methods [39]–[41] to solve task scheduling and load
model. This model consists of 3 objective functions balancing problems in the cloud computing environment. The
defined as 1) execution time, 2) cost, and 3) utilization related works are described as follows:
of resources. These are constraints in the proper task Many studies have used a heuristic algorithm to solve
scheduling and the load balancing of the system. problems in a cloud computing environment. For example,
• Propose a new approachthat is multi-objective opti- A Max-Min based task scheduling algorithm was proposed by
mization for the task scheduling problem using Mao et al. [22] to balance the load in the cloud and to predict
the Artificial Bee Colony algorithm (ABC) adapted the execution time of tasks. Then, the tasks were submitted
with the implementation of the Q-learning algorithm, to the VM based on their proposed Max-Min algorithm.
namely, MOABCQ, to optimize the multi-objective The VM utilized more resources and decreased response
scheduling problem in cloud computing and to reduce time. Patel et al. [24] proposed an Enhanced Load balanced
the makespan, cost, and simultaneous use of resources. Min-Min (ELBMM) algorithm for static task scheduling.
This new proposed method is combined with the First Jobs were assigned to VMs based on execution time, and the
Come First Serve (FCFS) heuristic task scheduling tasks were rescheduled to distribute to resources that idle.
called ‘‘MOABCQ_FCFS’’ and Largest Job First (LJF) Zhang et al. [25] studied the application of heuristic
heuristic task scheduling called ‘‘MOABCQ_LJF’’. algorithms in cloud scheduling problems. They proposed a
• Performance evaluation of our proposed method exam- method for scheduling tasks to minimize the task comple-
ined through CloudSim simulation. In this paper, tion time and execution cost (MCTE) in a smart grid-cloud
we improved the classical Q-learning algorithm for system. Then, they performed mathematical modeling for the
load balancing in the cloud environment by combining grid-cloud task scheduling problem.
it with ABC. This idea can improve the convergence Hussain et al. [26] proposed a resource-aware load-
rate, gain efficiency in load balancing, determine indi- balancing algorithm (RALBA), which is a dynamic schedul-
vidual VM loads, and balance them through the fitness ing technique that maps independent and non-preemptive
function. tasks to VMs. The process consisted of 2 parts. The
• The proposed method was compared with other first part was selecting the maximum size for the VM
optimization algorithms, such as: First Come First with the highest computational share. The second part was
Serve (FCFS) algorithm, Max-Min algorithm, Heuris- mapping the remaining tasks to the fastest working VM
tic Task Scheduling with ABC with Largest Job to perform. The results showed that RALBA was able to
First algorithm (HABC_LJF), Q-learning algorithm, reduce makespan. However, there were some problems with

VOLUME 10, 2022 17805

 B. Kruekaew, W. Kimpan: Multi-Objective Task Scheduling Optimization for Load Balancing in Cloud Computing Environment

load imbalance and reducing resource utilization. The load environment, the Heuristic Task Scheduling with Artificial
balancing approach was proposed to be modified using soft Bee Colony (HABC) Algorithm was proposed to schedule
computing-based stochastic hill climbing based on load bal- and manage cloud resources by Kruekaew and Kimpan [43].
ancing by Mondal et al. [27]. The experimental results showed that HABC with the Largest
A dynamic multi-workflow scheduling algorithm named Job First heuristic algorithm (HABC_LJF) was the most
the competent dynamic multi-workflow scheduling efficient in scheduling and managing cloud resources.
(CDMWS) algorithm was proposed by Adhikari and Gan et al. [44] proposed job scheduling using a genetic
Koley [28]. This method aimed to improve CPU utilization, simulated annealing algorithm. The primary purpose was
reduce makespan, and improve the makespan-deadline meet- to optimize the execution time of data center tasks.
ing ratio. This method was classified into 2 parts. The first Basu et al. [14] introduced a hybrid meta-heuristic algorithm
part served to estimate the processing time for each task based called GAACO, which was a combination of the GA algo-
on deadline and task dependencies. The second part was rithm and ACO algorithm to solve the task scheduling of
responsible for allocating VMs to reduce power consumption IoT applications in a multiprocessor cloud environment.
and increase resource utilization. The experiments showed The method guaranteed appropriate convergence when tested
that CDMWS outperformed the Enriched Workflow Schedul- with sizes of task graphs and number of different proces-
ing Algorithm (EWSA) and Round Robin (RR) algorithm. sors in terms of makespan and efficiency. They found that
Efficient resource allocation with a focus on solving the GAACO algorithm performed better than the GA algo-
energy efficiency problems using the multi-objective opti- rithm and ACO algorithm in a heterogeneous multiprocessor
mization (MOO) method was discussed by Shrimali and environment.
Patel [29]. A VM allocation approach has also been proposed For a multi-objective task scheduling problem in a cloud
using MOO. According to the experimental results, MOO led environment that is solved using a meta-heuristic algorithm
to energy savings due to the efficient allocation of resources and a hybrid meta-heuristic algorithm, Alsadie [45] proposed
without affecting the performance of the data center. a meta-heuristic framework for dynamic virtual machine allo-
Many studies have used a meta-heuristic algorithm and a cation with optimized task scheduling in a cloud computing
hybrid meta-heuristic algorithm to solve problems in a cloud environment (MDVMA). MDVMA was a multi-objective
computing environment. For example, Tawfeek et al. [35] dis- scheduling method using a non-dominated sorting genetic
cussed the adoption of the Ant Colony Optimization (ACO) algorithm to help reduce cost, makespan, and energy usage.
algorithm for reducing the execution time of tasks in a cloud In the experiments, theHeterogeneous Computing Schedul-
computing environment. The authors demonstrated that using ing Problems (HCSP) dataset was used with the CloudSim
ACO was efficient. ACO worked better than other methods, Simulator. The results showed that MDVMA was able to
such as the RR algorithm and FCFS algorithm. To reduce the optimize task scheduling better than the ABC algorithm,
makespan and optimize resource access in cloud computing, Whale Optimization Algorithm (WOA), and PSO algorithm
an improved PSO algorithm has been developed. The method in terms of reducing the cost, makespan, and energy usage of
used updating the weights of particles with the evolution of the cloud data center.
the number of iterations and to inject some random weights in Guo [34] proposed cloud computing multi-objective task
the final stages of the PSO. The objective was to avoid local scheduling optimization based on a fuzzy self-defense
optimum solutions being generated in the final stages of PSO algorithm, which had good performance in terms of the
was proposed by Luo et al. [12]. shortest completion time, deadline violation rate, and
Chen et al. [36] proposed the dynamic objective genetic utilization of virtual machine resources when compared
algorithm (DOGA) by focusing on optimizing the execution with A Multi-Objective Optimization Scheduling Method
time according to the deadline constraint to help reduce Based on the Ant Colony Algorithm in Cloud Computing
the cost of work and to work within the specified time. (PBACO) [33] and Task Scheduling Algorithm Based on
Amini et al. [37] proposed the resources allocation process RL. Zuo et al. [33] proposed a multi-objective optimization
for virtual machines in cloud computing using dragon- scheduling method in terms of efficiency and budget costs.
fly optimization algorithm. The experimental showed that This approach used an ant colony algorithm (PBACO) to
using dragonfly optimization algorithm helped improve task help determine the optimal solution. The experiment was
scheduling, load balancing, and resource allocations bet- compared with the classical heuristic algorithm, Min-Min
ter than ACO algorithm and Hybrid Algorithm Based on algorithm, and FCFS scheduling. PBACO was found to be
Particle Swarm Optimization and Ant Colony Optimization superior to the other comparison methods.
Algorithm (ACO-PSO). He et al. [46] proposed Adaptive Multi-objective Task
Li and Han [42] proposed a hybrid discrete ABC algo- Scheduling (AMTS) to try to improve resource utilization,
rithm for flexible task scheduling problems in a cloud sys- cost, energy consumption, and task completion time. AMTS
tem. The objective of this approach was to minimize the used a PSO-based approach for multi-objective scheduling
maximum completion time, total workloads of all devices, that considered process time and transmission time.
and maximum device workload. In terms of reducing the The proposed method applied the adaptive acceleration
completion time and balancing load in the cloud computing coefficient for particle diversity. After improving the PSO

17806 VOLUME 10, 2022

B. Kruekaew, W. Kimpan: Multi-Objective Task Scheduling Optimization for Load Balancing in Cloud Computing Environment

TABLE 1. Simulation environment. The advantages of this algorithm were the fast convergence
speed and strong search ability. The algorithm was compared
with EDA and GA using the CloudSim simulation experi-
ment platform. According to the testing results, the EDA-GA
hybrid algorithm effectively reduced job completion time and
improved load balancing ability.
Neelima and Reddy [52] proposed a load balancing task
scheduling algorithm in cloud using Adaptive Dragonfly
algorithm (ADA) which was a combination of dragonfly
algorithm (DA) and firefly algorithm (FA). The development
of a multi-objective function was based on three parameters:
completion time, processing costs, and load. The perfor-
mance of this method was measured in terms of execution
cost and time. When compared to DA and FA, the experi-
mental results showed that the proposed approach achieved
better load balancing results.
There is also research combining osmotic behavior with
bio-inspired algorithms, for example, Gamal et al. [53]
presented Osmotic hybrid artificial bee and ant colony
(OH_BAC) algorithm which derives from the osmosis the-
ory in the chemistry science. This algorithm was used to
form osmotic computing and find load balancing for VM
placement. OH_BAC used an osmosis approach to create a
low-energy cloud computing environment. In this algorithm,
ACO and ABC collaborated to find the optimum VM for
algorithm, AMTS was able to find the best solution for the migrating to the best physical machine (PM). To reduce
cloud-based scheduling problem. power consumption, OH_BAC activated the most appropriate
Jena [47] proposed an ABC-based approach for energy osmotic host among all PMs in the system. The algorithm
efficiency, processing time, cost, and computing resource was compared with ACO, ABC, H_BAC, and host overload-
utilization in the cloud computing environment. Tasks were ing detection algorithms. The experimental results showed
allocated to the data center using a multi-objective ABC that when compared to other algorithms in fixed and vari-
algorithm. An ant-based genetic algorithm was used to solve able loads, OH_BAC improved energy consumption, service
multi-objective scheduling problems to reduce latency and level agreement violation (SLAV), number of virtual machine
completion time while maximizing throughput in the cloud migration, and number of host shutdowns.
environment by Kumar and Venkatesan [48]. The multi- Machine learning algorithms have been used to solve chal-
objective WOA was proposed by Reddy and Kumar [49] to lenging problems in cloud computing environments [54]. For
develop scheduling in cloud computing. The authors tried to example, Farahnakian et al. [39] proposed the Reinforcement
create a fitness-based schedule based on three conditions: Learning-based Dynamic Consolidation method (RL-DC) to
quality of service, energy, and resource utilization. After reduce energy consumption and optimize resource usage in
considering the given parameters, the processing time and cloud data centers.
cost of the virtual machines were found to be reduced. Caviglione et al. [55] introduced a deep reinforcement
Madni et al. [50] proposed an innovative Multi-objective learning-based approach for the placement of VMs in
Cuckoo Search Optimization (MOCSO) algorithm for the cloud data centers. DRL was used to select the best loca-
resource scheduling problem in IaaS cloud computing envi- tion possible for each VM. Jena et al. [56] proposed a
ronment. The major goal of the resource scheduling challenge hybrid meta-heuristic algorithm called QMPSO for balancing
was to reduce cloud user costs and enhance the performance loads between VMs in cloud computing using hybridization
by reducing makespan time. The simulation results showed of modified particle swarm optimization (MPSO) and an
that MOSCO algorithm outperformed Multi-objective ACO improved Q-learning algorithm. QMPSO helped to improve
(MOACO), Multi-Objective GA (MOGA), Multi-Objective the makespan, throughput, and power consumption during
Min-Max (MOMM), and Multi-objective PSO (MOPSO) load balancing and effectively reduced the waiting time of
algorithm. tasks.
Pang et al. [51] developed an EDA-GA hybrid schedul- A framework for cloud resource allocation with the goal
ing algorithm to solve the multi-objective task scheduling of deploying resources in green cloud was proposed by
problem based on EDA (estimation of distribution algorithm) Thein et al. [57]. The proposed framework was based on a
and GA (genetic algorithm). The constraints of scheduling reinforcement learning mechanism and fuzzy logic. Its effi-
problem in this model were scheduling performance and time. ciency was measured by CPU utilization at the data center,

VOLUME 10, 2022 17807

 B. Kruekaew, W. Kimpan: Multi-Objective Task Scheduling Optimization for Load Balancing in Cloud Computing Environment

which measuring Power Usage Effectiveness (PUE) and Data • Let T = {t1 , t2 , t3 , . . . , tn } where T is the set of assigned
Center infrastructure Efficiency (DCiE). Simulation results tasks, and n is the number of independent tasks per-
using CloudSim showed that this framework can achieve formed on the VMs (m). tj represents the jth task. Each
effective performance for high data center energy efficiency task submission contains the number of instructions,
and prevent SLA violation. The goal of the task scheduling memory required, CPU required, etc.
and resource allocation model by hybrid ant colony opti- Output:
mization and deep reinforcement learning is to reduce the
completion time and improve the utilization of idle resources • Optimize scheduling, map n tasks and m VMs (resources)
with the use of a binary in-order traversal tree using weighted to minimize makespan, cost, and resource utilization and
values. For task scheduling, a DRL algorithm was used to find to increase load balancing in resource utilization.
idle resources and ACO to find the most suitable VM for each In general, multi-objective optimization problem consists
task by Rugwiro et al. [41]. of several fitness functions (Objective function) as: F (x) =
As mentioned previously, developing effective algorithms [f1 (x) , f2 (x) , . . . , fobj (x)] where obj is number of objec-
for task scheduling and selecting the right resources in tives, and fi (x) represents ith objective function. This problem
cloud computing environments was found to be very impor- does not have a single solution and a set of non-dominated
tant. Many studies put much effort into organizing tasks solutions can be found, known as a pareto optimal solutions.
or allocating proper resources for each task considering We propose the task scheduling algorithm in cloud
single objective, bi-objective and multi-objective scheduling. computing using conditions of multi-objective scheduling
Therefore, this paper proposed the multi-objective optimiza- approaches to increase the efficiency of schedule optimiza-
tion of the task scheduling problem, while previous studies tion and resource utilization, to maximize VM throughput,
focused mainly on the objectives of makespan or processing and create load balancing between VMs. The most com-
time and cost because both of these objectives meet the mon evaluation factors are cost, energy consumption, task
needs of the users. However, to work in cloud computing, completion time, task waiting time, flow time, failure rate,
it is necessary to consider various conditions or objectives. profit, carbon emission, makespan, and reliability [58]. The
In addition, load balancing in the cloud must be considered. proposed method determines the suitability conditions using
A hybrid meta-heuristic algorithm has been introduced to cost, resource utilization, and makespan as factors that help
help optimize performance in the cloud. Nevertheless, some in proper scheduling. Scheduling and load balancing in a
algorithms are weak in local search, while some algorithms cloud computing environment requires a fitness function
have a weakness in global search optimization. According to design to provide optimal solutions and also considers the
related studies, the ABC algorithm can be helpful in solv- multi-objective used to find the answer. We used weighted
ing the multi-objective task scheduling problem because it sum approach [59], [60] for solving the multi-objective
is capable of explorative behavior. In contrast, exploitative optimization problem. This approach is used to convert a
behavior, which is a part of the onlooker bee, was weak. multi-objective optimization problem to a single objective
In this case, reinforcement learning can help solve this prob- with weights which represents preferences among objectives
lem because Q-learning can improve the solution quality. by the decision maker. We have considered three objectives in
Therefore, we propose a method that can solve the multi- the article as follows:
objective task scheduling problem using the ABC algorithm 1) The first objective is defined in the condition of
and Q-learning (MOABCQ). MOABCQ helps determine the makespan or task execution time, which is the time that the
order of tasks for suitable available resources environments to system completes its last task. The makespan is a useful factor
find the most appropriate task scheduling solution. Moreover, for multi-objective scheduling approaches that can reduce the
MOABCQ also provides a load balancing system. execution time and allow tasks to be completed prematurely.
Each VM has a different execution time for completing the
III. PROBLEM DEFINITION AND FORMULATION OF task determined by the makespan. If a maximum of the
OBJECTIVE FUNCTION execution time value is high and the makespan value is also
Many users operate their works on the cloud environment, high, the system is considered poorly distributed tasks to the
which makes the scheduling process play an important role VMs. However, if the maximum execution time value is low,
in resource utilization, response time, latency, and load the makespan value is also low. Thus, the system can evenly
balancing. Task scheduling problems can be described as distribute tasks among the resources in the
system.
follows. Considering where each task tj tj ∈ T is assigned to the
Input: VM, vi (vi ∈ V) is represented by tji , so the VM task is
• Let V = {v1 , v2 , v3 , . . . , vm } where V is a collection of represented by vi = {txi , tyi , . . . , tzi }.
VMs, and m is the total number of VMs in the cloud The total execution time (ET ) of task processing on vi can
network. Each VM has its own resources (e.g., CPU, be obtained by (1).
RAM, and bandwidth) and the cost of usage and the P
computing power are defined differently; vi presents X tji ∈vi length(tj )
ET (vi ) =


the ith VM. ExtTime tji = (1)
tji ∈vi CPU (vi )
17808 VOLUME 10, 2022
B. Kruekaew, W. Kimpan: Multi-Objective Task Scheduling Optimization for Load Balancing in Cloud Computing Environment

where ExtTime tji is the execution time of tj processing in vi . task is sent to vi , we can calculate the memory load of vi
This value can be calculated by (2). by (10).

length(tj )
ExtTime tji = (2) RM j
CPU (vi ) LM i = AM i + (10)
TM i
where length(tj ) is the length of the jth task, the length of the
where AM i is the amount of memory usage before executing
task is defined in terms of the number of instructions (million
task tj at the ith VM
instructions), and CPU (vi ) is the CPU rate used to process the
RM j is the memory containing the request of the jth task
jth VM in the cloud. Makespan is the maximum value of the
TM i is the total memory available at the ith VM
execution time of all VMs and can be calculated by (3).
The next parameter is the CPU. If the requested task is sent
Makespan = Max(ExtTime(vi )), 1≤i≤m (3) to vi , we can calculate the CPU load of vi (LC i ) using (11).

MinMakespan is a lower bound of makespan, which is the RC j

LC i = AC i + (11)
minimum time required by the system to complete all tasks. TC i
MinMakespan is calculated by (4).
where AC i is the amount of CPU usage before executing task
MinMakespan = Min(ExtTime(vi )), 1≤i≤m (4) tj at the ith VM
RC j is the CPU that requests the jth task
Fitness function in terms of makespan (F1 ) can be calcu-
TC i is the total CPU available at the ith VM.
lated by (5).
VM utilization evaluation (VU i ) [61] can be calculated
MinMakespan by (12).
F1 = (5)
Makespan
ω1 ω2
2) The second objective is defined in terms of cost which F3 = VU i = ∗ (12)
1 − LM i 1 − LC i
is the cost of requesting to be processed by the task and
can be calculated from CPU cost, memory usage cost, and where ω1 and ω2 are the weights for the CPU and memory,
bandwidth usage cost. Estimated cost when tj is processed at respectively. ω1 + ω2 = 1 in this paper, given ω1 , ω2 are both
vi can be calculated by (6). equal to 0.5 because CPU and memory are equally important.

Total load on k host (LH ), where k is the total number of
Cost tji = (c1 ∗ ExtTime(tji )) + (c2 ∗ ExtTime(tji )) hosts in the system, can be calculated from (13).
+(c3 ∗ ExtTime(tji )) (6)
Xmk
where c1 , c2 , c3 are CPU usage cost per unit, memory usage LH k = VU ki (13)
i=0
cost per unit, and bandwidth usage cost per unit in vi ,
respectively. The average load on all physical machines in cloud (AL)
The total cost (TCost) is calculated as the sum of all tasks can be calculated by (14).
processing on all VMs, which can be calculated from (7). Pp
k=0 LH k
Xn Xm AL = (14)
TCost = Cost(tji ) (7) p
j=1 i=1

MinTCost is the lowest cost when the set of assigned tasks where p is the number of hosts in the cloud network.
T is processed in the VM where the VM process task tj The difference in load among each host and the average
gives the lowest cost, called the MinTCost(t j ) value, which load on the cloud network is calculated from |LH k − AL|.
means that MinTCost is only for task tj and can be calculated The fitness function is defined in terms of the utilization of
from (8). resources, which can be calculated by (15).
X
Xp
MinTCost = MinCost tj F3 = |LH k − AL| (15)
tj ∈T
X
k=0
= min1≤i≤m (Cost tji ) (8)
tji ∈T Fitness function is created by calculating the weighted
The fitness function in terms of cost (F2 ) can be calculated average of each individual fitness function. The proposed
by (9) fitness function (F) is shown in (16).
MinTCost F = (γ1 ∗ F1 ) + (γ2 ∗ F2 ) + (γ3 ∗ F3 ) (16)
F2 = (9)
TCost
3) The third objective is defined in terms of the utilization where γ1 , γ2 , γ3 , and γ  [0, 1] are the balance coefficients
of resources (CPU and memory) sent to a different number of between makespan, total cost, and resource utilization. Max-
processing units than in the cloud network. If the requested imizing the utility function (F) results in a better solution.

VOLUME 10, 2022 17809

 B. Kruekaew, W. Kimpan: Multi-Objective Task Scheduling Optimization for Load Balancing in Cloud Computing Environment

IV. HYBRID ABC ALGORITHM FOR LOAD BALANCING Algorithm 1 Q-Update(S)

A. Q-LEARNING ALGORITHM 1. Set values for learning rate α, discount factor γ ,
Q-learning [62] is one of the Reinforcement Learning (RL) reward r
algorithms. RL is one of the machine learning methods that 2. Initialize Q-values
allows agents to learn in their environment and action by 3. For i = 1 to n # n is the number of states
changing their state to receive rewards or penalties based 4. For j = 1 to p
# p is the number of actions in each state
on the feedback obtained from the environment. The main 5. Q si , aj = 0
purpose of RL is to learn the agent through trial and error 6. End for
between the agent and the environment. The agent is able to 7. End for
receive the environment situation through a state and choose 8. Select an action ai from A = {a1 , a2 , a3 , . . . , am }
an action that affects the environment to obtain the best and execute it and go to next state st+1
reward and learn through past mistakes. The Markov decision 9. Calculate the learning rate
process (MDP) is a framework for the decision-making of 10. Calculate the reward
agents because of the uncertain environment, and the result is 11. Update Qt+1 by (18)
sometimes stochastic. The agent chooses to perform the same 12. Repeat this step for new state until it converges
action for the same situation or state, but it may not always
obtain the same result. The Q-learning algorithm process is
shown in Fig. 2.
Given that the set of states S = {s1 , s2 , s3 , . . . , sn } is B. MODIFIED ABC ALGORITHM
in the environment, each state has a set of actions. The The ABC algorithm [19] is a meta-heuristic optimization
A = {a1 , a2 , a3 , . . . , am } agent selects action at A at time algorithm based on swarm intelligence. The swarm system
t in state st S to pass to the next state st+1 S through the consists of agents that communicate with other agents and
transition process and receives a reward rt+1 from the envi- their environment. In this algorithm, the goal of the agent is
ronment. To process the tasks, it is necessary to select the to find the best food source, where the food source represents
appropriate action to maximize the Q-value of each state, a set of possible answers in the search space and each agent
which is the primary objective of finding the optimal policy is represented by a bee.
in cloud computing. The Q-value function depends on the In ABC, agents are categorized according to the functions
selection of action in the state. Given the agent in state st of the bees and can be classified into 3 groups: employed
and selecting action at , the Q-value function is expected to bees, onlooker bees, and scout bees. Initially, each employed
move to the best state and gain to maximize the total expected bee finds a random food source, whereas in each itera-
reward in the environment. The Q-value can be calculated tion, employed bees find a new food source near a current
by (17) food source. After collecting the nectar, each employed bee
assesses the best food source. Then, the bee will move to a
Q (st , at ) = (1 − α) Q (st , at ) new food source only if the bees have determined that new
+ α [rt + γ max a+1 Q(st+1 , at+1 )] (17) food sources are better than the previous one. The employed
bees share information with the onlooker bees. The onlooker
where α is the learning rate, γ (0 < γ < 1) is the discount bees decide to choose new food sources based on information
factor and effect on the successive state by the previous obtained from the employed bees. If any food source has
action, and rt is the penalties or rewards awarded for per- much food or high quality, it will have a high chance of being
forming actions in the state st . The Q-value derives from chosen by onlooker bees. Then, each onlooker bee will find
creating a Q-table that stores all possible states, Q-values, a new food source around it. They select the food source and
and appropriate actions. The Q-learning algorithm attempts move to a new food source. The number of iterations is also
to establish the optimal state from their experience, and the predetermined, meaning that if no better food source is found,
greedy algorithm is implemented in the Q-learning algorithm. the employed bees who own the selected food source become
Q-value can be computed using (18). scout bees and are responsible for exploring the new food
source in a new area of search space.
Qt+1 (st , at ) = Q (st , at )+α rt + γ max a´t [Qt δ(st , at ), a´t ]


Each group of bees has different explorative and exploita-
− Qt (st , at )] (18) tive behaviors [63]. The explorative behavior of a search
agent involves searching for a new food source in the search
where α is the learning rate, calculated from α = 1/(1 + total space to avoid a local optimum. In contrast, exploitative
number of visits to state st ), and δ is the transition function. behavior involves searching for a better food source near
max a Q (st+1 , a) is an estimate of optimal future value. the current food source. In this paper, we use Q-learning to
A possible action in cloud computing involves load balanc- improve our solution to provide more appropriate solutions
ing, which allocates resources in VM and can be described in to problems. The ABC algorithm has both strong explorative
Algorithm 1. behavior and weak exploitative behavior; therefore, we aim to

17810 VOLUME 10, 2022

B. Kruekaew, W. Kimpan: Multi-Objective Task Scheduling Optimization for Load Balancing in Cloud Computing Environment

optimize exploitative behavior. The process can be described In the scout bee phase, if there are a number of unsuccessful
as follows: attempts to find a better neighbor, that food source will be
In the initialization phase, the ABC algorithm represents discarded, and the scout bee will randomly search for a new
the location of the food source with possible solutions to the food source.
problem. Initially, the food source is randomly generated as The pseudo-code of the MOABCQ method is presented in
shown in (19). Then, employed bees are associated with food Algorithm 2.
sources. The initial value of the Q-table is 0.
Algorithm 2 MOABCQ Method
0
xi,j = xjmin + rand (0, 1) ∗ (xjmax − xjmin ) (19) Initialization:
where xjmin is the lower bound of the jth optimization param- 1. Initialize the population and calculate individual
fitness values
eter, and xjmax is the upper bound of the jth optimization
2. Set up the parameter: best solution, maximum
parameter.
number of iterations, the population size
In addition to the initialization phase, the ABC algorithm
3. Find the best solution
separates the algorithm into three sub-phases: employed
4. while stopping criteria satisfied
bee phase, onlooker bee phase, and scout bee phase. The
Employed Bees Phase:
algorithm repeats all three phases until a certain maximum
5. For each position do
number of values is reached.
6. Update position of employed bee by (20)
In the employed bee phase, the employed bees [64] find
7. Estimate the new position
the location of the neighboring food source vti,j of the current
8. If fitness value of new position is better
food source (x̄it ) using (20).
9 Replace the current position with the
vti,j = xi,j
t t
+ rand [−1, 1] ∗ (xi,j t
− xk,j ) (20) new position
10 End if
where vti,j is the jth optimization parameter of v̄ti , and k is the 11. Calculate probability and update the Q-table
index of the food source. for select position in the onlooker bee phase
If the new food source v̄ti returns a
fitness value
greater than 12. End for
the current food source xit , Fit v̄ti > Fit x̄it , employed Onlooker Bees Phase:
bees forget the current food source and remember the new 13. For each onlooker do
location. In this step, the Q-table (reward-penalty scheme) 14. Select a position based on Q-value and
value is updated using (18). If the new food source provides a probability
better fitness value, the employed bee will not only replace the 15. Update position of onlooker bee by (21)
current food source with the new food source but also update 16. Estimate the new position
the Q-value by rewarding the selected new food source and 17. If fitness value of new position is better
penalty with the current food source. 18. Replace the current position with the new
In contrast, if the new food source does not provide a better position
fitness value, the new food source receives a penalty, and the 19. End if
current food source receives a reward. The Q-table is updated 20. Update the Q-table using (18)
every time that an employed bee finds a suitable food source. 21. End for
Therefore, if the number of employed bees is Emp, Q-table 22. Find the best solution (Q-table)
will update Emp once. Scout Bees Phase:
In the onlooker bee phase, the onlooker bee selects the 23. For each position do
employed bee’s food source from the Q-value in the Q-table. 24. Abandon the solution that have not been
In contrast, the onlooker bee searches for the new food source updated and generate new solutions randomly
using (21) and replacing the current food source with the new 25. update the Q-table using (18)
food source, if the new food source has a higher fitness value, 26. End for
then the Q-value will also be updated. 27. End while
vti,j = xi,k
t
+ ∅ti,j · rjt · (xi,k
t t
− xRFS,k ) (21)
where vti,j is the optimization parameter of a neighboring V. EXPERIMENTAL EVALUATION AND DISCUSSIONS
food source v̄ti , ∅ ∈ [−1, 1], and xRFS,k
t is the optimization In this section, the parameter settings and experimental
parameter of the optimal food source caused by random results are described. To evaluate the effectiveness of the
selection. proposed methodology (MOABCQ), we compared it with
Using (21) to improve exploitation instead of updating well-known heuristic task scheduling algorithms, such as
values in dimension, onlooker bees exploit current food and Max–Min task scheduling algorithm [22], First Come First
update all dimensions with different weight values, and in this Serve (FCFS) algorithm, and Largest Job First (LJF) algo-
step, Q-values (reward and penalty) are updated. rithm. Moreover, we compared MOABCQ method with the

VOLUME 10, 2022 17811

 B. Kruekaew, W. Kimpan: Multi-Objective Task Scheduling Optimization for Load Balancing in Cloud Computing Environment

TABLE 2. Parameter settings of meta-heuristic algorithms.

popular meta-heuristic task scheduling algorithms such as

the PSO algorithm and CS algorithm, and with our previous
method called Heuristic Task Scheduling with Artificial Bee
Colony algorithm and largest job first (HABC_LJF) [43].
We considered evaluating the performance of the proposed
method, which consists of a simulation environment, a bench-
mark datasets, parameter settings for the proposed method
and the comparison algorithms, experimental results, and the
time complexity of MOABCQ method.

A. SIMULATION ENVIRONMENT
This section presents an experiment conducted to evaluate
the performance of the proposed method (MOABCQ) when
comparing task scheduling with other methods in a heteroge-
neous cloud computing environment. In this paper, a simula-
tion was designed and developed using the CloudSim 3.0.3
simulator [65]. CloudSim is the most widely used simulator
to implement clouds. CloudSim is a tool that can simulate
virtual resources, and CloudSim can also support modeling,
simulation, and experimentation of virtualized cloud-based
data. This experiment was simulated on a computer with an
Intel Core i7-8750H CPU (clock speed of 2.20 GHz) and
16 GBs of RAM.
In this experiment, a virtual environment was simulated
to demonstrate the efficiency of the proposed method in
terms of scheduling and load balancing in a cloud computing
environment. The simulation environment of this experiment
was defined as shown in Table 1. FIGURE 3. Comparison of the performance in terms of makespan on
various datasets.

B. BENCHMARK DATASETS
To evaluate the scheduling efficiency of the proposed method, 2) Google Cloud Jobs (GoCJ) dataset [66], and 3) Synthetic
three different datasets were used: 1) Random dataset, workload dataset [26], which are described as:

17812 VOLUME 10, 2022

B. Kruekaew, W. Kimpan: Multi-Objective Task Scheduling Optimization for Load Balancing in Cloud Computing Environment

TABLE 3. Comparison of the performance in terms of ARUR on various datasets.

1) RANDOM DATASET to compare experimental results in terms of makespan,

We generated task sizes ranging from 1k – 70k Million throughput, ARUR [26], cost, and DI. In this experiment,
Instructions (MIs). The randomly generated dataset contains we assessed the effectiveness of the proposed method
a total of 1000 tasks. The dataset contains task size, number (MOABCQ). We combined the MOABCQ method with
of requested CPUs, and amount of RAM being requested. the First Come First Serve (FCFS) heuristic task schedul-
ing called ‘‘MOABCQ_FCFS’’ and Largest Job First (LJF)
2) GOCJ DATASET heuristic task scheduling called ‘‘MOABCQ_LJF’’ and
GoCJ dataset is considered a Google-like realistic dataset compared them with other well-known algorithms: FCFS
generated from the workload behaviors witnessed in Google scheduling algorithm, Max–Min task scheduling algo-
cluster traces using bootstrapped Monte Carlo, a well-known rithm [22], Heuristic Task Scheduling with Artificial Bee
simulation method. The task sizes in the GoCJ dataset range Colony algorithm and largest job first (HABC_LJF) [43],
from 15k – 900k MIs, and the datasets are classified as: small Q-learning algorithm, multi-objective particle swarm opti-
size jobs (15k – 55k MIs), medium size jobs (59k – 99k MIs), mization (MOPSO) algorithm, and multi-objective cuckoo
large size jobs (101k – 135k MIs), extra-large size jobs (150k search (MOCS) algorithm. Each dataset was run for
– 337.5k MIs), and huge size jobs (525k – 900k MIs). 20 rounds, and the average of the results is proposed in the
following section.
3) SYNTHETIC WORKLOAD DATASET
The first section presents a comparison of the performance
Synthetic workload dataset iscreated by random-number gen-
of the proposed method in terms of makespan. This experi-
erator mechanism using Monte Carlo simulation method.
ment was assigned to 100 VMs, and 200, 400, 600, 800, and
It consists of different tasks sizes from 1 – 45K MIs which are
1000 tasks were assigned to the system. In the experiment,
tiny size jobs (1-250 MIs), small size jobs (800–1200 MIs),
we used the 3 datasets mentioned earlier. The experimen-
medium size jobs (1800–2500 MIs), large size jobs
tal results are shown in Fig. 3. According to the experi-
(7k–10k MIs), and extra-large size jobs (30k–45k MIs).
mental results in Fig. 3(a), when the random dataset was
C. PARAMETER SETTINGS FOR THE PROPOSED METHOD tested, the MOABCQ method was found to reduce the
AND THE COMPARISON ALGORITHMS average makespan better than the Max-Min method, FCFS,
In this experiment, we defined population parameter and HABC_LJF, Q-learning, MOPSO, and MOCS. However,
other conditions from related articles which are: HABC algo- when 400 datasets were tested, MOCS gave the lowest
rithm [43], PSO algorithm [67], and CS algorithm [56]. As it average makespan compared to the other methods. MOCS
is already recognized that parameter setting has effects on took 3.69%, 3.77%, 8.82%, 13.85%, 20.77% 23.41%, and
the efficiency of algorithms and it depends on size or nature 82.39% less time to complete than FCFS, MOABCQ_LJF,
of the problem. For this reason, tuning the parameters must MOABCQ_FCFS, Q-learning, HABC_LJF, MOPSO, and
be done to ensure that we use the appropriate parameters Max-Min, respectively.
for the problem type and dataset. We did not claim, how- If we consider the insight of MOABCQ, after compar-
ever, that the proposed method or its parameters outperform ing MOABCQ_FCFS and MOABCQ_LJF, MOABCQ_LJF
alternative algorithms for all type of problems and datasets. can be found to give a lower average makespan than
In the experiment, the parameters of the ABC algorithm MOABCQ_FCFS in the case of 200, 800, and 1000 tasks.
proposed by Kruekaew and Kimpan [43] are defined in Table However, in the case of 600 tasks, MOABCQ_FCFS had an
2. In addition, to compare the scheduling performance, the average makespan 0.51% less than MOABCQ_LJF.
proposed method was compared with the HABC algorithm, Considering the experimental results in Fig. 3(b), in which
MOPSO algorithm, and MOCS algorithm. The parameters the GoCJ dataset was used, and Fig. 3(c), in which
are defined as shown in Table 2 which based on the original the Synthetic workload dataset was used, MOABCQ_LJF
papers. gave the lowest average makespan. When using the
GoCJ dataset, MOABCQ_LJF gave average makespan
D. EXPERIMENTAL RESULTS approximately 117.80%, 92.15%, 46.17%, 42.25%, 34.94%,
This section describes an experimental evaluation of a pro- 30.62%, and 8.21% less than the makespan of Max-Min,
posed scheduling approach using the benchmark dataset FCFS, Q-learning, MOPSO, HABC_LJF, MOCS, and

VOLUME 10, 2022 17813

 B. Kruekaew, W. Kimpan: Multi-Objective Task Scheduling Optimization for Load Balancing in Cloud Computing Environment

MOABCQ_FCFS, respectively. When using the Synthetic

workload dataset, MOABCQ_LJF yielded average makespan
of approximately 150.75%, 98.61%, 25.53%, 22.87%,
15.35%, 10.76%, and 2.97% less than the makespan of
Max-Min, FCFS, MOPSO, Q-learning, HABC_LJF, MOCS,
and MOABCQ_FCFS, respectively. It can be indicated that
MOABCQ_LJF outperformed the other methods in running
both datasets with all test conditions except the percentage of
the average makespan reduction.
However, overall examinations showed that the proposed
method can provide the lowest makespan value because
MOABCQ_LJF can allocate the task to the appropriate
resource. According to all these results, we can conclude that
the proposed method has the potential to efficiently allocate
resources in the system.
The second section presents a comparison of the effi-
ciency of the method presented in terms of throughput,
which is the number of tasks executed per unit of time.
The 3 previously used datasets were used in this exper-
iment. The experimental results of testing the efficiency
with a random dataset are shown in Fig. 4(a), which indi-
cates that MOABCQ had better throughput than other algo-
rithms. However, when testing in 400 tasks, MOCS gave
greater values than the others at 3.56%, 3.63%, 8.10%,
12.16%, 17.20%, 18.97%, and 45.17% when compared
to FCFS, MOABCQ_LJF, MOABCQ_FCFS, Q-learning,
HABC_LJF, MOPSO, and Max-Min, respectively. Consid-
ering the MOABCQ itself in depth, MOABCQ_LJF has a
higher throughput than MOABCQ_FCFS in 200, 800, and
1000 tasks. In the case of 600 tasks, MOABCQ_FCFS pro-
vided 0.50% higher throughput than MOABCQ_LJF.
After using the GoCJ and Synthetic workload datasets
to test throughput, we found that MOABCQ had bet-
ter performance than the other algorithms. The results
are shown in Fig. 4(b) and 4(c). When MOABCQ_LJF
and MOABCQ_FCFS were considered, MOABCQ_LJF
provides throughput approximately 6.14% more than
MOABCQ_FCFS on average. We can conclude from the
overall throughput test experiments that MOABCQ_LJF has
the potential to allocate the tasks to appropriate resources
in the same direction as the testing result in the first
section.
The third section presents a comparison of the efficiency of
the proposed method in terms of Average Resource Utiliza-
tion Ratio (ARUR), which is another important condition for
task scheduling in the system. In this experiment, we used
1000 tasks, 100 machines of VMs, and 3 dataset tests. The
experimental results are shown in Table 3.
We found that MOABCQ gave higher ARUR values than
the other methods. To consider in depth all 3 datasets, the
algorithms used to perform in this experiment that gave the
similarity of ARUR results were found to be Q-learning, FIGURE 4. Comparison of the performance in terms of throughput on
various datasets.
HABC_LJF, MOPSO, MOCS, MOABCQ_FCFS, and
MOABCQ_LJF, unlike Max-Min and FCFS. When testing
with the random dataset, MOABCQ_LJF provided larger 21.53% compared to MOABCQ_FCFS, MOCS, Q-learning,
ARUR values of 6.15%, 14.01%, 18.72%, 20.94%, and HABC_LJF, and MOPSO, respectively.

17814 VOLUME 10, 2022

B. Kruekaew, W. Kimpan: Multi-Objective Task Scheduling Optimization for Load Balancing in Cloud Computing Environment

When using the GoCJ dataset, MOABCQ_LJF gave

greater ARUR values then the others at 6.31%, 11.34%,
12.63%, 13.61%, and 22.99% when compared to
MOABCQ_FCFS, MOCS, HABC_LJF, MOPSO, and
Q-learning. After experimenting with the Synthetic work-
load dataset, MOABCQ_LJF gave higher ARUR values
than the MOABCQ_FCFS, MOCS, Q-learning, HABC_LJF,
and MOPSO methods at 0.62%, 4.99%, 5.74%, 9.74%, and
9.86%, respectively.
The testing with all three datasets revealed that
MOABCQ_LJF provides the highest ARUR value com-
pared to the other methods. Thus, we can conclude that the
MOABCQ_LJF method can efficiently schedule tasks in the
system and can help equally distribute tasks across available
resources, which can help the system stay in balance mode.
The fourth section presents a comparison of the per-
formance of the proposed method in terms of the degree
of imbalance (DI) to assess the load balancing of the
system. The experiments were conducted with the same
datasets as in the previous experiment sections. These exper-
iments were tested on 100 VMs and 200, 400, 600, 800,
and 1000 tasks. The proposed method (MOABCQ) was
compared with Max-Min, FCFS, Q-learning, HABC_LJF,
MOPSO, and MOCS. The results are shown in Table 4.
When testing on a random dataset and GoCJ dataset, the DI
values of MOABCQ_LJF were the lowest, indicating that
MOABCQ_LJF can distribute tasks more equally to existing
resources in the system than the other methods. However,
when using the Synthetic workload dataset, we found that
in the case of setting tasks in the system equal to 200,
600, 800 and 1000 tasks, MOABCQ_LJF gave a lower DI
value than the other methods. Unless testing with 400 tasks,
MOABCQ_FCFS had the lowest DI value. When compared
to MOABCQ_LJF, MOABCQ_FCFS can distribute tasks
better than MOABCQ_LJF at 4.37%.
According to the testing with all three datasets, the
MOABCQ method was found to be able to equally distribute
the task to the available resources in the system, which
resulted in a low DI value. If we consider the proposed
method in depth, it reveals that MOABCQ_LJF performed
more efficiently than the other compared methods. However,
it depends on the dataset being tested.
The final section presents a comparison of the perfor-
mance of the proposed method from a cost perspective to
assess the costs or overheads when accessing cloud com-
puting by executing 3 datasets. The experimental results are
shown in Fig. 5. Considering Fig. 5(a), using the random
dataset, MOABCQ was found to be able to reduce cost more
than the MOCS, MOPSO, Q-learning, HABC_LJF, FCFS,
and Max-Min. When MOABCQ_FCFS was compared with
MOABCQ_LJF, it indicated that MOABCQ_LJF has lower FIGURE 5. Comparison of the performance in terms of cost on various
dataset.
cost than MOABCQ_FCFS by approximately 3.38%. How-
ever, with 1000 tasks, the MOABCQ_LJF algorithm costs
4.88% more than the MOABCQ_FCFS algorithm. the other methods in the same way as when using the
When testing with the GoCJ dataset, the results in Fig. 5(b) random dataset. When comparing MOABCQ_FCFS with
show that MOABCQ was able to reduce costs more than MOABCQ_LJF and testing on 200, 400, and 800 tasks,

VOLUME 10, 2022 17815

 B. Kruekaew, W. Kimpan: Multi-Objective Task Scheduling Optimization for Load Balancing in Cloud Computing Environment

TABLE 4. Comparison of the performance in terms of DI on various datasets.

we found that the MOABCQ_LJF algorithm has a lower VI. CONCLUSION

cost than the MOABCQ_FCFS algorithm at approx- In this article, we propose the multi-objective optimization
imately 20.79%. In addition, when testing with 600 scheduling method in heterogeneous cloud computing using
and 1000 tasks, MOABCQ_FCFS was found to have a the MOABCQ method. This method considered the selection
higher cost than the MOABCQ_LJF method by 0.48% on of suitable VMs based on calculating the fitness of each
average. VM. Heuristic approaches which are FCFS and LJF were
When testing with the Synthetic workload dataset, the also included. The experiments were conducted with various
results in Fig. 5(c) show that the MOABCQ method has a datasets to observe the performance of the proposed algo-
lower cost than the other comparison methods in the same rithms. The proposed method helps load balancing tasks with
way as when testing with the previous two datasets. After existing resources in the system and also improves makespan
comparing MOABCQ_FCFS and MOABCQ_LJF, in the reduction, DI reduction, cost reduction, and throughput and
case of setting tasks in the system equal to 200, 600, 800, ARUR increases when compared to the Max-Min, FCFS,
and 1000 tasks, the MOABCQ_LJF algorithm was found Q-learning, HABC_LJF, MOPSO, and MOCS algorithms.
to have lower cost than MOABCQ_FCFS, except for 400 The experimental results indicated that the proposed method
tasks, MOABCQ_FCFS has lower cost than MOABCQ_LJF outperformed the others. However, we cannot guarantee that
method at 1.90%. the MOABCQ_LJF algorithm is optimal. Nevertheless, the
According to the testing with all three datasets, the pro- performance of the system cannot be optimized in every test
posed method of MOABCQ was able to reduce costs more dataset.
than the other comparison methods. However, when com- Task scheduling in a multi-cloud, fog cloud, or edge cloud
paring MOABCQ_LJF and MOABCQ_FCFS, we found environment can be challenging and interesting work in the
that MOABCQ_LJF can be more appropriately used for future. We propose a scheduling arrangement method in dif-
task scheduling with existing resources in the system than ferent environments. Other machine learning algorithms may
MOABCQ_FCFS. Considering the difference in the per- also be applied further. In addition, the proposed method can
centage between the two methods, MOABCQ_LJF has a also be tested in a real-world environment to observe the
smaller percentage. However, it depends on the dataset to be performance of the MOABCQ method.
tested.
REFERENCES
E. THE TIME COMPLEXITY OF MOABCQ METHOD
[1] T. S. George and V. P. S. Kumar, ‘‘Multicloud computing for on-demand
The time complexity of MOABCQ method can be calculated resource provisioning using clustering,’’ in Proc. 3rd Int. Conf. Sustain.
as: in ABC, an initial population of n is given and the bee is Energy Intell. Syst. (SEISCON), 2012, pp. 435–440.
classified into Employed bees and Onlooker bee. Therefore, [2] S. Yang, L. Pan, Q. Wang, S. Liu, and S. Zhang, ‘‘Subscription or pay-as-
you-go: Optimally purchasing IaaS instances in public clouds,’’ in Proc.
the number of iterations to find suitable VMs in the cloud IEEE Int. Conf. Web Services (ICWS), Jul. 2018, pp. 219–226.
is n and the number of updated data in the Q-table is n as [3] D. Ardagna, G. Casale, M. Ciavotta, J. F. Pérez, and W. Wang, ‘‘Quality-of-
well. As a result, the time complexity of MOABCQ is O(n). service in cloud computing: Modeling techniques and their applications,’’
J. Internet Services Appl., vol. 5, pp. 1–17, Dec. 2014.
If ABC repeats this step k times, the time complexity is equal
[4] K. Psychas and J. Ghaderi, ‘‘On non-preemptive VM scheduling in the
to k × O(n). Since k is a constant, the total time complexity cloud,’’ Proc. ACM Meas. Anal. Comput. Syst., vol. 1, no. 2, pp. 1–29,
of MOABCQ is equal to O(n). Dec. 2017, doi: 10.1145/3154493.

17816 VOLUME 10, 2022

B. Kruekaew, W. Kimpan: Multi-Objective Task Scheduling Optimization for Load Balancing in Cloud Computing Environment

[5] S. Crago, K. Dunn, P. Eads, L. Hochstein, D. Kang, M. Kang, D. Modium, [26] A. Hussain, M. Aleem, A. Khan, M. A. Iqbal, and M. A. Islam,
K. Singh, J. Suh, and J. Walters, ‘‘Heterogeneous cloud computing,’’ ‘‘RALBA: A computation-aware load balancing scheduler for cloud com-
in Proc. IEEE Int. Conf. Clust. Comput., Austin, TX, USA, Feb. 2011, puting,’’ Cluster Comput., vol. 21, no. 3, pp. 1667–1680, 2018, doi:
pp. 378–385, doi: 10.1109/CLUSTER.2011.49. 10.1007/s10586-018-2414-6.
[6] J. W. M. Bush, B. A. Thurber, and F. Blanchette, ‘‘Particle clouds in homo- [27] B. Mondal, K. Dasgupta, and P. Dutta, ‘‘Load balancing in cloud
geneous and stratified environments,’’ J. Fluid Mech., vol. 489, pp. 29–54, computing using stochastic hill climbing—A soft computing
Jul. 2003, doi: 10.1017/S0022112003005160. approach,’’ Proc. Technol., vol. 4, pp. 783–789, Jun. 2012, doi:
[7] R. Messier, ‘‘Virtual servers and platform as a service,’’ in Proc. Col- 10.1016/j.protcy.2012.05.128.
laboration Cloud Comput. Secur., Social Media, Unified Commun., 2014, [28] M. Adhikari and S. Koley, ‘‘Cloud computing: A multi-workflow schedul-
pp. 77–91, doi: 10.1016/B978-0-12-417040-7.00005-8. ing algorithm with dynamic reusability,’’ Arabian J. Sci. Eng., vol. 43,
[8] O. Alsaryrah, I. Mashal, and T.-Y. Chung, ‘‘Bi-objective optimiza- no. 2, pp. 645–660, Feb. 2018, doi: 10.1007/s13369-017-2739-0.
tion for energy aware Internet of Things service composition,’’ IEEE [29] B. Shrimali and H. Patel, ‘‘Multi-objective optimization oriented policy
Access, vol. 6, pp. 26809–26819, 2018, doi: 10.1109/ACCESS.2018. for performance and energy efficient resource allocation in cloud environ-
2836334. ment,’’ J. King Saud Univ.-Comput. Inf. Sci., vol. 32, no. 7, pp. 860–869,
[9] L. Liu, M. Zhang, R. Buyya, and Q. Fan, ‘‘Deadline-constrained coevo- Sep. 2020, doi: 10.1016/j.jksuci.2017.12.001.
lutionary genetic algorithm for scientific workflow scheduling in cloud [30] C.-W. Tsai and J. J. P. C. Rodrigues, ‘‘Metaheuristic scheduling for cloud:
computing,’’ Concurr. Comput. Pract. Exp., vol. 29, no. 5, p. e3942, 2017, A survey,’’ IEEE Syst. J., vol. 8, no. 1, pp. 279–291, Mar. 2014, doi:
doi: 10.1002/cpe.3942. 10.1109/JSYST.2013.2256731.
[10] Z. Wu, X. Liu, Z. Ni, D. Yuan, and Y. Yang, ‘‘A market-oriented hierar- [31] M. Kalra and S. Singh, ‘‘A review of metaheuristic scheduling techniques
chical scheduling strategy in cloud workflow systems,’’ J. Supercomput., in cloud computing,’’ Egyptian Informat. J., vol. 16, no. 3, pp. 275–295,
vol. 63, no. 1, pp. 256–293, 2013, doi: 10.1007/s11227-011-0578-4. 2015, doi: 10.1016/j.eij.2015.07.001.
[11] D. Yagyasen, M. Darbari, P. K. Shukla, and V. K. Singh, ‘‘Diversity and [32] F. Ramezani, J. Lu, J. Taheri, and F. K. Hussain, ‘‘Evolutionary algorithm-
convergence issues in evolutionary multiobjective optimization: Applica- based multi-objective task scheduling optimization model in cloud envi-
tion to agriculture science,’’ IERI Proc., vol. 5, pp. 81–86, Oct. 2013, doi: ronments,’’ World Wide Web, vol. 18, no. 6, pp. 1737–1757, 2015, doi:
10.1016/j.ieri.2013.11.074. 10.1007/s11280-015-0335-3.
[12] F. Luo, Y. Yuan, W. Ding, and H. Lu, ‘‘An improved particle swarm [33] L. Zuo, L. Shu, S. Dong, C. Zhu, and T. Hara, ‘‘A multi-objective
optimization algorithm based on adaptive weight for task scheduling in optimization scheduling method based on the ant colony algorithm in
cloud computing,’’ in Proc. 2nd Int. Conf. Comput. S. App. Eng., 2018, cloud computing,’’ IEEE Access, vol. 3, pp. 2687–2699, 2015, doi:
pp. 1–5, doi: 10.1145/3207677.3278089. 10.1109/ACCESS.2015.2508940.
[13] I. Alharkan, M. Saleh, M. A. Ghaleb, H. Kaid, A. Farhan, and A. Almar- [34] X. Guo, ‘‘Multi-objective task scheduling optimization in cloud computing
fadi, ‘‘Tabu search and particle swarm optimization algorithms for two based on fuzzy self-defense algorithm,’’ Alexandria Eng. J., vol. 60, no. 6,
identical parallel machines scheduling problem with a single server,’’ pp. 5603–5609, Dec. 2021.
J. King Saud Univ.-Eng. Sci., vol. 32, no. 5, pp. 330–338, Jul. 2020, doi: [35] M. A. Tawfeek, A. El-Sisi, A. E. Keshk, and F. A. Torkey, ‘‘Cloud task
10.1016/j.jksues.2019.03.006. scheduling based on ant colony optimization,’’ in Proc. 8th Int. Conf.
[14] S. Basu, M. Karuppiah, K. Selvakumar, and K. Li, ‘‘An intelli- Comput. Eng. Syst. (ICCES), Nov. 2013, pp. 64–69.
gent/cognitive model of task scheduling for IoT applications in cloud com- [36] Z. Chen, K. Du, Z. Zhan, and J. Zhang, ‘‘Deadline constrained cloud
puting environment,’’ Future Gener. Comput. Syst., vol. 88, pp. 254–261, computing resources scheduling for cost optimization based on dynamic
Nov. 2018, doi: 10.1016/j.future.2018.05.056. objective genetic algorithm,’’ in Proc. CEC, 2015, pp. 708–714, doi:
[15] J. Horn, N. Nafpliotis, and D. E. Goldberg, ‘‘A niched Pareto genetic 10.1109/CEC.2015.7256960.
algorithm for multiobjective optimization,’’ in Proc. 1st IEEE Conf. Evol. [37] Z. Amini, M. Maeen, and M. R. Jahangir, ‘‘Providing a load balancing
Comput. World Congr. Comput. Intell., vol. 1, Jun. 1994, pp. 82–87, doi: method based on dragonfly optimization algorithm for resource alloca-
10.1109/ICEC.1994.350037. tion in cloud computing,’’ Int. J. Netw. Distrib. Comput., vol. 6, no. 1,
[16] J. Knowles and D. Corne, ‘‘Approximating the nondominated front using pp. 35–42, 2018, doi: 10.2991/ijndc.2018.6.1.4.
the Pareto archived evolution strategy,’’ Evol. Comput., vol. 8, no. 2, [38] M. S. Sanaj and P. M. Joe Prathap, ‘‘An efficient approach to the map-
pp. 149–172, Jan. 2000, doi: 10.1162/106365600568167. reduce framework and genetic algorithm based whale optimization algo-
[17] D. Karaboga, ‘‘An idea based on honey bee swarm for numerical optimiza- rithm for task scheduling in cloud computing environment,’’ Mater. Today,
tion,’’ ERU, Kayseri, Turkey, Tech. Rep.-tr06, Oct. 2005. Process., vol. 37, pp. 3199–3208, Oct. 2021.
[18] B. Akay and D. Karaboga, ‘‘A modified artificial bee colony algorithm [39] F. Farahnakian, P. Liljeberg, and J. Plosila, ‘‘Energy-efficient virtual
for real-parameter optimization,’’ Inf. Sci., vol. 192, no. 1, pp. 120–142, machines consolidation in cloud data centers using reinforcement learn-
Apr. 2012, doi: 10.1016/j.ins.2010.07.015. ing,’’ in Proc. 22nd Euromicro Int. Conf. Parallel Distrib. Netw.-Based
[19] D. Karaboga and B. Gorkemli, ‘‘A combinatorial artificial bee colony Process., Turin, Italy, 2014, pp. 500–507.
algorithm for traveling salesman problem,’’ in Proc. Int. Symp. Innov. [40] S. Ismaeel, R. Karim, and A. Miri, ‘‘Proactive dynamic virtual-machine
Intell. Syst. Appl., Jun. 2011, pp. 50–53. consolidation for energy conservation in cloud data centres,’’ J. Cloud
[20] X. Li, D. Peng, B. Du, J. Guo, W. Xu, and K. Zhuang, ‘‘Hybrid artificial bee Comput., vol. 7, no. 1, pp. 1–28, Dec. 2018, doi: 10.1186/s13677-018-
colony algorithm with a rescheduling strategy for solving flexible job shop 0111-x.
scheduling problems,’’ Comput. Ind. Eng., vol. 113, pp. 10–26, Nov. 2017, [41] U. Rugwiro, C. Gu, and W. Ding, ‘‘Task scheduling and resource allo-
doi: 10.1016/j.cie.2017.09.005. cation based on ant-colony optimization and deep reinforcement learn-
[21] J. D. Ullman, ‘‘NP-complete scheduling problems,’’ J. Comput. Syst. Sci., ing,’’ J. Internet Technol., vol. 20, no. 5, pp. 1463–1475, 2019, doi:
vol. 10, no. 3, pp. 384–393, 1975. 10.3966/160792642019092005013.
[22] Y. Mao, X. Chen, and X. Li, ‘‘Max-min task scheduling algorithm for load [42] J.-Q. Li and Y.-Q. Han, ‘‘A hybrid multi-objective artificial bee colony
balance in cloud computing,’’ in Proc. Int. Conf. Comput. Sci. Inf. Technol., algorithm for flexible task scheduling problems in cloud computing sys-
2014, pp. 457–465. tem,’’ Cluster Comput., vol. 23, no. 4, pp. 2483–2499, Dec. 2020, doi:
[23] T. Islam and M. S. Hasan, ‘‘A performance comparison of load balancing 10.1007/s10586-019-03022-z.
algorithms for cloud computing,’’ in Proc. Int. Conf. Frontiers Adv. Data [43] B. Kruekaew and W. Kimpan, ‘‘Enhancing of artificial bee colony algo-
Sci. (FADS), Oct. 2017, pp. 130–135. rithm for virtual machine scheduling and load balancing problem in cloud
[24] G. Patel, R. Mehta, and U. Bhoi, ‘‘Enhanced load balanced min-min computing,’’ Int. J. Comput. Intell. Syst., vol. 13, no. 1, pp. 496–510, 2020,
algorithm for static meta task scheduling in cloud computing,’’ Proc. doi: 10.2991/ijcis.d.200410.002.
Comput. Sci., vol. 57, pp. 545–553, Jan. 2015, doi: 10.1016/j.procs.2015. [44] G.-N. Gan, T.-L. Huang, and S. Gao, ‘‘Genetic simulated annealing algo-
07.385. rithm for task scheduling based on cloud computing environment,’’ in Proc.
[25] H. Zhang, J. Shi, B. Deng, G. Jia, G. Han, and L. Shu, ‘‘MCTE: Minimizes Int. Conf. Intell. Comput. Integr. Syst., Oct. 2010, pp. 60–63.
task completion time and execution cost to optimize scheduling perfor- [45] D. Alsadie, ‘‘A metaheuristic framework for dynamic virtual machine allo-
mance for smart grid cloud,’’ IEEE Access, vol. 7, pp. 134793–134803, cation with optimized task scheduling in cloud data centers,’’ IEEE Access,
2019, doi: 10.1109/ACCESS.2019.2942067. vol. 9, pp. 74218–74233, 2021, doi: 10.1109/ACCESS.2021.3077901.

VOLUME 10, 2022 17817

 B. Kruekaew, W. Kimpan: Multi-Objective Task Scheduling Optimization for Load Balancing in Cloud Computing Environment

[46] H. He, G. Xu, S. Pang, and Z. Zhao, ‘‘AMTS: Adaptive multi-objective [61] A. Abdelsamea, A. A. El-Moursy, E. E. Hemayed, and H. Eldeeb, ‘‘Vir-
task scheduling strategy in cloud computing,’’ China Commun., vol. 13, tual machine consolidation enhancement using hybrid regression algo-
no. 4, pp. 162–171, Apr. 2016, doi: 10.1109/CC.2016.7464133. rithms,’’ Egyptian Inform. J., vol. 18, no. 3, pp. 161–170, Nov. 2017, doi:
[47] R. Jena, ‘‘Task scheduling in cloud environment: A multi-objective 10.1016/j.eij.2016.12.002.
ABC framework,’’ J. Inf. Optim. Sci., vol. 38, pp. 1–19, Jan. 2017, doi: [62] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction.
02522667.2016.1250460. Cambridge, MA, USA: MIT Press, 2018.
[48] A. Kumar and M. Venkatesan, ‘‘Multi-objective task scheduling using [63] S. Fairee, S. Prom-On, and B. Sirinaovakul, ‘‘Reinforcement learning for
hybrid genetic-ant colony optimization algorithm in cloud environ- solution updating in artificial bee colony,’’ PLoS ONE, vol. 13, no. 7,
ment,’’ Wireless Pers. Commun., vol. 107, pp. 1835–1848, 2019, doi: Jul. 2018, Art. no. e0200738, doi: 10.1371/journal.pone.0200738.
10.1007/s11277-019-06360-8. [64] D. Karaboga and B. Basturk, ‘‘A powerful and efficient algorithm for
[49] G. N. Reddy and S. P. Kumar, ‘‘Multi objective task scheduling algo- numerical function optimization: Artificial bee colony (ABC) algorithm,’’
rithm for cloud computing using whale optimization technique,’’ in Proc. J. Global Optim., vol. 39, pp. 459–471, Nov. 2007, doi: 10.1007/s10898-
Int. Conf. Next Gener. Comput., Technol. Singapore: Springer, 2017, 007-9149-x.
pp. 286–297. [65] R. Calheiros, R. Ranjan, A. Beloglazov, C. A. F. De Rose, and R. Buyya,
[50] S. H. H. Madni, M. S. A. Latiff, J. Ali, and S. M. Abdulhamid, ‘‘Multi- ‘‘CloudSim: A toolkit for modeling and simulation of cloud computing
objective-oriented cuckoo search optimization-based resource scheduling environments and evaluation of resource provisioning algorithms,’’ Softw.,
algorithm for clouds,’’ Arabian J. Sci. Eng., vol. 44, no. 4, pp. 3585–3602, Pract. Exper., vol. 41, no. 1, pp. 23–50, 2011, doi: 10.1002/spe.995.
2019, doi: 10.1007/s13369-018-3602-7. [66] A. Hussain and M. Aleem, ‘‘GoCJ: Google cloud jobs dataset for dis-
[51] S. Pang, W. Li, H. He, Z. Shan, and X. Wang, ‘‘An EDA-GA tributed and cloud computing infrastructures,’’ Data, vol. 3, no. 4, p. 38,
hybrid algorithm for multi-objective task scheduling in cloud com- 2018, doi: 10.3390/data3040038.
puting,’’ in IEEE Access, vol. 7, pp. 146379–146389, 2019, doi: [67] H. Saleh, H. Nashaat, W. Saber, and H. M. Harb, ‘‘IPSO task scheduling
10.1109/ACCESS.2019.2946216. algorithm for large scale data in cloud computing environment,’’ IEEE
[52] P. Neelima and A. R. M. Reddy, ‘‘An efficient load balancing system Access, vol. 7, pp. 5412–5420, 2018.
using adaptive dragonfly algorithm in cloud computing,’’ Cluster Comput.,
vol. 23, pp. 2891–2899, 2020, doi: 10.1007/s10586-020-03054-w.
[53] M. Gamal, R. Rizk, H. Mahdi, and B. E. Elnaghi, ‘‘Osmotic bio-inspired
load balancing algorithm in cloud computing,’’ IEEE Access, vol. 7,
pp. 42735–42744, 2019, doi: 10.1109/ACCESS.2019.2907615.
[54] U. A. Butt, M. Mehmood, S. B. H. Shah, R. Amin, M. W. Shaukat,
BOONHATAI KRUEKAEW received the B.Sc.
S. M. Raza, D. Y. Suh, and M. J. Piran, ‘‘A review of machine learn- degree in computer science from the Prince of
ing algorithms for cloud computing security,’’ Electronics, vol. 9, no. 9, Songkla University, Hat Yai, Songkhla, Thailand,
p. 1379, Aug. 2020, doi: 10.3390/electronics9091379. and the M.Sc. degree in computer science from
[55] L. Caviglione, M. Gaggero, M. Paolucci, and R. Ronco, ‘‘Deep reinforce- the King Mongkut’s Institute of Technology
ment learning for multi-objective placement of virtual machines in cloud Ladkrabang, Bangkok, Thailand, where she is cur-
datacenters,’’ Soft. Comput., vol. 25, pp. 12569–12588, Oct. 2021, doi: rently pursuing the Ph.D. degree. Her research
10.1007/s00500-020-05462-x. interests include cloud computing, algorithm,
[56] U. K. Jena, P. K. Das, and M. R. Kabat, ‘‘Hybridization of meta- artificial intelligence, and swarm intelligence.
heuristic algorithm for load balancing in cloud computing environment,’’
J. King Saud Univ.-Comput. Inf. Sci., early access. [Online]. Available:
https://www.sciencedirect.com/science/article/pii/S1319157819309267,
doi: 10.1016/j.jksuci.2020.01.012.
[57] T. Thein, M. M. Myo, S. Parvin, and A. Gawanmeh, ‘‘Reinforcement
learning based methodology for energy-efficient resource allocation in
cloud data centers,’’ J. King Saud Univ.-Comput. Inf. Sci., vol. 32, no. 10, WARANGKHANA KIMPAN (Member, IEEE)
pp. 1127–1139, Dec. 2020, doi: 10.1016/j.jksuci.2018.11.005. received the Ph.D. degree in system information
[58] Y. Chen, A. Ganapathi, R. Griffith, and R. Katz, ‘‘Analysis and lessons engineering from Kagoshima University, Japan.
from a publicly available Google cluster trace,’’ EECS Dep., RAD Lab,
She is currently an Assistant Professor with
Univ. California Berkeley, Berkeley, CA, USA, Tech. Rep. UCB/EECS-
2010-95, Jun. 2010.
the Department of Computer Science, School of
[59] R. T. Marler and J. S. Arora, ‘‘The weighted sum method for multi- Science, King Mongkut’s Institute of Technol-
objective optimization: New insights,’’ Struct. Multidisciplinary Optim., ogy Ladkrabang, Bangkok, Thailand. Her main
vol. 41, no. 6, pp. 853–862, Jun. 2010, doi: 10.1007/s00158-009-0460-7. research interests include swarm intelligence,
[60] I. Y. Kim and O. L. de Weck, ‘‘Adaptive weighted-sum method for bi- biomedical engineering, big data, data science and
objective optimization: Pareto front generation,’’ Struct. Multidisciplinary analytics, cloud computing, and the Internet of
Optim., vol. 29, no. 2, pp. 149–158, 2005, doi: 10.1007/s00158-004- Things.
0465-1.

17818 VOLUME 10, 2022