Swarm and Evolutionary Computation 77 (2023) 101248

Contents lists available at ScienceDirect

Swarm and Evolutionary Computation

journal homepage: www.elsevier.com/locate/swevo

Performance assessment and exhaustive listing of 500+ nature-inspired

metaheuristic algorithms
Zhongqiang Ma a, Guohua Wu a, *, Ponnuthurai Nagaratnam Suganthan b, c, *, Aijuan Song a,
Qizhang Luo a
a
School of Traffic and Transportation Engineering, Central South University, Changsha 410075, China
b
KINDI Center for Computing Research, College of Engineering, Qatar University, Doha, Qatar
c
School of Electrical and Electronic Engineering, Nanyang Technological University, 639798, Singapore

A R T I C L E I N F O A B S T R A C T

Keywords: Metaheuristics are popularly used in various fields, and they have attracted much attention in the scientific and
Metaheuristics industrial communities. In recent years, the number of new metaheuristic names has been continuously growing.
Nature-inspired Generally, the inventors attribute the novelties of these new algorithms to inspirations from either biology,
Parameter tuning
human behaviors, physics, or other phenomena. In addition, these new algorithms, compared against basic
Search bias to origin
versions of other metaheuristics using classical benchmark problems, show competitive performances. However,
Performance evaluation
Nonparametric tests many new metaheuristics are not rigorously tested on challenging benchmark suites and are not compared with
state-of-the-art metaheuristic variants. Therefore, in this study, we exhaustively tabulate more than 500 meta­
heuristics. In particular, several representative metaheuristics are introduced from two aspects, namely, the
inspirational source and the essential operators for generating solutions. To comparatively evaluate the per­
formance of the state-of-the-art and newly proposed metaheuristics, 11 newly proposed metaheuristics (generally
with high numbers of citations) and 4 state-of-the-art metaheuristics are comprehensively compared on the
CEC2017 benchmark suite. For fair comparisons, a parameter tuning tool named irace is used to automatically
configure the parameters of all 15 algorithms. In addition, whether these algorithms have a search bias to the
origin (i.e., the center of the search space) is investigated. All the experimental results are analyzed by several
nonparametric statistical methods, including the Bayesian rank-sum test, Friedman test, Wilcoxon signed-rank
test, critical difference plot and Bayesian signed-rank test. Moreover, the convergence, diversity, and the
trade-off between exploration and exploitation of these 15 algorithms are also analyzed. The results show that
the performance of the newly proposed EBCM algorithm performs similarly to the 4 compared algorithms and
has the same properties and behaviors, such as convergence, diversity, exploration and exploitation trade-offs, in
many aspects. However, the other 10 recent metaheuristics are less efficient and robust than the 4 state-of-the-art
metaheuristics. The performance of all 15 of the algorithms is likely to deteriorate due to certain transformations,
while the 4 state-of-the-art metaheuristics are less affected by transformations such as the shifting of the global
optimal point away from the center of the search space. It should be noted that, except EBCM, the other 10 new
algorithms are inferior to the 4 state-of-the-art algorithms in terms of convergence speed and global search ability
on CEC 2017 functions. Moreover, the other 10 new algorithms are rougher (i.e., present in their behavior with
high oscillations) in terms of the trade-off between exploitation and exploration and population diversity
compared with the 4 state-of-the-art algorithms. Finally, several important issues relevant to the metaheuristic
research area are discussed and some potential research directions are suggested.

1. Introduction problems can be modeled as optimization problems. Optimization al­

gorithms attempt to reach the optimal objective values (i.e., minimum
Optimization algorithms play an important role in the economy, or maximum) and satisfy the related constraints. Very complex problems
engineering, management, and medicine because many real-world are highly constrained, multimodal, discontinuous, noisy and of high

* Corresponding authors.
E-mail addresses: [email protected] (G. Wu), [email protected] (P.N. Suganthan).

https://doi.org/10.1016/j.swevo.2023.101248
Received 6 February 2022; Received in revised form 28 November 2022; Accepted 9 January 2023
Available online 14 January 2023
2210-6502/© 2023 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

dimension, all of which can make the traditional exact algorithms (e.g., with high numbers of citations) and 4 state-of-the-art metaheuristics
mathematical programming) ineffective. are selected to be comprehensively compared on the CEC2017
As an alternative method, approximate algorithms have attracted benchmark suite. In addition, whether these algorithms have a
much attention in recent decades. Approximate algorithms can be search bias to the origin is investigated. For fair comparisons, a
roughly divided into heuristic algorithms and metaheuristic algorithms. unified framework named irace is used to tune the parameters of all
Heuristic algorithms generally need to be elaborately designed for spe­ the comparative algorithms.
cific optimization problems and may have weak flexibility in solving • We use multiple nonparametric statistical methods to analyze the
other types of problems. In contrast, metaheuristics provide a general experimental results in depth. The statistical results show that the
optimization framework for solving various optimization problems and newly proposed EBCM algorithm performs similarly to the 4
benefit from the randomness embedded into the operators, which makes compared algorithms and has the same properties and behaviors,
it possible to find a satisfactory, or near-optimal solution, in a reasonable such as convergence, diversity, exploration and exploitation trade-
time, however, they cannot guarantee the optimum solution for complex offs, in many aspects. However, the other 10 recent metaheuristics
problems [1]. The merits of simplicity, less problem dependence, flexi­ are less efficient and robust than the 4 state-of-the-art meta­
bility, derivative-free mechanism, and local optima avoidance make heuristics. All 15 algorithms show certain degrees of search bias
metaheuristics user-friendly [2]. toward the origin, but the 4 state-of-the-art metaheuristics are less
Metaheuristics can be defined as high-level methodologies that affected by the shift operator on the functions. Furthermore, we find
embody the underlying heuristics to solve optimization problems [3]. that the other 10 new algorithms (i.e., except for EBCM) are inferior
The term metaheuristic was first proposed by Glover in 1986 [4], and to the 4 state-of-the-art algorithms in terms of convergence speed
most modern nature-inspired algorithms can be considered meta­ and global search ability on most of the CEC2017 functions. The
heuristics [5]. The concept of nature-inspired is about creating algo­ other 10 new algorithms show a rougher trade-off and diversity
rithms by mimicking natural phenomena or biological behaviors to solve compared with the 4 state-of-the-art algorithms. Finally, several
optimization problems. For example, simulated annealing (SA) [6] is important issues that should be considered in the metaheuristic
inspired by the idea of the solid annealing principle. Particle swarm research area are discussed and some potential research directions
optimization (PSO) [7] is derived from the interaction behaviors of birds are suggested.
in the flock. Ant colony optimization (ACO) [8] mimics the behaviors of
ants in finding the shortest path between a nest and a food source. The The paper is organized as follows: Section 2 presents a taxonomy of
classification criteria of metaheuristics can be varied. For instance, ac­ the metaheuristics and some representative metaheuristics are further
cording to the number of candidate solutions at each iteration, meta­ introduced and investigated by explaining the inspiration sources and
heuristics can be further divided into population-solution based the essential operators for generating solutions. Extensive experiments
metaheuristics and single-solution based metaheuristics [9]. Popular are conducted to evaluate the performance of the 15 comparative al­
single-solution based metaheuristics include SA, tabu search (TS) [4], gorithms in Section 3, and some properties of these algorithms are
iterated local search (ITS) [10], guided local search (GLS) [11], random further studied, including convergence, diversity, and the exploration
search (RS) [12], variable neighborhood search (VNS) [13], and large and exploitation trade-offs. Section 4 engages with some metaheuristics
neighborhood search (LNS) [14]. Population-solution based meta­ research issues and suggests several potential research directions. Sec­
heuristics include the genetic algorithm (GA) [15], differential evolution tion 5 draws the conclusion.
(DE) [16], pattern search (PS) [17], and others.
There are still some issues in the field of metaheuristics. With the 2. Literature overviews
increase in the number of recent metaheuristics, the necessity of irra­
tionally introducing new metaheuristic algorithms is questioned [18]. In the last few decades, not only various improved versions of met­
Molina et al. [19] found that there is no necessary significant relation­ aheuristics, but also many metaheuristics with new names mimicking
ship between the inspiration sources of algorithms and their perfor­ the behaviors of humans, animals and plants, and the phenomena of
mance. However, some researchers expect to improve the performance physics and chemistry have been proposed. We selected some of the
of metaheuristics through the inspiration source, which is still popular metaheuristics (i.e., 47 metaheuristics) to search for publica­
misleading. There is no work that comprehensively evaluates and tions of these algorithms in the Web of Science updates to November
compares the efficiency and effectiveness of the newly proposed and the 2021. Fig. 1 shows that the number of publications for DE, PSO, SA,
state-of-the-art metaheuristics [20,21]. Furthermore, some algorithms ACO, and the artificial immune system all exceeded 10,000. It can be
perform well on problems with the optimal solution located at the origin observed that many newly proposed metaheuristics have also received
(i.e., center of the search space) but are less efficient when the optimal many citations and substantial attention. Therefore, metaheuristics are
solutions are shifted [22–24]. This issue may affect the fair evaluation of still among the hot research topics and it is expected that the number of
the algorithms. publications of new metaheuristics and state-of-the-art metaheuristics
Motivated by the issues mentioned above, in this paper, we first will continue to increase in the future. Table 71 summarizes more than
summarize and analyze the related metaheuristics studies. Then, 500 metaheuristics, in which “B#” corresponds to reference [#] in
extensive experiments are conducted by using representative bench­ Appendix B of the related supplemental material. For details on the full
mark functions, to fairly evaluate and understand the performances and list of metaheuristics, please refer to the supplementary materials.
characteristics of the state-of-the-art and the recent metaheuristics with
a unified parameter tuning method. Furthermore, we test whether the 2.1. Taxonomy of metaheuristics
algorithms have a search bias to the origin. Therefore, the main research
contributions of our paper are outlined as follows: There are various taxonomy methods for metaheuristics in the
literature, and the most popular taxonomy is based on the source of
• More than 500 metaheuristics are collected and a taxonomy of the inspiration [19,20,25]. Fig. 2 illustrates a rough metaheuristics classi­
metaheuristics is proposed. In particular, several representative al­ fication, in which the metaheuristics are divided into population-based
gorithms are introduced from two aspects, including the inspiration
sources and the essential operators for generating solutions.
• We perform extensive experiments to evaluate and understand the 1
If a metaheuristic algorithm is not included in Table 7, please inform the
performances of the state-of-the-art and the recent metaheuristics. authors. Table 7 will be updated online at: https://github.com/P-N-Suganth
Eleven representative metaheuristics with new names (generally an/MHA-500Plus

2
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

Fig. 1. The number of publications about some popular metaheuristics.

Fig. 2. A classification of metaheuristics.

optimization algorithms (POAs) and single-solution based optimization 2.1.1. Evolutionary algorithms
algorithms (SOAs) according to the number of solutions generated in EAs are inspired by Darwinian evolutionary theory and mimic the
each iteration. SOAs generally require only one individual to search the behavior of evolution in nature, such as recombination, mutation, and
solution space. In contrast, POAs contain multiple individuals that selection [26], which fully embodies the idea of survival of the fittest.
search the solution space cooperatively and globally with some opera­ The first computer simulation of evolution can be traced back to 1954 by
tors and mechanisms, such as mutation, crossover, selection, informa­ the work of Barricelli [27] but his publication did not attract widespread
tion sharing, and search behavior learning. attention [28]. Until the 1960s and early 1970s, optimization methods
We mainly focus on the POAs in this paper. Compared with SOAs, the could be designed via artificial simulated evolution after the use of
most important characteristics of POAs are three-fold [1]. First, multiple evolutionary strategies (ES) to solve complex engineering problems in
points (i.e., solutions or individuals) are employed to search the solution Rechenberg’s work [29,30]. Currently, many variants of ES have been
space cooperatively. Second, mechanisms for information sharing and proposed in the literature, such as (1+1)-ES, (μ+1)-ES, (μ+λ)-ES, and (μ,
interactive learning among the individuals are adopted. Third, POAs are λ)-ES [31]. In 1960, evolutionary programming (EP) was first proposed
stochastic, as randomness is usually incorporated into search operators by Fogel to achieve artificial intelligence [32,33]. Originally EP used
such as mutation and crossover. In Fig. 2, the POAs can be further finite state machines as predictors to predict environments. Currently,
roughly divided into evolutionary algorithms (EAs), swarm intelligence EP is a popular evolutionary algorithm and has many different versions
algorithms (SIAs) and physics or chemistry-based algorithms (P/CBAs). including FEP (Fast EP) [34], AEP (Adaptive EP) [35], RLEP

3
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

(Evolutionary Programming based on Reinforcement Learning) [36], the poor and rich optimization algorithm (PRO) [62], human urbani­
and ENAEP (Ensemble algorithm of Gaussian and Cauchy mutation zation algorithm (HUS) [63], life choice-based optimizer (LCBO) [64]
operators using AEP) [37]. In the early 1970s, genetic algorithms and queuing search algorithm (QS) [65].
became popular through the work of Holland [15], and their perfor­
mance mainly depended on the efficient encoding and decoding of the (2) Nonhuman algorithms
solution, appropriate parameter configuration and operators, including
crossover, mutation and selection. GA and its variants are popular in a Nonhuman algorithms (NHAs) include animal-based algorithms
wide range of fields, such as planning [38] and scheduling [39], bio­ (AAs) and plant-based algorithms (PAs). AAs are inspired by the
logical [40] and chemical [41] engineering, and data mining [42]. Later, behavior of different animals, such as foraging, flocking, mating, and
genetic programming (GP) appeared and gradually became popular other behaviors [66]. For example, PSO is inspired by the behavior of a
starting from the early 1990s. The variants of GP include MGP (multi­ flock of birds or a school of fish, in which each particle can move
gene genetic programming) [43], and GGP (grammatical genetic pro­ throughout the solution space and update its current position in terms of
gramming) [44]. Subsequently, differential evolution introduced by a current best solution and a global best solution [53]. An artificial bee
Storn and Price in 1995 [16,45], emerged as a very competitive evolu­ colony (ABC) is a metaheuristic based on the intelligent behavior of a
tionary algorithm, especially in dealing with continuous optimization honey bee swarm. The bee colony consists of three types of bees,
problems. There are many powerful and efficient variants of DE pro­ employed bees, onlooker bees, and scout bees, and the search phases can
posed in the literature, such as MPEDE [46] (multi-population ensemble be divided into search, recruit, and abandon [67]. The bat algorithm
DE), EDEV [47] (ensemble of multiple DE variants), SaDE [48] (with (BA) [68] and cuckoo search (CS) [69] are inspired by the echolocation
adapted mutation strategies and parameters), jDE [49] (with behavior of bats and the brood parasitism of some cuckoo species,
self-adapted parameters) and CoDE [50] (composition of multiple stra­ respectively. Other popular AAs include the firefly algorithm (FA) [70],
tegies and parameter settings). gray wolf optimizer (GWO) [71] and grasshopper optimization algo­
rithm (GOA) [72]. PAs are inspired by plant behavior such as growth,
2.1.2. Swarm intelligence algorithms root expansion, weed invasion and flower pollinatio [65,73]. For
SIAs mimic the behaviors of animals, plants, and human groups in instance, the invasive weed optimization (IWO) algorithm [74] mimics
nature to optimize problems. Decentralized control and self- the process of weed invasion, and the flower-pollinating algorithm
organization are two important features of SIAs [51], which can be (FPA) [75] simulates the characteristics of flower pollination.
understood as a group of individuals achieving common goals through
cooperation. In other words, each individual of the swarm has its own 2.1.3. Physics/chemistry-based algorithms
intelligence and behaviors, and the integration of multiple individuals P/CBAs are mostly created by imitating the physical and chemical
has more power to solve complex problems [52]. Particularly, the law phenomena in nature, including electromagnetic force, inertia force,
following advantages of SIAs make them user-friendly optimizers [53]: gravity, electrical charges, river systems, movement, chemical changes
1) The general framework can be applied to various fields with only a of material, and others [73,76–79]. For instance, the gravitational
few modifications; 2) The information of the solution space and search search algorithm (GSA) [80] is inspired by the law of gravity and mass
states is reserved and used to guide the search during the optimization interactions, where the search individuals are a collection of masses.
process; 3) Relatively fewer parameters make SIAs require less tuning According to the concepts of the white hole, black hole and wormhole in
effort to cater to different optimization problems. However, there still cosmology, multi-verse optimization (MVO) [81] has been designed to
exist some critical issues that have not been well addressed in SIAs, such solve complex problems. In MVO, white holes and black holes are
as premature convergence, being stuck in a local optimum, and lack of correlated with explorations, and wormholes are responsible for sharing
good trade-offs between exploitation and exploration [54]. Compared and exploiting the information of the solution space. In addition, har­
with EAs, SIAs do not have crossovers, while evolutionary algorithms mony search (HS) [82] mimics the behavior of an orchestra to create the
usually have crossovers. SIAs do not include competitive selections, but most harmonious melody and measure it by aesthetic standards.
EAs usually have selections. In addition, SIAs can be hybridized with EAs Detailed information about HS is described in the literature [83]. Other
to include crossovers/selections. SIAs can be further categorized into typical P/CBAs include water evaporation optimization (WEO) [84],
human-related algorithms and nonhuman algorithms according to the transient search optimization (TSO) [85], chemical reaction optimiza­
inspiration source. tion (CRO) [86], and charged system search (CSS) [87].

(1) Human-related algorithms

2.2. Optimization mechanisms of metaheuristics
Human-related algorithms (HRAs) are inspired by the behaviors of
humans in society, such as learning, competition, political campaigns, In this section, the optimization frameworks of single-solution based
and cultural influence [55]. For example, inspired by the behavior of and population-based metaheuristics are presented. After that, several
human learning, the teaching-learning-based optimization (TLBO) al­ representative metaheuristics are reviewed from two different aspects,
gorithm works on the effect of a teacher on learners [56]. The the inspiration sources and the essential operators for generating
gaining-sharing knowledge-based algorithm (GSK) simulates the process solutions.
of obtaining and sharing knowledge during the human lifespan [52].
The group teaching optimization algorithm (GTOA) mimics the mech­ 2.2.1. Optimization framework of metaheuristics
anism of group teaching [57]. In terms of society competition, the soccer As Algorithm 1 [88,89] shows, typical single-solution based meta­
league competition algorithm (SLCA) is inspired by teams competing heuristics start from a single initial solution. It iteratively performs a
during a season in a soccer league [58] and the imperialist competitive generation and selection procedure for a single solution until a termi­
algorithm (ICA) simulates the competition among imperialists [59]. nation condition is met; then, a best-so-far solution will be returned. In
Inspired by political campaigns, Askari et al. [5] conducted a compre­ each iteration, a candidate solution set, C(st ), is generated based on the
hensive analysis of political mechanisms and proposed a new algorithm incumbent solution, st , in the generation procedure. In the selection
called the political optimizer (PO). In addition, the greedy reedy politics phase, a selection operation is performed on the set C(st ) to choose a new
optimization (GPO) [60] and the parliamentary optimization algorithm solution st+1 to replace the current solution.
(POA) [61] are also inspired by political mechanisms. There are many Population-based metaheuristics begin with an initial population
algorithms inspired by other human society inspiration sources, such as solution P0 , as shown in Algorithm 2 [88,90]. Afterward, the generation

4
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

and selection are iteratively executed to generate a new population Pt ,

′
/ /
and selects promising individuals to form a new population Pt+1 to DE current − to − rand 1:→
v i,G
replace the current population. Finally, the best-so-far solution, P∗ , is
( )
returned when a given stopping criterion is met. Moreover, the historical
information can be memorized in Algorithm 1 and Algorithm 2 to better =→
x i,G + F⋅ →x r1i ,G − →
x i,G
generate candidate solutions and to select promising solutions. ( )
Regardless of the kinds of optimization frameworks and classifica­
+ F⋅ →x r2i ,G − →
x r3i ,G (6)
tion criteria used, exploration and exploitation play crucial roles in
improving the performance of metaheuristics [47,91,92]. Exploration
refers to the ability to globally search the solution space and find a where →
x ri ,G , →
x ri ,G , →
x ri ,G , →
x ri ,G , and →
x ri ,G are mutually different vec­
promising region, which is associated with escaping from the local op­
1 2 3 4 5

tors, which are randomly chosen from the population at generation G.

timum and avoiding a premature convergence (i.e., increasing popula­ →x i,G is the target vector at generation G. →
x best,G is the vector with the
tion diversity). Exploitation denotes the capability of locally searching
best fitness in the population at generation G. F is the scaling factor
the promising region found by the exploration operators. The
within [0, 1].
well-known trade-off between exploration and exploitation is critical. In
Two widely used crossover methods in DE are the binomial crossover
regards to the trade-offs of the exploration and exploitation of meta­
and exponential crossover, and their formulas are shown as follows [97,
heuristics, Morales-Castañeda et al. [93] and Črepinšek et al.[94] con­
98].
ducted an in-depth investigation.
Binomial crossover:
2.2.2. Introduction of representative metaheuristics vi,j,G if randi (0, 1) ≤ CR or j = jrand
ui,j,G = { (7)
In this section, several representative state-of-the-art and new met­ xi,j,G otherwise
aheuristics are reviewed from two aspects: (1) the inspiration source for
proposing the algorithm, and (2) the essential operators for generating Exponential crossover:
solutions in each algorithm. These representative metaheuristics include
vi,j,G for j = 〈l〉D , 〈l + 1〉D , …, 〈l + L − 1〉D
some popular and competitive algorithms and the recently proposed ui,j,G = { (8)
xi,j,G otherwise
algorithms.
where ui,j,G , xi,j,G , and vi,j,G are the j-th components of vectors →
u i,G , →
x i,G ,
(1) Differential Evolution (DE) [16] →
and v , respectively, i = {1, 2, …, NP}, and j = {1, 2, …, D}. j
i,G is an
rand
integer, that is randomly generated in the range of [1,D]. randi (0, 1) is a
DE is a competitive metaheuristic inspired by the principle of sur­
number randomly generated from a uniform distribution in the range of
vival of the fittest. In DE, the population evolves through mutation,
[0, 1]. The notation < >D denotes the modulo function with modulus D
crossover, and selection in each generation, and the most frequently
and L is an integer number ranging in [1, D].
used mutation operator of DE is called DE/rand/1, which can be
The selection operation can be completed by comparing the fitness
formulated as
values of the target vector and the trial vector that determines which
( )
vectors can survive to the next generation.
→v =→
i,G x i + F⋅ →
r1 ,G x i − →x i r2 ,G r3 ,G (1) ( ) ( )
→
u i,G if f →u i,G ≤ f →x i,G
→x i,G+1 = { (9)
where →
v i,G is the mutation vector, →
x ri ,G , →
x ri ,G , and →
x ri ,G are three →
x i,G otherwise
1 2 3

randomly generated distinct vectors, and F is a mutation factor among [0,

1]. where → x i,G+1 and → x i,G are target vectors at the generations G and G + 1,
The other popular mutation schemes are summarized as follows [95, respectively, → u i,G+1 is the trial vector at generation G and f(∗) is the
96]: objective function considered.
/ / ( ) There exist many variants of DE in the literature. Some variants show
DE best 1 : → v =→ x + F⋅ → x i − → x i (2) competitive performance in solving complex problems. For example,
LSHADE-cnEpSin [99] combines a sinusoidal approach based on per­
i,G best,G r1 ,G r2 ,G

formance adaptation and a covariance matrix learning method for the

/ / ( ) ( )
crossover operator into LSHADE-EpSin, which achieved a competitive
DE best 2:→
v i,G = →
x best,G + F⋅ → x r2i ,G + F⋅ →
x r1i ,G − → x r3i ,G − →
x r4i ,G performance at the 2017 IEEE CEC. Mohamed et al. [100] proposed a
new version of DE named LSHADE-SPACMA by integrating
(3)
LSHADE-SPA and a modified version of CMA-ES. Furthermore,
/ / ( ) ( ) Mohamed et al. [97] proposed two new DE variants named EDE and
DE rand 2:→
v i,G = →
x r1i ,G + F⋅ →x i
r2 ,G − →
x r3i ,G + F⋅ →x i
r4 ,G − →
x r5i ,G EBDE, where EDE had a less greedy mutation strategy named
DE/current-to-ord_best/1, and EBDE introduced a more greedy muta­
(4) tion strategy named DE/current-to-ord_pbest/1. We named this newly
/ / proposed algorithm EDE-EBDE in our paper. MPEDE [46] and EDEV
[47] are two powerful variants proposed by Wu. In MPEDE [46], three
DE target − to − best 1:→
v i,G
mutation strategies simultaneously coexisted, i.e., “current-to-pbest/1”
( ) and “current-to-rand/1” and “rand/1”. EDEV [47] consists of three
=→
x i,G + F⋅ →x best,G − →x i,G highly popular and efficient DE variants, namely JADE, CoDE, and
( ) EPSDE. The entire population of EDEV is partitioned into four sub­
populations to coevolve to obtain better results. In the latest research,
+ F⋅ → x i − → x i (5)
Ghosh et al. [101] combined SHADE and LSHADE with the nearest
r1 ,G r2 ,G

spatial neighborhood-based parameter adaptive process modification

method to propose NSHADE and NLSHADE.

5
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

(2) Hybrid sampling evolution strategy (HSES) [102] multiplications.

HSES is a new version of ES that combines the covariance matrix (5) Multi-strategy enhanced sine cosine algorithm (MSCA) [106]
adaptation-evolution strategy (CMA-ES) and the univariate sampling
method, it achieved the best performance at the 2018 IEEE CEC. In the MSCA is an improved version of SCA [107], which is based on sine
HSES, the CMA-ES is mainly used for unimodal problems and the uni­ and cosine functions and randomly generates multiple initial individuals
variate sampling method is used for multimodal nonseparable problems. to fluctuate outward or toward the best solution. In MSCA, multiple
In particular, the method for calculating the mean value and the stan­ control mechanisms and operators are embedded into SCA, including
dard deviation of UMDAc (i.e., univariate marginal distribution algo­ the Cauchy mutation operator, chaotic local search mechanism, and
rithm continuous) is modified in the HSES. For detailed information opposition-based learning strategy, and two differential evolution op­
about the HSES, please refer to [102]. erators are used to achieve a better trade-off between exploration and
exploitation. The position update Eq.s of MSCA are the same as those of
(3) EBOwithCMAR [103] SCA and can be expressed as follows.
⃒ ⃒
Xit + r1 × sin(r2 ) × ⃒r3 Pti − Xit ⃒ r4 < 0.5
EBOwithCMAR (Effective Butterfly Optimizer with Covariance Ma­ Xit+1 = { t ⃒ t ⃒ (15)
trix Adapted Retreat Phase) is an improved variant of the butterfly X + r1 × cos(r2 ) × ⃒r3 P − X t ⃒ r4 ≥ 0.5
i i i
optimizer, which combines a self-adaptive butterfly optimizer and a
a
covariance matrix adapted retreat phase. The solution modification Eq.s r1 = a − t (16)
T
are shown as follows.
( ) where Xit is the position of the current solution in the i-th dimension at
Criss − cross modification : vz = x1ccz + F ∗ x1r1z − (X1 ∪ X2 )r2z (10)
the t-th iteration, and Pti is the position of the destination point in the i-th
( ) dimension at the t-th iteration. r1 is a random variable that is calculated
Toward − best modification : vz = x1bestz + F ∗ x1ccz − (X1 ∪ X2 )r2z (11) by Eq. (16), which is a constant, t is the current iteration, and T is the
maximum number of iterations. r2 is a random variable responsible for
where vz is a new vector, and x1ccz , x1r1z and (X1 ∪ X2 )r2z are three the movement (i.e., toward or outward Pti ) of the next solution (i.e.,
distinct individual vectors. x1bestz is the best neighbor of the z-th vector. F Xit+1 ). r3 is a random variable that gives random weights for Pti . r4 is a
is a positive real number that controls the population evolution random number within [0,1]. For detailed information on the mecha­
rate. X1 ∪ X2 is the combination of both populations. In particular, the nisms and operators adopted in MSCA please refer to [106].
crossover operator of EBOwithCMAR is based on the Eq. (9).
(6) Improved moth-flame optimization algorithm (IMFO) [108]
(4) Snap-drift cuckoo search (SDCS) [104]
IMFO is a newly improved algorithm that introduces a hybrid phase,
SDCS is a new version of CS [69] proposed by Rakhshani, that in­ dynamic crossover mechanism, and fitness-dependent weight factor into
tegrates the snap and drift modes into CS to establish the trade-off be­ MFO [109] to overcome the degeneration of the global search capability
tween exploration and exploitation. Moreover, a pair of new crossover and convergence speed. The main inspiration of IMFO is also the navi­
and mutation operators are employed to improve the search capability. gation behavior of moths in nature, which is referred to as a transverse
The updated rules of the SDCS are shown below. orientation. The position update Eq.s of moths in IMFO are shown as
{ follows.
max(0, pm − ω) if μ = snap
Snap and drift modes: pa = (12) ⃒
⃒f (M )⃒
⃒
⃒ ( best) ⃒
min(1, pm + ω) if μ = drift w=⃒ ⃒ (17)
⃒ f Mik ⃒
⎧ ( )
⎪
⎪
⎪ xti + a0 ⊗ xtj ⊗ Lévy(β) − xti if p < J
⎪
⎨ ( ) Mik = Dk−i 1 ebt cos(2πt) + w⋅Fik− 1 + (1 − w)⋅Mbest (18)
Crossover operator: xit+1 = xti + a0 ⊗ xtj − xti ⊗ Lévy(β) if J ≤ p ≤ 1 − J
⎪
⎪
⎪
⎪ ( ) where w is a weight factor that depends on fitness, f(Mbest ) is the fitness
⎩ xt + a ⊗ xt − xt ⊗ Lévy(β) if p ≥ 1 − J
i 0 j i value of the best solution Mbest and f(Mki ) represents the fitness values of
(13) the i-th moth at iteration k. Mki and Fik− 1 are the positions of the i-th moth
⎧ and the j-th flame at iterations k and k − 1, respectively. Dk− i
1
is the
( )
⎪
⎪
⎪ distance between the i-th moth and the j-th flame at iteration k − 1, b is a
⎪
⎪ xti + H(pa − ε) ⊗ xjt ⊗ r − xti if p < J
⎪
⎪
⎨ ( ) constant used to define the shape of the logarithmic spiral and t is a
t+1 t t t
Mutation operator: xi = xi + H(pa − ε) ⊗ xj − xi ⊗ r if J ≤ p ≤ 1 − J random number within [-1,1].
⎪
⎪ ( )
⎪
⎪
⎪ t t t
⎪
⎩ xi + H(pa − ε) ⊗ xj − xi
⎪ if p ≥ 1 − J (7) Aquila optimizer (AO) [110]

(14) AO is inspired by Aquila’s behavior in nature during the process of

catching prey. The optimization procedures of the proposed AO algo­
where pa is known as a switching parameter [105], which is applied to
rithm are represented in four methods, selecting the search space by a
trade-off the snap mode and drift mode. pm is the performance measure,
high soar with a vertical stoop, exploring within a divergent search
and ω is the increase (or decrease) rate of pa . μ is an auxiliary parameter
space by a contour flight with a short glide attack, exploiting within a
that equals snap if 0 ≤ pm ≤ 0.5; otherwise, μ = drift. xti and xtj are two
convergent search space by a low flight with a slow descent attack, and
mutually different positions at time t, and xt+1
i is the i-th position at time swooping by walking and grabbing the prey. In AO, the different steps
t + 1. a0 and β are the Lévy flight exponent and step size scaling factor, (four methods) have different mathematical expressions for solution
respectively. H refers to the Heaviside step function. r, p, and ε are three position updating and are shown as follows.
randomly generated numbers with uniform distributions, and J ∈ [0, 1]
is a possibility value. The notation ⊗ indicates entrywise

6
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

( t) [0,1] and the Levy distribution, respectively. β is the Levy index. The
Step 1 : X1 (t + 1) = Xbest (t) × 1 − + (XM (t) − Xbest (t) ∗ rand) (19)
T notation ⊕ in all the Eq.s represents the dot product operation.

Step 2 : X2 (t + 1) = Xbest (t) × Levy(D) + XR (t) + (y − x) ∗ rand (20) (9) Hyperbolic gravitational search algorithm (HGSA) [112]

Step 3 : X3 (t + 1)
GSA is a physically inspired population-based algorithm that solves
= (Xbest (t) − XM (t)) × α − rand + (UB − LB) × rand + LB) × δ problems based on the law of gravity and mass interactions [80]. HGSA
(21) is a new version of GSA, in which the hyperbolic acceleration coefficient,
dynamic regulation, and decreasing hyperbolic function are adopted to
Step 4 : X4 (t + 1) achieve a better trade-off between exploration and exploitation. The
= QF × Xbest (t) − (G1 × X(t) × rand) − G2 × Levy(D) + rand × G1 positions and velocities of the individuals in HGSA can be calculated as
follows.
(22)
( )/
vdi (t + 1) = randi × vdi (t) + c1 (t) × adi (t)Δt + c2 (t) × gbest − xdi (t) Δt
where Xi (t +1) is the solution of the next iteration of t, which is gener­
(26)
ated by each search method (XI ) and i = 1, 2, 3, 4. Xbest (t) is the best-
obtained solution until the tth iteration, XM (t) denotes the location xdi (t + 1) = xdi (t) + vdi (t + 1) (27)
mean value of the current solutions at the tth iteration, Levy(D) is the levy
flight distribution function, XR (t) is a random solution taken in the range where xdi (t) and vdi (t) are the position and velocity of the i-th individual
of [1, N] at the t th iteration, y = r × cos(θ) and x = r × sin(θ) are used to in the d-th dimension at iteration t, adi (t) is the acceleration of individual
present the spiral shape in the search and α and δ are the exploitation i at time t and c1 (t) and c2 (t) are the acceleration coefficients at time t.
adjustment parameters fixed at 0.1. LB and UB are the lower bound and randi is a uniform random variable in the interval [0,1]. gbest is the
upper bound of the given problem, respectively. QF denotes a quality position of the best-so-far solution. Δt is the time increment. For
function used to establish the equilibrium of the search strategies, G1 detailed information about HGSA, please refer to [112].
denotes various motions of the AO, G2 denotes the flight slope of the AO
that is used to follow the prey during the slope from the first location (1) (10) Memetic frog leaping algorithm (MFLA)
to the last location (t), rand is a random value between 0 and 1, and t and
T represent the current iteration and the maximum number of iterations, MFLA is an improved version of the shuffled frog leaping algorithm
respectively. For detailed parameter information calculations and the (SFLA) that was first proposed by Eusuff et al. [113]. SFLA is a meta­
AO, please refer to [110]. heuristic search approach that mimics the foraging behavior of frogs,
which is similar to PSO. In the frog population, each frog can commu­
(8) Improved grasshopper optimization algorithm (IGOA) [111] nicate with each other and the worst frog can jump to find the best food
source guided by the best frog. MFLA improves SFLA by integrating a
The grasshopper optimization algorithm (GOA) [71] is a recently memetic mechanism and a new search leaping rule. The mathematical
proposed metaheuristic algorithm that is inspired by the swarming formulas are shown below.
behavior of grasshoppers. IGOA improves GOA through the integration
of multiple mechanisms including Gaussian mutation, Levy-flight Qm = {
Qg if rand < 0.5
(28)
strategy and opposition-based learning. The improvement method in QC else
IGOA is similar to that of the MSCA. The mathematical expression of the
(29)
′
solution position update is as follows. Qw = Qw + rand(Qbest − Qw ) + rand(Qm − Qw )
⎛ ⎞
where Qw and Qbest are the worst and best frog, respectively. Qm is an
⎜∑
⎜ N ubd − lbd (⃒⃒ d ⃒) ⎟
⎟ auxiliary variable. Qg and QC represent the geometric center and grav­
d ⃒ xj − xi ⎟
Xid = c⎜
⎜ c s ⃒x − x ⃒ ̂d
⊕ G(α) + T (23) itational center, respectively. For the Eq.s for calculating Qg and QC ,
⎝ j=1 2 j i
dij ⎟⎠
please refer to [114].
j∕
=i

(11) Gaining‑sharing knowledge-based algorithm (GSK) [52]

Xilevy = Xi∗ + rand(d) ⊕ levy(β) (24)
( )〉 ( ) GSK is inspired by the human behaviors of gaining and sharing
Xilevy fitness Xilevy fitness Xi∗ knowledge, which can be divided into two phases: (1) the junior gaining
Xit+1 = { (25)
Xi∗ otherwise and sharing phase and (2) the senior gaining and sharing phase. The
differential Eq.s for generating the new solutions are proposed in two
In Eq. (23), Xid represents the updated position of grasshopper i in the
phases and described as follows:
d-th dimension. xdj and xdi are two different grasshoppers in the d-th
dimension. N is the number of grasshoppers, and c is a parameter xnew
xi + kf ∗ [(xi− 1 − xi+1 ) + (xr − xi )] f (xi ) > f (xr )
(30)
ij = {
calculated by the Eq. c = cmax − l(cmax − cmin)/L. ubd and lbd are the xi + kf ∗ [(xi− 1 − xi+1 ) + (xi − xr )] f (xi ) ≤ f (xr )
upper bound and lower bound in the d-th dimension, respectively. s(∗) is [( ) ]
the function that defines the social forces, and dij is the distance between xnew
ij = {
xi + kf ∗ [( xp− best − xp− worst ) + (xm − xi )] f (xi ) > f (xm )
(31)
xi + kf ∗ xp− best − xp− worst + (xi − xr ) f (xi ) ≤ f (xm )
the i-th and j-th grasshoppers. G(∗) and T ̂ d are the Gaussian step vector
and the value of the best-so-far solution in the d-th dimension, where xi is the i-th individual. Compared with the current individual xi ,
respectively. α is a Gaussian random number generated in the range of xi− 1 and xi+1 are the nearest better and worse individuals respectively, to
[0,1]. The generation and selection of the new solution are based on Eq. constitute the gain source of knowledge. xp− best and xp− worst are the best
s (24) and (25). Xilevy is a new solution generated based on the Levy flight individual and worst individual, respectively, among all the individuals.
mechanism, and Xi∗ is the new position of the i-th grasshopper after xr and xm are individuals randomly selected from the population, kf is a
updating. Xit+1 is a selected solution based on the fitness value between real number greater than 0 and f(∗) is the objective function.
Xilevy and Xi∗ . rand(d) and levy(∗) are the d-th dimension random vectors in

7
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

(12) Marine predators algorithm (MPA) [115] → →

where C is a concentration vector of the individuals, C eq is a vector that
→
contains the candidates in the equilibrium pool, F is an exponential
MPA mimics marine predators and uses the predation behavior of the →
Lévy and Brownian movements to optimize problems. The optimal term that includes an exponential function, V is considered a unit, and λ
encounter rate policy in the interaction between predator and prey is is a random vector in the range of [0,1]. In EO, the selection process is
→
also considered. In MPA, the optimization process is divided into three completed by comparing the fitness value of C i with the fitness values of
phases due to different velocity ratios. → → → →
C eq1 , C eq2 , C eq3 , and C eq4 and selecting the best one to replace the
Phase 1: When the velocity ratio is high or the prey is moving faster worst one.
than the predator As mentioned above, we can summarize the general mathematical
( )
̅̅̅̅→ → ̅̅→ → ̅̅→ model of the essential operators that generate solutions in the
stepsizei = R B ⊗ Elitei − R B ⊗ Preyi i = 1, ⋯n (32) population-based metaheuristics, which can be described as the new
solution xt+1i being equal to the sum of the current solution xti and the
̅̅→ ̅̅→ → ̅̅̅̅→
Preyi = Preyi + P⋅ R ⊗ stepsizei (33) modification increment or mutation vector Δxti [78], i.e.,

Phase 2: In the unit velocity ratio or when both predator and prey are xt+1
i = xti + Δxti (41)
moving at almost the same pace
The ways to determine Δxti reflect the essential differences among the
• For the first half of the population (i = 1, ⋯n/2) different metaheuristics.
( ) /
̅̅̅̅→ → ̅̅→ → ̅̅→
stepsizei = R L ⊗ Elitei − R L ⊗ Preyi i = 1, ⋯n 2 (34) 3. Experimental analyses

̅̅→ ̅̅→ → ̅̅̅̅→ To evaluate the performance and properties of the newly proposed
Preyi = Preyi + P⋅ R ⊗ stepsizei (35) algorithm, 11 newly named metaheuristics and 4 state-of-the-art meta­
heuristics are selected in this section. We first use a unified framework
named irace to automatically configure the parameters of all 15
• For the second half of the population (i = n/2, ⋯n)
comparative algorithms. Then, whether these algorithms have a search
( ) /
̅̅̅̅→ → → ̅̅→ ̅̅→ bias to the origin is investigated. For detailed description, the conver­
stepsizei = R B ⊗ R B ⊗ Elitei − Preyi i = n 2, ⋯n (36)
gence, diversity, and trade-off between the exploration and exploitation
of all 15 algorithms are also analyzed. All the experimental results were
̅̅→ ̅̅→ ̅̅̅̅→
Preyi = Elitei + P⋅CF ⊗ stepsizei (37) analyzed by nonparametric statistical methods, including the Friedman
test, Wilcoxon signed-ranks test, and Bayesian signed-rank test.

3.1. Experiment setup

Phase 3: In a low-velocity ratio when the predator is moving faster
than the prey In this section, 11 newly proposed representative metaheuristics that
( ) are popular and highly cited and 4 state-of-the-art metaheuristics are
̅̅̅̅→ → → ̅̅→ ̅̅→
stepsizei = R L ⊗ R L ⊗ Elitei − Preyi i = 1, ⋯n (38) selected for the comparison experiments. The 15 algorithms are

̅̅→ ̅̅→ ̅̅̅̅→ Table 1

Preyi = Elitei + P⋅CF ⊗ stepsizei (39)
Summary of the 15 comparison algorithms.
̅̅̅̅→ Algorithm Year Abbreviation
where stepsizei is the step size matrix of the search individuals (predator
̅̅→ • 11 new algorithms
and prey) and Elitei is the matrix that oversees the searching and finding
Aquila optimizer [110] 2017 AO
̅̅→
of the prey based on the information of the prey’s positions. Preyi is the Effective butterfly optimizer with covariance 2017 EBOwithCMAR
→ matrix adapted retreat phase [103] (EBCM)a
function matrix based on which the predators update their positions. R B Snap-drip cuckoo search [104] 2017 SDCS
is a vector containing random numbers that represents a Brownian Improved grasshopper optimization algorithm 2018 IGOA
→ [111]
motion. P is the constant number, and R is a vector that contains a
→ Hyperbolic gravitational search algorithms [112] 2019 HGSA
random number in the interval [0,1]. R L is a vector that contains Memetic frog leaping algorithm [114] 2019 MFLA
random numbers following the Lévy distribution. CF is an adaptive Improved moth-flame optimization algorithm 2020 IMFO
parameter used to control the predator step size. [108]
Multi-strategy enhanced Sine Cosine Algorithm 2020 MSCA
[106]
(13) Equilibrium optimizer (EO) [91] Gaining-sharing knowledge-based algorithm [52] 2019 GSK
Marine predators algorithm [115] 2020 MPA
EO is inspired by the control volume mass balance models that are Equilibrium optimizer [91] 2020 EO
• 4 state-of-the-art algorithms
used to estimate both the dynamic and equilibrium states. In EO, each
L-SHADE with nearest spatial neighborhood- 2017 NLSHADE
individual has its concentration (position), and the best-so-far solution is based modification [101]
named the equilibrium candidate. Each individual randomly updates LSHADE with semi-parameter adaptation hybrid 2017 LSHADE-SPACMA
their concentration around the equilibrium candidates to finally reach with CMA-ES [100] (LS-SPA)
the equilibrium state (optimal result). The updating rule of the in­ Hybrid sampling evolution strategy [102] 2018 HSES
Two enhanced DE variants EDE and EBDE [97] 2019 EDE-EBDE (ED-EB)
dividuals is shown as follows.
( ) → ( Note: In the rest of this paper, we use EBCM, LS-SPA, and ED-EB to represent
→ → → → → G →) EBOwithCMAR, LSHADE-SPACMA, and EDE-EBDE, respectively.
C = C eq + C − C eq ⋅ F + → 1 − F (40) a
λV As a top performer in a CEC competition, it was initially selected as a state-
of-the-art algorithm. But, a reviewer asked us to classify it under newer
algorithms.

8
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

summarized in Table 1. functions with 10 and 30 variables are used.

We select 2017 CEC bound-constrained numerical optimization All experimental results are analyzed by several nonparametric sta­
problems as the benchmark problems [116], which contain thirty tistical methods including the Bayesian rank-sum test, Friedman test,
functions that can be divided into four categories, unimodal functions Wilcoxon signed-rank test, and Bayesian signed-rank test to verify
(F1-F3), multimodal functions(F4-F10), hybrid functions (F11-F20) and whether the performance of two or more algorithms differs from each
composition functions (F21-F30). These functions have the same upper other statistically. If the p value obtained by any two algorithms is equal
bound (100) and lower bound (-100). The global minimum value of each to or less than 0.05, it indicates that there is a significant difference in
function is the product of the function index and 100. In this paper, the the performance of the two algorithms. Otherwise, the opposite is true.
maximum number of function evaluations is set to 10000 ∗D, and all the Details of the statistical results are summarized in the supplementary
experimental results are obtained from average values over 31 runs. file, where the result tables (or figures) are denoted as “TableS# (Fig.
To make fair comparisons [117,118], we first tuned the parameters S#)” and “#” is the table number. All the algorithms are coded in
of all 15 comparative algorithms on all the CEC 2017 functions with 10, MATLAB software and run on a Windows 10 operating system with a
30, and 50 variables. The tuned parameter values are presented in Core i7–10700CPU and 32 G RAM. The codes of this paper have been
Table 2. Afterward, further experiments are conducted from two aspects: published online (http://faculty.csu.edu.cn/guohuawu/zh_CN/zdylm/
performance evaluation and verification of whether these comparison 193832/list/index.htm).
algorithms have a search bias to the origin. In the performance evalu­
ation experiments, all the CEC 2017 functions with 10, 30, and 50
variables are used. To evaluate whether these comparative algorithms 3.2. Automatic parameter tuning
have a search bias to the origin, all the shifted and nonshifted CEC 2017
For a fair comparison, we employed the iterated racing (irace)

Table 2
Parameter tuning results of the 15 algorithms.
Algorithm Default parameters Tuned parameters
10 variables 30 variables 50 variables

AO [110] Population size n=25; number of Population size n=34; Exploitation Population size n=10; Exploitation Population size n=69; Exploitation
clusters m= 5; adjustment parameters α=0.9161; adjustment parameters α=0.4207; adjustment parameters α=0.186;
δ=0.3806 δ=09379 δ=0.6773
SDCS Population size n={15, 25, 35}; Population size n=10; Increase/decrease Population size n=24; Increase/ Population size n=10; Increase/
[104] Increase/decrease rate of rate of pa =0.3413; Movement variability decrease rate of pa =0.1854; Movement decrease rate of pa =0.9137;
pa ={0.005, 0.5, 1}; Movement parameter J=0.8281; Step size scaling variability parameter J=0.9618; Step Movement variability parameter
variability parameter J={0.1, 0.2, factor a0 =0.9491 size scaling factor a0 =0.5973 J=0.9316; Step size scaling factor
0.3}; Step size scaling factor a0 =0.5201
a0 ={0.01, 0.1, 1}
IGOA Population size n =30 Population size n=34 Population size n=35 Population size n=25
[111]
HGSA Population size n=30; Population size n=37; Gravitational Population size n=23; Gravitational Population size n=24; Gravitational
[112] Gravitational constant G0 =100 constant G0 =89 constant G0 =118 constant G0 =116
MFLA Number of memeplexes m={2, 4, Number of memeplexes m=5; Number of Number of memeplexes m=4; Number Number of memeplexes m=4; Number
[114] 5}; Number of frogs in a memeplex frogs in a memeplex n=5; beta=0.7563 of frogs in a memeplex n=6; of frogs in a memeplex n=5;
n={4, 5, 10}; beta=0.6 beta=0.5867 beta=1.4742
IMFO Population size n=100; Spiral Population size n=119; Spiral shape Population size n=118; Spiral shape Population size n=93; Spiral shape
[108] shape parameter b=1; Iteration parameter b=4; Iteration ratio P=0.0199 parameter b=4; Iteration ratio parameter b=3; Iteration ratio
ratio P=0.5 P=0.2963 P=0.3593
MSCA Population size n =30; Probability Population size n=27; Probability factor Population size n=31; Probability factor Population size n=31; Probability
[106] factor Pc =0.8; Constant number Pc =0.0659; Constant number a=1; μ=3 Pc =0.0319; Constant number a=1; μ=4 factor Pc =0.0116; Constant number
a=2; μ=4 is a parameter that is a parameter that controls the degree of is a parameter that controls the degree a=1; μ=4 is a parameter that controls
controls the degree of chaotic chaotic function. of chaotic function. the degree of chaotic function.
function.
GSK [52] Population size n=100; Top and Population size n =101; Top and bottom Population size n =93; Top and bottom Population size n =100; Top and
bottom percentage of individuals percentage of individuals P=0.1353; percentage of individuals P=0.052; bottom percentage of individuals
P=0.1; Knowledge factor kf =0.5; Knowledge factor kf =0.4822; Knowledge factor kf =0.485; Knowledge P=0.0521; Knowledge factor
Knowledge ratio kr =0.9; Knowledge ratio kr =0.9797; Knowledge ratio kr =0.991; Knowledge rate K=10 kf =0.4581; Knowledge ratio
Knowledge rate K=10 rate K=12 kr =0.9309; Knowledge rate K=9
MPA [115] Population size n=25; Probability Population size n=21; Probability factor Population size n=31; Probability factor Population size n=25; Probability
factor FADs={0.1, 0.2, 0.5, 0.7, FADs=0.8297; Constant number FADs=0.1014; Constant number factor FADs=0.3425; Constant
0.9}; Constant number P={0.1, 0.5, P=0.6737 P=0.1949 number P=0.5076
1, 1.5, 2}
EO [91] Population size n=30; Constant Population size n=33; Constant number Population size n=31; Constant number Population size n=20; Constant
number a1 =2; Constant number a1 =1.8876; Constant number a1 =1.9447; Constant number number a1 =1.8587; Constant number
a2 =1; Generation Probability a2 =0.9305; Generation Probability a2 =0.95021; Generation Probability a2 =1.1681; Generation Probability
GP=0.5 GP=0.2999 GP=0.5871 GP=0.7087
EBCM probls =0.1;σ=0.3; arch_rate=2.6; probls =0.9209;σ=0.2997; probls =0.4149;σ=0.9267; probls =0.818;σ=0.019;
[103] Memory size H=6 arch_rate=2.3947; Memory size H=5 arch_rate=3.2152; Memory size H=8 arch_rate=3.0527; Memory size H=4
NLSHADE Population size Np = {50,100}; MF Population size Np = 140; MF and MCr are Population size Np = 138; MF and MCr Population size Np = 164; MF and MCr
[101] and MCr are memory archive memory archive values, MF = 0.8404, are memory archive values, MF = 0.897, are memory archive values, MF =
values, MF = 0.5, MCr = 0.5. MCr = 0.9969. MCr = 0.7163. 0.9039, MCr = 0.792.
HSES Population size M=200; N=100 Population size M=182; N=90 Population size M=181; N=98 Population size M=214; N=92
[100]
LS-SPA pbest =0.3; pbestmin =0.15 pbest =0.416; pbestmin =0.1732 pbest =0.4765; pbestmin =0.1459 pbest =0.2438; pbestmin =0.1749
[102]
ED-EB L_Rate=0.8; EDE_best_rate=0.1; L_Rate=0.;0.2797 EDE_best_rate=0.4957; L_Rate=0.7763; EDE_best_rate=0.1264; L_Rate=0.5034;
[97] Memory size=5 Memory_size=5 Memory_size=6 EDE_best_rate=0.2124; Memory_size=4

Note: The adjustment parameters of each algorithm are determined based on the published paper and the codes are obtained from the authors’ websites.

9
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

method to automatically configure the main parameters. The iterated and F28), five (F4, F23-F24, F26, and F28), and two (F22 and F25)
racing method repeats three steps until it meets a termination criterion: functions. Particularly, the performance of GSK is equivalent to the 4
(1) Sampling new configurations according to a particular distribution; state-of-the-art algorithms (except NLSHADE) in solving the composi­
(2) Selecting the best configurations from the newly sampled configu­ tion function F22. Moreover, GSK is superior or similar to all 5 compe­
rations by means of racing, and (3) Updating the sampling distribution tition algorithms on multimodal function F4 and composition function
to bias the sampling toward the best configurations [119]. F28.
The following issues are interacting: (1) Different algorithms have a IMFO outperforms HSES, ED-EB, LS-SPA, and NLSHADE on five (F1-
different number of parameters requiring different number of function F2, F6, F20, and F25), two (F4 and F25), two (F4 and F25), and one
evaluations; (2) Having too few tuneable parameters may make an al­ (F25) functions.
gorithm to have a fixed characteristic. According to the "No Free Lunch" The MPA achieves better results than the HSES, ED-EB, LS-SPA, and
theorem, algorithms with fixed characteristic may not be efficient for NLSHADE on seven (F2, F6, F20-F21, F23, F26, and F28), three (F4, F26,
solving diverse types of optimization problems; (3) Allocating a single and F28), five (F4, F21, F23, F26, and F28), and one (F25) functions.
fixed number of function evaluations for all algorithms may result in too Particularly, MPA yields promising performance on the composition
few or too many function evaluations for tuning each algorithm. function F26 by surpassing all 4 state-of-the-art algorithms.
Considering all these, we tuned the main parameters of each algorithm In contrast, EO, AO, HGSA, IGOA, MSCA, and SDCS demonstrate less
independently. This approach offers different tuning budgets for efficiency than the 4 state-of-the-art algorithms. For instance, these 6
different algorithms. This is acceptable for offline applications. Param­ recent algorithms are only superior to the 4 state-of-the-art algorithms in
eter adjustment results of the 15 algorithms are shown in Table 2. less than 3 functions. In particular, AO is almost inferior to the 4 state-of-
The irace method is implemented through an R package named irace, the-art algorithms on all thirty functions.
developed by López-Ibáñez et al. [119]. Irace receives as input a In conclusion, EBCM shows competitive performance compared with
parameter space definition corresponding to the parameters of our 15 4 state-of-the-art algorithms. The performance of EBCM completely
algorithms that will be tuned, a set of training instances for which the surpasses HSES and is comparable to ED-EB and LS-SPA on the CEC 2017
parameters must be tuned, and a set of options for the irace that define functions with 30 variables. However, MFLA, GSK, IMFO, and MPA are
the configuration scenario. For example, we tune the AO parameters, inferior to the 4 state-of-the-art algorithms in most functions. EO, AO,
and the training instances are eight functions covering all types of the HGSA, IGOA, MSCA, and SDCS have less efficiency in dealing with CEC
CEC 2017 functions. Then, the irace searches in the parameter search 2017 functions with 30 variables since they are only superior to or
space for good-performing algorithm configurations by executing AO on comparable to the 4 state-of-the-art algorithms in a few functions. The
different functions with different parameter configurations. In other results show that MFLA/GSK/IMFO/MPA/EBCM is superior to HSES,
words, all the parameter configurations will be tested on all the func­ ED-EB, LS-SPA, and NLSHADE on 8/4/4/2/21, 15/3/5/2/14, 5/2/2/1/
tions to verify which is the best performing configuration. For a detailed 14, and 7/3/3/1/5 functions, respectively. In addition, MFLA and GSK
implementation of the irace method, please refer to [119–123]. are comparable to these 3 comparative algorithms (except NLSHADE)
on function F22. EO, AO, HGSA, IGOA, MSCA, and SDCS only perform
better than HSES, ED-EB, LS-SPA, and NLSHADE on 2/1/1/0, 0/0/0/1,
3.3. Experimental results and discussion 1/1/2/1, 0/0/0/1, 2/0/1/1, and 0/1/1/1 function(s). It is worth noting
that 11 recent algorithms become less efficient as the dimension of the
In this section, 11 recent algorithms and 4 state-of-the-art algorithms functions increases (i.e., from 10 variables to 30 variables).
are compared on CEC 2017 functions with 10, 30, and 50 variables,
respectively. The experimental results of the 15 algorithms are sum­ 3.3.1.2. Results of Wilcoxon signed-rank test. As seen from Table 3,
marized in Tables S1-S3. Nonparametric statistical methods, including EBCM performs competitively with 4 state-of-the-art algorithms on the
Friedman test, Bayesian signed-rank, and Wilcoxon signed-rank test. CEC 2017 functions with 30 variables. In addition, GSK exhibits signif­
The detailed statistical results can be found in Tables S4-S22 of the icantly similar performance to HSES in solving functions with 30 vari­
supplemental materials. Due to space limitations, we only show the ables which is consistent with the conclusion of the Bayesian rank-sum
analysis results on functions with 30 variables in the text. For more in­ test and the Friedman test. In contrast, the performances of EO, AO,
formation about the analysis results on functions with 10 and 50 vari­ HGSA, IGOA, IMFO, MFLA, MPA, MSCA, SDCS, and HSES are signifi­
ables, please refer to Sections 3 and 4 of Appendix A in the cantly different from those of the 4 state-of-the-art algorithms. In other
supplementary material. words, these 9 recent algorithms are not efficient in dealing with the
CEC 2017 functions with 30 variables. It is worth noting that some
3.3.1. Benchmark functions with 30 variables recent algorithms, such as MPA, SDCS and MFLA, demonstrate high
efficiency on functions with 10 variables but have a deteriorated per­
3.3.1.1. Comparison of each function. Some interesting observations can formance in solving functions with 30 variables.
be obtained from the statistical results of the functions with 30 variables
reported in Table S2. It is observed that MFLA, GSK, IMFO, MPA, AO, 3.3.1.3. Results of the CD plot. In the cases of the functions with 30
and EBCM exhibit competitive performance among the 11 recent algo­ variables, EBCM exhibits similar performance to the 4 state-of-the-art
rithms as compared with the 4 state-of-the-art algorithms. In particular, algorithms in Fig. 3, and the performance of GSK and MFLA are signif­
EBCM has the best performance among the 11 newly proposed algo­ icantly similar to HSES, LS-SPA, and ED-EB. In addition, there was no
rithms. EBCM outperforms HSES, ED-EB, LS-SPA and NLSHADE on significant difference between MPA and HSES. In contrast, the perfor­
twenty-one (F2, F5-F6, F7-F9, F11, F13, F15, F16-F17, F20-F21 and F23- mance of the other 7 recent algorithms (i.e., AO, IMFO, EO, SDCS,
F30), fourteen (F2, F4, F5, F7-F8, F11-F13, F16-F17, F21, F25-F26, and HGSA, MSCA, and IGOA) is significantly different from that of the 4
F28), fourteen (F5, F7-F8, F10-F11, F13, F16, F21, F23-F26, F28 and state-of-the-art algorithms. The conclusions drawn in this case are
F30), and five (F5-F6, F8, F13 and F25) functions, respectively. similar to the observation results of the Bayesian rank-sum test, the
MFLA is superior to HSES, ED-EB, LS-SPA and NLSHADE on eight Friedman test, and the Wilcoxon signed-rank test.
(F1-F2, F6, F20, and F25-F28), four (F4 and F25-F26), four (F4 and F25-
F26), and two (F22 and F25) functions. Moreover, MFLA exhibits high 3.3.1.4. Convergence analysis. The convergence plots of the 15 algo­
efficiency in dealing with the composition function F25. rithms on functions F1, F3, F4, F10, F11, F19, F21 and F24 with 30
GSK is superior to HSES, ED-EB, LS-SPA and NLSHADE on fifteen variables are shown in Fig. 4. According to Fig. 4, EBCM and 4 state-of-
(F1-F4, F6, F11, F13, F15-F16, F20, F23, and F26-F29), three (F4, F11,

10
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

Table 3 are mutually conflicting and complementary. For more information

The results with significant differences of the Wilcoxon signed-rank test in 10, about how to evaluate the trade-off of algorithms, please refer to [93].
30, and 50 variables. Fig.S8 shows the experimental results of the exploration and
Algorithms 30 variables exploitation trade-off of the 15 algorithms on functions F1, F3, F4, F7,
R+ R− p-value F11, F19, F22 and F24 with 30 variables. Due to space limitations, we
HSES VS EO 444.0 21.0 0.000013 only analyze the results on functions F1, F7, F11 and F22. For more
HSES VS AO 465.0 0.0 0.000002 information about the trade-off analysis of the 15 algorithms on other
HSES VS GSK 230.0 205.0 0.778632 functions, please read the supplementary material. The analysis results
HSES VS HGSA 449.0 16.0 0.000008
of Fig. S8 and Table S2 are summarized as follows.
HSES VS IGOA 465.0 0.0 0.000002
HSES VS IMFO 436.0 29.0 0.000027
HSES VS MFLA 346.0 89.0 0.005281 • Unimodal function F1: In terms of the 4 state-of-the-art algorithms,
HSES VS MPA 375.0 90.0 0.003269 LS-SPA and NLSHADE are the two most prominent algorithms on
HSES VS MSCA 458.0 7.0 0.000003 function F1, with an exploitation of 98.55% and 98.30%, and an
HSES VS SDCS 465.0 0.0 0.000002
exploration of 1.45% and 1.70%, respectively. ED-EB and HSES
HSES VS EBCM 53.0 412.0 1
ED-EB VS EO 460.0 5.0 0.000003 perform slightly worse than LS-SPA and NLSHADE and have trade-
ED-EB VS AO 465.0 0.0 0.000002 off behaviors of exploitation and exploration similar to these of
ED-EB VS GSK 388.0 47.0 0.000218 two top algorithms. They exploited the search space 99.19% and
ED-EB VS HGSA 449.0 16.0 0.000008
97.82% of the time, respectively. In terms of the 10 new algorithms,
ED-EB VS IGOA 465.0 0.0 0.000002
ED-EB VS IMFO 454.0 11.0 0.000005
EBCM performs slightly worse than LS-SPA and NLSHADE but has
ED-EB VS MFLA 386.0 49.0 0.000258 similar trade-off exploitation and exploration behaviors as LS-SPA
ED-EB VS MPA 426.0 39.0 0.000066 and NLSHADE. Meanwhile, the search space is exploited 96.95% of
ED-EB VS MSCA 465.0 0.0 0.000002 the time. MFLA, GSK, and IMFO also perform better among the 11
ED-EB VS SDCS 458.0 7.0 0.000003
newly proposed algorithms, and they spent 85.58%, 99.13%, and
ED-EB VS EBCM 153.0 282.0 1
LS-SPA VS EO 460.0 5.0 0.000003 99.15% of the time exploiting the search space. Although MFLA
LS-SPA VS AO 465.0 0.0 0.000002 employs a different exploration and exploitation rate compared with
LS-SPA VS GSK 347.0 88.0 0.004939 LS-SPA and NLSHADE, it benefits from multiple exploration peaks
LS-SPA VS HGSA 448.0 17.0 0.000009
appearing during the optimization process to jump into different
LS-SPA VS IGOA 465.0 0.0 0.000002
LS-SPA VS IMFO 454.0 11.0 0.000005
search zones and find better solutions. The other new algorithms
LS-SPA VS MFLA 385.0 50.0 0.00028 including EO, AO, MPA, HGSA, IGOA, MSCA and SDCS, are less
LS-SPA VS MPA 410.0 55.0 0.000251 efficient in terms of solution quality and exploit the search space
LS-SPA VS MSCA 461.0 4.0 0.000002 96.78%, 93.31%, 98.84%, 99.56%, 79.76%, 66.20%, and 0.00% of
LS-SPA VS SDCS 458.0 7.0 0.000003
the time, respectively. In particular. SDCS uses excessive exploration
LS-SPA VS EBCM 184.0 251.0 1
NLSHADE VS EO 435.0 0.0 0.000002 (i.e., 100% of the time) in its search process. EO, AO, MPA, and
NLSHADE VS AO 463.0 2.0 0.000002 HGSA maintain a behavior very close to the one used by the top two
NLSHADE VS GSK 417.0 18.0 0.000015 algorithms but the reason for finding different solutions is because of
NLSHADE VS HGSA 460.0 5.0 0.000003
the search mechanism used for exploration and exploitation. More­
NLSHADE VS IGOA 462.0 3.0 0.000002
NLSHADE VS IMFO 459.0 6.0 0.000003
over, MPA, IGOA and MSCA produce a very rough trade-off response.
NLSHADE VS MFLA 450.0 15.0 0.000007 In all cases, the incremental-decremental graph shows that the
NLSHADE VS MPA 428.0 7.0 0.000005 exploration effect is very short, while the exploitation action is
NLSHADE VS MSCA 463.0 2.0 0.000002 prolonged during most of the search time. The best trade-off can be
NLSHADE VS SDCS 460.0 5.0 0.000003
found to be more than 98% exploitation and less than 2% exploration
NLSHADE VS EBCM 121.0 314.0 1
on function F1.
• Multimodal function F7: The 4 state-of-the-art algorithms and EBCM
the-art algorithms have a fast convergence speed and can obtain better are the top five best-performing algorithms for solving function F7,
solutions on these selected functions compared with the other 10 new where NLSHADE achieves the best results with 90.49% exploitation
algorithms. Compared with the 4 state-of-the-art algorithms, AO, MFLA, and 9.52% exploration. Moreover, HSES, ED-EB, and LS-SPA spent
HGSA, IGOA, and MSCA have a slower convergence speed and the worst 98.20%, 93.69% and 96.26% of the time exploiting respectively. In
global search ability (i.e., less efficient) on these selected functions. The terms of the 11 new algorithms, EBCM shows competitive perfor­
other algorithms, such as EO, GAK, IMFO, MPA and SDCS, have a similar mance with 89.52% exploitation and 110.49% exploration. EO,
convergence speed to the 4 state-of-the-art algorithms, but they are MPA, IMFO, and HGSA perform slightly worse than the 4 state-of-
inferior to the 4 state-of-the-art algorithms on most of these select the-art algorithms but their trade-off levels between exploration
functions. Particularly, there is a clear gap between the 4 state-of-the-art and exploitation are close to the 4 state-of-the-art algorithms. The
algorithms and the 11 new algorithms on functions F3, F7, F11, F21 and performance of AO, IGOA, MFLA, GSK, SDCS, and MSCA widens the
F24. These results suggest that the 4 state-of-the-art algorithms have a gap with the 4 state-of-the-art algorithms and has different trade-off
faster convergence speed and a stronger global search ability on most of levels. They spent 97.01%, 78.41%, 83.98%, 92.54%, 98.40%, and
the selected functions. 76.86% of the time exploring the search space, respectively. Partic­
ularly, IGOA, MFLA, and MSCA focus less on exploitation compared
3.3.1.5. The trade-off of exploration and exploitation analysis. We with the 4 state-of-the-art algorithms. On the contrary, SDCS has a
consider the method proposed in Ref. [93] to evaluate the trade-off slightly higher exploitation rate. Moreover, AO, GSK, and SDCS have
between exploration and exploitation of the 15 algorithms. In partic­ similar trade-off levels between exploration and exploitation
ular, the percentage of exploration (i.e., XPL%) and the percentage of compared with the 4 state-of-the-art algorithms but have different
exploitation (i.e., XPT%) are used to evaluate the trade-off response. XPL solution qualities. Once again, it is a good example of how the dif­
% represents the level of exploration as a relationship betwee1111n the ference in the quality of the specific search mechanism of each al­
diversity in each iteration and the maximum reached diversity. XPT% gorithm greatly affects the performance. According to the
corresponds to the level of exploitation. Both elements, XPL% and XPT% incremental-decremental graph, all 15 algorithms focused on

11
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

Fig. 3. The CD plot of algorithms on the CEC 2017 functions with 30 variables.

Fig. 4. Convergence plots on functions with 30 variables.

exploitation, with a trade-off of more than 90% of the time exploring show that MFLA, AO, and MSCA focused less on exploitation
and less than 10% exploiting. compared with the 4 state-of-the-art algorithms. In other words, their
• Hybrid function F11: The top six best-performing algorithms for exploitation and exploration rates are not much different. In
solving function F11 are the 4 state-of-the-art algorithms, EBCM and contrast, EO, MPA, and HGSA seem to focus slightly more on
GSK. NLSHADE achieves the best results that exploited the search exploitation compared with the 4 state-of-the-art algorithms. More­
space 90.58% of the time and explored 9.42% of the time. HSES, ED- over, IGOA, HGSA and MSCA once again produce a rough trade-off
EB and LS-SPA exploited the search space 96.89%, 96.95%, and response, and always seem inefficient. The incremental-
95.14% of the time, respectively. GSK is slightly better than HSES decremental graph shows that the best-performing algorithms that
and ED-EB with an exploitation of 95.03% and 4.97% exploration, prefer exploitation to exploration, and are closer to 90% exploitation
respectively. Regarding the other new algorithms, EBCM obtains and 10% exploration, are used in their search process.
similar results to the 4 state-of-the-art algorithms and exploited the • Composition function F22: The results in Table S2 suggest that the
search space 88.99% of the time. MPA, MFLA, IMFO, and SDCS are most prominent algorithms for solving function F22 are 4 state-of-
inferior to the 4 state-of-the-art algorithms on function F11. They the-art algorithms and EBCM. In addition, GSK, MFLA, and SDCS
explored the search space 97.00%, 54.36%, 96.86%, and 97.04% of are the distant seconds. In terms of the 4 state-of-the-art algorithms,
the time, respectively. The worst five algorithms are AO, EO, IGOA, HSES, ED-EB, and LS-SPA exploited the search space 98.08%,
HGSA, and MSCA, which achieve the exploitation of 53.31%, 98.87%, and 98.55 of the times, respectively. On the contrary, they
97.88%, 79.10%, 99.59%, and 60.31%, respectively. These results spent 1.92%, 1.13%, and 1.45% of their time exploring, respectively.

12
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

NLSHADE focused less on exploitation than the other four state-of- algorithms perform well when solving problems whose optimal solu­
the-art algorithms, with an exploitation of 96.82% and an explora­ tions are located at the origin/center of the search space, but they are
tion of 3.182%. In terms of the 11 new algorithms, EBCM attains less efficient when dealing with the same functions whose optimal so­
similar results to the 4 state-of-the-art algorithms, which exploited lutions deviate from the origin. Liang et al. [124] first evaluated the
the search space 99.27% of the time and explored 0.735% of the performance of the multiagent genetic algorithm by considering the
time. GSK and MFLA have a better performance that achieves an searches biased to the origin. In addition, some newly proposed algo­
exploitation of 97.57% and 81.69%, respectively. AO, MPA, SDCS, rithms, such as TLBO [22,23] and GWO [24], have also been verified to
MSCA, and HGSA are slightly inferior to the seven top algorithms and have a search bias to the origin. In this section, extensive experiments
have different trade-off levels. They spent 87.53%, 97.82%, 99.00%, are carried out on the shifted and non-shifted CEC 2017 functions and
59.21% and 99.54% on exploitation, respectively. Moreover, EO, consider 10 and 30 variables to evaluate whether the 15 algorithms (11
IGOA, and IMFO are the three worst algorithms for solving function recent algorithms and 4 state-of-the-art algorithms) have a search bias to
F22. Although EO, GSK, MPA, HGSA, and SDCS obtain trade-off the origin. Detailed information on the experimental results is shown in
levels that are very similar to those of the best seven algorithms, Tables S23-S24. Nonparametric methods, such as the Friedman test and
they present bad performance in terms of the solution quality. This the Wilcoxon signed-ranks test are used to further analyze the experi­
once again shows the importance of the search mechanisms to obtain mental data.
a better performance. It is important to note that the 4 state-of-the- The results of the Friedman test in Table 4 show that SDCS, MSCA,
art algorithms, GSK and SDCS, produce the smoothest trade-off MFLA, and MPA have a better performance compared with the 4 state-
response, but EO, MPA, HGSA, IGOA, MPLA, and MSCA produce a of-the-art algorithms for solving the nonshifted and shifted functions
rough trade-off response. According to the incremental-decremental with 10 and 30 variables. Particularly, the SDCS achieves the lowest
graph, all the algorithms focused more time on exploitation, and the rank with scores of 3.4 and 3.6, respectively. Compared with the results
best trade-off for function F22 is closer to 99% exploitation and 1% obtained on the shifted functions, the performance of AO, SDCS, MSCA,
exploration. and MFLA are significantly improved for solving the nonshifted func­
tions with 10 and 30 variables. The performance of SDCS, MSCA and
In summary, EBCM has a similar performance and trade-off behavior MFLA are significantly affected by the shift operator on the functions. In
of exploitation and exploration compared with the 4 state-of-the-art other words, these algorithms have search that are biased to the origin.
algorithms. Furthermore, GSK, MPA, MFLA, and IMFO are slightly The performance of the other recent algorithms (i.e., EO, IGOA, and
inferior to the 4 state-of-the-art algorithms but demonstrate better HGSA) is slightly improved for solving the nonshifted functions
performance than the other 6 new algorithms (i.e., except for EBCM). compared with the algorithms dealing with the shifted functions. In
Although each algorithm has different exploitation and exploration addition, the performance of the 4 state-of-the-art algorithms has no
behaviors on each function, all the algorithms focus more time on significant difference in the solution between the nonshifted functions
exploitation, especially the better-performing algorithms. Due to and the shifted functions.
space limitations, we only show the balancing behavior of GSK and The results of the Wilcoxon Signed Ranks test are shown in Tables 5-
EBCM on functions F1, F7, F11 and F22, as shown in Fig. 5. 6. It can be found that the values of R+ are generally greater than the
values of R− , which means that the 15 algorithms can obtain better
3.3.1.6. Diversity analysis. To complement the analysis, an experiment performances in solving the nonshifted functions. In other words, all 15
of diversity on functions F3, F7, F11 and F24 with 30 variables is con­ algorithms have searches biased to the origin and are affected to varying
ducted and the results are presented in Fig. 6. In the experiments, we degrees. For example, the 4 state-of-the-art algorithms are less affected
consider Eq.s (1) and (2) defined in Ref. [93] for a diversity assessment by the shift operator on the functions compared with the 11 recent al­
and these two Eq.s are shown below. gorithms. In contrast, AO, SDCS, MFLA, MSCA, HGSA, and IGOA among
the 11 recent algorithms are greatly affected. In addition, we draw the
1∑ n ⃒
( ) ⃒
CD plots for the experimental results as shown in Figs. 7-8, which have
Divj = ⃒median xj − xij ⃒ (42)
n i=1 similar conclusions to the observations from the Friedman test and the
Wilcoxon signed ranks test.
1 ∑ In conclusion, all 15 comparative algorithms suffer from search
m
Div = Divj (43)
m j=1 biases to the origin to varying degrees. In particular, the 4 state-of-the-
art algorithms are less affected by the shift operator on the functions
where median(xj ) represents the median of dimension j in the whole compared with the 11 recent algorithms.
population. xij is the dimension j of search agent i. n corresponds to the
number of search agents in the population while m symbolizes the 3.3.2.2. The trade-off response of AO, MFLA, MSCA, and SDCS on the
number of design variables of the optimization problem. nonshifted functions. Because AO, MFLA, MSCA, and SDCS maintain
According to Fig. 6, it is clear that all 13 algorithms (i.e., except for good performance in problems where the optimal point is in the origin,
AO and MSCA) begin with a large diversity as a consequence of their we conduct experiments to investigate the trade-off response of these
random initialization. As the number of iterations increase, the popu­ four algorithms on functions F7 and F24 with 10 and 30 variables, and
lation diversity diminishes. AO and MSCA also begin with a large di­ the results are presented in Figures S13-S14. As can be seen from Fig.
versity but they have a certain population diversity at the final stage of S13, the evolution of the trade-off in the nonshifted functions for AO
iteration. Especially the diversity of AO on functions F11 and F24 first demonstrates consistency with the trade-off made by them on the shifted
decreases and then increases gradually with the iterations. Most of the functions. MFLA focuses slightly less on exploration when considering
11 new algorithms show a rough trade-off response, especially MPA, functions that are shifted. It exploited the search space 83.53% of the
MSCA, MFLA, IGOA and HGSA, which exhibit high oscillation behavior. time on the nonshifted function F7 while 72.68% of the time exploiting
Compared with the 11 new algorithms, the 4 state-of-the-art algorithms on the shifted function F7. Moreover, MFLA spent 46.87% and 42.70%
show the smoothest diversity responses. of the time exploiting the search space of the nonshifted and shifted
function F24, respectively. In contrast, MSCA and SDCS focused slightly
3.3.2. Results of CEC 2017 functions considering nonshifted and shifted more exploration on the nonshifted functions than the shifted functions.
According to Fig.S14, MFLA, MSCA, and SDCS present consistent trade-
3.3.2.1. Evaluate the search bias toward the origin. In the literature, some off responses between nonshifted and shifted functions. The case of AO

13
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

Fig. 5. The balancing behavior of GSK and EBCM on functions F1, F7, F11, and F22.

14
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

Fig. 6. Diversity analysis on functions F3, F7, F11, and F24 with 30 variables.

15
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

Table 4
The results of the Friedman test on the nonshifted and shifted CEC2017

NLSHADE-

NLSHADE-
nonshifted
functions.

shifted

>=0.2
309.0
126.0
Algorithms Average ranking on functions Average ranking on functions

VS
with 10 variables with 30 variables
nonshifted shifted nonshifted shifted

VS LS-SPA-
nonshifted

0.002464
LS-SPA-
EO 9.3333 (9) 11.0333 (12) 8.6 (8) 8.1167 (10)

shifted

375.5
MPA 5.9833 (5) 3.85 (1) 6.3667 (6) 7.55 (9)

89.5
GSK 10.1 (11) 7.75 (9) 11.2333 (13) 7.3667 (4)
MSCA 4.2833 (2) 10.4333 (11) 4.35 (3) 9.55 (14)
IMFO 11.6167 (14) 8.7667 (10) 13.7667 (15) 7.5 (8)

nonshifted
VS ED-EB-
MFLA 4.5333 (3) 5.5667 (5) 4.3833 (4) 7.3667 (4)

ED-EB-

0.1723
shifted

281.5
153.5
HGSA 11.75 (15) 12 (13) 12.3833 (14) 8.5333 (12)
IGOA 11.6 (13) 12.4667 (14) 9.4833 (11) 9.0833 (13)
SDCS 3.9333 (1) 7.65 (8) 4 (2) 8.25 (11)
AO 5.0333 (4) 13.8667 (15) 3.8167 (1) 9.85 (15)

nonshifted
VS EBCM-

0.05064
NLSHADE 9.3833 (10) 5.7 (6) 6.3 (5) 7.3667 (4)

EBCM-

shifted

308.0
127.0
EBCM 7.7667 (7) 4.1833 (2) 8.733 (9) 7.3667 (4)
HSES 10.1667 (12) 7.4333 (7) 10.1667 (12) 7.3667 (4)
LS-SPA 6.4333 (6) 4.75 (4) 7.45 (7) 7.3667 (4)
ED-EB 8.0833 (8) 4.55 (3) 8.9167 (10) 7.3667 (4)

nonshifted
VS HSES-
shifted

>=0.2
HSES-

274.0
161.0
on function F24 is interesting. AO produces a rougher trade-off response
on the shifted F24 than on the nonshifted F24. In particular, AO

nonshifted
employed a trade-off of 96.91% exploitation and 3.09% exploration on

1.9522E-5
VS SDCS-
the problem where the optimal solution is in the origin. On the shifted

shifted
SDCS-

423.5
41.5
function F24, AO spent 76.94% on exploiting and 23.07% on exploring.
Especially the SDCS maintains the same trade-off response between
nonshifted and shifted. Nevertheless, its performance in terms of quality

nonshifted

1.3038E-8
VS MSCA-
is significantly better than the nonshifted functions. This seems to

MSCA-

shifted

461.0
indicate that the SDCS maintains a fixed trade-off without considering

4.0
the function type. In a word, it is important to point out that these four
algorithms present a rough trade-off response on non-shifted and shifted

nonshifted
VS MPA-

0.01799
functions F7 and F24. These results suggest that the search mechanisms

shifted

346.5
118.5
MPA-
used by these algorithms seem to have a significant impact on their
performance, which we will investigate in our future work.
nonshifted
VS MFLA-

2.697E-4
4. Issues and suggestions for future research
The results of Wilcoxon signed-ranks test on the shifted and non-shifted functions with 10 variables.

shifted
MFLA-

400.5
64.5
Despite the fruitful results of metaheuristic research in the past few
decades, there are still some suggestions and interesting open problems
nonshifted
VS IMFO-

that need to be investigated in future research.

shifted

>=0.2
IMFO-

270.0
165.0

• Fair and comprehensive comparisons: For a fair comparison, it is

necessary to configure the parameters of all the comparative algo­
nonshifted

rithms using the same satisfactory parameter tuning approach, as the

VS IGOA-

8.718E-4
shifted
IGOA-

performance of metaheuristics is severely affected by the parameter

388.0
77.0

settings. In addition, when evaluating the performance of a newly

proposed algorithm, it is required to compare with state-of-the-art
nonshifted

algorithms on comprehensive and representative benchmark suits.

1.3966E-5
VS HGSA-
shifted

Almost all the metaheuristic algorithms are stochastic, which means

HGSA-

426.0
39.0

that they may obtain results of different quality in different runs.

Therefore, rigorous statistical tests are useful in comparing different
metaheuristic algorithms [125,126]. In some cases, the details of an
nonshifted
VS GSK-

algorithm are not fully explained due to space limitations, which

shifted

>=0.2
278.5
186.5
GSK-

may result in inaccurate replication and inconsistent computation

results. Thus, it is highly recommended that the authors make the
source codes publicly available.
nonshifted

0.000002

• Improve and propose metaheuristics from search behavior and

VS AO-
shifted

465.0

optimization mechanisms perspectives: Recent metaheuristics are

AO-

0.0

proposed according to phenomena from biology, nature, physics, and

so on. However, the effective performance of metaheuristics essen­
nonshifted

tially depends on the search behaviors and optimization mecha­

VS EO-

0.0083
shifted

337.5

nisms. For example, the neighborhood structures (e.g., one-point

97.5
EO-

exchange and multiple-point exchange) in single-solution based

metaheuristics, and the operators (e.g., crossover, mutation, and
p-value
Table 5

recombination) in population-solution based metaheuristics play

No.

R+
R−

crucial roles in the high performance of the optimizers. Besides, how

16
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

to guide the search directions to a promising region in the solution

space is another promising optimization mechanism. In particular,

NLSHADE-
nonshifted

NLSHADE
balancing exploration and exploitation to improve the performance

-shifted

>=0.2
289.0
146.0
of metaheuristics is significant. Therefore, we appeal to researchers

VS
to improve and propose metaheuristics not only from the inspiration
source but also from the perspectives of the essential search behav­

nonshifted
VS LS-SPA
iors and optimization mechanisms.

LS-SPA-

-shifted

0.0221
343.0
122.0
• Automatic design and configuration of metaheuristics: The
design and configuration of metaheuristics can be considered an
optimization problem. Traditional methods depend on prior knowl­
nonshifted
VS ED-EB

0.008301
-shifted edge and trial-and-error methods to obtain a configuration. Auto­
ED-EB-

337.5
matic design and configuration methods are attracting attention in

97.5
the fields of metaheuristics [127]. It not only saves a substantial
amount of human effort during the empirical analysis and design of
nonshifted
VS EBCM-

0.008008 metaheuristics but also leads to high-performance optimizers [128].

EBCM-

shifted

Therefore, it is worth using automatic methods to design and

338.0
97.0

configure metaheuristics. For example, there are many operators,

neighborhood structures, parameters, and mechanisms of informa­
nonshifted

tion sharing and learning in the component pool. These components

0.012834
VS HSES-
shifted

may be adaptively automatically selected from the component pool

HSES-

352.0
113.0

based on the features of the problems, and can be effectively com­

bined to design efficient algorithms for solving the specific problems.
With regard to metaheuristic design, LaTorre et al. [117] suggested
nonshifted

1.6392E-7
VS SDCS-

that simplicity should be considered one of the preferential aspects in

shifted
SDCS-

452.0
13.0

the design of new optimization techniques. Particularly, some new

algorithms are improved on previous algorithms by updating or
adding new strategies to their search procedure. Each improve­
nonshifted

1.8626E-9
VS MSCA-

ment/component that affects the performance of the new algorithm

MSCA-

shifted

465.0

needs to be further analyzed [129].

0.0

• Combining machine learning techniques with metaheuristics:

Machine learning (ML) has achieved fruitful results in recent de­
nonshifted

5.054E-4
VS MPA-

cades. ML’s powerful learning, prediction, and decision-making ca­

shifted

394.0
MPA-

pabilities have opened a new horizon for metaheuristic research. It is

71.0

promising to combine ML and metaheuristics in the following as­

pects: 1) A combination of meta-heuristics and deep learning, rein­
nonshifted
VS MFLA-

forcement learning, ensemble learning, etc., and reasonable

The results of Wilcoxon signed-ranks test on the shifted and non-shifted functions with 30 variables.

shifted
MFLA-

4.7E-5
392.0

recommendation of optimization algorithms for specific problems

43.0

[130,131]. 2) Using ML techniques to help to model optimization

problems, analyze the solution space, and perform problem decom­
nonshifted
VS IMFO-

position [132,133]. 3) ML can use historical data to dynamically

shifted

>=0.2
IMFO-

adjust parameter values during the optimization process of meta­

223.0
242.0

heuristics. Besides, when metaheuristics have multiple operators and

search mechanisms, ML is a prevalent and effective method for
nonshifted

learning the characteristics of these operators and mechanisms, and

VS IGOA-

9.314E-9
shifted

for generating the appropriate algorithmic configuration [134,135].

IGOA-

462.0
3.0

• Integrate problem domain knowledge into metaheuristics:

Integrating algorithms with problem domain knowledge can
improve the performance of the algorithms. For instance, designing
nonshifted
VS HGSA-

3.454E-5

the operators and search mechanisms of metaheuristics based on the

shifted
HGSA-

419.0
46.0

problem characteristics leads to having the search directions of the

algorithms based on the landscapes of the problem. In addition, the
optimality conditions of the problems can also be used to reduce the
nonshifted
VS GSK-

0.04048

variables and the difficulty of the problems considered [136].

shifted

332.0
133.0
GSK-

• Application to complex real-world optimization problems: Most

real-world optimization problems are large-scale, with complex
constraints, high-dimensional objectives, continuous variables and
nonshifted

0.000002

discrete variables. However, metaheuristics also face quite a few

VS AO-
shifted

465.0

challenges when solving these complex real-world optimization

AO-

0.0

problems. It is efficient to combine metaheuristics with surrogate

models [1] such as parallel acceleration and simulation optimization
nonshifted

5.592E-5

to solve complex real-world optimization problems.

VS EO-
shifted

415.0
50.0
EO-

5. Conclusions
p-value
Table 6

In this paper, we provide a comprehensive review of metaheuristics.

No.

R+
R−

More than 500 newly proposed and improved metaheuristics are

17
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

Table 7
List of metaheuristics (This list will be posted at: https://github.com/P-N-Suganthan).
Refs Year Full name & abbreviation Refs Year Full name & abbreviation

B1 1960 Evolutionary Programming, EP B257 2016 Water Evaporation Optimization, WEO

B2 1964 Evolution Strategies, ES B258 2016 Root Tree Optimization Algorithm, RTO
B3 1971 Genetic Algorithm, GA B259 2016 FIFA World Cup Algorithm, FIFAWC
B4 1977 Scatter Search Algorithm, SSA B260 2016 Sperm Whale Algorithm, SWA
B5 1981 Genetic Programming, GP B261 2016 Virus Optimization Algorithm, VOA
B6 1983 Simulated Annealing, SA B262 2016 Duelist Algorithm, DA
B7 1986 Tabu Search Algorithm, TSA B263 2016 Raven Roosting Optimization Algorithm, RROA
B8 1989 Stochastic Search Network, SSN B264 2016 Ring Seal Search, RSS
B9 1989 Memetic Algorithm, MA B265 2016 Flying Elephant Algorithm, FEA
B10 1992 Ant Colony Optimization, ACO B266 2016 Camel Algorithm, CA
B11 1993 Shuffled Complex Evolution, SCE B267 2016 Crystal Energy Optimization Algorithm, CEO
B12 1993 Great Deluge Algorithm, GDA B268 2016 Passing Vehicle Search, PVS
B13 1994 Cultural Algorithms, CA B269 2016 Tug Of War Optimization, TWO
B14 1995 Differential Evolution, DE B270 2016 Dynamic Virtual Bats Algorithm, DVBA
B15 1995 Particle Swarm Optimization, PSO B271 2016 Lion Optimization Algorithm, LOA
B16 1995 Old Bachelor Acceptance, OBA B272 2016 Natural Forest Regeneration Algorithm, NFR
B17 1996 Bacterial Evolutionary Algorithm, BEA B273 2016 Simulated Kalman Filter, SKF
B18 1997 Variable Neighbourhood Descent Algorithm, VND B274 2016 Shuffled Multi-Swarm Micro-Migrating Birds Optimization, SM2-MBO
B19 1998 Bee System, BS1 B275 2016 Yin-Yang-Pair Optimization, YYPO
B20 1998 Photosynthetic Learning Algorithm, PLA B276 2016 Virulence Optimization Algorithm, VOA
B21 1998 Chaos Optimization Algorithm, COA B277 2017 Artificial Butterfly Optimization, ABO
B22 1999 Sheep Flocks Heredity Model, SFHD B278 2017 Cyclical Parthenogenesis Algorithm, CPA
B23 1999 Extremal Optimization, EO B279 2017 Deterministic Oscillatory Search, DOS
B24 1999 Gravitational Clustering Algorithm, GCA B280 2017 Fractal-based Algorithm, FA
B25 2000 Clonal Selection Algorithm, CSA B281 2017 Neuronal Communication Algorithm, NCA
B26 2001 Harmony Search Algorithm, HSA B282 2017 Lightning Attachment Procedure Optimization, LAPO
B27 2001 Gene Expression Programming, GEP B283 2017 Bison Behavior Algorithm, BBA
B28 2001 Marriage In Honey Bees Optimization, MBO B284 2017 Drone Squadron Optimization, DSO
B29 2002 Bacterial Foraging Algorithm, BFA B285 2017 Human Behavior-Based Optimization, HBO
B30 2002 Bacteria Chemotaxis Algorithm, BCA B286 2017 Vibrating Particles System, VPS
B31 2002 Bee System, BS2 B287 2017 Spotted Hyena Optimizer, SHO
B32 2002 Popmusic Algorithm, POPMUSIC B288 2017 Salp Swarm Algorithm, SSA
B33 2002 Social Cognitive Optimization, SCO B289 2017 Grasshopper Optimisation Algorithm, GOA
B34 2003 Artificial Fish Swarm Algorithm, AFSA B290 2017 Rain Fall Optimization Algorithm, RFO
B35 2003 Covariance Matrix Adaptation–Evolution Strategy, CMA-ES B291 2017 Hydrological Cycle Algorithm, HCA
B36 2003 Society and Civilization, SC B292 2017 Killer Whale Algorithm, KWA
B37 2003 Artificial Immune System, AIS B293 2017 Camel Herd Algorithm, CHA
B38 2003 Queen-bee Evolution, QBE B294 2017 Collective Decision Optimization Algorithm, CDOA
B39 2003 Electromagnetism-Like Mechanism Optimization, EMO B295 2017 Laying Chicken Algorithm, LCA
B40 2004 Beehive Algorithm, BHA B296 2017 Kidney-Inspired Algorithm, KIA
B41 2004 Self-Organizing Migrating Algorithm, SOMA B297 2017 Golden Sine Algorithm, Gold-SA
B42 2005 Artificial Bee Colony Algorithm, ABCA B298 2017 Sperm Motility Algorithm, SMA
B43 2005 Bee Colony Optimization, BCO B299 2017 Rain Water Algorithm, RWA
B44 2005 Bees Swarm Optimization Algorithm, BSOA B300 2017 Thermal Exchange Optimization, TEO
B45 2005 Dendritic Cells Algorithm, DCA B301 2017 Porcellio Scaber Algorithm, PSA
B46 2005 The Bees Algorithm, BA B302 2017 Selfish Herd Optimizer, SHO
B47 2005 Wasp Swarm Optimization, WSO B303 2017 Polar Bear Optimization Algorithm, PBO
B48 2006 Shuffled Frog-Leaping Algorithm, SFLA B304 2017 Social Engineering Optimization, SEO
B49 2006 Big Bang–Big Crunch, BBC B305 2017 Sonar Inspired Optimization, SIO
B50 2006 Cat Swarm Optimization, CSO B306 2017 Weighted Superposition Attraction, WSA
B51 2006 Flocking base Algorithm, FA B307 2017 Satin Bowerbird Optimizer, SBO
B52 2006 Honey-bees Mating Optimization Algorithm, HBMO B308 2018 Artificial Atom Algorithm, A3
B53 2006 Small-World Optimization Algorithm, SWOA B309 2018 Artificial Swarm Intelligence, ASI
B54 2006 Saplings Growing Up Algorithm, SGUA B310 2018 Bees Life Algorithm, BLA
B55 2006 Seeker Optimization Algorithm, SOA B311 2018 Beetle Swarm Optimization Algorithm, BSOA
B56 2006 Weed Colonization Optimization, WCO B312 2018 Brunsvigia Optimization Algorithm, BVOA
B57 2007 Imperialist Competitive Algorithm, ICA B313 2018 Car Tracking Optimization Algorithm, CTOA
B58 2007 Monkey Search Algorithm, MSA B314 2018 Cheetah Based Algorithm, CBA
B59 2007 River Formation Dynamics, RFD B315 2018 Cheetah Chase Algorithm, CCA
B60 2007 Bacterial Swarming Algorithm, BSA B316 2018 Chaotic Crow Search Algorithm, CCSA
B61 2007 Bacterial-GA Foraging, BF B317 2018 Circular Structures of Puffer Fish Algorithm, CSPF
B62 2007 Parliamentary Optimization Algorithm, POA B318 2018 Competitive Learning Algorithm, CLA
B63 2007 Simplex Algorithm, SA B319 2018 Cricket Chirping Algorithm, CCA
B64 2007 Good Lattice Swarm Algorithm, GLSA B320 2018 Fibonacci Indicator Algorithm, FLA
B65 2007 Central Force Optimization, CFO B321 2018 Plant Self-Defense Mechanism Algorithm, PSDM
B66 2008 Fast Bacterial Swarming Algorithm, FBSA B322 2018 Emperor Penguin Optimizer, EPO
B67 2008 Biogeography-based Optimization, BBO B323 2018 Lion Pride Optimization Algorithm, LPOA
B68 2008 Bar Systems, BS B324 2018 Multi-Scale Quantum Harmonic Oscillator Algorithm, MQHO
B69 2008 Catfish Particle Swarm Optimization, CatfishPSO B325 2018 Mushroom Reproduction Optimization, MRO
B70 2008 Goose Team Optimizer, GTO B326 2018 Tree Growth Algorithm, TGA
B71 2008 Harmony Element Algorithm, HEA B327 2018 Moth Search Algorithm, MSA
B72 2008 Fish-School Search, FSF B328 2018 Farmland Fertility, FF
B73 2008 Roach Infestation Optimization, RIO B329 2018 Pity Beetle Algorithm, PBA
B74 2008 Viral Search, VS B330 2018 Mouth Brooding Fish Algorithm, MBF
(continued on next page)

18
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

Table 7 (continued )
Refs Year Full name & abbreviation Refs Year Full name & abbreviation

B75 2008 Plant Growth Optimization, PGO B331 2018 Artificial Flora Optimization Algorithm, AFOA
B76 2009 Artificial Beehive Algorithm, ABA B332 2018 Elephant Swarm Water Search Algorithm, ESWS
B77 2009 Artificial Physics Optimization, APO B333 2018 Sperm Swarm Optimization Algorithm, SSOA
B78 2009 Bee Colony-inspired Algorithm, BCiA B334 2018 Team Game Algorithm, TGA
B79 2009 Gravitational Emulation Local Search, GELS B335 2018 Coyote Optimization Algorithm, COA
B80 2009 Group Search Optimizer, GBO B336 2018 Queuing Search Algorithm, QSA
B81 2009 Cuckoo Search, CS B337 2018 Supernova Optimizer, SO
B82 2009 Gravitational Search Algorithm, GSA B338 2018 Spiritual Search, SS
B83 2009 Firefly Algorithm, FA B339 2018 School Based Optimization, SBO
B84 2009 Frog Call inspired Algorithm, FCA B340 2018 Weighted Vertices Optimizer, WVO
B85 2009 Glowworm Swarm Optimization, GSO B341 2018 Volleyball Premier League Algorithm, VPLA
B86 2009 League Championship Algorithm, LCA B342 2018 Yellow Saddle Goatfish Algorithm, YSGA
B87 2009 Paddy Field Algorithm, PFA B343 2019 Raccoon Optimization Algorithm, ROA
B88 2009 Dolphin Partner Optimization, DPO B344 2019 Andean Condor Algorithm, ACA
B89 2009 Dialectic Search, DS B345 2019 Anglerfish Algorithm, AA
B90 2009 Human-Inspired Algorithms, HIA B346 2019 Artificial Ecosystem-Based Optimization, AEO
B91 2009 Artificial Searching Swarm Algorithm, ASSA B347 2019 Atom Search Optimization Algorithm, ASOA
B92 2009 Bumble Bees Mating Optimization, BBMO B348 2019 Artificial Feeding Birds, AFB
B93 2009 Group Counseling Optimization, GCO B349 2019 Artificial Coronary Circulation System, ACCS
B94 2009 Hunting Search Algorithm, HSA B350 2019 Artificial Electric Field Algorithm, AEFA
B95 2009 Locust Swarm, LS B351 2019 Bus Transportation Algorithm, BTA
B96 2009 Intelligent Water Drops Algorithm, IWDA B352 2019 Biology Migration Algorithm, BMA
B97 2009 Water Flow Algorithm, WFA B353 2019 Buzzard Optimization Algorithm, BUZOA
B98 2010 Asexual Reproduction Optimization, ARO B354 2019 Blue Monkey Algorithm, BM
B99 2010 Bean Optimization Algorithm, BOA B355 2019 Chaotic Dragonfly Algorithm, CDA
B100 2010 Bat Algorithm, BA B356 2019 Cultural Coyote Optimization Algorithm, CCOA
B101 2010 Bee Swarm Optimization, BSO B357 2019 Dice Game Optimizer, DGO
B102 2010 Charged System Search, CSS B358 2019 Donkey Theorem Optimization, DTO
B103 2010 Chemical Reaction Optimization Algorithm, CRO B359 2019 Deer Hunting Optimization Algorithm, DHOA
B104 2010 Gravitational Field Algorithm, GFA B360 2019 Falcon Optimization Algorithm, FOA
B105 2010 Fireworks Algorithm, FA B361 2019 Find-Fix-Finish-Exploit-Analyze Algorithm, F3EA
B106 2010 Eagle Strategy, ES B362 2019 Flow Regime Algorithm, FRA
B107 2010 Grenade Explosion Algorithm, GEA B363 2019 Chaotic Optimal Foraging Algorithm, COFA
B108 2010 Wind Driven Optimization, WDO B364 2019 Naked Moled Rat, NMR
B109 2010 Termite Colony Optimization, TCO B365 2019 Xerus Optimization Algorithm, XOA
B110 2010 Consultant-Guided Search, CGS B366 2019 Nuclear Reaction Optimization, NRO
B111 2010 Social Emotional Optimization Algorithm, SEOA B367 2019 Hypercube Natural Aggregation Algorithm, HNAA
B112 2010 Hierarchical Swarm Model, HSM B368 2019 Sailfish Optimizer, SO
B113 2010 Reincarnation Algorithm, RA B369 2019 The Algorithm of the Innovative Gunner, AIG
B114 2011 Artificial Plants Optimization Algorithm, APO B370 2019 Supply-Demand-Based Optimization, SDBO
B115 2011 Brain Storm Optimization, BSO B371 2019 Butterfly Optimization Algorithm, BOA
B116 2011 Bioluminescent Swarm Optimization Algorithm, BSOA B372 2019 Emperor Penguins Colony, EPC
B117 2011 Cockroach Swarm Optimization, CSO B373 2019 Electron Radar Search Algorithm, ERSA
B118 2011 Group Escape Behavior, GEB B374 2019 Henry Gas Solubility Optimization, HGSO
B119 2011 Group Leaders Optimization Algorithm, GIOA B375 2019 Hitchcock Bird-Inspired Algorithm, HBIA
B120 2011 Teaching-Learning Base Optimization, TLBO B376 2019 Hammerhead Shark Optimization Algorithm, HOA
B121 2011 Cuckoo Optimization Algorithm, COA B377 2019 Fitness Dependent Optimizer, FDO
B122 2011 Artificial Chemical Reaction Optimization Algorithm, ACROA B378 2019 Life Choice-Based Optimizer, LCBO
B123 2011 Galaxy-Based Search Algorithm, GBSA B379 2019 Parasitism–Predation Algorithm, PPA
B124 2011 Spiral Dynamics Inspired Optimization, SDIO B380 2019 Pathfinder Algorithm, PA
B125 2011 Plant Propagation Algorithm, PPA B381 2019 Poor And Rich Optimization Algorithm, PROA
B126 2011 Eco-Inspired Evolutionary Algorithm, EIEA B382 2019 Seagull Optimization Algorithm, SOA
B127 2011 Gravitational Interactions Optimization, GIO B383 2019 Sooty Tern Optimization Algorithm, STOA
B128 2011 Stem Cells Algorithm, SCA B384 2019 Harris Hawks Optimization, HHO
B129 2011 Water-Flow Algorithm, WFA B385 2019 Bonobo Optimizer, BO
B130 2012 Anarchic Society Optimization, ASO B386 2019 Spherical Search Optimizer, SSO
B131 2012 Artificial Tribe Algorithm, ATA B387 2019 Squirrel Search Algorithm, SSA
B132 2012 Bat Intelligence, BI B388 2019 Flying Squirrel Optimizer, FSO
B133 2012 Collective Animal Behavior, CAB B389 2019 Bald Eagle Search Optimisation Algorithm, BESO
B134 2012 Cloud Model-based Differential Evolution Algorithm, CMDE B390 2019 Search And Rescue Optimization Algorithm, SAR
B135 2012 Flower Pollination Algorithm, FPA B391 2019 Wild Mice Colony Algorithm, WMC
B136 2012 Flock by Leader, FL B392 2019 Thieves And Police Algorithm, TPA
B137 2012 Krill Herd Algorithm, KHA B393 2020 Artificial Transgender Longicorn Algorithm, ATLA
B138 2012 Fruit Fly Optimization Algorithm, FFOA B394 2020 Barnacles Mating Optimizer, BMO
B139 2012 Water Cycle Algorithm, WCA B395 2020 Black Hole Mechanics Optimization, BHMO
B140 2012 Differential Search Algorithm, DSA B396 2020 Billiards-Inspired Optimization Algorithm, BIOA
B141 2012 Ray Optimization, RO B397 2020 Border Collie Optimization, BCO
B142 2012 Migrating Bird Optimization, MBO B398 2020 Bear Smell Search Algorithm, BSSA
B143 2012 Wolf Search Algorithm, WSA B399 2020 Buyer Inspired Meta-Heuristic Optimization Algorithm, BIMHO
B144 2012 Mine Blast Algorithm, MBA B400 2020 Darts Game Optimizer, DGO
B145 2012 Electro-Magnetism Optimization, EMO B401 2020 Dynamic Differential Annealed Optimization, DDAO
B146 2012 Bacterial Colony Optimization, BCO B402 2020 Dynastic Optimization Algorithm, DOA
B147 2012 Great Salmon Run, GSR B403 2020 Forensic Based Investigation, FBI
B148 2012 Japanese Tree Frogs Calling Algorithm, JTFC B404 2020 Plasma Generation Optimization, PGO
B149 2012 Community of Scientist Optimization, CSO B405 2020 Newton Metaheuristic Algorithm, NMA
B150 2012 Quantum-inspired Bacterial Swarming Optimization, QBSO B406 2020 Tunicate Swarm Algorithm, TSA
(continued on next page)

19
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

Table 7 (continued )
Refs Year Full name & abbreviation Refs Year Full name & abbreviation

B151 2012 Hoopoe Heuristic Optimization, HH B407 2020 Marine Predators Algorithm, MPA
B152 2012 Intelligent Gravitational Search Algorithm, IGSA B408 2020 Equilibrium Optimizer, EO
B153 2012 Lion Pride Optimizer, LPO B409 2020 Electric Fish Optimization, EFO
B154 2012 Zombie Survival Optimization, ZSO B410 2020 Slime Mould Algorithm, SMA
B155 2012 Artificial Photosynthesis and Phototropism Mechanism, APPM B411 2020 Black Widow Optimization Algorithm, BWOA
B156 2012 Superbug Algorithm, SA B412 2020 Manta Ray Foraging Optimization, MRFO
B157 2013 Artificial Plant Optimization Algorithm, APOA B413 2020 Mayfly Algorithm, MA
B158 2013 Artificial Reaction Algorithm, ARA B414 2020 Orcas Algorithm, OA
B159 2013 Adaptive Social Behavior Optimization, ASBO B415 2020 Political Optimizer, PO
B160 2013 Bat-Inspired Algorithm, BI B416 2020 Group Teaching Optimization Algorithm, GTOA
B161 2013 Co-Operation Of Biology Related Algorithm, COBRA B417 2020 Turbulent Flow Of Water-Based Optimization, TFWO
B162 2013 Global Neighborhood Algorithm, GNA B418 2020 Human Urbanization Algorithm, HUA
B163 2013 Mosquito Host-Seeking Algorithm, MHSA B419 2020 Chimp Optimization Algorithm, COA
B164 2013 Mobility Aware-Termite, MAT B420 2020 Coronavirus Optimization Algorithm, COA
B165 2013 Backtracking Search Optimization, BSO B421 2020 COVID-19 Optimizer Algorithm, CVA
B166 2013 Black Holes Algorithm, BHA B422 2020 Multivariable Grey Prediction Evolution Algorithm, MGPE
B167 2013 Social Spider Optimization, SSO B423 2020 Sandpiper Optimization Algorithm, SOA
B168 2013 Dolphin Echolocation, DE B424 2020 Shuffled Shepherd Optimization Method, SSOM
B169 2013 Artificial Cooperative Search, ACS B425 2020 Red Deer Algorithm, RDA
B170 2013 Gases Brownian Motion Optimization, GBMO B426 2020 Golden Ratio Optimization Method, GTOM
B171 2013 Swallow Swarm Optimization Algorithm, SSOA B427 2020 Gaining-Sharing Knowledge Based Algorithm, GSKA
B172 2013 Penguins Search Optimization Algorithm, PSOA B428 2020 Adolescent Identity Search Algorithm, AISA
B173 2013 Egyptian Vulture Optimization, EVO B429 2020 Capuchin Search Algorithm, CSA
B174 2013 Atmosphere Clouds Model Optimization, ACMO B430 2020 Giza Pyramids Construction, GPC
B175 2013 Magnetotactic Bacteria Optimization Algorithm, MBOA B431 2020 Grand Tour Algorithm, GTA
B176 2013 Blind, Naked Mole-Rats Algorithm, BNMR B432 2020 Groundwater Flow Algorithm, GFA
B177 2013 Soccer Game Optimization, SGO B433 2020 Gradient-Based Optimizer, GO
B178 2013 Seven-Spot Ladybird Optimization, SSLO B434 2020 Interactive Autodidactic School, IAS
B179 2013 Cuttlefish Algorithm, CA B435 2020 LÉVy Flight Distribution, LFD
B180 2013 African Wild Dog Algorithm, AWDA B436 2020 Momentum Search Algorithm, MSA
B181 2013 Mussels Wandering Optimization, MWO B437 2020 Nomadic People Optimizer, NPO
B182 2013 Swine Influenza Models Based Optimization, SIMB B438 2020 New Caledonian Crow Learning Algorithm, NCCL
B183 2013 Tree Physiology Optimization, TPO B439 2020 Horse Optimization Algorithm, HOA
B184 2014 Animal Behavior Hunting, ABH B440 2020 Rao Algorithms, RA
B185 2014 Artificial Raindrop Algorithm, ARA B441 2020 Rat Swarm Optimizer, RSO
B186 2014 Grey Wolf Optimizer, GWO B442 2020 Rain Optimization Algorithm, ROA
B187 2014 Symbiotic Organisms Search, SOS B443 2020 Student Psychology Based Optimization Algorithm, SPOA
B188 2014 Colliding Bodies Optimization, CBO B444 2020 Seasons Optimization Algorithm, SOA
B189 2014 Chicken Swarm Optimization, CSO B445 2020 Shell Game Optimization, SGO
B190 2014 Spider Monkey Optimization, SMO B446 2020 Sparrow Search Algorithm, SSA
B191 2014 Interior Search Algorithm, ISA B447 2020 Tiki-Taka Algorithm, TTA
B192 2014 Animal Migration Optimization Algorithm, AMOA B448 2020 Transient Search Optimization, TSO
B193 2014 Coral Reefs Optimization Algorithm, CROA B449 2020 Vapor-Liquid Equilibrium Algorithm, VLEA
B194 2014 Bird Mating Optimizer, BMO B450 2020 Virus Spread Optimization, VSO
B195 2014 Shark Smell Optimization, SSO B451 2020 Wingsuit Flying Search, WFS
B196 2014 Exchange Market Algorithm, EMA B452 2020 Water Strider Algorithm, WSA
B197 2014 Forest Optimization Algorithm, FOA B453 2020 Woodpecker Mating Algorithm, WMA
B198 2014 Golden Ball Algorithm, GBA B454 2020 Solar System Algorithm, SSA
B199 2014 Keshtel Algorithm, KA B455 2020 Arsh-Fati-Based Cluster Head Selection Algorithm, ARSH-FATI-CHS
B200 2014 Kaizen Programming, KP B456 2020 Teng-Yue Algorithm, TYA
B201 2014 Kinetic Gas Molecule Optimization, KGMO B457 2020 Projectiles Optimization, PO
B202 2014 Strawberry Algorithm, SA B458 2020 Color Harmony Algorithm, CHA
B203 2014 Heart Algorithm, HA B459 2020 Multi-Objective Beetle Antennae Search, MOBAS
B204 2014 Artificial Ecosystem Algorithm, AEA B460 2020 Orca Optimization Algorithm, OOA
B205 2014 The Scientific Algorithms, SA B461 2020 Photon Search Algorithm, PSA
B206 2014 Worm Optimization, WO B462 2020 Kernel Search Optimization, KSO
B207 2014 Greedy Politics Optimization, GPO B463 2020 Spherical Search Algorithm, SSA
B208 2014 Human Learning Optimization, HLO B464 2020 Triple Distinct Search Dynamics, TDSD
B209 2014 Soccer League Competition Algorithm, SLCA B465 2021 Chaos Game Optimization, CGO
B210 2014 Hyper-Spherical Search Algorithm, HSSA B466 2021 Chameleon Swarm Algorithm, CSA
B211 2014 Ecogeography-Based Optimization, EBO B467 2021 Atomic Orbital Search, AOS
B212 2014 Pigeon-Inspired Optimization, PIO B468 2021 Artificial Jellyfish Search Optimizer, JS
B213 2015 Ant Lion Optimization, ALO B469 2021 Cooperation Search Algorithm, CSA
B214 2015 Artificial Algae Algorithm, AAA B470 2021 Material Generation Algorithm, MGA
B215 2015 Artificial Showering Algorithm, ASA B471 2021 Crystal Structure Algorithm, CryStA1
B216 2015 Cricket Algorithm, CA B472 2021 Archimedes Optimization Algorithm, AOA
B217 2015 Gradient Evolution Algorithm, GEA B473 2021 Archerfish Hunting Optimizer, AHO
B218 2015 Moth-Flame Optimization Algorithm, MFOA B474 2021 Battle Royale Optimization Algorithm, BRO
B219 2015 Monarch Butterfly Optimization, MBO B475 2021 Artificial Lizard Search Optimization, ALSO
B220 2015 Water Wave Optimization, WWO B476 2021 Quantum Firefly Algorithm, QFA
B221 2015 Stochastic Fractal Search, SFS B477 2021 Flow Direction Algorithm, FDA
B222 2015 Elephant Herding Optimization, EHO B478 2021 Lichtenberg Algorithm, LA
B223 2015 Vortex Search Algorithm, VSA B479 2021 Pastoralist Optimization Algorithm, POA
B224 2015 Earthworm Optimization Algorithm, EOA B480 2021 Ebola Optimization Search Algorithm, EOSA
B225 2015 Lightning Search Algorithm, LSA B481 2021 Elephant Clan Optimization, ECO
B226 2015 Heat Transfer Search Algorithm, HTSA B482 2021 Red Colobuses Monkey, RCM
(continued on next page)

20
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

Table 7 (continued )
Refs Year Full name & abbreviation Refs Year Full name & abbreviation

B227 2015 Ions Motion Algorithm, IMA B483 2021 Golden Eagle Optimizer, GEO
B228 2015 Optics Inspired Optimization, OIO B484 2021 Group Mean-Based Optimizer, GMBO
B229 2015 Tree Seed Algorithm, TSA B485 2021 Dingo Optimizer, DO
B230 2015 Runner-Root Algorithm, RRA B486 2021 Coronavirus Herd Immunity Optimizer, CHIO
B231 2015 Elephant Search Algorithm, ESA B487 2021 Red Fox Optimization Algorithm, RFO
B232 2015 Election Algorithm, EA B488 2021 Arithmetic Optimization Algorithm, AOA
B233 2015 Locust Search, LS B489 2021 African Vultures Optimization Algorithm, AVOA
B234 2015 Invasive Tumor Growth Optimization Algorithm, ITWO B490 2021 Artificial Gorilla Troops Optimizer, GTO
B235 2015 Jaguar Algorithm, JA B491 2021 Artificial Hummingbird Algorithm, AHA
B236 2015 General Relativity Search Algorithm, GRSA B492 2021 Intelligent Ice Fishing Algorithm, IIFA
B237 2015 Root Growth Optimizer, RGO B493 2021 Komodo Mlipir Algorithm, KMA
B238 2015 Bull Optimization Algorithm, BOA B494 2021 Linear Prediction Evolution Algorithm, LPE
B239 2015 Prey-Predator Algorithm, PPA B495 2021 Multi-Objective Trader Algorithm, MOTR
B240 2015 African Buffalo Optimization, ABO B496 2021 Optimal Stochastic Process Optimizer, OSPO
B241 2016 Artificial Infectious Disease Optimization, AID B497 2021 Remora Optimization Algorithm, ROA
B242 2016 Across Neighborhood Search, ANS B498 2021 Ring Toss Game-Based Optimization Algorithm, RTGBO
B243 2016 Cricket Behavior-Based Algorithm, CBBA B499 2021 RUNge Kutta Optimizer, RUN
B244 2016 Competitive Optimization Algorithm, COOA B500 2021 Samw
B245 2016 Cognitive Behavior Optimization Algorithm, COA B501 2021 String Theory Algorithm, STA
B246 2016 Electromagnetic Field Optimization, EFO B502 2021 Success History Intelligent Optimizer, SHIO
B247 2016 Football Game Algorithm, FGA B503 2021 Tangent Search Algorithm, TSA
B248 2016 Intrusive Tumor Growth Inspired Optimization Algorithm, ITGO B504 2021 Tuna Swarm Optimization, TSO
B249 2016 Galactic Swarm Optimization, GSO B505 2021 Volcano Eruption Algorithm, VCA
B250 2016 Whale Optimization Algorithm, WOA B506 2021 Smart Flower Optimization Algorithm, SFOA
B251 2016 Sine Cosine Algorithm, SSA B507 2022 Ali baba and the Forty Thieves Optimization, AFT
B252 2016 Dragonfly Algorithm, DA B508 2022 Honey Badger Algorithm, HBA
B253 2016 Crow Search Algorithm, CSA B509 2022 Orca Predation Algorithm, OPA
B254 2016 Multi-Verse Optimizer, MVO B510 2022 Reptile Search Algorithm, RSA
B255 2016 Bird Swarm Algorithm, BSA B511 2022 Skip Salp Swam Algorithm, SSSA
B256 2016 Virus Colony Search, VCS

For fair comparisons, we first use a unified framework named irace to

Algorithm 1
automatically configure the parameters of all 15 comparative algo­
The common optimization framework of a single-solution based metaheuristics.
rithms. Then, whether these algorithms have a search bias to the origin
Input: initial solution s0 ; parameters is investigated. For a detailed description, the convergence, diversity
Output: The best solution
and trade-off between the exploration and exploitation of all 15 algo­
t←0;
Repeat rithms are also analyzed. All the experimental results were analyzed by
/* Generate candidate solutions (partial or complete neighborhood) from st */ nonparametric statistical methods, including the Friedman test, Wil­
Generate(C(st )); coxon signed-ranks test, and Bayesian signed-rank test. The results show
/* Select a solution from C(s) to replace the current solution st */
that the performance of EBCM is similar to the 4 compared algorithms,
st+1 = Select(C(st ));
t = t + 1;
and has the same properties and behaviors, such as convergence, di­
Until the termination condition is met. versity, exploration and exploitation trade-off, etc. But the other 10
recent algorithms are inferior to the 4 state-of-the-art algorithms for
solving the CEC 2017 benchmark suite with 10, 30, and 50 variables.
Algorithm 2
Besides, all 15 algorithms have searches biased to the origin but with
The common optimization framework of a population-based metaheuristics. different strengths. However, the 4 state-of-the-art algorithms are less
affected by the shift operator of the functions compared with the 11
Input: initial solution P0 ; parameters
Output: The best solution
recent algorithms. Except for EBCM, it should be noted that the other 10
t←0; new algorithms are inferior to the 4 state-of-the-art algorithms in terms
Evaluate the initial solutions and remember the best one as P∗ ; of convergence speed and global search ability on CEC 2017 functions.
Repeat Moreover, the other 10 new algorithms show a rougher trade-off and
Generate (Pt ); /*Generation a new population */ diversity compared to the 4 state-of-the-art algorithms. Finally, several
′

Pt+1 = Select Population(Pt ∪ Pt ); /*Select new population */

′

issues and suggestions based on the abovementioned review and ex­

Record the best solution found so far P∗ ;
t←t + 1; periments are proposed.
Until the termination condition is met In the next part of this survey series, we extend our work from the
return the best solution found P∗ . following aspects:

(1) Metaheuristics are a broad field of research. We need to focus on

collected, and a taxonomy of metaheuristics is further proposed to
comparative studies including examining more newly proposed
describe the metaheuristics from two aspects, including the inspiration
algorithms and state-of-the-art algorithms on benchmarks and
sources and the essential operators for generating solutions. We find that
real-world problems of different sizes, complexes, and categories.
the recent metaheuristics proposed in the literature are neither rigor­
(2) Due to space limitations, we investigated the performance and
ously tested on comprehensive and representative benchmark suites, nor
properties of different metaheuristics in the current study. We
compared with the state-of-the-art metaheuristics. Therefore, to eval­
need a thorough theoretical analysis to confirm why these met­
uate and understand the performance of the state-of-the-art and recent
aheuristics perform better or worse.
metaheuristics, 11 representative metaheuristics with new names are
(3) We investigated some metaheuristics on whether their search is
selected as recent algorithms to be compared with the 4 state-of-the-art
biased toward the origin. However, which parameters or strate­
algorithms on the CEC 2017 benchmark suite.
gies influence this property requires further study.

21
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

Fig. 7. The CD plot of algorithms on the nonshifted functions with 10 variables.

Fig. 8. The CD plot of algorithms on the nonshifted functions with 30 variables.

Overall, we hope that our study provides useful insight to guide Declaration of Competing Interest
future designs of more practicable metaheuristics that are capable of
handling complex, high-dimensional and large-scale real-world The authors declare that they have no known competing financial
problems. interests or personal relationships that could have appeared to influence
the work reported in this paper.
Credit for Authors’ Contributions
Data availability
Zhongqiang Ma: Programmed the methods, conducted experiments,
prepared the draft manuscript. Guohua Wu: Supervised and edited the Data will be made available on request.
manuscript. P. N. Suganthan: Proposed the overall project, supervised
and edited the manuscript. Aijuan Song: Assisted in the editing. Qiz­
hang Luo: Assisted in the editing. Acknowledgement

Open Access funding provided by the Qatar National Library.

22
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

Supplementary materials [32] D.B. Fogel, Evolutionary Computation: Toward A New Philosophy Of Machine
Intelligence, John Wiley & Sons, 2006.
[33] L.J. Fogel, Artificial intelligence through a simulation of evolution, in: Proc. 2nd
Supplementary material associated with this article can be found, in Cybern. Sci. Symp, 1965, p. 1965.
the online version, at doi:10.1016/j.swevo.2023.101248. [34] X. Yao, Y. Liu, G. Lin, Evolutionary programming made faster, IEEE Trans. Evol.
Comput. 3 (1999) 82–102.
[35] R. Mallipeddi, P.N. Suganthan, Evaluation of novel adaptive evolutionary
References programming on four constraint handling techniques, in: Proceedings of the 2008
IEEE Congr. Evol. Comput. (IEEE World Congr. Comput. Intell., IEEE, 2008,
[1] G. Wu, R. Mallipeddi, P.N. Suganthan, Ensemble strategies for population-based pp. 4045–4052.
optimization algorithms – a survey, Swarm Evol. Comput. 44 (2019) 695–711, [36] H. Zhang, J. Lu, Adaptive evolutionary programming based on reinforcement
https://doi.org/10.1016/j.swevo.2018.08.015. learning, Inf. Sci. (Ny). 178 (2008) 971–984.
[2] R. Moghdani, K. Salimifard, Volleyball premier league algorithm, Appl. Soft [37] R. Mallipeddi, S. Mallipeddi, P.N. Suganthan, Ensemble strategies with adaptive
Comput. 64 (2018) 161–185. evolutionary programming, Inf. Sci. (Ny). 180 (2010) 1571–1581.
[3] R. Rajabioun, Cuckoo optimization algorithm, Appl. Soft Comput. 11 (2011) [38] Q. Liu, X. Li, L. Gao, Y. Li, A modified genetic algorithm with new encoding and
5508–5518, https://doi.org/10.1016/j.asoc.2011.05.008. decoding methods for integrated process planning and scheduling problem, IEEE
[4] F. Glover, Future paths for integer programming and links to artificial Trans. Cybern. (2020).
intelligence, Comput. Oper. Res. 13 (1986) 533–549. [39] G. Zhang, Y. Hu, J. Sun, W. Zhang, An improved genetic algorithm for the flexible
[5] Q. Askari, I. Younas, M. Saeed, Political Optimizer: A novel socio-inspired meta- job shop scheduling problem with multiple time constraints, Swarm Evol.
heuristic for global optimization, Knowledge-Based Syst. (2020) 195, https://doi. Comput. 54 (2020), 100664.
org/10.1016/j.knosys.2020.105709. [40] S. Sayed, M. Nassef, A. Badr, I. Farag, A nested genetic algorithm for feature
[6] S. Kirkpatrick, C.D. Gelatt, M.P. Vecchi, Optimization by simulated annealing, selection in high-dimensional cancer microarray datasets, Expert Syst. Appl. 121
Science 220 (80-) (1983) 671–680. (2019) 233–243.
[7] J. Kennedy, R. Eberhart, Particle swarm optimization, in: Proc. ICNN’95- [41] F. Curtis, X. Li, T. Rose, A. Vazquez-Mayagoitia, S. Bhattacharya, L.
International Conf. Neural Networks, IEEE, 1995, pp. 1942–1948. M. Ghiringhelli, N. Marom, GAtor: a first-principles genetic algorithm for
[8] M. Dorigo, M. Birattari, T. Stutzle, Ant colony optimization, IEEE Comput. Intell. molecular crystal structure prediction, J. Chem. Theory Comput. 14 (2018)
Mag. 1 (2006) 28–39. 2246–2264.
[9] S.H. Aghay Kaboli, J. Selvaraj, N.A. Rahim, Rain-fall optimization algorithm: a [42] W. Chen, M. Panahi, H.R. Pourghasemi, Performance evaluation of GIS-based
population based algorithm for solving constrained optimization problems, new ensemble data mining techniques of adaptive neuro-fuzzy inference system
J. Comput. Sci. 19 (2017) 31–42, https://doi.org/10.1016/j.jocs.2016.12.010. (ANFIS) with genetic algorithm (GA), differential evolution (DE), and particle
[10] H.R. Lourenço, O.C. Martin, T. Stützle, Iterated local search: Framework and swarm optimization (PSO) for landslide spatial modelling, Catena 157 (2017)
applications. Handb. Metaheuristics, Springer, 2019, pp. 129–168. 310–324.
[11] C. Voudouris, E.P.K. Tsang, A. Alsheddy, Guided local search. Handb. [43] E.C. Pedrino, T. Yamada, T.R. Lunardi, J.C. de Melo Vieira Jr, Islanding detection
Metaheuristics, Springer, 2010, pp. 321–361. of distributed generation by using multi-gene genetic programming based
[12] L.A. Rastrigin, The convergence of the random search method in the extremal classifier, Appl. Soft Comput. 74 (2019) 206–215.
control of a many parameter system, Autom. Remote Control. 24 (1963) [44] C. Ryan, J.J. Collins, M.O. Neill, Grammatical evolution: Evolving programs for
1337–1342. an arbitrary language, in: Proceedings of the Eur. Conf. Genet. Program.,
[13] N. Mladenović, P. Hansen, Variable neighborhood search, Comput. Oper. Res. 24 Springer, 1998, pp. 83–96.
(1997) 1097–1100. [45] S. Das, P.N. Suganthan, Differential evolution: a survey of the state-of-the-art,
[14] D. Pisinger, S. Ropke, Large neighborhood search. Handb. Metaheuristics, IEEE Trans. Evol. Comput. 15 (2010) 4–31.
Springer, 2010, pp. 399–419. [46] G. Wu, R. Mallipeddi, P.N. Suganthan, R. Wang, H. Chen, Differential evolution
[15] J.H. Holland, Adaptation in natural and artificial systems, univ. of mich. press, with multi-population based ensemble of mutation strategies, Inf. Sci. (Ny). 329
Ann Arbor. (1975). (2016) 329–345.
[16] R. Storn, K. Price, Differential evolution–a simple and efficient heuristic for global [47] G. Wu, X. Shen, H. Li, H. Chen, A. Lin, P.N. Suganthan, Ensemble of differential
optimization over continuous spaces, J. Glob. Optim. 11 (1997) 341–359. evolution variants, Inf. Sci. (Ny). 423 (2018) 172–186, https://doi.org/10.1016/
[17] R. Hooke, T.A. Jeeves, ``Direct Search’’ solution of numerical and statistical j.ins.2017.09.053.
problems, J. ACM. 8 (1961) 212–229. [48] A.K. Qin, V.L. Huang, P.N. Suganthan, Differential evolution algorithm with
[18] A. Tzanetos, G. Dounias, Nature inspired optimization algorithms or simply strategy adaptation for global numerical optimization, IEEE Trans. Evol. Comput.
variations of metaheuristics? Artif. Intell. Rev. 54 (2021) 1841–1862. 13 (2008) 398–417.
[19] D. Molina, J. Poyatos, J. Del Ser, S. García, A. Hussain, F. Herrera, Comprehensive [49] J. Brest, S. Greiner, B. Boskovic, M. Mernik, V. Zumer, Self-adapting control
taxonomies of nature- and bio-inspired optimization: inspiration versus parameters in differential evolution: a comparative study on numerical
algorithmic behavior, critical analysis recommendations, Cognit. Comput. 12 benchmark problems, IEEE Trans. Evol. Comput. 10 (2006) 646–657.
(2020) 897–939, https://doi.org/10.1007/s12559-020-09730-8. [50] Y. Wang, Z. Cai, Q. Zhang, Differential evolution with composite trial vector
[20] A.H. Halim, I. Ismail, S. Das, Performance assessment of the metaheuristic generation strategies and control parameters, IEEE Trans. Evol. Comput. 15
optimization algorithms: an exhaustive review, Artif. Intell. Rev. (2020), https:// (2011) 55–66.
doi.org/10.1007/s10462-020-09906-6. [51] F.A. Hashim, E.H. Houssein, M.S. Mabrouk, W. Al-Atabany, S. Mirjalili, Henry gas
[21] X. Chu, T. Wu, J.D. Weir, Y. Shi, B. Niu, L. Li, solubility optimization: a novel physics-based algorithm, Futur. Gener. Comput.
Learning–interaction–diversification framework for swarm intelligence Syst. 101 (2019) 646–667, https://doi.org/10.1016/j.future.2019.07.015.
optimizers: a unified perspective, Neural Comput. Appl. 32 (2018) 1789–1809, [52] A.W. Mohamed, A.A. Hadi, A.K. Mohamed, Gaining-sharing knowledge based
https://doi.org/10.1007/s00521-018-3657-0. algorithm for solving optimization problems: a novel nature-inspired algorithm,
[22] Z. Chen, Y. Liu, Z. Yang, X. Fu, J. Tan, X. Yang, An enhanced teaching-learning- Int. J. Mach. Learn. Cybern. 11 (2020) 1501–1529, https://doi.org/10.1007/
based optimization algorithm with self-adaptive and learning operators and its s13042-019-01053-x.
search bias towards origin, Swarm Evol. Comput. (2021) 60, https://doi.org/ [53] G. Dhiman, M. Garg, A. Nagar, V. Kumar, M. Dehghani, A novel algorithm for
10.1016/j.swevo.2020.100766. global optimization: rat swarm optimizer, J. Ambient Intell. Humaniz. Comput.
[23] J.K. Pickard, J.A. Carretero, V.C. Bhavsar, On the convergence and origin bias of (2020), https://doi.org/10.1007/s12652-020-02580-0.
the teaching-learning-based-optimization algorithm, Appl. Soft Comput. 46 [54] W. Al-Sorori, A.M. Mohsen, New Caledonian crow learning algorithm: a new
(2016) 115–127, https://doi.org/10.1016/j.asoc.2016.04.029. metaheuristic algorithm for solving continuous optimization problems, Appl. Soft
[24] P. Niu, S. Niu, N. liu, L. Chang, The defect of the Grey Wolf optimization Comput. 92 (2020), https://doi.org/10.1016/j.asoc.2020.106325.
algorithm and its verification method, Knowledge-Based Syst 171 (2019) 37–43, [55] M.H. Sulaiman, Z. Mustaffa, M.M. Saari, H. Daniyal, Barnacles mating optimizer:
https://doi.org/10.1016/j.knosys.2019.01.018. a new bio-inspired algorithm for solving engineering optimization problems, Eng.
[25] A.E. Ezugwu, A.K. Shukla, R. Nath, A.A. Akinyelu, J.O. Agushaka, H. Chiroma, P. Appl. Artif. Intell. 87 (2020), https://doi.org/10.1016/j.engappai.2019.103330.
K. Muhuri, Metaheuristics: a comprehensive overview and classification along [56] R.V Rao, V.J. Savsani, D.P. Vakharia, Teaching–learning-based optimization: a
with bibliometric analysis, Artif. Intell. Rev. (2021), https://doi.org/10.1007/ novel method for constrained mechanical design optimization problems, Comput.
s10462-020-09952-0. Des. 43 (2011) 303–315, https://doi.org/10.1016/j.cad.2010.12.015.
[26] C. Pizzuti, Evolutionary computation for community detection in networks: a [57] Y. Zhang, Z. Jin, Group teaching optimization algorithm: a novel metaheuristic
review, IEEE Trans. Evol. Comput. 22 (2017) 464–483. method for solving global optimization problems, Expert Syst. Appl. (2020) 148,
[27] N.A. Barricelli, Symbiogenetic evolution processes realized by artificial methods, https://doi.org/10.1016/j.eswa.2020.113246.
1957. [58] N. Moosavian, B.Kasaee Roodsari, Soccer league competition algorithm: a novel
[28] A.S. Fraser, Simulation of genetic systems by automatic digital computers I. meta-heuristic algorithm for optimal design of water distribution networks,
Introduction, Aust. J. Biol. Sci. 10 (1957) 484–491. Swarm Evol. Comput. 17 (2014) 14–24, https://doi.org/10.1016/j.
[29] I. Rechenberg, Evolution strategy: nature’s way of optimization. Optim. Methods swevo.2014.02.002.
Appl. Possibilities Limitations, Springer, 1989, pp. 106–126. [59] E. Atashpaz-Gargari, C. Lucas, Imperialist competitive algorithm: an algorithm for
[30] L.J. Fogel, A.J. Owens, M.J. Walsh, Intelligent decision making through a optimization inspired by imperialistic competition, in: Proceedings of the 2007
simulation of evolution, Behav. Sci. 11 (1966) 253–272. IEEE Congr. Evol. Comput., 2007, pp. 4661–4667, https://doi.org/10.1109/
[31] D. Simon, Evolutionary Optimization Algorithms, John Wiley & Sons, 2013. CEC.2007.4425083.

23
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

[60] J.S.M.L. Melvix, Greedy politics optimization: Metaheuristic inspired by political [91] A. Faramarzi, M. Heidarinejad, B. Stephens, S. Mirjalili, Equilibrium optimizer: a
strategies adopted during state assembly elections, in: Proceedings of the 2014 novel optimization algorithm, Knowledge-Based Syst. 191 (2020), 105190,
IEEE Int. Adv. Comput. Conf., IEEE, 2014, pp. 1157–1162. https://doi.org/10.1016/j.knosys.2019.105190.
[61] A. Borji, A new global optimization algorithm inspired by parliamentary political [92] G. Dhiman, V. Kumar, Seagull optimization algorithm: theory and its applications
competitions, in: Proceedings of the Mex. Int. Conf. Artif. Intell, Springer, 2007, for large-scale industrial engineering problems, Knowledge-Based Syst. 165
pp. 61–71. (2019) 169–196, https://doi.org/10.1016/j.knosys.2018.11.024.
[62] S.H. Samareh Moosavi, V.K. Bardsiri, Poor and rich optimization algorithm: a new [93] B. Morales-Castañeda, D. Zaldívar, E. Cuevas, F. Fausto, A. Rodríguez, A better
human-based and multi populations algorithm, Eng. Appl. Artif. Intell. 86 (2019) balance in metaheuristic algorithms: does it exist? Swarm Evol. Comput. (2020)
165–181, https://doi.org/10.1016/j.engappai.2019.08.025. 54, https://doi.org/10.1016/j.swevo.2020.100671.
[63] H. Ghasemian, F. Ghasemian, H. Vahdat-Nejad, Human urbanization algorithm: a [94] M. Črepinšek, S.-H. Liu, M. Mernik, Exploration and exploitation in evolutionary
novel metaheuristic approach, Math. Comput. Simul. 178 (2020) 1–15, https:// algorithms: a survey, ACM Comput. Surv. 45 (2013) 1–33.
doi.org/10.1016/j.matcom.2020.05.023. [95] Z.-H. Zhan, Z.-J. Wang, H. Jin, J. Zhang, Adaptive distributed differential
[64] A. Khatri, A. Gaba, K.P.S. Rana, V. Kumar, A novel life choice-based optimizer, evolution, IEEE Trans. Cybern. 50 (2019) 4633–4647.
Soft Comput. 24 (2019) 9121–9141, https://doi.org/10.1007/s00500-019- [96] K.R. Opara, J. Arabas, Differential evolution: a survey of theoretical analyses,
04443-z. Swarm Evol. Comput. 44 (2019) 546–558.
[65] J. Zhang, M. Xiao, L. Gao, Q. Pan, Queuing search algorithm: a novel [97] A.W. Mohamed, A.A. Hadi, K.M. Jambi, Novel mutation strategy for enhancing
metaheuristic algorithm for solving engineering optimization problems, Appl. SHADE and LSHADE algorithms for global numerical optimization, Swarm Evol.
Math. Model. 63 (2018) 464–490, https://doi.org/10.1016/j.apm.2018.06.036. Comput. 50 (2019), 100455, https://doi.org/10.1016/j.swevo.2018.10.006.
[66] Q. Zhang, R. Wang, J. Yang, A. Lewis, F. Chiclana, S. Yang, Biology migration [98] A. Viktorin, R. Senkerik, M. Pluhacek, T. Kadavy, A. Zamuda, Distance based
algorithm: a new nature-inspired heuristic methodology for global optimization, parameter adaptation for success-history based differential evolution, Swarm
Soft Comput 23 (2019) 7333–7358. Evol. Comput. 50 (2019), 100462, https://doi.org/10.1016/j.
[67] D. Karaboga, B. Basturk, A powerful and efficient algorithm for numerical swevo.2018.10.013.
function optimization: artificial bee colony (ABC) algorithm, J. Glob. Optim. 39 [99] N.H. Awad, M.Z. Ali, P.N. Suganthan, Ensemble sinusoidal differential covariance
(2007) 459–471, https://doi.org/10.1007/s10898-007-9149-x. matrix adaptation with Euclidean neighborhood for solving CEC2017 benchmark
[68] X.-S. Yang, A new metaheuristic bat-inspired algorithm, in: Proceedings of the problems, in: Proceedings of the 2017 IEEE Congr. Evol. Comput., 2017,
Nat. Inspired Coop. Strateg. Optim. (NICSO 2010), Springer, 2010, pp. 65–74. pp. 372–379, https://doi.org/10.1109/CEC.2017.7969336.
[69] X.-S. Yang, S. Deb, Cuckoo search via Lévy flights, in: Proceedings of the 2009 [100] A.W. Mohamed, A.A. Hadi, A.M. Fattouh, K.M. Jambi, LSHADE with semi-
World Congr. Nat. Biol. Inspired Comput., Ieee, 2009, pp. 210–214. parameter adaptation hybrid with CMA-ES for solving CEC 2017 benchmark
[70] X.-S. Yang, Firefly algorithms for multimodal optimization, in: Proceedings of the problems, in: Proceedings of the 2017 IEEE Congr. Evol. Comput., IEEE, 2017,
Int. Symp. Stoch. Algorithms, Springer, 2009, pp. 169–178. pp. 145–152.
[71] S. Mirjalili, S.M. Mirjalili, A. Lewis, Grey wolf optimizer, Adv. Eng. Softw. 69 [101] A. Ghosh, S. Das, A.K. Das, R. Senkerik, A. Viktorin, I. Zelinka, A.D. Masegosa,
(2014) 46–61, https://doi.org/10.1016/j.advengsoft.2013.12.007. Using spatial neighborhoods for parameter adaptation: an improved success
[72] S.Z. Mirjalili, S. Mirjalili, S. Saremi, H. Faris, I. Aljarah, Grasshopper optimization history based differential evolution, Swarm Evol. Comput. 71 (2022), 101057.
algorithm for multi-objective optimization problems, Appl. Intell. 48 (2018) [102] G. Zhang, Y. Shi, Hybrid sampling evolution strategy for solving single objective
805–820. bound constrained problems, in: Proceedings of the 2018 IEEE Congr. Evol.
[73] S. Harifi, J. Mohammadzadeh, M. Khalilian, S. Ebrahimnejad, Giza pyramids Comput., 2018, pp. 1–7, https://doi.org/10.1109/CEC.2018.8477908.
construction: an ancient-inspired metaheuristic algorithm for optimization, Evol. [103] A. Kumar, R.K. Misra, D. Singh, Improving the local search capability of effective
Intell. (2020) 1–19. butterfly optimizer using covariance matrix adapted retreat phase, in:
[74] A.R. Mehrabian, C. Lucas, A novel numerical optimization algorithm inspired Proceedings of the 2017 IEEE Congr. Evol. Comput., 2017, pp. 1835–1842,
from weed colonization, Ecol. Inform. 1 (2006) 355–366. https://doi.org/10.1109/CEC.2017.7969524.
[75] X.-S. Yang, Flower pollination algorithm for global optimization, in: Proceedings [104] H. Rakhshani, A. Rahati, Snap-drift cuckoo search: a novel cuckoo search
of the Int. Conf. Unconv. Comput. Nat. Comput., Springer, 2012, pp. 240–249. optimization algorithm, Appl. Soft Comput. 52 (2017) 771–794, https://doi.org/
[76] S. Harifi, M. Khalilian, J. Mohammadzadeh, S. Ebrahimnejad, Emperor penguins 10.1016/j.asoc.2016.09.048.
colony: a new metaheuristic algorithm for optimization, Evol. Intell. 12 (2019) [105] X.-S. Yang, S. Deb, Cuckoo search: recent advances and applications, Neural
211–226. Comput. Appl. 24 (2014) 169–174, https://doi.org/10.1007/s00521-013-1367-1.
[77] V. Hayyolalam, A.A.P. Kazem, Black widow optimization algorithm: a novel [106] H. Chen, M. Wang, X. Zhao, A multi-strategy enhanced sine cosine algorithm for
meta-heuristic approach for solving engineering optimization problems, Eng. global optimization and constrained practical engineering problems, Appl. Math.
Appl. Artif. Intell. 87 (2020), 103249. Comput. 369 (2020), 124872, https://doi.org/10.1016/j.amc.2019.124872.
[78] E. Bogar, S. Beyhan, Adolescent Identity Search Algorithm (AISA): a novel [107] S. Mirjalili, SCA: a sine cosine algorithm for solving optimization problems,
metaheuristic approach for solving optimization problems, Appl. Soft Comput. 95 Knowledge-Based Syst 96 (2016) 120–133, https://doi.org/10.1016/j.
(2020), https://doi.org/10.1016/j.asoc.2020.106503. knosys.2015.12.022.
[79] M. Khishe, M.R. Mosavi, Chimp optimization algorithm, Expert Syst. Appl. 149 [108] D. Pelusi, R. Mascella, L. Tallini, J. Nayak, B. Naik, Y. Deng, An improved moth-
(2020), 113338. flame optimization algorithm with hybrid search phase, Knowledge-Based Syst
[80] E. Rashedi, H. Nezamabadi-pour, S. Saryazdi, GSA: a gravitational search 191 (2020), 105277, https://doi.org/10.1016/j.knosys.2019.105277.
algorithm, Inf. Sci. (Ny). 179 (2009) 2232–2248, https://doi.org/10.1016/j. [109] S. Mirjalili, Moth-flame optimization algorithm: A novel nature-inspired heuristic
ins.2009.03.004. paradigm, Knowledge-Based Syst. 89 (2015) 228–249, https://doi.org/10.1016/j.
[81] S. Mirjalili, S.M. Mirjalili, A. Hatamlou, Multi-Verse optimizer: a nature-inspired knosys.2015.07.006.
algorithm for global optimization, Neural Comput. Appl. 27 (2015) 495–513, [110] L. Abualigah, D. Yousri, M. Abd Elaziz, A.A. Ewees, M.A.A. Al-Qaness, A.
https://doi.org/10.1007/s00521-015-1870-7. H. Gandomi, Aquila optimizer: a novel meta-heuristic optimization algorithm,
[82] Z.W. Geem, J.H. Kim, G.V. Loganathan, A new heuristic optimization algorithm: Comput. Ind. Eng. 157 (2021), 107250.
harmony search, Simulation 76 (2001) 60–68. [111] J. Luo, H. Chen, Q. zhang, Y. Xu, H. Huang, X. Zhao, An improved grasshopper
[83] T. Zhang, Z.W. Geem, Review of harmony search with respect to algorithm optimization algorithm with application to financial stress prediction, Appl.
structure, Swarm Evol. Comput. 48 (2019) 31–43, https://doi.org/10.1016/j. Math. Model. 64 (2018) 654–668, https://doi.org/10.1016/j.apm.2018.07.044.
swevo.2019.03.012. [112] D. Pelusi, R. Mascella, L. Tallini, J. Nayak, B. Naik, Y. Deng, Improving
[84] A. Kaveh, T. Bakhshpoori, Water Evaporation Optimization: a novel physically exploration and exploitation via a hyperbolic gravitational search algorithm,
inspired optimization algorithm, Comput. Struct. 167 (2016) 69–85, https://doi. Knowledge-Based Syst 193 (2020), 105404, https://doi.org/10.1016/j.
org/10.1016/j.compstruc.2016.01.008. knosys.2019.105404.
[85] M.H. Qais, H.M. Hasanien, S. Alghuwainem, Transient search optimization: a new [113] M.M. Eusuff, K.E. Lansey, Optimization of water distribution network design
meta-heuristic optimization algorithm, Appl. Intell. 50 (2020) 3926–3941, using the shuffled frog leaping algorithm, J. Water Resour. Plan. Manag. 129
https://doi.org/10.1007/s10489-020-01727-y. (2003) 210–225.
[86] A.Y.S. Lam, V.O.K. Li, Chemical-reaction-inspired metaheuristic for optimization, [114] D. Tang, Z. Liu, J. Yang, J. Zhao, Memetic frog leaping algorithm for global
IEEE Trans. Evol. Comput. 14 (2009) 381–399. optimization, Soft Comput. 23 (2019) 11077–11105, https://doi.org/10.1007/
[87] A. Kaveh, S. Talatahari, A novel heuristic optimization method: charged system s00500-018-3662-3.
search, Acta Mech. 213 (2010) 267–289, https://doi.org/10.1007/s00707-009- [115] A. Faramarzi, M. Heidarinejad, S. Mirjalili, A.H. Gandomi, Marine predators
0270-4. algorithm: a nature-inspired metaheuristic, Expert Syst. Appl. 152 (2020),
[88] E.-G. Talbi, Metaheuristics: From Design To Implementation, John Wiley & Sons, 113377, https://doi.org/10.1016/j.eswa.2020.113377.
2009. [116] S. Debnath, W. Arif, S. Baishya, Buyer inspired meta-heuristic optimization
[89] B. Doğan, T. Ölmez, A new metaheuristic for numerical function optimization: algorithm, Open Comput. Sci. 10 (2020) 194–219, https://doi.org/10.1515/
vortex search algorithm, Inf. Sci. (Ny). 293 (2015) 125–145, https://doi.org/ comp-2020-0101.
10.1016/j.ins.2014.08.053. [117] A. LaTorre, D. Molina, E. Osaba, J. Poyatos, J. Del Ser, F. Herrera, A prescription
[90] F.A. Hashim, K. Hussain, E.H. Houssein, M.S. Mabrouk, W. Al-Atabany, of methodological guidelines for comparing bio-inspired optimization algorithms,
Archimedes optimization algorithm: a new metaheuristic algorithm for solving Swarm Evol. Comput. 67 (2021), 100973.
optimization problems, Appl. Intell. (2020), https://doi.org/10.1007/s10489- [118] E. Osaba, E. Villar-Rodriguez, J. Del Ser, A.J. Nebro, D. Molina, A. LaTorre, P.
020-01893-z. N. Suganthan, C.A.C. Coello, F. Herrera, A tutorial on the design, experimentation
and application of metaheuristic algorithms to real-world optimization problems,
Swarm Evol. Comput. 64 (2021), 100888.

24
Z. Ma et al. Swarm and Evolutionary Computation 77 (2023) 101248

[119] M. López-Ibáñez, J. Dubois-Lacoste, L. Pérez Cáceres, M. Birattari, T. Stützle, The [128] M. López-Ibánez, T. Stützle, Automatically improving the anytime behaviour of
irace package: Iterated racing for automatic algorithm configuration, Oper. Res. optimisation algorithms, Eur. J. Oper. Res. 235 (2014) 569–582.
Perspect. 3 (2016) 43–58, https://doi.org/10.1016/j.orp.2016.09.002. [129] A.P. Piotrowski, J.J. Napiorkowski, Some metaheuristics should be simplified, Inf.
[120] L.C.T. Bezerra, M. López-Ibáñez, T. Stützle, Automatically designing state-of-the- Sci. (Ny). 427 (2018) 32–62.
art multi- and many-objective evolutionary algorithms, Evol. Comput. 28 (2019) [130] E.-G. Talbi, Combining metaheuristics with mathematical programming,
195–226, https://doi.org/10.1162/evco_a_00263. constraint programming and machine learning, Ann. Oper. Res. 240 (2016)
[121] A.J. Nebro, M. López-Ibáñez, C. Barba-González, J. García-Nieto, Automatic 171–215.
configuration of NSGA-II with jMetal and irace, Proc. Genet. Evol. Comput. Conf. [131] M. Karimi-Mamaghan, M. Mohammadi, P. Meyer, A.M. Karimi-Mamaghan, E.-
Companion. (2019) 1374–1381, https://doi.org/10.1145/3319619.3326832. G. Talbi, Machine learning at the service of meta-heuristics for solving
[122] C. Huang, Y. Li, X. Yao, A survey of automatic parameter tuning methods for combinatorial optimization problems: A state-of-the-art, Eur. J. Oper. Res. 296
metaheuristics, IEEE Trans. Evol. Comput. 24 (2020) 201–216, https://doi.org/ (2022) 393–422.
10.1109/TEVC.2019.2921598. [132] E.-G. Talbi, Machine learning into metaheuristics: a survey and taxonomy, ACM
[123] D. Jiang, X. Li, Order fulfilment problem with time windows and synchronisation Comput. Surv. 54 (2021) 1–32.
arising in the online retailing, Int. J. Prod. Res. 59 (2021) 1187–1215. [133] J. Ma, D. Xia, Y. Wang, X. Niu, S. Jiang, Z. Liu, H. Guo, A comprehensive
[124] J.J. Liang, S. Baskar, P.N. Suganthan, A.K. Qin, Performance evaluation of comparison among metaheuristics (MHs) for geohazard modeling using machine
multiagent genetic algorithm, Nat. Comput. 5 (2006) 83–96, https://doi.org/ learning: insights from a case study of landslide displacement prediction, Eng.
10.1007/s11047-005-1625-y. Appl. Artif. Intell. 114 (2022), 105150.
[125] J. Carrasco, S. García, M.M. Rueda, S. Das, F. Herrera, Recent trends in the use of [134] S. Nematzadeh, F. Kiani, M. Torkamanian-Afshar, N. Aydin, Tuning
statistical tests for comparing swarm and evolutionary computing algorithms: hyperparameters of machine learning algorithms and deep neural networks using
practical guidelines and a critical review, Swarm Evol. Comput. 54 (2020), metaheuristics: a bioinformatics study on biomedical and biological cases,
https://doi.org/10.1016/j.swevo.2020.100665. Comput. Biol. Chem. 97 (2022), 107619.
[126] J. Derrac, S. García, D. Molina, F. Herrera, A practical tutorial on the use of [135] J.-S. Chou, T.-K. Nguyen, Forward forecast of stock price using sliding-window
nonparametric statistical tests as a methodology for comparing evolutionary and metaheuristic-optimized machine-learning regression, IEEE Trans. Ind.
swarm intelligence algorithms, Swarm Evol. Comput. 1 (2011) 3–18. Informatics. 14 (2018) 3132–3142.
[127] L.C.T. Bezerra, M. López-Ibánez, T. Stützle, Automatic component-wise design of [136] A. Song, G. Wu, W. Pedrycz, L. Wang, Integrating variable reduction strategy with
multiobjective evolutionary algorithms, IEEE Trans. Evol. Comput. 20 (2015) evolutionary algorithms for solving nonlinear equations systems, IEEE/CAA J.
403–417. Autom. Sin. 9 (2021) 75–89.

25