280 Information Technology and Control 2024/1/53

ITC 1/53 A Multi-level Surrogate-assisted Algorithm for ExpensiveOptimization Problems

Information Technology Received 2024/01/02 Accepted after revision 2024/02/06

and Control
Vol. 53 / No. 1 / 2024 HOW TO CITE: Hu, L., Wu, X., Che, X. (2024). A Multi-level Surrogate-assisted Algorithm for
pp.280-301 Expensive Optimization Problems. Information Technology and Control, 53(1), 280-301. https://doi.
DOI 10.5755/j01.itc.53.1.35922 org/10.5755/j01.itc.53.1.35922

A Multi-level Surrogate-assisted
Algorithm for Expensive
Optimization Problems
Liang Hu
Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education,
Jilin University, Changchun, China:
College of Computer Science and Technology, Jilin University, Changchun, China; e-mail: [email protected]

Xianwei Wu
College of Computer Science and Technology, Jilin University, Changchun, China; e-mail: [email protected]

Xilong Che
Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education,
Jilin University, Changchun, China;
College of Computer Science and Technology, Jilin University, Changchun, China; e-mail: [email protected]

Corresponding author: [email protected]

With the development of computer science, more and more complex problems rely on the help of computers for
solving. When facing the parameter optimization problem of complex models, traditional intelligent optimi-
zation algorithms often require multiple iterations on the target problem. It can bring unacceptable costs and
resource costs in dealing with these complex problems. In order to solve the parameter optimization of complex
problems, in this paper we propose a multi-level surrogate-assisted optimization algorithm (MLSAO). By con-
structing surrogate models at different levels, the algorithm effectively explores the parameter space, avoiding
local optima and enhancing optimization efficiency. The method combines two optimization algorithms, differ-
ential evolution (DE) and Downhill simplex method. DE is focused on global level surrogate model optimization.
Downhill simplex is concentrated on local level surrogate model update. Random forest and inverse distance
weighting (IDW) are constructed for global and local level surrogate model, respectively. These methods lever-
age their respective advantages at different stages of the algorithm. The MLSAO algorithm is evaluated against
other state-of-the-art approaches using benchmark functions of varying dimensions. Comprehensive results
from the comparisons showcase the superior performance of the MLSAO algorithm in addressing expensive
 Information Technology and Control 2024/1/53 281

optimization problems. Moreover, we implement the MLSAO algorithm for tuning precipitation parameters in
the Community Earth System Model (CESM). The outcomes reveal its effective enhancement of CESM’s sim-
ulation accuracy for precipitation in the North Indian Ocean and the North Pacific region. These experiments
demonstrate that MLSAO can better address parameter optimization problems under complex conditions.
KEYWORDS: Surrogate-assisted optimization; Random Forest; Inverse distance weighting; Multi level opti-
mization.

1. Introduction
Parameter optimization issues are prevalent in the values under different input parameters. The surro-
fields of science and engineering, necessitating the gate model offers the advantage of significantly low-
need for highly efficient computational algorithms to ering the cost and complexity of analysis, enhancing
address these challenges. In recent years, a variety of analysis efficiency and accuracy, and optimizing real
intelligent optimization algorithms, such as differen- systems during the design stage to minimize the need
tial evolution (DE) algorithm, genetic algorithm, par- for trial and error.
ticle swarm optimization (PSO) algorithm, have been Considering that the process of building surrogate
proposed and effectively employed in a multitude of models simplifies the complex processes of actual
engineering optimization scenarios. Their effective- complex systems into mathematical models, it be-
ness can be partly attributed to the circumstance
comes challenging to fully simulate the overall be-
that intelligent optimization methods do not require
havior of real systems. This error increases with the
objective functions to be analytical or differentiable,
growing complexity of real systems. As various opti-
while also possessing enhanced global search capa-
mization problems become increasingly complex, it
bilities. Nevertheless, the primary trait of these al-
becomes challenging to ensure the accuracy of a sin-
gorithms is their tendency to demand a substantial
gle-level surrogate model. Such models may struggle
number of fitness evaluations in order to pinpoint a
to adequately represent intricate systems and meet
solution that is close to optimal. Some optimization
precision requirements for minimizing errors. More-
problems may involve intensive computation and
over, there is a tendency for it to become ensnared in
costly simulation [16]. Applying these algorithms to
these problems with high computational costs pres- local optima, potentially failing to discover the genu-
ent a significant challenge. ine global optimum throughout the optimization pro-
cess. Thus, improving the fitting accuracy of surrogate
To tackle this issue, surrogate model is proposed as an
models and evading local optima stands out as one of
alternative to the costly performance evaluations in
the most crucial challenges confronted by optimiza-
order to mitigate the computational expenses. Surro-
tion algorithms based on surrogate model. In previ-
gate model is an effective tool for building a simplified
ous research, when dealing with such problems, more
model of the actual complex system for rapid testing,
emphasis was placed on using different algorithms in
verification and optimization. Surrogate model uses
various search stages, rather than constructing surro-
approximate methods to construct models instead
gate models at different levels. However, in practical
of complex models to simplify the optimization pro-
optimization problems, when the problem to be opti-
cess, so as to improve computational efficiency while
ensuring accuracy. Surrogate model predicts infor- mized is complex, it is necessary to set up multi-lev-
mation of unknown points through known sampling el surrogate models. Constructing multiple levels of
points, which can actually be attributed to construct surrogate models and conducting searches at various
an equation to instead of the large complex mod- stages can achieve a more optimal balance between
el involving costly simulation. It is an approximate exploration and exploitation, thereby increasing the
method based on experimental design technology. likelihood of identifying potential optimal solutions.
Surrogate model establishes a mathematical model In this paper, we propose a multi-level surrogate as-
between input parameter and output objective func- sisted optimization (MLSAO) algorithm. We attempt
tion values, predicts the output objective function to build a global-level surrogate model by a machine
282 Information Technology and Control 2024/1/53

learning method random forest, which can effec- 3 We conduct tests on multiple benchmark func-
tively handle high-dimensional data and non-linear tions with varying dimensions and apply MLSAO
relationships, and has good robustness and stability. to parameter tuning in CESM, demonstrating its
Then a differential evolution algorithm was applied effectiveness.
to obtain the optimal solution of the global surrogate The manuscript is structured as follows: In Section 2,
model and update the global surrogate model. To nav- we introduce some related works about the proposed
igate away from local optimal solution and explore the algorithm. We provide an overview of the algorithms
global optimum, we establish a local surrogate mod- incorporated in the proposed MLSAO in Section 3 and
el based on inverse distance weighting. This model elaborates on the details of the proposed algorithm in
helps explore regions where optimal solutions may Section 4. Experimental results are presented in Sec-
be present. The local model is simpler and smaller tion 5, and the study’s conclusions are summarized in
in scale than the global surrogate model. We select a Section 6.
small portion of high-quality data to build this new
surrogate model, aiming to construct a surrogate
model with high-quality and avoid searching mean-
ingless parameter space. Considering the data scale 2. Related Works
of the local surrogate model, we design a simplex In recent years, various methods have been employed
downhill method-oriented local model update strat- for constructing surrogate model, including poly-
egy. Compared with intelligent algorithms, the sim- nomial response surface (PRS) [15], support vector
plex downhill method converges quickly and does not regression (SVR) [50], Kriging [11], radial basis func-
require repetitive iterations. It is suitable for small- tion (RBF) [46], artificial neural networks (ANN) [12,
scale optimization problems. We combine two level 10, 28], multivariate adaptive regression [49], and
surrogates with different update strategies to realize random forests [2, 3], etc. Building upon this founda-
an algorithm that can solve complex model optimi- tion, numerous studies have explored the distinctions
zation problems. To assess the effectiveness of the between these models and the scenarios in which
proposed algorithm, experiments are conducted us- they are best suited [1, 9, 14, 41], contributing to their
ing diverse mathematical function benchmarks. The widespread application across various engineering
results indicate that the proposed algorithm has more domains. For instance, paper [24] utilized a PRS mod-
advantages compared to previous algorithms. Finally, el to assist in making parameter choices and to facil-
we apply the proposed algorithm to a complex opti- itate the comparison of sensitivity properties among
mization problem: the parameter tuning of the earth climate models. Xu et al. [44] introduced an optimi-
system model, the precipitation simulation results of zation method for land model parameter tuning using
CAM5 has been improved over several regions. a RBF surrogate model. Müller et al. [23] used RBF
The contributions of this work can be summarized as model for methane emission estimation. Yue et al.
follows: [48] used RBF adaptive surrogate model optimization
1 We propose the MLSAO algorithm, which utiliz- to search for a combination of parameters relevant to
es Random Forest to construct a global surrogate the geometry and elasticity of track structures. [32,
model and inverse distance weighting (IDW) to 45] proved that SVR model can be used for parameter
construct a local-level surrogate model. This ap- calibrate of finite element and satellite systems. Chu
proach enhances the search capabilities for target et al. [7] used Kriging model for resonance frequency
problem and helps avoid falling into local optimal analysis of dental.
solutions. To address computationally expensive problems
2 We integrate two powerful optimization algo- more effectively, researchers have introduced surro-
rithms, differential Evolution (DE) and Simplex gate-assisted optimization algorithms. Over the past
Downhill, into MLSAO, applying them to different decades, the literature has documented various al-
search stages. This capitalizes on the strengths of gorithms that leverage surrogate models to enhance
each method, striking a balance between explora- optimization processes. For example, Yu et al. [47]
tion and exploitation. propose a surrogate-assisted hierarchical particle
 These algorithms combine population.
surrogateThe modelpopulation through is thenthe updated
selection
These algorithmswith modelmany combine
kinds of surrogate through
optimization model the selection operation. The
algorithms. of key
DE inclu
steps
These algorithms combine surrogate of DE include:
with manyFurther kinds of optimization
enhance the algorithms.
of
advantages DE include:
of surrogate
with many kinds of optimization algorithms. Initialize
Further enhance theinadvantages
solving ofcomplex surrogateproblems. Initialize Initializatio optim
Further enhance the advantages ofmodels surrogate Initialize optimization conditions:
models However, in solving complex problems. Initialization of
models
Information Technology in andsolving
Control complex problems.some of 2024/1/53 theseInitialization
algorithms involve of optimization 283 involves
conditions deo
However, differentsome of level thesesurrogate
algorithms involve
model, they do not involves defining the
However, some of thesedifferent algorithms involve involves defining the control parameters differential
of the
level
consider surrogateusing model,
different theymodelsdo not at evolution
different algorithmdifferential evolution
different level surrogate model, they do not differential and function.
the fitness T
consider using search different
stages. models at different function. These contr
consider using differentsearch models at different function. These control parameters the populat
consist of
stages. the population size 𝑁𝑁
swarm optimization (SHPSO) algorithm, combine
search stages. the population size 𝑁𝑁𝑁𝑁, scaling factor crossover𝐹𝐹, and
RBF model with different PSO method. Liu et al. [21] 3. 3.Preliminaries
crossover probabilit
generate an
Preliminaries crossover probability generate 𝐶𝐶𝐶𝐶 . Subsequently,
and assess th
propose bagging-based 3.surrogate-assisted 3. Preliminaries generate and assess the initial initial
population. pop
The
Preliminaries evolu- InInthis section, we
this section, we provide provide background information initial
backgroundis shown as equations
initial population
population
follows in
is
(4
tionary algorithm (B-SAEA). This approach incor- In this pertinent
section, to
wethis study.to background
provide equations (4-5):
In this section, we provide information
background pertinent this study. (4-5):
equations X (0) ∣ �
porates bagging to construct high-quality surrogate information pertinent to this study. � � ∣ x{
information pertinent to this study. 3.1Problem Definition X� (0) ∣∣ x{�,�} (0) ≤ x
pro- 3.1. Problem Definition Definition X� (0) ∣∣ x{�,�} (0) ≤ x{�,�} ≤ x{�,�} (0),𝑡𝑡 ∈ [1,(4)
� �
model for each costly objective. Wang et al. [40]3.1Problem 𝑁𝑁𝑁𝑁
3.1Problem
posed the Evolutionary Sampling Assisted Optimi- Definition The optimization problem can be formulated 𝑡𝑡 ∈ [1, 𝑁𝑁𝑁𝑁]; 𝑤𝑤 ∈ [1
The
The optimization optimization
problem problem
can be(1-3): 𝑡𝑡 can
∈
formulated [1,be formulated
𝑁𝑁𝑁𝑁]; 𝑤𝑤 ∈ [1, as fol-
𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷], where X� (0 (5)
zation (ESAO) method, The optimization
leveraging problem
two capabilities can be as follows
formulated in equations where X�while (0) denote
as follows lows in equations
in equations (1-3):(1-3): where X (0) denotes the 𝑡𝑡 -th individual,
�
𝑥𝑥{�,�}
to account for both global asand
follows
local insearches.
equations Sun (1-3):et min�
�∈� f(x) � while 𝑥𝑥{�,�} �
and 𝑥𝑥bou �
minwhile f(x)𝑥𝑥 �
and 𝑥𝑥 denote the upper
lower and
{�,�
al. [36] proposed the Two-Layer Surrogate-Assisted min�∈� f(x)
�∈� {�,�} (1)
{�,�} upper bounds of
upper (1) bounds of the (1)𝑤𝑤 -th respectively dimension
PSO (TLSAPSO) algorithm, utilizing both global and (1)
𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 𝑡𝑡𝑡𝑡 𝑢𝑢� (𝑥𝑥) ≤ 0, 𝑖𝑖 = 1,2, … , 𝑛𝑛 respectively. each Theindiv fit
𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 𝑡𝑡𝑡𝑡 𝑢𝑢� (𝑥𝑥) ≤ 0, 𝑖𝑖 = respectively.
1,2, … , 𝑛𝑛 The fitnesseach
(2)
function
(2) individual value isforc
several local surrogate models for𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠
fitness𝑡𝑡𝑡𝑡approxima-
𝑢𝑢� (𝑥𝑥) ≤ 0, 𝑖𝑖 = 1,2, … , 𝑛𝑛 each individual(2) is calculated in population. the initial
population.
tion. Li et al. [19] proposed a surrogate-assisted hy- 𝑣𝑣� (𝑥𝑥) = 0, (2) 𝑗𝑗 = 1,2, … , 𝑟𝑟, population. (3) (3) Mutation:
𝑣𝑣� (𝑥𝑥) = 0, 𝑗𝑗 = 1,2, … , 𝑟𝑟, (3) Mutation: acquire Conduct
brid swarm optimization (SASHO).
𝑣𝑣� (𝑥𝑥) = 0, 𝑗𝑗 = Two
1,2, …swarms
, 𝑟𝑟, are where the (3) 𝑥𝑥 represents Mutation:the vector Conduct of mutation operations the to
respectively used in different optimization where
states. where
the 𝑥𝑥 the x represents
represents the the vector
vector of
of parameters need
acquire the intermedi
where the 𝑥𝑥 represents theto vector parameters need to be
acquire adjusted.
of f(x) represents the objective function. the intermediate
𝑓𝑓(𝑥𝑥) population. 𝑉𝑉 (𝑛𝑛 + 1) =
Xin et al. [42] proposed parameters
a surrogate and autoencod- parameters beneed adjusted. to be adjusted. 𝑓𝑓(𝑥𝑥) �
𝑉𝑉� (𝑛𝑛 + 1) = 𝑋𝑋�� (𝑛𝑛) +
need to be adjusted. 𝑓𝑓(𝑥𝑥) 𝑉𝑉� (𝑛𝑛 optimization
+ 1) = 𝑋𝑋�� (𝑛𝑛) problem,+ 𝐹𝐹�𝑋𝑋�� (𝑛𝑛) − 𝑋𝑋�� (𝑛𝑛)� (6)
er-assisted multitask particle swarm optimization While dealing with complex
algorithm to solve multimodal optimization prob- the objective function may not have a clear mathe-
matical expression or the expression may be difficult
lems. A surrogate-assisted differential evolution
to compute within a finite time. In the proposed al-
with knowledge transfer (SADE-KT) [20] integrate
gorithm, the surrogate is considered as the objective
knowledge transfer and the surrogate-assisted evo-
function to fit the parameters and corresponding val-
lutionary search proposed for expensive incremental
ues obtained by the complex model. The functions
optimization problems. u(x) and v(x) denote the inequality and equality con-
In recent years, some parallel-based optimization straints, respectively. They have different forms of ex-
methods have been proposed to address parameter pressions in different problems.
optimization for complex problems. They aim to en-
hance optimization efficiency by leveraging addition- 3.2. Differential Evolution Algorithm
al computational resources. For example: Parallel Differential evolution (DE) is an evolutionary algorithm
BNN-GA [6] has been demonstrated to better address for global optimization, introduced in [33]. The algo-
large-scale data optimization problems. HAS-EA rithm generates new individuals by differential muta-
[22] proposed a surrogate-assisted optimization al- tion of individuals in the population, and updates the
gorithm based on heterogeneous platforms. Xing et al population through selection operation to systemati-
[43] proposed a parallel kinging surrogate model opti- cally explore the search space for the global optimum.
mization method and Improved the expect improve- The primary stages of the DE algorithm encompass
ment function,which improved global performance population initialization, differential mutation oper-
and solution accuracy. However, these methods have ation, selection operation, and termination condition.
high requirements for computational resources. The core of the DE algorithm lies in the differential mu-
When computational resources are limited or the tar- tation operation, where new individuals are generated
get problem is difficult to parallelize, the optimization through the linear transformation of individuals in the
efficiency still cannot be improved. population. The population is then updated through the
These algorithms combine surrogate model with selection operation. The key steps of DE include:
many kinds of optimization algorithms. Further en- Initialize optimization conditions: Initialization of
hance the advantages of surrogate models in solving optimization conditions involves defining the control
complex problems. However, some of these algo- parameters of the differential evolution algorithm
rithms involve different level surrogate model, they and the fitness function. These control parameters
do not consider using different models at different consist of the population size NP, scaling factor F, nd
search stages. crossover probability CR. Subsequently, generate and
 e Initialize
Initialize
Initialize optimization
optimization
optimization conditions:
conditions:
conditions:
s. Initialization
Initialization ofof optimization
Initialization of optimization
optimization conditions conditions
conditions
e involves
involves
involves defining
defining
defining the
thethe control
control
control parameters
parameters
parameters ofofofthe
thethe
ot differential
differential
differential evolution
evolution
evolution algorithm
algorithm
algorithm andand
and the
thethefitness
fitness
fitness
284 function. These control parameters Information Technology and Control 2024/1/53
t function.These
function. Thesecontrol controlparameters parametersconsist consistof
consist ofof
the
thethepopulationpopulation
population sizesize𝑁𝑁𝑁𝑁,
size 𝑁𝑁𝑁𝑁,scaling
𝑁𝑁𝑁𝑁, scalingfactor
scaling factor𝐹𝐹,
factor 𝐹𝐹,𝐹𝐹,andand
and
ation (6) describes the mutation operation utilized optimization algorithm designed to
crossover
crossover probability
probability 𝐶𝐶𝐶𝐶 . Subsequently,
𝐶𝐶𝐶𝐶 . . Subsequently,
Subsequently,
ere 𝑚𝑚1 𝑚𝑚2 , crossover
and 𝑚𝑚3 are probability
three random 𝐶𝐶𝐶𝐶 locate the minimum or maximum of a given
generate
assess
generate
generate and
the
andand assess
initialassess
assess the
population.
the
the initial
initial
initial population.
The initial
population.
population. The
population
The
The is function.
timization algorithm designed to locate the minimum
mbers in intervals initial
[1, 𝑁𝑁𝑁𝑁]. 𝐹𝐹 is iscalled
population shown
the as follows inobjective It falls under the category of
ing factor, which initial
shown
initial population
as
population follows
is a fixed constant. 𝑛𝑛 in is
is shown
equations
shown asas
(4-5): follows
follows in
in or maximum
direct search algorithms, indicating that it does of a given objective function. It falls un-
d equations
equations(4-5):
equations (4-5):
(4-5):
dresents the 𝑛𝑛-th generation. der the category
not necessitate knowledge of the gradient of of direct search algorithms, indicat-
XX�X(0) ∣
∣ x∣
� �
(0) ≤ x ≤ x ��
(0), (4) (4) ing
(4)the objective function. that it does not necessitate knowledge of the gra-
� (0)∣∣ ∣xx (0)≤≤xx{�,�} (0),
� �
(0) {�,�} (0) {�,�}≤≤xx
{�,�} {�,�} (0), (4)
ssover: crossover� operations {�,�}
{�,�} are shown in {�,�}
{�,�}
dient of the objective function.
ation (7). 𝑡𝑡𝑡𝑡 𝑡𝑡∈∈∈[1, [1,
[1, 𝑁𝑁𝑁𝑁]; 𝑤𝑤𝑤𝑤𝑤𝑤∈∈∈[1,
𝑁𝑁𝑁𝑁];
𝑁𝑁𝑁𝑁]; [1, 𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷],
[1, 𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷],
𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷𝐷], (5)
(5)(5)The term (5) "simplex" denotes a geometric shape
dd where XX1) (0) denotes
denotesthe 𝑡𝑡𝑡𝑡 𝑡𝑡-th
-thindividual, created by aThe set of term 𝑛𝑛 +“simplex”
1 points,denotes rangingafrom geometric shape cre-
𝑉𝑉where
where (𝑔𝑔 + �X � (0)
(0) 𝑖𝑖𝑖𝑖 denotes
𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟(0,1) <the𝐶𝐶𝐶𝐶
the -th individual,
individual,
where �X� t(0) denotes �� the t-th individual, 𝑝𝑝� tox{t,𝑝𝑝w}� , in n-dimensional space. the shape from p0 to pn, in
while ated by a set of n + 1 points, ranging
�,� � � � L
𝑈𝑈�,� (𝑔𝑔 + 1) = �while while𝑥𝑥𝑥𝑥U{�,�}
while 𝑥𝑥{�,�} and and𝑥𝑥𝑥𝑥{�,�}
and 𝑥𝑥{�,�} denote
denotethe thelowerlowerand and
𝑥𝑥 𝑜𝑜𝑜𝑜ℎ𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 {�,�} denote the lower and
upper and x bounds w} denote
{�,�}
�,� the lower and upper bounds of the n-dimensional space. the shape satisfies
satisfies
upper {t,
upper bounds of
bounds ofof the the 𝑤𝑤𝑤𝑤
the 𝑤𝑤-th
(7) -th-th dimension,
dimension,
dimension,
1)
) w-th dimension,fitness
respectively. respectively. The fitness function
respectively.The
respectively. TheThe fitness fitnessfunction functionvalue
function valuefor
value forfor 𝑝𝑝 𝑝𝑝� 𝑝𝑝�
ere 𝐶𝐶𝐶𝐶 is theeach each value
crossover for
individual each
probability.individual
is calculated is
New inin the calculated
in the in
initial the initial 𝑑𝑑𝑑𝑑𝑑𝑑 � � ⋯ � ≠ 0.
each individual
individual isis calculated
calculated the initial
initial 1 1 1
2) population.
population.
)viduals are population. randomly generated by
population. It means that in one-dimensional
It means that in one-dimensional space, twospace, two points
bability.
3) Mutation:Conduct
Mutation: Conductmutation mutation operations operations to topoints
acquire cannot overlap, so the simplex is a line
) Mutation: Conduct
Mutation: Conduct mutation operations to mutation operations to cannot overlap, so the simplex is a line segment. In
ction: Choose the intermediate
acquire
individuals the intermediate
from population.
the population.
initial segment. Intwo-dimensionaltwo-dimensional space, three
of
f acquire
acquire the the intermediate
intermediate population.
population. space, three points cannot be collin-
points cannot besocollinear, so is the simplex is three-dimensional
adesigned to
)ulation and the
𝑥𝑥) Equation intermediate
𝑉𝑉𝑉𝑉�𝑉𝑉(𝑛𝑛
Equation� (𝑛𝑛
Equation
+
(𝑛𝑛
� + 1)(6)+1) (6)
1) =
(6) =
= 𝑋𝑋 describes
𝑋𝑋describes
𝑋𝑋
describes
��
population
(𝑛𝑛)
(𝑛𝑛) +
(𝑛𝑛) + 𝐹𝐹�𝑋𝑋+ the
𝐹𝐹�𝑋𝑋
the𝐹𝐹�𝑋𝑋
the ����
to(𝑛𝑛)−−𝑋𝑋��
mutation
(𝑛𝑛)
mutation
(𝑛𝑛) − 𝑋𝑋��
mutation
operation
(𝑛𝑛)�
operation
𝑋𝑋 (𝑛𝑛)� (6)
(𝑛𝑛)� (6)
operation .(6) (6) ear,utilized
utilized the simplex
optimization
optimization
utilized optimizationspace,
a triangle.
algorithm
algorithm
algorithm
In designed
designed to to
m a new generation where population.
𝑚𝑚1 𝑚𝑚2
�� ��
, and 𝑚𝑚3
��
are three
��
random triangle. Inspace,
three-dimensional
locate fourthe minimum
points cannot or maximum
be four
coplanar, ofsoaaathegiven simplex
where
where 𝑚𝑚1
Equation 𝑚𝑚1 𝑚𝑚2 ,
𝑚𝑚2 , and the
(6) describes and 𝑚𝑚3 are
are three
𝑚𝑚3mutation three random
randompoints cannotutilized
operation locate
locate the
the minimum
minimum
optimization or
or maximum
maximum
algorithm of
of
designed given
given to
numbers
� (𝑛𝑛 + 1) = 𝑈𝑈where
� (𝑛𝑛
Equation
numbers
numbers + 1),𝑚𝑚1in
in (6)
in𝑚𝑚2
intervals
, describes
intervals
intervals and 𝑚𝑚3
[1,
[1, 𝑁𝑁𝑁𝑁].
[1,the𝑁𝑁𝑁𝑁].
are(8)
𝑁𝑁𝑁𝑁]. 𝐹𝐹𝐹𝐹 is
𝐹𝐹
mutation
three is called
is called
called
random
the
operation the where
the abetetrahedron.
objective
islocate
objective
coplanar,
objective
function.
the function.
function.
minimum
so
This the It simplex
falls
ItItpattern
falls
orfalls
under
under
under
maximum
isthe
continuesthe
a category
theof afor
category
categorygivenhigherof
of di-
of
scaling
m1 m2, factor,
and m3arewhich threeis a fixed
random constant.
numbers in𝑛𝑛tetrahedron.
intervals This
direct
mensions, pattern
search where continues
algorithms,
the n+1 for
points higher
indicating cannot that lieit does
in
scaling
scaling factor,
numbers factor,
in intervals which
which[1,is𝑁𝑁𝑁𝑁].is a fixed
a fixed constant.
is constant. 𝑛𝑛
called the 𝑛𝑛dimensions, where direct search
direct search
objective thefunction.
algorithms,
algorithms,It falls indicating
indicating
under liethe
that it
that it does
incategory
doesof n-di-
an
𝑓𝑓�𝑈𝑈� (𝑛𝑛 + 1)� ≤[1,𝑓𝑓�𝑋𝑋
represents
represents NP]. � (𝑛𝑛)�,F the
the is 𝑛𝑛-th
called generation.
the
generation.scaling (9)𝐹𝐹 factor, which is a fixed not
not necessitate
necessitate n+1 points knowledge
knowledge cannot of
of the
the gradient
gradient of
represents
scaling factor, the which 𝑛𝑛-th
𝑛𝑛-th generation.
is a fixed constant. 𝑛𝑛 mensional
not necessitate
direct plane,
search algorithms, forming
knowledge an n-dimensional
indicating of thethat it doesof
gradient simplex.
of
an n-dimensional the
the objectiveplane,
objective function.forming an n-
(𝑛𝑛 + 1) = 𝑋𝑋�represents
(𝑛𝑛) constant.
Crossover:
Crossover:𝑜𝑜𝑜𝑜ℎ𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒.the n represents
crossover
𝑛𝑛-th
crossover generation. the(10)
operations
operations n-thare generation.
are shown
shown in
in not
The objective function.
thenecessitate
algorithm function.
knowledge
operates by of the
maintaining gradient a of
simplex,
Crossover: crossover operations are shown indimensional simplex.
Equation
Crossover: (7). crossover The
the term
objective "simplex"
function. denotes aa geometric shape
shown in Equations Equation
Equation
Crossover: (7).
(8)-(10),(7).
crossover in DE,operations a operations
greedy are are shown shown in in Equa- which
The
The is a geometric
term
term "simplex"
"simplex" shape
denotes
denotes consisting
a geometric
geometric of n+1 shape
shape vertices
tion (7). The algorithm in created
createdoperates
n-dimensionalby
by aa set
setby of
ofspace. 𝑛𝑛 +
maintaining 1At points,
points,
each ranging
a
ranging
iteration, from
from
ction strategy Equation is (𝑔𝑔 (7).
employed, 𝑉𝑉 (𝑔𝑔
�,�where + 1)
(𝑔𝑔 + 1) the 𝑖𝑖𝑖𝑖 𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟(0,1) < 𝐶𝐶𝐶𝐶 Thecreated
term by a set of denotes
"simplex" 𝑛𝑛 + 1
𝑛𝑛 + 1 points, a geometric ranging shapefromalgo-
the
𝑈𝑈 (𝑔𝑔 + 1) = ��𝑉𝑉𝑉𝑉�,� 𝑖𝑖𝑖𝑖 𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟(0,1) < 𝐶𝐶𝐶𝐶
�,� (𝑔𝑔 + 1) 𝑖𝑖𝑖𝑖 𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟(0,1) < 𝐶𝐶𝐶𝐶simplex, which 𝑝𝑝
𝑝𝑝 istoa
to 𝑝𝑝 ,, in
geometric
𝑝𝑝 in n-dimensional
shape
n-dimensional consisting space.
space. the
the shape
shape
erior individual𝑈𝑈�,� 𝑈𝑈 �,� +
is(𝑔𝑔chosen 1) =
+ 1) = �as the 𝑥𝑥𝑥𝑥�,� new 𝑜𝑜𝑜𝑜ℎ𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 rithm𝑝𝑝� to by
�
created
� evaluates
𝑝𝑝� ,a in
�� the
setn-dimensional objective
of 𝑛𝑛 + 1 points, function
space.
ranging at each
the from shape vertex
�,�
𝑉𝑉�,� (𝑔𝑔 + 1)𝑥𝑥�,� 𝑖𝑖𝑖𝑖𝑜𝑜𝑜𝑜ℎ𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒
𝑟𝑟𝑟𝑟𝑟𝑟𝑟𝑟(0,1) < 𝐶𝐶𝐶𝐶 of n+1 verticessatisfies
𝑜𝑜𝑜𝑜ℎ𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 in n-dimensional
satisfies space. At each
vidual. 𝑈𝑈�,� (𝑔𝑔 + 1) = �
�,�
(7)
(7) , (7) of𝑝𝑝�the to simplex,
satisfies and then performs
𝑝𝑝� , in n-dimensional space.a the series shape of opera-
𝑥𝑥�,� 𝑜𝑜𝑜𝑜ℎ𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 (7)iteration, thetions
algorithm evaluates 𝑝𝑝 𝑝𝑝 𝑝𝑝
satisfies to transform 𝑑𝑑𝑑𝑑𝑑𝑑 � �� the
𝑝𝑝𝑝𝑝the
� 𝑝𝑝 ��⋯objective
simplex.
�
𝑝𝑝 𝑝𝑝𝑝𝑝���� ≠These 0. operations in-
algorithm haswhere the advantages
where
where
𝐶𝐶𝐶𝐶
𝐶𝐶𝐶𝐶
𝐶𝐶𝐶𝐶
is
is
is
the
the
the
crossover
of simplicity,
crossover
crossover
probability.
probability.
probability.New
New
(7) function at each vertex of the
New
New clude reflection,
𝑑𝑑𝑑𝑑𝑑𝑑 ��1 1 ⋯
𝑑𝑑𝑑𝑑𝑑𝑑simplex,
11 𝑝𝑝 11 ⋯and
expansion,
1
1
�� ≠≠ 0.
then
1contraction, 0.
and shrink-
where
individuals CR is the
are crossover
randomly probability.
generated by individu- 𝑝𝑝 � � 𝑝𝑝�
y implementation, individuals no isneed are for randomlygradient generatedNew byperforms a series It
ItIt means that𝑑𝑑𝑑𝑑𝑑𝑑 in
� one-dimensional ⋯ � ≠ the 0. space, two
individuals
where als are
probability.
𝐶𝐶𝐶𝐶 randomly arecrossover
the randomly
generated bygenerated
probability. probability. by age, of operations
means
which
means that
that in
move in1 to
the transform
one-dimensional
simplex
1
one-dimensional 1towards space, the
space, minimum two
two or
rmation, and probability.
global convergence.
probability.
individuals are randomly It hasgenerated by simplex. These points
points cannot
operations
cannot overlap,
include
overlap, so
so the
reflection,
the simplex
simplex is
is aa line
line
Selection: Choose individuals from the initial maximum
It
popu- contraction,points
means of
cannot
that the in objective
overlap,
one-dimensional so function
the simplex value.
space, is atwo line
nd extensive Selection:
probability.
Selection:applications Choose
Choose in functionfrom
individuals
individuals from the
the initial
initial expansion, segment.
segment. In
and
In two-dimensional
shrinkage,
two-dimensional which space,
space, three
three
Selection: Choose individuals from the initial The segment.
points cannot
process In two-dimensional
overlap,
of Nelder-Mead so the simplex
algorithm space, is a three
line
is asis follows
mization, parameter lation and
population thethe
estimation,
and intermediate machinepopulation
intermediate population to form
tomove a newthe simplex points cannot be collinear, so the simplex a
population
population
Selection: Chooseand
and the
the intermediate
intermediate
individuals from population
the initialto
population to pointstowards
points
segment. cannot
cannot bethe
be
In istwo-dimensional minimum
collinear,
collinear, so
so the theor simplex
simplex
space, three is
is aa
ning, and various form generationaa new population.
generation population. and flowchart
triangle. In shown
three-dimensional in Figure 1. space, four
form other
form
population a new
new and
domains.
generation
generation
the intermediate population.
population. population to maximum ofpoints
triangle.
the objective
triangle. cannot In
In be three-dimensional
function
three-dimensional
collinear, value. so the simplex space,
space, is four
foura
points
Select
points cannot
initial
cannot be
be pointcoplanar,
coplanar, x , generateso
so the
the simplex
the
simplex is
remainingis aa n
Nelder-Mead form 𝑋𝑋 Algorithm
(𝑛𝑛
a new+ 1) = 𝑈𝑈
generation (𝑛𝑛 + 1), ,
population. (8) (8)
1 points
triangle. cannot
In be coplanar,
three-dimensional 0 so the simplex
space, four is a
𝑋𝑋𝑋𝑋��(𝑛𝑛
� (𝑛𝑛 + + 1) 1) = = 𝑈𝑈 𝑈𝑈��(𝑛𝑛
� (𝑛𝑛 + + 1), 1), (8)The process of
(8) Nelder-Mead
tetrahedron.
points.
tetrahedron. Construct This
This
algorithm
pattern
a simplex
pattern continues
continues
is
based
as for
onfor higher
these
higher points.
tetrahedron.
points cannot This
be pattern continues
coplanar, so 1: the simplex for higher is a
Nelder-Mead 𝑋𝑋algorithm
𝑖𝑖𝑖𝑖
� (𝑛𝑛 𝑓𝑓�𝑈𝑈
+ �1) (𝑛𝑛=+ [25],
1)�
� (𝑛𝑛≤
also
+𝑓𝑓�𝑋𝑋 referred
(𝑛𝑛)�,, (8)(9) follows and flowchart
dimensions, is shownwhere in
the Figure
n+1 points cannot lie in
𝑖𝑖𝑖𝑖
𝑖𝑖𝑖𝑖 𝑓𝑓�𝑈𝑈
𝑓𝑓�𝑈𝑈 ��(𝑛𝑛
(𝑛𝑛 + +𝑈𝑈1)�
1)� ≤≤ 1),
𝑓𝑓�𝑋𝑋���(𝑛𝑛)�,
𝑓𝑓�𝑋𝑋 (𝑛𝑛)�, (9)
(9) (9) dimensions,
dimensions,
2. Sort
2tetrahedron. the where
where
n+1
This points
pattern the
the n+1 n+1
according points
continuespoints cannot cannot
tofortheir
higherlie
lie in
in
objective
s the downhill simplex method, is a widely an
an n-dimensional
n-dimensional plane,
plane, forming
forming an
an n-
n-
𝑋𝑋 (𝑛𝑛
𝑖𝑖𝑖𝑖𝑋𝑋𝑋𝑋��𝑓𝑓�𝑈𝑈
(𝑛𝑛 +
+ 1)
(𝑛𝑛
1) + =
= 𝑋𝑋
1)�
𝑋𝑋 (𝑛𝑛)
≤
(𝑛𝑛) 𝑜𝑜𝑜𝑜ℎ𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒.
𝑓𝑓�𝑋𝑋 (𝑛𝑛)�,
𝑜𝑜𝑜𝑜ℎ𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒. (10)
(9)
(10) an
dimensions, n-dimensional
function where
values: f(p the )plane,
n+1
≤ f(p points) ≤ forming
... cannot
≤ f(p ). an
lie in n-
� (𝑛𝑛 + 1) = 𝑋𝑋�� (𝑛𝑛) 𝑜𝑜𝑜𝑜ℎ𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒.. (10)
�
� �
(10) dimensional
dimensional simplex.
simplex. 0 1 n
an dimensional
n-dimensional simplex. plane, forming an n-
As
As𝑋𝑋 shown
(𝑛𝑛
shown + 1) in=
in Equations
𝑋𝑋 (𝑛𝑛)
Equations (8)-(10),
𝑜𝑜𝑜𝑜ℎ𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒.
(8)-(10), inin DE,
DE, aa greedy
(10)
greedy 3 Remove the worst point p and calculate the cen-
AsAs � shown
shown inin Equations
�
Equations (8)-(10),
(8)-(10), in DE,
inwhere
DE, a greedy
a greedy selec- The
dimensional
The algorithm
algorithm simplex. operates
operates n – 1 pi by
byn maintaining
maintaining aa
ure 1 selection
selection strategy
strategy is
is employed,
employed, where the
the The
ter ofalgorithm
gravity p operates
= ∑ .by Reflectmaintaining
the worst apoint
As selection
shown
tion strategy in strategy
Equationsis employed, is employed,
(8)-(10), in
whereasthetheDE, where
a greedy
superior the individ- simplex,
simplex, which
which is g a geometric
n by shape shape consisting
which is is aa geometric shape consisting
i = 0
superior
superior individual
individual is
is chosen
chosen as the new
new Thesimplex,algorithm operates geometric maintaining consisting a
flowchart of Neldersuperior
selection
ual is Mead chosen individual
Algorithm.
strategy as the is new is chosen
employed,individual. as
where the the new of
of using a reflection
n+1 vertices in coefficient ρ space.
n-dimensional ≤ 0 andAt is usually
ρ each
individual.
individual. of n+1
simplex, n+1 vertices
vertices
which is in
inan-dimensional
n-dimensional
geometric shape space.
space. At
At each
consisting each
individual.
superior individual is chosen as the new set to
iteration,
iteration, 1. the
the algorithm
algorithm evaluates
evaluates the
the objective
objective
DE
DE algorithm
algorithm has the
has the advantages
advantages of
of simplicity,
simplicity,
easy iteration, the algorithm
of n+1 vertices in n-dimensional space. At each evaluates the objective
individual.
DE algorithm
algorithm has has the advantages of
of simplicity, function
function at
at <each
each vertex
vertex of
of the
the simplex, andthethen
DE implementation, the no advantages
need for simplicity,
gradient information, If f0 < frthe
4iteration,
function at falgorithm
n– 1, replace
each vertexevaluates the simplex,
ofpoint pn the
simplex, with and
and
objective then
reflected
then
easy
easy implementation,
implementation, no
no need
need for
for gradient
gradient performs
performs a
a series
series of
of operations
operations to
to transform
transform the
the
DE easy algorithm implementation,
and global convergence. has the no
advantages need of for
It has found extensive gradient
simplicity, appli- point
performs
function pat to
a
each construct
series vertex of a new
operations
of the simplex.
simplex,to transform
and then the
information,
information, and
and global
global convergence.
convergence. It
It has
has simplex.
simplex.
r
These
These operations
operations include
include reflection,
reflection,
information,
easy cations implementation, and
in function global no convergence.
need
optimization, for gradient
parameterIt has simplex.
performs
estima- Ifexpansion, a
fr < f0, it means These
series ofoperations
operations
the objective include
to transform
function valuereflection, the of the
found
found extensive
extensive applications
applications in
in function
function expansion, contraction,
contraction, and
and shrinkage,
shrinkage, which
which
found
information,
tion, machine extensive andlearning, globalapplications
convergence.
and various in other function
It has domains. expansion,
simplex. These contraction,
operations and includeshrinkage, reflection,whichin the
optimization,
optimization, parameter
parameter estimation,
estimation, machine
machine reflected
move point
the simplex p is smaller
towards the than all the
minimum or points
optimization,
found extensive parameterapplications estimation, in function machine move
move the simplex
simplex towards
the contraction,
r
towards the
the minimum
minimum or
learning,
learning, and
and various
various other
other domains.
domains.
expansion,
simplex.
maximum
maximum It isof of favorable
the
the objective
objective
andthe
for shrinkage,
function
function function value.
value. valueorto de-
which
3.3. Nelder-Mead
learning,
optimization, andparametervarious other Algorithm domains.machine
estimation, maximum
move the simplex of the objective towardsfunction the minimum value. or
3.3 Nelder-Mead crease in this direction. Extend in this direction pe =
3.3
3.3
learning, and variousAlgorithm
Nelder-Mead
Nelder-Mead Algorithm
Algorithm
other domains. The
maximum
The process
process of of
the
of Nelder-Mead
objective
Nelder-Mead function algorithm
value.
algorithm is
is as
as
The Nelder-Mead algorithm [25], also referred to as pg The + 𝒳(p process
= p ) of
with Nelder-Mead
a extension algorithm
coefficient 𝒳is >as1 and
3.3
The Nelder-Mead
Nelder-Mead Algorithm
algorithm [25], also referred follows
follows
r andg flowchart is shown in Figure 1:
and flowchart
The
The
to
the
as
Nelder-Mead
Nelder-Mead
the
downhill simplex
downhill
algorithm
algorithm
simplex
method, [25],
method,
[25],isalso also
a widely
is a
referred
referred
widely
utilized op- 𝒳 Thefollows
can process
be and
set asof If fe < fris
flowchart
2.Nelder-Mead is shown
shown
, which inin Figure
algorithm
representsFigureis1: 1:asexten-
the
to
Theto as as the
the downhill
Nelder-Mead downhillalgorithm simplex
simplex method, [25], alsois
method, aa widely
isreferred
widely follows and flowchart is shown in Figure 1:
to as the downhill simplex method, is a widely
Figure
Figure 1
Figure 11
The flowchart
Figure
The 1 of Nelder Mead Algorithm.
The flowchart
flowchart of
of Nelder
Nelder Mead
Mead Algorithm.
Algorithm.
The flowchart of Nelder Mead Algorithm.
 Information Technology and Control 2024/1/53 285

Figure 1
The flowchart of Nelder Mead Algorithm

sion is successful. Replace pl with pr, otherwise, and 4.1. The Procedure of the Proposed Algorithm
replace pn with pr. Building upon the aforementioned methods, we inte-
If fr ≤ fn – 1, it means the reflected point is still the worst grate them to formulate the multi-level surrogate-as-
point. A contraction operation is needed: If fn > fr ≥ fn –1, sisted optimization algorithm. The primary proce-
obtain the contraction point pc = pg + γ(pr – pg) , where dure is outlined in Figure 2.
0 < γ < 1 is the contraction coefficient, which can be set
In Figure 2, line 1-5 describes the process of sam-
as 0.5. This operation is called an outer contraction. If
pling. All the samples in the sampling set is generated
fr ≥ fn, replace the pr with pn, and update the value of pc
by selected sampling method which is described in
based on pc = pg + γ(pn – pg). This operation is called an
3.2.1. Each of these samples is subsequently forward-
inner contraction. If fc ≤ fn, the contraction is consid-
ed to the real complex model, which, in this problem,
ered successfully and replace pn with pc. Otherwise,
symbolizes the optimization objective. The model is
the contraction is failed. A new simplex is construct-
executed to compute the objective function value for
ed: keep p0 and halve the distance between p0 and the
each sample. The key-value pairs <parameters, ob-
other points, which is call a compression operation.
jective function value> will be utilized for construct-
ing the global surrogate model in line 6. The random
forest will fit the parameters and objective function
4. The Multi-level Surrogate-assist values. Given any vector within the range of each pa-
rameter, random Forest can generate predictions for
Optimization Algorithm the current vector based on the fitting results. In line
In this section, the procedure of the proposed 7-12, the constructed global surrogate model will up-
multi-level surrogate-assisted optimization algo- date by DE algorithm, during each iterative process,
rithm will be introduced in each subsection. DE will explore the entire parameter space of the
286 Information Technology and Control 2024/1/53

Figure 2 determining the quality of the sample set, and the ex-
The algorithm of the MLSAO cellence of the sample set directly influences the qual-
ity of the surrogate
the model. Various solution
optimal sampling methods
along with its
may introducecorresponding
distinct biases and variances function
objective in the value is
sample set. Improper sampling methods may result
output as the final result, concluding the
in insufficient samples or excessive sample biases,
algorithm.
thereby affecting the qualityThe of thefollowing
surrogate of this section will
model.
provideofdetailed
Hence, in the selection a samplingdescriptions
method, factorsof the critical
such as sampleparts in eachsample
distribution, step. size, and sample
quality should be taken into account to ensure that the
4.2 Sampling
sampled data accurately reflect the characteristics of
the original data.
At This, in turn, enhances
the beginning the predic- we select the
of the algorithm,
tion accuracy latin
of thehypercube
surrogate model. Latin
sampling (LHS) hyper-[30] method to
cube sampling (LHS) and Monte Carlo sampling [8]
generate samples. The sampling method plays
are suitable methods for building the surrogate mod-
a crucial role in determining the quality of the
el. They possess flexible sample sizes and excellent
sample set,
space-filling capabilities, and thecovering
efficiently excellence of the sample set
the en-
directly influences the quality
tire sample space with relatively few points [38]. LHS of the surrogate
model.
belongs to stratified Various
sampling technique sampling
employed for methods may
random sampleintroduce
selection indistinct
multi-dimensional
biases and space.
variances in the
This method aimssampleto better
set. represent
Improper thesampling
populationmethods may
using fewer sampling points. It is an
result in insufficient samples extension of the or excessive
Latin square sampling technique and introduces the
sample biases, thereby affecting the quality of
concept of hypercubes, which helps to distribute sam-
In Figure 2, line 1-5 describes the process of theacross
surrogate model. Hence,
surrogate model to search for the optimal solution ples more evenly the entire sample space. in the selection of
sampling. All the samples in the sampling set
of the surrogate model, which is then determined by One-dimensional a sampling method, factors such as sample
Latin hypercube sampling involves
isDE.generated by selected sampling method distribution,
A new key-value pair is added to the sampling set dividing the cumulative densitysample
functionsize,
(CDF)andinto sample
n quality
which is described in 3.2.1. Each of these
established in line 1-5, then we update the surrogate equal partitionsshould and thenbe taken into
selecting account
a random to ensure that the
data point
samples
model afteris increase
subsequently
in sampleforwarded
size. In thistoproposed
the real within each partition.
sampled Considering that each parame- reflect
data accurately the
method, we model,
complex construct which,
the local-level
in surrogate model ter follows a uniform
this problem, characteristics
distributionofwithinthe its
original
specifieddata. This, in
once the global-level
symbolizes surrogate modelobjective.
the optimization has converged.
The range, to obtain N samples, the the
cumulative density
turn, enhances prediction accuracy of the
Different from the global model. The role of the local function of the uniform distribution is utilized. Each
model is executed to compute the objective surrogate model. Latin hypercube sampling
surrogate model is to identify potential optimal solu- parameter is divided into N non-overlapping groups
function value for each sample. The key-value
tions based on the outcomes of the global model. So within its defined (LHS) and Monte Carlo sampling [8] are
range, with each group having a
pairs <parameters, objective function value>
that the sample size of the local model is much small- probability of 1/N suitable methods for building the surrogate
to be selected. Within each group’s
will
er, webe utilized
choose for subset
the optimal constructing
of samplesthe global interval, a parameter
according model.value Theyis possess
randomlyflexible sample sizes and
chosen. Fol-
surrogate model
to their objective in linevalues.
function 6. TheTherandom
subset willforest
up- lowing these rules,
excellent
M vectors space-filling
are generated,capabilities,
each rep- efficiently
date fit
will during
theeach iteration ifand
parameters generate better function
objective solution resenting the sampling
covering thefor entire
results sample
one parameter and space with
in line 14-18.
values. Given At last,
anythe optimal
vector solution
within thealong
rangewithof containing N elements.
relatively Thefew points [38].
an MLHS
goal is to create * N belongs to
its corresponding objective function value is output matrix, where each
each parameter, random Forest can generate stratified sampling techniqueByemployed for
row represents a sample point.
as the final result, concluding the algorithm. The fol- randomly selecting one element from each vector, a
predictions for the current vector based on the random sample selection in multi-dimensional
lowing of this section will provide detailed descrip- new M-dimensional vector is formed, resulting in a to-
fitting results. In line 7-12,
tions of the critical parts in each step. the constructed space. This method aims to better represent the
tal of N vectors. This ensures that the sampling points
global surrogate model will update by DE population using fewer sampling points. It is
4.2. Sampling are evenly distributed across the solution space.
algorithm, during each iterative process, DE an extension of the Latin square sampling
will explore
At the theofentire
beginning parameter
the algorithm, space
we select theof the
latin 4.3. Global Surrogate
techniqueModeland Construction
introduces the concept of
surrogate model (LHS)
hypercube sampling to search for the
[30] method optimal
to generate In the proposedhypercubes,
algorithm, thewhich helps tomodel
global surrogate distribute samples
samples. The
solution sampling
of the method
surrogate plays awhich
model, crucial is
role in
then is constructedmore
based evenly across
on random the
forest entireran-
method, sample space.
determined by DE. A new key-value pair is
One-dimensional Latin hypercube sampling
added to the sampling set established in line 1-
involves dividing the cumulative density
5, then we update the surrogate model after
function (CDF) into 𝑛𝑛 equal partitions and
increase in sample size. In this proposed
then selecting a random data point within each
method, we construct the local-level surrogate
partition. Considering that each parameter
model once the global-level surrogate model
follows a uniform distribution within its
 Information Technology and Control 2024/1/53 287

dom forest is a supervised ensemble learning method parameter range constraints, the random forest sur-
that belongs to the Bagging class and is implement- rogate model can generate simulated target function
ed based on decision trees. It improves prediction values based on the training.
accuracy by combining multiple decision trees. It is
the most commonly used ensemble learning model. 4.4. Update Global Surrogate Model
Firstly, the data set is sampled using the Bootstrap In general, solving such complex problems relies on
algorithm, and each group of data subsets is select- optimization algorithms that directly iterate in the
ed. Then, decision tree model parameters are set and real model. When we optimize a statistical model
trained on each data subset. Finally, these decision with relatively high accuracy using such algorithms,
tree models are used to vote and obtain the results. the iterations are almost cost-free. Achieving optimi-
The key to Random Forest is the decision tree struc- zation effects requires running only a small number
ture, and the key to decision trees is the feature split- of real models. This is the primary advantage of surro-
ting method, i.e., how to determine the quality of the gate models. However, constructing a surrogate mod-
splitting features. Random Forest uses the out-of-bag el involves a limited number of sample points, and the
error method, which does not use all samples when initial surrogate model may inevitably contain some
generating trees. Instead, a portion of the samples errors. Therefore, it is essential to judiciously in-
are reserved for validation. Approximately one-third crease the number of sampling points to continuously
of the samples in each decision tree are reserved for update the surrogate model, thereby reducing errors,
assessing the model’s performance and calculating its improving simulation accuracy, and enhancing the
error, not used during the model training process. The representation of the real model. There are numerous
generation rules of each tree are as follows: methods for updating the surrogate model, including
Initially, the parameter values are confirmed, where the minimum interpolating surface (MIS) [17], max-
N represents the number of training cases (samples), imum expected improvement (MEI) [31], and candi-
and M represents the number of features. The param- date point approach (CAND) [29]. In this work, we
eter m is crucial for determining the decision at a tree are more concerned about the optimal values rather
node, and it should be much smaller than M. Next, the than the complete shape of the fitting surface. MIS
training set is formed by repeatedly sampling with re- is a method that does not concentrate on construct-
placement from the N training cases (bootstrap sam- ing the entire surrogate but rather focuses on the
pling). The unselected cases are then used for making region containing the optimum. Hence, our focus is
predictions to evaluate their errors. For each node, on ensuring the accuracy of the surrogate model in
m features are randomly chosen, and the decision at the vicinity of the optimal point. Employing the MIS
each node is based on these selected features. The method, we utilize DE to discover the optimal value
best splitting method is determined based on these of the surrogate model. These optimal parameters are
m features. Each tree is allowed to grow fully without subsequently applied to the real model to iteratively
pruning, although pruning may be considered after acquire new key-value pairs, enrich the sampling set,
constructing a standard tree classifier. and augment the number of points surrounding the
In the proposed algorithm, the surrogate model plays optimal point. This iterative process enhances the
a crucial role in establishing the mapping between pa- modeling accuracy in the proximity of the optimum.
rameters and the objective function. Instead of engag- 4.5. Local Surrogate Model Construction
ing in the lengthy process of calculation and simula-
tion of the complex model, a mathematical statistical In this section, we select inverse distance weighting
model is employed to efficiently generate the objective (IDW) [4] model to construct local-level surrogate
function values. Hence, during the surrogate model model. It assigns weights to known values based on the
construction process, the essential key-value pairs inverse of their distances to the target location, result-
<parameters, objective function value> are utilized as ing in a weighted average calculation for the unknown
inputs and outputs for data fitting in the model. For value. IDW assumes that closer points have a greater
a surrogate model constructed through random for- influence on the estimation, while distant points have
est, given any set of parameter vectors that satisfy the less influence. The IDW method is as follows:
 known
distances values to thebased target on location,
the inverse of their
resulting in a crucial because
surrogate a high-quality
model is essential initialtosetefficiently
for the
distances to the target
weighted average calculation for the unknownlocation, resulting in a surrogate model is essential
identify optimal solutions that might be to efficiently
weighted
value. IDW average assumescalculationthat closer for the unknown
points have a identify optimal
challenging for thesolutions that might
global model to uncover.be
value. IDW assumes
greater influence on the estimation, that closer points have a
while challenging
When this for stepthe global model
is completed, to uncover.
the algorithm also
288 Information Technology and Control 2024/1/53
greater
distantinfluence points have on less the influence.
estimation,The while IDW When
ends,this stepthe
and is completed,
obtained optimalthe algorithmsolutionalso is
distant
method pointsis as have
follows: less influence. The IDW ends, and the obtained optimal solution is
output.
method is as follows: output.
�(�,��� )
∑�����(�,�� 𝑦𝑦𝑦𝑦� 𝑖𝑖𝑖𝑖 𝑝𝑝 ≠ 𝑝𝑝𝑝𝑝�
5. Experiments Results
�
𝑦𝑦 =∑���� �� ∑ �)
�(�,�� �) , (11)
𝑦𝑦𝑦𝑦� 𝑖𝑖𝑖𝑖 𝑝𝑝 ≠ 𝑝𝑝𝑝𝑝�
5. Experiments Results
���
𝑦𝑦 = � ∑ �(�,�� )
��� 𝑦𝑦𝑦𝑦 � 𝑖𝑖𝑖𝑖 𝑝𝑝 = 𝑥𝑥 ,, (11) (11)
�
𝑦𝑦𝑦𝑦� 𝑖𝑖𝑖𝑖 𝑝𝑝 = 𝑥𝑥�
�
5.section,
In this Experiments Results the effective-
we aim to demonstrate
where 𝑝𝑝 denotes the prediction input point, In this section, we aim to demonstrate the
nessInof this
the proposed
section, of algorithm
wethe to in
aimproposed addressing the
demonstrate the in
where
and denotes thethe
𝑝𝑝 represents
𝑦𝑦 prediction
prediction input
result point,
of the effectiveness algorithm
where p denotes the prediction input point, and y rep- optimization of complex
effectiveness
addressing of
the the model parameters.
proposed
optimization of algorithm
complex
At first,in
model
andpoint 𝑦𝑦 represents
𝑝𝑝. the prediction
� belongs to the result of
training set the =
resents the𝐷𝐷𝐷𝐷 prediction result of the point p. Dtj𝑇𝑇belongs someaddressing
mathematical the benchmark functions
optimization of complex optimiza-
model
point 𝑝𝑝. �𝐷𝐷𝐷𝐷 belongs and to the training set 𝑇𝑇 = parameters. At first, some mathematical
{𝑦𝑦𝑦𝑦the
to � , 𝑦𝑦𝑦𝑦 , …� , 𝑦𝑦𝑦𝑦
training � }, T
set = {yt𝑦𝑦𝑦𝑦 � is
, yt , ...,the
ytn}output
, and ytj value is the out- tion experiments are conducted.some Subsequently, a pre-
{𝑦𝑦𝑦𝑦for and 𝑦𝑦𝑦𝑦�1 point. is 2the output value parameters.
benchmark At first, functions mathematical
optimization
� , 𝑦𝑦𝑦𝑦the
� , … ,𝑗𝑗𝑦𝑦𝑦𝑦 � }, training
-th The weighting
put value for the j-th training point. The weighting cipitation parameter
benchmark tuning experiment with CESM
forfunction
the 𝑗𝑗 -th training point. The weighting experiments arefunctions optimization a
conducted. Subsequently,
function d𝑑𝑑is is defined
defined as: as: confirmed the capability
experiments are of the proposed
conducted.
precipitation parameter tuning experiment
algorithma
Subsequently,
function 𝑑𝑑 is defined ��as: in addressing intricate practical parameter optimiza-
𝑑𝑑(𝑝𝑝� , 𝑝𝑝� ) = |𝑝𝑝� , 𝑝𝑝� |� ,, precipitation
with CESM parameterconfirmed tuning experiment
the capability of the
𝑑𝑑(𝑝𝑝� , 𝑝𝑝� ) = |𝑝𝑝� , 𝑝𝑝� |� ,
�� tion with
challenges. These
CESM algorithm experiments
confirmed in are supported
the addressing
capability of by
the
where 𝑞𝑞 is termed the power parameter, and proposed intricate
where𝑞𝑞 qisistermed termed the power power parameter, and it must [5, 27, 37].
proposed
where
it must be strictlythe greater than parameter,
1 to ensure andthe practical algorithm
parameterinoptimization
addressing challenges.
intricate
be
it must strictly
be strictlygreater than 1 to ensure
greater than 1 to ensure the the continuity of practical parameter optimization
These experiments are supported by [5, 27, 37]. challenges.
continuity of derivatives.
derivatives.
continuity of derivatives.
TableThese
1 experiments are supported by [5, 27, 37].
The local surrogate model
The local surrogate model is constructed to leverage is constructed to Benchmark function description in this experiment
The local
leverage surrogate
thefrominsights modelfrom the is constructed
global surrogate to
the insights the global surrogate model and ex- Table 1
Function Global
leverage
model the
and insights
explore from the global
potential surrogate
improvements. Table 1 Function name
plore potential improvements. Therefore, to enhance number
Benchmark function description in thisoptimum
experiment.
model
Therefore, and explore potential improvements.
the accuracytoof enhance the surrogate the model,accuracy careful of the consid- Benchmark function description in this experiment.
Therefore,
surrogate to enhance
model, careful theconsideration
accuracy ofis given the F1 Function EllipsoidFunction name 0
Global
eration is given to both the quantity and quality of the
surrogate
to both model,
the careful
quantity and consideration
quality of theis selected
given number Function name
Function Global
optimum
selected points for constructing the model. For rela- F2 number Rosenbrock 0
optimum
to points
both the for quantity and qualitythe
constructing of the selectedFor
tively small surrogate models, IDWmodel. has higher mod- F1 Ellipsoid 0
points
relatively for small constructing
surrogate the models, model. IDW For has we F3 F1 AckleyEllipsoid 00
eling performance. When constructing the model, F2 Rosenbrock 0
relatively
higher the small
modeling surrogateperformance.models, IDW When has
choose top x fitness function-ranked points in the F4 F2 Griewank
Rosenbrock 00
higher
constructing modelingmodel, performance. When F3 Ackley 0
set to build thethe surrogate model we choose and manually the topremove
𝑥𝑥
Shifted Rotated Rastrigin (F10
constructing
fitnesspoor the model,
function-ranked we choose
points the top 𝑥𝑥 F5 F3 Ackley -330 0
those quality solutions. This in ensuresthe set theto quali- F4 in [34])Griewank 0
fitness
build function-ranked
the surrogate points
model in the manually set to
ty of the samples and allows theand surrogate model to F4
Rotated Hybrid
Griewank
composition
0
build
remove the those surrogate poorof model
quality and manually
solutions. This en- F6 F5 Shifted Rotated 10 -330
maintain a high level simulation performance, F5 functionShifted(F19 inRotated
Rastrigin[34])(F10 in -330
remove
ensures its those poor
themodeling quality
quality ofability the samples solutions. This
and allows
hancing in the region where the Rastrigin[34])(F10 in
ensures
the surrogate the quality model of themaintain
samplesaand highallows
optimal solution may to appear. level of [34])
thesimulation
surrogate model to maintain
performance, enhancing a high level ofits 5.1. Mathematical F6 FunctionRotated Benchmark
Hybrid 10
4.6.
simulation Get Final Optimal
performance, enhancing where itsthe F6 Rotated Hybrid
composition 10
modeling ability in the region Optimization Experiments
modeling ability in the region where the composition
function (F19 in
optimal solution may appear.
Unlike the updating strategy of the global model, we Benchmark functions with
optimal solution may appear. diverse
function characteristics
(F19
[34]) in are
4.6 Get
want Final Optimal
to quickly traverse the local surrogate model employed to assess the effectiveness [34]) of the MLSAO
4.6andGet findFinal Optimal
the possible optimal solution. Therefore, al- algorithm. We include GA [13], RBF-PSO, SA-COSO
Unlike the updating strategy of the global
gorithms
Unlike
model,the
with simple processes,
weupdating
want to quickly strategytraverse ofsuchtheasthe Nelder Mead [35], and
global local
5.1SAHOMathematical
[26] algorithms, Function
along with Benchmark
six prob-
Algorithm,
model, we want are more suitable to use. Compared with lems 5.1
of Mathematical
Optimization
varying dimensions, Function
Experiments to evaluate Benchmark
the perfor-
surrogate modeltoand quicklyfind the traversepossible theoptimal
local
algorithms
surrogate such as DE, it can explore the optimal mance Optimization
of the MLSAO Experiments
algorithm. These benchmark
solution.model and find
Therefore, the possible
algorithms with optimalsimple
value of Therefore,
solution. the current algorithms
target function withwith fewer iter- functions are detailed in Table 1. Each function un-
simple
ations. Similar to the updating of the global model, dergoes optimization thirty times, and the mean value
each iteration generates a new key-value pair. When and standard deviation are calculated. All compared
a new point is added, we will re-queue and continue algorithms are implemented in Python.
to select the top n points, rather than using a sampling
set with an increasing number of points. This is cru- 5.1.1. Experimental Results on 30d Benchmark
cial because a high-quality initial set for the surrogate Problems
model is essential to efficiently identify optimal solu- Table 2 and Figure 3 describe the performance of five
tions that might be challenging for the global model to algorithms on 30-dimensional problems. In the graphs,
uncover. When this step is completed, the algorithm the horizontal axis represents the complexity of the
also ends, and the obtained optimal solution is output. problem, which in this experiment is the number of
 Information Technology and Control 2024/1/53 289

Figure 3
Convergence curves of GA, RBF-PSO, SA-COSO, SAHO, and MLSAO on 30D functions

(a) 30D function1 (b) 30D function2

(c) 30D function3 (d) 30D function4

(e) 30D function5 (f ) 30D function6

RBF-PSO
Table 2
Statistical SA-COSO
RBF-PSO
Table 2 comparisons of results on 30-
290 Information Technology and Control 2024/1/53

Table 2 executions of benchmark test functions. It can be ob-

Statistical comparisons of results on 30-dimensional served that in most test cases, MLSAO outperforms the
benchmark problems other four methods. Among the six functions, MLSAO
achieves the best average performance in four of them.
Function
No
Algorithm Mean Std For function F1, both SA-COSO and SAHO, as well
as MLSAO, can find solutions close to the optimum.
F1 GA 4.4869E+02 1.1733E+02 SAHO performs the best, with MLSAO slightly behind
RBF-PSO 4.6740E+01 8.3591E+00 but significantly outperforming the other three control
functions. The optimization results for F2 show that
SA-COSO 4.3620E+00 2.9579E+00
MLSAO performs significantly better than the oth-
SAHO 1.3392E-01 1.5735E-01 er four algorithms, approaching the optimal solution
and having the lowest standard deviation, indicating
MLSAO 3.1476E-01 2.2098E-01
greater stability during optimization. Regarding the
F2 GA 8.4663E+02 1.7412E+02 Ackley function, MLSAO and SAHO continue to per-
form the best and significantly outperform GA, RBF-
RBF-PSO 1.4769E+02 5.3282E+01
PSO, and SA-COSO. Unlike the results for F1, MLSAO
SA-COSO 5.9851E+01 2.4556E+01 performs slightly better than SAHO in comparison to
SAHO 5.9070E+01 3.0016E+01 the control functions. The optimization results for F4
differ somewhat from the trends observed in the first
MLSAO 2.8445E+01 4.7331E-01 three functions. SAHO performs the best, while ML-
F3 GA 1.6114E+01 1.1471E+00 SAO and SA-COSO achieve optimization results of a
similar magnitude. This suggests that MLSAO still has
RBF-PSO 9.0326E+00 1.0429E+00 room for improvement in this type of problem. For the
SA-COSO 5.0152E+00 1.2214E+00 more complex functions F5 and F6, MLSAO demon-
strates superior performance and better exploration
SAHO 1.9901E+00 6.6895E-01
capabilities to varying degrees. Similar to F3, the opti-
MLSAO 1.4361E+00 6.3830E-01 mization results for F5 show MLSAO slightly outper-
forming SAHO, with both significantly outperforming
F4 GA 1.0908E+02 3.0358E+01
the other three algorithms. However, for the even more
RBF-PSO 1.3664E+01 5.2129E+00 complex F6, MLSAO exhibits a significantly superior
SA-COSO 8.8940E-01 1.1844E-01 ability compared to the other algorithms. In summary,
in the context of 30-dimensional problems, MLSAO
SAHO 1.2759E-01 6.5921E-01 demonstrates a significant advantage. Furthermore,
MLSAO 8.2555E-01 8.9065E-02 GA, which does not utilize surrogate models, lags far
behind the other algorithms, confirming the effective-
F5 GA 1.8780E+02 4.0290E+01 ness of surrogate model-based approaches for solving
RBF-PSO -3.5296E+01 3.7411E+01 complex problems.
SA-COSO -5.7357E+01 1.7545E+01 5.1.2. Experimental Results on 50d Benchmark
SAHO -2.1792E+02 3.7846E+01 Problems
Compared to the results on 30-dimensional bench-
MLSAO -2.3456E+02 2.2514E+01
mark functions, MLSAO performs even better on
F6 GA 1.0605E+03 5.1189E+01 50-dimensional benchmark functions. This is evident
RBF-PSO 9.8923E+02 1.2191E+01
in the results presented in Figure 4 and Table 3, where
MLSAO consistently outperforms the other four al-
SA-COSO 9.7358E+02 2.4022E+01 gorithms, demonstrating superior optimization per-
SAHO 9.8201E+02 3.9012E+01 formance. In all optimization results, MLSAO outper-
forms the other four control algorithms significantly.
MLSAO 9.4091E+02 1.1976E+01 For function F1, MLSAO’s average fitness value is ap-
 Information Technology and Control 2024/1/53 291

Figure 4
Convergence curves of GA, RBF-PSO, SA-COSO, SAHO, and MLSAO on 50D functions

(a) 50D function1 (b) 50D function2

(c) 50D function3 (d) 50D function4

(e) 50D function5 (f ) 50D function6

Table 3
292 Information Technology and Control 2024/1/53

Table 3 proximately 10% better than the second-best-perform-

Statistical comparisons of results on 50-dimensional ing algorithm, surpassing SA-COSO and SAHO by an
benchmark problems order of magnitude, while both SA-COSO and SAHO
Function
outperform RBF-PSO by approximately 10%, and GA,
Algorithm Mean Std the worst-performing algorithm, lags behind MLSAO
No
by nearly three orders of magnitude. The optimization
GA 2.2363E+03 4.3286E+02
results for the Rosenbrock function reveal that ML-
RBF-PSO 5.2168E+02 1.2364E+02 SAO achieves the second-best average optimization
results, which are approximately one-fourth of the
F1 SA-COSO 2.0327E+02 8.1653E+02
SAHO algorithm’s results, and MLSAO exhibits lower
SAHO 5.3468E+01 2.7324E+01 standard deviation. Similar patterns are observed in
MLSAO 4.3287E+00 2.6413e+00 the optimization results for the F3 function. MLSAO’s
average optimization results are roughly one-fourth of
GA 2.1705E+03 7.1463E+02 the second-best algorithm’s results, and MLSAO has
RBF_PSO 6.3548E+02 1.8932E+02 a lower and more stable standard deviation. Among
the other four algorithms, SAHO and SA-COSO per-
F2 SA-COSO 2.8811E+02 8.1744E+01 form similarly, while GA and RBF-PSO have similar
SAHO 1.9358E+02 4.5258E+01 results. MLSAO and SA-COSO perform the best on
the F4 function, with SA-COSO’s average results be-
MLSAO 5.0308E+01 4.0024E+00
ing approximately 5.65 times that of MLSAO. MLSAO
GA 1.5372E+01 3.6429E-01 significantly outperforms the other four algorithms in
RBF-PSO 1.3290E+01 8.4063E-01
this test. For the two more complex functions, in F5,
MLSAO’s average optimization results are improved
F3 SA-COSO 1.2346E+01 1.3853E+00 by 32.7% compared to the second-best algorithm, and
SAHO 8.6818E+00 1.9303E+00 in F6, this improvement is 6.75%. MLSAO also demon-
strates better stability with lower standard deviations
MLSAO 2.1369E+00 9.6370E-01 in both cases. In summary, MLSAO exhibits excellent
GA 4.0389E+02 4.4932E+01 optimization performance on 50-dimensional bench-
mark functions, consistently achieving the best op-
RBF-PSO 9.2733E+01 2.0329E+01
timization performance to varying degrees in each
F4 SA-COSO 3.3176E+01 1.4464E+01 function. This advantage is particularly pronounced in
most functions. Therefore, we consider MLSAO to be
SAHO 5.8982E+00 1.0977E+00
better suited for optimizing problems of mid-dimen-
MLSAO 9.4104E-01 7.8356E-02 sion complexity problems.
GA 4.2994E+02 6.4873E+01 5.1.3. Experimental Results on 100d Benchmark
RBF_PSO 2.7548E+02 5.7724E+01 Problems
F5 SA-COSO 3.1036E+02 5.2617E+01 We attempted to challenge more complex application
scenarios by testing various algorithms’ performance
SAHO 1.8805E+02 2.9981E+01 in solving high-dimensional problems using 100-di-
MLSAO 1.2672E+02 2.8506E+02 mensional benchmark functions. Table 4 displays
the average best fitness function values obtained by
GA 1.1894E+03 4.5169E+01
the five algorithms after 30 independent runs, high-
RBF_PSO 1.0067E+03 3.1456E+01 lighting the best average values for each test function.
MLSAO achieved the best fitness function values in
F6 SA-COSO 1.0818E+03 4.7503E+01
F2, F3, and F5, while SA-COSO performed the best
SAHO 9.9971E+02 2.4517E+01 in F1 and F6, and SAHO performed better in F4. Fig-
MLSAO 9.2985E+02 1.4371E+01 ure 5 shows the fitness changes of each algorithm on
different test functions. Combined with the table,
 SAHO, and MLSAO
Convergence curves of onGA, 100D functions.
RBF-PSO, SA-COSO,
ormance
Figure 5 in solving high-
SAHO, and MLSAO on 100D functions. Figure 5
blems using 100-dimensional
Information Technology and Control 2024/1/53 293
Convergence curves of GA, RBF-PSO, SA-COSO,
ions.SAHO,
Table
Figure 5
4 displays the
and MLSAO on 100D functions. Convergence curv
ess function values obtained
Convergence curves of GA, RBF-PSO, SA-COSO, SAHO, and MLSAO on 100D functions
SAHO, and MLSA
ithms after 30 independent
g the best average values for
n. MLSAO achieved the best
alues in F2, F3, and F5, while
med the best in F1 and F6, and
better in F4. Figure 5 shows
ges of each algorithm on (a) 100D function1 (b) 100D function2

nctions. Combined with the

performed the best in the
eriment for F1, but MLSAO
not far behind, with all three
e same level. MLSAO
significant advantage in the (c) 100D function3 (d) 100D function4
F2 function test, with an
tion result far higher than the
ns and the smallest standard
timization results for the F3
vored MLSAO, where the
the second-best SAHO was
5 times that of the best result.
(e) 100D function5 (f ) 100D function6

e other four algorithms were

The best optimization result
5.1.3 Experimental Results on 200d
ensional Griewank function
294 Information Technology and Control 2024/1/53

SA-COSO performed the best in the optimization Table 4

experiment for F1, but MLSAO and SAHO were not Statistical comparisons of results on 100-dimensional
far behind, with all three almost on the same level. benchmark problems
MLSAO demonstrated a significant advantage in the Function
100-dimensional F2 function test, with an average Algorithm Mean Std
No
optimization result far higher than the other four
F1 GA 1.2766E+04 1.5483E+03
functions and the smallest standard deviation. The
optimization results for the F3 function also favored RBF_PSO 8.2347E+03 9.0105E+02
MLSAO, where the average result for the second-best SA-COSO 1.3224E+03 3.0408E+02
SAHO was approximately 2.5 times that of the best
result. The results for the other four algorithms were SHAO 1.3861E+03 1.4210E+02
relatively similar. The best optimization result for MLSAO 1.3724E+03 2.3379E+01
the 100-dimensional Griewank function belonged to
F2 GA 1.0426E+04 1.0697E+03
SAHO. The difference between these two algorithms
and the other three was particularly pronounced. In RBF_PSO 4.1645E+03 5.9803E+02
the experiment for F5, MLSAO performed the best,
SA-COSO 4.0199E+02 1.8129E+02
followed by SHAO, with both achieving average re-
sults below 1000, while the results for the other three SAHO 5.7284E+02 4.3681E+01
algorithms were all above 1000. The experimental re- MLSAO 1.7192E+02 2.7322E+01
sults for the F6 function showed that the average re-
sults for all five algorithms were generally within the F3 GA 1.9894E+01 4.2056E-01
same range. However, SA-COSO performed the best, RBF_PSO 1.9021E+01 4.3126E-01
while MLSAO ranked only third, with its performance
SA-COSO 1.2127E+01 8.9256E-01
being less satisfactory. Overall, MLSAO achieved al-
most the best fitness function values or equivalent SAHO 1.5396E+01 4.8945E-01
levels in most functions. Even in cases where MLSAO MLSAO 6.1134E+00 6.0183E-01
did not achieve the optimal solution, it still managed
to come very close to an approximate optimal result F4 GA 1.1684E+03 1.0013E+02
to a large extent. However, in F6, the surrogate mod- RBF_PSO 6.1963E+02 5.7765E+01
el may have difficulty providing useful information,
SA-COSO 6.1253E+01 2.0021E+01
leading to the algorithm being trapped in a local opti-
mum and unable to discover the true global optimum. SAHO 9.3503E-01 1.0853E-01
Overall, MLSAO still performed well on most 100-di-
MLSAO 1.9258E+00 2.0905E-01
mensional problems, demonstrating its ability to han-
dle high-dimensional complex problems and proving F5 GA 2.1895E+03 2.5614E+02
its suitability for high-dimensional scenario. RBF_PSO 1.5842E+03 2.1104E+02
5.1.3. Experimental Results on 200d Benchmark SA-COSO 1.2561E+03 1.1033E+02
Problems SAHO 9.7908E+02 7.9437E+01
To further assess the capability of MLSAO in address-
MLSAO 8.1588E+02 8.9033E+01
ing high-dimensional and complex problems, we con-
ducted tests with several benchmark functions se- F6 GA 1.5017E+03 3.1302E+01
lected from Table 1, using a parameter dimension
RBF_PSO 1.4035E+03 4.7821E+01
of 200. We compared MLSAO with SAHSO[19]and
GL-SADE [39], and the results are shown in Table 5. SA-COSO 1.2677E+03 2.5304E+01
Despite suboptimal performance in F1, the outcomes SAHO 1.3793E+03 1.1042E+02
for F2 and F4 indicate that MLSAO remains the al-
gorithm with the strongest optimization capability. MLSAO 1.3998E+03 3.9211E+01
 Information Technology and Controlprocesses, large-scale
2024/1/53 circulation, 295 Parame
microphysical processes, boundary layer name
processes, etc. These schemes contain some
adjustable parameters, which can be tuned to zmconv_
Table 5 parameterization schemes, include cumulus convec-
Statistical comparisons of results on 200-dimensional change tion
the simulation result of CAM. The
processes, large-scale circulation, microphysi-
pdz
benchmark problems computational and time cost generated by
cal processes, boundary layer processes, etc. These
traditional algorithms are unacceptable during
Function
schemes contain some adjustable parameters, which
Algorithm Mean Std the optimization process. Therefore, we
can be tuned to change the simulation result of CAM.
No
applied the proposed algorithm to address the zmconv_
The computational and time cost generated by tradi-
F1 SAHSO 8.5726E+02 CESM parameter tuning problem. The results
9..8398E+01
ocn
tional algorithms are unacceptable during the opti-
GL-SADE 5.1297E+02 demonstrate that the proposed algorithm
8.9146E+01 mization process. Therefore, we applied the proposed
effectively enhanced the precipitation
MLSAO 2.7524E+03 1.8329E+02 algorithm to address the CESM parameter tuning
simulation results of the Community
problem. The results demonstrate that the proposed
F2 SAHSO 9.1682E+02 Atmosphere Model (CAM) in several regions
1.7957E+02
algorithm effectively enhanced the precipitation sim-
with a relatively small number of iterations.
GL-SADE 7.3085E+02 5.9803E+01 ulation results of the Community Atmosphere Model
In this (CAM)
work, in we use the
several proposed
regions MLSAO small num-
with a relatively
MLSAO 4.3583E+02 5.6634E+01
algorithm
ber to
of optimize
iterations.the precipitation related zmconv_
F3 SAHSO 7.5684E+00 7.3696E-01
parameters of CAM, so the
asproposed
to make the algorithm
In this work, we use MLSAO
GL-SADE 2.0751E+01 optimization
3.0311E-01 results of the CAM model more
to optimize the precipitation related parameters of
close toCAM,
the observation
so as to make data.
the This algorithm
optimization results of the
MLSAO 8.7712E+00 8.9768E-01
can greatly reduce the optimization
CAM model more close to the observationtime by data. This
F4 SAHSO 2.7841E+00 building
3.4071E-01 a surrogate
algorithm modelreduce
can greatly to replace the
the optimization time
GL-SADE 3.9983E+00 complicated
5.0702E-01 CAM model running process.
by building a surrogate model to replace the compli-
Therefore,
catedweCAMmustmodel
account for theprocess.
running disparities
Therefore, we
MLSAO 2.6304E+00 3.0015E-01
between must account for the disparities and
the model simulation results the the model
between
observational dataresults
in the objective function. To data in the The optimizat
simulation and the observational
guide the algorithm towards
In comparison with SAHSO, F3 shows only a slight objective function. To guide the algorithm optimizing by towards the numerical
minimizing these disparities, we have
disadvantage than SAHSO. Overall, compared to A optimizing by minimizing these disparities, we have chosen selected region
and B, MLSAO still achieves relatively better optimi- the root-mean-square error (RMSE)error
chosen the root-mean-square as (RMSE)
the as the default expe
zation results in the majority of the experimentalobjective function.
op- objective The RMSE
function. The is calculated
RMSE as
is calculated as fol- optimized exp
timization functions. These results demonstrate follows:
that lows: observed data.
MLSAO still possesses sufficient optimization prow- decreased by
ess when dealing with high-dimensional and complex 1
�
To demonstra
computational problems. RMSE = � �(mod� − obs� )� ,,
N indeed improv
���
5.2. CESM Parameter Tuning Experiments simulation, w
where where
𝑁𝑁 is the
N istotal number
the total number of of the
thegrid
grid points of sim- plots for each
The Community Earth System Model (CESM) points [18] of simulation region,𝑚𝑚𝑚𝑚𝑚𝑚 and 𝑜𝑜𝑜𝑜𝑜𝑜model
is a fully coupled global climate model that offersare ulation region, mod and obs
ad- the model simulated and observation
i
�
i are the � simulated precipitation
vanced computer simulations of Earth’s historical, and observation data values at grid point i. A small- optimized p
data values
er at grid
RMSE point 𝑖𝑖. aA smaller
indicates smaller RMSE
error between model
current, and future climate conditions. The commu- proposed algo
indicates a smaller
simulation and error betweendata.
observational model The optimization
nity atmosphere model (CAM) is the atmosphere simulation and observational data.for The Indian Ocean,
component of the community earth system model. It goal isgoal
optimization
to minimize
is to
the RMSE
minimize the RMSE
each region. The
in Figures 6
simulates various chemical reactions and energyfor ex-each region. The parameters are shownthe selected re-
parameters are shown in Table 6, and
introduce the c
change in the atmosphere. CESM simulation experi- gions are presented in Table 7. These parameters are
in Table 6, and the theselected regions are to the default p
that chosenfrom
ence is a high-performance computing program presented in Table
Zhang-McFarlane
7. These parameters are
parameterization
requires a significant amount of time and computa- scheme [51].
chosenfrom the Zhang-McFarlane
tional cost. A complete simulation process of CESM The optimization results are shown in Figure 6, the Table 7
parameterization scheme [51].
usually takes more than ten hours or even several numerical results indicate that the RMSE of selected
days. regions has decreased compared to the default exper- Regions selecte
Nam
The precipitation process in CAM is complicatedTableand 6 iment, indicating that the optimized experimental re-
precipitation-related processes is related to many sults are closer to the observed data. The RMSE for
Parameters description of Zhang-
McFarlane parameterization scheme. CAPE
means the convective available potential
energy.
296 Information Technology and Control 2024/1/53

Table 6
Parameters description of Zhang-McFarlane parameterization scheme. CAPE means the convective available potential
energy

Parameter name Meaning Range Default value

zmconv_dmpdz Parcel fractional mass entrainment rate –2.0 × 10 ~ –0.2 × 10

–3 –3
–1.0 × 10–3
zmconv_c0_ocn Deep convection precipitation efficiency over ocean 1.0 × 10–3 ~ 0.1 0.045
zmconv_tau Time scale for consumption rate deep CAPE 1800~28800 3600

Table 7 The improvement in the North Indian Ocean region

Regions selected in this study and their range is quite evident, as shown in the Figure 7. In the de-
fault experiment, it is noticeable that the precipita-
Name Gegion
tion in the ocean areas on both sides of the Indian
North Pacific Peninsula is considerably higher than the observed
15°-40°N, 150°-210°E
Ocean data, resulting in a significant positive error. In con-
North India 0°-20°N, 50°-90°E trast, in the optimized experiment, these errors have
been significantly reduced. This difference is espe-
North Pacific 15°-40°N, cially prominent on the westernhave
improvements side ofeffectively
the peninsu-reduced most
the two regions has decreased by150°-210°E
29.77% and 12.63%,
la, where precipitation values now align more close-
Ocean
respectively. To demonstrate that the decrease of
ly with thosethese positive
observed. errorserror
The positive without introducing ne
areas on
RMSE has indeed improved the accuracy of precipi-
eastern side oferrors.
the peninsula have also been signifi-
tation simulation, we draw precipitation distribution
cantly reduced, with some remain ing errors near the
plots for each region, detailing the changes in precip- Figure 7
North island of SriLanka. Furthermore, near the Arabian
itation over India 0°-20°N,
these regions 50°-90°E
caused by optimized pa-
Peninsula, the The
results of the optimized
precipitation experiment
distribution of the North India
rameters obtained from the proposed algorithm. The
are also superior to the default
optimization experiment.
result. The displays the
The left column
results for the North Indian Ocean, and the North
optimized parameters have reduced precipitation
default experiment, observation data, and the
Pacific are shown in Figures 6-7, respectively. We will
in this region, even eliminating
difference betweenthethe
positive errors
default experiment and th
now introduce the changes in precipitation compared
Figure 6 that were presentobservation data from top to In
in the default experiment. the The right
bottom.
to the default parameters.
southeastern marine area, unlike the improvement
column exhibits the optimal experiment,
RMSE of default experience and optimization seen in other regions, the optimized experiment has between the
observation data, and the difference
results
Figure 6over each region . increased precipitation
RMSE of default experience and optimization results over optimal in this area, bringing
experiment and the the re-
observation data from
sults closer to the observed
top to bottom. data. In summary, in the
each region
North Indian Ocean region, the default experiment
had significant positive errors. Our improvements
have effectively reduced most of these positive er-
rors without introducing new errors.
Figure 8 illustrates the optimization results for the
North Pacific region. Similar to the North Indian
Ocean region, the default experiment still exhibits
a significant amount of positive errors. Our optimi-
zation results aim to reduce these positive errors in
the North Pacific region. As shown in the figure, our
optimization results have substantially reduced pre-
cipitation in these ocean areas, primarily in the cen-
tral region of the selected area. Precipitation in this
central area has been significantly reduced and is now
very close to the results from observed data. The re-

The improvement in the North Indian Ocean

region is quite evident, as shown in the Figure 7.
In the default experiment, it is noticeable that the
precipitation in the ocean areas on both sides of
the Indian Peninsula is considerably higher than
 optimization result. The left column displays the
default experiment, observation data, and the
difference between the default experiment and
Information Technology and Control
the
2024/1/53 297

observation data from top to bottom. The right

column
Figure 7 exhibits the optimal experiment,
observation data, and the difference between the
The precipitation distribution of the North India optimization result. The left column displays the default experiment,
observation data, and the difference between the default experiment and the observation data from top to bottom. The
rightoptimal experiment
column exhibits andobservation
the optimal experiment, the observation data
data, and the difference from
between the optimal experiment and
the observation data from top to bottom
top to bottom.

Figure 8 illustrates the optimization results for

sults for the eastern and western sides of the region
derestimated precipitation, while the optimized ex-
the North Pacific region. Similar toincreased
the North
have also seen considerable improvements. Addition-
periment precipitation in this area, elimi-
ally, in the southeastern areas around the islands, pre-
nating negative errors. It is evident that the optimized
Indian Ocean region, the default experiment
cipitation has noticeably decreased. Furthermore, in
parameters can enhance the capabilities of CAM in
still exhibits a significant amount of positive
the northwestern region, the default experiment un-
the North Pacific region.

errors. Our optimization results aim to reduce

these positive errors in the North Pacific
region. As shown in the figure, our
optimization results have substantially
 observation data from top to bottom. The right
column exhibits the optimal experiment,
298 Information Technology and Control 2024/1/53

observation data, and the difference between the

optimal
Figure 8
experiment and the observation data from
The precipitation distribution of the North Pacific optimization result. The left column displays the default experiment,

topobservation
to
right bottom.
column
data, and the difference between the default experiment and the observation data from top to bottom. The
exhibits the optimal experiment, observation data, and the difference between the optimal experiment and
the observation data from top to bottom

6. Conclusions introduces a multi-level search mechanism that en-

hances its exploration capability of the parameter
In this work, we propose a multi-level surrogate-as- space, thereby mitigating the risk of falling into local
sisted optimization algorithm MLSAO, primari- optima to some extent. The proposed MLSAO inte-
ly designed to address the parameter optimization grates random forest and DE to construct and update
challenges associated with complex models. MLSAO the global level surrogate model, respectively. For lo-

6. Conclusions
In this work, we propose a multi-level
 Information Technology and Control 2024/1/53 299

cal level, we select IDW to build the surrogate model ering the introduction of parallel and multi-threading
and simplex downhill method to update this surro- techniques to enhance the efficiency and utilization
gate model. Whether it is the optimization tests on of computational resources for MLSAO.
benchmarks or the practical application of CESM pa-
rameter tuning, MLSAO has demonstrated excellent Competing Interests
optimization capabilities. The authors have no relevant financial or non-finan-
In future work, we aim to broaden the application cial interests to disclose.
scope of MLSAO, enhance its capability to tackle
multi-objective optimization, and facilitate its uti- Funding
lization in a wider range of domains to address re- This work is funded by: National Key R&D Plan of
al-world optimization challenges. Additionally, as China under Grant No. 2017YFA0604500, and by Key
data scales continue to increase, we aim to improve scientific and technological R&D Plan of Jilin Prov-
its optimization efficiency. Moreover, we are consid- ince of China under Grant No. 20180201103GX.

References
1. Alizadeh, R., Allen, J. K., Mistree, F. Managing Compu- 8. Dalgaty, T., Castellani, N., Turck, C., Harabi, K.-E., Qu-
tational Complexity Using Surrogate Models: A Critical erlioz, D., Vianello, E. In Situ Learning Using Intrinsic
Review. Research in Engineering Design, 2020, 31, 275- Memristor Variability via Markov Chain Monte Car-
298. https://doi.org/10.1007/s00163-020-00336-7 lo Sampling. Nature Electronics, 2021, 4(2), 151-161.
https://doi.org/10.1038/s41928-020-00523-3
2. Anderson, G. J., Lucas, D. D. Machine Learning Predic-
tions of a Multiresolution Climate Model Ensemble. 9. Diaz-Manriquez, A., Toscano, G., Coello Coello, C. A.
Geophysical Research Letters, 2018, 45, 4273-4280. Comparison of Metamodeling Techniques in Evolutio-
https://doi.org/10.1029/2018GL077049 nary Algorithms. Soft Computing, 2017, 21, 5647-5663.
https://doi.org/10.1007/s00500-016-2140-z
3. Aria, M., Cuccurullo, C., Gnasso, A. A Comparison
Among Interpretative Proposals for Random Forests. 10. Fan, F. L., Xiong, J. J., Li, M. Z., Wamh, G. On Interpretability
Machine Learning with Applications, 2021, 6, 100094. of Artificial Neural Networks: A Survey. IEEE Transacti-
https://doi.org/10.1016/j.mlwa.2021.100094 ons on Radiation and Plasma Medical Sciences, 2021, 5(6):
741-760. https://doi.org/10.1109/TRPMS.2021.3066428
4. Bemporad, A. Active Learning for Regression by Inver-
se Distance Weighting. Information Sciences, 2023, 11. Fuhg, J. N., Fau, A., Nackenhorst, U. State-of-the-Art
626, 275-292. https://doi.org/10.1016/j.ins.2023.01.028 and Comparative Review of Adaptive Sampling Me-
thods for Kriging. Archives of Computational Me-
5. Bouhlel, M. A., Hwang, J. T., Bartoli, N., Lafage, R., Mor- thods in Engineering, 2021, 28: 2689-2747. https://doi.
lier, J., Martins, J. R. R. A. A Python Surrogate Modeling org/10.1007/s11831-020-09474-6
Framework with Derivatives. Advances in Engineering
Software, 2019, p. 102662. https://doi.org/10.1016/j. 12. Golzari, A., Haghighat Sefat, M., Jamshidi, S. Develop-
advengsoft.2019.03.005 ment of an Adaptive Surrogate Model for Production
Optimization. Journal of Petroleum Science and Engi-
6. Briffoteaux, G., Gobert, M., Ragonnet, R., Gmys, J., neering, 2015, 133, 677-688. https://doi.org/10.1016/j.
Mezmaz, M., Melab, N., Tuyttens, D. Parallel Surroga- petrol.2015.07.012
te-assisted Optimization: Batched Bayesian Neural
Network-assisted GA versus q-EGO. Swarm and Evo- 13. Holland, J. H. Genetic Algorithms. Scientific American,
lutionary Computation, 2020, 57, 100717.https://doi. 1992, 267, 66-73. https://doi.org/10.1038/scientificame-
org/10.1016/j.swevo.2020.100717 rican0792-66

7. Chu, L., Shi, J., Souza de Cursi, E. Kriging Surrogate 14. Hwang, J. T. A Fast-Prediction Surrogate Model for
Large Datasets. Aerospace Science and Technology,
Model for Resonance Frequency Analysis of Dental Im-
2018. https://doi.org/10.1016/j.ast.2017.12.030
plants by a Latin Hypercube-Based Finite Element Me-
thod. Applied Bionics and Biomechanics, 2019, 2019. 15. Jiang, K., Wang, J., Chen, Z., Zhou, Q., Jiang, S.-H. Influ-
https://doi.org/10.1155/2019/3768695 ence of Coupling Factors on Structural Reliability Ba-
300 Information Technology and Control 2024/1/53

sed on Polynomial Response Surface Optimization Mo- of the National Academy of Sciences, 2010, 107, 21349-
del. Advances in Mechanical Engineering, 2023, 15(8). 21354. https://doi.org/10.1073/pnas.1015473107
https://doi.org/10.1177/16878132231184145 25. Nelder, J. A., Mead, R. A Simplex Method for Function
16. Jie, H., Wu, Y., Zhao, J., Ding, J., Liangliang. An Effici- Minimization. The Computer Journal, 1965, 7, 308-313.
ent Multi-Objective PSO Algorithm Assisted by Kriging https://doi.org/10.1093/comjnl/7.4.308
Metamodel for Expensive Black-Box Problems. Journal 26. Pan, J. S., Liu, N., Chu, S. C., Lai, T. An Efficient Sur-
of Global Optimization, 2017, 67, 399-423. https://doi. rogate-Assisted Hybrid Optimization Algorithm for
org/10.1007/s10898-016-0428-2 Expensive Optimization Problems. Information Sci-
17. Jones, D. R. A Taxonomy of Global Optimization Me- ences, 2021, 561, 304-325. https://doi.org/10.1016/j.
thods Based on Response Surfaces. Journal of Glo- ins.2020.11.056
bal Optimization, 2001, 21, 345-383. https://doi.or- 27. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel,
g/10.1023/A:1012771025575 V., Thirion, B., Grisel, O., Blondel, M., Louppe, G.,
18. Kay, J. E., Deser, C., Phillips, A., Mai, A., Hannay, C., Prettenhofer, P., Weiss, R., Weiss, R. J., Vanderplas,
Strand, G., Arblaster, J. M., Bates, S. C., Danabasoglu, J., Passos, A., Cournapeau, D., Brucher, M., Perrot,
G., Edwards, J., Holland, M., Kushner, P., Lamarque, J.- M., Duchesnay, E. Scikit-Learn: Machine Learning
F. ., Lawrence, D., Lindsay, K., Middleton, A., Munoz, E., in Python. J. Mach. Learn. Res, 2011. https://doi.
Neale, R., Oleson, K., Polvani, L. The Community Ear- org/10.5555/1953048.2078195.
th System Model (CESM) Large Ensemble Project: A 28. Rasamoelina, A. D., Adjailia, F., Sincak, P. A Review of
Community Resource for Studying Climate Change in Activation Function for Artificial Neural Network.
the Presence of Internal Climate Variability. Bulletin of 2020 IEEE 18th World Symposium on Applied Machi-
the American Meteorological Society, 2015, 96(8), 1333- ne Intelligence and Informatics (SAMI), 2020. https://
1349. https://doi.org/10.1175/BAMS-D-13-00255.1 doi.org/10.1109/SAMI48414.2020.9108717
19. Li, F., Li, Y., Cai, X. Gao, L. A Surrogate-Assisted Hybrid 29. Regis, R. G., Shoemaker, C. A. A Stochastic Radial Basis
Swarm Optimization Algorithm for High-Dimensional Function Method for the Global Optimization of Expen-
Computationally Expensive Problems. Swarm and Evo- sive Functions. INFORMS Journal on Computing, 2007,
lutionary Computation, 2022, 72, 101096. https://doi. 19, 497-509. https://doi.org/10.1287/ijoc.1060.0182
org/10.1016/j.swevo.2022.101096
30. Saurette, Daniel D., Asim Biswas, Adam W. Gillespie.
20. Liu, Y., Liu, J., Ding, J., Yang, S., Jin, Y. A Surrogate-As- Determining Minimum Sample Size for the Conditio-
sisted Differential Evolution with Knowledge Transfer ned Latin Hypercube Sampling Algorithm. Pedosphere,
for Expensive Incremental Optimization Problems. 2022. https://doi.org/10.1016/j.pedsph.2022.09.001
IEEE Transactions on Evolutionary Computation,
2023, 1-1. https://doi.org/10.1109/TEVC.2023.3291697 31. Schonlau, M., Welch, W. J., Jones, D. R. Global Versus
Local Search in Constrained Optimization of Computer
21. Liu, Y., Liu, J., Tan, S., Yang, Y., Li, F. A Bagging-Based Models. Lecture Notes-Monograph Series, 1998, pp. 11-
Surrogate-Assisted Evolutionary Algorithm for Expen- 25. https://doi.org/10.1214/lnms/1215456182
sive Multi-Objective Optimization. Neural Computing
and Applications, 2022, 34, 12097-12118. https://doi. 32. Shi, R., Liu, L., Long, T., Wu, Y., Wang, G. G. Multidisci-
org/10.1007/s00521-022-07097-5 plinary Modeling and Surrogate Assisted Optimizati-
on for Satellite Constellation Systems. Structural and
22. Li, Y., Zhong, J. HAS-EA: A Fast Parallel Surrogate-As- Multidisciplinary Optimization, 2018, 58, 2173-2188.
sisted Evolutionary Algorithm. Memetic Computing, https://doi.org/10.1007/s00158-018-2032-1
2022. https://doi.org/10.1007/s12293-022-00376-7
33. Storn, R., Price, K. Differential Evolution-A Simple and
23. Müller, J., Paudel, R., Shoemaker, C. A., Woodbury, J., Efficient Heuristic for Global Optimization over Conti-
Wang, Y. Mahowald, N. M. CH4 Parameter Estimation nuous Spaces. Journal of Global Optimization, 1997, 11,
in CLM4.5bgc Using Surrogate Global Optimization. 341. https://doi.org/10.1023/A:1008202821328
Geoscientific Model Development, 2015, 8(10), 3285-
3310. https://doi.org/10.5194/gmd-8-3285-2015 34. Suganthan, P. N., Hansen, N., Liang, J. J., Deb, K., Chen,
Y. P., Auger, A., Tiwari, S. Problem Definitions and Eva-
24. Neelin, J. D., Bracco, A., Luo, H., McWilliams, J. C., luation Criteria for the CEC 2005 Special Session on
Meyerson, J. E. Considerations for Parameter Optimi- Real-Parameter Optimization. KanGAL Report, 2005,
zation and Sensitivity in Climate Models. Proceedings 2005005.
 Information Technology and Control 2024/1/53 301

35. Sun, C., Jin, Y., Cheng, R., Ding, J., Zeng, J. Surrogate-As- 43. Xing, J., Luo, Y., Gao, Z. A Global Optimization Strate-
sisted Cooperative Swarm Optimization of High-Di- gy Based on the Kriging Surrogate Model and Parallel
mensional Expensive Problems. IEEE Transactions on Computing. Structural and Multidisciplinary Optimi-
Evolutionary Computation, 2017, 21, 644-660. https:// zation, 2020, 62(1), 405-417. https://doi.org/10.1007/
doi.org/10.1109/TEVC.2017.2675628 s00158-020-02495-6
36. Sun, C., Jin, Y., Zeng, J., Yu, Y. A Two-Layer Surro- 44. Xu, H., Zhang, T., Luo, Y., Huang, X., Xue, W. Parameter
gate-Assisted Particle Swarm Optimization Algori- Calibration in Global Soil Carbon Models Using Surro-
thm. Soft Computing, 2015, 19, 1461-1475. https://doi. gate-Based Optimization. Geoscientific Model Deve-
org/10.1007/s00500-014-1283-z lopment, 2018, 11, 3027-3044. https://doi.org/10.5194/
37. Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, gmd-11-3027-2018
M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, 45. Yan, C., Yin, Z., Shen, X., Mi, D., Guo, F., Long, D. Surro-
P., Weckesser, W., Bright, J., van der Walt, S. J., Brett, gate-Based Optimization with Improved Support Vector
M., Wilson, J., Millman, K. J., Mayorov, N., Nelson, A. R. Regression for Non-Circular Vent Hole on Aero-Engine
J., Jones, E., Kern, R., Larson, E., Carey, C. J.SciPy 1.0: Turbine Disk. Aerospace Science and Technology, 2020,
Fundamental Algorithms for Scientific Computing in 96, 105332. https://doi.org/10.1016/j.ast.2019.105332
Python. Nature Methods, 2020, 17(3), 261-272. https://
doi.org/10.1038/s41592-019-0686-2 46. You, X., Li, W., Chai, Y. A Truly Meshfree Method for
Solving Acoustic Problems Using Local Weak Form
38. Wang, C., Duan, Q., Gong, W., Ye, A., Di, Z., Miao, C. An and Radial Basis Functions. Applied Mathematics
Evaluation of Adaptive Surrogate Modeling Based Opti- and Computation, 2020, 365, 124694. https://doi.or-
mization with Two Benchmark Problems. Environmen- g/10.1016/j.amc.2019.124694
tal Modelling & Software, 2014, 60, 167-179. https://doi.
org/10.1016/j.envsoft.2014.05.026 47. Yu, H., Tan, Y., Zeng, J., Sun, C., Jin, Y. Surrogate-As-
sisted Hierarchical Particle Swarm Optimization. In-
39. Wang, W., Liu, H. L., Tan, K. C. A Surrogate-Assisted Di- formation Sciences, 2018, 454-455, 59-72. https://doi.
fferential Evolution Algorithm for High-Dimensional org/10.1016/j.ins.2018.04.062
Expensive Optimization Problems. IEEE Transactions
on Cybernetics, 2023, 53(4), 2685-2697. https://doi. 48. Shang, Y., Nogal, M., Teixeira, R., Wolfert, A. R. (Rogier)
org/10.1109/TCYB.2022.3175533 M. Optimal Design of Rail Level Crossings and Associ-
ated Transition Zones Using Adaptive Surrogate-As-
40. Wang, X., Wang, G. G., Song, B., Wang, P., Wang, Y. A No- sisted Optimization. Engineering Structures, 2023, 282,
vel Evolutionary Sampling Assisted Optimization Me- 115740. https://doi.org/10.1016/j.engstruct.2023.115740
thod for Highdimensional Expensive Problems. IEEE
Transactions on Evolutionary Computation, 2019, 23, 49. Zhang, C., Di, Z., Duan, Q., Xie, Z., Gong, W. Improved
815-827. https://doi.org/10.1109/TEVC.2019.2890818 Land Evapotranspiration Simulation of the Commu-
nity Land Model Using a Surrogate-Based Automatic
41. Williams, B., Cremaschi, S. Selection of Surrogate Parameter Optimization Method. Water, 2020, 12, 943.
Modeling Techniques for Surface Approximation and https://doi.org/10.3390/w12040943
Surrogate-Based Optimization. Chemical Engineering
Research and Design, 2021, 170, 76-89. https://doi.or- 50. Zhang, F., Lauren, J. O‘Donnell. Support Vector Regres-
g/10.1016/j.cherd.2021.03.028 sion. Machine Learning. Academic Press, 2020: 123-140.
https://doi.org/10.1016/B978-0-12-815739-8.00007-9
42. Ji, X., Zhang, Y., He, C., Cheng, J., Gong, D., Gao, X., Guo,
Y. Surrogate and Autoencoder-Assisted Multitask 51. Zhang, G. J., McFarlane, N. A. Sensitivity of Climate
Particle Swarm Optimization for High-Dimensional Simulations to the Parameterization of Cumulus Con-
Expensive Multimodal Problems. IEEE Transactions vection in the Canadian Climate Centre General Cir-
on Evolutionary Computation, 2023, 1-1. https://doi. culation Model. Atmosphere-Ocean, 1995, 33, 407-446.
org/10.1109/TEVC.2023.3287213 https://doi.org/10.1080/07055900.1995.9649539

This article is an Open Access article distributed under the terms and conditions of the Creative
Commons Attribution 4.0 (CC BY 4.0) License (http://creativecommons.org/licenses/by/4.0/).