See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/227050705

An efficient multiobjective differential evolution algorithm for engineering

design

Article in Structural and Multidisciplinary Optimization · April 2009

DOI: 10.1007/s00158-008-0269-9

CITATIONS READS

87 2,207

3 authors, including:

Wenyin Gong Zhihua Cai

96 PUBLICATIONS 2,939 CITATIONS

China University of Geosciences
263 PUBLICATIONS 7,211 CITATIONS
SEE PROFILE
SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Multi-objective optimization, logistics, vehicle routing problem View project

Cooperative Co-evolution Framework: CCFR View project

All content following this page was uploaded by Wenyin Gong on 08 April 2014.

The user has requested enhancement of the downloaded file.

Struct Multidisc Optim manuscript No.
(will be inserted by the editor)

An Efficient Multiobjective Differential Evolution Algorithm for

Engineering Design
Wenyin Gong · Zhihua Cai · Li Zhu

Received: date / Accepted: date

Abstract Solving engineering design and resources opti- Keywords Engineering design · multiobjective optimiza-
mization via multiobjective evolutionary algorithms (MOEAs) tion · differential evolution algorithm · orthogonal design
has attracted much attention in the last few years. In this method · Pareto-adaptive -dominance · constraint-handling
paper, an efficient multiobjective differential evolution al- method
gorithm is presented for engineering design. Our proposed
approach adopts the orthogonal design method with quan-
tization technique to generate the initial archive and evolu-
tionary population. An archive (or secondary population) is 1 Introduction
employed to keep the nondominated solutions found and it is
updated by a new relaxed form of Pareto dominance, called Many real-world engineering design or resources optimiza-
Pareto-adaptive -dominance (pa-dominance), at each gen- tion involve simultaneous optimization of multiple objec-
eration. In addition, in order to guarantee to be the best per- tives. Usually, these objectives stay in conflict with each
formance produced, we propose a new hybrid selection mech- other. As an example, in the design of an automobile an
anism to allow the archive solutions to take part in the gen- engineer may wish to maximize crash resistance for safety
erating process. To handle the constraints, a new constraint- and minimize weight for fuel economy. Instead of finding a
handling method is employed, which does not need any pa- single solution, the multiobjective optimization methods try
rameters to be tuned for constraint handling. The proposed to produce a set of good trade-off solutions called the non-
approach is tested on seven benchmark constrained prob- dominated solutions or Pareto-optimal solutions from which
lems to illustrate the capabilities of the algorithm in han- the decision maker may select one.
dling mathematically complex problems. Furthermore, four To deal with the multiobjective optimization problems
well-studied engineering design optimization problems are (MOPs), many mathematical programming techniques are
solved to illustrate the efficiency and applicability of the al- developed since the 1950s [4]. However, mathematical pro-
gorithm for multiobjective design optimization. Compared gramming techniques have certain limitations when tackling
with NSGA-II, one of the best MOEAs available at present, MOPs, such as (i) most of them can not find multiple solu-
the results demonstrate that our approach is found to be sta- tions in a single run, (ii) many of them are susceptible to the
tistically competitive. Moreover, the proposed approach is shape of the Pareto front and may not work when the Pareto
very efficient and is capable of yielding a wide spread of so- front is concave or disconnected, and (iii) multiple applica-
lutions with good coverage and convergence to true Pareto- tion of these methods do not guarantee finding widely differ-
optimal fronts. ent Pareto-optimal solutions. On the contrary, evolutionary
algorithms (EAs) deal simultaneously with a set of possible
solutions (the so-called population) which allows us to find
W.Y. Gong · Z.H. Cai · L. Zhu several members of the Pareto-optimal set in a single run
School of Computer Science, China University of Geosciences, Wuhan
430074, China
of the algorithm. Additionally, EAs are less susceptible to
Tel.: +86-15926402726 the shape or continuity of the Pareto front. Also, the use of
Fax: +86-27-67883716 diversity-preserving mechanisms can be added to the EAs to
E-mail: [email protected] find widely different Pareto-optimal solutions.
2

Since the 1980s, several multiobjective evolutionary al- at each generation. In order to handle the constraints, a new
gorithms (MOEAs) have been proposed and applied in MOPs, constraint-handling method is employed, which does not need
such as NSGA-II [8], SPEA2 [32], DEMO [23], etc. Re- any parameters to be tuned for constraint handling. Our pro-
cently, many researchers adopted the MOEAs to solve en- posed approach is validated using seven benchmark con-
gineering design problems [5], [12], [15], [20], and so on. strained MOPs and four engineering optimization problems
These algorithms share the same purpose - searching for a taken from the specialized literature, and its performance
uniformly distributed, near-optimal, and near-complete Pareto is compared against the Nondominated Sorting Genetic Al-
front for a given MOP. However, this ultimate goal is far gorithm II (NSGA-II) [8]. The results indicate that our ap-
from being accomplished by the existing MOEAs described proach is a viable alternative to efficient multiobjective op-
in literature. In one respect, most of the MOPs are very com- timization.
plicated and require the computational resources to be ho- The remainder of this paper is organized as follows. In
mogenously distributed in a high-dimensional search space. Section 2, we give the problem formulation of MOPs. The
On the other hand, those better-fit individuals generally have related work is introduced in Section 3. In Section 4, the DE
strong tendencies to restrict searching efforts within local algorithm is briefly described. Section 5 proposes a novel
areas because of the “genetic drift” phenomenon, which re- pa-ODEMO to deal with MOPs and describes its main com-
sults in the loss of diversity due to stochastic sampling. ponents in detail. In Section 6, we test our algorithm through
Differential evolution (DE) [26] algorithm is a novel EA a number of benchmark constrained MOPs and engineer-
for faster optimization, which mutation operator is based ing design problems. In addition, the experiment results are
on the distribution of solutions in the population. And DE compared with NSGA-II. The last section, Section 7, is de-
has won the third place at the first International Contest on voted to conclusions.
Evolutionary Computation on a real-valued function test-
suite. Unlike Genetic Algorithm (GA) that uses binary cod-
ing to represent problem parameters, DE is a simple yet 2 Problem formulation
powerful population based, direct search algorithm with the
Without loss the generality, an MOP includes a set of n
generation-and-test feature for globally optimizing functions
parameters (decision variables), a set of k objective func-
using real valued parameters. Among the DE’s advantages
tions, and a set of m constraints. Objective functions and
are its simple structure, ease of use, speed and robustness.
constraints are functions of the decision variables. The opti-
Price & Storn [26] gave the working principle of DE with
mization goal is to
single scheme. Later on, they suggested ten different schemes
of DE [27]. It has been successfully used in solving single- minimize : y = f (x) = (f1 (x), · · · , fk (x))
objective optimization problems. Hence, several researchers subject to : e(x) = (e1 (x), · · · , em (x)) ≥ 0
have tried to extend it to handle MOPs. Such as Pareto DE
where : x = (x1 , x2 , · · · , xn ) ∈ X (1)
(PDE) [1], Pareto DE Approach (PDEA) [18], Multiobjec-
tive DE (MODE) [28], DE for Multiobjective Optimization y = (y1 , y2 , · · · , yk ) ∈ Y
(DEMO) [23], GDE3 [14], and -MyDE [24], which uses where x is the decision vector, y is the objective vector, X
the -dominance [9] to get a good distribution of solutions denotes as the decision space, and Y represents the objec-
retained. A detailed survey of multiobjective DE has been tive space. Generally, for each variable xi it satisfies a con-
published recently [24], where the advantages and disadvan- strained boundary
tages of the most known existing multiobjective DE methods
are discussed. Thus, in the present paper we do not review li ≤ xi ≤ ui , i = 1, 2, · · · , n (2)
again this field. However, all of previous approaches gener-
ated the initial population randomly and most of them were The constraints e(x) ≥ 0 determine the set of feasible solu-
tested in unconstrained problems. Also little previous work tions.
pays attention to solve engineering design problems.
Definition 1 (Pareto Dominance) A vector x = (x1 , · · · , xk )
In order to further extend DE to solve multiobjective is said to Pareto dominate another vector y = (y1 , · · · , yk ),
engineering design problems efficiently, in this paper, we denoted as x ≺ y, if and only if
propose a novel multiobjective DE algorithm, called pa-
ODEMO, which integrates established techniques in exist- ∀i ∈ 1, · · · , k, xi ≤ yi and ∃j ∈ 1, · · · , k, xi < yi
ing EA’s in a single unique algorithm. The new approach
uses the orthogonal design method with quantization tech- Definition 2 (Pareto Optimality) A solution x ∈ X is said
nique to generate the initial population. Moreover, it adopts to be Pareto optimal in X if and only if ¬∃y ∈ X, v ≺
an archive to store the nondominated solutions and employs u, where u = f (x) = (f1 (x), · · · , fk (x)), v = f (y) =
the new Pareto-adaptive -dominance [13] to update the archive (f1 (y), · · · , fk (y)).
 3

Definition 3 (-dominance) Let f , g ∈ k . Then f is said 3.2 Orthogonal design method in EAs
to -dominate g for some  > 0, denoted as f ≺ g, if and
only if for all i ∈ {1, · · · , k}, (1 − )fi ≤ gi . In a discrete single objective optimization problem, when
there are N factors (variables) and each factor has Q lev-
Definition 4 (Pareto-optimal set) The Pareto-optimal set POS els, the search space consists of QN combinations of levels.
is defined as the set of all Pareto-optimal solutions, i.e., P OS = When N and Q are large, it may not be possible to do all
{x ∈ X|¬∃y ∈ X, f (y) ≺ f (x)}. QN experiments to obtain optimal solutions. Therefore, it is
desirable to sample a small, but representative set of com-
Definition 5 (Pareto-optimal front) The Pareto-optimal front binations for experimentation, and based on the sample, the
POF is defined as the set of all objective functions values optimal may be estimated. The orthogonal design was de-
corresponding to the solutions in POS, i.e., P OF = {f (x) = veloped for the purpose [11]. The selected combinations are
(f1 (x), · · · , fk (x))|x ∈ P OS}. scattered uniformly over the space of all possible combina-
tions QN .
Recently, some researchers applied the orthogonal de-
sign method incorporated with EAs to solve optimization
3 Related Work
problems. Leung et al. [17] incorporated orthogonal design
in GA for numerical optimization problems and found such
3.1 -domination based MOEAs
method was more robust and statistically sound than the
classical GAs. OMOEA [29] presented by Zeng et al. adopted
To achieve a faster convergence in multiobjective optimiza-
the orthogonal design method to solve MOPs. In OMOEA,
tion, one of the relaxed forms of Pareto dominance, -dominance
it uses the orthogonal design method to generate a group of
proposed by Laumanns et al. [16], has become popular in the
sub-niches, every sub-niche evolves at each generation. Be-
last few years. The -dominance acts as an archiving strategy
cause the orthogonal arrays (OAs) must be generated at each
to ensure both properties of convergence towards the Pareto-
generation, OMOEA is very time-consuming. Cai et al. [2]
optimal set and properties of diversity among the solutions
proposed a novel multiobjective DE, -ODEMO, which uses
found. Deb et al. [9], [10] proposed a steady-state MOEA,
the orthogonal design method to generate the initial popula-
-MOEA, based on the -dominance concept and efficient
tion and adopts -dominance to update the archive. Experi-
parent and archive update strategies to MOPs. The simula-
mental results indicate -ODEMO is very efficient in terms
tion results indicated that -MOEA is a good compromise in
of convergence near to the Pareto-optimal front, diversity of
terms of convergence near to the Pareto-optimal front, di-
solutions, and computational time. However, there are two
versity of solutions, and computational time. The authors
limitations of -ODEMO, (i) it may lose some nondomi-
concluded that the use of -dominance criterion has been
nated solutions in the final archive; and (ii) the choice of
found to have two advantages: (i) it helps in reducing the
the -vector is difficult of this approach.
cardinality of Pareto-optimal region and (ii) it ensures that
no two obtained solutions are within an i from each other
in the i-th objective. Later, Santana-Quintero et al. [24] and 4 Differential Evolution Algorithm
Cai et al. [2] incorporated the -dominance concept into DE
method to solve MOPs. However, the above-mentioned - DE algorithm [26] is a simple evolutionary algorithm that
domination based MOEAs do not overcome the main limi- creates new candidate solutions by combining the parent in-
tation of -dominance: the loss of several nondominated so- dividual and several other individuals of the same popula-
lutions from the hypergrid adopted in the archive because of tion. A candidate replaces the parent only if it has better
the way in which solutions are selected within each box [13]. fitness. This is a rather greedy selection scheme that of-
In order to remedy the limitation, Hernández-Dı́az et al. ten outperforms traditional EAs. Unlike GA that uses bi-
[13] proposed a new Pareto-adaptive -dominance method, nary coding to represent problem parameters, DE is a sim-
called pa-dominance, where different -dominance regions ple yet powerful population based, direct search algorithm
depending on the geometrical characteristics of the Pareto- with the generation-and-test feature for globally optimiz-
optimal front is used. This method remedies some limita- ing functions using real valued parameters. It has been suc-
tions of the original -dominance and can finds a higher cessfully used in solving single-objective optimization prob-
number of efficient points. However, in order to use the pa- lems. Among the DE’s advantages are its simple structure,
dominance method, an initial Pareto front approximation, ease of use, speed and robustness.
denoted by F , must be generated [13]. And the number of The DE algorithm in pseudo-code is shown in Algo-
efficient points in F can be critical for the final performance. rithm 1. Where N P is size of the evolutionary population.
If F is not generated efficiently, the final performance may CR is the probability of crossover operator. F is the scaling
be very poor. factor. rndint(1, n) is a randomly chosen index ∈ 1, 2, · · · , n
4

which ensures that U i gets at least one parameter from the the global minimum. It is desirable that an algorithm starts
mutant vector. And rndj [0, 1) is the j-th evaluation of a to explore those points that are scattered evenly in the fea-
uniform random number generator from [0, 1). Many vari- sible solution space. In our presented manner, the algorithm
ants of creation of a candidate are possible. We use the DE can evenly scan the feasible solution space once to locate
scheme DE/rand/1/bin (see lines 6 and 13 of Algorithm 1) good points for further exploration in subsequent iterations.
described in Algorithm 1 (more details on DE/rand/1/bin As the algorithm iterates and improves the population of
scheme and other DE schemes can be found in [27]). points, some points may move closer to the global mini-
mum. Based on these considerations, in order to obtain an
efficient F to generate the the first grid as soon as possible,
Algorithm 1 DE algorithm with DE/rand/1/bin scheme
we apply the quantization technique and the orthogonal de-
1: Generate the initial population of N P individuals P (0)
2: Evaluate the fitness for each individual in P (0) sign method to generate this initial archive and evolutionary
3: while The halting criterion is not satisfied do population (EP).
4: for i = 1 to N P do
5: Select uniform randomly r1 = r2 = r3 = i
6: jrand = rndint(1, n) 5.1.1 Design of the orthogonal array
7: for j = 1 to n do
8: if rndj [0, 1) < CR or j = jrand then To design an orthogonal array (OA), in this research, we use
9: Uji = Xjr1 + F × (Xjr2 − Xjr3 ) LR (QC ) to denote the OA with different level Q, where Q
10: else
is odd and use R = QJ to indicate the number of the rows
11: Uji = Xji
12: end if of OA, where J is a positive integer fulfilling
13: end for
14: Evaluate the offspring U i QJ − 1
C= (3)
15: if U i is better than X i then Q−1
16: Xi = Ui
17: end if C denotes the number of the columns of OA in the above
18: end for equation. The OA needs to find a proper J and Q to satisfy
19: end while
minimize : R = QJ
QJ − 1
subject to : C = ≥n (4)
Q−1
5 Our Proposed Approach: pa-ODEMO R ≥ NP
where n is the number of the variables, N P is the size of
From the literature review, the main deficiency in the ex- the evolutionary population. In this study, we adopt the al-
isting MOEAs lies on designing a suitable fitness assign- gorithm described in [17] to construct an OA. In particular,
ment strategy in order to search for a uniformly distributed, we use L(R, C) to indicate the OA; and ai,j to denote the
near-complete and near-optimal approximated Pareto front level of the jth factor in the ith combination in L(R, C). The
for the given optimization problem. Unfortunately, these ob- algorithm to generate the OA is described in Algorithm 2.
jectives are contradictory. Inspired by the ideas from the
orthogonal design method successfully used in EAs ([17],
5.1.2 Quantization
[29], and [2]) and pa-dominance proposed in [13], in this
work, we propose an extension of DE algorithm to solve For one decision variable Xj with the boundary [lj , uj ], we
constrained MOPs, which integrates established techniques quantize the domain into Q levels αj1 , αj2 , · · · , αjQ , where
in existing EA’s in a single unique algorithm. Our proposed the design parameter Q is odd and αi is given by
DE algorithm is named pa-ODEMO. Six crucial proce-
dures of pa-ODEMO will be discussed as follows. u j − lj
αji = lj + (i − 1)( ), 1 ≤ i ≤ Q (5)
Q−1

5.1 Orthogonal Initial Population In other words, the domain [lj , uj ] is quantized Q − 1 frac-
tions, and any two successive levels are same as each other.
As mentioned above, in order to use the pa-dominance method,
an initial Pareto front approximation F must be generated 5.1.3 Generation of Initial Population
[13]. And the number of efficient points in F can be critical
for the final performance. Obviously, the higher the number After constructing a proper OA and quantizing the domain
of the efficient points in F the better performance of the grid of each decision variable, we can generate the orthogonal
generated. However, before solving an optimization prob- population (OP) which can scatter uniformly over the de-
lem, we usually have no information about the location of cision space. Recall that Algorithm 2 can only construct
 5

Algorithm 2 Construction of Orthogonal Array that the parameters Q and J are very easy to be selected for
1: /* Construct the basic columns */ different problems.
2: for k = 1k−1
to J do
Q −1
3: j= Q−1
+1
4: for i = 1 to R do
ai,j =  Qi−1
Algorithm 3 Construction of Initial Archive and Evolution-
5: J−k  mod Q
6: end for ary Population
7: end for 1: /* Construction of orthogonal population (OP) */
8: /* Construct the nonbasic columns */ 2: eval = 0, ar size = 0
9: to J do
for k = 2k−1 3: for i = 1 to R do
10: j = Q Q−1−1 + 1 4: for j = 1 to n do
11: for s = 1 to j − 1 do 5: k = ai,j
12: for t = 1 to Q − 1 do 6: OPi,j = αjk
13: for i = 1 to R do 7: end for
14: ai,(j+(s−1)(Q−1)+t) = (ai,s × t + ai,j ) mod Q 8: Evaluate OPi and eval++
15: end for 9: end for
16: end for 10: /* Construction of initial archive (AR) */
17: end for 11: Find all of the nondominated solutions of OP
18: end for 12: Insert them into the initial archive, now ar size > 0
19: Increment ai,j by one for all i ∈ [1, R] and j ∈ [1, C] 13: /* Construction of initial evolutionary population (EP) */
14: if ar size ≥ N P then
15: Randomly select N P solutions from AR to generate initial EP
16: else
17: Insert all of the ar size solutions of AR into EP
LR (QC ), where R = QJ and J is a smallest positive in- 18: Select the remainder N P − ar size solutions from OP ran-
J
teger fulfilling C = QQ−1 −1
≥ n [see (4)]. Since the prob- domly
lem dimension n is given, when we execute Algorithm 2 to 19: Inert them into EP
20: end if
construct an OA with C factors, if C > n, then delete the
last C − n columns to get an array with n factors. And the
remainder of the array is still an OA [17]. The algorithm
for generating the OP is described in Algorithm 3, where
OPi,j is the j-th variable of the i-th individual of OP, R
is the number of rows in OA, n is the number of decision
variables, and eval is the current number of fitness function 5.2 Archiving the Candidate Solutions
evaluations (NFFEs). Regularly, the number of the rows R
of the OA is larger than the population size N P , so we cre- In the study of Zitzler et al. [31], it was clearly shown that
ate the initial archive with the nondominated solutions from elitist helps in achieving better convergence in MOEAs. In
OP first. And then we generate the initial EP from the initial pa-ODEMO, the elitism scheme is also adopted through
archive and OP with the following procedure (more details maintaining an external archive of nondominated solutions
are described in Algorithm 3). If ar size ≥ N P , we select found in evolutionary process. In order to achieve a faster
N P solutions from the initial archive randomly; or all of convergence, in this work, the Pareto-adaptive -dominance,
the ar size solutions in the initial archive are inserted into the so-called pa-dominance proposed by Hernández-Dı́az
EP, and the remainder N P − ar size solutions are selected et al. [13], is used to update the archive. At each genera-
from OP randomly. Compared with OGA/Q [17], our ap- tion, in order to include a solution into this archive, it is
proach and OGA/Q differ in two aspects with respect to the compared with respect to each member already contained
generation of initial population: (i) our approach integrates in the archive using pa-dominance after the grid is gen-
the orthogonal design method within MOEAs, and (ii) in our erated. Note, however, that although  can often have little
approach there are two population, archive and EP, we first meaning for test function analysis, the -dominance method
generate the initial archive with the nondominated solutions is very useful: i) it allows for the desired convergence prop-
from OP, and then the initial EP is generated from the ini- erties, and ii) it guarantees an optimal distribution of solu-
tial archive and OP. It is worth to point out that for a given tions [16]. Moreover, Laumanns et al. [16] pointed out that
MOP and the given Q and J, the initial archive generated none of the MOEAs has a proof of convergence to the true
by Algorithm 3 is the same for all runs, so we can gener- Pareto-optimal solutions with a wide diversity among the so-
ate the initial archive offline to avoid the time complexity of lutions, so the -dominance method used to solve engineer-
the Algorithm 2 and Algorithm 3. Thus we do not need to ing problems is meaningful and reasonable compared with
generate the initial archive each run, which makes the new other MOEAs. The procedure is described as follows.
algorithm more efficient. Furthermore, for a given MOP, we Every solution in the archive is assigned an identification
only need to generate a minimal OA which satisfies (4), so array (B = (B1 , B2 , · · · , Bk )T , where k is the total number
6

of objectives) as follows: a solution from the archive for mating starting from the evo-
lutionary process. In -MyDE [24], Santana-Quintero et al.
i1 pvi −(pvi −1)fi
 log( )  proposed two selection mechanisms, where the random se-
i1
Bi (f ) = +1 (6)
log( p1vi ) lection and elitist selection are interleaved. At the beginning,
all of the parents for mating are randomly selected from the
where, p controls the shape of the curve (or surface), T is EP to generate the offspring. When the current generations
the number of points desired by the decision maker, i1 is the is larger than a pre-defined number, the elitist selection is
size of the first box for each dimension, and vi controls the used and all of the parents are randomly selected from the
speed of variation. These parameters satisfy the following archive to generate the offspring. However, these two tech-
equations: niques have a limitation: the selection pressure is very high.
 vi (T −1)vi
Especially, in -MyDE, the main evolutionary population
i1 = (p −1)p
pT vi −1 (EP) is not used at all when the elitist selection is adopted.
(7)
(1 − 21/p )pT vi + 21/p pT vi /2 − 1 = 0 In our proposed approach, in order to regulate the se-
lection pressure we propose a hybrid selection mechanism,
The identification array divides the whole objective space which is similar to the method used in -MyDE, in which
into hyper-boxes. With the identification arrays calculated a random selection and an elitist selection are interleaved.
for the offspring c and each archive member a, the offspring The difference between -MyDE and our approach is that
c updates the archive as Algorithm 4 described (Ba indicates in elitist selection we only randomly choose one solution
the identification arrays of solution a). from the archive as the base parent, X r1 , in DE/rand/1/bin
scheme described in Algorithm 1. And the other two par-
Algorithm 4 Updating the Archive ents, X r2 and X r3 , are selected from EP randomly. We use
1: if Bc of the offspring -dominates Ba of any archive member a a selection parameter λ ∈ [0.1, 1.0] to regulate the selection
then pressure.
2: Delete all of the dominated archive members
3: Accept the offspring c 
random selection, eval < (λ × M ax eval)
4: else if Bc is -dominated by Ba of any archive member a then selection =
5: Reject c elitist selection, otherwise
6: else (8)
7: if c shares the same grid with an archive member a then
8: if c dominates a or c is closer to the grid than a then where, eval is the current NFFEs, and M ax eval is the
9: Delete a from the archive and accept c
maximal NFFEs pre-defined by the user. In our proposed
10: else
11: Reject c hybrid selection mechanism, one archive solution, which is
12: end if disturbed by the two randomly selected solutions from EP,
13: else is selected to take part in the generating process when the
14: Insert c into the archive
elitist selection is used. In this manner, our approach has
15: end if
16: end if three advantages: (i) it can guarantee to be the best solu-
tions found; (ii) because we do not use the archive solution
to guide the search at the beginning of evolutionary process,
Using the pa-dominance method, we can maintain the it can avoid to be misled by the inefficient archive solutions;
good properties of the original -dominance, such as en- and (iii) the solutions in EP can also guide the search in the
suring both properties of convergence towards the Pareto- elitist scheme.
optimal set and properties of diversity among the solutions
found in a small computation time, while overcoming the
main limitations of -dominance: the loss of several non- 5.4 Constraint-handling Method
dominated solutions from the hypergrid adopted in the archive.
Usually, most of engineering design problems have multi-
ple constraints. So it is essential to handle the constraints
5.3 Hybrid Selection Mechanism to solve the engineering design problems. A considerable
amount of researches on constraint handing techniques that
In [16], Laumanns et al. concluded that the archived mem- incorporate objective(s) and constraint(s) into the fitness func-
bers are really guaranteed to be the best solutions found. As tion of design candidates has been carried out (a good sum-
the archive is used to maintain the nondominated solutions mary is given in [3]). However, many previous constraint-
in pa-ODEMO, a key issue must be addressed is that how handling methods need to tune some parameters to balance
to allow the archive members to take part in the generating between the objective(s) and constraint(s). In this research,
process? In -MOEA [9], [10], Deb et al. randomly picked we employ a new constraint-handling method proposed by
 7

Oyama [19], which does not need any parameters to be tuned Algorithm 5 Main procedure of the proposed pa-ODEMO
for constraint handling. This method is based on the “Constraint- 1: Generate a proper OA and generate the orthogonal population OP
First-Objective-Next” model [15], where the constraints pre- 2: Create the initial archive AR1 with the nondominated solutions of
OP
cede the objectives because the feasibility of x is more im- 3: Create the initial evolutionary population EP1 from AR1 and OP
portant than minimization of f (x). The method is described 4: t = 1
as follows. 5: f lag = 0
6: while eval < M ax eval do
Definition 6 (Constrained Pareto dominance) A solution xi 7: child size = 0
is said to constrained-dominate a solution xj , if any of the 8: for i = 1 to N P do
following conditions is true: 9: if eval < λ × M ax eval then
10: Random selection
1. Solutions xi and xj are both feasible and solution xi 11: Produce the offspring c with DE/rand/1/bin scheme
dominates solution xj in objective function space. 12: else
13: Elitist selection
2. Solution xi is feasible and solution xj is not. 14: Produce the offspring c with DE/rand/1/bin scheme
3. Solutions xi and xj are both infeasible, but solution xi 15: end if
dominates solution xj in constraint space. 16: Evaluate the offspring c and eval++
17: if the offspring c dominates the target parent EPti then
where dominance in objective space is defined as Defini- 18: EPti = c
tion 1 while dominance in constraint space is defined as: 19: else if c is nondominated by EPti then
20: Add c to the child population CP
Definition 7 (Constraint space dominance) A solution xi 21: child size++
is said to dominate a solution xj in constraint space, if both 22: else
23: Discard c
of the following conditions are true: 24: end if
1. Solutions xi is no worse than solution xj in all con- 25: if f lag == 0 then
26: Update the archive with the usual dominance
straints, i.e., 27: else
28: Update the archive with pa-dominance
∀Ek (xi ) ≥ Ek (xj ) (9) 29: end if
30: end for
2. Solution xi is strictly better than solution xj in at least 31: if child size = 0 then
one constraint, i.e., 32: Combine CP and EPt
33: Prune the mixed population using nondominated ranking
∃Ek (xi ) > Ek (xj ) (10) method only
34: Get the next evolutionary population EPt+1
where Ek (x) = min(0, ek (x)), and k = 1, · · · , m. 35: end if
36: if ar size ≥ N F and f lag == 0 then
37: Generate the pa-dominance grid
38: f lag = 1
5.5 Handling the Constraint of the Variables 39: end if
40: t++
After using the DE/rand/1/bin scheme to generate a new so- 41: end while
lution, if one or more of the variables in the new solution
are outside their boundaries, i.e. xi ∈ / [li , ui ], the following
repair rule is applied:
 generated using DE/rand/1/bin scheme. The offspring re-
l + rndi [0, 1] × (ui − li ) if xi < li
xi = i (11) places the parent immediately if the parent is dominated by
ui − rndi [0, 1] × (ui − li ) if xi > ui the offspring. If the parent dominates the offspring, the off-
where rndi [0, 1] is the uniform random variable from [0,1] spring is discarded. Otherwise (when the offspring and par-
in each dimension i. ent are nondominated with regard to each other), the off-
spring is added to a temporary child population (CP). If
the first grid is not generated, insert the offspring into the
5.6 Main Procedure of pa-ODEMO archive with usual dominance concept. Otherwise, if the first
grid has been generated, insert the offspring into the archive
For an MOP, the proposed pa-ODEMO algorithm works with pa-dominance concept. This step is repeated until N P
as Algorithm 5. First, the temporary orthogonal population number of offspring are created. After that, combine the
(OP) is created using Algorithm 3, all of the nondominated temporary child population CP with EP, and we get a popu-
solutions of OP are inserted into the initial archive (AR). lation of the size between N P and 2×N P . If the population
And then, the initial evolutionary population (EP) is cre- has enlarged, we have to truncate it to prepare it for the next
ated from AR and OP. At each generation, an offspring is step of the algorithm.
8

The truncation sorts the individuals with nondominated selection parameter of λ = 0.9. As suggested in [17], to
sorting and if the individuals in the same front we only se- generate an OA, we use J = 2, and Q = 11 if n < 10;
lect them randomly. This is different from NSGA-II [8] and otherwise, Q = n − 1.
DEMO [23] to evaluate the individuals of the same front
with the crowding distance metric. Because in our approach
the diversity is maintained in the archive with pa-dominance. 6.2 Performance Metrics
The truncation procedure keeps in EP only the best N P in-
dividuals (with regard to the nondominated sorting metric). In order to make a fair comparison with other MOEAs, we
If the initial Pareto front approximation F is created, i.e. use two performance metrics proposed by Deb et al. [8] to
ar size is larger than N F that is the size of F , and the assess the performance. The two metrics are derived from
first grid is not generated, then we use the pa-dominance the final generation of 50 independent runs with different
concept to generate the grid (the algorithm is omitted here, random seeds.
interest readers can refer [13] for more details). The first metric is the Convergence metric γ. It mea-
sures the distance between the obtained nondominated front
Q and the set P OF of Pareto-optimal front:
6 Experiments and Results |Q|
di
γ = i=1 (12)
In order to validate the performance of our proposed ap- |Q|
proach, pa-ODEMO, it is tested on seven constrained bench- where di is the Euclidean distance (in the objective space,
mark problems to illustrate the capabilities of the algorithm hereinafter) between the solution i ∈ Q and the nearest
in handling mathematically complex problems. Also, four member of P OF . The lower the γ value, the better the con-
well-studied engineering design optimization problems are vergence of solutions. A result of γ = 0 indicates the con-
solved to illustrate the efficiency and applicability of the al- vergence Q = P OF ; any other value indicates Q deviates
gorithm for multiobjective design optimization. Moreover, from P OF .
our approach is compared with NSGA-II [8], which is one The second metric is Diversity metric Δ. This metric
of the best MOEAs available at present. For pa-ODEMO, measures the extent of spread achieved among the obtained
we have chosen a reasonable set of value and have not made nondominated front Q. It is desirable to get a set of solutions
any effort in finding the best parameter settings. We leave that spans the entire Pareto-optimal region. Δ is defined as
this task for a future study. follows:
k |Q|
de + i=1 |di − d|
Δ = i=1ki (13)
e
6.1 Parameter Settings i=1 di + |Q|d

where dei denotes the Euclidean distance between the i-th

All approaches are run 50 independent runs with different
random seeds on all test problems. The maximal NFFEs of coordinate for both extreme points in Q and P OF , and di
measures the Euclidean distance of each point in Q to its
different problems (described in the following sections) are
closer point in Q. From the above definition, it is easy to
different due to the complexity of the search space. For dif-
conclude that Δ ≥ 0 and the lower the Δ value, the better
ferent approaches, the parameter settings are as follows:
the distribution of solutions. A perfect distribution, that is
– For NSGA-II [8], the simulated binary crossover (SBX) Δ = 0, means that the extreme points of P OF have been
and polynomial mutation are used. The crossover prob- found and di is constant for all i.
ability of pc = 0.9 and a mutation probability of pm = Moreover, in order to compare the dominance relation-
1/n (where n is the number of decision variables). The ship between two populations resulting from two different
distribution indices for crossover and mutation operators MOEAs, the coverage of two sets (C value) [30], which is a
set as ηc = 20 and ηm = 20, respectively. The popula- binary metric [33], is measured to show how the final pop-
tion size of N P = 100. ulation of one algorithm dominates the final population of
– For pa-ODEMO, the crossover probability of pc = 0.9 another algorithm. The C value can be calculated as follows:
and the scaling factor of F = 0.5. The evolutionary pop-
|a ∈ X  ; ∃a ∈ X  : a a |
ulation size N P is set to 100. The size of initial Pareto C(X  , X  ) = (14)
front approximation F is N F = 1001 . The number of |X  |
points desired by the decision maker of T = 100. The where X  , X  ∈ X are two sets of objective vectors, and
1 For a grid with a maximum capacity of 100 vectors (i.e. T = 100), a a means that a covers a if and only if a ≺ a or
Hernández-Dı́az et al indicated that the best results are obtained when a = a . Function C maps the ordered pair (Xi , Xj ) to the


75 ≤ N F ≤ 125 [13]. interval [0, 1], where Xi and Xj denote the final populations
 9

resulting from algorithm i and j, respectively. The value f2 . The density of solutions gets thinner towards the Pareto-
C(Xi , Xj ) = 1 implies that all points in Xj are dominated optimal region. The Pareto-optimal curve lies on the in-
by or equal to points in Xi . The opposite, C(Xi , Xj ) = tersection of all 2 constraints. This feature of this prob-
0, represents the situation when none of the points in Xj lem may cause MOEAs difficulty in solving this prob-
are covered by the set Xi . Note that both C(Xi , Xj ) and lem.
C(Xj , Xi ) need to be considered independently since they
Table 2 compares the performances of pa-ODEMO and
have the distinct meanings.
NSGA-II. The statistics shown are based on 50 independent
runs with different random seeds. All the algorithms are im-
plemented in C++ and the experiments are done on a P-IV
6.3 Benchmark Constrained MOPs
2.8 GHz machine with 512 MB RAM under WIN-XP plat-
form. For the convergence metric and diversity metric, we
First, we select seven benchmark constrained MOPs (OSY,
need to know the true Pareto-optimal front for a problem.
SRN, TNK, Kita, Tamaki, DTLZ8, and DTLZ9) to test the
Since the benchmark constrained MOPs in Table 1 are arti-
capabilities of the algorithm in handling mathematically com-
ficial test problems, the true Pareto-optimal front is not dif-
plex problems. The details of these problems are described
ficult to be obtained. In our experiments we use uniformly
in Table 1, where OSY, SRN, and TNK are chosen from [6],
spaced Pareto-optimal solutions as the approximation of the
Kita and Tamaki are chosen from [24], DTLZ8 and DTLZ9
true Pareto-optimal front.
are selected from [7]. For OSY, SRN, TNK, and Kita, they
From Table 2, we can see that our approach is capa-
are two-objective MOPs. For Tamaki, DTLZ8, and DTLZ9,
ble of converging closely to the true Pareto-optimal front,
they are three-objective MOPs. The characters of these prob-
with good distribution of non-dominated solutions, for all of
lems are briefly described as follows.
the benchmark constrained MOPs. With respect to the set
– For OSY, there are six constraints. The Pareto-optimal coverage metric, pa-ODEMO produces better mean val-
region is a concatenation of five regions. Every region ues than NSGA-II for all of the seven MOPs. Also com-
lies on the intersection of certain constrains. Since the pared with the convergence metric, pa-ODEMO performs
entire Pareto-optimal region demands an MOEA popu- better than NSGA-II for these test problems. For the diver-
lation to maintain sub-populations at different intersec- sity metric, pa-ODEMO provides better mean values than
tions of constraint boundaries, it is difficult to solve for NSGA-II for six out of seven test problems, NSGA-II is
MOEAs. better than pa-ODEMO on OSY. However, from Fig. 1, it
– For SRN, it is relatively easy to solve for MOEAs. The can be seen that NSGA-II can not coverage the entire true
only difficulty the constraints introduce in this problem Pareto front sometimes. Moreover, pa-ODEMO needs less
is that they eliminate some part of the unconstrained CPU time than NSGA-II. The reason are two aspects. On the
Pareto-optimal set. one hand, for a give MOP and the given Q and J we only
– For TNK, this is a disconnected MOP, the Pareto-optimal need to generate the initial archive one run out of 50 runs.
sets are disconnected because of the constraints. Since On the other hand, as described in Algorithm 5, the crowd-
the Pareto-optimal solutions lie on a non-linear constraint ing distance metric is not used in our approach, and hence
space, an optimization algorithm may find difficulty in it can eliminate the complexity of O(k(2N P )log(2N P ))
finding a spread of solutions across the entire Pareto- of the crowding distance assignment and the complexity of
optimal region. O(2N P log(2N P )) of the crowding distance sorting com-
– For Kita, this problem has disconnected, concave Pareto- pared with NSGA-II [8].
optimal front, and the Pareto-optimal set is also discon- In order to demonstrate the working of the algorithm,
nected. Note that the Pareto-optimal set is in the bound- a typical simulation run using pa-ODEMO and NSGA-II
ary between the feasible and the infeasible region. is shown in Fig. 1 and Fig. 2 for all benchmark constrained
– For Tamaki, this a three-objective MOP. The Pareto-optimal problems. The graphical illustration also clearly demonstrates
front is connected and forms a curved surface. that for the two-objective MOPs pa-ODEMO is able to
– For DTLZ8, it is a high-dimensional and three-objective search for a uniformly distributed, near-complete and near-
problem. The Pareto-optimal front is a combination of optimal approximated Pareto front. Furthermore, pa-ODEMO
a straight line and a hyper-plane. An optimization algo- can obtain better nondominated solutions than NSGA-II for
rithm may find difficulty in finding solutions in both the all three-objective MOPs.
straight line and the hyper-plane in this problem and also From the comparison and analysis above, we can con-
in maintaining a good distribution of solutions on the clude that pa-ODEMO can produce competitive results com-
hyper-plane. pared with NSGA-II. It is able to search for a uniformly
– For DTLZ9, it is also a high-dimensional and three-objective distributed, near-complete and near-optimal approximated
problem. The Pareto-optimal front is a curve with f1 = Pareto front. Moreover, pa-ODEMO can deal with two-
10

Table 1 Benchmark constrained problems used in this study, n is the number of decision variables

Problem Max eval n Domain Objective functions Constraints


OSY 25,000 6 0 ≤ x1 , x2 , x6 ≤ 10 minf1 (x) = −25[ 2i=1 (xi − 2)2 + (x3 − 1)2 + e1 (x) = x1 + x2 − 2 ≥ 0
2 2
(x
6 4 − 4) + (x5 − 1) ]
2
1 ≤ x3 , x5 ≤ 5 minf2 (x) = x
i=1 i e2 (x) = 6 − x1 − x2 ≥ 0
0 ≤ x4 ≤ 6 e3 (x) = 2 + x1 − x2 ≥ 0
e4 (x) = 2 − x1 + 3x2 ≥ 0
e5 (x) = 4 − (x3 − 3)2 − x4 ≥ 0
e6 (x) = (x5 − 3)2 + x6 − 4 ≥ 0
SRN 5,000 2 −20 ≤ x1 ≤ 20 minf1 (x) = 2 + (x1 − 2)2 + (x2 − 1)2 e1 (x) = 225 − x21 − x22 ≥ 0
−20 ≤ x2 ≤ 20 minf2 (x) = 9x1 − (x2 − 1)2 e2 (x) = 10 − x1 + 3x2 ≥ 0
TNK 20,000 2 0 ≤ x1 ≤ π minf1 (x) = x1 e1 (x) = x21 + x22 − 1−
0.1 cos(16 arctan (x1 /x2 )) ≥ 0
0 ≤ x2 ≤ π minf2 (x) = x2 e2 (x) = 0.5 − 2i=1 (xi − 0.5)2 ≥ 0
Kita 5,000 2 x1 ≥ 0 maxf1 (x) = x2 − x21 e1 (x) = 6.5 − x1 /6 − x2 ≥ 0
x2 ≥ 0 maxf2 (x) = 0.5x1 + x2 + 1 e2 (x) = 7.5 − x1 /2 − x2 ≥ 0
e3 (x) = 30 − 5x1 − x2 ≥ 0
Tamaki 5,000 3 0 ≤ xi ≤ 1 maxf1 (x) = x1 e1 (x) = x21 + x22 + x23 > 0
maxf2 (x) = x2
maxf2 (x) = x3
1
 10
DTLZ8 50,000 30 0 ≤ xi ≤ 1 minf1 (x) = 10 i=1 xi
 20 e1 (x) = f3 (x) + 4f1 (x) − 1 ≥ 0
1
minf2 (x) = 10  i=11 xi e2 (x) = f3 (x) + 4f2 (x) − 1 ≥ 0
1 30
minf3 (x) = 10 i=21 xi e3 (x) = 2f3 (x) + f1 (x) + f2 (x) − 1 ≥ 0
 10 0.1
DTLZ9 50,000 30 0 ≤ xi ≤ 1 1
minf1 (x) = 10  i=1 xi e1 (x) = f32 (x) + f12 (x) − 1 ≥ 0
20 0.1
1
minf2 (x) = 10  30
i=11 x i e2 (x) = f32 (x) + f22 (x) − 1 ≥ 0
1 0.1
minf3 (x) = 10 i=21 xi

Table 2 Comparison of pa-ODEMO and NSGA-II on the seven benchmark constrained problems considered in the study showing the mean, and
standard deviation (SD) values for set coverage metric (C , where X1 and X2 represent pa-ODEMO and NSGA-II, respectively), convergence
metric (γ ), and spread metric (Δ). All results have been averaged over 50 independent runs. A result with boldface indicates better value obtained.
C γ Δ Time (s)
Problem Statistic
C(X1, X2) C(X2, X1) pa-ODEMO NSGA-II pa-ODEMO NSGA-II pa-ODEMO NSGA-II
OSY Mean 0.26040 0.16947 1.31837 1.56145 0.80109 0.74815 0.64000 0.72648
SD 0.11974 0.05659 0.16359 0.22319 0.07555 0.08818 0.02640 0.02360
SRN Mean 0.12900 0.01263 0.06575 0.23971 0.15080 0.39062 0.14470 0.22188
SD 0.03190 0.00935 0.00882 0.03456 0.01632 0.04516 0.00697 0.01411
TNK Mean 0.11880 0.09949 0.00146 0.00228 0.367440 0.84805 0.42648 0.49932
SD 0.03243 0.03350 2.28E-4 3.59E-4 0.042210 0.10055 0.02604 0.02156
Kita Mean 0.55318 0.01100 0.01964 0.07199 0.29728 0.81069 0.11532 0.17340
SD 0.05346 0.01093 0.00145 0.07644 0.03246 0.14839 0.00767 0.01007
Tamaki Mean 0.17960 0.08153 0.01903 0.02027 0.34264 0.51283 0.13316 0.22682
SD 0.04708 0.02684 0.00075 0.00249 0.01735 0.04984 0.00793 0.03209
DTLZ8 Mean 0.35160 0.00373 0.00982 0.03549 0.34844 0.58427 2.03898 2.75628
SD 0.06052 0.00402 7.86E-4 0.00593 0.02166 0.03546 0.07374 0.18688
DTLZ9 Mean 0.96980 0.81995 0.02018 0.02530 0.74592 0.78633 2.09844 2.56438
SD 0.04851 0.13507 0.00573 0.00356 0.02456 0.03416 0.05457 0.12258

and three-objective problems of diverse complexities; prob- to generate the offspring. In this case, the inefficient archive
lems with low and high dimensionality, with different types member may mislead the search. However, λ = 1.0 indi-
of Pareto fronts (concatenation, concave, discontinuous, thin cates that any archive solution is not selected to generate the
density and non-uniform spread), and with many constraints. offspring at all. It can be observed from Fig. 3 that different
λ provides different performance in terms of convergence
metric and diversity metric. For all benchmark constrained
6.4 Different Parameter Settings MOPs, the good control value λ is from [0.5, 0.9].

In this work, we do not make any serious attempt to find

the best parameter settings for pa-ODEMO. However in It is very interesting that apparently if λ = 0.0, pa-
this section, we perform additional experiments to show the ODEMO produces the worst performance in terms of both
effect of a couple of different parameter settings on the per- convergence metric and diversity metric for all benchmark
formance of pa-ODEMO. MOPs . It means that the archive member mislead the search
First, we keep all other parameters as described in Sec- if they take part in the generating process starting from evo-
tion 6.1, but perform our approach with λ taken from [0.0, 1.0] lution. In addition, for λ = 1.0, the performance of pa-
by step 0.1. Especially, when λ = 0.0, it means that start- ODEMO is poor for most of benchmark MOPs. It indicates
ing from evolution one of the archive solutions is selected that properly allowing the archive solutions to take part in
 11

OSY OSY
80 80
pae−ODEMO NSGA−II
True POF
70 70 True POF

60 60

50 50
f−2

f−2
40 40

30 30

20 20

10 10

0 0
−300 −250 −200 −150 −100 −50 0 −300 −250 −200 −150 −100 −50 0
f−1 f−1

SRN SRN
50 50
pae−ODEMO NSGA−II
True POF True POF
0 0

−50 −50
f−2

f−2

−100 −100

−150 −150

−200 −200

−250 −250
0 50 100 150 200 250 0 50 100 150 200 250
f−1 f−1

TNK TNK
1.4 1.4
pae−ODEMO NSGA−II
True POF True POF
1.2 1.2

1 1

0.8 0.8
f−2

f−2

0.6 0.6

0.4 0.4

0.2 0.2

0 0
0 0.2 0.4 0.6 0.8 1 1.2 1.4 0 0.2 0.4 0.6 0.8 1 1.2 1.4
f−1 f−1

Kita Kita
8.5 8.8
pae−ODEMO NSGA−II
8.4 True POF
True POF 8.6
8.3

8.2 8.4

8.1
8.2
f−2

f−2

8
8
7.9

7.8 7.8

7.7
7.6
7.6

7.5 7.4
−3 −2 −1 0 1 2 3 4 5 6 7 −4 −2 0 2 4 6 8
f−1 f−1

Fig. 1 Nondominated solutions of the final archive obtained by pa-ODEMO and NSGA-II on two-objective MOPs. The presented fronts are the
outcome of a typical run for pa-ODEMO and NSGA-II.
12

Tamaki Tamaki

1 pae−ODEMO
1 NSGA−II

0.8 0.8

0.6
0.6

f3
0.4
f3

0.4
0.2

0.2
0
0
0 0.2 1
0 1
0.4 0.8
0.2
0.4 0.6 0.6
0.5 0.4
0.6 0.8
0.8 0.2
1 0 1 0
f2 f2
f1
f1

DTLZ 8 DTLZ 8

0.7
pae−ODEMO
0.6 0.8 NSGA−II

0.5
0.6
0.4
f3

0.4
f3

0.3

0.2
0.2
0.1
0.8
0
0 0 0.6
0 0.8 0.2
0.2 0.6 0.4
0.4 0.4 0.4
0.6 0.2
0.6 0.2
0.8 0 0.8 0 f2
f2
f1 f1

DTLZ 9 DTLZ 9

pae−ODEMO NSGA−II
1
0.9
0.9
0.8
0.8

0.7 0.7
f3

f3

0.6
0.6
0.5
1
1 0.5
0.4 0.4
0.4 0.8
0.8 0.5
0.5 0.6
0.6 0.6 0.7 0.6
0.7
0.8 0.8
0.9 0.4 0.9 0.4
f2 f2
f1 f1

Fig. 2 Nondominated solutions of the final archive obtained by pa-ODEMO and NSGA-II on three-objective MOPs. The presented fronts are the
outcome of a typical run for pa-ODEMO and NSGA-II.

the generating process can guarantee to be the best perfor- initial population used in pa-ODEMO. The tick 0 of X axis
mance produced. in Fig. 4 and 5 represents the outcome of random initial pop-
ulation.
Second, we test different levels Q on the performance
of pa-ODEMO. Except for Q, all other parameters as de- From Fig. 4, it can be observed that the performance of
scribed in Section 6.1. For OSY, SRN, TNK, Kita, and Tamaki, random initial population is the worst in terms of both con-
Q sets with 11, 17, 21, 27, 31, 37, 41, 47, 51, 57 (ticks 1 - vergence metric and diversity metric. Moreover, the effect
10 of X axis in Fig. 4 and 5). For DTLZ8 and DTLZ9, Q of different levels on the performance of our approach is
sets with 29, 35, 39, 45, 49, 55, 59, 65, 69, 75 (ticks 1 - 10 small. From Fig. 5, we can see that when the Q increases
of X axis in Fig. 4 and 5). In addition, to validate the ef- the running time decreases. The reason is that for the given
fect of orthogonal initial population, we generate the initial Q and J we only need to generate the OA and the initial
evolutionary population randomly to replace the orthogonal archive one run. Since the maximal NFFEs is fixed, when Q
 13

Fig. 3 Effect of different selection parameter settings on the performance of all benchmark constrained MOPs. The results were averaged over
50 independent runs. Both convergence values and diversity values for each problem are normalized. Where problems 1-7 represent OSY, SRN,
TNK, Kita, Tamaki, DTLZ8, and DTLZ9, respectively, hereinafter.

Fig. 4 Effect of different levels on the performance of all benchmark constrained MOPs. The results were averaged over 50 independent runs.
Both convergence values and diversity values for each problem are normalized.

is larger, the algorithm needs more NFFEs to generate the and applicability of the algorithm for multiobjective design
initial archive and evolutionary population, and hence the optimization. For pa-ODEMO and NSGA-II, the parame-
remainder NFFEs for evolution process are smaller. In this ter settings are described in Section 6.1. The consistency of
manner, if Q is too large, the overall performance of pa- the algorithm was verified by running all the problems for
ODEMO may be poor [see Fig. 4]. 50 independent runs with different random seeds. However,
as the performance has already been proved (see previous
section), only the results for the representative sample run
6.5 Engineering Design Problems in 50 runs are reported here.

From the benchmark constrained MOPs experiments, we

can see that our approach is able to handle mathematically 6.5.1 Two-bar truss design
complex problems efficiently. So this section we will use our
approach to solve the engineering design problems. Four en- The first problem is the two-bar truss design problem, which
gineering design problems are selected to test the efficiency has been well studied by many researchers [12], [5], and
14

4 Two−bar−truss
x 10
10
pae−ODEMO
9

f−2
5

0
0 0.02 0.04 0.06 0.08 0.1
f−1

4 Two−bar truss
x 10
10
NSGA−II
9

Fig. 5 Effect of different levels on the running time of all benchmark 7

constrained MOPs. The results were averaged over 50 independent 6

runs. The time values for each problem are normalized.
f−2 5

0
0 0.01 0.02 0.03 0.04 0.05 0.06
f−1

Fig. 7 The nondominated solutions obtained by pa-ODEMO (top)

and NSGA-II (bottom) on two-bar truss design problem.

Fig. 6 The two-bar truss design problem. 99533.26kPa) and (0.052769m3, 8433.61kPa). Thus pa-
ODEMO provides a wider spread than NSGA-II. If mini-
mum volume is desired, our approach gives a value as low as
[22]. The problem is shown in Fig. 6, and the mathematical 0.004224m3. If minimization of stress is important, it finds
description of the problem is as follows [5]. a solution with stress as low as 1741.37kPa. Moreover, from
Minimize Fig. 7, we can see that pa-ODEMO is able to maintain
 a uniform distribution solutions and find a wide spread of
fvolume = f1 (x) = x1 (16 + y 2 )0.5 + x2 (1 + y 2 )0.5
Pareto-optimal solutions.
fstress,AC = f2 (x) = 20(16 + y 2 )0.5 /(x1 y)
(15) 6.5.2 Welded beam design
subject to
The second problem is selected from [21], which is to be de-
⎧
⎨ fvolume ≤ 0.1 signed for minimum cost and minimum end deflection sub-
fstress,AC ≤ 100, 000 ject to constraints on shear stress, bending stress and buck-
⎩
fstress,BC = 80(1 + y 2 )0.5 /(x2 y) ≤ 100, 000 ling load (Fig. 8). The mathematical formulation of the two
objective optimization problem is as follows.
where 1 ≤ y ≤ 3, and x1 , x2 ≥ 0. Minimize
The maximal NFFEs of this problem is 10,000 for both 
f1 (x) = 1.10471h2l + 0.04811tb(14.0 + t)
pa-ODEMO and NSGA-II. The nondominated solutions (16)
f2 (x) = δ(x) = 2.1952/(t3b)
obtained with pa-ODEMO and NSGA-II are shown in Fig. 7.
From Fig. 7, it can be observed that the Pareto front of pa- subject to
⎧
ODEMO is similar to the one of NSGA-II. The solutions ob- ⎪
⎪ e1 (x) = 13, 600 − τ (x) ≥ 0
tained with pa-ODEMO are spread in the range (0.004224m3, ⎨
e2 (x) = 30, 000 − σ(x) ≥ 0
99987kPa) and (0.098645m3, 1741.37kPa), and those ob- ⎪
⎪ e (x) = b − h ≥ 0
⎩ 3
tained with NSGA-II are spread in the range (0.0043319m3, e4 (x) = Pc(x) − 6, 000 ≥ 0
 15

Welded Beam
0.014
pae−ODEMO

0.012

0.01

0.008

f−2
0.006

0.004

0.002

0
0 5 10 15 20 25 30 35 40
f−1

−3 Welded Beam
Fig. 8 The welded beam design problem. 9
x 10

NSGA−II
8

where 7

⎧ 6

⎪ lτ  τ 
⎪ τ (x) = (τ ) + (τ ) + √0.25(l2 +(h+t)2 )
 2  2
⎪ 5
⎪
⎪ f−2
⎪
⎪ √ 4
⎪
⎪ 
⎨ τ = 6, 000/( 2hl) √ 3
6,000(14.0+0.5l) 0.25(l2 +(h+t)2 )
⎪ τ  = 2[0.707hl(l2 /12+0.25(l2 +(h+t)2 ))]
⎪
⎪
2

⎪ σ(x) = 504, 000/(t2b)

⎪
⎪
⎪ 1
⎪
⎪ Pc(x) = 64, 764.022(1 − 0.0282346t)tb3
⎩ 0
0.125 ≤ h, b ≤ 5.0, and0.1 ≤ l, t ≤ 10.0 0 5 10 15 20
f−1
25 30 35 40

Fig. 9 The nondominated solutions obtained by pa-ODEMO (top)

Fig. 9 shows the non-dominated solutions obtained af-
and NSGA-II (bottom) on welded beam design problem.
ter 15,000 NFFEs for both pa-ODEMO and NSGA-II. It
can be seen that pa-ODEMO is also capable of maintaining
a uniform distribution of solutions. pa-ODEMO found the
minimal cost solution as 2.8959 units with deflection 0.0131
inches, and the minimal deflection as 0.00044 with a cost
of 36.6172 units. For NSGA-II, the minimal cost is 3.0294
units for deflection of 0.0088 units, and the minimal deflec-
tion is 0.000439 with a cost of 37.4018 units. The extreme
solutions are captured well in pa-ODEMO. Again, it is in-
dicated that our proposed approach is efficient and is able to
Fig. 10 The speed reducer design problem.
find a wide variety of Pareto-optimal solutions.

6.5.3 Speed reducer design subject to

⎧
The third design problem is the speed reducer design prob- ⎪
⎪ e1 (x) = 1/27 − 1/(x1 x22 x3 ) ≥ 0
⎪
⎪ e2 (x) = 1/397.5 − 1/(x1 x22 x23 ) ≥ 0
lem shown in Fig. 10. This problem has also been studied in ⎪
⎪
⎪
⎪
[5], [12], [15], [24], and so on. The problem is described as ⎪
⎪ e3 (x) = 1/1.93 − x34 /(x2 x3 x46 ) ≥ 0
⎪
⎪
follows [5]. ⎪
⎪ e4 (x) = 1/1.93 − x35 /(x2 x3 x47 ) ≥ 0
⎪
⎪
Minimize ⎪
⎪ e (x) = 40 − x2 x − 3 ≥ 0
⎨ 5
⎧ e6 (x) = 12 − x1 /x2 ≥ 0
⎪
⎪ f = f1 (x) = ⎪
⎪ e7 (x) = x1 /x2 − 5 ≥ 0
⎪ weight
⎪ 2 2
⎪
⎪
⎪
⎪
⎪ 0.7854x 1 x2 (10x3 /3 + 14.933x3 − 43.0934) ⎪
⎪ e 8 (x) = x4 − 1.5x6 − 1.9 ≥ 0
⎨ ⎪
⎪
−1.508x1 (x6 + x27 ) + 7.477(x36 + x37 )
2
⎪
⎪ e 9 (x) = x5 − 1.1x7 − 1.9 ≥ 0
⎪
(17) ⎪
⎪
⎪
⎪ +0.7854(x4x26 + x5 x27 ) ⎪
⎪ e10 (x) = 1, 300 − f2 (x) ≥ 0
⎪
⎪ ⎪
⎪
⎪
745x4 2
⎩ fstress = f2 (x) = ( x2 x3 ) +1.69×10
7
⎪
⎩
745x
( x x 5 )2 +1.275×108
0.1x3
e11 (x) = 1, 100 − 2 3
0.1x3
≥0
6 7
16

1300
Speed reducer minimum distance between the radii, maximum length of
pae−ODEMO the brake, pressure, temperature and torque limitations. The
1200
problem is a mixed, constrained, multiobjective problem.
1100
The mathematical description of this problem is as follows
[21].
1000
Minimize

f−2

900 f1 (x) = 4.9 × 10−5 (x22 − x21 )(x4 − 1)

9.82×106 (x22 −x21 ) (18)
800
f2 (x) = x3 x4 (x3 −x 3)
2 1

700 subject to
⎧
600 ⎪ e1 (x) = (x2 − x1 ) − 20 ≥ 0
⎪
⎪
⎪
⎪
⎪ e2 (x) = 30 − 2.5(x4 + 1) ≥ 0
2500 3000 3500 4000 4500 5000 5500 6000
f−1
⎨
Speed reducer
e3 (x) = 0.4 − x3 /(3.14(x22 − x21 )) ≥ 0
1300 ⎪
⎪ e4 (x) = 1 − 2.22×10
−3
x3 (x32 −x31 )
≥0
NSGA−II ⎪
⎪ (x22 −x21 )2
⎪
⎪ 2.66×10−2 x3 x4 (x32 −x31 )
1200 ⎩ e5 (x) = − 900 ≥ 0
(x2 −x2 )
2 1

where 55 ≤ x1 ≤ 80, 75 ≤ x2 ≤ 110, 1, 000 ≤ x3 ≤

1100

1000 3, 000, and 2 ≤ x4 ≤ 20.

f−2

The maximal NFFEs of this problem is 5,000 for both

900
pa-ODEMO and NSGA-II. The nondominated solutions
800 obtained with pa-ODEMO and NSGA-II are shown in Fig. 12.
It is presented from Fig. 12 that both algorithms are able
700
to obtain a wide variety of solutions which are uniformly
600
2500 3000 3500 4000 4500 5000 5500 6000
spread. The solutions obtained with pa-ODEMO are spread
f−1
in the range (0.1274, 16.6549) and (2.8684, 2.0906), and
Fig. 11 The nondominated solutions obtained by pa-ODEMO (top) those obtained with NSGA-II are spread in the range (0.1293,
and NSGA-II (bottom) on speed reducer design problem. 17.598) and (2.7915, 2.1127). It is interesting to note that
NSGA-II found the minimal mass of the brake as 0.1293
with stopping time 17.598, this solution is dominated by
where 2.6 ≤ x1 ≤ 3.6, 0.7 ≤ x2 ≤ 0.8, 17 ≤ x3 ≤ 28, the minimal mass of the brake as 0.1274 with stopping time
7.3 ≤ x4 , x5 ≤ 8.3, 2.9 ≤ x6 ≤ 3.9, and 5.0 ≤ x7 ≤ 5.5. 16.6549 obtained with pa-ODEMO.
The Pareto fronts are presented in Fig. 11 obtained with Moreover, in order to verify our approach, the four met-
pa-ODEMO and NSGA-II after 15,000 NFFEs. pa-ODEMO rics (convergence metric, diversity metric, C metric, and
results in an extended Pareto curve between (2775.02, 1298.01) running time) are applied in the four engineering MOPs.
and (5897.4, 695.348). And the solutions obtained with NSGA- Note that in all the four problems we generated the true (or
II are spread in the range (2772.08, 1299.8) and (5728.87, global) Pareto-optimal front by enumeration, using parallel
696.726). Thus both have a wide variety of alternatives. The processing techniques (some of these fronts require a rather
proposed pa-ODEMO solutions are very competitive with high computational effort to be generated by enumeration).
NSGA-II solutions in terms of both closeness to the true op- Table 3 shows the results of pa-ODEMO and NSGA-II.
timum front and their spread. It is worth to point out that the The statistics shown are based on 50 independent runs with
solutions in the horizontal segment of the Pareto front are different random seeds. From this table it can be observed
obtained with pa-ODEMO because of the pa-dominance that NSGA-II requires more running time than pa-ODEMO
adopted in pa-ODEMO. for all four engineering problems, and hence it has larger
time complexity. pa-ODEMO is able to obtain better con-
6.5.4 Disc brake design vergence values than NSGA-II for all engineering MOPs.
In addition, for the C metric, except for Disc brake design
The fourth example consists in optimizing the disc brake de- problem, NSGA-II gives worse C values than pa-ODEMO.
sign problems studied by [21]. The objectives of the design Moreover, pa-ODEMO gives smaller diversity values than
are to minimize the mass of the brake and to minimize the NSGA-II for three engineering problems out of four. For the
stopping time. The variables are the inner radius of the discs, speed reducer design problem the diversity of pa-ODEMO
outer radius of the discs, the engaging force and the num- is worse than NSGA-II, the reason is that pa-ODEMO re-
ber of friction surfaces and are represented as x1 , x2 , x3 tains only a few solutions in the horizontal segment of the
and x4 respectively. The constraints for the design include Pareto-optimal front [see Fig. 11].
 17

Table 3 Comparison of pa-ODEMO and NSGA-II on the four engineering problems considered in the study showing the mean, and standard
deviation (SD) values for set coverage metric (C , where X1 and X2 represent pa-ODEMO and NSGA-II, respectively), convergence metric (γ ),
and spread metric (Δ). All results have been averaged over 50 independent runs. A result with boldface indicates better value obtained.
C γ Δ Time (s)
Problem Statistic
C(X1, X2) C(X2, X1) pa-ODEMO NSGA-II pa-ODEMO NSGA-II pa-ODEMO NSGA-II
Two-bar truss Mean 0.95900 0.00240 254.9408 439.7384 0.87393 0.93725 0.26560 0.30660
SD 0.01669 0.00497 48.2813 52.8976 0.05032 0.02425 0.02685 0.00876
Welded beam Mean 0.19620 0.14481 0.09169 0.16875 0.58607 0.88987 0.37520 0.46880
SD 0.08366 0.04019 0.00733 0.08030 0.04366 0.11976 0.01940 0.02451
Speed reducer Mean 0.26524 0.05000 2.69281 3.0771 0.84041 0.79717 0.42500 0.46260
SD 0.13005 0.03411 0.24051 0.10782 0.20085 0.06608 0.00671 0.00876
Disc brake Mean 0.19800 0.20418 0.05175 0.08203 0.61511 0.66542 0.13780 0.16880
SD 0.09348 0.09952 0.00579 0.01957 0.04581 0.01431 0.00716 0.00661

18
Disc brake 7 Conclusions
pae−ODEMO
16

14
In this paper, an efficient multiobjective DE algorithm is pre-
sented to solve constrained MOPs. It is characterized by a)
12
employing the orthogonal design method with quantization
technique to generate the initial population, b) adopting an
f−2

10

8 archive to store the nondominated solutions and employing

the new Pareto-adaptive -dominance to update the archive
6
at each generation, c) employing a new constraint-handling
4
method to handle the constraints, and d) using a hybrid se-
2 lection mechanism in which a random selection and elitist
0 0.5 1 1.5 2 2.5 3
f−1 selection are interleaved in order to allow the archive solu-
Disc brake tion to guide the search towards the Pareto-optimal front.
18
NSGA−II
16 Seven benchmark constrained MOPs are chosen to test
the capabilities of the algorithm in handling mathematically
14
complex problems. And four well designed problems are se-
12
lected to validate the efficiency and applicability of the al-
gorithm for multiobjective design optimization. Compared
f−2

10

8
with NSGA-II, one of the best MOEAs available at present,
the results show that pa-ODEMO are very competitive with
6
NSGA-II in terms of both closeness to the true optimum
4 front and their spread for the four all test problems. The
2 pa-ODEMO technique can yield efficient, uniformly dis-
0 0.5 1 1.5 2 2.5 3
f−1 tributed, near-complete and near-optimal Pareto-optimal so-
lutions for all test MOPs used in this work. So, we can con-
Fig. 12 The nondominated solutions obtained by pa-ODEMO (top)
and NSGA-II (bottom) on disc brake design problem. clude that our approach is a viable alternative to efficient
multiobjective optimization.

For the proposed pa-ODEMO, we have only chosen a

reasonable set of value and have not made any effort in find-
From the above analysis we can see that the pa-ODEMO ing the best parameter settings. Our future study consists on
solutions are very competitive with NSGA-II solutions in investigating the effect of different parameter settings on the
terms of both closeness to the true optimum front and their performance of pa-ODEMO.
spread for all test problems used in this study. pa-ODEMO
does not have any difficulty in achieving a good spread of
Pareto-optimal solutions for constrained multiobjective op- Acknowledgements The authors would like to acknowledge the anony-
timization. The results obtained for both benchmark con- mous reviewers and the Editor for their useful comments and construc-
tive suggestions. The authors also sincerely thank Dr. A.G. Hernández-
strained MOPs and engineering design problems amply demon-
Dı́az for providing us with the pa-MyDE code and his suggestions on
strate that the pa-ODEMO technique can yield efficient, our work. This work is supported by the Humanities Base Project of
uniformly distributed, near-complete and near-optimal Pareto- Hubei Province under grant No. 2004B0011 and the Natural Science
optimal solutions for multiobjective optimization. Foundation of Hubei Province under grant No. 2003ABA043.
18

References Optimization. Transactions of the Japan Society for Aeronautical

and Space Sciences 50:56 – 62
1. Abbass HA, Sarker R, Newton C (2001) PDE: A Pareto-frontier 20. Rajasekaran S, Lavanya S (2007) Hybridization of Genetic Algo-
Differential Evolution Approach for Multi-objective Optimization rithm with Immune System for Optimization Problems in Struc-
Problems. In: Proceedings of the 2001 IEEE Congress on Evolu- tural Engineering. Struct Multidisc Optim 34:415 – 429
tionary Computation CEC2001, pp 971 – 978 21. Ray T, Liew KM (2002) A Swarm Metaphor for Multiobjective
2. Cai ZH, Gong WY, Huang YQ (2007) A Novel Differential Evo- Design Optimization. Engineering Optimization 34:141 – 153
lution Algorithm based on -domination and Orthogonal Design 22. Reddy MJ, Kumar DN (2007) An efficient Multi-objective Opti-
Method for Multiobjective Optimization. In: Obayashi S et al (ed) mization Algorithm Based on Swarm Intelligence for Engineering
Proceedings of the Fourth International Conference on Evolution- Design. Engineering Optimization 39:49 – 68
ary Multi-Criterion Optimization (EMO-07), pp 286 – 301 23. Robič T, Filipič B (2005) DEMO: Differential Evolution for Mul-
3. Coello CAC (2002) Theoretical and Numerical Constraint- tiobjective Optimization. In: Coello CAC et al (ed) Proceedings
handling Techniques Used with Evolutionary Algorithms: A Sur- of the Third International Conference on Evolutionary Multi-
vey of the State of the Art. Computer Methods in Applied Me- Criterion Optimization (EMO-05), pp 520 – 533
chanics and Engineering 191:1245 – 1287 24. Santana-Quintero LV, Coello CAC (2005) An Algorithm Based on
4. Coello CAC (2006) Evolutionary Multi-objective Optimization: Differential Evolution for Multi-Objective Problems. International
A Historical View of the Field. IEEE Computational Intelligence Journal of Computational Intelligence Research 1:151 – 169
Magazine 1:28 – 36 25. Srinivas N, Deb K (2000) Multi-objective Function Optimization
5. Coello CAC, Pulido G (2005) Multiobjective Structural Optimiza- Using Nondominated Sorting Genetic Algorithms. Evolutionary
tion Using a Microgenetic Algorithm. Struct Multidisc Optim Computation 2:221 – 248
30:388 – 403 26. Storn R, Price K (1997) Differential Evolution - A Simple and
6. Deb K, Pratap A, Meyarivan T (2001) Constrained Test Problems Efficient Heuristic for Global Optimization Over Continuous
for Multi-objective Evolutionary Optimization. In: Zitzler E et al Spaces. Journal of Global Optimization 11:341 – 359
(ed) Proceedings of the First International Conference on Evolu- 27. Storn R, Price K (2003) Home page of differential evolution. Tech.
tionary Multi-Criterion Optimization (EMO-01) 284 – 298 rep., online at http://www.ICSI.Berkeley.edu/˜storn/code.html.
7. Deb K, Thiele L, Laumanns M, Zitzler E (2001) Scalable Test 28. Xue F, Sanderson AC, Graves RJ (2003) Pareto-based Multi-
Problems for Evolutionary Multi-Objective Optimization. Tech. objective Differential Evolution. In: Proceedings of the 2003 IEEE
Rep. TIK-Technical Report No. 112 Congress on Evolutionary Computation CEC2003, pp 862 – 869
8. Deb K, Pratap A, Agarwal, S, Meyarivan T (2002) A Fast and 29. Zeng SY, Kang LS, Ding LX (2004) An Orthogonal Multiob-
Elitist Multiobjective Genetic Algorithm: NSGA - II. IEEE Trans- jective Evolutionary Algorithm for Multi-objective Optimization
actions on Evolutionary Computation 6:182 – 197 Problems with Constraints. Evolutionary Computation 12:77 – 98
9. Deb K, Mohan M, Mishra S (2003) Towards a Quick Computa- 30. Zitzler E, Thiele L (1999) Multiobjective Evolutionary Algo-
tion of Well-spread Pareto-optimal Solutions. In: Fonseca CM et al rithms: A Comparative Sase Study and the Strength Pareto Ap-
(ed) Proceedings of the Second International Conference on Evo- proach. IEEE Transactions on Evolutionary Computation 3:257 –
lutionary Multi-Criterion Optimization (EMO-03), pp 222 – 236 271
10. Deb K, Mohan M, Mishra S (2005) Evaluating the -Domination 31. Zitzler E, Deb K, Thiele L (2000) Comparison of Multiobjective
Based Multi-Objective Evolutionary Algorithm for a Quick Com- Evolutionary Algorithms: Empirical Results. Evolutionary Com-
putation of Pareto-Optimal Solutions. Evolutionary Computation putation 8:173 – 195
13:501 – 525 32. Zitzler E, Laumanns M, Thiele L (2001) Spea2: Improving the
11. Fang KT, Ma CX (2001) Orthogonal and Uniform Design (in Chi- strength pareto evolutionary algorithm. Tech. Rep. Technical Re-
nese). Science Press, Beijing, 2001 port 103, Computer Engineering and Networks Laboratory
12. Farhang-Mehr A, Azarm S (2002) Entropy-based Multi-objective 33. Zitzler E, Thiele L, Laumanns M, Fonseca C, et al (2003) Perfor-
Genetic Algorithm for Design Optimization. Struct Multidisc Op- mance Assesement of Multiobjective Optimizer: An Analysis and
tim 24:351 – 361 Review. IEEE Transactions on Evolutionary Computation 7:117 –
132
13. Hernández-Dı́az AG, Santana-Quintero LV, Coello CAC,
et al (2006) Pareto-adaptive -dominance. Tech. Rep. Resport
EVOCINV-02-2006
14. Kukkonen S, Lampinen J (2005) GDE3: The third Evolution Step
of Generalized Differential Evolution. In: Proceedings of the 2005
IEEE Congress on Evolutionary Computation CEC2005, pp 443 –
450
15. Kurpati A, Azarm S, Wu J (2002) Constraint Handling Improve-
ments for Multiobjective Genetic Algorithms. Struct Multidisc
Optim 23:204 – 213
16. Laumanns M, Thiele L, Deb K, Zitzler E (2002) Combining Con-
vergence and Diversity in Evolutionary Multi-objective Optimiza-
tion. Evolutionary Computation 10:263 – 282
17. Leung YW, , Wang Y (2001) An Orthogonal Genetic Algo-
rithm with Quantization for Global Numerical Optimization. IEEE
Transactions on Evolutionary Computation 5:41 – 53
18. Madavan NK (2002) Multiobjective Optimization Using a Pareto
Differential Evolution Approach. In: Proceedings of the 2002
IEEE Congress on Evolutionary Computation CEC2002, pp 1145
– 1150
19. Oyama A, Shimoyama K, Fujii K (2007) New Constraint-handling
Method for Multi-objective and Multi-Monstraint Evolutionary

View publication stats