Problem Definitions and Evaluation Criteria for the CEC 2017

Special Session and Competition on Single Objective

Real-Parameter Numerical Optimization

N. H. Awad1, M. Z. Ali2, P. N. Suganthan1, J. J. Liang3 and B. Y. Qu3

1
School of EEE, Nanyang Technological University, Singapore
2
School of Computer Information Systems, Jordan University of Science and Technology, Jordan
3
School of Electrical Engineering, Zhengzhou University, Zhengzhou, China
[email protected], [email protected], [email protected], [email protected]

Technical Report, Nanyang Technological University, Singapore

And
Jordan University of Science and Technology, Jordan
And
Zhengzhou University, Zhengzhou China

Modified on October 15th 2016

 Research on the single objective optimization algorithms is the basis of the research on
the more complex optimization algorithms such as multi-objective optimizations algorithms,
niching algorithms, constrained optimization algorithms and so on. All new evolutionary and
swarm algorithms are tested on single objective benchmark problems. In addition, these
single objective benchmark problems can be transformed into dynamic, niching composition,
computationally expensive and many other classes of problems.
In the recent years various kinds of novel optimization algorithms have been proposed
to solve real-parameter optimization problems, including the CEC’05, CEC’13, CEC’14 and
CEC’16 Special Session on Real-Parameter Optimization[1][2][2]. Considering the comments
on the CEC’14 test suite, we organize a new competition on real parameter single objective
optimization.
For this competition, we are developing benchmark problems with several novel
features such as new basic problems, composing test problems by extracting features
dimension-wise from several problems, graded level of linkages, rotated trap problems, and
so on. This competition excludes usage of surrogates or meta-models.
This special session is devoted to the approaches, algorithms and techniques for solving
real parameter single objective optimization without making use of the exact equations of the
test functions. We encourage all researchers to test their algorithms on the CEC’17 test suite
which includes 29 benchmark functions. The participants are required to send the final
results in the format specified in the technical report to the organizers. The organizers will
present an overall analysis and comparison based on these results. We will also use statistical
tests on convergence performance to compare algorithms that generate similar final solutions
eventually. Papers on novel concepts that help us in understanding problem characteristics
are also welcome.
The C and Matlab codes for CEC’17 test suite can be downloaded from the website
given below:
http://www.ntu.edu.sg/home/EPNSugan/index_files/CEC2017
1. Introduction to the CEC’17 Benchmark Suite

1.1 Some Definitions:

All test functions are minimization problems defined as following:

Min f(x), x = [ x1 , x2 ,..., xD ]T

D: dimensions.
oi1 = [oi1 , oi 2 ,..., oiD ]T : the shifted global optimum (defined in “shift_data_x.txt”), which is

randomly distributed in [-80,80]D. Different from CEC’13 and similar to CEC’14 each
function has a shift data for CEC’17.
All test functions are shifted to o and scalable.
For convenience, the same search ranges are defined for all test functions.
Search range: [-100,100]D.
Mi: rotation matrix. Different rotation matrix are assigned to each function and each basic
function.
Considering that in the real-world problems, it is seldom that there exist linkages among all
variables. In CEC’17, same as CEC’15 the variables are divided into subcomponents
randomly. The rotation matrix for each subcomponents are generated from standard
normally distributed entries by Gram-Schmidt ortho-normalization with condition number c
that is equal to 1 or 2.
1.2 Summary of the CEC’17 Test Suite

Table I. Summary of the CEC’17 Test Functions

No. Functions Fi*=Fi(x*)

Unimodal 1 Shifted and Rotated Bent Cigar Function 100
Functions 2 Shifted and Rotated Zakharov Function 200
3 Shifted and Rotated Rosenbrock’s Function 300
4 Shifted and Rotated Rastrigin’s Function 400
5 Shifted and Rotated Expanded Scaffer’s F6 500
Function
Simple
6 Shifted and Rotated Lunacek Bi_Rastrigin 600
Multimodal
Function
Functions
7 Shifted and Rotated Non-Continuous Rastrigin’s 700
Function
8 Shifted and Rotated Levy Function 800
9 Shifted and Rotated Schwefel’s Function 900
10 Hybrid Function 1 (N=3) 1000
11 Hybrid Function 2 (N=3) 1100
12 Hybrid Function 3 (N=3) 1200
13 Hybrid Function 4 (N=4) 1300
Hybrid 14 Hybrid Function 5 (N=4) 1400
Functions 15 Hybrid Function 6 (N=4) 1500
16 Hybrid Function 6 (N=5) 1600
17 Hybrid Function 6 (N=5) 1700
18 Hybrid Function 6 (N=5) 1800
19 Hybrid Function 6 (N=6) 1900
20 Composition Function 1 (N=3) 2000
21 Composition Function 2 (N=3) 2100
22 Composition Function 3 (N=4) 2200
23 Composition Function 4 (N=4) 2300
24 Composition Function 5 (N=5) 2400
Composition
25 Composition Function 6 (N=5) 2500
Functions
26 Composition Function 7 (N=6) 2600
27 Composition Function 8 (N=6) 2700
28 Composition Function 9 (N=3) 2800
29 Composition Function 10 (N=3) 2900
Search Range: [-100,100]D

*Please Note: These problems should be treated as black-box problems. The explicit
equations of the problems are not to be used.
1.3 Definitions of the Basic Functions

1) Bent Cigar Function

D
f1 ( x ) = x + 10
2
1
6
x
i =2
2
i (1)

2) Zakharov Function
2 4
D
 D   D 
f3 ( x ) =  x +   0.5 xi  +   0.5 xi 
2
i (2)
i =1  i =1   i =1 

3) Rosenbrock’s Function
D −1
f 4 ( x ) =  (100( xi2 − xi +1 ) 2 + ( xi − 1) 2 ) (3)
i =1

4) Rastrigin’s Function
D
f5 ( x ) =  ( xi 2 − 10 cos(2 xi ) + 10) (4)
i =1

5) Expanded Schaffer’s F6 Function

(sin 2 ( x 2 + y 2 ) − 0.5)
Schaffer’s F6 Function: g ( x, y ) = 0.5 +
(1 + 0.001( x 2 + y 2 )) 2

f 6 ( x ) = g ( x1 , x2 ) + g ( x2 , x3 ) + ... + g ( xD −1 , xD ) + g ( xD , x1 ) (5)

6) Lunacek bi-Rastrigin Function

D D D
f 7 ( x) = min( ( xi − 0 ) 2 , dD + s  ( xi − 1 ) 2 ) + 10( D −  cos(2 zi ))
) ) )
i =1 i =1 i =1

02 − d 1
0 = 2.5, 1 = − , s = 1− ,d =1
s 2 D + 20 − 8.2
10( x − o) )
y= , xi = 2sign( xi *) yi + 0 , for i = 1, 2,..., D
100
)
z = 100 ( x − 0 )
(6)
7) Non-continuous Rotated Rastrigin’s Function
D
f8 ( x) =  ( zi 2 − 10 cos(2 zi ) + 10) + f13 *
i =1

) )
) 5.12( x − o)  xi if xi  0.5
x = M1 , yi =  ) ) for i = 1, 2,..., D
100 round(2 xi ) / 2 if xi  0.5 (7)
z = M110 M 2Tasy
0.2
(Tosz ( y ))
i −1

Where  : a diagonal matrix in D dimensions with the i diagonal element as ii = 
th 2( D −1)
,
i=1,2,…,D.
i −1
1+  xi

Tasy : if xi  0, xi = xi D −1
, for i = 1,..., D [4]

Tosz: for xi = sign( xi ) exp( xˆi + 0.049(sin(c1 xˆi ) + sin(c2 xˆi ))), for i = 1 and D [4]

−1 if xi  0
log( xi ) if xi  0 
where xˆi =  , sign( xi ) =  0 if xi = 0
 0 otherwise  1 otherwise


 10 if xi  0 7.9 if xi  0
c1 =  , and c2 = 
5.5 otherwise  3.1 otherwise

8) Levy Function

D −1
f9 ( x ) = sin 2 ( w1 ) +  ( wi − 1) 2 1 + 10sin 2 (πwi + 1)  + ( wD − 1) 2 1 + sin 2 (2 wD ) 
i =1

xi − 1
Where wi = 1 + ,  i = 1,..., D (8)
4

9) Modified Schwefel’s Function

D
f10 ( x ) = 418.9829  D −  g ( zi ), zi = xi +4.209687462275036e+002
i =1


 zi sin( zi ) if zi  500
1/2

 (9)
 ( z − 500) 2
g ( zi ) =  (500 − mod( zi ,500)) sin( 500 − mod( zi ,500) ) − i if zi  500
 10000 D
 ( zi + 500) 2
 (mod( z ,500) − 500) sin( mod( z , 500) − 500 ) − if zi  −500

i i
10000 D
10) High Conditioned Elliptic Function
D i −1
f11 ( x ) =  (106 ) D −1 xi2 (10)
i =1

11) Discus Function

D
f12 ( x ) = 106 x12 +  xi2 (11)
i =2

12) Ackley’s Function

1 D 2 1 D
f13 ( x ) = −20 exp(−0.2 
D i =1
xi ) − exp(  cos(2πxi )) + 20 + e
D i =1
(12)

13) Weierstrass Function

D k max k max
f14 ( x ) =  (  [a k cos(2πb k ( xi + 0.5))]) − D  [a k cos(2πb k  0.5)] (13)
i =1 k =0 k =0

a=0.5, b=3, kmax=20

14) Griewank’s Function

D
xi 2 D
x
f15 ( x ) =  −  cos( i ) + 1 (14)
i =1 4000 i =1 i

15) Katsuura Function

32 2 j x − round(2 j x ) 10
10 D 10
f16 ( x ) = 2  (1 + i  )D − 2
i i 1.2

j
(15)
D i =1 j =1 2 D

16) HappyCat Function

D 1/4 D D
f17 ( x ) = x −D
i =1
2
i + (0.5 xi2 +  xi ) / D + 0.5
i =1 i =1
(16)

17) HGBat Function

D D 1/2 D D
f18 ( x ) = ( xi2 ) 2 − ( xi ) 2 + (0.5 xi2 +  xi ) / D + 0.5 (17)
i =1 i =1 i =1 i =1

18) Expanded Griewank’s plus Rosenbrock’s Function

f19 ( x ) = f 7 ( f 4 ( x1 , x2 )) + f 7 ( f 4 ( x2 , x3 )) + ... + f 7 ( f 4 ( xD −1 , xD )) + f 7 ( f 4 ( xD , x1 )) (18)

19) Schaffer's F7 Function

( ( ))
2
 1 D −1 
f 20 ( x ) =   si . sin ( 50.0s ) + 1  , si = xi2 + xi2+1
0.2
(19)
 D − 1 i =1
i


1.4 Definitions of the CEC’17 Test Suite

A. Unimodal Functions:

1) Shifted and Rotated Bent Cigar

F1 ( x ) = f1 (M ( x − o1 )) + F1 * (20)

Figure 1. 3-D map for 2-D function

Properties:
➢ Unimodal
➢ Non-separable
➢ Smooth but narrow ridge

2) Shifted and Rotated Zakharov Function

F3 ( x ) = f3 (M( x − o3 )) + F3 * (21)
 Figure 3. 3-D map for 2-D function
Properties:
➢ Unimodal
➢ Non-separable

B. Multimodal Functions

3) Shifted and Rotated Rosenbrock’s Function

2.048( x − o4 )
F4 ( x ) = f 4 (M( ) + 1) + F4 * (22)
100

Figure 4(a). 3-D map for 2-D function

 Figure 4(b).Contour map for 2-D function
Properties:
➢ Multi-modal
➢ Non-separable
➢ Local optima’s number is huge

4) Shifted and Rotated Rastrigin’s Function

F5 ( x ) = f5 (M ( x − o5 )) + F5 * (23)

Figure 5(a). 3-D map for 2-D function

Figure 5(b).Contour map for 2-D function

Properties:
➢ Multi-modal
➢ Non-separable
➢ Local optima’s number is huge and second better local optimum is far from the
global optimum.

5) Shifted and Rotated Schaffer’s F7 Function

0.5( x − o6 )
F6 ( x ) = f 20 (M( )) + F6 * (24)
100
 Figure 6(a). 3-D map for 2-D function

Figure 6(b).Contour map for 2-D function

Properties:
➢ Multi-modal
➢ Non-separable
➢ Asymmetrical
➢ Local optima’s number is huge

6) Shifted and Rotated Lunacek Bi-Rastrigin’s Function

600( x − o7 )
F7 ( x ) = f 7 (M( )) + F7 * (25)
100
 Figure 7(a). 3-D map for 2-D function

Figure 7(b).Contour map for 2-D function

Properties:
➢ Multi-modal
➢ Non-separable
➢ Asymmetrical
➢ Continuous everywhere yet differentiable nowhere

7) Shifted and Rotated Non-Continuous Rastrigin’s Function

5.12( x − o8 )
F8 ( x ) = f8 ( ) + F8 * (26)
100
 Figure 8(a). 3-D map for 2-D function

Figure 8(b).Contour map for 2-D function

Properties:
➢ Multi-modal
➢ Non-separable
➢ Asymmetrical
➢ Local optima’s number is huge

8) Shifted and Rotated Levy Function

5.12( x − o9 )
F9 ( x ) = f9 (M( )) + F9 * (27)
100
 Figure 9(a). 3-D map for 2-D function

Figure 9(b).Contour map for 2-D function

Properties:
➢ Multi-modal
➢ Non-separable
➢ Local optima’s number is huge

9) Shifted and Rotated Schwefel’s Function

1000( x − o10 )
F10 ( x ) = f10 ( ) + F10 * (28)
100
 Figure 10(a). 3-D map for 2-D function

Figure 10(b).Contour map for 2-D function

Properties:
➢ Multi-modal
➢ Non-Separable
➢ Local optima’s number is huge and second better local optimum is far from the
global optimum.

C. Hybrid Functions
Considering that in the real-world optimization problems, different subcomponents of the
variables may have different properties[5]. In this set of hybrid functions, the variables are
randomly divided into some subcomponents and then different basic functions are used for
different subcomponents.
F ( x ) = g1 (M1 z1 ) + g 2 (M 2 z2 ) + ... + g N (M N z N ) + F * ( x ) (29)

F(x): hybrid function

gi(x): ith basic function used to construct the hybrid function
N: number of basic functions
z = [ z1 , z2 ,..., z N ]
z1 = [ yS1 , yS2 ,..., ySn ], z2 = [ ySn +1 , ySn +2 ,..., ySn +n ],..., z N = [ yS N −1 , yS N −1 ,..., ySD ]
1 1 1 1 2
 ni +1  ni + 2
i =1 k =1

y = x - oi , S = randperm(1: D)

pi : used to control the percentage of gi(x)

N
ni: dimension for each basic function n
i =1
i =D

N −1
n1 =  p1 D  , n2 =  p2 D  ,..., nN −1 =  pN −1D  , nN = D −  ni
i =1

Properties:
➢ Multi-modal or Unimodal, depending on the basic function
➢ Non-separable subcomponents
➢ Different properties for different variables subcomponents

10) Hybrid Function 1

N=3
p = [0.2, 0.4, 0.4]
g1 : Zakharov Function f3
g2 : Rosenbrock Function f4
g3: Rastrigin’s Function f5

11) Hybrid Function 2

N=3
p = [0.3, 0.3, 0.4]
g1 : High Conditioned Elliptic Function f11
g2 : Modified Schwefel’s Function f10
g3: Bent Cigar Function f1

12) Hybrid Function 3

N=3
p = [ 0.3, 0.3, 0.4]
g1 : Bent Cigar Function f1
g2 : Rosenbrock Function f4
g3: Lunache Bi-Rastrigin Function f7

13) Hybrid Function 4

N=4
p = [0.2, 0.2, 0.2, 0.4]
g1 : High Conditioned Elliptic Function f11
g2 : Ackley’s Function f13
g3: Schaffer’s F7 Function f20
g4: Rastrigin’s Function f5

14) Hybrid Function 5

N=4
p = [0.2, 0.2, 0.3, 0.3]
g1 : Bent Cigar Function f1
g2 : HGBat Function f18
g3: Rastrigin’s Function f5
g4: Rosenbrock’s Function f4

15) Hybrid Function 6

N=4
p = [0.2, 0.2, 0.3, 0.3]
g1 : Expanded Schaffer F6 Function f6
g2 : HGBat Function f18
g3: Rosenbrock’s Function f4
g4: Modified Schwefel’s Function f10

16) Hybrid Function 7

N=5
p = [0.1, 0.2, 0.2, 0.2, 0.3]
g1 : Katsuura Function f16
g2 : Ackley’s Function f13
g3: Expanded Griewank’s plus Rosenbrock’s Function f19
g4: Modified Schwefel’s Function f10
g4: Rastrigin’s Function f5

17) Hybrid Function 8

N=5
p = [0.2, 0.2, 0.2, 0.2, 0.2]
g1 : High Conditioned Elliptic Function f1
g2 : Ackley’s Function f13
g3: Rastrigin’s Function f5
g4: HGBat Function f18
g4: Discus Function f12

18) Hybrid Function 9

N=5
p = [0.2, 0.2, 0.2, 0.2, 0.2]
g1 : Bent Cigar Function f1
g2 : Rastrigin’s Function f5
g3: Expanded Griewank’s plus Rosenbrock’s Function f19
g4: Weierstrass Function f14
g5: Expanded Schaffer’s F6 Function f6

19) Hybrid Function 10

N=6
p = [0.1, 0.1, 0.2, 0.2, 0.2, 0.2]
g1 : Happycat Function f17
g2 : Katsuura Function f16
g3: Ackley’s Function f13
g4: Rastrigin’s Function f5
g5 : Modified Schwefel’s Function f10
g6: Schaffer’s F7 Function f20

D. Composition Functions
N
F ( x ) = {i *[i gi ( x ) + biasi ]} + F * (30)
i =1

F(x): composition function

gi(x): ith basic function used to construct the composition function
N: number of basic functions
oi: new shifted optimum position for each gi(x), define the global and local optima’s
position
biasi: defines which optimum is global optimum
i : used to control each gi(x)’s coverage range, a small  i give a narrow range for

that gi(x)
i : used to control each gi(x)’s height

wi : weight value for each gi(x), calculated as below:

D

1
 (x
j =1
j − oij ) 2
wi = exp(− ) (31)
D 2 D i 2
 (x
j =1
j − oij ) 2

n
Then normalize the weight i = wi /  wi
i =1
 1 j =i
So when x = oi ,  j =  for j = 1, 2,..., N , f ( x) = biasi + f *
0 ji
The local optimum which has the smallest bias value is the global optimum. The
composition function merges the properties of the sub-functions better and maintains
continuity around the global/local optima.
Functions Fi’= Fi-Fi* are used as gi. In this way, the function values of global optima of gi
are equal to 0 for all composition functions in this report.
In CEC’14, the hybrid functions are also used as the basic functions for composition
functions (Composition Function 7 and Composition Function 8). With hybrid functions as
the basic functions, the composition function can have different properties for different
variables subcomponents.
Please Note: All the basic functions that have been used in composition functions are shifted
and rotated functions.

20) Composition Function 1

N= 3，  = [10, 20, 30]
 = [ 1, 1e-6, 1]
bias = [0, 100, 200]
g1: Rosenbrock’s Function F4’
g2: High Conditioned Elliptic Function F11’
g3 Rastrigin’s Function F4’

Figure 11(a). 3-D map for 2-D function

 Figure 11 (b).Contour map for 2-D function
Properties:
➢ Multi-modal
➢ Non-separable
➢ Asymmetrical
➢ Different properties around different local optima

21) Composition Function 2

N=3
 = [10, 20, 30]
 = [1, 10, 1]
bias = [0, 100, 200]
g1: Rastrigin’s Function F5’
g2: Griewank’s Function F15’
g3 Modifed Schwefel's Function F10’

Figure 12(a). 3-D map for 2-D function

 Figure 12(b).Contour map for 2-D function
Properties:
➢ Multi-modal
➢ Non-separable
➢ Asymmetrical
➢ Different properties around different local optima

22) Composition Function 3

N=4
 = [10, 20, 30, 40]
 = [1, 10, 1, 1]
bias = [0, 100, 200, 300]
g1: Rosenbrock’s Function F4’
g2: Ackley’s Function F13’
g3: Modified Schwefel's Function F10’
g4: Rastrigin’s Function F5’

Figure 13(a). 3-D map for 2-D function

 Figure 13(b).Contour map for 2-D function

Properties:
➢ Multi-modal
➢ Non-separable
➢ Asymmetrical
➢ Different properties around different local optima

23) Composition Function 4

N=4
 = [10, 20, 30, 40]
 = [10, 1e-6, 10, 1]
bias = [0, 100, 200, 300]
g1: Ackley’s Function F13’
g2: High Conditioned Elliptic Function F11’
g3: Girewank Function F15’
g4: Rastrigin’s Function F5’

Figure 14(a). 3-D map for 2-D function

 Figure 14(b).Contour map for 2-D function
Properties:
➢ Multi-modal
➢ Non-separable
➢ Asymmetrical
➢ Different properties around different local optima

24) Composition Function 5

N=5
 = [10, 20, 30, 40, 50]
 = [10, 1, 10, 1e-6, 1]
bias = [0, 100, 200, 300, 400]
g1: Rastrigin’s Function F5’
g2: Happycat Function F17’
g3: Ackley Function F13’
g4: Discus Function F12’
g5: Rosenbrock’s Function F4’

Figure 15(a). 3-D map for 2-D function

 Figure 15(b).Contour map for 2-D function
Properties:
➢ Multi-modal
➢ Non-separable
➢ Asymmetrical
➢ Different properties around different local optima

25) Composition Function 6

N=5
 = [10, 20, 20, 30, 40]
 = [ 1e-26, 10, 1e-6, 10,5e-4]
bias = [0, 100, 200, 300, 400]
g1: Expanded Scaffer’s F6 Function F6’
g2: Modified Schwefel's Function F10’
g3: Griewank’s Function F15’
g4: Rosenbrock’s Function F4’
g5: Rastrigin’s Function F5’

Figure 16(a). 3-D map for 2-D function

 Figure 16(b).Contour map for 2-D function
Properties:
➢ Multi-modal
➢ Non-separable
➢ Asymmetrical
➢ Different properties around different local optima

26) Composition Function 7

N=6
 = [10, 20, 30, 40, 50, 60]
 = [10, 10, 2.5, 1e-26, 1e-6, 5e-4]
bias = [0, 100, 200, 300, 400, 500]
g1: HGBat Function F18’
g2: Rastrigin’s Function F5’
g3: Modified Schwefel's Function F10’
g4: Bent-Cigar Function F11’
g4: High Conditioned Elliptic Function F11’
g5: Expanded Scaffer’s F6 Function F6’

Figure 17(a). 3-D map for 2-D function

 Figure 17(b).Contour map for 2-D function
Properties:
➢ Multi-modal
➢ Non-separable
➢ Asymmetrical
➢ Different properties around different local optima

27) Composition Function 8

N=6
 = [10, 20, 30, 40, 50, 60]
 = [10, 10, 1e-6, 1, 1, 5e-4]
bias = [0, 100, 200, 300, 400, 500]
g1: Ackley’s Function F13’
g2: Griewank’s Function F15’
g3: Discus Function F12’
g4: Rosenbrock’s Function F4’
g4: HappyCat Function F17’
g5: Expanded Scaffer’s F6 Function F6’

Figure 18(a). 3-D map for 2-D function

 Figure 18(b).Contour map for 2-D function
Properties:
➢ Multi-modal
➢ Non-separable
➢ Asymmetrical
➢ Different properties around different local optima

28) Composition Function 10

N=3
 = [10, 30, 50]
 = [1, 1, 1]
bias = [0, 100, 200]
g1: Hybrid Function 5 F5’
g2: Hybrid Function 6 F6’
g3: Hybrid Function 7 F7’

Properties:
➢ Multi-modal
➢ Non-separable
➢ Asymmetrical
➢ Different properties around different local optima
➢ Different properties for different variables subcomponents

29) Composition Function 9

N=3
 = [10, 30, 50]
 = [1, 1, 1]
bias = [0, 100, 200]
g1: Hybrid Function 5 F5’
g2: Hybrid Function 8 F8’
g3: Hybrid Function 9 F9’
Properties:
➢ Multi-modal
➢ Non-separable
➢ Asymmetrical
➢ Different properties around different local optima
➢ Different properties for different variables subcomponents

2. Evaluation Criteria

2.1 Experimental Setting

Problems: 30 minimization problems

Dimensions: D=10, 30, 50, 100 (Results only for 10D and 30D are acceptable for the initial
submission; but 50D and 100D should be included in the final version)
Runs / problem: 51 (Do not run many 51 runs to pick the best run)
MaxFES: 10000*D (Max_FES for 10D = 100000; for 30D = 300000; for 50D = 500000;
for 100D = 1000000)
Search Range: [-100,100]D
Initialization: Uniform random initialization within the search space. Random seed is based
on time, Matlab users can use rand ('state', sum(100*clock)).
Global Optimum: All problems have the global optimum within the given bounds and there
is no need to perform search outside of the given bounds for these problems.

Fi ( x*) = Fi (oi ) = Fi *

Termination: Terminate when reaching MaxFES or the error value is smaller than 10-8.

2.2 Results Record

1) Record function error value (Fi(x)-Fi(x*)) after (0.01, 0.02, 0.03, 0.05, 0.1, 0.2, 0.3,
0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0)*MaxFES for each run.
In this case, 14 error values are recorded for each function for each run. Sort the error
values achieved after MaxFES in 51 runs from the smallest (best) to the largest (worst)
 and present the best, worst, mean, median and standard variance values of function
error values for the 51 runs.
Please Notice: Error value smaller than 10-8 will be taken as zero.

2.3 Performance Measure

a) The evaluation criteria will be divided into two parts:

1. 50% summation of mean error values of each problem for all dimensions as
follows:
29 29 29 29
SE = 0.1  ef10 D + 0.2   ef 30 D + 0.3  ef 50 D + 0.4   ef100 D
i =1 i =1 i =1 i =1

where ef is the mean error values for all the functions and SE is the sum of errors

and then find the score for this part as follows:

SE − SEmin
Score1 = (1 − )  50
SE

where SEmin is the minimal sum of errors from all the algorithms.

b) 50% rank based for mean error values for each problem in each dimension as
follows:
29 29 29 29
SR = 0.1  rank10 D + 0.2   rank30 D + 0.3  rank50 D + 0.4   rank100 D
i =1 i =1 i =1 i =1

where SR is the sum of ranks then find the score for this part as follows:
SR − SRmin
Score2 = (1 − )  50
SR

SRmin is the minimal sum of ranks from all the algorithms.

c) Combine the above two parts to find the final score as follows. Higher weight will
be given for higher dimensions:
Score = Score1 + Score2

3. Algorithm Complexity

a) Run the test program below:

 x= 0.55;
for i=1:1000000
x=x + x; x=x/2; x=x*x; x=sqrt(x); x=log(x); x=exp(x); x=x/(x+2);
end
Computing time for the above=T0;
b) Evaluate the computing time just for Function 18. For 200000 evaluations of a certain
dimension D, it gives T1;
c) The complete computing time for the algorithm with 200000 evaluations of the same D
dimensional Function 18 is T2.
)
d) Execute step c five times and get five T2 values. T 2 =Mean(T2)
) )
The complexity of the algorithm is reflected by: T 2 , T1, T0, and ( T 2 -T1)/T0
The algorithm complexities are calculated on 10, 30 and 50 dimensions, to show the
algorithm complexity’s relationship with dimension. Also provide sufficient details on the
computing system and the programming language used. In step c, we execute the complete
algorithm five times to accommodate variations in execution time due adaptive nature of
some algorithms.
Please Note: Similar programming styles should be used for all T0, T1 and T2.
(For example, if m individuals are evaluated at the same time in the algorithm, the
same style should be employed for calculating T1; if parallel calculation is employed for
calculating T2, the same way should be used for calculating T0 and T1. In other word,
the complexity calculation should be fair.)

4. Parameters

Participants must not search for a distinct set of parameters for each problem/dimension/etc.
Please provide details on the following whenever applicable:
a) All parameters to be adjusted
b) Corresponding dynamic ranges
c) Guidelines on how to adjust the parameters
d) Estimated cost of parameter tuning in terms of number of FEs
e) Actual parameter values used.

5. Encoding

If the algorithm requires encoding, then the encoding scheme should be independent of the
specific problems and governed by generic factors such as the search ranges.

6. Results Format

The participants are required to send the final results as the following format to the
organizers and the organizers will present an overall analysis and comparison based on
these results.
Create one txt document with the name “AlgorithmName_FunctionNo._D.txt” for each test
function and for each dimension.
For example, PSO results for test function 5 and D=30, the file name should be
“PSO_5_30.txt”.
Then save the results matrix (the gray shadowing part) as Table II in the file:
Table II. Information Matrix for D Dimensional Function X

***.txt Run 1 Run 2 … Run 51

Function error values when FES=0.01*MaxFES

Function error values when FES=0.02*MaxFES
Function error values when FES=0.03*MaxFES
Function error values when FES=0.05*MaxFES
……
Function error values when FES=0.9*MaxFES
Function error values when FES=MaxFES

Thus 30*4 (10D, 30D, 50D and 100D) files should be zipped and sent to the organizers.
Each file contains a 14*51 matrix.
Notice: All participants are allowed to improve their algorithms further after submitting
the initial version of their papers to CEC2014. And they are required to submit their results
in the introduced format to the organizers after submitting the final version of paper as soon
as possible.

6.1 Results Template

Language: Matlab 2008a

Algorithm: Particle Swarm Optimizer (PSO)
Results
Notice:
Considering the length limit of the paper, only Error Values Achieved with MaxFES are need
to be listed. While the authors are required to send all results (30*4 files described in section
2.2) to the organizers for a better comparison among the algorithms.

Table III. Results for 10D

Func. Best Worst Median Mean Std

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
 23
24
25
26
27
28
29
30

Table IV. Results for 30D

…
Table V. Results for 50D
…
Table VI. Results for 100D
…
Algorithm Complexity
Table VII. Computational Complexity
) )
T0 T1 T2 ( T 2 -T1)/T0
D=10
D=30
D=50

Parameters
a) All parameters to be adjusted
b) Corresponding dynamic ranges
c) Guidelines on how to adjust the parameters
d) Estimated cost of parameter tuning in terms of number of FES
e) Actual parameter values used.

References

[1] P. N. Suganthan, N. Hansen, J. J. Liang, K. Deb, Y.-P. Chen, A. Auger & S. Tiwari,
"Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on
Real-Parameter Optimization," Technical Report, Nanyang Technological University,
Singapore, May 2005 and KanGAL Report #2005005, IIT Kanpur, India, 2005.
[2] J. J. Liang, B. Y. Qu, P. N. Suganthan, Alfredo G. Hernández-Díaz, "Problem
Definitions and Evaluation Criteria for the CEC 2013 Special Session and Competition
on Real-Parameter Optimization", Technical Report 201212, Computational Intelligence
Laboratory, Zhengzhou University, Zhengzhou China and Technical Report,
Nanyang Technological University, Singapore, January 2013.
[3] J. J. Liang, B. Y. Qu, P. N. Suganthan, "Problem Definitions and Evaluation Criteria for
the CEC 2014 Special Session and Competition on Real-Parameter Optimization",
Technical Report 201212, Computational Intelligence Laboratory, Zhengzhou
University, Zhengzhou China and Technical Report, Nanyang Technological
University, Singapore, January 2014.
[4] Joaqu´ın Derrac, Salvador Garcia, Sheldon Hui, Francisco Herrera , Ponnuthurai N.
Suganthan, "Statistical analysis of convergence performance throughout the search: A
case study with SaDE-MMTS and Sa-EPSDE-MMTS," accepted by Symp. DE 2013,
IEEE SSCI 2013, 2012.
[5] Xiaodong Li, Ke Tang, Mohammad N. Omidvar, Zhenyu Yang, and Kai Qin,
Benchmark Functions for the CEC’2013 Special Session and Competition on
Large-Scale Global Optimization, Technical Report, 2013.