Particle Swarm optimisation

2002-04-24 [email protected]
Particle Swarm optimisation: A mini
tutorial
Particle Swarm optimisation
2002-04-24 [email protected]
The inventors (1)
Russell
Eberhart
[email protected]
Particle Swarm optimisation
2002-04-24 [email protected]
The inventors (2)
James
Kennedy
[email protected]
Particle Swarm optimisation
2002-04-24 [email protected]
Part 1: United we stand
Particle Swarm optimisation
2002-04-24 [email protected]
Cooperation example
Particle Swarm optimisation
2002-04-24 [email protected]
Initialization. Positions and velocities
Particle Swarm optimisation
2002-04-24 [email protected]
Neighbourhoods
geographical
social
Particle Swarm optimisation
2002-04-24 [email protected]
The circular neighbourhood
Virtual circle
1
5
7
6
4
3
8
2
Particle 1s 3-
neighbourhood
Particle Swarm optimisation
2002-04-24 [email protected]
Psychosocial compromise
Here I am! The best perf. of
my neighbours
My best
perf.
x
p
g

p
i

v
Particle Swarm optimisation
2002-04-24 [email protected]
The historical algorithm
( )
( )
( ) ( ) ( )
( ) ( ) ( )

+
+
= +
t x p rand
t x p rand
t v
t v
d d g
d d i
d
d
, 2
, 1
, 0
, 0
1
|
|
o
for each particle
update
the
velocity
( ) ( ) 1 ) 1 ( + + = + t v t x t x then move
for each component d
At each time step t
Particle Swarm optimisation
2002-04-24 [email protected]
Random proximity
x
p
g

p
i

v
Hyperparallelepiped => Biased
Particle Swarm optimisation
2002-04-24 [email protected]
Animated illustration
Global
optimum
Particle Swarm optimisation
2002-04-24 [email protected]
Part 2: How to choose parameters
Particle Swarm optimisation
2002-04-24 [email protected]
Type 1 form
2 1 2 1
' ' ) , 0 ( ) , 0 ( + = + = rand rand
2 1
2 1
' '
' '


+
+
=
g i
p p
p

+ + = +
+ = +
) ( ) 1 ( ) 1 (
))) ( ( ) ( ( ) 1 (
t x t v t x
t x p t v t v _
with

>
+ =

4 for
4 2
2
2
k

k
_
else
Usual values:
k=1
=4.1
=> _=0.73
swarm size=20
hood size=3
Particle Swarm optimisation
2002-04-24 [email protected]
5D complex space
4
2
0
-2
-4
2
1
0
-1
-2
6
5
4
3
2
1
0
}
}
Convergence
Non
divergence
A 3D section

Re(y)
Re(v)
Particle Swarm optimisation
2002-04-24 [email protected]
Move in a 2D section (attractor)
1
0.5 0 -0.5 -1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
Im(v)
Re(v)
Particle Swarm optimisation
2002-04-24 [email protected]
Some functions ...
Rosenbrock
Griewank
Rastrigin
Particle Swarm optimisation
2002-04-24 [email protected]
... and some results
30D function PSO Type 1" Evolutionary
algo.(Angeline 98)
Griewank [300] 0.003944 0.4033
Rastrigin [5] 82.95618 46.4689
Rosenbrock [10] 50.193877 1610.359
Optimum=0, dimension=30
Best result after 40 000 evaluations
Particle Swarm optimisation
2002-04-24 [email protected]
Beat the swarm!
Particle Swarm optimisation
2002-04-24 [email protected]
Part 3: Beyond real numbers
0
1
2
3
4
0
1
2
3
0
1
2
3
4
1
2
3
4
5
6 0
1
2
0
1
2
3
4
8
8
Bingo!
Particle Swarm optimisation
2002-04-24 [email protected]
Minimun requirements
( ) ( ) ( ) ( ) ( ) ( ) ' ' , ) ' , ( x f x f x f x f H H x x > v < e
( )
( )
( )
( ) position velocity position
velocity position position
velocity velocity velocity
velocity velocity t coefficien

,
,
,
,

Comparing positions in the

search space H
Algebraic operators
Particle Swarm optimisation
2002-04-24 [email protected]
velocity = pos_minus_pos(position
1
, position
2
)
velocity = linear_combin(o,velocity
1
,|,velocity
2
)
position = pos_plus_vel(position, velocity)
(position,velocity) = confinement(position
t+1
,position
t
)
Pseudo code form
}
algebraic
operators
=>
Particle Swarm optimisation
2002-04-24 [email protected]
Fifty-fifty

x
i
e 1...N
{ }
i = j x
i
= x
j
x
i
= x
i
D/ 2+1
D

1
D / 2

N=100, D=20. Search space: [1,N]
D

105 evaluations:
63+90+16+54+71+20+23+60+38+15
=
12+48+13+51+36+42+86+26+57+79 (=450)
Particle Swarm optimisation
2002-04-24 [email protected]
Knapsack

x
i
e 1...N
{ }
i = j x
i
= x
j
x
i
= S
ieI , I = D, I c 1, N
{ }

N=100, D=10, S=100,
870 evaluations:
run 1 => (9, 14, 18, 1, 16, 5, 6, 2, 12, 17)
run 2 => (29, 3, 16, 4, 1, 2, 6, 8, 26, 5)
Particle Swarm optimisation
2002-04-24 [email protected]
Graph Coloring Problem
1
4 5
2
1
1
5
5
3
2
2
1
1
5
5
5
0
2
0
-
1
4
-3
-1
-1
+
=
Particle Swarm optimisation
2002-04-24 [email protected]
The Tireless Traveller
Example of position: X=(5,3,4,1,2,6)
Example of velocity: v=((5,3),(2,5),(3,1))
Particle Swarm optimisation
2002-04-24 [email protected]
n
1

n
3

n
2

Apple trees
( ) ( )
2
3 2
2
2 1
n n n n f + =
Evaluation n1 n2 n3
0
1
2
3
Swarm size=3
Best position
7 7 6
3 11 6
6 4 10
3 0 17
Particle Swarm optimisation
2002-04-24 [email protected]
Part 4: Some variants
Particle Swarm optimisation
2002-04-24 [email protected]
0
0.2
0.4
0.6
0.8
1
1.2
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29
Unbiased random proximity
x
p
g

p
i

v
Hyperparallelepiped => Biased
Dimension
V
o
l
u
m
e

Hypersphere vs hypercube
Particle Swarm optimisation
2002-04-24 [email protected]
Clusters and queens
Each particle is weighted by its
perf.
Dynamic clustering
Centroids = queens =
temporary new
particles
Particle Swarm optimisation
2002-04-24 [email protected]
Think locally, act locally (Adaptive
versions)
Particle Swarm optimisation
2002-04-24 [email protected]
Adaptive swarm size
There has been enough
improvement
but there has been not enough
improvement
although I'm the worst
I'm the best
I try to kill myself
I try to generate a
new particle
Particle Swarm optimisation
2002-04-24 [email protected]
Adaptive coefficients
The better I
am, the more I
follow my own
way
The better is my best
neighbour, the more
I tend to go towards
him
ov
rand(0b)(p-x)
Particle Swarm optimisation
2002-04-24 [email protected]
Energies: classical process
Rosenbrock 2D. Swarm size=20, constant coefficients
Particle Swarm optimisation
2002-04-24 [email protected]
Energies: adaptive process
Rosenbrock 2D. Adaptive swarm size, adaptive coefficients
Particle Swarm optimisation
2002-04-24 [email protected]
Part 5: Real applications (hybrid)
Medical diagnosis
Industrial mixer
Electrical generator Electrical vehicle
Particle Swarm optimisation
2002-04-24 [email protected]
Real applications (stand alone)
Cockshott A. R., Hartman B. E., "Improving the fermentation medium for
Echinocandin B production. Part II: Particle swarm optimization", Process
biochemistry, vol. 36, 2001, p. 661-669.

He Z., Wei C., Yang L., Gao X., Yao S., Eberhart R. C., Shi Y., "Extracting
Rules from Fuzzy Neural Network by Particle Swarm Optimization", IEEE
International Conference on Evolutionary Computation, Anchorage, Alaska,
USA, 1998.

Secrest B. R., Traveling Salesman Problem for Surveillance Mission using
Particle Swarm Optimization, AFIT/GCE/ENG/01M-03, Air Force Institute of
Technology, 2001.

Yoshida H., Kawata K., Fukuyama Y., "A Particle Swarm Optimization for
Reactive Power and Voltage Control considering Voltage Security
Assessment", IEEE Trans. on Power Systems, vol. 15, 2001, p. 1232-1239.
Particle Swarm optimisation
2002-04-24 [email protected]
To know more
Clerc M., Kennedy J., "The Particle Swarm-Explosion,
Stability, and Convergence in a Multidimensional
Somplex space", IEEE Transaction on Evolutionary
Computation, 2002,vol. 6, p. 58-73.
Clerc M., "L'optimisation par essaim particulaire.
Principes et pratique", Herms, Techniques et
Science de l'Informatique, 2002.
Particle Swarm Central, http://www.particleswarm.net
THE site:
Self advert
Particle Swarm optimisation
2002-04-24 [email protected]
Appendix
Particle Swarm optimisation
2002-04-24 [email protected]
Canonical form
( ) ( ) ( ) ( )
( ) ( ) ( )

+ + = +
+ = +
1 1
1
t v t x t x
t x p t v t v
( ) ( ) t x p t y =
v t +1 ( )= ov t ( )+|y t ( )
y t +1 ( ) = v t ( )+ o q ( )y t ( )



o
|
oq
( )
( )
( )
( )
(

=
(

+
+
t y
t v
t y
t v

1 1
1
1
1
M
Eigen values e
1
and e
2

Particle Swarm optimisation
2002-04-24 [email protected]
Constriction
( ) ( ) ( )
( ) ( ) ( )

+
+ + +
=
+
+ + + +
=

o o | oq oq q q o o
_

o o | oq oq q q o o
_
4 2
2 2
4 2
2 2
2
2 2
2
2
2 2
1
Constriction coefficients
Particle Swarm optimisation
2002-04-24 [email protected]
Convergence criterion

6 5 4 3 2 1
3.5
3
2.5
2
1.5
1
0.5
_
1
e
1
<1
_
2
e
2
<1





:
_
1
<1
_
2
e
2
<1





k = _
2
e
2
Particle Swarm optimisation
2002-04-24 [email protected]
Magic Square (1)
(
(
(

D D D
j i
D
m m
m
m m
, 1 ,
,
, 1
1 , 1

( )
{ }

=
e
=
+
l k j i
j i
D i j
j i
m m
N m
x m
, ,
,
1
,
1
( )
( )
0
1
1
2
1
1 , ,
1
1
2
1
, 1 ,
=
|
|
.
|

\
|
+
|
|
.
|

\
|

= =
+

= =
+
D
j
D
i
j i j i
D
i
D
j
j i j i
m m
m m
Particle Swarm optimisation
2002-04-24 [email protected]
Magic Square (2)
D=3x3, N=100
10 runs
13430 evaluations
10 solutions
55 30 68
42 49 62
56 74 23
30 61 53
89 32 23
25 51 68
27 96 39
73 40 49
62 26 74
22 70 58
40 75 35
88 5 57
50 43 67
58 55 47
52 62 4
43 51 78
75 33 64
54 88 30
50 65 68
69 42 72
64 76 43
18 25 59
32 53 17
52 24 26
80 3 30
22 72 19
11 38 64
65 28 64
63 55 39
29 74 54
Particle Swarm optimisation
2002-04-24 [email protected]
Non linear system
( )

=
= +
0 10 sin
0 1
2 1
2
2
2
1
x x
x x
Search space
[0,1]
2

1 run
143 evaluations
1 solution
10 runs
1430 evaluations
3 solutions
Particle Swarm optimisation
2002-04-24 [email protected]