ARTICLE IN PRESS

Engineering Applications of Articial Intelligence ] (]]]]) ]]] ]]]

Contents lists available at ScienceDirect

Engineering Applications of Articial Intelligence

journal homepage: www.elsevier.com/locate/engappai

Particle swarm optimization with quantum infusion for system identication

Bipul Luitel, Ganesh K. Venayagamoorthy n
Real-Time Power and Intelligent Systems Laboratory, Missouri University of Science & Technology, Rolla, MO 65409, USA

a r t i c l e in fo
Article history: Received 2 February 2009 Received in revised form 26 November 2009 Accepted 16 January 2010 Keywords: Adaptive IIR lter DEPSO Dynamical system Power system PSO PSOQI Quantum principle System identication

abstract
System identication is a challenging and complex optimization problem due to nonlinearity of the systems and even more in a dynamic environment. Adaptive innite impulse response (IIR) systems are preferably used in modeling real world systems because of their reduced number of coefcients and better performance over the nite impulse response lters. Particle swarm optimization (PSO) and its other variants has been a subject of research for the past few decades for solving complex optimization problems. In this paper, PSO with quantum infusion (PSOQI) is used in identication of benchmark IIR systems and a real world problem in power systems. PSOQIs performance is compared with PSO and differential evolution PSO (DEPSO) algorithms. The results show that PSOQI has better performance over these algorithms in identifying dynamical systems. & 2010 Elsevier Ltd. All rights reserved.

1. Introduction Traditionally, least mean square (LMS) and other algorithms have been studied for the identication of linear and static systems (Widrow et al., 1976). But, almost all physical systems are nonlinear to certain extent and recursive in nature and hence it is more convincing to model such systems by using nonlinear models (Panda et al., 2007; Krusienski and Jenkins, 2005). Thus, nonlinear system identication has attracted attention in the eld of science and engineering. Hence these are better modeled as innite impulse response (IIR) models as they can provide better performance than a nite impulse response (FIR) lter with the same number of coefcients (Shynk, 1989a). Thus the problem of nonlinear system identication can also be viewed as a problem of adaptive IIR ltering. Also, IIR models are more efcient than the FIR models for implementation as they require less parameter and hence fewer computations for the same level of performance. However, there are few problems associated with the use of IIR models in identication of a system, such as instability of the algorithms, slow convergence and convergence to the local minimum (Netto et al., 1995). Different learning algorithms have been used in the past for nonlinear system identication. These techniques include use of neural network (Hongwei and Yanchun, 2005) and gradient based search techniques such as least mean square algorithm (Shynk, 1989(a)). Unfortunately, the error

Corresponding author. E-mail address: [email protected] (G.K. Venayagamoorthy).

surface of such recursive systems such as a multi-machine power system (Kundur, 1993) tends to be multi-modal and hence traditional techniques of parameter approximation fail as they get trapped into local minimum and cannot attain the global minimum (Krusienski and Jenkins, 2005). Various algorithms that are implemented in the adaptive IIR ltering for system identication are described in (Netto et al., 1995). Population based search algorithm such as genetic algorithm (GA) has also been used for the system identication. It uses a population of potential solutions encoded as chromosomes which go through genetic operations such as crossover and mutation to nd the best solution (Kristinsson and Dumont, 1992). But its effectiveness is affected by the convergence time (the time it takes to nd the global minimum). So to eliminate such deciencies, population based stochastic optimization techniques have been discussed in various literatures. Particle swarm optimization (PSO) is one of the most known techniques (Kennedy and Eberhart, 1995). Application of PSO in the system identication has been discussed in Panda et al. (2007). In Lee et al. (2006), a method for the identication of nonlinear system and parameter optimization of the obtained inputoutput model has been described. The proposed method uses least squares support vector machines regression based on PSO. In another work, PSO has been used for optimizing the parameters of Elman neural network which is used for speed identication of ultrasonic motors (Hongwei and Yanchun, 2005). A modied form of PSO called as the self-organizing particle swarm optimization and its application in the system identication has been discussed in Shen and Zeng (2007). Radial basis function

0952-1976/$ - see front matter & 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.engappai.2010.01.022

Please cite this article as: Luitel, B., Venayagamoorthy, G.K., Particle swarm optimization with quantum infusion for system identication. Engineering Applications of Articial Intelligence (2010), doi:10.1016/j.engappai.2010.01.022

ARTICLE IN PRESS
2 B. Luitel, G.K. Venayagamoorthy / Engineering Applications of Articial Intelligence ] (]]]]) ]]] ]]]

neural network (RBFNN) has been used for system identication in Chen et al. (2007), where a hybrid gradient-based PSO algorithm has been used to adjust the parameters of the RBFNN. In Liu et al. (2006), particle swarm optimization and quantumbehaved particle swarm optimization have been used for the system identication. Use of different types of stochastic optimization techniques in adaptive IIR lters and nonlinear systems has been explained in Krusienski and Jenkins (2005). Use of differential evolution (DE) and ant colony optimization (ACO) in IIR lter design has been presented in Karaboga (2005) and Karaboga et al. (2004), respectively. They also talk about the possible use of these approaches in system identication and other applications. But these algorithms have the tendency to get stuck in the local minimum when the complexity of the problem increases and in dynamic systems where time allowed for convergence is constrained. Hybrid algorithms are used to improve the performance by combining the best feature of participating individual algorithms. Differential evolution PSO (DEPSO) for digital lter design is discussed in Luitel and Venayagamoorthy (2008a). However, identication of systems without prior structural information is still a challenge and new algorithms and approaches are being studied. Also, identication of nonlinear time varying systems is computationally intensive and many traditional techniques fail. In this paper, PSO with quantum infusion (PSOQI) has been proposed to identify the pole zero parameters of an IIR system and in the identication of generator dynamics in a power system without prior structural information. PSOQI has better performance and is robust in the fact that its convergence characteristics is less affected by the dimension of the problem and has more consistent convergence than other algorithms. Also, PSOQI converges to a much lower value than PSO or DEPSO. In the identication of generator dynamics, PSOQI performs the best, whereas PSO and DEPSO cannot approximate the system transfer function every time as is seen from the standard deviation over a number of trials. The major contributions of the paper are listed below:

 Application of PSOQI in the identication of four generators 

in a two-area multimachine power system using inputoutput data without prior structural information. Comparison of three algorithms, PSOQI, PSO and DEPSO, on system identication problems. Based on lower values of mean squared error and standard deviation, PSOQI has shown to be the best algorithm compared to the other two for identication of IIR systems and generator dynamics in a multimachine power system.

The rest of the paper is organized as follows: In Section 2, an IIR system has been explained. The PSOQI algorithm is explained in Section 3. In Section 4, results of studies carried out on some benchmark problems and a practical power system problem are presented. The conclusions are presented in Section 5.

2. An IIR system System identication is the mathematical modeling of an unknown system by monitoring its inputoutput data. This is achieved by varying the parameters of the developed model so that for a set of given inputs, its output match that of the system under consideration. For a plant whose behavior is not known, an adaptive system can be modeled and its parameters can be continuously adjusted using any adaptive algorithms. By the use of such adaptive algorithms, the required parameters can be obtained such that the output of the plant and the model are same for the same set of inputs, which is the goal of system identication (Panda et al., 2007). Fig. 1 represents one such identication model of any arbitrary system. As said, most nonlinear systems are also recursive in nature. Hence, models for real world systems are better represented as IIR systems. By doing so, the problem of system identication now becomes the problem of adaptive IIR ltering, for which different adaptive algorithms can be applied for adjusting the feed forward and feedback parameters of the recursive system. An IIR system can be represented by the transfer function: Hz b0 b1 z1 b2 z2 bm zm 1 a1 z1 a2 z2 an zn 1

 Identication of benchmark IIR systems with full and reduced

order models using PSOQI, which results in lower mean squared error and more consistent convergence.

Fig. 1. Schematic for system identication.

Please cite this article as: Luitel, B., Venayagamoorthy, G.K., Particle swarm optimization with quantum infusion for system identication. Engineering Applications of Articial Intelligence (2010), doi:10.1016/j.engappai.2010.01.022

ARTICLE IN PRESS
B. Luitel, G.K. Venayagamoorthy / Engineering Applications of Articial Intelligence ] (]]]]) ]]] ]]] 3

where m and n are the number of numerator and denominator coefcients of the transfer function and an and bm are the pole and zero parameters of the IIR lter. This can be written as difference equation of the form (Krusienski and Jenkins, 2005; Karaboga, 2005): yk
L X n1

an kykn

M X n0

bn kxkn

where x(k) and y(k) represent the kth input and output of the system. Also, n = 1, 2, 3,y, L and n = 0, 1, 2,y, M represent the

coefcients of the IIR lter. Considering the block diagram of Fig. 1, the output y(k) for input x(k) to the system is mixed with a noise signal n(k). The output of the plant added with the noise gives the nal system output d(k). On the other hand, the output of the IIR lter in the modeled system for the same input x(k) has an output of y0 (n). The difference of the output from the actual system with that of the modeled system gives the error e(k). This error is used by the adaptive algorithm to adjust the parameters of the IIR lter, and thus reduce the error in a number of iterations so as to exactly identify the actual system. This has been shown in

START

Define: Search space, population of particles and fitness function Initialize: Position, Velocity, pbest and gbest of particles

Yes END

Min. error obtained? or Max. iterations reached?

No PSO pbest population gbest particle

Select a random particle

Find local attractor point J and mean best position mbest

Create offspring (gbest1) probabilistically

Evaluate fitness of gbest1

Is fitness of gbest > gbest1 ?

Yes New gbest = gbest

New gbest = gbest1 No

Fig. 2. Flowchart showing the PSOQI algorithm.

Please cite this article as: Luitel, B., Venayagamoorthy, G.K., Particle swarm optimization with quantum infusion for system identication. Engineering Applications of Articial Intelligence (2010), doi:10.1016/j.engappai.2010.01.022

ARTICLE IN PRESS
4 B. Luitel, G.K. Venayagamoorthy / Engineering Applications of Articial Intelligence ] (]]]]) ]]] ]]]

the following equations: dk yk nk ek dky0 k 3 4

velocity of the ith particle represented as xi xi1 ; xi2 ; . . .; xiD vi vi1 ; vi2 ; . . .; viD vid k 1 wvid k c1 rand1 Pid xid c2 rand2 Pgd xid 9 xid k 1 xid k vid k 1 10 7 8

For the identication of the system, the adaptive algorithm tries to minimize the error e(k) by adjusting the parameters of the modeled system, which are the pole-zero coefcients in case of an IIR system. The different kinds of algorithms that can be used for error minimization in adaptive systems are explained in Netto et al. (1995). In this paper, mean squared error (MSE) between the output of the actual system and the designed system as given by (5) has been considered as the feedback to the adaptive algorithm. F
N 1X dky0 k2 Nk1

These particles are randomly initialized over the search space with initial positions and velocities. They change their positions and velocities according to (9) and (10) where c1 and c2 are cognitive and social acceleration constants respectively, rand1()

The tness function used by the different algorithms that are illustrated in the paper is given by Fitness 1 1F 6

The numerator and denominator coefcients of the IIR lter are represented by D dimensions (D= L+ M). In Karaboga (2005), DE has been used for adjusting the parameters of the IIR system to reduce the MSE or to increase the tness of the system. In this paper, PSO, DEPSO and PSOQI are being used. The mentioned algorithms then nd the best parameters in a number of iterations by searching for the possible solutions in a D-dimensional search space.

3. Particle swarm optimization with quantum infusion Particle swarm optimization with quantum infusion is a new approach to hybridization of PSO and QPSO. Here, the quantum principle in QPSO is used to create a new offspring. After the positions and velocities of the particles are updated using standard PSO equations, a randomly chosen particle from PSOs pbest population is utilized to carry out the quantum operation (Luitel and Venayagamoorthy, 2009); and thus, create an offspring by mutating the gbest. The tness of the offspring is evaluated and the offspring replaces the gbest particle of PSO only if it has a better tness. This ensures that the tness of the gbest particle is equal to or better than its tness in the previous iteration. Thus, it is improved and pulled towards the best solution over iterations. By infusing the quantum theory to the standard PSO, a new hybrid algorithm is evolved which incorporates the best features of the respective individual algorithms and thus a better tness is achieved. In PSOQI, fast convergence property obtained by PSO in the rst few iterations, and the convergence to a lower average error property obtained by QPSO, have been combined and hence the performance is signicantly improved, as is shown in the results and Fig. 2. The owchart for PSOQI is illustrated in Fig. 2. It is described below in detail. PSO is an evolutionary-like algorithm developed by Eberhart and Kennedy in 1995 (delValle et al., 2008). It is a population based search algorithm and is inspired by the observation of natural habits of bird ocking and sh schooling. In PSO, a swarm of particles moves through a D dimensional search space. The particles in the search process are the potential solutions, which move around a dened search space with some velocity until the error is minimized or the solution is reached, which is decided by the tness function. The particles reach to the desired solution by updating their position and velocity according to the PSO equations. In PSO, each individual is treated as a volume-less particle in the D-dimensional space, with the position and

Fig. 3. Flowchart showing system identication using PSOQI.

Please cite this article as: Luitel, B., Venayagamoorthy, G.K., Particle swarm optimization with quantum infusion for system identication. Engineering Applications of Articial Intelligence (2010), doi:10.1016/j.engappai.2010.01.022

ARTICLE IN PRESS
B. Luitel, G.K. Venayagamoorthy / Engineering Applications of Articial Intelligence ] (]]]]) ]]] ]]] 5

and rand2() are two random functions uniformly distributed in the range of [0,1] and w is the inertia weight introduced to accelerate the convergence speed of PSO (delValle et al., 2008). Vector Pi = (Pi1, Pi2,y,PiD) is the best previous position (the position giving the best tness value) of particle i called the pbest, and vector Pg = (Pg1, Pg2,y, PgD) is the position of the best particle among all the particles in the swarm and is called the gbest. xid, vid, Pid are the dth dimension of vector of xi, vi, Pi. For the PSO equations to be dimensionally correct, the velocity term in (10) is taken over a unit time (Chakraborti et al., 2007). PSO is illustrated in the owchart in Fig. 3. Quantum behaved particle swarm optimization (QPSO) was introduced by Sun in 2004 (Sun et al., 2004a). According to the uncertainty principle, position and velocity of a particle in quantum world cannot be determined simultaneously. Thus QPSO differs from standard PSO mainly in the fact that exact values of x and v cannot be determined. In quantum mechanics, a particle, instead of having position and velocity, has a wavefunction given by

Df y

Zy
1

Q y dy e29y9=L

15

where the parameter L is the length of the potential eld which depends on the energy intensity and is called the creativity or imagination of the particle that determines its search scope (Sun et al., 2004b). L can be evaluated as the distance between the particles current position and point J as follows: L 2b9J x9 16

cr; t

11

which has no physical meaning but its amplitude squared gives the probability measure of its position in any one dimension r at time t. The governing equation of quantum mechanics is the Schrodingers equation given by j_ @ ^ r cr ; t cr; t H @t 12

The parameter b is the only parameter of the algorithm. It is called the creativity coefcient and is responsible for the convergence speed of the particle. In QPSO, search and solution spaces are two unique spaces of different quality. So a mechanism is necessary to map the position of a particle in the search space to the solution space. This is referred to as collapsing and is achieved by applying the Monte Carlo simulation. This is explained as follows (Sun et al., 2004a). Let s be any random number uniformly distributed between 0 and 1/L. For a uniform random number u in the interval [0,1], s is dened as s 1 u L 17

Now, equating (14) and (17), the following relation is achieved: u e29y9=L y 7   L 1 ln 2 u 18 19

where H is a time-independent Hamiltonian operator given by

2 ^ r _ r2 V r H 2m

13

where _ is Plancks constant, m is the mass of the particle and Vp(r) is the potential energy distribution (Mikki and Kishk, 2006). Based on the probability density function, a particles probability of appearing in position x can be determined. Therefore in QPSO, a delta-potential-well based probability density function has been used with center of the well at point J= (j1, j2,y, jD) in order to avoid explosion and help the particles in PSO to converge (Sun et al., 2004b). Assuming a particle in one-dimensional space having its center of potential at J, normalized probability density function Q and distribution function Df can be obtained (Sun et al., 2005). Let y=x j, then the form of this probability density function is given as follows and depends on the potential eld the particle lies in: Q y 1 29y9=L e L 14

The position equation is given as follows:   L 1 x J 7 ln 2 u

20

where the particles local attractor point J has coordinates given by the following equation: Jd a1 Pgd a2 Pid 21

where a1 = a/(a + b) and a2 = b/(a + b), and a and b are two uniformly distributed random numbers. From (16) and (19), the new position of the particle is calculated as   1 22 xk 1 Jk 7 b9Jkxk9 ln u This delta-potential-well based quantum PSO is called the QDPSO in Sun et al. (2004a). This has been improved further by dening a mainstream thought (Sun et al., 2005) or the mean best

Table 1 Study of Cases I and II. Case I (Krusienski and Jenkins, 2004) Transfer function 1:25z1 0:25z2 10:3z1 0:4z2 Case II (Ng et al., 1996) 0:2z1 0:4z2 0:5z3 1 0:6z1 0:25z2 0:2z3

Full order

L M Model L M Model

2 1 b0 b1 z1 1 a1 z1 a2 z2 1 0 b0 1 a1 z1

3 2 b0 b1 z1 b2 z2 1 a1 z1 a2 z2 a3 z3 2 1 b0 b1 z1 1 a1 z1 a2 z2

Reduced order

Please cite this article as: Luitel, B., Venayagamoorthy, G.K., Particle swarm optimization with quantum infusion for system identication. Engineering Applications of Articial Intelligence (2010), doi:10.1016/j.engappai.2010.01.022

ARTICLE IN PRESS
6 B. Luitel, G.K. Venayagamoorthy / Engineering Applications of Articial Intelligence ] (]]]]) ]]] ]]]

position, mbest, as
S 1X P k mbest k Si1 i

! S S 1X 1X P k; . . .; P k S i 1 i1 S i 1 iD

23

where S is the size of the population, D is the number of dimensions and Pi is the pbest position of each particle. Now the position update equation in (22) is given as (24), where the addition or subtraction is carried out with 50% probability:   1 24 xk 1 Jk 7 b9mbest kxk9 ln u By using (21) this can also be written as follows to show the mutation on gbest:   1 25 xk 1 a1 Pgd k a2 Pid k 7 b9mbestkxk9 ln u

4. Case studies and results Two different studies have been carried out for system identication. In the rst study, four benchmark IIR systems between second and sixth order are considered for the case study.
Fig. 5. Pole zero plot for full order model of Case I.

Fig. 4. Error graph for Case I: (a) full order and (b) reduced order for 500 iterations.

Fig. 6. Error graph for Case II: (a) full order and (b) reduced order for 500 iterations.

Please cite this article as: Luitel, B., Venayagamoorthy, G.K., Particle swarm optimization with quantum infusion for system identication. Engineering Applications of Articial Intelligence (2010), doi:10.1016/j.engappai.2010.01.022

ARTICLE IN PRESS
B. Luitel, G.K. Venayagamoorthy / Engineering Applications of Articial Intelligence ] (]]]]) ]]] ]]] 7

These IIR systems are modeled using two different models, one having the same order as the actual system and second having less order than that of the actual system. These reduced order cases pose challenge to the optimization algorithm in that they produce multimodal error surface which has multiple minima. In all cases, as the number of coefcients decreases, the degree of freedom reduces and it becomes more difcult to identify the actual system. In the second study, identication of dynamics of four generators in a power system is considered. With pseudorandom binary signal (PRBS) input to one of the generators, the speed deviations on all of the four generators is measured. From this inputoutput data, the transfer function of the four generators for dynamics of interest is identied using PSO, DEPSO and PSOQI. Identication of IIR systems using PSOQI is shown in the owchart in Fig. 3.

4.1. Study I Each case is simulated using PSO, DEPSO (Luitel and Venayagamoorthy, 2008a) and PSOQI (Luitel and Venayagamoorthy, 2008b) in MATLAB on the same computer using the following parameters. The PSO parameters used in the study are obtained by systematic study of the effect of various parameters in different case studies. The variation of the results with the parameters is, however, not a part of the results shown in this paper. The shown result is an average over 50 trials. D =number of dimension representing the weights to be optimized

Fig. 8. Error graph for Case III: (a) full order and (b) reduced order for 500 iterations.

Fig. 7. Pole-zero plot for full order model of Case II.

Table 2 Study of Case III (Shynk, 1989b). Transfer function z1 0:9z2 0:81z3 0:729z4 10:04z1 0:2775z2 0:2101z3 0:14z4 L M Model L M Model 4 3 b0 b1 z1 b2 z2 b3 z3 1 a1 z1 a2 z2 a3 z3 a4 z4 3 2 b0 b1 z1 b2 z2 1 a1 z1 a2 z2 a3 z3

Full order

Reduced order

Fig. 9. Pole-zero plot for full order model of Case III.

Please cite this article as: Luitel, B., Venayagamoorthy, G.K., Particle swarm optimization with quantum infusion for system identication. Engineering Applications of Articial Intelligence (2010), doi:10.1016/j.engappai.2010.01.022

ARTICLE IN PRESS
8 B. Luitel, G.K. Venayagamoorthy / Engineering Applications of Articial Intelligence ] (]]]]) ]]] ]]]

Table 3 Study of Case IV (Karaboga, 2005). Transfer function 10:4z2 0:65z4 0:26z6 10:77z2 0:8498z4 0:6486z6 L M Model L M Model 6 6 b0 b1 z1 b2 z2 b3 z3 b4 z4 b5 z5 b6 z6 1 a1 z1 a2 z2 a3 z3 a4 z4 a5 z5 a6 z6 5 5 b0 b1 z1 b2 z2 b3 z3 b4 z4 b5 z5 1 a1 z1 a2 z2 a3 z3 a4 z4 a5 z5

Full order

Reduced order

Fig. 11. Pole-zero plot for full order model of Case IV.

Fig. 10. Error graph for Case IV: (a) full order and (b) reduced order for 500 iterations.

P (population size) = 25 c1, c2 (cognitive and social acceleration constants for PSO) =2 w (inertia weight)= linearly decreasing from 1.4 to 0 CR (crossover rate)= 0.8 b = linearly increasing from 0.5 to 1 Number of inputs =50

Number of iterations=500 Maximum velocity=1.3 Maximum position= 1.3 Number of trials=50 The coefcients are randomly initialized within the periphery of the possible solution and the maximum velocity and position are also restricted to 1.3 which is the maximum value of the actual coefcients of the plant. The output of the plant is subjected to a white Gaussian noise of 30 dB signal to noise ratio. The transfer functions and their implementation in two different models for Cases I and II are shown in Table 1. Case I is a second order system. The simulation results of two different models for this case are shown in Fig. 4(a) and (b). The pole-zero plot of the coefcients obtained for this transfer function is shown in Fig. 5. Fig. 6(a) and (b) show the results of the third order system studied in Case 2. Fig. 7 shows the pole-zero plot of the coefcients obtained for the transfer function. Case 3 is a fourth order IIR system shown in Table 2. The error graphs for the two different models are shown in Fig. 8(a) and (b). The pole-zero plot for this case is shown in Fig. 9. Table 3 shows the transfer function of the plant and its models for the sixth order system studied in Case 4. The results are shown in Fig. 10(a) and (b) and the polezero plot in Fig. 11. The comparison of performance of PSO, DEPSO and PSOQI is shown in Table 4 where minimum, average and standard deviation of the results obtained from 50 trials over 500 iterations have been presented. These results show that

Please cite this article as: Luitel, B., Venayagamoorthy, G.K., Particle swarm optimization with quantum infusion for system identication. Engineering Applications of Articial Intelligence (2010), doi:10.1016/j.engappai.2010.01.022

ARTICLE IN PRESS
B. Luitel, G.K. Venayagamoorthy / Engineering Applications of Articial Intelligence ] (]]]]) ]]] ]]] 9

Table 4 Full order model (500 iterations). Case MSE (dB) Min. Case I PSO DEPSO PSOQI PSO DEPSO PSOQI PSO DEPSO PSOQI PSO DEPSO PSOQI 7.102e 4 7.102e 4 7.102e 4 7.791e 4 7.791e 4 7.791e 4 7.245e 4 7.245e 4 7.245e 4 7.821e 4 7.623e 4 7.984e 4 Avg. 8.612e 4 7.278e-4 7.102e 4 0.001 9.480e 4 9.215e 4 0.003 0.001 0.001 0.011 0.002 0.002 Std. 5.074e 4 4.391e 5 1.148e 7 5.222e 4 4.011e 4 3.627e 4 0.003 0.001 0.001 0.014 0.003 0.004 Time (s)a Min. 3.422 2.547 2.984 3.563 2.703 3.281 2.609 2.672 3.421 0.938 1.046 3.063 Avg. 3.769 3.166 3.227 3.778 2.826 3.432 3.404 3.056 3.734 2.240 2.329 4.008

Case II

Case III

Case IV

Performed on the same computer for 500 iterations.

Fig. 12. Error graph for: (a) Case I and (b) Case II for 50 iterations.

Fig. 13. Error graph for: (a) Case III and (b) Case IV for 50 iterations.

Please cite this article as: Luitel, B., Venayagamoorthy, G.K., Particle swarm optimization with quantum infusion for system identication. Engineering Applications of Articial Intelligence (2010), doi:10.1016/j.engappai.2010.01.022

ARTICLE IN PRESS
10 B. Luitel, G.K. Venayagamoorthy / Engineering Applications of Articial Intelligence ] (]]]]) ]]] ]]]

although PSOQI takes slightly more time to converge to the global minimum, it converges to a lower MSE. Moreover, the results show that PSOQI is fairly consistent in its performance and it deviates the least over a number of trials. The study is also carried out for full order with 50 iterations. These results for Cases I and II are shown in Fig. 12(a) and (b) and for Cases III and IV in Fig. 13(a) and (b), respectively. Table 5 shows the comparison of the performance of the three algorithms in 50 iterations. These results also conrm the robustness of PSOQI in its ability to converge faster and to a higher tness value. The similar results for the reduced order case are shown in Table 6. These results also indicate the better performance of PSOQI over PSO and DEPSO. Since mutation operation introduced by DE helps the DEPSO algorithm to come out of the local minima, it nds the global minimum faster, where PSO tends to get stuck. However, PSOQI has even better ability to overcome the local minima due to its quantum operation based mutation on the global best particle obtained from PSO.

^ k represents the output obtained by output samples at time k. Y the designed system at instant k
^ k a1 xk a2 xk1 a3 xk2 b1 yk1 b2 yk2 b3 yk3 Y

26

4.2. Study II In this study, identication of generator dynamics in a power system is carried out based on its inputoutput data with no prior information about the structure of the system. The system is implemented using (26) where x(k) and y(k) are the input and
Table 5 Full order model (50 iterations). Case MSE (dB) Min. Case I PSO DEPSO PSOQI PSO DEPSO PSOQI PSO DEPSO PSOQI PSO DEPSO PSOQI 9.448e 4 9.448e 4 9.447e 4 0.001 0.001 0.001 0.001 0.001 8.353e 4 0.001 8.688e 4 9.994e 4 Avg. 0.001 0.001 9.988e 4 0.002 0.002 0.001 0.013 0.004 0.003 0.024 0.007 0.004 Std. 5.011e 4 5.806e 4 1.222e 4 0.003 0.003 5.674e 4 0.045 0.002 0.002 0.032 0.010 0.006 Time (s)a Min. 0.218 0.234 0.265 0.234 0.235 0.297 0.233 0.250 0.343 0.234 0.250 0.375 Avg. 0.261 0.302 0.275 0.264 0.263 0.343 0.269 0.275 0.371 0.257 0.267 0.399

Fig. 14. Input pseudo-random binary signal.

Case II

Case III

Case IV

Performed on the same computer for 500 iterations.

Table 6 Reduced order model (500 iterations). Case MSE (dB) Min. Case I PSO DEPSO PSOQI PSO DEPSO PSOQI PSO DEPSO PSOQI PSO DEPSO PSOQI 0.006 0.006 0.006 0.004 0.004 0.004 0.005 0.005 0.005 0.001 0.001 0.001 Avg. 0.006 0.006 0.006 0.089 0.010 0.011 0.008 0.007 0.005 0.018 0.004 0.003 Std. 7.149e 4 4.214e 4 4.085e 18 0.443 0.005 0.006 0.003 0.003 0.001 0.042 0.004 0.001 Time (s)a Min. 2.234 2.125 2.500 3.625 3.609 3.079 0.766 0.859 2.312 2.281 2.515 3.375 Avg. 2.356 2.326 2.601 3.799 3.700 3.130 1.269 1.392 2.700 2.766 2.678 3.627

Case II

Case III

Case IV

Performed on the same computer for 500 iterations.

Please cite this article as: Luitel, B., Venayagamoorthy, G.K., Particle swarm optimization with quantum infusion for system identication. Engineering Applications of Articial Intelligence (2010), doi:10.1016/j.engappai.2010.01.022

ARTICLE IN PRESS
B. Luitel, G.K. Venayagamoorthy / Engineering Applications of Articial Intelligence ] (]]]]) ]]] ]]] 11

This can be written into a transfer function as follows: Y z a1 a2 z a3 z X z 1b1 z1 b2 z2 b3 z3 ^ k ek ykY

1 2

27 28

After ltering the input and the delayed output data, the output of the system is compared with the actual output and the MSE between these outputs is taken as the tness function. Using

the tness information, the algorithms continuously adjust the coefcients of the lters so as to minimize the error between the actual output of the system and the output of the designed system given by (28). In this study, four generators (G1G4) of a two-area power system (Venayagamoorthy, 2007) are considered. The generator G1 is subjected to a PRBS input and speed deviation (dSpeed) of the four generators is recorded for 20 s of input data (3125 samples). The rst 10 s of data is taken for identication of the system (training) and the next 10 s of data is used to verify the

Fig. 15. Training and testing plots for G1 using PSO.

Fig. 16. Training and testing plots for G1 using DEPSO.

Please cite this article as: Luitel, B., Venayagamoorthy, G.K., Particle swarm optimization with quantum infusion for system identication. Engineering Applications of Articial Intelligence (2010), doi:10.1016/j.engappai.2010.01.022

ARTICLE IN PRESS
12 B. Luitel, G.K. Venayagamoorthy / Engineering Applications of Articial Intelligence ] (]]]]) ]]] ]]]

Fig. 17. Training and testing plots for G1 using PSOQI.

Fig. 18. Training and testing plots for G3 using PSO.

system output (testing). The training and testing plots for different generators are shown in Figs. 1421. In this study, the dimension of the system is 6, each dimension representing the coefcient of the transfer function. The input PRBS signal is shown in Fig. 14. The training and testing plot for G1 obtained using PSO, DEPSO and PSOQI are presented in Figs. 1517, respectively. Figs. 1820 show the training and testing plots for G3 obtained using PSO, DEPSO and PSOQI, respectively. Units in Figs. 1520 for dspeed and time are

rad/s and s respectively. The minimum and average values of tness for the three algorithms obtained from the study are presented in Table 7. The standard deviation of the minimum values over 50 trials is also presented in the table. The results show that PSOQI is more consistent in identifying the dynamics of the system in every trial. Although PSO and DEPSO could also identify the system transfer function and predict the speed deviation, they were not able to do so in every trial. PSO and

Please cite this article as: Luitel, B., Venayagamoorthy, G.K., Particle swarm optimization with quantum infusion for system identication. Engineering Applications of Articial Intelligence (2010), doi:10.1016/j.engappai.2010.01.022

ARTICLE IN PRESS
B. Luitel, G.K. Venayagamoorthy / Engineering Applications of Articial Intelligence ] (]]]]) ]]] ]]] 13

Fig. 19. Training and testing plots for G3 using DEPSO.

Fig. 20. Training and testing plots for G3 using PSOQI.

DEPSO also suffered from the problem of identifying an out of phase system. Although the tness obtained by PSOQI is better than PSO, it has to go through more number of tness evaluations. Each iteration of PSO corresponds to P tness evaluations while that of PSOQI corresponds to P + 1 tness evaluations. However, if more number of pbest particles is considered for quantum operation, the tness evaluations will increase. In these studies, all of the pbest particles are considered for quantum operation

and hence the number of tness evaluations per iteration is 2P. However, given the equal number of tness evaluations, standard PSO does not show improvements in tness as is shown in Fig. 21. The gure also demonstrates how the PSOQI converges to a lower tness in less number of iterations, thus showing its promises in online applications, and how the tness of PSO does not meet that of PSOQI regardless of the number of tness evaluations (proportional to the number of iterations).

Please cite this article as: Luitel, B., Venayagamoorthy, G.K., Particle swarm optimization with quantum infusion for system identication. Engineering Applications of Articial Intelligence (2010), doi:10.1016/j.engappai.2010.01.022

ARTICLE IN PRESS
14 B. Luitel, G.K. Venayagamoorthy / Engineering Applications of Articial Intelligence ] (]]]]) ]]] ]]]

Fig. 21. Comparison of PSO and PSOQI in terms of tness.

Table 7 Comparison of PSO, DEPSO and PSOQI for system identication. PSO Training G1 Min. Avg. Std. Min. Avg. Std. Min. Avg. Std. Min. Avg. Std. 0.0288 0.3558 0.2610 0.0576 0.3843 0.2463 0.0126 0.3417 0.2528 0.0272 0.3394 0.2356 Testing 0.0231 0.2790 0.2046 0.0492 0.3219 0.2079 0.0127 0.3156 0.2365 0.0256 0.3082 0.2156 DEPSO Training 0.0090 0.3859 0.2828 0.0075 0.2675 0.2386 0.0024 0.2183 0.2553 0.0025 0.2807 0.3020 Testing 0.0090 0.3028 0.2212 0.0057 0.2237 0.2011 0.0021 0.1973 0.2369 0.0026 0.2543 0.2758 PSOQI Training 0.0014 0.0098 0.0058 8.0904e-4 0.0085 0.0057 0.0020 0.0200 0.0773 7.8092e-4 0.0089 0.0120 Testing 0.0011 0.0084 0.0054 7.0726e-4 0.0071 0.0046 0.0022 0.0190 0.0726 7.7778e-4 0.0086 0.0111

G2

G3

G4

5. Conclusion A hybrid particle swarm optimization with quantum infusion (PSOQI) algorithm for identication of IIR systems and generator dynamics in a multimachine power system has been presented in this paper. The performance of PSOQI is compared with that of PSO and a hybrid algorithm of PSO and differential evolution (DEPSO). These studies performed show that swarm, evolutionary and quantum behaved algorithms can be applied in system identication and hybrid algorithms perform better by combining the best features of the participating individual algorithms. The lower values of mean squared error and standard deviation show that PSOQI is the best algorithm among the three for system identication. The results show that PSOQI converges faster and with more consistency than the other algorithms, thus showing its promise in online implementation. However, it is computationally complex due to the increased number of tness evaluations and hence a trade-off between time complexity and tness is necessary.

To conrm its robustness and scalability, PSOQI needs to be applied to many different benchmark problems and dynamical real world applications. For the authors future work, the algorithm will be tested on different types of applications, including online and hardware implementation, using different tness functions.

Acknowledgments Funding provided by the National Science Foundation, USA under the CAREER grant ECCS #0348221 and EFRI #0836017 is gratefully acknowledged. References
Chakraborti, N., Das, S., Jaykanth, R., Pekoz, R., Erkoc, S., 2007. Genetic algorithms applied to Li+ ions contained in carbon nanotubes: an investigation using

Please cite this article as: Luitel, B., Venayagamoorthy, G.K., Particle swarm optimization with quantum infusion for system identication. Engineering Applications of Articial Intelligence (2010), doi:10.1016/j.engappai.2010.01.022

ARTICLE IN PRESS
B. Luitel, G.K. Venayagamoorthy / Engineering Applications of Articial Intelligence ] (]]]]) ]]] ]]] 15

particle swarm optimization and differential evolution along with molecular dynamics. Materials and Manufacturing Processes 22, 562569. Chen, S., Mei, T., Luo, M.,Yang, X., Identication of nonlinear system based on a new hybrid gradient-based PSO algorithm. In: International Conference on Information Acquisition, July 2007, pp. 265268. delValle, Y., Venayagamoorthy, G.K., Mohagheghi, S., Hernandez, J.C., Harley, R.G., 2008. Particle swarm optimization: basic concepts, variants and applications in power systems. IEEE Transactions on Evolutionary Computation 12, 171195. Hongwei, G., Yanchun L., Identication for nonlinear systems based on particle swarm optimization and recurrent neural network [ultrasonic motor control applications]. In: Proceedings of the International Conference on Communications, Circuits and Systems, vol. 2, May 2005. Karaboga, N., 2005. Digital IIR lter design using differential evolution algorithm. EURASIP Journal of Applied Signal Processing 8, 12691276. Karaboga, N., Kalinli, A., Karaboga, D., 2004. Designing digital IIR lters using ant colony optimization algorithm. Engineering Applications of Articial Intelligence 17, 301309. Kennedy, J., Eberhart, R., Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks, vol. 4, December 1995, pp. 19421948. Kristinsson, K., Dumont, G.A., 1992. System identication and control using genetic algorithms. IEEE Transactions on Systems, Man and Cybernetics 22, 10331046. Krusienski, D.J., Jenkins, W.K., 2005. Design and performance of adaptive systems based on structured stochastic optimization strategies. IEEE Circuits and Systems Magazine 5, 820. Krusienski, D.J., Jenkins, W.K., 2004. Particle swarm optimization for adaptive IIR lter structures. In: Congress on Evolutionary Computation vol. 1, 965970. Kundur, P. Power System Stability and Control. McGraw-Hill, New York, 1993. Lee, B.H., Kim, S., Seok, J., Won, S., Nonlinear system identication based on support vector machine using particle swarm optimization. In: International Joint Conference, SICE-ICASE, October 2006, pp. 56145618. Liu J., Wenbo, X., Sun, J., Nonlinear system identication of Hammerstien and Wiener model using swarm intelligence. In: IEEE International Conference on Information Acquisition, August 2006, pp. 12191223. Luitel B., Venayagamoorthy, G.K., A PSO with quantum infusion algorithm for training simultaneous recurrent neural networks. In: IEEE-INNS International Joint Conference on Neural Networks (IJCNN), June 2009, pp. 19231930.

Luitel B., Venayagamoorthy, G.K., Differential evolution Particle swarm optimization for digital lter design. In: Proceedings of the World Congress on Computational Intelligence, June 2008a, pp. 39543961. Luitel B., Venayagamoorthy, G.K., Particle swarm optimization with quantum infusion for the design of digital lters. In: Proceedings of the Swarm Intelligence Symposium (SIS), September 2008b, pp. 18. Mikki, S.M., Kishk, A., Quantum particle swarm optimization for electromagnetic. In: IEEE Transactions on Antennas and Propagation, vol. 54, October 2006. Netto, S.L., Diniz, P.S.R., Agathoklis, P., 1995. Adaptive IIR ltering algorithms for system identication: a general framework. IEEE Transactions on Education 38, 5466. Ng, S.C., Leung, S.H., Chung, C.Y., Luk, A., Lau, W.H., 1996. The genetic search approach. A new learning algorithm for adaptive IIR ltering. IEEE Signal Processing Magazine, 3846. Panda, G. Mohanty, D., Majhi, B., Sahoo, G., Identication of nonlinear systems using particle swarm optimization technique. In: IEEE Conference on Evolutionary Computation, September 2007, pp. 32533257. Shen, Y., Zeng C., A. Self-organizing particle swarm optimization algorithm and application. In: Third International Conference on Natural Computation, vol. 4, August 2007, pp. 668672. Shynk, J.J., 1989a. Adaptive IIR ltering. IEEE ASSP Magazine, 421. Shynk, J.J., 1989b. Adaptive IIR ltering using parallel form realization. IEEE Transaction on Acoustic, Speech, Signal Processing 37 (4), 519533. Sun, J., Feng, B., Xu, W., 2004a. Particle swarm optimization with particles having quantum behavior. In: Congress on Evolutionary Computation 1, 325331. Sun, J., Xu, W., Feng, B., Adaptive parameter control for quantum-behaved particle swarm optimization on individual level. In: IEEE International Conference on Systems, Man and Cybernetics, vol. 4, October 2005, pp. 30493054. Sun, J., Xu, W., Feng, B., Global search strategy of quantum-behaved particle swarm optimization. In: IEEE Conference on Cybernetics and Intelligent Systems, vol. 1, December 2004b, pp. 111116. Venayagamoorthy, G.K., 2007. Online design of an echo state network based wide area monitor for a multimachine power system. Neural Networks 20, 404413. Widrow, B., McCool, J.M., Larimore, M.G., Johnson, C.R. Jr., Stationary and nonstationary learning characteristics of the LMS adaptive lter. Proceedings of the IEEE 64 1976, pp. 11511162.

Please cite this article as: Luitel, B., Venayagamoorthy, G.K., Particle swarm optimization with quantum infusion for system identication. Engineering Applications of Articial Intelligence (2010), doi:10.1016/j.engappai.2010.01.022