1

AN EVOLUTIONARY QUANTUM BEHAVED PARTICLE SWARM

OPTIMIZATION FOR MINING ASSOCIATION RULES
--------------------------------------------------------------------------------------------------------------------Abstract: Association Rule Mining is a technique of data mining that aims to extract interesting
patterns, frequent correlations or associations among set of items in the transaction databases.
Particle Swarm Optimization (PSO) is one of several methods for mining Association Rules. It is
a technique used to explore the search space of a given problem to find the settings or parameters
required to maximize a particular objective. However, avoiding local optima and improving the
convergence speed is still a tedious task. In this work, an Evolutionary Quantum behaved Particle
Swarm Optimization (QPSO) algorithm is proposed by combining the classical PSO philosophy
and quantum mechanics to improve the performance of PSO. It is a global-convergence guaranteed
algorithm and has a better search ability than the traditional PSO. The performance of QPSO is
compared with basic PSO, and three other variants of PSO and the experimental results shows that
the proposed algorithm outperforms PSO quite significantly and is also found to be as effective as
the PSO variants with marginally better computational efficiency.
Keywords: Particle Swarm Optimization, Quantum behavior, Association Rule Mining

1. Introduction
Data mining, also known as Knowledge Discovery in Databases is the process of extracting the
patterns from data and taking an action based on the category of the pattern. As the amount of data
stored in databases and types of databases increases rapidly extracting the critical hidden
information from these databases has become an important issue. Several methods such as
association rules, clustering and classification have been used to obtain interesting facts from data.
Among these models, association rule mining is the most widely applied method.
Initially traditional algorithms like Apriori, Dynamic Itemset Counting, FP-Growth tree have been
used for extracting the interesting patterns. But, as the number of rules and the correlations in
databases increases, the computational complexity continues to grow rapidly. Later Evolutionary
algorithms such as Genetic Algorithms (GAs) and Particle Swarm Optimization (PSO) have been
introduced for mining association rules. The main motivation for using GA in the discovery of
association rules is that they cope better with attribute interaction through mutation and crossover
operators. Although GA discovers high level prediction rules with better attribute interaction, the
operators tends to increase the complexity of computation. Similar to GA, PSO is also a
population-based iterative algorithm. The fundamental problem in PSO is its imbalanced local and
global search and also while trying to reduce the convergence time it gets trapped in local optimal
region when solving complex multimodal problems. These weaknesses have restricted wider
applications of PSO.

Therefore, accelerating convergence speed and avoiding the local optima have become the two
most important and appealing goals in PSO research. In this work, an Evolutionary Quantum
behaved Particle Swarm Optimization algorithm that features better search efficiency by avoiding
local optima and improving the convergence speed is proposed.
The paper is organized as follows. In section 2 the various input and output parameters are
discussed. In section 3 a brief introduction to basic PSO is given, section 4 and 5 describes the
proposed system in detail. Section 6 explains the results and discussion and section 7 gives the
conclusion.

2. Methodology
2.1. Association Rule Mining
Association rule mining is a technique of data mining that is very widely used to deduce inferences
from large databases. Typically the relationship will be in the form of a rule: IF {X}, THEN {Y}.
Here the X part is called Antecedent and Y part is called Consequent. The rule should be such
that XY = .
The two parameter that indicate the importance of association rules are support and
confidence.
The support indicates how often the rule holds in a set of data. It is given by
support(X) =

No. of transactions containing X

Total No. of transactions

The confidence for a given rule is a measure of how often the consequent is true, given that the
antecedent is true. If the consequent is false while the antecedent is true, then the rule is also false.
If the antecedent is not matched by a given data item, then this item does not contribute to the
determination of the confidence of the rule. It is given by
confidence(XY) =

support( )
support()

2.2. Output Parameters

Much work on association rule mining has been done focusing on efficiency, effectiveness and
redundancy. Focus is also needed on the quality of rules mined. In this paper association rule
mining using QPSO is treated as a multi-objective problem where rules are evaluated
quantitatively and qualitatively based on the following measures.

2.2.1. Predictive Accuracy

Predictive Accuracy (PA) measures the effectiveness of the rules mined. The mined rules must
have high predictive accuracy. The formula is given by:
Predictive accuracy =

|X & Y|
|X|

Where |X&Y| is the number of records that satisfy both the antecedent X and consequent Y, |X| is
the number of rules satisfying the antecedent X.
2.2.2. Number of rules generated
The count of the rules generated above a certain PA.
2.2.3. Laplace
It is a confidence estimator that takes support into account, becoming more pessimistic as the
support of X decreases. It is useful to detect spurious rules that may occur by chance. It is defined
as
support(X Y) + 1
lapl(X Y) =
support(X) + 2
2.2.4. Fitness
Fitness value is utilized to evaluate the importance of each particle. It is given by
Fitness(k) = confidence(k) * log(support(k)) * length(k) + 1
The objective of this fitness function is maximization. The larger the particle support and
confidence, the greater the strength of the association, meaning that it is an important association
rule.
2.2.5. Conviction
Conviction measures the weakness of confidence. Conviction is infinite for logical implications
(confidence 1), and is 1 if X and Y are independent.

2.2.6. Convergence Rate

The generation in which the global optimum is obtained indicates the convergence of the
population.

3. PSO Algorithm
PSO introduced by Kennedy and Eberhart simulates the behaviors of bird flocking. In PSO, the
solutions to the problem are represented as particles in the search space. PSO is initialized with a
group of random particles (solutions) and then searches for optimum value by updating particles
in successive generations. In each iteration, all the particles are updated by following two best
values. The first one is the best solution (fitness) it has achieved so far. This value is called pbest.
Another "best" value that is tracked by the particle swarm optimizer is the best value, obtained so
far by any particle in the population. This best value is a global best and called gbest.
Particle Swarm has two primary operations: Velocity update and Position update. During each
generation each particle is accelerated toward the particles previous best position and the global
best position. At each iteration, a new velocity value for each particle is calculated based on its
current velocity, the distance from its previous best position, and the distance from the global best
position. The new velocity value is then used to calculate the next position of the particle in the
search space. This process is then iterated a set number of times or until a minimum error is
achieved.
After finding the two best values, the particle updates its velocity and positions with the following
formulae
v[t + 1] = v[t] + c1 rand1 (pbest[t] present[t] ) +
c2 rand2 (gbest [t] present[t])
present[t + 1] = present[t] + v[t]

v[ ] is the particle velocity, present[ ] is the current particle. pbest[ ] and gbest[ ] are local
best and global best position of particles. rand () is a random number between (0,1). c1 and c2 are
acceleration factors. Usually c1 = c2 = 2.05. [2]

The outline of basic Particle Swarm Optimizer is as follows

Step 1: Initialize the population
Step 2: Evaluate fitness of the individual particle (update pbest)
Step 3: Keep track of the individuals highest fitness (gbest)
Step 4: Modify velocities based on pbest and gbest
Step 5: Update the particle position
Step 6: Terminate if the condition is met
Step 7: Go to step 2

4. Quantum PSO
The Quantum inspired Particle Swarm Optimization (QPSO) is one of the recent optimization
methods based on quantum mechanics. Like any other evolutionary algorithm, a quantum inspired
particle swarm algorithm relies on the representation of the individual, the evaluation function and
the population dynamics. The particularity of quantum particle swarm algorithm stems from the
quantum representation it adopts which allows representing the superposition of all potential
solutions for a given problem. Moreover, the position of a particle depends on the probability
amplitudes a and b of the wave function . QPSO also stems from the quantum operators it uses
to evolve the entire population through generations. QPSO constitutes a powerful strategy to
diversify the QPSO population and enhance the QPSOs performance in avoiding premature
convergence to local optima.
In terms of classical mechanics, a particle is depicted by its position vector xi and velocity vector
vi, which determine the trajectory of the particle. The particle moves along a determined trajectory
following Newtonian mechanics, but this is not the case in quantum mechanics. In quantum world,
the term trajectory is meaningless, because xi and vi of a particle cannot be determined
simultaneously according to uncertainty principle. Therefore, if individual particles in a PSO
system have quantum behavior, the PSO algorithm is bound to work in a different fashion.
In the quantum model of a PSO, the state of a particle is depicted by wave function (x, t), instead
of position and velocity. The dynamic behavior of the particle is widely divergent from that of the
particle in traditional PSO systems. The particles move according to the following iterative
equations:
x(t+1) = p + * |mbest x(t)| * ln(1/u), if k >= 0.5
x(t+1) = p * |mbest x(t)| * ln(1/u), if k < 0.5

Where,
Local attractor, p = (c * pid + (1-c) * pgd)
c = (c1 * r1)/ (c1 * r1 + c2 * r2)
u,k,r1,r2 are uniformly distributed random numbers in the interval (0,1)
is the contraction-expansion coefficient in the interval (0,1)
c1 and c2 are the acceleration factors (usually c1 = 1.82 and c2 = 1.97) [8]
pid is the pbest of the ith particle
pgd is the global best particle
Mean best position or Mainstream Though point (mbest) of the population is defined as the
mean of the best position of all particles.

5. Adaptive QPSO
QPSO is mainly conducted by four key parameters important for the convergence and efficiency
of the algorithm: the contraction-expansion coefficient (), mean best position (mbest) and two
positive acceleration factors (c1 and c2). Contraction-expansion coefficient is a convergence factor
used to balance between exploration and exploitation by using the previous flying experience of
the particles. Mean best position is replaced with Weighted mean best position to determine
whether a particle is elitist or not. Acceleration parameters are typically two positive constants,
called the cognitive c1 and social parameters c2.
Adaptive Quantum Particle Swarm Optimization updates the algorithmic parameters dynamically
by identifying the evolutionary state that the particle belongs during each generation. Earlier QPSO
alternatives for parameter adaptation do not consider the particle state while AQPSO takes into
account the state of the particle obtained from diversity information. The Evolutionary State
Estimation (ESE) approach is adopted to identify the evolutionary states that the particle undergoes
in each generation. The parameters are adjusted according to the estimated state in order to provide
a better balance between global exploration and local exploitation. Additionally, an Elitist
Learning Strategy (ELS) is developed for the best particle to jump out of possible local optima.
5.1. Evolutionary State Estimation (ESE)
The Evolutionary State Estimation approach uses the population distribution information of the
particles to identify the state of the particle. Initially, the particles are dispersed throughout the
search space. During the evolutionary process the particles tend to crowd towards a globally
optimum region. Hence the population distribution information is necessary to identify the state
that the particle undergoes. The distribution information can be calculated by finding the mean
distance between each and every particles in the population. It is seen that the particles are closer
to globally best particle during the convergence state when compared to other states. The ESE
approach is detailed in the following steps:
Step 1: At the current position, calculate the mean distance of each particle i to all the other
particles. The mean distance can be measured using an Euclidian metricN

j=1,ji

k=1

1
di =
(xik xjk )2
N1
Where, N and D are population size and number of dimensions respectively.
Step 2: The distance of globally best particle is denoted as dg. Compare all distances and determine
the maximum (dmax) and minimum (dmin) distances. Determine the evolutionary factor f usingf=

dg dmin
[0,1]
dmax dmin

Step 3: Classify f into one of the four sets S1, S2, S3, and S4, which represent the states of
exploration, exploitation, convergence, and jumping out respectively.

Fig 5.1.1 Fuzzy membership functions for the four evolutionary states

Step 4: Adaptation of the Acceleration Factors: The acceleration factors c1 and c2 should be
controlled dynamically depending on the identified evolutionary state of the particle. Adaptive
control can be designed for the acceleration coefficients based on the following notion. Parameter
c1 denotes the self-cognition that brings the particle to its own historical best position, helping
explore local niches and maintaining the diversity of the swarm. Parameter c2 denotes the social
influence that pushes the swarm to converge to the current globally best region, helping with fast
convergence. Initialize c1 = 1.82 and c2 = 1.97, which satisfies the convergence condition of the
particles = (c1 + c2)/2 1. Since c2 > c1, the particles will converge faster to the global optimal
position than the local optimal position of each particle. [8]
These are two different learning mechanisms and should be given different treatments in different
evolutionary states. In this paper, the acceleration factors are both initialized to 1.82 and 1.97
respectively and adaptively controlled according to the evolutionary state, with strategies
developed as mentioned in table below. The values of c1 and c2 are varied in the range of 0.15.[8]

State
Exploration
Exploitation

Strategy
Strategy 1
Strategy 2

c1
c2
Increase
Decrease
Increase
Decrease
slightly
slightly
Convergence
Strategy 3 Increase
Increase
slightly
slightly
Jumping out
Strategy 4 Decrease Increase
Table 5.1.1 Strategies for the control of c1 and c2

Strategy 1: In Exploration state c1 is increased so that the particles search for the maximum
number of possible solutions instead of crowding at the same region. Decreasing c2 helps the
particles to refine its search in finding their historical best position instead of trying to find new
solutions.

Strategy 2: In exploitation state the particles make use of local information and group around the
current pbest of each particle. Thus, increasing c1 slightly will promote exploitation around pbest.
Since the global optima will not be found at this stage decreasing c2 slowly and keeping a low
value will avoid possible illusion of local optima.
Strategy 3: In convergence state the swarm is about to find the globally optimum region. Hence,
increasing c2 slowly will guide the other entire particles to this region. But c2 is decreased to avoid
probable local searches to let the particles converge fast.
Strategy 4: In jumping out state the globally best particle shifts towards a new optimal region as
soon as it finds that the particles are crowed in the local optimal region in convergence state. The
whole swarm should be updated to the new region as soon as possible. Hence maintaining a large
value of c2 with smaller c1 will help us in achieving this.

5.2. Adaptation of Contraction-Expansion Coefficient ()

The contraction-expansion coefficient is used to keep the balance between exploration and
exploitation by using the previous flying experience of the particles. The particles change its path
according to its best position and also by using the information from its neighbors. In addition, the
contraction-expansion coefficient is also an important convergence factor; from the results of
stochastic simulations, QPSO has relatively better performance by varying the value from 1.0 at
the beginning of the search to 0.5 at the end of the search to balance the exploration and
exploitation. [3]

5.3. Elitist Learning Strategy (ELS)

The most important problem is to determine whether a particle is an elitist or not, or say it exactly,
how to evaluate its importance in calculate the value of mean best position (m). It is natural, as in
other evolutionary algorithm, that we associate elitism with the particles fitness value. The greater
the fitness, the more important the particle is. Describing it formally, we can rank the particle in
descendent order according to their fitness value first. Then assign each particle a weight
coefficient i linearly decreasing with the particles rank, that is, the nearer the best solution, the
larger its weight coefficient is.
The Weighted Mean best position (or) Mainstream Thought point (m) is calculated as

Where,
i is the weight coefficient (decreases linearly from 1.5 to 0.5) [6]
M is the population size

The steps involved in Adaptive Quantum Particle Swarm Optimizer are as follows
Step 1: Initialize the population
Step 2: Set c1 to 1.82 and c2 to 1.97
Step 3: Evaluate fitness of the individual particle (update pbest)
Step 4: Keep track of the individuals highest fitness (gbest)
Step 5: Update mean best position
Step 6: Compute evolutionary factor by using the distribution information of particles in
the search space
Step 7: Estimate the evolutionary state using fuzzy classification and adaptively control
the algorithmic parameters as mentioned in ESE
Step 8: Perform ELS once the particles get into convergence state
Step 9: Update the particle position
Step 10: Terminate if the condition is met
Step 11: Go to step 2

6. Results and Discussion

The datasets used and the comparison of general PSO and QPSO are presented in this
section. To confirm the effectiveness of PSO and QPSO, both the algorithms were coded in Java.
Lenses, Car evaluation, Post-Operative Patient Care, Habermans Survival and Zoo datasets were
taken up from UCI Irvine repository for the experiment. The details of the datasets are listed in
Table 6.1. The association rule mining on the above mentioned datasets were performed by both
PSO and QPSO. The efficiency of both algorithms were compared by utilizing the output
parameters mentioned earlier.

Dataset
Lenses
Car Evaluation

Attributes
4
6

Swarm Size
24
700

Classes
3
4

Habermans
Survival
Post-operative
Patient Care
Zoo

310

87

16

101

Table 6.1 Dataset Description

10

6.1 Predictive Accuracy

6.2 Number of rules generated

Datasets

PSO

APSO
8

PSO +
SFLA
13

APSO +
SFLA
17

QPSO
(Phase1)
18

AQPSO
(Phase2)
21

Lenses(24)

Car(700)

33

386

49

619

685

691

Haberman(310)

34

258

72

281

301

303

Postop(87)

19

20

22

41

79

82

Zoo(101)

20

25

45

70

92

95

Datasets

PSO

APSO

Lenses(24)

0.501

0.636

PSO +
SFLA
0.560

APSO +
SFLA
0.641

QPSO
(Phase1)
0.576

AQPSO
(Phase2)
0.624

Car(700)

0.51

0.642

0.564

0.651

0.58

0.627

Haberman(310)

0.5

0.645

0.511

0.656

0.59

0.633

Postop(87)

0.5

0.560

0.532

0.634

0.563

0.61

Zoo(101)

0.5

0.570

0.534

0.570

0.56

0.589

6.3 Laplace

11

6.4 Conviction
The conviction value obtained for both PSO and QPSO is found to be infinity which implies that
the confidence of the rule is maximum.

7. Conclusion:
The QPSO algorithm is superior to the standard PSO mainly in three aspects. Firstly, quantum
theory is an uncertain system in which different state of the particles and a wider searching space
of the algorithm can be generated. Secondly, the introduction of mbest into QPSO is a benchmark
in Quantum Theory. In the standard PSO, it converges fast, but at times, the fast convergence
happens in the first few iterations but falls into a local optimal situation easily in the next few
iterations. With the introduction of mbest in QPSO, the average error is lowered, since each particle
cannot converge fast without considering its colleagues, which makes the frequency lower than
PSO. Lastly, QPSO has fewer parameters compared to standard PSO and it is much easier to
implement and run. Hence the performance of the algorithm is significantly improved by QPSO.

References
1. Kennedy, J. and Eberhart, R. Particle Swarm Optimization. IEEE International Conference
on Neural Networks (Perth, Australia), IEEE Service Center, Piscataway, NJ, IV:
19421948, 1995.
2. Zhan, Z-H. and Zhang, J. and Li, Y. and Chung, H.S-H, Adaptive particle swarm
optimization. IEEE Transactions on Systems Man, and Cybernetics Part B: Cybernetics,
39 (6), 2009,pp. 1362-1381.
3. L.D.S. Coelho, A Quantum Particle Swarm Optimizer with Chaotic Mutation Operator,
Chaos, Solitons and Fractals, Vol.37, No.5, 2008, pp. 1409-1418.
4. Layeb. A and Saidoini D.E : Quantum Genetic Algorithm for Binary Decision Diagram
Ordering Problem, In the proceeding of International Journal of Computer Science and
Network Securiy, Vol.7, No 9 (2007) 130-135.
5. K.Indira, S.Kanmani, P.Prashanth, V.Harish Konda Ramcharan Teja, R.Jeeva, Population
Based Search Methods in Mining Association Rules, in: Third International Conference on
Advances in Communication, Network, and Computing CNC 2012, LNCS, 2012, pp.
255261.
6. J. Sun, W.B. Xu, W. Fang, quantum-behaved particle swarm optimization algorithm with
controlled diversity, in: International Conference on Computational Science (3) 2006,
2006, pp. 847854.
7. J. Sun, W.B. Xu, W. Fang, Enhancing Global Search Ability of Quantum-Behaved Particle
Swarm Optimization by Maintaining Diversity of the Swarm, in: RSCTC 2006, 2006, pp.
736745.
8. Yourui Huang, Liguo Qo and Chaoli Tang, Optimal Coverage Scheme based on QPSO in
Wireless Sensor Networks, Journal of Networks, VOL.07, NO.09, September 2012.

12

8. CERTIFICATE
This is to certify that the write up was prepared after discussing with me and I approve the
content presented.

Signature of the project guide