See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/2673329

A Genetic Algorithm for Database Query Optimization

Article · February 1970

Source: CiteSeer

CITATIONS READS
106 886

3 authors:

Kristin P. Bennett Michael C. Ferris

Rensselaer Polytechnic Institute University of Wisconsin–Madison
153 PUBLICATIONS   6,924 CITATIONS    239 PUBLICATIONS   7,225 CITATIONS   

SEE PROFILE SEE PROFILE

Yannis Ioannidis
National and Kapodistrian University of Athens
318 PUBLICATIONS   8,299 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

DL.org - Coordination Action on Digital Library Interoperability, Best Practices, and Modelling Foundations View project

Optique View project

All content following this page was uploaded by Kristin P. Bennett on 11 March 2015.

The user has requested enhancement of the downloaded file.

 A Genetic Algorithm for Database Query Optimization

Kristin Bennett Michael C. Ferris Yannis E. Ioannidis

Computer Sciences Department
University of Wisconsin
1210 West Dayton Street
Madison, Wisconsin 53706

Abstract that satises

c(s0 ) = min
s2S
c(s):
Current query optimization techniques are
inadequate to support some of the emerging Query optimization has been an active area of research
database applications. In this paper, we out- ever since the beginning of the development of rela-
line a database query optimization problem tional DBMSs. Good surveys on query optimization
and describe the adaptation of a genetic algo- and other related issues can be found elsewhere [9, 10].
rithm to the problem. We present a method
for encoding arbitrary binary trees as chro- In the relational model, data is organized in relations,
mosomes and describe several crossover op- i.e., collections of similar pieces of information called
erators for such chromosomes. Preliminary tuples. Relations are the data units that are refer-
computational comparisons with the current enced by queries and processed internally. A strategy
best{known method for query optimization to answer a query q is a sequence of relational alge-
indicate this to be a promising approach. In bra operators applied to the relations in the database
particular, the output quality and the time that eventually produces the answer to q. The cost of
needed to produce such solutions is compara- a strategy is the sum of the costs of processing each
ble to and in general better than the current individual operator. Among these operators, the most
method. dicult one to process and optimize is the join, de-
noted by 1. It essentially takes as input two rela-
tions, combines their tuples one-by-one based on cer-
tain criteria, and produces a new relation as output.
1 INTRODUCTION Join is associative and commutative, so the number of
alternative strategies to answer a query grows expo-
Genetic algorithms [4, 6] are becoming a widely used nentially with the number of joins in it. Moreover, a
and accepted method for very dicult optimization DBMS usually supports a variety of join methods (al-
problems. In this paper, we describe the implemen- gorithms) for processing individual joins and a variety
tation of a genetic algorithm (GA) for a problem in of indices (data structures) for accessing individual re-
database query optimization. In order to give a care- lations, which increase the options even further. Thus,
ful formulation of our GA, we rst give a broad outline all query optimization algorithms primarily deal with
of this particular application. join queries. These are the focus of this paper as well.
The key to the success of a Database Management Sys- In current applications, each query usually involves a
tem (DBMS), especially of one based on the relational small number of relations, e.g., less than 10. Hence,
model [3], is the eectiveness of the query optimiza- although exponential in the number of joins, the size
tion module of the system. The input to this module of the strategy space is manageable. Most commercial
is some internal representation of a query q given to database systems use variations of the same query op-
the DBMS by the user. Its purpose is to select the timization algorithm, which performs an exhaustive
most ecient strategy (algorithm) to access the rele- search over the space of alternative strategies, and
vant data and answer the query. Let S be the set of whenever possible, uses heuristics to reduce the size
all strategies appropriate to answer a query q. Each of that space. This algorithm was rst proposed for
member s of S has an associated cost c(s) (measured the System{R prototype DBMS [14], so we refer to it
in terms of CPU and/or I/O time). The goal of any as the System{R algorithm.
optimization algorithm is to nd a member s0 of S Current query optimization techniques are inadequate
to support the needs of some of the newest database S T
application domains, such as articial intelligence 2 4
(e.g., expert and deductive DBMSs), CAD/CAM (e.g., P 3 H
engineering DBMSs), and other disciplines (e.g., sci-
entic DBMSs). Simply put, queries are much more
complex both in the number of operands and in the di-
1 5
R
versity and complexity of operators in the query. This M
greatly exacerbates the diculty of exploring the space Figure 1: Query Graph
of strategies and demands that new techniques be de-
veloped.
One of the proposed solutions is to use randomized
algorithms. Simulated Annealing, Iterative Improve- paper, we study tree queries, i.e., queries whose query
ment, and Two-Phase Optimization (a combination of graph is a tree. The answer to a given query is con-
the rst two) have already been successfully tried on structed by combining the tuples of all the relations in
query optimization [8, 15, 7], giving ample reason to a query based on the constraints imposed by the spec-
believe that a GA will perform well also. Many of the ied joins. This is done in a step-wise fashion, each
operators used in these studies can be adapted for use step involving a join between a pair of relations whose
in a GA and incorporated into a standard GA code. tuples are combined. These can be relations originally
An advantage of our version of the GA [1], is that stored in the database or results of operations from
it is designed for a parallel architecture and signi- previous steps (called intermediate relations). As a
cant computational savings over the other randomized very strong and eective heuristic, database systems
methods can be obtained by a parallel implementation. never combine relations that are not connected with a
This paper is organized as follows. Section 2 denes join in the original query. This is because such an op-
two strategy spaces that are of interest to query op- eration produces the cartesian product of the tuples in
timization, which were used in our experiments. It the two relations. Not only is this an expensive opera-
also contains a description of the System{R algorithm, tion, but its result is also very large, thus increasing the
which is used as a basis for comparison of our results. cost of subsequent operations. Most query optimizers
Section 3 describes the specic genetic algorithm that conne themselves into searching the subspace of S of
we developed, including the representation that we strategies with no cartesian products. This heuristic
used for the chromosomes for the two strategy spaces is adopted in the work presented in this paper as well.
and the adopted crossover operators. Section 4 con- Given the above, each strategy to answer a query can
tains the results of our experiments. Finally, Section be represented as a join processing tree. This is a tree
5 gives a summary and provides some direction for fu- whose leaves are database relations, internal nodes are
ture work. join operators, and edges indicate the ow of data from
bottom-up. In addition, the chosen index for each
2 QUERY OPTIMIZATION database relation and the chosen join method for each
join is specied. If all internal nodes of such a tree have
SPECIFICS at least one leaf as a child, then the tree is called lin-
ear. Otherwise, it is called bushy. Most join methods
2.1 STRATEGY SPACES distinguish the two join operands, one being the outer
(left) relation and the other being the inner (right) re-
Most query optimizers do not search the complete lation. An outer linear join processing tree (left{deep
strategy space S , but a subset of it, which is expected tree) is a linear join processing tree whose inner rela-
to contain the optimum strategy or at least one with tions of all joins are base relations. In this study, we
similar cost. To understand the various options, we deal with two strategy spaces: one that includes only
need some denitions related to databases. In a slight left-deep trees, which is denoted by L, and one that
abuse of notation, consider the following query: includes both linear and bushy ones, which is denoted
(A 1 C) and (B 1 C) and (C 1 D) by A. Examples of a left-deep tree and a bushy tree for
and (D 1 E) and (D 1 F) (1) query (1) are shown in Figure 2 (avoiding the details
of the join constraints and the join methods). The in-
Each join is associated with a constraint (omitted for terest in L stems from the fact that many DBMSs are
clarity of presentation) that species precisely which using it as their strategy space, and is the one on which
tuples of the joined relations are to appear in the re- the System{R algorithm can be applied. We experi-
sult. Query (1) can be represented by a query graph ment with A as well, because quite often the optimum
[16], which has the query relations as nodes and the strategy is not in L. We present results for applying
joins between relations as undirected edges, as shown a genetic algorithm on both spaces and compare them
in Figure 1. Throughout, we use capital letters to de- with the results of applying the System{R algorithm
note relations and numbers to represent joins. In this on L.
 4
rest cannot be part of the optimumnal strategy. This
process needs as many iterations as there are relations
5 E in the query to complete. The main disadvantage of
(a) the algorithm is that it needs to maintain and process
3 F a very large number of strategies during its execution,
a number that grows exponentially with the number
2 D of joins in the query. Especially towards the later it-
erations, this implies both a signicant cpu cost for
1 B processing and a signicant I/O cost due to increased
page faults. This makes the algorithm inapplicable to
A C queries with more than about 16 relations.
The above informal description of the algorithm is
4 slightly simplied. In the interest of space, we have
not discussed various complications that arise from
side-eects that the use of specic types of indices can
(b) 3 E have on the desirability of a strategy. However, the
version of the algorithm that was used in our experi-
ments did address all these complications as well. The
2 5
interested reader can nd further details in the original
1 B F D
paper on the System{R algorithm [14].
A C 3 GENETIC ALGORITHM
Figure 2: (a) Left-deep Tree; (b) Bushy Tree
In this section, we describe the implementation of a ge-
netic algorithm to solve the problem outlined above.
We should emphasize at this point that, even after ig- For completeness, we brie y review our terminology,
noring all strategies that contain cartesian products, details of which can be found in [1]. Our GA works
the number of the remaining strategies still grows ex- with a population of chromosomes, each of which can
ponentially with the number of joins in the query. be decoded into a solution of the problem. For each
The above is true even for the L space, let alone for chromosome i in the population, a measure of its qual-
the A space, which is a superset of the former. The ity is calculated, called its tness, f(i). Chromosomes
size of these spaces depends not only on the specic are selected from the population to become parents
query which is being optimized (being dependent on based on tness. Then, reproduction occurs between
the shape of its query graph), but also on the number pairs of chromosomes to produce ospring. The newly
of join methods supported by the system and the index created population becomes the next generation and
structures that exist on the query relations. Hence, the process is repeated. The model that we use has a
although cartesian products are excluded from strate- xed population size N.
gies, even for the small queries with which we deal in Our GA [1] uses a neighborhood scheme in which the
this paper (up to 16 joins), the corresponding L and A tness information is only transmitted within a local
spaces are very large and the associated optimization neighborhood, see for example [12]. A model algo-
problem is computationally very dicult. rithm for such a scheme is as follows.
2.2 THE SYSTEM{R ALGORITHM
Local Neighborhood Algorithm:
The System{R algorithm is based on dynamic pro-
gramming. Specically, the complete space L is con- repeat
structed, occasionally pruning parts of it that are iden- for each chromosome i do
tied as suboptimal. The space is constructed by iter- evaluate f(i)
ation on the number of relations joined so far. That is, broadcast f(i) in the neighborhood of i
at the k-th iteration, the best strategy to join k rela- receive f(j) for all chromosomes j
tions from the query is found, for all such sets of k rela- in the neighborhood
tions. In the next iteration, strategies of k+1 relations select chromosome k to mate from the
are constructed, by combining each strategy from the neighborhood of i based on tness
previous collection with the appropriate remaining re- reproduce using chromosomes i and k
lations. For each set of k+1 relations, multiple strate- replace chromosome i with one
gies are usually constructed, of which only the one with of the ospring
the least cost is kept, since it can be shown that all the until population variance is small
In the experiments that we report below, we have used gene consists of a relation and a join method. For
a neighborhood structure we call ring6, which consid- example,
ers chromosomes i and j to be neighbors if JA JC JB JD JF JE
min(ji ? j j ; ji + N ? j j ; ji ? N ? j j)  3 represents the left{deep strategy in Figure 2(a), with
This can be viewed as each chromosome residing on a J representing some join method.
ring with neighbors that are chromosomes no further In our code, we associate the join method with the in-
than three links away. This neighborhood structure ner (right) relation of each join. Thus, to recreate the
was chosen since it has proven very eective in other join processing tree, join the rst and second relations
problem instances [1]. For query optimization, the aim using the method associated with the second relation.
is to minimize the cost of the strategy, so to generate a Then join the resulting intermediate relation with the
tness distribution in a neighborhood, we take the neg- next relation according to the specied method. Re-
ative of the cost (to convert to maximization) and use peat until no relations remain. At each step verify that
a linear scaling of these values in each neighborhood an edge exists in the query graph (Figure 1) between
so that the maximum tness is some proportion (user the current relation and one of the relations that oc-
supplied) of the average tness. Details of this scaling curred previously in the chromosome. If no such edge
routine are found in [1]. Since each chromosome i only exists, then the query strategy contains a cartesian
selects one mating partner, selection is carried out by product, and the chromosome is penalized with an in-
choosing chromosome k from the neighborhood with
probability X
fk = fj :
nite cost. Note that the join method associated with
the rst relation is ignored and that if the crossover
method produces many cartesian products, then the
j 2nhd(i) GA will not perform well.
Reproduction produces two ospring. The current Using this encoding, the problem is similar to a con-
chromosome is replaced with its best ospring pro- strained traveling salesman problem (TSP) with a
vided this ospring is better than the worst chromo- choice of methods of transport between the cities. In
some in the neighborhood (see [12]). fact we use this analogy to motivate our choices of
For a GA to be an ecient optimization technique, crossover operator. However, the query optimization
we believe that reasonable parameter and algorithm function is much more expensive to evaluate than typ-
choices (such as those detailed above) can be made ical TSP functions, since each join, or equivalently the
without reference to a particular problem. However, cost of traveling directly between cities, can be depen-
the eectiveness of the overall algorithm depends cru- dent on the route previously taken and/or the future
cially on the representation of the problem and the op- cities to be visited. As an indication of the above com-
erators we use to exchange genetic information. Thus, plexity, we want to emphasize that the cost of a join
we now specialize to the particular problem of query between two relations is a function of their sizes. That
optimization. We describe two ways of encoding this size depends directly on the precise set of joins that
problem which correspond to the two strategy spaces have occurred previously. It also depends on the joins
L and A. We attempt to incorporate as much prob- that remain to be processed later, because much of the
lem specic information as possible (see [5]), and show data that is necessary for their execution is contained
how mutation, initial population choice and crossover in the two relations.
are carried out. Our discussion is broken into three Mutation is a secondary operator used to guarantee
parts, the rst dealing with left{deep strategies, the connectedness of the search pace. In our implementa-
second with bushy strategies, and the third dealing tion it is of two types. The rst type changes the join
with crossover operators. method randomly, and the second swaps the order of
two adjacent genes. A left{deep strategy generator
3.1 LEFT{DEEP STRATEGIES ensures that the initial population contains no carte-
Left{deep strategies (L) are a relatively small subset sian products. This is achieved by cycling through a
of the strategy space A. However, it has been observed randomly generated permutation of the relations, only
that frequently a strategy exists in L whose cost is very adding a relation to our chromosome if this can be
close to the optimal cost over A. System{R uses L as done without introducing a cartesian product. The
its strategy space, so it is appropriate to apply a GA in join methods are generated randomly.
L and compare the results of these two methods. The 3.2 BUSHY STRATEGIES
advantage of choosing L over A is that the search space
is much smaller (although still large in absolute terms); Quite often the best strategy for a query is in L. How-
the disadvantage is that we cannot beat System{R in ever, in order to produce better solutions that System{
terms of output quality using a GA in L. R, we must consider a larger strategy space, (A), which
Each chromosome represents a left{deep strategy. The contains both linear and bushy strategies. We en-
chromosome is an ordered list of genes, where each code such strategies into chromosomes by consider-
ing each join as a gene, so that koJ represents join randomly change the join method or the orientation.
k with some join method J and its constituent rela- The second is to perform reordering of genes on the
tions (found on the query graph) in o orientation (for chromosome by transposing a gene with its neighbor.
instance (a)lphabetically or (r)everse{alphabetically). Together, these guarantee that the search space is con-
A chromosome is then an ordered list of these genes. nected. The initial population is generated randomly.
An example is
1Ja 5Jr 2Jr 3Ja 4Ja 3.3 CROSSOVER
which represents the strategy given in Figure 2(b). In order to complete the discussion of our method, we
The decoding of the list into a solution is more costly describe the two crossover operators that we investi-
than the left{deep decoding but has the ability to rep- gated. In each of the encodings above, the chromosome
resent many more strategies. The decoding process is an ordered list of genes. As outlined above, in the
grows the bushy tree from the bottom up. It maintains left{deep case the genes can be identied by their rela-
a list of intermediate relations waiting to be joined. tion letter and in the bushy case by their join number.
Scanning the chromosome from left to right, it nds We describe the crossover operators solely in terms of
the constituent relations in each join (gene) by exam- these genes.
ining the query graph. The orientation of the gene The rst method, modied two swap (M2S), modies
indicates which relation is the outer (left) and inner the local improvement algorithm given in [11] to in-
(right) relation. If the right relation has been used in corporate information from both parents and was de-
the formation of some intermediate relation in the list, signed primarily for the left{deep case. It can be de-
the latter is substituted as the right relation of the join scribed as follows. Given two parent chromosomes, X
and the intermediate relation is removed from the list. and Y , randomly choose two genes in X and replace
The same process is done for the left relation. The left them by the corresponding genes from Y , retaining
and right relations are joined according to the method their order from Y , to create one ospring. For exam-
in the gene, and the resulting relation is added to the ple, in the left{deep case, suppose the parent chromo-
list. After all the genes are processed, one intermedi- somes X and Y are given by
ate relation remains in the list. This corresponds to X = mA nC mB mD nF nE ;
the root of the bushy tree. Y = mB nC mD mF mE nA
Note that although the representation is similar to the where m and n represent particular join methods and
one described for left{deep trees (it is an ordered list we randomly choose genes labeled A and D. The re-
of genes), there are three important dierences. First, sulting chromosome is
the decoding scheme guarantees that the correspond-
ing query strategy has no cartesian products. Thus, mD nC mB nA nF nE
we only consider \feasible" strategies of our problem. We interchange the roles of X and Y to create an-
Second, our representation is based on labeling joins, other ospring. The use of M2S was partly motivated
not relations. This is somewhat natural since there by the Swap transformation that has been successfully
may be several intermediate relations in use at any used for database query optimization in the context of
given step of the decoding scheme and these relations other randomized algorithms [7, 15]. The two transfor-
can be easily associated with a join. This association is mations are quite similar, except that, as a crossover,
critical for simple computation. In the left{deep case, the transformation takes into account two strategies,
only one intermediate relation is constructed at each whereas in its previous use it simply operates on one.
stage, so the problem of handling these relations does Note that most of the ordering information from one
not occur. Third, the representation is not uniquely of the chromosomes is retained, and it is this informa-
dened, i.e., several chromosomes can decode into the tion that is of primary importance in both decoding
same tree. schemes outlined above. Therefore, the crossover op-
The coding described above has the advantage that it erator retains most of the ordering information from
may now be possible to beat the System{R solution. one chromosome, and uses order information from the
However, the search space has been greatly increased other chromosome to exchange two genes. In the bushy
giving the GA a more dicult task. We believe that case, the modied chromosome can be very dierent
the decoding scheme is very important for this GA from both of its parents (depending on the locus of the
to perform well: the extra work that we carry out in genes being exchanged) due to the decoding scheme;
decoding guarantees that the algorithm only consid- in the left{deep case the solution will look very similar
ers feasible solutions of the problem, and furthermore, to one parent, although there is a small possibility of
that we can use standard crossover schemes motivated introducing a cartesian product.
by GA's for TSP and other database work to generate The second method, which we refer to as CHUNK, is
good chromosomes. adapted from [2, 12] and was designed primarily for
Mutation is carried out in two ways. The rst is to the bushy case. Here, we generate a random chunk
of the chromosome as follows. Suppose the number of [14]. Tree queries were generated randomly whose size
genes in the chromosome is l. The start of the chunk ranged from 5 to 16 joins. The limit on the query size
(of genes) is a uniformly generated random integer in was due to the inability of the System{R algorithm to
[0; l=2] and the length of the chunk is uniformly gen- run with larger queries, primarily because of its huge
erated from [l=4; l=2]. Suppose we randomly generate memory requirements. Moreover, not all generated 16-
the chunk [3; 4], then one resulting chromosome copies join queries were runnable by the System{R algorithm.
the third and fourth genes of X into the same position Thus, for large queries, the genetic algorithm is clearly
in the ospring, then deletes the corresponding genes superior to the traditional algorithm.
of Y , using the remainder of Y 's genes to ll up the In the interest of space, we do not present the pre-
remaining positions of the ospring. For example, if cise cost formulas that were used in this study. They
X and Y are given by capture the I/O cost of the various join methods and
X = 1nna 5nnr 2mmr 3mma 4mna ; indices used and can be found in any textbook on
Y = 3r 5a 1r 4a 2a databases. We also avoid presenting any details on the
assumed physical design of the database. The specics
where m and n represent particular join methods, the are exactly as in previous studies [7].
resulting chromosome is
We implemented all algorithms in C, and tested them
5na 1mr 2mr 3ma 4ma on a dedicated DecStation 3100 workstation. All ex-
Again, another chromosome is created by interchang- periments were conducted with a population size of
ing the roles of X and Y . We arrived at this method N = 64. Ten dierent queries were tested for each size
after some experimentation. The essential motiva- up to 16 joins. However, System{R managed to termi-
tion behind the above scheme is to force a reason- nate on only seven of the 16-join queries, so the results
ably sized subtree (between a quarter and a half of for that case represent a smaller number of queries.
the size of the original tree) from one parent to be For each query, each algorithm was run ve times.
incorporated into the other parent with minimal dis- 4.2 OUTPUT QUALITY
ruption to the latter. Of course, the chunk may or
may not correspond to a subtree, but to avoid exces- We compare output quality based on the lowest cost
sive computation in determining a chunk, the above chromosome in the nal generation. The cost of the
scheme was used. The ordering information on the average output strategy produced by the algorithms
chromosome crucially determines the strategy in our as a function of the query size is shown in Figure 3(a).
decoding scheme, so the crossover operator attempts The x{axis is the number of joins in the query. The
to minimize ordering changes. Since our representa- y{axis represents scaled cost, i.e., the ratio of the out-
tion is many to one, forcing the chunk to have a large put strategy cost over the cost found by the System-R
size generally results in some genetic information being algorithm. For each size, the average over all queries
exchanged, enabling the algorithm to look at a larger of that size, of the average scaled output cost over all
variety of solutions and hence generate better solutions ve runs of each query is shown.
in reasonable times.
Although the crossover operators were designed with The results are rather interesting. We observe that
particular strategy spaces in mind, both operators can on the average, when GA is applied to L, it fails to
be applied in the two strategy spaces since they essen- nd the optimum strategy, but except for the largest
tially consider the chromosomes as an ordered list of queries (16 joins), it is clearly within a small range
genes. Thus, we experimented with all combinations (10%) of optimality. For small queries (4-6 joins), both
of crossover operators and spaces. crossovers always nd the optimum. As the query size
grows, however, the algorithm becomes less stable and
the quality of its output deteriorates, primarily due to
4 PERFORMANCE RESULTS the dramatic increase in the size of the strategy space.
When GA is applied to A the results improve. On the
In this section, we report on an experimental evalua- average, for all three sizes, the algorithm found a bet-
tion of the performance and behavior of the above ge- ter strategy than the best left-deep tree, with the gains
netic algorithm on query optimization compared to the ranging up to 13%. Note that as query size grows,
System{R algorithm. First, we describe the testbed GA becomes relatively better than System{R. This is
that we used for our experiments, and then we discuss because, with increased query size, the relative dier-
the obtained results. ence between the best bushy strategies and the best
left-deep strategies increases as well, so by searching
4.1 TESTBED a richer space, GA is able to improve on the output
quality. A small set of experiments with an increased
For our experiments we assumed a DBMS that sup- population size has given very promising results for
ports the nested{loops and merge{scan join methods further improving the output quality in large queries.
 since it generates a large variety of strategies, thus
looking at a larger area of the space and using its time
1.5 GA in L with CHUNK
much more eectively.
To overcome some of the inherent problems of random-
GA in L with M2S

ized algorithms, it is occasionally proposed that such

System-R in L
1.4

algorithms are run multiple times on a given instance

GA in A with M2S

GA in A with CHUNK
problem, and the best solution among those found be
chosen. With that in mind, we also compare the best
1.3

output found among the ve runs of each version of the

S
c
a 1.2
l GA algorithm for each query. We show the average of
e
that over all queries of a given size in Figure 3(b). We
now see that in L, M2S is perfect, almost always nd-
d

C 1.1
o
ing the optimum strategy. CHUNK is considerably
improved as well, but is still has inferior performance
s
t
1.0
for the reasons explained above. Similar improvements
are seen in the A space as well. Especially in the large
joins, both crossovers nd very good strategies. All
these results indicate that multiple runs of the GA al-
0.9

gorithm may be a plausible way to avoid some of its

0.8
4 8 12 16
potential instabilities and produce high quality results.
Number of Relations

1.5
(a)
GA in L with CHUNK 4.3 TIME
GA in L with M2S
The average time results are presented in Figure 4,
1.4
System-R in L
where the x{axis represents the number of joins in the
GA in A with M2S
query, and the y{axis represents the processing time
GA in A with CHUNK
in seconds. For the 16-join queries on which System{R
1.3
failed to nish, we use the time-to-failure in this gure.
S
The results are as follows. System{R performs faster
for queries of size up to 14, but the GA in L is much
c
a 1.2

faster for queries of size 16. The increase in times for

l
e

GA in L is almost linear. There is a larger increase in

d

the time for GA in A than in L. This is to be expected

C 1.1
o
s
t since L is a much smaller space than A. As it is obvious
1.0 from Figure 4, this increase is much less steep than
the corresponding increase in time for System{R. We
believe that even if System{R was runnable for queries
0.9
beyond 16 joins, it would require much more time than
GA.
0.8
A nal comment that we want to make is that this ver-
sion of GA is designed to be ported to a parallel ma-
4 8 12 16
Number of Relations

chine. In a parallel implementation, each chromosome

(b)

Figure 3: Scaled Cost of Strategy at Convergence: (a) in the population resides on a processor and commu-
average of 5 runs and (b) best of 5 runs nication is carried out by message passing. The total
communication overhead is thus minimal. Based on
results on other optimization problems [1] where the
Another interesting comparison is that between the evaluation of the tness function dominates the pro-
two crossovers. When GA is applied to L, M2S is the cessing time, as is the case with query optimization, we
preferred crossover, with CHUNK having much worse expect linear speedups in execution time. Since only
performance. This is due to the fact that, when the limited parallelism can be incorporated into System{
relations are the genes of the chromosome, applying R, the time to execute the parallel GA should become
CHUNK produces many ospring with cartesian prod- much smaller than that of System{R.
ucts. Therefore, in that case, the algorithm spends
much time in useless matings, thus failing to converge
to a good strategy. On the other hand, M2S produces 5 CONCLUSIONS
much fewer strategies with cartesian products and is
the overall winner. Exactly the opposite happens when We have presented a genetic algorithm for database
GA is applied in A. CHUNK is the best performer, query optimization. In doing so we have intro-
 400 GA in L with CHUNK
/ Combinatorial Optimization Conference, Wa-
GA in L with M2S
terloo, September 1990, Ontario, Canada, 1990.
System-R in L University of Waterloo Press.
GA in A with M2S
[2] G.A. Cleveland and S.F. Smith. Using genetic al-
300 GA in A with CHUNK
gorithms to schedule ow shop releases. In Scha-
eer [13], pages 160{169.
[3] E. F. Codd. A relational model of data for large
T
i
shared data banks. CACM, 13(6):377{387, 1970.
m 200
[4] D.E. Goldberg. Genetic Algorithms in Search,
Optimization and Machine Learning. Addison{
e

Wesley, Reading MA, 1989.

(
s
e
c
[5] J.J. Grefenstette. Incorporating problem specic
knowledge into genetic algorithms. In L.D. Davis,
)
100
editor, Genetic Algorithms and Simulated Anneal-
ing. Pitman, London, 1987.
[6] J. Holland. Adaptation in Natural and Articial
Systems. The University of Michigan Press, Ann
Arbor, Michigan, 1975.
0
4 8 12 16
Number of Relations
[7] Y. E. Ioannidis and Y. Kang. Randomized al-
Figure 4: Average Processing Time gorithms for optimizing large join queries. In
Proc. of the 1990 ACM-SIGMOD Conference on
the Management of Data, pages 312{321, Atlantic
duced a novel encoding/decoding of chromosomes that City, NJ, May 1990.
represent binary trees together with associated new [8] Y. E. Ioannidis and E. Wong. Query optimiza-
crossover operators. Although we did not exploit it tion by simulated annealing. In Proc. of the 1987
in this paper, an important characteristic of the al- ACM-SIGMOD Conference on the Management
gorithm is its ecient parallelization. Our computa- of Data, pages 9{22, San Francisco, CA, May
tional experiments with sequential implementations of 1987.
the algorithm have shown the method to be a viable
alternative to the commercially established algorithm. [9] M. Jarke and J. Koch. Query optimization in
In fact, for large queries, one implementation of the database systems. ACM Computing Surveys,
GA found comparable solutions in much better time, 16(2):111{152, June 1984.
whereas a dierent implementation found better qual- [10] W. Kim, D. Reiner, and D. Batory. Query Pro-
ity solutions at the expense of additional time. More- cessing in Database Systems. Springer Verlag,
over, the GA was capable of optimizing large size prob- New York, N.Y., 1986.
lems on which the established algorithm fails. [11] S. Lin and B.W. Kernighan. An ecient heuris-
In the future, we plan to adapt our parallel implemen- tic algorithm for the traveling salesman problem.
tation of the GA to query optimization and verify our Operations Research, 21:498{516, 1973.
claims on its superiority over the System-R algorithm. [12] H. Muhlenbein. Parallel genetic algorithms, pop-
In addition, we plan to investigate its applicability to ulation genetics and combinatorial optimization.
query optimization in more complex database environ- In Schaeer [13], pages 416{421.
ments, e.g., parallel database machines. [13] J.D. Schaeer, editor. Proceedings of the Third
International Conference on Genetic Algorithms,
Acknowledgements San Mateo, California, 1989. Morgan Kaufmann
Publishers, Inc.
The work described in this paper has been partially [14] P. Selinger et al. Access path selection in a re-
supported by The Air Force Laboratory Graduate Fel- lational data base system. In Proc. of the 1979
lowship Program, the Air Force Oce of Scientic Re- ACM-SIGMOD Conference on the Management
search under Grant AFOSR{89{0410 and the National of Data, pages 23{34, Boston, MA, June 1979.
Science Foundation under Grant IRI-8703592.
[15] A. Swami and A. Gupta. Optimization of large
join queries. In Proc. of the 1988 ACM-SIGMOD
References Conference on the Management of Data, pages 8{
17, Chicago, IL, June 1988.
[1] E.J. Anderson and M.C. Ferris. A genetic al- [16] J. D. Ullman. Principles of Database Systems.
gorithm for the assembly line balancing prob- Computer Science Press, Rockville, MD, 1982.
lem. In Proceedings of the Integer Programming

View publication stats