PREPARED BY ARUN PRATAP SINGH MTECH 2nd SEMESTER

PREPARED BY ARUN PRATAP SINGH 1

1

QUERY PROCESSING AND OPTIMIZATION INTRODUCTION :
A database management system manages a large volume of data which can be retrieved by
specifying a number of queries expressed in a high-level query language such as SQL.
Whenever a query is submitted to the database system, a number of activities are performed
to process that query. Query processing includes translation of high-level queries into low-
level expressions that can be used at the physical level of the file system, query optimization,
and actual execution of the query to get the result. Query optimization is a process in which
multiple query-execution plans for satisfying a query are examined and a most efficient query
plan is identified for execution.

Basic Steps in Query Processing :-
1. Parsing and translation
2. Optimization
3. Evaluation


Parsing and translation :

Check syntax and verify relations.
Translate the query into an equivalent relational algebra expression.

Optimization :

Generate an optimal evaluation plan (with lowest cost) for the query plan.

Evaluation :

The query-execution engine takes an (optimal) evaluation plan, executes that plan, and returns
the answers to the query.



UNIT : II


PREPARED BY ARUN PRATAP SINGH 2

2





PREPARED BY ARUN PRATAP SINGH 3

3





PREPARED BY ARUN PRATAP SINGH 4

4



QUERY OPTIMIZATION :

Query optimization is a function of many relational database management systems. The query
optimizer attempts to determine the most efficient way to execute a given query by considering the
possible query plans.
Generally, the query optimizer cannot be accessed directly by users: once queries are submitted to
database server, and parsed by the parser, they are then passed to the query optimizer where
optimization occurs. However, some database engines allow guiding the query optimizer with hints.



PREPARED BY ARUN PRATAP SINGH 5

5
What is Query Optimization?
Suppose you were given a chance to visit 15 pre-selected different cities in Europe. The
only constraint would be Time
-> Would you have a plan to visit the cities in any order?
Plan:
-> Place the 15 cities in different groups based on their proximity () to each other.
-> Start with one group and move on to the next group.
Important point made over here is that you would have visited the cities in a more organized
manner, and the Time constraint mentioned earlier would have been dealt with efficiently.

Query Optimization works in a similar way:
There can be many different ways to get an answer from a given query. The result would
be same in all scenarios.

DBMS strive to process the query in the most efficient way (in terms of Time) to
produce the answer.

Cost = Time needed to get all answers

Starting with System-R, most of the commercial DBMSs use cost-based optimizers.
The estimation should be accurate and easy. Another important point is the need for
being logically consistent because the least cost plan will always be consistently low.


PREPARED BY ARUN PRATAP SINGH 6

6



PREPARED BY ARUN PRATAP SINGH 7

7

Steps in a Cost-based query optimization :

1. Parsing
2. Transformation
3. Implementation
4. Plan selection based on cost estimates

Query Flow :


PREPARED BY ARUN PRATAP SINGH 8

8
Query Parser Verify validity of the SQL statement. Translate query into an internal
structure using relational calculus.
Query Optimizer Find the best expression from various different algebraic
expressions. Criteria used is Cheapness
Code Generator/Interpreter Make calls for the Query processor as a result of the
work done by the optimizer.
Query Processor Execute the calls obtained from the code generator.

Cost-based query Optimization: Algebraic Expressions
If we had the following query-
SELECT p.pname, d.dname
FROM Patients p, Doctors d
WHERE p.doctor = d.dname
AND d.dgender = M




PREPARED BY ARUN PRATAP SINGH 9

9





PREPARED BY ARUN PRATAP SINGH 10

10



SYNTAX ANALYZER :

The syntax analyser takes the query from the users, parses it into tokens and analyses the
tokens and their order to make sure they comply with the rules of the language grammar. If
an error is found in the query submitted by the user, it is rejected and an error code together
with an explanation of why the query was rejected is returned to the user.

The syntax analyzer takes the query from the users, parses it into tokens andanalyses the tokens
and their order to make sure they follow the rules of the language grammar.
Is an error is found in the query submitted by the user, it is rejected and an error code together
with an explanation of why the query was rejected is return to the user.
A simple form of the language grammar that could use to implement SQL statement is given
bellow :
QUERY = SELECT + FROM + WHERE
SELECT = SELECT + <CLOUMN LIST>
FROM = FROM + <TABLE LIST>
WHERE = WHERE + VALUE1 OP VALUE2
VALUE1 = VALUE / COLUMN NAME
VALUE2 = VALUE / COLUMN NAME
OP = >, <, >=, <=, =, <>



PREPARED BY ARUN PRATAP SINGH 11

11
QUERY DECOMPOSITION :





The aims of query decomposition
(1) To transform a high-level query into a relational algebra query.
(2) To check that the query is syntactically and semantically correct.

The typical stages of query decomposition are analysis, normalization, semantic
analysis, simplification, and query restructuring.

ANALYSIS :

PREPARED BY ARUN PRATAP SINGH 12

12





PREPARED BY ARUN PRATAP SINGH 13

13





PREPARED BY ARUN PRATAP SINGH 14

14


NORMALIZATION :




PREPARED BY ARUN PRATAP SINGH 15

15



PREPARED BY ARUN PRATAP SINGH 16

16





PREPARED BY ARUN PRATAP SINGH 17

17
SEMANTIC ANALYSIS :




PREPARED BY ARUN PRATAP SINGH 18

18


PREPARED BY ARUN PRATAP SINGH 19

19


QUERY SIMPLIFIER :

PREPARED BY ARUN PRATAP SINGH 20

20




PREPARED BY ARUN PRATAP SINGH 21

21



PREPARED BY ARUN PRATAP SINGH 22

22



QUERY RECONSTRUCTION :


PREPARED BY ARUN PRATAP SINGH 23

23




PREPARED BY ARUN PRATAP SINGH 24

24



PREPARED BY ARUN PRATAP SINGH 25

25





PREPARED BY ARUN PRATAP SINGH 26

26
QUERY OPTIMIZATION :



PREPARED BY ARUN PRATAP SINGH 27

27



PREPARED BY ARUN PRATAP SINGH 28

28



PREPARED BY ARUN PRATAP SINGH 29

29




PREPARED BY ARUN PRATAP SINGH 30

30



PREPARED BY ARUN PRATAP SINGH 31

31


PREPARED BY ARUN PRATAP SINGH 32

32



PREPARED BY ARUN PRATAP SINGH 33

33




PREPARED BY ARUN PRATAP SINGH 34

34



PREPARED BY ARUN PRATAP SINGH 35

35



PREPARED BY ARUN PRATAP SINGH 36

36



PREPARED BY ARUN PRATAP SINGH 37

37



PREPARED BY ARUN PRATAP SINGH 38

38


The main aim of query optimization is to choose the most efficient way of implementing the
relational algebra operations at the lowest possible cost. Therefore, the query optimizer
should not depend solely on heuristics rules, but, it should also estimate the cost of executing
the different strategies and find out the strategy with the minimum cost estimate. The method
of optimising the query by choosing a strategy those results in minimum cost is called cost-
based query optimization. The cost-based query optimization uses formulae that estimate the

PREPARED BY ARUN PRATAP SINGH 39

39
costs for a number of options and selects the one with lowest cost and most efficient to
execute. The cost functions used in query optimization are estimates and not exact cost
functions. So, the optimization may select a query execution strategy that is not the optimal
one.
The cost of an operation is heavily dependent on its selectivity, that is, the proportion of the
input relation(s) that forms the output. In general, different algorithms are suitable for low-
and high-selectivity queries. In order for a query optimiser to choose a suitable algorithm for
an operation an estimate of the cost of executing that algorithm must be provided. The cost
of an algorithm is dependent on the cardinality of its input. To estimate the cost of different
query execution strategies, the query tree is viewed as containing a series of basic operations
which are linked in order to perform the query. Each basic operation has an associated cost
function whose argument(s) are the cardinality of its input(s). It is also important to know
the expected cardinality of an operations output, since this forms the input to the next
operation in the tree. The expected cardinalities are derived from statistical estimates of a
querys selectivity, that is, the portion of the tuple satisfying the query.
The main aim of query optimization is to choose the most efficient way of implementing the
relational algebra operations at the lowest possible cost.
Therefore the query optimizer should not depend solely on heuristic rules, but, it should also
estimate the cost of executing the different strategies and find out the strategy with the minimum
cost estimate.
The method of optimizing the query by choosing a strategy those result in minimum cost is called
cost-based query optimization.
The cost-based query optimization uses the formula that estimate the cost for a number of options
and selects the one with lowest cost and the most efficient to execute.
The cost functions used in query optimization are estimates and not exact cost functions.
The cost of an operation is heavily dependent on its selectivity, that is, the proportion of select
operation(s) that forms the output.
In general the different algorithms are suitable for low or high selectivity queries. In order for query
optimizer to choose suitable algorithm for an operation an estimate of the cost of executing that
algorithm must be provided.
The cost of an algorithm is depend of a cardinality of its input.
To estimate the cost of different query execution strategies, the query tree is viewed as containing
a series of basic operations which are linked in order to perform the query.
It is also important to know the expected cardinality of an operations output because this forms
the input to the next operation.
Cost Components of Query Execution :-
The success of estimating the size and cost of intermediate relational algebra operations depends
on the amount the accuracy of statistical data information stored with DBMS.



PREPARED BY ARUN PRATAP SINGH 40

40



PREPARED BY ARUN PRATAP SINGH 41

41


STRUCTURE OF QUERY EVALUATION PLAN :



PREPARED BY ARUN PRATAP SINGH 42

42




PREPARED BY ARUN PRATAP SINGH 43

43


PIPELINING AND MATERIALIZATION :


PREPARED BY ARUN PRATAP SINGH 44

44






PREPARED BY ARUN PRATAP SINGH 45

45



PREPARED BY ARUN PRATAP SINGH 46

46



PREPARED BY ARUN PRATAP SINGH 47

47


PREPARED BY ARUN PRATAP SINGH 48

48



PREPARED BY ARUN PRATAP SINGH 49

49


SOME QUESTIONS
Q . 1 Explain inter-query parallelism ?
Ans : Inter-query parallelism is a form of parallelism in the evaluation of database queries, in
which several different queries execute concurrently on multiple processors to improve the
overall throughput of the system.
When multiple non-conflicting requests are submitted to a database management system, then
the system can execute them in parallel to improve the overall throughput. This form of parallelism is
called inter-query parallelism. Inter-query parallelism is a consequence of the concurrency of user
requests. It is orthogonal to intra-query parallelism, in which several processors cooperate for the faster
execution of a single query.
Inter-query parallelism results from the ability to execute multiple queries at the same time while intra-
query parallelism is achieved by breaking up a single query into a number of subqueries each of which is
executed at a different site, accessing a different part of the distributed database.
If the user access to the distributed database consisted only of querying (i.e.,
read-only access), then provision of inter-query and intra-query parallelism would
imply that as much of the database as possible should be replicated. However, since most database
accesses are not read-only, the mixing of read and update operations requires the implementation of
elaborate concurrency control and commit protocols.
o Queries/transactions execute in parallel with one another.
o Increases transaction throughput; used primarily to scale up a transaction processing system to
support a larger number of transactions per second.
o Easiest form of parallelism to support, particularly in a shared-memory parallel database, because
even sequential database systems support concurrent processing.

PREPARED BY ARUN PRATAP SINGH 50

50
o More complicated to implement on shared-disk or shared-nothing architectures
o Locking and logging must be coordinated by passing messages between processors.
o Data in a local buffer may have been updated at another processor.
o Cache-coherency has to be maintained reads and writes of data in buffer must find
latest version of data.
Cache Coherency Protocol
o Example of a cache coherency protocol for shared disk systems:
o Before reading/writing to a page, the page must be locked in shared/exclusive mode.
o On locking a page, the page must be read from disk
o Before unlocking a page, the page must be written to disk if it was modified.
o More complex protocols with fewer disk reads/writes exist.
o Cache coherency protocols for shared-nothing systems are similar. Each database page is assigned
a home processor. Requests to fetch the page or write it to disk are sent to the home processor.

Q . 2 Discuss cost estimation in query optimization.
Ans: Explain above.