Introduction to Database Systems

CSE 344

Lecture 6:
Basic Query Evaluation and Indexes

CSE 344 - Winter 2013 1

 Announcements
•  Webquiz due tonight
•  Homework 2 is posted, due next Wednesday

•  Today: query execution, indexes

•  Reading: 14.1
•  Monday: no classes
•  Wednesday: guest lecturer Paris Koutris

CSE 344 - Winter 2013 2

 Where We Are
•  We learned importance and benefits of DBMSs
•  We learned how to use a DBMS
–  How to specify what our data will look like: schema
–  How to load data into the DBMS
–  How to ask SQL queries
•  Today:
–  How the DBMS executes a query
–  How we can help it run faster

CSE 344 - Winter 2013 3

Query Evaluation Steps
SQL query

Parse & Check Query

Decide how best to

answer query: query
optimization

Query Execution

Return Results
4
 Example
Student Takes Course
ID   fName   lName   studentID   courseID   courseID   name  

195428   Tom   Hanks   195428   344   344   Databases  

645947   Amy   Hanks   .  .  .   .  .  .  
.  .  .  

SELECT  *  
FROM    Student  x,  Takes  y  
WHERE  x.ID=y.studentID  AND  y.courseID  >  300  

How can the DBMS

CSE answer
344 - Winter 2013 this query? 5
 SELECT  *  
FROM    Student  x,  Takes  y  
WHERE  x.ID=y.studentID  AND  y.courseID  >  300  

Possible Query Plan 1

Nested-­‐
for  y  in  Takes   loop  join  
       if  courseID  >  300  then  
           for  x  in  Student  
                         if  x.ID=y.studentID  
                             output  *  

CSE 344 - Winter 2013 6

 SELECT  *  
FROM    Student  x,  Takes  y  
WHERE  x.ID=y.studentID  AND  y.courseID  >  300  

Possible Query Plan 2

sort  Student  on  ID  

sort  Takes  on  studentID  (and  filter  on  coursesID  >  300)  
merge  join  Student,  Takes  on  ID  =  studentID  
for  (x,y)  in  merged_result  output  *  

Merge  join  

CSE 344 - Winter 2013 7

 SELECT  *  
FROM    Student  x,  Takes  y  
WHERE  x.ID=y.studentID  AND  y.courseID  >  300  

Possible Query Plan 3

Hash-­‐join  
create  a  hash-­‐table    
for  x  in  Student  
           insert  x  in  the  hash-­‐table  on  x.ID  
 
for  y  in  Takes  
           if  courseID  >  300    
                   then  probe  y.courseID  in  hash-­‐table  
                                     if  match  found    
                                     then  output  *  

CSE 344 - Winter 2013 8

 Discussion
Which plan is best? Choose one: for  y  in  Takes  
       if  courseID  >  300  then  
•  Nested loop join            for  y  in  Takes  
                         if  x.ID=y.studentID  
                             output  *  

sort  Student  on  ID  

sort  Takes  on  studentID  (and  filter  on  coursesID  >  300)  
•  Merge join merge  join  Student,  Takes  on  ID  =  studentID  
return  results  

create  a  hash-­‐table    
for  x  in  Student  
           insert  x  in  the  hash-­‐table  on  x.ID  
 
•  Hash join for  y  in  Takes  
           if  courseID  >  300    
                   then  probe  y.courseID  in  hash-­‐table  
                                     if  match  found  and  addi[onal  condi[ons  
                                     then  return  match  

9
 Discussion
Which plan is best? Choose one: for  y  in  Takes  
       if  courseID  >  300  then  
•  Nested loop join: O(N2)            for  y  in  Takes  
                         if  x.ID=y.studentID  
–  Could be O(N)                              output  *  

when few courses>300 sort  Student  on  ID  

sort  Takes  on  studentID  (and  filter  on  coursesID  >  300)  
•  Merge join: O(N log N) merge  join  Student,  Takes  on  ID  =  studentID  
return  results  
–  Could be O(N)
create  a  hash-­‐table    
if tables already sorted for  x  in  Student  
           insert  x  in  the  hash-­‐table  on  x.ID  

•  Hash join: O(N) expectation  

for  y  in  Takes  
           if  courseID  >  300    
                   then  probe  y.courseID  in  hash-­‐table  
                                     if  match  found  and  addi[onal  condi[ons  
                                     then  return  match  

10
 Student

ID   fName   lName  

Data Storage 10   Tom   Hanks  

20   Amy   Hanks  
…  
•  DBMSs store data in files
•  Most common organization is row-wise storage
•  On disk, a file is split into 10 Tom Hanks
block 1
20 Amy Hanks
blocks 50 … …
block 2
•  Each block contains 200 …

220
block 3
a set of tuples 240

420
800

In the example, we have 4 blocks with 2 tuples each

CSE 344 - Winter 2013 11
 Student

ID   fName   lName  

Data File Types 10   Tom   Hanks  

20   Amy   Hanks  
…  
The data file can be one of:
•  Heap file
–  Unsorted
•  Sequential file
–  Sorted according to some attribute(s) called key

Note: key here means something different from primary key:

it just means that we order the file according to that attribute.
In our example we ordered by ID. Might as well order by fName,
if that seems a better idea for the applications running on
our database. 12
 Index
•  An additional file, that allows fast access to
records in the data file given a search key
•  The index contains (key, value) pairs:
–  The key = an attribute value (e.g., student ID or name)
–  The value = a pointer to the record
•  Could have many indexes for one table

Key = means here search key

CSE 344 - Winter 2013 13

 This Is Not A Key
Different keys:
•  Primary key – uniquely identifies a tuple
•  Key of the sequential file – how the datafile is
sorted, if at all
•  Index key – how the index is organized
 Student

Example 1: ID   fName  

10   Tom  
lName  

Hanks  
Index on ID 20   Amy   Hanks  

Index Student_ID on Student.ID Data File Student …  

10 Tom Hanks
10
20 20 Amy Hanks
50
50 … …
200
200 …
220
240 220
420
240
800
420
950
… 800

CSE 344 - Winter 2013 15

 Student

Example 2: ID   fName  

10   Tom  
lName  

Hanks  
Index on fName 20   Amy   Hanks  
Index Student_fName
on Student.fName Data File Student …  

10 Tom Hanks
Amy
Ann 20 Amy Hanks
Bob
50 … …
Cho
200 …
…
… 220
…
240
…
420
…
… 800
Tom

CSE 344 - Winter 2013 16

 Index Organization
Several index organizations:
•  Hash table
•  B+ trees – most popular
–  They are search trees, but they are not binary
instead have higher fanout
–  will discuss them briefly next
•  Specialized indexes: bit maps, R-trees,
inverted index

CSE 344 - Winter 2013 17

 B+ Tree Index by Example
d=2 Find the key 40

80
40 ≤ 80

20 60 100 120 140

20 < 40 ≤ 60

10 15 18 20 30 40 50 60 65 80 85 90

30 < 40 ≤ 40

10 15 18 20 30 40 50 60 65 80 85 90
CSE 344 - Winter 2013 18
 Clustered vs Unclustered

B+ Tree B+ Tree
Data entries
Data entries
(Index File)
(Data file)

Data Records Data Records

CLUSTERED UNCLUSTERED
Every table can have only one clustered and many unclustered indexes

CSE 344 - Winter 2013 19

 Index Classification
•  Clustered/unclustered
–  Clustered = records close in index are close in data
•  Option 1: Data inside data file is sorted on disk
•  Option 2: Store data directly inside the index (no separate files)
–  Unclustered = records close in index may be far in data
•  Primary/secondary
–  Meaning 1:
•  Primary = is over attributes that include the primary key
•  Secondary = otherwise
–  Meaning 2: means the same as clustered/unclustered
•  Organization B+ tree or Hash table

CSE 344 - Winter 2013 20

 Scanning a Data File
•  Disks are mechanical devices!
–  Technology from the 60s; density much higher
•  We read only at the rotation speed!
•  Consequence:
Sequential scan is MUCH FASTER than
random reads
–  Good: read blocks 1,2,3,4,5,…
–  Bad: read blocks 2342, 11, 321,9, …
•  Rule of thumb:
–  Random reading 1-2% of the file ≈ sequential
scanning the entire file; this is decreasing over
time (because of increased density of disks)
21
 SELECT  *  
FROM    Student  x,  Takes  y  
WHERE  x.ID=y.studentID  AND  y.courseID  >  300  

Query Plan 1 Revisited

for  y  in  Takes  

       if  courseID  >  300  then   Assume the database has indexes on these attributes:
           for  x  in  Student   •  index_takes_courseID = index on Takes.courseID
                         if  x.ID=y.studentID  
•  index_student_ID = index on Student.ID
                             output  *  

Index  selec[on  

for  y  in  index_Takes_courseID  where  y.courseID  >  300  

     for  x  in  Takes  where  x.ID  =  y.studentID  
                       output  *  
Index  join  

CSE 344 - Winter 2013 22

 Getting Practical:
Creating Indexes in SQL
CREATE    TABLE        V(M  int,      N  varchar(20),        P  int);  

CREATE    INDEX  V1  ON  V(N)  

CREATE    INDEX  V2  ON  V(P,  M)  

CREATE    INDEX  V3  ON  V(M,  N)  

CREATE  UNIQUE  INDEX  V4  ON  V(N)  

Not  supported  in  
CREATE  CLUSTERED  INDEX  V5  ON  V(N)   SQLite  

CSE 344 - Winter 2013 23

 Student

ID   fName   lName  

Which Indexes? 10   Tom   Hanks  

20   Amy   Hanks  
…  

•  How many indexes could we create?

•  Which indexes should we create?

In general this is a very hard problem

 Student

ID   fName   lName  

Which Indexes? 10   Tom   Hanks  

20   Amy   Hanks  
…  
•  The index selection problem
–  Given a table, and a “workload” (big Java
application with lots of SQL queries), decide which
indexes to create (and which ones NOT to create!)
•  Who does index selection:
–  The database administrator DBA

–  Semi-automatically, using a database

administration tool

CSE 344 - Winter 2013 25

Index Selection: Which Search Key
•  Make some attribute K a search key if the
WHERE clause contains:
–  An exact match on K
–  A range predicate on K
–  A join on K

CSE 344 - Winter 2013 26

 The Index Selection Problem 1
V(M, N, P);

Your workload is this

100000 queries: 100 queries:
SELECT * SELECT *
FROM V FROM V
WHERE N=? WHERE P=?

What indexes ?
CSE 344 - Winter 2013 27
 The Index Selection Problem 1
V(M, N, P);

Your workload is this

100000 queries: 100 queries:
SELECT * SELECT *
FROM V FROM V
WHERE N=? WHERE P=?

A: V(N) and V(P) (hash tables or B-trees)

CSE 344 - Winter 2013 28
 The Index Selection Problem 2
V(M, N, P);

Your workload is this

100000 queries: 100 queries: 100000 queries:
SELECT * SELECT * INSERT INTO V
FROM V FROM V VALUES (?, ?, ?)
WHERE N>? and N<? WHERE P=?

What indexes ?
CSE 344 - Winter 2013 29
 The Index Selection Problem 2
V(M, N, P);

Your workload is this

100000 queries: 100 queries: 100000 queries:
SELECT * SELECT * INSERT INTO V
FROM V FROM V VALUES (?, ?, ?)
WHERE N>? and N<? WHERE P=?

A: definitely V(N) (must B-tree); unsure about V(P)

CSE 344 - Winter 2013 30
 The Index Selection Problem 3
V(M, N, P);

Your workload is this

100000 queries: 1000000 queries: 100000 queries:
SELECT * SELECT * INSERT INTO V
FROM V FROM V VALUES (?, ?, ?)
WHERE N=? WHERE N=? and P>?

What indexes ?
CSE 344 - Winter 2013 31
 The Index Selection Problem 3
V(M, N, P);

Your workload is this

100000 queries: 1000000 queries: 100000 queries:
SELECT * SELECT * INSERT INTO V
FROM V FROM V VALUES (?, ?, ?)
WHERE N=? WHERE N=? and P>?

A: V(N, P)
CSE 344 - Winter 2013 32
 The Index Selection Problem 4
V(M, N, P);

Your workload is this

1000 queries: 100000 queries:
SELECT * SELECT *
FROM V FROM V
WHERE N>? and N<? WHERE P>? and P<?

What indexes ?
CSE 344 - Winter 2013 33
 The Index Selection Problem 4
V(M, N, P);

Your workload is this

1000 queries: 100000 queries:
SELECT * SELECT *
FROM V FROM V
WHERE N>? and N<? WHERE P>? and P<?

A: V(N) secondary, V(P) primary index

CSE 344 - Winter 2013 34
Basic Index Selection Guidelines
•  Consider queries in workload in order of importance

•  Consider relations accessed by query

–  No point indexing other relations

•  Look at WHERE clause for possible search key

•  Try to choose indexes that speed-up multiple queries

•  And then consider the following…

CSE 344 - Winter 2013 35
 Index Selection:
Multi-attribute Keys
Consider creating a multi-attribute key on K1,
K2, … if
•  WHERE clause has matches on K1, K2, …
–  But also consider separate indexes
•  SELECT clause contains only K1, K2, ..
–  A covering index is one that can be used
exclusively to answer a query, e.g. index R(K1,K2)
covers the query:
SELECT K2 FROM R WHERE K1=55
CSE 344 - Winter 2013 36
 To Cluster or Not
•  Range queries benefit mostly from clustering
•  Covering indexes do not need to be
clustered: they work equally well unclustered

CSE 344 - Winter 2013 37

 SELECT *
FROM R
WHERE K>? and K<?

Cost Sequential scan

0 100
Percentage tuples retrieved
CSE 344 - Winter 2013 38