Indexing and Hashing

Most of the slides are used from the

Silberschatz, Korth Sudarshan, Database
System Concepts book presentations
 Basic concepts

 Indexing mechanisms used to speed up access to desired data.

 E.g., author catalog in library
 Search Key –attribute, a set of attributes used to look up records in a file.
 An index file consists of records (called index entries) of the form

search-key pointer

 Retrieval Key is not the same as a key (Primary key)

 Index files are typically much smaller than the original file
 Two basic kinds of indices:
 Ordered indices: search keys are stored in sorted order
 Hash indices: search keys are distributed uniformly across “buckets” using a
“hash function”.
 How DBMS access to data

 Table scan
 Read all the rows (stored on the secondary memory) one by one
 Select the rows that mach the query
 Index
 Read (search) the index file, to select the rows that mach the query
 Access only selected rows find in index
 SQL server
 Find if index exist, optimizer decide if it will use index for efficient
selection of the rows, or will scan the table
 Index Evaluation Metrics

 Access types supported efficiently. e.g.,

 records with a specified value in the attribute
 or records with an attribute value falling in a specified range of
values.
 Access time
 Insertion time
 Deletion time
 Space overhead
 Index Classification

 Clustered/unclustered
 Clustered = records close in the index are close in the data
 Unclustered = records close in the index may be far in the data
 Primary/secondary
 Sometimes means this:
 Primary = includes primary key
 Secondary = otherwise
 Sometimes means clustered/unclustered
 Dense/sparse
 Dense = every key in the data appears in the index
 Sparse = the index contains only some keys
 B+ tree / Hash table / …
 Clustered vs. Unclustered Index

Data entries
Data entries
(Index File)
(Data file)

Data Records Data Records

CLUSTERED UNCLUSTERED
 Clustered Index

 File is sorted on the index attribute

 Dense index: sequence of (key pointer) pairs

10 10

20 20

30

40 30

40
50

60
50
70
60
80

70

80
 Clustered Index

 Sparse index

10 10

30 20

50

70 30

40
90

110
50
130
60
150

70

80
 Unclustered Indexes

 File is not sorted on the index attribute

 Used to index other attributes than primary key
 Always dense

10
20

10 30

20

20 30

20
20

30
10
30
20
30

10

30
 Ordered Indices

Indexing techniques:

 In an ordered index, index entries are stored on the search key

value. e.g., author catalog in library.
 Primary index: in a sequentially ordered file, the index whose
search key specifies the sequential order of the file.
 Also called clustering index
 The search key of a primary index is usually but not necessarily the
primary key.
 Secondary index: an index whose search key specifies an order
different from the sequential order of the file. Also called
non-clustering index.
 Index-sequential file: ordered sequential file with a primary index.
Example account file
 Dense Index Files

 Dense index — index record appears for every search-key

value in the file.
 Sparse Index Files

 Sparse Index: contains index records for only some search-key

values.
 Applicable when records are sequentially ordered on search-key
 To locate a record with search-key value K we:
 Find index record with largest search-key value < K
 Search file sequentially starting at the record to which the index
record points
 Less space and less maintenance overhead for insertions and
deletions.
 Generally slower than dense index for locating records.
 Good tradeoff: sparse index with an index entry for every block
in file, corresponding to least search-key value in the block.
Example of Sparse Index Files
 Multilevel Index

 If primary index does not fit in memory, access becomes

expensive.
 To reduce number of disk accesses to index records, treat
primary index kept on disk as a sequential file and construct a
sparse index on it.
 outer index – a sparse index of primary index
 inner index – the primary index file
 If even outer index is too large to fit in main memory, yet
another level of index can be created, and so on.
 Indices at all levels must be updated on insertion or deletion
from the file.
.
Multilevel Index(1)
 Index Update: Deletion

 If deleted record was the only record in the file with its particular
search-key value, the search-key is deleted from the index also.
 Single-level index deletion:
 Dense indices – deletion of search-key is similar to file record
deletion.
 Sparse indices –
 if an entry for the search key exists in the index, it is deleted by replacing the entry
in the index with the next search-key value in the file (in search-key order).
 If the next search-key value already has an index entry, the entry is deleted
instead of being replaced.
 Index Update: Insertion

 Single-level index insertion:

 Perform a lookup using the search-key value appearing in the
record to be inserted.
 Dense indices – if the search-key value does not appear in the
index, insert it.
 Sparse indices – if index stores an entry for each block of the file,
no change needs to be made to the index unless a new block is
created. In this case, the first search-key value appearing in the
new block is inserted into the index.
 Multilevel insertion (as well as deletion) algorithms are simple
extensions of the single-level algorithms
 Secondary Indices

 Frequently, one wants to find all the records whose values in a

certain field (which is not the search-key of the primary index)
satisfy some condition.
 Example 1: In the account database stored sequentially by account
number, we may want to find all accounts in a particular branch
 Example 2: as above, but where we want to find all accounts with a
specified balance or range of balances
 We can have a secondary index with an index record for each
search-key value; index record points to a bucket that contains
pointers to all the actual records with that particular search-key
value..
Secondary Index on balance field of account

bucket
 Primary and Secondary Indices

 Secondary indices have to be dense.

 Indices offer substantial benefits when searching for records.
 When a file is modified, every index on the file must be updated,
Updating indices imposes overhead on database modification.
 Sequential scan using primary index is efficient, but a sequential
scan using a secondary index is expensive
 each record access may fetch a new block from disk
 Clustered index

 Most efficient for

 Queries of the type range (<,=<,>,=>)
 Sorted data (ORDER BY, or GROUP BY)
 Combination of the data (JOIN)
 Where result is large set of data
 Index is defined on small number of fields
 Unique or many different values (IDENTITY, UNIQUE)
 Is used for sorting the retrieved data from tables
 It is not good choice if
 Data are updated frequently
 Un-clustered index

 Most efficient for

 Queries that contain join or GROUP BY clauses. Multiple non clustered
index is created on columns that are involved in join operation or group
operation, as also clustered index for columns of secondary index.
 Tables that are rarely updated but have huge volume of data
 Queries that do not return large set of rows
 Indexed are columns that are often used for query
 Indexed are columns that contain a large number of various values (e.g
name-surname)
 Not recommended
 To big number of un-clustered index in transaction databases where
updates are often (than also update of index is needed)
 Choice and usage/not usage of index

 FROM/WHERE clause  recommended usage of index

 INSERT/UPDATE clause  is not recommended usage of
index
 Choice of attributes that will be indexes  attributes for which
we have intensive search
 Index definition in SQL

 Creation of index
create index <index-name> on <relation-name>
(<attribute-list>)
example: create index b-index on branch(branch-name)
 We use create unique index for indirect specification that
search key is primary key
 This is not needed if SQL support unique declaration
 To drop the index
drop index <index-name>
 Examples

 For definition of relation

Person (name, age, city)

SELECT
SELECT**
FROM
FROM Person
Person
WHERE
WHERE name
name==“Smith”
“Smith”

Here to obtain answer sequential research of the file is made

Index is created by

CREATE
CREATEINDEX
INDEX nameIndex
nameIndexON
ONPerson(name)
Person(name)
 Examples (1)

 For rang search

SELECT
SELECT**
FROM
FROMPerson
Person
WHERE
WHEREage
age>>25
25AND
ANDage
age<<28
28

 Index is useful for

CREATE
CREATEINDEX
INDEXageIndex
ageIndexON
ON Person
Person(age)
(age)
 We do not define index for all the attributes because maintaining of
the index is costly operation
 Multiple-Key Access

 Use multiple indices for certain types of queries.

 Example:
select account-number
from account
where branch-name = “Perryridge” and balance = 1000
 Possible strategies for processing query using indices on
single attributes:
1. Use index on branch-name to find accounts with name
“Perryridge”, test balances of $1000.
2. Use index on balance to find accounts with balances of $1000;
test branch-name = “Perryridge”.
3. Use branch-name index to find pointers to all records pertaining
to the Perryridge branch. Similarly use index on balance. Take
intersection of both sets of pointers obtained
 Examples (2)

Suppose we have an index on combined search-key

(branch-name, balance).

 With the where clause

where branch-name = “Perryridge” and balance = 1000
the index on the combined search-key will fetch only records that
satisfy both conditions.
Using separate indices in less efficient — we may fetch many records
(or pointers) that satisfy only one of the conditions.
 Can also efficiently handle
where branch-name - “Perryridge” and balance < 1000
 But cannot efficiently handle
where branch-name < “Perryridge” and balance = 1000
May fetch many records that satisfy the first but not the second
condition.
 Examples (3)

 Index created on several attributes

Index on (age, city) CREATE

CREATEINDEX
INDEXdoubleindex
doubleindexONON
Person
Person(age,
(age,city)
city)
SELECT
SELECT**
FROM
FROM Person
Person
Help for search of WHERE
WHEREage
age==55
55AND
ANDcity
city==“Seattle”
“Seattle”

SELECT
SELECT**
And in FROM
FROM Person
Person
WHERE
WHEREage
age==55
55
 B+-Tree index files

B+-tree indices are an alternative to indexed-sequential files

 Disadvantage of indexed-sequential files: performance
degrades as file grows, since many overflow blocks get
created. Periodic reorganization of entire file is required.
 Advantage of B+-tree index files: automatically reorganizes
itself with small, local, changes, in the face of insertions and
deletions. Reorganization of entire file is not required to
maintain performance.
 Disadvantage of B+-trees: extra insertion and deletion
overhead, space overhead.
 Advantages of B+-trees outweigh disadvantages, and they
are used extensively.
 B+ Trees index file (1)

B+-tree is a tree structure (hierarchically structure) with following

features with root, leafs:
 All the paths from root to leafs are of the same length
 Parameter d = the degree
 Each node has >= d and <= 2d keys (except root)
30 120 240

Keys k < 30
Keys 30<=k<120 Keys 120<=k<240 Keys 240<=k

 Each leaf has >=d and <= 2d keys:

40 50 60

Next leaf

40 50 60
 B+ Tree Example

d=2 Find key 40

80

40  80

20 60 100 120 140

20 < 40  60

10 15 18 20 30 40 50 60 65 80 85 90

30 < 40  40

10 15 18 20 30 40 50 60 65 80 85 90
 Insertion in a B+ Tree

Insert K=19
80

20 60 100 120 140

10 15 18 20 30 40 50 60 65 80 85 90

10 15 18 20 30 40 50 60 65 80 85 90
 Insertion in a B+ Tree

After insertion
80

20 60 100 120 140

10 15 18 19 20 30 40 50 60 65 80 85 90

10 15 18 19 20 30 40 50 60 65 80 85 90
 Insertion in a B+ Tree

Now insert 25
80

20 60 100 120 140

10 15 18 19 20 30 40 50 60 65 80 85 90

10 15 18 19 20 30 40 50 60 65 80 85 90
 Insertion in a B+ Tree

After insertion
80

20 60 100 120 140

10 15 18 19 20 25 30 40 50 60 65 80 85 90

10 15 18 19 20 25 30 40 50 60 65 80 85 90
 Insertion in a B+ Tree

But now have to split !

80

20 60 100 120 140

10 15 18 19 20 25 30 40 50 60 65 80 85 90

10 15 18 19 20 25 30 40 50 60 65 80 85 90
 Insertion in a B+ Tree

After the split

80

20 30 60 100 120 140

10 15 18 19 20 25 30 40 50 60 65 80 85 90

10 15 18 19 20 25 30 40 50 60 65 80 85 90
 Deletion from a B+ Tree

Delete 30
80

20 30 60 100 120 140

10 15 18 19 20 25 30 40 50 60 65 80 85 90

10 15 18 19 20 25 30 40 50 60 65 80 85 90
 Deletion from a B+ Tree

After deleting 30
80
May change to
40, or not

20 30 60 100 120 140

10 15 18 19 20 25 40 50 60 65 80 85 90

10 15 18 19 20 25 40 50 60 65 80 85 90
 Deletion from a B+ Tree

Now delete 25
80

20 30 60 100 120 140

10 15 18 19 20 25 40 50 60 65 80 85 90

10 15 18 19 20 25 40 50 60 65 80 85 90
 Deletion from a B+ Tree

After deleting 25
Need to rebalance
80
Rotate

20 30 60 100 120 140

10 15 18 19 20 40 50 60 65 80 85 90

10 15 18 19 20 40 50 60 65 80 85 90
 Deletion from a B+ Tree

Now delete 40
80

19 30 60 100 120 140

10 15 18 19 20 40 50 60 65 80 85 90

10 15 18 19 20 40 50 60 65 80 85 90
 Deletion from a B+ Tree

After deleting 40
Rotation not possible
Need to merge nodes
80

19 30 60 100 120 140

10 15 18 19 20 50 60 65 80 85 90

10 15 18 19 20 50 60 65 80 85 90
 Deletion from a B+ Tree

Final tree
80

19 60 100 120 140

10 15 18 19 20 50 60 65 80 85 90

10 15 18 19 20 50 60 65 80 85 90
 Observations about B+-trees

 Since the inter-node connections are done by pointers,

“logically” close blocks need not be “physically” close.
 The non-leaf levels of the B+-tree form a hierarchy of sparse
indices.
 The B+-tree contains a relatively small number of levels
(logarithmic in the size of the main file), thus searches can be
conducted efficiently.
 Insertions and deletions to the main file can be handled
efficiently, as the index can be restructured in logarithmic time.
 Summary on B+ Trees

 Default index structure on most DBMS

 Very effective at answering ‘point’ queries:
productName = ‘Toy’
 Effective for range queries:
50 < price AND price < 100
 Less effective for multirange:
50 < price < 100 AND 2 < quant < 20
 Recommendation for using Index

 Exact key values: Select

Selectname
name
 Scan index, lookup relation From
Frompeople
people
 B+ trees or hash tables
Where
Wheresalary
salary==25
25

Select
Selectname
name
 Range queries: From
Frompeople
people
 B+ trees Where
Where20 20<=
<=age
ageand
and age
age<=
<=30
30
 Use index exclusively Select
Selectdistinct
distinctage
age
From
Frompeople
people
 Example B+-tree

B+-tree for account file (d = 1)

 Example B+-tree (1)

B+-tree for account датотека (d = 2)

 Static Hashing

 A bucket is a unit of storage containing one or more records (a

bucket is typically a disk block).
 In a hash file organization we obtain the bucket of a record
directly from its search-key value using a hash function.
 Hash function h is a function from the set of all search-key values
K to the set of all bucket addresses n. h(K)  {0, 1, …, n-1}
 Hash function is used to locate records for access, insertion as
well as deletion.
 Records with different search-key values may be mapped to the
same bucket; thus entire bucket has to be searched sequentially
to locate a record.
 Example of Hash File Organization

Hash file organization of account file, using branch-name as key

(See figure in next slide.)

 There are 10 buckets,

 The binary representation of the ith character is assumed to be the
integer i.
 The hash function returns the sum of the binary representations of
the characters modulo 10
 E.g. h(Perryridge) = 5 h(Round Hill) = 3 h(Brighton) = 3
Example 1. Hash File Organization

Hash file organization of

account file, using
branch-name as key
 Example 2. hash organization

 Assume 1 bucket (block) stores 2 keys + pointers

 h(e)=0
 h(b)=h(f)=1
e
 h(g)=2 0
 h(a)=h(c)=3
b
1
f

g
2
a
3
c
 Searching in a Hash Table

 Search for a:
 Compute h(a)=3
 Read bucket 3
e
 1 disk access 0
b
1
f

g
2
a
3
c
 Insertion in Hash Table

 Place in right bucket, if space

 e.g. h(d)=2

e
0
b
1
f

g
2
d

a
3
c
 Hash Functions

 Worst has function maps all search-key values to the same

bucket; this makes access time proportional to the number of
search-key values in the file.
 An ideal hash function is uniform, i.e., each bucket is assigned
the same number of search-key values from the set of all
possible values.
 Ideal hash function is random, so each bucket will have the
same number of records assigned to it irrespective of the actual
distribution of search-key values in the file.
 Typical hash functions perform computation on the internal
binary representation of the search-key.
 For example, for a string search-key, the binary representations of
all the characters in the string could be added and the sum modulo
the number of buckets could be returned.
 Inserting in hash tables with overflow

 Overflow bucket, if in existing buckets there is no place

 example h(k)=1

e
0
b k
1
f
 It is possible to g
chain several over- 2
d
flow buckets
a
3
c
 Hash Indices

 Hashing can be used not only for file organization, but also for
index-structure creation.
 A hash index organizes the search keys, with their associated
record pointers, into a hash file structure.
 Strictly speaking, hash indices are always secondary indices
 if the file itself is organized using hashing, a separate primary hash
index on it using the same search-key is unnecessary.
 However, we use the term hash index to refer to both secondary
index structures and hash organized files.
Example of Hash Index
 Deficiencies of Static Hashing

 In static hashing, function h maps search-key values to a fixed

set of B of bucket addresses.
 Databases grow with time. If initial number of buckets is too small,
performance will degrade due to too much overflows.
 If file size at some point in the future is anticipated and number of
buckets allocated accordingly, significant amount of space will be
wasted initially.
 If database shrinks, again space will be wasted.
 One option is periodic re-organization of the file with a new hash
function, but it is very expensive.
 These problems can be avoided by using techniques that allow
the number of buckets to be modified dynamically.
 Example definition index

CREATE TABLE student (

sid INT, PRIMARY KEY(sid)
title VARCHAR(100), automatic definition of
name VARCHAR(50), index
surname VARCHAR(50),
PRIMARY KEY(sid)
);
 Example definition index (1)

CREATE TABLE student (

sid INT, Explicit definition of
title VARCHAR(100), index
name VARCHAR(50),
surname VARCHAR(50),
);

CREATE UNIQUE INDEX student_sid ON student (sid); or

CREATE UNIQUE INDEX student_id ON student USING BTREE

(sid);

Alternate BTREE, HASH...

 Comparison of Ordered Indexing and Hashing

 Cost of periodic re-organization

 Relative frequency of insertions and deletions
 Is it desirable to optimize average access time at the expense of
worst-case access time?
 Expected type of queries:
 Hashing is generally better at retrieving records having a specified
value of the key.
 If range queries are common, ordered indices are to be preferred
 Bitmap Indices

 Bitmap indices are a special type of index designed for efficient

querying on multiple keys
 Records in a relation are assumed to be numbered sequentially
from, say, 0
 Given a number n it must be easy to retrieve record n
 Applicable on attributes that take on a relatively small number of
distinct values
 E.g. gender, country, state, …
 E.g. income-level (income broken up into a small number of levels
such as 0-9999, 10000-19999, 20000-50000, 50000- infinity)
 A bitmap is simply an array of bits
 Bitmap Indices (1)

 In its simplest form a bitmap index on an attribute has a bitmap

for each value of the attribute
 Bitmap has as many bits as records
 In a bitmap for value v, the bit for a record is 1 if the record has the
value v for the attribute, and is 0 otherwise
 Bitmap Indices (1)

 Bitmap indices are useful for queries on multiple attributes

 not particularly useful for single attribute queries
 Queries are answered using bitmap operations
 Intersection (and)
 Union (or)
 Complementation (not)
 Each operation takes two bitmaps of the same size and applies
the operation on corresponding bits to get the result bitmap
 E.g. 100110 AND 110011 = 100010
100110 OR 110011 = 110111
NOT 100110 = 011001
 Males with income level L1: 10010 AND 10100 = 10000
 Can then retrieve required tuples.
 Counting number of matching tuples is even faster
 Bitmap Indices (2)

 Bitmap indices generally very small compared with relation size

 E.g. if record is 100 bytes, space for a single bitmap is 1/800 of space
used by relation.
 If number of distinct attribute values is 8, bitmap is only 1% of relation size
 Deletion needs to be handled properly
 Existence bitmap to note if there is a valid record at a record location
 Needed for complementation
 not(A=v): (NOT bitmap-A-v) AND ExistenceBitmap)
 Should keep bitmaps for all values, even null value
 To correctly handle SQL null semantics for NOT(A=v):
 intersect above result with (NOT bitmap-A-Null)
 Efficient Implementation of Bitmap Operations

 Bitmaps are packed into words; a single word (a basic CPU

instruction) computes of 32 or 64 bits at once
 E.g. 1-million-bit maps can be handed with just 31,250 instruction
 Bitmaps can be used instead of Tuple-ID lists at leaf levels of
B+-trees, for values that have a large number of matching
records
 Worthwhile if > 1/64 of the records have that value, assuming a
tuple-id is 64 bits
 Above technique merges benefits of bitmap and B+-tree indices