SQL Query Optimization
and Indexing
Along the way
How is a db query executed
Schema optimization
Execution Plan
Indexing (Types of Indices)
Using indices
Lock Contention
Covering Indices
The DB Engine
How does a
database server
run a query ?
Server process
SQL Query
analysis
If new query?
Execution plan?
(Optimizer )
If cpu?
Table scan ?
index?
Require high
performance ??
Good optimized schema
Give indexes for specific
queries
Tradeoffs !!
Whats an Index??
adata structure
Retrieval
Inserts
Also
Denormalized db
=
faster 4 some
queries
+
slower for others
Choosing optimal Data Types
Smaller is better
Use less space
Require fewer CPU cycles
Simple is good
Integers easier to compare than characters
E.g.: Use MySQL built in types for date/time
Avoid NULL if possible
Harder for MySQL to optimize queries referring to
nullable columns
DATETIME and TIMESTAMP store same kind of data, but ??
String Types
VARCHAR and CHAR types
Their storage on disk is storage engine dependent
Usually the storage is different for disk, memory and
after retrieval from the storage engine
VARCHAR
Uses as much space as it needs
Uses 1 or 2 bytes extra for storing the length
1 byte if length up to 255 bytes, 2 for above 255 length
So VARCHAR(10) uses 11 bytes and VARCHAR(1000)
uses 1002 bytes
Improves performance as it saves space
Variableupdat
rows can
more
-
-
length
e
grow
work!!!
-
Use VARCHAR , max col length > avg length, updates are
rare
CHAR
Fixed length
For data changing frequently, char better than varchar
For very short columns, CHAR(1) = 1 BYTE and
VARCHAR(1) = 2 BYTES
The siblings of char and varchar are binary and
varbinary data types
Good for comparing as bytes that characters.
Comparing random strings
Strings produced by MD5(), SHA1() OR UUID().
Each new string generated will be distributed in arbitrary
ways over a large space
Can slow INSERT coz get inserted in a random loc in
indexes
They slow some SELECT queries as logically adjacent
rows will be widely dispersed in disk and memory
If you do store UUID values, you
should remove the dashes or, even
better, convert the UUID values to
16-byte numbers with UNHEX() and
store them in a BINARY(16) column.
You can retrieve the values in
hexadecimal format with the HEX()
function
IP Address
Usual case, use VARCHAR(15)
But, IP is really an unsigned 32 bit integer , not a string
Dotted-quad notation for humans to understand easily
MySQL provide INET_ATON() and INET_NTOA() fns to
convert btw 2 representations
The Execution Plan
Every SQL query is broken down in to series of
execution steps called as operators
Each operator performs basic operations like
insertion, search, scan, updation, aggregation etc.
There are 2 kinds of operators Logical operators
and physical operators.
Logical operators : describe how the execution
will be executed at a conceptual level
Physical operators : The actual logic / routine
which perform the action.
Checks
syntax
Query
process
or tree
is output
of parse
PARSE
OPTIMIZE
Calculate
cost and
gives out
estimated
plan and
an actual
plan
DATA STATISTICS
1. How many rows?
2. Unique data?
3. Does table span
over more than
one page?
EXECUTE
As per
plan
executio
n is
done
Into Indexing
TYPES
B-Tree Indexes
Hash Indexes
B-Tree
We use the term "B-Tree" for these indexes because
that's what MySQL uses in CREATE TABLE and other
statements
All the values are stored in order, and each leaf page is
the same distance from the root
Leaf nodes have pointers to the indexed data instead
of pointers to other pages
Because B-Trees store the indexed columns in order,
they're useful for searching for ranges of data
Hash Indexes
Built on hash tables and useful for exact lookups that
use every column in the index
Memory storage engine only supports this in MySQL
Forms hash codes of the indexed columns and stores a
pointer to each row in hash table
E.g. :
CREATE TABLE testhash (
fname VARCHAR(50) NOT NULL,
lname VARCHAR(50) NOT NULL,
KEY USING HASH(fname)
) ENGINE=MEMORY;
containing this data:
mysql> SELECT * FROM testhash
Fname
Darshan
Bijesh
Jophin
Vivek
lname
Raj
Chandran
Joseph
Babu
Suppose the index use a fn f(), which return following
values
f(Darshan) = 2323
f(Bijesh') = 7437
f(Jophin') = 8784
f('Vivek') = 2458
The index's data structure will look like this:
Slot
Value
2323
Pointer to row 1
2458
Pointer to row 4
7437
Pointer to row 2
8784
Pointer to row 3
A hash index on a TINYINT will be the same size as a hash
index on a large character, coz ???
the indexes store only the short hash values.
Non - Clustered Indexes
Data present in random order
Logical ordering specified by index
Typically created on column used in JOIN, WHERE and ORDER BY
Good for tables whose values may be modified frequently
Clustered Indexes
Data blocks arranged in order to match the index
Only one clustered index possible on a given table
Faster retrieval if data accessed in asc or desc order
MS SQL Server creates non-clustered
indices by default when CREATE INDEX is
given.
Using indices
Indexing the primary key
Usually automatically indexed to facilitate effective
information retrieval
Most effective access path
Other columns or combination of columns = secondary
index to improve performance in data retrieval
Secondary indexes
Indexes on other columns other than primary key
column
Create secondary indexes on tables that have more
reads than writes
Just copy of the db table but containing only the fields
specified in the index
Dont give more than 4 fields in an
index and more than 5 indexes for a
table. You are inviting trouble
otherwise !!
Index Column Order does matter !!
Not useful if lookup does not start from the leftmost side
of the indexed columns.
Cant skip columns in the index.
Join vs. Sub query
Join faster when we have less number of tables
Join faster when we have less data in tables
Sub query faster when there are large number of tables
as joining more tables is tedious
Sub query faster when we have huge data in tables
Explaining the explain ??!
A way to obtain information about how MySQL executes a
SELECT statement
Syntax : Explain SELECT select_options
Returns a row of information for each table used in the
SELECT statement
These are the info that MySQL gives for each table
id
Selec
t_typ
e
Tabl
e
Type Possibl
e_
Keys
Key
Key_
Length
Ref
Row
s
Id : select identifier
Select_type : type of select (Simple, Primary , Union ,
Dependent Union, Subquery, Dependent Subquery etc)
extr
a
Table : table to which row output refers
type : The join type (important)
possible keys : The possible indexes that can be used for
the query
keys : The indexes used in the query
rows : no: of rows scanned
Lock Contention??
1) DELETE FROM user WHERE
status = 9
Fully scan user
table, deleting if
status = 9;
User_id
(PK)
Name
status
100
What happens if query
1 does not lock row:
user_id = 100 ?
100000
2) UPDATE user SET status=9 WHERE
user_id =100
DATA CONSISTENCY IS
BROKEN !!
If STATUS column is
indexed
1) DELETE FROM user WHERE
status = 9
Status
PK
100
101
12345
100000
1) And 2) can run in
parallel
(CONCURRENCY
IMPROVED)
User_id
(PK)
Name
status
Roger
100
Rafael
01
100000
Andy
2) UPDATE user SET status=9 WHERE
user_id =100
Covering Index??
DB Engine ??
The underlying software component
that a database management
system(DBMS) uses to create , read ,
update , delete (CRUD) data from a
database
MySQL has InnoDB and MyISAM
InnoDB = transactional
MyISAM = non-transactional
InnoDB create a Clustered Index for
every table. If it has a primary key,
that is the clustered index. If not, it
created a six-byte unique ID and
makes it the clustered index.
All Indexes are B-Trees. The Primary
keys leaf nodes are the data.
References
High Performance MySQL Steven Feuerstein
Mastering the art of Indexing - Yoshinori Matsunobu
http://www.codeproject.com
http://www.databasejournal.com
SQL Best Practices Video Journal by Steven Feuerstein
MySQL 5.0 Reference manual
THANK YOU
