Presentation Outline
SQL Writing Process
SQL Standards
Using Indexes The Optimizer FROM, WHERE Clauses EXPLAIN SQL Trace Sub-Selects and Joins Tips and Tricks
1
SQL Writing Process
Step 1: What information do I need? Columns Step 2: Where is it? Tables Step 3: Write SQL:
SELECT columns FROM tables WHERE ... (joins, filters, subqueries)
I'M FINISHED!
SQL Writing Process
YOU'RE NOT FINISHED YET! You've got the results you want, but at what cost? There are many, many ways to get the right results, but only one is the fastest way1000-to-1 improvements are attainable! Inefficient SQL can dramatically degrade the performance of the entire system
Developers and DBAs must work together to tune the database and the application
SQL Standards
Why are SQL standards important?
Maintainability, readability
Performance: If SQL is the same as a (recently) executed statement, it can be re-used instead of needing to be reparsed
SQL Standards
Question: which of these statements are the same?
A. SELECT LNAME FROM EMP WHERE EMPNO = 12;
B. SELECT lname FROM emp WHERE empno = 12; C. SELECT lname FROM emp WHERE empno = :id; D. SELECT lname FROM
WHERE empno = 12; emp
SQL Standards
Answer: None
Whitespace, case, bind variables vs. constants all matter
Using standards helps to ensure that equivalent SQL can be reused.
Tables Used in the Examples
DEPT deptno dname loc
EMP empno mgr job deptno fname lname comm hiredate grade sal
7
SALGRADE grade losal hisal
SQL Standards: Example
SELECT E.empno, D.dname FROM emp E, dept D WHERE AND OR E.deptno = D.deptno (D.deptno = :vardept E.empno = :varemp); Keywords upper case and left-aligned Columns on new lines Use std. table aliases
Separate w/ one space
Use bind variables AND/OR on new lines
No space before/after parentheses
Indexes: What are they?
An index is a database object used to speed retrieval of rows in a table. The index contains only the indexed value--usually the key(s)--and a pointer to the row in the table. Multiple indexes may be created for a table Not all indexes contain unique values Indexes may have multiple columns
Indexes and SQL
If a column appears in a WHERE clause it is a candidate for being indexed. If a column is indexed the database can used the index to find the rows instead of scanning the table. If the column is not referenced properly, however, the database may not be able to used the index and will have to scan the table anyway. Knowing what columns are and are not indexed can help you write more efficient SQL
10
Example: Query without Index
No index exists for column EMPNO on table EMP, so a table scan must be performed:
Table: EMP SELECT * FROM emp WHERE empno = 8
empno 4 9 1 3 5 2 7 8 6 fname lisa jackie john larry jim mary harold mark gene lname... baker miller larson jones clark smith simmons burns harris
11
Example: Query with Index
Column EMPNO is indexed, so it can be used to find the requested row:
SELECT * FROM emp WHERE empno = 8 Index: PK_EMP EMP (EMPNO)
1, 4
Table: EMP
5 5, 9 empno 4 9 1 3 5 2 7 8 6 fname lisa jackie john larry jim mary harold mark gene lname ... baker miller larson jones clark smith simmons burns harris
1 2
7 8 9
12
Indexes: Caveats
Sometimes a table scan cannot be avoided Not every column should be indexed--there is performance overhead on Inserts, Updates, Deletes Small tables may be faster with a table scan Queries returning a large number (> 5-20%) of the rows in the table may be faster with a table scan
13
Indexes: Column Order
Example: Index on (EMPNO, DEPTNO)
SELECT * FROM emp WHERE deptno = 10;
SELECT FROM WHERE AND
* emp empno > 0 deptno = 10;
Question: which of these statements will use the index?
14
Indexes: Column Order
First statement will not use index. Second statement will use index.
Must use the leading column of the index.
15
Indexes: Functions
Question: which of these statements will use the index?
SELECT FROM WHERE ... WHERE * emp TRUNC(hiredate) = TRUNC(SYSDATE); fname || lname = 'MARYSMITH';
SELECT * FROM emp WHERE hiredate BETWEEN TRUNC(SYSDATE) AND TRUNC(SYSDATE)+1 ... WHERE fname = 'MARY' AND lname = 'SMITH';
16
Indexes: Functions
First statement will not use index. Second statement will use index. Using a function, calculation, or other operation on an indexed column disables the use of the Index.
17
Indexes: NOT
Question: which of these statements will use the index?
SELECT FROM WHERE ... ... * dept deptno != 0; deptno NOT = 0; deptno IS NOT NULL;
SELECT * FROM dept WHERE deptno > 0;
18
Indexes: NOT
First statement will not use index. Second statement will use index. Using NOT excludes indexed columns.
19
The Optimizer
The WHERE/FROM rules on the following pages apply to the Rule-based optimizer (Oracle). If the Cost-based Optimizer is used, Oracle will attempt to reorder the statements as efficiently as possible (assuming statistics are available). DB2 and Sybase use only a Cost-based optimizer The Optimizer's access paths can be overridden in Oracle and Sybase (not DB2)
20
The Optimizer: Hints
Return the first rows in the result set as fast as possible:
SELECT /*+ FIRST_ROWS */ empno FROM emp E dept D, WHERE E.deptno = D.deptno;
Force Optimizer to use index IDX_HIREDATE:
SELECT /*+ INDEX (E idx_hiredate) */ empno FROM emp E WHERE E.hiredate > TO_DATE('01-JAN-2000');
21
FROM Clause: Driving Table
Specify the driving table last in the FROM Clause:
SELECT * FROM dept D, -- 10 rows emp E -- 1,000 rows WHERE E.deptno = D.deptno;
SELECT * FROM emp E, -- 1,000 rows dept D -- 10 rows WHERE E.deptno = D.deptno;
Driving table is EMP
Driving table is DEPT
22
FROM Clause: Intersection Table
When joining 3 or more tables, use the Intersection table (with the most shared columns) as the driving table:
SELECT * FROM dept D, salgrade S, emp E WHERE E.deptno = D.deptno AND E.grade = S.grade;
EMP shares columns with DEPT and SALGRADE, so use as the driving table
23
WHERE: Discard Early
Use WHERE clauses first which discard the maximum number of rows:
SELECT FROM WHERE AND * emp E E.empno IN (101, 102, 103) E.deptno > 10;
3 rows 90,000 rows
24
WHERE: USING AND
Question: which of these statements will more effective?
SELECT FROM WHERE AND * emp E E.sal > 50000 25 > (SELECT COUNT(*) FROM emp M WHERE M.mgr = E.empno)
SELECT * FROM emp E WHERE 25 > (SELECT COUNT(*) FROM emp M WHERE M.mgr = E.empno) AND E.sal > 50000
25
WHERE: USING AND
When using an "AND" sub query, place it first. It will more cost effective.
26
WHERE: USING OR
Question: which of these statements will more effective?
SELECT * FROM emp E WHERE 25 > (SELECT COUNT(*) FROM emp M WHERE M.mgr = E.empno) OR E.sal > 50000 SELECT FROM WHERE OR * emp E E.sal > 50000 25 > (SELECT COUNT(*) FROM emp M WHERE M.mgr = E.empno)
27
WHERE: USING OR
Using an "OR" sub query, place it last. It will more cost effective.
28
WHERE: Filter First, Join Last
When Joining and Filtering, specify the Filter condition first, Joins last.
SELECT * FROM emp E, dept D WHERE (E.empno = 123 OR D.deptno > 10) AND E.deptno = D.deptno;
Filter criteria Join criteria
29
Subqueries: IN vs. EXISTS
Question: which of these statements will more effective?
SELECT E.* FROM emp E WHERE E.deptno IN ( SELECT D.deptno FROM dept D WHERE D.dname = 'SALES'); SELECT * FROM emp E WHERE EXISTS SELECT FROM WHERE AND
( 'X' dept D D.deptno = E.deptno D.dname = 'SALES');
30
Subqueries: IN vs. EXISTS
IN : Both tables are scanned. EXITS : Only outer table is scanned;Sub query used the index.
Use EXISTS instead of IN in sub queries.
31
Subquery vs. Join
Question: which of these statements will more effective?
SELECT * FROM emp E WHERE E.deptno IN ( SELECT D.deptno FROM dept D WHERE D.dname = 'SALES'); SELECT E.* FROM emp E, dept D WHERE D.dname = 'SALES' AND D.deptno = E.deptno; 32
Subquery vs. Join
IN : Both tables are scanned. JOIN : Only one table is scanned,other use index.
Use Join instead of Sub query.
33
Join vs. EXISTS
Best performance depends on subquery/driving table:
SELECT * FROM emp E WHERE EXISTS SELECT FROM WHERE AND EXISTS: better than Join if the number of matching rows in DEPT is small
( 'X' dept D D.deptno = E.deptno D.dname = 'SALES');
SELECT E.* FROM emp E, dept D WHERE D.dname = 'SALES' AND D.deptno = E.deptno;
34
JOIN: better than Exists if the number of matching rows in DEPT is large
Explain
Display the access path the database will use (e.g., use of indexes, sorts, joins, table scans) Oracle: Sybase: DB2: EXPLAIN SHOWPLAN EXPLAIN
Oracle Syntax:
EXPLAIN PLAN SET STATEMENT_ID = 'statement id' INTO PLAN_TABLE FOR
statement
Requires Select/Insert privileges on PLAN_TABLE
35
Explain
Example 1: IN subquery
SELECT * FROM emp E WHERE E.deptno IN ( SELECT D.deptno FROM dept D WHERE D.dname = 'SALES');
Result:
3 joins MERGE JOIN 1 dynamic view SORT (JOIN) 2 table scans TABLE ACCESS (FULL) OF EMP 3 sorts SORT (JOIN) VIEW SORT (UNIQUE) TABLE ACCESS (FULL) OF DEPT
36
Explain
Example 2: "EXISTS" subquery
SELECT * FROM emp e WHERE EXISTS SELECT FROM WHERE AND
( 'x' dept d d.deptno = e.deptno d.dname = 'SALES');
1 table scan 1 index scan 1 index access
Result:
FILTER TABLE ACCESS (FULL) OF EMP TABLE ACCESS (BY INDEX ROWID) OF DEPT INDEX (UNIQUE SCAN) OF PK_DEPT (UNIQUE)
37
Explain
Example 3: Join (no subquery)
SELECT E.* FROM emp E, dept D WHERE D.dname = 'SALES' AND D.deptno = E.deptno;
Result:
1 table scan 1 index scan 1 index access
NESTED LOOPS TABLE ACCESS (FULL) OF EMP TABLE ACCESS (BY INDEX ROWID) OF DEPT INDEX (UNIQUE SCAN) OF PK_DEPT (UNIQUE)
38
SQL Trace
Use SQL Trace to determine the actual time and resource costs for for a statement to execute. Step 1: ALTER SESSION SET SQL_TRACE TRUE; Step 2: Execute SQL to be traced:
SELECT E.* FROM emp E, dept D WHERE D.dname = 'SALES' AND D.deptno = E.deptno;
Step 3: ALTER SESSION SET SQL_TRACE FALSE;
39
SQL Trace
Step 4: Trace file is created in <USER_DUMP_DEST> directory on the server (specified by the DBA).
Step 5: Run TKPROF (UNIX) to create a formatted output file:
tkprof echd_ora_15319.trc $HOME/prof.out table=plan_table explain=dbuser/passwd
Trace file Formatted output file destination for Explain user/passwd for Explain
40
SQL Trace
Step 6: view the output file:
... SELECT E.* FROM emp E, dept D WHERE D.dname = 'SALES' AND D.deptno = E.deptno; call count ------- -----Parse 1 Execute 1 Fetch 2 ------- -----total 4
TIMED_STATISTICS must be turned on to get these values
rows ---------0 0 6 ---------6
cpu elapsed disk query current -------- ---------- ---------- ---------- ---------0.00 0.00 0 0 0 0.00 0.00 0 0 0 0.00 0.00 4 19 3 -------- ---------- ---------- ---------- ---------0.00 0.00 4 19 3
Misses in library cache during parse: 0 Optimizer goal: CHOOSE Parsing user id: 62 (PMARKS) Rows ------6 14 14 14 Row Source Operation --------------------------------------------------NESTED LOOPS TABLE ACCESS FULL EMP TABLE ACCESS BY INDEX ROWID DEPT INDEX UNIQUE SCAN (object id 4628)
EXPLAIN output
41
Tips and Tricks: UNION ALL
Use UNION ALL instead of UNION if there are no duplicate rows (or if you don't mind duplicates):
SELECT * FROM emp UNION SELECT * FROM emp_arch; SELECT * FROM emp UNION ALL SELECT * FROM emp_arch;
UNION: requires sort
UNION ALL: no sort
42
Tips and Tricks: HAVING vs. WHERE
With GROUP BY, use WHERE instead of HAVING (if the filter criteria does not apply to a group function):
SELECT deptno, AVG(sal) FROM emp GROUP BY deptno HAVING deptno IN (10, 20); SELECT deptno, AVG(sal) FROM emp WHERE deptno IN (10, 20) GROUP BY deptno;
43
HAVING: rows are filtered after result set is returned
WHERE: rows are filtered first--possibly far fewer to process
Tips and Tricks: EXISTS vs DISTINCT
Use EXISTS instead of DISTINCT to avoid implicit sort (if the column is indexed):
SELECT DISTINCT e.deptno, e.lname FROM dept d, emp e WHERE d.deptno = e.deptno;
DISTINCT: implicit sort is performed to filter duplicate rows
SELECT e.deptno, e.lname EXISTS: no sort FROM emp e WHERE EXISTS ( SELECT 'X' FROM dept d WHERE d.deptno = e.deptno);
44
Tips and Tricks: Consolidate SQL
Select from Sequences and use SYSDATE in the statement in which they are used:
SELECT SYSDATE INTO :vardate FROM dual; FROM dual;
SELECT arch_seq.NEXTVAL INTO :varid
INSERT INTO archive BEFORE: 3 statements VALUES (:vardate, :varid, ...) are used to perform 1 Insert
INSERT INTO emp_archive VALUES (SYSDATE, emp_seq.NEXTVAL, ...) AFTER: only 1 statement is needed
45
Tips and Tricks: Consolidate SQL
Consolidate unrelated statements using outer-joins to the the DUAL (dummy) table:
SELECT dname FROM dept WHERE deptno = 10; SELECT lname FROM emp WHERE empno = 7369;
SELECT d.dname, e.lname FROM dept d, emp e, dual x WHERE d.deptno AND e.empno AND NVL('X', AND NVL('X',
BEFORE: 2 round-trips
AFTER: only 1 round-trip
(+) = 10 (+) = 7369 x.dummy) = NVL('X', e.ROWID (+)) x.dummy) = NVL('X', d.ROWID (+));
46
Tips and Tricks: COUNT
Use COUNT(*) instead of COUNT(column):
SELECT COUNT(empno) FROM emp;
SELECT COUNT(*) FROM emp;
~ 50% faster
47
Tips and Tricks: Self-Join
Use a self-join (joining a table to itself) instead of two queries on the same table:
SELECT mgr INTO :varmgr FROM emp WHERE deptno = 10; LOOP... SELECT mgr, lname FROM emp WHERE mgr = :varmgr; SELECT E.mgr, E.lname FROM emp E, emp M WHERE M.deptno = 10 AND E.empno = M.mgr;
BEFORE: 2 round-trips
AFTER: only 1
48
Tips and Tricks: ROWNUM
Use the ROWNUM pseudo-column to return only the first N rows of a result set. (For example, if you just want a sampling of data):
SELECT * FROM emp WHERE ROWNUM <= 10; Returns only the first 10 employees in the table, in no particular order
49
Tips and Tricks: ROWID
The ROWID pseudo-column uniquely identifies a row, and is the fastest way to access a row:
CURSOR retired_emp_cur IS Instead of selecting the SELECT ROWID key column(s), ROWID is FROM emp used to identify the row WHERE retired = 'Y'; for later use ... FOR retired_emp_rec IN retired_emp_cur LOOP SELECT fname || ' ' || lname INTO :printable_name FROM emp WHERE ROWID = retired_emp_rec.ROWID; ...
50
Tips and Tricks: Sequences
Use a Sequence to generate unique values for a table:
SELECT INTO FROM ... INSERT VALUES MAX(empno) :new_empno emp; INTO emp (:new_empno + 1, ...); MAX(empno) requires a sort and an index scan INSERT could fail with a Duplicate error if someone else gets there first
Using a Sequence INSERT INTO emp VALUES (emp_seq.NEXTVAL, ...); ensures that you always have a unique number, or and does not require any SELECT emp_seq.NEXVAL table reads INTO :new_empno FROM dual;
51
THANK YOU
52
Tips and Tricks: Cartesian Products
Avoid Cartesian products by ensuring that the tables are joined on all shared keys:
SELECT FROM * dept, -- 10 rows salgrade, -- 20 rows emp; -- 1,000 rows
10 * 1000 * 20 = 200,000 rows
SELECT FROM * dept, -- 10 rows salgrade, -- 20 rows emp -- 1,000 rows E.deptno = D.deptno E.grade = S.grade;
53
WHERE AND
1,000 rows
