SQL Tuning & Performance Masterclass (2020) . Oa Course Slide for Students This documents prepared for our students to make a quick recap easily about what they have learnt: throughout the course. te SQL TUNING LLL ncWhat is SQL(Performance) Tuning and Why We Need? S ‘ & Whatis SQL Tuning? % Oracleis a Very Strong Database @ Better Performance - Less Hardware Cost @ SQL Tuning is the combination of some techniques | @ SQL Tuningis not so hard @ Who Will Do the Tuning? SQL TUNING ponaman ppeemeceseenWhat is SQL(Performance) Tuning and Why We Need? S s @ No Need to Scare SQL Tuning @ You Need to Know Database Architecture in Basic @ There will be many introductive lectures at first @ SQL Tuning Needs Expertise @ Some Tools will Not Be Explained l [ @ All The Subjects are Organized Strategically & Tosumup! SQL TUNING Coe ee eee oyOracle Database Architecture ? MIC een ee es 2° Architecture Diagram SQL TUNING ponaman ppeemeceseenOracle Database Architecture ? eensOracle Database Architecture ? ere eae SQL TUNING CSc ena caoOracle Database Architecture All the data is storedin Blocks A blockis the smallest unit of database storage A block can have 2KB to 32KB size (8KB default) A block stores row data or index data. RRAARAN Block Header Includes: + Block Type Information + Table Information + Row Directory @ Wecanuse PCTFREE or PCTUSE parameters to specify the space sizein blocks SQL TUNING CSc ena caoOracle Database Architecture Pane Column Data _— ee Row Header <—— I [| a] Ld] a NON a “s+. Row Piece in a Database Block [ T ——- ee Ro Overhead roe see rcs 10 (fcustered) Chained Row ies ay) mn tenth Column value SQL TUNING = Poe ec) pe) y } J 1 ao. | 2 [psec‘o> Oracle Database Architecture ‘ ‘ SESSION AREA + Stores session information of each user. SESSION AREA = Stores session variables, login information, session status, ete. @ PRIVATESQL AREA + Persistent area stores the bind variables Q + Runtime area stores the execution state Info. @ CURSORAREA + Stores the information of cursors @ SQLWORKAREA SQLTUNING Fon ~ ReteOracle Database Architecture Ea @ DATADICTIONARY CACHE + Stores the definitions of the database objects and their permissions @ RESULTCACHE = Stores the result of commonly used queries = Stores the result of functions @ LIBRARYCACHE + Stores the execution plans J * Stores procedures, packages control structures i ; &% OTHERAREAS BS KS = SQL TUNING oe ecOracle Database Architecture (ERs aed @ Largest memory area of SGA. @ Stores the copies of the data blocks read from the disc =. @ Why to read into the buffer cache? + Much faster than discs, gQ Maintained with a complex algorithm + Stores the most recently used & most touched ones, Database write process handles the Z write operations to the disc. gQ @ Stores the index data, too. SQL TUNING CSc ena caoOracle Database Architecture QR @ CDN eased Oracle Guarantees Not to Lose Data A Redo Log Entryis created when insert, update, delete, create, alter, drop occurs Redo Log Entries has the changes made to the database They are used for recovery operations Redo Log Entries are stored in the Redo Log Buffer Redo Log Buffer is a circular Buffer Rollback is not done with Redo Log Data SQL TUNING = Poe ec) pe) MeeeOracle Database Architecture % The original data stored into the memory (undo tablespace) is called as undo data. @ Another copy of the datais storedinthe buffer cache for the modifications. 4 Undo data is not modified because: UPDATE * Used for rollback operations DML operations * Used for providing read consistency * Used for providing flashback feature at Lj % Blocks > Extents > Segments > Tablespaces SQL TUNING Coe ee eee oyOracle Database Architecture ™% When we runa DMLcode, the server: Checks the Shared SQL. Area for similar statements to use. ‘Checks the Data Dictionary Cache and checks if our query is valid. Checks Buffer Cache & Undo Segments for the related data, Locks the related blocks. Makes the change to the blocks in the Buffer Cache. The changes are applied to the Redo Log Buffer before the Buffer Cache. ¥ The server returns the feedback for the change. ™@% When theusercommits: ¥ The server creates a commit record with SCN. ¥ The LGWr process writes redo log entries in the redo log buffer to the redo log files. ¥ The DBWn writes the dirty blocks to the disc & unlocks the blocks. v The server returns a feedback about the transaction completion. SQL TUNING CSc ena cao co. 4Oracle Database Architecture @ The size of each memory area is important for the execution performance of your queries 4% Oracle can manage the memory . automatically @ Itcan manage both SGA and PGA memories ! @ It is recommended to leave automatic memory management enabled & Enabling automatic memory management will prevent out of memory errors SQL TUNING op a poOracle Database Architecture @ Storage = Discs @ Control Files: Stores the pyhsical structure information of the database. @ Datafiles : Stores Data (Tables, procedures, application z data...) % Online Redo Log Files : Stores redo log entries Control es @ Archived Redo Log Files: Online redo log files are constantly moved here @ Backup Files : Stores the exact copy of the data files for ‘ Parameter le disaster recovery @ Parameter File : Stores the configure data of the database instance @ Password File: Stores the passwords of the admin users password sie sn aoe en ] (sysdba, sysoper, sysasm) ros Lm ke % AlertLog& Trace Files: Stores log messages and errors ‘occurredin the database. SQL TUNING CSc ena caoOracle Database Architecture TeSeIe NEN ee eee Blocks : Smallest units of storage (2KB-32KB) Lest Se Extents : Combination of several Database |p consecutive data blocks. Used for storing — i 1. specific type of info. Tablespace = Segments: Combination of several extents. ce Used for storing some big data (tables, ogenaees[ Segment indexes, etc). Tablespaces : Combination of many Ss segments. Used for grouping the related data in one container J Ohare eieinnpoeodtatio Oracle data session i + Permanent Tablespace :Stores the persistent schema KS objects SQL TUNINGPerformance Tuning Basics CEERI = SQL Tuni is a continuous process = You need to tune your queries: + While creating + After the creation = When to decide tuning? + By checking the top consuming queries frequently + After any complaints of bad performance + The reasons of performance loss + soucuuatenanges —_— + chonaesonine sta volume + Applicaton Changes + Aged suse + Database parades + Database Parameterchances — operating Sytem & Harvane Changes SQL TUNING oe ee eee ~ Bee eee eee MASTERCLASS par Hast 1c ovataPerformance Tuning Basics = Bad SQLs use more resources than necessary f§ PCT RTI KS ri + a = Characteristics of a Bad SQL 4 7 1 * Unnecessary parse time * Unnecessary VO operations " fr, + Unnecessary CPU time TONE UC TT ns 1a Te = Time on Wait (CPU) + Time on Execution = DB TIME = The reason of a Bad SQL: = Bad Design, Poor Coding, Inefficient Execution PlanPerformance Tuning Basics = Select the data types carefully + Assign data types as much as needed + Select exactly the same data type between parent-child keys I = Enforce data integrity = Use normalization well * Select right table type = Create Clusters = Use indexes often and select index type carefully i = Create index-organized tables (IOT) SQL TUNING Ec eau caoPerformance Tuning Basics UES Data’ Data 7 Data = Es WF pata SQL TUNING Ec eau caoPerformance Tuning Basics Partitioned ees) SQL TUNING Ec eau caoPerformance Tuning Basics ALTER TABLE employees MODIFY PARTITION BY RANGE (hire date) (PARTITION P_NANEL VALUES LESS THAN (TO_DATE('@1/01/2018", 'DD/MY/VYYY')) TABLESPACE USERS, PARTITION P_NANE2 VALUES LESS THAN (TO_DATE(‘C1/01/2019', ‘DO/MM/YYYY")) TABLESPACE USERS, PARTITION PLNGHES VALUES LESS THAN (TO_DATE(' 1/01/2020", 'DO/MA/YYYY")) TABLESPACE USERS, PARTITZON P_NANES VALUES LESS THAN (NAKVALUE) TABLESPACE” USERS, ) ONLINE UPDATE INDEXES (C1Oxa_SALARY oct, Tox2_ENP_ID GLOBAL PARTITION BY RANGE (employee_sd) (PARTITION TP1 VALUES LESS THAN (MAXVALUE)) % 2 TF SQL TUNING Ec eau caoPerformance Tuning Basics i, @ (Hard parse) / Yes(Soft parse) SQL TUNINGPerformance Tuning Basics (ERASMAS oad SELECT * FROM products WHERE prod_category = ‘Electronics’; ‘Check Schema Information ¥ Find Possible Access Pathe CheckStatistics Recultic i ofthe whole table, [[pesuiszanortnewnotetnie ‘ T T / f Useindex Read whole Table SQLTUNING os 7 oN oe ea eeePerformance Tuning Basics Ce oi very Transformer J Transformed Query peel pores Statistics ae, ‘ i” oe a Plan 4 Generator . as y (to Row Source Generation) eg SQL TUNING $B wv oracie-mastercom YA oraclemaster@outlookcomPerformance Tuning Basics PIES a Sad Query transformer transforms the query into a better performing one but semantically equivalent of it If the transform isn't better. it uses the original one Time restriction and old statistics may lead a wrong plan creation ORExpansion +_Using OR in the where clauce will preventindex ueages SELECT * FROM sales WHERE prod_id = 14 or prono_id = 335 SELECT * FROM sales WHERE prod_id = 14 UNION ALL SELECT * FROM sales MERE prono_id = 33 and prod_id <> 14; ‘Subquery Unnesting +_the optimizer transformsa nested queryintoajoin statement, SELECT * FROM sales WHERE cust_id IN (SELECT cust_id FROM customers); SELECT sales.* FROM sales , customers WHERE sales.cust_id = customers .cust_ids SQLTUNING = Poe ee) pe) 4 4 peer etePerformance Tuning Basics SELECT * FROM sales WHERE promo_id = 999; SELECT * FROM sales HNERE prono_id = 9995 + Sales Table nas 9RBBABrOvs + promo _id column has 4 distinct values. +The Resulthos 267.857 rows SELECT * FROM sales WERE promo_id SELECT num_distinet FROM dba_tab_coluans WHERE table_name = ‘SALES? +The Resulthae 2074 rows selectivity = Number of rows returning from the query SELECT * FROM sales WHERE cust_id = 1000035 Total numberof rows * 7059 distinct values, 7059 é + Why Selectivity and Cardinality important? SELECT * FROM sales WHERE cust_id Cardinality> Total numberof rows x Selectivity = Selectivity : =) secs eratci oe io pron. id= 968 0 cham 1 Stocco , + cectnatylennds etrinjansn and cone ee een ae . + nconccrelettyand carey newectlancosestnation aca cn ae ie SITs EO ean ta Pes oe ‘ MASTERCLASSPerformance Tuning Basics & Cost is the optimizer’s best estimate of the number of /Os to execute a statement. & To estimate the cost, the estimator uses; & Disk Yo, @ CPU usage, @ Memory usage. [Segoe [mae oc Single-block read time CPU Cost unter oe Cost = yey SQL TUNING $B wv oracie-mastercom YA oraclemaster@outlookcomPerformance Tuning Basics SELECT e.first_name, e.last_name, e.department_name from FROM employees e, departments d WHERE e.department_id = d.department_id; Join erder[2]+ DEPARTHENTS[O]R6 ENPLOEESTE]#2 Bestis Joanhethods Hash Join order[2]+ ERPLOYEES[E}#1 DEPARTHENTS[O]¥ int doin Wa Join @ | setecr stares | ! 1 | HASH 2026 | 106 | . 2 | TABLE access Fut | oepantmets| 27 | 3 | tame access eu | ewvoveer | 307 | f SQL TUNING Eo adPerformance Tuning Basics = Once the plan generator generates the optimum plan, it handles that to the row source generator = Row source generator generates an iterative execution plan usable for the database = Rowsourceis an area that we get the row set (Table, view, result of join or groups) = Row source generator produces a row source tree (A collection of row source: = Rowsource tree shows the following information : Bienes ; Meera ares ose sean mera] SQL TUNING $B wv oracie-mastercom YA oraclemaster@outlookcomPerformance Tuning Basics fo} aU NIN fes coe ee Pa eRenrne Sey es parPerformance Tuning Basics ETN eee Se % SQL Tuning Principles % SQL Tuning Strategies ¥ Identifying the issue v Parse time reduction Y Clarify the details of that issue. y¥ Plan comparison strategy ¥ Collecting data ¥ Quick solution strategy v Analyze the data ¥ Finding & implementing a good plan ¥ Choose an appropriate tuning ¥ Query analysis strategy > } strategy ‘ / f 4 SQL TUNING coy Booms We peamcasenPerformance Tuning Basics @ Weuse Query Analysis Strategy when: ¥ Quick tuning strategies did not work, and we have time to work on this problem ¥ Query can be modified ¥ Determine the underlying cause &@ Whatto doon this Strategy? y Statistics and Parameters ¥ Query Structure f 4 v Access Paths ¥ Join Orders & Join Methods ¥ Others . . / f 4 SQL TUNING = Coe eo ~~ Bee eaeeer neoy Pa eRenrne Sey es pe) Hast Loe LasPerformance Tuning Basics ‘Analyzing the Execution Plan = Collecting Data a v Execution Plan {SQL trace, TKPROF, V$_SQL_PLAN, DEMS MONITOR, AWRSQRPT-SQL, ete} ¥ Information of objects in the query HE stow to reste Secu ios v Statistics ¥ check the access pathe Check the join orcer and the join type Compare actual & estimated number of rows, Find the steps where cost and logical reads are different ¥ The available tools are ‘significant (SQLT, DBMS.STATS, TKPROF, AWR Report, ete) * Analyzing by considering the query tuning techniques * Find a possible solution + Updating statistics ¥ Histograms v. Parameter Settings + Pre-Analyze of the Query ¥ Check the volume of resulting data + Using aynamie atatstles ¥ Check the predicates + coating oF re-creating an indox ¥ Check the problematic constructs peace + using Hints + others SQL TUNING Se CS Ree ee ~ Bertone erst Sa 5 a @outhExecution Plan & Statistics SQL TUNING aeExecution Plan & Statistics * Statistic Types + Optimizer Statistics + System Statistics + Can be gathered manually or automatically ANALYZE TABLE CONPUTE STATISTICS; + GATHER_DATABASE_STATS Procedure + Optimizer Statistics = System Statistics EXEC dbms_stats. gather_database_stats; + Used by the Optimizer to estimate V0 and CPU 7 GATHER DICTIONARY. STATS Procedure costs, EXEC dbms_stats. gather_dictionary_stets; + Should be generated regularly + GATHER SCHEMA STATS Procedure + Should be gathered during a normal workload EXEC dbms_stats. gather_schena_stats(ownnane=>'SH’ EXEC dbns_stats. gather_systen_stats(‘Start”); Piacente eee EXEC db ,_stats. gather_table_stats(ownname=>‘SH?, ‘tabnane=> ‘SALES’, cascade=>true); ‘+ GATHER INDEX STATS Procedure SELECT * FRO" sys SQL TUNING Ec ea caeExecution Plan & Statistics @ Howcan wesee the Optimizer Statistics? * DBA_TABLES * DBA_TAB_STATISTICS DBA_TAB_COL_STATISTICS DBA_INDEXES DBA_CLUSTERS DBA_TAB_PARTITIONS f f DBA_IND_PARTITIONS DBA_PART_COL_STATISTICS - » 7, ag SQL TUNING Ec eee caoExecution Plan & Statistics = To analyze an execution plan: 1" Autotrace + V$SQL_PLAN = EXPLAIN PLAN EXPLAIN PLAN FOR 3 + Generates the explain plan and saves into plan_table SELECT * FROM TABLE(DBNS_XPLAN.DISPLAY())3 EXPLAIN PLAN SET statenent_id = “MyID? FOR SELECT FROM EMPLOYEES where employee id = 1005 EXPLAIN PLAN SET st nt_Ad ‘WyID? INTO MyPlanTable FOR SELECT FROM EMPLOYEES where employee_id = 1003 SQL TUNING Ec ea caeExecution Plan & Statistics + Autotrace traces our query and produces the execution plan and the statistics SET AUTOTRACE ON; SET AUTOTRACE ON [EXPLAIN|STATISTICS]; SET AUTOTRACE TRACE[ONLY] ON [EXPLAIN|STATISTICS] SET AUTOTRACE OFF3 * Autotrace uses plan_table like the explain plan. SQL TUNING Ec ea caeExecution Plan & Statistics * There are a lot of performance views that can be used for tuning VSSQLAREA V$SQLLWORKAREA vssqL. V$SQLPLAN V$SQL_PLAN_STATISTICS V$SQL_PLAN_ STATISTICS ALL * V$SQL_PLAN Actual execution plans are stored here It is very similarto plan_table tis connected to V$SQL view SELECT * FROM TABLE(DBMS_XPLAN. DISPLAY_CURSOR( “Sdaxtahvagh94? ))3 SQL TUNING Ferenc peer renExecution Plan & Statistics SELECT p.prod_id,p.prod_name, s.amount_sold, s.quantity_sold FROM sales s, products p, custoners € VERE s.pred_id = p.prod_id AND s.cust_id = c.cust_id AND sucust_id 2N (2,3,4)5)s Aan Nod * Cost fan 1 1 gat anes Sesto tse ' | Bl ie s Access Methods — [I+ 21 “mar sor I | at | daar S| Mx atcess woz | mocers || “ten Brel | S| "tner tracer ' —— JonMothede@2cinTypee|! | “SEE SEH av soe arnt soma masa) sues | sat |? a Ts WoT ‘BUIVAP CONVERSION TO ROWZDS: 1 1 1 = Partin Pruning |/2 3! gett mest mae | uss ase | | * Others ust 10%4 oR *3%.*C08T 10%-5) ox *o%.so0er a Et a0 SQL TUNING eas | eeTable & Index Access Paths > Types of Indexes v B-TREE Indexes INDEX (TABLE £00127 Tyler Bennett £10297 + Normal Index Eo1234 John Rappl 21437 £03033 George Woltman 00127 De ae Eos242 Adam Smith £69535 + Index-Organized Table (107) E1001 David McClellan E04242 |E10297 Rich Holcomb 01234 ¥ BITMAP Indexes E 163938 Nathan Adams Es1298° , 21437 Richard Potter’ = £43128 > Attributes of Indexes E27002 David Motsinger_ Ez7ooe E41208 Tim Sampair £03033 * Key Compression E4size Kim Atlich £10001 z 1 0 Te ten £63535 Timothy Grove _E 16398 * Ascending ~ Descending Ordered Indexes &/, SQLTUNING $B wv oracie-mastercom ZB [email protected] fwripT rire hTable & Index Access PathsTable & Index Acce: Dene Sales Table 10@01000e00000e01010008000010 @1e0011111100¢100101111111101) e@e119e00e00e111e19e0 02000 C e000) 2@0e00000020000 002000000000 Cee BeesTable & Index Access Paths > Table Access Paths > Index Access Paths ¥ Table Access Full v v Table Access by ROWID v v_ Sample Table Scan v ot UT emis Set BU he rer Index Unique Scan Index Range Scan Index Full Scan Index Fast Full Scan Index skip Scan Index Join Scan oy Index Organized Tables. Bitmap Access Paths ou Hoog = — Poe ee) Cee u Seo acy Coe fea foraTable & Index Access Paths Ee ae) » Table Access by ROWID occurs when : ¥ ROWID is used in the where clause directly v Byan Index Scan operation E SQL TUNING $B wv oracie-mastercom ZB [email protected]Table & Index Access Paths Doane}Table & Index Access Paths INDEX RANGE SCAN >the data we queried Is Bounded from one or both sides, the optimizer can use a > canbe applet ieee and mapinsre cy Cant epee ere tthe optimizer finds one or mere leading columns with= > «sings, will use indo rangescan. ene query includes an order by of group by clauses withthe Indexing colurnn, ange scan wll not do any sort. already sorted. Itshould net have null values order by clause hac desckaywor Invi read the data in descending order. > Youcan create your index as descensing, CREATE 108% andex_nane cit employees (departnent_id D850) a > Fumetion-based indexes canbe secessed 3s Index range scan > Mtwildeard characters are written on the ght, t wll perform indoxrange scan (OHERE doportnant_nane LIKE “A Pe ele) oe eee ee Pye ree SeeTable & Index Access Paths Esa > All the rows of the tables are indexed by their indexes » When the optimizer uses the index full scan? > Query has order by clause only with the indexed columns. > Query has group by clause only with the indexed columns. | > Query requires a sort-merge join. a a HHoHoOoOGoe eS é&/, SQLTUNING $B wv oracie-mastercom ZB [email protected] Pye ree SeeTable & Index Access Paths > If the query requests only the columns existing in the index, it uses IFF Scan > Can be applied to both b-tree and bitmap indexes. » Hints can be used to force the optimizer to use IFF Scan. > The differences of Index Full Scan vs Index Fast Full Scan > Index Fast Full Scan always reads only from the index X Index Full Scan may read from table, too. > Index Full Scan reads blocks one by one, sequentially X Index Fast Full scan reads multiple simultaneously, in unordered manner. > Index Fast Full Scanis faster than Index Full Scan most of the times. > Index Full Scan can be used to eliminate sorting, but Index Fast Full Scan cannot. Pe RL le oe eee ee MASTERCLASS®Table & Index Access Paths See > Ifyou don’t use the indexed columns on the where clause, the optimizer will not use the indexes. y We don’t create indexes for all the rows because of the costs. V If the second, third, column of a compositeindex is used as an access predicate, the optimizer will consider the index skip scan. y Index skip scan skipsthe leaves which do not have any chance to have any matching rows. » What are the advantages? ¥- Helps to reduce the number of indexes ¥ Decreasestheindex space ¥ Increases the overall performance by reducing index maintenance é&/, SQLTUNINGTable & Index Access Paths INDEX SKIP SCAN SELECT + FROn EAPLOVEES WHERE AoE BeTHEEN 20 ano 30] BETable & Index Access Paths > Ifan index stores the columns of a query, the optimizer will perform index fast full scan. > If the combination of multiple indexes store the columns of a query, the optimizer will join them and read the data from that join. (INDEX JOIN SCAN) » What to know about Index Join Scan? ¥ The combination of indexes must have every colurnn of the select clause, ¥ There is no join limit. More than two indexes can be jolned together to get the data. ¥ ‘There might be any index access path before the index join scan ¥ If you write ROWIDin the select clause, it will NOT perform index join scan. Pe RL le oe eee ee MASTERCLASS®Using Optimizer Hints > To command the optimizer, we use optimizer hints. Optimizer hints force the optimizer to pick a specific action. ‘The optimizer may not follow your hints. Ifthe hint is not reasonable, the optimizer will ignore it. Hints can be operating on a single table, multi-tables, a query block, a specific statement. Categories of the hints: + Hint for optimization approaches Y Access Pathe Hints Query Transformation Hints ¥ oin Order Hints ¥ Join Operations Hints Praia Exeettions Hint ¥ others QLTU Terr yeeUsing Optimizer Hints SELECT /*+ hint_name(p1 p2 p3..) */ first_name FROM EMPLOYEES; > Hints can be used after a SELECT, UPDATE or DELETE keywords. > You can use the table name or its alias as the hint parameter. But if there is an alias, you cannot use the table name! > There can be only one hint area. > Be careful on the hints you selected, especially if you are using multiple hints. You may lead the optimizer to a bad execution plan. «| SQLTUNING $B wv oracie-mastercom ZB [email protected] eer ee SeerJoin Operations é&/, SQLTUNING eer ee Seer P= Foe —_ “a Land Nosted loop join ie efficient when joining row sources are small. Sort merge jin is better than nested loop join if table is big and (or) one side is sorted. Hash join Ie better than sort merge jon for most cases if both sides are not sorted already. Cartesian products the most costly on: ItJoins all the rows of one side with all the rows of the other side. [ypeJoin Operations Fd | operation || DEFARTMENT: 1 AN | EMP DEPARTMENT 1X | BY INDEX ROWID| EMPLOY 4 > Ajoin operations done by the driving table (external table, outer table) and the inner table. Tableis the general name, but it is actually a row source If two row sources are small, or bigger one has an index, the optimizer may perform a nested loop Nested loop returns may be efficient if you need some rows immediately You can use USE_NL(table! table2) hint to force the optimizer to use nested loops é&/, SQLTUNING $B wv oracie-mastercom ZB [email protected] MASTERCLASS®Join Operations DESI Pins es Prins reg TABLE io ioe bat 5 1 5 1 a 5 4 4 5 2 3 2 3 7 3 7 3 2 z a 2 3 4 3 4 PARC o es ey ee MASTERCLASS®Join Operations > If the sort exceeds the sort area in PGA, it will write the sorted data into the disc. But this increases the cost so much. Index Full Scan or Index Range Scan will be very useful in sortsmerge join, since the data is already sorted in indexes. > Sort is done on joining keys. > After the sorts, both row sourees are merged. > Sort Merge is better than Nested Loop Join if table if row sources’ sizes are large > In sort merge, there is no driving table or inner table, > Ifthe row source is already sorted, there will be no sort key in the execution plan. > Sort merge join is efficient when the join condition is not an equijin > use_merge{tablel table2) hint is used to force the optimizer to perform sort merge join Pe RL le oe eee ee eer ee SeerJoin Operations Eon Ene INDEX Poe) peta) Petras) Petes) | of 1 5 1 2 : : aaa! 5 I P= ~~ Foe Cees Pod Rated Yo foneJoin Operations > Ahash table is built for the smallest row source » Hash table is created by generating the hash values of the key columns > The keys of second row source are hashed and checked against the hash table » Full table scan is performed to the table that will be hashed > Hash join is performed only when an equijoin is used >use_hash(table1 table2) hint is used to force the optimizer to perform hash join ease) a) G MASTERCLASS® a oe eee eeJoin Operations rowidl — (2) ee rowid3 (4) etaay rowidl 2 rowid4 1 rowid2 3 2 rowid3 4 3 rowida 4 3 rowids 5 ra rowidé 7 5 9 ease) a)Join Operations > Joins all the rows of a table with all the rows of the other table > It is nota realistic join for business » It is mostly done by mistake by forgetting to write the join condition > It results a very high cost. fl a oe eee eeJoin Operations > There are 4 join types : ¥ Equijoins and Nonequijoins v Outer Joins ¥ Semijoins v Antijoins > Equijoins return the matching rows with the equality operator > Joining other than equality operator is called as nonequijoins » Outer joins return matching and nonmatching rows > Semijoins return the rows matching with the EXISTS subquery > Antijoins return the rows which does not match with the NOTIN subquery «| SQLTUNING Ru Ben eer ee SeerJoin Operations > If the join condition contains an equality operator, it is an equijoin. > Equijoins are the most commonly used join types > To improve performance, you should use equijoins whenever you can > If you use another operator than the equality operator, it is nonequijoin. > Equijoins can use all join methods mF, > Noneq' ns cannot use the hash join method «| SQLTUNING $B wv oracie-mastercom ZB [email protected] eer ee SeerJoin Operations CEN » Outer joins return the matched and unmatched, TT Bla rows of the sources. 5 1 > Outer joins can be used with all join methods rR ie i 5 >» With Nested Loop and Outer Join, inner table is v S the one whose nonmatching rows will return. 7 3 2 2 » With Hash Join and Outer Join, hash table is bullt for the one whose nenmatching rows will 3 4 NOT return, Pe RL le oe eee ee MASTERCLASS®Join Operations » Semijoin returns the first match > Semijoin is the way of transforming the EXISTS Error) subquery into a join 5 = > Sometimes the optimizer may select a different a aaa! 5 method than semijoin even if you used the 2 3 EXISTS subquery 7 3 » Semijoins can be used with all join methods 5 5 » Use EXISTS instead of IN if possible 3 4 Pe RL le oe eee ee MASTERCLASS®Join Operations EN > Antijoins return the rows that do not match with the NOTIN subquery. » By default antijoins are used with sort-merge joins. » The optimizer may select a different one or you can use HASH_AJ or NL_AJ hints to change the join method. é&/, SQLTUNING MASTERCLASS® 5 a P= Foe —_ “a Petitat) 1 [ypeOther Optimizer Operators > Result Caches a memory areain SGA to store the results of queries for some time to increase the performance. There are two ways to store results in result cache: > MANUAL (DEFAULT- Needs resuit_cache int) > FORCE (n0_resuit_cachehin is used not tostorein the result cache) > DBMS_RESULT_CACHE package has statistics, information and some memory managing abilities V$RESULT_CACHE_OBJECTS view has the result cache data. Table annotations can be used as the default storage option to the result cache. CREATE TABLE table_name (..) RESULT_CACHE (MODE DEFAULT| FORCE) ; p ALTER TABLE table_name (..) RESULT_CACHE (MODE DEFAULT|FORCE)5 é&/, SQLTUNING $B wv oracie-mastercom ZB [email protected] MASTERCLASS®Other Optimizer Operators Every VIEW operator in the operations area does not mean that a view is therein the query Y Each seperate query ina query is pretended as an inline view. Y y What is View Merging? y View Merging means joining the inner query and the outer for a better performance v Eventually, every query insidethe outer query is shown with VIEW operator unless they cannot be merged with the outer query. mF, Pe RL le oe eee ee MASTERCLASS®Other Optimizer Operators Gag » The main goal of clustering is improving the performancewith a different way of storage ‘ i Sales Table » Types of clusters: Fa ae Cluster Table INDEX cLusTeRs aaa eta heer cere Prmeec era Peele Frees ent SINGLE TABLE HASH CLUSTERS: 101136 brea ’ SORTED HASH CLUSTERS Products Table id category corey one can ae Monitors FEC PA Site a Spucsmsaes [Se peusicooeiens Pearce de Sw LoOther Optimizer Operators Em > Sort Operator Types: > SORT AGGREGATE Operator > SORT UNIQUE Operator IF YOU WANT THE DATA RETURNED IN ORDER, YOU ieccamianaiees ca NEED TO USE THE ORDER BY CLAUSE. THESE ORDER > SORT GROUP BY Operator TYPES DOES NOT GUARANTEE THAT THE ROWS WILL > SORT ORDER BY Operator RETURN IN ORDER! > HASH GROUP BY Operator > HASH UNIQUE Operator aT. > BUFFER SORT Operator é&/, SQLTUNING $B wv oracie-mastercom ZB [email protected] yee rr ey SerrOther Optimizer Operators > When we use IN clause, if the values in IN clause are not too many, the optimizer tends to use the INLIST Operator > For INLIST Operator usage, the search must be on the indexed columns >» How does INLIST Operator work? SELECT * FROM EMPLOYEES WHERE employee_id IN (100, 110, 146) 100 110 146 a a a aie SQLTU 0 H NG $B wv oracie-mastercom ZB [email protected] rerSQL Tuning Techniques Query analysis strategy mainly focuses on modifying our queries to a better performing one. > The steps of solving the performance problem: v Check the execution plan basically for the problematic areas. ¥ Check the statistics ¥ Check your query for the common mistakes ” Check the execution plans detailed Check the Access paths ¥ Check join orders and join types v Compare the actual & estimated number of rows mF, Check the operations where the cost and logical reads differ significantly SQLTU yee a oe eee eeSQL Tuning Techniques » Common reasons for a Bad SQL: > Common possible solutions: Make the statistics up to date ¥ Create function-based indexes % Poorly written query Use dynamic statistics ¥ Use index-organized tables X Index used or not used Create or modify indexes ei haat coambnrnece X There is no index + Rewrite the query to use an index » Use parallelexecution X Predicates are not used Use hints / Use materialized views & Wrong types in predicates y Remove wrong hints ¥ Modify or disable triggers and Change the hints X wrong Join order saa Eliminate implicit data type conversion ® Other v Other SQLTU yee a oe eee eeSQL Tuning Techniques > There are many different tools to get the execution plans and the statistics. > Explain plan or Execution Plan? > There are 4 major tools to get the execution plan and the statistics : wv Autotrace ¥ SQL Monitor v-TKPROF ¥ DBMS_XPLAN » AUTOTRACE : > Itis free. » SQL Developer and SQL Plus shows different statistics. » Collapse or expand and focus only the problematic area in SQL Developer Bona RpeorsgeenSQL Tuning Techniques » AUTOTRACE (Continues) : » Wecan compare two different execution plans and statistics with SQL Developer. » SQL Developer has a hotspot button which shows the problematic area of the plan. » We can exporta plan and use it later. > By default, SQL Developer autotrace doesn't read the all of the rows. (Check Too/s->Preferences->Database->Autotrace/expiain Plan in SQL Developer) » SQL MONITORING: » It shows the execution plan and the statistics of the query being used right now » It can be viewed by SQL Developer or Enterprise Manager or by an SQL code > It captures the queries running longer than 5 seconds or running in parallel mode > If our query doesn’t suit to these, we can add MONITOR hint to make it show on the list. > Needs Diagnostics and Tuning packs to be licensed. SQL TUNING Poe yee [ypeSQL Tuning Techniques » TKPRO! Converts Oracle trace files into a human readable format. V > We need to create a trace file first, and use TKPROF tool for this trace file. Using TKPROF needs a bit more work to do comparing to the others. > You need to have the access privilegeto the database server to use this. It includes all the SQL statements run between the tracing starts and ends. It breaks down the execution time into parse, execute and fetch times. > DBMS_XPLAN : > It shows performancestatistics for each step of the plan V V v > If the query has already run before, there is no need to run fot DBMS_XPLAN. » If it is deleted from the cache, the AWR tool can show earlier plans. SQLTU yee a oe eee ee> What does the Real-Time SQL Monitoring Tool do for us? » Wecan check the real-time execution plans and statistics with graphical views. » Captures the execution plans and statistics automatically » Very useful to analyze the complex queries and PL/SQL code executions. » This tool can be used via SQL Developer, Enterprise Manager, or by Code » We can view SQL Monitoring information from v$sql_monitor and v$sql_plan_monitor database views. > Captures the queries running longer than 5 seconds or running in parallel mode. > To see the queries running less than 5 seconds, we can use the MONITOR hint. > Needs to have Diagnostics and Tuning Packs to be licensed. Pe RL le oe eee ee yee rr ey SerrSQL Tuning Techniques >» What are the benefits of this tool? » Shows the execution in real-time with lots of details » Tracks all the running queries, which enables us to find the top consuming queries » We can create active reports of the running queries which lets us analyze offline » Monitors the parallel execution > Helpsto analyze the large plans and complex queries easier » We can utilize poorly used indexes mF, » We can determinethe bind variables of the queries Pe RL le oe eee ee MASTERCLASS®SQL Tuning Techniques >» What are the benefits of SQL Trace files? » Performance information of our queries. » CPUTime and Elapsed Time » Wait Events: » Execution Plans leew cours » Call Counts (Parse, Execute, Fetch) mF, » Physical and Logical Reads Bene RpeosgeenSQL Tuning Techniques SELECT value FROM V$DIAG_INFO WHERE name = ‘Diag Trace’; >» How to enable/disable tracing? » We can enable/disabletracing for a specific user | specific session | entire database. > dbms_monitor.database_trace_enable(); » dbms_session. set_sql_trace(); » alter session set sql_trace = true; » We can trace user, session, database, application, service, module, etc. mF, SQLTUNIN yee [ypeSQL Tuning Techniques USING SQL TRACE FILES & TKPROF TOOL » How to track our own session? ALTER SESSION SET SQL_TRACE = TRUE|FALSE; DBMS_SESSION.SESSION_TRACE_ENABLE(waits => TRUE, binds =>FALSE); DBMS_MONITOR. SESSION_TRACE_ENABLE(session_id=>27, serial_num=>60, waits=>TRUE, binds=>FALSE) DBMS_SESSION. SESSION_TRACE_DISABLE(); DBMS_MONITOR. SESSION_TRACE_DISABLE(session_id=>27,serial_num=>60) Kk. » tresess utility can merge multipletrace files. «| SQLTUNINGSQL Tuning Techniques > How to generate TKPROF output? » TKPROF takes trace files as input, and generates a formatted output file. >» Doesn't show the commit and rollback operations. TKPROF trace_file name output_file_name [waits=yes|no] [sort=option] [printsn] [aggregate=yes|no] [insert=insert_file_name] [sys=yes|no] [table=schema.table] [explain=user/password] | [record=record_file_name] [width=n]; SQLTUNIN = — Poe ee) Cee u Seo acy oreo fea foraSQL Tuning Techniques > Do not use select * for all the queries » The optimizer may select a worse plan if you query for unnecessary columns > Whilej ing multiple tables or querying from views, selecting less columns might affect the performance > Ifyou use select *, the databaseneeds to check the data dictionary to get the table structure » select * will make the database perform more I/O operations » select * may decrease the performancesignificantly if the table has LOBs » select * will have a higher overload on the network. So there might be more network waits » select * is tend to have problems on maintenance el ele) oe eee ee MASTERCLASS®SQL Tuning Techniques > How do | make the optimizer use my indexes? > If the selectivity of the predicate is high, using Indexes may increase the performance very much > Add adequate predicates to the queries (Use indexed columns in your queries clearly) > Use reasonable hints in your queries > If possible, select only the indexed columns mF, ease) a)SQL Tuning Techniques > How do | make the optimizer use my indexes? » To make the optimizer use our indexes, the indexed columns must be used clearly. SELECT first_name, last_name, department_name FROM employees WHERE first_name||last_name = ‘StevenKING? SELECT first_name, last_name, department_name FROM employees WHERE first_name = ‘Steven’ AND last_name = ‘KING’; mF, é&/, SQLTUNING $B wv oracie-mastercom ZB [email protected] MASTERCLASS®SQL Tuning Techniques » How do | make the optimizer use my indexes? » One of the common mistakes for tuning aspect is using the arithmetic operations on the indexed columns. SQL TUNING yee rr ey Serr SELECT prod_id, cust_id, time_id FROM sales WHERE time_id + 10 = ‘20-2AN-98?5 SELECT prod_id, cust_id, time_id FROM sales WHERE time_id = ‘10-JAN-987 ; SELECT prod_id, cust_id, time_id From sales WHERE time_id = to_date(‘20-JAN-98, ‘DD-MON-RR?)-105 P= ~~ Foe Cees Pod Rated Yo foneSQL Tuning Techniques » How do | make the optimizer use my indexes? SELECT employee_id, first_name, last_name, salary FROM employees WHERE last_name LIKE ‘%on é&/, SQLTUNING MASTERCLASS® SELECT employee_id, first_name, last_name, salary FROM employees WHERE last_name LIKE ‘Ba%?; SELECT employee_id, first_name, last_name, reverse(last_name) FROM employees WHERE reverse(last_name) LIKE ‘rahh%? ; P= ~~ Foe Cees Pod Rated Yo foneSQL Tuning Techniques > How do | make the optimizer use my indexes? » Using functions on the indexed columns may suppress index usage SELECT employee_id, first_name, last_name, salary FROM employees WHERE TRUNC(hire_date, ‘YEAR?) = ‘@1-3AN-2002? 5 SELECT employee_id, first_name, last_name, salary FRON employees WHERE hire date BETWEEN ‘@1-JAN-2@02? AND ‘@1-JAN-2002"; mF, é&/, SQLTUNING $B wv oracie-mastercom ZB [email protected] MASTERCLASS®SQL Tuning Techniques » How do | make the optimizer use my indexes? » B-Tree indexes do not index the null values and this sometimes may suppress the index usages in our queries. » Ways to handle NULL value based performanceloss: > Ifreasonable, create a BITMAP Index instead of a B-TREE Index (BITMAP Indexes store the NULL values) mF, SQLTU yee a oe eee eeSQL Tuning Techniques Io Te oo Sacre ins andi @ELEGRS Ta), — = { sevecr * FROM T, (SELECT X FROM T2) T2 WHERE T2.X = T2.X)5 } (sevect * FRom Ta WHERE EXISTS (SELECT X FRON T2 WHERE TA.X = 12.x); | = — FOR X IN (SELECT * FROM T1) LOOP IF (EXISTS (SELECT X FROM T2)) THEN OUTPUT THE RECORD END IF5 ENDS SQL TUNING $B wv oracie-mastercom ZB [email protected]SQL Tuning Techniques >» Which one is faster? » If theouter tableis big and the subquery is small, using IN might have a better performance. » If the outer tableis small and the inner table is big, using EXISTS might have a better performance. » Some wrong beliefs: there are any nl ale, the NOTIN wl tur Rething. . » The new versions of database generally finds the optimum choice between EXISTS and IN SQLTU yee a oe eee eeSQL Tuning Techniques ENERO a a » TRUNCATE is always faster than the DELETE command (Truncate doesn't generate UNDO data, but delete generates) » Things to know about TRUNCATE > Truncate operation cannot be rollbacked. Flashback is also not so easy after truncate operations. > Truncate is a DDL operation. So it performs commits before and after the truncate operation. >» We can truncate a single partition as well. » Truncate doesn't fire the DML triggers. But it can fire the DDL triggers. » Truncate makes unusable Indexes usable again, But delete does not. mF, Pe RL le oe eee ee MASTERCLASS®SQL Tuning Techniques > How do | make the optimizer use my indexes? » If the data types of the column and compared value don't match, this may suppress the index usage. SELECT cust_id, cust_first_name, cust_last_name FROM customers WHERE cust_postal_code = 60332; SELECT cust_id, cust_first_name, cust_last_name FROM customers J¥ Vv WHERE cust_postal_code = ‘603325 SQL TUNING $B wv oracie-mastercom ZB [email protected] MASTERCLASS®SQL Tuning Techniques > Order by mostly requires sort operations > The sort operations are done in PGA or discs (If PGA doesn’t have enough memory) > How to tune the order by clauses? » Create or modify B-Tree indexes including the column used in the order by clause » Increasethe PGAsize » Query for only the indexed columnsin the select clause » Restrict the returning rows mF, Pe RL le oe eee ee yee rr ey SerrSQL Tuning Techniques Eee > B-Tree indexes increase the performance a lot for the min & max value searches V To find the min or max value, it needs to read the whole table (if there is no B-Tree index) Y In B-Tree indexes, the rightmost and leftmost leaves have the maximum and minimum values of the indexed column y If the returning rows are restricted, this time instead of leftmost and rightmost leaves, some other leaves have the min and max values. But again, it doesn’t need to read all the rows of the index. V If the query has multiple aggregate functions or another column, it will perfofm-index full scan or table access full. Pe ele) oe eee ee MASTERCLASS®SQL Tuning Techniques > If they return the same results or if you don't care about the duplicates, you should use UNION ALL instead of UNION for performance (UNION ALL doesn't perform sort) SELECT prod_id,cust_id,time_id,amount_sold,channel_id FROM sales | were channel_id = 3; Eq UNION SELECT prod_id,cust_id,time_id,amount_sold, channel_id FROM sales WHERE channel_id = 43 SELECT prod_id,cust_id,time_id,amount_sold,channel_id FROM sales WHERE channel_id = 3; UNION ALL SELECT prod_id,cust_id,time_id,amount_sold,channel_id FROM sales WHERE channel_id = 45 é&/, SQLTUNING $B wv oracie-mastercom ZB [email protected]SQL Tuning Techniques > How can I tune the having clauses? » Having clause restricts the rows after they areread! » Predicates in the having clause will not be used as access predicates! GROUP BY prod_id SELECT prod_id, SUM(amount_sold) FROM sales HAVING prod_id = 136; x SELECT prod_id, SUM(amount_sold) FROM sales WHERE prod_id = 136 GROUP BY prod_id; > é&/, SQLTUNING $B wv oracie-mastercom ZB [email protected] MASTERCLASS®SQL Tuning Techniques > Views on performance aspect » Do not use the views out of their purposes > Things to be careful on views! > If you don't need to use all the tables in a view, do not use this view on your queries. > Don't join the complex views with a table or another view » Avoid performing outer joins to the views > Be careful on subquery unnesting me? » Avoid using views insideof a view Pe RL le oe eee ee MASTERCLASS®SQL Tuning Techniques » Unlikethe basic and complex views, materialized views store both the query and the data » The materialized view data can be refreshed manually or by a PL/SQL job, or by auto-refresh on DMLs » Materialized view maintenance also is a burden to the database » Using materialized views may increasethe performance a lot >» We can create indexes, partitions etc, on materialized views (Its table is an ordinary table) >» When query rewrite is enabled in materialized views or in your session, the optimizer may use your materialized view even if you don't query from it FZ, Pe RL le oe eee ee MASTERCLASS®SQL Tuning Techniques ENTERS eee TO » For each DML operations, the database creates UNDO data > Performing commit frequently will not help on performance. (UNDO datais already carried to discs frequently) > For each DML operations, thedatabasecreates REDO data > Performing commit frequently will not help on performance (REDO datais already carried to discs frequently) > Any changeson the same blocks (which are carried to the redo log files) will be created in redo log buffer again >» Updates and deletes will lock the rows and that will make the other users wait to perform any ‘other operations on these rows >» How often do we need to commit? mF, > As soon as we finish the DML operations > For some business-specific reasons SQLTUNIN Pye ree See a GHeucemamcsaen [CO pectcoseenteesn oy Ce BG oraciemaster@outlookSQL Tuning Techniques PENa Teas » SQL Codes are executed in SQL Engine & PL/SQL Codes are executed in PL/SQL Engine v Transfer of control between SQL Engine and PL/SQL Engine is called as CONTEXT SWITCH > BULK COLLECT decreases the context switches (It reads multiple rows in one fetch) > Things to know about BULK COLLECT » By default, the bulk collect fetches all the rows in one context switch >» We can change the fetch count by using the LIMIT keyword » Implicit cursors use bulk collect by default mF, aCe aSQL Tuning Techniques PENa Teas » SQL Codes are executed in SQL Engine & PL/SQL Codes are executed in PL/SQL Engine > Transfer of control between SQL Engine and PL/SQL Engine is called as CONTEXT SWITCH Eo’ pane MASTERCLASS® | SQL TUNING $B wv oracie-mastercom ZB [email protected]SQL Tuning Techniques Partitioned > Ifthe query has low selectivity, the optimizer mostly prefers performing full-table scans > Creating partitioned tables increases the cost for the queries having low selectivity > Selecting from specific partitions is called as partition pruning » How can we prune the partitions? > Selecting directly from the partition by using the partition name. me, > Adding predicates to the where clause including the partition key (partitioned columns) é&/, SQLTUNING $B wv oracie-mastercom ZB [email protected] MASTERCLASS®SQL Tuning Techniques UNIS SELECT p.prod_name, s.quantity_sold, s.amount_sold FROM sales s, products p WHERE s.prod_id= p.prod_id; (S-@ SELECT STATEMENT. 6 " i DqHast 20 “55, “ 918643 3659140. & DdnesteD Loces swe ‘s1a43 3858140 THA CNESION TO ROMDS) STP BEE GOVGEVNLE) SES ROD SH Rees rede SP%00 39 ROD 0 TF; Banc access oe ROELROWE) SLES 8 we 1s 3: paxrmousance (ul) stenosis FE rane acces pu) sus Si St in aCe Spares peeSQL Tuning Techniques PEN Re TH sum_amount AS (SELECT SUM(amount_sold) amt_sold, prod_id FROM sales GROUP BY prod_id) num_of_prods AS (SELECT COUNT(*) num_prods FROM products) SELECT prod_name, amt_sold / (SELECT num_prods FROM num_of_prods) FROM products P, sum_amount S WHERE P.prod_id = S.prod_id AND amt_sold > 100000; » Ifyou use similar queries multiple times, using the WITH clause may increase the performance » With the WITH clause, Oracle stores the result of the queries in user's temp space » Ifthe result is not very big, it stores the data as a view in memory me, > Ifthe result is big, it stores the data in a global temporary table automatically created for that query. é&/, SQLTUNINGTuning with Advanced Indexing Techniques > Indexing is the most important thing in SQL and Performance Tuning. > There are lots of very useful index types in Oracle Database. > Poor indexing strategy will result poor performance. » Since the data is stored randomly in the discs, indexes will have a much faster access to the data by using their exact location IDs. > Without indexes, even if you select a very small fraction of the table, it will read the whole table anyway. > Index keys are selected from the most frequently queried and the most seléétive Columns. > Indexes store the ROWIDs which are the exact location of the rows in the discs. Pe RL le oe eee ee yee rr ey SerrTuning with Advanced Indexing Techniques CETaSENe) ING IMPORTANT? > Why shouldn't we create indexes for all columns? =o > Because of indexing cost > Storing the similar data more than ence » Maintenance cost > Why & How should we use the indexes? > four queries search for a small fraction of the table » Ifthe related columns are queried so often >» We should select the index type carefully > Indexed columns must be selective Pe RL le oe eee ee MASTERCLASS®Tuning with Advanced Indexing Techniques » Index selectivity is very important for index performance » Ifthe database reads from an index, there are multiple reads here. It first reads from branches, then leaves, and then the table. » Ifthe index is not selective, its cost might be higher than reading the whole table » There is not any restriction for the optimizer to or not to use an index. SELECT column_name, eae, WHERE table_name = '<> = $2 wwworacle-master.com SB [email protected] Coe fea foraTuning with Advanced Indexing Techniques EasesTuning with Advanced Indexing Techniques Pea Index entry SEU & | SQLTUNING CCE aoe ~ Boer teeta oy le-maste fee neTable & Index Acce: Eee Sales Table 10@01000e00000e01010008000010 @1e0011111100¢100101111111101) e@e119e00e00e111e19e0 02000 C e000) 2@0e00000020000 002000000000 Cee BeesTuning with Advanced Indexing Techniques » The benefits of Bitmap Indexes : » Works faster than B-Tree indexes for large number of rows in the resultset. > Uses less aise space than the B-Tree indexes > Bitmap indexas dosen't store llthe rowks, Instead, it atores the intervals and then converts when it needs > Bitmapindexes store the bimaps compressed » More efficient when the query contains multiple conditions in the where clause » Can be used for parallel DML or parallel queries eee > Indexes the NULL values > Bitmap Join indexes are useful for multiple table reads SQLTUNING $B wv oracie-mastercom ZB [email protected] MASTERCLASS®Tuning with Advanced Indexing Techniques >» What to know about Bitmap indexes for more? > Usually easier to remove and re-create than maintain >» Not suitable for concurrent transactions modifying the indexed column » Low selectivity is better for bitmap indexes » You need to select global or local index partitioning carefully mF, Pe RL le oe eee ee yee rr ey SerrTuning with Advanced Indexing Techniques > Bitmap Conversion To ROWID- Converts the bitmaps to the corresponding ROWIDs > Bitmap Conversion From ROWID~ Generates bitmap index from b-tree > Bitmap Conversion Count Calculates the count by Using the index > Bitmap Index Single Value - Gets. single value by using the index > Bitmap Range Sean - Performsa range scan over the bitmap index > Bitmap Full Scan- Reads the whole bitmap to return the result > Bitmap Merge Sean - Merges multiple bitmaps ( result of a range scan) inte one bitmap. > Bitmap AND - Performsan AND operation over the bits of two bitmaps > Bitmap OR- Performs an OR operation over the bits of two bitmaps > Bitmap Minus ~ Performsan AND operation between a bitmap and the negated version of another Bmap > Bitmap Key Iteration Takes each row froma table row source, finds the corresponding bitmaps and merges them é&/, SQLTUNING $B wv oracie-mastercom ZB [email protected] MASTERCLASS®Tuning with Advanced Indexing Techniques » Composite indexes are the ones created for multiple columns » Advantages of composite indexes : » Higher Selectivity > Less YO first_name last_name amd > Canbeused for one or multiple columns v Selecting column order in composite indexes is important! > Wrong column order will lead a worse plan or not the use the index! v Select the columns in order of mostly queried & most selective SQL TUNING $B wv oracie-mastercom ZB [email protected]Tuning with Advanced Indexing Techniques > An index including all the columns of the query is called covering index for that query > Benefits of covering indexes: > There is no need to look up the data in the table » Needs less I/O operations > Drawbacks of covering indexes: » Increase the index size » Will be used for fewer queries » Maintenance cost increases mF, Pe RL le oe eee ee MASTERCLASS®Tuning with Advanced Indexing Techniques > Simultaneous inserts/updates on the indexed tables may have performance problems because of the Index maintenance (especially for the sequential values) > Sequential value inserts may cause contention in the index blocks with some waits or locks » Reverse key index Is not an index used by reverse function on the indexed column! » Reverse key indexes store the bytes of the indexed columns in reverse order (ROWID's are not reversed) > Reversing the bytes will lead the database to store them in different index blocks > Drawbacks of reverse key indexes : > It works with only the equality searches mF, > Ituses more CPU to reverse the key values CREATE INDEX ix ON temp(a,b) REVERS! SQLTUNING $B wv oracie-mastercom ZB [email protected] Pye ree SeeTuning with Advanced Indexing Techniques > We can create bitmap join indexes over two or more tables > Bitmap Join Indexes need less space than materialized views. CREATE BITMAP INDEX sales_temp_bjx ON sales(P.prod_subcategory, C.cust_city) FROM sales S, products P, customers C WHERE S.prod_id = P.prod_id AND S.cust_id = C.cust_id LocaL; Maintenance Cost is higher Only one table among the indexed tables can be updated concurrently by different transactions Parallel DML is only supported on the fact table The joined columns of dimension table needs to have a unique or primary key constraint IFDimension table has a multi-column primary key, each column of that PK must be in the join No table can be Joined twice In the Index vvYvYYyYyY Bitmap join indexes cannot be created on temporary tables é&/, SQLTUNING $B wv oracie-mastercom ZB [email protected] MASTERCLASS®Tuning with Advanced Indexing Techniques ‘COMBINING BITMAP INDEXES SELECT * FROM custome Santos") ED EE, a ED Se, ECO A(T EL LO CC A PRSe RIS eeTuning with Advanced Indexing Techniques CREATE INDEX emp_last_name_fix ON employees(UPPER(Last_name)); } » Function-based index stores the result of the function for each row >» Using functions over the columns will suppress the index usages except for the function-based indexes » We can use any type of functions (built-in or user-defined) > We can use multiple functions in a column or in multiple columns. » The restrictions: > The function needs to be deterministic > Aggregate functions cannot be used aT. > Function needs to have a fixed-length data type Pe RL le oe eee ee MASTERCLASS®Tuning with Advanced Indexing Techniques » Store thenon-key columns as well, in the index leaves > Thereis not a table in addition to an index. Just the index. v Store the rows in the order of primary key values v It reads faster than the ordinary indexes over the primary key values v The changes are only over the index (since thereis no table) v IT needs less storage (no duplicate columns | rows) v Have full-functionality of ordinary tables (all the objects including indexes can be created over index-organized tables) TF, » Primary keys can be composite key Pe RL le oe eee ee MASTERCLASS®Tuning with Advanced Indexing Techniques Restrictions & Disadvantages of IOTs (Index-Organized Tables) : > Cannot create with a bitmap index > Needs to have a ur jue primary key » Can have max 100 columns (255 in index portion - rest in the overflow segment) » Primary keys can have max 32 columns » Cannot contain virtual columns > PCTTRESHOLD size cannot be larger than 50% of the index block » Faster in updates but slower in inserts. > There is no physical rowid in 1OTS. There are logical rowids —— > Secondary indexes use logical rowids which makes it work slower Pe RL le oe eee ee MASTERCLASS®Tuning with Advanced Indexing Techniques When to use IOTs? » Ifthe where clauses mostly have the primary key column, but select clause queries for other columns as well > Queries returning for small number of rows > Ifthe table data is not so often changing » Ifyou don't need additional indexes over the |OTs » Ifthe table is small in both row count and column count me? > Ifan index already needs the majority of the columns SNe SS Le rerTuning with Advanced Indexing Techniques INDEX - ORGANIZED TABLES REATE TABLE customers_iot (cust_id NUMBER, cust_first_name VARCHAR2(20), cust_last_name VARCHAR2(4@), cust_gender CHAR(1), cust_year_of_birth NUMBER(4,0), cust_marital_status VARCHAR2(2@), CONSTRAINT cid_pk PRIMARY KEY (cust_id)) ORGANIZATION INDEX TABLESPACE iot_tbs PCTTHRESHOLD 26 INCLUDING cust_year_of_birth OVERFLOW TABLESPACE iot_tbs2; mF, é&/, SQLTUNING P= ~~ Foe Cees Pod Rated Yo foneTuning with Advanced Indexing Techniques >» We can create indexes over the clusters Y We cannot create indexes for every clusters We can create indexes for the Index clusters We cannot create Indexes for hash-type clusters v Default cluster type is index cluster We cannot make DML operations over the index-clustered tables before the index is created vy Cluster Indexes are stored in the Index segment v Cluster indexes store the null values > Cluster indexes have entries for each cluster key value ‘CREATE INDEX emp_dept_index » Index Clusters cannot be used without the indexes | ow cluster emp dep. cluster TABLESPACE USERS STORAGE (INITIAL 25K NEXT 50); é&/, SQLTUNING $B wv oracie-mastercom ZB [email protected] MASTERCLASS®Tuning with Advanced Indexing Techniques » Reasons to make an index invisible : > Compare the performance with the new ene before changing it, > Check if dropping that index results in some problems > Invisible indexes are also alive like the visible Indexes. But the optimizer ignores them > OPTIMIZER_USE_INVISIBLE_INDEXES parameter can be set to TRUE to make the optimizer use the Invisible indexes » Invisible indexes are maintained by the database SELECT * FROM user_indexes WHERE visibility = ‘INVISIBLE? ; ALTER INDEX ix INVISIBLE; ) (auter inpex ix visteLe; eee CREATE INDEX ix ON TEMP(a,b) INVISIBLE; SQL TUNING Poe yer re se [ypeTuning with Advanced Indexing Techniques » Ifthere are lots of redundant data in the index, it might be useful to compress it (Will decrease the size and may increase the performance) > Can be applied to unique and non-unique indexes v It will work better for non-unique Indexes. Iteliminates the duplicate keys » Composite indexes can be compressed by the first N keys CREATE INDEX ix ON temp(a,b,c) COMPRESS [N]; CREATE INDEX ix ON temp(a,b,c) COMPRESS ADVANCED HIGH|LOW3] mF, SQLTU yee [ypeTuning with Advanced Indexing Techniques > Things to know about index compression: » Column order is important > Bitmap indexes cannot be compressed > Partitioned indexes cannot be compressed before Tig version » Can be alternative to bitmap indexes in some cases » Indexes are created as non-compressed by default mF, SNe SS Le MASTERCLASS®Tuning with Advanced Indexing Techniques » CONTEXT Type Index > performs search operations over large documents (PDF, MS Word, XML, HTML, Plain Text) » Converts the words in the documents into tokens » Documents are stored in BLOB or CLOB type columns CREATE TABLE my_doc ( id NUMBER(20) NOT NULL, name“ VARCHAR2(200) NOT NULL, document BLOB NOT NULL)5 (©REATE INDEX my_doc_idx ON my_docs (document) INDEXTYPE IS CTXYS.CONTEXT)] SELECT name FROM my_doc WHERE CONTAINS(document, ‘Search Text’) > @5 SELECT SCORE(1) score, name FROM my doc WHERE CONTAINS(document, ‘Search Text’, 1) > @ me, ORDER BY SCORE(1) DESC; Score Operator returns the relevance score of the row forthe specified search text SQLTUNING $B wv oracie-mastercom ZB [email protected]Tuning with Advanced Indexing Techniques >» CTXCAT Type Index > Ideal for smaller documents or text fragments » Larger than the context index and takes longer to build » It creates indexes over the index sets EXEC CTX_DDL.CREATE_INDEX_SET( ‘products_iset'); EXEC CTX_DDL.ADD_INDEX('products_iset', 'prod_list_price'); CREATE INDEX my_products_name_idx ON products (prod_desc) INDEXTYPE IS CTXSYS.CTXCAT PARAMETERS (‘index set products_iset')} SELECT prod_id, prod_list_price, prod_name, prod_subcategory FROM products WHERE CATSEARCH(prod_desc, ‘CD’, ‘prod_list_price>ie’) > 05 mF, é&/, SQLTUNINGTuning with Advanced Indexing Techniques > CTXRULE Type Index : > Used to build a document classification application > Documents inside of the table are classified based on their contets > Used forthe category searches CREATE INDEX temp_rule ON tem _table(text) INDEXTYPE IS CTXSYS.CTXTULE5 SELECT CLASSIFICATION FROM temp_table WHERE matches(text, ‘Lionel Messi is a famous footballer from Argentina’) > 5 > Synchronizing the Full-Text Indexes EXEC CTX_DDL.SYNC_INDEX(‘my_docs_idx"); EXEC CTX_DDL.OPTINIZE_INDEX(‘my_docs_idx', FULL"); SQLTUNIN mF, [ypeTuning with Advanced Indexing Techniques > Indexed columns are not used in the where clause > First column of the index is not used in the where clause > Index may not be selective enough > Leading wildcard(%) characters > Selectivity of the LIKE clause is low > Using functions on the indexed columns. > Implicit data type conversion » The column can contain NULL values > Invalid hint usage eee » Outdated database statistics «| SQLTUNING $B wv oracie-mastercom ZB [email protected] eer ee SeerTuning with Advanced Indexing Techniques » Create the indexes after inserting the table data » Create indexes for correct tables and columns » Order the indexed columns carefully > Do not keep the unncessary indexes » Consider creating and using indexes in parallel » Consider creating indexes with NOLOGGING >» Do not drop or disable the constraints having an index without thinking the indexing cost » Consider coalescing or rebuilding the index A Pilea $B wv oracie-mastercom ZB [email protected]Advanced Tuning Techniques > Using the BIND Variables may increase the performance by decreasing the parse counts SELECT AVG(salary) FROM employees WHERE department_id = 30; SELECT AVG(salary) FROM employees WHERE department_id = 40; SELECT AVG(salary) FROM employees WHERE department_id = 50; SELECT sql_id, executions, parse_calls, first_load_time, last_load_time,sql_text FROM v$sql WHERE sql_text LIKE ‘%avg(salary) from employees%" ORDER BY first_load_time DESC; SELECT AVG(salary) FROM employees WHERE department_id = é&/, SQLTUNING $B wv oracie-mastercom ZB [email protected]Tuning with Advanced Indexing Techniques » The optimizer peeks the bind variable values for the first execution » After the first plan is generated, it uses that plan for the next executions > This may cause the optimizer to select suboptimal plans for the next executions » Why not it peeks for all the values? » Toeliminate the hard parses » When to use the bind variables then? > Don't use the bind variables if the cardinality of the values in the column Is pretty different > Ifthe cardinalities are pretty similar and they all will need the same plan, use bind variables Pe RL le oe eee ee MASTERCLASS®Tuning with Advanced Indexing Techniques > The lifecycle of a query: > Open : Allocates memory for that cursor Y Parse : syntax analysis, semantic analysis, privilege checks etc Bind : Bind variable values are assigned Define : Defines how you want to see the data Execute vvryvy Fetch > The data structure allocated in the database for that query is called as cursor in the server side > Using these cursors by multiple executions is called as cursor sharing » Parent cursor stores the SQL statement and child stores the Information related to the differences Pe RL le oe eee ee MASTERCLASS®Tuning with Advanced Indexing Techniques » When the database can share the cursors? > When Bind variables are used > Only if the literals are different » CURSOR_SHARING parameter should be set to: > EXACT: The default cursor_sharing parameter. It allows cursor sharing only if the queries are exactly the same. > FORCE : Allows cursor sharing if everything but literals are the same. But it js not guaranteed. » Needs extra work to find a similar statements in the shared pool during the soft parse > It needs to use more memory > Star transformation Is not supported —, > The cursor sharing can be set by alter session or alter system commands Pe RL le oe eee ee MASTERCLASS®Tuning with Advanced Indexing Techniques » The main goal of adaptive cursor sharing is, not to have a new cursor for each bind value, but not to usethe same cursor for every query also. » Enabled by default (It is applied automatically if the query does not have over 14 bind variables) » It is independent of CURSOR_SHARING parameter » Benefits of adaptivecursor sharing: » It automatically detects if the query needs another execution plan or can use the existing one » Decreases the number of generated child cursors to minimum > It works automatically. You don't need to start it —: Pe RL le oe eee ee MASTERCLASS®Tuning with Advanced Indexing Techniques ind-Sensitive Cursor SELECT * FROM customers_temp WHERE country Bind-Aware Cursor 1 Pieters 2 Preeti 3 Presi) * Lowest Selectivity : 0.126162 * Lowest Selectivity : 0.001476 + Lowest Selectivity : 0.001476 + Highest Selectivity : 0.367063, + Highest Selectivity : 0.001804 ‘+ Highest Selectivity : 0.001804 SQLTUNING $B wv oracie-mastercom ZB [email protected]Tuning with Advanced Indexing Techniques » Useful views for adaptive cursor sharing : v V$SQL ~ Stores if the query is bind sensitive or bind aware Y V$SQL_CS_SELECTIVITY - Stores the lowest and highest acceptable selectivity values v V$SQL_CS STATISTICS ~ Stores some extra info like buffer gets, CPU time, ete. Y V$SQL_CS_HISTOGRAM - Stores the histogram statistics of the queries using bind variables » Hints about adaptive cursor sharing : » BIND_AWARE - Makes the database skip monitoring that query to check bind-sensitivity >» NO_BIND_AWARE - Makes the database ignore that query for bind-sensitiveness mF, Pe RL le oe eee ee MASTERCLASS®Tuning with Advanced Indexing Techniques » The statistics used by the optimize! > Table Statistics: > Index Statistics > Number of rows > Number of leaf blocks > Numberof blocks > Number of branch levels > Column Statistics > Number of distinet keys » Number of distinct valuesin that column > Index clustering factor » Number of NULL values in that column > System Statistics > Data distribution statistics (Histograms) > WO performance > Extended statisties > CPU performance The statistics are gathered by the optimizer by using the DBMS_STATS package Pe RL le oe eee ee yee rr ey SerrTuning with Advanced Indexing Techniques > Before database version 12c the execution plan was determined before the execution and this plan was applied > Starting with 12c the optimizer can change the plan on runtime. > While executing the query, the statistics collector gathers some new statistics about cardinality and histograms » Ifthe new statistics do not match with the first statistics, the optimizer picks one of its sub- plans it stored » Itwrites ‘This is an adaptive plan’ on the execution plan to express that > Itis enabled by default —: e SQLTU MI NG oe oeTuning with Advanced Indexing Techniques » Starting with 10g, dynamic sampling is introduced. It allows the optimizer to gather additional information at the parse time. » Dynamic sampling gathers the statistics by some recursive calls before generating the plan » Scans a fraction of random samples from the table blocks and calculates the st ics based on these random blocks > You can control the dynamic sampling with : > ALTER SYSTEMSET OPTIMIZER_DYNAMIC_SAMPLING= 4; > ALTER SESSIONSET OPTIMIZER_DYNAMIC_SAMPLING= 8; » f+ DYNAMIC_SAMPLING(II) */ » Before 12c, the dynamic sampling level can be set to between 0 to 10 (Default is 2) > Dynamic sampling is renamed to dynamic statistics in 12 and beyond. >» New level Tl has automatic sampling Pe ele) oe eee ee MASTERCLASS®Tuning with Advanced Indexing Techniques > Why to use dynamic statistics? » Ifthe current statistics are not enough to create an optimal plan > Ifthe query is executed multiple times > Ifthe time for gathering the dynamic statistics is ignorable compared to the overall execution time » When to use dynamic statistics? > Statistics are missing » Statistics are stale » Statistics are insufficient > There is a parallel execution » There is a SQL Plan directive mF, Pe RL le oe eee ee MASTERCLASS®Tuning with Advanced Indexing Techniques Ce ee Boe eros No Hint 90 932 Level 0 90 932 evel 1 90 932 Level 2 90 932 Level 3 90 932 Level 4 98 932 Level 5 ava 92 Level 6 985 932 Level 7 966 932 Level 8 943 032 Level 9 932 932 2g, Level 10 932 932 Level 12 826 932 Pelee oe yer re se