Welcome

Janis Griffin Oracle 19c Automatic

Senior Sales Engineer
Indexing - Inside & Out
Who Am I

Senior Sales Engineer / DBA

[email protected]
Twitter® - @DoBoutAnything
– Current – 30+ Years in Oracle®,
DB2®, ASE, SQL Server®,
MySQL®
– DBA and Developer
Specialize in Performance Tuning
Review Database Performance for
Customers Common Question – How do I
tune it?
Agenda
• 19c Automatic Indexing – What is it?
• High Level Steps
– Capture
– Verify
– Decide
– Monitor
– Report
• 19c Automatic Indexing – How It Works
– DBMS_AUTO_INDEX Package
– How to drop Automatic Indexes
• Several Case Studies
3 quest.com | confidential
19c Automatic Indexing – What is it?
• Implements indexes based expert index tuning knowledge
– Identifies ‘candidate indexes’ based on table column usage
– Without DBA involvement
o Except for DBA can set preferences
> View report of indexes and their impact on the application

• Works incrementally
– So needs to be iterative and continuous
– Created as invisible
o Uses ‘SYS_AI’ as the name prefix
– Automatic indexes are tested
o If improved performance – indexes made visible
o If no improvement – indexes are marked unusable
> Later removed
4 quest.com | confidential
19c Automatic Indexing High Level Steps
• No DBA interaction
• All tuning activities
– Auditable via reporting
Capture

Monitor Identify

Online
Verify
Validation
Decide

5 quest.com | confidential
19c Automatic Indexing – How It Works
• Capture
– Captures the application SQL history into a SQL repository
– Includes SQL, plans, bind values, execution statistics, etc. Capture

• Identify Candidate Indexes Monitor Identify

– That may help the newly captured SQL statements

Online
– Creates indexes as unusable invisible indexes Validation
Verify
Decide
o Metadata only
– Drops indexes obsoleted by newly created indexes (logical merge)
• Verify
– Ask optimizer if index candidates will be used for captured SQL statements
– Materialize indexes and run SQL to validate that performance improved
– All verification is done outside application workflow
6 quest.com | confidential
19c Automatic Indexing – How It Works
• Decide
– If performance is better for all statements, indexes are marked visible
– If performance worse for all statements, indexes remain invisible
– If performance worse for some statements
o Indexes are marked visible except for SQL statements that regressed

• Online Validation
– Validation of new indexes continues for other statements online
– Only one of the sessions executing a SQL statement
Capture

o is allowed to use the new indexes

Monitor Identify

• Monitor Online
Validation
Verify

Decide

– Index usage is continuously monitored

– Automatically created indexes will be dropped if not used in a long time
7 quest.com | confidential
19c Automatic Indexing Benefits
• Great for OLTP, OLAP, mixed workloads but critical for OLTP
• Applies to tuned and un-tuned applications
– If tuned
o Existing secondary indexes may be outdated
o Important indexes are missing
o Some secondary indexes can be dropped and auto indexes can be added
– If un-tuned
o Existing indexes support primary and unique key constraints Capture

• Can be used in all stages of application lifecycle Monitor Identify

• Support single and concatenated indexes Online

Validation
Verify

– Function-based indexes Decide

– Compression advanced low

8 quest.com | confidential
19c Automatic Indexing
• Automatic indexing defaults to run in same database as application
• Indexing task consumes CPU, memory and storage
– Resource manager plan limits task to 1 CPU
– DBA can control
o Which temp tablespace is used to build indexes
o Which tablespace and how much space can be used by auto indexing

Production Database

CPU Task
Memory Build
Storage Store

9 quest.com | confidential
Automatic Indexing Requirements
• Feature is only available to Enterprise Edition on Engineered Systems
– Exadata only

– Workaround for testing

o In CDB as sysdba
Alter system set “_Exadata_feature_on”=true scope=spfile;
Shutdown immediate;
Startup
– Unfortunately this is not supported
o Don’t use on production system
10 quest.com | confidential
19c DBMS_AUTO_INDEX Controls Auto Indexing
• Automatic indexing procedures
– CONFIGURE
o AUTO_INDEX_MODE – Turns on, off or report only
> IMPLEMENT - Turns on automatic indexing
> New indexes that improve performance are made visible & used by optimizer
> REPORT ONLY -Turns on automatic indexing
> New indexes remain invisible
> OFF - Turns off automatic indexing (DEFAULT)
o AUTO_INDEX_SCHEMA
> Can include/exclude schemas using ALLOW parameter
> Is case sensitive & can use wildcards
> If NULL, all schemas can use auto index

11 quest.com | confidential
Configure.Auto_Index_Mode Example

12 quest.com | confidential
Configure.Auto_Index_Schema Example

COLUMN parameter_name FORMAT A40

COLUMN parameter_value FORMAT A15

SELECT parameter_name, parameter_value

FROM cdb_auto_index_config
ORDER BY 1, 2;

13 quest.com | confidential
DBMS_AUTO_INDEX.CONFIGURE - Cont.
– CONFIGURE
o AUTO_INDEX_RETENTION_FOR_AUTO
> Number of days (default 373) auto indexes retained after last used date
o AUTO_INDEX_RETENTION_FOR_MANUAL
> Number of days (default NULL) manual indexes retained after last used date
o AUTO_INDEX_REPORT_RETENTION
> Number of days automatic indexing logs are retained before deletion
> Automatic indexing report is based of the logs (Default is 31 days)
o AUTO_INDEX_DEFAULT_TABLESPACE
> Tablespace to use to store auto indexes (Default is NULL)
o AUTO_INDEX_SPACE_BUDGET
> Percentage of tablespace size to use for auto indexes
> Can only be used when using default tablespace is used

14 quest.com | confidential
DBMS_AUTO_INDEX.CONFIGURE - Cont.
– CONFIGURE additional commands
o PARAMETER_VALUE is specific to parameter
> If NULL, setting is assigned a default value
o AUTO_INDEX_COMPRESSION enables/disables advanced compression
> ON for Advanced Low Compression
> OFF for no compression (Default)
o ALLOW for AUTO_INDEX_SCHEMA parameter
> TRUE adds the specified schema to the inclusion list
> FALSE adds the schema to the exclusion list
> NULL remove the schema from the list that it is currently added

15 quest.com | confidential
DBMS_AUTO_INDEX.CONFIGURE Examples

16 quest.com | confidential
DBMS_AUTO_INDEX.CONFIGURE Examples

17 quest.com | confidential
New (CDB/DBA) Views for Auto Indexes
• DBA_AUTO_INDEX_CONFIG *
– Display the current configuration of the automation index
• DBA_AUTO_INDEX_EXECUTIONS
– History of Automatic Indexing task executions
• DBA_AUTO_INDEX_IND_ACTIONS
– Actions performed on automatic indexes (e.g create, rebuild, etc…)
• DBA_AUTO_INDEX_SQL_ACTIONS
– Actions performed on SQL to verify automatic indexes
• DBA_AUTO_INDEX_STATISTICS
– Shows statistics related to automatic indexes
• DBA_AUTO_INDEX_VERIFICATIONS
– Shows statistics about PLAN_HASH_VALUE (original buffer gets, etc…)

18 quest.com | confidential
Additional Views for Auto Indexes
• DBA_ADVISOR_TASKS – new tasks

• DBA_INDEXES – new column (AUTO)

SELECT owner,
index_name,
auto,
index_type,
table_owner,
table_type
FROM dba_indexes
WHERE table_owner = 'TEST’
-- WHERE auto='YES'
ORDER BY owner, index_name;

19 quest.com | confidential
SMB$CONFIG Table
• Shows both documented & undocumented settings for Auto Indexes

20 quest.com | confidential
Other DBMS_AUTO_INDEX Functions
• DROP_SECONDARY_INDEXES
– Deletes all the indexes, except the ones used for constraints
o From a schema or a table
o Example - begin dbms_auto_index.drop_secondary_indexes('SH'); end;

• REPORT_ACTIVITY SELECT DBMS_AUTO_INDEX.report_activity(

activity_start => SYSTIMESTAMP-1,
– Returns a report of automatic indexing operations activity_end => SYSTIMESTAMP,
o Executed during a specific period type => 'TEXT’,
section => 'ALL’)
FROM dual;
declare
report clob :=null;
begin
report :=DBMS_AUTO_INDEX.REPORT_ACTIVITY (
activity_start => TO_TIMESTAMP('2020-01-01', 'YYYY-MM-DD'),
activity_end => TO_TIMESTAMP('2020-01-31', 'YYYY-MM-DD'),
type => 'TEXT',
section => 'SUMMARY',
level => 'BASIC');
dbms_output.put_line(report);
end;

• REPORT_LAST_ACTIVITY - Returns a report of the latest operation

21 quest.com | confidential
19c Automatic Indexing Reporting & Hints
• Each auto index task generates a report
– Reports can be generated via
o DBMS_AUTO_INDEX.REPORT_ACTIVITY function
> Date/Time range
> Format (XML, HTML, Text)
> Level (basic, typical, all)
> Section
> Summary, Index Details,
• Verification Details, Errors, All

• Use hints to control auto indexes

– /*+ USE_AUTO_INDEXES */
– /*+ NO_USE_AUTO_INDEXES */

22 quest.com | confidential
Report_Activity Example

23 quest.com | confidential
Report_Activity Cont.

24 quest.com | confidential
Report_Activity Cont.

25 quest.com | confidential
Other DBMS_AUTO_INDEX Procedures
• REPORT_LAST_ACTIVITY - Returns a report of the latest operation
declare
report clob := null; select dbms_auto_index.report_last_activity() from dual;
begin
report := DBMS_AUTO_INDEX.REPORT_LAST_ACTIVITY (
type => 'TEXT',
section=> 'ALL',
level=> 'TYPICAL');
dbms_output.put_line(report);
end;

– Type can be TEXT (default), HTML or XML

– Section can be SUMMERY, INDEX_DETAILS,VERIFICATION_DETAILS, ERROR or ALL
o Can combined
> SUMMARY + INDEX_DETAILS – shows summary and index_details
> ALL – ERRORS – shows every section except errors
– Level = Basic, Typical or All

26 quest.com | confidential
3 Case Studies
• Tuning Examples
– Used throughout the years with many Oracle versions – 10 & up
– Manual results were compared with the Tuning Advisor suggestions
o Consistent in previous releases
> Advisor usually missed the mark or got close but required additional DBA intervention

• Oracle 19C – Test Automatic Indexing

– Billing Query for a University
– Sale Order Query
– Popular Airline Flights in USA

27 quest.com | confidential
 Billing Query for a University
• Slow performance was reported by a customer
– Having trouble with their billing system
– The following query was identified as performing poorly
SELECT s.fname, s.lname, r.signup_date
FROM student s
INNER JOIN registration r ON s.student_id = r.student_id
INNER JOIN class c ON r.class_id = c.class_id
WHERE c.name = 'SQL TUNING'
AND r.signup_date BETWEEN
to_date(:beg_date,'DD-MON-YY') and to_date(:beg_date,'DD-MON-YY') +1
AND r.cancelled = 'N’;

– Table sizes
o Registration – 80,000 rows
o Student – 18,000 rows
o Class – 1000 rows

28 quest.com | confidential
Original Plan

29 quest.com | confidential
Original Performance

30 quest.com | confidential
Auto Indexes Enabled for Schema ‘Test’

31 quest.com | confidential
 Review Process of Registration (class_id, canceled)
SELECT a.execution_name, a.table_name,
a.index_name, b.stat_name, a.start_time
FROM dba_auto_index_ind_actions a, dba_auto_index_statistics b
WHERE a.execution_name = b.execution_name
ORDER BY 5,3;

DBA_AUTO_INDEX_VERIFICATIONS

32 quest.com | confidential
Auto Indexes Created
• Shows status of indexes
– 2 indexes are taking up space

33 quest.com | confidential
With Compression
EXEC DBMS_AUTO_INDEX.CONFIGURE('AUTO_INDEX_COMPRESSION','ON');

34 quest.com | confidential
Report_Activity

35 quest.com | confidential
Report_Activity Cont.

36 quest.com | confidential
Auto Index Plan

37 quest.com | confidential
Auto Index on Registration

38 quest.com | confidential
With Auto Index

39 quest.com | confidential
DBA Tuned Query

Create index cl_name

on class (name);

Create index cov_reg

on registration
(class_id,signup_date,
cancelled);

40 quest.com | confidential
Best Performance

41 quest.com | confidential
Did it measure up?

Auto Cost
107 (class)
76 (reg)

create index cov_reg on registration(class_id, signup_date, cancelled)

DBA Tuned Cost

5

42 quest.com | confidential
Sale Order Query
• HammerDB load utility – Slow running query
SELECT c_last, c_first, c_street_1, c_city, c_state, c_zip, Order_line 60,461,709
c_phone, o_entry_d, d_name, ol_delivery_d, ol_quantity, ol_amount
FROM order_line, orders, district, customer, stock
WHERE o_id = ol_o_id Orders 6,046,215
AND o_c_id=c_id
AND s_i_id = ol_i_id District 50
AND d_id = ol_d_id
AND ol_w_id = :B2
AND ol_d_id = :B4
Customer 150,000
AND (ol_o_id < :B3 )
AND ol_o_id >= (:B3 - 20) Stock 500,000
AND s_w_id = :B2
AND s_quantity < :B1
AND d_id = :B4
AND c_last like :B5 ;

43 quest.com | confidential
Existing Indexes

44 quest.com | confidential
Original Execution Plan

45 quest.com | confidential
Original Performance

46 quest.com | confidential
Include SOE Schema for Auto Indexing

SELECT index_name,table_name,
auto,visibility, compression,
segment_created, status
FROM user_indexes
WHERE auto='YES';

47 quest.com | confidential
Automatic Indexes

Total Space: 225m

Visible: 9m

48 quest.com | confidential
New Execution Plan

49 quest.com | confidential
Performance

50 quest.com | confidential
DBA Fine Tunes the Query
• create index orders_i2 on orders(o_id,o_c_id, o_entry_d);

51 quest.com | confidential
Popular Airline Flights in USA
SELECT
o.carrier, uc.description AS carrier_name
,ao.description AS origin_airport,co.Description AS origin_city
,o.fl_date,o.fl_num,o.tail_num
,ad.description AS destination_airport
,cd.Description AS destination_city ,w.Description Day_of_Week
FROM t_ontime o
INNER JOIN L_UNIQUE_CARRIERS uc ON uc.Code = o.UNIQUE_CARRIER
INNER JOIN L_AIRPORT_ID ao ON ao.Code = o.ORIGIN_AIRPORT_ID
INNER JOIN L_AIRPORT_ID ad ON ad.Code = o.DEST_AIRPORT_ID
INNER JOIN L_CITY_MARKET_ID co ON co.Code = o.ORIGIN_CITY_MARKET_ID
INNER JOIN L_CITY_MARKET_ID cd ON cd.Code = o.DEST_CITY_MARKET_ID
INNER JOIN L_WEEKDAYS w ON w.Code = o.DAY_OF_WEEK
WHERE to_date(fl_date,'YYYY-MM-DD') BETWEEN &beg_date AND &end_date
AND co.Description = &city
AND w.Description = &day_of_week;

L_UNIQUE_CARRIERS: 1620
L_AIRPORT_ID: 6438
L_CITY_MARKET_ID: 5823 US DOT - On-time Performance
L_WEEKDAYS: 8
T_ONTIME: 6784044

52 quest.com | confidential
Only Access Path is Full Table Scans
• No Original Indexes

53 quest.com | confidential
Automatic Indexes

• Visible Indexes

54 quest.com | confidential
Automatic Index Report
select DBMS_AUTO_INDEX.REPORT_LAST_ACTIVITY('TEXT','ALL','ALL') from dual

55 quest.com | confidential
New Plan

56 quest.com | confidential
Original Performance

57 quest.com | confidential
Auto Index Performance

58 quest.com | confidential
Summary
• Automatic Indexing can speed up performance
– 19c Optimizer has come along way
• Beware of just turning it on blindly
– Especially in production
– Watch out for baselines
o Created even when you have ‘capture’ turned off
– Auto Indexes can be Invisible / Visible
• Consider using in development / test
– Be cautious using in production
• Control at schema level
• Turn on compression for space savings
59 quest.com | confidential