<Insert Picture Here>
Upgrading to 11g Why, How and Best Practices
Roderick Manalac Consulting Technical Advisor - Center of Expertise
Tom Wright Customer Support Manager [email protected]
Agenda
Recap
Challenges & Best Practices AWR & STATSPACK SQL Plan Management Real Application Testing Q&A
Lifetime Support Policy
today
August 2012
August 2015
R2
July 2010
July 2013
January 2009
January 2012
Sustaining Support
Premier Support
R2
July 2007
Extended Support
July 2008
July 2010
t
2002 2003 2004 2005 2006 2007
http://www.oracle.com/support/library/brochure/lifetime-support-technology.pdf
2008
2009
2010
2011
2012
2013
2014
2016
2015
Lifetime Support Policy Deliverables
Feature
Major Product and Technology Releases Technical Support Access to Knowledge Base (MetaLink/Customer Connection/SupportWeb) Updates, Fixes, Security Alerts and Critical Patch Updates Tax, Legal and Regulatory Updates Upgrade Scripts Certification with existing Third Party Products/Versions Certification with New Third Party Products/Versions Certification with new Oracle Products
Premier Support
Extended Support
No
Sustaining Support
Pre-existing Only
No No No No No
11g Value Proposition
Easiest to Manage Highest Quality of Service
Highest Performance and Scalability, Fault Tolerant, Most Secure, Best Information
Lowest Risk of Change
Self Managing Database
Auto-Tuning
Advisory Instrumentation
Self Managing Database
Auto-Tuning
Advisory Instrumentation
Documentation
Note: 466181.1 Upgrade Companion 10gR2
Note: 601807.1 Upgrade Companion 11g
Documentation
Note: 601807.1 Upgrade Companion 11g
Documentation
Note: 601807.1 Upgrade Companion 11g
My Oracle Support Configuration Manager
What is it?
A Support capability that automates configuration information exchange between Oracle and our customers enabling faster resolution and proactive detection of customer issues. My Oracle Support utilizes core configuration management capabilities available from Oracle Enterprise Manager and provides: The ability to define configurations and organize projects A view of System details and changes Create, track, and status Service Requests Advanced Knowledge Management capabilities Proactive problem avoidance with HealthChecks Proactive Product and Security Alerts
OCM: The Big Picture
My Oracle Support or MetaLink
Config View Support
SCM
Knowledge Management
Oracle CCR
HTTPS Proxy Server
Simple install/ command UI
Customer Site
What is a system?
Systems are named collections of hardware, software, and operating system running on a machine
Sample System Data being pulled
Application Server level
Configuration files Console configuration DCM configuration Deployed applications General OPMN configuration
JServ level
Configuration files Zones
Database level
Config version Control files Data files Instance information Feature usage High water mark Initialization parameters Session high water mark Options Redo logs Rollback segments SGA Tablespaces Database users Statspack configuration
eBusiness Suite
# of active users Applied patches Custom forms Applications system summary Applications topology Comprising patches Component versions Config version Custom applications Profile options registered under custom app. Custom database objects Cust. Hooks Custom message count Custom objects owned by Apps Custom request sets Responsibility count Custom triggers Custom value sets table validated Custom value sets other Custom workflows Data groups Database version End-Dated users Generic service components parameters High priority patches not applied JTF hooks Localization modules NSL languages Other customizations Application patch levels Printers Printer drivers Profile options Registered Oracle users Technology stack inventory (DB)
HTTP Server level
Cipher suites Configuration files General Listen addresses Performance related settings Virtual hosts
Host level
Operating system initialized services Operating system ULIMITs Hardware CPUs IO devices Network interfaces Operating system Component File system Modules OS parameters Registered software
Infrastructure level
Apps web configuration Apps web configuration files Context files Database configuration Database configuration files Custom context variables Environment context variables Global context variables Host context variables Install context variables Process context variables System context variables Technology stack inventory (file system) Workflow mailer configuration Workflow mailer configuration files
Configuration Manager
Oracle Configuration Manager and Oracle Enterprise Manager You do not need to install Enterprise Manager (EM) in order to use the Oracle Configuration Manager. If EM Agent is already installed, you may also install the Oracle Configuration Manager EM and Oracle Configuration Manager send data to different repositories: Enterprise Manager sends to EM Repository, Oracle Configuration Manager sends to Customer Configuration Repository at Oracle Support. To manage your Oracle system with EM and also send Configuration data back to configuration manager in My Oracle Support or MetaLink, each must be installed on the system.
Service Request Severity Definitions
Severity Level 4 No Business Impact
No loss of service or resources
Severity Level 3 Minor Business Impact
Minor loss of service or resources
Severity Level 2 Serious Business Impact
Severe loss of service or resources w/o acceptable workaround
Severity Level 1 Critical Business Impact
Complete loss of service or resources and work cannot reasonably continue - the work is considered mission critical
Escalation Process
Call US: 800 223 1711
Other Global Support Hot Lines: http://www.oracle.com/support/contact.html
V.P. or Executive
Senior Manager or Director
Manager / Escalation Manager Support Representative Customer
Global Customer Care
Call US: 800 223 1711
Other Global Support Hot Lines: http://www.oracle.com/support/contact.html
Global Customer Care is a highly available expert resource that resolves customers' business-related inquiries while capturing and sharing feedback to enhance the customer relationship.
Responsibilities include but are not limited to:
Access and navigation of Oracle Support websites (including Metalink) Placing software orders Assisting with CSI questions Assisting with product or platform issues related to SR logging Locate and provide published information Create software upgrade orders
Oracle Support Services
http://www.oracle.com/support
Oracle Advisor Webcasts Now Available
Through My Oracle Support and MetaLink!
A new way to receive information about your Oracle EBusiness Suite solutions Go to MetaLink Note 398877.1 - Welcome to the Oracle Advisor Webcast Program!
Includes links to the Advisor Webcast page
Instructions for registering and viewing both live and archived webcasts
The current menu of scheduled webcasts to choose from
Process and Tools Advisor Webcasts
http://www.oracle.com/support/seminars.html
Customer Support Education
Delivery Options One-on-One Customer meetings and Webcasts Support Webcasts targeting a core Customer audience Regularly scheduled free Internet Seminars Regional Support Education Workshops User Groups, SIGs, RUGs, etc. Value to you Following attendance at one of our presentations, Customers have experienced, on average: Up to a 40% reduction in volume of new Service Requests As much as a 70% reduction in issues requiring further escalation A 10% increase in overall Customer Satisfaction
Agenda
Recap
Challenges & Best Practices AWR & STATSPACK SQL Plan Management Real Application Testing Q&A
Challenges & Best Practices
>90% of so-called upgrade problems aren't really upgrade problems but performance issues after the upgrade
It's important to use sufficient test scenarios and methods Typical problem areas: Optimizer - execution plans Queries are slow Reports or batches take longer to complete Increased resource requirements Code path changes
Challenges & Best Practices
Get as much performance information as possible BEFORE you upgrade to the new release - that's of vital importance!!!
Get accurate performance statistics Comparison: before after Measure query and batch timings Test upgrade the database Do functional testing Always use real production data Create a copy of the production database with RMAN Run performance tests with real life loads!!!
Challenges & Best Practices
Possibilities for performance analysis and prediction:
STATSPACK AWR SQL Plan Management Real Application Testing SQL Performance Analyzer Database Replay
Challenges & Best Practices
Optimizer - prevent execution plan changes:
Classical approach:
Rule Based Optimizer (Please note: RBO desupport since Oracle 10g) Hints Stored Outlines Rewriting SQL statements optimizer_features_enabled=n.n.n Change specific optimizer parameters Import and fix object and system statistics
Modern, efficient and better resource consumption:
SQL Plan Management SQL Profiling
Dictionary Statistics
Create dictionary statistics prior to the upgrade - otherwise it will take significantly longer
SQL> EXECUTE dbms_stats.gather_schema_stats
('SYS', options estimate_percent method_opt cascade
=> 'GATHER', => DBMS_STATS.AUTO_SAMPLE_SIZE, => 'FOR ALL COLUMNS SIZE AUTO', => TRUE);
Detailed scripts can be found here: Oracle Database Upgrade Guide Appendix B
Or in
SQL> EXECUTE dbms_stats.gather_dictionary_stats;
Post Upgrade
Create system statistics during a regular workload period otherwise non-appropriate values for the CBO will be used:
SQL> EXECUTE dbms_stats.gather_system_stats('start'); ... SQL> EXECUTE dbms_stats.gather_system_stats('stop');
SQL> select pname NAME, pval1 WERT, pval2 INFO from aux_stats$;
NAME WERT INFO -------------------- ---------- -----------------------------STATUS COMPLETED DSTART 04-07-2008 12:30 DSTOP 05-07-2008 12:30 FLAGS 1 CPUSPEEDNW 1392,39 IOSEEKTIM 11,405 IOTFRSPEED 25595,605 ...
Post Upgrade
Create fixed table statistics soon after the upgrade:
SQL> execute dbms_stats.gather_fixed_objects_stats;
Agenda
Recap
Challenges & Best Practices AWR & STATSPACK SQL Plan Management Real Application Testing Q&A
Automatic Workload Repository
Since Oracle 10g: AWR (Automatic Workload Repository)
Statistics repository in the SYSAUX tablespace Contains AWR snapshots (current and past periods) Requires a license for Diagnostic Pack Command line report:
@?/rdbms/admin/awrrpt.sql
Export the AWR:
begin DBMS_SWRF_INTERNAL.AWR_EXTRACT( dmpfile => 'awr_data.dmp', dmpdir => 'TMP_DIR', bid => 302, eid => 305); end; /
STATSPACK
Database performance tool - pre-10g
Note:394937.1 STATSPACK guide Export the PERFSTAT user
Agenda
Recap
Challenges & Best Practices AWR & STATSPACK SQL Plan Management Real Application Testing Q&A
What Starwood Hotels & Resorts Worldwide Is Saying
"The Database Replay and SQL Plan Management features alone are worth upgrading to Oracle Database 11g "
Arup Nanda Senior Director of Database Engineering and Architecture Starwood Hotels and Resorts Worldwide
Challenge
How do I ensure plan stability???
Without SQL Plan Management
Challenging to "freeze" execution plans and statistics Difficulty:
Statement has been parsed and a plan got created. Verification happens during execution:
GB
Parse
HJ
Execute
Plan acceptable
HJ
Now some conditions get changed (e.g. upgrade) A new plan will be created - is it better or worse???
GB
Parse
NL
Execute
Plan possibly not acceptable
NL
SQL Plan Management
First preventive and fully transparent database mechanism to ensure plan stability SQL execution plans will be recorded A SQL Baseline will be created
Consists of accepted execution plans Contains only plans for statements being parsed/executed more than once
Only known and verified and accepted plans will be used Package: DBMS_SPM
SQL Plan Management
3 phases for plan stability:
Capture Create a SQL Baseline representing trusted execution plans Baseline is stored in SQL Management Base in tablespace SYSAUX Selection Only accepted plans will be used New execution plans will be recorded in the plan history Evolution Evaluate all unverified plans for a given statement in the plan history to become either accepted or rejected
SQL Plan Management
White Paper:
http://www.oracle.com/technology/products/ manageability/database/pdf/ow07/ spm_white_paper_ow07.pdf
SQL Plan Management
Phase 1 - Capture
OPTIMIZER_CAPTURE_SQL_PLAN_BASELINE=TRUE (Default: FALSE)
GB
Parse
HJ
Execute
Plan acceptable
HJ
Initial plan will be accepted if repeated and will be added to the SQL Plan Baseline
SQL MANAGEMENT BASE
Plan History Residing in SYSAUX TS. Will occupy max. 10% of SYSAUX. Weekly job will delete plans not used since 53 weeks [default].
Plan Baseline
GB HJ HJ
SQL Plan Management
Phase 2 - Selection
OPTIMIZER_USE_SQL_PLAN_BASELINE=TRUE
(Default: TRUE)
Same statement parsed again but a different plan will be created
GB
Parse
NL
NL
New plan will be added to the Plan History but it won't be used unless it has been verified
Plan History
GB NL NL
Plan Baseline
GB
HJ HJ
SQL Plan Management
Phase 2 - Selection
OPTIMIZER_USE_SQL_PLAN_BASELINE=TRUE
GB
Parse
HJ
Execute
Plan known and acceptable
HJ
The optimizer will use only one of the VERIFIED plans stored in the SQL Baseline because only these will guarantee PLAN STABILITY
Plan History
GB NL NL GB HJ HJ
Plan Baseline
SQL Plan Management
Phase 3 - Evolution
Plan History
GB GB NL NL HJ HJ NL NL
Plan Baseline
GB
Worse plan will be kept in the Plan History and deleted automatically
Plan History
GB NL NL GB NL
Plan Baseline
GB HJ NL HJ
Equal or better plans can be added to the SQL Plan Baseline DBA schedules verification
DBA
Optimizer
SQL Plan Management
DatabaseControl:
SQL Plan Management
DatabaseControl - Configuration:
Start with 5-14 weeks: exec DBMS_SPM.CONFIGURE('plan_retention_weeks',5);
SQL Plan Management
DatabaseControl - details:
SQL Plan Management
DatabaseControl - evolve:
SQL Plan Management - Scenarios
Upgrade scenario 1:
Parameterize the optimizer back to the "old" behaviour Works for all database releases since 8.0.3
Upgrade scenario 2:
Transport all well known plans to the new release Works since 10gR2
Ship a new application (module) along with appropriate SQL plan baselines:
Ship the best execution plans for the new statements within a staging table and ensure that they'll be used Works since 10gR2
SQL Plan Management - Upgrade 1
Upgrade scenario 1
STS
Repeatable plans will be added to the Plan Baseline upon 2nd execution
Plan History
GB
Plan Baseline
GB NL NL HJ GB HJ GB NL NL
NL NL
Now: Different plans created with OFE=11 will be added to the Plan History for later verification
SQL Plan Management - Upgrade 2
Upgrade scenario 2
STS
Table Staging
exp imp expdp impdp DB-Link ...
STS
10.2 plans will be added to the SQL Plan Baseline
Plan History
GB
Plan Baseline
GB NL NL HJ GB HJ GB NL NL
NL NL
Every new better plan found in 11g will be added to the Plan History
SQL Plan Management - New Application
New application (module) gets delivered @Vendor @Customer
DBMS_SPM.CREATE_STGTAB_BASELINE Staging Staging
Table
exp imp expdp impdp
Table
DBMS_SPM.UNPACK_STGTAB_BASELINE DBMS_SPM.PACK_STGTAB_BASELINE
Plan Baseline
GB NL NL HJ GB HJ GB
Plan Baseline
GB NL NL NL HJ GB HJ GB NL NL
NL
Agenda
Recap
Challenges & Best Practices AWR & STATSPACK SQL Plan Management Real Application Testing Q&A
Real Application Testing
Goal: Enable successful adoption of new technology and features specifically new releases of the database - at low-cost and low-risk by:
Record and replay a real workload to see how the new system performs Finding regressions and changing plans before the upgrade
Licensable database pack "Real Application Testing"
Available with Oracle Database 11.1.0.6 Available with patch set 10.2.0.4 Available as single patch for 9.2.0.8 and 10.2.0.3 For patch numbers please see Note:560977.1 Can be used for upgrades from 9.2.0.8 to 10.2.0.3/4 9.2.0.8 to 11.1.0.x 10.2.0.3/4 to 11.1.0.x
Real Application Testing
Real Application Testing consists of:
Database Replay
Package DBMS_WORKLOAD_CAPTURE Package DBMS_WORKLOAD_REPLAY
SQL Performance Analyzer (SPA)
Package DBMS_SQLPA
SQL Tuning Sets (STS)
Package DBMS_SQLTUNE
Real Application Testing
White Paper:
SQL Performance Analyzer: Techinical White Paper: SQL Performance Analyzer
http://www.oracle.com/technology/products/manageability/database/pdf /ow07/spa_white_paper_ow07.pdf
Database Replay: Techinical White Paper: Database Replay
http://www.oracle.com/technology/products/manageability/database/pdf /ow07/db_replay_white_paper_ow07_1.pdf
Real Application Testing: Tools of the Trade
SQL Performance Analyzer
What is it? Predicts SQL performance deviations before end-users can be impacted, helps assess impact of change on SQL response time Executes each SQL, stored in SQL Tuning Set, in isolation using production context and then compares before and after execution plans and run-time statistics Unit testing of SQL with the goal to identify the set of SQL statements with improved/regressed performance
SQL
Database Replay
Replays real database workload on test system, helps assess impact of change on workload throughput Captures workloads and replays it with production characteristics including concurrency, synchronization & dependencies Comprehensive testing of all sub-systems of the database server using real production workload
SQL Concurrency
How it works?
When to use?
Dependency
Speed up/down
Real Application Testing
Database Control:
Agenda
Recap
Challenges & Best Practices AWR & STATSPACK SQL Plan Management Real Application Testing Q&A
SQL Performance Analyzer Database Replay
Challenge
How do I find out which plans will be changed? How do I verify parameter changes?
Common SPA Usage Scenarios
Common SPA usage scenario examples
Database upgrades, patch set changes Parameter changes Schema changes Optimizer statistics refresh, etc.
SPA interface
Rich GUI through Enterprise Manager Command line interface through DBMS_SQLPA PL/SQL package
SQL Performance Analyzer: Overview
Production
Client
Test
Re-execute SQL
Middle Tier
Capture SQL
Oracle DB
Transport SQL
Storage
Make Changes / Tuning Regressions
If adequate spare cycles available, optionally execute SQL here
* No middle & application tier setup required
SQL Performance Analyzer: Workflow
Production Test
Make Change
Steps (1) (2) (3) (4) (5)
Capture SQL (STS)
Transport STS
Execute SQL Pre-change
Execute SQL Compare Post-change Perf.
(6)
Reiterate
No
Done?
(7) Production Change / (7) Tuning Deployment Tuned System
Yes
SPA Workflows
Possible workflows:
Optimizer upgrade simulation (Start: 8.0.0) Init parameter change Guided workflow for manual changes
All are based on SQL Tuning Sets
Database Upgrade: 10g to 11g
Scenario 1: I want to upgrade from 10.x to 11.x database release to benefit from 11g functionality. How can I best accomplish the upgrade?
Goal: Assess impact of upgrade on SQL workload performance so that there are no surprises after upgrade. Once upgraded to 11g new features can be enabled one at a time.
Database Upgrade: 10g to 11g
Step 1:
Capture workload into STS: Preferred method (> 10.2.0.1)
EXEC DBMS_SQLTUNE.CAPTURE_CURSOR_CACHE_SQLSET( .. )
Create STS from Top SQL in AWR / AWR Baseline
For 9i database enable SQL tracing:
alter system set events '10046 trace name context forever, level 4';
Database Upgrade: 10g to 11g
Step 1 in EM:
Database Upgrade: 10g to 11g
Step 2: Upgrade a copy of 10g production to 11g test database Step 3: Run SPA in 11g test database Select Optimizer Upgrade workflow link
Database Upgrade: 10g to 11g
Step 3 in EM: Run SPA task
Database Upgrade: 10g to 11g
SPA results
3 2 4 1
Database Upgrade: 10g to 11g
After using SQL Plan Management with a baseline
Parameter Changes
SPA Recommendation: SQL Tuning Advisor
1 3
Parameter Changes
SQL Tuning Advisor offers SQL Profiles
Parameter Changes
SPA run after SQL Profiles had been applied
SQL Performance Analyzer
SPA: SQL statement details example
SPA for a 9i workload against 11g
Upgrade scenario from 9i directly to 11g
See Note: 562899.1 for the White Paper and Note:455889.1 for a SPA example
Event 10046
ORCL_ora_133674.trc ORCL_ora_133674.trc ORCL_ora_133674.trc ORCL_ora_133674.trc ...
copy
STS
SQL Performance Analyzer Mapping Table
Id
123 124
Owner
SH1 SH2
Name
SAL1 CUST
exp / imp
Agenda
Recap
Challenges & Best Practices AWR & STATSPACK SQL Plan Management Real Application Testing Q&A
SQL Performance Analyzer Database Replay
Challenge
How will my system perform after an upgrade? Will all application calls work in 11g?
Database Replay
Capture and replay an actual production database workload Identify, analyze and fix potential instabilities before making changes to production Capture Workload in Production
Capture full production workload with real load, timing & concurrency characteristics Move the captured workload to test system
Replay Workload in Test
Make the desired changes in test system Replay workload with full production characteristics Honor commit ordering
Analyze & Report
Errors Data divergence Performance divergence
Step 1: Workload Capture
All external client requests captured in binary files System background and internal activity excluded Minimal overhead
Avoids function call when possible Buffered I/O
Production System
Client
Client
Client
File System
Middle Tier
Independent of client protocol Can capture on 9.2.0.8 and replay on 11g Capture load for interesting time period, e.g., peak workload, month-end processing, etc.
Storage
File 1
File 2
File n
Step 2: Process Workload Files
Setup test system
Application data should be same as production system as of capture start time Use RMAN, Snapshot Standby, imp/exp, Data Pump, etc. to create test system Make change: upgrade db and/or OS, change storage, migrate platforms, etc.
File 1 File 2 File 2
Test System
File 1
File n
File n
Pre-processing transforms captured data into replayable format Once pre-processed, workload can be replayed many times
Metadata Capture Files Replay Files
Step 3: Replay Workload
Replays workload preserving timing, concurrency and dependencies of the captured system Replay Client is a special program that consumes processed workload and sends requests to the replay system For high concurrency workloads, it may be necessary to start multiple clients
Test System
Replay Clients
File 1 File 2
File n
Metadata
Replay Files
Database Replay
Step-By-Step
Database Replay
1. Create a copy of the database to replay the workload
RMAN Duplicate Snapshot Standby
Database Replay
2. Workflow in EM: Workload Capture
Database Replay
2. Preparation steps in EM: Workload Capture
Database Replay
3. Workload Filter:
Database Replay
4. Prepare an empty directory for capture files
Database Replay
Job scheduling - specify time frame for capture:
Database Replay
Database Control - capture review and submit:
Database Replay
Database Control - capture summary:
Database Replay
Database Control - capture statistics:
Database Replay
Preprocessing captured workload:
Database Replay
Replaying the captured workload:
Database Replay
Choose directory containing preprocessed capture files
Database Replay
Workload Replay - choose replay name:
Database Replay
Workload Replay: Wait for replay clients to connect
Database Replay
Workload Replay Clients: Calibrate
$> wrc mode=calibrate replaydir=/tmp/testing
Workload Replay Clients: Start
$> wrc system/oracle mode=replay
Database Replay
Workload Replay: WRC client(s) connected
Database Replay
Monitor Workload Replay during replay
Database Replay
Monitor divergence stats:
Database Replay
Workload Replay result:
Database Replay
Workload replay reporting:
Important Upgrade Notes at a glance
Upgrade to Oracle 10gR2
Note: 466181.1 Upgrade Companion 10gR2 Note: 316889.1 Complete Checklist for Manual Upgrade to 10gR2 Note: 555579.1 Known Issues 10.2.0.4
Upgrade to Oracle 11g
Note: 601807.1 Upgrade Companion 11g Note: 429825.1 Complete Checklist for Manual Upgrades to 11g Note: 454506.1 Known Issues and Alerts 11.1.0.6
Performance Testing
Note: 560977.1 Real Application Testing available for earlier releases Note: 562899.1 Using SQL Performance Analyzer for upgd. 9.2 to 10.2 Note: 394937.1 Statspack Guide Note: 295819.1 Potential Query Tuning Related Issues Note: 345048.1 GROUP BY does not sort if you don't use ORDER BY Upgrading to Oracle Database 10g: What to expect from the Optimizer
Upgrade Webpage on OTN
http://www.oracle.com/technology/products/database/oracle11g/upgrade/index.html
Latest Patchset 11.1.0.7.0
Patch Number 6890831 Available for all major platforms
Agenda
Recap
Challenges & Best Practices AWR & STATSPACK SQL Plan Management Real Application Testing Q&A
