1 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Oracle Confidential
Database Consolidation
Demystified: Plan, Test and
Consolidate
Ashish Agrawal
Consulting Product Manager
Kurt Engeleiter
Principal Product Manager
John Kanagaraj
Database Engineering, PayPal
2 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
 Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied
upon in making purchasing decisions. The development, release, and timing of any
features or functionality described for Oracle’s products remains at the sole discretion
of Oracle.
This document in any form, software or printed matter, contains proprietary information
that is the exclusive property of Oracle. Your access to and use of this confidential
material is subject to the terms and conditions of your Oracle Software License and
Service Agreement, which has been executed and with which you agree to comply.
This document and information contained herein may not be disclosed, copied,
reproduced or distributed to anyone outside Oracle without prior written consent of
Oracle. This document is not part of your license agreement nor can it be
incorporated into any contractual agreement with Oracle or its subsidiaries or affiliates.

3 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
  Introduction
Program
Agenda  Database Consolidation Strategies
 Consolidation Planning
 Consolidation Validation
 Capacity Planning: Advanced Use
Cases
 Customer Experiences

4 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
 Why Consolidate?
• Reduce complexity
– Servers running standard configurations
• Improve efficiency
– Drive up hardware utilization rates
• Lower costs
– Hardware/Software
– Energy and Floorspace
• Simplify maintenance
– Fewer servers in fewer locations
• Better security
– Smaller security perimeter

5 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
 DB Consolidation Challenges
 What consolidation strategy to use?
 How to test the chosen consolidation
strategy?
 Can the system handle peak workloads,
can workloads co-exist together?
 How to minimize consolidation risk?

6 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Database Consolidation
Strategies

7 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
 Server Consolidation
Consolidate Database in a VM
• Reasons for adoption
• Simple to implement
• Excellent isolation
• Mixed workloads
DW CRM ERP
• As-is consolidation
• Legacy support

DB

DB

DB
• Potential concerns OS OS OS
• Lower consolidation density Hypervisor
• Lower ROI
• Performance (latency)
• Managing sprawl
• Not suitable for all deployments

8 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Oracle Confidential
 OS Consolidation
Consolidate Database on Shared OS Platform
• Reasons for adoption
• Consolidation density
• Good ROI
• Performance DW ERP CRM
• Supports any app

DB

DB

DB
• Potential concerns
• Requires OS standardization OS OS
• Database only

9 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential © 2011 Oracle Corporation
 Database Consolidation
Consolidate Schema in Shared Database
• Reasons for adoption
• Efficient
• Fast provisioning
• Good ROI
DW ERP CRM
• Performance
• Potential concerns

DB
• App qualification required
OS OS
• Requires OS and DB standardization
• Isolation

10 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
 Next Generation Consolidation Technology
Pluggable Database
• Virtual databases with
their own:
Root – Schemas
– Tablespaces
– Data dictionary
PDB SEED ERP DB DW DB CRM DB – Synonyms
– Users
– Roles
– Services

• Ideal for consolidation

11 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
 Pluggable Database Consolidation
Consolidate Database as Pluggable Database
• Reasons for adoption
• Most efficient
• Extremely fast provisioning DW ERP CRM
• No application changes
PDB PDB PDB
• Best ROI
• Performance
• Efficient memory use
CDB
• Potential concerns
• Requires OS and DB standardization
• Misbehaved PDB

12 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
 Summary

Server OS Schema Pluggable

Business Value
Consolidation Consolidation Consolidation Database

Implementation Easy Easy Difficult* Easy

Isolation Highest High Limited High
Scalability Limited Limited High High
Performance Low Low High High
Resource mgmt Fair Fair Limited Excellent

Consolidation Density Low Medium High High

ROI Low Low High Highest

* Need to validate application schemas can co-exist
13 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Consolidation Planning

14 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
 Consolidation Suitability
120% 120% 120%
100%
80%
100%
80%
100%
80%
• Applications
60%
40%
60%
40%
60%
40%
have different
20%
0%
20%
0%
20%
0% workload
Application profiles
  
Application Combined
A CPU B CPU CPU

• Multiple
Utilization Utilization Utilization

resources need
120% 120% 120% to be analyzed
100% 100% 100%
80% 80% 80% – CPU
60% 60% 60%
40% 40% 40% – Memory
20% 20% 20%
0% 0% 0% – Storage

 
Application Application Combined
– Network

A CPU B CPU CPU
Utilization Utilization Utilization

15 Copyright © Oracle
2011, Oracle
15 Copyright © 2012, and/or and/or its affiliates.
its affiliates. All reserved.
All rights rights
OracleInsert Information Protection Policy Classification from Slide 8
Confidential
reserved.
Enterprise Manager Consolidation Planner

• Identifies under-utilized or over-utilized servers

• Helps administrator determine optimal solution for

consolidation

• Works for physical and virtual environments

16 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
 Consolidation Planner
• Leverages resource utilization and
configuration data from Enterprise
Manager repository
– CPU, memory, storage, network
– Over a representative period
• Administrator specifies servers and
constraints for workload migration
– Physical/virtual servers
– Existing/planned servers
– Business/technical constraints
• Detailed analysis on different scenarios
of consolidated workloads

17 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Consolidation Report
• Consolidation Scenario Report
available after running analysis:
• Provides details on:
– Consolidation ratio
– Destination server utilization
– Source to Target server mapping
– Confidence of meeting requirements
based on historical samples
– Manually mapped consolidation
constraint violations
– Servers that cannot be consolidated
(Exceptions)

18 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Consolidation Migration

19 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Consolidation Migration
• Server Consolidation
– Use VM OS migration tools
– Or do a fresh OS install and RMAN duplicate, replication or
export import
• OS Consolidation
– Use RMAN duplicate, replication or export import
• Schema Consolidation
– Resolve name space clashes
– Use schema level export import
• Pluggable Database
– Pre-12c use transportable tablespace, RMAN duplicate,
replication or export import
– 12c and above – plug and unplug

20 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
 Pluggable Database Migration
Consolidation of pre-12c Databases

• Provision empty PDBs

Root
per database that need to
be consolidated
• Use datapump or
Empty
ERP Empty
DW Empty
CRM
PDB SEED replication to migrate a
ERP PDB
PDB DW PDB
PDB CRM PDB
PDB
Database into a PDB
Data Pump Replication

11.2 10.2 11.2

ERP DB DW DB CRM DB

21 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
 Pluggable Database Migration
Consolidation of 12c Databases

Root • Upgrade an existing pre-

12c database to 12c
• Plug-in the database post
upgrade into a CDB
PDB SEED

12.1 12.1 12.1

ERP DB DW DB CRM DB

22 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Consolidation Validation

23 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Consolidated Database Performance Testing
End-to-end testing with real workloads
Create Test
Capture System
Workload

Replay
Workload Deploy Replay
Clients

Consolidation Type Tool Performance Test

Server, OS, Schema, CDB SQL Performance Analyzer SQL unit testing for response time
Server, OS Database Replay Load, performance testing for throughput

Schema, CDB Consolidated Database Replay Replay multiple workloads against one
database for throughput and scalability

24 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
SQL Performance Analyzer (SPA)

SQL Plans + Run-time Stats

Pre-Change Trial

SQL Workload Analysis Report

STS Compare SQL
SQL Plans + Run-time Stats Performance
Post-Change Trial

• Tests and predicts impact of system changes on SQL query performance

• Analyzes performance changes for improvements or regression
• Server, OS, CDB consolidation: run multiple SPA trials in parallel
• Schema Consolidation: merge SQL Tuning Sets, run trial

25 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
 Consolidated SQL Performance Analyzer
Maximize ROI while Reducing Effort and Risk

• Validates SQL
SALES performance for
SALES STS
consolidated database

• SQL workload captured

HR
HR STS for each database in STS
HR
SALES
ERP
CRM
• SPA executes all
ERP ERP STS
workloads together in
consolidated environment

• Identifies SQL
CRM CRM STS
regressions and helps
remediate them

26 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Database Replay
 Database load and performance
Production Test
testing with real production workloads Clients Replay Driver
– Production workload
characteristics such as timing,
transaction dependency, think
time, etc., fully maintained
 Test and measure transaction
throughput improvements
 Identify application scalability and
concurrency problems
 Use for server and OS consolidation
– Capture individual workloads Storage Storage
– Replay in parallel Analysis &
Capture Process Replay
Reporting

27 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Consolidated Database Replay
 Workload captured on different
Production Test
Clients Replay Driver
databases can be replayed
concurrently
 Works for schema consolidation and
Pluggable Databases
 Identify and remediate inter-application
scalability and concurrency problems
 Allows scaleup, subsetting, scheduling
of multiple workloads
 Use for schema and CDB
consolidation Storage Storage
 Available for 11.2.0.2 and above, MOS
Analysis &
Note: 1453789.1 Capture Process Replay
Reporting

28 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Consolidated Database Replay
Test System
Consolidated Replay Directory
Production Systems Directories
DB1 DB2 DB3 Capture 1 Capture 2 Capture 3

Windows
DB 10.2
AIX
DB 9.2.0.8
HP-UX
DB 11g
… Replay on CDB OR Non-CDB
CDB – DB12c
Non-CDB (11.2.0.2+)
Datafiles Control Redo Log files
files
Directories

Capture 1 Capture 2 Capture 3

root
May: Month-end June15: Daily June 18: DW -
Close Peak ETL
Datafiles Datafiles Datafiles

MFG PDB CRM PDB HR PDB

29 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Consolidated Database Replay
Container Database Support
Container Database (CDB)
 Workload Capture
Datafiles Control files Redo Log
– Capture container database level files

– Capture filters (include/exclude) can

specify multiple PDB targets by
 PDB Name root
 Service Name
Datafiles Datafiles Datafiles
 Workload Replay
– Multiple workload captures can be replayed
against PDBs by re-mapping capture
connections to services MFG PDB CRM PDB HR PDB

30 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
 EM Cloud Control 12c: Create Concurrent
Capture Wizard Workflow

31 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
 EM Cloud Control 12c: Create Concurrent
Replay Task Page

32 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
 EM Cloud Control 12c: Consolidated Database
Replay - Summary Page

33 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
 EM Cloud Control 12c: Consolidated Database
Replay - Reports

34 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
When to Use Each Tool?
• SQL Performance Analyzer for all consolidation cases
– Server, OS, Schema, CDB consolidation
– Important to verify individual SQL perform well in new
environment
• Database Replay
– Server and OS consolidation
– Verify target environment can handle consolidated load
• Consolidated Replay
– Schema and CDB consolidation
– New UI workflows simplify orchestration of load testing in
new environment

35 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Capacity Planning: Advanced
Use Cases

36 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Capacity Planning: Advanced Use Cases
• Will the consolidated system be able to
handle future workload growth?
• Three use cases:
– Time-shifting
– Workload Folding
– Schema Remapping

37 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Peak Workload Testing Through Time-Shifting
 Goal
– Exercise worst
case scenario
where workload
peaks line up
– Evaluate and
experience fallout
from the safety of
a test system

38 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Peak Workload Testing Through Time-Shifting

 Similar to consolidated replay, use parameters of add_capture() to create desired schedule for
each workload to line-up peaks
 API example: To delay a workload replay 2 hrs from start of replay:
– exec select dbms_workload_replay.add_capture('cap2’, 2* 60*60 ) from dual;

39 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
 Workload Folding
 High-Level Business Use Case
– Your company expects 2X more SALES workload during holiday season and
wants to test at this scale. CIO wants to make sure that the system being deployed
can handle this workload without performance and stability issues
 Scale-up Scenario: Assumptions
– Individual SALES transactions largely independent
– Initial DB state not very crucial to recorded transactions

 Solution
– Split a typical peak holiday 8h period into 2 workloads
– Schedule the 2 workloads to run simultaneously
– Restore the DB to the SCN at the beginning of capture. Second part of the
workload will operate on earlier than strictly required versions of data.
– For certain workloads this should be acceptable - mainly NEW sales are expected
and some replay divergence is acceptable

40 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
 Workload Folding

8 9 10 11 12 13 14 15 16

 Split workload into 2 subsets relative to capture  Schedule replay of the two workload
start time, 0-24 hr capture subsets to start at the same time (no
delay)
– 8am-12pm
 Replay steps are the same as for
– 12pm-16pm
consolidated replay previously covered

41 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Scale-up with Schema Remapping
 Useful for scale-up
testing when multiple
instances of the same
application are to be
SALES deployed
– Adding a new LOB or
geographical area
– Multi-tenant
applications
– DB Schema As a
Service

42 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
 Scale-up With Schema Remapping

SALES01

SALES

SALES02

 Setup Test system with multiple schemas from the initial schema using any of the supported
mechanisms (Data Pump, PDB migration, etc.)
 Copy workload multiple times into consolidated replay directory and remap users to different schemas
to perform consolidated replay
 API support: Example: EXEC DBMS_WORKLOAD_REPLAY.SET_USER_MAPPING (1001, ‘SALES',
SALES01’);
 This strategy can identify hardware, host bottlenecks, results in minimal divergence

43 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Customer Experiences

44 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Consolidated DB Replay in Oracle
Database 11gR2
John Kanagaraj, PayPal
 Speaker Qualifications
• Been working with Oracle Databases and
UNIX for too many years 
• Author and Technical editor
• Frequent speaker at OOW, IOUG
COLLABORATE and regional OUGs
• Oracle ACE
• Associate Editor, IOUG SELECT Journal
• Currently Database Engineer @ PayPal
• Loves to mentor new speakers & authors!
 The Case for RAT – Covering your risk of change
• Technology changes/upgrades rapidly
– Precipitated by DC moves, Capacity concerns,
Hardware/Software EOL, Org changes, etc.
• Database Consolidation usually driven by
– Cost: Many “miscellaneous” applications/DBs
• Not just “hard” costs!
– Server/DB sprawl: Optimistic/pessimistic models
• Data Center/Power costs are significant
• Upgrade testing is essential… but hard to do
– Load test prep, setup and execution is tedious
– Load testing may not cover all scenarios
• Note: A chain is as weak as the weakest link
 Consolidated Replay in 11.2.0.2
• Real Application Testing first available in Oracle DB 11.1.x
• Product maturity in Oracle Database 11gR2 – 11.2.0.2
• Consolidated Replay available in 11.2.0.2/0.3
• MOS Note 1453789.1 / Patch 13947480
– Patch needs to be requested; conflicts to be resolved
• Useful for testing consolidation of multiple schemas into a single
database – Now!!
– Addresses DB/Server sprawl
– Splices out “interesting” periods in long capture
– Provides for multiple “what if” scenarios
– Ability to “fold workload”
 Use cases: Consolidated Replay in 11.2.0.2
• Consolidating schemas/databases
– Understand application profiles: Trend from AWR and OS
– For Oracle Service based connections, use data from
DBA_HIST_SERV_STAT, DBA_HIST_SERVICE_WAIT_CLASS
– Use Consolidated Replay to validate!
• Using a long capture / “Interesting” periods
– Long captures => (Usually) required for better captures
– But… a Replay is always at least as long as a Capture
– Splicing out “interesting” periods in capture now possible
• “Folding” workload – Test scaling up Writes
– Previous version scaled up Read queries only
– Use two versions of same schema to double write workload
– Flush out Redo Writer/Redo log writer rate and other issues
 Best Practices: Consolidated Replay
• Understand your applications/schemas!
– Load, Batch and access patterns
– External/Internal application dependencies
• Backup source data just prior to capture
• Generate SQL Sets on source DB for SQL analysis
• Provide adequate disk space for capture files
• Provide realistic Test servers with DB Flashback
• Isolate the Test servers
• Run individual replays before consolidated replay
• Use SQL Performance Analyzer to fix regressing SQL
51 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Oracle Confidential
Oracle Confidential