What's New with

Oracle Data Pump

in Oracle Database 12c
Mike Dietrich
Senior Principal Technologist
Database Upgrades
Oracle Corporation

Updated: 4-APR-2014
 What's New with Oracle Data Pump in Oracle 12c

Introduction

Full Transportable Export

Views-as-Tables

Timestamps

Compression

Transform

Wrap Up

2 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 $> whoami
6 years
RDBMS Core & MissionCirtical Support

6 years
Technology Presales for DataGuard, Upgrades

Mike Dietrich
6 years
ST Upgrade Development Team
50% 50% + x%
Reference Workshops Development
Projects Worldwide Work

3 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Reference Involvement

4 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Reference Involvement: Results

>16000 Downloads
since June 2013

5 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Slides and Dates?

Download slides from:

– blogs.oracle.com/UPGRADE

6 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Data Pump Overview

The “new” faster export-import
– Available starting with Oracle 10.1
– Powerful concept:
– Master Note for Data Pump:
MOS Note:1264715.1
– For Compatibility and version changes:
MOS Note:553337.1

7 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Data Pump Best Practices

For full exports:
– Role EXP_FULL_DATABASE is required

For export consistency use:

– FLASHBACK_TIME=SYSTIMESTAMP

This will increase UNDO requirements for the duration of the export

Always set parameters:

– EXCLUDE=STATISTICS
– METRICS=YES

8 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Data Pump Best Practices

Speed up Data Pump:
– Let Data Pump create the objects
– PARALLEL=n

Typically n = 2x <number of CPU cores>
– EXCLUDE=STATISTICS on export
– EXCLUDE=INDEXES on import
1. Initial impdp with EXCLUDE=INDEXES
2. Second impdp with INCLUDE=INDEXES SQLFILE=indexes.sql
3. Split indexes.sql into multiple SQL files and run in multiple sessions
– Set COMMIT_WAIT=NOWAIT and COMMIT_LOGGING=BATCH during full imports

9 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Data Pump Best Practices

Direct import via database link
– Parameter: NETWORK_LINK

Run only impdp on the target system - no expdp necessary

No dump file written, no disk I/O, no file transfer needed

Restrictions of database links apply:

– Does not work with LONG/LONG RAW and certain object types

Performance: Depends on network bandwidth and target's CPUs

$ expdp ...

10 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Data Pump Best Practices

Real World Case:
Kaiser Permanente, Medicare (USA)
– impdp on NETWORK_LINK with
8 vs 16 CPU cores

10GBit connection leveraged up to 8 Gbit

1 TB table copied in ~15 min  4 TB/hour
– Network bandwidth and CPU bound

11 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 What's New with Oracle Data Pump in Oracle 12c

Introduction

Full Transportable Export

Views-as-Tables

Timestamps

Compression

Transform

Wrap Up

12 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Full Transportable Export

Full Transportable Export Database Link

PDB$
PDB1

Combining: SEED impdp

– Transportable Tablespaces
– Data Pump for manual steps
CDB
– Optional: Incremental Backups

Souce: Oracle 11.2.0.3 or newer Read Only

Target: Oracle 12.1.0.1 or newer

Works with or without Multitenant DB1

13 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Traditional Transportable Tablespaces
Rebuild meta information
(views, synonyms, trigger, roles etc)

SOURCE Database 10.2.0.4 DESTINATION Database 11.2.0.3

VIEWS VIEWS

Read Only
SYSTEM CODE
SYSTEM CODE

APP SYSAUX APP SYSAUX

UNDO PRIVS UNDO PRIVS

HUGO TEMP HUGO TEMP

14 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Full Transportable Export/Import with Copies

SOURCE Database 11.2.0.3 DESTINATION Database 12.1.0.1

VIEWS VIEWS

Read Only
SYSTEM CODE SYSTEM CODE

APP SYSAUX APP SYSAUX

Data Pump UNDO
UNDO PRIVS PRIVS

HUGO TEMP HUGO TEMP

15 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Full Transportable Export/Import with Backups

Convert and apply

backups

SOURCE Database 11.2.0.3 DESTINATION Database 12.1.0.1

VIEWS VIEWS

Read Only
SYSTEM CODE SYSTEM CODE

APP SYSAUX APP SYSAUX

Data Pump UNDO
UNDO PRIVS PRIVS

HUGO TEMP HUGO TEMP

16 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Full Transportable Export/Import into Multitenant

Create a fresh database/PDB

Create database link to source PDB$ Database Link
PDB1
SEED impdp

Tablespaces read-only – downtime!

Copy datafiles to destination CDB

Run impdp on NETWORK_LINK
impdp oow/passwd@PDB1
Read Only
NETWORK_LINK=DB1 VERSION=12 FULL=Y
TRANSPORTABLE=ALWAYS METRICS=Y
LOGFILE=oow_dir:src112fullimp.log DB1
TRANSPORT_DATAFILE='/oradata/ts1.dbf' …

17 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Step by Step: Full Transportable Export/Import

Incremental Backups
 – Enable archive logging in source:
ALTER DATABASE ARCHIVELOG;
 – Enable block change tracking in source:
 ALTER DATABASE ENABLE BLOCK CHANGE TRACKING USING FILE '<name>' REUSE;
 – Recovery area should be accessible by target instance
– Simple RMAN script:
BACKUP INCREMENTAL LEVEL 1 FOR RECOVER OF COPY WITH TAG 'xyz' DATABASE;
RECOVER COPY OF DATABASE WITH TAG 'xyz';

First execution creates level 0 copy of all datafiles

Subsequent invocations backup and apply only changed blocks to the datafiles since previous invocation

18 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Step by Step: Full Transportable Export/Import

Incremental Backups

Setup steps in the new database/PDB
 – Create a directory object
CREATE DIRECTORY xyz_dir AS '/reco/data';
 GRANT READ, WRITE ON DIRECTORY xyz_dir TO mike;
 – Create a database link back into source instance
CREATE PUBLIC DATABASE LINK v112 USING 'v112';

User who runs impdp on target must exist on source with DATAPUMP_EXP_FULL_DATABASE role

19 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Step by Step: Full Transportable Export/Import

Incremental Backups

Setup steps in the new database/PDB

Transport Phase
 – Set all data tablespaces read-only on source
ALTER TABLESPACE users READ ONLY;
 >>>>> DOWNTIME <<<<<

– Final execution of the RMAN backup and apply scripts

BACKUP INCREMENTAL LEVEL 1 FOR RECOVER OF COPY WITH TAG 'xyz' DATABASE;
RECOVER COPY OF DATABASE WITH TAG 'xyz';

20 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Step by Step: Full Transportable Export/Import

Incremental Backups

Setup steps in the new database/PDB

Transport Phase

Data Pump at Work
 – impdp mike/<passwd>@V112 NETWORK_LINK=v112
VERSION=12 FULL=Y TRANSPORTABLE=ALWAYS
METRICS=Y LOGFILE=xyz_dir:v112fullimp.log
TRANSPORT_DATAFILES='/reco/data/user1.dbf'

21 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Step by Step: Full Transportable Export/Import

Incremental Backups

Setup steps in the new database/PDB

Transport Phase

Data Pump at Work

Clean Up
– Check the logfile
– Target tablespaces  read-write
– Source tablespaces  read-only

22 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 What's New with Oracle Data Pump in Oracle 12c

Introduction

Full Transportable Export

Views-as-Tables

Timestamps

Compression

Transform

Wrap Up

23 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Exporting Views as Tables

VIEW1 expdp system/mgr

views_as_tables=scott.view1 ...

TABLE1
impdp system/mgr
remap_table=view1:scott.table1 ...

TABLE1

24 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Exporting Views as Tables

Parameter:
– VIEWS_AS_TABLES=[schema_name.]view_name[:table_name],

One or more views are to be exported as tables

– Exports a table with the same columns as view
– Row data is fetched from the view
– Dependent objects also exported, such as grants and constraints

TABLE_NAME: Template table serves as source of metadata
– Required if the database is read-only

25 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Exporting Views as Tables
Example: export
SQL> CREATE VIEW hr.my_view (first, last, dept) AS
select e.first_name, e.last_name,
d.department_name
from employees e, departments d
where e.department_id = d.department_id;

>$ expdp system/manager views_as_tables=hr.my_view …

Processing object type TABLE_EXPORT/VIEWS_AS_TABLES/TABLE
. . exported "HR"."MY_VIEW" 8.570 KB 106 rows

26 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Exporting Views as Tables
Example: import
>$ impdp system/manager remap_table=my_view:my_table …
Processing object type
TABLE_EXPORT/VIEWS_AS_TABLES/TABLE_DATA
. . imported "HR"."MY_TABLE" 8.570 KB 106 rows

Without the remap_table parameter, Data Pump creates a table

called my_view

remap_table:source_name:target_name remaps the name and
creates a table with the target name

27 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 What's New with Oracle Data Pump in Oracle 12c

Introduction

Full Transportable Export

Views-as-Tables

Timestamps

Compression

Transform

Wrap Up

28 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 LOGTIME Parameter

Include timestamps in the log file

LOGTIME = [ NONE | STATUS | LOGFILE | ALL ]
– NONE: No timestamps on status or log file messages
– STATUS: On status messages only
– LOGFILE: On log file messages only
– ALL: On both status and log file messages

29 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 LOGTIME Parameter

Without vs With LOGTIME=ALL

30 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 What's New with Oracle Data Pump in Oracle 12c

Introduction

Full Transportable Export

Views-as-Tables

Timestamps

Compression

Transform

Wrap Up

31 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Enhanced Compression Algorithm

COMPRESSION_ALGORITHM
– Defines the compression algorithm when compressing dump files
– Performance:

Compression ratio

CPU usage

32 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Enhanced Compression Algorithm

COMPRESSION_ALGORITHM
– BASIC : The same algorithm used in previous versions. Good
compression, without severely impacting on performance
– LOW : For use when reduced CPU utilization is a priority over
compression ratio
– MEDIUM: Recommended option. Similar characteristics to BASIC, but
uses a different algorithm
– HIGH: Maximum available compression, but more CPU intensive
$ expdp scott/tiger tables=emp directory=mydir
dumpfile=emp.dmp logfile=expdp_emp.log
compression=all compression_algorithm=medium

33 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Enhanced Compression Algorithm

Customer evaluation
– BASIC at 3.5 TB/hour

– MEDIUM at 7.0 TB/hour

2x

34 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 What's New with Oracle Data Pump in Oracle 12c

Introduction

Full Transportable Export

Views-as-Tables

Timestamps

Compression

Transform

Wrap Up

35 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Change Table Compression at Import Time

TRANSFORM option to enable Advanced/HCC Compression
– Example:

TRANSFORM=TABLE_COMPRESSION:"compress for query high"

36 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Change Table Compression at Import Time

But: Granularity only on the entire import
– Workarounds:

Precreate objects
– Downside: Will slow down import!!!
or:

Precreate the tablespace with COMPRESS option
– create tablespace ARCHIGH datafile 'archigh.ora' size 100G
default compress for archive high;

Then run Data Pump with TRANSFORM=TABLE_COMPRESSION:N
– This will drop all embedded compression attributes associated with the tables
– Now tablespace compression option will be used for all newly created tables

37 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 No Logging Options for Import

TRANSFORM=DISABLE_ARCHIVE_LOGGING:Y
– Disable redo logging when loading tables and/or creating indexes
– Applies to both TABLES and INDEXES
– Logging attributes restored to original settings after data is loaded

38 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 No Logging Options for Import

Notes:
– Redo logging for other operations still happens
– Valid for both file mode imports and network mode imports
– FORCE LOGGING mode?
DISABLE_ARCHIVE_LOGGING option will not disable any logging

39 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 No Logging Options for Import

TRANSFORM=LOB_STORAGE:SECURE_FILE
– Options:[ SECUREFILE | BASICFILE | DEFAULT | NO_CHANGE ]
– Transforms each LOB Segment into a SecureFile LOB

40 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 What's New with Oracle Data Pump in Oracle 12c

Introduction

Full Transportable Export

Views-as-Tables

Timestamps

Compression

Transform

Wrap Up

41 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Real World Checkpoint
Customer
Payback GmbH
– Belongs to Loyalty Partner GmbH
Project
which belongs to American Express
Constraints – HQ in Munich, Germany
– Develops and operates professional customer loyalty
Preparation programs based on customized IT solutions

Provider for Payback
Upgrade

Active in Germany, Poland,
Success? India and Mexico

Remarks

42 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Real World Checkpoint
Customer
Migrate 7TB / 1.5TB from HP-UX to Exadata V1
– Cross platform, cross Endianness, cross version
Project

Oracle 9.2.0.7 on HP-UX  Oracle 11.1.0.7 on OL

Constraints – 4 months planning and migration phase

August to November 2009
Preparation – Proposed go-live date

15-NOV-2009
Upgrade

Success?

Remarks

43 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Real World Checkpoint
Customer
Move everything in less than 24 hrs

Network bottleneck
Project
– Customer installed InfiniBand hardware into HP box
Constraints  ~ 3GB/sec throughput!

Preparation

Upgrade

Success?

Remarks

44 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Real World Checkpoint
Customer
Setup:

Project
PROD SWING
Constraints Restore
+
Preparation Upgrade
Upgrade HP-UX PA-RISC HP-UX PA-RISC OL 64bit

Success?

Remarks
Prod Load

45 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Real World Checkpoint
Data Pump on

Customer
Test migrations: NETWORK_LINK

Project
PROD SWING
Constraints

Preparation

Upgrade HP-UX PA-RISC HP-UX PA-RISCINSERT APPEND

OL 64bit
on database links
for tables >100 GB
Success?

Remarks
Prod Load

46 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Real World Checkpoint
Customer
Parallel live loads: Performance tests

Project
PROD SWING
Constraints

Preparation

Upgrade HP-UX PA-RISC HP-UX PA-RISC OL 64bit

Success?

Remarks
Redirect the production load by apps servers
Prod Load Prod Load

47 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Real World Checkpoint
Customer
Final test became LIVE migration

Project PROD SWING

Constraints

Preparation

Upgrade HP-UX PA-RISC HP-UX PA-RISC OL 64bit

Success?

Remarks
Prod Load

48 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Real World Checkpoint
Customer
Live? And alive?
– Yes! Go-live in early November 2009
Project

Two weeks earlier than proposed

Constraints – Total upgrade and migration time: ~20 hours

~ 8 hours: Restore and recovery
Preparation
~ 1 hour: Database upgrade to Oracle 11.1.0.7

~10 hours: Data migration to Exadata V1
Upgrade
~ 1 hour: Smoke testing and final verification

Success? – Dramatic performance improvements

Job runtimes decreased by 80%
Remarks
User complaints about too fast performance V really!!

49 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Real World Checkpoint
Customer
Not a single piece of SQL got changed!!!
– Most critical job: runtime from 30 hrs to < 2hrs
Project

Constraints

Preparation

Upgrade

Success?

Remarks

50 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Further information?

White Paper:
OTN Page:
Full Transportable Export/Import http://otn.oracle.com/goto/datapump
http://www.oracle.com/technetwork/database/enterprise-
edition/full-transportable-wp-12c-1973971.pdf

Documentation:
Oracle Database Utilities Guide

51 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
 Resources

Download slides as PDF from:
– http://blogs.oracle.com/UPGRADE

Full Day Upgrade Workshop in Oslo:

– Registration:
http://eventreg.oracle.com/profile/web/index.cfm?PKWebID=0x7
05481765&source=EMEAFM13033369MPP059

– May 20, 2014

52 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
53 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c
54 Copyright © 2014, Oracle and/or its affiliates. All rights reserved. Working with Oracle Multitenant in Oracle Database 12c