12 c

Best Practices Handbook

 About The Author
Prashanth Ashwathram is a seasoned Business
Intelligence professional and is experienced in the
analysis, design and development of Data
Warehousing and Business Intelligence applications
using ETL and OLAP tools. He has led multiple
successful BI Apps implementation projects and has
been instrumental in implementing ODI for various
customers including Ellucian, FireEye, and Nintendo.

Prior to KPI Partners, Prashanth was part of Oracle’s

BI Applications Core Engineering Development team
and was involved in the architecting and building of
various analytics modules.
 About KPI Partners

The Leader In Cloud Applications, Big Data, Business

Intelligence, and Data Discovery

KPI Partners provides strategic guidance and

technology systems for clients wishing to solve their
most complex and interesting business challenges
involving cloud applications, big data, business
intelligence, and data discovery.

KPI works with both technology departments and

business units to develop value-added decision
support solutions, not just new technology
deployments.

[email protected]
www.kpipartners.com
1-888-988-4KPI
 ODI 12c Best Practices Handbook

Preface
This eBook requires the reader to have a basic
familiarity with Oracle Data Integrator (ODI) and its
components. Please refer to our ODI blogs for a
quick introduction and understanding various
concepts of ODI.
ODI is a very powerful extract, load and transform
(ELT) product when handled correctly. Unfortunately,
a few common mistakes may lead to negative
results in integration projects. This e-book compiles a
few best practices that help avoid the most
common mistakes observed in integration projects
involving ODI.

Copyright © 2016 All rights reserved | www.kpipartners.com 5

 ODI 12c Best Practices Handbook

Introduction
ODI delivers high-performance data movement and
transformation among enterprise platforms with its
open and integrated ELT architecture
The latest ODI 12c version has many new features and
functionalities that deliver extreme performance,
higher productivity and is a future-ready solution that
enables you to keep up with the latest trends
including big data analytics, cloud computing, and
real-time BI

Copyright © 2016 All rights reserved | www.kpipartners.com 6

 ODI 12c Best Practices Handbook

Oracle Data Integrator

ELT: High Volume Data Processing and Fast Load Transformation

Cloud
Oracle Data Integrator • Certified for leading
technologies to deliver
fast time to value
Apps
High Performance ELT • High-performance, low
cost of ownership ELT
architecture
Declarative Design • Lightweight
Database
deployment

Extensible Knowledge
• Flexible, easy-to-enrich
Modules functionality
Big Data

Copyright © 2016 All rights reserved | www.kpipartners.com 7

 ODI 12c Best Practices Handbook

Industry Leading Performance

Extremely Fast Execution and Reduced Cost

Conventional E-TL Architecture

Extract
ELT provides a flexible architecture for
Load
optimized performance on any platform
Transform

Benefits
• Leverages set-based transformations
Next Generation ELT Architecture • Improves performance for loading, no
network hop
• Takes advantage of existing
infrastructure: hardware and software
Extract Load
Transform Transform

Copyright © 2016 All rights reserved | www.kpipartners.com 8

 ODI 12c Best Practices Handbook

Knowledge Modules
Simpler Physical Design and Shorter Implementation Time
Pluggable Knowledge Modules Architecture
Reverse Load from
Journalize Check Integrate;
Engineer Source to Data Service
(CDC) Constraints Transform
Metadata Staging

Sample out-of-the-box Knowledge Modules Key Architecture Benefits

• Faster development and
Oracle Spark Hive HBase Pig Sqoop simpler maintenance using
templates
• Easy to extend and add new
BICS
Oracle Oracle
JMS
External
Teradata
best practices
Datapump DBLink Tables
• Enforces predictability and
reduces cost of ownership
eBusiness
• Download Knowledge
SAP Siebel IBM DB2 Netezza SCD
Suite Modules from ODI
Exchange(akin to app store)

Copyright © 2016 All rights reserved | www.kpipartners.com 9

 ODI 12c Best Practices Handbook

Extended Connectivity
Leverages Existing IT, Faster Implementation Cloud
• Big Data: Spark, Hive, Pig, Hbase, Sqoop & Oozie
• Best for Oracle: Merge, Spatial, Multi-Table Insert, Big Data
Optimizer Hints and more
• Real-Time: CDC with GoldenGate
Applications
• Applications: E-Business Suite, Siebel, PeopleSoft,
JD Edwards Enterprise One, JD Edwards World,
SAP ERP and SAP BW
Databases
• Heterogeneous: Optimizations for all major
RDBMS: IBM DB2, Microsoft SQL Server, Teradata,
Netezza etc.
Legacy

Copyright © 2016 All rights reserved | www.kpipartners.com 10

 ODI 12c Best Practices Handbook

Topics
 Repository Management
 Knowledge Modules
 Topology Configurations
 Enforcing Data Quality
 Migration Utility

Copyright © 2016 All rights reserved | www.kpipartners.com 11

 ODI 12c Best Practices Handbook

Repository Management
ODI 12c repositories include a Master and Work
repository. Master repository holds the topology and
security information, whereas the work repository holds
the model/project information.

Repository Backups and restoration are very vital

during code migrations. Following key items to
consider during backups and migration:

• Smart exports are ideal for backups of subsets of

repositories as they export along with the
dependency

• Entire repository backups can be performed from

GUI and exported as xml for safe-keeping

Copyright © 2016 All rights reserved | www.kpipartners.com 12

 ODI 12c Best Practices Handbook

Repository Management
• Ensure when backups are performed, both Master
and Work repositories backups are created

• Other approach to backup schemas includes

creating regular backup schedule of the database
schemas on which Master and Work repositories
are deployed

ODI’s metadata repositories are on relational

databases and though it might be enticing to dally
with repository tables to get speed up the process. Its
always better to refrain from doing so as incorrect
usage might result in corrupted repository metadata.

Copyright © 2016 All rights reserved | www.kpipartners.com 13

 ODI 12c Best Practices Handbook

Repository Management
Each ODI repository gets a unique ID at the time of
creation. While creating multiple repositories, you
should make sure that all repositories are created with
strictly different IDs (even on different environments like
dev/qa/prod), and you should define and document
the process for moving objects between repositories
using import export, or versioning.

In addition to the Repository ID, all objects are

identified by an Internal ID that depends on the
Repository ID. These internal IDs uniquely identify an
object and are used by the ODI import system. Two
repositories with the same ID possibly contain objects
with the same internal ID, which mean the same
object for ODI. Transferring objects between these
repositories is similar to copying files with same names
across different directories, and may lead to object
replacement.

Copyright © 2016 All rights reserved | www.kpipartners.com 14

 ODI 12c Best Practices Handbook

Knowledge Modules
Knowledge Modules (KMs) are powerful framework
used at integration points in ODI. A large number of
KMs are available out of the box and support a large
number of database features. Its essential these KM’s
are used in according to the use case and not to
over-complicate mappings when simpler KM’s can do
the job.

• Don’t always import all the KM’s into project. Only

import the ones that are needed.

Copyright © 2016 All rights reserved | www.kpipartners.com 15

 ODI 12c Best Practices Handbook

KM Features
• The KM choice is critical when designing mappings. KM selection drives the features
available and the performances of the mappings.
• Start with simple KMs, do not over-engineer by selecting complex KMs or activating
complex options. Pay close attention to the KM options as simpler KM’s might be
sufficient in certain cases.
• KMs with extra features have an extra cost in terms of performance. For example,
performing simple insert only is faster and efficient than performing an incremental
update. If the use case deems only inserts are necessary or of targets are truncated
prior to load, using incremental update KM is an over-kill and could degrade the
performance.
• Similarly, activating or deactivating some of the KM features may bring in few
additional steps that may impact the performance.

Copyright © 2016 All rights reserved | www.kpipartners.com 16

 ODI 12c Best Practices Handbook

Topology Configuration
• In ODI, all executions are performed on a Logical Architecture (Logical schemas,
logical agent), that resolves to a Physical Architecture (real/physical source/targets
data servers/schemas and ODI run-time agents) with the help of Context.
• Contexts allow you to switch the execution of the artifacts from one environment
(context) to another easily.
• Another common mistake is forgetting to perform the logical/physical architecture
mapping in a given context. This is a topology administration mistake that leads to
executions not working in a given context. Ensure all the logical resources are
mapped to physical resources in every context used.
• You should never specify/use any qualified physical object names such as the
database name or schema name in the mappings from Designer components.
Because they may change depending on the execution context. It’s a good
practice to use substitution methods in such cases to make the design context
independent.

Copyright © 2016 All rights reserved | www.kpipartners.com 17

 ODI 12c Best Practices Handbook

Enforcing Data Quality

ODI allows you to enforce data quality of
source data using static checks as well as the
quality of the data before it is pushed into a
target via flow checks. Using both these checks,
you can make sure that the quality of the data
is improved or enforced when the data is
moved and transformed.
Not taking into account the quality of data,
data integration itself may end up moving and
transforming garbage, and propagating it over
the applications.

Copyright © 2016 All rights reserved | www.kpipartners.com 18

 ODI 12c Best Practices Handbook

Migration Utility
ODI being Oracle’s strategic data integration product,
oracle’s older integration suite, OWB is slowly entering
wilderness. Oracle has provided a migration tool for
customers using OWB to quickly migrate to ODI. Some
of the key things to keep in mind while performing
migration:

• Before running the migration utility, make sure the

environment variables ODI_HOME, ORACLE_HOME
and JAVA_HOME are correctly set. You may not be
able to run the migration utility if these variables are
pointing to incorrect path.

Copyright © 2016 All rights reserved | www.kpipartners.com 19

 ODI 12c Best Practices Handbook

Migration Utility
Ensure OWB and ODI clients connecting to the repository are closed before running the
migration utility. It can not read the OWB metadata if there is an active OWB repository
connection. Migration would also slow down if any an ODI connection is active.

It is a good practice to set migration property ‘STOP_ON_ERROR’ to false if you are

migrating in bulk. Any failures can be later analyzed by the Migration Report and Logs.

Choose the migration strategy wisely. If MIGRATION_STRATEGY is set to ‘NODUP’, the

migration utility will not migrate if any object with same name already exists in the ODI
repository. It doesn’t do a comparison between conflicting objects. When set to
‘CREATE’, It will migrate all the objects adding a number as suffix(_0 or _1) to the latest
migrated object

Copyright © 2016 All rights reserved | www.kpipartners.com 20

 ODI 12c Best Practices Handbook

Review Report File

It is always a good practice to review the .report file
post migration. The .report file contains a detailed log
for the migration process.

All the OWB operators should be bound to correct

locations. Though migration utility is capable of
migrating unbound operators, it automatically binds the
unbound object to the corresponding folder/project.
This may cause ambiguity in ODI and result in incorrect
data integration.

In OWB mapping, ensure correct target table name is

chosen in ‘Target Load Property’. Incorrect target table
name in ‘Target Load Property’ makes an ODI mapping
invalid.

Copyright © 2016 All rights reserved | www.kpipartners.com 21

 ODI 12c Best Practices Handbook

References
For ODI beginners, these resources provide valuable
insights and could get you started on the ODI
voyage:

Copyright © 2016 All rights reserved | www.kpipartners.com 22