Scripting and Automating ODI 11g using Groovy and the ODI SDK
Michael Rainey, Principal Consultant, Rittman Mead
Rittman Mead BI Forum 2013 Master Class, May 2013
T : +44 (0) 8446 697 995 E :
[email protected] W: www.rittmanmead.com
About me...
Michael Rainey, Principal Consultant, Rittman Mead
Oracle Data Integration Specialist
Oracle Data Integrator 11g, Oracle GoldenGate 11g
Instructor for Rittman Mead ODI 11g Bootcamp course
Writer for Rittman Mead Blog: http://www.rittmanmead.com/blog
Email :
[email protected]Twitter : @mRainey
T : +44 (0) 8446 697 995 E : [email protected] W: www.rittmanmead.com
About this presentation...
Introduction to the ODI SDK
Introduction to Groovy scripting language
Demonstrations
Developer s Quickstart
Metadata Setup
Initial Load: Interfaces, Scenarios, and Load Plans
ETL Migration to ODI
T : +44 (0) 8446 697 995 E : [email protected] W: www.rittmanmead.com
Intro to the ODI SDK
Java based API
Develop using Java, Jython, or Groovy
Perform nearly every action that can be done in ODI Studio
Generate metadata
Create / Modify Interfaces, Packages, Load Plans, etc
Manage execution sessions
Resources
Data Integration blog
https://blogs.oracle.com/dataintegration/tags/sdk
Rittman Mead blog
http://www.rittmanmead.com/tag/odi-11g-sdk/
Oracle Data Integrator Java API Reference
http://docs.oracle.com/cd/E28280_01/apirefs.1111/e17060/index.html
T : +44 (0) 8446 697 995 E :
[email protected] W: www.rittmanmead.com
Intro to Groovy
Agile, dynamic object oriented programming language for the Java Virtual Machine
Like Java...but simplified
Most Java syntax is valid in Groovy
Additional features inspired by Python, Ruby, and Smalltalk
Seamless integration with existing Java classes and libraries
SwingBuilder
Create Swing GUIs using Groovy builders
Easy to develop and maintain
Hierarchy of components
http://groovy.codehaus.org/
T : +44 (0) 8446 697 995 E : [email protected] W: www.rittmanmead.com
Groovy and ODI Integration
Oracle Data Integrator is delivered with a Groovy editor
Create and save .groovy script files
Integrates with the ODI instance and security manager
Transaction management simplified
Create transaction
Persist
Commit transaction
On exception...Rollback transaction
T : +44 (0) 8446 697 995 E : [email protected] W: www.rittmanmead.com
Develop Groovy in ODI Procedures
ODI Procedure and Knowledge Module steps
Groovy as a Technology
Less integrated than .groovy scripts
Must define
-Connection to repository
-ODI username & password
-ODI Instance for security and transaction
management
T : +44 (0) 8446 697 995 E : [email protected] W: www.rittmanmead.com
Use Case - Developer s Quickstart
Case:
Many find it difficult to use Oracle Data Integrator right out of the box
Other ETL tools provide a quick and easy way to begin development
Solution:
Determine which tasks need to be automated and develop Groovy script
-Create Data Server, Physical Schema, and Logical Schema
-Map Logical Schema/Physical Schema using the default Global context
-Create new Model
-Build and execute Package to reverse engineer tables into ODI Datastores
Initial code developed by Oracle s David Allan
ODI 11g Expert Accelerator for Model Creation
https://blogs.oracle.com/dataintegration/entry/odi_11g_expert_accelerator_for
T : +44 (0) 8446 697 995 E : [email protected] W: www.rittmanmead.com
Developer s Quickstart - Capture User Input
T : +44 (0) 8446 697 995 E : [email protected] W: www.rittmanmead.com
Demonstration
Developer s Quickstart - ODI SDK and Groovy
T : +44 (0) 8446 697 995 E :
[email protected] W: www.rittmanmead.com
Use Case - Metadata Setup
Case:
Setup of Staging and Foundation metadata
Add data warehouse audit columns
Solution:
Reverse engineer Source tables
Copy tables to Staging and Foundation Models
Add data warehouse audit columns to Foundation
Datastores via Groovy script
T : +44 (0) 8446 697 995 E : [email protected] W: www.rittmanmead.com
Demonstration
Metadata Setup - ODI SDK and Groovy
T : +44 (0) 8446 697 995 E :
[email protected] W: www.rittmanmead.com
Use Case - Initial Load Setup: Interfaces, Scenarios, and Load Plans
Case:
Setup objects for Source to Foundation initial load
Solution:
Groovy scripts
-Create Source to Foundation Interfaces
-Generate a Scenario for each Interface
-Build a Load Plan to execute all Scenarios
T : +44 (0) 8446 697 995 E : [email protected] W: www.rittmanmead.com
Initial Load Setup - InteractiveInterfaceHelperWithActions
T : +44 (0) 8446 697 995 E : [email protected] W: www.rittmanmead.com
Demonstration
Initial Load Setup - Interfaces, Scenarios, & Load Plans - ODI SDK and Groovy
T : +44 (0) 8446 697 995 E : [email protected] W: www.rittmanmead.com
Use Case - Migration to Oracle Data Integrator
Case:
Automate the migration of mappings from an old ETL tool to Oracle Data Integrator
Oracle Warehouse Builder is the ETL tool in this example
Solution:
Place the old ETL tool metadata in a database schema
Build Groovy scripts against the metadata to create ODI objects
-Topology
-Project
-Metadata
-Interfaces
T : +44 (0) 8446 697 995 E : [email protected] W: www.rittmanmead.com
Migration to ODI - OWB Metadata
T : +44 (0) 8446 697 995 E : [email protected] W: www.rittmanmead.com
Demonstration
Migration to Oracle Data Integrator - ODI SDK and Groovy
T : +44 (0) 8446 697 995 E :
[email protected] W: www.rittmanmead.com
In summary...
ODI SDK can perform nearly every action as the ODI Studio
Groovy script is the language of choice for developing against the ODI SDK
Java-based, but simple to develop
Developed in Groovy script files or Procedure steps
Automation of large, batch tasks in ODI can save loads of time
More information can be found at http://www.rittmanmead.com
Contact us at
[email protected] or
[email protected]Follow-us on Twitter (@rittmanmead & @mRainey) or Facebook
(facebook.com/rittmanmead)
T : +44 (0) 8446 697 995 E :
[email protected] W: www.rittmanmead.com
Scripting and Automating ODI 11g using Groovy and the ODI SDK
Michael Rainey, Principal Consultant, Rittman Mead
Rittman Mead BI Forum 2013 Master Class, May 2013
T : +44 (0) 8446 697 995 E : [email protected] W: www.rittmanmead.com
