Informatica PowerExchange Architecture Overview

Dan Smith
Principal Subject Matter Expert May 2012

Informatica PowerExchange

Batch Architecture Overview

Informatica PowerExchange
Batch Architecture

Data Integration Platform

SQL Data Records

PowerCenter

Data Maps for Non-Relational Access

Listener

PowerExchange

Mainframe and Mid-Range

Packaged Applications

Relational and Flat Files

Standards and Messaging

Remote Data

User Applications Operating Environment

Informatica PowerExchange
How to Create Metadatawhatever your source
A Datamap defines metadata
For non-relational databases, need to define metadata Contains all information regarding the underlying dataset

Created from existing information

VSAM copybooks IMS DBDs, then overlay segment copybooks ADABAS FDT IDMS - schemas

Stored on the same platform as the data Not needed for relational sources Provides the information to convert an SQL statement to actual data base requests
4

Informatica PowerExchange
Design Time Metadata Driven Approach

Data Integration Platform

(Create Datamap) Metadata Request

PowerExchange Navigator

PowerExchange

Metadata Records

Data Maps for Non-Relational Access

Copybooks Adabas Definitions Schemas

Listener

PowerExchange

Database Definitions

Hand Key

Mainframe and Mid-Range

Packaged Applications

Relational and Flat Files

User Applications

Standards and Messaging

Remote Data

Operating Environment

Informatica PowerExchange
Row Test the Data Map Preview the data

Informatica PowerExchange

Change Data Capture Architecture Overview

Informatica PowerExchange CDC

Defining the Changes to be Extracted

Data Integration Platform PowerExchange

(Create Datamap)
Navigator

PowerExchange

SQL

Data Records

Data Maps for Non-Relational Access Extraction Definitions

Listener

PowerExchange

Capture Registrations

Extract on definitions are based on capture registrations

Operating Environment

Informatica PowerExchange CDC

Capture-Once Deliver-Many

Data Integration Platform

SQL Data SQL Records Data Records

PowerCenter

PowerExchange

Listener

Listener

PowerExchange

Change Stream/ Logger

Collector
Physical VSAM Linear Datasets

Collector

Change Stream/ Logger

Database vendor - journal, archive, redo logs

Published Interfaces
ADABAS DATACOM DB2 IDMS IMS VSAM

Mainframe

Mid-Range

DB2/400 Oracle SQL Server DB2 LUW

Informatica PowerExchange CDC

Architecture for DB2 z/OS CDC

Data Integration Platform

3. PowerCenter issues standard SQL request

PowerCenter

SQL

Data Records

4. PowerExchange sends changed data records to PowerCenter

Listener

PowerExchange

DB2 z/OS

DB2 Logs
IFI 306 (z/OS)

Change Stream/ Logger

Collector

Agent

Data Maps for special use cases

DB2 SubSystem
1. PowerExchange collects changes

z/OS Environment

2. Changed data persisted in change stream

10

Informatica PowerExchange CDC

Architecture for DB2 i5/OS CDC

Data Integration Platform

SQL Data Records

PowerCenter

Journaling (i5/OS)

Listener

PowerExchange

Journal Reader

Data Maps for special use cases

DB2 i5/OS

Journals

DB2 SubSystem I5/OS Environment

11

Informatica PowerExchange CDC

Architecture for DB2 LUW CDC

Data Integration Platform

SQL Data Records

PowerCenter

Listener

PowerExchange

DB2 LUW

Archive Logs
LogRead API

Virtual Change Stream

Data Maps for special use cases

Collector

DB2 Instance Linux, UNIX, or Windows Environment

12

Informatica PowerExchange CDC

Architecture for SQL Server CDC

Data Integration Platform

SQL Data Records

PowerCenter

Listener

PowerExchange

SQL Server SQL Server Database Distribution DB

SQL query

Virtual Change Stream

Collector

Data Maps for special use cases

SQL Server Windows Environment

13

Informatica PowerExchange CDC

Architecture for PWX Oracle LogMiner CDC

Data Integration Platform

4. PowerCenter issues standard SQL request

PowerCenter

SQL

Data Records

5. PowerExchange sends changed data records to PowerCenter

Listener

PowerExchange

Change Stream Oracle

1. Oracle collects and persists changes in Redo Logs

Data Maps for special use cases

Oracle Logs
LogMiner Collector/PWXCCL
3. Changed data persisted in change stream

Oracle Instance
2. PowerExchange issues LogMiner request

Linux, UNIX, Windows, or z/Linux Environment

14

Informatica PowerExchange CDC

Architecture for PWX Express Oracle CDC

Data Integration Platform

4. PowerCenter issues standard SQL request

PowerCenter

SQL

Data Records

5. PowerExchange sends changed data records to PowerCenter

Listener

PowerExchange

Change Stream Oracle

1. Oracle collects and persists changes in Redo Logs

Data Maps for special use cases

Oracle Logs
Express Log Reader Collector/PWXCCL
3. Changed data persisted in change stream

Oracle Instance
2. PWX Express Log Reader captures changes

Linux, UNIX, Windows, or z/Linux Environment

15

Informatica PowerExchange CDC

Architecture for Mainframe VSAM CDC (Batch or CICS)

Data Integration Platform

3. PowerCenter issues standard SQL request

PowerCenter

SQL

Data Records

4. PowerExchange sends changed data records to PowerCenter

Listener

PowerExchange

VSAM
JRNAD Exit (Batch) File/User Exits (CICS)

Change Stream

Agent

Data Maps for Non-Relational Access

Collector
2. Changed data persisted in change stream

Batch or CICS Mainframe Environment

1.PowerExchange collects changes

16

Informatica PowerExchange CDC

Architecture for Mainframe IMS Synchronous CDC

Data Integration Platform

SQL Data Records

PowerCenter

Listener PWX Libraries

PowerExchange

DLI Job or DBCTL

Change Stream
COLLECTOR OPERATES IN Address Space of Batch DLI Job Or IMS Control Region

Agent

Data Maps for Non-Relational Access

Batch or Online IMS

Mainframe Environment

17

Informatica PowerExchange CDC

Architecture for Mainframe IMS Log-Based CDC

Data Integration Platform

SQL Data Records

PowerCenter

Listener Archived Logs

PowerExchange

DLI Job or DBCTL

Change Stream
Recons Collector

Agent

Data Maps for Non-Relational Access

Batch or Online IMS Mainframe Environment

18

Informatica PowerExchange CDC

Architecture for Mainframe ADABAS CDC

Data Integration Platform

SQL Data Records

PowerCenter

ADABAS PLOGs

Active PLOGs Active PLOGs ADABAS User Exit 2

Listener

PowerExchange

Change Stream

Agent

Data Maps for Non-Relational Access

ADABAS Nucleus

Collector

ADABAS Mainframe Environment

19

Informatica PowerExchange CDC

Architecture for Mainframe Datacom Synchronous CDC

Data Integration Platform

SQL Data Records

PowerCenter

Change Collector Subtask

Listener

PowerExchange

Datacom MUF
ECSA

Agent

Data Maps for Non-Relational Access

Change Stream
Change Controller

Datacom

Mainframe Environment

20

Informatica PowerExchange CDC

Architecture for Mainframe Datacom Table-Based CDC

Data Integration Platform

SQL Data Records

PowerCenter

Listener

PowerExchange

TSN Table Datacom MUF MNT Table Change Stream

Agent

Data Maps for Non-Relational Access

Collector

Datacom Mainframe Environment

21

Informatica PowerExchange CDC

Architecture for Mainframe IDMS Log-Based CDC

Data Integration Platform

SQL Data Records

PowerCenter

Journal Write Process

Listener

PowerExchange

IDMS

Journal

IDMS CV

IDMS Journal Switch

Change Stream

Agent

Data Maps for Non-Relational Access

Collector

Batch or Online IDMS Mainframe Environment

22

Informatica PowerExchange
Supported Sources & Targets
Informatica IMS VSAM DB2 z/OS ADABAS IDMS Datacom/DB DB2 i5/OS Oracle SQL Server DB2 LUW Batch R/W R/W R/W R/W R R R/W R/W R/W R/W CDC Y Y Y Y Y Y Y Y Y Y

23

24