DMM269 – Real-Time Change Data Capture with

SAP Data Services and SAP Replication Server

Ben Hofmans / Product Management

Public
Disclaimer

This presentation outlines our general product direction and should not be relied on in making a
purchase decision. This presentation is not subject to your license agreement or any other agreement
with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to
develop or release any functionality mentioned in this presentation. This presentation and SAP's
strategy and possible future developments are subject to change and may be changed by SAP at any
time for any reason without notice. This document is provided without a warranty of any kind, either
express or implied, including but not limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this
document, except if such damages were caused by SAP intentionally or grossly negligent.

© 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 2

Agenda

SAP Data Services

Enhanced integration between Data Services and Replication Server
Exercise scenario & Demo

© 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 3

SAP Data Services
Extract, Transform and Load & more

SAP Data Services

Business UI Technical UI
(Information (Data Services)
Steward)

SAP Data Services enables Unified Metadata

users with access to any data,
abilities to transform, improve, One Runtime ETL
and enrich structured or Architecture &
unstructured data, in batch or at Services Data Quality
real-time point of entry, all via a
single solution Profiling

Text Analytics

One Administration Environment

(Scheduling, Security, User Management)
One Set of Source/Target Connectors

© 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 4

SAP Data Services user interface

SAP Data Services Designer

 Drag & drop UI
 Rich library of transformations for
data integration, data quality and
text data processing.
 100+ built-in functions for advanced
data mappings.
 Design, debug, execute and monitor
from one UI.

© 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 5

Agenda

SAP Data Services

Enhanced integration between Data Services and Replication Server
Exercise scenario & Demo

© 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 6

 Real-time data integration with SAP Replication Server &
SAP Data Services

Source DB Target DB

SAP Replication Server

(formerly Sybase Replication
Server)

Value Proposition Use Cases How it works

Ease of use with rich capabilities. Real-time business insights Direct data integration

 Continuous capture of data changes from  Generate up-to-date consumer usage  Data Services connects directly to source
multiple sources to multiple targets reports for analysis systems for metadata information
 Rich tooling with transformation and  Realtime updates from non-SAP systems  Data Services connects to Replication
enrichment capabilities to SAP HANA Server/Agent for subscription,
 Automates and ease of administration configuration and changed data fetch

7
© 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 7
Real-time data integration with SAP Replication Server &
SAP Data Services – Phase II

Real-time CDC with Replication Server was first introduced in Data Services 4.2
Integration required use of PowerDesigner for Replication Agent subscriptions and ASE as external
staging between Replication Server and Data Services

In Data Services 4.2 SP3, a simplified architecture and configuration

Data Services connects directly to source systems for metadata information.
Data Services connects directly to Replication Server/Agent for subscription configuration and
changed data fetch.

Note: Data Services 4.2 SP3 in combination with Replication Server 15.7.1 SP 203 supports Oracle as source.
Support for other databases (Microsoft SQL Server, DB2, SAP ASE) will be added in upcoming SP releases
based upon customer demand.

© 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 8

Typical setup for initial + delta load
Step 1: Configure Replication Server

Using Rs_init.exe command line utility and a PRIMARY_CONNECTION.rs resource file.

The steps to setup Replication Server include:
 Creates a maintenance user in source and give it the required permissions.
 Install Replication Server system objects by running a script in the source database (rs_install_primary.sql)
 Create connection for source database in Replication Server
 Configure the replication agent on source database side (sp_config_rep_agent stored procedure).
 Start the replication agent thread in the source database (sp_start_rep_agent)

© 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 9

Typical setup for initial + delta load
Step 2: Datastores

 Create 2 datastores, one without CDC option for the

initial load, the other with CDC for the delta load.
 Import the same table in both datastores.

© 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 10

Typical setup for initial + delta load
Step 3: Job and initial load dataflow

 First step in the job is a conditional workflow to check this

job needs to do an initial load first or only delta load.
 In case of initial load:
– The built-in function begin_initial_load() is called.
– A dataflow reads from the non-CDC source table and
loads into the target table. The target table has the
option “delete data before loading”.

© 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 11

Typical setup for initial + delta load
Step 4: Continuous workflow

 A Replication Server CDC data flow can only be used

inside a Continuous Workflow.
 Once the job is started, it will continue running and
fetching changed data from Replication Server.
– Option to stop the job by using the “Stop when result of
the function is zero”
o E.g. use systime() to stop the CDC job each day at 9pm and
restart at 1am.
o E.g. use sql() function to read a stop flag from a database
control table.

© 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 12

Typical setup for initial + delta load
Step 5: CDC dataflow

 The CDC dataflow inside the continuous workflow will

read the table from the CDC datastore.
 In the dataflow use Map_CDC_Operation to generate
INSERT, UPDATE and DELETE statements for the target
table.

© 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 13

Agenda

SAP Data Services

Enhanced integration between Data Services and Replication Server
Exercise scenario & Demo

© 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 14

Scenario

Scenario details:
 Source: SAP ASE database with shop (store) information
 Target: Microsoft SQL Server database, needs replicated shop data in (near) real-time
 Extra requirement: whenever a shop is deleted in the source application, the shop in the target application
needs to be updated with a flag set to inactive. No physical delete in the target system in order to keep history.

SAP
MS SQL
(Sybase)
Server
ASE

SAP Replication Server

(formerly Sybase Replication
Server)

© 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 15

Demo
SAP d-code Virtual Hands-on Workshops and SAP d-code Online
Continue your SAP d-code education after the event!

SAP d-code Virtual Hands-on Workshops SAP d-code Online

 Access hands-on workshops post-event  Access replays of keynotes, Demo Jam, SAP d-code
 Starting January 2015 live interviews, select lecture sessions, and more!
 Complementary with your SAP d-code registration  Hands-on replays
http://sapdcodehandson.sap.com http://sapdcode.com/online

© 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 17

Further Information

SAP Public Web

Community @SCN: http://scn.sap.com/community/data-services
Wiki @SCN: http://wiki.scn.sap.com/wiki/display/EIM/Data+Services
Idea Place: https://ideas.sap.com/SAPBusinessObjectsDataServices
Documentation: http://help.sap.com/bods/

SAP Education and Certification Opportunities

www.sap.com/education

Watch SAP d-code Online

www.sapcode.com/online

© 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 18

Feedback
Please complete your session evaluation for
DMM269.

Thanks for attending this SAP TechEd && d-code session.

©©2014
2014SAP
SAPSE
SEororananSAP
SAPaffiliate
affiliatecompany.
company.AllAllrights
rightsreserved.
reserved. Public 1919
© 2014 SAP SE or an SAP affiliate company. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an
SAP affiliate company.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE
(or an SAP affiliate company) in Germany and other countries. Please see http://global12.sap.com/corporate-en/legal/copyright/index.epx for additional trademark
information and notices.
Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors.
National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its
affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or
SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing
herein should be construed as constituting an additional warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or
release any functionality mentioned therein. This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future
developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for
any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-
looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place
undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.

© 2014 SAP SE or an SAP affiliate company. All rights reserved. Public 20