NetWeaver PI 7.

30 File/FTP
Adapter - Duplicate Message
Handling Feature

Applies to:
SAP NetWeaver Process Integration (PI) 7.30+. For more information, visit the SOA Middleware homepage.

Summary
This feature increases the robustness by avoiding the processing of duplicate messages. It also provides
flexible threshold configuration and channel deactivation (if threshold exceeds) options. This section provides
the configuration documentation for this new feature which was developed in 7.30+ releases
Author: Sivasubramaniam Arunachalam
Company: SAP Labs
Created on: 14 September 2011

Author Bio
Sivasubramaniam Arunachalam is a senior developer at SAP Labs (Technology Innovation
Platform). He is currently occupied with PI 7.31 development/maintenace activities. Since
Sivasubramaniam joined SAP Labs in July 2010, he has developed new features in several
adapters/areas including File, JDBC, IDoc, SOAP/XI, HTTP, JPR, B2B(RNIF 1.1/2.0, CIDX &
PIDX) Adapters and XML Validation. Currenly, he is the component responsible for File,
JDBC, B2B Adapters and XML Validation and takes care of all new development,
enhancement and maintenance activities.
Before joining SAP Labs in 2010, He was working with Cognizant Technology Solutions in serveral areas
including SOA, A2A/B2B integration/middleware(webMethods) and intranet/internet web application
development(Java/J2EE).

SAP COMMUNITY NETWORK SDN - sdn.sap.com | BPX - bpx.sap.com | BA - boc.sap.com | UAC - uac.sap.com
© 2011 SAP AG 1
 NetWeaver PI 7.30 File/FTP Adapter - Duplicate Message Handling Feature

Table of Contents
Introduction ......................................................................................................................................................... 3
Overview ............................................................................................................................................................. 3
Enhanced Components ...................................................................................................................................... 3
New Configuration Options ................................................................................................................................. 4
Scenario Check List (File Sender Behaviour at various stages) ........................................................................ 5
Sample Scenario 1: ............................................................................................................................................ 5
Channel Configuration: ................................................................................................................................... 5
Scenario Description: ...................................................................................................................................... 5
Processing Logs: ............................................................................................................................................. 5
Sample Scenario 2: ............................................................................................................................................ 6
Channel Configuration: ................................................................................................................................... 6
Scenario Description: ...................................................................................................................................... 6
Processing Logs: ............................................................................................................................................. 6
Sample Scenario 3: ............................................................................................................................................ 7
Channel Configuration: ................................................................................................................................... 7
Scenario Description: ...................................................................................................................................... 7
Processing Logs: ............................................................................................................................................. 7
Sample Scenario 4: ............................................................................................................................................ 8
Channel Configuration: ................................................................................................................................... 8
Scenario Description: ...................................................................................................................................... 8
More about Duplicate Message Handling ........................................................................................................... 8
Related Content .................................................................................................................................................. 9
1. Configuring the Sender File Adapter ..................................................................................................... 9
2. Configuring the Sender FTP Adapter .................................................................................................... 9
3. What's New in SAP NetWeaver 7.3....................................................................................................... 9
Copyright........................................................................................................................................................... 10

SAP COMMUNITY NETWORK SDN - sdn.sap.com | BPX - bpx.sap.com | BA - boc.sap.com | UAC - uac.sap.com
© 2011 SAP AG 2
 NetWeaver PI 7.30 File/FTP Adapter - Duplicate Message Handling Feature

Introduction
To increase the robustness in file adapter by avoiding/controlling the processing of duplicate messages, this
feature has been developed. The File/FTP Sender Channel provides flexible threshold configuration and
channel deactivation (if threshold exceeds) options. This tutorial aims to provide how to guide which
including different ways of configuration and monitoring options.

Overview

If this feature is enabled, file/ftp sender channel determines if the current file is a duplicate of an already
processed file. If it is a duplicate, it will ignore and create an alert based on the configuration.
The duplicate file is identified by the combination of the following file properties.
 Fully qualified path including file name (with extension)
 Size
 Last modified time stamp
If any of these properties gets changed, the duplicate count also reset and considered as a new file

Enhanced Components
Three new fields have been added in File Sender Adapter Configuration for the following transport protocols.
 NFS
 FTP

SAP COMMUNITY NETWORK SDN - sdn.sap.com | BPX - bpx.sap.com | BA - boc.sap.com | UAC - uac.sap.com
© 2011 SAP AG 3
 NetWeaver PI 7.30 File/FTP Adapter - Duplicate Message Handling Feature

New Configuration Options

New Field Description

Enable Duplicate Handling To enable the Duplicate Message Handling feature
It will process the given number of duplicates and if the
Duplicate Message Alert Threshold duplicate count exceeds, the subsequent message will be
ignore and alerts will be created.
The channel will become inactive status and no more
Disable Channel if threshold exceeded
processing is allowed

SAP COMMUNITY NETWORK SDN - sdn.sap.com | BPX - bpx.sap.com | BA - boc.sap.com | UAC - uac.sap.com
© 2011 SAP AG 4
 NetWeaver PI 7.30 File/FTP Adapter - Duplicate Message Handling Feature

Scenario Check List (File Sender Behaviour at various stages)

The below table lists six different scenarios by configuring with different values for new fields.

Note:
The First Message (created from a file) will be always referred as ‘Original Message’
The Second Message (Created from the same file) will be referred as ‘First Duplicate Message’ and the current duplicate
count is 1

Sample Scenario 1:
Channel Configuration:

Scenario Description:
 The file will be processed only once ( Original Message + Zero Duplicates)
 All other messages starts from First Duplicate will be ignored
 Channel will be in active status but with errors (after threshold value exceeds the duplicate count)
 It will be process the further messages with duplicate check

Processing Logs:

SAP COMMUNITY NETWORK SDN - sdn.sap.com | BPX - bpx.sap.com | BA - boc.sap.com | UAC - uac.sap.com
© 2011 SAP AG 5
 NetWeaver PI 7.30 File/FTP Adapter - Duplicate Message Handling Feature

Error Message:

From the above logs, we can observe that

 The Original Message is processed successfully
 The Next message (First Duplicate) is skipped
 Duplicate alerts are sent to alert inbox
 The channel is still active with errors

Note: The same error message will be available in alert inbox also.

Sample Scenario 2:
Channel Configuration:

Scenario Description:
 The file will be processed only once ( Original Message + Zero Duplicates)
 All other messages starts from First Duplicate will be ignored
 Channel will be in inactive status (after threshold value exceeds the duplicate count)
 Since Channel is in inactive status, it won’t process the further messages

Processing Logs:

From the above logs, we can observe that

 The Original Message is processed successfully
 The Next message (First Duplicate) is skipped
 Duplicate alerts are sent to alert inbox
 The channel status has been changed to inactive

SAP COMMUNITY NETWORK SDN - sdn.sap.com | BPX - bpx.sap.com | BA - boc.sap.com | UAC - uac.sap.com
© 2011 SAP AG 6
 NetWeaver PI 7.30 File/FTP Adapter - Duplicate Message Handling Feature

Sample Scenario 3:
Channel Configuration:

Scenario Description:
 The same file will be processed 4 times ( Original Message + 3 Duplicates)
 All other messages from Fourth Duplicate will be ignored
 Channel will be still in active status (after threshold value exceeds the duplicate count)
 It will be process the further messages with duplicate check

Processing Logs:

SAP COMMUNITY NETWORK SDN - sdn.sap.com | BPX - bpx.sap.com | BA - boc.sap.com | UAC - uac.sap.com
© 2011 SAP AG 7
 NetWeaver PI 7.30 File/FTP Adapter - Duplicate Message Handling Feature

From the above logs we can observe that

 The Original Message is processed successfully
 The Second Message (First Duplicate) is processed successfully and current duplicate count = 1
 The Third Message (Second Duplicate) is processed successfully and current duplicate count = 2
 The Fourth Message (Third Duplicate) is processed successfully and current duplicate count = 3
 The Fifth Message (Fourth Duplicate) is skipped
 Duplicate alerts are sent to alert inbox
 The channel status has been changed to inactive

Sample Scenario 4:
Channel Configuration:

Scenario Description:
It is same as scenario 3 except,
 Channel will be in inactive status (after threshold value exceeds the duplicate count)
 Since Channel is in inactive status, it won’t process the further messages

More about Duplicate Message Handling

1) The duplicate count is maintained for every file(with name, size and last modified combination)
2) If you change any content or name of the file, the size (or) last modified time will also be changed.
So the duplicate count for the current file will be reset
3) The channel will never be disabled. It always changed to inactive based on the threshold
configuration
4) If you want to stop/start the inactive channel again, irrespective of the duplicate count, the same file
will be processed one more time
5) This feature is applicable only for File/FTP Sender Channel
6) The similar kind of feature is available for JMS and Mail Sender Adapters

SAP COMMUNITY NETWORK SDN - sdn.sap.com | BPX - bpx.sap.com | BA - boc.sap.com | UAC - uac.sap.com
© 2011 SAP AG 8
 NetWeaver PI 7.30 File/FTP Adapter - Duplicate Message Handling Feature

Related Content
1. Configuring the Sender File Adapter

2. Configuring the Sender FTP Adapter

3. What's New in SAP NetWeaver 7.3

SAP COMMUNITY NETWORK SDN - sdn.sap.com | BPX - bpx.sap.com | BA - boc.sap.com | UAC - uac.sap.com
© 2011 SAP AG 9
 NetWeaver PI 7.30 File/FTP Adapter - Duplicate Message Handling Feature

Copyright
© Copyright 2011 SAP AG. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG.
The information contained herein may be changed without prior notice.
Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.
Microsoft, Windows, Excel, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation.
IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5, System x, System z, System z10, System z9, z10, z9,
iSeries, pSeries, xSeries, zSeries, eServer, z/VM, z/OS, i5/OS, S/390, OS/390, OS/400, AS/400, S/390 Parallel Enterprise Server,
PowerVM, Power Architecture, POWER6+, POWER6, POWER5+, POWER5, POWER, OpenPower, PowerPC, BatchPipes,
BladeCenter, System Storage, GPFS, HACMP, RETAIN, DB2 Connect, RACF, Redbooks, OS/2, Parallel Sysplex, MVS/ESA, AIX,
Intelligent Miner, WebSphere, Netfinity, Tivoli and Informix are trademarks or registered trademarks of IBM Corporation.
Linux is the registered trademark of Linus Torvalds in the U.S. and other countries.
Adobe, the Adobe logo, Acrobat, PostScript, and Reader are either trademarks or registered trademarks of Adobe Systems
Incorporated in the United States and/or other countries.
Oracle is a registered trademark of Oracle Corporation.
UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.
Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of
Citrix Systems, Inc.
HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts
Institute of Technology.
Java is a registered trademark of Oracle Corporation.
JavaScript is a registered trademark of Oracle Corporation, used under license for technology invented and implemented by Netscape.
SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP Business ByDesign, and other SAP products and services mentioned
herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries.
Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and
other Business Objects products and services mentioned herein as well as their respective logos are trademarks or registered
trademarks of Business Objects S.A. in the United States and in other countries. Business Objects is an SAP company.
All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document
serves informational purposes only. National product specifications may vary.
These materials are subject to change without notice. These materials are provided by SAP AG and its affiliated companies ("SAP
Group") for informational purposes only, without representation or warranty of any kind, and SAP Group shall not be liable for errors or
omissions with respect to the materials. The only warranties for SAP Group products and services are those that are set forth in the
express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an
additional warranty.

SAP COMMUNITY NETWORK SDN - sdn.sap.com | BPX - bpx.sap.com | BA - boc.sap.com | UAC - uac.sap.com
© 2011 SAP AG 10