1 CIF Cookbook Version 2
1 CIF Cookbook Version 2
1 CIF Cookbook Version 2
Version 2.0
(This document is subjected to continuous improvement. All feedback please address to [email protected] and/or
[email protected] and/or [email protected])
Contents
Contents ............................................................................................................................................. 1
Introduction......................................................................................................................................... 4
Necessary Knowledge........................................................................................................................ 5
Chapter 1: APO Core Interface Overview .......................................................................................... 6
Transfer Technique......................................................................................................................... 6
qRFC Without Inbound Queues (Only Outbound Queues) ........................................................ 7
qRFC With Inbound Queues....................................................................................................... 7
Logical Unit of Work and Queued RFC ...................................................................................... 8
Integration Model ............................................................................................................................ 8
Transfer from R/3 to APO ........................................................................................................... 8
Transfer from APO to R/3 ........................................................................................................... 9
Data Transfer and Data Channels .............................................................................................. 9
Serialization Effect .................................................................................................................... 10
Chapter 2: CIF Monitoring Tools ...................................................................................................... 12
SMQ1 (Outbound monitoring) ...................................................................................................... 12
Outbound QUEUE Overview .................................................................................................... 13
Outbound Queue MENU OPTIONS.......................................................................................... 13
Outbound Queue STATUS SCREEN ....................................................................................... 13
Outbound Queue STATUS ....................................................................................................... 14
Outbound Queue LUW DETAIL................................................................................................ 15
SMQ2 (Inbound monitoring) ......................................................................................................... 17
© 2006 SAP AG
APO Core Interface Monitoring Cookbook
© 2006 SAP AG
APO Core Interface Monitoring Cookbook
Tools ......................................................................................................................................... 53
Resources/Setting problems..................................................................................................... 53
Long Accesses to Tables.......................................................................................................... 53
Long Process time of Integration jobs (R/3) ............................................................................. 55
Long processing time of publication job.................................................................................... 57
Long accesses to liveCache ..................................................................................................... 58
Appendix 1: List of Transactions ...................................................................................................... 59
Appendix 2: Related SAP Notes ...................................................................................................... 60
Appendix 3: Troubleshooting Flowchart........................................................................................... 61
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 4
Introduction
This document is intended to provide necessary knowledge and monitoring guideline to identify and
analyze problems related to SAP APO Core Interface (CIF). It is not an expert guide that contains
every minor detail of the corresponding topic. Instead this document provides quick, clearly
structured, step-by-step set of instructions – like a cookbook – as a centralized reference of
information and guideline.
The cookbook is separated into 2 sections. With each chapter in each section covering different
aspects of APO CIF. This structure enables reader from diverse CIF knowledge level to use this
cookbook without having to read the whole document.
The first section is providing necessary information surround APO CIF as an introduction and
background of CIF technology.
Chapter 1 “Core Interface Overview”, discuss about what CIF is, what kind of technology it’s based
on and introduction into CIF related terms and definitions.
Chapter 2 “CIF Monitoring Tools”, focus on specific tools use to monitor CIF
Chapter 3 “Performance and Configuration”, provides SAP recommended configuration for CIF;
including RFC and Gateway parameters setting, Integration setting, information on newest version
of qRFC and plug-ins and much more.
Section two is more focus on the Monitoring of CIF. This section provides instruction and guideline
on how to analyze and pint point CIF related problems.
Chapter 4 “General Checks”, covers CIF general checks that have to be verified before further
analysis of the problems.
Chapter 5 “CIF Monitoring and Analysis”, describes step-by-step actions to drill down further
queues errors and or CIF performance problems.
Last but not least, please find in the Appendix a further list of CIF related transactions, related SAP
notes and flowchart for troubleshooting.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 5
Section 1:
Necessary Knowledge
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 6
Transfer Technique
The SAP APO CIF uses queued remote function calls (qRFCs) provided by SAP Technology to ensure the
desired sequence and transactional security of data transmissions between SAP R/3 and SAP APO. With
qRFCs, asynchronous data transfers between SAP APO and SAP R/3 are established, thus enabling
business process steps to be finished in either one of the systems, without the need to wait until the data is
actually transferred to the other system(s).
qRFC itself is an enhancement of the tRFC (transaction Remote Function Calls) technology. The qRFC
requests are placed into a queue; they are processed in the correct sequence based on their dependencies to
other qRFC requests. Preceding requests have to be completed before subsequent requests are processed.
The qRFC requests can be collected to LUWs (Logical Unit of Work). One LUW on the sending system will
refer to exactly on LUW in the receiving system.
There is a queue for every logical unit of business objects like material changes, i.e. objects that logically
belong together are in the same queue, and they are processed one after the other (serialization). The queue
sequence will be guaranteed by the usage of identical queue names. The serialization is achieved via a queue
identifier that is comprised of: Client, Queue-Name, and Destination & Queue-Counter
Example:
Goods receipt for purchase orders is processed with a LUW with multiple function calls. The LUW guarantees,
that stock increase will only be processed in combination with the purchase order closing. All queues of the
LUW have to be processed, before the complete LUW can be finished.
qRFC with inbound and outbound queues involves a 3-phase processing and transfer model. All three phases
are completely independent of each other. Separation of these phases ensures that asynchronous processing
is as secure as possible. In the first step, the application data is written to the database in the outbound
queue. When the first step is completed, the data is saved in the database. In the second step, the QOUT
Scheduler transfers this data from the database of the client system, to the inbound queue in the database of
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 7
the target system. In the third step, the target system QIN Scheduler activates processing of the queue in the
target system. The following diagram illustrates the 3-phase communication concept:
Application
Application 1) Data security in the outbound queue
Database
Database 3) Processing
2) Transfer
QOUT Scheduler
QIN Scheduler
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 8
Integration Model
An integration model is used to control the data transfer from R/3 to APO. It is created in R/3 and gathers
together the data sets (master and transactional data) that are required for APO planning processes. An
integration model is uniquely defined by its name and application and is related to a dedicated target system.
By activating a generated integration model the actual data transfer is started. Typically there are models for
master data and models for transaction data.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 9
model. Thus, fewer amounts of data have to be transferred. Objects that have been changed in the
meantime are transferred, too. The ALE change pointers are evaluated for materials, customers,
vendors and sources of supply. Defining jobs enables the system to regenerate and then activate the
integration models in regular intervals in order to notify APO about a changed master data situation.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 10
Serialization Effect
When using qRFCs for a data transfer, the system adheres strictly to the sequence of objects to be transferred
(serialization), in contrast to the transfer using tRFC. Serialization applies if queues entries from different
logical units of work (LUWs) are being processed in this queue. For example, this could be the case if a
number of changes are made for an order in SAP R/3 at different points in time. The changes are in the same
queue due to the queue name and they are therefore subject to serialization. However, since the postings took
place at different points in time, the corresponding queue entries are in different LUWs. If a queue contains a
faulty queue entry, the subsequent queue entries will also not be transferred. This leads to a queue block. A
queue block not only affects the LUW containing the faulty queue entry, but also all LUWs containing
subsequent queue entries. This effect is called the serialization effect. This is a potential consequence of a
faulty queue entry during a data transfer using qRFC.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 11
Serialization effect can be suppressed (minimized) by new functionality of CIF Error Handling. For further
details see chapter 2 subchapter CIF Error Handling.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 12
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 13
Jump to hanging
queues display
Columns explanation:
• Cl: Client number where the qRFC was generated.
• Queue name: Name of the queue (generated from the application program). Clicking on this will
open a detailed screen for the corresponding queue.
• Destination: Destination that will be used within the R/3 or APO system, this destination name must
be present in SM59. Clicking on this will forward you to SM59.
• Entries: Number of LUWs in the queue.
Jump to running
queues display
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 14
READY
The queue is ready for transmission. This status should only be a temporary status. However, in the following
case this status can also be a permanent status: a queue was locked manually via transaction SMQ1 or via a
program and then unlocked without being activated at the same time. This queue must be activated explicitly.
RUNNING
The first LUW of this queue is currently being processed. If a queue in this STATUS hangs for more than 30
minutes, this may mean that the work process responsible for sending this LUW has terminated. In this case
you can activate this queue again. Note that activating a queue in status RUNNING may cause a LUW to be
executed several times if this LUW is processed in the target system at that time. We therefore recommend a
waiting time of at least 30 minutes before you activate the queue again.
EXECUTED
The first LUW of this queue is processed. The system waits for a qRFC-internal confirmation from the target
system before further LUWs are processed. If a queue in this STATUS hangs for more than 30 minutes, this
might mean that the work process responsible for sending this LUW has terminated. In contrast to status
RUNNING, this current LUW has definitely been executed successfully. You can activate this queue again
without problems. The qRFC Manager will automatically delete the LUW already executed and send the next
LUW.
SYSLOAD
At the time of the qRFC call, no DIALOG work processes were free in the sending system for sending the
LUW asynchronously. A batch job for subsequent sending has already been scheduled (see SAP Note
319860 for more details).
SYSFAIL
A serious error occurred in the target system while the first LUW of this queue was executed. The execution
was interrupted. When you double-click on this status, the system displays an error text. You can find
additional information on this error in the corresponding short dump in the target system (ST22). No batch job
is scheduled for a repetition and the queue is no longer processed. To solve the problem, information from the
affected application is required. Refer to SAP Note 335162 for the special error text "connection closed".
CPICERR
During transmission or processing of the first LUW in the target system, a network or communication error
occurred. When you double-click on this status, the system displays an error text. You can find additional
information on this error in the syslog (SM21), the trace files dev_rd or dev_rfc*. Depending on the definition in
transaction SM59 for the destination used. A batch job is scheduled for subsequent sending. Status CPICERR
may also exist in the following cases although no communication error occurred: a qRFC application finds out
that a LUW cannot be processed any further due to a temporary error in the application and therefore calls the
RESTART_OF_BACKGROUNDTASK function module to prompt the qRFC Manager to cancel the execution
of this LUW and to repeat this LUW later in accordance with the specification in transaction SM59.In this case,
qRFC simulates a communication error with the text "Command to tRFC/qRFC: Execute LUW once again." If
this error occurs very often, you must contact the corresponding application team.
STOP
On this queue or a generic queue (for example BASIS_*) a lock was set explicitly (SMQ1 or programs). Note
that the qRFC never locks a queue in its processing. After having informed the corresponding application
team, you can unlock and activate this queue using transaction SMQ1.
WAITSTOP
The first LUW of this queue has dependencies to other queues, and at least one of these queues is currently
still locked.
WAITING
The first LUW of this queue has dependencies to other queues, and at least one of these queues contains
other LUWs with higher priorities.
NOSEND
LUWs of this queue are never sent but retrieved by a special application. These queues are only used
internally at SAP (BW or CRM during communication with Mobile Clients). Even if a LUW was read by the
corresponding application (BW, CRM), this status does not change. This LUW is only deleted from the queue
if this application confirms collection (collective confirmation possible). Under no circumstances should this
status be reset using transaction SMQ1 and the queue activated.
NOSENDS
During the qRFC call, the application determines at the same time that the current LUW is not sent
immediately. This is used to debug the execution of an LUW via transaction SMQ1. Contact the corresponding
qRFC application to clarify this status since this is either a programming or configuration problem.
WAITUPDA
This status is set if qRFC is called within a transaction that also contains one or more update functions. As a
result of this status, the LUW and thus the queue is blocked until the update has been completed successfully.
If this status takes longer than a few minutes, check the status of the update or the update requests using
transaction SM13. After a successful retroactive update, the blocked LUW is sent automatically. You can also
restart the LUWs manually in the WAITUPDA status without a successful retroactive update (via transaction
SMQ1, Reset status, Activate queue). However, you may only perform this action after having informed the
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 15
qRFC application (for example, APO, BW, CRM). You must do this to avoid possible inconsistencies. This
WAITUPDA problem may be avoided as follows: If both qRFC calls and update calls occur within a
transaction, qRFC must be executed exclusively within the update. In this case, the qRFC LUW is only
created after the update has been completed successfully. Caution: If you are using SAP Basis Releases
4.0x, 4.5x, 4.6A or 4.6B, and an update function with the "collective run" type exists in a LUW, an error in the
kernel may cause this status. The queue also hangs in this case. This error has already been corrected with a
kernel patch (see SAP Note 333878).
RETRY / ARETRY
During LUW execution the application has diagnosed a temporary problem and has prompted the qRFC
Manager in the sending system via a specific qRFC call to schedule a batch job for a repetition on the basis of
the definition in transaction SM59 (Menu: Destination > TRFC Options)
ANORETRY
During the LUW execution, the application has found a serious error and prompted the qRFC Manager via a
specific qRFC call to cancel processing of this LUW. Information from the affected application is required to
solve the problem.
MODIFY
Processing this queue is locked temporarily because the LUW data is being modified.
Remark: Report RSQOWKEX reset the status of all queues (CPICERR, RETRY, SYSFAIL) and
restarts the outbound scheduler. This report can be used in exceptional cases; do not schedule this
report on regular basis, investigate the error before restarting the queue.
Jump to debugging
mode
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 16
• Rpts: Repeats that are planned in. Only if a CPIC error was detected repeats will be planned in. This
could be steered in SM59 (TRFC Options). If no entries were made the default values will be used
(30 attempts, one each 2 minutes).
When you are in the detailed view of a LUW (as shown in the screenshot above) the content of the LUW can
be shown by double clicking on the queue name. Depending on the setup of the system you will get a queue
display in binary mode (which is not very helpful for analyzing purposes) or you will get a clear overview of all
related tables in DDIC format and their content.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 17
Queue display on R/3 side (planned order with all relevant data, e.g. activities, BOM)
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 18
STOP
On this queue or a generic queue (for example, BASIS_*) a lock was set explicitly (SMQ2 or programs). Note
that the qRFC never locks a queue in its processing. After having informed the corresponding application team
you can unlock and activate this queue using transaction SMQ2.
WAITSTOP
The first LUW of this queue has dependencies to other queues, and at least one of these queues is currently
still locked.
WAITING
The first LUW of this queue has dependencies to other queues, and at least one of these queues contains
other LUWs with higher priorities.
NOEXEC
During the qRFC call, the application team simultaneously determines that the current LUW is not processed
automatically even if the queue to the QIN Scheduler (SMQR) is registered. This is used to debug the
execution of an LUW via transaction SMQ2. Contact the corresponding qRFC application to clarify this status
since this is either a programming or configuration problem.
RETRY
During LUW execution, the application has diagnosed a temporary problem and has used a specific qRFC call
to prompt the qRFC manager in the sending system to schedule a batch job. This batch job schedules a
repetition after two minutes.
ARETRY
During LUW execution the application has diagnosed a temporary problem and has prompted the qRFC
Manager in the sending system via a specific qRFC call to schedule a batch job for a repetition on the basis of
the registration in transaction SMQR.
ANORETRY
During the LUW execution, the application has found a serious error and prompted the qRFC Manager via a
specific qRFC call to cancel processing of this LUW. Information from the affected application is required to
solve the problem.
MODIFY
Processing this queue is locked temporarily because the LUW data is being modified.
Remark: Report RSQIWKEX reset the status of all queues (CPICERR, RETRY, SYSFAIL) and
restarts the inbound scheduler. This report can be used in exceptional cases; do not schedule this
report in regular basis, investigate the error before restarting the queue.
Remark: To distribute resources on the receiving system a logon group should be defined, using
/SMLG. In this way, you can exclude a specific server from the RFC processing (e.g. the database
server) to avoid overload
Jump to /SMQ1
• Scheduler Status: Status of the Scheduler. To see the current status you should refresh the screen.
• Last update: Time and date of the last activation of the scheduler.
• Name of AS group: Shows the name of the used RFC server group. The group name ‘DEFAULT’ is
standard and uses all available application servers in the R/3 system. The RFC server group can be
changed via menu ‘Edit’ ‘Change AS group’. Please be sure that the here used RFC group is
available in RZ12. It is not necessary to maintain AS group ‘DEFAULT’ in RZ12.
• Number of Entries Displayed: Number of the registered destinations
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 19
Columns explanation:
• Clt.: Client where the scheduler works for
• Dest.: Registered destination
• Type: Shows whether the destination is registered ( R ) or unregistered ( U ). You can change to
unregistered, if you first mark the queue and then select the Deregistration button.
• Max. connection: determine maximum work processes that are used in sending system (and target
system, if inbound is not set) to transfer the qRFC LUWs to the specified destination.
• Max. runtime: determine maximum time (in seconds) the scheduler needs to process/sort the
queues in one roundtrip. This is necessary for the scheduler to have a fair handling of a large number
of different registered destinations. New coming queues will not be included in the process until the
max time is reached or scheduler finish sorting/processing the current queues. When the max
runtime is reached, queues that are not yet sorted/processed will be process together with the new
coming queues.
• (In later Scheduler release, addition column is added) Act. Conn.: determine how many active
connections currently exist (active).
• Status: Status of the scheduler for a certain destination.
Reorganize Deletes all registrations. This does not affect the content of the tRFC/qRFC.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 20
Change AS group The Scheduler starts a server group for processing LUWs. The application
server group used is displayed in the third column. If the name of the
application server group has been changed (#DEFAULT), ensure that the
server group exists in transaction RZ12.
Activate Scheduler Start the Scheduler. The last start time is displayed in the second line in the
top left corner. The status in the top line changes throughout the runtime.
The status is now Active.
Change View Displays the status of the Scheduler.
This transaction is for displaying the state of the QIN-Scheduler and for register or deregister a queue. The
scheduler status could be seen at the top of the screen.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 21
By double clicking the corresponding line you can reach the first sub screen of SMQ2 to see more details
about the inbound queue content.
QIN Scheduler STATUS
The following statuses are possible:
• INACTIVE: The QIN Scheduler is not active.
• BATCHJOB SCHEDULED: The QIN Scheduler has scheduled a batch job.
• STARTING: The QIN Scheduler is currently in the START phase.
• ACTIVE: The QIN Scheduler is active.
• WAITING: The QIN Scheduler waits for free DIALOG work process.
• SYSFAIL: A serious error occurred in the current operations of the QIN Scheduler. Double-clicking
the status displays the respective error text. Other analyses of the syslog (SM21) or development
traces (dev_w*, dev_rfc* and dev_rd) may be required for the fault determination.
• CPICERR: A communication error occurred in the current operations of the QIN Scheduler. Double
clicking the statuses displays the respective error text. Other analyses of the syslog (SM21) or
Development traces (dev_rd, dev_rfc* and dev_w*) may be required for the fault determination.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 22
Queue status
Expand the root nodes by clicking on the node. The object types of the selected system are displayed. The
main window will display all queues for a particular system by double-clicking on the system name.
A symbol tells you the queue status of the individual objects:
No queues with errors
Queues are recorded
Queues manually stopped
Queues with errors
Queues are being processed
Queue is waiting for another queue to be processed
Column explanation:
• Queue name: Name of the relevant queue
• Destination: Destination system of the queue. By double-clicking on the destination column brings
you to the transaction SM59. Here, you can edit the settings for the connection to the relevant
system.
• Error text: Description of the queue status
• User: Person responsible for the queue
• Function module: Function module concerned
• Date/time: Time of the queue error
• Waiting: Queue is dependent on other queues
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 23
Prerequisites
To use the CIF cockpit, you need an RFC user and dialog authorization for every connected R/3 system. It is
recommended to create a special RFC destination for the CIF cockpit application for each system connected
to SAP APO. By this way the authorization of the user using the CIF cockpit can be restricted specifically for
using the CIF cockpit.
You can do this in the mySAP SCM Implementation Guide (IMG) under Integration with SAP Components →
Integration of SAP SCM and SAP R/3 → Basic Settings for Creating the System Landscape → Assign RFC
Destinations to Various Application Cases.
Settings Features:
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 25
In the ‘Settings’ area of the CIF cockpit you find the following information:
• SAP APO system
o Distribution Settings
Object specific settings for publication are read from the customizing (SAPAPO/CP3)
o Enhancement Projects
all user-exits (active and inactive) are shown (not only CIF related)
o under further settings
user parameter: user specific parameter from /SAPAPO/C4 are shown
strategy profiles: gives an overview of the existing strategy profiles used in PPDS for
scheduling
SNP settings: shows the setting regarding the transfer of SNP orders to an OLTP
system
• SAP R/3 system
o Block Sizes of Initial Data transfer
shows the settings regarding the filter and selection block size maintained in CFC3
o Enhancement Projects
All user-exits (active and inactive) are shown (not only CIF related)
o Further settings
Additional CIF related settings regarding change transfer of master data, initial data
transfer and change transfer for Resources. All of these settings are maintained in the
customizing transaction CFC9 in the R/3 system
• Transfer from SAP R/3 system to SAP APO system
o Integration model
Provides a list of all integration models (active and inactive) in the connected R/3
system with destination to the APO system. The list contains also the information about
the number of specific objects in each model.
o qRFC
Information about the installed qRFC version in the R/3 system and its supplement is
given. Also the queue type (inbound or outbound) is shown and the overall number of
resources (dialogue processes) and the number of resources which are reserved for
qRFC only.
o Inspection Lots
Setting regarding inspection lots is shown (transfer inspection lots or not)
o Returns
Setting for transferring SD returns from R/3 to APO (returns and return deliveries in
SAP APO Transportation Planning/Vehicle Scheduling (TP/VS)) => release information
to be added
• Transfer from SAP APO system to SAP R/3 system
o Distribution Criteria
Display of the distribution definition (maintained in customizing)
o qRFC
Information about the installed qRFC version in the APO and its supplement is given.
Also the queue type (inbound or outbound) is shown and the overall number of
resources (dialogue processes) and the number of resources which are reserved for
qRFC only.
o Error Handling
The type of error handling for CIF queues is shown (post processing records yes/no)
which is maintained in customizing in the maintenance transaction for ‘Assignment of
Logical System to Business System Group’.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 26
Monitoring Features:
In the ‘Monitoring’ section of the CIF cockpit you will find the following information:
• SAP APO system (in the example screen shot from system SMACLNT800)
information about installed software components, release and patch level
• CIF Background Jobs
you will get the information about the status of jobs which are scheduled in the APO system.
Only those jobs are shown for which an entry has been made in the user profile of the CIF
cockpit. Every type of program can be entered here, not only CIF related programs.
• Application Log
the application log of the APO is read according to the settings in the profile
• Comparison of Routing information (independent from target system)
unnecessary routing information from table /SAPAPO/CIFBEFCR are shown (look-up
information without a corresponding entry in table /SAPAPO/CIFLOOKU)
If there are entries shown: note 567601 provides a correction report to delete those entries.
• SAP R/3 system (in the example screen shot: form system GSBCLNT801)
information about installed software components, release and patch level
o CIF Background Jobs
you will get the information about the status of jobs which are scheduled in the R/3
system. Only those jobs are shown for which an entry has been made in the user profile
of the CIF cockpit. Every type of program can be entered here, not only CIF related
programs.
o Application Log
the application log of the R/3 system is read according to the settings in the profile
o Superfluous Filter Objects
The number of superfluous filter objects is shown for every type of filter object. From
the display of these objects you can jump to the corresponding R/3 system into the
report RCIFIMDL which can be used then to delete these filter objects (for more
information on this report see also note 593355).
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 27
With this function, the runtime is measured inside the CIF inbound function modules
itself (e.g. /SAPAPO/CIF_ORDER_INBOUND). The results can be evaluated in the CIF
Cockpit with different granularity (day, hour, minute) depending on the settings made in
the user profile.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 28
If the CIF error handling is active, the system does not create faulty queues in the inbound or outbound queue
anymore.
If a change to transaction data cannot be posted in the target system due to an error, the system creates a
post processing record with the processing status 1 (Still for Processing) and the error status 2 (error).
The system also creates post processing records for all other objects of the qRFC LUW in which the error
occurred (dependent post processing records).
This has the advantage that queues in the inbound/outbound of a system are no longer blocked because of
SYSFAIL’s and the above shown situation should not occur anymore.
The post-processing record is always stored in the target system. On APO side the record is stored in table
/SAPAPO/CIFERRLG, on R/3 side in table CIFERRLOG.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 29
Prerequisites
CIF error handling has to be switched on in IMG or in transaction /SAPAPO/C2 (Post processing, no splitting
of LUWS). By default it is switched off.
It is only necessary to switch it on for the SCM system. CIF post processing always works for both directions.
Restrictions
CIF error handling is not available in the following situations, which means that CIF queues hang when errors
occur:
• At the initial data transfer
• At the transfer of master data (initial and change transfer)
• At short dumps
• When the target system or the liveCache is unavailable
• When an object is locked in the target system (as before for the repetition of the transfer)
• If errors occur in customer exits or BAdIs that run in CIF inbound function modules during integration
You can find information about restrictions to CIF error handling when using certain applications and functions
in SAP Note 602484.
The screen is divided into 4 areas: on the left side the navigation tree, on the right side the work list of post
processing records on top, then a list with objects for which the re-processing was started in the current
session and on the bottom a list with messages regarding already processed records.
By clicking on one of the entries in the navigation tree you see the related post processing records in the work
list on the right side (in this case 4 records that were created during the CIF transfer from the system SMA to
GSB).
Note that the ‘Time of Error’ shown for the post processing record is shown in UTC (which might be different
from the system time).
The status of the record is indicated by the color of the line in the work list.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 30
Records in status 1 (not processed) are shown in light blue, status 2 (already processed) is shown in green
and status 3 and 4 ( status obsolete, set manually or automatically by the system) is indicated by the color
grey.
Records can also be marked as obsolete in the work list or the obsolete indicator can be withdrawn from here.
The reprocessing can be triggered by pressing the button for sending the record to APO or the
button for sending to R/3. The reprocessing can only be successful if the error cause for writing the
post processing record has been solved. You can check the reason by analyzing the application log. The
application log for every post processing record can be accessed by the appropriate button on the top of the
work list. If you observe any problems in using this functionality please check note 799647.
Reprocessing a post processing record means that the object for which the error occurs is read once again
from the source system and send to the target system with the actual data. By this way all changes which
were made on this object since the post processing record was written are taken into account.
Once the reprocessing is triggered the chosen records are moved from the work list to the list beneath:
Objects processed in this session.
The refresh button has to be used here to get the information if the reprocessing was successful or not (Note
that the reprocessing will never finish if you have maintained in your user settings that queues are only
recorded. Then in addition you have to execute the queue manually in the queue monitor).
If the reprocessing was successful the record becomes the status 2 (already processed). The record is not
deleted automatically from database, it has to be reorganized by the report
/SAPAPO/CIF_POSTPROC_REORG.
It is possible to set up the system in a kind that automatically a message is sent to a recipient if post
processing records are created (CIF Post processing Alert). The customer can schedule the program
/SAPAPO/CIF_POSTPROC_ALERT as a job to achieve this. Only for those records which are created new
since the last run of this job a message is sent to the recipient.
It is also possible to start the transaction /SAPAPO/CPPA to create a variant for this program or to trigger the
sending of these messages manually.
It is recommended to schedule this program on a regular basis (e.g. daily).
Note that activating CIF error handling leads to a different approach to the monitoring of integration. If CIF
error handling is not active, you first call the SCM queue manager and then the CIF comparison/reconciliation
of transaction data.
If CIF error handling is active, you first check whether post processing records exist, then call the SCM queue
manager, and then the CIF comparison/reconciliation of transaction data.
In case of performance problems based on a high reading time on table CIFERRLOG please see the
recommendation given by note 802980.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 31
You have to enter a value for the target system (the R/3 system). If you don’t see any values in the F4-Help for
this field, then the CIF error handling is not switched on in the customizing (see also note 652885).
On the bottom of the selection-screen you see the flag ‘Select Data from R/3’. This flag is activated in
standard and has the effect that reprocessing records in R/3 are also considered.
In certain scenarios (e.g. APO system is only used for GATP) the activation of this flag will cause the
termination of this program because in such scenarios there is no valid RFC connection defined from APO to
the R/3 system. In these cases the flag has to be deactivated.
You can use a retention period (in days) for the selection of the records which means that only those records
are selected which are older then the given period.
The report can be used for deleting post processing records and also to change the processing status to 3
(Obsolete set manually). But it is not possible to change the status to obsolete and delete the record in one
step.
It is recommended to define a variant for this report and run it on a daily basis as a background job.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 32
To optimize the performance, correct configuration for all layers is needed. This chapter discusses the definition
and recommendation on how to set the parameters, important jobs that have to be scheduled, and other tips and
tricks to optimize CIF performance.
During troubleshooting, these setting recommendations should be verified and exercised to match customer
conditions.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 33
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 34
Level 2: In addition to
level 1, all RFCs which
were started from
asynchronous RFC
sessions are monitored.
This also includes
synchronous RFCs. To
allow applications with a
lot of RFC calls
to run on this level, the
number of dialog
processes usable for
RFC may have to be
increased
Level 3: All RFCs
(synchronous and
asynchronous) are
monitored.
These settings are only effective on the receiver side IF the CIF communication has been switched to inbound.
Please also refer to SAP Note 74141 for setting the above parameters.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 35
SAPNote Description
697884 Queues in SMQ2 are not processed
744869 Performance Optimization Inbound Scheduler
742950 Performance Decrease on Oracle DBs
R3 Plug-In version
It’s relevant for data integration capability between R/3 and APO. For releases of R/3 and ECC 5.0, it‘s delivered
in R/3 Plug-in (PI) component. The last development of R/3 plug-in is 2004.1 SP 10. For releases > = ECC 6.0,
it‘s delivered in SAP application component (SAP_APPL). For further information regarding this, please refer to
service.sap.com alias ‘r3-plug-in’.
Resource Distribution
Use transaction RZ12 to define a group of application servers and their DIALOG work processes that is allowed to
use a QIN/QOUT scheduler. The group name must be assigned to the respective QIN/QOUT scheduler using
SMQR/SMQS. Doing so the qRFC calls are processed on a separate instance without disturbing dialog users.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 36
• Note 328355 describes and provides solution of long access to tables relevant to change pointer
(BDCPV, BDCP, BDCPS), due to storage quality index and table size.
• Note 436687 describes and provide solution of long access to integration model tables (CIF_IM*).
• Starting Oracle 10.G, Rule based optimizer is no longer exists. Due to this, SAP provides a predefined
statistics for those aRFC and tRFC tables. See SAP note 932975
Initial Supply
The initial supply of master and transaction data enables the planning in APO and has to be performed once
before Go Live. Once it is completed only deltas are transferred which is usually not a problem.
Recommendations that require a repeated initial load should be avoided (e.g. initialization of liveCache). Factors
influencing runtime are specified in the following subchapters.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 37
When transferring data via CIF, logs are recorded with object CIF (Core interface application log object). All logs
are given an expiry date by the application itself (for CIF it is 1 week in the future). After the expiry date is passed
logs can be deleted from the database. In order to keep performance optimal it is highly recommended to
schedule the appropriate delete jobs regularly/daily. The following reports are available:
R/3 APO
CIF application logs RDELALOG /SAPAPO/RDELLOG
Other application logs SBAL_DELETE SBAL_DELETE
/SLG2 /SLG2
Attention: liveCache locks can occur when separate APO processes try to access liveCache objects referring to
the same material/plant combination (pegging area). Typical object types causing locks are: Production orders
with operations and components, subcontracting purchase requisitions/orders, stock transport orders/requisitions.
To avoid locks it must be ensured that
• Only different material/plant combinations or resources are processed in parallel
• Identical material/plant combinations or same resources are run sequentially
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 38
RBDCPCLR (transaction /BD22) to delete them. If obsolete and processed change pointers are not deleted
periodically, performance could slow down significantly.
Online Transfer
The Online transfer means the data exchange of transaction data between R/3 and APO. Factors influencing
runtime are specified in the following.
Detailed Logging
Detailed logging can cause large database tables and a loss in performance in productive operation. For that
reason it should only be activated when the data is required. The records in the application log are not
automatically deleted by the system. To prevent the database from overflowing, the records have to be deleted at
regular intervals. See section ‘Application log writing’ under Initial Supply for more information.
Serialization Effect
As explained in the first Chapter, serialization effect can severely restrict data transfer and some data cannot be
transferred at all. Consequently, there are inconsistencies between source and target system.
Consequences:
• Data is completely missing in the target system
• Incorrect of incomplete transfer of data changes due to not transferred data
• Transfer of obsolete data in case queue entries are transferred later after they have been parked/saved
For that reason, it is of utmost importance to rectify incorrect queue entries in time. Monitoring concept/handbook
suitable to the Business Process (BP) is absolutely necessary and has to be established before go live involving
system administration AND business department as well.
The following Notes describe procedures how to prevent queue blockades:
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 39
The serialization effect can be suppressed for the most part if the new CIF error handling is used. Please refer to
chapter /SAPAPO/CPP (Error Handling).
Emergency scheduler
Due to technical problems or downtime of one or several components, a huge number of queue entries has been
accumulated in SCM inbound. With a huge number of queue entries (> 100.000), the inbound scheduler tends to
get very slow in processing them and becomes a bottleneck. To speed up CIF queue processing in SCM inbound
in such situations, the report /SAPAPO/CIF_EMRG_QINSCHED can be scheduled. See Note 869399 for
availability and implementation of this report.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 40
Prerequisite:
Set user logging at least to normal (standard) if not detailed logging. (in APO system use transaction /sapapo/C4
and/or in R3 system use transaction CFC2).
Step by step:
For APO system (but not for R3), you can refer to SAP Note 780034 for creating a report (program) to do this job
in more automatic way.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 41
Section 2:
Monitoring Roadmap
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 42
Alternative, you can check field QUEUETYPE of table CIFOPMODE for the appropriate logical system:
QUEUETYPE = I Inbound queues
QUEUETYPE = ‘ ‘ Outbound queues
APO Execute transaction /SAPAPO/C2, check column ‘Queue Type’ and column ‘Err.Hndlg.’:
If post-processing is used (see chapter /SAPAPO/CPP Error Handling), faulty queue entries are to be
checked using transaction /SAPAPO/CPP. In this case, it is very unlikely that queues are blocked.
The qRFC versions and supplements are compatible with each other. It is not necessary for the sending
system or receiving system to have the same version or supplement number.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 43
- Queue entries remain in state READY and scheduler (inbound / outbound) is continuously in state
WAITING.
- Destinations (outbound) or queue names (inbound) are not registered properly.
Determine the number of work processes available for qRFC / tRFC. Call transaction /SMQR or / SMQS,
choose Goto -> Queue RFC resources:
If DIA WPs for tRFC/qRFC are constantly exhausted (DIA-WPs for tRFC/qRFC = 0), this indicates a resource
problem. Either the RFC resources are not sufficient to accommodate the load or the qRFC processing is to
slow. Note that the number of available resources in the system is a snapshot which relates to the load status
of the system.
Check the entire RFC configuration of the connected systems (R/3 and SCM) according SAP Note 527481.
The following general rules must be fulfilled to guarantee that sufficient resources (work processes) are
available for qRFC processing:
- The number of DIA processes must be greater than or equal to the non-DIA processes for each instance
that is available for RFC processing and must be configured across all operation mode switches.
- The number of DIA processes available for RFC processing must be configured large enough to
accommodate the load of cascading RFC calls (e.g. function modules using ‘UPDATE TASK LOCAL’).
See Note 726148.
Use transaction RZ12 to verify the relevant parameters of the RFC server group assigned for RFC processing
(if is different from DEFAULT).
rdisp/rfc_use_quotas
rdisp/rfc_max_queue
rdisp/rfc_max_login
rdisp/rfc_max_own_login
rdisp/rfc_max_own_used_wp
rdisp/rfc_min_wait_dia_wp
rdisp/rfc_max_comm_entries
rdisp/rfc_max_wait_time
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 44
Having the necessary authorization, the values can be changed dynamically in transaction RZ12 (or RZ11).
Alternatively, the report RSARFCLD can be used to dynamically configure the RFC quotas on the server to
which you are logged on. Unlike parameter changes in the profile, these settings are lost when you next
restart the instance.
Using these transactions you are able to examine whether CIF problems are caused by a lack of available
RFC resources as well as configure them correctly to accommodate the CIF load.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 45
Monitoring steps:
1. Queue overview:
TA /SMQ2 (inbound) or /SMQ1 (outbound), leave field client as it is, enter CF* in field queue name, press
Execute.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 46
Monitoring step:
TA /ST11: Obtain further information from trace files dev_rd, dev_rfc* and dev_wp<n>, if problem persist for
longer time.
SYSFAIL
A high number of SYSFAIL entries in connection with the serialization effect (queues are in state WAITING on erroneous
queue) might lead to stuck queues. Since the number of entries is continuously growing the qRFC processing is slowed
down.
Monitoring steps:
1. Find error text:
Double-click on SYSFAIL.
b. TA /CFG1 (R/3) or TA /SAPAPO/C3 (APO), enter TID obtained from the queue monitor, press
Execute. If you cannot determine the TID, restrict the time frame as far as possible.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 47
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 48
Complex Analysis
SYSFAIL problems
SYSFAIL errors may have various reasons. They must be rescheduled manually after the error was solved.
Those errors are always recorded in the application log independent of the user settings (No logging, Normal,
Detailed logging).
Monitoring steps:
1. Find detailed error message:
Double-click error message in application log. Determine message number and long text, if existing.
2. Get detailed information about error codes, if not in application log visible:
a. For messages of type /SAPAPO/OM:
Transaction /SE38: display the report /SAPAPO/OM_ERROR_CODES and find your error code.
Transaction /SAPAPO/OM10, enter the error number. Detailed error message is shown (only
available in GERMAN).
Detailed LC error messages and their meaning can be obtained in SAPNet on the APO
developer page LC application errors. Number and short description should be included in case
an OSS message has to be created.
b. For all other messages of types:
Transaction /SE91, enter message class and number.
As result a list is shown with the number of integration models, where the object is contained in. From
here, drill-down is possible to the integration models.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 49
RETRY Problems
The RETRY status occurs in general when another user or process already locks data (usually on location
product level); therefore the queue processing is stopped. The queue gets the status RETRY and the inbound
scheduler will restart the queue depending on its settings.
Entries in status RETRY are not properly processed by the inbound scheduler, but must be triggered and
executed by a background job. This job named QRFC:<TID> schedules a repetition with a fixed period of 2
minutes. If locks frequently occurs that leads to enormous delay in inbound processing and to heavy system
load (CPU utilization).
Further bottlenecks are likely to occur because the number of background work processes in the system is
limited and the background scheduler assigns them work with a period of 60 seconds (default value for
rdisp/btc_time) though the runtime of the qRFC jobs is usually very short. Between 2 starts of the background
scheduler a lot of new entries might be created which leads to the consequence that the jobs are delayed up
to several hours.
Product allocation
A blocking conflict can happen during an update of sales orders and/or deliveries that contain items, which are
using product allocations within their ATP set up. That is, customers are working with product allocation with a
direct link to the planning area.
Locks are set only for the characteristics combination (location product) of the order that is currently
processed. If there are several sales orders (in different queues) having identical characteristics combinations
those cannot be processed in parallel but have to be processed one after the other. When events for such
orders are send simultaneously only the first will get the lock and will be processed, but all the following ones
will have to wait and will get the RETRY status because of the lock.
Forecast consumption
In a very general sense, forecast consumption is the comparison of two order types. Often SAP APO planned
independent requirements are consumed by other order types, such as sales orders, dependent demand, or
transfer requests. This functionality is fully integrated which means that a change of a consuming requirement
(e.g. sales order) is taken into account immediately.
Anyway, the pegging area of the corresponding product stored in liveCache needs to be updated.
To ensure consistency of data changes (forecast quantities and/or pegging area), SAP locks are set during
forecast consumption. If several sales orders arrive with identical location products, this can result in lock
problems and thus repetitive posting of the sales order. The higher the parallel load the more sales orders can
not be posted and are sent to RETRY by the qRFC scheduler.
This proceeding leads to problems in the area of consumer goods. Typically, those customers are having a
small number of products contained in nearly each order (top-sellers).
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 50
3. This leads to enormous processing delay (Example: LUW processed 5 times, first start time of LUW
03:27:51, last start time 03:43:51 => 15 minutes delay).
Alternatively, obtain number of qRFC jobs scheduled per day/per hour using /SE16, table TBTCP, enter
the fields JOBNAME (= qRFC*), SDLDATE and SDLTIME, press ‘Number of entries’:
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 51
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 52
These monitoring steps are useful to find out that massive occurrence of RETRYs hinder the CIF queue
processing. A general solution how to solve those problems cannot be provided in this document as it requires
deep knowledge of the business process in general and how the customer has implemented it. For that
reason, do NOT recommend any notes or workarounds without discussing the problem with application back
office.
BASIS Workaround for Basis release < 7.0 (you need the cooperation with back office to set this):
If application solution is exhausted, then other option is to speed up background scheduling:
• Setting the parameter rdisp/btctime to 30 seconds, which can be done using transaction /RZ11
without restarting the instance.
• Or by the usage of the so-called quick start program. This program serves as a job starter; see Note
599835 for details.
Report RSBTCSEXE:
With SAP Note 923228, this workaround become obsolete as enhancement of background job
scheduling (see SAP Note 923228 for details) is provided. This enhancement is delivered by default
starting basis release 7.0. For basis release below 7.0, a step by step configuration and coding
correction is needed to activate this enhancement.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 53
Resources/Setting problems
Wrong configuration of CIF setting/parameters and lack of resources can slow down the CIF transfer/process
or even worst, block the whole system.
We can monitor this using transaction /SM50 during CIF transfer/process. Observe work processes with RFC
user (i.e. ALEREMOTE).
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 54
Program RSTRFCCS (New enhanced version, available in SCM 5.0. Screenshot taken from test system Q1P)
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 55
Long access time to application log tables (both R/3 and SCM)
1. Check the logging level of the users used for RFC communication in the affected system (transactions
R/3 CFC2, SCM /SAPAPO/C4).
2. For Oracle databases: Check index storage quality of these tables. If index storage quality is bad,
recommend to recreate indexes regularly.
3. Check the size or the number of entries. If size or number of entries is huge check whether the delete
jobs are scheduled regularly (SBAL_DELETE in both systems, R/3: RDELALOG, SCM:
/SAPAPO/RDELLOG).
The lack of those options may cause high database request time because a large number of material /plant
combinations have to be read.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 56
2. A high number of superfluous filter objects hinder performance during activation of integration models. For
that reason it is recommended to delete unused entries for the appropriate tables. If the number of
unused filter objects is very high (> 200.000) the deletion must be performed in several steps. For details,
see Note 593355.
3. Check access path used if SQL trace is available. In some cases, indexes are missing (see Notes
844325, 799678).
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 57
1. The runtime per object type (purchase requisition = external procurement, planned / production order = in-
house production) depends on the block size for publication.
Check the block size using transaction /SAPAPO/CP3 and refer to Chapter 3 ‘Performance and
Configuration’ subchapter ‘Initial Supply’ and/or ‘Online Transfer’.
2. The publication job itself works in packages, that means a package of <n> objects is processed and sent
to R/3. The standard value for package size is 1000 and this should not be changed. The processing of
each package is recorded in the job-log of the appropriate background job:
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 58
3. Various selection options are available to set up variants for the job. It is possible to run disjoint variants in
parallel if the queues are processed fast enough.
Check the variant.
If you need to evaluate the runtime in detail, ask for creation of a variant with a limited number of change
pointers to be published.
Further analysis is not covered in this cookbook and should be carried on by liveCache expert.
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 59
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 60
© 2006 SAP AG
APO Core Interface Monitoring Cookbook 61
© 2006 SAP AG