Extracting Sales and Distribution Transaction Data from BW 2.

0B
BUSINESS INFORMATION WAREHOUSE 2.0B

Version 1

Last Changed on: 07.12.2000

SAP AG assumes no responsibility for errors or omissions in these materials. These materials are provided as is without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP shall not be liable for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. SAP does not warrant the accuracy or completeness of the information, text, graphics, links or other items contained within these materials. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third party web pages nor provide any warranty whatsoever relating to third party web pages.

2000 SAP AG

Contents
1 2 PREFACE ...................................................................................................................................................... 3 NEW EXTRACTORS/DATASOURCES WITH BW RELEASE 2.0B/R/3 PLUG-IN PI 2000.1 .......... 5 2.1 THE DESIGN OF THE NEW EXTRACT STRUCTURES ................................................................................... 6 2.2 ADMINISTRATIVE TABLES FOR DEFINING EXTRACT STRUCTURES AND FOR LINKING EXTRACT STRUCTURES TO DATASOURCES .......................................................................................................................... 9 3 PREPARED STEPS FOR EXTRACTION .............................................................................................. 10 3.1 TRANSFERRING BUSINESS CONTENT DATASOURCES ............................................................................. 10 3.2 MAINTAINING EXTRACT STRUCTURES ................................................................................................... 10 3.3 MAINTAINING THE DATASOURCE .......................................................................................................... 11 3.3.1 Canceling Fields ........................................................................................................................... 11 3.3.2 Specifying Selection Fields for InfoPackages ............................................................................... 11 3.3.3 Hiding Fields in BW ...................................................................................................................... 12 4 EXTRACTING TRANSACTION DATA ................................................................................................. 13 4.1 INITIALIZING TRANSACTION DATA (SETTING UP EXISTING DOCUMENTS) ............................................. 13 4.1.1 Special Features when Running the Set up in the OLTP ............................................................... 13 4.1.2 Extraction with Update Mode Full Update ................................................................................... 14 4.2 DELTA UPDATE OF EXTRACTION DATA IN THE OLTP ........................................................................... 15 4.2.1 Collecting Data for the V3 Update ................................................................................................ 15 4.2.2 Starting the V3 Update .................................................................................................................. 16 4.3 SPECIAL FEATURES OF THE DELTA UPDATE IN SD ................................................................................ 17 4.3.1 Sign Logic in the SD Extractors .................................................................................................... 17 4.3.2 Delta Update, with Relevant Changes Only .................................................................................. 17 4.3.3 SD DataSources: Compatibility with the ODS .............................................................................. 18 5 EXTRACTION SIMULATION AND EXTRACTION LOG ................................................................. 19 5.1 5.2 6 EXTRACTION SIMULATION ..................................................................................................................... 19 EXTRACTION LOG .................................................................................................................................. 19

ENHANCING / CHANGING EXTRACT STRUCTURES .................................................................... 20 6.1 6.2 SUBSEQUENT ENHANCEMENTS TO THE EXTRACT STRUCTURE............................................................... 22 FIELDS NOT PERMITTED IN LIS COMMUNICATION STRUCTURES ............................................................ 22

7 CHANGING OVER FROM LIS DATASOURCES TO THE NEW SD/LE-SHP DATASOURCES IN THE LOGISTICS EXTRACT STRUCTURES CUSTOMIZING COCKPIT ........................................ 24

2000 SAP AG

Extracting SD and LE-SHP Transaction Data

1 Preface
This document contains information about extractors and DataSources, that are delivered for the first time with release BW-2.0B and PI 2000.1 or. PI-A 2000.1 (valid from R/3-Release 4.0B ), for extracting sales and distribution transaction data. It serves as a supplement to the standard SAP documentation. The information contained in this document is valid at the time of publishing. It is not exhaustive and may need updating for further releases The target group of the paper is primarily consultants, as well as customer employees who are active in SAP BW during the BW project, or decision makers who are assessing the projects feasibility/future possibilities. Knowledge of the operational processes in SD, as well as knowledge of the Business Information Warehouse (SAP BW) area, are important, but not absolutely necessary, prerequisites for a full understanding of the following procedures. On the one hand, the document contains an overview of all the necessary steps for activating and carrying out successful data extraction. On the other hand, it examines, in detail, the conception and technical background of this extraction, so that you can gain an initial overview of the possibilities, and also the limitations, of extracting SD transaction data. Three new applications have been created for the extractors in question: 11 12 13 SD Sales BW LE Shipping BW SD Billing BW.

The Business Content belonging to these applications in BW is still grouped together under the application component SD. The following document relates exclusively to the extractors and DataSources for these applications. You can find general information, that is, not application-based detailed information, in the document Extraction of Logistics Transaction Data (SAPNet -> Alias BW -> Documentation -> Documentation Enhancements). You can find many of the functions described here in BWs OLTP Customizing, which you can get to via the context menu for a source system in the BW source system tree (a remote login is then carried out in the OLTP system). In the OLTP system you can also use the transaction SBIW. For the plug-in PI 2000.1 or. PI-A 2000.1, the composite note 340038 gives an overview of all the notes for the R/3 core, the plug-in and BW Content, that are relevant for extracting logistics transaction data, based on the customizing cockpit for the Logistics extract structure.

2000 SAP AG

Extracting SD and LE-SHP Transaction Data

A corresponding follow-up composite note, that also contains all necessary corrections for the plug-in, for R/3 and for BW Content, is planned for every future plug-in. To use these new extraction methods, you must have made changes in the R/3 standard to attach the plug-in. These changes are delivered via the R/3 support package and, for the extractors described here, they take place with the corrections described in note 194909. This and note 201207 as well as note 328534 are necessary prerequisites for using the plug-in for extracting SD transaction data without any mistakes. Since these corrections concern changes to the R/3 standard, and therefore cannot be delivered with a plug-in, when you use the plug-in, you must check whether these notes have already been installed on your computer with the R/3 support package. If this is not the case, you must install these notes manually.

2000 SAP AG

Extracting SD and LE-SHP Transaction Data

2 New Extractors/DataSources with BW Release 2.0B/R/3 Plug-In PI 2000.1

The logistics extractors extract transaction data and are designed to replace the transfer Info structures (S260 to S264) used until now, in the first version of the application dealt with here. You create, change or delete documents in the applications. The resulting data is staged online in LIS communication structures, which you can then update classically using the LIS in Info structures. The concept of the new extract structure applies here. In addition, or as an alternative, to the LIS update, data is transferred from the LIS communication structure, using extract structures, into a central delta management area. This transfer takes place in the V3 update and is therefore temporally detached from the application. The delta management acts as a buffer for data that can be requested from BW, via an InfoPackage, with update mode delta upload. The following diagram shows the interaction between the LIS communication structure and the new extraction technology.

Diagram 1: Online Update of Extract Structures.

In place of previous transfers, InfoPackages with update mode Delta Initialization are used to transfer existing or archived documents. In OLTP the InfoPackages retrieve data from the set up tables, which are assigned to the extract structures. You fill these tables using special set up transactions in OLTP. Details about these functions are described later in this document.

2000 SAP AG

Extracting SD and LE-SHP Transaction Data

Diagram 2: Set up of Extract Structures.

For each extract structure, one set up table with the name of the extract structure and SETUP as a suffix, exists. For example: Extract structure MC11VA0ITM, set up table MC11VA0ITMSETUP. Since you are using cluster tables, it does not make sense to display the entries with transaction SE16. You cannot determine the number of entries from the number of data records. You can get to a data display, however, using transaction RSA3, since this transaction uses data from the set up tables. 2.1 The Design of the New Extract Structures

The extract structures are a fundamental part of the new extraction concept. These are R/3DDIC structures that contain all fields, whose data contents are transferred from the transaction data via the DataSources to BW, when you activate the relevant extraction. The extract structures for applications 11, 12 and 13, are structured in such a way, so that most of the fields within them are filled directly from the field contents of the LIS communication structures. Some additional fields are determined in the extraction module. Every LIS communication structure, that is used to fill an extract structure, is assigned to an include structure in the extract structure. When you are adding extra fields, whose field names are identical in several LIS communication structures intended for use in an extractor, it is possible to assign uniquely, the LIS communication structures, from which the data contents are transferred. However, this does not mean that you can include a field with the same name several times in the same extract structure. This extract structure concept allows you to include other fields without modification as well as user-defined fields, which were included using append technology, in the corresponding include of the LIS communication structure - in the extract structures. They are
2000 SAP AG

Extracting SD and LE-SHP Transaction Data

then automatically filled with the value from the corresponding LIS communication structure (meaning the LIS communication structure that the field you selected refers to). The extractors are then assigned to different document levels (Document header = HDR, document item = ITM, schedule line = SCL). You can include fields in the extractor from either the same document level or a higher document level. You can only include the latter, that is, the transfer from the higher document level, for fields that you will use later as characters in BW (For example: The order item extractor can include characteristics from the order header (sold-to-party, sales document number, and so on)). Fields from a lower level can naturally not be offered. If you select fields, which you want to use as key figures in BW, from a higher level, the values would be transferred more than once and key figures would be duplicated in the BW data targets. Therefore, such fields are not included in the extract structure and are not included in the selection of fields that you can use to enhance the extract structure. Enhancing structures is discussed in detail in section 6 (Enhancing/Changing Extract Structures). Amongst other things, the section explains why not all fields of the LIS communication structure that you are using, are included in the selection of available fields when you want to enhance an extract structure. The following table gives an overview of the extractors for applications 11, 12 and 13, shows which LIS communication structures are based on them, for which events in OLTP you can transfer data into BW and what the DataSources assigned to the extractors are called. Event VA means creating, changing or deleting orders. Event VB means creating, changing or deleting quotations. Event VC means creating, changing or deleting deliveries. Event VD means creating, changing or deleting billing documents.
App Extract l. structure 11 MC11VA0HDR Com. structure MCVBAK MCVBUK 11 MC11VA0ITM MCVBAK MCVBUK MCVBAP MCVBUP MCVBKD 11 MC11VA0SCL MCVBAK MCVBUK MCVBAP MCVBUP MCVBKD MCVBEP 11 MC11V_0ITM MCVBAK MCVBUK MCVBAP MCVBUP MCVBKD 11 MC11V_0SCL MCVBAK MCVBUK MC11V_1SCL MC11V_2SCL MC11V_1ITM MC11V_2ITM MC11V_4ITM MC11V_5ITM MC11V_6ITM VA, VC 2LIS_11_V_SCL Allocation of order schedule line/ delivery item MC11VA1SCL MC11VA2SCL MC11VA4SCL MC11VA5SCL MC11VA6SCL MC11VA7SCL VA, VC 2LIS_11_V_ITM Allocation of order item/ delivery item MC11VA1ITM MC11VA2ITM MC11VA4ITM MC11VA5ITM MC11VA6ITM VA, VB 2LIS_11_VASCL Sales document schedule line Include Event VA, VB MC11VA1HDR MC11VA2HDR VA, VB 2LIS_11_VAITM Sales document item DataSource 2LIS_11_VAHDR Name Sales document header

2000 SAP AG

Extracting SD and LE-SHP Transaction Data

App Extract l. structure

Com. structure MCVBAP MCVBUP MCVBKD MCVBEP MCLIKP MCVBUK

Include MC11V_4SCL MC11V_5SCL MC11V_6SCL MC11V_7SCL

Event

DataSource

Name

12

MC12VC0HDR MC12VC1HDR MC12VC2HDR

VC

2LIS_12_VCHDR

Delivery header

12

MC12VC0ITM MCVBAK MCVBUK MCVBAP MCVBUP MC12VC1ITM MC12VC2ITM MC12VC4ITM MC12VC5ITM

VC

2LIS_12_VCITM

Delivery item

12

MC12VC0SCL

VC

2LIS_12_VCSCL

Schedule line delivery (dynamic assignment of delivery item to order schedule line)

MCVBAK MCVBUK MCVBAP MCVBUP MCVBEL 13 MC13VD0HDR MCLIKP MCVBUK 13 MC13VD0ITM MCVBAK MCVBUK MCVBAP MCVBUP

MC12VC1SCL MC12VC2SCL MC12VC4SCL MC12VC5SCL MC12VC7SCL VD MC13VD1HDR MC13VD2HDR VD MC13VD1ITM MC13VD2ITM MC13VD4ITM MC13VD5ITM 2LIS_13_VDITM Billing document item 2LIS_13_VDHDR Billing document header

The following extract structures contain special features that have made it necessary to restrict the possibilities for enhancing them: Extract structures MC11V_0ITM and MC11V_0SCL basically contain the open quantities or values of the order items or order schedule lines. The transfer of transaction data of an order item or order schedule line using these extractors occurs from two events that are assigned to different business objects: VA VC When you create, change or delete the order item or schedule line When you create, change or delete a delivery item for the above order item or schedule line.

To make sure that the data is updated for both events, with the right characteristics, in all BW cubes or ODS objects, in this case, only a reduced set of characteristics is available, whose field value assignment in the extraction modules has been explicitly programmed for the most part. You cannot map both of the events at the same time from the LIS communication structures in this case. The reason for this is that, on the one hand, not all fields for the order are also contained in delivery and, on the other hand, field contents can deviate between order and delivery (for example, storage location). In general, you cannot be sure that the fields are automatically filled with the correct value when the delivery event takes place. Therefore, in these extract structures, you are not permitted to choose those fields, which otherwise you would be able to choose, in addition to the delivered fields. Another special feature of these extract structures is that, in the case of set up, they transfer data to BW only when the order data is being set up. This is because, during set
2000 SAP AG 8

Extracting SD and LE-SHP Transaction Data

up, the current open quantities or values of an order item or schedule line are already determined in full, in the order. Furthermore, in these structures, order or delivery data is only updated when the corresponding order item is relevant for delivery (shown in the delivery status of the order item MCVBUP-LFSTA) or if the delivery item refers to an order item (shown in the delivery status of the order item MCLIPS-APLFSTA). The extract structure MC12VC0SCL refers fundamentally to the dynamic LIS communication structure MCVBEL for delivery, in which the delivered quantities are dynamically assigned (that is, via program logic at the time of extraction) to the schedule lines that are based on the delivery item. In OLTP itself, there is no delivery item to schedule line assignment stored in the transparent tables. The delivered quantities can only be transferred per schedule line as key figures in this extractor. From PI 2000.2/PI-A 2000.2, the fields WMENG, BMENG, CMENG and LMENG of the order schedule line, will no longer be allowed (this change should have been effective with PI 2000.1/PI-A 2000.1, but was not included due to an error). Do not choose these fields under any circumstances. If you do choose these fields, their values will be transferred to BW several times for partial delivery and subsequent delivery of an order item.

As well as the includes given above, all extract structures still contain fields that are entered directly in the DDIC structure (for example, MC11VA0HDR). Data in these fields is not assigned by mapping, using LIS communication structures. It is determined explicitly in the extraction modules. 2.2 Administrative Tables for Defining Extract Structures and for Linking Extract Structures to DataSources

The interaction of the LIS communication structures with the extract structures is controlled, amongst other things, by the table TMCEXCFS, in which, information about the fields that you have selected, or the fields which are not available for selection, is contained for all LIS communication structures. In table TMCEXCFZ, all additional fields that the customer has chosen are recorded. The extract structures are assigned to their DataSources using table TMCEXACT. In addition, in TMCEXACT, the activation status for the extraction is saved. The DataSource is generated on the basis of these tables (for example, according to the enhancement of an extract structure).

2000 SAP AG

Extracting SD and LE-SHP Transaction Data

3 Prepared Steps for Extraction

3.1 Transferring Business Content DataSources

The first step that you have to carry out to activate the process of extracting transaction data is to transfer, into the OLTP, those DataSources that have been delivered as D-version DataSources. This generates the A-versions of the DataSources. To carry out this step, choose the menu path Business Content DataSources -> Transfer Business Content DataSources (transaction RSA5) in the OLTP customizing of the Business Information Warehouse. You use the compare function in this transaction, to check whether or not the active version has been generated already, and whether there are any variations between the D-version and the A-version. Before you are able to transfer the DataSources from the D-version, you need to first transfer the application component hierarchy from the D-version into the A-version. 3.2 Maintaining Extract Structures

You process the extract structures in the Logistics Extract Structures Customizing Cockpit (transaction LBWE, or under the menu path Settings for Application-specific DataSources -> Logistics -> Managing Extract Structures in the OLTP customizing for the Business Information Warehouse) The extract structures you find here, all have a traffic light assigned to them. The color of the traffic light indicates the status of the extractor and the DataSource: Green: Yellow: DataSource has been generated, and the extraction activated DataSource has been generated, the extraction deactivated. (Delivered status of the SD extractors and DataSources) Extract structure has been changed, but the DataSource has not yet been generated. (If you are not able to generate a DataSource with this status, it may be that you have not carried out step 3.1 (Transferring Business Content DataSources).

Red:

You use Structure Maintenance to expand the extract structures (see section 6 Enhancing/Changing Extract Structures). After you have made changes to an extract structure, you have to use the DataSource Maintenance to regenerate the DataSource belonging to it. The next section details the maintenance process.
2000 SAP AG

10

Extracting SD and LE-SHP Transaction Data

The next step is to activate the extraction using Update Activation. If you change the Update Activation of an extract structure or the transport of these changes to another system, all the relevant users have to log out and then log on to the system again, in order to check that the activation has been applied to every transaction. 3.3 Maintaining the DataSource

When you generate DataSources in the Logistics Extract Structure Customizing Cockpit, you can set up the following properties for it: 3.3.1 Canceling Fields For those fields of the DataSource which can be used as key figures in the data targets of BW, you can set the indicator Field is inverted in case of cancellation, to specify whether you want to transfer old or canceled data records with an inverted +/- sign into BW. To ensure that the delta is posted successfully in a cube, you must set this flag for all key figures. This also applies for self-defined fields. The flag is already set for all the key figures that SAP delivers with the DataSources. It is strongly recommended that you do not change these settings. Example: If you change an order item (not a return) in the OLTP from an amount of 5 pieces to 4 pieces, the LIS communication structures contain two data records; one with an amount of 5 pcs and the other with 4 pcs. One of these data records shows the status of the fields before the changes (before-image, old data record) and the other shows the field status after the changes have been made. You use the field ROCANCEL, which is in all the descriptive extract structures, to determine whether you want the old amount to be copied into BW with a negative sign (-5 pcs). The sign is reversed only for those fields that are flagged as inverted when canceled. This allows you to use a cumulative update to post the old amount out of the cubes in BW, and post the new amount in. Because you can change more than just amounts or values in a document (for example, you can change the customer group from 001 to 002), it is important that an old and a new data record is copied to BW with each delta update. With this, you can post the data correctly into the cubes that use the changed fields as characteristics (-5 pcs with customer group 001 and +4 pcs with customer group 002). See section 4.3.1 Sign Logic in SD Extractors for a detailed description of how +/-sign logic works in SD extraction. 3.3.2 Specifying Selection Fields for InfoPackages You can use the flag Selection to determine whether you want the individual fields of the DataSource to act as selection criteria in the BW InfoPackages. Only use this function in moderation with extractors that are transferred into BW with a delta upload, because it can lead to different interpretations of the selection criteria.

2000 SAP AG

11

Extracting SD and LE-SHP Transaction Data

If you run one or more delta initializations with selection restrictions, you can no longer create future InfoPackages that have the update mode delta upload with different selection conditions, in an attempt to guarantee data consistency. The selection conditions of the delta upload are the same as all the selection conditions of the various delta initializations (orlinks). Furthermore, the selection conditions of the InfoPackages have no influence on the extraction module or the set up program in the OLTP. This means that this data is always gathered independently of the selection conditions (at the time of extraction, you do not necessarily have to know which InfoPackage is being used to load the data into BW, the data might be selected for different BWs with different selection criteria). Using the selection conditions in the InfoPackages does not improve performance for extracting in the OLTP. However, it must be pointed out, that after you carry out the V3 collective run, only data for which there has already been a delta initialization run, is retained to be transferred into BW. A subsequent delta initialization, aimed at extending the selection range of future delta uploads, will no longer have an effect on documents that have already had a V3 collective run. These documents can now only be converted into BW if they are set up. More particularly, you cannot use the selection conditions to write the extracted data into different cubes with InfoPackages, because the selection conditions of all delta upload InfoPackages are already defined by the selection conditions of each delta initialization.

3.3.3 Hiding Fields in BW You can use the flag Hide Field to determine whether you want to hide individual fields of the DataSource. If a field is hidden, it is no longer in the transfer structure, and so can no longer be selected in the transfer rules of the InfoSources. You can hide one of the fields of a DataSource selected from the standard delivery, but this can prevent an update into the SAP delivered cubes from being consistent. If a document has been changed, but the change affects only one field, and this field is hidden in the DataSource, the old and new records are still transferred into BW. In other words, it is not possible to reduce the number of data records that are transferred into BW, by hiding fields.

2000 SAP AG

12

Extracting SD and LE-SHP Transaction Data

4 Extracting Transaction Data

4.1 Initializing Transaction Data (Setting up Existing Documents)

To transfer information from documents that have already been created in the OLTP, you must follow this procedure: 1. Activate the required extraction in the Logistics Extract Structure Customizing Cockpit (LBWE). This takes place on the level of the current extract structure. 2. The data must first be retrieved by the set up functions in the OLTP and put into the set up tables. During this process, do not make any changes to the documents in the system. If the datasets are large, run a test to estimate the volume of data in the set up tables, and extend the database table spaces if necessary. 3. If the set up tables are filled, you can continue editing the documents in the OLTP. Users must log on to the system again to do this. This is the only way of making sure that the activation of the extraction is taken into account when documents are posted. You must make sure that until an InfoPackage is successfully posted into BW with the update mode Initializing the Delta Process, no V3 collective run is started. Otherwise, all the delta data that was posted in the mean time is lost for BW updates, and can only be retrieved if you refill the set up tables in the OLTP. 4. The data that is staged in the set up tables, must now be requested from the BW, using an InfoPackage in the update mode Initialize Delta Process. All the V3 posting entries for the extractors are prepared for requesting from a BW, only after this InfoPackage has been successfully processed. This happens when V3 collective processing is started in the central delta management area (transaction RSA7). The document information that is worked on in a previous V3 collective process (perhaps started by mistake), cannot be transferred into BW with a delta upload, because the system assumes, on the basis of the missing initialization, that you do not need the information in BW. 5. If you want to repeat a delta initialization, even though one has already been run, you must first delete the initialization information for the DataSource. You do this in the maintenance screen of an InfoPackage in the relevant InfoSource of the source system in question, by choosing Scheduler -> Initialization options for a Source System. Delete the relevant entries there. Then delete the requests from the PSA and the data targets. After this step, you must repeat steps 1 to 4. You must make sure that you delete old entries before the set up tables are filled again. Even information on the same document, which has not been changed in the interim, would appear in the set up table twice, if the selection of the set up is applied to this document again.

4.1.1 Special Features when Running the Set up in the OLTP

2000 SAP AG

13

Extracting SD and LE-SHP Transaction Data

To set up the data ready for transferring it into BW, SD uses the same selection programs that are used on LIS to refresh statistical data. Enhancements not delivered with the plug-ins, had to be made to these programs. The enhancements are either delivered with an R/3 support package, or manually installed. This affects notes 201207 and 328534, which must be installed. When you start the set up as a background job, make sure that the report variants you are using are created with the transaction OLI7BW, and so on, and not OLI7. Otherwise, the system cannot recognize their relevance to BW. If you want to schedule the set up process in the form of several parallel jobs, you must check beforehand, whether the corrections from note 339995 are already available in your plug-in, otherwise data will be lost. If the amount of data is too large to include the document changes while the set up is taking place, you can split up the set up into two phases: Phase 1: Set up all documents that will definitely no longer be changed before the end of the whole initialization (for example, all documents to a certain document number that was created approx. 12 months previously). This data is transferred as a full upload into BW. During this time, you can still continue to make changes to the (other) documents. Delta initialization with the remaining documents - during this phase, you must complete all work on the documents. Firstly, you must process all entries of the V3 update (accumulated in phase 1). Start by using the V3 control in the Logistics Extract Structures Customizing Cockpit. Since no InfoPackage has been transferred into BW yet, central delta management ignores this data. Neither is it needed, since the corresponding documents are transferred into BW in their present state during the next, and final, part of the set up. Before you start the set up in the OLTP, you must delete the documents from phase 1 (use the LBWG from the set up tables). When you have completed the set up of phase 2, this data is requested as an InfoPackage with the update mode Initialization of the Delta Process from BW. As described above in step 3, you can continue editing the documents as soon as the set up tables are filled, if you are sure that you are not starting the V3 collective processing too early. This split into two phases is only possible at present if you have not used ODS objects as data targets. At the moment only either Full Upload InfoPackages or Delta Init InfoPackages can be used for InfoSources with update rules in an ODS object.

Phase 2:

Note:

The technical realization of the transaction, for deleting the set up tables for an application, consists of the table being deleted on the database, and being created again. You must therefore make sure that the process is cross-client, meaning all entries are deleted in all the clients. 4.1.2 Extraction with Update Mode Full Update With the new extraction, the data is not retained in a table in the OLTP, unlike an extraction based on the LIS info structures.
2000 SAP AG 14

Extracting SD and LE-SHP Transaction Data

Extracting data by regularly requesting an InfoPackage with the update mode full update, therefore, involves a lot of extra work. This kind of extraction process is useful only (if ever) for transaction data that does not have any change functions (for example, material documents) in the OLTP, which means that for each full upload, you have to extract only those data records that have been created since the last full upload. However, this is not the case with the transaction data belonging to the components Sales, Shipping, and Billing, described here. Using this process is, therefore, quite questionable, since you would have to extract all the documents regularly, and delete all the corresponding requests in BW, to make sure that all the documents were extracted correctly. This process would result in an ever-increasing runtime whenever you extract data. Also, it would not be possible with this method, to write the differences (deltas) that result from any changes to the cube at the time of the change date. Since the extracted data is no longer retained in LIS info structures in the OLTP, you would have to run a complete set up before every full upload if you used this method. For these reasons, we strongly recommend that you do not use this option for the extractors described here. Nevertheless, the option of loading full upload InfoPackages into BW, can still be of great importance, even with this extraction, as described above. 4.2 Delta Update of Extraction Data in the OLTP

4.2.1 Collecting Data for the V3 Update If at least one extract structure, assigned to event VA, is active, the extraction modules will run when you create, change, or delete request data. If the changed request data causes a change in at least one of the fields of an active extract structure, a V3 update module is called, which is run when the V3 collective run is called again. The same applies to delivery (event VC) and billing (event VD). The extraction data is now staged in the update data, ready for further processing, until the V3 collective run is called. You can view this data if you call up transaction SM13 (selection with status V2 executed) and branch to the update module. Here, you can display data for the update modules (for example, MCEX_UPDATE_11) of the module type Collective run and you get the interface tables, and so on, in the same format as the extract structures. This is how an open update entry is created with active extract structures before the start of the V3 collective runs for every document change that takes place. This is not an error. The system purposely does not update data, until it receives explicit instructions to do so. All other update steps (updating R/3 documents, updating LIS-V2, and so on) are taken

2000 SAP AG

15

Extracting SD and LE-SHP Transaction Data

beforehand, independently of the V3 update of the BW extraction. The success of this process does not depend on whether there are still V3 update steps to be carried out or not. It is essential that you observe the following: Before you introduce an R/3 standard support package or a plug-in patch, or before you execute an R/3 upgrade, it is essential that all open update entries have been processed. This is especially valid for the update entries of the type V3 (collective processing). The next section describes how you process the entries. Otherwise, in the future you may no longer be able to update the update entries, as well as leading to errors, when one of the structures that was used in the update entry, has changed with the parch or upgrade. Here it is already enough, if data element of domain has been changed to a shared field. 4.2.2 Starting the V3 Update By starting a corresponding job with the button V3 control for an application in the Logistics Extract structures Customizing Cockpit, you can work on all update entries for which the update of the extract module for the corresponding application still has the status init. For the applications 11, 12, and 13, the modules are as follows: MCEX_UPDATE_11 MCEX_UPDATE_12 MCEX_UPDATE_13 This means that if you start a job for application 11, for example, all update entries with the module MCEX_UPDATE_11, which have the status init, are processed. After successfully processing these update entries, the extraction data is in the delta queue of the service API (central delta management area) ready to be retrieved by BW in the form of a delta InfoPackage. It is recommended, in the production operation, that you call the collective processing for the individual applications with one appropriate time period, since the update entries can possibly contain several different V3 modules. You have to request the BW delta queue data as soon as possible after the V3 collective processing from the BW. At the very least you have to make sure that the number of LUWs, that are displayed on the overview screen of transaction RSA7 by DataSource/target system in the field total, is not larger than 9999, since, otherwise, the data processing can require unusually long runtimes. If (because, for example, the above advice was not followed) it is necessary, to delete the entries DATA SOURCE/TARGET system from the BW of a delta queue, due to large cumulative quantities of data in the BW delta queue and associated problems with the transfer, then it is generally insufficient to delete the appropriate entry in the transaction RSA7 or to reset the delta initialization on the BW page. The procedure, in order to physically delete the data from the BW delta queue, is described in note 324622. You should also observe this note if you want to reset a delta initialization. The number of LUWs displayed in the BW delta queue does not correspond to the number of documents belonging to them.
2000 SAP AG 16

Extracting SD and LE-SHP Transaction Data

It is possible to summarize a large number of documents in one LUW. For this, it is strongly recommended that you consider note 358981. If one of the interface structures of the function module changes between the update data being created (saving the document) and executed (starting the V3 collective process) the update can no longer be carried out without errors, and the update is terminated. The chances of this happening are higher if an extract structure has been extended. To avoid this problem, see section 6 Enhancing / Changing Extract Structures 4.3 Special Features of the Delta Update in SD

4.3.1 Sign Logic in the SD Extractors As described in section 3.3.1 (Canceling Fields) you use the field ROCANCEL in the extract structures to control the +/- signs for the delta update. SD has the following additional feature: Document items in SD, where the credit/debit flag is set (for example, billing item MCVBRPSHKZG) return items, credit memo items, and so on (field Returns in the customizing mode of the order item types) are usually transferred with a negative sign into BW. This also applies to return items in document types, which are not of the returns type (for example, credit memo items in debit memos, or so-called mixed business transactions). In the LIS communication structures, all the value and amount fields are preceded with a positive sign. This applies in mixed business transactions as well, even if their document fields are supposed to be preceded by a negative sign. The value of ROCANCEL determines that new records are transferred into BW with a reverse sign (-5 pcs), and old records are transferred with an unchanged sign (+4 pcs), in delta updates, or in the set up of return items. On the other hand, to make sure the ODS processes the data records successfully, the data records are sorted, so that the old records are always transferred before the new records (as of note 335427 only). If you do want to update return items in return document types without negative signs, in cubes or ODS objects, this sign concept must be taken into account in the update rules. As an example, you can use the update rules for the cubes delivered by SAP. Return document types are identified by the value C (credit document) for the InfoObject 0DEB_CRED. 4.3.2 Delta Update, with Relevant Changes Only Unlike extraction using LIS info structures, the extraction modules in SD are designed in such a way that, old and new records are transferred only if at least one field of the extract structure has been changed. The field ROCANCEL is not taken into account here, since it always differentiates between new and old records.

2000 SAP AG

17

Extracting SD and LE-SHP Transaction Data

Changes made to documents in the OLTP, which do not affect any fields in the extract structure, do not therefore result in data being extracted into BW. Setting the flag Hide field for a field of the DataSource, however, does not mean you can prevent the old and new records from being transferred into BW, even if the document change was meant for this field only. The decision to transfer the data is based only on the extract structure. 4.3.3 SD DataSources: Compatibility with the ODS It is important that you refer to Notes 320863, 333492 and 335427 whenever you want to update SD DataSources into ODS objects. The InfoObject 0RECORDMODE is central to the process of updating into ODS objects. To delete data records from an ODS object, you have to transfer the data record that you want to delete with 0RECORDMODE = R (Remove). This is exactly what happens when the document or item is deleted in the OLTP.

2000 SAP AG

18

Extracting SD and LE-SHP Transaction Data

5 Extraction Simulation and Extraction Log

There is a common log transaction for LO extractors. In applications 11, 12, and 13, the log is filled both as a simulation, using a flag, in the set up functions, and as a set-get-parameter in the document processing. The log is application-specific and user-dependent. 5.1 Extraction Simulation

Using the log transaction, it is possible to simulate and analyze the extraction of document data that takes place as a result of the set up of an application (11, 12, 13) before the actual set up takes place. You are able to generate an extraction log for each user, and each application. If you choose the simulation flag when you are setting up an application (application-specific set up in the customizing of extractors) the set up tables are not filled. Instead, the log is filled with all the documents you have chosen. Generally, you use this function only when you are working with a limited number of documents. Do not use the simulation to help you estimate how long the runtime for an actual set up will take. Use the section BW Log in Customizing Extractors to read the log. It is not necessary to set any user parameters. The extraction log is independent from the set up tables, meaning that the log cannot be deleted explicitly, and is not deleted or overwritten when the set up tables are initialized. The log for an application belonging to a user is kept until the user makes a new entry for this application. This new entry deletes the last entry that was made. 5.2 Extraction Log

If, in the OLTP, a user sets the SET-GET-parameter (user parameter) MCL to X, and, provided that at least one extract structure for the current application is active, an entry is written to the extraction log used to log data that has been extracted for BW, every time a document is posted. When documents are modified, both the old and the new record are displayed. The previous entry in the log is overwritten with every new document that is posted. This is also true when the user carries out a simulation of the set up. For the purposes of logging a set up simulation, it is enough to set the corresponding parameters in the set up program. It is not necessary to set extra user parameters. It is also not possible to use the user parameter settings to fill the set up tables and the log simultaneously during a set up run. Two separate actions are always required to fill the set up tables and the log (one with and one without the simulation flag). The log allows you to monitor the extractors in development systems and test systems. For performance reasons, do not set the parameter MCL in productive systems.
2000 SAP AG

19

Extracting SD and LE-SHP Transaction Data

6 Enhancing / Changing Extract Structures

With the logistics extract structures customizing cockpit, a tool is provided for the extraction of logistics transaction data. This tool enables you to add fields from the LIS communication structures to the extract structures, without having to make any modifications. The LIS Customer Exits that are implemented to enhance the SD communication structures (enhancement MCS10001, MCS50001, and MCS60001) are used in the same way that they are used in the LIS, to fill customer-defined fields that have been included using append technology in the includes designed for this purpose in the LIS communication structures. These Customer Exits have already run in a BW data extraction, so that the relevant information is also available here. For more information on the procedure, see the documentation for the enhancements that you reach, for example, using transaction SMOD (subobject Documentation). Once you have enhanced the LIS communication structures, it is possible to include the fields in the logistics extract structures customizing cockpit in one or more extract structures, provided that the communication structure is included in the selection for the relevant extract structure you want to use. This ensures that the data filled in the Customer Exit is passed on to the BW. Since not all of the fields contained in the LIS communication structures are included in the SAP delivery for the extract structure, it is also possible to include additional standard fields from the LIS communication structure in the extract structure. All the selected fields are included automatically in a generated append structure for the corresponding include structure of the extract structure (for example, append ZZMC11VA4ITM for include MC11VA4ITM for additional fields in the order item extractor for LIS communication structure MCVBAP). If, in a subsequent plug-in, these same fields are included in the standard extract structure, they are removed from the customer enhancement using an XPRA program that is executed automatically with the Upgrade. For various reasons, it is not possible to offer all the fields contained in the LIS communication structure, for selection in the extract structure (see section Error! Reference source not found.). After an extract structure has been enhanced, the DataSource has to be regenerated, and replicated in BW. A function in the InfoSource tree in BW replicates the DataSource in BW, according to the current source system. It is extremely important that you regenerate the relevant transfer structures in the BW, because if you do not, the first InfoPackages after the enhancement terminate with an error message (Transfer structure has to be regenerated). In general, you add the inserted fields to the transfer structure, and adjust the transfer rules. For the time being, it is important that you generate the transfer structure, even if you do not want to change the transfer structure and the transfer rules for some reason. Before you change an extract structure, note the following important points: 1. There must be no V3 update entries that still need to be processed in the application, in which you are working. Any unprocessed V3 update entries inevitably result in an error when you start the update (the V3 collective processing) because the interface has changed the modules being used.
2000 SAP AG 20

Extracting SD and LE-SHP Transaction Data

To prevent this from happening, you have to trigger the processing of the entries in the Logistics Extract Structures Customizing Cockpit by using the V3 control (V3 collective process). If you failed to do this before changing the extract structure, you have to either use transaction SM13 to delete the update entries, or change the extract structure temporarily back to its previous status. 2. There must be no data in the central data management for the extract structure you are working in. If there is, you have to request this data, in the form of InfoPackages, out of the BW, before you change the extract structure. 3. There must be no data in the set up table for the extract structure you are working in. If there is, it will no longer be possible to transfer this data into the BW. The data has to be deleted, and, if it is required again at a later stage, you have to set it up after the extract structure has been changed and the update has been reactivated. 4. If the update log is active, a user is able to see the data for the relevant application again, only after he/she creates, changes, or deletes a document for the relevant application (or carries out a simulation of the set up function) once the extract structure has been changed, and the update has been reactivated. Before you transport the changes you have made to an extract structure into a target system, it is important that you check through the above points in the target system too. With PI-2000.2 and PI-A 2000.2, the Logistics Extract Structures Customizing Cockpit has been enhanced, for the extract structures described here, in such a way that makes it impossible to change the extract structures if one of the points 1 3 has not been applied. If a change is possible, all log entries are deleted immediately and automatically. The program RMCSBWCC checks that a transport complies with the points mentioned above. You start the program in the target system. Before you carry out the transport, make sure there are no more messages relating to the points above. It is particularly important that you run this program before you transport data into a productive system, because the consequences of transporting an extract structure with badly prepared changes, are particularly far reaching here. No users from the affected applications must be logged on to the system during the actual transport, because changes are also being made to the DDIC. Using the menu path Subsequent Processing of DataSources Edit DataSource, it is generally possible to enhance a DataSource in the OLTP customizing of the Business Information Warehouse. However, you are not able to use the LIS communication structures in the usual extraction module to fill the fields inserted here. Instead, you use a Customer-Exit with SAP enhancement RSAP0001 to fill the fields. Note that at this point, it is no longer possible to access the corresponding transaction data directly, and that this exit runs only in the V3 update (during V3 collective processing). Data from the application document not included in the extract structure has to be read from the database. This is inadvisable for a number of reasons, not least of which being that system performance is significantly impaired, and in this phase, additional changes may have also been made to documents. If you try to read transaction data in this customer exit, inconsistencies arise in the data.

2000 SAP AG

21

Extracting SD and LE-SHP Transaction Data

In general, SAP does not recommend you use the latter direct method to enhance extract structures of the Logistics Extract Structures Customizing Cockpit. As a rule, the best way is to enhance and fill the communication structures MCVBAK, MCVBAP, and so on. It is also possible to enhance the document tables VBAK, VBAP and so on, and fill them using the specialist user exits for the applications sales, shipping, and billing. The fields you added using an append are also included in the Logistics Extract Structures Customizing Cockpit to be used for enhancing the extract structures. Note that, in the procedure last referred, the data inserted into the document tables is saved with the document information on the database. If you have enhanced extract structures by adding append fields from the communication structures or document tables, but you do not require one of the fields anymore, you must first remove this field from the extract structure in the Logistics Extract Structures Customizing Cockpit. Then delete the field in the append of the communication structure or document table. 6.1 Subsequent Enhancements to the Extract Structure

If a) you want to add fields to an extract structure, b) the update into BW has been in use productively for a reasonable period of time, and c) you do not intend to, or it is not possible to reinitialize the data subsequently, it is important that you proceed as follows to ensure that individual delta updates are not lost during conversion. 1. Close all document updates for the application you are working in (lock the corresponding transactions to be extra sure it is not necessary to deactivate the updates, and there is no advantage if you do). 2. Use V3 control in the Logistics Extract Structures Customizing Cockpit to start processing all the open V3 update entries (check using SM13 or program RMCSBWCC as from PI 2000.2) 3. Delete the set up tables for the relevant applications (you should have deleted these set up tables after the initialization was completed successfully) 4. Get the delta-queue data for all the relevant DataSources by requesting the corresponding InfoPackages from all the relevant BW systems (using the program RMCSBWCC it is possible from PI 2000.2 to check whether the data has been requested before) 5. Enhance the extract structures or transport the changes made to the extract structure from a development system or a consolidation system 6. Activate the DataSources 7. Replicate the DataSources in the BW 8. Activate the relevant transfer structures in the BW 9. User restarts document processing, and V3 update runs can be started (users are able to restart document processing after they have carried out step 6, provided that the V3 update for the relevant extract structures is not started too soon) 6.2 Fields not permitted in LIS Communication structures

Not all of the fields contained in a LIS communication structure are included in the Logistics Extract Structures Customizing Cockpit in the selection of fields that you choose from to add as enhancements to extract structures. These fields are explicitly excluded from the selection by the setting in the control table TMCEXCFS. Note 351214 details the reasons for restricting the number of fields.
2000 SAP AG 22

Extracting SD and LE-SHP Transaction Data

SAP strongly recommends that you do not make any changes to the settings in the table TMCEXCFS, and accepts no responsibility for any consequences that result from this type of modification.

2000 SAP AG

23

Extracting SD and LE-SHP Transaction Data

7 Changing over from LIS DataSources to the new SD/LE-SHP DataSources in the Logistics Extract Structures Customizing Cockpit
With active updates, the new InfoSources delivered with BW 2.0B, are constructed in such a way as to allow you to change over from the LIS Info structures S260 to S264 to the new DataSources without losing any data, or having to set up any of the data. The following section describes the steps you need to follow to change over from the old DataSources to the new DataSources, if your BW update is active and based on the Info structures S260 to S264, and if you do not want to, or it is not possible to set up the data completely. 1. Make sure that all updates for the new extract structures are deactivated (LBWE). If updates have already been activated by mistake, you have to use the V3 control in all three applications to start a one-off V3 collective processing run, after you have deactivated the updates. 2. Transfer the new Business Content DataSources into the OLTP: The system generates the active versions of the DataSources. You replicate the DataSources, by triggering the replication process in the BW system for the corresponding source system. 3. Transfer the new Business Content objects into BW: Choose, in particular, the new InfoSources, the transfer rules, and the new update rules. 4. For all user-defined update rules in user-defined cubes that are based on the LIS structures S260 to S264, you have to create the corresponding update rules for the new InfoSources. Update rules based on the new Info structures are provided with the cubes that are delivered by SAP as Business Content. You just need to adjust these update rules, if you have modified the delivered update rules based on S260 and S264. 5. Make sure that all users authorized to make changes to SD documents are logged off the system. 6. Use transaction LBW1 to deactivate the update for the LIS transfer-Info structures S260 to S264. 7. Activate the update for the new extract structures in the Logistics Extract Structures Customizing Cockpit. 8. Once the update for the new extract structures is active, you are able to resume processing documents without any restrictions. 9. Transfer the last delta records that have accumulated for the old DataSources into BW. 10. Delete all the InfoPackages in the old InfoSources.

2000 SAP AG

24

Extracting SD and LE-SHP Transaction Data

11. Delete any data records that are still in the set up tables of the new extract structures. 12. Start the InfoPackages with the Delta initialization update mode for each of the new InfoSources. It is also possible to start the InfoPackages by setting the Initialization without Data Transfer switch. Information on this switch is found in the F1 Help. 13. It is only at this point that you are able to start the V3 collective processing.

2000 SAP AG

25