SAP BW/BI Hot Topics

1
In This Session …
• Gain insight into SAP NetWeaver® BW data load processes, how they work, and what
tools are available to monitor and optimize their performance
• Receive best practices to maximize data load performance while reducing long-term
maintenance costs
• Understand the benefits of optimized data load processes
• Find out how to enable version history to track code changes and how to create
reusable ETL logic to improve throughput and reduce data load time
• Get tips on when and how to use customer exits in DataSources and variables to
manage risk and reduce maintenance costs
• Identify the challenges and benefits of semantic partitioning and the importance of
efficient data models

2
What We’ll Cover …
• Loading data in SAP NetWeaver BW
• Finding performance bottlenecks
• Optimizing the database
• Optimizing the ABAP code
• Optimizing the data models
• Optimizing the data updates
• Wrap-up

3
SAP NetWeaver BW Data Load Processing Overview
• SAP NetWeaver BW data load processing consists of three main
activities:
 Extraction = Collecting the data in the source systems and preparing
it before sending it to SAP NetWeaver BW
 Transformation = Transforming the data using routines, lookups,
formulas, etc.
 Load = Updating the data into InfoProviders’ DataStore Objects
(DSOs), cubes, and master data

4
SAP NetWeaver BW Data Load Processing Overview

The ETL Process

As we already described at the beginning of this
book, the extraction, transformation, and loading
(ETL) process is the process of regularly updating
the SAP BW data.

For this process, the data must first be extracted

from the source systems, then checked for
consistency—and if necessary adapted—and finally
loaded into a data target
Phases of the ETL process
Extraction
In this first step, we will certainly select all the source data and
then provide
it for the transformation process that follows. In this phase, you
generally see a high level of heterogeneity because all initial data
is
generated in different data processing systems.
5
SAP NetWeaver BW Data Load Processing Overview

Transformation
Data transformation is the central task of the ETL process where
the
initial data must be adapted to the structure of the data target.
Furthermore,
the data quality should be analyzed and automatically
increased.
Loading
Once the data has been checked and made available, it is
integrated
into SAP BW. In this context, the work area where the data is
located is called the Persistent Staging Area (PSA). The process of
loading data always includes a probable aggregation of the
extracted and transformed data.

6
Dataflow in SAP NetWeaver BW

Source: SAP
7
Extraction Interface Types

Source: help.sap.com
8
DataSources Supported by SAP NetWeaver Extraction
• SAP NetWeaver BW Service API
 Allows data from SAP systems in standardized form to be extracted
and accessed directly
 These can be SAP application systems or SAP NetWeaver BW

systems
• File interface
 The file interface permits the extraction from and direct access to
files, such as csv files
• Web services
 Permit you to send data to the SAP NetWeaver BW system under
external control

9
DataSources Supported by SAP NetWeaver Extraction (cont.)
• Universal Data (UD) Connect
 Permits the extraction from and direct access to relational data

• Database (DB) Connect

 Permits the extraction from and direct access to data located in
tables or views of a database management system
• Staging Business Application Programming Interfaces (BAPIs)
 Open interfaces that SAP BusinessObjects DataServices and
certified third-party tools can use to extract data from older systems

10
Extraction Time Can Be Split into Two Categories
• Extraction time
 DB time to select the data to be extracted
 Logic applied during extraction such as joins, lookups, and filtering
• Middleware and network time
 The time used to transfer the data from the source system to the target SAP
NetWeaver BW system
 Interface types such as Web services and Universal Data (UD) Connect are good
for small amounts of data and cannot handle large volumes
 Fixed format files are larger to transfer but faster to load into SAP NetWeaver BW
 WAN Network time can become a bottleneck during peak hours

11
Transformation Types
• SAP NetWeaver BW supports the 3.x and the 7.x versions of transforming
the data
 3.x is using Transfer rules and Update rules
 Two steps of logic to process the dataset

 Loads to different targets must be processed together

 Used to have better performance than transformations

 Old method; no more development or performance enhancements; do not continue to

use
 7.x is using transformations
 Is using a single step of logic to process the dataset

 Loads to different targets can be processed independently

 Better performance

 Always use this option for new development

12
Loading Data to Information Providers Types
• Loading of the data to InfoProviders differs depending on type
 DSO
 Update of the activation queue
 Activation of data (update of active table and changelog)
 SID determination
 Should in general be switched off for DSOs

 Master data
 Update of master data tables
 SID determination
 Check duplicate key values
 Very time consuming for time-dependent attributes
 Attribute change run to activate the master data
 Generate navigation data

13
Loading Data to Information Providers Types (cont.)
• Loading of the data to InfoProviders differs depending on type (cont.)
 Cubes
 Update of data to the InfoCube star schema
 SID determination
 Roll up data to aggregates
 Update data to SAP NetWeaver BW Accelerator (SAP NetWeaver BWA)
• Performance considerations for loading the data
 Ensure that the database parameters are in place
 Implement the correct SAP NetWeaver BW settings for your InfoProviders

14
What We’ll Cover …
• Loading data in SAP NetWeaver BW
• Finding performance bottlenecks
• Optimizing the database
• Optimizing the ABAP code
• Optimizing the data models
• Optimizing the data updates
• Wrap-up

15
Tip 14: Data Modeling: Defining Dimensions
• Use as many dimensions as possible
 Separate common filter characteristics into own dimension
• Use line-item dimensions for high cardinality characteristics such as document
numbers
 Do not set the high cardinality flag!
• Define related characteristics in the same dimension
 Calculate expected number of dimensional entries
 Try not to exceed 10% of expected fact table entries
 Verify the dimension design after the first dataloads using program
SAP_INFOCUBE_DESIGNS
• Add all relevant time characteristics
 If 0CALMONTH is lowest granularity, add 0CALMONTH2, 0CALQUARTER,
0CALQUART1, 0HALFYEAR, and 0CALYEAR
 Provides greatest reporting flexibility without need to reload

16
 Tip 15: Implement Semantic Partitioning
• What is it?
 An architectural design to enable parallel data loading and query
execution
 Partitioning criteria: Year, Region, or Actual/Plan

Source: SAP
17
Tip 15: Implement Semantic Partitioning (cont.)
• Benefits of semantic partitioning:
 Reduction in SAP NetWeaver BWA footprint (when partitioned by
year)
 Parallel data loading (when not partitioned by year)

 Parallel query execution

 Best case when partitioning criterion is set as constant

 Almost as good to create variables to filter on 0INFOPROV

 Archival of a single InfoCube does not impact others

 Easier DB maintenance

Performance benefits are so significant …

semantic partitioning should be deployed
on virtually every data model!

18
 Tip 15: Implement Semantic Partitioning (cont.)
• Example: Semantic partitioning by year
History MultiProvider
(Summarized)

Current Year – 3 Current Year - 2 Current Year - 1 Current Year Current Year + 1

ALL years Current Year – 3 Current Year - 2 Current Year - 1 Current Year Current Year + 1
Write-Optimized (No SIDs)

Ex: Current Year + 1 = 2010

Current Year = 2009 DataSource

Current Year - 1 = 2008

Current Year - 2 = 2007
Current Year - 3 = 2006

Source: SAP
19