Berg 2016 Advancedperformanceoptimization v2
Berg 2016 Advancedperformanceoptimization v2
Berg 2016 Advancedperformanceoptimization v2
• Get practical tips and techniques for maintaining and cleaning an SAP BW system for
optimal performance, including PSA optimization, compression, maintaining statistical
cubes, and controlling growth, reducing log file sizes, removing DTP temporary storage,
DTP error logs, and temporary database objects
• Reduce the size of an SAP BW system by as much as 70% by taking steps such as
removing PSAs, aggregating, and optimizing InfoCubes, and implementing the new
LSA++ architecture
• See how to clean batch tables and reduce the footprints of un-needed data
• Learn how to take advantage of new performance features in SAP BW 7.3 and BW 7.4
1
What We’ll Cover
2
BW 7.3 (Non-HANA) InfoCube Design — Line Item Dimensions
Source: SAP AG
• This may result in improvements to query speeds for cubes not in BWA or HANA
Explore the use of line item dimensions for fields that are frequently
conditioned in queries. This model change can yield faster queries.
3
BW 7.3 (Non-HANA) InfoCube Design — High Cardinality Flags
• High-Cardinality flag for large InfoCubes with more than 10 million rows
Entries in dimension
InfoCube Number of rows compared for F table
FIUC_C03 12,859,780 37%
ZGAT_C01 20,793,573 46%
FIIF_C02 68,090,967 102%
FIGL_C01 156,738,973 88%
• At this company there were 11 InfoCubes with a ratio of more than 30% of the records in the dimensions vs.
fact table
• SAP recommends for Indexing and performance reasons to flag these as “high-cardinality” dimensions.
However, it has minor impact to smaller cubes.
• In this example, there were four medium and large InfoCubes that are not following the basic design
guidelines, and subsequently had slow performance
Millions
275
250
Additionally, 101 DSO objects were 225
200
flagged as being reportable. This 175
150
125
resulted in System IDs (SIDs) being 100
75
created during activation. 50
25
-
Partition DSOs. The lock on very large DSOs during parallel loads are well known and SAP has issued
several notes on the topic: 634458 “ODS object: Activation fails – DEADLOCK” and 84348 “Oracle deadlocks, ORA-00060.”
5
The BW 7.3 DataFlow Generation Wizard
This wizard reduces the number or manual steps needed to load data. It also simplifies
the development process and makes ETL work much easier.
6
Database Performance (Non-HANA Systems)
• Outdated indexes can lead to very poor search performance in all queries where
conditioning is used (i.e., mandatory prompts)
• The current sampling rates for this example were too low, and statistics should only be
run after major data loads, and could be scheduled weekly
For many systems, database statistics are outdated and may cause database performance to
perform significantly poorer than otherwise would be the case. Sampling should often be
changed and process chains may be re-scheduled.
7
Statistics and Indexes in BW: Top-10 InfoCubes Performance
In this real example, several of the largest InfoCubes have outdated Database statistics, which
may lead to sub-optimal performance. These should be updated as soon as possible.
DB Total
Technical DB Aggregate
Name Statistics Records
Name Indexes Indexes
(Oracle) (fact & dim)
Billing Conditions PCSL_010 205,802,748
HR Costs & Allocations PCCO_010 202,781,814
Sales Analysis PCSL_001A 199,642,247
Front-End and OLAP Statistics (Details) 0TCT_C02 111,585,386
Daily Shipment Analysis PCSL_C04 94,610,212
Billing Item PCSL_018 91,959,664
Sales and Margin Analysis 4 PCSL_022 60,395,396
PCBP: Cost Object Controlling PCCO_007 53,876,090
Benefits - Historical Months 0PABN_C01 52,800,098
Purchasing Data 0PUR_C01 50,190,148
Updating DB stats can be done in a weekly job that runs ever weekend.
8
Another example: Global Cache and Query performance
Today over 40% of all queries access the
database instead of the cache. This means
that the ‘cache hit-ratio’ is only 60%.
In the last month, the system had 31,955 query execution, of which 12,782 queries accessed
the database instead of the cache. If the hit-ratio is improved by 20%, it would result in
almost 59 minutes of less total wait times per day for the users.
BW comes with the default setting of 100 MB for local cache and
200 MB for global cache. 9
The OLAP Memory Cache Size Utilization
10
Changing the Cache Size
Since the global cache size is a logical value and the Export/Import SHM gives a physical limit, and
also considering that other applications (such as BCS) might use the Export/Import SHM, SAP
recommend to set the global cache parameter maximally to 90% of the Export/Import SHM buffer.
11
Example: Query Performance of most used InfoCubes
While many SD query executions are very fast (13-14 seconds), other reports in
Finance and Inventory Management is very slow
12
Example: Query Performance – 4,300 reports per month
Current with SAP BW on Oracle BW on HANA DB
Reporting Current total
Number of Avg. Avg. Time Avg. Time Est. query
Interface time spent Savings
Query Runtime DB [sec] OLAP execution time
waiting (hrs (hrs per year)
executions [sec] [sec] [sec] with HANA (sec)
per year)
Settings: BEx Broadcaster 220 45.22 32.20 9.86 33.16 8.00 27.29
Queries: BEx Web (JAVA) 2,809 44.88 31.90 8.45 420.23 7.00 354.68
Queries: BEx Web (ABAP) 1,261 15.51 9.32 1.73 65.19 5.00 44.18
Total Hrs saved 426.15
Of the total queries executed in the system in April-May 2015, 4,075 were from end-users.
The average query run-time took 39.7 seconds, while some reports took 8-9 minutes to run.
Of the slowest interface, the reports in BEx Web Java was executed 2,809 times and took an
average of 45 seconds to execute. In an HANA environment, the same query would have
completed in an estimated time of 7 second.
There are several temp tables that can be obsolete in the older systems (mostly from
previous jobs and internal reports run).
This could be cleaned up using the program SAP_DROP_TMPTABLES when users and
batch jobs are not running on the system (note: this program will remove all temp tables
except temp hierarchy tables and may stop jobs in-progress). It is therefore
recommended that this is done during a weekend outage.
Once this is done, you could consider running SAP_UPDATE_DBDIFF to clean all
obsolete temporary entries. These two efforts can reduce the BW system size and also
provide for a cleaner environment.
16
SAP HANA and SAP BW 7.4
• For BW 7.4 on HANA, SAP has continued to move more of the process intensive functions
from the application to the DB server
19
Pre-Steps — Cleaning Up Your BW System
• For example, an international company had a BW system with over 108TB, with only
38TB in the production box and the remaining data on their Near-Line Storage (NLS)
solution
• This cleaned BW system saved them potentially millions of dollars in hardware and
HANA licensing costs
1. Checks BW metadata with DDIC 7. Re-assigns requests written into the incorrect PSA partition
2. Deletes RSTT traces 8. Verifies DataSource segment assignment to PSA
3. Deletes BW statistical data 9. Deletes the entries no longer required in table RSIXW
4. Deletes aggregate data via deactivation 10. Clears all OLAP Cache parameters
5. Ensures partitioned tables are correctly indexed for PSA 11. Repairs InfoCube fact table indices at Data Dictionary level
6. Ensures request consistencies in the PSA 12. Reorganizes and deletes bookmark IDs and view IDs
You first have to install the program from SAP Note 1829728 before you can
generate the SAP_BW_HOUSEKEEPING task list using tcode STC01 21
A Tool to Help Migrate and Clean Up
22
More Tips to Make the Database Smaller
The sizing program in SAP Note 1736976 takes these size savings
settings into account when sizing your HANA system 23
What We’ll Cover
24
Demo: Optimal SAP BW on HANA Performance
25
What We’ll Cover
26
12 Pre-Steps — Cleaning Up Your BW System
1. Clean the Persistent Staging Area (PSA) for data already loaded to DSOs.
2. Delete the Aggregates (summary tables). They will not be needed again.
3. Compress the E and F tables in all InfoCubes. This will make InfoCubes much smaller.
4. Remove data from the statistical cubes (they start with the technical name of 0CTC_xxx).
These contain performance information for the BW system running on the relational
database. You can do this using the transaction RSDDSTAT or the program
RSDDSTAT_DATA_DELETE to help you.
5. Look at the log files, bookmarks, and unused BEx queries and templates (transaction
RSZDELETE).
6. Remove as much as possible of the DTP temporary storage, DTP error logs, and
temporary database objects. Help and programs to do this are found in SAP Notes
1139396 and 1106393.
27
12 Pre-Steps — Cleaning Up Your BW System (cont.)
7. For write-optimized DSOs that push data to reportable DSOs (LSA approach),
remove data in the write-optimized DSOs. It is already available in higher-level
objects.
8. Migrate old data to Near-Line Storage (NLS) on a small server. This will still provide
access to the data for the few users who infrequently need to see this old data. You will
also be able to query it when BW is on HANA, but it does not need to be in-memory.
9. Remove data in unused DSOs, InfoCubes, and files used for staging in the BW system.
This includes possible reorganization of master data text and attributes using process
type in RSPC.
28
12 Pre-Steps — Cleaning Up Your BW System (cont.)
10. You may also want to clean up background information stored in the table RSBATCHDATA. This
table can get very big if not managed. You should also consider archiving any IDocs and clean the
tRFC queues. All of this will reduce the size of the HANA system and help you fit the system
tables on the master node.
11. In SAP Note 706478, SAP provides some ideas on how to keep the Basis tables from growing too
fast in the future; if you are on Service Pack 23 on BW 7.0 or higher, you can also delete unwanted
master data directly (see SAP Note 1370848).
12. Finally, you can use the program RSDDCVER_DIM_UNUSED to delete any unused dimension
entries in your InfoCubes to reduce the overall system size.
29
Deep-Dive: Cleaning Up Your BW System
30
What We’ll Cover
31
Demo: SAP BW 7.4 Performance Monitoring
In this demo we will explore the BW 7.4 on HANA DBA Cockpit Features
32
Demo: SAP BW 7.4 Performance Monitoring (cont.)
33
What We’ll Cover
34
Converting InfoProviders and/or Data Flows
• While not required, InfoCubes can be optimized further
for HANA performance
• This basically means “flattening” the data structures
and removing the dimensions in BW from the physical
layer (they still look as if they exist)
Many refer to this optional step as a “functional migration” and do this after the HANA migration has been completed,
often as a separate initiative (see SAP Note 1849497) 35
HANA Optimized BW 7.4 DSOs and Performance Improvements
• This means that data activations are done much faster at the HANA database layer
• The change log is kept in a calculation view resulting in smaller DSOs
HANA optimized DSOs are also available for BW 7.3, but now they are created by
default, so do not convert DSOs to HANA-optimized. Fast activation is available for
all standard DSOs without conversion to HANA-optimized.
36
Converting InfoProviders and/or Data Flows
• You can also simulate the data flow before you do the real
conversion. When doing so, data is loaded for both versions (3.x
and 7.x) of the dataflows and the results are stored in cluster
tables. The data is then compared to verify that the dataflow after
migration calculates the same data as it did before migration.
• Since the differences are displayed separately, you can analyze the
results and changes in details
37
What We’ll Cover
38
EDW Design vs. Evolution
39
Data Design The Use of Layered Scalable
Architecture (LSA)
40
Example: Current LSA Data Architecture in SAP BW
DATA CORPORATE DATA BUSINESS FLEXIBLE DIMENSIONAL
ERP BW ACQUISITION MEMORY PROPAGATION TRANS. REPORTING REPORTING
Germany
6 LSA Layers
Europe Europe Europe Europe
(excl.Germany) (excl.Germany) (excl.Germany) (excl. Germany)
Europe
(excl. Germany)
ERP Table
Europe 2
41 total
Transferobjects
Data Source
Rule
Data
Acquisition
Europe 3
Info Source
USA USA USA
USA
USA
Americas 1 Americas 1 Americas 1 Americas 1
Americas 1
Americas 2
Asia
41
Example: Simplified LSA++ Data Architecture
DATA CORPORATE DATA BUSINESS FLEXIBLE DIMENSIONAL
ERP BW ACQUISITION MEMORY PROPAGATION
TRANS. REPORTING Remove
REPORTING3 LSA layers
ERP Table
41 shrinks to 9
Europe Europe Europe
Data Source
Transfer Rule
Info Source
total objects
Americas Americas Americas
Remove 5 semantic
partitions
42
EDW — Complex Layered Architectures
Consolidation Processes:
some was due to the technical configuration of the data store BPC Staging Cube
(BPC_C01)
architecture and data flow inside SAP BW
GL Summary Cube
(FIGL_C03)
Production Issues included:
1) Dependent jobs not running sequentially,
i.e., load from Summary cube to Staging Conformed
cube is sometimes executed before the Reportable FIGL_D21 FIGL_D20 FIGL_D17 FIGL_D14 FIGL_D18
DSO
Summary cube data is loaded and
activated, resulting in zero records in the Write
Staging cube. Optimized FIGL_D15S FIGL_D13S FIGL_D10S FIGL_D08 FIGL_D11S
DSO
2) Long latency with 6 layers of PSA, DSOs, Persistent Staging Area (PSA)
and InfoCubes before consolidation
processes can be executed. ECC 6.0 ECC 6.0 ECC 4.7 R/3 3.1i ECC 4.7
Asia- North-America Latin-America EU ASIA
Pacific 43
Fixes to Complex EDW Architecture
• The fix to this system included removing the conformed DSO layer, with BEx flags for
DataStores that are never reported on
• Also, the BPC Staging cube served
little practical purpose since the data is Consolidation Processes:
well as simplified system ECC 6.0 ECC 6.0 ECC 4.7 R/3 3.1i ECC 4.7
maintenance. Asia- North-America Latin-America EU ASIA
Pacific 44
EDW Data Design — Classical Use of MultiProvider Hints in BW
Problem: To reduce data volume in each InfoCube,
data is partitioned by Time period.
2002 2003 2004 2005 2006 2007 2008
Solution: We can add “hints” to guide the query execution. In the RRKMULTIPROVHINT table, you can
specify one or several characteristics for each MultiProvider, which are then used to partition
the MultiProvider into BasicCubes.
• If a query has restrictions on this characteristic, the OLAP processor is already checked to see which
part of the cubes can return data for the query. The data manager can then completely ignore the
remaining cubes.
• When DataStores and InfoCubes are allowed to grow over time, the data load and query performance
suffers
• Normally objects should be physically partitioned when the numbers of records exceed 100 – 200 million
However, this may be different depending on the size of your hardware and the type of database you use
• In SAP BW 7.3 we get an option to create a Semantic Partitioned Object (SPO) through wizards
You can partition based on fields such as calendar year, region, country, etc.
46
Data Design — Semantic Partitioned Objects
• When an SPO is created, a reference structure keeps track of the partitions. The structure is
placed in the MultiProvider for querying.
48
Where to Find More Information
• Bjarne Berg and Penny Silvia, Introduction to SAP HANA (3rd Edition) (SAP PRESS,
2014).
• http://scn.sap.com/docs/DOC-35096
Michaela Pastor, “SAP Business Warehouse 7.4” (SCN, November 2014).
• http://help.sap.com/nw_platform
SAP NetWeaver 7.4 on the SAP Help Portal
• www.stechno.net/sap-notes.html?view=sapnote&id=153967
SAP BI content release note for BW 7.4
49
7 Key Points to Take Home
• SAP BW 7.4 is the first release to take full advantage of SAP HANA
• Some of the functions in 7.4 are also available to non-HANA customers
• The new CompositeProviders and the Open ODS View make HANA and BW
tightly integrated and capable to support EDWs better
• You should break from the past and start designing with the new BW 7.4
features in mind
• The new monitoring features in the BW DBA Cockpit and the HANA systems
make it much easier to see what is occurring from a database level for the
non-basis team
• Before you size your system, clean it up and save hardware costs
• All customers should consider the BW move to HANA in 2016!
50
Your Turn!
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company) in Germany and other
countries. All other product and service names mentioned are the trademarks of their respective companies. Wellesley Information Services is neither owned nor controlled by SAP SE.
52