Performance Tuning Presentation
Performance Tuning Presentation
Performance Tuning Presentation
MultiProviders and Partitioning Aggregates Query Design & Caching Hardware & Servers InfoCubes and DSOs
BI- Accelerator
h Sizing and Implementation
h Management and Costs
In this session.
In this session we will cover the top 20 must-do technical performance tricks to help you optimize SAP NetWeaver BI reporting for your end users. We will look at performance modeling of InfoCubes, how to improve memory utilization by caching and how to use diagnostics to analyze performance issues. We will also explore best practices on how to develop and manage aggregates and MultiProviders, and see what the BI- Accelerator (BIA) can do for your organization. Finally, we will look at how to analyze EarlyWatch reports from Solution Manager 4.0 so they become actionable.
3
A survey of 353 top C-level officers in large companies, reported that the top BI concern was the scalability of their solutions.
Overall System Performance and capacity Server & Desktop processing capacity Determining ROI Security Data Integration Too many Business Intelligence tools in use Danger of distributing outdated data Difficult for end users to learn or use
Source: Intel, SAP & Business Week "Seizing the BI Opportunity" 2006.
7.9 7.88 7.65 7.62 7.51 7.31 7.21 7.03 7 7.25 7.5 7.75 8
4
MultiProviders and Partitioning Aggregates Query Design & Caching Hardware & Servers InfoCubes and DSOs
BI- Accelerator
h Sizing and Implementation
h Management and Costs
A query now have to search in all InfoProviders to find the data (i.e. billing docs from 2007). This is very slow. Solution: We can add hints to guide the query execution. In the RRKMULTIPROVHINT table, you can specify one or several characteristics for each MultiProvider which are then used to partition the MultiProvider into BasicCubes. If a query has restrictions on this characteristic, the OLAP processor is already checked to see which part cubes can return data for the query. The data manager can then completely ignore the remaining cubes.
An entry in RRKMULTIPROVHINT only makes sense if a few attributes of this characteristic (that is, only a few data slices) are affected in the majority of, or the most important, queries (SAP Notes: 911939. See also: 954889 and 1156681). 6
The MultiProvider query is then restarted automatically and processed sequentially What appears to be parallel processing corresponds to sequential processing plus the preceding phase of parallel processing up to the termination
Generally, its recommended that you keep the number of InfoProviders of a MultiProvider to no more than 10. However, even at 4-5 large InfoProviders you may experience performance degradation.
7
With SAP BW 3.0B SP14 (SAP BW 3.1 SP8 and later versions, you can change the default value of 30,000 rows refer to SAP Notes 629541, 622841, 607164, and 630500.
A larger number of base InfoProviders is likely to result in a scenario where there are many more base InfoProviders than available dialog processes, resulting in limited parallel processing and many pipelined sub-queries
You can also change the number of dialogs (increase the use of parallel processing) in RSADMIN by changing the settings for QUERY_MAX_WP_DIAG.
8
MultiProviders and Partitioning Aggregates Query Design & Caching Hardware & Servers InfoCubes and DSOs
BI- Accelerator
h Sizing and Implementation
h Management and Costs
Aggregates
Aggregates are much less used by the SAP installation base than training and common sense should dictate. The interface to build the summary tables (aggregates) are intuitive and easy to master, but few are taking real advantage of them. Even among those that are using aggregates, many have poorly defined solutions & seldom monitor the usage, thereby limiting the benefits of this simple technology.
To avoid poor definition and usage, aggregates should be developed after the system has been in production for a while and real user statistics are captured.
10
Select the run time of queries to be analyzed (e.g., 20 sec) Select time period to be analyzed
Only those queries executed in this time period will be reviewed to create the proposal
11
12
High valuation and high usage is what we are looking for. This indicates high reduction of records in aggregate and high benefits to users.
.
When using 3rd party query tools and ODBC to query directly into the DSO, you are bypassing the OLAP Processor. Therefore, you cannot accurately performance tune the system using aggregates (statistics), nor will the 3rd party tool benefit from aggregates.
13
MultiProviders and Partitioning Aggregates Query Design & Caching Hardware & Servers InfoCubes and DSOs
BI- Accelerator
h Sizing and Implementation
h Management and Costs
EarlyWatch Reports
Wrap-up
16
Read all data (all data is read from a database and stored in user memory space) Read data during navigation (data is read from a database only on demand during navigation) Read data during navigation and when expanding the hierarchy
2.
3.
Reading data during navigation minimizes the impact on the application server resources because only data that the user requires will be retrieved.
Source: Catherine Roze,
17
19
New in 7.0 BI: OLAP Engine can read deltas into the cache. Does not invalidate existing query cache. Turn off/on parallel processing When will the query program be regenerated based on database statistics
20
Tip 8: Filters
Leverage filters as much as possible. Using filters contributes to reducing the number of database reads and the size of the result set, thereby significantly improving query runtimes. Filters are especially valuable when associated with big dimensions where there is a large number of characteristics such as customers and document numbers.
If large reports have to be produced, leverage the BEx Broadcaster to generate batch reports and pre-deliver them each morning to their email, PDF or printer.
21
P1 of 3
22
P2 of 3
In this real case, aggregates was needed for those cubes flagged
23
Work with the basis team to research the settings and the Oracle issues. Focus on SAP notes and the index issue. The RSRT and RSRV codes are a gold mine for debugging and analyzing slow queries.
24
Notice the yellow flag for the 6 base cubes in the MultiProvider and the yellow flag for the 14 free chars.
(Note: no hints were used in this MultiProvider, which led to very poor performance).
You can also trace the front-end data transfers and OLAP performance by using RSTT in SAP 7.0 BI (RSRTRACE in BW 3.5)
25
Distribution Types
You can increase query speed by broadcasting the query result of commonly used queries to the cache. Users do not need to execute the query from the database. Instead the result is already in the system memory (much faster).
26
Here you can execute the query and see each breakpoint, thereby debugging the query and see where the execution is slow. Worth a try: Try running slow queries in debug mode with parallel processing deactivated to see if they run faster..
27
In 7.0 BI at all service packs upto number 14, it is also impossible to populate the OLAP cache by broadcasting query views. If you use earlier service packs, you may be forced to create many different queries to provide this performance.
The implementation of service pack 14 is highly recommended by SAP for these performance reasons. When implemented the Java execution will also improve.
28
Cockpit Expense Query for Detailed program Financial dashboard expense (actual vs. target) Financial dashboard [expense] Financial dashboard [non-earnings] Expense Query for Detailed org objective Financial dashboard [workforce costs] Expense Query for Detailed Capital Query for Detailed work type Financial performance Workforce financials Expense query for detailed Balance Query for Detailed program Balance Query for Detailed cost element Balance Query for Detailed MWC Non-earnings Query for Detailed org objective Financial dashboard (other balance) Balance query for organization detailed Balance Query for Detailed work type Non-earnings Query for Detailed cost element Capital Query for Detailed cost element Labor query for detailed org Standard costs variance detailed report Financial dashboard (non-earnings) Non-earnings Query for Detailed program Balance Query for Detailed org objective Capital Query for Detailed MWC Non-earnings query for organization detailed Non-earnings Query for Detailed work type Financial dashboard [workforce cost] Capital Query for Detailed org objective Financial dashboard [other balance sheet] Labor query for detailed cost element Expense Query for Detailed cost element PCC Expense Query for detailed Headcount detail fin-DB Capital - Actual Vs. Target Financial dashboard capital trend Non-earnings Query for Detailed MWC Average
#of Queries 1 7 2 2 1 6 1 1 9 9 1 1 1 1 1 7 1 1 1 1 1 1 7 1 1 1 1 1 2 1 2 1 1 1 1 7 2 1
Baseline SP 14 Improve 4/18/08 (4/21/08) ment 145 9 94% 150 18 88% 70 12 83% 42 13 69% 31 10 68% 50 17 66% 36 14 61% 22 9 59% 43 21 51% 29 16 45% 16 9 44% 14 8 43% 14 8 43% 14 8 43% 14 8 43% 29 17 41% 13 8 38% 13 8 38% 13 8 38% 14 9 36% 14 9 36% 20 13 35% 30 20 33% 12 8 33% 13 9 31% 14 10 29% 12 9 25% 12 9 25% 23 18 22% 15 12 20% 21 17 19% 12 10 17% 13 11 15% 14 12 14% 15 13 13% 24 22 8% 13 12 8% 13 13 0% 27.95 12.03
A Real Example
This company saw a 39% decrease in Query execution time after implementing SP-14. SPThey had 38 cockpits and 82 queries that improved substantially without any further changes..
29
39%
2. Line item dimensions are basically fields that are transaction oriented and therefore, once flagged as a line item dimension, is actually stored in the fact table. This results in faster query access (no table join).
Explore the use line item dimensions for fields that are frequently conditioned in queries.
30
31
CSSTidy
32
MultiProviders and Partitioning Aggregates Query Design & Caching Hardware & Servers InfoCubes and DSOs
BI- Accelerator
h Sizing and Implementation
h Management and Costs
EarlyWatch Reports
Wrap-up
33
The size of OLAP Cache is physically limited by the amount of memory set in system parameter rsdb/esm/buffersize_kb. The settings are available in RSPFPAR and RZ11.
Source: V. Rudnytskiy, 2008
35
Default
Flatfile
Change the logical file BW_OLAP_CACHE when installing the system (not valid name)
FILE
Binary Large Objects Optional (blob) Best for large result sets Blob/Cluster Enhanced (new in SAP 7.0 BI)
SP 14
No central cache directory or lock concept (enqueue). The Set mode is not available by RSR_CACHE_ACTIVATE_NEW RSADMIN VALUE=x default.
Source: SAP AG 2008.
36
Roll memory was never maxed out in the period 12/23/07 through 1/27/08 Paging memory was never maxed out in the period 12/23/07 through 1/27/08 Extended memory was never maxed out in the period 12/23/07 through 1/27/08 Only 3GB of 9 GB of Heap memory was ever used in the period 12/23/07 through 1/27/08
37
MultiProviders and Partitioning Aggregates Query Design & Caching Hardware & Servers InfoCubes and DSOs
BI- Accelerator
h Sizing and Implementation
h Management and Costs
EarlyWatch Reports
Wrap-up
38
Tip 17: Avoid Outdated Indexes and Database statistics Database statistics are used by the optimizer to route queries. Outdated statistics leads to performance degradation. Outdated indexes can lead to very poor search performance in all queries where conditioning is used (i.e. mandatory prompts).
Name Vendor history closed AR customer FIAR line items FIAR Payment history FIAR: Transaction data Multicube AR&billing Billing cube custom for AR trade Sales contract cube - anticipated billing Service orders - ZSLM Performance cube Headcount and personnel actionas Cycle count MM LIO interface infocube Material aging Lead time cube Tech-Nm XFIAP_C10 XFIAR_C10 0FIAR_C03 0FIAR_C05 0FIAR_C02 XSDARBIL XSDBILITM XSDCN_C10 ZCSCBZSLM ZCSCBPER ZHRPA_C02 XMMWM_C10 XLIO_C01 ZMMCBMAAG ZMMLTCUBE Object Infocube Infocube Infocube Infocube Infocube Infocube Infocube Infocube Infocube Infocube Infocube Infocube Infocube Infocube Infocube Indexes DB stats % used to create stats 10% 10% 10% 10% 10% n/a 10% 10% 10% 10% 10% 10% 10% 10% 10%
n/a
n/a
For high volume Infocubes, or cubes that have a high number of users, the percentage used to build the DB stats can be increased from the default 10% to 20%. This may yield more accurate query routing and better query performance (consider this especially for cubes with old data partitioned)
As much logic as possible should be moved to the extraction, and needed data fields should be denormalized and stored in logically organized ODSs and Infocubes.
Validate the high-cardinality of the data and reset the flag if needed this will give a better index type and performance
41
MultiProviders and Partitioning Aggregates Query Design & Caching Hardware & Servers InfoCubes and DSOs
BI- Accelerator
h Sizing and Implementation
h Management and Costs
EarlyWatch Reports
Wrap-up
42
The SAP BI Accelerator makes query response time 50-10,000 faster. You use process chains to maintain the HPA engine after each data load
HP, Sun and IBM have standard solutions ranging from $32K to $250K+ that can be installed and tested in as little as 2-4 weeks (+ SAP license fees)
43
You get BIA sizing estimates by running the SAP program available in SAP Note: 917803
44
45
Hardware Example
Environment Production Production Production Production Production Production Production Production QA QA QA QA QA QA QA QA Development Development Development Development Development Development Development Development Area Blade servers Memory Processors Processor speed Network cards External storage File system Chassis Blade servers Memory Processors Processor speed Network cards External storage File system Chassis Blade servers Memory Processors Processor speed Network cards External storage File system Chassis Recommended size 14 Blades 2x8 GB (2x4) DDR2 total 16 GB 2 x Quad Core Intel Xeon Processor 3.00 GHz+ 2 x Gigabit Cisco cards Dedicated disks (500 GB+) General Parallel file system (GPFS) 14 blades capacity 14 Blades 2x8 GB (2x4) DDR2 total 16 GB 2 x Quad Core Intel Xeon Processor 3.00 GHz+ 2 x Gigabit Cisco cards Dedicated disks (500 GB+) General Parallel file system (GPFS) 14 blades capacity 4 Blades 2x8 GB (2x4) DDR2 total 16 GB 2 x Quad Core Intel Xeon Processor 3.00 GHz+ 2 x Gigabit Cisco cards Dedicated disks (300 GB+) General Parallel file system (GPFS) 14 blades capacity IBM example* BladeCenter HS21 -8853G6U 39M5797 2 x Quad Core Intel Xeon Processor 3.00 GHz 32R1760 DS-4800 GPFS H-series (rack-mount/9U) 88524XU BladeCenter HS21 -8853G6U 39M5797 2 x Quad Core Intel Xeon Processor 3.00 GHz 32R1760 DS-4800 GPFS H-series (rack-mount/9U) 88524XU BladeCenter HS21 -8853G6U 39M5797 2 x Quad Core Intel Xeon Processor 3.00 GHz 32R1760 DS-4800 GPFS H-series (rack-mount/9U) 88524XU
The BIA should be sized for critical applications. Most companies use BIA only for Production, while others have a complete landscape
46
BIA is no longer something exotic. Many of the large BI systems have already implemented BIA and many more projects are under way in Europe and in the Americas.
Nike
Once you exceed a few hundred critical users and/or 3-4 Tb of data you should seriously consider SAP BIA
47
MultiProviders and Partitioning Aggregates Query Design & Caching Hardware & Servers InfoCubes and DSOs
BI- Accelerator
h Sizing and Implementation
h Management and Costs hEarlyWatch Reports
Wrap-up
48
recommendations
Run them periodically & read the details This is a real EarlyWatch report from a mid-sized company that has been running SAP BW for the last four years
On a large global project, system issues can be hard to pin-down without access to EarlyWatch reports. The monitoring reports allows you to tune the system before the user community gets access and complaints arise.
49
1 Performance Overview
In a 24-hour operational systems due to time-zones, you will have The performance of your system was analyzed with respect to the average response times and total workload. We did not detect any major problems that could affect the performance of your system. less time to react and fix issues. Therefore, early detection of system issues are critical to the success of a global project.
Avg. DB Avg. GUI Time in ms Time in ms 1110.9 6.3 585.4 80.8 5772.6 405.0
The following table shows the average response times for various task types: Task type DIALOG + RFC UPDATE UPDATE2 BATCH HTTP Dialog Steps 195240 5 48 59288 257762 Avg. Resp. Time in ms 3253.3 984.2 133.2 11599.3 693.5 Avg. CPU Time in ms 728.7 28.2 17.1 2091.2 183.7 Avg. Wait Time in ms 1.8 26.0 0.7 0.6 4.4 Avg. Load Time in ms 2.5 15.2 3.3 8.5 2.2
50
51
In this real example, the EarlyWatch report identified that there were no valid backups for almost one month.
52
MultiProviders and Partitioning Aggregates Query Design & Caching Hardware & Servers InfoCubes and DSOs
BI- Accelerator
h Sizing and Implementation
h Management and Costs hEarlyWatch Reports
Wrap-up
53
Resources
Presentations, tutorials & articles www.Comerit.net
SAP SDN Community web page for Business Intelligence Performance Tuning https://www.sdn.sap.com/irj/sdn/bi-performance-tuning
ASUG407 - SAP BW Query Performance Tuning with Aggregates by Ron Silberstein (requires SDN or Marketplace log-on). 54 min movie. https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/media/uuid/d9fd 84ad-0701-0010-d9a5-ba726caa585d Large scale testing of SAP BI Accelerator on a NetWeaver Platform https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/b00 e7bb5-3add-2a10-3890-e8582df5c70f
55
Your Turn!