SAP Basis Training II
SAP Basis Training II
SAP Basis Training II
OVERVIEW...........................................................................................................................................3
FLOW..................................................................................................................................................4
APPLICATION SERVER...........................................................................................................................4
DATABASE SERVER...............................................................................................................................5
PERFORMANCE BOTTLENECKS...............................................................................................................6
PERFORMANCE PROBLEMS AND TROUBLESHOOTING TECHNIQUES................................................................6
DIALOG RESPONSE................................................................................................................................6
UNIX...................................................................................................................................................7
DATABASE..........................................................................................................................................10
BASIS LAYER.....................................................................................................................................14
DIALOG (PER PROCESS).......................................................................................................................17
BATCH (PER PROCESS)........................................................................................................................17
ACUTE PROBLEMS..............................................................................................................................18
QUALITATIVE DATA............................................................................................................................18
QUANTITATIVE DATA..........................................................................................................................18
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 2 of 22
14011235.doc
Overview
FI SD BTC UPD PP
SAP Kernel
UNIX
Database
USUNX07 Application Data
Cuswtomizing (T* Tables)
ABAP/4
17x CPU (max 32) Data Dictionary (DDIC) {tables, fields/data types}
6GB RAM (max 32 GB) Master Data {users, vendors, customers}
IDS
UNIX
Mirror 2 Mirror 2
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 3 of 22
14011235.doc
Flow
Check with Central Instance which server is least loaded
Servers check with each other, one responds to CI
Make a straight connection to that server
Application Server
Connect
Ethernet
Apps Server
USUNX14
10 DIA
3 UPD
1 SPO
SAP Kernel
UNIX
FDDI
Task ABAP DB
Handler VM Interface
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 4 of 22
14011235.doc
Database Server
FDDI
Buffer Virtual
Virtual Threads
IDS
Unix
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 5 of 22
14011235.doc
Performance Troubleshooting
Performance Bottlenecks
Presentation
LAN/WAN Connectivity
Memory
ABAP
Application
Database
Disk
Dialog response
Measured from Dispatcher to Dispatcher
CPU (ABAP time on the Apps server)
DB (CPU and IO on USUNX07)
Roll (context swapping of user data)
Wait
Load/Gen (compiling of ABAP code)
Processing
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 6 of 22
14011235.doc
Bottom-Up Troubleshooting
Unix
OSS Collector
Basic, generalized data of the Operating System
ST06 runs OSS Collector for the instance you’re logged on to
Collector runs hourly
Has an option to compare all servers
On the USUNX07
Idle time not less then 20%
Not to much swap action
Paging must be < 20% of main memory
Disk utilization should be < 50%
Disk queue should be low
Disk wait time should be < 50/100 ms
On an Apps server
Dialog instances have low CPU usage
Batch instances use more CPU (max should be < 20%)
Perview (pv)
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 7 of 22
14011235.doc
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 8 of 22
14011235.doc
Memory Information:
physical page size = 4096 bytes, logical page size = 4096 bytes
Physical: 8384512 Kbytes, lockable: 7267600 Kbytes, available: 7412892 Kbytes
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 9 of 22
14011235.doc
Database
ST04
Unfortunatly this transaction is pretty useless because the numbers are squed due to
64/32 bit limitations
Onstat –p
Read quality > 95% (% cached reads)
Write quality > 82% (% cached writes)
Buffers shouldn’t be to big because the flush point will take to long (see checkpoint
duration in the logs). During a flush point the system halts.
Informix Dynamic Server Version 7.31.FC7X8 -- On-Line -- Up 1 days 15:47:41 -- 3878280 Kbytes
Profile
dskreads pagreads bufreads %cached dskwrits pagwrits bufwrits %cached
503518339 555131625 7922394078 93.64 13790707 12327788 63066879 78.13
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 10 of 22
14011235.doc
Onstat –m
Last 20 lines of the informix message log
Informix Dynamic Server Version 7.31.FC7X8 -- On-Line -- Up 1 days 15:48:12 -- 3878280 Kbytes
Onstat –g seg
Shared memory segments
Only 1 pool per type, otherwise you will get fragmentation
Resident buffers (general buffer) > 95% read quality
Virtual area
If a pool fills up it will add 128mb of extra space in a new pool
Informix Dynamic Server Version 7.31.FC7X8 -- On-Line -- Up 1 days 15:48:42 -- 3878280 Kbytes
Segment Summary:
id key addr size ovhd class blkused blkfree
1545 1381386241 c000000000476000 1513758720 24168 R* 184778 7
6666 1381386242 c00000005a818000 2457600000 38144 V 153899 146101
6155 1381386243 c0000000ecfd8000 729088 656 M 82 7
Total: - - 3972087808 - - 338759 146115
Onstat –g ioq
IO for database
Queue length should be < 100
KIO = asynchronous (calling app doesn’t for response, better performance)
IO = synchronous
Informix Dynamic Server Version 7.31.FC7X8 -- On-Line -- Up 1 days 15:52:10 -- 3878280 Kbytes
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 11 of 22
14011235.doc
Onstat –g rea
Ready threads (waiting for processes)
Informix WP’s which are ready but are not doing anything at the moment
No more then 8
Informix Dynamic Server Version 7.31.FC7X8 -- On-Line -- Up 1 days 15:53:00 -- 3878280 Kbytes
Ready threads:
tid tcb rstcb prty status vp-class name
144 c00000005b914ca8 0 4 ready 1cpu kaio
Onstat –g act
Active informix threads
Informix Dynamic Server Version 7.31.FC7X8 -- On-Line -- Up 1 days 15:53:31 -- 3878280 Kbytes
Running threads:
tid tcb rstcb prty status vp-class name
8 c00000005a898ca0 0 2 running 1cpu soctcppoll
142 c00000005b799d08 0 4 running 3cpu kaio
161 c00000005c068028 0 4 running 4cpu kaio
165 c00000005be5a028 0 4 running 6cpu kaio
186 c00000005ba29c18 0 4 running 5cpu kaio
1815 c000000067cbadb0 c0000000622db3d8 2 running 3cpu sqlexec
4493 c0000000927b1a90 c0000000622b9558 2 running 9cpu sqlexec
4649 c00000006cbf30c0 c000000066ef8648 2 running 7cpu sqlexec
Onstat –g ath
All informix threads
Informix Dynamic Server Version 7.31.FC7X8 -- On-Line -- Up 1 days 15:54:17 -- 3878280 Kbytes
Threads:
tid tcb rstcb prty status vp-class name
2 c00000005a8389e0 0 2 yield forever 18lio lio vp 0
3 c00000005a8391a0 0 2 yield forever 19pio pio vp 0
4 c00000005a839960 0 2 yield forever 20aio aio vp 0
5 c00000005a877100 0 2 yield forever 21msc msc vp 0
6 c00000005a8777e0 0 2 yield forever 22aio aio vp 1
7 c00000005a898540 c00000005a89c028 4 sleeping secs: 1 4cpu main_loop()
8 c00000005a898ca0 0 2 yield forever 1cpu soctcppoll
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 12 of 22
14011235.doc
Onstat –l
Shows logical logs usage
U = Used
B = Backup-ed
Used to roll the database back or forward
Informix Dynamic Server Version 7.31.FC7X8 -- On-Line -- Up 1 days 15:54:45 -- 3878280 Kbytes
Physical Logging
Buffer bufused bufsize numpages numwrits pages/io
P-2 13 512 2990072 86452 34.59
phybegin physize phypos phyused %used
4500035 70000 7132 2243 3.20
Logical Logging
Buffer bufused bufsize numrecs numpages numwrits recs/pages pages/io
L-3 14 16 41136289 3978892 527379 10.3 7.5
Subsystem numrecs Log Space used
OLDRSAM 41136289 7592216904
Sapdba
Db space reports (b, a)
Never < 1% free (except for logdbs & physdbs)
0% free will cause that table to be shut down
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 13 of 22
14011235.doc
Basis Layer
ST02
Tune Summary
Hit ratio +/- 99%
No swaps except for program
Generic Key -> buffered tables (customizing)
ST10
Table call statistics
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 14 of 22
14011235.doc
SM12
Lock entries shouldn’t be more then 6 hours old. If there are old locks, the user must
save or back out of the transaction
ST03
DB request time should be 50% of total request time
CPU 40%
Other 10%
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 15 of 22
14011235.doc
SM37
Check for Terminated Jobs
SM50
Make sure there are not too many processes in “private” mode
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 16 of 22
14011235.doc
When this limit is exceded you get the short dump: tsv_no_roll_memory.
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 17 of 22
14011235.doc
Top-Down Troubleshooting
Acute Problems
Abnormal system situation
Specific problem
CPU on the Apps server (poor ABAP)
Database (everything on USUNX07
This data can be reviewed by using /STAT
Main problems lay in:
Expensive SQL, indexes
IO Bottlenecks
Exclusive lock waits/locking situations (2000000 locks available)
Qualitative Data
Data received/asked from user when they report an issue
What runtimes are you seeing?
What runtimes do you usually have/need?
Which server?
Selection criteria/circumstances
Time of day
Anything else they can tell us for context
Quantitative Data
What changed (code, customizing, business process)?
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 18 of 22
14011235.doc
In Utils, Version Management you can see when the last version was transported.
Locks
There are 2 types of locks:
Database locks. Maintained by IDS for database integrity.
Enqueue locks. Arbitrated by SAP to maintain application integrity (follows
business logic). Enqueue locks should be released by killing the user session
from /SM04.
/SM12: Enqueue Lock entries. Blue are updates, black are user busy.
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 19 of 22
14011235.doc
A B C D
1 2
Lock on 2
Expensive SQL
/ST05: SQL Trace. Traces everything the ABAP processor does on a specified server.
This has the option to Explain SQL. The Estimated Cost (derived by the Informix
optimizer) shows how “heavy” the SQL is. A cost > 1000 is bad.
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 20 of 22
14011235.doc
/ST04: database analysis. The detail analysis shows lots of useful data for the database
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 21 of 22
14011235.doc
Transaction overview
/SE11 – Database Dictionary
/ST04 – Can also be used to calculate the cost of SQL for running processes. In Detail
Analysis, go to Informix Session.
/var/www/apps/scribd/scribd/tmp/scratch8/14011235.doc
Page 22 of 22