Simulating Oracle I/O Workloads To Configure Storage: James F. Koopmann Nocoug Fall Conference 2007 4:00 - 5:00
Simulating Oracle I/O Workloads To Configure Storage: James F. Koopmann Nocoug Fall Conference 2007 4:00 - 5:00
James F. Koopmann
NoCOUG
Fall Conference 2007
4:00 – 5:00
Who am I
Thanks James,
http://www.lsi.com/insight_center/industry_standards/storage_connectivity/raid/index.html
RAID Basics
RAID Level 1- Mirroring
http://www.lsi.com/insight_center/industry_standards/storage_connectivity/raid/index.html
RAID Basics
RAID Level 0+1 - Striping and Mirroring
http://www.lsi.com/insight_center/industry_standards/storage_connectivity/raid/index.html
RAID Basics
RAID Level 5 - Striping w/Rotational Parity
http://www.lsi.com/insight_center/industry_standards/storage_connectivity/raid/index.html
Determining the Application Mix
Therefore:
Small Reads = Total Reads – Large Reads
Small Writes = Total Writes – Large Writes
Determining the Workload for an
Existing Oracle Database
SQL> SELECT
'Number of Small Reads :'||
sum(decode(name,'physical read total IO requests',value,0)-
decode(name,'physical read total multi block requests',value,0)),
'Number of Small Writes:'||
sum(decode(name,'physical write total IO requests',value,0)-
decode(name,'physical write total multi block requests',value,0)),
'Number of Large Reads :'||
sum(decode(name,'physical read total multi block requests',value,0)),
'Number of Large Writes:'||
sum(decode(name,'physical write total multi block requests',value,0)),
'Total Bytes Read :'||
sum(decode(name,'physical read total bytes',value,0)),
'Total Bytes Written :'||
sum(decode(name,'physical write total bytes',value,0))
FROM gv$sysstat;
***Warning ****
DO NOT run ORION on a production Machine or any
machine that has data you are concerned about
KEEPING!
ORION Benchmarking
In Practice
ORION Benchmarking
Drive Tray
14
Drive
Trays
SAN
ORION Benchmarking
Test Scenarios / Methodology
Prove Performance
– Test various RAID Levels
• 0,1,0+1,5
– Test Various Read/Write Percentages
• 0/100, 75/25, 0/100
Prove Scalability
– Assume Entry Level
• 4 Trays (8TB)
– Expand Storage
• 8, 12, 14 Trays (17TB, 26TB, 30TB)
Automate All Tests
– Pre-Build a RAID System
– Present all volumes to Host
– Run Scripts
– 48 Tests / 250 Hours
ORION Benchmarking
Volume Group Layout – RAID 0+1
slot 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Tray 1 VG 1 VG 1 VG 2 VG 2 VG 3 VG 3 VG 4 VG 4 VG 5 VG 5 VG 6 VG 6 VG 7 VG 7 VG 8 VG 8
Tray 2 VG 17 VG 17 VG 18 VG 18 VG 19 VG 19 VG 20 VG 20 VG 21 VG 21 VG 22 VG 22 VG 23 VG 23 VG 24 VG 24
Tray 3 VG 33 VG 33 VG 34 VG 34 VG 35 VG 35 VG 36 VG 36 VG 37 VG 37 VG 38 VG 38 VG 39 VG 39 VG 40 VG 40
Tray 4 VG 49 VG 49 VG 50 VG 50 VG 51 VG 51 VG 52 VG 52 VG 53 VG 53 VG 54 VG 54 VG 55 VG 55 VG 56 VG 56
Tray 5 VG 1 VG 1 VG 2 VG 2 VG 3 VG 3 VG 4 VG 4 VG 5 VG 5 VG 6 VG 6 VG 7 VG 7 VG 8 VG 8
Tray 6 VG 17 VG 17 VG 18 VG 18 VG 19 VG 19 VG 20 VG 20 VG 21 VG 21 VG 22 VG 22 VG 23 VG 23 VG 24 VG 24
Tray 7 VG 33 VG 33 VG 34 VG 34 VG 35 VG 35 VG 36 VG 36 VG 37 VG 37 VG 38 VG 38 VG 39 VG 39 VG 40 VG 40
Tray 8 VG 9 VG 9 VG 10 VG 10 VG 11 VG 11 VG 12 VG 12 VG 13 VG 13 VG 14 VG 14 VG 15 VG 15 VG 16 VG 16
Tray 9 VG 25 VG 25 VG 26 VG 26 VG 27 VG 27 VG 28 VG 28 VG 29 VG 29 VG 30 VG 30 VG 31 VG 31 VG 32 VG 32
Tray 10 VG 41 VG 41 VG 42 VG 42 VG 43 VG 43 VG 44 VG 44 VG 45 VG 45 VG 46 VG 46 VG 47 VG 47 VG 48 VG 48
Tray 11 VG 49 VG 49 VG 50 VG 50 VG 51 VG 51 VG 52 VG 52 VG 53 VG 53 VG 54 VG 54 VG 55 VG 55 VG 56 VG 56
Tray 12 VG 9 VG 9 VG 10 VG 10 VG 11 VG 11 VG 12 VG 12 VG 13 VG 13 VG 14 VG 14 VG 15 VG 15 VG 16 VG 16
Tray 13 VG 25 VG 25 VG 26 VG 26 VG 27 VG 27 VG 28 VG 28 VG 29 VG 29 VG 30 VG 30 VG 31 VG 31 VG 32 VG 32
Tray 14 VG 41 VG 41 VG 42 VG 42 VG 43 VG 43 VG 44 VG 44 VG 45 VG 45 VG 46 VG 46 VG 47 VG 47 VG 48 VG 48
ORION Benchmarking
What to Test : 4-Tray Test : 64 Disks
slot 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Tray 1 VG 1 VG 1 VG 2 VG 2 VG 3 VG 3 VG 4 VG 4 VG 5 VG 5 VG 6 VG 6 VG 7 VG 7 VG 8 VG 8
Tray 2 VG 17 VG 17 VG 18 VG 18 VG 19 VG 19 VG 20 VG 20 VG 21 VG 21 VG 22 VG 22 VG 23 VG 23 VG 24 VG 24
Tray 3 VG 33 VG 33 VG 34 VG 34 VG 35 VG 35 VG 36 VG 36 VG 37 VG 37 VG 38 VG 38 VG 39 VG 39 VG 40 VG 40
Tray 4 VG 49 VG 49 VG 50 VG 50 VG 51 VG 51 VG 52 VG 52 VG 53 VG 53 VG 54 VG 54 VG 55 VG 55 VG 56 VG 56
Tray 5 VG 1 VG 1 VG 2 VG 2 VG 3 VG 3 VG 4 VG 4 VG 5 VG 5 VG 6 VG 6 VG 7 VG 7 VG 8 VG 8
Tray 6 VG 17 VG 17 VG 18 VG 18 VG 19 VG 19 VG 20 VG 20 VG 21 VG 21 VG 22 VG 22 VG 23 VG 23 VG 24 VG 24
Tray 7 VG 33 VG 33 VG 34 VG 34 VG 35 VG 35 VG 36 VG 36 VG 37 VG 37 VG 38 VG 38 VG 39 VG 39 VG 40 VG 40
Tray 8 VG 9 VG 9 VG 10 VG 10 VG 11 VG 11 VG 12 VG 12 VG 13 VG 13 VG 14 VG 14 VG 15 VG 15 VG 16 VG 16
Tray 9 VG 25 VG 25 VG 26 VG 26 VG 27 VG 27 VG 28 VG 28 VG 29 VG 29 VG 30 VG 30 VG 31 VG 31 VG 32 VG 32
Tray 10 VG 41 VG 41 VG 42 VG 42 VG 43 VG 43 VG 44 VG 44 VG 45 VG 45 VG 46 VG 46 VG 47 VG 47 VG 48 VG 48
Tray 11 VG 49 VG 49 VG 50 VG 50 VG 51 VG 51 VG 52 VG 52 VG 53 VG 53 VG 54 VG 54 VG 55 VG 55 VG 56 VG 56
Tray 12 VG 9 VG 9 VG 10 VG 10 VG 11 VG 11 VG 12 VG 12 VG 13 VG 13 VG 14 VG 14 VG 15 VG 15 VG 16 VG 16
Tray 13 VG 25 VG 25 VG 26 VG 26 VG 27 VG 27 VG 28 VG 28 VG 29 VG 29 VG 30 VG 30 VG 31 VG 31 VG 32 VG 32
Tray 14 VG 41 VG 41 VG 42 VG 42 VG 43 VG 43 VG 44 VG 44 VG 45 VG 45 VG 46 VG 46 VG 47 VG 47 VG 48 VG 48
ORION Benchmarking
What to Test : 8-Tray Test : 128 Disks
slot 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Tray 1 VG 1 VG 1 VG 2 VG 2 VG 3 VG 3 VG 4 VG 4 VG 5 VG 5 VG 6 VG 6 VG 7 VG 7 VG 8 VG 8
Tray 2 VG 17 VG 17 VG 18 VG 18 VG 19 VG 19 VG 20 VG 20 VG 21 VG 21 VG 22 VG 22 VG 23 VG 23 VG 24 VG 24
Tray 3 VG 33 VG 33 VG 34 VG 34 VG 35 VG 35 VG 36 VG 36 VG 37 VG 37 VG 38 VG 38 VG 39 VG 39 VG 40 VG 40
Tray 4 VG 49 VG 49 VG 50 VG 50 VG 51 VG 51 VG 52 VG 52 VG 53 VG 53 VG 54 VG 54 VG 55 VG 55 VG 56 VG 56
Tray 5 VG 1 VG 1 VG 2 VG 2 VG 3 VG 3 VG 4 VG 4 VG 5 VG 5 VG 6 VG 6 VG 7 VG 7 VG 8 VG 8
Tray 6 VG 17 VG 17 VG 18 VG 18 VG 19 VG 19 VG 20 VG 20 VG 21 VG 21 VG 22 VG 22 VG 23 VG 23 VG 24 VG 24
Tray 7 VG 33 VG 33 VG 34 VG 34 VG 35 VG 35 VG 36 VG 36 VG 37 VG 37 VG 38 VG 38 VG 39 VG 39 VG 40 VG 40
Tray 8 VG 9 VG 9 VG 10 VG 10 VG 11 VG 11 VG 12 VG 12 VG 13 VG 13 VG 14 VG 14 VG 15 VG 15 VG 16 VG 16
Tray 9 VG 25 VG 25 VG 26 VG 26 VG 27 VG 27 VG 28 VG 28 VG 29 VG 29 VG 30 VG 30 VG 31 VG 31 VG 32 VG 32
Tray 10 VG 41 VG 41 VG 42 VG 42 VG 43 VG 43 VG 44 VG 44 VG 45 VG 45 VG 46 VG 46 VG 47 VG 47 VG 48 VG 48
Tray 11 VG 49 VG 49 VG 50 VG 50 VG 51 VG 51 VG 52 VG 52 VG 53 VG 53 VG 54 VG 54 VG 55 VG 55 VG 56 VG 56
Tray 12 VG 9 VG 9 VG 10 VG 10 VG 11 VG 11 VG 12 VG 12 VG 13 VG 13 VG 14 VG 14 VG 15 VG 15 VG 16 VG 16
Tray 13 VG 25 VG 25 VG 26 VG 26 VG 27 VG 27 VG 28 VG 28 VG 29 VG 29 VG 30 VG 30 VG 31 VG 31 VG 32 VG 32
Tray 14 VG 41 VG 41 VG 42 VG 42 VG 43 VG 43 VG 44 VG 44 VG 45 VG 45 VG 46 VG 46 VG 47 VG 47 VG 48 VG 48
ORION Benchmarking
What to Test : 12-Tray Test : 192 Disks
slot 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Tray 1 VG 1 VG 1 VG 2 VG 2 VG 3 VG 3 VG 4 VG 4 VG 5 VG 5 VG 6 VG 6 VG 7 VG 7 VG 8 VG 8
Tray 2 VG 17 VG 17 VG 18 VG 18 VG 19 VG 19 VG 20 VG 20 VG 21 VG 21 VG 22 VG 22 VG 23 VG 23 VG 24 VG 24
Tray 3 VG 33 VG 33 VG 34 VG 34 VG 35 VG 35 VG 36 VG 36 VG 37 VG 37 VG 38 VG 38 VG 39 VG 39 VG 40 VG 40
Tray 4 VG 49 VG 49 VG 50 VG 50 VG 51 VG 51 VG 52 VG 52 VG 53 VG 53 VG 54 VG 54 VG 55 VG 55 VG 56 VG 56
Tray 5 VG 1 VG 1 VG 2 VG 2 VG 3 VG 3 VG 4 VG 4 VG 5 VG 5 VG 6 VG 6 VG 7 VG 7 VG 8 VG 8
Tray 6 VG 17 VG 17 VG 18 VG 18 VG 19 VG 19 VG 20 VG 20 VG 21 VG 21 VG 22 VG 22 VG 23 VG 23 VG 24 VG 24
Tray 7 VG 33 VG 33 VG 34 VG 34 VG 35 VG 35 VG 36 VG 36 VG 37 VG 37 VG 38 VG 38 VG 39 VG 39 VG 40 VG 40
Tray 8 VG 9 VG 9 VG 10 VG 10 VG 11 VG 11 VG 12 VG 12 VG 13 VG 13 VG 14 VG 14 VG 15 VG 15 VG 16 VG 16
Tray 9 VG 25 VG 25 VG 26 VG 26 VG 27 VG 27 VG 28 VG 28 VG 29 VG 29 VG 30 VG 30 VG 31 VG 31 VG 32 VG 32
Tray 10 VG 41 VG 41 VG 42 VG 42 VG 43 VG 43 VG 44 VG 44 VG 45 VG 45 VG 46 VG 46 VG 47 VG 47 VG 48 VG 48
Tray 11 VG 49 VG 49 VG 50 VG 50 VG 51 VG 51 VG 52 VG 52 VG 53 VG 53 VG 54 VG 54 VG 55 VG 55 VG 56 VG 56
Tray 12 VG 9 VG 9 VG 10 VG 10 VG 11 VG 11 VG 12 VG 12 VG 13 VG 13 VG 14 VG 14 VG 15 VG 15 VG 16 VG 16
Tray 13 VG 25 VG 25 VG 26 VG 26 VG 27 VG 27 VG 28 VG 28 VG 29 VG 29 VG 30 VG 30 VG 31 VG 31 VG 32 VG 32
Tray 14 VG 41 VG 41 VG 42 VG 42 VG 43 VG 43 VG 44 VG 44 VG 45 VG 45 VG 46 VG 46 VG 47 VG 47 VG 48 VG 48
ORION Benchmarking
What to Test : 14-Tray Test : 224 Disks
slot 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Tray 1 VG 1 VG 1 VG 2 VG 2 VG 3 VG 3 VG 4 VG 4 VG 5 VG 5 VG 6 VG 6 VG 7 VG 7 VG 8 VG 8
Tray 2 VG 17 VG 17 VG 18 VG 18 VG 19 VG 19 VG 20 VG 20 VG 21 VG 21 VG 22 VG 22 VG 23 VG 23 VG 24 VG 24
Tray 3 VG 33 VG 33 VG 34 VG 34 VG 35 VG 35 VG 36 VG 36 VG 37 VG 37 VG 38 VG 38 VG 39 VG 39 VG 40 VG 40
Tray 4 VG 49 VG 49 VG 50 VG 50 VG 51 VG 51 VG 52 VG 52 VG 53 VG 53 VG 54 VG 54 VG 55 VG 55 VG 56 VG 56
Tray 5 VG 1 VG 1 VG 2 VG 2 VG 3 VG 3 VG 4 VG 4 VG 5 VG 5 VG 6 VG 6 VG 7 VG 7 VG 8 VG 8
Tray 6 VG 17 VG 17 VG 18 VG 18 VG 19 VG 19 VG 20 VG 20 VG 21 VG 21 VG 22 VG 22 VG 23 VG 23 VG 24 VG 24
Tray 7 VG 33 VG 33 VG 34 VG 34 VG 35 VG 35 VG 36 VG 36 VG 37 VG 37 VG 38 VG 38 VG 39 VG 39 VG 40 VG 40
Tray 8 VG 9 VG 9 VG 10 VG 10 VG 11 VG 11 VG 12 VG 12 VG 13 VG 13 VG 14 VG 14 VG 15 VG 15 VG 16 VG 16
Tray 9 VG 25 VG 25 VG 26 VG 26 VG 27 VG 27 VG 28 VG 28 VG 29 VG 29 VG 30 VG 30 VG 31 VG 31 VG 32 VG 32
Tray 10 VG 41 VG 41 VG 42 VG 42 VG 43 VG 43 VG 44 VG 44 VG 45 VG 45 VG 46 VG 46 VG 47 VG 47 VG 48 VG 48
Tray 11 VG 49 VG 49 VG 50 VG 50 VG 51 VG 51 VG 52 VG 52 VG 53 VG 53 VG 54 VG 54 VG 55 VG 55 VG 56 VG 56
Tray 12 VG 9 VG 9 VG 10 VG 10 VG 11 VG 11 VG 12 VG 12 VG 13 VG 13 VG 14 VG 14 VG 15 VG 15 VG 16 VG 16
Tray 13 VG 25 VG 25 VG 26 VG 26 VG 27 VG 27 VG 28 VG 28 VG 29 VG 29 VG 30 VG 30 VG 31 VG 31 VG 32 VG 32
Tray 14 VG 41 VG 41 VG 42 VG 42 VG 43 VG 43 VG 44 VG 44 VG 45 VG 45 VG 46 VG 46 VG 47 VG 47 VG 48 VG 48
ORION Benchmarking
Scripts : crontab Entry
35 14 22 3 * /orion/Host/bin/orion_run.ksh
ORION Benchmarking
Scripts : orion_run.ksh
#!/usr/bin/ksh
cd /orion/Host/bin
cat /orion/Host/bin/runnames.txt |\
while read NAME
do
DISKS=`grep $NAME /orion/Host/bin/rundata.txt | awk '{print $2}'`
SMALL=`grep $NAME /orion/Host/bin/rundata.txt | awk '{print $3}'`
LARGE=`grep $NAME /orion/Host/bin/rundata.txt | awk '{print $4}'`
TYPE=`grep $NAME /orion/Host/bin/rundata.txt | awk '{print $5}'`
SIMULATE=`grep $NAME /orion/Host/bin/rundata.txt | awk '{print $6}'`
DURATION=`grep $NAME /orion/Host/bin/rundata.txt | awk '{print $7}'`
WRITE=`grep $NAME /orion/Host/bin/rundata.txt | awk '{print $8}'`
MATRIX=`grep $NAME /orion/Host/bin/rundata.txt | awk '{print $9}'`
/orion/Host/bin/tray01.ksh $NAME $DISKS $SMALL $LARGE $TYPE $SIMULATE $DURATION $WRITE $MATRIX &
/orion/Host/bin/tray02.ksh $NAME $DISKS $SMALL $LARGE $TYPE $SIMULATE $DURATION $WRITE $MATRIX &
/orion/Host/bin/tray03.ksh $NAME $DISKS $SMALL $LARGE $TYPE $SIMULATE $DURATION $WRITE $MATRIX &
/orion/Host/bin/tray04.ksh $NAME $DISKS $SMALL $LARGE $TYPE $SIMULATE $DURATION $WRITE $MATRIX &
/orion/Host/bin/tray05.ksh $NAME $DISKS $SMALL $LARGE $TYPE $SIMULATE $DURATION $WRITE $MATRIX &
/orion/Host/bin/tray06.ksh $NAME $DISKS $SMALL $LARGE $TYPE $SIMULATE $DURATION $WRITE $MATRIX &
/orion/Host/bin/tray07.ksh $NAME $DISKS $SMALL $LARGE $TYPE $SIMULATE $DURATION $WRITE $MATRIX &
/orion/Host/bin/tray08.ksh $NAME $DISKS $SMALL $LARGE $TYPE $SIMULATE $DURATION $WRITE $MATRIX &
/orion/Host/bin/tray09.ksh $NAME $DISKS $SMALL $LARGE $TYPE $SIMULATE $DURATION $WRITE $MATRIX &
/orion/Host/bin/tray10.ksh $NAME $DISKS $SMALL $LARGE $TYPE $SIMULATE $DURATION $WRITE $MATRIX &
/orion/Host/bin/tray11.ksh $NAME $DISKS $SMALL $LARGE $TYPE $SIMULATE $DURATION $WRITE $MATRIX &
/orion/Host/bin/tray12.ksh $NAME $DISKS $SMALL $LARGE $TYPE $SIMULATE $DURATION $WRITE $MATRIX &
/orion/Host/bin/tray13.ksh $NAME $DISKS $SMALL $LARGE $TYPE $SIMULATE $DURATION $WRITE $MATRIX &
/orion/Host/bin/tray14.ksh $NAME $DISKS $SMALL $LARGE $TYPE $SIMULATE $DURATION $WRITE $MATRIX &
sleep 300
count=`ls -1 /orion/Host/bin/*.flag | wc -l`
while [ $count -gt 0 ]
do
sleep 600
count=`ls -1 /orion/Host/bin/*.flag | wc -l`
done
done
ORION Benchmarking
Scripts : runnames.txt
R10DTC4SIO8randomLIO1024randomRPCT100
R10DTC4SIO8randomLIO1024randomRPCT75
R10DTC4SIO8randomLIO1024randomRPCT0
R10DTC4SIO8randomLIO1024sequentialRPCT0
R10DTC8SIO8randomLIO1024randomRPCT100
R10DTC8SIO8randomLIO1024randomRPCT75
R10DTC8SIO8randomLIO1024randomRPCT0
R10DTC8SIO8randomLIO1024sequentialRPCT0
R10DTC12SIO8randomLIO1024randomRPCT100
R10DTC12SIO8randomLIO1024randomRPCT75
R10DTC12SIO8randomLIO1024randomRPCT0
R10DTC12SIO8randomLIO1024sequentialRPCT0
R10DTC14SIO8randomLIO1024randomRPCT100
R10DTC14SIO8randomLIO1024randomRPCT75
R10DTC14SIO8randomLIO1024randomRPCT0
R10DTC14SIO8randomLIO1024sequentialRPCT0
ORION Benchmarking
Scripts : rundata.txt
testname num_disks size_small size_large type simulate duration write matrix
R10DTC4SIO8randomLIO1024randomRPCT100 16 8 1024 rand concat 120 0 basic
R10DTC4SIO8randomLIO1024randomRPCT75 16 8 1024 rand concat 120 25 basic
R10DTC4SIO8randomLIO1024randomRPCT0 16 8 1024 rand concat 120 100 basic
R10DTC8SIO8randomLIO1024randomRPCT100 16 8 1024 rand concat 120 0 basic
R10DTC8SIO8randomLIO1024randomRPCT75 16 8 1024 rand concat 120 25 basic
R10DTC8SIO8randomLIO1024randomRPCT0 16 8 1024 rand concat 120 100 basic
R10DTC12SIO8randomLIO1024randomRPCT100 16 8 1024 rand concat 120 0 basic
R10DTC12SIO8randomLIO1024randomRPCT75 16 8 1024 rand concat 120 25 basic
R10DTC12SIO8randomLIO1024randomRPCT0 16 8 1024 rand concat 120 100 basic
R10DTC14SIO8randomLIO1024randomRPCT100 16 8 1024 rand concat 120 0 basic
R10DTC14SIO8randomLIO1024randomRPCT75 16 8 1024 rand concat 120 25 basic
R10DTC14SIO8randomLIO1024randomRPCT0 16 8 1024 rand concat 120 100 basic
ORION Benchmarking
Scripts : tray01.ksh
#!/usr/bin/ksh
cd /orion/Host/bin
if [ -f 01_$2.lun ]
then
touch /orion/Host/bin/01.flag
cp /orion/Host/bin/01_$2.lun /orion/Host/bin/01_$1.lun
/orion/Host/bin/orion_linux
-run advanced -testname 01_$1 -num_disks $2
-size_small $3 -size_large $4 -type $5
-simulate $6 -duration $7 -write $8 -matrix $9
rm /orion/Host/bin/01.flag
fi
ORION Benchmarking
Scripts : 01_16.lun
/dev/sdb
/dev/sddl 4 Volume Groups (Luns)
/dev/sdh 16 Drives
/dev/sddq
slot 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Tray 1 VG 1 VG 1 VG 2 VG 2 VG 3 VG 3 VG 4 VG 4 VG 5 VG 5 VG 6 VG 6 VG 7 VG 7 VG 8 VG 8
Tray 2 VG 17 VG 17 VG 18 VG 18 VG 19 VG 19 VG 20 VG 20 VG 21 VG 21 VG 22 VG 22 VG 23 VG 23 VG 24 VG 24
Tray 3 VG 33 VG 33 VG 34 VG 34 VG 35 VG 35 VG 36 VG 36 VG 37 VG 37 VG 38 VG 38 VG 39 VG 39 VG 40 VG 40
Tray 4 VG 49 VG 49 VG 50 VG 50 VG 51 VG 51 VG 52 VG 52 VG 53 VG 53 VG 54 VG 54 VG 55 VG 55 VG 56 VG 56
Tray 5 VG 1 VG 1 VG 2 VG 2 VG 3 VG 3 VG 4 VG 4 VG 5 VG 5 VG 6 VG 6 VG 7 VG 7 VG 8 VG 8
tray01.ksh tray05.ksh
(01_16.lun) (05_16.lun)
ORION Benchmarking
Understanding the Test Results
ORION Outputs
– IOPS
– MBPS
– Latencies
– Summary
IOPS Overview
Graphical Results
– 0, 75, 100 percent Reads
ORION Benchmarking
IOPS Results
Input/outputs per second (IOPS) are primarily a
concern in an online transaction processing
(OLTP) application environment. OLTP
application environments have these
characteristics:
– High intensity of reads and writes
– Highly transaction-based applications
– Large bursts of small reads followed by inserts and
updates
– Small random I/Os, typically the size of the database
block size
– Read-to-write ratios, typically 70:30
ORION Benchmarking
IOPS Results - Overview
RAID Drive Tray Drive Read-to-Write Ratios
Level Count Count 0:100 75:25 100:0
0 4 64 16329 18759 20121
0 8 128 30960 35086 37500
0 12 192 42685 47302 50342
0 14 224 48290 52917 56288
1 4 64 9043 15536 20885
1 8 128 17414 29305 38845
1 12 192 24481 39376 51189
1 14 224 27885 43788 56467
10 4 64 8513 14720 19365
10 8 128 16577 27605 36186
10 12 192 23131 36766 47360
10 14 224 26104 40614 52034
5 4 64 4258 9997 18149
5 8 128 8180 18958 34283
5 12 192 11182 25128 44831
5 14 224 12564 27797 49295
ORION Benchmarking
IOPS Results - Overview
Read-to-Write Ratios IOPS / Drive
RAID Drive Tray Drive
Level Count Count 0:100 75:25 100:0 100:0
0 4 64 16329 18759 20121 314
0 8 128 30960 35086 37500 292
0 12 192 42685 47302 50342 254
0 14 224 48290 52917 56288 251
1 4 64 9043 15536 20885 326
1 8 128 17414 29305 38845 303
1 12 192 24481 39376 51189 266
1 14 224 27885 43788 56467 252
10 4 64 8513 14720 19365 302
10 8 128 16577 27605 36186 282
10 12 192 23131 36766 47360 246
10 14 224 26104 40614 52034 232
5 4 64 4258 9997 18149 283
5 8 128 8180 18958 34283 267
5 12 192 11182 25128 44831 233
5 14 224 12564 27797 49295 220
ORION Benchmarking
IOPS - RAID Comparison - Read=0%
I/O Throughput (RAID Comparison, Read=0% ) I/O Throughput (RAID Comparison, Read=0% )
60000 60000
50000 50000
40000 40000
64 Drives RAID 0 128 Drives RAID 0
IOPS
IOPS
64 Drives RAID 1 128 Drives RAID 1
30000 30000
64 Drives RAID 10 128 Drives RAID 10
64 Drives RAID 5 128 Drives RAID 5
20000 20000
10000 10000
0 0
0 20 40 60 80 100 0 20 40 60 80 100
Outstanding Asynchronous I/O (Load) Outstanding Asynchronous I/O (Load)
I/O Throughput (RAID Comparison, Read=0% ) I/O Throughput (RAID Comparison, Read=0% )
60000 60000
50000 50000
40000 40000
192 Drives RAID 0 224 Drives RAID 0
IOPS
IOPS
10000 10000
0 0
0 20 40 60 80 100 0 20 40 60 80 100 120
Outstanding Asynchronous I/O (Load) Outstanding Asynchronous I/O (Load)
ORION Benchmarking
IOPS - RAID Comparison - Read=75%
I/O Throughput (RAID Comparison, Read=75% ) I/O Throughput (RAID Comparison, Read=75% )
60000 60000
50000 50000
40000 40000
64 Drives RAID 0 128 Drives RAID 0
IOPS
IOPS
64 Drives RAID 1 128 Drives RAID 1
30000 30000
64 Drives RAID 10 128 Drives RAID 10
64 Drives RAID 5 128 Drives RAID 5
20000 20000
10000 10000
0 0
0 20 40 60 80 100 0 20 40 60 80 100
Outstanding Asynchronous I/O (Load) Outstanding Asynchronous I/O (Load)
I/O Throughput (RAID Comparison, Read=75% ) I/O Throughput (RAID Comparison, Read=75% )
60000 60000
50000 50000
40000 40000
192 Drives RAID 0 224 Drives RAID 0
IOPS
IOPS
10000 10000
0 0
0 20 40 60 80 100 0 20 40 60 80 100 120
Outstanding Asynchronous I/O (Load) Outstanding Asynchronous I/O (Load)
ORION Benchmarking
IOPS - RAID Comparison - Read=100%
I/O Throughput (RAID Comparison, Read=100% ) I/O Throughput (RAID Comparison, Read=100% )
60000 60000
50000 50000
40000 40000
64 Drives RAID 0 128 Drives RAID 0
IOPS
IOPS
64 Drives RAID 1 128 Drives RAID 1
30000 30000
64 Drives RAID 10 128 Drives RAID 10
64 Drives RAID 5 128 Drives RAID 5
20000 20000
10000 10000
0 0
0 20 40 60 80 100 0 20 40 60 80 100
Outstanding Asynchronous I/O (Load) Outstanding Asynchronous I/O (Load)
I/O Throughput (RAID Comparison, Read=100% ) I/O Throughput (RAID Comparison, Read=100% )
60000 60000
50000 50000
40000 40000
192 Drives RAID 0 224 Drives RAID 0
IOPS
IOPS
10000 10000
0 0
0 20 40 60 80 100 0 20 40 60 80 100 120
Outstanding Asynchronous I/O (Load) Outstanding Asynchronous I/O (Load)
Existing System
Can I Validate?
Existing System
What Can You Do
Disk Characteristics
– Check Profile of SAN
Calculating IOPS from Vendor Data
Calculating MBPS from Vendor Data
Monitoring Your System
– Database Statistics
– Sample Scripts
– Validating IOPS & MBPS
Re-Configuration
Existing System
Disk Characteristics - What a Vendor Gives You
RPM
Revolutions Per Minute. The
number of rotations completed
in one minute.
•Average Latency
(Rotational Latency) Average
time after head is over the
track for the target sector to
rotate under the head before a
read or write.
Existing System
Disk Characteristics - Calculate IOPS for Disk
RPM Rotations Rotations Full Rotation Rotational Latency Average IO IOPS
Rotations Per Per (Half Rotation) Seek Time
Per Second Mili-second Time
Minute
Y Z (Y+Z) (1/[Y+Z])*1000
X Y Z (X*Y*Z)/1024
71 1 128K 9MBPS
71 1 128K 9MBPS
ORION Benchmarking
MBPS - RAID Comparison-Read=various%
Data Transfer Rate (RAID Comparison, Read=0%) Data Transfer Rate (RAID Comparison, Read=75%)
1800 1800
1600 1600
1400 1400
1200 1200
224 Drives RAID 0 224 Drives RAID 0
MBPS
MBPS
1000 224 Drives RAID 1 1000 224 Drives RAID 1
800 224 Drives RAID 10 800 224 Drives RAID 10
224 Drives RAID 5 224 Drives RAID 5
600 600
400 400
200 200
0 0
0 10 20 30 40 50 0 10 20 30 40 50
Outstanding Asynchronous I/O (Load) Outstanding Asynchronous I/O (Load)
1800
1600
1400
1200
224 Drives RAID 0
MBPS
1000 224 Drives RAID 1
800 224 Drives RAID 10
224 Drives RAID 5
600
400
200
0
0 10 20 30 40 50
Outstanding Asynchronous I/O (Load)
Existing System
Scripts – Validate IOPS & MBPS
Set of scripts to run against your existing
database
Check / Validate what the database sees
against what you expect from hardware
Run against current statistics
Run against Snapshots
Existing System
Report IOPS & MBPS over time
Purpose
– Report IOPS & MBPS
• For small and large I/O
• Percentage of Reads to Writes
• User Defined Sample Period
Importance
– Matching Expected I/O rates on the Disk Subsystem
to What is Actually Seen in the Database
Script
– vsysstat_ioworkload.sql
Existing System
vsysstat_ioworkload.sql - Output
First Sample
Number of Small Reads : 6773457
Number of Small Writes: 8150855
Number of Large Reads : 2114186
Number of Large Writes: 5860450
Total Bytes Read : 3.1256E+11
Total Bytes Written : 2.8618E+11
Enter the amount of time (in seconds) you would like this process to sleep for sampling data
Enter value for sleeptime: 300
Sleep Time (secs): 300
Second Sample
Number of Small Reads : 6778045
Number of Small Writes: 8162002
Number of Large Reads : 2114206
Number of Large Writes: 5867811
Total Bytes Read : 3.1260E+11
Total Bytes Written : 2.8651E+11
Results :
Small Read IOPS = 15.293 IOPS
Small Write IOPS = 37.157 IOPS
Total Small IOPS = 52.45 IOPS
Small Read I/O % = 29.158 %
Small Write I/O % = 70.842 %
Large Read IOPS = .067 IOPS
Large Write IOPS = 24.537 IOPS
Total Large IOPS = 24.603 IOPS
Large Read I/O % = .271 %
Large Write I/O % = 99.729 %
Total Read = .127 MBPS
Total Written = 1.049 MBPS
Total MBPS = 1.176 MBPS
Existing System
Report IOPS & MBPS over Snapshots
Purpose
– Report IOPS & MBPS
• Uses Snapshots ; Workload Repository History (WRH)
• For small and large I/O
• Percentage of Reads to Writes
• User Defined Sample Period
Importance
– Matching Expected I/O rates on the Disk Subsystem
to What is Actually Seen in the Database
Warning
– Performance Tuning Pack Requirement
Script
– wrh_sysstat_ioworkload.sql
Existing System
wrh_vsysstat_ioworkload.sql - Output
First Sample
Number of Small Reads : 24352077
Number of Small Writes: 21296716
Number of Large Reads : 3305589
Number of Large Writes: 14464153
Total Bytes Read : 5.6736E+11
Total Bytes Written : 5.6482E+11
Second Sample
Number of Small Reads : 24759830
Number of Small Writes: 21652738
Number of Large Reads : 3314771
Number of Large Writes: 14664751
Total Bytes Read : 5.7180E+11
Total Bytes Written : 5.7518E+11
Results :
Small Read IOPS = 113.265 IOPS
Small Write IOPS = 98.895 IOPS
Total Small IOPS = 212.16 IOPS
Small Read I/O % = 53.387 %
Small Write I/O % = 46.613 %
Large Read IOPS = 2.551 IOPS
Large Write IOPS = 55.722 IOPS
Total Large IOPS = 58.272 IOPS
Large Read I/O % = 4.377 %
Large Write I/O % = 95.623 %
Total Read = 1.176 MBPS
Total Written = 2.744 MBPS
Total MBPS = 3.921 MBPS
Existing System
Report IOPS & MBPS over Snapshots
Purpose
– Report IOPS & MBPS
• Uses Snapshots ; Workload Repository History (WRH)
• For small and large I/O
• Percentage of Reads to Writes
• EVERYTHING
Importance
– Matching Expected I/O rates on the Disk Subsystem
to What is Actually Seen in the Database
Warning
– Performance Tuning Pack Requirement
Script
– wrh_sysstat_ioworkload_ALL.sql
Existing System
wrh_vsysstat_ioworkload_ALL.sql - Output
Small Small Total Small Small
Read Write Small Read Write
END_TIME IOPS IOPS IOPS I/O% I/O%
------------------------- ---------- ---------- ---------- ---------- ----------
13-AUG-07 07.00.54.540 AM 43.117 53.969 97.087 44.411 55.589
13-AUG-07 08.00.56.844 AM 41.677 53.642 95.318 43.724 56.276
13-AUG-07 09.00.00.297 AM 83.613 66.803 150.417 55.588 44.412
13-AUG-07 10.00.04.307 AM 143.974 122.113 266.087 54.108 45.892
13-AUG-07 11.00.07.604 AM 178.109 149.302 327.41 54.399 45.601
13-AUG-07 12.00.10.494 PM 100.027 86.229 186.256 53.704 46.296
13-AUG-07 01.00.13.213 PM 119.235 94.052 213.288 55.904 44.096
13-AUG-07 02.00.17.176 PM 239.61 198.644 438.254 54.674 45.326
13-AUG-07 03.00.20.998 PM 139.792 108.863 248.655 56.219 43.781
13-AUG-07 04.00.26.942 PM 457.685 277.04 734.725 62.293 37.707
13-AUG-07 05.00.30.527 PM 150.167 118.779 268.946 55.835 44.165
13-AUG-07 06.00.33.084 PM 30.141 24.554 54.695 55.108 44.892
13-AUG-07 07.00.36.907 PM 76.618 70.219 146.837 52.179 47.821
13-AUG-07 08.00.39.735 PM 31.321 45.13 76.451 40.968 59.032
13-AUG-07 09.00.42.588 PM 197.372 150.143 347.515 56.795 43.205
13-AUG-07 10.00.45.432 PM 37.311 32.712 70.022 53.284 46.716
13-AUG-07 11.00.48.299 PM 265.166 69.311 334.477 79.278 20.722
14-AUG-07 12.00.50.723 AM 186.941 67.7 254.641 73.413 26.587
14-AUG-07 01.00.52.585 AM 118.794 94.839 213.633 55.607 44.393
14-AUG-07 02.00.55.051 AM 199.069 159.653 358.721 55.494 44.506
14-AUG-07 03.00.56.688 AM 38.899 38.201 77.1 50.452 49.548
Importance
– Investigate which mount point are experiencing the
most read or writes
– Use to see if any mount point is being used more
than others
Script
– vfilestat_iopercent_mount.sql
Existing System
vfilestat_iopercent_mount.sql - Output
Percentage
Mount of Total Percentage Percentage
Point I/O of Reads of Writes
------------------------------ ---------- ---------- ----------
/u00/app/oracle/oradata/prd .81 .59 1.18
/u02/app/oracle/oradata/prd .87 .94 .76
/u03/app/oracle/oradata/prd 7.20 .22 19.07
/u04/app/oracle/oradata/prd 46.31 45.90 47.01
/u05/app/oracle/oradata/prd 26.85 27.69 25.41
/u06/app/oracle/oradata/prd 5.12 7.49 1.08
/u07/app/oracle/oradata/prd .00 .00 .00
/u09/app/oracle/oradata/prd 4.03 4.45 3.33
/u10/app/oracle/oradata/prd .54 .71 .25
/u11/app/oracle/oradata/prd 8.28 12.02 1.90
10 rows selected.
Existing System
Report I/O Percentages
Purpose
– Report I/O Percentages
• Across mount points
• For small and large I/O
• Uses Snapshots ; Workload Repository History (WRH)
Importance
– Investigate which mount point are experiencing the most read
or writes
– Use to see if any mount point is being used more than others
Script
– wrh_vfilestat_iopercent_mount.sql
Existing System
wrh_vfilestat_iopercent.sql - Output
Percentage
Mount of Total Percentage Percentage
Point I/O of Reads of Writes
------------------------------ ---------- ---------- ----------
/u02/app/oracle/oradata/prd .93 .96 .89
/u03/app/oracle/oradata/prd 7.17 .22 19.32
/u04/app/oracle/oradata/prd 45.99 45.13 47.48
/u05/app/oracle/oradata/prd 27.99 29.32 25.67
/u06/app/oracle/oradata/prd 5.10 7.39 1.09
/u07/app/oracle/oradata/prd .00 .00 .00
/u09/app/oracle/oradata/prd 4.01 4.38 3.36
/u10/app/oracle/oradata/prd .57 .74 .25
/u11/app/oracle/oradata/prd 8.24 11.86 1.92
Conclusion / Questions
Simulating
Oracle I/O Workloads to
Configure Storage
James F. Koopmann
NoCOUG
Fall Conference 2007
4:00 – 5:00