DS8000 - Training
DS8000 - Training
IBM TotalStorage
ibm.com/storage
Architecture
aspects of the DS8000 series architecture
IBM TotalStorage
Cooling plenum disk enclosure pair disk enclosure pair disk enclosure pair disk enclosure pair disk enclosure pair
Cooling plenum disk enclosure pair disk enclosure pair disk enclosure pair disk enclosure pair disk enclosure pair disk enclosure pair
eServer p5 570
Primary power supply
eServer p5 570
I/O Enclosure 1
I/O enclosure 0
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
DS8000 Architecture
SAN fabric
host adapter
RIO-G
host adapter
N-way SMP
Volatile memory
Persistent memory
Complex 0 Complex 1
device adapter
Volatile memory
Persistent memory
N-way SMP
RIO-G
device adapter
Fibre channel switch Fibre channel switchFibre channel switch Fibre channel switch
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Virtualisation
virtualization concepts as they apply to the DS8000
IBM TotalStorage
LIC
Memory Processor
LIC
Memory Processor
LPAR Hypervisor
takes part of takes part of takes part of
Memor y Processor
I/ O
RIOG
I/ O
Memor y Processor
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Physical Layer
I/O Enclosure H H D H H D A A A A A A
RIO-G
I/O Enclosure H H D H H D A A A A A A
Server0
Switches
Server1
Switched loop 1
Switched loop 2
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Array Site
Array Site
Switch
Loop 1
Copyright IBM Corporation 2001. All rights reserved. 2005.
Loop 2
IBM TotalStorage
Creation of an Array
Array Site
D1 D2 D3 D7 D8 D9 D10 D11 P D12 D13 D14 D15 D16 P D17 D18 ... ... ... ... ... ... ...
Creation of an array
Data Data Data Data Data Data Parit y Spare
D4 D5 D6 P
RAID Array
Spar e
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Forming a Rank
Data Data Data Data Data Data Parity Spare
D1 D2 D3 D4 D5 D6 P D7 D8 D9 D10 D11 P D12 D13 D14 D15 D16 P D17 D18 ... ... ... ... ... ... ...
RAID Array
Creation of a Rank
.. . . .. . .
.. . .
1GB
1GB
1GB
1GB
.. . .
.. . .
.. . .
.. . .
.. . .
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Extent Pools
Extent Pool CKD0
1113 Cyl. CKD 1113 Cyl. CKD 1113 Cyl. CKD 1113 Cyl. CKD
For performance and ease of management create 1 Extent Pool per Rank
Server0
Extent Pool FBtest
1GB FB 1GB FB 1GB FB 1GB FB
Server1
1GB FB
1GB FB
1GB FB
1GB FB
1GB FB
1GB FB
1GB FB
1GB FB
1GB FB
1GB FB
1GB FB
1GB FB
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Creation of an FB LUN
Extent Pool FBprod Rank-a
1 GB 1 GB 1 GB 1 GB free
Logical 3 GB LUN
3 GB LUN Rank-b
used 1 GB free used 1 GB free
Allocate a 3 GB LUN
3 GB LUN Rank-b
used 1 GB used used 1 GB used
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Volume group: DB2test Host att: Test WWPN5 WWPN6 WWPN7 Host att: Prog WWPN8
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Virtualisation Hierarchy
Array Site RAID Array Data Data Data Data Data Data Parity Spare Rank Type FB Extent Pool Logical Volume 1 GB FB 1FB FB 1 GB GB 1 GB GB 1 GB FB 1FB FB
Server0
1 GB FB 1FB FB 1 GB GB
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Performance Considerations
early performance considerations regarding the DS8000 series
IBM TotalStorage
Non-balanced method: LUNs across ranks More than 1 rank per extent pool
2 GB LUN 1
Extent pool 1
8GB LUN
Exten t
Rank 2
1GB
Extent pool 2
2GB LUN 2
Extent pool 3
2GB LUN 3
Rank 7
Rank 3
Extent pool 4
2GB LUN 4
Rank 8
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
RIO-G Module
L1,2 Memory
Memory
Processo r
Processo r
Memory
L3 Memory
L1,2 Memory
Processo r
Processo r
L1,2 Memory
L3 Memory
Server0
POWER5 2-way SMP
RIO-G Module
Server1
RIO-G Module POWER5 2-way SMP
DAs Rank
Extent pool0
1
Extent pool2
3
Extent pool4
5
Extent pool1
2
Extent pool3
4
Extent pool5
6
11
10
12
Extent pool6
Extent pool8
Extent pool10
Extent pool7
Extent pool9
Extent pool11
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Implementation Tasks
IBM TotalStorage
Physical Installation
Hardware install - IBM CE
DS HMC - interna land external (if fitted) Call Home and Remote Support DS Storage Manager - CLI Must use DS CLI package shipped with DS Code DS Storage Manager - GUI http://<ipaddress>:8451/DS8000/Logon Note: DS Stoage Manger on Internal HMC can only provide real-time configuration
User Mangement
Default logon to DS GUI and CLI is admin/admin Password must be changed straight away: chuser -pw passw0rd admin
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
DS CLI
Must use CLI code shipped with DS code Runs on all OS Create profiles for each DS
Windows - stored in C:\Program Files\IBM\dscli\Profile\dsxxxx.profile
# # DS CLI Profile # # Management Console/Node IP Address(es) # hmc1 and hmc2 are equivalent to -hmc1 and -hmc2 command options. hmc1: 9.149.20.163 hmc2: 9.149.20.173 # # Default target Storage Image ID # "devid" and "remotedevid" are equivalent to # "-dev storage_image_ID" and "-remotedev storeage_image_ID" command options, respectively. devid: IBM.2107-75AZRY1
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
=============================================================== I0000 50050763040000D0 Online Fibre Channel-SW SCSI-FCP 0 I0001 50050763040040D0 Online Fibre Channel-SW SCSI-FCP 0 I0002 50050763040080D0 Online Fibre Channel-SW SCSI-FCP 0 I0003 500507630400C0D0 Online Fibre Channel-SW SCSI-FCP 0 I0030 50050763040300D0 Online Fibre Channel-SW SCSI-FCP 0 I0031 50050763040340D0 Online Fibre Channel-SW SCSI-FCP 0 I0032 50050763040380D0 Online Fibre Channel-SW SCSI-FCP 0 I0033 500507630403C0D0 Online Fibre Channel-SW SCSI-FCP 0 I0100 50050763040800D0 Online Fibre Channel-SW SCSI-FCP 0 I0101 50050763040840D0 Online Fibre Channel-SW SCSI-FCP 0 I0102 50050763040880D0 Online Fibre Channel-SW SCSI-FCP 0 I0103 500507630408C0D0 Online Fibre Channel-SW SCSI-FCP 0 I0130 50050763040B00D0 Online Fibre Channel-SW SCSI-FCP 0 I0131 50050763040B40D0 Online Fibre Channel-SW SCSI-FCP 0 I0132 50050763040B80D0 Online Fibre Channel-SW SCSI-FCP 0 I0133 50050763040BC0D0 Online Fibre Channel-SW SCSI-FCP 0 I0200 50050763041000D0 Online Fibre Channel-SW SCSI-FCP 0 I0201 50050763041040D0 Online Fibre Channel-SW SCSI-FCP 0 . .
IBM TotalStorage
IBM TotalStorage
Logical Configuration
1 2 3 4 5 6
Create arrays Create ranks Create extent pool Create logical volumes Create volume groups Create hosts & host groups
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Create Arrays
RAID5 or RAID10 Formed from Array Site of 8 DDMs Formed across the FC fabric
Similar to Arrays across loops in an ESS
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
=========================================== S1 S2 S3 S4 S5 S6 S7 S8 S9 S10 S11 S12 S13 S14 S15 . . . S48 7 146.0 Assigned A47 2 2 2 2 2 2 2 2 0 0 0 0 0 0 0 146.0 Assigned A0 146.0 Assigned A1 146.0 Assigned A2 146.0 Assigned A3 146.0 Assigned A4 146.0 Assigned A5 146.0 Assigned A6 146.0 Assigned A7 146.0 Assigned A8 146.0 Assigned A9 146.0 Assigned A10 146.0 Assigned A11 146.0 Assigned A12 146.0 Assigned A13 146.0 Assigned A14
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
================================================================== A0 A1 A2 A3 A4 A5 A6 A7 A8 A9 Assigned Normal 5 (6+P+S) S1 Assigned Normal 5 (6+P+S) S2 Assigned Normal 5 (6+P+S) S3 Assigned Normal 5 (6+P+S) S4 Assigned Normal 5 (7+P) S5 Assigned Normal 5 (7+P) S6 Assigned Normal 5 (7+P) S7 Assigned Normal 5 (7+P) S8 Assigned Normal 5 (6+P+S) S9 Assigned Normal 5 (6+P+S) S10 R0 2 R1 2 R2 2 R3 2 R4 2 R5 2 R6 2 R7 2 R8 0 R9 0 R10 0 R11 0 R12 0 R13 0 R14 0 R15 0 R16 6 146.0 146.0 146.0 146.0 146.0 146.0 146.0 146.0 146.0 146.0 146.0 146.0 146.0 146.0 146.0 146.0 146.0
A10 Assigned Normal 5 (6+P+S) S11 A11 Assigned Normal 5 (6+P+S) S12 A12 Assigned Normal 5 (7+P) S13 A13 Assigned Normal 5 (7+P) S14 A14 Assigned Normal 5 (7+P) S15 A15 Assigned Normal 5 (7+P) S16 A16 Assigned Normal 5 (6+P+S) S17 . . . A47 Assigned Normal 5 (7+P) S48
R47 7
IBM TotalStorage
146.0
Create Ranks
Format CKD or FB
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Rank can be assinged to Extent Pool with mkrank mkrank -Array A0 -stgtype fb -extpool P0 Extent Pool must exist
IBM TotalStorage
IBM TotalStorage
fb
Each Extent Pool is either CKD or FB RAID types not intermixed in same Extent Pool Extent Pools associated with a server Divided into equal sized extents
FB - 1 GB (binary)
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Address Groups
Addresses are grouped into Address Groups of 4096 addresses Each Address Group must be CKD or FB No real impact to FB implementations
dscli> lsaddressgrp Date/Time: 15 February 2006 10:22:15 GMT IBM DSCLI Version: 5.1.0.236 DS: IBM.2107-75AZRY1 ID stgtype basevolnum Vols LSSs confgvols ========================================= 0 fb 1 fb 2 fb 0000 1000 2000 4096 16 4096 16 4096 16 189 189 186
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Volume addressing
Format is hexadecimal XYZZ X = Address group (0-F) XY = LSS Number (00-FE) ZZ = Volume Number (00-FF)
Recommendations
Create a single LSS for each Extent Pool Define this during volume creation
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Create FB Volumes
Ensure OS support exists for larger LUNs Maximum number of volumes is 65280
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
# --------------------------------# 6+p+s small Ranks we want 36.5GB (decimal) vols = 34 extents # # # #LSS 00 mkfbvol -dev IBM.2107-75AZRY1 -extpool P0 -cap 34 -name P0_#h 0000-000D # #LSS 01 mkfbvol -dev IBM.2107-75AZRY1 -extpool P1 -cap 34 -name P1_#h 0100-0107 # #LSS 02 mkfbvol -dev IBM.2107-75AZRY1 -extpool P2 -cap 68 -name P2_#h 0200-0205 mkfbvol -dev IBM.2107-75AZRY1 -extpool P2 -cap 34 -name P2_#h 0206-0208 73GB (decimal) vols = 68 extents
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
========================================================================================================= == P0_0000 0000 Online Normal P0_0001 0001 Online Normal P0_0002 0002 Online Normal P0_0003 0003 Online Normal P0_0004 0004 Online Normal P0_0005 0005 Online Normal P0_0006 0006 Online Normal P0_0007 0007 Online Normal P0_0008 0008 Online Normal P0_0009 0009 Online Normal P0_000A 000A Online Normal P0_000B 000B Online Normal P0_000C 000C Online Normal P0_000D 000D Online Normal Normal Normal Normal Normal Normal Normal Normal Normal Normal Normal Normal Normal Normal Normal 2107-900 FB 512 P0 2107-900 FB 512 P0 2107-900 FB 512 P0 2107-900 FB 512 P0 2107-900 FB 512 P0 2107-900 FB 512 P0 2107-900 FB 512 P0 2107-900 FB 512 P0 2107-900 FB 512 P0 2107-900 FB 512 P0 2107-900 FB 512 P0 2107-900 FB 512 P0 2107-900 FB 512 P0 2107-900 FB 512 P0 34.0 34.0 34.0 34.0 34.0 34.0 34.0 34.0 34.0 34.0 34.0 34.0 34.0 34.0
71303168 71303168 71303168 71303168 71303168 71303168 71303168 71303168 71303168 71303168 71303168 71303168 71303168 71303168
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Provides LUN masking feature Volumes can be assigned to one or more volume groups Up to 8320 Volume Groups
Any Host Port (or Host Group) can only access one Volume Group Volume Groups can be accessed by multiple hosts
Providing they use same blocksize and addressing mode Data integrity reposnsibility of the host OS/software
Clustering support
Volumes can be asssigned to multiple Volume Groups Mutliple hosts can be in the same Volume Group
Recommendations
Create one Volume Group per Hoat Group Name Volume Groups with host names (up to 16 chars)
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
All Fixed Block-512 V20 SCSI All futsm01_vg V21 SCSI Mask
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Host Connection created by assigning FC HBA WWN Profiles used to identify host type Recommendations
Group all host connections from a single host into a Host Group
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
============================================================================ fprdb06_p1 0000 10000000C94F16F4 fprdb06_p2 0001 10000000C94F1663 fprdb06_p3 0002 10000000C94F1525 fprdb06_p4 0003 10000000C94F1655 fprdb07_p1 0004 10000000C94F1320 fprdb07_p2 0005 10000000C94F1498 fprdb07_p3 0006 10000000C94F1147 fprdb07_p4 0007 10000000C94F1170 IBM pSeries - AIX IBM pSeries - AIX IBM pSeries - AIX IBM pSeries - AIX IBM pSeries - AIX IBM pSeries - AIX IBM pSeries - AIX IBM pSeries - AIX 5 V5 5 V5 6 V5 5 V5 7 V6 7 V6 7 V6 7 V6 all all all all all all all all
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Copy Service
Duplication of Data
You can create back up data with little or no disruption to your application Allow you to create immediate copy of production data Mission Critical Application
Supports
Open Systems Z Series
Advantage
Disaster Recovery
Faster restore
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Mirror Functions
Metro Mirror Global Mirror
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.
Thank You
IBM TotalStorage
Copyright IBM Corporation 2001. All rights reserved. 2005.