Guide To Integrate IBM System Storage With SAP HANA TDI V2.9
Guide To Integrate IBM System Storage With SAP HANA TDI V2.9
Guide To Integrate IBM System Storage With SAP HANA TDI V2.9
Table of Contents
1
Preface .................................................................................................................... 3
Requirements .......................................................................................................... 4
10
11
12
13
Setup of /hana/shared/global.ini...................................................................... 13
14
15
SAP HANA TDI High Availability setup with IBM System Storage ................... 13
16
SAP HANA TDI Disaster Recovery setup with IBM System Storage ................ 14
17
18
19
Resources .............................................................................................................. 15
20
Trademarks ............................................................................................................ 16
21
Disclaimers............................................................................................................. 17
Page - 2
1 Preface
This paper is intended as an architecture and configuration guide to setup the
IBM System Storage for the SAP HANA tailored datacenter integration (SAP
HANA TDI) within a SAN environment.
This document has been written for IT technical specialists and architects, with
advanced skill levels on SUSE Linux Enterprise Server or RedHat LINUX and IBM
System Storage, with a focus on architecting and setting up an IBM System Storage environment for SAP HANA TDI. The SAP HANA TDI allows the SAP customer to
use external storage to attach to the SAP HANA server.
This document provides the necessary information to select, verify and connect
IBM System Storage to the SAP HANA server through a SAN (fiber channel) attached storage network. The recommendations in this guideline apply to both
single node and scale out configurations, as well as Intel or IBM POWER based
SAP HANA systems.
For a list of all storage systems certified for SAP HANA production please visit:
http://global.sap.com/community/ebook/2014-09-02-hana-hardware/enEN/enterprise-storage.html
IBM Storwize family, model V7000, V5030, V5020, V5010, V5000, V3700,
and IBM Spectrum Virtualize (SVC)
IBM FlashSystem family, model 820, 840, 900, V840, V9000
A9000, A9000R
IBM XIV Gen 3 Storage System
IBM DS8800, DS8870, DS8880
IBM Spectrum Scale ESS GS2,4 & GL2,4,6
Page - 3
Verify all server & storage components and firmware requirements given
by SAP are met
Install SLES or Red Hat operating system on all SAP HANA nodes
If required: setup NFS services for the SAP HANA shared directory
Setup SAN storage systems
Create SAN zone for SAP HANA environment
On the storage systems map the storage volumes (LUNs)
to all target SAP HANA nodes
Setup of multipath
On each LINUX host create the file systems for SAP HANA data, log,
shared
Setup of /hana/shared/global.ini
Install and configure the SAP HANA software
Optional: HA, DR, and Backup considerations
4 Requirements
Before starting to deploy a SAP HANA TDI, ensure that the following key items have
are in place:
Page - 4
Basic setup of the HANA nodes (NFS clients), add this line to /etc/fstab
nfs-server:/hana/shared
Page - 5
/hana/shared
nfs
rw,soft,intr,rsize=8192,wsize=8192 0 0
7.1
To achieve the required storage performance according to SAP KPIs (key performance indicators) for SAP HANA TDI, one of the following configurations listed
should be chosen nevertheless all IBM systems listed can be used as stand-alone
system as well, used for DATA and LOG.
IBM Spectrum Virtualized is referred as SVC.
Storage virtualization
SVC
IBM FlashSystem
SVC
IBM Storwize
SVC
IBM FlashSystem
IBM FlashSystem
SVC
IBM DS8000
Because IBM SVC, and for smaller configurations IBM Storwize, provisions volumes
(LUNs) as a single volume type and single storage system, it will ease the administration of storage pool tiers with Flash, SSD, and HDD types; and optional tasks
like setup of storage high availability, FlashCopy base backup, storage based mirroring, or storage maintenance. If the IBM storage system will be direct attached
the SAP HANA TDI nodes, then the following table specifies the recommended
disk types.
IBM Storwize
IBM DS8000
IBM XIV
Module
Module
IBM FlashSystem
Flash Card
Flash Card
Page - 6
7.2
# HANA
Storwize V3700
10/8/10/12
10 / 14
FlashSystem V9000
40
FlashSystem A9000R
40
12
DS8870
16
DS8880 (4/6/8)
16 / 32 / 40
XIV Gen 3
15
10 / 40
The minimum number of HDD for DATA is 24: three RAID 5 as 7+1, spares not included. The HDD or SDD disk capacity can be chosen arbitrary to match HANA
capacity requirements.
IBM Storwize Use distributed RAID 5, this provides higher performance.
Page - 7
7.3
The SAP HANA Data and Log have different I/O characteristics reflected by the
different storage systems configurations as described earlier. This has to be applied by pooling the appropriate storage systems accordingly.
For all storage systems:
Create 4 volumes for HANA DATA and 1 volume for HANA LOG,
for each node
These volumes would also server as MDISKs if SVC will be used
Page - 8
7.4
The following diagrams show the relationship between Linux file system, LINUX LV,
VG, and PV, and the Virtual Volumes (VD, VDISK) and Managed Disks (MD, MDISK,
RAIDs/Arrays) of a Storage Systems.
We recommend to create just one DATA and just one LOG pool, if feasible, and
just one non-Prod (HDD / shared) pool, if needed. This ensures best resource utilization (max striping) if more than one HANA system needs to be deployed.
Page - 9
7.5
Every IBM storage system can be shared between SAP HANA (production) and
any other SAP or non-SAP (production) workload.
To do so proper sizing of the entire storage infrastructure is required, including configuration of RAID controller, number of HDD, SSD, storage of Flash Modules, number and type of HBA and FC ports/links.
IBM and IBM Business Partner are pleased to provide support for this sizing task,
including the usage of the IBM tools HANAmagic.
7.6
The IBM storage systems provide the option to compression data on VDISK / Volume level.
Because the data of the HANA database are just integers, these data cannot
be compressed further. SAP calls this Data Compression in the Column Store.
We strongly urge you to not use IBM storage compression (RtC) for SAP HANA, else
this would lead to performance degradation.
8 Setup of multipath
Setup multipath according SLES or Red Hat documentation, in addition:
/etc/multipath.conf:
defaults {
polling_interval
failback
no_path_retry
user_friendly_names
30
immediate
5
no
devices {
device {
vendor
product
failback
no_path_retry
rr_min_io
dev_loss_tmo
path_checker
path_grouping_policy
path_selector
prio
rr_weight
}
}
Page - 10
"IBM"
"2145"
immediate
5
10
120
tur
group_by_prio
"round-robin 0"
const
uniform
Use the xfs file system type with a 4KB block size for the DATA and LOG file
systems
To use LVM 2 use this schema:
vgcreate lvreate mkfs.xfs
Use useful names for volume groups and logical volumes do not use
the hyphen (minus) sign - as part of the name, because the LINUX mapper uses this sign as separator between VG and LV name.
Create Volume Groups and Logical Volumes for DATA and LOG, for each
HANA mount point (as listed in global.ini) create one Volume Group with
exactly one Logical Volume.
It has turned out that these settings are most beneficial for SAP HANA workload
in combination with SVC:
1. Use 4 PV for Volume Group DATA
2. Create a volume group with an extent size of 1MB and an alignment size
of 1MB
vgcreate vgcreate -s 1M --dataalignment 1M VG_ANA_DATA_1 /dev/mapper/WWID_1 /dev/mapper/WWID_2 ...
Page - 11
If HANA scale-out or HA node fail-over is used, it is not necessary to add the file
system to /etc/fstab, mounting will be done by SAP HANA.
11 VMware VVols
The same setup applies like for LINUX native installation as described in chapter
10; the only deviation is that VMware (ESX server) controls the SAN multipath,
not the LINUX OS. The volumes are typically accessible under the path:
/dev/sd... instead of /dev/mapper/...
Please have a look in the IBM Redbook how to setup VMware VVols with IBM storage: http://www.redbooks.ibm.com/abstracts/sg248328.html?Open
Page - 12
13 Setup of /hana/shared/global.ini
Enter the Logical Volume names as shown in directory /dev/mapper into
/hana/shared/global.ini :
[communication]
listeninterface = .global
[persistence]
basepath_datavolumes = /hana/data/ANA
basepath_logvolumes = /hana/log/ANA
[storage]
ha_provider
partition_*_*__prtype
partition_1_data__lvmname
partition_1_log__lvmname
partition_2_data__lvmname
partition_2_log__lvmname
=
=
=
=
=
=
hdb_ha.fcClientLVM
5
VG_ANA_DATA_1-LV_ANA_DATA_1
VG_ANA_LOG_1-LV_ANA_LOG_1
VG_ANA_DATA_2-LV_ANA_DATA_2
VG_ANA_LOG_2-LV_ANA_LOG_2
The [storage] section is only needed, if you want to use the node fail-over functionality provided by SAP HANA software and for HANA scale-out. If you use the
system replicating capability leave this section empty.
15 SAP HANA TDI High Availability setup with IBM System Storage
The SAP HANA system gains high availability through an N+1 concept one or
more server act as standby SAP HANA node with dedicated storage for each
active node. If one active SAP HANA node fails, the SAP HANA cluster software
Page - 13
16 SAP HANA TDI Disaster Recovery setup with IBM System Storage
The disaster recovery capabilities of SAP HANA are documented on
www.saphana.com:
If IBM Metro Mirror (synchronous) is used for storage replication for SAP HANA TDI,
all DATA and all LOG volumes of all nodes from one HANA system must be included in one storage Consistency Group.
That implies (when using native attached storage) that both Data and Log
must come from the same storage unit.
18 Change History
V1
V1.2
V1.3
V1.4
V1.5
V1.6
V1.7
Page - 14
Initial version
Update of chapter 7.6 to clarify files system setup
Included new Storwize V5000
Specified NFS section to be used only for /hana/shared/
Minor updates add some clarifications
Add setup of multipath and global.ini
Rework of document
19 Resources
For any product or documentation provided by SAP please contact SAP.
For any product or documentation provided by SUSE please contact SUSE.
For any product or documentation provided by IBM regarding SAP send an email
to [email protected].
IBM documentations:
This document is ID WP102347 on www.ibm.com
SAP HANA on IBM Power Systems and IBM System Storage Guides
http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102502
IBM Storwize product manuals:
http://www-01.ibm.com/support/docview.wss?uid=ssg1S7003318
Installation, configuration and usage of Tivoli Storage Manager for Enterprise Resource Planning V6.4 Data Protection for SAP HANA
http://www-01.ibm.com/support/docview.wss?uid=swg21608240
SAP documentation:
Overview - SAP HANA tailored data center integration
http://www.saphana.com/docs/DOC-3633
Page - 15
Access to sapjam and documentation will be provided by SAP the listed links
only work after login: login first, then access the links.
SAP HANA Fiber Channel Storage Connector Admin Guide
SAP HANA Storage Requirements (Storage Whiltepaper.pdf)
https://jam4.sapjam.com/wiki/show/60549
Highly Available NFS Storage with DRBD and Pacemaker with SUSE Linux Enterprise High Availability Extension 11
https://www.suse.com/documentation/sle_ha/singlehtml/book_sleha_techguides/book_sleha_techguides.html
20 Trademarks
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or
both. These and other IBM trademarked terms are marked on their first occurrence in this information with the appropriate symbol ( or ), indicating US registered or common law trademarks owned by IBM at the time this information was
published. Such trademarks may also be registered or common law trademarks
in other countries. A current list of IBM trademarks is available on the Web at
http://www.ibm.com/legal/copytrade.shtml
The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both:
AIX, BladeCenter, DB2, Global Business Services, Global Technology Services, GPFS, IBM SmartCloud, IBM, Intelligent Cluster, Passport Advantage, POWER, PureFlex, RackSwitch, Redbooks, Redpaper,
Redbooks (logo) , System Storage, System x, System z, Tivoli, z/OS.
Page - 16
21 Disclaimers
This information was developed for products and services offered in Germany.
IBM may not offer the products, services, or features discussed in this document in
other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be
used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service.
IBM may have patents or pending patent applications covering subject matter
described in this document. The furnishing of this document does not grant you
any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive, Armonk, NY 10504-1785
U.S.A.
The following paragraph does not apply to the United Kingdom or any other
country where such provisions are inconsistent with local law: INTERNATIONAL
BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT
Page - 17
Page - 18