HPE 3PAR StoreServ

Remote Copy Workshop

© 2015 Hewlett-Packard Development Company, L.P. – Peter Mattei

Agenda

• Remote Copy Overview

• Supported Topologies
• Transport Layers
• Restrictions & Limitations
• Remote Copy Setup – CLI
• Remote Copy – SSMC
• Failure Scenarios
• Peer Persistence - Overview
3PAR Remote Copy
Overview
3PAR Remote Copy (RC)
Possible Configs & Topologies 1:1 Configuration
RC sync, async streaming and periodic
Smart S P
• Simple and setup in minutes via GUI (SSMC) or CLI P S
Complete
• Native FC SAN and/or IP LAN-based replication 4:4 Configuration
• Reservation-less – No extra copies or infrastructure needed RC sync, async streaming and periodic
• Fully compatible with all 3PAR Thin & Data Reduction Technologies
• Remote Copy configurations from 1:1 up to 4:4
Support details are in the “3PAR Remote Copy Software User Guide”
• VMware vSphere Site Recovery Manager (SRM) certified
• VMware vSphere Metro Storage Cluster (vMSC) certified
Scalable
• Up to 5 replication links per node – up to 40 per array
• 1 RCIP link per node (1GbE or 10GbE)
• 1 to 4 FC and/or FCIP links per node

See a demo video here https://www.youtube.com/watch?v=7RVozzXOuRQ

Read the whitepaper Disaster-tolerant solutions with HPE 3PAR Remote Copy
3PAR Remote Copy
Remote Copy Directions

Unidirectional – A given storage system

pair is said to be under unidirectional
replication if replication occurs only in one
direction.

Bi-directional – each storage system in

the pair functions as both the primary and
secondary system, and each system
contains both primary and secondary
volume groups.
3PAR Remote Copy
Transport Layers

?
3PAR Remote Copy
Transport Layers
– Remote Copy over Fibre Channel (RCFC) – connected over a Fibre Channel SAN.
Up to 4 RCFC links are supported per node.
Only 1:1 relationship between RCFC links and targets is allowed
(you cannot use the same RCFC link to connect to more than one target).

– Remote Copy over Internet Protocol (RCIP) – connected to each other over an IP network and use
native Remote Copy IP ports to replicate data to each other.
1-to-2 relationship between RCIP links and targets is allowed
(the same pair of RCIP links can be used to connect to two source/target arrays).

– Remote Copy over FCIP (Fibre Channel over IP) – enables Fibre Channel frames to be sent over
an IP network. Storage systems use FC ports, the same ones they use for RCFC, and the SAN they
use is extended across an IP network by using FC-to-IP routing with external FCIP gateways.
3PAR Remote Copy link performance enhancements
Dramatically increased performance with 3.3.1 for RCFC and RCIP

3PAR 2-node configuration, 8kB, 100% random write 3PAR 4-node configuration, 8kB, 100% random write
Synchronous Remote Copy FC (4 x 16Gb FC links) Synchronous Remote Copy IP (4 x 10Gb RCIP links)

7 7
Response time (ms)

Response time (ms)

6 6

5 5

4 4

3 3

2 2

1 1

40 000 60 000 80 000 100 000 120 000 100 000 120 000 140 000 160 000 180 000
Host performance (IOPS) Host performance (IOPS)

3.2.2 3.2.2
3.3.1 with IRQ map disabled 3.3.1 with IRQ map disabled
3.3.1 with IRQ map enabled 3.3.1 with IRQ map & jumbo frames enabled
IRQ = Processor Interrupt Request IRQ = Processor Interrupt Request
Remote Copy Replication Types
3PAR Remote Copy Synchronous
Continuous operation and synchronization

Primary Secondary
Real-time Mirror 1 2
• Highest I/O currency
• Lock-step data consistency 4 3

Space Efficient
P S
• Thin provisioning aware

Targeted Use
1 : Host server writes I/O to primary write cache
• Campus-wide business continuity 2 : Primary array writes I/O to secondary write cache
• Guaranteed Consistency 3 : Remote array acknowledges the receipt of the I/O
• Enabled by Volume Groups 4 : Host I/O acknowledged to host
3PAR Remote Copy Asynchronous Periodic
Initial setup and synchronization

Primary Local Secondary

• Efficient even with high latency links Snapshots
• Local writes acknowledgement
• Bandwidth-friendly
2 3
• Just delta replication

• Space efficient P S
• Thin aware

• Guaranteed Consistency Step 1 : Secondary volume created

• Enabled by Volume Groups
Step 2 : Local snapshot created
• Based on snapshots
Step 3 : Initial synchronization
3PAR Remote Copy Asynchronous Periodic
Continuous operation and periodic resynchronization

Ideal for long-distance implementations Primary Local Secondary Remote

1 Snapshots Snapshot
Efficient even with high latency links
• Local write acknowledgement 2
Bandwidth-friendly 4 3
P 3 S
• Just delta replication
Space efficient
• Thin aware 1 : Host server writes I/O to primary write cache
2 : Host I/O acknowledged to host
Guaranteed Consistency
• Enabled by Volume Groups Manual or scheduled resync
• Based on snapshots 3 : Create local and remote snapshot
4 : Replicate snapshot delta
3PAR Remote Copy Asynchronous Streaming
Continuous operation with RPO close to 0

Efficient with higher latency links Primary Secondary

1 3
• Local write acknowledgement
• Maintains local server write IO performance and
latency 2 4
• Write IO data is queued in local cache and
replicated at max available link speed
• constantly places I/O data from the primary P S
onto the replication link for delivery at the
secondary array, applying the data in 100 ms
delta sets
1 : Host server writes I/O to primary write cache
Guaranteed Consistency 2 : Host I/O acknowledged to host
• Enabled by Volume Groups and Write IO tickets 3 : Primary array writes I/O to secondary write cache
(tags)
4 : Remote array acknowledges the receipt of the I/O
Periodic snapshots –
• every 1 hour default (can be changed
setrcopygroup snap_freq)
3PAR Synchronous Long Distance Configuration
A specialized 1:2 Disaster Recovery (DR) solution
• Combines the ability to maintain concurrent metro-distance synchronous Remote Copies with RTO=0
AND continental distance asynchronous Remote Copies for Disaster Tolerance (DT)

• Asynchronous streaming mode targets are not Primary Synchronous Long Distance
supported in SLD 1:2 Configuration

Metropolitan distance
P
• Remote copy links for SLD can be all IP, all FC,
or a mixture of each (one leg IP and the other
leg FC). Sync RC

Async Periodic RC
S1 S2
Standby

Secondary Tertiary

Find 4 SLD demo videos here https://www.youtube.com/playlist?list=PL9UfCHCZQuNDmU8WRXT_RU7yG_sL-eweV

General
Requirements & Limitations
3PAR Remote Copy
General Requirements & Restrictions
• You must have Remote Copy licenses for all storage systems participating in RC replication
 StoreServ * License Cap
3PAR 8000 and 20000 Software Details 8200
8400, 8450
8440
48
168
320

Legacy licensing before February 2017 20450

20850
168
320
20800 480
20840 640

Replication SW Suite *
• Virtual Copy (VC)
Recovery Manager Central Suite
• Remote Copy (RC)
• Peer Persistence (PP) • vSphere
• Cluster Extension Windows (CLX) Policy Server • MS SQL
• Policy Manager Software
• Oracle
Data Optimization SW Suite * • SAP HANA
• Dynamic Optimization Data Encryption • 3PAR File Persona
• Adaptive Optimization
• Peer Motion
• Priority Optimization
File Persona SW Suite Application SW Suite for MS Exchange

Security SW Suite *
• Virtual Domains Smart SAN for 3PAR Application SW Suite for MS Hyper-V
• Virtual Lock

• System Reporter 3PAR Operating System SW Suite * • Full Copy

• 3PARInfo • Thin Provisioning
• Online Import license (1 year) • Rapid Provisioning • Adaptive Flash Cache • Thin Copy Reclamation
• System Tuner • Autonomic Groups • Persistent Cache • Thin Persistence
• Host Explorer • Autonomic Replication Groups • Persistent Ports • Thin Conversion
• Multi Path IO SW • Autonomic Rebalance • Management Console • Thin Deduplication for SSD
• VSS Provider • LDAP Support • Web Services API • 3PAR OS Administration Tools
• Scheduler • Access Guard • SMI-S • CLI client
• Host Personas • Real Time Performance Monitor • SNMP
3PAR 8000, 9000 and 20000 Software Details
All-inclusive licensing model
All-inclusive Multi-System Software - optional Policy Server – optional
Policy Manager Software
Remote Copy (RC) Federation
Peer Motion Peer Persistence
Cluster Extension Windows (CLX) Data Encryption – optional
Requires encrypted drives

All-inclusive Single-System Software - included with every system

Dynamic Optimization Recovery Manager Central Suite
Adaptive Optimization File Persona • vSphere
Priority Optimization • MS SQL
Smart SAN for 3PAR • Oracle
Virtual Domains
Application Suite for MS Hyper-V • SAP HANA
Virtual Lock • 3PAR File Persona
Application Suite for MS Exchange
Virtual Copy
System Reporter
3PARInfo Rapid Provisioning Adaptive Flash Cache Full Copy
Online Import license (1 year) Autonomic Groups Persistent Cache Thin Provisioning
System Tuner Autonomic Replication Groups Persistent Ports Thin Copy Reclamation
Host Explorer Autonomic Rebalance Management Console Thin Persistence
Multi Path IO SW LDAP Support Web Services API Thin Conversion
VSS Provider Access Guard SMI-S Thin Deduplication for SSD
Scheduler Host Personas Real Time Performance Monitor 3PAR OS Administration Tools
3PAR Remote Copy
General Requirements & Restrictions
• You must have Remote Copy licenses for all storage systems participating in RC replication

• Remote Copy requires the use of a minimum of two 3PAR StoreServ systems

• Remote Copy does not support self-mirroring configurations. It cannot use a storage system to
replicate its own primary volumes to itself.

• Remote Copy does not support multi-hop configurations. It cannot replicate a primary volume
group to a secondary system and then replicate the volume group again from the secondary
system to a third storage system.

• The physical connections between all storage systems used with Remote Copy must be made
through an IP-capable network or an FC SAN network
3PAR Remote Copy
General Requirements & Restrictions

• To maintain availability, you must use more than one link to connect storage systems

• Remote Copy uses all the available links that are configured for the same replication mode
(synchronous or asynchronous periodic) to transmit data in parallel

• HPE recommends that synchronous mode replication be used to limit the potential for data loss,
whenever the additional write latency induced by the network, plus the write latency induced by the
target array, will not exceed the maximum write latency tolerable by the application whose data is
being replicated
3PAR Remote Copy
Limitations 1/2
• Max Size of Mirrored Volume - 64 TB

• Max Mirrored Volumes per HPE 3PAR StoreServ Storage for synchronous RC:
800 - 2 Nodes 3000 - 4 Nodes 4000 - 8 Nodes

• Max Mirrored Volumes per HPE 3PAR StoreServ Storage for async periodic RC:
2400 - 2 Nodes 6000 - 4 Nodes and more

• Max Mirrored Volumes per HPE 3PAR StoreServ Storage for async streaming RC:
512 - All Node configuration
3PAR Remote Copy
Limitations 2/2
• Max Consistency Groups

6000 Periodic or 3000 Sync (for 4 nodes and greater configuration)

2400 Periodic and 800 Sync (for 2 nodes)
512 Asynchronous Streaming

• Max Mirrored Volumes per consistency group: 300

• Max Mirrored Volumes per 3PAR for async streaming: 512 - All Node configuration
Transport Layers
Requirements & Limitations
3PAR Remote Copy
Transport Layers RCFC
• Each storage system should have a pair of HBAs installed

• HBAs in each storage system connect those systems through FC SAN (dual fabric)

• HBA can be shared by the host and RCFC for the following HPE 3PAR StoreServ Storage systems:
20000, 10000, 8000, 7000
• Each pair of RCFC ports that support an RCFC link must exist in an exclusive zone (pWWN members)

• RCFC requires a dedicated node pair for each secondary system

• Only one-to-one relationship between RCFC links and targets is allowed. i.e. you cannot use the same
RCFC link to connect to more than one target.

• HPE recommends that interrupt coalescing be disabled on the RCFC ports

controlport intcoal disabled -f 2:0:1
3PAR Remote Copy
Transport Layers RCIP 1/2
• Each storage system in the remote copy configuration must have at least two nodes

• For RCIP, each network adapter (NIC) interface must use a unique IP address

• The network adapter (NIC) interface and the management Ethernet port of the 3PAR controller node
must be on different IP subnets

• A pair of IP ports on the node pairs in an array may have a RC relationship with up to 2 other arrays
(pair of RCIP ports on an array may send data to up to two different arrays, and may be the remote
copy target for those same two)
3PAR Remote Copy
Transport Layers RCIP 2/2
• RCIP configurations can use up to 8 links between systems. Up to 8 nodes can each have one
network adapter (NIC) port contributing links to an RCIP remote copy pair.

• The network used by RCIP does not have to be dedicated to remote copy, but there should be a
guaranteed network bandwidth (minimum of 500 KB/s) between any pair of arrays. Guaranteed
bandwidth on the network is especially important when replicating synchronously over RCIP

• Traffic on the firewall must be enabled for RCIP ports: TCP 5785 & 5001
• Supported Array Ports over IP for async streaming RC: 10GbE only (1GbE not supported)!
3PAR Remote Copy
Transport Layers Limitation

• Transport layer protocols cannot be mixed for a given mode of replication between a pair
of Remote Copy volume groups.

E.g. when doing replication between a pair of volume group (between a pair of 3PAR
StoreServ systems), the transport layer can be either RCIP, or RCFC but not mixed.
3PAR Remote Copy
Maximum latencies
Max Supported Latency Round Trip Time (RTT )
Remote Copy Type
3PAR OS ≤ 3.2.1 3PAR OS 3.2.2 3PAR OS 3.3.1
Synchronous RCFC 2.6ms ≤ MU1: 5ms - ≥ MU2: 10ms
10ms
Synchronous RCIP 2.6ms 5ms 10ms
Synchronous RC FCIP NA 10ms 10ms
Asynchronous Periodic RCFC 2.6ms 5ms 5ms
Asynchronous Periodic RCIP 150ms 150ms 150ms
Asynchronous Periodic RC FCIP 120ms 120ms 120ms
Asynchronous Streaming RCFC NA 5ms 10ms
Asynchronous Streaming RCIP NA NA 30ms
Asynchronous Streaming FCIP NA 10ms 30ms

Optical Fibre networks typically have a delay of ~5 us/km (0.005ms/km)

Thus 2.6ms RTT will allow Fibre link distances of up to 260km, 10ms RTT gives you ~1000km
2.6ms : 0.005ms/km = 520km forth and back  260km site to site
10.0ms : 0.005ms/km = 2000km forth and back  1000km site to site Go back to
SW and
Features
3PAR Remote Copy Asynchronous Streaming
Requirements & Limitations 1/2

• Support for firmware 3.3.1 only (patches for 3.2.2 to enable Async Streaming were for Proof Of Concept ONLY).

• Support only for these Models 8k, 9k, 20k series

• Support for RCFC, RCIP and RCFC with FCIP -

RCIP is only supported using the 10GbE interfaces with Asynchronous Streaming
Async streaming is not supported over a 1GbE RCIP link.

• Source and target arrays must contain the same number of nodes.

• Every node must have at least 1 RC link

• 1:1 configurations only. No support for 1:2, 2:1, M x N

3PAR Remote Copy Asynchronous Streaming
Requirements & Limitations 2/2

• The IP networks should be adequately provisioned to support the required workload.

• Maximum latency on the network used for any link will not exceed the specified values in the Feature
availability matrix for the particular link over 24 hours using the packet load size of the network
(10 ms, 30 ms)

• Network latency jitter (the difference between the highest and lowest network latency, measured
constantly over 24 hours with a packet load the size of the network's MTU) must not exceed
2ms or 20% of the end-to-end average network latency

• Packet loss ratio does not exceed 0.0012% average measured over 24 hours
Transport Layers
Link Timeout & Failures
3PAR Remote Copy
Link failure

• When a link between storage systems fails, the system issues an alert on each
system as soon as it detects the link failure.
• As long as 1 remote copy link between the two systems remains active, Remote Copy
remains in normal operation and sends all data through the remaining links.
• The system might experience a slight reduction in throughput (bandwidth), but a
single link failure in a multiple-link remote copy pair does not incur errors under normal
operating conditions other than the link failure itself.
3PAR Remote Copy
Link & Target Timeout..

RC type Link Heartbeat Timeout Target Timeout

Synchronous 5s 10 s

Asynchronous Streaming 5s 5s

Asynchronous Periodic 60 s 200 s

RC declares a secondary
RC declares an
system down after
unresponsive link down
all links have gone down
3PAR Remote Copy
Target Failure

• When all links between remote copy targets in a remote copy relationship fail, neither
side can communicate. Each side reports that its remote copy target has failed.
Following a remote copy target failure, the following actions occur:
 Both systems declare the other system to be down.
 Both systems generate alerts regarding the other system’s failure.

The system handles complete link failure differently depending on whether the links are
used for synchronous, asynchronous streaming, or asynchronous periodic mode volume
groups
Remote Copy Volume Groups
3PAR Remote Copy Volume Groups
Assured Data integrity
New Target
Create new
Single Volume Volume created
Source
autonomically
Volume
• All writes to the secondary volume are completed
in the same order as they were written on the
primary volume

Autonomic Remote Copy Volume Group

• Volumes grouped together to maintain write
RC Volume Group RC Volume Group
ordering across sets of volumes
• Useful for databases or other applications with Primary 3PAR Storage Secondary 3PAR
Storage
dependencies
• Secondary groups and volumes are autonomically
created and credentials inherited
3PAR Remote Copy Volume Groups
Naming

• When you create and name a volume group on the primary system, HPE 3PAR
Remote Copy automatically creates and names the associated secondary volume
group on the secondary system.

• The secondary volume group is named:

<primary_group_name>.r<primary_system_system_ID>

• For example, if the primary volume group is Group1 and the primary system ID is 96,
HPE 3PAR Remote Copy names the secondary volume group Group1.r96
3PAR Remote Copy
Volumes Synchronization States...
New — Remote copy for the primary virtual volume has not started.

Syncing — Remote copy is currently synchronizing the secondary virtual volume with the primary

Synced — The primary and secondary virtual volumes are currently in sync (for asynchronous periodic mode volumes,
the output displays the last synchronization time)

NotSynced — The primary and secondary virtual volumes are not in synchronization

Stopped — The volume group is stopped. The primary and secondary virtual volumes were in synchronization the last
time the group was in the Started state, but might be out of synchronization now

Stale — The secondary virtual volume has a valid point-in-time copy of the primary volume; however, the last attempt at
synchronization failed

Failsafe — In the event of a network failure that prompts a failover, any primary volumes will be placed into a failsafe
state to prevent data corruption and inconsistency between primary and secondary volumes.

NOTE: The Failsafe state is relevant only to volumes that have matching WWNs. If the secondary volumes were created using the
admitrcopyvv –createvv option, it will affect these groups.
Remote Copy Group operations
3PAR Remote Copy
RC Group States...
Macierz DC RC Macierz DRC
Operation Group status
Status Read/Write direction Status Read/Write

Normal Operation started & sync Primary R/W ---> Secondary -

Primary-
Failover stopped Primary R/W - R/W
Reverse

Secondary- Primary-
Recover started & sync - <--- R/W
Reverse Reverse

Restore started & sync Primary R/W ---> Secondary -

Revert Failover
started & sync Primary R/W ---> Secondary -
Undo Failover

Go back to
SW and
Features
Remote Copy States - Normal

Read/Write
LUN
Normal

A A
Primary Group started Secondary
& sync
3PAR Array A 3PAR Array B

Site A Site B
Remote Copy States - Failover

Read/Write
LUN
Failover

Read/Write
LUN
A A
Primary Group Primary-Reverse
stopped
3PAR Array A 3PAR Array B

Site A Site B
Remote Copy States - Failover

Recover

Read/Write
LUN
A A
Secondary-Rev Group started Primary-Reverse
& sync
3PAR Array A 3PAR Array B

Site A Site B
Remote Copy States - Normal

Read/Write
LUN
Restore

A A
Primary Group started Secondary
& sync
3PAR Array A 3PAR Array B

Site A Site B
Remote Copy –
Track synchronization
3PAR Remote Copy tracks synchronization details
Synchronous volume groups

• For synchronous mode volume groups, the startrcopygroup

command prompts resynchronization each time it is used.

• To verify that a resynchronization took place, check the tasks by

using the showtask command

• The syncrcopy command manually synchronizes remote copy

volume groups
3PAR Remote Copy tracks synchronization details
Asynchronous Streaming volume groups

• For asynchronous streaming mode volume groups, the

startrcopygroup command prompts resynchronization each time it
is used.

• To verify that a resynchronization took place, check the tasks by

using the showtask command

• The syncrcopy command manually synchronizes remote copy

volume groups
3PAR Remote Copy tracks synchronization details
Asynchronous Periodic volume groups

• startrcopygroup command starts synchronization only the first time the command is
issued for a volume group. Subsequent instances of the command do not initiate
resynchronization of asynchronous periodic mode volume groups.
• HPE 3PAR Remote Copy does not create a new task for each resynchronization of
an asynchronous periodic volume group.
• Instead, HPE 3PAR Remote Copy keeps the initial synchronization task active for as
long as the group is in the Started state, and updates task details when
resynchronizations occur
Remote Copy - CLI
CLI Overview – SSH connection

1. Z poziomu systemu klienta:

# ssh-key-gen –b 2048 –t rsa
(tworzona jest lokalnie para kluczy id_rsa i id_rsa.pub)
2. Zalogować się do 3Par użytkownikiem user1 dla którego ma być utworzona autoryzacja
3. Z poziomu 3par:
3par cli% setsshkey
Please enter the SSH public key below. When finished, press enter twice. The key
is usually long. It's better to copy it from inside an editor and paste it here.
(Please make sure there is no extra blanks.)

ssh–rsa AF5afPdciUTJ0PYzB6msRxFrCuDSqDwPshqWS5tGCFSoSZdE= user1’s pubic key

SSH public key successfully set!

4. Z poziomu systemu klienta:

3par-local# ssh user1@IP_address
3PAR Info v 1.6
• Red Hat Enterprise Linux 5.7, 5.8, 6.1, 6.2, 6.3, 6.4, 6.5, 6.6, 7.0 (x64 and x86_32),
• SUSE LINUX Enterprise Server 10.0 SP4 (x64 and x86_32),
• SUSE LINUX Enterprise Server 11.0 SP1 and SP2, 12.0 SP1 (x64 and x86_32),
• HP-UX 11i v1 (PA-RISC), HP-UX 11i v2, 11i v3 (PA-RISC and IA64),
• Windows 2008 SP1 and SP2 (x64 and X86_32),Win 2008 R2, 2008 R2 SP1, Win 2012, 2012 R2,Win Server 2016,
• ESX 4.0, ESX 4.1 and ESXi 5.1, ESXi 5.5, ESXi 6.0, ESXi 6.5
• AIX 6.1, 7.1
• Solaris 10,11 (SPARC and x86),
• Oracle Linux 6.8 (x64), Oracle Linux 7.3, Oracle Linux UEK 5, UEK 6, UEK 7
CLI Overview
Rodzaje komend
CLI Overview – Remote Copy
RC commands descriptions
startrcopy Enable remote copy on a storage system

stoprcopy Stop remote copy functionality for all started remote copy volume groups

creatercopytarget Specify the targets within a remote copy pair and create additional links

admitrcopylink Create a link to a secondary system

creatercopygroup Create a remote copy volume group (can be run by users with edit privileges)

admitrcopytarget Add a secondary system to HPE 3PAR StoreServ Storage system

admitrcopyvv Add an existing virtual volume to an existing remote copy volume group

setrcopygroup Set a remote copy volume group’s policies, data transfer direction, resynchronization period, and mode

setrcopytarget Set a remote copy target’s name and policies, and the target link’s throughput definition
CLI Overview – Remote Copy
RC commands descriptions
showport View remote copy port information

showrcopy View remote copy configuration details

showrctransport View the status and information about end-to-end remote copy transport

checkrclink Perform a connectivity, latency, and throughput test between two connected systems

controlport Set remote copy interface information

dismissrcopylink Remove a sending link that was created with the admitrcopylink command

dismissrcopytarget Remove a secondary system from HPE 3PAR StoreServ Storage system

dismissrcopyvv Remove a virtual volume from a remote copy volume group

removercopygroup Delete a remote copy volume group (can be run by users with Edit privileges)

removercopytarget Remove a target definition from a remote copy system and remove all links affiliated with that target definition
CLI Overview – Remote Copy
RC commands descriptions
startrcopygroup Enable remote copy for a remote copy volume group

stoprcopygroup Stop remote copy functionality for a specific remote copy volume group

syncrcopy Synchronize remote copy volume groups

growvv Increase the size of a virtual volume

histrcvv View a histogram of remote copy service times

srstatrcopy View historical performance data reports for remote copy links

srstatrcvv View historical performance data reports for remote copy volumes

statport View I/O statistics for remote copy ports

statrcopy View statistics for remote copy volume groups

statrcvv View statistics for remote copy volumes

Remote Copy – CLI
Transport Layer
Remote Copy - CLI
Transport Layer RCIP 3par-local
1. Start Remote Copy
3par-local# startrcopy
2. Konfiguracja portów RCIP:
3par-local# controlport rcip addr -f 192.168.20.11 255.255.255.0 0:3:1
3par-local# controlport rcip addr -f 192.168.20.12 255.255.255.0 1:3:1
3par-local# controlport rcip gw -f 192.168.20.1 0:3:1
3par-local# controlport rcip gw -f 192.168.20.1 1:3:1
3par-local# controlport rcip speed auto full 0:3:1
3par-local# controlport rcip speed auto full 1:3:1
3par-local# controlport rcip state up -f 0:3:1
3par-local# controlport rcip state up -f 1:3:1
3. Weryfikacja portów RCIP:
3par-local# showport –rcip
3par-local# showrcopy
4. Te same czynności na drugiej macierzy
Remote Copy - CLI
Transport Layer RCIP 3par-remote
1. Start Remote Copy
3par-remote# startrcopy
2. Konfiguracja portów RCIP:
3par-remote# controlport rcip addr -f 192.168.20.13 255.255.255.0 0:3:1
3par-remote# controlport rcip addr -f 192.168.20.14 255.255.255.0 1:3:1
3par-remote# controlport rcip gw -f 192.168.20.1 0:3:1
3par-remote# controlport rcip gw -f 192.168.20.1 1:3:1
3par-remote# controlport rcip speed auto full 0:3:1
3par-remote# controlport rcip speed auto full 1:3:1
3par-remote# controlport rcip state up -f 0:3:1
3par-remote# controlport rcip state up -f 1:3:1
3. Weryfikacja portów RCIP:
3par-remote# showport –rcip
3par-remote# showrcopy
Remote Copy - CLI
Transport Layer RCIP
1. Test komunikacji IP między macierzami i poszcz. portami
3par-local# controlport rcip ping 192.168.20.13 0:3:1
3par-local# controlport rcip ping 192.168.20.14 1:3:1
2. Test przepustowości między macierzami i poszcz. portami (max MTU)
3par-remote# checkrclink startserver 0:3:1 192.168.20.11
3par-remote# checkrclink startserver 1:3:1 192.168.20.12

3par-local# checkrclink startclient 0:3:1 192.168.20.13 -time 60

3par-local# checkrclink startclient 1:3:1 192.168.20.14 -time 60
....
3par-remote# checkrclink stopserver 0:3:1
3par-remote# checkrclink stopserver 1:3:1

3par-local# checkrclink stopclient 0:3:1

3par-local# checkrclink stopclient 1:3:1
Remote Copy - CLI
Transport Layer RCIP - MTU

1. Set MTU
3par-local# controlport rcip mtu 9000 0:3:1
3par-local# controlport rcip mtu 9000 1:3:1

3par-remote# controlport rcip mtu 9000 0:3:1

3par-remote# controlport rcip mtu 9000 0:3:1

3par-local# controlport rcip ping -s 9000 -pf 192.168.20.13 0:3:1

3par-local# controlport rcip ping -s 9000 -pf 192.168.20.14 1:3:1
Remote Copy - CLI
Logical Relation between 3Par Arrays - FCIP

1. Relacja logiczna między macierzami i poszcz. portami

3par-local# creatercopytarget 3par-remote IP 0:3:1:192.168.20.13
3par-local# admitrcopylink 3par-remote 1:3:1:192.168.20.14

2. Te same czynności z drugiej macierzy

3par-remote# creatercopytarget 3par-local IP 0:3:1:192.168.20.11
3par-remote# admitrcopylink 3par-local 1:3:1:192.168.20.12
3. showrcopy
Remote Copy - CLI
Removing logical Relation between 3Par Arrays - FCIP

1. Usunięcie linków logicznych między poszcz. portami

3par-local# dismissrcopylink 3par-remote 0:3:1:192.168.20.13
3par-local# dismissrcopylink 3par-remote 1:3:1:192.168.20.14
2. Analogiczne czynności z drugiej macierzy
3par-remote# dismissrcopylink 3par-local 0:3:1:192.168.20.11
3par-remote# dismissrcopylink 3par-local 1:3:1:192.168.20.12
3. Usunięcie relacji logicznej między macierzami
3par-local# removercopytarget 3par-remote
3par-remote# removercopytarget 3par-local
4. Zatrzymanie serwisu RC
3par-local# stoprcopy
3par-remote# stoprcopy
Remote Copy - CLI
Transport Layer RCFC
1. Start Remote Copy
3par-local# startrcopy
2. Konfiguracja portów RCFC (dla wszystkich portów):
3par-local# controlport offline -f 0:2:3
3par-local# controlport config rcfc -ct point -f 0:2:3
3par-local# controlport rcfc init -f 0:2:3
3. Sprawdzenie stanu portów dla RCFC
3par-local# showrctransport –rcfc
4. Relacja logiczna RC między macierzami i poszcz. portami
3par-local# creatercopytarget 3par-remote FC 2FF70002AC01BC1D
0:2:3:20230002ac01bc1d
#admitrcopylink 3par-remote 1:2:4:21240002ac01bc1d
5. showrcopy
6. Te same czynności z drugiej macierzy
Remote Copy - CLI
Removing logical Relation between 3Par Arrays - RCFC

1. Usunięcie linków logicznych między poszcz. portami

3par-local# controlport offline -f 0:2:3
3par-local# controlport offline -f 1:2:4
3par-local# controlport rcfc delete -f 0:2:3
3par-local# controlport rcfc delete -f 1:2:4
2. Analogiczne czynności z drugiej macierzy
3par-remote# controlport rcfc delete -f 0:2:3
3par-remote# controlport rcfc delete -f 1:2:4
3. Usunięcie relacji logicznej między macierzami
3par-local# removercopytarget 3par-remote
3par-remote# removercopytarget 3par-local
4. Zatrzymanie serwisu RC
3par-local# stoprcopy
3par-remote# stoprcopy
Remote Copy - CLI
Removing a remote copy setup completely

stoprcopy -stopgroups -clear

The stoprcopy -stopgroups -clear command:

• Stops remote copy on all volume groups
• Deletes all remote copy configuration data
• Releases all resources used by remote copy
• Is not reversible
Remote Copy – CLI
RC Volume Groups
Remote Copy - CLI
RC Group – normal VV admit
1. Utworzenie Remote Copy Group
3par-local# creatercopygroup –domain domain1 Group1 3par-
remote:sync/async/periodic

2. Dodanie wolumenów do RC
3par-local# admitrcopyvv -pat vv1.* Group1 3par-remote:@vvname@

3. Uruchomienie RC grupy
3par-local# startrcopygroup Group1
3par-local# showrcopy

4. Zatrzymanie RC grupy
3par-local# stoprcopygroup Group1

For remote copy to operate, you must name the domain correctly. When volumes are
admitted to a remote copy group for which a virtual domain has been defined, the volumes
on both sides must share the same domain name
Remote Copy - CLI
RC Groups – auto VV creation
1. Utworzenie Remote Copy Grupy
3par-local# creatercopygroup –domain domain1 Group1 3par-remote:async

2. Konfiguracja CPG dla RC grupy do automatycznego tworzenia wolumenów

3par-local# setrcopygroup -usr_cpg CPG_SSD 3par-remote:CPG_SSD
-snp_cpg CPG_SSD 3par-remote:CPG_SSD Group1

3. Dodanie wolumenów do grupy i automatycznie ich utworzenie na macierzy remote

3par-local# admitrcopyvv -nosync -createvv -pat vv1.*
Group1 3par-remote:@vvname@

4. Uruchomienie RC grupy
3par-local# startrcopygroup Group1
3par-local# showrcopy
5. Stop RC grupy
3par-local# stoprcopygroup Group1
Remote Copy - CLI
RC Groups – time interval
1. Utworzenie interwału czasowego dla grupy async periodic
3par-local# setrcopygroup period <period_value>{s|m|h|d} <target_name>
<group_name>

365d > period_value > 5m

period_value =0 (no automatic synchr)

1. Utworzenie interwału czasowego dla grupy async streaming

3par-local# setrcopygroup period <period_value>{s|m|h|d}
<target_name> <group_name>

period_value - used to define the order in which groups will be stopped and restarted
automatically (min 30s)
Remote Copy - CLI
RC Groups – changing mode
1. Zatrzymanie grupy
3par-local# stoprcopygroup <group_name>

2. Zmiana modu dla grupy

3par-local# setrcopygroup mode <mode_value> <target_name> <group_name>

<mode_value> New remote copy mode (sync, async, or periodic)

3. Start grupy
3par-local# startrcopygroup <group_name>
Remote Copy - CLI
RC Groups – manual Sync
1. Zatrzymanie grupy
3par-local# stoprcopygroup <group_name>

2. Zmiana modu dla grupy

3par-local# syncrcopygroup <group_name>
3par-local# syncrcopygroup -ovrd <group_name>
3. Start grupy
3par-local# startrcopygroup <group_name>
Remote Copy – CLI
Volume Operations
CLI– Remote Copy
Grow VV
The size of a remote copy volume can be increased by using the growvv command. The growth of a
volume is coordinated across primary and secondary targets.
A coordinated grow can be started from either the primary or secondary target. If the volume to be
grown is in a remote copy group, the group must be stopped before the grow operation is permitted.

Volumes on remote targets are grown to the intended size of the local volume. If a target cannot be
contacted or remote copy is not started, only the local volume will be grown.

1. Zatrzymanie Remote Copy Grupy

3par-local# stoprcopygroup Group1

2. Powiększenie wolumenu
3par-local# growvv VV_name <size>[g|G|t|T]

3. Uruchomienie RC grupy
3par-local# startrcopygroup Group1
3par-local# showrcopy
Remote Copy - CLI
RC Group –VV admit

1. Dodanie wolumenów do RC
3par-local# admitrcopyvv -pat vv1.* Group1 3par-remote:@vvname@

2. Konfiguracja CPG dla RC grupy do automatycznego tworzenia wolumenów

3par-local# setrcopygroup -usr_cpg CPG_SSD 3par-remote:CPG_SSD
-snp_cpg CPG_SSD 3par-remote:CPG_SSD Group1

3. Dodanie wolumenów do grupy i automatycznie ich utworzenie na macierzy remote

3par-local# admitrcopyvv -nosync -createvv -pat vv1.*
Group1 3par-remote:@vvname@
CLI– Remote Copy
Remove VV from Group
1. Zatrzymanie Remote Copy Grupy
3par-local# stoprcopygroup Group1
3par-local# stoprcopygroup –pat Group1*

2. Usunięcie wolumenów z grupy

3par-local# dismissrcopyvv –pat VV_name* Group1

3. Usunięcie wolumenów z grupy i skasowanie wolumenu z macierzy remote

3par-local# dismissrcopyvv –removevv VV_name Group1

4. Uruchomienie RC grupy
3par-local# startrcopygroup Group1
3par-local# showrcopy
Remote Copy
Group Operations and States
3PAR Remote Copy
Target Failure for synchronous groups

• Remote copy stops the remote copy groups

• Remote Copy continues to allow writes to the primary volume group after snapshots of the primary
volumes have been taken
• Remote copy creates snapshots of all primary virtual volumes that completed the initial
synchronization process.
• While replication is stopped, remote copy does not create snapshots for secondary virtual volumes.
• The system records failed I/O replication separately because remote copy creates the snapshots of
the primary virtual volumes on the primary system after I/O is written to the primary virtual volumes,
but before remote copy can write the I/O to the secondary virtual volumes.
• These recorded failed I/O operations are completed when the replication links return and before
resynchronization of the volumes occurs
3PAR Remote Copy
Target Failure for asynchronous streaming groups
• Target failure and asynchronous streaming volume groups
 Remote Copy continues to allow writes to the primary volume group
 Remote Copy stops replication from the primary to the secondary system
 While replication is stopped, remote copy does not create coordinated snapshots for the primary and
secondary virtual volumes

• After recover at least one RC link the following occurs:

 If the auto_recover policy is set, the volume groups automatically restart.

 If the no_auto_recover policy is set (the default), manually restart groups by the startrcopygroup command on
the primary system
 Promotes the secondary to the last coordinated snapshot point
 Determines the difference between the base volume and the coordinated snapshot taken before replication
stopped.
 Copies any new writes from the primary volume groups to the secondary volume groups .
3PAR Remote Copy
Target Failure for periodic groups

• Remote Copy stops the remote copy groups

• Remote Copy does not create any new snapshots
• If a resynchronization was in progress when the failure occurred, HPE 3PAR Remote Copy
automatically promotes the pre-existing snapshots on the secondary volume group — but only for
those volumes that did not complete resynchronization.
3PAR Remote Copy
Failure of the system

• Failure of a primary system

When a primary system fails and you need access to the data on the secondary virtual volumes, you
must switch the role of the volume groups containing those volumes (from secondary to primary):

stoprcopygroup <group_name>

setrcopygroup failover <group_name>

setrcopygroup recover <group_name>

startrcopygroup <group_name>

• Failure of a secondary system

Remote Copy handles the failure like a failure of all links
Remote Copy – Snapshots
Snapshots in synchronous mode
3PAR Remote Copy snapshots
Synchronous mode

In synchronous mode, remote copy creates snapshots only when:

• An error occurs.
• Disaster recovery is in process.
• A volume group is manually stopped.

If a failure occurs during the initial (full) synchronization of a virtual volume:

• Remote copy does not create a snapshot of that volume.
• After you recover the system and restart remote copy, remote copy performs another initial
(full) synchronization of the volume
3PAR Remote Copy snapshots
Synchronous mode - Snapshots taken before or during disaster recovery

If the secondary system fails

If the secondary system fails, the primary system:
• Stops the replication of all volume groups
• Takes snapshots of all volumes that were completely synchronized

If the primary system fails

When remote copy restarts after a primary system failure (either manually using the startrcopygroup
command or via the auto_recover policy), Remote Copy first looks for a valid resynchronization
snapshot for each virtual volume.

• If a resynchronization snapshot exists, HPE 3PAR Remote Copy:

1. Creates snapshots of all the secondary virtual volumes that were previously synced
2. Resynchronizes the secondary virtual volume by sending only the differences between
the resynchronization snapshot and the current data in the primary virtual volume.

• If a resynchronization snapshot does not exist, Remote Copy performs a full synchronization.
Remote Copy – Snapshots
Snapshots in asynchronous streaming
3PAR Remote Copy snapshots
Asynchronous Streaming mode

Snapshots taken during initial synchronization

During the initial synchronization, HPE 3PAR Remote Copy performs the following:

1. On an initial synchronization of an Async Streaming group, snapshots (coordinated

snapshots) are created at the point when full syncs have completed on all volumes in the group.

2. After synchronization is completed, deletes the sync snapshots created on the primary and
secondary system.

3. Saves the newer snapshot (resync) for future resynchronization.

Snapshots taken periodically

Periodically resynchronization points are created on both the primary and secondary sides, HPE 3PAR
Remote Copy takes new snapshots of the secondary and primary volumes. (Coordinated snapshots).
3PAR Remote Copy snapshots
Asynchronous Streaming mode – resync process

During resynchronization

• The primary uses its coordinated snapshots to resynchronize with the secondary arrays volumes.

• The primary and secondary volume state is set to syncing.

• As each volume completes its resync, its volume state changes to synced.

• When the resynchronization completes on all volumes in the group, remote copy takes new
coordinated snapshots before deleting the now old coordinated snapshots on the primary and the
backup systems.

• After resynchronization, the base volume on the secondary storage system matches the new
snapshots on the primary and secondary storage systems. The new snapshot on the primary system
and secondary system can now be used in the next resynchronization operation.
3PAR Remote Copy snapshots
Asynchronous Streaming mode – resync failure

Snapshots and resynchronization failure in asynchronous streaming mode

If, during resynchronization in asynchronous streaming mode, the primary system fails, remote copy
behaves in the following manner for the volumes in the remote copy target volume group:

• For all volumes in the remote copy volume group that completed resynchronizing before the failure,
HPE 3PAR Remote Copy automatically promotes the pre-synchronization snapshot for all of these
volumes.

• For all volumes in the remote copy volume group that were in the process of resynchronizing at the
time of the failure, but that did not complete resynchronizing, HPE 3PAR Remote Copy automatically
promotes the pre-synchronization snapshot for all of these volumes.
Remote Copy – Snapshots
Snapshots in asynchronous periodic
3PAR Remote Copy snapshots
Asynchronous periodic mode initial sync

Snapshots taken during initial synchronization

During the initial synchronization, HPE 3PAR Remote Copy:

1. Takes a snapshot of the primary volume.

2. Sends that snapshot over to initialize the secondary volume
3PAR Remote Copy snapshots
Asynchronous periodic mode – resync process

Snapshots taken during resynchronization:

• At the next scheduled resynchronization, or whenever you issue the syncrcopy, Remote Copy:

• Takes new snapshots of the secondary and primary volumes.

• Sends the differences between the old primary snapshot and the new primary snapshot over to
resynchronize the secondary base volume.

• The secondary volume state is syncing.

• When the resynchronization completes, remote copy deletes the old snapshots on the primary and
the backup systems.

• After resynchronization, the base volume on the secondary storage system matches the new
snapshot on the primary storage system. The new snapshot on the primary system can now be
used in the next resynchronization operation.
3PAR Remote Copy snapshots
Asynchronous periodic mode – resync failure

Snapshots and resynchronization failure in asynchronous periodic mode

If, during resynchronization in asynchronous periodic mode, the primary system fails, remote copy
behaves in the following manner for the volumes in the remote copy target volume group:

• For all volumes in the remote copy volume group that completed resynchronizing before the failure,
HPE 3PAR Remote Copy takes no action on these volumes and retains all pre-synchronization
snapshots for these volumes.

• For all volumes in the remote copy volume group that were in the process of resynchronizing at the
time of the failure, but that did not complete resynchronizing, HPE 3PAR Remote Copy automatically
promotes the pre-synchronization snapshot for all of these volumes.
3PAR Remote Copy snapshots
Asynchronous periodic mode – resync failure

Some of the volumes in the volume group successfully synchronized before the failure, and some
volumes did not finish resynchronizing before the failure, then the volumes that had not completed
resynchronization will be promoted to the last recovery point

To make the volumes I/O consistent again, one of two actions must be performed:

• The remote copy volume group must be restarted after the failure has been recovered from, at which
time a new resynchronization will occur, resulting in all the volumes becoming I/O consistent with one
another at the new resynchronization point in time.

• The remote copy volume group must be used for recovery, by means of a failover, at which time all of
the volumes whose snapshots were not promoted following the failure (the ones that completed
synchronization) will have their pre-synchronization snapshots promoted, and all the volumes in the
volume group will then revert to their I/O consistent pre-synchronization state.
Disaster Tolerant Solutions
- Cluster Extension
- Metrocluster
- Peer Persistence

Go back to
SW and
Features
Cluster Extension for Windows
Clustering solution protecting against server and storage failure
What does it provide?
• Manual or automated site-failover for Server and Storage
resources
• Transparent Hyper-V Live Migration between site
Supported environments:
• Microsoft Windows Server 2003, 2008, 2012
• HPE StoreEasy (Windows Storage Server)
• Max supported distances
• Remote Copy sync supported up to 5ms RTT (~500km)
• Up to MS Cluster heartbeat max of 20ms RTT
• 1:1 and SLD configuration
• Sync or async Remote Copy
Requirements:
• 3PAR Disk Arrays
• 3PAR Remote Copy (RC)
• Microsoft Cluster
• HPE Cluster Extension (CLX)
• Max 20ms cluster IP network RTT
Licensing options:
• Option 1- per Cluster Node
1 LTU per Windows Cluster Node (i.e. 4 LTUs for the configuration to the left)
• Option 2 - per 3PAR Array
1 LTU per 3PAR array (i.e. 2 LTUs for the configuration to the left)
Also see the HPE CLX references
Serviceguard Metrocluster for HP-UX and Linux
End-to-end clustering solution to protect against server and storage failure
What does it provide?
• Manual or automated site-failover for Server and Storage
resources
Supported environments:
• HP-UX 11i v2 & v3 with Serviceguard
• RHEL 5 and 6 with HPE Serviceguard 11.20.10
• SLES 11with HPE Serviceguard 11.20.10
• Max supported distances
• Up to Remote Copy sync max 5ms RTT (~500km)
• Up to Remote Copy async max 150ms RTT
Requirements:
• 3PAR Disk Arrays
• 3PAR Remote Copy
• HPE Serviceguard & HPE Metrocluster
Licensing for Linux:
• 1 LTU SGLX per CPU core and 1 LTU MCLX per CPU core
Licensing Options for HP-UX:
• Option 1: Per CPU socket for SGUX and MCUX
• Option 2: Per Cluster with up to 16 nodes for SGUX and MCUX

Find documentation here http://h20565.www2.hp.com/portal/site/hpsc/public/psi/home/?sp4ts.oid=4161971#manuals

3PAR Peer Persistence
Never lose access to your data volumes P Primary RC Volume
S Secondary RC Volume
What does it provide? VM Clustered Application
• High Availability across data centers active paths standby paths

How does it work?

• Automatic or manual transparent LUN swap based on
3PAR Remote Copy and OS MPIO
• Primary RC Volume presented with active paths
• Secondary RC Volume presented with passive paths
• Automated LUN swap arbitrated by a Quorum Witness
(on a 3rd site)
P
Requirements:
P
• FC, iSCSI or FCoE cross-site Server SAN
• Two synchronous Remote Copy links (RCFC or RCIP)
• Max replication link latency of 10ms RTT (~1000km)
• 3PAR Remote Copy and Peer Persistence Licenses S
Supported environments: S
• All OS stated on the next slide
• Oracle RAC on Windows, HP-UX, RHEL, SUSE

Also see the Peer Persistence product page

Peer Persistence for VMware
Never lose access to your volumes VMware vSphere 5.x Metro Storage Cluster (single
subnet)
• Each host is connected to each array on
both sites via redundant fabrics (FC or
iSCSI or FCoE)
• Synchronous copy of the volume is kept on
the partner array/site (RCFC or RCIP)
Fabric A
• Each Volume is exported in R/W mode with
same WWN from both arrays on both sites Fabric B
• Volume paths for a given volume are “Active” up to 5ms RTT latency
only on the Array where the “Primary” copy
of the volume resides. Other volume paths
are marked “Standby” Vol B Vol B prim
B sec B
• Both arrays can host active and passive Vol A Vol A sec
volumes A prim A
• Quorum Witness on 3rd site acts as arbitrator
QW
in case of failures
vSphere
Active Path Vol A 3PAR Array 3PAR Array
Active Path (Vol B) Site A Site C Site B
Passiv0e/Standby
Path
Peer Persistence for VMware – ALUA Path View

VMware vSphere 5.x Metro Storage Cluster (single

subnet)

2’

vCenter Path Management View

Vol A prim Vol A sec

2 2’

QW
vSphere
3PAR Array 3PAR Array
Site A Site C Site B

3PAR MC Remote Copy View

Peer Persistence Overview

– Peer Persistence is a high availability (HA) storage OS Min. 3PAR OS

Host 3PAR Host
Connectivity Persona
configuration between two sites with the ability to
transparently redirect host I/O, from the primary to VMware vSphere 5.0, 5.1 3.1.2 MU2 FC 11

the secondary storage system 3.1.3 FC

VMware vSphere 5.5, 6.x 1) 11
3.2.1 FC, iSCSI, FCoE
• ”switchover” Windows 2008 R2, 2012 R2 3.2.1
is a manual process allowing to facilitate service 2016 3.2.2 MU3 FC, iSCSI, FCoE 15
optimization and storage system maintenance (Cluster and standalone)
activities within a high-availability data storage RHEL 6.x, 3.2.2
solution FC, iSCSI, FCoE 2
7.x 3.2.2 MU1
• “failover” HP-UX v3 3.2.2 FC, FCoE 13
is an automatic process initiated by the secondary Citrix Xen Server 6.0, 6.1, 6.2 1
3.2.2.MU2 FC
array redirecting host I/O from the failed primary to 6.5 2
the secondary array. SLES 10, 11, 12 3.2.2.MU1 FC, iSCSI 2
A functional 3PAR Quorum Witness is required for
Solaris 10, 11 3.2.2.MU1 FC 2
an automatic transparent failover to occur
– The Peer volumes must be synchronously RC Link 1) ESX VVOL not supported
3PAR OS
replicated over RCFC, RCIP or RCIP and will have Topology
the same WWNs RCFC ≥ 3.1.2 MU2
RCIP ≥ 3.1.3
HP 3PAR Peer Persistence – Automatic Transparent Failover
Catastrophic Failover Window timing

• For 3PAR OS 3.2.2 MU2 and RC links declared

down and RC group
beyond the timeout value for stopped
RC link
Peer Persistence to detect a failure Last
detected successful
catastrophic failure can be set Last RC QW check
Link fails
from 10 to 30 seconds 10 second default Additional wait time (0 – 20 seconds)
wait time
• The default value is 10 Time in
seconds
seconds 0 5 11 15 >31 fail_timeout (<= 35)

• An automatic transparent
failover will occur if both the Primary Array’s connection to the Quorum Witness QW failure here will
must fail and be detected within this window for not be detected in
RC links and the QW access Failsafe on the Primary and an ATF to occur time for an ATF

get lost within the selected 10

to 30 seconds window
3PAR Peer Persistence versus VMware SRM

Functionality 3PAR Peer Persistence 3PAR integrated with SRM

Dual-site active-active + optional 3rd standby
Concept Dual-site bi-directional active-standby datacenters
datacenters
Use case High Availability + Disaster Recovery Disaster Recovery
Transparent non-disruptive failover of active 3PAR Manually triggered storage failover and restart of selected
Disaster on primary site volumes. If vSphere Metro Storage Cluster VMs in DR site. Both sites can be primary site for one
(vMSC) deployed VMs can fail over automatically environment and DR site for another one.
Allows balancing load over the two datacenters – Provides extensive failover test capabilities on the remote site
Additional use
active LUNs can be swapped transparently on copies of production data

vMotion / Storage vMotion YES => One cluster over two datacenters Yes => Requires SRM ≥ 6.1 AND Peer Persistence

Granularity 3PAR Remote Copy Group 3PAR Remote Copy Group

Volume switch / failover Manually or automated with Quorum on a 3rd site Manually

3PAR Multi-System SW Suite

3PAR Single + Multi-System SW Suite + VMware SRM
Requirements Host access across both sites
RC link requirements see SPOCK
RC link requirements see SPOCK
Peer Persistence for VMware vSphere
Requirements
• This non-disruptive failover configuration is currently supported with VMware vSphere 5.0, 5.1 and
5.5 and 6.x hosts
• All volumes exported on all primary and secondary arrays must share the same volume WWN
• The sites must be set up in a 1-to-1 Remote Copy FC or IP configuration in synchronous mode
• For uniform VMware vSphere Metro Storage Cluster (vMSC) implementations and vMotion support
– A common SAN across both sites is required
– All associated hosts are connected to both the primary and secondary arrays
– The ESX hosts that these volumes are exported to must be configured with host persona 11 (VMware), which
supports host capability Asymmetric Logical Unit Access (ALUA)
• If Remote Copy secondary volumes are being exported to ESX hosts that are not also connected to
the primary volume, host persona 6 (Generic-Legacy) should be configured. Host persona 6 does not
support non-disruptive failover
• To avoid split-brain situations a Quorum Witness instance in a third location is required
– Canned ESX VM with Red Hat Enterprise Linux
– Can be deployed on vSphere 5.0 and above (any version including free Essentials)
Peer Persistence – 3PAR Storage & VMware vSphere
Easy Setup
– Steps to configure the QW VM :
– Install the canned HP QW RedHat VM thinly provisioned and on a
vSphere server located at a preferably 3rd site
– Setup the QW Network
– Define the QW Hostname
– Set the QW Password
– From the 3PAR management console or the CLI configure the
communication between the 3PAR StoreServs and the QW

– Now you can setup Peer Persistence

– Zone ESX host to the 3PAR arrays (SAN)
– create Hosts with persona 11, VVs, LUNs on primary 3PAR
– create Datastores in vSphere
– create Hosts with persona 11, VVs on secondary 3PAR
– Setup and sync Remote Copy sync
– Add Remote Copy Group to Enabled Automatic Failover Remote Copy
Groups in Peer Persistence
– create LUNs on secondary 3PAR
– Test your setup
HP 3PAR Peer Persistence –
Arbitrator

Quorum Witness (QW)

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HP 3PAR Peer Persistence - Arbitrator
What it is
– Quorum Witness (QW)
Site 3
– Application running on Linux on a Virtual Machine
in a Hypervisor (vSphere 5.x or Hyper-V 2012 R2)
– Comes as a package including a Linux VM and the Quorum Witness
QW software
Linux
– Minimal configuration required
– Network setup, Hostname, Password Hypervisor
– Takes less than 5 minutes
– The QW only requires IP connectivity to both 3PARs
– Install the QW in a third independent site and do not place the QW
datastore on one of the protected
HP 3PAR StoreServ systems!
HP 3PAR Peer Persistence - Arbitrator
Quorum Witness Requirements*
– VMware ESXi 5.x or Windows Hyper-V 2012 R2
– 2 GB memory
– 20 GB disk space
– Network interface with access to a network with connectivity to both HP 3PAR StoreServ
Systems
– Static IP Address assignment
– Maximum Latency/RTT (Round Trip Time)
– Maximum round trip latency: 150 ms
– Connection timeout: 250 ms
– Response timeout: 3s

*Source: Remote Copy User Guide

Peer Persistence
Failover Scenarios

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
3PAR Peer Persistence failures handling
Remote Copy (RC) Links failure
Active Path Vol A
Cluster Active Path (Vol B)
Passive/Standby
LUN A.123
Path

LUN B.456

LUN B.456

LUN A.123
Fabric B
• Remote Copy Links failure
does not cause any
Fabric A Automatic transparent failover
if communication to QW
remains operational
• Replication of I/O across RC
B B links stops due to failure
RC Links
Synchronous Remote Copy
Secondary Failure Primary • Host I/Os continue to go to
with RTT of up to 5ms RTT
A latency A their primary volumes
Primary Secondary • Manual Switchover not
QW
3PAR Array A 3PAR Array B possible

Site A Site C Site B

Quorum Witness failure
Active Path Vol A
Cluster Active Path (Vol B)
Passive/Standby
LUN A.123
Path

LUN B.456

LUN B.456

LUN A.123
Fabric B

Fabric A • QW failure does not cause

any failover as long as the
RC link remains operational
• Manual failover can still be
B Synchronous Remote Copy
B issued; However, automatic
Secondary Primary failover cannot happen
with RTT of up to 5ms RTT
A latency A without QW
Primary Secondary
QW Failure
QW
3PAR Array A 3PAR Array B

Site A Site C Site B

Communication to Quorum Witness (QW) failure
Active Path Vol A
Cluster Active Path (Vol B)
Passive/Standby
LUN A.123
Path

LUN B.456

LUN B.456

LUN A.123
Fabric B

• If either site A or Site B lose

connectivity to QW, an
Fabric A
automatic failover does not
occur
• Host I/O and replication of I/O
across RC links continues
B Synchronous Remote Copy
B normally
Secondary Primary
with RTT of up to 5ms RTT • Manual switchover can still be
A latency A issued
Primary Secondary
Comm. QW Comm.
3PAR Array A Failure Failure 3PAR Array B

Site A Site C Site B

Both sites to QW and RC communication failure
Active Path Vol A
Cluster Active Path (Vol B)
Passive/Standby
LUN A.123
Path

LUN B.456

LUN B.456

LUN A.123
Fabric B • Both arrays will be isolated as
they can neither
communicate with each other
Fabric A over the RC links nor with
Quorum Witness (QW).
• Remote Copy (RC) groups
that are primary will go into
failsafe mode and stop
B RC Links
Synchronous Remote Copy
B serving I/O, resulting in host
Secondary Failure Primary
with RTT of up to 5ms RTT I/O failure
A latency A • No failover actions will be
Primary Secondary
Comm. QW Comm. performed, replication of I/O
3PAR Array A Failure Failure 3PAR Array B across RC links will stop

Site A Site C Site B

Server loss – Automated Server Recovery
Active Path Vol A
Server Cluster Active Path (Vol B)
failure Passive/Standby
LUN A.123
Path

LUN B.456

LUN B.456

LUN A.123
Fabric B • VMware vSphere
VMware HA automatically
restarts VMs on other servers
in the cluster (including on
Fabric A
remote servers)
• Microsoft Windows Server
MS Failover Cluster
automatically restarts all
B Synchronous Remote Copy
B affected services and/or
Secondary Primary
with RTT of up to 5ms RTT Hyper-V VMs on other servers
A latency A in the cluster (including on
Primary Secondary remote servers)
QW
3PAR Array A 3PAR Array B • No intervention required on
storage systems
Site A Site C Site B
Storage component failure – Storage still accessible
Active Path Vol A
Cluster Active Path (Vol B)
Passive/Standby
LUN A.123
Path

LUN B.456

LUN B.456

LUN A.123
Fabric B

Fabric A • Single component failure

(controller node, FC port,
drive, FC switch etc.) within
the 3PAR array or the SAN
Single
Component B B are handled transparently
Failure Synchronous Remote Copy and do not cause a failover
Secondary Primary
with RTT of up to 5ms RTT
A latency A
Primary Secondary
QW
3PAR Array A 3PAR Array B

Site A Site C Site B

Complete Storage System failure on either side
Active Path Vol A
Cluster Active Path (Vol B)
Passive/Standby
LUN A.123
Path

LUN B.456

LUN B.456

LUN A.123
• Upon loss of an entire storage
Fabric B
system, the volumes that
were ‘primary’ on the failed
system have to be failed over.

Fabric A • The surviving system will take

over the ‘primary’ role based
on arbitration with the QW.
• The ‘standby’ paths to these
Storage volumes become ‘active’
System B Synchronous Remote Copy
B paths. Previously, ‘active’
Failure Primary
with RTT of up to 5ms RTT paths to these volumes from
A latency A the failed array go into ‘failed’
New Primary status.
QW
3PAR Array A 3PAR Array B • I/O to the host continues to be
served and VMs remain
Site A Site C Site B
online.
Complete Site Failure
Active Path Vol A
Cluster Active Path (Vol B)
Passive/Standby
LUN A.123
Path

LUN B.456

LUN B.456

LUN A.123
• The volumes that were
Fabric B ‘primary’ on the failed site have
to be failed over.
• The surviving system will take
Fabric A over the ‘primary’ role based
on arbitration with the QW.
• The ‘standby’ paths to these
volumes become ‘active’
B B paths. Previously, ‘active’
Synchronous Remote Copy
with RTT of up to 5ms RTT
Primary paths go into ‘failed’ status.
A latency A • VMs on Site 2 remain online.
New Primary
QW • VMs previously running on Site
3PAR Array A 3PAR Array B 1 can be restarted on Site 2

Site A Site C Site B

Network Partition Handling – VMware vSphere
Active Path Vol A
Active Path (Vol B)
APP APP APP APP APP APP APP APP APP APP APP APP APP APP APP APP APP APP
Passive/Standby
OS OS OS OS OS OS OS OS OS
Cluster OS OS OS OS OS OS OS OS OS

Path
APP
OS

vSphere VM
LUN A.123

LUN B.456

LUN B.456

LUN A.123
• If a SAN partition occurs
Fabric B (Host and RC IO), volumes
from both arrays remain
available to local hosts.
Fabric A Volume paths to remote site
hosts are lost. Passive
volume paths remain passive.
VMs that reside on local
storage array stay online
B Synchronous Remote Copy
B
Secondary Primary VMs that reside on remote
with RTT of up to 5ms RTT
latency storage array shutdown or
A A enter zombie state
Primary Secondary
3PAR Array A
QW
3PAR Array B • Depending on the Remote
Copy transport used
Site A Site C Site B replication might still be
running or stopped.
Network Partition Handling – MS Windows Server
Redirected IO Redirected IO Cluster Network
Active Path Vol A
Active Path (Vol B)
Passive/Standby
Cluster APP
Path
OS

Hyper-V VM
LUN A.123

LUN B.456

LUN B.456

LUN A.123
Clustered Application

Fabric B
• If a SAN partition occurs
volumes from both arrays
remain available to local
Fabric A hosts. Volume paths to
remote site hosts get
redirected through the cluster
network using SMB2 and stay
B B online until the problem has
Synchronous Remote Copy been solved.
Secondar Primary
with RTT of up to 5ms RTT
y latency • Depending on the Remote
A A
Copy transport used
Primary Secondar
QW
y B replication might still be
3PAR Array A 3PAR Array
running or stopped.
Site A Site C Site B
The End

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.