Chapter 12 ■ Remote Replication 1

Chapter 12
Remote Replication
Modes of Remote Replication
 The two basic modes of remote replication are synchronous and asynchro-
nous. In synchronous remote replication, writes must be committed to the
source and remote replica (or target), prior to acknowledging “write
complete” to the host (see Figure 12-1).

 Additional writes on the source cannot occur until each preceding write has
been completed and acknowledged. This ensures that data is identical on the
source and replica at all times. Further, writes are transmitted to the remote
site exactly in the order in which they are received at the source.

Therefore, write ordering is maintained.

 If a source-site failure occurs, synchronous remote replication

provides zero or near-zero recovery- point objective (RPO).

1 The host writes data to the source.

2 Data from the source is replicated to the target at a remote site.

3 The target acknowledges back to the source.

4 The source acknowledges write complete to the host.

Figure 12-1: Synchronous replication

2 Section III ■ Backup, Archive, and Replication

 However, application response time is increased with synchronous remote

replication because writes must be committed on both the source and target
before sending the “write complete” acknowledgment to the host.

 The degree of impact on response time depends primarily on the distance between
sites, band- width, and quality of service (QOS) of the network connectivity
infrastructure.

 Figure 12-2 represents the network bandwidth requirement for synchronous

replication. If the bandwidth provided for synchronous remote replication is
less than the maximum write workload, there will be times during the day when
the response time might be excessively elongated, causing applications to time
out.

 In asynchronous remote replication, a write is committed to the source and

immediately acknowledged to the host. In this mode, data is buffered at the
source and transmitted to the remote site later (see Figure 12-3).

 Figure 12-4 shows the network bandwidth requirement for asynchronous

replication. In this case, the required bandwidth can be provisioned equal to or
greater than the average write workload. Data can be buffered during times when
the bandwidth is not enough and moved later to the remote site. Therefore,
sufficient buffer capacity should be provisioned.
 Chapter 12 ■ Remote Replication 3

1 The host writes data to the source.

2 The write is immediately acknowledged to the host.

3 Data is transmitted to the target at a remote site later.

4 The target acknowledges back to the source.

Figure 12-3: Asynchronous replication

 In asynchronous replication, data at the remote site will be behind the source by at
least the size of the buffer.

 Therefore, asynchronous remote replication provides a finite (nonzero) RPO

disaster recovery solution. RPO depends on the size of the buffer, the available
network bandwidth, and the write workloadto the source.
4 Section III ■ Backup, Archive, and Replication

 Asynchronous replication implementation can take advantage of locality of

reference (repeated writes to the same location).

 If the same location is written multiple times in the buffer prior to transmission
to the remote site, only the final version of the data is transmitted. This feature
conserves link bandwidth.

 In both synchronous and asynchronous modes of replication, only writes to the

source are replicated; reads are still served from the source.

Remote Replication Technologies

Remote replication of data can be handled by the hosts or storage arrays. Other options
include specialized network-based appliances to replicate data over theLAN or SAN.

12.2.1. Host-Based Remote Replication

Host-based remote replication uses the host resources to perform and manage the
replication operation. There are two basic approaches to host-based remote replication:
Logical volume manager (LVM) based replication and database replication via log
shipping.

LVM-Based Remote Replication

 LVM-based remote replication is performed and managed at the volume group level.
Writes to the source volumes are transmitted to the remote host by the LVM.

 The LVM on the remote host receives the writes and commits them to the remote
volume group.

 Prior to the start of replication, identical volume groups, logical volumes, and file
systems are created at the source and target sites. Initial synchro- nization of data
between the source and replica is performed.

 After the initial synchronization, production work can be started on the source
volumes and replication of data can be performed over an existing standard IP
network (see Figure 12-5).
 Chapter 12 ■ Remote Replication 5

IP

LV1 LV2 LV1 LV2

LV3 LV4 LV3 LV4

Production Host Remote Host

Figure 12-5: LVM-based remote replication

 LVM-based remote replication supports both synchronous and asynchronous modes of

replication. If a failure occurs at the source site, applications can be restarted on the
remote host, using the data on the remote replicas.
 LVM-based remote replication is independent of the storage arrays and there- fore
supports replication between heterogeneous storage arrays.

Host-Based Log Shipping

 Database replication via log shipping is a host-based replication technology
supported by most databases.

 Transactions to the source database are captured in logs, which are

periodically transmitted by the source host to the remote host (see Figure 12-
6).

 The remote host receives the logs and applies them to the remote database.
6 Section III ■ Backup, Archive, and Replication

Log Log

IP

Data Data

Source Database Standby Database

Production Host Remote Host

Figure 12-6: Host-based log shipping

 All DBMSs switch log files at preconfigured time intervals or when a log file is full.
The current log file is closed at the time of log switching, and a new logfile is opened.

 When a log switch occurs, the closed log file is transmitted by the source host to the
remote host. The remote host receives the log and updates the standby database.

 This process ensures that the standby database is consistent up to the last committed
log. RPO at the remote site is finite and depends on the size of the log and the
frequency of log switching.

 Available network bandwidth, latency, rate of updates to the source database, and the
frequency of log switching should be considered when determining the optimal size of
the log file.

 Similar to LVM-based remote replication, the existing standard IP network can be used for
replicating log files. Host-based log shipping requires low network bandwidth because
it transmits only the log files at regular intervals.
 Chapter 12 ■ Remote Replication 7

Storage Array-Based Remote Replication

 In storage array-based remote replication, the array-operating environment and
resources perform and manage data replication.

 This relieves the burden on the host CPUs, which can be better used for applications
running on the host.

 A source and its replica device reside on different storage arrays. Data can be
transmitted from the source storage array to the target storage array over a shared or a
dedicated network.

 Replication between arrays may be performed in synchronous, asynchronous, or disk-

buffered modes.

Synchronous Replication Mode

 In array-based synchronous remote replication, writes must be committed to the source
and the target prior to acknowledging “write complete” to the produc- tion host.

 Additional writes on that source cannot occur until each preceding write has been
completed and acknowledged.

 Figure 12-7 shows the array-based synchronous remote replication process.

 In the case of synchronous remote replication, to optimize the replication process and
to minimize the impact on application response time, the write is placed on cache of
the two arrays.

 The intelligent storage arrays destage thesewrites to the appropriate disks later.

 If the network links fail, replication is suspended; however, production work can
continue uninterrupted on the source storage array.

 The array operating environment keeps track of the writes that are not transmitted to
the remote storage array.

 When the network links are restored, the accumulated data is transmitted to the
remote storage array. During the time of network link out- age, if there is a failure at
the source site, some data will be lost, and the RPO at the target will not be zero.
8 Section III ■ Backup, Archive, and Replication

Source Site Remote Site

1 2

4 3
So u rce

Production Source Storage Remote Storage Remote

Host Array Array Host

1 Write from the production host is received by the source storage array.

2 Write is then transmitted to the remote storage array.

3 Acknowledgment is sent to the source storage array by the remote storage array.

4 Source storage array signals write-completion to the production host.

Figure 12-7: Array-based synchronous remote replication

Asynchronous Replication Mode

In array-based asynchronous remote replication mode, as shown in Figure 12-8, a
write is committed to the source and immediately acknowledged to the host. Data is
buffered at the source and transmitted to the remote site later. The source and the
target devices do not contain identical data at all times. The data on the target device
is behind that of the source, so the RPO in this case is not zero.

Source Site Remote Site

1 3

2 4
So u rce

Production Source Storage Remote Storage Remote

Host Array Array Host

1 The production host writes to the source storage array.

2 The source array immediately acknowledges the production host.

3 These writes are then transmitted to the target array.

4 After the writes are received by the target array, it sends an acknowledgment to the source array.

Figure 12-8: Array-based asynchronous remote replication

 Chapter 12 ■ Remote Replication 9

Disk-Buffered Replication Mode

 Disk-buffered replication is a combination of local and remote replication technolo- gies.
A consistent PIT local replica of the source device is first created. This is then
replicated to a remote replica on the target array.

 Figure 12-9 shows the sequence of operations in a disk-buffered remote replica- tion. At
the beginning of the cycle, the network links between the two arrays are suspended, and
there is no transmission of data.

 While production applicationruns on the source device, a consistent PIT local replica
of the source device is created. The network links are enabled, and data on the local
replica in the source array transmits to its remote replica in the target array.

 After synchroni- zation of this pair, the network link is suspended, and the next local
replica of the source is created. Optionally, a local PIT replica of the remote device on
thetarget array can be created.

 The frequency of this cycle of operations depends on the available link bandwidth and
the data change rate on the source device. Because disk-buffered technology uses local
replication, changes made to the source and its replica are possible to track.

 Therefore, all the resynchronization operations between the source and target can be
done incrementally.

 When compared to synchronous and asynchronous replications, disk-buffered remote

replication requires less bandwidth.

 In disk-buffered remote replication, the RPO at the remote site is in the order of
hours.

 For example, a local replica of the source device is created at 10:00 a.m., and this
data transmits to the remote replica, which takes 1 hour to complete.

 Changes made to the source device after 10:00 a.m. are tracked. Another local replica
of the source device is created at 11:00 a.m. by applying track changes between the
source and local replica (10:00 a.m. copy)
10 Section III ■ Backup, Archive, and Replication

 During the next cycle of transmission (11:00 a.m. data), the source data has
moved to 12:00 p.m.

 The local replica in the remote array has the 10:00 a.m. data until the 11:00
a.m. data is successfully transmitted to the remote replica.

 If there is a failure at the source site prior to the completion of transmission,

then the worst-case RPO at the remote site would be 2 hours because the remote
site has10:00 a.m. data.

Source Device

Local Replica

2 4
3

Production
Host
Remote Replica
Source Storage Array Target Storage Array

Local Replica

1 The production host writes data to the source device.

2 A consistent PIT local replica of the source device is created.

3 Data from the local replica in the source array is transmitted to its remote replica in the target array.

4 Optionally, a local PIT replica of the remote device on the target array is created.

Figure 12-9: Disk-buffered remote replication

Network-Based Remote Replication

In network-based remote replication, the replication occurs at the network layer between
the host and storage array. Continuous data protection technology, discussed in the
previous chapter, also provides solutions for network-based remote replication.

CDP Remote Replication

 In normal operation, CDP remote replication provides any-point-in-time recov-
ery capability, which enables the target LUNs to be rolled back to any
previouspoint in time.

 Similar to CDP local replication, CDP remote replication typically uses a

 Chapter 12 ■ Remote Replication 11

journal volume, CDP appliance, or CDP software installed on a separate

host (host-based CDP), and a write splitter to perform replication between
sites.

 The CDP appliance is maintained at both source and remote sites.

 Figure 12-10 describes CDP remote replication. In this method, the replica is
synchronized with the source, and then the replication process starts.

 After the replication starts, all the writes from the host to the source are split
into two copies.

 One of the copies is sent to the local CDP appliance at the source site, and the
other copy is sent to the production volume.

 After receiving the write, the appliance at the source site sends it to the
appliance at the remote site.

 Then, the write is applied to the journal volume at the remote site. For an
asynchronous operation, writes at the source CDP appliance are accumulated,
and redundantblocks are eliminated.

 Then, the writes are sequenced and stored with their corresponding
timestamp. The data is then compressed, and a checksum is generated.

 It is then scheduled for delivery across the IP or FC network to the remote

CDP appliance.

 After the data is received, the remote appliance verifies the checksum to ensure
the integrity of the data.

 The data is then uncompressed and written to the remote journal volume.

 As a next step, data from the journal volume is sent to the replica at
predefined intervals.
12 Section III ■ Backup, Archive, and Replication
Host

Write Splitter

Local Remote
CDP Appliance CDP Appliance
SAN SAN/WAN SAN
Production
Volume

Source
Storage Array

CDP Replica
Journal

Remote
Storage Array

Figure 12-10: CDP remote replication