Vdocument - in - Appsync With Dell Emc Unity Table of Contents Executive Summary 3 Audience
Vdocument - in - Appsync With Dell Emc Unity Table of Contents Executive Summary 3 Audience
August, 2018
AUDIENCE
This white paper is intended for storage administrators, application owners, and database administrators who are
currently administering AppSync in their environment, Dell EMC internal field personnel, as well as partners who assist
with deploying AppSync.
INTRODUCTION
Dell EMC AppSync enables integrated copy data management (iCDM) with Dell EMC's primary storage systems.
AppSync simplifies and automates the process of generating and consuming copies of production data. By abstracting the
underlying storage and replication technologies, and through deep application integration, AppSync empowers application
owners to satisfy copy demand for operational recovery and data repurposing. In turn, storage administrators need only
be concerned with initial setup and policy definition management, resulting in an agile, frictionless environment. AppSync
automatically discovers the application, learns of the layout structure, and maps it through the virtualization layer, to the
underlying storage device. AppSync orchestrates all the activities required from copy creation and validation through
mounting, at the target host, and launching, or recovering, the application copy. The supported workflows additionally
include refresh, expire, and restore production.
The architecture of AppSync has three major components: the AppSync server, AppSync host plug-ins, and the AppSync
user interface.
• The AppSync server is deployed on a Windows server system, either physical or vitual, and controls all workflow
activities, manages the alerting and monitoring aspects, and persists internal data in a PostgreSQL database.
• The AppSync host plug-ins are installed on all source and mount hosts, providing the ability to integrate with the
operating systems, and the applications hosted on those operating systems. This includes such applications as
Microsoft Exchange, Microsoft SQL, Oracle, or VMware datastores. In the case of VMware datastore replication, there
is no host plug-in, as AppSync communicates directly with the vCenter server.
• The AppSync user interface is typically a web-based interface, or graphical user interface (GUI), used specifically
for AppSync world class copy management features, however, AppSync can also be managed using a vSphere VSI
plug-in, RESTapi, as well as a command line interface (CLI).
Note: For more information about each component, refer to the AppSync Installation and Configuration Guide, the
AppSync User and Administration Guide, and to validate supported versions, please refer to the latest AppSync Support
Matrix.
AppSync supports block FC and iSCSI protocols, as well as NFS file storage on Unity arrays, however does not support
CIFS.
• Unified Snapshots are supported on block (FC and iSCSI) resources (LUNs, Consistency Groups, Thin Clones,
VMware VMFS Datastore) as well as file resources (Filesystems, VMware NFS Datastore). Unity OE version 4.2 and
later supports Unity Thin Clones.
• Native Asynchronous Replication provides local, remote, both local & remote protection. Native synchronous
replication is supported on block resources (LUNs, Consistency Groups, and VMware VMFS Datastores).
• Dell EMC UnityVSA is a Software Defined Storage (SDS) solution that runs as a VMware vApp on the VMware ESXi
Server platform. It provides a flexible storage option for environments that do not require purpose built storage
hardware.
Note: For more information on Unity, refer to the following whitepapers: Dell-EMC-unity-introduction-to-the-platform, Dell
EMC Unity Replication Technologies, and Dell EMC Unity: Snapshots and Thin Clones A Detailed Review.
APPSYNC AND DELL EMC UNITY PREREQUISITES
• The AppSync server and host plug-in software are assumed to be installed and configured, with the host’s
applications discovered, as well as any vCenter server configured if applicable, according to the AppSync Installation
and Configuration Guide.
• In order to support Unity Native Replication, a minimum of Unity OE version 4.2 and AppSync version 3.8 is
required, and both the local and remote Unity arrays must be registered within AppSync.
− Each Dell EMC Unity system requires a valid full AppSync license that must be configured using the AppSync
console.
• Add the Dell EMC Unity systems to AppSync, ensuring the AppSync user has the Resource Administrator role.
• AppSync creates local, remote, or both local & remote snapshots and Thin Clones of application data, and names
them with the prefix AppSyncSnap or AppSynClone respectively.
• AppSync supports Unity Native Replication for block storage
− Neither VVols nor file storage is supported as of AppSync 3.8
• AppSync supports two types of Unity Native Replication types, Synchronous and Asynchronous replication
• All typical workflows, such as protecting applications using the Bronze, Silver, and Gold Service Plans, repurposing
local and remote copies, and mounting, refreshing, and restoring copies are supported.
− Please refer to the AppSync User and Administration Guide for more details on what functions are supported
for each type of application
• AppSync support Unity file storage with the following limitations
− NFS 4.x is supported with Unix/Linux NFS filesystems. NFS 4.x version filesystems used as VMWare NFS
datastores are not supported, thought NFSv3 for NFS datastores are.
− Unity file storage carved from VMWare virtual disks, from NFS datastores, is supported by Oracle and Unix
filesystem applications only
− AppSync repurposing workflows are not supported with Unity file storage
− AppSync does not support creating remote/target snapshots of Unity Native Replication sessions created for
any File storage resource.
Note: Please refer to the Service plan overview section in the AppSync User and Administration Guide, which depicts
the latest support for each Unity replication type.
Add a Dell EMC Unity system by navigating to Settings > Storage Infrastructure, and then selecting Unity from the Add
drop down box, as shown in Figure 2 - Adding Dell EMC Unity to AppSync.
Figure 2 - Adding Dell EMC Unity to AppSync
It is necessary to apply a proper license file by navigating to Settings > Licenses, and either clicking Obtain License
File Online, to generate the licensing file, if the LAC and sales order number are known, or, if the license file, depicted
with the .lic file extension, is already obtained, Upload and Install License File, as shown insFigure 3 - Applying the
Unity License. This applies to all licensing models.
Figure 6 - MS SQL Gold Service Plan Event Dialog depicts the events of a typical Gold Service Plan protection
workflow of a Microsoft SQL database located on a Unity array using Synchronous native replication.
• When using an Asynchronous native replication session, AppSync creates a snapshot of the application, then
creates a snapshot on the remote array using Dell EMC Unity snapshot shipping technology. A SYNC operation is
issued on the replication session to synchronize the data. Once the remote snapshot is created, the local snapshot is
removed.
• When using a Synchronous native replication session, a snapshot gets created directly on the remote array, as
synchronous replication session is always In-SYNC.
• Native Replication sessions are supported as of AppSync 3.8 with Unity OE version 4.2
DISPLAYING COPIES
View the copies created from either Copy Management or the Service Plans.
Figure 7 - Copy Management Copies shows how copies appear in Copy Management.
Figure 8 - Service Plan Copies Tab depicts the Copies tab of a Service Plan.
Figure 10 - Mounting a Linux Filesystem depicts the event dialog during a typical Mount operation for a Linux
Filesystem using a Bronze Service Plan.
REPURPOSING WORKFLOWS
AppSync supports the creation of multi-generational Unity Snapshots and Thin Clones, with a prefix of either
AppSyncSnap or AppSynClone, respectively. AppSync supports Unity repurposing workflows as of AppSync 3.5 and
Unity OE version 4.2, or later.
AppSync repurposing supports a number of different types of workflows, such as creating application consistent local or
remote copies, automating mounting and application recovery scenarios, as well as scheduling these operations.
Repurposing workflows focus on a single application at a time, and does not utilize a copy count rotation policy, as occurs
with Service Plan workflows. Therefore, repurposing workflows are not generally utilized for protecting the application, or
also considered backup solutions. Repurposing is most often considered suitable for replicating production environments,
and using them quality assurance testing, development, offloading reporting, patch management, to name but a few. More
details on repurposing workflow can be referenced in the AppSync User and Administration Guide.
Repurposing workflows offer multi-generation copies. This is to say, a 1st generation copy, or one copy removed from the
source, can be optionally copied to a 2nd generation copy, or a copy that is twice removed from the source.
Figure 13 - Repurposing Overview represents an overview of multi-generational copies-
Figure 13 - Repurposing Overview
The following are typical repurposing use cases, which AppSync is known to be utilized. This is not an exhaustive list,
rather, helps identify some of the possible workflows.
• On-demand copies – a copy of a single application is used for an extended period of time, and then discarded – this
copy is not used for backup purposes to restore from - maintaining copy retention. Copies can be used for performing
patch management testing, performance tuning against non-production environments, or offloading reporting, thus
reducing the amount of IO performed against a production environment.
• Data Masking – a 1st generation copy is created and mounted in order for sensitive data to be masked. Once the
sensitive data is masked, the copy is then unmounted in order to create a 2nd generation copy. This 2nd generation
copy, which has an inauthentic version of the data, can then be utilized.
• Remote Copy Retention – long term retention on a remote array, sometimes identified as a disaster recovery copy,
can be accomplished using repurposing. Remote copies can be created off RecoverPoint bookmarks, by selecting the
Use Bookmark as an intermediate step, as seen in Figure 14 - Repurpose Intention, as well as off native
replication sessions. When using RecoverPoint, however, Remote repurposing is the only way to support remote
copy retention - on the remote array. AppSync supports using the Silver and Gold Service Plans with native
replication for long term retention on the remote array.
• Snap-of-Snap – similar to a data masking requirement, repurposing supports creating a 1st generation application
consistent copy of a single application, used as the source for multiple 2nd generation copies. These 2nd generation
copies can then be utilized for things such as:
− Providing multiple copies of the same point-in-time (PIT) to developers - identical copies for training purposes
or to offer as a baseline for collaboration efforts
− Alleviating the need to quiesce the production environment unnecessarily for many copies
− Refreshing the 2nd generation copy whilst not having to change the PIT
REPURPOSING CONSIDERATIONS
• Used primarily for testing or development purposes for extended periods of time, then discarded/expired when done
• Repurposed copies do not figure into RPO calculations (refer to the AppSync User and Administration Guide for more
details regarding RPO alerts)
• Restores are supported from the 1st generation copies only
• 2nd generation copies are not application consistent (no application discovery, mapping, application integration such
as freeze and thaw of a database – callouts are supported for unmounting only – refer to the AppSync User and
Administration Guide for more details on callout scripts)
− For this reason, 2nd generation copies should never be taken when a 1st generation copy is mounted
• A 2nd generation copy is considered “application consistent,” if the 1st generation copy is and has also not been
mounted, or more importantly, altered by the time the 2nd generation copy is taken
− The 2nd generation copy is identical to the 1st generation copy, at time of creation
− A mounted/altered 1st generation copy should be unmounted before creating the 2nd generation copy
REPURPOSING SPECIFICS
• 1st generation copies must be Unity Snapshots (Unity Thin Clones are not supported as 1st generation copies)
• 2nd generation copies can either be Unity Snapshots or Unity Thin Clones
• AppSync does not provide a way to mount read-only copies of Unity snapshots or Unity Thin Clones
• When creating 2nd generation Unity Thin Clones, the 1st generation snapshot cannot be in a mounted state
• Per the Dell EMC Unity: Snapshots and Thin Clones A Detailed Review white paper:
“Prior to creating a Thin Clone, users need to create a snapshot. The snapshot can have read-only host access or
be an unattached snapshot.”
− For AppSync, this means the 1st generation copy must not be in a mounted state, when creating or
refreshing a 2nd generation Thin Clone. Figure 14 - Thin Clone Error depicts the error seen if the 1st
generation snapshot is mounted, or in read-write mode.
Figure 16 - Creating a Repurposed Filesystem depicts the repurpose initiation of two UNIX filesystems.
Once the object(s) are highlighted, clicking Repurpose Create Repurposed Copy launches the Repurposing wizard,
as seen in Figure 18 - Repurpose Intention.
Creating a repurposed copy always originates with the creation of a 1st generation copy. In the case of Dell EMC Unity,
this will always be a snapshot. Once the 1st generation copy is created, any number of 2nd generation copies, either snap,
thin clone, or both, can be created. The initialization wizard provides the option to create a 2nd generation copy at the
same time the 1st generation copy is created. This is for convenience and does not require the creation of the 2nd
generation copy, at the time of the creation of the 1st generation copy.
The following steps outline how to create an initial 1st generation copy and then creating a 2nd generation copy. This way,
it is clear on how one can create multiple 2nd generation copies once a 1st generation copy exists.
The initial wizard provides the ability to create only one 2nd generation copy, but there may be times when many additional
2nd generation copies are required. The process one may follow for creating multiple 2nd generation copies, is as follows.
Configuring the 1st Generation Copy
1. The repurpose wizard’s copy generation option default, is to create a 1st generation copy. Site options are discussed
in a later section titled: Site Settings – Local or Remote.
2. Leave the default Create a 1st gen copy selected, and click Next to begin the repurposing wizard.
The following two sections depict options one would encounter repurposing that specific type of application. Refer to the
AppSync User and Administration Guide for more details on each application’s copy options.
The default copy type is generally needed to be changed from Full to Copy. This is because the repurposing wizard is
being performed on a MS SQL database, for which there is already in a protection service plan. Repurposing copies are
not generally used for protection purposes, and no two workflows should utilize a SQL full copy type, as a full backup
alters the transaction log sequence numbers, thus disrupts a restore operation taken from another workflow process.
Please refer to the MS SQL section of the AppSync User and Administration Guide for more details on MS SQL settings,
and also other application types being repurposed, such as Oracle.
There is a limited number of options when repurposing filesystems, as compared to repurposing MS SQL or Oracle
databases. Filesystem repurposing workflows require a label to be configured, unlike other types of workflows, where a
default label exits.
There are additional settings which can be configured for 1st generation copies only. These are described as follows:
1. The Configure storage options settings are for managing VPLEX and advanced VMAX configuration options.
These settings are not discussed within this white paper, so please refer to the AppSync User and Administration
Guide for more details. If an environment is behind VPLEX, then proper configuration is required. Please ensure the
Cluster selection and Array preference are configured appropriately.
2. The Advanced Settings only applies to Windows environments, so is not seen when repurposing Oracle
databases. The configuration options seen once clicked, provides an ability to retry Microsoft’s Volume Shadow Copy
service, or VSS, if a timeout occurs. The default setting is to retry three times, but can be extendedand an additional
amount of time between each retry can be configured.
Once the copy settings have been configured, click Next to configure the Schedule settings, and then to complete the
wizard, initiating the 1st generation copy.
Once the repurposing workflow completes, the 1st generation copy will be visible in two locations, under Copy
Management and also through the Repurposed Copies menu.
Figure 20 - Copy Management Repurposed Copies List
There is an alternative way to manage repurposed copies, including the ability to create 2nd generation copies of a 1st
generation copy. Navigate to the Repurposed Copies menu, as seen in Figure 21 - Repurposed Copies Menu Button,
highlight the 1st generation copy, and then click Repurpose.
Mounting options can also be configured for both 1st and 2nd generation copies. Please refer to the AppSync User and
Administration Guide for mounting option details, as each application type has different mount options not covered within
this white paper, however, the following notes should be considered.
• The 1st generation copy is not typically mounted when planning on taking 2nd generation copies, unless using the
repurposing workflow for data masking purposes - specifically changing the data on the 1st generation copy prior to
creating the 2nd generation copy.
• The 1st generation copy should never be mounted while creating the 2nd generation copy since there is no application
integration with the 2nd generation copy, it is an exact duplicate of the 1st generation copy. If there are IO on the 1st
generation copy at the time of creating the 2nd generation copy, it is likely the 2nd generation copy will not be in a
consistent state.
• Since 1st generation copies are application aware, in cases with MS SQL and Oracle, they can be recovered,
however, since 2nd generation copies are not, they cannot be recovered in an automated fashion with AppSync. Post-
mount scripts can be utilized for this purpose, however, and the unmount callout script will then need to be utilized
prior to refreshing the 2nd generation copy. Please refer to the AppSync User and Administration Guide for more
details on unmount callout scripts for 2nd generation copies.
AppSync provides the ability to refresh both 1st and 2nd generation copies, in one of two ways.
1. Repurposed Copies – similar to the menu seen in Figure 21 - Repurposed Copies Menu Button, clicking the
Repurposed Copies button under Copy Management, launches a holistic view of all repurposed copies. One need
to simply select the copy and click Refresh.
2. Copy Management Copies – Navigating to the Copies menu under Copy Management, such as seen in Figure 22
– Repurposing an MS SQL Copy, one need only select the copy and then click Refresh.
Windows 1st Gen Windows 2nd Gen Unix 1st Gen Unix 2nd Gen
1st Generation is not AppSync Way AppSync Way Native Array AppSync Way
“Attached”
1st Generation is AppSync Way Native Array Native Array Native Array
“Attached”
CONCLUSION
In conclusion, this whitepaper explains AppSync integration with Dell EMC Unity arrays in order to create copies of
application data on local and remote storage arrays, including repurposing. It also covers key information and concepts
when protecting an application residing on Dell EMC Unity All Flash block and NFS file storage using AppSync. It
discusses the inner operations of AppSync software while protecting applications using Dell EMC Unity Snapshot and
Thin Clone technologies, concepts when repurposing an application residing on Dell EMC Unity All Flash block storage
as well as other workflows like mounting and restore operations.
REFERENCES
AppSync 3.8 User and Administration Guide
https://support.emc.com/docu90291_AppSync-3.8-User-and-Administration-Guide-.pdf
AppSync 3.8 Installation and Configuration Guide
https://support.emc.com/docu90292_AppSync-3.8-Installation-and-Configuration-Guide-.pdf
AppSync Release Notes
https://support.emc.com/docu90294_AppSync-3.8-Release-Notes.pdf
Dell EMC Unity Replication Technologies – A detailed Review
https://support.emc.com/docu69886_Dell-EMC-Unity:-Replication-Technologies---A-Detailed-Review.pdf
AppSync Simple Support Matrix
https://elabnavigator.emc.com/vault/pdf/EMC_AppSync.pdf
Dell EMC Unity: Snapshots and Thin Clones A Detailed Review
https://support.emc.com/docu69887_Dell-EMC-Unity:-Snapshots-and-Thin-Clones---A-Detailed-Review
LIST OF FIGURES
Figure 1 - User Interface ......................................................................................................................................................... 3
Figure 2 - Adding Dell EMC Unity to AppSync........................................................................................................................ 7
Figure 3 - Applying the Unity License ..................................................................................................................................... 7
Figure 4 – MS SQL Bronze Service Plan Event Dialog .......................................................................................................... 8
Figure 5 - MS SQL Silver Service Plan Event Dialog ............................................................................................................. 9
Figure 6 - MS SQL Gold Service Plan Event Dialog ............................................................................................................... 9
Figure 7 - Copy Management Copies ................................................................................................................................... 10
Figure 8 - Service Plan Copies Tab ...................................................................................................................................... 10
Figure 9 - Mounting and Recovering a MS SQL Database .................................................................................................. 11
Figure 10 - Mounting a Linux Filesystem .............................................................................................................................. 11
Figure 11 - Restoring a Remote MS SQL Copy.................................................................................................................... 12
Figure 12 - MS SQL Affected Entities ................................................................................................................................... 13
Figure 13 - Repurposing Overview ....................................................................................................................................... 14
Figure 14 - Thin Clone Error ................................................................................................................................................. 15
Figure 15 - Create Repurposed Copy ................................................................................................................................... 16
Figure 16 - Creating a Repurposed Filesystem .................................................................................................................... 16
Figure 17 - Repurpose Wizard Intention ............................................................................................................................... 16
Figure 18 - MS SQL Repurpose Copy Options..................................................................................................................... 17
Figure 19 - UNIX Filesystem Repurposing Copy Options .................................................................................................... 18
Figure 20 - Copy Management Repurposed Copies List ...................................................................................................... 19
Figure 21 - Repurposed Copies Menu Button ...................................................................................................................... 19
Figure 22 – Repurposing an MS SQL Copy ......................................................................................................................... 19
Figure 23 - Repurposing a Filesystem Copy......................................................................................................................... 20
Figure 24 - Repurpose Intention ........................................................................................................................................... 21