APPSYNC VMAX ARRAY SUPPORT GUIDE

Including comparisons between VMAX2 and VMAX3

ABSTRACT
This white paper discusses and prov ides guidelines for users who are integrating their

application environment hosted on a Dell EMC VMAX storage system using Dell EMC

AppSync as their application protection tool.

October, 2016

W HITE PAPER

The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect
to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.
Use, copy ing, and distribution of any EMC software described in this publication requires an applicable software license.
EMC 2, EMC, the EMC logo, Dell EMC AppSync, and EMC VMAX are registered trademarks or trademarks of EMC Corporation in the
United States and other countries. All other trademarks used herein are the property of their respective owners. Copyright 2016 EMC
Corporation. All rights reserved. Published in the USA., 09/16 White Paper, H13621.1
EMC believes the information in this document is accurate as of its publication date. The information is subject to change without notice.
EMC is now part of the Dell group of companies.

TABLE OF CONTENTS
EXECUTIVE SUMMARY................................................................................................. 5
AUDIENCE ........................................................................................................................................................ 5

INTRODUCTION............................................................................................................ 5
USING THE GUIDELINES .............................................................................................. 5
VMAX CONFIGURATION REQUIREMENTS .................................................................... 6
VMAX Block Storage Considerations............................................................................................................ 6
Ad vanced VMAX3 Features Supported........................................................................................................ 6
VMAX SMI-S Pro vider ..................................................................................................................................... 7

VMAX ARRAY DISCOV ERY AND CONFIGURATION ...................................................... 7

Modifying Credentials ...................................................................................................................................... 9
Provider Preference......................................................................................................................................... 9

PREVENTING AUTOMATIC DISCOVERY OF EMC ARRAYS ........................................... 9

Procedure to Prevent Automatic Discovery of Arrays ................................................................................ 9

STORAGE CONFIGURATION ...................................................................................... 10

VMAX2 ............................................................................................................................................................. 10
VMAX3 ............................................................................................................................................................. 10
VMAX Storage Configuration Comparison................................................................................................. 11
Using Storage Groups For Copy Target Devices...................................................................................... 11
Modifying Storage Groups ............................................................................................................................ 12
Storage Pools: Enabling A Virtual Pool For AppSync .............................................................................. 13
Device Management on the Backend VMAX............................................................................................. 14

REMOVING A VMAX ARRAY FROM APPSYNC ............................................................ 14

REMOVING AN SMI-S PROVIDER................................................................................ 15
VMAX3 SNAPVX CONFIGURATION DETAILS .............................................................. 15
SnapVX Linking .............................................................................................................................................. 16

PROTECTING APPLICATIONS USING APPSYNC SERVICE PLANS ............................. 16

Application Discovery Phase ........................................................................................................................ 18
Application Mapping Phase .......................................................................................................................... 18
Create Copy Phase........................................................................................................................................ 18
Mount Copy Phase......................................................................................................................................... 21

REPURPOSING WORKFLOWS.................................................................................... 23
RESTORE FROM APPLICATION COPIES .................................................................... 26
LUN Level Restores from VP Snap and TF Clone copies ....................................................................... 26
VM Le vel Restore from VP Snap and TF Clone copies ........................................................................... 26
VMAX3 Restores with SnapVX .................................................................................................................... 26

TROUBLESHOTTING TIPS .......................................................................................... 26

CONCLUSION ............................................................................................................. 28

EXECUTIVE SUMMARY
This document prov ides technical information about integrating Dell EMC AppSync software with EMC VMAX storage sy stems,
including the environmental caveats that should be taken into consideration when using both AppSync nc and VMAX. This document also
prov ides guidance while deploying the AppSync nc software in environments where application data resides on a VMAX storage system.

AUDIENCE
This white paper is intended f or Dell EMC customers that are application and/or storage administrators who are currently administering
AppSync nc in their environment and Dell EMC internal f field personnel and partners who assist customers with deploying AppSync.

INTRODUCTION
Dell EMC AppSync is software that enables Integrated Copy Data Management (iCDM) with Dell EMC's primary storage systems.
AppSync simplifies and automates the process of generating and consuming copies of production data. By abstracting the underlying
storage and replication technologies, and through deep application integration, AppSync empowers application owners to satisfy copy
demand f or operational recovery and data repurposing on their own. In turn, storage administrators need only be concerned with initial
setup and policy management, resulting in an agile, frictionless environment. AppSync automatically discovers application databases,
learns the database structure, and maps it through the virtualization layer to the underlying storage LUN. It then orchestrates all the
activities required from copy creation and validation through mounting at the target host and launching or recovering the application.
Supported workflows also include refresh, expire, and restore production.

USING THE GUIDELINES

The information and guidelines described in this document have been provided by Dell EMC AppSync nc Engineering. This information is
supplemental and should be used in conjunction with other AppSync nc documentation, including the AppSync User and Administration
Guide, the Installation and Configuration Guide, and the version specific Release Notes. AppSync terms seen directly within the user
interface (UI) are bold in blue. This prov ides a direct link to what is seen with the user console, offering clearer readability. For example,
within the user interface, there is a menu called Settings where you can click Storage Infrastructure which prov ides a wizard to add
storage array s. As seen in Figure 1, the words in blue are seen verbatim.

Figure 1 - AppSync Console UI

VMAX CONFIGURATION REQUIREMENTS

AppSync supports creating and managing copies of applications on VMAX array s using TimeFinder and SnapVX replication
technologies. AppSync also supports remote copy management in an SRDF configuration.
Please also refer to the AppSync Performance and Scalability Guidelines document f or detailed performance guidelines that will help
optimize AppSync nc f or performance and scalability, found here - http://www.emc.com/collateral/white-papers/h11153-appsyncperf-and-scale-wp.pdf . Also, always refer to the latest support matrix for up-to-date support details, found here https://elabnavigator.emc.com/eln/modernHomeDataProtection.
The f ollowing tables depict the current supported environments.
Table 1 - AppSync Supported VMAX Technologies

Replication Technology
TimeFinder Clone
TimeFinder Mirror
TimeFinder Snap
TimeFinder VP Snap
SnapVX
RecoverPoint Local*
RecoverPoint Remote*
RecoverPoint
Local and Remote*
SRDF/S
SRDF/A (non-MS Apps only)

VMAX Support
RAID LUN
Pool LUN
Yes
No
No
Not applicable
Not applicable
Yes
Yes

Yes
No
No
Yes
Yes
Yes
Yes

Yes

Yes

Yes
Yes

Yes
Yes

*Note: Applies to VMAX2 only RecoverPoint with VMAX3 is not currently supported.

Table 2 - AppSync Supported Applications

Supported
Applications
Oracle on Linux and AIX
Microsoft Exchange
Microsoft SQL
Microsoft file systems
Linux and AIX file systems
VMware Datastores
VMAX BLOCK STORAGE CONSIDERATIONS
As noted in the previous section, AppSync supports TimeFinder and SnapsVX technologies. VMAX2 array s support thin pool based
LUNS f or snap operations, and due to this restriction, AppSync only supports TimeFinder VP Snap technology. Traditional TimeFinder/
Snap which utilizes VDEVs, are not supported with AppSync. TimeFinder/Clone and RecoverPoint based copies, with a VMAX2 backend, are the only technologies which support RAID LUNs. On the VMAX3, SnapVX Clone and Snap are both supported, and both
utilize the default SRP (storage resource pool).

ADVANCED VMAX3 FEATURES SUPPORTED

ENA S
AppSync 3.0 offers support for VMAX3s eNAS (embedded network attached storage) feature. VMAX3 eNAS offers consolidated file
storage and supports features such as Service Level Objectives (SLO)-based provisioning. Please reference the VMAX3 eNAS
Deployment For Microsoft Windows And SQL Server Environments white paper f or more details here http://www.emc.com/collateral/white-papers/h14241-vmax3-enas-deployment-microsoft-sql-server-wp.pdf
6

A FA
AppSync 2.2.2.2 and greater offers support for VMAX All Flash Array s; the first true fully all flash based VMAX array . Please reference
The VMAX All Flash Storage Family white paper f or more details here - https://www.emc.com/collateral/white-papers/h14920-introto-v max-af-storage.pdf

VMAX SMI-S PROVIDER

AppSync uses the EMC SMI-S provider software (now found bundled with the Solutions Enabler package) to communicate with the
VMAX storage systems. Alway s refer to the SMI-S Provider table, within the latest Dell EMC AppSync support matrix, found at the
following address f or the most current details. https://elabnavigator.emc.com/eln/modernHomeDataProtection

COMMUNICA TING WITH THE SMI-S PROVIDER:

Queries all required information, such as mapping, device, storage groups, pools, etc.

Performs active management tasks, such as snapshot and clone sessions creating, refreshing, terminating, restoring, LUN
masking etc.

The SMI-S provider software can be installed on a Windows or Linux, virtual or physical, machine that has physical connectivity to the
VMAX(s) being managed. The SMI-S provider must be registered as a resource within AppSync. There should be at least six
gatekeepers exported from the VMAX being managed to the SMI-S provider host. They should be spread across two fabrics/HBAs for
redundancy .
Up to five arrays using a single SMI-S instance can be supported, however, due to performance considerations, ascertained by multiple f
actors, caution should be used and so the recommendation is one SMI-S provider per local and its remote array. Consider other
applications using the provider, total volumes and copies under management, etc, when determining how many arrays one SMI-S
provider can support.
If the SMI-S Provider and AppSync server share the same host, then the SMI-S provider should be installed first. If AppSync is already
installed, install SMI-S provider and restart the AppSync server services (Dell EMC AppSync Datastore service, Dell EMC AppSync
Security Server, and Dell EMC AppSync Server Service).

VMAX ARRAY DISCOVERY AND CONFIGURATION

VMAX array s are discovered using an SMI-S providers that manage the VMAX. VMAX array s are discovered as seen in Figure 2,
Figure 3, Figure 4, and Figure 5. AppSync requires admin, or equivalent credentials, to the SMI-S provider. As part of the discovery
phase, AppSync establishes a connection with the SMI-S provider, obtaining a list of all the array s managed, along with detailed
information about the array s, such as the microcode information and model number. A Rediscover should be performed when changes
are made to the array , such as adding additional disks to a storage group. AppSync uses a secure connection over port 5989 to
communicate with the SMI-S provider. The same SMI-S provider can be used to discover both VMAX2 and VMAX3 array s. Ensure the
latest support matrix is referenced to determine which version to use with the particular environment.

Figure 2 - Adding VMAX Storage to AppSync

Figure 3 - Registering the SMI-S Provider

Figure 4 - Selecting the VMAX Array (s)

Figure 5 - Selecting Storage Groups and Pools

Please note: When adding a VMAX3 array , optionally choose one or more storage groups, however, only the default SRP is
supported. AppSync environments with additional SRPs are supported, however, this option on VMAX3 is not configurable.

MODIFYING CREDENTIALS
If the SMI-S provider credentials are modified by the SMI-S admin, they must also be change within AppSync. Navigate to the Storage
Infrastructure menu, highlight the VMAX, highlight the SMI-S host, and then clicking Edit, in order to change the credentials.

PROVIDER PREFERENCE
AppSync uses the SMI-S provider that was used to initially discover the VMAX array , as the preferred provider. The pref erred order, in
the case of multiple SMI-S providers, can be changed within the AppSync UI by highlighting the provider and clicking Set Preferred.

PREVENTING AUTOMATIC DISCOVERY OF EMC ARRAYS

EMC VMAX array s are automatically discovered when they are physically connected and zoned through HBAs to the server where the
EMC SMI-S provider is installed. It is best practice to limit each SMI-S host to one local, and its corresponding remote array, however,
may manage up to five, based on performance considerations. The performance of the SMI-S provider is critical to AppSyncs
performance. The discovery of arrays on an SMI-S provider host, which is not used by AppSync, may impact the performance of
AppSync jobs. Below is a short description of the procedure to prevent automatic discovery of an array which is not required to be
managed by a given SMI-S provider, for which AppSync utilizes.

PROCEDURE TO PREVENT AUTOMATIC DISCOVERY OF ARRAYS

Please dont perform this procedure if the SMI-S provider host is shared by other EMC applications , e.g. ECC, SRM,
Unisphere, etc., as those applications may need the connectivity and information from all the arrays.
1.

Either stop the ECOM service on the Windows host, or execute the following command to stop the EMC SMI-S provider service:

2.

<install_dir>\ECIM\ECOM\bin\sm_service stop ecom.exe

3.

Execute the command SYMCLI\bin\symcfg list to list the VMAX array ID(s) to disable

4.

In a plain text file, list the IDs of the array(s) that should not be auto-discovered

5.

Save the file as symavoid in the SYMAPI/config directory (ensure no extension exists, such as the Windows .txt extension, which is hidden by
default)

6.

Rename the SYMAPI/db/symapi_db.bin file to something like symapi_db.bin.old{todays date}. The database will be recreated the next time the
SMI-S provider tries to discover arrays.

7.

Start ECOM service or execute the following command to start the EMC SMI-S provider service:

8.

<install_dir>\ECIM\ECOM\bin\sm_service start ecom.exe

9.

Perform a discovery for the changes to take effect. Do this by running the discover command in ECIM\ECOM\bin \TestSmisProvider.exe binary
shipped with the SMI-S Provider, or running the Solutions Enabler command SYMCLI\bin\symcfg discover and then running the
SYMCLI\bin\symcfg list command to validate.

STORAGE CONFIGURATION
AppSync requires copy devices to create successful application copies using TimeFinder Clone and VP Snap technologies, or in the
case of VMAX3, SnapVX technology using targetless snapshots. AppSync provides two choices to the storage administrator for
configuring a copy device to be used by AppSync; creating and pairing targets for TimeFinder Clone and VP Snap, and one optional
choice f or SnapVX copies, which are:

Storage Groups: Using a dedicated storage group (one or more), which contain preconfigured target devices

Storage Pools: Enabling a virtual provisioning pool(s) for AppSync

o

Only applicable f or VMAX2

One or more pools can be used

Prov ides thresholds, so if a pool reaches a certain used percentage (90% by default), AppSync will no longer use that
pool to create new devices

VMAX3 array s use the default Storage Resource Pool (SRP) and cannot be configured differently

There are fundamental differences between VMAX2 and VMAX3 array s, when it comes to how storage is configured. With the VMAX2
array s, optionally choose to use storage groups, and/or storage pools. With the VMAX3 array , optionally choose a storage group. There
is also a difference with how VMAX3 uses storage with repurposing workflows versus service plan workflows. Please refer to the
Repurposing Workf low section for more details.

VMAX2
Storage administrators can use either or both options. If neither option is used, AppSync jobs will f ail, because it has no devices in a
storage group to use as target devices, nor does it have the authority to use a storage pool to create new devices. If just the storage group
is defined, then AppSync will attempt to use devices within that storage group, and if none are available, then the process will f ail. When
both options are used, AppSync first tries to get copy storage from previously-provisioned devices (in a storage group defined), and if
none are found, storage pools are used to provision new targets. The devices AppSync provisions are not added to the defined storage
group, so some administrators may wish to manually add them for proper housekeeping, otherwise, the devices AppSync creates are
easily identifiable, as seen in the section titled Device Management on the Backend VMAX and depicted in Figure 12 and Figure 13.

VMAX3
Storage administrators can use an optional storage group f or copy devices. If a storage group is not defined, AppSync will use the SRP to
create targetless/unlinked devices, during service plan workflows, and will create target/linked devices from the SRP during
repurposing workflows. If a storage group is defined, AppSync still uses the SRP during the copy phase of a service plan workflow,
however, will use the devices in the storage group during the mount operation; when a device requires linking. If no device is available in
a storage group defined within AppSync, the mount phase will fail. When running a repurposing workflow with a storage group
st
defined, AppSync requires the target device to be present during the 1 generation copy phase.

Configuring a storage group prevents AppSync from creating devices in the SRP devices must be available in the configured
storage group
10

Serv ice Plans do not require target dev ices during the copy phase, but requires during the mounting phase

Repurposing workf lows require target dev ices for the copy phase for the 1 generation copies, but not for the 2 generation copies
please ref er to the SnapVX Linking section for more details.

st

nd

VMAX STORAGE CONFIGURATION COMPARISON

Table 3 depicts the differences between VMAX2 and VMAX3, with regards to AppSync storage configuration options.
Table 3 - VMAX Storage Configuration Comparison
Settings
VMAX2 Behavior

VMAX3 Behavior

No storage conf iguration

(def ault settings)

Copy process will f ail

Copy process is successful, using the

def ault SRP

Storage Pool Only

Copy process is successful if below

threshold. New devices created from
configured pool

Not Applicable Def ault SRP is used

and is not conf igurable

Storage Group Only

If dev ices are av ailable in the SG, then

the copy process will be successful. If
there are no f ree dev ices, the copy
process f ails as new dev ices cannot be
created since a pool is not conf igured.

Both Storage Group and

Storage Pool

If dev ices are not av ailable in the

storage group, AppSy nc will create new
dev ices f rom the configured pool.

Serv ice Plan copy process will be

successful regardless of available
dev ices in a storage group, but will
f ail if dev ices are not available during
the mount operation.
Repurposing workf lows require that
dev ices are av ailable in the storage
st
group during the 1 generation copy
phase, and will f ail if none are
nd
av ailable. 2 generation copies do
not require target dev ices.
Not Applicable same as Storage
Group Only , which, when conf igured,
prev ent AppSy nc from creating target
dev ices in the SRP.

USING STORAGE GROUPS FOR COPY TARGET DEVICES

Create one or more Storage Groups with no host connectivity (not attached to any LUN masking view). Please take note of the
f ollowing rules, when prov isioning new target dev ices for copy processing:

Make sure the target copy device is of the same size and conf iguration (i.e. meta configuration) as source device

For VP Snaps, make sure the target dev ices are thin provisioned

If the source dev ices are already in a VP Snap session f rom a thin pool, make sure the target copy devices are also provisioned in
the same pool (VP Snaps of same source must all be f rom same pool) please ref er to the TimeFinder VP Snap documentation
f or more details

If an application already has an existing TimeFinder clone or VP Snap session, ensure the target devices of that existing session are
placed in one of the storage groups conf igured within AppSync. AppSync reuses existing sessions.
All dev ices in the configured storage group(s) are used as copy devices, so never add production volumes into AppSync assigned
storage group(s) as it will result in data loss. Do not share the same storage group(s) across multiple AppSync servers.
As seen in Figure 5Figure 5 - Selecting Storage Groups and Pools, AppSy nc discovers storage groups which are not part of any
masking v iew. To conf igure a storage group f or an AppSync target device selection, enable it by clicking on the checkbox. AppSync
11

allows selecting multiple storage groups. Once selected, a warning states that the devices within in the selected storage group will be
used as target f or copies created by AppSync, as seen in Figure 6. The data on those dev ices will be ov erwritten.

Figure 6 - Storage Dev ice Warning

Rev iew the storage details within AppSy nc, under Storage Infrastructure, by clicking on the VMAX, as seen in Figure 7. Notice that
the VMAX2 shows the selected storage pool, but the VMAX3 does not. The only supported pool on VMAX3 is the def ault SRP. When
multiple storage groups are added, the Number of Devices is the sum of all storage groups.

VMAX2

VMAX3

Figure 7 - Storage Groups & Pools

MODIFYING STORAGE GROUPS

Once a storage group is modif ied on the array , such as through Unisphere for VMAX, the storage array within AppSync must be
rediscov ered, for example, when adding additional target v olumes to a storage group. To rediscover the VMAX storage group, nav igate
to Storage Infrastructure, click the VMAX array , then click Rediscover as seen in Figure 8.

Figure 8 - Rediscov ering a VMAX

Conf iguring/de-configuring storage groups is accomplished by clicking Manage Copy Storage as seen in Figure 9.

12
Figure 9 - Managing Copy Storage

STORAGE POOLS: ENABLING A VIRTUAL POOL FOR APPSYNC

AppSy nc utilizes new or existing storage pools on the VMAX2 array , where AppSy nc can prov ision and/or utilize existing devices for
copy targets. These target devices are used when creating TimeFinder VP Snap and Clone replicas.
Af ter AppSy nc chooses a specific pool for provisioning target devices for an application copy, all subsequent copies of that application
are prov isioned in the same pool until the pool consumption limit breaches the threshold value. Pool thresholds are configurable, and
are set at 90% by def ault, as seen in Figure 10. This means that when the pool consumption reaches 90%, AppSync will no longer use
that particular pool.

Figure 10 - Storage Pool Thresholds

Some of the key points to consider while enabling v irtual provisioning pools for VMAX2 array s are listed below:

If there is a pre-existing VP Snap session f or a particular application AppSync is protecting, then that pool must be added to
AppSy nc in order to prov ision new dev ices.

The Meta dev ice configurations must match, as when creating copies outside of AppSync, so the pool that is selected must also
match the RAID type of the source.

Ensure there is enough space in the pool to accommodate the number of copies. AppSync requires an additional dev ice (n+1) per
copy . If planning on creating one copy , for example, ensure there is enough space and dev ice count for two; for every source
dev ice. AppSy nc ensures there is always at least one copy available, alleviating the concern that a f ailure in production will occur
during the copy ref resh process, or the copy process does not complete successfully.

As seen in Figure 10, AppSync discovers all thin pools av ailable on the VMAX2 array , displays them in the wizard, and allows one or
more to be conf igured. If a pools consumed percentage is beyond the threshold limit, it is no longer used by AppSync for target
prov isioning, and the job will f ail if no other pool can be utilized (see key points abov e). Pools can be configured, adjusted, or deconf igured at any time, by using the option seen in Figure 9.
Should a storage pool need to be de-conf igured, an option to terminate the clone sessions AppSync created is presented, as seen in
Figure 11.

Figure 11 - De-conf iguring Storage Pools

13

DEVICE MANAGEMENT ON THE BACKEND VMAX

Dev ices provisioned by AppSync are named AppSync-{AppSync_Server_Name}. An example is AppSync-lrmp081, where lrmp081
is the AppSy nc serv er name. This can be used to identify all dev ices which have been prov isioned by AppSync as seen in Figure 12.
Do not delete dev ices on VMAX array s used by AppSync, unless those devices are no longer managed by AppSync. AppSync does not
delete dev ices which it has prov isioned in the pool when the pool is de-conf igured by default, however, it does provide an option to
terminate the clone session(s) created by AppSync, as seen in Figure 11. Retaining device pairing af ter de-configuring the pool is the
def ault option. If all dev ices provisioned by AppSync are to be deleted, query the VMAX sy stem for all dev ices by filtering the Volume
Identifier Name equals AppSync-{ServerName} where the ServerName is the name of AppSync server host name, as seen in
Figure 12. VMAX3 sy stems can be f iltered with an asterix (*), such as AppSync*, such as when multiple AppSy nc servers exist in the
env ironment. Please note that with VMAX3 and AppSy nc v ersions prior to 3.0.2, no hyphen exists between AppSync and the {Server
Name}, howev er, post AppSy nc 3.0.2, the hyphen is present as seen in Figure 13.

Figure 12- VMAX2 Volume Identif ier

Figure 13 - Pre-3.0.2 v s. Post 3.0.2 on VMAX3

REMOVING A VMAX ARRAY FROM APPSYNC

VMAX array s can be remov ed from AppSync only when no activ e copies exist for which AppSy nc created - all copies AppSync created
but f irst be expired. If storage groups and/or storage pools are configured, they too must be de-configured. Remove the VMAX by
nav igating to Settings > Storage Infrastructure, highlight the array and click Remove. If anything remains configured, an error will be
display ed and will prev ent the VMAX f rom being remov ed as seen in Figure 14.
If a VMAX array is remov ed from the SMI-S provider, before being removed from AppSync, AppSync removes the association between
the two. If there are no AppSy nc created VMAX copies, storage groups, or storage pools, then the VMAX array itself is also removed
upon rediscov ery.

14

Figure 14 - Remov ing VMAX Error

REMOVING AN SMI-S PROVIDER

Remov ing an SMI-S prov ider is accomplished by navigating to Storage Infrastructure, highlighting the VMAX array , selecting the SMIS prov ider host, and clicking Remove, as seen in Figure 15. If any VMAX array uses this particular SMI-S prov ider as a preferred
array , if an array is managed purely by this particular SMI-S provider and has v alid copies, storage, or pools configured, the removal
f ails with an appropriate error message.

Figure 15 - Remov ing SMI Prov ider

VMAX3 SNAPVX CONFIGURATION DETAILS

The VMAX3 storage array utilizes SnapVX, a new technology introduced which replaces the VMAX2 TimeFinder technology . SnapVX
eliminates the need f or the traditional dev ice management tasks, often associated with TimeFinder, such as configuring save pools,
creating target dev ices for pairing operations, and prov isioning tasks. Creating point-in-time copies utilize VMAXs Storage Resource
Pool (SRP), to maintain the changed tracks, which needs no conf iguration for AppSync to utilize. AppSync utilizes the default SRP out
of the box, so of fers a rapid setup. AppSync supports environments where multiple SRPs exist, but AppSync does not offer the ability to
utilize those other pools. For more details regarding the way SnapVX utilizes the SRP, please ref er to the f ollowing white paper http://www.emc.com/collateral/technical-documentation/h13697-emc-vmax3-local-replication.pdf.
AppSy nc prov ides two copy methods with TimeFinder on VMAX2, Clone and Snap. A clone is a 100% like-f or-like copy of the source
data, while the Snap is based on pointers, directing changed writes to a pre-conf igured virtualized storage pool. With SnapVX, this
nomenclature has changed. Like-f or-like target devices and pre-configured storage pools are no longer required. AppSy nc can utilize
pre-conf igured dev ices, added to a pre-configured storage group. These devices are only utilized during the mount phase, or during the
1st generation repurposing workf low.
AppSy nc prov ides support for both SnapVX Copy and NoCopy modes. Copy mode is similar to the way VMAX2 used TimeFinder
Clones, and the NoCopy mode is similar to TimeFinder Snaps.
When thinking about how TimeFinder works, there is a concept of create and activate, where create essentially pairs the device with a
pre-conf igured target dev ice, and activate creates the point-in-time copy, also readies the device; to be presented to a host. SnapVX
does not subscribe to the create, then activate nomenclature. With SnapVX, the point-in-time copy is established as a target-less
dev ice during the copy phase, only assigning a target device during link operations, for which AppSy nc initiates during the mount
phases. When utilizing the repurposing workf low, howev er, AppSync assigns a linked target device during the copy phase. This is
discussed in more detail in the Repurposing Workflow section. Please also see the SnapVX Linking section for more details.
15

SNAPVX LINKING
Target dev ices, as observed when working with VMAX2 and TimeFinder technology , are not necessary when creating copies with
serv ice plan workf lows during the create copy phase. The copy becomes linked to a target device only when accessed, such as when
mounted. If storage groups are configured with pre-configured, and av ailable, devices, AppSync uses those devices, otherwise, if
storage groups are not utilized, AppSy nc creates new target dev ices from the default SRP. When devices are unmounted, they are not
unlinked. Target dev ices are unlinked only when they are expired from within AppSync, though are not deleted on the array , even if
AppSy nc creates them.
st

Repurposing workf lows behav e somewhat differently than service plan workf lows. Repurposing the 1 generation copy creates a
snapshot of the dev ice, also linking to a device, regardless if that device is set to mount. This differs from a service plan copy phase in
st
that a target dev ice is required f or 1 generation copies. If storage groups are utilized, and no target dev ice is available, the
st
repurposing job f ails. Refreshing the 1 generation copy will create a new snapshot and relink to the same target dev ice. Mounting
will not perf orm the linking as would be the case when working with serv ice plans. Unmounting the device does not unlink the device,
expiring the dev ice does, just as a service plan. Service plans and Repurposing workflows are described in more detail, in the AppSync
User and Administration Guide.
nd

st

nd

The 2 generation copy is created from the 1 generation copy, though differs in behavior. The 2 generation copy will not link to a
st
target during the copy phase, as does the 1 generation copy. This process is more like service plan workf lows, in that target devices
are linked upon the need to access the data, such as through a mount operation. Refreshing the copy will create a new snapshot and
relink only if the snapshot had already been linked, such as if it had been mounted. Expiring the copy, again, will unlink the dev ice and
delete the snapshot.
Some f requently asked questions (FAQs) regarding SnapVX and Linking are below:
1.

Q: When AppSync mounts VMAX3 copies, it links them to target devices. Does it unlink the target device when un-mounting?
A: No, it does not unlink when un-mounting, only when expiring.

2.

Q: Does it not make more sense to have AppSync unlink when un-mounting?
A: This is a feature of SnapVX which AppSync utilizes to increase efficiency during the next mount operation of the same SnapVX snapshot. The
device maintains the same address in the masking view settings, specifically the port group, so behaves much more efficiently.

3.

Q: How is the repurposing relink feature utilized?

A: The relink use case use case with the repurposing workflows provides an incremental refresh process since the device is not unlinked till
expired. The refresh process is only incremental if the device remains linked, so the 1st generation refresh always links to a device, in order to allow
quicker refreshing and source for the 2nd generation copies.

4.

Q: Can Appsync use the same targets for the next mount operation?
A: Yes, when refreshing, AppSync does an unlink and relink internally, which is transparent to the end user.

PROTECTING APPLICATIONS USING APPSYNC SERVICE PLANS

AppSy nc is an application-centric copy management solution, with the approach of protecting dynamic application environments.
AppSy nc uses a concept of service plans, which provides a robust class of service for each of the application types supported. Each
application ty pe has three def ault tiered service plans:

A Bronze plan f or creating local copies (local array based copies and RecoverPoint local bookmarks)

A Silver plan f or creating remote copies (remote array based copies with SRDF integration or RecoverPoint remote bookmarks)

A Gold plan f or creating local and remote copies (Local and remote bookmarks gold plans do not support SRDF)

Applications can be subscribed to dif ferent service plans based on their required SLA agreements, and/or recovery point objectives
(RPO). Serv ice plans are composed of several phases which def ine the copy schedule/on-demand, copy technology type, copy
retention, application backup type, mount options, and any other options that may be required. The dif ferent phases are depicted in
Figure 16.

16

Figure 16 - Serv ice Plan Phases

Af ter registering and conf iguring VMAX storage, and af ter adding application hosts, subscribe the application to a service plan to create
and manage copies. Bronze and Silver plans are supported for applications residing on VMAX storage sy stems. Snapshot is the
def ault replication technology, howev er can be change. For instance, if a clone is desired, change the Storage Preference to where
Clone is on top as seen in Figure 17. AppSync 3.0.2 and greater, as seen in Figure 18, also offers the ability to de-select technologies.
Please ref er to the AppSync User and Administration Guide f or more details on Serv ice Plan settings.

Figure 17 - Storage Pref erence pre-3.0.2

Figure 18 - Storage Pref erence post-3.0.2

17

APPLICATION DISCOVERY PHASE

Bef ore creating the copy , AppSync must examine the application in order to account for changes, such as device expansion, addition,
or remov al, or if a change in status has occurred, such as whether the previous databases remain on online. This phase is not
conf igurable, nor can it be disabled.

APPLICATION MAPPING PHASE

Similar to the discov ery phase, this phase alway s runs, and is not user configurable. During the aapplication mapping, AppSync maps
the application components, for example, the databases and log f iles, to underlying file systems and physical disk objects. AppSync
then maps these f ile ssystems and disk objects to the storage array, i.e. LUNs. AppSync gather all the storage system device
inf ormation pertaining to these LUNs. If the underlying LUNs are on a VMAX sy stem, the following operations are perf ormed by
AppSy nc:

AppSy nc uses the SMI-S prov ider to map the world wide names (WWNs) of the source LUNs to a corresponding VMAX array to
obtain their details. These details include the ty pe of disk, RAID type, etc. This occurs every time a service plan is run, and is not
delectable, so if there are any changes to the device on the array , AppSync is updated of those changes, such as if the size is
changed.

SMI-S prov iders should be kept online, otherwise delay s may be experienced. If an SMI-S prov ider host is taken down f or
maintenance, f or example, it should be removed from within AppSync, or the pref erred list should be modified.

For Silv er serv ice plans, AppSync also tries to find the remote device, i.e.R2, after getting the R1 details from the SMI-S Prov ider.
Both local and remote array s must be added to AppSy nc.

A FFINITIZATION RULES
Once the mapping phase completes, AppSy nc affintizes the applications based on certain rules allowing group applications to be
replicated together, i.e. when multiple applications are subscribed to the same service plan. Affinitization rules are application-specific,
as well as storage-specif ic. For example, each application has rules specific to that application, eg. SQL, Oracle, Vmware, as well as
rules specif ic to a storage system on which the application data resides. In the case where the application data resides on a VMAX
storage sy stem, the following affintization rules apply:
o

Affinitize by VMAX array ID: This rule is based on the VMAX ID (serial Number). If two databases, datastores, etc., are
on the same host but are on two dif f erent VMAX sy stems, AppSync separates them into different point-in-time copies.

Affinitize by VMAX device type: For example, thin pool dev ice vs. thick (RAID devices). Multiple databases on the same
VMAX array spread across thin and thick LUNs are split based on the type of underlying device. All applications on thin
dev ices, for example, which are on the same host, will be grouped together as part of one point-in-time copy.

Affinitize by RA Group (only for silver plans using SRDF): As part of the silver service plan execution, all devices
belonging to the same RA group are protected as one point-in-time copy .

CREATE COPY PHASE

Just like the prev ious two phases, the copy phase must run, though is user configurable. During the create copy phase execution,
AppSy nc interacts with the array and application components to create application consistent, point-in-time copies of the applications
which were grouped together as one af f initization set. For each set of applications, one create copy phase is executed as part of a
single serv ice plan execution cycle. For example, if affinitizer results in 3 sets of applications as part of an affinitization process, three
create copy phases are executed during a single service plan execution cycle. Details pertaining to applications on VMAX f ollow:

AppSy nc supports TimeFinder VP Snap and TimeFinder Clone f or VMAX2 and SnapVX Snap and SnapVX Clone f or VMAX3 as
the supported copy technologies

TimeFinder Snap utilizing VDEVs are not supported on VMAX2, and instead, utilize VP Snaps.

By def ault all service plans have snapshot as the preferred storage technology type. If clones are desired, change the storage
order pref erence or deselect the other technologies, as seen in figures Figure 17and Figure 18.

For application data residing on a VMAX2 array s:

18

If the source storage devices are thick LUNs, AppSync always creates TimeFinder Clone copies even if snapshot is
higher in the storage order pref erence

If the source storage devices are thin LUNs, AppSync always creates VP Snap copies as part of the service plan run

If the application data resides on a mix of thin and thick LUNs, AppSync alway s creates clone copies since it cannot
create VP Snaps f or thick dev ices

A specif ic copy rotation value in service plan settings can be configured. The def ault value is seven, whereas the schedule is every
24 hours, by def ault, thus creating seven copies, or one weeks worth of copies by default. In the case of VMAX2 env ironments,
plan to ensure there are at least n+1 target dev ices for each source; where n is the rotation value. This extra set of dev ices is
required due to AppSy nc expiring copies only after completing the next copy. This ensures there is always one copy, in case the
new copy f ails, or a corruption/disaster occurs, during the creation of the new copy .
o

This extra dev ice is not required with VMAX3, since with VMAX3, AppSy nc re-links the device upon refreshing the
mounted v olume not needing a target dev ice during the copy phase.

AppSy nc alway s uses the differential copy option. This is done so that AppSync does not need to perform a full synchronization in
the ev ent that a clone copy is rotated (expired from AppSync and new clone created at dif ferential point-in-time).

VMAX2 allows a maximum of eight clone copies per source device. VP Snap restores require one differential TF clone session.
Since AppSy nc creates a differential clone copy and also supports VP Snap copies for same source device, it is alway s advisable
to limit the number of clone copies to a maximum of six in the create copy phase rotation value setting.

When considering creating remote copies, such as taking snaps and clones off of the R2:
o

AppSy nc does not manage the SRDF sessions.

AppSy nc only supports SRDF/S and SRDF/A please refer to the support matrix for the current list of what
applications are supported f or each technology, e.g. supporting SRDF/A with Oracle is currently the only supported
application with SRDF/A.

For SRDF/A the link state should be in Consistent or Sy nchronized f or replications to work properly. For SRDF/S,
link state should be Sy nchronized.

Use the Silver serv ice plan to create copies off the R2. AppSync does not support Gold service plans for SRDF at
the time of publication.

Creating copies of f R2 with asynchronous mode (SRDF/A), the following should be enabled:

Dev ice lev el write pacing should be enabled please ref er to the SRDF product guide f or more details

All the dev ices which constitute a single application should belong to the same RA Group

TA RGET DEVICE SELECTION & PROVISIONING ON VMA X2*

Bef ore creating VMAX2 copies, AppSy nc needs to pair each of the source devices on which application data resides to a corresponding
target dev ice. In case AppSync doesnt find any suitable devices to pair, it is capable of provisioning target devices in one of the
suitable storage pools conf igured for AppSync (see the Storage Configuration section for more details). AppSync follows a set of rules
f or pairing source and target dev ices for VP Snap and TF clone copies, as well as prov isioning a target device in storage pools. The
rules f or creating VP Snap copies and TF clone copies are slightly different. Listed below are f ew of the rules captured while pairing
source and target dev ices:
1.

If the source device has pre-existing session with target devices, and are configured for AppSync, those devices receive the highest priority to be
used by AppSync

2.

In case the previous rule does not yield a target device, AppSync queries the SMI-S provider in order to find information about the virtual
provisioned pool VP Snap copies. Appsync uses this pool information to get a suitable virtual provisioned pool from the list of all enabled thin
pools. AppSync stores this suitable pool information in memory so that if no suitable target devices are found, AppSync can provision new targets
for the source LUNs.

3.

When AppSync queries its database, the devices are prioritized by the following specifications:

19

3.1 All thin dev ices matching size requirements

3.2 No thin dev ice in any VP Snap pool
3.3 No thick dev ices in any clone session. These devices get the next highest priority.
4.

If AppSync does not find target devices, and there are no valid pools configured, AppSync considers target devices which had a pre-existing
session, but is no longer being used for any copy session. It terminates the earlier session and re-uses the targets for the new copy.

5.

If a source LUN is a META LUN, AppSync searches for target META LUNs matching the same size as the source and same type as source META
i.e. stripped or /concatenated. In a situation where the target meta size and type matches, but the number of members are different, AppSync does
not consider that target as eligible.

6.

While provisioning META targets, AppSync requires a virtual provisioned pool protection level i.e. RAID type, that matches the RAID type of source
meta device. For example, if the source META LUN is at RAID-5 device, AppSync requires a thin provisioned pool with RAID-5 to provision a target
META LUN for this source LUN.

7.

If both storage group and storage pool for target and source is a META thin LUN, make sure META targets match the RAID level of META source
LUNs in the configured storage group, otherwise AppSync may not be able pick an appropriate pool for provisioning new META targets (since it
matches the RAID level wile selecting the pool). This restriction is not applicable if pools are not configured for provisioning new META LUNs.

*NOTE: For VMAX3 target device selection and provisioning, see the Storage Configuration and VMAX3 SnapVX sections.

CONFIGURE CUSTOMIZED STORA GE POOL OPTIONS FOR VMA X2 COPIES

AppSy nc of fers the ability to customize which storage pools to use when prov isioning target devices for VMAX2 env ironments, on a
per-serv ice plan basis. VMAX3 has the one def ault pool (SRP), so does not of fer customization. The storage pool(s) are selectable per
serv ice plan, rather than by environment, offering more control over which pools to use f or each service plan configured. This is to say,
when conf iguring the VMAX2 array s, within AppSy nc, add the storage pool to AppSync, to create target devices, and further customize
which serv ice plans use a particular pool(s), as seen in Figure 19.

Figure 19 - VMAX2 Storage Pool Options

SPECIA L HA NDLING FOR ORA CLE A PPLICA TIONS

AppSy nc splits Oracle copies into two distinct point-in-time copies. The f irst point-in-time copy is created for all the database files
(control f iles, redo logs and data f iles), and is created while the database is in hot-backup mode. The second point-in-time copy is
created f or the archiv e log volumes, and optionally for the Fast Recovery Area. This copy is made after Oracle has been taken out of
hot-backup mode, and only when the f irst point-in-time copy phase is completed. Since VMAX clones can take a longer time to
complete (i.e. all tracks must synchronize), after the first point in time copy is activated, a second point-in-time copy can take a
signif icantly longer time to after the first point-in-time copy. To av oid this, AppSync efficiently completes one f ull clone synchronization
cy cle between the source and target dev ices, before triggering the first point-in-time copy. Since all tracks are completely synchronized
20

bef ore activ ating the first point-in-time copy, the second point-in-time copy doesnt need to wait, as it is just an update of changed
tracks.

EXPIRING COPIES
Expiring copies with AppSy nc, on a VMAX array , does not delete the dev ices on the array , even if provisioned by AppSync. The
AppSy nc expire command does not equate to symsnap, symclone, or symsnapvx terminate. AppSync will re-use the target devices
f or the next snapshot, or clone, of the same source device. This reduces the time it takes to provision devices as targets. Optionally run
SY MCLI commands to validate, for example symdev show {target ID}, symsnap list, symclone list, or symsnapvx list.

MOUNT COPY PHASE

AppSy nc supports dynamic mounting of application copies created on VMAX f or all AppSy nc-supported platform types, i.e. Linux, AIX,
and Windows. AppSy nc doesn't support static mounting of VMAX copies except through using RecoverPoint. AppSync relies on
VMAXs auto prov isioning capability to provision devices to mount hosts. AppSync requires the mount host to be zoned to the VMAX
array and should hav e a masking v iew with the appropriate initiator, port, and storage group, in anticipation for mount operations.
To perf orm dy namic mounts of VMAX copies, AppSy nc first finds the mount hosts FC/iSCSI adapter information and then interacts with
VMAX v ia the SMI-S prov ider to f ind an appropriate storage group. Once AppSync finds the appropriate storage group, the target copy
dev ices are added as shown in Figure 20.

Figure 20 - Mounting Wizard

MOUNT HOST STORA GE GROUP RULES

1.

For virtual machines, AppSync performs mounts by masking the copies to the ESX host, and then adds the devices to the virtual machine.

2.

In case the host is connected to VMAX via multiple masking views, and the host is not an ESX host in a cluster (for virtual or RDM mounts),
AppSync gives first preference to a masking view dedicated for that host. For example, the initiator group connected to the masking view has only
initiators for that host). In this scenario AppSync adds the target devices to that storage group; connected to the dedicated masking view.

21

3.

If AppSync doesn't find a dedicated storage group, it creates a list of storage groups with all storage groups attached to each of the masking devices
that are connected to the host and then picks up the first storage group. If the selected storage group is dedicated for GK devices (i.e. has only
gatekeeper devices), AppSync picks up the next storage group in a sorted list. If no other storage group exists except the one with only GK devices,
AppSync selects that storage group.

4.

For an ESX host in a cluster, AppSync tries to find a masking view which has connectivity to the maximum number of nodes of the ESX cluster. If it
finds such a common masking view, it uses the storage group to add the devices during the mount phase. In case any ESX nodes in cluster are not
connected to this common masking view, AppSync searches for a masking view for that node as outlined in step 2 and 3. If no storage group for
any node of the cluster is found, the mount operation fails with appropriate exception.

5.

When the selected masking view is connected to a cascaded storage group i.e. storage group containing a list of other storage groups, AppSync
sorts the storage group by its name and picks up the first storage group checking the storage group is not dedicated for GK devices. If the selected
storage group is dedicated for GK devices, AppSync picks the next storage group in sorted list.

6.

When working with SQL cluster mounts, if the use dedicatedStorageGroup option is selected (selected by default), and if AppSync is unable to
find any dedicated storage group for the mount host, an exception is displayed and the mount fails.

7.

During unmount, AppSync finds the storage group using the same rules as described previously and removes the devices from the storage group
as shown in Figure 21.

Figure 21 - Unmounting Copies

22

MOUNTING WITH FA ST VP ON VMA X2

AppSy nc 3.0 and greater of fers the ability to mount to a storage group associated with a particular FAST VP policy . For VMAX V2
env ironments, select the desired FAST VP policy within the service plans Mount Copy settings, or during an on-demand mount
operation. Each FAST VP policy is associated with a particular storage group. AppSync will add the target dev ices to this particular
storage group, and if no storage group exists, then AppSync will add the dev ices to any storage group associated with the particular
mount host selected. Please note that target copies cannot move between different storage tiers, so ensure that the storage pool of the
copy dev ices is associated the same storage pool type (sharing the same FAST VP policies).

MOUNTING WITH THE DESIRED SERVICE LEVEL OBJECTIVE ON VMA X3

AppSy nc of fers the ability to choose the desired Service Level Objective (SLO) when mounting target devices. AppSync will add the
target dev ices to a storage group associated with the desired SLO, if configured within AppSy nc. If no storage group exists that match
the SLO selection, then the dev ices are added to any storage group associated with the mount host selected.

REPURPOSING WORKFLOWS
Repurposing workf lows are v ery similar to the service plan workflows, in that the same phases are followed, as seen in Figure 16.
There are dif f erences, howev er, such as the repurposing workflow allows taking multi-generation copies, and also, offers the ability to
take array based copies off of a RecoverPoint bookmark (placing RecoverPoint bookmarks in image access mode as a temporary
st
placeholder in order to create an array based copy ). The repurposing workf low offers a 1 generation copy which can be used f or
nd
restore purposes, since it can be used as a gold copy, and also serves as a source for multiple 2 generation copies. Both
st
generation copies can be mounted and written to, but only the 1 can be restored to production.
st

nd

1 Generation Copies: Can be used f or restore purposes and serves as the same source for multiple 2 generation copies

2 Generation Copies: Cannot be used f or restore purposes, and is created from a 1 generation copy (e.g. snap-of-snap)

nd

st

Repurposing also dif f ers in the way target devices are managed in a VMAX3 env ironment. When working with VMAX3 array s and
serv ice plan workf lows, devices are only required when linking, as occurs when the dev ices are mounted. With the repurposing
st
nd
workf low, howev er, the 1 generation copy automatically links to devices, regardless if the devices are mounted. The 2 generation
copy shares the same rules as service plan copies, where linking only occurs during the mount operation. With VMAX2, howev er, the
repurposing workf low behav es the same as the service plan workf low, as target devices are alway s required. With this knowledge,
ensure that there are either target dev ices available in the storage group AppSy nc is configured with, or no storage group is configured,
allowing AppSy nc to create dev ices as necessary.
st

Lastly , repurposing 1 generation copies do not require the additional target devices, as do service plans. Using Service Plans with
VMAX2 and TimeFinder, one additional target dev ice per source is required. The prev ious copy is expired only after the new copy is
f ully complete. This means that VMAX2 TimeFinder serv ice plans require an N+1 copy count. That is to say, AppSync requires one
additional target dev ice f or every source device, for every service plan job. For example, a SQL database is set to a copy expiration
count of sev en (default expiration count), so when working with VMAX2 and TimeFinder, there needs to be eight copy targets available
in a storage group, or av ailable f or AppSy nc to create. With the repurposing workflow, however, AppSync does not require the N+1
rotation count, as it will reuse the same target dev ices. When working with VMAX3 SnapVX copies, repurposing workf lows are similar
to VMAX2, in that only one dev ice is required, and does not require the N+1 copy count.

RULES FOR REPURPOSING WORKFLOWS WITH SNA PVX (LINKING/RE-LINKING)

Repurposing gen 1:
o

Create copy phase will create the copy of the device and link it to a target

Refresh will create a new copy and relink to the same target dev ice

Mount will not do the linking, as the copy is already linked

Unmount will not unlink the target dev ice, and only unmounts from the host (speeds up re-mounting)

Expire will unlink the target dev ice and delete the copy (any target device created remains on the array)

Repurposing gen 2:
o

Create copy phase will create the copy , howev er, will not be linked to a target dev ice
23

When the copy needs to be accessed, i.e. mounted, it is linked to a target device, then mounted

Refresh will create a new copy , and relinked if the original copy was already linked to a target device

Expire will unlink the target dev ice, if linked, and deleted (any target device created remains on the array )

REPURPOSING USE CA SE
The f ollowing depicts a common repurposing workf low use case. A database developing group wishes to create multiple copies of
production. Users require a point-in-time copy to serves as the baseline copy, the gold copy, for the development groups testing.
st
Since multiple users need the same PIT copy , they must use a common sourced copy, from the same PIT. This is the 1 generation
copy , also commonly referred to as a gold copy . This copy, by default, is application consistent.
nd

Each user has the ability to create 2 generation copies, presenting them to different hosts. Each copy is autonomous, and can be
st
ref reshed upon demand, or schedule. The 1 generation copy can be ref reshed as well, independently, thus providing a div erse work
st
env ironment. The 1 copy , being that it is not mounted or changed, can also serve as a restorable copy.

1ST GENERA TION REPURPOSING WORKFLOW PROCESS FLOW

st

The f ollowing expert of the process log depicts a common flow for creating a 1 generation copy, for the first time, with a mounting
option.

Phase
Master Phase
Application discovery
Application discovery
Application mapping
Application mapping
Create 1st gen copy
Create 1st gen copy
Create 1st gen copy
Create 1st gen copy
Create 1st gen copy
Create 1st gen copy
Create 1st gen copy
Create 1st gen copy
Create 1st gen copy
Create 1st gen copy
Create 1st gen copy
Create 1st gen copy
Create 1st gen copy
Unmount previous
copy
Unmount previous
copy
Mount copy
Mount copy

Message
Beginning execution of service plan
Application discovery phase beginning
Application discovery phase completed successfully
Application mapping phase beginning
Application mapping phase completed successfully
Create 1st gen copy phase beginning
Skipping unmount because there were no previously mounted copies found for
the applications under protection during this cycle.
Attempt number 1 to create a VSS copy of the application.
Starting backup of SQL Server databases.
VSS application freeze succeeded on host amssqlp1.
Backup of database completed successfully.
VSS application thaw succeeded on host amssqlp1.
The SnapVX snapshot for the source device 003B8 on the array is
EMC_SYNC_ASPECT1474905607970.
The SnapVX snapshot for the source device 003B9 on the array is
EMC_SYNC_ASPECT1474905607970.
Refreshing the SnapVX replication relationships in storage array.
The SnapVX snapshot EMC_SYNC_ASPECT1474905607970 for the source device
003B8 on the array is linked in nocopy mode with target device 003BC.
The SnapVX snapshot EMC_SYNC_ASPECT1474905607970 for the source device
003B9 on the array is linked in nocopy mode with target device 003BD.
Create 1st gen copy phase completed successfully
Skipping unmount because there were no previously mounted copies found for
the applications under protection during this cycle.
Unmount previous copy phase for applications completed successfully
Mount copy phase beginning
Beginning mount on mount host
24

Mount copy
Mount copy
Mount copy
Mount copy
Mount copy
Mount copy
Mount copy
Mount copy
Mount copy
Mount copy
Mount copy
Mount copy
Mount copy
Mount copy
Mount copy
Mount copy
Mount copy
Mount copy
Mount copy
Recover copy
Recover copy
Recover copy

Mounting file systems on mount host

Making replication storage accessible to the mount host
Adding copy device 003BD,003BC in storage group on VMAX array .
Rescan HBA and datastore refresh on ESX
All the target devices are visible on ESX
Started to mount copy of datastore
All the target devices are visible on ESX
Mounting the VMFS on ESX
Cluster mount is enabled
Rescan HBA and datastore refresh on ESX
Copy of datastore is mounted a on host
Beginning hot add of virtual disk on virtual machine
Successfully added virtual disk on virtual machine
Mounting file systems in read-write mode on mount host
VSS volume import and filesystem mount succeeded on host
SQL metadata file saved
Saved the VSS backup components document file
Copy mount completed successfully on mount host
Mount copy phase completed successfully
Recover copy phase beginning
Beginning recovery of databases in mode recovery.
Recover copy phase completed successfully

25

RESTORE FROM APPLICATION COPIES

LUN LEVEL RESTORES FROM VP SNAP AND TF CLONE COPIES
AppSy nc supports restoring copies created using the bronze lev el plan (i.e. local copies only). LUN restores from remote copies (silver
plan) is not supported using SRDF, though are with RecoverPoint. Review the following considerations for a LUN level restore:
1.

Since AppSync supports TimeFinder VP Snap (thin pools), not TimeFinder Snap (using VDEVs), while restoring a VP Snap snapshot, VMAX
consumes one TimeFinder clone session to perform the restore operation. It is very important to ensure AppSync is configured to make no more
than six copies of the same application, when also creating snaps, due to this restriction.

2.

Once the restore completes, AppSync splits the pair in order to perform a restore from the same copy again. AppSync can also refresh the same
copy (in case it is expired from AppSync as part of rotation), performing a differential copy during next service plan run.

3.

At the end of restore from VP Snap copies, VMAX creates one additional session in "RESTORED" state for the same source-target pair. AppSync
terminates this RESTORED session at the end of restore, so that the same copy can be used again for restore as well as restore from other copy. If
the session is left in the RESTORED state, it cannot restore from other copies.

VM LEVEL RESTORE FROM VP SNAP AND TF CLONE COPIES

VM RESTORE FROM VP SNA P COPIES
Instant restore option is not av ailable for restore from VP Snap copies since AppSync cannot use storage migration techniques which
are required f or perf orming instant restore. To perf orm storage migration, AppSync requires creating another copy from the current
copy (not allowed f or VP Snap copies) since VMAX does not allow a snap-of -VP Snap copy . Due to this restriction, AppSync mounts
the VP Snap datastore copy , registers the VMs, and then clones the snapshot datastore to the restore location.

VM RESTORE FROM TF CLONE COPIES

Depending on the conf iguration of source and clone devices involved in the TF Clone copy operation, AppSync can perform instant
restore options in certain cases. If all the source devices and clone devices of the datastores are thin devices, AppSync can create a
VP Snap of a clone and use it to mount the datastore on the f ly and perf orm live storage migration. This enables instant restore of VM.

VMAX3 RESTORES WITH SNAPVX

SnapVX copies are restored and then the restore session is terminated. If a linked target is used, that modified data is not restored, and
a restore does not af fect the point-in-time copy on the linked target device.

TROUBLESHOTTING TIPS
AppSy nc uses SMI-S prov ider to manage VMAX storage sy stem and many times failure of an AppSync service plan and other
operations related to VMAX array s are due to SMI-S prov ider issues. In such cases, it is always recommended to collect log files
related to SMI-S prov ider. Below is a list of log f iles which need to be collected from SMI-S provider host :
1.

Collect all log files:

1.1 On Windows in C:\Program Files\EMC\SYMAPI\log and C:\Program Files\EMC\ECIM\ECOM\log

1.2 On Linux in /var/symapi/log and /opt/emc/ECIM/ECOM/log
2.

Collet all dump files:

2.1 On Windows in C:\Program Files\EMC\ECIM\ECOM\Providers

2.2 On Linux in /opt/emc/ECIM/ECOM/bin
3.

Collect the Solutions Enabler symapi_db.bin database file:

3.1 On Windows in C:\Program Files\EMC\SYMAPI\db

3.2 On Linux in /var/symapi/db
There are times when enabling debug mode with the SMI-S prov ider and Solutions Enabler may be required to troubleshoot. Turning
the debugging mode on/of f is depicted below, and should only be necessary upon request by Dell EMC support. Once changed, the
logs can be collected again.
26

STEPS TO TURN ON SMI-S PROVIDER DEBUG LOG

1.

Stop ecom

1.1 On Windows C:\Program Files\emc\ECIM\ECOM\bin\sm_service stop ecom.exe

1.2 On Linux use kill -9 to stop ecom.
2.

Stop the Solutions Enabler daemons

2.1 On Windows C:\Program Files\EMC\SYMCLI\bin\stordaemon shutdown all

2.2 On Linux /opt/emc/usr/symcli/bin/stordaemon shutdown all
3.

Cleanup all files

3.1 On Windows in C:\Program Files\EMC\SYMAPI\log and C:\Program Files\EMC\ECIM\ECOM\log

3.2 On Linux in /var/symapi/log and /opt/emc/ECIM/ECOM/log
4.

Edit the file oslsporvider.conf and change the following:

4.1 On Windows in C:\Program Files\EMC\ECIM\ECOM\providers

4.2 On Linux in /opt/emc/ECIM/ECOM/providers
4.3 Change
*/com.emc.cmp.ofl.log.Control.severity.id = INFO
-To*/com.emc.cmp.ofl.log.Control.severity.id = DEBUG
5.

Start ecom on windows c:\program files\emc\ecim\ecom\bin\sm_ser vice start ecom.exe on Linux from the directory /opt/emc/ecim/ecom/bin
./ECOM d

6.

Wait for ecom to fully start use the testsmiporvider dv command to verify ecom is fully started.

7.

Reproduce the issue and perform steps 0-4 above.

STEPS TO TURN ON SOLUTIONS ENA BLER DEBUGGING

1.

Instead of launching ecom as a service on Windows or a background process on Linux , run it from the command line to collect Solutions Enabler
traces. To do that, open a command prompt on Windows or Linux.

2.

On Windows: c:\program files\emc\ecim\ecom\bin or on Linux: /opt/emc/ECIM/ECOM/bin

3.

Set the following two environment variables:

SYMAPI_DEBUG=-1
SYMAPI_DEBUG=C:\trace.txt

4.

Re-start ecom from the command line on Windows by typing ecom and hitting enter, or on Linux, type in ./ECOM and hit enter.

5.

Wait for ecom to fully start, and use the testsmiporvider dv command to verify ecom is fully started.

6.

Reproduce the issue and perform collection steps outlined previously

LIST OF A PPSYNC ERROR A ND TROUBLESHOOTING STEPS

1.

Issue: A copy fails after one of the devices provisioned by AppSync was deleted outside of AppSync
Troubleshooting steps: Never delete devices which are being used by AppSync as targets.

2.

Issue: Discovery of VMAX systems using the SMI-S Provider hangs or is very slow
Troubleshooting steps: Run discovery outside of AppSync using TestSmisProvider.exe to determine if it is also behaving in a similar way.
Check if there are arrays which the SMI-S provider is discovering, that are not correctly configured on host, do not have the necessary gatekeepers,
or redundant to the SMI-S host. In such case, remove those arrays from automatic discovery using SYMAVOID file as described in Procedure to
prevent Automatic Discovery of Arrays section of this whitepaper.

3.

Issue: Replication fails if the SRDF link is not in a consistent state

27

Troubleshooting steps: From the symdev show cli command, verify that the link is not in the consistent state. Per the AppSync User and
Administration Guide, ensure the R1>R2 link state is Synchronized for SRDF/S or Consistent for SRDF/A.
4.

Issue: Clone replication fails with error pointing to exceeding number of sessions
Troubleshooting steps: TimeFinder is limited to eight differential pairing sessions. AppSync utilizes differential sessions, and offers no more than
seven copies, since one session must be reserved for restoring VP Snap sessions. In addition, since an extra copy is needed for rotation (N+1)
AppSync does not delete the prior copy till the next copy is complete, the maximum number of clone copies should be set to six. Please refer to the
Copy session limits section in the AppSync User and Administration Guide, for more details.

5.

Issue: Service Plan is configured with an expiration value of 1 but AppSync is creating two clone sessions
Troubleshooting steps: AppSync always maintains two clone sessions minimum, due to the fact that AppSync ensures a copy is fully created
before expiring the previous copy. This method is considered an N+1 copy count, and cannot be changed.

6.

Issue: Copy creation fails with strange SMI-S provider errors

Troubleshooting steps: Validate the SMI-S version and Enginuity/microcode level against the latest ESM -

https://elabnavigator.emc.com/eln/modernHomeDataProtection
7.

Issue: Progress appears to be HUNG, repeating the same percentage value

Troubleshooting steps: Ensure the AppSync UI session is not timed out, and ensure the virtual provisioned storage pool is configured and not full.

8.

Issue: AppSync does not delete devices which it has provisioned

Troubleshooting steps: Do not delete devices provisioned by AppSync until the AppSync server is de-configured. Once the AppSync server is deconfigured and uninstalled, query the VMAX to find all devices per the Device Management On The Backend VMAX section, and remove these
devices. AppSync uses the naming convention while provisioning devices to make this process easy.

CONCLUSION
In conclusion, this whitepaper explains key information and concepts when protecting an application residing on VMAX storage using
AppSync. It discusses the inner operations of AppSync software while protecting applications on VMAX2 using TimeFinder VP Snap
and Clone, and VMAX3 using SnapVX Snap and Clone copies along with deployment and troubleshooting tips. The main differences
between deploying in a VMAX2 and VMAX3 environment is also identified.

28