Вы находитесь на странице: 1из 332

RecoverPoint

Version 5.0

Administrator's Guide
P/N 302-003-386
REV 01

Copyright 2016 EMC Corporation. All rights reserved. Published in the USA.
Published October 2016
EMC believes the information in this publication is accurate as of its publication date. The information is subject to change
without notice.
The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with
respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a
particular purpose. Use, copying, and distribution of any EMC software described in this publication requires an applicable
software license.
EMC, EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and other
countries. All other trademarks used herein are the property of their respective owners.
For the most up-to-date regulatory document for your product line, go to EMC Online Support (https://support.emc.com).
EMC Corporation
Hopkinton, Massachusetts 01748-9103
1-508-435-1000 In North America 1-866-464-7381
www.EMC.com

RecoverPoint 5.0 Administrator's Guide

CONTENTS

Figures

Tables

11

Chapter 1

Getting Started

13

The RecoverPoint interfaces.......................................................................... 14


Logging into RecoverPoint............................................................................. 14
The Getting Started wizard............................................................................ 15
Add Licenses screen........................................................................ 15
Enable Support screen..................................................................... 16
Register RecoverPoint screen........................................................... 17
The Unisphere for RecoverPoint interface...................................................... 17
The RecoverPoint workflow............................................................................18
First steps in RecoverPoint............................................................... 19
Configuring replication.....................................................................19
Performing recovery activities...........................................................19
Monitoring and managing RecoverPoint........................................... 19
System analysis............................................................................... 20
RecoverPoint support....................................................................... 20

Chapter 2

Dashboard

21

The RecoverPoint Dashboard.........................................................................22


Overall Health............................................................................................... 23
RPA Clusters Map............................................................................. 23
Multipath monitoring....................................................................... 24
Alerts............................................................................................... 25
System Traffic.................................................................................. 26
Consistency Group Transfer Status................................................... 26
System Limits................................................................................................28
System.............................................................................................28
Groups.............................................................................................29
Storage............................................................................................ 29
Splitters........................................................................................... 29
System Events...............................................................................................30
Display events................................................................................. 30
Display additional event information................................................31
Sort events.......................................................................................31
Filter events..................................................................................... 31
Copy an event to the clipboard.........................................................32
Collect system information around a specific event.......................... 33

Chapter 3

Protection

35

RecoverPoint Protection................................................................................ 36
Protect Volumes............................................................................................ 37
How to protect volumes....................................................................37
Policy Templates........................................................................................... 52
RecoverPoint 5.0 Administrator's Guide

CONTENTS

Display all policy templates..............................................................52


Create new policy templates............................................................ 53
Modify existing policy templates...................................................... 53
Remove a policy template................................................................ 54
Apply a policy template to a new consistency group......................... 54
Apply a policy template to an existing consistency group................. 55
Group Sets.................................................................................................... 56
How group sets work........................................................................56
Display all group sets.......................................................................57
Create a group set............................................................................ 57
Edit a group set................................................................................ 58
Remove a group set..........................................................................59
Enable or disable all groups in a group set....................................... 60
Pause or start transfer for all groups in a group set........................... 60
Apply a bookmark to all of the groups in a group set........................ 60
Initiate recovery activities for a group set......................................... 60
Manage Protection........................................................................................ 61
The RecoverPoint protection policies................................................63
Monitoring and managing protection................................................73
Monitoring and managing groups.....................................................73
Monitoring and managing copies..................................................... 87
Modifying consistency groups.......................................................... 97
Modifying copies............................................................................112
Modifying links.............................................................................. 114
Modifying replication sets.............................................................. 115
Modifying journals......................................................................... 118
Modifying snapshots......................................................................120

Chapter 4

Recovery

127

RecoverPoint Recovery................................................................................ 128


Test a Copy................................................................................................. 128
How to test a copy..........................................................................128
Test a Copy and Recover Production............................................................ 132
How to recover production............................................................. 132
Test a Copy and Fail Over............................................................................ 135
How to fail over.............................................................................. 136
Manage Recovery........................................................................................ 139
Testing a copy................................................................................ 140
Failing over.................................................................................... 140
Recovering production................................................................... 142

Chapter 5

RPA Clusters

143

RecoverPoint clusters.................................................................................. 144


General Cluster Info.....................................................................................144
Storage....................................................................................................... 147
Display registered storage..............................................................148
Register storage............................................................................. 149
Unregister storage..........................................................................154
Edit storage registration details......................................................155
Display registered resource pools.................................................. 157
Register resource pools..................................................................158
Unregister resource pools.............................................................. 159
Splitters...................................................................................................... 160
Display splitter information............................................................ 160
4

RecoverPoint 5.0 Administrator's Guide

CONTENTS

Display volume information........................................................... 161


Add splitters.................................................................................. 162
Remove splitters............................................................................ 164
Attach volumes to the splitter........................................................ 164
Detach volumes from the splitter....................................................165
RPAs........................................................................................................... 166
Properties...................................................................................... 167
Status............................................................................................ 168
Interface........................................................................................ 169
Traffic Statistics............................................................................. 169
Usage Statistics............................................................................. 171
vCenter Servers........................................................................................... 173
Before you begin............................................................................ 173
Display vCenter Server information................................................ 174
Register vCenter Server.................................................................. 174
Edit vCenter Server Registration..................................................... 175
Remove vCenter Server.................................................................. 175
Rescan vCenter Server....................................................................176

Chapter 6

Admin

177

RecoverPoint Administration....................................................................... 178


Manage Licenses........................................................................................ 178
Install a license..............................................................................179
Remove licenses............................................................................ 180
Monitor your license limits............................................................. 180
Export a detailed license report......................................................180
Display your RecoverPoint license information............................... 181
Register RecoverPoint..................................................................................182
Register your RecoverPoint system................................................. 182
Export the post-deployment form to a CSV file................................186
Registering RecoverPoint by email or phone................................... 186
Users and Security...................................................................................... 187
Manage Users................................................................................ 187
Manage Roles................................................................................ 191
Manage User Authentication.......................................................... 195
Change Password.......................................................................... 198
Switch User....................................................................................199
Change Security Level.................................................................... 199
Change Idle Logout Time................................................................ 200
System Analysis Tools................................................................................. 200
Detect Bottlenecks......................................................................... 201
Balance Load................................................................................. 204
Collect System Information............................................................ 209
System Notifications................................................................................... 215
Manage Event Filters...................................................................... 215
Configure System Reports and Alerts..............................................226
Manage Call Home Events.............................................................. 230

Chapter 7

Support

235

RecoverPoint Support..................................................................................236
RecoverPoint Documentation......................................................... 236
EMC Online Support....................................................................... 237
The RecoverPoint Community......................................................... 238
RecoverPoint 5.0 Administrator's Guide

CONTENTS

Chapter 8

Troubleshooting

239

Licensing RecoverPoint............................................................................... 240


Before you begin............................................................................ 240
License RecoverPoint..................................................................... 240
Upgrade from RecoverPoint 4.0 or later.......................................... 247
Upgrade from RecoverPoint versions prior to 4.0............................ 247
Get licensing support..................................................................... 250
Notification of new RecoverPoint software releases..................................... 250
The Upgrade Recommendation dialog box..................................... 250
What should I do when I see the Upgrade Recommendation dialog
box?...............................................................................................251
Deciding whether or not you want to upgrade.................................251
Upgrading your RecoverPoint environment..................................... 252
Set the time display of Unisphere for RecoverPoint......................................252
Change the display language of Unisphere for RecoverPoint........................253
First-time initialization is taking too long..................................................... 253
How does it work............................................................................253
Initialize from backup.................................................................... 254
Protecting your data during initialization..................................................... 255
Host applications are hanging.....................................................................256
When does application regulation happen?....................................256
How does application regulation work?.......................................... 256
How do I know application regulation is happening?...................... 256
What can I do to stop my group from being regulated?................... 256
One-phase distribution workarounds.......................................................... 256
Copy is being regulated...............................................................................257
When does control action regulation happen?................................257
How do I know control action regulation is happening?.................. 257
How does control action regulation work?...................................... 257
How do I release a copy from control action regulation?................. 257
How do I verify that regulation is over?........................................... 257
Copy is experiencing a high load................................................................. 258
How do I know a copy is experiencing a high load?.........................258
What is a permanent high load?..................................................... 258
When do permanent high loads occur?...........................................258
How do permanent high loads work?..............................................258
How can I tell a copy is under permanent high load?...................... 258
What can I do to come out of permanent high load?....................... 259
How do I verify that a permanent high load is over?........................ 259
What is a temporary high load?...................................................... 259
When do temporary high loads occur?............................................259
How do temporary high loads work?...............................................259
How can I tell a copy is under temporary high load?....................... 260
What should I know about temporary high loads?.......................... 260
How do I verify that a temporary high load is over?......................... 260
RPA keeps rebooting................................................................................... 260
When does reboot regulation happen?........................................... 261
How does reboot regulation work?................................................. 261
How do I know reboot regulation is happening?............................. 261
What should I do to stop reboot regulation?................................... 261
Host applications crash during initialization................................................261
Storage read-rate during initialization............................................ 261
How does I/O throttling work?........................................................ 262
When should I enable I/O throttling?..............................................262
How do I enable I/O throttling?.......................................................262
6

RecoverPoint 5.0 Administrator's Guide

CONTENTS

How do I check how throttling affects my system performance?......264


How can I tell whether or not I/O throttling is enabled?.................. 264
Add more LUNs than allowed per VNX storage group................................... 264
How do I know I need more VNX LUNs?...........................................264
What can I do if I need more VNX LUNs?......................................... 264
How can I verify that the storage groups were added successfully?.265
Swap the LUN numbers of journal volumes..................................................265
Find your array serial numbers.....................................................................266
Find the serial numbers of VNX arrays............................................ 266
Find the serial numbers of Symmetrix arrays.................................. 266
Find the serial numbers of VPLEX arrays......................................... 267
Find your RPA cluster IDs............................................................................. 267
Find your XtremIO cluster IDs.......................................................................267
Find your XtremIO cluster IDs (PSNTs) in XtremIO............................268
Find your XtremIO cluster IDs (PSNTs) in RecoverPoint....................268
Understanding the RecoverPoint license..................................................... 268
Calculate your required journal size............................................................ 271
Minimum journal size with snapshot consolidation........................272
Validate replication integrity....................................................................... 278
How can replication integrity be compromised?..............................278
When should I validate replication integrity?.................................. 278
How the integrity check tool works................................................. 279
How the periodic integrity validation tool works............................. 279
How do I configure integrity validation?..........................................280
How do I know whether or not integrity validation was successful?.281
Can I disable or enable integrity validation globally?...................... 281
What should I do if an issue is detected?........................................282
MetroPoint group conversion...................................................................... 282
Recovering from a cluster disaster...............................................................282

Chapter 9

Events

283

Normal Events.............................................................................................284
Detailed events........................................................................................... 294
Advanced events.........................................................................................330

RecoverPoint 5.0 Administrator's Guide

CONTENTS

RecoverPoint 5.0 Administrator's Guide

FIGURES

1
2
3
4
5
6
7
8
9
10
11
12
13

Getting Started Wizard Enable Support screen............................................................... 16


Alerts widget..................................................................................................................25
Alerts table.................................................................................................................... 26
Dashboard System Limits tab........................................................................................ 28
DashboardSystem Events.............................................................................................. 30
Identifying a distributed group.......................................................................................82
Identifying a distributed group.......................................................................................82
RPA Clusters: General Cluster Info................................................................................145
RPA Clusters: Storage.................................................................................................. 147
RPA Clusters: Splitters................................................................................................. 160
RPA Clusters: vCenter Servers...................................................................................... 173
License file format....................................................................................................... 268
Example licenses.........................................................................................................269

RecoverPoint 5.0 Administrator's Guide

FIGURES

10

RecoverPoint 5.0 Administrator's Guide

TABLES

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

Dashboard options........................................................................................................ 22
Status icons.................................................................................................................. 24
Tooltip links...................................................................................................................24
Transfer state.................................................................................................................27
System LimitsSystem tab...............................................................................................29
System LimitsGroup tab.................................................................................................29
System LimitsStorage tab ............................................................................................. 29
System LimitsSplitters tab............................................................................................. 30
Event filter settings........................................................................................................ 31
Protection options......................................................................................................... 36
Toolbar options............................................................................................................. 40
Group summary screen.................................................................................................. 49
Multiple consistency group monitoring options............................................................. 62
Multiple consistency group actions................................................................................62
Group policies............................................................................................................... 63
Link policies.................................................................................................................. 65
Copy policies................................................................................................................. 69
Journal policies..............................................................................................................69
Status tab......................................................................................................................74
Status tab options when a group is selected..................................................................75
Priority setting default values........................................................................................ 80
Consolidated statistics output....................................................................................... 83
Status tab......................................................................................................................88
Status tab options when a group is selected..................................................................88
Copy journal tab............................................................................................................ 92
Journal Tab: Snapshot Consolidation Progress information............................................ 94
Journal tab image list.....................................................................................................94
Options....................................................................................................................... 105
Snapshot policies........................................................................................................121
RPA cluster options......................................................................................................144
RPA cluster information............................................................................................... 145
Storage information.....................................................................................................148
Resource pool storage information.............................................................................. 157
Volume information..................................................................................................... 162
RPA table information.................................................................................................. 166
Properties tab information........................................................................................... 167
Status tab information.................................................................................................168
Virtual machine info.................................................................................................... 174
RPA Clusters: Add vCenter Server.................................................................................175
Edit vCenter Server Settings.........................................................................................175
Admin options............................................................................................................. 178
Displayed license information..................................................................................... 181
Automatically registering your RecoverPoint system.....................................................183
Pre-defined users........................................................................................................ 188
Pre-defined roles......................................................................................................... 192
Permissions.................................................................................................................192
Permission settings..................................................................................................... 194
LDAP settings.............................................................................................................. 196
System information collection settings........................................................................ 210
Collect system information options..............................................................................214
Event Filter settings..................................................................................................... 217
RecoverPoint 5.0 Administrator's Guide

11

TABLES

52
53
54
55
56
57
58
59
60
61
62
63

12

Email settings..............................................................................................................219
Syslog settings............................................................................................................ 220
SNMP settings............................................................................................................. 221
RecoverPoint SNMP trap variables............................................................................... 222
System parameters sent to SyR....................................................................................226
Licensing support........................................................................................................ 250
Cluster information......................................................................................................268
License file variables................................................................................................... 269
Journal size with snapshot consolidation equation legend...........................................273
Normal scope events................................................................................................... 284
Detailed scope events................................................................................................. 294
Advanced scope events............................................................................................... 330

RecoverPoint 5.0 Administrator's Guide

CHAPTER 1
Getting Started

This section describes the first steps to setting up your RecoverPoint system.
l
l
l
l
l

The RecoverPoint interfaces.................................................................................. 14


Logging into RecoverPoint..................................................................................... 14
The Getting Started wizard.................................................................................... 15
The Unisphere for RecoverPoint interface.............................................................. 17
The RecoverPoint workflow....................................................................................18

Getting Started

13

Getting Started

The RecoverPoint interfaces


Unisphere for RecoverPoint
A graphical user interface (GUI) for administrators of the RecoverPoint system. Initial
setup, as well as the routine monitoring and managing of the RecoverPoint system,
can all be performed through Unisphere for RecoverPoint.
The RecoverPoint Command Line Interface (CLI)
For IT professionals and advanced users who want to run automated scripts and
perform batch functions on their RecoverPoint copies. For instructions on how to
launch and use the RecoverPoint CLI, see the EMC RecoverPoint Command Line
Interface Reference Guide for your RecoverPoint version.
The RecoverPoint REST (REpresentative State Transfer) API
For developers who wish to integrate RecoverPoint with their own applications and
for those who want to write scripts to automate RecoverPoint operations. For
instructions on how to use the RecoverPoint API, see the EMC RecoverPoint REST API
Programming Guide for your RecoverPoint version.

Logging into RecoverPoint


Before you begin
l

Obtain your RPA cluster management IP.

Optionally obtain your RecoverPoint username and password.

Procedure
1. Open a web browser and enter the cluster management IP into your browser address
bar. The RecoverPoint login screen is displayed.

2. In the RecoverPoint login screen, log in with your username and password, or enter the
default RecoverPoint Username admin and Password admin.
3. If:
l

14

this is your first time logging into Unisphere for RecoverPoint, The Getting Started
wizard on page 15 is displayed. Use the Getting Started wizard to add your
RecoverPoint licenses, enable support, and register your RecoverPoint system.

RecoverPoint 5.0 Administrator's Guide

Getting Started

this is not your first time logging into Unisphere for RecoverPoint and a new
version of RecoverPoint is available The Upgrade Recommendation dialog box on
page 250 is displayed.

Results
Unisphere for RecoverPoint is displayed.
After you finish
Select RPA Clusters > RPA-Cluster-Name > RPAs. Click the tab of each RPA cluster and
ensure that a green checkbox appears in both the Status and Connectivity columns of all
RPAs at each RPA cluster.

The Getting Started wizard


The first time that you access Unisphere for RecoverPoint after initial system installation
or upgrade, the Getting Started Wizard is displayed to guide you, step-by-step through
the process of installing your RecoverPoint licenses, enabling support and registering
your RecoverPoint system. Click Next Add Licenses in the first screen of the Getting
Started Wizard to get started.

Add Licenses screen


The Add Licenses screen allows you to install your RecoverPoint license(s) and display
your license information.

Note

The settings in the Add Licenses screen are displayed both through the Getting Started
Wizard the first time you launch Unisphere for RecoverPoint, and by selecting Admin >
Manage Licenses.
Procedure
1. Install a license on page 179.
2. Click Next Enable Support to go to the Enable Support screen on page 16.

The Getting Started wizard

15

Getting Started

Enable Support screen


The Enable Support screen allows you to configure the RecoverPoint system reports
mechanism (SyR) and provide one-way communication between a RecoverPoint
installation and the EMC System Reports database (ESRS).
Figure 1 Getting Started Wizard Enable Support screen

Note

The settings in the Enable Support screen are displayed both through the Getting Started
Wizard the first time you launch Unisphere for RecoverPoint, and by selecting Admin >
System Notifications > Configure System Reports and Alerts.
Procedure
1. Configure system reports and alerts on page 227.
2. Ensure that system reports are configured correctly on page 229 by clicking the Test
Connectivity button.
Note

The connectivity test, if successful, sends a call home event (number 30999) to the
EMC Install Base and opens a Service Request with EMC Customer Support. If a
verification email is provided, EMC Customer Support sends an email to the entered
email address to verify that the system reports mechanism (SyR) has been
successfully configured.
3. Click Next Register RecoverPoint.

16

RecoverPoint 5.0 Administrator's Guide

Getting Started

Register RecoverPoint screen


The Register RecoverPoint screen allows you to fill out and automatically send the
RecoverPoint post-deployment form to the RecoverPoint IB group.

Note

The settings in the Register RecoverPoint screen are displayed through both the Getting
Started Wizard the first time you launch Unisphere for RecoverPoint, and by selecting
Admin > Register RecoverPoint each subsequent time Unisphere for RecoverPoint is
launched.
Procedure
1. Register your RecoverPoint system on page 182.
2. Click the Finish button to close the Getting Started Wizard and apply your changes.
A dialog box is displayed allowing you to decide whether or not you would like to
display the Getting Started Wizard in subsequent RecoverPoint launches.

NOTICE

It is not recommended to select the checkbox in the pop-up until you have completed
the tasks of adding your licenses, enabling RecoverPoint support, and registering your
RecoverPoint system.

The Unisphere for RecoverPoint interface


The RecoverPoint interface has been developed using Adobe Flex technology and
simplified to promote ease of use and a common user experience between all EMC
products. The RecoverPoint wizards are streamlined and optimized for a simpler and
more intuitive user experience, targeted at the IT generalist. The Dashboard provides an
overview of system health at a glance, and all of the information displayed through
Unisphere for RecoverPoint is consistently auto-refreshed.
Register RecoverPoint screen

17

Getting Started

Note

Up to 20 instances of Unisphere for RecoverPoint can be run concurrently on each RPA


cluster.

The Status Bar that consistently appears at the bottom of the interface displays the
numbers of alerts that are currently in the system, and their severity (for example, Alerts:
1 Critical, 3 Errors, 3 Warnings. When you click the status bar from any screen in the
application, you are redirected back to The RecoverPoint Dashboard on page 22.

The RecoverPoint workflow


Perform the following steps after completing the RecoverPoint system installation
process, and before starting to configure your replication environment:
1. One time only, you will need to Step 1: Quickly access your entitlements on page
242 and Step 2 - Activate your entitlements on page 244 on support.emc.com.
2. Install your RecoverPoint licenses, enable support, and register your RecoverPoint
system using the The Getting Started wizard on page 15.
3. (Optionally) Change the display language of Unisphere for RecoverPoint on page 253
and Set the time display of Unisphere for RecoverPoint on page 252.
4. Manage Users on page 187, Manage Roles on page 191, and Manage User
Authentication on page 195.
5. Register storage on page 149 and Register resource pools on page 158.
6. (Optionally) Change Security Level on page 199.
When you're done, configure the RecoverPoint system to start protecting your data. This
process is described step-by-step in Protect Volumes on page 37.
In case of a disaster, or simply to perform routine maintenance on your production site,
you can test a copy, recover production or fail over operations to another site. Use cases
and procedures for doing so are outlined in Recovery on page 127.
18

RecoverPoint 5.0 Administrator's Guide

Getting Started

Once replication has been configured and started, you normally wont need more than
minimal involvement with the system.
To troubleshoot your system, if required, refer to The RecoverPoint Dashboard on page
22, Manage Protection on page 61 and Manage Recovery on page 139.
The types of event notification in RecoverPoint, and how to configure events is detailed in
System Notifications on page 215.
Note

RecoverPoint contains intelligent System Analysis Tools on page 200. Use these tools to
help you make your replication environment as efficient as possible.

First steps in RecoverPoint


Perform the following steps after completing the installation process, and before starting
to configure your replication environment.
Procedure
1. Access your licenses on page 242 and activate them on page 244 on
support.emc.com.
2. Install your RecoverPoint licenses, enable support, and register your RecoverPoint
system using the Getting Started Wizard on page 15.
3. (Optionally) Set the language on page 253 and the time display on page 252.
4. Manage users on page 187, manage roles on page 191, and manage user
authentication on page 195.
5. Register storage on page 149 and Register resource pools on page 158.
6. (Optionally) Change the user security level on page 199.

Configuring replication
After you have completed the First steps in RecoverPoint on page 19, configure the
RecoverPoint system to start replication. The entire process is described step-by-step in
Protecting Volumes on page 37.

Performing recovery activities


In case of a disaster, or simply to perform routine maintenance on your production site,
you may wish to test a copy, recover production or fail over operations to another site.
Use cases and procedures for doing so are outlined here on page 127.

Monitoring and managing RecoverPoint


Once replication has been configured and started, you normally wont need more than
minimal involvement with the system.
To troubleshoot your system, if required, refer to The RecoverPoint Dashboard on page
22, Managing Protection on page 61 and Managing Recovery on page 139.
The types of event notification in RecoverPoint, and how to configure events is detailed in
System Notifications on page 215.

First steps in RecoverPoint

19

Getting Started

System analysis
RecoverPoint contains intelligent System Analysis Tools on page 200 to help you make
your replication environment as efficient as possible.

RecoverPoint support
If you need support, use the Support on page 236 tab to easily access all of the
RecoverPoint support options on support.emc.com.
For troubleshooting issues, be sure to collect system information on page 209 before
contacting EMC Customer Support.

20

RecoverPoint 5.0 Administrator's Guide

CHAPTER 2
Dashboard

This section describes the RecoverPoint dashboard and how to use it to analyze and
monitor replication in RecoverPoint.
l
l
l
l

The RecoverPoint Dashboard.................................................................................22


Overall Health....................................................................................................... 23
System Limits........................................................................................................28
System Events.......................................................................................................30

Dashboard

21

Dashboard

The RecoverPoint Dashboard


The RecoverPoint Dashboard provides a high-level overview of the RecoverPoint system.
It presents important system information to help you analyze and monitor your
RecoverPoint environment.

Table 1 Dashboard options

Option

When to use

Overall Health

To graphically display an overview of the current status of the RecoverPoint


system.

System Limits

To monitor how close you are to breaching your system limitations.

System Events To display the RecoverPoint event log.

22

RecoverPoint 5.0 Administrator's Guide

Dashboard

Overall Health
The Overall Health tab graphically displays an overview of the current status of the
RecoverPoint system. The overall health tab contains widgets that allow you to easily
monitor and analyze the current state of the system.

The Overall Health tab of the Dashboard contains the following widgets:
l

RPA Clusters Map on page 23

Alerts on page 25

System Traffic on page 26

Consistency Group Transfer Status on page 26

RPA Clusters Map


The RPA Clusters Map widget provides a schematic representation of all RPA clusters in
the RecoverPoint system, the connectivity between them, and any system alerts received
by the clusters.

Overall Health

23

Dashboard

The RPA clusters map displays the type of connection that exists between all RPA clusters
in the system. A dashed line represents an IP (WAN) connection and a solid line
represents a Fibre Channel (LAN) connection.
The RPA clusters map also displays the status of all major components in the system that
are seen by the RPA cluster, including; hosts, switches, storage devices, individual RPAs,
the LAN, and the WAN.
When the system detects a problem with one of these major components, the RPA
clusters map displays:
Table 2 Status icons

Icon

Description
An error icon on each component that is not functioning properly.
A warning icon on each component that has warnings associated with it.
The status of a component, particularly until you have completed logging in to
the system, may also be Unknown. In that case, the icon is displayed on the
component.

Note

The errors and warnings in the RPA clusters map widget are also displayed in the
Alerts on page 25 widget.
Hover your cursor over an RPA cluster in the map to display a tooltip with the following
links:
Table 3 Tooltip links

Option

Description

Display Alerts

To display the Alerts dialog box containing the number of alerts registered by
the RPA cluster.
Click Details in the Alerts dialog box to display the alert texts.

Display RPA
To display the General Cluster Info on page 144.
Cluster Details

Multipath monitoring
When multipathing monitoring is active, the system analyzes network errors at the level
of individual paths, and generates a warning in the RPA Clusters Map whenever there is
not full redundancy between the RPA and splitters or volumes. Full redundancy is defined
as follows:

24

For RPA-to-volume links, there must be at least two distinct paths between each RPA
and volume; that is, each RPA has access to at least two storage WWNs (and
controllers, where relevant) via non-overlapping paths.

For RPA-to-splitter links, there must be at least two distinct paths between each RPA
and each splitter using different RPA ports and host (or switch) WWNs.

RecoverPoint 5.0 Administrator's Guide

Dashboard

When RPA Multipath Monitoring is enabled, the system issues a warning upon logging in
regarding any existing links without full redundancy. In addition, warning events are
written to the log.
By default, multipath monitoring is active for all copies, for links both to storage and to
splitters.
To enable or disable RPA multipath monitoring, use the
config_multipath_monitoring CLI command.
Note

Path information is displayed in the Storage on page 147 screen.

Alerts
The Alerts widget provides a graphical overview of system health categorized by alert
level.
Figure 2 Alerts widget

Click the More details link at the bottom-right corner of the graph to display the alerts
table.
The alerts table contains system errors and warnings. The errors and warnings are
composed of messages from the System Limits screen, and other general system
warnings.

Alerts

25

Dashboard

Figure 3 Alerts table

Hover over an error or warning in the table to display a tooltip containing the error or
warning properties.
When relevant, the dialog will show a severity, category, relevant RPA cluster, and
description of the error or warning.
Note

A system limit with a status of Critical equates to an Error in the alerts table. A system limit
with a status of Major or Minor equates to a Warning in the alerts table.
Click the Less details link at the bottom of the alerts table to close the alerts table and
display the Alerts widget.

System Traffic
The System Traffic widget provides a graphical overview of the amount of SAN and WAN
traffic passing through each RPA cluster.

Consistency Group Transfer Status


The Consistency Group Transfer Status widget displays the transfer state of all
consistency groups in the RecoverPoint system.

26

RecoverPoint 5.0 Administrator's Guide

Dashboard

Each consistency group and consistency group copy can be in any one of the following
transfer states:
Table 4 Transfer state

Transfer
State

Description

Active

Data is being transferred asynchronously to a copy.

Active
Data is being transferred synchronously to a copy.
(Synchronized
)
Paused

Data is not being transferred to a copy, because transfer has been paused by the
user.

Paused by
system

Data is not being transferred to a copy, because transfer has been paused by the
system.

Init (n%)

A copy is being initialized, or undergoing a full sweep or volume sweep.

High-load

The system enters a permanent high-load state while data is being transferred to
a copy.

High-load (n
%)

The system enters a temporary high-load state while data is being transferred to
a copy.

Replicating
snap

The link is replicating in snap-based replication mode.

Snap idle

The link is about to begin or has completed a snap-based replication process.

N/A

Data is not being transferred to a copy, because the copy has been disabled by
the user.

The pie chart in the consistency group transfer status widget presents a breakdown of the
consistency groups in the system based on their current data transfer states, as follows:
l

Active (green) - includes active, active (synchronized), Replicating snap and Snap idle

Error (red)

Init (orange) - includes initializing, high load, and high load (n%)

Paused (yellow) - includes paused and paused by system

Disabled (grey) - includes N/A

Consistency Group Transfer Status

27

Dashboard

Other (brown) - includes flushing, flushing buffer, flushing remote buffer to journal, and
unknown

Note

A consistency groups transfer state is determined by its most severe link state. For
example, in a RecoverPoint system that is replicating to two copies, if one link is paused
and one link is active, the transfer state will be set as paused.
If there are no consistency groups in the system, the chart will be replaced by the text No
consistency groups.
Hover your mouse over a transfer state in the chart to see the names of the consistency
groups in that state.

System Limits
The System Limits tab monitors important system values to let the user know how close
they are to their limits. The limits may be determined by the system, policies, licensing, or
limitations of external technologies.
Figure 4 Dashboard > System Limits tab

You can sort the system limits by clicking in a column header.


You can filter monitored limits by clicking the Filter icon in the upper-right corner and
entering text to filter by in the text box.
All status severities of minor, major, and critical are also displayed in the Alerts on page
25 widget.

System
The following information is displayed in the System tab:

28

RecoverPoint 5.0 Administrator's Guide

Dashboard

Table 5 System Limits > System tab

Column

Description

Status

OK, minor, major, critical. Indicates how close a monitored setting is to its
limit.

Description

The description of the monitored system limit.

RPA Cluster

The relevant RPA cluster.

RPA

The relevant RecoverPoint appliance.

Current Value of
Limit

The current value of the setting and its limit.

Groups
The following information is displayed in the Groups tab:
Table 6 System Limits > Group tab

Column

Description

Status

OK, minor, major, critical. Indicates how close a monitored setting is to its limit.

Description

The description of the monitored system limit.

Group

The relevant consistency group.

Copy

The relevant consistency group copy.

Current Value
of Limit

The current value of the setting and its limit.

Storage
The following information is displayed in the Storage tab:
Table 7 System Limits > Storage tab

Column

Description

Status

OK, minor, major, critical. Indicates how close a monitored setting is to its limit.

Description

The description of the monitored system limit.

RPA Cluster

The relevant RPA cluster.

Array

The name of the relevant storage array.

Current Value
of Limit

The current value of the setting and its limit.

Splitters
The following information is displayed in the Splitters tab (not relevant to XtremIO):
Groups

29

Dashboard

Table 8 System Limits > Splitters tab

Column

Description

Status

OK, minor, major, critical. Indicates how close a monitored setting is to its limit.

Description

The description of the monitored system limit, including:


l

The number of RPA clusters attached to the splitter.

The total number of volumes attached to the splitter.

RPA Cluster

The relevant RPA cluster.

Splitter

The name of the relevant splitter.

Current Value
of Limit

The current value of the setting and its limit.

System Events
The System Events tab displays the RecoverPoint event log which contains all events
logged by the RecoverPoint system, in the order that they are logged. A total of 9000
events can be displayed in the event log.
Figure 5 Dashboard > System Events

Display events
To display system events; select Dashboard > System Events tab.
The RecoverPoint event log is displayed, containing the last 9000 events logged in the
RecoverPoint system. Each event in the event log is displayed along with its relevant
event ID, summary, level, scope, point in time, RPA cluster, and topic.
30

RecoverPoint 5.0 Administrator's Guide

Dashboard

Note

By default, only Normal scope events are displayed, but you can also Filter events on
page 31 to display Detailed events.

Display additional event information


When an event is selected in the event log, the Details tab under the event log displays
additional event information. To display more information, such as a recommended user
action in cases that warrant user intervention, click the More Information tab.

Sort events
To sort the events in the event log by ID, summary, level, scope, point in time, RPA
cluster, or topic, click on the relevant column header but you can also Filter events on
page 31 based on advanced criteria.

Filter events
Use the following procedure to filter the events in the System Events on page 30 log, and
save the settings for the current RecoverPoint user.
Procedure
1. Select Dashboard and click the System Events tab.
2. Click Filter Events. The Event Filter dialog box is displayed.

3. Events can be filtered using the settings in the table below.


Table 9 Event filter settings

Setting

Values

From

The date and time (in GMT) from which events are to be displayed.
Select Unbounded to display the 9000 events preceding the date and time
specified in the To field.

To

The date and time (in GMT) until which events are to be displayed. Select
Unbounded to display all events up until the current date and time.

Display additional event information

31

Dashboard

Table 9 Event filter settings (continued)

Setting

Values

Events to
Exclude

The IDs of the events that should not be displayed. Multiple event IDs can be
entered. Separate multiple event IDs with a comma.

Topics

The topic of the events to display in the event log. Valid values are:

Scope

Level

RPA cluster

Splitter

Consistency group

Management

RPA

Array

The scope of the events to display in the event log:


l

Normal: To display only the root cause for an entire set of Detailed and
Advanced events. In most cases, these events are sufficient for effective
monitoring of system behavior.

Detailed: To display all events.

Advanced: In specific support cases, EMC Customer Support may ask you to
display events in an Advanced scope. These events contain information that
is intended primarily for the technical support engineers.

The level of the events to display in the event log:


l

Info: Messages are informative in nature, usually referring to changes in the


configuration or normal system state.

Warning: Message indicates a warning, usually referring to a transient state


or an abnormal condition that does not degrade system performance.

Error: Message indicates an important event that is likely to disrupt normal


system behavior and/or performance.

Containing the Enter the terms that must exist in the event summary in order for the event to be
text
displayed in the event log. Use commas to separate multiple terms. Select:
l

Match all to display all events containing all of the specified terms in the
event summary.

Match any to display all events containing any of the specified terms in the
event summary.

Reset to
Default

To reset all of the settings in the Event Filter dialog box to their default settings.

OK

To close the Event Filter dialog box, and apply the specified settings.

Cancel

To close the Event Filter dialog box, without applying the specified settings.

Copy an event to the clipboard


Procedure
1. Select Dashboard and click the System Events tab.
32

RecoverPoint 5.0 Administrator's Guide

Dashboard

2. Select the event in the event log and click Copy to Clipboard.
Results
The event info that is copied to the clipboard is in the following format:
Summary: RPA is successfully communicating with its cluster.
Level: Clear
ID: 3000
Topic: RPA
Scope: Normal
Point in Time (GMT): 4/17/2013 8:46:35 AM
RPA Cluster: London
Description : RPA is successfully communicating with its
cluster(RPA2)

Collect system information around a specific event


Procedure
1. Attach volumes to the splitter on page 164 for all splitters in the system.
2. Select Dashboardand click the System Events tab.
3. Select an event in the event log.
4. Click Collect System Information. The collect system information wizard is displayed
and the collection start and end fields are automatically set to 10 minutes before and
after the selected event.
5. Continue from step 3 on page 210 of Collect system information on page 209.

Collect system information around a specific event

33

Dashboard

34

RecoverPoint 5.0 Administrator's Guide

CHAPTER 3
Protection

This section describes how to protect volumes, and how to monitor and manage
protection and consistency.
l
l
l
l
l

RecoverPoint Protection........................................................................................ 36
Protect Volumes.................................................................................................... 37
Policy Templates................................................................................................... 52
Group Sets............................................................................................................ 56
Manage Protection................................................................................................ 61

Protection

35

Protection

RecoverPoint Protection
In Unisphere for RecoverPoint, data protection is guided by wizards, and performed
through the Protection menu.

Table 10 Protection options

36

Option

When to Use

Protect
Volumes

To add a new consistency group and define its protection plan.


Displays the Protect Volumes wizard to guide you through the process of
defining the production and its protection plan, the copies and their protection
plans, and the production and copy journals and their protection plans.

Policy
Templates

To view or modify existing protection policy templates, or create a new protection


policy template for copies or links.

Group Sets

To create or modify sets of consistency groups, perform recovery activities or


periodically bookmark the same consistent point in time, across volumes in
multiple groups, simultaneously.

Manage
Protection

To manage and monitor existing consistency groups and their protection plans,
control transfer, or bookmark consistent points in time across groups and sets of
groups.

RecoverPoint 5.0 Administrator's Guide

Protection

Protect Volumes
Use Protection > Protect Volumes to create consistency groups (CGs) to protect your
volumes. In RecoverPoint, CGs are used to configure protection policies, and set RPO and
RTO policies according to specific resource allocation and prioritization.

How to protect volumes


Creates a new consistency group.
Before you begin
l

Journal volumes can be automatically provisioned. To automatically provision journal


volumes, RecoverPoint requires dedicated resource pools. If you are planning to use
the automatic journal provisioning feature, make sure you have dedicated resource
pools on storage. You must register storage and register resource pools to enable
automatic volume provisioning.

For RecoverPoint with Unity, VNXe, or VNX with RecoverPoint/SE licenses, there is a
limitation of one array per side. If you have a local and a remote copy, make sure the
local copy is stored on the same Unity/VNXe/VNX array as the production copy.

Ensure you have your RecoverPoint environment planning document describing the
groups and journals you will need to create, the volumes that you will need to define
in each, and the RPAs that will handle the data transfer. If you do not have this
document, contact your RecoverPoint account executive.
Your environment planning document should detail:

The number of consistency groups that you will need, and their names.

The LUNs to include in each group, and at each copy.

The journal LUNs at each copy.

Which consistency groups to set as distributed.

The required RPO and RTO settings per consistency group.

The RPA that will handle the replication of each group.

The priority of each consistency group with relation to the other groups in the
system.

Decide whether you will want to Initialize from backup on page 254, and prepare
accordingly.
When a consistency group is first created, or a replication set is first added to the
system, the system automatically starts a first-time initialization process at the
Protect Volumes

37

Protection

copies (that is, synchronizing the copy volumes with the production volumes by
sending the complete image of each production volume to each copys storage).
Depending on the amount of data being transferred, this process can be both timeconsuming and bandwidth-consuming. To save time and bandwidth, see First-time
initialization is taking too long on page 253.
l

If one volume at a copy is on an XtremIO array, all other XtremIO volumes at that copy
must be on the same array.

Ensure you have mapped all replication LUNs to RecoverPoint WWNs. Only the
volumes that are mapped and zoned to the RPA cluster are displayed in the available
volumes list in RecoverPoint.
Note

In RecoverPoint/SE only, all of the volumes of the Unity or VNX array are displayed in
the available volumes list.
l

There is a limit on the number of LUNs that are supported per storage group (see
the EMC RecoverPoint Release Notes for this limit).

If you need more LUNs than are currently available to support your replication sets
and journals, either because you have calculated your system needs or because
you have received a message that you have exceeded the maximum number of
supported LUNs, you may be able to create additional storage groups, according
to the procedure presented in Add more LUNs than allowed per VNX storage
group on page 264.

The Unity/VNXe storage system does not use storage groups.

In Symmetrix splitter environments:

38

In VNX storage systems:

Volumes attached to VNX splitters and volumes attached to Symmetrix splitters


may be mixed in one consistency group. See the EMC RecoverPoint Deploying with
Symmetrix Arrays and Splitter Technical Notes for other splitter-specific limitations.

If any volume added is a device on a Symmetrix VMAX 10K, 20K or 40K array, refer
to Device Tagging in the EMC RecoverPoint Deploying with Symmetrix Arrays and
Splitter Technical Notes.

In VNX splitter environments:


n

The same consistency group can contain volumes attached to multiple splitters.

The same consistency group can contain volumes using a VNX splitter to split
writes to one RPA cluster and volumes using a different splitter type to split writes
to a different RPA cluster.

In Unity/VNXe splitter environments, a consistency group can contain only volumes


attached to Unity/VNXe splitters.

In VPLEX splitter environments:


n

The VPLEX consistency group name will only be displayed in the VPLEX Group
column of the volume list if you Attach volumes to the splitter on page 164 for the
VPLEX splitter (including certificate).

After you register your VPLEX storage, you will not be able to select VPLEX volumes
and non-VPLEX volumes in the same consistency group copy. It is recommended
that all volumes in a VPLEX consistency group be configured in a single
consistency group copy, and all volumes in a non-VPLEX consistency group be
configured in another consistency group copy.

RecoverPoint 5.0 Administrator's Guide

Protection

A VPLEX MetroPoint consistency group can contain a maximum of one remote


copy and from zero to two local copies. If there is no remote copy, there must be
two local copies, one at each side of the VPLEX Metro.

To turn a non-MetroPoint group into a MetroPoint group, when the production


volume is a VPLEX distributed volume and there exists no more than one remote
copy, Add a copy to a group on page 99.

To turn a MetroPoint group into a non-MetroPoint group, Remove a copy from a


group on page 106.

For replicating Oracle databases, see the instructions on how to replicate an Oracle
database, including using Oracle hot backup procedures with RecoverPoint
bookmarks for point-in-time snapshots and quick testing and disaster recovery in the
EMC RecoverPoint Replicating Oracle with EMC RecoverPoint Technical Notes.

Procedure
1. Select Protection > Protect Volumes.
2. In the Select Production Volumes screen:
a. Define the consistency group name, the production name, and the RPA cluster that
will manage the production.

Note

A MetroPoint Group checkbox is displayed if one or more distributed volumes


exists at the production copy, and is mapped to the RPA cluster of the production
copy. To create a MetroPoint group, select the MetroPoint group check box. When
selected, only VPLEX distributed volumes are displayed in the volume list.
b. Select your production volumes in the volume list.
Note

If one volume at a copy is on an XtremIO array, all other XtremIO volumes at that
copy must be on the same array. Use the toolbar at the top-right corner of the
screen to help you identify the volumes.

How to protect volumes

39

Protection

Table 11 Toolbar options

Icon

Description
To filter the volumes in the list by Size, Product, Vendor, Name, UID, LUN,
or VPLEX Group.
Note

The VPLEX Group column is only displayed after you Attach volumes to the
splitter on page 164 for your VPLEX splitter.
To select the table columns that you want to display.
Note

You can also click and drag the column dividers to make the columns
wider or thinner. In most cases, RecoverPoint will remember the column
width that you define, and display the columns accordingly in each
subsequent launch of this wizard.
To export the data in the volume list to a *.CSV file.
To display context-sensitive help for this wizard screen.

c. Optionally, click Modify Policies to define the group policies and copy policies
now, or skip this step and define the policies later.
The policy settings are optional. The default values provide a practical
configuration. It is recommended to accept the default settings unless there is a
specific business need to set other policies.
d. Click Next Define the Production Journal to select journal volumes that have
already been provisioned on storage. on page 40

Defining the production journal


Use the Define Production Journal screen to define the production journal, after you have
manually provisioned it on storage.
Before you begin

40

If your copy volumes are located on a Unity, VNXe, or VNX array and a resource pool
has been registered on page 158 on the same array for the journal volume, select
Automatically Provision Journal Volumes to allow RecoverPoint to provision your
journal volumes.

Copy journal volumes have been manually provisioned on storage.

For applications running on a host cluster, the journal volumes must not be resources
of the host cluster.

If you selected the MetroPoint group checkbox in the previous screen of the wizard,
the Standby Production screen is displayed.

RecoverPoint 5.0 Administrator's Guide

Protection

In the Standby Production screen, enter a name for the copy in the Standby
production copy name field before you continue with this procedure.
Procedure
1. Ensure Select Provisioned Journal Volumes is selected.

2. Select the volumes to add to the journal.


For best performance, select volumes that are identical in size. If identically sized
volumes are not available, select volumes that are similar in size.
Note

The journal for an XtremIO array need not be larger than 10 GB for a copy with nondistributed consistency groups, 40 GB for distributed consistency groups.
In RecoverPoint/CL and RecoverPoint/EX, only the volumes that are mapped and
zoned to the RPA cluster are displayed in the available volumes list.
In RecoverPoint/SE, all of the volumes of the supported Unity Series and VNX Series
arrays are displayed in the available volumes list.
How to protect volumes

41

Protection

3. Use the toolbar icons at the top-right corner of the screen to help you identify the
required volumes in the volume list. For best performance, select volumes that are
identical in size. If identically sized volumes are not available, select volumes that are
similar in size.
Note

In RecoverPoint/CL and RecoverPoint/EX, only the volumes that are mapped and
zoned to the RPA cluster are displayed in the available volumes list. In
RecoverPoint/SE, all of the volumes of the supported Unity Series and VNX Series
arrays are displayed in the available volumes list.
4. Optionally, click Modify Policy to define the journal policies now, or skip this step and
define the policies later using the Manage Protection screen.
Note

The Modify Policy link is only enabled after volumes are selected in the volume list.
5. Optionally, apply a policy template.
a. Use the Select Template drop-down to select one of your policy templates.
b. Click Load to load the settings in the template.
c. Click OK to apply the settings.
6. Click Next Add a Copy on page 44.

Automatically provisioning the production journal


Use the Journal Provisioning Method screen to automatically provision the copy journal.
Before you begin
l

Copy volumes are located on a Unity, VNXe, or VNX array.

A resource pool has been registered on the Unity, VNXe, or VNX array containing the
copy volumes.

Procedure
1. Select Automatically Provision Journal Volumes.

42

RecoverPoint 5.0 Administrator's Guide

Protection

2. Define the journal settings.


Option

Description

Journal size

Enter a value for the journal size to define an absolute journal size
at the copy.

Predicted
Select this option and enter a value for predicted incoming writeincoming write- rate and required protection window to have RecoverPoint
rate
calculate the required journal size based on your consistency
groups predicted write-rate and a required protection window.
As you change this value, the journal size that is required to
sustain the entered values for predicted incoming write-rate and
the required protection window is dynamically updated in the
Calculated journal size field.
Required
protection
window

Note

Only relevant if Predicted incoming write-rate is selected.


Enter a value for the required protection window to have
RecoverPoint calculate the required journal size based on your
consistency groups predicted write-rate and a required protection
window.
As you change this value, the journal size that is required to
sustain the entered values for predicted incoming write-rate and
the required protection window is dynamically updated in the
Calculated journal size field.

Tiering Policy

(Optionally) Select one of the available tiering policies displayed in


the Resource pool provisioning information table below to
automatically apply the policy to the selected volumes.

Back to Select
Production
Volumes

Go back to the previous screen and re-select the production


volumes, or (optionally) redefine the production protection policy.

3. If there are Available Tiering Policies on storage, you can select a Tiering policy to
apply to the journal volumes from the combo-box at the bottom of the resource pool
table now.
4. Select the relevant resource pool from the table.
The following information is displayed for each resource pool in the table:
Information

Description

Name

The name of the registered resource pool in the RecoverPoint


system.

Type

The resource pool type. Valid resource pool types are:

Total Size

RAID groups

Storage groups

Datastores

The total capacity of the resource pool.


How to protect volumes

43

Protection

Information

Description

Available Size

The total capacity of the resource pool minus the capacity of


the volumes that were already allocated in the resource pool.

Tiers

The physical type of the storage. For example, Fibre Channel,


SATA, etc.

Available Tiering
Policies

The tiering policies that are available on storage and can be


automatically applied to volumes in RecoverPoint when users
automatically provision volumes.

Storage Name

The name of the storage array in RecoverPoint.

Serial Number

Serial number of the storage array.

Array Type

The type of the array.

IP

The arrays IP address.

5. Click Next Add a Copy on page 44.

Adding a copy
Procedure
1. In the Add a Copy screen:

a. Define the copy name, the RPA cluster that will manage the copy volumes, and the
replication mode on page 65.
For each production volume, click the Select volume link to add a volume to the
copy. The selected volume is the volume that the production volume is replicated
to.
The volume list is displayed. The volume list only contains volumes of sizes that
are equal to, or greater than, the specified production volume.

44

RecoverPoint 5.0 Administrator's Guide

Protection

Note

If one volume in a copy is on an XtremIO array, all the volumes in the consistency
group at that copy must be on an XtremIO array. If the copy is on an XtremIO array,
it must be exactly the same size as all other copies in the consistency group. It
cannot be either larger or smaller.

Note

In RecoverPoint/CL and RecoverPoint/EX, only the volumes that are mapped and
zoned to the RPA cluster are displayed in the available volumes list. In
RecoverPoint/SE, all of the volumes of the supported Unity Series or VNX Series
array are displayed in the available volumes list.
Use the toolbar icons at the top-right corner of the screen to help you identify the
required volumes in the volume list. For best performance during failover, select
volumes that are the same size as the production volume. If volumes of the same
size are not available, select volumes that are as similar in size as possible.
NOTICE

You cannot proceed to the next step until the replication sets icon under the
volume list shows a green check mark to indicate that the number of replication
sets is equal to the number of production volumes.

b. Optionally, click Modify Policy to define the copy policies and link policies now, or
skip this step and define the policies later from the Manage Protection screen.
If you want to change the protection settings using policy templates, in the Link
tab, use the Select Template drop-down to select one of your templates. Click Load
to load the settings in the template and click OK to apply the settings.
How to protect volumes

45

Protection

Note

The Modify Policy link is only enabled after volumes are selected in the volume
list. You can set a copy policy for the production, however, the copy policy of the
production is only relevant after failover, when the production becomes a copy.
c. Click Next Define Copy Journal and:
l

Allow RecoverPoint to automatically provision the copy journal volumes on


page 47.
or

Select copy journal volumes that have already been provisioned on storage on
page 46.

Defining the copy journal


Use the Define Copy Journal screen to define the copy journal, after you have manually
provisioned it on storage.
Before you begin
l

Copy journal volumes have been manually provisioned on storage.

For applications running on a host cluster, the journal volumes must not be resources
of the host cluster.

Procedure
1. Ensure Select Provisioned Journal Volumes is selected.

2. Select the volumes to add to the journal.


Note

The journal for an XtremIO array need not be larger than 10 GB for a copy with nondistributed consistency groups, 40 GB for distributed consistency groups.
3. Use the toolbar icons at the top-right corner of the screen to help you identify the
required volumes in the volume list. For best performance, select volumes that are
46

RecoverPoint 5.0 Administrator's Guide

Protection

identical in size. If identically sized volumes are not available, select volumes that are
similar in size.
Note

In RecoverPoint/CL and RecoverPoint/EX, only the volumes that are mapped and
zoned to the RPA cluster are displayed in the available volumes list. In
RecoverPoint/SE, all of the volumes of the supported Unity Series or VNX Series array
are displayed in the available volumes list.
4. Optionally, click Modify Policy to define the journal policies now, or skip this step and
define the policies later using the Manage Protection screen.
Note

The Modify Policy link is only enabled after volumes are selected in the volume list.
5. Optionally, apply a policy template.
a. Use the Select Template drop-down to select one of your policy templates.
b. Click Load to load the settings in the template.
c. Click OK to apply the settings.
6. Click Next Display Group Summary on page 49.

Automatically provisioning the copy journal


Use the Journal Provisioning Method screen to automatically provision the copy journal.
Before you begin
l

Copy volumes are located on a Unity, VNXe, or VNX array.

A resource pool has been registered on the Unity, VNXe, or VNX array containing the
copy volumes.

Procedure
1. Select Automatically Provision Journal Volumes.

2. Define the journal settings.


How to protect volumes

47

Protection

Option

Description

Journal size

Enter a value for the journal size to define an absolute journal size
at the copy.

Predicted
Select this option and enter a value for predicted incoming writeincoming write- rate and required protection window to have RecoverPoint
rate
calculate the required journal size based on your consistency
groups predicted write-rate and a required protection window.
As you change this value, the journal size that is required to
sustain the entered values for predicted incoming write-rate and
the required protection window is dynamically updated in the
Calculated journal size field.
Required
protection
window

Note

Only relevant if Predicted incoming write-rate is selected.


Enter a value for the required protection window to have
RecoverPoint calculate the required journal size based on your
consistency groups predicted write-rate and a required protection
window.
As you change this value, the journal size that is required to
sustain the entered values for predicted incoming write-rate and
the required protection window is dynamically updated in the
Calculated journal size field.

Tiering Policy

(Optionally) Select one of the available tiering policies displayed in


the Resource pool provisioning information table below to
automatically apply the policy to the selected volumes.

Back to Select
Production
Volumes

Go back to the previous screen and re-select the production


volumes, or (optionally) redefine the production protection policy.

3. If there are Available Tiering Policies on storage, you can select a Tiering policy to
apply to the journal volumes from the combo-box at the bottom of the resource pool
table now.
4. Select the relevant resource pool from the table.
The following information is displayed for each resource pool in the table:

48

Information

Description

Name

The name of the registered resource pool in the RecoverPoint


system.

Type

The resource pool type. Valid resource pool types are:


l

RAID groups

Storage groups

Datastores

Total Size

The total capacity of the resource pool.

Available Size

The total capacity of the resource pool minus the capacity of


the volumes that were already allocated in the resource pool.

RecoverPoint 5.0 Administrator's Guide

Protection

Information

Description

Tiers

The physical type of the storage. For example, Fibre Channel,


SATA, etc.

Available Tiering
Policies

The tiering policies that are available on storage and can be


automatically applied to volumes in RecoverPoint when users
automatically provision volumes.

Storage Name

The name of the storage array in RecoverPoint.

Serial Number

Serial number of the storage array.

Array Type

The type of the array.

IP

The arrays IP address.

5. Click Next Display Group Summary on page 49.

Displaying the group summary


Procedure
1. In the Group Summary screen:

a. Use the following options until the consistency group diagram reflects your
requirements:
Table 12 Group summary screen

Action

Description

Edit

Opens the Add a Copy screen of the copy whose Edit link you clicked,
allowing you to edit the copy settings, and modify the copy volume
configuration, replication mode, or protection plan.

Delete

Removes the copy whose Delete link you clicked, from the consistency
group.

How to protect volumes

49

Protection

Table 12 Group summary screen (continued)

Action

Description
Note

You cannot delete the production copy.

Replication
Sets

Displays detailed volume information on all replication set volumes that


have been defined for the consistency group. Select a row in the
replication sets table in the top section to display the associated volume
information in the bottom section.
For each volume the following information is displayed; vendor, array,
serial number, volume, UID, size and VPLEX group.
Click on the replication set name to open the Edit Replication Set dialog
box, and rename the replication set.

Note

When you add a replication set to a consistency group, RecoverPoint


ensures consistency between the copies and the production source by
performing a volume sweep synchronization on all of the volumes in the
new replication set. Selecting the Attach as Clean checkbox overrides the
default synchronization process by informing the system that the new copy
volumes are known to be an exact image of their corresponding production
volumes. If Attach as Clean is checked, and the copy volumes are
inconsistent with their corresponding production volumes, they will remain
inconsistent. To ensure consistency, best practice is not to use Attach as
Clean and allow RecoverPoint to perform the volume sweep.
When selected, starts transfer for the group as soon as the group is
Start
created.
replication .
..when I
NOTICE
click
If a consistency group contains one journal created using automatic volume
'Finish'
provisioning and one journal whose volumes were selected manually this
setting will not work and you will have to start transfer for the group
manually.
A first-time initialization (or fast first-time initialization) occurs when a
consistency group is enabled for the first time.

50

RecoverPoint 5.0 Administrator's Guide

Protection

Table 12 Group summary screen (continued)

Action

Description

Add a Copy

Opens the Add a Copy screen, allowing you to create another copy, define
the new copys settings, and define the copys volume configuration,
replication mode, and protection plan.

Finish

Note

Before you start transfer to any copy, ensure that the copy volumes are
unmounted from any hosts and any volume groups are deported from the
logical volume manager.
Creates the group, applies all of the specified settings, and starts a firsttime initialization process to synchronize the production volumes to the
copy volumes.
Before you click Finish, note the following:
l

If First-time initialization is taking too long on page 253, instead of


allowing RecoverPoint to send the initialization data to the copy
storage, you can initialize the copy volumes from a backup disk, saving
the time it would otherwise take to synchronize the data over a WAN or
FC connection.

By default, RecoverPoint writes the first snapshot directly to the copy


storage, bypassing the copy journal. Using the fast_first_init
parameter of the config_link_policy CLI command, you can
override the default setting and write the initialization snapshot first to
the journal (which is more time-consuming but provides greater data
protection).

To enable failover during initialization, set the value of the


allow_long_resync parameter of the config_copy_policy
CLI command to no (default is yes).

b. Click Finish to create the group, and all of it's copies.


The group is created and the Manage Protection screen is displayed.
After you finish
You can now Manage Protection on page 61 or start RecoverPoint Recovery on page
128.

How to protect volumes

51

Protection

Policy Templates
Use Protection > Policy Templates to view or modify existing protection policy templates
or create new protection policy templates, for copies or links.

Policy templates are re-useable pre-defined protection settings for both copies and links.
Three default policy templates are included in RecoverPoint; default copy, default remote
link and default local link.
Policy templates work in the following way, the user:
1. Modifies an existing policy template or creates a new policy template.
2. Applies the settings in the policy template to existing links or copies using the
Protect Volumes wizard or the Manage Protection screen.

Display all policy templates


Displays the list of policy templates currently configured in the RecoverPoint system.
Procedure
1. Select Protection > Policy Templates.

2. You cannot remove the default policy templates, but you can create new ones on page
53 based on the default template settings and modify on page 53 any of the userdefined policy templates.

52

RecoverPoint 5.0 Administrator's Guide

Protection

Create new policy templates


Creates a new user-defined policy template.
Before you begin
l

Only users with Group permissions can create new policy templates.

Use the Copy policy template to control your Journal policies on page 69.

If a policy template contains an invalid value for an array, you will not be able to
apply it to a consistency group. For example, if you select Enable RecoverPoint
Snapshot Consolidation and try to apply the template to copy on an XtremIO array,
loading the template will fail because RecoverPoint snapshot consolidation is not
supported on XtremIO arrays.

Procedure
1. Select Protection > Policy Templates.
2. In the list of policy templates, click Add.

The Add Policy Template dialog box is displayed.


3. In the Type field, select the type of policy template that you want to create.
Valid types of policy templates are Copy and Link.
The settings change according to the type of template that you select.
4. In the Name field, enter a descriptive name for your new policy template.
5. Depending on the type of template you selected, configure the Journal policies on
page 69 and/or the Table 16 on page 65 according to your requirements.
6. Click OK.

Modify existing policy templates


Modifies an existing user-defined policy template.
Before you begin
l

Only users with Group permissions can modify policy templates.

When you modify a policy template, all settings based on the template do not
automatically change. You will need to re-load and apply the modified policy
template to use the new settings.
Create new policy templates

53

Protection

Use the Copy policy template to control your Journal policies on page 69.

If a policy template contains an invalid value for an array, you will not be able to load
it to the array. For example, if you select Enable RecoverPoint Snapshot
Consolidation and then try to apply the template to a copy on an XtremIO array,
loading the template will fail because RecoverPoint snapshot consolidation is not
supported on XtremIO arrays.

You cannot modify any of the default Policy Templates on page 52.

Procedure
1. Select Protection > Policy Templates.
2. In the list of policy templates, select a template and click Edit.

The Edit Policy Template dialog box is displayed.


3. Depending on the type of template you selected, configure the Journal policies on
page 69 and/or the Table 16 on page 65 according to your requirements.
4. Click OK.

Remove a policy template


Removes a user-defined policy template from the RecoverPoint system.
Before you begin
l

Only users with Group permissions remove policy templates.

You cannot remove the default Copy and Link policy templates, and you cannot
change their settings. You can, however, create new policy templates and base them
on the settings of the default policy templates.

Procedure
1. Select Protection > Policy Templates.
2. In the list of policy templates, select a template.
3. Click Remove, and click Yes in the confirmation dialog box.

Apply a policy template to a new consistency group


Use the Protect Volumes on page 37 wizard to apply policy templates to copies, journals
and links during group creation.

54

RecoverPoint 5.0 Administrator's Guide

Protection

Apply a policy template to an existing consistency group


Use the Protection > Manage Protection to apply policy templates after group creation.

Define a protection policy for existing group links


Procedure
1. Select Protection > Manage Protection
2. In the left pane, expand the consistency group and select one of its copies.
3. In the right pane, click the Link Policy tab.
4. In the Link Policy tab, for each link:
a. Select the link to which you want to apply the template settings.
b. Use the Select Template drop-down to select the template.
c. Click Load to load the settings in the template.
The settings have not yet been applied, and they can now be modified to reflect
your requirements.
d. Click Apply to apply the settings to the selected link.

Define a protection policy for existing group journals


Procedure
1. Select Protection > Manage Protection
2. In the left pane, expand the consistency group and select one of its copies.
3. In the right pane, click the Copy Policy tab.
4. In the Copy Policy tab:
a. Use the Select Template drop-down to select the template.
b. Click Load to load the settings in the template.
The settings have not yet been applied, and they can now be modified to reflect
your requirements.
c. Click Apply to apply the settings to the copy journal.

Define policies while adding a copy to an existing group


Procedure
1. Select Protection > Manage Protection
2. In the left pane, select a consistency group.
3. In the right pane, select Status > Add Copy.
4. In the production journal and copy journal creation screens, define the journal
policies on page 69:
a. Select the Modify Policy link.
b. Use the Select Template drop-down to select one of your templates.
c. Click Load to load the settings in the template.
The settings have not yet been applied, and they can now be modified to reflect
your requirements.
Apply a policy template to an existing consistency group

55

Protection

d. Click OK to apply the settings.


5. In the copy creation screen, define the link policies on page 65:
a. Select the Modify Policies link.
b. Select the Link tab.
c. Use the Select Template drop-down to select one of your templates.
d. Click Load to load the settings in the template.
The settings have not yet been applied, and they can now be modified to reflect
your requirements.
e. Click OK to apply the settings.

Group Sets
Use Protection > Group Sets to create or modify sets of consistency groups, perform
operational and recovery activities, and periodically bookmark the same consistent point
in time across volumes in multiple groups, simultaneously.

Note

Parallel bookmarks cannot be enabled for group sets containing consistency groups with
volumes that reside on an XtremIO array or a Data Domain.

How group sets work


The user selects a set of groups, enters a name for the group set, and (optionally) sets the
frequency at which parallel bookmarks will be periodically applied to the group set.
If parallel bookmarking is enabled, a bookmark is generated at the specified interval for
all groups in the group set. The bookmark will consist of the group set name that the user
specifies and an automatically incremented number starting at zero, and incremented up
to 65535 (and then, beginning at zero again).
Group sets can be used to perform actions on all of the groups defined in it
simultaneously, such as; enabling and disabling image access, enabling direct access,
undoing writes, failing over, recovering production, starting and stopping transfer, and
enabling or disabling groups.

56

RecoverPoint 5.0 Administrator's Guide

Protection

Display all group sets


Displays all existing group sets in the RecoverPoint system.
Before you begin
l

Only users with Group permissions can add, edit or remove group sets.

The Enable, Disable, Start Transfer, and Pause Transfer buttons are only enabled if
they are relevant for at least one of the groups in the group set.

Procedure
1. Select Protection > Group Sets.

2. All of the group sets in the system are displayed, along with their system information.

Create a group set


Before you begin
l

Parallel bookmarks cannot be enabled for group sets containing consistency groups
with volumes that reside on an XtremIO array or a Data Domain.

Only users with Group permissions can add, edit or remove group sets.

The maximum number of group sets in a RecoverPoint system is limited to 128.


However, each group set can contain an unlimited number of consistency groups,
unless parallel bookmarking is enabled.

While parallel bookmarking is enabled for a group set:


n

The maximum number of groups in the group set is limited to 32.

You cannot add or remove groups from the group set.

The parallel bookmarking interval should be greater than 20 seconds.

You cannot create the group set if any of the groups that you want to include are
already part of another group set that has parallel bookmarking enabled.

You cannot create the group set if any of the groups that you want to include
contain links for which snap-based replication mode has been enabled.

You cannot create a group set if one of its groups is not replicating in the same
direction (that is, from the same source) as other groups in the group set.

Although groups with mixed topologies (that is, with a different number of copies)
can be included in the same group set, for recovery purposes, this is not
recommended.

Display all group sets

57

Protection

Procedure
1. Select Protection > Group Sets .
2. Click Add. The Add Group Set dialog box is displayed.

3. In the Add Group Set dialog box:


a. In the Name field, enter a descriptive name for the group set.
b. In the list of consistency groups, select one or more consistency groups to add to
the group set.
All consistency groups in a group set must be replicating in the same direction.
You cannot enable parallel bookmarking for a group set if any of the groups in the
group set are part of another group set that has parallel bookmarking enabled.
c. Ensure that the Enable Parallel Bookmarking checkbox reflects your requirements.
If it is selected, ensure that the Frequency list box contains the required value.
Enabling parallel bookmarking limits the maximum number of groups in the group
set to 32. You cannot add or remove groups from a group set with parallel
bookmarking enabled. The parallel bookmarking frequency should be no less than
20 seconds. Parallel bookmarks cannot be enabled for group sets containing
consistency groups with volumes that reside on an XtremIO array.
d. Click OK.

Edit a group set


Modifies the settings of an existing group set.
Before you begin
l

Parallel bookmarks cannot be enabled for group sets containing consistency groups
with volumes that reside on an XtremIO array.

Only users with Group permissions can edit group sets.

Procedure
1. Select Protection > Group Sets .
2. Select a group set in the list of group sets.
58

RecoverPoint 5.0 Administrator's Guide

Protection

3. Click Edit. The Edit Group Set dialog box is displayed.

4. In the Edit Group Set dialog box:


a. In the Name field, modify the group set name as required.
Although you can modify the group set name, it is not recommended.
b. Select or clear the checkboxes of groups in the list of groups to include or exclude
them from the groups set.
You will not be able to enable parallel bookmarking for the group set if any of the
groups in the group set are part of another group set that has parallel bookmarking
enabled.
c. Ensure that the Enable Parallel Bookmarking checkbox reflects your requirements.
If it is selected, ensure that the Frequency list box contains the required value.
Parallel bookmarks cannot be enabled for group sets containing consistency
groups with volumes that reside on an XtremIO array. You cannot disable parallel
bookmarking if it is enabled, but you can change the Frequency. Enabling parallel
bookmarking limits the maximum number of groups in the group set to 32. You
cannot add or remove groups from a group set with parallel bookmarking enabled.
The parallel bookmarking frequency should be no less than 20 seconds.
d. Click OK.

Remove a group set


Deletes a group set from the RecoverPoint system.
Before you begin
l

Only users with Group permissions can remove group sets.

This action cannot be undone.

Procedure
1. Select Protection > Group Sets .
2. Select one or more group sets in the list of group sets.
3. Click Remove.

Remove a group set

59

Protection

Enable or disable all groups in a group set


Simultaneously enables or disables all groups in a group set.
Before you begin
l

When transfer starts for a disabled group, a full sweep occurs.

The Enable and Disable buttons are only enabled if they are relevant for at least one
of the groups in the group set.

Procedure
1. Select Protection > Group Sets .
2. Select the group set from the list of group sets.
3. Click Enable or Disable.

Pause or start transfer for all groups in a group set


Simultaneously starts or stops the transfer of writes between the production and all of its
copies for all groups in a group set.
Before you begin
l

After transfer starts, a short initialization occurs.

The Start Transfer and Pause Transfer buttons are only enabled if they are relevant
for at least one of the groups in the group set.

Procedure
1. Select Protection > Group Sets .
2. Select the group set from the list of group sets.
3. Click Start Transfer or Pause Transfer.

Apply a bookmark to all of the groups in a group set


Procedure
1. Select Protection > Group Sets .
2. Select the group set from the list of group sets.
3. Click Apply Bookmark.

Initiate recovery activities for a group set


Simultaneously initiates a recovery activity for all groups in a group set.
You can trigger recovery activities for a group set from both the Protection and the
Recovery menus.
l

60

To test a copy, undo writes, enable direct access or disable image access for all
consistency groups in a group set:
n

Select Protection > Group Sets. Select a group set from the list of group sets, click
Test Copies, and follow the instructions in Test a Copy on page 128.

Select Recovery > Test a Copy and follow the instructions in Test a Copy on page
128.

To test a copy and recover production of all consistency groups in a group set:

RecoverPoint 5.0 Administrator's Guide

Protection

Select Protection > Group Sets. Select a group set from the list of group sets, click
Recover Production, and follow the instructions in Test a Copy and Recover
Production on page 132.

Select Recovery > Recover Production and follow the instructions in Test a Copy
and Recover Production on page 132.

To test a copy and fail over all consistency groups in a group set:
n

Select Protection > Group Sets. Select a group set from the list of group sets, click
Fail Over, and follow the instructions in Test a Copy and Fail Over on page 135.

Select Recovery > Fail Over and follow the instructions in Test a Copy and Fail
Over on page 135.

Manage Protection
Use the Manage Protection screen to manage and monitor your existing consistency
groups and their protection plans, control transfer, and mark consistent points in time
across groups and sets of groups.

When the Manage Protection screen is first displayed, the Consistency Groups node is
selected in the left pane, and the list of all consistency groups in the system is displayed
in the right pane along with the status of every consistency group in the RecoverPoint
system.

Use this screen for monitoring all consistency groups, as follows:

Manage Protection

61

Protection

Table 13 Multiple consistency group monitoring options

Column

Description

Name

The name of the consistency group.

Status

Whether a group is Enabled or Disabled.

Transfer

The group transfer status (Paused, Active, Initializing, etc.). When a group is
configured to transfer data synchronously, the word Synchronized is displayed
next to the state of transfer.
Note

You can also see the data transfer state on page 26 of consistency groups
through the Consistency Group Transfer Status on page 26 widget in the
Dashboard.
Alerts

The number of alerts that have been logged for a group. If alerts have been
logged, click the link to display the alert texts.

Protected Size The size of the data that is being protected by the group.
Active RPA

The RPA that is the preferred RPA.

Group Set

The group set name, if the group has been added to a group set.

You can also perform the following actions on multiple groups in the system
simultaneously, by clicking the relevant button at the bottom of the screen:
Table 14 Multiple consistency group actions

Button

Description

Remove Group Remove the selected consistency groups.


Note

Cannot be undone.
Enable Group
Note

Only enabled if the selected consistency groups are disabled. Enables the
selected consistency groups.
Enabling a disabled consistency group starts replication and causes a full
sweep.
Disable Group

Only enabled if the selected consistency groups are enabled. Disables the
selected consistency groups.
Note

Disabling a consistency group stops all replication, deletes journals, and causes
a full sweep when the group is enabled.
Create
Bookmark

62

RecoverPoint 5.0 Administrator's Guide

Create a bookmark and define snapshot policies across multiple groups on page
123

Protection

The RecoverPoint protection policies


RecoverPoint protection is policy-driven. A protection policy, based on the particular
business needs of your company, is uniquely specified for each consistency group, each
copy (and copy journal) and each link. Each policy comprises settings that collectively
govern the way in which replication is carried out. Replication behavior changes
dynamically during system operation in light of the policy, the level of system activity,
and the availability of network resources.
Note

There are various advanced protection policies that cannot be accessed through
Unisphere for RecoverPoint, and can only be configured using the RecoverPoint CLI. See
the EMC RecoverPoint CLI Reference Guide for your RecoverPoint product version for more
information on these policies.

Group policies
Describes the policy settings for consistency groups.
To display these settings, select Protection > Manage Protection, select a consistency
group in the left pane, and click the Group Policy tab in the right pane.
Table 15 Group policies

Setting

Description

Group Name

The name of the consistency group.

Primary RPA

The RPA that you prefer to replicate the consistency group. When the primary RPA
is not available, the consistency group will switch to another RPA in the RPA
cluster. Whether data will transfer when replication is switched to another RPA
depends on the value of the transfer_by_non_preffered parameter of the
config_group_policy CLI command.
Note

Best practice is to ensure that groups that replicate in synchronous replication


mode are set to use different RPAs than groups that replicate in asynchronous
replication mode . Mixing between the two may result in low I/O rates for the
synchronous groups. It is also recommended that dynamic sync and purely
synchronous consistency groups reside on different RPAs, whenever possible.

Priority

Default = Normal
Only relevant for remote replication over the WAN or Fibre Channel, when two or
more consistency groups are using the same Primary RPA.
Select the priority assigned to this consistency group. The priority determines the
amount of bandwidth allocated to this consistency group in relation to all other
consistency groups using the same Primary RPA.
Possible values are:
l

Idle

Low

Normal

The RecoverPoint protection policies

63

Protection

Table 15 Group policies (continued)

Setting

Description
l

High

Critical

In asynchronous replication, groups with a priority of Critical are provided ten


times the priority of normal groups. Groups with a priority of High are provided
three times the priority of normal groups. Groups with a priority of Low are
provided 50% of the priority of normal groups. Groups with a priority of Idle are
provided 1% of the priority of normal groups.

Preferred
Cluster

Note

Only relevant when MetroPoint group is selected.


Default = Follow VPLEX bias rules
Sets the preferred RPA cluster of the MetroPoint group to the active or standby
production, or to follow the VPLEX bias rules.
Default=disabled
Distribute
group writes
Only supported if the journal is at least 40 GB.
across
multiple
Note
RPAs

Both enabling and disabling this setting causes the journal of all copies in the
consistency group to be lost.

Distributed consistency groups across multiple RPAs to significantly increase the


maximum available RPA throughput, allowing for a significantly larger group. For
throughput performance statistics (during synchronous and asynchronous
replication) and feature limitations, see the EMC RecoverPoint Performance Guide.
When enabled, a minimum of one and a maximum of three secondary RPAs can
be selected.
Note

Before changing this setting, ensure all preferred RPAs (both primary and
secondary) are connected by Fibre Channel and can see each other in the SAN .

External
Application

Recovery
Copy

64

RecoverPoint 5.0 Administrator's Guide

Default=none
Possible values are None and SRM.
l

None: Using an external management application (SRM) is disabled.

SRM: To enable support for VMware Site Recovery Manager. This option is
only valid if a RecoverPoint Storage Replication Adapter for VMware Site
Recovery Manager is installed on the vCenter Servers. For more information,
refer to the EMC RecoverPoint Adapter for VMware Site Recovery Manager
Release Notes.

Specifies which copy the external application should fail over to.

Protection

Table 15 Group policies (continued)

Setting

Description

External
Only relevant for MetroPoint configuration and if External Application = SRM.
Management
SRM supports only one production copy. When RecoverPoint is managed by
Side

external management, the external management application (SRM) must be


connected via a fixed management IP address. If the other cluster becomes
active, SRM will fail. External Management Side determines the RecoverPoint IP
address of which RecoverPoint production copy is used for management by SRM.
This IP address cannot be changed, even if the active production copy changes
over to the other side, and SRM therefore fails.
The External Management Side parameter cannot be changed unless:
l

RecoverPoint is replicating correctly.

In vCenter, neither SRM Test nor a Recovery Plan is running.


After changing the External Management Side parameter, Rescan Scan must
be run in vCenter, otherwise SRM will fail.

Managed by

When any value other than None is selected, Managed By is enabled and you
have the following options:
l

RecoverPoint: Check this option for planned or unplanned maintenance of


the RecoverPoint system. When activated, external application support is
disabled and user-initiated RecoverPoint capabilities are enabled. When
activated, RecoverPoint user-initiated capabilities, such as image access,
image testing, changing policies and creating bookmarks are available.

External Application: When activated, the specified external application


manages RecoverPoint. All RecoverPoint user-initiated capabilities are
disabled. The user cannot access images, change policies, or change
volumes. Bookmarks cannot be created in the RecoverPoint GUI, but they
can be created using the RecoverPoint CLI bookmark commands.
When Managed by = External Application, Site Recovery Manager manages
the group, and can perform automatic failover and test failover.

Link policies
Describes the policy settings for the links between two copies.
Table 16 Link policies

Setting

Description

Asynchronous

Default=enabled
When enabled, RecoverPoint replicates the consistency group data in
asynchronous replication mode.

Snap Replication

Default=disabled
When enabled, RecoverPoint replicates the consistency group data in
asynchronous, snap-based replication mode.
Possible values:
The RecoverPoint protection policies

65

Protection

Table 16 Link policies (continued)

Setting

Description
l

None: Disables snap-based replication mode.

Periodic: A new snap starts after the specified interval has passed
since the previous snap was started. If the time interval has passed,
but the previous snap has not been finished, the next snap will start
as soon as the previous one is finished. Sets the interval between
snaps to any value between 30 and 1440 minutes. When set to
Periodic, an interval should be defined. The default interval is 30
minutes. The RPO will be twice the specified interval, and cannot be
changed.
Note

The recommended snap replication mode for XtremIO replication is


Periodic. For sizing guidelines when using Continuous snap-based
replication mode consult EMC Customer Support.

Synchronous

On Highload: Uses asynchronous replication mode as the default


replication mode and dynamically switches to snap-based
replication mode upon high load. When highload is over,
dynamically switches back to asynchronous replication mode .

Continuous: A new snap starts as soon as the previous snap is


completed. Specify the RPO. RPO is determined by the XtremIO
snapshot pruning policy, which cannot be changed and by the
Maximim Number of Snaps specified. The RPO specified here can
shorten the RPO determined by the other parameters, but cannot
lengthen it. Default RPO = 1 minute.

Note

Not relevant in ProtectPoint and XtremIO.


Default=disabled
When enabled, RecoverPoint replicates the consistency group data in
synchronous replication mode.

Dynamic by latency

Note

Not relevant in ProtectPoint.


Default=disabled
Only relevant for synchronous replication mode.
When enabled, RecoverPoint uses dynamic sync mode and alternates
between synchronous and asynchronous replication modes, as
necessary, according to latency conditions (the number of milliseconds
or microseconds between the time the data is written to the local RPA
and the time that it is written to the RPA or journal at the remote site).
l

66

RecoverPoint 5.0 Administrator's Guide

Start async replication above: When the specified limit is reached,


RecoverPoint automatically starts replicating in asynchronous
replication mode.

Protection

Table 16 Link policies (continued)

Setting

Description
l

Dynamic by
throughput

Resume sync replication below: When the specified limit is reached,


RecoverPoint goes back to replicating in synchronous replication
mode.

Note

Not relevant in ProtectPoint.


Default=disabled
Only relevant for synchronous replication mode.
When enabled, RecoverPoint uses dynamic sync mode and alternates
between synchronous and asynchronous replication modes, as
necessary, according to throughput conditions (the total writes that
reach the local RPA, per copy, in kb/s).

RPO

Start async replication above: When the specified limit is reached,


RecoverPoint automatically starts replicating in asynchronous
replication mode.

Resume sync replication below: When the specified limit is reached,


RecoverPoint goes back to replicating in synchronous replication
mode.

Default = 25 seconds
This setting defines the required lag of the each link in a consistency
group, and is set manually in MB, GB, writes, seconds, minutes, or hours.
In RecoverPoint, RPO starts being measured when a write made by the
production host reaches the local RPA, and stops being measured when
the write reaches either the target RPA or the target journal (depending
on the value of the transfer_by_non_preferred parameter of the
config_group_policy CLI command).
When the value of the regulate_application parameter of the
config_link_policy CLI command is set to no (default is no), the
specified RPO is not guaranteed. RecoverPoint will try its best to replicate
within the specified RPO without affecting host performance.

Compression

Note

Not relevant in ProtectPoint.


Default = Low
To compress data before transferring it to a remote RPA cluster. Can
reduce transfer time significantly.

The RecoverPoint protection policies

67

Protection

Table 16 Link policies (continued)

Setting

Description
Note

Only relevant for asynchronous remote replication. Both the enabling and
disabling of compression causes a short pause in transfer and a short
initialization. Compression decreases transfer time, but increases the
source RPAs CPU utilization.
l

High: Yields the highest bandwidth reduction ratio and requires the
most RPA resources.

Medium: Yields an average bandwidth reduction ratio and requires


an average amount of RPA resources.

Low: Yields the lowest bandwidth reduction ratio and requires the
least RPA resources.

None: Compression is disabled and requires no RPA resources.


See the EMC RecoverPoint Release Notes for the supported throughput
for each compression level.

Enable
Deduplication

Note

Not relevant in ProtectPoint.


Default = disabled
To eliminate repetitive data before transferring the data to a remote RPA
cluster. Can reduce transfer time significantly.
Note

Only relevant for asynchronous remote replication. Compression must


be enabled before deduplication can be enabled. Both the enabling and
disabling of deduplication causes a short pause in transfer and a short
initialization. Deduplication decreases transfer time, but increases the
source RPAs CPU utilization.

Snapshot
Granularity

Note

Not relevant in ProtectPoint and XtremIO.


Default = fixed (per second)

68

RecoverPoint 5.0 Administrator's Guide

Fixed (per write): To create a snapshot for every write operation, over
a specific (local or remote) link.

Fixed (per second): To create one snapshot per second, over a


specific (local or remote) link.

Dynamic: To have the system determine the snapshot granularity of


a specific (local or remote) link, according to available resources.

Protection

Table 16 Link policies (continued)

Setting

Description
Note

When you use distributed consistency groups, the snapshot


granularity of all links in the consistency group can be no finer than
one second.

Copy policies
To display the copy policy settings, select Protection > Manage Protection, expand a
consistency group, select one of its copies, and click the Copy Policy tab.
Table 17 Copy policies

Setting

Description

Host OS

Default = Other/Mixed
The operating system of the host writing to the volumes in the consistency group.

Clear journal
Note
upon
Only relevant in ProtectPoint
removal

Journal policies
To display the journal policies, select Protection > Manage Protection, expand a
consistency group and select one of its copies. Then, click the Copy Policy tab and note
the settings in the Journal Policy section.
Table 18 Journal policies

Setting

Description

Journal
Compression

Note

Not relevant in ProtectPoint and XtremIO.


Default = none
When enabled, instructs the target RPA of the consistency group to compress
the snapshots in the journal so that more images (that is, a longer protection
window) can be saved in the same journal capacity, saving storage cost.
Compression impacts the CPU resources of the target RPA of the consistency
group, and can impact that RPAs ability to sustain its write-load. If the target
RPA of the consistency group for which you wish to enable this option is also
transferring the data of other consistency groups across the WAN, note that
enabling this setting will affect the RPAs transfer rate. See the EMC
RecoverPoint Release Notes for throughput limitations.

The RecoverPoint protection policies

69

Protection

Table 18 Journal policies (continued)

Setting

Description
Note

Enabling journal compression while a consistency group is enabled will result


in the loss of all snapshots in the journal. To change the value of journal
compression, the consistency group must be in one of the following states:
disabled, direct image access, or distributing.

Maximum
Journal Lag

Note

Not relevant in ProtectPoint and XtremIO.


Default = unlimited
Defines the data access aspects of RTO for each copy.
Note

The industry term RTO includes the time during which the disaster was being
fixed, the actual disaster recovery, testing, and communication to the users.
The RTO setting in RecoverPoint only defines the actual disaster recovery
time.
When data is received by the RPA faster than it can be distributed to storage
volumes, it accumulates in the journal. The Maximum Journal Lag is the
maximum amount of snapshot data (in bytes, KB, MB, or GB) that is
permissible to hold in the copy journal before distribution to the copy
storage. In other words, the maximum amount of data that would have to be
distributed to the copy storage before failover to the latest image could take
place. In terms of RTO, this is the maximum time that would be required in
order to bring the copy up-to-date with production.
When the Maximum Journal Lag value is reached, the system switches to
three-phase distribution mode, and no longer retains rollback information. As
soon as the lag is within the allowed limits, rollback data is retained again.

Enable
RecoverPoint
snapshot
consolidation

Note

Not relevant in ProtectPoint and XtremIO.


Default = disabled
When selected, enables automatic snapshot consolidation to allow for the
storage of a longer history in the copy journal.
Automatic snapshot consolidation cannot be enabled for a group that is part
of a consistent group set.
When enabled, the Predicted Protection Window is not calculated.

Do not consolidate any snapshots for at least


Default = 2 days
Minimum = 12 hours.
The period during which snapshot data is not to be consolidated. The
periods start time is always today, and the periods end time is <n>
hours / days / weeks / months ago.

70

RecoverPoint 5.0 Administrator's Guide

Protection

Table 18 Journal policies (continued)

Setting

Description
If no daily or weekly consolidations are specified, the remaining
snapshots are consolidated monthly.

Consolidate snapshots that are older than x to one snapshot per day
for y days
Default = 5 days
Snapshots are consolidated every ~24 hours.
Select the Indefinitely checkbox to consolidate all subsequent snapshots
in ~24 hour intervals.
l

If the Indefinitely checkbox is not selected, and no weekly


consolidations are specified, the remaining snapshots are
consolidated monthly.
If the Indefinitely checkbox is selected, weekly and monthly
consolidations are disabled, and the remaining snapshots are
consolidated daily.
Note

The snapshot consolidation process will fail if at least 24 hours


worth of data do not exist in the journal in addition to the
unconsolidated duration, or if there is not a minimum of 1 GB of
data between the starting and closing points of the consolidation.

Consolidate snapshots that are older than x to one snapshot per


week for y weeks
Default = 4 weeks
Snapshots are consolidated every ~168 hours.
Select the Indefinitely checkbox to consolidate all subsequent snapshots
in ~168 hour intervals.

Maximum
Number of
Snaps

If the Indefinitely checkbox is selected, the remaining snapshots are


consolidated weekly. The first weekly consolidation cannot happen
unless there are at least seven daily snapshots in the journal, in
addition to the required number of weekly snapshots and the
specified unconsolidated duration.

If the Indefinitely checkbox is not selected, the remaining snapshots


are consolidated monthly. The first monthly consolidation cannot
happen unless there are at least four weekly snapshots, or 28 daily
snapshots in the journal in addition to the required number of
monthly snapshots.

Note

Only relevant for XtremIO.


Default=80

The RecoverPoint protection policies

71

Protection

Table 18 Journal policies (continued)

Setting

Description
The maximum number of snaps to be saved at the copy (2500). The value of
this setting may change if the detailed consolidation policy is modified.

Required
Protection
Window

Default = disabled
Note

In XtremIO, this setting is enabled by default, with a value of 2 days, and it


cannot be disabled.
Indicates how far in time the copy image can be rolled back.
When a required protection window is specified, the status of the current
protection window and the predicted protection window are displayed in the
Journal tab of the copy.

Edit Detailed
Consolidation
Settings

Note

Only relevant for XtremIO copies.


Displays the number of snapshots to retain and consolidate per time duration
(specified in minutes, hours, days, weeks, or months.
The detailed consolidation policy is deduced by RecoverPoint when a
Maximum Number of Snaps and a Required Protection Window is
defined, or when an Snap-based replication interval is defined in the copy
Table 16 on page 65. One to three consolidation policies may be deduced.
You can specify between one and three detailed consolidation policies in
parallel. The system uses the entered consolidation policy values as
guidelines. The actual number of snapshots per timeframe is displayed
beside each policy.
Note

For the detailed consolidation policy values to be valid, the first consolidation
policy in the list must be of the shortest duration and the last policy in the list
must be of the longest duration. The first average snapshot frequency in the
list must be lower than the last.

72

RecoverPoint 5.0 Administrator's Guide

Protection

Monitoring and managing protection


After a consistency group has been created, use the Manage Protection screen to monitor
and modify the group's settings and policies.

Use Manage Protection for:


l

Monitoring and managing groups on page 73

Monitoring and managing copies on page 87

Monitoring group performance on page 76

Monitoring copy journals on page 91

Modifying consistency groups on page 97

Modifying copies on page 112

Modifying links on page 114

Modifying replication sets on page 115

Modifying journals on page 118

Modifying snapshots on page 120

Monitoring and managing groups


Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select a group.
3. In the right pane, select the Status tab.

Monitoring and managing protection

73

Protection

Results

The Status tab displays a graphical representation of the consistency group and copy
statuses. The following information is displayed:
Table 19 Status tab

Label

Displays

<Group
Name>
Running on
<RPA>

The group name and the RPA which is handling the group data, both of which can
be modified through the group policies. Group policies on page 63 includes
more information about group policies.

<Copy Name>
at <Cluster
Name>

The relevant copy and RPA cluster names. The copy name can be modified
through the copy policies. Copy policies on page 69 includes more information
about copy policies.

Replication
modes and
states

A visual representation of the current replication modes and states.


l

A dashed green line means that the system is replicating asynchronously.

A dashed green line on top of a solid line means that the system is
replicating synchronously.

A greyed out line means that replication has stopped.

Note

The lines move in the direction of replication.


The transfer state is displayed in the center of each link between the production
and the copy. The transfer state displayed here correlates to what is displayed in
the CG Transfer State on page 26 widget of the system Dashboard.
When replicating synchronously, the word synchronized is displayed.

Role

74

RecoverPoint 5.0 Administrator's Guide

The current role of the copy with regards to failover and regulation.
l

Before failover: Production, Local Copy and Remote Copy

After failover: Target at Production, Local Source and Remote Source

During regulation: Regulated

Protection

Table 19 Status tab (continued)

Label

Displays

Storage

The image access modes.

Image

The image currently being distributed to storage in the distribution phase.

When a group is selected in the left pane, you can click the buttons at the bottom of the
Status tab to perform the following actions:
Table 20 Status tab options when a group is selected

Button

Description

Create
Bookmark

Create a bookmark and define snapshot policies for one group on page 122 by
providing a bookmark name for the current snapshot.

Add Copy

Displays the Add a copy to a group on page 99 wizard, enabling you to add a
copy to the selected consistency group.

Test Copy

Displays the Test a Copy on page 128 wizard, enabling you to select and test a
copy of the selected consistency group.

Fail Over

Note

Not relevant in ProtectPoint


Displays the Test a Copy and Fail Over on page 135 wizard, enabling you to
select and test a copy of the selected consistency group, as well as fail over to it
permanently or temporarily.

Recover
Production

Displays the Test a Copy and Recover Production on page 132 wizard, enabling
you to select and test a copy of the selected consistency group, and recover
production from it.

Start
Transfer

Only displayed if transfer is paused for the selected consistency group. Causes
the transfer of writes from the production host to all copies of the selected
consistency group to start.

Pause
Transfer

Only displayed if transfer is active for the selected consistency group. Causes the
transfer of writes from the production host to all copies of the selected
consistency group to pause.
Note

Best practice is not to use this command. Use Group policies on page 63 to set
policies for use of bandwidth. Pause transfer may be used when WAN bandwidth
is very limited and you wish to give the largest bandwidth possible to another
consistency group. In that case, you may temporarily pause transfer for lowerpriority consistency groups.

Disable
Group

Only displayed if the group is enabled for the selected consistency group.
Disables the selected consistency group.
Note

Disabling a consistency group stops all replication, deletes journals, and causes
a full sweep when the group is enabled.
Monitoring and managing groups

75

Protection

Table 20 Status tab options when a group is selected (continued)

Button

Description

Enable
Group

Only displayed if the group is disabled for the selected consistency group.
Enables the selected consistency group.
Note

Enabling a disabled consistency group starts replication and a full sweep


commences.

Remove
Group

Removes the selected consistency group from the system.


Note

Cannot be undone.

Monitoring group performance


Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select a group.
3. In the right pane, select the Statistics tab.

The following traffic statistics are displayed in the Traffic sub-tab:


Note

When the production copy is on an XtremIO array, only Total Traffic and Initialization
Traffic statistics are available.

76

The Total Traffic widget displays the total traffic that the consistency group data
generates on the RPA transferring the group data at the production and copy sites,
as well as between RPA clusters in your RecoverPoint system (in Mb/sec). The total
traffic includes the Initialization Traffic and the Application Traffic.

The Application Traffic widget displays the traffic that the consistency group
generates on the RPA transferring the group data at the production and copy sites,

RecoverPoint 5.0 Administrator's Guide

Protection

as well as between RPA clusters in your RecoverPoint system (in Mb/sec). The
application traffic includes Incoming Writes as well as RecoverPoint replication
process traffic.
l

The Initialization Traffic widget displays the traffic that the consistency group
generates between copies and RPA clusters in your RecoverPoint system, during
consistency group initialization (in Mb/sec). Initialization traffic can be high, as
old writes are transferred between source and target copies to synchronize
between the production volumes and the copy volumes.

The Incoming Writes widget displays the writes to the production storage that the
consistency group generates at copies in your RecoverPoint system (in writes per
second), where each write is one I/O to the production storage.

The following statistics are displayed in the Replication Performance sub-tab:

The Bandwidth Reduction widget displays the ratio by which RecoverPoint was
able to compress the consistency group data that is transferred between RPA
clusters in your RecoverPoint system. The bandwidth reduction ratio includes both
data compression and deduplication.
Note

Compression and deduplication are only relevant for asynchronous replication to a


remote RPA cluster.
The gains realized will vary depending on the compressibility of the replicated
data. Typically, enterprises can save 50% 90% of their bandwidth by enabling
the compression setting.
NOTICE

While enabling compression saves bandwidth, it also increases RPA CPU


utilization, which lowers RPA throughput. See the Performance section of the
EMC RecoverPoint Release Notes for the impact of each compression setting on RPA
throughput. Also, both the enabling and disabling of compression causes a short
pause in transfer and a short initialization.

Monitoring and managing groups

77

Protection

The Time Lag widget displays the lag of the selected consistency group (in
seconds), when transferring data between all local and remote RPA clusters in your
RecoverPoint system.

The Write Lag widget displays the lag of the selected consistency group (in writes),
when transferring data between all local and remote RPA clusters in your
RecoverPoint system. The write lag includes any writes generated by RecoverPoint
replication processes as well as internal RecoverPoint overhead.

The Data Lag widget displays the lag of the selected consistency group (in MB),
when transferring data between all local and remote RPA clusters in your
RecoverPoint system.

Enabling deduplication
Before you begin
l

Deduplication is only relevant for asynchronous remote replication.

Both the enabling and disabling of deduplication causes a short pause in transfer
and a short initialization.

Data compression must be enabled before deduplication can be enabled.

Deduplication is only supported in RecoverPoint environments in which all (local and


remote) RPAs are:
n

Physical RPAs with at least 8GB of RAM

Virtual RPAs with at least 4GB RAM

Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select a consistency group.
3. In the right pane, click the Link Policy tab.
a. In the Select link to set policy diagram, select the relevant link.
b. In the Bandwidth Reduction section, if Compression is set to None, enable
compression by changing this value to Low, Medium, or High.
c. Check the Enable Deduplication checkbox.
4. Click the Apply button.
NOTICE

There is a short pause in transfer, followed by a short initialization.


After you finish
To display the effects of deduplication on your bandwidth:

78

Display short-term deduplication statistics on page 79the system displays the


current (sampled over the past seconds) and average (sampled over the past
minutes) performance statistics.

Display long-term deduplication statistics on page 79the user defines the period
in time for which to generate performance statistics.

RecoverPoint 5.0 Administrator's Guide

Protection

Note

A bandwidth reduction or deduplication ratio value that is not significantly higher than 1
would indicate that the deduplication (and/or compression) process is not very effective.
If so, consider disabling these policies (as they do consume RPA CPU resources).

Display short-term deduplication statistics


To display the short-term effects of deduplication on your bandwidth:
Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select a consistency group.
3. In the right pane; click the Statistics tab and click on the Replication Performance tab.
Note the performance statistics in the Bandwidth Reduction pane. The Bandwidth >
Reduction pane displays the current bandwidth reduction ratio, including
compression and deduplication.
1. To display the short-term effects of deduplication on your bandwidth through the
RecoverPoint CLI:
4. Run the get_group_statistics command.
5. Note the Current bandwidth reduction ratio and Average bandwidth reduction ratio
values. These values are the current (seconds) and average (minutes) bandwidth
reduction ratios, including compression and deduplication.
6. Note the Current deduplication ratio, and Average deduplication ratio values. These
values are the current (seconds) and average (minutes) bandwidth reduction ratios, of
only the deduplication process.

Display long-term deduplication statistics


The long-term effects of deduplication on your bandwidth can only be monitored through
the RecoverPoint CLI.
To display the long-term effects of deduplication on your bandwidth:
l

Detect bottlenecks on page 201

Export consolidated statistics on page 82

Both bottleneck detection and exporting consolidated statistics allows you to define the
period in time for which to generate long-term bandwidth reduction and deduplication
ratios.

Configuring lag optimization


Describes how to set the lag optimization of a specific consistency group.
Before you begin
l

You should:
n

Document the RPO of your consistency groups (for example, according to


regulation rules).

Evaluate the importance and value of your consistency groups data in relation to
each other, and verify that the default values of the priority on page 63 setting (as
described in the following table) reflect your business objectives. If they do not,
contact EMC Customer Support and ask them to change the values.

Monitoring and managing groups

79

Protection

Table 21 Priority setting default values

Setting Value
Idle

0.01 x Normal

Low

0.5 x Normal

Normal

Normal

High

3 x Normal

Critical

10 x Normal

Note

Changing the default values is optional, as they provide a practical configuration.


It is recommended to accept the default settings unless there is a specific
business need to set other policies.
l

Lag optimization is performed per RPA.

The lag is optimized only if the bandwidth is the cause of the lag. It is not relevant, for
example, if the lag is caused by the distribution process at the remote site (which can
happen when replicating over Fibre Channel).

The lag optimizer is only relevant for groups replicating asynchronously, that are not
experiencing a high load or an initialization. Groups replicating synchronously are
always prioritized over groups replicating asynchronously.

Procedure
1. Select Protection > Manage Protection
2. In the left pane, select a consistency group.
3. In the right pane, select the Group Policy tab
4. Specify the group Policy setting.
5. In the left pane, expand the group and select a copy.
6. Specify the copy RPO policy, according to your company's business objectives.

Distributing group writes across RPAs


Before you begin
NOTICE

When modifying a groups topology, journal loss will occur.

80

The snapshot granularity of all links in the consistency group can be set in the link
policies to no smaller than one second.

Journal loss will occur when modifying a groups topology (setting a non-distributed
group as distributed, or setting a distributed group as non-distributed).

When configuring journals for a distributed consistency group, keep the following in
mind:
n

All copies of distributed consistency groups must have journals that are at least
20 GB in size.

The recommended journal size for distributed groups with snapshot consolidation
enabled is at least 40GB.

RecoverPoint 5.0 Administrator's Guide

Protection

If the capacity of an existing copy journal is less than the minimum journal size
required for distributed consistency groups (see the EMC RecoverPoint Release
Notes for this limit) the consistency group will need to be disabled and then
enabled again after you add journal volumes or change the size of a journal LUN
on storage
Enabling a disabled group causes a full sweep.

Distributed consistency groups are only supported if there is a Fibre Channel


connection between all RPAs in each RPA cluster in the RecoverPoint system,
therefore:
n

In Fibre Channel environments, make sure all of the RPAs at each RPA cluster are
connected to the SAN through a Fibre Channel switch, and zoned together so that
they all see each other in the SAN.

Groups running on vRPAs cannot be distributed.

If any of the primary or secondary RPAs associated with a consistency group becomes
unavailable, there will be a brief pause in transfer on all of the groups primary and
secondary RPAs, and all of the group segments will undergo a short initialization.

Under certain circumstances (for example, if one of the primary or secondary RPAs
becomes unavailable) two consistency group segments could be handled by the
same RPA.

In general, distributed consistency groups offer better performance than nondistributed (regular) consistency groups, as distributed groups run on a minimum of
two RPAs (one primary RPA and one secondary RPA). There is only a small
improvement in performance when a group is run on three RPAs. However, there is a
steep improvement in performance when a group is run on four RPAs.

The following are example journal configurations for distributed consistency groups:
n

One 40 GB volume + optional smaller volumes

Two 20 GB volumes + optional smaller volumes

Three 16 GB volumes + optional smaller volumes

Four 10 GB volumes + optional smaller volumes

Procedure
1. Review the relevant limitations and performance statistics of distributed groups in the
EMC RecoverPoint Release Notes
2. Select Protection > Manage Protection.
3. In the left pane, select a consistency group.
4. In the right pane, click the Group Policy tab
5. Check or clear the Distribute group checkbox. The Secondary RPA checkboxes are
enabled or disabled accordingly.
6. If you are enabling this feature, select one to three secondary RPAs, by checking the
relevant checkboxes in the Secondary RPA section.
Results
The group writes are distributed across the selected RPAs.
After you finish
To monitor each group segments performance separately, and in relation to the group as
a whole, detect botttlenecks on page 201.
You can tell whether or not a group is distributed by navigating to Protection > Manage
Protection, clicking on a consistency group name in the left pane and looking at the top
of the consistency groups Status tab. If the group is distributed, the text (primary)
Monitoring and managing groups

81

Protection

followed by a comma-separated list of numbers indicating the designated secondary


RPAs is displayed.

Figure 6 Identifying a distributed group

If the group is not distributed, only one RPA is specified in this area. You can also tell
which of all of your groups are distributed by navigating to Protection > Manage
Protection, clicking on Consistency Groups in the left pane and looking in the Active RPA
column of the consistency group table. Distributed consistency groups are displayed with
the text (primary) followed by a comma-separated list of numbers indicating the
designated secondary RPAs.

Figure 7 Identifying a distributed group

For groups that are not distributed, only one RPA is specified in this column.

Exporting consolidated statistics


The export_consolidated_statistics CLI command provides data series for a selection of
important RecoverPoint operational statistics. It enables advanced users, customer
support representatives, and implementation specialists to analyze system traffic and
workload trends, to identify correlation between spikes in two or more settings, and to
discover the root causes of high loads and other significant system behaviors.
With the export_consolidated_statistics command, you specify the granularity at which to
collect statistics (minute, hour, and/or day) and, for each granularity, the time frame over
which to collect the statistics. The resulting CSV file organizes the output for each entry
according to the standard bottleneck detection settings.
Procedure
1. Create a secure SSH connection to the RecoverPoint CLI and log in one of the predefined users with admin privileges.
2. Run the export_consolidated_statistics command.
3. When prompted, specify the required time frame and report granularity.
The consolidation statistics are collected, placed in multiple comma-separated-value
(*.csv) files, and archived into one compressed file with the extension tar.gz.

82

RecoverPoint 5.0 Administrator's Guide

Protection

4. Open an Internet browser window and connect to an RPA with the address:
http://<RPA IP address>/info
Note

You can connect to the RPA through either an HTTP or an HTTPS connection.
5. When prompted, log in to the RPA using boxmgmt privileges.
6. In the browser, navigate to the following directory:
../info/long_term_stats
7. Download and unzip the long_term_stats.tar.gz file into a folder on your local
machine.
Results
The following table lists the statistics that are output by the
export_consolidated_statistics command. All statistics are valid for the sampled time
interval, that is, the difference between the time from and time to values on that row.
Table 22 Consolidated statistics output

Statistic

Unit of
measure

Description

Incoming writes rate for


link

MB/sec

Throughput of writes that arrived at a link on the


production side.

Incoming IOs rate for


link

IOs/sec

Number of writes that arrived at a link on the production


side. Average IO size can be computed as (Incoming
writes rate for link) / (Incoming IOs rate for link).

Non - initialization
output rate for link

MB/sec

Rate at which normal write traffic was transferred for the


link.

Initialization output rate MB/sec


for link

Rate at which data was actually transferred for the link


for purpose of synchronization.

Data synchronization
rate for link

MB/sec

Rate at which data was checked for possible transfer for


the link for purpose of synchronization. In first-time
initialization, this is the rate at which the data from the
replication set volumes is transferred. In other cases,
comparison of signatures increases the rate.

Compression CPU
utilization

Depending on the relevance (link, site or RPA), either:


l

Link: Portion of processor time used to compress


the incoming data over a link. This statistic is
relevant only for remote links.

Site: Portion of processor time used to compress the


incoming data over all links for all RPAs at a site.

RPA: Portion of processor time used to compress the


incoming data over all links on the RPA.

Percentage time in
transfer

% of time

Portion of time link was in active transfer state.

Monitoring and managing groups

83

Protection

Table 22 Consolidated statistics output (continued)

84

Statistic

Unit of
measure

Description

Percentage time of
initialization

% of time

Portion of time link was in initializing transfer state.

RPO - lag in time


sec
between replicas during
transfer after init

Actual recovery point objective for link, as measured in


seconds. It is measured only when the transfer state for
the link was active.

RPO - lag in data


MB
between replicas during
transfer after init

Actual recovery point objective for link, as measured in


megabytes of data. It is measured only when the transfer
state for the link was active.

RPO - lag in IOs


IOs
between replicas during
transfer after init

Actual recovery point objective for link, as measured in


number of IOs. It is measured only when the transfer
state for the link was active.

Group-Link utilization

Aggregate measure of portion of RPA capacity used by


the link, based on number of IOs, writes throughput, and
CPU utilization.

Distributor receiver
regulation duration

% of time

Depending on the relevance (link, site or RPA), either:


l

Link: Portion of time that the distribution process for


a copy was forced to regulate the incoming data rate.
A high value indicates a slow storage journal
volumes relative to the rate of the incoming data.

RPA: Portion of time that the distribution process


across all copies served by the RPA was forced to
regulate the incoming data rate. A high value
indicates slow storage journal volumes relative to
the rate of the incoming data.

Site: Portion of time that the distribution process


across all copies at a site was forced to regulate the
incoming data rate. A high value indicates slow
storage journal volumes relative to the rate of the
incoming data.

Distributor phase 1
thread load

% of time

Portion of time used to receive data and write it to the


journal volumes for the link. The value is dependent on
the performance of the journal volumes.

Distributor phase 1
effective speed

MB/sec

Depending on the relevance (link, site or RPA), either:

RecoverPoint 5.0 Administrator's Guide

Link: Rate at which data was received and written to


the journal volumes for the link. The value is
dependent on the performance of the journal
volumes.

RPA: Rate at which data was received and written to


the journal volumes for all copies served by the RPA.
The value is dependent on the performance of the
journal volumes.

Site: Rate at which data was received and written to


the journal volumes for all copies at the site. The

Protection

Table 22 Consolidated statistics output (continued)

Statistic

Unit of
measure

Description
value is dependent on the performance of the
journal volumes.

Distributor phase 2
thread load

% of time

Portion of time used to read data from the journal


volumes and write to the copy volumes, using either
normal or fast-forward distribution mode. The value is
dependent on the performance of the journal and copy
volumes.

Distributor phase 2
effective speed

MB/sec

Depending on the relevance (link, site or RPA), either:


l

Link: Rate at which data was read from the journal


volumes and written to the copy volumes, using
either normal or fast-forward distribution mode. The
value is dependent on the performance of the
journal and copy volumes.

RPA: Rate at which data was read from the journal


volumes and written to the copy volumes for all
copies served by the RPA, using either normal or
fast-forward distribution mode. The value is
dependent on the performance of the journal and
copy volumes.

Site: Rate at which data was read from the journal


volumes and written to the copy volumes for all
copies at a site, using either normal or fast-forward
distribution mode. The value is dependent on the
performance of the journal and copy volumes.

Fast forward
distribution duration

% of time

Portion of time at copy when distribution was in fastforward mode.

WAN throughput from


box (over IP)
or
Cross-site throughput
from box (over FC)

Mb/sec

Outgoing throughput from RPA to remote site.

Total incoming writes


rate for box

MB/sec

Throughput of writes that arrived at an RPA on the


production side. For concurrent local and remote
replication, this is the sum of throughputs for all local
and remote links on this RPA (which is double the actual
incoming writes rate).

Incoming IOs rate for


box

IOs/sec

Number of writes that arrived at an RPA on the


production side. For concurrent local and remote
replication, this is the sum of incoming IOs for all local
and remote links (which is double the actual number of
incoming writes). Average IO size can be computed as
(Total incoming writes rate for box) / (Incoming IOs rate
for box).

Monitoring and managing groups

85

Protection

Table 22 Consolidated statistics output (continued)

86

Statistic

Unit of
measure

Description

Non - initialization
output rate for box
(average over all
period)

MB/sec

Rate at which normal write traffic was transferred by the


RPA.

Initialization output rate MB/sec


for box (average over all
period)

Rate at which data was actually transferred by the RPA


for purpose of synchronization.

Data synchronization
rate for box (average
over all period)

MB/sec

Rate at which data was checked for possible transfer by


the RPA for purpose of synchronization. In first-time
initialization, this is the rate at which the data from the
replication set volumes is transferred. In other cases,
comparison of signatures increases the rate.

Replication process
CPU utilization per
box

Portion of processor time used for the replication process


on the RPA, including compression and other replication
activities.

SAN target thread load

% of time

Portion of time used by RPA for initial processing of


writes, before assigning them to the relevant links.

Box utilization

Aggregate measure of portion of RPA capacity used,


based on both IO load and processor utilization.

WAN throughput from


site (over IP)
or
Cross-site throughput
from site (over FC)

Mb/sec

Combined outgoing throughput from a site (to remote


site).

Total incoming writes


rate for site

MB/sec

Throughput of writes that arrived on the production side.


For concurrent local and remote replication, this is the
sum of throughputs for all local and remote links at the
site (which is double the actual incoming writes rate).

Incoming IOs rate for


site

IOs/sec

Number of writes that arrived on the production side. For


concurrent local and remote replication, this is the sum
of incoming IOs for all local and remote links at the site
(which is double the actual number of incoming writes).
Average IO size can be computed as (Total incoming
writes rate for box) / (Incoming IOs rate for box).

Non - initialization
output rate for site
(average over all
period)

MB/sec

Rate at which normal write traffic was transferred by all


RPAs at the site.

Initialization output rate MB/sec


for site (average over all
period)

Rate at which data was actually transferred by all RPAs at


the site for purpose of synchronization.

Data synchronization
rate for site (average
over all period)

Rate at which data was checked for possible transfer by


all RPAs at the site for purpose of synchronization. In
first-time initialization, this is the rate at which the data

RecoverPoint 5.0 Administrator's Guide

MB/sec

Protection

Table 22 Consolidated statistics output (continued)

Statistic

Unit of
measure

Description
from the replication set volumes is transferred. In other
cases, comparison of signatures increases the rate.

Line latency between


sites

msec

Time it takes for data to be transferred from a site to the


other site.

Packet loss

% of
packets

Measure of the reliability of the line between a site and


the other site, as measured in portion of packets that
must be resent. Latency and packet loss impact the
effective throughput on a line between two sites.

Table 22 Consolidated statistics output


(1) In an RPA, a link is a logical channel from production to copy, either local or remote, for a
given consistency group.
(2) An actual recovery point objective is the time/data/writes that were waiting to be
transferred over the link to the copy, averaged over the sampled time interval.

Monitoring and managing copies


Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand a group, and select a copy.
3. In the right pane, select the Status tab.
Results

The Status tab displays a graphical representation of the copy statuses. The following
information is displayed:

Monitoring and managing copies

87

Protection

Table 23 Status tab

Label

Displays

<Group
Name>
Running on
<RPA>

The group name and the RPA which is handling the group data, both of which can
be modified through the group policies. Group policies on page 63 includes
more information about group policies.

<Copy Name>
at <Cluster
Name>

The relevant copy and RPA cluster names. The copy name can be modified
through the copy policies. Copy policies on page 69 includes more information
about copy policies.

Replication
modes and
states

A visual representation of the current replication modes and states.


l

A dashed green line means that the system is replicating asynchronously.

A dashed green line on top of a solid line means that the system is
replicating synchronously.

A greyed out line means that replication has stopped.


Note

The lines move in the direction of replication.


The transfer state is displayed in the center of each link between the production
and the copy. The Transfer state displayed here correlates to what is displayed
in the Consistency Group Transfer Status on page 26 widget of the system
Dashboard.
When replicating synchronously, the word synchronized is displayed.

Role

The current role of the copy with regards to failover and regulation.
l

Before failover: Production, Local Copy and Remote Copy

After failover: Target at Production, Local Source and Remote Source

During regulation: Regulated

Storage

The image access mode.

Image

The image currently being distributed to storage in the distribution phase.

When a copy is selected in the left pane, you can click the buttons at the bottom of the
Status tab to perform the following actions:
Table 24 Status tab options when a group is selected

Button

Description

Test Copy

Displays the Test a Copy on page 128 wizard, enabling you to test the selected
copy of the selected consistency group.

Fail Over

Note

Not relevant in ProtectPoint


Displays the Test a Copy and Fail Over on page 135 wizard, enabling you to test
and fail over to the selected copy permanently or temporarily.

88

RecoverPoint 5.0 Administrator's Guide

Protection

Table 24 Status tab options when a group is selected (continued)

Button

Description

Recover
Production

Displays the Test a Copy and Recover Production on page 132 wizard, enabling
you to test and recover production from the selected copy.

Start
Transfer

Only displayed if transfer is paused for the selected copy.

Pause
Transfer

Only displayed if transfer is active for the selected copy.

Causes the transfer of writes from the production host to the selected copy to
start.

Causes the transfer of writes from the production host to the selected copy to
pause.
Note

Best practice is not to use this command. Use Group policies on page 63 to set
policies for use of bandwidth. Pause transfer may be used when WAN bandwidth
is very limited and you wish to give the largest bandwidth possible to another
consistency group. In that case, you may temporarily pause transfer for lowerpriority consistency groups.

Disable Copy Only displayed if the selected copy is enabled. Disables the selected copy.
Note

Disabling a copy stops all replication, deletes journals, and causes a volume
sweep when the copy is re-enabled.

Enable Copy

Only displayed if the selected copy is disabled. Enables the selected copy.
Note

Enabling a disabled copy starts replication and a volume sweep commences.

Remove
Copy

Removes the copy from the consistency group.


Note

You cannot remove the production copy.

Controlling RTO
Describes how to set and monitor the RPO at a specific copy.
Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand a consistency group, and select a copy.
3. In the right pane, click the Copy Policy tab.
4. Control the part of RTO dedicated to data access using the Maximum journal lag
setting in the Journal policies on page 69.
Results
When the system approaches the value of Maximum journal lag, it moves to three-phase
distribution mode.
Monitoring and managing copies

89

Protection

After you finish


To monitor the RTO:
1. Select Protection > Manage Protection.
2. In the left pane, select a consistency group.
3. In the right pane, click the Journal tab.
The value displayed in the Journal Lag field indicates the RTO in terms of data size.
Alternatively, run the get_group_statistics CLI command to display the journal
lag.

Controlling RPO
Describes how to set and monitor the RTO at a specific copy.
Before you begin
Only relevant in asynchronous replication mode
Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select a consistency group.
3. In the right pane, click the Link Policy tab.
a. In the Select link to set policy diagram, select the relevant link.
b. In the Protection Settings, set the RPO policy and click the Apply button.
4. Control the RPO for each consistency group through the RPO setting in the group Link
policies on page 65.

90

RecoverPoint 5.0 Administrator's Guide

Protection

In RecoverPoint, RPO can be expressed in terms of time, quantity of data, ornumber of


writes. To guarantee the RPO, host applications can be throttled upon approaching the
defined RPO setting, using application regulation. To control application regulation,
run the regulate_application parameter of the config_link_policy CLI
command. In synchronous replication mode, the regulate_application
parameter of the config_link_policy CLI command is automatically enabled,
and cannot be modified.
After you finish
To monitor the RPO:
1. Select Protection > Manage Protection.
2. In the left pane, select a consistency group.
3. In the right pane, click the Statistics tab and then, click the Replication Performance
sub-tab to monitor the consistency group's RPO in terms of time, writes, or quantity of
data.
Note

You can also select Dashboard > System Limits, and then click on the Groups tab to
monitor the RPO of all consistency groups in RecoverPoint simultaneously.
In the RecoverPoint CLI, run the get_group_statistics command, and identify the
Lag output in the Link stats area.

Monitoring copy journals


Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand a group and select a copy.
3. In the right pane, select the Journal tab.

Monitoring and managing copies

91

Protection

Results

The following table describes the information displayed in the Journal tab of each copy.
Table 25 Copy journal tab

Info

Description

Current

Snapshot currently being distributed to the journal.

Storage

Storage access status. The current condition of a volume at the copy storage can
be:

Journal Lag

Compression
Ratio

No access, during normal replication.

Direct Access, during direct image access.

Logged Access, during logged access.

Amount of data (represented by snapshots) in the copy journal that has not yet
been distributed to the copy storage. The maximum journal lag setting defines
the current recovery point objective. In the event of a disaster, this is the
maximum amount of data loss that may be incurred.
Note

This feature is relevant for both synchronous and asynchronous replication.


Default = none
When enabled, instructs the target RPA of the consistency group to compress the
snapshots in the journal so that more images (that is, a longer protection
window) can be saved with the same journal capacity (saving storage cost).
Enabling this option is encouraged if you have cost considerations, a low
incoming write-rate, and/or limited bandwidth.
Take note of the following:

92

RecoverPoint 5.0 Administrator's Guide

Compression is not available in RecoverPoint/SE.

Compression is not relevant for the production journal (since the production
journal does not contain snapshots).

Compression impacts the CPU resources of the target RPA of the consistency
group, and can impact that RPAs ability to sustain its write-load. If the target

Protection

Table 25 Copy journal tab (continued)

Info

Description
RPA of the consistency group for which you want to enable this option is also
transferring the data of other consistency groups across the WAN, note that
enabling this setting will affect the RPAs transfer rate. See the EMC
RecoverPoint Release Notes for throughput limitations.

Required
Protection
Window

Default = disabled

Current
Protection
Window

Indicates how far the copy journal can be rolled back. If the Required Protection
Window is defined, the Current Protection Window will be in one of the following
statuses (indicated in parentheses after the Current Protection Window):

Indicates how far in time the copy image can be rolled back.

Sufficient: Image can be rolled back far enough to meet the Required
Protection Window

Insufficient: Image cannot be rolled back far enough to meet the Required
Protection Window

Extending: Replication has not been running long enough to be roll backed
as far as the Required Protection Window.
If the Required Protection Window is not defined, the status will be N/A.

Predicted
Protection
Window

Systems prediction of the eventual size of the protection window. Note that
there is no guarantee on how long it will take to reach the predicted protection
window, and no guarantee that it will ever be reached (conditions may change
before it is reached).
If the Required Protection Window is defined, the Predicted Protection Window is
in one of the following statuses:
l

Sufficient: Predicted Protection Window is large enough to meet the


Required Protection Window

Insufficient: Predicted Protection Window is not large enough to meet the


Required Protection Window
If the Required Protection Window is not defined or replication has not been
running long enough to make predictions, no status will be indicated. It can
take 24 hours or longer of journal entries before the system finishes
calculating the predicted protection window.
Note

When snapshot consolidation is enabled, the Predicted Protection Window


value is displayed as N/A.
Space Saved
by
Consolidation

Note

Not relevant for XtremIO snapshot consolidation


Amount of space saved by RecoverPoint snapshot consolidation. This value is
updated only after a consolidation process completes.

Monitoring and managing copies

93

Protection

If a snapshot consolidation job is in process, the following information is also displayed


(not relevant in XtremIO):
Table 26 Journal Tab: Snapshot Consolidation Progress information

Info

Description

Consolidation
Type

Type of consolidation: Manual, Daily, Weekly, Monthly.

Consolidation
Range

Start and end times of snapshots being consolidated.

Progress

Completion percentage. A pending status indicates that the consolidation is


waiting for additional snapshots in the journal to be distributed to the user
volume. The consolidation will begin automatically once the snapshots have
been distributed.

Stop

Cancels the consolidation process. The consolidation process stops after it


completes processing the PIT it is currently working on. Stopping a consolidation
process returns the journal to the same state it was in before the consolidation
started.

The image list in the copy Journal tab displays the following information:

Table 27 Journal tab image list

Info

Description

Point in Time

Closing time of the snapshot.

Size

Size of snapshot.

Bookmark

Displays the bookmark name.


For consolidated snapshots, displays the consolidation type (manual, daily,
weekly, monthly). A tool tip indicates how much space was saved by the
consolidation.

Consistency

94

RecoverPoint 5.0 Administrator's Guide

Whether the snapshot is known by the user to be application-consistent or not.


The two possible values for this field are Application-Consistentand CrashConsistent. All snapshots are crash-consistent by default. Snapshots are
automatically marked application-consistent by KVSS during snapshot creation,

Protection

Table 27 Journal tab image list (continued)

Info

Description
and can be manually marked by the user using the Snapshot Policy button in
the copy Journal tab.
Note

Marking a snapshot as application-consistent does not create an applicationconsistent snapshot, it only tags the snapshot as application-consistent.
Snapshots marked as application-consistent have a higher probability of being
included in the snapshots displayed in the image list. If snapshot consolidation
is enabled, consider setting a Consolidation Policy of Never Consolidate for
snapshots that are marked as application-consistent.
Consolidation
Policy

Consolidation policy applied to this snapshot (Always Consolidate, Never


Consolidate, Survive Daily, Survive Weekly, Survive Monthly).
XtremIO snapshot pruning policy cannot be changed. It is displayed as Always
Consolidate.

Monitoring distribution performance


Describes how to monitor distribution performance at a specific copy.
Procedure
1. Detect Bottlenecks on page 201.
2. Select the General detection including initialization and high load periods with peak
writing analysis option.
3. Select both Advanced overview and Detailed overview.
4. Specify the other required information.
5. In the output, scroll-down until you reach a section that starts with the text: System
overview of the copy:<copyname

6. In five-phase distribution mode, note the process steps labeled as Distributor phase
1 and Distributor phase 2, and note their performance statistics.

Monitoring and managing copies

95

Protection

7. In three-phase distribution mode, note the process labeled Fast forward distribution
duration, and note the statistics.

Enabling snapshot consolidation for non-XtremIO copies


Before you begin
l

The total size of all snapshots between the specified start and end times must be at
least 1 GB.

Snapshots that account for 90% of the consolidation period must be available in
journal.
For example, for daily consolidation to take place, the starting and ending snapshots
must be at least 22 hours apart. Likewise, automatic consolidation will not take place
if the snapshots in the journal exceed 110% of the consolidation period. For
example, daily consolidation will not take place if the starting and ending snapshots
are more than 26 hours apart.

You cannot enable automatic consolidation on a consistency group that belongs to a


consistent group set.

Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand the consistency group and select a copy.
3. In the right pane, click the Copy Policy tab.
4. Select the Enable RecoverPoint Snapshot Consolidation checkbox to begin
consolidating the snapshots in your copy journal according to the default settings, or
adjust the settings to your specific requirements.
Results
Automatic snapshot consolidation begins according to the values set in the Copy Policy
tab, provided the system is in the distribution phase and initialization is over.
After you finish
To display the consolidation progress and space saved by the process:
1. Select Protection > Manage Protection.
2. In the left pane, expand the consistency group and select a copy.
3. In the right pane, click the Journal tab.

Changing the snapshot consolidation policy of XtremIO copies


Before you begin

96

There can be a minimum of one and a maximum of three snapshot consolidation


policies.

Changing the periodic Interval of the Snap-based replication policy in the group
Table 16 on page 65 automatically updates the XtremIO detailed snapshot
consolidation policies.

RecoverPoint 5.0 Administrator's Guide

Protection

Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand the consistency group and select a copy.
3. In the right pane, click the Copy Policy tab.
4. Change the XtremIO copy Table 18 on page 69:

(Quick path) Define a Maximum Number of Snaps and a Required Protection


Window. The system automatically deduces the most efficient snapshot
consolidation policy based on your settings. Up to three policies may be deduced.

(Advanced users only) Click the Edit Detailed Consolidation Settings link.
In the Edit Detailed Consolidation Settings dialog box, specify one to three
consolidation policies.

The system uses the entered values as guidelines. The actual number of
snapshots per timeframe is displayed beside each policy.
Note

For the consolidation policy values to be valid, the first consolidation policy in the
list must be of the shortest duration and the last policy in the list must be of the
longest duration. The first average snapshot frequency in the list must be higher
than the last.

Modifying consistency groups


Use the Manage Protection screen to:
l

Modify a groups protection policy on page 98

Modify a links protection policy on page 98

Remove a group on page 99

Add a copy to a group on page 99


Modifying consistency groups

97

Protection

Remove a copy from a group on page 106

Add replication sets on page 106

Remove replication sets on page 110

Pause transfer for a group on page 110

Start transfer for a group on page 111

Disable groups on page 111

Enable groups on page 111

Rename a replication set on page 115

Modify a groups protection policy


Modifies the protection policies of an existing consistency group.
Procedure
1. Select Protection > Manage Protection.
2. Select the consistency group in the left pane.
3. Click the Group Policy tab in the right pane.

Results
All of the group protection policies can be modified through the Group Policy tab. Group
policies on page 63 includes more information about group protection policies.

Modify a links protection policy


Modifies the protection policies of the links between an existing consistency group's
copies.
Procedure
1. Select Protection > Manage Protection.
2. select the consistency group in the left pane and click the Link Policy tab in the right
pane.

98

RecoverPoint 5.0 Administrator's Guide

Protection

Results
All of the link protection policies on page 65 can be modified through the Link Policy tab.
If you have created policy templates on page 52, use the Select Template drop-down to
select a policy template. Click Load to load the template settings into the dialog box.
NOTICE

After loading a policy template, modify the link policies on page 65 as required, and click
Apply to use the settings. The settings are not applied until the Apply button is clicked.

Remove a group
NOTICE

This action cannot be undone.


To remove one or more groups from the system, select Protection > Manage Protection,
and then, either:
l

Remove one group by selecting the consistency group in the left pane, clicking the
Status tab, and clicking Remove Group.

Remove multiple groups by clicking Consistency Groups in the left pane, selecting
one or more groups in the right pane, and clicking Remove Group.

Add a copy to a group


Before you begin
l

The Cancel button closes the wizard and disables image access without creating the
copy.

The Finish button closes the wizard, creates the copy, and optionally, starts
replication to the new copy.
NOTICE

VPLEX Metro users can also use this procedure to turn a non-MetroPoint group into a
MetroPoint group.
Modifying consistency groups

99

Protection

Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select a consistency group.
3. In the right pane, select the Status tab, and click Add Copy. The Add a Copy wizard is
displayed.
4. In the Add a Copy screen:
Note

This screen changes depending on whether or not VPLEX distributed volumes have
been mapped to RPA cluster at the production, and the group contains no remote
copies, or one remote copy.
l

If the there are VPLEX distributed volumes mapped to the RPA cluster at the
production, and the group contains no remote copies, or one remote copy, the Add
a Copy screen will contain two options for adding a copy.

If you are not a VPLEX Metro user or you do not want to turn a non-MetroPoint
group into a MetroPoint group, skip to the next bullet. If you are a VPLEX Metro
user, and want to turn a non-MetroPoint group into a MetroPoint group:
a. Select the Add Standby > Production Copy radio button.
b. Enter a name for the copy in the Standby production copy name field.
c. In the volume list, mark the checkboxes of each journal volume that you want
to use in your standby production journal.
Use the toolbar options at the top-right corner of the screen to help you identify
the required volumes in the volume list. For best performance, select volumes
that are identical in size. If identically sized volumes are not available, select
volumes that are similar in size.
These are the copy journal volumes for the standby copy, not the copy volumes.
d. Optionally, click Modify Policy to define the Journal policies on page 69 now, or
skip this step and define the policies later when you Modify a copys protection
policies on page 112.
The policy settings are optional. The default values provide a practical
configuration. It is recommended to accept the default settings unless there is
a specific business need to set other policies.
100

RecoverPoint 5.0 Administrator's Guide

Protection

If you want to change the protection settings using Policy Templates on page
52, use the Select Template drop-down to select one of your templates. Click
Load to load the settings in the template and click OK to apply the settings.
Note

This link is only enabled after volumes are selected in the volume list.
e. Click Next Display Copy Summary, and continue to Step 5 on page 213 on page
105.
l

Configure the copy settings:


If the groups production volumes are not VPLEX distributed volumes or the group
contains more than one remote copy, in the Add a Copy screen:

a. Define the copy name, the RPA cluster that will manage the copy volumes, and
the replication mode.
b. For each production volume, click the Select volume link to add a volume to
the copy. The selected volume is the volume that the production volume is
replicated to.
Note

If one volume at a copy is on an XtremIO array, all other XtremIO volumes at


that copy must be on the same array.
The volume list is displayed. The volume list only contains volumes of sizes
that are equal to, or greater than, the specified production volume.

Modifying consistency groups

101

Protection

Note

In VPLEX splitter environments, the VPLEX consistency group name will only be
displayed in the VPLEX Group column of the volume list if you Attach volumes
to the splitter on page 164 for the VPLEX splitter (including certificate). After
you enter VPLEX credentials, you will not be able to select VPLEX volumes and
non-VPLEX volumes in the same consistency group copy. It is recommended
that all volumes in a VPLEX consistency group be configured in a single
consistency group copy, and all volumes in a non-VPLEX consistency group be
configured in another consistency group copy.
In RecoverPoint/CL and RecoverPoint/EX, only the volumes that are mapped
and zoned to the RPA cluster are displayed in the available volumes list. In
RecoverPoint/SE, all of the volumes of the supported VNX Series array are
displayed in the available volumes list. See the appropriate technical notes for
your splitter for more information.
Use the toolbar options at the top-right corner of the screen to help you identify
the required volumes in the volume list. For best performance during failover,
select volumes that are the same size as the production volume. If volumes of
the same size are not available, select volumes that are as similar in size as
possible.
NOTICE

You cannot proceed to the next step until the replication sets icon under the
volume list shows a green check mark to indicate that the number of
replication sets is equal to the number of production volumes.

a. Optionally, click Modify Policy to define the Copy policies on page 69 and Link
policies on page 65 now.
102

RecoverPoint 5.0 Administrator's Guide

Protection

The policy settings are optional. The default values provide a practical
configuration. It is recommended to accept the default settings unless there is a
specific business need to set other policies.
If you want to change the protection settings using Policy Templates on page 52, in
the Link tab, use the Select Template drop-down to select one of your templates.
Click Load to load the settings in the template and click OK to apply the settings.
b. To continue configuring the group, click Next Define Copy Journal.
5. In the Define Copy Journal step:
If the volume that you specified in the last step is not on a Unity/VNXe/VNX array, skip
to step a on page 104
l

If the volume that you specified in the last step resides on a Unity/VNXe/VNX array
that has already been registered in RecoverPoint, and a resource pool was already
registered for the same Unity/VNXe/VNX array, the Journal Provisioning Method
screen is displayed.

If you want to allow RecoverPoint to automatically provision your journal volumes


for you, keep the Automatically Provision Journal Volumes radio button selected,
and select one of the following options:
Option

Description

Journal size

Enter a value for the journal size to define an absolute journal


size at the copy.

Predicted
incoming
write-rate

Select this option and enter a value for predicted incoming writerate and required protection window to have RecoverPoint/SE
calculate the required journal size based on your consistency
groups predicted write-rate and a required protection window.
As you change this value, the journal size that is required to sustain the
entered values for predicted incoming write-rate and the required
protection window is dynamically updated in the Calculated journal
size field.

Modifying consistency groups

103

Protection

Option

Description

Required
protection
window

Note

Only relevant if Predicted incoming write-rate is selected.


Enter a value for the required protection window to have
RecoverPoint/SE calculate the required journal size based on
your consistency groups predicted write-rate and a required
protection window.
As you change this value, the journal size that is required to sustain the
entered values for predicted incoming write-rate and the required
protection window is dynamically updated in the Calculated journal
size field.

When you are done configuring your journal settings, click Next Display Copy
Summary.
l

If you do not want RecoverPoint to automatically provision your journal volumes on


storage, ensure the Select Provisioned Journal Volumes radio button is selected.

For applications running on a host cluster, the journal volumes must not be
resources of the host cluster.
In the volume list:
a. Select the volumes to add to the journal.
On XtremIO arrays, the journal does not need to be larger than 10 GB for nondistributed consistency groups, 40 GB for distributed consistency groups.
Use the toolbar options at the top-right corner of the screen to help you identify
the required volumes in the volume list. For best performance, select volumes
that are identical in size. If identically sized volumes are not available, select
volumes that are similar in size.
b. Optionally, click Modify Policy to define the Journal policies on page 69 now.
The policy settings are optional. The default values provide a practical
configuration. It is recommended to accept the default settings unless there is
a specific business need to set other policies.
104

RecoverPoint 5.0 Administrator's Guide

Protection

If you want to change the protection settings using Policy Templates on page
52, use the Select Template drop-down to select one of your templates. Click
Load to load the settings in the template and click OK to apply the settings.
c. Click Next Display Copy Summary.
6. In the Copy Summary screen:
a. Select a copy to display its volume details and ensure the copy summary reflects
your anticipated copy configuration:

In RecoverPoint/SE:
l

If you selected to Automatically provision journal volumes based on a predefined Journal Size, the Journal Size is displayed at the bottom of the Copy
Summary screen.

If you selected to Automatically provision journal volumes based on a Predicted


incoming write-rate, and a Required protection window, the Calculated Journal
Size is displayed at the bottom of the Copy Summary screen.

b. Use the following options to create the copy, and optionally, start replication.
Table 28 Options

Action

Description

Start
replication to
copy when I
click 'Finish'

When selected, starts transfer to the copy as soon as the copy is


created.
Note

Before you start transfer to any copy, ensure that the copy volumes are
unmounted from any hosts and any volume groups are deported from
the logical volume manager.

Finish

Creates the copy, applies all of the specified settings, and starts a firsttime initialization process to synchronize the production volumes to the
copy volumes.
Before you click Finish, note the following:
Modifying consistency groups

105

Protection

Table 28 Options (continued)

Action

Description
l

If First-time initialization is taking too long on page 253, instead of


allowing RecoverPoint to send the initialization data to the copy
storage, you can initialize the copy volumes from a backup disk,
saving the time it would otherwise take to synchronize the data over
a WAN or FC connection.

By default, RecoverPoint writes the first snapshot directly to the


copy storage, bypassing the copy journal. Using the
fast_first_init parameter of the config_link_policy CLI
command, you can override the default setting and write the
initialization snapshot first to the journal (which is more timeconsuming but provides greater data protection).

To enable failover during initialization, set the value of the


allow_long_resync parameter of the config_copy_policy
CLI command to no (default is yes).

Remove a copy from a group


NOTICE

This action cannot be undone.


Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand the consistency group and select the copy.
Note

If you want to turn a MetroPoint group into a non-MetroPoint group, select the standby
production copy.
3. In the right pane; select the Status tab, click Remove Copy.

Add replication sets


Before you begin
l

The journal of all copies of the consistency group will be lost if all RPA clusters in your
system are not running RecoverPoint 4.4 or later.

Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select the consistency group.
3. In the right pane; select the Replication Sets tab, and click Add Replication Sets.

106

RecoverPoint 5.0 Administrator's Guide

Protection

The Add Replication Sets wizard opens and the Select Production Volumes screen is
displayed.
4. In the Select Production Volumes screen:
a. Select your production volumes in the volume list.
Note

If one volume at a copy is on an XtremIO array, all other XtremIO volumes at that
copy must be on the same array.

Use the toolbar options at the top-right corner of the screen to help you identify
the volumes.
The VPLEX consistency group name will only be displayed in the VPLEX Group
column if you have entered the VPLEX storage credentials (including certificate).
When you enter VPLEX credentials, you will not be able to select VPLEX volumes
and non-VPLEX volumes in the same RecoverPoint consistency group copy. It is
Modifying consistency groups

107

Protection

recommended that all volumes in a VPLEX consistency group be configured in a


single RecoverPoint consistency group copy, and vice versa.
b. To continue configuring the replication set, click Next Select Copy Volumes.
5. In the Select Copy Volumes screen:

a. Click the Select volume link to add a volume to the copy.


The selected volume is the volume that the production volume is replicated to. The
volume list is displayed.
The volume list only contains volumes of sizes that are equal to, or greater than,
the specified production volume.
Note

If one volume at a copy is on an XtremIO array, all other XtremIO volumes at that
copy must be on the same array.

Use the toolbar options at the top-right corner of the screen to help you identify
the required volumes in the volume list.
For best performance during failover, select volumes that are the same size as the
production volume. If volumes of the same size are not available, select volumes
that are as similar in size as possible.
108

RecoverPoint 5.0 Administrator's Guide

Protection

You cannot go on to the next step until the Replication sets icon under the volume
list shows a green check mark to indicate that the number of Replication sets is
equal to the number of production volumes.

b. When you are done selecting the volumes at each copy, click Next Review
Replication Sets.
6. In the Review Replication Sets screen:

a. Select a row in the replication sets table in the top section of the screen to display
the associated volume information in the bottom section.
b. Click on the replication set name to open the Edit Replication Set dialog box.

Modifying consistency groups

109

Protection

c. If required, rename the replication set.


d. Optionally, select the Attach as Clean checkbox (advanced users ONLY).
NOTICE

When you add a new replication set to an existing consistency group, RecoverPoint
ensures consistency between the new copy volume(s) and their production
source(s) by performing a volume sweep on all of the volumes in the new
replication set. Selecting the Attach as Clean checkbox overrides the default
synchronization process by informing the system that the new copy volumes are
known to be an exact image of their corresponding production volumes. If Attach
as Clean is checked, and the new copy volumes are inconsistent with their
corresponding production volumes, they will remain inconsistent. To ensure
consistency, best practice is not to use this option and allow the default
RecoverPoint synchronization process to ensure consistency between the new
copy volumes and the production.
7. Click Finish.
Results
All of the consistency group volumes undergo a short pause in transfer and a short
initialization, and a volume sweep occurs on the newly added volumes.

Remove replication sets


Before you begin
l

Enabled consistency groups must contain at least one replication set. If you want to
remove all of the replication sets in the group, click the Status tab and then click
Disable Group.

Procedure
1. Select Protection > Manage Protection.
2. Select the consistency group.
3. Select the Replication Sets tab.
4. Select one or more replication sets to remove.
Note

Clicking on a cell in any of the copy columns displays the relevant volume information
in the bottom pane.
5. Click Remove Replication Set.
Results
All of the consistency group volumes undergo a short pause in transfer and a short
initialization.

Pause transfer for a group


Pauses transfer between the production and all copies of a consistency group.
Procedure
1. Select Protection > Manage Protection.
110

RecoverPoint 5.0 Administrator's Guide

Protection

2. In the left pane, select the consistency group.


3. In the right pane; select the Status tab, click Pause Transfer, and click yes in the
confirmation dialog box.
Results
Transfer is paused between the production and all copies of the consistency group.

Start transfer for a group


Starts transfer between the production and all copies of a consistency group.
Before you begin
l

Before you start transfer to any copy, ensure that the copy volumes are unmounted
from any hosts and any volume groups are deported from the logical volume
manager.

Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select the consistency group.
3. In the right pane; select the Status tab, click Start Transfer, and click yes in the
confirmation dialog box.
Results
Transfer starts between the production and all copies of the consistency group.

Disable groups
Note

Disabling a group stops replication, deletes all copy journals, and causes a full sweep on
all copies in the group when the group is re-enabled.
Procedure
1. Select Protection > Manage Protection.
2. Disable the group(s):
l

To disable one group, select the group in the left pane, click the Status tab, and
click Disable Group.

To disable multiple groups, click Consistency Groups in the left pane, select one or
more groups in the right pane, and click Disable Group.

Enable groups
Before you begin
l

Enabling a consistency group automatically starts transfer.

Before you start transfer to any copy, ensure that the copy volumes are unmounted
from any hosts and any volume groups are deported from the logical volume
manager.

Procedure
1. Select Protection > Manage Protection.
2. Enable the group(s):
Modifying consistency groups

111

Protection

To enable one group, select the group in the left pane, click the Status tab, and
click Enable Group.

To enable multiple groups, click Consistency Groups in the left pane, select one or
more groups in the right pane, and click Enable Group.
NOTICE

When transfer starts, a full sweep occurs.

Modifying copies
Use the Manage Protection screen to:
l

Modify a copys protection policies on page 112

Pause transfer to a copy on page 113

Start transfer to a copy on page 113

Disable a copy on page 113

Enable a copy on page 114

Modify a copys protection policies


Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand the consistency group and select a copy.
3. In the right pane, click the Copy Policy tab.

4. Modify the Copy policies on page 69 and the copy Journal policies on page 69 as
required.
5. If you have created Policy Templates on page 52, use the Select Template drop-down
to select one of them, and click Load to load their settings into the dialog box.
6. Click Apply to use the settings.
112

RecoverPoint 5.0 Administrator's Guide

Protection

Note

The settings are not applied until the Apply button is clicked.

Remove a copy from a group


NOTICE

This action cannot be undone.


Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand the consistency group, and select the copy.
3. In the right pane; select the Status tab, click Remove Copy, and click yes in the
confirmation dialog box.

Pause transfer to a copy


Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand a consistency group, and select the copy.
3. In the right pane; select the Status tab, click Pause Transfer, and click yes in the
confirmation dialog box.

Start transfer to a copy


Before you begin
l

Ensure that the copy volumes are unmounted from any hosts and any volume groups
are deported from the logical volume manager.

Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand a consistency group, and select the copy.
3. In the right pane; select the Status tab, click Start Transfer, and click yes in the
confirmation dialog box.

Disable a copy
Before you begin
l

Disabling a copy stops replication, deletes the copy journal, and causes a volume
sweep when the copy is re-enabled.

Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand a consistency group, and select the copy.
3. In the right pane; select the Status tab, click Disable Copy, and click yes in the
confirmation dialog box.

Modifying copies

113

Protection

Enable a copy
Before you begin
l

Enabling a copy automatically starts transfer.

Before you start transfer to any copy, ensure that the copy volumes are unmounted
from any hosts and any volume groups are deported from the logical volume
manager.

Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand a consistency group, and select the copy.
3. In the right pane; select the Status tab, click Enable Copy, and click yes in the
confirmation dialog box.
NOTICE

A volume sweep commences when transfer starts.

Modifying links
To modify the protection policy of the links between existing consistency group copies:
Procedure
1. Select Protection > Manage Protection.
2. Select the consistency group in the left pane
3. In the right pane, click the Link Policy tab.

Results
All of the Link policies on page 65 can be modified through the Link Policy tab.
If you have created Policy Templates on page 52, select a link in the diagram and use the
Select Template drop-down to select a policy template. Click Load to load the template
settings into the Link Policy dialog box.
114

RecoverPoint 5.0 Administrator's Guide

Protection

NOTICE

After loading a policy template, modify the Link policies on page 65 as required, and click
Apply to use the settings. The settings are not applied until the Apply button is clicked.

Modifying replication sets


Use the Replication Sets tab to:
l

Rename a replication set on page 115

Modify (replace volumes) in a replication set on page 115

Resize volumes in a replication set on page 116

Remove replication sets on page 110

Rename a replication set


Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select the consistency group.
3. In the right pane; select the Replication Sets tab, select a replication set, and click
Edit Replication Set.

4. Change the value in the Name field.


5. Click OK.

Edit replication sets


Before you begin
Best practice is that all volumes in a replication set should be of the same size.
Procedure
1. Select Protection > Manage Protection.
2. Select the consistency group.
3. Select the Replication Sets tab, select a replication set, and click Edit Replication Set.
In the Edit Replication Set dialog box:

Modifying replication sets

115

Protection

a. Click the link next to the RPA cluster containing the volume you want to replace.

In the Select Volume dialog box, note the production volume specified in the
Production volume area at the top of the screen.
b. Select the volume to which you want to replicate the specified production volume.
c. Click OK.
Results
Data transfer is briefly paused for the group. A volume sweep occurs on any added
volume(s) and a short initialization occurs on all other volumes in the group, when data
transfer re-starts.

Resize volumes in a replication set


There are two procedures that can be used to resize volumes in a replication set:
l

Resizing volumes dynamically on page 116

Resizing volumes on page 118 (not relevant in XtremIO)

Resizing volumes dynamically


Before you begin
Only dynamic expanding of volumes is supported, not shrinking.
In XtremIO environments:

116

XtremIO 4.0.4-41 or later must be available at all XtremIO copies.

If you resize a production volume without resizing all of the copy volumes, transfer is
paused until the copy volumes are resized.

RecoverPoint 5.0 Administrator's Guide

Protection

If you resize a copy volume without resizing its production volume, failover and
production recovery will not be possible until the production volume is resized.

To minimize the time that transfer is paused, best practice is to resize the copy
volumes first, and then the production volumes.

In Unity/VNXe/VNX environments:
l

All volumes in a replication set must be attached to a Unity/VNXe/VNX splitter.

VNX OE for Block 05.32 or later must be available at all copies.

Thick volumes cannot be resized while snap-based replication is enabled.

Procedure
1. Resize the LUNs on storage.
l

On the Unity/VNXe/VNX array, in Unisphere:


a. Click the LUN tab.
b. Select the copy LUN or LUNs, right-click on the LUN name, select Expand, and
enter the new size.
c. Select the production LUN, select Expand, and enter the new size.
Once the LUNs have been resized, RecoverPoint will fake the sizes (as if they were
still the old sizes). Replication continues using the faked sizes.

On the XtremIO array:


a. Access the XtremIO array containing the copy volumes.
b. Click the Configuration icon.
c. Right-click on the copy volume, click Modify, and enter a new size.
d. Click OK.
e. Repeat these steps on the production array.

In RecoverPoint, a short initialization process starts on all volumes in the group.


2. Display the new volume sizes in Unisphere for RecoverPoint.
Note

It can take up to half an hour for RecoverPoint to display the new LUN sizes. To update
the display immediately, run the rescan_san CLI command.
l

For Unity/VNXe/VNX arrays:


a. Select Protection > Manage Protection.
b. In the left pane, select the consistency group.
c. In the right pane; select the Replication Sets tab, select the replication set, and
click Edit Replication Set.

Modifying replication sets

117

Protection

It may take a few minutes for RecoverPoint to display the new volume sizes.
l

For XtremIO arrays:


a. Select Protection > Manage Protection.
b. Select the consistency group.
c. Select the Replication Sets tab.

Resizing volumes
Before you begin
This procedure is not relevant for XtremIO.
Procedure
1. In RecoverPoint:
a. In Unity/VNXe/VNX only: Detach volumes from the splitter on page 165.
b. Remove replication sets on page 110.
2. On the storage, resize all LUNs in the replication set.
3. In RecoverPoint:
a. Create a new replication set containing the resized LUNs according to Add
replication sets on page 106.
A short initialization process starts on all volumes in the group.
b. In Unity/VNXe/VNX only: Attach the volumes to the splitter according to Attach
volumes to the splitter on page 164.
Results
A volume sweep starts on the replication set.

Modifying journals
There are two ways to change the storage capacity of an existing journal:
l

Add or edit journal volumes on page 118; this process does not trigger a full sweep
synchronization, nor does it erase all history in the existing journal. After this
procedure, the copy can still be rolled back to a previous point-in-time.

Change the size of a journal LUN on storage on page 119; this process does trigger a
full sweep synchronization, and it does erase all history in the existing journal. After
this procedure, the copy cannot be rolled back to a previous point-in-time.

Add or edit journal volumes


Changes the storage capacity of a journal at a copy by adding or removing journal
volumes.
Before you begin

118

In RecoverPoint/CL and RecoverPoint/EX only: Only the volumes that are exposed to
the RPA cluster are displayed in the available volumes list. If the required volumes
are not in the available volumes list, mask the LUNs to RecoverPoint WWNs, and click
the Rescan button to update the list of available volumes.

In RecoverPoint/SE, all of the volumes of the supported array are displayed in the
available volumes list.

For best performance, select volumes that are identical in size. If identically sized
volumes are not available, select volumes that are similar in size.

RecoverPoint 5.0 Administrator's Guide

Protection

Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand the consistency group, and select a copy.
3. In the right pane:
a. Select the Journal tab.
b. Click Edit Journal Volumes. The Edit Journal Volumes dialog box is displayed.

c. Mark the checkboxes of the volumes that you want to add to the journal.
Clear the checkboxes of the volumes that you want to remove from the journal.
Multiple volumes can be selected.
d. Click OK.
After you finish
If you performed this procedure to enlarge the journal capacity after distributing the
writes of this consistency group across multiple RPAs, disable and then re-enable this
group to trigger a full sweep.

Change the size of a journal LUN on storage


Resizes journals by resizing an existing journal volume LUN on storage.
Note

Not relevant in XtremIO.


Procedure
1. In RecoverPoint, select Recovery > Test a Copy:
a. Enable access to the Latest Image at the copy containing the journal volume(s), in
Logged Access mode.
b. Exit the wizard without disabling image access.

Modifying journals

119

Protection

Note

This ensures that all writes up until the latest image are transferred from the copy
journal to the copy storage, and only the host writes from that point forward will
need to be synchronized at the end of this procedure.
2. In RecoverPoint, disable the copy and remove the journal volume(s). Select Protection
> Manage Protection:
a. In the left pane, expand the group that the journal volume belongs to and select
the copy.
b. In the right pane; select the Journal tab, and click Edit Journal > Volumes. The Edit
Journal Volumes dialog box is displayed.

c. Clear the checkbox of the journal volume whose size you want to change and click
OK. The journal volume is removed from the RecoverPoint system.
3. At the copys SAN, dedicate more storage resources to the required LUN(s).
4. In RecoverPoint, click Edit Journal Volumes .
The RecoverPoint SAN discovery utility automatically detects the change in the
physical size of the volume.
5. Select the resized LUN(s).
6. Click the OK button to apply the changes and exit the dialog box.
7. In the right pane, select the Status tab.
8. Select Recovery > Manage Recovery, and select Finish Testing in the Select Next
Action combo box of the copy containing the expanded journal volume(s).
Results
Data transfer is briefly paused for the group, and a short initialization may occur, but only
the writes that occurred after image access was enabled to the copy will be synchronized.

Modifying snapshots
All of the points in time (snapshots) created by RecoverPoint are crash-consistent by
default. You can set or modify the policies of one or more snapshots by using the
following procedures:
l

120

Modify the policies of one snapshot on page 122

RecoverPoint 5.0 Administrator's Guide

Protection

Create a bookmark and define snapshot policies for one group on page 122
Note

To automatically create parallel bookmarks at pre-defined intervals, Create a group


set on page 57.
l

Create a bookmark and define snapshot policies across multiple groups on page 123

Use KVSS to Create and mark an application-consistent point in time on page 124.
Table 29 Snapshot policies

Setting

Description

Bookmark

The name applied to a point in time snapshot. Bookmarks are used to


uniquely identify a consistent point in time across all volumes in the group.

Mark as

The two possible values for this field are Application-Consistentand CrashConsistent. All snapshots are Crash-Consistent by default. Select ApplicationConsistentfrom the drop down if the selected snapshot is known to be
application-consistent.
Selecting Application-Consistent does not create an application-consistent
snapshot, it only tags the snapshot as application-consistent.
Note

Snapshots are automatically marked as application-consistent by KVSS


during snapshot creation, and can be manually marked by the user using the
Snapshot Policy button. Snapshots marked as application-consistent have
a higher probability of being displayed in the image list. If snapshot
consolidation is enabled, consider selecting Define a Consolidation Policy for
this Snapshot and Never Consolidate this Snapshot for snapshots that are
marked as application-consistent.

Define a
Consolidation
Policy for this
Snapshot

Note

Not relevant in ProtectPoint


Sets the consolidation policy for the snapshot.
The default consolidation policy for a snapshot is Always Consolidate, which
means that the snapshot is consolidated the next time that the consolidation
process runs.
Select Never Consolidate this Snapshot if you never want this snapshot
consolidated.
Select Snapshot must survive Daily/Weekly/Monthly consolidations
for:
l

Daily: Snapshot remains after daily consolidations, but is consolidated in


weekly, monthly, and manual consolidations.

Weekly: Snapshot remains after daily and weekly consolidations, but is


consolidated in monthly and manual consolidations.

Monthly: Snapshot remains after daily, weekly, and monthly


consolidations, but is consolidated in manual consolidations.

Modifying snapshots

121

Protection

Table 29 Snapshot policies (continued)

Setting

Description
Note

The XtremIO snapshot pruning policy cannot be changed.

Modify the policies of one snapshot


Note

Not relevant for XtremIO.


Procedure
1. Select Protection > Manage Protection.
2. In the left pane, expand a consistency group, and select a copy.
3. In the right pane; select the Journal tab, select a snapshot in the list, and click
Snapshot Policy. The Snapshot Policy dialog box is displayed.

4. In the Snapshot Policy dialog box, enter or modify any of the snapshot policies on
page 121 and click OK.

Create a bookmark and define snapshot policies for one group


Changes a snapshots settings and policies across all volumes in a group.
Before you begin
You can only bookmark a snapshot in a consistency group that is enabled and actively
transferring.
Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select a consistency group.
3. In the right pane; select the Status tab, and click Create Bookmark.

122

RecoverPoint 5.0 Administrator's Guide

Protection

4. Enter a descriptive name in the Bookmark field.


5. Enter or modify any of the Table 29 on page 121.
6. Click OK.
Note

All RecoverPoint snapshots are crash-consistent. Use KVSS to Create and mark an
application-consistent point in time on page 124.

Create a bookmark and define snapshot policies across multiple groups


Identifies a similar recovery point across multiple consistency groups.
Before you begin
l

You can only create a bookmark for a consistency group that is enabled and actively
transferring.

When creating a bookmark for multiple groups in parallel, the system creates a
bookmark with the same name and snapshot policies at a similar point in time on
each of the relevant consistency group volumes.

Note

This procedure does not create parallel bookmarks. To create parallel (consistent)
bookmarks at pre-defined intervals, create a group set on page 57.
Procedure
1. Select Protection > Manage Protection.
2. In the left pane, select Consistency Groups.
3. In the right pane; select two or more consistency groups, click Create Bookmark.

Modifying snapshots

123

Protection

The Create Bookmark dialog box is displayed.


4. Enter a descriptive name in the Bookmark field.
5. Enter or modify any of the snapshot policies on page 121.
6. Click OK.

Create and mark an application-consistent point in time


The RecoverPoint KVSS utility is a command-line utility that enables applying bookmarks
to Windows 2012 R2 and 2008-based applications that support Microsoft Volume
Shadow Copy Service (VSS). Microsoft Exchange and SQL are examples of Windows
applications that support Volume Shadow Copy Service.
A single bookmark can be used to bookmark VSS-aware applications in many
consistency groups. VSS guarantees that the applications are in a consistent state at the
point-in-time when each bookmark is applied to an image. As a result, recovery using an
image with a KVSS bookmark is faster than recovering from normal RecoverPoint images.
The EMC RecoverPoint Replicating Microsoft SQL Server Technical Notes and the EMC
RecoverPoint Replicating Microsoft Windows File Systems Technical Notes contain additional
detailed information on how to download, install and use RecoverPoint KVSS to create
application-consistent bookmarks.
KVSS bookmarks are created using the kvss.exe bookmark command.
The working folder for running KVSS commands is %SystemDriver%/
EMCRecoverPointVSSProvider/.
Note

Any bookmarks created using KVSS are automatically marked as Application-Consistent in


the RecoverPoint Table 29 on page 121.
When using KVSS to apply bookmarks:

124

Parameter values should be surrounded by quotation marks.

You can use the vssadmin list writers command to obtain a list of registered
writers on the host machine.

You can use the kvss.exe list command to display the components of each of
the writers found using the vssadmin list writers command.

RecoverPoint 5.0 Administrator's Guide

Protection

You can run the kvss.exe set_credentials command once per Windows user
to define the ip, user and password. After doing so, you will not need to enter these
values again.

Multiple writers and groups can be entered simultaneously if they are separated by a
space.

To use KVSS in a Microsoft Cluster Server environment with Symmetrix DMX storage, the
SPC-2 flag must be enabled on the Symmetrix ports.The syntax is as follows:
kvss.exe bookmark
bookmark=<bookmark_name>
writers=<writer_name> <writer_name>
[groups=<group_name> <group_name>]
[consolidation_policy=never|survive_daily|survive_weekly|
survive_monthy|always]
[type=[FULL|COPY]]
[ip=<RecoverPoint_cluster_management_ip_address>]
[user=<RecoverPoint_username>]
[password=<RecoverPoint password>]
Note

Parameters that are surrounded by square brackets [ ] are optional.


where:
l

writer = a VSS-aware host application

group = RecoverPoint consistency group

bookmark = name by which you can identify the applied bookmark

consolidation_policy = consolidation policy to set for this snapshot. Valid values are:
n

never; Snapshot is never consolidated.

survive_daily; Snapshot remains after daily consolidations, but is consolidated in


weekly, monthly, and manual consolidations.

survive_weekly; Snapshot remains after daily and weekly consolidations, but is


consolidated in monthly and manual consolidations.

survive_monthly; Snapshot remains after daily, weekly, and monthly


consolidations, but is consolidated in manual consolidations.

always; Snapshot is consolidated in every consolidation process, whether manual


or automatic.
Note

The default policy is always. If the consolidation_policy parameter is not


specified, the snapshot is consolidated in both automatic and manual
consolidation processes.
l

type =The shadow copy type: either FULL or COPY. This setting is optional. The default
is COPY. The settings full and copy are implemented by the writer application.
Generally, when type = full,backup logs are truncated; when type = copy, backup logs
are not truncated.

ip = RPA cluster management IP

user = RecoverPoint username

password = RecoverPoint password

Modifying snapshots

125

Protection

NOTICE

You can use the kvss.exe set_credentials command to define the IP address,
user, and password one time per Windows user, instead of entering these values each
time you create a bookmark.
For example, to create a bookmark for a Microsoft Exchange application for the first time:
kvss.exe set_credentials
ip=10.10.0.145
user=admin
password=admin
kvss.exe bookmark
writers="Microsoft Exchange Writer"
groups="exchange\comp1" exchange\comp2
bookmark="exchange hourly snapshot"
consolidation_policy=survive_daily

To create a bookmark every subsequent time for a Microsoft Exchange application after
defining the ip, user, and password through the kvss.exe set_credentials
command:
kvss.exe bookmark
writers="Microsoft Exchange Writer"
groups="exchange\comp1" exchange\comp2
bookmark="exchange hourly snapshot"
consolidation_policy=survive_daily

126

RecoverPoint 5.0 Administrator's Guide

CHAPTER 4
Recovery

This section describes how to test copies, recover from disasters, and migrate to a
different storage system.
l
l
l
l
l

RecoverPoint Recovery........................................................................................ 128


Test a Copy......................................................................................................... 128
Test a Copy and Recover Production.................................................................... 132
Test a Copy and Fail Over.....................................................................................135
Manage Recovery................................................................................................ 139

Recovery

127

Recovery

RecoverPoint Recovery
In Unisphere for RecoverPoint, data testing and recovery is guided by wizards, and
performed through the Recovery menu.
NOTICE

Test a Copy and Fail Over is not relevant in ProtectPoint.

In Unity/VNXe/VNX environments you cannot enable image access in Virtual Access


mode to a snapshot created before replication volumes were resized. To access an image
created before volumes were resized, use Logged Access mode.
In XtremIO, if you enable access to a snapshot that was created before you resized LUNs,
RecoverPoint automatically adjusts the LUNs at the accessed point in time to the resized
volume size.

Test a Copy
Use the Test a Copy wizard to briefly access and test your copy images.

How to test a copy


Procedure
1. At the copy host, mount the copy volumes.
128

RecoverPoint 5.0 Administrator's Guide

Recovery

2. In Unisphere for RecoverPoint, select Recovery > Test a Copy.


3. In the Select a Copy screen, select a consistency group or a group set:
l

If you selected a consistency group, select a non-production copy from the list of
copies or the copy diagram.

If you selected a group set, select a target RPA cluster.

4. Click Next Select an Image on page 129.

Selecting an image
Procedure
1. In the Select an Image screen, select the copy image that you want to access.
To test the image, you may want to start with the last image known to be valid. To
analyze data, you generally want the latest image.

How to test a copy

129

Recovery

2. Unless your copy is on an XtremIO array, you will have to select the required Image
access mode. If your copy is on an XtremIO array, skip this step.
Virtual Access without Roll image in background is preferred. However, if you need to
test images for a longer period of time or need maximum performance while testing,
select Virtual Access and check the Roll to image in background checkbox, or select
Logged Access.
3. Mount the copy volumes that you want to access and import any volume groups.
4. Click Next Test the Image on page 130 to start enabling image access to the copy
volumes.
After you finish

Testing the image


The Test the Image screen differs depending upon whether you selected a consistency
group or a group set.

130

RecoverPoint 5.0 Administrator's Guide

Recovery

If any of the groups or copies in a group set are not legal for the requested operation, they
are not displayed in the Relevant copies at target cluster table.
Procedure
1. Use the Test the Image screen to monitor your testing activity.
2. Before you begin testing, wait for image access to be enabled.
If you are accessing the image in logged access mode, wait for the system to roll to
the selected image.
Note the status of the Image access log capacity progress bar. It indicates the amount
of time that you have left to access the image before writes to the copy are blocked
and you will have to add journal volumes, undo writes, enable direct access or disable
image access.
3. Test the copy image, to ensure its suitability to your purposes.
While testing the copy image, you can click Roll to Image to physically roll to the
desired image (in Virtual Access mode), Enable Direct Access to bypass the image
access log and write directly to the copy storage, Undo Writes to undo all writes to the
copy volumes, Cancel to disable image access, or Save & Close to exit the wizard
without disabling image access and display the Manage Recovery on page 139
screen.
Note
l

If you Enable Direct Access you cannot roll back to an earlier image of the
accessed copy if you discover corrupted data. Moreover, in the event of a disaster
at the production during testing, you will not be able to remove any new data
written to the copy during testing unless the group contains another nonproduction copy.

Before you Undo Writes, at the copy host, shut down all applications accessing the
group volumes, unmount all copy volumes, and deport any volume groups. After
the writes are undone, at the copy host, mount the copy volumes and import
volume groups to continue testing.

4. When you are done testing, click Cancel to disable image access.

How to test a copy

131

Recovery

5. At the copy host, stop all applications and unmount the copy volumes. Deport any
copy volume groups.

Test a Copy and Recover Production


Use Test a Copy and Recover Production to display the Recover Production wizard. The
Recover Production wizard guides you through the process of correcting file or logical
corruption, by rolling the production back to a previous point-in-time.

How to recover production


Procedure
1. Select Recovery > Test a Copy and Recover Production.
2. In the Select a Copy screen, select either a consistency group or a group set.

132

If you selected a consistency group, select a non-production copy from the list of
copies or the copy diagram.

If you selected a group set, select a target RPA cluster from the list.

RecoverPoint 5.0 Administrator's Guide

Recovery

3. Click Next Select an Image on page 133.

Selecting an image
Procedure
1. In the Select an Image screen, select the image that you want to recover production
from.
To restore your production from a copy, you generally want the most recent copy
image that you know to be valid. For example, if you are using Microsoft Volume
Shadow Copy Service, you probably want to select the most recent shadow copy.

2. Select your preferred image access mode.


3. Mount the copy volumes that you want to access and import any volume groups.
4. Click Next Start Recovery on page 134.
RecoverPoint starts enabling access to the selected image at the copy.

How to recover production

133

Recovery

Starting recovery
The Start Recovery screen differs depending upon whether you selected a consistency
group or a group set.

If any of the groups or copies in a group set are not legal for the requested operation, they
are not displayed in the Relevant copies at target cluster table.
Procedure
1. Use the Start Recovery screen to monitor your copy testing, and start recovery.
2. Wait for image access to be enabled.
Note the status of the Image access log capacity progress bar. It indicates the amount
of time that you have left to access the image before writes to the copy are blocked
and you will have to add journal volumes, undo writes, enable direct access, or
disable image access.
3. Test the copy image, to ensure its suitability to your purposes.
134

RecoverPoint 5.0 Administrator's Guide

Recovery

Note the status of the Image access log capacity progress bar. It indicates the amount
of time that you have left before writes to the copy are blocked and you will have to
add journal volumes, undo writes, enable direct access or disable image access.
While testing the copy image, click Undo Writes to undo all writes to the copy volumes
without disabling image access, Cancel to disable image access, or Save & Close to
exit the wizard without disabling image access.
Note

Before you click Undo Writes, at the copy host, shut down all applications accessing
the copy volumes, unmount all copy volumes, and deport any volume groups. After
the writes are undone, mount the copy volumes and import any volume groups to
continue testing.
4. Before starting recovery, if necessary, click Undo Writes to remove any writes to the
copy before production recovery. Then, at the copy host, unmount the copy volumes
and deport any volume groups.
Note

If you do not Undo Writes now, any writes made to the copy while image access was
enabled will be transferred to the production when you click Finish. Before you click
Undo Writes, at the copy host, shut down all applications accessing the copy
volumes, unmount all copy volumes, and deport any volume groups. After the writes
are undone, mount the copy volumes and import any volume groups to start
production recovery.
5. Click Finish to start production recovery and display the Manage Recovery on page
139 screen.
Note

The marking information in the production journal is deleted (not relevant in XtremIO).
After production recovery is complete, the group undergoes a short initialization to
synchronize the new production data at the copies.

Test a Copy and Fail Over


Use Test a Copy and Fail Over to display the Failover wizard. The failover wizard guides
you through the process of migrating all of your production activities to a copy. For groups
with more than one non-production copy, you can also use the failover wizard to
temporarily fail over to a copy while performing routine maintenance on the production
system, and then fail back to the production.

Test a Copy and Fail Over

135

Recovery

How to fail over


Before you begin
l

You can only fail over temporarily if you have more than one non-production copy in
the group.

If your group(s) only contain one non-production copy, the failover will be permanent.
Before you begin a permanent failover, shut down any production host activities on
Windows Server platforms. To ensure an up-to-date image of the file system, flush all
file systems that reside on the production volumes. Some applications, such as
Exchange, have their own cache, which should be flushed as well. Close all
applications that are using the consistency group volumes at the production site and
place each drive volume used by the consistency group into an offline state.

Procedure
1. Select Recovery > Test a Copy and Fail Over.
2. In the Select a Copy screen, select either a consistency group or a group set.

136

If you selected a consistency group, select a non-production copy from the list of
copies or the copy diagram.

If you selected a group set, select a target RPA cluster from the list.

RecoverPoint 5.0 Administrator's Guide

Recovery

3. Click Next Select an Image on page 137.

Selecting an image
Procedure
1. In the Select an Image screen, select the image to fail over to.

To permanently fail over to a copy, you generally want the most recent copy image that
you know to be valid, that was created after placing all consistency group drives into
an offline state at the production. For example, if you are using Microsoft Volume
Shadow Copy Service, you probably want to select the most recent shadow copy.
2. Select your preferred image access mode.
3. Mount the copy volumes that you want to access and import any volume groups.
4. Click Next Start Failover on page 138.
RecoverPoint starts enabling access to the specified copy image.

How to fail over

137

Recovery

Starting failover
The Start Failover screen differs depending upon whether you selected a consistency
group or a group set.

If any of the groups or copies in a group set are not legal for the requested operation, they
will not be displayed in the Relevant copies at target cluster table.
Procedure
1. Use the Start Failover screen monitor your testing activity, and start failover.
2. Wait for image access to be enabled.
Note the status of the Image access log capacity progress bar. It indicates the amount
of time that you have left to access the image before writes to the copy are blocked
and you will have to add journal volumes, undo writes, enable direct access or disable
image access.
3. If you are only failing over temporarily, and planning to fail back, skip this step.
138

RecoverPoint 5.0 Administrator's Guide

Recovery

If you are failing over permanently; resume production disk activities by placing each
production drive volume used by the consistency group into an online state now.
4. At the copy host, mount the copy volumes that you want to access and import any
volume groups.
5. Test the copy image, to ensure its suitability to your purposes.
While testing the copy image, click Undo Writes to undo all writes to the copy volumes
without disabling image access, Cancel to disable image access, or Save & Close to
exit the wizard without disabling image access and display the Manage Recovery on
page 139 screen.
Note

Before you click Undo Writes, at the copy host, shut down all applications accessing
the group volumes, unmount all group volumes, and deport any volume groups. After
the writes are undone, mount the copy volumes and import any volume groups to
continue testing.
6. When testing is complete and you are ready to fail over, if your group only contains
one non-production copy, the failover will be permanent.
At the copy host, unmount the copy volumes and deport any volume groups now.
7. Click Finish.
l

If there is only one non-production copy in your group(s), the marking information
in the production journal is deleted, the copy journal is deleted, and the group
undergoes a full sweep before normal replication is resumed.

If there is more than one non-production copy in your group(s), the Manage
Recovery on page 139 screen is displayed. You can repair the production while
your applications and business operations continue at the copy. When you are
done, use the Manage Recovery on page 139 screen to:
n

Fail back to production

Set your copy (or copies) as production

Manage Recovery
Use Manage Recovery to monitor and manage all of the accessed copies and recovery
activities currently occurring in the system.

The Manage Recovery screen displays an activity bar for each copy (or copies, in the case
of a group set) on which an image has been accessed by RecoverPoint. In each copy
Manage Recovery

139

Recovery

activity bar, click the Show Activity Diagram link to monitor the activity progress and use
the Select Next Action drop-down to complete the activity.

Note

If an action that is part of a recovery activity fails for one or more of the groups in a group
set, you may want to manually perform the action for each group on which the action
failed in order to bring all of the group set's groups to the same recovery state.

Testing a copy
While Testing a copy, click Select Next Action to:
Continue Testing
Opens the Test a Copy wizard to the Test the Image screen on page 130.
Finish Testing
Stops testing and disables image access to the relevant group or group set copies.

Failing over
While Failing Over, click Select Next Action to:
Fail Back to Production
When repairs at the production site are complete, resumes production activities at
the original production site.
Opens the Failover wizard to the Start Failover screen on page 138. In the Start
Failover screen, click Fail Over to start failing back.
Note

The production journal is deleted.


Replication is resumed. The direction of replication is restored from the original
production to the original copies.
After failback, at the copy host(s), unmount the copy volumes and deport any
volume groups.

140

RecoverPoint 5.0 Administrator's Guide

Recovery

Set Copy/Copies as Production


Fails the production over to the selected accessed copy (or copies).
Note

Before selecting this option, at the relevant copy host(s), close any applications
accessing the copy(s), unmount the copy volumes and deport any volume groups.
If the group contained more than one non-production copy, the Set Production
dialog box is displayed, prompting you to decide what you want to do with any target
copies of the new production after failover. If you are failing over a group set
containing groups with more than one non-production copy, the Set Production
dialog box is not displayed. In this case, before setting a copy as the new
production, you will need to manually configure the links on page 98 between the
new production and all of the new target copies.

If you want to use a copy, configure a new link for it and define a link policy on page
65 (not necessary for Data Domain copies). Removing a copy deletes the copy data
and settings from storage. Disabling a copy retains the copy data and settings on
storage.
Note

The journal of the new production is deleted. Replication starts from the new
production to the new copies.
Back to Wizard
Opens the Failover wizard to the Start Failover screen on page 138.

Failing over

141

Recovery

Recovering production
While recovering production only Back to Wizard is available. Click Back to Wizard to open
the Recover Production wizard to the Start Recovery screen on page 134. In the Start
Recovery screen, click Recover Production to start the recovery activity.

142

RecoverPoint 5.0 Administrator's Guide

CHAPTER 5
RPA Clusters

This section describes the RPA Clusters menu and how to use it to manage and monitor
replication in RecoverPoint.
l
l
l
l
l
l

RecoverPoint clusters.......................................................................................... 144


General Cluster Info.............................................................................................144
Storage............................................................................................................... 147
Splitters.............................................................................................................. 160
RPAs................................................................................................................... 166
vCenter Servers................................................................................................... 173

RPA Clusters

143

RPA Clusters

RecoverPoint clusters
Monitoring or administration of components in a RecoverPoint system that are managed
by RPA clusters is performed through the RPA Clusters interface. The information for each
RPA cluster is displayed in a separate tab of the interface.

Table 30 RPA cluster options

Option

When to use

General
Cluster Info

To display the specified RPA cluster graphically as part of a site map and view
general cluster information, or to display any alerts that may have been logged
for any of the physical components of your RecoverPoint system, at the selected
RPA cluster.

Storage

To register arrays, vCenter Servers, and resource pools and manage their
registration in the RecoverPoint system.

Splitters

Note

Not relevant in ProtectPoint


To display the splitter table, add/remove splitters, or attach/detach volumes
from splitters at the selected RPA cluster.

RPAs
vCenter
Servers

To display the RPA table containing all RPAs in the selected RPA cluster, and the
status and properties of each.
Note

Not relevant in ProtectPoint


To monitor, register, or remove vCenter Servers.

General Cluster Info


Use the RPA Clusters > RPA-Cluster-Name tab > General Cluster Info link to graphically
display an overview of the current state of individual components in your RecoverPoint
system.

144

RecoverPoint 5.0 Administrator's Guide

RPA Clusters

Figure 8 RPA Clusters: General Cluster Info

The General Cluster Info section contains a Cluster Diagram and a Cluster Details section
that allow you to easily monitor and analyze the current state of the RPA cluster. The
cluster diagram contains all of the major components of the RecoverPoint system (host,
storage, and RPAs) and the connectivity between them and the RPA cluster. An icon is
displayed on any system component for which system alerts on page 25 have been
registered.

The following information is displayed in the details section below the cluster diagram:

Table 31 RPA cluster information

Information

Description

RecoverPoint
Version

The version of the RecoverPoint software that is currently installed on the RPA
cluster.
If the system is in maintenance mode, the RecoverPoint version will display N/A
System is in maintenance mode.

General Cluster Info

145

RPA Clusters

Table 31 RPA cluster information (continued)

Information

Description

Internal RPA
Cluster Name

The internal system name given to RPA clusters, and used in upgrade scenarios.

Management
IPv4

The IP address assigned to the LAN interface of the RPA cluster in order to define
the management interface network, in IPv4 format.
Note

Use this IP to connect to the RPA in the RPA cluster that is running the cluster
control, in IPv4 format.
Management
IPv6

The IP address assigned to the LAN interface of the RPA cluster in order to define
the management interface network, in IPv6 format.
Note

Use this IP to connect to the RPA in the RPA cluster that is running the cluster
control, in IPv6 format.
Software
Serial ID

One of the system parameters sent to SyR on page 226.

Number of
RPAs

The number of RPAs in the RPA cluster.

RPA Type

The type of the RPA cluster. Valid values are Physical or Virtual.

Maintenance
Mode

Unisphere for RecoverPoint will enter maintenance mode:


l

During minor version upgrade.

During major version upgrade.

When replacing an RPA in an existing cluster.

When adding new RPAs to existing clusters.

When converting an environment to a RecoverPoint/SE or RecoverPoint/CL


configuration.

When entering maintenance mode through CLI.

If an external system causes the system to enter maintenance mode.


When in this mode, Unisphere for RecoverPoint can only monitor the system;
user-initiated capabilities are disabled. Once the system exits this mode,
Unisphere for RecoverPoint will return to standard managing and monitoring
function.
Whether or not the system is in maintenance mode. If so, the specific
maintenance activity that is currently being performed is displayed.
RecoverPoint system maintenance is performed using the Deployment
Manager. See the EMC RecoverPoint Installation and Deployment Guide for
more information on maintenance mode.

Valid values are:

146

RecoverPoint 5.0 Administrator's Guide

None (when the system is not in maintenance mode)

Minor Upgrade

Major Upgrade

RPA Clusters

Table 31 RPA cluster information (continued)

Information

Time Zone

Description
l

Hardware Replacement

RPA Cluster Upgrade

Repository Change

RP/SE Conversion

System Modification

User-Initiated Maintenance Mode

RPA Cluster Connection

Unknown

The time zone used by the RPA cluster.

Storage
Use the RPA Clusters > RPA-Cluster-Name tab > Storage link to enable RecoverPoint
storage awareness by registering new arrays, vCenter Servers and resource pools or
manage existing registered arrays, vCenter Servers, and resource pools.
Note

In RecoverPoint, Data Domain block services for ProtectPoint pools are referred to as
resource pools

Figure 9 RPA Clusters: Storage

Use the Storage link to:


l

Display registered storage on page 148

Register storage on page 149

Storage

147

RPA Clusters

Figure 9 RPA Clusters: Storage (continued)


l

Unregister storage on page 154

Edit storage registration details on page 155

Display registered resource pools on page 157

Register resource pools on page 158

Unregister resource pools on page 159

Display registered storage


Use the following procedure to display all of the storage systems currently registered at
an RPA cluster.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, select Storage.

Results
The Registered Storage table is displayed. The Registered Storage table contains the list
of all of the storage currently registered at the selected RPA cluster with the following
storage-specific information:
Table 32 Storage information

148

Information

Description

Name

The name of the registered storage in the RecoverPoint system.

Serial
Number

The name of the splitter in the RecoverPoint system.

Type

The type of storage. Valid storage types are:

RecoverPoint 5.0 Administrator's Guide

VNX

ScaleIO

VPLEX

RPA Clusters

Table 32 Storage information (continued)

Information

Description
l

vCenter Server

XtremIO

Data Domain

IP

The IP address of the array or vCenter Server management interface. In the case
of ScaleIO storage, it may be a list of IP addresses.

Version

The version of the operating system running on the storage.

Splitter

The name of the splitter in the RecoverPoint system.


Note

Only available in VNX.

Connectivity Whether or not there is connectivity between the storage and the RPA cluster.
Status
Connectivity Whether the connection between the storage and the RPA cluster is via Fibre
Channel (FC) or IP.
Type

Register storage
Before you begin
Note

This procedure is not relevant to Unity or VNXe, which registers itself with RecoverPoint
automatically.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click Storage.
Note

You can also register vCenter Servers by selecting RPA Clusters > <RPA-Cluster-Name>
tab > vCenter Servers .

Register storage

149

RPA Clusters

In the right pane, the Registered Storage table is displayed. The Registered Storage
table contains the list of all of the Data Domains, arrays and vCenter Servers that are
currently registered at the selected RPA cluster and their storage information on page
148.
3. In the right pane, in the Registered Storage tab, click Add. The Register Storage
dialog box is displayed.
4. In the Register Storage dialog box:
l

Register an array seen by this RPA cluster


To register any Data Domain, XtremIO, VPLEX, ScaleIO, VNX or vCenter Server that
has already been masked to the RPA cluster:

a. Select an array from the list


b. Define the storage login credentials:

150

RecoverPoint 5.0 Administrator's Guide

RPA Clusters

Define Data Domain login credentials on page 152

Define ScaleIO login credentials on page 151

Define vCenter login credentials on page 154

Define Unity/VNXe/VNX login credentials on page 153

Define VPLEX login credentials on page 153

Define XtremIO login credentials on page 152

Register any storage


To register any Data Domain, XtremIO, VPLEX, ScaleIO, VNX or vCenter Server,
regardless of whether or not it is masked to the RPA cluster:

a. Select the storage type from the drop-down.


b. Define the storage login credentials:
n

Define Data Domain login credentials on page 152

Define ScaleIO login credentials on page 151

Define vCenter login credentials on page 154

Define Unity/VNXe/VNX login credentials on page 153

Define VPLEX login credentials on page 153

Define XtremIO login credentials on page 152

Define ScaleIO login credentials


For ScaleIO storage, the ScaleIO MDM credentials and IP1 address are mandatory.

Register storage

151

RPA Clusters

Define XtremIO login credentials

Procedure
1. Enter the IP address of the XtremIO Management Server (XMS).
If the XtremIO Management Server manages more than one cluster, register each
cluster separately.
2. Enter the Serial number (PSNT) of the XtremIO cluster.
3. Enter the XtremIO rp_user as the User.
4. Enter the rp_user password as the Password. This password can only be changed
using the XtremIO user interface.

Define Data Domain login credentials

152

RecoverPoint 5.0 Administrator's Guide

RPA Clusters

Procedure
1. Enter the Data Domain IP address.
2. Enter the username and password for the block services for ProtectPoint pool and DD
Boost storage-unit in the Username and Password fields.
3. Enter the DD Boost Storage Unit to use as the ProtectPoint catalog.
4. By default, the Use FC for data transfer checkbox is selected. Uncheck to use IP only
for data transfer.

Define VPLEX login credentials


For VPLEX storage, the storage IP address, username and password are mandatory.
Alternatively, retrieve a certificate from system or manually select it.

Define Unity/VNXe/VNX login credentials


For VNX storage, all of the fields are mandatory. Enter the IP addresses of SP-A and SP-B,
the username and password of your storage and an authentication scope.

Register storage

153

RPA Clusters

Define vCenter login credentials


For vCenter Server storage, all of the fields are mandatory and the default TCP port is 443.
Best practice is to configure the vCenter Server to require the use of a certificate because
once RecoverPoint has read the certificate, it does not need further access to the
location. The default certificate locations are:
l

Windows 2003 Server:


C:\Documents and Settings\All Users\Application Data\VMware
\VMware VirtualCenter\SSL\rui.crt.

Windows 2008 Server:


C:\Users\All Users\Application Data\VMware\VMware
VirtualCenter\SSL\rui.crt.
For more information about the location of the security certificate, refer to Replacing
vCenter Server Certificates VMware vSphere 4.0, available at www.vmware.com.

Unregister storage
Unregisters storage from an RPA cluster.
Note

You can also unregister vCenter Servers by selecting RPA Clusters > RPA-Cluster-Name >
vCenters Server.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click Storage.

154

RecoverPoint 5.0 Administrator's Guide

RPA Clusters

In the right pane, the Registered Storage tab is displayed. The Registered Storage tab
contains the list of all of the storage currently registered at the selected RPA cluster,
and its storage information on page 148.
3. In the right pane:
a. Select your storage in the Registered Storage tab.
b. Click Remove.

Edit storage registration details


Use the following procedure to edit the registration details of a Data Domain, an array, or
a vCenter Server at an RPA cluster.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click Storage.
Note

You can also edit the registration details of a vCenter Servers from the vCenter
Servers on page 173 link.
In the right pane, the Registered Storage table is displayed. The Registered Storage
table contains the list of all of the arrays and vCenter Servers that are currently
registered at the selected RPA cluster and their storage information on page 148.

Edit storage registration details

155

RPA Clusters

3. In the right pane:


a. Select storage in the Registered Storage table.
b. Click Edit. The Edit Storage Registration dialog box is displayed.

c. In the Edit Storage Registration dialog box:


If required, modify the storage name.
a. If required, define the login credentials for your array:

156

Define Data Domain login credentials on page 152

Define ScaleIO login credentials on page 151

Define vCenter login credentials on page 154

Define Unity/VNXe/VNX login credentials on page 153

Define VPLEX login credentials on page 153

RecoverPoint 5.0 Administrator's Guide

RPA Clusters

Define XtremIO login credentials on page 152

b. Click OK.

Display registered resource pools


Use the following procedure to display all of the resource pools that have been registered
at an RPA cluster.
Note

This procedure is not relevant for XtremIO arrays.


Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, select Storage.

In the right pane, the Registered Storage table is displayed.


3. In the Registered Storage table, select an array or vCenter Server.
The Registered Resource Pools table is displayed under the Registered Storage tab.
The Registered Resource Pools table contains the list of all of the resource pools of
the selected storage, that are currently registered at the selected RPA cluster. The
following storage-specific information is displayed for each registered resource pool:
Table 33 Resource pool storage information

Information

Description

Name

The name of the registered resource pool in the RecoverPoint system.

Type

The resource pool type.


Valid resource pool types are:
l

RAID groups

Storage pools

Data stores

Display registered resource pools

157

RPA Clusters

Table 33 Resource pool storage information (continued)

Information

Description
l

vDisk pools

Total Size

The total capacity of the resource pool.

Available
Size

The total capacity of the resource pool minus the capacity of the volumes that
were already allocated in the resource pool.

Tiers

The physical type of the storage. For example, Fibre Channel, SATA, etc.

Available
Tiering
Policies

The tiering policies that are available on storage and can be automatically
applied to volumes in RecoverPoint when users automatically provision volumes.

Register resource pools


Registers resource pools at an RPA cluster.
Procedure
1. Select RPA Clusters.
2. Select the tab of the relevant RPA cluster.
3. In the left pane, select Storage.
In the right pane, the Registered Storage tab is displayed. The Registered Storage tab
contains the list of all of the storage systems that are currently registered at the
selected RPA cluster and their storage information.

4. In the Registered Storage tab, select an array or vCenter Server.


5. Under the Registered Resource Pools table, click Add.
The Register Available Resource Pools dialog box is displayed.

158

RecoverPoint 5.0 Administrator's Guide

RPA Clusters

6. In the Register Available Resource Pools dialog box:


a. Select the resource pool that you want to register.
b. Click OK.

Unregister resource pools


Use the following procedure to unregister resource pools of an array or vCenter Server at
an RPA cluster.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, select Storage.
In the right pane, the Registered Storage table is displayed. The Registered Storage
table contains the list of all of the arrays and vCenter Servers that are currently
registered at the selected RPA cluster and their storage information on page 148.

3. In the Registered Storage tab, select the array or vCenter Server whose resource pools
you want to unregister.
The Registered Resource Pools table is displayed in the Registered Storage tab. The
Registered Resource Pools table contains the list of all of the resource pools of the
Unregister resource pools

159

RPA Clusters

selected array or vCenter Server, that are currently registered at the selected RPA
cluster and their storage information on page 157.
4. In the Registered Resource Pools table:
a. Select the resource pool you want to unregister.
b. Click Remove.

Splitters
Note

This section is not relevant in XtremIO, which is a splitter-less solution.


Select RPA Clusters > RPA-Cluster-Name > Splitters to display all of the splitters that have
been added to RecoverPoint, add and remove splitters, and attach or detach volumes
from splitters.
Figure 10 RPA Clusters: Splitters

In RecoverPoint, all installed splitters are automatically added to the system, and all
volumes that are masked and zoned to the RPA are automatically attached to all existing
splitters at a site. In the rare case that a splitter is added to the system after the system is
already installed, you will need to add the splitter to the system, and attach the relevant
volumes to the splitter. Multiple splitters can be attached to one RPA cluster. See the EMC
RecoverPoint Release Notes for the limitation on the maximum number of splitters that can
be attached to a cluster.

Display splitter information


Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, select Splitters.

160

RecoverPoint 5.0 Administrator's Guide

RPA Clusters

Results

The Splitters table is displayed in the right pane, with the following splitter-specific
information:
Information

Description

Status

Whether the splitter is up or down.

Name

The name of the splitter in the RecoverPoint system.

Type

The type of storage that the splitter is installed on.

RPA Link

Whether or not there is connectivity between the RPA and the splitter.

The number of RPA clusters attached to the splitter.


Attached
RPA Clusters

When a VPLEX splitter is selected in the Splitters table, the Directors are displayed above
the Volumes Attached to Selected Splitter section.

Display volume information


Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, select Splitters. The Splitter table is displayed in the right pane.
3. In the right pane; select a splitter in the Splitter table.
The Volumes Attached to Selected Splitter section displays the volumes attached to
the selected splitter, and the following volume-specific information:

Display volume information

161

RPA Clusters

Table 34 Volume information

Information

Description

Group

The name of the group that the volume belongs to.

Copy

The name of the copy that the volume belongs to.

Replication
Set

The name of the replication set that the volume belongs to.

Access

The data access state of the volume, with regards to the splitter. Once a volume
is configured in RecoverPoint, including attachment to a splitter, the RPA can
place the volume in any of the displayed states.

Also
Replicated
by SRDF

Whether or not the volumes is also replicated by the Symmetrix replication


technology, SRDF.

Add splitters
Connects your RecoverPoint write-splitters to RecoverPoint.
Before you begin

162

Not relevant for XtremIO, which is a splitter-less solution.

If splitters have been installed on your arrays before installing RecoverPoint, they will
automatically be added to the RecoverPoint system, see splitter installation in the
EMC RecoverPoint Installation and Deployment Guide and the RecoverPoint technical
notes for your specific splitter type. However, if a storage array was added to the
system after deployment, the splitter will need to be added to the system.

All splitters that are masked and zoned to the RPAs are automatically added to the
system, unless they are already attached to another RPA cluster. VPLEX, VNX and
Symmetrix splitters are automatically added to the system regardless of masking and
zoning, unless they are already attached to the maximum allowed number of RPA
clusters. See the EMC RecoverPoint Release Notes for the maximum number of RPA
clusters that can be attached to a splitter.

Splitters residing on arrays that are not licensed are not automatically added to the
system.

RecoverPoint 5.0 Administrator's Guide

RPA Clusters

Splitters removed by the user will not be re-added automatically by the system.

ScaleIO splitters are added to the system after the automatic SAN rescan cycle is
complete. If the ScaleIO splitter is not visible in the system, manually run the
rescan_san CLI command.

Volumes added to replication sets are automatically attached to all splitters.

For VNX splitters: Although the two storage processors are displayed as separate
entities (Splitter 1-A and Splitter 1-B), they are managed as a single entity. If you add
or remove a splitter, the second storage processor instance is automatically added or
removed.

If a production storage volume is rolled back by a VNX SnapView session, the VNX
splitter will automatically initialize a full sweep of any of the volumes attached to
the splitter.

A single VNX splitter can be shared by up to four 4 RPA clusters. While attaching a
VNX splitter to a fifth RPA cluster appears to succeed, the splitter is in an error
state for the newly attached RPA cluster. All splitter operations for this RPA cluster
fail and return the error: Maximum RPA clusters per splitter exceeded. In this case,
remove the VNX splitter from the fifth RPA cluster.

For Unity/VNXe splitters:


n

The Unity/VNXe splitter is displayed as a single entity.

The Unity/VNXe splitter has a designated Snapshot Mount Point for every LUN. No
full sweep occurs when adding or rolling back array snaps.

The VNXe splitter does not support multiple clusters.

Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click Splitters.
3. In the right pane, click Add.

Note

The Add button is only enabled when there are splitters installed on the storage and
ready to add to the system.
4. In the Add Splitter dialog box; select the splitter that you want to add, and click OK.
If the selected splitter is a Unity/VNXe/VNX splitter, both SP-A and SP-B are added
when one of them is selected.

Add splitters

163

RPA Clusters

After you finish


Attach volumes to the splitter on page 164

Remove splitters
Note

You must Detach volumes from the splitter on page 165 before you can remove it from
the system. Only splitters that are not attached to any volumes can be removed.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click Splitters.
3. In the right pane; select one or more splitters in the Splitters table, click Remove, and
click Yes in the confirmation dialog box. If the selected splitter is a VNX splitter, both
SP-A and SP-B are removed when one of them is selected.

Attach volumes to the splitter


Before you begin
l

RecoverPoint automatically detects all paths from the splitter to the volume. If no
path exists between the splitter and the volume, you cannot attach the volume to that
splitter.

You cannot replicate a volume until it is attached to a splitter. You cannot fail over to
a local or a remote copy of a volume until it is attached to a splitter.

For VNX splitters:


n

Although the two storage processors are listed as separate entities (Splitter 1-A
and Splitter 1-B), they are managed as a single entity. If you attach or detach a
volume at one instance, the same volume will automatically be attached or
detached at the other storage processor instance.

If you attach a volume to an RPA cluster, and that volume is already attached to a
different RPA cluster that shares the same VNX splitter, the volume appears to
attach successfully to the second RPA cluster, but then faults to the Attached to
other RPA cluster/s error state. The volume cannot be used by the RPA cluster to
which it was just attached. To correct this error, click Detach to detach the volume
from the RPA cluster.
Note

To avoid this problem, a volume should be masked to a single RPA cluster. A


volume that is masked for one RPA cluster should not be masked for another RPA
cluster.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click Splitters.
3. In the right pane; select a splitter in the Splitters table, and click Attach in the
Volumes Attached to Selected Splitter section. The volumes that can be attached to
the splitter are displayed. Select one or more volumes to attach to the splitter. Click
OK to attach to selected volume to the splitter.

164

RecoverPoint 5.0 Administrator's Guide

RPA Clusters

NOTICE

When you attach a volume to a splitter, RecoverPoint ensures consistency between


the copy and the production source by performing a full sweep synchronization of the
volume. Checking the Attach as Clean checkbox overrides the default synchronization
process by informing the system that the copy volume is known to be an exact image
of its corresponding production volume. If Attach as Clean is checked, and the copy
volume being attached is inconsistent with its corresponding production volume, it
will remain inconsistent. To ensure consistency, best practice is not to select this
checkbox and use the default RecoverPoint synchronization process.

Detach volumes from the splitter


Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click Splitters.
3. In the right pane; select a splitter in the Splitters table, select a volume in the
Volumes Attached to Selected Splitter section, click Detach, and click Yes in the
confirmation dialog box.

Detach volumes from the splitter

165

RPA Clusters

RPAs
Select RPA Clusters > RPA-Cluster-Name > RPAs to display all of the RPAs in the
RecoverPoint system and the status and properties of each RPA.

The RPA table displays the following information for each RPA.
Table 35 RPA table information

Column

Description

Status

Indicates if the RecoverPoint appliance, LAN interface card, and WAN interface
card are alive.

ID

The ID of the RPA. Each RPA in an RPA cluster has an RPA ID that starts with the
string RPA and ends with a number from 1-8. Only RPA1 and RPA2 can run the
cluster control.

WAN IP

The WAN IP address of the RPA.

Management
IPv4

The LAN IP address of the RPA, in IPv4 format. Use this address to open a direct
session with an RPA.

Connectivity

Indicates if communication to all RPAs in the RPA cluster is alive and if the
storage link, repository volume, data link, and communication with the remote
RPA are all alive; if one RPA is down, connectivity of every RPA in the cluster will
report an error.

When an RPA in the RPA table is selected, the tabs at the bottom of the screen display the
RPA:

166

Properties on page 167

Status on page 168

Interfaces on page 169

RecoverPoint 5.0 Administrator's Guide

RPA Clusters

Properties
Displays the main properties of each RPA.

Select an RPA in the RPA table, and then click the Properties tab.
Table 36 Properties tab information

Field

Description

RPA Cluster

The RPA cluster that the RPA belongs to.

ID

The ID of the RPA. Each RPA in an RPA cluster has an RPA ID that starts with the
string RPA and ends with a number from 1-8. Only RPA1 and RPA2 can run the
cluster control.

Properties

167

RPA Clusters

Status
Displays the connectivity statuses of each RPA.

Select an RPA in the RPA table, and then click the Status tab.
Table 37 Status tab information

Field

Description

RPA Status

Whether or not the RPA is connected.

Repository
Volume

Whether or not the RPA can access the Repository Volume.

Storage Link

Whether or not the RPA can access storage.

WAN Interface

Whether or not the physical WAN port is alive, the interface card is functional,
and communication with other local RPAs exists.

FC Connectivity

Whether or not the RPA has access to the intranet.

Communication Whether or not the RPA has access to the remote RPA that it is paired with for
replication.
with Remote
RPA Cluster

168

Data Link

Whether or not data can be replicated to the remote RPA that the selected
RPA is paired with for replication.

LAN IPv4
Interface

Whether or not the physical LAN port is alive, the interface card is functional,
and communication with other local RPA (if any) exists.

RecoverPoint 5.0 Administrator's Guide

RPA Clusters

Interface
The Interface tab displays the Fabric Interface settings of each RPA.

Select an RPA in the RPA table and click the Interface tab to display the Fabric Interface
settings for each RPA HBA port. These settings are required when replacing a faulty RPA.

Traffic Statistics
Select an RPA in the RPA table and click the Traffic Statistics tab to display the RPAs
traffic statistics.

The Traffic Statistics tab displays the following statistics for the selected RPA:
l

Total traffic on page 170

Application traffic on page 170

Initialization traffic on page 170

Interface

169

RPA Clusters

Total traffic
Displays the total amount of data that is transferred by the selected RPA in the SAN and
between clusters, in Mb/s.

Application traffic
Displays the total amount of application data that is transferred by the selected RPA in
the SAN and between clusters, in Mb/s.

Initialization traffic
Displays the total amount of data that is transferred by the selected RPA in the SAN and
between clusters during RecoverPoint initialization, in Mb/s.

170

RecoverPoint 5.0 Administrator's Guide

RPA Clusters

Usage Statistics
Select an RPA in the RPA table and click the Usage Statistics tab to display the RPAs
resource usage statistics.

The Usage Statistics tab displays the following statistics for the selected RPA:
l

Incoming writes on page 171

Bandwidth reduction on page 172

CPU usage on page 172

Incoming writes
Displays the amount of incoming writes that are transferred by the selected RPA, per
second.

Usage Statistics

171

RPA Clusters

Bandwidth reduction
Displays the ratio by which the bandwidth usage of the selected RPA is reduced per
minute due to RecoverPoint.

CPU usage
Displays the percent of the selected RPAs CPU that is utilized per minute.

172

RecoverPoint 5.0 Administrator's Guide

RPA Clusters

vCenter Servers
Select RPA Clusters > RPA-Cluster-Name > vCenter Servers to display the vCenter server
table, monitor, register, or remove vCenter Servers. The vCenter user should have the
necessary privileges to perform the desired tasks.
Figure 11 RPA Clusters: vCenter Servers

When the vCenter Server's IP address is selected, the bottom pane displays the vCenter
Servers object, and under it, all VMware vCenter Servers registered with RecoverPoint.
The view displays data extracted from the VMware vCenter Server together with
RecoverPoint replication data.
In addition to displaying ESX servers and all their virtual machines, datastores, and RDM
drives, the RecoverPoint vCenter Server view also displays the replication status of each
volume. The RecoverPoint vCenter Servers view is for monitoring only (read-only).
Select the vCenter Servers link to:
l

Display vCenter Server information on page 174

Register vCenter Server on page 174

Edit vCenter Server Registration on page 175

Remove vCenter Server on page 175

Rescan vCenter Server on page 176

Before you begin


Before you begin, note that the best practice is to configure the vCenter Server to require
the use of a certificate because once RecoverPoint has read the certificate, it does not
need further access to the location.
The default certificate locations are:
l

Windows 2003 Server:


C:\Documents and Settings\All Users\Application Data\VMware
\VMware VirtualCenter\SSL\rui.crt.

Windows 2008 Server:


C:\Users\All Users\Application Data\VMware\VMware
VirtualCenter\SSL\rui.crt.
vCenter Servers

173

RPA Clusters

For more information about the location of the security certificate, refer to Replacing
vCenter Server Certificates VMware vSphere 4.0, available at www.vmware.com.

Display vCenter Server information


To display data from the VMware vCenter Server in Unisphere for RecoverPoint, select
RPA Clusters > RPA-Cluster-Name > vCenter Servers. When selected, the top-right pane
displays the vCenter Server table containing all vCenter Servers registered with
RecoverPoint and their user names. For detailed information about a specific vCenter
Server, in the top pane, click the vCenter Server's IP address to display:
l
Each ESX server in the vCenter Server and its IP address
l
Each virtual machine running in the ESX server, with the following details:
Table 38 Virtual machine info

Information
Replication
status

Icon

Description
Fully configured for replication
Partially configured for replication
Not configured for replication

IP address
l

The IP address of the virtual machine

Every LUN and raw device mapping accessed by each virtual machine, with the
following details:
n
Replication status: fully configured for replication or not configured for replication.
n
For LUNs or devices configured for replication by RecoverPoint, the following are
displayed: consistency group, copy (Production, Local, Remote), replication set,
and which datastore for each LUN or raw device mapping is configured for
replication.

Register vCenter Server


Creates a connection between RecoverPoint and a VMware vCenter Server, which allows
RecoverPoint to display the VMware view of volumes configured for replication.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click vCenter Servers.
3. In the right pane, click Add. The Register vCenter Server dialog box is displayed.

174

RecoverPoint 5.0 Administrator's Guide

RPA Clusters

4. In the Register vCenter Serverdialog box, enter the following information:


Table 39 RPA Clusters: Add vCenter Server

Setting

Description

vCenter Server IP address of the vCenter Server. This is also the display name of the vCenter
IP
Server in the top pane.
Port

TCP port number of the vCenter Server. Default = 443.

Username

vCenter Server username.

Password

vCenter Server password.

Certificate

If you wish to specify a certificate, browse to and select the certificate file.

Edit vCenter Server Registration


Use the following procedure to edit the credentials of an existing connection between
RecoverPoint and a VMware vCenter Server.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click vCenter Servers.
3. In the right pane; select a vCenter Server, and click Edit. The Edit vCenter Server
Registration dialog box is displayed.
4. In the Edit vCenter Server Registration dialog box, you can edit the following
information:
Table 40 Edit vCenter Server Settings

Setting

Description

vCenter Server IP address of the vCenter Server. This is also the display name of the vCenter
IP
Server in the top pane.
Port

TCP port number of the vCenter Server.

Username

vCenter Server username.

Password

vCenter Server password.

Certificate

If you wish to specify a certificate, browse to and select the certificate file.

Remove vCenter Server


Use the following procedure to remove a connection between RecoverPoint and a VMware
vCenter Server.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click vCenter Servers.
Edit vCenter Server Registration

175

RPA Clusters

3. In the right pane; select a vCenter Server, click Remove, and click Yes in the
confirmation dialog box.

Rescan vCenter Server


Use the following procedure to obtain the latest information from the VMware vCenter
Servers registered with RecoverPoint, and update Unisphere for RecoverPoint.
Procedure
1. Select RPA Clusters, and select the tab of the relevant RPA cluster.
2. In the left pane, click vCenter Servers.
3. In the right pane, click the Rescan icon.

176

RecoverPoint 5.0 Administrator's Guide

CHAPTER 6
Admin

This section describes how to license and register RecoverPoint, how to manage users,
roles, and user authentication, how to configure system notifications, detect bottlenecks
in replication, analyze the load on each RPA, and collect system information for support
purposes. This section also describes how to export statistics and use the statistics tool
through the RecoverPoint CLI.
l
l
l
l
l
l

RecoverPoint Administration............................................................................... 178


Manage Licenses................................................................................................ 178
Register RecoverPoint..........................................................................................182
Users and Security.............................................................................................. 187
System Analysis Tools......................................................................................... 200
System Notifications........................................................................................... 215

Admin

177

Admin

RecoverPoint Administration
Administration of RecoverPoint users and the RecoverPoint system is performed through
the Admin menu.

Table 41 Admin options

Option

When to Use

Manage
Licenses

To add or remove your RecoverPoint license files from the system or view your
RecoverPoint license details.

Fill out or update your RecoverPoint post-deployment form and automatically


Register
RecoverPoint send it to the EMC Install Base Group, registering your RecoverPoint product.

Users and
Security

To manage users, roles, and user authentication. To log off and log back in as a
different user, to change the currently logged in user, the password of the
currently logged in user, or after how much idle time to log off the user.

System
Analysis
Tools

To detect bottlenecks in replication, analyze the load on each RPA, and collect
system information for support purposes.

To configure system reports and alerts, manage event filters and call home
System
Notifications events.

Manage Licenses
Select Manage Licenses to add or remove RecoverPoint licenses from the system.

178

RecoverPoint 5.0 Administrator's Guide

Admin

Install a license
Before you begin
l

Access on page 242 and activate on page 244 your entitlements.

Obtain a RecoverPoint license file.

Procedure
1. Select Admin > Manage Licenses to display the Manage Licenses screen.

2. Then for each license file:


a. Click Add.

b. Click Browse...
c. Select the license file.
Install a license

179

Admin

d. Click OK.

Remove licenses
Procedure
1. Select Admin > Manage Licenses.
2. Select the license that you want to remove, and click Remove.
Note

A warning is issued if removing the license may cause the system to exceed the
allowed replication capacity (for example, when an array license is removed but an
existing consistency group contains volumes that reside on the removed array).

Monitor your license limits


Use the following procedure to ensure that you are replicating well within the limits of
your RecoverPoint license.
Procedure
1. Select Dashboard > System Limits > System.
2. Note the value of the following licensing-related system limits:
l

Number of bytes replicated locally per licensed RPA cluster

Number of bytes replicated remotely per licensed RPA cluster

Number of bytes replicated locally per licensed array

Number of bytes replicated remotely per licensed array

Number of days until RecoverPoint license expires

Export a detailed license report


You can export a detailed license report. The report is exported as a comma-delimited file
(*.csv) that can be opened in MS Excel or any other reader of CSV files.
Procedure
1. Select Admin > Manage Licenses.
2. Click the Export button at the top right corner of the screen.
3. In the Export dialog box:

a. Select the desired export options and click Next.

180

RecoverPoint 5.0 Administrator's Guide

Admin

b. Click Download to download the file to your computer, select a location on your
computer to store the file, and click Save.
4. Open the exported file:
a. Open MS Excel.
b. Open the *.csv file from the location you selected in the previous step. The Excel
Text Import Wizard is displayed to help you set the import options.
c. In the Excel dialog box, select Delimited, and click Next.
d. In the Delimiters field, select Comma, and click Next.
e. Click Finish.

Display your RecoverPoint license information


After you have added one or more licenses to RecoverPoint, you can display your
RecoverPoint license information through the Admin > Manage Licenses screen.

The following attributes are displayed, per license:


Table 42 Displayed license information

Parameter Values
Type

Description

RecoverPoint/CL Uses the RecoverPoint/CL licensing model.


RecoverPoint/EX Uses the RecoverPoint/EX licensing model.
RecoverPoint/SE Uses the RecoverPoint/SE licensing model.
RecoverPoint/BK Uses the RecoverPoint/BK licensing model.

RPA Cluster <RPA Cluster>

The name of the licensed RPA cluster.

Array Serial <Array Serial>

The serial number of the licensed storage array.

Replication Local

RecoverPoint/CL: None

RecoverPoint/SE and RecoverPoint/EX:


An array serial number.

The license supports replication to a local copy.

Remote

The license supports replication to a remote copy.

Capacity

<Capacity>

The replication capacity supported by the license (in terabytes).

Expiration

<Date>

The date at which the license expires.

Permanent

The license does not have an expiration date.

Active

The license is active.

Expired

The expiration date of the license has passed.

Status

Display your RecoverPoint license information

181

Admin

Table 42 Displayed license information (continued)

Parameter Values
Inactive

Description
The license is not active. This can happen when:
l

If a licensed array is not mapped and zoned to the RPA.

If a RecoverPoint/EX license and a RecoverPoint/CL license


are both added at the same RPA cluster, and the
RecoverPoint/EX license becomes inactive.

Note

Check the displayed information against the license file variables on page 269 for
correctness and Monitor your license limits on page 180 to ensure that you are
replicating well within the limits of your RecoverPoint license.

Register RecoverPoint
Select the Register RecoverPoint menu to fill out or update your RecoverPoint postdeployment form and automatically send it to the EMC Install Base Group, registering
your RecoverPoint product.

Register your RecoverPoint system


Before you begin
l

182

Register your RecoverPoint system after:


n

installing a RecoverPoint system

connecting RPA clusters in a RecoverPoint system

upgrading a RecoverPoint system

The RecoverPoint registration process will fail if you do not first configure your SMTP
address on page 227.

RecoverPoint 5.0 Administrator's Guide

Admin

The registration process will be incomplete if valid values are not provided for every
field in the post-deployment form.

When a new RPA cluster is added to the system, it is automatically added to the postdeployment form with empty parameter values. Ensure you have entered the values
for all new RPA clusters before sending the post-deployment form to the IB group.

Procedure
1. Select Admin > Register RecoverPoint to display the RecoverPoint Post-Deployment
Form.

2. Click Update Form to display the Update Post-Deployment Form Details dialog box.

3. In the Update Post-Deployment Form Details dialog box:


a. Select the relevant RPA cluster from the RPA Cluster drop-down.
b. Update the form information according to the instructions in the following table.
Table 43 Automatically registering your RecoverPoint system

Field

Description

For Example

Activity Type

Enter the type of activity you are


performing (upgrade, installation)

Upgrade

Location

The city, state and country where


the customer is located.

New York, New York, USA

Register your RecoverPoint system

183

Admin

Table 43 Automatically registering your RecoverPoint system (continued)

Field

Description

For Example

Upgrade/
Installation
Performing
Resource

Enter the role of the person


performing this upgrade or
installation activity

Customer

Sales Order
Number

The customer or Customer


Engineer should provide this
information

62276566

VCE

Indicate whether or not this


RecoverPoint implementation is
operating within a VCE (Vblock)
environment.

Selected=Yes
Unselected=No
or Unknown

VCE = VMware+Cisco+EMC
Connect Home

The method used to send


configuration reports and alerts to
EMC. Enabling this feature is
recommended as it allows EMC to
pro-actively address issues within
your RecoverPoint environment,
should they arise.

Software: ESRS (preferred), E-mail,


Not Allowed
Hardware: Not Applicable

Configure System Reports and


Alerts on page 226 to establish
connectivity for your RecoverPoint
environment.
Connect In

The method used to allow remote


connectivity to your RecoverPoint
environment. Enabling this feature
is recommended as it allows
secure access to your
RecoverPoint environment to
gather logs and resolve issues as
expeditiously as possible.
If you already have an ESRS
Gateway servicing other EMC
products, simply use the ESRS
Config Tool (refer to the EMC
Secure Remote Support Gateway
Operation Guide for further
instructions on Config Tool usage)
to add your RecoverPoint devices
(or have your gateway
administrator do it for you) to the
list of ESRS monitored
environments. Once the device is
added, click the request update
button to send the new device
information to EMC and contact

184

RecoverPoint 5.0 Administrator's Guide

Software: ESRS (preferred), WebEx,


or Not Allowed
Hardware: ESRS (preferred, WebEx,
or Not Allowed

Admin

Table 43 Automatically registering your RecoverPoint system (continued)

Field

Description

For Example

your local EMC Customer Engineer


to approve the update.
If you do not have an ESRS
Gateway at your site, contact your
EMC Account Manager to find out
more about the benefits of ESRS.

c. Click Apply to save each change to the form, without closing the dialog box.
d. Click OK to save all changes and close the dialog box.
NOTICE

If your company does not have SyR connectivity, and therefore, you cannot
Configure system reports and alerts on page 227, you can skip the rest of the
steps in this procedure, Export the post-deployment form to a CSV file on page
186 and Registering RecoverPoint by email or phone on page 186 now.
4. Click Send Form to automatically send the post-deployment form to the EMC Install
Base group. The Send Form dialog box is displayed.

5. In the Send Form dialog box, enter the email address of the person at your company
that is in charge of RecoverPoint maintenance and operation.
Note

Call home event 8000 is sent to EMC with your updated registration information and
an email is sent to the specified email address from EMC Customer Support to verify
that your registration details were updated successfully in the EMC Install Base.
6. Wait a couple of minutes.
7. Select Dashboard > System Events and look for events 8000 Calling homeCalling
home. If event 8000 is logged; the registration details were successfully sent to the
EMC Install Base.
Results
The following information is automatically sent along with the information that you
entered into the post-deployment form, per RPA cluster:
l

Software serial ID

Hardware serial IDs for RPAs 1-8

RecoverPoint software version

Cluster name

Register your RecoverPoint system

185

Admin

License type

Connect home method

The System ID (identical for all RPA clusters in a RecoverPoint system)

Export the post-deployment form to a CSV file


You can save your RecoverPoint registration information or Registering RecoverPoint by
email or phone on page 186 by exporting the RecoverPoint post-deployment form and all
of its contents as a comma-delimited file (*.csv) that can be opened in MS Excel or any
other reader of CSV files. The Activity Date field in the exported CSV file contains the date
and time that the file was exported.
Procedure
1. Select Admin > Register RecoverPoint.
2. Click the Export button at the top right corner of the screen.

3. In the Export dialog box:

a. Select the desired export options and click Next.


b. Click Download to download the file to your computer, select a location on your
computer to store the file, and click Save.
4. Open the exported file:
a. Open MS Excel.
b. Open the *.csv file from the location you selected in the previous step. The Excel
Text Import Wizard is displayed to help you set the import options.
c. In the Excel dialog box, select Delimited, and click Next.
d. In the Delimiters field, select Comma, and click Next.
e. Click Finish.

Registering RecoverPoint by email or phone


Registers RecoverPoint if your company does not have external connectivity, and
therefore you cannot register your RecoverPoint system online.
Before you begin
l

Register your RecoverPoint system after:


n

186

installing a RecoverPoint system

RecoverPoint 5.0 Administrator's Guide

Admin

connecting RPA clusters in a RecoverPoint system

upgrading a RecoverPoint system

The registration process will be incomplete if valid values are not provided for every
field in the post-deployment form.

Procedure
1. Access https://support.emc.com
2. Search for the term Post-Deployment Form
3. Download and fill out the EMC RecoverPoint and RP for VMs Post-Deployment Form
4. Send the information to the EMC Install Base group:
l

EMC customers and partners: Email the post-deployment form to the Install Base
group at rp.registration@emc.com.

EMC employees:
n

(Preferred) Use the IB Portal at http://emc.force.com/BusinessServices.

Call in the information to the Install Base group at 1-866-436-2411 Monday


to Friday (regular Eastern Time Zone working hours).

Users and Security


Select the Users and Security menu to manage users, roles and user authentication, to
log off and log in as a different user, to change the password of the currently logged in
user, the security level, or after how much idle time to log off the user.

Manage Users
Select Admin > Users and Security > Manage Users to view, add, remove and modify
RecoverPoint users.

Users and Security

187

Admin

How users work


You add users, edit users and remove users by selecting Admin > Users and Security >
Manage Users.
The RecoverPoint appliance is shipped with the following local users already defined:
Table 44 Pre-defined users

User

Role

Initial Password Permissions

security-admin security

security-admin

Security: changing users and roles, security levels,


LDAP configuration

admin

admin

admin

All, except security

boxmgmt

boxmgmt boxmgmt

Boxmgmt/Upgrade

Note

User authorization grants or denies users access to resources managed by RecoverPoint.


User authorization is identical, regardless of whether the user was authenticated by
RecoverPoint or by a Microsoft Windows Active Directory LDAP server. User authorization
can be limited to specific consistency groups.

Add new users


Before you begin
l

Only users with security permissions on page 192 can add users, and can remove
and edit permissions for users that have previously been added.

You cannot remove pre-defined users on page 188.

You cannot change the permissions of pre-defined users on page 188.

To change the passwords of pre-defined users on page 188, Switch User on page
199 and Change Password on page 198.

It is recommended that you change the passwords of predefined users.

If you wish to implement a purely LDAP-based authentication system, you need not
give out the passwords of any pre-defined users on page 188.

To be able to access RecoverPoint, every user in an LDAP Active Directory must be


added to RecoverPoint users list and assigned a role, or be a member of a group that
has access.

Procedure
1. Select Admin > Users and Security > Manage Users. The RecoverPoint user list is
displayed.
2. Click Add. The Add User dialog box is displayed.
188

RecoverPoint 5.0 Administrator's Guide

Admin

3. Select Local User or LDAP User or Group.


l

To configure a local user:


a. Provide a username and a password, according to your security level.
b. Select the role in the user settings section.
c. To limit access to specific consistency groups, click Only for selected groups
and select the consistency groups to which you are granting this user access.

To configure an LDAP user or group:


a. Click Select... to select either a user name or a group from the lists of users and
groups.
b. Select the role from the drop-down in the user settings section.
c. To limit access to specific consistency groups, click Only for selected groups
and select the consistency groups to which you are granting this user access.

Binding to the LDAP server will fail unless at least one of the following is true:
a. The connection to the server uses LDAP over SSL (only if the Active Directory
supports SSL).
b. Anonymous binding is used.
c. Signing requirements are set to "none" in Active Directory.
Settings

Description

Local User
User Name

Name of new user to add for appliance-based


authentication. A user name must start with a lower-case
alphabetic character. All subsequent characters must be
lower-case alphabetic, numeric, or hyphen (-). No other
characters are legal in user names.

Password

Password for new user. All printing ASCII characters are


legal in passwords.

Confirm
Password

Password for new user. All printing ASCII characters are


legal in passwords.

Manage Users

189

Admin

Settings

Description

LDAP User or
Group

To add users that already exist in the Active Directory on


the LDAP server, and grant them access to the
RecoverPoint appliance.

User Name

Name of a user to add to RecoverPoint user list.

Groups

Name of a group or groups (in Active Directory) to add to


the authorized users of RecoverPoint.

User Settings
Role

The role of the specified local user, LDAP user, or LDAP


group.

Only for selected When checked, limits the access of this local user, LDAP
groups
user, or LDAP group to the specified consistency groups.
Groups

Select the consistency groups that the specified user or


group may access.

Edit existing users


Before you begin
l

Only users with security permissions on page 192 can edit users.

You cannot edit pre-defined users on page 188.

Procedure
1. Select Admin > Users and Security > Manage Users.
2. To edit a local user, an LDAP user or LDAP group:
l

To modify the users password:


a. Read how security levels work in RecoverPoint.
b. Change Password on page 198

To modify the users username:


a. Remove the user according to Remove users on page 191.
b. Add new users on page 188 with a new username.

To modify the users role:


a. Select a user.
b. Click Edit.
The Edit Local User dialog box is displayed. The name of this dialog box
depends on the user type (Local, LDAP User, LDAP Group or LDAP User and
Group), but all of these dialog boxes contain the same options except for the
Edit Local User dialog, which additionally allows you to change the user and
password of the local user.

190

RecoverPoint 5.0 Administrator's Guide

Admin

c. Select one of the pre-defined roles on page 192 from the drop-down in the
User Settings section.
d. If required, modify the access to specific consistency groups by selecting Only
for selected groups.
e. Click OK.

Remove users
Before you begin
l

Only users with security permissions on page 192 can remove users.

You cannot remove pre-defined users on page 188.

Procedure
1. Select Admin > Users and Security > Manage Users.
2. Select a user.
3. Click Remove.

Manage Roles
Select Admin > Users and Security > Manage Roles to view, add, remove and modify
RecoverPoint roles.

Manage Roles

191

Admin

How roles work


You add roles, edit roles and remove roles by selecting Admin > Users and Security >
Manage Roles.
Each RecoverPoint user is defined by a user name, a password, and a role. A role is a
named set of access permissions. By assigning a role to users, the users receive all the
access permissions defined by the role.
Note

User authorization grants or denies users access to resources managed by RecoverPoint.


User authorization is identical, regardless of whether the user was authenticated by
RecoverPoint or by a Microsoft Windows Active Directory LDAP server. User authorization
can be limited to specific consistency groups.
The following pre-defined roles exist in RecoverPoint:
Table 45 Pre-defined roles

Role

Permissions

security

Security

Read only

System configuration

Storage management

Splitter configuration

Upgrade

Data transfer

Failover

Boxmgmt

Group configuration

Target image

Read only

admin

boxmgmt

Boxmgmt

monitor

Read only

Roles define the following permissions:


Table 46 Permissions

192

Permission

Description

Splitter
Configuration

Add or remove splitters, and attach or detach splitters to volumes.

Group
Configuration

Create and remove consistency groups, and modify all group settings except
those that are included in the data transfer, target image, and failover
permissions, may bookmark images, and resolve settings conflict.

RecoverPoint 5.0 Administrator's Guide

Admin

Table 46 Permissions (continued)

Permission

Description

Storage
Management

Register arrays and vCenter Servers, and update storage registration details for
arrays and vCenter Servers.

Data Transfer

Enable and disable access to images, and undo writes to the image access log.

Target Image

Enable and disable access to images, resume distribution, and undo writes to
the image access log.

Failover

Modify replication direction (in temporary failover and recover production),


initiate failover, verify failover.

System
Configuration

Configure and manage e-mail alerts, SNMP, System notifications, alert rules,
licenses, syslog, and other system configuration settings.

Security

All commands dealing with roles, users, LDAP configuration, and security level.

Upgrade

Installing RPA software, RPA maintenance, including upgrading to a minor


RecoverPoint release, upgrading to a major RecoverPoint release, replacing an
RPA, adding new RPAs, converting an environment to a RecoverPoint/SE or
RecoverPoint configuration, or modifying system settings.

Web
download

Download RecoverPoint installation packages from the EMC web site.

Add a new role


Before you begin
l

Only users with security Table 46 on page 192 can add, edit, or remove roles added
by the user.

You cannot edit or remove Table 45 on page 192.

Procedure
1. Select Admin > Users and Security > Manage Roles.
2. Click Add. The Add Role dialog box is displayed.

Manage Roles

193

Admin

3. Enter a new role name and set the required access permissions using the following
settings.
Table 47 Permission settings

Setting

Description

Role

Name of the new RecoverPoint role to add.

Permissions

Select the access Table 46 on page 192 to be granted to all persons who are
assigned to this role.

4. Click OK.

Edit an existing role


Before you begin
l

Only Table 44 on page 188 with security Table 46 on page 192 can edit roles.

You cannot edit Table 45 on page 192.

Procedure
1. Select Admin > Users and Security > Manage Roles.
2. Select a role.
3. Click Edit. The Edit Role dialog box is displayed.

4. Change the role name and/or select new Table 46 on page 192.

Remove a role
Before you begin

194

Only pre-defined users on page 188 with security permissions on page 192 can
remove roles.

You cannot remove pre-defined roles on page 192.

RecoverPoint 5.0 Administrator's Guide

Admin

Procedure
1. Select Admin > Users and Security > Manage Roles.
2. Select a role.
3. Click Remove, and click yes in the confirmation dialog box that appears.

Manage User Authentication


Select Admin > Users and Security > Manage user Authentication to configure access to an
LDAP Active Directory for user authentication.

How user authentication works


User authentication configures RecoverPoint to use the organizations Microsoft Windows
Active Directory LDAP server for authentication.
RecoverPoint provides two independent mechanisms for authenticating users: appliancebased authentication and authentication via the organizations Microsoft Windows Active
Directory LDAP (Lightweight Directory Access Protocol) server. The two authentication
mechanisms can be used simultaneously, LDAP may be used exclusively, or appliancebased authentication can be used exclusively.

Authenticate users using LDAP


Before you begin
l

Only users with security permissions on page 192 can configure user authentication
through an organizations Microsoft Windows Active Directory LDAP server.

Although optional, it is strongly recommended to have secondary LDAP servers at


both the production and at the copy sites. If no LDAP server is available, users may
not be able access RPAs or use RecoverPoint.

RecoverPoint only accepts LDAP certificates in PEM format. To format the certificate
created on the LDAP server in PEM format, use the following procedure:
1. On the LDAP server, export a copy of the server certificate from the Active Directory
server. Use the Certification Authority application's Copy File to option to export
the certificate in Base-64 Encoded X.509 (.cer) format.
2. Copy the server certificate to a system with OpenSSL Certificate Authority software
installed. You can use any Linux or Windows system.
3. Log into the system where you copied the certificate, and run the following
command to create the PEM certificate:
n

On Linux: > /opt/symas/bin/openssl x509 -in


AD_certificate_name -out OpenLDAP_certificate name

On Windows: > openssl x509 -in drive:/path/AD_file.cer inform d -out drive:/path/OpenLDAP_file.pem

Manage User Authentication

195

Admin

Procedure
1. Select Admin > Users and Security > Manage User Authentication. The Manage User
Authentication screen is displayed.

2. Configure the LDAP settings.


Table 48 LDAP settings

Settings

Description

Enable Active
Directory Support

Check to activate RecoverPoint authentication and authorization using a


Microsoft Windows Active Directory LDAP server.

LDAP Configuration
Primary LDAP server IP address of the primary LDAP server
Secondary LDAP
server

(optional) IP address of the secondary LDAP server.


Although optional, it is strongly recommended to have secondary LDAP
servers at both the production and at the copy sites. If no LDAP server is
available, users may not be able access RPAs or use RecoverPoint.

Base Distinguished Node in the LDAP tree from which to start a search for users:
Name
dc=Klaba,dc=COM
Search Base
Distinguished
Name

Root of the LDAP user search tree. The suffix of the Search Base
Distinguished Name must be the Base Distinguished Name. The format will
be similar to the following:
cn=Users,dc=Klaba,dc=COM

Binding Type:

196

RecoverPoint 5.0 Administrator's Guide

Admin

Table 48 LDAP settings (continued)

Settings

Description

To specify the type of binding (authentication against the Active Directory).


Bind Distinguished
Name

Distinguished name to use for initial binding when querying the LDAP
server. The format of the Bind Distinguished Name will be similar to the
following:
cn=Administrator,cn=Users,dc=Klaba,dc=COM
The bind distinguished name can be any user on the LDAP server who has
read permission for the directory in the defined search base.

Password

Password of the bind distinguished name to use for initial binding when
querying LDAP server

Directory Access Protocol


LDAP

To send LDAP query over non-secure connection.

LDAP over SSL

To send LDAP query over a secure connection.

Upload certificate
from file

Select if you want to enter a path to an Active Directory certificate to use for
secure communication with LDAP server.
To export the certificate and format it in PEM format:
On the LDAP server, export a copy of the server certificate from the Active
Directory server. Use the Certification Authority application's Copy File to
option to export the certificate in Base-64 Encoded X.509 (.cer) format.
Copy the server certificate to a system with OpenSSL Certificate Authority
software installed. You can use any Linux or Windows system.
Log into the system to which you copied the certificate, and run the
following command:
On Linux:
> /opt/symas/bin/openssl x509 -in AD_certificate_name -out
OpenLDAP_certificate name
On Windows:
> openssl x509 -in drive:/path/AD_file.cer -inform d -out drive:/path/
OpenLDAP_file.pem

Advanced settings
The advanced settings are optional and can be left at their default values.
Search time limit

Default 30 sec.

3. Click Apply.
4. Test LDAP settings on page 198 to verify that you have configured LDAP correctly.

Manage User Authentication

197

Admin

Test LDAP settings


Procedure
1. Select Admin > Users and > Security > Manage User > Authentication.
2. Enter the LDAP Configuration settings on page 196.
3. Click Apply.
4. Click Test configuration.
Note

An error is displayed if the system is unable to connect to the primary or secondary


LDAP servers.

Modify LDAP settings


Procedure
1. Select Admin > Users and Security > Manage User > Authentication.
2. Click Clear LDAP Settings, and click yes in the confirmation dialog box.
3. Modify the LDAP Configuration settings on page 196.
4. Click Apply.
5. Test LDAP settings on page 198 to verify that you have configured LDAP correctly.

Change Password
Select Admin > Users and Security > Change Password to change the password of the
currently logged-in user.

Procedure
1. Enter your current password.

2. Enter the new password in the New Password and Confirm Password fields according
to the password rules of your user's security level.
198

RecoverPoint 5.0 Administrator's Guide

Admin

3. Click OK.

Switch User
Use the following procedure to log off, and log back in as a different RecoverPoint user.
Select Admin > Users and Security > Switch User to log off, and log back in as a different
RecoverPoint user.

Procedure
1. Enter the username and password of the user as whom you want to log into
RecoverPoint.

2. Click Login.

Change Security Level


Use Admin > Users and Security > Change Security Level to set the password strength and
security rules for the currently logged-in user.

Change a users security level


Sets the security level of user passwords that are required to access the RPAs in your
system.
Before you begin
l

Only pre-defined users on page 188 with security permissions on page 192 can
change the security level of a user.
Switch User

199

Admin

Regardless of the security level, any user who tries unsuccessfully three times to log
in will be locked out. To unlock the user, use the CLI command unlock_user. Only
users with security permission can unlock a user.

The default security level is set to Basic. It is recommended that RecoverPoint


administrators set the level to High to meet relevant security standards, such as those
of the USA Department of Defense Security Technical Implementation Guides (DoD STIG).

When the security level is set to High, user passwords:

must contain a minimum of eight characters; at least two characters must be


lower-case, at least two upper-case, and at least two characters non-alphabetical
(either digits or special characters).

can only be reset once every 24 hours.

cannot be reused until at least ten other passwords have been used.

expire in 90 days.

When the security level is set to Basic, user passwords must have a minimum of five
characters.

Procedure
1. Select Admin > Users and Security > Change Security Level.
2. Select either High or Basic.
3. Click OK.

Change Idle Logout Time


Use Admin > Users and Security > Change Idle Logout Time to specify after how much time
without user input to automatically log off the user.

System Analysis Tools


Use the System Analysis Tools menu to detect bottlenecks in replication, analyze the
load on each RPA, and collect system information for support purposes.

200

RecoverPoint 5.0 Administrator's Guide

Admin

Note

You can also access additional system analysis tools through the RecoverPoint CLI. Use
the RecoverPoint CLI to export statistics , export consolidated statistics , and load the
statistics analysis tool.

Detect Bottlenecks
Use Admin > System Analysis Tools > Detect Bottlenecks to display statistics about
system performance by group, RPA, or RPA cluster, and use the statistics to identify and
mitigate bottlenecks in transfer.

Procedure
1. Define the Bottleneck Detection Settings as described below. Note that in bottleneck
detection, the entered times are according to the local time and the analysis output is
according to GMT.

Detection Mode
Default=System overview and bottleneck analysis
Type of analysis to perform.
Possible values are System overview and bottleneck analysis, Initialization periods,
High load periods, Initialization, high load, and peak analysis, Advanced analysis,
and Peak analysis.
Start
Default=Earliest available time
Start time for analysis. Possible values are:
Detect Bottlenecks

201

Admin

l
l

Earliest available time


A user-specified date and time, where the date and time, if not specified, is
the current local time today.
Note

Enter times according to local time. Output is according to GMT.


End
Default=The current date and time
End time for analysis, where the date and time, if not specified, is the current
local time today.
Note

Enter times according to local time. Output is according to GMT.


Advanced Overview
Default=no
Whether the analysis covers a broader set of system performance indicators (yes)
or not (no).
Detailed Overview
Default=no
Whether the analysis is confined to the RPA cluster only (no) or also includes
analysis of the groups and RPAs (yes).
Peak Duration
Default=5 minutes
Includes an analysis of the periods in which the write peaks were of the specified
time duration.
Groups to Include
Note

Not relevant when System overview and bottleneck analysis is selected.


Default=all
The consistency groups on which to perform the analysis. To analyze only global
statistics, without statistics on individual groups, clear all of the group
checkboxes.
2. Click Start to start the analysis. A progress bar is displayed.

202

RecoverPoint 5.0 Administrator's Guide

Admin

[Wait for the process to complete]


3. During analysis, you can use any of the following options.
Action

Description

Start New
Analysis

To stop the current analysis and display the screen containing the bottleneck
analysis settings.
Note

Only available when analysis is in progress.


Stop Analysis

To stop bottleneck analysis.


Note

Only available when analysis is in progress.


Display
Results

To open a window containing the bottleneck analysis results.


Note

Only available when analysis is complete.

4. Click Display Results. The bottleneck detection results are displayed.

Detect Bottlenecks

203

Admin

5. Copy the results and paste them into a text file.


6. Save the text file for later reference.

Balance Load
Use this procedure to analyze the load on each RPA and display a load balancing
strategy.
Before you begin

204

Load balancing is only available on the RPA running the cluster control, or through
the floating Cluster Management IP. Load balancing will not work if the RPA running
the cluster control is down. If this RPA goes down, and then back up again, five
minutes must pass before load balancing can be performed again.

For best results, groups should be configured and replicating for seven days before
performing load balancing, so that a long enough traffic history is available for the
load balancing analysis.

The load balancing analysis is performed on all RPAs at all sites.

Groups can be excluded from the load balancing recommendation. However, even
when excluded from the recommendation, all groups are included in the analysis.

The load balancing analysis is performed on all RPAs in the environment. If there are
three RPAs in total, and one of them goes down for a week, and it is the primary RPA
of a group included in the analysis, the load balancing recommendation calculations
are still based on the total number of RPAs, as if they were all working. In other
words, the load balancing mechanism accumulates group statistics for its
calculations, as if groups were always running on their preferred RPAs, even if
flipovers happened during the week.

Load balancing can identify cases in which distributed group segments do not
actually require four separate RPAs to run on, and recommend RPA assignments
accordingly. Distributed group segments can be run on different RPAs, or the same
RPA, and each group segment is treated as a separate group in the load balancing
analysis and recommendation. If you are not well acquainted with distributed
consistency groups, it is recommended that you become acquainted with how they

RecoverPoint 5.0 Administrator's Guide

Admin

work before using the load balancing tool (see the EMC RecoverPoint Glossary of Terms
for more information on distributed consistency groups).
l

To ensure consistency, consistency groups are initialized when moving from RPA to
RPA. During switchover, all groups running on the preferred RPA are initialized once
when they move to the non-preferred RPA, and then another time when they switch
back to their preferred RPAs. Therefore, any re-assignment of RPAs during replication
should be carefully planned out, as not to affect the performance of the production
host applications.

Since the modification of preferred RPA assignments causes flipovers that cause
initializations, if the load balancing analysis finds that no significant changes in
workload or number of consistency groups have occurred, the mechanism will not
recommend preferred RPA re-assignment.

The load balancing mechanism smooths-out I/O statistics, so that any sudden peaks
or lapses in traffic do not cause it to recommend unnecessary changes.

Procedure
1. Select Admin > System Analysis Tools > Balance Load.
2. Select the consistency groups to exclude from the analysis.

Note

The groups that you select will be preceded by an asterisk in the load balancing
recommendation, and their traffic statistics will be included in the analysis upon
which the recommendation is based.
3. Click Start to start the analysis. A progress bar is displayed.

[Wait for the process to complete]

Balance Load

205

Admin

4. During analysis, you can use any of the following options:


Action

Description

Start New
Analysis

To stop the current analysis and display the screen containing the load
balancing settings.
Note

This option is only enabled when analysis is in progress.


Stop Analysis

To stop the load balancing analysis.


Note

This option is only enabled when analysis is in progress.


Display Results

To open a window containing the load balancing analysis results.


Note

This option is only enabled when analysis is complete.

5. When the analysis is complete:


l

Click Display Results. The load balancing recommendation is displayed.

A normal scope event is logged indicating that the load balancing process has
ended. The event contains a copy of the load balancing recommendation output.

6. Understand the load balancing results:


l

If the load balancing recommendation is based on less than 7 days-worth of data,


a note is displayed indicating so.

One or more of the following load balancing recommendations are displayed:


n

206

No action is necessary. The environment is stable, and groups are evenly


distributed across all RPAs.

RecoverPoint 5.0 Administrator's Guide

Admin

Action is necessary. The environment is not stable because groups are not
evenly distributed across all RPAs. To correct this, manually modify the
Preferred RPA of each group.

Action may be necessary. The environment is stable, although groups are not
evenly distributed across all RPAs, and this may affect future performance. To
distribute groups evenly across all RPAs, manually modify the Preferred RPA of
each group.

The current preferred and recommended RPA assignments are displayed per
consistency group, along with each RPAs average throughput and incoming writerate, in a Recommended Load Distribution table.
Note

For distributed groups, each consistency group segment is treated as a separate


group, and the performance statistics are displayed per group segment.

Two tables, Traffic per RPA before Application of Recommendation and Traffic per
RPA after Application of Recommendation, are displayed in order to aid you in the
process of understanding the performance implications on all RPAs, if you do
choose to apply the recommendation. Each RPAs average throughput and IOPS are
displayed, and the RPA with the least amount of traffic is highlighted, in each of
the tables.

Balance Load

207

Admin

If any non-distributed (regular) consistency groups have experienced a minute of


throughput that exceeded the maximum throughput rate of a single RPA (as
described in EMC RecoverPoint Release Notes) at least 50 times during the week,
the load balancing recommendation indicates that these groups should be set as
distributed. If any distributed consistency groups have experienced less than 40
minutes of throughput at a rate of over 60 MB/sec during the week, the load
balancing recommendation indicates that these groups should be set as nondistributed.

If the total group throughput is exceeded, a message is displayed indicating that


you should add additional RPAs to your RPA cluster, and run load balancing
analysis again after 7 days. See the EMC RecoverPoint Release Notes for the total
throughput per distributed consistency group and total throughput per RPA
limitations.

7. Retrieve the load balancing results:


a. Open a browser window and type the following path into your browsers address
bar:
<Cluster_Management_IP>/info/load_balancing/
b. Log in using boxmgmt credentials.
c. Download the relevant text file(s) and store them for later reference. The load
balancing output is saved to text files with the name:
balance_load_yyyy-mm-dd_hh-mm.txt
Note

Ten load balancing output files are stored in this location at a time. When ten files
have already been stored, the older files are removed to make way for newer files.
The date and time of the output file is indicated in the filename.
208

RecoverPoint 5.0 Administrator's Guide

Admin

d. Save the text files to a local directory for later reference.


8. Change the preferred RPAs of consistency groups based on the load balancing
recommendation:
a. Select Protection > Manage Protection.
b. In the left pane, select the group whose preferred RPA you want to change.
c. In the right pane, select the Group Policy tab.
d. Change the value of Preferred RPA according to the load balancing
recommendation. If the group is a distributed group and the load balancing
recommendation indicates so, change the value of Secondary RPAs too.
e. Click Apply.
Each time a consistency groups preferred or secondary RPAs are modified, the
group undergoes an initialization to ensure consistency. For distributed
consistency groups, if one group segment is initialized, all group segments are
initialized.
9. Verify that load balancing improved performance on page 209

Verify that load balancing improved performance


After running load balancing, use the following procedure to check the implications on
performance.
Procedure
1. Wait 7 days, for a new traffic history to be available.
2. Balance Load on page 204.
3. Compare the old load balancing results and performance statistics to the new load
balancing results and performance statistics.

Collect System Information


Use Admin > System Analysis Tools > Collect System Information to retrieve system log
files and environment information for support purposes.

How system information collection works


To analyze and resolve support cases, you can Collect system information on page 209
for the past thirty days or you can Collect system information around a specific event on
page 33. System information is collected using the Collect System Information wizard.

Collect system information


Before you begin
l

In order to collect system information for VNX and VPLEX splitters, you must first
Attach volumes to the splitter on page 164.
Collect System Information

209

Admin

The system information collected from RPAs also includes Dell OMSA hardware
configuration information, provided the RPAs are running on Dell PowerEdge
platforms.

To retrieve the output file containing the system information, you must log in as one
of the Table 44 on page 188 with boxmgmt Table 46 on page 192.

It is not possible to collect Unity/VNXe system information remotely through the RPA.
Collect Unity/VNXe and RecoverPoint Unity/VNXe splitter system information
manually using the Unity/VNXe Unisphere. Unity/VNXe credentials are required for
system information collection.

Procedure
1. Attach volumes to the splitter on page 164 for all splitters in the system.
2. Select Admin > System Analysis Tools > Collect System Information.
3. If this is your first time collecting system information, click Start. Otherwise, click
Start New Collection Process.
The Collect System Information wizard is displayed, to guide you through the
collection process.

NOTICE

You can also collect system information around a specific event by selecting
Dashboard > System Events. Then select an event in the event log and click Collect
System Information. The Collect System Information wizard is displayed and the
collection start and end fields are automatically set to 10 minutes before and after the
selected event.
4. Define the Table 49 on page 210.
Table 49 System information collection settings

210

Setting

Description

From

Default=yesterday, the current hour


Specify the date and time (in GMT or your local time) of the first system
information that you want to include in the output.

RecoverPoint 5.0 Administrator's Guide

Admin

Table 49 System information collection settings (continued)

Setting

Description
Although the system information of the past thirty days is available for
collection, only three days of system information can be collected at a
time.
GMT is not adjusted for daylight savings time.

To

Default=today, the current hour.


Specify the date and time (in GMT or your local time) of the latest system
information that you want to include in the output.
Although the system information of the past thirty days is available for
collection, only three days of system information can be collected at a
time.
GMT is not adjusted for daylight savings time.

Include Information
from RPA Clusters

Optional
Default=selected
When selected, enables the collection of system information from RPA
clusters.

RPA clusters

Optional
Default=all RPA clusters
The RPA clusters from which to collect system information. If none of the
checkboxes are selected, only the system information of the RPA cluster
from which the collection process is triggered is collected.
Only relevant when Include Information from RPA Clusters is selected.

Include core files

Optional
Default=disabled
Whether or not to include core files in the output. Core files may be large.
Subsequently, including these files in the collection process may
substantially increase collection time.
Only relevant when Include Information from RPA Clusters is selected.

Full system log


collection

Optional
Default= selected
When selected, enables the collection of all system logs.

Advanced

Optional
Default= disabled
Use only when explicitly instructed to do so by EMC Customer Support.
When selected, enables the collection of system logs. Use this option to
reduce log collection time.
Select one or more of these categories:
l

CLI

Control

Replication

Storage
Collect System Information

211

Admin

Table 49 System information collection settings (continued)

Setting

Include Information
from Splitters

Description
l

Monitor

Management

Optional
Default=selected
When selected, enables the collection of system information from the
selected splitters.
Note

Attach volumes to the splitter on page 164 before system information


can be collected.
Note

Unity/VNXe splitter system information must be collected manually from


the array.
Place the output
Optional
file(s) on my local RPA Default=enabled
When selected, copies the collection process output file(s) to the RPA
from which the collection process was triggered.
The output file(s) are placed in: http://[RPA IP address]/info or https://
[RPA IP address]/info. Use boxmgmt credentials to log in and view the
information.
Copy the output file(s) Optional
to an FTP server
Default=disabled
When selected, copies the collection process output file to the specified
FTP server.
When Copy the output file(s) to an FTP server is selected:
FTP Server Address

The IP or DNS address of the FTP server to which to upload the collection
process output file.
For example: 10.250.20.40 or ftp.EMC.com

Port

The port through which to access the specified FTP server.

User Name

The username to use when logging into the specified FTP server.

Password

The password to use when logging into the specified FTP server.

Remote Path

Optional
The path to the copy of the output file stored on the specified FTP server.
For example: / (to access the rootdir)

Override default
filename (Not
recommended)

Optional
Default=disabled
Whether to override the default file name of the output file placed on the
FTP server.
It is recommended to keep the default file name.

212

RecoverPoint 5.0 Administrator's Guide

Admin

Table 49 System information collection settings (continued)

Setting

Description
When selected, RecoverPoint renames the output file uploaded to the FTP
server according to the new file name specified in the Filename field.

New File Name

Optional
The new file name for the output file placed on the FTP server.
Only relevant when Override default filename is selected. It is
recommended to clear the Override default filename checkbox, and keep
the default file name as is.

5. Click Next Test Connectivity. The Test Connectivity screen is displayed.

6. In The Test Connectivity screen, review the displayed information to verify that your
configuration settings are correct, and the system can access all RPAs and splitters:
l

Verify that the Collection Settings are configured as intended. If required, click
Back to Define Settings to edit your settings.

Verify that a green check mark exists in the Status column of all listed splitters and
RPAs, and that the text Action succeeded exists in the Details column of all listed
splitters and RPAs. If all splitters and RPAs do not have green check marks in their
Status columns, you can collect the information directly from individual hosts on
which the feature is enabled.

7. Click the Finish button to initiate the collection process. A progress bar is displayed,
and the collection process begins.

Collect System Information

213

Admin

Note

Be patient. The collection process can take awhile, depending on the time span and
the amount of data being collected.
8. During the collection process, you have the following options:
Table 50 Collect system information options

Option

Description

Start New Collection Process

To stop the current collection process and display the screen


containing the collection settings.

Stop Collecting System


Information

To stop the current collection process.

Display Results

To open a window containing the collected system information.

9. After the collection process is complete, click Display Results.

214

RecoverPoint 5.0 Administrator's Guide

Admin

10. In the Collection Results dialog box:


a. Verify that the specified system information has been successfully collected. Verify
that a green check mark exists in the Status column of all listed splitters and RPAs,
and that the text Action succeeded exists in the Details column of all splitters and
RPAs. If all splitters and RPAs do not have green check marks in their Status
columns, you can collect the information directly from individual hosts on which
the feature is enabled.
b. Retrieve the system information. Click the Output file (HTTP) link, or Output file
(HTTPS) link, and enter the username and password of Table 44 on page 188 with
boxmgmt Table 46 on page 192, and click Close.
Upon completion of the collection process, the output file is placed in: http://[RPA
IP address]/info or https://[RPA IP address]/info. Use boxmgmt credentials to log
in and view the information.
11. If you enabled the Copy output file to an FTP server option, you can now retrieve the
remote copy of the output file from the FTP server:
a. Open a Web browser window.
b. Enter the FTP Server Address or DNS name of the FTP server into the address bar.
c. At the login prompt, enter the User Name and Password.
d. Browse to the Remote Path .

System Notifications
Use the System Notifications menu to manage event filters, configure system reports and
alerts and manage call home events.

Manage Event Filters


Use Admin > System Notifications > Manage Event Filters to define and modify the
settings for the sending of outbound system events through email, SNMP, and/or Syslog.

System Notifications

215

Admin

Display all event filters


To display all of the event filters currently defined in the system, select Admin > System
Notifications > Manage Event Filters.

Create an event filter


Procedure
1. Select Admin > System Notifications > Manage Event > Filters.
2. Click Add. The Event Filter wizard is displayed.
3. In the Define Filter screen:

216

RecoverPoint 5.0 Administrator's Guide

Admin

a. Define the event filter settings according to the instructions in the following table.
Table 51 Event Filter settings

Setting

Description

Filter Name

Enter a unique name of the event filter to be created.

Topic

Select which events to report, according to the component of the


RecoverPoint system on which the events occur:

Groups

All Topics

Site

RPA

Consistency Group

Splitter

Array

Only relevant when Topic is set to All Topics or Consistency Group.


Select the names of the groups that are to be subject to this filter. There is
no default value for this parameter.

Level

Scope

Event IDs to
Exclude

Info: Event is informative in nature, usually referring to changes in the


configuration or normal system state. When Level=Info, events of all
levels are returned; there is no filtering of events as a result of the
Level setting.

Warning: Event indicates a warning, usually referring to a transient


state or an abnormal condition that does not degrade system
performance. When Level=Warning, all events with level of Warning or
Error are returned; Info-level events are filtered. The Warning level
includes Warning Off events.

Error: Event indicates an important event that is likely to disrupt normal


system behavior and/or performance. When Level=Error, only events at
the Error level are returned; Warning-level and Info-level events are
filtered. The Error level includes Error Off and Brief Error events.

Normal: To report selected basic events and root-cause events. A rootcause event returns a single description for a system event that can
spawn an entire set of detailed and advanced events. When
Scope=Normal, all of those detailed and advanced events are filtered
out, leaving only the normal events. This is the default setting, and in
most cases, it is sufficient for effective monitoring of system behavior.

Detailed: This category includes all events (that is, detailed and
normal), with respect to all components, that are generated for use by
users.

Advanced: In specific cases (for instance, for troubleshooting a


problem) EMC Customer Support may ask you to retrieve information
from the advanced log events. These events contain information that is
intended primarily for the technical support engineers.

The IDs of the Events that are not subject to this filter.

Manage Event Filters

217

Admin

Table 51 Event Filter settings (continued)

Setting

Description
Note

Use a comma to separate multiple event IDs.

b. Click Next Define Method of Transfer.


4. In the Define Method of Transfer screen:

a. Select one or more methods of transfer.


b. If you selected Email:
l

Select the frequency with which the events will be sent.

Enter the email addresses of all recipients in a comma-separated list.

c. Click Finish to save the event filter and exit the wizard.
5. Configure the transfer method on page 218 for each method of transfer that you
defined.

Configure the transfer method


Events can be transferred to the specified servers and/or email addresses by email,
syslog or SNMP.

Configure Email event notification


Use the following table to enable the transfer of events by email.
Procedure
1. Select Admin > System Notifications > Manage Event > Filters.
2. In the bottom pane, select the Email tab.

218

RecoverPoint 5.0 Administrator's Guide

Admin

3. Enter the values for the fields in the Email tab according to the instructions in the
following table.
Table 52 Email settings

Setting

Description

Enable Email
Notifications

Select to enable the events defined in any event filters whose method of transfer
is set to Email to be transferred to the specified server from the specified sender.
Note

Selecting this checkbox also enables the sending of email alerts to the specified
server address if you enabled both reports and alerts in the Configure System
Reports and Alerts on page 226 screen.
SMTP server
address

Enter the IP address or DNS name of the server to which to send the emails.
Note

When you click Apply, any change to this field automatically updates the SMTP
server address field in the Configure System Reports and Alerts on page 226
screen.
Sender
address

Enter the email address that should appear in the sender field of all emails
concerning sent to the specified server regarding event notifications.

4. Click Apply.
Results
All of the event filters whose Method of Transfer includes Email will be transferred to the
recipients defined in the filter (at the frequency defined in the filter) using the specified
server address and sender.

Configure Syslog event notification


Use the following procedure to enable the transfer of events by syslog.
Procedure
1. Select Admin > System Notifications > Manage Event Filters.
2. In the bottom pane, select the Syslog tab.

Manage Event Filters

219

Admin

3. Enter the values for the fields in the Syslog tab according to the following table.
Table 53 Syslog settings

Setting

Description

Enable Syslog
Notifications

Select to enable the events defined in any event filters whose method of transfer
is set to Syslog to be transferred to the specified servers with the specified label.

Facility

Select one of the available labels to be attached to all messages.

Specify target
host for

(Optional) Specify the syslog server at each RPA cluster to which you wish to
deliver notifications. The address may be either in IP or DNS format. A DNS
address will work only if a DNS server is configured in the RecoverPoint system.

4. Click Apply.
Results
All of the event filters whose Method of Transfer includes Syslog will be transferred by
Syslog using the specified target host and facilities.

Configure SNMP event notification


Use the following procedure to enable the transfer of events by SNMP.
RecoverPoint supports the standard Simple Network Management Protocol (SNMP),
including support for SNMP version 3 (SNMPv3). RecoverPoint supports various SNMP
queries and can be configured to generate SNMP traps (notification events), which are
sent to designated network management systems.
Procedure
1. Select Admin > System Notifications > Manage Event Filters.
2. In the bottom pane, select the SNMP tab.

220

RecoverPoint 5.0 Administrator's Guide

Admin

3. Enter the values in the fields of the SNMP tab according to the following table.
Table 54 SNMP settings

Setting

Description

Enable SNMP
Agent

Select to enable the RecoverPoint SNMP agent to send all of the events defined
in any of the event filters whose method of transfer is set to SNMP to the
specified servers with the specified label.
The RecoverPoint SNMP agent must be enabled in order to send SNMP traps
(notification events) and to respond to host-initiated SNMP queries.

Allow secure
transports
only

When selected, only encrypted SNMP queries over secure transport will be
enabled; use of community strings and SNMP over an unsecured port will be
disabled.

Send event
traps

When selected, sends SNMP traps (notification events) to the RecoverPoint


SNMP agent according to your SNMP trap configuration on page 222. Does not
affect host queries.

Specify trap
(Optional) The network management server to which you wish to deliver
destination for notifications. The address may be either in IP or DNS format. A DNS address will
work only if a DNS server is configured in the RecoverPoint system.
SNMP v1 RO
community

If you are using SNMP version 1, enter an SNMPv1 read-only community string to
enable the use of SNMPv1 community strings. If not enabled, only SNMPv3 user
names with passwords or certificates are enabled.
Note

The read-only community string is transmitted in cleartext.

4. To add an SNMPv3 user:


a. Click Add in the SNMPv3 Users area. The Add SNMPv3 User dialog box is
displayed.

Manage Event Filters

221

Admin

b. Enter the required information.


c. To add a certificate, click Browse and select the certificate. The certificate file can
contain more than one certificate; select one. All certificates in the file are equally
valid. For optimal security, specify a user certificate and do not specify a password.
5. To remove an SNMPv3 user, select the user in the SNMPv3 Users table and click
Remove.
6. Click Apply to save your changes.
7. Perform SNMP trap configuration on page 222.

SNMP trap configuration


RecoverPoint supports the default MIB-II. The RecoverPoint MIB can be downloaded from
powerlink.emc.com at the following location:
Home > Support > Software Downloads and Licensing > Downloads PR > RecoverPoint
The application MIB OID is:
1.3.6.1.4.1.21658
The trap identifiers for RecoverPoint traps are as follows:
l

Info

Warning

Error

The RecoverPoint SNMP trap variables and their possible values are listed in the
following table.
Table 55 RecoverPoint SNMP trap variables

222

Variable

OID

Description

dateAndTime

3.1.1.1

Date and time that trap was sent

eventID

3.1.1.2

Unique event identifier; the values are listed in the Events chapter
of this Administrators Guide.

siteName

3.1.1.3

Name of site where event occurred

eventLevel

3.1.1.4

Valid values are:

RecoverPoint 5.0 Administrator's Guide

info

warning

warning off

error

Admin

Table 55 RecoverPoint SNMP trap variables (continued)

Variable

OID

Description
l

eventTopic

3.1.1.5

error off

Valid values are:


l

site

K-Box

group

splitter

management

hostName

3.1.1.6

Name of host

kboxName

3.1.1.7

Name of K Box

volumeName

3.1.1.8

Name of volume

groupName

3.1.1.9

Name of group

eventSummary

3.1.1.10 Short description of event

eventDescription 3.1.1.11 More detailed description of event

Note

OpenManage Server Administrator (OMSA) support is available only for systems in which
the RPAs are running on Dell PowerEdge platforms.
OMSA support provides RecoverPoint customers with the ability to:
l

Display Dell hardware event notifications, together with RecoverPoint event


notifications, in real-time, through the same management console.

Collect System Information on page 209 that includes Dell hardware configuration
information.

RecoverPoint generates events that result in Simple Network Management Protocol


(SNMP) traps. When an event with predefined characteristics (defined in RecoverPoint
and OMSA MIBs) occurs on your system, the SNMP subagent sends information about the
event, along with trap variables, to a specified management console.
Use the following procedure to view OMSA and RecoverPoint events in real-time.
Procedure
1. Configure SNMP event notification on page 220 to send event traps to your event
management console. Ensure that the Enable SNMP Agent and Send event traps
checkboxes are checked and that the IP address of the machine that has been
dedicated as the event management console is defined as a Trap destination.
2. Install a MIB browser on the machine dedicated as the event management console.
Note

MIB browsers are used to manage SNMP-enabled devices and applications in a


network. MIB browsers enable users to load MIBs, issue SNMP requests, and receive
SNMP traps.
Manage Event Filters

223

Admin

3. Open the MIB Browser on your management console, and


a. Enter the cluster management IP of the RPA cluster into the address bar. The
cluster management IP is a virtual IP address assigned to the RPA that is currently
active in the cluster. In the event that this RPA fails, the cluster management IP
address dynamically switches to the RPA that assumes operation.
b. Enable the MIB browsers Trap Receiver. In the Trap Receiver, both the
RecoverPoint and OMSA events are displayed in real-time. RecoverPoint events are
preceded with their severity level (Info, Warning, or Error), the Dell OMSA event
OIDs have the prefix 1.3.6.1.4.1.674.
After you finish
See the Dell OpenManage Server Administrator Version 1.9 SNMP Reference Guide,
which can be found at the Dell website:
http://support.dell.com/support/edocs/software/svradmin/1.9.2/en/SNMP/

Modify an existing event filter


Procedure
1. Select Admin > System Notifications > Manage Event Filters.

2. Select an event filter in the table.


3. Click Edit. The Event Filter wizard is displayed, with the current settings for the
selected event filter.

224

RecoverPoint 5.0 Administrator's Guide

Admin

4. Modify the event filter settings according to the instructions in Create an event
filter on page 216.
5. Click Next Define Method of Transfer.
6. If necessary, in the Define Method of Transfer screen, modify the methods of transfer.
For each new method of transfer that you define, Configure the transfer method on
page 218.
7. Click Finish to save the new event filter settings and exit the wizard.

Remove an existing event filter


Procedure
1. Select Admin > System Notifications > Manage Event > Filters.

2. Select an event filter in the table.


3. Click Remove, and then, click Yes in the confirmation dialog box.

Manage Event Filters

225

Admin

Configure System Reports and Alerts


Use Admin > System Notifications > Configure System Reports and Alerts to modify the
system report and alert settings, define or edit a transfer method, and test your settings
to ensure that they work. You can also use this screen to configure notification of new
RecoverPoint software releases.

Depending on the severity of the event, system alerts will automatically open a service
request with EMC Customer Support and will attach the latest system report to the service
request. System alerts are sent in real-time (at the time that the event occurs), allowing
EMC to provide preemptive support for RecoverPoint issues.
System reports contain the configuration and state of the RecoverPoint system, and they
are sent to the EMC system report (SyR) database every Sunday. Only one system report
is sent at a time (containing the information of all RPA clusters) unless there are
connectivity issues between RPA clusters, or an RPA cluster in the RecoverPoint system is
unable to send a system report. In this case, the problematic RPA cluster will send its own
system report. All system reports contain the name of the RPA cluster that they are sent
from.
The following parameters are sent to the EMC system report (SyR) database and included
in any opened service requests:
Table 56 System parameters sent to SyR

Parameter

Description

System ID

The unique identifier of all activated RPA clusters in a RecoverPoint system. The
System ID:
l

Is generated for the first RPA cluster that is added to the system during
RecoverPoint deployment. When adding additional RPA clusters to the
system, the additional RPA clusters receive the System ID of the first RPA
cluster.

Does not change with upgrades. When upgrading to RecoverPoint 4.0 from
previous RecoverPoint versions, the System ID value is taken from the value
of Account ID in the previous RecoverPoint version.

Can be displayed by running the get_account_settings CLI command,


and can only be modified by EMC Customer Support.

Also known as: Account ID

SSID

226

RecoverPoint 5.0 Administrator's Guide

The Software Serial ID. The unique identifier of an RPA cluster in the RecoverPoint
system. The SSID is the identification used by the install base to support EMC
equipment installed at customer sites, through the system reporting and ESRS
mechanisms.

Admin

Table 56 System parameters sent to SyR (continued)

Parameter

Description
Note

The format and function of the SSID has changed from previous RecoverPoint
versions.
The SSID is in the format <SN>EID<RPA cluster ID>, where <SN> and <RPA cluster ID>
are described in the licensing section of the EMC RecoverPoint Product Guide. Its
length can be no more than 22 characters, and therefore, the RPA cluster ID will
be truncated as required. The SSID:

Site ID

Is generated for each RPA cluster when the first license related to the RPA
cluster is added to the system. When additional licenses are added to the
system, the SSID of the RPA cluster does not change.

If no license was added for an RPA cluster, the SSID of that RPA cluster will
be empty.

Can be displayed by running the get_system_settings CLI command, and can


only be modified by EMC Customer Support.

Does not change with upgrades. When upgrading to RecoverPoint 4.0 from
previous RecoverPoint versions, the SSID value is taken from the value of the
SSIDs in the previous RecoverPoint version.

Does not change even if the SSID SN from which it was generated no longer
exists in any of the installed licenses.

The ID used in various EMC global processes as a unique identifier of the


physical location of an RPA cluster.
The Site ID:
l

Is obtained from the value of the <Site_Info> parameter in the license file
variables. Its value is set by the first installed license for an RPA cluster.
Note

If no license was installed for an RPA cluster, the Site ID of that RPA cluster
will be empty.
l

Can be displayed by running the get_system_settings CLI command,


and can only be modified by EMC Customer Support.

Does not change even if the <Site_Info> from which it was generated no longer
exists in any of the installed licenses.

Configure system reports and alerts


Before you begin
l

Best practice is to keep both system reports and alerts, and compression and
encryption enabled.

System reports and alerts will only be sent provided a method of transfer (SMTP,
ESRS, or FTPS) is configured. You can configure the method of transfer using the
Getting Started Wizard or by selecting Admin > System Notifications > Configure
System Reports and Alerts.
Configure System Reports and Alerts

227

Admin

To transfer system reports and alerts using SMTP or ESRS, ensure that port 25 is open
and available for SMTP traffic.

To transfer system reports and alerts using FTPS, ensure that ports 990 and 989 are
open and available for FTPS traffic.

To send system reports and alerts, a permanent RecoverPoint license must exist in
the system; system reports and alerts will not work with a temporary license.

By default:
l

Both system reports and system alerts are enabled.

System reports and alerts are sent through SMTP, but they can be configured for
transfer through a customer's ESRS server, or RecoverPoints built-in FTPS server.

System reports and alerts are compressed and encrypted with RSA encryption using a
256-bit key before they are sent.

Procedure
1. Select Admin > System Notifications > Configure System Reports and Alerts.

2. Ensure that the Enable System Reports and Alerts checkbox is selected, and the radio
button is set to System reports and system alerts .
3. Ensure that the Compression and Encryption checkboxes are selected.
4. Define the transfer method.
l

To transfer system notifications through an SMTP server:


a. In the Transfer Method section, ensure the SMTP radio button is selected.
b. In the SMTP server address field, specify the IP address or DNS name of your
dedicated SMTP server, in IPv4 or IPv6 format.
c. In the Sender email field, specify the email address from which the system
notifications will be sent.
d. Click Apply.

To transfer system notifications through RecoverPoints built-in FTPS server:


a. In the Transfer Method section, select the FTPS radio button.
b. Click Apply.

To transfer system notifications through an ESRS gateway:


a. In the Transfer Method section, select the ESRS radio button.
b. In the ESRS gateway IP address field, specify the IP address of your ESRS
gateway in IPv4 format.

228

RecoverPoint 5.0 Administrator's Guide

Admin

c. Click Apply.

Configure notification of new RecoverPoint software releases


This feature is enabled by default.
Procedure
1. Select Admin > System Notifications > Configure System Reports and Alerts.

2. Select or clear the Enable New Release Notification checkbox.

Ensure that system reports are configured correctly


Use the following procedure to ensure that system reports can be sent using the
configured transfer method.
Procedure
1. Configure system reports on page 227
2. Click Test Connectivity
3. In the Test Connectivity dialog box, enter the email address to which EMC Customer
Support can send a verification email.

Note

The connectivity test, if successful, opens a service request containing call home
event number 30999 and sends an email to the specified verification email address
from EMC Customer Support to verify that the system reports mechanism (SyR) has
been successfully configured.
4. Wait ten minutes.
5. Select Dashboard > System Events and look for event 1020 Failed to send system
report.
Configure System Reports and Alerts

229

Admin

If this event does not appear in your event logs, the system notifications
mechanism has been properly configured.
If you do receive event 1020; check whether there is an issue with your selected
method of transfer. If a problem exists, fix it, configure system reports on page 227
and click Test Connectivity again. If problem persists contact EMC Customer
Support.

Display the current system report


To view the current system report information, run the get_system_report CLI
command.

Send the current system report to an email address


To send the current system report information to a specified e-mail address, run the
get_system_report CLI command.

Display the current system report settings


To display the current system notification settings, run the
get_system_report_settings CLI command.

Manage Call Home Events


Use Admin > System Notifications > Manage Call Home Events to view and suppress the
events upon which a service request is automatically opened with EMC Customer
Support.

RecoverPoints call home feature is a proactive online service capability of RPAs that
enables RPAs to continuously monitor their own health, and the health of the
RecoverPoint system, using a pre-defined set of event-filtering rules. The call home event
mechanism automatically notifies EMC Customer Support of potential issues, using
secure transmissions. If a serious problem arises, the call home mechanism
automatically detects it and opens a service request with EMC Customer Support. When
the service request is received, EMC Customer Support pro-actively engages the relevant
EMC personnel to start working on the issues resolve, and immediately start working with
the relevant customer parties, or uses a configured ESRS gateway, to resolve the issue.
The call home event mechanism is customizable so that any specific environment or
specific customer requirement can be accommodated. EMC Customer Support have the
ability to create additional call home events, as required, to meet a specific customer's
needs. They can also permanently disable or temporarily suppress call home events. For
example, if an RPA cluster keeps reporting a problem that EMC Customer Support are
already aware of and have scheduled a customer visit to fix next week, the event
reporting the issue can be suppressed for a period of 2 weeks.

How call home events work


The call home events mechanism works in the following way, whenever the logic in the
call home event script is fulfilled:
230

RecoverPoint 5.0 Administrator's Guide

Admin

1. RecoverPoint event number 8000 "Calling home" is displayed in the event log,
and in parallel; a call home event is sent to SyR.
2. SyR opens an SR (Service Request) with EMC Customer Support, and relays the
following information:
l

The ID of the call home event is sent as the symptom code.

The event's Severity.

A description of the event scripts logic.

The email address and telephone number defined in the contact_info


parameter of the config_user_account CLI command.
Note

If contact information is not specified, and a call home event is triggered, a


Service Request is still opened, but it will not contain this information, which will
elongate the cases time to resolve.
3. EMC Customer Support engages the customer (or uses an ESRS Gateway) to proactively resolve the issue.
Note

Whenever the logic of a call home event script cannot be fulfilled, RecoverPoint event
8001 is generated in the event logs.

Before you begin


l

RecoverPoint ships with a set of pre-defined call home events that are enabled by
default.

The call home events mechanism is not available when the RPA running the Cluster
control is down.

The call home events mechanism is disabled during upgrade and while RecoverPoint
is in maintenance mode. The call home events mechanism discards any events
issued while the system is in maintenance mode.

The call home events, and the last call home time for each event, will survive RPA
failure. In the case of a disaster, all of the user-defined call home events, and any call
home events that may have been logged up until the time of the disaster, are
retrievable. This information will not, however, survive repository volume format.

When upgrading to a new RecoverPoint version that introduces no change in the


system-defined call home events, the existing system-defined rules are preserved.
When upgrading to a new RecoverPoint version that does introduce a change in the
system-defined call home events, the new system-defined events will override the old
ones.

The call home event mechanism is based upon with the most current system settings.
Therefore, if one of the rules in a call home event references a system property (e.g.
replication type), the value of the property must be the current value of the property
in the system settings. The call home mechanism does not save previous settings. If a
rule refers to a system entity (e.g. group, splitter, etc), and the entity has been
deleted from the system (and therefore, the system settings), no call home event is
issued.

The call home events mechanism automatically includes the contact information
defined in the contact_info parameter of the config_user_account CLI
command in the Service Request which is opened with EMC Customer Support. In
Manage Call Home Events

231

Admin

order to ensure a swift turnaround time, ensure that the contact_info parameter
contains both the email address and the telephone number of the person at your
company that is designated to mitigate RecoverPoint support situations. If contact
information is not specified and a call home event is triggered, a Service Request is
still opened, but it will not contain this information, which will elongate the cases
time to resolve.

Display all call home events


To display all of the call home events in the system, select Admin > System Notifications >
Manage Call Home Events and select an event in the table to display the call home event
script, description and status in the bottom pane.

Suppress one call home event until a specific date and time
Use the following procedure to suppress one call home event for a specific period of time.
Procedure
1. Select Admin > System Notifications > Manage Call Home Events
2. Select the event.
3. In the Status area of the bottom pane, select the Suppress Until radio button, and set
the date and time fields.
4. Click Apply.
NOTICE

Note the change in the event status in the Status column of the event table. To enable
suppressed events, select the event in the table, change the date and time fields to
the current date and time, and click Apply.

Suppress all call home events until a specific date and time
Use the following procedure to suppress all call home events for a specific period of time.
232

RecoverPoint 5.0 Administrator's Guide

Admin

Procedure
1. Select Admin > System Notifications > Manage Call Home Events
2. Select the event.
3. Click the Suppress All button under the event list. The Suppress All dialog box is
displayed.

4. In the Suppress All dialog box, set the date and time fields, and click OK.
NOTICE

Note the change in the status of all of the events in the Status column of the event
table. To enable all of the suppressed events, click Suppress All, set the date and
time fields to the current date and time, and click OK.

Modify call home events


To modify call home events, contact EMC Customer Support.

Manage Call Home Events

233

Admin

234

RecoverPoint 5.0 Administrator's Guide

CHAPTER 7
Support

This section describes all of the RecoverPoint support options and how to quickly access
your Recoverpoint documentation, product support, or online community of users from
within Unisphere for RecoverPoint.
l

RecoverPoint Support..........................................................................................236

Support

235

Support

RecoverPoint Support
Unisphere for RecoverPoint product support is accessed through the Support menu.

RecoverPoint Documentation
Click Support > RecoverPoint Documentation to display the RecoverPoint documentation
on EMC Online Support.

When clicked, the RecoverPoint documentation page is displayed, enabling you to search
for the RecoverPoint documentation that you require by RecoverPoint version, content
type, support task or OS.

236

RecoverPoint 5.0 Administrator's Guide

Support

EMC Online Support


Click Group > EMC Online Support to download EMC products, manage licenses, open
service requests, chat with EMC technical support, or contact EMC.

When clicked, the EMC Online Support page is displayed, enabling you to:

Register your product to access software downloads or obtain and manage license
keys.

Track, view, manage and create service requests for yourself and your company.

Engage with EMC Technical Support for answers to your questions.

View and manage your company information - Sites, products, contacts and
contracts.

Manage site preferences to guide your service center experience.

Download the new Mobile App to manage and track your Service Requests anytime,
anywhere.

Contact EMC

Learn about EMC's Maintenance, Warranty Information and Parts Return Site.

Learn how to fully leverage the capabilities of the new EMC experience.

Gain one-stop access to technical and support-related information by EMC product.

Download the latest EMC products and patches.

Connect with EMC customers and technical experts in this worldwide community of
your peers.
EMC Online Support

237

Support

Submit your ideas for improving serviceability on EMC products. SER is transparent
data repository where all requests can be submitted, managed and tracked.

The RecoverPoint Community


Click Support > The RecoverPoint Community to interact with other RecoverPoint users
and share best practices and tips, or communicate directly with the experts responsible
for the EMC RecoverPoint family.

When clicked, the RecoverPoint community page is displayed, enabling you to join the
community, start a discussion or view feeds.

238

RecoverPoint 5.0 Administrator's Guide

CHAPTER 8
Troubleshooting

This section describes occurrences in RecoverPoint, how to identify them, and how to
mitigate them from a users perspective.
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l
l

Licensing RecoverPoint....................................................................................... 240


Notification of new RecoverPoint software releases............................................. 250
Set the time display of Unisphere for RecoverPoint..............................................252
Change the display language of Unisphere for RecoverPoint................................253
First-time initialization is taking too long............................................................. 253
Protecting your data during initialization............................................................. 255
Host applications are hanging.............................................................................256
One-phase distribution workarounds.................................................................. 256
Copy is being regulated.......................................................................................257
Copy is experiencing a high load......................................................................... 258
RPA keeps rebooting........................................................................................... 260
Host applications crash during initialization........................................................261
Add more LUNs than allowed per VNX storage group........................................... 264
Swap the LUN numbers of journal volumes..........................................................265
Find your array serial numbers.............................................................................266
Find your RPA cluster IDs..................................................................................... 267
Find your XtremIO cluster IDs...............................................................................267
Understanding the RecoverPoint license............................................................. 268
Calculate your required journal size.....................................................................271
Validate replication integrity............................................................................... 278
MetroPoint group conversion.............................................................................. 282
Recovering from a cluster disaster.......................................................................282

Troubleshooting

239

Troubleshooting

Licensing RecoverPoint
This section describes how to license new RecoverPoint purchases, and upgrade your
RecoverPoint license.

Before you begin


l

All RecoverPoint licenses are per array, an XtremIO cluster or an RPA cluster.
Licensing is enforced using the Locking ID.

You cannot add a volume that resides on an unlicensed array to a replication set.
Only volumes that reside on licensed arrays are displayed in the volume list. This
limitation does not exist for journal volumes or repository volumes.

If a license Locking ID contains an array serial number that is not recognized by


RecoverPoint, you will be able to install the license in RecoverPoint but it will not be
tied to an RPA cluster and the license wont be activated until the array is mapped and
zoned to an RPA cluster.

For consistency groups containing both a local copy and a remote copy: If you select
to fail over to the remote copy and the remote RPA cluster does not have enough
licensed replication capacity for two times the capacity of the remote copy, the
failover will fail and you will be prompted to remove the local copy for the failover to
succeed.

When trying to add or modify a group, copy or replication set, the licenses of all
relevant arrays and RPA clusters are checked and the action will fail if it causes any of
the arrays or RPA clusters in the system to exceed their maximum licensed replication
capacity.

In RecoverPoint/CL:
n

If a license Locking ID contains an RPA Cluster ID that is not recognized by


RecoverPoint, you will not be able to install the license.

RecoverPoint/CL licenses cannot be added if a RecoverPoint/SE license already


exists.

RecoverPoint/CL licenses cannot coexist on the same RPA cluster with


RecoverPoint/EX licenses. If both are added to the system, the RecoverPoint/EX
license is disabled until the RecoverPoint/CL license is removed.

When RecoverPoint/SE is installed, only RecoverPoint/SE licenses can be added.

RecoverPoint/EX licenses cannot:


n

Be added if a RecoverPoint/SE license already exists.

Coexist on the same RPA cluster with RecoverPoint/CL licenses. If both are added
to the system, the RecoverPoint/EX license is disabled until the RecoverPoint/CL
license is removed.

License RecoverPoint
RecoverPoint license activation is performed through EMC Online Support at http://
support.emc.com.
1. Immediately upon submission of your sales order, a LAC email is sent to the address
provided during order entry, containing a new License Authorization Code (LAC) for all
of your purchased entitlements. A sales order can contain multiple LACs. In this case,
a separate LAC email is sent for each LAC contained in the sales order.
240

RecoverPoint 5.0 Administrator's Guide

Troubleshooting

2. Click the Click here link in the LAC email to automatically access the RecoverPoint
licensing area of the EMC Online Support site and display all of the entitlements
associated with your LAC.
3. In the RecoverPoint licensing area of the EMC Online Support site, activate and
generate a License File for each entitlement. You will generate a separate license for
each RecoverPoint license model associated with each unique array or RPA cluster.
4. Install your license files on the RecoverPoint system through Unisphere for
RecoverPoint.

What you will need


What you will need before you start licensing RecoverPoint.
l

Your LAC email(s) (or sales order numbers) to perform Step 1: Quickly access your
entitlements on page 242.

Access to EMC Online Support to perform Step 2 - Activate your entitlements on page
244.

Unisphere for RecoverPoint to perform Step 3 - Install your license file(s) in


RecoverPoint and enable support on page 247.
Note

If you do not have Unisphere for RecoverPoint installed, see the EMC RecoverPoint
Installation and Deployment Guide for installation instructions, and install
RecoverPoint before starting this procedure.

Search for entitlements associated with a sales order


Procedure
1. Access and log into EMC Online support at http://support.emc.com
l

Existing EMC Online Support users; enter your EMC Support username and
password.

New EMC Online Support users; click Register Now and complete the registration
process.

2. Select Service Center > Get and Manage Licenses > R > RecoverPoint Family.
3. Click the Activate link in the RecoverPoint version 4.0 and later section to access the
EMC licensing portal.

License RecoverPoint

241

Troubleshooting

4. In the LAC Request screen, click Manage Entitlements to display the Search
Entitlements screen:

5. In the Search Entitlements screen:

a. Enter a sales order number in the Sales Order # field.


b. Select Available in the Status field.
c. Click Search Entitlements to display all unactivated entitlements associated with
the entered sales order.
6. Go on to Step 2 - Activate your entitlements on page 244.

Step 1: Quickly access your entitlements


You will need to create a new RecoverPoint purchase order if you are a new RecoverPoint
customer, or if you want to:

242

(In XtremIO) Upgrade a basic license to a full-featured license.

Add replication capacity in RecoverPoint/CL.

RecoverPoint 5.0 Administrator's Guide

Troubleshooting

Add registered capacity in RecoverPoint/EX.

Upgrade from RecoverPoint/SE to RecoverPoint/EX.

Upgrade from RecoverPoint/EX to RecoverPoint/CL.

Add storage arrays in RecoverPoint/EX.

Add RPAs to your RecoverPoint system.

Add an RPA cluster to your RecoverPoint system.

Add raw capacity (for RecoverPoint/EX customers with VMAX 10K arrays with
registered capacity).

Add back-end array replication (for RecoverPoint/EX customers with VPLEX arrays).

Dynamically change capacity allocations on RPA clusters (for ELA customers).

In all of these cases, you will need to make a purchase with EMC sales, and obtain a
sales order. As soon as your sales order is approved, a LAC email is automatically sent to
the email addresses provided during order entry containing the LAC (License Authorization
Code) issued to you, validating your purchase and your entitlement(s).

A unique LAC is usually generated and emailed to you for each purchase, but this is not
always the case for evaluation licenses or purchases of certain products. Also, in some
cases (such as the addition of multiple arrays in RecoverPoint/EX), multiple LACs will be
generated and multiple LAC emails will be emailed to you.
To start the licensing process:
l

If you have your LAC email(s), for each LAC, click the Click here link in the LAC email
to automatically access the Powerlink licensing area of EMC Online Support, and
display all entitlements associated with the LAC. Then, activate your entitlements on
page 244.

License RecoverPoint

243

Troubleshooting

Note

If you do not have your LAC email, search your email inbox for an email containing
EMC License Authorization in the subject line. If you still cannot find your LAC email(s),
contact EMC Worldwide Licensing Support at: https://powerlinklicensing.emc.com/
poeticWeb/session/license_request_emc.jsp
l

If you have your sales order number(s), search for entitlements associated with a
sales order on page 241.

Step 2 - Activate your entitlements


Before you begin
l

Licensing is enforced using the Locking ID.

In the licensing section of the EMC Online Support Web site:


l

Existing EMC Online Support users; enter your EMC Support username and password.

New EMC Online Support users; click Register Now and complete the registration
process.

After you have logged-into EMC Online Support, the Search Entitlements to Activate
screen is displayed. The Search Entitlements to Activate screen is the first screen of a
five screen wizard, that will guide you, step-by-step, through the process of turning your
LACs into license files (redeeming your entitlements).
Procedure
1. In the Search Entitlements to Activate screen:
The link in the LAC email pre-populates the LAC field and searches for all entitlements
associated with the specified LAC. All relevant entitlements are displayed.

a. Select ONE entitlement to activate. Each entitlement must be selected and


activated separately. There can be multiple RecoverPoint models listed under each
entitlement.
b. Click Start Activation Process. The Search Machines dialog box is displayed.
244

RecoverPoint 5.0 Administrator's Guide

Troubleshooting

c. In the Search Machines dialog box, click Add a Machine. The Add Machine dialog
box is displayed.
d. In the Add Machine dialog box, enter a new machine name, and click Save.
A machine name is like a folder, it is used to group items together logically. A
unique machine name must be specified for each entitlement. Machine names can
be whatever unique name you choose but it is best practice to relate the machine
name to the Locking ID. For example, in RecoverPoint/SE and RecoverPoint/EX, the
Locking ID is an array serial number. Therefore, the machine name should be RPAPM00120701240 if APM00120701240 is the array serial number used for the
Locking ID. In RecoverPoint/CL, the Locking ID is the RPA Cluster ID, so the
machine name should be the RPA Cluster ID.

2. In the Register screen, verify the machine name, and click Next.
3. In the Activate screen, enter the Locking ID, and click Next.

The Locking ID is the field that is displayed under the Machine Information column;
its value is whichever entity the license is enforced to.
Depending on the entitlement, the Locking ID can be either an RPA Cluster ID (in
RecoverPoint/CL), or an array serial number (in RecoverPoint/SE and RecoverPoint/
License RecoverPoint

245

Troubleshooting

EX). To display instructions for finding your RPA Cluster IDs and array serial numbers,
either click the Locking ID help link or see Find your RPA cluster IDs on page 267, Find
your XtremIO cluster IDs on page 267 and Find your array serial numbers on page
266.
4. In the Confirm screen, enter the email addresses of the recipients of the license file in
the Email to field of the Additional Email Options section, and click Finish.
By default, the customer contact provided during order entry and the user performing
the entitlement activation are automatically emailed an activation certificate
containing the license file as an attachment. You can enter multiple email addresses
in the Email to field. Separate multiple email addresses with commas. You can also
include optional comments which will be displayed in the certificate e-mail in the
Email comments field, or add optional notes which can only be viewed online in
Powerlink Licensing in the Notes for this transaction section.

5. In the Complete screen, click Save to File to download the license file and save the file
locally. The resulting license file has a *.lic extension and is in plain text format
(can be opened in any text editor).

6. Redo this procedure from Step 1: Quickly access your entitlements on page 242 for
each entitlement in each LAC email or sales order.

246

RecoverPoint 5.0 Administrator's Guide

Troubleshooting

7. After you have turned all of your entitlements into license files, physically transfer the
license file(s) to the computer from which you will be running Unisphere for
RecoverPoint, so you can install them in the RecoverPoint system.

Step 3 - Install your license file(s) in RecoverPoint and enable support


Before you begin
l

Multiple warnings will be issued before an expiring license is about to expire. After
license expiration, replication continues but no further operations will be allowed on
the RecoverPoint system.

The license limitations and their current values (for example, capacity, expiration) for
each licensed RPA cluster and storage array is displayed in the Dashboard > System
Limits pane.

Adding a new permanent or expiring license with the same Locking ID as an existing
expiring license replaces the existing expiring license.

After you have your license files, launch Unisphere for RecoverPoint as described in the
EMC RecoverPoint Installation and Deployment Guide.
When Unisphere for RecoverPoint loads for the first time The Getting Started wizard on
page 15 is displayed. The Getting Started Wizard will guide you, step-by-step, through the
process of installing your license file(s) in RecoverPoint and enabling RecoverPoint
support. If The Getting Started wizard on page 15 is not displayed, install your license
files using Manage Licenses on page 178 and Configure System Reports and Alerts on
page 226.

Upgrade from RecoverPoint 4.0 or later


When upgrading from RecoverPoint 4.0 or later, all existing permanent licenses are
automatically carried over and no further licensing is required.

Upgrade from RecoverPoint versions prior to 4.0


Use the following procedure to upgrade your RecoverPoint software license from
RecoverPoint 3.5.x to this version of RecoverPoint.
Before you begin
Note

When upgrading from RecoverPoint versions prior to 4.0, previous licenses are
automatically converted to the new license format. The new licenses are temporary and
valid for 90 days from the date of upgrade. After that, you must have installed one or
more permanent license files to replicate with RecoverPoint appliances.
Procedure
1. Open a Service Request with EMC Customer Support:
You can open the Service Request yourself or have an EMC representative open the
Service Request for you.
To have an EMC representative open the Service Request for you:
l

Call 24/7 License Key Live Support for North America, EMEA and APJK by dialing
1-800-782-4362, selecting option 4, and selecting option 4 again.

Go to http://support.emc.com and click the Live Chat button.


To open the Service Request yourself:
Upgrade from RecoverPoint 4.0 or later

247

Troubleshooting

a. Go to http://support.emc.com
l

When upgrading to RecoverPoint 4.0, previous licenses will automatically be


converted to the new license format. The new licenses are temporary and valid
for 90 days from the date of upgrade. After that, you must have installed one or
more permanent license files to replicate with RecoverPoint appliances.

Existing EMC Online Support users; when prompted, enter your EMC Support
username and password.

New EMC Online Support users; click Register Now and complete the
registration process.

b. In EMC Online Support, click Service Center in the main toolbar, and then click
Create a Service Request.
c. Fill out the fields in the service request according to the instructions in the
following table:
Field

Value

Service
Select Technical Problem/Question from the drop-down.
Request Type
Company and Product Information
Search by
Site ID

This is the geographical location of the data center for which you are
requesting the license. If you know the site ID, enter it in this field.
If you don't know the site ID:
Expand the Advanced Site Search.
Enter your company name and details.
Click Search.
Select the relevant site ID in the Search Results table.
Note

If you don't see your companys site ID in the list, contact EMC Customer
Support.
Product
Details

Select the Choose from all EMC Products radio button.


Start typing the product name into the field to bring up all of the relevant
EMC products.
Select the EMC product for which you are requesting an upgrade from the
search results.

Version
Number

Select the version number of the EMC product that you want to upgrade
from the drop-down.

Contact Information
Initial
Contact is
Best Made
By

248

RecoverPoint 5.0 Administrator's Guide

Select Email.
Note

If you select Online, all communication regarding your RecoverPoint


license key will be through the SR tracking system on EMC Online Support.

Troubleshooting

Field

Value

Primary
Contact
Name

Your license will be emailed to the email address specified here.


Select the name of the primary contact person at your company for
RecoverPoint management.
Note

If the email address/es listed under the primary contact name are incorrect
or missing, edit them to update the contact information.
Phone

Enter the phone number as an alternative contact method.


Note

If the phone numbers listed under the primary contact name are incorrect
or missing, edit them to update the contact information.
Problem Details
Problem
Severity

Enter a problem severity of 4

Problem
Summary

Type the following text into the problem summary field:


License Upgrade Request for SO#: <The Sales Order Number of Your Last
RecoverPoint Purchase>

Problem
Description

Enter the following:


l

Your current RecoverPoint System ID

All of the array serial numbers of the EMC arrays (VNX, VMAX, VPLEX,
etc.) that require RecoverPoint protection.

Your required replication types (local replication, remote replication)

Your required replication capacity

Your required configuration details (splitter type, splitter sharing,


Stretch CDP, etc.)

d. When youre done filling out the SR details, click the Submit button. An SR
Tracking number is displayed. Write this number down and save it for future
reference.
Note

In most cases, 48 hours after submitting your service request, your RecoverPoint
licenses will be emailed to the specified email address. If you have not received
your RecoverPoint licenses within a month from the time you submitted your
service request, contact EMC Customer Support.
2. Install your RecoverPoint license(s).
When Unisphere for RecoverPoint loads for the first time The Getting Started wizard on
page 15 is displayed. The Getting Started Wizard will guide you, step-by-step, through
the process of installing your license file(s) in RecoverPoint and enabling RecoverPoint
support. If The Getting Started wizard on page 15 is not displayed, install your license
files using Manage Licenses on page 178 and Configure System Reports and Alerts on
page 226.
Upgrade from RecoverPoint versions prior to 4.0

249

Troubleshooting

Get licensing support


Licensing support is available for product ordering, upgrades, license activation and
license installation, according to the following table.
Table 57 Licensing support

Process

Americas

EMEA

APJ

Ordering

1-866-EMC-7777> Option 2

+1-508-435-1000, ext. 54777 > Option 2

Presalessupportcenter@emc.com
Upgrades

1-508-249-5401

License Activation
or Installation

http://support.emc.com

US number: +1-508-249-5401

Service Center > Create Service Request


licensing@emc.com
1-800-782-4362 > Option 4 > 353 (0) 21 487-9862 >
Option 4
Option 1

353 (0) 21 487-9883

Notification of new RecoverPoint software releases


RecoverPoint software upgrades are described in relation to major RecoverPoint releases.
In RecoverPoint, major releases are labeled with an X.x (for example, RecoverPoint 4.1).
RecoverPoint upgrades are considered:
l

MajorIf you must move to another major release to upgrade them. For example,
RecoverPoint 3.5, 4.0 and 4.1 are all different major releases of RecoverPoint, and
upgrading from one major release to another requires a major upgrade.

MinorIf they are in the same major release. For example, upgrading to any of the
service packs or patches of RecoverPoint 4.1 are all minor upgrades.

The Upgrade Recommendation dialog box


Each time you log into Unisphere for RecoverPoint, if RecoverPoint detects that there is a
new recommended minor version or the target version is newer than the version of
RecoverPoint currently installed at the RPA cluster to which you have connected, the
Upgrade Recommendation dialog box is automatically displayed, containing the:

250

RecoverPoint version currently installed at the RPA cluster you are connected to.

RecoverPoint target versionThe target version is the most stable and current
RecoverPoint software code version across all major RecoverPoint releases. Best
practice is to ensure all RPA clusters in your RecoverPoint system are always running
the RecoverPoint target version.

RecoverPoint recommended minor versionThe recommended minor version is the


most current minor software version available for the major release of RecoverPoint
that you are running. If your environment is not qualified, or you do not wish to
upgrade to the target version, consider upgrading your RPA clusters to the
RecoverPoint recommended minor version. There is one recommended minor version
per major release of RecoverPoint.

RecoverPoint 5.0 Administrator's Guide

Troubleshooting

You can enable or disable the display of the Upgrade Recommendation dialog box using
the Admin > System Notifications > System Reports and Alerts screen, by clearing or
marking the Enable New Release Notification checkbox. It is enabled by default.
If you connected to RecoverPoint using the management (floating) IP address, note the
relevant cluster name in the Upgrade Recommendation dialog box, as the dialog box is
displayed per RPA cluster.

What should I do when I see the Upgrade Recommendation dialog box?


When the Upgrade Recommendation dialog box is displayed, you should start thinking
about upgrading your RecoverPoint system to the target version or recommended minor
version indicated in the pop-up.

Deciding whether or not you want to upgrade


To help you decide whether or not you are qualified for, or want to upgrade, use the:
l

EMC RecoverPoint Release NotesThe RecoverPoint release notes contain descriptions


of all of the new features and know bugs of each release. Use them to decide whether
or not the main features of the target or recommended release warrant an upgrade for
your company. For:
n

RecoverPoint/SEdownload the release notes from https://support.emc.com/


products/1249_RecoverPoint-SE/Documentation/

RecoverPoint/CLdownload the release notes from https://support.emc.com/


products/1226_RecoverPoint-CL/Documentation/

RecoverPoint/EXdownload the release notes from https://support.emc.com/


products/27704_RecoverPoint-EX/Documentation/

EMC Service CenterThe EMC Service Center at https://support.emc.com/


servicecenter/ is the one stop shop where you manage all of your support-related
activities and information. If you are using RecoverPoint/CL or RecoverPoint/EX, use
the Service Center to open a Service Request and start planning your RecoverPoint
upgrade. All RecoverPoint users can use the Service Center to:
n

Track, view, manage and create service requests.

Download the latest EMC products and patches.

Engage with EMC Technical Support for answers to your questions.

Contact EMC Technical support.

Access information for sites, products, contacts, contracts, and environments.

Download the EMC Support mobile application.

Learn about EMC's Maintenance, Warranty Information and Parts Return Site.

Connect with EMC customers and technical experts in a worldwide community.

Register your product to access software downloads, manage or obtain licenses.

EMC MatrixThe RecoverPoint EMC Simple Support Matrices (ESSMs) can be found
on EMCs E-lab Web site. E-Lab has tested and qualified millions of different data
center configurations and EMC solutions. To find out if your environment is qualified
for upgrade to the suggested target and recommended versions, download the
relevant RecoverPoint ESSM:
n

For upgrade to RecoverPoint 4.1.xDownload the ESSM from https://


elabnavigator.emc.com/vault/pdf/EMC_RecoverPoint_4.1.x_ESSM.pdf

What should I do when I see the Upgrade Recommendation dialog box?

251

Troubleshooting

For upgrade to RecoverPoint 4.0.xDownload the ESSM from https://


elabnavigator.emc.com/vault/pdf/EMC_RecoverPoint_4.0.x_ESSM.pdf

Upgrading your RecoverPoint environment


If you are upgrading your RecoverPoint system and are running:
l

RecoverPoint with Unity, VNXe, or VNX storage and RecoverPoint/SE licensesYour


RecoverPoint system is customer upgradeable.
1. Download the RecoverPoint Customer Upgrade Kit from the Recommended
Resources area in the upper right corner of: https://support.emc.com/products/
1249_RecoverPoint-SE
2. Unzip the kit.
3. From the kit, open the RecoverPointCustomer Upgrade Guide, and follow the
instructions to upgrade your system.
Any other RecoverPoint systemYour RecoverPoint system is not customer
upgradeable. EMC Customer Support will plan and execute the upgrade for you.
Contact your EMC representative or open a service request at https://
support.emc.com/servicecenter/

Set the time display of Unisphere for RecoverPoint


Use the following procedure from any screen in Unisphere for RecoverPoint to set the time
display.
Procedure
1. Click Time displayed on the Unisphere for RecoverPoint status bar.

2. In the Set Time Display dialog box, select the radio button of your preferred time
display option.

252

RecoverPoint 5.0 Administrator's Guide

Troubleshooting

NOTICE

Local time is the local time at the RPA that Unisphere for RecoverPoint is connected to.
Results
The time display option you choose will be used by RecoverPoint when performing
operations which require a time stamp such as collecting system information and logging
events.

Change the display language of Unisphere for RecoverPoint


Procedure
1. In Unisphere for RecoverPoint, select the Settings icon in the upper-right corner.

2. In the Preference Settings dialog box, select another display language.

First-time initialization is taking too long


You can save both time and bandwidth by initializing copy volumes from a backup, when
they are first added to the system.

How does it work


When a consistency group is initialized for the first time, the RecoverPoint system cannot
identify which blocks are identical between the production and copy volumes, and must
therefore mark all blocks for that volume. This is true both following the creation of a new
consistency group and following the addition of a volume (or volumes) to an existing
group. While initializing, the RecoverPoint system efficiently determines which blocks are
actually different between the production and copies, and sends only the data for those
blocks to the copy storage, as the initialization snapshot.
The volumes at the local and remote RPA clusters can be initialized while the host
applications are either running or not. Initialization of one consistency group does not
interfere with the operation of other consistency groups. Initialization can be carried out
automatically over IP or Fibre Channel.

Change the display language of Unisphere for RecoverPoint

253

Troubleshooting

Alternatively, you can back up the current production data, manually transfer it to the
remote cluster, and then copy the production image to the copy volumes, where it
becomes the pre-replication image.
The state of data transfer prior to any type of initialization is always paused.
NOTICE

First-time initialization can cause changes in the partition table of the copy volume or
volumes. In a Windows environment, you must clear the OS cache before changing the
partition table of a copy volume. To clear the OS cache, disable the LUN or LUNs on which
the volume resides, and re-enable it. You can do so from the Disk drives interface of the
Windows Device Manager.
You can initialize new consistency groups by creating a backup of your production
volumes, physically transferring them to the remote site, and copying the backup images
onto the remote storage volumes.
During this process, applications at the production site can be running or not. Although
transfer is paused during this process, unless the production host is completely shut
down during the creation and physical transfer of the production image to the remote
site, the host applications keep writing to storage. In this time, the production and copy
volumes become inconsistent, and any writes to production volumes during this process
are subsequently synchronized, upon completion of the process, and start of transfer.
Only synchronizing the changes made to the production volumes during this time results
in a relatively small amount of additional traffic, and takes substantially less time than to
synchronize over IP or Fibre Channel.

Initialize from backup


Use the following procedure to initialize from a backup image, when the production and
copy volumes are not consistent.
Procedure
1. At the production RPA cluster Attach volumes to the splitter on page 164 and ensure
that all splitters that have access to replication volumes in the group are attached to
those volumes.
2. Create a new consistency group, and define replication and journal volumes using
Protect Volumes on page 37 or Policy Templates on page 52.
3. Pause transfer for a group on page 110 for the group, to stop the replication of data
from production to the copy.
4. In the RecoverPoint CLI, run the clear_markers command to inform the system
that the copy volume at the remote RPA cluster is known to be identical to its
corresponding production volume, and a full volume sweep synchronization is not
required.

254

RecoverPoint 5.0 Administrator's Guide

Troubleshooting

NOTICE

For the selected consistency groups, clearing markers clears all markers of the
selected copy from the production journal volume; that is, it treats the selected copy
as identical to the production source. Caution: To clear markers, the production
source and the selected copy volume must be absolutely identical. If they are not, the
inconsistencies will remain. The clear markers command should be used only with
extreme caution. It is useful when a production source and a copy have been
synchronized manually by initializing from backup and adequate bandwidth is not
available to synchronize using the storage network. Otherwise, the best practice is not
to use this command.
5. Create a block-based backup of the production volumes.
6. Physically transfer the backup to the remote cluster.
7. At the remote RPA cluster, in Unisphere for RecoverPoint:
a. Test a Copy on page 128 to enable image access to the image at the remote RPA
cluster in Logged Accessmode, and enable direct access. Do not select the prereplication image.
If your consistency group contains more than one non-production copy, for all
other copies, the journal is erased and all journal history is lost.
b. Click Finish to close the wizard without disabling image access.
8. Restore the backup onto the remote copy volumes. The backup image becomes the
pre-replication image.
9. In Unisphere for RecoverPoint:
a. Select Recovery > Manage Recovery.
b. Select Finish Testing from the Select Next Action combo box of the accessed copy
at the remote RPA cluster, to disable image access and start transfer to the copy.
Upon start of transfer, the system synchronizes the data of the copy volume with
the corresponding production data.
10. At the remote RPA cluster, if the group contains more than one non-production copy,
Start transfer to a copy on page 113 to all of the other copies.

Protecting your data during initialization


When replicating in any mode other than snap-based replication mode, by default, you
cannot recover production from a copy image or fail over to a copy image during
initialization. For a recovery activity to succeed, there must be a complete image in the
copy journal to recover the data from; therefore, recovery is not available during
initialization, while the snapshot being distributed is larger than the capacity of the copy
journal.
To enable recovery during initialization , set the value of the allow_long_resync
parameter of the config_copy_policy CLI command to no (default is yes). When
allow_long_resync is set to no, the system pauses transfer whenever the snapshot
being transferred is larger than the capacity of the journal, at which point you can Add or
edit journal volumes on page 118 to increase the journal size. Note that any additional
journal volumes should be permanent, as removing a journal volume causes all of the
snapshots in the journal to be deleted and requires a volume sweep.
Protecting your data during initialization

255

Troubleshooting

Host applications are hanging


Some RecoverPoint users set a policy that enables RecoverPoint to control the
acknowledgment of writes back to the host in the case of bottlenecks or insufficient
resources that would otherwise prevent RecoverPoint from replicating the data.
If your host applications experience delays, loss of client connectivity, or slow response
times, check whether the value of the regulate_application parameter in the
config_link_policy CLI command is set to yes (default is no).

When does application regulation happen?


Application regulation happens when a user enables the regulate_application
parameter in the config_link_policy CLI command.

How does application regulation work?


The system slows host applications when approaching the RPO policy limit. When the
system cannot replicate the current incoming write-rate while guaranteeing the RPO
setting, the system slows host applications to guarantee that the RPO is always enforced.
Additionally, if there is a bottleneck in the system, the system will regulate the host
applications instead of entering a High load state.
Link policies on page 65 includes more information about the RPO setting.

How do I know application regulation is happening?


If your host applications experience delays, loss of client connectivity, or slow response
times, check whether the value of the regulate_application parameter in the
config_link_policy CLI command is set to yes. If it is, your host applications are
being regulated to ensure your RPO.

What can I do to stop my group from being regulated?


To come out of this state, set the value of the regulate_application parameter in
the config_link_policy CLI command to no.
Note

Before disabling application regulation, make sure you are familiar with replication
modes and understand the implications of doing so.

One-phase distribution workarounds


To avoid long initializations, perform one of the following procedures in the RecoverPoint
Administrator's Guide:

256

Use the initialization from backup procedure First-time initialization is taking too
long on page 253.

Set the value of the allow_long_resync parameter of the


config_copy_policy CLI command to no(default is yes).

Add volumes to the copy journal. Note that the additional journal volumes must be
permanent or a full sweep will occur when the volumes are removed.

RecoverPoint 5.0 Administrator's Guide

Troubleshooting

Copy is being regulated


RecoverPoint includes a smart mechanism that protects the system from adverse affects
and over-consumption of system resources, when a system component is operating
improperly in the system. This mechanism is referred to as Control action regulation.
This section answers the questions:

When does control action regulation happen?


Control action regulation happens when a system component is operating improperly in
the system, and jittering (quickly changing) between two states for a set period of time.

How do I know control action regulation is happening?


You know the control action regulation mechanism has been enabled, when:
l

Event number 4133 (Copy regulation has started)is displayed in the event log.

In the consistency group Status tab of the GUI, the Role of a copy becomes Regulated
and is displayed in red.

How does control action regulation work?


When control action regulation happens, to allow the environment to stabilize, the
control action regulation mechanism places the copy in a Regulated state, in which the
system will protect itself by closing the link to the copy, limiting any adverse affects and
over-consumption of system resources. The copy stays in the state it was in before
regulation began for 30 minutes or until corrective action is taken.

How do I release a copy from control action regulation?


When control action regulation happens, you can:
l

Release all groups at all copies from this state by running the
unregulate_all_copies command from the CLI.

Check previous event logs. Look for repetitive errors that may indicate a specific
problem in the system.

Check SAN/IP events outside of RecoverPoint, as instabilities may not originate from
RecoverPoint.

If regulation persists, Collect System Information on page 209 and contact EMC
Customer Support for further instruction.

How do I verify that regulation is over?


The following indicators can help you verify that your copy is no longer being regulated:
l

Event 4132 (Copy regulation has ended due to a user action or internal timeout.) is
displayed in the event log.

The Role of a copy is no longer displayed in red in the consistency group Status tab.

Copy is being regulated

257

Troubleshooting

Copy is experiencing a high load


High load is a system state that indicates resource depletion during replication.
There are two kinds of high loads in RecoverPoint:
l

Permanent High-loads - In these cases, RecoverPoint stops and waits for a user action
in order to come out of high load.

Temporary High-loads - In these cases, RecoverPoint tries to recover from the high
load and will keep trying until the condition that triggered the high load changes.

How do I know a copy is experiencing a high load?


You know that a copy has entered a high load state, when:
l

Warning events are logged specifying that the copy is experiencing high load.

In Unisphere for RecoverPoint:


n

In the consistency group or copy Status tab, the transfer state of a copy becomes
High load.

In the Consistency Group Transfer State widget in the Dashboard, a state of Init
(orange) is displayed for the group.

What is a permanent high load?


A permanent high load is a system state that happens during replication, when the size
of the journal, or the queue of snapshots waiting for distribution of the journal at the
copy, is insufficient.

When do permanent high loads occur?


A permanent high load generally happens in one of two cases:
l

During copy testing in RecoverPoint products with a splitter, if a user accesses a copy
in logged or virtual access modes for a long time and the queue of snapshots waiting
for distribution of the journal reaches its maximum capacity, in this case the system
will Pause Transfer and wait for user input.

When the system is in initialization mode, and the journal volume has reached its
maximum capacity, while the value of the allow_long_resync parameter of the
config_copy_policy CLI command is set to no.

How do permanent high loads work?


When any of the events in When do permanent high loads occur? on page 258 occurs, the
system stops transfer and waits for user input.

How can I tell a copy is under permanent high load?


The following indicators are displayed when your copy is experiencing a permanent high
load:
l

Warning events are logged specifying that the copy is experiencing high load.

The transfer state is displayed as High load. You can display the transfer state:
n

258

In the RecoverPoint CLI, by running the get_group_states command.

RecoverPoint 5.0 Administrator's Guide

Troubleshooting

In Unisphere for RecoverPoint:


In the consistency group Status tab, by selecting Protection > Manage
Protection, selecting a group, and clicking the groups Status tab.
In the Consistency Groups Transfer Status widget of the Dashboard of the
Overall Health tab, groups undergoing permanent high load are labeled Init
(orange).

What can I do to come out of permanent high load?


To release a copy from a permanent high load:
l

During copy testing in RecoverPoint products with a splitter, if the user accessed a
copy in logged or virtual access mode for a long time and the queue of snapshots
waiting for distribution of the journal reached its maximum capacity, you should
disable image access or enable direct access according to image access modes.

If the system was in initialization mode, and the journal volume became full, while
the value of the allow_long_resync parameter of the config_copy_policy
CLI command is set to no.

How do I verify that a permanent high load is over?


The following indicators can help you verify that your copy is no longer experiencing a
permanent high load:
l

Warning events are logged specifying that the copy is no longer experiencing high
load.

The transfer state is no longer displayed as High load. You can display the transfer
state:
n

In the RecoverPoint CLI, by running the get_group_states command.

In Unisphere for RecoverPoint:


In the consistency group Status tab, by selecting Protection > Manage
Protection, selecting a group, and clicking the groups Status tab.
In the Consistency Groups Transfer Status widget of the Dashboard > Overall
Health tab.

What is a temporary high load?


A temporary high load is a system state that happens during replication, when the RPA
resources at the production site are insufficient.

When do temporary high loads occur?


Temporary high loads occur:
l

In extreme cases, during uncommonly great durations of uncommonly extreme write


loads

When the copy or journal volumes are not fast enough to handle distribution

If the WAN is too slow

If the compression level is too high

How do temporary high loads work?


When any of the events in When do temporary high loads occur? on page 259 occurs,
traffic is paused and started immediately. If resources are still low, the system waits five
What can I do to come out of permanent high load?

259

Troubleshooting

minutes and then tries to pause and start transfer again until the required resources are
available.
NOTICE

Upon every start of transfer, a short initialization occurs.

How can I tell a copy is under temporary high load?


The following indicators are displayed when your copy is experiencing a temporary high
load:
l

Warning events are logged specifying that the copy is experiencing high load.

The transfer state is displayed as high load, followed by a progress status. You can
display the transfer state:
n

In the RecoverPoint CLI, by running the get_group_states command.

In Unisphere for RecoverPoint:


In the consistency group Status tab, by selecting Protection > Manage
Protection, selecting a group, and clicking the groups Status tab.
In the Consistency Groups Transfer Status widget of the Dashboard of the
Overall Health tab, groups undergoing temporary high load are labeled Init
(orange).

What should I know about temporary high loads?


Temporary high loads are a common occurrence and expected to happen from time to
time.
If the high load lasts for an extreme period of time or occurs too frequently (and will
eventually impact the business RPO) contact EMC Customer Support for a mitigation
plan.

How do I verify that a temporary high load is over?


The following indicators can help you verify that your copy is no longer experiencing a
temporary high load:
l

Warning events are logged specifying that the copy is no longer experiencing high
load.

The transfer state is no longer displayed as High load. You can display the transfer
state:
n

In the RecoverPoint CLI, by running the get_group_states command.

In Unisphere for RecoverPoint:


In the consistency group Status tab, by selecting Protection > Manage
Protection, selecting a group, and clicking the groups Status tab.
In the Consistency Groups Transfer Status widget of the Dashboard > Overall
Health tab.

RPA keeps rebooting


Reboot regulation is a state of regulation that allows the system to detach an RPA from its
RPA cluster in the event of frequent unexplained reboots or internal failures.
260

RecoverPoint 5.0 Administrator's Guide

Troubleshooting

When does reboot regulation happen?


Reboot regulation happens when an RPA is frequently and unexpectedly rebooting, or
undergoing a repeated internal failure.

How does reboot regulation work?


When an RPA behaves according to When does reboot regulation happen? on page 261,
the system detaches the RPA from the RPA cluster.

How do I know reboot regulation is happening?


Reboot regulation is happening when the user receives the following message when
logging into the RPA as a boxmgmt user:

What should I do to stop reboot regulation?


To stop reboot regulation, contact EMC Customer Support for further instructions.

Host applications crash during initialization


During initialization RecoverPoint reads the data at the production storage in order to
synchronize the data of the copy volumes with the production volumes. If the production
storage cannot handle the read-rate it will hang, causing RecoverPoint and the
production host to timeout, and the user's applications to crash, lose access to the
storage, or experience I/O or path failures.
RecoverPoints I/O throttling mechanism enables users to limit the storage read-rate of
each array or vCenter Server seen by an RPA cluster or all arrays and vCenter Servers seen
by an RPA cluster, allowing the storage to handle the I/O rate during initialization.
When the I/O throttling mechanism is enabled (Default is disabled) RecoverPoint limits
the I/O rate that it sends to the storage according to the throttling level defined by the
user.

Storage read-rate during initialization


Use the following procedure to check your storage read-rate during initialization.
Procedure
1. Connect to the RecoverPoint CLI at the relevant site and run the
detect_bottlenecks command.
2. Note the relevant read-rates:
l

If you are connected to an RPA cluster at the production site, note the value of the
Speed of reading from local user volume during
initialization parameter.

If you are connected to a remote RPA cluster, note the value of the Speed of
reading from remote user volume during initialization
parameter.

When does reboot regulation happen?

261

Troubleshooting

How does I/O throttling work?


In RecoverPoint, I/O throttling works in the following way:
1. The user sets the I/O throttling level to limit the storage read-rate during initialization.
2. During initialization, I/Os are delayed, so they do not exceed the value of the
throttling level.
NOTICE

The higher the throttling level, the longer initialization will take.

When should I enable I/O throttling?


Enable I/O throttling if your host applications experience I/O or path failures, crash, or
lose access to the storage during initialization.

How do I enable I/O throttling?


Before you begin
The following limitations apply to the I/O throttling feature:
l

I/O throttling can only be configured using the RecoverPoint CLI or REST API. You
cannot configure I/O throttling using the Unisphere for RecoverPoint GUI.

I/O throttling can only be configured for arrays and vCenter Servers that have been
mapped to, and are therefore seen by, the RPA cluster.

When selecting to configure I/O throttling for all arrays or vCenter Servers seen by an
RPA cluster, the performance of all consistency groups residing on the same array or
vCenter Server is affected. For example, setting the throttling level to high limits the
I/O rate of all RPAs in an RPA cluster to a maximum read-rate of 50 MBps, per array or
vCenter Server.

The higher the throttling level, the longer initialization will take. Ensure that a low
throttling level is insufficient before setting the throttling level to high.

You must be logged into RecoverPoint as an admin user to configure I/O throttling.

To ensure efficiency, RecoverPoint sends I/Os to storage in parallel. The I/O throttling
mechanism regulates parallel I/Os in the following way:

When set to low, the I/O throttling mechanism allows for a maximum of 10 MB of
parallel I/Os per second.

When set to high, the I/O throttling mechanism allows for a maximum of 1 MB of
parallel I/Os per second.

When set to custom, the I/O throttling mechanism allows for a maximum of
<CustomValue>/20 of parallel I/Os. For example, if the custom value is 40 MBps,
the I/O throttling mechanism will allow for 40/20=2 MB of parallel I/Os per
second.

In previous RecoverPoint versions, I/O throttling was only available per RPA cluster.
Therefore:
n

262

When upgrading to RecoverPoint 4.1 from a previous RecoverPoint version, the


I/O throttling settings in the previous RecoverPoint version are applied per array
and vCenter Server that are seen by the RPA cluster at the time of the upgrade.
Any arrays or vCenter Servers mapped to the RPA cluster after upgrade will require
additional I/O throttling configuration.

RecoverPoint 5.0 Administrator's Guide

Troubleshooting

If your RecoverPoint system contains at least one RPA cluster running a


RecoverPoint version prior to RecoverPoint 4.1:
you will not be able to set the I/O throttling level per array or vCenter Server.
the I/O throttling settings of all RPA clusters will be the RecoverPoint 4.0 and
earlier version.
Note

See the RecoverPoint 4.0 version of the EMC RecoverPoint Administrators Guide
and the EMC RecoverPoint CLI Guide for the I/O throttling settings if at least one
of the RPA clusters in your RecoverPoint system is running a RecoverPoint
version prior to 4.1.
Use the following procedure to enable I/O throttling for a specific array or vCenter Server.
Procedure
1. Run the config_io_throttling command in the RecoverPoint CLI.
2. Select the RPA cluster to which the array or vCenter Server is mapped.
3. Select the array or vCenter Server from the list.
4. Set the throttling level: Set the throttling level to low to limit the I/O rate of the
specified array or vCenter Server to a maximum read-rate of 200 MBps.
l

Set the throttling level to high to limit the I/O rate of the specified array or vCenter
Server to a maximum read-rate of 50 MBps.

Set the throttling level to custom, and specify the custom value in MBps to limit
the I/O rate of the specified array or vCenter Server to any other read-rate.

Set the throttling level to none to disable I/O throttling on the specified array or
vCenter Server.

5. To enable I/O throttling for all arrays or vCenter Servers mapped to an RPA cluster:
a. Run the config_io_throttling command in the RecoverPoint CLI.
b. Select the RPA cluster to which the arrays and/or vCenter Servers are mapped.
c. Scroll down to the bottom of the list of seen arrays and vCenter Servers, and select
the last option in the list.
d. Set the throttling level:
l

Set the throttling level to low to limit the I/O rate of all RPAs in an RPA cluster to a
maximum read-rate of 200 MBps, per array.

Set the throttling level to high to limit the I/O rate of all RPAs in an RPA cluster to a
maximum read-rate of 50 MBps, per array.

Set the throttling level to custom, and specify the custom value in MBps to limit
the I/O rate of all RPAs in an RPA cluster to any other read-rate, per array.

Set the throttling level to none to disable I/O throttling of all RPAs in the specified
RPA cluster.

Results
The read-rate of the RPA on which the consistency group is running is limited to 50 MBps,
200 MBps, or the specified custom setting divided by the number of RPAs in the cluster.

How do I enable I/O throttling?

263

Troubleshooting

How do I check how throttling affects my system performance?


To check the effect that I/O throttling has on your system performance, check your
Storage read-rate during initialization on page 261 before and after enabling I/O
throttling.

How can I tell whether or not I/O throttling is enabled?


When I/O throttling is enabled, initializations take longer to complete. The higher the
throttling level, the longer initialization will take to complete.
To verify whether I/O throttling is enabled, run the get_system_settings CLI
command and note the state of the I/O Throttling section per RPA cluster.

Add more LUNs than allowed per VNX storage group


Note

This section is not relevant to Unity or VNXe.


RecoverPoint requires the following types of volumes at both production and remote
copies:
l

Replication

Journal

In VNX storage systems, the number of LUNs per storage group is limited. This limit is
typically 256, 512, or 1024, depending on the specific VNX model, as presented in the
EMC RecoverPoint Release Notes.

How do I know I need more VNX LUNs?


You know that you need more VNX LUNs when:
l

The following message is displayed in the RecoverPoint or Unisphere UI:


The maximum number of supported LUNs per storage group has
been reached. Either delete existing consistency
groups or contact Customer
Support for further assistance.

You calculate in advance the number of LUNs that you expect to need to support the
replication, journal, and repository volumes for your RecoverPoint configuration, and
that number exceeds the number of LUNs currently available.

What can I do if I need more VNX LUNs?


To overcome this limit, you can create additional storage groups, up to 16 per VNX
storage system. An equal number of virtual WWNs is created for each RPA Fibre Channel
port, and one virtual WWN from each RPA Fibre Channel port is assigned to each
additional storage group. Use the following procedure to create additional VNX storage
groups.
Procedure
1. Calculate the total number of storage groups to create, by dividing the estimated total
number of LUNs you need by the limit for your VNX model.
2. Log into RecoverPoint as user boxmgmt.
264

RecoverPoint 5.0 Administrator's Guide

Troubleshooting

3. Use the Number of virtual ports per port option to create the virtual WWNs at each
site, as follows:
a. Open theSetup > Modify settings > Site-1 details menu.
b. Use the Number of virtual ports per port option to set the desired number of virtual
WWNs.
c. Apply the modified settings.
4. Zone the newly created WWNs:
a. For Brocade switches that support NPIV, enable NPIV and set ports as F_Ports;
otherwise, set ports as L_Ports. For all other switches, every switch port to which a
RecoverPoint Fiber Channel port that uses virtual ports is attached must be
configured as a Fabric Loop port.
b. Create RPA-to-Storage zones with the new WWNs in the same way that existing
WWNs are already zoned.
5. Repeat step 3 on page 265 and step 4 on page 265 for each RPA cluster in your
RecoverPoint system.
Results
The RPAs automatically create new storage groups, and register the new WWNs to those
storage groups.

How can I verify that the storage groups were added successfully?
Upon successful completion of registration (which may take between several minutes
and an hour), the event, Registration of all the site RPAs initiators with the CLARiiON array
is complete., is generated for each site.

Swap the LUN numbers of journal volumes


Swapping LUN numbers for LUNs that have already been exposed to an RPA cluster
should be avoided when possible. When it cannot be avoided, it should be done
according to the following procedure.
Use the following procedure to swap LUN numbers on LUNs that have already been
exposed to an RPA cluster.
Procedure
1. Disable the consistency group.
Disabling the consistency group causes a full sweep of the consistency group when it
is enabled. This procedure will also cause journal loss.
2. Remove journals from the consistency group.
3. Swap the LUNs (on the storage array).
4. Add LUNs as journals.
5. Enable the consistency group.
NOTICE

The journal is deleted and a full sweep of the consistency group occurs.

How can I verify that the storage groups were added successfully?

265

Troubleshooting

Find your array serial numbers


This section will help you find the serial numbers of your storage arrays, when activating
RecoverPoint licenses.

Find the serial numbers of VNX arrays


For VNX Series arrays, use the Naviseccli getagent command to find the array serial
number.
For example:
naviseccli -Scope 0 -user <username> -password <password> -address
<spaddress> getagent
Agent Rev:
7.32.0 (5.2)
Name:
K10
Desc:
Node:
A-APM00120701240
Physical Node:
K10
Signature:
2998645
Peer Signature:
2998612
Revision:
05.32.000.4.007
SCSI Id:
0
Model:
VNX7500
Model Type:
Rackmount
Prom Rev:
8.10.00
SP Memory:
24576
Serial No:
APM00120701240
SP Identifier:
A
Cabinet:
SPE5
NOTICE

spaddress is the address of SPA or SPB, not the control station.

Find the serial numbers of Symmetrix arrays


For Symmetrix arrays, you can display the array serial number on the Symmetrix array or
through RecoverPoint.
On the Symmetrix array, run the SYMCLI command to find the array serial number.
For example:
# symcfg list
SYMMETRIX
Mcode Cache Num Phys Num Symm
SymmID Attachment Model Version Size (MB) Devices Devices
000194900874 Local VMAX-1SE 5876 28672 32 695

In RecoverPoint, run the get_splitter_states CLI command to display the array


serial number.
For example:
SiteC> get_splitter_states cluster=SiteC
Cluster:

266

RecoverPoint 5.0 Administrator's Guide

Troubleshooting

SiteC:
Splitters:
SYMM- 000194900874
Status: OK

Find the serial numbers of VPLEX arrays


For VPLEX arrays, you can display the array serial number on the VPLEX array or through
RecoverPoint.
For example:
VPlexcli:/> ls -t /engines/*::top-level-assembly
/engines/engine-2-1:
Name
Value
------------------ -------------top-level-assembly FNM00104500511
/engines/engine-1-1:
Name
Value
------------------ -------------top-level-assembly FNM00104500511

In RecoverPoint, run the get_splitter_states CLI command to display the array


serial number.
ClusterA> get_splitter_states cluster=ClusterA
Cluster:
ClusterA:
Splitters:
FNM00104500511
Status: OK

Find your RPA cluster IDs


Procedure
1. Run the CLI command get_internal_cluster_uids.
2. Enter the cluster name, or alternately, press Enter to see a list of all of the RPA cluster
IDs in the system.
3. Note the output:
Internal cluster uids:
Cluster1: 123123123123123123
Cluster2: 345345345345345345
Cluster3: 456456456456456456

Find your XtremIO cluster IDs


The licenses for XtremIO are enforced to the XtremIO cluster ID via the license Locking ID.
The XtremIO cluster ID (called a PSNT, or Product PSNT Serial Number in XtremIO)
contains 5 alphanumeric characters followed by 9 numeric characters.
For example: XIO00123456789

Find the serial numbers of VPLEX arrays

267

Troubleshooting

Find your XtremIO cluster IDs (PSNTs) in XtremIO


Your XtremIO PSNTs can be displayed:
l

In the Inventory of the XtremIO GUI, for example:

By running the show-clusters-info command in the XtremIO CLI, for example:


xmcli (admin)> show-clusters-info
Table 58 Cluster information
ClusterName

Index
State

ConnState

ActivationTime

Start-Time

SW-Version PSNT

EncryptionMode

xbrick414

active

connected
Wed Jun 10
17:30:15
2015

Wed Jun 10
17:30:15
2015

4.0.0-54
XIO00150201280

disabled

Find your XtremIO cluster IDs (PSNTs) in RecoverPoint


RecoverPoint discovers XtremIO PSNTs automatically and they are displayed when you
Register storage on page 149. When multiple PSNTs exist the user should register each
PSNT, one at a time.

Understanding the RecoverPoint license


The license file format illustrated below is described in the following table.
Figure 12 License file format

Figure 13 on page 269 contains some common examples of RecoverPoint licenses. One
license file can include multiple licenses. Each license starts with the word INCREMENT.
268

RecoverPoint 5.0 Administrator's Guide

Troubleshooting

Figure 13 Example licenses

The following table described the variables in the license file, and their possible values.
Table 59 License file variables

Variable

Description

Examples

{Replication} The RecoverPoint license model.


Valid values are:

{Type}

{Prod}

LOC for local replication

REM for remote replication

RP.xx.REM.xx
RP.xx.LOC.xx

The RecoverPoint license model.


Valid values are:

RP.CL.xxx.xx
RP.SE.xxx.xx

CL for RecoverPoint/CL

RP.EX.xxx.xx

EX for RecoverPoint/EX

SE for RecoverPoint/SE

The array model. Valid values are:


l

00 - for RecoverPoint/CL licenses

ELA - for licenses based on Enterprise


License Agreements

VMAX

VMAX10K, VMAX20K, VMAX40K

VPLEX

RP.xx.xx.VNX5700

Understanding the RecoverPoint license

269

Troubleshooting

Table 59 License file variables (continued)

Variable

Description
l

Unity models (for example, Unity 300)

VNXe3200

VNX arrays (for example, VNX7600)

CX, CX4-120, CX4-240, CX4-480,


CX4-960, AX4-5

{Version}

The license schema version.

{Expiration}

The license expiration date.


Licenses can be:
l

Permanent (for an unlimited time)

Expiring:
- Evaluation: Up to 90 days

Examples

1.0
l

15-dec-2014

permanent

Unity/VNXe/VNX array serial


numbers are in the format:
APM00120801240

VPLEX array serial numbers are


in the format:
FNM00105500511

VMAX array serial numbers are


in the format: 000194910874

RPA Cluster IDs are in the


format:
7754643987555540854

12 (for 12 TB)

0 (for unlimited capacity)

- Temporary: Up to 365 days


- Emergency: Up to a few days
Note

Warnings will appear 90 days before


license expiration.
Valid values are:

{LockingID}

{Capacity}

270

RecoverPoint 5.0 Administrator's Guide

permanent

A date in the format day-month-year

The unique identifier of the entity that the


license is tied to.
A Locking ID can be:
l

The serial number of a Unity/VNXe/VNX


Series array.

The serial number of a VPLEX array.

The SYMM ID of a Symmetrix array.

An RPA Cluster ID.

A System ID

The amount of data licensed for replication at


a given site.
l

In RecoverPoint/CL: as purchased, per


RPA cluster.

In RecoverPoint/SE: the entire Unity/


VNXe/VNX array capacity (unlimited)

In RecoverPoint/EX: For Unity/VNXe/VNX,


the entire array capacity. All other arrays;
as purchased per array.

Troubleshooting

Table 59 License file variables (continued)

Variable

Description

Examples

A value of zero means that the


replication capacity is unlimited.
{Unit}

The unit of data being replicated.

TB

{Customer}

The customers name and physical address,


as entered in the RecoverPoint sales order.

EMC, MyStreet, Hopkington, MA

{Issued}

The license creation date, in the format daymonth-year.

15-Dec-2014

{Site_info}

The Site ID is a global EMC setting that


identifies the geographical location of the
customer site.
In RecoverPoint, the EMC site ID is called a
Location ID.

ACTIVATED TO SiteID: 112207055

The site ID is received by the RecoverPoint


system from ELMS and is sent to the system
report mechanism (SyR) and used (in
RecoverPoint/CL licensing) to associate an
RPA cluster to a geographical location.
{SN}

The Activation ID, a uniquely generated value 2055509


specific to the activation. Every time a license
is activated, the SN changes. This value is
sent to the system report mechanism (SyR)
for support purposes.

{Sign}

An encrypted hash segment containing the


license information which is used by
RecoverPoint to validate the license and
ensure its uniqueness.

0004 0EAA 52A0 F74E 6EG4 BE6A


6CF0 1400 \
10E3 A333 704C 11E1 9DAF DA5B
1C0B

Calculate your required journal size


The calculation of journals depends on whether or not snapshot consolidation is
enabled.
The recommended minimum journal size when not utilizing snapshot consolidation is:
MinJournalSize = 1.05 * [(D data per second)*(required rollback time
in seconds)/(1 image access log size)] + (reserved for marking)

For example, if:


D data per second = 5 Mb/s
required rollback time = 24 hr = 86 400 s
image access log = 0.20
reserved for marking = 1.5 GB

Calculate your required journal size

271

Troubleshooting

Then the minimum journal size would be:


1.05 * 5 Mb/s * 86 400 s / (1 - 0.20) + 1.5 GB = 579 000 Mb
579 000 Mb = 579 000/8 MB = 72 375 MB = 72.4 GB

You can use iostat (UNIX) or perfmon (Windows) to determine the value for D data per
second.
The default image access log size is 20%, and it can only be changed using the
RecoverPoint CLI.
See the EMC RecoverPoint Release Notes for the minimum and maximum journal size
limitations.

Minimum journal size with snapshot consolidation


Note

Not relevant for XtremIO.


Journal sizing with snapshot consolidation must take into account the incremental
change of data over the period of consolidation. Therefore, when snapshot consolidation
is enabled, the formula for estimating the minimum required journal size is:
MinJournalSize =
SpaceForMarkingAndReplication
+ AvgHourlyChangeRate
* (UnconsolidatedDuration
+ DailyConsolidations
+ WeeklyConsolidations
+ MonthlyConsolidations)
/ spaceAfterConsolidationProcess
/ spaceAfterImageAccessProcesses
* spaceAfterCompression
Note

See the EMC RecoverPoint Release Notes for the minimum and maximum journal size
limitations, when snapshot consolidation is enabled.
For example, if an application has the following requirements:
l

An average data change rate of 0.5MBps

An image access log size that is 20% of the total journal size

A distributed consistency group

A 12 hour unconsolidated duration (i.e. protection window)

10 daily snapshots

2 weekly snapshots

3 monthly snapshots

Journal compression is disabled

The minimum size for the journal at this copy would be calculated as (if performance data
was unavailable for the localities of reference):
MinJournalSize = 6
+ { [0.5*(60*60/1024)]
* 12
+ [24 + (10-1)*24*(1-0.20)]
+ [7*24*(1-0.20)+ (2-1)*7*24*(1-0.20)*(1-0.20)]

272

RecoverPoint 5.0 Administrator's Guide

Troubleshooting

+ [7*24*4*(1-0.20)*(1-0.20)+(3-1)*28*24*(1-0.20)*
(1-0.20)*(1-0.20)]
/ (1-0.25)
/ [1-(0.20 + 0.10)]
* (1-0)}

Where:
Table 60 Journal size with snapshot consolidation equation legend

Field

Description

MinJournalSize

The minimum journal size required to sustain the specified


journal configuration, in GB.

SpaceForMarkingAndReplication

The journal space required (mainly) for the storage of the


delta marking information, and other replication processes, in
GB.
Can be one of two constant values:
l

If the group is a regular (non-distributed) consistency


group, set this value to 1.5.

If the group is a distributed consistency group, set this


value to 6.

Note

The space required for marking and replication processes is


four times larger for distributed consistency groups,
regardless of the number of RPAs configured. This is due to
the four consistency group segments that are created when
you use distributed consistency groups.
AvgHourlyChangeRate

The average change rate of the replicated data based on


iostat.a, in GB per hour.
For example:
If the average data change rate of the replicated data is
0.5MBps, the AvgHourlyChangeRate is:
0.5MBps * (60Sec * 60Min / 1024forGB) =
1.75 GBps

Note

If this value is unknown, create the consistency group and


allow it to replicate during a period in time that adequately
represents your average change rate. Then, Detect
Bottlenecks on page 201 for this period of time and note the
Incoming writes rate for link.
UnconsolidatedDuration

The period of time during which snapshot data is not to be


consolidated, in hours. The periods start time is always
today, and the periods end time is n hoursago.Also referred to
as the protection window, or the continuous protection period.
For example:

Minimum journal size with snapshot consolidation

273

Troubleshooting

Table 60 Journal size with snapshot consolidation equation legend (continued)

Field

Description
If the past two days-worth of unconsolidated data must
always be available, the value of UnconsolidatedDuration is:
2 * 24Hrs =

48

Note

Value must be at least 12 hours. This value is set and can be


modified through the Do not consolidate snapshots for at
least setting in the journal policies.
DailyConsolidations

The required number of daily snapshots less the journal


space saved during daily consolidations (i.e. the
%DailyLoR), in hours.
This value is computed as:
24 + (NumberOfDailyConsolidations - 1) * 24
* (1-%DailyLoR)
For example:

If three daily snapshots are required, the value of


DailyConsolidations is:
24 + (3-1) * 24Hrs * (1-%DailyLoR)

Note

This value is set, and can be modified, through the one


snapshot per day for x days setting in the journal policies.
There must be 24 hours-worth of unconsolidated data
available in order to create the first daily snapshot, and
therefore, the first 24 hours are not multiplied by the
%DailyLoR.
%DailyLoR

The percent of journal space saved during each daily


consolidation due to the data that is repeated to the same
location, and therefore, write-folded upon consolidation.
Note

The locality of reference depends on each consistency groups


(i.e. applications) specific I/O pattern, how many blocks get
re-written, and other factors. If you do not know this value,
you can use a value of 0.2 (i.e. 20%) for a general estimation.
For a more precise value based on your system performance,
Collect System Information on page 209 and send it to EMC
Customer Support.
To calculate the journal space left after daily consolidations,
use a value of:
1 - %DailyLoR, For example: 1 - 0.20 (if performance
data is not available).

274

RecoverPoint 5.0 Administrator's Guide

Troubleshooting

Table 60 Journal size with snapshot consolidation equation legend (continued)

Field

Description
Note

The %DailyLoR is not applied to the first daily snapshot, as


there must be 24 hours-worth of unconsolidated data
available in order to create the first daily snapshot. See the
DailyConsolidations parameter for a detailed
description.
WeeklyConsolidations

The required number of weekly snapshots, less the journal


space saved during daily and weekly consolidations (i.e. the
%DailyLoR and the %WeeklyLoR), in hours.
This value is computed as:
7 * 24 * (1-%DailyLoR) +
(NumberOfWeeklyConsolidations - 1) * 7 * 24
*
(1 - %DailyLoR ) * (1 - %WeeklyLoR )
For example:

If four weekly snapshots are required, the value of


WeeklyConsolidations is:
7 * 24 * (1-0.20) + (4 - 1) * 7 * 24 * (1 0.20 ) * (1 - 0.20 )
Note

This value is set, and can be modified, through the one


snapshot per week for y weeks setting in the journal policies.
There must be 7 daily snapshots available in order to create
the first weekly snapshot, and therefore, the first 7 days (7 *
24Hrs) are only multiplied by the %DailyLoR (as opposed to
the %DailyLoR and %WeeklyLoR).
%WeeklyLoR

The percent of journal space saved during each weekly


consolidation due to the data that is repeated to the same
location, and therefore, write-folded upon consolidation.
Note

The locality of reference depends on each consistency groups


(i.e. applications) specific I/O pattern, how many blocks get
re-written, and other factors. If you do not know this value,
you can use a value of 0.2 (i.e. 20%) for a general estimation.
For a more precise value based on your system performance,
Collect System Information on page 209 and send it to EMC
Customer Support.
To calculate the journal space left after weekly
consolidations, use a value of:
1 - %WeeklyLoR, For example: 1 - 0.20 (if performance
data is not available).

Minimum journal size with snapshot consolidation

275

Troubleshooting

Table 60 Journal size with snapshot consolidation equation legend (continued)

Field

Description
Note

The %WeeklyLoR is not applied to the first weekly snapshot,


as there must be 7 daily snapshots available in order to
create the first weekly snapshot. See the
WeeklyConsolidations parameter for a detailed description.
MonthlyConsolidations

The required number of monthly snapshots, less the journal


space saved during daily, weekly, and monthly consolidations
(i.e. the %DailyLoR, the %WeeklyLoR, and the
%MonthlyLoR), in hours.
This value is computed as:
7 * 24 * 4 * (1 - %DailyLoR) * (1 %WeeklyLoR) +
(NumberOfMonthlyConsolidations - 1 ) * 28 *
24 * (1 - %DailyLoR) *
(1 - %WeeklyLoR) * (1 - %MonthlyLoR)
For example:

If two monthly snapshots are required, the value of


MonthlyConsolidations is:
7 * 24 * 4 * (1 - 0.20) * (1 - 0.20) + (2 1 ) * 28 * 24 * (1 - 0.20) * (1 - 0.20) *
(1 - 0.20)
Note

This value is set, and can be modified, through the journal


policies. In the snapshot consolidation process, a month is
represented as 28 days. There must be 28 daily snapshots or
4 weekly snapshots available in order to create the first
weekly snapshot, and therefore, the first 28 days (7 *
24Hrs * 4Days) are only multiplied by the %DailyLoR
and the %WeeklyLoR (as opposed to the %DailyLoR, the
%WeeklyLoR and the %MonthlyLoR.
%MonthlyLoR

The percent of journal space saved during each monthly


consolidation due to the data that is repeated to the same
location, and therefore, write-folded upon consolidation.
Note

The locality of reference depends on each consistency groups


(i.e. applications) specific I/O pattern, how many blocks get
re-written, and other factors. If you do not know this value,
you can use a value of 0.2 (i.e. 20%) for a general estimation.
For a more precise value based on your system performance,
Collect System Information on page 209 and send it to EMC
Customer Support.
To calculate the journal space left after monthly
consolidations, use a value of:
276

RecoverPoint 5.0 Administrator's Guide

Troubleshooting

Table 60 Journal size with snapshot consolidation equation legend (continued)

Field

Description
1 - %MonthlyLoR, For example: 1 - 0.20 (if
performance data is not available).
Note

The %MonthlyLoR is not applied to the first monthly


snapshot, as the data of the first daily snapshot is not
repeated to the same location. See the
MonthlyConsolidations parameter for a detailed description.
%LAProcess

The percentage of the journal space that is reserved for


logged access processes.
Default value=20%
Note

This value is set, and can be changed through the


image_access_log_size parameter of the
config_copy_policy CLI command.
%VAProcess

The percentage of the journal space that is reserved for virtual


access processes.
Constant value=10%
This value is constant, and cannot be changed.

spaceAfterImageAccessProcesses The percentage of the journal space that is left after the space
reserved for the logged (%LAProcess) and virtual
(%VAProcess) image access processes is subtracted.
Note

This value is calculated as: 1 - %LAProcess +


%VAProcess, for example: 1 - 0.20 + 0.10
spaceAfterConsolidationProcess

The percentage of the journal space that is left after the space
reserved for the snapshot consolidation process is
subtracted, where the snapshot consolidation process uses
25% of the total journal space.
This value is constant, and is calculated as: 1 - 0.25

spaceAfterCompression

The percentage of the journal space that is left after the space
saved by journal compression is subtracted:
l

If the data contained in the journal is not compressible,


use a value of: 1 - 0.

If the data contained in the journal is somewhat


compressible, use a value of: 1 - 0.20.

If the data contained in the journal is highly


compressible, use a value of: 1 - 0.40.

This value is set and can be modified through the journal


compression setting in the Journal policies on page 69.
Value depends highly upon the compressibility of the data
being stored in the journal, which will vary per consistency
Minimum journal size with snapshot consolidation

277

Troubleshooting

Table 60 Journal size with snapshot consolidation equation legend (continued)

Field

Description
group. The specified values are general estimations, and if
journal compression is used, they should be replaced with
actual values. To obtain actual values, enable the journal
compression setting for a representative period of time
(Warning: both enabling and disabling this setting causes
journal loss), and then Collect System Information on page
209. Send the collected system information to EMC Customer
Support to obtain journal compression values based on the
performance data of the group being replicated.
Note

Enabling journal compression is not recommended, as this


process requires RPA CPU resources, and enabling it has
adverse effects on the distribution process, and by
association, the RPA throughput. See the EMC RecoverPoint
Release Notes for performance statistics of consistency
groups, with and without journal compression.
a.

You can use iostat (UNIX) or perfmon (Windows) to determine the value for data per second.

Validate replication integrity


Integrity validation ensures replication integrity between the production and copy
volumes of a specific consistency group, or all consistency groups in a RecoverPoint
system, without necessitating the stopping of I/Os at the production.

How can replication integrity be compromised?


Replication integrity can be compromised, for example, due to:
l

Configuration issues. If the host replicates data without going through a RecoverPoint
write splitter, as in ESX environments, for example.

Faulty procedures. Due to incorrect usage of the clear_markers command or


incorrectly performing a LUN resize, for example.

When should I validate replication integrity?


You should validate replication integrity:

278

If image access fails


RecoverPoint protects you from possible integrity issues by not allowing you to test,
fail over to, or recover production from, an image whose replication integrity has been
compromised. During copy testing, failover, and production recovery; when accessing
an image in logged access mode, image access will fail if the selected image is not
consistent with its production source. In such cases, you should check the integrity of
the group Using the integrity check tool on page 280 to ensure data consistency
between the groups production and copy volumes.

Periodically

RecoverPoint 5.0 Administrator's Guide

Troubleshooting

You can validate replication integrity periodically Using the periodic integrity
validation tool on page 281 to ensure data consistency between all of the production
and copy volumes in your RecoverPoint system on an ongoing basis.

How the integrity check tool works


The integrity check tool enables users to perform an integrity check of all of the volumes
of a consistency group copy to ensure consistency between all of the copy volumes and
their production sources.

Before you begin


Before you use the integrity check tool, note the following limitations:
l

The integrity check process only runs on the groups primary RPA.

The integrity check process reads all of the data of all of the production and copy
volumes of the group, which can put extra load on the production storage, and then
compares their signatures, which can put extra load on the WAN. The extra load
placed on the WAN may lead to high load or WAN disconnection, which will cause the
integrity check to stop before completing the process on all consistency group
volumes.

Integrity check is not supported during snap-based replication.

How it works
When enabled, the integrity check tool:
1. Reads a section of the groups production and copy volumes.
2. Creates hash signatures of that section of the production and copy volumes.
3. Compares the two signatures.
4. Repeats this process as many times as required across all group volumes.
5. If no inconsistency is found, event 4107 - Integrity check completed successfully, is
logged.
NOTICE

If at any time during this process a possible inconsistency is found, event 4108 - A
possible replication integrity issue has been identified is logged.

How the periodic integrity validation tool works


The periodic integrity validation tool enables users to perform an integrity check on all of
the volumes of a consistency group on an ongoing bases, to ensure consistency between
all production and copy volumes of all groups in a RecoverPoint system, at all times.

Before you begin


Before you use the periodic integrity validation tool, note the following limitations:
l

By default, periodic integrity validation is disabled.

Integrity validation during snap-based replication is not supported.

The periodic integrity validation tool is only enabled if all RPA clusters in your
RecoverPoint system are running RecoverPoint 4.1 SP1 or later.

To temporarily disable or enable all of the link settings in your entire RecoverPoint
system that were set Using the periodic integrity validation tool on page 281 see Can
I disable or enable integrity validation globally? on page 281.
How the integrity check tool works

279

Troubleshooting

How it works
For each copy whose integrity you want to periodically validate, you will need to run the
config_periodic_integrity_validation CLI command, specify the copy name,
and select a mode of operation.
The following modes are available:
l

Light: In this mode, each RPA will read 1MB every 5 seconds.

Heavy: In this mode, each RPA will read 10MB every 0.5 second.
Note

When using this mode, the maximum degradation in total performance (throughput
and IOPs) should not exceed 10%. However, using this mode is not recommended,
and this mode should not be used without explicit instruction from EMC Customer
Support.
l

Disabled

When enabled, the tool randomly selects an area of the production storage protected by
each consistency group:
1. Each RPA sequentially scans the volumes of the consistency groups that are running
on it (selecting a different group each time) and randomly selects an area of the
storage protected by the group. Each volumes beginning and end receive priority in
the random selection process.
2. If the randomly selected area of a production volume has already been replicated but
has not been written to the target storage yet, this area is not scanned and is
considered consistent.
3. If a possible replication inconsistency is detected, the tool will re-scan the same area
of the production volume to ensure that the results of the first scan are correct.
4. If the second scan detects a possible replication integrity issue, event 4165 is
logged, and a call home event will automatically open a service request with EMC
Customer Support with the relevant consistency group name, volume name and
metadata, while the data itself is written in the RPA logs.
5. If no replication integrity issues are detected no events are logged.

How do I configure integrity validation?


Replication integrity is enabled separately per tool:
l

How the integrity check tool works on page 279

How the periodic integrity validation tool works on page 279

Note

Both integrity validation tools can only be accessed using the RecoverPoint CLI.

Using the integrity check tool


All of the settings of the integrity check tool are configured per link of a consistency
group.
You can:
l

280

Start a new integrity check by running the start_integrity_check CLI


command.

RecoverPoint 5.0 Administrator's Guide

Troubleshooting

Stop an integrity check that is in progress by running the stop_integrity_check


CLI command.

Display the progress of an integrity check by running the


show_integrity_check_status CLI command.

Using the periodic integrity validation tool


All of the settings of the periodic integrity validation tool are configured per link of a
consistency group.
You can:
l

Enable periodic integrity validation by running the


config_periodic_integrity_validation CLI command, and setting the
mode to light.
NOTICE

Setting the integrity validation mode to heavy might affect performance. Consult EMC
Customer Support before setting the mode to heavy.
l

Disable periodic integrity validation by running the


config_periodic_integrity_validation CLI command, and setting the
mode to disabled.

Display the current integrity validation settings by running the


get_group_settings command and noting the value of the Integrity validation
policy field.

How do I know whether or not integrity validation was successful?


The output of the integrity validation process differs between the integrity check tool and
the periodic integrity validation tool.

The integrity check tool output


After Using the integrity check tool on page 280 one of the following events is logged to
indicate whether or not a possible integrity issue has been detected:
l

Event 4107 - Integrity check completed successfully.

Event 4108 - A possible replication integrity issue has been identified.

The periodic integrity validation tool output


After Using the periodic integrity validation tool on page 281, if a possible replication
integrity issue is detected, event 4165 is logged and a call home event automatically
opens a service request with EMC Customer Support containing the name of the relevant
group and volume, and the relevant metadata. After receiving the service request, EMC
Customer Support will pro-actively contact you to help resolve any possible integrity
issues.

Can I disable or enable integrity validation globally?


You can temporarily disable or enable all of the link settings in your entire RecoverPoint
system that were set Using the periodic integrity validation tool on page 281, without
deleting the link settings by running the
config_global_periodic_integrity_validation CLI command. Running this
command and setting enabled to no, will override all of link-level settings. Running this
command and setting enabled to yes, will use the configured link-level settings.
How do I know whether or not integrity validation was successful?

281

Troubleshooting

To display whether or not global periodic integrity validation is enabled, run the
get_global_policy CLI command.

What should I do if an issue is detected?


If a possible replication integrity issue is detected, use the clear_markers CLI command
to trigger a full sweep. If problem persists, contact EMC Customer Support.

MetroPoint group conversion


Use the following procedure to convert a regular consistency group into a MetroPoint
group, or a MetroPoint group into a regular consistency group.
Before you begin
l

To become a MetroPoint group, the production volume must be a VPLEX distributed


(DR1) volume, and it may not have more than one remote copy.

To protect a VPLEX Metro distributed volume at both VPLEX clusters simultaneously,


use MetroPoint topology.

Procedure
1. Select Protection > Manage Protection
2. Select a group
3. Click Add Copy and follow the instructions in Add a copy to a group on page 99.
4. Add a standby production copy
Results
The group becomes a MetroPoint group.
Note

Follow the instructions in Remove a copy from a group on page 106 and remove the
standby production copy to turn a MetroPoint group into a regular group.

Recovering from a cluster disaster


After a full cluster disaster or a switch disaster, it may take 10 minutes or more for all the
components of the RPA system to reboot, reconnect, and restore full operation.

282

RecoverPoint 5.0 Administrator's Guide

CHAPTER 9
Events

This section describes all of the events issued by the RecoverPoint system.
l
l
l

Normal Events.....................................................................................................284
Detailed events................................................................................................... 294
Advanced events.................................................................................................330

Events

283

Events

Normal Events
The following table contains the Normal scope events that are issued by the RecoverPoint
system.
Table 61 Normal scope events

ID

Topic

Level

Summary

More Details

User Action

1002

Mgmt

Info

User logged out

None

1010

Mgmt

Warning

License grace period


expires in one day

Grace period expires in 1 day. You


must activate each license and
enter your licenses into
RecoverPoint.

1012

Mgmt

Warning

License grace period


about to expire

License grace period expires in less than <expiration interval> days. You
must obtain a new RecoverPoint
license.

1013

Mgmt

Error

License expired

None

Obtain a new RecoverPoint


license.

1021

RPA

Error

HBA firmware error

None

2000

RPA
Cluster

Info

Cluster management
running on RPA

None

3002

RPA

Warning

Cluster management
switched over to RPA

None

3007

RPA

Warning
off

RPA is up

None

3014

RPA

Warning

RPA unable to access


volume

None

3022

RPA

Warning
off

LAN link to RPA restored

None

3023

RPA

Warning

LAN link to RPA down

None

3155

Group

Info

Snap interfaces created


on storage

None

3156

Group

Info

Snap interfaces deleted


from storage

None

3170

Mgmt

Info

New site keys and


certificates are
generated and being
synchronized between
clusters.

None

3171

Mgmt

Info

New site keys and


certificates are
synchronized between
clusters.

None

4000

Group

Info

Group capabilities OK

None

284

RecoverPoint 5.0 Administrator's Guide

Events

Table 61 Normal scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

4001

Group

Warning

Minor problem in group


capabilities

None

4003

Group

Error

Group capabilities
problem

None

4007

Group

Info

Pausing data transfer for


group

None

4008

Group

Warning

Pausing data transfer to


copy

1. To determine the cause of this


event, check adjacent events for
this group, both before and after
this event.
2. Verify that transfer resumes in a
timely manner. To do so, check the
transfer state in the RecoverPoint
GUI or CLI, or check subsequent
events for this group.
3. If transfer does not resume,
collect logs for the period when the
event occurred, and contact EMC
Customer Support

4009

Group

Error

Pausing data transfer for


group

None

4010

Group

Info

Starting data transfer to


copy

None

4015

Group

Info

Transferring latest
snapshot

Transferring latest snapshot before


pausing transfer (no data loss)

4016

Group

Warning

Transferring latest
snapshot

Transferring latest snapshot before


pausing transfer (no data loss)

4017

Group

Error

Transferring latest
snapshot

Transferring latest snapshot before


pausing transfer (no data loss)

4018

Group

Warning

Transfer of latest
snapshot complete

Transfer of latest snapshot from


source complete (no data loss)

4019

Group

Warning

Group under high load

Group in high load -- transfer to be


paused temporarily

4020

Group

Warning
off

Group high load over

None

4021

Group

Error

Initialization paused

Initialization has been paused


because the journal is full.

To complete initialization, add


journal volumes to the journal
or enable long initialization.

4022

Group

Error off

Initialization resumed

None

4023

Group

Error

Transfer paused

Transfer has been paused because


the journal is full.

To restart transfer, first disable


access to the accessed image.

4024

Group

Error off

Transfer restarted

None

Normal Events

285

Events

Table 61 Normal scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

4025

Group

Warning

Group under high load

Group is undergoing a high load so


initialization will be restarted.

4026

Group

Warning
off

Group high load is over

None

4027

Group

Error

Group under high load

Roll to the physical image will be


paused and transfer will be paused.

4028

Group

Error off

Group high load is over

None

4040

Group

Error

Journal error

Full sweep to be performed.

4041

Group

Info

Group side activated

None

4042

Group

Info

Group side deactivated

None

4043

Group

Warning

Group side deactivated

None

4044

Group

Error

Group side deactivated

None

4051

Group

Info

Resuming distribution

Disabling access to image and


resuming distribution.

4054

Group

Error

Enabling access to image None

4057

Group

Error

Specified image removed The specified image has been


from journal
removed from the journal.

4062

Group

Info

Access enabled to latest


image

None

4063

Group

Warning

Access enabled to latest


image

None

4064

Group

Error

Access enabled to latest


image

None

4080

Group

Warning

Current lag exceeds


maximum lag

None

4081

Group

Warning
off

Current lag within policy


limit

None

4084

Group

Info

Markers cleared

None

4085

Group

Warning

Unable to clear markers

None

4086

Group

Info

Synchronization started

None

4087

Group

Info

Synchronization
completed

None

4091

Group

Error

Image access log is full

Access to copy volumes for this


group at this RPA cluster has been
disabled. If you do not wish to
retain the writes that were added to
the copy image while image access
was enabled, disable image
access; the relevant writes are

286

RecoverPoint 5.0 Administrator's Guide

Events

Table 61 Normal scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

automatically undone.
Alternatively, use the undo writes
command to undo the same writes;
image access remains enabled.
- If, however, you wish to retain the
writes that were added to the copy
image while image access has been
enabled, you must increase the size
of the copy journal by adding one or
more volumes to the journal.
Alternatively, you can set the image
access mode to direct; however, in
that case, the copy journal is
erased, and a full sweep
resynchronization will be required
to restore consistency between the
production and copy images.
- To help prevent this problem from
recurring, increase the size of your
image access log, using one or both
of the following:
- - Add volumes to the journal.
- - Modify the policy setting for the
portion of the journal allocated to
the image access log. This is
permitted only when image access
is disabled.
4095

Group

Info

Writes to image access


log cannot be undone

The image access log is being


written to storage, so the writes to
the image access log cannot be
undone.

4097

Group

Warning

Starting three-phase
distribution

Max journal lag exceeded or


insufficient journal

Check the maximum journal lag


setting value, and verify that the
current value is indeed
required. If the maximum
journal lag value has not been
defined or the current journal
lag is lower than the maximum
journal lag value, this is
probably a journal space or
performance issue. In this case,
enlarge your journal or increase
the speed of the journal by
placing it in a faster array, on
faster media, or any other
procedure that can increase the
journal write-rate.

4098

Group

Warning
off

Maximum journal lag


within limit

Distribution normal -- rollback


information retained.

Normal Events

287

Events

Table 61 Normal scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

4099

Group

Warning

Starting long
initialization

None

4110

Group

Info

Enabling virtual access


to image

None

4111

Group

Info

Virtual access to image


enabled

None

4112

Group

Info

Rolling to physical image None

4113

Group

Info

Roll to physical image


stopped

None

4114

Group

Info

Roll to physical image


complete

Logged access to physical image


enabled.

4115

Group

Error

Cannot enable access to


virtual image

Unable to enable access to virtual


image due to a partition table error.

4116

Group

Error

Virtual writes buffer full

Virtual access is disabled.

4118

Group

Error

Cannot enable virtual


access

None

4119

RPA

Error

Initiator issued out of


bounds I/O

None

Contact EMC Customer Support

4120

Group

Warning

Journal usage exceeding


threshold

Because you are accessing a copy


image in logged access mode for an
unusually long time, the copy
journal usage is exceeding the
normal threshold, and this may
affect the image access log
capacity.

4122

Group

Error off

Trying to regain write


permissions

Trying to regain write permissions


to non-production copy.

4124

Group

Error off

Read and write to volume None


restored

4129

Group

Warning

Logged access enabled


for unusually long time

Logged access has been enabled


on a group copy for an unusually
long time.

4130

Group

Warning
off

Logged access no longer


enabled for unusually
long time

Logged access on group copy is


now disabled or it has been
enabled to a different image.

4135

Group

Info

Copy entered
maintenance mode

All copy activities have been


suspended.

4136

Group

Info

Copy exited maintenance All copy activities have resumed.


mode

4137

Group

Info

Snapshot consolidation
has been successful

288

RecoverPoint 5.0 Administrator's Guide

None

Events

Table 61 Normal scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

4138

Group

Warning

Snapshot consolidation
failed

None

4140

Group

Info

Change pipe mode


because of sync/async
dynamic settings

None

4141

Group

Info

Snapshot consolidation
policy change

The user has changed the snapshot


consolidation policy setting(s).

4142

Group

Info

Snapshot consolidation
request registered

None

4156

Group

Warning

Maximum lag exceeded

Maximum lag between source and


target cluster exceeded.

4159

Group

Info

Volume was resized

Replication set will be resized


automatically.

4160

Group

Info

Volumes were resized

None

4161

Group

Info

Short initialization
started

Short initialization started because


of resize

4162

Group

Info

Replication set resize


enabled

You may now resize the replication


set capacity in RecoverPoint.

4174

RPA
Cluster

Info

volume migration started Migration of volume configuration


data has started

4210

RPA
Cluster

Warning

VM no longer being
replicated

None

4211

RPA
Cluster

Warning

VM partially replicated

None

4212

RPA
Cluster

Info

VM is fully replicated

None

4213

RPA
Cluster

Error

vCenter Server is not


accessible

None

4300

RPA
Cluster

Warning

Writing rate to the


production journal is
slow

If bottleneck detection shows a


delay factor >> 1 for this journal
replication of the group may be
disrupted.

4301

RPA

Warning

RPA is unable to handle


incoming data rate due
to high compression
level

None

Try to reduce the load on the


RPA by lowering the
compression level. Alternatively

4302

Group

Warning

Journal is unable to
handle incoming data
rate

None

If the event recurs, try to


configure journal volumes for
better performance.

4303

Group

Warning

Journal and replication


volumes at copy are

None

If the event recurs, try to


configure journal volumes for
better performance.
Normal Events

289

Events

Table 61 Normal scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

unable to handle
incoming data rate
4304

Group

Warning

Remote storage is
unable to handle
incoming data rate;
regulating distribution

In distribution, there are reads and If the event recurs, try to


writes to the journal and replication configure remote storage for
volumes on the remote storage. If
better performance.
I/O access to this storage is too
slow, high load states may occur.

4305

Group

Warning

Remote cluster is unable


to handle incoming data
rate

This may cause high load states at


the local cluster.

4306

RPA
Cluster

Warning

Data transfer rate


between clusters is slow

This may increase RPO, and cause a If this event recurs, examine the
high-load state.
WAN/FC link for possible high
latency or packet loss. Consult
your network administrator if
needed.

4307

RPA
Cluster

Warning

Reading rate from local


replication volumes
during resynchronization
is slow

A slow reading rate during


resynchronization (which involves
reading a very large volume of data
from the local replication volumes)
may increase RPO.

If the event recurs, try to


configure local replication
volumes for better performance.

4308

RPA
Cluster

Warning

Reading rate from


replication volumes at
copy during
resynchronization is slow

A slow reading rate during


resynchronization (which involves
reading a very large volume of data
from replication volumes at the
copy) may increase RPO.

If the event recurs, try to


configure replication volumes at
the copy for better performance.

4309

RPA

Warning

RPA utilization reached


80%

High utilization of RPA resources


may be due to high volume of
incoming data, high volume of
incoming IOPS, or high
compression level.

If bottleneck detection shows


that compression is using 80%
or more of CPU resources,
reduce the compression setting.
If incoming data is near the
limit, consider adding an RPA or
moving groups to other RPAs .

4310

Group

Warning

Link utilization reached


80%

High utilization of the link may be


due to a high volume of incoming
data, high volume of incoming
IOPS, or high compression level.

If bottleneck detection shows


that compression for this link is
using 80% or more of CPU
resources, reduce the
compression setting. If
incoming data is near the limit,
consider splitting the existing
group onto two or more RPAs .

4312

Group

Error

Switching to virtual
access

After resizing, virtual image access


with roll is not allowed to a point-intime before the volume resize.
Virtual image access was
performed instead.

290

RecoverPoint 5.0 Administrator's Guide

If the event recurs, try to


configure remote storage for
better performance.

Events

Table 61 Normal scope events (continued)

ID

Topic

Level

Summary

More Details

4400

RPA
Cluster

Warning
off

Writing rate to the


production journal is no
longer slow

None

4401

RPA

Warning
off

RPA handles the


incoming data rate
successfully

None

4402

Group

Warning
off

Journal handles
incoming data rate
successfully

None

4403

Group

Warning
off

Journal and replication


volumes at copy handle
incoming data rate
successfully

None

4404

Group

Warning
off

Remote storage is able to None


handle incoming data
rate and no longer
regulates distribution

4405

Group

Warning
off

Remote cluster handles


incoming data rate
successfully

None

4406

RPA
Cluster

Warning
off

Data transfer rate


between clusters is no
longer slow

None

4407

RPA
Cluster

Warning
off

Reading rate from local


None
replication volumes
during resynchronization
is normal

4408

RPA
Cluster

Warning
off

Reading rate from


replication volumes at
copy during
resynchronization is
normal

None

4409

RPA

Warning
off

RPA utilization is back to


normal

None

4410

Group

Warning
off

Link utilization is back to None


normal

4412

Group

Error off

Able to roll to physical


image again

None

5013

Splitter

Error

Splitter is down

None

5014

Splitter

Info

Writes to replication
volumes disabled, reads
are allowed

None

5015

Splitter

Error off

Splitter is up

None

User Action

Normal Events

291

Events

Table 61 Normal scope events (continued)

ID

Topic

Level

Summary

More Details

5016

Splitter

Warning

Splitter has restarted

None

5017

Splitter

Error off

Splitter version is
supported

None

5018

Splitter

Error

Splitter version is not


supported

None

5019

Splitter

Warning

Writes to replication
volumes disabled, reads
are allowed

None

5020

Splitter

Error

Writes to replication
volumes disabled, reads
are allowed

None

5029

Splitter

Error

LUN mismatch detected

Volume sweep will follow.

5030

Splitter

Error

Possible splitter write


failure

None

5031

Splitter

Warning

Splitter not splitting to


replication volumes

Volume sweep(s) required.

5032

Splitter

Info

Splitter splitting to
replication volumes

None

5035

Splitter

Info

Writes to replication
volumes disabled

None

5036

Splitter

Warning

Writes to replication
volumes disabled

None

5037

Splitter

Error

Writes to replication
volumes disabled

None

5038

Splitter

Info

Splitter delaying writes

None

5039

Splitter

Warning

Splitter delaying writes

None

5040

Splitter

Error

Splitter delaying writes

None

5041

Splitter

Info

Splitter not splitting to


replication volumes

None

5042

Splitter

Warning

Splitter not splitting to


replication volumes

None

5043

Splitter

Error

Splitter not splitting to


replication volumes

None

5045

Group

Warning

Splitter and RPA problem Simultaneous problems reported in


detected
splitter and RPA. Full-sweep
resynchronization will be required
upon restarting data transfer.

5046

Splitter

Warning

Splitter write re-issued to RPA did not acknowledge an I/O in


RPA
a timely manner. As a result, the
splitter has re-issued the write.

292

RecoverPoint 5.0 Administrator's Guide

User Action

From the relevant hosts, rescan


the SAN.

Following this event, it is


recommended to:
- Monitor subsequent events, to

Events

Table 61 Normal scope events (continued)

ID

Topic

Level

Summary

More Details

User Action
determine if the problem recurs.
- If the problem persists, or
increases in frequency, contact
EMC Customer Support .

5052

Splitter

Error

Splitter does not support Group was successfully configured


synchronous replication to replicate synchronously but
since then the splitter has been
replaced. The new splitter does not
support synchronous replication
and consequentially replication
mode has been changed to
asynchronous.

Replace your current splitter


with one that supports
synchronous replication. If
problem persists contact EMC
Customer Support.

5053

Group

Error off

Splitter now supports


synchronous replication

Group is configured to replicate


synchronously and the splitter now
supports synchronous replication.

5059

Splitter

Info

Array roll started

An array roll has been triggered.


RecoverPoint transfer will be
paused until roll is completed.

No user action is required.

5060

Splitter

Info

VPLEX inter-cluster link


recovery started

VPLEX inter-cluster link recovery


started, RecoverPoint transfer will
be paused until recovery is
completed.

No user action is required.

5061

Splitter

Info

VNX roll started

Failover started at another RP


system attached to VPLEX, no
snapshots can be created by
RecoverPoint.

No user action is required.

7111

RPA
Cluster

Error

Volumes switched on
storage

System has detected that volumes


were switched in the storage
configuration without following
proper procedures.

7112

Splitter

Error

Volumes switched on
storage

System has detected that volumes


were switched in the storage
configuration without following
proper procedures.

9999

RPA

Info

Test event

None

18036

RPA

Error

Negotiation of transfer
protocol failed again

Negotiation of transfer protocol


failed due to network issues in TCP
layer between clusters. Problem
was temporarily corrected, but has
since returned.

If problem persists, check your


TCP network.

Normal Events

293

Events

Detailed events
The following table contains the Detailed scope events that are issued by the
RecoverPoint system.
Table 62 Detailed scope events

ID

Topic

Level

Summary

More Details

1000

Mgmt

Info

User logged in

None

1001

Mgmt

Warning

User login failed

None

1003

Mgmt

Warning

Failed to generate SNMP


trap

None

1004

Mgmt

Warning

Failed to send email alert Failed to send email alert to the


specified address.

1005

Mgmt

Warning

Failed to update local file None

1006

Mgmt

Info

Settings changed

None

1007

Mgmt

Info

Settings change failed

None

1008

Mgmt

Info

User action succeeded

None

1009

Mgmt

Info

User action failed

None

1011

Mgmt

Error

License grace period


expired

You must obtain a new


RecoverPoint license and then
activate each license and enter your
licenses into RecoverPoint.

1014

Mgmt

Info

User bookmarked an
image

None

1017

Mgmt

Warning

RPA to host multipathing None


problem

1018

Mgmt

Warning
off

RPA multipathing
problem fixed

None

1019

Mgmt

Warning

User action performed


successfully

None

1020

Mgmt

Warning

Failed to send system


report

None

294

RecoverPoint 5.0 Administrator's Guide

User Action

Check whether there is an issue


with your selected method of
transfer. If a problem exists, fix it
and then, perform a connectivity
test by running the
test_syr_connectivity CLI
command. If problem persists
contact EMC Customer Support.

Check whether there is an issue


with your selected method of
transfer. If a problem exists, fix it
and then, perform a connectivity
test by running the
test_syr_connectivity CLI
command. If problem persists
contact EMC Customer Support.

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

1031

Mgmt

Warning

System limit exceeded


low watermark

None

1032

Mgmt

Info

System limit passed


below low watermark

None

1033

Mgmt

Warning

System limit exceeded


high watermark

None

1034

Mgmt

Info

System limit passed


below high watermark

None

1035

Mgmt

Error

System limit exceeded

None

1036

Mgmt

Info

System limit passed


below limit

None

3000

RPA

Warning
off

RPA successfully
communicating with
cluster

None

3001

RPA

Warning

RPA problem
communicating with
cluster

RPA is currently experiencing a


problem communicating with its
cluster.

3005

RPA
Cluster

Error

Settings conflict between A settings conflict has been


clusters
discovered between two RPA
clusters.

3006

RPA
Cluster

Error off

Settings conflict between The settings conflict between RPA


clusters resolved
clusters has been resolved by a
user action.

3008

RPA

Warning

RPA appears to be down

None

3030

RPA

Warning

RPA switched path to


storage

None

3036

RPA

Error

RPA clock out of sync

None

3037

RPA

Error off

RPA clock is back in sync None

3038

RPA
Cluster

Info

Director automatically
added

None

3040

RPA
Cluster

Info

RPA cluster added

None

3041

RPA
Cluster

Info

RPA cluster removed

None

3157

Group

Error

Snap based replication


stopped

Array resources usage exceeded


maximum threshold

3158

Group

Error off

Snap based replication


resumed

Array resources usage is back to


normal

3159

RPA
Cluster

Error

System was unable to


None
remove snap interface(s)
from the array

User Action

Contact EMC Customer Support.

Clear array resources

Remove the snap interface(s)


manually

Detailed events

295

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

3160

Group

Error

System was unable to


remove a consistency
group from the array

None

Remove the array consistency


group manually

3161

Group

Error

System was unable to


None
remove consistency snap
from array

Remove array consistency snap


manually

3162

Group

Error

System was unable to


promote consistency
snap on array

None

Remove the array consistency


group manually

3163

Group

Error

Array operation took too


long

None

Array may be loaded. Increase snap


based replication interval.

4056

Group

Error

No image in journal
matches query

None

4078

Group

Info

Replication set resized

None

4082

Group

Warning

Next synchronization will The copy marked all of its


be a full sweep
replication volumes as dirty.

4083

Group

Warning

Next synchronization will The copy marked some of its


be a volume sweep
replication volumes as dirty. Next
synchronization will be a volume
sweep on these volumes.

4090

Group

Warning

Image access log 80%


full

296

RecoverPoint 5.0 Administrator's Guide

Image access is enabled to a nonproduction copy.


- The capacity of the copy journals
image access log has nearly been
reached; only limited further
information can be written to it. If
the log becomes completely full,
the system will block access by
host applications to the replication
volumes that belong to that group
(and the application will receive I/O
errors).
- When the system is in this state, it
is recommended that immediate
action be taken to prevent the log
from becoming 100% full.

If you do not wish to retain the


writes that were added to the copy
image while image access was
enabled, disable image access; the
relevant writes are automatically
undone. Alternatively, use the undo
writes command to undo the same
writes; image access remains
enabled.
- If, however, you wish to retain the
writes that were added to the copy
image while image access has been
enabled, you can do one of the
following:
- - Increase the size of the copy
journal by adding one or more
volumes to the journal.
- - Fail over to the copy.
- Note: The copy journal is erased.
- - Set the image access mode to
direct.
- Note: The copy journal is erased,
and a full sweep resynchronization
will be required to restore
consistency between the
production and copy images.
- To help prevent this problem from

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action
recurring, increase the size of your
image access log, using one or both
of the following:
- - Add volumes to the journal.
- - Modify the policy setting for the
portion of the journal allocated to
the image access log. This is
permitted only when image access
is disabled.

4106

Group

Warning

Starting full sweep

Cannot write additional replication


data for this group to volume
because its capacity has been
reached. Starting full sweep.

4107

Group

Info

Replication integrity
check successful

None

4108

Group

Error

Replication integrity
issue detected

A possible replication integrity


issue has been detected.

Contact EMC Support

4109

Group

Error

Replication integrity
check stopped before it
finished

Replication integrity check stopped


before it finished because transfer
has been paused, and no
inconsistency has been found.

Check your system events to better


understand the root cause of the
pause in transfer. Run the
replication integrity check again
after restarting transfer.

4117

Group

Warning

Virtual writes buffer is


90% full

When virtual writes buffer becomes


full virtual access is disabled.

4121

Group

Error

Error writing to volume

Ensure that RecoverPoint has write


permission to all volumes. System
will retry automatically. If the
problem recurs repeatedly,
possible causes may be: volume
contains bad sector; volume is thin
device not attached to any storage
pool; volume is attached to storage
pool that is full.

4123

Group

Error

Read and write to volume Read from and write to volume


failed
failed. System retried and failed
again. System will retry
automatically. If the problem recurs
repeatedly, contact EMC Customer
Support.

4125

Group

Error

Journal capacity
insufficient

Journal capacity is currently


insufficient for the required
protection window.

4126

Group

Error off

Journal capacity
sufficient

Journal capacity is currently


sufficient for the required
protection window.

If you extended a storage pool or


attached a device to a storage pool,
run the CLI command
rescan_san If problem persists,
contact EMC Customer Support.

Detailed events

297

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

4127

Group

Warning

Predicted journal
capacity insufficient

Journal capacity is predicted to be


insufficient for the required
protection window.

4128

Group

Warning
off

Predicted journal
capacity sufficient

None

4131

Group

Warning

Transfer paused or
synchronizing for
unusually long time

None

4132

Group

Warning
off

Transfer resumed

Transfer has resumed (following


long pause or synchronization)

4133

Group

Error

Starting copy regulation

The system has detected an


unstable state in this copy and
therefore the copy is being
regulated for a period of <x> minutes
in order to allow for the
environment to stabilize. See
Details for cause of regulation.
Event logs can also be a good
source for the detection of system
issues. Look for repetitive errors,
that may indicate a specific
problem in the system. Also, please
check SAN/IP events outside of
RecoverPoint, as instabilities may
not originate from RecoverPoint. If
regulation persists, collect all
system information, and contact
EMC Customer Support

4134

Group

Error off

Copy regulation has


ended

Copy regulation has ended due to a


user action or internal timeout.

4139

Splitter

Warning

Storage unreachable

The configured login credentials are Ensure that there is connectivity


incorrect or the storage is
between the RPA cluster and the
unreachable.
storage, and that the entered
credentials are correct, and try
again.

4158

Group

Warning

Journal capacity
extended

Journal capacity is currently


extended for the required
protection window

4163

Group

Error

Snap-based replication
has been disabled

The storage cannot perform the


operations necessary for snapbased replication. Snap-based
replication has been paused for a
period of %
%t_SC_ArrayOperationsFreezeInter
val%% minutes in order to allow
you to fix any issues with the
storage.

298

RecoverPoint 5.0 Administrator's Guide

User Action

See the More Information area for a


description of the exact problem.
After fixing the problem, run the
clear_all_storage_errors
CLI command to restart snap-based
replication. If problem persists,
collect system information, and
contact EMC Customer Support.

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

4164

Group

Error off

Snap-based replication
has been restored

Snap-based replication error has


ended due to user action or internal
system timeout.

4165

Group

Error

Possible integrity issue


detected

A possible replication integrity


issue has been detected when
using the periodic integrity check.

4166

Group

Error

Replication integrity
check failed to run

Replication integrity check failed to Wait for the current integrity check
run because another integrity check to end, or stop the current process.
is already running on the same RPA.

4167

Group

Error

Replication integrity
check stopped before it
finished

Replication integrity check stopped


before it finished, and no
inconsistency has been found.

Run the replication integrity check


again.

4175

RPA

Info

System has entered


maintenance mode

The RecoverPoint system is adding


an RPA to the cluster.

No action required.

4176

RPA
Cluster

Info

System has entered


maintenance mode

The RecoverPoint system is


performing a major version
upgrade.

No action required.

4177

RPA
Cluster

Info

System has entered


maintenance mode

The RecoverPoint system is


performing a minor version
upgrade.

No action required.

4178

RPA

Info

System has entered


maintenance mode

The RecoverPoint system is


performing an RPA replacement.

No action required.

4179

RPA

Info

System has exited


maintenance mode

The RecoverPoint system is no


No action required.
longer adding an RPA to the cluster.

4180

RPA
Cluster

Info

System has exited


maintenance mode

The RecoverPoint system is no


longer performing a major version
upgrade.

No action required.

4181

RPA
Cluster

Info

System has exited


maintenance mode

The RecoverPoint system is no


longer performing a minor version
upgrade.

No action required.

4182

RPA

Info

System has exited


maintenance mode

The RecoverPoint system is no


longer performing an RPA
replacement.

No action required.

4183

RPA

Info

System has entered


maintenance mode

The RecoverPoint system is


performing an RPA conversion from
class to RPSE.

No action required.

4184

RPA

Info

System has entered


maintenance mode

The RecoverPoint system is no


longer performing an RPA
conversion from class to RPSE.

No action required.

4185

RPA
Cluster

Info

System has entered


maintenance mode

System has entered user-initiated


maintenance mode

No action required.

Use the set_markers CLI


command to trigger a full sweep. If
problem persists, contact EMC
Customer Support.

Detailed events

299

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

4186

RPA
Cluster

Info

System has exited


maintenance mode

System has exited user-initiated


maintenance mode

No action required.

4230

RPA
Cluster

Info

All initiators registered


with storage

All RPAs initiators were successfully


registered with the storage system.

4231

RPA

Info

RPA initiators registered


with storage

RPA initiators were successfully


registered with the storage system.

4232

RPA
Cluster

Info

Storage group created on Storage group was successfully


storage
created on the storage system

4233

RPA
Cluster

Info

All storage groups


created on storage

All storage groups were


successfully created on the storage
system

4311

Group

Info

Load balancing
recommendation

None

4501

Mgmt

Warning

License invalid in
RecoverPoint

You have installed a


RecoverPoint/SE license although
the system was installed with the
Deployment Manager RecoverPoint
Installer Wizard.

To benefit from RecoverPoint/SE


simplifications and the Unisphere
RecoverPoint/SE plug-in, you must
perform one of the following
actions:
- - If using a release earlier than 3.4
SP1: 1) upgrade the system to
release 3.4 SP1 or later using the
Deployment Manager Upgrade RPAs
Wizard, 2) convert the environment
to RecoverPoint/SE using the
Deployment Manager Converter
Wizard.
- - If using release 3.4 SP1 or later,
convert the environment to
RecoverPoint/SE using the
Deployment Manager Converter
Wizard.

4502

Mgmt

Warning

License invalid in
RecoverPoint/SE

You have installed a RecoverPoint


license although the system was
installed with the Deployment
Manager RecoverPoint/SE Installer
Wizard.

To benefit from full RecoverPoint


functionality, such as bandwidth
reduction and support for
additional capacity, you must
perform the following actions:
- - If using a release earlier than 3.4
SP1: 1) upgrade the system to
release 3.4 SP1 or later using the
Deployment Manager Upgrade RPAs
Wizard, 2) convert the environment
to RecoverPoint using the
Deployment Manager Converter
Wizard.
- - If using release 3.4 SP1 or later:
1) convert the environment to
RecoverPoint using the Deployment

300

RecoverPoint 5.0 Administrator's Guide

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action
Manager Converter Wizard, 2)
upgrade the existing
RecoverPoint/SE license to a
RecoverPoint license.

5008

Splitter

Warning

Host shut down

None

5010

Splitter

Warning

Splitter stopped

Depending on the policy, writing by


host may be disabled for some
groups, and a full sweep may be
required for other groups.

5011

Splitter

Info

Splitter stopped

Full sweep will be required.

5012

Splitter

Warning

Splitter stopped

Writes to replication volumes


disabled

5050

Splitter

Warning

Failed to collect system


info

None

5051

Splitter

Warning

Unregistered array or
vCenter Server found

No login credentials have been


defined for this array or vCenter
Server. Define login credentials to
extend the period in which system
info is saved, from three days to
thirty days. It is recommended that
you do so as soon as possible.

5054

Group

Warning

Consistency group is
configured with a LUN
greater than 2 TB and a
VNX/CLARiiON splitter
version that does not
support the LUN.

None

5055

Group

Warning
off

Consistency group was


configured with a LUN
greater than 2 TB and a
VNX/CLARiiON splitter
version that did not
support the LUN. Now
the splitter version
supports the LUN.

None

5056

Splitter

Warning

SCSI errors detected in


SANTap based
replication short
synchronization might
occur.

None

5057

Group

Warning

Volume configuration not Consistency group contains a


supported by VNX/
replication set with volumes that
CLARiiON splitter version are not equal in size, and this is not
supported by the VNX/CLARiiON

Verify that all arrays and vCenter


Severs have been registered with
the correct credentials.

No action required.

Reconfigure the replication set so


that all of its volumes are of the
same size. Note: Modifying the
volumes in a replication set causes

Detailed events

301

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

splitter version currently installed


on the system.

a volume sweep on all of the


volumes in the replication set.

5058

Group

Warning
off

Volume configuration
Replication set volume
supported by VNX/
configuration is now supported by
CLARiiON splitter version the VNX/CLARiiON splitter version
installed on the system.

6000

Group

Error

Image cannot be
accessed

An unrecognised error has


occurred. The specified image of
your distributed group cannot be
accessed.

Try accessing a different image. If


you cannot access any other
images, contact EMC Customer
Service.

6001

Group

Error off

Stopped trying to access


inaccessible image

The system has stopped trying to


access an inaccessible image of a
distributed group.

No user action is required.

8000

Mgmt

Error

Calling home

None

A service request is being opened


for you with EMC Customer Service.

8001

Mgmt

Error

Call home event failure

None

Contact EMC Customer Support.

8002

RPA
Cluster

Info

VM created

None

8003

RPA
Cluster

Info

VM network reconfigured None

8004

RPA
Cluster

Info

Private network created


on ESX

None

8005

RPA
Cluster

Info

Private network on ESX


erased

None

8101

RPA
Cluster

Info

Array provider registered

None

8102

RPA
Cluster

Info

Array provider
unregistered

None

8103

RPA
Cluster

Info

Storage was registered

None

8104

RPA
Cluster

Info

Storage was
unregistered

None

8105

RPA
Cluster

Warning

Lost connectivity to
storage

None

8106

RPA
Cluster

Info

Restored connectivity to
storage

None

8107

RPA
Cluster

Info

Resource pool was


registered

None

302

RecoverPoint 5.0 Administrator's Guide

Ensure that there is connectivity


with the storage and that the
entered credentials are correct, and
try again. If problem persists,
contact EMC Customer Support.

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

8108

RPA
Cluster

Info

Resource pool was


unregistered

None

8200

Group

Info

Consistency group was


created on an array

None

8201

Group

Info

Snap-based replication
is done

None

8202

Group

Info

Bookmark has been


applied

None

8203

Group

Error

Failed to create snap on


storage

None

8204

Group

Error

Failed to replicate snap


on time

None

8205

Group

Error

Failed to attach snapshot None


to snap interface

9998

RPA

Info

Cannot identify event

None

10203

RPA

Error off

RPA cluster was down

None

10206

RPA

Brief
error

Internal process
restarted

None

10207

RPA

Brief
error

Internal process
restarted

None

10210

RPA

Error

High load occurring


during group
initialization

High load situation is ongoing.

Run detect_bottlenecks for


the consistency group in which the
high load condition has been
detected.Consider enabling fast
first-time initialization (for details,
see RecoverPoint Administrators
Guide).

10211

RPA

Error

SAN communication
problem between
splitters and RPAs
existed but has resolved
itself

None

Temporary issue on the SAN caused


communication problems between
splitters and RPAs at the specified
cluster. The problem resolved itself.

10212

RPA

Error off

Fibre Channel is back up

Temporary problem in Fibre


Channel link between splitters and
RPAs has been resolved.

10213

RPA

Error off

High load occurring


during group
initialization

High load situation is ongoing.

Run detect_bottlenecks for


the consistency group in which the
high load condition has been
detected. Consider enabling fast
first-time initialization (for details,
see RecoverPoint Administrators
Guide).

Detailed events

303

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

10216

RPA

Brief
error

Internal process
restarted

None

10501

RPA

Info

Synchronization
complete

None

10502

RPA

Info

Access to image enabled Access to an image at the copy has


been enabled.

10503

RPA

Error

Transferring latest
snapshot

Transferring latest snapshot before


pausing transfer (no data loss)

10504

RPA

Info

Access to image
disabled

Access to an image at the copy has


been disabled.

10505

RPA

Info

Image access log writes


have been undone

None

10506

RPA

Info

Roll to physical image


complete

Logged access to physical image


now enabled.

10507

RPA

Info

Journal available again

Due to system changes the journal


was temporarily out of service, but
it is now available again.

10508

RPA

Info

Data flushed from source All data has been flushed from the
RPA
source RPA. Failover will proceed.

10509

RPA

Info

Initial long initialization


complete

10510

RPA

Info

Following a pause
None
transfer system now
cleared to restart transfer

10511

RPA

Info

Finished recovering
replication backlog

10512

RPA

Info

Following a pause
None
transfer system now
cleared to restart transfer

12000

RPA

Error

Stable root cause

None

12001

RPA

Error

Splitter is down

None

12002

RPA

Error

Error in links to all other


clusters

None

304

RecoverPoint 5.0 Administrator's Guide

User Action

None

None

- Check if other cluster is down.


- - In case of IP connectivity:
- o Check RecoverPoint gateways
and IPs configuration.
- o Check if there are any ISPrelated issues.
- o Check RecoverPoint firewall
configuration.
- - In case of FC connectivity, check
RecoverPoint zoning.
- - If you recently changed the
network configuration, please verify

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action
that RecoverPoint configuration
matches the new network
configuration.
- - If none of the above instructions
resolve this issue, collect system
information from all clusters
experiencing connectivity issues,
and contact EMC Customer Service.
-

12003

RPA

Error

Error in link to an RPA at


other cluster(s)

None

12004

RPA

Error

Error in data link to other Error in data link. All RPAs at cluster
cluster(s)
are unable to transfer replicated
data to other cluster(s) due to
communication failure over the
WAN link. Data transfer between
RPA clusters has stopped.

1. Check if replication process at


other cluster is down.
- 2. In case of IP connectivity:
- a. check RP gateways/IPs
configuration.
- b. Check firewall configuration.
- 3. If you recently changed network
configuration, please verify that RP
configuration matches the new
network configuration.
-

12005

RPA

Error

Error in data link over


WAN. RPA is unable to
transfer replicated data
to other cluster

None

1. Check if replication process at


other cluster is down.
- 2. In case of IP connectivity:
- a. check RP gateways/IPs
configuration.
- b. Check firewall configuration.
- 3. If you recently changed network
configuration, please verify that RP
configuration matches the new
network configuration.
-

12006

RPA

Error

RPA is disconnected
from the RPA cluster

None

12007

RPA

Error

All RPAs are


disconnected from the
RPA cluster

None

12008

RPA

Error

RPA is down

None

12009

RPA

Error

Link entered high load

Due to heavy I/O activity a link


entered a high load state in order to
prevent I/O failures on that link.
The following are among the
possible causes of the high load:RPA is unable to handle the large
volume of incoming data. (RPA
performance statistics are

If high load persists, consider


running the balance_load
command and applying the load
balancing recommendation, or
manually modifying the preferred
RPA of each group according to the
recommendation. Check for
scheduled activities in your

Detailed events

305

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

presented in the Release Notes that


accompany each RecoverPoint
product release.) - Journal reaches
capacity, because the rate of the
distribution process consistently
lags behind the rate of incoming
data to the copy journal.
- - WAN is too slow to handle the
data rate. - Compression for WAN
optimization is too high, such that
the RPA is unable to handle the
volume of incoming data. Peak I/O
activity in the SAN causes a
temporary bottleneck in the
environment. This can be
considered normal behavior, and
does not necessarily require user
action.

environment. When schedule


permits, run
detect_bottlenecks for the
consistency group in which the high
load condition was detected. If
relevant, consider enabling fast
first-time initialization. (For details,
see the RecoverPoint
Administrators Guide).

12010

RPA

Error

Journal error -- full sweep None


to be performed after
error is corrected

12011

RPA

Error

Image access log or


virtual buffer is full

306

RecoverPoint 5.0 Administrator's Guide

Access to copy volumes for this


group at this RPA cluster has been
disabled.

If you do not wish to retain the


writes that were added to the copy
image while image access was
enabled, disable image access; the
relevant writes are automatically
undone. Alternatively, use the undo
writes command to undo the same
writes; image access remains
enabled.
- If, however, you wish to retain the
writes that were added to the copy
image while image access has been
enabled, you must increase the size
of the copy journal by adding one or
more volumes to the journal.
Alternatively, you can set the image
access mode to direct; however, in
that case, the copy journal is
erased, and a full sweep
resynchronization will be required
to restore consistency between the
production and copy images.
- To help prevent this problem from
recurring, increase the size of your
image access log, using one or both
of the following:
- - Add volumes to the journal.
- - Modify the policy setting for the
portion of the journal allocated to

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action
the image access log. This is
permitted only when image access
is disabled.

12012

RPA

Error

Cannot enable virtual


access

None

12013

RPA

Error

Cannot enable image


access

Cannot enable image access to the


specified image.

12014

RPA

Error

Fibre Channel is down

Fibre Channel link between all RPAs


and all splitters and storage is
down.

12016

RPA

Error

Fibre Channel is down

Fibre Channel link between all RPAs


and all storage is down.

12022

RPA

Error

Fibre Channel is down

Fibre Channel link between RPA and


splitters or storage volumes (or
both) is down.

12023

RPA

Error

Fibre Channel is down

Fibre Channel link between RPA and


all splitters and storage is down.

12024

RPA

Error

Fibre Channel is down

Fibre Channel link between RPA and


all splitters is down.

12025

RPA

Error

Fibre Channel is down

Fibre Channel link between RPA and


all storage is down.

12026

RPA

Error

Error occurred in link to


None
an RPA at other cluster(s)

12027

RPA

Error

RPAs are unable to


access any volume at
this cluster that belongs
to this consistency group

RPAs cannot complete I/O


operations at this cluster to any
volume -- replication or journal -- in
the consistency group.

12029

RPA

Error

Fibre Channel is down

Fibre Channel link between all RPAs


and one or more replication
volumes is down.

12030

RPA

Error

Fibre Channel is down

Fibre Channel link between all RPAs


and one or more journal volumes is
down.

12031

RPA

Error

Error occurred in link


between one or more
RPAs at cluster and all
RPAs at other cluster(s)

None

Suggestions: Check your SAN and


storage configurations for any
irregularities; for example, incorrect
LUN masking or zoning. Ensure that
the Reservations Support policy
parameter setting (enabled/not
enabled) is appropriate for your
environment, as described in the
RecoverPoint Administrators
Guide.

Ensure that at RPA(s) cluster are


connected properly to the other
cluster(s).

Detailed events

307

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

12032

RPA
Cluster

Error

Error occurred in link


between all RPAs at
cluster and all RPAs at
other cluster(s)

None

Ensure that all RPAs at cluster are


connected properly to the other
cluster(s)

12033

RPA

Error

volume is not accessible; None


repository data may be
lost

12034

RPA

Error

Writes to storage may


have occurred without
corresponding writes to
RPA

None

12035

RPA

Error

Cluster control at other


cluster(s) unavailable

Error occurred in link to cluster


leader RPA at other cluster(s).

12036

RPA

Error

Negotiation of transfer
protocol failed

Negotiation of transfer protocol


failed due to network issues in TCP
layer between clusters

12037

RPA

Error

All replication volumes in None


the consistency group (or
groups) are not
accessible

12038

RPA

Error

All journal volumes in


None
the consistency group (or
groups) are not
accessible

12039

RPA

Warning

Long initialization
started

None

12040

RPA

Error

System detected bad


sectors in volume

None

12041

RPA

Error

Error writing to volume

Ensure that RecoverPoint has write


permission to all volumes. Possible
causes: volume is read-only;
volume contains bad sector;
volume is thin device not attached
to any storage pool; volume is
attached to storage pool that is full.

12043

RPA

Error

Splitter writes may have


failed

None

12044

RPA

Error

Problem with IP link


None
between RPAs (in at least
in one direction)

12045

RPA

Error

Problem with all IP links


between RPAs

None

12046

RPA

Error

Problem with IP link


between RPAs

None

308

RecoverPoint 5.0 Administrator's Guide

If problem persists, check your TCP


network.

If you extended a storage pool or


attached a device to a storage pool,
run the rescan_san CLI
command. If problem cannot be
solved, contact EMC Customer
Support.

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

12047

RPA

Error

RPA network interface


card (NIC) problem

None

12048

RPA

Error

Unsupported splitter
version

None

12049

RPA

Info

RPA has entered


maintenance mode

None

12050

RPA

Error

RecoverPoint has
None
dynamically started
replicating
asynchronously to one of
the replicas of this
group. The group will
now be initialized.
During initialization data
is not transferred
synchronously

12057

RPA
Cluster

Info

Replication volume
reservation conflict

This is OK for disabled groups.

12058

RPA

Error

RPA to splitter
communication error

None

12059

RPA

Error

RPA communication
problem with volumes

RPA has encountered a


communication problem with all
volumes

12060

RPA

Error

RPA communication
problem with journal
volumes

None

12061

RPA

Error

RPA communication
problem with replication
volumes

None

12062

RPA

Error

RPA communication
problem with replication
volume

None

12063

RPA

Error

RPA communication
problem with journal
volume

None

12072

RPA

Error

Fibre Channel is down

FC link between RPAs down.

12073

RPA

Error

RPA communication
None
problems with repository
volume

12074

RPA

Error

FC link between RPAs


and repository volume
down

User Action

None

Detailed events

309

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

12075

RPA
Cluster

Error

Splitter errors while


controlling volume

None

12076

RPA
Cluster

Error

Splitter errors while


controlling group

None

12077

RPA
Cluster

Error

Splitter errors while


controlling storage

None

12078

RPA

Error

Splitter communication
error

Although the RPA sees the splitter,


the cluster control has been unable
to communicate with it for the past
five minutes, and therefore, cannot
detect whether or not any writes
have passed through it.
- To ensure consistency between
the copy and production, transfer
will be paused. When the system
re-starts transfer, an initialization or
full sweep will follow.

12080

RPA

Error

Communications
problem in internal
process

The cause of this problem may be:


Failure of a RecoverPoint process Disconnection between internal
RecoverPoint processes, To
determine the cause of this
problem, collect logs for the period
when the event occurred, and
contact EMC Customer Support.

12081

RPA

Error

Communications
problem in internal
process

The cause of this problem may be:


Failure of a RecoverPoint process Disconnection between internal
RecoverPoint processes, To
determine the cause of this
problem, collect logs for the period
when the event occurred, and
contact EMC Customer Support.

12082

RPA

Error off

Communications
problem in internal
process

The cause of this problem may be:


Failure of a RecoverPoint process Disconnection between internal
RecoverPoint processes, To
determine the cause of this
problem, collect logs for the period
when the event occurred, and
contact EMC Customer Support.

12083

RPA

Error

Communications with
Communications with Symmetrix
Symmetrix director failed director has failed repeatedly.

12085

Mgmt

Info

User logged in

310

RecoverPoint 5.0 Administrator's Guide

None

User Action

To ensure data integrity, it is highly


recommended that any snapshots
created in the past five minutes are
not selected for failover.

Check for communication failure,


port failure, faulty zoning, or
configuration errors.

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

12086

RPA

Warning

RPA disk drive failure


detected

None

Contact EMC Customer Support.

12087

RPA

Warning

RPA fan failure detected

None

Contact EMC Customer Support.

12088

Array

Error

RPA has lost connection


to specific array

None

None

12089

Array

Error

The RPA cluster has lost


connection to all
volumes and splitters on
the array

None

None

12090

Site

Error

The RPA cluster has lost


connection to specific
host

None

None

12091

Site

Error

ESX is down

None

None

12092

Group

Error

RPA has lost connection


to the VM

None

None

12093

Group

Error

RPA cluster has lost


connection to the VM

None

None

12094

Group

Error

VM is powered down

None

None

12095

Site

Error

RPA has lost connection


to data store

None

None

12096

Site

Error

RPA cluster has lost


connection to data store

None

None

12097

Site

Error

RPA cluster has lost


connection to all data
stores

None

None

12098

Site

Error

RPA has lost connection


to all data stores

None

None

14001

RPA

Error off

Splitter is up and version None


is supported

14002

RPA

Error off

All links to all other


clusters have been
restored

None

14003

RPA

Error off

Link to an RPA at other


cluster(s) has been
restored

None

14004

RPA

Error off

Data link has been


restored. All RPAs at
cluster are able to
transfer replicated data
to other cluster(s).

None

14005

RPA

Error off

Data link over WAN has


been restored. RPA is

None

Detailed events

311

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

able to transfer
replicated data to other
cluster
14006

RPA

Error off

Connection of RPA to the


RPA cluster is restored

14007

RPA

Error off

Connection of all RPAs to None


the RPA cluster is
restored

14008

RPA

Error off

RPA is up

None

14009

RPA

Error off

Link was in high load,


but has now retuned to
normal operation

Due to heavy I/O activity, the link


entered a high load state in order to
prevent I/O failures on that link.
The link has since exited the high
load state, and has returned to
normal operation. The following are
among the possible causes of the
high load:
- 1. RPA is unable to handle the
large volume of incoming data.
(RPA performance statistics are
presented in the Release Notes that
accompany each RecoverPoint
product release.)
- 2. Journal reaches capacity,
because the rate of the distribution
process consistently lags behind
the rate of incoming data to the
copy journal.
- 3. WAN is too slow to handle the
data rate. -Compression for WAN
optimization is too high, such that
the RPA is unable to handle the
volume of incoming data.
- 4. Peak I/O activity in the SAN
causes a temporary bottleneck in
the environment. This can be
considered normal behavior, and
does not necessarily require user
action.

14010

RPA

Error off

Journal error corrected -full sweep required

None

14011

RPA

Error off

Image access log or


virtual buffer no longer
full

None

14012

RPA

Error off

Virtual access to image


enabled

None

312

RecoverPoint 5.0 Administrator's Guide

None

If high load persists, consider


running the balance_load
command and applying the load
balancing recommendation, or
manually modifying the preferred
RPA of each group according to the
recommendation. Check for
scheduled activities in your
environment. When schedule
permits, run
detect_bottlenecks for the
consistency group in which the high
load condition was detected. If
relevant, consider enabling fast
first-time initialization (for details,
see RecoverPoint Administrators
Guide).

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

14013

RPA

Error off

No longer trying to
access a diluted image

None

14014

RPA

Error off

Fibre Channel is back up

Fibre Channel link between all RPAs


and all splitters and storage is
restored.

14016

RPA

Error off

Fibre Channel is back up

Fibre Channel link between all RPAs


and all storage is restored.

14022

RPA

Error off

Fibre Channel is back up

Fibre Channel link that was down


between RPA and splitters is
restored.

14023

RPA

Error off

Fibre Channel is back up

Fibre Channel link between RPA and


all splitters and storage is restored.

14024

RPA

Error off

Fibre Channel is back up

Fibre Channel link between RPA and


all splitters is restored

14025

RPA

Error off

Fibre Channel is back up

Fibre Channel link between RPA and


all storage is restored

14026

RPA

Error off

The link to an RPA at


None
other cluster(s) has been
restored

14027

RPA

Error off

Access to all volumes in None


the consistency group (or
groups) is restored

14029

RPA

Error off

Fibre Channel is back up

Fibre Channel link between all RPAs


and one or more user volumes is
restored.

14030

RPA

Error off

Fibre Channel is back up

Fibre Channel link between all RPAs


and one or more journal volumes is
restored.

14031

RPA

Error off

Link between one or


None
more RPAs at cluster and
all RPAs at the other
cluster(s) has been
restored

14032

RPA
Cluster

Error off

Link between all RPAs at


cluster and all RPAs at
the other cluster(s) has
been restored

None

14033

RPA

Error off

Access to volume
restored

Restored repository data integrity.

14034

RPA

Error off

Replication consistency
to storage restored

Replication consistency in writes to


storage has been restored.

14035

RPA

Error off

Cluster control at other


cluster(s) available

The link to cluster leader RPA at


other cluster(s) has been restored.

User Action

Detailed events

313

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

14036

RPA

Error off

Negotiation of transfer
protocol successful
again

Negotiation of transfer protocol,


which failed to due to network
issues in TCP layer between
clusters, has been restored.

14037

RPA

Error off

Access to all replication


volumes in the
consistency group (or
groups) has been
restored

None

14038

RPA

Error off

Access to all journal


volumes in the
consistency group (or
groups) is restored

None

14039

RPA

Warning
off

Long initialization
completed

None

14040

RPA

Error off

Bad sectors in volume


have been corrected

None

14041

RPA

Error off

Write access to volume


restored

None

14042

RPA

Error off

Synchronization in
progress to restore any
failed writes in group

None

14043

RPA

Error off

Synchronization in
progress to restore any
failed writes

None

14044

RPA

Error off

Problem with IP link


None
between RPAs (in at least
in one direction)
corrected

14045

RPA

Error off

All IP links between RPAs None


restored

14046

RPA

Error off

IP link between RPAs


restored

None

14047

RPA

Error off

RPA network interface


card (NIC) problem
corrected

None

14049

RPA

Info

RPA is out of
maintenance mode

None

14050

RPA

Error off

RecoverPoint has
None
dynamically resumed
synchronous replication
to one of the replicas of
this group. The group will
now be initialized.

314

RecoverPoint 5.0 Administrator's Guide

User Action

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

During initialization data


is not transferred
synchronously
14057

RPA
Cluster

Info

Group volumes
reservation conflicts
were fixed

None

14058

RPA

Error off

End of splitter to RPA


connectivity error

None

14059

RPA

Error off

End of connectivity
problems between all
group volumes and RPA

None

14060

RPA

Error off

End of connectivity
None
problems between group
journal volumes and RPA

14061

RPA

Error off

End of connectivity
None
problems between group
replication volumes and
RPA

14062

RPA

Error off

End of connectivity
None
problems between group
replication volume and
RPA

14063

RPA

Error off

Connectivity problems
with group journal
volume and RPA were
fixed

14072

RPA

Error off

Fibre Channel is up again FC link between RPAs restored

14073

RPA

Error off

Connectivity between
repository volume and
RPA restored

None

14074

RPA

Error off

FC link between all RPAs


and repository volume
restored

None

14075

RPA
Cluster

Error off

Control issues between


volume and splitter
corrected

None

14076

RPA
Cluster

Error off

Control issues between


group and splitter
corrected

None

14077

RPA
Cluster

Error off

Control issues between


storage and splitter
corrected

None

None

Detailed events

315

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

14078

RPA

Error off

End of splitter
communication error

The cluster control is able to


communicate with the splitter
again. To ensure consistency
between the copy and production,
initialization or full sweep will
follow.

No user action is required.

14080

RPA

Error off

Fix of communications
problem in internal
process

None

14081

RPA

Error off

Internal process
communication problem
has been fixed

None

14082

RPA

Error off

Fix of communications
problem in internal
process

None

14083

RPA

Error off

Communication with
Symmetrix director
restored

After a failure, communications


with a Symmetrix director has been
restored successfully.

14086

RPA

Warning
off

RPA disk drive failure


resolved

None

14087

RPA

Warning
off

RPA fan failure resolved

None

14088

Array

Error off

RPA has restored


connection to specific
array

None

None

14089

Array

Error off

The RPA cluster has


restored connection to
all volumes and splitters
on the array

None

None

14090

Site

Error off

The RPA cluster has


restored connection to
specific host

None

None

14091

Site

Error off

ESX is up

None

None

14092

Group

Error off

RPA has restored


connection to the VM

None

None

14093

Group

Error off

RPA cluster has restored


connection to the VM

None

None

14094

Group

Error off

VM is powered up

None

None

14095

Site

Error off

RPA has restored


connection to data store

None

None

14096

Site

Error off

RPA cluster has restored


connection to data store

None

None

316

RecoverPoint 5.0 Administrator's Guide

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

14097

Site

Error off

RPA cluster has restored


connection to all data
stores

None

None

14098

Site

Error off

RPA has lost connection


to all data stores

None

None

16000

RPA

Brief
error

Transient root cause

None

16001

RPA

Brief
error

Brief splitter error


corrected. Problem has
been corrected.

Splitter was down. Problem has


been corrected.

16002

RPA

Brief
error

Brief error in links to all


other clusters corrected

Error occurred in all links to all


other clusters. Problem has been
corrected.

16003

RPA

Brief
error

Brief error in link to an


RPA at other cluster(s)
corrected

Error occurred in link to an RPA at


other cluster(s). Problem has been
corrected.

16004

RPA

Brief
error

Brief error in data link to Error in data link. All RPAs at cluster
other cluster(s) corrected are unable to transfer replicated
data to other cluster(s). Problem
has been corrected.

16005

RPA

Brief
error

Error occurred in data


link over WAN. RPA is
unable to transfer
replicated data to other
cluster. Problem has
been corrected

None

16006

RPA

Brief
error

RPA was disconnected


from the RPA cluster.
Connection has been
restored

None

16007

RPA

Brief
error

All RPAs were


None
disconnected from the
RPA cluster. Problem has
been corrected

16008

RPA

Brief
error

RPA was down. Problem


has been corrected

16009

RPA

Brief
error

Link was in high load but Due to heavy I/O activity, a link
has now returned to
entered a high load state in order to
normal operation
prevent I/O failures on that link.
The link has since exited the high
load state, and has returned to
normal operation. The following are
among the possible causes of the
high load: RPA is unable to handle
the large volume of incoming data.

None
If high load persists, consider
running the balance_load
command and applying the load
balancing recommendation, or
manually modifying the preferred
RPA of each group according to the
recommendation. Check for
scheduled activities in your
environment. When schedule
Detailed events

317

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

(RPA performance statistics are


presented in the Release Notes that
accompany each RecoverPoint
product release.) Journal reaches
capacity, because the rate of the
distribution process consistently
lags behind the rate of incoming
data to the copy journal. WAN is too
slow to handle the data rate.
Compression for WAN optimization
is too high, such that the RPA is
unable to handle the volume of
incoming data. Peak I/O activity in
the SAN causes a temporary
bottleneck in the environment. This
can be considered normal behavior,
and does not necessarily require
user action.

permits, run
detect_bottlenecks for the
consistency group in which the high
load condition was detected. If
relevant, consider enabling fast
first-time initialization (for details,
see RecoverPoint Administrators
Guide).

16010

RPA

Brief
error

Journal error occurred.


Problem has been
corrected -- full sweep
required

None

16011

RPA

Brief
error

Briefly image access log


or virtual buffer was full

Problem has been corrected.

16012

RPA

Brief
error

Briefly unable to enable


virtual access to image

Problem has been corrected.

16013

RPA

Brief
error

Was unable to enable


access to specified
image. Problem has
been corrected

None

16014

RPA

Brief
error

Fibre Channel is back up


again

Fibre Channel link between all RPAs


and all splitters and storage was
down. Problem has been corrected.

16016

RPA

Brief
error

Fibre Channel is back up


again

Fibre Channel link between all RPAs


and all storage was down. Problem
has been corrected.

16022

RPA

Brief
error

Fibre Channel is back up


again

Fibre Channel link between RPA and


splitters was briefly down. Problem
has been corrected.

16023

RPA

Brief
error

Fibre Channel is back up


again

None

16024

RPA

Brief
error

Fibre Channel is back up


again

Fibre Channel link between RPA and


all splitters was down. Problem has
been corrected.

318

RecoverPoint 5.0 Administrator's Guide

Fibre Channel link between RPA and


all splitters and storage was down.
Problem has been corrected.

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

16025

RPA

Brief
error

Fibre Channel is back up


again

Fibre Channel link between RPA and


all storage was down. Problem has
been corrected.

16026

RPA

Brief
error

Brief RPA and all storage


error corrected

Error occurred in link to an RPA at


other cluster(s). Problem has been
corrected.

16027

RPA

Brief
error

Brief group(s) volumes


accessibility error
corrected

All volumes in the consistency


group (or groups) were not
accessible. Problem has been
corrected.

16029

RPA

Brief
error

Fibre Channel is back up


again

Fibre Channel link between all RPAs


and one or more replication
volumes was down. Problem has
been corrected.

16030

RPA

Brief
error

Fibre Channel is back up


again

Fibre Channel link between all RPAs


and one or more journal volumes
was down. Problem has been
corrected.

16031

RPA

Brief
error

Brief link error between


RPA(s) at cluster and all
RPAs at other cluster(s)
corrected

Error occurred in link between one


or more RPAs at cluster and all RPAs
at the other cluster(s). Problem has
been corrected.

16032

RPA
Cluster

Brief
error

Brief link error between


all RPAs at cluster and all
RPAs at other cluster(s)
corrected

Error occurred in link between all


RPAs at cluster and all RPAs at
other cluster(s). Problem has been
corrected.

16033

RPA

Brief
error

Brief volume
accessibility error
corrected

volume was not accessible.


Problem has been corrected.

16034

RPA

Brief
error

Brief write consistency


error corrected

Writes to storage may have


occurred without corresponding
writes to RPA. Problem has been
corrected.

16035

RPA

Brief
error

Cluster control at other


Error occurred in link to cluster
cluster(s) available again leader RPA at other cluster(s).
Problem has been corrected.

16036

RPA

Brief
error

Negotiation of transfer
protocol successful
again

16037

RPA

Brief
error

Brief group(s) replication All replication volumes in the


volumes accessibility
consistency group (or groups) were
error corrected
not accessible. Problem has been
corrected.

User Action

Negotiation of transfer protocol


failed due to network issues in TCP
layer between clusters. Problem
has been corrected.

Detailed events

319

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

16038

RPA

Brief
error

Brief group(s) journal


volumes accessibility
error corrected

All journal volumes in the


consistency group (or groups) were
not accessible. Problem has been
corrected.

16039

RPA

Info

System ran long resync

None

16040

RPA

Brief
error

System had detected


bad sectors in volume.
Problem has been
corrected

None

16041

RPA

Brief
error

There was a problem


writing to volume.
Problem has been
corrected

None

16042

RPA

Brief
error

Splitter write may have


failed (while group was
transferring data).
Problem has been
corrected

None

16043

RPA

Brief
error

Splitter writes may have


failed

None

16044

RPA

Brief
error

There was a problem


with an IP link between
RPAs (in at least in one
direction). Problem has
been corrected

None

16045

RPA

Brief
error

There was a problem


with all IP links between
RPAs. Problem has been
corrected

None

16046

RPA

Brief
error

There was a problem


with an IP link between
RPAs. Problem has been
corrected

None

16047

RPA

Brief
error

There was a RPA network None


interface card (NIC)
problem. Problem has
been corrected

16048

RPA

Brief
error

Brief unsupported
splitter version error
corrected

Splitter version was not supported.


Problem has been corrected.

16049

RPA

Info

RPA temporarily entered


maintenance mode but
has since exited

None

16050

RPA

Error

RecoverPoint had
dynamically resumed

None

320

RecoverPoint 5.0 Administrator's Guide

User Action

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

synchronous replication
to one of the replicas of
this group but has since
started replicating
asynchronously again.
Consequentially the
group has been
initialized twice. During
initialization data was
not transferred
synchronously. If this is
not the expected
behavior contact EMC
Customer Support
16057

RPA
Cluster

Info

Brief reservation
conflicts on group
volumes

None

16058

RPA

Brief
error

Brief connectivity
None
problem between splitter
and RPA

16059

RPA

Brief
error

Brief connectivity
problem between all
group volumes and RPA

16060

RPA

Brief
error

Brief connectivity
None
problem between group
journal volumes and RPA

16061

RPA

Brief
error

Brief connectivity
problem between group
replication volumes and
RPA

None

16062

RPA

Brief
error

Brief connectivity
problem between group
replication volume and
RPA

None

16063

RPA

Brief
error

Brief connectivity
problem between group
journal volume and RPA

None

16072

RPA

Brief
error

Fibre Channel is back up


again

Fibre channel link between RPAs


was down but problem has been
corrected and the link is back up
again.

16073

RPA

Brief
error

Brief connectivity
problem between
repository volume and
RPA

None

None

Detailed events

321

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

16074

RPA

Brief
error

FC link between RPAs


and repository volume
restored

None

16075

RPA
Cluster

Brief
error

Brief control issues


between volume and
splitter corrected

None

16076

RPA
Cluster

Brief
error

Brief control issues


between group and
splitter corrected

None

16077

RPA
Cluster

Brief
error

Brief control issues


between storage and
splitter corrected

None

16078

RPA

Brief
error

Brief splitter
communication error

A temporary splitter communication No user action is required.


error occurred, but the problem has
since been corrected. To ensure
consistency between the copy and
production, initialization or full
sweep will follow.

16080

RPA

Brief
error

Internal process
communication problem

The cause of this problem may be:


Failure of a RecoverPoint process Disconnection between internal
RecoverPoint processes.

To determine the cause of this


problem, collect logs for the period
when the event occurred, and
contact EMC Customer Support.

16081

RPA

Brief
error

Internal process
communication problem

The cause of this problem may be:


Failure of a RecoverPoint process Disconnection between internal
RecoverPoint processes.

To determine the cause of this


problem, collect logs for the period
when the event occurred, and
contact EMC Customer Support.

16082

RPA

Brief
error

Internal process
communication problem
briefly resolved

The cause of this problem may be:


Failure of a RecoverPoint process Disconnection between internal
RecoverPoint processes.

To determine the cause of this


problem, collect logs for the period
when the event occurred, and
contact EMC Customer Support.

16083

RPA

Brief
error

Communications with
Symmetrix director
restored again

Communications with a Symmetrix


director failed briefly and has now
been restored

If this recurs frequently, contact


EMC Customer Support.

16086

RPA

Warning
off

Brief RPA power supply


failure resolved

None

Check electrical power to both


power supplies. If problem persists,
contact EMC Customer Support.

16087

RPA

Warning
off

Brief generic hardware


failure resolved

None

Contact EMC Customer Support.

16088

Array

Brief
error

RPA has lost connection


temporarily to specific
array

None

None

16089

Array

Brief
error

The RPA cluster has lost


connection temporarily
to all volumes and
splitters on the array

None

None

322

RecoverPoint 5.0 Administrator's Guide

User Action

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

16090

Site

Brief
error

The RPA cluster has lost


connection temporarily
to specific host

None

None

16091

Site

Brief
error

ESX was down


temporarily

None

None

16092

Group

Brief
error

RPA has lost connection


temporarily to the VM

None

None

16093

Group

Brief
error

RPA cluster has lost


connection temporarily
to the VM

None

None

16094

Group

Brief
error

VM was powered down


temporarily

None

None

16095

Site

Brief
error

RPA has lost connection


temporarily to data store

None

None

16096

Site

Brief
error

RPA cluster has lost


connection temporarily
to data store

None

None

16097

Site

Brief
error

RPA cluster has lost


connection temporarily
to all data stores

None

None

16098

Site

Brief
error

RPA has lost connection


to all data stores
temporarily

None

None

18001

RPA

Error

Splitter problem, which


was considered to have
been corrected, has
reoccurred

None

18002

RPA

Error

Error occurred in all links None


to all other clusters.
Problem was temporarily
corrected, but has since
returned

18003

RPA

Error

Error occurred in link to


an RPA at other
cluster(s). Problem was
temporarily corrected,
but has since returned

None

18004

RPA

Error

Error in data link. All


RPAs at cluster are
unable to transfer
replicated data to other
cluster(s). Problem was
temporarily corrected,
but has since returned

Due to communication failure over


the WAN link, data transfer between
the clusters has stopped.

Detailed events

323

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

18005

RPA

Error

RPA is unable to transfer None


replicated data to other
cluster. Data link was
temporarily restored, but
problem has returned

18006

RPA

Error

Connection of RPA to the


RPA cluster was
temporarily restored but
problem has returned

None

18007

RPA

Error

All RPAs were


temporarily restored to
the RPA cluster but
problem has returned

None

18008

RPA

Error

RPA was temporarily up


but problem has
returned -- RPA is down

None

18009

RPA

Error

Group temporarily exited None


high load but problem
has returned

18010

RPA

Error

Journal error was


None
temporarily corrected but
problem has returned

18011

RPA

Error

Image access log or


None
virtual buffer was
temporarily no longer full
and writing by hosts at
copy was re-enabled -but problem has
returned

18012

RPA

Error

Virtual access to image


was temporarily enabled
but problem has
returned

None

18013

RPA

Error

Access to image was


temporarily enabled but
problem has returned

None

18014

RPA

Error

Fibre Channel is down


again

Fibre Channel link between all RPAs


and all splitters and storage was
temporarily restored but problem
has returned

18016

RPA

Error

Fibre Channel is down


again

Fibre Channel link between all


splitters and all storage was
temporarily restored but problem
has returned.

324

RecoverPoint 5.0 Administrator's Guide

More Details

User Action

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

18022

RPA

Error

Fibre Channel is down


again

Fibre Channel link that was down


between RPA and splitters or
storage volumes (or both) was
temporarily restored but problem
has returned.

18023

RPA

Error

Fibre Channel is down


again

Fibre Channel link between RPA and


all storage was temporarily restored
but problem has returned.

18024

RPA

Error

Fibre Channel is down


again

Fibre Channel link between RPA and


all splitters was temporarily
restored but problem has returned.

18025

RPA

Error

Fibre Channel is back


down again

Fibre Channel link between RPA and


all storage was temporarily restored
but problem has returned.

18026

RPA

Error

Error occurred in link to


an RPA at other
cluster(s). Problem was
temporarily corrected,
but has since returned

None

18027

RPA

Error

Access to all volumes in None


the consistency group (or
groups) was temporarily
restored but problem has
returned

18029

RPA

Error

Fibre Channel is back


down again

Fibre Channel link between all RPAs


and one or more user volumes was
temporarily restored but problem
has returned

18030

RPA

Error

Fibre Channel is back up


again

Fibre Channel link between all RPAs


and one or more journal volumes
was temporarily restored but
problem has returned

18031

RPA

Error

Error occurred in link


between one or more
RPAs at cluster and all
RPAs at the other
cluster(s). Problem was
temporarily corrected,
but has since returned

None

18032

RPA
Cluster

Error

Error occurred in link


between all RPAs at
cluster and all RPAs at
other cluster(s). Problem
was temporarily
corrected, but has since
returned

None

User Action

Detailed events

325

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

18033

RPA

Error

Access to volume was


temporarily restored but
problem has returned

None

18034

RPA

Error

Replication consistency
in writes to storage and
writes to RPAs was
temporarily restored but
problem may have
returned

None

18035

RPA

Error

Cluster control at other


cluster(s) unavailable
again

Error occurred in link to cluster


leader RPA at other cluster(s).
Problem was temporarily corrected,
but has since returned.

18037

RPA

Error

Access to all replication None


volumes in the
consistency group (or
groups) has been
temporarily restored, but
problem has returned

18038

RPA

Error

Access to all journal


None
volumes in the
consistency group (or
groups) was temporarily
restored but problem has
returned

18039

RPA

Warning

Long resync was


completed but has now
restarted

None

18040

RPA

Error

User marked volume as


OK but bad sectors
problem persists

None

18041

RPA

Error

Problem writing to
volume was corrected,
but has recurred

None

18042

RPA

Error

Synchronization had
None
restored any failed writes
in group but problem has
returned

18043

RPA

Error

Internal problem

18044

RPA

Error

Problem with IP link


None
between RPAs (in at least
in one direction) was
corrected but problem
has returned

326

RecoverPoint 5.0 Administrator's Guide

None

User Action

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

18045

RPA

Error

Problem with IP link


None
between RPAs (in at least
in one direction) was
corrected but problem
has returned

18046

RPA

Error

Problem with IP link


between RPAs was
corrected but problem
has returned

None

18047

RPA

Error

RPA network interface


card (NIC) problem was
corrected but problem
has returned

None

18049

RPA

Info

RPA temporarily exited


maintenance mode but
has since re-entered

None

18050

RPA

Error off

RecoverPoint had
dynamically resumed
synchronous replication
to one of the replicas of
this group but has since
started replicating
asynchronously again.
Consequentially the
group has been
initialized twice. During
initialization data was
not transferred
synchronously. If this is
not the expected
behavior contact EMC
Customer Support

None

18057

RPA
Cluster

Info

Reservation conflicts
from group replication
volumes were briefly
fixed

None

18058

RPA

Error

Splitter to RPA
connectivity error has
returned

None

18059

RPA

Error

Connectivity problems
between all group
volumes and RPA were
briefly corrected

None

18060

RPA

Error

Connectivity problems
between group journal
volumes and RPA were
briefly corrected

None

User Action

Detailed events

327

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

18061

RPA

Error

Connectivity problems
between group user
volumes and RPA were
briefly corrected

None

18062

RPA

Error

Connectivity problems
between group user
volume and RPA were
briefly corrected

None

18063

RPA

Error

Connectivity problems
between group journal
volume and RPA were
briefly corrected

None

18072

RPA

Error off

Fibre Channel is down


again

Fibre channel link between RPAs


was temporarily restored but the
problem has returned and the link
is back down again.

18073

RPA

Error

Connectivity between
repository volume and
RPA restored

None

18074

RPA

Error

Fibre Channel is down


again

Fibre Channel link between all RPAs


and repository volume was
temporarily restored but problem
has returned.

18075

RPA
Cluster

Error

Control issues between


volume and splitter
corrected

None

18076

RPA
Cluster

Error

Control issues between


group and splitter
corrected

None

18077

RPA
Cluster

Error

Control issues between


storage and splitter
corrected

None

18078

RPA

Error

Splitter communication
error returned

The cluster control was temporarily


able to communicate with the
splitter, but since then, it has lost
communication with the splitter
again.
- To ensure consistency between
the copy and production, transfer
will be paused. When the system
re-starts transfer, an initialization or
full sweep will follow.

To ensure data integrity, it is highly


recommended that any snapshots
created in the past five minutes are
not selected for failover.

18080

RPA

Error

Internal process
communication problem

The cause of this problem may be:


Failure of a RecoverPoint process Disconnection between internal
RecoverPoint processes.

To determine the cause of this


problem, collect logs for the period
when the event occurred, and
contact EMC Customer Support.

328

RecoverPoint 5.0 Administrator's Guide

User Action

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

18081

RPA

Error

Internal process
communication problem

The cause of this problem may be:


Failure of a RecoverPoint process Disconnection between internal
RecoverPoint processes.

To determine the cause of this


problem, collect logs for the period
when the event occurred, and
contact EMC Customer Support.

18082

RPA

Error

Internal process
communication problem

The cause of this problem may be:


Failure of a RecoverPoint process Disconnection between internal
RecoverPoint processes.

To determine the cause of this


problem, collect logs for the period
when the event occurred, and
contact EMC Customer Support.

18083

RPA

Error

Communication with
After a brief recovery,
Symmetrix director failed communications with a Symmetrix
again
director has failed again.

18086

RPA

Brief
warning

RPA power supply failure None


is back

18087

RPA

Brief
warning

Generic hardware failure


is back again

None

18088

Array

Error

RPA has restored


connection temporarily
to specific array

None

None

18089

Array

Error

The RPA cluster has


restored connection
temporarily to all
volumes and splitters on
the array

None

None

18090

Site

Error

The RPA cluster has lost


connection temporarily
to specific host

None

None

18091

Site

Error

ESX was up temporarily

None

None

18092

Group

Error

RPA has restored


connection to the VM
temporarily

None

None

18093

Group

Error

RPA cluster has restored


connection temporarily
to the VM

None

None

18094

Group

Error

VM was powered up
temporarily

None

None

18095

Site

Error

RPA has restored


connection temporarily
to data store

None

None

18096

Site

Error

RPA cluster has restored


connection temporarily
to data store

None

None

Check for communication failure,


port failure, faulty zoning, or
configuration errors.

Detailed events

329

Events

Table 62 Detailed scope events (continued)

ID

Topic

Level

Summary

More Details

User Action

18097

Site

Error

RPA cluster has restored


connection temporarily
to all data stores

None

None

18098

Site

Error

RPA has restored


connection to all data
stores temporarily

None

None

Advanced events
The following table contains the Normal scope events that are issued by the RecoverPoint
system.
Table 63 Advanced scope events

ID

Topic

Level

Summary

More Details

2004

RPA
Cluster

Info

Repository volume
created

None

2005

RPA
Cluster

Info

Repository volume
removed

None

2006

RPA
Cluster

Info

Splitter(s) added

None

2007

RPA
Cluster

Info

Splitter(s) removed

None

3003

RPA

Warning

Stopping a system
process

None

3004

RPA

Info

Starting a system
process

None

3009

RPA

Warning

A system process
stopped

None

3010

RPA

Warning

A system process
stopped

None

3031

RPA

Warning

An internal process
restarted

None

3032

RPA

Info

An internal process
restarted

None

3034

RPA

Info

Negotiating transfer
protocol

None

3035

RPA

Info

An internal process
restarted

None

4005

Group

Info

Negotiating transfer
protocol

None

330

RecoverPoint 5.0 Administrator's Guide

User Action

Events

Table 63 Advanced scope events (continued)

ID

Topic

Level

Summary

More Details

4013

RPA

Info

Negotiating transfer
protocol

None

4013

RPA

Info

Negotiating transfer
protocol

None

4014

Group

Info

Negotiating transfer
protocol

None

4050

Group

Info

Enabling access to image None

4052

Group

Warning

Enabling access to image None

4072

Group

Info

Journal cleared

None

4073

Group

Warning

Journal cleared

None

4074

Group

Error

Journal cleared

None

4075

Group

Info

Journal available

None

4076

Group

Info

Adding volume(s)

None

4077

Group

Info

Removing volume(s)

None

4092

Group

Info

Accessing image

Writing by hosts to the accessed


image has been enabled.

4093

Group

Info

Undoing writes to image


access log

None

4094

Group

Info

Writes to image access


log have been undone

None

4096

Group

Info

Image access log has


been written to storage

None

4100

RPA
Cluster

Info

Group created

Creating a new group modifies the


load distribution across RPAs.

4101

RPA
Cluster

Info

Group removed

None

4102

Group

Info

Journal volume(s)
removed

None

4103

Group

Info

Journal volume(s) added

None

4104

Group

Info

Group started accepting


writes

None

4105

Group

Info

Group stopped accepting None


writes

5000

Splitter

Info

Volume(s) attached to
splitter(s)

5001

Splitter

Info

Volume(s) detached from None


splitter(s)

User Action

To balance the write load across all


RPAs run the balance_load CLI
command in seven days and apply
the recommendation.

None

Advanced events

331

Events

Table 63 Advanced scope events (continued)

ID

Topic

Level

Summary

More Details

5049

Splitter

Warning

Splitter write to RPA


failed

None

10000

RPA

Info

Analyzing changes in
system

None

10001

RPA

Info

System is stable

System changes have occurred.


System is now stable.

10002

RPA

Info

Issuing intermediate
report

System activity has not stabilized


and therefore, an intermediate
report is being issued.

10101

RPA

Error

Unrecognised system
activity

Cause of system activity unclear.

10102

RPA

Info

Internal changes
occurred

Cluster control recorded internal


changes that do not impact system
operation.

10201

RPA

Info

Settings have changed

None

10202

RPA

Info

System changes at other


cluster

System changes have occurred at


the another cluster.

12042

RPA

Error

Splitter write may have


failed during transfer

Splitter write may have failed (while


group was transferring data).
Synchronization will be required.

332

RecoverPoint 5.0 Administrator's Guide

User Action

To obtain more information filter


events using Detailed scope.

For details of the system changes,


refer to the event log at the other
cluster(s).

Вам также может понравиться