STF-4 Business Continuity PDF

Copyright 2006 EMC Corporation. Do not Copy - All Rights Reserved.
Business Continuity - 1
2006 EMC Corporation. All rights reserved.
Section 4 Business Continuity
Section 4 Business Continuity
Introduction
Welcome to Section 4 of Storage Technology Foundations Business Continuity.
Copyright 2006 EMC Corporation. All rights reserved.
These materials may not be copied without EMC's written consent.
EMC believes the information in this publication is accurate as of its publication date. The
information is subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED AS IS. EMC
CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND
WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY
DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A
PARTICULAR PURPOSE.
Use, copying, and distribution of any EMC software described in this publication requires an
applicable software license.
EMC
2
, EMC, Navisphere, CLARiiON, and Symmetrix are registered trademarks and EMC
Enterprise Storage, The Enterprise Storage Company, The EMC Effect, Connectrix, EDM,
SDMS, SRDF, Timefinder, PowerPath, InfoMover, FarPoint, EMC Enterprise Storage Network,
EMC Enterprise Storage Specialist, EMC Storage Logix, Universal Data Tone, E-Infostructure,
Access Logix, Celerra, SnapView, and MirrorView are trademarks of EMC Corporation.
All other trademarks used herein are the property of their respective owners.
2006 EMC Corporation. All rights reserved. Business Continuity - 2
Section Objectives
Upon completion of this section, you will be able to:
Describe what business continuity is
Describe the basic technologies that are enablers of data
availability
Describe basic disaster recovery techniques
The objectives for this section are shown here. Please take a moment to read them.
In This Section
This section contains the following modules:
Business Continuity Overview
Backup and Recovery
Business Continuity Local Replication
Business Continuity Remote Replication
This section contains the following 4 modules:
Backup and Recovery
Business Continuity Local Replication
Business Continuity Remote Replication.
After completing this module, you will be able to:
Define and differentiate between Business Continuity and
Disaster Recovery
Differentiate between Disaster Recovery and Disaster
Restart
Define terminology such as Recovery Point Objective and
Recovery Time Objective
Give a high level description of Business Continuity
Planning
Identify Single Points of Failure and describe solutions to
eliminate them
The are the objectives for this module. Please take a moment to review them.
What is Business Continuity?
Business Continuity is the preparation for, response to,
and recovery from an application outage that adversely
affects business operations
Business Continuity Solutions address systems
unavailability, degraded application performance, or
unacceptable recovery strategies
Since information is a primary asset for most businesses, business continuity is a major concern.
This is not just a concern for the Information Technology department, it impacts the entire
business. At one time, data storage was viewed as a simple issue. The requirements have
become more sophisticated. Businesses must now contend with information availability, storage
and business continuation in adverse events large or small, man-made or natural. Before we
can talk about business continuity and solutions for business continuity, we must first define the
terms. Business Continuity is the preparation for, response to, and recovery from an application
outage that adversely affects business operations. Business Continuity Solutions address
systems unavailability, degraded application performance, or unacceptable recovery strategies.
Lost Revenue
Know the downtime costs (per
hour, day, two days...)
Number of employees
impacted (x hours out *
hourly rate)
Damaged Reputation
Customers
Suppliers
Financial markets
Banks
Business partners
Financial Performance
Revenue recognition
Cash flow
Lost discounts (A/P)
Payment guarantees
Credit rating
Stock price
Other Expenses
Temporary employees, equipment rental, overtime
costs, extra shipping costs, travel expenses...
Why Business Continuity
Direct loss
Compensatory payments
Lost future revenue
Billing losses
Investment losses
Lost Productivity
There are many factors that need to be considered when calculating the cost of downtime. A
formula to calculate the costs of the outage should capture both the cost of lost productivity of
employees and the cost of lost income from missed sales.
The Estimated average cost of 1 hour of downtime = (Employee costs per hour) *( Number
of employees affected by outage) + (Average Income per hour).
Employee costs per hour is simply the total salaries and benefits of all employees per week,
divided by the average number of working hours per week.
Average income per hour is just the total income of an institution per week, divided by
average number of hours per week that an institution is open for business.
Information Availability
20 min 10 sec 17 hrs 31 min 0.2% 99.8%
1 hr 41 min 3.65 days 1% 99%
10 min 5 sec 8 hrs 45 min 0.1% 99.9%
0.6 sec 31.5 sec 0.0001% 99.9999%
6 sec 5.25 min 0.001% 99.999%
1 min 52.5 min 0.01% 99.99%
3hrs 22 min 7.3 days 2% 98%
Downtime per Week Downtime per Year % Downtime % Uptime
Information Availability ensures that applications and business units have access to information
whenever it is needed. The primary components of information availability are:
Protection from data loss
Ensuring data access
Appropriate data security
The online window for some critical applications has moved to 99.999% of time.
Information availability depends upon robust, functional IT systems.
Importance of Business Continuity and Planning
Millions of US Dollars per Hour in Lost Revenue
6.5
3.6
2.8
2.6
2.0
1.6
1.6
1.5
1.3
1.2
1.1
Retail brokerage
Point of sale
Energy
Credit card sales authorization
Telecommunications
Call location
Manufacturing
Financial institutions
Information technology
Insurance
Retail
Source Meta Group, 2005
This chart shows how much money each industry loses for each hour of downtime. As you can
see, downtime is expensive.
T
a
p
e

B
a
c
k
u
p
P
e
r
i
o
d
i
c

R
e
p
l
i
c
a
t
i
o
n
Recovery Point Objective (RPO)
Wks Days Hrs Mins Secs
Recovery Point Recovery Time
T
a
p
e

B
a
c
k
u
p
P
e
r
i
o
d
i
c

R
e
p
l
i
c
a
t
i
o
n
A
s
y
n
c
h
r
o
n
o
u
s

R
e
p
l
i
c
a
t
i
o
n
A
s
y
n
c
h
r
o
n
o
u
s

R
e
p
l
i
c
a
t
i
o
n
S
y
n
c
h
r
o
n
o
u
s

R
e
p
l
i
c
a
t
i
o
n
S
y
n
c
h
r
o
n
o
u
s

R
e
p
l
i
c
a
t
i
o
n
Secs Mins Hrs Days Wks
Recovery Point Objective (RPO) is the point in time to which systems and data must be
recovered after an outage. This defines the amount of data loss a business can endure. Different
business units within an organization may have varying RPOs.
Recovery Time Objective (RTO)
Recovery Time includes:
Fault detection
Recovering data
Bringing apps back online
G
l
o
b
a
l

C
l
u
s
t
e
r
Wks Days Hrs Mins Secs Secs Mins Hrs Days Wks
G
l
o
b
a
l

C
l
u
s
t
e
r
M
a
n
u
a
l

M
i
g
r
a
t
i
o
n
M
a
n
u
a
l

M
i
g
r
a
t
i
o
n
T
a
p
e

R
e
s
t
o
r
e
T
a
p
e

R
e
s
t
o
r
e
Recovery Time Objective (RTO) is the period of time within which systems, applications, or
functions must be recovered after an outage. This defines the amount of downtime that a
business can endure, and survive.
Disaster Recovery versus Disaster Restart
Most business critical applications have some level of data
interdependencies
Disaster recovery
Restoring previous copy of data and applying logs to that copy to bring it to
a known point of consistency
Generally implies the use of backup technology
Data copied to tape and then shipped off-site
Requires manual intervention during the restore and recovery processes
Disaster restart
Process of restarting mirrored consistent copies of data and applications
Allows restart of all participating DBMS to a common point of consistency
utilizing automated application of recovery logs during DBMS initialization
The restart time is comparable to the length of time required for the
application to restart after a power failure
Disaster recovery is the process of restoring a previous copy of the data and applying logs or
other necessary processes to that copy to bring it to a known point of consistency.
Disaster restart is the restarting of dependent write consistent copies of data and applications,
utilizing the automated application of DBMS recovery logs during DBMS initialization to bring
the data and application to a transactional point of consistency.
There is a fundamental difference between Disaster Recovery and Disaster Restart. Disaster
recovery is the process of restoring a previous copy of the data and applying logs to that copy to
bring it to a known point of consistency. Disaster restart is the restarting of mirrored consistent
copies of data and applications.
Disaster recovery generally implies the use of backup technology in which data is copied to tape
and then it is shipped off-site. When a disaster is declared, the remote site copies are restored
and logs are applied to bring the data to a point of consistency. Once all recoveries are
completed, the data is validated to ensure it is correct.
Disruptors of Data Availability
Disaster (<1% of Occurrences)
Natural or man made
Flood, fire, earthquake
Contaminated building
Unplanned Occurrences (13% of
Occurrences)
Failure
Database corruption
Component failure
Human error
Planned Occurrences (87% of Occurrences)
Competing workloads
Backup, reporting
Data warehouse extracts
Application and data restore
Source: Gartner, Inc.
Elevated demand for increased application availability confirms the need to ensure business
continuity practices are consistent with business needs.
Interruptions are classified as either planned or unplanned. Failure to address these specific
outage categories seriously compromises a companys ability to meet business goals.
Planned downtime is expected and scheduled, but it is still downtime causing data to be
unavailable. Causes of planned downtime include:
New hardware installation/integration/maintenance
Software upgrades/patches
Backups
Application and data restore
Data center disruptions from facility operations (renovations, construction, other)
Refreshing a testing or development environment with production data
Porting testing/development environment over to production environment
Causes of Downtime
Human Error
System Failure
Infrastructure Failure
Disaster
Today, the most critical component of an organization is information. Any disaster occurrence
will affect information availability critical to run normal business operations.
In our definition of disaster, the organizations primary systems, data, applications are damaged
or destroyed. Not all unplanned disruptions constitute a disaster.
Business Continuity vs. Disaster Recovery
Business Continuity has a broad focus on prevention:
Predictive techniques to identify risks
Procedures to maintain business functions
Disaster Recovery focuses on the activities that occur
after an adverse event to return the entity to normal
functioning.
Business Continuity is a holistic approach to planning, preparing, and recovering from an
adverse event. The focus is on prevention, identifying risks, and developing procedures to
ensure the continuity of business function. Disaster recovery planning should be included as
part of business continuity.
BC objectives include:
Facilitate uninterrupted business support despite the occurrence of problems.
Create plans that identify risks and mitigate them wherever possible.
Provide a road map to recover from any event.
Disaster Recovery is more about specific cures, to restore service and damaged assets after an
adverse event. In our context, Disaster Recovery is the coordinated process of restoring systems,
data, and infrastructure required to support key ongoing business operations.
Business Continuity Planning (BCP)
Includes the following activities:
Identifying the mission or critical business functions
Collecting data on current business processes
Assessing, prioritizing, mitigating, and managing risk
Risk Analysis
Business Impact Analysis (BIA)
Designing and developing contingency plans and disaster
recovery plan (DR Plan)
Training, testing, and maintenance
Business Continuity Planning (BCP) is a risk management discipline. It involves the entire
business--not just IT. BCP proactively identifies vulnerabilities and risks, planning in advance
how to prepare for and respond to a business disruption. A business with strong BC practices in
place is better able to continue running the business through the disruption and to return to
business as usual.
BCP actually reduces the risk and costs of an adverse event because the process often uncovers
and mitigates potential problems.
Objectives
Train, Test, and
Document
Implement,
Maintain, and
Assess
Analysis
Design
Develop
Business Continuity Planning Lifecycle
The Business Continuity Planning process includes the following stages:
1. Objectives
Determine business continuity requirements and objectives including scope and budget
Team selection (include all areas of the business and subject matter expertise (internal/external)
Create the project plan
2. Perform analysis
Collect information on data, business processes, infrastructure supports, dependencies, frequency of use
Identify critical needs and assign recovery priorities.
Create a risk analysis (areas of exposure) and mitigation strategies wherever possible.
Create a Business Impact Analysis (BIA)
Create a Cost/benefit analysis identify the cost (per hour/day, etc.) to the business when data is unavailable.
Evaluate Options
3. Design and Develop the BCP/Strategies
Evaluate options
Define roles/responsibilities
Develop contingency scenarios
Develop emergency response procedures
Detail recovery, resumption, and restore procedures
Design data protection strategies and develop infrastructure
Implement risk management/mitigation procedures
4. Train, test, and document
5. Implement, maintain, and assess
Business Impact Analysis (BIA)
$1800
$8000
18000
$55619
$55768
$69517
Loss p/y
$400
$16,000
$16,000
$279,098
$279,066
$279,056
Single Loss
Expectancy
1.0
0.5
1.0
0.2
0.2
.25
# Event
p/y
$5,000
$122,000
$80,000
$10,000
$66,456
$5,800
Est cost of
mitigation
No failover for development
webserver
1 2 IT-
Intranet/B2B
6
Computer room does not
have sufficient UPS
capacity to run on single
unit
3 4 Entire
Company
5
Primary dev platforms dont
have failover
3 4 IT-All 4
Relocate net equip to a
separate physical rack
1 5 Entire
Company
3
Cisco net backbone switch
not redundant
1 5 Entire
Company
2
No redundant UPS for
Networking/phone equip
1 5 Entire
Company
1
High Risk SPOF Item Probability
(1-5)
Impact
(1 -5)
Business Area
Affected
#
This is an example of Business Impact Analysis (BIA). The dollar values are arbitrary and are
used just for illustration. BIA quantifies the impact that an outage will have to the business and
potential costs associated with the interruption. It helps businesses channel their resources based
on probability of failure and associated costs.
User & Application
Clients
IP
Identifying Single Points of Failure
Primary
Node
Consider the components in the picture and identify the Single Points of Failure.
HBA Failures
Configure multiple HBAs, and use
multi-pathing software
Protects against HBA failure
Can provide improved
performance (vendor
dependent)
HBA
HBA
Host
Switch
Storage
Port
Port
HBA
HBA
Configuring multiple HBAs and using multi-pathing software provides path redundancy. Upon
detection of a failed HBA, the software can re-drive the I/O through another available path.
Switch/Storage Array Port Failures
Configure multiple switches
Make the devices available
via multiple storage array
ports
HBA
HBA
Host
Switch
Storage
Port
Port
HBA
HBA
Port
Port
This configuration provides switch redundancy as well as protects against storage array port
failures.
Disk Failures
Use some level of RAID
HBA
HBA
Host
Switch
Storage
Port
Port
HBA
HBA
Port
Port
As seen earlier, using some level of RAID, such as RAID-1 or RAID-5, will ensure continuous
operation in the event of disk failures.
Host Failures
Clustering protects against production host failures
HBA
HBA
Host
Switch
Storage
Port
Port
HBA
HBA
Port
Port
Storage
Host
Planning and configuring clusters is a complex task. At a high level:
A cluster is two or more hosts with access to the same set of storage (array) devices
Simplest configuration is a two node (host) cluster
One of the nodes would be the production server while the other would be configured as a
standby. This configuration is described as Active/Passive.
Participating nodes exchange heart-beats or keep-alives to inform each other about their
health.
In the event of the primary node failure, cluster management software will shift the
production workload to the standby server.
Implementation of the cluster failover process is vendor specific.
A more complex configuration would be to have both the nodes run production workload on
the same set of devices. Either cluster software or application/database should then provide a
locking mechanism so that the nodes do not try to update the same areas on disk
simultaneously. This would be an Active/Active configuration.
Site/Storage Array Failures
Remote replication helps protect against either entire site
or storage array failures
HBA
HBA
Host
Switch
Storage
Port
Port
HBA
HBA
Port
Port
Storage
Remote replication will be explored in-depth in a later module in this section. What is not shown
in the picture is host connectivity to the storage array in the remote site.
User & Application
Clients
IP
Resolving Single Points of Failure
Primary
Node
IP
Redundant
Network
K
e
e
p

A
l
i
v
e
Clustering
Software
Failover
Node
Redundant Paths
Redundant Disks
RAID 1/RAID5
Redundant
Site
This example combines the methods that we have discussed to resolve single points of failure. It
uses clustering, redundant paths and redundant disks, a redundant site, and a redundant network.
Local Replication
Data from the production devices is copied over to a set
of target (replica) devices
After some time, the replica devices will contain identical
data as those on the production devices
Subsequently copying of data can be halted. At this point-
in-time, the replica devices can be used independently of
the production devices
The replicas can then be used for restore operations in
the event of data corruption or other events
Alternatively the data from the replica devices can be
copied to tape. This off-loads the burden of backup from
the production devices
Local replication technologies offer fast and convenient methods for ensuring data availability.
The different technologies and the uses of replicas for BC/DR operations will be discussed in a
later module in this section. Typically local replication uses replica disk devices. This greatly
speeds up the restore process, thus minimizing the RTO. Frequent point-in-time replicas also
help in minimizing RPO.
Backup/Restore
Backup to tape has been the predominant method for
ensuring data availability and business continuity
Low cost, high capacity disk drives are now being used
for backup to disk. This considerably speeds up the
backup and the restore process
Frequency of backup will be dictated by defined
RPO/RTO requirements as well as the rate of change of
data
Far from being antiquated, periodic backup is still a widely used method for preserving copies of
data. In the event of data loss due to corruption or other events, data can be restored up to the
last backup. Evolving technologies now permit faster backups to disks. Magnetic tape drive
speeds and capacities are also continually being enhanced. The various backup paradigms and
the role of backup in B-C/D-R planning will be discussed in detail later in this section.
Module Summary
Key points covered in this module:
Importance of Business Continuity
Types of outages and their impact to businesses
Business Continuity Planning and Disaster Recovery
Definitions of RPO and RTO
Difference between Disaster Recovery and Disaster
Restart
Identifying and eliminating Single Points of Failure
These are the key points covered in this module. Please take a moment to review them.
Backup and Recovery
Upon completion of this module, you will be able to:
Describe best practices for planning Backup and
Recovery.
Describe the common media and types of data that are
part of a Backup and Recovery strategy.
Describe the common Backup and Recovery topologies.
Describe the Backup and Recovery Process.
Describe Management considerations for Backup and
Recovery.
This lesson looks at Backup and Recovery. Backup and Recovery are a major part of the
planning for Business Continuity.
Lesson: Planning for Backup and Recovery
Upon completion of this lesson, you be able to:
Define Backup and Recovery.
Describe common reasons for a Backup and Recovery
plan.
Describe the business considerations for Backup and
Recovery.
Define RPO and RTO.
Describe the data considerations for Backup and
Recovery
Describe the planning for Backup and Recovery.
This lesson provides an overview of the business drivers for backup and recovery and introduces
some of the common terms used when developing a backup and recovery plan.
What is a Backup?
Backup is an additional copy of data that can be used for
restore and recovery purposes.
The Backup copy is used when the primary copy is lost
or corrupted.
This Backup copy can be created as a:
Simple copy (there can be one or more copies)
Mirrored copy (the copy is always updated with whatever is written
to the primary copy.)
A Backup is a copy of the online data that resides on primary storage. The backup copy is
created and retained for the sole purpose of recovering deleted, broken, or corrupted data on the
primary disk.
The backup copy is usually retained over a period of time, depending on the type of the data,
and on the type of backup. There are three derivatives for backup: disaster recovery, Archival,
and operational backup. We will review them in more detail, on the next slide.
The data that is backed up may be on such media as disk or tape, depending on the backup
derivative the customer is targeting. For example, backing up to disk may be more efficient than
tape in operational backup environments.
Backup and Recovery Strategies
Several choices are available to get the data to the backup
media such as:
Copy the data.
Mirror (or snapshot) then copy.
Remote backup.
Copy then duplicate or remote copy.
Several choices are available to get the data written to the backup media.
You can simply copy the data from the primary storage to the secondary storage (disk or
tape), onsite. This is a simple strategy, easily implemented, but impacts the production
server where the data is located, since it will use the servers resources. This may be
tolerated on some applications, but not high demand ones.
To avoid an impact on the production application, and to perform serverless backups, you
can mirror (or snap) a production volume. For example, you can mount it on a separate
server and then copy it to the backup media (disk or tape). This option will completely free
up the production server, with the added infrastructure cost associated with additional
resources.
Remote Backup, can be used to comply with offsite requirements. A copy from the primary
storage is done directly to the backup media that is sitting on another site. The backup media
can be a real library, a virtual library or even a remote filesystem.
You can do a copy to a first set of backup media, which will be kept onsite for operational
restore requirements, and then duplicate it to another set of media for offsite purposes. To
simplify thr procedure, you can replicate it to an offsite location to remove any manual
procedures associated with moving the backup media to another site.
Its All About Recovery!
Businesses back up their data to enable its recovery in
case of potential loss.
Businesses also back up their data to comply with
regulatory requirements.
Types of backup derivatives:
Disaster Recovery
Archival
Operational
There are three different Backup derivatives:
Disaster Recovery addresses the requirement to be able to restore all, or a large part of, an IT
infrastructure in the event of a major disaster.
Archival is a common requirement used to preserve transaction records, email, and other
business work products for regulatory compliance. The regulations could be internal,
governmental, or perhaps derived from specific industry requirements.
Operational is typically the collection of data for the eventual purpose of restoring, at some
point in the future, data that has become lost or corrupted.
Reasons for a Backup Plan
Hardware Failures
Human Factors
Application Failures
Security Breaches
Disasters
Regulatory and Business Requirements
Reasons for a backup plan include:
Physical damage to a storage element (such as a disk) that can result in data loss.
People make mistakes and unhappy employees or external hackers may breach security and
maliciously destroy data.
Software failures can destroy or lose data and viruses can destroy data, impact data integrity,
and halt key operations.
Physical security breaches can destroy equipment that contains data and applications.
Natural disasters and other events such as earthquakes, lightning strikes, floods, tornados,
hurricanes, accidents, chemical spills, and power grid failures can cause not only the loss of
data but also the loss of an entire computer facility. Offsite data storage is often justified to
protect a business from these types of events.
Government regulations may require certain data to be kept for extended timeframes.
Corporations may establish their own extended retention policies for intellectual property to
protect them against litigation. The regulations and business requirements that drive data as
an archive generally require data to be retained at an offsite location.
How does Backup Work?
Client/Server Relationship
Server
Directs Operation
Maintains the Backup Catalog
Client
Gathers Data for Backup (a backup client sends backup data to a
backup server or storage node).
Storage Node
Backup products vary, but they do have some common characteristics. The basic architecture of
a backup system is client-server, with a backup server and some number of backup clients or
agents. The backup server directs the operations and owns the backup catalog (the information
about the backup). The catalog contains the table-of-contents for the data set. It also contains
information about the backup session itself.
The backup server depends on the backup client to gather the data to be backed up. The backup
client can be local or it can reside on another system, presumably to backup the data visible to
that system. A backup server receives backup metadata from backup clients to perform its
activities.
There is another component called a storage node. The storage node is the entity responsible for
writing the data set to the backup device. Typically there is a storage node packaged with the
backup server and the backup device is attached directly to the backup servers host platform.
Storage nodes play an important role in backup planning as it can be used to consolidate backup
servers.
How does Backup Work?
Disk
Storage
Tape
Backup
Data Set
Metadata
Catalog
Backup Server
& Storage Node
Servers
Clients
The following represents a typical Backup process:
The Backup Server initiates the backup process (starts the backup application).
The Backup Server sends a request to a server to send me your data.
The server sends the data to the Backup Server and/or Storage Node.
The Storage Node sends the data to the tape storage device and the Backup Server begins
building the catalog (metadata) of the backup session.
When all of the data has been transferred from the server to the Backup Server, the Backup
Server writes the catalog to a disk file and closes the connection to the tape device.
Business Considerations
Customer business needs determine:
What are the restore requirements RPO & RTO?
Where and when will the restores occur?
What are the most frequent restore requests?
Which data needs to be backed up?
How frequently should data be backed up?
hourly, daily, weekly, monthly
How long will it take to backup?
How many copies to create?
How long to retain backup copies?
Some important decisions that need consideration before implementing a Backup/Restore
solution are shown above. Some examples include:
The Recovery Point Objective (RPO)
The Recovery Time Objective (RTO)
The media type to be used (disk or tape)
Where and when the restore operations will occur especially if an alternative host will be
used to receive the restore data.
When to perform backups.
The granularity of backups Full, Incremental or cumulative.
How long to keep the backup for example, some backups need to be retained for 4 years,
others just for 1 month
Is it necessary to take copies of the backup or not
Data Considerations: File Characteristics
Location
Size
Number
Location:
Many organizations have dozens of heterogeneous platforms that support a complex
application. Consider a data warehouse where data from many sources is fed into the
warehouse. When this scenario is viewed as The Data Warehouse Application, it easily
fits this model. Some of the issues are:
How the backups for subsets of the data are synchronized
How these applications are restored
Size:
Backing up a large amount of data that consists of a few big files may have less system
overhead than backing up a large number of small files. If a file system contains millions of
small files, the very nature of searching the file system structures for changed files can take
hours, since the entire file structure is searched.
Number: a file system containing one million files with a ten-percent daily change rate will
potentially have to create 100,000 entries in the backup catalog. This brings up other issues
such as:
How a massive file system search impacts the system
Search time/Media impact
Is there an impact on tape start/stop processing?
Data Considerations: Data Compression
Compressibility depends on the data type, for example:
Application binaries do not compress well.
Text compresses well.
JPEG/ZIP files are already compressed and expand if
compressed again.
Many backup devices such as tape drives, have built-in hardware compression technologies. To
effectively use these technologies, it is important to understand the characteristics of the data.
Some data, such as application binaries, do not compress well. Text data can compress very
well, while other data, such as JPEG and ZIP files, are already compressed.
Data Considerations: Retention Periods
Operational
Data sets on primary media (disk) up to the point where most restore
requests are satisfied, then moved to secondary storage (tape).
Disaster Recovery
Driven by the organizations disaster recovery policy
Portable media (tapes) sent to an offsite location / vault.
Replicated over to an offsite location (disk).
Backed up directly to the offsite location (disk, tape or emulated tape).
Archiving
Driven by the organizations policy.
Dictated by regulatory requirements.
As mentioned before, there are three types of backup models (Operational, Disaster Recovery,
and Archive). Each can be defined by its retention period. Retention Periods are the length of
time that a particular version of a dataset is available to be restored.
Retention periods are driven by the type of recovery the business is trying to achieve:
For operational restore, data sets could be maintained on a disk primary backup storage
target for a period of time, where most restore requests are likely to be achieved, and then
moved to a secondary backup storage target, such as tape, for long term offsite storage.
For disaster recovery, backups must be done and moved to an offsite location.
For archiving, requirements usually will be driven by the organizations policy and
regulatory conformance requirements. Tapes can be used for some applications, but for
others a more robust and reliable solution, such as disks, may be more appropriate.
Lesson: Summary
Topics in this lesson included:
Backup and Recovery definitions and examples.
Common reasons for Backup and Recovery.
The business considerations for Backup and Recovery.
Recovery Point Objectives and Recovery Time
Objectives.
The data considerations for Backup and Recovery
The planning for Backup and Recovery.
In this lesson we reviewed the business and data considerations when planning for Backup and
Recovery including:
What is a Backup and Recovery?
What is the Backup and Recovery process?
Business recovery needs
RPO Recovery point objectives
RTO Recovery time objectives
Data characteristics
Files, compression, retention
Lesson: Backup and Recovery Methods
Describe Hot and Cold Backups.
Describe the levels of Backup Granularity.
Weve discussed the importance and considerations for a Backup Plan, now this lesson provides
an overview of the different methods for creating a backup set.
Database Backup Methods
Hot Backup: production is not interrupted.
Cold Backup: production is interrupted.
Backup Agents manage the backup of different data
types such as:
Structured (such as databases)
Semi-structured (such as email)
Unstructured (file systems)
Backing up databases can occur useing two different methods:
A Hot backup, which means that the application is still up and running, with users accessing
it, while backup is taking place.
A Cold backup, which means that the application will be shut down for the backup to take
place.
Most backup applications offer various Backup Agents to do these kinds of operations. There
will be different agents for different types of data and applications.
Backup Granularity and Levels
Full Backup
Cumulative (Differential)
Incremental
Full Cumulative Incremental
The granularity and levels for backups depend on business needs, and, to some extent,
technological limitations. Some backup strategies define as many as ten levels of backup. IT
organizations use a combination of these to fulfill their requirements. Most use some
combination of Full, Cumulative, and Incremental backups.
A Full backup is a backup of all data on the target volumes, regardless of any changes made to
the data itself.
An Incremental backup contains the changes since the last backup, of any type, whichever was
most recent.
A Cumulative backup, also known as a Differential backup, is a type of incremental that
contains changes made to a file since the last full backup.
Files 1, 2, 3, 4, 5
Production Production
Restoring an Incremental Backup
Key Features
Files that have changed since the last full or incremental backup are
backed up.
Fewest amount of files to be backed up, therefore faster backup and less
storage space.
Longer restore because last full and all subsequent incremental backups
must be applied.
Incremental Incremental
Tuesday
File 4
Wednesday
File 3
Thursday
File 5 Files 1, 2, 3
Monday
Full Backup Full Backup
The following is an example of an incremental backup and restore:
A full backup of the business data is taken on Monday evening. Each day after that, an
incremental backup is taken. These incremental backups only backup files that are new or that
have changed since the last full or incremental backup.
On Tuesday, a new file is added, File 4. No other files have changed. Since File 4 is a new file
added after the previous backup on Monday evening, it will be backed up Tuesday evening.
On Wednesday, there are no new files added since Tuesday, but File 3 has changed. Since File
3 was changed after the previous evening backup (Tuesday), it will be backed up Wednesday
evening.
On Thursday, no files have changed but a new file has been added, File 5. Since File 5 was
added after the previous evening backup, it will be backed up Thursday evening.
On Friday morning, there is a data corruption, so the data must be restored from tape.
The first step is to restore the full backup from Monday evening. Then, every incremental
backup that was done since the last full backup must be applied, which, in this example,
means the:
Tuesday,
Wednesday, and
Thursday incremental backups.
Restoring a Cumulative Backup
Key Features
More files to be backed up, therefore it takes more time to backup
and uses more storage space.
Much faster restore because only the last full and the last cumulative
backup must be applied.
Files 1, 2, 3, 4, 5, 6
Cumulative Cumulative
Tuesday
File 4 Files 1, 2, 3
Monday
Full Backup Full Backup Cumulative Cumulative
Wednesday
Files 4, 5
Cumulative Cumulative
Thursday
Files 4, 5, 6
The following is an example of cumulative backup and restore:
A full backup of the data is taken on Monday evening. Each day after that, a cumulative backup
is taken. These cumulative backups backup ALL FILES that have changed since the LAST
FULL BACKUP.
On Tuesday, File 4 is added. Since File 4 is a new file that has been added since the last full
backup, it will be backed up Tuesday evening.
On Wednesday, File 5 is added. Now, since both File 4 and File 5 are files that have been added
or changed since the last full backup, both files will be backed up Wednesday evening.
On Thursday, File 6 is added. Again, File 4, File 5, and File 6 are files that have been added or
changed since the last full backup; all three files will be backed up Thursday evening.
On Friday morning, there is a corruption of the data, so the data must be restored from tape.
The first step is to restore the full backup from Monday evening.
Then, only the backup from Thursday evening is restored because it contains all the
new/changed files from Tuesday, Wednesday, and Thursday.
Lesson: Summary
Hot and Cold Backups.
The levels of Backup Granularity.
This lesson provided an introduction to Backup methods and granularity levels, including hot
and cold backups and the levels of backup granularity.
Lesson: Backup Architecture Topologies
Describe DAS, LAN, SAN, Mixed topologies.
Describe backup media considerations.
We have discussed the importance of the Backup plan and the different methods used when
creating a backup set. This lesson provides an overview of the different topologies and media
types that are used to support creating a backup set.
Backup Architecture Topologies
There are 3 basic backup topologies:
Direct Attached Based Backup
LAN Based Backup
SAN Based Backup
These topologies can be integrated, forming a mixed
topology
There are three basic topologies that are used in a backup environment: Direct Attached Based
Backup, LAN Based Backup, and SAN Based Backup.
There is also a fourth topology, called Mixed, which is formed when mixing two or more of
these topologies in a given situation.
Direct Attached Based Backups
Catalog
Backup Server
LAN
Metadata
Media
Backup
Storage Node
Data
Here, the backup data flows directly from the host to be backed up to the tape, without utilizing
the LAN. In this model, there is no centralized management and it is difficult to grow the
environment.
Direct Attached Based Backups are performed directly from the backup clients disk to the
backup clients tape devices. The advantages and disadvantages are outlined here. The key
advantage of direct-attached backups is speed. The tape devices can operate at the speed of the
channels. Direct-attached backups optimize backup and restore speed since the tape devices are
close to the data source and dedicated to the host. Disadvantages are Direct-attached backups
impact the host and application performance since backups consume host I/O bandwidth,
memory, and CPU resources. Direct-attached backups potentially have distance restrictions, if
short-distance connections such as SCSI are used.
LAN Based Backups
Backup Server
LAN
Metadata
Storage Node
Data
Mail Server File Server Database Server
Metadata
Data
In this model, the backup data flows from the host to be backed up to the tape through the LAN.
There is centralized management, but there may be an issue with the LAN utilization since all
data goes through it.
As we have defined previously, Backup Metadata contains information about what has been
backed up, such as file names, time of backup, size, permissions, ownership, and most
importantly, tracking information for rapid location and restore. It also indicates where it has
been stored, for example, which tape. Data, the contents of files, databases, etc., is the primary
information source to be backed up. In a LAN Based Backup, the Backup Server is the central
control point for all backups. The metadata and backup policies reside in the Backup Server.
Storage Nodes control backup devices and are controlled by the Backup Server.
The advantages of LAN Based Backup include the following:
LAN backups enable an organization to centralize backups and pool tape resources.
The centralization and pooling can enable standardization of processes, tools, and backup
media. Centralization of tapes can also improve operational efficiency.
Disadvantages are:
The backup process has an impact on production systems, the client network, and the
applications.
It consumes CPU, I/O bandwidth, LAN bandwidth, and memory.
In order to maintain finite backup points, applications might have to be halted and databases
shut down.
SAN Based Backups (LAN Free)
LAN
Metadata
Storage Node
Data
Mail Server
SAN
Backup Server
Data
A SAN based backup, also known as LAN Free backup, is achieved when there is no backup
data movement over the LAN. In this case, all backup data travels through a SAN to the
destination backup device.
This type of backup still requires network connectivity from the Storage Node to the Backup
Server, since metadata always has to travel through the LAN.
LAN-free backups use Storage Area Networks (SANs) to move backup data rapidly and reliably.
The SAN is usually used in conjunction with backup software that supports tape device sharing.
A SAN-enabled backup infrastructure introduces these advantages to the backup process. It
provides Fibre Channel performance, reliability, and distance. It requires fewer processes and
reduced overhead. It does not use the LAN to move backup data and eliminates or reduces
dedicated backup servers. Finally, it improves backup and restore performance.
SAN/LAN Mixed Based Backups
LAN
Metadata
Storage Node
Data
Mail Server Database Server
Data
SAN
Backup Server
Data
A SAN/LAN Mixed Based Backup environment is achieved by using two or more of the
topologies described in the previous slides. In this example, some servers are SAN based while
others are LAN based.
Backup Media
Tape
Traditional destination for backups
Sequential access
No protection
Disk
Random access
Protected by the storage array (RAID, hot spare, etc)
There are two common types of Backup media, tape and disk.
Multiple Streams on Tape Media
Multiple streams interleaved to achieve higher
throughput on tape
Keeps the tape streaming, for maximum write performance
Helps prevent tape mechanical failure
Greatly increases time to restore
Tape Tape
Data from
Stream 1 Data from
Stream 2
Data from
Stream 3
Tape drive streaming is recommended from all vendors, in order to keep the drive busy. If you
do not keep the drive busy during the backup process (writing), performance will suffer.
Multiple streaming helps to improve performance drastically, but it generates one issue as well:
the backup data becomes interleaved, and thus the recovery times are increased.
Backup to Disk
Backup to disk minimizes tape in backup environments
by using disk as the primary destination device
Cost benefits
No processes changes needed
Better service levels
Backup to disk aligns backup strategy to RTO and
RPO
Backup to disk replaces tape and its associated devices, as the primary target for backup, with
disk. Backup to disk systems offer major advantages over equivalent scale tape systems, in
terms of capital costs, operating costs, support costs, and quality of service. It can be
implemented fully on day 1 or over a phased approach.
While no changes are needed, any number of enhancements to the process, and the services
provided, are now possible. Backup to disk can be a great enabler. Instead of having tape
technology drive the business processes, the business goals drive the backup strategy.
Tape versus Disk Restore Comparison
Typical Scenario:
800 users, 75 MB mailbox
60 GB database
Source: EMC Engineering and EMC IT
*Total time from point of failure to return of service to e-mail users
56
0 10 20 30 40 50 60 70 80 90 100 120 110
Recovery Time in Minutes*
Tape
Backup / Restore
Disk
Backup / Restore
108
Minutes
108
Minutes
24
Minutes
24
Minutes
This example shows a typical recovery scenario using tape and disk. As you can see, recovery
with disk provides much faster recovery than does recovery with tape.
This example shows a typical recovery scenario using tape and disk. As you can see, recovery
with disk provides much faster recovery than recovery with tape.
Keep in mind that this example involves data recovery only. The time it takes to bring the
application online is a separate matter. Even so, you can see in this example that the benefit was
a restore roughly five times faster than it would have gone with tape. What you dont see is the
mitigated risk of media failure, and time saved in not having to locate and load the correct tapes
before being able to begin the recovery process.
Three Backup / Restore Solutions based on RTO
Time of last image dictates
the log playback time
Larger data sets extend the
recovery time (ATA and tape)
*Total time from point of failure to return of service to e-mail users
0 10 20 30 40 50 60 70 80 90 100 120 110
Recovery Time in Minutes*
Backup on tape
Backup on ATA
108 Min. 108 Min.
24 Min. 24 Min.
Typical Scenario:
800 users, 75 MB mailbox
60 GB DB restore time
500 MB logs log playback
130
BCV / Clone
2 Min.
41 Minutes
19 Minutes
125 Minutes
17 Min.
17 Min.
17 Min.
Restore time
Log playback
The diagram shows typical recovery scenarios using different technical solutions. As you can
see recovery with Business Continuance Volumes (BCVs) clones provides the quickest recovery
method.
It is important to note that using BCV or clones on Disk, enables you to be able to make more
copies of your data more often. This will improve RPO (the point from which they can recover).
It will also improve RTO because the log files will be smaller and that will reduce the log
playback time.
Traditional Backup, Recovery and Archive Approach
Production environment grows
Requires constant tuning and data placement to
maintain performance
Need to add more tier-1 storage
Backup environment grows
Backup windows get longer and jobs do not complete
Restores take longer
Requires more tape drives and silos to keep up with
service levels
Archive environment grows
Impact flexibility to retrieve content when requested
Requires more media, adding management cost
No investment protection for long term retention
requirements
Backup
Process
Backup
Process
Archive
Process
Archive
Process
In a traditional approach for backup and archive, businesses take a backup of production.
Typically backup jobs use weekly full backups and nightly incremental backups. Based on
business requirements, they will then copy the backup jobs and eject the tapes to have them sent
offsite, where they will be stored for a specified amount of time.
The problem with this approach is simple - as the production environment grows, so does the
backup environment.
Differences Between Backup / Recovery & Archive
Data typically maintained for
analysis, value generation, or
compliance
Data typically overwritten on
periodic basis (e.g., monthly)
Useful for compliance and should
take into account information-
retention policy
Not for regulatory compliance
though some are forced to use
Typically long-term (months, years,
or decades)
Typically short-term (weeks or
months)
Adds operational efficiencies by
moving fixed / unstructured content
out of operational environment
Improves availability by enabling
application to be restored to a
specific point in time
Available for information retrieval Used for recovery operations
Primary copy of information A secondary copy of information
Archive Backup / Recovery
Backup/Recovery and Archiving support different business and goals. This slide compares and
contrasts some of the differences that are significant.
New Architecture for Backup, Recovery & Archive
OUnderstand the environment
OActively archive valuable information to tiered
storage
OBack up active production information to disk
ORetrieve from archive or recover from backup
Backup
Process
Backup
Process
Archive
Process
Archive
Process
1
3
4
2
4
The recovery process is much more important than the backup process. It is based on the
appropriate recovery-point objectives (RPOs) and recovery-time objectives (RTOs). The process
usually drives a decision to have a combination of technologies in place, from online Business
Continuance Volumes (BCVs), to backup to disk, to backup to tape for long-term, passive
RPOs.
Archive processes are determined not only by the required retention times, but also by retrieval-
time service levels and the availability requirements of the information in the archive.
For both processes, a combination of hardware and software is needed to deliver the appropriate
service level. The best way to discover the appropriate service level is to classify the data and
align the business applications with it.
Lesson: Summary
The DAS, LAN, SAN, and Mixed topologies.
Backup media considerations.
This lesson provided an overview of the different topologies and media types that support
creating a backup set.
Lesson: Managing the Backup Process
Describe features and functions of common
Backup/Recovery applications.
Describe the Backup/Recovery process management
considerations.
Describe the importance of the information found in
Backup Reports and in the Backup Catalog.
We have discussed the planning and operations of creating a Backup. This lesson provides an
overview of Management activities and applications that help manage the Backup and Recovery
process.
How a Typical Backup Application Works
Backup clients are grouped and associated with a Backup
schedule that determines when and which backup type will
occur.
Groups are associated with Pools, which determine which
backup media will be used.
Each backup media has a unique label.
Information about the backup is written to the Backup Catalog
during and after it completes. The Catalog shows:
when the Backup was performed, and
which media was used (label).
Errors and other information is also written to a log.
The process for using a Backup application includes the following:
Backup clients are grouped and associated with a Backup schedule that determines when and
which backup type will occur.
Groups are associated with Pools, which determine which backup media will be used. Each
backup media has a unique label.
Information about the backup is written to the Backup Catalog during and after it completes.
The Catalog shows when the Backup was performed, and which media was used (label).
Errors and other information are also written to a log.
Backup Application User Interfaces
There are typically two types of user interfaces:
Command Line Interface CLI
Graphical User Interfaces GUI
There are typically two types of user interfaces. With Command Line Interface, CLI, backup
administrators usually write scripts to automate common tasks, such as sending reports via email.
Graphical User Interfaces, GUI, controls the backup and restore process, multiple backup
servers, multiple storage nodes, and multiple platforms/operating systems. It is a single and
easy to use interface that provides the most common (if not all) administrative tasks.
Managing the Backup and Restore Process
Running the B/R Application: Backup
The backup administrator configures it to be started, most (if not all)
of the times, automatically
Most backup products offer the ability for the backup client to initiate
their own backup (usually disabled)
Running the B/R Application: Restore
There is usually a separate GUI to manage the restore process
Information is pulled from the backup catalog when the user is
selecting the files to be restored
Once the selection is finished, the backup server starts reading from
the required backup media, and the files are sent to the backup
client
There are common tasks associated with managing a Backup or Restore activity using the B/R
Application. These include backup and restore. In backup, it configures a backup to be started,
most (if not all) of the times, automatically, and enables the backup client to initiate its own
backup (Note: usually this feature is disabled).
In restore, there is usually a separate GUI to manage the restore process. Information is pulled
from the backup catalog when the user is selecting the files to be restored. Once the selection is
finished, the backup server starts reading from the required backup media, and the files are sent
to the backup client.
Backup Reports
Backup products also offer reporting features.
These features rely on the backup catalog and log files.
Reports are meant to be easy to read and provide
important information such as:
Amount of data backed up
Number of completed backups
Number of incomplete backups (failed)
Types of errors that may have occurred
Additional reports may be available, depending on the
backup software product used.
Backup products also offer reporting features. These features rely on the backup catalog and log
files. Reports are meant to be easy to read and provide important information such as amount of
data backed up, number of completed backups, number of incomplete backups (failed), and
types of errors that may have occurred. Additional reports may be available, depending on the
backup software product used.
Importance of the Backup Catalog
As you can see, backup operations strongly rely on the
backup catalog
If the catalog is lost, the backup software alone has no
means to determine where to find a specific file backed
up two months ago, for example
It can be reconstructed, but this usually means that all of
the backup media (i.e. tapes) have to be read
Its a good practice to protect the catalog
By replicating the file system where it resides to a remote location
By backing it up
Some backup products have built-in mechanisms to
protect their catalog (such as automatic backup)
As you can see, backup operations strongly rely on the backup catalog. If the catalog is lost, the
backup software alone has no means to determine where to find a specific file backed up in the
past. It can be reconstructed, but this usually means that all of the backup media (i.e. tapes) has
to be read. Its a good practice to protect the catalog by replicating the file system where it
resides, to a remote location, and by backing it up. Some backup products have built-in
mechanisms to protect their catalog (such as automatic backup).
Lesson: Summary
The features and functions of common Backup/Recovery
applications.
The Backup/Recovery process management
considerations.
The importance of the information found in Backup
Reports and in the Backup Catalog.
This lesson provided an overview of Backup and Recovery management activities and tools.
Module Summary
The best practices for planning Backup and Recovery.
The common media and types of data that are part of a
Backup and Recovery strategy.
The common Backup and Recovery topologies.
The Backup and Recovery Process.
Management considerations for Backup and Recovery.
Local Replication
After completing this module you will be able to:
Discuss replicas and the possible uses of replicas
Explain consistency considerations when replicating file
systems and databases
Discuss host and array based replication technologies
Functionality
Differences
Considerations
Selecting the appropriate technology
In this section, we will look at what replication is, technologies used for creating local replicas,
and things that need to be considered when creating replicas.
What is Replication?
Replica - An exact copy (in all details)
Replication - The process of reproducing data
Original Replica
REPLICATION REPLICATION
Local replication is a technique for ensuring Business Continuity by making exact copies of
data. With replication, data on the replica will be identical to the data on the original at the
point-in-time that the replica was created.
Examples:
Copy a specific file
Copy all the data used by a database application
Copy all the data in a UNIX Volume Group (including underlying logical volumes, file
systems, etc.)
Copy data on a storage array to a remote storage array
Possible Uses of Replicas
Alternate source for backup
Source for fast recovery
Decision support
Testing platform
Migration
Replicas can be used to address a number of Business Continuity functions:
Provide an alternate source for backup to alleviate the impact on production.
Provide a source for fast recovery to facilitate faster RPO and RTO.
Decision Support activities such as reporting.
For example, a company may have a requirement to generate periodic reports. Running
the reports off of the replicas greatly reduces the burden placed on the production
volumes. Typically reports would need to be generated once a day or once a week, etc.
Developing and testing proposed changes to an application or an operating environment.
For example, the application can be run on an alternate server using the replica volumes
and any proposed design changes can be tested.
Data migration.
Migration can be as simple as moving applications from one server to the next, or as
complicated as migrating entire data centers from one location to another.
Considerations
What makes a replica good?
Recoverability
Considerations for resuming operations with primary
Consistency/re-startability
How is this achieved by various technologies
Kinds of Replicas
Point-in-Time (PIT) = finite RPO
Continuous = zero RPO
How does the choice of replication technology tie back
into RPO/RTO?
Key factors to consider with replicas:
What makes a replica good:
Recoverability from a failure on the production volumes. The replication technology
must allow for the restoration of data from the replicas to the production and then allow
production to resume with a minimal RPO an RTO.
Consistency/re-startability is very important if data on the replicas will be accessed
directly or if the replicas will be used for restore operations.
Replicas can either be Point-in-Time (PIT) or continuous:
Point-in-Time (PIT) - the data on the replica is an identical image of the production at
some specific timestamp
For example, a replica of a file system is created at 4:00 PM on Monday. This replica
would then be referred to as the Monday 4:00 PM Point-in-Time copy.
Note: The RPO will be a finite value with any PIT. The RPO will map to the time when the PIT
was created to the time when any kind of failure on the production occurred. If there is a failure
on the production at 8:00 PM and there is a 4:00 PM PIT available, the RPO would be 4 hours (8
4 = 4). To minimize RPO with PITs, take periodic PITs.
Continuous replica - the data on the replica is synchronized with the production data at
all times.
The objective with any continuous replication is to reduce the RPO to zero.
Replication of File Systems
Host
Apps
Volume Management
DBMS Mgmt Utilities
File System
Multi-pathing Software
Device Drivers
HBA HBA HBA
Operating System
Physical Volume
Buffer
Most OS file systems buffer data in the host before the data is written to the disk on which the
file system resides.
For data consistency on the replica, the host buffers must be flushed prior to the creation of
the PIT. If the host buffers are not flushed, the data on the replica will not contain the
information that was buffered on the host.
Some level of recovery will be necessary
Note: If the file system is unmounted prior to the creation of the PIT no recovery would be
needed when accessing data on the replica.
A database application may be spread out over
numerous files, file systems, and devicesall of which
must be replicated
Database replication can be offline or online
Replication of Database Applications
Logs Data
Database replication can be offline or online:
Offline replication takes place when the database and the application are shutdown.
Online replication takes place when the database and the application are running.
Database: Understanding Consistency
Databases/Applications maintain integrity by following the
Dependent Write I/O Principle
Dependent Write: A write I/O that will not be issued by an application
until a prior related write I/O has completed
A logical dependency, not a time dependency
Inherent in all Database Management Systems (DBMS)
e.g. Page (data) write is dependent write I/O based on a successful log
write
Applications can also use this technology
Necessary for protection against local outages
Power failures create a dependent write consistent image
A Restart transforms the dependent write consistent to transactionally
consistent
+i.e. Committed transactions will be recovered, in-flight transactions will be
discarded
All logging database management systems use the concept of dependent write I/Os to maintain
integrity. This is the definition of dependent write consistency. Dependent write consistency is
required for the protection against local power outages, loss of local channel connectivity, or
storage devices. The logical dependency between I/Os is built into database management
systems, certain applications, and operating systems.
Database Replication: Transactions
Data
Log
Database
Application
4 4
3 3
2 2
1 1
Buffer
Database applications require that for a transaction to be deemed complete a series of writes
have to occur in a particular order (Dependent Write I/O), these writes would be recorded on the
various devices/file systems.
In this example, steps 1-4 must complete for the transaction to be deemed complete.
Step 4 is dependent on Step 3 and will occur only if Step 3 is complete
Step 3 is dependent on Step 2 will occur only if Step 2 is complete
Step 2 is dependent on Step 1 will occur only if Step 1 is complete
Steps 1-4 are written to the databases buffer and then to the physical disks.
Database Replication: Consistency
Data
Log
Source
Replica
Consistent
4 4
3 3
2 2
1 1
Log
Data
Note: In this example, the database is online.
At the point in time when the replica is created, all the writes to the source devices must be
captured on the replica devices to ensure data consistency on the replica.
In this example, steps 1-4 on the source devices must be captured on the replica devices for
the data on the replicas to be consistent.
Database Replication: Consistency
Data
Log
Source
Replica
Inconsistent
Note: In this example, the database is online.
4 4
3 3
2
1
Creating a PIT for multiple devices happens quickly, but not instantaneously.
Steps 1-4 which are dependent write I/Os have occurred and have been recorded successfully
on the source devices
It is possible that steps 3 and 4 were copied to the replica devices, while steps 1 and 2 were
not copied.
In this case, the data on the replica is inconsistent with the data on the source. If a restart
were to be performed on the replica devices, Step 4 which is available on the replica might
indicate that a particular transaction is complete, but all the data associated with the
transaction will be unavailable on the replica making the replica inconsistent.
Database
Application
(Offline)
Database Replication: Ensuring Consistency
Data
Log
Source
Replica
Consistent
Off-line Replication
If the database is offline or
shutdown and then a replica is
created, the replica will be
consistent
In many cases, creating an offline
replica may not be a viable due to
the 24x7 nature of business
Database replication can be performed with the application offline (i.e., application is shutdown,
no I/O activity) or online (i.e., while the application is up and running). If the application is
offline, the replica will be consistent because there is no activity. However, consistency is an
issue if the database application is replicated while it is up and running.
Online Replication
Some database applications allow
replication while the application is up
and running
The production database would have to
be put in a state which would allow it to
be replicated while it is active
Some level of recovery must be
performed on the replica to make the
replica consistent
Data
Log
Source
Replica
Inconsistent
4 4
3 3
2
1
In the situation shown, Steps 1-4 are dependent write I/Os. The replica is inconsistent because
Steps 1 & 2 never made it to the replica. To make the database consistent, some level of
recovery would have to be performed. In this example, this could be done by simply discarding
the transaction that was represented by Steps 1-4. Many databases are capable of performing
such recovery tasks.
5
Source
Replica
Consistent
4 4
3 3
2 2
1 1
5
An alternative way to ensure that an online replica is consistent is to:
Hold I/O to all the devices at the same instant.
Create the replica.
Release the I/O.
Holding I/O is similar to a power failure and most databases have the ability to restart from a
power failure.
Note: While holding I/O simultaneously one ensures that the data on the replica is identical to
that on the source devices, the database application will timeout if I/O is held for too long.
Tracking Changes After PIT Creation
At PIT
Source = Target
Later
Source Target
Resynch
Source = Target
Changes will occur on the production volume after the creation of a PIT, changes could also
occur on the target. Typically the target device will be re-synchronized with the source device at
some future time in order to obtain a more recent PIT.
Note: The replication technology employed should have a mechanism to keep track of changes.
This makes the re-synchronization process will be much faster. If the replication technology
does not track changes between the source and target, every resynchronization operation will
have to be a full operation.
Local Replication Technologies
Host based
Logical Volume Manager (LVM) based mirroring
File System Snapshots
Storage Array based
Full volume mirroring
Full volume: Copy on First Access
Pointer based: Copy on First Write
Replication technologies can classified by:
Distance over which replication is performed - local or remote
Where the replication is performed - host or array based
Host based - all the replication is performed by using the CPU resources of the host
using software that is running on the host.
Array based - all replication is performed on the storage array using CPU resources on
the array via the arrays operating environment.
Note: In the context of this discussion, local replication refers to replication that is performed
within a data center if it is host based and within a storage array if it is array based.
Logical Volume Manager: Review
Physical Storage
Logical Storage
LVM
Host resident software responsible for
creating and controlling host level logical
storage
Physical view of storage is converted to a
logical view by mapping. Logical data
blocks are mapped to physical data blocks.
Logical layer resides between the physical
layer (physical devices and device drivers)
and the application layer (OS and
applications see logical view of storage).
Usually offered as part of the operating
system or as third party host software
LVM Components:
Physical Volumes
Volume Groups
Logical Volumes
Logical Volume Managers (LVMs) introduce a logical layer between the operating system and
the physical storage. LVMs have the ability to define logical storage structures that can span
multiple physical devices. The logical storage structures appear contiguous to the operating
system and applications.
The fact that logical storage structures can span multiple physical devices provides flexibility
and additional functionality:
Dynamic extension of file systems
Host based mirroring
Host based striping
The Logical Volume Manager provides a set of operating system commands, library
subroutines, and other tools that enable the creation and control of logical storage.
Volume Groups
Physical
Disk
Block
Volume Group
Physical
Volume 1
Physical
Volume 2
Physical
Volume 3
One or more Physical Volumes
form a Volume Group
LVM manages Volume Groups
as a single entity
Physical Volumes can be added
and removed from a Volume
Group as necessary
Physical Volumes are typically
divided into contiguous equal-
sized disk blocks
A host will always have at least
one disk group for the Operating
System
Application and Operating
System data maintained in
separate volume groups
A Volume Group is created by grouping together one or more Physical Volumes. Physical
Volumes:
Can be added or removed from a Volume Group dynamically.
Cannot be shared between Volume Groups, the entire Physical Volume becomes part of a
Volume Group.
Each Physical Volume is partitioned into equal-sized data blocks. The size of a Logical Volume
is based on a multiple of the equal-sized data block.
The Volume Group is handled as a single unit by the LVM.
A Volume Group, as a whole, can be activated or deactivated.
A Volume Group would typically contain related information. For example, each host would
have a Volume Group which holds all the OS data, while applications would be on separate
Volume Groups.
Logical Volumes are created within a given Volume Group. A Logical Volume can be thought
of as a virtual disk partition, while the Volume Group itself can be though of as a disk. A
Volume Group can have a number of Logical Volumes.
Logical Volumes
Logical Disk
Block
Volume Group
Physical Disk
Block
Physical Volume 1 Physical Volume 2 Physical Volume 3
Logical Volume
Logical Volume
Logical Volumes (LV) form the basis of logical storage. They contain logically contiguous data
blocks (or logical partitions) within the volume group. Each logical partition is mapped to at
least one physical partition on a physical volume within the Volume Group. The OS treats an
LV like a physical device and accesses it via device special files (character or block). A Logical
Volume:
Can only belong to one Volume Group. However, a Volume Group can have multiple LVs.
Can span multiple physical volumes.
Can be made up of physical disk blocks that are not physically contiguous.
Appears as a series of contiguous data blocks to the OS.
Can contain a file system or be used directly. Note: There is a one-to-one relationship
between LV and a File System.
Note: Under normal circumstances there is a one-to-one mapping between a logical and physical
Partition. A one-to-many mapping between a logical and physical partition leads to mirroring of
Logical Volumes.
Host Based Replication: Mirrored Logical Volumes
Host Logical Volume
Logical Volume
Physical
Volume 1
PVID1
VGDA
Physical
Volume 2
PVID2
VGDA
Logical Volumes may be mirrored to improve data availability. In mirrored logical volumes
every logical partition will map to 2 or more physical partitions on different physical volumes.
Logical volume mirrors may be added and removed dynamically
A mirror can be split and data contained used independently
The advantages of Mirroring a Logical Volume are high availability and load balancing during
reads if the parallel policy is used. The cost of mirroring is additional CPU cycles necessary to
perform two writes for every write and the longer cycle time needed to complete the writes.
Host Based Replication: File System Snapshots
Many LVM vendors will allow the creation of File System
Snapshots while a File System is mounted
File System snapshots are typically easier to manage
than creating mirrored logical volumes and then splitting
them
Many Logical Volume Manager vendors will allow the creation of File System Snapshots while
a File System is mounted. File System snapshots are typically easier to manage than creating
mirrored logical volumes and then splitting them.
Host (LVM) Based Replicas: Disadvantages
LVM based replicas add overhead on host CPUs
If host devices are already Storage Array devices then
the added redundancy provided by LVM mirroring is
unnecessary
The devices will have some RAID protection already
Host based replicas can be usually presented back to the
same server
Keeping track of changes after the replica has been
created
Host based replicas can be usually presented back to the same server:
Using the replica from the same host for any BC operation will add an additional CPU
burden on the server
Replica is useful for fast recovery if there is any logical corruption on the source at the File
System level
Replica itself may become unavailable if there is a problem at the Volume Group level
If the Server fails, then the replica and the source would be unavailable until the server is
brought online or another server is given access to the Volume group
Presenting a LVM based local replica to a second host is usually not possible because the
replica will still be part of the volume group which is usually accessed by one host at any
given time
Keeping track of changes after the replica has been created:
If changes are not tracked all future resynchronization will be a full operation
Some LVMs may offer incremental resynchronization
Replication performed by the Array Operating
Environment
Replicas are on the same array
Storage Array Based Local Replication
Production
Server
Business
Continuity Server
Array
Replica Source
With storage array based local replication:
Replication performed by the Array Operating Environment
Array CPU resources are used for the replication operations
Host CPU resources can be devoted to production operations instead of replication
operations
Replicas are on the same array
Can be accessed by an alternate host for any BC operations
Typically array based replication is performed at a array device level.
Need to map storage components used by an application back to the specific array
devices used then replicate those devices on the array.
A database could be laid out on over multiple physical volumes which belong. One
would have to replicate all the devices for a PIT copy of the database.
Typically Array based replication is done at a array device
level
Need to map storage components used by an application/file system
back to the specific array devices used then replicate those
devices on the array
Array 1
Storage Array Based Local Replication Example
File System 1
Volume Group 1
Logical Volume 1
Source
Vol 1
Replica
Vol 1
Source
Vol 2
Replica
Vol 2
c12t1d1 c12t1d2
In this example, File System 1 has to be replicated.
File System 1 is actually built on Logical Volume 1, which in turn is a part of Volume
Group 1 which is made up of two Physical Volumes c12t1d1 and c12t1d2.
These physical volumes are actually residing in Array 1 and are Source Vol1 and Source
Vol2.
In order to replicate File System 1, one has to actually replicate the two Array Devices.
Since 2 Array Volumes have to replicated we need two Array Volumes to act as the replica
volumes. In this example Replica Vol1 and Replica Vol2 will be used for the replication.
Array Based Local Replication: Full Volume Mirror
Source Target
Attached
Array
Read/Write Not Ready
Full volume mirroring is achieved by attaching the target device to the source device and then
copying all the data from the source to the target. The target is unavailable to its host while it is
attached to the source and the synchronization occurs.
Target (Replica) device is attached to the Source device and the entire data from the source
device is copied over to the target device
During this attachment and synchronization period the Target device is unavailable
Source Target
Detached - PIT
Read/Write Read/Write
Array
After the synchronization is complete the target can be detached from the source and be made
available for Business Continuity operations. The point-in-time (PIT) is determined by the time
of detachment or separation of the Source and Target. For example, if the detachment time is
4:00 PM, the PIT of the replica is 4:00 PM.
For future re-synchronization to be incremental, most
vendors have the ability to track changes at some level of
granularity (e.g., 512 byte block, 32 KB, etc.)
Tracking is typically done with some kind of bitmap
Target device must be at least as large as the Source
device
For full volume copies the minimum amount of storage required is
the same as the size of the source
For future re-synchronization to be incremental, most vendors have the ability to track changes
at some level of granularity, such as 512 byte block, 32 KB, etc. Tracking is typically done with
some kind of bitmap. The target device must be at least as large as the Source device. For full
volume copies the minimum amount of storage required is the same as the size of the source.
Copy on First Access (COFA)
Target device is made accessible for BC tasks as soon
as the replication session is started
Point-in-Time is determined by time of activation
Can be used in Copy First Access mode (deferred) or in
Full Copy mode
Target device is at least as large as the Source device
Copy on First Access (COFA) provides an alternate method to create full volume copies. Unlike
Full Volume mirrors, the replica is immediately available when the session is started (no waiting
for full synchronization).
The PIT is determined by the time of activation of the session. Just like the full volume
mirror technology this method requires the Target devices to be at least as large as the
source devices.
A protection map is created for all the data on the Source device at some level of granularity
(e.g., 512 byte block, 32 KB, etc.). Then the data is copied from the source to the target in
the background based on the mode with which the replication session was invoked.
Write to Source
Copy on First Access Mode: Deferred Mode
Source Target
Write to Target
Read from Target
Source Target
Source Target
In the Copy on First Access mode (or the deferred mode), data is copied from the source to the
target only when:
A write is issued for the first time after the PIT to a specific address on the source
A read or write is issued for the first time after the PIT to a specific address on the target.
Since data is only copied when required, if the replication session is terminated the target device
will only have data that was copied (not the entire contents of the source at the PIT). In this
scenario, the data on the Target cannot be used as it is incomplete.
Copy on First Access: Full Copy Mode
On session start, the entire contents of the Source device
is copied to the Target device in the background
Most vendor implementations provide the ability to track
changes:
Made to the Source or Target
Enables incremental re-synchronization
In Full Copy mode, the target is made available immediately and all the data from the source is
copied over to the target in the background.
During this process, if a data block that has not yet been copied to the target is accessed, the
replication process will jump ahead and move the required data block first.
When a full copy mode session is terminated (after full synchronization), the data on the
Target is still usable as it is a full copy of the original data.
Array: Pointer Based Copy on First Write
Targets do not hold actual data, but hold pointers to
where the data is located
Actual storage requirement for the replicas is usually a small fraction
of the size of the source volumes
A replication session is setup between the Source and
Target devices and started
When the session is setup based on the specific vendors
implementation a protection map is created for all the data on the
Source device at some level of granularity (e.g 512 byte block, 32
KB etc.)
Target devices are accessible immediately when the session is
started
At the start of the session the Target device holds pointers to the
data on the Source device
Unlike full volume replicas, the target devices for pointer based replicas only hold pointers to
the location of the data but not the data itself. When the copy session is started the target device
holds pointers to the data on the source device. The primary advantage of pointer based copies is
the reduction in storage requirement for the replicas.
Pointer Based Copy on First Write Example
Source Save Location
Target
Virtual Device
The original data block from the Source is copied to the save location, when a data block is first
written to after the PIT.
Prior to a new write to the source or target device:
Data is copied from the source to a save location
The pointer for that specific address on the Target then points to the save location
Writes to the Target result in writes to the save location and the updating of the
pointer to the save location
If a write is issued to the source for the first time after the PIT the original data block is
copied to the save location and the pointer is updated from the Source to the save
location.
If a write is issued to the Target for the first time after the PIT the original data is copied
from the Source to the Save location, the pointer is updated and then the new data is
written to the save location.
Reads from the Target are serviced by the Source device or from the save location based
on the where the pointer directs the read.
Source When data has not changed since PIT
Save Location When data has changed since PIT
Data on the replica is a combined view of unchanged data on the Source and the save location.
Hence if the Source device becomes unavailable the replica will no longer have valid data.
Array Replicas: Tracking Changes
Changes will/can occur to the Source/Target devices
after PIT has been created
How and at what level of granularity should this be
tracked?
Too expensive to track changes at a bit by bit level
Would require an equivalent amount of storage to keep track of which bit
changed for each the source and the target
Based on the vendor some level of granularity is chosen and a bit
map is created (one for Source and one for Target)
One could choose 32 Kb as the granularity
For a 1 GB device changes would be tracked for 32768 32Kb chunks
If any change is made to any bit on one 32Kb chunk the whole chunk is
flagged as changed in the bit map
1 GB device map would only take up 32768/8/1024 = 4Kb space
It is too expensive to track changes at a bit by bit level because it would require an equivalent
amount of storage to keep track of which bit changed for each the source and the target. Some
level of granularity is chosen and a bit map is created (one for the Source and one for the Target).
The level of granularity is vendor specific.
Source 0 0 0 0 0 0 0 0
Target 0 0 0 0 0 0 0 0
Array Replicas: How Changes Are Determined
1 0 1 1 0 1 0 1
0 = unchanged 1 = changed
Resynch
At PIT
Target 0 0 1 1 0 0 0 1
Source 1 0 0 1 0 1 0 0
After PIT
Differential/incremental re-synchronization:
The bitmaps for the source and target are all set to 0 at the PIT
Any changes to the source or target after PIT are flagged by setting appropriate flag to 1 in
the bit map
When a re-synchronization is required the two bitmaps are compared and only those chunks
that have either changed on the source or target are synchronized
The benefit is that re-synchronization times are minimized.
Array Replication: Multiple PITs
06:00 A.M.
: 12 : 01 : 02 : 03 : 04 : 05 : 06 : 07 : 08 : 09 : 10 : 11 : 12 : 01 : 02 : 03 : 04 : 05 : 06 : 07 : 08 : 09 : 10 : 11 :
P.M. A.M.
12:00 P.M.
06:00 P.M.
12:00 A.M.
Source
Target Devices
Point-In-Time
Most array based replication technologies will allow the Source devices to maintain replication
relationships with multiple Targets.
This can also reduce RTO because the restore can be a differential restore.
Each PIT could be used for a different BC activity and also as restore points.
In this example, a PIT is created every six hours from the same source. If any logical or physical
corruption occurs on the Source, the data can be recovered from the latest PIT and at worst the
RPO will be 6 hours.
Array Replicas: Ensuring Consistency
Inconsistent
`
Consistent
Source Replica
4 4
3 3
2 2
1 1
Source Replica
4 4
3 3
2
1
Most array based replication technologies will allow the creation of Consistent replicas by
holding I/O to all devices simultaneously when the PIT is created.
Typically applications are spread out over multiple devices
Could be on the same array or multiple arrays
Replication technology must ensure that the PIT for the whole application is consistent
Need mechanism to ensure that updates do not occur while PIT is created
Hold I/O to all devices simultaneously for an instant, create PIT and release I/O
Cannot hold I/O for too long, application will timeout
Mechanisms to Hold I/O
Host based
Array based
What if the application straddles multiple hosts and
multiple arrays?
Mechanisms to hold I/O:
Host based
Some host based application could be used to hold IO to all the array devices that are to
be replicated when the PIT is created
Typically achieved at the device driver level or above before the I/O reaches the HBAs
Some vendors implement this at the multi-pathing software layer
Array based
I/Os can be held for all the array devices that are to be replicated by the Array Operating
Environment in the array itself when the PIT is created
What if the application straddles multiple hosts and multiple arrays?
Federated Databases
Some array vendors are able to ensure consistency in this situation
Array Replicas: Restore/Restart Considerations
Production has a failure
Logical Corruption
Physical failure of production devices
Failure of Production server
Solution
Restore data from replica to production
The restore would typically be done in an incremental manner and the
Applications would be restarted even before the synchronization is
complete leading to very small RTO
-----OR------
Start production on replica
Resolve issues with production while continuing operations on replicas
After issue resolution restore latest data on replica to production
Failures can occur in many different ways:
There could be a logical corruption of the data on the production devices, the devices are
available but the data on them is corrupt. In this case, one would opt to restore the data to the
production from the latest replica.
Production devices may become unavailable due to physical failures (Production server
down, physical drive failure etc.). In this case, one could start the production on the latest
replica and then while the production is being done from the replicas fix the physical
problems on the Production side. Once the situation has been resolved, the latest information
from the replica devices can be restored back to the production volumes.
Array Replicas: Restore/Restart Considerations
Before a Restore
Stop all access to the Production devices and the Replica devices
Identify Replica to be used for restore
Based on RPO and Data Consistency
Perform Restore
Before starting production on Replica
Stop all access to the Production devices and the Replica devices
Identify Replica to be used for restart
Based on RPO and Data Consistency
Create a Gold copy of Replica
As a precaution against further failures
Start production on Replica
RTO drives choice of replication technology
Based on the type of failure on has to choose to either perform a restore to the production
devices or to shift production operations to the replica devices. In either case the
recommendation would be to stop access to the production and replica devices, then identify the
replica that will be used for the restore or the restart operations.
The choice of replica depends on the consistency of the data on the replica and the desired RPO
(E.g., A business may create PIT replicas every 2 hours, if a failure occurs then at most only 2
hours of data would have been lost). If a replica has been written (application testing for
example) to after the creation of the PIT then this replica may not be a viable candidate for the
restore or restart.
Note: RTO is a key driver in the choice of replication technology. The ability to restore or
restart almost instantaneously after any failure is very important.
Array Replicas: Restore Considerations
Full Volume Replicas
Restores can be performed to either the original source device or to
any other device of like size
Restores to the original source could be incremental in nature
Restore to a new device would involve a full synchronization
Pointer Based Replicas
Restores can be performed to the original source or to any other
device of like size as long as the original source device is healthy
Target only has pointers
+Pointers to source for data that has not been written to after PIT
+Pointers to the save location for data was written after PIT
Thus to perform a restore to an alternate volume the source must be
healthy to access data that has not yet been copied over to the target
With Full Volume replicas, all the data that was on the source device when the PIT was created
is available on the Replica (either will Full Volume Mirroring or with Full Volume Copies).
With Pointer Based Replicas and Full Volume Copies in deferred mode (COFA), access to all
the data on the Replica is dependent on the health (accessibility) of the original source volumes.
If the original source volume is inaccessible for any reason, pointer based or Full Volume Copy
on First Access replicas are of no use in either a restore or a restart scenario.
Array Replicas: Which Technology?
Full Volume Replica
Replica is a full physical copy of the source device
Storage requirement is identical to the source device
Restore does not require a healthy source device
Activity on replica will have no performance impact on the source
device
Good for full backup, decision support, development, testing and
restore to last PIT
RPO depends on when the last PIT was created
RTO is extremely small
Full Volume replicas have a number of advantages over pointer based (COFA) technologies.
The replica has the entire contents of the original source device from the PIT and any
activity to the replica will have no performance impact on the source device (there is no
COFA or COFW penalty).
Full Volume replicas can be used for any BC activity.
The only disadvantage is that the storage requirements for the replica are at least equal to
that of the source devices.
Array Replicas: Which Technology? (continued)
Pointer based - COFW
Replica contains pointers to data
Storage requirement is a fraction of the source device (lower cost)
Restore requires a healthy source device
Activity on replica will have some performance impact on source
Any first write to the source or target will require data to be copied to the
save location and move pointer to save location
Any read IO to data not in the save location will have to be serviced by
the source device
Typically recommended if the changes to the source are less than
30%
RPO depends on when the last PIT was created
RTO is extremely small
The main benefit of Pointer based copies is the lower storage requirement for the replicas. This
technology is also very useful when the changes to the source are expected to be less that 30%
after the PIT has been created. Heavy activity on the Target devices may cause performance
impact on the source because any first writes to the target will require data to be copied from the
source to the save location, also any reads which are not in the save area will have to be read
from the source device. The source device needs to be accessible for any restart or restore
operations from the Target.
Array Replicas: Which Technology? (continued)
Full Volume COFA Replicas
Replica only has data that was accessed
Restore requires a healthy source device
Activity on replica will have some performance impact
Any first access on target will require data to be copied to target before
the I/O to/from target can be satisfied
Typically replicas created with COFA only are not as useful as
replicas created with the full copy mode Recommendation would
be to use the full copy mode it the technology allows such an option
The COFA technology requires at least the same amount of storage as the source. The
disadvantages of the COFW penalty and the fact that the replica would be of no use if the source
volume were inaccessible make this technology less desirable. In general this technology should
only be recommended if a full copy mode is available. If a full copy mode is available then one
should always use the full copy mode and then the advantages are identical to that discussed for
Full Volume replicas.
Array Replicas: Full Volume vs. Pointer Based
< 30% No limits Data change
Requires a healthy
source device
Source need not be
healthy
Restore
Very small Very small RTO
Some None Performance Impact
Fraction of Source 100% of Source Required Storage
Pointer Based Full Volume
This table summarizes the differences between Full Volume and Pointer Base replication
technologies.
Module Summary
Replicas and the possible use of Replicas
Consistency considerations when replicating File
Systems and Databases
Host and Array based Replication Technologies
Advantages/Disadvantages
Differences
Considerations
Remote Replication
After completing this module, you will be able to:
Explain Remote Replication Concepts
Synchronous/Asynchronous
Connectivity Options
Discuss Host and Array based Remote Replication
Technologies
Functionality
Differences
Considerations
This module introduces the challenges and solutions for remote replication and describes two
possible implementations.
Remote Replication Concepts
Replica is available at a remote facility
Could be a few miles away or half way around the world
Backup and Vaulting are not considered remote replication
Synchronous Replication
Replica is identical to source at all times Zero RPO
Asynchronous Replication
Replica is behind the source by a finite margin Small RPO
Connectivity
Network infrastructure over which data is transported from source
site to remote site
The Replication concepts/considerations that were discussed for Local Replication apply to
Remote Replication as well. We will explore the concepts that are unique to Remote replication.
Synchronous and Asynchronous replication concepts and considerations will be explained in
more detail in the next few slides.
Data has to be transferred from the source site to a remote site over some network This can be
done over IP networks, over the SAN, using DWDM (Dense Wave Division Multiplexing) or
SONET (Synchronous Optical Network) etc. We will discuss the various options later in the
module.
A write has to be secured on the
remote replica and the source before
it is acknowledged to the host
Ensures that the source and remote
replica have identical data at all times
Write ordering is maintained at all times
Replica receives writes in exactly the
same order as the source
Synchronous replication provides the
lowest RPO and RTO
Goal is zero RPO
RTO is as small as the time it takes to
start application on the remote site
1
3
4
2
Data Write
Data Acknowledgement
Server
Disk
Disk
Synchronous Data is committed at both the source site and the remote site before the write is
acknowledged to the host. Any write to the source must be transmitted to and acknowledged by
the remote before signaling a write complete to the host. Additional writes cannot occur until
each preceding write has been completed and acknowledged. Ensures that data at both sites are
identical at all times.
Response Time Extension
Application response time will be
extended due to synchronous
replication
Data must be transmitted to remote
site before write can be acknowledged
Time to transmit will depend on
distance and bandwidth
Bandwidth
To minimize impact on response
time, sufficient bandwidth must be
provided for at all times
Rarely deployed beyond 200 km
Average
Time
Writes
MB/s
Max
Applications response times will be extended with any kind of Synchronous Replication, this
is due to the fact that any write to source must be transmitted to and acknowledged by remote
before signaling write complete to the host. The response time depends on the distance
between sites, available bandwidth and the network connectivity infrastructure.
The longer the distance the more the response time Speed of light is finite every 200 Km
(125 miles) will add 1ms to the response time.
Insufficient bandwidth will also cause response time elongation. With Synchronous replication
one should have sufficient bandwidth all the time. The picture on the slide shows the amount
of data that has to replicated as a function of time. To minimize the response time elongation
one must ensure that the Max bandwidth is provided by the network at all times. If we assume
that only the average bandwidth is provided for then there will be times during the day (the
shaded section) when response times may be unduly elongated causing applications to time
out.
The distances over which Synchronous replication can be deployed really depends on an
applications ability to tolerate the extension in response time. It is rarely deployed for
distances greater than 200 Km (125 miles).
Write is acknowledged to host
as soon as it is received by the
source
Data is buffered and sent to
remote
Some vendors maintain write ordering
Other vendors do not maintain write
ordering, but ensure that the replica will
always be a consistent re-startable
image
Finite RPO
Replica will be behind the Source by
a finite amount
Typically configurable
1
4
2
3
Data Write
Data Acknowledgement
Server
Disk
Disk
Asynchronous - Data is committed at the source site and the acknowledgement is sent to the
host. The Data is buffered and then forwarded to the remote site as the network capabilities
permit. The Data at the remote site will be behind the source by a finite RPO, typically the RPO
would be a configurable value.
The primary benefit of Asynchronous replication is that there is no response time elongation.
Asynchronous replications are typically deployed over extended distances. The response time
benefit is offset by the Finite RPO.
Response Time unaffected
Bandwidth
Need sufficient bandwidth on average
Buffers
Need sufficient buffers
Can be deployed over long distances
Average
Time
Writes
MB/s
Max
Extended distances can be achieved with Asynchronous replication because there is no impact
on the application response time. Data is buffered and then sent to the remote site. The available
bandwidth should be at least equal to the average write workload. Data will be buffered during
times when the bandwidth is not enough, thus sufficient buffers should be designed into the
solution.
Remote Replication Technologies
Host based
Logical Volume Manager (LVM)
Log Shipping
Storage Array based
Synchronous
Asynchronous
Disk Buffered - Consistent PITs
Combination of Local and Remote Replication
In the context of our discussion Remote Replication refers to replication that is done between
data centers if it is host based and between Storage arrays if it is array based.
Host based implies that all the replication is done by using the CPU resources of the host using
software that is running on the host. Array based implies that all replication is done between
Storage Arrays and is handled by the Array Operating Environment.
We will discuss each of the technologies listed in turn.
LVM Based Remote Replication
Network
Volume Group
Physical
Volume 1
Physical
Volume 2
Physical
Volume 3
Physical
Volume 1
Physical
Volume 2
Physical
Volume 3
Local Site Remote Site
Volume Group
Log
Log
Duplicate Volume Groups at local and remote sites
All writes to the source Volume Group are replicated to
the remote Volume Group by the LVM
Synchronous or Asynchronous
Some LVM vendors provide remote replication at the Volume Group level
Duplicate Volume Groups need to exist at both the local and remote sites before replication
starts
This can be achieved in a number of ways Over IP, Tape backup/restore etc.
All writes to the source Volume Group are replicated to the remote Volume Group by the LVM
Typically the writes are queued in a log file and sent to the remote site in the order received
over a standard IP network
Can be done synchronously or asynchronously
Synchronous Write must be received by remote before the write is acknowledged locally
to the host
Asynchronous Write is acknowledged immediately to the local host and queued and sent in
order
In the event of a network failure
Writes are queued in the log file
When the issue is resolved the queued writes are sent over to the
remote
The maximum size of the log file determines the length of outage
that can be withstood
In the event of a failure at the source site, production
operations can be transferred to the remote site
Production work can continue at the source site if there is a network failure, the writes that need
to replicated will be queued in the log file and sent over to the remote site when the network
issue is resolved. If the log files fill up before the network outage is resolved a complete
resynchronization of the remote site would have to be performed. Thus the size of the log file
determine the length of network outage that can be tolerated.
In the event of a failure at the source site (e.g. server crash, site wide disaster), production
operations can be resumed at the remote site with the remote replica. The exact steps that need
to performed to achieve this depend on the LVM that is in use.
Advantages
Different storage arrays and RAID protection can be used at the
source and remote sites
Standard IP network can be used for replication
Response time issue can be eliminated with asynchronous mode,
with extended RPO
Disadvantages
Extended network outages require large log files
CPU overhead on host
For maintaining and shipping log files
A significant advantage of using LVM based remote replication is the fact that storage arrays
from different vendors can be used at the two sites. E.g. At the production site a high-end array
could be used while at the remote site a second tier array could be used. In a similar manner the
RAID protection at the two sites could be different as well.
Most of the LVM based remote replication technologies allow the use of standard IP networks
that are already in place, eliminating the need for a dedicated network. Asynchronous mode
supported by many LVMs eliminates the response time issue of synchronous mode while
extending the RPO.
Log files need to be configured appropriately to support extended network outages. Host based
replication technologies use host CPU cycles.
Host Based Log Shipping
Offered by most DB Vendors
Advantages
Minimal CPU overhead
Low bandwidth
Standby Database consistent
to last applied log
Original
Logs
Stand By
Logs
IP Network
Log Shipping is a host based replication technology for databases offered by most DB Vendors
Initial State - All the relevant storage components that make up the database are replicated to
a standby server (done over IP or other means) while the database is shutdown
Database is started on the production server As and when log switches occur the log file
that was closed is sent over IP to the standby server
Database is started in standby mode on the standby server, as and when log files arrive they
are applied to the standby database
Standby database is consistent up to the last log file that was applied
Advantages
Minimal CPU overhead on production server
Low bandwidth (IP) requirement
Standby Database consistent to last applied log
RPO can be reduced by controlling log switching
Disadvantages
Need host based mechanism on production server to periodically ship logs
Need host based mechanism on standby server to periodically apply logs and check for
consistency
IP network outage could lead to standby database falling further behind
Array Based Remote Replication
Replication performed by the Array Operating
Environment
Host CPU resources can be devoted to production operations
instead of replication operations
Arrays communicate with each other via dedicated channels
ESCON, Fibre Channel or Gigabit Ethernet
Replicas are on different arrays
Primarily used for DR purposes
Can also be used for other BC operations
Production Array
Remote Array
Distance
Source
Replica
Network
DR Server
Production
Server
Replication Process
A Write is initiated by an application/server
Received by the source array
Source array transmits the write to the remote array via dedicated channels (ESCON, Fibre
Channel or Gigabit Ethernet) over a dedicated or shared network infrastructure
Write received by the remote array
Only Writes are forwarded to the remote array
Reads are from the source devices
Array Based Synchronous Replication
Network links
Write is received by the source array from host/server
Write is transmitted by source array to the remote array
Remote array sends acknowledgement to the source
array
Source array signals write complete to host/server
Source Target
Synchronous Replication ensures that the replica and source have identical data at all times. The
source array issues the write complete to the host/server only when the write has been received
both at the remote array and the source array. Thus when the write complete is sent the Replica
and Source are identical.
The sequence of operations is:
Write is received by the source array from host/server.
Write is transmitted by source array to the remote array.
Remote array sends acknowledgement to the source array.
Source array signals write complete to host/server.
Array Based Asynchronous Replication
No impact on response time
Extended distances between arrays
Lower bandwidth as compared to Synchronous
Network links
Write is received by the source array from host/server
Write is transmitted by source array to the remote array
Source array signals write complete to host/server
Remote array sends acknowledgement to the source
array
Source Target
Applications do not suffer any response time elongation with Asynchronous replication, because
any write is acknowledged to the host as soon as the write is received by the source array. Thus
asynchronous replication can be used for extended distances. Bandwidth requirements for
Asynchronous will be lower than Synchronous for the same workload. Vendors ensure data
consistency in different ways
Array Based Asynchronous Replication
Ensuring Consistency
Maintain write ordering
Some vendors attach a time stamp and sequence number with each of
the writes, then ship the writes to the remote array and apply the writes
to the remote devices in the exact order based on the time stamp and
sequence numbers
Remote array applies the writes in the exact order they were received,
just like synchronous
Dependent write consistency
Some vendors buffer the writes in the cache of the source array for a
period of time (between 5 and 30 seconds)
At the end of this time the current buffer is closed in a consistent manner
and the buffer is switched, new writes are received in the new buffer
The closed buffer is then transmitted to the remote array
Remote replica will contain a consistent, re-startable image on the
application
The data on the remote replicas will be behind the source by a finite amount in Asynchronous
replication, thus steps must be taken to ensure consistency. Some vendors achieve consistency
by maintaining write ordering, i.e. the remote array applies writes to the replica devices in the
exact order that they were received at the source. Other vendors leverage the dependent write
I/O logic that is built into most databases and applications.
Array based Disk Buffered Consistent PITs
Local and Remote replication technologies can be
combined to create consistent PIT copies of data on
remote arrays
RPO usually in the order of hours
Lower Bandwidth requirements
Extended distance solution
Disk buffered consistent PITs is a combination of Local and Remote replications technologies.
The idea is to make a Local PIT replica and then create a Remote replica of the Local PIT. The
advantage of disk buffered PITs is lower bandwidth requirements and the ability to replicate
over extended distances. Disk buffered replication is typically used when the RPO requirements
are of the order of hours or so, thus a lower bandwidth network can be used to transfer data from
the Local PIT copy to the remote site. The data transfer may take a while, but the solution would
be designed to meet the RPO.
We will take a look at a two disk buffered PIT solutions.
Extended Distance Consistent PIT
Create a Consistent PIT Local Replica on Source Array
Create a Remote Replica of this Local Replica
Optionally create another replica of the Remote replica on the
remote array if needed
Repeatas automation, link bandwidth, change rate permit
SOURCE
REMOTE
Network Links
Remote
Replica
Local
Replica
Local
Replica
Source
Disk buffered replication allows for the incremental resynchronization between a Local Replica
which acts as a source for a Remote Replica.
Benefits include:
Reduction in communication link cost and improved resynchronization time for long-
distance replication implementations
The ability to use the various replicas to provide disaster recovery testing, point-in-time
backups, decision support operations, third-party software testing, and application upgrade
testing or the testing of new applications.
Synchronous + Extended Distance Consistent PIT
Synchronous replication between the Source and Bunker
Site
Create consistent PIT Local Replica at bunker
Create Remote Replica of bunker Local Replica
Optionally create additional Local Replica at Target site
from the Remote Replica if needed
Repeatas automation, link bandwidth, change rate
permit
SOURCE REMOTE BUNKER
Sync
Source
Remote
Replica
Local
Replica
Local
Replica
Remote
Replica
Network
Links
Network
Links
Synchronous + Extended Distance Buffered Replication benefits include:
Bunker site provides a zero RPO DR Replica
The ability to resynchronize only changed data between the intermediate Bunker site and the
final target site, reducing required network bandwidth
Reduction in communication link cost and improved resynchronization time for long-
distance replication implementations
The ability to use the replicas to provide disaster recovery testing, point-in-time backups,
decision support operations, third-party software testing, and application upgrade testing or
the testing of new applications.
Remote Replicas Tracking Changes
Remote replicas can be used for BC Operations
Typically remote replication operations will be suspended when the
remote replicas are used for BC Operations
During BC Operations changes will/could happen to both
the source and remote replicas
Most remote replication technologies have the ability to track
changes made to the source and remote replicas to allow for
incremental re-synchronization
Resuming remote replication operations will require re-
synchronization between the source and replica
Tracking changes to facilitate incremental re-synchronization between the source devices and
remote replicas is done via the use of bitmaps in a manner very similar to that discussed in the
Local Replication lecture. Two bitmaps one for the source and one for the replica would be
created, some vendors may keep the information of both bitmaps at both the source and remote
sites, while others may simply keep the source bitmap at the source site and the remote bitmap
at the remote site. When a re-synchronization (source to replica or replica to source) is required
the source and replica bitmaps will be compared and only data that was changed will be
synchronized.
Primary Site Failure Operations at Remote Site
Remote replicas are typically not available for use while
the replication session is in progress
In the event of a primary site failure the replicas have to
be made accessible for use
Create a local replica of the remote devices at the remote
site
Start operations at the Remote site
No remote protection while primary site issues are resolved
After issue resolution at Primary Site
Stop activities at remote site
Restore latest data from remote devices to source
Resume operations at Primary (Source) Site
While remote replication is in progress the remote devices will typically not be available for use.
This is to ensure that the no changes are made to the remote replicas, the purpose of the remote
replica is to provide a good starting point for any recovery operations.
Prior to any recovery efforts with the remote replicas, it is always a good idea to create a local
replica of the remote devices. The local replica can be use as a fall back if the recovery process
somehow corrupts the remote replicas.
Array Based Which Technology?
Synchronous
Is a must if zero RPO is required
Need sufficient bandwidth at all times
Application response time elongation will prevent extended distance
solutions (rarely above 125 miles)
Asynchronous
Extended distance solutions with minimal RPO (order of minutes)
No Response time elongation
Generally requires lower Bandwidth than synchronous
Must design with adequate cache/buffer or sidefile/logfile capacity
Disk Buffered Consistent PITs
Extended distance solution with RPO in the order of hours
Generally lower bandwidth than synchronous or asynchronous
The choice of the appropriate array based remote replication depends on specific needs.
What are the RPO requirements? What is the distance between sites? What is the primary reason
for remote replication? etc.
Storage Array Based Remote Replication
Network Options
Most vendors support ESCON or Fibre Channel adapters for remote
replication
Can connect to any optical or IP networks with appropriate protocol
converters for extended distances
+DWDM
+SONET
+IP Networks
Some Vendors have native Gigabit Ethernet adapters which allows
the array to be connected directly to IP Networks without the need
for protocol converters
A dedicated or a shared network must be in place for remote replication. Storage arrays will
have dedicated ESCON, Fibre Channel or Gigabit Ethernet adapters which are used for remote
replication. The network between the two arrays could be ESCON or Fibre Channel for the
entire distance. Such networks would be typically used for shorter distance. For extended
distances an optical or IP network must be used. Examples of optical networks are DWDM and
SONET (discussed later). To connect the ESCON or Fibre Channel adapters from the arrays to
these networks protocol converters may have to be used. Gigabit Ethernet adapters can be
connected directly to the IP network.
Dense Wavelength Division Multiplexing (DWDM)
DWDM is a technology that puts data from different
sources together on an optical fiber with each signal
carried on its own separate light wavelength (commonly
referred to as a lambda or ).
Up to 32 protected and 64 unprotected separate
wavelengths of data can be multiplexed into a light
stream transmitted on a single optical fiber.
ESCON
Fibre Channel
Gigabit Ethernet
Optical Channels
Optical Electrical
Optical
Lambda
Dense Wavelength Division Multiplexing (DWDM) multiplexes wavelengths (often referred to
as lambdas or represented by the symbol ) onto a single pair (transmit and receive paths) of
optical fibers.
A key benefit of DWDM is protocol transparency. Since DWDM is an optical transmission
technique, the same interface type can be used to transport any bit rate or protocol. It also allows
different bit rates and protocol data streams to be mixed on the same optical fiber. DWDM
alleviates the need for protocol conversion, associated complexity and the resulting transmission
latencies.
Synchronous Optical Network (SONET)
SONET is Time Division Multiplexing
(TDM) technology where traffic from
multiple subscribers is multiplexed
together and sent out onto the SONET
ring as an optical signal
Synchronous Digital Hierarchy (SDH)
similar to SONET but is the European
standard
SONET/SDH, offers the ability to
service multiple locations, its
reliability/availability, automatic
protection switching, and restoration
SONET
OC3
OC48
OC48
SDH
STM-1
STM-16
STM-16
Synchronous Optical Networks (SONET) is a standard for optical telecommunications transport
formulated by the Exchange Carriers Standards Association (ECSA) for the American National
Standards Institute (ANSI). The equivalent international standard is referred to as Synchronous
Digital Hierarchy and is defined by the European Telecommunications Standards Institute
(ETSI). Within Metropolitan Area Networks (MANs) today, SONET/SDH rings are used to
carry both voice and data traffic over fiber.
Rated Bandwidth
2488.0 OC48/STM16
622.08 OC12/STM4
155.5 OC3/STM1
51.8 OC1
34 E3
2 E1
45 T3
1.5 T1
1024 Gigabit Ethernet
1024 or 2048 Fibre Channel
200 Escon
Bandwidth Mb/s Link
The slide lists the rated bandwidth in Mb/s for standard WAN (T1, T3, E1, E3), SONET (OC1,
OC3, OC12, OC48) and SDH (STM1, STM4, STM16) Links. The rated bandwidth of ESCON,
Fibre Channel and Gigabit Ethernet is also listed.
Module Summary
Remote Replication Concepts
Connectivity Options
Host and Array based Remote Replication Technologies
Functionality
Differences
Considerations
Section Summary
Key points covered in this section:
Business continuity overview
Basic technologies that are enablers of data availability
Basic disaster recovery techniques
This completes Section 4 Business Continuity.
Please take a moment to review the key points covered in this section.

STF-4 Business Continuity PDF

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

STF-4 Business Continuity PDF

Загружено:

Авторское право:

Доступные форматы

Copyright 2006 EMC Corporation. Do not Copy - All Rights Reserved.

Вам также может понравиться