Вы находитесь на странице: 1из 26

Module 7 - Replication

Replication
Why replicate?
Reliability
Avoid single points of failure
Performance
Scalability in numbers and geographic area
Why not replicate?
Replication transparency
Consistency issues
Updates are costly
Availability may suffer if not careful
CS454/654 7-2
Logical vs Physical Objects
There are physical copies of logical objects in the system.
Operations are specified on logical objects, but translated to
operate on physical objects.
x
1
write(x)
T
x
2
x
3
write(x
1
) write(x
2
) write(x
3
)
CS454/654 7-3
From Coulouris, Dollimoreand Kindberg, Distributed Systems: Concepts and Design, 3rd ed.
Addison-Wesley Publishers 2000
Replication Architecture
FE
Requests and
replies
C
Replica
C
Service
Clients
Front ends
managers
RM
RM
FE
RM
CS454/654 7-4
From Coulouris, Dollimoreand Kindberg, Distributed Systems: Concepts and Design, 3rd ed.
Addison-Wesley Publishers 2000
Object Replication (1)
Organization of a distributed remote object shared by
two different clients.
CS454/654 7-5
From Tanenbaumand van Steen, Distributed Systems: Principles and Paradigms
Prentice-Hall, Inc. 2002
Object Replication (2)
a) A remote object capable of handling concurrent invocations on its own.
b) A remote object for which an object adapter is required to handle concurrent
invocations
CS454/654 7-6
From Tanenbaumand van Steen, Distributed Systems: Principles and Paradigms
Prentice-Hall, Inc. 2002
Object Replication (3)
a) A distributed system for replication-aware distributed objects.
b) A distributed system responsible for replica management
CS454/654 7-7
From Tanenbaumand van Steen, Distributed Systems: Principles and Paradigms
Prentice-Hall, Inc. 2002
What will we study
Consistency models - How do we reason about the consistency of the
global state?
Data-centric consistency
Strict consistency
Linearizability
Sequential consistency
Client-centric consistency
Eventual consistency
Update propagation - How does an update to one copy of an item get
propagated to other copies?
Replication protocols - What is the algorithm that takes one update
propagation method and enforces a given consistency model?
CS454/654 7-8
Strict Consistency
Any read(x) returns a value corresponding to the result of the
most recent write(x).
CS454/654 7-9
Relies on absolute global time; all writes are instantaneously
visible to all processes and an absolute global time order is
maintained.
Cannot be implemented in a distributed system
t
1
R(x)
t
2
W
2
(x)b
R(x,b)
WRONG!
Machine 1
Machine 2
From Tanenbaumand van Steen, Distributed Systems: Principles and Paradigms
Prentice-Hall, Inc. 2002
Linearizability
The result of the execution should satisfy the following
criteria:
Read and write by all processes were executed in some serial order and
each processs operations maintain the order of specified;
If ts
op
1
(x) < ts
op
2
(y) then op
1
(x) should precede op
2
(y) in this sequence.
This specifies that the order of operations in itnerleavingis conssitent
with the real times at which the operations occurred in the actual
implementation.
Requires synchronization according to timestamps, which
makes it expensive.
Used only in formal verification of programs.
CS454/654 7-10
Sequential Consistency
Similar to linearizability, but no requirement on timestamp
order.
The result of execution should satisfy the following criteria:
Read and write operations by all processes on the data store were
executed in some sequential order;
Operations of each individual process appear in this sequence inthe
order specified by its program.
These mean that all processes see the same interleaving of
operations similar to serializability.
CS454/654 7-11
From Tanenbaumand van Steen, Distributed Systems: Principles and Paradigms
Prentice-Hall, Inc. 2002
Sequential Consistency
Process 1 Process 2 Process 3
R
1
(x)
W
1
(y)
R
2
(x)
W
2
(z)
R
3
(a)
W
3
(x)
R
1
(a)
W
1
(z) R
2
(x)
W
2
(a)
R
3
(x)
W
3
(y)
Switch
Order between ops
from Process i is
preserved
Interleaving between ops
between different
processes is arbitrary
CS454/654 7-12
Merged
Operations
Transactional Replica Consistency
Implementation of sequential consistency requires
transactions.
One-copy serializability
The effect of transactions performed by clients on replicated objects
should be the same as if they had been performed one at-a-time on a
single set of objects.
This is done within transactional boundaries.
CS454/654 7-13
Client-Centric Consistency
CS454/654 7-14
More relaxed form of consistency only concerned with replicas being eventually
consistent (eventual consistency).
In the absence of any further updates, all replicas converge to identical copies of
each other only requires guarantees that updates will be propagated.
Easy if a user always accesses the same replica; problematic if the user accesses
different replicas.
Client-centric consistency: guarantees for a single client the consistency of access to a
data store.
From Tanenbaumand van Steen, Distributed Systems: Principles and Paradigms
Prentice-Hall, Inc. 2002
Client-Centric Consistency (2)
Monotonic reads
If a process reads the value of a data item x, any successive read
operation on x by that process will always return that same value or a
more recent value.
Monotonic writes
A write operation by a process on a data item x is completed before any
successive write operation on x by the same process.
Read your writes
The effect of a write operation by a process on data item x will always
be seen by a successive read operation on x by the same process.
Writes follow reads
A write operation by a process on a data item x following a previous
read operation on x by the same process is guaranteed to take place on
the same or more recent value of x that was read.
CS454/654 7-15
Replica Placement Alternatives
Permanent replicas
Put a number of replicas at
specific locations
Mirroring
Server-initiated replicas
Server decides where and
when to place replicas
Push caches
Client-initiated replicas
Client caches
CS454/654 7-16
From Tanenbaumand van Steen, Distributed Systems: Principles and Paradigms
Prentice-Hall, Inc. 2002
Update Propagation
What to propagate?
Propagate only a notification
Invalidation
Propagate updated data
Possibly only logs
Propagate the update operation
Active replication
Who propagates?
Server: push approach
Client: pull approach
Epidemic protocols
Update propagation in eventual-consistency data stores.
CS454/654 7-17
Pull versus Push Protocols
Issue Push-based Pull-based
State of server List of client replicas and caches None
Messages sent Update (and possibly fetch update later) Poll and update
Response time at client Immediate (or fetch-update time) Fetch-update time
CS454/654 7-18
From Tanenbaumand van Steen, Distributed Systems: Principles and Paradigms
Prentice-Hall, Inc. 2002
Replication Protocols
We focus on those that enforce sequential consistency.
Primary-based protocols
Remote-Write protocols
Local-Write protocols
Replicated Write protocols
Active replication
Quorum-based protocols
Read-one-Write-All (ROWA)
CS454/654 7-19
CS454/654 7-20
Primary Copy Remote-Write
Protocol
From Tanenbaumand van Steen, Distributed Systems: Principles and Paradigms
Prentice-Hall, Inc. 2002
Primary Copy Local-Write
Protocol
CS454/654 7-21
From Tanenbaumand van Steen, Distributed Systems: Principles and Paradigms
Prentice-Hall, Inc. 2002
Active Replication
Requires a process, for each replica, that can perform the
update on it
How to enforce the update order?
Totally-ordered multicast mechanism needed
Can be implemented by Lamport timestamps
Can be implemented by sequencer
Problem of replicated invocations
If an object A invokes another object B, all replicas of A will invoke B
(multiple invocations)
CS454/654 7-22
Replicated Invocations Problem
CS454/654 7-23
From Tanenbaumand van Steen, Distributed Systems: Principles and Paradigms
Prentice-Hall, Inc. 2002
Solution to Replicated Invocations
a) Forwarding an invocation request from a replicated object.
b) Returning a reply to a replicated object.
CS454/654 7-24
From Tanenbaumand van Steen, Distributed Systems: Principles and Paradigms
Prentice-Hall, Inc. 2002
Quorum-Based Protocol
Assign a vote to each copy of a replicated object (say V
i
) such
that
i
V
i
= V
Each operation has to obtain a read quorum (V
r
) to read and a
write quorum (V
w
) to write an object
Then the following rules have to be obeyed in determining the
quorums:
V
r
+ V
w
> V an object is not read and written by two
transactions concurrently
V
w
> V/2 two write operations from two transactions cannot
occur concurrently on the same object
CS454/654 7-25
Quorum Example
Three examples of the voting algorithm:
a) A correct choice of read and write set
b) A choice that may lead to write-write conflicts
c) ROWA
CS454/654 7-26
From Tanenbaumand van Steen, Distributed Systems: Principles and Paradigms
Prentice-Hall, Inc. 2002