Mutual Exclusion for DS

Distributed Mutual Exclusion Algorithms

Two Approaches:

(1) Token-based approach (2) Permission-based approach

Permission-Based Approach: (1) Centralized Algorithm (2) Fully Distributed Algorithm

Centralized Algorithms

Centralized Algorithm:
1. One process is elected as the coordinator. 2. Whenever a process wants to enter a Critical Section, it sends a REQUEST message to the coordinator asking for permission. 3. If no other process is currently in CS, the coordinator sends back a REPLY granting permission. 4. When the process finishes execution of CS, it sends a RELEASE message to the coordinator.

Centralized Algorithms
Advantages:
1. GUARANTEES mutual exclusion and it is also FAIR. 2. NO STARVATION. 3. Only 3 messages per entry are required to enter the CS: REQUEST, GRANT, and RELEASE.

Drawbacks:
1. If the coordinator crashes, the entire system may go down. 2. There is no distinction between a dead coordinator and permission denied, since in both cases no messages come back to the requesting process. 3. The single coordinator can become a performance bottleneck.

Fully Distributed Algorithm (FDA)

>> When process Pi wants to enter the CS, it generates a new TS (timestamp) and sends the message REQUEST (Pi, TS) to all processes in the system. >> On receiving a REQUEST message, a process Pj may reply immediately by sending a REPLY message back to Pi, or it may defer sending a REPLY back. (See below for this decision.) >> A process that has received a REPLY message from all other processes in the system can enter the CS, queuing incoming message and deferring them. >> After exiting the CS, the process sends REPLY messages to all its deferred requests.

Decision
DECISION on whether Pj replies immediately to a REQUEST (Pi, TS) or defer its REPLY: >> If Pj is in the CS, then it defers its REPLY to Pi. >> If Pj does not want to enter the CS, then it sends a REPLY immediately. >> If Pj wants to enter the CS but has not yet entered it, then it compares its own REQUEST TS (timestamp) with the TS of the incoming message made by Pi. If Pjs TS > Pis TS (meaning that Pj is younger), then it sends a REPLY immediately to Pi. Otherwise, it defers the REPLY.

Fully Distributed Algorithm (FDA)

(1) Mutual exclusion is obtained. (2) Freedom from deadlock is ensured. (3) Freedom from starvation is ensured, since entry to the CS is scheduled based on the TS ordering. (4) The number of messages per entry to CS is 2(n-1), where n is the number of processes in the system.

Disadvantages with FDA

(1) The process need to know the identity of all other processes in the system. When a new process joins the group of processes participating in the mutual exclusion algorithm: -- The new process must receive the names of all other processes in the group. -- The name of the new process must be distributed to all the other processes in the
group.

(2) If one process fails, then the entire scheme collapses. Resolution: By continuously monitoring the state of all processes in the group. If one process fails, then all other processes are notified, so that they will no longer send REQUEST messages to the failed process. When a process recovers, it must initiate the procedure that allows it to rejoin the group.

Token-Based Algorithm for a Ring Structure

>> When a process receives the token, it may enter the CS, keeping the token. >> After the process exits the CS, the token is passed around again. >> If the process receiving the token does not want to enter the CS, it passes the token to its neighbor. >> We assume that the processes in the systems are logically organized in a ring structure. The physical communication network need not be a ring. As long as the processes are connected to one another, it is possible to implement a logical ring. >> If the ring is unidirectional, freedom from starvation is ensured. >> The number of messages per entry to CS varies from 1 to infinite. >> Two types of failure: (1) If the token is lost, an election must be called to generate a new token. (2) If a process fails, a new logical ring must be established.

Abstract Ring
p2 p3 p2 p4 p6 p1 p3

p1

p4

p2 p5

p5
p6 p5

token

(a) System graph

(b) Abstract ring

Token-based algorithm for a ring topology

1. A process Pi wishing to enter the CS sends a request message (request, Pi) along its outedge and blocks itself. 2. When a process not possessing the token receives a request message, it forwards the message along its out-edge. 3. When a process possessing the token receives the message (request, Pi), it performs: a. if it is not in CS, it forms the message (token, request queue, Pi) and sends it along its out-edge. b. If it is in CS, it enters Pi in the request queue associated with the token.

Token-based algorithm (conti.)

4. When a process completes execution of the CS, it checks if the request queue is empty. If not, it removes the first process id from the queue. Let this id be Pi. It now forms a message (token, Pi, request queue) and sends it along its out-edge. 5. When a process Pj receives the message (token, Pi, request queue), it checks if Pj = Pi. If so, it creates a local data structure to store the token and copies the request queue from the message. It now becomes active and enters the CS. If Pj is not equal to Pi, it forwards the message along its out-edge.

Raymonds Algorithm for an Abstract Inverted Tree

>> Phold designates the process in possession of the privilege token. >> The Algorithm maintains three invariants: (1) Process Phold is the root of the tree. (2) Each process in the system belongs to the tree. (3) Each process Pi other than Phold has exactly one outgoing edge (Pi, Pj) where Pj is the parent of Pi in the tree. >> A path from Pi to Phold exists in the system for every Pi other than Phold. >> The algorithm requires O(log n) messages per entry to CS, where n is the number of messages in the system.

Abstract Inverted Tree

p2 p4 p6 p1 p1 p5

p3

p3

p5

p6

p2

p4

(a) System graph

(b) Abstract inverted tree

Raymonds Algorithm for Mutual Exclusion

(1) A process Pi wishing to enter a CS enters its own id in the local queue. Also, it sends a REQUEST message containing its own id on its outgoing edge. (2) A process Pr, which receives a REQUEST from another process, performs following actions: > Put the id of the requester in its local queue. > Send a REQUEST containing its own id on its outgoing edge, if Pr is not equal to Phold. (3) On completing the execution of a CS, Phold performs following: a. Remove the process at the head of the local queue. Let it be Pi. b. Send the token to Pi. c. Reverse the tree edge (Pi, Phold). d. If the local queue is not empty, send a REQUEST containing its own id to Pi. (4) A process receiving the token performs following: > If its own request is at the top of the local queue, remove it from the queue and enter CS. > Else, perform the four actions in steps (3).

An Example of Raymonds Algorithm

>> The following Figure illustrates operation of Raymonds algorithm when processes P4 and P1 make requests to enter CS. Figure (c) shows the situation after the requests made by P4 and P1 have reached P5, the holder of the token (see Steps (1) and (2) of Raymond Algorithm).

>> When process P5 releases the CS, it (see Step (3)) (3)a. removes the P3 from its local queue, (3)b. passes the token to P3, (3)c. reverses the edge (P3, P5). (3)d. P5 now sends a request to P3 since its local queue is not empty. The result is shown in Figure (d).
>> P3 performs similar actions (see Step 4), which result in sending the token to process P4, reversal of edge (P4, P3) and sending of a request by P3 to P4. The result is shown in Figure (e). >> P4 now enter s CS. After P4 completes the CS, the token is transferred to process P1 via P3 and P5 in an analogous manner, which enables P1 to enter the CS. Note that this would not have been possible if Step 3(d) had not been executed.

Illustrating Example
p3 p1
p5 p1

p5

p1

p1

p3

p4

p1

p1

p3

p4 p5

p6

p2

p4

p4

p6

p2

p4

p4

(c)

(d)

Illustrating Example
p1
p5 p1

p5

p1

p1

p3

p4 p5

p1

p1

p3

p5

p6

p2

p4

p4

p6

p2

p4

p3

(d)

(e)

ELECTION Algorithm
>> The algorithms that determine where a new copy of the coordinator should be restarted if the coordinator fails. >> Election algorithms assume that a unique priority number is associated with each active process in the system. Further, assume the priority number for Pi to be i. >> The coordinator is always the process with the highest priority number.

The Bully Algorithm

(a) Pi sends an ELECTION message to each process with a higher priority number. (b) If no response is received within time T, Pi assumes that all processes with numbers greater than i have failed and elect itself the NEW coordinator. (Pi informs all active processes with priority numbers less than i that Pi is the new coordinator.) (c) If an answer is received, Pi has done its election. NOTE: If Pi is not the coordinator, then at any time during execution, it may receive one of the following TWO messages: (1) Pj is the new coordinator (j > i). (2) Pj has started an ELECTION (j < i). Pi sends a RESPONSE to Pj and begins its own ELECTION, provided that Pi has not already initiated such an election.

An Example
a. Assumes the system consisting of processes P1 through P4, and initially, all processes are active; P4 is the coordinator. b. P1 and P4 fail. P2 determines that P4 has failed by sending a REQUEST that is not answered within time T. P2 then begins its ELECTION by sending a REQUEST to P3 (This is the only active process with priority number greater than p2.) c. P3 receives the REQUEST, responds to P2, and begins its own ELECTION by sending an ELECTION message to P4 (P3 does not know P4 has failed.) d. P2 receives P3s RESPONSE and begins waiting to receive a message informing it that a process with a higher priority number has been elected. (What if no message is received within a time interval T? This means P3 failed after it sent a RESPONSE message to P2. P2 should RESTART the ELECTION algorithm.)

(conti.)
e. P4 does not respond within an interval T, so P3 elects itself the new coordinator and informs P1 and P2. (P1 does not receive the message, since it has failed.) f. Later, when P1 recovers, it sends an ELECTION message to P2, P3, and P4. g. P2 and P3 respond to P1 and begin their own ELECTION. P3 will again be elected, though the same events as before. h. Finally, P4 recovers and notifies P1, P2, and P3 that it is the current coordinator. (Note: P4 sends no ELECTION requests, since it is the process with the highest number in the system.)

Another Example

(a) Process 4 holds an election. (b) Processes 5 and 6 respond, telling 4 to stop. (c) Now 5 and 6 each hold an election.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5

Another Example (Conti)

(d) Process 6 tells 5 to stop. (e) Process 6 wins and tells everyone.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5

The Ring Algorithm

The RING Algorithm for a system where processes are organized as a Ring: >> A ring can be logically or physically, so each process knows who its successor is. >> When a process notices that the coordinator is not functioning, it builds an ELECTION message containing a new ACTIVE LIST with its number being the only number, and sends the message to its successor, or the one after that, until a running process is located. >> At each step along the way, the sender adds its own process number to the ACTIVE LIST in the message effectively making itself a candidate to be elected as coordinator. >> Eventually, the message gets back to the process that started it all. That process recognizes this event when it receives an incoming message containing its own process number in the ACTIVE LIST. >> At this point, the message type is changed to COORDINATOR and circulated once again, this time to inform everyone else who the coordinator is and who the members of the new ring are. When this message has circulated once, it is removed and everyone goes back to work.

Election Algorithm Using a Ring

Figure
[5,6,0] 0 previous coordinator has crashed 1 2 Election message

[2]
7 [5,6] 3 [2,3] no response 6 [5] 5 4

Explanation for the above Figure

What happens if two processes, say 2 and 5, discover simultaneously that the previous coordinator, process 7, has crashed? Answer: Each of these builds an ELECTION message and each of them starts circulating its message, independent of the other one. Eventually, both messages will go all the way around, and both 2 and 5 will convert them into COORDINATOR messages, with exactly the same members and in the same order. When both have gone around again, both will be removed. It does no harm to have extra messages circulating; at worst it consumes a little bandwidth, but this not considered wasteful.

Elections in Wireless Environments

>> The Bully and Ring Election Algorithms are based on assumptions that message passing is reliable and the topology of the network does not change, which are not realistic in wireless environments.
>> Vasudevan (2004) proposed a solution that elects a best leader rather than just a random for a wireless ad hoc network. >> Any node in the network, called the source, can initiate an election by sending an ELECTION message to its immediate neighbors (i.e., the nodes in its range). >> When a node receives an ELECTION for the first time, it designates the sender as its parent, and subsequently sends out an ELECTION message to all its immediate neighbors, except for the parent.

(conti.)
>> When a node receives an ELECTION message from a node other than its parent, it merely acknowledges the receipt. >> When node R has designated node Q as its parent, it forwards the ELECTION message to its immediate neighbors (excluding Q) and waits for acknowledgements to come in before acknowledging the ELECTION message from Q. >> This waiting has an important consequence: 1st: Note that neighbors that have already selected a parent will immediately respond to R. 2nd: If all neighbors already have a parent, R is a leaf node and will be able to report back to Q quickly. In doing so, it will also report information such as its battery lifetime and other resource capacity.

(conti.)
>> This information will later allow Q to compare Rs capacities to that of other downstream nodes, and select the best eligible node for leadership. >> Of course, Q had sent an ELECTION message only because its own parent P had done so as well. >> In turn, when Q eventually acknowledges the ELECTION message previously sent by P, it will pass the most eligible node to P as well. >> In this way, the source will eventually get to know which node is best to be selected as leader, after which it will broadcast this information to all other nodes.

Elections in Wireless Environments (1)

Election algorithm in a wireless network, with node a as the source. (a) Initial network. (b)(e) The build-tree phase
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5

Elections in Wireless Environments (2)

Election algorithm in a wireless network, with node a as the source. (a) Initial network. (b)(e) The build-tree phase
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5

Elections in Wireless Environments (3)

Figure 6-22. (e) The build-tree phase. (f) Reporting of best node to source.
Tanenbaum & Van Steen, Distributed Systems: Principles and Paradigms, 2e, (c) 2007 Prentice-Hall, Inc. All rights reserved. 0-13-239227-5

Deadlock Handling for DS

Deadlock prevention algorithms presented in a centralized system can be used in a distributed system. For example, we can use the resourceordering deadlock-prevention technique by simply defining a global ordering among the system resources, and allowing each process to request a resource (at any site) with unique number i only if it is not holding a resource with a unique number greater than i.

A deadlock-prevention scheme based on timestamp ordering with resource preemption

Each process in the system is assigned a unique timestamp when it is created. The wait-die scheme: Based on a nonpreemptive technique When process Pi requests a resource currently held by Pj, Pi is allowed to wait only if it has a smaller timestamp than does pj (i.e., Pi is older than Pj). Otherwise, Pi is rolled back (dies).

(conti.)
The wound-wait scheme: Based on a preemptive technique When process Pi requests a resource currently held by Pj, Pi is allowed to wait only if it has a larger timestamp than does pj (i.e., Pi is younger than Pj). Otherwise, Pj is rolled back (Pj is wounded by Pi.)

Example
Suppose that processes P1, p2, and P3 have timestamps 5, 10, and 15, respectively. With the wait-die scheme, if P1 requests a resource held by P2, P1 will wait. If P3 requests a resource held by P2, P3 will be rolled back. With the wound-wait scheme, if P1 requests a resource held by P2, then the resource will be preempted from P2, and P2 will be rolled back. If P3 requests a resource held by P2, then P3 will wait.

 Both schemes can avoid starvation provided that, when a process is rolled back, it is not assigned a new timestamp. Differences: wait-die: an older process must wait for a younger one to release its resource. wound-wait : an older process never waits for a younger process. wait-die: if Pi dies and is rolled back because it has requested a resource held by Pj, then Pi may reissue the same sequence of requests when it is restarted. If the resource is still held by Pj, then Pi will die again. Thus, Pi may rollback (die) several times before acquiring the needed resource. wound-wait : Pi is wounded and rolled back because Pj has requested a resource it holds. When Pi is restarted and requests the resource now being held by Pj, Pi waits. Thus, fewer rollbacks occur in the wound-wait scheme.

Discussion

Deadlock Detection
The deadlock-prevention algorithm may preempt resources even if no deadlock has occurred. To prevent unnecessary preemptions, we can use a deadlock-detection algorithm. If we assume only a single resource of each type, a cycle in the wait-for graph represents a deadlock. Problem in DS: How to maintain the wait-for graph?

Wait-For Graph
Local wait-for graphs are constructed as usual. When a process Pi in site S1 needs a resource held by process Pj in site S2, a request message is sent by Pi to site S2. The edge Pi Pj is then inserted in the local wait-for graph of site S2. If any local wait-for graph has a cycle, deadlock has occurred. The fact that we find no cycles in any of the local wait-for graphs does not mean that there are no deadlocks, however.

Example
p1 p2 p2 p4

p5

p3

p3 Site S2

Site S1
p2 p1

p4

p5
p3

Global wait-for graph