Академический Документы
Профессиональный Документы
Культура Документы
Companion slides for The Art of Multiprocessor Programming by Maurice Herlihy & Nir Shavit
Art of Multiprocessor Programming
Fundamentals
What is the weakest form of communication that supports mutual exclusion? What is the weakest shared object that allows shared-memory computation?
Alan Turing
Showed what is and is not computable on a sequential machine. Still best model there is.
Turing Computability
0 1 1 0 1 0
Mathematical model of computation What is (and is not) computable Efficiency (mostly) irrelevant
Art of Multiprocessor Programming 5
Shared-Memory Computability?
Shared Memory
10011
Mathematical model of concurrent computation What is (and is not) concurrently computable Efficiency (mostly) irrelevant
6
10
Register *
Holds a (binary) value
10011
* A memory location: name is historical
Art of Multiprocessor Programming 11
Register 10011
Can be read
10011
Art of Multiprocessor Programming 12
Register
Can be written
01100
10011
13
Registers
public interface Register<T> { public T read(); public void write(T v); }
14
Registers
public interface Register<T> { public T read(); public void write(T v); }
Single-Reader/Single-Writer Register
Multi-Reader/Single-Writer Register
10011
01100
10011
17
Multi-Reader/Multi-Writer Register
mumble
10011
mumble
10011
01010
mumble
10011 11011
Art of Multiprocessor Programming 18
Jargon Watch
SRSW
Single-reader single-writer
MRSW
Multi-reader single-writer
MRMW
Multi-reader multi-writer
19
Safe Register
OK if reads and writes dont overlap
read(1001)
write(1001)
20
Safe Register
Some valid value if reads and writes do overlap
$*&v 1111
21
write(1001)
read(????)
0000
1001
Art of Multiprocessor Programming
Regular Register
write(0) write(1)
read(1)
read(0)
Regular or Not?
write(0) write(1)
read(1)
read(0)
23
Regular or Not?
write(0) write(1)
read(1)
read(0)
24
Regular or Not?
write(0) write(1)
read(0)
25
Regular or Not?
write(0) write(1)
read(1)
read(0)
26
Regular Linearizable
write(0) write(1)
read(1)
read(0)
Atomic Register
write(1001) write(1010) read(1010)
read(1001)
read(1010)
Atomic Register
write(1001) write(1010) read(1010)
read(1001)
read(1010)
29
Register Space
MRMW
MRSW M-valued Boolean
SRSW
Safe
Regular Atomic
30
Weakest Register
Single writer
1 0 1
Single reader
Weakest Register
Single writer
0 1 0
Single reader
0 1 0
flipflop
Results
From SRSW safe Boolean register
All the other registers Mutual exclusion
34
Wait-Free Implementations
Definition: An object implementation is wait-free if every method call completes in a finite number of steps
No mutual exclusion
Thread could halt in critical section Build mutual exclusion from registers
Art of Multiprocessor Programming 35
M-valued Boolean
Road Map
SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic Atomic snapshot
Art of Multiprocessor Programming 37
Road Map
SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic Atomic snapshot
Next
38
Register Names
public class SafeBoolMRSWRegister implements Register<Boolean> { public boolean read() { } public void write(boolean x) { } }
39
Register Names
public class SafeBoolMRSWRegister implements Register<Boolean> { public boolean read() { } public void write(boolean x) { } }
property
40
Register Names
public class SafeBoolMRSWRegister implements Register<Boolean> { public boolean read() { } public void write(boolean x) { } }
property
Size matters
Art of Multiprocessor Programming 41
Register Names
public class SafeBoolMRSWRegister implements Register<Boolean> { public boolean read() { } public void write(boolean x) { } }
property
type
(3)
zzz
0 0
writer
0 0
43
0 0 0 0
0
44
1 0
0
45
0
46
Whew!
1
1 1
1
48
method
53
1000 0
1000
1011
Road Map
SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic Questions? MRMW atomic Atomic snapshot
Art of Multiprocessor Programming 56
Road Map
SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic Atomic snapshot
Art of Multiprocessor Programming
Next
57
1 0
0 1 0 1
58
0 0
0 1 0 1
59
0 Last written: 0 0
0 0
60
wrote
62
Road Map
SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic Questions? Atomic snapshot
Art of Multiprocessor Programming 68
Road Map
SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic Atomic snapshot
Art of Multiprocessor Programming
Next
69
Writing M-Valued
Unary representation: bit[i] means value i
1 0 0 0 00 0
Initially 0
01234567
70
Writing M-Valued
Write 5
1 0 0 00 01234567
71
Writing M-Valued
Write 5
Initially 0
0 1 0 0 0 1 01234567
72
Writing M-Valued
Write 5
5 01234567
0 1 0 0 0 1
73
74
bit[i] public int read() { for (int i=0; i < M; i++) if (this.bit[i].read()) return i; }}
means value i
75
Set bit x
76
77
78
Road Map
SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic Questions? Atomic snapshot
Art of Multiprocessor Programming 79
Road Map
SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic Atomic snapshot
Art of Multiprocessor Programming 80
1234 5678
1234
Instead of 5678
82
1234 5678
Initially 1234
5678
time
Art of Multiprocessor Programming 83
1234 5678
Initially 1234
1234
Instead of 5678
Reg write(5678) Reg read(1234)
time
Art of Multiprocessor Programming 84
1234 5678
Initially 1234
1234
Instead of 5678
Reg write(5678) Reg read(1234)
Reg read(5678)
85
Timestamped Values
1234 1:45 5678 2:00 Writer writes value and stamp together
2:00
5678
Reader saves last read (value,stamp) and returns new value only if higher stamp
86
writer
read(1:45 1234)
time
>
88
Another Scenario
Writer starts write stamp 2:00 1:45 1:45 1:45 value 5678 1234 1234 1234
89
Another Scenario
zzz stamp 2:00 1:45 1:45 1:45 value 5678 1234 1234 1234 2:00, 5678
reader reads
later reader
1:45 1234
Yellow was completely after Blue but read earlier valuenot linearizable!
Art of Multiprocessor Programming
90
Multi-Reader Redux
One per thread
1 1:45 1:45 1:45 1234 1234 1234 1:45 1:45 1:45 2 1234 1234 1234 1:45 1:45 1:45 3 1234 1234 1234 1 2 3
91
Multi-Reader Redux
2:00, 5678
1 1 1:45 2:00 1:45 2:00 2:00 1:45 1234 5678 1234 5678 5678 1234 1:45 1:45 1:45 2 1234 1234 1234
2 1 2 3
92
Multi-Reader Redux
2:00, 5678
reader writes column to 1 notify others of what it 2 read 1 2 3 1:45 1234 1:45 1234 2:00 5678 1:45 1234 1 2:00 5678 1:45 1234 1:45 1234 2:00 5678 1:45 1234 2 2:00 5678 1:45 1234 2:00 1:45 5678 1234 1:45 1234 3
Yellow reader will read new value in column written by earlier Blue reader
Art of Multiprocessor Programming 93
1:45 1234
read(2:00 5678)
In which case Blue will complete writing 2:00 5678 to its column
read(2:00 5678)
time
95
Road Map
SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic Next MRMW atomic Atomic snapshot
Art of Multiprocessor Programming 96
value
time time
Art of Multiprocessor Programming 98
Linearization Points
time time
Art of Multiprocessor Programming 99
Linearization Points
Look at Writes First
write(4)
time time
Art of Multiprocessor Programming 100
Linearization Points
Order writes by TimeStamp
write(4)
time time
Art of Multiprocessor Programming 101
Linearization Points
Order reads by max stamp read
time time
Art of Multiprocessor Programming 102
Linearization Points
Order reads by max stamp read
time time
Art of Multiprocessor Programming 103
Linearization Points
The linearization point depends on the execution (not a line in the code)!
time time
Art of Multiprocessor Programming 104
Road Map
SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic Questions? MRMW atomic Atomic snapshot
Art of Multiprocessor Programming 105
Road Map
SRSW safe Boolean MRSW safe Boolean MRSW regular Boolean MRSW regular MRSW atomic MRMW atomic Next Atomic snapshot
Art of Multiprocessor Programming 106
Atomic Snapshot
update
scan
107
Atomic Snapshot
Array of SWMR atomic registers Take instantaneous snapshot of all Generalizes to MRMW registers
108
Snapshot Interface
public interface Snapshot { public int update(int v); public int[] scan(); }
109
Snapshot Interface
Thread i writes v to its register
public interface Snapshot { public int update(int v); public int[] scan(); }
110
Snapshot Interface
Instantaneous snapshot of all theads registers public interface Snapshot { public int update(int v); public int[] scan(); }
111
Atomic Snapshot
Collect
Read values one at a time
Problem
Incompatible concurrent collects Result not linearizable
112
Clean Collects
Clean Collect
Collect during which nothing changed Can we make it happen? Can we detect it?
113
Simple Snapshot
Put increasing labels on each entry Collect Problem:twice Scanner might not be collecting a snapshot! If both agree,
Were done
Collect1 x y z w r z x Art of Multiprocessor Programming
Otherwise,
Try again
Collect2
x
y z w r z x 114
Scanner
Updater
Updater time
y
z w
y
z
Scanner reads x and z with Must Use Labels different labels and recognizes this is not a clean collect
Scanner 1,x 1,z 3,x 3,z
Updater
Updater time
1,x
2,y
1,z 2,w
3,x
4,y
3,z
116
Simple Snapshot
Collect twice If both agree,
Were done
Otherwise,
Try again
Collect2
1
22 1 7 13 18 12 117
118
121
Simple Snapshot
private LabeledValue[] collect() { LabeledValue[] copy = new LabeledValue[n]; for (int j = 0; j < n; j++) copy[j] = this.register[j].read(); return copy; }
Just read each register into array
122
once
124
On mismatch,
126
Simple Snapshot
Linearizable Update is wait-free
No unbounded loops
128
Wait-Free Snapshot
Add a scan before every update Write resulting snapshot together with update value If scan is continuously interrupted by updates, scan can take the updates snapshot
129
Wait-free Snapshot
If As scan observes that B moved twice, then B completed an update while As scan was in progress
Collect Collect Collect 26 24 12
26 24 12 Update
26 24 12
B time
130
Wait-free Snapshot
A
Collect Collect Collect 26 24 12
26 24 12
Update
26 24 12
time
Art of Multiprocessor Programming 131
Wait-free Snapshot
A
Collect Collect Collect 26 24 12
26 24 12 Scan
Write
26 24 12
Update
time
Art of Multiprocessor Programming 132
Wait-free Snapshot
Bs 1st update must have written during 1st collect
Collect
Collect
Collect
26 24 12 Scan
Write
26 24 12 Scan
Write
26 24 12
So A can steal result of Bs scan So scan of Bs second update must be within interval of As scan time
Update
Art of Multiprocessor Programming 133
Wait-free Snapshot
A
Collect Collect Collect 26 24 12 Scan
Write
26 24 12 Scan
Write
26 24 12
But no guarantee that scan of Bs 1st update can be used Why? time
Art of Multiprocessor Programming 134
Write
26 24 12
Scan
Update
time
Why cant A steal result of Bs scan Update Because another update might have interfered before the scan
Art of Multiprocessor Programming 135
26 24 12 Update
26 24 12
B
time
Scan is Wait-free
scan
update
scan
update scan
138
139
Actual value
140
1101111010100010110000
label
value
Art of Multiprocessor Programming
Last snapshot
142
Wait-free Update
public void update(int value) { int i = Thread.myIndex(); int[] snap = this.scan(); SnapValue oldValue = r[i].read(); SnapValue newValue = new SnapValue(oldValue.label+1, value, snap); r[i].write(newValue); }
Art of Multiprocessor Programming 143
Wait-free Scan
public void update(int value) { Take scan int i = Thread.myIndex(); int[] snap = this.scan(); SnapValue oldValue = r[i].read(); SnapValue newValue = new SnapValue(oldValue.label+1, value, snap); r[i].write(newValue); }
Art of Multiprocessor Programming 144
Wait-free Scan
public void update(int value) { Take scan int i = Thread.myIndex(); int[] snap = this.scan(); SnapValue oldValue = r[i].read(); SnapValue newValue = new SnapValue(oldValue.label+1, value, snap); r[i].write(newValue); Label value with scan }
Art of Multiprocessor Programming 145
Wait-free Scan
public int[] scan() { SnapValue[] oldCopy, newCopy; boolean[] moved = new boolean[n]; oldCopy = collect(); collect: while (true) { newCopy = collect(); for (int j = 0; j < n; j++) { if (oldCopy[j].label != newCopy[j].label) { }} return getValues(newCopy); }}}
146
Wait-free Scan
public int[] scan() { SnapValue[] oldCopy, newCopy; boolean[] moved = new boolean[n]; oldCopy = collect(); collect: while (true) { newCopy = collect(); for (int j = 0; j < n; j++) { if (oldCopy[j].label != newCopy[j].label) { Keep track of who moved }} return getValues(newCopy); }}}
147
Wait-free Scan
public int[] scan() { SnapValue[] oldCopy, newCopy; boolean[] moved = new boolean[n]; oldCopy = collect(); collect: while (true) { newCopy = collect(); for (int j = 0; j < n; j++) { if (oldCopy[j].label != newCopy[j].label) { }} return getValues(newCopy); }}}
148
Wait-free Scan
public int[] scan() { SnapValue[] oldCopy, newCopy; boolean[] moved = new boolean[n]; oldCopy = collect(); collect: while (true) { newCopy = collect(); for (int j = 0; j < n; j++) { if (oldCopy[j].label != newCopy[j].label) { }} return getValues(newCopy); }}}
149
Mismatch Detected
if (oldCopy[j].label != newCopy[j].label) { if (moved[j]) { // second move return newCopy[j].snap; } else { moved[j] = true; oldCopy = newCopy; continue collect; }}} return getValues(newCopy); }}}
150
Mismatch Detected
if (oldCopy[j].label != newCopy[j].label) { if (moved[j]) { return newCopy[j].snap; } else { moved[j] = true; oldCopy = newCopy; continue collect; If thread moved twice, }}} just steal its second return getValues(newCopy); snapshot }}}
151
Mismatch Detected
if (oldCopy[j].label != newCopy[j].label) { if (moved[j]) { // second move return newCopy[j].snap; } else { moved[j] = true; Remember that oldCopy = newCopy; thread moved continue collect; }}} return getValues(newCopy); }}}
152
Observations
Uses unbounded counters
can be replaced with 2 bits
153
Summary
We saw we could implement MRMW multi valued snapshot objects From SRSW binary safe registers (simple flipflops) But what is the next step to attempt with read-write registers?
154
Grand Challenge
Snapshot means
Write any one array element Read multiple array elements
155
Grand Challenge
What about atomic writes to multiple locations?
Writes to 0 and 1
Writes to 1 and 2
156
For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to
http://creativecommons.org/licenses/by-sa/3.0/.
Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights.
157