GRAPH ALGORITHMS

Dr. Tanzima Hashem

Assistant Professor
CSE, BUET
DEFINITION OF MST

 Let G=(V,E) be a connected, undirected graph.

 For each edge (u,v) in E, we have a weight w(u,v)
specifying the cost (length of edge) to connect u and v.
 We wish to find a (acyclic) subset T of E that connects
all of the vertices in V and whose total weight is
minimized.
 Since the total weight is minimized, the subset T must
be acyclic.
 Thus, T is a tree. We call it a minimum spanning tree.
 The problem of determining the tree T is called the
minimum-spanning-tree problem.
MINIMUM SPANNING TREES
 Spanning Tree
 A tree (i.e., connected, acyclic graph) which contains
all the vertices of the graph
 Minimum Spanning Tree
 Spanning tree with the minimum sum of weights
8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
 Spanning forest g
1
g 2
f
 If a graph is not connected, then there is a spanning
tree for each connected component of the graph
APPLICATION OF MST: AN EXAMPLE
 In the design of electronic circuitry, it is
often necessary to make a set of pins
electrically equivalent by wiring them
together.
 Running cable TV to a set of houses. What’s
the least amount of cable needed to still
connect all the houses?
PROBLEM: LAYING TELEPHONE WIRE

5
Central office
WIRING: NAÏVE APPROACH

6
Central office

Expensive!
WIRING: BETTER APPROACH

7
Central office

Minimize the total length of wire connecting the customers

EXAMPLE OF MST

Here is an example of a connected graph

and its minimum spanning tree:

8 7
b c d 9
4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

Notice that the tree is not unique:

replacing (b,c) with (a,h) yields another spanning tree
with the same minimum weight.
EXAMPLE OF MST

Here is an example of a connected graph

and its minimum spanning tree:

8 7
b c d 9
4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

Notice that the tree is not unique:

replacing (b,c) with (a,h) yields another spanning tree
with the same minimum weight.
GENERIC ALGORITHM
GENERIC_MST(G,w)
1 A:={}
2 while A does not form a spanning tree do
3 find an edge (u,v) that is safe for A
4 A:=A∪{(u,v)}
5 return A

 Set A is always a subset of some minimum

spanning tree.
 An edge (u,v) is a safe edge for A if by adding (u,v)
to the subset A, we still have a minimum spanning
tree.

10
HOW TO FIND A SAFE EDGE
We need some definitions and a theorem.
 A cut (S,V-S) of an undirected graph
G=(V,E) is a partition of V.
 An edge crosses the cut (S,V-S) if one of its
endpoints is in S and the other is in V-S.
 An edge is a light edge crossing a cut if its
weight is the minimum of any edge crossing
the cut.
HOW TO FIND A SAFE EDGE

8 7
b c d 9
4
2
S↑ 11 14 e ↑S
a i 4
7 6
10
8
V-S↓ h g f ↓ V-S
1 2

• This figure shows a cut (S,V-S) of the graph.

• The edge (d,c) is the unique light edge crossing the cut.
THE ALGORITHMS OF KRUSKAL AND PRIM
 The two algorithms are elaborations of the
generic algorithm.
 They each use a specific rule to determine a
safe edge in the GENERIC_MST.
 In Kruskal's algorithm,
 The set A is a forest.
 The safe edge added to A is always a least-
weight edge in the graph that connects two
distinct components.
 In Prim's algorithm,
 The set A forms a single tree.
 The safe edge added to A is always a least-
weight edge connecting the tree to a vertex not in
the tree.
KRUSKAL’S ALGORITHM
 Basic idea:
 Grow many small trees
 Find two trees that are closest (i.e., connected
with the lightest edge), join them with the
lightest edge
 Terminate when a single tree forms
CLAIM
 Ifedge (u, v) is the lightest among all
edges, (u, v) is in a MST
 Proof by contradiction:
 Suppose that (u, v) is not in any MST
 Given a MST T, if we connect (u, v), we create a
cycle
 Remove an edge in the cycle, have a new tree T’
 W(T’) < W(T)

By the same argument, the

second, third, …, lightest
u v
edges, if they do not create a
cycle, must be in MST
KRUSKAL’S ALGORITHM IN WORDS
 Procedure:
 Sort all edges into non-decreasing order
 Initially each node is in its own tree
 For each edge in the sorted list
 If the edge connects two separate trees, then

 join the two trees together with that edge

 EXAMPLE
c-d: 3
b-f: 5 a
b-a: 6
6 12
5 9
f-e: 7 b f g
b-d: 8 14 7
8 15
f-g: 9 c e h
d-e: 10
3 10
a-f: 12 d
b-c: 14
e-h: 15
 EXAMPLE
c-d: 3
b-f: 5 a
b-a: 6
6 12
5 9
f-e: 7 b f g
b-d: 8 14 7
8 15
f-g: 9 c e h
d-e: 10
3 10
a-f: 12 d
b-c: 14
e-h: 15
 EXAMPLE
c-d: 3
b-f: 5 a
b-a: 6
6 12
5 9
f-e: 7 b f g
b-d: 8 14 7
8 15
f-g: 9 c e h
d-e: 10
3 10
a-f: 12 d
b-c: 14
e-h: 15
 EXAMPLE
c-d: 3
b-f: 5 a
b-a: 6
6 12
5 9
f-e: 7 b f g
b-d: 8 14 7
8 15
f-g: 9 c e h
d-e: 10
3 10
a-f: 12 d
b-c: 14
e-h: 15
 EXAMPLE
c-d: 3
b-f: 5 a
b-a: 6
6 12
5 9
f-e: 7 b f g
b-d: 8 14 7
8 15
f-g: 9 c e h
d-e: 10
3 10
a-f: 12 d
b-c: 14
e-h: 15
 EXAMPLE
c-d: 3
b-f: 5 a
b-a: 6
6 12
5 9
f-e: 7 b f g
b-d: 8 14 7
8 15
f-g: 9 c e h
d-e: 10
3 10
a-f: 12 d
b-c: 14
e-h: 15
 EXAMPLE
c-d: 3
b-f: 5 a
b-a: 6
6 12
5 9
f-e: 7 b f g
b-d: 8 14 7
8 15
f-g: 9 c e h
d-e: 10
3 10
a-f: 12 d
b-c: 14
e-h: 15
 EXAMPLE
c-d: 3
b-f: 5 a
b-a: 6
6 12
5 9
f-e: 7 b f g
b-d: 8 14 7
8 15
f-g: 9 c e h
d-e: 10
3 10
a-f: 12 d
b-c: 14
e-h: 15
 EXAMPLE
c-d: 3
b-f: 5 a
b-a: 6
6 12
5 9
f-e: 7 b f g
b-d: 8 14 7
8 15
f-g: 9 c e h
d-e: 10
3 10
a-f: 12 d
b-c: 14
e-h: 15
 EXAMPLE
c-d: 3
b-f: 5 a
b-a: 6
6 12
5 9
f-e: 7 b f g
b-d: 8 14 7
8 15
f-g: 9 c e h
d-e: 10
3 10
a-f: 12 d
b-c: 14
e-h: 15
 EXAMPLE
c-d: 3
b-f: 5 a
b-a: 6
6 12
5 9
f-e: 7 b f g
b-d: 8 14 7
8 15
f-g: 9 c e h
d-e: 10
3 10
a-f: 12 d
b-c: 14
e-h: 15
DISJOINT-SET
 Keep a collection of sets S1, S2, .., Sk,
 Each Si is a set, e,g, S1={v1, v2, v8}.
 Three operations

 Make-Set(x)-creates a new set whose only

member is x.
 Union(x, y) –unites the sets that contain x and y,
say, Sx and Sy, into a new set that is the union of
the two sets.
 Find-Set(x)-returns a pointer to the
representative of the set containing x.

28
MULTIPLE OPERATIONS

 Suppose multiple operations:

 n: #MAKE-SET operations (executed at beginning).
 m: #MAKE-SET, UNION, FIND-SET operations.
 mn, #UNION operation is at most n-1.
 LINKED-LIST IMPLEMENTATION

 Each set as a linked-list, with head and tail, and each node
contains value, next node pointer and back-to
representative pointer.
 Example:
 MAKE-SET costs O(1): just create a single element list.
 FIND-SET costs O(1): just return back-to-representative
pointer.
 LINKED-LISTS FOR TWO SETS
Set {c,h,e}
head c h e

tail

Set {f, g}
head f g

tail

UNION of
two Sets

head f g c h e

tail
UNION IMPLEMENTATION

A simple implementation: UNION(x,y) just

appends x to the end of y, updates all back-to-
representative pointers in x to the head of y.
 Each UNION takes time linear in the x’s length.
 Suppose n MAKE-SET(xi) operations (O(1) each)
followed by n-1 UNION
 UNION(x1, x2), O(1),
 UNION(x2, x3), O(2),
 …..
 UNION(xn-1, xn), O(n-1)
 The UNIONs cost 1+2+…+n-1=(n2)
 So 2n-1 operations cost (n2), average (n) each.
WEIGHTED-UNION HEURISTIC

 The simple implementation is inefficient because

 We may be appending a longer list to a shorter
List during a UNION operation
 We must update the representative pointer of each
member of the longer list
 Weighted Union Heuristic
 Maintain the length of each list
 Always append the smaller list to the longer list
 With ties broken arbitrarily
 A sequence of m MAKE-SET, UNION & FIND-
SET operations, n of which are MAKE-SET
operations, takes O(m+nlgn) time
DISJOINT-SET IMPLEMENTATION: FORESTS
 Rooted trees, each tree is a set, root is the
representative. Each node points to its parent. Root
points to itself.

c cf cf

c d
h e d

Set {c,h,e} Set {f,d} h e UNION

STRAIGHTFORWARD SOLUTION

 Three operations
 MAKE-SET(x): create a tree containing x. O(1)
 FIND-SET(x): follow the chain of parent pointers
until to the root. O(height of x’s tree)
 UNION(x,y): let the root of one tree point to the root
of the other. O(1)
 It is possible that n-1 UNIONs results in a tree
of height n-1. (just a linear chain of n nodes).
 So n FIND-SET operations will cost O(n2).
UNION BY RANK & PATH COMPRESSION

 Union by Rank: Each node is associated with a

rank, which is the upper bound on the height of
the node (i.e., the height of subtree rooted at the
node), then when UNION, let the root with
smaller rank point to the root with larger rank.
 Path Compression: used in FIND-SET(x)
operation, make each node in the path from x to
the root directly point to the root. Thus reduce the
tree height.
PATH COMPRESSION
f f

e c d e

c
 ALGORITHM FOR DISJOINT-SET FOREST
UNION(x,y)
MAKE-SET(x) 1. LINK(FIND-SET(x),FIND-SET(y))
1. p[x]x
2. rank[x]0 LINK(x,y)
FIND-SET(x)
1. if x p[x]
1. if rank[x]>rank[y]
2. then p[x] FIND-SET(p[x])
2. then p[y] x
3. return p[x]
3. else p[x] y
4. if rank[x]=rank[y]
5. then rank[y]++

Worst case running time for m MAKE-SET, UNION, FIND-SET

operations is: O(m(n)) where (n)4. So nearly linear in m.
 KRUSKAL'S ALGORITHM

MST-Kruskal(G,w)
1 A  
2 for each vertex v  V[G] do
3 Make-Set(v) //creates set containing v (for initialization)
4 sort the edges of E
5 for each (u,v)E do
6 if Find-Set(u)  Find-Set(v) then // different component
7 A  A  {(u,v)}
8 Union(Set(u),Set(v)) // merge
9 return A
 EXAMPLE
c-d: 3 WITH DISJOINT SET UNION

b-f: 5 a
b-a: 6
6 12
5 9
f-e: 7 b f g
b-d: 8 14 7
8 15
f-g: 9 c e h
d-e: 10
3 10
a-f: 12 d
b-c: 14
e-h: 15 a b c d e f g h
 EXAMPLE
c-d: 3 WITH DISJOINT SET UNION

b-f: 5 a
b-a: 6
6 12
5 9
f-e: 7 b f g
b-d: 8 14 7
8 15
f-g: 9 c e h
d-e: 10
3 10
a-f: 12 d
b-c: 14
e-h: 15 a b c d e f g h
 EXAMPLE
c-d: 3 WITH DISJOINT SET UNION

b-f: 5 a
b-a: 6
6 12
5 9
f-e: 7 b f g
b-d: 8 14 7
8 15
f-g: 9 c e h
d-e: 10
3 10
a-f: 12 d
b-c: 14
e-h: 15 a b f c d e g h
 EXAMPLE
c-d: 3 WITH DISJOINT SET UNION

b-f: 5 a
b-a: 6
6 12
5 9
f-e: 7 b f g
b-d: 8 14 7
8 15
f-g: 9 c e h
d-e: 10
3 10
a-f: 12 d
b-c: 14
e-h: 15 b f a c d e g h
 EXAMPLE
c-d: 3 WITH DISJOINT SET UNION

b-f: 5 a
b-a: 6
6 12
5 9
f-e: 7 b f g
b-d: 8 14 7
8 15
f-g: 9 c e h
d-e: 10
3 10
a-f: 12 d
b-c: 14
e-h: 15 b f a e c d g h
 EXAMPLE
c-d: 3 WITH DISJOINT SET UNION

b-f: 5 a
b-a: 6
6 12
5 9
f-e: 7 b f g
b-d: 8 14 7
8 15
f-g: 9 c e h
d-e: 10
3 10
a-f: 12 d
b-c: 14
e-h: 15 b f a e c d g h
 EXAMPLE
c-d: 3 WITH DISJOINT SET UNION

b-f: 5 a
b-a: 6
6 12
5 9
f-e: 7 b f g
b-d: 8 14 7
8 15
f-g: 9 c e h
d-e: 10
3 10
a-f: 12 d
b-c: 14
e-h: 15 b f a e c d g h
 EXAMPLE
c-d: 3 WITH DISJOINT SET UNION

b-f: 5 a
b-a: 6
6 12
5 9
f-e: 7 b f g
b-d: 8 14 7
8 15
f-g: 9 c e h
d-e: 10
3 10
a-f: 12 d
b-c: 14
e-h: 15 b f a e c d g h
 KRUSKAL'S ALGORITHM

MST-Kruskal(G,w)
1 A  
2 for each vertex v  V[G] do
3 Make-Set(v) //creates set containing v (for initialization)
4 sort the edges of E
5 for each (u,v)E do
6 if Find-Set(u)  Find-Set(v) then // different component
7 A  A  {(u,v)}
8 Union(Set(u),Set(v)) // merge
9 return A
RUNNING TIME OF KRUSKAL’S ALGORITHM
 Kruskal’s Algorithm consists of two stages.
 Initializing the set A in line 1 takes O(1) time.
 Sorting the edges by weight in line 4.
 takes O(E lg E)

 Performing
 |V| MakeSet() operations for loop in lines 2-3.
 |E| FindSet(), for loop in lines 5-8.
 |V| - 1 Union(), for loop in lines 5-8.
 which takes O(E lg E)

 The total running time is

 O(E lg E)
 We have lg │E│ = O(lg V) because # of E = V-1
 So total running time becomes O(E lg V).
PRIM’S ALGORITHM
 Basic idea:
 Start from an arbitrary single node
 A MST for a single node has no edge

 Gradually build up a single larger and larger

MST

6
5
Not yet discovered
7

Fully explored nodes

Discovered but not fully explored nodes
PRIM’S ALGORITHM
 Basic idea:
 Start from an arbitrary single node
 A MST for a single node has no edge

 Gradually build up a single larger and larger

MST

6 2
5 9
4 Not yet discovered
7

Fully explored nodes

Discovered but not fully explored nodes
PRIM’S ALGORITHM
 Basic idea:
 Start from an arbitrary single node
 A MST for a single node has no edge

 Gradually build up a single larger and larger

MST

6 2
5 9
4
7
NOTATION
 Tree-vertices: in the tree constructed so far
 Non-tree vertices: rest of vertices

Prim’s Selection rule

• Select the minimum weight edge between a
tree-node and a non-tree node and add to
the tree

53
PRIM’S ALGORITHM IN WORDS
 Randomly pick a vertex as the initial tree T
 Gradually expand into a MST:

 For each vertex that is not in T but directly

connected to some nodes in T
 Compute its minimum distance to any vertex

in T
 Select the vertex that is closest to T
 Add it to T
EXAMPLE
a
6 12
5 9
b f g
14 7
8 15
c e h
3 10
d
EXAMPLE
a
6 12
5 9
b f g
14 7
8 15
c e h
3 10
d
EXAMPLE
a
6 12
5 9
b f g
14 7
8 15
c e h
3 10
d
EXAMPLE
a
6 12
5 9
b f g
14 7
8 15
c e h
3 10
d
EXAMPLE
a
6 12
5 9
b f g
14 7
8 15
c e h
3 10
d
EXAMPLE
a
6 12
5 9
b f g
14 7
8 15
c e h
3 10
d
EXAMPLE
a
6 12
5 9
b f g
14 7
8 15
c e h
3 10
d
EXAMPLE
a
6 12
5 9
b f g
14 7
8 15
c e h
3 10
d
EXAMPLE
a
6 12
5 9
b f g
14 7
8 15
c e h
3 10
d
EXAMPLE
a
6 12
5 9
b f g
14 7
8 15
c e h
3 10
d
EXAMPLE
a
6 12
5 9
b f g
14 7
8 15
c e h
3 10
d
EXAMPLE
a
6 12
5 9
b f g
14 7
8 15
c e h
3 10
d
PRIM ALGORITHM:VARIABLES
 r:Grow the minimum spanning tree from the root vertex “r”.

 Q: is a priority queue, holding all vertices that are not in the

tree now.

 key[v]: is the minimum weight of any edge connecting v to a

vertex in the tree.

 p [v]: names the parent of v in the tree.

 T[v] – Vertex v is already included in MST if T[v]==1,

otherwise, it is not included yet.

 Removing v from set Q adds it to set Q-V of vertices in tree,

thus adding (v, p[ v]) to A.
PRIM ALGORITHM (2)
MST-Prim(G,r)
01 Q  V[G] // Q – vertices out of T
02 for each u  Q
03 key[u]  
04 key[r]  0 // r is the first tree node, let r=1
05 p[r]  NIL
06 while Q   do
07 u  ExtractMin(Q) // making u part of T
08 for each v  Adj[u] do
09 if v  Q and w(u,v) < key[v] then
10 p[v]  u
11 key[v]  w(u,v)
 The execution of Prim's algorithm
8 7
the root b c d 9
vertex 4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

V a b c d e f g h i
T 1 0 0 0 0 0 0 0 0
Key 0 - - - - - - - -
p -1 - - - - - - - -
69
 The execution of Prim's algorithm
8 7
the root b c d 9
vertex 4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

V a b c d e f g h i
T 1 0 0 0 0 0 0 0 0
Key 0 4 - - - - - 8 -
p -1 a - - - - - a -

70
 The execution of Prim's algorithm
8 7
the root b c d 9
vertex 4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2
Important: Update Key[v] only if T[v]==0

V a b c d e f g h i
T 1 1 0 0 0 0 0 0 0
Key 0 4 8 - - - - 8 -
p -1 a b - - - - a -

71
 The execution of Prim's algorithm
8 7
the root b c d 9
vertex 4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

V a b c d e f g h i
T 1 1 1 0 0 0 0 0 0
Key 0 4 8 7 - 4 - 8 2
p -1 a b c - c - a c

72
 The execution of Prim's algorithm
8 7
the root b c d 9
vertex 4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

V a b c d e f g h i
T 1 1 1 0 0 0 0 0 1
Key 0 4 8 7 - 4 6 7 2
p -1 a b c - c i i c

73
 The execution of Prim's algorithm
8 7
the root b c d 9
vertex 4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

V a b c d e f g h i
T 1 1 1 0 0 1 0 0 1
Key 0 4 8 7 10 4 2 7 2
p -1 a b c f c f i c

74
 The execution of Prim's algorithm
8 7
the root b c d 9
vertex 4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

V a b c d e f g h i
T 1 1 1 0 0 1 1 0 1
Key 0 4 8 7 10 4 2 1 2
p -1 a b c f c f g c

75
 The execution of Prim's algorithm
8 7
the root b c d 9
vertex 4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

V a b c d e f g h i
T 1 1 1 0 0 1 1 1 1
Key 0 4 8 7 10 4 2 1 2
p -1 a b c f c f g c

76
 The execution of Prim's algorithm
8 7
the root b c d 9
vertex 4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

V a b c d e f g h i
T 1 1 1 1 0 1 1 1 1
Key 0 4 8 7 9 4 2 1 2
p -1 a b c d c f g c

77
 The execution of Prim's algorithm
8 7
the root b c d 9
vertex 4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

V a b c d e f g h i
T 1 1 1 1 1 1 1 1 1
Key 0 4 8 7 9 4 2 1 2
p -1 a b c d c f g c

78
COMPLEXITY: PRIM ALGORITHM
MST-Prim(G,r)
01 Q  V[G] // Q – vertices out of T
02 for each u  Q
O(V)
03 key[u]  
04 key[r]  0
05 p[r]  NIL
06 while Q   do O(V)
07 u  ExtractMin(Q) // making u part of T Heap: O(lgV)
08 for each v  Adj[u] do Overall: O(E)
09 if v  Q and w(u,v) < key[v] then
10 p[v]  u
11 key[v]  w(u,v) Decrease Key: O(lgV)

Overall complexity: O(V)+O(V lg V+E lg V) = O(E lg V)

 SUMMARY
Kruskal’s algorithm Prim’s algorithm

1. Select the shortest edge in a 1. Select any vertex

network
2. Select the shortest edge
2. Select the next shortest edge connected to that vertex
which does not create a cycle
3. Select the shortest edge
3. Repeat step 2 until all connected to any vertex
vertices have been connected already connected

4. Repeat step 3 until all

vertices have been
connected
SUMMARY
 Both are greedy algorithms
 Both have the same output MST

 Kruskal’s begins with forest and merge into a tree

 Prim’s always stays as a tree

THE END