Академический Документы
Профессиональный Документы
Культура Документы
1
Announcements!
• HW3 due Friday
• Follow the Piazza HW3 post for updates – we fixed some
typos and also added a hint.
• HW4 released Friday
2
Prologue
• Real quick recap of hash functions!
• Go to section this week for a longer recap!
3
Hash tables
• U is the universe. It has size M.
• As hash table stores items of U in n buckets.
• A hash table comes with a hash function ℎ: # → 1, … , (
which says what element goes in what bucket.
1 n buckets
22
2
13 )= 2
h (2 2 3
h (1
3) =
3
Universe U 4
Hash families
• A hash family is a collection of hash functions.
• For example:
• H = { all of the functions h:U -> {1,..,n} }
• H = { least-significant-digit, most-significant-digit }
• We can build a hash table by drawing a random
function uniformly at random from a hash family
and using that hash function.
5
Example 2. You, the algorithm, chooses a random hash
function ℎ: ( → {0, … , 9}. Choose it
h 0 = Most_significant_digit
h 1 = Least_significant_digit randomly from H.
H = {h 0, h 1}
uk
ui 9 19 99 6
Example 2. You, the algorithm, chooses a random hash
function ℎ: ( → {0, … , 9}. Choose it
h 0 = Most_significant_digit
h 1 = Least_significant_digit randomly from H.
H = {h 0, h 1}
1 11 101
2 141
uj
…
121
uk
9
ui 131 7
Example 2. You, the algorithm, chooses a random hash
function ℎ: ( → {0, … , 9}. Choose it
h 0 = Most_significant_digit
h 1 = Least_significant_digit randomly from H.
H = {h 0, h 1}
1 11 101
2 That’s bad! It takes a 141
long time to
uj
…
search/insert/delete!
121
uk
9
ui 131 8
Universal hash family
• H is a universal hash family if, when h is chosen
uniformly at random from H,
for all &' , &) ∈ +with &' ≠ &) ,
1
12∈3 ℎ &' = ℎ &) ≤
8
• In English: For any pair of items in the universe, the
probability that they end up in the same bucket is small.
• We came up with this definition because it implied that
(in expectation) not too many items land in any given
bucket.
• There exist universal hash families of size O(M2 ).
9
Hashing a universe of size M into n
buckets, where at most n of the
items in M ever show up.
Conclusion
• We can build a hash table that:
• supports INSERT/DELETE/SEARCH in O(1)
expected time
• requires O(n log(M)) bits of space.
• O(n) buckets
• O(n) items with log(M) bits per item
• O(log(M)) bits to store the hash function
10
Questions about hash functions?
• Section this week
• Office hours
• CLRS
• (Optional) lecture notes posted on website
11
More detailed schedule on the website!
Roadmap
re s
u
2
c t
le
e R a n d o m ize d
5l
ct
ur
A lg s
es
str
Da ture
Asymptotic
1 st
uc
ta s
c la
ss
Analysis Sorting
We
D iv id e an d en ces
c u rr
are
co nq ue r Re
MIDTERM
her
e
Dynamic
Greedy Algs Programming
Longe Graphs!
st, Sh
Max a ortest
nd Mi ,
1l
n… 9 le c t u
r es The
ec
tu
12
Future!
re
Outline
• Part 0: Graphs and terminology
13
Part 0: Graphs
14
Graphs
Citation graph of
literary theory
academic papers
16
Graphs
Theoretical Computer
Science academic
communities
17
Game of Thrones Character
Graphs Interaction Network
18
Graphs jetblue flights
19
Graphs Complexity Zoo
containment graph
20
Graphs debian dependency (sub)graph
21
Graphs
Immigration
flows
22
Graphs Potato trade
23
Graphs
Soybeans
Water
24
Graphs
Graphical models
25
Graphs
26
Graphs Neural connections
in the brain
27
Graphs
• There are a lot of graphs.
• We want to answer questions about them.
• Efficient routing?
• Community detection/clustering?
• From pre-lecture exercise:
• Computing Bacon numbers
• Signing up for classes without violating pre-req constraints
• How to distribute fish in tanks so that none of them will fight.
28
Undirected Graphs 2
• Example 4
• V = {1,2,3,4}
• E = { {1,3}, {2,4}, {3,4}, {2,3} }
• Example 4
• V = {1,2,3,4}
• E = { (1,3), (2,4), (3,4), (4,3), (3,2) }
• The in-degree of vertex 4 is 2.
• The out-degree of vertex 4 is 1.
• Vertex 4’s incoming neighbors are 2,3
• Vertex 4’s outgoing neighbor 30is 3.
How do we represent graphs?
• Option 1: adjacency matrix
2
1 2 3 4 1
0 0 1 0
1
3
0 0 1 1
2
4
1 1 0 1
3
4
0 1 1 0 31
How do we represent graphs?
• Option 1: adjacency matrix
2
1 2 3 4 1
1 0 1 0
1
3
0 0 1 1
2
4
1 1 0 1
3
4
0 1 1 0 32
How do we represent graphs?
• Option 1: adjacency matrix 2
Destination
1 2 3 4 1
0 0 1 0 3
Source
1 2 3
0 0 0 1 4
0 1 0 1
0 0 1 0
4
33
How do we represent graphs?
• Option 2: adjacency lists.
2
1 2 3 4
1
3
3 4 1 2 4
35
G e n e ra
l l y b e tt e
fo r s p a r s r
Trade-offs 1 2
e g ra p h
3 4
s
0 0 1 0
Say there are n vertices 0 0 1 1 3 4 1 2
and m edges. 1 1 0 1
0 1 1 0 3 4 3
2
Edge membership O(deg(v)) or
Is e = {v,w} in E? O(1) O(deg(w))
Neighbor query
Give me v’s neighbors.
O(n) O(deg(v))
Space requirements
O(n2) O(n + m)
See Lecture 9 IPython notebook for an actual We’ll assume this
implementation! representation for
36
the rest of the class
Part 1: Depth-first search
37
How do we explore a graph?
At each node, you can get a list of neighbors,
and choose to go there if you want.
6
5 7
2
1 8
3
4
38
Depth First Search
Exploring a labyrinth with chalk and a piece of string
39
Depth First Search
Exploring a labyrinth with chalk and a piece of string
40
Depth First Search
Exploring a labyrinth with chalk and a piece of string
41
Depth First Search
Exploring a labyrinth with chalk and a piece of string
42
Depth First Search
Exploring a labyrinth with chalk and a piece of string
43
Depth First Search
Exploring a labyrinth with chalk and a piece of string
44
Depth First Search
Exploring a labyrinth with chalk and a piece of string
45
Depth First Search
Exploring a labyrinth with chalk and a piece of string
46
Depth First Search
Exploring a labyrinth with chalk and a piece of string
47
Depth First Search
Exploring a labyrinth with chalk and a piece of string
48
Depth First Search
Exploring a labyrinth with chalk and a piece of string
49
Depth First Search
Exploring a labyrinth with chalk and a piece of string
50
Depth First Search
Exploring a labyrinth with chalk and a piece of string
51
Depth First Search
Exploring a labyrinth with chalk and a piece of string
52
Depth First Search
Exploring a labyrinth with chalk and a piece of string
53
Depth First Search
Exploring a labyrinth with chalk and a piece of string
yrinth:
Lab !
ored
expl
54
Depth First Search
Exploring a labyrinth with pseudocode
• Each vertex keeps track of whether it is:
• Unvisited
• In progress
• All done
64
DFS finds all the nodes reachable
from the starting point
In an undirected graph, this is
called a connected component.
start
start
start
66
Why is it called depth-first?
• We are implicitly building a tree:
YOINK! D E A
F
B
A E
C
B
68
Running time
To explore just the connected component we started in
or
72
Application of DFS: topological sorting
• Find an ordering of vertices so that all of the
dependency requirements are met.
• Aka, if v comes before w in the ordering, there is not an
edge from w to v.
coreutils
multiarch-
support
tar
dpkg
libselinux1
libbz2 Suppose the dependency graph has no cycles:
it is a Directed Acyclic Graph (DAG) 73
Can’t always eyeball it.
74
Let’s do DFS What do you notice about the
finish times? Any ideas for how
we should do topological sort?
start:9
finish:10
coreutils
start:7
finish:8 multiarch
-support
tar start:3
finish:4
dpkg start:0
finish:11
libselinux1
libbz2 start:2
start:1 finish:5
finish:6
75
Suppose the underlying
graph has no cycles
Finish times seem useful
Claim: In general, we’ll always have:
A B
finish: [larger] finish: [smaller]
• Then
B.startTime A.finishTime
A.startTime B.finishTime
B
• aka, B.finishTime < A.finishTime. 79
Theorem
• If we run DFS on a directed acyclic graph,
If A B
Then B.finishTime < A.finishTime
80
Back to
If A B
Then B.finishTime < A.finishTime
topological sorting
• In what order should I install packages?
• In reverse order of finishing time in DFS!
start:3
start:9
finish:4
coreutils finish:10
start:7 multiarch-
finish:8 support
tar
start:3
start:9
finish:4
coreutils finish:10
start:7 multiarch-
finish:8 support
tar
83
What did we just learn?
• DFS can help you solve the topological sorting
problem
• That’s the fancy name for the problem of finding an
ordering that respects all the dependencies
84
This example skipped in
Example: class – here for reference.
B
Unvisited
In progress
All done
A
Start:0
85
This example skipped in
Example class – here for reference.
B
Unvisited
In progress
All done
A
Start:0
Start:1
86
This example skipped in
Example class – here for reference.
B
Unvisited
In progress
All done
A
Start:0
Start:1
Start:2 87
This example skipped in
Example class – here for reference.
Start:3
B
Unvisited
In progress
All done
A
Start:0
Start:1
Start:2 88
This example skipped in
Example class – here for reference.
Start:3
B Leave:4
Unvisited
In progress
All done
A
Start:0
Start:1
D B
Start:2 89
This example skipped in
Example class – here for reference.
Start:3
B Leave:4
Unvisited
In progress
All done
A
Start:0
Start:1
D D B
Start:2
90
Leave:5
This example skipped in
Example class – here for reference.
Start:3
B Leave:4
Unvisited
In progress
All done
A
Start:0
Start:1
Leave: 6
D C D B
Start:2
91
Leave:5
This example skipped in
Example class – here for reference.
Start:3
B Leave:4
Unvisited
In progress
All done
A
Start:0
Leave: 7
C
Start:1
Leave: 6 Do them in this order:
D A C D B
Start:2
92
Leave:5
Another use of DFS
that we’ve already seen
• In-order enumeration of binary search trees
HINT for Problem 6 on the
3 7
2 4 8
Do DFS and print a node’s
label when you are done with
the left child and before you
1 begin the right child.
93
Part 2: breadth-first search
94
How do we explore a graph?
If we can fly
5
4
1 3
7
8
6
2
9
95
Breadth-First Search
Exploring the world with a bird’s-eye view
Not been there yet
96
Breadth-First Search
Exploring the world with a bird’s-eye view
Not been there yet
97
Breadth-First Search
Exploring the world with a bird’s-eye view
Not been there yet
98
Breadth-First Search
Exploring the world with a bird’s-eye view
Not been there yet
99
Breadth-First Search
Exploring the world with a bird’s-eye view
Not been there yet
World:
lored!
exp 100
Same disclaimer as for DFS: you may have seen other ways to implement this,
this will be convenient for us.
Breadth-First Search
Exploring the world with pseudocode
• Set Li = [] for i=1,…,n Li is the set of nodes
• L0 = [w], where w is the start node we can reach in i
• Mark w as visited steps from w
• For i = 0, …, n-1:
• For u in Li:
• For each v which is a neighbor of u: -
• If v isn’t yet visited:
• mark v as visited, and put it in Li+1 L0
L1
Go through all the nodes L2
in Li and add their
unvisited neighbors to Li+1 101 L3
BFS also finds all the nodes
reachable from the starting point
start
A
F L1 B
E
B L2 C
F
D
C
L3 G Call this the
“BFS tree”
• First we go as broadly as we can.
104
Pre-lecture exercise
• What Samuel L. Jackson’s Bacon number?
c
r a ssi
Ju k
r
Pa
Samuel L.
Kevin Tre Jackson
m o rs
Bacon
Ariana Richards
(Answer:
105 2)
I wrote the pre-lecture exercise
before I realized that I really wanted
an example with distance 3
James
X- m McAvoy
en
Kevin
Bacon ia
a rn
N
B j o r k M et
W h e n ro u g h Oliver Sacks It is really hard to find
Tilda bo
A tt e n people with Bacon
Swinton 106
number 3!
Application of BFS: shortest path
• How long is the shortest path between w and v?
v
107
Application of BFS: shortest path
• How long is the shortest path between w and v?
some proof!
See skipped
• A shortest path between w and v is
slide. given by the path in the BFS tree. L1
• If we never found v, the distance
is infinite.
L2
M odify the BFS pseudocode
to return shortest paths!
111
Another application of BFS
• Testing bipartite-ness
112
Pre-lecture exercise: fish
• You have a bunch of fish and two fish tanks.
• Some pairs of fish will fight if put in the same tank.
• Model this as a graph: connected fish will fight.
• Can you put the fish in the two tanks so that there is no fighting?
113
Bipartite graphs
Can color the vertices red
and orange so that there
• A bipartite graph looks like this: are no edges between any
same-colored vertices
Example:
are in tank A
are in tank B
if the fish fight
Example:
are students
are classes
if the student is
enrolled in the114class
Is this graph bipartite?
115
How about this one?
116
How about this one?
117
This one?
118
Application of BFS:
Testing Bipartiteness
• Color the levels of the BFS tree in
alternating colors.
• If you never color two connected
A
nodes the same color, then it is
bipartite.
B
• Otherwise, it’s not. E
F
C D
G
119
Breadth-First Search
For testing bipartite-ness
Not been there yet
120
Breadth-First Search
For testing bipartite-ness
Not been there yet
121
Breadth-First Search
For testing bipartite-ness
Not been there yet
122
Breadth-First Search
For testing bipartite-ness
Not been there yet
123
Breadth-First Search
For testing bipartite-ness
Not been there yet
IPARTITE!
LEARLY B
C 124
Breadth-First Search
For testing bipartite-ness
Not been there yet
125
Breadth-First Search
For testing bipartite-ness
Not been there yet
126
Breadth-First Search
For testing bipartite-ness
Not been there yet
127
Breadth-First Search
For testing bipartite-ness
Not been there yet
128
Breadth-First Search
For testing bipartite-ness
Not been there yet
O A N OT two steps
WH
IPAR TITE! Can reach there in
B three steps
129
Hang on now.
• Just because this coloring doesn’t
work, why does that mean that
there is no coloring that works?
I can come up
with plenty of bad
colorings on this
legitimately
bipartite graph…
Plucky the
pedantic penguin
130
Make this proof
sketch formal!
Some proof required
Ollie the over-achieving ostrich
There must
B be an even
E
number of start
these edges
F
C D
132
What have we learned?
BFS can be used to detect
bipartite-ness in time O(n + m).
133
Outline
• Part 0: Graphs and terminology
Recap 134
Recap
• Depth-first search
• Useful for topological sorting
• Also in-order traversals of BSTs
• Breadth-first search
• Useful for finding shortest paths
• Also for testing bipartiteness
• Both DFS, BFS:
• Useful for exploring graphs, finding connected
components, etc
135
Still open (next few classes)
• We can now find components in undirected graphs…
• What if we want to find strongly connected components
in directed graphs?
136
Next Time
• Strongly Connected Components
137