Randomized Algorithms

南京大学

尹一通
 Course Info
• 尹一通
yitong.yin@gmail.com
yinyt@nju.edu.cn
• office hour:
804, Thursday 2-4pm
• course homepage: http://tcs.nju.edu.cn/wiki
 Textbooks
Rajeev Motwani and Prabhakar Raghavan.
Randomized Algorithms.
Cambridge University Press, 1995.

Michael Mitzenmacher and Eli Upfal.

Probability and Computing:
Randomized Algorithms and Probabilistic Analysis.
Cambridge University Press, 2005.
References
CLRS

Feller
An Introduction to Probability
Theory and Its Applications

Aldous and Fill

Reversible Markov Chains and
Random Walks on Graphs

Alon and Spencer

The Probabilistic Method
 Randomized Algorithms
“algorithms which use randomness in computation”

Turing
Machine random
coin

Why?
• Simpler.
• Faster.
• Can do impossibles.
• Can give us clever deterministic algorithms.
• Random input.
• Deterministic problem with random nature.
• ... ...
Checking Matrix Multiplication
three n×n matrices A, B, C:

?
A × B = C

best matrix multiplication algorithm: O(n 2.373

)
Checking Matrix Multiplication
three n×n matrices A, B, C:

?
A × B
O(n )
= C

best matrix multiplication algorithm: O(n 2.373

)
Checking Matrix Multiplication
three n×n matrices A, B, C:

?
A × B = C

best matrix multiplication algorithm: O(n 2.373

)

Freivald’s Algorithm
pick a uniform random r ∈{0,1}n;
check whether A(Br) = Cr ;

time: O(n2) if AB = C, always correct

 Freivald’s Algorithm
pick a uniform random r ∈{0,1}n;
check whether A(Br) = Cr ;

if AB = C, always correct
if AB ≠ C, let D = AB C = 0n n
say Dij = 0
n 1
2 1
Pr[ABr = Cr] = Pr[Dr = 0] =
2n 2
n
(Dr)i = Dik rk =0
D r k=1
1
n
i rj = Dik rk
Dij
k=j
 Freivald’s Algorithm
pick a uniform random r ∈{0,1}n;
check whether A(Br) = Cr ;

if AB = C, always correct

Theorem (Freivald, 1979)

If AB ⇥= C, for a uniformly random r {0, 1}n ,
1
Pr[ABr = Cr] .
2

repeat independently for 100 times

time: O(n2) if AB ≠ C, error probability ≤ 2 100
 Monte Carlo vs Las Vegas
Two types of randomized algorithms:

Monte Carlo Las Vegas

running time is fixed always correct

correctness is random running time is random
 Polynomial Identity Testing
(PIT)
Input: two polynomials f, g F[x] of degree d
Output: f g?
d
X
f F[x] of degree d : f (x) = ai x i
for ai 2 F
i=0

Input: a polynomial f F[x] of degree d

Output: f 0?

f is given as black-box
 Input: a polynomial f F[x] of degree d
Output: f 0?

simple deterministic algorithm:

check whether f(x)=0 for all x {1, 2, . . . , d + 1}

Fundamental Theorem of Algebra:

A degree d polynomial has at most d roots.

Randomized Algorithm
pick a uniform random r ∈S ; S F
check whether f(r) = 0 ;
 Randomized Algorithm
pick a uniform random r ∈S ; S F
check whether f(r) = 0 ; |S| = 2d
if f 0
d 1
Pr[f (r) = 0] =
|S| 2

Fundamental Theorem of Algebra:

A degree d polynomial has at most d roots.
 Checking Identity
北京

database 1
Are they
identical?
南京

database 2
Communication Complexity
(Yao 1979)
f (a, b)
# of bits
communicated
a b

Han Meimei Li Lei

n n
EQ : {0, 1} × {0, 1} → {0, 1}
!
Theorem (Yao, 1979) 1 a = b
EQ(a, b) =
There is no deterministic communication
0 a ̸= b protocol
solving EQ with less than n bits in the worst-case.
 Communication Complexity
n 1 n
X1
i f(r)=g(r) ? i
f= ai x g= bi x
i=0 i=0
r, g(r) n
a ∈{0, 1} n b ∈{0, 1}

Han Meimei Li Lei

pick uniform
by PIT: random r ∈[2n]
1
one-sided error
2
# of bit communicated: too large!
 Communication Complexity
n 1 n
X1
i f(r)=g(r) ? i
f= ai x g= bi x
i=0 i=0
r, g(r) n
a ∈{0, 1} n b ∈{0, 1}
O(log n) bits
Han Meimei Li Lei

pick uniform
random r ∈[p]
k = log2 (2n)

choose a prime p k
[2 , 2 k+1
] let f, g Zp [x]
 Polynomial Identity Testing
(PIT)
Input: f, g 2 F[x1 , x2 , . . . , xn ] of degree d
Output: f g?

F[x1 , x2 , . . . , xn ] : ring of n-variate polynomials over field F

f 2 F[x1 , x2 , . . . , xn ] :
X
i1 i2 in
f (x1 , x2 , . . . , xn ) = ai1 ,i2 ,...,in x1 x2 · · · xn
i1 ,i2 ,...,in 0

degree of f : maximum i1 + i2 + · · · + in with ai1 ,i2 ,...,in 6= 0

 Input: f, g 2 F[x1 , x2 , . . . , xn ] of degree d
Output: f g?

equivalently:

Input: f 2 F[x1 , x2 , . . . , xn ] of degree d

Output: f 0?

X
i1 i2 in
f (x1 , x2 , . . . , xn ) = ai1 ,i2 ,...,in x1 x2 · · · xn
i1 ,i2 ,...,in 0
i1 +i2 +···+in d
 Input: f, g 2 F[x1 , x2 , . . . , xn ] of degree d
Output: f g?

equivalently:

Input: f 2 F[x1 , x2 , . . . , xn ] of degree d

Output: f 0?

f is given as block-box: given any ~x = (x1 , x2 , . . . , xn )

returns f (~x)
or as product from: e.g. Vandermonde determinant
2 n 13
1 x1 x21 ... x1
61 x2 x22 ... xn2 1 7
Y
6 7 f (~x) = det(M ) = (xi xj )
M = 6. .. .. .. .. 7
4 .. . . . . 5
j<i
1 xn x2n ... xnn 1
PIT: Polynomial Identity Testing
Input: f 2 F[x1 , x2 , . . . , xn ] of degree d
Output: f 0?

f is given as block-box or product from

if ∃ a poly-time deterministic algorithm for PIT:

either: NEXP ≠ P/poly
or: #P ≠ FP
 Input: f 2 F[x1 , x2 , . . . , xn ] of degree d
Output: f 0?

fix an arbitrary S ✓ F
pick random r1, r2, ... , rn ∈S
uniformly and independently at random;
check whether f(r1, r2, ... , rn) = 0 ;

f ⌘0 f (r1 , r2 , . . . , rn ) = 0
 Input: a polynomial f F[x] of degree d
Output: f 0?

fix an arbitrary S ✓ F
pick a uniform random r ∈S;
check whether f(r) = 0 ;
f ⌘0 f (r) = 0

Fundamental Theorem of Algebra:

A degree d polynomial has at most d roots.

d
f 6⌘ 0 Pr[f (r) = 0] 
|S|
 Input: f 2 F[x1 , x2 , . . . , xn ] of degree d
Output: f 0?

fix an arbitrary S ✓ F
pick random r1, r2, ... , rn ∈S
uniformly and independently at random;
check whether f(r1, r2, ... , rn) = 0 ;

f ⌘0 f (r1 , r2 , . . . , rn ) = 0

Schwartz-Zippel Theorem
d
f 6⌘ 0 Pr[f (r1 , r2 , . . . , rn ) = 0] 
|S|
 Schwartz-Zippel Theorem
d
f 6⌘ 0 Pr[f (r1 , r2 , . . . , rn ) = 0] 
|S|
X
i1 i2 in
f (x1 , x2 , . . . , xn ) = ai1 ,i2 ,...,in x1 x2 · · · xn
i1 ,i2 ,...,in 0
i1 +i2 +···+in d

f can be treated as a single-variate polynomial of xn:

d
X
i
f (x1 , x2 , . . . , xn ) = xn fi (x1 , x2 , . . . , xn 1 )
i=0
= gx1 ,x2 ,...,xn 1 (xn )

Pr[f (r1 , r2 , . . . , rn ) = 0] = Pr[gr1 ,r2 ,...,rn 1 (rn ) = 0]

gr1 ,r2 ,...,rn 1

6⌘ 0? done?
 Schwartz-Zippel Theorem
d
f 6⌘ 0 Pr[f (r1 , r2 , . . . , rn ) = 0] 
|S|

induction on n :

basis: n=1 single-variate case, proved by

the fundamental Theorem of algebra

I.H.: Schwartz-Zippel Thm is true for all smaller n

 Schwartz-Zippel Theorem
d
f 6⌘ 0 Pr[f (r1 , r2 , . . . , rn ) = 0] 
|S|

induction step:
n
fk 6⌘ 0
k: highest power of xn in f
degree of fk  d k
k
X
f (x1 , x2 , . . . , xn ) = xin fi (x1 , x2 , . . . , xn 1)
i=0

= k
xn fk (x1 , x2 , . . . , xn 1 ) + f¯(x1 , x2 , . . . , xn )
k
X1
where f¯(x1 , x2 , . . . , xn ) = i
xn fi (x1 , x2 , . . . , xn 1 )
i=0

highest power of xn in f¯ < k

 Schwartz-Zippel Theorem
d
f 6⌘ 0 Pr[f (r1 , r2 , . . . , rn ) = 0] 
|S|

f (x1 , x2 , . . . , xn ) = xkn fk (x1 , x2 , . . . , xn 1 ) + f¯(x1 , x2 , . . . , xn )

n f 6⌘ 0
k
highest power of xn in f¯ < k
degree of fk  d k

law of total probability:

kd
Pr[f (r1 , r2 , . . . , rn ) = 0] I.H. 
|S|

= Pr[f (~r) = 0 | fk (r1 , . . . , rn 1) = 0] · Pr[fk (r1 , . . . , rn 1) = 0]

+ Pr[f (~r) = 0 | fk (r1 , . . . , rn 1) 6= 0] · Pr[fk (r1 , . . . , rn 1) 6= 0]
k
= Pr[gr1 ,...,rn 1
(rn ) = 0 | fk (r1 , . . . , rn 1 ) 6= 0] 
|S|
where gx1 ,...,xn 1 (xn ) = f (x1 , . . . , xn )
 Schwartz-Zippel Theorem
d
f 6⌘ 0 Pr[f (r1 , r2 , . . . , rn ) = 0] 
|S|

d k k d
Pr[f (r1 , r2 , . . . , rn ) = 0]  + =
|S| |S| |S|
 Input: f 2 F[x1 , x2 , . . . , xn ] of degree d
Output: f 0?

fix an arbitrary S ✓ F
pick random r1, r2, ... , rn ∈S
uniformly and independently at random;
check whether f(r1, r2, ... , rn) = 0 ;

f ⌘0 f (r1 , r2 , . . . , rn ) = 0

Schwartz-Zippel Theorem
d
f 6⌘ 0 Pr[f (r1 , r2 , . . . , rn ) = 0] 
|S|