(Algorithms)

Course 9

NP Theory

An Introduction to the Theory of NP

Outlines

Polynomial Time
Intractability
Optimization Problems vs. Decision Problems
The Theory of NP

P problem
NP problem
NP-complete problem
NP-hard problem

NP-complete

Polynomial Time ()

(Polynomial Time)?

Polynomial
Time

Non-Polynomial
Time

Def:

(Polynomial-time Algorithm)
(input size)

(Worst-case)

ninput size p (n)

Polynomial-time

computable

f(x) polynomial-time computable

xPolynomial Time f

Intractability ()

[ 1]:

[ 2]:

(tractable) (intractable)

(Worst-case)
(Intractable)

(Worst-case)
Polynomial-Time Algorithm
Polynomial-Time Algorithm

Intractable.

For example:

brute-force algorithm ()
(Chained Matrix Multiplication problem) non-polynomial
time.

dynamic programming algorithm (Algorithm 3.6)

(n3).

Problems for which polynomial-time algorithms have been found

MST

Problems that have been proven to be intractable

()

(Halting
Problem)(Equivalence Problem)

Problems that have not been proven to be intractable, but for which
polynomial-time algorithms have never been found

(Traveling
Salesman Problem)

The Traveling Salesman Problem; TSP

TSP

n(A salesman spends

his time visiting n cities cyclically. )

(In one tour he visits

each city exactly once, and finishes up where he started.)

()
(In what order should he visit the cities to minimize the distance
traveled?)

10

TSP

(Exponentially) !!

3 cities 1 solution.

10 cities 181,440 possible tours

n cities (n-1)!/2 possible tours

11

n=26 25! /2
25!=155112100433309859840000001.55 x 1025

106 3.15 x 107

3.15 x 1013

n=26

12

Optimization Problem vs. Decision Problem

(Optimization Problem)
: (Decision Problem)

Decision Problem:
yes no

13

The

partition problem ():

S={a1, a2, , an}:

S1S2

Ex: Let S = {13, 2, 17, 20, 8}. The answer to this problem instance is "yes"
because we can partition S into S1 = {13, 17} and S2 = {2, 20, 8}.

The

Sum of Subset Problem ():

S={a1, a2, , an}c: S

SSc

Ex: Let S = {12, 9, 33, 42, 7, 10, 5} and c = 24.

The answer of this problem instance is "yes" as there exists S = {9, 10, 5}
and the sum of the elements in S is equal to 24.
If c is 6, the answer will be "no".

14

The Satisfiability Problem (; SAT):

EETrue
FalseTrue

Ex: Let E = (-x1 x2 - x3)(x1 -x2 ) (x2 x3). Then the following assignment
will make E true and the answer will be yes.
x1 F, x2 F, x3 T

If E is -x1 x1 , there will be no assignment which can make E true and the

answer will be no.

NP-Complete (by S. A. Cook, 1971).

The Minimal Spanning Tree Problem ():

Given a graph G, find a spanning tree T of G with the minimum length.

15

The

Traveling Salesperson Problem ():

G = (V,E)
cyclecycle

Ex: Consider following graph. There are two cycles satisfying our
condition. They are C1 = abedcfa and C2 = acbe
dfa. C1 is shorter and is the solution of this problem instance.

16

Some

problems:

The Partition Problem

The Sum of Subset Problem

The Satisfiability Problem

The Minimal Spanning Tree Problem

The Traveling Salesperson Problem

(Decision Problems)

(Optimization Problems)

Problems)
(Decision Problems)!!

(Optimization

17

For

example (MST Problem):

Given a graph G and a constant c.

The total length of the spanning tree of the graph G is a:

If a < c, then the answer is yes,

otherwise, its answer is no.

(The minimal spanning tree decision problem)

version of the
optimization problem)

(Decision

18

The Theory of NP

19

20

!!

21

(TSP)

TSP

22

!!

(
)
()

23

24

Computer Science

()

25

Deterministic v.s. Non-deterministic

Yes/No

(Deterministic Algo.)

(Non-deterministic Algo.)

26

Deterministic Algorithm ()

Def:
(Permitting at most one next move at any step in a

computation)

(Deterministic Machine)

27

Non-deterministic Algorithm ()

Def:
(Permitting more than one choice of

next move at some step in a computation)

(Nondeterministic Machine)

28

Non-deterministic

Algorithm

(Guess)

Non-deterministic

(
)

(Verification)

(True)

29

Nondeterministic SAT
nE

(SAT)

O(2n)

O(n)

/* Guess */

for i = 1 to n do
xi choice(true, false);
/* Verification */

if E(x1, x2, ,xn) is true then

success;

else failure;

30

Polynomial-Time Reducible ()
Def:

Q1Q2L1L2

Q1Q2

(L1 pL2 Q1 pQ2)

f(x) /**/

f(x) polynomial-time computable

f(x) xxL1 f(x)L2 (xL1 f(x)L2)

Q1

L1

f(x)

Q2

L2

L1f(x)L2
L1f(x)L2
Reduce

Q1Q2

31

Reduce?

Q1 reduce Q2Q1Q2

Q1Input

Reduce
f(x)

Q2Input

Q2Algo.

Reduce
Q2Output
xL1 f(x)L2

Q1Output

32

Q1Q2Q2
Q2Q1

f(x) polynomial-time computable

Q1 f(x) + Q2

Q2
Q1

L1 L2 Q1 Q2

33

Q1

Q2

Q2Q1 ()

Q2Q1

Q1Q2Q1 pQ2 Q2 pQ1

Q1 Q2Q2 Q3Q1 Q3 ()

Q1: 4?

Q2:4G=(V,E)?

Q1 Q2

34

i vi

i j (vi, vj)E

i j (vi, vj)E

f polynomial time computable

G=(V,E) |V|2
O(|V|2)
fPolynomial time computable

f(x)xxL1 f(x)L2

G
()I j k vi vjvk vi vj
vk
()()I j k

Q1 Q2

35

P, NP, NP hard, NP complete

P:

Decision ProblemDeterministic
AlgorithmWorst CasePolynomial Time

NP:

NP Non-deterministicPolynomial

Polynomial Time
(Verify)

PNP

PNP ()

P=NP (?)

36

NP-hard:

Decision Problem Q NP-hard

NPDecision Problem Q (QNP) Q
(QQ)

NP-complete:

Decision Problem QNP-complete

:

QNP

QNP-hard

37

NP

NP-hard
NPcomplete

P
NP-hardNP-complete

NP-completeNP-hard ()
NP-hardNP-complete (
NP)

NP-complete NP-hard

NP-hard
NP-complete

38

Summary
Nearly
In

all of the decision problems are NP problems.

NP problems, there are some problems which have

polynomial algorithms. They are called P problems.

Every P problem must be an NP problem.

There

are a large set of problems which, up to now, have no

polynomial algorithms.

39

Some important properties of NP-complete problems:

Up to now, no NP-complete problem has any worst case polynomial

algorithm.

If any NP-complete problem can be solved in polynomial time, NP = P.

If the decision version of an optimization problem is NP-complete,

this optimization problem is called NP-hard.

We can conclude that all NP-complete and NP-hard

problems must be difficult problems because

They do not have polynomial algorithms at present.

It is quite unlikely that they can have polynomial algorithms in the future.

40

NP-complete
NPC

NP-completeNPcomplete
NP-complete

QNPC

() QNP (can guess an answer, and check it in polynomial time)

() NPC Q QQ

f(x)

f(x)polynomial-time computable

f(x)xxL1 f(x)L2
(L1QL2Q)

41

QNPC

1 QNP

2 QNP-hard

NP-completeNPNP-hard
Q NP-complete Q NP
NP-hard
Q NP-hardNP

NP Q QQ
NP Q (NP Q Q)
Q NP-hard

QNPNP-hardQNPcomplete

42

(Cooks Theorem)

SATNP-Complete
(SATNPC)

NPC

NPC

NPC

()

43

3-SATNP-Complete

Clique () NP-Complete

Vertex-Cover () NP-Complete

Dominating Set () NP-Complete

44

3-SATNP-Complete

3-SAT:

SATSAT
3E
TrueFalseTrue

Ex: Let E = (-x1 x2 - x3)(x1 -x2 - x4) (-x5 x2 x3). Then the
following assignment will make E true and the answer will be yes.
x1 T, x2 T, x3 T, x4 F, x5 F

QNPC

QNP (can guess an answer, and check it in polynomial time)

NPC QQQ

f(x)

f(x)polynomial-time computable

f(x)xxL1 f(x)L2

45

QNP

Can guess an answer, and check it in polynomial time

O(n)
O(1)Polynomial Time
NP
/* Guess */
for i = 1 to n do
xi choice(true, false);
/* Verification */
if E(x1, x2, ,xn) is true then
success;
else failure.

46

QQ
SATNPC

problem (by )

Q3-SATQ
QQ:

f(x)QQ

f(x)polynomial-time computable

f(x)xxL1 f(x)L2
(L1QL2Q)

47

1f(x)QQ

SATE E
E E E

Q Q f(x)

(x1)

(x1 y1 y2) (x1 1 y2) (x1 y1 2) (x1 1 2)

(x1 x2)

(x1 x2 y1) (x1 x2 1)

(x1 x2 x3)

(x1 x2 xk)

(x1 x2 y1) (1 x3 y2) (2 x4 y3)

(k-4 xk-2 yk-3) (k-3 xk-1 xk)

QQf(x)

48

2f(x)polynomial-time computable

SAT E n k
E 3-SAT E
O(nk)

SAT E

(
)

(k > 3) k-2
O(k)

n O(nk)

f(x)polynomial-time computable

49

3f(x)xxL1 f(x)L2
(L1QL2Q)

SAT E True
3-SAT E True

(E is satisfiable E is satisfiable )
E is satisfiable E is satisfiable
E is True

SAT E TrueTrue

TrueTrue

E is True

SAT E True3-SAT
E True E True

SATTrue xi yi False
SATFalse yi True E
True

50

E is satisfiable E is satisfiable

3-SAT E TrueTrue

True yi xi

E is True

True

E is True

3-SAT E TrueSAT
E True

SAT E
3-SAT E

SAT 3-SAT (Q Q)

3-SAT NP-Complete

Problem

51

An instance E in SAT :

The

(x1 x2)(-x3) (x1 -x2 x3 -x4 x5 x6)

x1 v x2
-x3
x1 v -x2 v x3 v -x4 v x5 v x6

instance E in 3-SAT :
x1 x2 y1
x1 x2 -y1
-x3 y2 y3
-x3 -y2 y3
-x3 y2 -y3
-x3 -y2 -y3
x1 -x2 y4
-y4

x3 y5

-y5 -x4 y6

-y6 x5 x6

SAT
E

f(x)

3-SAT
E

52

(Approximation Algorithm)

QNP-complete
(Polynomial
Time)

NP-complete!!

Approximation Algorithm

Issue:

(Polynomial Time)

53

Approximation Ratio
Approximation

Ratio

Opt(x)AA(x)
A-approximation
max(

A(x)
Opt(x)

Opt(x)
A(x)

) , for all x.

Approximation Ratio

()

54

Approximation ratio

A(x)
Opt(x)

Opt(x)
A(x)

A(x)
Opt(x)

TSPNP-complete

A(x)
Opt(x)

[Algo. 1] (9.6)minapprox < 2 mindist

[Algo. 2] (9.7)minapprox2 < 1.5 mindist

A(x)

Opt(x)

Opt(x)
A(x)

Approximation ratio
Opt(x)
A(x)

Opt(x)
A(x)