[next] [prev] [prev-tail] [tail] [up]

5.2 A Time Hierarchy

Lower Bounds on Time Complexity Tractability and Intractability Intuitively, it seems obvious that some problems require more time to solve than others. The following result confirms this intuitive assessment while implying the existence of a time hierarchy for the class of language recognition problems. Definitions A function T(n) is said to be time-constructible if there exists a T(n) timebounded, deterministic Turing machine that for each n has an input of length n on which it makes exactly T(n) moves. The function is said to be fully time-constructible if there exists a deterministic Turing machine that makes exactly T(n) moves on each input of length n. A function S(n) is said to be space-constructible if there exists an S(n) space-bounded, deterministic Turing machine that for each n has an input of length n on which it requires exactly S(n) space. The function is said to be fully space-constructible if there exists a deterministic Turing machine that requires exactly S(n) space on each input of length n. Example 5.2.1 The deterministic Turing machine M in Figure 5.2.1

Figure 5.2.1 A T(n) = 2n time-bounded, deterministic Turing machine. makes exactly t(x) = |x| + (number of 1's in x) moves on a given input x. t(x) = 2|x| when x contains no 0's, and t(x) < 2|x| when x contains 0's. The existence of M implies that T(n) = 2n is a time-constructible function, because a. M is 2n time-bounded, and b. For each n there exists the input 1n of length n on which M makes exactly 2n moves. The existence of the deterministic Turing machine M does not imply that 2n is fully timeconstructible, because M does not make exactly 2n moves on each input of length n. However, M can be modified to show that 2n is a fully time-constructible function.

Convention In this section Mx denotes a Turing machine that is represented by the string x of the following form. If x = 1jx0 for some j 0 and for some standard binary representation x0 of a deterministic Turing machine M, then Mx denotes M. Otherwise, Mx denotes a deterministic Turing machine that accepts no input. The string x is said to be a padded binary representation of Mx. Theorem 5.2.1 Consider any function T1(n) and any fully time-constructible function T2(n), that for each c > 0 have an nc such that T2(n) c(T1(n))2 for all n nc. Then there is a language which is in DTIME (T2(n)) but not in DTIME (T1(n)). Proof Let T1(n) and T2(n) be as in the statement of the theorem. Let U be a universal Turing machine similar to the universal Turing transducer in the proof of Lemma 5.1.1. The main difference is that here U assumes an input (M, x) in which M is represented by a padded binary representation instead of a standard binary representation. U starts each computation by going over the "padding" 1j until it reaches the first 0 in the input. Then U continues with its computation in the usual manner while ignoring the padding. U uses a third auxiliary work tape for keeping track of the distance of its input head from the end of the padding. The result is shown by diagonalization over the language L = { v | v is in {0, 1}*, and U does not accept (Mv, v) in T2(|v|) time }. L is obtained from the diagonal of the table Tuniversal (see Figure 5.2.2).

Figure 5.2.2

Hypothetical table Tuniversal indicating membership in the language { (Mw, u) | U accepts (Mw, u) in T2(|u|) time }.

In the table Tuniversal the entry at row Mw and column u is equal to 1 if U accepts (Mw, u) in T2(|u|) time, and it is equal to 0 if U does not. The proof relies on the observation that each O(T1(n))

time-bounded, deterministic Turing machine Mx0 that accepts L has also a padded representation x for which U can simulate the whole computation of Mx on x in T2(|x|) time. Consequently, Mx accepts x if and only if U does not accept (Mx, x) or, equivalently, if and only if Mx does not accept x. Specifically, for the purpose of showing that L is not in DTIME (T1(n)), assume to the contrary that L is in the class. Under this assumption, there is a dT1(n) time-bounded, deterministic Turing machine M that accepts L, for some constant d. Let x0 be a standard binary representation of M, and c be the corresponding constant cM implied by Lemma 5.1.1 for the representation x0 of M. Let x = 1jx0 for some j that satisfies j + c(dT1(j + |x0|))2 T2(j + |x0|), that is, x = 1jx0 for a large enough j to allow U sufficient time T2(|x|) for simulating the whole computation of Mx on input x. Such a value j exists because for big enough j the following inequalities hold. j+c
2

(j + |x 0|) + c T1(j + |x0|) + c (1 + cd2) T2(j + |x0|)

2

2 2

Consider the string x = 1jx0. By definition, |x| = j + |x0| and so j + c(dT1(|x|))2 T2(|x|). Moreover, x is a padded binary representation of M. For the string x one of the following two cases must hold. However, neither of them can hold, so implying the desired contradiction to the assumption that L is in DTIME (T1(n)). Case 1 x is in L. The assumption together with L = L(Mx) imply that Mx accepts x in dT1(|x|) time. In such a case, by Lemma 5.1.1 U accepts (Mx, x) in j + c(dT1(|x|))2 T2(|x|) time. On the other hand, x in L together with the definition of L imply that U does not accept x in T2(|x|) time. The contradiction implies that this case cannot hold. Case 2 x is not in L. The assumption together with L = L(Mx) imply that Mx does not accept x. In such a case, U does not accept (Mx, x) either. On the other hand, x not in L together with the definition of L imply that U accepts (Mx, x). The contradiction implies that this case cannot hold either. To show that L is in DTIME (T2(n)) consider the deterministic four auxiliary-work-tape Turing machine M that on input x proceeds according to the following algorithm. Step 1 M stores (Mx, x) in its first auxiliary work tape. That is, M stores the string x, followed by the separator 01, followed by the representation 011 of the left endmarker , followed by x, followed by the representation 0111 of the right endmarker $. In addition, M encloses the sequence of strings above between the "left endmarker" and the "right endmarker" , respectively. Step 2 M computes the value of T2(|x|) and stores it in the second auxiliary work tape. Step 3

M follows the moves of U on the content of its first auxiliary work tape, that is, on (Mx, x). M uses its third and fourth auxiliary work tapes for recording the content of the two auxiliary work tapes of U. During the simulation M interprets as the left endmarker , and as the right endmarker $. M halts in an accepting configuration if it determines that U does not reach an accepting state in T2(|x|) moves. Otherwise, M halts in a nonaccepting configuration. By construction, the Turing machine M is of O(T2(|x|)) time complexity. The fully timeconstructibility of T2(n) is required for Step 2. Example 5.2.2 Let T1(n) = nk and T2(n) = 2n. T1(n) and T2(n) satisfy the conditions of Theorem 5.2.1. Therefore the class DTIME (2n) properly contains the class DTIME (nk). Lower Bounds on Time Complexity In addition to implying the existence of a time hierarchy for the language recognition problems, Theorem 5.2.1 can be used to show lower bounds on the time complexity of some problems. Specifically, consider any two functions T1(n) and T2(n) that satisfy the conditions of Theorem 5.2.1. Assume that each membership problem Ki for a language in DTIME (T2(n)) can be reduced by a T3(n) time-bounded, deterministic Turing transducer Mi to some fixed problem K (see Figure 5.2.3).

Figure A set of Turing transducers M1, M2, . . . for reducing the problems K1, K2, . . . in DTIME (T2(n)) to a given language recognition problem K. Each Mi on instance x of 5.2.3 Ki provides an instance y of K, where K has the answer yes for y if and only if Ki has the answer yes for x. In addition, assume that each such Mi on input x of length n provides an output y of length f(n) at most. Then the membership problems for the languages in DTIME (T2(n)) are decidable in T3(n)

+ T(f(n)) time if K can be solved in T(n) time. In such a case, a lower bound for the time complexity T(n) of K is implied, since by Theorem 5.2.1 the class DTIME (T2(n)) contains a problem that requires more than cT1(n) time for each constant c, that is, the inequality T3(n) + T(f(n)) > cT1(n) must hold for infinitely many n's. The lower bound is obtained by substituting m for f(n) to obtain the inequality T(m) > cT1(f-1(m)) - T3(f-1(m)) or, equivalently, the inequality T(n) > cT1(f-1(n)) - T3(f-1(n)). Example 5.2.3 Consider the time bounds T1(n) = 2an, T2(n) = 2bn for b > 2a, and T3(n) = f(n) = n log n. For such a choice, T3(n) + T(f(n)) > cT1(n) implies that n log n + T(n log n) > c2an. By substituting m for n log n it follows that T(m) > c2an - m = c2am/log n - m c2am/log m - m 2dm/log m or, equivalently, that T(n) > 2dn/log n for some constant d. The approach above for deriving lower bounds is of special interest in the identification of intractable problems, that is, problems that require impractical amounts of resources to solve. Such an identification can save considerable effort that might otherwise be wasted in trying to solve intractable problems. Tractability and Intractability In general, a problem is considered to be tractable if it is of polynomial time complexity. This is because its time requirements grow slowly with input length. Conversely, problems of exponential time complexity are considered to be intractable, because their time requirements grow rapidly with input length and so can be practically solved only for small inputs. For instance, an increase by a factor of 2 in n, increases the value of a polynomial p(n) of degree k by at most a factor of 2k. On the other hand, such an increase at least squares the value of 2p(n). The application of the approach above in the identification of intractable problems employs polynomially time-bounded reductions. A problem K1 is said to be polynomially time reducible to a problem K2 if there exist polynomially time-bounded, deterministic Turing transducers Tf and Tg that for each instance I1 of K1 satisfy the following conditions (see Figure 5.2.4).

Figure 5.2.4

Reduction by polynomially time-bounded, deterministic Turing transducers Tf and Tg.

a. Tf on input I1 gives an instance I2 of K2. b. K1 has a solution S1 at I1 if and only if K2 has a solution S2 at I2, where S1 is the output of Tg on input S2. In the case that K1 and K2 are decision problems, with no loss of generality it can be assumed that Tg computes the identity function g(S) = S, that is, that Tg on input S2 outputs S1 = S2. A given complexity class C of problems can be used to show the intractability of a problem K by showing that the following two conditions hold. a. C contains some intractable problems. b. Each problem in C is polynomially time reducible to K, that is, K is at least as hard to solve as any problem in C. Once a problem K is determined to be intractable, it then might be used to show the intractability of some other problems by showing that K is polynomially time reducible to . In such a case, the easier K is the easier the reductions are, and the larger the class of such applicable problems is. The observation above sparks our interest in the "easiest" intractable problems K, and in the complexity classes C whose intractable problems are all "easiest" intractable problems. In what follows, a problem K is said to be a C-hard problem with respect to polynomial time reductions, or just a C-hard problem when the polynomial time reductions are understood, if every problem in the class C is polynomially time reducible to the problem K. The problem K is said to be C-complete if it is a C-hard problem in C. Our interest here is in the cases that C = NP and C = PSPACE. [next] [prev] [prev-tail] [front] [up] [next] [prev] [prev-tail] [tail] [up]

5.4 More NP-Complete Problems

The 0 - 1 Knapsack Problem From Boolean Expression to a System of Linear Equations From a System of Linear Equations to an Instance of the 0 - 1 Knapsack Problem The Clique Problem The NP-hardness of the satisfiability problem was demonstrated by exhibiting the existence of a polynomial time reduction, from each problem in NP to the satisfiability problem. A similar approach was used for showing the NP-hardness of the 3-satisfiability problem. However, in

general the proof of the NP-hardness of a given problem need not be generic in nature, but can be accomplished by polynomial time reduction from another NP-hard problem. A proof by reduction is possible because the composition of polynomial time reductions is also a polynomial time reduction. That is, if a problem Ka is reducible to a problem Kb in T1(n) time, and Kb is reducible to a problem Kc in T2(n) time, then Ka is reducible to Kc in T2(T1(n)) time. Moreover, T2(T1(n)) is polynomial if T1(n) and T2(n) are so. The 0 - 1 Knapsack Problem The proofs of the following two theorems exhibit the NP-hardness of the problems in question by means of reduction. Theorem 5.4.1 The problem defined by the following pair, called the 0 - 1 knapsack problem, is an NP-complete problem. Domain: { (a1, . . . , aN , b) | N 1, and a1, . . . , aN , b are natural numbers }. Question: Are there v1, . . . , vN in {0, 1} such that a1v1 + + aN vN = b for the given instance (a1, . . . , aN , b)? Proof Consider a Turing machine M that on any given instance (a1, . . . , aN , b) of the problem nondeterministically assigns values from {0, 1} to v1, . . . , vN , checks whether a1v1 + + aN vN = b, and accepts the input if and only if the equality holds. M can be of polynomial time complexity. Therefore the 0 - 1 knapsack problem is in NP. To show that the 0 - 1 knapsack problem is NP-hard consider any instance E of the 3satisfiability problem. Let x1, . . . , xm denote the variables in the Boolean expression E. E is a conjunction c1 ck of some clauses c1, . . . , ck. Each Ci is a disjunction ci 1 ci 2 ci 3 of some literals ci 1, ci 2, ci 3. Each ci j is a variable xt, or a negation xt of a variable xt, for some 1 t m. From Boolean Expression to a System of Linear Equations From the Boolean expression E a system S of linear equations of the following form can be constructed. x1 + xm +
1

=1 =1

c1 1 + c1 2 + c1 3 + y1 1 + y1 2 = 3 ck 1 + ck 2 + ck 3 +yk 1 +yk 2 = 3

The system S has the variables x1, . . . , xm, 1, . . . , m, y1 1, . . . , yk 2. The variable xt in S corresponds to the literal xt in E. The variable t in S corresponds to the literal xt in E. ci j stands for the variable xt in S, if xt is the jth literal in Ci. ci j stands for the variable t in S, if xt is the jth literal in Ci. Each equation of the form xi + i = 1 has a solution over {0, 1} if and only if either xi = 1 and = 0, or xi = 0 and i = 1. Each equation of the form ci 1 + ci 2 + ci 3 + yi 1 + yi 2 = 3 has a solution over {0, 1} if and only if at least one of the equalities ci 1 = 1, ci 2 = 1, and ci 3 = 1 holds. It follows that the system S has a solution over {0, 1} if and only if the Boolean expression E is satisfiable. From a System of Linear Equations to an Instance of the 0 - 1 Knapsack Problem The system S can be represented in a vector form as follows.
i

The variables z1, . . . , z2m+2k in the vector form stand for the variables x1, . . . , xm, 1, . . . , m, y1 1, . . . , yk 2 of S, respectively. ai j is assumed to be the coefficient of zj in the ith equation of S. bi is assumed to be the constant in the right-hand side of the ith equation in S. Similarly, the system S can also be represented by the equation H of the following form.

In H, each aj stands for the integer whose decimal representation is a1 j am+k j. Similarly, b stands for the integer whose decimal representation is b1 bm+k. The representation is possible because the sum ai 1 + + ai 2m+2k is either equal to 2 or to 5 for each 1 i m + k. That is, the ith digit in the sum c = a1 + + a2m+2k depends only on the ith digits of a1, . . . , a2m+2k. It follows that S is satisfiable over {0, 1} if and only if H is satisfiable over {0, 1}. As a result, the instance E of the 3-satisfiability problem is satisfiable if and only if the instance (a1, . . . , a2m+2k, b) of the 0 - 1 knapsack problem has a positive solution. Moreover, a polynomially time-bounded, deterministic Turing transducer can similarly construct corresponding instance of the 0 - 1 knapsack problem, from each instance E of the 3-satisfiability problem. Consequently, the NP-hardness of the 0 - 1 knapsack problem follows from the NPhardness of the 3-satisfiability problem. Example 5.4.1 Consider the Boolean expression E of the form (x1 x2 x3) (x2 x3 x4) (x1 x3 x4) (x1 x2 x4). E is an instance of the 3-satisfiability problem. The Boolean expression is satisfiable if and only if the following system S of linear equations has a solution over {0, 1}.

On the other hand, the system S has a solution over {0, 1} if and only if the equation H of the following form has a solution over {0, 1}. The leading zeros are ignored in the constants of H.

The expression E is satisfiable if and only if the instance (10001010, 1001001, 100110, 10011, 10000001, 1000100, 101000, 10100, 1000, 1000, 100, 100, 10, 10, 1, 1, 11113333) of the 0 - 1 knapsack problem has a positive solution. The Clique Problem The previous examples of NP-complete problems deal with Boolean expressions and linear equations. The following example deals with graphs. Theorem 5.4.2 The problem defined by the following pair, called the clique problem, is an NP-complete problem. Domain: { (G, k) | G is a graph and k is a natural number }. Question: Does G has a clique of size k for the given instance (G, k)? (A clique is a subgraph with an edge between each pair of nodes. The number of nodes in a clique is called the size of the clique.) Proof Consider a Turing machine M that on a given instance (G, k) of the clique problem proceeds as follows. M starts by nondeterministically choosing k nodes in G. Then it determines whether there is an edge in G between each pair of the k chosen nodes. If so, then M accepts the input; otherwise it rejects the input. M is of polynomial time complexity. Consequently the clique problem is in NP.

To show the NP-hardnes of the clique problem consider any instance E of the 3-satisfiability problem. As in the proof of the previous result, let x1, . . . , xm denote the variables in the Boolean expression E. E is a conjunction c1 ck of some clauses c1, . . . , ck. Each Ci is a disjunction ci 1 ci 2 ci 3 of some literals ci 1, ci 2, ci 3. Each ci j is a variable xt, or a negation xt of a variable xt, for some 1 t m. From the Boolean expression E a graph G of the following form can be constructed. The graph G has a node corresponding to each pair (ci, (d1, d2, d3)) of an assignment (d1, d2, d3) that satisfies a clause Ci. The node that corresponds to a pair (ci, (d1, d2, d3)) is labeled by the set {xi 1 = d1, xi 2 = d2, xi 3 = d3}, where xi 1, xi 2, xi 3 are assumed to be the variables used in ci 1, ci 2, ci 3, respectively. It follows that for each Ci, the graph G has seven associated nodes. The graph G has an edge between a node labeled by a set {xi 1 = d1, xi 2 = d2, xi 3 = d3} and a node labeled by a set {xj 1 = d'1, xj 2 = d'2, xj 3 = d'3} if and only if no variable xt has conflicting assignments in the two sets, 1 t m. By construction, no pair of nodes associated with the same clause Ci have an edge between them. On the other hand, the edges between the nodes that correspond to each pair of clauses, relate exactly those assignments to the variables that satisfy both clauses simultaneously. Consequently, the Boolean expression E is satisfiable if and only if G has a clique of size k. A polynomially time-bounded, deterministic Turing transducer can in a similar way determine a corresponding instance (G, k) of the clique problem for each instance E of the 3-satisfiability problem. Therefore, implying the NP-hardness of the clique problem. Example 5.4.2 Let E be the Boolean expression (x1 x2 x3) (x2 x3 x4) (x1 x3 x4) (x1 x2 x4). Let G be the graph in Figure 5.4.1.

Figure 5.4.1

A graph G which relates the assignments that satisfy the clauses of the Boolean expression (x1 x2 x3) (x2 x3 x4) (x1 x3 x4) (x1 x2 x4).

Then by the proof of the last theorem, E is satisfiable if and only if (G, 4) is satisfiable. The assignment (x1, x2, x3, x4) = (1, 0, 0, 1) that satisfies E corresponds to the clique in G whose nodes are shaded. From the definition of NP-completeness, it follows that P is equal to NP if and only if there is an NP-complete problem in P.

It should be noticed that all the known algorithms, for the NP-complete problems, are in essence based on exhaustive search over some domain. For instance, in the case of the satisfiability problem, an exhaustive search is made for an assignment to the variables that satisfies the given expression. In the case of the 0 - 1 knapsack problem, the exhaustive search is made for a subset of a given multiset {a1, . . . , aN }, whose values sum up to some given value b. In the case of the clique problem, the exhaustive search is made for a clique of the desired size. In all of these cases the search is over a domain of exponential size, and so far it seems this is the best possible for the NP-complete problems. [next] [prev] [prev-tail] [front] [up]