DESIGN AND ANALYSIS OF ALGORITHMS

UNIT IV BACKTRACKING

Objectives

Ø Describe about backtracking

Ø Explain how to construct state space tree
Ø Explain how to solve n-queen problem
Ø Compare backtracking with greedy and dynamic programming
Ø Solve and construct state space tree for subset sum problems using backtracking
Ø Explain about graph coloring

Backtracking
Definition: Backtracking is a process where steps are taken towards the final solution and the
details are recorded. If these steps do not lead to a solution some or all of them may have to be
retraced and the relevant details discarded. In theses circumstances it is often necessary to search
through a large number of possible situations in search of feasible solutions.

1. General method
• Useful technique for optimizing search under some constraints
• Express the desired solution as an n-tuple (x1, . . . , xn) where each xi 2 Si, Si being a finite set
• The solution is based on finding one or more vectors that maximize, minimize, or satisfy a
criterion function P(x1, . . . , xn)
• Sorting an array a[n]
– Find an n-tuple where the element xi is the index of ith smallest element in a
– Criterion function is given by a[xi] _ a[xi+1] for 1 _ i < n
– Set Si is a finite set of integers in the range [1,n]
• Brute force approach
– Let the size of set Si be mi
– There are m = m1m2 · · ·mn n-tuples that satisfy the criterion function P
– In brute force algorithm, you have to form all the m n-tuples to determine the optimal
solutions
• Backtrack approach
– Requires less than m trials to determine the solution
– Form a solution (partial vector) and check at every step if this has any chance of
success
– If the solution at any point seems not-promising, ignore it
– If the partial vector (x1, x2, . . . , xi) does not yield an optimal solution, ignore mi+1 ·
· ·mn possible test vectors
even without looking at them

2. Solution space and tree organization

State-space search methods in problem solving have often been illustrated using tree diagrams.
We explore a set of issues related to coordination in collaborative problem solving and design,

Design and Analysis of algorithms-P.Vasantha Kumari 1

and we present a variety of interactive features for state-space search trees intended to facilitate
such activity.

A node in the state space tree is promising if it corresponds to the partials constructed solution
that may lead to the complete solution otherwise the nodes are called non-promising. Leaves of
the tree represent either the non- promising dead end or complete solution found by the
algorithm.

The tree organization of the solution space is state space tree

Each node in the state space tree defines problem state

Solution state are those problem states s for which the path from the root to s
defines a tuple in the solution space

Answer state are those solution states s for which the path from root to s defines a
tuple that is a member of solutions (it satisfies the implicit constraints)

Solution space is partitioned into disjoint sub-solution space at each internal node

Static vs. dynamic tree
 Static trees are independent of the problem instance
 Dynamic trees are dependent of the problem instance

A node which has been generated and all of whose children have not yet been
generated is called a live node

The live node whose children are currently being generated is called E-node

A dead node is a generated node which is not to be expanded further or all of
whose children have been generated

3. The Eight Queens problem

Place 8 queens in a chessboard so that no two queens are in the same row, column, or diagonal.

Design and Analysis of algorithms-P.Vasantha Kumari 2

Formulation :
Ø States: any arrangement of 0 to 8 queens on the board
Ø Initial state: 0 queens on the board
Ø Successor function: add a queen in any square
Ø Goal test: 8 queens on the board, none attacked

Idea of solution:

• Each recursive call attempts to place a queen in a specific column

– A loop is used, since there are 8 squares in the column
• For a given call, the state of the board from previous placements is known (i.e. where
are the other queens?)
• Current step backtracking: If a placement within the column does not lead to a
solution, the queen is removed and moved "down" the column
• Previous step backtracking: When all rows in a column have been tried, the call
terminates and backtracks to the previous call (in the previous column)
• Pruning: If a queen cannot be placed into column i, do not even try to place one onto
column i+1 – rather, backtrack to column i-1 and move the queen that had been placed
there
• Using this approach we can reduce the number of potential solutions even more.

Algorithm
void NQueens(int k, int n)
// Using backtracking, this procedure prints all
// possible placements of n queens on an nXn
// chessboard so that they are nonattacking.
{
for (int i=1; i<=n; i++) {
if (Place(k, i)) {
x[k] = i;
if (k==n) { for (int j=1;j<=n;j++)
cout << x[j] << ' '; cout << endl;}
else NQueens(k+1, n);
}
}
}

bool Place(int k, int i)

// Returns true if a queen can be placed in kth row and
// ith column. Otherwise it returns false. x[] is a
// global array whose first (k-1) values have been set.
// abs(r) returns the absolute value of r.
{
for (int j=1; j < k; j++)
if ((x[j] == i) // Two in the same column
|| (abs(x[j]-i) == abs(j-k))) // or in the same diagonal
return(false);
return(true);
}

Design and Analysis of algorithms-P.Vasantha Kumari 3

State Space Tree of the Four-queens Problem

4. Subset sum problem

An instance of the Subset Sum problem is a pair (S, t), where S = {x1, x2, ..., xn} is a set of
positive integers and t (the target) is a positive integer. The decision problem asks for a subset of
S whose sum is as large as possible, but not larger than t. This problem is NP-complete.
In sum of subset problem, there are n positive integers wi and a positive integer W. The goal is to
find the all subsets of the integers that sum to W. It is similar to the 0-1 knapsack problem, of
which the solution is only one. If the values and weights of each item are equal, we have the sum
of subsets problem.

• Problem: Given n positive integers w1, ... wn and a positive integer S. Find all subsets of
w1, ... wn that sum to S.
• Example:
n=3, S=6, and w1=2, w2=4, w3=6

Design and Analysis of algorithms-P.Vasantha Kumari 4

 • Solutions:
{2,4} and {6}
• We will assume a binary state space tree.
• The nodes at depth 1 are for including (yes, no) item 1, the nodes at depth 2 are for item
2, etc.
• The left branch includes wi, and the right branch excludes wi.
• The nodes contain the sum of the weights included so far

First sort the items so that weight is non decreasing.

● Two conditions that allow backtracking:
1)At level i, if the total weight is not W, and adding wi+1 would bring the total weight above W.
● (because all weights after wi+1 are >= wi+1)
2)At level i, if the total weight is not W, and all following weights can't bring it to W.

Algorithm : sum of subsets

void sum_of_subsets (index i, int weight, int total)

{
if (promising(i, weight, total))
{
if (weight == W)
cout << include[1] through include[i];
else
{
include[i+1] = true;
sum_of_subsets(i+1, weight + w[i+1], total w[i+1]);
include[i+1] = false;
sum_of_subsets(i+1, weight, total w[i+1]);
}
}

void bool promising (index i, int weight, int total)

{
return ((weight + total >= W) &&(weight == W || weight + w[i+1] <= W));

Example

The figure shows the pruned state space tree when backtracking is used with n = 4, W = 13, and
w1 = 3, w2 = 4, w3 = 5, w4 = 6

Design and Analysis of algorithms-P.Vasantha Kumari 5

5. Graph coloring

The graph (or vertex) coloring problem, which involves assigning colors to vertices in a graph
such that adjacenct vertices have distinct colors, arises in a number of scientific and engineering
applications such as scheduling , register allocation , optimization and parallel numerical
computation.

Mathmatically, a proper vertex coloring of an undirected graph G=(V,E) is a map c: V -> S such
that c(u) != c(v) whenever there exists an edge (u,v) in G. The elements of set S are called the
available colors. The problem is often to determine the minimum cardinality (the number of
colors) of S for a given graph G or to ask whether it is able to color graph G with a certain
number of colors.

However, coloring a general graph with the minimum number of colors (the cardinality of set S)
is known to be an NP-complete problem.
n 3-color example

Design and Analysis of algorithms-P.Vasantha Kumari 6

6. Knapsack problems

The knapsack is nothing but a sack where in which we need to place the given items according
to the capacity of the knapsack. In case of backtracking we consider the profits but in dynamic
programming we consider weights.

The state space tree for knapsack

Design and Analysis of algorithms-P.Vasantha Kumari 7
 •
Each node v will include 3 values:
– profit (v) = sum of profits of all items included in the knapsack (on a path from
root to v)
– weight (v)= the sum of the weights of all items included in the knapsack (on a
path from root to v)
– upperBound(v). upperBound(v) is greater or equal to the maximum benefit that
can be found by expanding the whole subtree of the state space tree with root v.
• The nodes are numbered in the order of expansion
• Node v is promising if weight(v) < C, and upperBound(v)>maxprofit
• Otherwise it is not promising
• Note that when weight(v) = C, or maxprofit = upperbound(v) the node is non promising
Example

Suppose n = 4, W = 16, and we have the following:

i pi wi pi / wi
1 $40 2 $20
2 $30 5 $6
3 $50 10 $5
4 $10 5 $2
• Note the the items are in the correct order

The calculation for node 1

maxprofit = $0 (n = 4, C = 16 )
Node 1
a) profit = $ 0
weight = 0
b) bound = profit + p1 + p2 + (C - 7 ) * p3 / w3
= $0 + $40 + $30 + (16 -7) X $50/10 =$115
c) 1 is promising because its weight =0 < C = 16
and its bound $115 > 0 the value of maxprofit.

The calculation for node 2

Item 1 with profit $40 and weight 2 is included
maxprofit = $40
a) profit = $40
weight = 2

b) bound = profit + p2 + (C - 7) X p3 / w3
= $40 + $30 + (16 -7) X $50/10 =$115

c) 2 is promising because its weight =2 < C = 16

and its bound $115 > $40 the value of maxprofit.

Design and Analysis of algorithms-P.Vasantha Kumari 8

Summary
Ø Backtracking is a process where steps are taken towards the final solution and the details
are recorded.

Ø A node in the state space tree is promising if it corresponds to the partials constructed
solution that may lead to the complete solution otherwise the nodes are called non-
promising. Leaves of the tree represent either the non- promising dead end or complete
solution found by the algorithm.

Ø The tree organization of the solution space is state space tree. Each node in the state
space tree defines problem state. Solution state is those problem states s for which the
path from the root to s defines a tuple in the solution space

Ø N queen problem is defined as to place n queens in a n ×n matrix. So that no two queens

are in the same row, column, or diagonal.
Ø The graph (or vertex) coloring problem, which involves assigning colors to vertices in a
graph such that adjacent vertices have distinct colors, arises in a number of scientific and
engineering applications such as scheduling, register allocation , optimization and
parallel numerical computation.
Ø The knapsack is nothing but a sack where in which we need to place the given items
according to the capacity of the knapsack.

Design and Analysis of algorithms-P.Vasantha Kumari 9

Key Terms
>> backtracking >> n-queen problem >> knapsack problem
>> subset sum problem >> graph coloring

Key Term Quiz

1. --------------------- is a process where steps are taken towards the final solution and the details
are recorded.
2. A node in the state space tree is ------------------- if it corresponds to the partials constructed
solution that may lead to the complete solution otherwise the nodes are called -----------------
3. The tree organization of the solution space is called ----------------
4. --------------- is defined as to place n queens in a n ×n matrix. So that no two queens are in the
same row, column, or diagonal.
5. --------------- which involves assigning colors to vertices in a graph such that adjacent vertices
have distinct colors, arises in a number of scientific and engineering applications such as
scheduling, register allocation , optimization and parallel numerical computation.
6. Knapsack problem that uses the concept of -----------------
7. N-queen problem is solved by ------------------

Objective Type Questions

1. Define backtracking
(a) Is the process of finding the optimal solution by bounding the unwanted solution
(b)Backtracking is the process of finding the optimal solution by tracing back the previous
solutions t
(c)Is the process of finding the optimal solution by performing the minimum route tracing.
(d)None of the above

2. The application problems that are solved by backtracking are

(a)N queen problem (b) Subset sum problem
(c)Hamiltonian circuit problem (d) All the above

3. Promising nodes
(a)Are the nodes that do not lead to the solution
(b)Are the nodes that contains minimum cost
(c)The nodes that forms a minimum spanning tree
(d)Are the nodes that lead to the solution in the state space tree

4. Non promising nodes

(a)Nodes that lead to the solution
(b)Does not lead to the solution in the state space tree
(c)Both are true
(d)None is true

5. N queen problem
(a) is a problem with n queens and n-1x n-1 squares.
(b)Is a problem with n-1 queens and n x n squares

Design and Analysis of algorithms-P.Vasantha Kumari 10

(c)Is a problem with n queens and n x n squares
(d)None of the above

6. Graph coloring problem

(a)The problem that uses minimum colors for adjacent vertices of the graph
(b)No two vertices should have the same color
(c)Both are true
(d)None of the above

7. N-queen problem is solved by

(a) backtracking (b) greedy (c) branch and bound (d) dynamic programming

8. Subset sum problem is solved by

(a) backtracking (b) greedy (c) branch and bound (d) dynamic programming

9. Grah coloring problem is solved by

(a) backtracking (b) greedy (c) branch and bound (d) dynamic programming

10. What is the minimum number of colors is needed to color the following graph?

(a) 2 (b) 3 (c) 4 (d) 5

11. What is the minimum number of colors is needed to color the following graph?

(a) 2 (b) 3 (c) 4 (d) 5

Review Questions

Two Mark Questions

1. What is back tracking?

Design and Analysis of algorithms-P.Vasantha Kumari 11

 2. What is state space tree?
3. Define promising and non promising node.
4. What do you mean by n-queen problem?
5. Write short notes on subset sum problem.
6. Define knapsack problem.
7. What do you mean by graph coloring?
8. What are the applications of graph coloring?
9. Write the differences between backtracking and branch-and-bound techniques.

Big Questions

1. Explain N-queens problem with an algorithm

2. Give a backtracking solution to the 0/1 Knapsack problem.
3. Explain subset sum problem and discuss possible solution strategies using backtracking.
4. What is backtracking? Explain in detail.
5. Write short notes on graph coloring.

Lesson Labs
1. Apply backtracking to solve the following instances of subset sum problem S={1,3,4,5}
and d=11.

-------------------- END OF FOURTH UNIT -----------------------

Design and Analysis of algorithms-P.Vasantha Kumari 12