Divide & Conquer

Design and Analysis of

Algorithms
(Lecture 14)
Divide And Conquer
Fall-29
National University Of Computer & Emerging Sciences
 Divide & Conquer

Introduction
• General techniques that often yield effective algorithms for solving large classes
of problems.

• Some of the more important techniques are (i) Greedy (ii) Dynamic Programming
and (iii) Divide & Conquer

• Before using any technique, better ask about the quality of the solution obtained.

• Remember that for NP-Complete problems, these or any other known techniques
will NOT yield efficient solutions.

National University Of Computer & Emerging Sciences

 Divide & Conquer

Divide and Conquer

• Breaking up large problems into smaller units that are easier
to handle

• Example: Cleaning the house, implementing a word

processor

• Top-down methodology and object-oriented methodology

are based on the principle of divide and conquer

National University Of Computer & Emerging Sciences

 Divide & Conquer

Divide and Conquer

Hard Problem

Easy Problem Hard Problem Easy Problem

Easy Problem Easy Problem

National University Of Computer & Emerging Sciences

 Divide & Conquer

Divide & Conquer

• Divide & conquer paradigm involves 3
steps:
– Divide the problem into sub-problems
– Conquer the sub-problems P
– Combine the solutions to the sub-problems.
P1 P2
• This nature of divide & conquer S1 S2
algorithms automatically lends to
recursion. S
• Classic examples:
– Merge sort : T(n) = 2 T(n/2) + O(n)
– Binary search: T(n) = T(n/2) + 1

• Proof: usually by induction method.

National University Of Computer & Emerging Sciences
 Divide & Conquer

Divide & Conquer (Constructing Tennis Tournament)

• Round robin tournament of n = 2k players.

• Each player must play every other player.

• Each player must play one match per day for n – 1 days.

• The tournament schedule is thus an n row by n-1 column

table, whose entry is row i and column j is that player i
must play with which player on jth day.

• Work by finding schedule for one half of players, keep on

doing it till we get down to two players.

National University Of Computer & Emerging Sciences

 Divide & Conquer

Divide & Conquer (Constructing Tennis Tournament)

• Divide the matrix into four parts

• The upper left corner comes from the smallest schedule,

recursively.

• The bottom left corner is obtained by adding n/2 to each player

code of the top left corner

• The top right corner is obtained by pitting the high order players
against low order players, by cyclically permuting the orders.

• The bottom right corner is similarly obtained by pitting the low order
players against high order players, by cyclically permuting the
orders.

National University Of Computer & Emerging Sciences

 Divide & Conquer

Divide & Conquer (Constructing Tennis Tournament)

1 2 4 5 6
1 23 3 5 76
2 7
4 8
3
1 4 6 7
4 8
Player 37
5 6 1 54
6 4 1 7 38
8 52
7 28
5 2 61
8 3 2 8 45
63
Will7 play player
National University1Of Computer
7 & Emerging Sciences
 Divide & Conquer

Divide & Conquer (Constructing Tennis Tournament)

1 1 2 1 2 4 5 6
1 2 1 23 3 1 23 3 5 76
7
2 1 2 1 4 2
4 8
3 4 1 3
42 1 4 6 7
4 3 3 4 8
5 6 37 1 54
2
6 4 1 7 38
1 8 52
7 28
5 2 61
8 3 2 8 45
7 63
National University Of Computer & Emerging Sciences
 Divide & Conquer

Example: Min-Max Problems

• Problem: Find the largest (heaviest) and

smallest (lightest) elements in a set of n
elements (e.g., gold nuggets, counterfeit coins)

• Simple Algorithm: Make n-1 comparisons to

find largest, n-2 more comparisons to find
smallest for a total of 2n-3 comparisons.

National University Of Computer & Emerging Sciences

 Divide & Conquer

Min-Max Cont’d

• Divide-and-Conquer:
– For n ≤ 2, make 1 comparison
– For large n, divide set into two smaller sets and determine
largest/smallest element for each set
– Compare largest/smallest from two subsets to determine
smallest/largest of combined sets
– Do recursively

• Let n =8
• How many comparisons?

National University Of Computer & Emerging Sciences

 Divide & Conquer

Min-Max Cont’d

• Recursive solution: recurrence equation

1, n=2
• c(n) =
2* c(n/2) + 2, n =2k
• Solve using substitution
c(n) = (3n/2) - 2, n=2k

National University Of Computer & Emerging Sciences

 Divide & Conquer

Misc
• Divide-and-Conquer methodology naturally leads to
recursive algorithms
• In general, implement algorithm as recursive
program
– Few instances where non-recursive is better

• Rule of thumb: Balance sub-problems, i.e., try and

make sub-problems of equal size
– See merge sort

National University Of Computer & Emerging Sciences

 Divide & Conquer

MULTIPLICATION OF TWO n-BIT

NUMBERS
• For this problem, the basic operations should be
bit operations.
• Let X and Y be two n-bit numbers, the traditional
method requires O(n2) bit operations, say

National University Of Computer & Emerging Sciences

 Divide & Conquer

MULTIPLICATION OF TWO n-BIT

NUMBERS (cont.)
• Assume n=2m, we treat X and Y as two n-bit strings and partition
them into two halves as X = A * B, Y= C * D, i.e., concatenation of
A, B and C, D. As binary numbers, we can express

XY = (A2n/2 + B) (C2n/2 + D)
= AC2n + (AD + BC) 2n/2 + BD ............ (*)
• The above computation of XY reduces to four multiplications of (n/
2)-bit numbers plus some additions and shifts (multiplications by p
owers of 2).

National University Of Computer & Emerging Sciences

 Divide & Conquer

Two n-bits Number Analysis

• Let T(n) be the number of bit-operations
required to multiply two n-bit numbers.

National University Of Computer & Emerging Sciences

 Divide & Conquer

Two n-bits Number Analysis (cont.)

National University Of Computer & Emerging Sciences

 Divide & Conquer

Two n-bits Number Analysis (cont.)

• Thus T(n) depends very much the number of

subproblems and also the size of the subproblems.
• If the number of subproblems were 1, 3, 4, 8, then the
algorithms would be of order n, nlog3, n2, n3 respectively.

National University Of Computer & Emerging Sciences

 Divide & Conquer

A better Algorithm
• So, the multiplication of two n-bit numbers remains O
(n2) if the number of half-size subproblems remains 4
• In order to reduce the time complexity, we attempt to
reduce the number of subproblems (i.e., multiplication
s ) by a trick which uses more additions and shifts, spe
cifically, by reducing one multiplication with many addit
ions and shifts.
• This trick pays off asymptotically, i.e., when n is
large.

National University Of Computer & Emerging Sciences

 Divide & Conquer

A better Algorithm (cont.)

U = (A + B)(C + D) XY= AC2n + (AD + BC) 2n/2 + BD ............ (*)
V = AC
W = BD
Z = U - V - W = AD + BC

So, XY can be written as V2n+ Z2 n/2 + W (from Eq (*)).

• This approach requires only three multiplications of

two (n/2)-bit numbers, plus some additions and shift
s, to multiply two n-bit numbers.

National University Of Computer & Emerging Sciences

 Divide & Conquer

Analysis

• One can use the multiplication routine recursively to

evaluate the products u, v, and w. The additions and shif
ts require O(n) time. Thus the time complexity of multiplyi
ng two n-bit numbers is bounded from above by

• Where k is a constant reflecting the additions and shifts.

From the above discussion, the solution to the recurrenc
e is then O(n log3 ) = O(n 1.59 ).

National University Of Computer & Emerging Sciences

 Divide & Conquer

Example

This divide and conquer technique for multiplication of two

n-bit number can also applied to the multiplication problem
of two n-digit integers and n-degree polynomial.

National University Of Computer & Emerging Sciences

 Divide & Conquer

Is it really better than O(n )? 2

If this technique is so good, why don’t we teach it in school?

1. The technique is quite complex to understand. Most likely the

students will never learn multiplication.

2. The constant ignored is very large i.e. 500. Useful for large
multiplications.

3. Usually don’t multiply numbers 500 bit numbers by paper

pencil technique in schools.

National University Of Computer & Emerging Sciences