11/14/2014

Divide and Conquer

11/14/2014

Three Steps of The Divide and

Conquer Approach
The most well known algorithm design strategy:
1. Divide the problem into two or more smaller
subproblems.
2.

3.

Conquer the subproblems by solving them

recursively.
Combine the solutions to the subproblems
into the solutions for the original problem.

11/14/2014

A Typical Divide and Conquer Case

a problem of size n

subproblem 1
of size n/2

subproblem 2
of size n/2

a solution to
subproblem 1

a solution to
subproblem 2

11/14/2014

a solution to
the original problem

11/14/2014

An Example: Calculating a0 + a1 + + an-1

ALGORITHM RecursiveSum(A[0..n-1])
//Input: An array A[0..n-1] of orderable elements
//Output: the summation of the array elements
if n > 1
return ( RecursiveSum(A[0.. n/2 1]) + RecursiveSum(A[n/2 .. n 1]) )

k
Efficiency: (for n = 2 )

A(n) = 2A(n/2) + 1, n >1

A(1) = 0;

11/14/2014

General Divide and Conquer recurrence

The Master Theorem
the time spent on solving a subproblem of size n/b.

T(n) = aT(n/b) + f (n), where f (n) (nd)

1. a < bd
T(n) (nd)
d
2. a = b
T(n) (nd log n )
d
3. a > b
T(n) (
)
the time spent on dividing the problem
into smaller ones and combining their solutions.
11/14/2014

Divide and Conquer Examples

Sorting: mergesort and quicksort

Binary search
Tree traversals
Matrix multiplication-Strassens algorithm

11/14/2014

11/14/2014

Mergesort
Algorithm:

The base case: n = 1, the problem is naturally solved.

The general case:

Divide: Divide array A[0..n-1] in two and make copies of each half in
arrays B[0.. n/2 - 1] and C[0.. n/2 - 1]
Conquer: Sort arrays B and C recursively using merge sort

Combine: Merge sorted arrays B and C into array A as follows:

Repeat the following until no elements remain in one of the arrays:

compare the first elements in the remaining unprocessed portions of
the arrays
copy the smaller of the two into A, while incrementing the index
indicating the unprocessed portion of that array
Once one of the arrays is exhausted, copy the remaining unprocessed
elements from the other array into A.

11/14/2014

The Mergesort Algorithm

ALGORITHM Mergesort(A[0..n-1])
//Sorts array A[0..n-1] by recursive mergesort
//Input: An array A[0..n-1] of orderable elements
//Output: Array A[0..n-1] sorted in nondecreasing order
if n > 1
//divide
copy A[0.. n/2 1] to B[0.. n/2 1]
copy A[ n/2 .. n 1] to C[0.. n/2 1]
//conquer
Mergesort(B[0.. n/2 1)
Mergesort(C[0.. n/2 1)
//combine
Merge(B, C, A)
11/14/2014

The Merge Algorithm

ALGORITHM Merge(B[0..p-1], C[0..q-1], A[0..p+q-1])
//Merges two sorted arrays into one sorted array
//Input: Arrays B[0..p-1] and C[0..q-1] both sorted
//Output: Sorted Array A[0..p+q-1] of the elements of B and C
i 0; j 0; k 0
while i < p and j < q do //while both B and C are not exhausted
if B[i] <= C[j]
//put the smaller element into A
A[k] B[i]; i i + 1
else A[k] C[j]; j j + 1
kk+1
if i = p
//if list B is exhausted first
copy C[j..q-1] to A[k..p+q-1]
else
//list C is exhausted first
copy B[i..p-1] to A[k..p+q-1]
11/14/2014

11/14/2014

Mergesort Examples

83297154

Efficiency?
Worst case: the smaller element comes from alternating arrays.

11/14/2014

10

The Divide, Conquer and

Combine Steps in Quicksort

Divide: Partition array A[l..r] into 2 subarrays, A[l..s-1]

and A[s+1..r] such that each element of the first array
is A[s] and each element of the second array is A[s].
(computing the index of s is part of partition.)

Implication: A[s] will be in its final position in the sorted array.

Conquer: Sort the two subarrays A[l..s-1] and A[s+1..r] by

recursive calls to quicksort
Combine: No work is needed, because A[s] is already in its
correct place after the partition is done, and the two
subarrays have been sorted.

11/14/2014

11

Quicksort

Select a pivot w.r.t. whose value we are going to divide the

sublist. (e.g., p = A[l])
Rearrange the list so that it starts with the pivot followed by a
sublist (a sublist whose elements are all smaller than or equal to
the pivot) and a sublist (a sublist whose elements are all
greater than or equal to the pivot )
Exchange the pivot with the last element in the first sublist(i.e.,
sublist) the pivot is now in its final position
Sort the two sublists recursively using quicksort.

11/14/2014

A[i]p

A[i] p

12

11/14/2014

The Quicksort Algorithm

ALGORITHM Quicksort(A[l..r])
//Sorts a subarray by quicksort
//Input: A subarray A[l..r] of A[0..n-1],defined by
its left and right indices l and r
//Output: The subarray A[l..r] sorted in
nondecreasing order
if l < r
s Partition (A[l..r]) // s is a split position
Quicksort(A[l..s-1])
Quicksort(A[s+1..r]
11/14/2014

13

The Partition Algorithm

ALGORITHM Partition (A[l ..r])
//Partitions a subarray by using its first element as a pivot
//Input: A subarray A[l..r] of A[0..n-1], defined by its left and right indices
l and r (l < r)
//Output: A partition of A[l..r], with the split position returned as this
functions value
P A[l]
i l; j r + 1;
Repeat
repeat i i + 1 until A[i]>=p //left-right scan
repeat j j 1 until A[j] <= p//right-left scan
if (i < j)
//need to continue with the scan
swap(A[i], A[j])
until i >= j
swap(A[l], A[j])
return j

11/14/2014

14

Quicksort Example
5 3 1 9 8 2 4 7
Exercise:
15 22 13 27 12 10 20 25

11/14/2014

15

11/14/2014

Efficiency of Quicksort
Based on whether the partitioning is balanced.
Best case: split in the middle ( n log n)

for n>1 //2 subproblems of size n/2 each

C(1)=0

Worst case: sorted array! ( n2)

C(n) = 2C(n/2) + n

C(n) = C(n-1) + n+1 //2 subproblems of size 0 and n-1 respectively

Average case: random arrays ( n log n)

11/14/2014

16

Binary Search an Iterative Algorithm

ALGORITHM BinarySearch(A[0..n-1], K)
//Implements nonrecursive binary search
//Input: An array A[0..n-1] sorted in ascending order and
//
a search key K
//Output: An index of the arrays element that is equal to K
//
or 1 if there is no such element

l 0, r n 1
while l r do
m (l + r) / 2
if K = A[m] return m
else
if K < A[m] r m 1
else
l m+1
return -1

//l and r crosses over cant find K.

//the key is found
//the key is on the left half of the array
// the key is on the right half of the array

11/14/2014

17

Binary Search a Recursive Algorithm

ALGORITHM BinarySearchRecur(A[0..n-1], l, r, K)
if l > r
//base case 1: l and r cross over cant find K
return 1
else
m (l + r) / 2
if K = A[m]
//base case 2: K is found
return m
else
//general case: divide the problem.
if K < A[m]
//the key is on the left half of the array
return BinarySearchRecur(A[0..n-1], l, m-1, K)
else
//the key is on the left half of the array
return BinarySearchRecur(A[0..n-1], m+1, r, K)
11/14/2014

18

11/14/2014

Binary Search -- Efficiency

What is the recurrence relation?

C(n) = C(n / 2) + 1

Efficiency

C(n) ( log n)

11/14/2014

19

Binary Tree Traversals

Definitions

Example:

A binary tree T is defined as a finite set of nodes

that is either empty or consists of a root and two
disjoint binary trees TL and TR called, respectively,
the left and right subtree of the root.
The height of a tree is defined as the length of the
longest path from the root to a leaf.

Problem: find the height of a binary tree.

11/14/2014

20

Find the Height of a Binary Tree

ALGORITHM Height(T)
//Computes recursively the height of a binary tree
//Input: A binary tree T
//Output: The height of T
if T =
return 1
else
return max{Height(TL), Height(TR)} + 1

Efficiency?

11/14/2014

C(n) = n + x =2n + 1, x=n+1

21

11/14/2014

Binary Tree Traversals preorder, inorder,

and postorder traversal

Binary tree traversal: visit all nodes of a binary tree recursively.

Write a recursive preorder binary tree traversal algorithm.

Algorithm Preorder(T)
//Implement the preorder traversal of a binary tree
//Input: Binary tree T (with labeled vertices)
//Output: Node labels listed in preorder
if T
write label of Ts root
Preorder(TL)

Preorder(TR)

How
11/14/2014

many recursive calls ?

22

Strassens Matrix Multiplication

Brute-force algorithm

c00 c01

a00 a01

b00 b01

c10 c11

a10 a11

b10 b11

a00 * b00 + a01 * b10

a00 * b01 + a01 * b11

a10 * b00 + a11 * b10

a10 * b01 + a11 * b11

8 multiplications

Efficiency class: (n3)

4 additions
11/14/2014

23

Strassens Matrix Multiplication

Strassens Algorithm (two 2x2 matrices)

c00 c01

c10 c11

a00 a01
a10 a11

m1
m2
m3
m4
m5
m6

m11/14/2014
7 = (a01 - a11) * (b10 + b11)

=
=
=
=
=
=

b00 b01
b10 b11

m1 + m4 - m5 + m7
m2 + m4

(a00 + a11) * (b00 + b11)

(a10 + a11) * b00
a00 * (b01 - b11)
a11 * (b10 - b00)
(a00 + a01) * b11
(a10 - a00) * (b00 + b01)

m3 + m5
m1 + m3 - m2 + m6

7 multiplications
18 additions

24

11/14/2014

Strassens Matrix Multiplication

Two nxn matrices, where n is a power of two (What

about the situations where n is not a power of two?)
C00 C01

A00 A01
=

C10 C11

B00 B01
*

A10 A11

B10 B11

M1 + M 4 - M5 + M 7

M3 + M 5

=
M2 + M 4

M1 + M 3 - M2 + M 6

11/14/2014

25

Submatrices:

M1 = (A00 + A11) * (B00 + B11)

M2 = (A10 + A11) * B00

M3 = A00 * (B01 - B11)

M4 = A11 * (B10 - B00)

M5 = (A00 + A01) * B11

M6 = (A10 - A00) * (B00 + B01)

M7 = (A01 - A11) * (B10 + B11)

11/14/2014

26

Efficiency of Strassens Algorithm

If n is not a power of 2, matrices can be padded with rows

and columns with zeros
Multiplication operation

Recurrence Relation, M(n)=7M(n/2), M(1)=1

Efficiency, M(n)=n2.807
Addition operation

Recurrence Relation, A(n)=7A(n/2)+18(n/2) 2, A(1)=0

Efficiency, A(n)=n2.807

11/14/2014

27