Growth of Functions

M Satpathy/ IIT Bhubaneswar

Week - 2

Lecture 2

Slide 1 of 66.

Asymptotic Analysis
How running time off an algorithm increases when input

size becomes very very large (input size increases without

bounds)
An algorithm which is asymptotically more efficient will be

the better choice, but

for small inputs other algorithms may outperform.

Lecture 2

Slide 2 of 66.

 Functions whose domains are

T(n), n is the input size, a natural number

Lecture 2

Slide 3 of 66.

Lecture 2

Slide 4 of 66.

 Function f(n) is equal to g(n) within a constant factor

Lecture 2

Slide 5 of 66.

Example
We will show that

Lecture 2

Slide 6 of 66.

 Not true for when n >>

or a constant function

Lecture 2

Slide 7 of 66.

Lecture 2

Slide 8 of 66.

is a subset of

Lecture 2

Slide 9 of 66.

Lecture 2

Slide 10 of 66.

 This means some constant multiple of g(n) is an

asymptotic upper bound on f(n).

Lecture 2

Slide 11 of 66.

O notation
Running time of an algorithm can be found by inspecting

the algorithm structure.

A doubly nested loop usually makes the algorithm of

complexity

Example: Insertion sort algorithm; Bubble sort

means, for any input size n, there exists f(n) s.t.

running time of the algorithm is bounded above by f(n).

i.e. worst case running time is

Lecture 2

Slide 12 of 66.

Lecture 2

Slide 13 of 66.

Bubble sort

Lecture 2

Slide 14 of 66.

Horners Rule

Lecture 2

Slide 15 of 66.

asymptotic lower bound on a function

Lecture 2

Slide 16 of 66.

Theorem

Lecture 2

Slide 17 of 66.

 When running time of an algorithm is

Then no matter what is the input size n is chosen, the

running time of the input is constant times g(n)

for n >>

Lecture 2

Slide 18 of 66.

 Best case: linear

Worst case: quadratic

Lecture 2

Slide 19 of 66.

Monotonocity

Lecture 2

Slide 20 of 66.

Divide and Conquer Algorithms

Lecture 2

Slide 21 of 66.

 Divide the problem into smaller sub-problems

Conquer the sub-problems by solving them (recursively)

Combine the solutions to obtain the final result

e.g. Merge Sort

Lecture 2

Slide 22 of 66.

Quicksort
Given A[pr], pick any element A[q] as the pivot.
Partition A[pr] into sub-arrays

A[p (q 1)] and A[(q + 1) r] s.t.

all elements in A[p (q 1)] A[q]
all elements in A[(q + 1) r] A[q]
Sort the sub-arrays by calling Quicksort on sub-arrays

A[p (q 1)]
A[(q + 1) r]

Since IN-PLACE update, in the end, we get the sorted array.

Lecture 2

Slide 23 of 66.

Algorithm
Quicksort(A, p, r)
if p < r

q = partition(A,p,r)
Quicksort(A,p,q-1)
Quicksort(A,q+1, r)

Initial call: Quicksort(A,1,n) // n: length of the array

Lecture 2

Slide 24 of 66.

Lecture 2

Slide 25 of 66.

Lecture 2

Slide 26 of 66.

Look at the in-place updates.

Lecture 2

Slide 27 of 66.

Lecture 2

Slide 28 of 66.

Lecture 2

Slide 29 of 66.

Lecture 2

Slide 30 of 66.

Loop invariants
Can be proved by induction
They help in understanding the algorithm

They help in showing that the algorithm is correct.

Lecture 2

Slide 31 of 66.

Complexity

Lecture 2

Loop runs (r-1-p) times

If n = r 1 p , T(n) =

Slide 32 of 66.

QS: worst case time

In worst case, one side of partition is (n-1) and the other

side is empty.
T(n) for an empty array is T(0)

Lecture 2

Slide 33 of 66.

Best case partitioning

Lecture 2

Slide 34 of 66.

 Suppose, the array always splits into in the ration 1/10 vs

9/10

Lecture 2

Slide 35 of 66.

Lecture 2

Slide 36 of 66.

 Same remains true if the array always splits in ration 1:99.

Lecture 2

Slide 37 of 66.

Quicksort
Worst case hardly occurs in practice
Even if the array splits to n/100 & 99n/100, complexity

remains at

Lecture 2

Slide 38 of 66.

Quicksort
Worst case hardly occurs in practice

Lecture 2

Slide 39 of 66.

Quicksort
Widely used in practice
Performs better than Merge-sort
In average case both are
But Quicksort has smaller const of proportionality
Also In-PLACE update: no extra memory

Lecture 2

Slide 40 of 66.

Quicksort

Donald Knuth
Edsger Dijkstra
Niklaus Wirth
Dana Scott

Raj Reddy

Tony Hoare [Turing award winner]

Lecture 2

Slide 41 of 66.