You are on page 1of 76

DESIGN AND ANALYSIS OF ALGORITHMS

Text Book:
Introduction to the Design and Analysis of Algorithms. - Anany Levitin

UNIT I
Introduction Notion of Algorithm Fundamentals of Algorithmic Solving Important Problem Types Fundamentals of the Analysis Framework Asymptotic Notations Basic Efficiency Classes

Introduction
Algorithmics is more than a branch of computer science.
It is the core of computer science, and, in all fairness, can be said to be relevant to most of science, business, and technology.

Notion of Algorithm

An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.

Notion of Algorithm

GCD of 2 Numbers
Euclids algorithm for computing gcd(m, n) Step 1: If n = 0, return the value of m as the answer and stop; otherwise, proceed to Step 2.
Step 2: Divide m by n and assign the value of the remainder to r. Assign the value of n to m and the value of r to n. Go to Step 1.

Step 3:

GCD of 2 Numbers
ALGORITHM Euclid(m, n) //Computes gcd (m, n) by Euclids algorithm //Input: Two nonnegative, not-both-zero integers m and n //Output: Greatest common divisor of m and n while n != 0 do r m mod n mn nr return m

Notion of Algorithm
Consecutive integer checking algorithm for computing gcd(m, n) Step 1:
Step 2:

Assign the value of min{m, n} to t.


Divide m by t. If the remainder of this division is 0, go to Step 3; otherwise, go to Step 4.

Notion of Algorithm
Step 3: Divide n by t. If the remainder of this division is 0, return the value of t as the answer and stop; otherwise, proceed to Step 4.
Decrease the value of t by 1. Go to Step 2.

Step 4:

Notion of Algorithm
Middle-school procedure for computing gcd(m, n)
Step 1: Step 2: Find the prime factors of m. Find the prime factors of n.

Notion of Algorithm
Middle-school procedure for computing gcd(m, n)
Step 3: Identify all the common factors in the two prime expansions found in Step 1 and Step 2.

Step 4:

Compute the product of all the common factors and return it as the greatest common divisor of the numbers given.

Notion of Algorithm
For the numbers 60 and 24, we get
60 = 2 . 2 . 3 . 5 24 = 2 . 2 . 2 . 3 gcd(60, 24) = 2 . 2 . 3 = 12.

Notion of Algorithm
Sieve of Eratosthenes method of generating prime numbers:

Notion of Algorithm
ALGORITHM Sieve(n)
//Implements the sieve of Eratosthenes //Input: A positive integer n > 1 //Output: Array L of all prime numbers less than or equal to n for p2 to n do A[p]p

Notion of Algorithm
for p2 to n do if A[p] !=0 jpp while j n do A[j ]0 j j + p end while End for

Notion of Algorithm
//copy the remaining elements of A to array L of the primes i 0 for p2 to n do if A[p] = 0 L[i]A[p] i i + 1 return L

Notion of Algorithm

Find gcd(31415, 14142) Euclids algorithm.

by

applying

Notion of Algorithm
gcd(31415, 14142) gcd(14142, 3131) gcd(3131, 1618) gcd(1618, 1513) gcd(1513, 105) gcd(1513, 105) gcd(105, 43) gcd(43, 19) = gcd(19, 5) = gcd(5, 4) gcd(4, 1) = gcd(1, 0) = 1.

Fundamentals of Algorithmic Problem Solving


Algorithms can be the procedural solutions to problems. These solutions are not answers but specific instructions for getting answers.

Fundamentals of Algorithmic Problem Solving


Understanding the Problem
Ascertaining the Capabilities Computational Device of the

Choosing between Exact and Approximate Problem Solving Algorithm Design Techniques

Fundamentals of Algorithmic Problem Solving


Designing an Algorithm and Data Structures Methods of Specifying an Algorithm Proving an Algorithms Correctness Analyzing an Algorithm Coding an Algorithm

Important Problem Types


Sorting Searching String processing Graph problems Combinatorial problems Geometric problems Numerical problems

Examples
Write an algorithm for the sorting problem that sorts an array by counting, for each of its elements, the number of smaller elements and then uses this information to put the element in its appropriate position in the sorted array. Apply this algorithm to sorting the list 60, 35, 81, 98, 14, 47.

Examples
ALGORITHM ComparisonCountingSort(A[0..n 1]) //Sorts an array by comparison counting //Input: Array A[0..n 1] of orderable values //Output: Array S[0..n 1] of As elements sorted in nondecreasing order

Examples
for i 0 to n 1 do Count[i]0 for i 0 to n 2 do for j i + 1 to n 1 do if A[i]<A[j ] Count[j ]Count[j ]+ 1 else Count[i]Count[i]+ 1

Examples
for i 0 to n 1 do S[Count[i]] = a[i] Return S

Fundamentals of the Analysis of Algorithm Efficiency


The Analysis Framework:

There are two kinds of efficiency:


Time efficiency Space efficiency.

Fundamentals of the Analysis of Algorithm Efficiency


Time efficiency, also called time complexity, indicates how fast an algorithm in question runs. Space efficiency, also called space complexity, refers to the amount of memory units required by the algorithm in addition to the space needed for its input and output.

Measuring an Inputs Size


Lets start with the obvious observation that almost all algorithms run longer on larger inputs.

For example, it takes longer to sort larger arrays, multiply larger matrices, and so on. Therefore, it is logical to investigate an algorithms efficiency as a function of some parameter n indicating the algorithms input size.

Units for Measuring Running Time


We can simply use some standard unit of time measurementa second, or millisecond, and so onto measure the running time of a program implementing the algorithm. There are obvious drawbacks to such an approach.

Units for Measuring Running Time


Dependence on the speed of a particular computer, dependence on the quality of a program implementing the algorithm and of the compiler used in generating the machine code, and the difficulty of clocking the actual running time of the program. Since we are after a measure of an algorithms efficiency, we would like to have a metric that does not depend on these extraneous factors.

Units for Measuring Running Time


One possible approach is to count the number of times each of the algorithms operations is executed. The thing to do is to identify the most important operation of the algorithm, called the basic operation, the operation contributing the most to the total running time, and compute the number of times the basic operation is executed.

Units for Measuring Running Time


As a rule, it is not difficult to identify the basic operation of an algorithm: it is usually the most time-consuming operation in the algorithms innermost loop. For example, most sorting algorithms work by comparing elements (keys) of a list being sorted with each other; for such algorithms, the basic operation is a key comparison.

Units for Measuring Running Time


As another example, algorithms for mathematical problems typically involve some or all of the four arithmetical operations: addition, subtraction, multiplication, and division.

Of the four, the most time-consuming operation is division, followed by multiplication and then addition and subtraction, with the last two usually considered together.

Units for Measuring Running Time


Thus the established framework for the analysis of an algorithms time efficiency suggests measuring it by counting the number of times the algorithms basic operation is executed on inputs of size n.

Units for Measuring Running Time


Let cop be the execution time of an algorithms basic operation on a particular computer, and let C(n) be the number of times this operation needs to be executed for this algorithm. Then we can estimate the running time T (n) of a program implementing this algorithm on that computer by the formula:

Orders of Growth

Worst-Case, Best-Case, and Average-Case Efficiencies


The worst-case efficiency of an algorithm is its efficiency for the worst-case input of size n, which is an input (or inputs) of size n for which the algorithm runs the longest among all possible inputs of that size.

The worst-case analysis provides very important information about an algorithms efficiency by bounding its running time from above.

Worst-Case, Best-Case, and Average-Case Efficiencies


The best-case efficiency of an algorithm is its efficiency for the best-case input of size n, which is an input (or inputs) of size n for which the algorithm runs the fastest among all possible inputs of that size.

Worst-Case, Best-Case, and Average-Case Efficiencies


However, that neither the worst-case analysis nor its best-case counterpart yields the necessary information about an algorithms behavior on a typical or random input.
This is the information that the average-case efficiency seeks to provide. To analyze the algorithms average case efficiency, we must make some assumptions about possible inputs of size n.

Worst-Case, Best-Case, and Average-Case Efficiencies

Amortized Efficiency
It applies not to a single run of an algorithm but rather to a sequence of operations performed on the same data structure. It turns out that in some situations a single operation can be expensive, but the total time for an entire sequence of n such operations is always significantly better than the worst-case efficiency of that single operation multiplied by n.

Amortized Efficiency
So we can amortize the high cost of such a worst-case occurrence over the entire sequence in a manner similar to the way a business would amortize the cost of an expensive item over the years of the items productive life.

Asymptotic Notations and Basic Efficiency Classes


The efficiency analysis framework concentrates on the order of growth of an algorithms basic operation count as the principal indicator of the algorithms efficiency.

To compare and rank such orders of growth, computer scientists use three notations:
O(big oh), (big omega), and (big theta).

Asymptotic Notations and Basic Efficiency Classes


The idea of these definitions is to establish a relative order among functions. Given two functions, we compare their relative rates of growth. Example 1000n and n2

O - Notation
A function t (n) is said to be in O(g(n)), denoted t (n) O(g(n)), if t (n) is bounded above by some constant multiple of g(n) for all large n, i.e., if there exist some positive constant c and some nonnegative integer n0 such that t (n) cg(n) for all n n0.

O - Notation
Example:
100n + 5 O(n2)

O - Notation

- Notation
A function t (n) is said to be in (g(n)), denoted t (n) (g(n)), if t (n) is bounded below by some positive constant multiple of g(n) for all large n, i.e., if there exist some positive constant c and some nonnegative integer n0 such that t (n) cg(n) for all n n0.

- Notation
Example

n3 (n2)

- Notation

- Notation
A function t (n) is said to be in (g(n)), denoted t (n) (g(n)), if t (n) is bounded both above and below by some positive constant multiples of g(n) for all large n, i.e., if there exist some positive constants c1 and c2 and some nonnegative integer n0 such that c2g(n) t (n) c1g(n) for all n n0.

- Notation
Example:

1/2 n (n-1) (n2)

- Notation

THEOREM
If t1(n) O(g1(n)) and t2(n) O(g2(n)), then

t1(n) + t2(n) O(max{g1(n), g2(n)}).

PROOF

if a1 <= b1 and a2 <= b2, then a1 + a2 < = 2 max{b1, b2}

- Notation

RULES

If T1(n) = O(f(n)) and T2(n)=O(g(n)), then

(a)
(b)

T1(n) + T2(n) = max (O(f(n)), O(g(n)))


T1(n) * T2(n) = O(f(n) * g(n))

RULES

If T(x) is a polynomial of degree n, then T(x) = (x n).

Log k n = O(n) for any constant k. This tells us that logarithms grow very slowly.

RULES

If two programs are expected to take similar times, probably the best way to decide which is faster is to code them both up and run them.

Using Limits for Comparing Orders of Growth


Though the formal definitions of O, , and are indispensable for proving their abstract properties, they are rarely used for comparing the orders of growth of two specific functions.

A much more convenient method for doing so is based on computing the limit of the ratio of two functions in question.

Using Limits for Comparing Orders of Growth

Using Limits for Comparing Orders of Growth


Compare the orders of growth of the following: n(n 1) and n2

Log 2 n and n
n! And 2n

ASSIGNMENT
1. What is an algorithm? Explain the notion of an algorithm with a neat diagram. 2. Give and explain the important characteristics of an algorithm. 3. Write and explain the Euclids algorithm for computing GCD of any two integers. Hence find gcd(31415, 14142).

ASSIGNMENT
4. Explain with an example the consecutive integer checking algorithm for computing GCD of any two integers.
5. Implement Sieves algorithm to generate prime numbers between 1 and n. Trace the algorithm for n = 25. 6. Explain with a neat diagram, the sequence of steps to be followed in designing and analyzing an algorithm.

ASSIGNMENT
7. Write an algorithm for finding the distance between the two closest elements in an array of numbers.
8. Explain the important problem types. Give two examples for each. 9. Implement an algorithm that sorts an array by counting, for each of its elements, the number of smaller elements and then uses this information to put the element in its appropriate position in the sorted array. Apply this algorithm to sorting the list 60, 35, 81, 98, 14, 47.

ASSIGNMENT
10. Explain the terms: time and space efficiency.
11. Explain the units for measuring the running time. 12. Compare the orders of growth of various algorithms. 13. Define the following: worst case, best case and average case analysis of an algorithm.

ASSIGNMENT

14. Give the worst case, best case and average case analysis of sequential search algorithm. 15. What is amortized efficiency? Explain with an example.

ASSIGNMENT
16. For each of the following algorithms, indicate (i) a natural size metric for its inputs, (ii) its basic operation, and (iii) whether the basic operation count can be different for inputs of the same size:
a. computing the sum of n numbers b. computing n! c. finding the largest element in a list of n numbers d. Euclids algorithm e. sieve of Eratosthenes f. pen-and-pencil algorithm for multiplying two n-digit decimal integers

ASSIGNMENT
17. Explain the different asymptotic notations with an example for each notation.
18. If If t1(n) O(g1(n)) and t2(n) O(g2(n)), then show that t1(n) + t2(n) O(max{g1(n), g2(n)}). 19. Compare the orders of growth for the following: a). n(n 1) and n2 b) Log 2 n and n c) n! And 2n

ASSIGNMENT
20. Indicate whether the first function of each of the following pairs has a smaller, same or larger order of growth than second function.
a. b. c. d. e. f. n (n+1) and 2000n2 100n2 and 0.01n3 Log 2 n and ln n 2n-1 and 2n Log 2 n and log 2 n2 (n-1)! And n!

ASSIGNMENT
21. For each of the following functions, indicate the class (g(n)) the function belongs to. Prove your assertions.

22. Prove that if t1(n) (g1(n)) and t2(n) (g2(n)), then t1(n) + t2(n) (max{g1(n), g2(n)}).

ASSIGNMENT
23. Prove that every polynomial of degree k, p(n) = aknk + ak1nk1+ . . . + a0 with ak > 0, belongs to (nk). 24.List the following functions according to their order of growth from the lowest to the highest:

ASSIGNMENT
25. Find the Big oh notation for the following:
a. b. c. Log n + n n + n log n 6n + 2n4 + 4n5

26. With the help of a flowchart, explain the various stages of algorithm design and analysis process. 27. Explain the different problem types and give one algorithm for each type.

ASSIGNMENT
28. Use the informal definitions of O,, and to determine whether the following assertions are true or false.