Manipal Univ -MS (CS),CSE 607: DAA 1.

Algorithms Definitions Syllabus

2. Correctness, time and Space Complexity 3. Average and Worst case analysis 4. Optimality , P and NP Complete Problems 5. Algorithmic languages and data structures 6. Numeral , Non-numeral , combinational and graph algorithms 7. Polynomial , matrix and vector manipulations 8. String matching and data compression algorithms 9. Algorithms for linear , Non-linear , integer and Dynamic Programming Problems. 10. Searching, Sorting, merging sorted lists, String matching , Trees, Decision Trees ,game Trees, Branch and Bound Algorithms. 11. Set representation and components of a Graph 12. Shortest Path Problems, traveling Salesman Problem. 13. Parallel Algorithms and Distributed Processing.

Previous Question paper -2007

1. Define "Big Oh" and "Omega" notations with example. Find "Big Oh" notation for the following equations: i) log n + Sqrt (n) ii) 6n+2n4+4n5 iii) n + log n iv) log n + n. 2. Calculate optimal job sequencing using greedy strategy, given n = 5. (pI, p2, p3, p4, p5) = (15, 20, 10,5, 1) (dl, d2, d3, d4, d5) = (2, 2,1,3,3) 3. Compare and contrast the Prim's and Kruskal's Algorithms with a suitable example for each. 4. Derive the time complexity of Quick Sort and Merge Sort using Master's Theorem. Compare quick sort and merge sort with respect to their time complexities 5. Explain the concepts of Breadth First Search (BFS) and Depth First Search (DFS) of a graph. Also suggest the pseudo codes for same.

6. Explain the concept of polynomial reducibility and NP Completeness. Explain any two Problems which are NPC. 7. Explain Presorting with the help of an algorithm. Show how presort can help in calculating mode of give datasets. 8. What is an AVL tree? Explain the four rotations used in AVL trees while balancing the tree with an example. 9. Explain Binary Search Algorithm and derive its time complexities in best, average and worst cases. 10. Write the general plan for analyzing efficiency of recursive algorithms. Give a recursive solution to the Tower of Hanoi Puzzle.

Fundamentals of Algorithmic Problem Solving

1. Understanding the Problem 2. Ascertaining the Capabilities of a computational device 3. Choosing between Exact and Approximate Problem Solving 4. Deciding on Appropriate Data Structures 5. Algorithm Design Techniques 6. Methods of Specifying an Algorithm 7. Proving an Algorithms Correctness 8. Analyzing an Algorithm 9. Coding an Algorithm

An Input to an algorithm specifies an instance of the problem the algorithm solves

Ascertaining the Capabilities of a computational device :

If the machine executes the instructions one after the other, one operation at a time then the algorithm is said to be Sequential algorithms. If the machine executes the instructions in parallel then the algorithms are said to Parallel Algorithms.

Choosing between Exact and Approximate Problem Solving :

The next principle decision is to choose between solving the problem exactly or solving it approximately. Why would one opt for an approximation algorithms? 1. There are important problems that simply can not be solved exactly , such as Extracting square roots , Solving non linear equations and Evaluating definite integrals.

1. Available algorithms for solving a problem exactly can be unacceptably slow because of the Problems intrinsic Complexity. The most well known of them is the Traveling Salesman Problem of finding the shortest tour through n cities.

Deciding on Appropriate Data Structures such as Stacks,

Queues , Sets , Graphs etc.

An Algorithm Design Technique is a general approach to

solving problems algorithmically that is applicable to a variety of problems from different area of computing.

Methods of Specifying an Algorithm : Algorithm can be

specified in Pseudo code or Flowchart. A Pseudo code is a mixture of natural language and programming language like constructs. A Flowchart is a method of expressing an algorithm by a collection of connected geometric shapes containing descriptions of the algorithms steps.

Proving an Algorithms Correctness :

You have to prove that the algorithm yields a required result for every legitimate input in a finite amount of time. A common technique for proving correctness is to use mathematical induction because an algorithms iterations provide a natural sequence of steps needed for such proofs. To show that an algorithm is incorrect, you need just one instance of its input for which the algorithm fails. If the algorithm is found to be incorrect, you need to either redesign it under same decisions regarding the data structures, the design technique and so on. For an approximation algorithm, we usually would like to be able to show that the error produced by algorithm does not exceed a predefined limit.

Analyzing an algorithm :
There are two kinds of algorithm efficiency : Time efficiency and Space efficiency. Time efficiency indicates how fast the algorithm runs. Space efficiency indicates how much extra memory the algorithm needs. Another desirable characteristics of an algorithm is Simplicity. Simpler algorithms are easier to understand and easier to program, consequently, the resulting programs usually contain fewer bugs. Another desirable characteristics of an algorithm is generality. There are two issues : generality of the problem the algorithm solves and the range of inputs it accepts. It is Some times easier to design an algorithm for a problem posed on general terms.

Coding an algorithm :
Most algorithms are destined to be ultimately implemented as Computer programs. Programming an algorithm presents a peril and an opportunity. The Peril lies in the possibility of making the transition from an algorithm to a program either incorrectly or very inefficiently. As a practical matter , the validity of programs is still established by testing. Testing of computer programs is an art rather than a science. Testing and Debugging is essential to implement an algorithm. As a rule, a good algorithm is a result of repeated effort and rework.

Algorithm Specification :
1. Comments begin with // and continue until the end of line. 2. Blocks are indicated with matching braces { } 3. An identifier begins with a letter . The Data types of variables are not explicitly declared. 4. Assignment of values to variables is done using the assignment statement < variable > : = < expression > ; 5. There are two boolean values True and False. The Logical operators and , or , not and relational operators <, <=,=, <> , >, >= are provided. 6. Elements of multidimensional arrays are accessed using [ and ] . 7. The following loop statements are employed.

While loop While < condition > do { < Statement 1 > . < Statement n > } For loop for variable : = value 1 to value 2 step step do { < Statement 1 > . < Statement n > }

Repeat Until loop repeat < Statement 1 > < Statement n > until < condition > 1. A Conditional sttement has the following forms : if < condition > then < statement > if < condition > then < statement 1 > else < statement 2 > Case Statement case { : < condition > : < statement 1> . : < condition n > : < statement n > : else : < statement n+1 > }

1. Input and Output are done using instructions read and write 2. There is only one type of procedure : Algorithm . An Algorithm consists of a heading and a body . The heading takes the form Algorithm Name ( < parameter list > ) where Name is the name of the procedure and (<parameter list >) is a listing of the procedure parameters.

Algorithm to find maximum of n

1. Algorithm Max ( A , n) 2. // A is an array of size n. 3. { 4. Result := A[1]; 5. for i:= 2 to n do 6. if A[i] > Result then Result := A[i]; 7. return Result ; 8. } A and n are procedure parameters, Result and i are local variables.

Space Complexity:
The Space needed by an algorithm is seen to be the sum of the following components: A fixed part that is independent of the Characteristics (ex: number, size) of the inputs and outputs. This part includes the Instruction Space (Space for the code), Space for constants A variable part that consists of the space needed by component variables whose size is dependent on the particular problem instance being solved, the space needed by the referenced variables and the recursion stack space. The Space requirement S (P) of any algorithm P may be written as S (P) = c + Sp (Instance characteristics), where c is constant.

Example 1: 1. Algorithm abc(a,b,c) 2. { 3. return a+ b+ b*c+ (a +b- c) /(a +b)+ 4.0 ; 4. } If we assume that one word is adequate to store the values of each of a, b, c and the result then the space needed by abc algorithm is 4 words Since the space needed by abc is independent of the instance characteristic, Sp = 0.

Example 2: 1. Algorithm Sum(a,n) 2. { 3. s := 0.0 ; 4. for i: = 1 to n do 5. s: = s + a[i]; 6. return s ; 7. } The Space needed by n is one word since it is of type integer. The Space needed by a is the space needed by variables of type array of floating point numbers. This requires at least n words to store n elements in array and 3 words to store the values of n, i and s.The Space required for Sum algorithm is S >= (n+3).

Example 3 : 1. Algorithm RSum(a,n) 2. { 3. if (n<= 0 ) then return 0.0 ; 4. else return RSum (a,n-1) + a[n] ; 5. } Assume that the return address requires one word of memory, each call to RSum requires at least 3 words (one word each for n , return address and a pointer to a[ ]) The depth of the recursion is n+1. The space required is > = 3(n+1)

Time Complexity :
The Time T (P) taken by a program P is the sum of the Compile time and Run time. The Compile time does not depend on the instance characteristics. We may assume that a compiled program will be run several times with out re compilation. The run time is denoted by tp (instance characteristics) tp (n) = ca ADD (n) + cs SUB (n) + cm MUL (n) + cd DIV (n)+ .Where ca , cs, cm ,cd respectively denote the time needed for an addition, subtraction ,multiplication ,division and so on and ADD,SUB ,MUL , DIV denotes the functions of addition, subtraction ,multiplication ,division respectively.

The time complexity is depending on the number of program steps. A program step is defined as a syntactically or semantically meaningful segment of a program that has an execution time that is independent of instance characteristics. Ex : The entire statement return a +b + b*c + (a+ b- c) / (a+ b) + 4.0; could be regarded as a step since its execution time is independent of instance characteristics We can determine the number of steps needed by a program to solve a particular problem instance in one of two ways.

1. Introduce a new variable, count, into the program. This is a global variable with initial value 0. Count is incremented by the step count of that statement in the program. 2. Build a table in which we list total number of steps contributed by each statement. Determine the number of steps per execution (s/e) of that statement and the total number times each statement is executed. The total number of contributions of all statements will give the step count of the program.

Example 1 : 1. Algorithm Sum(a,n) 2. { 3. s: = 0.0 ; 4. count := count +1; 5. for i: = 1 to n do 6. { 7. count := count + 1; // for For 8. s: = s +a[i]; count := count + 1; // for assignment 9. } 10. count := count + 1; // for last time of for 11. count := count + 1; // for return 12. return s; 13. }

 The above algorithm can be simplified for count variable as follows. 1. Algorithm Sum(a,n) 2. { 3. for i:=1 to n do count := count+2; 4. count:= count + 3; 5. } From the algorithm, the value of count will increase by a total of 2n .If count is zero to start with, and then it will be 2n+3 on termination. So each invocation above algorithm executes a total of 2n +3 steps.

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. { 11.count := count +1 ; 12. // for the addition ,function invocation and return 13. return RSum( a,n-1) + a[n]; 14. } 15. }

Example 2: Algorithm RSum(a,n) { count := count +1; // for the if conditional if (n<=0)then { count := count + 1 ; // for the return return 0.0; } else

Let t RSum(n) be the increase in the value of count when algorithm terminates.The recursive formula for step count is given by t RSum(n) = 2 if n=0 = 2 + t RSum(n-1) if n>0 The above recurrence relation can be solved by substitution method. t RSum(n) = 2 + t RSum(n-1) = 2 + 2 + t RSum(n-2) = 2(2) + t RSum(n-2) = 2 + 2 + 2 + t RSum(n-3) = 2(3) + t RSum(n-3) =. = 2(n) + t RSum(0) = 2(n) + 2 for n >= 0. So, the step count for RSum algorithm is 2n + 2

The Second method to determine the step count of an algorithm is to build a table in which we list the total number of steps contributed by each statement. The s/e (Steps per execution) of a statement is the amount by which the count changes as a result of the execution of that statement. The total number times the step executed is known as Frequency. By combing these two quantities, the total contribution of each statement is obtained. By adding contributions of all statements, the step count for the entire algorithm is obtained.

Example 1: Statement s/e frequency Total steps

Algorithm Sum(a,n) { s := 0.0 ; for i: = 1 to n do s: = s + a[i]; return s ; }

0 0 1 1 1 1 0

1 n+1 n 1 -

0 0 1 n+1 n 1 0

Total

2n + 3

Example 2:

Statement 1. Algorithm RSum(a,n) 2. { 3. if (n<=0)then 4. return 0.0; 5. else return 6. RSum( a,n-1) + a[n]; 7. } Total

s/e 0 0 1 1 1 0

frequency Total steps n =0 n >0 n =0 n >0 0 0 0 0 1 1 1 1 1 0 1 0 0 1+x 0 1+x 0 0 2+x

Where x = t RSum(n-1)

Example 3: Statement 1 Algorithm Add(a,b,c,m,n) 2. { 3. for i:= 1 to m do 4. for j: = 1 to n do 1. c[i,j]:= a[i,j] + b[i,j]; 2. } Total s/e 0 0 1 1 1 0 frequency m+1 m(n + 1) mn Total steps 0 0 m+1 mn + m mn 0 2mn+2m +1

For m > n

Asymptotic Notation (O,,)

Big O notation : The function f(n) = O(g(n)) iff there exist positive constants c and n0 such that f(n) <= c * g(n) for all n , n>= n0 . Examples: 1. The function 3n + 2 = O(n) as 3n + 2 <= 4n for all n >= 2 2. The function 3n + 3 = O(n) as 3n + 3 <= 4n for all n >= 3 3. The function 100n + 6 = O(n) as 100n + 6 <= 101n for all n >= 6 4. The function 10n2 + 4n+2 = O(n2) as 10n2 + 4n + 2 <= 11n2 for all n >= 5

5. The function 6*2n + n2 = O(2n) as 6*2n + n2 <= 7*2n for all n>=4 6. The function 10n2 + 4n+2 = O(n4) as 10n2 + 4n + 2 <= 10n4 for all n >= 2 7. The function 3n + 2 O(1) as 3n + 2 is not less than or equal to c for any constant c and all n >= n0 8. The function 10n2 + 4n+2 O(n)

The statement f(n) = O(g(n)) states only that g(n) is an upper bound on the value of f(n) for all n , n >= n0 . If f(n) = amnm + +a1n + a0 then f(n) = O(nm) .

Omega notation () : The function f(n) = (g(n)) iff there exist positive constants c and n0 such that f(n) >= c * g(n) for all n , n>= n0 . Examples: 1. The function 3n + 2 = (n) as 3n + 2 >= 3n for all n >= 2 2. The function 3n + 3 = (n) as 3n + 3 >= 3n for all n >= 1 3. The function 100n + 6 = (n) as 100n + 6 >= 100n for all n >= 1 4. The function 10n2 + 4n+2 = (n2) as 10n2 + 4n + 2 >= n2 for all n >= 1 5. The function 6*2n + n2 = (2n) as 6*2n + n2 >= 2n for all n>=1 For the statement f(n) = (g(n)) to be informative , g(n) should be as large a function of n as possible for which the statement f(n) = (g(n)) is True. If f(n) = amnm + +a1n + a0 and am > 0 then f(n) = (nm) . Theta notation(): The function f(n) = (g(n)) iff there exist

Examples: The function 3n + 2 = (n) as 3n + 2 >= 3n for all n >= 2 and 3n + 2 <= 4n for all n >= 2 so c1 = 3 , c2 = 4 and n0 = 2 The function 10n2 + 4n+2 = (n2) The function 6*2n + n2 = (2n) The function 3n + 2 (1) The function f(n) = (g(n)) iff g(n) is both an upper and lower bound on f(n). If f(n) = amnm + +a1n + a0 and am > 0 then f(n) = (nm) .

Little oh Notation (o): The function f (n) = o (g(n)) iff lim n f(n)/ g(n) = 0. Examples: 1. The function 3n + 2 = o (n2) since lim n 3n + 2 / n2 = 0 2. The function 3n + 2 = o (n log n) 3. The function 3n + 2 = o (n log log n) 4. The function 6*2n + n2 =o (3n) 5. The function 3n + 2 o (n ) 2. The function 6*2n + n2 o (2n) Little Omega Notation (): The function f(n) = (g(n)) iff lim n g(n)/ f(n) = 0.

Asymptotic Complexity of algorithms: Example 1: Statement 1 Algorithm Sum(a,n) 2{ 3. s := 0.0 ; 4. for i: = 1 to n do 1. s: = s + a[i]; 2. return s ; 3. } Total s/e 0 0 1 1 1 1 0 frequency 1 n+1 n 1 Total steps (0) (0) (1) (n) (n) (1) (0) (n)

Example 2:
Statement s/e Frequency n=0n>0 Total steps n=0n>0

Algorithm RSum(a,n) 1. { 2. if (n<=0)then 3. return 0.0; 4. else return 5. RSum( a,n-1) + a[n]; 6. }

0 0 1 1 1+x 0

1 1 0 -

1 0 1 -

0 0 1 1 0 0

(0) (0) (1) (0) (1+x) (0)

Total

(1+x)

Where x = t RSum(n-1)

Example 3:
Statement s/e frequency Total steps

1 Algorithm Add(a,b,c,m,n)
2. { 3. for i:= 1 to m do 4. for j: = 1 to n do 1. c[i,j]:= a[i,j] + b[i,j]; 2. }

0 0 1 1 1 0

(m) (mn) (mn) -

(0) (0) (m) (mn) (mn) (0) (mn)

Total

For m > n

Important Problem types

Sorting Searching String processing Graph problems Combinatorial problems Geometric Problems Numerical problems

Sorting
The Sorting problem asks us to rearrange the items of a given list in ascending order. We usually need to sort lists of numbers, Characters from an alphabet, Character strings, records maintained at the Schools , Companies. A piece of information to guide sorting is known as Key. Two properties of the algorithm are : Stable and In place. A sorting algorithm is called stable if it preserves the relative order of any two equal elements in its input. If an input list contains two equal elements in position i and j where i < j, then in the sorted list they have to be in position i and j respectively such that i < j

 The second notable feature of sorting algorithm is the amount of extra memory the algorithm requires. An algorithm is said to be in place if it does not require extra memory, except , possibly , for a few memory units.

Searching :
The Searching problem deals with finding a given value, called a search key, in a given set. There are different types of searching algorithms. There is no single algorithm that fits all situations best. Some algorithms work faster than others but require more memory. Some are very fast but applicable only to sorted arrays. There is no stability , but different issues arise.

 Specifically, in applications where the underlying data may change frequently relative to the number of searches, searching has to be considered in conjunction with two other operations, addition and deletion from the data set of an item. In this situations, data structures and algorithms should be chosen to strike a balance among the requirements of each operation.

String Processing:
A String is a sequence of characters from an alphabet. Text strings comprise letters, numbers and special characters. Bit Strings comprise of Zeros and ones. String processing is useful in Computer languages and compiling. Searching for given word in the text is known as string matching.

Graph Problems:
A graph can be thought of as a collection of points called vertices , some of which are connected by line segments called edges. Graphs can be used for modeling a wide variety of real life applications , including Transportation and Communication Networks, Project Scheduling and Games. Basic graph algorithms include Graph Traversal Algorithms, Shortest-path Algorithms , Graph Coloring Problems.

Combinatorial Problems:
The traveling salesman problem and the Graph coloring problems are the examples of Combinatorial problems. The combinatorial problems are very difficult since 1. The number of combinatorial objects grows extremely fast with a problems size , reaching unimaginable magnitudes even for moderated sized instances. 2. There are no known algorithms for solving most such problems exactly in an acceptable amount of time.

Geometric Problems:
Geometric algorithms deal with geometric objects such as points , lines and Polygons. Two classic problems are : The closest pair problem and the convex hull problem. The closest pair problem is self explanatory: given n points in the plane , find the closest pair among them. The convex hull problem asks us to find the smallest convex polygon that would include all the points of a given set.

Numerical Problems:
These problems involve mathematical objects of continuous nature: solving equations and systems of equations , computing definite integrals ,evaluating functions and so on. These problems can be solved in approximately.