MT 203 Analysis and Design of Algorithms

Contents Unit 1 Introduction to Algorithms Unit 2 Fundamentals of the Analysis of Algorithm Efficiency Unit 3 Mathematical Aspects and Analysis of Algorithms Unit 4 Mathematical Aspects and Analysis of Algorithms 2 Unit 5 Brute Force Method Unit 6 Divide and Conquer Unit 7 Decrease and Conquer Unit 8 Transform and Conquer Unit 9 Space and Time Tradeoffs
Edition: Fall 2011 BKID B1480 2
nd

1 20 42 64 87 107 133 155 182

May 2011

Unit 10 Dynamic Programming 1 Unit 11 Dynamic Programming 2 Unit 12 Greedy Technique Unit 13 Limitations of Algorithm Power Unit 14 Coping with the Limitations of Algorithm Power

207 229 254 275 296

Dean Directorate of Distance Education Sikkim Manipal University (SMU DDE) Board of Studies Chairman Head IT SMU DDE, Manipal 576 104 Additional Registrar SMU DDE Manipal 576 104 Addl. Registrar (S.E.) Office of the Student Evaluation Manipal 576 104 Dr. N. V. S. Subba Reddy Director, Sikkim Manipal Institute of Technology Gangtok, Sikkim Dr. Ramprasad Vardhachar Director, Computer Studies Dayanand Sagar Institutions, Bangalore Prof. A. Srinivas Director - Research and Consultancy and Professor of Computer Science PES Institute of Technology Bangalore 560 085 Content Preparation Team Team Triumph Triumph India Software Services Pvt. Ltd. Bangalore 560 094 Edition: Fall 2011 Printed: May 2011 This book is a distance education module comprising a collection of learning material for our students. All rights reserved. No part of this work may be reproduced in any form by any means without permission in writing from Sikkim Manipal University, Gangtok, Sikkim. Printed and published on behalf of Sikkim Manipal University, Gangtok, Sikkim by Mr. Rajkumar Mascreen, GM, Manipal Universal Learning Pvt. Ltd., Manipal 576 104. Printed at Manipal Press Limited, Manipal. Mr. Nirmal Kumar Nigam Asst. Prof., Dept. of IT SMU-DDE, Manipal Dr. A. Kumaran Research Manager, Multilingual Research, Microsoft (India) Mr. Ashwin Krishna Director, Keystone Business Bangalore Mr. Srinivas Deeduvanu Engineering Manager Yahoo (India), Bangalore Dr. Rajasekar Image Quality Leader Wipro - GE (India), Bangalore Dr. Ashok Kumar T. K. General Manager IBM (India), Bangalore

Content Editor Mr. Arvind Singh Lecturer, Dept. of IT SMU DDE, Bangalore 560 008

SUBJECT INTRODUCTION
Algorithm was first formulated by a Persian mathematician named Abu Abdullah Muhammad Ibn Musa al-Khwarizmi in the early 9th century. Since then algorithms have been in use in various different areas of research and study. Each of these areas has a different definition for algorithm with respect to their objective. In software programming, algorithms are defined as a finite set of rules that defines the sequence of operations to solve a specific set of goals and has five important features: finiteness, definiteness, input, output, efficiency. For the assurance of these five features, there are methods to analyze and design algorithms. The three ways to analyze algorithms are Mathematical analysis, Empirical analysis and Algorithm visualization. There are some standards to calculate the algorithm efficiency as well, known as asymptotic notations. Algorithms are formed using different methods. They are as given below: Brute force method It is a problem solving technique where a series of possible answers are computed and the possibility for each accurate answer is tested. Divide and Conquer method This method divides the problem into sub problems, solves it recursively and finally combines these solutions to generate a solution for the main problem. Decrease and Conquer method The principle idea this algorithm is to solve a problem by reducing its instance to a smaller one and then extending the obtained solution to get a solution to the original instance. Transform and Conquer method This method transforms the problem into a level where the solution can be obtained and then solves the problem. Dynamic Programming method It is an optimization technique for particular classes of backtracking algorithms which repeatedly solve subproblems to find the solution of the main problem. Greedy method The Greedy technique constructs a solution to an optimization problem through a sequence of choices, each expanding a

partially constructed solution until a complete solution to the problem is reached. The execution of an algorithm can be done either in a short time interval with more memory space or larger time interval with less memory space. This selection of one alternative over the other is known as the tradeoff. Some problems which can be solved using algorithms are not solved within polynomial time. Even if the some problems are solved within the polynomial time, the efficiency of the algorithm is low. These limitations are overcome by solving those problem instances which tends to give a solution in polynomial time and discarding the exponential time solutions. Unit 1: Introduction to Algorithms This unit covers the various definitions of algorithm, its types, properties and the steps for designing it. This unit gives a brief idea on solving a problem using different algorithmic problem solving techniques. It also introduces data structures, their classification and characteristics. Unit 2: Fundamentals of the Analysis of Algorithm Efficiency This unit covers algorithm analysis framework with respect to complexity of an algorithm. It gives an idea about various methods used for analyzing algorithms and explains the techniques involved in amortization. It also includes some case studies in algorithm analysis. Unit 3: Mathematical Aspects and Analysis of Algorithms This unit explains the types of asymptotic notations. It lists the basic asymptotic efficiency classes. It also describes the efficient analysis of nonrecursive algorithms with illustrations. Unit 4: Mathematical Aspects and Analysis of Algorithms 2 This unit defines recursion and recursive algorithm with examples. It also discusses the empirical analysis of algorithms and algorithm visualization with examples. Unit 5: Brute Force Method This unit defines brute force method and analyzes selection sort and bubble sort with this method. It analyzes and computes sequential search and brute force string matching. It also explains and discusses the exhaustive search.

Unit 6: Divide and Conquer This unit explains the Divide and Conquer methodology which is a wellknown general design methodology. It performs different sorting techniques, builds and traverses a binary tree. It also applies the Strassens matrix method using the Divide and Conquer methodology. Unit 7: Decrease and Conquer This unit explains the concepts of decrease and conquer and the methodology it uses in various algorithms. It explains the algorithm used for insertion sort, Depth-first search, Breadth-first search and topological sorting. It also analyzes the algorithm for generating combinatorial objects. Unit 8: Transform and Conquer This unit defines the technique of transform and conquers. It describes the method of presorting, the Gaussian elimination technique and the approach of AVL and 2-3 trees in balanced search trees. It also defines heap sort and the problem reduction strategy. Unit 9: Space and Time Tradeoffs This unit explains the importance of space-time tradeoff in programming. It describes the process of sorting by counting and the input enhancement in string matching. It explains hashing technique the B-Tree technique with respect to space and time tradeoffs. Unit 10: Dynamic Programming 1 This unit introduces the Dynamic Programming technique with an example to calculate the nth Fibonacci number. It explains the dynamic programming approach to compute binomial coefficients. It also describes the Warshalls and Floyds algorithms. Unit 11: Dynamic Programming 2 This unit defines the Principle of Optimality and analyzes binary search trees using dynamic programming. It also introduces the Knapsack problem and solves an instance of it using dynamic programming and memory functions. Unit 12: Greedy Technique This unit you covers the concepts of Greedy technique algorithms that are used for optimization problems such as Kruskal's algorithm and Prim's algorithm for finding minimum spanning trees. It also describes the Dijkstra's

algorithm for finding single-source shortest paths, and the algorithm for finding optimum Huffman trees. Unit 13: Limitations of Algorithm Power This unit covers the limitations of algorithm power with respect to lower bound arguments of algorithms. It explains decision trees with examples. It also analyzes P, NP and NPcomplete problems. Unit 14: Coping with the Limitations of Algorithm Power This unit explains how to cope with some of the limitations of algorithms power. It analyzes the algorithm technique of Backtracking and explains the solution strategy of Branch-and-Bound algorithms. This unit also discusses approximation algorithms for NP-Hard problems like the Traveling Salesman and the Knapsack problems. Objectives of studying the subject After studying this subject, you should be able to: Describe the fundamentals of algorithms and data structures Explain different methods for analyzing the algorithms Explain the types of asymptotic notations Define and describe recursion and recursive algorithms Explain the empirical analysis of algorithms and algorithm visualization Define and describe brute force method with examples Explain the Divide and Conquer methodology with examples Analyze Decrease and Conquer methodology with examples Define the technique of Transform and Conquer with examples Explain the importance of space-time tradeoff in programming Explain the Dynamic Programming approach Describe the Greedy technique with examples List the limitations of algorithm power Describe the ways to cope with the limitations of algorithm power.