Algorithm Correctness & Analysis

Algorithm
A finite set of instructions which if followed accomplish a
particular task. In addition every algorithm must satisfy
following criteria:
1. Input: zero or more quantities externally supplied
2. Output: at least one quantity is produced
3. Definiteness: Each instruction must be clear and
unambiguous.
4. Finiteness: In all cases algorithm must terminate after
finite number of steps.
5. Effectiveness: each instruction must be sufficiently
basic.

 Two algorithms on two systems

Algorithm A1
Algorithm A2
A2

A1

50 n lg n
2 n2

Super computer
108 ins/sec
Time taken by Super Computer
= 2.(106)2/ 108 = 20,000 sec
P.C
106 ins /sec

Time taken by P .C.

= 50 .106 lg 106 / 106 = 1,000 sec

Thus by using a fast algorithm, the personal computer

gives results
20 times faster than the result given by super computer
using a slow algorithm.
Thus a good algorithm is like a sharp knife, it does
exactly what it is supposed to do with a minimum
amount of effort.

Complexity
Some questions to answer:
How fast can we solve a problem?
There may be many algorithms for a given problem.
Which algorithm to use?
What are the classical algorithm design techniques ?
Are there problems inherently difficult to solve?

How do we express the complexity of algorithm?

Resources : Time and Space

Complexity lower bounds for problems.

Complexity classes P, NP etc.

Pseudocode
Pseudocode is an English language like representation of

the code required for an algorithm

It is partly English, partly structured code.
The English part provides a relaxed syntax that is easy to
read.
The code part consists of an extended version of the basic
algorithmic constructs-sequence, selection and iteration.

Sequence, selection, loop

A sequence is a series of statements that do not alter the

execution path within an algorithm.

Statements such as assign and add are sequence statements.
A call to another algorithm is also considered a sequence
statement.
Selection statements evaluate one or more alternatives.
Paths are followed based on its result.
The typical selection statement is the two way selection if
(condition) action 1 else action 2.
The part of the loop are identified by indentation.
Loop iterates a block of code. It closely resembles the while
loop. It is a pretest loop.

Example
Algorithm: deviation

It finds deviations from average.

Pre: nothing

Post: numbers are read and deviation from average

printed

1
2

3
4
5
6

7
8

i= 0
Loop(all data are read)
1
I=I+1
2
read numbers into array[i]
3
sum = sum + number
Average = sum / I
Print (average)
J=0
Loop (j < i)
1
j = j+ 1
2
dev = array[j] average
3
print (array [ j] . Dev)
Return
End deviation

Summary
Confidence in algorithms from testing and
correctness proof

Correctness of recursive algorithms proved

directly by induction
Examples: Fibonacci numbers, maximum,
multiplication

Correctness
How do we know that an algorithm works?

Logical method for checking correctness

Testing
Correctness proof
Testing vs. Correctness Proofs
Testing: try the algorithm on sample inputs
Correctness Proof: Prove mathematically; testing may
not found obscure bugs
Using testing alone can be dangerous

Correctness of Recursive
algorithms
To prove correctness of recursive algorithm:
Prove it by induction on the size of the problem being
solved
Base of recursion is base of induction
Need to prove the recursive calls are given sub-problems,
i.e., no infinite recursion
Inductive step: assume that recursive calls work
correctly, and use this assumption to prove that the
current call works correctly

Recursive Fibonacci Numbers

Fibonacci numbers: F0 = 0, F1 =1, and for all n 2,
Fn = Fn-2 + Fn-1

Recursive Fibonacci Numbers

Claim: For all n 0, fib(n) return Fn
Base: For n =0, fib(n ) returns 0 as claimed. For n = 1, fib(n)
returns 1 as claimed.
Induction: Suppose that n 2 and for all 0 m < n, fib (m)
returns Fm .
Required to prove fib(n) returns Fn
What does fib(n) returns?

fib(n-1) + fib(n-2) = Fn-1 + Fn-2

= Fn

(by Ind. Hyp.)

Recursive Maximum

Recursive maximum

Recursive Multiplication

Recursive Multiplication

Recursive Multiplication

Analysis

Summary

Constant Multiples
Analyze the resource usage of an algorithm to within a constant
multiple.
Why? Because other constant multiples creep in when translating
from an algorithm to executable code:
Programmer ability
Programmer effectiveness
Compiler
Computer hardware
Recall: Measure resource usage as a function of input size

Big oh

Example
Most big-Os can be proved by induction.
First Example: log n = O(n).

Claim : For all n >1, log n <n . The proof is by induction on n.

The claim is trivially for n=1,since 0<1. Now suppose n > 1 and
log n < n. Then,
log (n+1)
< log 2n
= log n+1
< n+1 (by ind. hyp.)

Second example

Note that we need

Big Omega

Big Theta

True or false

Adding Big Ohs

Continue

Multiplying Big Ohs

Types of Analysis

Example

Time Complexity

Multiplication

Bubble sort

Analysis Trick

Example

Lies, Damn Lies, and Big-Os

Algorithms and Problems

Algorithms Analysis 2

The Heap

Contd

To Delete the Minimum

But we have lost the tree structure

Contd

But we have violated the structure condition

Contd

Contd

What Does it work?

Contd

Contd

To Insert a New Element

Contd

Contd

Contd

Why Does it Work

Contd

Implementing a Heap

Analysis of Priority Queue

Operation

Analysis of l(n)

Contd

Example

Heapsort

Building a Heap Top Down

Contd

Contd

Building a Heap Bottom Up

Continue

Continue

Algorithms Course Notes

Algorithm Analysis 3
summary
Analysis of recursive algorithms:
Recurrence relations
How to derive them
How to solve them

Deriving Recurrence Relations

Continue

Example

Continue

Analysis of Multiplication

Solving Recurrence Relations

The Multiplication Example

Repeated Substitution

Warning

Reality Check

Merge Sorting

Continue

Continue

A General Theorem

Proof Sketch

Continue

Geometric Progressions

Back to the Proof

Continue

Messy Details

Continue

Example