PROBLEM TO PROGRAM

Mathematical Model
Informal algorithm

Abstract Data Type

(ADT)
Pseudo Language Code

Data Structure
(DS)
Program
(C-Program)

1) Model the problem using an appropriate mathematical model (Infromal algorithm)

2) The informal algorithm is written in pseudo language
3) The stepwise refinement of pseudo language gives various types of data used and
Operations to be performed on data. (i.e., data type)
4) We create ADT for each data type.
5) We choose an appropriate DS to implement each ADT.
6) Finally replace informal statements in pseudo language code by C-code.
An algorithm is a finite sequence of computational steps that transform the input into the output
in finite number of steps.
A data type is a collection of objects and a set of operations that act on those objects.
An abstract data type (ADT) is a data type that is organized in such a way that the specification
of the objects and the operations on the objects is separated from the representation of the objects
and the implementation of the operations. ADT is mathematical model of data type
A data structure(DS) is a way to store and organize data in order to facilitate access and
modifications. No single data structure works well for all purposes, and so it is important to
know the strengths and limitations of several of them. We use DS to implement ADT.
Pseudo code: Mixture of natural language and high level programming language constructs that
describes algorithm.
A program is an expression of an algorithm in a programming language.

1 INTRODUCTION
1. Non-Computational problem
A problem that has no precise and simple specification
Example: Convince your boss for salary hike, convince your faculty for marks.
2. Computational Problem
Specification of input

Specification output as function of input

Example: The sorting problem:

Input: a sequence <a1,a2,...,an> of n numbers.
Output: a permutation <a1',a2',...,an'> of the input with a1' <= a2' <= ... <= an'.
3. Algorithm Definition
Definition
An Algorithm is well-defined computational procedure that takes some value, or set of values, as
input and produces some value, or set of values as output.
Definition
A particular input is called an instance of a computational problem.
Example: The input sequence <31, 41, 59, 26, 41, 58> is an instance of the sorting problem.
4. Algorithm Characteristics
All algorithms should satisfy the following criteria.
Input: Zero or more quantities are externally supplied.
Output: At least one quantity is produced.
Definiteness: Each instruction is clear and unambiguous.
Finiteness: For all cases, the algorithm terminates after a finite number of steps.
Effectiveness: instruction is basic enough to be carried out
Definition: Algorithms that are definite and effective are called computational procedures.
Example: Digital computer.
Definition: An algorithm is said to be correct if, for every input instance, it halts with the correct
output.
5. The study of Algorithm includes the following important areas
Design an algorithm: Creating and algorithm is an art which may never be fully
automated.
Different design strategies:
Divide and Conquer, Greedy, Dynamic programmingetc.
Express an algorithm: Algorithm specification using Pseudo code.

Validate an algorithm (correctness): To show that algorithm computes the correct

answer for all possible legal inputs.
Analysis an algorithm: Find the time and space complexity. Prove that we cannot
solve the problem any faster using asymptotic analysis.
Implementation: Implementing algorithm in a programming language
Verification: Test the program (debugging and profiling)

6. Performance Evaluation
As an algorithm is executed, it uses the computers CPU to perform operations and its memory to
hold the program and data.
An efficient algorithm
needs less running time
uses less space
6.1 Space Complexity: The space complexity of an algorithm is the amount of memory it needs
to run to completion.
6.2 Time Complexity: The time complexity of an algorithm is the amount of computer time it
needs to run to completion.
6.3 Performance evaluation of an algorithm refers to the task of computing space and time
complexity of an algorithm.
Performance evaluation can be loosely divided into two major phases:
1) a priory estimates(Performance Analysis)
Uses analytical methods
Machine independent
2) a posterior testing (Performance Measurement or profiling):
It is the process of executing a correct program on data sets and measuring time
and space it takes to compute results
Machine dependent
Performance Analysis is general methodology because
It uses high level description of an algorithm(Pseudo-code)
All possible input instances are taken into account
Machine independent

7. Space Complexity
Components of space complexity
7.1 Instruction space: Space needed for code
7.2 Data Space:
(1) Space needed for constants and simple variables
(2) Space needed for dynamically allocated objects (Such as arrays, structures, etc.)
7.3 Environment stack space: It is used to save information needed to resume execution of
partially executed function.
Each time a function is invoked the following data are saved on the environment stack
The return address
The values of local variables and formal parameters
7.4 Recursion stack space: Amount of stack space needed by recursive functions is called
recursion stack space. It depends on
Space needed by local variables and formal parameters
Maximum depth of recursion (i.e., maximum number of nested recursive calls)
Compiler being used
The total space needed by a program is divided into two parts:
A fixed part independent of instance characteristics (e.g., size, number, value)
1. Instruction space
2. Data space( space needed for constants and simple variables and some dynamically
allocated objects)
Note: The space needed by some of the dynamically allocated memory may also be
independent of
problem size
3. Environment stack space for non-recursive functions
A variable part dependent on instance characteristics
1. Dynamically allocated space
2. Recursion stack space
7.5 Definition
The space complexity S(P) of any algorithm P can be written as
S(P) = C + SP(I)
C constant that denotes fixed part
SP Variable part that depends on instance characteristics (I) (e.g., size, number, value)

Examples
1.

Algorithm abc(a, b, c)
{
return a+b+b*c/(a+b+4.0);
}
C= Space needed for a, b, c and result;
SP(abc)=0
2.
Algorithm Sum(a,n)
{
s=0;
for(i=1 to n)
s=s+a[i];
return s;
}
Space required for
formal parameters a and n
local variables s ,i and constant 0
instruction space.
This amount of space needed does not depend on value of n.
Ssum(n) =0
Since a is actually the address of the first element in a[](i.e., a[0]), the space needed by it is also
constant.
3.
Algorithm rsum(int a[], int n)
{
if(n>0)
return rsum(a, n-1)+a[n-1];
return 0;
}
Let
reference of a =4 bytes;
value of n = 4 bytes;
return address = 4 bytes
are stored on recursion stack for each recursion call.
Each recursive call require 12 bytes
Depth of recursion=(n+1)
recursion stack space =12(n+1)
Srsum(n) = 12 (n+1)

8. Time Complexity
Time taken by a program P is sum of compile time and runtime
T (P) =C+TP (I)
C (compile time) is independent of instance characteristics ( constant).
TP (I) (Run time) is dependent on instance characteristics.
(i) However, analytical approach to determine the exact runtime is complicated.

Since runtime depends on machine dependent issues like i) Type ofprocessor, ii) Access
rate (read/write operations), iii) Architecture and machine independent issue iv) input
size.

(ii) Run time expression should be machine-independent.

Therefore, we estimate runtime as function of input size. i.e., we find rate of growth of time
with respect to input size.
Running time= f(input size)
(iii) We use parameters that characterize input size.
Example:
Problem
Searching
Sorting
Matrix multiplication
Graphs

Size of input
number of elements in array(one parameter)
Number of items to be sorted.
i.e., number of elements in array(one parameter)
Dimension of matrices(pxq, qxr)(three parameters)
numbers of vertices and edges.(Two parameters)

(iv) Assumption
We define notion of step as machine independent as possible.
Let each line/statement/step of our pseudo code execute in constant time so that analysis of
algorithm is machine independent
Definition: Total runtime (Time Complexity)
The running time of the algorithm is the sum of running times for each statement executed.

=

( ) (. )

Example:

Program/Pseudo code
int sum(int a[], int n)
{
int s=0;
int i;
for(i=0;i<n;i++)
s+=a[i];
return s;
}

cost No. of Times

C1
C2
C3
C4
C5

1
0
n+1
n
1

T (n) =C1+C3(n+1)+C4 *n+C5

= (C3+C4)n+(C1+C3+C5)= An + B
8.1 Order of growth
Another abstraction to ease analysis and focus on the important features.Look only at the leading
term of the formula for running time.
Drop lower-order terms.(For large n, their effect is negligible)
T (n) ~ An
Ignore the constant coefficient in the leading term.(As they are machine dependent and
less significant)
T (n) ~ n
i.e., worst-case running time T (n) grows like n.
We write
T(n) = (n)

8.2 BEST CASE, WORST CASE, AVERAGE CASE

Best-case time-complexity: The smallest running time performed by an algorithm
given any input of size n.
Worst-case time-complexity: The smallest running time performed by an algorithm
given any input of size n.
Average case time-complexity: The expected running time performed by an algorithm
given any input of size n.
NOTE: Mostly the worst-case complexity is used to compare the efficiency of two algorithms
since
1) The worst-case running time gives a guaranteed upper bound on the running time for any
input.
2) Worst case occur fairly often.
2) Average case is as bad as worst case .
3) Finding average case is difficult.

Example:Linear search
Pseudo code
LinearSearch(A, n, k)
A[n+1]=k;
i=1;
while(A[i]k)
i++;
if(i<=n)
return i;
else
return -1;

cost times
C1
C2
C3
C4
C5

1
1
ti
ti-1
1

Where, ti= No. of times while loop executed.

Best case: Search element is the first element
ti = 1
T (n) = C
T(n) = (1)
Worst case: Search element is not found or last element
ti= n+1
T (n) =An+B
T (n) = (n)
Average case:
Let probability of key being found at any position is equally likely.
i.e., prob. that key found at position i=Prob(i)= 1/n for each index 1<=i<=n
The probability distribution
X
1
2
3
. n
p(X) 1/n 1/n 1/n .. 1/n
Expected value of X
T (n) = E(X) = x. p(x) = 1.1/n+2.1/n+3.1/n+..+n.1/n
= 1+2+3+..+n/n
= n(n+1)/2.n
= (n+1)/2
T (n) = (n)
Best case Worst case Average case
Successful search
(1)
(n)
(n)
Unsuccessful search (n)
(n)
(n)

9. Sorting Problem
Input: a sequence <a1,a2,...,an> of n numbers.
Output: a permutation <a1',a2',...,an'> of the input with a1' <= a2' <= ... <= an'.
Usually, the numbers to be sorted are part of a collection of data called a record. Each record
contains a key, which is the value to be sorted.
Example of a record
Key Other data
9.1 Some definitions
Internal Sort
The data to be sorted is all stored in the computers main memory(RAM).
External Sort
Some of the data to be sorted might be stored in some external, slower, device.
In Place Sort
The amount of extra space required to sort the data is constant w.r.to input size.
Stable sort
A STABLE sort preserves relative order of records with equal keys
Ideal Sorting
The ideal sorting algorithm would have the following properties:
Stable: Equal keys aren't reordered.
Operates in place, requiring O(1) extra space.
Worst-case O(nlg(n)) key comparisons.
Worst-case O(n) swaps.
Adaptive: Speeds up to O(n) when data is nearly sorted or when there are few unique
keys.
There is no sorting algorithm that has all of these properties, and so the choice of sorting
algorithm depends on the application.
Definition
Invariant
An invariant is a property of data that is unmodified by an algorithm, typically a set of
relationships that hold between elements.
Finding invariants makes it easier to prove properties of the algorithm.

SELECTION SORT
Pseudocode
Selection sort (A,n)

Cost Times

for (i=1to n-1)

C1

min=i;

C2

n-1

for (j=i+1 to n)

C3

n
n1
n1
i=1 j=i+1 1 = i=1 (n i)// Comaprisons

if (A[j] < A[min]) C4

min=j;
swap(A[i],A[min]);

C5

n1
i=1 (n i) //Swaps
n-1

Loop Invariant
At start of each iteration of outer for loop, the sub array A[1.j-1] consists the j-1 smallest
elements in the array A[1n] in the sorted order.
Worst Case Analysis
n1 n1

() = 1
i=1 j=i+1
n1
i=1 n

=
i1
=(n-2)+(n-3)+.+1=(n-1)(n-2)/2=(n2)

Properties
1 Complexity
Best-case Worst-case Average-case
(n2)
(n2)
(n2)
2. Not Stable
3. In-place
4. (n2) comparisons
5. (n) swaps(Exchanges)
6. Selection sort takes about n2/2 comparisons and n swaps
7. Not Adaptive
8. Selection sort is used for sorting files with very large values and small keys.

BUBBLE SORT
BubbleSort(A, n)
for i = 1 to n-1 do
for j = 1 to n-1-i do
If (a[j+1] < a[j])
swap a[j] anda[j+1]
BubbleSort_modified(A, n)
for i = 1 to n-1 do
count=0;
for j = 1 to n-1-i do
If (a [j+1] < a[j])
Swap (a[j] ,a[j+1])
count++;
if(count==0) return
Loop Invariant:
At the start of each iteration of the outer for loop, the subarray A [n-i+1..n] consists of the i
largest elements in the array A[1 . . n] in sorted order.
Properties
1 Complexity
Best-case
Worst- Averagecase
case
2
(n ) BubbleSort
(n2)
2
(n) BubbleSort_modified (n )
2. Stable
3. In-place
4. O (n2) comparisons and O (n2) swaps
5. About n2/2 comparisons and n2/2 swaps in both average and worst case
6. Adaptive

INSERTION SORT
Pseudo code
INSERTION-SORT(A, n)

Cost Times

for j= 2 to n

C1

C2

n-1

i =j 1

C3

n-1

while (i> 0 and A[i] > key) //comparisons

C4

nj=2 t j // comparisons

key= A[ j]
//Insert A[ j ] into the sorted sequence A[1 . .j -1]

A[i + 1] = A[i]

//exchanges

C5

i=i 1

C6

A[i + 1] = key

C7

nj=2 t j 1 // exchanges

nj=2 t j 1
n-1

tj: No. of times the while statement is executed at iteration j

T (n) = c1 n + c 2 (n 1) + c3 (n 1) + c 4 t j + c5 (t j 1) + c6 (t j 1) + c7 (n 1)
n

j =2

j =2

j =2

Best Case Analysis

The array is already sorted. i.e., Always A[i] key in the while loop test
tj = 1
T (n) = c1n + c2(n -1) + c3(n -1) + c4(n -1) + c7(n-1) = (c1 + c2 + c3 + c4 + c7)n (c2 + c3 + c4 + c7)
T (n) = A n + B = (n)
Worst Case Analysis
The array is in reverse sorted order i.e., Always A[i] > key in while loop test
Have to compare key with all elements to the left of the j-th position
Compare with j-1 elements
tj = j
Using n
n
n
n(n + 1)
n(n + 1)
n(n 1)
=
=
>
=

=
>
j
j
1
( j 1) =

2
2
2
j 2
j 2
=j 1 =
=
n(n 1)
n(n 1)
n(n + 1)
T (n) = c1 n + c 2 (n 1) + c3 (n 1) + c 4
1 + c 5
+ c6
+ c7 (n 1)
2
2
2

T (n) = A n2+ Bn+C

a quadratic function of n
2
T(n) = (n )
order of growth in n2

Average case analysis

On average, the key in A[j] is less than half the elements in A[1 j 1] and its greater than the
other half.
On average, the while loop has to look halfway through the sorted subarray A[1 j 1] to
decide where to drop key.
tj= j/2.
T(n) = (n2)
order of growth in n2
Example:
Input array
5

Loop Invariant:
At the start of each iteration of the outer for loopthe loop indexed by jthe subarray
A[1 . . j-1] consists of the elements originally in A[1 . . j-1] but in sorted order.
Properties
1. Complexity
Best-case
(n)

Worst-case
(n2)

Average-case
(n2)

2. Stable
3. In-place
4. O (n2) comparisons and O (n2) swaps
5. Insertion sort takes about n2/4 comparisons and n2/8 swaps on random input (average case)
and in the worst case, they are double.
6. Adaptive (Insertion sort is linear for almost sorted arrays)

General rules
1. Assignment O(1)
2. Linear for loop
O(Number of iterations in a loop * Number of statements inside the loop)
Example:for( i=0 to n)
{
print ( hi );
x = x+1;
y =y+1;
}
T ( n ) = ( 3 * n ) = (n)
3. Nested Loops:
Analyze the innermost loop first, complexity of next outer loop = number of iterations in this
loop * complexity of inner loop, etc..
Example:
sum = 0;
for (i=0; i < N; i++)
for (j=0; j < N; j++)
sumsum + 1;
Inner loop: (N)
Outer loop: N iterations
Overall: (N2)
4. If-then-else statements:
Worst-case running time: the test, plus the if part or the else part (whichever is the larger).
if (Condition)
S1
else
S2
5. Consecutive statements: Add the time complexities of each statement.

Practice Problems
1. Best case for an algorithm
(a) takes the same time for all data;
(b) assumes the data that the algorithm handles in the greatest time;
(c) assumes the data that the algorithm handles in the least time;
(d) is the expected time considering all possible input data;
2. Worst case for an algorithm
(a) takes the same time for all data;
(b) assumes the data that the algorithm handles in the greatest time;
(c) assumes the data that the algorithm handles in the least time;
(d) is the expected time considering all possible input data;
3. Average case for an algorithm
(a) takes the same time for all data;
(b) assumes the data that the algorithm handles in the greatest time;
(c) assumes the data that the algorithm handles in the least time;
(d) is the expected time considering all possible input data;
4. The time complexity of Insertion sort is
(a) O(lg n); (b) O(n);
(c) O(n2);

(d) O(2n)

5. Selection sort uses

(a) divide and conquer;
(c) the greedy approach;

(b) brute force;

(d) dynamic programming

6. What is the complexity of the following C code segment:

for(i=1;i<=n;i++)
x=x+1;
(a) (lg n)
(b) (n)
(c) (n lg n)

(d) (n2)

7. What is the complexity of the following C code segment:

for(i=1;i<=n;i+=2)
x=x+1;
(a) (lg n)
(b) (n)
(c) (n lg n)

8. What is the complexity of the following C code segment:

for(i=1;i<=n;i*=2)
x=x+1;
(a) (lg n)

(b) (n)

(c) (n)

9. What is the complexity of the following C code segment:

for(i=n;i>=1;i/=2)
x=x+1;
(a) (lg n)
(b) (n)
(c) (n lg n)

10. What is the complexity of the following C code segment:

for(i=1;i<=n*n;i++)
x=x+1;

(d) (n2)

(d) (n lg n)

(d) (n2)

(a) (lg n)

(b) (n)

(c) (n2)

(d) (n n))

(a) (lg n)

(b) (n)

(c) (n2)

(d) (n n)

11. What is the complexity of the following C code segment:

for(i=1;i*i<=n;i++)
x=x+1;

12. What is the complexity of the following C code segment:

for(i=n/2;i<=n;i++)
x=x+1;
(a) (n) (b) (n)

(c) (n2)

(d) (n n)

(a) (1)

(b) (lg n)

(c) (n)

(d) (n)

(a) (1)

(b) (lg n)

(c) (n)

(d) (n)

13. What is the complexity of the following C code segment:

for(i=n/2;i<=n;i*=2)
x=x+1;

14. What is the complexity of the following C code segment:

j=0;
for(i=1;i<=n;i=i+j)
j++;

15. What is the complexity of the following C code segment:

j=0;
for(i=1;i<=n;i=j+i)
{
j++;
break;
}
(a) (1)

(b) (lg n)

(c) (n)

(d) (n)

(a) (1)

(b) (lg n)

(c) (n)

(d) (n)

16. What is the complexity of the following C code segment:

j=0;
for(i=1;i<=n;i=j+i)
{
j++;
continue;
}

17. What is the complexity of the following C code segment:

for(i=1;i<=n;i++)
for(j=1;j<=n;j++)
x=x+1;
(a) (n)
(b) (n logn) (c) (n2)
(d) (n3)

18. What is the complexity of the following C code segment:

for(i=1;i<=n;i++)
for(j=1;j<=n;j*=2)
x=x+1;
(a) (n)
(b) (n logn) (c) (n2)

19. What is the complexity of the following C code segment:

for(i=1;i<=n;i++)
for(j=n/3;j<=2n;j+=n/3)
x=x+1;
(a) (lg n)
(b) (n)
(c) (n lg n)
20. What is the complexity of the following C code segment:
for(i=1;i<=n;i++)
for(j=1;j<=n;j=2*j)
x=x+1;
(a) (lg n)
(b) (n)
(c) (n lg n)

(d) (n3)

(d) (n2)

(d) (n2)

21. What is the complexity of the following C code segment:

for(i=n;i>=1;i/=2)
for(j=1;j<=n;j=2*j)
x=x+1;
2
(a) (lg n)
(b) (n)
(c) (n lg n)

22. What is the complexity of the following C code segment:

for(i=1;i<=n;i++)
for(j=1;j<=n;j=2*j)
for(k=1;k<=n;k=k+n/2)
x=x+1;
(a) (nlg n)
(b) (n2 lg n) (c) (n lg2 n)

23. What is the complexity of the following C code segment:

for(i=1;i<=n;i++)
for(j=1;j<=n;j=j+i)
x=x+1;
(a) (lg n)
(b) (n)
(c) (n lg n)
24. What is the complexity of the following C code segment:
for(i=n/2;i<=n;i++)
for(j=1;j<=n;j+=n/2)
for(k=1;k<=n;k*=2)
x=x+1;
(a) (lg n)
(b) (n lg n) (c) (n2 lg n)

25. What is the complexity of the following C code segment:

for(i=n/2;i<=n;i++)
for(j=1;j<=n;j*=2)
for(k=1;k<=n;k*=2)
x=x+1;
(a) (n)
(b) (n lg n) (c) (n lg 2n)
26. What is the complexity of the following C code segment:
for(i=1;i<=n;i++)
{
for(j=1;j<=n;j++)
for(l=1;i<=i;l++)
for(k=1;k<=i;k++)
x=x+1;
}
(a) (n2)
(b) (n3)
(c) (n3/2)

(d) (n2)

(d) (n2)

(d) (n2)

(d) (n)

(d) (n2 lg n)

(d) (n5/2)

27. What is the complexity of the following C code segment:

for(i=1;i<=n;i++){
for(j=1;j<=n;j++)
x=x+1;
for(j=1;j<=n;j*=2)
x=x+1;
}
(a) (lg n)
(b) (n )
(c) (nlg n)

28. What is the complexity of the following C code segment:

void function (int n)
{
int i,j,k,count=0;
for(i=1;i<=n;i++)
for(j=1;j<=n*n;j++)
for(k=1;k*k<=n;k++)
count++;
}
(a) (n5/2)
(b) (n3 )
(c) (n7/2)
29. What is the complexity of the following C code segment:
void function (int n){
int i,j;
if(n==1)
return;
for(i=1;i<=n;i++)
for(j=1;j<=n*n;j++)
break;
}
(a) (n)
(b) (n2 )
(c) (n3)
30. What is the complexity of the following C code segment:
void function (int n){
int i=1;
while(i<n)
{
int j=n;
while(j>0)
j=j/2;
i=2*i;
}
}
(a) (n2)
(b) (lg2n )
(c) (n lgn)

(d) (n2)

(d) (n5)

(d) (n4)

(d) (n)

31. Consider the following C-program fragment in which i, j and n are integer variables.
j=0;
for (i = n, ; i > 0; i /= 2)
j +=i;
Let val (j) denote the value stored in the variable j after termination of the for loop. Which
one of the following is true?
(A) val (j) = ( log n)
(B) val (j) = ( n)
(C) val (j) = (n )
(D) val (j) = (n log n)
32. Consider the following function:
int unknown int n {
int i, j, k=0;
for(i=n/2; in; i++)
for(j=2; jn; j=j*2)
k=k+n/2;
return k;
}
The return value of the function is
(A) (n2)

(B) (n2 lg n) (C) (n3)

(D) (n3 lg n)