Algorithm definition

Different types of efficiency

worst-case, best-case, average-case, amortized

Algorithm types
Polynomial time algorithms, exponential algorithms, exact algorithms, approximation
algorithms

Asymptotic notations
Big theta, Big omega, Big oh

Methods of specifying an algorithms
pseudocode, flowchart

Analysing efficiency of non-recursive algorithm
input size
basic operation
case efficiency
setup a sum
find closed form formula or order of growth

Analysing efficiency of recursive algorithm
input size
basic operation
case efficiency
setup a recurrence relation
solve recurrence or find order of growth

Divide and Conquer technique
algorithm
recurrence relation
Problems: Quick sort, Merge sort, Matrix multiplication, multiplication of large integers

Multiplication of large integers
pad with leading zeros
number of digits should be power of 2
formula
M(n) = n
1.585

Brute force approach, M(n) = n
2

C = a * b a first number bsecond number n digits
o a = a
1
a
o
b= b
1
b
0

Formula
C= C
2
10
n
+ C
1
10
n/2
+ C
0

C
2
= a
1
* b
1

C
0
= a
0
* b
0

C
1
= (a
1
+ a
0
)* (b
1
+ b
0
) (C
2
+C
0
)

Strassens matrix multiplication
order of matrix should be power of 2


pad with zeros
7 multiplication and 18 additions/subtractions
Brute force algorithm requires 8 multiplications and 4 additions, M(n)=n
3

M(n) = n
2.807

( A
11
A
12
) ( B
11
B
12
) ( C
11
C
12
)
( A
21
A
22
) * ( B
21
B
22
) = ( C
21
C
22
)
C
11
= m
1
+m
4
-m
5
+m
7

C
12
= m
3
+m
5

C
21
= m
2
+ m
4

C
22
= m
1
+m
3
-m
2
+m
6

Where
m
1
= (A
11
+A
22
) (B
11
+B
22
)
m
2
= (A
21
+A
22
)B
11

m
3
= A
11
(B
12
-B
22
)
m
4
= A
22
(B
21
-B
11
)
m
5
= (A
11
+A
12
)B
22

m
6
= (A
21
-A
11
)(B
11
+B
12
)
m
7
= (A
12
-A
22
)(B
21
+B
22
)

Dynamic Programming
Bellmans principle of optimality
Table is maintained
Uses previously computed value to obtain current value
Problems: Computing binomial coefficient, optimal binary search trees, knapsack
problem, transitive closure, all pairs shortest path

Greedy Method
locally optimal, feasible, irrevocable
algorithm
Problems: Minimum spanning tree

Warshall Algorithm
transitive closure
Input: adjacency matrix
Find R
0
,R
1
,,R
n

R
ij
k
= r
ij
k-1
OR (r
ik
k-1
and r
kj
k-1
)
If no edge, edge value is zero
Output: Transitive closure binary(boolean) matrix

Floyds Algorithm
all pairs shortest path
Input: weight matrix, w
Find D
0
,D
1
,,D
n

D
ij
k
=min{d
ij
k-1
, d
ik
k-1
+ d
kj
k-1
} k>=1
D
ij
0
= w
ij

Diagonal values are zero
If no edge, edge value is infinity
Output: all pairs shortest path - weight matrix

Optimal Binary Search trees


catalan number
average number of comparisons
{ }

=
s s
+ + + =
j
i s
s
j k i
p j k C k i C j i C )} , 1 ( ) 1 , ( min ) , (
C(i,i-1)=0 1in+1
C(i,i)=p
i
1in
R(i,i)=1, 1in RRoot table

Knapsack problem
technique: Greedy / Dynamic programming / backtracking / branch & bound
problem statement
Maximize E v
i
x
i
i=1 to n
Subject to Ew
i
x
i
<W
where v
i
>0, w
i
>0 and x
i
e{0,1} / 0<x
i
<1
Formula
V(i,j)=Max{V(i-1,j), V(i-1,j-w
i
)+v
i
} } j-w
i
0
V(i-1,j) j-w
i
<0
If V(i-1,j) is larger, the object i is discarded.
If V(i-1,j-W
i
)+v
i
is larger, the object i is included

o initial conditions are
V(0,j)=0 j>0 V(i,j)=- for all i when j<0
V(i,0) = 0 i0
o solution is found at V(n,W)

Prims Algorithm
finds minimum spanning tree
steps, data structures used for implementation

Kruskals Algorithm
finds minimum spanning tree
sort the edges, forms a forest
Union & Find algorithms
data structures used for implementation

Dijkstras Algorithm
finds single-source shortest path

Backtracking
State-space tree
Promising node, Non-promising node
Implicit constraints, Explicit constraints
Solution tuple, D-search
Non-optimization problem
Stack, queue method in numbering nodes in state-space tree

N-queens problem
Problem statement
Diagonal iff |j-l|=|i-k|


Each row has its corresponding queen
Stack method node numbering
Implicit constraints, Explicit constraints

Hamiltonian Circuit
problem statement
Implicit constraints, Explicit constraints

Sum of Subset
Problem statement
2 types of formulation, Implicit constraints, Explicit constraints

Branch and Bound
optimization problems
idea behind branch & bound, best first branch & bound strategy

Assignment problem
problem statement
cost matrix
no two selected elements are in the same column & their sum is the smallest possible

Knapsack problem
problem statement
v
1
/w
1
v
2
/w
2
. v
n
/w
n

ub = v + (W-w)(v
i+1
/w
i+1
)

Traveling Salesperson problem
problem statement
Lower bound = s/2 (s two nearest city from each vertex)

P, NP, Np-Complete, Np-hard problems, Deterministic, non-deterministic, polynomial time
problems, tractable, intractable, decidable, undecidable problems, Polynomially-Reducible
problem

Approximation algorithms for Np-hard problems
relative error of approximation for minimizing problem
) (
) ( ) (
) (
*
a
a
a
S f
S f S f
S re

= S* is an exact solution to the problem
accuracy ratio of approximation solutions to minimization problems
) (
) (
) (
*
S f
S f
S r
a
a
=
accuracy ratio of approximation solutions to maximization problems
) (
) (
) (
*
a
a
S f
S f
S r =
performance ratio R
a
is the principal metric indicating the quality of the Approximation
algorithm. R
a
is as close to 1 as possible

Approximation algorithm for Traveling salesperson problem
nearest neighbor algorithm
twice around the tree algorithm



Approximation algorithm for knapsack problem
greedy algorithm for discrete knapsack problem (0/1)
greedy algorithm for continuous knapsack problem (Fractional)