Вы находитесь на странице: 1из 44

CSE-259 Linear Programming and

Analysis of Algorithms
Instructor : Dr. Abu Sayed Md. Mostafizur Rahaman
Time
: Monday 8:30-10:20
Thursday 08:30-10:20

Course Information
Course webpage
http://chyon.wikispaces.com/courses

Course materials

Lecture slides
Lecture books
News and schedule updates

Grading policy

TBA

Course materials are password protected


2

Books

Topics covered in this course


Introduction and review
Algorithm analysis
Linear Programming
Search trees
Priority queues
Sorting algorithms
Hash functions and hash tables
Graph algorithms
Algorithm design

What is an algorithm?
An algorithm is a sequence of unambiguous instructions
for solving a problem, i.e., for obtaining a required
output in a finite amount of time.
problem

algorithm

input

computer

output
5

Basic Issues Related to Algorithms


How to design algorithms
How to express algorithms
Proving correctness
Efficiency

Theoretical analysis

Empirical analysis

Optimality

Algorithm design techniques/strategies


1. Brute force
2. Divide and conquer
3. Decrease and conquer
4. Transform and conquer
5. Space and time tradeoffs
6. Greedy approach
7. Dynamic programming
8. Iterative improvement
9. Backtracking
10. Branch and bound

Mathematical background

Floor and ceiling functions


The floor function maps any real number x onto the greatest
integer less than or equal to x:

3.2
5.2

3 3
6 6

Consider it rounding towards negative infinity

The ceiling function maps x onto the least integer greater than
3.2 4 4
or equal to x:
Necessary because double

5.2
5 5

have a range just under 21024


long can only represent
numbers as large as 263 1

Consider it rounding towards positive infinity


The cmath library implements these as
double floor( double );
double ceil( double );

LHpitals rule
If you are attempting to determine
lim

f n

g n

f n lim g n
but both lim
n
n

lim

f n

g n

lim

, it follows

f 1 n

g 1 n

Repeat as necessary
Note: the kth derivative will always be shown as f

n
10

Logarithms
We will begin with a review of logarithms:
If n = em, we define
m = ln( n )
It is always true that eln(n) = n; however, ln(en) = n
requires that n is real

11

Logarithms
A more interesting observation we will repeatedly
use:
nlog (m) = mlog (n),
b

a consequence of n b logb n :
nlog (m) = (blog (n))log (m)
= blog (n) log (m)
= (blog (m))log (n)
= mlog (n)
b

12

Arithmetic series
Next we will look various series
Each term in an arithmetic series is increased by a
constant value (usually 1) :
n

0 1 2 3 L n k
k 0

n n 1
2

13

Arithmetic series
Proof 1: write out the series twice and add each
column
1 +
2 +
3 + ... + n 2 + n 1 + n
+ n + n 1 + n 2 + ... +
3 +
2 + 1
(n + 1) + (n + 1) + (n + 1) + . . . + (n + 1) + (n + 1) + (n + 1)
= n (n + 1)

Since we added the series twice, we must divide


the result by 2

14

Other polynomial series


We could repeat this process, after all:
n n 1 2n 1
2
k

6
k 0
n

n 2 n 1
3
k

4
k 0
n

however, it is easier to see the pattern:


n

k
k 0

n n 1
2

n2

3
n
n

1
2
n

n
k2

6
3
k 0
n

n 2 n 1
n4
3
k

4
4
k 0
n

15

Geometric series
The next series we will look at is the geometric
series with common ratio r:
n 1
n
1

r
k
r

1 r
k 0
and if |r| < 1 then it is also true that

k
r

k 0

1
1 r

16

Geometric series
1 r
1
1 r

Proof: multiply by
n

Telescoping series:
all but the first and last terms cancel

1 r k 0 r k
n

1 r

k 0

k
r

r
r
k 0
k 0
n

1 r
(1 r r 2 L r n ) (r r 2 L r n r n 1 )

1 r
1 r n 1

1 r

17

Mathematical induction

Defination
Mathematical induction is a form of mathematical proof.
Just because a rule, pattern, or formula seems to work for
several values of n, you cannot simply decide that it is valid
for all values of n without going through a legitimate proof.
The Principle of Mathematical Induction
Let Pn be a statement involving the positive integer n. If
1. P1 is true, and
2. the truth of Pk implies the truth of Pk+1 , for every positive
integer k, then Pn must be true for all integers n.

19

Ex. Use mathematical induction to prove the following formula.


Sn = 1 + 3 + 5 + 7 + . . . + (2n-1) = n2
First, we must show that the formula works for n = 1.
1. For n = 1
S1 = 1 = 12
The second part of mathematical induction has two steps. The first
step is to assume that the formula is valid for some integer k. The
second step is to use this assumption to prove that the formula is
valid for the next integer, k + 1.

2. Assume Sk = 1 + 3 + 5 + 7 + . . . + (2k-1) = k2
is true, show that Sk+1 = (k + 1)2 is true.

Sk+1 = 1 + 3 + 5 + 7 + . . . + (2k 1) + [2(k + 1) 1]


= [1 + 3 + 5 + 7 + . . . +(2k 1)] + (2k + 2 1)
= Sk + (2k + 1)
= k2 + 2k + 1
= (k + 1)2

Ex. Use mathematical induction to prove the following


formula.
Sn = 1 + 2 + 3 + 4 +
2

...

+n =
2

1. Show n = 1 is true.

n n 1 2n 1
6
Sn = 12 =

2. Assume that Sk is true.


Sk =

1 +2 +3 +4 +
2

Show that Sk+1 =

...

+k =
2

k k 1 2k 1
6

k 1 k 2 2k 3
6

1 2 3
6

is true.

Sk+1

= (12 + 22 + 32 + 42 + . . . + k2) + (k + 1)2

k k 12k 1 + (k + 1)2

6
2
k k 1 2k 1 6 k 1

k 1 k 2k 1 6 k 1

k 1 2k

7k 6

Factor out a (k + 1)

k 1 k 2 2k 3

Example 1

Prove that

k
k 0

n n 1
2

is true for n 0

0 0 1
k 0

2
k 0
0

When n = 0:

Assume that thenstatement


isn true for a given n:
1
k n 1 k
We now show:
k 0
k 0

k
k 0

n n 1
2

n n 1
2
2 n 1 n n 1

n 1

2
n 2 n 1 n 1 n 2

2
2

24

Example 2
2
Prove that the sum of
the
first
n
odd
integers
is
n
:
n

2k 1 n

When n = 1:

for n 1

k 1

2k 1 1 1

k 1

Assume that the statement is true for a given n:


n 1

k 1

k 0

2k 1 n

k 1

2k 1 2 n 1 1 2k 1

We now show:

25

Example 2
Prove that the sum of the first n odd integers is n2:
n

2k 1 n
1

for n 1

k 1

2k 1 1 1

When n = 1: k 1
Assume that the statement is true for a given n:
n
We now show:
2k 1 n 2

k 1

n 1

k 1

k 1

2k 1 2 n 1 1 2k 1
2 n 1 1 n 2
2n 2 1 n 2
n 2 2n 1
n 1

26

Example 3
Prove that

k
n 1
2

2
1

for n 0

k 0

20 1 201 1

When n = 0: k 0
Assume that the statement is true for a given n:
We now show:
n 1

2
k 0

n 1

2n 1 1

k 0

2k
k 0

27

Example 3

Prove that

k
n 1
2

2
1

k 0

for n 0

20 1 20 1 1

When n = 0: k 0
Assume that the statement is true for a given n:
We now show:

n 1

2
k 0

n 1

2n 1 1

k 0

2k
k 0

2n 1 2n 1 1
2 2n 1 1
2n 2 1

28

Example 4
Prove that

n
n

2
k 0 k
n

n 0
1 20

When n = 0: k 0 k 0
0

n
n
Assume that the statement is true for a given n: k 2
k 0
n

We now show:

n 1

k
k 0

n 1

n 1

n 1 n 1

k
n

1
k 1

n

29

Example 4
Prove that

n
n

2
k 0 k
n

n 0
1 20

When n = 0: k 0 k 0
0

n
n

2
Assume that the statement is true for a given n:
k 0 k
We now show:

n 1

k
k 0
n 1

n 1

n 1 n 1

k 1 k
n 1
n

n
n
1
1
k 1 k k 1
n

n
n
n

n n n
1
1

k
k

k 1
k 1
1 0 k 1 k
n

n 1 n
n


k 0 k
n

30

Example 4
Prove that

n
n

2
k 0 k
n

n 0
0

1 2
k 0 k
0
0

When n = 0:
Assume that the statement is true for a given n:
We now show:
n 1

k
k 0
n 1

n 1 n n 1 n 1

k 1 k n 1
n n
n
1
1
k
k

1
k 1
n
n
n
n

n n n
1
1

k
k

1
0
k 1
k 1
k


1 k
n
n
2
k 0 k

n
n

2
k 0 k
n

n 1 n
n



k
0 k
n

31

Example 4
Prove that

n
n

2
k 0 k
n

n 0
1 20

When n = 0: k 0 k 0
0

n
2n

Assume that the statement is true for a given n: k 0 k


n

We now show:
n 1

k
k 0
n 1

n
1

n
n


1
k
k

1
k 1
n
n
n
n

n n n
1
1

0
k 1 k
k 1 k 1
k


1 k
n
n
2 2 2n 2n 1
k 0 k
1

n 1 n 1

k
k 1
n 1
n

n 1 n
n



k
0 k
n

32

Example 5
Prove that

1 r n 1
r

1 r
k 0
n

1 r1
r 1
When n = 0:
1 r
k 0
0

Assume that the statement is true for a given n:


We now show:
n 1

r
k 0

n 1

1 r n 1
r

1 r
k 0
n

rk
k 0

33

Example 5
Prove that

1 r n 1
r

1 r
k 0
n

1 r1
r 1
When n = 0:
1 r
k 0
0

1 r n 1
Assume that the statement is true for a given n: r 1 r
k 0
n

We now show:
n 1

n 1

k 0

rk
k 0

1 r n 1
r
1 r
1 r r n 1 1 r n 1

1 r
1 r
1 r r n 1 1 r n 1

1 r
r n 1 r r n 1 1 r n 1 1 r n 2

1 r
1 r
n 1

34

Example 6
Prove by induction that n3 n is divisible by 3 for all integers
This is slightly different
Choose a base case, say n = 0
Next, prove that the truth of the formula for n implies the
truth for n + 1
Also, prove that the truth of the formula for n also implies
the truth for n 1
When n = 0: 03 0 = 0 is divisible by 3
Assume that the statement is true for a given n: n3 3 is
divisible by 3
3
We now show:
n 1 n 1 n3 3n3 3n 1 n 1

n 1

n 1 n3 3n 3 3n 1 n 1
3 n 2 n n3 n

3 n n 2 n3 n

In both cases, the first term is divisible by 3,


the second term is divisible by 3 by assumption;
Consequently, their sum must also be divisible by 3

35

Example 6
Of course, proof-by-induction may not always be the only
approach:
Proving that n3 n is divisible by 3 for all integers
An alternative proof could follow by observing that all integers
may be written as either
3m
3m + 1
3m + 2
and then observing that
n3 n 3m 3m
3

3m 3m 1 3m 1

n3 n 3m 1 3m 1
3

3m 3m 1 3m 2

n3 n 3m 2 3m 2
3

3 m 1 3m 1 3m 2

36

Example 7
Prove that the derivative of xn w.r.t. x is nxn 1 for n
1 using

the chain rule, and

x 1
Proof

When n = 1: the derivative of x is 1


Assume that the derivative of xn w.r.t. x is nxn 1
x n 1 x n x
We now show:

xn x xn x

37

Example 7
Prove that the derivative of xn w.r.t. x is nxn 1 for n
1 using

the chain rule, and

x 1

Proof

When n = 1: the derivative of x is 1


Assume that the derivative of xn w.r.t. x is nxn 1

x x
x


We now show:
n 1

xn x xn x
nx n 1 x x n
nx n x n
n 1 x n

38

Example 8
Prove that for integer values of n 1
When n = 1:
ln n ! n ln(n)
Assume that
ln(1!) ln(1) 0 1 ln(1)
We now show:
ln n ! n ln(n)
ln n 1 ! ln n 1 ln n !

39

Example 8
Prove that ln n ! n ln(n)
for integer values of n
1
ln(1!) ln(1) 0 1ln(1)

When n = 1:
Assume that ln n !
We now show:

n ln(n)

ln n 1 ! ln n 1 ln n !

ln n 1 n ln n

ln n 1 n ln n 1
n 1 ln n 1

40

Example 9
Prove that

2n

1
n
H 2n 1
2
k 1 k
20

1
0
H 20 1 1
2
k 1 k

When n = 0:

Assume that the statement is true for a given n


We now show:

H 2n1

1

k 1 k

2n1

k
k 1

2n

1
n k
k 2 1
2n1

41

Example 9

2n

1
n
Prove that H 2n 1
2
k 1 k
20
1
0
When n = 1:
H 20 1 1
2
k 1 k

Assume that the statement is true for a given n


We now show:
1

k 1 k

2n1

H 2n1

k
k 1
2n

1
n k
k 2 1
2n1

n1

n 2 1
1
2 k 2n 1 k
n1

n 2
1
1 n 1
2 k 2n 1 2
n1

n
1 2
1 n 1 1
2 2 k 2n 1
n 2n
n 1
n 1
1 n 1 1 1
2 2
2 2
2

42

Example 10

k k
When n = 1: k 1

Prove that

k 1
1

k 1

k
Assume that the statement is true for a given n
We now show:
k 1

k 1

k n 1 k
n 1

k 1

k 1

n 1 2 n 1
2

k k
n

k 1

k 1

43

Example 10
Prove that

k 1
1

k
3

k
1 k
2

k 1

k
When n = 1:
k 1
k 1
Assume that the statement is true for a given n
We now show:

k n 1 k
n 1

k 1

k 1

n 1 2 n 1
2

k k
n

k 1

k 1

n n 1
n
3
n 2 2n 1 2 n 1

k 1
2

n 2 2n 1 n3 2n 2 n k 1 k 3
n

n3 3n 2 3n 1 k 1 k 3
n

n 1 k 1 k 3
3

n 1 k 1 k 3
3

k 1 k 3
n 1

44