CSE 326

Asymptotic Analysis
David Kaplan

Dept of Computer Science & Engineering
Autumn 2001
Asymptotic Analysis CSE 326 Autumn 2001 2
Housekeeping
Homework 1 status
Join cse326@cs
Poll:
Slide format(s)?
Office hours today?
Asymptotic Analysis CSE 326 Autumn 2001 3
Analyzing Algorithms
Analyze algorithms to gauge:
Time complexity (running time)
Space complexity (memory use)

Input size is indicated by a number n
sometimes have multiple inputs, e.g. m and n

Running time is a function of n
n, n
2
, n log n, 18 + 3n(log n
2
) + 5n
3
Asymptotic Analysis CSE 326 Autumn 2001 4
RAM Model
RAM (random access machine)
Ideal single-processor machine (serialized
operations)
Standard instruction set (load, add, store, etc.)
All operations take 1 time unit (including, for our
purposes, each C++ statement)
Asymptotic Analysis CSE 326 Autumn 2001 5
Order Notation (aka Big-O)
Big-O
T(n) = O( f(n) )
Exist positive constants c, n
0
such that
T(n) s cf(n) for all n > n
0
Upper bound
Omega
T(n) = O( f(n) )
Exist positive constants c, n
0
such that
T(n) > cf(n) for all n > n
0
Lower bound
Theta
T(n) = ( f(n) )
T(n) = O(f(n)) AND T(n) = O(f(n))
Tight bound
little-o
T(n) = o( f(n) )
T(n) = O(f(n)) AND T(n) != (f(n))
Strict upper
bound
Asymptotic Analysis CSE 326 Autumn 2001 6
Simplifying with Big-O
By definition, Big-O allows us to:
Eliminate low order terms
4n + 5 4n
0.5 n log n - 2n + 7 0.5 n log n
Eliminate constant coefficients
4n n
0.5 n log n n log n
log n
2
= 2 log n log n
log
3
n = (log
3
2) log n log n

But when might constants or low-order terms matter?
Asymptotic Analysis CSE 326 Autumn 2001 7
Big-O Examples
n
2
+ 100 n = O(n
2
)
follows from ( n
2
+ 100 n ) s 2 n
2
for n > 10

n
2
+ 100 n = O(n
2
)
follows from ( n
2
+ 100 n ) > 1 n
2
for n > 0

n
2
+ 100 n = u(n
2
)
by definition

n log n = O(n
2
)
n log n = u(n log n)
n log n = O(n)
Asymptotic Analysis CSE 326 Autumn 2001 8
Big-O Usage
Order notation is not symmetric:
we can say 2n
2
+ 4n = O(n
2
)
but never O(n
2
) = 2n
2
+ 4n
Right-hand side is a crudification of the left

Order expressions on left can produce unusual-
looking, but true, statements:
O(n
2
) = O(n
3
)
O(n
3
) = O(n
2
)
Asymptotic Analysis CSE 326 Autumn 2001 9
Big-O Comparisons
Function A

n
3
+ 2n
2


n
0.1


n + 100n
0.1


5n
5


n
-15
2
n
/100

8
2log n


Function #2

100n
2
+ 1000

log n

2n + 10 log n

n!

1000n
15


3n
7
+ 7n

vs.
Asymptotic Analysis CSE 326 Autumn 2001 10
Race 1
100n
2
+ 1000
vs. n
3
+ 2n
2

Asymptotic Analysis CSE 326 Autumn 2001 11
Race 2
n
0.1
log n
vs.
In this one, crossover point is very late! So, which algorithm is really better???
Asymptotic Analysis CSE 326 Autumn 2001 12
Race C
n + 100n
0.1
2n + 10 log n
vs.
Is the better algorithm asymptotically better???
Asymptotic Analysis CSE 326 Autumn 2001 13
Race 4
5n
5
n!
vs.
Asymptotic Analysis CSE 326 Autumn 2001 14
Race 5
n
-15
2
n
/100 1000n
15

vs.
Asymptotic Analysis CSE 326 Autumn 2001 15
Race VI
8
2log(n)
3n
7
+ 7n
vs.
Asymptotic Analysis CSE 326 Autumn 2001 16
Big-O Winners (i.e. losers)
Function A

n
3
+ 2n
2


n
0.1


n + 100n
0.1


5n
5


n
-15
2
n
/100

8
2log n


Function #2

100n
2
+ 1000

log n

2n + 10 log n

n!

1000n
15


3n
7
+ 7n

vs.
Winner

O(n
2
)

O(log n)

O(n) TIE

O(n
5
)

O(n
15
)

O(n
6
) why???

Asymptotic Analysis CSE 326 Autumn 2001 17
Big-O Common Names
constant: O(1)
logarithmic: O(log n)
linear: O(n)
log-linear: O(n log n)
superlinear: O(n
1+c
) (c is a constant > 0)
quadratic: O(n
2
)
polynomial: O(n
k
) (k is a constant)
exponential: O(c
n
) (c is a constant > 1)
Asymptotic Analysis CSE 326 Autumn 2001 18
Kinds of Analysis
Running time may depend on actual input, not
just length of input

Distinguish
Worst case
Your worst enemy is choosing input
Average case
Assume probability distribution of inputs
Amortized
Average time over many runs
Best case (not too useful)
Asymptotic Analysis CSE 326 Autumn 2001 19
Analyzing Code
C++ operations
Consecutive stmts
Conditionals
Loops
Function calls
Recursive functions
constant time
sum of times
larger branch plus test
sum of iterations
cost of function body
solve recursive equation
Asymptotic Analysis CSE 326 Autumn 2001 20
Nested Loops
2
1
1
1
1 n n
n
i
n
j
n
i
= =

=
=
=
for i = 1 to n do
for j = 1 to n do
sum = sum + 1

Asymptotic Analysis CSE 326 Autumn 2001 21
Dependent Nested Loops
for i = 1 to n do
for j = i to n do
sum = sum + 1

= + = + =

= = = = =
i n i n
n
i
n
i
n
i
n
i j
n
i 1 1 1 1
) 1 ( ) 1 ( 1
2
2
) 1 (
2
) 1 (
) 1 ( n
n n n n
n n ~
+
=
+
+
Asymptotic Analysis CSE 326 Autumn 2001 22
Recursion
A recursive procedure can often be
analyzed by solving a recursive equation
Basic form:
T(n) =
base case: some constant
recursive case: T(subproblems) + T(combine)
Result depends upon
how many subproblems
how much smaller are subproblems
how costly to combine solutions (coefficients)
Asymptotic Analysis CSE 326 Autumn 2001 23
Sum of Queue
SumQueue(Q)
if (Q.length == 0 )
return 0
else
return Q.dequeue() + SumQueue(Q)
One subproblem
Linear reduction in size (decrease by 1)
Combining: constant (cost of 1 add)
T(0) s b
T(n) s c + T(n 1) for n>0
Asymptotic Analysis CSE 326 Autumn 2001 24
Sum of Queue Solution
T(n) s c + c + T(n-2)
s c + c + c + T(n-3)
s kc + T(n-k) for all k
s nc + T(0) for k=n
s cn + b = O(n)
T(0) s b
T(n) s c + T(n 1) for n>0
Equation:
Solution:
Asymptotic Analysis CSE 326 Autumn 2001 25
Binary Search
BinarySearch(A, x)
Search A, a sorted array, for item x
One subproblem, half as large
7 12 30 35 75 83 87 90 97 99
T(1) s b
T(n) s T(n/2) + c for n>1
Equation:
Asymptotic Analysis CSE 326 Autumn 2001 26
Binary Search: Solution
T(n) s T(n/2) + c
s T(n/4) + c + c
s T(n/8) + c + c + c
s T(n/2
k
) + kc
s T(1) + c log n where k = log n
s b + c log n = O(log n)
T(1) s b
T(n) s T(n/2) + c for n>1
Equation:
Solution: