Data Structures & Algorithms

Lecture 14: Asymptotic Analysis (Continued)

Dr. Muhammad Shahzad
muhammad.shehzad@seecs.edu.pk
Department Of Computing (DOC),
School of Electrical Engineering & Computer Science (SEECS),
National University of Sciences & Technology (NUST)

15/11/2016

Recap

Asymptotic analysis

Determines the running time in a particular notation

(Big-O, Omega or Theta notation)

Big-O most commonly used notation

Given two positive-valued functions f and g:
f (n) is O(g(n))
if there exist positive numbers c and N such that
f (n) cg(n) for all n N
f is big-O of g if there is a positive number c such
that f is not larger than cg for sufficiently large ns;
that is, for all ns larger than some number N

M. Shahzad: Data Structures & Algorithms

Lecture 14: Asymptotic Complexity (Continued)

Relationship b/w f and g

The relationship between f and g can be expressed by

stating either that g(n) is an upper bound on the value
of f (n) or that, in the long run, f grows at most as fast
as g
c g(n)

f(n)

cg(n) is an approximation to f(n), bounding from above

M. Shahzad: Data Structures & Algorithms

Lecture 14: Asymptotic Complexity (Continued)

Calculating c and g

Usually infinitely many pairs of cs and Ns that can be

given for the same pair of functions f and g
f (n) = 2n2 + 3n + 1 cg(n) = cn2 = O(n2)
where g(n) = n2, candidate values for c and N are

Different values of c and N for function f (n) = 2n2 + 3n + 1 = O(n2)

calculated according to the definition of big-O
M. Shahzad: Data Structures & Algorithms

Lecture 14: Asymptotic Complexity (Continued)

Calculating c and g

To choose the best c and N, it should be determined

for which N, a certain term in f becomes the largest
and stays the largest
In f(n), the only candidates for the largest term are 2n2
and 3n; these terms can be compared using the
inequality 2n2 > 3n that holds for n > 1.5
Thus, the chosen values are N = 2 and c 3(3/4)

Different values of c and N for function f (n) = 2n2 + 3n + 1 = O(n2)

calculated according to the definition of big-O
M. Shahzad: Data Structures & Algorithms

Lecture 14: Asymptotic Complexity (Continued)

g(n) vs c

N is always a
point where the
functions cg(n)
and f intersect
each other
function g is plotted with different coefficients c
M. Shahzad: Data Structures & Algorithms

Lecture 14: Asymptotic Complexity (Continued)

Best g(n)
The inherent imprecision of the big-O notation goes even
further, because there can be infinitely many functions g for
a given function f
E.g., f (n) = 2n2 + 3n + 1 is big-O not only of n2, but also of
n3, n4, . . . , nk, . . . for any k 2
To avoid this, the smallest function g is chosen, i.e., n2 in
this case
Also, the approximation of f can go further
I.e., f (n) = 2n2 + 3n + 1 can be approximated as f(n) =
2n2 + O(n)
Or the function n2 + 100n + log10n + 1,000 can be
approximated as n2 + 100n + O(log10n)
M. Shahzad: Data Structures & Algorithms

Lecture 14: Asymptotic Complexity (Continued)

Big-O Examples

Prove that running time T(n) = n3 + 20n + 1 is O(n3)

Proof:
By the Big-O definition, T(n) is O(n3)
if T(n) cn3 for some n N
Check the condition: n3 + 20n + 1 cn3
or equivalently 1 + 20/n2 + 1/n3 c
Therefore, the Big-O condition holds for n N = 1 and
c 22 (= 1 + 20 + 1)
Larger values of N result in smaller factors c (e.g., for
N = 10, c 1.201 and so on) but in any case the above
statement is valid
M. Shahzad: Data Structures & Algorithms

Lecture 14: Asymptotic Complexity (Continued)

Big-O Examples

Prove that running time T(n) = n3 + 20n + 1 is not O(n2)

Proof:
By the Big-O definition, T(n) is O(n2)
if T(n) cn2 for some n N
Check the condition: n3 + 20n + 1 cn3
or equivalently n + 20/n + 1/n2 c
Therefore, the Big-O condition cannot hold since the left
side of the latter inequality is growing infinitely, i.e., there
is no such constant factor c
M. Shahzad: Data Structures & Algorithms

Lecture 14: Asymptotic Complexity (Continued)

Big-O Examples

T(n) examples
3n + 4
4n2 + 17n + 5344
6n3+ 3n2 + 5n + 57
15log2(n) + 37
15 x 2n + 4n57 +16
37

M. Shahzad: Data Structures & Algorithms

O(n)
O(n2)
O(n3)

O(log(n)) Logarithmic
O(2n)
Exponential
O(1)
Constant time

Lecture 14: Asymptotic Complexity (Continued)

Linear
Quadratic
Cubic

10

Typical functions applied in Big-O estimates

M. Shahzad: Data Structures & Algorithms

Lecture 14: Asymptotic Complexity (Continued)

11