CS210: Data Structures and Algorithms July-Nov 2013

Notes 1 4 Aug, 2013

1 O(), () and () notation
The order notations, namely, Big-Oh, Omega and Theta are dened as follows. Let f and g be
non-negative and non-decreasing functions of their argument, which is the set of natural numbers
N = {0, 1, 2, . . .}.
Denition 1. 1. A function f(n) is said to be of the order of g(n), also written as f(n)
O(g(n)), or more simply as f(n) = O(g(n)), if there exists a constant c > 0 and a positive
integer n
0
such that f(n) c g(n), for all n n
0
.
2. f(n) is said to be (g(n)), also written as f(n) (g(n)), or more simply as f(n) = (g(n)),
if there exists a constant c > 0 and a positive integer n
0
such that f(n) c g(n), for all
n n
0
.
3. f(n) is said to be (g(n)), also written as f(n) = (g(n)) if there exist positive constants c
1
and c
2
and a positive integer n such that c
1
g(n) f(n) c
2
g(n), for all n n
0
.
Formally, the notations O(g(n)), (g(n)) and (g(n)) denote sets of functions. The denitions are
as follows:
O(g(n)) ={f(n) : there exist a positive constant c and positive integer n
0
such that f(n) c g(n)
for all n > n
0
.}
(g(n)) ={f(n) : there exist a positive constant c and positive integer n
0
such that f(n) c g(n)
for all n > n
0
.}
(g(n)) ={f(n) : there exist positive constants c
1
and c
2
and a positive integer n
0
such that
c
1
g(n) f(n) c
2
g(n) for all n > n
0
.}
Since, O(g(n)) is a set of functions, the correct way of writing would be to say that f(n) O(g(n)).
However, we often use the informal notation f(n) = O(g(n)) to mean that f(n) O(g(n)). The
same simplication is used for the notations (g(n)) and (g(n)). Figure 1 illustrates a picture of
such functions.
O(): f(n) = O(g(n)), or, f(n) is asymptotically upper bounded by a constant times g(n), if f(n)
c g(n) asymptotically. Another way of looking at is is that lim
n
f(n)
g(n)
is at most a constant.
Example 1. Let f(n) = 5n
2
+100n 100, for n 1. Then, f(n) = O(n
2
), since, f(n) 6n
2
, for
n > 100. Also, f(n) = O(n
2
log
2
(n)), for the same reason, that is, f(n) 6n
2
log
2
(n), for n > 100.
And, similarly, f(n) = O(n
3
), f(n) = O(n
k
), for any k > 2, as well as f(n) = O(2
n
).
The () notation means the opposite of O().
1
Figure 1: Illustrations of the O, and notations. For each gure, any value of n larger than n
0
as shown will also work.
2
(): f(n) is said to be (g(n)) if f(n) is asymptotically lower bounded by some constant times
g(n). That is, lim
n
g(n)
f(n)
is at most a constant.
(): f(n) is said to be (g(n)) if f(n) = O(g(n)) and f(n) = (g(n)). In other words, f(n) and
g(n) are asymptotically of the same order, that is, their asymptotic growth is the same, up
to constant factors.
(): f(n) is (g(n)) if there exist positive constants c
1
and c
2
such that f(n) is sandwiched
between c
1
g(n) and c
2
g(n). That is, the asymptotic limit of the function ratios satisfy c
1

lim
n
f(n)
g(n)
c
2
, where, c
1
and c
2
are constants.
Property 1. f(n) = (g(n)) if and only if g(n) = O(f(n)) and vice-versa.
Proof. Suppose f(n) = (g(n)). Then, there exist positive constant c and positive integer n
0
such
that f(n) c g(n), for all n n
0
. In other words, for n n
0
, g(n) c
1
f(n), and hence,
g(n) = O(f(n)), where the role of the constant c is played by c
1
and n n
0
. This proves the
only if part.
Suppose g(n) = O(f(n)). Then there exists a positive constant c

and positive integer n

1
such
that g(n) c

f(n), for n n
1
. Equivalently, f(n) (1/c

)g(n), for n n
1
. This shows that
f(n) = (g(n)), proving the if part. Thus both parts of the lemma are proved.
Property 2. f(n) = (g(n)) if and only if f(n) = O(g(n)) and g(n) = O(f(n)).
Proof. Suppose f(n) = (g(n)). Then there exist positive constants c
1
and c
2
and positive integer
n
0
such that c
1
g(n) f(n) c
2
g(n), for all n n
0
. By the rst part of the inequality, c
1
g(n)
f(n), for n n
0
, or, that, g(n) (1/c)f(n). Hence, g(n) = O(g(n)). By the second inequality,
f(n) c
2
g(n), for n n
0
, we have, f(n) = O(g(n)). This proves the only if part.
Conversely, suppose f(n) = O(g(n)) and g(n) = O(f(n)). From the former we obtain that there
exists a positive constant c
1
and a positive integer n
0
such that for n > n
0
, f(n) c
1
g(n).
Since, g(n) = O(f(n)), there exist c
2
and n
1
such that for n > n
1
, g(n) c
2
f(n), or, that
(1/c
2
)g(n) f(n). Hence, for n > max(n
0
, n
1
), we have,
(1/c
2
)g(n) f(n) c
1
g(n)
or that f(n) = (g(n)). This proves the if part of the statement of the lemma.
Example 2. As an example, consider the function
n
2
2
3n. Intuitively, it appears to be (n
2
). To
see this, we have to show that there exist constants c
1
and c
2
, and a value n
0
such that
c
1
n
2

n
2
2
3n c
2
n
2
holds for n > n
0
. Dividing by n
2
yields
c
1

1
2

3
n
c
2
.
For n 12 = n
0
, we can choose c
1
= 1/4 to satisfy the left-inequality. The right-inequality is
satised by choosing c
2
= 1/2. Hence,
n
2
2
3n = (n
2
). Note that other choices of the constants
c
1
, c
2
and n
0
exist as well (for e.g., n
0
= 10, c
1
= 0.2 and c
2
= 1/2; or, n
0
= 100, c
1
= 0.47, c
2
= 0.5
etc.). However, to show that f(n) = (g(n)), any one choice of the constants suces.
3
Example 3. As another example, we can use the denition to show that 10n (n
2
). Suppose
10n = (n
2
). Then, there exists positive constants c
1
, c
2
and n
0
such that
c
1
n
2
10n c
2
n
2
Dividing by n
2
, we have,
c
1

10
n
c
2
However, as n , 10/n 0, and there cannot be any positive constant c
1
satisfying c
1
10/n
for all n > n
0
, for any n
0
.
Example 4. The above examples suggest that a quadratic function f(n) = an
2
+ bn + c, with
a > 0, is (n
2
). Informally, we note that the asymptotic growth is quadratic (ignoring the lower
order terms and by dropping the constant a, we get (n
2
). This may be justied formally.
Let the upper constant c
2
= a + |b| + |c|, so that c
2
n
2
an
2
+ bn + c for all n 1. Also,
(an
2
+bn+c) = n
2
(a+b/n+c/n
2
). For n n
0
= max(4|b|/a, 2

|c|/

a), we have b/n a/4

and c/n
2
a/4 and therefore,
a + b/n + c/n
2
a a/4 a/4 = a/2 .
Hence, for n max(n
0
, 1),
(a/2)n
2
f(n) (a +|b| +|c|)n
2
that is, f(n) = (n
2
).
Informal summary.
1. f(n) = O(g(n)) means that f(n) is asymptotically upper bounded by some constant times
g(n). f = O(g) is similar to f c g.
2. f(n) = (g(n)) means that f(n) is asymptotically lower bounded by some constant time
g(n). f = (g) is similar to f c g.
3. f(n) = (g(n)) means that f is both upper and lower bounded by some constants times g(n).
f = (g) is similar to f and g are of the same order (something like f = g).
The advantage of thinking of O() as and () as (over the space of non-negative, monotonic
function over integers) is that it naturally makes us think of the transitive properties.
Transitivity:
f(n) = O(g(n)) and g(n) = O(h(n)) imply f(n) = O(h(n)) .
f(n) = (g(n)) and g(n) = (h(n)) imply f(n) = (h(n)) .
f(n) = (g(n)) and g(n) = (h(n)) imply f(n) = (h(n)) .
Example. 1. 2n = O(n) and n = O(n
2
) and hence 2n = O(n
2
).
Reexivity:
f(n) = O(f(n)), f(n) = (f(n)), and f(n) = (f(n)) .
The reexive properties may be proved by noting that for all n, the ratio
f(n)
f(n)
is 1.
4
2 o() and () notation
The asymptotic upper bound given by O() notation may or may not be tight. For example, the
bound 2n +log(n) = O(n) is tight, but 2n = O(n
2
) is not tight. The little-oh notation o() is used
to denote an upper bound that is not asymptotically tight.
o(g(n)) ={f(n) : for any constant c > 0, there exists a constant n
0
> 0 such that
f(n) < c g(n), for all n n
0
}
The point is that no matter what the positive constant c > 0 is, f(n) < c g(n), asymptotically.
That is, lim
n
f(n)
g(n)
< c, for any constant c. In this sense, f is of strictly smaller order than g.
For example, n = o(nlog(n)), n = o(n
1.25
), n
2
= o(2
n
), etc..
The little-omega -notation denotes a lower bound that is not asymptotically tight. Formally,
(g(n)) ={ for any constant c > 0, there exists a constant n
0
> 0 such that
c g(n) < f(n), for all n n
0
}
For example, n
2
/2 = (nlog(n)) and n
2
3n = (n
1.75
).
Informal summary. f(n) = o(g(n)) says that f has strictly lower order than g. f(n) = (g(n))
says that f has strictly higher order than g.
Exercises
1. Show that (a) if f(n) = o(g(n)) then f(n) = O(g(n)), and, (b) if f(n) = (g(n)) then
f(n) = (g(n)).
2. For non-negative and non-decreasing functions f(n) and g(n), show that max(f(n), g(n)) =
(f(n) + g(n)).
3. Give a tight bound for

n
i=1
i
a
, where a > 0 is a constant.
4. Is 2
n+1
= O(2
n
)? Is 2
2n
= O(2
n
)?
5. Show that o(g(n)) (g(n)) is the empty set.
5