Series

Ravi S. Iyer, Ph.D

ravi@analytics-applied.com

Contents
1 Introduction

2 Expanding functions

2.1 Maclaurin series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.2.1 cos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.2.2 sin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.2.3

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2.2.5 ( + )

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.2.4 ( + )
2.2.6 (1 )

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.2.7 log (1 ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3 Taylors series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.4 Approximating using Taylor and Maclaurin series . . . . . . . . . . . . . 17
2.4.1 Applying Lagranges form of the residual . . . . . . . . . . . . . . 22
A A few important derivations

26

Series

1 Introduction
In this document we take the algebraic view of series and sequences and look at the
algebraic implications of sequences, series and convergence.

2 Expanding functions
Consider nonlinear functions such as log, exp, trigonometric functions, (1) etc. We
will try to express these as polynomials in their variable.

Let () be a function of some variable , , and . Then, we

define
() B

2.1 Maclaurin series

Let us assume that we are given a nonlinear function (), , and .
Let us also assume that () is defined and infinitely differentiable in some interval
(, ) about 0. Then, we could express () as the infinite power series
() = 0 + 1 + 2 2 + + +

Draft

(C) Copyright Dr. Ravi S. Iyer

(1)

Series

Where, the coefficients 0 , 1 , are computed as follows:

0

() | = 0 = (0)

() | = 0 = (0)

(2) (0)
1 (2)
() | = 0 =
2!
2!
(3) (0)
1 (3)
() | = 0 =
3!
3!

() (0)
1 ()
() | = 0 =
!
!

The general expression is of the form

=

() (0)
,
!

= 0, 1,

(2)

Equation 1 can therefore be written as follows:

() =

() (0)

!
=0

(3)

Such a representation of a function is called its corresponding Maclaurin series.

Equation 3 is easy to prove.
Consider ().
It is obvious that by setting to 0, we have
()| = 0 = 0 .
Similarly,
() = 1 + 22 + 33 2 +

Draft

(C) Copyright Dr. Ravi S. Iyer

Series

Therefore,
(0) = 1
Again, from equation 1, we have
(2) () = (2.1) 2 + (3.2.1)3 + (4.3.2.1)4 2 +
= 2! 2 + 3! 3 + 4! 4 2 +

Therefore,
(2) (0) = 2! 2

2 =

(2)
2!

Similarly,
(3) () = (3.2.1) 3 + (4.3.2.1)4 + (5.4.3.2.1)5 2 +
= 3! 3 + 4! 4 + 5! 5 2 +

Therefore,
(3) (0) = 3! .3

3 =

(3)
3!

The other results follow from similar arguments.

Please note: All this is possible only if () is defined and infinitely differentiable in
some interval (, ) about 0, where > 0.

2.2 Examples
2.2.1 cos
Consider the function () = cos , in the interval * . In this interval cos
is defined and continuously differentiable.
* Remember, is in radians.

Draft

(C) Copyright Dr. Ravi S. Iyer

Series

(0) = cos (0) = 1

(1) (0) = sin (0) = 0
(2) (0) = cos (0) = 1
(3) (0) = sin (0) = 0
(4) (0) = cos (0) = 1

and the pattern repeats.

So, cos can be expressed as
cos

1 + 0. + (1).

2 4
+
+ ,
2! 4!

=0

2
3
4
+ 0. + 1. +
2!
3!
4!

(1)

2
(2)!

(4)
(5)

Consider equation 4. As the powers of are always even, it tells us that cos is an
even function, i.e., cos () = cos (), which is indeed a property of cos .

2.2.2 sin
Next, Consider the function () = sin , in the interval . In this interval
sin is defined and continuously differentiable.
Remember, is in radians.

Draft

(C) Copyright Dr. Ravi S. Iyer

Series

Figure 1: Using equation 5 to evaluate cos at 3 . The plot shows the partial sums
for different values of . Note that the partial sums quickly converge (by the 3rd
iteration) to the expected value (0.5).

(0) = sin (0) = 0

(1) (0) = cos (0) = 1
(2) (0) = sin (0) = 0
(3) (0) = cos (0) = 1
(4) (0) = sin (0) = 0

Draft

(C) Copyright Dr. Ravi S. Iyer

Series

Figure 2: Evaluating cos . Note that the partial sums quickly converge (by the 6th
iteration) to the expected value (-1).

and the pattern repeats.

So, sin can be expressed as
sin

2
3
4
+ (1). + 0. +
2!
3!
4!
3 5 7
=
+

+ ,

3! 5! 7!

0 + 1. + 0.

=0

Draft

(1)

2+1
(2 + 1)!

(C) Copyright Dr. Ravi S. Iyer

(6)
(7)

Series

Consider equation 6. As the powers of are always odd, it tells us that sin is an odd
function, i.e., sin () = sin (), and this is indeed a property of sin .

Figure 3: Using equation 7 to evaluate sin at 3 . The plot shows the partial sums
for different values of . Note that the partial sums quickly converge to the expected
value ( 3
= 0.866 ).
2

2.2.3
Next, Consider the function () = . is defined and infinitely differentiable in
the complex plane .
As

= ,
( )
Draft

(C) Copyright Dr. Ravi S. Iyer

Series

(0) = 0 = 1
(1) (0) = 0 = 1
(2) (0) = 0 = 1
(3) (0) = 0 = 1

and this pattern continues.

So, can be expressed as
= 1 + +

2 3
+
+
2! 3!

(8)

!
=0

(9)

Let us digress a bit and evaluate at = , = 1.

From equation 8, we have,
()2 ()3 ()4 ()5
+
+
+
+
2!
3!
4!
5!
2
3 4
5
= 1 + + (1) + () +
+ +
2!
3! 4!
5!
2 4 6
3 5 7
=
1
+

+ +
+

+
(
) (
)
2! 4! 6!
3! 5! 7!

= 1 + +

=0

(1)

2
2+1
+
(1)

(2)!
(2 + 1)!
=0

= cos + sin
Let us rewrite the conclusion of the above derivation (after renaming the parameter
as ), we have
= cos + sin

(10)

Equation 10 is called Eulers formula.

Draft

(C) Copyright Dr. Ravi S. Iyer

Series

Eulers Identity
Evaluating equation 10 at = , we have
= cos + sin = 1 + .(0) = 1
Or,
+ 1 = 0

(11)

Equation 11 is called Eulers identity.

This identity helps us define logarithms for negative numbers.

Let

, > 0.
From equation 11, we have

= 1

log () =

(multiplying both sides by )

log ( )

(Taking log (.) on both sides)

= log () +
Summarizing, given , > 0
log () = log () +

(12)

Figure 4 illustrates this.

2.2.4 ( + )
Let () = ( + ) , where 0, , , 0.

Draft

(C) Copyright Dr. Ravi S. Iyer

10

Series

Figure 4: Plot of log(). It lies on a plane parallel to the Real plane at a distance
along the positive imaginary axis.

Then,
(0)

(1) (0)

(2) (0)

.( 1) 2

() (0)

!

( )!

() (0)

So, the Maclaurin series expansion for ( + ) has only + 1 terms.

From equation 2, we can compute the coefficients, , = 0, , . These are:
() (0)
!
1
!
=

! ( )!

= ( )

Draft

(C) Copyright Dr. Ravi S. Iyer

(13)

11

Series
So, the Maclaurin series for ( + ) is

( + ) =

()

=0

which is the Binomial theorem.

2.2.5 ( + )
Let () = ( + ) , where 0, , , , 0.
Then,
(0)

(1) (0)

(2) (0)

.( 1) 2

() (0)

( 1) ( + 1)

Please Note: () can disappear only if or = 0, both of which have been precluded in the problem statement.
From equation 2, we can compute the coefficients, , = 0, , . These are:
=
=

() (0)
!
1
( 1) ( + 1).
!
(14)

So, the Maclaurin series for ( + ) is

( + ) =

Draft

( 1) ( + 1)

!
=0

(C) Copyright Dr. Ravi S. Iyer

12

Series

The term

(1) (+1)
!

is often written as ( ). So, the

( + ) =

()

=0

2.2.6 (1 )1
Consider (1 )1 , 0.
This can be written as (() + 1)1 , and setting = 1, we can use the results from
section 2.2.5.

(1 )1 =

1
(1)
( )
=0

Let us look at the coefficients, which are nothing but (

=1

= (1)
2!

= (1)(2)
3!

1
, for = 0, 1, .
)

= 1
(1).(2)
=
=1
2
(1).(2).(3)
=
= 1
3!

= (1) !(+1)

Draft

= (1)

(1).(2) ()
= (1)
!

(C) Copyright Dr. Ravi S. Iyer

13

Series

So,

(1 )1 =

.(1) .

=0

(1) .(1) .

Note that there are two (1) terms

=0

=0

= 1 + + 2 + 3 + + +
This is the geometric series with ratio . So, the nth partial sum is
=

1
1

The series converges only for || < 1.

2.2.7 log (1 )
Consider log (1 ). We will assume that > 0.
We will derive the polynomial expansion for log (1 ) in two different ways.
1. Using Maclaurin series expansion

Draft

(C) Copyright Dr. Ravi S. Iyer

14

Series

(0)
(1) (0)

= log (1)

=0

1
= 1
|

= 1

=0

(2) (0) = (1) (1).(1)

(1)2 |

= 1

(3) (0) = (1) (1).(2)

(1)3 |

= 2

(4) (0) = (1) (2).(3)

(1)3 |

= 6

(5) (0) = (1) (6).(4)

(1)3 |

= 24

=0
=0
=0
=0

() (0)

(1)!
= (1) (1)
|

=0

= (1)( 1)!

So, the coefficients of , > 1 are:

= (1)( 1)!

1
1
=
!

So, the expansion for log (1 ), > 0 is

log (1 ) = 0

]
[=1

2 3 4

2
3
4

(15)

This series converges only if || < 1.

2. Using (1 )1
Consider (1 )1 . We have seen in section 2.2.6 that
(1 )1 = 1 + + 2 + 3 +

Draft

(C) Copyright Dr. Ravi S. Iyer

15

Series

Now, note that

1
log ) =
(

>0

Therefore,

1
log (1 ) ) = (1)
,
(

where, (1 ) > 0

Which means,

log (1 ) = (1)

= (1)

0 ( =0 )

= (1)

(
0

=0

= (1)

=
+1
+|
( +1
=0 )
=0

= (1)

+1
+1
=0

2 3 4

2
3
4

which is identical to the expression 15.

2.3 Taylors series

In the definition of the Maclaurin series, we had assumed that the function () is
defined and infinitely differentiable in some interval (, ) ( , > 0). Consider
two scenarios:
1. The function is not defined in the neighborhood of 0, but is defined and continuously differentiable in some some neighborhood of = .
2. We are interested in the behavior of the function in the neighborhood of some
= , 0.

Draft

(C) Copyright Dr. Ravi S. Iyer

16

Series

To solve this problem, we shall resort to changing variables and shifting origins. Let
us consider expressing () as
() 0 + 1 ( ) + 2 ( )2 + + ( ) +

(16)

A function () is real analytic on an open set D in the real line if, for any
0 in D, one can express () as

() =

( 0 ) = 0 + 1 ( 0 ) + 2 ( 0 )2 + 3 ( 0 )3 +

(17)

=0

such that the coefficients 0 , 1 , are real numbers, and the series converges to () for all in some neighborhood of 0 .

Then, as in section 2.1, the coefficients 0 , 1 , are computed as

=

() ()
( )
!

(18)

and the series is given by

() =

() ()
( )
!
=0

(19)

Equation 19 is the Taylor series expansion of the function ().

Setting = 0 in equation 19 gives us the corresponding Maclaurin series.

2.4 Approximating using Taylor and Maclaurin series

It is often desirable to truncate the Taylor (or Maclaurin) series to only a few terms.
For example, in evaluating a function at a point using computers. This amounts to
approximating the function by an nth-degree polynomial (finite n). The error incurred

Draft

(C) Copyright Dr. Ravi S. Iyer

17

Series

in approximating a function by its nth-degree Taylor polynomial is called the remainder or residual and is denoted by the function (). Taylor's theorem can be used
to obtain a bound on the size of the remainder.
Let () be the nth partial sum of the Taylor series expansion of the function ().
This is nothing but the nth-degree Taylor polynomial of () evaluated at = , with
() defined and infinitely differentiable in the neighborhood | | < .
() = () +

(2) ()
(1) ()
() ()
( ) +
( )2 + +
( )
1!
2!
!

Then, the residual () is

() = () ()

(20)

Let us use this opportunity to show that lim () = 0. That is, in the neighborhood | | < , () indeed converges to the value of () as .
To do this, we will make use of the following result (theorem 2).
Theorem 1. Consider an open interval ( , + ). If (+1) () is continuous in (
, + ), then, for all ( , + ),
() =

1
( ) (+1) ()
!

Proof. The proof is by induction over .

P(1). For = 1,
1 () = () 1 () = () () (1) ()( )
For = 1, the integral in the theorem is

( ) (2) ()

Let us evaluate this integral by parts, by setting

= ( ). Therefore, = .
= (2) (). Therefore, = (1) ().
Draft

(C) Copyright Dr. Ravi S. Iyer

18

Series

Therefore, using the relationship

We get,

( ) (2) () = ( ) (1) ()|=

(1) ()()

= 0 ( ) (1) () () + ()
= () () (1) ()( )
= 1 ()
The theorem therefore holds good for = 1.

P(2). Let us suppose that the theorem is true for some = . That is,
() =

1
( ) (+1) ()
!

We will show that if it is true for = , then it is also true for = + 1. That
is,
+1 () =

1
( )+1 (+2) ()
( + 1)!

Let us evaluate this integral by parts, by setting

= ( )+1 . Therefore, = ( + 1)( ) .
= (+2) (). Therefore, = (+1) ().
Therefore, using the relationship

Draft

(C) Copyright Dr. Ravi S. Iyer

19

Series

We get,

1
( )+1 (+2) () =
( + 1)!

=
=
=
=
=

1
( )+1 (+1) ()] |=
=
( + 1)! [

+1
(1)( ) (+1) ()

( + 1)!

1
1
0
( )(+1) (+1) () +
( ) (+1)
( + 1)!
!

( )(+1) (+1) () + ()
( + 1)!
1
() ()
( )(+1) (+1) ()
( + 1)!
1
() () +
( )(+1) (+1) ()
(
)
( + 1)!
() +1 ()

= +1 ()
Hence the proof.

Let us illustrate the use of this result with an example. Let us prove that the Maclaurin series for cos is valid for all , < < .
We are required to prove that irrespective of the value of , the residual value tends
to 0 as the number of terms in the partial sum tends to . This is tantamount to
proving that for all , < < , | ()| 0, as .
By setting = 0 in the expression for () in theorem 2, we get
() =

1
( ) (+1) ()
!
0

As () = cos , (+1) () is either sin or cos , and as | sin | 1 and | cos | 1,

|

Draft

(+1)

() | 1.

(C) Copyright Dr. Ravi S. Iyer

20

Series

We will use the following two identities involving absolute values in our proof. For
,

() |

| () |

and
|

()()
() | | () |
|
|

Now, for > 0,

| ()| =

=
=

1
( ) (+1) () |
! |
0

1
( ) | (+1) () |
!
0

1
( )
!
0

1 +1
! ( + 1)
+1
( + 1)!

Similarly, for < 0,

| ()| =

0
1
( ) (+1) () |
|
!

1
(+1)
(

)
() |
|
| |
!
0

0
1
( )
!

=
=

(as < 0)

1 ()+1
! ( + 1)
()+1
( + 1)!

In either case,
| () |

| |+1
( + 1)!

Now, consider the R.H.S of equation 21. The term

(21)
| |+1
(+1)!

0 as , irrespective

of the value of (see theorem 4). Hence the L.H.S also tends to 0 as , i.e.,
| () | 0, as .
Draft

(C) Copyright Dr. Ravi S. Iyer

21

Series

As the residual value tends to 0, cos is equal to the sum of its Maclaurin series.
Theorem 2 however is not very easy to work with, and an alternative expression of
the residual error is called for.

Theorem 2. Consider an open interval ( , + ). If (+1) () is continuous in (

, + ), then, there exists a point between and , such that
() =

(+1) ()
( )+1
( + 1)!

This expression is called the Lagranges Form of the residual or remainder term.
Proof. The function () = ( ) does not change sign in the interval between and
. Therefore, from theorem 7, there exists a between and such that

( ) (+1) () = (+1) ()

= (+1) ()
= (+1) ()

( )

( )+1 =
| =
1

( )+1
+1

(22)

Therefore, from theorem 2, we have

() =

1
( ) (+1) ()
!

1 (+1) ( )+1

()
!
+1
+1
(

)
= (+1) ()
( + 1)!
=

from equation (22)

(23)

2.4.1 Applying Lagranges form of the residual

Example 1.
Let us revisit the example used earlier (see page 20), that is to prove that the MaclauDraft

(C) Copyright Dr. Ravi S. Iyer

22

Series

rin series for cos represents cos for all .

Using the Lagranges form of the residual at = 0, we have
() = (+1) ()

+1
( + 1)!

0 .

Here () = cos . Therefore, (+1) () is either sin or cos . In either case, | (+1) ()|
1. So,
+1

| ()| = |

(+1)

| |
() |
( + 1)!

+1

| |
( + 1)!

(24)

From theorem 4, the R.H.S of equation 24 tends to 0 as . Therefore, | ( )| 0

as . This shows that as the order of the polynomial approaches , for all
values of , the Maclaurin series for cos converges to the true value cos , because
the remainder term converges absolutely to 0.
3
Let us approximate () =
by a Taylor polynomial of degree 2 at = 8, and see

how accurate this approximation is, when 7 9?

1

() = 3
1 2
(1) () = 3
3
2 5
(2)
() = 3
9
8
10
(3) () =
3
27

(8) = 2
(1) (8) =

1
12

(2) (8) =

1
144

Thus, the second-degree Taylor polynomial is

(1) (8)
(2) (8)
( 8) +
( 8)2
1!
2!
1
1
= 2 + ( 8)
( 8)2
12
288

2 () = (8) +

which is the desired approximation.

Draft

(C) Copyright Dr. Ravi S. Iyer

23

Series

Now, using the Lagrange form of the remainder term, we have,

2 () =
=
=

(3) ()
( 8)3
3!
1 10 83
( 8)3
3! 27
5 83
( 8)3
81

Figure 5: Neighborhood of interest in evaluating the second degree Taylor polynomial

3
of
, 7 9.

Draft

(C) Copyright Dr. Ravi S. Iyer

24

Series

Therefore,
5 8
3
|2 () | = 81 3 | ( 8) |
Here, lies between 8 and , and 7 9 (see figure 5 ).
Now, 7 9 implies that 1 ( 8) 1, or, | 8 | 1.
Also, since > 7,

3 < 7 3 = 0.005577059.

Therefore,
5 8
5
3
|2 () | = 81 3 | ( 8) | < 81 0.005577059 1 = 0.000344263
3
Hence, if 7 9, the error in estimating
, evaluated at 8, using a second order

Taylor polynomial approximation, is less than 0.00035.

Draft

(C) Copyright Dr. Ravi S. Iyer

25

Series

A A few important derivations

Theorem 3. For , if
+1

1
=
log
+ 1
=1
Then is strictly monotone increasing. That is, for all , +1 > .
Proof. For all > 0, ,

+1

1
log
+ 1
=1
+1

<

1
log ( + 2)
+ 1
=1

for all < ( + 2), log < log ( + 2).

= log ( + 2)
< log ( + 2)

(25)

+2

+1 =

1
log
+ 2
=1

+1

1
log + log ( + 2)

+ 2
=1

1
( + 1) + log ( + 2)]
=
+2 [
1
>
( + 1) + ] From (25)
+2 [
=

+1 >

(26)

Hence the proof.

Theorem 4.

| |+1
= 0
( + 1)!
lim

Draft

(C) Copyright Dr. Ravi S. Iyer

26

Series

Proof. This is equivalent to proving that

+1

( + 1) log ||
log , as

=1

(27)

Let
+1

1
log .
=
+ 1
=1
Then the L.H.S of expression 27 can be written as
( + 1) log || ( + 1) = ( + 1) (log || )
Now, is a strictly monotone increasing sequence of (see theorem 3), and log || is
a constant with respect to . So, for a given , there exists 0 such that, for all
> 0 , > log ||. So, for all , > 0 , (log || ) < 0. This means that as ,
the expression ( + 1) (log || ) .
Hence the proof.
Theorem 5 (Extreme Value Theorem). If a function () is continuous on a closed
interval [, ], then () has both a maximum and a minimum on [, ].
Proof. See
http://math.duke.edu/~cbray/Stanford/2002-2003/Math%2041/EVTProof.pdf

Theorem 6. Consider a function () is continuous on a closed interval [, ], and let

() < < () (assuming () < () ), or () > > () (assuming () > () ).
Then there exists [, ] such that () = .
Proof. See
http://en.wikipedia.org/wiki/Intermediate_value_theorem#Theorem

Theorem 7 (Weighted Mean Value Theorem for Integrals). If and are continuous
functions on [, ] and does not change sign in [, ], then there exits a number in

Draft

(C) Copyright Dr. Ravi S. Iyer

27

Series

[, ] such that

()() = ()

()

Proof. Because does not change sign, either 0 or 0. Without loss of generality, let us assume that () 0.
By the Extreme Value Theorem, has an absolute minimum (say, ) and an absolute
maximum (say, ) in the interval [ , ].
()

So, for , since () 0,

() () () ()
Therefore,

()

()()

()

(28)

If () = 0, then these inequalities show that ()() = 0, and so theorem

7 is trivially true.
If If () 0, and as we have assumed that If () 0, If () > 0.
We can therefore divide throughout in equation 28 by (), and get

()()
()

(29)

Then, by the Intermediate Value Theorem, there exists a number in [, ] such that
() =

()()
()

Therefore, as () > 0, there exists a number in [, ] such that

()
Draft

() =

()()

(C) Copyright Dr. Ravi S. Iyer

28

Series

Hence the proof.

Draft

(C) Copyright Dr. Ravi S. Iyer

29