Submission Number: 1

Group Number: 5-C

Group Members:

Name Location (Country) E-Mail Address Non-Contributing

Member (X)
Ravi Shankar Mumbai (India) ravishnkr.sinha@gmail.com
Gaurav Roongta Kolkata (India) roongtagaurav@gmail.com
Hitesh Sachani Nagpur (India) hiteshsachanI@gmail.com
MScFE 620 Discrete Time Stochastic Processes

Group # _5-C_

Submission # 1
I. Kolmogorov and his axioms of probability
In 1933, Kolmogorov put forward his three axioms of probability which helped us develop the
understanding of the modern theory of probability and develop the mathematical theory of
probability. The axioms made us realize that the general theory of probability can be described
technically on the basis of concepts such as measures, measurable spaces, measurable sets and
measurable functions. Let us have a look at what the axioms are1:

(1) Axiom 1: Let {Ω, F, P}be a probability space where Ω represents the set of all possible
outcomes of a random experiment, i.e. the sample space, F represents the σ-algebra of all the
subsets of Ω and P represents a measure on the measurable space (Ω, F) such that P(Ω) = 1.
Kolmogorov called the sample space, “The space of elementary events”. The elements of F
correspond to an event related to the experiment and for any event A∈F, P(A) represents the
probability of the event A and it is a number in the set [0,1]. Mathematically, it can be written as:

P(A) ∈ R, P(A) ≥ 0 ∀ A∈F.

(2) Axiom 2: Let X: Ω→R represent a random variable, ω∈Ω :X(ω)< a represent an event in
the σ−algebra of F ∀ a∈R and Px represent “the law of X” given by:

Px(B) =P(X-1(B))
Where “B” represents a Borel subset on the real line. The random variable induces a probability
in the Borel σ−algebra of the real line. The probability of occurrence of at least one event from
the sample space σ is equal to 1, i.e. P(σ) = 1. This is also called the “unit measure” assumption.

(3) Axiom 3: The third axiom states that for mutually exclusive events “a” and “b” such that for
{a, b} ∈F,

P (a ∪ b) = P(a) +P(b).
This can be extended to any number of events. i.e. for {a, b} ∈F,

P (a1 ∪ a2 ∪ a3…..∪ an) =P(a1) +P(a2) +...+P(an)

These three axioms lead to various theorems which helped formulate the mathematical aspects of
the theory of probability. The theorems that were proven using these axioms are listed below:

(1) The probability of the empty set being 0.

(2) The complement rule :P(A) = 1−P(Ac) Where Ac denotes the complementary set of A.

2
MScFE 620 Discrete Time Stochastic Processes

Group # _5-C_

(3) The rule of monotonicity: If A⊆B, then P(A)≤P(B).

(4) The strong law of large numbers which states that as the number of observations become
very large, the sample means converge to the value of the population mean.

(5) These axioms helped Kolomogorov to prove the 0-1 law which states that any event A in
an asymptotic σ-algebra has the probability of either 0 or 1. We define an asymptotic σ-
algebra as the intersection of the sequence ofσ-algebras generated by the sequence of a
random variable. let’s denote the sequence of sigma-algebras by Fn ∀ n≥1. Mathematically:

F= F1 ∩ F2∩ F3………. ∩ Fn
The work of Kolomogorov acted as the foundation of the calculus of probability theory based
on which other theories such as the Markov process were developed.

II. Markov Chain Process

Definition: Markov Chain is a discrete or continuous time process which describes a possible
sequence of events, their main characteristic being the fact that they are only dependent on their
previous event. Since the past events become irrelevant while determining the future one, it can
be said that such processes are memory-less.

It was developed by Andrey Andreyevich Markov who was a disciple of the famous Russian
mathematician Pafnuty Chebyshev in Petersburg University. He is also famous for the Gauss-
Markov theorem which is widely used in linear regression.
It is so versatile in its application in the real word processes that it’s used from operational
research to exchange rates to baseball to Music3. This list is very long and diverse stating the
omnipresence of this concept is real life.

Mathematically, this can be represented by:

P {Xn+1 = j|X0 = i0, X1 = i1, …., Xn = in} = P {Xn+1 = j|X0 = i0}

Where,
Xn represents a stochastic process (also known as the states of the Markovian process), S is the
sample space (also known as state space) and i,j ∈ S and n ≥ 0. In case of a discrete stochastic
process, S is countable.

3
MScFE 620 Discrete Time Stochastic Processes

Group # _5-C_

In probability’s language, the conditional probability distribution of the future state is dependent
only upon the state in which the system is at present. If this conditional probability is constant
through time, then the Markov chain is said to be time homogeneous. Mathematically,

P (Xn+1 = j| Xn = i) = pij
Similarly, if this conditional probability is dependent not only on the latest state but on latest m
states, then it’s called a Markov chain with order m.

The defining characteristic of a Markov chain is the transition probability. This is nothing but the
conditional probability distribution of the future states given the current state. In case of a
discrete Markov chain, this is represented by a transition matrix whose elements can be
represented by:
pij = P (Xt+1 = j|Xt = i)

Here “i” refers to the current state and “j” refers to the future state.

Similarly, in case of continuous Markov chain, we’ve to solve a first order differential equation
to get the transition probability:

P {Xn+1 = j|X0 = i0, X1 = i1, …., Xn = in} = p(j,in)*Δt

Assuming that the indexing is done with respect to time.
Here,

p(j,in) is the solution of the equation:

dP
(t) = P(t)*Q
dt
Here, P (0) is the identity matrix and Q is the transition rate matrix. Q has the dimensions of the
finite state space S. Interestingly, sum of rows of Q add up to “0”.

4
MScFE 620 Discrete Time Stochastic Processes

Group # _5-C_

III. J. L. Doob and Development of Martingales

The story of Joseph L Doob getting into probability is as interesting as his theories. In the
aftermaths of the Great depression, J.L. Doob was unemployed. By the help of a colleague, he
got the chance to work with Harold Hotelling who was a statistician. Thus, the great depression
was responsible for bringing J.L. Doob to probability theory, who was otherwise working on the
boundary values of analytical functions.
The contributions of Kolomogorov had allowed him to interpret the existing probability
theorems mathematically and his work on the mathematical theory of probability only grew to
Martingale theory in his second book Classical Potential Theory and Its Probabilistic
Counterpart.

Let’s look at a few of his many prominent works:

1. Doob’s stopping theorem: We know that the expectation of a martingale is constant at
every deterministic time instant “t”. Doob’s theorem proved that this notion is still valid
when the time “t” is replaced with a stopping time “τ”. This stopping has to bounded in
nature (τ < ∞). This theorem allows us to find out the characteristic function and “the
Law” of Markovian processes.

2. Doob’s transform of a Brownian motion: The transform is denoted by

h(x) = x
where “h” denotes a harmonic function. This transform modifies the original
characteristics of the original Brownian motion X. Let’s denote the transformed process
as X’. This has the unique property of shifting away from the origin which X cannot do.
This gives interesting results in the multiple dimensions.

3. Doob’s martingale convergence theorems:

These state that any super-martingale must converge if it satisfies any condition which
proves that it is bounded.

The First martingale convergence theorem:

This provides a sufficient condition for a random variable to have a limit as time
approaches infinity. Let us define a few terms,

“t” indicates time and t≥ 0.

Ft = Maximum of {-Ft, 0}

Let ω be a path in the sample space Ω. Then,

F(ω) = limt→∞ Ft(ω)

5
MScFE 620 Discrete Time Stochastic Processes

Group # _5-C_

Given,
Supt>0 E[Ft] < +∞.
Here, the point-wise limit is finite ∀ ω∈Ω.

The second martingale convergence theorem:

The first convergence theorem is pointwise and not in Lp space and thus it’s not uniform.
Chebyshev’s inequality suggests that convergence in L1 space means convergence in
probability and distribution.

A/q this theorem, the following statements are equivalent:

(Ft) ∀ t ≥ 0 is uniformly integrable. And

Nt → N as t→ ∞ P-a.s.
Here, N is an integrable random variable and N ∈ L1(Ω, P, R).

Both these theorems are proved using Doob’s upcrossing inequality. The application of these
theorems includes the convergence property observed for conditional expectations and the
convergence of pth power of any continuous martingale in a Lp probability space ∀ p >1. (given
that the supremum of the expectation of the pth power of the martingale is bounded).

The contributions of Doob can be summarised by the comment of P. A. Mayer about the first
book of Doob, Stochastic Processes (1953): He called it the “Bible of new probability”2. The
contributions further extend from the decomposition of a sub-martingale to Levy’s zero-one law.

The legacy of Doob’s work can be found from the mathematics of probability theory to the risk
neutral pricing of financial assets.

6
MScFE 620 Discrete Time Stochastic Processes

Group # _5-C_

Bibliography & References:

1. Kolmogorov and Probability Theory: David Nualart, (August 2004), Page: 607-619,
https://core.ac.uk/download/pdf/268083255.pdf
2. The Annals of Probability (2009), Vol. 37, No. 5, 1664–1670, DOI: 10.1214/09-AOP480
c Institute of Mathematical Statistics, Biographical Note by M. Yor
3. Sean Meyn; Richard L. Tweedie (2 April 2009). Markov Chains and Stochastic Stability.
Cambridge University Press. p. 3. ISBN 978-0-521-73182-9. Archived from the original on
23 March 2017.