BST 401 Probability Theory

Xing Qiu Ha Youn Lee

Department of Biostatistics and Computational Biology

University of Rochester

October 5, 2010

Qiu, Lee BST 401

Outline

1 Product Measures and Fubini’s Theorem

2 Kolmogorov’s Extension Theorem

Qiu, Lee BST 401

Product measure space

The usual Euclidean space equipped with the usual

Lebesgue measure, (Rn , B(Rn , µ(n) ) is a product of
several (R, B, µ).
Definition:
the wholespace: Cartesian product.
the σ-algebra: minimum σ-algebra that contains all
measurable rectangles, which are sets that have form
A1 × A2 × . . . An , A1 , . . . , An are measurable sets in each
one-dim space.
the product measure: well defined on measurable
rectangles, µ(A1 × A2 × . . . An ) = µ1 (A1 )µ2 (A2 ) . . . µn (An ).

Qiu, Lee BST 401

Product measure space

The usual Euclidean space equipped with the usual

Lebesgue measure, (Rn , B(Rn , µ(n) ) is a product of
several (R, B, µ).
Definition:
the wholespace: Cartesian product.
the σ-algebra: minimum σ-algebra that contains all
measurable rectangles, which are sets that have form
A1 × A2 × . . . An , A1 , . . . , An are measurable sets in each
one-dim space.
the product measure: well defined on measurable
rectangles, µ(A1 × A2 × . . . An ) = µ1 (A1 )µ2 (A2 ) . . . µn (An ).

Qiu, Lee BST 401

Product measure space

The usual Euclidean space equipped with the usual

Lebesgue measure, (Rn , B(Rn , µ(n) ) is a product of
several (R, B, µ).
Definition:
the wholespace: Cartesian product.
the σ-algebra: minimum σ-algebra that contains all
measurable rectangles, which are sets that have form
A1 × A2 × . . . An , A1 , . . . , An are measurable sets in each
one-dim space.
the product measure: well defined on measurable
rectangles, µ(A1 × A2 × . . . An ) = µ1 (A1 )µ2 (A2 ) . . . µn (An ).

Qiu, Lee BST 401

Product measure space

The usual Euclidean space equipped with the usual

Lebesgue measure, (Rn , B(Rn , µ(n) ) is a product of
several (R, B, µ).
Definition:
the wholespace: Cartesian product.
the σ-algebra: minimum σ-algebra that contains all
measurable rectangles, which are sets that have form
A1 × A2 × . . . An , A1 , . . . , An are measurable sets in each
one-dim space.
the product measure: well defined on measurable
rectangles, µ(A1 × A2 × . . . An ) = µ1 (A1 )µ2 (A2 ) . . . µn (An ).

Qiu, Lee BST 401

Product measure space

The usual Euclidean space equipped with the usual

Lebesgue measure, (Rn , B(Rn , µ(n) ) is a product of
several (R, B, µ).
Definition:
the wholespace: Cartesian product.
the σ-algebra: minimum σ-algebra that contains all
measurable rectangles, which are sets that have form
A1 × A2 × . . . An , A1 , . . . , An are measurable sets in each
one-dim space.
the product measure: well defined on measurable
rectangles, µ(A1 × A2 × . . . An ) = µ1 (A1 )µ2 (A2 ) . . . µn (An ).

Qiu, Lee BST 401

The Classical Product Measure Theorem

Calculus analogy: double integral. Draw a diagram.

For E ∈ F1 × F2 , let Ex (Ey ) be the cross-section at x (y ),

Ex = {y : (x, y ) ∈ E} , Ey = {x : (x, y ) ∈ E} .

The measure of E can be calculated by iterated integration:

Z Z
µ(E) = µ2 (Ex )dµ1 (x) = µ1 (Ey )dµ2 (y ).
Ω1 Ω2

There is one and only one measure which satisfies

µ(A1 × A2 ) = µ1 (A1 )µ2 (A2 ). (There is a mathematical
terminology for this property. µ(·) is called the tensor
product of µ1 (·) and µ(·), denoted by µ1 ⊗ µ2 ).

Qiu, Lee BST 401

The Classical Product Measure Theorem

Calculus analogy: double integral. Draw a diagram.

For E ∈ F1 × F2 , let Ex (Ey ) be the cross-section at x (y ),

Ex = {y : (x, y ) ∈ E} , Ey = {x : (x, y ) ∈ E} .

The measure of E can be calculated by iterated integration:

Z Z
µ(E) = µ2 (Ex )dµ1 (x) = µ1 (Ey )dµ2 (y ).
Ω1 Ω2

There is one and only one measure which satisfies

µ(A1 × A2 ) = µ1 (A1 )µ2 (A2 ). (There is a mathematical
terminology for this property. µ(·) is called the tensor
product of µ1 (·) and µ(·), denoted by µ1 ⊗ µ2 ).

Qiu, Lee BST 401

The Classical Product Measure Theorem

Calculus analogy: double integral. Draw a diagram.

For E ∈ F1 × F2 , let Ex (Ey ) be the cross-section at x (y ),

Ex = {y : (x, y ) ∈ E} , Ey = {x : (x, y ) ∈ E} .

The measure of E can be calculated by iterated integration:

Z Z
µ(E) = µ2 (Ex )dµ1 (x) = µ1 (Ey )dµ2 (y ).
Ω1 Ω2

There is one and only one measure which satisfies

µ(A1 × A2 ) = µ1 (A1 )µ2 (A2 ). (There is a mathematical
terminology for this property. µ(·) is called the tensor
product of µ1 (·) and µ(·), denoted by µ1 ⊗ µ2 ).

Qiu, Lee BST 401

Fubini’s Theorem

Motivation:
R f (x, y ) defined on D = A × B. How do we know
that D f (x, yR)dxdy
R can be computed

by iterated
integration: A B f (x, y )dy dx?
R
The answer: D |f (x, y )|dxdy < ∞. (Fubini’s theorem in
real analysis)
The same theorem still holds if the Lebesgue measureR is
replaced by a σ-finite measure. If f is integrable ( Ω |f |dµ is
finite), we have
Z Z Z  Z Z 
f dµ = f dµ2 dµ1 = f dµ1 dµ2 .
Ω1 ×Ω2 Ω1 Ω2 Ω2 Ω1

Qiu, Lee BST 401

Fubini’s Theorem

Motivation:
R f (x, y ) defined on D = A × B. How do we know
that D f (x, yR)dxdy
R can be computed

by iterated
integration: A B f (x, y )dy dx?
R
The answer: D |f (x, y )|dxdy < ∞. (Fubini’s theorem in
real analysis)
The same theorem still holds if the Lebesgue measureR is
replaced by a σ-finite measure. If f is integrable ( Ω |f |dµ is
finite), we have
Z Z Z  Z Z 
f dµ = f dµ2 dµ1 = f dµ1 dµ2 .
Ω1 ×Ω2 Ω1 Ω2 Ω2 Ω1

Qiu, Lee BST 401

Fubini’s Theorem

Motivation:
R f (x, y ) defined on D = A × B. How do we know
that D f (x, yR)dxdy
R can be computed

by iterated
integration: A B f (x, y )dy dx?
R
The answer: D |f (x, y )|dxdy < ∞. (Fubini’s theorem in
real analysis)
The same theorem still holds if the Lebesgue measureR is
replaced by a σ-finite measure. If f is integrable ( Ω |f |dµ is
finite), we have
Z Z Z  Z Z 
f dµ = f dµ2 dµ1 = f dµ1 dµ2 .
Ω1 ×Ω2 Ω1 Ω2 Ω2 Ω1

Qiu, Lee BST 401

Fubini’s Theorem (II)

Without the integrability condition, sometimes you may end

up with different answers by using two different iterations.
See Example 6.1 in the book (page 468).
In some sense, Fubini’s theorem is a “sister theorem” of
the dominated convergence theorem because
Z Z Z  
f (x, y )dµ(x)dµ(y ) = lim fn dµ(y ),
Ω1 Ω2 Ω1 n→∞

where fn are rectangle approximations of f from inside out

(picture it for the students) so that fn is dominated by |f |.
The
R theorem in the book allows another case, f > 0 (even
if Ω f = +∞. The analogy: the monotone convergence
theorem, with +∞ being a valid limit.

Qiu, Lee BST 401

Fubini’s Theorem (II)

Without the integrability condition, sometimes you may end

up with different answers by using two different iterations.
See Example 6.1 in the book (page 468).
In some sense, Fubini’s theorem is a “sister theorem” of
the dominated convergence theorem because
Z Z Z  
f (x, y )dµ(x)dµ(y ) = lim fn dµ(y ),
Ω1 Ω2 Ω1 n→∞

where fn are rectangle approximations of f from inside out

(picture it for the students) so that fn is dominated by |f |.
The
R theorem in the book allows another case, f > 0 (even
if Ω f = +∞. The analogy: the monotone convergence
theorem, with +∞ being a valid limit.

Qiu, Lee BST 401

Fubini’s Theorem (II)

Without the integrability condition, sometimes you may end

up with different answers by using two different iterations.
See Example 6.1 in the book (page 468).
In some sense, Fubini’s theorem is a “sister theorem” of
the dominated convergence theorem because
Z Z Z  
f (x, y )dµ(x)dµ(y ) = lim fn dµ(y ),
Ω1 Ω2 Ω1 n→∞

where fn are rectangle approximations of f from inside out

(picture it for the students) so that fn is dominated by |f |.
The
R theorem in the book allows another case, f > 0 (even
if Ω f = +∞. The analogy: the monotone convergence
theorem, with +∞ being a valid limit.

Qiu, Lee BST 401

The L1 space

Denote all integrable functions on measure space

(Ω, F , µ) by L1R(Ω) := L1 (Ω, F , µ).
f ∈ L1 (Ω) ⇐⇒ Ω |f |dµ < ∞. This L1 space is a linear
space equipped with a “length”, the L1 norm.
We can change the order of iterative integrals of L1
functions.
The dominated convergent theorem says that if f1 , f2 , . . .
are a sequence of L1 (Ω) functions and they are dominated
by g ∈ L1 (Ω), then we can change the order of (almost
everywhere) limit and integration.
To put it in another way, there is no “surprise” in the L1
space. We have to pay special attention to functions
(random variables) that are not in L1 space (do not have
finite expectation).
Qiu, Lee BST 401
The L1 space

Denote all integrable functions on measure space

(Ω, F , µ) by L1R(Ω) := L1 (Ω, F , µ).
f ∈ L1 (Ω) ⇐⇒ Ω |f |dµ < ∞. This L1 space is a linear
space equipped with a “length”, the L1 norm.
We can change the order of iterative integrals of L1
functions.
The dominated convergent theorem says that if f1 , f2 , . . .
are a sequence of L1 (Ω) functions and they are dominated
by g ∈ L1 (Ω), then we can change the order of (almost
everywhere) limit and integration.
To put it in another way, there is no “surprise” in the L1
space. We have to pay special attention to functions
(random variables) that are not in L1 space (do not have
finite expectation).
Qiu, Lee BST 401
The L1 space

Denote all integrable functions on measure space

(Ω, F , µ) by L1R(Ω) := L1 (Ω, F , µ).
f ∈ L1 (Ω) ⇐⇒ Ω |f |dµ < ∞. This L1 space is a linear
space equipped with a “length”, the L1 norm.
We can change the order of iterative integrals of L1
functions.
The dominated convergent theorem says that if f1 , f2 , . . .
are a sequence of L1 (Ω) functions and they are dominated
by g ∈ L1 (Ω), then we can change the order of (almost
everywhere) limit and integration.
To put it in another way, there is no “surprise” in the L1
space. We have to pay special attention to functions
(random variables) that are not in L1 space (do not have
finite expectation).
Qiu, Lee BST 401
The L1 space

Denote all integrable functions on measure space

(Ω, F , µ) by L1R(Ω) := L1 (Ω, F , µ).
f ∈ L1 (Ω) ⇐⇒ Ω |f |dµ < ∞. This L1 space is a linear
space equipped with a “length”, the L1 norm.
We can change the order of iterative integrals of L1
functions.
The dominated convergent theorem says that if f1 , f2 , . . .
are a sequence of L1 (Ω) functions and they are dominated
by g ∈ L1 (Ω), then we can change the order of (almost
everywhere) limit and integration.
To put it in another way, there is no “surprise” in the L1
space. We have to pay special attention to functions
(random variables) that are not in L1 space (do not have
finite expectation).
Qiu, Lee BST 401
Differentiating under the integral

We have established conditions under which we can

interchange the order of limit/integration
(monotone/dominated convergence theorem) and
integration/integration (Fubini’s theorem).
Sometimes we also want to interchange the order of
differentiation and integration.
The key of proof: turn the difference u(y + h) − u(y ) into a
double integral and interchange the order of integral by the
Fubini’s theorem. See A.9. (page 478).
An important application, which will be used in the central
limit theorem (and more):
Theorem (9.3)
If ϕ(θ) = EeθZ < ∞ for θ ∈ [−, ], then EZ = ϕ0 (0).

Qiu, Lee BST 401

Differentiating under the integral

We have established conditions under which we can

interchange the order of limit/integration
(monotone/dominated convergence theorem) and
integration/integration (Fubini’s theorem).
Sometimes we also want to interchange the order of
differentiation and integration.
The key of proof: turn the difference u(y + h) − u(y ) into a
double integral and interchange the order of integral by the
Fubini’s theorem. See A.9. (page 478).
An important application, which will be used in the central
limit theorem (and more):
Theorem (9.3)
If ϕ(θ) = EeθZ < ∞ for θ ∈ [−, ], then EZ = ϕ0 (0).

Qiu, Lee BST 401

Differentiating under the integral

We have established conditions under which we can

interchange the order of limit/integration
(monotone/dominated convergence theorem) and
integration/integration (Fubini’s theorem).
Sometimes we also want to interchange the order of
differentiation and integration.
The key of proof: turn the difference u(y + h) − u(y ) into a
double integral and interchange the order of integral by the
Fubini’s theorem. See A.9. (page 478).
An important application, which will be used in the central
limit theorem (and more):
Theorem (9.3)
If ϕ(θ) = EeθZ < ∞ for θ ∈ [−, ], then EZ = ϕ0 (0).

Qiu, Lee BST 401

Differentiating under the integral

We have established conditions under which we can

interchange the order of limit/integration
(monotone/dominated convergence theorem) and
integration/integration (Fubini’s theorem).
Sometimes we also want to interchange the order of
differentiation and integration.
The key of proof: turn the difference u(y + h) − u(y ) into a
double integral and interchange the order of integral by the
Fubini’s theorem. See A.9. (page 478).
An important application, which will be used in the central
limit theorem (and more):
Theorem (9.3)
If ϕ(θ) = EeθZ < ∞ for θ ∈ [−, ], then EZ = ϕ0 (0).

Qiu, Lee BST 401

Differentiating under the integral

We have established conditions under which we can

interchange the order of limit/integration
(monotone/dominated convergence theorem) and
integration/integration (Fubini’s theorem).
Sometimes we also want to interchange the order of
differentiation and integration.
The key of proof: turn the difference u(y + h) − u(y ) into a
double integral and interchange the order of integral by the
Fubini’s theorem. See A.9. (page 478).
An important application, which will be used in the central
limit theorem (and more):
Theorem (9.3)
If ϕ(θ) = EeθZ < ∞ for θ ∈ [−, ], then EZ = ϕ0 (0).

Qiu, Lee BST 401

Infinite Product Space
An n-product space describes a random event with up to n
trials. This is usually good enough for describing an
application.
It would be nice to also define product measure/integral for
infinitely many random trials. This is because the large
sample behavior can be studied in a precise way only if we
allow n → ∞.
It turns out that there is one and only one way to extend
the measures of member spaces to the infinite product
space, just like the classical product measure theorem. It
was discovered by Kolmogorov.
Infinite product probability space is the basis of LLN, CLT,
etc. We also learn that there is a division between the
theoretic probability and its applications (statistical
applications with finite sample).
Qiu, Lee BST 401
Infinite Product Space
An n-product space describes a random event with up to n
trials. This is usually good enough for describing an
application.
It would be nice to also define product measure/integral for
infinitely many random trials. This is because the large
sample behavior can be studied in a precise way only if we
allow n → ∞.
It turns out that there is one and only one way to extend
the measures of member spaces to the infinite product
space, just like the classical product measure theorem. It
was discovered by Kolmogorov.
Infinite product probability space is the basis of LLN, CLT,
etc. We also learn that there is a division between the
theoretic probability and its applications (statistical
applications with finite sample).
Qiu, Lee BST 401
Infinite Product Space
An n-product space describes a random event with up to n
trials. This is usually good enough for describing an
application.
It would be nice to also define product measure/integral for
infinitely many random trials. This is because the large
sample behavior can be studied in a precise way only if we
allow n → ∞.
It turns out that there is one and only one way to extend
the measures of member spaces to the infinite product
space, just like the classical product measure theorem. It
was discovered by Kolmogorov.
Infinite product probability space is the basis of LLN, CLT,
etc. We also learn that there is a division between the
theoretic probability and its applications (statistical
applications with finite sample).
Qiu, Lee BST 401
Infinite Product Space
An n-product space describes a random event with up to n
trials. This is usually good enough for describing an
application.
It would be nice to also define product measure/integral for
infinitely many random trials. This is because the large
sample behavior can be studied in a precise way only if we
allow n → ∞.
It turns out that there is one and only one way to extend
the measures of member spaces to the infinite product
space, just like the classical product measure theorem. It
was discovered by Kolmogorov.
Infinite product probability space is the basis of LLN, CLT,
etc. We also learn that there is a division between the
theoretic probability and its applications (statistical
applications with finite sample).
Qiu, Lee BST 401
The construction

Let (Ωi , Fi , µi ), i = 1, 2, . . . be an infinite sequence of

probability spaces.
the whole space Ω = ∞
Q
i=1 Ωi is again the Cartesian
product.
Measurable rectangles are:
A1 × A2 × · · · × An × Ωn+1 × Ωn+2 × . . .. I.e., only finitely
many non-trivial terms.
the σ-algebra, F is generated by measurable rectangles.
Product probability: a probability on F such that
n
Y
P {ω ∈ Ω : ω1 ∈ A1 , . . . , ωn ∈ An } = Pi (Ai ).
i=1

Qiu, Lee BST 401

The construction

Let (Ωi , Fi , µi ), i = 1, 2, . . . be an infinite sequence of

probability spaces.
the whole space Ω = ∞
Q
i=1 Ωi is again the Cartesian
product.
Measurable rectangles are:
A1 × A2 × · · · × An × Ωn+1 × Ωn+2 × . . .. I.e., only finitely
many non-trivial terms.
the σ-algebra, F is generated by measurable rectangles.
Product probability: a probability on F such that
n
Y
P {ω ∈ Ω : ω1 ∈ A1 , . . . , ωn ∈ An } = Pi (Ai ).
i=1

Qiu, Lee BST 401

The construction

Let (Ωi , Fi , µi ), i = 1, 2, . . . be an infinite sequence of

probability spaces.
the whole space Ω = ∞
Q
i=1 Ωi is again the Cartesian
product.
Measurable rectangles are:
A1 × A2 × · · · × An × Ωn+1 × Ωn+2 × . . .. I.e., only finitely
many non-trivial terms.
the σ-algebra, F is generated by measurable rectangles.
Product probability: a probability on F such that
n
Y
P {ω ∈ Ω : ω1 ∈ A1 , . . . , ωn ∈ An } = Pi (Ai ).
i=1

Qiu, Lee BST 401

The construction

Let (Ωi , Fi , µi ), i = 1, 2, . . . be an infinite sequence of

probability spaces.
the whole space Ω = ∞
Q
i=1 Ωi is again the Cartesian
product.
Measurable rectangles are:
A1 × A2 × · · · × An × Ωn+1 × Ωn+2 × . . .. I.e., only finitely
many non-trivial terms.
the σ-algebra, F is generated by measurable rectangles.
Product probability: a probability on F such that
n
Y
P {ω ∈ Ω : ω1 ∈ A1 , . . . , ωn ∈ An } = Pi (Ai ).
i=1

Qiu, Lee BST 401

The construction

Let (Ωi , Fi , µi ), i = 1, 2, . . . be an infinite sequence of

probability spaces.
the whole space Ω = ∞
Q
i=1 Ωi is again the Cartesian
product.
Measurable rectangles are:
A1 × A2 × · · · × An × Ωn+1 × Ωn+2 × . . .. I.e., only finitely
many non-trivial terms.
the σ-algebra, F is generated by measurable rectangles.
Product probability: a probability on F such that
n
Y
P {ω ∈ Ω : ω1 ∈ A1 , . . . , ωn ∈ An } = Pi (Ai ).
i=1

Qiu, Lee BST 401

Measure Extension
This definition is good for rectangles because rectangles
are essentially finite dimensional.
F of course includes much more than just rectangles
because we allow countable infinite set operations. If we
fix finite dimensions, these set operations lead to finer and
finer measurable sets (a la Carathéodory extension
theorem). We also have another countable infinity: the
dimensions, which is not a problem for finite-dim Q space. A
typical example is an infinite-dim rectangle: A = ∞ n=1 Ai ,
Ai ∈ Fi .
P (A) = ∞
Q
Heuristically speaking,
P∞ n=1 P(Ai ) or equivalently,
log (P (A)) = n=1 log(P(Ai )). This series converges
because P is a probability measure so log(P(Ai )) 6 0 (so
the monotone convergence theorem may apply). Other
measures will run into trouble.
Qiu, Lee BST 401
Measure Extension
This definition is good for rectangles because rectangles
are essentially finite dimensional.
F of course includes much more than just rectangles
because we allow countable infinite set operations. If we
fix finite dimensions, these set operations lead to finer and
finer measurable sets (a la Carathéodory extension
theorem). We also have another countable infinity: the
dimensions, which is not a problem for finite-dim Q space. A
typical example is an infinite-dim rectangle: A = ∞ n=1 Ai ,
Ai ∈ Fi .
P (A) = ∞
Q
Heuristically speaking,
P∞ n=1 P(Ai ) or equivalently,
log (P (A)) = n=1 log(P(Ai )). This series converges
because P is a probability measure so log(P(Ai )) 6 0 (so
the monotone convergence theorem may apply). Other
measures will run into trouble.
Qiu, Lee BST 401
Measure Extension
This definition is good for rectangles because rectangles
are essentially finite dimensional.
F of course includes much more than just rectangles
because we allow countable infinite set operations. If we
fix finite dimensions, these set operations lead to finer and
finer measurable sets (a la Carathéodory extension
theorem). We also have another countable infinity: the
dimensions, which is not a problem for finite-dim Q space. A
typical example is an infinite-dim rectangle: A = ∞ n=1 Ai ,
Ai ∈ Fi .
P (A) = ∞
Q
Heuristically speaking,
P∞ n=1 P(Ai ) or equivalently,
log (P (A)) = n=1 log(P(Ai )). This series converges
because P is a probability measure so log(P(Ai )) 6 0 (so
the monotone convergence theorem may apply). Other
measures will run into trouble.
Qiu, Lee BST 401
Kolmogorov Extension Theorem

For simplicity, the book only states a very special case, i.e.,
the indices of the product space is countable infinite and
the member state spaces are R.
(Page 471) This version of the theorem basically says, if
we define the probability for intervals of each member
space, there is one and only one way to define a countable
infinite product measure.
By the way, X1 , X2 , . . . is called a stochastic process with
index set N. And yes, i.i.d. sequence of random variables
are the simplest stochastic process.
In fact, KET works for uncountable index set. KET is the
foundation of stochastic processes (time course analysis,
spatial statistics, Brownian motions, stochastic differential
equations).
Qiu, Lee BST 401
Kolmogorov Extension Theorem

For simplicity, the book only states a very special case, i.e.,
the indices of the product space is countable infinite and
the member state spaces are R.
(Page 471) This version of the theorem basically says, if
we define the probability for intervals of each member
space, there is one and only one way to define a countable
infinite product measure.
By the way, X1 , X2 , . . . is called a stochastic process with
index set N. And yes, i.i.d. sequence of random variables
are the simplest stochastic process.
In fact, KET works for uncountable index set. KET is the
foundation of stochastic processes (time course analysis,
spatial statistics, Brownian motions, stochastic differential
equations).
Qiu, Lee BST 401
Kolmogorov Extension Theorem

For simplicity, the book only states a very special case, i.e.,
the indices of the product space is countable infinite and
the member state spaces are R.
(Page 471) This version of the theorem basically says, if
we define the probability for intervals of each member
space, there is one and only one way to define a countable
infinite product measure.
By the way, X1 , X2 , . . . is called a stochastic process with
index set N. And yes, i.i.d. sequence of random variables
are the simplest stochastic process.
In fact, KET works for uncountable index set. KET is the
foundation of stochastic processes (time course analysis,
spatial statistics, Brownian motions, stochastic differential
equations).
Qiu, Lee BST 401
Kolmogorov Extension Theorem

For simplicity, the book only states a very special case, i.e.,
the indices of the product space is countable infinite and
the member state spaces are R.
(Page 471) This version of the theorem basically says, if
we define the probability for intervals of each member
space, there is one and only one way to define a countable
infinite product measure.
By the way, X1 , X2 , . . . is called a stochastic process with
index set N. And yes, i.i.d. sequence of random variables
are the simplest stochastic process.
In fact, KET works for uncountable index set. KET is the
foundation of stochastic processes (time course analysis,
spatial statistics, Brownian motions, stochastic differential
equations).
Qiu, Lee BST 401
Implications

What we can describe (measure) in the infinite product

space that can not be described by finite product? Limits.
We can measure for example these sets:
n
( )
1X
A = ω : lim Xi (ω) = EX ,
n→∞ n
i=1
n
( )
1 X
B = ω : lim √ (Yi (ω) − EY ) ∈ (−1.96σ, 1.96σ) .
n→∞ n i=1

Qiu, Lee BST 401

Implications

What we can describe (measure) in the infinite product

space that can not be described by finite product? Limits.
We can measure for example these sets:
n
( )
1X
A = ω : lim Xi (ω) = EX ,
n→∞ n
i=1
n
( )
1 X
B = ω : lim √ (Yi (ω) − EY ) ∈ (−1.96σ, 1.96σ) .
n→∞ n i=1

Qiu, Lee BST 401

Homework

Page 470, Exercise 6.3.

Pg 121, Ash’s book, # 2.

Qiu, Lee BST 401

Homework

Page 470, Exercise 6.3.

Pg 121, Ash’s book, # 2.

Qiu, Lee BST 401