Outline

Contents
1 Product Measures and Fubini’s Theorem 1

2 Kolmogorov’s Extension Theorem 3

1 Product Measures and Fubini’s Theorem

Product measure space

• The usual Euclidean space equipped with the usual Lebesgue measure, (Rn , B(Rn , µ(n) )
is a product of several (R, B, µ).
• Definition:

– the wholespace: Cartesian product.

– the σ-algebra: minimum σ-algebra that contains all measurable rectangles,
which are sets that have form A1 ×A2 ×. . . An , A1 , . . . , An are measurable
sets in each one-dim space.
– the product measure: well defined on measurable rectangles, µ(A1 × A2 ×
. . . An ) = µ1 (A1 )µ2 (A2 ) . . . µn (An ).

The Classical Product Measure Theorem

• Calculus analogy: double integral. Draw a diagram.

• For E ∈ F1 × F2 , let Ex (Ey ) be the cross-section at x (y),

Ex = {y : (x, y) ∈ E} , Ey = {x : (x, y) ∈ E} .

The measure of E can be calculated by iterated integration:

Z Z
µ(E) = µ2 (Ex )dµ1 (x) = µ1 (Ey )dµ2 (y).
Ω1 Ω2

• There is one and only one measure which satisfies µ(A1 ×A2 ) = µ1 (A1 )µ2 (A2 ).
(There is a mathematical terminology for this property. µ(·) is called the tensor
product of µ1 (·) and µ(·), denoted by µ1 ⊗ µ2 ).
Fubini’s Theorem
R
• Motivation: f (x, y) defined on D = A×B. How do we know
that D f (x, y)dxdy
R R
can be computed by iterated integration: A B f (x, y)dy dx?
R
• The answer: D |f (x, y)|dxdy < ∞. (Fubini’s theorem in real analysis)
• The same theorem still holds R if the Lebesgue measure is replaced by a σ-finite
measure. If f is integrable ( Ω |f |dµ is finite), we have
Z Z Z  Z Z 
f dµ = f dµ2 dµ1 = f dµ1 dµ2 .
Ω1 ×Ω2 Ω1 Ω2 Ω2 Ω1

Fubini’s Theorem (II)

• Without the integrability condition, sometimes you may end up with different
answers by using two different iterations. See Example 6.1 in the book (page
468).
• In some sense, Fubini’s theorem is a “sister theorem” of the dominated conver-
gence theorem because
Z Z Z  
f (x, y)dµ(x)dµ(y) = lim fn dµ(y),
Ω1 Ω2 Ω1 n→∞

where fn are rectangle approximations of f from inside out (picture it for the
students) so that fn is dominated by |f |.
R
• The theorem in the book allows another case, f > 0 (even if Ω f = +∞. The
analogy: the monotone convergence theorem, with +∞ being a valid limit.

The L1 space

• Denote all integrable

R functions on measure space (Ω, F , µ) by L1 (Ω) := L1 (Ω, F , µ).
1 1
f ∈ L (Ω) ⇐⇒ Ω |f |dµ < ∞. This L space is a linear space equipped with
a “length”, the L1 norm.

• We can change the order of iterative integrals of L1 functions.

• The dominated convergent theorem says that if f1 , f2 , . . . are a sequence of
L1 (Ω) functions and they are dominated by g ∈ L1 (Ω), then we can change
the order of (almost everywhere) limit and integration.

• To put it in another way, there is no “surprise” in the L1 space. We have to pay

special attention to functions (random variables) that are not in L1 space (do not
have finite expectation).

2
Differentiating under the integral
• We have established conditions under which we can interchange the order of
limit/integration (monotone/dominated convergence theorem) and integration/integration
(Fubini’s theorem).
• Sometimes we also want to interchange the order of differentiation and integra-
tion.
• The key of proof: turn the difference u(y + h) − u(y) into a double integral and
interchange the order of integral by the Fubini’s theorem. See A.9. (page 478).
• An important application, which will be used in the central limit theorem (and
more):
Theorem 1 (9.3). If ϕ(θ) = EeθZ < ∞ for θ ∈ [−, ], then EZ = ϕ0 (0).

2 Kolmogorov’s Extension Theorem

Infinite Product Space
• An n-product space describes a random event with up to n trials. This is usually
good enough for describing an application.
• It would be nice to also define product measure/integral for infinitely many ran-
dom trials. This is because the large sample behavior can be studied in a precise
way only if we allow n → ∞.
• It turns out that there is one and only one way to extend the measures of mem-
ber spaces to the infinite product space, just like the classical product measure
theorem. It was discovered by Kolmogorov.
• Infinite product probability space is the basis of LLN, CLT, etc. We also learn
that there is a division between the theoretic probability and its applications (sta-
tistical applications with finite sample).

The construction
• Let (Ωi , Fi , µi ), i = 1, 2, . . . be an infinite sequence of probability spaces.
Q∞
• the whole space Ω = i=1 Ωi is again the Cartesian product.
• Measurable rectangles are: A1 × A2 × · · · × An × Ωn+1 × Ωn+2 × . . .. I.e., only
finitely many non-trivial terms.
• the σ-algebra, F is generated by measurable rectangles.
• Product probability: a probability on F such that
n
Y
P {ω ∈ Ω : ω1 ∈ A1 , . . . , ωn ∈ An } = Pi (Ai ).
i=1

3
Measure Extension
• This definition is good for rectangles because rectangles are essentially finite
dimensional.
• F of course includes much more than just rectangles because we allow count-
able infinite set operations. If we fix finite dimensions, these set operations lead
to finer and finer measurable sets (a la Carathéodory extension theorem). We
also have another countable infinity: the dimensions, which is not a problem
Q∞ for
finite-dim space. A typical example is an infinite-dim rectangle: A = n=1 Ai ,
Ai ∈ F i .
Q∞
• Heuristically
P∞ speaking, P (A) = n=1 P (Ai ) or equivalently, log (P (A)) =
n=1 log(P (Ai )). This series converges because P is a probability measure
so log(P (Ai )) 6 0 (so the monotone convergence theorem may apply). Other
measures will run into trouble.

Kolmogorov Extension Theorem

• For simplicity, the book only states a very special case, i.e., the indices of the
product space is countable infinite and the member state spaces are R.
• (Page 471) This version of the theorem basically says, if we define the probability
for intervals of each member space, there is one and only one way to define a
countable infinite product measure.
• By the way, X1 , X2 , . . . is called a stochastic process with index set N. And
yes, i.i.d. sequence of random variables are the simplest stochastic process.
• In fact, KET works for uncountable index set. KET is the foundation of stochas-
tic processes (time course analysis, spatial statistics, Brownian motions, stochas-
tic differential equations).

Implications
• What we can describe (measure) in the infinite product space that can not be
described by finite product? Limits.
• We can measure for example these sets:
( n
)
1X
A = ω : lim Xi (ω) = EX ,
n→∞ n
i=1
( n
)
1 X
B = ω : lim √ (Yi (ω) − EY ) ∈ (−1.96σ, 1.96σ) .
n→∞ n i=1

Homework
• Page 470, Exercise 6.3.
• Pg 121, Ash’s book, # 2.