BST 401 Probability Theory

Xing Qiu Ha Youn Lee

Department of Biostatistics and Computational Biology

University of Rochester

September 9, 2010

Qiu, Lee BST 401

Outline

1 σ-algebras

Qiu, Lee BST 401

Motivation I

In this lecture we introduce the σ-algebra (or σ-algebra),

which is a fundamentally useful tool in modern probability
and statistics theory.
We will define notions such as probability, probability
space and random variables formally later. Some informal
knowledge about probability is useful for this lecture,
though.
At the intuitive level, a probability is an assessment of how
“likely” a particular set of outcomes might be. In other
words, a probability is a set function: it takes a set as input,
and then churns out a real number as its output.

Qiu, Lee BST 401

Motivation I

In this lecture we introduce the σ-algebra (or σ-algebra),

which is a fundamentally useful tool in modern probability
and statistics theory.
We will define notions such as probability, probability
space and random variables formally later. Some informal
knowledge about probability is useful for this lecture,
though.
At the intuitive level, a probability is an assessment of how
“likely” a particular set of outcomes might be. In other
words, a probability is a set function: it takes a set as input,
and then churns out a real number as its output.

Qiu, Lee BST 401

Motivation I

In this lecture we introduce the σ-algebra (or σ-algebra),

which is a fundamentally useful tool in modern probability
and statistics theory.
We will define notions such as probability, probability
space and random variables formally later. Some informal
knowledge about probability is useful for this lecture,
though.
At the intuitive level, a probability is an assessment of how
“likely” a particular set of outcomes might be. In other
words, a probability is a set function: it takes a set as input,
and then churns out a real number as its output.

Qiu, Lee BST 401

Motivation II

Again by our intuition, a probability P should have the

following properties:
1 If A, B are two well defined sets of outcomes, we should be
able to “talk about” the probability of A ∩ B, A ∪ B, and Ac or
Bc .
2 Other important properties such as non-negativity and
additivity which will be introduced later.
In other words, the domain of P is not just any collection of
sets, it must have certain algebraic properties.

Qiu, Lee BST 401

Motivation II

Again by our intuition, a probability P should have the

following properties:
1 If A, B are two well defined sets of outcomes, we should be
able to “talk about” the probability of A ∩ B, A ∪ B, and Ac or
Bc .
2 Other important properties such as non-negativity and
additivity which will be introduced later.
In other words, the domain of P is not just any collection of
sets, it must have certain algebraic properties.

Qiu, Lee BST 401

Motivation II

Again by our intuition, a probability P should have the

following properties:
1 If A, B are two well defined sets of outcomes, we should be
able to “talk about” the probability of A ∩ B, A ∪ B, and Ac or
Bc .
2 Other important properties such as non-negativity and
additivity which will be introduced later.
In other words, the domain of P is not just any collection of
sets, it must have certain algebraic properties.

Qiu, Lee BST 401

Motivation II

Again by our intuition, a probability P should have the

following properties:
1 If A, B are two well defined sets of outcomes, we should be
able to “talk about” the probability of A ∩ B, A ∪ B, and Ac or
Bc .
2 Other important properties such as non-negativity and
additivity which will be introduced later.
In other words, the domain of P is not just any collection of
sets, it must have certain algebraic properties.

Qiu, Lee BST 401

Motivation III

If a set function satisfies certain properties, it will be called

a measure. Later we will learn that all valid probabilities
are measures.
An important family of non-probability measure is the
family of Lebesgue measures, which is the usual length for
R1 , area for R2 , volume for R3 , etc.

Qiu, Lee BST 401

Motivation III

If a set function satisfies certain properties, it will be called

a measure. Later we will learn that all valid probabilities
are measures.
An important family of non-probability measure is the
family of Lebesgue measures, which is the usual length for
R1 , area for R2 , volume for R3 , etc.

Qiu, Lee BST 401

Set with infinite number of subsets

Def: An forms an increasing sequence of sets with limit A:

A1 ⊂ A2 ⊂ . . . and ∪∞
n=1 An = A. Denote as An ↑ A.
Similarly, we can define An ↓ A.

Qiu, Lee BST 401

Set with infinite number of subsets

Def: An forms an increasing sequence of sets with limit A:

A1 ⊂ A2 ⊂ . . . and ∪∞
n=1 An = A. Denote as An ↑ A.
Similarly, we can define An ↓ A.

Qiu, Lee BST 401

Upper/Lower Limit of a sequence of sets

Review the upper/lower limit of a sequence of real

numbers.
The analogy in set theory:
∞ [
\ ∞
lim supn An = Ak .
n k =n

ω ∈ lim infn An iff ω ∈ An for infinite many times.

Similarly
∞ \
[ ∞
lim infn An = Ak .
n k =n

ω ∈ lim infn An iff ω ∈

/ An for only finite many times.
If lim supn An = lim infn An = A, we say A = limn An .

Qiu, Lee BST 401

Upper/Lower Limit of a sequence of sets

Review the upper/lower limit of a sequence of real

numbers.
The analogy in set theory:
∞ [
\ ∞
lim supn An = Ak .
n k =n

ω ∈ lim infn An iff ω ∈ An for infinite many times.

Similarly
∞ \
[ ∞
lim infn An = Ak .
n k =n

ω ∈ lim infn An iff ω ∈

/ An for only finite many times.
If lim supn An = lim infn An = A, we say A = limn An .

Qiu, Lee BST 401

Upper/Lower Limit of a sequence of sets

Review the upper/lower limit of a sequence of real

numbers.
The analogy in set theory:
∞ [
\ ∞
lim supn An = Ak .
n k =n

ω ∈ lim infn An iff ω ∈ An for infinite many times.

Similarly
∞ \
[ ∞
lim infn An = Ak .
n k =n

ω ∈ lim infn An iff ω ∈

/ An for only finite many times.
If lim supn An = lim infn An = A, we say A = limn An .

Qiu, Lee BST 401

Upper/Lower Limit of a sequence of sets

Review the upper/lower limit of a sequence of real

numbers.
The analogy in set theory:
∞ [
\ ∞
lim supn An = Ak .
n k =n

ω ∈ lim infn An iff ω ∈ An for infinite many times.

Similarly
∞ \
[ ∞
lim infn An = Ak .
n k =n

ω ∈ lim infn An iff ω ∈

/ An for only finite many times.
If lim supn An = lim infn An = A, we say A = limn An .

Qiu, Lee BST 401

Algebras

Def: algebra of collection of subsets: closure under Ac and

A ∪ B, which implies closure under ∩.
Finite set algebra: always atomizable. So it is easy to
make it close under set operations.

Qiu, Lee BST 401

Algebras

Def: algebra of collection of subsets: closure under Ac and

A ∪ B, which implies closure under ∩.
Finite set algebra: always atomizable. So it is easy to
make it close under set operations.

Qiu, Lee BST 401

Why algebras? (I)

Why closure under mathematical operations?

A = 1, 2, 3, 4. A is not closed under +. Solution: extend A
to N.
For N, ·· is not well defined. (Partial) solution: extend N to
Q.
Q is not closed under the limit operation. Solution: extend
Q to R.
Strictly speaking, R is not closed under division since a0
(singular points) is undefined. It creates a lot of trouble!

Qiu, Lee BST 401

Why algebras? (I)

Why closure under mathematical operations?

A = 1, 2, 3, 4. A is not closed under +. Solution: extend A
to N.
For N, ·· is not well defined. (Partial) solution: extend N to
Q.
Q is not closed under the limit operation. Solution: extend
Q to R.
Strictly speaking, R is not closed under division since a0
(singular points) is undefined. It creates a lot of trouble!

Qiu, Lee BST 401

Why algebras? (I)

Why closure under mathematical operations?

A = 1, 2, 3, 4. A is not closed under +. Solution: extend A
to N.
For N, ·· is not well defined. (Partial) solution: extend N to
Q.
Q is not closed under the limit operation. Solution: extend
Q to R.
Strictly speaking, R is not closed under division since a0
(singular points) is undefined. It creates a lot of trouble!

Qiu, Lee BST 401

Why algebras? (I)

Why closure under mathematical operations?

A = 1, 2, 3, 4. A is not closed under +. Solution: extend A
to N.
For N, ·· is not well defined. (Partial) solution: extend N to
Q.
Q is not closed under the limit operation. Solution: extend
Q to R.
Strictly speaking, R is not closed under division since a0
(singular points) is undefined. It creates a lot of trouble!

Qiu, Lee BST 401

Why algebras? (II)

Why closure under set operations?

We do not need to worry about the validity of set
operations.
p
Real/complex number example: f (x) = (x).

Qiu, Lee BST 401

Why algebras? (II)

Why closure under set operations?

We do not need to worry about the validity of set
operations.
p
Real/complex number example: f (x) = (x).

Qiu, Lee BST 401

Σ-algebras

Def: σ-algebra: a algebra closed under countable infinite

unions/intersections.
The minimum σ-algebras.
The maximum σ-algebras.
Algebra but not σ-algebra. Ω = N. Collection F is defined
to be all subsets of finitely many numbers. Is the set of
even numbers a member of F?

Qiu, Lee BST 401

Σ-algebras

Def: σ-algebra: a algebra closed under countable infinite

unions/intersections.
The minimum σ-algebras.
The maximum σ-algebras.
Algebra but not σ-algebra. Ω = N. Collection F is defined
to be all subsets of finitely many numbers. Is the set of
even numbers a member of F?

Qiu, Lee BST 401

Σ-algebras

Def: σ-algebra: a algebra closed under countable infinite

unions/intersections.
The minimum σ-algebras.
The maximum σ-algebras.
Algebra but not σ-algebra. Ω = N. Collection F is defined
to be all subsets of finitely many numbers. Is the set of
even numbers a member of F?

Qiu, Lee BST 401

Σ-algebras

Def: σ-algebra: a algebra closed under countable infinite

unions/intersections.
The minimum σ-algebras.
The maximum σ-algebras.
Algebra but not σ-algebra. Ω = N. Collection F is defined
to be all subsets of finitely many numbers. Is the set of
even numbers a member of F?

Qiu, Lee BST 401

Why σ-algebras?

Closure
P∞ under countable infinite union makes it easy to use
n=1 , or replace the summation operation by integrals.
You can consider it as the Q to R extension: to ensure
taking limit is a valid operation.
Without this we still can talk about the finite step arithmetic
(for Q) or set (for sets) operations, yet we can not utilize
most of the modern mathematical tools (that is, pretty
much every theorem since calculus).

Qiu, Lee BST 401

Why σ-algebras?

Closure
P∞ under countable infinite union makes it easy to use
n=1 , or replace the summation operation by integrals.
You can consider it as the Q to R extension: to ensure
taking limit is a valid operation.
Without this we still can talk about the finite step arithmetic
(for Q) or set (for sets) operations, yet we can not utilize
most of the modern mathematical tools (that is, pretty
much every theorem since calculus).

Qiu, Lee BST 401

Why σ-algebras?

Closure
P∞ under countable infinite union makes it easy to use
n=1 , or replace the summation operation by integrals.
You can consider it as the Q to R extension: to ensure
taking limit is a valid operation.
Without this we still can talk about the finite step arithmetic
(for Q) or set (for sets) operations, yet we can not utilize
most of the modern mathematical tools (that is, pretty
much every theorem since calculus).

Qiu, Lee BST 401

σ-algebras and “information”

Taking as a whole, a σ-algebra represents some kind of

information: Some sets are valid, some sets are
“unspeakable”.
Finite case, 2 × 2 diagram: a coarser σ-algebra (minimum
one), and a finer one (row algebra, or the max algebra).
The column σ-algebra and the row σ-algebra represents
different information.
k × k grids. A trivial digital photo compression algorithm:
local average. (their out in the wild cousins are designed
with functional transformations, a subject we will briefly
touch when we discuss the characteristic functions.)
Infinite case, stock price prediction as a function of days.

Qiu, Lee BST 401

σ-algebras and “information”

Taking as a whole, a σ-algebra represents some kind of

information: Some sets are valid, some sets are
“unspeakable”.
Finite case, 2 × 2 diagram: a coarser σ-algebra (minimum
one), and a finer one (row algebra, or the max algebra).
The column σ-algebra and the row σ-algebra represents
different information.
k × k grids. A trivial digital photo compression algorithm:
local average. (their out in the wild cousins are designed
with functional transformations, a subject we will briefly
touch when we discuss the characteristic functions.)
Infinite case, stock price prediction as a function of days.

Qiu, Lee BST 401

σ-algebras and “information”

Taking as a whole, a σ-algebra represents some kind of

information: Some sets are valid, some sets are
“unspeakable”.
Finite case, 2 × 2 diagram: a coarser σ-algebra (minimum
one), and a finer one (row algebra, or the max algebra).
The column σ-algebra and the row σ-algebra represents
different information.
k × k grids. A trivial digital photo compression algorithm:
local average. (their out in the wild cousins are designed
with functional transformations, a subject we will briefly
touch when we discuss the characteristic functions.)
Infinite case, stock price prediction as a function of days.

Qiu, Lee BST 401

σ-algebras and “information”

Taking as a whole, a σ-algebra represents some kind of

information: Some sets are valid, some sets are
“unspeakable”.
Finite case, 2 × 2 diagram: a coarser σ-algebra (minimum
one), and a finer one (row algebra, or the max algebra).
The column σ-algebra and the row σ-algebra represents
different information.
k × k grids. A trivial digital photo compression algorithm:
local average. (their out in the wild cousins are designed
with functional transformations, a subject we will briefly
touch when we discuss the characteristic functions.)
Infinite case, stock price prediction as a function of days.

Qiu, Lee BST 401