Вы находитесь на странице: 1из 36

BST 401 Probability Theory

Xing Qiu

Ha Youn Lee

Department of Biostatistics and Computational Biology University of Rochester

September 9, 2010

Qiu, Lee

BST 401

Outline

1

σ-algebras

Qiu, Lee

BST 401

Motivation I

In this lecture we introduce the σ-algebra (or σ-algebra), which is a fundamentally useful tool in modern probability and statistics theory.

We will deﬁne notions such as probability, probability space and random variables formally later. Some informal knowledge about probability is useful for this lecture, though.

At the intuitive level, a probability is an assessment of how “likely” a particular set of outcomes might be. In other words, a probability is a set function: it takes a set as input, and then churns out a real number as its output.

Qiu, Lee

BST 401

Motivation I

In this lecture we introduce the σ-algebra (or σ-algebra), which is a fundamentally useful tool in modern probability and statistics theory.

We will deﬁne notions such as probability, probability space and random variables formally later. Some informal knowledge about probability is useful for this lecture, though.

At the intuitive level, a probability is an assessment of how “likely” a particular set of outcomes might be. In other words, a probability is a set function: it takes a set as input, and then churns out a real number as its output.

Qiu, Lee

BST 401

Motivation I

In this lecture we introduce the σ-algebra (or σ-algebra), which is a fundamentally useful tool in modern probability and statistics theory.

We will deﬁne notions such as probability, probability space and random variables formally later. Some informal knowledge about probability is useful for this lecture, though.

At the intuitive level, a probability is an assessment of how “likely” a particular set of outcomes might be. In other words, a probability is a set function: it takes a set as input, and then churns out a real number as its output.

Qiu, Lee

BST 401

Motivation II

Again by our intuition, a probability P should have the following properties:

1 If A, B are two well deﬁned sets of outcomes, we should be able to “talk about” the probability of A B, A B, and A c or B c .

2 Other important properties such as non-negativity and additivity which will be introduced later.

In other words, the domain of P is not just any collection of sets, it must have certain algebraic properties.

Qiu, Lee

BST 401

Motivation II

Again by our intuition, a probability P should have the following properties:

1 If A, B are two well deﬁned sets of outcomes, we should be able to “talk about” the probability of A B, A B, and A c or B c .

2 Other important properties such as non-negativity and additivity which will be introduced later.

In other words, the domain of P is not just any collection of sets, it must have certain algebraic properties.

Qiu, Lee

BST 401

Motivation II

Again by our intuition, a probability P should have the following properties:

1 If A, B are two well deﬁned sets of outcomes, we should be able to “talk about” the probability of A B, A B, and A c or B c .

2 Other important properties such as non-negativity and additivity which will be introduced later.

In other words, the domain of P is not just any collection of sets, it must have certain algebraic properties.

Qiu, Lee

BST 401

Motivation II

Again by our intuition, a probability P should have the following properties:

1 If A, B are two well deﬁned sets of outcomes, we should be able to “talk about” the probability of A B, A B, and A c or B c .

2 Other important properties such as non-negativity and additivity which will be introduced later.

In other words, the domain of P is not just any collection of sets, it must have certain algebraic properties.

Qiu, Lee

BST 401

Motivation III

If a set function satisﬁes certain properties, it will be called

a measure. Later we will learn that all valid probabilities are measures.

An important family of non-probability measure is the family of Lebesgue measures, which is the usual length for R 1 , area for R 2 , volume for R 3 , etc.

Qiu, Lee

BST 401

Motivation III

If a set function satisﬁes certain properties, it will be called

a measure. Later we will learn that all valid probabilities are measures.

An important family of non-probability measure is the family of Lebesgue measures, which is the usual length for R 1 , area for R 2 , volume for R 3 , etc.

Qiu, Lee

BST 401

Set with inﬁnite number of subsets

Def: A n forms an increasing sequence of sets with limit A:

A 1 A 2

Similarly, we can deﬁne A n A.

and

n=1 A n = A. Denote as A n A.

Qiu, Lee

BST 401

Set with inﬁnite number of subsets

Def: A n forms an increasing sequence of sets with limit A:

A 1 A 2

Similarly, we can deﬁne A n A.

and

n=1 A n = A. Denote as A n A.

Qiu, Lee

BST 401

Upper/Lower Limit of a sequence of sets

Review the upper/lower limit of a sequence of real numbers.

The analogy in set theory:

lim sup n A n =

A k .

n k=n

ω lim inf n A n iff ω A n for inﬁnite many times.

Similarly

lim inf n A n =

∞ ∞

A k .

n k=n

ω lim inf n A n iff ω / A n for only ﬁnite many times.

If lim sup n A n = lim inf n A n = A, we say A = lim n A n .

Qiu, Lee

BST 401

Upper/Lower Limit of a sequence of sets

Review the upper/lower limit of a sequence of real numbers.

The analogy in set theory:

lim sup n A n =

A k .

n k=n

ω lim inf n A n iff ω A n for inﬁnite many times.

Similarly

lim inf n A n =

A k .

n k=n

ω lim inf n A n iff ω / A n for only ﬁnite many times.

If lim sup n A n = lim inf n A n = A, we say A = lim n A n .

Qiu, Lee

BST 401

Upper/Lower Limit of a sequence of sets

Review the upper/lower limit of a sequence of real numbers.

The analogy in set theory:

lim sup n A n =

A k .

n k=n

ω lim inf n A n iff ω A n for inﬁnite many times.

Similarly

lim inf n A n =

∞ ∞

A k .

n k=n

ω lim inf n A n iff ω / A n for only ﬁnite many times.

If lim sup n A n = lim inf n A n = A, we say A = lim n A n .

Qiu, Lee

BST 401

Upper/Lower Limit of a sequence of sets

Review the upper/lower limit of a sequence of real numbers.

The analogy in set theory:

lim sup n A n =

A k .

n k=n

ω lim inf n A n iff ω A n for inﬁnite many times.

Similarly

lim inf n A n =

∞ ∞

A k .

n k=n

ω lim inf n A n iff ω / A n for only ﬁnite many times.

If lim sup n A n = lim inf n A n = A, we say A = lim n A n .

Qiu, Lee

BST 401

Algebras

Def: algebra of collection of subsets: closure under A c and A B, which implies closure under .

Finite set algebra: always atomizable. So it is easy to make it close under set operations.

Qiu, Lee

BST 401

Algebras

Def: algebra of collection of subsets: closure under A c and A B, which implies closure under .

Finite set algebra: always atomizable. So it is easy to make it close under set operations.

Qiu, Lee

BST 401

Why algebras? (I)

Why closure under mathematical operations?

A = 1, 2, 3, 4. A is not closed under +. Solution: extend A to N.

For N, · · is not well deﬁned. (Partial) solution: extend N to

 Q. Q is not closed under the limit operation. Solution: extend Q to R.

a

Strictly speaking, R is not closed under division since (singular points) is undeﬁned. It creates a lot of trouble!

0

Qiu, Lee

BST 401

Why algebras? (I)

Why closure under mathematical operations?

A = 1, 2, 3, 4. A is not closed under +. Solution: extend A to N.

For N, · · is not well deﬁned. (Partial) solution: extend N to

 Q. Q is not closed under the limit operation. Solution: extend Q to R.

a

Strictly speaking, R is not closed under division since (singular points) is undeﬁned. It creates a lot of trouble!

0

Qiu, Lee

BST 401

Why algebras? (I)

Why closure under mathematical operations?

A = 1, 2, 3, 4. A is not closed under +. Solution: extend A to N.

For N, · · is not well deﬁned. (Partial) solution: extend N to

 Q. Q is not closed under the limit operation. Solution: extend Q to R.

a

Strictly speaking, R is not closed under division since (singular points) is undeﬁned. It creates a lot of trouble!

0

Qiu, Lee

BST 401

Why algebras? (I)

Why closure under mathematical operations?

A = 1, 2, 3, 4. A is not closed under +. Solution: extend A to N.

For N, · · is not well deﬁned. (Partial) solution: extend N to

 Q. Q is not closed under the limit operation. Solution: extend Q to R.

a

Strictly speaking, R is not closed under division since (singular points) is undeﬁned. It creates a lot of trouble!

0

Qiu, Lee

BST 401

Why algebras? (II)

Why closure under set operations?

We do not need to worry about the validity of set operations.

Real/complex number example: f (x) = (x).

Qiu, Lee

BST 401

Why algebras? (II)

Why closure under set operations?

We do not need to worry about the validity of set operations.

Real/complex number example: f (x) = (x).

Qiu, Lee

BST 401

Σ-algebras

Def: σ-algebra: a algebra closed under countable inﬁnite unions/intersections.

The minimum σ-algebras.

The maximum σ-algebras.

Algebra but not σ-algebra. Ω = N. Collection F is deﬁned to be all subsets of ﬁnitely many numbers. Is the set of even numbers a member of F ?

Qiu, Lee

BST 401

Σ-algebras

Def: σ-algebra: a algebra closed under countable inﬁnite unions/intersections.

The minimum σ-algebras.

The maximum σ-algebras.

Algebra but not σ-algebra. Ω = N. Collection F is deﬁned to be all subsets of ﬁnitely many numbers. Is the set of even numbers a member of F ?

Qiu, Lee

BST 401

Σ-algebras

Def: σ-algebra: a algebra closed under countable inﬁnite unions/intersections.

The minimum σ-algebras.

The maximum σ-algebras.

Algebra but not σ-algebra. Ω = N. Collection F is deﬁned to be all subsets of ﬁnitely many numbers. Is the set of even numbers a member of F ?

Qiu, Lee

BST 401

Σ-algebras

Def: σ-algebra: a algebra closed under countable inﬁnite unions/intersections.

The minimum σ-algebras.

The maximum σ-algebras.

Algebra but not σ-algebra. Ω = N. Collection F is deﬁned to be all subsets of ﬁnitely many numbers. Is the set of even numbers a member of F ?

Qiu, Lee

BST 401

Why σ-algebras?

Closure under countable inﬁnite union makes it easy to use

n=1 , or replace the summation operation by integrals.

You can consider it as the Q to R extension: to ensure taking limit is a valid operation.

Without this we still can talk about the ﬁnite step arithmetic (for Q) or set (for sets) operations, yet we can not utilize most of the modern mathematical tools (that is, pretty much every theorem since calculus).

Qiu, Lee

BST 401

Why σ-algebras?

Closure under countable inﬁnite union makes it easy to use

n=1 , or replace the summation operation by integrals.

You can consider it as the Q to R extension: to ensure taking limit is a valid operation.

Without this we still can talk about the ﬁnite step arithmetic (for Q) or set (for sets) operations, yet we can not utilize most of the modern mathematical tools (that is, pretty much every theorem since calculus).

Qiu, Lee

BST 401

Why σ-algebras?

Closure under countable inﬁnite union makes it easy to use

n=1 , or replace the summation operation by integrals.

You can consider it as the Q to R extension: to ensure taking limit is a valid operation.

Without this we still can talk about the ﬁnite step arithmetic (for Q) or set (for sets) operations, yet we can not utilize most of the modern mathematical tools (that is, pretty much every theorem since calculus).

Qiu, Lee

BST 401

σ-algebras and “information”

Taking as a whole, a σ-algebra represents some kind of information: Some sets are valid, some sets are “unspeakable”.

Finite case, 2 × 2 diagram: a coarser σ-algebra (minimum one), and a ﬁner one (row algebra, or the max algebra). The column σ-algebra and the row σ-algebra represents different information.

k × k grids. A trivial digital photo compression algorithm:

local average. (their out in the wild cousins are designed with functional transformations, a subject we will brieﬂy touch when we discuss the characteristic functions.)

Inﬁnite case, stock price prediction as a function of days.

Qiu, Lee

BST 401

σ-algebras and “information”

Taking as a whole, a σ-algebra represents some kind of information: Some sets are valid, some sets are “unspeakable”.

Finite case, 2 × 2 diagram: a coarser σ-algebra (minimum one), and a ﬁner one (row algebra, or the max algebra). The column σ-algebra and the row σ-algebra represents different information.

k × k grids. A trivial digital photo compression algorithm:

local average. (their out in the wild cousins are designed with functional transformations, a subject we will brieﬂy touch when we discuss the characteristic functions.)

Inﬁnite case, stock price prediction as a function of days.

Qiu, Lee

BST 401

σ-algebras and “information”

Taking as a whole, a σ-algebra represents some kind of information: Some sets are valid, some sets are “unspeakable”.

Finite case, 2 × 2 diagram: a coarser σ-algebra (minimum one), and a ﬁner one (row algebra, or the max algebra). The column σ-algebra and the row σ-algebra represents different information.

k × k grids. A trivial digital photo compression algorithm:

local average. (their out in the wild cousins are designed with functional transformations, a subject we will brieﬂy touch when we discuss the characteristic functions.)

Inﬁnite case, stock price prediction as a function of days.

Qiu, Lee

BST 401

σ-algebras and “information”

Taking as a whole, a σ-algebra represents some kind of information: Some sets are valid, some sets are “unspeakable”.

Finite case, 2 × 2 diagram: a coarser σ-algebra (minimum one), and a ﬁner one (row algebra, or the max algebra). The column σ-algebra and the row σ-algebra represents different information.

k × k grids. A trivial digital photo compression algorithm:

local average. (their out in the wild cousins are designed with functional transformations, a subject we will brieﬂy touch when we discuss the characteristic functions.)

Inﬁnite case, stock price prediction as a function of days.

Qiu, Lee

BST 401