Вы находитесь на странице: 1из 36
BST 401 Probability Theory Xing Qiu Ha Youn Lee Department of Biostatistics and Computational Biology

BST 401 Probability Theory

Xing Qiu

Ha Youn Lee

Department of Biostatistics and Computational Biology University of Rochester

September 9, 2010

Youn Lee Department of Biostatistics and Computational Biology University of Rochester September 9, 2010 Qiu, Lee

Qiu, Lee

BST 401

Outline 1 σ -algebras Qiu, Lee BST 401

Outline

1

σ-algebras

Qiu, Lee

BST 401

Outline 1 σ -algebras Qiu, Lee BST 401
Motivation I In this lecture we introduce the σ -algebra (or σ -algebra), which is

Motivation I

In this lecture we introduce the σ-algebra (or σ-algebra), which is a fundamentally useful tool in modern probability and statistics theory.

We will define notions such as probability, probability space and random variables formally later. Some informal knowledge about probability is useful for this lecture, though.

At the intuitive level, a probability is an assessment of how “likely” a particular set of outcomes might be. In other words, a probability is a set function: it takes a set as input, and then churns out a real number as its output.

is a set function : it takes a set as input, and then churns out a

Qiu, Lee

BST 401

Motivation I In this lecture we introduce the σ -algebra (or σ -algebra), which is

Motivation I

In this lecture we introduce the σ-algebra (or σ-algebra), which is a fundamentally useful tool in modern probability and statistics theory.

We will define notions such as probability, probability space and random variables formally later. Some informal knowledge about probability is useful for this lecture, though.

At the intuitive level, a probability is an assessment of how “likely” a particular set of outcomes might be. In other words, a probability is a set function: it takes a set as input, and then churns out a real number as its output.

is a set function : it takes a set as input, and then churns out a

Qiu, Lee

BST 401

Motivation I In this lecture we introduce the σ -algebra (or σ -algebra), which is

Motivation I

In this lecture we introduce the σ-algebra (or σ-algebra), which is a fundamentally useful tool in modern probability and statistics theory.

We will define notions such as probability, probability space and random variables formally later. Some informal knowledge about probability is useful for this lecture, though.

At the intuitive level, a probability is an assessment of how “likely” a particular set of outcomes might be. In other words, a probability is a set function: it takes a set as input, and then churns out a real number as its output.

is a set function : it takes a set as input, and then churns out a

Qiu, Lee

BST 401

Motivation II Again by our intuition, a probability P should have the following properties: 1

Motivation II

Again by our intuition, a probability P should have the following properties:

1 If A, B are two well defined sets of outcomes, we should be able to “talk about” the probability of A B, A B, and A c or B c .

2 Other important properties such as non-negativity and additivity which will be introduced later.

In other words, the domain of P is not just any collection of sets, it must have certain algebraic properties.

words, the domain of P is not just any collection of sets, it must have certain

Qiu, Lee

BST 401

Motivation II Again by our intuition, a probability P should have the following properties: 1

Motivation II

Again by our intuition, a probability P should have the following properties:

1 If A, B are two well defined sets of outcomes, we should be able to “talk about” the probability of A B, A B, and A c or B c .

2 Other important properties such as non-negativity and additivity which will be introduced later.

In other words, the domain of P is not just any collection of sets, it must have certain algebraic properties.

words, the domain of P is not just any collection of sets, it must have certain

Qiu, Lee

BST 401

Motivation II Again by our intuition, a probability P should have the following properties: 1

Motivation II

Again by our intuition, a probability P should have the following properties:

1 If A, B are two well defined sets of outcomes, we should be able to “talk about” the probability of A B, A B, and A c or B c .

2 Other important properties such as non-negativity and additivity which will be introduced later.

In other words, the domain of P is not just any collection of sets, it must have certain algebraic properties.

words, the domain of P is not just any collection of sets, it must have certain

Qiu, Lee

BST 401

Motivation II Again by our intuition, a probability P should have the following properties: 1

Motivation II

Again by our intuition, a probability P should have the following properties:

1 If A, B are two well defined sets of outcomes, we should be able to “talk about” the probability of A B, A B, and A c or B c .

2 Other important properties such as non-negativity and additivity which will be introduced later.

In other words, the domain of P is not just any collection of sets, it must have certain algebraic properties.

words, the domain of P is not just any collection of sets, it must have certain

Qiu, Lee

BST 401

Motivation III If a set function satisfies certain properties, it will be called a measure.

Motivation III

If a set function satisfies certain properties, it will be called

a measure. Later we will learn that all valid probabilities are measures.

An important family of non-probability measure is the family of Lebesgue measures, which is the usual length for R 1 , area for R 2 , volume for R 3 , etc.

measures, which is the usual length for R 1 , area for R 2 , volume

Qiu, Lee

BST 401

Motivation III If a set function satisfies certain properties, it will be called a measure.

Motivation III

If a set function satisfies certain properties, it will be called

a measure. Later we will learn that all valid probabilities are measures.

An important family of non-probability measure is the family of Lebesgue measures, which is the usual length for R 1 , area for R 2 , volume for R 3 , etc.

measures, which is the usual length for R 1 , area for R 2 , volume

Qiu, Lee

BST 401

Set with infinite number of subsets Def: A n forms an increasing sequence of sets

Set with infinite number of subsets

Def: A n forms an increasing sequence of sets with limit A:

A 1 A 2

Similarly, we can define A n A.

and

n=1 A n = A. Denote as A n A.

we can define A n ↓ A . and ∪ ∞ n = 1 A n

Qiu, Lee

BST 401

Set with infinite number of subsets Def: A n forms an increasing sequence of sets

Set with infinite number of subsets

Def: A n forms an increasing sequence of sets with limit A:

A 1 A 2

Similarly, we can define A n A.

and

n=1 A n = A. Denote as A n A.

we can define A n ↓ A . and ∪ ∞ n = 1 A n

Qiu, Lee

BST 401

Upper/Lower Limit of a sequence of sets Review the upper/lower limit of a sequence of

Upper/Lower Limit of a sequence of sets

Review the upper/lower limit of a sequence of real numbers.

The analogy in set theory:

lim sup n A n =

A k .

n k=n

ω lim inf n A n iff ω A n for infinite many times.

Similarly

lim inf n A n =

∞ ∞

A k .

n k=n

ω lim inf n A n iff ω / A n for only finite many times.

If lim sup n A n = lim inf n A n = A, we say A = lim n A n .

many times. If lim sup n A n = lim inf n A n = A

Qiu, Lee

BST 401

Upper/Lower Limit of a sequence of sets Review the upper/lower limit of a sequence of

Upper/Lower Limit of a sequence of sets

Review the upper/lower limit of a sequence of real numbers.

The analogy in set theory:

lim sup n A n =

A k .

n k=n

ω lim inf n A n iff ω A n for infinite many times.

Similarly

lim inf n A n =

A k .

n k=n

ω lim inf n A n iff ω / A n for only finite many times.

If lim sup n A n = lim inf n A n = A, we say A = lim n A n .

many times. If lim sup n A n = lim inf n A n = A

Qiu, Lee

BST 401

Upper/Lower Limit of a sequence of sets Review the upper/lower limit of a sequence of

Upper/Lower Limit of a sequence of sets

Review the upper/lower limit of a sequence of real numbers.

The analogy in set theory:

lim sup n A n =

A k .

n k=n

ω lim inf n A n iff ω A n for infinite many times.

Similarly

lim inf n A n =

∞ ∞

A k .

n k=n

ω lim inf n A n iff ω / A n for only finite many times.

If lim sup n A n = lim inf n A n = A, we say A = lim n A n .

many times. If lim sup n A n = lim inf n A n = A

Qiu, Lee

BST 401

Upper/Lower Limit of a sequence of sets Review the upper/lower limit of a sequence of

Upper/Lower Limit of a sequence of sets

Review the upper/lower limit of a sequence of real numbers.

The analogy in set theory:

lim sup n A n =

A k .

n k=n

ω lim inf n A n iff ω A n for infinite many times.

Similarly

lim inf n A n =

∞ ∞

A k .

n k=n

ω lim inf n A n iff ω / A n for only finite many times.

If lim sup n A n = lim inf n A n = A, we say A = lim n A n .

many times. If lim sup n A n = lim inf n A n = A

Qiu, Lee

BST 401

Algebras Def: algebra of collection of subsets: closure under A c and A ∪ B

Algebras

Def: algebra of collection of subsets: closure under A c and A B, which implies closure under .

Finite set algebra: always atomizable. So it is easy to make it close under set operations.

under ∩ . Finite set algebra: always atomizable. So it is easy to make it close

Qiu, Lee

BST 401

Algebras Def: algebra of collection of subsets: closure under A c and A ∪ B

Algebras

Def: algebra of collection of subsets: closure under A c and A B, which implies closure under .

Finite set algebra: always atomizable. So it is easy to make it close under set operations.

under ∩ . Finite set algebra: always atomizable. So it is easy to make it close

Qiu, Lee

BST 401

Why algebras? (I) Why closure under mathematical operations? A = 1 , 2 , 3

Why algebras? (I)

Why closure under mathematical operations?

A = 1, 2, 3, 4. A is not closed under +. Solution: extend A to N.

For N, · · is not well defined. (Partial) solution: extend N to

Q.

Q

is not closed under the limit operation. Solution: extend

Q

to R.

a

Strictly speaking, R is not closed under division since (singular points) is undefined. It creates a lot of trouble!

0

R is not closed under division since (singular points) is undefined. It creates a lot of

Qiu, Lee

BST 401

Why algebras? (I) Why closure under mathematical operations? A = 1 , 2 , 3

Why algebras? (I)

Why closure under mathematical operations?

A = 1, 2, 3, 4. A is not closed under +. Solution: extend A to N.

For N, · · is not well defined. (Partial) solution: extend N to

Q.

Q

is not closed under the limit operation. Solution: extend

Q

to R.

a

Strictly speaking, R is not closed under division since (singular points) is undefined. It creates a lot of trouble!

0

R is not closed under division since (singular points) is undefined. It creates a lot of

Qiu, Lee

BST 401

Why algebras? (I) Why closure under mathematical operations? A = 1 , 2 , 3

Why algebras? (I)

Why closure under mathematical operations?

A = 1, 2, 3, 4. A is not closed under +. Solution: extend A to N.

For N, · · is not well defined. (Partial) solution: extend N to

Q.

Q

is not closed under the limit operation. Solution: extend

Q

to R.

a

Strictly speaking, R is not closed under division since (singular points) is undefined. It creates a lot of trouble!

0

R is not closed under division since (singular points) is undefined. It creates a lot of

Qiu, Lee

BST 401

Why algebras? (I) Why closure under mathematical operations? A = 1 , 2 , 3

Why algebras? (I)

Why closure under mathematical operations?

A = 1, 2, 3, 4. A is not closed under +. Solution: extend A to N.

For N, · · is not well defined. (Partial) solution: extend N to

Q.

Q

is not closed under the limit operation. Solution: extend

Q

to R.

a

Strictly speaking, R is not closed under division since (singular points) is undefined. It creates a lot of trouble!

0

R is not closed under division since (singular points) is undefined. It creates a lot of

Qiu, Lee

BST 401

Why algebras? (II) Why closure under set operations? We do not need to worry about

Why algebras? (II)

Why closure under set operations?

We do not need to worry about the validity of set operations.

Real/complex number example: f (x) = (x).

need to worry about the validity of set operations. Real/complex number example: f ( x )

Qiu, Lee

BST 401

Why algebras? (II) Why closure under set operations? We do not need to worry about

Why algebras? (II)

Why closure under set operations?

We do not need to worry about the validity of set operations.

Real/complex number example: f (x) = (x).

need to worry about the validity of set operations. Real/complex number example: f ( x )

Qiu, Lee

BST 401

Σ -algebras Def: σ -algebra: a algebra closed under countable infinite unions/intersections. The minimum σ

Σ-algebras

Def: σ-algebra: a algebra closed under countable infinite unions/intersections.

The minimum σ-algebras.

The maximum σ-algebras.

Algebra but not σ-algebra. Ω = N. Collection F is defined to be all subsets of finitely many numbers. Is the set of even numbers a member of F ?

F is defined to be all subsets of finitely many numbers. Is the set of even

Qiu, Lee

BST 401

Σ -algebras Def: σ -algebra: a algebra closed under countable infinite unions/intersections. The minimum σ

Σ-algebras

Def: σ-algebra: a algebra closed under countable infinite unions/intersections.

The minimum σ-algebras.

The maximum σ-algebras.

Algebra but not σ-algebra. Ω = N. Collection F is defined to be all subsets of finitely many numbers. Is the set of even numbers a member of F ?

F is defined to be all subsets of finitely many numbers. Is the set of even

Qiu, Lee

BST 401

Σ -algebras Def: σ -algebra: a algebra closed under countable infinite unions/intersections. The minimum σ

Σ-algebras

Def: σ-algebra: a algebra closed under countable infinite unions/intersections.

The minimum σ-algebras.

The maximum σ-algebras.

Algebra but not σ-algebra. Ω = N. Collection F is defined to be all subsets of finitely many numbers. Is the set of even numbers a member of F ?

F is defined to be all subsets of finitely many numbers. Is the set of even

Qiu, Lee

BST 401

Σ -algebras Def: σ -algebra: a algebra closed under countable infinite unions/intersections. The minimum σ

Σ-algebras

Def: σ-algebra: a algebra closed under countable infinite unions/intersections.

The minimum σ-algebras.

The maximum σ-algebras.

Algebra but not σ-algebra. Ω = N. Collection F is defined to be all subsets of finitely many numbers. Is the set of even numbers a member of F ?

F is defined to be all subsets of finitely many numbers. Is the set of even

Qiu, Lee

BST 401

Why σ -algebras? Closure under countable infinite union makes it easy to use n =

Why σ-algebras?

Closure under countable infinite union makes it easy to use

n=1 , or replace the summation operation by integrals.

You can consider it as the Q to R extension: to ensure taking limit is a valid operation.

Without this we still can talk about the finite step arithmetic (for Q) or set (for sets) operations, yet we can not utilize most of the modern mathematical tools (that is, pretty much every theorem since calculus).

not utilize most of the modern mathematical tools (that is, pretty much every theorem since calculus).

Qiu, Lee

BST 401

Why σ -algebras? Closure under countable infinite union makes it easy to use n =

Why σ-algebras?

Closure under countable infinite union makes it easy to use

n=1 , or replace the summation operation by integrals.

You can consider it as the Q to R extension: to ensure taking limit is a valid operation.

Without this we still can talk about the finite step arithmetic (for Q) or set (for sets) operations, yet we can not utilize most of the modern mathematical tools (that is, pretty much every theorem since calculus).

not utilize most of the modern mathematical tools (that is, pretty much every theorem since calculus).

Qiu, Lee

BST 401

Why σ -algebras? Closure under countable infinite union makes it easy to use n =

Why σ-algebras?

Closure under countable infinite union makes it easy to use

n=1 , or replace the summation operation by integrals.

You can consider it as the Q to R extension: to ensure taking limit is a valid operation.

Without this we still can talk about the finite step arithmetic (for Q) or set (for sets) operations, yet we can not utilize most of the modern mathematical tools (that is, pretty much every theorem since calculus).

not utilize most of the modern mathematical tools (that is, pretty much every theorem since calculus).

Qiu, Lee

BST 401

σ -algebras and “information” Taking as a whole, a σ -algebra represents some kind of

σ-algebras and “information”

Taking as a whole, a σ-algebra represents some kind of information: Some sets are valid, some sets are “unspeakable”.

Finite case, 2 × 2 diagram: a coarser σ-algebra (minimum one), and a finer one (row algebra, or the max algebra). The column σ-algebra and the row σ-algebra represents different information.

k × k grids. A trivial digital photo compression algorithm:

local average. (their out in the wild cousins are designed with functional transformations, a subject we will briefly touch when we discuss the characteristic functions.)

Infinite case, stock price prediction as a function of days.

we discuss the characteristic functions.) Infinite case, stock price prediction as a function of days. Qiu,

Qiu, Lee

BST 401

σ -algebras and “information” Taking as a whole, a σ -algebra represents some kind of

σ-algebras and “information”

Taking as a whole, a σ-algebra represents some kind of information: Some sets are valid, some sets are “unspeakable”.

Finite case, 2 × 2 diagram: a coarser σ-algebra (minimum one), and a finer one (row algebra, or the max algebra). The column σ-algebra and the row σ-algebra represents different information.

k × k grids. A trivial digital photo compression algorithm:

local average. (their out in the wild cousins are designed with functional transformations, a subject we will briefly touch when we discuss the characteristic functions.)

Infinite case, stock price prediction as a function of days.

we discuss the characteristic functions.) Infinite case, stock price prediction as a function of days. Qiu,

Qiu, Lee

BST 401

σ -algebras and “information” Taking as a whole, a σ -algebra represents some kind of

σ-algebras and “information”

Taking as a whole, a σ-algebra represents some kind of information: Some sets are valid, some sets are “unspeakable”.

Finite case, 2 × 2 diagram: a coarser σ-algebra (minimum one), and a finer one (row algebra, or the max algebra). The column σ-algebra and the row σ-algebra represents different information.

k × k grids. A trivial digital photo compression algorithm:

local average. (their out in the wild cousins are designed with functional transformations, a subject we will briefly touch when we discuss the characteristic functions.)

Infinite case, stock price prediction as a function of days.

we discuss the characteristic functions.) Infinite case, stock price prediction as a function of days. Qiu,

Qiu, Lee

BST 401

σ -algebras and “information” Taking as a whole, a σ -algebra represents some kind of

σ-algebras and “information”

Taking as a whole, a σ-algebra represents some kind of information: Some sets are valid, some sets are “unspeakable”.

Finite case, 2 × 2 diagram: a coarser σ-algebra (minimum one), and a finer one (row algebra, or the max algebra). The column σ-algebra and the row σ-algebra represents different information.

k × k grids. A trivial digital photo compression algorithm:

local average. (their out in the wild cousins are designed with functional transformations, a subject we will briefly touch when we discuss the characteristic functions.)

Infinite case, stock price prediction as a function of days.

we discuss the characteristic functions.) Infinite case, stock price prediction as a function of days. Qiu,

Qiu, Lee

BST 401