BST 401 Probability Theory

Xing Qiu Ha Youn Lee

Department of Biostatistics and Computational Biology

University of Rochester

September 2, 2010

Qiu, Lee BST 401

Outline

1 Set and Functions

Qiu, Lee BST 401

Set Operations

The whole set Ω, the empty set φ; an element x in a set A:

x ∈ A.
A ⊂ B, A ⊆ B; A ⊃ B, A ⊇ B.
Set operations: A ∩ B, A ∪ B, Ac (w.r.t. Ω).
De Morgan’s laws: (A ∪ B)c = Ac ∩ B c , (A ∩ B)c = Ac ∪ B c .
For more than two sets:
!c !c
[ \ \ [
An = Acn , An = Acn .
n n n n

Qiu, Lee BST 401

Set Operations

The whole set Ω, the empty set φ; an element x in a set A:

x ∈ A.
A ⊂ B, A ⊆ B; A ⊃ B, A ⊇ B.
Set operations: A ∩ B, A ∪ B, Ac (w.r.t. Ω).
De Morgan’s laws: (A ∪ B)c = Ac ∩ B c , (A ∩ B)c = Ac ∪ B c .
For more than two sets:
!c !c
[ \ \ [
An = Acn , An = Acn .
n n n n

Qiu, Lee BST 401

Set Operations

The whole set Ω, the empty set φ; an element x in a set A:

x ∈ A.
A ⊂ B, A ⊆ B; A ⊃ B, A ⊇ B.
Set operations: A ∩ B, A ∪ B, Ac (w.r.t. Ω).
De Morgan’s laws: (A ∪ B)c = Ac ∩ B c , (A ∩ B)c = Ac ∪ B c .
For more than two sets:
!c !c
[ \ \ [
An = Acn , An = Acn .
n n n n

Qiu, Lee BST 401

Set Operations

The whole set Ω, the empty set φ; an element x in a set A:

x ∈ A.
A ⊂ B, A ⊆ B; A ⊃ B, A ⊇ B.
Set operations: A ∩ B, A ∪ B, Ac (w.r.t. Ω).
De Morgan’s laws: (A ∪ B)c = Ac ∩ B c , (A ∩ B)c = Ac ∪ B c .
For more than two sets:
!c !c
[ \ \ [
An = Acn , An = Acn .
n n n n

Qiu, Lee BST 401

Set Operations

The whole set Ω, the empty set φ; an element x in a set A:

x ∈ A.
A ⊂ B, A ⊆ B; A ⊃ B, A ⊇ B.
Set operations: A ∩ B, A ∪ B, Ac (w.r.t. Ω).
De Morgan’s laws: (A ∪ B)c = Ac ∩ B c , (A ∩ B)c = Ac ∪ B c .
For more than two sets:
!c !c
[ \ \ [
An = Acn , An = Acn .
n n n n

Qiu, Lee BST 401

Disjoint “atoms”: the classical three-circle-diagram.

Figure: Three sets can generate 7 atoms.

Qiu, Lee BST 401

Functions

Please review the basic definitions of a function. Pay

attention to a function’s domain and its image.
I assume you know the definition and basic properties of
the following elementary functions:
1 Power functions, x a , a ∈ R. When a is a non-integer
rational number, without confusion we take the principle
branch of nth root operation; when a is irrational, its
definition is given by the principle branch of a log function.
2 Exponential and logarithmic functions, ex and log(x).
3 Trigonometric functions and their inverse functions. Their
domain, image, etc.
4 The combination of the above by +, ×, ÷, and functional
composition.

Qiu, Lee BST 401

Functions

Please review the basic definitions of a function. Pay

attention to a function’s domain and its image.
I assume you know the definition and basic properties of
the following elementary functions:
1 Power functions, x a , a ∈ R. When a is a non-integer
rational number, without confusion we take the principle
branch of nth root operation; when a is irrational, its
definition is given by the principle branch of a log function.
2 Exponential and logarithmic functions, ex and log(x).
3 Trigonometric functions and their inverse functions. Their
domain, image, etc.
4 The combination of the above by +, ×, ÷, and functional
composition.

Qiu, Lee BST 401

Functions

Please review the basic definitions of a function. Pay

attention to a function’s domain and its image.
I assume you know the definition and basic properties of
the following elementary functions:
1 Power functions, x a , a ∈ R. When a is a non-integer
rational number, without confusion we take the principle
branch of nth root operation; when a is irrational, its
definition is given by the principle branch of a log function.
2 Exponential and logarithmic functions, ex and log(x).
3 Trigonometric functions and their inverse functions. Their
domain, image, etc.
4 The combination of the above by +, ×, ÷, and functional
composition.

Qiu, Lee BST 401

Functions

Please review the basic definitions of a function. Pay

attention to a function’s domain and its image.
I assume you know the definition and basic properties of
the following elementary functions:
1 Power functions, x a , a ∈ R. When a is a non-integer
rational number, without confusion we take the principle
branch of nth root operation; when a is irrational, its
definition is given by the principle branch of a log function.
2 Exponential and logarithmic functions, ex and log(x).
3 Trigonometric functions and their inverse functions. Their
domain, image, etc.
4 The combination of the above by +, ×, ÷, and functional
composition.

Qiu, Lee BST 401

Functions

Please review the basic definitions of a function. Pay

attention to a function’s domain and its image.
I assume you know the definition and basic properties of
the following elementary functions:
1 Power functions, x a , a ∈ R. When a is a non-integer
rational number, without confusion we take the principle
branch of nth root operation; when a is irrational, its
definition is given by the principle branch of a log function.
2 Exponential and logarithmic functions, ex and log(x).
3 Trigonometric functions and their inverse functions. Their
domain, image, etc.
4 The combination of the above by +, ×, ÷, and functional
composition.

Qiu, Lee BST 401

Functions

Please review the basic definitions of a function. Pay

attention to a function’s domain and its image.
I assume you know the definition and basic properties of
the following elementary functions:
1 Power functions, x a , a ∈ R. When a is a non-integer
rational number, without confusion we take the principle
branch of nth root operation; when a is irrational, its
definition is given by the principle branch of a log function.
2 Exponential and logarithmic functions, ex and log(x).
3 Trigonometric functions and their inverse functions. Their
domain, image, etc.
4 The combination of the above by +, ×, ÷, and functional
composition.

Qiu, Lee BST 401

Limit of a sequence of real numbers (I)

For simplicity, we are going to use “increasing” to mean

“non-decreasing”. “decreasing” to mean “non-increasing”.
A sequence of real numbers (ai ) = (a1 , a2 , . . .) converges
to a∗ if for any given “precision criterion”  > 0, there exists
an integer N such that the “error”, defined as
dist(ai − a∗ ) = |ai − a∗ |, is smaller than  for all i ≥ N.
An increasing, bounded sequence of real numbers
a1 ≤ a2 ≤ . . . always converges to a limit. (if you consider
“∞” as a valid limiting point, the boundedness part can be
omitted.)
Similarly, a decreasing sequence of real numbers always
converges to a limit (if you don’t like “−∞”, you can add the
bounded from below condition).

Qiu, Lee BST 401

Limit of a sequence of real numbers (I)

For simplicity, we are going to use “increasing” to mean

“non-decreasing”. “decreasing” to mean “non-increasing”.
A sequence of real numbers (ai ) = (a1 , a2 , . . .) converges
to a∗ if for any given “precision criterion”  > 0, there exists
an integer N such that the “error”, defined as
dist(ai − a∗ ) = |ai − a∗ |, is smaller than  for all i ≥ N.
An increasing, bounded sequence of real numbers
a1 ≤ a2 ≤ . . . always converges to a limit. (if you consider
“∞” as a valid limiting point, the boundedness part can be
omitted.)
Similarly, a decreasing sequence of real numbers always
converges to a limit (if you don’t like “−∞”, you can add the
bounded from below condition).

Qiu, Lee BST 401

Limit of a sequence of real numbers (I)

For simplicity, we are going to use “increasing” to mean

“non-decreasing”. “decreasing” to mean “non-increasing”.
A sequence of real numbers (ai ) = (a1 , a2 , . . .) converges
to a∗ if for any given “precision criterion”  > 0, there exists
an integer N such that the “error”, defined as
dist(ai − a∗ ) = |ai − a∗ |, is smaller than  for all i ≥ N.
An increasing, bounded sequence of real numbers
a1 ≤ a2 ≤ . . . always converges to a limit. (if you consider
“∞” as a valid limiting point, the boundedness part can be
omitted.)
Similarly, a decreasing sequence of real numbers always
converges to a limit (if you don’t like “−∞”, you can add the
bounded from below condition).

Qiu, Lee BST 401

Limit of a sequence of real numbers (I)

For simplicity, we are going to use “increasing” to mean

“non-decreasing”. “decreasing” to mean “non-increasing”.
A sequence of real numbers (ai ) = (a1 , a2 , . . .) converges
to a∗ if for any given “precision criterion”  > 0, there exists
an integer N such that the “error”, defined as
dist(ai − a∗ ) = |ai − a∗ |, is smaller than  for all i ≥ N.
An increasing, bounded sequence of real numbers
a1 ≤ a2 ≤ . . . always converges to a limit. (if you consider
“∞” as a valid limiting point, the boundedness part can be
omitted.)
Similarly, a decreasing sequence of real numbers always
converges to a limit (if you don’t like “−∞”, you can add the
bounded from below condition).

Qiu, Lee BST 401

Limit of a sequence of real numbers (II)

In general a sequence of real numbers (a1 , a2 , . . .) always

has a subsequence which approaches the upper limit
(including “∞” as a possible limit) of this sequence.
Similarly, it contains a subsequence which approaches its
lower limit. Once the upper limit equals the lower limit, we
say this sequence converges to this limit.
Two companion subsequences, denoted as (b1 , b2 , . . .)
and (c1 , c2 , . . .), can be quite useful:

bi = sup ai , ci = inf ai .
i≥n i≥n

(bi ) is decreasing and (ci ) is increasing and they converge

to lim supi→∞ ai and lim infi→∞ ai , respectively.

Qiu, Lee BST 401

Limit of a sequence of real numbers (II)

In general a sequence of real numbers (a1 , a2 , . . .) always

has a subsequence which approaches the upper limit
(including “∞” as a possible limit) of this sequence.
Similarly, it contains a subsequence which approaches its
lower limit. Once the upper limit equals the lower limit, we
say this sequence converges to this limit.
Two companion subsequences, denoted as (b1 , b2 , . . .)
and (c1 , c2 , . . .), can be quite useful:

bi = sup ai , ci = inf ai .
i≥n i≥n

(bi ) is decreasing and (ci ) is increasing and they converge

to lim supi→∞ ai and lim infi→∞ ai , respectively.

Qiu, Lee BST 401

Limit of a sequence of real numbers (II)

In general a sequence of real numbers (a1 , a2 , . . .) always

has a subsequence which approaches the upper limit
(including “∞” as a possible limit) of this sequence.
Similarly, it contains a subsequence which approaches its
lower limit. Once the upper limit equals the lower limit, we
say this sequence converges to this limit.
Two companion subsequences, denoted as (b1 , b2 , . . .)
and (c1 , c2 , . . .), can be quite useful:

bi = sup ai , ci = inf ai .
i≥n i≥n

(bi ) is decreasing and (ci ) is increasing and they converge

to lim supi→∞ ai and lim infi→∞ ai , respectively.

Qiu, Lee BST 401

Limit of a sequence of real numbers (II)

In general a sequence of real numbers (a1 , a2 , . . .) always

has a subsequence which approaches the upper limit
(including “∞” as a possible limit) of this sequence.
Similarly, it contains a subsequence which approaches its
lower limit. Once the upper limit equals the lower limit, we
say this sequence converges to this limit.
Two companion subsequences, denoted as (b1 , b2 , . . .)
and (c1 , c2 , . . .), can be quite useful:

bi = sup ai , ci = inf ai .
i≥n i≥n

(bi ) is decreasing and (ci ) is increasing and they converge

to lim supi→∞ ai and lim infi→∞ ai , respectively.

Qiu, Lee BST 401

Limit of a sequence of real numbers (IV)
For a sequence (an ), If its upper limit equals its lower limit
(lim supn an = lim infn an = a∗ ), then (an ) converges to a.
The distance function which quantifies error is important.
For real numbers, there is essentially one way to measure
the error term: |ai − a∗ |. This is because a distance
function needs to satisfy several axioms (use wikipedia).
For a sequence of n-dimensional points(vectors), the
natural way to measure the error term is the Euclidean
distance. But other distance functions do exist, such as the
Manhattan distance (google it). Fortunately, a sequence of
vectors is convergent in one distance implies it is
convergent in all other distances.
Unfortunately, you can define quite a few non-compatible
distances of random numbers. So there are many different
convergences of random numbers.
Qiu, Lee BST 401
Limit of a sequence of real numbers (IV)
For a sequence (an ), If its upper limit equals its lower limit
(lim supn an = lim infn an = a∗ ), then (an ) converges to a.
The distance function which quantifies error is important.
For real numbers, there is essentially one way to measure
the error term: |ai − a∗ |. This is because a distance
function needs to satisfy several axioms (use wikipedia).
For a sequence of n-dimensional points(vectors), the
natural way to measure the error term is the Euclidean
distance. But other distance functions do exist, such as the
Manhattan distance (google it). Fortunately, a sequence of
vectors is convergent in one distance implies it is
convergent in all other distances.
Unfortunately, you can define quite a few non-compatible
distances of random numbers. So there are many different
convergences of random numbers.
Qiu, Lee BST 401
Limit of a sequence of real numbers (IV)
For a sequence (an ), If its upper limit equals its lower limit
(lim supn an = lim infn an = a∗ ), then (an ) converges to a.
The distance function which quantifies error is important.
For real numbers, there is essentially one way to measure
the error term: |ai − a∗ |. This is because a distance
function needs to satisfy several axioms (use wikipedia).
For a sequence of n-dimensional points(vectors), the
natural way to measure the error term is the Euclidean
distance. But other distance functions do exist, such as the
Manhattan distance (google it). Fortunately, a sequence of
vectors is convergent in one distance implies it is
convergent in all other distances.
Unfortunately, you can define quite a few non-compatible
distances of random numbers. So there are many different
convergences of random numbers.
Qiu, Lee BST 401
Limit of a sequence of real numbers (IV)
For a sequence (an ), If its upper limit equals its lower limit
(lim supn an = lim infn an = a∗ ), then (an ) converges to a.
The distance function which quantifies error is important.
For real numbers, there is essentially one way to measure
the error term: |ai − a∗ |. This is because a distance
function needs to satisfy several axioms (use wikipedia).
For a sequence of n-dimensional points(vectors), the
natural way to measure the error term is the Euclidean
distance. But other distance functions do exist, such as the
Manhattan distance (google it). Fortunately, a sequence of
vectors is convergent in one distance implies it is
convergent in all other distances.
Unfortunately, you can define quite a few non-compatible
distances of random numbers. So there are many different
convergences of random numbers.
Qiu, Lee BST 401
Limit of a function

We can now define the limit of a function f (x) as x

approaches x0 . x0 could be ±∞, for the sake of simplicity
we assume x0 is finite for the following definition.
limx→x0 f (x) = y ∗ if and only if

∀ > 0, ∃δ > 0 such that dist(f (x)−y ∗ ) <  for all x ∈ Ball(x0 , δ).

The logic negation of a sequence/function converges to a

value is that this sequence/function “breaks the precision
rule” infinitely often. More precisely, a sequence is not
convergent if for a given  > 0, we have

dist(ai , a∗ ) >  i.o.

where i.o. stands for infinitely often.

Qiu, Lee BST 401

Limit of a function

We can now define the limit of a function f (x) as x

approaches x0 . x0 could be ±∞, for the sake of simplicity
we assume x0 is finite for the following definition.
limx→x0 f (x) = y ∗ if and only if

∀ > 0, ∃δ > 0 such that dist(f (x)−y ∗ ) <  for all x ∈ Ball(x0 , δ).

The logic negation of a sequence/function converges to a

value is that this sequence/function “breaks the precision
rule” infinitely often. More precisely, a sequence is not
convergent if for a given  > 0, we have

dist(ai , a∗ ) >  i.o.

where i.o. stands for infinitely often.

Qiu, Lee BST 401

Limit of a function

We can now define the limit of a function f (x) as x

approaches x0 . x0 could be ±∞, for the sake of simplicity
we assume x0 is finite for the following definition.
limx→x0 f (x) = y ∗ if and only if

∀ > 0, ∃δ > 0 such that dist(f (x)−y ∗ ) <  for all x ∈ Ball(x0 , δ).

The logic negation of a sequence/function converges to a

value is that this sequence/function “breaks the precision
rule” infinitely often. More precisely, a sequence is not
convergent if for a given  > 0, we have

dist(ai , a∗ ) >  i.o.

where i.o. stands for infinitely often.

Qiu, Lee BST 401