MATH 2209: Probability

Table of Contents
Distributions ............................................................................................................................................................................................. 3
Discrete .......................................................................................................................................................... 3
Binomial X~Bin(n,p) .............................................................................................................................................................3
Poisson X~Poi() ..................................................................................................................................................................3

Continuous ..................................................................................................................................................... 4
Uniform U~(a,b) ..................................................................................................................................................................4
Exponential X~Exp() ...........................................................................................................................................................4
Gaussian X~N(,2) ..............................................................................................................................................................5

Joint ............................................................................................................................................................................................................. 5
Discrete .......................................................................................................................................................... 5
Discrete Joint PMF ...............................................................................................................................................................5
Finding Discrete Marginal PMF ............................................................................................................................................5
Discrete Joint CDF................................................................................................................................................................5
Stating CDF ..........................................................................................................................................................................5

Continuous ..................................................................................................................................................... 6
Continuous Joint PDF ...........................................................................................................................................................6
Continuous Marginal PDF ....................................................................................................................................................6
Continuous Joint CDF...........................................................................................................................................................6

Proving Joint CDF ............................................................................................................................................ 6

Conditional Probability .................................................................................................................................... 7
Discrete ...............................................................................................................................................................................7
Continuous ..........................................................................................................................................................................7
Multiplication Rule ..............................................................................................................................................................7
Discrete: ..............................................................................................................................................................................7
Continuous: .........................................................................................................................................................................7

Independence ................................................................................................................................................. 7
Discrete Independence ........................................................................................................................................................7
Continuous Independence ...................................................................................................................................................7

Expectancy................................................................................................................................................................................................. 8
Discrete .......................................................................................................................................................... 8
Continuous ..................................................................................................................................................... 8
Symmetry Theorem ......................................................................................................................................... 8
Covariance and Correlation ............................................................................................................................. 8
Covariance ..........................................................................................................................................................................8
Correlation ..........................................................................................................................................................................9
Independence .....................................................................................................................................................................9

Properties of Covariance and Correlation ......................................................................................................... 9

Properties of Expectation ................................................................................................................................ 9
Functions of Single Variables ............................................................................................................................................................... 9
Definition........................................................................................................................................................ 9
Determining SY ..................................................................................................................................................................10

CDF of functions of Single Variables ............................................................................................................... 10

PDF of functions of Single Variables ............................................................................................................... 10
Process to determine pdf ..................................................................................................................................................11
Corollary: Special Case.......................................................................................................................................................11
Determining PDF via Direct Approach ................................................................................................................................11

E(X) in Single Variable Functions .................................................................................................................... 11

Independence ............................................................................................................................................... 11
Multivariable Functions ...................................................................................................................................................................... 12
Multivariate CDF ........................................................................................................................................... 12
Process of finding multivariate CDF ...................................................................................................................................12

Multivariate PDF ........................................................................................................................................... 12

Callum Biggs 20498066

MATH 2209: Calculus

pg. 1

Independence between X and Y ..................................................................................................................... 12

Sums of Two Variables .................................................................................................................................. 13
Moment Generating Functions ......................................................................................................................................................... 13
Definition...................................................................................................................................................... 13
Finding the MGF ................................................................................................................................................................14
Finding E(Xn) ......................................................................................................................................................................14
Tabular ..............................................................................................................................................................................14
Case 1 .......................................................................................................................................................................................................... 14
Case 2 .......................................................................................................................................................................................................... 14

Uniqueness ................................................................................................................................................... 14
Single Variable .............................................................................................................................................. 15
MGF for Functions of Single Variables ...............................................................................................................................15
Corollary ..................................................................................................................................................................................................... 15

PDF Convolutions ..............................................................................................................................................................15

Summation of Independent Random Variables......................................................................................................................................... 15
Corollary: IID Random Variables............................................................................................................................................................... 15

Multivariate .................................................................................................................................................. 16

Callum Biggs 20498066

MATH 2209: Calculus

pg. 2

Distributions
Discrete
Discrete Distributions take on finite values
Probabilities are specified by a pmf
Discrete functions are given in a table form
= =

= =

= 2 2

Binomial X~Bin(n,p)
Models the number of success in a series of independent events, each with a probability of success p
Parameters:
n = number of trials
p = probability of success

= ( = ) =
1 , , 1

! !
= 1 +
=
= (1 )

Poisson X~Poi()
Probability of events occurring in a fixed time period occurs with a known average rate and are independent of
the time since the last event
Parameter: Average rate is
For the probability of exactly n occurrences, with an average of in the time frame
=

= exp{ 1 }
=
=

Callum Biggs 20498066

MATH 2209: Calculus

pg. 3

Continuous
Continuous functions are those in which there are infinite outcomes
For a pdf ()

= =

= =

= 2 2

Uniform U~(a,b)
Parameter: Uniform on (a,b)
1
= ,
0,
otherwise
1,

=
,
<

0,
<

=

1
= ( + )
2
1
=

12

Exponential X~Exp()
Parameter: Describes events between Poisson process, i.e. continuous and independent
,
0
=
0,
otherwise

= 1 ,
0,

= 1
=

0
otherwise

1
2

Callum Biggs 20498066

MATH 2209: Calculus

pg. 4

Gaussian X~N(,2)
Parameter: Mean , Variance 2
1
2
=
exp
2 2
2 2

1
2 2

exp

2 2

1
= exp{ 2 2 }
2
=
= 2

Stating CDF
1. If is bounded between two real values , , () is three piecewise,
= 0 if < , = if < < , = 1 if >
2. If is bounded from a real value a and infinity, only two parts are given, = 0 if < , and
= if >
3. If across only the function needs to be given

Joint
Discrete
Discrete Joint PMF
Discrete Joints are given as a table
= , = = ,
Satisfying the following conditions
i , 0 for all pairs ,
ii

(, ) = 1

Therefore to check if a joint discrete probability function exists, the above conditions must be met
Marginal PMF
=

(, )

Finding Discrete Marginal PMF

To find for ()
1. Sum up all the joint probabilities for X=k, for all the possible values that Y=l can take
2. The summation is your answer
3. To find the complete mpmf, must perform this for each value of k, and tabulate the results.
Answer is in form of table!

Discrete Joint CDF

, =

(, ) , (, )

Note: The use of dummy variables , . These must be used, or great confusion will occur
Callum Biggs 20498066

MATH 2209: Calculus

pg. 5

Continuous
Continuous Joint PDF
Continuous Joint PDF specifies the distributions of probabilities over a space of all pairs (, ) of
possible values of two random variables X and Y
The domain of a joint pdf can be taken as the plane
Continuous PDFs are draw in 3D, with x axis, y axis, and axis
For I and J, intervals on , plane,
, =

Such that the following

i , 0 for all pairs ,
ii

, = 1

Therefore to check if a joint continuous probability function exists, the above conditions must be met

Continuous Marginal PDF

=

Note: and match up with and

Continuous Joint CDF

, =

, , (, )

Once again note the use of dummy variables , . These must be used, or great confusion will occur
Applications of the Joint CDF
, =

2
,

< , < = , , , + (, )

Proving Joint CDF

To prove the existence of a joint CDF, you must prove that the function meets the following properties
, = ( , )
i) = lim , ,

= lim ,

ii) lim , = 1
,

iii)

lim , = 0 ,

lim , = 0

iv) is and from the right in each variable

Callum Biggs 20498066

MATH 2209: Calculus

pg. 6

Conditional Probability
Discrete
| , =

,
,

Continuous
| =

,
,

Multiplication Rule
Discrete:
|= =

| |

| | ()

Note: If X and Y have joint discrete distribution,

=

Continuous:
|= =

|= ()

Where and are the set of all values x and y can take

Independence
Discrete Independence
, =
| = ()

Continuous Independence
, =
| (|) = ()

Callum Biggs 20498066

MATH 2209: Calculus

pg. 7

Expectancy
Discrete
= =

Provided that the RHS converges, i.e.

<

Continuous

= =

Provided that the RHS converges, i.e.

<

Symmetry Theorem
Some functions cannot be integrated to determine the expected value. This occurs if they are not convergent,
and will result in integrals approaching infinity. These problems are solved by using the symmetry theorem.

Symmetry Theorem
If X has a pdf , which is symmetrical around 0 , in the sense that
0 + = 0 ,
for all
And if () exists then = 0
i.e. if symmetrical, point of symmetry is the mean

Covariance and Correlation

Covariance
Suppose = +
var =
=
=
Now defining
cov , =

2
+ 2
2 +

+ 2

Discrete Case

, ,

Continuous Case

cov , =
Therefore
var = var + var 2cov ,
cov , =
() is the expected value of the joint function
Callum Biggs 20498066

MATH 2209: Calculus

pg. 8

Correlation
The correlation coefficient between X and Y is a measure of the degree of linear dependence between X and Y.
The coefficient between two random variables X and Y is defined as
cov ,
=

=

Independence
If X and Y are independent then
cov , = 0
= 0
However, the converse is not necessarily true, cov , = 0, DOES NOT MEAN X and Y are independent
If cov , = 0, i.e. X and Y are uncorrelated, it follows that
var = var + var

Properties of Covariance and Correlation

1. If = + and = + where , , , are constants, then
, = ,
=
2. If = + , where and are constants, then
=
, =
=
3. For any random variables X and Y, it follows that 1 1
Extreme values of = are only obtained if X and Y are linearly related,
i.e. = + , with a being positive or negative respectively for the two cases

It follows that

var = 2 var + 2 var 2cov ,

Properties of Expectation
1.
2.
3.
4.
5.

If = = 1 then =
If = then = ()
If = + then = +
If = + then = +
If = + then = +

Functions of Single Variables

Definition
Let X be a random variable and be a real valued function of a real variable.
Define the random variable Y as
= ()
Let denote the space of all possible values of , this is the support of the distribution of
o is of continuous type, = : > 0
o is of discrete type = : > 0
Then any point in is mapped by the random variable function to the real variable ,
which is then mapped by to
Also, if = (), then
= ( )
Callum Biggs 20498066

MATH 2209: Calculus

pg. 9

Determining SY
1.
2.
3.
4.
5.

Determine in terms of
Determine 1 =
Replace with 1 ()
Manipulate until just in terms of
A number-line helps

CDF of functions of Single Variables

If is a function of
=
=
To find the CDF, () must be defined
By creating 1 () it is easy to rearrange ( ) to be in terms of X, allowing to determine () using x. This
leads to two unique cases

1. is strictly increasing
Results in X less than 1 ()

= 1
1 ()

Note:
1 () goes on the top
The integral does not have to go from , but from the lower limits of ()
u is used as a dummy variable to avoid confusion,

2. is strictly decreasing
Results in X greater then 1

= 1

1 ()

Note:
1 () goes on the bottom
The integral does not have to go to , but to the upper limits of ()
u is used as a dummy variable to avoid confusion,

PDF of functions of Single Variables

If the following conditions are met
is a random variable of continuous type
is a one-to-one mapping of = : > 0
1 is differentiable on = ( )), with a continuous derivative
1 ()
1

,

=

0,
otherwise
Note: Absolute value of derivative is used, this accounts for any negatives which would screw up everything
Proof: Consider definition of CDF, particularly starting with ( ), then derive the CDF

Callum Biggs 20498066

MATH 2209: Calculus

pg. 10

Process to determine pdf

1.
2.
3.
4.

Check if conditions are met

Consider
Define = ( )
Determine 1 (), rearrange in terms of

5. Determine

, answer should be positive

6. Follow formula above

Corollary: Special Case

If = +

=
,

As 1 =

, and

1 ()

1
||

Determining PDF via Direct Approach

1.
2.
3.
4.
5.

State =
Determine 1 ()
State = ,
= ( ), replace with 1 ()
State = ( ), and using where g is increase/decreasing, determine
the appropriate integral
6. State cdf, =, note do appropriately with the correct amount of piecewise

7. = ()

E(X) in Single Variable Functions

Obviously if =

Continous

Discrete

, , ,

Joint

Independence
If two variables are independent
, = 1 2
1 , 2

Callum Biggs 20498066

= 1 2

MATH 2209: Calculus

pg. 11

Multivariable Functions
Multivariate CDF
Suppose that a transformation , (, ), is invertible s.t. the random variables Z and
= , ,
= (, )
And and have solutions
= , ,
= (, )
If the inverse transforms are differentiable then let a Jacobian equal

, =


Then
1 , 1 =

, , ,

, ,

, 2

1 1

Where the region of integration is 1 1 =

, 2 : 1 , 1

Process of finding multivariate CDF

1.
2.
3.
4.
5.

Write down the functions = (, ) and = (, )

Determine the solutions = , , = (, )
Determine , , hence determine , ,
Determine and , using X and Y, careful consideration must be made!
Calculate 1 , 1
1 , 1 =

, , ,

, ,

, 2

Multivariate PDF
From multivariate CDF, assuming the transformation has continuous first order partial derivatives
, = ( , , , ,
For , in the range of the transformation and zero otherwise

Independence between X and Y

1 =

Derived by combining independent joint with Sum of two variables

Notice for the CDF that w is replaced with y

Callum Biggs 20498066

MATH 2209: Calculus

pg. 12

Sums of Two Variables

This method is a direct route to determine the distribution of the function , s.t. = +
Consider the transformation
, ,
= + ,
=
The inverse of this transform yields
= ,
=
The Jacobian of this inverse
1 1
=1
0 1
Hence for 1 and 1 =

, 2 : 1
1 =

, 1
1 1
1

(1 ) =

And, differentiating reveals the pdf

1 =

, ,

So it can be seen that the introduction of a transform and another variable allows for easier pathway
to determining the marginal.
Remember to check the domain for , this could result in avoiding integrals to and from

Moment Generating Functions

Definition

The mgf of a random variable X is essentially the Laplace transform of the pdf of
In fact = =
One reason for working with mgfs is to avoid the nuisance of the minus signs which result in Laplace transform
equivalents

The mgf of a random variable is defined as

=
This definition occurs whenever the expectation is finite for all t in some non-degenerating interval,
including the origin. E.g. some interval , .
Note: = =

The name arises because the expansion of leads, in suitable cases (allowing interchange of
expectation and infinite sum), to the expansion
2
3
= 1 + + 2 + ( 2 )
2!
3!

2
3
In which the moments of X are , ( ), , appear, with ( ) as the coefficient of ! , this
allows the moments of X to be obtained via differentiation of ()

=0

Callum Biggs 20498066

MATH 2209: Calculus

pg. 13

Finding the MGF

1. Define ()

2. =

3. Simple evaluate this, the key is to consider the domain of ()

Finding E(Xn)
1. Define ()

2. =

3. Check exists in an open interval around = 0

4. =
, =

=0

Tabular
Case 1
f(x) differentiates to zero eventually,
g(x) continually differentiates

Case 2
f(x) continually differentiates,
g(x) continually differentiates
f(x) Derivative
g(x) Integration
+()
()
()
()
+ ()
()
() = +

()

Uniqueness
There exists a 1-1 correspondence between mgfs and probability distributions from which an mgf exists
This means that for every element in the distribution corresponds to exactly one element in the
mgf
o i.e. Mapping between mgfs and probability distributions is a bijection
This means that for functions of random variables, mgfs are able to be obtained directly, rather
than through determination of the cdf or pdf

Callum Biggs 20498066

MATH 2209: Calculus

pg. 14

Single Variable
MGF for Functions of Single Variables
For a random variable Y that is a function of a random variable X
=
=

Proof:
For a fixes real , consider a new variable =
The expectation of , from the Single Variable chapter

Let = ()

= ()

Corollary
If = +
= ()
This corollary is very useful for transformations to Gaussian Distributions

PDF Convolutions
= 1 , 2 , ,
= 1 ,2 ,,

Summation of Independent Random Variables

(Important Theorem, essentially a corollary of the above theorem, but stated separately due to significance)

If = 1 + 2 + + , where 1 , 2 , , are independent variables, with respective mgfs

1 , 2 , ,
= 1 2
Summations of independent random variables translates into simple pointwise multiplication of
corresponding mgfs

Corollary: IID Random Variables

If = 1 + 2 + + , where 1 , 2 , , are iid (independent and identically distributed) random
variables with (common) mgf
=
Gamma Distribution Example

Callum Biggs 20498066

MATH 2209: Calculus

pg. 15

Multivariate
It is simple to extend all the previous content into multivariate form by considering the random
column vector = 1 , 2 , ,

=
Whenever the expectation exists for any in some non-degenerate box in that contains the origin,
i.e. a rectangle in 2

Considering that = 1 1 + 2 2 ++ ) , that by letting some = 0, the mgfs of any

component of , or the joint mgf of any sub collection of can be obtained.
Example
Let = 5, letting 1 = 2 = 4 = 0 will give 3 5 (3 , 5 ), joint mgf

This can be extended, via manipulation of the vector to obtain the mgf of functions of multiple
random variables
Example
Moreover, the expression for = 31 23 , in = 5
Then = where = 3, 0, 2, 0,0

Callum Biggs 20498066

MATH 2209: Calculus

pg. 16