Mathematical Statistics (MA212M)

Lecture Slides
Lecture 21
Univariate Normal Distribution

Def: A continuous random variable X is said to have a univarite

normal distribution if the PDF of X is given by
1 1 x−µ 2
f (x) = √ e − 2 ( σ ) for all x ∈ R,
σ 2π
where µ ∈ R and σ > 0.
Notation: X ∼ N(µ, σ 2 ).
Theorem: If X ∼ N(µ, σ 2 ), all moments of X exist. In particular,
E (X ) and Var (X ) exist, and they are given by E (X ) = µ and
Var (X ) = σ 2 .
Remark: This means that a normal distribution is completely specified
by its mean and variance.
Bivariate Normal

Def: A two dimensional random vector X = (X1 , X2 ) is said to have

a bivariate normal distribution if aX1 + bX2 is a univariate normal for
all (a, b) ∈ R2 \ (0, 0).
Remark: If X has bivariate normal distribution, then each of X1 and
X2 is univariate normal. Hence E (X1 ), E (X2 ), Var (X1 ), Var (X2 ), and
Cov (X1 , X2 ) exist.
   
µ1 σ11 σ12
Notation: µ = E (X ) = and Σ = Var (X ) = .
µ2 σ21 σ22
Theorem: Let X be a bivariate normal random vector. If µ = E (X )
and Σ = Var (X ), then for any fixed u = (a, b) ∈ R2 \ (0, 0),
u 0 X ∼ N(u 0 µ, u 0 Σu).
Bivariate Normal

Theorem: Let X be a bivariate normal random vector, then

0 1 0
MX (t) = e t µ+ 2 t Σt for all t ∈ R2 .
Remark: Thus the bivariate normal distribution is completely specified
by the mean vector µ and the variance-covariance matrix Σ.
Notation: X ∼ N2 (µ, Σ).
Corollary: If X ∼ N2 (µ, Σ), then X1 ∼ N(µ1 , σ11 ) and
X2 ∼ N(µ2 , σ22 ).
Remark: The converse of the above theorem is not true.
Remark: If X ∼ N2 (µ, Σ) and Cov (X1 , X2 ) = 0, then X1 and X2 are
independent.
Probability Density Function

Theorem: Let X ∼ N2 (µ, Σ) be such that Σ is invertible, then, for

all x ∈ R2 , X has a joint PDF given by
 
1 1 0 −1
f (x) = exp − (x − µ) Σ (x − µ)
2π|Σ|1/2 2
1
= p e A(x1 , x2 , µ1 , µ2 , σ1 , σ2 , ρ) ,
2πσ1 σ2 1 − ρ 2

√ √
where σ1 = σ11 , σ2 = σ22 , ρ is correlation coefficient between X1
and X2 , and
( 2     2 )
1 x1 − µ1 x1 − µ1 x2 − µ2 x2 − µ2
A=− − 2ρ + .
2(1 − ρ2 ) σ1 σ1 σ2 σ2
Conditional Probability Density Function

Theorem: Let X ∼ N2 (µ, Σ) be such that Σ is invertible, then

1 for all x2 ∈ R, the conditional PDF of X1 given X2 = x2 is given
by
" 2 #
1 x1 − µ1|2

1
fX1 |X2 (x1 |x2 ) = √ exp − for x1 ∈ R,
σ1|2 2π 2 σ1|2

where µ1|2 = µ1 + ρ σσ21 (x2 − µ2 ) and σ1|2

2
= σ12 (1 − ρ2 ).
 
2
2 X1 |X2 = x2 ∼ N µ1|2 , σ1|2 .
3 E (X1 |X2 = x2 ) = µ1|2 = µ1 + ρ σσ21 (x2 − µ2 ) for all x2 ∈ R.
2
4 Var (X1 |X2 = x2 ) = σ1|2 = σ12 (1 − ρ2 ) for all x2 ∈ R. Hence the
conditional varinace does not depend on x2 .