Generalized Linear Models

Generalized Linear Models (GLM)

General class of linear models that are made up of 3

components: Random, Systematic, and Link unction
!
Random component: "dentifies dependent #aria$le
(Y) and its pro$a$ility distri$ution
!
Systematic %omponent: "dentifies the set of
e&planatory #aria$les (X
'
,(((,X
k
)
!
Link unction: "dentifies a function of the mean
that is a linear function of the e&planatory
#aria$les
k k
X X g + + + =
1 1
) (
Random %omponent

%onditionally Normally distri$uted response )ith constant

standard de#iation * Re+ression models )e ha#e fit so far(

,inary outcomes (Success or ailure)* Random

component has Binomial distri$ution and model is called
Logistic Regression(

%ount data (num$er of e#ents in fi&ed area and-or len+th

of time)* Random component has Poisson distri$ution and
model is called Poisson Regression

.hen %ount data ha#e /(0) 1 2(0), model fit can $e

Negative Binomial Regression

%ontinuous data )ith ske)ed distri$ution and #ariation

that increases )ith the mean can $e modeled )ith a
Gamma distri$ution
%ommon Link unctions

"dentity link (form used in normal and gamma

re+ression models):

Lo+ link (used )hen cannot $e ne+ati#e as

)hen data are Poisson counts):

Lo+it link (used )hen is $ounded $et)een 3 and

' as )hen data are $inary):
= ) ( g
) log( ) ( = g

1
log ) ( g
Lo+istic Re+ression

Lo+istic Re+ression * 4ichotomous Response

#aria$le and numeric and-or cate+orical e&planatory
#aria$le(s)
!
Goal: Model the pro$a$ility of a particular outcome as a
function of the predictor #aria$le(s)
!
5ro$lem: 5ro$a$ilities are $ounded $et)een 3 and '

4istri$ution of Responses: ,inomial

Link unction:

1
log ) ( g
Lo+istic Re+ression )ith ' 5redictor

Response - Presence/Absence of characteristic

Predictor - Numeric variable observed for each case

Model - (x) Probabilit of presence at predictor level x

x
x
e
e
x
1 !
1 !
1
) (

+
+
+
=

1
" ! P(Presence) is the same at each level of x

1
# ! P(Presence) increases as x increases

1
$ ! P(Presence) decreases as x increases
Lo+istic Re+ression )ith ' 5redictor

0
,
1
are unkno)n parameters and must $e estimated
usin+ statistical soft)are such as S5SS, S6S, R or
S7676 (or in a matri& lan+ua+e)
8
5rimary interest in estimatin+ and testin+ hypotheses
re+ardin+
1
8
Lar+e*Sample test (.ald 7est):
8 H
3
:
1
9 3 H
6
:
1
3
) ( %
% & &
% & &
' '
'
1 (
'
'
)
1
)
'
1
)
obs
obs
obs
X P val P
X R R
X S T

:ote: Some soft)are packa+es

perform this as an e;ui#alent <*
test or t*test
=dds Ratio

"nterpretation of Re+ression %oefficient ():

!
"n linear re+ression, the slope coefficient is the chan+e in the
mean response as x increases $y ' unit
! "n lo+istic re+ression, )e can sho) that:

= =
+
) ( 1
) (
) (
) (
) 1 (
x
x
x odds e
x odds
x odds

*hus e

represents the change in the odds of the outcome

(multiplicativel) b increasing x b 1 unit

+f " !( the odds and probabilit are the same at all x levels (e

"1)

+f # ! ( the odds and probabilit increase as x increases (e

#1)

+f $ ! ( the odds and probabilit decrease as x increases (e

$1)
>?@ %onfidence "nter#al for =dds Ratio
Step ': %onstruct a >?@ %" for :

+
) ) )
) ) ) ) ) )
,- & 1 ( ,- & 1 ,- & 1

.tep '% Raise e " '&/10 to the lo1er and upper bounds of the 2+%

+
)
) )
)
) )
,- & 1 ,- & 1
(


e e

+f entire interval is above 1( conclude positive association

+f entire interval is belo1 1( conclude negative association

+f interval contains 1( cannot conclude there is an association

Multiple Lo+istic Re+ression
2&tension to more than one predictor #aria$le (either numeric or
dummy #aria$les)(
.ith k predictors, the model is )ritten:
k k
k k
x x
x x
e
e

+ + +
+ + +
+
=

1 1 !
1 1 !
1
Ad3usted 4dds ratio for raising x
i
b 1 unit( holding
all other predictors constant%
i
e OR
i

=

Man models have nominal/ordinal predictors( and

1idel ma5e use of dumm variables
7estin+ Re+ression %oefficients

7estin+ the o#erall model:

) (
& &
)) log( ' ( )) log( ' ( & &
! all Not %
! %
' '
'
(
'
1 !
'
1 !
obs
k obs
obs
i A
k
X P P
X R R
L L X S T
H
H
=

=
=
= = =

L
!
, L
1
are values of the ma6imi7ed li5elihood function( computed b
statistical soft1are pac5ages& *his logic can also be used to compare
full and reduced models based on subsets of predictors& *esting for
individual terms is done as in model 1ith a single predictor&
5oisson Re+ression

Generally used to model %ount data

4istri$ution: 5oisson (Restriction: 2(0)9/(0))

Link unction: %an $e identity link, $ut typically use the

lo+ link:

( )
k k
X X
k
k k
e X X
X X g

+ + +
=
+ + + = =
&&&
1
1 1 !
1 1 !
(&&&(
&&& ) ln( ) (
7ests are conducted as in Lo+istic re+ression
.hen the mean and #ariance are not e;ual (o#er*dispersion), often replace the
5oisson 4istri$ution replaced )ith :e+ati#e ,inomial 4istri$ution