Newsom 1

USP 656 Multilevel Regression

Winter 2006
yfuufulfulglg and Fixed:
Variables, Effects, and Coefficients
The terms random and i!ed are used re"uentl# in the hierar$hi$al linear modeling literature% The
distin$tion is a dii$ult one to &egin with and &e$omes more $onusing &e$ause the terms are used to
reer to dierent $ir$umstan$es% 'ere are some summar# $omments that ma# hel(%
Random and Fixed Variables
) i!ed varia&le is one that is assumed to &e measured without error% *t is also assumed that the
values o a i!ed varia&le in one stud# are the same as the values o the i!ed varia&le in another
stud#% Random varia&les are assumed to &e values that are drawn rom a larger (o(ulation o values
and thus will re(resent them% +ou $an thin, o the values o random varia&les as re(resenting a
random sam(le o all (ossi&le values o that varia&le% Thus- we e!(e$t to generali.e the results
o&tained with a random varia&le to all other (ossi&le values o that random varia&le% Most o the time
in )N/0) and regression anal#sis we assume the inde(endent varia&les are i!ed%
Random and Fixed Effects
The terms random and i!ed are used in the $onte!t o )N/0) and regression models- and reer to
a $ertain t#(e o statisti$al model% )lmost alwa#s- resear$hers use i!ed ee$ts regression or )N/0)
and the# are rarel# a$ed with a situation involving random ee$ts anal#ses% ) i!ed ee$ts )N/0)
reers to assum(tions a&out the inde(endent varia&le and the error distri&ution or the varia&le% )n
e!(erimental design is the easiest e!am(le or illustrating the (rin$i(al% Usuall#- the resear$her is
interested in onl# generali.ing the results to e!(erimental values used in the stud#% 1or instan$e- a
drug stud# using 0 mg- 5 mg- or 20 mg o an e!(erimental drug% This is when a i!ed ee$ts )N/0)
would &e a((ro(riate% *n this $ase- the e!tra(olation is to other studies or treatments that might use
the same values o the drug 3i%e%- 0 mg- 5 mg- and 20 mg4% 'owever- i the resear$her wants to ma,e
ineren$es &e#ond the (arti$ular values o the inde(endent varia&le used in the stud#- a random
ee$ts model is used% ) $ommon e!am(le would &e the use o (u&li$ art wor,s re(resenting low-
moderate- and high a&stra$tness 3e%g%- statue o a war hero vs% a (ivoting geometri$ design4% The
resear$her would li,e to ma,e ineren$es &e#ond the three (ie$es used- so the art (ie$es are
$on$e(tuali.ed as (ie$es randoml# drawn rom a larger universe o (ossi&le (ie$es and the
ineren$es are made to a larger universe o art wor, and range o a&stra$tness values% Su$h a
generali.ation is more o an inerential lea(- and- $onse"uentl#- the random ee$ts model is less
(owerul% Random ee$ts models are sometimes reerred to as Model ** or varian$e $om(onent
models% )nal#ses using &oth i!ed and random ee$ts are $alled mi!ed models%
Fixed and Random Coefficients in Multilevel Regression
The random vs% i!ed distin$tion or varia&les and ee$ts is im(ortant in multilevel regression% *n
multilevel regression models- &oth level52 and level52 predictors are assumed to &e i!ed% 'owever-
level52 intercepts and slopes are t#(i$all# assumed to var# randoml# a$ross grou(s% 6e$ause o the
assum(tions a&out their error distri&utions- we $all their varian$es- oo=var(U0j) and 11=var(U1j),
random $oei$ients%
This means that we $an thin, a&out 0j and 1j as a,in to the random variables * des$ri&ed
a&ove% *nstead o attem(ting to generali.e &e#ond the (arti$ular values o the inde(endent varia&le-
we are attem(ting to generali.e &e#ond the (arti$ular grou(s in the stud#% 1or instan$e- we ma# have
50 $om(anies- &ut we wish to generali.e to a larger universe o $om(anies when we e!amine the
means 3inter$e(ts4 or the !5# relationshi( 3slo(es4%
*n '7M- varian$es or the inter$e(ts and slo(es are estimated &# deault% That is- when #ou
irst $onstru$t a model- U0j and U1j are estimated &# deault% 'owever- the resear$her has the $hoi$e
o setting U0j and U1j to &e .ero% 6# setting them to .ero- we are testing a model in whi$h we assume
0j and 1j do not var# randoml# a$ross grou(s% *n a$t- their varian$e is assumed to &e .ero- so the#
Newsom 2
USP 656 Multilevel Regression
Winter 2006
are assumed to &e $onstant or nonvar#ing a$ross grou(s% 1or e!am(le- i!ed- nonvar#ing inter$e(ts
would im(l# the grou( average or the de(endent varia&le is assumed to &e e"ual in ea$h grou(%
)lthough we t#(i$all# reer to this $onstraint as i!ing the inter$e(ts or i!ing the slo(es- the term is
somewhat loosel# a((lied- &e$ause we are reall# assuming the# are i!ed and nonvarying%
Intercept Model and Random Effects A!VA
*n the sim(lest '7M model- there are no (redi$tors% We have one level52 e"uation and one level52
e"uation8
0 ij j ij
Y R = +
where Y is the de(endent varia&le- 0j is the inter$e(t- and Rij is the residual or error% 0j $an &e
thought o as the mean o ea$h grou(% The level52 e"uation also has no (redi$tors in it9s sim(lest
orm8
0 00 0 j j
U = +
where 0j is the de(endent varia&le- 00 is the level52 inter$e(t- and U0j is the level52 error% *n this
e"uation- 00 re(resents the grand mean or the mean o the inter$e(ts% U0j re(resents the deviation o
ea$h mean rom the grand mean% When the average deviation is large- there are large grou(
dieren$es%
*t $an &e shown that this is reall# the )N/0) model% )N/0) e!amines the deviations o grou( means
rom the grand mean% 6e$ause we assume that the grou( means- re(resented &# 0j and- thus- their
deviations are randoml# var#ing- this model is e"uivalent to the random effects )N/0) model% We
$an rewrite our two '7M e"uations as a single e"uation- i we (lug in the level52 terms into level52
e"uation%
00 0 ij j ij
Y U R = + +
*n this e"uation- 00 re(resents the grand mean- the U0j term re(resents the deviations rom the mean
3or mean dieren$es or treatment ee$t- i #ou would li,e4- and Rij re(resents the within5grou( error or
variation% *n more mathemati$all# oriented )N/0) te!t&oo,s- #ou will see an )N/0) model written
something li,e8
ij j ij
Y e = + +
* the model is an random ee$ts )N/0)- j is assumed to have a random distri&ution% *n multilevel
regression- &e$ause we assume U0j to var# randoml#- the sim(le '7M model with no level52 or level52
(redi$tors is e"uivalent to the random ee$ts )N/0) model%
When we add (redi$tors to the level52 e"uation- the# are $ovariates and the model &e$omes a
random ee$ts )N:/0) in whi$h the means are ad;usted or the $ovariate%
Newsom 3
USP 656 Multilevel Regression
Winter 2006
"ummary #able
Random vs$
Fixed
%efinition Example &se in Multilevel
Regression
Variables Random variable8 324 is assumed to &e measured with
measurement error% The s$ores are a un$tion o a true
s$ore and random error< 324 the values $ome rom and
are intended to generali.e to a mu$h larger (o(ulation
o (ossi&le values with a $ertain (ro&a&ilit# distri&ution
3e%g%- normal distri&ution4< 3=4 the num&er o values in
the stud# is small relative to the values o the varia&le as
it a((ears in the (o(ulation it is drawn rom% Fixed
variable8 324 assumed to &e measured without
measurement error< 324 desired generali.ation to
(o(ulation or other studies is to the same values< 3=4
the varia&le used in the stud# $ontains all or most o the
varia&le9s values in the (o(ulation%
*t is im(ortant to distinguish &etween a varia&le that is
varying and a varia&le that is random% ) i!ed varia&le
$an have dierent values- it is not ne$essaril# invariant
3e"ual4 a$ross grou(s%
Random variable:
(hotogra(hs
re(resenting individuals
with diering levels o
attra$tiveness
mani(ulated in an
e!(eriment- $ensus
tra$,s
Fixed variable:
gender- ra$e- or
intervention vs% $ontrol
grou(%
Predi$tor
varia&les in M7R
generall#
assumed to &e
i!ed
Effects Random effect8 324 dierent statisti$al model o
regression or )N/0) model whi$h assumes that an
inde(endent varia&le is random< 324 generall# used i
the levels o the inde(endent varia&le are thought to &e
a small su&set o the (ossi&le values whi$h one wishes
to generali.e to< 3=4 will (ro&a&l# (rodu$e larger
standard errors 3less (owerul4% Fixed effect8 324
statisti$al model t#(i$all# used in regression and )N/0)
assuming inde(endent varia&le is i!ed< 324
generali.ation o the results a((l# to similar values o
inde(endent varia&le in the (o(ulation or in other
studies< 3=4 will (ro&a&l# (rodu$e smaller standard
errors 3more (owerul4%
Random effect8
random ee$ts )N/0)-
random ee$ts
regression
Fixed effect8 i!ed
ee$ts )N/0)- i!ed
ee$ts regression
*nter$e(t onl#
models in M7R
are e"uivalent to
random ee$ts
)N/0) or
)N:/0)%
Coefficients Random coefficient: term a((lies onl# to M7R
anal#ses in whi$h inter$e(ts- slo(es- and varian$es $an
&e assumed to &e random% M7R anal#ses most
t#(i$all# assume random $oei$ients% /ne $an
$on$e(tuali.e the $oei$ients o&tained rom the level52
regressions as a t#(e o random varia&le whi$h $omes
rom and generali.es to a distri&ution o (ossi&le values%
>rou(s are $on$eived o as a su&set o the (ossi&le
grou(s%
Fixed coefficient: a $oei$ient $an &e i!ed to &e non5
var#ing 3invariant4 a$ross grou(s &# setting its &etween
grou( varian$e to .ero%
Random $oei$ients must &e varia&le a$ross grou(s%
:on$e(tuall#- i!ed $oei$ients ma# &e invariant or
var#ing a$ross grou(s%
Random coefficient8
the level52 (redi$tor-
average in$ome- is
used to (redi$t s$hool
(erorman$e in ea$h
s$hool% *nter$e(t
values or s$hool
(erorman$e are
assumed to &e a
sam(le o the inter$e(ts
rom a larger (o(ulation
o s$hools%
Fixed coefficient8
slo(es or inter$e(ts
$onstrained to &e e"ual
over dierent s$hools%
6oth used in
M7R% Slo(es and
inter$e(t values
$an &e
$onsidered to &e
i!ed or random-
de(ending on
resear$hers
assum(tions and
how the model is
s(e$iied% The
varian$e o the
slo(es or
inter$e(ts are
$onsidered
random
$oei$ients%