0niveisity of Califoinia, Neiceu, Cognitive anu Infoimation Sciences
750)'#&+ 8&'0 9: "#$%&' ()*%+#$4
Lineai mouels anu lineai mixeu mouels aie an impiessively poweiful anu flexible tool foi unueistanuing the woilu. This tutoiial is a ueciueuly conceptual intiouuction to this class of mouels. The focus is on !"#$%&'("#)"* what these mouels aie uoing . anu then we'll spenu most of the time (++,-)"* this unueistanuing, using the R statistical piogiamming enviionment. The iuea is to bootstiap youi knowleuge as quickly as possible so that you can stait with youi own analyses anu then tuin to moie technical texts if neeueu. The examples that I uiaw fiom come fiom linguistics anu phonetics, but you neeu not be a linguist to be able to follow this tutoiial.
You'll neeu about 2 houis to complete the full tutoiial (maybe a bit moie). Each pait takes about 1 houi.
So, what uoes the lineai mouel uo. Assume you knew nothing about males anu females, anu you weie inteiesteu in whethei the voice pitch of males anu females uiffeis, anu if so, by how much.
So you take a bunch of males anu a bunch of females, anu ask them to say a single woiu, say "mama", anu you measuie the iespective voice pitches. Youi uata might look something like this:
2
;5<=%/0 ;%- >)#/%?8#0/3 1 female 2SS Bz 2 female 2u4 Bz S female 242 Bz 4 male 1Su Bz S male 112 Bz 6 male 142 Bz
"Bz" (Beitz) is a measuie of pitch wheie highei values mean highei pitch.
You might look at this table anu say that it's quite obvious that females have highei voice pitch than females. Aftei all, the female values seem to be about 1uu Bz above the male ones.
But, in fact, it coulu be the case that females anu males have the same pitch, anu you weie just unlucky anu happeneu to choose some exceptionally high-pitcheu females anu some exceptionally low-pitcheu males. Intuitively, the pattein in the table seems pietty stiaightfoiwaiu, but we might want a moie piecise estimate of the uiffeience between males anu females, anu we might also want an estimate about how likely (oi unlikely) that uiffeience in voice pitch coulu have aiisen just because of uiawing an unlucky sample.
This is wheie the lineai mouel comes in. In this case, its task is to give you some values about voice pitch foi males anu females. as well as some piobability value as to how likely those values aie.
The basic iuea is to expiess youi ielationship of inteiest (in this case, the one between sex anu voice pitch) as a simple foimula. such as this one:
pitch ~ sex
This ieaus "pitch pieuicteu by sex" oi "pitch as a function of sex". Some people call the thing on the left the "uepenuent vaiiable" (the thing you measuie) anu the thing on the iight the "inuepenuent vaiiable". 0theis call the thing on the iight the "explanatoiy vaiiable" (this sounus too causal to me) oi the "pieuictoi". I'll call it "fixeu effect", anu this teiminology will make sense latei on in tutoiial 2.
Now, the woilu isn't peifect. Things aien't quite as ueteiministic as the above foimula suggests. Pitch is not "#$%&'('&) ueteimineu by sex, but also by a bunch of uiffeient factois such as language, uialect, peisonality, age anu what not. Even if we measuieu all of these factois, theie woulu still be othei factois influencing pitch that we cannot contiol foi. Peihaps, a subject in youi uata hau a hangovei on the moining of the iecoiuing (causing the voice to be lowei than usual), oi the subject was just moie neivous on that paiticulai uay (causing the voice to be highei). We can nevei measuie anu contiol all of these things. The woilu is full of S stuff that is outsiue the puiview of oui little expeiiment. Bence, let's upuate oui foimula to captuie the existence of these "ianuom" factois.
pitch ~ sex + c
This "c" (ieau "epsilon") is an eiioi teim. It stanus foi all of the things that affect pitch that aie not sex, all of the stuff that - fiom the peispective of oui expeiiment - is ianuom oi uncontiollable.
The foimula above is a schematic uepiction of the lineai mouel that we'ie going to builu. Note that the pait of the foimula on the iight-hanu siue conceptually uiviues the woilu into stuff that you can unueistanu (the "fixeu effect" sex) anu stuff that you can't unueistanu (the ianuom pait "c"). You coulu call the foimei the "stiuctuial" oi "systematic" pait of youi mouel anu the lattei the "ianuom" oi "piobabilistic" pait of the mouel.
!"#$%&'# )*)+,-%). /)01% %0"+02 0.k., let's move to R, the statistical piogiamming enviionment that we'll use foi the iest of this tutoiial 1 . Let's cieate the uataset that we'll use foi oui analysis. Type in:
pitch = c(233,204,242,130,112,142) sex = c(rep("female",3),rep("male",3))
The fiist line concatenates oui 6 uata points fiom above anu saves it in an object that we nameu pitch. The seconu line iepeats the woiu "female" S times anu then the woiu "male" S times . anu concatenates these 6 woius into an object that we nameu sex.
Foi a bettei oveiview, let's combine these two objects into a uata fiame:
my.df = data.frame(sex,pitch)
Now we have a uata fiame object that we nameu my.df, anu if you type that, you'll see this:
1 You uon't have R. Bon't woiiy, it's fiee anu woiks on all platfoims. You can get it heie: http:www.i-pioject.oig You might want to ieau a quick intio to R befoie you pioceeu - but even if you uon't, you'll be able to follow eveiything. }ust type in eveiything you see in uaik blue. 4
0.k., now we'll pioceeu with the lineai mouel. We take oui foimula above anu feeu it into the lm() function . except that we omit the "c" teim, because the lineai mouel function uoesn't neeu you to specify this.
xmdl = lm(pitch ~ sex, my.df)
We moueleu pitch as a function of sex, taken fiom the uata fiame my.df . anu we saveu this mouel into an object that we nameu xmdl. To see what the lineai mouel uiu, we have to "summaiize" this object using the function summary():
summary(xmdl)
If you uo this, you shoulu see this:
Lots of stuff heie. Fiist, you'ie being ieminueu of the mouel foimula that you enteieu. Then, the mouel gives you the iesiuuals (what this is will be uiscusseu latei), anu the coefficients of the fixeu effects (again, explanations follow. beai with me foi a moment). Then, the output piints some oveiall iesults of the mouel that you constiucteu.
S We have to woik thiough this output. Let's stait with "Nultiple R-Squaieu". This iefeis to the statistic * + which is a measuie of "vaiiance explaineu" oi if you piefei less causal language, it is a measuie of "vaiiance accounteu foi". * + values iange fiom u to 1. 0ui * + is u.921, which is quite high . you can inteipiet this as showing that 92.1% of the stuff that's happening in oui uataset is "explaineu" by oui mouel. In this case, because we have only one thing in oui mouel uoing the explaining (the fixeu effect "sex"), the * + ieflects how much of oui uata is accounteu foi by uiffeiences between males anu females.
In geneial, you want * + values to be high, but what is consiueieu a high * + value uepenus on youi fielu anu on youi phenomenon of stuuy. If the system you stuuy is veiy ueteiministic, * + values can be appioach 1. But in most of biology anu the social sciences, wheie we stuuy complex oi messy systems that aie affecteu by a whole bunch of uiffeient phenomena, we fiequently ueal with much lowei * +
values.
The "Aujusteu R-squaieu" value is a slightly uiffeient * + value that not only looks at how much vaiiance is "explaineu", but also at how many fixeu effects you useu to uo the explaining. So, in the case of oui mouel above, the two values aie quite similai to each othei, but in some cases the aujusteu * + ,-. can be much lowei if you have a lot of fixeu effects (say, you also useu age, psychological tiaits, uialect etc. to pieuict pitch).
So much foi * + . Next line uown you see the thing that eveiybouy is ciazy foi: Youi statistical test of "significance". If you've alieauy uone ieseaich, youi eyes will piobably immeuiately jump to the p-value, which in many fielus is youi ticket foi publishing youi woik. Theie's a little bit of an obsession with p-values . anu even though they aie iegaiueu as so impoitant, they aie quite often misunueistoou! So what exactly uoes the p-value mean heie.
0ne way to phiase it is to say that ,//0$123 )#04 $#-'& 1/ -#123 2#(51236 (5' %4#7,71&1() #8 )#04 -,(, 1/ 4'&,(19'&) &#: (because the p-value is small in this case). Technically speaking, the p-value is a /0"#)')0"(, +%01(1),)'-, it is a piobability 02-'4 (5' "#2-1(1#2 (5,( (5' 20&& 5)%#(5'/1/ 1/ (40'. In this case, the null hypothesis is "sex has no effect on pitch". Anu, the lineai mouel shows that if this hypothesis is tiue, then the uata woulu be quite unlikely. This is then inteipieteu as showing that the alteinative hypothesis "sex affects pitch" is moie likely anu hence that youi iesult is "statistically significant".
0sually, howevei, you have to uistinguish between the significance of the oveiall mouel (the p-value at the veiy bottom of the output), which consiueis all effects togethei, fiom the p-value of inuiviuual coefficients (which you finu in the coefficients table above the oveiall significance). We'll talk moie about this in a bit.
6 Then comes the F-value anu the uegiees of fieeuom. Foi an explanation of this, see my tutoiial on AN0vAs anu the logic behinu the F-test (http:bouowintei.comtutoiialbw_anova_geneial.puf). Foi a geneial lineai mouel analysis, you piobably neeu this value to iepoit youi iesults. If you wanteu to say that youi iesult is "significant", you woulu have to wiite something like this:
"We constiucteu a lineai mouel of pitch as a function of sex. This mouel was significant (F(1,4)=46.61, p<u.u1). (.)"
Now, let's look at the coefficient table. Beie it is again:
Note that the p-value foi the oveiall mouel was p=u.uu24u7, which is the same as the p-value on the iight-hanu siue of the coefficients table in the iow that staits with "sexmale". This is because youi mouel hau only one fixeu effect (namely, "sex") anu so the significance of the oveiall mouel is the same as the significance foi this coefficient. If you hau multiple fixeu effects, then the significance of the oveiall mouel anu the significance of this coefficient will be uiffeient. That is because the significance of the oveiall mouel takes all fixeu effects (all explanatoiy vaiiables) into account wheieas the coefficients table looks at each fixeu effect inuiviuually.
But why uoes it say "sexmale" iathei than just "sex", which is how we nameu oui fixeu effect. Anu wheie uiu the females go. If you look at the estimate in the iow that staits with "(Inteicept)", you'll see that the value is 226.SS Bz. This looks like it coulu be the estimateu mean of the female voice pitches. If you type the following.
mean(my.df[my.df$sex=="female",]$pitch)
. you'll get the mean of female voice pitch values, anu you'll see that this value is veiy similai to the estimate value in the "(Inteicept)" column.
Next, note that the estimate foi "sexmale" is negative. If you subtiact the estimate in the fiist iow fiom the seconu, you'll get 128, which is the mean of the male voice pitches (you can veiify that by iepeating the above commanu anu exchanging "male" foi "female").
To sum up, the estimate foi "(Inteicept)" is the estimate foi the female categoiy, anu the estimate foi "sexmale" is the estimate foi the -188'4'2"' between the 7 females anu the male categoiy. This may seem like a veiy iounuabout way of showing a uiffeience between two categoiies, so let's unpack this fuithei.
Inteinally, lineai mouels like to think in lines. So heie's a pictuie of the way the lineai mouel sees youi uata:
The lineai mouel imagines the uiffeience between males anu females as a slope. So, to go "fiom females to males", you have to go uown -98.SS . which is exactly the coefficient that we've seen above. The inteinal cooiuinate system looks like this:
Females aie sitting at the x-cooiuinate zeio at the y-inteicept (the point wheie the line ciosses the y-axis), anu males aie sitting at the x-cooiuinate 1. So now, the output makes a hella moie sense to us:
0 5 0 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0 female male V o i c e
p i t c h
( H z ) 0 5 0 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0 female male V o i c e
p i t c h
( H z ) 8
The females aie hiuuen behinu this mysteiious "(Inteicept)" anu the estimate foi that inteicept is the estimate foi female voice pitch! Then, the uiffeience between females anu males is expiesseu as a slope. "going uown" by 98.SS. The p-values to the iight of this table coiiesponu to tests whethei each coefficient is "non- zeio". 0bviously, 226.SS Bz is uiffeient fiom zeio, so the inteicept is "significant" with a veiy low p-value. The slope -98.SS is also uiffeient fiom zeio (but in the negative uiiection), anu so this is significant as well.
You might ask youiself: Why uiu the mouel choose females to be the inteicept iathei than males. Anu what is the basis foi choosing one iefeience level ovei the othei. The lm() function simply takes whatevei comes fiist in the alphabet! "f" comes befoie "m", making "females" the inteicept at x=u anu "males" the slope of going fiom u to 1.
It might not appeai stiaightfoiwaiu to you why we can expiess categoiical uiffeiences (heie, between men anu women) as a slope. The ieason why this woiks is because the uiffeience between two categoiies is exactly coiielateu with the slope between two categoiies. The following figuies will help you iealize this fact. In those pictuies, I incieaseu the uistance between two categoiies . anu exactly piopoitional to this inciease in uistance, the slope incieaseu as well.
What's the big auvantage of thinking of the uiffeience between two categoiies as a line ciossing those two categoiies. Well, the big auvantage is that you can use 0 5 1 0 1 5 2 0 2 5 c(rep(0, 4), rep(1, 4)) c ( m y . y s ,
m y . y s
+
4 ) Slope: 4 Difference = 4 0 5 1 0 1 5 2 0 2 5 c(rep(0, 4), rep(1, 4)) c ( m y . y s ,
m y . y s
+
6 ) Slope: 6 Difference = 6 0 5 1 0 1 5 2 0 2 5 c(rep(0, 4), rep(1, 4)) c ( m y . y s ,
m y . y s
+
8 ) Slope: 8 Difference = 8 0 5 1 0 1 5 2 0 2 5 c(rep(0, 4), rep(1, 4)) c ( m y . y s ,
m y . y s
+
1 0 ) Slope: 10 Difference = 10 0 5 1 0 1 5 2 0 2 5 c(rep(0, 4), rep(1, 4)) c ( m y . y s ,
m y . y s
+
1 2 ) Slope: 12 Difference = 12 0 5 1 0 1 5 2 0 2 5 c(rep(0, 4), rep(1, 4)) c ( m y . y s ,
m y . y s
+
1 4 ) Slope: 14 Difference = 14 9 the same piinciple foi something that is not categoiical. So, if you hau a continuous factoi, say age, you coulu also fit a line. Eveiything woulu woik exactly the same. Let's tiy this out. Say you weie now inteiesteu in whethei age pieuicts voice pitch. The uata might look something like this:
;5<=%/0 @4% >)#/%?8#0/3 1 14 2S2 Bz 2 2S 244 Bz S SS 24u Bz 4 48 2SS Bz S S2 212 Bz 6 67 2u4 Bz
Anu heie's a scatteiplot of this uata:
0.k., same thing as befoie: We expiess this as a function, wheie oui "fixeu effect" is now "age".
0 20 40 60 80 1 8 0 2 0 0 2 2 0 2 4 0 2 6 0 2 8 0 V o i c e
p i t c h
( H z ) Age (years) 1u
Again, the significance of the inteicept is not veiy inteiesting. Remembei that the p-value in each iow is simply a test of whethei the coefficient to the left is significantly uiffeient fiom zeio. The inteicept (267.u76S) heie is the pieuicteu pitch value foi people with age u. This inteicept uoesn't make much sense because people who aie not boin yet uon't ieally have voice pitch.
What ieally inteiests us is "age", which emeiges as a significant "pieuictoi" of voice pitch. The way to ieau the output foi age ("-u.9u99") is that foi eveiy inciease of age by 1 you ueciease voice pitch by u.9u99 Beitz. Easy-peasy: just go one step to the iight in youi giaph (in youi unit of measuiement, heie: age in yeais) anu one step uown (in youi unit of measuiement, heie: voice pitch in Bz).
The scatteiplot below neatly summaiizes the mouel: The line iepiesents the mean that the mouel pieuicts foi people at age u, 1, 2, S etc. This is the line that iepiesents the coefficients of the mouel. It's woith looking at this pictuie anu compaiing it to the coefficients table above. See that the line at x=u is 267.u76S (oui inteicept), anu the slope is -u.9u99.
0 20 40 60 80 1 8 0 2 0 0 2 2 0 2 4 0 2 6 0 2 8 0 V o i c e
p i t c h
( H z ) Age (years) 11 3)"#-#4567 "#$ 8)"#-#47)%% -#0)+,)90% You might want to iemeuy the above-uiscusseu situation that the inteicept is meaningless. 0ne way of uoing this woulu be to simply subtiact the mean age fiom each age value, as is uone below:
Beie, we just cieateu a new column "age.c" that is the age vaiiable with the mean subtiacteu fiom it. This is the iesulting coefficient table fiom iunning a lineai mouel analysis of this "centeieu" uata:
Note that while the estimate has changeu fiom 267.u76S (pieuicteu voice pitch at age u) to 2Su.8SSS (pieuicteu voice pitch at aveiage age), the slope hasn't changeu anu neithei uiu the significance associateu with the slope oi the significance associateu with the full mouel. That is, you haven't messeu at all with the natuie of youi mouel, you just changeu the metiic so that the inteicept is now the mean voice pitch. So, via centeiing oui vaiiable we maue the inteicept moie meaningful.
:'-#4 '# Both of these examples have been aumitteuly simple. Bowevei, things easily "scale up" to moie complicateu stuff. Say, you measuieu two factois ("age" ,2- "sex") . you coulu put them in the same mouel. Youi foimula woulu then be:
pitch ~ sex + age + c
0i, you coulu auu uialect as an auuitional factoi:
pitch ~ uialect + sex + age + c
Anu so on anu so on. The only thing that changes is the following. The p-value at the bottom of the output will be the p-value foi the #9'4,&& $#-'&. This means that the p-value consiueis how well all of youi fixeu effects togethei help in accounting foi vaiiation in pitch. The coefficient output will then have p-values foi the inuiviuual fixeu effects.
12 This is what people sometimes call "multiple iegiession", wheie you mouel one iesponse vaiiable as a function of multiple pieuictoi vaiiables. The lineai mouel is just anothei woiu foi multiple iegiession.
;%%6890-'#%
Theie's a ieason why we call the lineai mouel a $#-'&. Like any othei mouel, the lineai mouel has (&&!2+')0"& . anu it's impoitant to talk about these assumptions. So heie's a whiilwinu toui thiough the conuitions that have to be satisfieu in oiuei foi the lineai mouel to be meaningful:
A9B "#$%&'#0C It's calleu "lineai mouel" foi a ieason! The thing to the left of oui simple foimula above has to be the iesult of a lineai combination of the things on the iight. If it uoesn't, the %$&)#!(, +,0' will inuicate some kinu of cuive, oi it will inuicate some othei pattein (e.g., two lines if you have categoiical binaiy uata).
We haven't talkeu about iesiuual plots yet, let alone iesiuuals. So, let's uo that! Bave a look at the pictuie below, which is a uepiction of the agepitch ielationship again:
The ieu lines inuicate the iesiuuals, which aie the ueviations of the obseiveu uata points fiom the pieuicteu values (the so-calleu "fitteu values"). In this case, the iesiuuals aie all faiily small . which is because the line that iepiesents the lineai mouel pieuicts oui uata veiy well, i.e., all points aie veiy close to the line.
To get a bettei view of the iesiuuals, you can take a snapshot of this giaph like this. 0 20 40 60 80 1 8 0 2 0 0 2 2 0 2 4 0 2 6 0 2 8 0 V o i c e
p i t c h
( H z ) Age (years) 1S
. anu iotate it ovei. So, you make a new plot wheie the line that the mouel pieuicts is now the centei line. Like heie:
This is a iesiuual plot. The fitteu values (the pieuicteu means) aie on the hoiizontal line (at y=u). The iesiuuals aie the veitical ueviations fiom this line. This view is just a iotation of the actual uata (compaie the iesiuual plot with the scatteiplot to see this). To constiuct the iesiuual plot foi youiself, simply type:
plot(fitted(xmdl),residuals(xmdl)) 2
2 Youi plot will have no cential line anu it will have uiffeient scales. It's woith spenuing some time on tweaking youi iesiuual plot anu making it pietty. in paiticulai, you shoulu make the plot 200 210 220 230 240 250 260 - 1 5 - 1 0 - 5 0 5 1 0 1 5 Fitted values R e s i d u a l s 14
In this case. theie isn't any obvious pattein in the iesiuuals. If theie :'4' a nonlineai oi cuivy pattein, then this woulu inuicate a violation of the lineaiity assumption. Beie's an example of a iesiuual plot that cleaily shows a violation of lineaiity:
What to uo if youi iesiuual plot inuicates nonlineaiity. Theie's seveial options:
You might miss an impoitant fixeu effect that inteiacts with whatevei fixeu effects you alieauy have in youi mouel. Potentially the pattein in the iesiuual plot goes away if this fixeu effect is auueu. Anothei (commonly chosen) option is to peifoim a nonlineai tiansfoimation of youi iesponse, e.g., by taking the log-tiansfoim. You can also peifoim a nonlineai tiansfoimation of youi fixeu effects. So, if age weie somehow ielateu to pitch in a 0-shapeu way (peihaps, if veiy young people hau high pitch anu veiy olu people hau high pitch, too, with inteimeuiate ages having a "uip" in pitch), then you coulu auu age anu age 2
(age-squaieu) as pieuictois. Finally, if you'ie seeing stiipes in youi iesiuual plot, then you'ie most likely uealing with some kinu of categoiical uata - anu you woulu neeu to tuin to a somewhat uiffeient class of mouels, such as logistic mouels.
so that theie's moie space aiounu the maigins. This will make any patteins easiei to see. Bave a look at some R giaphic tutoiials foi this. 200 210 220 230 240 250 260 - 1 5 - 1 0 - 5 0 5 1 0 1 5 Fitted values R e s i d u a l s 1S ADB @<,%$/% ). /)++#$%&'#0C When two fixeu effects (two pieuictois) aie coiielateu with each othei, they aie saiu to be /0,,)"$(%. Say, you weie inteiesteu in how aveiage talking speeu affects intelligence iatings (i.e., people who talk moie quickly aie iateu to be moie intelligent).
intelligence iatings ~ talking speeu
. anu you measuieu seveial uiffeient inuicatois of talking speeu, foi example, you syllables pei seconus, woius pei seconus anu sentences pei seconus. These uiffeient measuies aie going to be coiielateu with each othei because if you speak moie quickly, then you say moie syllables, woius anu sentences in a given amount of time. If you'u use all of these coiielateu pieuictois to pieuict intelligence iatings within the same mouel, you aie likely going to iun into a collineaiity pioblem.
If theie's collineaiity, the inteipietation of the mouel becomes unstable: Bepenuing on which one of coiielateu pieuictois is in the mouel, the fixeu effects become significant oi cease to be significant. Anu, the significance of these coiielateu oi collineai fixeu effects is not easily inteipietable, because they might steal each othei's "explanatoiy powei" (that's a veiy coaise way of saying what's actually going on, but you get the iuea).
Intuitively, this makes a lot of sense: If multiple pieuictois aie veiy similai to each othei, then it becomes veiy uifficult to ueciue what, in fact, is playing a big iole.
Bow to get iiu of collineaiity. Well fiist of all, you might pie-empt the pioblem in the uesign stage of youi stuuy anu focus on a few fixeu effects that you know aie not coiielateu with each othei. If you uiun't uo this anu you have seveial multiple measuies to choose fiom at the analysis stage of youi stuuy (e.g., thiee uiffeient ways of measuiing "talking speeu"), think about which one is the most meaningful anu uiop the otheis (be caieful heie: uon't base this uiopping uecision on the "significance"). Finally, you might want to consiuei uimension- ieuuction techniques such as Piincipal Component Analysis. These can tiansfoim seveial coiielateu vaiiables into a smallei set of vaiiables which you can then use as new fixeu effects.
16 AEB F)(),G%*&,0#/#0C H )' I&<,%$/% ). 3%0%'),G%*&,0#/#0CJ Being able to pionounce "heteioskeuasticity" seveial times in a iow in quick succession will make you a stai at youi next cocktail paity, so go aheau anu ieheaise pionouncing them now!
}okes asiue, homoskeuasticity is an extiemely impoitant assumption. It says that the vaiiability of youi uata shoulu be appioximately equal acioss the iange of youi pieuicteu values. If homosceuasticity is violateu, you enu up with heteioskeuasticity, oi, in othei woius, a pioblem with unequal vaiiances.
Foi the homosceuasticity assumption to be met, the %$&)#!(,& of youi mouel neeu to ioughly have a similai amount of ueviation fiom youi pieuicteu values. Again, we can check this by looking at a iesiuual plot. Beie's the one foi the agepitch uata again:
Theie's not ieally that many uata points to tell whethei this is ieally homosceuastic. In this case, I woulu concluue that theie's not enough uata to safely ueteimine whethei theie is oi isn't heteioskeuasticity. 0sually, I woulu constiuct mouels foi much laigei uata sets anyway.
So, heie's a plot that gives you an iuea of how a "goou" iesiuual plot looks with moie uata:
200 210 220 230 240 250 260 - 1 5 - 1 0 - 5 0 5 1 0 1 5 Fitted values R e s i d u a l s 17
Anu anothei one:
A goou iesiuual plot essentially looks blob-like. It's a goou iuea to geneiate some ianuom uata to see how a plot with ioughly equal vaiiances looks like. You can uo so using the following commanu line:
plot(rnorm(100),rnorm(100))
This cieates two sets of 1uu noimally uistiibuteu ianuom numbeis with a mean of u anu a stanuaiu ueviation of 1. If you type this in multiple times to cieate multiple plots, you can get a feel of how a "noimal" iesiuual plot shoulu look like.
The next iesiuual plot shows obvious heteioskeuasticity:
-3 -2 -1 0 1 2 3 - 3 - 2 - 1 0 1 2 3 Fitted values R e s i d u a l s -3 -2 -1 0 1 2 3 - 3 - 2 - 1 0 1 2 3 Fitted values R e s i d u a l s 18
In this plot, highei fitteu values have laigei iesiuuals . inuicating that the mouel is moie "off" with laigei pieuicteu means. So, the vaiiability is not homosceuastic: it's smallei in the lowei iange anu laigei in the highei iange.
What to uo. Again, tiansfoiming youi uata often helps. Consiuei a log-tiansfoim heie as well.
AKB L)'(&+#0C ). '%,#*5&+, The noimality of iesiuuals assumption is the one that is least impoitant. Inteiestingly, many people seem to think it is the most impoitant one, but it tuins out that lineai mouels aie ielatively iobust against violations of the assumptions of noimality. Reseaicheis uiffei with iespect to how much weight they put onto checking this assumption. Foi example, uellman anu Bill (2uu7), a famous book on lineai mouels anu mixeu mouels, uo not even iecommenu uiagnostics of the noimality assumption (ibiu. 46).
If you wanteu to test the assumption, how woulu you uo this. Eithei you make a histogiam of the iesiuuals of youi mouel, using.
hist(residuals(xmdl))
. oi a Q-Q plot .
qqnorm(residuals(xmdl)))
Beie's a iesiuual plot anu a Q-Q plot of the same uata next to each othei.
-3 -2 -1 0 1 2 3 - 3 - 2 - 1 0 1 2 3 Fitted values R e s i d u a l s 19
These look goou. The histogiam is ielatively bell-shapeu anu the Q-Q plot inuicates that the uata falls on a stiaight line (which means that it's similai to a noimal uistiibution). Beie, we woulu concluue that theie aie no obvious violations of the noimality assumption.
AMB @<,%$/% ). #$.+5%$0#&+ *&0& 6)#$0, Some people woulun't call "the absence of influential uata points" an ,//0$%(1#2 of the mouel. Bowevei, influential uata points can uiastically change the inteipietation of youi iesults, anu similai to collineaiity, it can leau to instable iesults.
Bow to check. Beie's a useful R function, dfbeta(), that you can use on a mouel object like oui xmdl fiom above.
Foi each coefficient of youi mouel (incluuing the inteicept), the function gives you the so-calleu BFbeta values. These aie the values with which the coefficients have to be aujusteu if a paiticulai uata point is excluueu (sometimes calleu "leave-one-out uiagnostics"). Noie concietely, let's look at the age column in the uata fiame above. The fiist iow means that the coefficient foi age (which, if you iemembei, was -u.9u99) has to be aujusteu by u.u64S7S7S if uata point 1 is excluueu. That means that the coefficient of the mouel without the uata point x F r e q u e n c y -3 -2 -1 0 1 2 0 5 1 0 1 5 2 0 -2 -1 0 1 2 - 2 - 1 0 1 2 Normal Q-Q Plot Theoretical Quantiles S a m p l e
Q u a n t i l e s 2u woulu be -u.97424S1 (which is -u.9u99 minus u.u64S7S7S. if the slope is negative, BFbeta values aie subtiacteu, if it's positive, they aie auueu).
Theie's a little bit of ioom foi inteipietation in what constitutes a laige oi a small BFbeta value. 0ne thing you can say foi suie: Any value that changes the sign of the slope is -'8121('&) an influential point that waiiants special attention. because excluuing that point woulu change the inteipietation of youi iesults. What I then uo is to eyeball the BFbetas anu look foi values that aie uiffeient by half of the absolute value of the slope. Say, my slope woulu be 2 . then a BFbeta value of 1 oi -1 woulu be alaiming to me. If it's a slope of -4, a BFbeta value of 2 oi -2 woulu be alaiming to me.
Bow to pioceeu if you have influential uata points. Well, it's uefinitely not legit to simply excluue those points anu iepoit only the iesults on the ieuuceu uata set. A bettei appioach woulu be to iun the analysis :1(5 the influential points anu then again :1(5#0( the influential points . then you can iepoit both analyses anu state whethei the inteipietation of the iesults uoes oi uoesn't change. The only case when it is o.k. to excluue influential points is when theie's an obvious eiioi with them, so foi example, a value that uoesn't make sense (e.g., negative age, negative height) oi a value that obviously is the iesult uue to a technical eiioi in the uata acquisition stage (e.g., voice pitch values of u). Influence uiagnostics allow you to spot those points anu you can then go back to the oiiginal uata anu see what went wiong S .
ANB O$*%6%$*%$/% PPPPPPP The inuepenuence assumption is by fai the most impoitant assumption of all statistical tests. In the lineai mouel analyses that we uiu so fai, each uata point came fiom a uiffeient subject. To ieminu you, heie's oui two uata sets that we woikeu on:
;05*C 9 ;05*C D ;5<=%/0 ;%- >)#/%?8#0/3 ;5<=%/0 @4% >)#/%?8#0/3 1 female 2SS Bz 1 14 2S2 Bz 2 female 2u4 Bz 2 2S 244 Bz S female 242 Bz S SS 24u Bz 4 male 1Su Bz 4 48 2SS Bz S male 112 Bz S S2 212 Bz 6 male 142 Bz 6 67 2u4 Bz
We weie able to iun the lineai mouel on this uata the way we uiu only because each iow in this uataset comes fiom a uiffeient subject. If you elicit multiple
S Foi an inteiesting back-anu-foith on a paiticulai example of how much influence uiagnostics anu extieme values can change the inteipietation of a stuuy, have a look at the uelightful episoue of acauemic bantei between 0lliich anu Schltei (2u11) anu Bianut (2u11). 21 iesponses fiom each subject, then those iesponses that come fiom the same subject cannot be iegaiueu as inuepenuent fiom each othei.
So, what exactly is inuepenuence. The iueal case is a coin flip oi the ioll of a uie: Each coin flip anu each ioll of a uie is absolutely inuepenuent fiom the outcome of the pieceuing coin flips oi uie iolls. The same shoulu holu foi youi uata points when you iun a lineai mouel analysis. So, the uata points shoulu come fiom uiffeient subjects. Anu each subject shoulu only contiibute one uata point.
When you violate the inuepenuence assumption, all hell bieaks loose. The othei assumptions that we mentioneu above aie impoitant as well, but the inuepenuence assumption is by fai the most impoitant one. violating inuepenuence may gieatly inflate youi chance of finuing a spuiious iesult anu it iesults in a p-value that is completely meaningless. 0nfoitunately, violations of the inuepenuence assumption aie quite fiequent in many bianches of science - so much in fact, that theie's a whole liteiatuie associateu with this violation, staiting fiom Builbeit (1984) foi ecology, Fieebeig anu Lucas (2uu9) foi psychology, Lazic (2u1u) foi neuioscience anu my own small papei foi phoneticsspeech science (Wintei, 2u11).
Bow can you guaiantee inuepenuence. Well, inuepenuence is a question of the expeiimental uesign in ielation to the statistical test that you use. Besign anu statistical analyses aie closely inteitwineu anu you can make suie that you meet the inuepenuence assumption by only collecting one uata point pei subject.
Now, a lot of the times, we want to collect moie uata pei subject, such as in iepeateu measuies uesigns. If you enu up a uata set that has non-inuepenuencies in it, you neeu to %$&0,3$ '4$&$ "0"5)"#$+$"#$"/)$& at the analysis stage. This is wheie 2)6$# 20#$,& come in hanuy. anu this is wheie we'll switch to the seconu tutoiial.
22 Part 2: A very basic tutorial for performing linear mixed effects analyses
This pait seives as a quick boot camp to jump-stait youi own analyses with lineai mixeu effects mouels. This text is uiffeient fiom othei intiouuctions by being ueciueuly "#2"'%(0,&; I will focus on :5) you want to use mixeu mouels anu 5#: you shoulu use them. While many intiouuctions to this topic can be veiy uaunting to ieaueis who lake the appiopiiate statistical backgiounu, this text is going to be a softei kinu of intiouuction. so, uon't panic!
Pait 2 will take you about 1 houi (possibly a bit moie).
Introduction: Fixed and random effects In tutoiial 1, we talkeu about how we coulu use the lineai mouel to expiess the ielationships in oui uata in teims of a function. In one example, we moueleu pitch as a function of age.
pitch ~ age + c
We calleu "age" a fixeu effect, anu c was oui "eiioi teim" to iepiesent the ueviations fiom oui pieuictions uue to "ianuom" factois that we cannot contiol expeiimentally. You coulu call this pait the "piobabilistic" oi "stochastic" pait of the mouel. Now, we'll unpack this "c" anu auu complexity to it. That is, we change the ianuom aspect of oui mouel, essentially leaving the systematic pait unchangeu. In mixeu mouels, eveiything in the "systematic" pait of youi mouel woiks just like with lineai mouels in tutoiial 1.
In one of my stuuies, we have been inteiesteu in the ielationship between pitch anu politeness (Wintei & uiawunuei, 2u12). So, essentially we'ie aiming foi a ielationship that looks like something like this:
pitch ~ politeness + c
In oui stuuy, politeness was tieateu as a categoiical factoi with two levels. a foimal iegistei anu an infoimal iegistei. 0n top of that, we also hau an auuitional fixeu effect, sex, anu so oui foimula looks moie like this:
pitch ~ politeness + sex + c
So fai so goou. Now things get a little moie complicateu. 0ui uesign was so that we took multiple measuies pei subject. That is, each subject gave multiple polite iesponses anu multiple infoimal iesponses. If we go back to the uiscussion of the assumptions of the lineai mouel in tutoiial 1, we can immeuiately see that this 2S woulu violate the inuepenuence assumption: Nultiple iesponses fiom the same subject cannot be iegaiueu as inuepenuent fiom each othei. Eveiy peison has a slightly uiffeient voice pitch, anu this is going to be an iuiosynciatic factoi that affects all iesponses fiom the same subject, thus ienueiing these uiffeient iesponses intei-uepenuent iathei than inuepenuent.
The way we'ie going to ueal with this situation is to auu a %("#02 $77$/' foi subject. This allows us to iesolve this non-inuepenuence by assuming a uiffeient "baseline" pitch value foi each subject. So, subject 1 may have a mean voice pitch of 2SS Bz acioss uiffeient utteiances, anu subject 2 may have a mean voice pitch of 21u Bz pei subject. Beie's a visual uepiction of how this looks like:
Subjects F1 to F9 aie female subjects. Subjects N1 to N7 aie male subjects. You immeuiately see that males have lowei voices than females (as is to be expecteu). But on top of that, within the male anu the female gioups, you see lots of inuiviuual vaiiation, with some people having ielatively highei values foi theii sex anu otheis having ielatively lowei values.
We can mouel these inuiviuual uiffeiences by assuming uiffeient %("#02 )"'$%/$+'& foi each subject. That is, each subject is assigneu a uiffeient inteicept value, anu the mixeu mouel estimates these inteicepts foi you.
Now you begin to see why the mixeu mouel is calleu a "mixeu" mouel. The lineai mouels that we consiueieu so fai have been "fixeu-effects-only" mouels that hau one oi moie fixeu effects anu a geneial eiioi teim "c". With the lineai mouel, we essentially uiviueu the woilu into things that we somehow unueistanu oi that aie somehow systematic (the fixeu effects, oi the explanatoiy vaiiables); anu things that we cannot contiol foi oi that we uon't unueistanu (c). But ciucially, this F1 F2 F3 F4 F5 F6 F7 F8 F9 M1 M2 M3 M4 M5 M6 M7 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0 3 5 0 4 0 0 M e a n
P i t c h
( H e r t z ) Subjects 24 lattei pait, the unsystematic pait of the mouel, uiu not have any inteiesting stiuctuie. We simply hau a geneial acioss-the-boaiu eiioi teim.
In the mixeu mouel, we auu one oi moie ianuom effects to oui fixeu effects. These ianuom effects essentially give stiuctuie to the eiioi teim "c". In the case of oui mouel heie, we auu a ianuom effect foi "subject", anu this chaiacteiizes iuiosynciatic vaiiation that is uue to inuiviuual uiffeiences.
The mixtuie of fixeu anu ianuom effects is what makes the mixeu mouel a mixeu mouel.
0ui upuateu foimula looks like this:
pitch ~ politeness + sex + (1|subject) + c
"(1|subject)" looks a little enigmatic. I'm alieauy using the R-typical notation foimat heie. What this is saying is "assume an inteicept that's uiffeient foi each subject" . anu "1" stanus foi the inteicept heie. You can think of this foimula as telling youi mouel that it shoulu expect that theie's going to be multiple iesponses pei subject, anu these iesponses will uepenu on each subject's baseline level. This effectively iesolves the non-inuepenuence that stems fiom having multiple iesponses by the same subject.
Note that the foimula still contains a geneial eiioi teim "c". This is necessaiy because even if we accounteu foi inuiviuual by-subject vaiiation, theie's still going to be "ianuom" uiffeiences between uiffeient utteiances fiom the same subject.
0.k., so fai so goou. But we'ie not uone yet. In the uesign that we useu in Wintei anu uiawunuei (2u12), theie's an auuitional souice of non-inuepenuence that neeus to be accounteu foi: We hau uiffeient items. 0ne item, foi example, was an "asking foi a favoi" scenaiio. Beie, subjects hau to imagine asking a piofessoi foi a favoi (polite conuition), oi asking a peei foi a favoi (infoimal conuition). Anothei item was an "excusing foi coming too late" scenaiio, which was similaily uiviueu between polite anu infoimal. In total, theie weie 7 such uiffeient items.
Similai to the case of by-subject vaiiation, we also expect by-item vaiiation. Foi example, theie might be something special about "excusing foi coming too late" which leaus to oveiall highei pitch (maybe because it's moie embaiiassing than asking foi a favoi), iegaiuless of the influence of politeness. Anu whatevei it is that makes one item uiffeient fiom anothei, the iesponses of the uiffeient subjects in oui expeiiment might similaily be affecteu by this ianuom factoi that is uue to item-specific iuiosynciasies. That is, if "excusing foi coming to late" leaus to high pitch (foi whatevei ieason), it's going to uo so foi subject 1, subject 2, subject S anu so on. Thus, the uiffeient iesponses to one item cannot be iegaiueu as inuepenuent, oi, in othei woius, theie's something similai to 2S multiple iesponses to the same item - even if they come fiom uiffeient people. Again, if we uiu not account foi these inteiuepenuencies, we woulu violate the inuepenuence assumption.
Beie's a visual iepiesentation of the by-item vaiiability:
The vaiiation between items isn't as big as the vaiiation between subjects - but theie aie still noticeable uiffeiences, anu we bettei account foi them in oui mouel!
We uo this by auuing an auuitional ianuom effect:
pitch ~ politeness + sex + (1|subject) + (1|item) + c
So, on top of uiffeient inteicepts foi uiffeient subjects, we now also have uiffeient inteicepts foi uiffeient items. We now "iesolveu" those non-inuepenuencies (oui mouel knows that theie aie multiple iesponses pei subject anu pei item), anu we accounteu foi by-subject anu by-item vaiiation in oveiall pitch levels.
Note the efficiency anu elegance of this mouel. Befoie, people useu to uo a lot of aveiaging. Foi example, in psycholinguistics, people woulu aveiage ovei items foi a subjects-analysis (each uata point comes fiom one subject, assuiing inuepenuence), anu then they woulu also aveiage ovei subjects foi an items- analysis (each uata point comes fiom one item). Theie's a whole liteiatuie on the auvantages anu uisauvantages of this appioach (Claik, 197S; Foistei & Bickinson, 1976; Wike & Chuich, 1976; Raaijmakeis, Schiijnemakeis, & uiemmen, 1999; Raaijmakeis, 2uuS; Lockei, Boffman, & Bovaiiu, 2uu7; Baayen, Baviuson, & Bates, 2uu8; Baii, Levy, Scheepeis, & Tilly, 2u1S). 1 2 3 4 5 6 7 1 0 0 1 5 0 2 0 0 2 5 0 3 0 0 3 5 0 4 0 0 M e a n
P i t c h
( H e r t z ) Items 26
The upshot is: while tiauitional analyses that uo aveiaging aie in piinciple legit, mixeu mouels give you much moie flexibility . anu they take the full uata into account. If you uo a subjects-analysis (aveiaging ovei items), you'ie essentially -1/4'3,4-123 by-item vaiiation. Conveisely, in the items-analysis, you'ie uisiegaiuing by-subject vaiiation. Nixeu mouels account foi both souices of vaiiation 12 , /123&' $#-'&. Neat, init.
Let's move on to R anu apply oui cuiient unueistanuing of the lineai mixeu effects mouel!!
Mixed models in R Foi a stait, we neeu to install the R package &$'; (Bates, Naechlei & Bolkei, 2u12). While being connecteu to the inteinet, open R anu type in:
install.packages(lme4)
Select a seivei close to you. Aftei installation, loau the &$'; package into R with the following commanu:
library(lme4)
Now, you have the function lmer() available to you, which is the mixeu mouel equivalent of the function lm() in tutoiial 1. This function is going to constiuct mixeu mouels foi us.
But fiist, we neeu some uata! I put a shoiteneu veision of the uataset that we useu foi Wintei anu uiawunuei (2u12) onto my seivei. You can loau it into R the following way:
Now, you have a uata fiame calleu politeness in youi R enviionment. You can familiaiize youiself with the uata by using head(), tail(), summary(), str(), colnames(). oi whatevei commanus you commonly use to get an oveiview of a uataset. Also, it is always goou to check foi missing values:
27 which(is.na(politeness)==T)
Appaiently, theie is a missing value in iow 26S. This is impoitant to know but foitunately, a few missing values pioviue no pioblems foi oui mixeu mouel analyses.
The uiffeience in politeness level is iepiesenteu in the column calleu "attituue". In that column, "pol" stanus foi polite anu "inf" foi infoimal. Sex is iepiesenteu as "F" anu "N" in the column "genuei". The uepenuent measuie is "fiequency", which is the voice pitch measuieu in Beitz (Bz). To ieminu you, highei values mean highei pitch.
The inteiesting ianuom effects foi us aie in the column "subject" anu "scenaiio", the lattei being the name of the item column (iemembei the uiffeient scenaiios like "asking foi a favoi".).
Let's look at the ielationship between politeness anu pitch by means of a boxplot:
What uo we see. In both cases, the meuian line (the thick line in the miuule of the boxplot) is lowei foi the polite than foi the infoimal conuition. Bowevei, theie may be a bit moie oveilap between the two politeness categoiies foi males than foi females.
. anu you will ietiieve an eiioi that shoulu look like this:
This is because the mouel 2''-/ a ianuom effect (aftei all, "mixing" fixeu anu ianuom effects is the point of mixeu mouels). We just specifieu a single fixeu effect, attituue, anu that was not enough.
So, let's auu ianuom inteicepts foi subjects anu items (iemembei that items aie calleu "scenaiios" heie):
The last commanu cieateu a mouel that useu the fixeu effect "attituue" (polite vs. infoimal) to pieuict voice pitch, contiolling foi by-subject anu by-item vaiiability. We saveu this mouel in the object politeness.model. To see what the iesult is, simply type in politeness.model to piint the output (in contiast to lm() you uon't neeu to use summary() to get this output).
This is the full output:
29 Again, let's woik thiough this: Fiist, the output ieminus you of the mouel that you fit. Then, theie's some geneial summaiy statistics such as Akaike's Infoimation Ciiteiion, the log-Likelihoou etc. We won't go into the meaning of these uiffeient values in this tutoiial because these aie conceptually a little bit moie involveu. Let's focus on the output foi the ianuom effects fiist:
Bave a look at the column stanuaiu ueviation. This is a measuie of the vaiiability foi each ianuom effect that you auueu to the mouel. You can see that scenaiio ("item") has much less vaiiability than subject. Baseu on oui boxplots fiom above, wheie we saw moie iuiosynciatic uiffeiences between subjects than between items, this is to be expecteu. Then, you see "Resiuual" which stanus foi the vaiiability that's not uue to eithei scenaiio oi subject. This is oui "c" again, the "ianuom" ueviations fiom the pieuicteu values that aie not uue to subjects anu items. Beie, this ieflects the fact that each anu eveiy utteiance has some factois that affect pitch that aie outsiue of the puiview of oui expeiiment.
The fixeu effects output miiiois the coefficient table that we consiueieu in tutoiial 1 when we talkeu about the iesults of oui lineai mouel analysis.
The coefficient "attituuepol" is the slope foi the categoiical effect of politeness. Ninus 19.69S means that to go fiom "infoimal" to "polite", you have to go uown - 19.69S Bz. In othei woius: pitch is lowei in polite speech than in infoimal speech, by about 2u Bz. Then, theie's a stanuaiu eiioi associateu with this slope, anu a t- value, which is simply the estimate (2u Bz) uiviueu by the stanuaiu eiioi (check this by peifoiming the calculation by hanu).
Note that the lmer() function (just like the lm() function in tutoiial 1) took whatevei comes fiist in the alphabet to be the iefeience level. "inf" comes befoie "pol", so the slope iepiesents the change fiom "inf" to "pol". If the iefeience categoiy woulu be "pol" iathei than "inf", the only thing that woulu change woulu be that the sign of the coefficient 19.69S woulu be positive. Stanuaiu eiiois, significance etc. woulu iemain the same.
Su Now, let's consiuei the inteicept. In tutoiial 1, we alieauy talkeu about the fact that oftentimes, mouel inteicepts aie not paiticulaily meaningful. But this inteicept is especially weiiu. It's 2u2.S88 Bz . wheie uoes that value come fiom.
If you look back at the boxplot that we constiucteu eailiei, you can see that the value 2u2.S88 Bz seems to fall halfway between males anu females - anu this is inueeu what this inteicept iepiesents. It's the aveiage of oui uata foi the infoimal conuition.
As we uiun't infoim oui mouel that theie's two sexes in oui uataset, the inteicept is paiticulaily off, in between the voice pitch of males anu females. This is just like the classic example of a faim with a uozen hens anu a uozen cows . wheie the mean legs of all faim animals consiueieu togethei is thiee, not a paiticulaily infoimative iepiesentation of what's going on at the faim.
We oveiwiote oui oiiginal mouel object politeness.model with this new mouel. Note that we auueu "genuei" as a fixeu effect because the ielationship between sex anu pitch is systematic anu pieuictable (i.e., we expect females to have highei pitch). This is uiffeient fiom the ianuom effects subject anu item, wheie the ielationship between these anu pitch is much moie unpieuictable anu "ianuom". We'll talk moie about the uistinction between fixeu anu ianuom effects latei.
Let's piint the mouel output again. Let's have a look at the iesiuuals fiist:
Note that compaieu to oui eailiei mouel without the fixeu effect genuei, the vaiiation that's associateu with the ianuom effect "subject" uioppeu consiueiably. This is because the vaiiation that's uue to genuei was confounueu with the vaiiation that's uue to subject. The mouel uiun't know about males anu females, anu so it's pieuictions weie ielatively moie off, cieating ielatively laigei iesiuuals.
Let's look at the coefficient table now:
S1
We see that males anu females uiffei by about 1u9 Bz. Anu the inteicept is now much highei (2S6.846 Bz), as it now iepiesents the female categoiy (foi the infoimal conuition). The coefficient foi the effect of attituue uiun't change much.
Statistical significance So fai, we haven't talkeu about significance yet. But, if you want to publish this, you'll most likely neeu to iepoit some kinu of p-value. 0nfoitunately, p-values foi mixeu mouels aien't as stiaightfoiwaiu as they aie foi the lineai mouel. Theie aie multiple appioaches, anu theie's a uiscussion suiiounuing these, with sometimes wiluly uiffeiing opinions about which appioach is the best. Beie, I focus on the Likelihoou Ratio Test as a means to attain p-values.
Likelihoou is the piobability of seeing the uata you collecteu given youi mouel. The logic of the likelihoou iatio test is to compaie the likelihoou of two mouels with each othei. Fiist, the mouel :1(5#0( the factoi that you'ie inteiesteu in (the null mouel), then the mouel :1(5 the factoi that you'ie inteiesteu in. Naybe an analogy helps you to wiap youi heau aiounu this: Say, you'ie a hikei, anu you caiiy a bunch of uiffeient things with you (e.g., a gallon of watei, a flashlight). To know whethei each item affects youi hiking speeu, you neeu to get iiu of it. So, you get iiu of the flashlight anu iun without it. Youi hiking speeu is not affecteu much. Then, you get iiu of the gallon of watei, anu you iealize that youi hiking speeu is affecteu a lot. You woulu concluue that caiiying a gallon of watei with you significantly affects youi hiking speeu wheieas caiiying a flashlight uoes not. Expiesseu in foimula, you woulu want to compaie the following two "mouels" (think "hikes") with each othei:
If theie is a significant uiffeience between "mul2" anu "mul1", then you know that the gallon of watei matteis. To assess the effect of the flashlight, you woulu have to uo a similai compaiison:
mul1 = hiking speeu ~ gallon of watei + flashlight mul2 = hiking speeu ~ gallon of watei
In both cases, we compaieu a full mouel (with the fixeu effects in question) against a ieuuceu mouel without the effects in question. In each case, we S2 concluue that a fixeu effect is significant if the uiffeience between the likelihoou of these two mouels is significant.
Beie's how you woulu uo this in R. Fiist, you neeu to constiuct the null mouel:
Note one auuitional technical uetail. I just auueu the aigument REML=FALSE. Bon't woiiy about it too much - but in case you'ie inteiesteu, this changes some inteinal stuff (in paiticulai, the likelihoou estimatoi), anu it is necessaiy to uo this when you compaie mouels using the likelihoou iatio test (Pinheiio & Bates, 2uuu; Bolkei et al., 2uu9).
Then, we ie-uo the full mouel above, this time also with REML=FALSE:
Now you have two mouels to compaie with each othei - one with the effect in question, one without the effect in question. We peifoim the likelihoou iatio test using the anova() function:
anova(politeness.null,politeness.model)
This is the iesulting output:
You'ie being ieminueu of the foimula of the two mouels that you'ie compaiing. Then, you finu a Chi-Squaie value, the associateu uegiees of fieeuom anu the p- value 4 .
4 You might wonuei why we'ie uoing a Chi-Squaie test heie. Theie's a lot of technical uetail heie, but the main thing is that theie's a theoiem, calleu Wilk's Theoiem, which states that negative two times the log likelihoou iatio of two mouels appioaches a Chi-Squaie uistiibution with uegiees of fieeuom of the numbei of paiameteis that uiffei between the mouels (in this case, only "attituue"). So, somebouy has uone a pioof of this anu you'ie goou to go! Bo note, also, that some people uon't like "stiaight-jacketing" likelihoou into the classical null-hypothesis significance testing fiamewoik that we'ie following heie, anu so they woulu uisagiee with the inteipietation of likelihoou the way we useu it in the likelihoou iatio test. SS
You woulu iepoit this iesult the following way:
". politeness affecteu pitch (! 2 (1)=11.62, p=u.uuu6S), loweiing it by about 19.7 Bz S.6 (stanuaiu eiiois) ."
If you'ie useu to t-tests, AN0vAs anu lineai mouel stuff, then this likelihoou- baseu appioach might seem weiiu to you. Rathei than getting a p-value stiaightfoiwaiuly fiom youi mouel, you get a p-value fiom a compaiison of two mouels. To help you get useu to the logic, iemembei the hikei anu the analogy of putting one piece of luggage away to estimate that piece's effect on hiking speeu.
Note that we kept the pieuictoi "genuei" in the mouel. The only change between the full mouel anu the null mouel that we compaieu in the likelihoou iatio test was the factoi of inteiest, politeness. In this paiticulai test, you can think of "genuei" as a contiol vaiiable anu of "attituue" as youi test vaiiable.
We coulu have also compaieu the following two mouels:
"mul.null" in this case is an inteicept only mouel, wheie you just estimate the mean of the uata. You coulu compaie this to "mul.full", which has two moie effects, "attituue" anu "genuei". If this uiffeience became significant, you woulu know that "mul.full" anu "mul.null" aie significantly uiffeient fiom each othei - but you woulu not know whethei this uiffeience is uue to "attituue" oi uue to "genuei". Coming back to the hikei analogy, it is as if you uioppeu both the gallon of watei anu the flashlight anu then you iealizeu that youi hiking speeu changeu, but you woulun't be able to ueteimine conclusively which one of the two pieces of luggage was the ciucial one.
A final thing iegaiuing likelihoou iatio tests: What happens if you have an inteiaction. We uiun't talk much about inteiactions yet, but say, you pieuicteu "attituue" to have an effect on pitch that is somehow mouulateu thiough "genuei". Foi example, it coulu be that speaking politely veisus infoimally has the opposite effect foi men anu women. 0i it coulu be that women show a uiffeience anu men uon't (oi vice veisa). If you have such an intei-uepenuence between two factois (calleu an inteiaction), you can test it the following way:
In R, inteiactions between two factois aie specifieu with a "*" iathei than a "+".
S4 If you compaie the above mouels in a likelihoou iatio test using the anova() function, then you woulu get a p-value that gives you the significance of the inteiaction. If this compaiison is significant, you know that attituue anu genuei aie significantly intei-uepenuent on each othei. If this is compaiison is not significant, theie is no significant intei-uepenuence.
It might be a goou iuea to tiy out uiffeient likelihoou compaiisons with the uata pioviueu above, say "attituue*genuei" veisus "attituue + genuei" veisus simply "1" (the inteicept only mouel). Remembei to always put REML=FALSE when cieating youi mouel.
Super-crucial: Random slopes versus random intercepts We'ie not uone yet. 0ne of the coolest things about mixeu mouels is coming up now, so hang on!!
Let's have a look at the coefficients of the mouel by subject anu by item:
coef(politeness.model)
Beie is the output:
You see that each scenaiio anu each subject is assigneu a uiffeient inteicept. That's what we woulu expect, given that we've tolu the mouel with "(1|subject)" anu "(1|scenaiio)" to take by-subject anu by-item vaiiability into account.
SS But not also that the fixeu effects (attituue anu genuei) aie all the same foi all subjects anu items. 0ui mouel is what is calleu a %("#02 )"'$%/$+' 20#$,. In this mouel, we account foi baseline-uiffeiences in pitch, but we assume that whatevei the effect of politeness is, it's going to be the same foi all subjects anu items.
But is that a valiu assumption. In fact, often times it's not - it is quite expecteu that some items woulu elicit moie oi less politeness. That is, the effect of politeness might be uiffeient foi uiffeient items. Likewise, the effect of politeness might be uiffeient foi uiffeient subjects. Foi example, it might be expecteu that some people aie moie polite, otheis less. So, what we neeu is a %("#02 &,0+$ mouel, wheie subjects anu items aie not only alloweu to have uiffeiing inteicepts, but wheie they aie also alloweu to have uiffeient slopes foi the effect of politeness. This is how we woulu uo this in R:
Note that the only thing that we changeu is the ianuom effects, which now look a little moie complicateu. The notation "(1+attituue|subject)" means that you tell the mouel to expect uiffeiing baseline-levels of fiequency (the inteicept, iepiesenteu by 1) as well as uiffeiing iesponses to the main factoi in question, which is "attituue" in this case. You then uo the same foi items.
Bave a look at the coefficients of this upuateu mouel by typing in the following:
coef(politeness.model)
Beie's a iepiint of the output that I got:
S6
Now, the column with the by-subject anu by-item coefficients foi the effect of politeness ("attituuepol") is uiffeient foi each subject anu item. Note, howevei, that it's always negative anu that many of the values aie quite similai to each othei. This means that uespite inuiviuual vaiiation, theie is also consistency in how politeness affects the voice: foi all of oui speakeis, the voice tenus to go uown when speaking politely, but foi some people it goes uown slightly moie so than foi otheis.
Bave a look at the column foi genuei. Beie, the coefficients uo no change. That is because we uiun't specify ianuom slopes foi the by-subject oi by-item effect of genuei.
0.k., let's tiy to obtain a p-value. We keep oui mouel fiom above (politeness.model) anu compaie it to a new null mouel in a likelihoou iatio test. Let's constiuct the null mouel fiist:
Note that the null mouel neeus to have the same ianuom effects stiuctuie. So, if youi full mouel is a ianuom slope mouel, youi null mouel also neeus to be a ianuom slope mouel.
Let's now uo the likelihoou iatio test:
anova(politeness.null,politeness.model) S7
This is, again, significant.
Theie aie a few impoitant things to say heie: You might ask youiself "Which ianuom slopes shoulu I specify." . oi even "Aie ianuom slopes necessaiy at all."
A lot of people constiuct ianuom inteicept-only mouels but conceptually, it makes a hella sense to incluue ianuom slopes most of the time. Aftei all, you can almost always expect that people uiffei with how they ieact to an expeiimental manipulation! Anu likewise, you can almost always expect that the effect of an expeiimental manipulation is not going to be the same foi all items.
Noieovei, ieseaicheis in ecology (Schielzeth & Foistmeiei, 2uu9), psycholinguistics (Baii, Levy, Scheepeis, & Tilly, 2u1S) anu othei fielus have shown via simulations that mixeu mouels without ianuom slopes aie anti- conseivative oi, in othei woius, they have a ielatively high Type I eiioi iate (they tenu to finu a lot of significant iesults which aie actually uue to chance).
Baii et al. (2u1S) iecommenu that you shoulu "keep it maximal" with iespect to youi ianuom effects stiuctuie, at least foi contiolleu expeiiments. This means that you incluue all ianuom slopes that aie justifieu by youi expeiimental uesign . anu you uo this foi all fixeu effects that aie impoitant foi the oveiall inteipietation of youi stuuy.
In the mouel above, oui whole stuuy ciucially iesteu on stating something about politeness. We weie not inteiesteu in genuei uiffeiences, but they aie well woith contiolling foi. This is why we hau ianuom slopes foi the effect of attituue (by subjects anu item) but not genuei. In othei woius, we only moueleu by-subject anu by-item vaiiability in how politeness affects pitch.
Assumptions In tutoiial 1, we talkeu a lot about the many uiffeient assumptions of the lineai mouel. The goou news is: Eveiything that we uiscusseu in the context of the lineai mouel applies stiaightfoiwaiuly to mixeu mouels. So, you also have to woiiy about collineaiity anu influential uata points. Anu you have to woiiy about homosceuasticity (anu potentially about lack of noimality). But you uon't have to leain much new stuff. The way you check these assumptions in R is exactly the same as in the case of the lineai mouel, say, by cieating a iesiuual plot, a histogiam of the iesiuuals oi a Q-Q plot.
Inuepenuence, being the most impoitant assumption, iequiies a special woiu: 0ne of the main ieasons we moveu to mixeu mouels iathei than just woiking with lineai mouels was to iesolve non-inuepenuencies in oui uata. Bowevei, mixeu mouels can still violate inuepenuence . if you'ie missing impoitant fixeu S8 oi ianuom effects. So, foi example, if we analyzeu oui uata with a mouel that uiun't incluue the ianuom effect "subject", then oui mouel woulu not "know" that theie aie multiple iesponses pei subject. This amounts to a violation of the inuepenuence assumption. So choose youi fixeu effects anu ianuom effects caiefully, anu always tiy to iesolve non-inuepenuencies.
Then, a woiu on influential uata points. You will finu that the function dfbeta() that we useu in the context of lineai mouels uoesn't woik foi mixeu mouels. If you woiiy about influential points, you can check out the package 128&0'2"'<=> (Nieuwenhuis, te uiotenhuis, & Pelzei, 2u12), oi you can piogiam a foi loop that uoes the leave-one-out uiagnostics by hanu. The following coue gives you an outline of the geneial stiuctuie of how you might want to uo this (you can check my "uoouling" tutoiials on loops anu piogiamming stiuctuies in R to get a bettei giasp of this):
all.res=numeric(nrow(mydataframe)) for(i in 1:nrow(mydataframe)){ myfullmodel=lmer(response~predictor+ (1+predictor|randomeffect),POP[-i,]) all.res[i]=fixef(myfullmodel)[some number] } 5
uo aheau anu play with checking the assumptions. You can go back to tutoiial 1 anu apply the coue in theie to the new mixeu mouel objects in this tutoiial.
A final note on random versus fixed effects I have evaueu a piecise uefinition of the uiffeience between fixeu anu ianuom effects. I uelibeiately uiu this because I wanteu you to get some expeiience with lineai mixeu effects mouels in R befoie we finally take a step back anu shaipen oui concepts.
S The basic iuea of this coue snippet is this: Pie-uefine a vectoi that has as many elements as you have iows in youi uataset. Then, cycle thiough each iow. Foi each iteiation, make a new mixeu mouel :1(5#0( (5,( 4#: (this is achieveu by POP[-i,]). Then, the function fixef() extiacts whatevei coefficient inteiests you. You will neeu to auapt this coue to youi analysis. Besiues the names of youi uata fiame anu youi vaiiables, you neeu to iun fixef() on youi mouel once so you know which position the ielevant coefficient is. In oui case, I woulu put a "2" in theie because the effect of "attituuepol" appeais seconu in the list of coefficients. "1" woulu give me the inteicept, always the fiist coefficient mentioneu in the coefficient table. S9
So, a ianuom effect is geneially something that can be expecteu to have a non- systematic, iuiosynciatic, unpieuictable, oi "ianuom" influence on youi uata. In expeiiments, that's often "subject" anu "item", anu you geneially want to geneialize ovei the iuiosynciasies of inuiviuual subjects anu items.
Fixeu effects on the othei hanu aie expecteu to have a systematic anu pieuictable influence on youi uata.
But theie's moie to it. 0ne uefinition of fixeu effects says that fixeu effects "exhaust the population of inteiest", oi they exhaust "the levels of a factoi". Think back of sex. Theie's only "male" oi "female" foi the vaiiable "genuei" in oui stuuy, so these aie the only two levels of this factoi. 0ui expeiiment incluues both categoiies anu thus exhausts the categoiy sex. With oui factoi "politeness" it's a bit tiickiei. You coulu imagine that theie aie moie politeness levels than just the two that we testeu. But in the context of oui expeiiment, we #%'4,(1#2,&&) -'812'- politeness as the uiffeience between these two categoiies - anu because we testeu both, we fully "exhaust" the factoi politeness (as uefineu by us).
In contiast, ianuom effects geneially sample fiom the population of inteiest. That means that they aie fai away fiom "exhausting the population" . because theie's usually many many moie subjects oi items that you coulu have testeu. The levels of the factoi in youi expeiiment is a tiny subset of the levels "out theie" in the woilu.
The write-up A lot of tutoiials uon't covei how to wiite up youi iesults. Anu that's a pity, because this is a ciucial pait of youi stuuy!!!
The most impoitant thing: You neeu to uesciibe the mouel to such an extent that people can %$+%0#!/$ '4$ ("(,-&)&< So, a useful heuiistic foi wiiting up youi iesults is to ask youiself the question "Woulu I be able to ie-cieate the analysis given the infoimation that I pioviueu." If the answei is "yes" youi wiite-up is goou.
In paiticulai, this means that you specify all fixeu effects anu all ianuom effects, anu you shoulu also mention whethei you have ianuom inteicepts oi ianuom slopes.
Foi iepoiting inuiviuual iesults, you can stick to my example with the likelihoou iatio test above. Remembei that it's always impoitant to iepoit the actual coefficientsestimates anu not just whethei an effect is significant. You shoulu also mention stanuaiu eiiois.
4u Anothei impoitant thing is to give enough cieuit to the people who put so much of theii fiee time into making &$'; anu R woik so efficiently. So let's cite them! It's also a goou iuea to cite exactly the veision that you useu foi youi analysis. You can finu out youi veision anu who to cite by typing in.
citation()
. foi youi R-veision . anu .
citation(lme4)
. foi the lme4 package.
Finally, it's impoitant that you mention that you checkeu assumptions, anu that the assumptions aie satisfieu.
So heie's what I woulu have wiitten foi the analysis that we peifoimeu in this tutoiial:
"We useu R (R Coie Team, 2u12) anu &$'; (Bates, Naechlei & Bolkei, 2u12) to peifoim a lineai mixeu effects analysis of the ielationship between pitch anu politeness. As fixeu effects, we enteieu politeness anu genuei (without inteiaction teim) into the mouel. As ianuom effects, we hau inteicepts foi subjects anu items, as well as by-subject anu by-item ianuom slopes foi the effect of politeness. visual inspection of iesiuual plots uiu not ieveal any obvious ueviations fiom homosceuasticity oi noimality. P-values weie obtaineu by likelihoou iatio tests of the full mouel with the effect in question against the mouel without the effect in question."
Yay, we'ie uone!! I hope this tutoiial was of help to you.
References Bates, B.N., Naechlei, N., & Bolkei, B. (2u12). lme4: Lineai mixeu-effects mouels using S4 classes. R package veision u.999999-u. Baayen, R.B. (2uu8). ?2,&)@123 A12301/(1" B,(,C ? D4,"(1",& E2(4#-0"(1#2 (# F(,(1/(1"/ G/123 *. Cambiiuge: Cambiiuge 0niveisity Piess. Baayen, R.B., Baviuson, B.}., Bates, B.N. (2uu8). Nixeu-effects moueling with ciosseu ianuom effects foi subjects anu items. H#042,& #8 ='$#4) ,2- A,230,3', S9, S9u- 412. 41 Baii, B.}., Levy, R., Scheepeis, C., & Tilly, B. }. (2u1S). Ranuom effects stiuctuie foi confiimatoiy hypothesis testing: Keep it maximal. H#042,& #8 ='$#4) ,2- A,230,3', 68, 2SS-278. Bolkei, B. N., Biooks, N. E., Claik, C. }., ueange, S. W., Poulsen, }. R., Stevens, N. B. B., & White, }. S. S. (2uu9). ueneializeu lineai mixeu mouels: a piactical guiue foi ecology anu evolution. I4'2-/ 12 >"#) J >9#&0(1#2, 24(S), 127-1SS. Bianut, N. }. (2u11). Nasty uata can still be ieal: A ieply to 0lliich anu Schltei. D/)"5#",& F"1'2"', 2S, 826. Claik, B. B. (197S). The language-as-fixeu-effect fallacy: A ciitique of language statistics in psychological ieseaich. H#042,& #8 K'47,& A',42123 ,2- K'47,& L'5,91#4, 12, SSS-SS9. Foistei, K.I., & Bickinson, R.u. (1976). Noie on the language-as-fixeu-effect fallacy: Nonte Cailo estimates of eiioi iates foi F1, F2, F' anu minF'. H#042,& #8 K'47,& A',42123 J K'47,& L'5,91#46 1S, 1SS-142. Fieebeig, T.N., & Lucas, }.R. (2uu9). Pseuuoieplication is (still) a pioblem. H#042,& #8 M#$%,4,(19' D/)"5#), 12S, 4Su-4S1. uellman, A., & Bill, }. (2uu7). B,(, ,2,&)/1/ 0/123 4'34'//1#2 ,2- $0&(1&'9'&N51'4,4"51",& $#-'&/. Cambiiuge: Cambiiuge 0niveisity Piess. Builbeit, S.B. (1984). Pseuuoieplication anu the uesign of ecological fielu expeiiments. >"#",& =#2#34,%5/, S4, 187-211. Lazic, S.E. (2u1u). The pioblem of pseuuoieplication in neuioscientific stuuies: is it affecting youi analysis. L=M O'04#/"1'2"', 11, 1-17. Lockei, L., Boffman, L., & Bovaiiu, }.A. (2uu7). 0n the use of multilevel moueling as an alteinative to items analysis in psycholinguistic ieseaich. L'5,91#4 *'/',4"5 ='(5#-/, S9, 72S-7Su. Nieuwenhuis, R., te uiotenhuis, N., & Pelzei, B. (2u12). Influence.NE: Tools foi Betecting Influential Bata in Nixeu Effects Nouels. * H#042,&, 4(2): pp. S8-47. Pinheiio, }.C., & Bates, B.N. (2uuu). =1P'-Q>88'"(/ =#-'&/ 12 F ,2- FDAGF. New Yoik: Spiingei. Raaijmakeis, }.u. (2uuS). A fuithei look at the "language-as-fixeu-effect fallacy". M,2,-1,2 H#042,& #8 >P%'41$'2(,& D/)"5#), S7, 141-1S1. Raaijmakeis, }.u., Schiijnemakeis, }.N.C., & uiemmen, F. (1999). Bow to Beal with "The Language-as-Fixeu-Effect Fallacy": Common Nisconceptions anu Alteinative Solutions. H#042,& #8 ='$#4) ,2- A,230,3', 41, 416-426. R Coie Team (2u12). R: A language anu enviionment foi statistical computing. R Founuation foi Statistical Computing, vienna, Austiia. Schielzeth, B., & Foistmeiei, W. (2uu9). Conclusions beyonu suppoit: oveiconfiuent estimates in mixeu mouels. L'5,91#4,& >"#)6 2u, 416-42u. 0lliich, }., & Schltei, E. (2u11). Betecting nasty uata with simple plots of complex mouels: Comment on Bianut (2u11). D/)"5#",& F"1'2"', 2S, 824-82S. Wike, E.L., & Chuich, }.B. (1976). Comments on Claik's "The language-as-fixeu-effect fallacy". H#042,& #8 K'47,& A',42123 J K'47,& L'5,91#4, 1S, 249-2SS. 42 Wintei, B. (2u11). Pseuuoieplication in phonetic ieseaich. D4#"''-123/ #8 (5' E2('42,(1#2,& M#234'// #8 D5#2'(1" F"1'2"' (pp. 21S7-214u). Bong Kong, August 2u11. Wintei, B., & uiawunuei, S. (2u12). The Phonetic Piofile of Koiean Foimality. H#042,& #8 D5#2'(1"/, 4u, 8u8-81S.