Ch.

12 The Analysis of Variance

• Residual Analysis and Model Assessment

◦ Model Assumptions:
The measurements Xi,j

◦◦ can be modelled as normal random variables.

◦◦ are statistically independent.

◦◦ come from populations having the same variance.

◦ These assumptions can be checked by examining plots

of residuals.
◦ For the completely randomized design, the residuals
are
ei,j = xi,j − x̄i·
for i = 1, 2, . . . , I, j = 1, 2, . . . , J.

◦ Compare:
εi,j = xi,j − µi

ei,j = xi,j − x̄i·

◦ ei,j contains information about variability left

unexplained by effects of the differences in the
treatment groups.
◦ If the model assumptions are reasonable, then the
residuals

◦◦ should appear to be normally distributed

◦◦ should be close to statistically independent

◦◦ have a constant variance; the residual variance

should not differ for different treatment groups
◦ Residual Plots:

◦◦ Normal QQ Plot: Residuals should be scattered

about a straight line.

◦◦ Run Chart: There should be no systematic pattern.

◦◦ Plot of Residuals Versus Fitted Values (x̄i·): There

should be no systematic pattern.
◦ Remedial Measures:

◦◦ If the residuals are not close to normal, a data

transformation might help

◦◦ A pattern on the run chart may be indicating that

the measurements are not independent. Run order
may be an important factor – it should be included in
the experimental design
◦◦ If variability in the residuals is increasing with the
treatment means, two common approaches are:

1. work on square root scale if treatment variances are

approximately proportional to treatment means

2. work on log scale if treatment standard deviations

are approximately proportional to treatment means
◦ Example. An experiment was conducted to see if
temperature had an effect on the number of surface
flaws resulting after painting car exteriors.

◦◦ 24 cars were assigned randomly to each of

◦◦ 3 temperature groups (10 C, 20 C, 30 C)

◦◦ A boxplot of the number of surface flaws against

temperature suggests that there might be a
difference in mean number of flaws at the different
temperatures.
 Boxplot of Car Surface Defects

10
8
Number of Surface Flaws

6
4
2
0

10 20 30

temperature
◦◦ The treatment means are x̄1· = 4.875, x̄2· = 1.375, and
x̄3· = 6.125.

◦◦ Residuals: e1,1 = 3 − 4.875 = −1.875, etc.

◦◦ Can the analysis of variance be used?

Check Residual Plots:

1. Are the residuals normally distributed? No – they

are discrete... but
 QQ−Plot of Car Finish Residuals

4
2
Surface Flaw Residuals

0
−2
−4

−2 −1 0 1 2

Theoretical Quantiles
 This normal QQ-plot suggests no serious departure
from normality

2. Are the residuals independent?

 Residuals Residuals

−4 −2 0 2 4 −2 −1 0 1 2 3

1
1

2
2

3
3

4
4

Time
Time

5
5

6
6

7
7

8
8

Residuals

−1.0 0.0 1.0

1
2
3
4

Time
5
6
7
8
 Systematic patterns on the run charts are not
evident

3. Are the residual variances constant?

 Residuals vs Fitted

4
21
17
2
Residuals

0
−2
−4

24

2 3 4 5 6

Fitted values
aov(formula = SurfaceFlaws ~ factor(Temperature))
 the treatment variance appears to increase with
treatment mean
Remedy: For count data, a square root
transformation often works.

◦ Strategy: Take square root of all 24 defect counts.

◦ Compute treatment means on the square root scale:

x̄1· = 2.16, x̄2· = 0.985, x̄3· = 2.41
Surface Flaw Residuals QQ−Plot of Car Finish Residuals
0.5
0.0
−1.0

−2 −1 0 1 2

Theoretical Quantiles

Residuals vs Fitted
0.0 0.5
Residuals

−1.0

16
10 24

1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4

◦ Normal probability plot of residuals indicates a
departure from normality; ANOVA results will be
somewhat conservative

◦ Plot of residuals versus treatment means indicates a

constant variance.

◦ Proceed to ANOVA using measurements on a square

root scale.
◦◦ ANOVA Results:

◦ SST = 16.69, d.f. = 23

◦ SSTr = 9.26, d.f. = 2

◦ Ultimately, we find F = 13.11 which is very large

relative to the tabulated F values: e.g.
f.001,2,21 = 9.77. Thus, we conclude that the mean
number of surface flaws is not always the same for
the different temperatures.
• Multiple Comparisons

◦ ANOVA leads to 1 of two possible conclusions:

1. Treatment group means are the same.

2. Treatment group means are different.

◦ If treatment means are the same, STOP.

◦ If treatment means are different, you may conduct a

Post-Hoc study:
1. Which means are different?

2. Find a confidence interval for a linear combination of

the true treatment means.

◦ Tukey’s Procedure: gives simultaneous confidence

intervals for all possible pairs of treatment means.
◦ We can have (1 − α)100% confidence that the true
difference µi − µk lies in the interval
q
x̄i· − x̄k· ± Qα,I,I(J−1) MSE/J
for each i, k = 1, 2, . . . , I.

◦ The Q factor is tabulated in Appendix A.10.

◦ Trusses Example. Find simultaneous 95% confidence
intervals for all pairs of true treatment group means.

◦◦ First, recall that we demonstrated that mean axial

stiffness index is different for trusses constructed
with plates of different lengths,

◦◦ ⇒ it makes sense to find out which treatment means

are different.
◦◦ Estimated mean axial stiffness for each of the plate
lengths:

333.2 (4 in.)
368 (6 in.)
375.1 (8 in.)
407.4 (10 in.)
437.2 (12 in.)

◦◦ I = 5, J = 7, α = .05 ⇒
Q.05,5,30 = 4.1 (m = 5, ν = 30)

◦◦ M SE: 1052.2
q
w = Q M SE/J
q
= 4.1 1052.2/7 = 50.2
◦◦ The 95% confidence intervals for µ1 − µi for
i = 2, 3, 4, 5 are

333.2 − 368 ± 50.2 = −34.8 ± 50.2

333.2 − 375.1 ± 50.2 = −41.9 ± 50.2

333.2 − 407.4 ± 50.2 = −74.2 ± 50.2

333.2 − 437.2 ± 50.2 = 104 ± 50.2

◦◦ For µ2 − µi, i = 3, 4, 5, we have

−7.1 ± 50.2

−39.4 ± 50.2

−69.2 ± 50.2
◦◦ For µ3 − µi, i = 4, 5, we have
−32.3 ± 50.2

−51.9 ± 50.2

◦◦ and µ4 − µ5 is in
−29.8 ± 50.2

◦◦ Lots of intervals.

◦◦ In some cases, the signal is stronger than the noise.

◦◦ These are the significant differences at level α=.05

◦◦ A quick way of seeing which treatment means are

significantly different is to list the means in ascending
order and identify all means that are within the
margin of error (50.2) of each other.
◦ Summary: to compute simultaneous confidence
intervals for true treatment means, Tukey’s method is
as follows:

1. Obtain Qα,I,I(J−1) from Tukey’s studentized range

table
q
2. Compute w = Q M SE/J

3. List the treatment group means in ascending order

and underline those which do not differ by more than
w.
• Confidence intervals for linear combinations of treatment
group means
I
X
a i µi
i=1
e.g.
µ 1 + µ 2 + µ 3 µ 4 + µ5
−
3 2
◦ Point estimator:
x̄ + x̄2· + x̄3· x̄4· + x̄5·
p.e. = 1· −
3 2
◦ Standard error: square root of
3σ 2 2σ 2
+
9J 4J

b 2 = M SE so
◦ σ
 ◦ Estimated Standard error is
s s
M SE 3 2
s.e. = +
J 9 4

◦ A (1 − α)100% confidence interval for this linear

combination is then

p.e. ± tα/2,I(J−1)s.e.

• Trusses Example. Compute p.e. and s.e. and a 95%

confidence interval for the above linear combination.
• 10.3 Unbalanced Designs

◦ Unequal sample sizes for different treatment groups

◦ SST: numerator of the sample variance of all

measurements

◦ SSE: sum of treatment group variance numerators

◦ SSTr = SST - SSE

◦ Total d.f. = number of measurements - 1

◦ Treatment d.f. = I - 1

◦ Error d.f. = number of measurements - I

◦ ANOVA table and F -test: as in the balanced case

◦ slight change to multiple comparisons formula (see

page 425)
• Example The yield of tomatoes (kg/plot) for four
different salinity levels (measured in terms of electrical
conductivity EC) are given. EC levels = 1.6, 3.8, 6.0 and
10.2 nmhos/cm

1.6: 59.5 53.3 56.8 63.1 58.7

3.8: 55.2 59.1 52.8 54.5
6.0: 51.7 48.8 53.9 49.0
10.2: 44.6 48.5 41.0 47.3 46.1

The scatterplot of the tomato yields for the various

salinities suggests that there is a difference. It also
indicates that the within treatment variance is constant.
 Scatterplot of Tomato Data
60
tomato yield (kg/plot)

55
50
45

2 4 6 8 10

salinity (EC) level

Use the F test at level α = .05 to test for any differences
in true average yield due to the different salinity levels.
◦ SST = 581

◦ The sample variances at each EC level are 13.0, 7.1,

5.9, and 8.4, giving us

SSE = 124.5

◦ After filling out the ANOVA table, we find that the

observed F value is 51.3.

◦ f.05,3,14 = 3.34 so we conclude that there is a difference

among the mean yields. Salinity level makes a
difference.
• Compute a 95% confidence interval for
µ10.2 − 1
3 (µ1.6 + µ3.8 + µ6 ).