Академический Документы
Профессиональный Документы
Культура Документы
Part 2
The reduced model has to be a subset of the full (i.e. they are
nested models)
E.g. all terms in reduced model are also in full; full has additional
terms unique to it, but reduced has no terms unique to it
2
Model comparison by F-Test
> anova(model.polynomial.reduced, full.model.2ndinteractions)
Analysis of Variance Table
3
What if relationship between Y and
one or more Xs is nonlinear?
Option 1: transform data.
Option 2: use non-linear regression.
Option 3: use polynomial regression.
4
The polynomial regression model
In polynomial
regression, the 1000
regression model
5
The polynomial regression model: procedure
(mgDM/m)
test for increase in SSmodel .
100
Continue with higher order Linear model
(cubic, quartic, etc.) until 2nd order
there is no further polynomial model
10
significant increase in
10 30 50 70 90 110
SSmodel .
Current velocity (cm/s)
6
Polynomial regression: caveats
Y
be large (precision is low)
and will increase with the
order of the terms.
X1
7
Example
Lowess curve
1
y
-1
-3
10 30 50 70 90
x
8
Coefficients:
Value Std. Error t value Pr(>|t|)
(Intercept) -0.3701 0.1562 -2.3699 0.0188
x 0.0065 0.0027 2.4114 0.0168
1
0
-1
-2
0 20 40 60 80 100
9
Coefficients:
Value Std. Error t value Pr(>|t|)
(Intercept) 1.7718 0.1244 14.2486 0.0000
x -0.1195 0.0057 -21.0307 0.0000
I(x^2) 0.0012 0.0001 22.8826 0.0000
0
-1
-2
0 20 40 60 80 100
x
10
Value Std. Error t value Pr(>|t|)
(Intercept) 1.0295 0.1499 6.8655 0.0000
x -0.0335 0.0128 -2.6150 0.0096
I(x^2) -0.0009 0.0003 -2.9716 0.0033
I(x^3) 0.0000 0.0000 7.3217 0.0000
3
2
1
y
0
-1
-2
0 20 40 60 80 100
x
11
Coefficients:
Value Std. Error t value Pr(>|t|)
(Intercept) 1.0258 0.1923 5.3334 0.0000
x -0.0328 0.0261 -1.2547 0.2111
I(x^2) -0.0009 0.0010 -0.8652 0.3880
I(x^3) 0.0000 0.0000 0.9337 0.3516
I(x^4) 0.0000 0.0000 -0.0307 0.9755
0 20 40 60 80 100
x
12
> anova(degre1,degre2)
Analysis of Variance Table
Response: y
> anova(degre2,degre3)
Analysis of Variance Table
Response: y
> anova(degre4,degre3)
Analysis of Variance Table
Response: y
13
Overfitting/Underfitting
More examples/discussion:
https://stats.stackexchange.com/questions/128616/whats-a-real-world-example-of-
overfitting
14
Multiway ANOVA vs Multiple regression
15
When, and why, do we pool in ANOVA
16
How and why do you seek simple models in
multiple regression?
17
Multiple regression: the general idea for
inference
Evaluate significance of a
Model A
variable by fitting two models:
(X1 in)
one with the term in, the other
with it removed. MF
Test for change in model fit (e.g. R2)
associated with removal of the Model B
term in question. (X1 out)
Unfortunately, change in
model fit may depend on what
Retain X1
other variables are in model if
( large)
there is multicollinearity!
Delete X1
( small)
18
Fitting multiple regression models
19
Selection of independent variables
(contd)
20
Strategy I: computing all possible
models
Compute all possible
models and choose the {X1, X2, X3}
best one.
cons:
time-consuming
leaves definition of best {X1} {X1, X2} {X1, X2, X3}
to researcher
pros: {X2} {X2, X3}
if the best model is
defined, you will find it! {X3} {X1, X3}
21
Strategy II: stepwise forward selection
22
Strategy III: stepwise backward selection
23
AIC: Akaike Information Criteria
24
Example
25
Example (all variables)
Call:
lm(formula = logherp ~ logarea + cpfor2 + thtden, data = mydata)
Residuals:
Min 1Q Median 3Q Max
-0.30729 -0.13779 0.02627 0.11441 0.29582
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.284765 0.191420 1.488 0.149867
logarea 0.228490 0.057647 3.964 0.000578 ***
cpfor2 0.001095 0.001414 0.774 0.446516
thtden -0.035794 0.015726 -2.276 0.032055 *
---
Signif. codes: 0 *** 0.001 ** 0.01 * 0.05 . 0.1 1
26
Example: stepwise backward
Start: AIC=-98.27 Akaike Information criterion.
logherp ~ logarea + cpfor2 + thtden To be minimized
Df Sum of Sq RSS AIC
- cpfor2 1 0.01571 0.64508 -99.576
<none> 0.62937 -98.267
- thtden 1 0.13585 0.76522 -94.794
- logarea 1 0.41198 1.04135 -86.167
Step: AIC=-99.58
logherp ~ logarea + thtden
27
Example: forward stepwise
model.null <- lm(logherp ~ 1, data = mydata)
> step <- stepAIC(model.null, scope = ~. + logarea +
+ cpfor2 + thtden, direction = "forward")
Start: AIC=-82.09
logherp ~ 1
Step: AIC=-92.38
logherp ~ logarea
Step: AIC=-99.58
logherp ~ logarea + thtden
28
Information Theoretic Approach
29
> #####################################################################
> # Dredging and the information theoretical approach
> library(MuMIn)
> dd <- dredge(full.model.2ndinteractions)
>
> # get all possible models
> top.models.1 <- get.models(dd, subset = delta < 4)
> model.avg(top.models.1)
AICc for smaller data sets (N < 40k)
Model summary:
Deviance AICc Delta Weight
2+3+4+5+7 0.23 -36.46 0.00 0.34
2+3+4+5 0.26 -35.91 0.55 0.26
1+2+3+4+5+7 0.22 -33.75 2.72 0.09
2+3+4+5+7+8 0.22 -33.67 2.79 0.08
1+2+3+4+5 0.25 -33.25 3.21 0.07
2+3+4+5+8 0.25 -33.02 3.44 0.06
2+3+4+5+6+7 0.22 -32.90 3.56 0.06
2+3+4+5+6 0.26 -32.50 3.97 0.05
Variables:
1 2 3 4 5
cpfor2 I(swamp^2) logarea swamp thtden
6 7 8
logarea:swamp logarea:thtden swamp:thtden
30
Averaged model parameters:
Coefficient SE Adjusted SE Lower CI Upper CI
(Intercept) -2.02e-01 2.50e-01 2.61e-01 -0.713000 0.310000
cpfor2 -1.27e-04 4.84e-04 5.02e-04 -0.001110 0.000856
I(swamp^2) -2.68e-04 4.91e-05 5.16e-05 -0.000370 -0.000167
logarea 1.27e-01 1.20e-01 1.23e-01 -0.114000 0.369000
swamp 3.20e-02 6.13e-03 6.45e-03 0.019400 0.044700
thtden -7.02e-02 5.35e-02 5.49e-02 -0.178000 0.037500
logarea:swamp 4.73e-05 5.53e-04 5.84e-04 -0.001100 0.001190
logarea:thtden 2.23e-02 2.52e-02 2.58e-02 -0.028300 0.072900
swamp:thtden -3.49e-05 1.46e-04 1.52e-04 -0.000333 0.000263
31
What is the best approach?
32
What is the partial regression coefficient?
8
j measures the amount by X2 = -3
which Y changes when Xj is
4
increased by one unit and X2 = -1
all other independent
X2 = 1
variables are held constant. Y 0
-4
X2 = 3
Simple -8
regression -4 -2 0 2 4
Partial regression X1
33
https://stats.stackexchange.com/questions/78828/is-there-a-difference-
between-controlling-for-and-ignoring-other-variables-i/78830#78830
34
How to check the normality of data in multiple regression?
I don't understand why independent variables are always considered fixed? why
not random?
I was a bit confused on the difference between when two variables (ie: X1 and
X2) have an interaction versus when two variables are correlated. Would it be
possible to get an example of both?
35
I did not fully understand the definition of a parameter in the context of multiple regression,
and how these relate with the X variables.
Linear regression
Yi 0 1 X i i
dependent
variable independent unexplained
variable variation
parameters
36
If you know with two dependent variable that one is determined by the other are does it
make sense to omit it in a multiple regression
I still do not understand what Cook's distance is and how to interpret the graphs of
residuals vs leverage
How is tolerance useful if we can gain the same information from VIF?
I dont think I really understand the difference between interactions and multicolinearity (in
terms of how to identify them).
Are we required to memorize the degrees of freedom formulae that would be used to
calculate an F ratio, or just know that they are different for random or fixed effects?
In lecture 16: Slide 43: I don't understand "sensitivity of parameter estimates to small
changes in data (multicolinearity)
Table 37.3 (pg. 353) mentions repeated measures (multiple observations of the same
subject at different times) is a violation of independence in multiple regression. Would the
appropriate test(s) be a paired-t-test, or 2/Multi-Way ANOVA?
37