Trivariate Regression By Hand

It is not totally unreasonable to conduct a multiple regression analysis by hand, as long as you have
only two predictor variables. Consider the analysis we did in PSYC 6430 (with the program
CorrRegr.sas) predicting attitudes towards animals (AR) from idealism and misanthropy. SAS gave
us the following univariate and bivariate statistics:
Variable
ar
ideal
misanth

Mean

Std Dev

Sum

Minimum

Maximum

154
2.37969
0.53501
366.47169
1.21429
4.21429
154
3.65024
0.53278
562.13651
2.30000
5.00000
154
2.32078
0.67346
357.40000
1.00000
4.00000

Pearson Correlation Coefficients, N = 154

Prob> |r| under H0: Rho=0
ar
ar
ideal

ideal

misanth

1.00000

0.05312

0.05312

1.00000
0.5129

0.22064
0.5129
0.0060
-0.13975

0.0839

Let us first obtain the beta weights for a standardized regression equation,

r y 1 ry 2 r12
1 r

2
12

Z y 1Z1 2 Z 2

.05312 .22064( 0.13975)

.0856
1 .139752

Look at the numerator of the ratio above. In computing the partial standardized slope for
predicting Y from X1, we remove from the standardized slope for predicting Y from X 1 ignoring X2 (ry1)
the overlap among Y, X1, and X2 (-ry2r12).

r y 2 r y 1r12
1 r

2
12

.22064 .05312( 0.13975)

.2326
1 .13975 2

Now for the unstandardized equation, Y a b1 X 1 b2 X 2

bi

i sy
si

b1

.0856(.535)
.086
.53278

a Y bX i
b2

.2326(.535)
.185
.67346

a 2.38 .086(3.65) .185(2.32) 1.637

The Multiple R2

Copyright 2014, Karl L. Wuensch - All rights reserved.

MultReg-ByHand.docx

r r
2
y1

2
y2

2r y 1r y 2 r12

.05312 .22064 2(.05312)(. 22064)( 0.13975)

.0559
1 r
1 .13975 2
Look at the numerator above. We sum the portion of variance in Y that is explained by X 1
ignoring X2(ry1) and the portion of variance in Y that is explained by X 2 ignoring X1(ry2) and then we
remove from that the portion of variance that has been counted twice.
R y212

2
12

R y212...i ...p i r yi r y2y

forp 2

(p is number of predictors)

R y212 .0856(.05312) .2326(.22064 ) .0559

Semipartials

sr1

ry 1 ry 2 r12
1 r122

sr2

r y 2 r y 1r12

sr i 2 R y212...i ...p R y212...(i )...p

1 r122

(but easier to do by method below)

forp 2

sr12 .0559 .22064 2 .0072 , so sr = .085

1
sr22 .0559 .05312 2 .0531 , so sr = .230
2
Partials

pr1

sr1
1 r

pr2

2
y2

pri 2

sri 2
1 R y212...(i )...p

sr2
1 r y21

forp 2

Tests of significance of partials

Testing the null that in the population I = 0 is absolutely equivalent to testing the null that the
semipartial or partial correlation coefficient is 0.

SE 1

1 R y212

1
N p 1 1 r122

H0: Pop. = 0

SE

df = N - p - 1 (but easier to get with next formula)

H0: Pop. sr = 0
t 1 .085

N p 1
1 R 2

t sr

154 2 1
1.07
1 .0559

df = N - p - 1
t 2 .230

154 2 1
2.91
1 .0559

df= 151

Shrunked (Adjusted) R2
(1 R 2 )( N 1)
(1 .0559)(153)
shrunken R 2 1
1
.043
N p 1
151
Please re-read this document about shrunken R2.
ANOVA summary table:
Source
Regression
Error
Total

SS
R SSY
(1-R2)SSY
SSY
2

df
p
n-p-1
n-1

MS

F tests the null that population R2 = 0

For our example, SSy = (N-1)s2 = 153(.53501)2 = 43.794
SSregr = .0559(43.794) = 2.447
SSerror= 43.794 - 2.447 = 41.347
F(2, 151) = 4.47, p = .013

standard error of estimate MSE 0.52

Testing the Significance of the R2
It is possible to get the value of F without computing the Mean Squares. Just use this formula:
R n p 1
.0559153 2 1
F
F
4.47
2
p 1 R
21 .0559
. For our data,
. Do note that even small rounding
2

errors in the computation of R2 can have noticeable effects on the value of F here, so obtain the R2
with several decimal point precision.
For a hierarchical analysis,
R y212...p r y21 sr y221 sr y2312 sr yp2 12...(p 1)
Return to Wuenschs Stats Lessons Page

Copyright 2014, Karl L. Wuensch - All rights reserved.