REGRESI BERGANDA

Isi:
Multiple correlation and regression
Types of Multiple regression methods

Multiple Regression
When looking to make predictions, third (or
fourth) variables that correlate with the
criterion are not nuisances, but may provide
additional information.
With several potential predictor variables, use
Multiple Regression analyses
Built upon multiple correlation (just as with
bivariate regression and correlation are
related)

Multiple Regression Equation

regardless of the number of predictors, the
multiple linear regression equation is
Y'=a+b1x1+b2x2+b3x3+ bkxk
where Y = is the predicted value of Y
a = the regression constant
b1 - bk = regression weights or coefficients
and x1 - xk = predictor variables

Multiple Regression
Y

f
a

g
X1

How can you find area abc?

(which = R2YX1X2)
Remember
r2YX1X2 = a/(a+f)
r2YX2X1 = c/(c+f)

c
b
d

r2Y(X1X2) = a
r2Y(X2X1) = c

e
X2

r2X1Y = a+b
r2X1X2 = b+d
r2X2Y= b+c

Multiple Regression
Y

g
X1

X2

R2YX1X2= r2YX1+ r2Y(X2X1)

or
find area bc = r2(YX2)

b
d

find area ab = r2(YX1)

then add c = r2Y(X2X1)

f
a

Finding area abc:

then add a = r2Y(X1X2)

R2YX1X2= r2YX2+ r2Y(X1X2)

Get the same R2 but,

order of entry can importantly
influence your conclusions.

Multiple Regression Coefficient Equation

R2YX1X2X3Xn= r2YX1+ r2Y(X2X1) +
r2Y(X3X1X2)+ + r2Y(XnX1X2X3Xn-1)

Considerations about R2
R2 Coefficient of determination (total
proportion of variance you can account for.
1-R2: coefficient of non determination.
R2 cannot be less than the largest single
bivariate r2yx

Considerations about R2
With additional predictors, R2 will increase only to
the extent that the new predictor is not correlated
with predictors already in the equation.
Increase dependent upon the squared semi-partial
correlation coefficient:
R2YX1X2X3Xn= r2YX1+ r2Y(X2X1) + r2Y(X3X1X2)+ + r2Y(XnX1X2X3
Xn-1)

Additional variables entering the equation may

simply be fitting noise so test the significance of
improved fit between observed Y and predicted Y
after each step (except simple).
All assumptions & pitfalls of simple linear regression

Multiple Regression Methods

Direct (Simple) Regression
Forward regression
Backward regression
Stepwise Regression
Hierarchical Regression

Direct (Simple) Regression

All available predictor variable are put into the
equation at once and they are assessed as if
they had been entered last
They are assessed on the basis of the
proportion of variance in the criterion
variable (Y) they uniquely account for
(Squared Semi-Partial Correlations).
called simple regression in Bordens and Abbott
With SPSS, specify them with method =ENTER

Forward Regression
Sequentially add variables, one at a time
based on the strength of their squared
semi-partial correlations (or simple
bivariate correlation in the case of the first
variable to be entered into the equation)

Backward Regression
Start with them all in the equation then
delete them on the bases of smallest
change in the R2

Stepwise Regression
A combination of forward and backward
At each step one variable can be entered (on
basis of greatest improvement in R2) but
one may also be removed if the change
(reduction) in R2 is not significant.
In the Bordens and Abbott text, it sounds
like they use this term to mean Forward
regression.

Hierarchical Regression
This is the only method with which the researcher
assumes control over the analyses.
On basis of theory or practicality
This is especially important when multicollinearity
is a problem

Multiple Regression

R2YX1X2= r2YX1+ r2Y(X2X1)

=.15+.36=.51

f
a

R2YX1X2= r2YX2+ r2Y(X1X2)

X2
c

=.45+.06=.51
r2X1Y=.15
r2X1X2 =.15
r2X2Y =.45

e
d

r2Y(X1X2) = .06
r2Y(X2X1) = .36
r2YX1X2 = .06/(.06+.49)=.11

X1

r2YX2X1 = .36/(.36+.49)=.42

Multiple Regression

R2YX1X2= r2YX1+ r2Y(X2X1)

=.20+.04=.24

R2YX1X2= r2YX2+ r2Y(X1X2)

=.19+.05=.24
r2X1Y=.20
r2X1X2 =.80

a
b
g
X1

r2X2Y =.19

r2Y(X1X2) = .05

e
X2

r2Y(X2X1) = .04
r2YX1X2 = .05/(.05+.80)=.06
r2YX2X1 = .04/(.04+.80)=.05

Multicollinearity Examples
predicting political development (77
countries) from information re.
communication, urbanization,
education, and agricultural
development: among predictors, lowest
r was .69, highest was .88
R2y 1, R2y 12, R2y 123, R2y 1234
were .66, .67, .67, .67

MR is Powerful and Easy to (Mis)Use

Anton de Mann's talk: suicide ideation
Anonymous support service, n.s. after things that
could not be controlled.
Another colloquia on adolescent diabetics:
After demographic information added, education
(about the dangers -- potential fatal -- of not
taking insulin regularly and maintaining strict
diets) was n.s.
Conclude: No sense in increasing efforts to
education this kids!!!
Didn't even know if it would be significant if
entered first

Better examples: Layton & Swanson

Coleman Report
Explain school achievement inequalities:
1 important DV was verbal abilities, 60
IVs! 5 chosen (based on assumed
importance) - blacks and whites
examined separately
3 different orders of entry

Coleman Report
Explain school achievement inequalities:
For whites, Self-concept consistently
important
For blacks, less so: Control of the
environment more important
Interesting that both are quite subjective attitude - measures

Warsaw Study
Assessing massive efforts by government
to achieve educational equality
spread individuals around city (mixing
SES groups that might differ in ability
and achievement)
1300 kids tested on non-verbal IQ test
3 orders of entry

Warsaw Study

Warsaw Study
Successfully removed any school or area effects:
Family consistently significant.
Concluded that societal changes over a
generation failed to override forces that
determine social class distribution in mental
performance

Multivariate Designs and Analyses

are all very powerful and easy
statistics to use, and misuse.
To use these the techniques
appropriately depends upon
careful research design and
thought.

SPSS Multiple Regression Output

Variables Entered/Removedb
Model
1

Variables
Entered
EMPATEND,
CONTROL,
EFFICACY,
SYMPATHY,
a
ANGER

Variables
Removed

Method

Enter

a. All requested variables entered.

b. Dependent Variable: ZHELP
Model Summary
Model
1

R
.626a

R Square
.392

Adjusted
R Square
.352

Std. Error of
the Estimate
1.0085

a. Predictors: (Constant), EMPATEND, CONTROL,

EFFICACY, SYMPATHY, ANGER

SPSS Multiple Regression Output

ANOVAb

Model
1

Regression
Residual
Total

Sum of
Squares
49.272
76.281
125.553

df
5
75
80

Mean Square
9.854
1.017

F
9.689

Sig.
.000a

a. Predictors: (Constant), EMPATEND, CONTROL, EFFICACY, SYMPATHY, ANGER

b. Dependent Variable: ZHELP

Coefficientsa

Model
1

Unstandardized
Coefficients
B
Std. Error
(Constant)
-4.407
.790
CONTROL
-1.12E-03
.076
SYMPATHY
.456
.107
ANGER
.288
.090
EFFICACY
.431
.133
EMPATEND 1.089E-02
.009

a. Dependent Variable: ZHELP

Standardized
Coefficients
Beta
-.001
.416
.315
.297
.119

t
-5.580
-.015
4.276
3.186
3.228
1.255

Sig.
.000
.988
.000
.002
.002
.214

SPSS Variables
Multiple
Regression Output
Entered/Removed
a

Model
1

Variables
Entered

Variables
Removed

SYMPATHY

ANGER

EFFICACY

Method
Stepwise (Criteria:
Probability-of-F-to-enter <= .050,
Probability-of-F-to-remove >= .100).
Stepwise (Criteria:
Probability-of-F-to-enter <= .050,
Probability-of-F-to-remove >= .100).
Stepwise (Criteria:
Probability-of-F-to-enter <= .050,
Probability-of-F-to-remove >= .100).

a. Dependent Variable: ZHELP

Model Summary
Model
1
2
3

R
.455a
.548b
.616c

R Square
.207
.300
.380

Adjusted
R Square
.197
.282
.355

Std. Error of
the Estimate
1.1225
1.0612
1.0058

a. Predictors: (Constant), SYMPATHY

b. Predictors: (Constant), SYMPATHY, ANGER
c. Predictors: (Constant), SYMPATHY, ANGER, EFFICACY

SPSS Multiple Regression Output

ANOVAd
Model
1

Regression
Residual
Total
Regression
Residual
Total
Regression
Residual
Total

Sum of
Squares
26.008
99.544
125.553
37.715
87.837
125.553
47.654
77.899
125.553

df
1
79
80
2
78
80
3
77
80

Mean Square
26.008
1.260

F
20.641

Sig.
.000a

18.858
1.126

16.746

.000b

15.885
1.012

15.701

.000c

a. Predictors: (Constant), SYMPATHY

b. Predictors: (Constant), SYMPATHY, ANGER
c. Predictors: (Constant), SYMPATHY, ANGER, EFFICACY
d. Dependent Variable: ZHELP

SPSS Multiple Regression Output

Coefficientsa

Model
1
2

(Constant)
SYMPATHY
(Constant)
SYMPATHY
ANGER
(Constant)
SYMPATHY
ANGER
EFFICACY

Unstandardized
Coefficients
B
Std. Error
-1.892
.510
.498
.110
-2.660
.537
.539
.104
.281
.087
-4.308
.732
.494
.100
.284
.083
.412
.132

a. Dependent Variable: ZHELP

Standardized
Coefficients
Beta
.455
.492
.308
.451
.310
.284

t
-3.713
4.543
-4.950
5.159
3.224
-5.885
4.938
3.429
3.134

Sig.
.000
.000
.000
.000
.002
.000
.000
.001
.002

SPSS Multiple Regression Output

Excluded Variablesd

Model
1

CONTROL
ANGER
EFFICACY
EMPATEND
CONTROL
EFFICACY
EMPATEND
CONTROL
EMPATEND

Beta In
.157a
.308a
.282a
.073a
.042b
.284b
.086b
.013c
.119c

t
1.549
3.224
2.911
.702
.394
3.134
.869
.128
1.269

Sig.
.125
.002
.005
.485
.694
.002
.387
.898
.208

Partial
Correlation
.173
.343
.313
.079
.045
.336
.099
.015
.144

Collinearity
Statistics
Tolerance
.966
.985
.979
.922
.819
.979
.920
.812
.909

a. Predictors in the Model: (Constant), SYMPATHY

b. Predictors in the Model: (Constant), SYMPATHY, ANGER
c. Predictors in the Model: (Constant), SYMPATHY, ANGER, EFFICACY
d. Dependent Variable: ZHELP

SPSS Multiple Regression: Correlations

Correlations
CONTROL

SYMPATHY

ANGER

EFFICACY

ZHELP

EMPATEND

Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N
Pearson Correlation
Sig. (2-tailed)
N

CONTROL SYMPATHY
1.000
-.184
.
.100
81
81
-.184
1.000
.100
.
81
81
.403**
-.120
.000
.284
81
81
.053
.145
.640
.195
81
81
.067
.455**
.550
.000
81
81
.023
.280*
.837
.011
81
81

**. Correlation is significant at the 0.01 level (2-tailed).

*. Correlation is significant at the 0.05 level (2-tailed).

ANGER
EFFICACY
.403**
.053
.000
.640
81
81
-.120
.145
.284
.195
81
81
1.000
-.026
.
.818
81
81
-.026
1.000
.818
.
81
81
.248*
.342**
.025
.002
81
81
-.071
-.061
.530
.586
81
81

ZHELP
EMPATEND
.067
.023
.550
.837
81
81
.455**
.280*
.000
.011
81
81
.248*
-.071
.025
.530
81
81
.342**
-.061
.002
.586
81
81
1.000
.195
.
.081
81
81
.195
1.000
.081
.
81
81