Академический Документы
Профессиональный Документы
Культура Документы
Chapter 11
Simple Regression
Chap 12-1
Chapter Goals
After completing this chapter, you should be
able to:
Explain the correlation coefficient and perform a
hypothesis test for zero population correlation
Chap 12-2
Chapter Goals
(continued)
Chap 12-3
Correlation Analysis
Correlation analysis is used to measure
strength of the association (linear relationship)
between two variables
Correlation is only concerned with strength of the
relationship
No causal effect is implied with correlation
Correlation was first presented in Chapter 3
Chap 12-4
Correlation Analysis
The population correlation coefficient is
denoted (the Greek letter rho)
The sample correlation coefficient is
sxy
r
sx sy
where
s xy
(x x)(y y)
n 1
Chap 12-5
H0 : 0
r (n 2)
(1 r )
Chap 12-6
Decision Rules
Hypothesis Test for Correlation
Lower-tail test:
Upper-tail test:
Two-tail test:
H0: 0
H1: < 0
H0: 0
H1: > 0
H0: = 0
H1: 0
a
-ta
ta
r (n 2)
(1 r )
2
a/2
-ta/2
a/2
ta/2
has n - 2 d.f.
Chap 12-7
Introduction to
Regression Analysis
Regression analysis is used to:
Predict the value of a dependent variable based on
the value of at least one independent variable
Explain the impact of changes in an independent
variable on the dependent variable
Dependent variable: the variable we wish to explain
(also called the endogenous variable)
Chap 12-8
Yi 0 1xi i
Where 0 and 1 are the population model
coefficients and is a random error term.
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
Chap 12-9
Population
Slope
Coefficient
Independent
Variable
Random
Error
term
Yi 0 1Xi i
Linear component
Random Error
component
Chap 12-10
Yi 0 1Xi i
Observed Value
of Y for Xi
i
Predicted Value
of Y for Xi
Slope = 1
Random Error
for this Xi value
Intercept = 0
Xi
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
X
Chap 12-11
Estimate of
the regression
intercept
Estimate of the
regression slope
y i b0 b1xi
Value of x for
observation i
Chap 12-12
Chap 12-13
b1
(x x)(y y)
i
i1
x
2
(x
x
)
i
rxy
sY
sX
i1
b0 y b1x
The regression line always goes through the mean x, y
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
Chap 12-14
Chap 12-15
E[ i ] 0 and E[ i ] 2
2
for (i 1, ,n)
for all i j
Chap 12-16
Interpretation of the
Slope and the Intercept
b0 is the estimated average value of y
when the value of x is zero (if x = 0 is
in the range of observed x values)
Chap 12-17
Chap 12-18
Square Feet
(X)
245
1400
312
1600
279
1700
308
1875
199
1100
219
1550
405
2350
324
2450
319
1425
255
1700
Chap 12-19
Graphical Presentation
House price model: scatter plot
House Price ($1000s)
450
400
350
300
250
200
150
100
50
0
0
500
1000
1500
2000
2500
3000
Square Feet
Chap 12-20
Chap 12-21
Excel Output
Regression Statistics
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
41.33032
Observations
10
ANOVA
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Coefficients
Intercept
Square Feet
Standard Error
t Stat
P-value
Significance F
0.01039
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-22
Chap 12-23
Graphical Presentation
Intercept
= 98.248
400
350
Slope
= 0.10977
300
250
200
150
100
50
0
0
500
1000
1500
2000
2500
3000
Square Feet
Chap 12-24
Interpretation of the
Intercept, b0
house price 98.24833 0.10977 (square feet)
b0 is the estimated average value of Y when the
value of X is zero (if X = 0 is in the range of
observed X values)
Here, no houses had 0 square feet, so b0 = 98.24833
just indicates that, for houses within the range of
sizes observed, $98,248.33 is the portion of the
house price not explained by square feet
Chap 12-25
Interpretation of the
Slope Coefficient, b1
house price 98.24833 0.10977 (square feet)
Chap 12-26
Measures of Variation
Total variation is made up of two parts:
SST
SSR
SSE
Total Sum of
Squares
Regression Sum
of Squares
Error Sum of
Squares
SSR (y i y)2
SSE (yi y i )2
where:
Chap 12-27
Measures of Variation
(continued)
Chap 12-28
Measures of Variation
(continued)
Y
yi
2
SSE = (yi - yi )
_
y
xi
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
_
y
X
Chap 12-29
Coefficient of Determination, R2
The coefficient of determination is the portion
of the total variation in the dependent variable
that is explained by variation in the
independent variable
The coefficient of determination is also called
R-squared and is denoted as R2
SSR regressionsum of squares
R
SST
total sum of squares
2
note:
0 R 1
Chap 12-30
Examples of Approximate
r2 Values
Y
r2 = 1
r2 = 1
r2
=1
Chap 12-31
Examples of Approximate
r2 Values
Y
0 < r2 < 1
X
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
Chap 12-32
Examples of Approximate
r2 Values
r2 = 0
No linear relationship
between X and Y:
r2 = 0
Chap 12-33
Excel Output
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
SSR 18934.9348
R
0.58082
SST 32600.5000
2
Regression Statistics
41.33032
Observations
10
ANOVA
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Coefficients
Intercept
Square Feet
Standard Error
t Stat
P-value
Significance F
0.01039
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-34
Chap 12-35
Correlation and R2
The coefficient of determination, R2, for a
simple regression is equal to the simple
correlation squared
R r
2
2
xy
Chap 12-36
Estimation of Model
Error Variance
An estimator for the variance of the population model
error is
n
2
e
i
SSE
s
n2 n2
2
2
e
i1
Chap 12-37
Excel Output
Regression Statistics
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
se 41.33032
41.33032
Observations
10
ANOVA
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Coefficients
Intercept
Square Feet
Standard Error
t Stat
P-value
Significance F
0.01039
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-38
Chap 12-39
small se
large se
Chap 12-40
2
2
(xi x) (n 1)sx
where:
sb1
SSE
se
n2
Chap 12-41
Excel Output
Regression Statistics
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
sb1 0.03297
41.33032
Observations
10
ANOVA
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Coefficients
Intercept
Square Feet
Standard Error
t Stat
P-value
Significance F
0.01039
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-42
small Sb1
large Sb1
Chap 12-43
Test statistic
b1 1
t
sb1
d.f. n 2
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
where:
b1 = regression slope
coefficient
1 = hypothesized slope
sb1 = standard
error of the slope
Chap 12-44
Square Feet
(x)
245
1400
312
1600
279
1700
308
1875
199
1100
219
1550
405
2350
324
2450
319
1425
255
1700
Chap 12-45
Square Feet
b1
Standard Error
sb1
t Stat
P-value
98.24833
58.03348
1.69296
0.12892
0.10977
0.03297
3.32938
0.01039
b1 1 0.10977 0
t
3.32938
t
sb1
0.03297
Chap 12-46
Square Feet
d.f. = 10-2 = 8
b1
Standard Error
sb1
t Stat
P-value
98.24833
58.03348
1.69296
0.12892
0.10977
0.03297
3.32938
0.01039
t8,.025 = 2.3060
a/2=.025
Reject H0
a/2=.025
Do not reject H0
-tn-2,/2
-2.3060
Reject H0
tn-2,/2
2.3060 3.329
Decision:
Reject H0
Conclusion:
There is sufficient evidence
that square footage affects
house price
Chap 12-47
Coefficients
Standard Error
t Stat
P-value
98.24833
0.10977
Lower 95%
Upper 95%
58.03348
1.69296
0.12892
-35.57720
232.07386
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-48
Intercept
Square Feet
Coefficients
Standard Error
t Stat
P-value
98.24833
0.10977
Lower 95%
Upper 95%
58.03348
1.69296
0.12892
-35.57720
232.07386
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-49
MSR
SSR
k
SSE
MSE
n k 1
where F follows an F distribution with k numerator and (n k - 1)
denominator degrees of freedom
(k = the number of independent variables in the regression model)
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
Chap 12-50
Excel Output
Regression Statistics
Multiple R
0.76211
R Square
0.58082
Adjusted R Square
0.52842
Standard Error
MSR 18934.9348
F
11.0848
MSE 1708.1957
41.33032
Observations
10
P-value for
the F-Test
ANOVA
df
SS
MS
F
11.0848
Regression
18934.9348
18934.9348
Residual
13665.5652
1708.1957
Total
32600.5000
Coefficients
Intercept
Square Feet
Standard Error
t Stat
P-value
Significance F
0.01039
Lower 95%
Upper 95%
98.24833
58.03348
1.69296
0.12892
-35.57720
232.07386
0.10977
0.03297
3.32938
0.01039
0.03374
0.18580
Chap 12-51
Test Statistic:
H0: 1 = 0
H1: 1 0
a = .05
df1= 1
df2 = 8
MSR
F
11.08
MSE
Decision:
Reject H0 at a = 0.05
Critical
Value:
Fa = 5.32
Conclusion:
a = .05
Do not
reject H0
Reject H0
F.05 = 5.32
Chap 12-52
Prediction
The regression equation can be used to
predict a value for y, given a particular x
For a specified value, xn+1 , the predicted
value is
y n1 b0 b1xn1
Chap 12-53
Predictions Using
Regression Analysis
Predict the price for a house
with 2000 square feet:
Chap 12-54
450
400
350
300
250
200
150
100
50
0
0
500
1000
1500
2000
2500
3000
Risky to try to
extrapolate far
beyond the range
of observed Xs
Square Feet
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
Chap 12-55
y = b0+b1xi
Prediction Interval
for an single
observed y, given xi
Statistics for Business and Economics, 6e 2007 Pearson Education, Inc.
xi
X
Chap 12-56
1 (x n1 x)2
2
n (x i x)
(xn1 x)2
Chap 12-57
1 (xn1 x)2
1
2
n (xi x)
Chap 12-58
y n1 t n-2,/2 se
1 (xi x)2
317.85 37.12
2
n (xi x)
Chap 12-59
y n1 t n-1,/2 se
1
(Xi X)2
1
317.85 102.28
2
n (Xi X)
Chap 12-60
Chapter Summary
Introduced the linear regression model
Chap 12-61
Chap 12-62