Chap 2

15/11/2011
Introductory Econometrics: A modern approach (Wooldridge)
Chapter 2
The Simple Regression Model
Dr. L Vn Chn FTU, 2011
Chap 9-1
2.1 Definition of Simple Regression Model

2
Applied econometric analysis often begins with 2 variables y and x. We are interested in studying how y varies with changes in x. E.g., x is years of education, y is hourly wage. x is number of police officers, y is a community crime rate. In the simple linear regression model: y = 0 + 1 x + u
(2.1)
y is called the dependent variable, the explained variable, or the regressand. x is called the independent variable, the explanatory variable, or the regressor. u, called error term or disturbance, represents factors other than x that affect y. u stands for unobserved.

3
If the other factors in u are held fixed, u = 0 , then x has a linear y = 1x effect on y:
1 is the slope parameter. This is of primary interest in applied economics.

One-unit change in x has the same effect on y, regardless of the initial value of x. Unrealistic. E.g., wage-education example, we might want to allow for increasing returns.
15/11/2011

4
An assumption: the average value of u in the population is zero. E(u) = 0 (2.5) This assumption is not restrictive since we can always use 0 to normalize E(u) to 0. Because u and x are random variables, we can define conditional distribution of u given any value of x. Crucial assumption: average value of u does not depend on x. E(u|x) = E(u) (2.5) + (2.6) This implies the zero conditional mean assumption. E(y|x) = 0 + 1 x
(2.6)

5
Population regression function (PRF): E(y|x) is a linear function of x. For any value of x, the distribution of y is centered about E(y|x).
y
f(y)
E ( y | x ) = 0 + 1 x
x1
x2
2.2 Ordinary Least Squares

6
How to estimate population parameters 0 and 1 from a sample? Let {(xi,yi): i = 1, 2, , n} denote a random sample of size n from the population. For each observation in this sample, it will be the case that
yi = 0 + 1 xi + ui
15/11/2011

7
y y4
PRF, sample data points and the associated error terms:
u4 {
} u3
E ( y | x ) = 0 + 1 x
y3 y2
} u1
u2 {
y1
x1
x2
x3
x4

8
To derive the OLS estimates, we need to realize that our main assumption of E(u|x) = E(u) = 0 also implies that Cov(x,u) = E(xu) = 0 (2.11) Why? Cov(x,u) = E(xu) E(x)E(u) = Ex[E(xu|x)] = Ex[xE(u|x)] = 0. We can write 2 restrictions (2.5) and (2.11) in terms of x, y, 0 and 1
E ( y 0 1 x) = 0 E[ x( y 0 1 x )] = 0
(2.12) (2.13)
(2.12) and (2.13) are 2 moment restrictions with 2 unknown parameters. They can be used to obtain good estimators of 0 and 1 .

9
Method of moments approach to estimation implies imposing the population moment restrictions on the sample moments. Given a sample, we choose estimates 0 and 1 to solve the sample versions: 1 n (2.14) ( yi 0 1xi ) = 0 n i =1
1 n xi ( yi 0 1xi ) = 0 n i =1
(2.15)
Given the properties of summation, (2.14) can be rewritten as (2.16) y = + x

0 1
or
0 = y 1 x
(2.17)
15/11/2011

10
Drop 1/n in (2.15) and plug (2.17) into (2.15):
x ( y [ y x ] x ) = 0 x ( y y ) = x ( x x ) ( xi x )( yi y ) = 1 ( xi x ) 2
i =1 i =1 i =1 n i i 1 i =1 i i n n i =1 n i i 1 1 i n
Provided that
( xi x ) 2 > 0
i =1
(2.18)
the estimated slope is
1 =
( x x )( y
i =1 i n i =1 i
y)
(2.19)
2
(x x)

11
Summary of OLS slope estimate: - Slope estimate is the sample covariance between x and y divided by the sample variance of x. - If x and y are positively correlated, the slope will be positive. - If x and y are negatively correlated, the slope will be negative. -Only need x to vary in the sample.
0 and 1 given in (2.17) and (2.19) are called the ordinary least squares (OLS) estimates of 0 and 1 .

12
To justify this name, for any 0 and 1 , define a fitted value for y given x = xi: (2.20) yi = 0 + 1 xi
The residual for observation i is the difference between the actual yi and its fitted value: ui = yi yi = yi 0 1 xi Intuitively, OLS is fitting a line through the sample points such that the sum of squared residuals is as small as possible term ordinary least squares. Formal minimization problem:
min ui2 = ( yi 0 1 xi ) 2
0 , 1
i =1 i =1
(2.22)
15/11/2011

13
Sample regression line, sample data points and residuals:
y y4
4 { } 3 2 {
y = 0 + 1 x
y3 y2
y1
.} 1
x1
x2
x3
x4

14
To solve (2.22), we obtain 2 first order conditions, which are the same as (2.14) and (2.15), multiplied by n.
Once we have determined the OLS 0 and 1 , we have the OLS regression line: (2.23) y = 0 + 1 x
(2.23) is also called the sample regression function (SRF) because it is the estimated version of the population regression function (PRF) E ( y | x) = 0 + 1 x . Remember that PRF is fixed but unknown. Different samples generate different SRFs.

15
Slope estimate 1 is of primary interest. It tells us the amount by which y changes when x increases by 1 unit. y = 1 x
E.g., we study the relationship between firm performance and CEO compensation. salary = 0 + 1 roe + u salary = CEOs annual salary in thousands of dollars, roe = average return (%) on the firms equity for previous 3 years. Because a higher roe is good for the firm, we think 1 > 0. Data set CEOSAL1 contains information on 209 CEOs in 1990. OLS regression line: salry = 963.191 + 18.501roe
(2.26)
15/11/2011

16
E.g., for the population of the workforce in 1976, let y = wage, $ per hour, x = educ, years of schooling. Using data in WAGE1 with 526 observations, we obtain the OLS regression line: wge = -0.90 + 0.54educ Implication of the intercept? Why? Only 18 people in the sample have less than 8 years of education. the regression line does poorly at very low levels. Implication of the slope?
(2.27)
2.3 Mechanics of OLS

17
Fitted Values and Residuals Given 0 and 1 , we can obtain the fitted value yi for each observation. Each yi is on the OLS regression line.
OLS residual associated with observation i, ui , is the difference between yi and its fitted value. If ui is positive, the line underpredicts yi.
If ui is negative, the line overpredicts yi.
In most cases, every ui 0 , none of the data points lie on the OLS line.

18
Algebraic Properties of OLS Statistics (1) The sum and thus the sample average of the OLS residuals is zero. n 1 n ui = 0 and thus n ui = 0 i =1 i =1 (2) The sample covariance between the regressors and the OLS residuals is zero. n xi u i = 0
i =1
(3) The OLS regression line always goes through the mean of the sample. y = + x
0 1
15/11/2011

19
We can think of each observation i as being made up of an explained part and an unexplained part yi = yi + ui . We define the following:
(y
i =1 n
i =1 n
y ) 2 is the total sum of squares (SST),

y ) 2 is the explained sum of squares (SSE),
is the residual sum of squares (SSR). SST = SSE + SSR
(y u
i =1 2 i
Then
(2.36)

20
Proof:
(y
i =1
y ) 2 = [( yi yi ) + ( yi y )]2 = [ui + ( yi y )]2 = ui2 + 2 ui ( yi y ) + ( yi y ) 2

i =1 i =1 n i =1 n n
= SSR + 2 ui ( yi y ) +SSE
i =1
i =1 n
i =1
and we know that
u (y
i =1 i
y) = 0

21
Goodness-of-Fit How well the OLS regression line fits the data? Divide (2.36) by SST to get: 1 = SSE/SST + SSR/SST The R-squared of the regression or the coefficient of determination SSE SSR R2 = 1 (2.38) SST SST It implies the fraction of the sample variation in y that is explained by the model. 0 R2 1
15/11/2011

22
E.g., CEOSAL1. roe explains only about 1.3% of the variation in salaries for this sample. 98.7% of the salary variations for these CEOs is left unexplained! Notice that a seemingly low R2 does not mean that an OLS regression equation is useless. It is still possible that (2.26) is a good estimate of the ceteris paribus relationship between salary and roe.
2.4 Units of Measurement

23
OLS estimates change when the units of measurement of the dependent and independent variables change. E.g., CEOSAL1. Rather than measuring salary in $000, we measure it in $, salardol = 1,000.salary. Without regression, we know that salrdol = 963,191 + 18,501roe. Multiply the intercept and the slope in (2.26) by 1,000 (2.40) have the same interpretations. Define roedec = roe/100 where roedec is a decimal. salry = 963.191 + 1850.1roedec.
(2.40) (2.26) and
(2.41)
2.4 Units of Measurement

24
What happens to R2 when units of measurement change? Nothing.
15/11/2011
2.4 Nonlinearities in Simple Regression

25
It is rather easy to incorporate many nonlinearities into simple regression analysis by appropriately defining y and x.
E.g., WAGE1. 1 of 0.54 means that each additional year of education increases wage by 54 cents. maybe not reasonable.
Suppose that the percentage increase in wage is the same given one more year of education. (2.27) does not imply a constant percentage increase. New model: log(wage) = 0 + 1 educ + u (2.42)
where log(.) denotes the natural logarithm.


26
For each additional year of education, the percentage change in wage is the same. the change in wage increases. (2.42) implies an increasing return to education. Estimating this model and the mechanics of OLS are the same: lg(wage) = 0.584 + 0.083educ (2.44) wage increases by 8.3 percent for every additional year of educ.

27
Another important use of the natural log is in obtaining a constant elasticity model. E.g., CEOSAL1. We can estimate a constant elasticity model relating CEO salary ($000) to firm sales ($ mil): log(salary) = 0 + 1 log(sales) + u (2.45) where 1 is the elasticity of salary with respect to sales. If we change the units of measurement of y, what happens to 1 ? Nothing.
15/11/2011
2.4 Meaning of Linear Regression

28
We have seen a model that allows for nonlinear relationships. So what does linear mean? An equation y = 0 + 1 x + u is linear in parameters, 0 and 1 . There are no restrictions on how y and x relate to the original dependent and independent variables. Plenty of models cannot be cast as linear regression models because they are not linear in their parameters. E.g., cons = 1/( 0 + 1 inc) + u
2.5 Unbiasedness of OLS

29
Unbiasedness of OLS is established under a set of assumptions: Assumption SLR.1 (Linear in Parameters) The population model is linear in parameters as y = 0 + 1 x + u (2.47)
where 0 and 1 are the population intercept and slope parameters. Realistically, y, x, u are all viewed as random variables. Assumption SLR.2 (Random Sampling) We can use a random sample of size n, {(xi,yi): i = 1, 2, , n}, from the population model.

30
Not all cross-sectional samples can be viewed as random samples, but many may be. We can write (2.47) in terms of the random sample as yi = 0 + 1 xi + ui , i = 1, 2, , n
(2.48)
To obtain unbiased estimators of 0 and 1 , we need to impose Assumption SLR.3 (Zero Conditional Mean) E(u|x) = 0 This assumption implies E(ui|xi) = 0 for all i = 1, 2, , n.
10
15/11/2011

31
Assumption SLR.4 (Sample Variation in the Independent Variable) In the sample, xi, i = 1, 2, , n are not all equal to a constant. This assumption is equivalent to
n
(x x)
i =1 i
n
>0
i
From (2.19) :
1 =
( x x )( y y ) ( x x ) y
i =1 i i
(x x)
i =1 i i 0
i =1 n
(x x)
i =1 i n
Plug (2.48) into this:
1 =
( x x )(
i =1 n i =1 i
+ 1 xi + ui )
2
(x x)
= 1 +
( x x )u
i =1 i
SSTx

32
Errors uis are generally different from 0.
1 differs from 1 .
The first important statistical property of OLS: Theorem 2.1 (Unbiasedness of OLS) Using Assumptions SLR.1 through SLR.4, E ( 0 ) = 0 , and E ( 1 ) = 1 (2.53)
The OLS estimates of 0 and 1 are unbiased.
Proof: E ( 1 ) = 1 + E[(1 / SSTx ) ( xi x )ui ]

i =1
= 1 + (1 / SSTx ) ( xi x )E (ui ) = 1
i =1

33
(2.17) implies 0 = y 1 x = 0 + 1 x + u 1 x = 0 + ( 1 1 ) x + u E ( ) = + E[( ) x ] =

0 0 1 1 0
Remember unbiasedness is a feature of the sampling distributions of 0 and 1 .It says nothing about the estimate we obtain for a given sample. If any of four assumptions fails, then OLS is not necessarily unbiased. When u contains factors affecting y that are also correlated with x can result in spurious correlation.
11
15/11/2011

34
E.g., let math10 denote % of tenth graders at a high school receiving a passing score on a standardized math exam. Let lnchprg denote % of students eligible for the federally funded school lunch program. We expect the lunch program has a positive effect on performance: math10 = 0 + 1 ln chprg + u MEAP93 has data on 408 Michigan high school for the 1992-1993 school year. mth10 = 32.14 0.319lnchprg Why? u contains such as the poverty rate of children attending school, which affects student performance and is highly correlated with eligibility in the lunch program.
2.5 Variances of the OLS Estimators

35
Now we know that the sampling distribution of our estimate is centered about the true parameter. How spread out is this distribution? We need to add an assumption. Assumption SLR.5 (Homoskedasticity)
2 Var(u|x) =
the variance.
This assumption is distinct from Assumption SLR.3: E(u|x) = 0. This assumption simplifies the variance calculations for 0 and 1 and it implies OLS has certain efficiency properties.

36
2 Var(u|x) = E(u2|x) [E(u|x)]2 = E(u2|x) = 2 Var(u) = E(u2) =
2 is often called the error variance. , the square root of the error variance, is called the standard
deviation of the error. We can say that
E ( y | x ) = 0 + 1 x
(2.55) (2.56)
Var ( y | x ) = 2
12
15/11/2011

37
Homoskedastic case:
y
f(y|x)
E(y|x) = 0 + 1x
x1
x2

38
Heteroskedastic case:
f(y|x)
E(y|x) = 0 + 1x
x1
x2
x3

39
Theorem 2.2 (Sampling variances of the OLS estimators) Under Assumptions SLR.1 through SLR.5,
Var ( 1 ) =
(x x)
i =1 i
=
2
2
SSTx
(2.57)
and
Var ( 0 ) =
2
n
1 n 2 xi n i =1
2 i
(2.58)
(x x)
i =1
2
Proof: Var ( 1 ) =
1 SSTx2
( x x ) Var (u ) = SST
i =1 i i
SSTx
2 x
2 =
2
SSTx
13
15/11/2011

40
(2.57) and (2.58) are invalid in the presence of heteroskedasticity. (2.57) and (2.58) imply that: (i) The larger the error variance, the larger are Var( j ) .
(ii) The larger the variability in the xi, the smaller are Var( j ).
2 Problem: the error variance is unknown because we dont observe the errors, ui.
2.5 Estimating the Error Variance

41
What we observe are the residuals, ui . We can use the residuals to form an estimate of the error variance.
We write the residuals as a function of the errors: ui = yi 0 1 xi = ( 0 + 1 xi + ui ) 0 1 xi
ui = ui ( 0 0 ) ( 1 1 ) xi
2 An unbiased estimator of is 1 n 2 SSR 2 = ui = n 2 n 2 i =1
(2.59)
(2.61)
2.5 Estimating the Error Variance

42
= 2 = standard error of the regression (SER).

Recall that sd ( 1 ) =
SSTx
SSTx
2 , if we substitute for 2 , then we
have the standard error of 1 :

se( 1 ) = = n ( xi x ) 2 i =1
14
15/11/2011
2.6 Regression through the Origin

43
In rare cases, we impose the restriction that when x = 0, E(y|0) = 0. E.g., if income (x) is zero, income tax revenues (y) must also be zero. Equation
~ ~ y = 1 x + u
(2.63)
Obtaining (2.63) is called regression through the origin. We still use OLS method with the corresponding first order condition
x (y
i i =1
1 xi ) = 0
1 =
x y
i =1 n i
x
i =1
(2.66)
2 i
~ If 0 0 , then 1 is a biased estimator of 1.
15

Chap 2

Загружено:

Сведения о документе

Исходное описание:

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Chap 2

Загружено:

Авторское право:

Доступные форматы

15/11/2011

Introductory Econometrics: A modern approach (Wooldridge)

The Simple Regression Model

Dr. L Vn Chn FTU, 2011

2.1 Definition of Simple Regression Model

2.1 Definition of Simple Regression Model

1 is the slope parameter. This is of primary interest in applied economics.

Dr. L Vn Chn FTU, 2011

2.1 Definition of Simple Regression Model

2.1 Definition of Simple Regression Model

2.2 Ordinary Least Squares

Dr. L Vn Chn FTU, 2011

2.2 Ordinary Least Squares

PRF, sample data points and the associated error terms:

Dr. L Vn Chn FTU, 2011

2.2 Ordinary Least Squares

2.2 Ordinary Least Squares

Given the properties of summation, (2.14) can be rewritten as (2.16) y = + x

2.2 Ordinary Least Squares

Drop 1/n in (2.15) and plug (2.17) into (2.15):

the estimated slope is

Dr. L Vn Chn FTU, 2011

2.2 Ordinary Least Squares

Dr. L Vn Chn FTU, 2011

2.2 Ordinary Least Squares

Dr. L Vn Chn FTU, 2011

2.2 Ordinary Least Squares

Sample regression line, sample data points and residuals:

Dr. L Vn Chn FTU, 2011

2.2 Ordinary Least Squares

2.2 Ordinary Least Squares

2.2 Ordinary Least Squares

2.3 Mechanics of OLS

Dr. L Vn Chn FTU, 2011

2.3 Mechanics of OLS

Dr. L Vn Chn FTU, 2011

2.3 Mechanics of OLS

y ) 2 is the total sum of squares (SST),

2.3 Mechanics of OLS

y ) 2 = [( yi yi ) + ( yi y )]2 = [ui + ( yi y )]2 = ui2 + 2 ui ( yi y ) + ( yi y ) 2

and we know that

Dr. L Vn Chn FTU, 2011

2.3 Mechanics of OLS

Dr. L Vn Chn FTU, 2011

2.3 Mechanics of OLS

Dr. L Vn Chn FTU, 2011

2.4 Units of Measurement

(2.40) (2.26) and

2.4 Units of Measurement

What happens to R2 when units of measurement change? Nothing.

Dr. L Vn Chn FTU, 2011

2.4 Nonlinearities in Simple Regression

where log(.) denotes the natural logarithm.

2.4 Nonlinearities in Simple Regression

Dr. L Vn Chn FTU, 2011

2.4 Nonlinearities in Simple Regression

Dr. L Vn Chn FTU, 2011

2.4 Meaning of Linear Regression

Dr. L Vn Chn FTU, 2011

2.5 Unbiasedness of OLS

2.5 Unbiasedness of OLS

2.5 Unbiasedness of OLS

Plug (2.48) into this: