ECON 332

Business Forecasting Methods

Multiple Regression Models
Chapter 5
Prof. Kirti K. Katkar
5-2
Multiple Regression Models
• An extension of simple bivariate regression models
• The dependent variable Y is a function of multiple
independent variables X1, X2, ….., Xn.
Y = f (X1, X2, ….., Xn)
= βo + β1X1 + β2X2 + ………. +βnXn +ε
where βo is the intercept and βis are slope terms associated
with respective independent variables and ε represents the
population error term i.e. (Actual Y – Predicted Ŷ per the
model)
• The ordinary least squares criterion for multiple regression
models is
min Σε2 = Σ (Y-Ŷ)2
= Σ (Y - βo - β1X1 - β2X2 + ………. -βkXk)2
5-3
Multiple Regression Models (Contd.)
• Values of the true regression parameters βi are estimated from sample data.
The resulting sample multiple regression model is
Y = = bo + b1X1 + b2X2 + ………. +bkXk
where bi are sample statistics and are best estimates of population
parameters βi.
• Residuals – forecast errors - deviations are Y – Ŷ
• Invariably bi are determined by computer software packages which also
provide the model evaluation statistics such as
– Standard errors
– t ratios and F ratios
– Coefficient of determination R2
– Adjusted coefficient of determination – Adjusted R2
– Durbin-Watson statistic
in addition to the table of residuals
5-4 Multiple Regression Challenges:
Selecting Dependent/ Independent Variables
• Dependent variable tends to be a prime mover such as
disposable personal income (DPI), or an effect of many
actions (causes) such as company sales, patient-days in a
hospital etc. These are the first ones to be chosen.
• To choose independent variables
– Think of what contributes to changes in the dependent variable
• DPI impact on Retail Sales (RS)
• Advertising/ Promotion impact on short term product sales
• Affluence factor on Police Service calls
– Make sure the chosen independent variables do not measure the same
fundamental phenomenon
• DPI and Gross Domestic Product (GDP)
• Population and DPI
• % Employed and % Unemployed
5-5 Multiple Regression Challenges:
Selecting Dependent/ Independent Variables (Contd.)
• To choose independent variables (Contd.)
– Avoid the problem of multicollinearity
• Find and use only those variables that are not highly correlated with one
another
– Lastly make sure that independent variables can be forecast into future
5-6
Forecasting RS with Multiple Regression Model

RSF1 = 1,052,224 - 59,926 MR (Mortgage Interest Rate)

RSF2 = - 1,422,517 + 110 DPI – 9,945 MR (Disposable Personal Income)
5-7

DPI & MR Data and Multiple Regression Results for RS (RSF2)

5-8
DPI & MR Data and Multiple Regression Results for RS (RSF2) (Contd.)
5-9
Actual RS vs. Forecasted RS using RSF2

RSF2 = - 1,422,517 + 110 DPI – 9,945 MR (Disposable Personal Income)

5-10
Actual RS vs. Forecasted RS using RSF1

RSF1 = 1,052,224 - 59,926 MR (Mortgage Interest Rate)

5-11
Three Dimensional View of RS , MR and DPI
Regression Plane Concept
5-12
Three Dimensional View of RS , MR and DPI:
Regression Plane Superimposed
5-13

Process for Statistical Evaluation of Multiple

Regression Results
1. Do the results make sense in terms of the slope sign?
2. Is the slope term statistically +ve or –ve at the desired
significance level using the t test?
3. How much of the variation in the dependent variable is
explained by using the R-squared value?
4. Does the model exhibit serial correlation? Use of Durbin-
Watson statistic
5. F statistic to ensure that all slope terms are non-zero
6. Multicollinearity – are the independent variables chosen
highly correlated with each other?
5-14

Statistical Evaluation of Slope Signs

• Does the sign ( + or -) make sense?
– In the RS case, we have RS = -1,422,517 – 9,945 MR + 110.77 DPI
– Slope of -9,945 for Mortgage Rate makes sense
– Slope of +110.77 for Disposable Personal Income makes sense
• What if the signs do not make sense?
– Clear indication that the model is wrong
– Model could be incomplete and may require more than two
independent variables to explain the underlying phenomenon i.e. it is
under-specified
– Should never use models where signs do not make sense
• If the signs do make sense, is the slope significantly positive
or negative?
– The slope closer to zero would indicate that there is no linear
relationship between X and Y, rather Y and X are completely
independent of each other
5-15

Are the slope terms statistically +ve or –ve at the desired

significance level using the t test?

•The appropriate test here is t-test and for 95% confidence level or 5%
significance level with df = n – (K+1), where
–n = # of observations
–K = # of independent variables
For DPI and MR we need to test
For MR For DPI
H0:β1≥ 0 H0:β2≤ 0
H1:β1< 0 H1:β2> 0

• Now tcalc = (bi-0)/ Standard Error of Estimate (SEE) bi

And critical value of t at 5% significance and df = 36-(2+1)=33 is 1.645
•Since t distribution is symmetrical we always consider absolute value of
tcalc
5-16

Are the slope terms statistically +ve or –ve at the desired

significance level using the t test? (Contd.)

• From the ForecastX output we find that tcalc for MR = -1.3 and tcalc for
DPI = 9.66.
For MR For DPI
H0:β1≥ 0 H0:β2≤ 0
H1:β1< 0 H1:β2> 0
Abs tcalc = 1.3 Abs tcalc = 1.3
tT = 1.645 tT = 9.66
Since 1.3 is not ≥ but<1.645, Since 9.66>1.645,
We accept H0 i.e. at 5% We reject H0 i.e. at 5%
significance level we cannot significance level we can say
say that the slope of MR is that the slope of DPI is
significantly -ve significantly +ve
At 10%( tT = 1.282) or higher significance level
we will reject H0 for MR
5-17
How much of the variation in the dependent
variable is explained by using the Coefficient of
Determination?

• R2 is the coefficient of determination and ranges between 0

and 1
• R2 should be as close to 1 as possible
• For Multiple Regression models we modify the R2 by the
degrees of freedom or by the # of independent variables
used. This is called Adjusted R2 or R-bar-squared and is
used for evaluation
Adjusted R2 = 1- (Σei2/(n-k+1))/(Σ(Yi – Y)2/(n-1)) or
= R2 - (1-R2)(k/(n-k+1))
5-18
How much of the variation in the dependent
variable is explained by using the Coefficient of
Determination?
• R2 can be statistically tested to see if it is different from zero
using the F statistic. Since R2 measures the degree of linear
relationship between dependent and independent variables, the
hypothesis are constructed as
H0: β1 = β2 = ……… = βk = 0 All βis are simultaneously zero
H1: All βis are not simultaneously zero
ExplainedVariation K
• Fcalc = Un − ExplainedVariation [ n − ( K + 1)]

• To test the hypothesis, Fcalc is compared with FT from the table

for K df in numerator and n-(K+1) df in the denominator
• We reject the H0 if Fcalc> FT i.e. the multiple regression model
passes the F test.
5-19
F distribution @ 95% confidence level or 5%
significance level
5-20

F distribution @ 95% confidence level or 5% significance

level (Contd.)
5-21
Multicollinearity
• As covered earlier, a major assumption in multiple regression
is that independent variables are not correlated with each
other or with linear combinations of other independent
variables
• If this is violated, multicollinearity results. The slope signs
differ from what would make sense and large forecast errors
occur
• Illustration: RS = b0 + b1MR + b2DPI + b3BI – we have added
a third independent variable BI – Bond Interest to model
Retail Sales (RS). The economic theory and reality would lead
us to expect 1. b1 and b3 to be –ve and 2. b2 to be +ve. The
model results per ForecastX are:
RS = -1,354,134 + 8,018MR +108DPI – 20,490BI
+ve b1=8,018 makes no sense.
5-22
Multicollinearity
• As covered earlier, a major assumption in multiple regression
is that independent variables are not correlated with each
other or with linear combinations of other independent
variables
• If this is violated, multicollinearity results. The slope signs
differ from what would make sense and large forecast errors
occur
• Illustration: RS = b0 + b1MR + b2DPI + b3BI – we have added
a third independent variable BI – Bond Interest to model
Retail Sales (RS). The economic theory and reality would lead
us to expect 1. b1 and b3 to be –ve and 2. b2 to be +ve. The
model results per ForecastX are:
RS = -1,354,134 + 8,018MR +108DPI – 20,490BI
+ve b1=8,018 makes no sense.
5-23
Multicollinearity Deciphered
• To understand the source of Multicollinearity it is useful to
look at the correlation matrix. In the case of 3 independent
variable RS model we have as correlations
DPI MR BI
DPI 1
MR -0.6745 1
BI -0.7154 0.989759 1
• Very strong linear relationship exists between MR and BI. In
fact they both measure the same economic phenomenon.
• To avoid Multicollinearity, no firm rules - but rules of thumb
– Avoid using independent variables which have correlation very close
to 1 – e.g. MR and BI as above
– Avoid situations where correlation between independent variables is
higher than their correlations with the dependent variable.
• Modeling the first difference rather than the original series can
also help here.
5-24

Does the model exhibit serial correlation?

• One of the major assumptions in the ordinary least squares

regression model is that the error terms are independent and
normally distributed. This implies that there is no pattern
between errors.
• If the serial correlation between errors exists, it estimates the
standard error of estimate to be smaller than they really are. It
does not impact the estimates of slope and intercept.
• This would mean that the calculated t would be overstated and
that we may reject the null hypothesis that should not be
rejected.
5-25

Examples of +ve and –ve Serial Correlation

-ve Serial Correlation +ve Serial Correlation Errors follow

Errors alternate in sign previous error’s sign. Most
common in business/ economic data
5-26

Statistical Test for Existence of Serial Correlation

Durbin-Watson (DW) Statistic

• DW statistic is calculated as ∑ (et − et − 1) ∑

2
et
2

where et is the residual for time period t

and et-1 is the residual for previous time period t-1
• DW statistic will always range between 0 and 4
• A value closer to 2, say between 1.75 and 2.25 indicates that
there is no serial correlation
• A value closer to 0 would indicate positive serial correlation
• A value closer to 4 would indicate negative serial correlation
• For precise evaluation we use the DW table
5-27
Schematic for Evaluating Serial Correlation
5-28
The DW Table
5-29
The DW Table (Contd.)
5-30

Bivariate RS Model Statistics

5-31

Multiple Regression RS Model Statistics

5-32

Application of DW Statistic to the RS

Models
• The bivariate model RS = b0 + b1MR had DW = 0.97 thereby
leading us to conclude that there was a +ve serial correlation
• The multiple regression model RS = b0 + b1MR + b2DPI has
DW = 2.6 . And for K=2 and N=36, from DW Table we have
du=1.59 and dl=1.35 . This falls in Region B since,
4-du<DW<dl, i.e. (4-1.59)<2.6<(4-1.35), or 2.41<2.6<2.65
• This will lead us to conclude that the results are indeterminate
i.e. we cannot statistically confirm that there is –ve serial
correlation.
• A common reason for serial correlation is that the model is
underspecified and has not taken into consideration most
important explanatory variable. This can be fixed by adding
the independent variable to the model.
5-33
Alternative Variable Selection – Addition
or Deletion - Criteria
• The initial step is to choose independent variables based on solid
understanding of the phenomenon being modeled.
• The addition or deletion of a variable can be achieved by using two other
statistical criteria – AIC and BIC statistics.
• AIC – Akaike Information Criterion – provides accuracy of estimation and
best approximation to reality. It is based on the measure of accuracy and
parsimony (the fewest independent variables the better). The best model
has the minimum AIC statistic.
• BIC/ SIC – Bayesian/ Schwartz Information Criterion - is similar to AIC. It
uses the Bayesian arguments about the prior probability of “true” model to
indicate the best model. The best model has minimum BIC statistic.
• In choosing the best multiple regression model, besides high Adjusted R2;
it is always useful to look at AIC and BIC statistics.
• In fact experimental results indicate that AIC and BIC are BETTER
measures than Adjusted R2 given that the chosen independent variables
make Business/ Economic sense.
5-34

Including Seasonality in Multiple

Regression Models
• This is done by including a dummy variable. It assumes values
of 0 when seasonality is absent and 1 when it is present. This
is similar to how we modeled promotions using the event
model.
• Several useful applications would include modeling
– Beer sales
– Christmas sales
– Private housing starts
5-35
Dummy Variable Illustration: Private Housing Starts (PHS)

PHS = b0 + b1MR + b2Q2 +b3Q3 + b4Q4

5-36
Modeling Non-linear Phenomenon Using Multiple
Regression Models
• This makes sense where the dependent variable has non-linear
relationship with independent variable(s) such as diminishing
returns modeling.
• Some forms this can take are:
Y = b0 + b1X +b2X2 + …….. + bnXn
Y = eaXb