Quantitative Methods by SM

9/19/14
Simple Linear Regression
Thanks to Arun Kumar, Ravindra Gokhale, and Nagarajan Krishnamurthy
Sujay K Mukhoti
Quantitative Techniques-I
Indian Institute o Management Indore
Quantitative Methods by SM
9/19/14
Relationship !et"een t"o varia!les

Y: Expenditure on redit ard

!: limit on the redit ard

"oes higher limit result in higher expenditure#

Quantitative Methods by SM
9/19/14
#ata and Scatter $lot
Expenditure Credit Limit
18008.0!1" #844".01$81
1$!$!.89981 "!"#!.!!"
1"0!$.!9"81 ##9!."4##
1#"4.408" #$1#!.!#"#8
84$".10"4# #9$"$.19#"$
1"11!.10$## 084!."48
184#4.$4!!8 1"".0#18
18$89.!0#81 8"4#.4$084
1988.$$!! !190.88$$4
1841."88! #8490.1$"1#
Quantitative Methods by SM
9/19/14
%orrelation and %ausality

Any relation $et%een ! & Y: '()*+,

is symmetri and %e seek ausality

Assumption: one %ay ausal relation

-tatistially: is ! ause o. Y#

Y: dependent

!: independent

Y' f/!0 #

Not all f : only linear .untions

0 / 0 /
0 , /
Y V X V
Y X Cov
=
Quantitative Methods by SM
9/19/14
Linear relation or one "ay causality

-imple 1inear Regression 2odel o. Y on !:

Y ' $
(
3 $
4
! 3 5

Y: "ependent varia$le 6 random varia$le

!: 7ndependent varia$le 6 non8random or 9given .ats:, no error

in measurement

5 : error due to unkno%n auses, exluded varia$les, exogenous

/maro8eonomi0 .ators or even 9$lak s%an: events
$
(
: general level o. Y , average Y %ithout making any
o$servation on !
$
4
: rate o. hange o. Y %rt !, slope
Quantitative Methods by SM
9/19/14
&ssumptions'

-imple 1inear Regression 2odel .or o$served data:

Y
i
' $
(
3 $
4
!
i
3 5
i
E;5
i
< ' ( , i.e. E;Y
i
= !
i
< ' ' $
(
3 $
4
!
i
>/5
i
0 ' ?
@
: onstant
Aov/5
i
, 5
j
0 ' Aov/y
i
, y
j
0 ' ( : individuals are independent
5
i
B N/(, ?
@
0 i.e. Y
i
B N/$
(
3 $
4
!
i
, ?
@
0 and iid)
Quantitative Methods by SM
9/19/14
Method o (stimation' )rdinary Least Square

-tandard -imple 1inear Regression

2odel:
Y
i
' $
(
3 $
4
!
i
3 5
i

- '

Ahoose the line %hih gives

minimum -
Ahoie is .or b
0
and b
4

=

n
i
i i
X b b y
4
@
4 (
0 /
Quantitative Methods by SM
9/19/14
)LS estimates

Citted line and predited value:

--E'
2-E ' --ED/n8@0 B t
n8@
, s '
x
y
xy
x
s
s
r
s
Y X Cov
b = =
@
4
0 , /
E
x b y b
4 (
E
=

=
@ @
4 (
0
E
/ 0
E E
/
i i i i
Y Y X b b Y
i i
X b b Y
4 (
E E E
+ =
MSE
Quantitative Methods by SM
9/19/14
(*%(L )utputs
Fhat is ANG>A in regression#

ANG>A : Cator and treatments

Regression is Cator ating on data:

Treatment 4: Y'$
(
3 5
i
+,-LL M)#(L.
Treatment @: Y'$
(
3 $
4
! 3 5
i
+/-LL
M)#(L.

"C .or the Cator 9Regression:: @84 '4

"C .or the errors : n-2

Quantitative Methods by SM
9/19/14
)LS estimates

Aon.idene 7nterval o. estimated o8e..iient

-igni.iane o. the varia$le: Test o. hypothesis

@
@
4
, 0
E
/

= =

n
X
X SS
SS
s
b SE
i
i X
X
X
i
nSS
X s
b SE

=
@
(
0
E
/
@
4
4
@
(
(
B
0
E
/
E
, B
0
E
/
E
n n
t
b SE
b
t
b SE
b
Quantitative Methods by SM
9/19/14
%o-eicient o determination ' 0oodness o it

--T ' --R 3 --E

: good .it i. lose to one

Test o. hypothesis: Ho% good is the .it : E!AE18ANGRE ta$le

Adjusted
i i i
i
Y Y Y Y
Y Y
Deviation Error deviation Explained
Deviation otal
E E

+ =
SS
SSE
SS
SS!
! = = 4
@
0 4 D/
0 @ D/
4
D
D
4
@

= =
n SS
n SSE
df SS
df SSE
!
SS
SSE
Quantitative Methods by SM
9/19/14
$ilgrim 1ank ' %ase o SLR model

Alan Green has $een given the responsi$ility o. .inding out

%hether a .ee should $e harged .or online $anking or inentives
should $e given .or online $anking)

>aria$le seletion .rom domain kno%ledge: Ialane or pro.it#

Jro.it'/Ialane in "eposit Aounts0 K /Net 7nterest

-pread03Cees37nterest .rom 1oans8Aost to serve

Gnline $anking is one o. the the least ostly hannel through

%hih $anks provide servie to ustomers /$ranhes, AT2
mahine, @L hour all entre, automated voie response units0)
Quantitative Methods by SM
9/19/14
$ilgrim 1ank ' %ase o SLR model
"t# "
=
"t# "

Fhy online only#

2ost ost saving: Yet ))) may not $e most pro.ita$le

"ata: -ample o. M((((

Cirst test: >s means are di..erent

T%o sample t8test:

Gnline G..line
N MN*L @++N(
2ean 44O)OO+ 44()+NO
-t "ev @NM)OO* @+4)M(4
Quantitative Methods by SM
9/19/14
$ilgrim 1ank ' %ase o SLR model

7s there any di..erential e..et among young and old#

2odel:

Jro.it'$
(
3 $
4
AP A'Age

Regression -ummary .or N+ online hannel users:

2ean pro.it : +@)OM, sd/pro.it0 ' @@4)(,

2ean age: M)ML , sd/Age0 ' 4)M+

Aor/pro.it, age0 ' ()4(LP R

@
' ()(4(+
Quantitative Methods by SM
9/19/14
$ilgrim 1ank ' %ase o SLR model

7s there any di..erential e..et o. hannel among young and old

#

2odel:

Jro.it'$
(
3 $
4
G 3$
@
AP G ' dummy .or onlineP
A'Age

Result: R
@
' ()4NMMP

Aoe..iients:
Estimate -td) Error t value Jr/Q=t=0
$4 M@)*+,L *)@NN@ O)4O4 +)MOe84( RRR
$@ @,)MO(M ()LMMN O+)ONO S @e84O RRR
Model 1uilding
Quantitative Methods by SM
9/19/14

"etermine the dependent and independent varia$les

Jlot satter diagram: exlude spurious orrelation

>alidate inlusion o. interept $ased on domain

kno%ledge) e.$. Feight versus volume

Ahek .or regime hange: dummy varia$le

Model 1uilding
Quantitative Methods by SM
9/19/14

>alidate the assumptions :

(rror mean 2ero: No ! is le.t out %hih may

have signi.iant e..et on average Y

(rror 3ariance is constant: Jort.olio does

not inlude 9market makers:: e)g Ced) Iank is
not in your sample 8 it may set exogenous
.ators to ontrol ?
@

(rrors are uncorrelated aross sampled ases:

Your ustomers are not herding to reate trou$le:
No money launderingD terrorist .unding
Model 1uilding
Quantitative Methods by SM
9/19/14

%heck or outliers and exlude : -peial treatment

.or speial ones

Treat missing values

Run regression and determine goodness o. .it

Test signi.iane o. varia$les

Re8run a.ter dropping insigni.iant varia$les

Jredit $ased on .itted model

Quantitative Methods by SM
9/19/14
THANK YOU
&
ALL THE BEST