WorkedExamplesforChapter16

ExampleforSection16.4
ConsidertheMarkovchainthathasthefollowing(onestep)transitionmatrix.
State01234

P=

0 0 0.2 0.5 0.3 0

1 0
0
0
1
0

2 0 0.2 0 0.1 0.7

3 0
1
0
0
0

4 0.8
0.1 0 0.1 0

(a)DeterminetheclassesofthisMarkovchainand,foreachclass,determine
whetheritisrecurrentortransient.
States1and3areaccessiblefromeachother(p31=1andp13=1),butnoother
statesareaccessiblefromthesestates(p1j=0andp3j=0forj=0,2,4).Therefore,states
1and3communicateandformoneclassoftheMarkovchain.Uponenteringeitherstate,
theprocesswillreturntothatstateintwosteps,so{1,3}isarecurrentclass.
State0isaccessiblefromstate4(p40=0.8),state2isaccessiblefromstate0(p02
=0.5),andstate4isaccessiblefromstate2(p24=0.7),soeachofthesestatesis
accessiblefromeachoftheseotherstates.Therefore,states0,2,and4communicateand
formasecondclassoftheMarkovchain.Theprocesscanmovefromanyofthesestates
tostate1orstate3,inwhichcasetheprocessneverwouldreturntostates0,2,and4
again.Therefore,{0,2,4}isatransientclass.
(b)Foreachoftheclassesidentifiedinpart(a),determinetheperiodofthestatesin
thatclass.
WecalculateP(2)andP(3).

0
0

P(2)=P*P= 0.56

0.4
1
0.17
0
0.26

0
0
0
0
0.4

0.25
0
0.27
1
0.34

0.35
0

0 .

0
0

0.28
0

P(3)=P(2)*P= 0

0.285
0
0.382
1
0.42

0
0
0.28
0
0

0.435
1
0.338
0
0.3

0
0

0 .

0
0.28

Sincep11=p33=0andp(2)11=p(2)33=1,theclass{1,3}hasperiod2.
Nownotethatp00=p22=p44=0,p(2)00=p(2)22=p(2)44=0,andp(3)000,p(3)220,
(3)
p 440.Thisindicatesthattheclass{0,2,4}hasperiod3.

ExampleforSection16.5
Asoapcompanyspecializesinaluxurytypeofbathsoap.Thesalesofthissoapfluctuate
betweentwolevels"Low"and"High"dependingupontwofactors:(1)whether
theyadvertise,and(2)theadvertisingandmarketingofnewproductsbeingdoneby
competitors.Thesecondfactorisoutofthecompany'scontrol,butitistryingto
determinewhatitsownadvertisingpolicyshouldbe.Forexample,themarketing
manager'sproposalistoadvertisewhensalesarelowbutnottoadvertisewhensalesare
high.Advertisinginanyquarterofayearhasitsprimaryimpactonsalesinthefollowing
quarter.Therefore,atthebeginningofeachquarter,theneededinformationisavailable
toforecastaccuratelywhethersaleswillbeloworhighthatquarterandtodecidewhether
toadvertisethatquarter.
Thecostofadvertisingis$1millionforeachquarterofayearinwhichitisdone.
Whenadvertisingisdoneduringaquarter,theprobabilityofhavinghighsalesthenext
quarteris1/2or3/4,dependinguponwhetherthecurrentquarterssalesareloworhigh.
Theseprobabilitiesgodownto1/4or1/2whenadvertisingisnotdoneduringthecurrent
quarter.Thecompanysquarterlyprofits(excludingadvertisingcosts)are$4million
whensalesarehighbutonly$2millionwhensalesarelow.(Hereafter,useunitsof
millionofdollars.)
(a)Constructthe(onestep)transitionmatrixforeachofthefollowingadvertising
strategies:(i)neveradvertise,(ii)alwaysadvertise,(iii)followthemarketing
managersproposal.
Letstate0indicatetheLowlevelofsalesandstate1indicatetheHighlevelofsales
duringthecurrentquarter,whereeachtransitionoftheprocessgoesfromonequarterto
thenext.
.
(i)Theonesteptransitionmatrixfortheneveradvertisestrategyis

State01
P=

0
1

3 / 4 1 / 4
1 / 2 1 / 2 .

(ii)Theonesteptransitionmatrixforthealwaysadvertisestrategyis

State01
0 1 / 2 1 / 2
.
P=
1 1/
4 3 / 4
(iii)Theonesteptransitionmatrixforthemarketingmanagersproposalis

State01
P=

0
1

1 / 2 1 / 2
1 / 2 1 / 2 .

(b)Determinethesteadystateprobabilitiesmanuallyforeachofthethreecasesin
part(a).
Todeterminethesteadystateprobabilities(0,1)foreachofthetransitionmatrices
definedabove,wesolvethefollowingsystemoflinearequations:
0 =0p00+1p10
1 =0p01+1p11
1 =0 +1,
whereoneofthefirsttwoequations(say,thesecond)isredundantandsocanbeignored.
For(i)theneveradvertisestrategy,thissystemofequationsbecomes

so

3
1
0= 0+ 1,
4
2

0=21.
Since
0+1=1,

thisimpliesthat0=2/3,1=1/3.
Proceedinginthesamewayfortheothertwooptionsyieldsthefollowingresults.
For(ii)thealwaysadvertisestrategy,0=1/3,1=2/3.
For(iii)themarketingmanagersproposal,0=1/2,1=1/2.
(c)Findthelongrunexpectedaverageprofit(includingadeductionforadvertising
costs)perquarterforeachofthethreeadvertisingstrategiesinpart(a).Whichof
thesestrategiesisbestaccordingtothismeasureofperformance?
Recallthatthecompanysquarterlyprofits(excludingadvertisingcosts)are$4
millionwhensalesarehigh(State=1)and$2millionwhensalesarelow(State=0).
Therefore,inunitsofmillionsofdollars,thelongrunexpectedaveragecost(excluding
advertisingcosts)is20+41.Totakeadvertisingcosts($1million)intoaccount,we
needtosubtract1fromeachcoefficientwhereadvertisingisdone.Thisleadstothe
calculationsshownbelow.
For(i)theneveradvertisestrategy,thelongrunexpectedaverageprofitis
profit=2(2/3)+4(1/3)=$8/3million.
For(ii)thealwaysadvertisestrategy,thelongrunexpectedaverageprofitis
profit=2(1/3)+4(2/3)1=$7/3million.
For(iii)themarketingmanagersproposal,thelongrunexpectedaverageprofitis
profit=(21)(1/2)+4(1/2)=$5/2million.
Therefore,whentheobjectiveistomaximizethelongrunexpectedaverage
profit,thebeststrategyisneveradvertise.