Better Bootstrap Confidence Intervals: Comment

Author(s): Wei-Yin Loh and C. F. J. Wu

Source: Journal of the American Statistical Association, Vol. 82, No. 397 (Mar., 1987), pp. 188-
190
Published by: American Statistical Association
Stable URL: http://www.jstor.org/stable/2289147 .
Accessed: 10/06/2014 13:17

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp

.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.

American Statistical Association is collaborating with JSTOR to digitize, preserve and extend access to Journal
of the American Statistical Association.

http://www.jstor.org

This content downloaded from 62.122.79.46 on Tue, 10 Jun 2014 13:17:07 PM

All use subject to JSTOR Terms and Conditions
188 Journal of the American Statistical Association, March 1987

1, andcorrelation 0, andconfidence intervalsin thisone- interval,replacingobservedwithexpectedinformation.

parameter familyare to be based on the MLE 0. In a The conditional interval does notagreewiththeBCa in-
simulation ofthisproblemwith0 = .9,theBCa procedure tervalto Op(lIn) (inthesenseofRemarkG), becausethe
performed poorly,shifting thepercentile intervalin the conditioning is an Op(1I\/n)effect. Can theBCa method
wrongdirectionand thusachievinga coveragelevel of be adaptedto produceintervals withgood conditional
71% insteadof thetarget90%. In thecalculationof a, properties?
thedensityof 0 was too difficult to compute,so we re- 4. Are the intervalsdiscussedin Section7 nonpara-
placedthelikelihood basedon 0 withthelikelihood based metricin the"distribution-free" sense?
on theentiredataset(see Efron'sRemarkF). We suspect 5. A different approachto multiparameter and hence
thatthesepoorresultsmayhavearisenfromthecompu- nonparametric confidence interval construction canbe de-
tationofa, becausea andexpression (4.1) forzowerevery velopedthrough the"profile likelihood." Theprofile like-
different (largenegativevaluesfora, largepositiveones lihoodis constructed by maximizing overall of thenui-
forzo). Now Efron'sTheorem2 saysthatzo and goagree sanceparameters foreach fixedvaluefortheparameter
to Op(lln), henceit appearsthata is not estimating go ofinterest 0. Fromthis,one canconstruct a second-order
verywell.Unfortunately, directcomputation of gois dif- correct confidence interval byinverting thebias-corrected
ficult.We wonderifEfronwouldcomment on thediffi- signedsquarerootofthelog-likelihood ratiostatistic.By
cultyarisinghereand on therobustness offormula (4.4). carrying thisoutin themultinomial family, we geta non-
It is importantto add thatifone insteadbasestheconfi- parametric likelihoodand confidenceintervalfor0. It
denceintervalon thesamplecorrelation coefficient (the turnsout, however,thatin the multinomial the profile
morecommonapproach),theBCa interval workswell. likelihoodis verydifficult to compute;thisproblemcan
2. As his firstexample,Efronconsidersthe Fieller be alleviatedthrough use ofStein'sleastfavorable family
problemin thecase (Yl, Y2) = (8, 4), forwhichtheBC as an approximation. We notethatthisprocedure hastwo
methodcloselymatchestheexact(Fieller)interval. In'the potentialadvantagesovertheBCa procedure:(a) it has
moreextremecase (Yl, Y2) = (1, 15), theexactinterval goodconditional as wellas unconditional properties, and
is the exclusive interval (- oo, - 15.65) U (5.012, oo), (b) itis basedon a likelihood andhencedoesnotrequire
whereasthe BC methodgave (-87.1, 71.6) with500 an initialreduction of problemto theMLE, and thisin
bootstrap replications.
Hencethebootstrap methodgives turnmightresultin increasedefficiency. We wouldwel-
a verymisleading answerinthiscase. We notethatthisis comeanythoughts Efronmighthaveon thisprocedure.
not a pathological example:in a simulation experiment Detailscan be foundin DiCiccioand Tibshirani (1986).
reportedin DiCiccio and Tibshirani (1986), witha true
meanof(3, 3), 10% oftheFiellerintervals wereexclusive. ADDITIONALREFERENCES
3. RemarkG is somewhat misleading becausethework DiCiccio,T., and Tibshirani, R. (1986), "Approximating the Profile
ofMcCullagh(1984)andCox (1980)concentrates on con- LikelihoodThroughStein'sLeastFavourableFamily,"TechnicalRe-
ditionalconfidence intervals,thatis,conditional on a first- port2, University ofToronto,Dept. ofStatistics.
(1987),"BootstrapConfidence Intervals
andBootstrap Approx-
orderancillary.Efron'sintervalsare unconditional and imations,"Journalof the AmericanStatisticalAssociation,82, 163-
(11.5)is an unconditionalversionofMcCullaghandCox's 170.

Corment
WEI-YINLOH and C. F. J. WU*

Efronshouldbe congratulated thetheory worsethanthe classicalt interval.This background

forpioneering ap-
anduseofthebootstrap, a subjectthathasbeendiscussed pearsto be themotivation behindEfron'spresentwork.
frequentlyin recentyears.Despitethelargebodyofthe- It is clearfromhislucidpresentation thattheBCa in-
oreticalresultsshowingits superiorasymptotic perfor- tervalis theoretically betterthanthe BC interval.The
mance,simulation studieson the coverageprobabilities former, however,is notnecessarily betterthanotherin-
of variousbootstrapintervals(Efron1982; Loh 1987; tervals(see ourTables 1 and 2). We findhisidea of the
Schenker 1985;Wu 1986)indicatethatno one versionis acceleration constanta to be originaland usefulin many
a clearwinner.In somecases,bootstrap do even instances.
intervals He hasgivenan elegantjustification
oftheBCa

* Wei-YinLoh is Assistant
Professorand C. F. J. Wu is Professor,
bothat theDepartmentofStatistics,
UniversityofWisconsin, Madison, ? 1987AmericanStatistical
Association
WI 53706.Thisworkis supported by jointNSF-AFOSR GrantDMS JournaloftheAmericanStatistical
Association
8502303andISSA 86-0068. March1987,Vol.82, No.397,Theory
and Methods

This content downloaded from 62.122.79.46 on Tue, 10 Jun 2014 13:17:07 PM

All use subject to JSTOR Terms and Conditions
Lohand Wu:Comment 189

Table 1. NonparametricEstimationof the Mean (nominal90%/o; Table 2. NonparametricEstimationof the Variance (nominal90%;
n = 20; B = 200; 500 replicates) n = 20; B = 200; 500 replicates)

Distribution PER BC BC. BST t interval Distribution PER BC BC. BST CSR

Normal Normal
Coverage .91 .90 .89 .92 .90 Coverage .79 .82 .83 .88 .83
E(length) .71 .71 .72 .77 .76 E(length) .86 .89 .94 1.5 1.0
t2 t5
Coverage .84 .81 .79 .83 .91 Coverage .68 .75 .77 .85 .87
E(length) 1.7 1.7 1.8 2.7 1.8 E(length) 1.1 1.2 1.3 3.2 1.6
Weibull Exponential
Coverage .73 .73 .75 .83 .75 Coverage .65 .67 .69 .83 .71
E(length) 2.6 2.7 3.0 5.5 2.8 E(length) 1.3 1.3 1.5 5.5 1.4

methodin simpleparametricsituations.For theimportant and (3.7): (i) Estimationofthemedianor otherquantile.

nonparametric case and theparametriccase involvingnui- (ii) Estimation of an endpoint(Loh 1984).Here theem-
sance parameters,furthertheoreticalinvestigationsare piricalinfluence functionUi(7.2) anda (7.3) arenotcom-
needed beforethe methodcan be used withconfidence. putable.Extensionsof the BCa methodto nonsmooth
Our discussionwill focuson the followingfouraspects. situationsare calledfor.

1. DOES THE BCa METHODPROVIDESUBSTANTIAL2.2 Smooth0

IMPROVEMENT OVER THEBC METHOD? Evenwhentheformula fora exists,theremaybe other
Tables 1 and 2 contain Monte Carlo estimatesof the computational problems.In thecase of nuisanceparam-
coverage probabilitiesand expected lengthsof the non- eters,formula (6.5) appearsto be difficult to deriveand
parametricBCa and otherintervalsforestimating themean computein general.The formula (6.7) is foran exponen-
and variance, respectively.(The Weibull distributionin tialfamily withnaturalparameter q. For a curvedexpo-
Table1 hasdensity andthedistributionsnentialfamily,
:y-1/2exp(- y112), does a stillhavea niceand simpleform?
in Table 2 are scaled so that each has unit variance.) We nowconsiderthenonparametric case. The formula
"PER" is the percentileinterval,"BST" is the bootstrap- (7.3) fora dependson Ui. In manycases Uihasno closed
t intervalin Efron (1982), and "CSR" is a new bootstrap form.WhatA shouldbe chosenin computing Ui? Is a
methodproposed in Loh (1987). The tables suggestthat, sensitive to thechoiceofA?
forfinitesample sizes, the BCa methoddoes not improve
upon the BC methodsignificantly. For the situationscon- 3. THEACCELERATIONCONSTANTMAY BE BIG
sidered here, the classical-t,BST, and CSR intervalsap-
pear to be superiorto theBCa. (AlthoughtheBST interval Efronwrites,"The constanta willtypically be in the
is excellentin termsof coverage probability,it should be range lal < .2 . . ." (p. 173). From formulas(4.4) and
noticedthatits lengthtends to be on average quite a bit (7.3), a willbe largeor infinite if 10or Ui has largeor
longerthan thatof the otherintervals.) infinite
thirdmoment.If 0 is thevarianceor skewness,
finitesixthor ninthmomentsmaybe requiredforthe
methodtowork.In thesesituations, doestheBCa interval
2. COMPUTATIONOF THEACCELERATION
yieldanyrealimprovement overtheBC interval? Should
CONSTANTa
a be modified?
2.1 Nonsmooth0 To illustrate
our point,considertheone-sample mean
0 = Y. Here
How is a computedwhen0 is not smooth?Considerthe n

followingexamples. a = (6\) -13s3, A

A = n- E (X, - X)3E
1. No nuisanceparameters.Estimationof the centerof
symmetry 0 fromthe familyof double exponentialdistri- It can be shown,byan Edgeworth expansionoftheboot-
butions{(2z)-1 exp[ - -1ly- 01],- known}.Here formulas strapdistributionG, thatz1= a + o(n12). The values
(4.4) and (4.5) are not computable. z[a] and z[1 - a] in the BCa interval(3.8) can be ap-
2. One nuisanceparameter.Estimationof the scale pa- proximated
by
rametera in the familyof two-parameter exponentialdis-
tributions{a-' exp[-c-r(y - ,u)], y > ,u,,u unknown}.
a + [a - z(1-a)][1 - a2 + az(l-a)]-l

The MLE a is approximatelynormal,but the Fisher in- and

formationbound does not exist.Here formula(6.5) is not
a + [a + z(l-a)][l - a2 1.
computable.
-

3. Nonparametric
case. Examplesof functions
thatdo Letr0= A{z(1a() - [(Z(la())2 andrl 2{Z(la) +
+ 4]1/2} -

not possesstransformation-to-normality
properties
(3.6) [(Z(O-a))2 + 4]1/2} oftheequationa2
be thesolutions -

This content downloaded from 62.122.79.46 on Tue, 10 Jun 2014 13:17:07 PM

All use subject to JSTOR Terms and Conditions
190 Journal of the American Statistical Association, March 1987

az(1-a) - 1 = 0. The BCa interval

(3.8) is degenerateif 4. ON THEASSUMPTIONOF TRANSFORMATION
z[a] 2 z[l - a], whichoccurswhenIrol< lal< rl.Typically TO NORMALITY
r1is too largeto be of anyreal concern.For lalsmaller
thanbutclose to Irol,theBCa interval is nondegenerate The BCa methodis motivated bytheassumption (3.6)
that,on the transformed scale, 0 = 0 + (1 + a4)(Z -
butlargebecauseone ofz[a] or z[1 - a] is large.From
thevaluesof Irol givenhere,we advisecautionin theuse zO),whereZ is standard normal.To whatextentdoes the
method depend on the normality of Z? If Z is assumed
oftheBCa methodwhenlalis large.
to haveanotherdistribution F, theconstant zo is changed
2a: .1 .05 .025 .01 .005 to F-( G(6)). The constant a, on theotherhand,appears
Irol: .473 .420 .381 .342 .320 to havea morecomplicated form,sincethederivation of
(4.4) usesthefirstsixmoments ofthenormaldistribution.
One mayarguethatthesevaluesof Irol are too largeto The moments of a generalF are usuallylesssimple.
be of concern.For example,if Irol= .32, thenlal = Irol
impliesthati3s-3 = 6 x .32 = 1.92An,whichequals ADDITIONALREFERENCES
7.7 and 9.6 forn = 16 and 25. That is, the BCa interval
Loh, Wei-Yin(1984),"Estimating
an Endpointof a Distribution
With
maybe largeordegenerate withvery ResamplingMethods," The Annals of Statistics,12, 1543-1550.
onlyfordistributions
largeskewness.The situationcan, however,be worse, (1987), "CalibratingConfidence Coefficients,"
Journalof the
becausethesampleskewnessis quitevariableand,in ad- AmericanStatisticalAssociation,82, 155-162.
Wu,C. F. J.(1986),"Jackknife,Bootstrap andOtherResampling Meth-
exceeding
dition,becauseskewness 10is notunlikely
with odsinRegression Analysis"(withdiscussion), TheAnnalsofStatistics,
estimatorsmorenonlinearthanthesamplemean. 14.

Comment
D. R. COX*

The articleaddressesimportant generalissues with whichsome parametervalues are logicallyinconsistent

powerandgenerality. Nevertheless I wassurprised to see withthedata.
no mention ofwhatin manywaysseemsthesimplest and 3. A simpleBartlettadjustmentimprovescoverage
mostnaturalwayof finding approximate confidence re- probabilities and whend = 1 in (1) a furthercorrection
gions,namelyviadirectuseofthelikelihood ratiostatistic. adjusts likelihood-basedregionsto equitailedregions
Fora parameter 0 ofdimension d, therebeingno nuisance (McCullagh1984).
parameters, we obtainregionsoftheform 4. Whenthereareno nuisanceparameters (1) hasfairly
appealing second-orderoptimality
properties basedon lo-
{0; 2[1(0) - 1(0)] - (1)
cal ancillarity
(Cox 1980;McCullagh1984). Whenthere
whereI denoteslog-likelihood,1(6)denotesitssupremum, are nuisanceparameters suchoptimality in generalfails,
andthechi-squared distributionwithd degreesoffreedom andwhentherearemanynuisanceparameters thefailure
is used to forman approximate 1 - a level region.If maybe severe.Someconditioning is probablythewayout
0 = (V/,A),whereV/is theparameter of interest,(1) is and is studiedand a possibleformaljustification
outlined
replacedin thefirstinstancebytheprofilelikelihood in as yetunpublishedworkbyN. Reid and me.
{V; 2[1(A, A 1(VI,AV)]
- Xd,} (2)It is unclearto mehowtheseprocedures comparecom-
putationally in complexcases withthoseproposedin the
whereA,.is themaximum likelihoodestimateof A given
article.In preserving
information forpossiblesubsequent
y,and d is nowthedimension of V/.
reanalysis and combinationwithotherdata,storageof a
Theseprocedures havethefollowing advantages: condensedformofI has advantages, butforsimplepres-
1. Thereis exactinvarianceunderreparameterization.entation ofconclusions
anestimate oflocationwitha single
sensibleanswersare obtainedformul- estimateoferrorhas,wherereasonably
2. Qualitatively appropriate,ob-
timodal1,forI thathavesupremum at oo,andforcasesin viousattractions.

? 1987AmericanStatistical
Association
* D. R. Cox is Senior Research Fellow, Departmentof Mathematics, JournaloftheAmericanStatistical
Association
ImperialCollege, London SW7 2BZ, England. and Methods
March1987,Vol.82, No.397,Theory

This content downloaded from 62.122.79.46 on Tue, 10 Jun 2014 13:17:07 PM

All use subject to JSTOR Terms and Conditions