Berger The Likelihho Principle 1988 Book

Institute of Mathematical Statistics
LECTURE NOTES-MONOGRAPH SERIES

The Likelihood Principle
(Second Edition)
James O. Berger
Purdue University
Robert L. Wolpert
Duke University
T H E L I K E L I H O O D P R I N C I P L E :
A R E V I E W , G E N E R A L I Z A T I O N S , A N D S T A T I S T I C A L I M P L I C A T I O N S
S E C O N D E D I T I O N
LECTURE NOTES-MONOGRAPH SERIES
Shanti S. Gupta, Series Editor
Volume 6
The Likelihood Principle
(Second Edition)
James O. Berger
Purdue University
Robert L. Wolpert
Duke University
Hayward, California
Lecture Notes-Monograph Series
Series Editor, Shanti S. Gupta, Purdue University
The production of the IMS Lecture Notes-Monograph Series is
managed by the IMS Business Office: Nicholas P. Jewell, IMS
Treasurer, and Jose L. Gonzalez, IMS Business Manager.
Library of Congress Catalog Card Number: 88-81456
International Standard Book Number 0-940600-13-7
Copyright 1988 Institute of Mathematical Statistics
All rights reserved
Printed in the United States of America
T o m y p a r e n t s , O r v i s a n d T h e l m a
J a m e s B e r g e r
T o m y w i f e , R u t a
R o b e r t W o l p e r t
P R E F A C E
T h i s m o n o g r a p h b e g a n w i t h r e s e a r c h d e s i g n e d t o p r o v i d e a g e n e r a l i -
z a t i o n o f t h e L i k e l i h o o d P r i n c i p l e ( L P ) t o q u i t e a r b i t r a r y s t a t i s t i c a l
s i t u a t i o n s . T h e p u r p o s e o f s e e k i n g s u c h a g e n e r a l i z a t i o n w a s t o p a r t i a l l y
a n s w e r c e r t a i n c r i t i c i s m s t h a t h a d b e e n l e v i e d a g a i n s t t h e L P , c r i t i c i s m s w h i c h
s e e m e d t o p r e v e n t m a n y s t a t i s t i c i a n s f r o m s e r i o u s l y c o n s i d e r i n g t h e L P a n d i t s
i m p l i c a t i o n s . T h e r e s e a r c h e f f o r t s e e m e d w o r t h w h i l e b e c a u s e o f t h e s i m p l i c i t y ,
c e n t r a l i m p o r t a n c e , a n d f a r r e a c h i n g i m p l i c a t i o n s o f t h e L P .
B a c k g r o u n d r e a d i n g f o r t h e r e s e a r c h r e v e a l e d a w i d e r t h a n
e x p e c t e d r a n g e o f p u b l i s h e d c r i t i c i s m s o f t h e L P . In a n a t t e m p t t o b e c o m p l e t e
a n d a d d r e s s a l l s u c h c r i t i c i s m s , t h e r e s e a r c h p a p e r e x p a n d e d c o n s i d e r a b l y .
E v e n t u a l l y it s e e m e d s e n s i b l e t o e n l a r g e t h e p a p e r t o a m o n o g r a p h . T h i s a l s o
a l l o w e d f o r d i s c u s s i o n o f c o n d i t i o n i n g i d e a s in g e n e r a l a n d f o r a r e v i e w o f t h e
i m p l i c a t i o n s o f t h e L P . It w a s d e c i d e d , h o w e v e r , t o s t o p s h o r t o f a g e n e r a l
r e v i e w o f c o n d i t i o n a l m e t h o d s in s t a t i s t i c s . I n p a r t i c u l a r , t h e m o n o g r a p h d o e s
n o t d i s c u s s t h e m a n y l i k e l i h o o d b a s e d s t a t i s t i c a l m e t h o d o l o g i e s t h a t h a v e b e e n
d e v e l o p e d , a l t h o u g h r e f e r e n c e s t o t h e s e m e t h o d o l o g i e s w i l l b e g i v e n . T h i s
l i m i t a t i o n w a s , in p a r t , b e c a u s e s u c h a n e n d e a v o r w o u l d b e f a r t o o a m b i t i o u s ,
a n d , in p a r t , b e c a u s e w e f e e l ( a n d i n d e e d a r g u e i n C h a p t e r 5 ) t h a t B a y e s i a n
i m p l e m e n t a t i o n o f t h e L P i s t h e c o r r e c t c o n d i t i o n a l m e t h o d o l o g y .
T h e m a t h e m a t i c a l l e v e l o f t h e m o n o g r a p h i s , f o r t h e m o s t p a r t ,
k e p t a t a n o n t e c h n i c a l l e v e l . T h e m a i n e x c e p t i o n is t h e g e n e r a l i z a t i o n o f t h e
L P i n S e c t i o n 3 . 4 , w h i c h i s ( n e c e s s a r i l y ) p r e s e n t e d a t a m e a s u r e - t h e o r e t i c
l e v e l , b u t c a n b e s k i p p e d w i t h n o l o s s in c o n t i n u i t y . A l s o , t h e m o n o g r a p h
vii
v i i i P R E F A C E
p r e s u p p o s e s n o f a m i l i a r i t y w i t h c o n d i t i o n i n g c o n c e p t s . I n d e e d C h a p t e r 2
p r o v i d e s a n e l e m e n t a r y r e v i e w o f c o n d i t i o n i n g , w i t h m a n y e x a m p l e s .
T h i s s e c o n d e d i t i o n w a s p r o d u c e d u n d e r t h e r a t h e r s e v e r e c o n s t r a i n t
t h a t t h e o r i g i n a l m a n u s c r i p t , u s e d f o r p h o t o - o f f s e t p r i n t i n g , w a s i n a d v e r t a n t l y
d e s t r o y e d ; o n l y t h e p h o t o s w e r e k e p t . T h u s c h a n g e s c o u l d o n l y b e m a d e b y
r e t y p i n g e n t i r e p a g e s o r i n s e r t i n g n e w p a g e s . A l i s t o f c o r r e c t i o n s t h a t w e r e
t o o m i n o r t o j u s t i f y t h e r e t y p i n g o f a n e n t i r e p a g e i s g i v e n a t t h e e n d o f t h e
m o n o g r a p h . I n s e r t e d p a g e s r e c e i v e d d e c i m a l p a g e n u m b e r s : e . g . 7 4 . 1 , 7 4 . 2 . A
l i s t o f a d d i t i o n a l r e f e r e n c e s w a s a d d e d , a n d n e w d i s c u s s i o n s w e r e k i n d l y c o n t r i -
b u t e d b y M . J . B a y a r r i a n d M . H . D e G r o o t , B r u c e H i l l , a n d L u c i e n L e C a m .
S u b s t a n t i a l c h a n g e s o r a d d i t i o n s w e r e m a d e in S e c t i o n s 3 . 1 , 3 . 5 ,
4 . 2 . 1 , 4 . 4 , a n d 4 . 5 . T h e c h a n g e s i n S e c t i o n 4 . 4 c o r r e c t a g l a r i n g o v e r s i g h t in
t h e f i r s t e d i t i o n : t h e f a i l u r e t o e m p h a s i z e t h e m i s l e a d i n g c o n c l u s i o n s t h a t
c a n r e s u l t f r o m v i o l a t i o n o f t h e L i k e l i h o o d P r i n c i p l e in s i g n i f i c a n c e t e s t i n g
o f a p r e c i s e h y p o t h e s i s . A n o t h e r v e r y w e a k p a r t o f t h e f i r s t e d i t i o n w a s
S e c t i o n 3 . 5 , w h i c h d i s c u s s e d p r e d i c t i o n , d e s i g n , a n d n u i s a n c e p a r a m e t e r s . T h e
n e w m a t e r i a l i n c o r p o r a t e s r e c e n t s u b s t a n t i v e i n s i g h t s f r o m t h e l i t e r a t u r e .
N u m e r o u s o t h e r m i n o r c h a n g e s a n d l i t e r a t u r e u p d a t i n g s w e r e m a d e
t h r o u g h o u t t h e m o n o g r a p h . W e d i d n o t a t t e m p t c o m p l e t e c o v e r a g e o f r e c e n t l i t -
e r a t u r e , h o w e v e r .
W e a r e g r a t e f u l t o a n u m b e r o f p e o p l e f o r v a l u a b l e d i s c u s s i o n s o n
t h i s s u b j e c t a n d / o r f o r c o m m e n t s a n d s u g g e s t i o n s o n o r i g i n a l d r a f t s o r t h e
f i r s t e d i t i o n o f t h e m o n o g r a p h . I n p a r t i c u l a r , w e w o u l d l i k e t o t h a n k G e o r g e
B a r n a r d , M . J . B a y a r r i , M a r k B e r l i n e r , L a w r e n c e B r o w n , G e o r g e C a s e l l a , M o r r i s
D e G r o o t , J . L . F o u l l e y , L e o n G l e s e r , P r e m G o e l , C l y d e H a r d i n , B r u c e H i l l ,
J i u n n H w a n g , R a j e e v K a r a n d i k a r , L u c i e n L e C a m , K e r - C h a u L i , D e n n i s L i n d l e y ,
G e o r g e M c C a b e , G e o r g e s M o n e t t e , J o h n P r a t t , D o n R u b i n , H e r m a n R u b i n , M y r a
S a m u e l s , S t e v e S a m u e l s , a n d T o m S e l l k e . W e a r e e s p e c i a l l y g r a t e f u l t o M . J.
B a y a r r i a n d M . H . D e G r o o t f o r a n e x c e p t i o n a l l y c o m p l e t e a n d i n s i g h t f u l s e t o f
c o r r e c t i o n s a n d c o m m e n t s o n t h e f i r s t e d i t i o n . W e a r e a l s o g r a t e f u l t o
S h a n t i G u p t a f o r t h e e n c o u r a g e m e n t t o t u r n t h e m a t e r i a l i n t o a m o n o g r a p h .
P R E F A C E ix
T h a n k s a r e a l s o d u e t o t h e A l f r e d P . S l o a n F o u n d a t i o n , t h e N a t i o n a l S c i e n c e
F o u n d a t i o n ( G r a n t s M C S - 7 8 0 1 7 3 7 , M C S - 8 1 0 1 6 7 0 A 1 , a n d D M S - 8 7 0 2 6 2 0 ) , a n d t h e
C e n t e r f o r S t o c h a t i c P r o c e s s e s a t t h e U n i v e r s i t y o f N o r t h C a r o l i n a f o r s u p p o r t
o f t h e r e s e a r c h in t h e m o n o g r a p h . F i n a l l y , w e a r e e x t r e m e l y g r a t e f u l t o
N o r m a L u c a s , T e e n a C h a s e , a n d B e t t y G i c k f o r e x c e l l e n t t y p i n g o f t h e m a n u s c r i p t .
M a r c h , 1 9 8 8 J A M E S 0 . B E R G E R
P u r d u e U n i v e r s i t y , W e s t L a f a y e t t e
R O B E R T W O L P E R T
D u k e U n i v e r s i t y , D u r h a m
T A B L E OF C ON T E N T S
C HA PT E R 1. IN T RODUC T ION 1
C HA PT E R 2. C ON DIT ION IN G 5
2.1. S imple E xamples 5
2.2. Relevant S ubsets 11
2.3. A ncillarity 13
2.4. C onditional Frequentist Procedures 14
2.4.1. Conditional Confidence 14
2.4.2. Estimated Confidence 16
2.5. C riticisms of Partial C onditioning 16
C HA PT E R 3. T HE L IKE L IHOOD PRIN C IPL E A N D GE N E RA L IZA T ION S . 19
3.1. Introduction 19
3.2. History of the L ikelihood Principle 22
3.3. B irnbaum's Development the Discrete C ase 24
3.3.1. Evidence
Conditionally, and Sufficiency 24

3.3.2. Axiomatic Development 26
3.4. Generalizations B eyond the Discrete C ase 28
3.4.1. Difficulties in the Nondiscrete Case 29
3.4.2. Evidencej Conditionality> and Sufficiency 31
3.4.3. The Relative Likelihood Principle 32
3.5. Prediction, Design, N uisance P a r a m e t e r s , and the L P 36
3.5.1. Introduction 36
3.5.2. Unobserved Variables: Prediction and Design 37
3.5.3. Nuisance Variables and Parameters 41
x
TABLE OF CONTENTS xi
3.6. Criticisms of Birnbaum's Axiomatic Development 42
3.6.1. The Model Assumption 43
3.6.2. The Evidence Assumption 45
3.6.3. The Weak Conditionality Principle 45
3.6.4. The Sufficiency Principle 46
3.7 . Violation of the Likelihood Principle: Inadmissibility and
Incoherency 50
3.7 .1. Introduction 50
3.7 .2. Decision Theoretic Evaluation 52
3.7 .3. Betting Evaluation 59
CHAPTER 4. CONSEQUENCES AND CRITICISMS OF THE LIKELIHOOD PRINCIPLE AND THE
RELATIVE LIKELIHOOD PRINCIPLE 65
4.1. Incompatibility with Frequentist Concepts 65
4.1.2. Objectivity 67
4.1.3. Procedures for Nonspecialists 68
4.1.4. Repeatability 7 0
4.1.5. The Confidence Principle 71
4.2. The Irrelevance of Stopping Rules 7 4
4.2.1. Introduction 7 4
4.2.2. The (Discrete) Stopping Rule Principle 7 5
4.3.2. Positive Implications 7 7
4.2.4. Criticisms 80
4.2.5. Stopping Rules and Inadmissibility 83
4.2.6. The General Stopping Rule Principle 86
4.2.7 . Informative Stopping Rules 88
4.3. The Irrelevance of Censoring Mechanisms
4.3.2. F ix ed Censoring and Eq uivalent Censoring Mechanisms . . . 9 2
4 . 3 . 3 . Random Censoring 95
4.3.4. Informative Censoring 103
x TABLE OF CONTENTS
4.4. Significance Testing 104
4.4.1. Conflict with the Likelihood Principle 104
4.4.2. Averaging Over "More Extreme" Observations 105
4.4.3. Testing a Single Null Model 109
4.4.4. Conclusions 109.3
4.5. Randomization Analysis 110
4.5.2. Finite Population Sampling 112
4.5.3. Randomization Tests 117
CHAPTER 5. IMPLEMENTATION OF THE LIKELIHOOD PRINCIPLE 121
5.1. Introduction 121
5.2. NonBayesian Likelihood Methods 122
5.3. Arguments for Bayesian Implementation 124
5.3.1. General Considerations 124
5.3.2. The FraserMonetteNg_, Stone, and Stein Examples . . . 127
5.4. Robust Bayesian Analysis 136
5.5. Conclusions 141
RE FE RE N C E S 143
A DDIT ION A L OR UPDA T E D RE FE RE N C E S 160
DIS C US S ION BY M. J. B A YA RRI A N D M. H. DE GROOT 160.3
DIS C US S ION BY B RUC E HIL L 161
DIS C US S ION BY DA VID L A N E 175
DIS C US S ION BY L UC IE N L E C A M 182
RE PL Y T O T HE DIS C US S ION 186
A DDIT ION A L RE FE RE N C E S IN T HE DIS C US S ION 198
IN DE X OF E XA MPL E S 200
A UT HOR IN DE X 201
S UB JE C T IN DE X 204
E RRA T A A N D C L A RIFIC A T ION S 207
C H A P T E R 1 . I N T R O D U C T I O N
A m o n g a l l p r e s c r i p t i o n s f o r s t a t i s t i c a l b e h a v i o r , t h e L i k e l i h o o d
P r i n c i p l e ( L P ) s t a n d s o u t a s t h e s i m p l e s t a n d y e t m o s t f a r r e a c h i n g . I t e s s e n
t i a l l y s t a t e s t h a t a l l e v i d e n c e , w h i c h is o b t a i n e d f r o m a n e x p e r i m e n t , a b o u t a n
u n k n o w n q u a n t i t y , i s c o n t a i n e d in t h e l i k e l i h o o d f u n c t i o n o f f o r t h e g i v e n
d a t a . T h e i m p l i c a t i o n s o f t h i s a r e p r o f o u n d , s i n c e m o s t n o n B a y e s i a n a p p r o a c h e s
t o s t a t i s t i c s a n d i n d e e d m o s t s t a n d a r d s t a t i s t i c a l m e a s u r e s o f e v i d e n c e ( s u c h
a s c o v e r a g e p r o b a b i l i t y , e r r o r p r o b a b i l i t i e s , s i g n i f i c a n c e l e v e l , f r e q u e n t i s t
r i s k , e t c . ) a r e t h e n c o n t r a i n d i c a t e d .
T h e L P w a s a l w a y s i m p l i c i t in t h e B a y e s i a n a p p r o a c h t o s t a t i s t i c s ,
b u t i t s d e v e l o p m e n t a s a s e p a r a t e s t a t i s t i c a l p r i n c i p l e w a s d u e in l a r g e p a r t
t o i d e a s o f R . A . F i s h e r a n d G . B a r n a r d ( s e e S e c t i o n 3.2 f o r r e f e r e n c e s ) . I t
r e c e i v e d m a j o r n o t i c e w h e n B i r n b a u m ( 1 9 6 2 a ) s h o w e d it t o b e a c o n s e q u e n c e o f
t h e m o r e c o m m o n l y t r u s t e d S u f f i c i e n c y P r i n c i p l e ( t h a t a s u f f i c i e n t s t a t i s t i c
s u m m a r i z e s t h e e v i d e n c e f r o m a n e x p e r i m e n t ) a n d C o n d i t i o n a l i t y P r i n c i p l e ( t h a t
e x p e r i m e n t s n o t a c t u a l l y p e r f o r m e d s h o u l d b e i r r e l e v a n t t o c o n c l u s i o n s ) . S i n c e
t h e n t h e L P h a s b e e n e x t e n s i v e l y d e b a t e d b y s t a t i s t i c i a n s i n t e r e s t e d in f o u n d a
t i o n s , b u t h a s b e e n i g n o r e d b y m o s t s t a t i s t i c i a n s . T h e r e a r e p e r h a p s s e v e r a l
r e a s o n s f o r t h i s . F i r s t , t h e c o n s e q u e n c e s o f t h e L P s e e m s o a b s u r d t o m a n y
c l a s s i c a l s t a t i s t i c i a n s t h a t t h e y f e e l it a w a s t e o f t i m e t o e v e n s t u d y t h e
i s s u e . S e c o n d , a c u r s o r y i n v e s t i g a t i o n o f t h e L P r e v e a l s c e r t a i n o f t s t a t e d
o b j e c t i o n s , f o r e m o s t o f w h i c h is t h e a p p a r e n t d e p e n d e n c e o f t h e p r i n c i p l e o n
a s s u m i n g e x a c t k n o w l e d g e o f t h e ( p a r a m e t r i c ) m o d e l f o r t h e e x p e r i m e n t (so t h a t
a n e x a c t l i k e l i h o o d f u n c t i o n e x i s t s ) . S i n c e t h e m o d e l is r a r e l y t r u e , ( h a s t y )
1
2 T H E L I K E L I H O O D P R I N C I P L E
r e j e c t i o n o f t h e L P m a y r e s u l t . T h i r d , t h e L P d o e s n o t s a y h o w o n e i s t o p e r -
f o r m a s t a t i s t i c a l a n a l y s i s ; i t m e r e l y g i v e s a p r i n c i p l e t o w h i c h a n y m e t h o d o f
a n a l y s i s s h o u l d a d h e r e . I n d e e d B a y e s i a n a n a l y s i s i s o f t e n p r e s e n t e d a s t h e w a y
t o i m p l e m e n t t h e L P ( w i t h w h i c h w e e s s e n t i a l l y a g r e e ) , a v e r y u n a p p e a l i n g
p r o s p e c t t o m a n y c l a s s i c a l s t a t i s t i c i a n s .
T h e m a j o r p u r p o s e o f t h i s ( m o s t l y r e v i e w ) m o n o g r a p h i s t o a d d r e s s
t h e s e c o n c e r n s . A s e r i o u s e f f o r t w i l l b e m a d e , t h r o u g h e x a m p l e s a n d a p p e a l s t o
c o m m o n s e n s e , t o a r g u e t h a t t h e L P i s i n t u i t i v e l y s e n s i b l e , m o r e s o t h a n t h e
c l a s s i c a l m e a s u r e s w h i c h i t i m p u n e s . A l s o , a g e n e r a l i z e d v e r s i o n o f t h e L P
w i l l b e i n t r o d u c e d , a v e r s i o n w h i c h r e m o v e s t h e r e s t r i c t i o n o f a n e x a c t l y k n o w n
l i k e l i h o o d f u n c t i o n , a n d y e t h a s e s s e n t i a l l y t h e s a m e i m p l i c a t i o n s . ( O t h e r
c r i t i c i s m s o f t h e L P w i l l a l s o b e d i s c u s s e d . ) F i n a l l y , t h e q u e s t i o n o f i m p l e -
m e n t a t i o n o f t h e L P w i l l b e c o n s i d e r e d , a n d i t w i l l b e a r g u e d t h a t B a y e s i a n
a n a l y s i s ( m o r e p r e c i s e l y r o b u s t B a y e s i a n a n a l y s i s ) i s t h e m o s t s e n s i b l e a n d
r e a l i s t i c m e t h o d o f i m p l e m e n t a t i o n . A t h o r o u g h d i s c u s s i o n o f t h i s i s s u e i s ,
h o w e v e r , o u t s i d e t h e s c o p e o f t h e m o n o g r a p h , s o t h e m a i n t h e s i s w i l l s i m p l y b e
t h a t t h e L P i s b e l i e v a b l e a n d t h a t b e h a v i o r i n v i o l a t i o n o f i t s h o u l d b e
a v o i d e d t o t h e e x t e n t p o s s i b l e .
A c c e p t a n c e o f s u c h a t h e s i s r a d i c a l l y a l t e r s t h e w a y o n e v i e w s
s t a t i s t i c s . I n d e e d , t o m a n y B a y e s i a n s , b e l i e f i n t h e L P i s t h e b i g d i f f e r e n c e
b e t w e e n B a y e s i a n s a n d f r e q u e n t i s t s , n o t t h e d e s i r e t o i n v o l v e p r i o r i n f o r m a t i o n .
T h u s S a v a g e s a i d ( i n t h e D i s c u s s i o n o f B i r n b a u m ( 1 9 6 2 a ) )
" I , m y s e l f , c a m e t o t a k e . . . B a y e s i a n s t a t i s t i c s . . .
s e r i o u s l y o n l y t h r o u g h r e c o g n i t i o n o f t h e l i k e l i -
h o o d p r i n c i p l e . "
M a n y B a y e s i a n s b e c a m e B a y e s i a n s o n l y b e c a u s e t h e L P l e f t t h e m l i t t l e c h o i c e .
S u f f i c i e n t t i m e h a s p a s s e d s i n c e t h e a x i o m a t i c d e v e l o p m e n t o f
B i r n b a u m t o h o p e t h a t a n y v a l i d o b j e c t i o n s t o t h e L P w o u l d b y n o w h a v e b e e n
f o u n d . I n d e e d , t h e r e a r e n u m e r o u s a r t i c l e s i n t h e l i t e r a t u r e p r e s e n t i n g
e x a m p l e s , c o u n t e r e x a m p l e s , a r g u m e n t s , a n d c o u n t e r a r g u m e n t s f o r t h e L P . W e w i l l
I N T R O D U C T I O N 3
a t t e m p t t o d i s c u s s a l l m a j o r i s s u e s r a i s e d , a n d t h u s w i l l n e c e s s a r i l y c o v e r
m u c h o f t h e s a m e g r o u n d a s t h e s e o t h e r a r t i c l e s . T h e c o l l e c t i o n o f r e l e v a n t
a r g u m e n t s i n o n e p l a c e w i l l h o p e f u l l y m a k e s t u d y o f t h i s c r u c i a l i s s u e m u c h
e a s i e r .
C l e a r l y , w e c a n n o t c l a i m i m p a r t i a l i t y in t h i s m o n o g r a p h ; i n d e e d t h e
m o n o g r a p h i s e s s e n t i a l l y a i m e d a t p r o m o t i n g t h e L P . T h i s c a n b e s t b e d o n e , h o w -
e v e r , b y p u r p o s e l y r a i s i n g a n d a n s w e r i n g a l l o b j e c t i o n s t o i t ( o f w h i c h w e a r e
a w a r e ) , s o a s u b s t a n t i a l a c c o u n t i n g o f t h e " o t h e r s i d e " w i l l b e g i v e n . A l s o ,
a l t h o u g h o u r c r i t i c i s m o f c l a s s i c a l m o d e s o f t h o u g h t m a y s e e m r a t h e r s e v e r e a t
t i m e s , i t w o u l d b e w r o n g t o c o n c l u d e t h a t w e a r e c o m p l e t e l y r e j e c t i n g c l a s s i c a l
s t a t i s t i c s , a s i t i s p r a c t i c e d . M o s t c l a s s i c a l p r o c e d u r e s w o r k v e r y w e l l m u c h
o f t h e t i m e . I n d e e d , m a n y c l a s s i c a l p r o c e d u r e s a r e e x a c t l y w h a t a n " o b j e c t i v e
c o n d i t i o n a l i s t " w o u l d u s e , a l t h o u g h f o r d i f f e r e n t r e a s o n s a n d w i t h d i f f e r e n t
i n t e r p r e t a t i o n s . T h e r e a r e e x c e p t i o n s ( e . g . s i g n i f i c a n c e t e s t i n g a n d m u c h o f
s e q u e n t i a l a n a l y s i s - s e e C h a p t e r 4 ) , w h e r e it c a n b e a r g u e d t h a t c l a s s i c a l
a n a l y s e s o f t e n y i e l d y e r y m i s l e a d i n g i n f e r e n c e s b e c a u s e o f t h e i r v i o l a t i o n o f
t h e L P .
O f c o u r s e , c l a s s i c a l s t a t i s t i c i a n s d o ( i n p r a c t i c e ) c o n d i t i o n a l l
t h e t i m e ; w h e n e v e r a n e x p e r i m e n t a l p r o t o c o l i s a l t e r e d o r a l o o k a t t h e d a t a
r e v e a l s t h e n e c e s s i t y t o a l t e r t h e h y p o t h e s i z e d m o d e l , c o n d i t i o n i n g h a s t a k e n
p l a c e . ( C o n d i t i o n i n g f o l l o w e d b y t h e u s e o f u n c o n d i t i o n a l f r e q u e n t i s t e v a l u a -
t i o n s i s , h o w e v e r , h i g h l y s u s p e c t , a n d i s t h e s o u r c e o f m u c h o f t h e h o s t i l i t y
t o w a r d s t h e L P . ) C o n d i t i o n i n g s e e m s u n a v o i d a b l e in p r a c t i c e , a n d s o it i s a
w o n d e r f u l p r a c t i c a l i m p l i c a t i o n o f t h e L P t h a t s u c h c o n d i t i o n i n g i s n o t o n l y
l e g i t i m a t e , b u t i s p r o p e r , p r o v i d i n g a s u i t a b l e c o n d i t i o n a l a n a l y s i s i s t h e n
p e r f o r m e d . C l i n i c a l t r i a l s i s j u s t o n e a r e a w h e r e v e r y d e s i r a b l e s i m p l i c i t y in
e x p e r i m e n t a t i o n a n d a n a l y s i s r e s u l t s f r o m a d o p t i o n o f t h e c o n d i t i o n a l v i e w p o i n t .
D i s c u s s i o n o f s u c h p r a c t i c a l i m p l i c a t i o n s i s g i v e n i n C h a p t e r 4 .
T h e m a t h e m a t i c s a n d t h e o r e t i c a l s t a t i s t i c s u s e d i n t h e m o n o g r a p h
w i l l , f o r t h e m o s t p a r t , b e k e p t a t a n e a s y - t o - r e a d l e v e l . ( T h e e x c e p t i o n i s
S e c t i o n 3 . 4 , w h e r e t h e g e n e r a l L P i s d e v e l o p e d . ) A l s o , e x a m p l e s w i l l f r e q u e n t l y
b e g i v e n in s i m p l e a r t i f i c i a l s e t t i n g s , r a t h e r t h a n r e a l i s t i c a l l y c o m p l i c a t e d
s t a t i s t i c a l s i t u a t i o n s , a g a i n f o r e a s e o f r e a d i n g a n d b e c a u s e c o m p l i c a t e d s i t
u a t i o n s a r e o f t e n t o o i n v o l v e d t o c l e a r l y r e v e a l k e y i s s u e s . A d v a n c e m e n t o f a
s u b j e c t u s u a l l y p r o c e e d s b y a p p l y i n g t o c o m p l i c a t e d s i t u a t i o n s t r u t h s d i s c o v
e r e d i n s i m p l e s e t t i n g s .
T h r o u g h o u t t h e m o n o g r a p h , X w i l l d e n o t e t h e r a n d o m q u a n t i t y t o b e
o b s e r v e d , X t h e s a m p l e s p a c e , x ( t h e o b s e r v e d d a t a ) a r e a l i z a t i o n o f X , a n d
P () t h e p r o b a b i l i t y d i s t r i b u t i o n o f X o n 2 , w h e r e G i s u n k n o w n . A l t h o u g h
w i l l b e c a l l e d t h e p a r a m e t e r a n d t h e p a r a m e t e r s p a c e , t h e f a m i l y { P
( )>
E } n e e d n o t b e a t y p i c a l p a r a m e t r i c f a m i l y ; c o u l d j u s t d e n o t e s o m e ( p o s s i
b l y n o n p a r a m e t r i c ) i n d e x . A l s o , w i l l b e u n d e r s t o o d t o c o n s i s t o f a l l u n k
n o w n f e a t u r e s o f t h e p r o b a b i l i t y d i s t r i b u t i o n . O f t e n , t h e r e f o r e , o n l y p a r t o f
w i l l b e o f i n t e r e s t , t h e r e m a i n d e r b e i n g a n u i s a n c e " p a r a m e t e r . " I n d i s c u s
s i n g s e q u e n t i a l a n d p r e d i c t i o n p r o b l e m s i t w i l l s o m e t i m e s b e c o n v e n i e n t t o c o n
s i d e r u n o b s e r v e d r a n d o m v a r i a b l e s Z , a s w e l l a s t h e u n k n o w n ; z w i l l t h e n d e
n o t e a p o s s i b l e v a l u e o f Z . T o s i m p l i f y t h e e x p o s i t i o n in t h e m o n o g r a p h , h o w
e v e r , w e w i l l u s u a l l y o n l y c o n s i d e r t h e s i m p l e r c a s e in w h i c h Z i s a b s e n t . N o t e
t h a t f o r s o m e s t a t i s t i c a l p r o b l e m s it i s i m p o s s i b l e t o s e p a r a t e Z a n d { P _ ( ) }

S e e S e c t i o n 3 . 5 f o r d i s c u s s i o n o f s u c h p r o b l e m s .
W h e n n e c e s s a r y , ^ w i l l d e n o t e t h e f i e l d o f m e a s u r a b l e e v e n t s i n X .
I f a d e n s i t y f o r X e x i s t s i t w i l l b e d e n o t e d f
Q
( x ) , a n d w e w i l l p r e s u m e t h e
e x i s t e n c e o f a s i n g l e d o m i n a t i n g f i n i t e m e a s u r e v ( ) f o r { P
( )> E ) s u c h
t h a t P
Q
( B ) = J f
Q
( x ) v ( d x ) f o r e a c h B e 3 . I n a l l t h e e x a m p l e s v w i l l b e t a k e n
t o b e c o u n t i n g m e a s u r e in t h e d i s c r e t e c a s e a n d L e b e s g u e m e a s u r e in t h e c o n t i n
u o u s c a s e , w h e n X i s a s u b s e t o f E u c l i d e a n s p a c e . U s u a l l y w e w i l l w r i t e t h e
r e f e r e n c e m e a s u r e s i m p l y a s " d x " ( i m p l i c i t l y t a k i n g L e b e s g u e m e a s u r e f o r v ) ;
t h e f o r m u l a s w i l l r e q u i r e m i n o r c h a n g e s f o r c a s e s ( i n c l u d i n g t h o s e i n v o l v i n g
d i s c r e t e d i s t r i b u t i o n s ) i n w h i c h o t h e r r e f e r e n c e m e a s u r e s a r e m o r e c o n v e n i e n t .
C H A P T E R 2 . C O N D I T I O N I N G
The most commonly used measures of accuracy of evidence in
st a t i st i c s are pre experimental. A pa r t i cul a r procedure is decided upon f or
use, and the accuracy of the evidence from an experiment is i de nt i f i e d wit h the
long run behavior of the procedure, were the experiment repeatedly performed.
This long run behavior is evaluated by averaging the performance of the proce
dure over the sample space Z. I n cont rast , the LP st at es t hat post experimental
reasoning should be used, wherein only the act ual observation x (and not the
ot her observations i n% t hat could have occured) is relevant . There are a
var iet y of int ermediat e posit ions which cal l f or pa r t i a l condit ioning on x and
pa r t i a l long run frequency i nt e r pr e t at i ons. Par t ly f or hi st or i ca l purposes,
and par t l y to indicat e t hat the case f or at least some sort of condit ioning is
compelling, we discuss in t his chapter various condit ioning viewpoints.
2 . 1 SIMPLE EXAMPLES
The f ollowing simple examples reveal the necessity of at least sometimes
t hinking condit ionally, and wi l l be important l a t e r .
EXAMPLE 1 . Suppose X, and X
2
are independent and
P
( X
= 1 ) = P
Q
( X. = +1 ) = , i = 1 , 2 .
H e r e > < < is an u nk no w n p ar am e t e r t o be e st im at e d f r o m X, and X
2
I t is
e asy t o se e t h at a 7 5 % c o nf id e nc e se t o f sm al l e st siz e f o r is
c ( x
v
x
2
) =
t h e p o int y Xj + Xg ) if X
1
f
t h e p o int X ^ l if X
1
=
5
T h u s, if r e p e at e d l y u se d in t h is p r o bl e m , C ( X,,X
2
) w o u l d c o nt ain w it h
p r o babil it y . 7 5 .
N o t ic e , h o w e v e r , t h at w h e n x , f x it is a b s o l u t e l y c e r t a i n t h at
= Xn+ Xp ) , w h il e w h e n x ^ = x ~ it is e q u al l y u nc e r t ain w h e t h e r = xj1 o r
= x ,+ l ( assu m ing no p r io r k no w l e d g e abo u t ) . T h u s, f r o m a p o st e x p e r im e nt al
v ie w p o int , o ne w o u l d say t h at C ( x
1 9
x
2
) c o nt ains w it h " c o nf id e nc e " 10 0 % w h e n
x , f x
2
, bu t o nl y w it h " c o nf id e nc e " 5 0 % w h e n x , = x
2
C o m m o n se nse c e r t ainl y
su p p o r t s t h e p o st e x p e r im e nt al v ie w h e r e . I t is t e c h nic al l y c o r r e c t t o c al l
C U p Xo ) a 7 5 % c o nf id e nc e se t , bu t , if af t e r se e ing t h e d at a w e k no w w h e t h e r it
is r e al l y a 10 0 % o r 5 0 % se t , r e p o r t ing 7 5 % se e m s r at h e r sil l y .
T h e abo v e e x am p l e f o c u se s t h e issu e so m e w h at : d o e s it m ak e se nse
t o r e p o r t a p r e e x p e r im e nt al m e asu r e w h e n it is k no w n t o be m isl e ad ing af t e r
se e ing t h e d at a? T h e ne x t e x am p l e al so se e m s int u it iv e l y c l e ar , y e t is t h e k e y
t o al l t h at f o l l o w s.
E XA M P L E 2 . S u p p o se a su bst anc e t o be anal y z e d c an be se nt e it h e r t o a
l abo r at o r y in N e w Y o r k o r a l abo r at o r y in C al if o r nia. T h e t w o l abs se e m
e q u al l y g o o d , so a f air c o in is f l ip p e d t o c h o o se be t w e e n t h e m , w it h " h e ad s"
d e no t ing t h at t h e l ab in N e w Y o r k w il l be c h o se n. T h e c o in is f l ip p e d and
c o m e s u p t ail s, so t h e C al if o r nia l ab is u se d . A f t e r aw h il e , t h e e x p e r im e nt al
r e su l t s c o m e bac k and a c o nc l u sio n m u st be r e ac h e d . S h o u l d t h is c o nc l u sio n
t ak e int o ac c o u nt t h e f ac t t h at t h e c o in c o u l d h av e be e n h e ad s, and h e nc e t h at
t h e e x p e r im e nt in N e w Y o r k m ig h t h av e be e n p e r f o r m e d inst e ad ?
T h is, o f c o u r se , is a v ar iant o f t h e f am o u s C o x e x am p l e ( C o x ( 19 5 8 )
se e al so C o r nf ie l d ( 19 6 9 ) ) , w h ic h c o nc e r ns be ing g iv e n ( at r and o m ) e it h e r an
ac c u r at e o r an inac c u r at e m e asu r ing inst r u m e nt ( and k no w ing w h ic h w as g iv e n) .
S h o u l d t h e c o nc l u sio n r e ac h e d by e x p e r im e nt at io n d e p e nd o nl y o n t h e inst r u m e nt
ac t u al l y u se d , o r sh o u l d it t ak e int o ac c o u nt t h at t h e o t h e r inst r u m e nt m ig h t
h av e be e n o bt aine d ?
I n sy m bo l ic f o r m , w e c an p h r ase t h is e x am p l e as a " m ix e d e x p e r im e nt "
C O N D I T I O N I N G 1
in w h ic h w it h p r o babil it ie s ^ ( ind e p e nd e nt o f ) e it h e r e x p e r im e nt E , o r
e x p e r im e nt E p ( bo t h p e r t aining t o ) w il l be p e r f o r m e d . S h o u l d t h e anal y sis
d e p e nd o nl y o n t h e e x p e r im e nt ac t u al l y p e r f o r m e d , o r sh o u l d t h e p o ssibil it y o f
h av ing d o ne t h e o t h e r e x p e r im e nt be t ak e n int o ac c o u nt ?
T h e o bv io u s int u it iv e answ e r t o t h e q u e st io ns in t h e abo v e e x am p l e
is t h at o nl y t h e e x p e r im e nt ac t u al l y p e r f o r m e d sh o u l d m at t e r . Bu t t h is is
c o u nt e r t o p r e e x p e r im e nt al f r e q u e nt ist r e aso ning , w h ic h say s t h at o ne sh o u l d
av e r ag e o v e r al l p o ssibl e o u t c o m e s ( h e r e , inc l u d ing t h e c o in f l ip ) . O ne c o u l d
ar g u e t h at it is c o r r e c t t o c o nd it io n o n t h e c o in f l ip , and t h e n u se t h e
f r e q u e nt ist m e asu r e s f o r t h e e x p e r im e nt ac t u al l y p e r f o r m e d , bu t t h e L P d is
al l o w s t h is and is ( su r p r ising l y ) d e r iv abl e sim p l y f r o m c o nd it io ning o n t h e
c o in f l ip p l u s su f f ic ie nc y ( se e C h ap t e r 3 ) .
E XA M P L E 3. Fo r a t e st ing e x am p l e , su p p o se it is d e sir e d t o t e st H Q : = 1
v e r su s H : = 1, base d o n X ^ 7 ? ( ,. 2 5 ) . T h e r e j e c t io n r e g io n X >_ 0 g iv e s a
t e st w it h e r r o r p r o babil it ie s ( t y p e I and t y p e I I ) o f . 0 2 2 8 . I f x = 0 is
o bse r v e d , it is t h e n p e r m issibl e t o st at e t h at H
Q
is r e j e c t e d , and t h at t h e
e r r o r p r o babil it y is

= . 0 2 2 8 . C o m m o n se nse , h o w e v e r , ind ic at e s t h at t h e
d at a x = 0 f ail s t o d isc r im inat e at al l be t w e e n H
n
and H . A ny se nsibl e
u a
p e r so n w o u l d be e q u al l y u nc e r t ain as t o t h e t r u t h o f H
n
o r H ( base d j u st o n
u a
the data x = 0 ) . Suppose on the ot her hand, t hat x = 1 is observed. Then
( pre experiment ally) one can s t i l l only r e j e ct at = . 0 2 2 8 , but x = 1 is four
standard deviat ions from = 1 , so the evidence against H
Q
seems overwhelming.
Cl e ar l y, the act ual i nt ui t i ve evidence conveyed by x can be quit e
di f f e r e nt from the pre experimental evidence. This has led many f requent ist s
to pr ef er the use of P values to f ixe d e r r or pr oba bi l i t i e s. The P value
( against H
Q
) would here be P _^( X _> x ) , a measure of evidence against H
Q
wit h
much more dependence on the act ual observat ion, x, than mere r e j e ct i on at
= . 0 2 2 8 . (Even P values can be c r i t i c i z e d from a condit ional viewpoint ,
however see Section 4 . 4 . )
8
T H E L I K E L I H O O D P R I N C I P L E
N o t e t h at t h e r e is no t h ing l o g ic al l y w r o ng w it h r e p o r t ing e r r o r
p r o babil it ie s in E x am p l e 3; it j u st se e m s t o be an inad e q u at e r e f l e c t io n o f t h e
e v id e nc e c o nv e y e d by t h e d at a t o r e p o r t = . 0 2 2 8 f o r b o t h x = 0 and x = 1.
P r at t ( 19 7 7 ) ( p e r h ap s so m e w h at t o ng u e inc h e e k ) t h u s c o ins
T H E P R I N C I P L E O F A D E Q U A C Y . A c o n c e p t o f s t a t i s t i c a l e v i d e n c e i s ( v e r y )
i n a d e q u a t e i f i t d o e s n o t d i s t i n g u i s h e v i d e n c e o f ( v e r y ) d i ffe r e n t s t r e n g t h s .
E XA M P L E 4 a. S u p p o se X is 1, 2 , o r 3 and is 1 o r 2 , w it h P
( x ) g iv e n in t h e
f o l l o w ing t abl e :
p
o
P
l
1
. 0 0 9
. 0 0 1
X
2
. 0 0 1
. 9 8 9
3
. 9 9
. 01
T h e t e st , w h ic h ac c e p t s P
Q
w h e n x = 3 and ac c e p t s P , o t h e r w ise , is a m o st
p o w e r f u l t e st w it h b o t h e r r o r p r o babil it ie s e q u al t o . 0 1. H e nc e , it w o u l d be
v al id t o m ak e t h e f r e q u e nt ist st at e m e nt , u p o n o bse r v ing x = 1, " M y t e st h as
r e j e c t e d P
Q
and t h e e r r o r p r o babil it y is . 0 1. " T h is se e m s v e r y m isl e ad ing ,
sinc e t h e l ik e l ih o o d r at io is ac t u al l y 9 t o 1 in fa v o r o f P
Q
, w h ic h is be ing
r e j e c t e d .
E XA M P L E 4 b. O ne c o u l d o bj e c t in E x am p l e 4 a, t h at t h e .01 l e v e l t e st is
inap p r o p r iat e , and t h at o ne sh o u l d u se t h e .001 l e v e l t e st , w h ic h r e j e c t s o nl y
w h e n x = 2 . C o nsid e r , h o w e v e r , t h e f o l l o w ing sl ig h t l y c h ang e d v e r sio n:
p
o
P
l
1
. 0 0 5
. 0 0 5 1
X
2
. 0 0 5
. 9 8 4 9
3
. 9 9
. 01
A g ain t h e t e st w h ic h r e j e c t s P
Q
w h e n x = 1 o r 2 and ac c e p t s o t h e r w ise h as e r r o r
p r o babil it ie s e q u al t o . 0 1, and no w it ind e e d se e m s se nsibl e t o t ak e t h e
ind ic at e d ac t io ns ( if w e su p p o se an ac t io n m u s t be t ak e n) . I t st il l se e m s
unreasonable, however, to report an er r or pr obabi l i t y of .01 upon r ej ecti ng P
Q
when x = 1, since the data provides yery l i t t l e evidence i n favor of P,.
EXAMPLE 5. For a decision theor eti c example, consider the i nt er est i ng Stein
phenomenon, concerned wi th estimation of a p variate normal mean (p >_ 3) based
on X ^7? ( , I) and under sum of squares er r or loss. The usual pre experimental
measure of the performance of an estimator 6 i s the r i sk function (or expected
loss)
R( ) = E
0
j (
. ( X) )
2
.
The c l assi c al est i mat or here i s 6 (x) = x, but James and St ei n (1960) showed
that
J S
( x ) = ( 1
_ Bz|
)
f
has R( , " ) < R( , ) = p f o r a l l . One can thus r ep or t 6 " as always be
i ng b e t t e r than from a pr e exper i mental vi ewpoi nt . However, i f p = 3 and
x = ( 0, . 01, . 01) i s obser ved, then
6
J
"
S
( x) = ( 0, 49.99, 49.99),
w h ic h is an absu r d e st im at e o f . H e nc e " c an be t e r r ibl e f o r c e r t ain x .
O f c o u r se t h e p o sit iv e p ar t v e r sio n o f 6 " ,
6
J s+
( x
, = ( i . fi=| ,
+
Xi
X
corrects t hi s gl ar i ng problem, but the point is that a procedure which looks

great pre experimentally could be t er r i b l e f or par t i c ul ar x, and i t may not
always be so obvious when t hi s i s the case.
Confidence sets f or can also be developed (see Casella and
Hwang (1982 )) which have l ar ger pr obabi l i t i es of coverage than the cl assical
confidence el l i p soi d s, are never larger i n si ze, and for small | x|
consist of the si ngl e point {0}. Indeed, these sets are of the simple form
{ : !
J
~
S +
( x ) |
2
< p ( l ) } if | x | >
C ( x ) =
{ 0 } if | x | < ,
2
where x (1 ) i s the 1 t/z per centi l e of the chi square d i st r i b ut i on wi th p
degrees of freedom, and i s sui tabl y smal l . Although t hi s confidence proce
dure looks great pre experimentally, one would look rather fool i sh to conclude
when p = 3 and x = (0, .01, .01) that i s the point {0} with confidence 95%.
The above examples, though simple, i ndi cate most of the i nt ui t i ve
reasons f or condi ti oni ng. There are a wide var i ety of other such examples.
The Uniform ( , + ) d i st r i b ut i on ( ,3 known) provides a host of examples
where conditional reasoning d i f f er s considerably from pre experimental reason
ing ( c . f . Welch (1939) and Pr att (1961)). The Stein 2 stage procedure f or
2
o bt aining a c o nf id e nc e int e r v al o f f ix e d w id t h f o r t h e m e an o f a ??( , ) d is
2
t r ibu t io n is ano t h e r e x am p l e . A p r e l im inar y sam p l e al l o w s e st im at io n o f ,
f r o m w h ic h it is p o ssibl e t o d e t e r m ine t h e sam p l e siz e ne e d e d f o r a se c o nd
sam p l e in o r d e r t o g u ar ant e e an o v e r al l p r o babil it y o f c o v e r ag e f o r a f ix e d
w id t h int e r v al . Bu t w h at if t h e se c o nd sam p l e ind ic at e s t h at t h e p r e l im inar y
2
e st im at e o f w as w o e f u l l y l o w ? T h e n o ne w o u l d r e al l y h av e m u c h l e ss r e a l
c o nf id e nc e in t h e p r o p o se d int e r v al ( c . f . L ind l e y ( 19 5 8 ) and S av ag e e t . al .
( 19 6 2 ) ) . A no t h e r e x am p l e is r e g r e ssio n o n r and o m c o v ar iat e s. I t is c o m m o n
p r ac t ic e t o p e r f o r m t h e anal y sis c o nd it io nal l y o n t h e o bse r v e d v al u e s o f t h e
c o v ar iat e s, r at h e r t h an g iv ing c o nf id e nc e st at e m e nt s, e t c . , v al id in an
av e r ag e se nse o v e r al l c o v ar iat e s t h at c o u l d h av e be e n o bse r v e d . R o binso n
( 19 7 5 ) al so g iv e s e x t r e m e l y c o m p e l l ing ( t h o u g h ar t if ic ial ) e x am p l e s o f t h e
ne e d t o c o nd it io n. P ic c inat o ( 19 8 1) g iv e s so m e int e r e st ing d e c isio nt h e o r e t ic
e x am p l e s.
A f inal im p o r t ant e x am p l e is t h at o f r o bu st e st im at io n. A c o n
v inc ing c ase c an be m ad e t h at inf e r e nc e st at e m e nt s sh o u l d be m ad e c o nd it io nal l y
o n t h e r e sid u al s; if t h e d at a l o o k s c o m p l e t e l y l ik e no r m al d at a, u se no r m al
t h e o r y . Bar nar d ( 19 8 1) say s
C O N D I T I O N I N G
" W e sh o u l d r e c o g nise t h at ' r o bu st ne ss
1
o f
inf e r e nc e is a c o nd it io nal p r o p e r t y so m e
inf e r e nc e s f r o m so m e sam p l e s ar e r o bu st .
Bu t o t h e r inf e r e nc e s, o r t h e sam e inf e r e nc e s
f r o m o t h e r sam p l e s, m ay d e p e nd st r o ng l y o n
d ist r ibu t io nal assu m p t io ns. "
D e m p st e r ( 19 7 5 ) c o nt ains w e r y c o nv inc ing d isc u ssio n and a h o st o f int e r e st ing
e x am p l e s c o nc e r ning t h is issu e . R e l at e d t o c o nd it io nal r o bu st ne ss is l ar g e
sam p l e inf e r e nc e , w h ic h sh o u l d o f t e n be d o ne c o nd it io nal l y o n sh ap e f e at u r e s
o f t h e l ik e l ih o o d f u nc t io n. T h u s, in u sing asy m p t o t ic no r m al t h e o r y f o r t h e
m ax im u m l ik e l ih o o d e st im at o r , , o ne sh o u l d g e ne r al l y u se I ( ) ~ , t h e inv e r se
o f o b s e r v e d Fish e r inf o r m at io n, as t h e c o v ar ianc e m at r ix , r at h e r t h an I ( ) ~ ,
t h e inv e r se o f e x p e c t e d Fish e r inf o r m at io n. Fo r e x t e nsiv e d isc u ssio n o f
t h e se and r e l at e d issu e s se e J e f f r e y s ( 19 6 1) , P r at t ( 19 6 5 ) , A nd e r se n ( 19 7 0 ) ,
E f r o n and H ink l e y ( 19 7 8 ) , Bar nd o r f f N ie l se n ( 19 8 0 ) , C o x ( 19 8 0 ) , and H ink l e y
( 19 8 0 a,19 8 2 ) .
2 . 2 R E L E V A N T S U BS E T S
Fisher ( c . f . Fisher (1956a)) long advocated conditioning on what he
cal l ed relevant subsets of X (also cal l ed "recognizable subsets", "reference
set s", or "conditional experimental frames of reference"). There i s a con
siderable l i t er at ur e on the subj ect, which tends to be more formal than the
i nt ui t i ve type of reasoning presented i n the examples of Section 2 . 1. The
basic idea i s to f i nd subsets of X (often determined by st at i st i c s) which,
when conditioned upon, change the pre experimental measure. In Example 1, f or
instance,
X = {x: x^ = x^} U {x: x
1
t x
2
h
and t h e c o v e r ag e p r o babil it ie s o f C U ^ ^ ) c o nd it io ne d o n o bse r v ing X in t h e
" r e l e v ant " su bse t s { x : x
= x
2
) o r { x : x
]
f x
2
> ar e 1 and . 5 , r e sp e c t iv e l y .
I n E x am p l e 2 , t h e t w o o u t c o m e s o f t h e c o in f l ip d e t e r m ine t w o r e l e v ant su bse t s.
I n E x am p l e s 3, 4 , and 5 it is no t c l e ar w h at su bse t s sh o u l d be c o nsid e r e d
r e l e v ant , bu t m any r e aso nabl e c h o ic e s g iv e c o nd it io nal r e su l t s q u it e d if f e r e nt
f r o m t h e p r e e x p e r im e nt al r e su l t s.
Fo r m al t h e o r ie s o f r e l e v ant su bse t s ( c . f . Bu e h l e r ( 19 5 9 ) ) p r o c e e d
in a f ash io n anal o g o u s t o t h e f o l l o w ing . S u p p o se C ( x ) is a c o nf id e nc e p r o c e d u r e
w it h c o nf id e nc e c o e f f ic ie nt 1 f o r al l , i. e . ,
( 2 . 2 . 1) P
( C ( X) c o nt ains ) = 1 f o r al l .
T h e n B is c al l e d a r e l e v ant su bse t o f % if , f o r so m e > 0 , e it h e r
( 2 . 2 . 2 ) P
Q
( C ( X) c o nt ains | X B) < _ ( 1) f o r al l
o r
( 2 . 2 . 3 ) P
( C ( X ) c o n t a i n s | X B) > ( 1 ) + f o r a l l .

When ( 2 . 2 . 2 ) or ( 2 . 2 . 3 ) holds and x G B i s obse r ve d, i t i s que st i ona bl e whet her
( 2 . 2 . 1 ) should be t he measure of e vide nce r e p or t e d . This f ormed t he basi s of
Fi she r ' s ob j e c t i on ( Fi she r ( 1 9 5 6 b) ) t o t he Aspin Welch ( 1 9 4 9 ) sol ut i on t o t he
Be hr e ns Fi she r problem ( se e a l so Yat es ( 1 9 6 4 ) and Co r nf i e l d ( 1 9 6 9 ) ) . Anot her
example f ol l ows. ( For more examples, see Cor nf i e l d ( 1 9 6 9 ) , Olshen ( 1 9 7 7 ) , and
Fr ase r ( 1 9 7 7 ) . )
EXAMPLE 6 . (Brown ( 1 9 6 7 ) , wi t h e a r l i e r r e l a t e d examples by St e i n ( 1 9 6 1 ) and
p
Bue hle r and Fedderson ( 1 9 6 3 ) ) . I f X , , . . . , X
n
i s a sample f rom a 7 ?( , )
d i s t r i b u t i o n , bot h and unknown, t he usual 1 0 0 ( l ) % conf i de nce i nt e r va l f o r
i s
C (x.s) = ( x t
/ 2 ^ *
+ t
/ 2 ^
where x and s are the sample mean and standard de viat ion, r e spe ct ive ly, and
t i2 is the appropriat e c r i t i c a l value f or the t di st r i but i on wit h n 1 degrees
of freedom. For n = 2 and = . 5 we thus have
P
2
( C( X, S) contains ) = . 5 f or a l l ,
2
,
bu t Br o w n ( 19 6 7 ) sh o w e d t h at
P
2
( C ( X,S ) c o nt ains | X| / S <_ 1 + / 2 ) >_ f f o r al l ,
2
,
and h e nc e t h e se t
B = { ( x
r
. . . , x
n
) : | x | / s <. 1 + / 2 }
f o r m s a r e l e v ant su bse t .
T h e r e is a c o nsid e r abl e l it e r at u r e c o nc e r ning t h e e st abl ish m e nt o f
c o nd it io ns u nd e r w h ic h r e l e v ant su bse t s d o o r d o no t e x ist ( c . f . Bu e h l e r ( 19 5 9 ) ,
W al l ac e ( 19 5 9 ) , S t e in ( 19 6 1) , P ie r c e ( 19 7 3) , Bo nd ar ( 19 7 7 ) , R o binso n ( 19 7 6 ,
19 7 9 a, 19 7 9 b) , and P e d e r se n ( 19 7 8 ) ) . T h o u g h int e r e st ing , a st u d y o f t h e se
issu e s w o u l d t ak e us t o o f ar af ie l d . ( S e e S e c t io n 3. 7 . 3 f o r so m e r e l at e d m at e r
ial , h o w e v e r . ) A l so , m u c h o f t h e t h e o r y is st il l base d o n f r e q u e nt ist ( t h o u g h
p ar t l y c o nd it io nal ) m e asu r e s, and h e nc e v io l at e s t h e L P . O f c o u r se , m any
r e se ar c h e r s in t h e f ie l d st u d y t h e issu e so l e l y t o p o int o u t inad e q u ac ie s in
t h e f r e q u e nt ist v ie w p o int , and no t t o r e c o m m e nd sp e c if ic c o nd it io nal f r e q u e nt ist
m e asu r e s. I nd e e d , it is f air l y c l e ar t h at t h e e x ist e nc e o f r e l e v ant su bse t s,
su c h as in E x am p l e 6 , is no t ne c e ssar il y a p r o bl e m , sinc e w h e n v ie w e d c o m p l e t e l y
c o nd it io nal l y ( say f r o m a Bay e sian v ie w p o int c o nd it io ne d o n t h e d at a ( x ,s) ) , t h e
int e r v al C ( x ,s) is v e r y r e aso nabl e . T h u s t h e e x ist e nc e o f r e l e v ant su bse t s
m ainl y p o int s t o a ne e d t o t h ink c ar e f u l l y abo u t c o nd it io ning .
2 . 3 A N C I L L A R I T Y
T h e m o st c o m m o n t y p e o f p ar t ial c o nd it io ning ad v o c at e d in
st at ist ic s is c o nd it io ning o n an anc il l ar y st at ist ic . A n a n c i l l a r y s t a t i s t i c ,
as int r o d u c e d by Fish e r ( se e Fish e r ( 19 5 6 a) f o r d isc u ssio n and e ar l ie r r e f e r
e nc e s) , is a st at ist ic w h o se d ist r ibu t io n is ind e p e nd e nt o f . ( Fo r a
d e f init io n w h e n nu isanc e p ar am e t e r s ar e p r e se nt , se e S e c t io n 3. 5 . 5 . ) T h u s, in
E x am p l e 1, S = | XiXj is an anc il l ar y st at ist ic w h ic h , w h e n c o nd it io ne d u p o n,
g iv e s " c o nd it io nal c o nf id e nc e " f o r C ( X) o f 10 0 % o r 5 0 % as s is 1 o r 0 ,
r e sp e c t iv e l y . A nd , in E x am p l e 2 , t h e o u t c o m e o f t h e c o in f l ip is an anc il l ar y
st at ist ic . T h e f o l l o w ing is a m o r e int e r e st ing e x am p l e .
E XA M P L E 7 . S u p p o se X
1
,. . . ,X
n
ar e i. i. d . U nif o r m ( p + ^ ) . T h e n
T = ( U ,V ) = ( m in X. , m ax X. ) is a su f f ic ie nt st at ist ic , and S = V U is an
anc il l ar y st at ist ic ( h av ing a d ist r ibu t io n c l e ar l y ind e p e nd e nt o f ) . T h e
c o nd it io nal d ist r ibu t io n o f T g iv e n S = s is u nif o r m o n t h e se t
X = { ( u ,v ) : v u = s and 2 < u < + 2 " s } .
I nf erence wit h respect to t his condit ional di st r i but i on is st raight f orward.
For inst ance, a 1 0 0 ( l ) % ( condit ional) confidence i nt e r val f or is
C(U, V) = 1 (U+V) l ( l ) O s ) ,
one of t he solut ions proposed by Welch ( 1 9 3 9 ) . This condit ional i nt e r val is
considerably more appealing than various "opt imal" nonconditional i nt e r va l s,
as discussed in Pr at t ( 1 9 6 1 ) .
There are a number of d i f f i c ul t i e s in the de f i ni t i on and use of
a nci l l a r y st a t i st i c s ( c . f . Basu ( 1 9 6 4) and Cox ( 1 9 7 1 ) ) . Nevert heless, condi
t ioning on a nci l l a r i e s goes a long way towards providing be t t e r condit ional
procedures. A few ref erences, from which others can be obt ained, are Fisher
( 1 9 5 6 a) , Anderson ( 1 9 7 3 ) , Barnard ( 1 9 7 4 ) , Cox and Hinkley ( 1 9 7 4 ) , Cox ( 1 9 7 5 ) ,
Dawid ( 1 9 7 5 , 1 9 8 1 ) , Efron and Hinkley ( 1 9 7 8 ) , Barndorf f Nielsen ( 1 9 7 8 , 1 9 8 0 ) ,
Hinkley ( 1 9 7 8 , 1 9 8 0 ) , Seidenf eld ( 1 9 7 9 ) , Grambsch ( 1 9 8 0 ) , Amari ( 1 9 8 2 ) ,
Barnet t ( 1 9 8 2 ) , and Buehler ( 1 9 8 2 ) .
2 . 4 CONDITIONAL FREQUENTIST PROCEDURES
An ambitious attempt to f ormalize condit ioning wi t hi n a f r e que nt ist
framework was undertaken by Kief er ( 1 9 7 7 ) . (See also Kief er ( 1 9 7 5 , 1 9 7 6 ) ,
Brown ( 1 9 7 7 ) , Brownie and Kief er ( 1 9 7 7 ) , and Berger ( 1 9 8 4 c, 1 9 8 4 d) . ) The
f ormalizat ion was in two di st i nct di r e ct i ons, which Kief er called condit ional
confidence and estimated confidence.
2 . 4 . 1 Condit ional Confidence
The basic idea of condit ional confidence is to def ine a pa r t i t i on
W
$
: s <} of x (the sets in the part i t i on are the relevant subsets of %)>
and then associate with each set in the part i t i on the appropriate conditional
frequency measure for the procedure considered. In Example 1, the part i t i on
would be into the sets X^ = {x: x^ = x} and X^ = {x: x-j t x
2
)
I n
CONDITIONING 15
Example 2, the part i t i on would be i nto the sets where heads and t ai l s are
observed, respect i vely.
When dealing wi th a confidence procedure { C( X)} , the conditional
frequency measure t hat would be reported, i f x X were observed, is
r ( ) = P_( C( X) cont ains | X <E z) .
E XA M P L E 7 ( c o nt inu e d ) . L e t t h e p ar t it io n be { z
$
: 0 <_ s <_ 1} ( se e E x am p l e 7 ) .
T h e n, f o r t h e p r o c e d u r e { C ( U ,V ) } ,
r
s
( ) = P
( C ( U ,V ) c o nt ains | ( U ,V )
$
) 1.
I n E x am p l e s 1, 2 , and 7 it is r e l at iv e l y c l e ar w h at t o c o nd it io n
o n. I n E x am p l e s 3, 4 , 5 , and 6 , h o w e v e r , t h e r e is no c l e ar c h o ic e o f a
p ar t it io n. I n a sit u at io n su c h as E x am p l e 3, t h e f o l l o w ing c h o ic e is at t r ac
t iv e .
E XA M P L E 3 ( c o nt inu e d ) . L e t z
$
= { s,s} ( i. e . , t h e t w o p o int s s and s) f o r
s > 0 . ( W e w il l ig no r e x = 0 , sinc e it h as z e r o p r o babil it y o f o c c u r r ing . )
T h e " nat u r al " m e asu r e o f c o nd it io nal c o nf id e nc e in a t e st ing sit u at io n is t h e
c o nd it io nal e r r o r p r o babil it y f u nc t io n, d e t e r m ine d h e r e by
( 2 . 4 . 1) r
s
( l ) = r
s
( l ) = P _,( R e j e c t ing | z
s
)
P _
]
( X=s)
=
P _
1
( X=s) + P _
1
( X=s)
One would thus report the t e st outcome along wi t h t he condi t i onal e r r or
pr oba bi l i t y ( 1 +e l
x
' ) ~ . This condit ional e r r or pr oba bi l i t y has t he appealing
propert y of being close t o 1 / 2 i f | x| is near ze r o, while being very small i f
| x| i s l a r ge . Thus i t sa t i sf i e s Pr a t t ' s " Pr i nc i pl e of Adequacy. "
The reason ( f rom a frequency viewpoint ) f or f or mal l y int r oducing
a p a r t i t i o n is t o prevent such "abuses" as condit ioning on "f avorable
1 1
r e l e vant
subset s, but ignor ing unf avorable ones and present ing t he uncondit ional
measure when x i s i n an unf avorable r e l e vant subset .
2 . 4 . 2 E st im at e d C o nf id e nc e
A n al t e r nat iv e ap p r o ac h t o c o nd it io ning , w h ic h c an be j u st if ie d
f r o m a f r e q u e nt ist p e r sp e c t iv e ( c . f . K ie f e r ( 19 7 7 ) o r Be r g e r ( 19 8 4 c ) ) , is t o
p r e se nt a d at a d e p e nd e nt c o nf id e nc e f u nc t io n. I f a c o nf id e nc e se t p r o c e d u r e
C ( x ) is t o be u se d , f o r inst anc e , o ne c o u l d r e p o r t l ( x ) as t h e " c o nf id e nc e "
in C ( x ) w h e n x is o bse r v e d . P r o v id ing
( 2 . 4 . 2 ) E ( X ) ) < P
( C ( X) c o nt ains ) f o r al l ,

t h is " r e p o r t " h as t h e u su al f r e q u e nt ist v al id it y t h at , in r e p e at e d u se , C ( X)
w il l c o nt ain w it h at l e ast t h e av e r ag e o f t h e r e p o r t e d c o nf id e nc e s. T h u s,
in E x am p l e 2 , o ne c o u l d r e p o r t l ( x ) = 1 o r j as x , ^ x
2
o r x = x
2
>
r e sp e c t iv e l y . E st im at e d c o nf id e nc e ( o r , m o r e g e ne r al l y , e st im at e d r isk ) c an
be v e r y u se f u l in a nu m be r o f sit u at io ns w h e r e c o nd it io nal c o nf id e nc e f ail s
( se e K ie f e r ( 19 7 7 ) o r Be r g e r ( 19 8 4 c ) ) .
2 . 5 C R I T I C I S M S O F P A R T I A L C O N D I T I O N I N G
T h e ne e d t o at l e ast so m e t im e s c o nd it io n se e m s t o be w e l l
r e c o g niz e d , as t h e br ie f r e v ie w in t h is c h ap t e r h as ind ic at e d . T h e ap p r o ac h e s
d isc u sse d in S e c t io ns 2 . 2 , 2 . 3, and 2 . 4 . 1 c o nsid e r o nl y p ar t ial c o nd it io ning ,
h o w e v e r ; o ne st il l d o e s a f r e q u e nc y anal y sis, bu t w it h t h e c o nd it io nal d ist r i
bu t io n o f X o n a su bse t . T h e r e ar e se v e r al m aj o r c r it ic ism s o f su c h p ar t ial
c o nd it io ning . ( T h e e st im at e d c o nf id e nc e ap p r o ac h in S e c t io n 2 . 4 . 2 h as a q u it e
d if f e r e nt basis; c r it ic ism o f it w il l be g iv e n at t h e e nd o f t h is se c t io n. )
Fir st , t h e c h o ic e o f a r e l e v ant su bse t o r an anc il l ar y st at ist ic
o r a p ar t it io n { z
$
: s e S ) c an be y e r y u nc e r t ain. I nd e e d , it se e m s f air l y
c l e ar t h at it is h ar d t o ar g u e p h il o so p h ic al l y t h at o ne sh o u l d c o nd it io n o n a
c e r t ain se t o r p ar t it io n, bu t no t o n a su bse t o r su bp ar t it io n. ( A f t e r al l , it
se e m s so m e w h at st r ang e t o o bse r v e x , no t e t h at it is in, say , % , and t h e n f o r
g e t abo u t x and p r e t e nd o nl y t h at
%
is k no w n t o h av e o bt aine d . ) R e se ar c h e r s
w o r k ing w it h anc il l ar it y at t e m p t t o d e f ine " g o o d " anc il l ar y st at ist ic s t o c o n
d it io n u p o n, bu t , as m e nt io ne d e ar l ie r , t h e r e ap p e ar t o be no c o m p l e t e l y
sat isf ac t o r y d e f init io ns. A l so , anc il l ar y st at ist ic s d o no t e x ist in m any
sit u at io ns w h e r e it se e m s im p o r t ant t o c o nd it io n, as t h e f o l l o w ing sim p l e
e x am p l e sh o w s.
E XA M P L E 8 . S u p p o se = [ 0 , ] ) , and
' w it h p r o babil it y 1
X =
0 w it h p r o babil it y .
( A n inst r u m e nt m e asu r e s e x ac t l y , bu t w il l e r r o ne o u sl y g iv e a z e r o r e ad ing
w it h p r o babil it y e q u al t o . ) C o nsid e r t h e c o nf id e nc e p r o c e d u r e C ( x ) = { x }
( t h e p o int x ) . H e r e P
Q
( C ( X) c o nt ains ) = 1. I t is c l e ar , h o w e v e r , t h at o ne
w ant s t o c o nd it io n o n { x : x > 0 } , sinc e C ( x ) = { } f o r su r e if x > 0 . Bu t
t h e r e is no anc il l ar y st at ist ic w h ic h p r o v id e s su c h a c o nd it io ning .
I n sit u at io ns su c h as E x am p l e s 3, 4 , 5 , and 6 , t h e se l e c t io n o f a
p ar t it io n f o r a c o nd it io nal c o nf id e nc e anal y sis se e m s q u it e ar bit r ar y . K ie f e r
( 19 7 7 ) sim p l y say s t h at t h e c h o ic e o f a p ar t it io n m u st u l t im at e l y be l e f t t o
t h e u se r , al t h o u g h h e d o e s g iv e c e r t ain g u id e l ine s. T h e d e v e l o p m e nt o f
int u it io n o r t h e o r y f o r t h e c h o ic e o f a p ar t it io n se e m s v e r y h ar d , h o w e v e r
( se e al so K ie f e r ( 19 7 6 ) , Br o w n ( 19 7 7 ) , and Be r g e r ( 19 8 4 c ) ) .
E v e n m o r e d ist u r bing ar e e x am p l e s, su c h as E x am p l e 4 ( b) , w h e r e it
se e m s im p o ssibl e t o p e r f o r m t h e ind ic at e d se nsibl e t e st and r e p o r t c o nd it io nal
e r r o r p r o babil it ie s r e f l e c t ing t h e t r u e u nc e r t aint y w h e n x = 1 is o bse r v e d .
( A t h r e e p o int s c anno t be p ar t it io ne d int o t w o no nd e g e ne r at e se t s, and o n a
d e g e ne r at e se t t h e c o nd it io nal e r r o r p r o babil it y m u st be z e r o o r o ne . ) A ny
t h e o r y w h ic h c anno t h and l e su c h a sim p l e e x am p l e is c e r t ainl y su sp e c t .
T h e sit u at io n f o r e st im at e d c o nf id e nc e t h e o r y is m o r e am big u o u s,
be c au se it h as no t be e n v e r y e x t e nsiv e l y st u d ie d . I n p ar t ic u l ar , t h e c h o ic e
o f a p ar t ic u l ar e st im at e d c o nf id e nc e o r r isk is v e r y d if f ic u l t , in all bu t t h e
sim p l e st sit u at io ns. A nd , in sit u at io ns su c h as E x am p l e s 3 and 4 ( b) ,
e st im at e d c o nf id e nc e f u nc t io ns w il l h av e c e r t ain u nd e sir abl e p r o p e r t ie s. I n
E x am p l e 3, f o r inst anc e , any e st im at e d e r r o r p r o babil it y , ( x ) , w h ic h is
d e c r e asing in | x | and sat isf ie s t h e f r e q u e nt ist v al id it y c r it e r io n ( sim il ar t o
( 2 . 4 . 2 ) )
E
( X) > P
( T e st is in e r r o r ) f o r al l ,
m u st h av e ( 0 ) > A ( sinc e P
( T e st is in e r r o r ) = j ) . I t se e m s st r ang e ,
h o w e v e r , t o r e p o r t an e r r o r l ar g e r t h an j ( w h ic h c o u l d , int u it iv e l y , be
o bt aine d by sim p l e g u e ssing ) . Fo r m o r e e x t e nsiv e d isc u ssio n o f e st im at e d
c o nf id e nc e , se e Be r g e r ( 19 8 4 c ) .
T h e f inal ar g u m e nt ag ainst p ar t ial c o nd it io ning is t h e al r e ad y
al l u d e d t o f ac t t h at t h e m o st c l e ar c u t and " o bv io u s" f o r m o f c o nd it io ning
( E x am p l e 2 ) im p l ie s ( t o g e t h e r w it h su f f ic ie nc y ) t h e L P , w h ic h st at e s t h at
c o m p l e t e c o nd it io ning ( d o w n t o x it se l f ) is ne c e ssar y . S inc e t h is w o u l d
e l im inat e t h e p o ssibl e ap p l ic at io n o f f r e q u e nc y m e asu r e s, ne w m e asu r e s o f
e v id e nc e w o u l d c l e ar l y be c al l e d f o r .
I t sh o u l d be m e nt io ne d t h at c e r t ain o t h e r f o r m s o f st at ist ic al
inf e r e nc e ar e v e r y c o nd it io nal in nat u r e , su c h as f id u c ial inf e r e nc e d e v e l o p e d
by Fish e r ( se e H ac k ing ( 19 6 5 ) , P l ac k e t t ( 19 6 6 ) , W il k inso n ( 19 7 7 ) , P e d e r se n
( 19 7 8 ) , and D aw id and S t o ne ( 19 8 2 ) f o r t h e o r y and c r it ic ism s) , st r u c t u r al in
f e r e nc e d e v e l o p e d by Fr ase r ( c . f . Fr ase r ( 19 6 8 , 19 7 2 , 19 7 9 ) ) , and p iv o t al in
f e r e nc e d e v e l o p e d by Bar nar d ( c . f . Bar nar d ( 19 8 0 , 19 8 1) and Bar nar d and S p r o t t
( 19 8 3) ) . ( Bar ne t t ( 19 8 2 ) g iv e s a g o o d int r o d u c t io n t o al l o f t h e se ap p r o ac h e s. )
T h e sim il ar it ie s am o ng t h e se m e t h o d s ( and al so " o bj e c t iv e Bay e sian" anal y sis and
f r e q u e nt ist " inv ar ianc e " anal y sis) ar e c o nsid e r abl e , bu t t h e m o t iv at io ns c an be
q u it e d if f e r e nt . T h e se m e t h o d s r ar e l y r e su l t in u nr e aso nabl e c o nc l u sio ns f r o m
a c o nd it io nal v ie w p o int , and h e nc e d o h av e m any u se f u l im p l ic at io ns f o r
c o nd it io nal anal y sis. S p ac e p r e c l u d e s e x t e nsiv e d isc u ssio n o f t h e se
ap p r o ac h e s. ( S o m e d isc u ssio n o f st r u c t u r al and p iv o t al anal y sis w il l be g iv e n
in S e c t io ns 3.6 and 3. 7 , in t h e c o u r se o f answ e r ing a sp e c if ic c r it ic ism o f t h e
L P . ) S u f f ic e it t o say t h at t h e y ar e base d o n " int u it iv e " p r inc ip l e s w h ic h c an
be at o d d s w it h t h e L P ( and Bay e sian anal y sis) , and h e nc e l e av e u s d o u bt ing
t h e ir u l t im at e t r u t h .
C H A P T E R 3 . T H E L I K E L I H O O D P R I N C I P L E A N D G E N E R A L I Z A T I O N S
3 . 1 I N T R O D U C T I O N
T h e L P d e a l s w i t h s i t u a t i o n s in w h i c h X h a s a d e n s i t y f
0
( x ) ( w i t h
r e s p e c t t o s o m e m e a s u r e v ) f o r a l l e<B>. O f c r u c i a l i m p o r t a n c e i s t h e l i k e l i
h o o d f u n c t i o n f o r g i v e n x , g i v e n b y
( 3 . 1 . 1 )
( ) = f
( x ) ,
i . e . , t h e d e n s i t y e v a l u a t e d a t t h e o b s e r v e d v a l u e X = x a n d c o n s i d e r e d a s a
f u n c t i o n o f . O f t e n w e w i l l c a l l
v
( ) t h e l i k e l i h o o d f u n c t i o n f o r o r s i m
x
p l y t h e l i k e l i h o o d f u n c t i o n . T h e L P , w h i c h f o l l o w s , i s s t a t e d in a f o r m s u i t
a b l e f o r e a s y i n i t i a l u n d e r s t a n d i n g ; c e r t a i n i m p l i c i t q u a l i f i c a t i o n s a r e d i s
c u s s e d a t t h e e n d o f t h e s e c t i o n .
T H E L I K E L I H O O D P R I N C I P L E . A l l t h e i n f o r m a t i o n a b o u t o b t a i n a b l e f r o m a n e x
p e r i m e n t i s c o n t a i n e d i n t h e l i k e l i h o o d f u n c t i o n f o r g i v e n x . T w o l i k e l i h o o d
f u n c t i o n s f o r ( f r o m t h e s a m e o r d i f f e r e n t e x p e r i m e n t s ) c o n t a i n t h e s a m e i n f o r
m a t i o n a b o u t i f t h e y a r e p r o p o r t i o n a l t o o n e a n o t h e r .
I t h a s b e e n k n o w n s i n c e F i s h e r ( 1 9 2 5 , 1 9 3 4 ) t h a t t h e " r a n d o m " l i k e
l i h o o d f u n c t i o n
( ) is a m i n i m a l s u f f i c i e n t s t a t i s t i c f o r , a n d h e n c e c o n
t a i n s all i n f o r m a t i o n a b o u t f r o m a c l a s s i c a l v i e w p o i n t . T h e L P g o e s c o n s i d
e r a b l y f a r t h e r , h o w e v e r , m a i n t a i n i n g t h a t o n l y
( ) f o r t h e a c t u a l o b s e r v a t i o n
X = x i s r e l e v a n t .
1 9
2 0 T H E L I K E L I H O O D P R I N C I P L E
E X A M P L E 9 . S u p p o s e Y , Y
2
, . . . a r e i . i . d . B e r n o u l l i ( ) r a n d o m v a r i a b l e s . I n
e x p e r i m e n t E.., a f i x e d s a m p l e s i z e o f 1 2 o b s e r v a t i o n s i s d e c i d e d u p o n , a n d t h e
1 2
s u f f i c i e n t s t a t i s t i c X . = J Y . t u r n s o u t t o b e x , = 9 . I n e x p e r i m e n t E ~ , it
1
i = l

i s d e c i d e d t o t a k e o b s e r v a t i o n s u n t i l a t o t a l o f 3 z e r o e s h a s b e e n o b s e r v e d ,
a t w h i c h p o i n t t h e s u f f i c i e n t s t a t i s t i c X
2
= Y . t u r n s o u t t o 9 . T h e d i s t r i
b u t i o n o f X ^ in E . i s b i n o m i a l w i t h d e n s i t y
w h i c h f o r x , = 9 y i e l d s t h e l i k e l i h o o d f u n c t i o n
* g (
Q
) = ( g
2
)
9
( l )
3
T h e d i s t r i b u t i o n o f X
2
in E
2
i s n e g a t i v e b i n o m i a l w i t h d e n s i t y
f
2
(
2
) = (
X
f )
X 2
( I )
3
,
which for x
2
= 9 yi el ds the l i kel i hood function
) = (
1
9
1
)
9
(l )
3
.
I n t h i s s i t u a t i o n , t h e L P s a y s t h a t ( i ) f o r e x p e r i m e n t E . a l o n e ,
t h e i n f o r m a t i o n a b o u t i s c o n t a i n e d s o l e l y in & Q ( ) ; a n d ( i i ) s i n c e & i ( ) a n d
2
& ( ) a r e p r o p o r t i o n a l a s f u n c t i o n s o f , t h e i n f o r m a t i o n a b o u t in e x p e r i
m e n t s E
1
a n d E
2
i s i d e n t i c a l .
T h e s e c o n c l u s i o n s a r e , o f c o u r s e , a t o d d s w i t h f r e q u e n t i s t
r e a s o n i n g . T h e b i n o m i a l a n d n e g a t i v e b i n o m i a l d i s t r i b u t i o n s w i l l t e n d t o g i v e
d i f f e r e n t f r e q u e n t i s t m e a s u r e s . F o r i n s t a n c e , a o n e t a i l e d s i g n i f i c a n c e t e s t
o f H
Q
: = Y w i l l g i v e s i g n i f i c a n c e l e v e l s o f = . 0 7 3 0 a n d = . 0 3 3 8 in t h e
T H E L I K E L I H O O D P R I N C I P L E A N D G E N E R A L I Z A T I O N S
21
b i n o m i a l a n d n e g a t i v e b i n o m i a l c a s e s , r e s p e c t i v e l y , s o , i f s i g n i f i c a n c e a t t h e
= . 0 5 l e v e l w a s s o u g h t , o n e w o u l d e i t h e r r e j e c t o r n o t r e j e c t d e p e n d i n g o n
t h e m o d e l . ( S e e L i n d l e y a n d P h i l l i p s ( 1 9 7 6 ) f o r f u r t h e r d i s c u s s i o n . )
T h i s e x a m p l e a l s o e v i d e n c e s a c o n s e q u e n c e o f t h e L P t h a t w i l l b e
d i s c u s s e d l a t e r , n a m e l y t h a t t h e " s t o p p i n g r u l e " is i r r e l e v a n t w h e n d r a w i n g
i n f e r e n c e s a b o u t . H e r e , it d o e s n o t m a t t e r w h e t h e r t h e s t o p p i n g r u l e w a s t o
s a m p l e u n t i l t h e t w e l f t h o b s e r v a t i o n o r u n t i l 3 z e r o e s w e r e o b t a i n e d ; t h e d a t a
t h a t 9 o n e s a n d 3 z e r o e s w e r e o b t a i n e d is all t h a t s h o u l d b e r e l e v a n t .
I t i s i n t e r e s t i n g t h a t e v e n c e r t a i n B a y e s i a n s w o u l d , a t l e a s t f o r
m a l l y , a l s o e s p o u s e v i o l a t i o n o f t h e L P in t h i s e x a m p l e . F o r i n s t a n c e , t h e
n o n i n f o r m a t i v e ( g e n e r a l i z e d ) p r i o r s f o r t h a t a r e r e c o m m e n d e d b y Je f f r e y s
( 1 9 6 1 ) a r e ^ ) ~
?
( 1 ) ~ % in t h e b i n o m i a l c a s e , a n d
2
( ) ' ^( l ) "
1
,
in t h e n e g a t i v e b i n o m i a l c a s e . T h e s e w i l l l e a d t o d i f f e r e n t p o s t e r i o r d i s t r i
b u t i o n s a n d h e n c e ( t y p i c a l l y ) d i f f e r e n t i n f e r e n c e s , e v e n w h e n t h e l i k e l i h o o d
f u n c t i o n s a r e p r o p o r t i o n a l . ( S e e H i l l ( 1 9 7 4 a ) f o r f u r t h e r d i s c u s s i o n . )
E X A M P L E 1 0 . L e t X = { 1 , 2 , 3 } a n d = { 0 , 1 } , a n d c o n s i d e r e x p e r i m e n t s E , a n d E
2
w h i c h c o n s i s t o f o b s e r v i n g X j a n d X
2
w i t h t h e a b o v e X a n d t h e s a m e , b u t w i t h
p r o b a b i l i t y d e n s i t i e s a s f o l l o w s :
f
S
( x
l
}
1
. 9 0
. 0 9
"1
2
. 0 5
. 0 5 5
3
. 0 5
. 85 5
f
0
( x
2
}
f (
2
)
1
. 26
. 0 26
C
M

C
M
. 7 3
. 80 3
3
. 0 1
. 1 7 1
I f , n o w , x , = 1 is o b s e r v e d , t h e L P s t a t e s t h a t t h e i n f o r m a t i o n
a b o u t s h o u l d d e p e n d o n t h e e x p e r i m e n t o n l y t h r o u g h ( f
o
( l ) , f
1
( 1 ) ) = (.9, . 0 9 ) .
F u r t h e r m o r e , s i n c e t h i s is p r o p o r t i o n a l t o (.26 , . 0 2 6 ) = ( f g ( l ) , f ^( l ) ) , it
s h o u l d b e t r u e t h a t x = 1 p r o v i d e s t h e s a m e i n f o r m a t i o n a b o u t a s d o e s x.. = 1.
A n o t h e r w a y o f s t a t i n g t h e L P f o r t e s t i n g s i m p l e h y p o t h e s e s , a s h e r e , i s t h a t
t h e e x p e r i m e n t a l i n f o r m a t i o n a b o u t is c o n t a i n e d in t h e l i k e l i h o o d r a t i o f o r
2 1 . 1 T H E L I K E L I H O O D P R I N C I P L E
t h e o b s e r v e d x . N o t e t h a t t h e l i k e l i h o o d r a t i o s f o r t h e t w o e x p e r i m e n t s a r e
a l s o t h e s a m e w h e n 2 is o b s e r v e d , a n d a l s o w h e n 3 is o b s e r v e d . H e n c e , n o
m a t t e r w h i c h e x p e r i m e n t i s p e r f o r m e d , t h e s a m e c o n c l u s i o n a b o u t s h o u l d b e
r e a c h e d f o r t h e g i v e n o b s e r v a t i o n . T h i s e x a m p l e c l e a r l y i n d i c a t e s t h e s t a r t
l i n g n a t u r e o f t h e L P . E x p e r i m e n t s E
1
a n d E
2
a r e v e r y d i f f e r e n t f r o m a
f r e q u e n t i s t p e r s p e c t i v e . F o r i n s t a n c e , t h e t e s t w h i c h a c c e p t s = 0 w h e n t h e
o b s e r v a t i o n i s 1 a n d d e c i d e s = 1 o t h e r w i s e is a m o s t p o w e r f u l t e s t w i t h e r r o r
p r o b a b i l i t i e s ( o f T y p e I a n d T y p e I I , r e s p e c t i v e l y ) . 1 0 a n d .09 f o r E j , a n d .7 4
a n d . 0 2 6 f o r E
2
T h u s t h e c l a s s i c a l f r e q u e n t i s t w o u l d r e p o r t d r a s t i c a l l y
d i f f e r e n t i n f o r m a t i o n f r o m t h e t w o e x p e r i m e n t s . ( A n d t h e c o n d i t i o n a l f r e q u e n
t i s t i s a l s o l i k e l y t o r e p o r t E , a n d E
2
d i f f e r e n t l y ; i n d e e d , f o r E
2
it is h a r d
t o p e r f o r m a n y s e n s i b l e c o n d i t i o n a l f r e q u e n t i s t a n a l y s i s b e c a u s e o f t h e t h r e e
p o i n t Z a n d t h e w i d e l y d i f f e r i n g e r r o r p r o b a b i l i t i e s . )
T h i s e x a m p l e e m p h a s i z e s a v e r y i m p o r t a n t i s s u e . I t i s c l e a r t h a t
e x p e r i m e n t E , is m o r e l i k e l y t o p r o v i d e u s e f u l i n f o r m a t i o n a b o u t , a s
r e f l e c t e d b y t h e o v e r a l l b e t t e r e r r o r p r o b a b i l i t i e s . T h e L P in n o s e n s e
c o n t r a d i c t s t h i s . T h e L P a p p l i e s o n l y t o t h e i n f o r m a t i o n a b o u t t h a t i s
a v a i l a b l e f r o m k n o w l e d g e o f t h e e x p e r i m e n t a n d t h e o b s e r v e d x . E v e n t h o u g h E
h a s a m u c h b e t t e r c h a n c e o f y i e l d i n g g o o d i n f o r m a t i o n , t h e L P s t a t e s t h a t t h e
c o n c l u s i o n , o n c e x i s a t h a n d , s h o u l d b e t h e s a m e , r e g a r d l e s s o f w h e t h e r x c a m e
f r o m E
1
o r E
2
T h e c o n f l i c t o f t h e L P w i t h f r e q u e n t i s t j u s t i f i c a t i o n s s e e m s
i n e s c a p a b l e . ( S e e a l s o B i r n b a u m ( 1 9 7 7 ) . )
H i l l ( 1 9 8 7 a , b ) d i s c u s s e s a n u m b e r o f i m p o r t a n t c l a r i f i c a t i o n s o r
q u a l i f i c a t i o n s o f t h e L P . S e v e r a l o f t h e s e a r e d i s c u s s e d in d e p t h l a t e r in t h e
m o n o g r a p h , b u t it i s p e r h a p s p e d a g o c i c a l l y b e s t t o a t l e a s t m e n t i o n t h e m h e r e .
T h e f i r s t h a s t o d o w i t h t h e r o l e o f . A s p r e s e n t e d u p u n t i l n o w ,
r e p r e s e n t s o n l y t h e u n k n o w n a s p e c t o f t h e p r o b a b i l i t y d i s t r i b u t i o n o f X . F o r
t h e b u l k o f t h e m o n o g r a p h w e w i l l c o n f i n e a t t e n t i o n t o t h i s c a s e , it b e i n g t h e
T H E L I K E L I H O O D P R I N C I P L E A N D G E N E R A L I Z A T I O N S 2 1 . 2
m o s t f a m i l i a r s t a t i s t i c a l s i t u a t i o n . O f t e n , h o w e v e r , t h e r e a r e u n k n o w n s w h i c h
a r e r e l e v a n t t o a s t a t i s t i c a l p r o b l e m b u t w h i c h d o n o t d i r e c t l y a f f e c t t h e d i s
t r i b u t i o n o f X . O n e e x a m p l e i s p r e d i c t i o n , in w h i c h it i s d e s i r e d t o p r e d i c t
a n u n k n o w n r a n d o m v a r i a b l e Z , a f t e r o b s e r v i n g X . O t h e r e x a m p l e s a r i s e in
d e s i g n a n d s e q u e n t i a l a n a l y s i s p r o b l e m s , w h e r e a s y e t u n o b s e r v e d d a t a c a n
a f f e c t t h e d e c i s i o n t o b e m a d e . E x a m p l e s a r e g i v e n in S e c t i o n 3 . 5 .
I n g e n e r a l , t h e r e f o r e , t h e L P s h o u l d b e f o r m u l a t e d i n s u c h a w a y
t h a t c o n s i s t s o f a l l u n k n o w n v a r i a b l e s a n d p a r a m e t e r s t h a t a r e r e l e v a n t t o
t h e s t a t i s t i c a l p r o b l e m . ( A n y a t t e m p t t o p r e c i s e l y d e f i n e " r e l e v a n t t o t h e
s t a t i s t i c a l p r o b l e m " w o u l d i n v o l v e b o t h d e c i s i o n t h e o r y a n d m o d e l f o r m u l a t i o n ,
a n d l e a d u s t o o f a r a s t r a y . ) T h e m a j o r d i f f i c u l t y w i t h w o r k i n g in s u c h g e n e r
a l i t y i s t h a t o f d e f i n i n g w h a t is t h e n m e a n t b y a l i k e l i h o o d f u n c t i o n f o r
( c f . B a y a r r i , D e G r o o t , a n d K a d a n e ( 1 9 8 7 ) ) . W e h a v e o p t e d f o r d i s c u s s i n g t h i s
g e n e r a l s i t u a t i o n o n l y in S e c t i o n 3 . 5 , t h o u g h w e b e l i e v e t h a t v i r t u a l l y a l l
i s s u e s r a i s e d f o r t h e s p e c i a l c a s e o f b e i n g t h e m o d e l p a r a m e t e r a l s o a p p l y
t o a p p r o p r i a t e f o r m u l a t i o n s o f t h e g e n e r a l s i t u a t i o n . I n a n y c a s e , it i s
i m p o r t a n t t o k e e p in m i n d t h e q u a l i f i c a t i o n t h a t m u s t c o n t a i n a l l u n k n o w n s
r e l e v a n t t o t h e p r o b l e m f o r t h e L P t o b e v a l i d in i t s s i m p l e f o r m .
A s e c o n d q u a l i f i c a t i o n f o r t h e L P i s t h a t it o n l y a p p l i e s f o r a
f u l l y s p e c i f i e d m o d e l { f
f l
} I f t h e r e i s u n c e r t a i n t y in t h e m o d e l , a n d i f o n e
d e s i r e s t o g a i n i n f o r m a t i o n a b o u t w h i c h m o d e l is c o r r e c t , t h a t u n c e r t a i n t y m u s t
b e i n c o r p o r a t e d i n t o t h e d e f i n i t i o n o f .
A t h i r d q u a l i f i c a t i o n i s t h a t , in a p p l y i n g t h e L P t o t w o d i f f e r e n t
e x p e r i m e n t s , it i s i m p e r a t i v e t h a t b e t h e s a m e u n k n o w n q u a n t i t y in e a c h .
T h u s , in E x a m p l e 9 , w e a s s u m e d t h a t r e p r e s e n t e d t h e s a m e s u c c e s s p r o b a b i l i t y
in e i t h e r t h e b i n o m i a l o r n e g a t i v e b i n o m i a l e x p e r i m e n t . I n a p p l y i n g t h e L P t o
t w o d i f f e r e n t e x p e r i m e n t s , w e a l s o r e q u i r e t h a t t h e c h o i c e o f a n e x p e r i m e n t b e
n o n i n f o r m a t i v e ( e . g . i m p l e m e n t e d b y a c h a n c e m e c h a n i s m n o t i n v o l v i n g ) ;
t h i s m i g h t b e v i o l a t e d i f t h e e x p e r i m e n t e r c h o o s e s a m o n g p o s s i b l e e x p e r i m e n t s
on t h e b a s i s o f p r i o r b e l i e f s . I n f o r m a t i v e e x p e r i m e n t a l c h o i c e s m a y b e h a n d l e d
by t h e m e t h o d s d i s c u s s e d in S e c t i o n 4 . 2 . 7 .
F u r t h e r e l a b o r a t i o n a n d o t h e r q u a l i f i c a t i o n s w i l l b e i n t r o d u c e d a s
w e p r o c e e d . U n d e r s t a n d i n g t h e l i m i t a t i o n s a n d t h e d o m a i n o f a p p l i c a b i l i t y o f
t h e L P is a l m o s t a s i m p o r t a n t a s u n d e r s t a n d i n g i t s b a s i s a n d i m p l i c a t i o n s .
3 . 2 H I S T O R Y O F T H E L I K E L I H O O D P R I N C I P L E
F o r a h i s t o r y o f t h e c o n c e p t o f l i k e l i h o o d , s e e E d w a r d s ( 1 9 7 4 ) .
T h e n a m e " l i k e l i h o o d " f i r s t a p p e a r e d in F i s h e r ( 1 9 2 1 ) . F i s h e r m a d e c o n s i d e r -
a b l e u s e o f l i k e l i h o o d a n d c o n d i t i o n i n g c o n c e p t s ( c f . F i s h e r ( 1 9 2 5 , 1 9 3 4 ,
1 9 5 6 a ) ) a n d c a m e c l o s e t o e s p o u s i n g t h e L P in F i s h e r ( 1 9 5 6 a ) , b u t r e f r a i n e d
f r o m c o m p l e t e c o m m i t t m e n t t o t h e p r i n c i p l e . V e r s i o n s o f t h e LP w e r e d e v e l o p e d
a n d p r o m o t e d b y B a r n a r d in a s e r i e s o f w o r k s ( B a r n a r d ( 1 9 4 7 a , 1 9 4 7 b , 1 9 4 9 ) ) .
L i k e l i h o o d c o n c e p t s w e r e a l s o e m p l o y e d by a n u m b e r o f o t h e r s t a t i s t i c i a n s ,
c f . B a r t l e t t ( 1 9 3 6 , 1 9 5 3 ) .
T h e L P r e c e i v e d m a j o r n o t i c e in 1 9 6 2 , d u e t o B a r n a r d , J e n k i n s ,
a n d W i n s t e n ( 1 9 6 2 ) a n d B i r n b a u m ( 1 9 6 2 a ) . B o t h p a p e r s ( and t h e D i s c u s s i o n s o f
t h e m ) c o n t a i n e d n u m e r o u s c o m p e l l i n g e x a m p l e s in f a v o r o f t h e L P , a n d a l s o
p r o v i d e d a x i o m a t i c d e v e l o p m e n t s o f t h e L P f r o m t h e s i m p l e r ( and m o r e
b e l i e v a b l e ) c o n c e p t s o f s u f f i c i e n c y a n d c o n d i t i o n a l i t y . B i r n b a u m ' s d e v e l o p -
m e n t is m o r e c o n v i n c i n g , a n d wil"
1
b e g i v e n in t h e n e x t s e c t i o n . T h e w o r k s i n c e
t h e n o n t h e L P a n d i t s c o n s e q u e n c e s is c o n s i d e r a b l e , a s c a n b e s e e n f r o m t h e
r e f e r e n c e s . N o t e w o r t h y g e n e r a l d i s c u s s i o n s c a n b e f o u n d in P r a t t ( 1 9 6 5 ) , C o x
a n d H i n k l e y ( 1 9 7 4 ) , D a w i d ( 1 9 8 1 ) , B a r n e t t ( 1 9 8 2 ) , a n d e s p e c i a l l y B a s u ( 1 9 7 5 ) .
In f a i r n e s s , it s h o u l d b e m e n t i o n e d t h a t B a r n a r d c a m e t o s u p p o r t
o n l y a l i m i t e d v e r s i o n o f t h e L P a n d B i r n b a u m u l t i m a t e l y c a m e c l o s e t o
T H E L I K E L I H O O D P R I N C I P L E A N D G E N E R A L I Z A T I O N S 2 3
r e j e c t i n g i t . T h e r e a s o n s w i l l b e d i s c u s s e d in S e c t i o n s 3 . 6 . 4 a n d 4 . 1 ,
r e s p e c t i v e l y .
T h e a b o v e d e v e l o p m e n t i s a b r i e f h i s t o r y o f t h e L P f r o m a n o n
B a y e s i a n p e r s p e c t i v e . T h e L P w a s a l w a y s i m p l i c i t i n t h e B a y e s i a n a p p r o a c h t o
s t a t i s t i c s . T h i s is b e c a u s e , i f ( ) is a p r i o r d e n s i t y f o r , t h e n t h e
p o s t e r i o r d e n s i t y is
( | x ) = ( H ( ) / m ( x )
( a s s u m i n g m ( x ) = E
( ) > 0 ) , w h i c h d e p e n d s o n t h e e x p e r i m e n t o n l y t h r o u g h
x
i ( ) ( p r e s u m i n g t h a t s e l e c t i o n o f is i n d e p e n d e n t o f E a n d x ) . S i n c e a l l
B a y e s i a n i n f e r e n c e f o l l o w s f r o m t h e p o s t e r i o r , t h e L P is a n i m m e d i a t e c o n s e
q u e n c e o f t h e B a y e s i a n p a r a d i g m . T h u s Je f f r e y s ( 1 9 6 1 ) s a y s
" C o n s e q u e n t l y t h e w h o l e o f t h e i n f o r m a t i o n
c o n t a i n e d i n t h e o b s e r v a t i o n t h a t is r e l e
v a n t t o t h e p o s t e r i o r p r o b a b i l i t i e s o f
d i f f e r e n t h y p o t h e s e s is s u m m e d u p in t h e
v a l u e s t h a t t h e y g i v e t o t h e l i k e l i h o o d . "
A n i m p o r t a n t p o i n t h e r e i s t h a t
( ) is a l l t h a t m a t t e r s t o a
B a y e s i a n , n o m a t t e r w h a t p r i o r d e n s i t y is u s e d . I t is t e m p t i n g , t h e r e f o r e ,
t o s a y t h a t , i f i ( ) c o n t a i n s a l l t h e s a m p l e i n f o r m a t i o n a b o u t r e g a r d l e s s
o f t h e k n o w n p r i o r , t h e n i ( ) s h o u l d c o n t a i n a l l t h e s a m p l e i n f o r m a t i o n e v e n
x
w h e n t h e p r i o r i s u n k n o w n .
T h e a b o v e r e l a t i o n s h i p b e t w e e n t h e L P a n d B a y e s i a n a n a l y s i s s h o u l d
p r o b a b l y b e q u a l i f i e d t o s o m e e x t e n t , in t h a t i t i s p o s s i b l e t o b e a
" f r e q u e n t i s t B a y e s i a n . " O n e c a n b e l i e v e t h a t o n l y f r e q u e n t i s t m e a s u r e s o f
p r o c e d u r e p e r f o r m a n c e h a v e v a l i d i t y , a n d y e t , b e c a u s e o f v a r i o u s r a t i o n a l i t y
o r a d m i s s i b i l i t y a r g u m e n t s , b e l i e v e t h a t t h e o n l y r e a s o n a b l e p r o c e d u r e s a r e
B a y e s p r o c e d u r e s , a n d t h a t t h e b e s t m e t h o d o f c h o o s i n g a p r o c e d u r e i s t h r o u g h
c o n s i d e r a t i o n o f p r i o r i n f o r m a t i o n a n d a p p l i c a t i o n o f t h e B a y e s i a n p a r a d i g m .
T h e p o s t e r i o r d i s t r i b u t i o n w o u l d p r o v i d e a c o n v e n i e n t m a t h e m a t i c a l d e v i c e
f o r d e t e r m i n i n g t h e b e s t p r o c e d u r e , f r o m t h i s v i e w p o i n t , b u t o v e r a l l
f r e q u e n t i s t B a y e s m e a s u r e s o f p e r f o r m a n c e , n o t p o s t e r i o r B a y e s m e a s u r e s , w o u l d
b e t h e r e l e v a n t m e a s u r e s o f a c c u r a c y . T h e L P d i r e c t l y a t t a c k s t h i s v i e w ,
a r g u i n g t h a t t h i n k i n g " c o n d i t i o n a l B a y e s ,
1 1
n o t " f r e q u e n t i s t B a y e s , " is
i m p o r t a n t .
A s s o m e w h a t o f a n a s i d e h e r e , t h e r e a r e t w o o t h e r r e a s o n s w h y
B a y e s i a n s s h o u l d b e v e r y i n t e r e s t e d i n t h e L P . T h e f i r s t is t h a t , i n c o m p l i c a
t e d r e a l p r o b l e m s , B a y e s i a n s w i l l o f t e n s p e n d m u c h o f t h e i r t i m e s i m p l y l o o k i n g
a t l i k e l i h o o d f u n c t i o n s a n d d o i n g m a x i m u m l i k e l i h o o d a n a l y s e s , d u e t o c a l c u l a
t i o n a l c o m p l e x i t i e s o f a f u l l B a y e s i a n a n a l y s i s . E m p h a s i z i n g t h e i m p o r t a n c e o f
t h e o b s e r v e d l i k e l i h o o d f u n c t i o n is t h u s t o b e e n c o u r a g e d . F i n a l l y , t h e r e is
t h e v e r y p r a g m a t i c r e a s o n t h a t p r o m o t i n g t h e B a y e s i a n p o s i t i o n c a n o f t e n b e
m o s t e f f e c t i v e l y d o n e b y f i r s t s e l l i n g t h e L P , s i n c e t h e l a t t e r c a n b e d o n e
w i t h o u t i n t r o d u c i n g t h e e m o t i o n a l l y c h a r g e d i s s u e o f p r i o r d i s t r i b u t i o n s ( s e e
B e r g e r ( 1 9 8 4 b ) ) .
3 . 3 B I R N B A U M ' S D E V E L O P M E N T T H E D I S C R E T E C A S E
Birnbaum's ( 1 9 6 2 a) development of the LP from the i nt ui t i ve l y
simpler and more plausible concepts of suf f icie ncy and condit ionalit y is
f ormally correct only in the case of experiments wit h discret e densit ies ( see
Section 3 . 4 . 1 ) . Since the discret e case is also the easiest t o understand
i nt ui t i ve l y, we r e st r i c t ourselves in t his sect ion to a discret e sample space
X. We car e f ul l y out l i ne Birnbaum's argument, to allow easy dissect ion by those
who f i nd i t hard to believe the conclusion. The mathematical st yl e is kept
f a i r l y inf ormal; r igor poses no problem because of the discret eness.
3 . 3 . 1 Evidence, C ondi t i onal i t y, and Suf f iciency
By an experiment E, we herein mean the t r i p l e ( X, , { f
n
} ) , where
t h e r andom v a r i a b l e X, t a k i n g va l ue s X\ X a nd ha vi ng d e n s i t y f
n
( x ) f o r some
in , is o b s e r v e d . ( B e c a u s e o f t h e d i s c r e t e n e s s , t h e d e n s i t y c a n b e a s s u m e d t o
e x i s t , a n d w e w i l l t a k e a l l s u b s e t s o f % t o b e m e a s u r a b l e . ) F o r s i m p l i c i t y o f
n o t a t i o n , x a n d w i l l b e s u p p r e s s e d in t h e d e s c r i p t i o n o f E . V i r t u a l l y a l l
s t a t i s t i c a l m e t h o d o l o g i e s r e q u i r e o n l y t h e a b o v e i n f o r m a t i o n c o n c e r n i n g a n
e x p e r i m e n t . ( T h e " s t r u c t u r a l t h e o r y " o f F r a s e r a n d t h e " p i v o t a l t h e o r y " o f
B a r n a r d d e e m a d d i t i o n a l i n f o r m a t i o n r e l a t i n g X , , a n d t h e r a n d o m n e s s t o b e
i m p o r t a n t , h o w e v e r . T h i s i s s u e w i l l b e d i s c u s s e d i n S e c t i o n s 3 . 6 . 4 a n d 3 . 7 . )
T h e o u t c o m e o f t h e e x p e r i m e n t is t h e d a t a X = x , a n d f r o m E a n d x
w e a r e t o i n f e r o r c o n c l u d e s o m e t h i n g a b o u t ( o r a b o u t s o m e t h i n g r e l a t e d t o
) . F o l l o w i n g B i r n b a u m ( 1 9 6 2 a ) , w e w i l l c a l l t h i s i n f e r e n c e , c o n c l u s i o n , o r
r e p o r t t h e e v i d e n c e a b o u t a r i s i n g f r o m E a n d x , a n d w i l l d e n o t e t h i s b y
E v ( E , x ) . W e p r e s u p p o s e n o t h i n g a b o u t w h a t t h i s e v i d e n c e i s ; it c o u l d ( a t t h i s
s t a g e ) b e a n y s t a n d a r d m e a s u r e o f e v i d e n c e , o r s o m e t h i n g e n t i r e l y n e w . ( S i n c e
E is an a r g u m e n t , it c o u l d c e r t a i n l y b e a f r e q u e n t i s t m e a s u r e . ) A l s o , w e d o
n o t p r e c l u d e t h e p o s s i b i l i t y t h a t E v ( E , x ) d e p e n d s o n " o t h e r i n f o r m a t i o n , " s u c h
as p r i o r i n f o r m a t i o n a b o u t , o r a l o s s f u n c t i o n in a d e c i s i o n p r o b l e m . T h e
f o c u s w i l l b e o n t h e m a n n e r in w h i c h t h e " r e p o r t " E v ( E , x ) s h o u l d d e p e n d o n E a n d
x . ( D a w i d ( 1 9 7 7 ) p r e f e r s t o t a l k a b o u t m e t h o d s o f i n f e r e n c e b a s e d o n E a n d x ,
a n d p r i n c i p l e s w h i c h t h e s e m e t h o d s s h o u l d s a t i s f y . I n a s e n s e , b y l e t t i n g
E v ( E , x ) d e n o t e w h a t e v e r c o n c l u s i o n o n e is g o i n g t o r e p o r t , w e a r e a l s o t a k i n g
t h i s v i e w , w h i l e k e e p i n g B i r n b a u m ' s n o t a t i o n . ) A s o n e f i n a l p o i n t , E v ( E , x )
c o u l d b e a c o l l e c t i o n o f " e v i d e n c e s " a b o u t , o b v i a t i n g t h e c r i t i c i s m t h a t t h e
L P is b a s e d o n t h e a s s u m p t i o n t h a t a s i n g l e m e a s u r e o f e v i d e n c e e x i s t s .
T h e C o n d i t i o n a l i t y P r i n c i p l e e s s e n t i a l l y s a y s t h a t , i f a n e x p e r i
m e n t is s e l e c t e d b y s o m e r a n d o m m e c h a n i s m i n d e p e n d e n t o f , t h e n o n l y t h e
e x p e r i m e n t a c t u a l l y p e r f o r m e d is r e l e v a n t . ( T h e s e l e c t i o n m e c h a n i s m is
a n c i l l a r y , s o t h i s is a v e r s i o n o f c o n d i t i o n i n g o n a n a n c i l l a r y s t a t i s t i c . )
T h e g e n e r a l c o n d i t i o n a l i t y p r i n c i p l e is n o t n e e d e d h e r e . I n d e e d w e n e e d o n l y
t h e f o l l o w i n g c o n s i d e r a b l y w e a k e r p r i n c i p l e , n a m e d b y B a s u ( 1 9 7 5 ) .
W E A K C O N D I T I O N A L I T Y P R I N C I P L E ( W C P J . S u p p o s e t h e r e a r e t w o e x p e r i m e n t s
1 2
E
= ( X
, , { f } ) a n d E
9
= ( X
9
, , ( f
Q
} ) w h e r e o n l y t h e u n k n o w n p a r a m e t e r
n e e d b e c o m m o n t o t h e t w o e x p e r i m e n t s . C o n s i d e r t h e m i x e d e x p e r i m e n t E * ,
w h e r e b y J = 1 o r 2 i s o b s e r v e d , e a c h h a v i n g p r o b a b i l i t y ^ ( i n d e p e n d e n t o f ,
X , , o r X p j j a n d e x p e r i m e n t E , i s t h e n p e r f o r m e d . F o r m a l l y , E * = ( X * , , f } ) >
w h e r e X * = ( J, X j ) a n d f * ( ( j , X j ) ) = \
f
(
j )
h e n
>
E v ( E * , ( j , . ) ) = E v ( E . , . ) ,
j J J
i . e . , t h e e v i d e n c e a b o u t f r o m E * i s j u s t t h e e v i d e n c e f r o m t h e e x p e r i m e n t
a c t u a l l y p e r f o r m e d .
T h e W C P is n o t h i n g b u t a f o r m a l i z a t i o n o f E x a m p l e 2 , a n d h e n c e is
e s s e n t i a l l y d u e t o C o x ( 1 9 5 8 ) . I t is h a r d t o d i s b e l i e v e t h e W C P , y e t , a s
m e n t i o n e d a f t e r E x a m p l e 2 , e v e n t h e W C P a l o n e h a s s e r i o u s c o n s e q u e n c e s .
T u r n i n g f i n a l l y t o t h e f a m i l i a r c o n c e p t o f s u f f i c i e n c y , w e s t a t e
t h e f o l l o w i n g w e a k v e r s i o n ( n a m e d by D a w i d ( 1 9 7 7 ) ) .
W E A K S U F F I C I E N C Y P R I N C I P L E ( W S P ) . C o n s i d e r a n e x p e r i m e n t E = ( X , , { f
) ) , a n d
s u p p o s e T ( X ) i s a s u f f i c i e n t s t a t i s t i c f o r . T h e n , i f T ( x ^) = T ( x 2 ) ,
E v t E ^ ) = E v ( E , x
2
) .
T h e L P w i l l b e s e e n t o f o l l o w d i r e c t l y f r o m t h e W C P a n d W S P . A
v a r i e t y o f a l t e r n a t e p r i n c i p l e s a l s o l e a d t o t h e L P ( c f . B a s u ( 1 9 7 5 ) , D a w i d
( 1 9 7 7 ) , B a r n d o f f N i e l s e n ( 1 9 7 8 ) , B e r g e r ( 1 9 8 4 a ) , B h a v e ( 1 9 8 4 ) , a n d E v a n s ,
F r a s e r , a n d M o n e t t e ( 1 9 8 5 c , 1 9 8 6 ) ) . T h e W C P a n d W S P a r e t h e m o s t f a m i l i a r ,
h o w e v e r . A n o t h e r p r o m i n e n t p r i n c i p l e i s " M a t h e m a t i c a l E q u i v a l e n c e , " g i v e n in
B i r n b a u m ( 1 9 7 2 ) . T h i s p r i n c i p l e is a w e a k v e r s i o n o f t h e s u f f i c i e n c y p r i n c i p l e ,
s t a t i n g t h a t i f , in a g i v e n e x p e r i m e n t E , f
( x , ) = f
( x
0
) f o r all , t h e n
E v ( E , X j ) = E v ( E , x
2
) . O n e c o u l d b a s e t h e L P o n m a t h e m a t i c a l e q u i v a l e n c e , p l u s
a m i n o r g e n e r a l i z a t i o n o f t h e W C P . T h e w e a k e n i n g o f s u f f i c i e n c y i s c a r r i e d t o
t h e u l t i m a t e in E v a n s , F r a s e r , a n d M o n e t t e ( 1 9 8 6 ) , w h i c h d e r i v e s t h e L P s o l e l y
f r o m a g e n e r a l i z e d v e r s i o n o f t h e c o n d i t i o n a l i t y p r i n c i p l e .
3 . 3 . 2 A x i o m a t i c D e v e l o p m e n t
T h e f o r m a l s t a t e m e n t o f t h e L P is a s f o l l o w s .
F O R M A L L I K E L I H O O D P R I N C I P L E . C o n s i d e r t w o e x p e r i m e n t s E , = { X, , , { f
1
} ) a n d
E
=
( X o s { f g } ) > w h e r e i s t h e s a m e q u a n t i t y i n e a c h e x p e r i m e n t . S u p p o s e
t h a t f o r t h e p a r t i c u l a r r e a l i z a t i o n s x a n d x | f r o m E , a n d E 2 r e s p e c t i v e l y ,
T H E L I K E L I H O O D P R I N C I P L E A N D G E N E R A L I Z A T I O N S
27
*
x
* ( ) = * ( )
x
1
x
2
const ant c ( i . e . , f
( x ) = c f . ( x ) / o r ) . T/zerc
E v ( E
x * ) = Ev( E
2
, x * ) .
LIKELIHOOD PRINCIPLE COROLLARY. If E = (X, , {f
Q
}) is an experiment, then
Ev(E, x) should depend on E and x only through
( ) .
THEOREM 1 (Birnbaum ( 1 9 6 2a) ) . The Formal Likelihood Principle follows from the
WCP and the SP. The converse is also true.
Proof. I f E
1
and E,, are the two experiments about , consider the mixed
experiment E* as defined in the WCP. From the WCP we know t hat
( 3 . 3 . 1 ) Ev( E* , ( j, x. . ) ) = Ev( E
j f
Xj ) .
N e x t , t h i n k i n g s o l e l y o f E * w i t h r a n d o m o u t c o m e ( J, X j ) , c o n s i d e r
t h e s t a t i s t i c
f
( l . x f ) i f J = 2 , X
2
= x *
T ( J. X j ) =
( J, X j ) o t h e r w i s e .
( T h u s t h e t w o o u t c o m e s ( l , x ^) a n d ( 2 , x ^) r e s u l t in t h e s a m e v a l u e o f T . ) T is
a s u f f i c i e n t s t a t i s t i c f o r . T h i s is c l e a r , s i n c e
P
( X * = ( j , X j ) | T = t f ( l , x | ) ) =
1 i f ( j . x J = t
j
0 o t h e r w i s e ,
and
P
( X * = ( l , x f ) | T = 1 P
( X * = ( 2 , x | ) | T = ( l . x f ) )
( x
) +
\
all o f w h i c h a r e i n d e p e n d e n t o f . T h e W S P t h u s i m p l i e s t h a t
( 3 . 3 . 2 ) E v ( E * , ( l , x f ) ) = E v ( E * , ( 2 , x * ) ) .
C o m b i n i n g ( 3 . 3 . 1 ) a n d ( 3 . 3 . 2 ) e s t a b l i s h e s t h e r e s u l t .
T o p r o v e t h a t t h e L P i m p l i e s t h e W C P , o b s e r v e t h a t , f o r E * ,
T h i s is c l e a r l y p r o p o r t i o n a l t o f ? ( x , ) , t h e l i k e l i h o o d f u n c t i o n i n E . w h e n x .
j j j
is o b s e r v e d , s o t h e L P i m p l i e s t h a t
E v ( E * , ( j ,
X j
) ) = E v E j . X j ) .
T o p r o v e t h a t t h e L P i m p l i e s t h e W S P , i t s u f f i c e s t o n o t e t h a t , i f
T (xj ) = T ( x
2
) i n a n e x p e r i m e n t f o r w h i c h T is s u f f i c i e n t , t h e n x
1
a n d x ^ h a v e
p r o p o r t i o n a l l i k e l i h o o d f u n c t i o n s . ||
P r o o f o f t h e L P C o r o l l a r y . F o r g i v e n x * X , d e f i n e
1 i f X = x *
Y =
0 i f X f x * ,
a n d n o t e t h a t Y h a s d i s t r i b u t i o n g i v e n b y
( 3 . 3 . 3 ) f j ( l ) = f
( x * ) = l f j ( ) .
F o r t h e e x p e r i m e n t E * o f o b s e r v i n g Y , i t f o l l o w s f r o m t h e L P t h a t
E v ( E , x * ) = E v ( E * , l ) .
B u t E * , a n d h e n c e E v ( E * , l ) , d e p e n d o n l y o n f . ( x * ) =
v
* ( ) ( u s i n g ( 3 . 3 . 3 ) ) . ||
T h e a b o v e r e s u l t s a r e w o r t h d w e l l i n g u p o n f o r a m o m e n t . T h e L P i s
e x t r e m e l y r a d i c a l f r o m t h e v i e w p o i n t o f c l a s s i c a l s t a t i s t i c s , a s w i l l b e s e e n
i n C h a p t e r 4 . Y e t t o r e j e c t t h e L P , o n e m u s t l o g i c a l l y r e j e c t e i t h e r t h e W C P
o r t h e W S P . B u t t h e W S P i s , i t s e l f , a c o r n e r s t o n e o f c l a s s i c a l s t a t i s t i c s , a n d
t h e r e is n o t h i n g in s t a t i s t i c s a s " o b v i o u s " a s t h e W C P ( o r E x a m p l e 2 ) .
3 . 4 G E N E R A L I Z A T I O N S B E Y O N D T H E D I S C R E T E C A S E
B a s u ( 1 9 7 5 ) a n d o t h e r s h a v e a r g u e d t h a t t h e s a m p l e s p a c e x in a n y
p h y s i c a l l y r e a l i z a b l e e x p e r i m e n t m u s t b e f i n i t e , d u e t o o u r i n a b i l i t y t o
m e a s u r e w i t h i n f i n i t e p r e c i s i o n . T h i s s u g g e s t s t h a t t h e L i k e l i h o o d P r i n c i p l e
f o r d i s c r e t e e x p e r i m e n t s ( a s i n S e c t i o n 3 . 3 ) is a l l t h a t o n e n e e d s . W e a r e
THE LIKELIHOOD PRINC IPLE AND GENERALIZATIONS 29
philosophically i n agreement wit h t hi s.
On the ot her hand, continuous and ot her more general pr obabi l i t y
di st r i but i ons are enormously usef ul in simplif ying s t a t i s t i c a l computations
and in providing numerical approximations which are of t en qui t e accurat e. I t
is possible f or the l i ke l i hood f unct ion f or a continuous model to d i f f e r
st r i ki ngl y from t hat of t he discr et e model i t is intended t o approximate, so
it i s not obvious t hat t he va l i di t y of t he LP in discr e t e problems extends t o
i t s va l i di t y in t he approximating continuous problems. In any case, ext ension
of the LP t o more general si t uat i ons can only st rengt hen i t s case. Such an
ext ension is our t ask in t he present sect ion.
As i n Sect ion 3 . 3 , an experiment E = ( X, , { P J ) w i l l be understood
t o involve t he observat ion of t he random var iable X, having pr oba bi l i t y d i s t r i
but ion P
on X
9
. ( I t wi l l not be necessary t o assume t he exist ence of
a densit y. ) There i s , unavoidably, measure t heoret ic mathematics i n t hi s
se ct i on, but the sect ion can be skipped, i f desir ed, wit hout any e sse nt ial
loss of cont i nui t y.
The sample space % wi l l be assumed t o be a locally compact
Hausdorff space whose topology admits a countable base (LCCB space, f or shor t ) ,
and the P
wi l l be assumed t o be Borel measures. Of course, X of t en arises as
a n % v a l u e d r a ndom v a r i a b l e on a p r o b a b i l i t y s p a c e ( , 3 , {
} ) e q u i p p e d w i t h
u
a f a m i l y o f p r o b a b i l i t y m e a s u r e s i n d e x e d b y 6 . S u c h u n d e r l y i n g s t r u c t u r e
w i l l n o t b e r e l e v a n t i n o u r a n a l y s i s , h o w e v e r .
3 . 4 . 1 D i f f i c u l t i e s in t h e N o n d i s c r e t e C a s e
I n a n e x p e r i m e n t E = ( X , , { P
Q
} ) f o r w h i c h t h e r e is a n x X
s a t i s f y i n g P ( {x }) = 0 f o r e v e r y , i t is d i f f i c u l t t o a s s i g n a n y p a r t i c u
l a r m e a n i n g t o
n
E v ( E , x ) " . F o r e x a m p l e , B a s u ( 1 9 7 5 ) a n d Jo s h i ( 1 9 7 6 ) h a v e
o b s e r v e d t h a t a n a i v e a p p l i c a t i o n o f B i r n b a u m ' s ( 1 9 6 2 a ) s u f f i c i e n c y p r i n c i p l e
w o u l d s u g g e s t f o r s u c h a n x t h a t E v ( E , x ) = E v ( E , y ) f o r e v e r y y X > s i n c e t h e
m a p T : x - % w h i c h t a k e s x o n t o y a n d l e a v e s all o t h e r p o i n t s ( i n c l u d i n g y )
f i x e d is s u f f i c i e n t f o r . T h i s is p a r t i c u l a r l y d i s t u r b i n g f o r c o n t i n o u s
30 THE LIKELIHOOD PRINCIPLE
d i s t r i b u t i on s , si nce then P
Q
( { x} ) =0 f or every x e % and every e ; Birnbaum' s
s u f f i c i en c y p r i n c i p l e then suggests t hat al l p ossi b l e ob servat i ons l end
p r ec i sel y t he same evi dence (and t her ef or e none) about .
The unique s p e c i f i c at i on of a l i k e l i h ood f unc t i on causes s i m i l ar
probl ems. I f t here i s no si ng l e f i n i t e measure v on z whose n u l l sets
c oi nc i de wi t h those Borel sets N f or which P.(N) = 0 f or al l 6 , then no
l i k e l i h ood f unc t i on e xi s t s . Thi s i s t he usual st at e of af f ai r s i n nonpara
met r i c problems ( r e c al l t hat coul d be an ar b i t r ar y index set ) and can even
ar i se i n si mpl e p aramet ri c examples; f or example, P
Q
(A) = j / / \ dx
+
j U(
) >
= X = [ 0 , 1 ] , descri b es an experi ment i n which X= wi t h p r ob ab i l i t y j and
i s ot herwi se uni f orml y d i s t r i b u t e d over the u n i t i n t e r v al ; no f i n i t e measure
v dominates {P } , and no l i k e l i h ood f unc t i on e xi s t s . ( I n c i d e n t al l y , t h i s seems
t o be a source of conf usi on i n c er t ai n " counterexampl es" t o t he LP such as the
second example i n Sec t i on 2.5 of Birnbaum ( 1 969) . )
Even i n problems where t here i s a measure v wi t h t he i nd i c at ed
p r op er t i es , t he Radon Nikodym d er i vat i ves
U ) = f
f l
( x) = P
fl
(dx)/ v(dx)
are determined only up to sets of v measure zero; these functions of could be
specified i n an enti rel y arb i trary manner for al l x i n any set N c z with
v(N) = 0. One way to salvage a l i kel i hood pri nci pl e in the face of such
ambiguity is to specify a parti cul ar version of P (dx)/ v(dx) for each ; for
example, i n case a (v almost everywhere) continuous density exists we could

set
= {open neighborhoods of x %} and put

U ) = i nf sup (P_(U)/ v(U))
ve

U

UcV
for x i n the support of v, i ( ) = 0 otherwise.
By rest ri c t i ng our attenti on to (v almost everywhere) continuous
densi ti es, continuous suf f i c i ent st at i st i c s, etc. we could develop versions of
the condi t i onal i t y, suffi ci ency, and l i kel i hood principl es very si mi l ar to
those i n the discrete set t i ng .
T H E L I K E L I H O O D P R I N C I P L E A N D G E N E R A L I Z A T I O N S 31
I n s t e a d w e w i l l d e v e l o p v e r s i o n s o f t h e s e p r i n c i p l e s a p p l i c a b l e
f o r a l l e x p e r i m e n t s , i n c l u d i n g t h o s e w i t h d i s c o n t i n u o u s d e n s i t y f u n c t i o n s a n d
e v e n t h o s e f o r w h i c h n o l i k e l i h o o d f u n c t i o n e x i s t s . T h e p r i c e w e p a y f o r s u c h
g e n e r a l i t y i s t h a t o u r c o n c l u s i o n s w i l l a l l b e w e a k e n e d b y t h e q u a l i f i c a t i o n
" f o r a l l x X o u t s i d e a f i x e d s e t N w i t h P
( N ) = 0 f o r a l l 6 " , w h i c h w e s h a l l
ab b revi ate " f or { P> a.e. x" . I t i s i mportant t o note t hat N wi l l be unknown
to the s t at i s t i c i an , and hence the onl y assurance t hat the actual ob servati on
x i s not i n N i s the f ai t h t hat events of p r ob ab i l i t y zero do not happen. This
i s , of course, a statement i n the c l assi c al f r eq uent i st framework, but estab
l i shi ng a versi on of the LP wi t hi n t hi s framework shoul d, at l east , be con
vi nc i ng to f r eq uent i st s.
3. 4. 2. Evidence, Con d i t i on al l y , and Suf f i c i enc y
As b ef ore, denote by Ev(E, x) the ( undefi ned) evi d ent i al content of
an ob servati on x i n an experiment E = ( X, , { P
Q
} ) . The f ol l owi ng are the
ap p rop ri ate g eneral i zat i ons of the WCP and suf f i c i enc y p r i nc i p l e f or non
di sc ret e experiments.
WEAK CONDITIONALITY PRINCIPLE. Consider the mixture, E*, of two experiments
E
l
= ( X
> { P
} ) a n d E
2
= ( X
2
9 { P
} ) >
d e f i n e d a s E * = ( X* , , {P * } ) ,
where X* = ( J , Xj ) , J = 1 or 2 {as E, is performed) with probability y each
{independent of ) , and
P*(A) = \ P J ( {
X I
: ( l , . , ) A}) + \ P^({x
2
: ( 2, x
2
) A} ) .
Then,
Ev(E*
>
(j , x
j
) ) = Ev(E
j f
x..) for {?*} - a.e. ( j . Xj ) .
I f the sample spaces i n E, and E
2
are countable, we could del ete
"i mpossi bl e" outcomes ( i . e . , x. f or which P x
) = 0 f or al l ) and d i s
pense wi t h the " { P* } a. e. " q u al i f i c at i on above, thus recoveri ng the di sc ret e
W C P .
A f o r m a l d e f i n i t i o n o f s u f f i c i e n c y is a s f o l l o w s . L e t
E = ( X , , { P
Q
} ) b e an e x p e r i m e n t a n d T : z > J a m e a s u r a b l e m a p f r o m X t o
32 THE LIKELIHOOD PRINC IPLE
anot her LCCB sp a c e r . The s t a t i s t i c T determines a f ami l y { P } of Borel
measures on j by
P I ( A ) = P
( A ) ) ,
a n d h e n c e a n e x p e r i m e n t E = ( T , j , { P
} ) U n l e s s T is 11 w e e x p e c t ( i n
g e n e r a l ) t h a t E w i l l t e l l u s l e s s a b o u t t h a n E , s i n c e d i f f e r e n t o u t c o m e s
x X w i t h p o s s i b l y d i f f e r e n t e v i d e n t i a l i m p o r t c a n b e m a p p e d o n t o t h e s a m e
T ( x ) e x . T h e e x c e p t i o n a l c a s e is t h a t in w h i c h T is s u f f i c i e n t .
D E F I N I T I O N . For the exp eri m en t E , s u p p os e there exi s ts a f am i l y {g . : t J"}
of B orel p rob ab i l i ty m eas u res on X s ati s f y i n g
P
( A ) = / g
t
( A ) p J( d t ) = / g
( )
( A ) P
( d x )
for all Borel sets A c t Then T is called "sufficient
11
(or sometimes
11
sufficient for ").
Note t ha t g. i s not pe r mi t t e d t o depend upon ; ot herwise g. = P
Q
would always work. Any one t o one measurable mapping T i s s uf f i c i e nt ; j ust
l e t g
t
be a poi nt mass a t T ( t ) X.
The Suffi ci ency Pri nci pl e makes precise the notion that T(x) in j
te l l s as much about as x i n E;
SUFFICIENCY PRINCIPLE (SP). If T: X + J is sufficient, then
Ev(E, x) = Ev( E
T
, T( x) ) for {P
} a . e. x X.
Again we may del ete the impossible outcomes when x i s countable to
remove the "{P
Q
} - a.e." qual i f i cati on and conclude that Ev(E,x) = Ev(E,y)
w h e n e v e r a s u f f i c i e n t s t a t i s t i c T s a t i s f i e s T ( x ) = T ( y ) , a n d s o r e c o v e r t h e
d i s c r e t e W S P o f S e c t i o n 3 . 3 . 1 .
3 . 4 . 3 . T h e R e l a t i v e L i k e l i h o o d P r i n c i p l e
L e t E ] = (X.|, , { P ' } ) a n d E p = ( X
2
, , { P }) b e t w o e x p e r i m e n t s
a n d s u p p o s e ( f o r m o t i v a t i o n a l p u r p o s e s ) t h a t e a c h a d m i t s a l i k e l i h o o d f u n c t i o n ,
THE LIKELIHOOD PRINC IPLE AND GENERALIZATIONS 3 3
i . e . a f i n i t e measure v. on t he sample space Z and a f a mi l y { f l ( ) } of
1 I
i n t e g r a b l e f u n c t i o n s s a t i s f y i n g
P Q ( A ) = / f j ( x ) v . ( d x ) , A c z . .
T h e L i k e l i h o o d P r i n c i p l e ( w e r e it t o h o l d h e r e ) w o u l d a s s e r t t h a t
E v ( E
1
, x
1
) = E v ( E
2
, x
2
)
w h e n e v e r f
n
( x
) = c f
Q
( x
o
) f o r all a n d s o m e c o n s t a n t c = c ( x , , x
0
) n o t
1 2
d e p e n d i n g o n , i . e . w h e n e v e r t h e r e l a t i v e l i k e l i h o o d c = f
Q
( x
) / f
Q
( x
o
) d o e s
I D C
not depend on . Our freedom to specif y ^ Q( X ^ ) a r b i t r a r i l y whenever
v ^ x ^ } ) = 0 makes i t clear t hat t his pr i nci pl e needs ref ormulat ion before i t is
suit able f or experiments wit h uncountable sample spaces. ( However, at points
x, and x which are atoms of v, and v> r espect ively, t he LP is reasonable, and
can be shown to f ollow from the WCP and SP as in Section 3 . 3 . )
To develop a suit able general pr i nci pl e , we generalize the concept
t hat the r e l a t i ve like lihood of x j and x
2
is independent of . Basi cal l y, i f
a mapping exist s between two subsets of ^ and X^ f or which the Radon Nikodym
de r ivat ive of the induced measure wit h respect to the e xist ing measure ( on, say,
Z, ) is independent of , then we can est ablish an equivalence of evidence
between the corresponding observations in the subsets. The reasons f or
generalizing the LP in t his dir e ct ion are: ( i ) I t can be st at ed in great
ge ne r al i t y, without requiring models or de nsit ie s; ( i i ) I t wi l l be shown to
f ollow from the WCP and SP, as did the LP; and ( i i i ) I t , in t ur n, can be shown
t o imply ( i n subst ant ial ge ne r alit y) the Stopping Rule Pr inciple and C ensoring
Pr i nci pl e , besides having di r e ct l y important implicat ions of i t s own. The
major l i mi t a t i on of the RLP (compared to the LP) is t hat i t does not provide
any such convenient summarization of evidence as the like lihood f unct ion ( which
need not e xi st in the general case) .
RELATIVE LIKELIHOOD PRINCIPLE (RLP). Let : Uj + l ^ be a Borel bimeasurable
one to one mapping from U, c %.. onto IL <= ^
9
and suppose there exists a
strictly positive function c on U, such that for all %>
( 3 . 4 . 1 ) PQ( A) = / / c ^ P ^ d x . , ) , A c U
2
.
"
( A)
T h e n E v ( E
r
x
1
) = E v ( E
2
, x ^ ) f o r { ? ] } a . e . x
]
U
]
.
N o t e t h a t t h e R L P d o e s n ot s a y a n y t h i n g f o r p a r t i c u l a r x ^ . I n d e e d ,
i f x , h a s z e r o p r o b a b i l i t y f o r a l l , t h e n c o u l d b e d e f i n e d a r b i t r a r i l y at
x , a n d s t i l l s a t i s f y ( 3 . 4 . 1 ) . T h u s t h e R L P can o n l y b e i n t e r p r e t e d in a
p r e e x p e r i m e n t a l s e n s e : i f s a t i s f i e s ( 3 . 4 . 1 ) , e v i d e n t i a r y e q u i v a l e n c e h o l d s
w i t h p r o b a b i l i t y o n e o n U.. W h e r e o r U . c o m e f r o m is i r r e l e v a n t . T h e
f o l l o w i n g t h e o r e m s h o w s t h a t t h e R L P is i n d e e d a g e n e r a l i z a t i o n o f t h e L P .
T H E O R E M 2 . F o r t w o e x p e r i m e n t s E
]
= ( X ^ , { P ^} ) a n d E
2
= ( X
2
, , { P ^} ) w i t h
c o u n t a b l e s a m p l e s p a c e s d e v o i d o f o u t c o m e s i m p o s s i b l e u n d e r a l l , t h e L P a n d
t h e R L P a r e e q u i v a l e n t .
P r o o f . W i t h o u t l o s s o f g e n e r a l i t y , w e t a k e t h e d o m i n a t i n g m e a s u r e s v , a n d v
2
t o b e c o u n t i n g m e a s u r e o n z , a n d Z o > r e s p e c t i v e l y , s o t h e l i k e l i h o o d f u n c t i o n s
a r e f \ [ x . ) = P ^ x . } ) . F i r s t , a s s u m e t h e v a l i d i t y o f t h e L P , a n d l e t
1 1
P ? ( A ) = / [ l / c ( x ) ]P . j ( d x )
" ' ( A )
f or some : U, > U
2
and a l l A c \ ] Fix any x. U-i and set x
2
= ( x i ) ,
A = { x
2
} . Then f g ( x
2
) = t l / c t x ^ x , ) f or a l l , SO t he LP assert s t ha t
Ev( E
1
, x
1
) = Ev( E
2
, x ^ ) .
1 2
C o n v e r s e l y , a s s u m e t h e R L P h o l d s , a n d s u p p o s e t h a t f ' ( x
) = c f ^( x
0
)
I L
f or some x j Z-j , x
2
Z
2 >
c > 0 , and a l l . Put U^ = { x j } , U
2
= { x
2
} ,
and de f i ne : U
1
> U
2
by ( x j) = x
2
( Note t ha t we are f r e e t o choose U j, U
2 >
and i n any f ashion compat ible wi t h t he condit ions i n t he RLP, but e vi de nt i a r y
equivalence need not hold on any nul l se t . ) Regard c as t he const ant value of
a s t r i c t l y posi t i ve f unct i on on U, . Then t he RLP assert s t ha t
E v t E ^ x ^ = Ev( E
2
, t x ^ ) f or { P^} a . e . x
]
U
]
,
i . e . that Ev(E, , x^) = Ev(E
2
tXo) (by hypothesis X, contains no point at which
f ( x , ) v a n i s h e s f o r all , s o t h e " { P
} a . e . " q u a l i f i c a t i o n is
U I
u n n e c e s s a r y ) . ||
T H E O R E M 3. T h e W C P a n d t h e S P t o g e t h e r i m p l y t h e R L P .
P r o o f . L e t E , a n d E p b e t w o e x p e r i m e n t s , a b i m e a s u r a b l e m a p p i n g f r o m a B o r e l
s e t U^ e %
1
o n t o U ^ ^ , a n d c: U^ > (0,) a m e a s u r a b l e f u n c t i o n s a t i s f y i n g
P
<
A
) = / . ] C V c ( x ) ] p J( d x )
(A)
f or a l l Borel Ac I L, a l l . Let E* be the mixture of E, and E
2
, and
def ine a mapping T: X* + X* by
) i f i = 1 and x
]
T ( i , x
. ) =
]
e l s e .
T h i s d e t e r m i n e s a n e w e x p e r i m e n t E *
T
= ( T , X*, { P ! } ) , w h e r e P I ( A ) =

F i r s t w e s h o w t h a t T is s u f f i c i e n t . F o r e a c h t = ( i , x ) 6 X*
d e f i n e a m e a s u r e g. o n z * b y
( A . ) = e
t
( A ) i f i = 1 o r x .
g
t
( A ) =
( c
( A , ) +
( A
2
) ) / ( l + c ) i f 1 = 2 , , a n d x., = "
]
( x
2
) .
H e r e c = c ( x ) a n d e , , . d e n o t e t h e u n i t p o i n t m a s s e s a t xj Z - , x
2
Z o >
t X * r e s p e c t i v e l y ; A . d e n o t e s {x . X ^ : ( l ^ x ^ ) A } . It is s t r a i g h t f o r w a r d
t o v e r i f y t h a t
P * ( A ) = / g
t
( A ) p J ( d t )
f o r e a c h B o r e l A c ^ * , s o T is s u f f i c i e n t .
B y t h e S P w e c a n c o n c l u d e t h a t
E v ( E * , ( l ,
X l
) ) = E v ( E *
T
, ( 2 , ( . , ) ) ) a n d
Ev( E* , ( 2 , x
2
) ) = Ev( E*
T
, ( 2 , x
2
) )
^ 6 X
and { P^} a. e. x
2
X
r
In particular, for
for
- a . e .
36 HE LIKELIHOOD PRINCIPLE
( { P* } a. e. ) x
G U
and X
2
= ^ x ^ we have
Ev ( E* , ( l , x
1
) ) = Ev( E*
T
, ( 2, x
2
) ) = Ev ( E* , ( 2, x
2
) ) .
By t he WCP we have
Ev UM l . Xj ) ) = Ev E^Xj ) and Ev( E* , ( 2, x
2
) ) = Ev ( E
2 >
x
2
) ,
so we can concl ude t h at
EvEj.Xj) = Ev(E
2
,
))
for { ?h a.e. x , EU
| |
T h e R L P w i l l b e u s e d in C h a p t e r 4 t o e s t a b l i s h g e n e r a l v e r s i o n s o f
i m p o r t a n t c o n s e q u e n c e s o f t h e L P . T h e o r e m 3 d e m o n s t r a t e s t h a t r e j e c t i o n o f
t h e s e c o n s e q u e n c e s ( a n d s e v e r a l a r e q u i t e u n p a l a t a b l e f r o m t h e f r e q u e n t i s t
v i e w p o i n t ) i m p l i e s r e j e c t i o n o f t h e W C P o r t h e S P .
3.5 P R E D I C T I O N , D E S I G N , N U I S A N C E P A R A M E T E R S , A N D T H E L P
3 . 5 . 1 I n t r o d u c t i o n
T h e L P a s s t a t e d a b o v e h a s t h e w e r y i m p o r t a n t q u a l i f i c a t i o n t h a t it
d o e s n o t a p p l y i f d o e s n o t i n c l u d e all u n k n o w n q u a n t i t i e s g e r m a n e t o t h e e x
p e r i m e n t o r p r o b l e m . F o r i n s t a n c e , in d e s i g n o r p r e d i c t i o n p r o b l e m s t h e u n
k n o w n f u t u r e o b s e r v a t i o n i s o b v i o u s l y r e l e v a n t , a n d y e t i s n o t n e c e s s a r i l y a
p a r t o f t h e p a r a m e t e r d e f i n i n g t h e d i s t r i b u t i o n o f t h e o b s e r v a b l e X . A
r e l a t e d d i f f i c u l t y is t h a t , o f t e n , o n l y a p a r t o f i s r e a l l y o f i n t e r e s t , t h e
r e m a i n d e r b e i n g a " n u i s a n c e " p a r a m e t e r . T h e s e i s s u e s a r e e x p l o r e d in t h i s
s e c t i o n .
W e b e g i n by e x p a n d i n g t h e d e f i n i t i o n o f t o i n c l u d e u n o b s e r v e d a n d
n u i s a n c e v a r i a b l e s . D e f i n e
= (z ) = ( y , w ; , ) ,
w h e r e z = ( y , w ) i s t h e v a l u e o f a n u n o b s e r v e d v a r i a b l e Z , w i t h y b e i n g o f i n
t e r e s t a n d w b e i n g a n u i s a n c e v a r i a b l e , a n d w h e r e = ( , n ) is t h e p a r a m e t e r
t h a t d e t e r m i n e s t h e d i s t r i b u t i o n s o f b o t h X a n d Z , w i t h b e i n g o f i n t e r e s t a n d
b e i n g a n u i s a n c e p a r a m e t e r . ( W e w i l l p u r p o s e f u l l y r e m a i n v a g u e o n t h e d e f i
n i t i o n o f " n u i s a n c e v a r i a b l e " a n d " n u i s a n c e p a r a m e t e r " ; f o r m a l d e f i n i t i o n s c o u l d
b e a t t e m p t e d a l o n g d e c i s i o n t h e o r e t i c l i n e s , b u t w o u l d t a k e u s t o o f a r a f i e l d . )
T o i n d i c a t e t h a t e v i d e n c e a b o u t a n d y is d e s i r e d f r o m E w e w i l l w r i t e
E
V
?
(E
'
X)
f o r t h e e v i d e n c e a b o u t a n d y f r o m t h e o b s e r v a t i o n o f x in a n e x p e r i m e n t E .
T w o d i f f i c u l t i e s a r i s e in a t t e m p t i n g t o a p p l y t h e L P in t h i s m o r e
g e n e r a l c o n t e x t . T h e f i r s t is t h a t t h i s g e n e r a l i z e d i s n o l o n g e r j u s t t h e
p a r a m e t e r d e f i n i n g t h e d i s t r i b u t i o n o f X . T h u s t h e d e f i n i t i o n in ( 3 . 1 . 1 ) o f
i ( ) a s t h e d e n s i t y o f X g i v e n m a y n o l o n g e r b e a s u i t a b l e d e f i n i t i o n . I n
d e e d , i f Z i s c o n d i t i o n a l l y i n d e p e n d e n t o f X g i v e n , t h e n (by t h e d e f i n i t i o n
o f c o n d i t i o n a l i n d e p e n d e n c e ) it c a n b e s h o w n t h a t ( 3 . 1 . 1 ) b e c o m e s
I ( )

f ( x ) = f ( x ) ,
x z ,01
w h i c h d o e s n o t e v e n i n v o l v e z . T h e s e c o n d d i f f i c u l t y i s t h a t t h e n u i s a n c e
p a r a m e t e r , , w i l l a p p e a r in t h i s l i k e l i h o o d f u n c t i o n e v e n t h o u g h it is n o t
o f i n t e r e s t .
T o r e s o l v e t h e s e d i f f i c u l t i e s a n d i n d i c a t e t h e r o l e o f t h e L P , w e
w i l l d i s c u s s a l t e r n a t i v e d e f i n i t i o n s o f t h e l i k e l i h o o d f u n c t i o n w h i c h b r i n g o u t
t h e r o l e o f i m p o r t a n t u n o b s e r v e d v a r i a b l e s a n d s u p p r e s s t h e r o l e o f n u i s a n c e
p a r a m e t e r s , a n d w e w i l l i n d i c a t e u n d e r w h a t c i r c u m s t a n c e s t h e s e f o r m s o f t h e
l i k e l i h o o d f u n c t i o n m a y b e s u b s t i t u t e d f o r t h e s i m p l e ( 3 . 1 . 1 ) .
3 . 5 . 2 U n o b s e r v e d V a r i a b l e s : P r e d i c t i o n a n d D e s i g n
T h e f o l l o w i n g e x a m p l e s h o w s t h a t a n a i v e a p p l i c a t i o n o f t h e L P c a n
b e m i s l e a d i n g i f f u t u r e o b s e r v a t i o n s a r e o f i n t e r e s t .
E X A M P L E 1 1 . W e h a v e a v a i l a b l e a s e q u e n c e o f o b s e r v a t i o n s X . = ( U . , V . )
(i = 1 , 2 , . . . ) w h e r e
P ( v
i + 1
= i | v . = l ) = 1/ 2 , p ( v .
+ 1
= o | v . = l ) = 1/ 2
P ( v
i + 1
= i | v . = o ) = o , P ( V .
+ 1
= o | v . = o ) = l .
( D e f i n e V
Q
= 1 ) . W h e n V . = 1 , U .
+ 1
w i l l b e i n d e p e n d e n t o f t h e p r e v i o u s U .
w i t h a 7 ( , l ) d i s t r i b u t i o n . W h e n V . = 0 , o n t h e o t h e r h a n d , U .
+
, w i l l b e z e r o .
( T h i s w o u l d c o r r e s p o n d t o a s i t u a t i o n in w h i c h a m e a s u r i n g i n s t r u m e n t is u s e d
t o o b t a i n t h e i m p o r t a n t o b s e r v a t i o n U . , w h i l e V . t e l l s w h e t h e r t h e e q u i p m e n t
w i l l w o r k t h e n e x t t i m e ( V . = 1 ) o r h a s i r r e p a r a b l y b r o k e n (V = 0 ) ) .
I m a g i n e t h a t x . , . . . ,x^ h a v e b e e n o b s e r v e d , a n d t h a t v . = 1 f o r
i = l , . . . , n l . T h e l i k e l i h o o d f u n c t i o n f o r is t h e n g i v e n b y
n ,
K U ) = f
( u . ) a 7?(u , n ~ ) d e n s i t y .
T h e L P t h u s s a y s t h a t t h e e v i d e n c e a b o u t i s c o n t a i n e d in I ( ) , a n d i f w e a r e
s t o p p i n g t h e e x p e r i m e n t n o t h i n g e l s e is n e e d e d . H o w e v e r , in d e c i d i n g w h e t h e r
o r n o t t o t a k e a n o t h e r o b s e r v a t i o n , it i s o b v i o u s t h a t k n o w l e d g e o f v i s
c r u c i a l . I f v = 1 it m a y b e d e s i r a b l e t o t a k e a n o t h e r o b s e r v a t i o n , b u t if
v = 0 it w o u l d b e a w a s t e o f t i m e ( s i n c e t h e m e a s u r i n g i n s t r u m e n t i s b r o k e n ) .
T h i s e x a m p l e i s r e l a t e d t o a l i m i t a t i o n o f s u f f i c i e n c y ( c f . B a h a d u r ( 1 9 5 4 ) ) .
T h e a p p a r e n t f a i l u r e o f t h e L P i n ' E x a m p l e 11 i s r e a l l y t h e f a i l u r e
t o i n c l u d e a l l u n k n o w n s in t h e s p e c i f i c a t i o n o f ; o n l y i s i n c l u d e d . F o r
t h i s p r o b l e m t h e n e x t o b s e r v a t i o n , X ( a n d p e r h a p s f u r t h e r o b s e r v a t i o n s ) , a r e
a l s o i m p o r t a n t u n k n o w n s . A n d t h e l i k e l i h o o d f u n c t i o n f o r t h i s f u t u r e o b s e r
v a t i o n a n d d o e s d e p e n d o n v . E x a m p l e s s u c h a s t h i s h a v e o f t e n b e e n t o u t e d
a s c o u n t e r e x a m p l e s t o t h e L P . T h e r e a r e a t l e a s t t w o p o s s i b l e r e p l i e s .
T h e f i r s t p o s s i b l e r e s p o n s e i s t o s i m p l y e x c l u d e p r o b l e m s i n v o l v i n g
s u c h u n o b s e r v e d Z f r o m c o n s i d e r a t i o n . T h i s w a s e s s e n t i a l l y t h e t a c k w e t o o k
e a r l i e r in t h e m o n o g r a p h , m o t i v a t e d b y a d e s i r e f o r s i m p l i c i t y o f e x p o s i t i o n .
T h i s r e s p o n s e i s c l e a r l y n o t v e r y s a t i s f y i n g .
A s e c o n d p o s s i b l e r e s p o n s e i s t o r e d e f i n e t h e l i k e l i h o o d f u n c t i o n
s o a s t o i n c o r p o r a t e Z . I n t h e f i r s t e d i t i o n o f t h i s m o n o q r a p h it w a s e s s e n
t i a l l y s u g g e s t e d t h a t o n e d e f i n e t h e l i k e l i h o o d f u n c t i o n f o r = ( z , ) =
( y , w ; , ) t o b e
( 3 . 5 . 1 )
( ) = f
( j )
( x , y , w ) ;
t h i s i s , o f c o u r s e , j u s t t h e j o i n t d e n s i t y o f ( X , Z ) g i v e n t h e p a r a m e t e r
= (
5
n ) , b u t h e r e it is t o b e c o n s i d e r e d a f u n c t i o n o f t h e u n k n o w n = ( z , )
w h e n t h e o b s e r v e d v a l u e X = x is i n s e r t e d . S u c h r e d e f i n i t i o n o f
( ) i n d e e d
w o r k s , in t h e s e n s e t h a t t h e L P w i l l s t i l l t h e n a p p l y a n d b e d e r i v a b l e f r o m
a p p r o p r i a t e v e r s i o n s o f t h e C o n d i t i o n a l i t y P r i n c i p l e a n d S u f f i c i e n c y P r i n c i p l e .
W e h a v e n o t c a r e f u l l y i n v e s t i g a t e d t h i s , h o w e v e r . (I t s h o u l d b e e m p h a s i z e d
t h a t ( 3 . 5 . 1 ) is n o t t h e d e n s i t y o f X , g i v e n , s o t h a t t h i s l i k e l i h o o d f u n c t i o n
is q u i t e d i f f e r e n t f r o m ( 3 . 1 . 1 ) . F o r B a y e s i a n s , t h e d i s t i n c t i o n i s w h e t h e r t o
i n c l u d e t h e u n o b s e r v e d v a r i a b l e Z a s p a r t o f t h e m o d e l p a r a m e t e r o r a s p a r t o f
t h e o b s e r v a t i o n ; w e w i l l a r g u e in t h e n e x t s e c t i o n t h a t it m a k e s n o d i f f e r e n c e . )
W h i l e ( 3 . 5 . 1 ) c a n b e u s e d t o e s t a b l i s h t h e L P in t h i s m o r e g e n e r a l
c o n t e x t , it h a s c e r t a i n p r a c t i c a l l i m i t a t i o n s a s a d e f i n i t i o n o f l i k e l i h o o d .
T h e m o s t s e r i o u s l i m i t a t i o n i s t h a t i t m u s t b e u t i l i z e d v e r y c a u t i o u s l y .
C o m m o n t e c h n i q u e s s u c h a s m a x i r r r t m l i k e l i h o o d c a n o f t e n b e d i s a s t r o u s i f
a p p l i e d d i r e c t l y t o t h i s a ( ) . F o r e x a m p l e s , s e e B a y a r r i , D e G r o o t a n d K a d a n e
( 1 9 8 7 ) ; h e n c e f o r t h , B D K .
A r e l a t e d o b j e c t i o n t o ( 3 . 5 . 1 ) i s t h a t i t s d e f i n i t i o n i s , in a
s e n s e , q u i t e a r b i t r a r y . E x t e n s i v e d i s c u s s i o n o f t h i s p o i n t c a n a l s o b e f o u n d
in B D K , w i t h m a n y e x a m p l e s . I t is a p o i n t w i t h w h i c h w e e s s e n t i a l l y a g r e e b u t ,
f o l l o w i n g B e r l i n e r ( 1 9 8 7 ) , v i e w a s t a n g e n t i a l t o t h e L P . T h e L P l e a p s i n t o
a c t i o n a f t e r X , Z , , a n d f
( x > z ) h a v e b e e n d e f i n e d , a n d X = x o b s e r v e d . T h e
p r o c e s s o f g e t t i n g t o t h i s p o i n t i s i n h e r e n t l y v a g u e a n d r a t h e r a r b i t r a r y ;
b u t t h a t d o e s n ' t a l t e r t h e f a c t t h a t , h a v i n g r e a c h e d t h i s p o i n t a n d a s s u m i n g
t h a t t h e m o d e l is c o r r e c t , all i n f o r m a t i o n a b o u t = ( z , ) i s c o n t a i n e d in
( 3 . 5 . 1 ) f o r t h e g i v e n d a t a .
W h i l e ( 3 . 5 . 1 ) is t h u s f o r m a l l y s a t i s f a c t o r y f o r u s e in t h e L P , t h e
p r a c t i c a l d i f f i c u l t i e s s u r r o u n d i n g i t s u s e a n d d e f i n i t i o n s u g g e s t l o o k i n g f o r
a n a l t e r n a t i v e " l i k e l i h o o d f u n c t i o n . " A w ery a p p e a l i n g p o s s i b i l i t y is p r e s e n t -
ed in B u t l e r ( 1 9 8 7 ) , d i s c u s s i o n o f w h i c h w e d e f e r t o t h e n e x t s e c t i o n . A m o n g
t h e m a n y o t h e r r e f e r e n c e s d i s c u s s i n g l i k e l i h o o d f o r u n o b s e r v e d v a r i a b l e s
( t y p i c a l l y in p r e d i c t i o n ) a r e G e i s s e r ( 1 9 7 1 ) , K a l b f l e i s c h ( 1 9 7 1 ) , L a u r i t z e n
( 1 9 7 4 ) , A i t c h i s o n a n d D u n s m o r e ( 1 9 7 5 ) , H i n k l e y ( 1 9 7 9 ) , a n d B u t l e r ( 1 9 8 6 ) .
D e s i g n p r o b l e m s d e s e r v e s p e c i a l e m p h a s i s . B e f o r e t h e e x p e r i m e n t i s
c o n d u c t e d , X i t s e l f is t h e u n o b s e r v e d v a r i a b l e , a n d s h o u l d h e n c e b e i d e n t i f i e d
w i t h Z in t h e a b o v e f o r m u l a t i o n . ( In s e q u e n t i a l o r m u l t i s t a g e e x p e r i m e n t s , a t
e a c h s t e p o r s t a g e t h e p r e v i o u s l y t a k e n o b s e r v a t i o n s a r e x , w h i l e t h e f u t u r e
o b s e r v a t i o n s a r e Z . ) T h e L P d o e s n o t f o r b i d a v e r a g i n g o v e r u n ob s erv ed v a r i -
a b l e s , a n d so d o e s n o t f o r m a l l y c o n t r a i n d i c a t e u s e o f m a n y c l a s s i c a l d e s i g n
c r i t e r i a . F o r i n s t a n c e , t h e L P d o e s n ot s a y t h a t it is w r o n g t o c h o o s e t h e
s a m p l e s i z e in a t e s t i n g p r o b l e m b y c o n s i d e r a t i o n o f t y p e I a n d t y p e II e r r o r
p r o b a b i l i t i e s . ( Of c o u r s e , a f t e r t h e d a t a h a v e b e e n t a k e n , t h e L P w o u l d
a r g u e a g a i n s t u s e o f t h e s e p r e - e x p e r i m e n t a l e r r o r p r o b a b i l i t i e s a s m e a s u r e s o f
e v i d e n c e f o r o r a g a i n s t t h e h y p o t h e s e s . )
W h i l e n o t d i s a l l o w i n g t h e u s e o f c l a s s i c a l d e s i g n c r i t e r i a , t h e
LP c a n h a v e a s u b s t a n t i a l p r a c t i c a l e f f e c t o n d e s i g n ; a p r o p o n e n t o f t h e LP
( i . e . a c on d i ti on al i s t) w o u l d w a n t t o d e s i g n a n e x p e r i m e n t s o a s t o h a v e a
h i g h p r o b a b i l i t y o f o b t a i n i n g a c c u r a t e c o n d i t i o n a l ( p o s t - e x p e r i m e n t a l ) c o n c l u -
s i o n s , r a t h e r t h a n m e r e p r e - e x p e r i m e n t a l f r e q u e n t i s t a s s u r a n c e s o f a c c u r a c y .
T h e d i f f e r e n c e in v i e w p o i n t c a n b e s i g n i f i c a n t in t h a t t h e c o n d i t i o n a l i s t c a n
b e m o r e f l e x i b l e in h i s a p p r o a c h t o d e s i g n , o f t e n s i m p l y s a m p l i n g d a t a u n t i l
e n o u g h ( c o n d i t i o n a l ) e v i d e n c e h a s b e e n a c c u m u l a t e d . By t h e S t o p p i n g R u l e
P r i n c i p l e ( d i s c u s s e d in S e c t i o n 4 . 2 a n d s h o w n t o b e a c o n s e q u e n c e o f t h e L P ) it
i s q u i t e v a l i d f o r t h e c o n d i t i o n a l i s t t o e m p l o y s u c h s t o p p i n g r u l e s o f c o n v e n -
i e n c e . A f r e q u e n t i s t a n a l y s i s , o n t h e o t h e r h a n d , r e q u i r e s t h a t t h e p r o b a b i l i -
t i e s o f s t o p p i n g f o r e a c h p o s s i b l e r e a s o n b e k n o w n a t t h e o u t s e t , a n d t h a t all
t h e s e s t o p p i n g p r o b a b i l i t i e s b e i n c o r p o r a t e d in t h e a n a l y s i s .
S i m i l a r l y t h e L P g i v e s l i t t l e g u i d a n c e in a s s e s s i n g t h e o v e r a l l
p e r f o r m a n c e o f a d e c i s i o n p r o c e d u r e . S u c h a n a s s e s s m e n t m i g h t b e d e s i r e d in
q u a l i t y c o n t r o l a n d o t h e r s i t u a t i o n s w h e r e a p a r t i c u l a r p r o c e d u r e w i l l b e u s e d
r e p e a t e d l y . T h u s s u p p o s e o n e f a c e s a s e q u e n c e o f p r o b l e m s X . ^ P , o n e a c h o f
i .
w h i c h a c e r t a i n p r o c e d u r e w i l l b e u s e d . E v a l u a t i o n o f t h e p r o c e d u r e 6 w i l l
t y p i c a l l y i n v o l v e s o m e t y p e o f a v e r a g e o v e r t h e s a m p l e s p a c e b e c a u s e f u t u r e
o b s e r v a t i o n s X . a r e u n k n o w n ; a s w i t h d e s i g n p r o b l e m s , h o w e v e r , t h i s in n o w a y
c o n t r a d i c t s t h e L P . ( T h e L P d o e s , o f c o u r s e , s a y t h a t i t i s w r o n g t o r e p o r t
s u c h p r o c e d u r e p e r f o r m a n c e a s s e s s m e n t s a s t h e e v i d e n c e a b o u t a p a r t i c u l a r .
u p o n o b s e r v i n g a p a r t i c u l a r . ) S e e S e c t i o n 4 . 1 f o r f u r t h e r d i s c u s s i o n .
3 . 5 . 3 N u i s a n c e V a r i a b l e s a n d P a r a m e t e r s
W h e n = ( , ) w i t h a n u i s a n c e v a r i a b l e , t h e L P s a y s t h a t a l l
e v i d e n c e a b o u t is c o n t a i n e d i n t h e l i k e l i h o o d f u n c t i o n i ( ) ; it s e e m s
x
r e a s o n a b l e t o i n t e r p r e t t h i s b r o a d l y e n o u g h t o i n f e r t h a t i ( ) s h o u l d a l s o
x
c o n t a i n all e v i d e n c e a b o u t t h e p a r t o f . T h i s c a n b e m a d e f o r m a l t h r o u q h t h e
N U I S A N C E V A R I A B L E L I K E L I H O O D P R I N C I P L E . S i n c e e v i d e n c e a b o u t d e p e n d s o n E
a n d X o n l y t h r o u g h I ( )
3
E v ( E , x ) a l s o d e p e n d s o n E a n d X o n l y t h r o u g h
x
I ( ) . M o r e g e n e r a l l y w h e n = ( y , w ; , ) j w h e r e y a n d a r e t h e i m p o r t a n t
x
u n o b s e r v e d v a r i a b l e s a n d u n k n o w n p a r a m e t e r s w h i l e w a n d a r e n u i s a n c e v a r i
a b l e s a n d p a r a m e t e r s , E v ( E , x ) d e p e n d s o n E a n d X o n l y t h r o u g h I ( )
y JS x
a s d e f i n e d i n ( 3 . 5 . 1 ) .
W i t h t h i s a m e n d m e n t , t h e L P s a y s t h a t E v ( E , x ) ( o r m o r e g e n e r a l l y
E v
r
( E , x ) ) i n v o l v e s E a n d x o n l y t h r o u g h ( ) = a ( , n ) ( o r m o r e g e n e r a l l y
y j x x
A ( ) = l ( y , w ; , ) ) , b u t d o e s n o t s a y w h a t t o d o a b o u t ( o r ( w , ) ) ; t h e L P
X X
d o e s n o t s a y h o w t o i n t e r p r e t
( ) s o a s t o i s o l a t e t h e e v i d e n c e a b o u t y a n d
x
. W h i l e t h i s f o r m a l l y f a l l s in t h e d o m a i n o f " u t i l i z a t i o n o f t h e l i k e l i h o o d
f u n c t i o n , " a t o p i c t h a t w e a r e a v o i d i n g , a b r i e f d i s c u s s i o n o f c e r t a i n m e t h o d s
o f d e a l i n g w i t h s u c h n u i s a n c e q u a n t i t i e s i s d e s i r a b l e .
T h e f i r s t k e y o b s e r v a t i o n is a f o r m a l i z a t i o n o f t h e s u g g e s t i o n in
B u t l e r ( 1 9 8 7 ) f o r d e a l i n g w i t h n u i s a n c e v a r i a b l e s o r p a r a m e t e r s t h a t h a v e k n o w n
d i s t r i b u t i o n s :
M A R GI N A L I ZA T I O N P R I N C I P L E : I f t h e d i s t r i b u t i o n o f a n u n o b s e r v e d n u i s a n c e v a r i
a b l e o r p a r a m e t e r i s g i v e n , f o r m a m a r g i n a l l i k e l i h o o d f u n c t i o n f r o m t h e j o i n t
d e n s i t y o f X a n d t h e n u i s a n c e v a r i a b l e o r p a r a m e t e r b y s i m p l y i n t e g r a t i n g
o u t t h e n u i s a n c e v a r i a b l e o r p a r a m e t e r i n t h i s j o i n t d e n s i t y .
T h e f i r s t s t e p in t h i s m a r g i n a l i z a t i o n p r o c e s s c a n a l w a y s b e d o n e ;
w c a n b e i m m e d i a t e l y e l i m i n a t e d ( i f p r e s e n t ) b e c a u s e i ( ) = f# \ ( x , y
5
w )
x l jTi /
s p e c i f i e s i t s d i s t r i b u t i o n . T h u s I ( ) c a n b e r e d u c e d t o
x
, , ) = f f /
( , y , w ) d w .
A f u r t h e r m a r g i n a l i z a t i o n s t e p c a n b e t a k e n w h e n t h e d i s t r i b u t i o n
O p
o f ( o r p a r t o f ) is g i v e n . T h u s i f = ( , ) , a n d it is g i v e n t h a t h a s
2 1
d e n s i t y ( | , ) , t h e l i k e l i h o o d f u n c t i o n c a n b e f u r t h e r m a r g i n a l i z e d t o
( 3 . 5 . 2 ) j y . . n
1
) = /
f
(
) (
> y >
w
) ^ l e . n
1
) d w d
2
.
E X A M P L E 1 1 . 1 . C o n s i d e r t h e r a n d o m e f f e c t s p r o b l e m w h e r e
=
i
r\ 0 0
t h e . . b e i n g i . i . d . ^( , ) a n d t h e . b e i n g i . i . d . 7 ? ( , ) ; h e r e , y , a n d
I J I
2
a r e u n k n o w n . S u p p o s e t h a t i n t e r e s t c e n t e r s o n t h e " h y p e r p a r a m e t e r s "
= ( y , ) . T h e n t h e p a r a m e t e r s = a n d = ( . , p , . . . , , ) a r e n u i s a n c e
2 2
p a r a m e t e r s , a n d t h e d i s t r i b u t i o n o f is g i v e n . I n d e e d ( | ) is
2 t
77j(yl., I ) , w h e r e ^ = ( 1 , . . . , 1 ) a n d ^ i s t h e i d e n t i t y m a t r i x . A s t a n d a r d c a l
c u l a t i o n ( c f . B e r g e r ( 1 9 8 5 ) ) t h e n y i e l d s f o r ( 3 . 5 . 2 ) ( n o t e t h a t ( y , w ) is n o t
p r e s e n t h e r e )
U , n
I
? ?
2 ? ?
e x p { I ( x . v ) / [ 2 ( + ^ ) ] } e x p { s * 7 ( 2 ) }
J
2 2
w h e r e x . = I ../ J a n d s = ( X . . x . ) .
" 1 "i "
T h e s u g g e s t i o n t o u s e ( 3 . 5 . 2 ) a s t h e l i k e l i h o o d w a s m a d e in B u t l e r
( 1 9 8 7 ) t o a n s w e r t h e c r i t i c i s m s in B a y a r r i , D e G r o o t , a n d K a d a n e ( 1 9 8 7 ) c o n c e r n
ing t h e a r b i t r a r i n e s s a n d d i f f i c u l t y in u s e o f t h e l i k e l i h o o d d e f i n e d in
( 3 . 5 . 1 ) ; u s e o f ( 3 . 5 . 2 ) s e e m s t o b e q u i t e s u c c e s s f u l in t h i s r e g a r d . W e
s u p p o r t u s i n g ( 3 . 5 . 2 ) a s t h e " p r a c t i c a l " d e f i n i t i o n o f l i k e l i h o o d , n o t i n g t h a t
it is f u l l y c o n s i s t e n t w i t h o u r p r e f e r r e d ( s e e C h a p t e r 4 ) B a y e s i a n a p p r o a c h t o
u t i l i z a t i o n o f l ( ) . M o s t n o n B a y e s i a n s w o u l d a l s o p r o b a b l y a p p r o v e o f
( 3 . 5 . 2 ) a s t h e d e f i n i t i o n o f l i k e l i h o o d ; f a i l u r e t o d o s o l e a v e s o n e o p e n t o
t h e s e r i o u s c r i t i c i s m s in B a y a r r i , D e G r o o t , a n d K a d a n e ( 1 9 8 7 ) . I t is a l s o
p r o b a b l y t r u e t h a t a v e r s i o n o f t h e L P b a s e d o n ( 3 . 5 . 2 ) c o u l d b e s h o w n t o
f o l l o w ( w i t h c e r t a i n q u a l i f i c a t i o n s c f . t h e c o m m e n t s a t t h e e n d o f t h e
s e c t i o n ) f r o m v e r s i o n s o f t h e C o n d i t i o n a l i t y P r i n c i p l e a n d S u f f i c i e n c y P r i n c i
p l e . W e h a v e n o t l o o k e d i n t o t h e m a t t e r , h o w e v e r .
U s e o f ( 3 . 5 . 2 ) d o e s n o t c o m p l e t e l y s o l v e t h e n u i s a n c e p a r a m e t e r
p r o b l e m , o f c o u r s e , b e c a u s e * ( y , , n ) s t i l l d e p e n d s o n t h e n u i s a n c e p a r a m e t e r
. T h e r e i s , u n f o r t u n a t e l y , n o " c o n s e n s u s " a p p r o a c h t o e l i m i n a t i o n o f . I n
t h e r e m a i n d e r o f t h e s e c t i o n , a b r i e f i n t r o d u c t i o n t o s o m e o f t h e p r o p o s e d
m e t h o d s f o r e l i m i n a t i o n o f w i l l b e g i v e n .
T h e B a y e s i a n a p p r o a c h t o t h e p r o b l e m is c o n c e p t u a l l y s t r a i g h t f o r
w a r d . O n e s i m p l y d e t e r m i n e s ( | ) , t h e c o n d i t i o n a l p r i o r d e n s i t y o f g i v e n
, a n d c a l c u l a t e s t h e r e d u c e d l i k e l i h o o d f u n c t i o n
D 1 1 1
( oc oN v f ) = f * ( v f n l k ) dn
T h e p r o d u c t o f t h i s a n d t h e m a r g i n a l p r i o r d e n s i t y , ( ) , w i l l b e p r o p o r t i o n a l
D
t o t h e p o s t e r i o r d i s t r i b u t i o n o f ( y , ) g i v e n x , s o t h a t i ( y , ) c l e a r l y s u f f i c e s
f o r t h e B a y e s i a n . A s t r o n g c a s e c a n b e m a d e t h a t e v e n t h e n o n B a y e s i a n c o n d i
t i o n a l i s t s h o u l d o p e r a t e b y u s i n g ( 3 . 5 . 3 ) , w i t h ( | ) c h o s e n t o b e s o m e
" n o n i n f o r m a t i v e " p r i o r d e n s i t y f o r g i v e n . P r e s e n t a t i o n o f t h i s c a s e w o u l d ,
u n f o r t u n a t e l y , t a k e u s t o o f a r a f i e l d .
T h e m o s t c o m m o n n o n B a y e s i a n a p p r o a c h t o e l i m i n a t i o n o f n i s
t h r o u g h m a x i m i z a t i o n : i . e . , c o n s i d e r a t i o n o f
A
y
( y ,
= s p
n
1
T h e d a n g e r s in u s e o f
h a v e b e e n w e l l d o c u m e n t e d a n d h a v e r e s u l t e d in a
s e a r c h f o r a l t e r n a t i v e m e t h o d s ( s e e S e c t i o n 5.2 f o r r e f e r e n c e s ) .
A l t e r n a t i v e n o n B a y e s i a n m e t h o d s t y p i c a l l y a p p r o a c h t h e p r o b l e m o f
e l i m i n a t i n g t h r o u g h i d e a s o f p a r t i a l o r c o n d i t i o n a l l i k e l i h o o d . T h e i d e a o f
p a r t i a l l i k e l i h o o d ( c f . K a l b f l e i s c h ( 1 9 7 4 ) , S p r o t t ( 1 9 7 5 ) , C o x ( 1 9 7 5 ) , D a w i d
( 1 9 7 5 , 1 9 8 0 ) , B a r n d o r f f N i e l s e n ( 1 9 7 8 , 1 9 8 0 ) , H i n k l e y ( 1 9 8 0 ) , a n d K a y ( 1 9 8 5 ) )
i s t o f a c t o r t h e l i k e l i h o o d a s ( i g n o r i n g , f o r s i m p l i c i t y , f u t u r e o b s e r v a t i o n s
Z = ( y , w ) a n d t h e p o s s i b i l i t y t h a t p a r t o f h a s a k n o w n d i s t r i b u t i o n )
( 3 . 5 . 4 ) I ( ) = ) , ) ,

1 2
a n d t h e n to w o r k w i t h i ( ) e x c l u s i v e l y . T h i s is s u c c e s s f u l w h e n a d o e s n o t
X X
c o n t a i n m u c h i n f o r m a t i o n a b o u t , o r w h e n t h e i n f o r m a t i o n is v e r y h a r d t o
e x t r a c t b e c a u s e o f h i g h v a r i a t i o n d u e to . I t is p a r t i c u l a r l y a t t r a c t i v e in
t h e s p e c i a l c a s e (to w h i c h w e r e t u r n in C h a p t e r 4 ) in w h i c h c o n t a i n s n o
i n f o r m a t i o n a b o u t , i . e . in w h i c h
( 3 . 5 . 5 ) l ( ) = ^( )
2
U ) .

T h i s a r i s e s w h e n an a n c i l l a r y s t a t i s t i c T e x i s t s f o r , a n c i l l a r y in t h e s t r o n g
s e n s e t h a t
f
( x ) = 9
( x | T ) h
( T ) ;
( 3 . 5 . 5 ) is t h e n i m m e d i a t e . ( O t h e r , b r o a d e r , d e f i n i t i o n s o f a n c i l l a r i t y a l s o
a p p e a r in t h e l i t e r a t u r e , b u t l e a d to e x p r e s s i o n s a s in ( 3 . 5 . 4 ) r a t h e r t h a n
( 3 . 5 . 5 ) . A l s o , a t t e m p t s h a v e b e e n m a d e t o f i n d a p p r o x i m a t e d e c o m p o s i t i o n s o f
t h e f o r m ( 3 . 5 . 5 ) ; c f . H i n d e a n d A i t k i n ( 1 9 8 6 ) . )
E X A M P L E 1 2 . S u p p o s e E c o n s i s t s o f o b s e r v i n g
X = ( ( Y
1
, Z
1
) , . . . , ( Y
n
, Z
n
) ) ,
w h e r e t h e ( Y
1
> Z
i
) a r e i . i . d . p a i r s h a v i n g a c o m m o n b i v a r i a t e n o r m a l d i s t r i b u
t i o n w i t h u n k n o w n m e a n (
,
z
) a n d c o v a r i a n c e m a t r i x
l l

1 2 \
\
1 2

2 2 / .
p
O f i n t e r e s t i s t h e r e g r e s s i o n o f Y o n Z ; t h u s i n t e r e s t c e n t e r s o n = ( , 3 , ) ,
w h e r e
=
3
7
, =
i A
, =
n
( l
l
) ,
Y L V ? 11 99
2
s i n c e E ( Y . | Z . ) = + 3 Z . a n d is t h e c o n d i t i o n a l v a r i a n c e o f Y . g i v e n Z . .
L e t t i n g = (
, ^, , , , .
2
,
2 2
) , n = (
2 2
, O , a n d
T = ( Z
1 9
. . . , Z ) , a s t a n d a r d c a l c u l a t i o n g i v e s
1 ?
i = l
U C m C I X
g
( | ) h
( ) .
T h u s ( 3 . 5 . 5 ) is s a t i s f i e d ( a n d , i n d e e d , T is a n c i l l a r y f o r ) .
I t s e e m s n a t u r a l , w h e n ( 3 . 5 . 5 ) h o l d s , t o s t a t e t h a t all e v i d e n c e
a b o u t a v a i l a b l e f r o m E a n d x is s u m m a r i z e d in
v
( ) . T h u s , in E x a m p l e 1 2 ,
x
it s e e m s n a t u r a l t o b a s e t h e r e g r e s s i o n a n a l y s i s o n g
( x | T ) , t h e c o n d i t i o n a l
d i s t r i b u t i o n o f t h e Y . g i v e n t h e o b s e r v e d z . . T h i s i s , i n d e e d , v i r t u a l l y
a l w a y s d o n e in r e g r e s s i o n ; t h e z
a r e t r e a t e d a s n o n r a n d o m , i . e . , a r e
c o n d i t i o n e d u p o n .
B a s i n g t h e a n a l y s i s o n l y u p o n Jr ( ) is n o t a l w a y s j u s t i f i e d . I f
k n o w l e d g e o f w o u l d c o m m u n i c a t e i n f o r m a t i o n a b o u t , t h e n ( ) c a n n o t ,
t h e o r e t i c a l l y , b e i g n o r e d . ( F o r p r a c t i c a l r e a s o n s , h o w e v e r , o n e m i q h t
f r e q u e n t l y i g n o r e s u c h i n f o r m a t i o n s e e S e c t i o n 4 . 5 . 4 ) T h e m o s t n a t u r a l
w a y to r i g o r o u s l y s t a t e t h i s is in t e r m s o f B a y e s i a n a n a l y s i s : if a n d
2
a r e a p r i o r i i n d e p e n d e n t , t h e n i ( ) c o n t a i n s n o i n f o r m a t i o n a b o u t . T h i s is
x
c l e a r , s i n c e t h e n ( 3 . 5 . 3 ) b e c o m e s ( i q n o r i n g y )
l \ U ) = / * J( ) l \ M
2
( ) d ex * J( ) .
T h e s t a n d a r d c o n d i t i o n i n g o n t h e z . in E x a m p l e 1 2 is t h u s r i g o r o u s l y
j u s t i f i a b l e o n l y w h e n y ^ a n d a
2 2
a r e f e l t to b e a p r i o r i i n d e p e n d e n t o f , ,
2
a n d , a r e a s o n a b l e a s s u m p t i o n in m a n y s i t u a t i o n s .
A l t h o u g h B a y e s i a n r e a s o n i n g p r o v i d e s t h e i n t u i t i v e b a s i s f o r
s t a t i n g t h a t a n u i s a n c e p a r a m e t e r c a r r i e s no i n f o r m a t i o n a b o u t , w e w i l l
s i d e s t e p t h e i s s u e a n d s i m p l y g i v e an o p e r a t i o n a l d e f i n i t i o n c o m p a t i b l e w i t h
t h e L P .
D E F I N I T I O N . S u p p o s e E i s s u c h t h a t ( 3 . 5 . 5 ) i s s a t i s f i e d . L e t E
b e t h e
"t h o u g h t " e x p e r i m e n t i n w h i c h > i n a d d i t i o n t o X., i s o b s e r v e d . T h e n i s a
n o n i n f o r m a t i v e n u i s a n c e p a r a m e t e r i f E v (E , ( x , n ) ) i s i n d e p e n d e n t o f .
N O N I N F O R M A T I V E N U I S A N C E P A R A M E T E R P R I N C I P L E ( N N P P ) . I f E i s a s i n ( 3 . 5 . 5 ) a n d
i s a n o n i n f o r m a t i v e n u i s a n c e p a r a m e t e r ^ t h e n
E V
( E , x ) = E v
( E \ ( x , ) ) .
T h e N N P P s t a t e s t h e " o b v i o u s , " t h a t i f o n e w e r e t o r e a c h t h e
i d e n t i c a l c o n c l u s i o n f o r e v e r y , w e r e k n o w n , t h e n t h a t s a m e c o n c l u s i o n
s h o u l d b e r e a c h e d e v e n i f is u n k n o w n . T h i s p r i n c i p l e w i l l b e u s e d in t h e
d i s c u s s i o n o f r a n d o m s t o p p i n g r u l e s a n d r a n d o m c e n s o r i n g in C h a p t e r 4 .
A s a f i n a l q u a l i f i c a t i o n , it s h o u l d b e n o t e d t h a t e a c h o f t h e s e
m e t h o d s f o r s u p p r e s s i n g t h e r o l e o f n u i s a n c e p a r a m e t e r s i s o n l y a p p l i c a b l e w h e n
a d e c i s i o n o r a c t i o n i s t o b e t a k e n o n t h e b a s i s o f e v i d e n c e a l r e a d y r e c o r d e d ,
a n d n o f u r t h e r t a k i n g o f e v i d e n c e is c o n t e m p l a t e d . F o r e x a m p l e , t h e L i k e l i h o o d
P r i n c i p l e d o e s n o t i m p l y t h a t t h e B a y e s i a n ' s r e d u c e d l i k e l i h o o d f u n c t i o n ,
D
v
( y > ) > s u m m a r i z e s all e v i d e n c e f r o m a n e x p e r i m e n t E a b o u t a p a r a m e t e r o f
i n t e r e s t a n d a n u n o b s e r v e d v a r i a b l e o f i n t e r e s t y , i f t h a t e v i d e n c e m u s t
l a t e r b e c o m b i n e d w i t h o t h e r e v i d e n c e f r o m f u r t h e r t r i a l s a l s o g o v e r n e d b y t h e
s a m e n u i s a n c e p a r a m e t e r . F u t u r e o b s e r v a t i o n s m a y o f f e r n e w e v i d e n c e a b o u t
t h e j o i n t d i s t r i b u t i o n o f a n d ; b y i n t e g r a t i n g a w a y ( o r b y m a x i m i z i n g a w a y )
t h e n u i s a n c e p a r a m e t e r w e w o u l d l o s e t h e c h a n c e t o u s e t h a t n e w e v i d e n c e t o
t r a n s f o r m p r e s e n t e v i d e n c e a b o u t i n t o e v i d e n c e a b o u t . T h u s , in E x a m p l e 1 1 . 1 ,
it w o u l d n o t s u f f i c e t o c a r r y a l o n g o n l y z * ( y , , ) i f a d d i t i o n a l r e p l i c a t i o n s
x . . ( f o r i = 1 1 ) w e r e t o b e o b t a i n e d at a l a t e r t i m e . E v e n i f f u t u r e
p
o b s e r v a t i o n s w i l l n o t b e t a k e n , a B a y e s i a n c o u l d n o t r e p o r t ( ) a s a c o m p l e t e
x
s u m m a r y o f t h e e v i d e n c e t o a n o t h e r B a y e s i a n w h o m i g h t u s e a d i f f e r e n t c o n d i t i o n
al p r i o r ( | ) ; d e s p i t e t h e n u i s a n c e , t h e e n t i r e l i k e l i h o o d f u n c t i o n
* ( , ) m u s t b e r e p o r t e d i n o r d e r t o c o n v e y all i n f o r m a t i o n .
3.6 C R I T I C I S M S O F B I R N B A U M ' S A X I O M A T I C D E V E L O P M E N T
B i r n b a u m ' s a x i o m a t i c d e v e l o p m e n t o f t h e L P h a s b e e n s u b j e c t e d t o
c o n s i d e r a b l e s c r u t i n y . E r r o r s in B i r n b a u m ' s a r g u m e n t s d i d e x i s t , a s w a s m e n
t i o n e d in S e c t i o n 3 . 4 . 1 ( s e e a l s o B i r n b a u m ( 1 9 7 2 ) , B a s u ( 1 9 7 5 ) , Jo s h i ( 1 9 7 6 ) ,
a n d G o d a m b e ( 1 9 7 9 ) ) , b u t t h e s e e r r o r s w e r e c o r r e c t a b l e a n d d i d n o t a f f e c t t h e
b a s i c t r u t h o f t h e a r g u m e n t s . A l s o e a s i l y h a n d l e d a r e c e r t a i n c r i t i c i s m s o f
t h e L P a r i s i n g f r o m i t s m i s a p p l i c a t i o n o r m i s i n t e r p r e t a t i o n . S e v e r a l s u c h
m i s a p p l i c a t i o n s a n d m i s i n t e r p r e t a t i o n s h a v e a l r e a d y b e e n m e n t i o n e d ; f o r
c o m p l e t e n e s s w e r e s t a t e t h e m h e r e .
( i ) T h e L P a p p l i e s o n l y w h e n i n c l u d e s all u n k n o w n s r e l e v a n t t o
t h e p r o b l e m . F o r d e s i g n , p r e d i c t i o n , s e q u e n t i a l a n a l y s i s , m e t a a n a l y s i s , a n d
i n m a n y s c e n a r i o s , t h e i m p o r t a n t u n k n o w n s o f t e n i n c l u d e m o r e t h a n j u s t ,
t h e u n k n o w n p a r a m e t e r o f t h e p r o b a b i l i t y m o d e l . B u t t h e L P c a n b e r e f o r m u
l a t e d t o i n c l u d e s u c h u n k n o w n s ; s e e S e c t i o n 3 . 5 .
( i i ) S o m e t i m e s a f r e q u e n t i s t m e a s u r e o f t h e p e r f o r m a n c e o f a p r o
c e d u r e s u c h a s a s a m p l i n g i n s p e c t i o n p l a n o r a d i a g n o s t i c t e s t is s p e c i f i e d ,
b y c o n t r a c t o r l a w , t o b e o f p r i m a r y i n t e r e s t . T h e n , o f c o u r s e , t h e L P ( w h e n
s t a t e d f o r a l o n e ) d o e s n o t a p p l y .
( i i i ) T h e r e c a n b e a m b i g u i t i e s in t h e d e f i n i t i o n o f t h e l i k e l i h o o d
f u n c t i o n . T h e p r o b l e m c a n u s u a l l y b e r e s o l v e d , h o w e v e r , b y t h e a p p r o a c h e s
d i s c u s s e d in S e c t i o n s 3 . 4 a n d 3 . 5 .
( i v ) T h e r e c a n b e s i t u a t i o n s in w h i c h t h e c h o i c e o f e x p e r i m e n t
c o n v e y s i n f o r m a t i o n a b o u t . F o r i n s t a n c e , o n e m i g h t j u d g e t h a t t h e e x p e r i m e n
t e r n e v e r w o u l d h a v e c h o s e n t h e g i v e n e x p e r i m e n t u n l e s s h e s u s p e c t e d t h a t , s a y ,
w a s s m a l l . T h e L P w i l l s t i l l t h e n a p p l y , in t h e s e n s e t h a t t h e e x p e r i m e n t a l
e v i d e n c e i s s t i l l c o n t a i n e d in ( ) ; it is j u s t t h a t o n e w i l l t h e n h a v e a d d i
t i o n a l e v i d e n c e p r o v i d e d b y t h e c h o i c e o f e x p e r i m e n t . ( I n a s e n s e , t h e c h o i c e
o f e x p e r i m e n t s h o u l d b e t r e a t e d a s a d d i t i o n a l d a t a . )
( v ) T h e r e a r e p e r i o d i c a l l y a t t e m p t s t o p r o v e t h e L P w r o n g b y a r
g u i n g , in a g i v e n e x a m p l e , t h a t a p a r t i c u l a r l i k e l i h o o d b a s e d m e t h o d ( e . g . ,
m a x i m u m l i k e l i h o o d e s t i m a t i o n ) g i v e s a b a d r e s u l t . B u t t h e L P p r e s c r i b e s n o
p a r t i c u l a r m e t h o d f o r u t i l i z a t i o n o f ( ) . T h i s i s s u e i s e x t e n s i v e l y d i s c u s s e d
in C h a p t e r 5 .
( v i ) T h e L P d o e s n o t a p p l y t o t h e i n f o r m a t i o n c o n v e y e d a b o u t d i f
f e r e n t p a r a m e t e r s f r o m d i f f e r e n t e x p e r i m e n t s . I t m a y b e t e m p t i n g t o s a y t h a t ,
i f E ^ is b i n o m i a l ( n , , ) a n d E p is b i n o m i a l ( n , p ) a n d 1 0 s u c c e s s e s ( o r o n e s )
a r e o b s e r v e d in e a c h o f t h e e x p e r i m e n t s , t h e n s i n c e t h e l i k e l i h o o d f u n c t i o n s
f o r t h e t w o s i t u a t i o n s a r e t h e s a m e ( a s f u n c t i o n s ) , o n e s h o u l d r e a c h t h e s a m e
c o n c l u s i o n s a b o u t j a n d ^ B u t t h e L P d o e s n o t s a y t h i s ; it a p p l i e s o n l y
w h e n j a n d
2
a r e t h e s a m e p a r a m e t e r , i . e . , a r e p h y s i c a l l y o r c o n c e p t u a l l y
t h e s a m e q u a n t i t y .
T h e r e h a v e b e e n a n u m b e r o f c r i t i c i s m s d i r e c t e d a t t h e e x p l i c i t a n d
i m p l i c i t p r i n c i p l e s u s e d i n B i r n b a u m ' s d e v e l o p m e n t o f t h e L P . W e a d d r e s s t h e s e
c r i t i c i s m s in t h i s a n d t h e f o l l o w i n g s e c t i o n s .
3 . 6 . 1 T h e M o d e l A s s u m p t i o n
T h e m o s t f r e q u e n t l y e x p r e s s e d c r i t i c i s m o f t h e L P is t h a t i t is
s u p p o s e d l y y e r y d e p e n d e n t o n a s s u m i n g a p a r t i c u l a r p a r a m e t r i c m o d e l w i t h a
d e n s i t y f o r X ; s i n c e m o d e l s a r e a l m o s t n e v e r k n o w n e x a c t l y , i t is f e l t t h a t
t h e L P is o n l y r a r e l y a p p l i c a b l e . I t i s , o f c o u r s e , e a s y t o c r i t i c i z e a l m o s t
a n y s t a t i s t i c a l t h e o r y f o r b e i n g m o d e l d e p e n d e n t , b u t l e t us e x a m i n e t h e
i s s u e s e r i o u s l y a n y w a y .
T h e f i r s t p o i n t t o n o t e i s t h a t , e v e n i f t h e r e a r e v a r i o u s
p o s s i b l e m o d e l s u n d e r c o n s i d e r a t i o n , t h e L P s t i l l s a y s t h a t t h e i n f o r m a t i o n
i n t h e d a t a , f o r a n y p o s s i b l e m o d e l , is c o n t a i n e d i n t h e l i k e l i h o o d f u n c t i o n
f o r t h a t m o d e l . T h e e v i d e n c e c o n v e y e d b y t h e d a t a c e r t a i n l y c h a n g e s a s
d i f f e r e n t m o d e l s a r e c o n s i d e r e d , b u t t h e l i k e l i h o o d f u n c t i o n s s h o u l d s t i l l b e
c o n s i d e r e d t h e v e h i c l e s o f t h i s e v i d e n c e .
T o b e m o r e f o r m a l a b o u t t h i s , w e n e e d o n l y r e c a l l t h a t n e e d n o t
b e r e s t r i c t e d t o b e i n g a t y p i c a l p a r a m e t e r , a n d i n d e e d c a n r e p r e s e n t v a r i o u s
m o d e l s . T h e s i t u a t i o n o f d i s c r e t e X is e a s i e s t t o s e e : t h u s , i f
X = ( X i x
2
}
w e
c o u l d s i m p l y l e t = ( ^ g . . ) d e n o t e a p o i n t o n t h e
i n f i n i t e d i m e n s i o n a l s i m p l e x
: 0 <_ . <_ 1 and IB. = 1 } ,
and de f i ne
T h e n {P } is t h e c l a s s o f a l l p r o b a b i l i t y d i s t r i b u t i o n s o n
9
a n d t h e L P
a ppl i e s t o t h i s compl e t e l y nonpar ame t r i c se t up, as we l l as t o any s i t u a t i o n

w h e r e a r e s t r i c t e d c l a s s o f m o d e l s ( c o r r e s p o n d i n g t o s o m e s u b s e t o f ) is
c o n s i d e r e d . O f c o u r s e , w e w i l l u s u a l l y o n l y b e i n t e r e s t e d in s o m e f u n c t i o n
( ) , b u t i f a l l t h e e v i d e n c e a b o u t is c o n t a i n e d i n t h e l i k e l i h o o d f u n c t i o n ,
t h e n t h e s a m e s h o u l d b e t r u e o f ( ) . T h e a r g u m e n t i n S e c t i o n 3 . 4 in f a v o r o f
c o n s i d e r i n g o n l y d i s c r e t e s i t u a t i o n s ( i n f o u n d a t i o n s ) t h u s i n d i c a t e s t h a t t h e
L P a l w a y s a p p l i e s .
E v e n in c o n t i n u o u s s i t u a t i o n s , t h e r e is n o n e e d t o t i e t h e L P
t o r e s t r i c t i v e p a r a m e t r i c m o d e l s . F o r i n s t a n c e , c o n s i d e r t h e f o l l o w i n g e x a m p l e .
E X A M P L E 1 3 . S u p p o s e X
l f
. . . , X a r e i . i . d . o b s e r v a t i o n s f r o m s o m e d i s t r i b u t i o n ,
k n o w n t o h a v e a d e n s i t y ( w i t h r e s p e c t t o a g i v e n m e a s u r e v ) , b u t o t h e r w i s e
u n k n o w n . L e t b e t h e s e t o f a l l s u c h d e n s i t i e s , s o t h a t t h e d e n s i t y o f
X = ( X
. . . , X
n
) is
n
f ( x ) = ( x . ) .
i = l

For instance, thi s would be the si tuati on i f the x. were known to have a
di st ri b ut i on with a continuous density with respect to Lebesgue measure on a
Euclidean space. Thus a l i kel i hood function does exi st in such nonparametric
si tuati ons, and the LP (more properly the heuri sti c LP discussed i n Section
3.4.1) would apply. "Robustness" problems t yp i cal l y f al l i nto the setti ng
where a subset of (say, al l densities close to some prescribed parametric
family of densities) i s under consideration. Again, the LP wi l l usually apply.
I t can be argued, of course, that one may be dealing with a
general non dominated family { P> or, al t ernat i vel y, that the LP does not
r e a l l y a p p l y t o t h e n o n d i s c r e t e c a s e , b u t t h e r e is s t i l l t h e R L P t o c o n t e n d
w i t h . A g a i n , c o u l d j u s t b e u s e d t o i n d e x t h e d i s t r i b u t i o n , s o t h e R L P w i l l
e s s e n t i a l l y a l w a y s b e a p p l i c a b l e , y e t i t is i n c o n s i s t e n t w i t h f r e q u e n t i s t
r e a s o n i n g a n d w i l l b e s e e n t o y i e l d s t r o n g c o n c l u s i o n s s u c h a s t h e S t o p p i n g
R u l e a n d C e n s o r i n g P r i n c i p l e s . I n c o n c l u s i o n , t h e r e f o r e , a l t h o u g h t h e L P i s
u s u a l l y s t a t e d i n t e r m s o f a p a r t i c u l a r p a r a m e t r i c m o d e l w i t h d e n s i t i e s , i t
( o r i t s g e n e r a l i z a t i o n s ) a r e e s s e n t i a l l y a l w a y s a p p l i c a b l e . ( I m p l e m e n t i n g t h e
L P c a n , o f c o u r s e , b e m u c h m o r e d i f f i c u l t i n n o n p a r a m e t r i c s i t u a t i o n s , a s w i l l
b e d i s c u s s e d i n C h a p t e r 5 . )
3 . 6 . 2 T h e E v i d e n c e A s s u m p t i o n
A l e s s c o m m o n c r i t i c i s m o f B i r n b a u m ' s d e v e l o p m e n t is t h e q u e s t i o n
i n g o f t h e e x i s t e n c e o r m e a n i n g o f E v ( E , x ) . A s n o t e d i n S e c t i o n 3 . 3 . 1 , h o w e v e r ,
t h i s c a n h a v e e s s e n t i a l l y a n y i n t e r p r e t a t i o n ( i n i t i a l l y ) a n d n e e d n o t c o n s i s t
o f a n y s i n g l e m e a s u r e , s o it is h a r d t o s e e t h e f o r c e o f t h i s o b j e c t i o n .
3 . 6 . 3 T h e W e a k C o n d i t i o n a i i t y P r i n c i p l e
A p o s s i b l e p o i n t o f c r i t i c i s m is t h e W e a k C o n d i t i o n a l i t y P r i n c i p l e .
I n d e e d , a c o m m i t t e d f r e q u e n t i s t m i g h t w e l l r e j e c t t h i s p r i n c i p a l , s a y i n g i t is
b a s e d o n t h e e r r o n e o u s b e l i e f t h a t o n e c a n o b t a i n e v i d e n c e ( i n t h e i n t u i t i v e
s e n s e ) a b o u t a p a r t i c u l a r f r o m a p a r t i c u l a r e x p e r i m e n t ( c . f . , N e y m a n ( 1 9 5 7 ,
1 9 7 7 ) ) . I n s t e a d , t h e a r g u m e n t g o e s , o n e c a n o n l y s t a t e t h e p e r f o r m a n c e o f a
p r o c e d u r e t h a t w i l l b e u s e d r e p e a t e d l y , a n d t h i s s h o u l d ( o r a t l e a s t c o u l d )
i n v o l v e a v e r a g i n g o v e r b o t h E. a n d . ^ *
n a
s e n s e , t h i s p o s i t i o n is l o g i c a l l y
v i a b l e . I t s s c i e n t i f i c d e s i r a b i l i t y is y e r y q u e s t i o n a b l e , h o w e v e r , as E x a m p l e
2 in S e c t i o n 2.1 i l l u s t r a t e s . T h i s i s s u e w i l l b e d i s c u s s e d f u r t h e r i n S e c t i o n
4 . 1 .
D u r b i n ( 1 9 7 0 ) r a i s e s t h e p o i n t t h a t i f t h e W e a k C o n d i t i o n a l i t y
P r i n c i p l e is a l l o w e d t o a p p l y o n l y t o c o n d i t i o n i n g v a r i a b l e s w h i c h d e p e n d
s o l e l y o n a m i n i m a l s u f f i c i e n t s t a t i s t i c , t h e n t h e L P d o e s n o t f o l l o w . ( T h i s
i s b e c a u s e , in t h e p r o o f o f T h e o r e m 1 , t h e c o n d i t i o n i n g s t a t i s t i c , J, is n o t
p a r t o f t h e m i n i m a l s u f f i c i e n t s t a t i s t i c w h e n t h e t w o l i k e l i h o o d f u n c t i o n s a r e
p r o p o r t i o n a l . S u f f i c i e n c y s a y s " d i s c a r d J, " a f t e r w h i c h i t is c l e a r l y
i m p o s s i b l e t o c o n d i t i o n o n J. ) N o p l a u s i b l e r e a s o n h a s b e e n a d v a n c e d f o r s o
r e s t r i c t i n g t h e W e a k C o n d i t i o n a l i t y P r i n c i p l e , h o w e v e r , a n d t h e i d e a s e e m s
u n r e a s o n a b l e a s a r e e x a m i n a t i o n o f E x a m p l e 2 s h o w s .
E X A M P L E 2 ( c o n t i n u e d ) . L e t x ~ d e n o t e t h e o u t c o m e o f t h e C a l i f o r n i a e x p e r i m e n t ,
a n d s u p p o s e t h a t t h e r e w a s s o m e p o s s i b l e o u t c o m e x ^ o f t h e N e w Y o r k e x p e r i m e n t
f o r w h i c h i ( ) w o u l d h a v e b e e n p r o p o r t i o n a l t o i ( ) . T h e n , in t h e m i x e d
X
C
X
N
e x p e r i me nt E* , t h e out comes x~ and x* . woul d be i d e n t i f i e d by a mi ni ma l s u f f i
cient s t a t i s t i c , precluding applicat ion of the r e st r i ct e d WCP. I f , however,
t here was no x
N
, then condit ioning on the C al i f or ni a experiment would be
allowed. Thus, by Durbin' s argument, whether or not onechooses t ocondit ion on
the act ually performed C al i f or ni a experiment wit h observation x~ would depend
on the exist ence, or lack t hereof , of an observation x*. , in the unperformed
New York experiment, having a like lihood f unct ion proport ional to t hat of x
c
Such dependence of condit ioning on the incident al st ruct ure of an unperformed
experiment would be rat her bi z a r r e .
Other rejoinders to Durbin' s cr i t i ci sm can be found in Birnbaum
( 1 9 7 0) and Savage ( 1 9 7 0) . Savage invokes a "cont inuit y" argument, showing
t hat f ollowing Durbin' s r e st r i ct e d WCP can involve drawing subst ant ially
di f f e r e nt conclusions when a problem is changed in an i nsi gni f i ca nt way ( such
as sl i ght l y pert urbing the like lihood f unct ion of x
N
above) .
3 . 6 . 4 . The Suf f iciency Pr inciple
Sur pr isingly, the most common and serious axiomatic cr it icisms of
the LP are those direct ed at t he Suf f iciency Pr i nci pl e . This may seem st range,
suf f iciency being such a cent ral part of classical st a t i st i c s, but issues can
be raised.
The f i r s t issue is a val i d l i mi t a t i on of the SP: i f one faces a
decision i n which the consequences ( or loss) depend on x, and not just on the
act ion taken and unknown , then the SP need not be va l i d. Such sit uat ions
are r e l a t i ve l y r ar e , however, and could be handled wit h a ref ormulat ion of the
LP to the e f f e c t t hat Ev( E, x) should depend on i ( ) and x.
x
A second issue, raised by Kalbf leisch ( 1 9 7 4 , 1 9 7 5 ) , is t hat the LP
does not f ollow from the WCP and SP i f suf f icie ncy is not allowed t o apply to
simple mixture experiments. The problems wit h such a r e st r i c t i on of suf f iciency
are t hat ( i ) I t seems a r t i f i c i a l , t here being no i nt ui t i ve reason to r e st r i c t
suf f iciency to ce r t ain types of experiments; ( i i ) I t is d i f f i c ul t and perhaps
i m p o s s i b l e t o c l e a r l y d i s t i n g u i s h b e t w e e n m i x t u r e a n d n o n m i x t u r e e x p e r i m e n t s
( c f . t h e d i s c u s s i o n in K a l b f l e i s c h ( 1 9 7 5 ) ) ; ( i i i ) M i x t u r e e x p e r i m e n t s c a n o f
t e n b e s h o w n t o b e e q u i v a l e n t t o n o n m i x t u r e e x p e r i m e n t s ( c f . B i r n b a u m ( 1 9 6 2 a ) ) ,
m a k i n g t h e d i s t i n c t i o n s e e m u n r e a s o n a b l e ; a n d ( i v ) I n a l m o s t a n y s i t u a t i o n ,
b e h a v i o r in v i o l a t i o n o f s u f f i c i e n c y c a n b e s h o w n t o be i n f e r i o r ( s e e S e c t i o n
3 . 7 ) . E v a n s , F r a s e r , a n d M o n e t t e ( 1 9 8 6 ) c o n t a i n s f u r t h e r d i s c u s s i o n .
T h e m o s t s e r i o u s c r i t i c i s m o f t h e S P c o m e s f r o m i d e a s o f B a r n a r d
( c f . B a r n a r d , Je n k i n s , a n d W i n s t e n ( 1 9 6 2 ) , B a r n a r d ( 1 9 8 0 , 1 9 8 1 ) , B a r n a r d
a n d G o d a m b e ( 1 9 8 2 ) , a n d t h e d i s c u s s i o n s in B i r n b a u m ( 1 9 6 2 a ) , B a s u ( 1 9 7 5 ) , a n d
W i l k i n s o n ( 1 9 7 7 ) ) a n d F r a s e r ( c f . F r a s e r ( 1 9 6 3 , 1 9 6 8 , 1 9 7 2 , a n d 1 9 7 9 ) ) . T h e y
q u e s t i o n t h e " s u f f i c i e n c y " o f r e p r e s e n t i n g t h e e x p e r i m e n t a l s t r u c t u r e s o l e l y
in t e r m s o f p r o b a b i l i t y d i s t r i b u t i o n s on t h e s a m p l e s p a c e i n d e x e d b y t h e
u n k n o w n ; D a w i d ( 1 9 7 7 ) c a l l e d t h i s t h e D i s t r i b u t i o n P r i n c i p l e ( D P ) . T h e
c r i t i c i s m o f t h e D P ( a n d h e n c e t h e S P ) is t h a t t h e r e m a y b e i m p o r t a n t i n f o r
m a t i o n l o s t c o n c e r n i n g t h e r e l a t i o n s h i p b e t w e e n X , , a n d t h e " r a n d o m n e s s " in
t h e p r o b l e m . (A n i m p o r t a n t o b s e r v a t i o n is t h a t , w h i l e r e l e v a n t t o t h e L P ,
t h i s c r i t i c i s m i s n o t r e l e v a n t t o c e r t a i n o f t h e m o s t c o n t r o v e r s i a l r e l a t i v e s
o f t h e L P , s u c h a s t h e S t o p p i n g R u l e P r i n c i p l e ; c f . D a w i d ( 1 9 8 6 ) . )
T h i s c r i t i c i s m t u r n s o u t t o b e q u i t e d i f f i c u l t t o a n s w e r , s t r i k i n g
at t h e c o r e o f v i r t u a l l y all a p p r o a c h e s t o s t a t i s t i c s . O n e r e s p o n s e i s t o
a t t e m p t an a x i o m a t i c d e v e l o p m e n t o f t h e L P w h i c h i n c o r p o r a t e s " s t r u c t u r a l "
i n f o r m a t i o n . S u c h a d e v e l o p m e n t c a n b e f o u n d in B e r g e r ( 1 9 8 4 a ) , b u t i s s o m e
t h i n g o f a f a i l u r e , c o n t a i n i n g a s u s p e c t a x i o m f r o m t h e a b o v e v i e w p o i n t . A l s o
in B e r g e r ( 1 9 8 4 a ) , t h e r e f o r e , t h e i s s u e is a d d r e s s e d f r o m t h e v i e w p o i n t o f
c o h e r e n c y a n d a d m i s s i b i l i t y ; it is s h o w n t h a t i n c o r p o r a t i n g " s t r u c t u r a l " i n
f o r m a t i o n in v i o l a t i o n o f s u f f i c i e n c y r e s u l t s in i n f e r i o r b e h a v i o r . T h e s e
a r g u m e n t s a r e f a m i l i a r , b u t b e c a u s e o f t h e i m p o r t a n c e o f t h e i s s u e a n d t h e
b e a r i n g t h e s e a r g u m e n t s h a v e on a n y p r o p o s e d v i o l a t i o n o f t h e L P , t h e y a r e
r e v i e w e d in S e c t i o n 3 . 7 . ( E v a n s , F r a s e r , a n d M o n e t t e ( 1 9 8 6 ) a l s o c o n t a i n s
r e l e v a n t d i s c u s s i o n . ) I n c i d e n t a l l y , t h e n e e d t o r e s o r t t o c o h e r e n c y a n d
a d m i s s i b i l i t y b e a r s o u t I . J. G o o d ' s d i s c u s s i o n o f B i r n b a u m ( 1 9 6 2 a ) , t h a t
d e r i v a t i o n o f t h e L P v i a t h e W C P a n d S P is m a i n l y a s o c i o l o g i c a l c o n t r i b u t i o n
t o s t a t i s t i c s , s i n c e B a y e s i a n c o h e r e n c y a x i o m a t i c s w o u l d g i v e t h e L P d i r e c t l y .
W h i l e a g r e e i n g , w e f e e l t h a t t h e s o c i o l o g i c a l c o n t r i b u t i o n is y e r y s u b s t a n
t i a l ; m a n y p e o p l e w i l l ( f o r w h a t e v e r r e a s o n s ) a c c e p t t h e W C P a n d S P , y e t
r e s i s t t h e L P .
I n t h e r e m a i n d e r o f t h i s s e c t i o n , w e b r i e f l y o u t l i n e t h e o b j e c t i o n
t o t h e S P t h a t is r a i s e d in t h e t h e o r i e s o f P i v o t a l I n f e r e n c e ( c f . B a r n a r d
( 1 9 8 0 , 1 9 8 2 ) a n d B a r n a r d a n d S p r o t t ( 1 9 8 3 ) ) a n d S t r u c t u r a l I n f e r e n c e ( c f .
F r a s e r ( 1 9 6 8 , 1 9 7 2 , 1 9 7 9 ) ) . T h e k e y i d e a is t h a t it m a y b e k n o w n t h a t
X = h ( , ) ,
w h e r e is a n u n k n o w n r a n d o m q u a n t i t y t a k i n g v a l u e s in a c c o r d i n g t o a k n o w n
d i s t r i b u t i o n Q , a n d h i s a k n o w n f u n c t i o n f r o m x > . ( O f t e n in
S t r u c t u r a l a n d P i v o t a l i n f e r e n c e , Q is k n o w n o n l y t o b e l o n g t o s o m e c l a s s . .
F o r s i m p l i c i t y , w e a s s u m e Q is k n o w n . ) T h i s is a c t u a l l y m o r e o r l e s s t h e
" s t r u c t u r a l " f o r m u l a t i o n o f t h e p r o b l e m . T h e f o r m u l a t i o n in P i v o t a l I n f e r e n c e
is b a s e d o n " p i v o t a l s " = g ( X , ) h a v i n g k n o w n d i s t r i b u t i o n s . T y p i c a l l y g
w i l l b e a n a p p r o p r i a t e i n v e r s e f u n c t i o n o f h , s o t h e t w o a p p r o a c h e s a r e v e r y
r e l a t e d . W e w i l l , f o r t h e m o s t p a r t , c o n s i d e r t h e s t r u c t u r a l f o r m u l a t i o n ,
a l t h o u g h c o m m e n t s a b o u t d i f f e r e n c e s f o r t h e p i v o t a l m o d e l w i l l b e m a d e . T h e
s t r u c t u r a l m o d e l is s o m e t i m e s c a l l e d a f u n c t i o n a l m o d e l ( c f . B u n k e ( 1 9 7 5 ) a n d
D a w i d a n d S t o n e ( 1 9 8 2 ) ) , b u t w e w i l l s t i c k w i t h F r a s e r ' s o r i g i n a l t e r m . T h e
f o l l o w i n g e x a m p l e , f r o m F r a s e r ( 1 9 6 8 ) ( a n d r e l a t e d t o a n e x a m p l e in M a u l d o n
( 1 9 5 5 ) ) , i l l u s t r a t e s t h e k e y i s s u e .
E X A M P L E 1 4 . S u p p o s e X = ( X ^ X ^ , = (
1 5
, ) , a n d P
Q
i s b i v a r i a t e n o r m a l
w i t h m e a n z e r o a n d c o v a r i a n c e m a t r i x
2
T h i s c o u l d a r i s e f r o m e i t h e r o f t h e f o l l o w i n g t w o t n . a c t u A a J m o d e l s :
( i ) = (
1 >
2
) is b i v a r i a t e n o r m a l , m e a n z e r o a n d i d e n t i t y c o v a r i a n c e m a t r i x ,
a n d
( 3 . 6 . 1 ) X = h ( , ) = (ji, + p ) ;
( i i ) is t h e s a m e b u t
( 3 . 6 . 2 ) X = h * ( , ) = (
l
1
+
l
2
, ^ ) ,
w h e r e
2
= + , ' = | /
2
,
anc
*
1 =
i/ o
I n
P i v o t a l I n f e r e n c e , o n e
w o u l d w r i t e ( 3 . 6 . 1 ) a n d ( 3 . 6 . 2 ) a s
( 3 . 6 . 1 ) ' = (j U^) = (X|/ |, ( X
2
X | / | ) / ) ,
( 3 . 6 . 2 )
1
= (
r
2
) = ( X ^ , ( X j ' X g / g J/
1
) .
a n d . a n d
2
w o u l d b e t h e p i v o t a l s w i t h k n o w n d i s t r i b u t i o n u p o n w h i c h t h e
i n f e r e n c e w o u l d b e b a s e d . I n p u r s u i n g t h i s e x a m p l e l a t e r w e w i l l a s s u m e t h a t
i n d e p e n d e n t o b s e r v a t i o n s X , . . . , X
n
f r o m t h e m o d e l a r e t a k e n , g i v i n g t h e
" s u f f i c i e n t " s t a t i s t i c S = \ ( X
)
t
( X
1
) , w h i c h h a s a W i s h a r t ( n , \ ) d i s t r i b u
i = l
t i o n .
I n t h e a b o v e t y p e o f s i t u a t i o n , w h i c h w e w i l l c a l l a P S ( f o r
P i v o t a l S t r u c t u r a l ) s i t u a t i o n , a n e x p e r i m e n t is s p e c i f i e d b y
E = ( X , , h ,

, Q ) . A s in E x a m p l e 1 4 , o n e c o u l d h a v e a s i n g l e p r o b a b i l i t y
m o d e l e d e x p e r i m e n t , E = ( X , , { P
} ) , a r i s i n g f r o m m o r e t h a n o n e P S e x p e r i m e n t .
I n s u c h s i t u a t i o n s t h e r e is a d e f i n i t e l o s s o f s t r u c t u r e i n r e d u c t i o n t o a
p r o b a b i l i t y m o d e l . T h e q u e s t i o n t h a t w i l l b e a d d r e s s e d i n t h e n e x t s e c t i o n is
w h e t h e r t h i s s t r u c t u r e c o n t a i n s a n y u s e f u l i n f o r m a t i o n . O f c o u r s e , t h e p o i n t
is m o o t u n l e s s P S t h e o r y a c t u a l l y r e c o m m e n d s d i f f e r i n g a c t i o n s o r c o n c l u s i o n s
f o r d i f f e r i n g P S m o d e l s w h i c h h a v e t h e s a m e p r o b a b i l i t y m o d e l . A n e x a m p l e
w h e r e t h i s is t h e c a s e f o r P i v o t a l t h e o r y c a n b e f o u n d in t h e d i s c u s s i o n b y
B a r n a r d i n B e r g e r ( 1 9 8 4 a ) . A p o s s i b l e e x a m p l e f o r S t r u c t u r a l t h e o r y is
E x a m p l e 1 4 .
E X A M P L E 1 4 ( c o n t i n u e d ) . A p a r t o f S t r u c t u r a l I n f e r e n c e i s t h e c o n s t r u c t i o n o f
" s t r u c t u r a l d i s t r i b u t i o n s " f o r . T h e s e c a n p r e s u m a b l y b e u s e d , i n t h e s a m e
m a n n e r as p o s t e r i o r o r f i d u c i a l d i s t r i b u t i o n s , t o m a k e i n f e r e n c e s o r p r o b a b i l i t y
s t a t e m e n t s a b o u t . T h e s t r u c t u r a l d e n s i t i e s , b a s e d o n S , f o r = ( ^, , )
a r e g i v e n f o r t h e t w o m o d e l s ( 3 . 6 . 1 ) a n d ( 3 . 6 . 2 ) , r e s p e c t i v e l y , by ( s e e
F r a s e r ( 1 9 6 8 ) )
( 3 . 6 . 3 ) I
1
( | S ) = K ^ s j y s j "
1
,
a n d
(3.6.4)
2
( | s ) = K
2
( s ) f
( s ) ( A
2
r V
.
( T h e s e h a p p e n t o c o r r e s p o n d t o t h e p o s t e r i o r d i s t r i b u t i o n s w i t h r e s p e c t t o t h e
r i g h t i n v a r i a n t H a a r m e a s u r e s o n t h e l o w e r a n d u p p e r t r i a n g u l a r g r o u p d e c o m p o
s i t i o n s o f $ . ) E x a m p l e s w i l l b e g i v e n i n t h e n e x t s e c t i o n w h i c h s h o w t h a t u s e
o f t h e s e d i f f e r i n g s t r u c t u r a l d i s t r i b u t i o n s c a n l e a d t o d i f f e r i n g c o n c l u s i o n s .
3 . 7 V I O L A T I O N O F T H E L I K E L I H O O D P R I N C I P L E : I N A D M I S S I B I L I T Y A N D I N C O H E R E N C Y
3 . 7 . 1 I n t r o d u c t i o n
T h e a l t e r n a t i v e t o j u s t i f i c a t i o n o f t h e L P f r o m " f i r s t p r i n c i p l e s "
i s t o s h o w t h a t b e h a v i o r i n v i o l a t i o n o f t h e L P is i n f e r i o r . T h e o n l y c o n v i n c
i n g m e t h o d o f d e m o n s t r a t i n g s u c h i n f e r i o r i t y is t o s h o w t h a t s u c h b e h a v i o r c a n
b e i m p r o v e d u p o n in r e p e a t e d u s e . W e t h u s t u r n t o m e a s u r e s o f l o n g r u n p e r
f o r m a n c e o f s t a t i s t i c a l p r o c e d u r e s o r m e t h o d s . W e w i l l n o t a r g u e t h a t
m e a s u r e s o f l o n g r u n p e r f o r m a n c e h a v e a n i m p o r t a n t p r a c t i c a l r o l e i n
s t a t i s t i c s ( a s f r e q u e n t i s t s w o u l d a r g u e ) , b u t w e w i l l a r g u e t h a t t h e y h a v e t h e
i m p o r t a n t t h e o r e t i c a l r o l e o f p r o v i d i n g a t e s t f o r p r o p o s e d m e t h o d o l o g i e s : i t
c a n n o t b e r i g h t ( p h i l o s o p h i c a l l y ) t o r e c o m m e n d r e p e a t e d u s e o f a m e t h o d i f t h e
m e t h o d h a s " b a d " l o n g r u n p r o p e r t i e s . B o t h o f t h e m a i n a p p r o a c h e s t o l o n g r u n
e v a l u a t i o n , d e c i s i o n t h e o r y a n d b e t t i n g c o h e r e n c y , w i l l b e d i s c u s s e d . W e w i l l
f u r t h e r a r g u e t h a t t h e d e c i s i o n t h e o r e t i c a p p r o a c h is t h e m o r e s a t i s f a c t o r y
o f t h e t w o ( e v e n f o r " i n f e r e n c e " p r o b l e m s ) , a l t h o u g h e i t h e r a p p r o a c h s t r o n g l y
c o n t r a i n d i c a t e s v i o l a t i o n o f t h e L P .
A v i o l a t i o n o f t h e L P w i l l o c c u r ( i n t h e d i s c r e t e c a s e ) w h e n t h e r e
a r e t w o e x p e r i m e n t s E ^ a n d E
2
, w i t h x ^ X^ a n d x e X~ s a t i s f y i n g ( f o r s o m e
p o s i t i v e c o n s t a n t c )
( 3 . 7 . 1 ) f j ( x ' ) = c f * ( x ) f o r a l l ,
a n d f o r w h i c h
( 3 . 7 . 2 ) E v ( E
x j ) f E v ( E
2
, x ' ) .
C o n s i d e r n o w t h e m i x e d e x p e r i m e n t E * , i n w h i c h J = 1 o r 2 , w i t h p r o b a b i l i t y ~
e a c h , is o b s e r v e d ( i n d e p e n d e n t o f a l l e l e m e n t s o f t h e E . ) , a n d e x p e r i m e n t E , is
t h e n p e r f o r m e n d . A c c o r d i n g t o t h e W C P ,
E v ( E * , ( j
f
X j ) ) = E v E j . X j ) ,
w h i c h c o m b i n e d w i t h ( 3 . 7 . 2 ) y i e l d s t h e c o n c l u s i o n
( 3 . 7 . 3 ) E v ( E * , ( l , x p ) f E v ( E * , ( 2 , x ) ) .
I t w i l l b e b e h a v i o r a c c o r d i n g t o ( 3 . 7 . 3 ) t h a t is s h o w n t o b e i n f e r i o r i n
r e p e a t e d u s e .
I n t h e n o n d i s c r e t e c a s e , w e c a n c o n s i d e r v i o l a t i o n o f t h e R L P ( s e e
S e c t i o n 3 . 4 . 1 ) . T h u s s u p p o s e t h a t , in t h e s i t u a t i o n o f t h e R L P , t h e r e e x i s t s a
s e t A c U , , w i t h P g ( A ) > 0 f o r a l l , a n d s u c h t h a t
( 3 . 7 . 4 ) E v ( E
1
, x
1
) t E v ( E
2
, x , ) ) .
A g a i n c o n s i d e r i n g t h e m i x e d e x p e r i m e n t E * a n d a p p l y i n g t h e W C P , o n e o b t a i n s
t h a t , f o r x
1
A ,
( 3 . 7 . 5 ) E v E M l . x , ) ) f E v ( E * , ( 2 , t x , ) ) ) ,
b e h a v i o r w h i c h w i l l b e s h o w n t o a l s o h a v e b a d l o n g r u n p r o p e r t i e s .
T h e e x p e r i m e n t E * w i l l p r e s e r v e a l l " s t r u c t u r a l " f e a t u r e s o f Ej a n d
E
2
, s o t h e o n l y o b j e c t i o n t h a t c o u l d b e r a i s e d c o n c e r n i n g t h e a b o v e l i n e o f
r e a s o n i n g i s t h e u s e o f t h e W C P . A l t h o u g h s o m e f r e q u e n t i s t s w i l l r e j e c t t h e
W C P ( a n d a r e t h e n e x e m p t f r o m t h e c o n c l u s i o n s o f t h i s s e c t i o n ) m o s t w i l l f i n d
s u c h r e j e c t i o n d i f f i c u l t . V i r t u a l l y a l l o t h e r t h e o r i e s a c c e p t t h e W C P , a n d
a r e h e n c e s u b j e c t t o e v a l u a t i o n t h r o u g h E * . A m o n g t h e t h e o r i e s w h i c h s e e m t o
a c c e p t t h e W C P , a n d y e t s o m e t i m e s a d v o c a t e v i o l a t i o n o f t h e L P , a r e ( t h e
a l r e a d y d i s c u s s e d ) P i v o t a l I n f e r e n c e a n d S t r u c t u r a l I n f e r e n c e , F i d u c i a l
I n f e r e n c e , P l a u s i b i l i t y I n f e r e n c e ( s e e B a r n d o r f f N i e l s e n ( 1 9 7 6 ) ) , a n d c e r t a i n
n o n i n f o r m a t i v e p r i o r B a y e s i a n t h e o r i e s ( s e e E x a m p l e 9 in S e c t i o n 3 . 1 ) . I t
s h o u l d b e n o t e d t h a t it is a c t u a l l y r a t h e r r a r e f o r t h e s e t h e o r i e s t o c o n f l i c t
w i t h t h e L P . I n d e e d t h e c o n f l i c t w o u l d n o t b e w o r t h m a k i n g a n i s s u e o f , w e r e it
n o t f o r t h e p u r p o r t e d r e f u t a t i o n s o f t h e L P t h a t s e e m t o a r i s e f r o m t h e s e
t h e o r i e s . T h e " r e f u t a t i o n s " a r e a l w a y s o f t h e f o r m " f o l l o w i n g t h e o r y A
c o n f l i c t s w i t h t h e L P , s o t h e L P m u s t b e w r o n g . " W e w i l l a r g u e ( v i a l o n g r u n
e v a l u a t i o n ) t h a t t h e r e v e r s e is t r u e .
3 . 7 . 2 D e c i s i o n T h e o r e t i c E v a l u a t i o n
T h e d e c i s i o n t h e o r e t i c a p p r o a c h s u p p o s e s t h a t t h e r e s u l t o f t h e
s t a t i s t i c a l i n v e s t i g a t i o n is t o t a k e a n a c t i o n a e G ( w h i c h c o u l d c o n c e i v a b l y
b e t h e a c t i o n t o t a k e a p a r t i c u l a r " i n f e r e n c e " ) , t h e c o n s e q u e n c e o f w h i c h , f o r
g i v e n d a t a x a n d w h e n o b t a i n s , is t h e l o s s L ( a , ) . I t is a l s o s u p p o s e d t h a t
t h e s t a t i s t i c a l m e t h o d b e i n g e v a l u a t e d p r o v i d e s a n a c t i o n t o t a k e f o r e a c h
p o s s i b l e x , t h u s d e f i n i n g a s t a t i s t i c a l p r o c e d u r e ( ) : Z + G . ( F o r t h e
m o s t p a r t w e w i l l s t i c k t o n o n r a n d o m i z e d p r o c e d u r e s f o r s i m p l i c i t y . ) A s u s u a l
in f r e q u e n t i s t d e c i s i o n t h e o r y , w e d e f i n e t h e f r e q u e n t i s t r i s k a n d t h e B a y e s
r i s k ( w i t h r e s p e c t t o a p r i o r d i s t r i b u t i o n o n ) a s , r e s p e c t i v e l y ,
R ( , ) = E L ( ( X ) , ) , a n d r ( , ) = E
R ( , ) .
F o l l o w i n g H i l l ( 1 9 7 4 b ) a n d B e r g e r ( 1 9 8 4 a ) , a n d in a s i m i l a r m a n n e r
t o m a n y b e t t i n g s c e n a r i o s , w e c o n s i d e r t h e f o l l o w i n g g a m e .
1 2
E V A L U A T I O N GA M E . P l a y e r 1 p r o p o s e s u s e o f 6 a n d P l a y e r 2 p r o p o s e s 6 . A
m a s t e r o f c e r e m o n i e s w i l l c h o o s e a s e q u e n c e = ( p p ) C {a c l as s of
rel ev an t s equ en c es ) , an d f or eac h . t h e e x p e r i m e n t E w i l l b e i n d e p e n d e n t l y
p e r f o r m e d y i e l d i n g a n o b s e r v a t i o n X . ( f r o m t h e d i s t r i b u t i o n P . ) . P l a y e r j
i
w i l l u s e
J
( x
i
)> p a y i n g t o t h e o t h e r p l a y e r h i s "l o s s " L (
J
( x
i
) ,
i
) . A f t e r n
p l a y s
3
P l a y e r 2 w i l l h a v e w o n
n
n
= X [L^x.h,.
I f , f o r a n y C ,
( 3 . 7 . 6 ) P (lim i n f ^ S
n
> 0 ) = 1 ,
2 1
t h e n 6 w i l l b e c a l l e d C - b e t t e r t h a n 6 .
A l t h o u g h t h e r e a r e a n u m b e r o f r e a s o n a b l e c h o i c e s f o r c in t h e
E v a l u a t i o n G a m e , a p a r t i c u l a r l y a t t r a c t i v e c h o i c e is
Z
n
= { : t h e r e e x i s t s a c omp ac t s e t K c @ f o r wh i c h . K f o r ev er y i } .
T h i s c h o i c e is a t t r a c t i v e b e c a u s e r e a l i t y is b o u n d e d , b u t t h e b o u n d is o f t e n
u n k n o w n ( a n d , h e n c e , w e e n t e r t a i n u n b o u n d e d m o d e l s ) . W i t h such a C , t h e
E v a l u a t i o n G a m e s e e m s to b e a f a i r w a y o f t e s t i n g t h e p e r f o r m a n c e o f a p r o c e -
d u r e . If i s c e r t a i n to l ose an a r b i t r a r i l y l a r g e a m o u nt in c o m p a r i s o n w i t h
2 1
6 , i t w o u l d c e r t a i nl y s e e m u nw i s e to cal l f u nd a m e nt a l l y s o u nd . T he f o l l o w
i ng t he o r e m is useful in d e a l i ng w i t h c ^ .
T H E O R E M 4 . S u p p o s e R ( ,
2
) < R ( ,
) f o r a l l , t h a t [ R ( ,
]
) R ( ,
2
) ]
i s c o n t i n u o u s i n
3
a n d t h a t t h e ra n d o m v a ri a b l e s
X . J. . JU ^X . J. . ) ]
h a v e u n i f o rm l y b o u n d e d v a ri a n c e s ( w h i c h i s t ri v i a l l y s a t i s f i e d i f L i s
2 1
b o u n d e d ) . T h e n i s C ^ b e t t e r t h a n i n t h e E v a l u a t i o n G a m e .
P ro o f . D e f i ne
(
. ) = E

( Z
i
) = R (
) R ( . ,
2
) .
By the s t r o ng law o f l arge n u m b e r s ,
1
n
y [Z . ( . ) J > 0 a l m o s t s u r e l y ,
n
i = l
1
so t h a t , f o r any ,
( 3 . 7 . 7 ) P (l i m i nf 1 S > 0 ) = P (1 1 m i nf 1 J ( . ) > 0 ) .
K n ^ n i = l
im i nf 1 S > 0 ) = P (1 1 m i nf 1
B u t s i nc e t he . l i e in s o m e c o m p a c t set and ( ) is c o nt i nu o u s and
posi t i ve ,
i nf ( . ) > 0.
i <
The conclusion is immediate from ( 3 . 7 . 7 ) . | |
2
The condit ion "R( , ) < R( , ) f or a l l " i n Theorem 4 implies
t hat 6 is inadmissible in a f r e que nt ist decision t heoret ic sense. This is
r e a l l y the key condit ion in the f a i l ur e of 6 in the Evaluat ion Game. Indeed
we can, in a loose sense, equate such f a i l ur e wit h i nadmi ssi bi l i t y. The
exact r elat ionship depends on the choice of C in the Evaluat ion Game, so we
wi l l sometimes use the term " inadmissibilit y" to encompass the whole idea.
Adopting a decision t heoret ic viewpoint f or evaluat ion can be
c r i t i c i z e d , especially f or inf erence problems in which losses ( i f they e xist
at a l l ) are vague or hard to f ormulat e. This is not the place to argue the
case f or a decision t heoret ic out look, and indeed a j ust i f i c a t i on of decision
theory is not needed f or our purpose here. Our goal is t o judge the claim i n
PS analysis ( and ot her approaches) t hat the LP is i nva l i d, because i t ignores
important f eat ures of the experiment. We wi l l e sse nt i al l y t r y to argue t ha t ,
in any decision problem, repeated vi ol at i on of the LP wi l l r e sult in long run
loss. M ost st a t i st i c i a ns would probably have qualms about t r yi ng t o argue
t ha t , even i f the LP should be f ollowed in any decision problem, i t need not be
f ollowed in inf erence problems. Essent ially such an argument would be of the
var ie t y " I know m r i ght , but wi l l not allow any quant i f i abl e evaluat ion
of my methods. "
We wi l l avoid the "unf air" possi bi l i t y of t aking an inf erence
procedure and evaluat ing i t wit h respect t o a pa r t i cul a r loss f unct ion. I t is
somewhat more f a i r t o evaluat e i t wit h respect to a very wide range of loss
f unct ions, and i nf e r i or performance f or a wide range of reasonable losses
should be a serious concern. M ore commonly, however, we wi l l consider
pa r t i cul a r losses as given, and see where the f ollowing of PS ( or ot her)
reasoning might lead us. C r i t i c i z i ng P S reasoning ( i n pa r t i c ul a r , possible
v i o l a t i o n o f t h e L P ) in d e c i s i o n s e t t i n g s f o r w h i c h it w a s n e v e r i n t e n d e d i s ,
o f c o u r s e , a n u n c e r t a i n u n d e r t a k i n g , e s p e c i a l l y s i n c e i t is n o t c l e a r w h a t
P S r e a s o n i n g in d e c i s i o n c o n t e x t s w o u l d b e . O f r e l e v a n c e h e r e is t h e
f o l l o w i n g c o m m e n t o f H i l l ( 1 9 7 4 b ) :
" B u t n o m a t t e r w h a t i s m e a n t b y i n f e r e n c e ,
i f i t is t o b e o f a n y v a l u e , t h e n s o m e h o w
it m u s t b e u s e d , o r a c t e d u p o n , a n d t h i s
d o e s i n d e e d l e a d b a c k t o t h e d e c i s i o n
t h e o r e t i c f r a m e w o r k . I s u s p e c t t h a t f o r
s o m e ' i n f e r e n c e ' is u s e d a s a s h i e l d t o
d i s c o v e r y t h a t t h e i r a c t i o n s a r e i n c o h e r e n t . "
A s an e x a m p l e o f a r e a s o n a b l e " i n f e r e n c e " l o s s , i m a g i n e t h a t a
g i v e n " c o n f i d e n c e " s e t C is t o b e u s e d , a n d t h a t t h e d e s i r e d i n f e r e n c e is a
m e a s u r e , ( x ) , o f t h e " c h a n c e " o r " c o n f i d e n c e " w i t h w h i c h w e w i s h t o a s s e r t
t h a t C c o n t a i n s . N o m a t t e r w h a t i n t e r p r e t a t i o n is a t t a c h e d t o ( x ) , it
s e e m s r e a s o n a b l e t o m e a s u r e i t s p e r f o r m a n c e v i a a l o s s f u n c t i o n w h i c h r e f l e c t s
w h e t h e r o r n o t 6 d o e s a g o o d j o b o f i n d i c a t i n g t h e p r e s e n c e o f in C . O n e
s u c h l o s s f u n c t i o n i s
( 3 . 7 . 8 ) L ( ( x ) , ) = ( I
c
( ) ( x ) )
2
,
e s s e n t i a l l y t h e q u a d r a t i c s c o r i n g f u n c t i o n o f d e F i n e t t i ( 1 9 6 2 ) . ( A n y o t h e r
p r o p e r s c o r i n g f u n c t i o n w o u l d a l s o b e r e a s o n a b l e c . f . L i n d l e y ( 1 9 8 2 ) . ) N o t e
t h a t f o r a n y " p o s t e r i o r " d i s t r i b u t i o n , ( | x ) , f o r , t h e o p t i m a l c h o i c e o f
( x ) in ( 3 . 7 . 8 ) is
( 3 . 7 . 9 )
( x ) = E
(
l
x )
l
c
( ) = P
(
l
x )
( C ) ,
i . e . , t h e p o s t e r i o r p r o b a b i l i t y o f C . T h u s , t o t e s t t h e i n f e r e n c e s
p r o v i d e d b y S t r u c t u r a l I n f e r e n c e in E x a m p l e 1 4 , it s e e m s r e a s o n a b l e t o u s e
t h e s t r u c t u r a l d i s t r i b u t i o n s p r o v i d e d b y ( 3 . 6 . 3 ) a n d ( 3 . 6 . 4 ) t o d e t e r m i n e
i H o
( s ) a n d ( s ) v i a ( 3 . 7 . 9 ) , a n d t h e n t e s t t h e i m p l i e d p r o c e d u r e i n t h e
E v a l u a t i o n G a m e f o r t h e m i x e d e x p e r i m e n t E * ( s e e S e c t i o n 3 . 7 . 1 ) . W e w i l l
r e t u r n t o t h i s e x a m p l e l a t e r .
The si mpl est s i t u at i on , i n which vi ol at i on of the LP ( or RLP)
resul t s i n f ai l i n g the Eval uati on Game f or E*, i s when L i s s t r i c t l y convex
i n V f o r al l . ( For some other s i t u at i on s , see Berger ( 1 984a) .) Consider
f i r s t the di sc ret e case i n Secti on 3 . 7 . 1 . A vi ol at i on of the LP (see ( 3 . 7 . 3 ) )
would imply use of a i n E* f or which
( 3.7 .1 0)
( ( l , x' ) ) f
]
( ( 2, xp ) .
Consider, however, the procedure
^
1
l
1
( ( 2 , x p ) f o r
( 3.7 .1 1 ) 6
2
( ( j , Xj ) ) =
X j
= x \ o r x
2
6
( ( j , . ) ) ot herwi se,
J
where c i s from ( 3 . 7 . 1 ) . Using the s t r i c t convexi ty of L, one ob tai ns t hat
( 3.7 .1 2) L(
2
( ( j , x. j ) ) , ) < ^ L(
]
( ( l , xp ) , )
+
(
|
T
y L (
1
( ( 2 , x ' ) ) , ) .
A n e a s y c a l c u l a t i o n , u s i n g ( 3 . 7 . 1 ) , t h e n s h o w s t h a t
( 3 . 7 . 1 3 ) R ( ,
) R ( ,
2
) = ^ f J( x p ( ) ,
w h e r e ( ) is t h e d i f f e r e n c e b e t w e e n t h e r i g h t a n d l e f t h a n d s i d e s o f ( 3 . 7 . 1 2 ) .
U n d e r t h e a d d i t i o n a l e a s i l y s a t i s f i a b l e c o n d i t i o n s o f T h e o r e m 4 , it i s
i m m e d i a t e t h a t f a i l s t h e E v a l u a t i o n G a m e f o r a l l 6 C ~ , ( T h i s is a l l , o f
c o u r s e , a f o r m o f t h e R a o B l a c k w e l l T h e o r e m . )
E X A M P L E 9 ( c o n t i n u e d s e e S e c t i o n 3 . 1 ) . S u p p o s e i t is d e s i r e d t o e s t i m a t e
u n d e r t h e l o s s L = ( e a ) ( o r a n y o t h e r s t r i c t l y c o n v e x l o s s ) , a n d t h a t ,
w o u l d b e r e c o m m e n d e d f o r E ^ a n d
2
f o r E
2 >
w h e r e i ( 9 ) f
2
( 9 ) ; t h u s a
v i o l a t i o n o f t h e L P w i l l h a v e o c c u r r e d . ( N e i t h e r P i v o t a l n o r S t r u c t u r a l
i n f e r e n c e w o u l d n e c e s s a r i l y r e c o m m e n d d i f f e r e n t a c t i o n s h e r e , b u t t h e
Je f f r e y s n o n i n f o r m a t i v e p r i o r B a y e s t h e o r y a n d a l s o A k a i k e ( 1 9 8 2 ) w o u l d s e e m t o
s o r e c o m m e n d . ) T h e s i t u a t i o n m e s h e s e x a c t l y w i t h t h e d i s c r e t e s e t t i n g d i s c u s s e d
THE LIKELIHOOD PRINC IPLE AND GENERALIZATIONS 5 7
above, and so i f one ( f ol l owi ng t he WCP) used
( 3 . 7 . 1 4 ) ^ U . X j ) ) = j xj )
f or t he mixed experiment E* , ( 3 . 7 . 1 3 ) would hold. I t f ollows from Theorem 4
t hat
1
f a i l s the Evaluat ion Game f or C^ Note that
1
would not f a i l the
Evaluat ion Game f or any which converged to zero or one. The f a i l ur e of 6 f or
any e C ^,, or even more generally f or any which l i e s wit hin a compact subset
of some posit ive f r act i on of the t ime, st r ike s us, however, as strong enough
evidence t o rule out using .
The non discrete version of the above argument f or convex loss
would assume ( see the discussion around ( 3 . 7 . 4 ) ) t ha t , in vi ol at i on of the
RLP f or E*
f
( 3 . 7 . 1 5 ) Ô . x , ) ) 7 ^
]
( ( 2 , f x j ) ) , f o r x , A .
T h e a n a l o g o f ( 3 . 7 . 1 1 ) is n o w
( 3 . 7 . 1 6 ) 6
2
( ( j , x
j
) ) = E [
1
( ( J, X
J
) ) | T ( j , x
J
) 3 ,
t h e c o n d i t i o n a l e x p e c t a t i o n o f 6 g i v e n T , w h e r e T is t h e s u f f i c i e n t
s t a t i s t i c (in E * )
( 2 , t x j ) ) i f j = 1 and x
]

]
( j , x . ) ot herwise,
j
The appropriat e versions of ( 3 . 7 . 1 2 ) and ( 3 . 7 . 1 3 ) can e asily be est ablished
and under reasonable condit ions, f a i l ur e of in the Evaluat ion Game f ollows.
EXAM PLE 14 ( cont inued) . Suppose i t is desired to estimate J ( which is
equivalent to ) under the st r i c t l y convex loss
( 3 . 7 . 1 7 ) L( , J) = t r U I ' V l o g et { ^) 2 .
1 2
( The loss L( 4 ) = t r ( $ ~ I ) would work si mi l a r l y see James and St ein
( 1 9 6 1 ) a n d S e l l i a h ( 1 9 6 4 ) . ) I f o n e t r e a t s , ( | s ) a n d
9
( | s ) in ( 3 . 6 . 3 ) a n d
( 3 . 6 . 4 ) a s p o s t e r i o r s a n d c a l c u l a t e s t h e o p t i m a l e s t i m a t o r s w i t h r e s p e c t to
( 3 . 7 . 1 7 ) , o n e o b t a i n s
( 3 . 7 . 1 8 )
( s ) = s
n 1 )
1
0
0
( n+ i:
\
1
s
u
w h e r e s = s . s * = S y S y , s. a n d S y b e i n g l o w e r a n d u p p e r t r i a n g u l a r , r e s p e c t i v e l y .
I f t h e s e e s t i m a t o r s w o u l d b e u s e d in Ej a n d E,,, t h e W C P w o u l d l e a d t o u s i n g t h e
e s t i m a t o r 6 ( ( j , s ) ) = . ( s ) in t h e m i x e d e x p e r i m e n t E * .
J
T o e s t a b l i s h f a i l u r e o f 6 i n t h e E v a l u a t i o n G a m e , l e t
A = { s : ^( s ) f 6
2
( s ) } a n d n o t e t h a t A h a s p r o b a b i l i t y o n e f o r a l l . T h i s
s i t u a t i o n s a t i s f i e s t h e c o n d i t i o n s o f t h e R L P w i t h Uj a n d U
2
b e i n g t h e e n t i r e
s a m p l e s p a c e , c( ) = 1 , a n d b e i n g t h e i d e n t i t y m a p ( s i n c e t h e p r o b a b i l i t y
s p a c e is i d e n t i c a l f o r E , a n d E
2
) , a n d a l s o s a t i s f i e s ( 3 . 7 . 1 5 ) . T h e e s t i m a t o r
p
6 in ( 3 . 7 . 1 6 ) is s i m p l y
2
( ( j , s ) ) = ^ ( ( l . s ) ) + }
2
( ( 2 , s ) )
\
2
( s ) ,
a n d , f r o m t h e s t r i c t c o n v e x i t y o f t h e l o s s , i t f o l l o w s e a s i l y t h a t ( f o r E * )
R ( ,
2
) < R ( ,
) f o r a l l .
F u r t h e r m o r e , t h e c o n d i t i o n s o f T h e o r e m 4 c a n e a s i l y b e v e r i f i e d i n t h i s
2 1
s i t u a t i o n , a n d s o t h e c o n c l u s i o n o f t h e t h e o r e m a p p l i e s : is b e t t e r t h a n
i n t h e E v a l u a t i o n G a m e f o r a l l b o u n d e d s e q u e n c e s .
O f c o u r s e , t h i s s a m e a n a l y s i s w o u l d h o l d f o r a n y e s t i m a t o r s t h a t
d i f f e r f o r E
]
a n d E
2
, n o t j u s t f o r
]
a n d
2
i n ( 3 . 7 . 1 8 ) . T h u s v i o l a t i n g t h e
R L P b y u s i n g d i f f e r e n t e s t i m a t o r s i n t h e t w o c a s e s s e e m s d e f i n i t e l y c o n t r a
i n d i c a t e d .
T h e s a m e k i n d o f c o n c l u s i o n f o l l o w s i n t h e " i n f e r e n c e " s i t u a t i o n o f
g i v i n g t h e " c o n f i d e n c e " t o b e a t t a c h e d t o a s e t C , u s i n g a l o s s s u c h a s ( 3 . 7 . 8 ) .
I f , ( | s ) a n d
2
( | s ) a r e u s e d a s p o s t e r i o r s t o p r o d u c e p r o b a b i l i t i e s t h a t
is in C ( v i a ( 3 . 7 . 9 ) ) a n d t h e s e p r o b a b i l i t i e s d i f f e r ( a s w i l l u s u a l l y b e t h e
c a s e ) , a n a n a l y s i s v i r t u a l l y i d e n t i c a l t o t h a t a b o v e s h o w s t h a t t h e v i o l a t i o n
o f t h e R L P r e s u l t s in a n i n f e r e n c e f o r E * w h i c h f a i l s t h e E v a l u a t i o n G a m e f o r
a l l b o u n d e d . A g a i n , o n e c o u l d o b j e c t t o e v a l u a t i n g i n f e r e n c e s v i a ( 3 . 7 . 8 ) ,
b u t u s e o f a n y r e a s o n a b l e m e a s u r e o f t h e p e r f o r m a n c e o f i n f e r e n c e s w o u l d l e a d
t o t h e s a m e c o n c l u s i o n .
3 . 7 . 3 B e t t i n g E v a l u a t i o n
S t u d y i n g c o h e r e n c e in b e t t i n g h a s a l o n g t r a d i t i o n i n s t a t i s t i c s ,
e s p e c i a l l y B a y e s i a n s t a t i s t i c s . T h e t y p i c a l s c e n a r i o d e a l s w i t h e v a l u a t i o n
o f m e t h o d s ( u s u a l l y i n f e r e n c e m e t h o d s ) w h i c h p r o d u c e , f o r e a c h x , e i t h e r a
p r o b a b i l i t y d i s t r i b u t i o n f o r , s a y q
v
( ) ( w h i c h c o u l d b e a p o s t e r i o r d i s t r i b u
x
t i o n , a f i d u c i a l d i s t r i b u t i o n , a s t r u c t u r a l d i s t r i b u t i o n , e t c . ) , o r a s y s t e m o f
c o n f i d e n c e s t a t e m e n t s { C ( x ) , ( x ) } w i t h t h e i n t e r p r e t a t i o n t h a t is f e l t t o b e
i n C ( x ) w i t h p r o b a b i l i t y ( x ) . F o r s i m p l i c i t y , w e w i l l r e s t r i c t o u r s e l v e s t o
t h e c o n f i d e n c e s t a t e m e n t f r a m e w o r k ; a n y q
( ) } c a n b e a t l e a s t p a r t i a l l y
e v a l u a t e d t h r o u g h c o n f i d e n c e s t a t e m e n t s b y c h o o s i n g { C ( x ) } a n d l e t t i n g ( x ) b e
t h e p r o b a b i l i t y ( w i t h r e s p e c t t o q ) t h a t is i n C ( x ) .
T h e a s s u m p t i o n is t h e n m a d e ( m o r e o n t h i s l a t e r ) t h a t , s i n c e ( x ) is
t h o u g h t t o b e t h e p r o b a b i l i t y t h a t is i n C ( x ) , t h e p r o p o s e r o f { C ( x ) , ( x ) }
s h o u l d b e e q u a l l y w i l l i n g t o a c c e p t e i t h e r t h e b e t t h a t is in C ( x ) , a t o d d s o f
( l ( x ) ) t o ( x ) , o r t h e b e t t h a t is n o t i n C ( x ) , a t o d d s o f ( x ) t o
( l ( x ) ) . A n e v a l u a t i o n s g a m e , a s i n S e c t i o n 3 . 7 . 2 , i s t h e n p r o p o s e d , i n
w h i c h t h e m a s t e r o f c e r e m o n i e s a g a i n g e n e r a t e s ^ a n d X ^
t
P l a y e r 1 s t a n d s r e a d y
t o a c c e p t b e t s o n { C ( x ) , ( x ) } , a n d P l a y e r 2 b e t s s ( x ) a t o d d s d e t e r m i n e d b y
( x ) . H e r e , s ( x ) = 0 m e a n s n o b e t is o f f e r e d ; s ( x ) > 0 m e a n s t h a t a n a m o u n t
s ( x ) is b e t t h a t C ( x ) ; a n d s ( x ) < 0 m e a n s t h a t t h e a m o u n t | s ( x ) | is b e t
t h a t $ C ( x ) . ( A s d i s c u s s e d i n R o b i n s o n ( 1 9 7 9 a ) , r e s t r i c t i n g s ( x ) t o s a t i s f y
I s ( x ) I <_ 1 is a l s o s e n s i b l e . ) T h e w i n n i n g s o f P l a y e r 2 a t t h e i t h p l a y a r e
W
i "cx^ i ^ i ^ V
1
n
and of i nt e r e st is again the l i mi t i ng behavior of I W . I f , f or some e > 0,
( 3 . 7 . 1 9 ) PJ l i m i nf J [ W. > e) = 1
f o r a l l s e q u e n c e s = (
]
, .
2
, . . . ) , t h e n { C ( x ) , ( x ) } w i l l b e c a l l e d i n c o h e r e n t ,
o r a l t e r n a t i v e l y s ( x ) w i l l b e s a i d t o b e a s u p e r r e l e v a n t b e t t i n g s t r a t e g y . I f
it is m e r e l y t h e c a s e t h a t ( 3 . 7 . 1 9 ) h o l d s f o r a n y 6 C ^ w i t h = 0 , t h e n
C ( x ) , ( x ) } w i l l b e c a l l e d w e a k l y i n c o h e r e n t o r s ( x ) w i l l b e s a i d t o b e w e a k l y
r e l e v a n t . ( T h e s e c o n c e p t s c a n b e f o u n d i n d i f f e r e n t , b u t c l o s e l y r e l a t e d , f o r m s
in s u c h w o r k s a s B u e h l e r ( 1 9 5 9 , 1 9 7 6 ) , W a l l a c e ( 1 9 5 9 ) , F r e e d m a n a n d P u r v e s
( 1 9 6 9 ) , C o r n f i e l d ( 1 9 6 9 ) , P i e r c e ( 1 9 7 3 ) , B o n d a r ( 1 9 7 7 ) , H e a t h a n d S u d d e r t h
( 1 9 7 8 ) , R o b i n s o n ( 1 9 7 9 a , 1 9 7 9 b ) , L e v i ( 1 9 8 0 ) , a n d L a n e a n d S u d d e r t h ( 1 9 8 3 ) . )
I f { C ( x ) , ( x ) } i s i n c o h e r e n t o r w e a k l y i n c o h e r e n t , t h e n P l a y e r 1
w i l l f o r s u r e l o s e m o n e y i n t h e a p p r o p r i a t e e v a l u a t i o n s g a m e , w h i c h c e r t a i n l y
c a s t s d o u b t o n t h e v a l i d i t y o f t h e p r o b a b i l i t i e s ( x ) . A n u m b e r o f o b j e c t i o n s
t o t h e s c e n a r i o c a n , a n d h a v e , b e e n r a i s e d , h o w e v e r , a n d c a r e f u l e x a m i n a t i o n
o f t h e s e o b j e c t i o n s i s w o r t h w h i l e .
O b j e c t i o n 1 . P l a y e r 1 w i l l h a v e n o i n c e n t i v e t o b e t u n l e s s h e p e r c e i v e s t h e
o d d s a s s l i g h t l y f a v o r a b l e . T h i s t u r n s o u t t o b e n o p r o b l e m i f i n c o h e r e n c e i s
p r e s e n t , s i n c e t h e o d d s c a n b e a d j u s t e d b y / 2 i n P l a y e r l' s f a v o r , a n d
P l a y e r 2 w i l l s t i l l w i n . I f o n l y w e a k i n c o h e r e n c e i s p r e s e n t , i t i s s t i l l
o f t e n p o s s i b l e t o a d j u s t t h e o d d s b y a f u n c t i o n g ( x ) s o t h a t P l a y e r 1 p e r c e i v e s
t h a t t h e g a m e i s i n h i s f a v o r , y e t w i l l l o s e in t h e l o n g r u n , b u t t h i s is n o t
c l e a r l y a l w a y s t h e c a s e .
O b j e c t i o n 2 . W e a k i n c o h e r e n c e h a s b e e n d e e m e d n o t v e r y m e a n i n g f u l , s i n c e a
s e q u e n c e = ( ^
f
2 > . ) c o u l d b e c h o s e n s o t h a t P l a y e r 1 i s n o t a s u r e l o s e r .
H o w e v e r , t h e f a c t t h a t P l a y e r 1 i s a s u r e l o s e r f o r a n y e Q ,
n
s e e m s q u i t e
s e r i o u s .
O b j e c t i o n 3 . O f c o u r s e , f r e q u e n t i s t s w h o q u o t e a c o n f i d e n c e l e v e l f o r
{ C ( x ) } r e m o v e t h e m s e l v e s f r o m t h e g a m e , s i n c e t h e y d o n o t c l a i m t h a t i s t h e
p r o b a b i l i t y t h a t i s in C ( x ) , a n d h e n c e w o u l d f i n d t h e b e t t i n g s c e n a r i o
t o t a l l y i r r e l e v a n t .
O b j e c t i o n 4 . T h e g a m e is u n f a i r t o P l a y e r 1 , s i n c e P l a y e r 2 g e t s t o c h o o s e
w h e n , h o w m u c h , a n d w h i c h w a y t o b e t . V a r i o u s p r o p o s a l s h a v e b e e n m a d e t o
" e v e n t h i n g s u p . " T h e p o s s i b i l i t y m e n t i o n e d in O b j e c t i o n 1 is o n e s u c h , b u t
d o e s n ' t c h a n g e t h e c o n c l u s i o n s m u c h . A m o r e r a d i c a l p o s s i b i l i t y , s u g g e s t e d
b y F r a s e r ( 1 9 7 7 ) , is t o a l l o w P l a y e r 1 t o d e c l i n e b e t s . T h i s c a n h a v e a
d r a s t i c e f f e c t , b u t s t r i k e s us a s t o o r a d i c a l , i n t h a t it g i v e s P l a y e r 1
l i c e n s e t o s t a t e c o m p l e t e l y s i l l y ( x ) f o r s o m e x . I t is a f t e r a l l { ( x ) }
t h a t is b e i n g t e s t e d , a n d t e s t i n g s h o u l d b e a l l o w e d f o r a l l x .
O b j e c t i o n 5 . T h e m o s t s e r i o u s o b j e c t i o n w e p e r c e i v e t o t h e b e t t i n g g a m e is
t h a t { ( x ) } is g e n e r a l l y n o t s e l e c t e d f o r u s e in t h e g a m e , b u t r a t h e r t o
c o m m u n i c a t e i n f o r m a t i o n a b o u t . I t m a y b e t h a t t h e r e is n o b e t t e r c h o i c e o f
{ 6 ( x ) } f o r c o m m u n i c a t i n g t h e d e s i r e d i n f o r m a t i o n . C o n s i d e r t h e f o l l o w i n g
e x a m p l e , w h i c h c a n b e f o u n d i n B u e h l e r ( 1 9 7 1 ) , a n d is e s s e n t i a l l y s u c c e s s i v e
m o d i f i c a t i o n s b y B u e h l e r a n d H . R u b i n o f a n e a r l i e r e x a m p l e o f D . B l a c k w e l l .
E X A M P L E 1 5 . S u p p o s e z a n d a r e t h e i n t e g e r s , a n d t h a t P
( X = + l ) =
P . ( X = 1 ) = Q . W e a r e t o e v a l u a t e t h e c o n f i d e n c e w e a t t a c h t o t h e s e t s
C ( x ) = { x + 1 } ( t h e p o i n t ( x + ) ) , a n d a n a t u r a l c h o i c e is ( x ) = j ( s i n c e is
e i t h e r x 1 o r x + 1 , a n d in t h e a b s e n c e o f f a i r l y s t r o n g p r i o r i n f o r m a t i o n a b o u t
, e i t h e r c h o i c e s e e m s e q u a l l y p l a u s i b l e ) . T h i s c h o i c e c a n b e b e a t e n i n t h e
b e t t i n g g a m e , h o w e v e r , b y b e t t i n g t h a t i s n o t in C ( x ) w i t h p r o b a b i l i t y g ( x ) ,
w h e r e 0 < g ( x ) < 1 is a n i n c r e a s i n g f u n c t i o n . ( A l l o w i n g P l a y e r 2 t o h a v e a
r a n d o m i z e d b e t t i n g s t r a t e g y d o e s n o t s e e m u n r e a s o n a b l e . ) I n d e e d , t h e e x p e c t e d
g a i n p e r b e t o f o n e u n i t , f o r a n y f i x e d , is y [ g ( + l ) g ( l ) ] > 0 , f r o m
w h i c h it is e a s y t o c h e c k t h a t ( x ) = ^
i s
w e a k l y i n c o h e r e n t . (A c o n t i n u o u s
v e r s i o n o f t h i s e x a m p l e , m e n t i o n e d in R o b i n s o n ( 1 9 7 9 a ) , h a s X ^ T ^ ( J) ,
= I R
, C ( X ) = (co, x ) , a n d ( x ) = | . )
I n t h i s a n d o t h e r e x a m p l e s w h e r e ( ( x ) } l o s e s in b e t t i n g , o n e c a n
a s k t h e c r u c i a l q u e s t i o n I s t h e r e a b e t t e r t h a t c o u l d b e u s e d ? T h e
q u e s t i o n h a s n o c l e a r a n s w e r , b e c a u s e t h e p u r p o s e o f is n o t c l e a r l y d e f i n e d .
O n e p o s s i b l e j u s t i f i c a t i o n f o r ( x ) = j
n
t h e a b o v e e x a m p l e is t h a t i t is t h e
u n i q u e l i m i t i n g p r o b a b i l i t y o f C ( x ) f o r s e q u e n c e s o f w h a t c o u l d b e c a l l e d
i n c r e a s i n g l y v a g u e p r i o r d i s t r i b u t i o n s . (A m o r e f o r m a l B a y e s i a n j u s t i f i c a t i o n
a l o n g t h e s e l i n e s w o u l d b e a r o b u s t B a y e s i a n j u s t i f i c a t i o n , t o t h e e f f e c t t h a t
t h e c l a s s o f p o s s i b l e p r i o r s is s o l a r g e t h a t t h e r a n g e o f p o s s i b l e p o s t e r i o r
p r o b a b i l i t i e s f o r C ( x ) w i l l i n c l u d e 1/2 f o r all x . ) A n a l t e r n a t i v e j u s t i f i c a
t i o n c a n b e f o u n d b y r e t r e a t i n g t o d e c i s i o n t h e o r y , a n d a t t e m p t i n g t o q u a n t i f y
h o w w e l l ( x ) p e r f o r m s u s i n g a l o s s s u c h a s ( 3 . 7 . 8 ) . O n e c a n t h e n a s k i f
t h e r e is a b e t t e r 6 in t e r m s , s a y , o f t h e d e c i s i o n t h e o r e t i c E v a l u a t i o n G a m e
f o r b o u n d e d . T h e a n s w e r in t h e c a s e o f E x a m p l e 1 5 is n o ! A s t a n d a r d
l i m i t i n g B a y e s a r g u m e n t c a n b e u s e d t o s h o w t h a t ( x ) = j i s d e c i s i o n
t h e o r e t i c a l l y a d m i s s i b l e f o r t h i s l o s s , f r o m w h i c h it f o l l o w s t h a t , f o r a n y
o t h e r <S* , a b o u n d e d ( i n d e e d c o n s t a n t ) s e q u e n c e c a n b e f o u n d s u c h t h a t 6 is
b e t t e r t h a n 6 * in t h e E v a l u a t i o n G a m e .
T h e E v a l u a t i o n G a m e ( o r d e c i s i o n t h e o r e t i c i n a d m i s s i b i l i t y ) w i t h
r e s p e c t t o l o s s e s s u c h a s ( 3 . 7 . 8 ) c a n b e r e l a t e d t o i n c o h e r e n c y , a n d s e e m s t o
b e a c r i t e r i o n s o m e w h e r e b e t w e e n w e a k i n c o h e r e n c y a n d i n c o h e r e n c y ( c . f .
R o b i n s o n ( 1 9 7 9 a ) ) . T h i s s u p p o r t s t h e f e e l i n g t h a t it m a y b e a m o r e v a l i d
c r i t e r i o n t h a n t h e b e t t i n g c r i t e r i o n . T h i s is n o t t o s a y t h a t t h e b e t t i n g
s c e n a r i o s a r e n o t i m p o r t a n t . B u e h l e r , i n d i s c u s s i o n o f F r a s e r ( 1 9 7 7 ) , m a k e s
t h e i m p o r t a n t p o i n t t h a t , a t t h e v e r y l e a s t , b e t t i n g s c e n a r i o s s h o w w h e n
q u a n t i t i e s s u c h a s ( x ) " b e h a v e d i f f e r e n t l y f r o m o r d i n a r y p r o b a b i l i t i e s . " A n d
a s H i l l ( 1 9 7 4 b ) s a y s
" . . . t h e d e s i r e f o r c o h e r e n c e . . . i s n o t
p r i m a r i l y b e c a u s e h e f e a r s b e i n g m a d e
a s u r e l o s e r b y a n i n t e l l i g e n t o p p o n e n t
w h o c h o o s e s a j u d i c i o u s s e q u e n c e o f
g a m b l e s . . . b u t r a t h e r b e c a u s e h e f e e l s
t h a t i n c o h e r e n c e is s y m p t o m a t i c o f s o m e
t h i n g b a s i c a l l y u n s o u n d in h i s a t t i t u d e s . "
T o s h o w t h a t v i o l a t i o n o f t h e L P ( o r R L P ) l e a d s t o s o m e f o r m o f
i n c o h e r e n c e , i t is a g a i n n e c e s s a r y t o c o n s i d e r t h e s e t u p i n S e c t i o n 3 . 7 . 1 .
Taking the discret e case f i r s t , suppose a f ixe d set C c is assigned
"confidence" c^ in Ej when xj is observed, but "confidence" a f , ) in E
2
when xL is observed. I f the WCP is followed f or the mixed experiment E* , the
confidence f unct ion employed sa t i sf i e s
) ) ) =
1
t
2
the appropriat e version of ( 3 . 7 . 3 ) . C onsider now the be t t ing st rat egy ( see the
beginning of the sect ion f or i nt e r pr e t at i on)
i f x . f x^ o r x
i f j = k a nd x . = x
or
c ( l . ) i f j f k a nd x . = xJ o r
J J J '
w h e r e cj = 1 , c
2
= c ( f r o m ( 3 . 7 . 1 ) ) , a nd k = 1 o r 2 as j <
2
o r c^ >
2 >
r e s p e c t i ve l y . If t hi s s t r a t e g y i s u s e d w i t h o d d s c o r r e s p o nd i ng to . w h e n
j
( j,x ' ) is o b s e r ve d , t he e x p e c t e d g a i n can b e e a s i l y c a l c u l a t e d t o b e
j
If f ( x p is b o u n d e d a w a y f r o m ze r o f o r all b o u nd e d s e qu e nc e s , i t f o l l o w s
e a s i l y t ha t is w e a kl y i n c o h e r e n t .
In t he n o n d i s c r e t e c a s e , o n e r e p l a c e s . a b o ve by . ( x . ) ( t he
j J J
"c o n f i d e n c e " i n C i f x i s o b s e r ve d i n E . ) , a nd a s s u m e s t h a t , f o r s o m e
j j
Ac U
wit h P^( A) > 0 f or a l l ,

^X ^ t
2
f or A.
The corresponding confidence f unct ion i n the mixed experiment E* is
( ( j , x ) ) = . ( . ) > which again violat es the RLP. C onsider, now, the be t t ing
J j J
st rat egy
0 i f ( j. X j) 4 A*
c
j
( x
j
) c
j
( x
j
) i f
^ "
k
t J
x
j ^
a n d
C J( X J) ( 1 O J( X J) ) if
and
A *
A * ,
w h e r e
1
= 1 , c
2
( x
2
) = c ( "' ( x
2
) ) ( s e e ( 3 . 4 . 1 ) ) ,
64
T H E L I K E L I H O O D P R I N C I P L E
A * = A } U A },
and
o r
1 i f j = l a n d ^X j ) < ^ U ^ )
j
=
2 a n d ( ( x ^
2 o t h e r w i s e .
T h e e x p e c t e d g a i n f o r t h i s b e t t i n g s t r a t e g y c a n e a s i l y b e c a l c u l a t e d t o b e
/ \ |
]
( x
1
)
2
(
( x
1
) ) | p j ( d x
1
) .
Weak incoherency wi l l again f ollow under reasonable condit ions.
For general theorems on coherence, consult Heath and Sudderth
( 1 9 7 8 ) and Lane and Sudderth ( 1 9 8 3 ) and the references t he r e i n. These theorems
indicat e t ha t , unless f or E* is compatible wit h some post erior di st r i but i on,
incoherency wi l l r e sul t . A coherent 6 wi l l not vi ol at e the LP ( or RLP) , and so
incoherence of vi ol a t i on of the LP is quit e general. Again, however, t hi s may
not be as convincing as the decision t heoret ic r e f ut at i on of vi ol at i on of the
LP which was discussed i n Sect ion 3 . 7 . 2 .
C H A P T E R 4 , C O N S E Q U E N C E S A N D C R I T I C I S M S O F T H E L I K E L I H O O D
P R I N C I P L E A N D R E L A T I V E L I K E L I H O O D P R I N C I P L E
M o s t p e o p l e w h o r e j e c t t h e L P d o s o b e c a u s e it h a s c o n s e q u e n c e s
t h e y d o n o t l i k e . O f c o u r s e a n y t h e o r y d e s e r v e s t o b e r e j e c t e d i f i t s c o n s e -
q u e n c e s a r e e r r o n e o u s , b u t g r e a t c a r e m u s t b e t a k e n in m a k i n g s u r e t h a t t h e
c o n s e q u e n c e s r e a l l y a r e w r o n g a n d n o t j u s t in o p p o s i t i o n t o t h e i n t u i t i o n
c u r r e n t l y d o m i n a n t in t h e f i e l d . I n t h i s s e c t i o n w e d i s c u s s s o m e o f t h e m o r e
s u r p r i s i n g c o n s e q u e n c e s o f t h e L P a n d R L P , a n d i n v e s t i g a t e t h e c o n f l i c t s w i t h
p r e v a l e n t s t a t i s t i c a l i n t u i t i o n . I t w i l l c o m e a s n o s u r p r i s e t h a t w e f e e l
t h a t t h e c o n f l i c t s a r e a l w a y s r e s o l v e d i n f a v o r o f t h e L P a n d R L P .
4.1 I N C O M P A T I B I L I T Y W I T H F R E Q U E N T I S T C O N C E P T S
4 . 1 . 1 I n t r o d u c t i o n
T h e p h i l o s o p h i c a l i n c o m p a t i b i l i t y o f t h e L P a n d t h e f r e q u e n t i s t
v i e w p o i n t i s c l e a r , s i n c e t h e L P d e a l s o n l y w i t h t h e o b s e r v e d x , w h i l e f r e q u e n -
t i s t a n a l y s e s i n v o l v e a v e r a g e s o v e r p o s s i b l e o b s e r v a t i o n s . I t c a n n o t b e s a i d ,
h o w e v e r , t h a t a n y p a r t i c u l a r f r e q u e n t i s t p r o c e d u r e c o n f l i c t s w i t h t h e L P ,
s i n c e t h e p r o c e d u r e c o u l d h a p p e n t o c o r r e s p o n d t o a s e n s i b l e c o n d i t i o n a l
p r o c e d u r e . S u c h a c o r r e s p o n d e n c e d o e s , in f a c t , o c c u r in m a n y s t a t i s t i c a l s i t u -
a t i o n s . F o r i n s t a n c e , m u c h o f f r e q u e n t i s t n o r m a l d i s t r i b u t i o n t h e o r y i n f e r e n c e
p r o v i d e s t h e s a m e n u m e r i c a l m e a s u r e s o f " c o n f i d e n c e " a s d o e s n o n i n f o r m a t i v e
p r i o r c o n d i t i o n a l B a y e s i a n t h e o r y ( b e c a u s e o f t h e s y m m e t r i e s o r g r o u p s t r u c t u r e
o f t h e p r o b l e m ) , a l t h o u g h t h e i n t e r p r e t a t i o n s o f t h e s e m e a s u r e s a r e d i f f e r e n t .
(A c y n i c m i g h t a r g u e t h a t f r e q u e n t i s t s t a t i s t i c s h a s s u r v i v e d p r e c i s e l y b e c a u s e
o f s u c h l u c k y c o r r e s p o n d e n c e s . ) N e v e r t h e l e s s , e n o u g h d i r e c t c o n f l i c t s h a v e b e e n
6 5
( a n d w i l l b e ) s e e n t o j u s t i f y v i e w i n g t h e L P as r e v o l u t i o n a r y f r o m a
f r e q u e n t i s t p e r s p e c t i v e .
W e h a v e a l r e a d y a l l u d e d t o t h e f a c t t h a t a f r e q u e n t i s t c a n
l o g i c a l l y d i s m i s s t h e L P , e s s e n t i a l l y b y r e j e c t i n g t h e W C P a n d c o n c l u d i n g t h a t
t h e c o n c e p t o f l e a r n i n g o r d r a w i n g c o n c l u s i o n s a b o u t , f o r a p a r t i c u l a r
e x p e r i m e n t , is m e a n i n g l e s s . T h u s N e y m a n ( c . f . N e y m a n ( 1 9 5 7 , 1 9 6 7 , 1 9 7 7 ) )
e s p o u s e s t h e v i e w p o i n t t h a t o n l y t h e p e r f o r m a n c e o f a p r o c e d u r e i n r e p e a t e d u s e
i s r e l e v a n t , a n d t h a t it is a m i s t a k e t o t h i n k in t e r m s o f l e a r n i n g a b o u t
p a r t i c u l a r . T h o u g h l o g i c a l l y v i a b l e , t h i s v i e w p o i n t is s c i e n t i f i c a l l y
u n a p p e a l i n g . E x p e r i m e n t s a r e d o n e p r e c i s e l y t o o b t a i n " e v i d e n c e " a b o u t
u n k n o w n , a n d i n v e s t i g a t o r s w i l l n o t t a k e k i n d l y t o b e i n g t o l d t h a t t h i s is
m e a n i n g l e s s . T h u s B i r n b a u m ( 1 9 7 7 ) a r g u e s t h a t N e y m a n P e a r s o n c o n c l u s i o n s a r e
v i r t u a l l y a l w a y s u s e d in a n " e v i d e n t i a r y " f a s h i o n , r a t h e r t h a n as m e a s u r e s o f
p r o c e d u r e p e r f o r m a n c e i n r e p e a t e d u s e . S a v a g e p u t t h i s v e r y s u c c i n c t l y w h e n
t a l k i n g a b o u t c o n f i d e n c e s e t s in S a v a g e e t . a l . ( 1 9 6 2 ) :
" T h e o n l y u s e I k n o w f o r a c o n f i d e n c e
i n t e r v a l is t o h a v e c o n f i d e n c e i n i t . "
S u p p o s i n g t h e n t h a t w e a r e g o i n g t o u s e a f r e q u e n c y m e a s u r e a s a
m e a s u r e o f e v i d e n c e a b o u t , w h a t c l a s s i c a l j u s t i f i c a t i o n s f o r s u c h b e h a v i o r
c a n b e a d v a n c e d ? T h e r e a r e a t l e a s t t h e f o l l o w i n g f o u r :
( i ) F r e q u e n c y m e a s u r e s a r e " o b j e c t i v e " , h a v i n g a w e l l d e f i n e d p h y s i c a l
i n t e r p r e t a t i o n , a n d s c i e n c e d e m a n d s o b j e c t i v e s t a t i s t i c a l m e a s u r e s .
( i i ) T h e u s e o f f r e q u e n c y m e a s u r e s ( a n d p r o c e d u r e s b a s e d o n t h e m ) i s
r e a s o n a b l y s o u n d a n d s a f e f o r n o n s p e c i a l i s t s .
( i i i ) O n e n e e d s " r e p e a t a b l e " e x p e r i m e n t s i n s c i e n c e , i . e . , a n y e v i d e n c e
g a t h e r e d a b o u t s h o u l d a l s o b e l i k e l y t o b e f o u n d i f t h e e x p e r i m e n t is
r e p e a t e d ; t h i s w i l l s u p p o s e d l y b e t r u e i f f r e q u e n c y m e a s u r e s o f e v i d e n c e a r e
u s e d .
( i v ) T h e f o l l o w i n g p r i n c i p l e s h o u l d b e f o l l o w e d :
C O N F I D E N C E P R I N C I P L E . A n y s t a t i s t i c i a n w h o u s e s a m e t h o d o l o g y i n w h i c h h e m a k e s
C O N S E Q U E N C E S A N D C R I T I C I S M S O F T H E L P A N D R L P 6 7
o n , d U i a w c o n c Z u A l o n A W A k pec^cec a c c u A a c y , s h o u l d b e g u u v i a n t z z d
t h a t I n t h z l o n g n a n h a c X u a a c c u A a c y v o J U L L b e o u t l o . c u > t t k a Z p n . o n U & d .
W e w i l l b r i e f l y e x a m i n e t h e s e f o u r j u s t i f i c a t i o n s .
4 . 1 . 2 O b j e c t i v i t y
It s h o u l d b e o b s e r v e d , f i r s t o f a l l , t h a t t h e L P is e n t i r e l y o b j e c
t i v e , s t a t i n g o n l y t h a t t h e e v i d e n c e a b o u t is c o n t a i n e d in t h e l i k e l i h o o d
f u n c t i o n . A l s o , t h e l i k e l i h o o d f u n c t i o n h a s a s m u c h p h y s i c a l r e a l i t y a s a n y
f r e q u e n c y m e a s u r e c a l c u l a t e d f o r a p r e s u m e d m o d e l . It w o u l d t h u s b e l o g i c a l l y
s o u n d t o p a s s o n t o t h e n e x t i s s u e . W e d a l l y , h o w e v e r , b e c a u s e o f t h e p r o b l e m
o f u s i n g t h e l i k e l i h o o d f u n c t i o n . I n d e e d , s i n c e in C h a p t e r 5 w e w i l l a r g u e f o r
B a y e s i a n u s e o f t h e l i k e l i h o o d f u n c t i o n , i s s u e s o f o b j e c t i v i t y w i l l b e c o m e r e l e v a n t .
T h e B a y e s i a n a n s w e r s t o c r i t i c i s m s o f o b j e c t i v i t y a r e e i t h e r ( i )
o b j e c t i v i t y i s a m y t h , o r ( i i ) o n l y t h r o u g h " n o n i n f o r m a t i v e " p r i o r B a y e s i a n
a n a l y s i s c a n o b j e c t i v i t y b e r e a l l y a t t a i n e d . A s a n e x a m p l e o f t h e f i r s t
a r g u m e n t , B o x ( 1 9 8 0 ) s t a t e s :
"In t h e p a s t , t h e n e e d f o r p r o b a b i l i t i e s
e x p r e s s i n g p r i o r b e l i e f h a s o f t e n b e e n t h o u g h t
o f , n o t a s a n e c e s s i t y f o r a l l s c i e n t i f i c
i n f e r e n c e , b u t r a t h e r a s a f e a t u r e p e c u l i a r t o
B a y e s i a n i n f e r e n c e . T h i s s e e m s t o c o m e f r o m
t h e c u r i o u s i d e a t h a t a n o u t r i g h t a s s u m p t i o n
d o e s n o t c o u n t a s a p r i o r b e l i e f . . . I b e l i e v e
t h a t it i s i m p o s s i b l e l o g i c a l l y t o d i s t i n g u i s h
b e t w e e n m o d e l a s s u m p t i o n s a n d t h e p r i o r
d i s t r i b u t i o n o f t h e p a r a m e t e r s . "
A g e n e r a l r e v i e w o f t h i s o b j e c t i v i t y i s s u e i s g i v e n in B e r g e r a n d B e r r y ( 1 9 8 7 ) .
( S e e a l s o B e r g e r ( 1 9 8 5 ) . ) T h e o n l y p o r t i o n o f f r e q u e n t i s t t h e o r y f o r m a l l y
e x e m p t f r o m t h e a r g u m e n t i s ( c o m p l e t e l y ) n o n p a r a m e t r i c a n a l y s i s , a n d , e v e n
t h e n , t h e c h o i c e o f a p a r t i c u l a r p r o c e d u r e t o u s e c a n b e a r g u e d t o b e a
h i g h l y s u b j e c t i v e i n p u t .
I f t h e m o d e l c a n b e c l a i m e d t o h a v e s o m e o b j e c t i v e s t a t u s , t h e r e
is s t i l l a r g u m e n t ( i i ) ( a b o v e ) t o c o n t e n d w i t h . T h e i d e a b e h i n d t h i s a r g u m e n t
is t h a t o n e c a n l a y c l a i m t o o b j e c t i v i t y o n l y b y p u r p o s e l y s t r i v i n g f o r i t ,
t h r o u g h u s e o f w h a t is d e e m e d t o b e a n " o b j e c t i v e p r i o r . " S u b s t a n t i a l
s u p p o r t f o r t h i s p o s i t i o n c a n b e f o u n d in J e f f r e y s ( 1 9 6 1 ) , B o x a n d T i a o ( 1 9 7 3 ) ,
Z e l l n e r ( 1 9 7 1 ) , R o s e n k r a n z ( 1 9 7 7 ) , B e r n a r d o ( 1 9 7 9 ) , B e r g e r ( 1 9 8 0 , 1 9 8 4 e ) , a n d
J a y n e s ( 1 9 8 1 , 1 9 8 2 ) . R e g a r d l e s s o f t h e v a l i d i t y o f a r g u m e n t ( i i ) , it is a
f a c t t h a t u s e o f n o n i n f o r m a t i v e p r i o r s is o b j e c t i v e , p u r p o s e l y n o t i n v o l v i n g
s u b j e c t i v e p r i o r o p i n i o n s , a n d is c o n s i s t e n t w i t h t h e L P . T h e m e a s u r e s o f
e v i d e n c e u s e d a r e , o f c o u r s e , p r o b a b i l i s t i c s t a t e m e n t s a b o u t t h e u n k n o w n
i t s e l f ( t h r o u g h t h e f o r m a l p o s t e r i o r d i s t r i b u t i o n o f ) a n d h e n c e m a y b e
d e e m e d l e s s " r e a l " , b u t a v e r y s t r o n g c a s e c a n b e m a d e t h a t " e v i d e n c e " a b o u t
u n c e r t a i n q u a n t i t i e s s h o u l d o n l y b e q u a n t i f i e d p r o b a b i l i s t i c a l l y ( c . f .
d e F i n e t t i ( 1 9 7 2 , 1 9 7 4 ) ) . T h e r e a r e a l s o o t h e r l i k e l i h o o d b a s e d m e t h o d s w h i c h
c a n b e c l a s s i f i e d a s o b j e c t i v e , a s w i l l b e s e e n i n C h a p t e r 5 . H e n c e , e v e n i f
d e e m e d o b t a i n a b l e a n d d e s i r a b l e , o b j e c t i v i t y is n o t a r e a s o n t o r e j e c t t h e L P
i n f a v o r o f f r e q u e n c y m e a s u r e s .
4 . 1 . 3 P r o c e d u r e s f o r N o n s p e c i a l i s t s
W e a c c e p t t h e a r g u m e n t t h a t it is i m p o r t a n t t o d e v e l o p r e a s o n a b l y
s i m p l e s t a t i s t i c a l p r o c e d u r e s w h i c h c a n b e s a f e l y u s e d b y n o n s p e c i a l i s t s .
H o w e v e r , i t is n o t a t a l l c l e a r t h a t t h i s n e e d b e d o n e f r o m a f r e q u e n c y
v i e w p o i n t . F i r s t , f r e q u e n c y m e t h o d s o f t e n a t t a i n f o r m a l s i m p l i c i t y b y
o b s c u r i n g d i f f i c u l t i s s u e s , s u c h a s t h e c h o i c e o f e r r o r p r o b a b i l i t i e s in a
t e s t o r t h e c h o i c e o f a p a r t i t i o n in a c o n d i t i o n a l c o n f i d e n c e p r o c e d u r e
( s e e S e c t i o n 2 . 5 ) . S e c o n d , r e l a t i v e l y s i m p l e p r o c e d u r e s a n d m e t h o d s o f
e v a l u a t i o n c o n s i s t e n t w i t h t h e L P c a n b e d e v e l o p e d ( w / o t h e i n t r o d u c t i o n o f
s u b j e c t i v e p r i o r s ) a s t h e b o o k s o f J e f f r e y s ( 1 9 6 1 ) , B o x a n d T i a o ( 1 9 7 3 ) , a n d
Z e l l n e r ( 1 9 7 1 ) i n d i c a t e . W e a r e c o n t i n u a l l y s u r p r i s e d a t t h e e a s e w i t h w h i c h
t h e u s e o f n o n i n f o r m a t i v e p r i o r s , a s in t h e s e b o o k s , g i v e s e x c e l l e n t
( c o n d i t i o n a l ) p r o c e d u r e s . I n d e e d , a s m e n t i o n e d e a r l i e r , m a n y r e a s o n a b l e
f r e q u e n t s t p r o c e d u r e s a r e , a t l e a s t a p p r o x i m a t e l y , n o n i n f o r m a t i v e p r i o r B a y e s
p r o c e d u r e s , a n d " f r e q u e n c y c o n f i d e n c e " t h e n o f t e n c o i n c i d e s w i t h " p o s t e r i o r
c o n f i d e n c e . " W h e n t h i s c o r r e s p o n d e n c e d o e s n o t o c c u r , s u c h a s i n u n c o n d i t i o n a l
f r e q u e n t i s t a p p r o a c h e s t o t h e e x a m p l e s in S e c t i o n 2 . 1 , t h e f r e q u e n t i s t a p p r o a c h
i s d e f i n i t e l y s u s p e c t . F u r t h e r d i s c u s s i o n a n d r e f e r e n c e s c a n b e f o u n d i n
B e r g e r ( 1 9 8 0 ) . N o t e t h a t w e a r e n o t m a i n t a i n i n g t h a t t h e u s e o f n o n i n f o r m a t i v e
p r i o r s s o l v e s a l l p r o b l e m s a n d is f o o l p r o o f , b u t o n l y t h a t , i f p r o c e d u r e s
w h i c h a r e s i m p l e t o u s e a n d i n t e r p r e t a r e d e e m e d n e c e s s a r y , t h e n t h e r e a r e
g o o d c o n d i t i o n a l a l t e r n a t i v e s t o f r e q u e n t i s t d e v e l o p m e n t o f p r o c e d u r e s . W e
h a v e a l s o s l i g h t e d t h e s u b j e c t i v e B a y e s s o l u t i o n t o t h e p r o b l e m , w h i c h w i l l ,
h o w e v e r , b e d i s c u s s e d in C h a p t e r 5 .
In t h i s s i t u a t i o n , w h e r e a p r o c e d u r e is d e v e l o p e d f o r u s e b y
n o n s p e c i a l i s t s , t h e p e r f o r m a n c e o f t h e p r o c e d u r e in r e p e a t e d u s e is c e r t a i n l y
r e l e v a n t ( s e e S e c t i o n 3 . 5 . 4 ) , t h o u g h n o t n e c e s s a r i l y o f p r i m a r y i m p o r t a n c e .
G o o d f r e q u e n c y p e r f o r m a n c e c a n e v e n b e o f i n t e r e s t t o t h e s t r i c t c o n d i t i o n a l i s t ,
a s t h e f o l l o w i n g e x a m p l e i n d i c a t e s .
E X A M P L E 1 6 . S u p p o s e a c o n f i d e n c e p r o c e d u r e C ( x ) is t o b e u s e d ( i . e . , w h e n
X = x is o b s e r v e d it w i l l b e s t a t e d t h a t C ( x ) ) , h a v i n g f r e q u e n t i s t
c o v e r a g e p r o b a b i l i t y
r ( ) = P
( C ( X ) c o n t a i n s ) >_ 1 .
A c ondi t i ona l Bayesian ( f o r s i mp l i c i t y ) would, f o r a p r i o r d i s t r i b u t i o n on ,

be i nt e r e st e d i n having good post e r i or p r o b a b i l i t y t ha t i s i n C( x ) , i . e . ,
would want
( x) P
(
l
x )
( C( x) )
t o b e l a r g e , w h e r e ( | x ) is t h e p o s t e r i o r p r o b a b i l i t y d i s t r i b u t i o n o f
g i v e n x . B u t , l e t t i n g m d e n o t e t h e m a r g i n a l d i s t r i b u t i o n o f X ( i . e . ,
m ( ) = E
P ()) a n d I
( y ) d e n o t e t h e u s u a l i n d i c a t o r f u n c t i o n o n a s e t B , it
is c l e a r t h a t
E
m
( X ) = E V
(
l
X )
( C ( X ) )
= E
j o i n t d i s t b n . (
'
) [ I
c ( )
= E
P
n
( C ( X ) c o n t a i n s )
S i n c e t h i s r e l a t i o n s h i p h o l d s r e g a r d l e s s o f , a c o n d i t i o n a l i s t c o u l d f e e l
t h a t ( x ) is " l i k e l y
1 1
t o b e l a r g e i f C ( x ) is u s e d a n d is s m a l l , a n d h e n c e b e
w i l l i n g t o u s e C ( x ) w h e n u n a b l e t o c a r r y o u t a t r u s t w o r t h y B a y e s i a n a n a l y s i s .
S e e P r a t t ( 1 9 6 5 ) a n d B e r g e r ( 1 9 8 4 b ) f o r m o r e g e n e r a l d e v e l o p m e n t a n d s p e c i f i c
e x a m p l e s .
It is i m p o r t a n t t o e m p h a s i z e t h a t t h e p r i m a r y g o a l in s i t u a t i o n s
s u c h as E x a m p l e 1 6 s h o u l d s t i l l b e g o o d c o n d i t i o n a l p e r f o r m a n c e , a n d t h a t t h e
f r e q u e n t i s t m e a s u r e d o e s n o t g u a r a n t e e t h i s . C o n c e i v a b l y , ( x ) c o u l d b e v e r y
s m a l l f o r s o m e x ( a n d a l l m ) , w h i c h is c e r t a i n l y r e l e v a n t s i n c e s u c h x c o u l d b e
o b s e r v e d . T h u s o u r v i e w i s t h a t p r o c e d u r e s s h o u l d u s u a l l y b e d e v e l o p e d f r o m a
c o n d i t i o n a l v i e w p o i n t , a n d t h e i r f r e q u e n c y p r o p e r t i e s p e r h a p s i n v e s t i g a t e d t o
e n s u r e r o b u s t n e s s . O f c o u r s e t h e a l r e a d y e x i s t i n g c l a s s i c a l p r o c e d u r e s w h i c h
h a v e g o o d c o n d i t i o n a l p r o p e r t i e s a r e f i n e . O t h e r d i s c u s s i o n s o f t h i s p o i n t c a n
b e f o u n d in G o d a m b e a n d T h o m p s o n ( 1 9 7 7 ) , G o d a m b e ( 1 9 8 2 a , b ) , a n d B e r g e r ( 1 9 8 4 e ) .
4 . 1 . 4 R e p e a t a b i l i t y
T h e r e is c e r t a i n l y t r u t h t o t h e o b s e r v a t i o n t h a t , i f a s c i e n t i f i c
e x p e r i m e n t c l a i m s t o h a v e o b t a i n e d s t r o n g e v i d e n c e a b o u t , t h e n m a n y
s c i e n t i s t s e x p e c t f u t u r e s i m i l a r e x p e r i m e n t s t o a l s o p r o v i d e s t r o n g e v i d e n c e .
T h e f r e q u e n c y m e a s u r e s , b a s e d o n i m a g i n i n g r e p e t i t i o n s o f t h e e x p e r i m e n t ,
s e e m i d e a l l y s u i t e d t o a c h i e v e t h i s . T h e r e is a s e r i o u s c o n c e r n h e r e , h o w e v e r ,
a s t h e f o l l o w i n g e x a m p l e i n d i c a t e s .
E X A M P L E 1 7 . S u p p o s e X h a s t h e t w o p o i n t d i s t r i b u t i o n g i v e n b y P
( X = 0 ) = .9 9
a n d P
Q
( X = ) = . 0 1 . ( E i t h e r w i l l b e m e a s u r e d e x a c t l y , o r n o o b s e r v a t i o n
w i l l b e r e c o r d e d . ) I f n o w x = 5 is o b s e r v e d , i t s h o u l d c e r t a i n l y b e c o n c l u d e d
C O N S E Q U E N C E S A N D C R I T I C I S M S O F T H E LP A N D R L P 71
t h a t = 5 e x a c t l y ( v e r y s t r o n g " e v i d e n c e " ) , b u t r e p e t i t i o n s o f t h e e x p e r i m e n t
a r e v e r y u n l i k e l y t o r e p r o d u c e t h e r e s u l t .
It c o u l d p e r h a p s b e a r g u e d t h a t s c i e n c e s h o u l d n o t b e l i e v e " l u c k y "
o b s e r v a t i o n s , a s in t h e p r e v i o u s e x a m p l e , a n d h e n c e s h o u l d n o t t h i n k
c o n d i t i o n a l l y o n t h e d a t a . T h i s s e e m s t o o s e v e r e a s t r a i g h t j a c k e t , h o w e v e r .
O n e c a n a l w a y s b e s k e p t i c a l o f l u c k y o b s e r v a t i o n s a n d s e e k p o s s i b l e a l t e r n a t i v e
r e a s o n s f o r t h e m , b u t t h e i r c o n d i t i o n a l e v i d e n t i a l i n t e r p r e t a t i o n s h o u l d b e
a l l o w e d . S u c h c o n d i t i o n a l i n t e r p r e t a t i o n s c a n , o f c o u r s e , a l s o b e v e r i f i e d o r
d i s p r o v e d b y f u t u r e i n v e s t i g a t i o n s .
4 . 1 . 5 T h e C o n f i d e n c e P r i n c i p l e
T h e C o n f i d e n c e P r i n c i p l e w a s i m p l i c i t i n m u c h o f N e y m a n 's e a r l y
d e v e l o p m e n t o f t h e f r e q u e n t i s t v i e w p o i n t ( c . f . N e y m a n ( 1 9 6 7 ) a n d a l s o N e y m a n
( 1 9 5 7 , 1 9 7 7 ) a n d B e r g e r ( 1 9 8 4 c ) ) , a n d w a s s t a t e d e x p l i c i t l y b y B i r n b a u m ( c . f .
G i e r e ( 1 9 7 7 ) a n d B i r n b a u m ( 1 9 6 8 , 1 9 7 0 , 1 9 7 7 ) ) , w h o u l t i m a t e l y c a m e t o r e j e c t
t h e LP b e c a u s e o f i t s c o n f l i c t w i t h t h e C o n f i d e n c e P r i n c i p l e . O t h e r d i s c u s s i o n s
o f t h i s o r r e l a t e d p r i n c i p l e s c a n b e f o u n d in C o x a n d H i n k l e y ( 1 9 7 4 ) ( w h i c h
d i s t i n g u i s h e s b e t w e e n s t r o n g a n d w e a k v e r s i o n s , t h e w e a k v e r s i o n a l l o w i n g
c o n d i t i o n i n g o n r e l e v a n t s u b s e t s ) , K i e f e r ( 1 9 7 7 b ) , L e C a m ( 1 9 7 7 ) , a n d B a r n a r d
a n d G o d a m b e ( 1 9 8 2 ) . C r i t i c a l d i s c u s s i o n c a n b e f o u n d in J e f f r e y s ( 1 9 6 1 ) ,
H a c k i n g ( 1 9 6 5 ) , E d w a r d s ( 1 9 7 2 ) , d e F i n e t t i ( 1 9 7 2 , 1 9 7 4 ) , P r a t t ( 1 9 7 7 ) , a n d
J a y n e s ( 1 9 8 1 , 1 9 8 2 ) . T h e f o l l o w i n g m a t h e m a t i c a l f o r m u l a t i o n o f t h e
C o n f i d e n c e P r i n c i p l e w i l l b e u s e f u l in t h e d i s c u s s i o n , a n d is r e l a t e d t o t h e
E v a l u a t i o n G a m e in S e c t i o n 3 . 7 . 2 .
T H E F O R M A L C O N F I D E N C E P R I N C I P L E . A p r o c e d u r e 6 i s t o b e u s e d f o r a s e q u e n c e o f
p r o b l e m s c o n s i s t i n g o f o b s e r v i n g X. ^ P . A c r i t e r i o n , L ( . , ( x . ) ) , m e a s u r e s
t h e p e r f o r m a n c e o f 6 i n e a c h p r o b l e m ( s m a l l L b e i n g g o o d ) . O n e s h o u l d r e p o r t ,
a s t h e " c o n f i d e n c e " i n u s e o f 6
1
n
( 4 . 1 . 1 ) R ( ) = s u p T i m ^ I L ( . , ( x . ) ) ,
n * i = l
a s s u m i n g t h e l i m i t e x i s t s w i t h p r o b a b i l i t y o n e . ( I t c a n u s u a l l y b e s h o w n t h a t
R ( ) = s u p R ( , ) , w h e r e R ( , ) = E A ( , ( X ) ) . )
E X A M P L E 1 8 . S u p p o s e 6 is a c o n f i d e n c e p r o c e d u r e , s o t h a t ( x . ) c @ w i l l b e t h e
c o n f i d e n e
( x ) i s
c o n f i d e n c e s e t w h e n x
i
is o b s e r v e d . T h e n a t u r a l m e a s u r e o f t h e p e r f o r m a n c e o f
s i n c e t h i s m e a s u r e s , w h e t h e r o r n o t ( x . ) d o e s c o n t a i n . . T h e r i s k o f 6 i s
R ( , ) = E L ( e . ( X ) ) = l P
Q
( ( X ) c o n t a i n s ) ,

and i t i s e asy t o show, f o r t h i s p r ob l e m, t h a t
R ( ) = sup R ( e > ) = 1 i n f P
( ( X ) c o nt a i ns ) .

H e n c e t h e " r e p o r t , " a c c o r d i n g t o t h e C o n f i d e n c e P r i n c i p l e , s h o u l d b e o n e m i n u s
t h e m i n i m u m c o v e r a g e p r o b a b i l i t y o f .
A l t h o u g h t h e C o n f i d e n c e P r i n c i p l e is f o r m u l a t e d a b o v e o n l y i n t e r m s
o f r e p e t i t i v e u s e o f f o r p r o b l e m s o f t h e s a m e f o r m ( b u t p o s s i b l y d i f f e r i n g
) , it c a n e a s i l y b e g e n e r a l i z e d t o i n c l u d e u s e o f f o r d i f f e r e n t t y p e s o f
p r o b l e m s . S u c h a g e n e r a l i z a t i o n a d d s l i t t l e c o n c e p t u a l l y , h o w e v e r . T h e a p p e a l
o f t h e C o n f i d e n c e P r i n c i p l e i s u n d e n i a b l e . B y f o l l o w i n g i t , t h e a c t u a l a v e r a g e
p e r f o r m a n c e o f i n r e p e a t e d u s e w i l l b e a t l e a s t a s g o o d a s t h e r e p o r t e d
p e r f o r m a n c e R ( ) . T h e r e a r e s e v e r a l p r o b l e m s in f o l l o w i n g t h e C o n f i d e n c e
P r i n c i p l e , h o w e v e r .
T h e f i r s t d i f f i c u l t y is t h a t , in v i r t u a l l y a l l s t a t i s t i c a l i n v e s t i
g a t i o n s , e x t e n s i v e a s s u m p t i o n s c o n c e r n i n g t h e m o d e l , e t c . , a r e m a d e . T h u s a
p e r s o n c l a i m i n g t o e r r n o m o r e t h a n 5% o f t h e t i m e b e c a u s e h e f o l l o w s t h e
C o n f i d e n c e P r i n c i p l e , is r e a l l y s a y i n g h e e r r s n o m o r e t h a n 5 % o f t h e t i m e i f
a l l t h e m o d e l a s s u m p t i o n s h e m a k e s a r e c o r r e c t . T h i s r e m o v e s s o m e o f t h e
l u s t r e f r o m t h e p r i n c i p l e .
A s e c o n d s e r i o u s i s s u e is t h e n e e d t o h a v e a v a l i d b o u n d , R ( ) , o n
t h e p e r f o r m a n c e o f . T h i s is a n o f t e n u n a p p r e c i a t e d a s p e c t o f t h e f r e q u e n
t i s t p o s i t i o n . I n d e e d , t h e f r e q u e n t i s t p o s i t i o n is o f t e n v i e w e d a s r e q u i r i n g
o n l y t h e r e p o r t i n g o f t h e f u n c t i o n R ( , ) . W i t h o u t t h e b o u n d , R ( ) , h o w e v e r ,
n o g u a r a n t e e o f l o n g r u n p e r f o r m a n c e , in a c t u a l u s e o f o n d i f f e r e n t p r o b l e m s ,
c a n b e g i v e n .
E X A M P L E 1 9 . C o n s i d e r s i m p l e v e r s u s s i m p l e h y p o t h e s i s t e s t i n g , a n d s u p p o s e o n e
a l w a y s u s e s t h e m o s t p o w e r f u l t e s t o f l e v e l

= . 0 1 . O n e c a n m a k e t h e
f r e q u e n t i s t s t a t e m e n t t h a t o n l y 1% o f t r u e n u l l h y p o t h e s e s w i l l b e r e j e c t e d
( i . e . , R ( , ) = .01 f o r e q u a l t o t h e n u l l ) , b u t t h i s s a y s n o t h i n g a b o u t h o w
o f t e n o n e e r r s w h e n r e j e c t i n g . F o r i n s t a n c e , i f t h e t e s t h a s p o w e r o f .01
( a d m i t t e d l y t e r r i b l e p o w e r , b u t u s e f u l f o r m a k i n g t h e p o i n t ) a n d t h e n u l l a n d
a l t e r n a t i v e h y p o t h e s e s o c c u r e q u a l l y o f t e n in r e p e t i t i v e u s e o f t h e t e s t , t h e n
h a l f o f a l l r e j e c t i o n s w i l l b e in e r r o r . T h u s o n e c a n n o t m a k e m e a n i n g f u l
s t a t e m e n t s a b o u t a c t u a l e r r o r i n c u r r e d i n r e p e t i t i v e u s e , w i t h o u t a n a p p r o p r i a t e
b o u n d o n R (
s
) f o r a l l .
T h e p r o b l e m w i t h n e e d i n g R ( ) i s , o f c o u r s e , t h a t it c o u l d b e a
u s e l e s s b o u n d ( o r c o u l d e v e n b e i n f i n i t e ) . I n d e e d , w h e n e v e r R ( , ) is h i g h l y
v a r i a b l e a s a f u n c t i o n o f
>
t h e r e p o r t i n g o f R ( ) is l i k e l y t o b e e x c e s s i v e l y
c o n s e r v a t i v e . T h e c o n d i t i o n a l f r e q u e n t i s t a p p r o a c h e s d i s c u s s e d in S e c t i o n 2 . 4
h a v e c o n s i d e r a b l e p r o m i s e in o v e r c o m i n g t h i s d i f f i c u l t y , h o w e v e r , a n d c a n b e
g i v e n i n t e r p r e t a t i o n s c o m p a t i b l e w i t h t h e C o n f i d e n c e P r i n c i p l e .
U l t i m a t e l y , t h e o n l y c l e a r o b j e c t i o n t o t h e C o n f i d e n c e P r i n c i p l e is
t h a t it c o n f l i c t s w i t h t h e L P . T h i s w a s i n d i c a t e d in t h e e x a m p l e s a n d d i s c u s
s i o n in C h a p t e r 2 , a n d w i l l b e s e e n i n l a t e r e x a m p l e s a l s o . M o s t c o n d i t i o n
a l i s t s v i e w t h e C o n f i d e n c e P r i n c i p l e , w h i l e a t t r a c t i v e , a s a n u n a t t a i n a b l e
g o a l . ( N o t e , h o w e v e r , t h a t a B a y e s i a n c o n d i t i o n a l i s t f o l l o w s t h e C o n f i d e n c e
P r i n c i p l e t o t h e e x t e n t t h a t h i s s t a t e m e n t s o f a c c u r a c y w i l l b e c o r r e c t , in
t h e l o n g r u n a v e r a g e s e n s e , i f h i s p r i o r a s s u m p t i o n s a r e c o r r e c t ; o n e c o u l d ,
i n d e e d , a r g u e t h a t i t is t h e B a y e s i a n w h o is h o n e s t l y t r y i n g t o f o l l o w t h e
C o n f i d e n c e P r i n c i p l e b y c l e a r l y s t a t i n g t h e b e l i e f s a n d a s s u m p t i o n s h i s
a s s e s s m e n t s a r e b a s e d o n . ) I n c h o o s i n g b e t w e e n t h e L P a n d t h e C o n f i d e n c e
P r i n c i p l e , it is i m p o r t a n t t o r e c a l l t h e s i m p l e a x i o m a t i c b a s i s o f t h e L P , a n d
t o r e a l i z e t h a t n o s u c h b a s i s h a s b e e n f o u n d f o r t h e C o n f i d e n c e P r i n c i p l e .
I n d e e d , t h e l o n g r u n p e r f o r m a n c e v i e w is d e e m e d r a t h e r p e c u l i a r b y m o s t
u n i n i t i a t e d p e o p l e ( c . f . , t h e d i s c u s s i o n s in t h e e a r l y p a p e r s o f N e y m a n in
N e y m a n ( 1 9 6 7 ) ) .
4 . 2 T H E I R R E L E V A N C E O F S T O P P I N G R U L E S
4 . 2 . 1 I n t r o d u c t i o n
O n e o f t h e m o s t i m p o r t a n t a p p l i c a t i o n s o f t h e L P a n d R L P i s t h e
S t o p p i n g R u l e P r i n c i p l e ( S R P ) . S t a t e d i n f o r m a l l y , t h e S R P is s i m p l y t h a t t h e
r e a s o n f o r s t o p p i n g e x p e r i m e n t a t i o n ( t h e s t o p p i n g r u l e ) s h o u l d b e i r r e l e v a n t t o
e v i d e n t i a r y c o n c l u s i o n s a b o u t . T h e t h e o r e t i c a l a n d p r a c t i c a l i m p l i c a t i o n s o f
t h e S R P t o s u c h f i e l d s a s s e q u e n t i a l a n a l y s i s a n d c l i n i c a l t r i a l s a r e
e n o r m o u s , a n d w i l l b e p a r t i a l l y d i s c u s s e d i n S e c t i o n s 4 . 2 . 3 a n d 4 . 2 . 4 . T h e
S R P i t s e l f w i l l b e d i s c u s s e d a t t w o l e v e l s : in S e c t i o n 4 . 2 . 2 it w i l l b e
p r e s e n t e d i n a r e l a t i v e l y s i m p l e s e q u e n t i a l s e t t i n g , in w h i c h it w i l l b e s h o w n
t o f o l l o w s o l e l y f r o m t h e L P , w h i l e i n S e c t i o n 4 . 2 . 6 a y e r y g e n e r a l v e r s i o n
w i l l b e d e v e l o p e d f r o m t h e R L P . S e c t i o n 4 . 2 . 7 d i s c u s s e s s i t u a t i o n s in w h i c h
t h e S R P is n o t a p p l i c a b l e , a n d S e c t i o n 4 . 2 . 5 p o i n t s o u t a n i n t e r e s t i n g c o n f l i c t
b e t w e e n f r e q u e n t i s t a d m i s s i b i l i t y a n d t h e f r e q u e n t i s t b e l i e f i n t h e i m p o r t a n c e
o f c o n s i d e r i n g s t o p p i n g r u l e s .
T h e S t o p p i n g R u l e P r i n c i p l e w a s f i r s t e s p o u s e d b y B a r n a r d
( 1 9 4 7 a , 1 9 4 9 ) , w h o s e m o t i v a t i o n a t t h e t i m e w a s e s s e n t i a l l y a r e l u c t a n c e t o
a l l o w a n e x p e r i m e n t e r ' s i n t e n t i o n s t o a f f e c t c o n c l u s i o n s d r a w n f r o m d a t a .
( M o r e w i l l b e s a i d o f t h i s s h o r t l y . ) T h e p r i n c i p l e w a s s h o w n t o b e a c o n s e
q u e n c e o f t h e L P in B i r n b a u m ( 1 9 6 2 a ) , a n d B a r n a r d , J e n k i n s a n d W i n s t e n ( 1 9 6 2 ) ,
a n d a r g u e d t o h o l d in e s s e n t i a l l y c o m p l e t e g e n e r a l i t y b y P r a t t ( 1 9 6 5 ) . O t h e r
g o o d d i s c u s s i o n s o f t h e p r i n c i p l e c a n b e f o u n d in A n s c o m b e ( 1 9 6 3 ) , C o r n f i e l d
( 1 9 6 6 ) , B a r t h o l o m e w ( 1 9 6 7 ) , B a s u ( 1 9 7 5 ) , B e r g e r ( 1 9 8 0 ) , a n d in m a n y B a y e s i a n
w o r k s s u c h a s E d w a r d s , L i n d m a n , a n d S a v a g e ( 1 9 6 3 ) .
C O N S E Q U E N C E S A N D C R I T I C I S M S O F T H E L P A N D R L P 7 4 .1
B e f o r e f o r m a l l y i n t r o d u c i n g s t o p p i n g r u l e s a n d t h e s t o p p i n g r u l e
p r i n c i p l e , it i s u s e f u l t o i l l u s t r a t e c e r t a i n o f t h e i d e a s t h r o u g h a s i m p l e
e x a m p l e . T h e f o l l o w i n g e x a m p l e , f r o m B e r g e r a n d B e r r y ( 1 9 8 7 ) , d e m o n s t r a t e s
t h e p o s s i b l e e x t r e m e d e p e n d e n c e o f f r e q u e n t i s t m e a s u r e s u p o n t h e i n t e n t i o n s
o f t h e e x p e r i m e n t e r c o n c e r n i n g s t o p p i n g t h e e x p e r i m e n t . T h e e x a m p l e c l e a r l y
q u e s t i o n s t h e s e n s i b i l i t y o f s u c h e x t r e m e d e p e n d e n c e . ( B e r g e r a n d B e r r y ,
1 9 8 7 , a l s o c o n t a i n s o t h e r s i m p l e e x a m p l e s , o n b o t h s i d e s o f t h e i s s u e . )
E X A M P L E 1 9 . 1 . A s c i e n t i s t e n t e r s t h e s t a t i s t i c i a n ' s o f f i c e w i t h 1 0 0 o b s e r v a
t i o n s , a s s u m e d t o b e i n d e p e n d e n t a n d f r o m a W ( , l ) d i s t r i b u t i o n . T h e s c i e n t i s t
w a n t s t o t e s t H
Q
: = 0 v e r s u s Hj: f 0 . T h e c u r r e n t a v e r a g e i s x"
n
= 0 . 2 , so
t h e s t a n d a r d i z e d t e s t s t a t i s t i c is z = ^ | x " 0 | = 2 . A c a r e l e s s c l a s s i c a l
s t a t i s t i c i a n m i g h t s i m p l y c o n c l u d e t h a t t h e r e is s i g n i f i c a n t e v i d e n c e a g a i n s t
H
Q
a t t h e 0 . 0 5 l e v e l . B u t a m o r e c a r e f u l o n e w i l l a s k t h e s c i e n t i s t , " W h y d i d
y o u c e a s e e x p e r i m e n t a t i o n a f t e r 1 0 0 o b s e r v a t i o n s ? " If t h e s c i e n t i s t r e p l i e s ,
"I j u s t d e c i d e d t o t a k e a b a t c h o f 1 0 0 o b s e r v a t i o n s , " t h e r e w o u l d s e e m t o b e n o
p r o b l e m , a n d y e r y f e w c l a s s i c a l s t a t i s t i c i a n s w o u l d p u r s u e t h e i s s u e . B u t
t h e r e is a n o t h e r i m p o r t a n t q u e s t i o n t h a t s h o u l d b e a s k e d ( f r o m t h e c l a s s i c a l
p e r s p e c t i v e ) , n a m e l y : " W h a t w o u l d y o u h a v e d o n e h a d t h e f i r s t 1 0 0 o b s e r v a t i o n s
n o t y i e l d e d s i g n i f i c a n c e ? "
T o s e e t h e r e a s o n s f o r t h i s q u e s t i o n , s u p p o s e t h e s c i e n t i s t
r e p l i e s : "I w o u l d t h e n h a v e t a k e n a n o t h e r b a t c h o f 1 0 0 o b s e r v a t i o n s . " T h i s
r e p l y d o e s n o t c o m p l e t e l y s p e c i f y a s t o p p i n g r u l e , b u t t h e s c i e n t i s t m i g h t
a g r e e t h a t h e w a s i m p l i c i t l y c o n s i d e r i n g a p r o c e d u r e o f t h e f o r m :
( a ) t a k e 1 0 0 o b s e r v a t i o n s ;
( b ) i f / T O O ] X Q Q I > k t h e n s t o p a n d r e j e c t H Q ,
( c ) i f /Tt|x",
00
1< k t h e n t a k e a n o t h e r 1 0 0 o b s e r v a t i o n s a n d r e j e c t if
2 0 0
| > k .
F o r t h i s p r o c e d u r e t o h a v e l e v e l = 0 . 0 5 , k m u s t be c h o s e n t o b e 2 . 1 8
( P o c o c k , 1 9 7 7 ) . S i n c e t h e a c t u a l d a t a h a d / T 0 0 " | x "
1 0 0
| = 2 < 2 . 1 8 , t h e s c i e n t i s t
c o u l d n o t a c t u a l l y c o n c l u d e s i g n i f i c a n c e , a n d h e n c e w o u l d h a v e t o t a k e t h e
n e x t T O O o b s e r v a t i o n s .
T h i s s t r i k e s m a n y p e o p l e a s p e c u l i a r . T h e i n t e r p r e t a t i o n o f t h e
r e s u l t s o f an e x p e r i m e n t d e p e n d s n o t o n l y o n t h e d a t a o b t a i n e d a n d t h e w a y it
w a s o b t a i n e d , b u t a l s o u p o n t h o u g h t * o f t h e e x p e r i m e n t e r c o n c e r n i n g p l a n s f o r
t h e f u t u r e .
O f c o u r s e , t h i s c a n b e c a r r i e d f u r t h e r . S u p p o s e t h e p u z z l e d
s c i e n t i s t l e a v e s a n d g e t s t h e n e x t 1 0 0 o b s e r v a t i o n s , a n d b r i n g s t h e m b a c k .
C o n s i d e r t w o c a s e s . I f /2 0 0 1 ><2 OI
= 2 > 1 < 2
^
8 t h e n t I i e r e s
u l t s a r e n o t
s i g n i f i c a n t . B u t t h e y w o u l d h a v e b e e n s i g n i f i c a n t h a d t h e s c i e n t i s t n o t
p a u s e d h a l f w a y t h r o u g h t h e s t u d y t o c a l c u l a t e z ! (It w o u l d c e r t a i n l y b e
t e m p t i n g n o t t o d i s c l o s e t h i s i n t e r i m c a l c u l a t i o n , a n d e s s e n t i a l l y i m p o s s i b l e
t o d e t e r m i n e w h e t h e r o r n o t t h e s c i e n t i s t h a d m a d e an i n t e r i m c a l c u l a t i o n ! )
O n t h e o t h e r h a n d , s u p p o s e / 2 0 0 | X "
2
Q Q | = 2 . 2 > 2 . 1 8 , s o n o w s i g n i f i c a n c e h a s
b e e n o b t a i n e d . B u t w a i t ! A g a i n t h e s t a t i s t i c i a n a s k s w h a t t h e s c i e n t i s t
w o u l d h a v e d o n e h a d t h e r e s u l t s n o t b e e n s i g n i f i c a n t . S u p p o s e t h e
s c i e n t i s t s a y s , " I f m y g r a n t r e n e w a l w e r e t o b e a p p r o v e d , I w o u l d t h e n t a k e
a n o t h e r 1 0 0 o b s e r v a t i o n s ; i f t h e g r a n t r e n e w a l w e r e r e j e c t e d , I w o u l d h a v e
n o m o r e f u n d s a n d w o u l d h a v e t o s t o p t h e e x p e r i m e n t in a n y c a s e . " T h e
a d v i c e o f t h e c l a s s i c a l s t a t i s t i c i a n m u s t t h e n b e : " W e c a n n o t m a k e a
c o n c l u s i o n u n t i l w e f i n d o u t t h e o u t c o m e o f y o u r g r a n t r e n e w a l ; i f it i s
n o t r e n e w e d , y o u c a n c l a i m s i g n i f i c a n t e v i d e n c e a g a i n s t H ~ , w h i l e i f i t i s
r e n e w e d y o u c a n n o t c l a i m s i g n i f i c a n c e a n d m u s t t a k e a n o t h e r 1 0 0 o b s e r v a t i o n s . "
T h e u p t o n o w h o n e s t s c i e n t i s t h a s h a d e n o u g h , a n d h e s e n d s in a r e q u e s t t o
h a v e t h e g r a n t r e n e w a l d e n i e d , v o w i n g n e v e r a g a i n t o t e l l t h e s t a t i s t i c i a n
w h a t h e w o u l d h a v e d o n e u n d e r a l t e r n a t i v e s c e n a r i o s .
N o t e t h a t w e a r e n o t f a u l t i n g t h e c l a s s i c a l s t a t i s t i c i a n h e r e f o r
a s c e r t a i n i n g a n d i n c o r p o r a t i n g t h e s t o p p i n g r u l e in t h e a n a l y s i s . I f o n e i n
s i s t s o n u t i l i z a t i o n o f f r e q u e n t i s t m e a s u r e s , s u c h i n v o l v e m e n t o f t h e s t o p p i n g
r u l e ( e v e n i f it e x i s t s o n l y in t h e i m a g i n a t i o n o f t h e e x p e r i m e n t e r ) is m a n d a
t o r y . T h e n e e d h e r e f o r i n v o l v e m e n t o f t h e s t o p p i n g r u l e c l e a r l y c a l l s t h e
b a s i c f r e q u e n t i s t p r e m i s e i n t o q u e s t i o n , h o w e v e r .
4 . 2 . 2 T h e ( D i s c r e t e ) S t o p p i n g R u l e P r i n c i p l e
S o a s n o t t o o b s c u r e t h e e s s e n t i a l n a t u r e o f t h e S R P , t h e d i s c u s
s i o n in t h i s s e c t i o n w i l l b e r e s t r i c t e d t o t h e f o l l o w i n g f a i r l y s i m p l e
s i t u a t i o n . S u p p o s e E
is a s e q u e n t i a l e x p e r i m e n t c o n s i s t i n g o f ( i ) a
s e q u e n c e o f i n d e p e n d e n t o b s e r v a t i o n s Xj, X,,,..., w h i c h w i l l b e o b s e r v e d o n e
a t a t i m e a n d w h i c h h a v e c o m m o n d e n s i t y f
( x ) ; a n d ( i i ) a n o n r a n d o m i z e d
stopping rule, , which can be r epr esent ed by a sequence of set s,

A c
m
= ... c ( t he m f ol d Car t esi an pr oduct of %) ,
having t he pr oper t y t hat
( 4 . 2 . 1 ) i f x
m
= ( x
. . . , x
m
) A
m
, s a m p l i n g s t o p s ;
m c
i f x A , s a m p l i n g c o n t i n u e s .
S i n c e t h e o b s e r v a t i o n s w i l l b e o b s e r v e d s e q u e n t i a l l y , it is c l e a r l y u n n e c e s s a r y
t o h a v e A c o n t a i n p o i n t s w h o s e f i r s t j c o o r d i n a t e s w e r e i n A . f o r a n y j < m ;
t h u s w e h e n c e f o r t h a s s u m e t h a t
A
m
A . x
m
~
J
' = 0 f o r j < m .
T h e s t o p p i n g t i m e , N , c o r r e s p o n d i n g t o is t h a t ( r a n d o m ) m f o r w h i c h A
m
;
t h e r e a l i z a t i o n o f N w i l l b e d e n o t e d b y n . A s u s u a l , o n l y p r o p e r s t o p p i n g
r u l e s w i l l b e c o n s i d e r e d , i . e . , t h o s e w h i c h h a v e N f i n i t e w i t h p r o b a b i l i t y o n e
f o r a l l . T h e p r o b a b i l i t y d e n s i t y o f t h e r a n d o m e x p e r i m e n t a l o u t c o m e
X
N
X
N
= ( X
r
. . . , X
N
) is t h e n
( 4 . 2 . 2 ) f
( x
n
)
E X A M P L E 2 0 . S u p p o s e t h e X . a r e ^ ( , l ) .
C a s e 1 . C o n s i d e r t h e s t o p p i n g r u l e , , d e f i n e d b y
0 i f m k
A !
i f m = k .
1
T h e e x p e r i m e n t E
is t h u s t h e f i x e d s a m p l e s i z e e x p e r i m e n t w h i c h a l w a y s
o b s e r v e s p r e c i s e l y k o b s e r v a t i o n s .
2
C a s e 2 . C o n s i d e r t h e s t o p p i n g r u l e , , d e f i n e d b y
( 4 . 2 . 3 ) = {
m
e *
m
: | x j > K m * } .
w h e r e x
m
is t h e m e a n o f ( x ^ . . j x j a n d K is a f i x e d p o s i t i v e c o n s t a n t . (By
2
u s i n g t h e L a w o f t h e I t e r a t e d L o g a r i t h m , can b e s h o w n t o b e a p r o p e r s t o p p i n g
r u l e . ) T h i s s t o p p i n g r u l e is r a t h e r p e c u l i a r , in t h a t it s a y s t o s t o p s a m p l i n g
w h e n t h e s a m p l e m e a n is K s t a n d a r d d e v i a t i o n s f r o m z e r o .
E X A M P L E 2 1 . S u p p o s e t h e X
i
a r e B e r n o u l l i ( ) .
1
C a s e 1. L e t E
b e t h e f i x e d s a m p l e s i z e e x p e r i m e n t w h i c h t a k e s k o b s e r v a
t i o n s , w h e r e k <_ 2 .
p
C a s e 2 . L e t b e d e f i n e d b y
A * = { ! } , k \ = { ( 0 , 0 ) , ( 0 , 1 ) } , A ? = 0 f o r j > 2
2
( i . e . , s t o p i f X, = 1 , a n d o t h e r w i s e s t o p a f t e r o b s e r v i n g X ^ ) , a n d let E
b e
t h e c o r r e s p o n d i n g s e q u e n t i a l e x p e r i m e n t .
S T O P P I N G R U L E P R I N C I P L E ( S R P ) : I n a s e q u e n t i a l e x p e r i m e n t E
, w i t h o b s e r v e d
f i n a l d a t a x , E v ( E
, x ) s h o u l d n o t d e p e n d o n t h e s t o p p i n g r u l e .
T h e S R P w o u l d i m p l y , i n E x a m p l e 2 0 , t h a t i f t h e o b s e r v a t i o n in
C a s e 2 h a p p e n e d t o h a v e n = k , t h e n t h e e v i d e n t i a r y c o n t e n t o f t h e d a t a w o u l d
b e t h e s a m e a s i f t h e d a t a h a d a r i s e n f r o m t h e f i x e d s a m p l e s i z e e x p e r i m e n t in
C a s e 1. A s i m i l a r c o n c l u s i o n w o u l d h o l d in E x a m p l e 21 i f n = k o c c u r r e d .
W h e n % is d i s c r e t e , t h e S R P is a n i m m e d i a t e c o n s e q u e n c e o f t h e L P .
n
T h i s is i m m e d i a t e f r o m ( 4 . 2 . 2 ) i n t h a t I
n
( ) is p r o p o r t i o n a l t o f
Q
( x . )
x
1 1
i = l
1
w h i c h d o e s n o t d e p e n d o n t h e s t o p p i n g r u l e . F o r d e r i v a t i o n o f t h e S R P in
g e n e r a l ( f r o m t h e R L P ) s e e S e c t i o n 4 . 2 . 6 .
4 . 2 . 3 P o s i t i v e I m p l i c a t i o n s
A r e c u r r i n g p r o b l e m i n c l a s s i c a l s t a t i s t i c s is t h a t o f o p t i o n a l
s t o p p i n g . I d e a l l y ( f r o m a c l a s s i c a l v i e w p o i n t ) a n e x p e r i m e n t e r c h o o s e s h i s
s t o p p i n g r u l e b e f o r e e x p e r i m e n t a t i o n , a n d t h e n f o l l o w s it e x a c t l y . A c t u a l
p r a c t i c e i s , h o w e v e r , a c k n o w l e d g e d t o b e q u i t e d i f f e r e n t . E x p e r i m e n t s m a y e n d
b e c a u s e t h e d a t a l o o k s c o n v i n c i n g e n o u g h , b e c a u s e m o n e y r u n s o u t , o r b e c a u s e
t h e e x p e r i m e n t e r h a s a d i n n e r d a t e . I n d e e d , l i t t l e o r n o t h o u g h t m a y h a v e b e e n
g i v e n t o t h e s t o p p i n g r u l e p r i o r t o e x p e r i m e n t a t i o n , in w h i c h c a s e , u p o n s t o p
p i n g f o r w h a t e v e r r e a s o n , t h e d a t a is o f t e n t r e a t e d a s h a v i n g a r i s e n f r o m a
f i x e d s a m p l e s i z e d e s i g n . O p t i o n a l s t o p p i n g m a y o f t e n b e h a r m l e s s ( s u c h a s
w h e n t h e e x p e r i m e n t e r q u i t s t o h a v e d i n n e r ) , b u t s t o p p i n g " w h e n t h e d a t a l o o k s
g o o d " c a n b e a s e r i o u s e r r o r w h e n c o m b i n e d w i t h f r e q u e n t i s t m e a s u r e s o f
e v i d e n c e . F o r i n s t a n c e , i f o n e u s e d t h e s t o p p i n g r u l e in C a s e 2 o f E x a m p l e 2 0 ,
b u t a n a l y z e d t h e d a t a a s i f a f i x e d s a m p l e h a d b e e n t a k e n , o n e c o u l d g u a r a n t e e
a r b i t r a r i l y s t r o n g f r e q u e n t i s t " s i g n i f i c a n c e " a g a i n s t H
: = 0 b y m e r e l y
c h o o s i n g l a r g e e n o u g h K .
O p t i o n a l s t o p p i n g p o s e s a s i g n i f i c a n t p r o b l e m f o r c l a s s i c a l
s t a t i s t i c s , e v e n w h e n t h e e x p e r i m e n t e r s a r e e x t r e m e l y s c r u p u l o u s . H o n e s t
f r e q u e n t i s t s f a c e t h e p r o b l e m o f g e t t i n g e x t r e m e l y c o n v i n c i n g d a t a t o o s o o n
( i . e . , b e f o r e t h e i r s t o p p i n g r u l e s a y s t o s t o p ) , a n d t h e n f a c i n g t h e d i l e m m a
o f h o n e s t l y f i n i s h i n g t h e e x p e r i m e n t , e v e n t h o u g h a w a s t e o f t i m e o r d a n g e r o u s
t o s u b j e c t s , o r o f s t o p p i n g t h e e x p e r i m e n t w i t h t h e p r e m a t u r e l y c o n v i n c i n g
e v i d e n c e a n d t h e n n o t b e i n g a b l e t o g i v e f r e q u e n c y m e a s u r e s o f e v i d e n c e . O n e
c o u l d a r g u e t h a t e x p e r i m e n t s s h o u l d b e d e s i g n e d t o a l l o w f o r e a r l y s t o p p i n g in
r e s p o n s e t o c l e a r e v i d e n c e ( a n d , i n d e e d , m a n y s u c h s t o p p i n g r u l e s h a v e b e e n
c r e a t e d , a s in t h e t h e o r y o f " r e p e a t e d s i g n i f i c a n c e t e s t i n g " ) , b u t t h e r e w i l l
o f t e n b e u n f o r e s e e n e v e n t u a l i t i e s t h a t c r o p u p in s e q u e n t i a l e x p e r i m e n t a t i o n ,
l e a v i n g a s t r i c t f r e q u e n t i s t in a n e m b a r a s s i n g p o s i t i o n .
C o n t r a s t t h i s e n o r m o u s d i l e m m a w i t h t h e s t a r t l i n g s i m p l i c i t y
r e s u l t i n g f r o m u s e o f t h e S R P . T h e S R P s a y s t h a t it j u s t d o e s n ' t m a t t e r ; s t o p
f o r w h a t e v e r r e a s o n s , w h i c h ( c o n d i t i o n a l o n t h e d a t a ) d o n o t d e p e n d o n ( s e e
S e c t i o n 4 . 2 . 7 ) , a n d u s e a n a p p r o p r i a t e c o n d i t i o n a l a n a l y s i s b a s e d o n i
n
( )
n *
( o r , a l t e r n a t i v e l y , f
( x . ) ) . T h e r e a s o n f o r s t o p p i n g is s i m p l y n o t r e l e v a n t .
i = l

A s E d w a r d s , L i n d m a n , a n d S a v a g e ( 1 9 6 3 ) s a y
" T h e i r r e l e v a n c e o f s t o p p i n g r u l e s t o
s t a t i s t i c a l i n f e r e n c e r e s t o r e s a s i m p l i
c i t y a n d f r e e d o m t o e x p e r i m e n t a l d e s i g n . . .
M a n y e x p e r i m e n t e r s w o u l d l i k e t o f e e l f r e e
t o c o l l e c t d a t a u n t i l t h e y h a v e e i t h e r
c o n c l u s i v e l y p r o v e d t h e i r p o i n t , c o n c l u s i v e l y
d i s p r o v e d i t , o r r u n o u t o f t i m e , m o n e y , o r
p a t i e n c e . "
A n s c o m b e ( 1 9 6 3 ) s i m p l y m a k e s t h e b l u n t s t a t e m e n t " S e q u e n t i a l a n a l y s i s is a
h o a x . " T h e s e c o m m e n t s s h o u l d b e q u a l i f i e d , o f c o u r s e , t o t h e e x t e n t t h a t
d e s i g n w i l l d e p e n d o n t h e s t o p p i n g r u l e . In o t h e r w o r d s , c h o o s i n g b e t w e e n t w o
s e q u e n t i a l d e s i g n s o b v i o u s l y i n v o l v e s c o n s i d e r a t i o n o f s t o p p i n g r u l e s . I n d e e d ,
t h e m o s t d i f f i c u l t p a r t o f ( t h e o r e t i c a l ) s e q u e n t i a l ( d e c i s i o n ) a n a l y s i s is t h a t
o f d e c i d i n g , a t a g i v e n s t a g e , w h e t h e r t o s t o p s a m p l i n g o r t o t a k e a n o t h e r
o b s e r v a t i o n ( i . e . , c h o o s i n g t h e s t o p p i n g r u l e ) . M u c h o f t h e w o r k d o n e in
c l a s s i c a l s e q u e n t i a l a n a l y s i s h a s a d d r e s s e d t h i s p r o b l e m , a n d i s h e n c e o f
c o n s i d e r a b l e r e l e v a n c e .
T h e o t h e r d e s i r a b l e i m p l i c a t i o n o f t h e S R P is t h a t a n a l y s i s o f a n
e x p e r i m e n t c a n b e d o n e o b j e c t i v e l y , i n t h e s e n s e t h a t i t is n o l o n g e r n e c e s s a r y
t o k n o w t h e e x p e r i m e n t e r ' s i n t e n t i o n s t o w a r d s s t o p p i n g . It s e e m s \ / e r y s t r a n g e
t h a t a f r e q u e n t i s t c o u l d n o t a n a l y z e a g i v e n s e t o f d a t a , s u c h a s ( x , , . . . , x )
i n E x a m p l e 2 0 , i f t h e s t o p p i n g r u l e is n o t g i v e n . I f t h e e x p e r i m e n t e r f o r g o t t o
r e c o r d t h e s t o p p i n g r u l e a n d t h e n d i e d , it is u n a p p e a l i n g t o h a v e t o g u e s s h i s
s t o p p i n g r u l e i n o r d e r t o c o n d u c t t h e a n a l y s i s . A s m e n t i o n e d e a r l i e r , i t w a s
a p p a r e n t l y t h i s f e e l i n g , t h a t d a t a s h o u l d b e a b l e t o s p e a k f o r i t s e l f , t h a t led
B a r n a r d t o f i r s t s u p p o r t t h e S t o p p i n g R u l e P r i n c i p l e .
C O N S E Q U E N C E S A N D C R I T I C I S M S O F T H E L P A N D R L P 79
T h e a b o v e i d e a is a c t u a l l y a g e n e r a l c o n s e q u e n c e o f t h e L P , a n d is
u s e f u l t o a p p l y i n a r e a s o t h e r t h a n o p t i o n a l s t o p p i n g . C o n s i d e r t h e f o l l o w i n g
e x a m p l e .
E X A M P L E 2 2 . A n e x p e r i m e n t w a s c o n d u c t e d w i t h t w o t r e a t m e n t g r o u p s (T, a n d T )
a n d a c o n t r o l g r o u p ( C ) , t h e o u t c o m e s f o r e a c h e x p e r i m e n t a l u n i t b e i n g s i m p l y
s u c c e s s ( S ) o r f a i l u r e ( F ) . T h e d a t a w a s
s
F
C
8
1 2
T
l
1 2
8
T
2
2
8
In a n a l y z i n g t h e r e s u l t s , t h e e x p e r i m e n t e r n o t e d t h a t , in c o m p a r i n g T , w i t h C ,
a s t a n d a r d a n a l y s i s u n d e r t h e n u l l h y p o t h e s i s o f n o t r e a t m e n t e f f e c t w a s n o t
s i g n i f i c a n t at l e v e l = .1 ( o n e t a i l e d ) , b u t t h a t i f t h e p a t i e n t s i n T a n d C
w e r e p o o l e d , t h e n T , w a s s i g n i f i c a n t l y b e t t e r a t t h e = . 0 2 l e v e l . T h e
e x p e r i m e n t e r w e n t o n t o s a y t h a t T , w a s r e a l l y t h e t r e a t m e n t o f i n t e r e s t a n d
t h a t T
2
w a s t h o u g h t t o h a v e n o e f f e c t b u t w a s j u s t i n c l u d e d f o r t h o r o u g h n e s s ,
a n d h e n c e t h a t p o o l i n g T p a n d C is a c c e p t a b l e .
T o t h e c r i t i c i a l a p p r a i s e r , t h i s c r e a t e s d o u b t s c o n c e r n i n g
h y p o t h e s i s s e l e c t i o n a n d c o n f i r m a t i o n f r o m t h e s a m e s e t o f d a t a . O n t h e o t h e r
h a n d , m a y b e t h e e x p e r i m e n t e r r e a l l y w a s p l a n n i n g t o p o o l T p a n d C a l l a l o n g
( a n d w a s s u r e J ^ w a s n o w o r s e t h a n C ) , a n e s p e c i a l l y p l a u s i b l e p o s s i b i l i t y
c o n s i d e r i n g t h a t o n l y 1 0 p a t i e n t s w e r e g i v e n T . In a n y c a s e , it is d i s c o n
c e r t i n g t h a t t o a n a l y z e t h e p r o b l e m f r o m a f r e q u e n t i s t p e r s p e c t i v e w e h a v e t o
k n o w w h a t t h e e x p e r i m e n t e r ' s i n t e n t i o n s w e r e . T r y i n g t o a n a l y z e h a r d d a t a
b y g u e s s i n g w h a t t h e e x p e r i m e n t e r w a s t h i n k i n g b e f o r e d o i n g t h e e x p e r i m e n t
s e e m s r a t h e r s t r a n g e . ( O f c o u r s e , a B a y e s i a n w o n ' t n e c e s s a r i l y b e a b l e t o
a v o i d s u c h c o n s i d e r a t i o n s , s i n c e t h e e x p e r i m e n t e r ' s s t a t e m e n t s m a y w e l l a f f e c t
p r i o r p r o b a b i l i t y j u d g e m e n t s . T h e u n c e r t a i n t y w i l l b e u p f r o n t i n t h e p r i o r
w h e r e i t b e l o n g s , h o w e v e r , w i t h t h e d a t a s p e a k i n g f o r i t s e l f t h r o u g h t h e
l i k e l i h o o d f u n c t i o n . )
4 . 2 . 4 C r i t i c i s m s
T h e r o s y s t a t e m e n t s in t h e p r e v i o u s s e c t i o n c o n c e r n i n g t h e S R P c a n
b e v i e w e d a s h o p e l e s s l y m i s g u i d e d b y f r e q u e n t i s t s , s i n c e f r e q u e n c y m e a s u r e s a r e
s o d e p e n d e n t o n s t o p p i n g r u l e s . C o n s i d e r E x a m p l e s 2 0 a n d 2 1 , f o r i n s t a n c e .
E X A M P L E 2 1 ( c o n t i n u e d ) . In t h e f i x e d s a m p l e s i z e e x p e r i m e n t , X
k
w o u l d b e a n
u n b i a s e d e s t i m a t o r o f f o r e i t h e r k = 1 o r 2 . I f o n e w e r e t o i g n o r e t h e
s t o p p i n g r u l e , , in C a s e 2 , h o w e v e r , a n d s t i l l u s e t h e s a m p l e m e a n a s t h e
e s t i m a t o r , a " p r o b l e m " o f b i a s a r i s e s . I n d e e d , t h e s a m p l e m e a n , 5 L , h a s
E
"
X
N
=
P
( X
1 ) E
" l l
X
1 ] +
VY
0 ) E
[
'
X
2 l
X
l < l
= + 1 ( I ) ,
w h i c h is b i a s e d u p w a r d s . T h u s i f a c o n d i t i o n a l i s t s t a t e d h e w o u l d b e u s i n g
5 L , r e g a r d l e s s o f t h e s t o p p i n g r u l e , t h e e x p e r i m e n t e r c o u l d u s e a n d " m a k e
a p p e a r l a r g e r t h a n i t r e a l l y i s " ( i f d e s i r e d ) .
E X A M P L E 2 0 ( c o n t i n u e d ) . T h i s e x a m p l e h a s b e e n e x t e n s i v e l y d i s c u s s e d , i n t e r m s
o f i t s r e l a t i o n s h i p t o t h e S R P a n d t h e L P . A r m i t a g e ( 1 9 6 1 ) p u b l i s h e d ( t o o u r
k n o w l e d g e ) t h e f i r s t s u c h d i s c u s s i o n . B a s u ( 1 9 7 5 ) g i v e s a p a r t i c u l a r l y
t h o r o u g h e x a m i n a t i o n o f a v e r s i o n o f t h e e x a m p l e . F o r d e f i n i t e n e s s in h i g h l i g h t
i n g t h e " p a r a d o x , " l e t us a s s u m e t h a t a 9 5 % " c o n f i d e n c e i n t e r v a l " f o r is
d e s i r e d , a n d t h a t a n " o b j e c t i v e " c o n d i t i o n a l i s t s t a t e s t h a t , i f a f i x e d s a m p l e
o f s i z e n w e r e t a k e n , h e w o u l d u s e t h e i n t e r v a l
( 4 . 2 . 4 ) C
n
(
n
) = ( ^ ( 1 . 9 6 ) 0 " * , ^ + ( 1 . 9 6 ) r f * ) .
O f c o u r s e , h e w o u l d n o t i n t e r p r e t c o n f i d e n c e i n t h e f r e q u e n c y s e n s e , b u t
i n s t e a d w o u l d ( p r o b a b l y ) u s e a p o s t e r i o r B a y e s i a n v i e w p o i n t w i t h t h e n o n i n f o r m
a t i v e p r i o r d e n s i t y

( ) = 1 , w h i c h l e a d s t o a 7 ? ( x
n
, n ~ 2 ) p o s t e r i o r d i s t r i b u t i o n
f o r ( a l s o , t h e u s u a l f i d u c i a l d i s t r i b u t i o n a n d t h e l i k e l i h o o d f u n c t i o n f o r
) .
S u p p o s e n o w t h a t t h e e x p e r i m e n t e r h a s a n i n t e r e s t i n s e e i n g t h a t
= 0 is n o t i n t h e c o n f i d e n c e i n t e r v a l . H e c o u l d t h e n u s e t h e s t o p p i n g r u l e
in ( 4 . 2 . 3 ) f o r s o m e K > 1 . 9 6 . T h e c o n d i t i o n a l i s t , b e i n g b o u n d t o i g n o r e t h e
s t o p p i n g r u l e , w i l l s t i l l u s e ( 4 . 2 . 4 ) a s h i s c o n f i d e n c e i n t e r v a l , b u t t h i s c a n
n e v e r c o n t a i n z e r o . H e n c e t h e f r e q u e n t i s t p r o b a b i l i t y o f c o v e r a g e o f ( 4 . 2 . 4 ) ,
n a m e l y
2
( ) = ? l ( C
N
( X
N
) c o n t a i n s ) ,
is s u c h t h a t r ( 0 ) = 0 a n d ( b y c o n t i n u i t y ) r ( ) is n e a r z e r o f o r s m a l l . T h e
e x p e r i m e n t e r h a s t h u s s u c c e e d e d i n g e t t i n g t h e c o n d i t i o n a l i s t t o p e r c e i v e t h a t
t 0 , a n d h a s d o n e s o h o n e s t l y .
E x a m p l e s 2 0 a n d 21 a r e t y p i c a l o f h o w t h e S R P ( o r t h e L P ) s e e m s t o
a l l o w t h e e x p e r i m e n t e r t o m i s l e a d a c o n d i t i o n a l i s t . T h e " m i s l e a d i n g " , h o w e v e r ,
is s o l e l y f r o m a f r e q u e n t i s t v i e w p o i n t , a n d w i l l n o t b e o f c o n c e r n t o a
c o n d i t i o n a l i s t . B e f o r e d i s c u s s i n g w h y , t w o c o m m e n t s a b o u t E x a m p l e 2 0 s h o u l d
b e g o t t e n o u t o f t h e w a y .
( i ) U s e o f a s t o p p i n g r u l e , s u c h a s t h a t in ( 4 . 2 . 3 ) , c a n b e c h a n c y f o r a n
e x p e r i m e n t e r i f = 0 is a r e a l p o s s i b i l i t y , s i n c e t h e n N is l i k e l y t o
b e e x t r e m e l y l a r g e . ( T h i s h a s n o r e a l b e a r i n g o n t h e a r g u m e n t s h e r e ,
h o w e v e r . )
( i i ) A B a y e s i a n c o n d i t i o n a l i s t m i g h t n o t c o m p l e t e l y i g n o r e a s t o p p i n g r u l e
s u c h a s t h a t i n ( 4 . 2 . 3 ) , i f h e s u s p e c t s it is b e i n g u s e d b e c a u s e t h e
e x p e r i m e n t e r t h i n k s m i g h t b e z e r o . T h e B a y e s i a n m i g h t t h e n a s s i g n s o m e
p o s i t i v e p r i o r p r o b a b i l i t y , \ , t o b e i n g e q u a l t o z e r o , in r e c o g n i t i o n
o f t h e e x p e r i m e n t e r ' s p o s s i b l e k n o w l e d g e . ( T h e s t o p p i n g r u l e i s a f f e c t
i n g o n l y t h e p r i o r , h o w e v e r , n o t " w h a t t h e d a t a s a y s . " ) A B a y e s i a n
a n a l y s i s in t h i s s i t u a t i o n is s t r i k i n g l y d i f f e r e n t t h a n t h a t in t h e
" n o n i n f o r m a t i v e
1 1
c a s e . I n d e e d , a s a p a r t i c u l a r e x a m p l e , i f t h e t 0 a r e
p
g i v e n p r i o r d e n s i t y ( 1 ) t i m e s a 7?(0,p ) d e n s i t y , t h e n t h e p o s t e r i o r
p r o b a b i l i t y t h a t = 0 , g i v e n t h e o b s e r v a t i o n x
p
= K n " ^ , is
8 2 THE LIKELIHOOD PR INCIPLE
o
F or some sp e c i f i c numbers, suppose t ha t
p
= 1 0 , K = 3 , and n = 1 0 , 0 0 0 . T hen,
( 0 | x
n
= 3 n" * ) = [ H ( "
l ) ( . 2 8 5 ) ] "
1
.
F o r m o d e r a t e , t h i s s a y s t h a t = 0 is q u i t e p l a u s i b l e w h e n n is l a r g e , e v e n
t h o u g h x i s t h r e e s t a n d a r d d e v i a t i o n s f r o m 0 . ( T h i s is e s s e n t i a l l y
" J e f f r e y ' s " o r " L i n d l e y ' s " P a r a d o x . )
F i n a l l y , l e t us r e t u r n t o E x a m p l e s 2 0 a n d 2 1 a n d s e e i f t h e
c o n d i t i o n a l p e r s p e c t i v e m i g h t n o t a f t e r a l l b e m o r e i n t u i t i v e l y a p p e a l i n g .
T h e u s e o f a b i a s e d e s t i m a t o r i n E x a m p l e 2 1 i s r e a l l y n o t t h a t t r o u b l i n g , s i n c e
b i a s h a s l o n g b e e n a s u s p e c t c r i t e r i o n ( e s p e c i a l l y w h e n c o m p a r e d t o , s a y , t h e
p l a u s i b i l i t y o f t h e W e a k C o n d i t i o n a l i t y P r i n c i p l e ) . W e w i l l c o n c e n t r a t e o n
t h e m o r e d i s t u r b i n g E x a m p l e 2 0 , t h e r e f o r e .
E X A M P L E 2 0 ( c o n t i n u e d ) . F i r s t o f a l l , t h e l i k e l i h o o d f u n c t i o n f o r ( w h e n w e
s t o p a t t i m e n ) i s p r o p o r t i o n a l t o a 7 l ( x , r \ ~
2
) d e n s i t y . T h i s c l e a r l y i n d i c a t e s
t h a t a n y p a r t i c u l a r v a l u e o f n e a r x
n
i s m o r e p l a u s i b l e t h a n a v a l u e f a r f r o m
x
n
T h e i n t e r v a l i n ( 4 . 2 . 4 ) i s a r e a s o n a b l e c h o i c e f r o m t h i s v i e w p o i n t ,
a l t h o u g h o t h e r c o n d i t i o n a l i s t s m i g h t v a r y t h e c o n s t a n t 1 . 9 6 o r s h i f t s o m e w h a t
t o w a r d s a s u s p e c t e d p r i o r m e a n .
C o n t r a s t t h i s w i t h t h e r a t h e r u n r e a s o n a b l e w a y i n w h i c h a f r e q u e n
t i s t m u s t b e h a v e t o o b t a i n , s a y , c o v e r a g e p r o b a b i l i t y o f a t l e a s t . 9 5 f o r a l l
w h e n K is l a r g e . It c a n b e s h o w n t h a t a f r e q u e n t i s t s h o u l d s t i c k t o
c o n n e c t e d i n t e r v a l s ( t o m i n i m i z e s i z e f o r a g i v e n c o v e r a g e p r o b a b i l i t y ) a n d
t h a t , w h e n ( s a y ) x
n
i s s l i g h t l y b i g g e r t h a n K n ~ ^ a n d n i s f a i r l y l a r g e ( w h i c h
w i l l t y p i c a l l y b e t h e c a s e f o r l a r g e K a n d t h e s t o p p i n g r u l e ( 4 . 2 . 3 ) ) , t h e s e
i n t e r v a l s m u s t u s u a l l y i n c l u d e b o t h z e r o a n d x . H e n c e , in o r d e r t o e n s u r e t h e
d e s i r e d c o v e r a g e p r o b a b i l i t y a t z e r o w h e n K i s l a r g e , a f r e q u e n t i s t w i l l m o d i f y
( 4 . 2 . 4 ) b y r e p l a c i n g a s m a l l p o r t i o n o f t h i s i n t e r v a l o f " l i k e l y " e , s u c h a s
( x
n
+ ( 1 . 9 6
n
) n ~ ^ , x
n
+ ( 1 . 9 6 ) n ~ * ) , w i t h a b i g i n t e r v a l , [ 0 , x
n
( 1 . 9 6 ) r f * ) ,
o f u n l i k e l y . T h i s s e e m s u n r e a s o n a b l e . T h e c o n d i t i o n a l i s t k n o w s t h a t a n x
s a t i s f y i n g x
p
> K n ~ ^ ( w i t h n v e r y l a r g e ) c o u l d h a v e a r i s e n f r o m = 0 , b u t
v a l u e s n e a r x
n
a r e s o m u c h m o r e l i k e l y t o b e t h e t r u e t h a t h e " b e t s " o n
t h e s e . It s h o u l d b e r e e m p h a s i z e d t h a t t h e c o n d i t i o n a l a n a l y s i s is p r e d i c a t e d
o n = 0 h a v i n g n o s p e c i a l p l a u s i b i l i t y ; i f it d o e s , t h e B a y e s i a n c o n c l u s i o n s
( s e e ( i i ) a b o v e ) w i l l b e q u i t e d i f f e r e n t .
T h e a b o v e a t t e m p t s a r e p r o b a b l y u n l i k e l y t o s a t i s f y a f r e q u e n t i s t ' s
v i o l a t e d i n t u i t i o n , i f t h e f r e q u e n t i s t is n o t p r a c t i c e d in t h i n k i n g c o n d i t i o n
a l l y . A s S a v a g e s a i d i n S a v a g e e t . a l . ( 1 9 6 2 )
"I l e a r n e d t h e s t o p p i n g r u l e p r i n c i p l e
f r o m P r o f e s s o r B a r n a r d , in c o n v e r s a t i o n
in t h e s u m m e r o f 1 9 5 2 . F r a n k l y , I t h e n
t h o u g h t i t a s c a n d a l t h a t a n y o n e in t h e
p r o f e s s i o n c o u l d a d v a n c e a n i d e a s o
p a t e n t l y w r o n g , e v e n a s t o d a y I c a n s c a r c e l y
b e l i e v e t h a t s o m e p e o p l e r e s i s t a n i d e a s o
p a t e n t l y r i g h t . "
Of some f orce may be the argument t hat , i f one's i nt ui t i on gives cont radict ory
insight s, i t should be t rust ed in simple si t uat i ons, such as Example 2 , rat her
than in extremely complex sit uat ions such as Example 2 0. The next sect ion
also lends support t o the case f or ignoring the stopping r ul e .
4 . 2 . 5 Stopping Rules and I nadmissibilit y
I n Section 3. 7 i t was argued t hat behavior in vi ol at i on of the LP,
but consistent wit h the WCP, tends to be decision t heor et ically inadmissible.
We rephrase the conclusion, in t his sect ion, to show t hat behavior dependent
on the stopping r ule wi l l of t en be inadmissible.
Suppose we have possible observations X ^X ^, . . . , as in Section
1 2
4 . 2 . 2 , and are considering two possible stopping r ule s, and , wit h
1 2 1 2
respect ive stopping sets { A
m
) and { A
m
} . T he stopping r ul e s, and , are
presumed t o have the possi bi l i t y of yi e l di ng common dat a,
; i . e . , t here is
presumed t o be some n* and A c A
1
* A^ such t hat A has posit ive pr obabilit y
1 2 n n
in both E
and E
f or a l l . Examples 2 0 and 2 1 are of t hi s t ype, since the

2
2
s e t s A, h a v e p o s i t i v e p r o b a b i l i t y f o r a l l ( u n d e r b o t h E
a n d E
) , s o t h a t
?
A = A. w o r k s .
Suppose t hat we face a decision problem concerning , consist ing of
choice of an act ion a G under a loss f unct ion L( a, ) which is st r i c t l y
convex in "a" f or each . (More general loss functions can of t en be handled
1 2
also. ) Proposed f or use in E
and E
, respect ively, are decision rules

^x
1 1
) and 6
2
( x
n
) . I f , now, the stopping rule is f e l t to make a dif f e r e nce , j
and
2
should di f f e r f or at least some of the possible common observations.
Thus we suppose t hat t here is some A* c A f or which
( 4 . 2 . 5 )
5 ]
( x
n
* )
2
( x
n
* ) f or x
n
* <= A*.
Consider, next , the mixed experiment, E*, consist ing of observing
1
J
J = 1 or 2 wit h pr obabilit y ? each and then performing E . T his is a well
N,
defined sequential experiment wit h random observation ( J , jK ) , N, being the
J
stopping time f or E
. I f the WCP is f ollowedf or E* and ( 4 . 2 . 5 ) holds, then

t he decision r ul e , 6 , used f or E* should sat i sf y
( ( l ,
n
* ) ) f ( ( 2 ,
n
* ) ) f or x
n
*<E A*,.
( Al t e r na t i ve l y, t his inequalit y should hold on some A* i f i t is f e l t t hat the
stopping r ule act ually used i . e . , the value of j r e al l y is relevant to
t he decision. ) But, the est imat or
\ ( ( , $
n
* ) ) + \ ( ( 2 ,
n
* ) ) i f n = n* and x
n

A*
n
) ) o t h e r w i s e
s a t i s f i e s ( b e c a u s e o f t h e s t r i c t c o n v e x i t y o f L )
( 4 . 2 . 6 ) L ( * ( ( J , x
* ) ) , ) < \ L ( ( ( l , x
n
* ) ) , ) + \ L ( ( ( 2 ,
n
* ) ) , ) .
E . s t a n d f o r e x p e c t a t i o n i n e x p e r i m e n t s E * E
p U
E
, r e s p e c t i v e l y , t h e f r e q u e n t i s t r i s k ( i n E * ) o f * s a t i s f i e s
1 2
H e n c e , l e t t i n g E J , E _, a n d E . s t a n d f o r e x p e c t a t i o n i n e x p e r i m e n t s E * , E
, a n d
R ( , * ) == E* L( * ( (
= E* L( ( ( J
Nj
N
l
N
l
N.
. r . >
2
9
E L( ^
:
L ( 6 ( (
N
= R ( , ) .
( T h e i n e q u a l i t y a b o v e is s t r i c t b e c a u s e o f ( 4 . 2 . 6 ) , t h e f a c t t h a t A * h a s
1 2
p o s i t i v e p r o b a b i l i t y f o r a l l in E
a n d E
, a n d p r o v i d i n g R ( , ) is f i n i t e . )
T h i s e s t a b l i s h e s t h e i n a d m i s s i b i l i t y o f a l l o w i n g t h e s t o p p i n g r u l e t o a f f e c t
t h e d e c i s i o n m a k i n g .
E X A M P L E 2 1 ( c o n t i n u e d ) . S u p p o s e t h a t t h e g o a l is t o e s t i m a t e u n d e r s q u a r e d
e r r o r l o s s , a n d t h a t , b e c a u s e o f t h e b i a s in u s e o f X ^ f o r t h e s t o p p i n g r u l e
, a n e s t i m a t o r
2
( x ) w o u l d b e u s e d (in E
) s u c h t h a t p ( x ) is n o t e q u a l
t o x
p
f o r a t l e a s t o n e p o s s i b l e o b s e r v a t i o n , s a y , n = 1 , x , = 1. L e t E
b e
t h e f i x e d s a m p l e s i z e e x p e r i m e n t o f s i z e k = 1 , a n d s u p p o s e t h a t |(x . ) = x ,
w o u l d b e u s e d f o r t h i s e x p e r i m e n t . H o w e v e r , t h e e x p e r i m e n t e r c h o o s e s b e t w e e n
1 2
p e r f o r m i n g E
a n d E
o n t h e b a s i s o f a f a i r c o i n f l i p (J = 1 o r 2 ) . T h i s is
e x a c t l y t h e s i t u a t i o n d i s c u s s e d a b o v e , a n d i f t h e e x p e r i m e n t e r f o l l o w s h i s
" i n s t i n c t s " a n d u s e s d i f f e r e n t e s t i m a t e s ( d e p e n d i n g o n J o r t h e a c t u a l
s t o p p i n g r u l e e m p l o y e d ) w h e n x^ = 1 is o b s e r v e d , h e w i l l b e b e h a v i n g i n a n
i n a d m i s s i b l e f a s h i o n .
T h e d e v e l o p m e n t a b o v e is j u s t a s p e c i a l c a s e o f t h a t i n S e c t i o n
3 . 7 , w h i c h in t u r n is b a s i c a l l y j u s t a v e r s i o n o f t h e R a o B l a c k w e l l t h e o r e m .
( H e r e , J is n o t p a r t o f t h e s u f f i c i e n t s t a t i s t i c f o r i n E * w h e n x
1 1
A *,
a n d d e c i s i o n r u l e s s h o u l d b e b a s e d o n l y o n t h e s u f f i c i e n t s t a t i s t i c . ) T h e
r e a s o n f o r e x p l i c i t l y g i v i n g t h e d e v e l o p m e n t in t h e s e q u e n t i a l f r a m e w o r k is
t o c l e a r l y e x h i b i t t h e c o n f l i c t b e t w e e n t h e f r e q u e n t i s t d e s i r e f o r
a d m i s s i b i l i t y a n d t h e i n t u i t i v e n o t i o n t h a t t h e s t o p p i n g r u l e u s e d s h o u l d
m a t t e r .
4 . 2 . 6 T h e G e n e r a l S t o p p i n g R u l e P r i n c i p l e
T he SRP can be ge ne r a l i z e d t o an e s s e nt i a l l y a r b i t r a r y sequence of
e xpe r i me nt s, and shown ( i n t hi s g e ne r a l i t y ) t o f ol l ow f rom t he RLP. T hus
suppose we have a va i l a b l e a sequence E | , E
2
, . . . of experiment s ( r e p l a c i ng t he
i . i . d . obse r va t i ons, X ^X ^, . . . , of Se ct i on 4 . 2 . 2 ) c onsi st i ng of obse r vi ng X .
on% . We can consi de r , f o r each m, t he composit e exper iment
E
m
= ( X
m
, , { P} ) c onsi st i ng of obse r ving X
m
= ( X
, . . . , X ) o n / = L wi t h
^ I Tl . i J
p r o b a b i l i t y d i s t r i b u t i o n P" \ ( I f t he exper iment s ar e i nde pe nde nt , P
1
? w i l l

s i mp l y by t h e p r o d u c t me a sur e o f t h e i n d i v i d u a l d i s t r i b u t i o n s o n . . )
j
We conside r se que nt i a l procedures i n which we de c i de , a f t e r
pe r f or mi ng experiment s E
, . . . , E , whet her or not t o pe r f or m E , , . As usua l ,

I m m +1
w e c a n a l l o w t h i s d e c i s i o n t o d e p e n d u p o n t h e o u t c o m e o f a n a u x i l l i a r y c h a n c e
m e c h a n i s m , l e a d i n g t o t h e f o l l o w i n g g e n e r a l n o t i o n o f a s t o p p i n g r u l e .
D E F I N I T I O N . A s t o p p i n g r u l e i s a s e q u e n c e = (
Q
, , , . . . ) i n w h i c h
Q
[ 0 , 1 ]
i s a co n s t an t an d : X * [ 0 , 1 ] a m e a s u r a b l e f u n c t i o n o n X f o r m >_ 1 .
T h e i n t e n t i o n is t h a t
m
( x
m
) r e p r e s e n t t h e c o n d i t i o n a l p r o b a b i l i t y
o f s t o p p i n g a f t e r o n l y m o b s e r v a t i o n s , g i v e n t h a t w e h a v e t a k e n m o b s e r v a t i o n s
a n d h a v e o b s e r v e d x
m
= ( x , , . . . , x ) . T h e n o n r a n d o m i z e d s t o p p i n g r u l e s d i s c u s s e d
in S e c t i o n 4 . 2 . 2 a r e t h e s p e c i a l c a s e w h e r e t h e c a n o n l y a s s u m e t h e v a l u e s
0 a n d 1 . W h e n c o n v e n i e n t , w e s h a l l r e g a r d
Q
a s a f u n c t i o n o n t h e o n e p o i n t
s e t = { 0 } , t h e " s a m p l e s p a c e " f o r t h e n u l l e x p e r i m e n t E = (%, , { P ^ } ) ,
w i t h P
t h e p o i n t m a s s a t X 's o n l y p o i n t f o r a l l .
N o w d e f i n e X * = { ( m , x
m
) : m I N ,
m
%
m
} . F o r
m
= ( x , , . . . , x
m
) X a n d 0 < j < m , l e t
m > J
= (x-,,.. . , x . ) 7? d e n o t e t h e
i n i t i a l s e g m e n t ; o f c o u r s e x = 0 X n o m a t t e r w h a t x
m
X
m
m i g h t b e . F o r
e a c h s t o p p i n g r u l e , , d e t e r m i n e a f a m i l y { P ^ } o f m e a s u r e s o n X * b y s e t t i n g
m 1 ~ ~
f o r e a c h m a n d B o r e l s e t A c ^
m
, W i t h t h i s d e f i n i t i o n ,
Q
is t h e p r o b a b i l i t y
o f p e r f o r m i n g E , i . e . o f t a k i n g n o d a t a a t a l l . A f t e r o b s e r v i n g x
m
, ( x
m
) is
t h e c o n d i t i o n a l p r o b a b i l i t y o f t a k i n g n o m o r e o b s e r v a t i o n s .
If P ( z * ) = 1 f o r all , t h e n t h e p r o c e d u r e is c e r t a i n t o s t o p
e v e n t u a l l y a n d is c a l l e d p r o p e r ; o t h e r w i s e is i m p r o p e r a n d , f o r a t l e a s t
o n e , t h e r e is a p o s i t i v e p r o b a b i l i t y ( 1 P ( * ) ) t h a t t h e s e q u e n t i a l p r o c e d u r e
w o u l d r e q u i r e s a m p l i n g a n i n f i n i t e n u m b e r o f t i m e s . F o r a p r o p e r s t o p p i n g
r u l e , , w e c a n c o n s i d e r t h e s e q u e n t i a l e x p e r i m e n t
E * = ( ( N , X
N
) , , { P } ) ,
w h e r e N d e n o t e s t h e ( r a n d o m ) s t o p p i n g t i m e . (It is n o t a t i o n a l l y c o n v e n i e n t t o
i n c l u d e N a s p a r t o f t h e o b s e r v a t i o n a l t h o u g h it c o u l d , o f c o u r s e , b e
r e c o v e r e d f r o m X . )
T h e S t o p p i n g R u l e P r i n c i p l e f o r t h i s g e n e r a l s e t t i n g is
f o r m a l i z e d i n t h e f o l l o w i n g t h e o r e m , a n d is s h o w n t o f o l l o w f r o m t h e R L P .
T H E O R E M 5 ( T h e S t o p p i n g R u l e P r i n c i p l e ) . F r o m t h e R L P , i t f o l l o w s t h a t , f o r
a n y ( p r o p e r ) s t o p p i n g r u l e ,
E v ( E l , ( n , x
n
) ) = E v ( E
n
, x
n
)
f o r { P } a . e . ( n , x ) , i . e . t h e e v i d e n c e c o n c e r n i n g i n E ^ i s i d e n t i c a l w i t h
t h a t f o r t h e f i x e d s a m p l e s i z e e x p e r i m e n t ( w i t h t h e o b s e r v e d r \ )
s o t h a t
i s i r r e l e v a n t .
P r o o f . P i c k n IN a n d l e t U , e z * b e t h e s e t o f p o i n t s ( n , x
n
) w i t h
n

n
s a t i s f y i n g 0 <
n
( x
n
) ( 1 . ( x
n > J
) ) , a n d l e t c: U
]
* ( 0 , ) b e t h e i n d i c a t e d
p r o d u c t . M a p U
]
o n t o U
2
= { x
n
z " : ( n , x
n
) U ^ } b y s e t t i n g ( n , ^
n
) =
n
.
T h e n is o n e t o o n e a n d b i m e a s u r a b l e , a n d
P " ( A ) = / [ l / c t y ) ] p ( d y ) .
'
( A )

T h e a s s e r t i o n o f t h e t h e o r e m n o w f o l l o w s f r o m t h e R L P . ||
N o t i c e t h a t w a s n o t r e q u i r e d t o b e a s u b s e t o f s o m e E u c l i d e a n
s p a c e , n o r w a s { P } r e q u i r e d t o b e a d o m i n a t e d f a m i l y ; t h u s e v e n in s i t u a t i o n s
w h e r e n o v e r s i o n o f t h e u s u a l L P c a n a p p l y , t h e S R P is v a l i d ( p r o v i d e d , o f
c o u r s e , t h a t t h e W C P a n d S P , a n d h e n c e t h e R L P , a r e a c c e p t e d ) . T h i s w a s
o b s e r v e d i n P r a t t ( 1 9 6 5 ) .
4 . 2 . 7 I n f o r m a t i v e S t o p p i n g R u l e s
E v e n t h e d e f i n i t i o n o f a s t o p p i n g r u l e g i v e n in t h e l a s t s e c t i o n
m a y s e e m s o m e w h a t n a r r o w w h e n c o m p a r e d w i t h t h e v a s t p o s s i b i l i t i e s f o r
i n f o r m a l s t o p p i n g d i s c u s s e d i n S e c t i o n 4 . 2 . 3 . S t o p p i n g r u l e s w h i c h a p p e a r t o
b e m o r e g e n e r a l c a n b e c r e a t e d b y i n t r o d u c i n g an a u x i l l i a r y v a r i a b l e Y
( p o s s i b l y r a n d o m ) , a n d a l l o w i n g , t h e c o n d i t i o n a l p r o b a b i l i t y o f s t o p p i n g a t
s t a g e m , t o d e p e n d o n t h e v a l u e o f Y , a s w e l l a s o n
m
. T h i s a c t u a l l y a d d s
v e r y l i t t l e g e n e r a l i t y , h o w e v e r , s i n c e t h e v a l u e s o f Y a t e a c h s t a g e c o u l d
s i m p l y b e i n c o r p o r a t e d i n t o t h e d a t a X . . T h e f o l l o w i n g e x a m p l e i l l u s t r a t e s
t h e i m p o r t a n c e o f s o m e t i m e s d o i n g t h i s .
E X A M P L E 2 3 . S u p p o s e X , , X
2
, . . . a r e i n d e p e n d e n t B e r n o u l l i ( ) r a n d o m v a r i a b l e s ,
w i t h = . 4 9 o r = . 5 1 . T h e o b s e r v a t i o n s , h o w e v e r , a r r i v e r a n d o m l y . I n d e e d ,
i f = . 4 9 , t h e o b s e r v a t i o n s a r r i v e a s a P o i s s o n p r o c e s s w i t h m e a n r a t e o f 1
p e r s e c o n d , w h i l e i f = . 5 1 , t h e o b s e r v a t i o n s w i l l a r r i v e a s a P o i s s o n
p r o c e s s w i t h m e a n r a t e o f 1 p e r h o u r . T h e " s t o p p i n g r u l e " t h a t w i l l b e u s e d
i s t o s t o p t h e e x p e r i m e n t a t t h e f i r s t o b s e r v a t i o n t h a t a r r i v e s a f t e r 1 m i n u t e
h a s e l a p s e d . O n e c a n h e r e i n t r o d u c e Y = t i m e , a n d w r i t e d o w n t h e s t o p p i n g
r u l e in t e r m s o f Y a n d t h e X . .
It is i n t u i t i v e l y c l e a r t h a t t h i s s t o p p i n g r u l e c a n n o t b e i g n o r e d
s i n c e , i f o n e e n d s up w i t h 6 0 o b s e r v a t i o n s , k n o w i n g w h e t h e r t h e e x p e r i m e n t r a n
f o r 1 m i n u t e o r 2 j d a y s is c r u c i a l k n o w l e d g e . I n c o r p o r a t i n g Y i n t o t h e d a t a
r e s o l v e s a l l a m b i g u i t i e s , h o w e v e r . T h u s , s i m p l y d e f i n e Y . a s t h e ( r a n d o m )
t i m e a t w h i c h t h e i o b s e r v a t i o n a r r i v e s , a n d c o n s i d e r t h e e x p e r i m e n t t o
c o n s i s t o f o b s e r v i n g ( X , , Y , ) , { X n ^ o ^ ^
e s t o
P P
i n
9
r u
^
e W #
H b e g i v e n
b y
O i f y
m
< 1
a n d is o f t h e f o r m d i s c u s s e d i n S e c t i o n 4 . 2 . 6 ( o r e v e n S e c t i o n 4 . 2 . 2 ) . T h e
i m p o r t a n c e o f t h e n u m b e r o f o b s e r v a t i o n s a r r i v i n g d u r i n g t h e t i m e s p a n o f
t h e e x p e r i m e n t w i l l b e r e f l e c t e d in t h e p o r t i o n o f t h e l i k e l i h o o d f u n c t i o n
d u e t o t h e y . .
S l i g h t l y m o r e g e n e r a l i t y m i g h t b e n e e d e d t h a n a f f o r d e d b y s i m p l y
o b s e r v i n g t h e a u x i l l i a r y v a r i a b l e s a t t h e o b s e r v a t i o n t i m e s ( a s in E x a m p l e 2 3 )
a n d i n c l u d i n g t h e m a s p a r t o f t h e o b s e r v a t i o n s , b u t t h e i d e a is c l e a r :
c o n s i d e r a l l a v a i l a b l e o b s e r v a t i o n a l i n f o r m a t i o n a s p a r t o f t h e d a t a X . . ( O f
c o u r s e , s o m e a u x i l l i a r y i n f o r m a t i o n m a y b e c o n s i d e r e d t o o i n f o r m a l t o i n c l u d e
a s p a r t o f t h e d a t a , a n d y e t m a y h a v e s o m e e f f e c t o n s t o p p i n g , b u t s u c h
i n f o r m a t i o n s h o u l d o n l y b e i g n o r e d i f it s e e m s r e l a t i v e l y u n i m p o r t a n t , i n
w h i c h c a s e i t s e f f e c t o n s t o p p i n g c a n p r o b a b l y a l s o b e i g n o r e d . )
E v e n w i t h i n t h e a b o v e m o r e g e n e r a l p e r s p e c t i v e o n s t o p p i n g r u l e s ,
a d i f f i c u l t y m i g h t s t i l l a r i s e . T h i s d i f f i c u l t y is t h a t t h e s t o p p i n g r u l e
m i g h t b e u n k n o w n o r p a r t i a l l y u n k n o w n , in t h a t c e s s a t i o n o f t h e s e q u e n t i a l
e x p e r i m e n t c o u l d d e p e n d o n u n o b s e r v a b l e r a n d o m q u a n t i t i e s w h o s e p r o b a b i l i t y
d i s t r i b u t i o n s a r e n o t c o m p l e t e l y k n o w n . F o l l o w i n g t h e c o n v e n t i o n o f S e c t i o n
3 . 5 a n d l e t t i n g d e n o t e a l l u n k n o w n q u a n t i t i e s , w e c o u l d t h u s w r i t e a g e n e r a l
s t o p p i n g r u l e i n t e r m s o f ( x
m
, ) . ( A c t u a l l y , b y i n c l u d i n g a u n i f o r m r a n d o m
v a r i a b l e in , it w o u l d b e p o s s i b l e t o h a v e t h e a s s u m e o n l y t h e v a l u e s
z e r o o r o n e . ) T h e g e n e r a l d e n s i t y o n x * ( d e n s i t i e s , a n d d i s c r e t e n e s s i f
n e c e s s a r y , b e i n g a s s u m e d t o r e t a i n c o m p a t i b i l i t y w i t h S e c t i o n 3 . 5 ) w o u l d t h e n
b e
f((n,x
n
)) = [ V l j t ^ J . l ^ . f ; ^
1 1
) .
J ^
where f
n
i s t he de nsi t y cor r e sponding t o P" Again f ol l owi ng Se c t i on 3 . 5 , one
"
c o u l d w r i t e = ( , n ) , w h e r e is o f i n t e r e s t a n d is a n u i s a n c e v a r i a b l e .
I f , f o r t h e o b s e r v e d ( n , x
n
) , ( x
n
" \ ) d o e s n o t d e p e n d o n f o r j n , a n d
i f is a n o n i n f o r m a t i v e n u i s a n c e p a r a m e t e r ( s e e S e c t i o n 3 . 5 ) f o r t h e f i x e d
s a m p l e s i z e e x p e r i m e n t s i n v o l v i n g o b s e r v a t i o n o f X
n
, t h e n t h e LP a n d N N P P
( s e e S e c t i o n 3 . 5 ) i m p l y t h a t is i r r e l e v a n t . S u c h a T i s c a l l e d
n o n i n f o r m a t i v e ; o t h e r w i s e i s s a i d t o b e i n f o r m a t i v e a n d t h e S R P w i l l n o t
a p p l y . ( R a i f f a a n d S c h l a i f e r ( 1 9 6 1 ) i n t r o d u c e d t h e s e t e r m s . )
W e d o n o t p u r s u e t h e m a t t e r f u r t h e r b e c a u s e i n f o r m a t i v e s t o p p i n g
r u l e s o c c u r o n l y r a r e l y in p r a c t i c e ( p r o v i d i n g a l l o b s e r v a t i o n a l i n f o r m a t i o n
i s i n c o r p o r a t e d i n t o t h e X . , a s i n E x a m p l e 2 3 ) . T h e r e e x i s t s a c e r t a i n a m o u n t
o f d i s a g r e e m e n t c o n c e r n i n g t h i s p o i n t , b u t t h e d i s a g r e e m e n t s e e m s t o b e
p r i m a r i l y d u e t o t h e m i s c o n c e p t i o n t h a t a n i n f o r m a t i v e s t o p p i n g r u l e is o n e
f o r w h i c h N c a r r i e s i n f o r m a t i o n a b o u t . T h i s is n o t t h e d e f i n i t i o n o f a n
i n f o r m a t i v e s t o p p i n g r u l e . ) l e r y o f t e n N w i l l c a r r y i n f o r m a t i o n a b o u t , b u t
t o b e i n f o r m a t i v e a s t o p p i n g r u l e m u s t c a r r y i n f o r m a t i o n a b o u t a d d i t i o n a l t o
N
t h a t a v a i l a b l e i n X , a n d t h i s l a s t w i l l b e r a r e i n p r a c t i c e .
4 . 3 T H E I R R E L E V A N C E O F C E N S O R I N G M E C H A N I S M S
4 . 3 . 1 I n t r o d u c t i o n
A n o t h e r g r e a t s i m p l i f i c a t i o n t h a t a p p l i c a t i o n o f t h e L P ( o r R L P )
m a k e s p o s s i b l e is i n t h e h a n d l i n g o f c e n s o r i n g . D a t a is o f t e n o b s e r v e d i n
c e n s o r e d f o r m , a n d t h e m e c h a n i s m s c a u s i n g t h e c e n s o r i n g c a n b e q u i t e i n v o l v e d .
In m o s t s u c h c a s e s , t h e L P ( o r R L P ) w i l l i m p l y t h a t o n l y t h e r e s u l t o f t h e
c e n s o r i n g , a n d n o t t h e c e n s o r i n g m e c h a n i s m o r d i s t r i b u t i o n , i s r e l e v a n t t o
c o n c l u s i o n s a b o u t .
S e c t i o n 4 . 3 . 2 c o n s i d e r s t h e s i t u a t i o n o f f i x e d ( n o n r a n d o m )
c e n s o r i n g , a n d e s t a b l i s h e s a v e r s i o n o f t h e i r r e l e v a n c e o f c e n s o r i n g
m e c h a n i s m s c a l l e d C e n s o r i n g P r i n c i p l e 1 . O n e o f t h e i m p l i c a t i o n s o f C e n s o r i n g
P r i n c i p l e 1 is t h a t t h e e v i d e n t i a l i m p o r t o f a n u n c e n s o r e d o b s e r v a t i o n , f r o m
a n e x p e r i m e n t i n w h i c h c e n s o r i n g w a s p o s s i b l e , is t h e s a m e a s t h e i d e n t i c a l
o b s e r v a t i o n f r o m a n u n c e n s o r e d v e r s i o n o f t h e e x p e r i m e n t .
C O N S E Q U E N C E S A N D C R I T I C I S M S O F T H E L P A N D R L P 91
S e c t i o n 4 . 3 . 3 c o n s i d e r s r a n d o m c e n s o r i n g , a n d e s t a b l i s h e s
c o n d i t i o n s u n d e r w h i c h t h e d i s t r i b u t i o n o f t h e c e n s o r i n g r a n d o m v a r i a b l e is
i r r e l e v a n t . T h e m a i n c o n d i t i o n is o n t h e c e n s o r i n g m e c h a n i s m i t s e l f , a n d
l e a d s t o t h e c o n c e p t o f a n o n i n f o r m a t i v e c e n s o r i n g m e c h a n i s m . T h i s c o n c e p t
is s u r p r i s i n g l y s i m p l e a n d p o w e r f u l . It is n o t t h e c a s e , h o w e v e r , t h a t all
s e n s i b l e c e n s o r i n g m e c h a n i s m s a r e n o n i n f o r m a t i v e , a l t h o u g h m a n y c o m m o n o n e s
a r e . T h i s i s s u e is d i s c u s s e d i n S e c t i o n 4 . 3 . 4 .
T h e C e n s o r i n g P r i n c i p l e , a s it a p p l i e s t o u n c e n s o r e d o b s e r v a t i o n s
in n o n r a n d o m c e n s o r i n g , s e e m s t o b e d u e t o J o h n P r a t t ( s e e P r a t t ( 1 9 6 1 , 1 9 6 5 ) ,
h i s d i s c u s s i o n in B i r n b a u m ( 1 9 6 2 a ) , a n d t h e d i s c u s s i o n i n S a v a g e e t . a l .
( 1 9 6 2 ) ) . T h e g e n e r a l C e n s o r i n g P r i n c i p l e s d e v e l o p e d h e r e a n d t h e c o n c e p t o f a
n o n i n f o r m a t i v e c e n s o r i n g m e c h a n i s m a p p e a r t o b e n e w , h o w e v e r . B e f o r e
p r o c e e d i n g w i t h t h e s e g e n e r a l d e v e l o p m e n t s , it is w o r t h w h i l e t o p r e s e n t an
i l l u m i n a t i n g ( a n d e n t e r t a i n i n g ) e x a m p l e f r o m P r a t t ' s d i s c u s s i o n o f B i r n b a u m
( 1 9 6 2 a ) . T h e e x a m p l e m a k e s a s i m p l e v e r s i o n o f t h e C e n s o r i n g P r i n c i p l e s e e m
i n t u i t i v e l y o b v i o u s .
E X A M P L E 2 4 ( P r a t t ) . A s a m p l e o f 2 5 o b s e r v a t i o n s w a s t a k e n f r o m a 7 ? ( ,
2
)
p o p u l a t i o n , a n d i n f e r e n c e a b o u t t h e p o p u l a t i o n m e a n w a s d e s i r e d . A l l o b s e r v a
t i o n s w e r e f o u n d t o l i e b e t w e e n 7 2 a n d 9 9 , a n d a s t a n d a r d n o r m a l a n a l y s i s w a s
p e r f o r m e d b y a f r e q u e n t i s t s t a t i s t i c i a n . T h e s t a t i s t i c i a n r e p o r t e d t h e
a n a l y s i s t o t h e e x p e r i m e n t e r , b u t , c u r i o u s a b o u t t h e o b s e r v e d 9 9 , a s k e d t h e
e x p e r i m e n t e r h o w h i g h h i s m e a s u r i n g i n s t r u m e n t ( a s s u m e d t o b e p e r f e c t l y
a c c u r a t e ) r e a d . T h e e x p e r i m e n t e r s a i d t h a t t h e i n s t r u m e n t o n l y r e a d t o 1 0 0 ,
b u t t h a t , i f h e h a d o b s e r v e d a r e a d i n g o f 1 0 0 , h e w o u l d h a v e s w i t c h e d t o
a n o t h e r i n s t r u m e n t w h i c h h a d a r a n g e o f 1 0 0 t o 1 0 0 0 . T h e s t a t i s t i c i a n w a s
h a p p y w i t h t h i s r e s p o n s e , a n d s a t i s f i e d w i t h a j o b w e l l d o n e .
T h e n e x t d a y t h e e x p e r i m e n t e r c a l l e d a b o u t s o m e t h i n g e l s e , a n d
m e n t i o n e d t h a t h e h a d j u s t c h e c k e d t h e h i g h r a n g e i n s t r u m e n t a n d f o u n d t h a t it
w a s b r o k e n . T h e s t a t i s t i c i a n a s k e d i f t h e e x p e r i m e n t e r w o u l d h a v e h a d t h e
i n s t r u m e n t r e p a i r e d b e f o r e c o m p l e t i n g t h e p r e v i o u s e x p e r i m e n t , t o w h i c h t h e
e x p e r i m e n t e r s a i d n o . T h e s t a t i s t i c i a n t h e n s a i d t h a t w h a t w e r e r e a l l y b e i n g
o b s e r v e d w e r e o b s e r v a t i o n s , X . , f r o m t h e t r u n c a t e d d i s t r i b u t i o n w i t h t h e
u s u a l n o r m a l d e n s i t y f o r x . < 1 0 0 b u t t h e p o i n t m a s s
o(lOO) = 7 V e x p { ~ (x)
2
}dx
), 1 0 0 ( 2 ) ~
2
~ 2
P
5
a t x . = 1 0 0 . T h i s , s a i d t h e s t a t i s t i c i a n , c a l l s f o r a d i f f e r e n t a n a l y s i s ; f o r
i n s t a n c e , t h e u s u a l 1 0 0 ( l ) % c o n f i d e n c e i n t e r v a l f o r in t h e n o r m a l s i t u a t i o n
w o u l d n o l o n g e r h a v e p r o b a b i l i t y o f c o v e r a g e o f a t l e a s t 1 i n t h e t r u n c a t e d
s i t u a t i o n . T h e e x p e r i m e n t e r r e a c t e d t o t h i s w i t h o u t r a g e , s a y i n g t h a t h e
o b s e r v e d p r e c i s e l y w h a t h e w o u l d h a v e o b s e r v e d h a d t h e h i g h r a n g e i n s t r u m e n t
b e e n w o r k i n g (all o b s e r v a t i o n s w e r e l e s s t h a n 1 0 0 ) , a n d t h a t t h e c o n d i t i o n o f
a n i n s t r u m e n t n e v e r u s e d in t h e e x p e r i m e n t h a r d l y s e e m e d r e l e v a n t t o t h e
i n f o r m a t i o n a b o u t o b t a i n e d f r o m t h e e x p e r i m e n t . T h e f r e q u e n t i s t s t a t i s t i c i a n
m e r e l y s h o o k h i s h e a d a t t h e n a i v e t e o f e x p e r i m e n t e r s .
4 . 3 . 2 F i x e d C e n s o r i n g a n d E q u i v a l e n t C e n s o r i n g M e c h a n i s m s
Co n s i d e r a n e x p e r i me n t E = ( X , , { P
Q
} ) F i x e d c e n s o r i n g o c c ur s whe n,
i nst e a d of X , one observes Y = g ( X ) , where g i s a known f unc t i on f rom % i nt o

fl
o y . T h u s t h e e x p e r i m e n t r e a l l y p e r f o r m e d is E ^ = ( Y , , { P
f l
o g } ) . ( A s
u s u a l , i f A c ^ , g "
( A ) = { x %: g ( x ) A } . )
E X A M P L E 2 5 . S u p p o s e X = ( X j . . .X ) , w h e r e t h e X
i
r e p r e s e n t t h e t i m e s o f
d e a t h d u e t o c a n c e r o f p a t i e n t s in a c a n c e r s u r v i v a l e x p e r i m e n t . S u p p o s e ,
h o w e v e r , t h a t t h e e x p e r i m e n t w i l l l a s t o n l y t e n y e a r s , s o t h a t t h e r e a l d a t a
w i l l , f o r t h e i p a t i e n t , b e
( 4 . 3 . 1 ) Y . = ( Y ] , Y J ) = ( m i n { X
. , 1 0 } , I [
O >
l o ]
( X
i
} )
( i . e . , t h e t r u n c a t e d s u r v i v a l t i m e a n d a n i n d i c a t o r a s t o w h e t h e r t h e
o b s e r v a t i o n is o r is n o t t r u n c a t e d ) . T h u s
( 4 . 3 . 2 ) Y = g ( X ) ( Y
r
. . . , Y
n
) .
T h i s is a n e x a m p l e o f w h a t is c o m m o n l y c a l l e d t y p e I c e n s o r i n g . E x a m p l e 2 4 is
C O N S E Q U E N C E S A N D C R I T I C I S M S O F T H E LP A N D R L P 9 3
a l s o o f t h i s t y p e .
E X A M P L E 2 6 . S u p p o s e t h a t X = ( X
]
X
R
) , b u t t h a t t h e n r l a r g e s t o f t h e X .
w i l l b e t r u n c a t e d a t t h e r ^ l a r g e s t . T h u s l e t
(4.3 .3 ) Y. = ( ] , ) = ( m i n { X . , X
( r )
h I ^ jtX.)),
w h e r e X/^ x <_ X ^ x <.. . <_ X/ \ a r e t h e o r d e r s t a t i s t i c s f o r X . A g a i n
( 4 . 3 . 4 ) Y = g ( X ) E ( Y
r
. . . , Y
n
) .
T his is an example of what is commonly called type I I censoring.
EXAMPLE 2 7. Suppose X = R
n
, y = % x { 0 , 1 } , and f or some f ixe d p > 0 ,
(X . O) i f | X| p
( 4 . 3 . 5 ) g(X ) =
( pX / | X | , l) i f IX I > p.
T h e n E ^ r e p r e s e n t s t h e e x p e r i m e n t in w h i c h t h e r a d i u s o f X is t r u n c a t e d a t p ,
b u t t h e d i r e c t i o n , X / | X | , o f X is f a i t h f u l l y r e p o r t e d . T h i s is n o t a
s t a n d a r d " t y p e " o f c e n s o r i n g , b u t f i t s e a s i l y w i t h i n o u r f r a m e w o r k .
O u r g o a l in t h i s s e c t i o n is t o i n d i c a t e t h a t t h e o n l y e f f e c t a
c e n s o r i n g m e c h a n i s m s h o u l d h a v e o n a c o n c l u s i o n is t o c o n v e y k n o w l e d g e c o n c e r n
ing t h e a c t u a l l o c a t i o n o f x i n % . T h i s m a y s e e m i n t u i t i v e l y o b v i o u s , b u t
E x a m p l e 2 4 is a p r i m e i l l u s t r a t i o n o f h o w t h i s is n o t t h e c a s e c l a s s i c a l l y .
W e f o r m a l i z e t h i s n o t i o n in t h e f o l l o w i n g d e f i n i t i o n .
D E F I N I T I O N . L e t E = ( X , , {P } ) i ? e g i v e n u n o e n s o r e d e x p e r i m e n t
a n d
c o n s i d e r t w o f i x e d c e n s o r i n g m e c h a n i s m s g , a n d g ^ T h e s e m e c h a n i s m s w i l l b e
s a i d t o b e e q u i v a l e n t o n A c % i f , f o r a l l x A ,
( 4 . 3 . 6 ) g ^ t g ^ x ) ) = g 2
( g
2
(
) )
A s a s p e c i a l c a s e
a s i n g l e f i x e d c e n s o r i n g m e c h a n i s m ^ g
3
w i l l b e s a i d t o b e
e q u i v a l e n t t o n o c e n s o r i n g o n h a Z i f g " ( g ( x ) ) = x f o r a l l x A .
T h e i d e a in t h e a b o v e d e f i n i t i o n is t h a t , f o r c e n s o r i n g m e c h a n i s m
g , o n e o b s e r v e s Y . = g ( X ) a n d t h a t t h e o n l y i n f o r m a t i o n c o m m u n i c a t e d b y t h e
c e n s o r e d d a t a , y . , is t h a t x w a s in gT ( y ^ . I f ( 4 . 3 . 6 ) is s a t i s f i e d , t h e n
g.j a n d g
?
w i l l a l w a y s c o n v e y t h e s a m e i n f o r m a t i o n ( f o r x A ) . A n d a g w h i c h
is e q u i v a l e n t t o n o c e n s o r i n g ( f o r x A ) c o n v e y s e x a c t l y t h e s a m e i n f o r m a t i o n
t h a t x d o e s . I n E x a m p l e 2 4 , i t is c l e a r t h a t t h e c e n s o r i n g m e c h a n i s m is
e q u i v a l e n t t o n o c e n s o r i n g o n A = { x : x . < 1 0 0 , i = 1 , . . . , 2 5 } ; in E x a m p l e 2 5 ,
g is e q u i v a l e n t t o n o c e n s o r i n g o n A = { x : x . < 1 0 , i = l , . . . , n } ; a n d , in
E x a m p l e 2 7 , g is e q u i v a l e n t t o n o c e n s o r i n g o n A = { x : |x| < p } . A s a n
e x a m p l e o f p o s s i b l e e q u i v a l e n c e o f t w o d i f f e r e n t c e n s o r i n g m e c h a n i s m s , c o n s i d e r
t h e f o l l o w i n g c o m b i n a t i o n o f E x a m p l e s 2 5 a n d 2 6 .
E X A M P L E 2 8 . S u p p o s e X = ( X
1 >
. . . , X
n
) , w h e r e t h e X^ c a n a s s u m e o n l y p o s i t i v e
i n t e g e r v a l u e s . L e t g
1
b e a s in ( 4 . 3 . 1 ) a n d ( 4 . 3 . 2 ) , g
2
b e a s in ( 4 . 3 . 3 ) a n d
( 4 . 3 . 4 ) , a n d A = { x : x , % = 1 0 } . I t is e a s y t o c h e c k t h a t , f o r x A ,
g i
( g
( ) ) = 9 2
1
( g
2
(
) ) = t* e
A :
î = ^
f
^ i "
H e n c e t h e t y p e I a n d t y p e II c e n s o r i n g w o u l d , in t h i s c a s e , b e e q u i v a l e n t o n A .
( N o t e t h a t c l a s s i c a l a n a l y s i s t e n d s t o t r e a t t h e t w o t y p e s o f c e n s o r i n g
d i f f e r e n t l y . )
W e n o w f o r m a l l y s t a t e , a n d j u s t i f y , t h e p r i n c i p l e t h a t e q u i v a l e n t
c e n s o r i n g m e c h a n i s m s c o n v e y t h e s a m e i n f o r m a t i o n a b o u t , f o r x A .
9
1
g
2
C E N S O R I N G P R I N C I P L E 1. I f E an d E ar e t w o e x p e r i me n t s ar i s i n g f r o m
ce n s o r i n g me ch an i s ms e q u i v al e n t o n A f o r an e x p e r i me n t E
t h e n
( 4 . 3 . 7 ) E v ( E
9
\
g ]
( x ) ) = E v ( E
9 2
,
f o r a l l x A i f X i s di s cr e t e , an d f o r {P } - a . e . x A i n g e n e r al . A s a
s p e ci al cas e , i f g~ ( g ( x ) ) = x f o r al l x A , t h e n ( 4 . 3 . 7 ) can b e r e p l ace d b y
( 4 . 3 . 8 ) E v ( E
g
, g ( x ) ) = E v ( E , x ) .
C e n s o r i n g P r i n c i p l e 1 f o l l o w s f r o m t h e L P in t h e d i s c r e t e c a s e
s i n c e , b y d e f i n i t i o n , t h e p r o b a b i l i t i e s o f gj(x) a n d g x ) a r e e q u a l ( t o
P
( g ^ ( g
( x ) ) ) ) f o r a l l . In t h e g e n e r a l c a s e i t f o l l o w s f r o m t h e R L P b y
s e t t i n g U
]
= g ^ x ) : x A } . , U
2
= g
2
( x ) : x A } , g ^ x ) ) = g
2
( x ) a n d
c g ^ x ) ) = 1 f o r x e A .
T h e g r e a t e s t p r a c t i c a l u s e o f C e n s o r i n g P r i n c i p l e 1 is in t h e c a s e
w h e r e a c e n s o r i n g m e c h a n i s m , g , is e q u i v a l e n t t o n o c e n s o r i n g o n A , a s w a s t h e
c a s e i n E x a m p l e s 2 4 , 2 5 , a n d 2 7 w h e n n o c e n s o r i n g h a p p e n e d t o o c c u r . T h e
c e n s o r i n g m e c h a n i s m c a n t h e n b e c o m p l e t e l y i g n o r e d .
4 . 3 . 3 R a n d o m C e n s o r i n g
T o g e n e r a l i z e t h e n o t i o n o f c e n s o r i n g t o i n c l u d e r a n d o m c e n s o r i n g ,
l e t b e a c e n s o r i n g v a r i a b l e w i t h p r o b a b i l i t y d e n s i t y v o n . ( T o
a v o i d t e c h n i c a l i t i e s , d i s c r e t e n e s s o f a n d X w i l l b e a s s u m e d u n t i l t h e e n d
o f t h e s e c t i o n . ) S u p p o s e t h a t X a n d a r e i n d e p e n d e n t ( w i t h o u t w h i c h y e r y
l i t t l e p r o g r e s s c a n b e m a d e ) , a n d t h a t
Y = g ( x , ) e y
is o b s e r v e d . T h e a c t u a l e x p e r i m e n t p e r f o r m e d c a n t h u s b e w r i t t e n
E
g , v
= ( > > { f
g , v
} ) >
where the density of Y is
( 4 . 3. 9 ) f 9 '
v
( y) = I f
( x) v( ) .
( x, ) : g(x, ) = y}
E X A M P L E 2 9 . S u p p o s e X r e p r e s e n t s t h e t i m e a t w h i c h a p a t i e n t i n a c a n c e r
s u r v i v a l s t u d y w o u l d s u f f e r d e a t h d u e t o c a n c e r , a n d l e t r e p r e s e n t t h e
d e a t h t i m e d u e t o c o m p e t i n g r i s k s . ( W e w i l l s i d e s t e p t h e i s s u e o f w h e t h e r o r
n o t X a n d c a n b e w e l l d e f i n e d . ) T h e a c t u a l o b s e r v a t i o n f o r t h e p a t i e n t w i l l
b e
( 4 . 3 . 1 0 ) Y = ( Y
, Y
2
) = g ( X , ) = ( m i n { X , } I [
0 >
] (
) ) >
1 2
i . e . , the act ual time of deat h, Y , and an indicat or , Y , as to the cause of
death. Generalizat ion to involve data from n pat ient s and a var iet y of
c o m p e t i n g r i s k s i s s t r a i g h t f o r w a r d , a n d a l l t h e s u b s e q u e n t t h e o r y w i l l a p p l y
e q u a l l y w e l l t o s u c h a g e n e r a l i z a t i o n .
T h e L P , o f c o u r s e , i m p l i e s t h a t t h e l i k e l i h o o d f u n c t i o n , d e t e r m i n e d
f r o m ( 4 . 3 . 9 ) f o r t h e o b s e r v e d y , c o n t a i n s a l l t h e i n f o r m a t i o n a b o u t a v a i l a b l e
f r o m t h e e x p e r i m e n t . T h e d i f f i c u l t y i n u t i l i z i n g t h i s l i k e l i h o o d f u n c t i o n l i e s
in t h e p r e s e n c e o f
v
in t h e e x p r e s s i o n : t y p i c a l l y , v w i l l b e u n k n o w n ( a n d
c o m p l i c a t e d ) . I f , h o w e v e r ,
v
w e r e j u d g e d t o c o n v e y n o i n f o r m a t i o n a b o u t ( s e e
S e c t i o n 3 . 5 a n d S e c t i o n 4 . 3 . 4 ) a n d f
9 > v
( y ) c o u l d b e s h o w n t o f a c t o r i n t o
s e p a r a t e t e r m s i n v o l v i n g a n d
v
, t h e n t h e d i f f i c u l t y w o u l d d i s a p p e a r . T h i s
w o u l d r e s u l t in an e n o r m o u s s i m p l i f i c a t i o n o f t h e a n a l y s i s , a n d is a n o t h e r o f
t h e g r e a t p r a c t i c a l g a i n s t h a t c a n b e r e a l i z e d t h r o u g h a d o p t i o n o f t h e L P .
T h e f o l l o w i n g d e f i n i t i o n g i v e s t h e k e y c h a r a c t e r i z a t i o n o f c e n s o r i n g m e c h a n i s m s
f o r w h i c h t h i s p r o g r a m is p o s s i b l e .
D E F I N I T I O N . A c e n s o r i n g m e c h a n i s m g: X x * y i s n o n i n f o r m a t i v e a t y y i f
g ( y ) i s a p r o du ct s e t
3
i . e .
i f
g
1
( y ) = A x B , w h e r e A c % a n d B c: .
E X A M P L E 2 9 ( c o n t i n u e d ) . H e r e
g'W. y
2
))
( y
1
. ) x { y
1
} i f y
2
= o
{ y
1
} x [ y\ ) i f y
2
= l ,
s o t h a t g is a n o n i n f o r m a t i v e c e n s o r i n g m e c h a n i s m a t a l l y y .
E X A M P L E 2 7 ( c o n t i n u e d ) . C o n s i d e r t h e s i t u a t i o n in E x a m p l e 2 7 , b u t a s s u m e t h a t
p is n o w a r a n d o m v a r i a b l e ( a n d , h e n c e , r e p l a c e g ( X ) b y g ( X , p ) ) . S i n c e
g 'W . y
2
) ) =
{ y
1
} x E l y
1
! , - )
i f
{cy
1
: c > 1} x { | y
]
| } i f y
2
= 1,
g is a noninformative censoring mechanism at all y y.
CONSEQ UENCES AND CR IT ICISMS OF THE LP AND RLP 9 7
I f g i s noniref ormat ive a t y , t hen ( 4 . 3 . 9 ) becomes ( employing a l so
t he independence of X and )
( 4.3. ) f f
v
( y) = [ I f
( ) ] [ I v( ) ] .
s o t h a t ( f o r k n o w n v ) , t h e L P i m p l i e s t h a t a l l i n f o r m a t i o n c o n c e r n i n g f r o m
t h e e x p e r i m e n t is c o n t a i n e d i n
( 4 . 3 . 1 2 ) * ( ) = I f
( x ) .
I f v is u n k n o w n b u t " n o n i n f o r m a t i v e " f o r ( s e e S e c t i o n s 3 .5 a n d 4 . 3 . 4 ) , t h e
s a m e c o n c l u s i o n f o l l o w s f r o m t h e N N P P in S e c t i o n 3 . 5 . 5 . T h e s e c o n c l u s i o n s c a n
b e s u m m a r i z e d a s f o l l o w s .
C E N S O R I N G P R I N C I P L E 2 . I f X a n d a r e d i s c r e t e , X a n d a r e i n d e p e n d e n t , g i s
n o n i n f o r m a t i v e a t t h e o b s e r v e d y , a n d e i t h e r v i s k n o w n o r i t i s u n k n o w n b u t
n o n i n f o r m a t i v e , t h e n E v ( E ^
> v
, y ) d e p e n d s o n l y o n l * { & ) { f r o m ( 4 . 3 . 1 2 ) ) .
N o t e t h a t t h i s p r i n c i p l e d o e s n o t s a y t h a t c e n s o r i n g h a s n o
e f f e c t o n t h e a n a l y s i s . I n d e e d , *,*() w i l l o f t e n f a i l t o b e p r o p o r t i o n a l t o
v
( ) = f ( x ) , w h i c h w o u l d b e u s e d i f n o c e n s o r i n g o c c u r r e d . A n o t h e r p o i n t is
x
t h a t t h e o n l y c e n s o r i n g m e c h a n i s m s w h i c h c a n g u a r a n t e e t h a t E v ( E
9 > v
, y ) d o e s n o t
d e p e n d o n v ( f o r v a s in t h e p r i n c i p l e ) a r e n o n i n f o r m a t i v e c e n s o r i n g m e c h a n i s m s .
T h i s i s e s t a b l i s h e d in t h e f o l l o w i n g t h e o r e m .
T H E O R E M 6 . I f g : X x + y i s n o t a n o n i n f o r m a t i v e c e n s o r i n g m e c h a n i s m a t y ,
t h e n t h e r e e x i s t s { f } o n X s u c h t h a t E v ( E
9 > v
, y ) d e p e n d s o n v.

Proof. I f g ( y) is not a product se t , i t f ollows t hat t here e xi st two points
| i
2
such t hat e i t he r
1
= { x: g( x,
) = y and g( x,
2
) f y } ,
or
2
= ( x: gx ^ t y and g( x,
2
) = y } ,
or both are nonempty. Consider v t hat are concentrated on {
1
,
2
} , and def ine
o = x : g ( . , ) = g ( ,
9
) = y } .
O I C
E q u a t i o n ( 4 . 3 . 9 ) c a n t h e n b e w r i t t e n
f j
V
( y ) = v (
l
) P
(
l
) + V U
2
) P
(
2
) + P
(
3
) .
= v (
1
) [ P
Q
(
1
) P
(
9
) ] + P
Q
(
9
U J .
T h u s , a s l o n g as { f j i s c h o s e n s o t h a t [ P
Q
(
1
) P
O
(
9
) ] a n d P . (
9
U ) a r e
b l o c c . 0
n o t p r o p o r t i o n a l a s f u n c t i o n s o f , t h e l i k e l i h o o d f u n c t i o n w i l l d e p e n d o n
F i n a l l y , w e l e a v e t h e d i s c r e t e s e t t i n g a n d d e v e l o p a v e r y g e n e r a l
v e r s i o n o f C e n s o r i n g P r i n c i p l e 2 , b a s e d o n t h e R L P . W e w i l l a s s u m e t h a t a n d
y a r e L C C B s p a c e s , t h a t v is a B o r e l p r o b a b i l i t y m e a s u r e , a n d t h a t
g: X x +y is a B o r e l f u n c t i o n . T h e a c t u a l e x p e r i m e n t o f o b s e r v i n g
Y = g ( X , ) is E
g > v
= ( Y , , { P
g
'
v
} ) , w h e r e
( 4 . 3 . 1 3 ) p 9 >
v
(
C
) = ( P
v ) ( ( x , ) : g ( x , ) C } ) .
T h e d e f i n i t i o n o f a n o n i n f o r m a t i v e c e n s o r i n g m e c h a n i s m a t y r e m a i n s u n c h a n g e d ,
a n d l e a d s t o t h e f o l l o w i n g p r i n c i p l e .
C E N S O R I N G P R I N C I P L E V . L e t C c y b e a B o r e l s e t s u ch t h at g i s a n o n i n f o r ma-
t i v e ce n s o r i n g me ch an i s m at al l y C . S u p p o s e v , an d \>
2
ar e B o r e l p r o b ab i l i t y
me as u r e s ( f o r ) w h i c h a r e m u t u a l l y a b s o l u t e l y c o n t i n u o u s o n C * = U B
y
( w h e r e g " ( y ) = A * B ) . T h e n , i f e i t h e r ( i ) v , a n d v
2
a r e k n o w n , o r ( i i )
t h e y a r e u n k n o w n b u t n o n i n f o r m a t i v e ( s e e S e c t i o n s 3 .5 a n d 4 . 3 . 4 ^ i t s h o u l d b e
t h e c a s e t h a t
( 4 . 3 . 1 4 ) E v ( E ', y ) = E v ( E % y ) f o r { ?
e
'} a . e . y C .
T h e c o n c l u s i o n in C e n s o r i n g P r i n c i p l e V is n o t q u i t e a s s t r o n g
a s t h a t in t h e o r i g i n a l C e n s o r i n g P r i n c i p l e 2 , in t h a t e v i d e n t i a r y e q u i v a l e n c e
i s o n l y s t a t e d t o h o l d a m o n g e q u i v a l e n c e c l a s s e s o f v (on C ) . O f c o u r s e , i f
t h e p o s s i b l e v u n d e r c o n s i d e r a t i o n a r e k n o w n t o b e a b s o l u t e l y c o n t i n u o u s w i t h
r e s p e c t t o s o m e m e a s u r e u , t h e n i t c a n b e s t a t e d t h a t v is i r r e l e v a n t ( i f it is
C O N S E Q U E N C E S A N D C R I T I C I S M S O F T H E LP A N D R L P 9 9
n o n i n f o r m a t i v e ) . F o r i n s t a n c e , in E x a m p l e 2 9 it m a y b e r e a s o n a b l e t o a s s u m e
t h a t v is a b s o l u t e l y c o n t i n u o u s w i t h r e s p e c t t o L e b e s g u e m e a s u r e , a n d is t h u s
i g n o r a b l e ( i f n o n i n f o r m a t i v e ) .
It s e e m s l i k e l y t h a t C e n s o r i n g P r i n c i p l e V is a g e n e r a l c o n s e q u e n c e
o f t h e R L P . T h i s is b e c a u s e o n e c a n d e f i n e ( s e e t h e R L P ) U , = U
2
= C , t o b e
t h e i d e n t i t y m a p , a n d
( 4 . 3 . 1 5 ) c ( y ) = c ( ( x , ) ) = v
a n d s e e k t o s h o w t h a t ( f o r a n y B o r e l s u b s e t , D , o f C )
( 4 . 3 . 1 6 )
P
'
V 2 ( D )
>V l
S i n c e ( 4 . 3 . 1 6 ) is e s s e n t i a l l y ( 3 . 4 . 1 ) o f t h e R L P ( w h e r e 1/c h a s b e e n r e p l a c e d
b y c f o r c o n v e n i e n c e in w h a t f o l l o w s ) , C e n s o r i n g P r i n c i p l e 2
1
w o u l d b e a n
i m m e d i a t e c o n s e q u e n c e o f t h e R L P ( a n d t h e N N P P o f S e c t i o n 3 . 5 , i f t h e v
a r e
u n k n o w n b u t n o n i n f o r m a t i v e ) . A n d ( 4 . 3 . 1 6 ) s e e m s t o b e a c o r r e c t e q u a t i o n :
i t c a n t r i v i a l l y b e v e r i f i e d t o h o l d in t h e d i s c r e t e s e t t i n g , f o r i n s t a n c e .
U n f o r t u n a t e l y , s e v e r e m e a s u r a b i t y d i f f i c u l t i e s ( d u e t o t h e p o s s i b l e n a s t y
n a t u r e o f g ) p r e v e n t e d us f r o m v e r i f y i n g ( 4 . 3 . 1 6 ) , in g e n e r a l . U n d e r
a d d i t i o n a l c o n d i t i o n s , h o w e v e r , w e w e r e a b l e t o s h o w t h a t ( 4 . 3 . 1 6 ) d o e s h o l d
f o r s o m e p o s i t i v e c , w h i c h s u f f i c e s , b y t h e a b o v e a r g u m e n t , t o e s t a b l i s h
C e n s o r i n g P r i n c i p l e 2 ' as a c o n s e q u e n c e o f t h e R L P . F u r t h e r m o r e , t h o u g h
s o m e w h a t t e c h n i c a l , t h e s e a d d i t i o n a l c o n d i t i o n s i n v o l v e o n l y t h e c e n s o r i n g
m e c h a n i s m , g , a n d n o t t h e P
o r v . T h i s m a k e s g e n e r a l v e r i f i c a t i o n o f t h e
i r r e l e v a n c e o f a n y s p e c i f i c c e n s o r i n g m e c h a n i s m p o s s i b l e .
T H E O R E M 7 . L e t g b e a n o n i n f o r m a t i v e c e n s o r i n g m e c h a n i s m a t a l l y C , an d
s u p p o s e t h at t h e r e e x i s t s e q u e n ce s { } a n d { } o f m e a s u r a b l e m a p p i n g s
p
: X + X a n d : > , s u c h t h a t t h e f u n c t i o n s g ( x , ) = g ( c p
n
( x )
n
( ) )
a r e c o u n t a b l y v a l u e d a n d t h e a a l g e b r a s ,
3 , a n d J > , g e n e r a t e d o n % x b y
g ( X j ) ,
n
( x ) > a n d g ( x
s
) , r e s p e c t i v e l y
s a t i s f y t h e c o n d i t i o n s
1 0 0 T H E L I K E L I H O O D P R I N C I P L E
00 OO
) V Jr = ir.
m =l n =m
T h e n f o r a n y t w o p r o b a b i l i t y m e a s u r e s v , a n d v o n A , w h i c h a r e m u t u a l l y
a b s o l u t e l y c o n t i n u o u s o n C ,
( 4 . 3 . 1 7 ) / h ( y ) P
2
( d y ) = / h ( y ) c ( y ) P ' ^ ( d y )
for every bounded measurable function h ony and every probability measure P
onX. (Note that (4.3.16) follows trivially from (4. 3. 17). Hence, under the
above conditions, Censoring Principle V is a consequence of the RLPJ
Proof. We w i l l prove t he t heorem f or C = y. T he modi f i c a t i ons needed f o r
a r b i t r a r y C a r e obvious. F or n _> 1 l e t t y ?} ^! be t he count ably many values
of g
n
; t he al ge br a J<
n
i s ge ne r at e d by t he count abl e p a r t i t i o n
P
n
= { ! J B ^ ) o f Z
i nt o t he measurable r e ct a ngl e s ( or product se t s)

A*?
x
B? = g ( y
1
?) , where A" = (A J and B" = " (B
n
) ; her e ( as be f or e ) A
J 3 n j J n n j n y
J _ 1 J
and B ar e det er mined by t he r e l a t i o n g ( y) = A x B . F or ( x, ) X * ,
de f i ne
' BJ MB! ?) if ^(B?) > 0,
( 4 . 3 . 1 8 ) c
n
( x , ) =
) = v ( B ) =
I 1 if v^Bj ^ Bj ) =
c ( x , ) = l i m s u p c ( x , ) ,
where j is determined by the r e l at i on g
n
( x, ) = y
n
. .
A dir ect computation ve r i f i e s t ha t , f or any pr obabilit y measure P
on
9
(
4 3 1 9
) "
c
n =
E
'
I n d e e d , t o s h o w t h i s it is s u f f i c i e n t t o t a k e a n y b o u n d e d m e a s u r a b l e f u n c t i o n ,
h , o n x * y a n d n o t e t h a t
/ h (
n
( x ) , g
n
( x , ) ) c
n
( x , ) P ( d x ) v
1
( d )
= / / h (
n
( ) , g
n
( , ) ) P ( d ) v
?
( d ) .
z A
L
By ( 4 . 3 . 1 9 ) and Condition ( i ) , c is a uniformly int egrable martingale on
( Z x , ( 3
n
v ^
n
)
n>
| Pxv^) , f or every P. Hence c
p
converges to c wit h
Pxvn measure 1 f or every P, and sa t i sf i e s
P v
( 4 . 3 . 2 0 ) I = E [ c | 3
n
v ,* ] f or every n > 1 .
n n
Since we may take P t o be concentrated on any single point x z> we have
act ually shown t hat c
n
( x, ) converges to c( x, ) f or every x e Z and v, almost
every in (where the except ional set of v, measure zero may depend on x) .
I t is obvious from the de f i ni t i on of c
n
t hat c
n
( x, ) depends on x and
only through y!? = g ( x, ) , and t heref ore t hat c is & measurable. I t
00
follows t hat c is measurable over v jr f or each m, and so (by Condition ( i i ) )
n=m
c is measurable over A Since any ^ measurable f unct ion may be wr i t t e n as a
Borel measurable f unct ion of g, t here exist s some posit ive f unct ion, c, on y
wit h
( 4 . 3 . 2 1 ) c( x, ) = cog( , ) .
N o w let h b e b o u n d e d a n d m e a s u r a b l e on'J/, l e t P b e t h e p r o b a b i l i t y
m e a s u r e o n z , a n d s e t
Pv
9
(4.3 .2 2 ) h
n
= E h o g | 3
n
v ^ ] .
Again the martingale convergence theorem implies t hat h
n
( x, ) converges t o
hg( x, ) f or Pxv , almost every ( x, ) , since hg is ^ measurable and
Conditions ( i ) and ( i i ) imply t h a t j ^c v i c v (a?_ v i ) . By Lebesgue's
n= l
n
n
= l
n n
dominated convergence theorem, ( 4 . 3 . 2 0 ) , and ( 4 . 3 . 1 9 ) ,
g . v ,
/ h e dP ' = / / h o g c o g P ( d x K ( d )
V X
= l i m / / h
n
c P ( d x ) v , ( d ) (by D C T )
n~> X A

'
= lim / / h
n
c
n
P d x j v ^ d ) (by ( 4 . 3 . 2 0 ) )
n* X A
= l i m / / h
n
P ( d x ) v
2
( d ) (by ( 4 . 3 . 1 9 ) )
n X A
= / / h o g P ( d x ) v
?
( d ) (by D C T )
X A
= /h dP '
2
.
T h i s v e r i f i e s ( 4 . 3 . 1 7 ) a n d c o m p l e t e s t h e p r o o f . ||
R e m a r k 1 . In c a s e it is p o s s i b l e t o f i n d { } a n d { } s o t h a t & c & . , ,
C o n d i t i o n ( i ) in t h e t h e o r e m m a y b e e l i m i n a t e d a n d C o n d i t i o n ( i i ) c a n b e
00
si mpl i f i e d t o v jr &.
n= l
n
Remark 2 . I f and are themselves countably valued, then obviously g is
also, so the theorem applies i f Conditions ( i ) and ( i i ) are sa t i sf i e d.
EXAMPLE 2 9 ( cont inued) . Let t ing <a> denote the closest int eger t o a ( t he
larger int eger in case of a t i e ) , def ine
n
( x) = 2 "
n
<2
n
x> and
n
( ) = 2 "
n
<2
n
> .
I t is st raight f orward to ve r i f y t hat Conditions ( i ) and ( i i ) in Theorem 7 are
sa t i sf i e d, and hence t hat Censoring Pr inciple 2
1
follows in complete
generalit y from the RLP ( f or t hi s si t ua t i on) .
EXAMPLE 2 7 ( cont inued) . Let p = {A
1
?} be a sequence of par t i t i ons of the
i < 1
unit sphere ( i n R
n
) int o f i ni t e l y many Borel sets such t hat P
n + 1
ref ines p
p
and
lim max dia Â
1
?) = 0 .
CONSEQ UENCES AND CR IT ICISMS OF THE LP AND RLP 1 03
Let { j } be a c o l l e c t i o n of poi nt s such t ha t ? A*?, and def i ne
n
( x ) = i 2 "
n
j i f i 2
n
| x| < i + 1 and x/ | x| e AJ ,
n
(
P
) = k2 ~
n
i f k <_ 2
n
p < k + 1 .
A g a i n t h e C o n d i t i o n s ( i ) a n d ( i i ) o f T h e o r e m 7 a r e e a s i l y v e r i f i e d , s o t h a t
t h i s c e n s o r i n g m e c h a n i s m is a l s o g e n e r a l l y i r r e l e v a n t .
4 . 3 . 4 I n f o r m a t i v e C e n s o r i n g
It i s , o f c o u r s e , n o t a l w a y s t h e c a s e t h a t t h e c e n s o r i n g m e c h a n i s m o r
d i s t r i b u t i o n c a n b e i g n o r e d . T h e r e a r e y e r y f e w i n s t a n c e s o f f i x e d c e n s o r i n g
w h e r e i n t h e m e c h a n i s m s c a n b e l a b e l e d i n f o r m a t i v e , s o w e w i l l c o n c e n t r a t e in
t h i s s e c t i o n o n r a n d o m c e n s o r i n g .
T h e m o s t c o m m o n r e a s o n f o r b e i n g u n a b l e t o i g n o r e t h e c e n s o r i n g
d i s t r i b u t i o n , v , in r a n d o m c e n s o r i n g is d e p e n d e n c e o f t h e r a n d o m v a r i a b l e X
a n d t h e r a n d o m c e n s o r i n g v a r i a b l e . In E x a m p l e 2 9 , f o r i n s t a n c e , o n e m a y
h a v e a n o n c a n c e r d e a t h w h i c h o c c u r r e d b e c a u s e c a n c e r s u b s t a n t i a l l y l o w e r e d
o v e r a l l h e a l t h . I n d e e d i n c o m p e t i n g r i s k t h e o r y , i n g e n e r a l , d e p e n d e n c e
b e t w e e n X a n d t h e c e n s o r i n g v a r i a b l e s m a y b e t h e r u l e r a t h e r t h a n t h e e x c e p t i o n .
S u c h d e p e n d e n c e m a k e s C e n s o r i n g P r i n c i p l e 2 i n a p p l i c a b l e , a n d i n d e e d i ( )
w i l l t y p i c a l l y d e p e n d u p o n v in s u c h s i t u a t i o n s . ( T h e L P is s t i l l v a l i d ,
o f c o u r s e . )
A s e c o n d p o s s i b l e r e a s o n t h a t t h e c e n s o r i n g d i s t r i b u t i o n m i g h t b e
i n f o r m a t i v e is t h a t t h e c e n s o r i n g m e c h a n i s m , g , m i g h t f a i l t o b e n o n i n f o r m a t i v e .
A s a \ / e r y s i m p l e e x a m p l e , s u p p o s e t h e a c t u a l o b s e r v a t i o n is
Y = g ( X , ) = X + ,
w h e r e X X = ( 0 , ~ ) a n d = ( 0 , ) . It i s e a s y t o c h e c k t h a t g " ( y ) is n o t
a p r o d u c t s e t i n X x f o r a n y y , s o t h a t g c l e a r l y f a i l s t o b e n o n i n f o r m a t i v e .
F o r s u c h g , % ( ) w i l l t y p i c a l l y d e p e n d o n v .
A t h i r d r e a s o n t h a t v m i g h t n o t b e i g n o r a b l e is t h a t v w i l l o f t e n b e
u n k n o w n , a n d t h e r e c o u l d b e s o m e " p r i o r " r e l a t i o n s h i p b e t w e e n v a n d . A g a i n ,
t h e n o t a t i o n o f S e c t i o n 3 . 5 is c o n v e n i e n t h e r e . T h u s l e t s t a n d f o r a l l
u n k n o w n a s p e c t s o f t h e s i t u a t i o n a n d w r i t e = ( , n ) , w h e r e i s o f i n t e r e s t
a n d is a n u i s a n c e v a r i a b l e ( p r e s u m a b l y c o n t a i n i n g u n k n o w n a s p e c t s o f t h e
d i s t r i b u t i o n , v , o f ) . F o r i n s t a n c e , i f X * P a n d ^ v
a r e s i m i l a r
c o m p e t i n g r i s k s , t h e r e m i g h t w e l l b e s u s p e c t e d r e l a t i o n s h i p s b e t w e e n a n d
w h i c h p r e v e n t v f r o m b e i n g i g n o r e d ( e v e n i f X a n d a r e i n d e p e n d e n t a n d g is
n o n i n f o r m a t i v e ) . W e w i l l n o t r e p e a t t h e d i s c u s s i o n o f S e c t i o n 3 . 5 c o n c e r n i n g
w h e n a n d w h y ( a n d h e n c e v ) c a n b e i g n o r e d i n s u c h s i t u a t i o n s .
A f i n a l k i n d o f i n f o r m a t i v e c e n s o r i n g s h o u l d b e m e n t i o n e d , e v e n
t h o u g h it is n o t c e n s o r i n g i n t h e f o r m a l s e n s e w e h a v e d e f i n e d . T h i s is
c e n s o r i n g i n w h i c h c e n s o r e d d a t a is s i m p l y n o t o b s e r v e d o r r e c o r d e d . T h u s ,
f o r t h e c e n s o r i n g m e c h a n i s m d e s c r i b e d i n ( 4 . 3 . 1 ) a n d ( 4 . 3 . 2 ) , it c o u l d b e t h e
c a s e t h a t a n X . > 1 0 is n o t o b s e r v e d o r e v e n k n o w n t o h a v e e x i s t e d . S u c h a
s i t u a t i o n is e a s i l y d e a l t w i t h b y r e c o g n i z i n g t h a t t h e r e l e v a n t p r o b a b i l i t y
d i s t r i b u t i o n o f t h e o b s e r v e d X . is t h e c o n d i t i o n a l d i s t r i b u t i o n , g i v e n t h a t
X . _< 1 0 . T h e c e n s o r i n g m e c h a n i s m w i l l u s u a l l y e n t e r i n t o t h i s c o n d i t i o n a l
d i s t r i b u t i o n i n a n o n i g n o r a b l e f a s h i o n , h o w e v e r .
I n t e r e s t i n g l y e n o u g h , t h i s o m i s s i o n o f d a t a d u e t o c e n s o r i n g c a n
a r i s e f r o m t h e m e t h o d s o f r e p o r t i n g d a t a ( c . f . D a w i d a n d D i c k e y ( 1 9 7 7 ) ) . A n
o b v i o u s e x a m p l e is t h a t o f a t r a d e j o u r n a l w h i c h o n l y p u b l i s h e s r e s u l t s o f
e x p e r i m e n t s w h i c h p r o v i d e " s i g n i f i c a n t " e v i d e n c e a c c o r d i n g t o s o m e c r i t e r i a .
T h e d a t a o f i n t e r e s t , f o r a g i v e n i s s u e , w o u l d b e a l l d a t a f r o m e x p e r i m e n t s o n
t h a t i s s u e , b u t o n l y t h a t d a t a l e a d i n g t o " s i g n i f i c a n c e " w i l l b e c o m e a v a i l a b l e ;
t h e r e s t w i l l b e c e n s o r e d . T h i s is a v e r y c o m p l i c a t e d p r o b l e m , a n d i t i s n o t
a t a l l c l e a r h o w t o a n a l y z e t h e s i t u a t i o n . T h e c e n s o r i n g o f t h e j o u r n a l
c l e a r l y c a n n o t b e i g n o r e d , h o w e v e r .
4 . 4 S I G N I F I C A N C E T E S T I N G
4 . 4 . 1 C o n f l i c t w i t h t h e L P
S i g n i f i c a n c e t e s t i n g o f a h y p o t h e s i s ( u s e d h e r e in t h e s e n s e o f
P v a l u e s , r a t h e r t h a n l e v e l t e s t i n g ) is v i e w e d b y m a n y a s a c r u c i a l e l e m e n t
C O N S E Q U E N C E S A N D C R I T I C I S M S O F T H E L P A N D R L P 1 0 5
o f s t a t i s t i c s , y e t it p r o v i d e s a s t a r t l i n g a n d p r a c t i c a l l y s e r i o u s e x a m p l e o f
c o n f l i c t w i t h t h e L P . A s i g n i f i c a n c e t e s t o f t h e h y p o t h e s i s H
Q
, t h a t X h a s
d i s t r i b u t i o n P , p r o c e e d s b y d e f i n i n g s o m e s t a t i s t i c T ( X ) , w h e r e l a r g e v a l u e s
o f T s u p p o s e d l y c a s t d o u b t on H
Q
, a n d t h e n c a l c u l a t i n g , f o r t h e g i v e n o b s e r v a
t i o n x , t h e s i g n i f i c a n c e l e v e l ( o r P v a l u e ) o f x ,
y :
J
T ( y )
( 4 . 4 . 1 ) p = P ( T ( X ) > T ( x ) ) = f P ( d y )
v :
J
T ( v ) >T ( x ) >
( i . e . , t h e p r o b a b i l i t y u n d e r P o f o b s e r v i n g x o r s o m e t h i n g m o r e " e x t r e m e " ) .
If t h i s i s s m a l l , t h e n o n e s u p p o s e d l y d o u b t s t h a t H
Q
c o u l d b e t r u e . G e n e r a l
d i s c u s s i o n s o f s i g n i f i c a n c e t e s t i n g ( i n c l u d i n g d i s c u s s i o n s o f i m p o r t a n t
p r a c t i c a l i s s u e s s u c h a s " r e a l " v e r s u s " s t a t i s t i c a l " s i g n i f i c a n c e ) c a n b e
f o u n d in E d w a r d s , L i n d m a n , a n d S a v a g e ( 1 9 6 3 ) , H a c k i n g ( 1 9 6 5 ) , M o r r i s o n a n d
H e n k e l ( 1 9 7 0 ) , E d w a r d s ( 1 9 7 2 ) , C o x a n d H i n k l e y ( 1 9 7 4 ) , D e m p s t e r ( 1 9 7 4 a , b ) ,
P r a t t ( 1 9 7 6 , 1 9 7 7 ) , C o x ( 1 9 7 7 ) , B a r n a r d ( 1 9 8 0 ) , G o o d ( 1 9 8 1 ) , B a r n e t t ( 1 9 8 2 ) ,
B e r g e r ( 1 9 8 5 ) , H a l l a n d S e l i n g e r ( 1 9 8 6 ) , a n d B e r g e r a n d D e l a m p a d y ( 1 9 8 7 ) .
A v e r y c o m m o n s e t t i n g f o r s i g n i f i c a n c e t e s t i n g is t h e p a r a m e t r i c
f r a m e w o r k o f t e s t i n g H
Q
: =
Q
v e r s u s Hj: f
Q
. T h e n t h e n u l l d i s t r i b u t i o n ,
P , is s i m p l y P
Q
in o u r u s u a l n o t a t i o n ( o r f () i f d e n s i t i e s e x i s t ) . In
t h i s p a r a m e t r i c s e t t i n g it i s c l e a r t h a t r e p o r t i n g s i g n i f i c a n c e l e v e l s v i o l a t e s
t h e L P , s i n c e s i g n i f i c a n c e l e v e l s i n v o l v e a v e r a g i n g o v e r s a m p l e p o i n t s o t h e r
t h a n j u s t t h e o b s e r v e d x ( s e e ( 4 . 4 . 1 ) ) . T h e e x t r e m e l y s e r i o u s p r a c t i c a l
p r o b l e m s t h a t c a n r e s u l t a r e d i s c u s s e d in S e c t i o n 4 . 4 . 2 .
S i g n i f i c a n c e t e s t i n g is a l s o f r e q u e n t l y u s e d w h e n o n l y a s i n g l e
m o d e l P i s b e i n g c o n t e m p l a t e d . T e s t i n g o f f i t t o a s p e c i f i e d m o d e l is a
c o m m o n e x a m p l e . S i n c e o n l y o n e p r o b a b i l i t y d i s t r i b u t i o n i s t h e n i n v o l v e d ,
t h e r e is n o l i k e l i h o o d f u n c t i o n ; it i s h e n c e o f t e n a r g u e d t h a t t h e L P c a n n o t
a p p l y t o s u c h a s i t u a t i o n . A r g u m e n t s t o t h e c o n t r a r y w i l l b e g i v e n in
S e c t i o n 4 . 4 . 3 .
4 . 4 . 2 A v e r a g i n g O v e r " M o r e E x t r e m e " O b s e r v a t i o n s
T h e l o g i c b e h i n d i n c l u d i n g all d a t a " m o r e e x t r e m e " t h a n t h e g i v e n
x , w h e n c a l c u l a t i n g p , is n o t p a r t i c u l a r l y c o n v i n c i n g . C o n s i d e r t h e f o l l o w i n g
a r t i f i c i a l e x a m p l e , r e l a t e d t o an e x a m p l e in C o x ( 1 9 5 8 ) .
E X A M P L E 3 0 . S u p p o s e , u n d e r P
Q
a n d P
]
, r e s p e c t i v e l y , t h a t X h a s t h e
d i s t r i b u t i o n s g i v e n in t h e f o l l o w i n g t a b l e .
X
P
0
( )
0
.7 5
.7 0
1
.1 4
.2 5
2
.0 4
.0 4
3
.0 37
.0 0 5
4
.0 33 .
.0 0 5
I f T ( x ) = x w e r e u s e d a s t h e t e s t s t a t i s t i c f o r a s i g n i f i c a n c e t e s t o f e i t h e r
P Q o r P | ( i . e . , i f l a r g e x w e r e c o n s i d e r e d " e x t r e m e " ) , a n d i f x = 2 w e r e
o b s e r v e d , t h e n t h e s i g n i f i c a n c e l e v e l a g a i n s t P
Q
a l o n e w o u l d b e
p
o
= P
0
( X >. 2 ) = . 1 1 ,
w h i l e t h e s i g n i f i c a n c e l e v e l a g a i n s t P. a l o n e w o u l d b e
P
1
= P ^ X >_2 ) = . 0 5 .
(We a r e n o t t h i n k i n g h e r e o f t e s t i n g P
Q
v e r s u s Pj t h e f o c u s is o n c o m p a r i n g
s i g n i f i c a n c e t e s t s o f e a c h s e p a r a t e h y p o t h e s i s . ) T h u s x = 2 w o u l d p r o v i d e
" s i g n i f i c a n t e v i d e n c e a g a i n s t P^ a t t h e 5 % l e v e l , " b u t w o u l d n o t e v e n p r o v i d e
" s i g n i f i c a n t e v i d e n c e a g a i n s t P
Q
a t t h e 1 0 % l e v e l . "
T h e c o n c e r n h e r e , o f c o u r s e , is t h a t w e r e P~ a n d P . b e i n g
c o n s i d e r e d s i m u l t a n e o u s l y a s p o s s i b l e m o d e l s , l i k e l i h o o d r e a s o n i n g w o u l d a r g u e
t h a t t h e y a r e e q u a l l y s u p p o r t e d b y x = 2 ; t h e i r l i k e l i h o o d r a t i o is t h e n e q u a l
t o o n e . W h e n c o n s i d e r e d in i s o l a t i o n t h e r e f o r e , i t i s d e f i n i t e l y s t r a n g e t h a t
x = 2 p r o v i d e s s u c h d i f f e r e n t s i g n i f i c a n c e l e v e l s f o r P
Q
a n d P , .
J e f f r e y s ( 1 9 6 1 ) c l e a r l y e x p o s e d t h e q u e s t i o n a b l e l o g i c b e h i n d
s i g n i f i c a n c e l e v e l s , s t a t i n g
" . . . a h y p o t h e s i s w h i c h m a y b e t r u e m a y b e
r e j e c t e d b e c a u s e it h a s n o t p r e d i c t e d
o b s e r v a b l e r e s u l t s w h i c h h a v e n o t o c c u r r e d . "
In t h e e x a m p l e h e r e , n e i t h e r P
Q
n o r P
]
" p r e d i c t s " t h a t x = 3 o r x = 4 w i l l
o c c u r , a n d i n d e e d t h e y d o n o t o c c u r , b u t P
1
w o u l d b e r e j e c t e d a t t h e 5 % l e v e l ,
w h i l e P
Q
w o u l d n o t , b e c a u s e Pj " p r e d i c t s " t h e s e u v i o b Z v z d r e s u l t s e v e n l e s s
t h a n P
Q
.
Q u e s t i o n a b l e l o g i c c o u l d p e r h a p s b e o v e r l o o k e d i f it m a d e l i t t l e
d i f f e r e n c e in p r a c t i c e , b u t h e r e t h e a v e r a g i n g o v e r o t h e r o b s e r v a t i o n s w i l l
v i r t u a l l y a l w a y s h a v e a p r o f o u n d e f f e c t . C o n s i d e r t h e f o l l o w i n g e x a m p l e f r o m
E d w a r d s , L i n d m a n , a n d S a v a g e ( 1 9 6 3 ) .
E X A M P L E 3 0 . 1 . S u p p o s e X = ( X
, . . . , X
n
) , w h e r e t h e X
i
a r e i . i . d . ? 7 ( ,
2
) ,

2
k n o w n . T h e u s u a l t e s t s t a t i s t i c f o r t e s t i n g h L : =
Q
v e r s u s H,: /
Q
i s
T ( X ) = v ^ | X
o
| / ,
w h e r e X" i s t h e s a m p l e m e a n . I f t = T ( x ) i s t h e o b s e r v e d t e s t s t a t i s t i c , t h e
s i g n i f i c a n c e l e v e l is t h e n
P = 2 ( 1 ( t ) ) ,
w h e r e i s t h e s t a n d a r d n o r m a l c . d . f . .
C o n s i d e r , n o w , t h i s t e s t i n g s c e n a r i o f r o m a l i k e l i h o o d p e r s p e c t i v e .
W e r e H , g i v e n b y H , : = ., it w o u l d h a v e b e e n n a t u r a l t o u s e , a s t h e c o m p a r
a t i v e e v i d e n c e f o r t h e t w o h y p o t h e s e s , t h e o b s e r v e d l i k e l i h o o d r a t i o
L
l
= V
X ) /
S
( X )
U n f o r t u n a t e l y , t h e a c t u a l H , c o n s i s t s o f a l l f
Q
, m a k i n g i t d i f f i c u l t t o
d e f i n e a t r u e l i k e l i h o o d r a t i o , L , o f H
Q
t o H,. It s e e m s c l e a r , h o w e v e r , t h a t
a l o w e r b o u n d o n L i s
L = f ( x ) / s u p f ( x ) .
"

0 ?
0

T h e e v i d e n c e a g a i n s t H
Q
is c e r t a i n l y n o s t r o n g e r t h a n L_.
A n e a s y c a l c u l a t i o n s h o w s t h a t , in t h i s e x a m p l e ,
t = e x p { J t
2
} .
T h e f o l l o w i n g t a b l e g i v e s v a l u e s o f 1^ f o r v a r i o u s t , a n d a l s o g i v e s t h e
s i g n i f i c a n c e l e v e l s a s s o c i a t e d w i t h t h e s e t . ( T h e L.g r o w i s d i s c u s s e d l a t e r . )
T a b l e 1 . L i k e l i h o o d R a t i o B o u n d s a n d S i g n i f i c a n c e L e v e l s
t
P
k
kg
1 . 64 5
. 1 0
. 2 5 8
. 64 4
1 . 9 60
. 05
. 1 4 6
. 4 09
2 . 5 76
. 01
. 036
. 1 2 3
3 . 2 9 1
. 001
. 004 4
. 01 8
T h e s u r p r i s e h e r e i s t h a t ^ i s m u c h l a r g e r t h a n p . W h e n p i s . 0 5
f o r i n s t a n c e , l _ i s . 1 4 6 , i n d i c a t i n g t h a t t h e d a t a p r o v i d e s n o m o f i n t h a n 1 t o 7
e v i d e n c e a g a i n s t H
Q
.
j^ i t s e l f c a n b e a r g u e d t o b e m i s l e a d i n g l y s m a l l b e c a u s e i t i s
b a s e d o n m a x i m i z i n g t h e " l i k e l i h o o d o f H j . " M o r e r e a s o n a b l e i s t o u s e , a s
t h e " l i k e l i h o o d o f Hj", a n a v e r a g e o f f
( x ) o v e r a l l f
Q
. T h i s l e a d s t o
a M Q s i g k t c d Z k o J U k o o d h a L L o
L g = f
f t
( x ) / J f
f l
( x ) g ( ) d ,
0 { ^ }

w h e r e g is s o m e d e n s i t y ( o r " w e i g h t f u n c t i o n " ) . A B a y e s i a n w o u l d c h o o s e g
t o b e t h e c o n d i t i o n a l p r i o r d e n s i t y on H j , in w h i c h c a s e L g w o u l d b e t h e
R e g a r d l e s s o f i n t e r p r e t a t i o n , o n e c a n g a i n i n s i g h t i n t o t h e
i m p a c t o f s u c h e v i d e n c e m e a s u r e s b y c a l c u l a t i n g l o w e r b o u n d s o n L g o v e r
r e a s o n a b l e c l a s s e s o f g. F o r i n s t a n c e , in B e r g e r a n d S e l l k e ( 1 9 8 7 ) it i s
s h o w n t h a t f o r a n y d e n s i t y g w h i c h i s a n o n i n c r e a s i n g f u n c t i o n o f | J ,
L g i s a t l e a s t a s l a r g e a s L g , g i v e n in t h e l a s t r o w o f T a b l e 1 . T h e
i n d i c a t i o n is t h u s t h a t , w h e n p = .0 5 s a y , t h e e v i d e n c e a g a i n s t H
Q
i s a c t u a l l y
n o s t r o n g e r t h a n 1 t o 2^. ( A n d i f o n e t r i e d " n a t u r a l " f u n c t i o n s g , o n e w o u l d
f i n d t h a t L g i s t y p i c a l l y 1 o r m o r e w h e n p = . 0 5 ; s e e , e . g . , J e f f r e y s ( 1 9 6 1 ) . )
T h e a b o v e e x a m p l e i s q u i t e d i s t u r b i n g . It i n d i c a t e s t h a t t h e
c l a s s i c a l s t a t i s t i c i a n a n d t h e c o n d i t i o n a l i s t w i l l o f t e n r e a c h \ / e r y d i f f e r e n t
c o n c l u s i o n s w i t h t h e s a m e d a t a , p r e c i s e l y b e c a u s e o n e a v e r a g e s o v e r a l l
" e x t r e m e " s a m p l e p o i n t s w h i l e t h e o t h e r u s e s o n l y t h e o b s e r v e d d a t a . ( B e r g e r
a n d S e l l k e ( 1 9 8 7 ) s p e c i f i c a l l y s h o w t h a t t h i s a v e r a g i n g i s t h e s o u r c e o f t h e
d i s c r e p a n c y . ) F u r t h e r m o r e , t h e d i s c r e p a n c y b e t w e e n s i g n i f i c a n c e l e v e l s a n d
c o n d i t i o n a l m e a s u r e s o f e v i d e n c e ( e . g . , L_, L g o r L g , t h e p o s t e r i o r p r o b a b i l i t y
o f H Q , a n d e v e n c o n d i t i o n a l f r e q u e n t i s t m e a s u r e s c f . B e r g e r a n d S e l l k e
( 1 9 8 7 ) ) h a s b e e n s h o w n t o h o l d in a h u g e v a r i e t y o f s i g n i f i c a n c e t e s t i n g
p r o b l e m s i n v o l v i n g a " p r e c i s e " h y p o t h e s i s . ( H
Q
n e e d n o t b e a p o i n t n u l l f o r
t h e d i s c r e p a n c y t o a r i s e s e e B e r g e r a n d S e l l k e , 1 9 8 7 , a n d B e r g e r a n d
D e l a m p a d y , 1 9 8 7 b u t i f H
Q
i s , s a y , a o n e s i d e d h y p o t h e s i s , t h e n t h e
d i s c r e p a n c y m a y n o t a r i s e s e e C a s e l l a a n d B e r g e r , 1 9 8 7 . ) N o t e a l s o t h a t
t h i s d i s c r e p a n c y is v e r y r e l a t e d ( b u t n o t i d e n t i c a l ) t o " J e f f r e y s ' s P a r a d o x "
o r " L i n d l e y ' s P a r a d o x " . T h e s e i s s u e s a r e e x p l o r e d , in d e p t h , in E d w a r d s ,
L i n d m a n , a n d S a v a g e ( 1 9 6 3 ) , B e r g e r a n d S e l l k e ( 1 9 8 7 ) a n d B e r g e r a n d
D e l a m p a d y ( 1 9 8 7 ) . O t h e r r e l e v a n t w o r k s i n c l u d e L i n d l e y ( 1 9 5 7 , 1 9 7 7 ) ,
J e f f r e y s ( 1 9 6 1 ) , D e G r o o t ( 1 9 7 3 ) , D e m p s t e r ( 1 9 7 4 b ) , D i c k e y ( 1 9 7 7 ) , S m i t h a n d
S p i e g e l h a l t e r ( 1 9 8 0 ) , G o o d ( 1 9 8 1 , 1 9 8 4 ) , S h a f e r ( 1 9 8 2 ) , Z e l l n e r ( 1 9 8 4 ) ,
B e r g e r ( 1 9 8 5 ) , D e l a m p a d y a n d B e r g e r ( 1 9 8 7 ) , a n d D e l a m p a d y ( 1 9 8 6 a , b ) .
O n e d e f e n s e o f a v e r a g i n g o v e r o t h e r o b s e r v a t i o n s ( a n d at t h e s a m e
t i m e a n a t t a c k on t h e L P ) t h a t i s s o m e t i m e s a d v a n c e d i s t h e c l a i m t h a t it is
n e c e s s a r y t o c o n s i d e r w h a t o b s e r v a t i o n s m i g h t k a v e , o c c u r r e d . It i s , h o w e v e r ,
a m i s c o n c e p t i o n t o b e l i e v e t h a t t h e L P f a i l s t o d o t h i s . I n d e e d , in d e t e r m i n
i n g t h e l i k e l i h o o d f u n c t i o n ( o r f a m i l y o f d i s t r i b u t i o n s f o r X ) , it i s c r u c i a l
t o c o n s i d e r a n d c o m p a r e t h e p o s s i b l e x t h a t m i g h t be o b s e r v e d . O n c e t h i s h a s
b e e n d o n e , h o w e v e r , a n d t h e d a t a o b t a i n e d , t h e L P s t a t e s t h a t o n l y t h e o b s e r v e d
a
v
( ) i s n e e d e d ,
x
4 . 4 . 3 T e s t i n g A S i n g l e N u l l M o d e l
W h e n o n l y P h a s b e e n f o r m u l a t e d , it h a s b e e n a r g u e d t h a t s i g n i f
i c a n c e t e s t i n g d o e s n o t v i o l a t e t h e L P b e c a u s e n o t h i n g r e s e m b l i n g a l i k e l i h o o d
f u n c t i o n e x i s t s . A l t h o u g h c o r r e c t in a c e r t a i n f o r m a l s e n s e , t h e r e a r e s e v e r
al w e a k n e s s e s t o t h e a r g u m e n t .
P e r h a p s t h e m o s t s e r i o u s w e a k n e s s f o l l o w s f r o m t h e o b s e r v a t i o n s
in t h e p r e v i o u s s e c t i o n : i f a v e r a g i n g o v e r " e x t r e m e " s a m p l e p o i n t s is
1 0 9 . 1 T H E L I K E L I H O O D P R I N C I P L E
v i r t u a l l y a J b o a y ^ b a d in t e s t i n g a " p r e c i s e " n u l l w h e n a l t e r n a t i v e s a r e g i v e n ,
it s e e m s i n c r e d i b l y o p t i m i s t i c t o b e l i e v e t h a t s u c h a v e r a g i n g w i l l b e r e a s o n
a b l e w h e n a l t e r n a t i v e s a r e n o t g i v e n . T h e a r g u m e n t t h a t " s i g n i f i c a n c e t e s t i n g
is t h e o n l y a v a i l a b l e s t a t i s t i c a l p r o c e d u r e " i s h a r d l y p e r s u a s i v e w h e n it is
k n o w n t h a t t h i s a v a i l a b l e s t a t i s t i c a l p r o c e d u r e i s b a d f o r t e s t i n g p r e c i s e
h y p o t h e s e s .
A s e c o n d w e a k n e s s o f t h e a r g u m e n t t h a t o n l y P e x i s t s is t h a t
i m p l i c i t a l t e r n a t i v e s t o P o f t e n a r e p r e s e n t . I n d e e d , a l t e r n a t i v e s m u s t
e n t e r , a t l e a s t i n f o r m a l l y , i n t o t h e c h o i c e o f t h e t e s t s t a t i s t i c T ( x ) .
F o r i n s t a n c e , i n E x a m p l e 3 0 it s e e m s j u s t i f i a b l e t o u s e T ( x ) =x t o m e a s u r e
" e x t r e m e " o n l y i f t h e a l t e r n a t i v e s t h a t o n e h a s in m i n d a r e , s a y , a l t e r n a t i v e s
w h i c h a r e s t o c h a s t i c a l l y l a r g e r t h a n P
Q
( s o t h a t a l a r g e x t e n d s t o s u p p o r t
t h e a l t e r n a t i v e s m o r e t h a n it t e n d s t o s u p p o r t P Q . ) A S a n o t h e r e x a m p l e o f
t h e i m p l i c i t p r e s e n c e o f a l t e r n a t i v e s , c o n s i d e r c h i s q u a r e t e s t i n g o f f i t .
E X A M P L E 3 0 . 2 . C o n s i d e r a s t a t i s t i c a l e x p e r i m e n t in w h i c h n i n d e p e n d e n t a n d
i d e n t i c a l l y d i s t r i b u t e d r a n d o m q u a n t i t i e s X,, X 2 , ..., X
n
a r e o b s e r v e d f r o m
a d i s t r i b u t i o n F. I t is d e s i r e d t o c o n d u c t a s i g n i f i c a n c e t e s t o f t h e
h y p o t h e s i s H
Q
: F = F
Q
, w h e r e F
Q
i s a s p e c i f i e d d i s t r i b u t i o n . A c o m m o n t e s t
p r o c e d u r e , w h e n n o a l t e r n a t i v e s a r e s p e c i f i e d , i s t h e c h i s q u a r e t e s t o f f i t .
C k S g u a e . T e t P o c ^ d u . 2 , : F i r s t , a p a r t i t i o n a . ^ . Q o f t h e r e a l l i n e i s
s e l e c t e d . T h e n t h e s a m p l e f r e q u e n c i e s o f t h e n o b s e r v a t i o n s in t h e c e l l s o f
t h e p a r t i t i o n a r e c a l c u l a t e d . L e t z = ( z p ..., z
m
)
t
d e n o t e t h e s e f r e q u e n c i e s ;
t h u s z
i
= n u m b e r o f X ^ s i n ( a
i
,, a . ] . D e f i n e
1
=F ( a
i
) F a ^ , ) = P
F
( a
i
_
1
< X <
a
i
) ,
?
and
T h e n t h e c h i s q u a r e t e s t p r o c e d u r e i s t o c a l c u l a t e t h e t e s t s t a t i s t i c
C O N S E Q U E N C E S A N D C R I T I C I S M S O F T H E L P A N D R L P 1 0 9 . 2
m ( z . n j )
2
t =

i=l n V
a n d a p p r o x i m a t e t h e s i g n i f i c a n c e l e v e l b y
2
w h e r e , is a c h i s q u a r e r a n d o m v a r i a b l e w i t h m 1 d e g r e e s o f f r e e d o m .
T h e i m p l i e d a l t e r n a t i v e s h e r e a r i s e f r o m t h e f a c t t h a t Z h a s a
U i x L t i o m l a J ( n , ) d i s t r i b u t i o n , s o t h a t b a s i n g t h e t e s t o n z is e q u i v a l e n t
t o a c k n o w l e d g i n g t h e t e s t t o b e t h a t o f H
Q
: = v e r s u s H , : ^
0
. ( U s e o f
t c a n b e a r g u e d t o f u r t h e r i m p l y t h a t t h e a l t e r n a t i v e s , f , a r e r o u g h l y
m
0 2 0
o r d e r e d in p l a u s i b i l i t y a c c o r d i n g t o = ( V ) / V , S O t h a t o n e is
i=l

r e a l l y t e s t i n g H
Q
: = 0 v e r s u s H ,: > 0 . ) B u t t h i s is a p a r a m e t r i c p r o b l e m
w i t h s p e c i f i e d a l t e r n a t i v e s ( a n d h e n c e a l i k e l i h o o d f u n c t i o n ) s o t h a t
L P c o m p a t i b l e t e s t i n g m e t h o d s c a n a p p l y . I n d e e d , in D e l a m p a d y a n d B e r g e r
( 1 9 8 7 ) it is s h o w n t h a t t h e s a m e t y p e o f d i f f i c u l t y f o r s i g n i f i c a n c e t e s t i n g ,
t h a t w a s d i s c u s s e d in S e c t i o n 4 . 4 . 2 , e x i s t s h e r e : t h e s i g n i f i c a n c e l e v e l i s
t y p i c a l l y m u c h s m a l l e r t h a n s e n s i b l e c o n d i t i o n a l m e a s u r e s o f t h e e v i d e n c e
f o r H
o
.
T h e a b o v e a r g u m e n t , t h a t t h e r e a r e i m p l i c i t a l t e r n a t i v e s in
s i g n i f i c a n c e t e s t i n g , c a n a c t u a l l y b e g i v e n a q u i t e g e n e r a l f o r m a l f o u n d a t i o n .
It h a s p r e v i o u s l y b e e n m e n t i o n e d t h a t t h e a c t u a l s a m p l e s p a c e # w i l l b e
d i s c r e t e in p r a c t i c e . B u t t h e n , a s d i s c u s s e d in S e c t i o n 3 . 6 . 1 , e v e n t h e
s e t { P
} o f o J U L d i s t r i b u t i o n s o n X a c t u a l l y r e s u l t s in a d e f i n a b l e l i k e l i h o o d
f u n c t i o n . F u r t h e r m o r e , a s i g n i f i c a n c e t e s t o f P c a n b e i d e n t i f i e d w i t h a
t e s t o f H
n
: =
n
v e r s u s H
: f
, w h e r e P
= P . T h u s t h e L P c a n a p p l y ,
u u i
a n d a r g u e s a g a i n s t t h e u s e o f s i g n i f i c a n c e l e v e l s .
A l t h o u g h f o r m a l l y c o r r e c t , w e d o n o t a s c r i b e m u c h p r a c t i c a l
i m p o r t a n c e t o t h i s l a s t a r g u m e n t , b e c a u s e t h e c l a s s o f a Z Z a l t e r n a t i v e s t o
P is t y p i c a l l y m u c h t o o b i g t o s u g g e s t a s e n s i b l e a n a l y s i s . In p r a c t i c e ,
s o m e c o n s i d e r a t i o n o f t h e t y p e o f a l t e r n a t i v e s t h a t a r e e x p e c t e d is n e c e s s a r y ,
e v e n in c l a s s i c a l s i g n i f i c a n c e t e s t i n g . In c h o o s i n g a t e s t s t a t i s t i c T ( x ) ,
f o r i n s t a n c e , w e e a r l i e r o b s e r v e d t h a t it i s o f t e n n e c e s s a r y t o c o n s i d e r
a l t e r n a t i v e s w h e n d e f i n i n g " e x t r e m e . " It h a s b e e n a r g u e d t h a t it m a y b e
e a s i e r t o g u e s s a r e a s o n a b l e T , r e f l e c t i n g i n t u i t i v e j u d g e m e n t s a s t o w h i c h
o b s e r v a t i o n s s u p p o r t H
Q
a n d w h i c h s u p p o r t a l t e r n a t i v e s , t h a n t o a t t e m p t
e x p l i c i t c o n s i d e r a t i o n o f a l t e r n a t i v e s a n d c o n s t r u c t i o n o f T b y , s a y , l i k e l i
h o o d r a t i o c o m p a r i s o n s o f P w i t h t h e a l t e r n a t i v e s . T h e a r g u m e n t t h a t o n e
c a n d o b e t t e r b y u s e o f i n t u i t i o n , t h a n b y e x p l i c i t c o n s i d e r a t i o n o f i m p o r t a n t
r e l e v a n t f e a t u r e s o f a p r o b l e m ( h e r e , t h e a l t e r n a t i v e s ) , i s d i f f i c u l t t o
r e f u t e , b u t i s a n a r g u m e n t t h a t w e w o u l d f e e l y e r y u n c o m f o r t a b l e h a v i n g a s a
b a s i s f o r o u r a p p r o a c h t o s c i e n c e a n d u n d e r s t a n d i n g . E v e n m o r e t r o u b l i n g i s
t h e f a c t t h a t s i g n i f i c a n c e t e s t i n g a l l o w s o n e t o " h i d e " t h i s u s e o f p e r s o n a l
i n t u i t i o n . T h u s , w h i l e P r a t t ( 1 9 6 5 ) a d m i t s t h a t c o n s i d e r a t i o n o f a l t e r n a t i v e s
c a n b e h a r d a n d a s o u r c e o f c o n t r o v e r s y in m a n y s i t u a t i o n s d e a l t w i t h b y
s i g n i f i c a n c e t e s t i n g , h e a r g u e s t h a t
" C o m p u t i n g a P v a l u e r u n s t h e d a n g e r o f
h i d i n g t h i s r e a l u n c e r t a i n t y a n d l e g i t i m a t e
d i s a g r e e m e n t b e h i n d a s c r e e n o f i r r e l e v a n t
p r e c i s i o n . "
A s a f i n a l p o i n t , it h a s b e e n e x t e n s i v e l y a r g u e d ( c f . H a c k i n g
( 1 9 6 5 ) ) t h a t o n e c a n n e v e r r e a l l y r e j e c t P u n t i l o n e h a s s o m e t h i n g b e t t e r ,
n a m e l y a n o t h e r m o d e l P w h i c h i s b o t h " r e a s o n a b l e " a n d b e t t e r s u p p o r t e d b y
t h e d a t a . In E x a m p l e 3 0 , f o r i n s t a n c e , t h e o b s e r v a t i o n x = 2 i s q u i t e u n l i k e l y
t o o c c u r u n d e r P Q , b u t it is e q u a l l y u n l i k e l y t o o c c u r u n d e r P , t h u s i f P
n
a n d Pj a r e k n o n t o b e t h e o n l y p o s s i b i l i t i e s , t h e n x = 2 p r o v i d e s n o e v i d e n c e
a g a i n s t P Q . T h u s c o n s i d e r a t i o n o f a l t e r n a t i v e s i s i m p e r a t i v e i f o n e a c t u a l l y
s e e k s t o r e j e c t P .
4 . 4 . 4 C o n c l u s i o n s
W h a t i s t o b e c o n c l u d e d a b o u t s i g n i f i c a n c e t e s t i n g ? F i r s t o f a l l ,
it s h o u l d b e a d m i t t e d t h a t , a s t h e s i g n i f i c a n c e l e v e l ( o r P v a l u e ) d e c r e a s e s ,
t h e e v i d e n c e a g a i n s t H
Q
w i l l b e i n c r e a s i n g ( a s s u m i n g t h a t T h a s b e e n c h o s e n
C O N S E Q U E N C E S A N D C R I T I C I S M S O F T H E L P A N D R L P 1 0 9 . 4
a p p r o p r i a t e l y ) . I n d e e d , in a f e w s p e c i a l s i t u a t i o n s ( p r i m a r i l y o n e - s i d e d
t e s t i n g s i t u a t i o n s ) t h e s i g n i f i c a n c e l e v e l c a n c o r r e s p o n d t o a r e a s o n a b l e
c o n d i t i o n a l ( B a y e s i a n ) m e a s u r e o f t h e v a l i d i t y o f H
Q
( c f . J e f f r e y s ( 1 9 6 1 ) ,
P r a t t ( 1 9 6 5 ) , D e G r o o t ( 1 9 7 3 ) , F r a s e r a n d M a c k a y ( 1 9 7 6 ) , D i c k e y ( 1 9 7 7 ) ,
Z e l l n e r ( 1 9 8 2 ) , a n d C a s e l l a a n d B e r g e r ( 1 9 8 7 ) ) . I n g e n e r a l t h o u g h , t h e
m a g n i t u d e o f a s i g n i f i c a n c e l e v e l n e e d b e a r n o r e l a t i o n s h i p ( f r o m p r o b l e m
t o p r o b l e m ) t o t h e a c t u a l a m o u n t o f e v i d e n c e a g a i n s t H
Q
, a n d s i g n i f i c a n c e
l e v e l s in t e s t i n g p r e c i s e h y p o t h e s e s a r e t y p i c a l l y s o m i s l e a d i n g l y s m a l l
t h a t t h e i r u s e f o r a c t u a l l y r e j e c t i n g a h y p o t h e s i s i s s t r o n g l y c o n t r a i n d i c a t e d
A l t h o u g h a g i v e n s i g n i f i c a n c e l e v e l c a n m e a n v a s t l y d i f f e r e n t
t h i n g s in d i f f e r e n t s i t u a t i o n s , it c a n b e a r g u e d t h a t , t h r o u g h f r e q u e n t u s e
in v a r i o u s s i t u a t i o n s , i n s i g h t i n t o i t s t r u e s t r e n g t h o f e v i d e n c e a g a i n s t H Q
c a n b e o b t a i n e d . T h i s i s p e r h a p s t r u e : c a p a b l e p e o p l e c a n b e c o m e y e r y g o o d
a t d o i n g t a s k s w i t h g r o s s l y i n a d e q u a t e t o o l s . T h i s is n o t t o s a y , h o w e v e r ,
t h a t b e t t e r t o o l s s h o u l d b e i g n o r e d o r , m o r e i m p o r t a n t l y , t h a t i n e x p e r i e n c e d
p e o p l e w i l l d o w e l l w i t h t h e i n a d e q u a t e t o o l s .
O n e p o s s i b l y v a l i d u s e o f s i g n i f i c a n c e t e s t i n g is t o p r o v i d e an
a l e r t t h a t f u r t h e r i n v e s t i g a t i o n (in p a r t i c u l a r c o n s i d e r a t i o n o f a l t e r n a t i v e s )
is n e e d e d . A s B a r n a r d ( 1 9 8 1 ) s a y s
" T h e q u e s t i o n t o b e a n s w e r e d is w h e t h e r
t h e f e a t u r e ( T ( x ) ) p r e s e n t e d i s so
i m p r o b a b l e on H
Q
a s t o j u s t i f y t h e e f f o r t
i n v o l v e d in e x e r c i s i n g o u r i m a g i n a t i o n t o
p r o d u c e a n h y p o t h e s i s t h a t c o u l d a c c o u n t
f o r i t . "
T h e r e i s n o g u a r a n t e e f r o m a s m a l l s i g n i f i c a n c e l e v e l t h a t P is w r o n g ( i . e . ,
t h a t an a l t e r n a t i v e h y p o t h e s i s c a n b e f o u n d w h i c h is s u b s t a n t i a l l y m o r e
s u p p o r t e d b y t h e d a t a ) , b u t w i t h o u t a s m a l l s i g n i f i c a n c e l e v e l t h e r e m a y b e
n o n e e d t o l o o k p a s t P . T h i s u s e o f s i g n i f i c a n c e t e s t i n g c a n b e a r g u e d t o
b e i m p o r t a n t e v e n t o B a y e s i a n s , a s e x t e n s i v e l y d i s c u s s e d in B o x ( 1 9 8 0 ) : f o r
a g i v e n m o d e l a n d p r i o r , t h e m a r g i n a l ( o r p r e d i c t i v e ) d e n s i t y o f X c a n b e
u s e d t o c o n d u c t a s i g n i f i c a n c e t e s t w h i c h c o u l d a l e r t o n e t o q u e s t i o n t h e
m o d e l o r p r i o r .
O f c o u r s e , e v e n t h i s u s e o f s i g n i f i c a n c e t e s t i n g a s a n a l e r t
c o u l d b e q u e s t i o n e d , b e c a u s e o f t h e m a t t e r o f a v e r a g i n g o v e r u n o b s e r v e d x .
It is h a r d t o s e e w h a t e l s e c o u l d b e d o n e w i t h P a l o n e . , h o w e v e r , a n d it is
s o m e t i m e s a r g u e d t h a t t i m e c o n s t r a i n t s p r e c l u d e c o n s i d e r a t i o n o f a l t e r n a t i v e s .
T h i s m a y o c c a s i o n a l l y b e t r u e , b u t is p r o b a b l y f a i r l y r a r e . E v e n c u r s o r y
c o n s i d e r a t i o n o f a l t e r n a t i v e s a n d a f e w r o u g h l i k e l i h o o d r a t i o c a l c u l a t i o n s
w i l l t e n d t o g i v e s u b s t a n t i a l l y m o r e i n s i g h t t h a n w i l l a s i g n i f i c a n c e l e v e l ,
a n d w i l l u s u a l l y n o t b e m u c h m o r e d i f f i c u l t t h a n s e n s i b l y c h o o s i n g T a n d
c a l c u l a t i n g t h e s i g n i f i c a n c e l e v e l . ( S e e a l s o D e m p s t e r ( 1 9 7 4 b ) . )
A d m i t t e d l y , s u c h an a p p r o a c h w i l l b e s o m e w h a t i m p r e c i s e , b u t w h a t i s t h e
a d v a n t a g e o f " i r r e l e v a n t p r e c i s i o n " ?
4 . 5 R A N D O M I Z A T I O N A N A L Y S I S
4 . 5 . 1 I n t r o d u c t i o n
In c l a s s i c a l f i n i t e p o p u l a t i o n s a m p l i n g ( o r s u r v e y s a m p l i n g ) a n d
r a n d o m i z a t i o n t e s t i n g , t h e r a n d o m i z a t i o n in t h e e x p e r i m e n t a l d e s i g n ( u s e d t o
s e l e c t t h e s a m p l e o r a l l o c a t e t r e a t m e n t s ) is a d o m i n a n t f a c t o r in t h e
c o n s t r u c t i o n o f m e a s u r e s o f e v i d e n c e a b o u t . T h e s e m e a s u r e s a r e
p r e e x p e r i m e n t a l in n a t u r e , a n d t h e i r u s e d i r e c t l y v i o l a t e s t h e L P a n d R L P .
( T h e o u t c o m e o f t h e r a n d o m i z a t i o n is u s u a l l y k n o w n , a n d h e n c e a v e r a g i n g o v e r
s a m p l e s o r t r e a t m e n t a l l o c a t i o n s t h a t m i g h t h a v e o c c u r r e d i s s u p p o s e d l y
i r r e l e v a n t . ) H e n c e , b e l i e f in t h e L P w o u l d h a v e a p r o f o u n d e f f e c t o n o n e ' s
v i e w o f t h e s e a r e a s o f s t a t i s t i c s .
P e r h a p s n o t s u r p r i s i n g l y , it is in t h e s e a r e a s , s o d r a s t i c a l l y
a f f e c t e d b y t h e L P , t h a t s o m e o f t h e s t r o n g e s t i n t u i t i v e a r g u m e n t s a g a i n s t
t h e L P c a n b e r a i s e d . T h e i s s u e s i n v o l v e d a r e y e r y c o m p l e x , s o m u c h s o t h a t
a l l w e c a n h o p e t o d o i s s k i m t h e s u r f a c e o f t h e s u b j e c t . I n d e e d , w e w i l l
e s s e n t i a l l y r e s t r i c t o u r s e l v e s t o a d e f e n c e o f t h e L P in a f e w s i m p l e
e x a m p l e s , t r y i n g t o e s t a b l i s h , a s p l a u s i b l e , t h e a r g u m e n t t h a t a n y t h i n g
s e n s i b l e in r a n d o m i z a t i o n a n a l y s i s is s e n s i b l e p r e c i s e l y b e c a u s e it h a s a
s e n s i b l e i n t e r p r e t a t i o n f r o m a l i k e l i h o o d v i e w p o i n t .
A l t h o u g h o u r m a i n e m p h a s i s w i l l n o t b e on c r i t i c i z i n g
r a n d o m i z a t i o n a n a l y s i s , it i s i m p o r t a n t t o k e e p s e v e r a l i s s u e s in m i n d .
F i r s t , r a n d o m i z a t i o n a n a l y s i s c a n c l e a r l y b e \ / e r y s i l l y c o n d i t i o n a l l y , i f
f o l l o w e d b l i n d l y . E v e n p r o p e r r a n d o m i z a t i o n c a n r e s u l t ( b y b a d l u c k ) in
t r e a t m e n t g r o u p s u n b a l a n c e d w i t h r e s p e c t t o u n a n t i c i p a t e d ( b u t o b s e r v e d t o
b e i m p o r t a n t ) c o v a r i a t e s , o r in a s a m p l e w h i c h is c l e a r l y u n r e p r e s e n t a t i v e
o f t h e p o p u l a t i o n , a n d y e t c l a s s i c a l r a n d o m i z a t i o n a n a l y s i s d o e s n o t t r e a t
s u c h s i t u a t i o n s a n y d i f f e r e n t l y t h a n s i t u a t i o n s w h e r e t h e o u t c o m e o f t h e
r a n d o m i z a t i o n is " g o o d " . T h u s , i f o n e r a n d o m l y s a m p l e s f r o m t h e p o p u l a t i o n
o f v o t e r s in a s u r v e y o n p r e f e r e n c e in t h e n e x t P r e s i d e n t i a l e l e c t i o n a n d
f i n d s t h a t , u n f o r t u n a t e l y , a l l m e m b e r s o f t h e r a n d o m s a m p l e h a p p e n t o b e
R e p u b l i c a n , it is p e r m i s s i b l e ( c l a s s i c a l l y ) t o i g n o r e t h i s f a c t a n d p r o c e e d
w i t h t h e u s u a l a n a l y s i s . A s e c o n d p r o b l e m w i t h r a n d o m i z a t i o n a n a l y s i s ( o r
a t l e a s t r a n d o m i z a t i o n t e s t i n g ) i s t h a t it is o f t e n i m p l e m e n t e d t h r o u g h
s i g n i f i c a n c e t e s t i n g , a n d t h e s e r i o u s c o n c e r n s o f t h e p r e v i o u s s e c t i o n t h e n
a p p l y . T h e t h i r d , a n d m o s t i m p o r t a n t , p r o b l e m i s t h a t r a n d o m i z a t i o n a n a l y s i s
d o o v i o l a t e , t h e L P . In m u r k y s i t u a t i o n s , w h e r e i n t u i t i o n s t u m b l e s , i t s e e m s
e s p e c i a l l y n e c e s s a r y t o d e p e n d o n f o u n d a t i o n s .
B e c a u s e o f t h e a b o v e ( a n d v a r i o u s s p e c i f i c ) c r i t i c i s m s o f
r a n d o m i z a t i o n a n a l y s i s , s u c h a n a l y s i s is u s u a l l y a d v a n c e d , n o t a s an a l w a y s
s o u n d w a y o f p r o c e e d i n g , b u t a s t h e m o s t u s e f u l p r a c t i c a l m e t h o d o f o b t a i n i n g
a r e a s o n a b l e a n s w e r . W e w i l l t r y t o a r g u e t h a t t h e c a s e f o r t h i s i s w e a k ,
a t b e s t .
O f c o u r s e , e v e n t h o u g h w e a r g u e t h a t t h e b a s i s o f r a n d o m i z a t i o n
a n a l y s i s i s f u n d a m e n t a l l y in e r r o r , m a n y o f t h e s p e c i f i c p r o c e d u r e s u s e d in
s u r v e y s a m p l i n g a n d r a n d o m i z a t i o n t e s t i n g a r e p e r f e c t l y s a t i s f a c t o r y . ( I f s o ,
h o w e v e r , it i s p r o b a b l y b e c a u s e t h e y h a v e s o m e s e n s i b l e i n t e r p r e t a t i o n
c o n s i s t e n t w i t h t h e L P . ) A l s o , t h e v a l u e o f r a n d o m i z a t i o n i t s e l f , in
t r e a t m e n t a l l o c a t i o n a n d t h e c h o i c e o f a s a m p l e , i s n o t b e i n g a d d r e s s e d h e r e .
S u c h r a n d o m i z a t i o n is o f t e n a r g u e d t o b e v a l u a b l e ( e v e n b y m a n y c o n d i t i o n a l
i s t s ) in h e l p i n g t o r e d u c e s y s t e m a t i c e f f e c t s t h a t p e r h a p s m i g h t u n w i t t i n g l y
b e i n t r o d u c e d b y t h e e x p e r i m e n t a l d e s i g n o r s a m p l i n g p l a n . A n e x p e r i m e n t
in w h i c h r a n d o m i z a t i o n is u s e d p r o p e r l y w i l l , m o s t o f t h e t i m e , t u r n o u t
t o b e r e a s o n a b l y b a l a n c e d w i t h r e s p e c t t o e x p e r i m e n t a l l y i n d u c e d ( a n d
u n a n t i c i p a t e d ) c o v a r i a t e s . R a n d o m i z a t i o n a l s o h e l p s g r e a t l y in c o n v i n c i n g
o t h e r s , w h o d o n o t h a v e a c c e s s t o t h e e x p e r i m e n t a l s e t u p o r d a t a , t h a t n o
s y s t e m a t i c b i a s e s w e r e p r e s e n t . E m p l o y i n g m e a s u r e s o f e v i d e n c e b a s e d o n t h e
r a n d o m i z a t i o n p r o b a b i l i t i e s is an e n t i r e l y d i f f e r e n t m a t t e r , h o w e v e r .
I n d e e d , a c o n d i t i o n a l i s t w i l l n o t o n l y i g n o r e t h e r a n d o m i z a t i o n p r o b a b i l i t i e s ,
s i n c e t h e o u t c o m e o f t h e r a n d o m i z a t i o n i s k n o w n , b u t w i l l a l s o c h e c k t o s e e
t h a t b a l a n c e w i t h r e s p e c t t o i m p o r t a n t n e w c o v a r i a t e s w a s i n d e e d o b t a i n e d .
4 . 5 . 2 F i n i t e P o p u l a t i o n S a m p l i n g
A t y p i c a l c l a s s i c a l s e t u p i s t h a t o f h a v i n g a p o p u l a t i o n
V
=
{ y j , . . . , y ^ } o f N u n i t s , w h e r e e a c h u n i t y . c a n b e r e p r e s e n t e d a s a
v e c t o r y.j = (u , v . ) u.j r e p r e s e n t i n g a l a b e l ( o r o t h e r k n o w n i n f o r m a t i o n )
a b o u t t h e u n i t a n d v
r e p r e s e n t i n g s o m e t h i n g u n k n o w n ( b u t o b s e r v a b l e ) . It
i s d e s i r e d t o i n f e r s o m e t h i n g a b o u t = ( V , , . . . , V M ) , f r o m a s a m p l e
V
s
=
( y
> y j >. . . >y i )> w h i c h i s a s u b s e t o f % h e r e s = {ij. , i

m
} c l =
{ 1 , . . . , N } i n d i c a t e s w h i c h u n i t s f r o m t h e p o p u l a t i o n a r e s e l e c t e d t o b e p a r t
o f t h e s a m p l e . N o t e t h a t it is t y p i c a l l y a l s o p o s s i b l e t o u s e t h e k n o w n
l a b e l s u = ( u j , . . . , u
N
) in m a k i n g i n f e r e n c e s a b o u t . L e t S d e n o t e t h e
c o l l e c t i o n o f a l l s u b s e t s o f I , a n d s u p p o s e P i s a p r o b a b i l i t y d i s t r i b u t i o n
on S . A p r o c e d u r e ( ^
s
, u ) i s t o b e u s e d , a n d s o m e c r i t e r i o n f u n c t i o n
L ( ( ^
s
, u ) , ) e m p l o y e d . F i n a l l y , t h e o v e r a l l s t a t i s t i c a l p r o c e d u r e ( P , ) , b y
w h i c h it i s m e a n t t h a t s w i l l b e c h o s e n a c c o r d i n g t o t h e p r o b a b i l i t y
d i s t r i b u t i o n P o n S a n d ( ^
s
, u ) w i l l b e u s e d , i s e v a l u a t e d c l a s s i c a l l y b y t h e
f r e q u e n t i s t m e a s u r e o f p e r f o r m a n c e
R ( P , , ) = L ( ( v , u ) , ) P ( s ) .
N
E X A M P L E 3 1 . S u p p o s e it i s d e s i r e d t o e s t i m a t e t h e p o p u l a t i o n t o t a l = v ,
i=l

u s i n g a n e s t i m a t o r ( V u ) a n d s q u a r e d e r r o r l o s s
L ( ( V
s 5
u ) , ) = ( ( V
s
, u ) )
2
. S u p p o s e P ( s ) = 1/(JJ) f o r a l l s S o f s i z e n ,
c o r r e s p o n d i n g t o s e l e c t i o n o f a s i m p l e r a n d o m s a m p l e o f s i z e n . T h e e s t i m a t o r
( 4 . 5 . 1 ) ( V
s
, u ) = Jj v.
S
~
n
j =l

j
( r e c a l l t h a t % = ( ( u ,v ) , . . . , ( u . ,v_. ) ) ) is u n b i a s e d in t h e s e n s e t h a t
s
l

l

n

n
a ( V , u ) P ( s ) = ,
s t S
s
~
a n d h e n c e R ( P , , ) c a n b e c o n s i d e r e d t o b e t h e v a r i a n c e o f t h e p r o c e d u r e ( P , ) ,
w e r e it r e p e a t e d l y u s e d .
T o i n v e s t i g a t e t h i s s i t u a t i o n f r o m t h e v i e w p o i n t o f l i k e l i h o o d ,
n o t e t h a t t h e o n l y r a n d o m n e s s h e r e is in t h e g e n e r a t i o n o f s , a n d h e n c e t h a t
P ( s )
w h e r e
Q
is t h e s e t o f a l l p o s s i b l e v e c t o r s , ^
s
> w h i c h c o u l d a r i s e a s s a m p l e s
f o r t h e g i v e n u a n d . ( N o t e t h a t t h e i m p l i c i t s a m p l e s p a c e i s t h e u n i o n ,
o v e r a l l , o f s u c h . ) T h u s t h e l i k e l i h o o d f u n c t i o n f o r , w h e n ^
s
i s
o b s e r v e d , is s i m p l y
fP(s) i f U
( 4 . 5 . 2 ) P ( I ) = J
S
1
( 4 . 5 . 3 )
where ( f o r ^
s
= ( ( u , v ) , . . . , ( u
i
, v
i
) ) )
1 1 m m
( V ) = G : f or j = l , . . . , m, t he i component of e qual s v } .
J
j
S i n c e ( ) i s c o n s t a n t f o r G ( ^ ) , it c o n v e y s n o i n f o r m a t i o n a b o u t ,
o t h e r t h a n t h a t t h e p a r t o f o b s e r v e d (in ^
$
) is k n o w n . T h i s is d e e m e d b y
s o m e t o b e a f a i l u r e o f t h e L P , in t h a t t h e s t a t i s t i c a l p r o c e d u r e i s t h o u g h t
t o p r o v i d e c o n s i d e r a b l e i n f o r m a t i o n a b o u t t h a t p a r t o f n o t o b s e r v e d in ^
s
,
c a l l it * .
T h e l i k e l i h o o d ( o r m a y b e B a y e s i a n ) v i e w is i n d e e d , t h a t t h e d a t a
c o n t a i n s n o i n h e r e n t i n f o r m a t i o n a b o u t * , a n d t h a t t h e o n l y w a y o f i n f e r i n g
a n y t h i n g a b o u t * is t o r e l a t e it s o m e h o w t o t h e o b s e r v e d s a m p l e . V a r i o u s
r e l a t i o n s h i p s w h i c h m i g h t b e d e e m e d r e a s o n a b l e a r e :
( i ) A l l v . a r e t h o u g h t t o b e s i m i l a r , a n d t h e l a b e l s u
i
c o n t a i n n o
i n f o r m a t i o n . O f t h e m a n y w a y s t o m o d e l t h i s , a s i m p l e ( o f t e n t o o
s i m p l e ) p o s s i b i l i t y is t o p r e s u m e t h a t t h e v a r e i n d e p e n d e n t
2 2
o b s e r v a t i o n s f r o m a 7 ( , ) d i s t r i b u t i o n . T h e n e s t i m a t e a n d ,
u s i n g t h e s a m p l e y , a n d i n f e r w h a t e v e r i s d e s i r e d a b o u t . In t h e
s i t u a t i o n o f E x a m p l e 3 1 , t h e a n s w e r s w o u l d b e e s s e n t i a l l y t h e s a m e
a s t h e c l a s s i c a l a n s w e r s ,
( i i ) S u p p o s e t h e v a r e t h o u g h t t o b e l i n e a r l y r e l a t e d t o t h e u , s a y
v
i
= + 3 u
i
+
i 9
w h e r e t h e a r e p r e s u m e d t o h a v e s o m e d i s t r i b u t i o n . C l e a r l y a q u i t e
d i f f e r e n t a n a l y s i s w o u l d b e a p p r o p r i a t e .
( i i i ) S u p p o s e t w o d i s t i n c t s i m i l a r g r o u p s w i t h i n t h e p o p u l a t i o n c a n b e
i d e n t i f i e d f r o m ^ K n o w l e d g e a b o u t e a c h g r o u p c a n b e o b t a i n e d
f r o m V , a s in ( i ) , a n d t h e p r o p o r t i o n o f e a c h g r o u p in t h e p o p u l a t i o n
e s t i m a t e d . ( O f c o u r s e , a s t r a t i f i e d s a m p l e w o u l d p r o b a b l y h a v e b e e n
d e s i r a b l e h a d t h e g r o u p s b e e n i d e n t i f i a b l e s o l e l y f r o m t h e l a b e l s . )
( i v ) S u p p o s e it is f e l t t h a t t h e s a m p l e d o e s n o t l o o k t y p i c a l o f t h e
r e m a i n d e r o f t h e p o p u l a t i o n . (A n u n l u c k y s a m p l e w a s d r a w n , o r t h e
s a m p l e r e v e a l e d an u n a n t i c i p a t e d b i a s in t h e s a m p l i n g p l a n . ) It is
n o t c l e a r w h a t t o d o , b u t it c e r t a i n l y c a n n o t b e r i g h t t o p r o c e e d
w i t h a c l a s s i c a l a n a l y s i s , a s i f t h e s a m p l e w a s s a t i s f a c t o r y .
In t h e s i t u a t i o n s a b o v e , c l a s s i c a l s a m p l i n g t h e o r i s t s w o u l d , o f
c o u r s e , r e c o m m e n d d i f f e r e n t p r o c e d u r e s f o r t h e v a r i o u s p r e s u m e d m o d e l s . T h e
p o i n t o f t h e d i s c u s s i o n i s t o i n d i c a t e t h a t t h e d a t a , ' y
9
r e a l l y d o e s n ' t s a y
a n y t h i n g a b o u t * , u n l e s s t h e r e is s o m e b a c k g r o u n d i n f o r m a t i o n r e l a t i n g t h e
d a t a t o t h e p o p u l a t i o n . It m i g h t b e a r g u e d t h a t , e v e n w h e n n o t h i n g is k n o w n
a b o u t t h e p o p u l a t i o n , a s i m p l e r a n d o m s a m p l e w i l l p r o b a b l y p r o d u c e a r e p r e s e n
t a t i v e s u b s e t o f t h e p o p u l a t i o n , s o t h a t a n e s t i m a t o r s u c h a s ( 4 . 5 . 1 ) is
r e a s o n a b l e f o r t h e p o p u l a t i o n t o t a l . W e d o n o t d i s a g r e e , b u t j u d g e t h a t
( 4 . 5 . 1 ) i s t h e n r e a s o n a b l e p r e c i s e l y b e c a u s e t h e s a m p l e is t h o u g h t t o b e
r e p r e s e n t a t i v e , in w h i c h c a s e ( 4 . 5 . 1 ) w o u l d b e j u s t i f i a b l e f r o m a v a r i e t y o f
B a y e s i a n a r g u m e n t s . T h e r a n d o m i z a t i o n m a y h e l p t o c o n v i n c e o n e t h a t t h e
s a m p l e is r e p r e s e n t a t i v e , b u t , o n c e c o n v i n c e d o f t h a t f a c t , t h e r e i s n o
f u r t h e r n e e d t o c o n s i d e r t h e s a m p l e s e l e c t i o n p r o b a b i l i t i e s .
M o d e l i n g t h e p o p u l a t i o n is o f t e n c a l l e d t h e s u p e r p o p u l a t i o n
a p p r o a c h t o s u r v e y s a m p l i n g . A l t h o u g h w e h a v e p r e s e n t e d it a s B a y e s i a n in
n a t u r e , t h e m o d e l i n g o f t h e p o p u l a t i o n c a n a l s o b e a r g u e d t o b e a s " o b j e c t i v e "
a s a n y m o d e l i n g u s u a l l y d o n e in s t a t i s t i c s ( c f . t h e d i s c u s s i o n b y R o y a l ! o f
B a s u ( 1 9 7 1 ) ) , in w h i c h c a s e o n e c a n a r g u e t h a t a d i r e c t l y m e a n i n g f u l l i k e l i
h o o d f u n c t i o n f o r t h e s u p e r p o p u l a t i o n p a r a m e t e r s w i l l e x i s t . T o a B a y e s i a n ,
t h e c h o i c e o f a m o d e l is j u s t p a r t o f t h e p r i o r s p e c i f i c a t i o n ( a n d o f t e n t h e
m o s t i m p o r t a n t a n d u n c e r t a i n p a r t ) , s o t h e d i s t i n c t i o n s e e m s u n n e c e s s a r y .
T h i s d i s c u s s i o n h a s a s s u m e d t h a t t h e s e l e c t i o n p r o b a b i l i t i e s ,
P ( s ) , a r e k n o w n . I f t h e y a r e p a r t i a l l y u n k n o w n a n d d e p e n d o n o r o n a n
i n f o r m a t i v e n u i s a n c e p a r a m e t e r ( s e e S e c t i o n 3 . 5 ) t h e y c o u l d b e r e l e v a n t t o
c o n c l u s i o n s a b o u t . R u b i n ( 1 9 8 4 ) a d d r e s s e s t h i s i s s u e , d i s t i n g u i s h i n g
b e t w e e n " i g n o r a b l e " a n d " n o n i g n o r a b l e " s a m p l e s e l e c t i o n m e c h a n i s m s , a n d
r a i s e s t h e r e l a t e d p o i n t t h a t t h e P ( s ) m a y b e u s e f u l a s c r u d e c o v a r i a t e s in
c e r t a i n s i t u a t i o n s o f s t r a t i f i e d s a m p l i n g .
A n o t h e r i s s u e t h a t h a s b e e n r a i s e d i s t h e p o s s i b i l i t y o f i n v o l v i n g
t h e P ( s ) b y p u r p o s e l y i g n o r i n g t h e r a n d o m i z a t i o n o u t c o m e . I n d e e d , R a o ( 1 9 7 1 )
a r g u e s t h a t o n e c a n o b t a i n a n " i n f o r m a t i v e " l i k e l i h o o d f u n c t i o n b y i g n o r i n g
t h e l a b e l s u
in t h e s a m p l e ^ . T h e a v a i l a b l e d a t a is t h e n o n l y v , a n
m v e c t o r o f t h e o b s e r v e d v , w i t h n o r e c o r d o f w h i c h e l e m e n t s o f t h e p o p u l a
t i o n it is a s s o c i a t e d w i t h . It i s e a s y t o c a l c u l a t e , u s i n g ( 4 . 5 . 2 ) a n d
( 4 . 5 . 3 ) , t h a t t h e l i k e l i h o o d f u n c t i o n c o r r e s p o n d i n g t o y i s
( 4 . 5 . 4 ) ( ) = P ( s ) I , v ( ) .
a l l ^
s
o f s i z e m
M
* s '
T h i s l i k e l i h o o d f u n c t i o n m a y s e e m t o c o n t a i n m o r e i n f o r m a t i o n a b o u t . In
E x a m p l e 3 1 , f o r i n s t a n c e , it is e a s y t o s e e t h a t , i f N / m i s a n i n t e g e r , t h e
M . L . E . f o r is a n y v e c t o r c o n t a i n i n g N / m c o p i e s o f v . T h e M . L . E . f o r
w o u l d t h u s b e ( 4 . 5 . 1 ) .
In d i s c u s s i n g t h e r e a s o n a b l e n e s s o f t h e a b o v e p r o p o s a l , it i s
i m p o r t a n t t o f i r s t n o t e t h a t i g n o r i n g d a t a is o f t e n a s e n s i b l e p r a c t i c a l
n e c e s s i t y , a s t h e f o l l o w i n g e x a m p l e i n d i c a t e s .
E X A M P L E 3 2 . S u p p o s e w e o b s e r v e ( X , Y ) h a v i n g a j o i n t d e n s i t y f ( x | ) g ( y | )
( i . e . , X a n d Y a r e i n d e p e n d e n t ) , b u t t h a t f is k n o w n w h i l e g is c o m p l e t e l y
u n k n o w n . I f w e h a v e v e r y l i t t l e p r i o r i n f o r m a t i o n a b o u t g , s o l i t t l e t h a t
y c o n v e y s n o c l e a r k n o w l e d g e a b o u t , t h e n b a s i n g t h e a n a l y s i s on x a l o n e
s e e m s r e a s o n a b l e . O f c o u r s e , i g n o r i n g y c a n b e v i e w e d a s a f o r m a l v i o l a t i o n
o f t h e L P , s i n c e it e s s e n t i a l l y i n v o l v e s i n t e g r a t i n g y o u t o f t h e j o i n t
d e n s i t y o f X a n d Y . It is n o t a v i o l a t i o n o f t h e s p i r i t o f t h e L P , h o w e v e r ,
p r o v i d i n g & ( ) = f ( x l ) is f e l t t o b e r e a s o n a b l y c l o s e t o w h a t w o u l d h a v e
X
b e e n o b t a i n e d w e r e y i n c l u d e d ( s a y , b y p u t t i n g a p r i o r d i s t r i b u t i o n o n g a n d
i n t e g r a t i n g o u t o v e r t h i s p r i o r ) . F u r t h e r d i s c u s s i o n a n d r e f e r e n c e s on t h i s
i s s u e c a n b e f o u n d in P r a t t ( 1 9 6 5 ) , w h o c a l l s X an " i n s u f f i c i e n t s t a t i s t i c , "
a n d in B e r g e r ( 1 9 8 3 ) .
W h i l e i g n o r i n g d a t a m a y o f t e n b e a p r a c t i c a l n e c e s s i t y , t h e r e i s
a c r u c i a l d i f f e r e n c e b e t w e e n d o i n g s o in E x a m p l e 3 2 a n d d o i n g s o in t h e
s a m p l e s u r v e y p r o b l e m . In E x a m p l e 3 2 a n u n k n o w n e l e m e n t g w a s e l i m i n a t e d b y
i g n o r i n g d a t a , w h i l e R a o ( 1 9 7 1 ) s u g g e s t s r e p l a c i n g t h e k n o w n l i k e l i h o o d
f u n c t i o n in ( 4 . 5 . 3 ) b y t h e v e r s i o n in ( 4 . 5 . 4 ) t h a t w o u l d r e s u l t i f t h e l a b e l s
in s w e r e i g n o r e d . N o real s i m p l i f i c a t i o n is i n v o l v e d in t h e l a t t e r
s i t u a t i o n ; i n d e e d ( 4 . 5 . 4 ) s e e m s m o r e c o m p l i c a t e d t h a n ( 4 . 5 . 3 ) . In s o m e
s i t u a t i o n s a n o n B a y e s i a n l i k e l i h o o d a n a l y s i s o f ( 4 . 5 . 4 ) m a y s e e m e a s i e r
t h a n a s i m i l a r a n a l y s i s o f ( 4 . 5 . 3 ) , b u t s u c h is p r o b a b l y o n l y t h e c a s e in
s i m p l e s i t u a t i o n s l i k e t h a t o f E x a m p l e 31 w h e r e P ( s ) is c o n s t a n t , ( a n d t h e n
d i r e c t r e a s o n i n g o f a m o d e l c o n s t r u c t i o n o r B a y e s i a n n a t u r e w i t h ( 4 . 5 . 3 ) is
a l s o e a s y ) . A n d it is e a s y t o c o n s t r u c t e x a m p l e s w h e r e t h e u s e o f ( 4 . 5 . 4 )
w i t h h i g h l y v a r i a b l e P ( s ) c a n g i v e c o m p l e t e l y u n r e a s o n a b l e a n s w e r s f o r
p a r t i c u l a r o b s e r v e d y .
W e h a v e b a r e l y t o u c h e d t h e s u r f a c e o f s u r v e y s a m p l i n g . D e e p e r
d i s c u s s i o n s o f t h e s e i s s u e s a n d o t h e r r e f e r e n c e s c a n b e f o u n d in G o d a m b e
( 1 9 6 6 , 1 9 8 2 a , 1 9 8 2 b ) , C o r n f i e l d ( 1 9 6 9 ) , B a s u ( 1 9 6 9 , 1 9 7 1 , 1 9 7 8 ) , E r i c s o n
( 1 9 6 9 ) , K a l b f l e i s c h a n d S p r o t t ( 1 9 6 9 ) , R a o ( 1 9 7 1 ) , R o y a l l ( 1 9 7 1 , 1 9 7 6 ) ,
G o d a m b e a n d T h o m p s o n ( 1 9 7 6 ) , S m i t h ( 1 9 7 6 ) , C a s s e l e t . a l . ( 1 9 7 7 ) , a n d
T h o m p s o n ( 1 9 8 0 ) . A p a r t i c u l a r l y c o n v i n c i n g c a s e f o r t h e B a y e s i a n v i e w c a n
be f o u n d in B a s u ( 1 9 7 8 ) .
4 . 5 . 3 R a n d o m i z a t i o n T e s t i n g
R a n d o m i z a t i o n t e s t i n g w a s i n t r o d u c e d b y F i s h e r ( c f . F i s h e r ( I 9 6 0 ) )
a n d w a s f u r t h e r d e v e l o p e d b y K e m p t h o r n e a n d o t h e r s . ( S e e K e m p t h o r n e a n d
F o l k e s ( 1 9 7 1 ) a n d B a s u ( 1 9 8 0 ) f o r s o m e o f t h e s e d e v e l o p m e n t s a n d o t h e r
r e f e r e n c e s ) . T h e b a s i s o f r a n d o m i z a t i o n t e s t i n g is u s i n g t h e r a n d o m i z a t i o n
m e c h a n i s m i n v o l v e d in t r e a t m e n t a l l o c a t i o n t o e x p e r i m e n t a l u n i t s t o f o r m
p r o b a b i l i t y a s s e s s m e n t s o f e v i d e n c e . T h e f o l l o w i n g s i m p l e e x a m p l e e x h i b i t s
t h e k e y f e a t u r e s o f t h e a p p r o a c h . S e e B a s u ( 1 9 8 0 ) , a n d t h e d i s c u s s a n t s
t h e r e o f , f o r a m o r e g e n e r a l d i s c u s s i o n .
E X A M P L E 3 3 . In an e x p e r i m e n t , n i n d e p e n d e n t p a i r s o f m a t c h e d s u b j e c t s
( S ? , s ] ) , . . . , (S^ Sj j)} a r e t o b e u t i l i z e d t o c o m p a r e t w o t r e a t m e n t s , T
Q
( t h e
" s t a n d a r d " ) a n d T j ( t h e " n e w t r e a t m e n t " ) . W i t h i n e a c h p a i r , t h e t w o t r e a t
m e n t s a r e r a n d o m l y a s s i g n e d : l e t r
e q u a l 0 o r 1 a s t r e a t m e n t T
Q
o r T ^ ,
r e s p e c t i v e l y , i s a s s i g n e d t o sV ( s o t h a t t r e a t m e n t T ^
r
\ is a s s i g n e d t o S ^ ) ,
a n d d e f i n e r = ( r
] f
. . . , r
n
) . N o t e t h a t P ( r = 0 ) = P ( r
= 1 ) = \ . T h e r e s u l t
o f t h e e x p e r i m e n t w i l l b e a v e c t o r X = (Xi,. . , X
n
) , w h e r e , f o r t h e i t h p a i r ,
0 i f T Q is j u d g e d t o h a v e w o r k e d b e t t e r
1 i f Tj is j u d g e d t o h a v e w o r k e d b e t t e r .
( F o r s i m p l i c i t y o f d i s c u s s i o n , w e a s s u m e t h a t e q u a l i t y o f t r e a t m e n t s is n o t
a p o s s i b l e o b s e r v a t i o n , a n d t h a t o n l y t h e c r u d e m e a s u r e s X
a r e o b s e r v a b l e . )
R a n d o m i z a t i o n t e s t i n g , h e r e , w o u l d i n v o l v e c o n s i d e r a t i o n o f t h e
h y p o t h e s i s ( H
Q
) t h a t t h e t r e a t m e n t s h a v e an i d e n t i c a l e f f e c t , in t h e s e n s e
.
t h a t a g i v e n s u b j e c t in e a c h p a i r , s a y s u b j e c t S ( = 0 o r 1 ) , w o u l d d o
b e s t n o m a t t e r w h i c h t r e a t m e n t it r e c e i v e d . It is e a s y t o c h e c k t h a t H
Q
c a n
b e w r i t t e n m a t h e m a t i c a l l y a s
( 4 . 5 . 5 ) H
Q
: X . = ( + ) ^
2
f o r i = l , . . . , n .
Also, l et t i ng = ( p . . . ,
n
) , i t i s clear t hat , ph.Q. zxp<LnJjn<Lvvtallij
9
X has
density (under HQ) f
f i
( x) = 2 "
n
(since there i s only one assignment r which
wi l l match x t o , and each r has pr obabi l i t y 2 "
n
of occur r i ng) .
S u p p o s e t h a t it i s d e s i r e d t o p e r f o r m a s i g n i f i c a n c e t e s t o f H Q
against the one sided al t er nat i ve t hat T . i s a bet t er treatment than T
Q
. The
n
nat ural t est st at i st i c would be X = X , wit h large values of X providing
evidence against H
Q
. The signif icance level (or P value) of an observat ion,
x, would then be
= Pu

(X > X = X,) = ( " ) 2 "
.
0 ' 2 i =l j=
J
I f , f o r e x a m p l e , a l l x = 1 , t h e n = 2
n
w h i c h , f o r l a r g e n , w o u l d s e e m t o
c a s t d o u b t o n H
Q
.
T h e p r e e x p e r i m e n t a l m e a s u r e o f e v i d e n c e , , in t h e a b o v e e x a m p l e
is b a s e d o n t h e r a n d o m i z a t i o n p r o b a b i l i t i e s . S i n c e t h e a c t u a l r a n d o m i z a t i o n
o u t c o m e r b e c o m e s k n o w n , h o w e v e r , c o n d i t i o n a l r e a s o n i n g w o u l d a r g u e t h a t s u c h
p r o b a b i l i t i e s a r e i r r e l e v a n t . A c o n d i t i o n a l a n a l y s i s o f t h e p r o b l e m m i g h t
g o a s f o l l o w s .
E X A M P L E 3 3 ( c o n t i n u e d ) . B e c a u s e o f t h e p a i r i n g ( a n d t h e r a n d o m i z a t i o n ) it
m i g h t b e d e e m e d r e a s o n a b l e t o p r e t e n d t h a t t h e s u b j e c t s w i t h i n e a c h p a i r a r e
i d e n t i c a l . I f t h e p a i r s c a n b e c o n s i d e r e d t o b e a r a n d o m s a m p l e f r o m t h e
e n t i r e p o p u l a t i o n o f p a i r s , a n d d e n o t e s t h e ( h y p o t h e t i c a l ) p r o p o r t i o n o f
t h e p o p u l a t i o n f o r w h i c h t r e a t m e n t T , w o u l d b e b e t t e r t h a n T
Q
, t h e n o n e
c o u l d w r i t e t h e j o i n t d e n s i t y o f x a n d r a s
f
( x , r ) =
A l i k e l i h o o d a n a l y s i s c o u l d t h e n b e p e r f o r m e d , b a s e d o n t h i s ( b i n o m i a l )
l i k e l i h o o d f o r . ( O f c o u r s e , a s i g n i f i c a n c e t e s t o f = ^ w o u l d g i v e t h e
s a m e r e s u l t a s t h e r a n d o m i z a t i o n a n a l y s i s , a n d w e w i l l a r g u e t h a t t h i s i s
r e a l l y w h y t h e r a n d o m i z a t i o n a n a l y s i s i s , a t a l l , s e n s i b l e . )
T h e r a n d o m i z a t i o n m e c h a n i s m p l a y s n o d i r e c t r o l e in t h e a b o v e
l i k e l i h o o d a r g u m e n t . I n d e e d , t h e u s e o f r a n d o m i z a t i o n is l i m i t e d t o m a k i n g
m o r e b e l i e v a b l e t h e a s s u m p t i o n t h a t t h e p a i r e d s u b j e c t s a r e e q u i v a l e n t : t h e
r a n d o m i z a t i o n h o p e f u l l y e l i m i n a t e s t h e p o s s i b i l i t y o f e x p e r i m e n t e r i n d u c e d
b i a s t h a t m i g h t b e i n t r o d u c e d b y , s a y , g i v i n g t r e a t m e n t T
Q
t o t h e s u b j e c t s
( p e r h a p s s u b c o n s c i o u s l y ) t h o u g h t t o b e h e a l t h i e s t . It m i g h t b e a r g u e d , b y
s o m e , t h a t t h e c l a s s i c a l r a n d o m i z a t i o n a n a l y s i s s e e m s i n t u i t i v e l y m o r e
s e n s i b l e t h a n t h e m o d e l e d l i k e l i h o o d a n a l y s i s . T h e f o l l o w i n g i l l u s t r a t i o n
o f b i a s e d r a n d o m i z a t i o n ( a s d i s c u s s e d in B a s u ( 1 9 8 0 ) ) c a s t s d o u b t o n t h e
v a l i d i t y o f s u c h a n a r g u m e n t .
E X A M P L E 3 3 ( c o n t i n u e d ) . S u p p o s e t h e t r e a t m e n t s a r e a s s i g n e d b y a r a n d o m i z a
t i o n m e c h a n i s m h a v i n g t h e p r o p e r t y t h a t t h e s u b j e c t s S^ ( i n d e p e n d e n t l y )
r e c e i v e t r e a t m e n t T
Q
w i t h p r o b a b i l i t y ^ a n d t r e a t m e n t Tj w i t h p r o b a b i l i t y ^.
S u p p o s e , f u r t h e r , t h a t t h e r a n d o m i z a t i o n o u t c o m e h a p p e n s t o b e t h a t e a c h S
r e c e i v e s t r e a t m e n t T
, a n d t h e e x p e r i m e n t a l o u t c o m e h a p p e n s t o b e t h a t e a c h
x = 1 . I f t h e n u l l h y p o t h e s i s is t r u e , t h e n it m u s t b e t h e c a s e t h a t
i
= 1
f o r a l l i ( s e e ( 4 . 5 . 5 ) ) . B u t it f o l l o w s t h a t t h e s i g n i f i c a n c e l e v e l a g a i n s t
H
Q
i s
= V = (1
1 )
( X l x = n) = P(all r , - 0) = 4"".
T h i s s i g n i f i c a n c e l e v e l s e e m s m i s l e a d i n g l y l o w , d u e t o t h e " u n l i k e l y
1 1
r a n d o m i z a t i o n o u t c o m e . T h e e v i d e n c e a g a i n s t H Q c e r t a i n l y s e e m s n o s t r o n g e r
t h a n it w o u l d h a v e b e e n h a d a n u n b i a s e d r a n d o m i z e r b e e n u s e d . T h e m o d e l e d
l i k e l i h o o d a n a l y s i s w o u l d , o f c o u r s e , b e u n a f f e c t e d b y t h e u s e o f t h e b i a s e d
r a n d o m i z e r . T h u s it s e e m s t h a t t h e r a n d o m i z a t i o n a n a l y s i s m a y b e r a t h e r
s u s p e c t , u n l e s s it c o r r e s p o n d s t o a s e n s i b l e m o d e l e d l i k e l i h o o d a n a l y s i s .
A s w i t h f i n i t e p o p u l a t i o n s a m p l i n g , t h e l i k e l i h o o d a p p r o a c h t e n d s
t o i n v o l v e f u r t h e r m o d e l i n g o f t h e s i t u a t i o n u n d e r i n v e s t i g a t i o n . W h i l e t o
s o m e e x t e n t u n a p p e a l i n g ( m o r e a s s u m p t i o n s m u s t b e i n t r o d u c e d ) , t h e r e s e e m s
t o b e l i t t l e c h o i c e . I n E x a m p l e 3 3 , i f o n e w e r e n o t c o m f o r t a b l e in t r e a t i n g
t h e s u b j e c t s w i t h i n a p a i r a s i d e n t i c a l , o r t h e p a i r s a s r e p r e s e n t a t i v e o f
t h e p o p u l a t i o n , t h e n t h e r a n d o m i z a t i o n a n a l y s i s w o u l d a l s o b e y e r y s u s p e c t .
( I f i t s o h a p p e n e d t h a t a c e r t a i n s u b j e c t in e a c h p a i r c o u l d b e i d e n t i f i e d
a s " h e a l t h i e r " , a c a r e f u l i n v e s t i g a t i o n o f t h e m a t c h u p s o f t r e a t m e n t s a n d
s u b j e c t s w o u l d b e i n d i c a t e d . ) E x t e n s i v e d i s c u s s i o n s o f t h e s e i s s u e s c a n b e
f o u n d ( w i t h o t h e r r e f e r e n c e s ) in S a v a g e e t . a l . ( 1 9 6 2 ) , H i l l ( 1 9 7 0 ) ,
G o o d ( 1 9 7 6 ) , R u b i n ( 1 9 7 8 ) , L i n d l e y a n d N o v i c k ( 1 9 8 1 ) , a n d e s p e c i a l l y
B a s u ( 1 9 8 0 ) .
C H A P T E R 5 . I M P L E M E N T A T I O N O F T H E L I K E L I H O O D P R I N C I P L E
5.1 I N T R O D U C T I O N
T h e L P s t r i k e s us as c o r r e c t , a n d b e h a v i n g in v i o l a t i o n o f it
w o u l d b e a s o u r c e o f c o n s i d e r a b l e d i s c o m f o r t . Y e t t h e L P d o e s n o t t e l l o n e
w h a t to do ( a l t h o u g h i n s i s t i n g o n m e t h o d s b a s e d o n t h e o b s e r v e d l i k e l i h o o d
f u n c t i o n c e r t a i n l y r e d u c e s t h e p o s s i b i l i t i e s ) . I t c a n i n d e e d b e a r g u e d t h a t
t h e r e is s o m e t i m e s n o s e n s i b l e m e t h o d o f b e h a v i o r w h i c h is c o m p l e t e l y c o n s i s -
t e n t w i t h t h e L P .
T h i s r a i s e s a y e r y i m p o r t a n t d i s t i n c t i o n w h i c h is o f t e n m i s u n d e r -
s t o o d in f o u n d a t i o n a l m a t t e r s . " F o u n d a t i o n s " u s u a l l y p r o c e e d s b y f o r m u l a t i n g
p r o p e r t i e s o f d e s i r a b l e b e h a v i o r , a n d t h e n s e e i n g w h a t c a n b e d e d u c e d f r o m
t h e s e p r o p e r t i e s . T h e q u i n t e s s e n t i a l e x a m p l e is t h a t f r o m ( v e r y r e a s o n a b l e )
a x i o m s o f " c o n s i s t e n t " o r " r a t i o n a l " b e h a v i o r , it c a n b e d e d u c e d t h a t a n y
" c o n s i s t e n t " a n a l y s i s c o r r e s p o n d s t o s o m e B a y e s i a n a n a l y s i s . T h i s d o e s n o t
i m p l y , h o w e v e r , t h a n a n y p a r t i c u l a r f o r m o f c o n s i s t e n t ( B a y e s i a n ) a n a l y s i s is
n e c e s s a r i l y s a t i s f a c t o r y , s i n c e , a s C . A . B . S m i t h s a i d in S a v a g e , e t . a l . ( 1 9 6 2 ) ,
" C o n s i s t e n c y is n o t n e c e s s a r i l y a v i r t u e :
o n e c a n b e c o n s i s t e n t l y o b n o x i o u s . "
A n d t h e r e is n o g u a r a n t e e t h a t a n o n o b n o x i o u s c o n s i s t e n t w a y o f b e h a v i n g e x i s t s .
( S e e B e r g e r ( 1 9 8 4 e ) f o r f u r t h e r d i s c u s s i o n . ) T h u s f o u n d a t i o n a l a r g u m e n t s
( i n c l u d i n g t h e L P ) c a n l o g i c a l l y b e c o n s i d e r e d i r r e l e v a n t f r o m a n o p e r a t i o n a l
p e r s p e c t i v e .
T h i s is c e r t a i n l y o v e r s t a t i n g t h e c a s e , s o m e w h a t , in t h a t , a t t h e
v e r y l e a s t , f o u n d a t i o n a l a r g u m e n t s c a n b e i n v a l u a b l e in g i v i n g d i r e c t i o n t o
121
o u r e f f o r t s . T h u s t h e " c o n s i s t e n c y " t h e o r y s t r o n g l y s u g g e s t s t h a t t r u t h l i e s
in a B a y e s i a n d i r e c t i o n , a n d t h e L P s t r o n g l y s u g g e s t s t h a t t r u t h l i e s in t h e
d i r e c t i o n o f m e t h o d s b a s e d o n d e t e r m i n a t i o n a n d u t i l i z a t i o n o f t h e l i k e l i h o o d
f u n c t i o n ( f o r t h e o b s e r v e d x ) . L u c k i l y ( o r i n e v i t a b l y ) t h e s e t w o d i r e c t i o n s
a r e c o m p a t i b l e .
T o s h o w t h a t t h e L P i s n o t i r r e l e v a n t , w e m u s t a r g u e t h a t a
s e n s i b l e m e t h o d o f a n a l y s i s e x i s t s w h i c h i s c o m p a t i b l e w i t h i t . T h i s is s i m p l y
t o o m u c h t o a s k ; i t w o u l d i n v o l v e d e m o n s t r a t i n g t h a t s u c h a m e t h o d o l o g y w o r k s
w e l l " a c r o s s t h e b o a r d " i n s t a t i s t i c s . I n s t e a d , w e w i l l c o n t e n t o u r s e l v e s t o
a r g u i n g f o r w h a t , w e f e e l , t h i s m e t h o d o l o g y m u s t b e , n a m e l y r o b u s t B a y e s i a n
a n a l y s i s . W e s t a r t o u t , h o w e v e r , w i t h a v e r y b r i e f d e s c r i p t i o n o f n o n B a y e s i a n
l i k e l i h o o d m e t h o d s . U n t i l S e c t i o n 5 . 4 , w e w i l l a s s u m e t h a t t h e l i k e l i h o o d
f u n c t i o n i e ) ( f o r t h e o b s e r v e d x ) e x i s t s .
5 . 2 N O N B A Y E S I A N L I K E L I H O O D M E T H O D S
I t s h o u l d f i r s t b e m e n t i o n e d t h a t t h e r e a r e c l a s s i c a l l y b a s e d
l i k e l i h o o d m e t h o d s s u c h a s m a x i m u m l i k e l i h o o d e s t i m a t i o n a n d l i k e l i h o o d r a t i o
t e s t i n g . A l t h o u g h t h e s e a r e u s u a l l y g i v e n e v i d e n t i a l i n t e r p r e t a t i o n s i n
f r e q u e n t i s t t e r m s , t h e c o n c e p t s t h e m s e l v e s a r e c l e a r l y o f g r e a t i m p o r t a n c e i n
l i k e l i h o o d m e t h o d s . T h e l i t e r a t u r e o n t h e s e s u b j e c t s i s t o o v a s t t o e v e n
a t t e m p t m e n t i o n i n g .
S i n c e t h e L P s t a t e s t h a t a l l e v i d e n c e a b o u t is c o n t a i n e d i n
&
v
( ) > o n e c o n c e i v a b l e s o l u t i o n t o t h e p r o b l e m o f w h a t t o d o i s s i m p l y t o
x
r e p o r t %
( h l e a v i n g i t s u s e a n d i n t e r p r e t a t i o n " t o t h e u s e r " ( c . f . , F i s h e r
( 1 9 5 6 a ) a n d B o x a n d T i a o ( 1 9 7 3 ) ) . T h i s i s n o t n e c e s s a r i l y u n r e a s o n a b l e , a s
" e y e b a l l i n g " a l i k e l i h o o d f u n c t i o n o f t e n r e v e a l s m o s t t h i n g s o f i n t e r e s t , a t
l e a s t w h e n is l o w d i m e n s i o n a l . M a n y p e o p l e p r o b a b l y c o u l d l e a r n t o u s e f u l l y
d e a l w i t h l i k e l i h o o d f u n c t i o n s a s t h e b a s i c e l e m e n t s o f s t a t i s t i c s ( a n d i n d e e d
m a n y n o w d o ) . E v e n a B a y e s i a n s h o u l d e n c o u r a g e r e p o r t i n g o f l i k e l i h o o d f u n c
t i o n s . T h u s G o o d ( 1 9 7 6 ) s a y s
I M P L E M E N T A T I O N O F T H E L I K E L I H O O D P R I N C I P L E 1 2 3
" I f a B a y e s i a n i s a s u b j e c t i v i s t h e w i l l
k n o w t h a t t h e i n i t i a l p r o b a b i l i t y d e n s i t y
v a r i e s f r o m p e r s o n t o p e r s o n a n d s o h e w i l l
s e e t h e v a l u e o f g r a p h i n g t h e l i k e l i h o o d
f u n c t i o n f o r c o m m u n i c a t i o n . A D o o g i a n w i l l
c o n s i d e r t h a t e v e n h i s o w n i n i t i a l p r o b a b i l i t y
d e n s i t y i s n o t u n i q u e s o h e s h o u l d a p p r o v e
e v e n m o r e " .
N e v e r t h e l e s s , r e p o r t i n g o f I ( ) c a n n o t b e c o n s i d e r e d t o b e t h e e n d o f t h e
x
s t a t i s t i c i a n ' s j o b ; pr ope r l y using a ( ) can be d i f f i c u l t and c r u c i a l . Al so,
t h e na t ur a l vi sua l i n t e r p r e t a t i o n t h a t w i l l be a sc r i be d t o i ( ) by most users
x
is t h a t o f a p r o b a b i l i t y d i s t r i b u t i o n f o r , a n i n t e r p r e t a t i o n n e e d i n g c a r e f u l
h a n d l i n g .
M o s t o f t h e l i k e l i h o o d m e t h o d s t h a t h a v e b e e n p r o p o s e d a r e
d e p e n d e n t o n t h e i n t e r p r e t a t i o n t h a t
v
(,)/fi, (
9
) m e a s u r e s t h e r e l a t i v e
X I X C.
s u p p o r t o f t h e d a t a f o r i a n d p E x t e n s i v e d e v e l o p m e n t o f t h i s i d e a c a n b e
f o u n d i n H a c k i n g ( 1 9 6 5 ) a n d E d w a r d s ( 1 9 7 2 ) . O t h e r l i k e l i h o o d d e v e l o p m e n t s c a n
b e f o u n d i n F i s h e r ( 1 9 5 6 a ) , B a r n a r d , J e n k i n s a n d W i n s t e n ( 1 9 6 2 ) , B i r n b a u m
( 1 9 6 2 a ) , B a r n a r d ( 1 9 6 7 a ) , S p r o t t a n d K a l b f l e i s c h ( 1 9 6 9 ) , K a l b f l e i s c h a n d S p r o t t
( 1 9 7 0 ) , A n d e r s e n ( 1 9 7 0 , 1 9 7 1 , 1 9 7 3 ) , K a l b f l e i s c h ( 1 9 7 1 , 1 9 7 8 ) , B a r n d o r f f N i e l s e n
( 1 9 7 1 ) , S p r o t t ( 1 9 7 3 a , 1 9 7 3 b ) , C o x a n d H i n k l e y ( 1 9 7 4 ) , C o x ( 1 9 7 5 ) , T j u r ( 1 9 7 8 ) ,
H i n k l e y ( 1 9 7 8 , 1 9 7 9 , 1 9 8 0 , 1 9 8 2 ) , G r a m b s c h ( 1 9 8 0 ) , B a r n e t t ( 1 9 8 2 ) , a n d m a n y o f
t h e r e f e r e n c e s g i v e n i n C h a p t e r 2 . " P l a u s i b i l i t y I n f e r e n c e " ( c . f . B a r n d o r f f
N i e l s e n ( 1 9 7 6 ) ) i s a l s o r e l a t e d . ( N o t a l l o f t h e s e a u t h o r s n e c e s s a r i l y
s u b s c r i b e t o t h e L P , o f c o u r s e . )
W e d o n o t d e t a i l t h e s e d e v e l o p m e n t s f o r s e v e r a l r e a s o n s . F i r s t ,
t h e s p a c e r e q u i r e m e n t w o u l d s i m p l y b e p r o h i b i t i v e . S e c o n d , m a n y o f t h e
t e c h n i q u e s p r o p o s e d , w h i l e v a l u a b l e , a r e e i t h e r d e s i g n e d o n l y f o r a n a r r o w c l a s s
o f p r o b l e m s , a n d h e n c e d o n o t p r o v i d e a b a s i s f o r a g e n e r a l l i k e l i h o o d b a s e d
t h e o r y , o r a t t e m p t g e n e r a l i t y b u t f a l l p r e y t o c o u n t e r e x a m p l e s . ( S e e B i r n b a u m
( 1 9 6 2 a ) , t h e d i s c u s s i o n in K a l b f l e i s c h a n d S p r o t t ( 1 9 7 0 ) , P l a n t e ( 1 9 7 1 ) , B a s u
( 1 9 7 5 ) , H i l l ( 1 9 7 3 , 1 9 7 5 ) , a n d L e v i ( 1 9 8 0 ) f o r s o m e s u c h c o u n t e r e x a m p l e s . )
F i n a l l y , a n d m o s t i m p o r t a n t l y , w e w i l l a r g u e i n t h e n e x t s e c t i o n t h a t t h e r e
a r e c o m p e l l i n g r e a s o n s f o r u t i l i z i n g ( ) t h r o u g h B a y e s i a n a n a l y s i s , a n d h e n c e
t h a t n o n B a y e s i a n l i k e l i h o o d t e c h n i q u e s a r e i n h e r e n t l y l i m i t e d . S u c h t e c h n i q u e s
c a n o f f e r s u b s t a n t i a l i m p r o v e m e n t s o v e r c l a s s i c a l m e t h o d s , h o w e v e r , a n d s h o u l d
b e u s e f u l f o r t h o s e u n w i l l i n g t o a c c e p t a B a y e s i a n a p p r o a c h . A l s o , m a n y o f
t h e t e c h n i c a l d e v e l o p m e n t s i n t h e s e a r t i c l e s c a n b e u s e f u l e v e n t o a B a y e s i a n .
5 . 3 A R G U M E N T S F O R B A Y E S I A N I M P L E M E N T A T I O N
S a v a g e , i n t h e d i s c u s s i o n o f B i r n b a u m ( 1 9 6 2 a ) , s a i d
" . . . I s u s p e c t t h a t o n c e t h e l i k e l i h o o d
p r i n c i p l e i s w i d e l y r e c o g n i z e d , p e o p l e w i l l
n o t l o n g s t o p a t t h a t h a l f w a y h o u s e b u t w i l l
g o f o r w a r d a n d a c c e p t t h e i m p l i c a t i o n s o f
p e r s o n a l i s t i c p r o b a b i l i t y f o r s t a t i s t i c s . "
I t w o u l d b e i n a p p r o p r i a t e h e r e t o p r e s e n t t h e f u l l r a n g e o f a r g u m e n t s f o r
B a y e s i a n a n a l y s i s . I n s t e a d , w e w i l l c o n c e n t r a t e o n i n d i c a t i n g h o w s e n s i b l e
u s e o f t h e l i k e l i h o o d f u n c t i o n s e e m s p o s s i b l e o n l y t h r o u g h B a y e s i a n a n a l y s i s .
5 . 3 . 1 G e n e r a l C o n s i d e r a t i o n s
F i r s t , b e l i e v e r s i n t h e L P s h o u l d , i t s e e m s , b e e s p e c i a l l y w a r y o f
w h a t G o o d ( 1 9 7 6 ) c a l l e d ' a d h o c k e r i e s
1
. T h e s e a r e s u p e r f i c i a l l y r e a s o n a b l e
m e t h o d s o f a n a l y s i s w h i c h , h o w e v e r , h a v e n o f i r m f o u n d a t i o n a l b a s i s . C a r e f u l
i n v e s t i g a t i o n o f a d h o c k e r i e s a l w a y s s e e m s t o r e v e a l a f l a w . N o n B a y e s i a n u s e
o f l i k e l i h o o d f u n c t i o n s v i r t u a l l y a l w a y s p r o c e e d s b y d e v e l o p i n g a n a d h o c
m e t h o d o f d e a l i n g w i t h i n v o l v e d s i t u a t i o n s . N o a d h o c m e t h o d e v e r s e e m s t o b e
s u f f i c i e n t . I n d e e d , t h e r a t i o n a l i t y o r c o n s i s t e n c y j u s t i f i c a t i o n f o r B a y e s i a n
a n a l y s i s g i v e s a s t r o n g i n d i c a t i o n t h a t n o a d h o c m e t h o d w i l l e v e r p r o v e
f o o l p r o o f .
A n e x a m p l e o f t h e p r o b l e m s f a c e d b y n o n B a y e s i a n s i s t h a t ,
d i s c u s s e d in S e c t i o n 3 . 5 , o f d e a l i n g w i t h i n f o r m a t i v e n u i s a n c e p a r a m e t e r s .
S u c h n u i s a n c e p a r a m e t e r s a r e p a r t o f t h e l i k e l i h o o d f u n c t i o n , y e t n e e d t o
e s s e n t i a l l y b e e l i m i n a t e d b e f o r e p r o g r e s s c a n b e m a d e . T h e B a y e s i a n a p p r o a c h
p r o v i d e s a n a t u r a l ( t h o u g h m a y b e d i f f i c u l t ) w a y o f d o i n g t h i s ; d e t e r m i n e a
p r i o r d i s t r i b u t i o n a n d i n t e g r a t e o u t t h e n u i s a n c e p a r a m e t e r ( a f t e r m u l t i p l y i n g
t h e l i k e l i h o o d f u n c t i o n a n d t h e p r i o r ) . S i m p l e a l t e r n a t i v e s , s u c h a s m a x i m i z i n g
o v e r t h e n u i s a n c e p a r a m e t e r , a r e s i m p l y t o o c r u d e t o g i v e g e n e r a l h o p e o f
s u c c e s s ( s e e L i n d l e y i n t h e d i s c u s s i o n o f B i r n b a u m ( 1 9 6 2 a ) ) , a l t h o u g h f a i r l y
s o p h i s t i c a t e d m e t h o d s ( s u c h a s t h o s e in H i n d e a n d A i t k e n ( 1 9 8 4 ) ) m a y o f t e n
w o r k r e a s o n a b l y w e l l .
T h e o n l y s i t u a t i o n s in w h i c h p u r e l i k e l i h o o d m e t h o d s a r e c o m p l e t e l y
c o n v i n c i n g a r e s i m p l e o n e s ( s u c h a s t e s t i n g t w o s i m p l e h y p o t h e s e s ) , w h e r e t h e y
i n f a c t c o r r e s p o n d t o B a y e s p r o c e d u r e s . T h u s B i r n b a u m ( 1 9 6 2 a ) s a y s ( a n d
s u p p o r t s w i t h e x a m p l e s )
" A n d , a t l e a s t f o r s u c h s i m p l e p r o b l e m s ,
o n e m i g h t s a y t h a t ( L P ) i m p l i e s ( B a y e s )
i n t h e \ / e r y b r o a d a n d q u a l i t a t i v e s e n s e
t h a t u s e o f s t a t i s t i c a l e v i d e n c e a s
c h a r a c t e r i z e d b y t h e l i k e l i h o o d f u n c t i o n
a l o n e e n t a i l s t h a t i n f e r e n c e o r d e c i s i o n
m a k i n g b e h a v i o r w i l l b e e x t e r n a l l y i n d i s
t i n g u i s h a b l e f r o m ( s o m e c a s e o f ) a B a y e s i a n
m o d e o f i n f e r e n c e . "
T h e a b o v e a r g u m e n t s w i l l n o t b e \ / e r y c o m p e l l i n g t o m o s t n o n
B a y e s i a n s , s o l e t u s t u r n t o t h e k e y i s s u e t h a t % ( ) n e e d m a k e l i t t l e s e n s e
u n l e s s i n t e r p r e t e d t h r o u g h a B a y e s i a n f i l t e r . I f is a p r i o r ( d e n s i t y f o r
c o n v e n i e n c e ) o n , t h e n a B a y e s i a n b e l i e v i n g i s r e a s o n a b l e o r p l a u s i b l e w o u l d
v i e w i ( ) t h r o u g h t h e p o s t e r i o r d i s t r i b u t i o n
( | x ) = A
y
( ) ( ) / / ( H ( ) d ,
w h i c h e s s e n t i a l l y c o r r e s p o n d s t o v i e w i n g A ( ) a s a p r o b a b i l i t y d e n s i t y w . r . t .
x
t h e ( p r o p e r l y n o r m a l i z e d v e r s i o n o f ) . T h e p r i o r n e e d n o t b e p r o p e r , a n d
i n d e e d t h o s e w a n t i n g " o b j e c t i v i t y " m i g h t d e s i r e t o u s e a " n o n i n f o r m a t i v e " p r i o r
a s t h e b a s i s o f t h e n o r m a l i z i n g m e a s u r e . In a n y c a s e , t h e k e y t o t h e
B a y e s i a n a p p r o a c h i s t o t r e a t * ( ) a s a n a c t u a l p r o b a b i l i t y d e n s i t y a n d it i s
x
r e a s o n a b l e t o d o s o o n l y w h e n i t is c o n s i d e r e d a d e n s i t y w . r . t . t h e p r e s u m e d
p r i o r m e a s u r e f o r .
A n u m b e r o f j u s t i f i c a t i o n s f o r t h i s v i e w h a v e b e e n a d v a n c e d . F i r s t
i s t h e q u i t e p e r s u a s i v e a r g u m e n t t h a t p r o b a b i l i t y is t h e l a n g u a g e o f
u n c e r t a i n t y , s o t h e u n c e r t a i n t y a b o u t , r e f l e c t e d in ( ) , s h o u l d b e
x
e x p r e s s e d p r o b a b i l i s t i c a l l y . S e c o n d , i t u s u a l l y is n e c e s s a r y t o c o m p a r e o r
r e l a t e o n e s u b s e t o f % t o a n o t h e r , a n d s o m e m e t h o d o f a v e r a g i n g o v e r ,
() i s
t h e n n e e d e d . I n d e e d , B a s u ( 1 9 7 5 ) p r e s e n t s r e a s o n a b l e a r g u m e n t s t h a t &
( )
s h o u l d b e " a d d i t i v e " w h e n is d i s c r e t e . ( H i s a r g u m e n t , h o w e v e r , t h a t in
r e a l i t y i s a l w a y s d i s c r e t e , is m u c h l e s s c o n v i n c i n g t h a n t h e c o r r e s p o n d i n g
a r g u m e n t t h a t % i s d i s c r e t e ; w e m e a s u r e X t o o n l y a c e r t a i n a c c u r a c y , b u t
c o u l d s t i l l b e a n y t h i n g . )
N o n B a y e s i a n a v e r a g i n g o f ( ) h a s t h e s e v e r e p r o b l e m t h a t
r e p a r a m e t e r i z a t i o n c a n c h a n g e t h e a n s w e r . O n e c a n m a k e a c h a n g e o f v a r i a b l e s
= ( ) , w h e r e i s a 11 f u n c t i o n , a n d t h e r e s u l t i n g l i k e l i h o o d f u n c t i o n f o r
, n a m e l y
( ( ) ) , c o u l d l o o k c o m p l e t e l y d i f f e r e n t . A d h o c a v e r a g i n g m e t h o d s
w i l l v i r t u a l l y a l w a y s g i v e d i f f e r e n t c o n c l u s i o n s f o r t h e r e p a r a m e t e r i z e d
l i k e l i h o o d f u n c t i o n ( a s w i l l m a n y o t h e r i n t u i t i v e l i k e l i h o o d t e c h n i q u e s ) , a
v e r y d i s t u r b i n g p r o s p e c t . O f c o u r s e , t h e i n t e r p r e t a t i o n o f a ( ) a s a
p r o b a b i l i t y d e n s i t y w . r . t . t h e p r i o r m e a s u r e is i m m u n e t o t h i s p r o b l e m , s i n c e
a r e p a r a m e t e r i z a t i o n s i m p l y i n t r o d u c e s a J a c o b i a n i n t h e t r a n s f o r m e d p r i o r .
In s o m e s i t u a t i o n s , i t is c l e a r l y i m p e r a t i v e t o d e t e r m i n e a n d i n
t r o d u c e . O n e s u c h s i t u a t i o n i s t h a t o f S e c t i o n 4 . 5 . 2 , i n w h i c h t h e l i k e l i
h o o d f u n c t i o n i t s e l f c o n v e y s a l m o s t n o i n f o r m a t i o n u n l e s s i s s e v e r e l y
r e s t r i c t e d t h r o u g h ( i . e . , a s u i t a b l e m o d e l f o r t h e p o p u l a t i o n i s i n t r o d u c e d ) .
I n d e e d t h e n o n p a r a m e t r i c s i t u a t i o n d i s c u s s e d i n S e c t i o n 3 . 6 . 1 i s t h e g e n e r a l
p r o t o t y p e f o r t h i s s i t u a t i o n , in t h a t t h e l i k e l i h o o d f u n c t i o n i s v e r y d i f f i c u l t
t o u s e u n l e s s i s s u b s t a n t i a l l y r e s t r i c t e d a p r i o r i , c o r r e s p o n d i n g t o p r o p o s i n g
a m o d e l ( o r c l a s s o f m o d e l s ) f o r t h e d i s t r i b u t i o n o f X. T h e " g e n e r a l i z e d
i n v e r s e " p r o b l e m s d i s c u s s e d in J a y n e s ( 1 9 8 1 ) a l s o h a v e t h i s s a m e f l a v o r .
T h e f a i l u r e o f t h e l i k e l i h o o d f u n c t i o n t o p r o v i d e c l e a r l y i n t e r
p r e t a b l e i n f o r m a t i o n , w h e n i s h u g e , is s o m e t i m e s d e e m e d a c r i t i c i s m o f t h e
L P . I n s t e a d , w e v i e w it a s a n i n d i c a t i o n t h a t p r i o r i n f o r m a t i o n m u s t b e u s e d
in s u c h s i t u a t i o n s . ( S e e a l s o t h e d i s c u s s i o n in S e c t i o n 4 . 5 . )
5 . 3 . 2 T h e F r a s e r M o n e t t e N g , S t o n e , a n d S t e i n E x a m p l e s
N e x t , w e t u r n t o t h r e e i m p o r t a n t e x a m p l e s w h i c h h a v e b e e n v i e w e d
a s c o u n t e r e x a m p l e s t o t h e L P , b u t i n s t e a d a r e v i e w e d b y u s a s i n d i c a t i o n s t h a t
a B a y e s i a n ( r a t h e r t h a n i n t u i t i v e ) i n t e r p r e t a t i o n o f t h e l i k e l i h o o d f u n c t i o n is
n e e d e d . T h e f i r s t i s a n e x a m p l e f r o m F r a s e r , M o n e t t e , a n d N g ( 1 9 8 4 ) . ( S e e
a l s o E v a n s , F r a s e r , a n d M o n e t t e ( 1 9 8 6 ) a n d t h e d i s c u s s i o n t h e r e i n f o r
a d d i t i o n a l d e v e l o p m e n t . )
E XA M P L E 3 4 . S u p p o s e X= = { 1 , 2 , . . . } , a n d
/ 2 , 2 , 2 + l w h e n i s e v e n
( 5 . 3 . 1 ) f
( x ) = i f o r x = ( l ) / 2 , 2 , 2 + l w h e n f is o d d
1 , 2 , 3 w h e n = 1 .
T h e l i k e l i h o o d f u n c t i o n is e a s i l y s e e n t o b e
x / 2 , 2 x , 2 x + l w h e n x i s e v e n
a
( ) = \ f o r =
( x l ) / 2 , 2 x , 2 x + l w h e n x f 1 is o d d
1 , 2 , 3 w h e n x = 1 .
T h u s , f o r a n y x , t h e d a t a i n t u i t i v e l y g i v e s e q u a l s u p p o r t t o t h e t h r e e
p o s s i b l e c o m p a t i b l e w i t h t h a t o b s e r v a t i o n . O n s o l e l y l i k e l i h o o d b a s e d
g r o u n d s , t h e r e f o r e , a n y o f t h e t h r e e w o u l d b e a s u i t a b l e e s t i m a t e . C o n s i d e r ,
t h e r e f o r e , t h r e e p o s s i b l e e s t i m a t o r s , j,
2
, a n d 6
3
, c o r r e s p o n d i n g t o u s i n g
t h e f i r s t , m i d d l e , a n d l a s t p o s s i b l e , r e s p e c t i v e l y : t h u s
x / 2 w h e n x is e v e n
( x D / 2 w h e n x M i s o d d
1 w h e n x = 1 ,
6
2
( x ) = 2 x , a n d
3
( x ) = 2 x + l .
Now
P
(
2
( X) = ) = P
( X = / 2 )
and
P
( < *
3
( X) = ) = P
Q
( X = ( l ) / 2 ) =
w h i l e , a m a z i n g l y ,
P
( { 1 , 2 , 3 } ) = 1
1/3 w h e n i s e v e n
1/3 w h e n t 1 is o d d
when = 1
( 5 . 3 . 2 ) P
( X) = ) =
P
( { 2 , 2 + l } ) = 2 / 3 ot h erwise.
Even more surprising is t h at th e confidence set C j(x) = { 2 x, 2 x+ l} seems t wice
as good from a "pure l i ke l i h ood" viewpoint as Cp(x) = , ( x ) } , and ye t
0 w h e n = 1
P
( C
( X) c o n t a i n s ) =
1/3 o t h e r w i s e
w h i l e
P
Q
( C
0
( X) c o n t a i n s ) =
1 w h e n = 1
2 / 3 o t h e r w i s e .
O f c o u r s e , t h e m e a s u r e s h e r e a r e f r e q u e n t i s t m e a s u r e s , b u t t h e d e c i s i o n
t h e o r e t i c o r c o h e r e n c y e v a l u a t i o n a r g u m e n t s o f S e c t i o n 3 . 7 c a n b e a p p l i e d t o
i n d i c a t e s u b s t a n t i a l i n f e r i o r i t y o f 6
?
, 6
3
, o r C , i n r e p e a t e d u s e .
L e t us n o w c o n s i d e r w h a t h a p p e n s w h e n &
( ) is p a s s e d t h r o u g h a
B a y e s i a n f i l t e r . A B a y e s i a n h a s a p r i o r d e n s i t y IT f o r , a n d h i s p o s t e r i o r
d e n s i t y w i l l b e
IMPLEMENTATION OF THE LIKELIHOOD PRINCIPLE 129
&
x
( e ) ( e )
( ) I
{
( x ) ,
2
( x ) , 6
3
( x ) }
( )
ft ~
=
( 6
1
( x) ) + (
2
( x) ) + (
3
( x) ) '
T h u s , i n d e e d , t h e d a t a c o n v e y s n o t h i n g t o t h e B a y e s i a n e x c e p t t h a t is
<5 | ( x ) , 6
2
( x ) , o r
3
( x ) . Is t h e B a y e s i a n i n d i f f e r e n t b e t w e e n , ( x ) , p ( x ) ,
a n d J x ) , h o w e v e r ? H e is o n l y i f ( , ( x ) ) = (
2
( x ) ) = (
3
( x ) ) , w h i c h c a n n o t
h o l d f o r a l l x ( w h e n is a p r o p e r d e n s i t y ) . I n d e e d i t w i l l t y p i c a l l y b e
t h e c a s e , a t l e a s t f o r d e n s i t i e s w h i c h a r e m o n o t o n i c a l l y d e c r e a s i n g f o r l a r g e
, t h a t ( , ( x ) ) > (
2
( x ) ) + (
3
( x ) ) . T h u s a B a y e s i a n w o u l d n e v e r a l w a y s u s e
2
>
3
, o r C , , a n d w o u l d , in f a c t , t e n d t o u s e , . T h e B a y e s i a n t h u s a v o i d s
t h e d a n g e r i n h e r e n t in p u r e l i k e l i h o o d r e a s o n i n g .
A s a f i n a l c o m m e n t o n t h i s e x a m p l e , n o t e t h a t a ( s o p h i s t i c a t e d )
n o n i n f o r m a t i v e p r i o r B a y e s i a n o b t a i n s a r e a s o n a b l e ( o b j e c t i v e ) a n s w e r t o t h i s
p r o b l e m . A l t h o u g h o n e m i g h t n a i v e l y g i v e a c o n s t a n t ( i m p r o p e r ) p r i o r d e n s i t y ,
r e s u l t i n g i n t h e i l l a d v i s e d ( | x ) =
( ) , i t i s c l e a r f r o m ( 5 . 3 . 1 ) t h a t
x
is approximately a scale parameter. Th is would lead a noninf ormat ive pr i or
Bayesian t o use t h e Je f f r e y' s ( 1 96 1 ) pr i or densit y f or a scale parameter,
namely ( ) = . With t h i s noninf ormative pr i or , not only is , ( x) again t h e
cl e ar ch oice f or , but t h e post e r ior pr oba bi l i t y t h at = , ( x) is
approximately 2 / 3 f or large x. ( Th i s, i nc i de nt a l l y, provides a condit ional
j us t i f i c a t i o n f or t h e f r e que nt i st report in ( 5 . 3 . 2 ) . )
Th us, e i t h e r a proper pr i or Bayesian or a car e f ul noninf ormat ive
pr i or Bayesian wi l l e asi l y a r r i ve at a sensible likelih ood based conclusion in
t h i s example. We h ave seen no pure l i ke l i h ood methods wh ich can make t h e same
claim.
EXAMPLE 3 5 . Stone ( 1 97 6 ) (see also Hi l l ( 1 98 1 ) f or a discussion si mi l a r t o
t h e f ollowing) considers a very i nt e r e st i ng example in wh ich a drunken sol di e r ,
st a r t i ng at an i nt e r se ct i on 0 in a c i t y (wh ich h as square bl ocks) , staggers
around on a random path t r a i l i ng a t aut st rong. Event ually t h e sol di e r stops
at an i nt e r se ct i on ( a f t e r walking at l e ast one block) and buries a t r e asur e .
Let denote t h e path of t h e st r i ng from 0 t o t h e t r e asur e . Le t t i ng N, S, E,
and W stand f or a path segment one block long in th e indicat ed di r e ct i on, can
be expressed as a sequence of such l e t t e r s , say
= N N E S WS WW.
( N o t e t h a t N S , S N , E W , a n d W E c a n n o t a p p e a r , a s t h e t a u t s t r i n g w o u l d j u s t b e
r e w o u n d . I n e x p r e s s i o n s b e l o w , h o w e v e r , w e a l l o w s u c h c o m b i n a t i o n s t o a p p e a r
f o r n o t a t i o n a l c o n v e n i e n c e , a l t h o u g h t h e y a r e t o b e u n d e r s t o o d t o c a n c e l . )
A f t e r b u r y i n g t h e t r e a s u r e , t h e s o l d i e r r a n d o m l y c h o o s e s o n e o f
t h e f o u r p o s s i b l e d i r e c t i o n s a n d w a l k s o n e b l o c k in t h a t d i r e c t i o n ( s t i l l
k e e p i n g t h e s t r i n g t a u t ) . L e t X d e n o t e t h i s a u g m e n t e d p a t h , s o t h a t X i s o n e
o f t h e p a t h s { N , S , E , W } , w i t h p r o b a b i l i t y ^ e a c h . W e o b s e r v e X, a n d a r e
t o f i n d t h e t r e a s u r e .
N o t e f i r s t t h a t , f o r g i v e n X = x , t h e o n l y p o s s i b l e v a l u e s o f
a r e { x N , x S , x E , x W } , a n d s i n c e t h e p r o b a b i l i t y t h a t X = x w h e n e a c h o f t h e s e
o b t a i n s i s j , w e h a v e t h e l i k e l i h o o d f u n c t i o n
X
( ) = J
f o r e a c h o f t h e f o u r p o s s i b l e .
S t o n e u s e s t h i s e x a m p l e t o i n d i c a t e a p r o b l e m w i t h u s e o f t h e
" n o n i n f o r m a t i v e " p r i o r

( )
s
1 f o r a l l p o s s i b l e p a t h s , s i n c e a n e a s y
c a l c u l a t i o n t h e n s h o w s t h a t t h e p o s t e r i o r p r o b a b i l i t y o f e a c h o f t h e f o u r
p o s s i b l e . g i v e n x , i s j. T h i s s u p p o s e d l y c o n f l i c t s w i t h t h e i n t u i t i o n t h a t ,
g i v e n , X i s t h r e e t i m e s a s l i k e l y t o e x t e n d t h e p a t h a s t o b a c k t r a c k ( t h e r e
a r e 3 d i r e c t i o n s t o e x t e n d t h e p a t h a n d o n l y o n e t o b a c k t r a c k ) , s o t h a t x
" m o s t l i k e l y " a r o s e f r o m t h e one ( a m o n g t h e f o u r p o s s i b i l i t i e s ) f o r w h i c h x i s
a n e x t e n s i o n . F r a s e r , i n t h e d i s c u s s i o n s o f S t o n e ( 1 9 7 6 ) a n d H i l l ( 1 9 8 1 ) ,
i n d i c a t e s t h a t t h i s s t r i k e s h i m a s a c o n c l u s i v e c o u n t e r e x a m p l e t o t h e L P i t s e l f ,
t h e " l i k e l i
v i e w p o i n t .
t h e " l i k e l i h o o d s " o f ^ s e e m i n g a b s u r d f r o m a f r e q u e n t i s t ( c o n d i t i o n a l o n )
T o u s , t h i s e x a m p l e a g a i n s e r v e s t o i n d i c a t e t h a t t h e l i k e l i h o o d
f u n c t i o n c a n r e a l l y b e u t i l i z e d o n l y t h r o u g h B a y e s i a n a n a l y s i s . F o r i n s t a n c e ,
f o r g e t f o r a m o m e n t t h e a m u s i n g s t r u c t u r e o f S t o n e ' s e x a m p l e , a n d j u s t c o n s i d e r
IMPLEMENTATION OF THE LIKELIHOOD PRINCIPLE 131
th e st a t i st i c a l problem involving and X. Suppose was in a c t ua l i t y generated
according t o t h e ( pr i or ) di st r i but i on
1 / ( 2 3
n
l ) i f length of < _ n
( ) =
0 i f length of > n,
and t h at a path x of length < n is observed. Th en a Bayesian analysis is
ce r t a i nl y cor r e ct , and th e post erior pr obabi l i t y of each of th e possible
given x is indeed j. Returning t o th e example of th e sol di e r , t h i s makes cle ar
t h at i f is f e l t t o h ave e sse nt i al l y a uniform pr i or in th e neigh borh ood of x,
th en th e analysis decried by Stone and Fraser is cor r e ct .
The d i f f i c ul t y h ere is t h at i t was never described "why" th e
soldie r stopped at a given int e r se ct ion and buried t h e t r e asur e , i . e . , how
was generated. We, i n f a c t , would doubt t h at was l oca l l y uniform at any x.
Far more reasonable would be t o assume t h at th e soldie r stops a f t e r a path of
lengt h n, wit h some pr obabi l i t y p
n
, and t h at a l l path s of length n ( t h e r e are
N
n
= 4 3 of them when n _>!) h ave equal pr obabi l i t y of occurring. Th en, i f
is a path of length n,
w( ) = P
n
/ N
n
.
F o r a g i v e n x o f l e n g t h m ^ 1 , t h r e e o f t h e p o s s i b l e a r e o f l e n g t h m + 1 w h i l e
o n e is o f l e n g t h m 1 . T h e p o s t e r i o r p r o b a b i l i t i e s o f t h e s e a r e
(
W W
p
+
V/ V>
( f or th e of length m+1) and
( f or t h e of length m 1) . I f p
+ 1
= P
m
. i> then th ese pr oba bi l i t i e s are y^
and j , r e spe ct ive ly, indicat ing t h at i t ce r t ai nl y is sensible t o presume t h at
t h e t reasure is buried at t h a t f or wh ich x is an extension of th e pat h .
( Th is analysis is yery si mi l ar to t h at of Dickey in th e discussion of Stone
( 197 6 ) and t o th e analysis in Hi l l ( 198 1) using a f i ni t e l y addit ive pr i or on n) .
Th e above considerations can be reinf orced by considering a second
model proposed by Fraser i n t h e discussion of Hi l l ( 1 9 8 1 ) . Fraser' s model is
t h at th e observat ion X is generated from according to t h e f ollowing sch eme,
wh ere x
Q
denotes a given pa r t i cul a r path and 0 th e or i gi n:
W
=
i
and f
e
( 0 ) =
1
when { x
o
N > x
o
E
'
x
o
s
'
x
o
W }
f
( 0 ) = 1 w h e n = x
Q
,
f ( ) = ^ a n d f
Q
( 0 ) = | f o r t h e r e m a i n i n g .
( T h e s o l d i e r t r a i l s a n e l a s t i c s t r i n g , a n d a f t e r b u r y i n g t h e t r e a s u r e a t t h e
e n d o f / x
Q
h e p a s s e s o u t a n d h a s a 7 5 % c h a n c e o f b e i n g s n a p p e d b a c k t o 0 ; t h e
e n d o f x
Q
, h o w e v e r , is v e r y s l i p p e r y , s o i f t h e s o l d i e r b u r i e s t h e t r e a s u r e
t h e r e a n d p a s s e s o u t h e w i l l b e s n a p p e d b a c k t o 0 f o r s u r e . T h e r e a l s o h a p p e n s
t o b e a g o o d S a m a r i t a n w h o w a l k s t h e s t r e e t s w i t h i n o n e b l o c k o f a s h e l t e r a t
x
Q
, a n d i f t h e s o l d i e r p a s s e s o u t a t x
Q
N , X Q S , X
Q
W , o r x ~E a n d d o e s n ' t g e t
s n a p p e d b a c k t o 0 , t h e g o o d S a m a r i t a n w i l l t a k e h i m b a c k t o x
Q
. ) S u p p o s e
n o w t h a t t h e o b s e r v a t i o n f r o m t h i s m o d e l j u s t h a p p e n s t o b e x
Q
, s o t h a t t h e
l i k e l i h o o d f u n c t i o n f o r i s t h e s a m e a s t h a t o b t a i n e d f r o m S t o n e ' s m o d e l f o r
t h e o b s e r v a t i o n x
Q
. T h e L P s a y s t h a t t h e c o n c l u s i o n s in e a c h c a s e s h o u l d b e
t h e s a m e , a n d w e c o n c u r . S i n c e i s s t i l l t h e p a t h g e n e r a t e d b y t h e d r u n k e n
s o l d i e r , t h e p r i o r d e f i n e d b y ( ) = P
n
/ N
n
is s t i l l a p p r o p r i a t e , a n d t h e
r e s u l t i n g B a y e s i a n a n a l y s i s s e n s i b l e . ( A l t e r n a t i v e l y , i f h a d b e e n g e n e r a t e d
i n s u c h a w a y t h a t t h e p r i o r w a s f e l t t o b e l o c a l l y u n i f o r m n e a r x
Q
n o t e
t h a t a n y p r o p e r p r i o r c o u l d o n l y b e l o c a l l y u n i f o r m n e a r s o m e o f t h e p o s s i b l e
o b s e r v a t i o n s t h e B a y e s i a n a n a l y s i s w i t h ( ) = K w o u l d b e a p p r o p r i a t e . )
T h i s B a y e s i a n r e a s o n i n g is i n c o n f l i c t w i t h f r e q u e n t i s t r e a s o n i n g ,
w h i c h s t a t e s i n t h e s i t u a t i o n o f S t o n e t h a t , c o n d i t i o n a l o n , X i s t h r e e
t i m e s a s l i k e l y t o e x t e n d t h e p a t h a s t o b a c k t r a c k , w h i l e i n t h e s i t u a t i o n o f
F r a s e r t h e r e is n o r e a s o n t o t h i n k t h i s . S u c h r e a s o n i n g s e e m s t o b e t h e b a s i s
o f t h e c l a i m b y F r a s e r ( i n t h e d i s c u s s i o n o f H i l l ( 1 9 8 1 ) ) t h a t t h e s i t u a t i o n
p r o v i d e s a c o u n t e r e x a m p l e t o t h e L P . T o u s i t i n s t e a d p r o v i d e s y e t a n o t h e r
c o u n t e r e x a m p l e to f r e q u e n t i s t r e a s o n i n g . If t h e r e is d o u b t as to t h i s ,
i m a g i n e that really w a s g e n e r a t e d a c c o r d i n g to o n e o f t h e p r i o r s c o n s i d e r e d
h e r e (and a c o m p e l l i n g c a s e can b e m a d e t h a t t h e d r u n k e n s o l d i e r a c t u a l l y does
g e n e r a t e a c c o r d i n g to ( ) = P
n
/ N
n
) , in w h i c h c a s e t h e r e s e e m s l i t t l e d o u b t
t h a t t h e t w o m o d e l s g i v e t h e s a m e a n s w e r . (See a l s o B e r g e r ( 1 9 8 4 a ) . )
T h i s e x a m p l e a g a i n s h o w s t h e p o s s i b l e e r r o r in a t t e m p t i n g to b a s e
an a n a l y s i s s o l e l y on ( ) , and s h o w s how t h e B a y e s i a n p e r s p e c t i v e r e s o l v e s
x
th e d i f f i c ul t i e s . One i nt e r e st i ng f eat ure of t h i s example is t h at th e nat ural
noninformative pr i or densit y is const ant , and result s i n th e i l l a dvi se d
( | x) = , f or t h e f our possible . Th e d i f f i c ul t y wit h th e noninformative
pr i or approach h ere is t h at th e parameter space can be viewed as th e f r e e group
on two generators and, as shown by Peisakof f ( 1 9 5 0 ) , t h i s group is too large
f or group based st a t i st i c a l analyses t o work. ( Peisakof f discusses th e
problem from th e viewpoint of invariance t h eory, but invariance th eory has a
very close r e lat ionsh ip wit h nonif ormat ive pr i or Bayesian th eory c . f . Berger
( 1 9 8 0 ) . ) Bondar and Milnes ( 198 1) provide ext ensive discussion concerning when
such groups are "too l ar ge . "
EXAMPLE 3 6 . St ein ( 196 2 ) constructed t h e f ollowing example t o show th e
d i f f i c ul t y in casually applying th e LP. An unknown quant it y > 0 can be
2 2
measured by X ^7 j ( , ) ( known) or by Y h aving density
2
( 5 . 3 . 3 ) f ( y | ) = c y "
1
e x p { ^ (1 ^)
2
} I
( Q b )
( y ) ,
l o
i o o o
w h e r e c is t h e a p p r o p r i a t e n o r m a l i z i n g c o n s t a n t , b is e n o r m o u s ( s a y , 10 ) ,
a n d d is large ( s a y , 5 0 ) . T h e l i k e l i h o o d f u n c t i o n s i ( ) a n d a ( ) for t h e
x y
r e s p e c t i v e e x p e r i m e n t s w o u l d b e ( i g n o r i n g m u l t i p l i c a t i v e c o n s t a n t s a n d
r e c a l l i n g t h a t > 0 )
*
( ) = e x p { \ ( X )
2
} I
( 0 J O O )
( ) ,
2
\
( 0 J O O )
2
S u p p o s e n o w t h a t t h e o b s e r v a t i o n s a r e s u c h t h a t x = y = d . T h e n
t h e o n l y d i f f e r e n c e b e t w e e n l i e ) a n d i ( ) is t h e d i f f e r e n c e b e t w e e n t h e
x y
f a c t o r s I
/ n N
( ) a n d 1/
/ k
J ) , w h i c h c a n b e s h o w n t o b e n e g l i g i b l e b e c a u s e
( 0 , ; \ y / o> < * > )
b is s o h u g e . T h u s t h e L P s a y s t h a t t h e g i v e n o b s e r v a t i o n s , x a n d y , p r o v i d e
( e s s e n t i a l l y ) t h e s a m e i n f o r m a t i o n a b o u t . W e a g r e e w i t h t h i s e n t i r e l y .
N e x t , S t e i n o b s e r v e s t h a t t h e u s u a l , s a y 9 5 % , f r e q u e n t i s t o r
o b j e c t i v e c o n d i t i o n a l c o n f i d e n c e i n t e r v a l f o r w h e n x is o b s e r v e d i s
( 5 . 3 . 4 ) ( x ( 1 . 9 6 ) , x + ( 1 . 9 6 ) )
( n o t e t h a t x / = d = 5 0 , s o t h e r e s t r i c t i o n t o > 0 is e s s e n t i a l l y i r r e l e v a n t ) ,
a n d h e n c e t h a t a p p l i c a t i o n o f t h e L P i m p l i e s t h a t t h e i n t e r v a l
( 5 . 3 . 5 ) ( y ( 1 . 9 6 ) [ y / d j , y + ( 1 . 9 6 ) [ y / d ] )
sh ould be used i f y is observed. Again we agree, providing t h e i nt e r va l in
( 5 . 3 . 4 ) is inappr opr iat e.
Considering now t h e i nt e r va l i n ( 5 . 3 . 5 ) as a f r e que nt ist i nt e r val
( t o be used f or a l l y ) , a calculat ion shows (see Berger ( 1 9 8 0 ) ) t h at
(5.3.6) P(Y UML < < Y + H L) < 10
1 0 0
.
T h i s , t o a f r e q u e n t i s t , c a s t s e x t r e m e d o u b t o n t h e p r e m i s e t h a t t h e i n t e r v a l
in ( 5 . 3 . 5 ) c o n t a i n s , a n d s e e m s t o i n d i c a t e a f a i l u r e o f t h e L P .
T o a B a y e s i a n , t h e r e i s n o r e a l p r o b l e m w i t h t h i s e x a m p l e . T h e
u s e o f ( 5 . 3 . 5 ) w a s p r e d i c a t e d o n t h e v a l i d i t y o f ( 5 . 3 . 4 ) , w h i c h i n t u r n f o l l o w s
o n l y i f t h e p r i o r i s a p p r o x i m a t e l y l o c a l l y u n i f o r m w i t h i n s e v e r a l s t a n d a r d
d e v i a t i o n s o f t h e a c t u a l o b s e r v a t i o n x ( a n d is w e l l b e h a v e d o u t s i d e t h i s
r e g i o n ) . I n r e a l i t y , t h i s w i l l n e v e r b e t h e c a s e f o r a l l x a n d ; a n y p r op e r
p r i o r w i l l g i v e s u b s t a n t i a l l y d i f f e r e n t r e s u l t s a s x a n d p a r t i c u l a r l y v a r y .
I n d e e d , n o t e t h a t i t w a s a s s u m e d t h a t x = y = d i n t h e a b o v e c o n d i t i o n a l
a n a l y s i s , a n d s i n c e i t c a n b e s h o w n t h a t Y is a l m o s t c e r t a i n t o b e e n o r m o u s
( o n t h e o r d e r o f b i n s i z e ) , i t f o l l o w s t h a t w e m u s t i m a g i n e t h a t x a n d a r e
a l s o e n o r m o u s . T h e u s e o f ( 5 . 3 . 4 ) , w h e n x a n d a r e e n o r m o u s , w i l l r a r e l y b e
c o n d i t i o n a l l y s o u n d . It i s t h i s u s e o f ( 5 . 3 . 4 ) , n o t t h e u s e o f t h e L P , w h i c h
i s i n e r r o r . A n d i f a v e r y s m a l l y j u s t h a p p e n s t o o c c u r , t h e n a n d o n l y t h e n
is u s e o f ( 5 . 3 . 4 ) , a n d h e n c e ( 5 . 3 . 5 ) , i n d i c a t e d .
T w o o b s e r v a t i o n s c o n c e r n i n g a B a y e s i a n a n a l y s i s o f t h i s p r o b l e m a r e
i n o r d e r . T h e f i r s t is t h a t c l e v e r B a y e s i a n r e a s o n i n g i s n o t r e q u i r e d t o s h o w
t h e i n a d e q u a c y o f t h e i n t e r v a l i n ( 5 . 3 . 5 ) f o r a l l b u t v e r y s m a l l y . I n d e e d ,
f o r v i r t u a l l y a n y p r i o r d i s t r i b u t i o n , t h e i n t e r v a l i n ( 5 . 3 . 5 ) w i l l h a v e
p o s t e r i o r p r o b a b i l i t y n e a r z e r o . T h e s e c o n d o b s e r v a t i o n is t h a t a s t a n d a r d
n o n i n f o r m a t i v e p r i o r B a y e s i a n a n a l y s i s d oe s w o r k w e l l h e r e . F o r t h e d e n s i t y
i n ( 5 . 3 . 3 ) , c a n e a s i l y b e s e e n t o b e a s c a l e p a r a m e t e r , a n d a g a i n t h e
s t a n d a r d n o n i n f o r m a t i v e p r i o r d e n s i t y w o u l d b e ( ) = . A B a y e s i a n a n a l y s i s
w i t h t h i s i m p r o p e r p r i o r g i v e s w e r y s e n s i b l e a n s w e r s a n d s h o w s t h e i n t e r v a l i n
( 5 . 3 . 5 ) t o b e s e r i o u s l y i n a d e q u a t e f o r a l l b u t y e r y s m a l l y .
It i s w o r t h w h i l e t o s u m m a r i z e t h e t h r e e m a i n p o i n t s t h a t a r e
i l l u s t r a t e d i n t h e a b o v e e x a m p l e s .
1. I n t u i t i v e u t i l i z a t i o n o f l i k e l i h o o d f u n c t i o n s c a n b e m i s l e a d i n g . In
E x a m p l e s 3 4 a n d 3 5 , f o r i n s t a n c e , t h e u s u a l i n t e r p r e t a t i o n o f a l i k e l i h o o d
f u n c t i o n a s a m e a s u r e o f t h e c o m p a r a t i v e s u p p o r t o f t h e d a t a f o r t h e v a r i o u s ,
w h i l e f o r m a l l y c o r r e c t , c a n l e a d t o a n e r r o n e o u s c o n c l u s i o n i f p r i o r i n f o r m a t i o n
i s n o t c o n s i d e r e d .
E x a m p l e 3 6 a l s o d e m o n s t r a t e s t h a t i n t u i t i v e a p p r o a c h e s w h i c h w o r k
w e l l i n a c e r t a i n s i t u a t i o n s h o u l d n o t b e c a r e l e s s l y t r a n s f e r r e d t o d i f f e r e n t
s i t u a t i o n s w i t h a s i m i l a r l i k e l i h o o d f u n c t i o n . It is t r u e t h a t , w h e n p r i o r
i n f o r m a t i o n i s v a g u e i n t h e n o r m a l m e a n s i t u a t i o n , t h e " c o n f i d e n c e " i n t e r v a l
( 5 . 3 . 7 ) ( x ( 1 . 9 6 ) , x + ( 1 . 9 6 ) )
is a r e a s o n a b l e c o n d i t i o n a l p r o c e d u r e . N a i v e l y t r a n s f e r r i n g t h i s t o t h e Y
s i t u a t i o n f a i l s , h o w e v e r , b e c a u s e ( 5 . 3 . 7 ) is r e a s o n a b l e o n l y w h e n is s m a l l
e n o u g h f o r t h e p r i o r i n f o r m a t i o n t o i n d e e d b e v a g u e , a n d t h e Y p r o b l e m i n v o l v e s
o b s e r v a t i o n s w h i c h w i l l u s u a l l y c o r r e s p o n d t o h u g e . T h i s " e r r o r " i s n o t e d
a n d e x t e n s i v e l y d i s c u s s e d i n B a s u ( 1 9 7 5 ) .
2 . W h i l e n o t d i r e c t l y r e l a t e d t o o u r c e n t r a l t h e s i s , t h e s e e x a m p l e s i n d i c a t e
t h e c a r e n e e d e d in t h e u s e o f i m p r o p e r " n o n i n f o r m a t i v e " p r i o r s . W h e n p r i o r
o p i n i o n s a r e i n d e e d r e f l e c t e d b y a l o c a l l y n o n i n f o r m a t i v e p r i o r ( i n t h e r e g i o n
o f f o r w h i c h t h e l i k e l i h o o d f u n c t i o n is s i g n i f i c a n t ) , t h e u s e o f n o n i n f o r m a
t i v e p r i o r s is r e a s o n a b l e a s a n a p p r o x i m a t i o n . ( S e e a l s o B o x a n d T i a o ( 1 9 7 3 ) ,
D i c k e y ( 1 9 7 6 ) , a n d B e r g e r ( 1 9 8 4 e ) . ) It a p p e a r s , h o w e v e r , e s p e c i a l l y f r o m
E x a m p l e 3 5 , t h a t a u t o m a t i c u s e o f n o n i n f o r m a t i v e p r i o r s c a n l e a d o n e a s t r a y .
T h i s i s n o t t o s a y t h a t u s e o f n o n i n f o r m a t i v e p r i o r s is t o b e a v o i d e d ; i n d e e d
w e f e e l t h a t t h e y a r e i n v a l u a b l e i n o b t a i n i n g r e l a t i v e l y s i m p l e , g o o d , a n d
" o b j e c t i v e " s t a t i s t i c a l p r o c e d u r e s .
3 . T h e s e e x a m p l e s c a n b e t u r n e d a r o u n d a n d u s e d a s i n d i c t m e n t s o f f r e q u e n c y
r e a s o n i n g . F r e q u e n c y r e a s o n i n g i n e a c h e x a m p l e w o u l d c o r r e s p o n d ( a t b e s t ) t o
B a y e s i a n a n a l y s i s w i t h r e s p e c t t o a c e r t a i n , v e r y s p e c i a l , p r i o r . Qu i t e
d i f f e r e n t a n s w e r s w e r e s e e n t o o b t a i n i f o t h e r p r i o r b e l i e f s w e r e h e l d . T h i s ,
o f c o u r s e , i s a n o t h e r g e n e r a l j u s t i f i c a t i o n f o r t h e B a y e s i a n p o s i t i o n : a " g o o d "
f r e q u e n t i s t p r o c e d u r e i s u s u a l l y a B a y e s p r o c e d u r e w i t h r e s p e c t t o s o m e p r i o r ,
a n d i f t h e c o r r e s p o n d i n g p r i o r d o e s n o t s e e m r e a s o n a b l e , u s e o f t h e p r o c e d u r e
i s s u s p e c t .
5 . 4 R O B U S T B A Y E S I A N A N A L Y S I S
W e s e e m t o h a v e b e e n i n e x o l o r a b l y l e d t o B a y e s i a n a n a l y s i s . O u r
i n t e r p r e t a t i o n o f t h e s i t u a t i o n a t t h i s p o i n t i s t h a t w e c a n b e s t i n t e r p r e t
t h e i n f o r m a t i o n f r o m t h e d a t a , n a m e l y ,
v
( ) , a s a p r o b a b i l i t y d e n s i t y o n
x
w . r . t . s o m e p r i o r m e a s u r e , , r e f l e c t i n g o u r p r i o r b e l i e f s ( o r l a c k t h e r e o f )
c o n c e r n i n g . T h u s o n e n e e d o n l y e l i c i t h i s p r i o r d i s t r i b u t i o n ,
Q
, a n d
p e r f o r m a B a y e s i a n a n a l y s i s .
U n f o r t u n a t e l y , e l i c i t a t i o n o f
Q
i s n o t e a s y , a n d i n d e e d c a n n o t b e
d o n e w i t h c o m p l e t e a c c u r a c y i n a f i n i t e a m o u n t o f t i m e . ( W e a r e t h i n k i n g o f
Q
a s t h e p r i o r w h i c h w o u l d b e t h e r e s u l t o f i n f i n i t e l y l o n g r e f l e c t i o n o n
t h e p r o b l e m . ) It is n o t c l e a r t h a t w r i t i n g d o w n a q u i c k g u e s s a t
Q
a n d
p e r f o r m i n g a B a y e s i a n a n a l y s i s w i t h t h i s g u e s s i s b e t t e r t h a n o t h e r n o n
B a y e s i a n m e t h o d s o f a n a l y s i s . T h e f e a r is t h a t t h e g u e s s f o r
Q
m i g h t c o n t a i n
f e a t u r e s w h i c h w o u l d b e d e e m e d t o b e in e r r o r u p o n f u r t h e r r e f l e c t i o n , a n d t h a t
t h e s e f e a t u r e s m i g h t h a v e s u c h a n o v e r w h e l m i n g l y d e t r i m e n t a l e f f e c t o n t h e
a n a l y s i s t h a t a c l a s s i c a l a n a l y s i s w h i c h i g n o r e s p r i o r i n f o r m a t i o n m i g h t b e
p r e f e r a b l e .
T h e o b v i o u s m e t h o d o f a l l e v i a t i n g s u c h f e a r s i s t o d o r o b u s t
B a y e s i a n a n a l y s i s ( s e e B e r g e r 1 9 8 4 e , 1 9 8 5 , a n d 1 9 8 7 f o r g e n e r a l s u r v e y s a n d
r e f e r e n c e s ) , w h e r e i n o n e c o n s i d e r s , i n s t e a d o f a s i n g l e g u e s s f o r r
, a c l a s s r
o f p l a u s i b l e p r i o r d i s t r i b u t i o n s f e l t c e r t a i n t o c o n t a i n
Q
. F r o m r ( a n d t h e
l i k e l i h o o d f u n c t i o n ) o n e o b t a i n s a c l a s s o f p o s s i b l e p o s t e r i o r d i s t r i b u t i o n s
t o w o r k w i t h . ( N o t e t h a t , in t h i s r o b u s t B a y e s i a n s e n s e , E v ( E , x ) r e a l l y i s a
s e t o f " e v i d e n c e s " . ) I f t h e c o n c l u s i o n o r a c t i o n t o b e t a k e n is e s s e n t i a l l y
t h e s a m e f o r a l l s u c h p o s t e r i o r s , t h e n t h e p r o b l e m i s s o l v e d . I n d e e d , in a
s e n s e t h i s i s t h e ov u y s i t u a t i o n in w h i c h t h e r e c a n b e s a i d t o b e a n
u n e q u i v o c a l a n s w e r t o a p r o b l e m . ( T h i s h o l d s t r u e a l s o w h e n r i s a c l a s s o f
p r i o r s o f v a r i o u s i n d i v i d u a l s w h o m u s t c o m e t o a j o i n t c o n c l u s i o n . )
It m a y h a p p e n , h o w e v e r , t h a t t h e c o n c l u s i o n o r a c t i o n t o b e t a k e n
i s q u i t e d i f f e r e n t f o r v a r i o u s p o s t e r i o r s in t h e c l a s s . W h e n t h i s i s t h e
c a s e t h e r e a r e f o u r o p t i o n s : ( i ) A t t e m p t f u r t h e r p r i o r e l i c i t a t i o n ( r e s u l t i n g
in a n a r r o w i n g o f r ) ; ( i i ) O b t a i n m o r e d a t a ; ( i i i ) C o n c l u d e t h a t t h e r e is n o
a n s w e r ; a n d ( i v ) C h o o s e a m o n g t h e p o s s i b l e a n s w e r s a c c o r d i n g t o s o m e c r i t e r i a
n o t i n v o l v i n g f u r t h e r p r i o r e l i c i t a t i o n . S o l u t i o n s ( i ) a n d ( i i ) a r e c e r t a i n l y
t o b e a t t e m p t e d , i f p o s s i b l e , b u t l i m i t e d t i m e o r r e s o u r c e s m a y p r e c l u d e s u c h
s o l u t i o n s (an e x a m p l e o f G o o d ' s T y p e II r a t i o n a l i t y ) . N o t e t h a t s o l u t i o n
( i ) m a y b e s o m e w h a t s i m p l e r t h a n it s e e m s a t f i r s t s i g h t , s i n c e t h e o b s e r v e d
d a t a m a y e f f e c t i v e l y r u l e o u t a l a r g e p o r t i o n o f r , m e a n i n g t h a t f u r t h e r p r i o r
e l i c i t a t i o n c a n b e c o n c e n t r a t e d o n s p e c i f i c a s p e c t s o f t h e p r o b l e m . S o l u t i o n
( i i i ) i s c e r t a i n l y r e a s o n a b l e , a n d is i n s o m e s e n s e t h e o n l y t r u l y h o n e s t
c o n c l u s i o n i f ( i ) o r ( i i ) c a n n o t b e p u r s u e d . B u t in m a n y s i t u a t i o n s it i s
n e c e s s a r y t o p r o c e e d a n y w a y a n d o b t a i n a n " i n t e l l i g e n t " g u e s s a t t h e a n s w e r .
T h i s b r i n g s u s t o s o l u t i o n ( i v ) , i . e . , t h e u s e o f a l t e r n a t e
c r i t e r i a . T h e r e a r e m a n y p o s s i b i l i t i e s h e r e , w i t h t h e f o l l o w i n g f i v e b e i n g
t h e m o s t i m p o r t a n t :
1 . P u t a p r i o r d i s t r i b u t i o n on r i t s e l f , a n d c a r r y o u t a f o r m a l B a y e s i a n
a n a l y s i s . ( N o t e t h a t t h i s w o u l d b e s i m p l y a f o r m a l p r i o r d i s t r i b u t i o n o f
s o m e s o r t , s i n c e t h e p r i o r e l i c i t a t i o n p r o c e s s h a s s u p p o s e d l y c e a s e d . )
2 . U s e m i n i m a x t y p e c r i t e r i a o n p o s t e r i o r m e a s u r e s ( e . g . , p o s t e r i o r e x p e c t e d
l o s s e s ) f o r e r.
3 . U s e f r e q u e n t i s t m e a s u r e s t o s e l e c t a " g o o d " p r o c e d u r e c o m p a t i b l e w i t h
G .
4 . U s e s o m e m e a s u r e o f " i n f o r m a t i o n " t o s e l e c t a p r i o r in r , s u c h a s a
" m a x i m u m e n t r o p y " p r i o r ( c f . J a y n e s ( 1 9 8 2 ) ) o r a " r e f e r e n c e " p r i o r
( c f . B e r n a r d o ( 1 9 7 9 ) ) .
5 . U s e T y p e II m a x i m u m l i k e l i h o o d m e t h o d s ( c f . G o o d ( 1 9 6 5 ) ) , e s s e n t i a l l y
c h o o s i n g t h e p r i o r G w h i c h m a x i m i z e s t h e m a r g i n a l o r p r e d i c t i v e
d e n s i t y m ( x | ) = E
[ f
0
( x ) ] f o r t h e g i v e n d a t a x ( s u c h a p r i o r b e i n g t h e
" m o s t p l a u s i b l e " p r i o r in r in l i g h t o f t h e d a t a ) . T h i s is a s t a n d a r d
a d h o c B a y e s i a n a n d e m p i r i c a l B a y e s i a n t e c h n i q u e .
D i s c u s s i o n a n d o t h e r r e f e r e n c e s f o r t h e s e m e t h o d s c a n b e f o u n d in B e r g e r
( 1 9 8 4 e , 1 9 8 5 ) a n d B e r g e r a n d B e r l i n e r ( 1 9 8 6 ) . O f i n t e r e s t h e r e is t h a t t w o o f
t h e s e m e t h o d s , n a m e l y m e t h o d s 3 a n d 4 , c a n v i o l a t e t h e L P . ( M e t h o d 4 c a n
v i o l a t e t h e L P b e c a u s e t h e s e l e c t e d p r i o r w i l l t y p i c a l l y d e p e n d o n a l l o f E ,
n o t j u s t t h e o b s e r v e d l i k e l i h o o d f u n c t i o n . ) W e w i l l n o t e n t e r i n t o a d i s c u s
s i o n o f t h e r e l a t i v e m e r i t s o f t h e f i v e m e t h o d s , b u t d o n o t e t h a t t h e r e s e e m t o
b e s t a t i s t i c a l p r o b l e m s t h a t a r e m o s t a m e n a b l e t o s o l u t i o n b y e a c h o f t h e m e t h
o d s . F o r i n s t a n c e , t h e r e a r e m a n y h i g h d i m e n s i o n a l a n d n o n p a r a m e t r i c p r o b l e m s
w h e r e it i s h a r d t o f i n d a ny r e a s o n a b l e p r i o r d i s t r i b u t i o n , m u c h l e s s d o a r o b u s t
B a y e s i a n a n a l y s i s , a n d y e t r e l a t i v e l y s i m p l e f r e q u e n t i s t p r o c e d u r e s e x i s t w h i c h
c a n b e m e a n i n g f u l t o a c o n d i t i o n a l i s t i n t h e s e n s e o f E x a m p l e 1 6 in S e c t i o n
4 . 1 . 3 . C o n s i d e r t h e f o l l o w i n g e x a m p l e , w h i c h w e l e a r n e d f r o m B r a d E f r o n .
E XA M P L E 3 7 . T h e e x p e r i m e n t , E , c o n s i s t s o f o b s e r v i n g X, , . . .
f
X
1 5
, w h i c h a r e
i . i . d . o b s e r v a t i o n s f r o m a c o m p l e t e l y u n k n o w n c o n t i n u o u s d e n s i t y f o n R
.
( H e r e w e i d e n t i f y w i t h t h e u n k n o w n f, s o i s t h e s e t o f a l l c o n t i n u o u s
d e n s i t i e s o n R . ) O f i n t e r e s t is , t h e m e d i a n o f t h e u n k n o w n d e n s i t y . A
s i m p l e b i n o m i a l c a l c u l a t i o n s h o w s t h a t a 9 6 . 5 % f r e q u e n t i s t c o n f i d e n c e i n t e r v a l
f o r i s g i v e n b y [ X(
4
\ .
x
( ) l >
w h e r e t h e X
(
)
a r e t h e o r d e r
s t a t i s t i c s .
D u e t o t h e e x t r e m e d i f f i c u l t y o f c o n s t r u c t i n g r e a s o n a b l e p r i o r d i s t r i b u t i o n s
o n , a B a y e s i a n m i g h t w e l l c h o o s e t o s i m p l y u s e [ X/ ^v , X /
1 1
% ] , w i t h t h e
i n t e r p r e t a t i o n p r o v i d e d b y E x a m p l e 1 6 .
T h u s , b e c a u s e o f d i f f i c u l t i e s i n p e r f o r m i n g a r o b u s t B a y e s i a n
a n a l y s i s , a c o n d i t i o n a l i s t m i g h t f o r m a l l y v i o l a t e t h e L P . O f c o u r s e , t h i s
c o u l d b e v i e w e d a s m e r e l y a t e m p o r a r y c o n d i t i o n d u e t o t h e l a c k o f d e v e l o p m e n t
o f B a y e s i a n t h e o r y ; c e r t a i n l y g r e a t e r e f f o r t h a s b e e n e x p e n d e d b y s t a t i s t i c i a n s
o n d e v e l o p m e n t o f n o n B a y e s i a n t h e o r y . A l s o , t h e n e e d t o c o m p r o m i s e s h o u l d n o t
b e v i e w e d a s p r o v i d i n g l e g i t i m a c y t o t h e c o m p r o m i s e s , b u t s h o u l d i n s t e a d b e
v i e w e d a s a f o r c e d s t a b in t h e d a r k . T h u s S a v a g e , i n S a v a g e e t . a l . ( 1 9 6 2 ) ,
s t a t e s
"I u s e d t o b e b o w e d b y c r i t i c s w h o s a i d ,
w i t h a p p a r e n t t e c h n i c a l j u s t i f i c a t i o n , t h a t
c e r t a i n p o p u l a r n o n p a r a m e t r i c t e c h n i q u e s
a p p l y i n s i t u a t i o n s w h e r e i t s e e m s m e a n i n g
l e s s e v e n t o t a l k o f a l i k e l i h o o d f u n c t i o n ,
b u t I h a v e l e a r n e d t o e x p e c t t h a t e a c h o f t h e s e
t e c h n i q u e s e i t h e r h a s a B a y e s i a n v a l i d a t i o n o r
w i l l b e f o u n d t o h a v e o n l y i l l u s o r y v a l u e a s a
m e t h o d o f i n f e r e n c e . "
A s e c o n d r e a s o n f o r p o s s i b l e v i o l a t i o n o f t h e L P , a s d i s c u s s e d i n
S e c t i o n 4 . 1 . 3 , is t h a t m a n y u s e r s o f s t a t i s t i c s w i l l b e u n a b l e t o p e r f o r m
c a r e f u l r o b u s t B a y e s i a n a n a l y s e s . F o r t h e s e u s e r s w e m u s t p r o v i d e s i m p l e
B a y e s i a n p r o c e d u r e s w i t h " b u i l t i n " r o b u s t n e s s . I n p a r t , t h i s r o b u s t n e s s
s h o u l d b e m e a s u r e d i n a f r e q u e n c y s e n s e , s i n c e t h e p r o c e d u r e s w i l l b e u s e d
r e p e a t e d l y ( i . e . , f o r d i f f e r e n t X ) . O f c o u r s e , g o o d c o n d i t i o n a l p e r f o r m a n c e
o f t h e s e p r o c e d u r e s s h o u l d s t i l l b e o f p a r a m o u n t c o n c e r n . N o t e , i n p a r t i c u l a r ,
t h a t t h e n o n i n f o r m a t i v e p r i o r o r " o b j e c t i v e " B a y e s i a n p r o c e d u r e s a r e u s u a l l y
v e r y g o o d p r o c e d u r e s f r o m t h i s p e r s p e c t i v e o f u s e b y n o n s p e c i a l i s t s , a n d m a y
f o r m a l l y v i o l a t e t h e L P t h r o u g h d e p e n d e n c e o f t h e n o n i n f o r m a t i v e p r i o r o n E
( a n d n o t j u s t t h e o b s e r v e d
v
( ) ) . In a s i m i l a r v e i n , H a j e k ( 1 9 6 7 , 1 9 7 1 ) a r g u e s
x
t h a t a s y m p t o t i c t h e o r y ( w h i c h c a n p r o v i d e u s e f u l s i m p l e p r o c e d u r e s f o r
n o n s p e c i a l i s t s in c o m p l i c a t e d s i t u a t i o n s ) c a n s o m e t i m e s b e m o r e d i f f i c u l t i f
o n e i s r e s t r i c t e d t o b a s i n g i t o n l y o n t h e g i v e n l i k e l i h o o d f u n c t i o n , a n d n o t
o n E as a w h o l e .
A s a f i n a l c o m m e n t c o n c e r n i n g B a y e s i a n a n a l y s i s , it s h o u l d b e
m e n t i o n e d t h a t c h o i c e o f a p r i o r ( o r c l a s s r ) w i l l o f t e n h a v e t o w a i t u n t i l
a f t e r t h e d a t a is a t h a n d a n d ,
() is a v a i l a b l e . T h u s i n B a r n a r d , J e n k i n s ,
a n d W i n s t e n ( 1 9 6 2 ) i t is s t a t e d ( w h e r e t h e y r e f e r t o " w e i g h t s
1 1
i n s t e a d o f a
" p r i o r " )
" T h e a d v a n t a g e o f l o o k i n g f i r s t a t t h e
l i k e l i h o o d f u n c t i o n a n d t h e n c o n s i d e r i n g
t h e w e i g h t s , l i e s in t h e f a c t t h a t t h e
l i k e l i h o o d f u n c t i o n w i l l o f t e n b e s o n e a r
z e r o o v e r m u c h o f t h e r a n g e o f t h a t t h e
w e i g h t s i n t h e s e r e g i o n s c a n b e q u i c k l y
d i s m i s s e d f r o m c o n s i d e r a t i o n . "
T h i s " c h o o s i n g t h e p r i o r a f t e r s e e i n g t h e d a t a " s t r i k e s m a n y a s u n s a v o r y , b u t
i t i s a b s o l u t e l y e s s e n t i a l w h e n i s h i g h d i m e n s i o n a l o r o t h e r w i s e c o m p l i c a t e d .
It is l e s s d i s t u r b i n g w h e n v i e w e d f r o m t h e r o b u s t B a y e s i a n v i e w p o i n t , w h e r e a
c o n c l u s i o n is d e e m e d c l e a r c u t o n l y w h e n a n y r e a s o n a b l e p r i o r p a s s e d o v e r i ( )
g i v e s e s s e n t i a l l y t h e s a m e a n s w e r . S e e B e r g e r ( 1 9 8 4 e ) f o r f u r t h e r d i s c u s s i o n .
I M P L E M E N T A T I O N O F T H E L I K E L I H O O D P R I N C I P L E 141
5 . 5 C O N C L U S I O N S
A t f i r s t s i g h t , w e s e e m t o h a v e c o m e t o t h e c o n c l u s i o n t h a t t h e L P
i s n o t a l w a y s a p p l i c a b l e , i n t h a t t h e o n l y s a t i s f a c t o r y m e t h o d o f a n a l y s i s
b a s e d o n t h e L P s e e m s t o b e r o b u s t B a y e s i a n a n a l y s i s , w h i c h b e c a u s e o f
t e c h n i c a l d i f f i c u l t i e s m a y s o m e t i m e s r e q u i r e u s e o f t e c h n i q u e s t h a t f o r m a l l y
v i o l a t e t h e L P . W e e m p h a t i c a l l y b e l i e v e , h o w e v e r , t h a t t h e L P i s a l w a y s v a l i d ,
in t h e s e n s e t h a t t h e e x p e r i m e n t a l e v i d e n c e c o n c e r n i n g i s c o n t a i n e d i n
x
( ) .
B e c a u s e o f l i m i t e d t i m e a n d r e s o u r c e s , h o w e v e r , i n t e r p r e t i n g o r m a k i n g u s e o f
t h i s e v i d e n c e m a y i n v o l v e u s e o f m e a s u r e s v i o l a t i n g t h e L P . O f c o u r s e , w h e n e v e r
s u c h a m e a s u r e i s u s e d o n e s h o u l d m a k e s u r e t h a t i t h a s n o t l e d t o a
r e c o g n i z a b l y e r r o n e o u s c o n d i t i o n a l c o n c l u s i o n .
U n t i l n o w ( i n t h i s s e c t i o n ) w e h a v e a s s u m e d t h e e x i s t e n c e o f i ( ) .
A s m e n t i o n e d i n S e c t i o n s 3 . 4 a n d 3 . 6 . 1 , t h i s a s s u m p t i o n is ( i n a s e n s e ) a l w a y s
v a l i d , s i n c e t h e s a m p l e s p a c e i s a l w a y s f i n i t e in r e a l i t y a n d t h e n ( )
x
a l w a y s e x i s t s , e v e n w h e n t h e m o d e l is u n c e r t a i n o r u n k n o w n . P r a c t i c a l
c o n s i d e r a t i o n s o f t e n c a l l f o r t h e u s e o f c o n t i n u o u s a p p r o x i m a t i o n s , h o w e v e r ,
f o r w h i c h t h e l i k e l i h o o d f u n c t i o n m a y b e i l l d e f i n e d o r n o t e x i s t . ( O f c o u r s e ,
a s m e n t i o n e d i n S e c t i o n 3 . 6 . 1 , e v e n in m a n y c o n t i n u o u s n o n p a r a m e t r i c s i t u a t i o n s
t h e l i k e l i h o o d f u n c t i o n c a n b e c o n s i d e r e d t o e x i s t . ) In a n y c a s e , t h e R L P
a l w a y s a p p l i e s , a n d a g o o d c a s e c a n a l s o b e m a d e t h a t r o b u s t B a y e s i a n a n a l y s i s
i s t h e o n l y r e a s o n a b l e m e t h o d o f a n a l y s i s c o n s i s t e n t w i t h i t . M o r e f r e q u e n t
c o m p r o m i s e s m a y , h o w e v e r , b e n e e d e d i n t h e s e m o r e d i f f i c u l t s i t u a t i o n s .
E v e n f o r t h o s e w h o f i n d t h e m s e l v e s u n a b l e t o a c c e p t B a y e s i a n
m e t h o d s , t h e L P s h o u l d n o t b e i g n o r e d a n d t h e c o n d i t i o n a l v i e w p o i n t s h o u l d b e
k e p t in m i n d . I f a c l a s s i c a l p r o c e d u r e i s b e i n g u s e d , a q u i c k c h e c k t o m a k e
s u r e t h a t i t is s a y i n g s o m e t h i n g w h i c h is a t l e a s t s e n s i b l e c o n d i t i o n a l l y s e e m s
o n l y p r u d e n t . S t a t i s t i c s l o o k s v e r y b a d w h e n it r e c o m m e n d s a c o n c l u s i o n t h a t
c l e a r l y c o n t r a d i c t s c o m m o n s e n s e .
R E F E R E N C E S
A I T C H I S O N , J. a n d D U N S M O R E , I. R. ( 1 9 7 5 ) . S t a t i s t i c a l P r e d i c t i o n A n a l y s i s .
C a m b r i d g e U n i v e r s i t y P r e s s , C a m b r i d g e .
A K A I K I , H . ( 1 9 8 2 ) . O n t h e f a l l a c y o f t h e l i k e l i h o o d p r i n c i p l e . S t a t i s t i c s a n d
P r o b a b i l i t y L e t t e r s 1 , 7 5 7 8 .
A M A R I , S. ( 1 9 8 2 ) . G e o m e t r i c a l t h e o r y o f a s y m p t o t i c a n c i l l a r i t y a n d c o n d i t i o n a l
i n f e r e n c e . B i o m e t r i k a 6 9
3
1 1 8 .
A N D E R S E N , E . B . ( 1 9 7 0 ) . A s y m p t o t i c p r o p e r t i e s o f c o n d i t i o n a l m a x i m u m l i k e l i
h o o d e s t i m a t o r s . J . R o y . S t a t i s t . S o c . B 3 2
2 8 3 3 0 1 .
A N D E R S E N , E. B . ( 1 9 7 1 ) . A s t r i c t l y c o n d i t i o n a l a p p r o a c h in e s t i m a t i o n t h e o r y .
S h a n d . A k t u a r i e t i d s k r . 5 4
3
3 9 4 9 .
A N D E R S E N , E. B . ( 1 9 7 3 ) . C o n d i t i o n a l I n f e r e n c e a n d M o d e l s f o r M e a s u r i n g .
M e n t a l h y g i e j n i s k F o r l a g , C o p e n h a g e n .
A N S C O M B E , F. J. ( 1 9 6 3 ) . S e q u e n t i a l m e d i c a l t r i a l s . J . A m e r . S t a t i s t . A s s o c .
5 8
3
3 6 5 3 8 3 .
A R M I T A G E , P . ( 1 9 6 1 ) . C o n t r i b u t i o n t o t h e d i s c u s s i o n o f C . A . B . Sm i t h
' C o n s i s t e n c y in s t a t i s t i c a l i n f e r e n c e a n d d e c i s i o n
1
. J . B o y . S t a t i s t . S o c .
B 2 3 , 1 3 7 .
A S P I N , A . A . ( 1 9 4 9 ) . T a b l e s f o r u s e in c o m p a r i s o n s w h o s e a c c u r a c y i n v o l v e s t w o
v a r i a n c e s , s e p a r a t e l y e s t i m a t e d ( w i t h a n a p p e n d i x b y B . L . W e l c h ) .
B i o m e t r i k a 3 6 > 2 9 0 2 9 6 .
B A H A D U R , R. R. ( 1 9 5 4 ) . S u f f i c i e n c y a n d s t a t i s t i c a l d e c i s i o n f u n c t i o n s . A n n .
M a t h . S t a t i s t . 2 5 , 4 2 3 4 6 2 .
143
BARNARD, 6. A. ( 1 9 4 7 a ) . A review of ' Sequent ia l Ana lysis' by Abraham Wald.
J. Amer. Statist. Assoo. 42, 65 8 669 .
BARNARD, G. A. ( 1 9 4 7 b) . The meaning of significa nce l e ve l . Biometrika 34,
1 7 9 1 8 2 .
BARNARD, G. A. ( 1 9 4 9 ) . St a t i st i c a l inference ( wit h Discussion) . J. Roy.
Statist. Soc. B 11, 1 1 5 1 3 9 .
BARNARD, G. A. ( 1 9 62 ) . Comments on St e in' s remark on the l i ke l i hood
p r i nc i p l e
1
. J. Roy. Statist. Soc. A 125, 5 69 5 7 3 .
BARNARD, G. A. ( 1 9 67 a ) . The use of the l i ke l i hood funct ion in st a t i st i c a l
inference. In Proc. 5th Berkeley Symp. on Math. Statist, and Prob.,
Universit y of Ca l i for ni a Press, Berkeley.
BARNARD, G. A. ( 1 9 67 b) . The Ba yesia n controversy in st a t i st i c a l infe r e nce .
J. Inst. Actuaries 93, 2 2 9 2 69 .
BARNARD, G. A. ( 1 9 7 1 ) . Sc i e nt i f i c inferences and da y to da y decisions. I n
Foundations of Statistical Inference, V. P. Godambe and D. A. Sprott
( e ds. ) . Hol t , Rineha rt and Winston, Toronto.
BARNARD, G. A. ( 1 9 7 4 ) . On l i ke l i hood. I n t he Proceedings of the Conference on
Foundational Questions in Statistical Inference, 0. Barndorff Nielsen,
P. Bl a e si l d, and G. Schou ( e ds. ) . Department of Theoret ica l St a t i st i c s,
Unive r sit y of Aa rhus.
BARNARD, G. A. ( 1 9 7 5 ) . Condit iona l inference is not i ne f f i c i e nt . Scandinavian
J. of Statist. 3, 1 3 2 1 3 4 .
BARNARD, G. A. ( 1 9 8 0 ) . Pivot a l inference and t he Ba yesia n controversy ( wit h
Discussion) . I n Bayesian Statistics, J. M. Berna rdo, M. H. DeGroot,
D. V. Lindle y, and A.F.M. Smith ( e ds. ) . Unive r sit y Press, Va lencia .
BARNARD, G. A. ( 1 9 8 1 ) . A coherent view of st a t i st i c a l infe r e nce . Presented a t
t he Symposium on St a t i st i c a l I nference and Applica t ions, Universit y of
Wa t erloo, August, 1 9 8 1 .
BARNARD, G. A. and GODAMBE, V. P. ( 1 9 8 2 ) . Alla n Birnbaum, A memorial a r t i c l e .
Ann. Statist. 10, 1033 1039.
R E F E R E N C E S 1 4 5
B A R N A R D , G . A . , J E N K I N S , G . M . , a n d W I N S T E N , C . B . ( 1 9 6 2 ) . L i k e l i h o o d i n f e r e n c e
a n d t i m e s e r i e s . J . R o y . S t a t i s t . S o c . A 1 2 5 , 3 2 1 - 3 7 2 .
B A R N A R D , G. A . a n d S P R O T T , D . A . ( 1 9 8 3 ) . T h e g e n e r a l i z e d p r o b l e m o f t h e N i l e :
r o b u s t c o n f i d e n c e s e t s f o r p a r a m e t r i c f u n c t i o n s . A n n . S t a t i s t . 1 1 , 1 0 4 -
1 1 3 .
B A R N D O R F F - N I E L S E N , 0 . ( 1 9 7 1 ) . O n C o n d i t i o n a l S t a t i s t i c a l I n f e r e n c e .
M a t e m a t i s k I n s t i t u t e , A a r h u s U n i v e r s i t y .
B A R N D O R F F - N I E L S E N , 0 . ( 1 9 7 6 ) . P l a u s i b i l i t y i n f e r e n c e ( w i t h D i s c u s s i o n ) . J .
R o y . S t a t i s t . S o c . B 3 8 , 1 0 3 - 1 3 1 .
B A R N D O R F F - N I E L S E N , 0 . ( 1 9 7 8 ) . I n f o r m a t i o n a n d E x p o n e n t i a l F a m i l i e s i n S t a t i s t i -
c a l T h e o r y . W i l e y , N e w Y o r k .
B A R N D O R F F - N I E L S E N , 0 . ( 1 9 8 0 ) . C o n d i t i o n a l i t y r e s o l u t i o n s . B i o m e t r i k z 6 7 , 2 9 3 -
3 1 0 .
B A R N E T T , V . ( 1 9 8 2 ) . C o m p a r a t i v e S t a t i s t i c a l I n f e r e n c e ( 2 n d E d i t i o n ) . J o h n
W i l e y a n d S o n s , N e w Y o r k .
B A R T H O L O M E W , D . J. ( 1 9 6 7 ) . H y p o t h e s i s t e s t i n g w h e n t h e s a m p l e s i z e is t r e a t e d
a s a r a n d o m v a r i a b l e . J . R o y . S t a t i s t . S o c . B 2 9 , 5 3 - 8 2 .
B A R T L E T T , M . S . ( 1 9 3 6 ) . S t a t i s t i c a l i n f o r m a t i o n a n d p r o p e r t i e s o f s u f f i c i e n c y .
P r o c . R o y a l S o c . A 1 5 4 , 1 2 4 .
B A R T L E T T , M . S . ( 1 9 5 3 ) . A p p r o x i m a t e c o n f i d e n c e i n t e r v a l s , I , B i o m e t r i k a 4 0 ,
1 3 - 1 9 . I I , B i o m e t r i k a 4 0 , 3 0 6 - 3 1 7 .
B A S U , D . ( 1 9 6 4 ) . R e c o v e r y o f a n c i l l a r y i n f o r m a t i o n . In C o n t r i b u t i o n s t o
S t a t i s t i c s , C . R . R a o ( e d . ) , 7 - 2 0 . P e r g a m o n P r e s s , O x f o r d .
B A S U , D . ( 1 9 6 9 ) . R o l e o f t h e s u f f i c i e n c y a n d l i k e l i h o o d p r i n c i p l e s in s a m p l e
s u r v e y t h e o r y . S a n k h y a A 3 1 , 4 4 1 - 4 5 4 .
B A S U , D . ( 1 9 7 1 ) . A n e s s a y o n t h e l o g i c a l f o u n d a t i o n s o f s u r v e y s a m p l i n g , p a r t
o n e . In F o u n d a t i o n s o f S t a t i s t i c a l I n f e r e n c e , V . P. G o d a m b e , a n d D . A .
S p r o t t ( e d s . ) . H o l t , R i n e h a r t a n d W i n s t o n , T o r o n t o .
B A S U , D . ( 1 9 7 5 ) . S t a t i s t i c a l i n f o r m a t i o n a n d l i k e l i h o o d ( w i t h d i s c u s s i o n s ) .
S a n k h y a S e r . A 3 7 , 1 - 7 1 .
B A S U , D . ( 1 9 7 8 ) . O n t h e r e l e v a n c e o f r a n d o m i z a t i o n in d a t a a n a l y s i s ( w i t h
d i s c u s s i o n ) . In S u r v e y S a m p l i n g a n d M e a s u r e m e n t , N. K. Na m b o o d i r i ( e d . ) .
A c a d e m i c P r e s s , Ne w Y o r k .
B A S U , D. ( 1 9 8 0 ) . R a n d o m i z a t i o n a n a l y s i s o f e x p e r i m e n t a l d a t a : t h e F i s h e r
r a n d o m i z a t i o n t e s t . J . A m e r . S t a t i s t . A s s o c . 7 5 , 5 7 5 5 9 5 .
B E R G E R , J. ( 1 9 8 0 ) . S t a t i s t i c a l D e c i s i o n T h e o r y : F o u n d a t i o n s , C o n c e p t s , a n d
M e t h o d s . S p r i n g e r V e r l a g , Ne w Y o r k .
B E R G E R , J. ( 1 9 8 4 a ) . In d e f e n s e o f t h e l i k e l i h o o d p r i n c i p l e : a x i o m a t i c s a n d
c o h e r e n c y . In B a y e s i a n S t a t i s t i c s I I , J. M . B e r n a r d o , M . H . D e G r o o t ,
D . L i n d l e y , a n d A . Sm i t h ( e d s . ) .
B E R G E R , J . ( 1 9 8 4 b ) . B a y e s i a n s a l e s m a n s h i p . In B a y e s i a n I n f e r e n c e a n d D e c i s i o n
T e c h n i q u e s w i t h A p p l i c a t i o n s : E s s a y s i n H o n o r o f B r u n o d e F i n e t t i , P. K.
G o e l a n d A . Z e l l n e r ( e d s . ) . N o r t h H o l l a n d , A m s t e r d a m .
B E R G E R , J . ( 1 9 8 4 c ) . T h e f r e q u e n t i s t v i e w p o i n t a n d c o n d i t i o n i n g . T o a p p e a r i n
t h e P r o c e e d i n g s o f t h e B e r k e l e y C o n f e r e n c e i n H o n o r o f J . K i e f e r a n d J .
N e y m a n , L. L e C a m a n d R. O l s h e n ( e d s . ) . W a d s w o r t h , B e l m o n t C a l i f o r n i a .
B E R G E R , J . ( 1 9 8 4 d ) . A r e v i e w o f J. K i e f e r ' s w o r k o n c o n d i t i o n a l f r e q u e n t i s t
s t a t i s t i c s . T o a p p e a r in T h e C o l l e c t e d W o r k s o f J a c k K i e f e r ( L . B r o w n ,
I. O l k i n , J. S a c k s , H . W y n n , e d s . ) .
B E R G E R , J . ( 1 9 8 4 e ) . T h e r o b u s t B a y e s i a n v i e w p o i n t ( w i t h d i s c u s s i o n ) . In
R o b u s t n e s s i n B a y e s i a n S t a t i s t i c s * J. K a d a n e ( e d . ) , 6 3 1 4 4 .
N o r t h H o l l a n d , A m s t e r d a m .
B E R G E R , J . a n d B E R L I N E R , L . M . ( 1 9 8 3 ) . Ro b u s t B a y e s a n d e m p i r i c a l B a y e s
a n a l y s i s w i t h c o n t a m i n a t e d p r i o r s . T e c h n i c a l Re p o r t # 8 3 3 5 , S t a t i s t i c s
D e p t . , P u r d u e U n i v . , W . L a f a y e t t e .
B E R N A R D O , J . M . ( 1 9 7 9 ) . R e f e r e n c e p o s t e r i o r d i s t r i b u t i o n s f o r B a y e s i a n
i n f e r e n c e ( w i t h d i s c u s s i o n ) , j . B o y . S t a t i s t . S o c . B 4 1 , 1 1 3 1 4 7 .
B I R N B A U M , A . ( 1 9 6 1 a ) . O n t h e f o u n d a t i o n s o f s t a t i s t i c a l i n f e r e n c e : b i n a r y
e x p e r i m e n t s . A n n . M a t h . S t a t i s t . 3 2 , 4 1 4 4 3 5 .
B I R N B A U M , A . ( 1 9 6 1 b ) . C o n f i d e n c e c u r v e s : a n o m n i b u s t e c h n i q u e f o r e s t i m a t i o n
a n d t e s t i n g s t a t i s t i c a l h y p o t h e s e s . J . A m e r . S t a t i s t . A s s o c . 5 6 , 2 4 6 - 2 4 9 .
B I R N B A U M , A . ( 1 9 6 2 a ) . O n t h e f o u n d a t i o n s o f s t a t i s t i c a l i n f e r e n c e ( w i t h
d i s c u s s i o n ) . J . A m e r . S t a t i s t . A s s o c . 5 7 , 2 6 9 - 3 0 6 .
B I R N B A U M , A . ( 1 9 6 2 b ) . I n t r i n s i c c o n f i d e n c e m e t h o d s . B u l l e t i n o f t h e I n t .
S t a t i s t . I n s t . 3 9 , 3 7 5 - 3 8 3 .
B I R N B A U M , A . ( 1 9 6 8 ) . L i k e l i h o o d . In I n t e r n a t i o n a l E n c y c l o p e d i a o f t h e S o c i a l
S c i e n c e s , V o l . 9 .
B I R N B A U M , A . ( 1 9 6 9 ) . C o n c e p t s o f s t a t i s t i c a l e v i d e n c e . In P h i l o s o p h y ,
S c i e n c e , a n d M e t h o d : E s s a y s i n H o n o r o f E r n e s t N a g e l , S . M o r g e n b e s s e r ,
P . S u p p e s , a n d M . W h i t e ( e d s . ) . S t . M a r t i n ' s P r e s s , N e w Y o r k .
B I R N B A U M , A . ( 1 9 7 0 a ) . S t a t i s t i c a l m e t h o d s in s c i e n t i f i c i n f e r e n c e . N a t u r e 2 2 5 ,
1 0 3 3 .
B I R N B A U M , A . ( 1 9 7 0 b ) . O n D u r b i n ' s m o d i f i e d p r i n c i p l e o f c o n d i t i o n a l i t y . J .
A m e r . S t a t i s t . A s s o c . 6 5 , 4 0 2 - 4 0 3 .
B I R N B A U M , A . ( 1 9 7 2 ) . M o r e o n c o n c e p t s o f s t a t i s t i c a l e v i d e n c e . J . A m e r .
S t a t i s t . A s s o c . 6 7 , 8 5 8 - 8 6 1 .
B I R N B A U M , A . ( 1 9 7 7 ) . T h e N e y m a n - P e a r s o n t h e o r y a s d e c i s i o n t h e o r y a n d a s
i n f e r e n c e t h e o r y : w i t h a c r i t i c i s m o f t h e L i n d l e y - S a v a g e a r g u m e n t f o r
B a y e s i a n t h e o r y . S y n t h e s e 3 6 , 1 9 - 4 9 .
B O N D A R , J. V . ( 1 9 7 7 ) . O n a c o n d i t i o n a l c o n f i d e n c e p r i n c i p l e . A n n . S t a t i s t . 5 ,
8 8 1 - 8 9 1 .
B O N D A R , J. V . a n d M I L N E S , P. ( 1 9 8 1 ) . A m e n a b i l i t y : A s u r v e y f o r s t a t i s t i c a l
a p p l i c a t i o n s o f H u n t - S t e i n a n d r e l a t e d c o n d i t i o n s o n g r o u p s . Z e i t s c h r .
W a h r s c h . V e r w . G e t . 5 7 , 1 0 3 - 1 2 8 .
B O X , G . E . P . ( 1 9 8 0 ) . S a m p l i n g a n d B a y e s
1
i n f e r e n c e in s c i e n t i f i c m o d e l l i n g a n d
r o b u s t n e s s ( w i t h d i s c u s s i o n ) . J . R o y . S t a t i s t . S o c . B 1 4 3 , 3 8 3 - 4 3 0 .
B O X , G . E . P . a n d T I A O , G . C . ( 1 9 7 3 ) . B a y e s i a n I n f e r e n c e i n S t a t i s t i c a l A n a l y s i s .
A d d i s o n - W e s l e y , R e a d i n g .
B R O W N , L . D . ( 1 9 6 7 ) . T h e c o n d i t i o n a l l e v e l o f S t u d e n t ' s t - T e s t . A n n . M a t h .
S t a t i s t . 3 8 , 1 0 6 8 - 1 0 7 1 .
BROWN, L. D. (1978). A cont r i but i on to Ki ef er ' s theory of condi t i onal
confidence procedures. Ann. Statist. 6, 59- 71.
BROWNIE, C. and KIEFER, J . (1977). The ideas of condi t i onal confidence i n the
si mpl est set t i ng. Comrmm. Statist. A6(8), 691-751.
BUEHLER, R. J. (1959). Some val i di t y c r i t er i a f or st at i st i cal i nference.
Ann. Math. Statist. 30, 845-863.
BUEHLER, R. J. (1971). Measuring i nformati on and uncert ai nt y. In Foundations
of Statistical Inference, V. P. Godambe and D. A. Spr ot t ( e d s . ) . Hol t ,
Rinehart and Winston, Toronto.
BUEHLER, R. J. (1976). Coherent preferences. Ann. Statist. 4, 1051-1064.
BUEHLER, R. J. (1982). Some anci l l ar y st at i st i cs and t hei r properti es (wi th
di scussi on). J. Amer. Statist. Assoc. 77, 581-594.
BUEHLER, R. J . and FEDDERSON, A. P. (1963). Note on a condi t i onal property of
Student' s t . Ann. Math. Statist. 34, 1098-1100.
BUNKE, H. (1975). St at i st i cal i nference: Fi duci al and st r uct ur al versus
l i kel i hood. Math. Operations forsch. U. Statist. 6, 667-676.
CASELLA, G. and HWANG, J. T. (1982). Zero-radius confidence procedures. Tech-
ni cal Report, Cornell Uni ver si t y.
CASSEL, C. M., SARNDAL, C. E., and WRETMAN, J. H. (1977). Foundations of
Inference in Survey Sampling. Wiley, New York.
CORNFIELD, J. (1966). Sequential t r i a l s , sequential anal ysi s, and the l i k e l i -
hood pr i nci pl e. The American Statist. 20, No. 2, 18-23.
CORNFIELD, J. (1969). The Bayesian outlook and i t s appl i cat i on (wi t h
di scussi on). Biometrics 26, 617-657.
COX, D. R. (1958). Some problems connected wi t h s t at i s t i c al i nference. Ann.
Math. Statist. 29, 357-372.
COX, D. R. (1971). The choice between anci l l ar y s t at i s t i c s . J. Roy. Statist.
Soc. B 33, 251-255.
COX, D. R. (1975). Part i al likelihood. Biometrika 62, 269-276.
REFERENCES 149
COX, D. R. (1977). The rol e of si gni fi cance t es t s . Saand. J. Statist. 4, 49-
70.
COX, D. R. (1978). Foundations of st at i st i cal inference: the case for
eclecticism. Austral. J. Statist. 20, 43-59.
COX, D. R. (1980). Local anc i l l ar i t y. Biometriha 67, 279-286.
COX, D. R. and HINKLEY, D. V. (1974). Theoretical Statistics. Chapman and
Hal l , London.
DAWID, A. P. (1975). On the concepts of sufficiency and anci l l ari t y in the
prescence of nuisance parameters. J. Roy. Statist. Soc. B 37, 248-258.
DAWID, A. P. (1977). Conformity of inference patterns. In Recent
Developments in Statistics, J. R. Barra, et . al . ( eds . ) . North-Holland,
Amsterdam.
DAWID, A. P. (1980). A Bayesian look at nuisance parameters. In Bayesian
Statistics, J. M. Bernardo, M. H. DeGroot, D. V. Lindley, and A.F.M. Smith
( eds . ) . Uni versi ty Press, Valencia.
DAWID, A. P. (1981). St at i s t i c al i nference. In Encyclopedia of Statistical
Sciences, S. Kotz and N. L. Johnson (eds. ). Wiley, New York.
DAWID, A. P. and DICKEY, J. M. (1977). Likelihood and Bayesian inference from
selectively reported data. J. Amer. Statist. Assoc. 72, 845-850.
DAWID, A. P. and STONE, M. (1982). The functional-model basis of fiducial
inference (with discussion). Ann. Statist. 10, 1054-1074.
DE FINETTI, B. (1962). Does i t make sense to speak of 'Good probability
apprai sers' ? In The Scientist Speculates, I. J. Good ( ed. ) . Basic Books,
New York.
DE FINETTI, B. (1972). Probability, Induction, and Statistics. Wiley, New
York.
DE FINETTI, B. (1974). Theory of Probability, Volumes 1 and 2. Wiley, New
York.
D E G R O O T , M . H . ( 1 9 7 3 ) . D o i n g w h a t c o m e s n a t u r a l l y : i n t e r p r e t i n g a t a i l a r e a
a s a p o s t e r i o r p r o b a b i l i t y o r a s a l i k e l i h o o d r a t i o . J . A m e r . S t a t i s t .
A s s o . 6 8 , 9 6 6 9 6 9 .
D E M P S T E R , A . P . ( 1 9 7 4 a ) . Re m a r k s o n i n f e r e n c e . In t h e P r o c e e d i n g s o f t h e
C o n f e r e n c e o n F o u n d a t i o n a l Q u e s t i o n s i n S t a t i s t i c a l I n f e r e n c e , 0 .
B a r n d o r f f N i e l s e n , P. B l a e s i l d , a n d G. S c h o u ( e d s . ) . D e p a r t m e n t o f
T h e o r e t i c a l S t a t i s t i c s , U n i v e r s i t y o f A a r h u s .
D E M P S T E R , A . P. ( 1 9 7 4 b ) . T h e d i r e c t u s e o f l i k e l i h o o d f o r s i g n i f i c a n c e t e s t
i n g . In t h e P r o c e e d i n g s o f t h e C o n f e r e n c e o n F o u n d a t i o n a l Q u e s t i o n s i n
S t a t i s t i c a l I n f e r e n c e , 0 . B a r n d o r f f N i e l s e n , P. B l a e s i l d , a n d G . S c h o u
( e d s . ) . D e p a r t m e n t o f T h e o r e t i c a l S t a t i s t i c s , U n i v e r s i t y o f A a r h u s .
D E M P S T E R , A . P . ( 1 9 7 5 ) . A s u b j e c t i v i s t l o o k a t r o b u s t n e s s . B u l l , o f t h e
I n t e r n a t i o n a l S t a t i s t . I n s t . 4 6 , 3 4 9 3 7 4 .
D I C K E Y , J. M . ( 1 9 7 6 ) . A p p r o x i m a t e p o s t e r i o r d i s t r i b u t i o n s . J . A m e r . S t a t i s t .
A s s o c . 7 1 , 6 8 0 6 8 9 .
D I C K E Y , J. M . ( 1 9 7 7 ) . Is t h e t a i l a r e a u s e f u l a s a n a p p r o x i m a t e B a y e s f a c t o r ?
J . A m e r . S t a t i s t . A s s o c . 7 2 , 1 3 8 1 4 2 .
D U R B I N , J . ( 1 9 7 0 ) . O n B i r n b a u m ' s t h e o r e m o n t h e r e l a t i o n b e t w e e n s u f f i c i e n c y ,
c o n d i t i o n a l i t y , a n d l i k e l i h o o d . J . A m e r . S t a t i s t . A s s o c . 6 6 , 3 9 5 3 9 8 .
E D W A R D S , A . W . F . ( 1 9 7 2 ) . L i k e l i h o o d . C . U . P . , C a m b r i d g e .
E D W A R D S , A . W . F . ( 1 9 7 4 ) . T h e h i s t o r y o f l i k e l i h o o d . I n t . S t a t i s t . R e v . 4 2 ,
9 1 5 .
E D W A R D S , W . , L I N D M A N , H . , a n d S A V A G E , L . J. ( 1 9 6 3 ) . B a y e s i a n s t a t i s t i c a l
i n f e r e n c e f o r p s y c h o l o g i c a l r e s e a r c h . P s y c h o l o g i c a l R e v i e w 7 0 , 1 9 3 2 4 2 .
E F R O N , B . a n d H I N K L E Y , D . V . ( 1 9 7 8 ) . A s s e s s i n g t h e a c c u r a c y o f t h e m a x i m u m
l i k e l i h o o d e s t i m a t o r : o b s e r v e d v e r s u s e x p e c t e d F i s h e r i n f o r m a t i o n .
B i o m e t r i k a 6 6 , 4 5 7 4 8 2 .
E R I C S O N , W . A . ( 1 9 6 9 ) . S u b j e c t i v e B a y e s i a n m o d e l s in s a m p l i n g f i n i t e
p o p u l a t i o n s . J . R o y . S t a t i s t . S o c . B l , 1 9 5 2 3 3 .
REFERENCES 151
F ISHER, R. A. ( 1921) . On t he ' Probabl e E rror
1
of a coef f i ci ent of cor r el at i on
deduced from a small sample. Metron. I, part 4, 3- 32.
FISHER, R. A. ( 1925) . Theory of s t a t i s t i c a l est i mat i on. Proc. Cambridge Phil.
Soc. 22, 700- 725.
FISHER, R. A. ( 1934) . Two new propert i es of mathematical l i kel i hood. Proa.
Royal Soc. A 144, 285-307.
FISHER, R. A. (1956a). Statistical Methods and Scientific Inference. Oliver
and Boyd. Edinburgh.
FISHER, R. A. (1956b). On a test of significance in Pearson's Biometrika
Tables (no. 11). J. Roy. Statist. Soc. B 18, 56-60.
FISHER, R. A. (1960). The Design of Experiments (7th ed.J. Oliver and Boyd,
Edinburgh.
FRASER, D.A.S. (1963). Cn the sufficiency and likelihood principles. J. Amer.
Statist. Assoc. 58, 641-647.
FRASER, D.A.S. (1968). The Structure of Inference. Wiley, New York.
FRASER, D.A.S. (1972). Bayes, likelihood, or structural . Ann. Math. Statist.
43, 777-790.
FRASER, D.A.S. (1973). Inference and redundant parameters. In Multivariate
Analysis-Ill, P. R. Krishnaiah ( ed. ) . Academic Press, New York.
FRASER, D.A.S. (1976). Necessary analysis and adaptive inference (with
discussion). J. Amer. Statist. Assoc. 71, 99-113.
FRASER, D.A.S. (1977). Confidence, posterior probability, and the Buehler
example. Ann. Statist. 5, 892-898.
FRASER, D. A. S . ( 1979) . Inference and Linear Models. McGraw- Hi l l , New York.
FRASER, D. A. S . and MACKAY, J . ( 1976) . On the eq ui val ence of standard i nference
procedures. In Foundations of Probability Theory, Statistical Inference,
and Statistical Theories of Science, Vol. II, W. L. Harper and C. A.
Hooker ( e ds . ) . R ei del , Dordrecht.
FREEDMAN, D. A. and PURVES, R. A. ( 1969) . Bayes methods f or booki es. Ann.
Math. Statist. 40, 1177-1186.
G E I S S E R , S . ( 1 9 7 1 ) . T h e i n f e r e n t i a l u s e o f p r e d i c t i v e d i s t r i b u t i o n s . In
F o u n d a t i o n s o f S t a t i s t i c a l I n f e r e n c e , V . P. G o d a m b e a n d D . A . S p r o t t
( e d s . ) . H o l t , R i n e h a r t , a n d W i n s t o n , T o r o n t o .
G I E R E , R. N . ( 1 9 7 7 ) . A l l a n B i r n b a u m ' s c o n c e p t i o n o f s t a t i s t i c a l e v i d e n c e .
S y n t h e s e 3 6 , 5 - 1 3 .
G O D A M B E , V . P. ( 1 9 6 6 ) . A n e w a p p r o a c h t o s a m p l i n g f r o m f i n i t e p o p u l a t i o n s , I.
J . R o y . S t a t i s t . S o c . B 2 8 , 3 1 0 - 3 1 9 .
G O D A M B E , V . P . ( 1 9 7 9 ) . O n B i r n b a u m ' s m a t h e m a t i c a l l y e q u i v a l e n t e x p e r i m e n t s .
J . R o y . S t a t i s t . S o c . B 4 1 , 1 0 7 - 1 1 0 .
G O D A M B E , V . P. ( 1 9 8 2 a ) . L i k e l i h o o d p r i n c i p l e a n d r a n d o m i z a t i o n . In S t a t i s t i c s
a n d P r o b a b i l i t y : E s s a y s i n H o n o r o f C . R . R a o , G. K a l l i a n p u r , P. R.
K r i s h n a i a h , a n d J . K . G h o s h ( e d s . ) . N o r t h - H o l l a n d , A m s t e r d a m .
G O D A M B E , V . P. ( 1 9 8 2 b ) . E s t i m a t i o n in s u r v e y s a m p l i n g : r o b u s t n e s s a n d
o p t i m a l i t y . J . A m e r . S t a t i s t . A s s o c . 7 7 , 3 9 3 - 4 0 6 .
G O D A M B E , V . P . a n d T H O M P S O N , M . E . ( 1 9 7 6 ) . P h i l o s o p h y o f s u r v e y - s a m p l i n g
p r a c t i c e . In F o u n d a t i o n s o f P r o b a b i l i t y T h e o r y , S t a t i s t i c a l I n f e r e n c e ,
a n d S t a t i s t i c a l T h e o r i e s o f S c i e n c e , V o l . I I , W . L . H a r p e r a n d C . A .
H o o k e r ( e d s . ) . R e i d e l , D o r d r e c h t .
G O D A M B E , V . P. a n d T H O M P S O N , M . E . ( 1 9 7 7 ) . R o b u s t n e a r o p t i m a l e s t i m a t i o n in
s u r v e y p r a c t i c e . B u l l e t i n o f t h e I n t e r n a t . S t a t i s t . I n s t . 4 7 , 1 2 7 - 1 7 0 .
G O O D , I. J. ( 1 9 5 0 ) . P r o b a b i l i t y a n d t h e W e i g h i n g o f E v i d e n c e . G r i f f i n , L o n d o n .
G O O D , I . J. ( 1 9 6 5 ) . T h e E s t i m a t i o n o f P r o b a b i l i t i e s . M . I. T . P r e s s ,
C a m b r i d g e .
G O O D , I . J . ( 1 9 7 6 ) . T h e B a y e s i a n i n f l u e n c e , o r h o w t o s w e e p s u b j e c t i v i s m u n d e r
t h e c a r p e t . In F o u n d a t i o n s o f P r o b a b i l i t y T h e o r y , S t a t i s t i c a l I n f e r e n c e ,
a n d S t a t i s t i c a l T h e o r i e s o f S c i e n c e , V o l . I I , W . L . H a r p e r a n d C . A .
H o o k e r ( e d s . ) . R e i d e l , D o r d r e c h t .
G O O D , I . J. ( 1 9 8 1 ) . S o m e l o g i c a n d h i s t o r y o f h y p o t h e s i s t e s t i n g . In
P h i l o s o p h y i n E c o n o m i c s , J . C . P i t t ( e d . ) . R e i d e l , D o r d r e c h t .
G R A M B S C H , P . ( 1 9 8 0 ) . L i k e l i h o o d i n f e r e n c e . P h . D . D i s s e r t a t i o n , U n i v e r s i t y o f
M i n n e s o t a .
H A C K I N G , I. ( 1 9 6 5 ) . L o g i c o f S t a t i s t i c a l I n f e r e n c e . C a m b r i d g e U n i v e r s i t y
P r e s s , C a m b r i d g e .
H A J E K , J. ( 1 9 6 7 ) . O n b a s i c c o n c e p t s o f s t a t i s t i c s . In P r o c e e d i n g s o f t h e
F i f t h B e r k e l e y S y m p o s i u m o n M a t h e m a t i c a l S t a t i s t i c s a n d P r o b a b i l i t y ,
L e C a m a n d J. N e y m a n ( e d s . ) . U n i v e r s i t y o f C a l i f o r n i a P r e s s , B e r k e l e y .
H A J E K , J. ( 1 9 7 1 ) . L i m i t i n g p r o p e r t i e s o f l i k e l i h o o d s a n d i n f e r e n c e . In
F o u n d a t i o n s o f S t a t i s t i c a l I n f e r e n c e , V . P. G o d a m b e a n d D . A . S p r o t t
( e d s . ) . H o l t , R i n e h a r t , a n d W i n s t o n , T o r o n t o .
H E A T H , D . a n d S U D D E R T H , W . ( 1 9 7 8 ) . O n f i n i t e l y a d d i t i v e p r i o r s , c o h e r e n c e ,
a n d e x t e n d e d a d m i s s i b i l i t y . A n n . S t a t i s t . 6 , 3 3 3 - 3 4 5 .
H I L L , B . ( 1 9 7 0 ) . S o m e c o n t r a s t s b e t w e e n B a y e s i a n a n d c l a s s i c a l i n f e r e n c e i n
t h e a n a l y s i s o f v a r i a n c e a n d in t h e t e s t i n g o f m o d e l s . In B a y e s i a n
S t a t i s t i c s , D . L . M e y e r a n d R . 0 . C o l l i e r , J r . ( e d s . ) . P e a c o c k P u b l i s h e r s ,
I t a s c a , I l l i n o i s .
H I L L , B . ( 1 9 7 3 ) . R e v i e w o f " L i k e l i h o o d " b y A . W . F . E d w a r d s , J . A m e r . S t a t i s t .
A s s o c . 6 8 , 4 8 7 - 4 8 8 .
H I L L , B . ( 1 9 7 4 a ) . R e v i e w o f " B a y e s i a n I n f e r e n c e in S t a t i s t i c a l A n a l y s i s " by
G . E . P . B o x a n d G . T i a o . T e c h n o m e t r i c s 1 6 , 4 7 8 - 4 7 9 .
H I L L , B . ( 1 9 7 4 b ) . O n c o h e r e n c e , i n a d m i s s i b i l i t y , a n d i n f e r e n c e a b o u t m a n y
p a r a m e t e r s in t h e t h e o r y o f l e a s t s q u a r e s . In S t u d i e s i n B a y e s i a n
E c o n o m e t r i c s a n d S t a t i s t i c s , S . F i e n b e r g a n d A . Z e l l n e r ( e d s . ) . N o r t h -
H o l l a n d , A m s t e r d a m .
H I L L , B . ( 1 9 7 5 ) . A b b e r a n t b e h a v i o r o f t h e l i k e l i h o o d f u n c t i o n in d i s c r e t e
c a s e s . J . A m e r . S t a t i s t . A s s o c . 7 0 , 7 1 7 - 7 1 9 .
H I L L , B . ( 1 9 8 1 ) . O n s o m e s t a t i s t i c a l p a r a d o x e s a n d n o n - c o n g l o m e r a b i l i t y . In
B a y e s i a n S t a t i s t i c s , J. M . B e r n a r d o , M . H . D e G r o o t , D . V . L i n d l e y , a n d
A . F . M . S m i t h ( e d s . ) . U n i v e r s i t y P r e s s , V a l e n c i a .
H I N D E , J. a n d A I T K I N , M . ( 1 9 8 4 ) . N u i s a n c e p a r a m e t e r s , c a n o n i c a l l i k e l i h o o d s ,
a n d d i r e c t l i k e l i h o o d i n f e r e n c e . T e c h n i c a l R e p o r t , C e n t r e f o r A p p l i e d
S t a t i s t i c s , U n i v e r s i t y o f L a n c a s t e r .
H I N K L E Y , D . V . ( 1 9 7 8 ) . L i k e l i h o o d i n f e r e n c e a b o u t l o c a t i o n a n d s c a l e p a r a m e -
t e r s . B i o m e t r i k a 6 5 , 2 5 3 - 2 6 2 .
H I N K L E Y , D . V . ( 1 9 7 9 ) . P r e d i c t i v e l i k e l i h o o d . A n n . S t a t i s t . 7 , 7 1 8 - 7 2 8 .
H I N K L E Y , D . V . ( 1 9 8 0 a ) . F i s h e r ' s d e v e l o p m e n t o f c o n d i t i o n a l i n f e r e n c e . In
R . A . F i s h e r : A n A p p r e c i a t i o n , S . E . F i e n b e r g a n d D . V . H i n k l e y ( e d s . ) .
S p r i n g e r - V e r l a g , N e w Y o r k .
H I N K L E Y , D . V . ( 1 9 8 0 b ) . L i k e l i h o o d as a p p r o x i m a t e p i v o t a l d i s t r i b u t i o n .
B i o m e t r i k a 6 7 , 2 8 7 - 2 9 2 .
H I N K L E Y , D . V . ( 1 9 8 3 ) . C a n f r e q u e n t i s t i n f e r e n c e s b e v e r y w r o n g ? A c o n d i t i o n a l
' y e s ' . In S c i e n t i f i c I n f e r e n c e , D a t a A n a l y s i s , a n d R o b u s t n e s s , G . E . P .
B o x , T . L e o n a r d , a n d C . F . W u ( e d s . ) . A c a d e m i c P r e s s , N e w Y o r k .
J A M E S , W . a n d S T E I N , C . ( 1 9 6 1 ) . E s t i m a t i o n w i t h q u a d r a t i c l o s s . In F o u r t h
B e r k e l e y S y m p o s i u m M a t h . S t a t i s t , a n d P r o b . U n i v e r s i t y o f C a l i f o r n i a
P r e s s , B e r k e l e y .
J A Y N E S , E . T . ( 1 9 8 1 ) . T h e i n t u i t i v e i n a d e q u a c y o f c l a s s i c a l s t a t i s t i c s .
P r e s e n t e d at t h e I n t e r n a t i o n a l C o n v e n t i o n o n F u n d a m e n t a l s o f P r o b a b i l i t y
a n d S t a t i s t i c s , L u i n o , I t a l y .
J A Y N E S , E . T . ( 1 9 8 2 ) . P a p e r s o n P r o b a b i l i t y , S t a t i s t i c s , a n d S t a t i s t i c a l
P h y s i c s , a r e p r i n t c o l l e c t i o n . D . R e i d e l , D o r d r e c h t .
J E F F R E Y S , H . ( 1 9 6 1 ) . T h e o r y o f P r o b a b i l i t y ( Z r d e d n . ) . C l a r e n d o n P r e s s ,
O x f o r d .
J E F F R E Y S , H . ( 1 9 7 3 ) . S c i e n t i f i c I n f e r e n c e ( Z r d e d n . ) . C . U . P . , C a m b r i d g e .
J O S H I , V . M . ( 1 9 7 6 ) . A n o t e o n B i r n b a u m ' s t h e o r y o f t h e l i k e l i h o o d p r i n c i p l e .
J . A m e r . S t a t i s t . A s s o c . 7 1 , 3 4 5 - 3 4 6 .
K A L B F L E I S C H , J. D . ( 1 9 7 1 ) . L i k e l i h o o d m e t h o d s o f p r e d i c t i o n . In F o u n d a t i o n s
o f S t a t i s t i c a l I n f e r e n c e , V . P. G o d a m b e a n d D . A . S p r o t t ( e d s . ) . H o l t ,
R i n e h a r t , a n d W i n s t o n , T o r o n t o .
K A L B F L E I S C H , J. D . ( 1 9 7 4 ) . S u f f i c i e n c y a n d c o n d i t i o n a l i t y . In t h e P r o c e e d i n g s
o f t h e C o n f e r e n c e o n F o i m d a t i o n a l Q u e s t i o n s i n S t a t i s t i c a l I n f e r e n c e ,
0 . B a r n d o r f f N i e l s e n , P. B l a e s i l d , a n d G . S c h o u ( e d s . ) . D e p a r t m e n t o f
T h e o r e t i c a l S t a t i s t i c s , U n i v e r s i t y o f A a r h u s .
K A L B F L E I S C H , J. D . ( 1 9 7 5 ) . S u f f i c i e n c y a n d c o n d i t i o n a l i t y . B i o m e t r i k a 6 2 ,
2 5 1 2 6 8 .
K A L B F L E I S C H , J . D . ( 1 9 7 8 ) . L i k e l i h o o d m e t h o d s a n d n o n p a r a m e t r i c t e s t s . J .
A m e r . S t a t i s t . A s s o c . 7 3 , 1 6 7 1 7 0 .
K A L B F L E I S C H , J. D . a n d S P R O T T , D . A . ( 1 9 6 9 ) . A p p l i c a t i o n s o f l i k e l i h o o d a n d
f i d u c i a l p r o b a b i l i t y t o s a m p l i n g f i n i t e p o p u l a t i o n s . In N e w D e v e l o p m e n t s
i n S u r v e y S a m p l i n g . W i l e y , Ne w Y o r k .
K A L B F L E I S C H , J . D . a n d S P R O T T , D . A . ( 1 9 7 0 ) . A p p l i c a t i o n o f l i k e l i h o o d m e t h o d s
t o m o d e l s i n v o l v i n g l a r g e n u m b e r s o f p a r a m e t e r s ( w i t h d i s c u s s i o n ) . J .
R o y . S t a t i s t . S o . B 3 2 , 1 7 5 2 0 8 .
K E M P T H O R N E , 0 . a n d F O L K E S , J. L . ( 1 9 7 1 ) . P r o b a b i l i t y , S t a t i s t i c s , a n d D a t a
A n a l y s i s . I o w a S t a t e U n i v e r s i t y P r e s s , A m e s .
K I E F E R , J. ( 1 9 7 5 ) . Co n d i t i o n a l c o n f i d e n c e a p p r o a c h in m u l t i d e c i s i o n p r o b l e m s .
In P r o c e e d i n g s o f t h e F o u r t h D a y t o n M u l t i v a r i a t e C o n f e r e n c e , P . R.
K r i s h n a i a h ( e d . ) . 1 4 3 1 5 8 . N o r t h H o l l a n d , A m s t e r d a m .
K I E F E R , J. ( 1 9 7 6 ) . A d m i s s i b i l i t y o f c o n d i t i o n a l c o n f i d e n c e p r o c e d u r e s . A n n .
M a t h . S t a t i s t . 4 , 8 3 6 8 6 5 .
K I E F E R , J. ( 1 9 7 7 a ) . Co n d i t i o n a l c o n f i d e n c e s t a t e m e n t s a n d c o n f i d e n c e e s t i m a
t o r s ( w i t h d i s c u s s i o n ) . J . A m e r . S t a t i s t . A s s o c . 7 2 , 7 8 9 8 2 7 .
K I E F E R , J. ( 1 9 7 7 b ) . T h e f o u n d a t i o n s o f s t a t i s t i c s a r e t h e r e a n y ? S y n t h e s e
3 6 , 1 6 1 1 7 6 .
K I E F E R , J. ( 1 9 8 0 ) . Co n d i t i o n a l i n f e r e n c e . In t h e E n c y c l o p e d i a o f S t a t i s t i c s ,
S. K o t z a n d N. J o h n s o n ( e d s . ) . W i l e y , Ne w Y o r k .
L A N E , D . A . a n d S U D D E R T H , W . D . ( 1 9 8 3 ) . C o h e r e n t a n d c o n t i n u o u s i n f e r e n c e .
A n n . S t a t i s t . 1 1 , 1 1 4 1 2 0 .
LAURITZEN, S. L. ( 19 7 4 ) . Sufficiency, prediction, and extreme models. Scand.
J. Statist. l
y
128134 .
L E C A M , L. ( 19 7 7 ) . A note on metastatistics or 'an essay toward stating a
problem in the doctrine of c h a n c e s
1
. Synthese 36
1 3 3 1 6 0 .
L E V I , I. ( 19 80 ) . The Enterprise of Knowledge. MIT Press, Cambridge.
LINDLEY, D. V. ( 19 58) . A survey of the foundations of statistics. Appl.
Statist. 7
y
18619 8.
LINDLEY, D. V. ( 19 7 1) . Bayesian Statistics Review. S.I.A.M., Philadelphia.
LINDLEY, D. V. ( 19 82) . Scoring rules and the inevitability of probability.
I nt. Statist. Rev. SO, 1 2 6 .
LINDLEY, D. V. and NOVICK, M. ( 1 9 8 1 ) . The role of exchangeability in inference.
Ann. Statist. 9, 4 558.
LINDLEY, D. V. and PHILLIPS, L. D. ( 1 9 7 6 ) . Inference for a Bernoulli process
(a Bayesian v i e w ) . American Statist. 30, 112119 .
MAULDON, J. G. ( 19 55) . Pivotal quantities for W i s h a r f s and related distribu
tions and a paradox in Fiducial theory. J. Roy. Statist. Soc B 17,
7 9 85.
MORRISON, D. F. and HENKEL, R. E. ( 1 9 7 0 ) . The Significance Test Controversy.
Aldine, Chicago.
NEYMAN, J. ( 1 9 5 7 ) , 'Inductive behavior
1
as a basic concept of philosophy of
science. Rev. I ntl. Statist. I nst. 25, 7 22.
NEYMAN, J. ( 19 67 ) . A Selection of Early Statistical Papers of J. Neyman.
University of California Press, Berkeley.
NEYMAN, J. ( 19 7 7 ) . Frequentist probability and frequentist statistics.
Synthese 36, 9 7 1 3 1 .
OLSHEN, R. A. ( 19 7 3) . The conditional level of the Ftest. J. Amer. Statist.
Assoc. 68, 69 269 8.
PEDERSEN, J. G. ( 19 7 8) . Fiducial inference. I nt. Statist. Review 46, 14 7 17 0 .
PEISAKOFF, M. ( 19 50 ) . Transformation Parameters. Ph.D. Thesis, Princeton
University.
P I C C I N A T O , L . ( 1 9 8 1 ) . O n t h e o r d e r i n g s o f d e c i s i o n f u n c t i o n s . S y m p o s i a
M a t h e m a t i o a X X V . A c a d e m i c P r e s s , L o n d o n .
P I E R C E , D . A . ( 1 9 7 3 ) . O n s o m e d i f f i c u l t i e s w i t h a f r e q u e n c y t h e o r y o f
i n f e r e n c e . A n n . S t a t i s t . 1 , 2 4 1 - 2 5 0 .
P L A C K E T T , R. L . ( 1 9 6 6 ) . C u r r e n t t r e n d s in s t a t i s t i c a l i n f e r e n c e . J . R o y .
S t a t i s t . S o c . A 1 2 9 , 2 4 9 - 2 6 7 .
P L A N T E , A . ( 1 9 7 1 ) . C o u n t e r - e x a m p l e s a n d l i k e l i h o o d . In F o u n d a t i o n s o f
S t a t i s t i c a l I n f e r e n c e , V . P. G o d a m b e a n d D . A . S p r o t t ( e d s . ) . H o l t ,
R inehart, and Winston, Toronto.
PRATT, J. W. (1961). Review of Lehmann's Testing S tati sti cal Hypotheses.
J. Amer. Statist. Assoc. 56, 163-166.
PRATT, J. W. (1965). Bayesian interpretation of standard inference statements
(with discussion). J. Roy. Statist. Soc. B 27, 169-203.
PRATT, J. W. (1976). A discussion of the question : for what use are tests
of hypotheses and tests of significance. Commun. Statist.-Theor. Meth. A3,
779-787.
PRATT, J. W. (1977). 'Decisions' as st at i st i cal evidence and Birnbaum's
'confidence concept
1
. Synthese 36, 59-69.
RAIFFA, H. and SCHLAIFER, R. (1961). Applied Statistical Decision Theory.
Graduate School of Business Administration, Harvard University.
RAO, C. R. (1971). Some aspects of st at i st i cal inference in problems of
sampling from finite populations. In Foundations of Statistical Inference,
V. P. Godambe and D. A. S prott (eds. ). Hol t, R inehart, and Winston,
Toronto.
ROBINSON, G. K. (1975). Some counterexamples to the theory of confidence
intervals. Biometrika 62, 155-161.
ROBINSON, G. K. (1976). Properties of Student's t and of the Behrens-Fisher
solution to the two means problem. Ann. Statist. 5, 963-971.
ROBINSON, G. K. (1979a). Conditional properties of st at i st i cal procedures.
Ann. Statist. 7, 742-755.
R O B I N S O N , G. K . ( 1 9 7 9 b ) . C o n d i t i o n a l p r o p e r t i e s o f s t a t i s t i c a l p r o c e d u r e s f o r
l o c a t i o n a n d s c a l e p a r a m e t e r s . A n n . S t a t i s t . 7 , 7 5 6 - 7 7 1 .
R O S E N K R A N T Z , R. D . ( 1 9 7 7 ) . I n f e r e n c e , M e t h o d , a n d D e c i s i o n : T o w a r d s a B a y e s i a n
P h i l o s o p h y o f S c i e n c e . R e i d e l , B o s t o n .
R O Y A L L , R. ( 1 9 7 1 ) . L i n e a r r e g r e s s i o n m o d e l s in f i n i t e p o p u l a t i o n s a m p l i n g
t h e o r y . In F o u n d a t i o n s o f S t a t i s t i c a l I n f e r e n c e , V . P. G o d a m b e a n d D . A .
S p r o t t ( e d s . ) . H o l t , R i n e h a r t , a n d W i n s t o n , T o r o n t o .
R O Y A L L , R. ( 1 9 7 6 ) . L i k e l i h o o d f u n c t i o n s in f i n i t e p o p u l a t i o n s a m p l i n g
s u r v e y . B i o m e t r i k a 6 3 , 6 0 5 - 6 1 7 .
R U B I N , D . B . ( 1 9 7 8 ) . B a y e s i a n i n f e r e n c e f o r c a u s a l e f f e c t s : t h e r o l e o f
r a n d o m i z a t i o n . A n n . S t a t i s t . 6 , 3 4 - 5 8 .
R U B I N , D . B . ( 1 9 8 4 ) . T h e u s e o f p r o b a b i l i t y s c o r e s in a p p l i e d B a y e s i a n
i n f e r e n c e . In B a y e s i a n S t a t i s t i c s I I
3
J. M . B e r n a r d o , M . H . D e G r o o t ,
D . V . L i n d l e y , a n d A . F . M . S m i t h ( e d s . ) .
S A V A G E , L . J. ( 1 9 5 4 ) . T h e F o u n d a t i o n s o f S t a t i s t i c s . J o h n W i l e y a n d S o n s ,
N e w Y o r k .
S A V A G E , L . J. ( 1 9 7 0 ) . C o m m e n t s o n a w e a k e n e d p r i n c i p l e o f c o n d i t i o n a l i t y . J .
A m e r . S t a t i s t . A s s o c . 6 5 , 3 9 9 - 4 0 1 .
S A V A G E , L . J. ( 1 9 7 6 ) . O n r e r e a d i n g R. A . F i s h e r ( w i t h d i s c u s s i o n ) . A n n .
S t a t i s t . 4 , 4 4 1 - 5 0 0 .
S A V A G E , L . J., e t . a l . ( 1 9 6 2 ) . T h e F o u n d a t i o n s o f S t a t i s t i c a l I n f e r e n c e .
M e t h u e n , L o n d o n .
S E I D E N F E L D , T . ( 1 9 7 9 ) . P h i l o s o p h i c a l P r o b l e m s o f S t a t i s t i c a l I n f e r e n c e .
R e i d e l , B o s t o n .
S E L L I A H , J. ( 1 9 6 4 ) . E s t i m a t i o n a n d t e s t i n g p r o b l e m s in a W i s h a r t d i s t r i b u t i o n .
P h . D . T h e s i s , D e p t . o f S t a t i s t i c s , S t a n f o r d U n i v e r s i t y .
S M I T H , T . M . F . ( 1 9 7 6 ) . T h e f o u n d a t i o n s o f s u r v e y s a m p l i n g : a r e v i e w . J . R o y .
S t a t i s t . S o c . A 1 3 9 , 1 8 3 - 2 0 4 .
S P R O T T , D . A . ( 1 9 7 3 a ) . N o r m a l l i k e l i h o o d s a n d t h e i r r e l a t i o n to l a r g e s a m p l e
t h e o r y o f e s t i m a t i o n . B i o m e t r i k a 6 0 , 4 5 7 - 4 6 5 .
S P R O T T , D . A . ( 1 9 7 3 b ) . P r a c t i c a l u s e s o f t h e l i k e l i h o o d f u n c t i o n . In
I n f e r e n c e a n d D e c i s i o n , U n i v e r s i t y P r e s s o f C a n a d a , T o r o n t o .
S P R O T T , D . A . ( 1 9 7 5 ) . M a r g i n a l a n d c o n d i t i o n a l s u f f i c i e n c y . B i o m e t r i k a 6 2 ,
5 9 9 - 6 0 5 .
S P R O T T , D . A . a n d K A L B F L E I S C H , J . D . ( 1 9 6 9 ) . E x a m p l e s o f l i k e l i h o o d s a n d
c o m p a r i s o n s w i t h p o i n t e s t i m a t e s a n d l a r g e s a m p l e a p p r o x i m a t i o n s . J . A m e r .
S t a t i s t . A s s o c . 6 4 , 4 6 8 - 4 8 4 .
S T E I N , C . ( 1 9 6 1 ) . E s t i m a t i o n o f m a n y p a r a m e t e r s . I n s t . M a t h . S t a t i s t . W a l d
L e c t u r e s , U n p u b l i s h e d .
S T E I N , C . ( 1 9 6 2 ) . A r e m a r k o n t h e l i k e l i h o o d p r i n c i p l e . J . R o y . S t a t i s t . S o c .
A 1 2 6 , 5 6 5 - 5 6 8 .
S T O N E , M . ( 1 9 7 6 ) . S t r o n g i n c o n s i s t e n c y f r o m u n i f o r m p r i o r s ( w i t h d i s c u s s i o n ) .
J . A m e r . S t a t i s t . A s s o c . 7 1 , 1 1 4 - 1 2 5 .
T H O M P S O N , M . E . ( 1 9 8 0 ) . L i k e l i h o o d p r i n c i p l e a n d r a n d o m i z a t i o n in s u r v e y
s a m p l i n g . R e p o r t 7 8 - 0 4 , D e p t . o f S t a t i s t i c s , U n i v e r s i t y o f W a t e r l o o .
T J U R , T . ( 1 9 7 8 ) . S t a t i s t i c a l i n f e r e n c e u n d e r t h e l i k e l i h o o d p r i n c i p l e .
P r e p r i n t 1 , I n s t i t u t e o f M a t h e m a t i c a l S t a t i s t i c s , U n i v e r s i t y o f C o p e n h a g e n ,
C o p e n h a g e n .
W A L L A C E , D . L . ( 1 9 5 9 ) . C o n d i t i o n a l c o n f i d e n c e level p r o p e r t i e s . A n n . M a t h .
S t a t i s t . 3 0 , 8 6 4 - 8 7 6 .
W E L C H , B . L . ( 1 9 3 9 ) . O n c o n f i d e n c e l i m i t s a n d s u f f i c i e n c y w i t h p a r t i c u l a r
r e f e r e n c e t o p a r a m e t e r s o f l o c a t i o n . A n n . M a t h . S t a t i s t . 1 0 , 5 8 - 6 9 .
W I L K I N S O N , G. N . ( 1 9 7 7 ) . O n r e s o l v i n g t h e c o n t r o v e r s y in s t a t i s t i c a l i n f e r e n c e
( w i t h d i s c u s s i o n ) . J . R o y . S t a t i s t . S o c . B 3 9 , 1 1 9 - 1 7 1 .
Y A T E S , F. ( 1 9 6 4 ) . F i d u c i a l p r o b a b i l i t y , r e c o g n i z a b l e s u b s e t s , a n d B e h r e n s
1
t e s t . B i o m e t r i c s 2 0 , 3 4 3 - 3 6 0 .
Z E L L N E R , A . ( 1 9 7 1 ) . A n I n t r o d u c t i o n t o B a y e s i a n I n f e r e n c e i n E c o n o m e t r i c s .
W i l e y , N e w Y o r k .
Z E L L N E R , A . ( 1 9 8 2 ) . A p p l i c a t i o n s o f B a y e s i a n a n a l y s i s in e c o n o m e t r i c s . P r e -
s e n t e d at t h e I n s t i t u t e o f S t a t i s t i c i a n s I n t e r n a t i o n a l C o n f e r e n c e o n P r a c -
t i c a l B a y e s i a n S t a t i s t i c s at S t . J o h n ' s C o l l e g e , C a m b r i d g e .
A D D I T I O N A L OR U P D A T E D R E F E R E N C E S
B A Y A R R I , M. J., D E G R O O T , M. H . , and K A D A N E , J.B . ( 1 9 8 7 ) . What is the likeli
hood function? (With D i s c u s s i o n ) . In Statistical Decision Theory and
Related Topics IV, Volume 1 , S. S. Gupta and J. Berger ( e d s . ) . Springer
V e r l a g , New Y ork.
B ER GER , J. ( 1 9 8 5 ) . Statistical Decision Theory and Bayesian Analysis.
SpringerVerlag, New York.
B ER GER , J. ( 1 9 8 7 ) . Robust Bayesian analysis: sensitivity to the prior. To
appear in the Proceedings of the Conference in Honor of I. J. Goody
K. Hinkelmann ( e d . ) .
B ER GER , J. and BERLINER, L. M. ( 1 9 8 6; previously 1 9 8 3 ) . Robust Bayes and
empirical Bayes analysis with contaminated priors. Ann. Statist. 14,
4 6 1 4 8 6 .
B ER GER , J. and B E R R Y , D. ( 1 9 8 7 ) . The relevance of stopping rules in statisti
cal inference. In Statistical Decision Theory and Related Topics IV,
Volume 1 , S. S. Gupta and J. Berger ( e d s . ) . SprinqerVerlag, New Y ork.
B ER GER , J. and B ER R Y , D. ( 1 9 8 8 ) . Statistical analysis and the illusion of
objectivity. To appear in American Scientist.
B ER GER , J. and DELA MPA DY , M. ( 1 9 8 7 ) . Testing precise hypotheses (with
D i s c u s s i o n ) . Statistical Science 2, 3 1 7 3 5 2 .
B ER GER , J. and S E L L K E , T . ( 1 9 8 7 ) . Testing a point null hypothesis: the
irreconcilability of Pvalues and evidence (with D i s c u s s i o n ) . J. Amer.
Statist. Assoc. 82, 1 1 21 3 9 .
ADDITIONAL OR UPDATED REFERENCES 160.1
BERLINER, M. ( 1 9 8 7 ) . Discussion of B a y a r r i , D e G r o o t , and Kadane ( 1 9 8 7 ) . In
Statistical Decision Theory and Related Topics IV
3
Volume 1, S. S. Gupta
and J. Berger ( e d s . ) . S p r i n g e r - V e r l a g , New York.
BHAVE, S. V. ( 1 9 8 4 ) . Two concepts of conditionality. J. Statist. Plann. and
Inf. 10, 131-135.
BUTLER, R. W. ( 1 9 8 6 ) . Predictive likelihood inference with applications (with
D i s c u s s i o n ) . J. Roy. Statist. Soc. B 47, 1-38 .
B U T L E R , R. W . ( 1 9 8 7 ) . A l i k e l y a n s w e r to ' what is t h e l i k e l i h o o d f u n c t i o n ?
1
Discussion of Bayarri et. a l . in Statistical Decision Theory and Related
Topics IV, Volume 1, S. S. Gupta and J. Berger ( e d s . ) . Springer-Verlag,
New York.
CASELLA, G. and BERGER, R. ( 1 9 8 7 ) . Reconciling Bayesian and frequentist
evidence in the one-sided testing problem. J. Amer. Statist. Assoc. 82,
1 0 6 - 1 1 1 .
DAWID, A. P. ( 1 9 8 6 ) . Discussion of E v a n s , F r a s e r , and Monette ( 1 9 8 6 ) . Canad.
J. Statist. 14, 1 9 6 - 1 9 7 .
DELAMPADY, M. and BERGER, J. ( 1 9 8 7 ) . Lower bounds on posterior probabilities
for multinomial and chi-squared t e s t s . Technical Report # 8 6 - 3 7 , Department
of S t a t i s t i c s , Purdue University, West L a f a y e t t e .
EVANS, M . , FRASER, D . A . S., and M O N E T T E , G. ( 1 9 8 5 a ) . M i x t u r e s , e m b e d d i n g s ,
and ancillarity. Canad. J. Statist. 13, 1-6.
EVANS, M . , FRASER, D . A. S., and M O N E T T E , G. ( 1 9 8 5 b ) . Regularity conditions
for statistical m o d e l s . Canad. J. Statist. 13, 1 3 7 - 1 4 4 .
EVANS, M., FRASER, D . A . S., and M O N E T T E , G. ( 1 9 8 5 c ) . On the role of principles
in statistical inference. In Statistical Theory and Data Analysis,
K. Matusita ( e d . ) . N o r t h - H o l l a n d , Amsterdam.
EVANS, M . , FRASER, D . A. S., and M O N E T T E , G. ( 1 9 8 6 ) . On principles and argu-
ments to likelihood (with d i s c u s s i o n ) . Canad. J. Statist. 14, 18 1-19 9 .
FRASER, D . A . S. ( 1 9 8 4 ) . Structural m o d e l s . In Encyclopedia of Statistical
Science, Volume 6, N. L. Johnson and S. Kotz ( e d s . ) . W i l e y , New Y o r k .
160.2 THE L I K E L I H O O D PRINCIPLE
FRASER, D . A . S., M O N E T T E , G., and NG, K. W . ( 1 9 8 4 ) . M a r g i n a l i z a t i o n , likeli-
hood, and structural m o d e l s . In Multivariate Analysis VI, P. R.
Krishnaiah ( e d . ) . N o r t h - H o l l a n d , Amsterdam.
G O O D , I. J. ( 1 9 8 4 ) . Notes C 1 4 0 , C 1 4 4 , C 1 9 9 , C 2 0 0 , and C 2 0 1 . J. Statist.
Computation and Simulation 19.
H A L L , P. and S E L I N G E R , B . ( 1 9 8 6 ) . Statistical s i g n i f i c a n c e : balancing e v i -
dence against d o u b t . Australian J. of Statist. 28, 3 5 4 - 3 7 0 .
H A R T I G A N , J. A . ( 1 9 6 7 ) . The likelihood and invariance p r i n c i p l e s . Ann. Math.
Statist. 3, 5 3 3 - 5 3 9 .
H I L L , B. ( 1 9 8 7 a ) . On the validity of the likelihood p r i n c i p l e . In Statistical
Decision Theory and Related Topics IV, Volume 1, S. S. Gupta and J. Berger
( e d s . ) . S p r i n g e r - V e r l a g , New Y o r k .
H I L L , B . ( 1 9 8 7 b ) . The validity of the likelihood p r i n c i p l e . The American
Statistician 41, 9 5 - 1 0 0 .
H I N D E , J. and A I T K I N , M . ( 1 9 8 7 , previously 1 9 8 4 ) . Canonical likelihood: a
new likelihood treatment of nuisance p a r a m e t e r s . Biometrika 74, 4 5 - 5 8 .
J O S H I , V. M. ( 1 9 8 3 ) . Likelihood p r i n c i p l e . In Encycl. Statist. Sci. 4,
6 4 4 - 6 4 7 , S. Kotz and N. L. Johnson ( e d s . ) . W i l e y , New Y o r k .
KAY, R. ( 1 9 8 5 ) . Partial likelihood. In Encycl. Statist. Sci. 6, 5 9 1 - 5 9 3 .
S. Kotz and N. L. Johnson ( e d s . ) . W i l e y , New Y o r k .
L I N D L E Y , D . V. ( 1 9 5 7 ) . A statistical p a r a d o x . Biometrika 44, 1 8 7 - 1 9 2 .
L I N D L E Y , D . V. ( 1 9 7 7 ) . A problem in forensic s c i e n c e . Biometrika 64, 2 0 7 - 2 1 3 .
P O C O C K , S. J. ( 1 9 7 7 ) . Group sequential m e t h o d s in the design and analysis of
clinical t r i a l s . Biometrika 64, 1 9 1 - 1 9 9 .
S H A F E R , G. ( 1 9 8 2 ) . L i n d l e y ' s p a r a d o x . J. Amer. Statist. Assoc. 77, 3 2 5 - 3 5 1 .
S M I T H , A . F. M. and S P I E G E L H A L T E R , D . J. ( 1 9 8 0 ) . Bayes factors and choice
criteria for linear m o d e l s . J. Roy. Statist. Soc B 42, 2 1 3 - 2 2 0 .
Z E L L N E R , A . ( 1 9 8 4 ) . Posterior odds ratios for regression h y p o t h e s e s : general
c o n s i d e r a t i o n s and some specific r e s u l t s . In Basic Issues in Econometrics.
U n i v e r s i t y of Chicago P r e s s , C h i c a g o .
D I S C U S S I O N : A U X I L I A R Y P A R A M E T E R S A N D
S I M P L E L I K E L I H O O D F U N C T I O N S
B Y P R O F E S S O R S M. J. B A Y A R R I A N D M. H. D E G R O O T
( U n i v e r s i t y o f V a l e n c i a a n d C a r n e g i e M e l l o n U n i v e r s i t y )
W e a r e g r a t e f u l t o o u r f r i e n d s J i m B e r g e r a n d R o b e r t W o l p e r t f o r
g i v i n g u s t h i s o p p o r t u n i t y t o c o n t r i b u t e t o t h e i r v a l u a b l e a n d c o m p r e h e n s i v e
s t u d y o f t h e l i k e l i h o o d p r i n c i p l e . O u r p r i o r d i s t r i b u t i o n w a s h i g h l y c o n c e n
t r a t e d o n t h e i r w r i t i n g a n e x c e l l e n t m o n o g r a p h , a n d t h e e v i d e n c e p r o v i d e d b y
t h e d a t a w e n o w h a v e c o n f i r m s o u r p r i o r o p i n i o n . T h e i r t r e a t m e n t i s c a r e f u l
a n d t h o u g h t f u l ( t h i s m e a n s t h a t w e a g r e e w i t h t h e m ) a n d l e a v e s l i t t l e r o o m f o r
f u r t h e r d i s c u s s i o n . N e v e r t h e l e s s , h a r e m o s t o d o l o p o s i b l e ; w e w i l l t r y .
O u r c o m m e n t s w i l l b e r e s t r i c t e d t o t h e m a t e r i a l i n S e c t i o n 3.5
p e r t a i n i n g t o t h e c o n s t r u c t i o n o f a l i k e l i h o o d f u n c t i o n t o b e u s e d i n s t a t i s t i
c a l p r o b l e m s i n v o l v i n g n u i s a n c e v a r i a b l e s , n u i s a n c e p a r a m e t e r s , a n d f u t u r e
o b s e r v a t i o n s . W e w i l l u s e t h e n o t a t i o n x , y , w , , a n d t o r e p r e s e n t t h e s a m e
q u a n t i t i e s a s i n S e c t i o n 3 . 5 . 2 . H e r e , x i s t h e o b s e r v a t i o n a n d a l l t h e o t h e r
q u a n t i t i e s a r e u n o b s e r v e d , y a n d w a r e r e g a r d e d a s v a r i a b l e s , a n d a r e
r e g a r d e d a s p a r a m e t e r s , a n d y a n d a r e o f i n t e r e s t . It w i l l b e c o n v e n i e n t f o r
u s t o u s e t h e n o t a t i o n f ( x , y , w | , ) r a t h e r t h a n f
( c
( x , y , w ) t o d e n o t e a
c o n d i t i o n a l d e n s i t y .
T h e b a s i c p u r p o s e o f a l i k e l i h o o d f u n c t i o n i s t o s e r v e a s a f u n c
t i o n t h a t r e l a t e s o b s e r v e d a n d u n o b s e r v e d q u a n t i t i e s , a n d c o n v e y s a l l t h e
r e l e v a n t i n f o r m a t i o n p r o v i d e d b y t h e o b s e r v e d d a t a a b o u t t h e u n o b s e r v e d q u a n t i
t i e s . F r o m t h e B a y e s i a n p o i n t o f v i e w , w h i c h w e s h a l l a d o p t i n t h i s d i s c u s s i o n ,
1 6 0 . 3
w e a r e i n t e r e s t e d i n f i n d i n g f ( y , | x ) . If t h e d e s i g n o f t h e e x p e r i m e n t i s n ot
u n d e r c o n s i d e r a t i o n , w e c ou l d s i m p l y w a i t u n t i l x i s o b s e r v e d a n d t h e n a s s e s s
t h e d e n s i t y f ( y , [ x ) d i r e c t l y . H o w e v e r , t o g u i d e o u r t h i n k i n g a n d t o h e l p m a k e
o u r c o n c l u s i o n s m o r e c o n v i n c i n g t o o t h e r s , w e w o u l d t y p i c a l l y i n t r o d u c e s ome
s t r u c t u r e i n t o o u r l e a r n i n g p r o c e s s b y w r i t i n g f ( y , | x ) i n t h e for m
( 1 ) f ( y , | ) f ( | ) f ( y | ) f ( c ) .
If t h e r e i s g e n e r a l a g r e e m e n t a b ou t t h e for m o f t h e d e n s i t i e s f ( x | ) a n d
f ( y | x ) t h e n t h e s e d e n s i t i e s c a n b e r e g a r d e d a s "g i v e n " a n d in t h e s p i r i t o f
( 3 . 5 . 1 ) , a l i k e l i h o o d f u n c t i o n c o u l d b e d e f i n e d a s
( 2 )
( y , ) = f ( x | ) f ( y | x , ) = f ( x , y | ) .
In B D K w e r e f e r r e d t o t h i s l i k e l i h o o d f u n c t i o n a s LF b e c a u s e i t i s d e r i v e d
fr om t h e c o n d i t i o n a l d e n s i t y o f t h e "v a r i a b l e s " g i v e n t h e "p a r a m e t e r s ".
W e r e g a r d t h e l i k e l i h o o d f u n c t i o n ( 2 ) a s u n s u i t a b l e a s a g e n e r a l
d e f i n i t i o n b e c a u s e w e do n ot b e l i e v e t h e r e i s a c l e a r c u t d i s t i n c t i o n b e t w e e n
u n o b s e r v e d v a r i a b l e s a n d p a r a m e t e r s . T h e for m o f ( 2 ) r e l i e s on t h e d e n s i t y
f ( x , y | ) b e i n g g i v e n o r a g r e e d o n . W e c a n r e w r i t e t h i s d e n s i t y a s
( 3 ) f ( , y | ) = f ( | y , ) f ( y | ) ,
a n d a g r e e m e n t a b o u t f ( x , y | ) i s e q u i v a l e n t t o a g r e e m e n t a b ou t b ot h f a c t o r s on
t h e r i g h t h a n d s i de o f ( 3 ) . In t h i s c a s e t h e l i k e l i h o o d f u n c t i o n w o u l d b e
g i v e n b y ( 2 ) . H o w e v e r , it i s p o s s i b l e t h a t t h e r e i s a g r e e m e n t a b ou t t h e for m
o f f ( x | y ) w h i l e t h e for m of f ( y | ) i s c o n s i d e r e d a s h i g h l y s u b je c t i v e . In
t h i s c a s e , a l i k e l i h o o d f u n c t i o n for y a n d b a s e d on t h e o b s e r v a t i o n x w o u l d
b e s i m p l y
( 4)
( y , ) = f ( x | y , ) .
T h u s , w h e n a n e x p e r i m e n t e r r e p o r t s t o u s s ome p a r t i c u l a r f u n c t i o n
o f y a n d w h i c h i s h i s or h e r l i k e l i h o o d f u n c t i o n b a s e d on t h e o b s e r v e d d a t a ,
w e w o u l d s t i l l n e e d f u r t h e r i n f o r m a t i o n fr om t h e e x p e r i m e n t e r in o r d e r t o b e
W h e n x i s a v e c t o r o f o b s e r v a t i o n s , o n e t y p i c a l w a y i n w h i c h a c o n
v e n i e n t c h o i c e o f t h e a u x i l i a r y p a r a m e t e r c a n s i m p l i f y t h e d e n s i t y f ( x | , n )
i s m a k i n g t h e c o m p o n e n t s o f x c o n d i t i o n a l l y i n d e p e n d e n t . M o r e i m p o r t a n t l y , a
c o n v e n i e n t c h o i c e o f m a y m a k e y a n d w c o n d i t i o n a l l y i n d e p e n d e n t o f x g i v e n
a n d . In t h i s c a s e , f ( y , w | x , , ) r e d u c e s t o f ( y , w | , ) a n d t h e l i k e l i h o o d
f u n c t i o n ( 6 ) b e c o m e s
(7) *
(y,W,,) = f(x|,) f(y,w|,).

R e g a r d l e s s of w h e t h e r t h e d e n s i t y f ( y , w | , ) i s g i v e n o r s u b j e c t i v e , it d o e s
n ot i n v o l v e t h e d a t a x , s o a l l t h e e v i d e n c e i n x a b o u t t h e u n k n o w n s i s c o n t a i n
e d i n t h e f i r s t f a c t o r f ( x | , ) on t h e r i g h t h a n d s i de o f ( 7 ) . T h u s , w e b e
l i e v e t h a t i t i s t h e o n l y f a c t o r t h a t s h ou l d b e i n c l u d e d i n t h e l i k e l i h o o d
f u n c t i o n . T h e i n c l u s i o n o f o t h e r f u n c t i o n s o f t h e u n k n o w n s , s u c h a s f ( y , w | , )
or t h e p r i o r f ( , n ) , w h i c h do n ot d e p e n d on t h e d a t a , s e e ms a r t i f i c i a l .
It s h ou l d b e n ot e d t h a t t h e l i k e l i h o o d f u n c t i o n t h a t w e a r e r e c o m
m e n d i n g i n t h i s s i t u a t i o n , n a m e l y
( 8 ) ,
( y , w, , ) = f ( x | , n ) ,
c a n a l s o b e e x p r e s s e d b e c a u s e o f c o n d i t i o n a l i n d e p e n d e n c e a s
( 9 ) *
( y , w , , ) = f ( x | y , w , , ) .
In o t h e r w o r d s , t h i s l i k e l i h o o d f u n c t i o n i s s i m p l y t h e c o n d i t i o n a l d e n s i t y o f
t h e o b s e r v a t i o n s g i v e n t h e u n o b s e r v e d q u a n t i t i e s . In B DK t h i s l i k e l i h o o d
f u n c t i o n w a s c a l l e d LF . a n d i s i n a c c or d w i t h t h e b a s i c d e f i n i t i o n ( 3 . 1 . 1 )
g i v e n b y B e r g e r a n d W o l p e r t .
M o r e g e n e r a l l y , e y e r y B a y e s i a n a n a l y s i s p r o c e e d s fr om a s p e c i f i c a
t i on o f t h e jo i n t d e n s i t y f ( x , y , w , , ) . If w e l e t s d e n o t e t h e s e t { x , y , v
, }
o f a l l t h e c o m p o n e n t s of a l l t h e q u a n t i t i e s c o n s i d e r e d i n t h e p r o b l e m , a n d l e t
s
1
a n d Sp d e n o t e n o n e m p t y s u b s e t s o f s s u c h t h a t s , s = a n d s , u
s
o
= s
t h e n t h e jo i n t d e n s i t y f ( s ) c a n b e e x p r e s s e d a s t h e p r o d u c t f ( s ) = f ( s J s >
. T h e v a r i o u s l i k e l i h o o d f u n c t i o n s u n d e r c o n s i d e r a t i o n i n t h i s d i s c u s s i o n
D I S C U S S I O N B Y B A Y A R R I A N D D E G R O O T 1 6 0 . 5
a b l e t o m a k e i n f e r e n c e s o r c a l c u l a t e p o s t e r i o r d i s t r i b u t i o n s . W e m u s t k n ow
w h e t h e r or n ot t h e d e n s i t y f ( y | ) h a s b e e n i n c l u d e d i n t h e l i k e l i h o o d f u n c t i o n .
In o t h e r w o r d s , i n o r d e r t o b e a b l e t o u s e t h i s l i k e l i h o o d f u n c t i o n , w e m u s t
k n ow n ot o n l y t h e f u n c t i o n i t s e l f b u t a l s o w h i c h f a c t o r s h a v e b e e n u s e d t o d e
r i v e i t . ( W e wi l l a r g u e l a t e r i n t h i s d i s c u s s i o n t h a t i t i s u n n e c e s s a r y e v e r
t o i n c l u d e t h e f a c t o r f ( y | ) i n t h e l i k e l i h o o d f u n c t i o n i n o r d e r t o c o n v e y t h e
e v i d e n c e p r o v i d e d b y t h e da t a x , s i n c e t h i s f a c t o r d o e s n ot i n v o l v e x . )
It s h o u l d b e n ot e d t h a t t h e f a c t o r s on t h e r i g h t h a n d s i d e o f ( 2 )
c o n t a i n o n l y t h e v a r i a b l e s y a n d p a r a m e t e r s t h a t a r e o f i n t e r e s t . In m a n y
p r o b l e m s , t h e d e n s i t i e s f ( x | ) a n d f ( y | x , ) c a n s t i l l b e d i f f i c u l t t o s p e c i f y
o r c a n s t i l l b e c o n s i d e r e d h i g h l y s u b je c t i v e b y o t h e r s . T h e s e d i f f i c u l t i e s a r e
u s u a l l y r e d u c e d b y i n t r o d u c i n g f u r t h e r s t r u c t u r e i n t o t h e l e a r n i n g p r o c e s s b y
m e a n s o f a m o r e d e t a i l e d s p e c i f i c a t i o n o f t h e "p a r a m e t e r s p a c e " o f a n d t h e
"s a m p l e s p a c e " o f y . T h e s e s p e c i f i c a t i o n s a r e r e p r e s e n t e d b y a "n u i s a n c e
p a r a m e t e r " a n d a "n u i s a n c e v a r i a b l e " w . A s a r e s u l t , ( 1 ) n o w b e c o m e s
( 5) f ( y , | x )

/ / f ( x | > n ) f ( y , w | x , , ) f ( , n ) dw d .
It s h o u l d b e e m p h a s i z e d t h a t a n d w a r e s e l e c t e d b y u s f o r o u r
c o n v e n i e n c e . If w e h a v e b e e n s u c c e s s f u l i n o u r s e l e c t i o n , t h e n t h e r e wi l l b e
g e n e r a l a g r e e m e n t a mon g o t h e r s on t h e for m o f f ( x | , n ) a n d f ( y , w | x , , ) . It
i s p r e s u m a b l y b e c a u s e o f s u c h a g r e e m e n t t h a t B e r g e r a n d W o l p e r t r e g a r d t h e s e
d e n s i t i e s a s b e i n g "g i v e n ", a n d d e f i n e a l i k e l i h o o d f u n c t i o n ( 3 . 5 . 1 ) t o b e
t h e i r p r o d u c t
(6)
(y,,,) = f(x|,) f(y,w|x,,)

= f(>y>w|,).
In v i e w o f t h e s e c o m m e n t s , t h e t r a d i t i o n a l e x p r e s s i o n "n u i s a n c e p a r a m e t e r " f o r
s e e m s i n a p p r o p r i a t e . B e c a u s e i t h e l p s u s t o b u i l d m o d e l s a n d t o a c h i e v e
a g r e e m e n t a b o u t t h o s e m o d e l s , m i g h t b e t t e r b e c a l l e d a n a u x i l i a r y p a r a m e t e r .
D I S C U S S I O N B Y B A Y A R R I A N D D E G R O O T 1 6 0 . 7
a r e of t h e for m f ( s J s p ) for s ome p a r t i c u l a r c h o i c e of s , , o r a r e d e r i v e d fr om
f s ^ s ^ ) b y i n t e g r a t i n g ou t q u a n t i t i e s t h a t a r e n ot o f i n t e r e s t . T h e s u b s e t
s
1
i s a l w a y s t a k e n t o c o n t a i n x a n d u s u a l l y , a s i n LF , t o c o n t a i n o t h e r
" v a r i a b l e s " w i t h g i v e n d i s t r i b u t i o n s . H o w e v e r , i t s h o u l d b e e m p h a s i z e d t h a t
Sj i s s o m e t i m e s a l s o t a k e n t o c o n t a i n c o m p o n e n t s o f t h e " p a r a m e t e r s " , a s i n
p
B e r g e r a n d W o l p e r t ' s ( 3 . 5 . 2 ) w h e r e i s i m p l i c i t l y m o v e d i n t o s, a n d t h e n
i n t e g r a t e d o u t . ( In m o r e c o l l o q u i a l t e r m s , t h e c h o i c e o f a l i k e l i h o o d f u n c t i o n
i s e s s e n t i a l l y t h e c h o i c e o f w h e r e t o p u t t h e b a r i n t h e jo i n t d e n s i t y
f ( x
5
y > w , , ) . In LF i t i s p u t b e t w e e n w a n d , w h e r e a s i n LF . i t i s p u t
b e t w e e n x a n d y . )
C l e a r l y , t h e r e a r e v e r y m a n y d i f f e r e n t p o s s i b l e c h o i c e s o f s , , a n d
t h e d e f i n i t i o n o f t h e l i k e l i h o o d f u n c t i o n c a n b e c o m e y e r y a r b i t r a r y . T h e
f u n d a m e n t a l i de a i s t h a t i n o r d e r t o c o n v e y t h e e v i d e n c e a b o u t t h e u n k n o w n s
p r o v i d e d b y t h e d a t a , i t i s u n n e c e s s a r y t o i n c l u d e a n y q u a n t i t i e s o t h e r t h a n x
i n s , . I n d e e d , t h e p o s s i b l e i n c l u s i o n o f o t h e r q u a n t i t i e s c a n o n l y l e a d t o
c o n f u s i o n f o r t h e u s e r s o f t h e s e l i k e l i h o o d f u n c t i o n s . T h u s w e c l a i m t h a t t h e
e v i d e n c e p r o v i d e d b y t h e d a t a i s c o n v e y e d m o s t e f f i c i e n t l y a n d m o s t g e n e r a l l y
b y t h e l i k e l i h o o d f u n c t i o n t h a t w e h a v e c a l l e d L F
Q b s
, a s g i v e n b y ( 9 ) .
It s h o u l d b e e m p h a s i z e d t h a t w e a r e m a k i n g a n i m p o r t a n t d i s t i n c t i o n
b e t w e e n t h e e v i d e n c e p r o v i d e d b y x a b o u t a n d y , a n d t h e i n f o r m a t i o n t h a t i s
n e e d e d t o m a k e i n f e r e n c e s a b o u t a n d y . T h i s d i s t i n c t i o n i s c l e a r i n t h e
B a y e s i a n a p p r o a c h , b u t l e s s c l e a r i n t h e l i k e l i h o o d b a s e d f r e q u e n t i s t a p p r o a c h .
H o w e v e r , e v e n i n t h a t a p p r o a c h , t h e d i s t i n c t i o n b e c o m e s c l e a r i f L F
Q b s
is
a l w a y s u s e d b u t i n f e r e n c e s i n c o r p o r a t e o t h e r f a c t o r s s u c h a s f ( y | ) i n ( 3 ) .
T h u s , a l a r g e v a r i e t y o f i n f e r e n t i a l a i m s c a n b e a c c o m p l i s h e d w i t h ju s t L F ^
r a t h e r t h a n a n e q u a l l y l a r g e v a r i e t y o f l i k e l i h o o d f u n c t i o n s .
D I S C U S S I O N B Y P R O F E S S O R B R U C E M. H I L L
( U n i v e r s i t y o f M i c h i g a n )
I s h o u l d l i k e t o c o n g r a t u l a t e B e r g e r a n d W o l p e r t o n t h e i r l u c i d
a n d i n f o r m a t i v e p r e s e n t a t i o n o f t h e h i s t o r y a n d s u b s t a n c e o f t h e l i k e l i h o o d
p r i n c i p l e , a n d t h e i r e x t e n s i o n o f t h e l i k e l i h o o d p r i n c i p l e . A l t h o u g h I f o u n d
t h e i r e x t e n s i o n i n t e r e s t i n g , a n d h o p e t h a t it m a y r e s o l v e s o m e d o u b t s c o n c e r n -
i n g t h e s t a t u s o f t h e l i k e l i h o o d p r i n c i p l e in t h e i n f i n i t e c a s e , m y o w n v i e w
is t h a t t h e l i k e l i h o o d p r i n c i p l e r e a l l y s t a n d s o r f a l l s in t h e f i n i t e c a s e .
T h e p a r t o f t h e i r a r t i c l e t h a t I w o u l d l i k e t o d i s c u s s is t h a t c o n c e r n i n g t h e
v a r i o u s e x a m p l e s t h a t h a v e b e e n p r e s e n t e d a g a i n s t t h e l i k e l i h o o d p r i n c i p l e ,
w h e r e m y v i e w s a r e p e r h a p s d i f f e r e n t f r o m t h o s e o f B e r g e r a n d W o l p e r t ( B W ) , a n d
in t h e c o u r s e o f t h e d i s c u s s i o n m y a p p r o a c h t o t h e i n f i n i t e c a s e s h o u l d b e c o m e
c l e a r . B e f o r e d o i n g s o I w a n t t o p r e f a c e m y r e m a r k s w i t h t w o c o m m e n t s . F i r s t ,
I t h i n k t h a t w e B a y e s i a n s s h o u l d b e g r a t e f u l t o S t e i n , S t o n e , F r a s e r a n d
M o n e t t e , f o r t h e i r i n t e r e s t i n g e x a m p l e s , a l l o f w h i c h h a v e s o m e r e a l s u b s t a n c e
t o t h e m . T h e o r i e s r e q u i r e g o o d c r i t i c i s m in o r d e r t o g r o w , a n d t h e l a c k o f
s u c h c r i t i c i s m h a s b e e n d e t r i m e n t a l t o t h e B a y e s i a n t h e o r y . S e c o n d l y , I t h i n k
it is e s s e n t i a l t h a t w e k e e p in m i n d t h e d i s t i n c t i o n b e t w e e n t h e l i k e l i h o o d
p r i n c i p l e (by w h i c h I m e a n t h e f o r m a l l i k e l i h o o d p r i n c i p l e o f B W ) a n d v a r i o u s
i m p l e m e n t a t i o n s o r i n t e r p r e t a t i o n s o f t h e l i k e l i h o o d p r i n c i p l e . I s h a l l t r y t o
d e m o n s t r a t e t h a t n o n e o f t h e e x a m p l e s s p e a k a g a i n s t t h e l i k e l i h o o d p r i n c i p l e a s
s u c h , b u t r a t h e r t h a t t h e y c o n s t i t u t e f r e q u e n t i s t i c a r g u m e n t s a g a i n s t t h e u s e
o f s p e c i f i c i m p r o p e r ( o r d i f f u s e f i n i t e l y a d d i t i v e ) p r i o r d i s t r i b u t i o n s . I
s h a l l t h e n e x p l a i n w h y I t h i n k s u c h a r g u m e n t s h a v e n o r e a l t e e t h t o t h e m .
161
Le t us b e g i n w i t h t h e e x a m p l e o f S t e i n . A l t h o u g h it w a s o r i g i n a l l y
p r e s e n t e d b y S t e i n as an a r g u m e n t a g a i n s t t h e l i k e l i h o o d p r i n c i p l e , w i t h a n
a r g u m e n t a g a i n s t l a z y B a y e s i a n s t a c k e d o n a t t h e e n d , I r e g a r d it a s p r i m a r i l y
an a r g u m e n t a g a i n s t B a y e s i a n s ( s u c h a s m y s e l f ) w h o u s e i m p r o p e r o r f i n i t e l y
a d d i t i v e p r i o r d i s t r i b u t i o n s t o o b t a i n a p p r o x i m a t e p o s t e r i o r d i s t r i b u t i o n s , a n d
a l s o a g a i n s t t h e t h e o r y o f d e Fi n e t t i ( w h i c h I f o l l o w ) w h i c h in p r i n c i p l e d o e s
n o t r u l e o u t a n y f i n i t e l y a d d i t i v e p r i o r d i s t r i b u t i o n . T o b e g i n w i t h , t h e
l i k e l i h o o d p r i n c i p l e d o e s n o t j u s t i f y e i t h e r ( 5 . 3 . 4 ) o r ( 5 . 3 . 5 ) o f B W , s i n c e it
d o e s n o t s u g g e s t a w a y o f a t t a c h i n g p r o b a b i l i t y t o s e t s . It is t r u e o f c o u r s e
t h a t s o m e i n d i v i d u a l s w h o s u p p o r t t h e l i k e l i h o o d p r i n c i p l e ( p e r h a p s w i t h q u a l i -
f i c a t i o n s ) , s u c h a s G e o r g e B a r n a r d a n d A . W . F. E d w a r d s , a l s o s o m e t i m e r e c o m -
m e n d t h e u s e o f s u c h p r o b a b i l i s t i c i n t e r p r e t a t i o n s o f t h e l i k e l i h o o d f u n c t i o n ,
b u t t h a t is b y v i r t u e o f a d d i t i o n a l a s s u m p t i o n s , w h e t h e r e x p l i c i t o r i m p l i c i t ,
a n d is n o t r e a l l y p a r t o f t h e l i k e l i h o o d p r i n c i p l e . BW a p p a r e n t l y a c c e p t t h a t
( 5 . 3 . 6 ) is a s t r o n g a r g u m e n t a g a i n s t t h e u s e o f a u n i f o r m i m p r o p e r p r i o r d i s -
t r i b u t i o n f o r t h e t a , b u t s u g g e s t t h a t t h e r e is n o d i f f i c u l t y f o r B a y e s i a n s b e -
c a u s e o n t h e o n e h a n d t h e t a is a s c a l e p a r a m e t e r a n d s o it is t h e l o g a r i t h m o f
t h e t a ( i f a n y t h i n g ) t h a t s h o u l d b e g i v e n a u n i f o r m p r i o r d i s t r i b u t i o n , a s in
B a r n a r d ' s r e p l y t o S t e i n ; a n d o n t h e o t h e r h a n d , a n d m o r e i m p o r t a n t l y , t h e y
a r g u e t h a t w i t h p r o p e r p r i o r d i s t r i b u t i o n s t h e t y p e o f i n t e r v a l t h a t S t e i n
s h o w s h a s b a d f r e q u e n t i s t i c p r o p e r t i e s c a n o c c u r o n l y r a r e l y s i n c e "Y is a l m o s t
c e r t a i n t o b e e n o r m o u s . " A l t h o u g h I a g r e e w i t h b o t h a r g u m e n t s o f B W , i t s e e m s
t o m e t h a t t h e i s s u e b e i n g r a i s e d b y S t e i n is n o t w h e t h e r a s e n s i b l e B a y e s i a n
c a n a v o i d t h e i n t e r v a l s ( 5 . 3 . 5 ) , b u t r a t h e r w h e t h e r b y v i r t u e o f c a r e l e s s n e s s
o r b e c a u s e h i s t h e o r y p e r m i t s s u c h i n t e r v a l s ( a s f o r e x a m p l e is t r u e o f t h e
d e Fi n e t t i t h e o r y ) t h e u n w a r y o r e v e n w a r y B a y e s i a n w i l l b e c o m e f r e q u e n t i s t i c
p r e y . If t h e r e r e a l l y w e r e a t r a p w i t h t e e t h t o it t h e n S t e i n ' s e x a m p l e w o u l d
s u g g e s t e i t h e r t h a t o n e s t i c k w i t h p r o p e r p r i o r d i s t r i b u t i o n s , o r e l s e b e q u i t e
c a r e f u l in t h e c h o i c e o f i m p r o p e r , o r m e r e l y f i n i t e l y a d d i t i v e , p r i o r d i s t r i b u -
t i o n s , a n d a s S t e i n s a y s , t h i s w o u l d m a k e t h e " p r i o r d i s t r i b u t i o n u s e d d e p e n d o n
D I S C U S S I O N B Y H I L L 1 6 3
a c c i d e n t a l f e a t u r e s o f t h e d e c i s i o n p r o b l e m . " N o w t o m e t h i s s e e m s t o b e a
real a n d i m p o r t a n t i s s u e . S u p p o s e w e a r e d i s c u s s i n g a r e a l - w o r l d p a r a m e t e r ,
w h o s e e x i s t e n c e , d e f i n i t i o n , a n d m e a n i n g , in n o w a y d e p e n d u p o n t h e e x p e r i m e n t
t o b e p e r f o r m e d . ( T h e e x i s t e n c e o f s u c h p a r a m e t e r s m a y b e f a r l e s s c o m m o n
t h a n is u s u a l l y a s s u m e d , b u t p r e s u m a b l y w e c o u l d a l l a g r e e t h a t a t l e a s t s o m e
s u c h p a r a m e t e r s e x i s t s , o r a t a n y r a t e a r e w o r t h d i s c u s s i n g , a n d c o n f i n e a t t e n -
t i o n t o t h e s e . W h e n t h e " p a r a m e t e r " d e p e n d s u p o n t h e e x p e r i m e n t f o r i t s
e x i s t e n c e a n d m e a n i n g , t h e n , o f c o u r s e , t h e l i k e l i h o o d p r i n c i p l e d o e s n o t
a p p l y . ) In t h e s u b j e c t i v e B a y e s i a n t h e o r y o f d e Fi n e t t i a n d L. J . S a v a g e , t h e
p r i o r d i s t r i b u t i o n f o r s u c h a p a r a m e t e r w o u l d b e c h o s e n t o r e p r e s e n t o n e ' s
o p i n i o n s a b o u t t h a t p a r a m e t e r , a n d w h e t h e r t h e m e a s u r e m e n t i s t o b e m a d e
a c c o r d i n g t o t h e n o r m a l m o d e l o r a c c o r d i n g t o ( 5 . 3 . 3 ) s h o u l d n o t i n a n y w a y
a f f e c t t h e p r i o r d i s t r i b u t i o n . If f o r s o m e r e a s o n I t h o u g h t t h a t a u n i f o r m
i m p r o p e r p r i o r f o r t h e t a w a s a p p r o p r i a t e a s a n a p p r o x i m a t i o n u n d e r t h e n o r m a l
m o d e l , a n d t h e n l e a r n e d t h a t i n f a c t t h e m e a s u r e m e n t e r r o r w a s d i s t r i b u t e d
a c c o r d i n g t o ( 5 . 3 . 3 ) , b u t w i t h t h e n e a r l y i d e n t i c a l l i k e l i h o o d f u n c t i o n s t h a t
S t e i n p r o d u c e s , t h e n it s e e m s t o m e t h a t t h e u n i f o r m p r i o r s h o u l d s t i l l p r o v i d e
a s a t i s f a c t o r y a p p r o x i m a t i o n in o b t a i n i n g m y p o s t e r i o r d i s t r i b u t i o n . F u r t h e r -
m o r e , a B a y e s i a n w h o w o u l d u s e t h e u n i f o r m p r i o r f o r t h e t a w h e n t h e m e a s u r e -
m e n t e r r o r is n o r m a l l y d i s t r i b u t e d , b u t w o u l d u s e a u n i f o r m p r i o r d i s t r i b u t i o n
f o r t h e l o g a r i t h m o f t h e t a w h e n t h e m e a s u r e m e n t e r r o r is d i s t r i b u t e d a c c o r d i n g
t o ( 5 . 3 . 3 ) , is c o m i n g v e r y c l o s e t o v i o l a t i n g t h e l i k e l i h o o d p r i n c i p l e in
t h i s e x a m p l e , s i n c e h e is m a k i n g v e r y d i f f e r e n t i n f e r e n c e a b o u t t h e t a in t h e
t w o c a s e s e v e n t h o u g h t h e l i k e l i h o o d f u n c t i o n s a r e i n a c e r t a i n s e n s e v e r y
" c l o s e , " a n d t h e t a is t h e s a m e f i x e d q u a n t i t y . S e e S a v a g e ( 1 9 7 0 ) f o r
a r e l a t e d a r g u m e n t . ( S o m e B a y e s i a n s , f o r e x a m p l e B o x a n d T i a o , a c t u a l l y
r e c o m m e n d t h a t p r i o r d i s t r i b u t i o n s b e m a d e t o d e p e n d u p o n t h e s a m p l i n g s c h e m e ,
a n d s o w o u l d u s e a d i f f e r e n t p r i o r d i s t r i b u t i o n f o r t h e p a r a m e t e r o f a
B e r n o u l l i s e q u e n c e i f t h e e x p e r i m e n t w e r e o f b i n o m i a l f o r m t h a n i f t h e e x p e r i -
m e n t w e r e o f n e g a t i v e b i n o m i a l f o r m , e v e n w h e n t h e c h o i c e o f t h e e x p e r i m e n t is
m a d e b y r a n d o m i z a t i o n a n d i s t h u s u n i n f o r m a t i v e , a n d t h i s c l e a r l y v i o l a t e s
t h e l i k e l i h o o d p r i n c i p l e . W h a t e v e r e l s e m a y b e s a i d o f s u c h a n a p p r o a c h , i t
is c e r t a i n l y n o t a p a r t o f t h e o r d i n a r y s u b j e c t i v e B a y e s i a n t h e o r y , i n w h i c h
t h e p r i o r d i s t r i b u t i o n f o r a p a r a m e t e r o f t h e t y p e w e a r e d i s c u s s i n g d o e s n o t
d e p e n d u p o n w h a t e x p e r i m e n t s m a y o r m a y n o t b e p e r f o r m e d a t s o m e f u t u r e t i m e .
F u r t h e r m o r e , i f B W c h o o s e t o u s e i m p r o p e r p r i o r d i s t r i b u t i o n s , b u t o n l y w h e n
t h e s e d o n o t l e a d t o b a d f r e q u e n t i s t p r o p e r t i e s , t h e n t h e y t o o a r e p e r i l o u s l y
c l o s e t o a v i o l a t i o n o f t h e l i k e l i h o o d p r i n c i p l e , s i n c e t h e i r c h o i c e w i l l t u r n
o u t t o d e p e n d u p o n t h e s a m p l i n g d i s t r i b u t i o n , j u s t a s w i t h B o x a n d T i a o . )
W h a t t h e S t e i n e x a m p l e a c t u a l l y d e m o n s t r a t e s is t h a t i f a B a y e s i a n u s e s t h e
u n i f o r m p r i o r d i s t r i b u t i o n f o r t h e t a , t h e n h i s p o s t e r i o r p r o b a b i l i t y f o r t h e
i n t e r v a l ( 5 . 3 . 5 ) , g i v e n a n y y , is a t l e a s t . 9 5 , w h i l e g i v e n a n y t h e t a , t h e
f r e q u e n t i s t p r o b a b i l i t y f o r t h e i n t e r v a l is v e r y t i n y a c c o r d i n g t o ( 5 . 3 . 6 ) .
T h i s is t h e p h e n o m e n o n o f n o n c o n g l o m e r a b i l i t y . C o n g l o m e r a b i l i t y is a p r o p e r t y
o f a p r o b a b i l i t y d i s t r i b u t i o n , a n d w a s d e f i n e d b y d e F i n e t t i ( 1 9 7 2 , p . 9 9 )
a s f o l l o w s : i f t h e c o n d i t i o n a l p r o b a b i l i t y o f a n e v e n t , g i v e n e a c h
e l e m e n t o f a p a r t i t i o n , l i e s b e t w e e n p a n d q , t h e n a l s o t h e p r o b a b i l i t y o f
t h e e v e n t l i e s b e t w e e n p a n d q . C o n g l o m e r a b i l i t y a l w a y s h o l d s f o r c o u n t a b l y
a d d i t i v e p r o b a b i l i t y d i s t r i b u t i o n s a n d c o u n t a b l e p a r t i t i o n s , b u t n e e d n o t
h o l d f o r m e r e l y f i n i t e l y a d d i t i v e d i s t r i b u t i o n s , a n d i n f a c t , a s s h o w n
r e c e n t l y i n H i l l a n d L a n e ( 1 9 8 3 ) u s i n g o n l y e l e m e n t a r y m a t h e m a t i c s a n d
v e r i f y i n g a c o n j e c t u r e o f d e F i n e t t i , c o n g l o m e r a b i l i t y a n d c o u n t a b l e a d d i t i v i t y
a r e e q u i v a l e n t f o r c o u n t a b l e s p a c e s . T h e u n i f o r m i m p r o p e r p r i o r d i s t r i b u t i o n
c a n b e g i v e n a f i n i t e l y a d d i t i v e i n t e r p r e t a t i o n , w h i c h is w h y t h e n o n c o n g l o m e r -
a b i l i t y e x h i b i t e d b y S t e i n c a n o c c u r . T h u s f o r t h e p a r t i t i o n b a s e d u p o n t h e
v a l u e o f t h e t a , w e h a v e ( 5 . 3 . 6 ) , w h i l e f o r t h e p a r t i t i o n b a s e d u p o n t h e v a l u e
o f Y , t h e i n t e r v a l s ( 5 . 3 . 5 ) h a v e p o s t e r i o r p r o b a b i l i t y a t l e a s t . 9 5 f o r a l l
p o s s i b l e s u c h Y v a l u e s , w h e n t h e u n i f o r m p r i o r d i s t r i b u t i o n f o r t h e t a i s u s e d .
T h e u n c o n d i t i o n a l p r o b a b i l i t y o f t h e i n t e r v a l h a s n o t b e e n d e f i n e d , b u t
w h a t e v e r v a l u e i t is g i v e n m u s t e x h i b i t a n o n c o n g l o m e r a b i l i t y w i t h r e s p e c t t o
o n e o r t h e o t h e r o f t h e t w o p a r t i t i o n s . In H i l l ( 1 9 8 1 ) I
g a v e s o m e g e n e r a l a r g u m e n t s a s t o w h y n o n c o n g l o m e r a b i l i t y c a n n o t b e a v o i d e d in
t h e s u b j e c t i v e B a y e s i a n f r a m e w o r k , a n d a s t o w h y I b e l i e v e t h e r e is r e a l l y y e r y
l i t t l e t h a t is o p e r a t i o n a l l y m e a n i n g f u l in t h e t y p e o f s u p e r f i c i a l l y f r i g h t e n -
i n g c a l c u l a t i o n e x h i b i t e d i n ( 5 . 3 . 6 ) . A f t e r d i s c u s s i n g t h e o t h e r e x a m p l e s , I
w i l l r e t u r n t o t h i s i s s u e , a n d s u g g e s t a n e w a r g u m e n t a s t o w h y t h e r e is n o
w a y t o d e m o n s t r a t e a n y u n d e s i r a b l e c o n s e q u e n c e s i f o n e u s e s a n i m p r o p e r p r i o r
d i s t r i b u t i o n . T h u s a l t h o u g h I a g r e e w i t h BW t h a t t h e B a y e s i a n c a n a l w a y s a v o i d
t h e t r a p b y u s i n g p r o p e r d i s t r i b u t i o n s , I a l s o l i k e t o u s e i m p r o p e r p r i o r d i s -
t r i b u t i o n s o r m e r e l y f i n i t e l y a d d i t i v e p r i o r d i s t r i b u t i o n s w h e n I t h i n k t h e y
y i e l d a s i m p l e a n d s a t i s f a c t o r y a p p r o x i m a t i o n t o m y p o s t e r i o r d i s t r i b u t i o n , a n d
d o n o t a c c e p t ( a s B W s e e m t o d o ) t h a t t h e r e a r e a n y o p e r a t i o n a l l y m e a n i n g f u l
ill c o n s e q u e n c e s t o s o u s i n g s u c h d i s t r i b u t i o n s ( e v e n f o r a l l p o s s i b l e v a l u e s
o f Y in t h e S t e i n e x a m p l e ) . A g e n e r a l t h e o r y p e r t a i n i n g t o t h e t y p e o f c o n s e -
q u e n c e s t h a t a r i s e in n o n c o n g l o m e r a b l e s i t u a t i o n s h a s b e e n f o r m u l a t e d a n d
e l e g a n t l y p r e s e n t e d b y H e a t h a n d S u d d e r t h ( H S ) i n H e a t h a n d S u d d e r t h ( 1 9 7 8 )
a n d b y L a n e a n d S u d d e r t h ( 1 9 8 4 ) , a n d a s w e s h a l l s e e l a t e r a l l o f t h e
e x a m p l e s p u r p o r t e d l y a g a i n s t t h e l i k e l i h o o d p r i n c i p l e , a r e in f a c t m e r e l y m o r e
e x a m p l e s o f t h e t y p e o f i n c o h e r e n c e d i s c u s s e d b y H S . ( Se e H S e x a m p l e 5 . 2 f o r a
v e r y s i m p l e e x a m p l e s i m i l a r t o t h a t o f S t e i n . ) It i s m y o p i n i o n , h o w e v e r , t h a t
t h e H S r e q u i r e m e n t f o r c o h e r e n c e , t o t h e e x t e n t it g o e s b e y o n d t h e d e F i n e t t i
f o r m o f c o h e r e n c e ( w h i c h o n l y r e q u i r e s a v o i d a n c e o f s u r e l o s s w i t h a f i n i t e
n u m b e r o f g a m b l e s ) , is t o o r e s t r i c t i v e , a n d a t l e a s t in t h e s p e c i a l c a s e o f
t h e M o n e t t e - F r a s e r e x a m p l e , I w i l l a r g u e t h a t t h e a p p a r e n t ill c o n s e q u e n c e s o f
v i o l a t i n g t h e H S c o n d i t i o n f o r c o h e r e n c e c a n n o t r e a l l y b e m a d e o p e r a t i o n a l .
T h e S t o n e e x a m p l e d o e s n o t d i r e c t l y p e r t a i n t o t h e l i k e l i h o o d
p r i n c i p l e , a n d h a s b e e n a n a l y s e d b y m y s e l f in H i l l ( 1 9 8 1 ) f r o m a
f i n i t e l y a d d i t i v e p o i n t o f v i e w . In a d d i t i o n t o o b s e r v i n g t h a t a f i n i t e l y
a d d i t i v e d i f f u s e u n i f o r m d i s t r i b u t i o n o n t h e " l e n g t h " o f p a t h y i e l d s t h e
s t a n d a r d c o n f i d e n c e r e s u l t , i t w a s a l s o p o i n t e d o u t t h a t i n o r d e r t o o b t a i n t h e
u n i f o r m d i s t r i b u t i o n o n t h e l o c a t i o n o f t h e t r e a s u r e t h a t S t o n e c r i t i c i z e s it
is n e c e s s a r y t o e m p l o y a d i f f u s e f i n i t e l y a d d i t i v e p r i o r d i s t r i b u t i o n w h i c h
g i v e s o d d s o f n i n e t o o n e in f a v o r o f p a t h s o f l e n g t h j + 1 v e r s u s p a t h s o f
l e n g t h j - 1 , f o r a l l j > 1 , a n d s u c h a p r i o r d i s t r i b u t i o n s e e m s r a t h e r s i l l y in
t h i s e x a m p l e . N o n e t h e l e s s , j u s t as in t h e S t e i n e x a m p l e t h e d e Fi n e t t i t h e o r y
d o e s n o t r u l e o u t s u c h p r i o r d i s t r i b u t i o n s , a n d t h e q u e s t i o n is o n c e a g a i n
w h e t h e r a s e r i o u s c a s e c a n b e m a d e a g a i n s t t h e i r u s e . It m a y b e n o t e d t h a t t h e
p o s t e r i o r o b t a i n e d w i t h t h i s p r i o r is a l s o i n c o h e r e n t in t h e s e n s e o f H e a t h a n d
S u d d e r t h .
F r a s e r in h i s d i s c u s s i o n o f my V a l e n c i a a r t i c l e Hi l l ( 1 9 8 1 )
m a i n t a i n e d t h a t t h e S t o n e e x a m p l e a l s o h a s i m p l i c a t i o n s w i t h r e g a r d t o
t h e l i k e l i h o o d p r i n c i p l e , a n d g a v e t h e e x a m p l e r e p o r t e d b y B W . T h e e x a m p l e a s
i n i t i a l l y p r e s e n t e d d i d n o t s e e m a p p r o p r i a t e t o m e , s i n c e i t r e q u i r e d t h a t
f i r s t t h e t a , t h e t r u e p a t h t o t h e t r e a s u r e , b e s e l e c t e d a s in S t o n e ' s e x a m p l e ,
n e x t t h a t t h e o b s e r v e d p a t h o f t h e S t o n e e x p e r i m e n t b e g i v e n , a n d f i n a l l y t h a t
a r a n d o m i z a t i o n b e p e r f o r m e d t h a t l e a v e s o n e w i t h t h e s a m e l i k e l i h o o d f u n c t i o n
a s b e f o r e . In t h i s s i t u a t i o n , w h e r e t h e s e c o n d e x p e r i m e n t c o n s i s t s o f t h e
f i r s t e x p e r i m e n t t o g e t h e r w i t h a n i r r e l e v a n t r a n d o m i z a t i o n , t h e l i k e l i h o o d
p r i n c i p l e f o l l o w s f r o m j u s t t h e s u f f i c i e n c y p r i n c i p l e , a n d is b a r e l y w o r t h
c o m m e n t i n g o n . H o w e v e r , t h e F r a s e r e x a m p l e c a n b e m o d i f i e d s o t h a t t h i s is
n o l o n g e r t h e c a s e , f o r e x a m p l e , o n e c a n i m a g i n e t h a t a n e w e x p e r i m e n t E * is
p e r f o r m e d a s f o l l o w s : f i r s t a p a t h z f r o m t h e o r i g i n is s e l e c t e d a c c o r d i n g t o
a p r o b a b i l i t y d i s t r i b u t i o n t h a t d e p e n d s u p o n t h e t a , in s u c h a w a y t h a t z is
e q u a l l y l i k e l y t o b e a n y o f t h e f o u r p a t h s f o r w h i c h t h e t a is a o n e b l o c k
e x t e n s i o n o r r e t r a c t i o n o f z , a n d w e o b s e r v e t h i s z . N e x t , s o m e o n e e l s e w h o
s o m e h o w o r o t h e r h a p p e n s t o k n o w t h e t r u e t h e t a , u n o b s e r v e d l y r e t r a c t s o r
e x t e n d s z b a c k ( o r f o r t h ) t o t h e t r u e t h e t a ( w h i c h t h e r e f o r e r e m a i n s t h e t r u e
p a r a m e t e r o f t h e e x p e r i m e n t ) , a n d f r o m t h e r e d o e s t h e e x p e r i m e n t w i t h t h e
F r a s e r l i k e l i h o o d f u n c t i o n a s p r e s e n t e d b y B W , w i t h z = x ( 0 ) . O n e t h e n
o b s e r v e s in t h i s l a s t e x p e r i m e n t a p a t h X . T h e l i k e l i h o o d f u n c t i o n f o r t h e t a
b a s e d u p o n t h e d a t a Z = z , X = z , in t h e e x p e r i m e n t E * , is t h e n i d e n t i c a l w i t h
t h e l i k e l i h o o d f u n c t i o n d e r i v e d f r o m t h e S t o n e e x p e r i m e n t w i t h t h e s a m e
o b s e r v e d p a t h z , a n d s o w i t h t h i s m o d i f i c a t i o n t h e F r a s e r e x a m p l e d o e s m e e t t h e
c o n d i t i o n s f o r t h e l i k e l i h o o d p r i n c i p l e t o a p p l y . If o n e a d o p t s a B a y e s i a n
p o i n t o f v i e w , t h e n a s B W a r g u e , o n e h a s p r e c i s e l y t h e s a m e a p r i o r i i n f o r m a t i o n
a b o u t t h e t a n o m a t t e r w h i c h e x p e r i m e n t is p e r f o r m e d , a n d it i s c e r t a i n l y
r e a s o n a b l e t o d r a w t h e s a m e i n f e r e n c e in e a c h e x p e r i m e n t . S u p p o s e , h o w e v e r ,
o n e i m a g i n e s t h a t i t is m e a n i n g f u l t o c o n s i d e r t h e c a s e o f n o p r i o r i n f o r m a t i o n
( w h a t e v e r t h i s m e a n s ) , s o t h a t B a y e s i a n i n f e r e n c e is n o t p o s s i b l e . It w o u l d b e
i n t e r e s t i n g t o k n o w w h a t t h e a p p r o p r i a t e n o n - B a y e s i a n i n f e r e n c e a b o u t t h e t a
w o u l d b e u n d e r E * a s o p p o s e d t o t h e S t o n e e x p e r i m e n t . W o u l d , f o r e x a m p l e , a
n o n - B a y e s i a n n o w t r e a t t h e t a a s t h o u g h i t w e r e e q u a l l y l i k e l y t o b e a n y o f t h e
f o u r p o s s i b l e p a t h s ? R a t h e r t h a n c a l l i n g i n t o q u e s t i o n t h e l i k e l i h o o d p r i n c i -
p l e i t s e e m s t o m e t h a t t h i s e x a m p l e m a y r a i s e s o m e s e r i o u s p r o b l e m s f o r n o n -
B a y e s i a n s .
N o w l e t u s t u r n t o t h e n e w e x a m p l e b y M o n e t t e a n d F r a s e r ( M F ) .
T h i s e x a m p l e d o e s n o t s e e m t o p e r t a i n d i r e c t l y t o t h e l i k e l i h o o d p r i n c i p l e ,
s i n c e t h e r e is o n l y o n e e x p e r i m e n t u n d e r d i s c u s s i o n . It d o e s , l i k e t h e o t h e r
e x a m p l e s , s u g g e s t t h a t a c c o r d i n g t o f r e q u e n t i s t s t a n d a r d s a c e r t a i n i m p r o p e r ,
o r d i f f u s e f i n i t e l y a d d i t i v e , p r i o r d i s t r i b u t i o n is u n s a t i s f a c t o r y , a n d B W ,
a s i n t h e S t e i n e x a m p l e , a r g u e t h a t f o r p r o p e r p r i o r d i s t r i b u t i o n s , a n d e v e n
f o r t h e c o n v e n t i o n a l i m p r o p e r p r i o r d i s t r i b u t i o n f o r s o m e t h i n g a k i n t o a
s c a l e p a r a m e t e r , t h e r e is n o d i f f i c u l t y . A l t h o u g h a g a i n I a g r e e w i t h B W t h a t
o r d i n a r i l y o n e n e e d o n l y c o n s i d e r q u i t e p r o p e r p r i o r d i s t r i b u t i o n s , a n d a l s o
t h a t t h e p a r t i c u l a r i m p r o p e r o r f i n i t e l y a d d i t i v e d i s t r i b u t i o n s t h a t a r e b e i n g
c a s t i g a t e d m a y b e o f n o s p e c i a l i n t e r e s t , I w o u l d n o n e t h e l e s s l i k e t o a r g u e
t h a t a s y e t v e r y l i t t l e h a s b e e n d e m o n s t r a t e d a g a i n s t t h e u s e o f s u c h p r i o r
d i s t r i b u t i o n s . M y a r g u m e n t w o u l d b e m u c h t h e s a m e in a l l e x a m p l e s , b u t w i l l
b e p r e s e n t e d h e r e i n c o n n e c t i o n w i t h t h e M F e x a m p l e , w h i c h is t h e s i m p l e s t .
W h a t h a s b e e n s h o w n is t h a t c h o i c e o f a n i m p r o p e r u n i f o r m p r i o r d i s t r i b u t i o n
( o r a f i n i t e l y a d d i t i v e d i f f u s e p r i o r d i s t r i b u t i o n ) f o r t h e t a w o u l d l e a d t o a
p o s t e r i o r d i s t r i b u t i o n , s u c h t h a t i f I w e r e t o b e t i n a c c o r d w i t h i t , I w o u l d
b e a l o s e r in t h e H e a t h - S u d d e r t h s e n s e ( t h i s is c l o s e l y r e l a t e d t o a l a c k o f
e x t e n d e d a d m i s s i b i l i t y ) . S i n c e I r e g a r d t h e f i n i t e l y a d d i t i v e u n i f o r m d i s t r i
b u t i o n a s u s e f u l f o r a p p r o x i m a t i o n s , a n d a s h a v i n g a s m u c h j u s t i f i c a t i o n a s a n y
o t h e r d i s t r i b u t i o n ( t o b e g i v e n f u l l r i g h t s , a s d e F i n e t t i s a y s ) , a n d i n a n y
c a s e I d o n ' t t h i n k t h a t i t m a t t e r s w h e t h e r t h e t a i s a k i n t o a s c a l e p a r a m e t e r ,
s o t h a t I c a n n o t t a k e r e f u g e i n t h e B W a r g u m e n t u n l e s s I d i s p e n s e e n t i r e l y
w i t h b o t h m e r e l y f i n i t e l y a d d i t i v e d i s t r i b u t i o n s a n d i m p r o p e r p r i o r s , I a m
l o a t h e t o g i v e i t u p s o e a s i l y . S o s u p p o s e I f a l l i n t o t h e t r a p a n d a g r e e t o
p o s t o d d s i n a c c o r d w i t h t h e p o s t e r i o r d i s t r i b u t i o n t h a t i s u n i f o r m o v e r t h e
t h r e e p o s s i b l e v a l u e s f o r t h e t a , g i v e n x . L e t u s s e e t o w h a t e x t e n t M F c a n
t a k e a d v a n t a g e o f s u c h f o o l i s h n e s s a s I a m w i l l i n g t o e x h i b i t . In o r d e r t o
d o s o t h e y m u s t c o n s t r u c t a r e a l w o r l d v e r s i o n o f t h e i r m a t h e m a t i c a l m o d e l . S o
f i r s t o f a l l t h e y m u s t s o m e h o w o r o t h e r p i c k a t h e t a , a n d t h e n p i c k a n x i n
a c c o r d w i t h t h e i r m o d e l . T h e H e a t h S u d d e r t h g a m b l i n g s c e n a r i o s e e m s t o b e a
c o n v e n i e n t a n d a p p r o p r i a t e w a y o f d e s c r i b i n g t h e o p e r a t i o n a l c o n s e q u e n c e s o f
m y p o t e n t i a l i n c o h e r e n c y ( e v e n f o r t h o s e w h o t h i n k t h a t t h e y d o n ' t g a m b l e ) ,
a n d i f d e s i r e d , c a n e a s i l y b e t r a n s l a t e d i n t o n o n g a m b l i n g t e r m s . T h u s s u p p o s e
t h a t t h e t a i s p i c k e d f r o m a m o n g s t t h e p o s i t i v e i n t e g e r s b y t h e m a s t e r o f c e r e
m o n i e s i n a n y w a y h e l i k e s , a n d t h e n X i s s e l e c t e d a c c o r d i n g t o t h e M F d i s t r i
b u t i o n f o r X , g i v e n t h e t a . A f t e r w e a r e a l l g i v e n t h e v a l u e x t h a t X t a k e s
o n , I t h e n u s e t h e p o s t e r i o r d i s t r i b u t i o n b a s e d u p o n t h e u n i f o r m p r i o r d i s t r i
b u t i o n f o r t h e t a t o d e t e r m i n e t h e o d d s t h a t I , a s b o o k i e , w i l l g i v e f o r t h e
v a r i o u s v a l u e s o f t h e t a . A l s o , a f t e r o b s e r v i n g x , M F a r e e n t i t l e d t o p l a c e
a n y f i n i t e n u m b e r o f b e t s c o n c e r n i n g t h e t a t h e y w i s h , a n d f i n a l l y t h e t a i s
r e v e a l e d b y t h e m a s t e r o f c e r e m o n i e s a n d a l l b e t s a r e p a i d o f f . S u p p o s e t h a t
M F b e t a d o l l a r o n t h e e v e n t t h a t t h e t a t a k e s o n t h e v a l u e , ( X ) , a n d l e t G
d e n o t e t h e f i n a l p a y o f f f r o m m e t o t h e m . G i v e n t h e t a , t h e r e i s a t l e a s t
p r o b a b i l i t y 2 / 3 t h a t ^ X ) w i l l e q u a l t h e t a , a n d s o t h e e x p e c t a t i o n o f G ,
g i v e n t h e t a , i s a t l e a s t $ 1 , f o r a l l p o s s i b l e t h e t a , a n d I a m i n c o h e r e n t i n
t h e s e n s e o f H e a t h S u d d e r t h . H o w e v e r , t o m a k e t h e t r a n s a c t i o n o p e r a t i o n a l l y
m e a n i n g f u l i t i s n e c e s s a r y t o s p e c i f y p r e c i s e l y h o w X w i l l b e r e v e a l e d , f o r
e x a m p l e , t h a t X w i l l b e e x p r e s s e d t o t h e b a s e 1 0 ( o r in a n y o t h e r s p e c i f i e d
f o r m w h a t s o e v e r ) , a n d t h a t a c e r t a i n f i n i t e t i m e l i m i t is p r e s c r i b e d d u r i n g
w h i c h t h e g a m e is t o b e p l a y e d . N o w I t h i n k t h a t a l l o f u s c o u l d c o m e t o
a g r e e m e n t t h a t g i v e n t h e c o n s t r a i n t s o f t h e w o r l d w e l i v e i n , t h e r e i s a n
u p p e r b o u n d , s a y N , t o t h e v a l u e o f X t h a t c a n b e r e p o r t e d t o us a s d a t a in
t h e p r e s c r i b e d f o r m a n d in t h e p r e s c r i b e d t i m e , f o r e x a m p l e , a n N s u c h t h a t in
t h e p r e s e n t s t a t e o f t e c h n o l o g y e v e n t h e f a s t e s t c o m p u t e r c o u l d n o t d i s p l a y a n
i n t e g e r g r e a t e r t h a n N i n t h e t i m e a l l o t t e d f o r t h e e x p e r i m e n t . ( T o b e e v e n
m o r e r e a l i s t i c , t h e s a m e is t r u e w i t h r e g a r d t o t h e t a , b u t f o r t h e p u r p o s e o f
t h e p r e s e n t a r g u m e n t w e n e e d n o t a s s u m e a n y c o n s t r a i n t o n t h e m a g n i t u d e o f
t h e t a , a n d s h a l l f o l l o w M F i n a s s u m i n g t h a t t h e m a s t e r o f c e r e m o n i e s c a n c h o o s e
a n y v a l u e w h a t s o e v e r , a n d t h e n c a n a n d d o e s s e l e c t a n X in t h e w a y t h a t M F
s p e c i f y . O f c o u r s e t h e t a , l i k e X , c a n n o t a c t u a l l y b e r e p o r t e d i t i f e x c e e d s N ,
b u t o n e m i g h t w i s h t o c o n s i d e r c a s e s w h e r e t h e m a s t e r o f c e r e m o n i e s h a s e x t r a -
o r d i n a r y p o w e r s , a n d is e n t r u s t e d t o a n n o u n c e w h o w i n s t h e g a m b l e i n s i t u a t i o n s
w h e r e X d o e s n o t e x c e e d N b u t 2 X d o e s . T h i s p o i n t s o u t t h a t t h e r e a r e i n
f a c t a v a r i e t y o f w a y s t o m a k e t h e H e a t h - S u d d e r t h s c e n a r i o o p e r a t i o n a l l y m e a n -
i n g f u l , a n d t h a t o u r a s s u m p t i o n t h a t X c a n n o t b e r e p o r t e d i f i t e x c e e d s s o m e
k n o w n N , is m e r e l y t h e m i n i m a l r e a l - w o r l d c o n s t r a i n t . T h i s g i v e s t h e p r e s e n t
a r g u m e n t g r e a t e r g e n e r a l i t y in t h a t i t m a y a p p l y e v e n w h e n t h e t a i s a r e a l -
w o r l d p h y s i c a l p a r a m e t e r f o r w h i c h t h e r e w o u l d b e n o k n o w n b o u n d s . If a b o u n d
o n t h e t a w e r e a v a i l a b l e t h e n o f c o u r s e t h e a r g u m e n t w o u l d a p p l y a l l t h e m o r e .
H o w e v e r , t h e p o i n t i s t h a t w h e t h e r o r n o t t h e r e i s s u c h a b o u n d o n t h e t a , t h e r e
is n e c e s s a r i l y a b o u n d o n t h e p o s s i b l e v a l u e o f X t h a t c a n b e r e p o r t e d . If w e
d o t a k e i n t o a c c o u n t k n o w n b o u n d s o n p o s s i b l e t h e t a , o r o n p o s s i b l e r e p o r t e d
v a l u e s o f t h e t a , t h e n t h i s w o u l d l e a d u s t o p r o p e r p r i o r d i s t r i b u t i o n s a s in
B W . H o w e v e r , it is n o t n e c e s s a r y t o i n t r o d u c e s u c h c o n s i d e r a t i o n s in t h e
p r e s e n t e x a m p l e s i n c e , a s w e s h a l l s o o n s e e , t h e b o u n d e d n e s s o f t h e X t h a t c a n
b e r e p o r t e d a l r e a d y d e s t r o y s t h e f r e q u e n t i s t i c a r g u m e n t . ) S u p p o s e t h e n t h a t
t h e t a a n d X a r e s e l e c t e d b y t h e m a s t e r o f c e r e m o n i e s in a c c o r d w i t h t h e M F
m o d e l , w i t h o u t a n y c o n s t r a i n t u p o n t h e m a g n i t u d e o f e i t h e r , a n d t h a t N i s a
k n o w n u p p e r b o u n d f o r a n y X t h a t c a n p o s s i b l y b e r e p o r t e d a s d a t a . W e d o n o t
a s s u m e t h a t N i s t h e l e a s t p o s s i b l e u p p e r b o u n d f o r a r e p o r t a b l e X , b u t m e r e l y
t h a t i t i s a n u p p e r b o u n d . ( I t i s , o f c o u r s e , d e s i r a b l e t h a t N b e n o t t o o m u c h
l a r g e r t h a n t h e l e a s t u p p e r b o u n d , b u t t h e a r g u m e n t d o e s n o t d e p e n d u p o n t h i s . )
T h u s o u r e x p e r i m e n t n o w c o n s i s t s i n p r e c i s e l y t h e M F e x p e r i m e n t , t o g e t h e r w i t h
t h e m o d e s t r e a l w o r l d c o n s t r a i n t t h a t i f X > N , t h e n n o v a l u e o f X w i l l b e
r e p o r t e d ( s i n c e i t w o u l d b e i m p o s s i b l e t o d o s o ) , a n d h e n c e t h a t a n y b e t s t h a t
d e p e n d u p o n t h e v a l u e o f X w i l l b e c a l l e d o f f . In t h i s s i t u a t i o n t h e a c t u a l
g a m b l e a s t o w h e t h e r t h e t a i s 6.(X) i s c a l l e d o f f w h e n e v e r X > N , a n d w e a r e
d e a l i n g w i t h a c o n d i t i o n a l g a m b l e i n t h e s e n s e o f d e F i n e t t i ( 1 9 7 4 , C h . 4 ) .
C o n s e q u e n t l y t h e p a y o f f f r o m m e t o M F i s n o w a s b e f o r e i f X is
a c t u a l l y r e p o r t e d , b u t a l l g a m b l e s a r e c a l l e d o f f i f X > N . ( T h e r e i s n o t h i n g
u n d e r h a n d e d h e r e w i t h r e g a r d t o t h e r e d u c t i o n t o c o n d i t i o n a l g a m b l e s : i n o r d e r
t h a t t r a n s a c t i o n s c a n o c c u r , s o t h a t t h e s c e n a r i o h a s o p e r a t i o n a l m e a n i n g , i t
is n e c e s s a r y t h a t t h e b e t s a r e c o n d i t i o n a l b e t s , g i v e n t h a t a v a l u e o f X is
r e p o r t e d , a n d h e n c e c o n d i t i o n a l u p o n t h e e v e n t t h a t t h e X s e l e c t e d i n t h e M F
e x p e r i m e n t d o e s n o t e x c e e d N . I f X > N t h e n n o X i s r e p o r t e d a n d n o g a m b l e s
c a n b e m a d e c o n c e r n i n g w h e t h e r t h e t a = i ( X ) . N o t e a l s o t h a t i t i s n o t n e c e s
s a r y t o a s s u m e t h a t X m u s t b e r e p o r t e d i f X < N , b u t m e r e l y t h a t X c a n n o t b e
r e p o r t e d i f X > N , a n d t h a t X m u s t b e r e p o r t e d i f i t is p o s s i b l e t o d o s o i n
t h e f a s h i o n p r e s c r i b e d . ) I t i s i n t e r e s t i n g n o w t o s e e w h a t b e c o m e s o f t h e
f r e q u e n t i s t a r g u m e n t t h a t s h o w e d t h a t t h e c o n d i t i o n a l e x p e c t a t i o n o f m y l o s s ,
g i v e n t h e t a , i s a t l e a s t $ 1 , f o r a l l p o s s i b l e t h e t a . I a m s t i l l u s i n g t h e s a m e
p r i o r d i s t r i b u t i o n a s b e f o r e , s o t h a t i f I a m a c t u a l l y g i v e n a v a l u e o f X
( n e c e s s a r i l y <_ N ) t h e n I p o s t t h e s a m e o d d s a s b e f o r e a g a i n s t t h e e v e n t t h a t
t h e t a = j ( x ) . I f t h e t a i s s u f f i c i e n t l y s m a l l s o t h a t X b o t h c a n a n d m u s t b e
r e p o r t e d ( h e n c e n e c e s s a r i l y t h e t a < N / 2 ) , t h e n t h e e x p e c t a t i o n o f G , g i v e n s u c h
a t h e t a , i s t h e s a m e a s b e f o r e , a t l e a s t $ 1 . O n t h e o t h e r h a n d , i f t h e t a > N / 2 ,
t h e n t h e o n l y v a l u e s o f X t h a t c a n p o s s i b l y b e r e p o r t e d a r e X = t h e t a / 2 o r
( t h e t a l ) / 2 , d e p e n d i n g o n w h e t h e r t h e t a is e v e n o r o d d . H e n c e g i v e n a v a l u e
o f t h e t a > N / 2 , a n d g i v e n t h a t t h e g a m b l e i s n o t c a l l e d o f f , i t i s c e r t a i n t h a t
t h e t a i s n o t e q u a l t o , ( X ) , a n d s o t h e c o n d i t i o n a l e x p e c t a t i o n o f G , g i v e n
s u c h a t h e t a a n d t h a t t h e g a m b l e i s n o t c a l l e d o f f , i s $ 1 , w h i l e t h e u n c o n d i
t i o n a l e x p e c t a t i o n o f G , g i v e n s u c h a t h e t a , l i e s b e t w e e n $ . 3 3 a n d $ 0 .
W h e t h e r i n g a m b l i n g t e r m s o r i n c o v e r a g e p r o b a b i l i t y t e r m s , i t i s t h u s s e e n
t h a t w h e n a r e a l w o r l d c o n s t r a i n t a s t o t h e v a l u e o f X t h a t c a n b e r e p o r t e d i s
i n c o r p o r a t e d i n t o t h e M F e x a m p l e , t h e n t h e e x a m p l e b r e a k s d o w n , a n d i n f a c t i f
a v a l u e o f X i s a c t u a l l y r e p o r t e d , t h e n t h e y e r y s a m e j(X) t h a t a p p e a r e d s o
d e s i r a b l e f r o m t h e M F p o i n t o f v i e w , b e c o m e s i m p o s s i b l e a s t h e v a l u e o f t h e t a
w h e n t h e t a > N / 2 . ( A v a r i a t i o n o f t h i s s c e n a r i o w o u l d r e q u i r e m e a l s o t o p o s t
o d d s o n t h e t a , g i v e n t h e i n f o r m a t i o n t h a t X e x c e e d s N . T h i s w o u l d r e q u i r e c a r e
i n o b t a i n i n g t h e p o s t e r i o r d i s t r i b u t i o n f o r a f i n i t e l y a d d i t i v e p r i o r d i s t r i b u
t i o n , b u t i n a n y e v e n t t h e 6 . ( X ) a r e s t i l l n o t a v a i l a b l e , a n d t h e f r e q u e n t i s t i c
a r g u m e n t s t i l l b r e a k s d o w n . )
T h e a b o v e f o r m o f a r g u m e n t s u g g e s t s w h y t h e r e n e e d n o t b e a n y t h i n g
w r o n g w i t h u s i n g t h e f i n i t e l y a d d i t i v e u n i f o r m d i s t r i b u t i o n i n c o n n e c t i o n w i t h
e x p e r i m e n t s c o n d u c t e d b y h u m a n b e i n g s , i . e . , w h e r e t h e r e p o r t a b l e o b s e r v a t i o n
X , i f n o t t h e t a i t s e l f , m u s t b e b o u n d e d , a n d o n e c a n w i t h a l i t t l e t h o u g h t
a l w a y s c h o o s e a g e n e r o u s u p p e r b o u n d . M o r e g e n e r a l l y , w h e n t h e t a i s n o t
c h o s e n b y a n y h u m a n , b u t i s a p a r a m e t e r o f t h e r e a l w o r l d , t h e n o n e m a y n o t b e
a b l e t o a r g u e f o r a n y u p p e r b o u n d f o r t h e t a , b u t i n m y o p i n i o n n e i t h e r w i l l
t h e r e b y a n y o p e r a t i o n a l l y m e a n i n g f u l s c e n a r i o i n w h i c h o n e w h o c h o o s e s a
f i n i t e l y a d d i t i v e d i s t r i b u t i o n c a n b e s h o w n t o b e i n t r o u b l e b y v i r t u e o f
f r e q u e n t i s t p r o p e r t i e s . B W s u g g e s t u s i n g p r o p e r p r i o r d i s t r i b u t i o n s f o r t h e t a
a s a w a y o f a v o i d i n g t h e a p p a r e n t f r e q u e n t i s t i c d i f f i c u l t i e s i n t h e a b o v e
e x a m p l e s . H o w e v e r , i f B W o r H e a t h S u d d e r t h w i s h t o u s e i m p r o p e r o r m e r e l y
f i n i t e l y a d d i t i v e p r i o r d i s t r i b u t i o n s , a n d i f t h e y c h o o s e t o a v o i d n o n c o n g l o m
e r a b i l i t y a n d i t s f r e q u e n t i s t i c c o n s e q u e n c e s , a s i n t h e v a r i o u s e x a m p l e s , t h e n
i t s e e m s t o m e t h a t t h e y a r e i n f a c t g o i n g t o v i o l a t e t h e l i k e l i h o o d p r i n c i p l e ,
s i n c e t h e p a r t i c u l a r i m p r o p e r o r f i n i t e l y a d d i t i v e d i s t r i b u t i o n s t h a t t h e y m u s t
r u l e o u t in o r d e r t o a v o i d n o n c o n g l o m e r a b i l i t y w i l l d e p e n d u p o n t h e f o r m o f
the experiment, just as the prior distributions that Box and Tiao recommend
depend upon the form of the experiment. (BW can avoid violating the likelihood
principle by either restri cti ng themselves to proper prior distributions, or by
using improper prior distributions only when they provide an "adequate
11
approx-
imation to the posterior distribution based upon some proper prior distribution.
But I think i t is too restri cti ve always to restri ct oneself to proper prior
distributions, and although, as mentioned earl i er, I too ordinarily take the
approximation point of view, I don't think the notion of what is an adequate
approximation should depend upon frequentistic properties.) In the Stein
example and in the Heath-Sudderth example ( 5. 2) , where according to the model
(taken l i t er al l y) the parameter and data are not discrete and the set of theta
compatible with the data is not f i ni t e, the argument I have given above must
be modified, but I think that here too, when real-world constraints are
allowed for, the frequentistic argument wi l l again break down, and I hope that
my discussion of the MF example at least suggests some of the di f f i cul t i es
involved in trying to make the frequentistic argument operational. In my
Valencia art i cl e I also suggested that as yet no serious argument for conglom-
erabi l i t y had ever been given (since that time Lane and Sudderth (1984) have
given such an argument, but I do not agree with thei r views concerning the
appropriate gambles with which to define coherency), and suggested also that
Stone's example had an i mpl i ci t assumption of conglomerability for i ts castiga-
tion of the uniform prior. (Stone (1979) replied by asserting that Hi l l is like
a prisoner condemned to death by gui l l oti ne who rejoices that the gui l l oti ne
wi l l be chosen from an i nf i ni t e collection. I replied "Yes, Mervyn, but al l your
guillotines are made of butter." At a deeper level this concerns the appro-
priate interpretation of conditional probability, whether in terms of gambles
that are called off i f the conditioning event does not occur, as in de Finetti
(1972, p. 81), or in the more usual way, but there is not space to go
into this here.) Si r Harold Jeffreys once cri ti ci zed conventional tests of
significance because they reject hypotheses that may be true on the basis of
data that have not occurred. Apparently some would also have us reject the
u s e o f i m p r o p e r p r i o r d i s t r i b u t i o n s b e c a u s e o f e x p e r i m e n t s t h a t c a n n o t b e
p e r f o r m e d .
F i n a l l y , l e t m e m e n t i o n a n i m p o r t a n t r e a l w o r l d p r o b l e m w h e r e
e x a c t l y s u c h c o n s i d e r a t i o n s a s I h a v e b e e n d i s c u s s i n g a r i s e . C o n s i d e r a
b a l a n c e d o n e w a y r a n d o m e f f e c t s a n a l y s i s o f v a r i a n c e m o d e l , w i t h I r o w s a n d J
c o l u m n s . In H i l l ( 1 9 8 0 ) I e x a m i n e d t h e c o n s e q u e n c e s o f d r a w i n g i n f e r e n c e a b o u t
t h e r a t i o o f t h e b e t w e e n t o t h e w i t h i n v a r i a n c e , , u s i n g a s d a t a o n l y t h e
r a t i o o f t h e m e a n s q u a r e b e t w e e n t o t h e m e a n s q u a r e w i t h i n . I t w a s s h o w n t h a t
t h i s c a n i n f a c t b e j u s t i f i e d b y a f u l l y B a y e s i a n a n a l y s i s , a n d i s a p p r o p r i a t e
w h e n t h e p r i o r d i s t r i b u t i o n o f t h e t w o v a r i a n c e c o m p o n e n t s i s s u c h t h a t t h e i r
r a t i o i s i n d e p e n d e n t o f t h e w i t h i n v a r i a n c e , a n d t h e o v e r a l l m e a n i s g i v e n a
d i f f u s e p r i o r d i s t r i b u t i o n . T h e p r o b l e m t h e n r e d u c e s t o o n e o f i n f e r e n c e a b o u t a
s i m p l e l o c a t i o n p a r a m e t e r , = l n ( l + J ) , b a s e d u p o n d a t a = l n ( M S B / M S W ) , a n d
w i t h t h e d i s t r i b u t i o n o f , g i v e n , b e i n g t h a t o f t h e l o g a r i t h m o f a r a n d o m
v a r i a b l e h a v i n g t h e F d i s t r i b u t i o n w i t h 11 a n d I ( J l ) d e g r e e s o f f r e e d o m .
T h e l i k e l i h o o d f u n c t i o n f o r b a s e d u p o n t h e d a t a i s t h e n t h e d e n s i t y o f t h i s
l n ( F ) d i s t r i b u t i o n , t r a n s l a t e d s o t h a t t h e m o d e i s a t ( a n d w i t h d e g r e e s o f
f r e e d o m r e v e r s e d ) , e x c e p t t h a t t h e d e n s i t y m u s t b e t r u n c a t e d f r o m b e l o w a t 0
b e c a u s e i s n o n n e g a t i v e ( i t i s c o n v e n i e n t a n d h a r m l e s s t o t h i n k o f t h e l i k e l i
h o o d f u n c t i o n a s b e i n g d e f i n e d f o r a l l , s o t h a t e v e n i f i s n e g a t i v e , t h e
m o d e i s a t , a n d t h e n t o m a k e t h e t r u n c a t i o n f r o m b e l o w a t 0 s t e m f r o m t h e
p r i o r d i s t r i b u t i o n . ) I f o n e u s e s t h e u n i f o r m p r i o r d i s t r i b u t i o n f o r , w i t h
> 0 , t h e n o n e i s i n p r e c i s e l y t h e t y p e o f s i t u a t i o n t h a t t h e S t e i n , H S ( 5 . 2 ) ,
a n d M F e x a m p l e s , d e a l w i t h . A l t h o u g h t h e r e i s n o t h i n g m a g i c a l o r m a n d a t o r y
a b o u t u s e o f t h i s p a r t i c u l a r p r i o r d i s t r i b u t i o n , a n d i n f a c t t h e r e i s u s u a l l y
a g r e a t d e a l o f p r i o r i n f o r m a t i o n a b o u t t h e r a t i o o f v a r i a n c e c o m p o n e n t s i n
s u c h p r o b l e m s , s o t h a t I w o u l d r e c o m m e n d u s e o f a p r o p e r p r i o r d i s t r i b u t i o n
f o r , a t t h e s a m e t i m e , I t h i n k a g r e a t d e a l o f i n s i g h t c a n b e o b t a i n e d f r o m
t h e i m p r o p e r u n i f o r m p r i o r o n , a n d d o n o t t h i n k i t s h o u l d b e a u t o m a t i c a l l y
r u l e d o u t m e r e l y b e c a u s e i t m a y l e a d t o b a d f r e q u e n t i s t i c r i s k p r o p e r t i e s . A s
I a r g u e i n H i l l ( 1 9 8 0 ) , t h e p o s t e r i o r e x p e c t a t i o n b a s e d u p o n t h i s i m p r o p e r p r i o r
y i e l d s , a t l e a s t i n s o m e r e s p e c t s , a m o r e p l a u s i b l e e s t i m a t o r f o r a m u l t i
v a r i a t e m e a n ( t h e r e a l i z e d r a n d o m e f f e c t s ) t h a n d o e s t h e p o s i t i v e p a r t S t e i n
e s t i m a t o r . F o r e x a m p l e , t h e p o s t e r i o r e x p e c t a t i o n c a n n o t s h r i n k a l l t h e w a y
t o t h e g r a n d m e a n o f t h e o b s e r v a t i o n s , s i n c e t h e w e i g h t g i v e n t o t h e r o w m e a n s
y .
#
d e c r e a s e s o n l y t o 2 / ( 1 + 1 ) a s t h e r a t i o o f t h e b e t w e e n t o w i t h i n m e a n
s q u a r e g o e s t o 0 , w h e r e a s o f c o u r s e t h e p o s i t i v e p a r t S t e i n e s t i m a t o r c a n g i v e
z e r o w e i g h t t o t h e r o w m e a n s , a n d t h i s i s n o t a l w a y s s e n s i b l e . T h e b e h a v i o r
o f t h e p o s t e r i o r e x p e c t a t i o n s t e m s p a r t l y f r o m t h e p a r t i c u l a r f o r m o f t h e
p r i o r d i s t r i b u t i o n f o r t h e v a r i a n c e c o m p o n e n t s ( e s p e c i a l l y t h e f a c t t h a t t h e
r a t i o o f t h e v a r i a n c e c o m p o n e n t s i s a p r i o r i i n d e p e n d e n t o f t h e w i t h i n v a r i a n c e
c o m p o n e n t ) , a n d p a r t l y f r o m t h e t r u n c a t i o n o f t h e p o s t e r i o r d i s t r i b u t i o n o f
f r o m b e l o w a t 0 , n e i t h e r o f w h i c h d o n o n B a y e s i a n s i n c o r p o r a t e i n t o t h e i r
a n a l y s i s . In m y o p i n i o n d u e r e s p e c t f o r t h e l i k e l i h o o d p r i n c i p l e , a n d p r o p e r
a l l o w a n c e f o r t h e s e a s p e c t s o f t h e p r o b l e m , a r e f a r m o r e i m p o r t a n t t h a n a n y
f r e q u e n t i s t i c a r g u m e n t s a g a i n s t t h e u s e o f i m p r o p e r p r i o r d i s t r i b u t i o n s , w h i l e
a t t h e s a m e t i m e , a s B W w o u l d p r e s u m a b l y a g r e e , a p r o p e r p r i o r d i s t r i b u t i o n
f o r t h e v a r i a n c e c o m p o n e n t s w o u l d o r d i n a r i l y b e r e a s o n a b l e , a n d g i v e t h e b e s t
o f b o t h w o r l d s .
D I S C U S S I O N O F T H E S E C O N D E D I T I O N B Y P R O F E S S O R H I L L
S i n c e t h e p u b l i c a t i o n o f t h e f i r s t e d i t i o n o f t h e m o n o g r a p h b y
B e r g e r a n d W o l p e r t , I h a v e w r i t t e n s e v e r a l a r t i c l e s p e r t a i n i n g t o t h e v a l i d i t y
o f t h e l i k e l i h o o d p r i n c i p l e , a n d t o i t s r o l e i n B a y e s i a n d a t a a n a l y s i s . I
b e l i e v e t h a t t h e e x a m p l e o f H i l l ( 1 9 8 7 a , b ) c l e a r l y s h o w s t h a t t h e o r i g i n a l
s t a t e m e n t o f t h e l i k e l i h o o d p r i n c i p l e b y B i r n b a u m i n t e r m s o f a n a b s t r a c t c o n
c e p t o f e v i d e n c e w a s f a u l t y . T h e d i f f i c u l t y i n t h e l i k e l i h o o d p r i n c i p l e i s
e a s i l y r e m e d i e d , h o w e v e r , a n d t h i s w a s d o n e i n m y s t a t e m e n t o f t h e r e s t r i c t e d
l i k e l i h o o d p r i n c i p l e i n t h o s e a r t i c l e s . In m y f o r m u l a t i o n o n e s p e a k s n o t o f
t h e e v i d e n c e i n s o m e u n d e f i n e d a b s t r a c t s e n s e , b u t r a t h e r o n l y o f t h e e v i d e n c e
a b o u t t h e v a l u e o f , a n d e x c l u d e s f r o m t h e d i s c u s s i o n a n y a s s e r t i o n a b o u t h o w
m i g h t r e l a t e t o o t h e r u n k n o w n s , w h e t h e r h y p o t h e s e s o r p a r a m e t e r s . T h u s m y
D I S C U S S I O N B Y H I L L 1 7 4 . 1
e x a m p l e c a n b e v i e w e d a s s h o w i n g t h a t t w o d i f f e r e n t e x p e r i m e n t s t h a t y i e l d
p r o p o r t i o n a l l i k e l i h o o d f u n c t i o n s f o r d o n o t n e c e s s a r i l y p r o v i d e t h e s a m e
e v i d e n c e a b o u t , s i n c e w e c a n l e a r n , f o r e x a m p l e , t h a t h a s a d i f f e r e n t
' c o l o r ' i n t h e t w o e x p e r i m e n t s . T h e c o l o r m i g h t b e a n i m p o r t a n t p a r t o f t h e
o v e r a l l e v i d e n c e a b o u t . O f c o u r s e t h e c o l o r c a n b e i n c l u d e d i n t h e p a r a m e t e r ,
b u t t h e l i k e l i h o o d p r i n c i p l e , a s u s u a l l y f o r m u l a t e d , d o e s n o t r e q u i r e o n e t o d o
s o . It i s h o p e d t h a t o n c e t h i s p o i n t i s u n d e r s t o o d , o t h e r s w i l l , l i k e m y s e l f ,
b e c o m e e v e n s t r o n g e r s u p p o r t e r s o f t h e e s s e n t i a l p a r t o f t h e l i k e l i h o o d
p r i n c i p l e .
T h e b a s i c p o i n t o f m y e x a m p l e i s r e l a t e d t o f u n d a m e n t a l q u e s t i o n s
t h a t a r i s e i n t h e o r i e s o f c a u s a l i t y , f o r e x a m p l e , c o n c e r n i n g d e t e r m i n i s m a n d
t h e p o s s i b i l i t y o f i n d e p e n d e n c e i n t h e r e a l w o r l d . S u c h q u e s t i o n s a r i s e i n
c r i t i c a l d i s c u s s i o n s o f q u a n t u m m e c h a n i c s a n d r e l a t i v i t y t h e o r y , f o r e x a m p l e ,
in c o n n e c t i o n w i t h B e l l ' s i n e q u a l i t y , a s w e l l a s i n p h i l o s o p h y .
In H i l l ( 1 9 8 5 8 6 , p . 2 2 3 ) I h a v e g i v e n a n a c c o u n t o f h o w t h e l i k e
l i h o o d p r i n c i p l e m u s t b e f u r t h e r m o d i f i e d t o d e a l w i t h B a y e s i a n d a t a a n a l y s i s ,
w h e r e t h r o u g h e x p l o r a t i o n o f t h e d a t a , o n e m a y m o d i f y t h e o r i g i n a l m o d e l . T h e
s a m e a r t i c l e , p . 2 0 2 f , a r g u e s t h a t e v e n a p a r t f r o m i n a d m i s s i b i l i t y , i n c o h e r e n c e ,
a n d t h e f a i l u r e t o u t i l i z e a v a i l a b l e i n f o r m a t i o n , t h e f r e q u e n t i s t a p p r o a c h
b r e a k s d o w n c o m p l e t e l y i n c o n n e c t i o n w i t h s u c h d a t a a n a l y s i s , s i n c e a l l f r e
q u e n t i s t i c a s s e r t i o n s m u s t b e c o n d i t i o n a l n o t o n l y u p o n t h e d i a g n o s t i c s u s e d ,
b u t t h e i r o r d e r , a n d e v e n t h e t h o u g h t s t h a t c r o s s o n e ' s m i n d . S u c h c o n d i t i o n a l
p r o b a b i l i t i e s a r e p l a i n l y b o t h u n k n o w n a n d u n k n o w a b l e . F i n a l l y , H i l l ( 1 9 8 8 )
g i v e s a v e r y s h o r t , a n d p a r t l y n e w , p r o o f o f t h e s t o p p i n g r u l e p r i n c i p l e , i . e . ,
t h a t t h e s t o p p i n g r u l e i s i r r e l e v a n t f o r i n f e r e n t i a l a n d d e c i s i o n m a k i n g p u r
p o s e s , o r t h a t " s e q u e n t i a l a n a l y s i s i s a h o a x , " a s c o n c l u d e d b y A n s c o m b e ( 1 9 6 3 ,
p . 3 8 1 ) . H e r e t h e p r o o f d o e s n o t d e p e n d u p o n t h e l i k e l i h o o d p r i n c i p l e , o r
e v e n t h e r e s t r i c t e d l i k e l i h o o d p r i n c i p l e . I n s t e a d , i t i s s h o w n t h a t o n a p o s t
d a t a b a s i s , i . e . , g i v e n t h e r e a l i z e d d a t a , s e q u e n t i a l a n a l y s t s p u r p o r t t o
e x t r a c t i n f o r m a t i o n o v e r a n d a b o v e t h a t f o l l o w i n g f r o m t h e c o r r e s p o n d i n g f i x e d
s a m p l e s i z e e x p e r i m e n t , f r o m a l o g i c a l l y c e r t a i n e v e n t . In t h i s a r t i c l e t h e
i m p o r t a n t d i s t i n c t i o n b e t w e e n t h e p r e - d a t a a n d p o s t - d a t a c o n s i d e r a t i o n s i s
e m p h a s i z e d . O n c e o n e i s g i v e n t h e d a t a , t h e p r i m a r y a i m m u s t b e t o m a k e
i n t e l l i g e n t a n d r a t i o n a l d e c i s i o n s , f o r w h i c h t h e B a y e s i a n a p p r o a c h s e e m s
q u i t e w e l l s u i t e d . O f c o u r s e s e q u e n t i a l d e s i g n n e e d n o t n e c e s s a r i l y b e a h o a x ,
b u t it a p p e a r s t h a t n o t v e r y m u c h is k n o w n a b o u t t h i s p o t e n t i a l l y i m p o r t a n t
s u b j e c t , p e r h a p s b e c a u s e o f t h e c o n f u s i o n b e t w e e n p r e - d a t a a n d p o s t - d a t a c o n s i d -
e r a t i o n s , a s d i s c u s s e d i n Hi l l ( 1 9 8 8 ) .
T h e l i k e l i h o o d p r i n c i p l e is o f t e n m i s t a k e n l y a s s u m e d t o b e l a r g e l y
e q u i v a l e n t t o t h e B a y e s i a n a p p r o a c h . T h e l i k e l i h o o d p r i n c i p l e , a s p r o p o s e d b y
B i r n b a u m , in t e r m s o f a n a b s t r a c t a n d e m p t y c o n c e p t o f e v i d e n c e , w a s in f a c t
t h e l a s t g a s p ( i n t e l l e c t u a l l y s p e a k i n g ) o f t h e t h e o r y o f c l a s s i c a l s t a t i s t i c s ,
w i t h i t s n a i v e p r e t e n c e a t o b j e c t i v i t y . I n d e e d , B i r n b a u m ( 1 9 6 2 , p . 2 7 7 ) q u o t e s
J i m m i e S a v a g e a s f o l l o w s . " R e j e c t i n g b o t h n e c e s s a r y a n d p e r s o n a l i s t i c v i e w s o f
p r o b a b i l i t y l e f t s t a t i s t i c i a n s n o c h o i c e b u t t o w o r k a s b e s t t h e y c o u l d w i t h
f r e q u e n t i s t v i e w s . . . T h e f r e q u e n t i s t i s r e q u i r e d , t h e r e f o r e , t o s e e k a c o n c e p t
o f e v i d e n c e , a n d o f r e a c t i o n t o e v i d e n c e , d i f f e r e n t f r o m t h a t o f t h e p r i m i t i v e ,
o r n a t u r a l , c o n c e p t t h a t is t a n t a m o u n t t o a p p l i c a t i o n o f B a y e s
1
t h e o r e m . "
" S t a t i s t i c a l t h e o r y h a s b e e n d o m i n a t e d b y t h e p r o b l e m t h u s c r e a t e d ,
a n d i t s m o s t p r o f o u n d a n d i n g e n i o u s e f f o r t s h a v e g o n e i n t o t h e s e a r c h f o r n e w
m e a n i n g s f o r t h e c o n c e p t s o f i n d u c t i v e i n f e r e n c e a n d i n d u c t i v e b e h a v i o r .
O t h e r p a r t s o f t h i s l e c t u r e w i l l a t l e a s t s u g g e s t c o n c r e t e l y h o w t h e s e e f f o r t s
h a v e f a i l e d , o r c o m e t o a s t a l e m a t e . F o r t h e m o m e n t , s u f f i c e i t t o s a y t h a t a
p r o b l e m w h i c h a f t e r s o m a n y y e a r s s t i l l r e s i s t s s o l u t i o n i s s u s p e c t o f b e i n g ill
f o r m u l a t e d , e s p e c i a l l y s i n c e t h i s is a p r o b l e m o f c o n c e p t u a l i z a t i o n , n o t a
t e c h n i c a l m a t h e m a t i c a l p r o b l e m l i k e F e r m a t ' s l a s t t h e o r e m o r t h e f o u r - c o l o r
p r o b l e m . "
B i r n b a u m t h e n s t a t e s t h a t " T h e p r e s e n t p a p e r i s c o n c e r n e d p r i m a r i l y
w i t h a p p r o a c h e s t o i n f o r m a t i v e i n f e r e n c e w h i c h d o n o t d e p e n d u p o n t h e B a y e s i a n
p r i n c i p l e o f i n v e r s e p r o b a b i l i t y . " It w o u l d t h e r e f o r e a p p e a r t h a t B i r n b a u m
r e g a r d e d h i s a p p r o a c h t o e v i d e n c e a s m e e t i n g t h e o b j e c t i o n s t h a t S a v a g e a n d
o t h e r s h a d r a i s e d . H o w e v e r , j u s t a s t h e M i c h e l s o n - M o r e l y e x p e r i m e n t s p e l t t h e
D I S C U S S I O N B Y H I L L 1 7 4 . 3
d e a t h k n e l l f o r c l a s s i c a l p h y s i c s ( w h i c h w a s a t l e a s t a h i g h l y s u c c e s s f u l a n d
u s e f u l s u b j e c t ) , o n e m u s t w o n d e r w h a t i s l e f t o f c l a s s i c a l s t a t i s t i c s , w i t h o u t
e v e n B i r n b a u m ' s l i k e l i h o o d p r i n c i p l e t o s u s t a i n i t . A l l t h a t a p p e a r s t o b e
l e f t i s t h e r e s t r i c t e d l i k e l i h o o d p r i n c i p l e , w h i c h i s i m p l i e d b y t h e B a y e s i a n
a p p r o a c h , a n d i s s o m e w h a t m o r e g e n e r a l t h a n t h e B a y e s i a n a p p r o a c h , s i n c e it
a l l o w s f o r v e r s i o n s o f B a y e s i a n d a t a a n a l y s i s s u c h a s in Hi l l ( 1 9 8 8 ) . I k n o w
o f n o w a y t o d e m o n s t r a t e e v e n t h e r e s t r i c t e d l i k e l i h o o d p r i n c i p l e , h o w e v e r ,
o t h e r t h a n t h r o u g h t h e B a y e s i a n a p p r o a c h .
I t h i n k t h a t n o w a d a y s it w i l l b e r e a d i l y u n d e r s t o o d t h a t t h e p r e -
t e n c e a t o b j e c t i v i t y in c l a s s i c a l s t a t i s t i c s w a s e q u i v a l e n t t o t a k i n g a p a r t i c -
u l a r s u b j e c t i v i s t i c B a y e s i a n v i e w , t h a t b a s e d u p o n d i f f u s e p r i o r d i s t r i b u t i o n s ,
a n d b y f i a t d e c l a r i n g t h a t t h i s c o n s t i t u t e s o b j e c t i v i t y . S u c h p r i o r d i s t r i b u -
t i o n s p l a y a n i m p o r t a n t r o l e in B a y e s i a n s t a t i s t i c s , v i a t h e s t a b l e e s t i m a t i o n
a r g u m e n t o f J i m m i e S a v a g e , b u t d o n o t a c q u i r e a n y m a g i c a l s t a t u s i n t h e B a y e s -
i a n t h e o r y .
T h e n a t u r e o f " o b j e c t i v i t y " w a s n e v e r s e r i o u s l y d i s c u s s e d in c l a s s -
i c a l s t a t i s t i c s , d e s p i t e t h e f a c t t h a t t h i s w a s a n d i s a n o t o r i o u s l y d i f f i c u l t
q u e s t i o n i n p h i l o s o p h y . E v e n in s t a t i s t i c s , n u m e r o u s e x a m p l e s e x i s t s h o w i n g
t h a t t h i s p r e t e n c e c a n n o t b e m a d e , w i t h o u t l e a d i n g t o a b s u r d i t i e s . T h e r e a r e
m a n y e x a m p l e s in w h i c h t h e r e a l i z e d l i k e l i h o o d f u n c t i o n i s n e a r l y f l a t , n o
m a t t e r w h a t t h e p r e - d a t a e x p e c t e d i n f o r m a t i o n m a y h a v e b e e n . T h i s o c c u r s , f o r
e x a m p l e , i n i n f e r e n c e a b o u t v a r i a n c e c o m p o n e n t s w h e n t h e c l a s s i c a l u n b i a s e d
e s t i m a t o r o f t h e b e t w e e n v a r i a n c e c o m p o n e n t i s n e g a t i v e , a s in Hi l l ( 1 9 6 5 ,
1 9 6 7 ) . A m o r e s o p h i s t i c a t e d e x a m p l e o f t h e n e e d f o r a s u b j e c t i v e v i e w o c c u r s
in d e c i d i n g w h e t h e r a p a r t i c u l a r o b s e r v a t i o n i s a n " o u t l i e r , " a s in Hi l l ( 1 9 7 4 b ,
S e c t i o n 4 ) a n d Hi l l ( 1 9 8 8 , S e c t i o n 3 ) . W h a t t h e s o - c a l l e d o b j e c t i v i s t s d o , a s
J a c k G o o d s a y s , i s t o S U T C ( s w e e p t h e s u b j e c t i v e a s p e c t s u n d e r t h e c a r p e t ) .
P r o b a b i l i t y a n d s t a t i s t i c s , a s r e l a t e d t o t h e r e a l w o r l d , a r e f u n d a m e n t a l l y
s u b j e c t i v e o r p e r s o n a l i s t i c . In c e r t a i n s i t u a t i o n s , h o w e v e r , o n e m a y o b t a i n
p r a c t i c a l o b j e c t i v i t y b y m e a n s o f a c o n s e n s u s a s t o a p p r o p r i a t e p r i o r d i s t r i -
b u t i o n s a n d m o d e l s . S e e Hi l l ( 1 9 8 5 - 8 6 , 1 9 8 8 ) . A l s o , s o m e t i m e s c e r t a i n
" o b j e c t i v i s t i c " m e t h o d s , s u c h a s t h e f i d u c i a l a p p r o a c h , c a n b e j u s t i f i e d
B a y e s i a n l y , a s f o r e x a m p l e w i t h A ( n ) in B a y e s i a n n o n p a r a m e t r i c s t a t i s t i c s ,
Hi l l ( 1 9 8 7 c ) . F i n a l l y , b y a d e l i c i o u s i r o n y , i t a l s o t u r n s o u t t h a t t h e f e w
i m p o r t a n t o b j e c t i v e c r i t e r i a t h a t f r e q u e n t i s t s h a v e r e c o m m e n d e d , s u c h a s
a d m i s s i b i l i t y , e x t e n d e d a d m i s s i b i l i t y , e t c . , l e a d i n e v i t a b l y b a c k t o t h e
B a y e s i a n a p p r o a c h .
T h e d i s t i n g u i s h e d p h i l o s o p h e r a n d p s y c h o l o g i s t , W i l l i a m J a m e s
( 1 8 9 6 , p . 9 7 ) p u t s it q u i t e w e l l : " O b j e c t i v e e v i d e n c e a n d c e r t i t u d e a r e d o u b t -
l e s s y e r y f i n e i d e a l s t o p l a y w i t h , b u t w h e r e o n t h i s m o o n l i t a n d d r e a m - v i s i t e d
p l a n e t a r e t h e y t o b e f o u n d ? I a m , t h e r e f o r e , m y s e l f a c o m p l e t e e m p i r i c i s t s o
f a r a s m y t h e o r y o f h u m a n k n o w l e d g e g o e s . I l i v e , t o b e s u r e , b y t h e p r a c t i c a l
f a i t h t h a t w e m u s t g o o n e x p e r i e n c i n g a n d t h i n k i n g o v e r o u r e x p e r i e n c e , f o r
o n l y t h u s c a n o u r o p i n i o n s g r o w m o r e t r u e ; b u t t o h o l d a n y o n e o f t h e m - I
a b s o l u t e l y d o n o t c a r e w h i c h - a s i f it n e v e r c o u l d b e r e i n t e r p r e t a b l e o r
c o r r i g i b l e , I b e l i e v e t o b e a t r e m e n d o u s l y m i s t a k e n a t t i t u d e , a n d I t h i n k t h a t
t h e w h o l e h i s t o r y o f p h i l o s o p h y w i l l b e a r m e o u t . "
J a m e s ' s e l o q u e n t s t a t e m e n t c a n s e r v e a s a p r e a m b l e t o t h e t h e o r y
a n d p r a c t i c e o f B a y e s i a n d a t a a n a l y s i s a n d d e c i s i o n - m a k i n g , w h i c h is a s y n t h e -
s i s o f t h e e m p i r i c i s m - p r a g m a t i s m o f J o h n L o c k e , D a v i d H u m e , C h a r l e s P e i r c e ,
a n d W i l l i a m J a m e s , w i t h t h e r a t i o n a l i s m o f P l a t o , D e s c a r t e s , K a n t , a n d o t h e r s ,
a n d t o w h i c h I b e l i e v e t h a t t h e n e x t c e n t u r y w i l l b e d e v o t e d .
D I S C U S S I O N BY P R O F E S S O R D A V I D A, L A N E
( Universit y of Minnesota)
Berger and Wolpert have done the st a t i st i c s community a service by
ca l l i ng our at t e nt i on once again to the like lihood pr i nci pl e (LP) and i t s
implicat ions. They repeat Birnbaum's(1962a) message, already admirably
r ecapit ulat ed by Basu ( 197 5 ) and Dawid ( 1 9 7 7 ) : if you work wit hin the
classical ( X, , { P
Q
} ) paradigm, you want to make inferences about "t rue "

o n the b a s i s o f " o b s er v ed x," a n d yo u w i s h to r es p ec t c er ta i n fun d a m en ta l
p r i n c i p l es o f i n fer en c e (fo r exa m p l e, the s uffi c i en c y a n d w ea k c o n d i ti o n a l i ty
p r i n c i p l es ) , t h e n yo ur i n fer en c e ha d b etter d ep en d up o n the o b s er v a ti o n x
thr o ug h the l i k el i ho o d fun c ti o n tha t x i n d uc es o n . In p a r ti c ul a r , yo u m us t
a c c ep t the i m p l i c a ti o n s o f s o m e o ther p r i n c i p l es tha t m a n y s ta ti s ti c i a n s r eg a r d
a s fa l s e, n ev er m i n d fun d a m en ta l , l i k e the s to p p i n g ti m e a n d c en s o r i n g
p r i n c i p l es .
T her e a r e s ev er a l b a i l o ut o p ti o n s fo r s ta ti s ti c i a n s w ho c ho o s e
n ei ther to fo l l o w the LP to ful l y c o n d i ti o n a l a n a l ys i s n o r to r a i s e a d ho c k er y
to a s c i en ti fi c p r i n c i p l e. T hey c a n r ej ec t the (X, , {P }) p a r a d i g m b y
requiring e i t he r more st ruct ure (as do st r uc t ur a l i st s, pivot eers, and, perhaps,

some "object ive" Bayesians) or less (as do defenders of a l t e r na t i ve f r e e
signif icance t est s and, more dr a st i c a l l y, explorat ory data anal ysi s) ; or they
can modify the fundamental pre principles so t hat the LP and the object ionable
post principles f a i l t o be derivable from them, as did Durbin ( 197 0 ) and
Kalbf leisch ( 1 9 7 5 ) ; or they can claim t hat ot her, more fundamental, pr i nci pl e s,
l i ke the Confidence Pr i nci pl e , conf l i ct wit h the LP, making an ideological
choice among competing pr inciples necessary.
175
Since Bayesian pr act ice is consist ent wit h t he LP, Bayesians have
no need t o r e f ut e Birnbaum's work. Indeed, t o Berger and Wolpert , the LP is a
trump card in t he Bayesian salesman's hand. They argue, as did Basu ( 1 9 7 5 ) ,
t hat only Bayesian ideas permit t he LP t o be properly implemented and t hat
Bayesian considerat ions unravel t he "counterexamples" t o the LP produced by
Armitage, St e i n, Fraser and ot hers.
But even t o Bayesians, consistency wit h Bayesian ideas should be
no guarantee of f oundat ional cogency. For example, t he f act t hat ( e sse nt i a l l y)
admissible decision rules are Bayes does not recommend Wald's f ormulat ion of
decision theory t o most Bayesians. So t he question ar i se s: should Bayesians
promote Birnbaum's f ormulat ion and de r ivat ion of the LP as a cornerstone of
t he foundations of st a t i st i c s? I t hink not , f or two reasons. Fi r st , t he LP
i s embedded in a paradigm which is not di r e ct l y applicable t o many, i f not
most, of t he important real problems of st a t i st i c a l inf erence. Because of the
ambiguity and l i mi t a t i ons of t hi s paradigm, t he proof of t he LP is not
compelling. Second, t he LP ignores what I regard as the fundamental t enet of
Bayesianit y: t he purpose of an inf erence is t o quant if y uncert aint y. When
t hi s t enet i s properly taken int o account, f oundat ional arguments can be
adduced t hat lead di r e c t l y t o Bayesian methods.
The next sect ion elaborat es t he f i r s t of these reasons in some
d e t a i l . For a development of t he second, see Lane ( 198 1) and Lane and
Sudderth ( 1 9 8 4 ) .
( X, , { P
} ) and t he LP
I shall discuss t hree problems wit h t he LP. The f i r s t r e lat e s t o
t he meaning, the second t o t he adequacy, and t he t hi r d t o t he relevance of the
( Xj > { P
Q
} ) paradigm. Both t he f i r s t and t he second of these problems ca l l
the de r ivat ion of t he LP int o quest ion.
1 . What do the elements of represent? This question is import ant , since
t he proof of t he LP requires us. t o consider t he mixture of two di f f e r e nt
experiments wit h "t he same . " There at le ast t hree possible i nt e r pr e t at i ons
D I S C U S S I O N BY L A N E 1 7 7
o f the el em en ts o f :
a ) i s the d i s tr i b uti o n P.;
b ) is a n a b s tr a c t s et a n d m er el y i n d exes the d i s tr i b uti o n P
n
;
c ) is a p o s s i b l e v a l ue fo r s o m e " r ea l " p hys i c a l p a r a m eter , a n d P
is to
b e r eg a r d ed a s the d i s tr i b uti o n o f the r a n d o m q ua n ti ty X s ho ul d b e the

tr ue v a l ue o f tha t p a r a m eter .
I n ter p r eta ti o n s a ) a n d b ) a r e m a them a ti c a l l y p r ec i s e. T hey a r e
d efi n ed in ter m s o f the a s s um ed m o d el a n d do n o t r efer to the p hys i c a l
r ea l i ty tha t m o d el is i n ten d ed to r ep r es en t.
I n ter p r eta ti o n c ) ha s an en ti r el y d i ffer en t c ha r a c ter a n d r a i s es
d i ffi c ul t p hi l o s o p hi c a l i s s ues . W hen a n d i n w ha t s en s e d o " r ea l " p hys i c a l
p a r a m eter s exi s t? If I o p t fo r i n ter p r eta ti o n c ) , m us t I b el i ev e tha t a c o i n
ha s a p r o p en s i ty to c o m e up hea d s x 1 0 0 % o f the ti m e in an (i n fi n i tel y) l o n g
s er i es o f r ep ea ted fl i p s ? I a m i n c l i n ed to b el i ev e tha t ther e m a y b e " r ea l "
p hys i c a l p a r a m eter s in m ea s ur em en t er r o r p r o b l em s , a l tho ug h ev en her e a s tr i c t
o p er a ti o n a l 1 s t c o n s tr uc ti o n l ea d s to i n ter p r eta ti o n a ) r a ther tha n c ) fo r the
p a r a m eter : the m ea s ur i n g p r o c es s , en c o d ed a s P., d efi n es the q ua n ti ty
m ea s ur ed . In few o ther p r o b l em s to w hi c h s ta ti s ti c a l i n fer en c e is a p p l i ed
a r e ther e m o d el fr ee p hys i c a l q ua n ti ti es s ta n d i n g b ehi n d ea c h m o d el p a r a m eter .
T o d ec i d e w hether o r n o t yo u a g r ee, thi n k a b o ut yo ur l a s t r eg r es s i o n o r ti m e
s er i es a n a l ys i s .
Bo th Ber g er a n d W o l p er t (p p . 4 2 3 ) a n d D a w i d ( 9 7 7 , p . 2 5 2 ) s eem
to fa v o r i n ter p r eta ti o n c ) . F o r exa m p l e, Ber g er a n d W o l p er t s a y tha t the LP
a p p l i es o n l y w hen the el em en ts o f the tw o p a r a m eter s ets a r e " the s a m e
p a r a m eter , i .e. a r e p hys i c a l l y o r c o n c ep tua l l y the s a m e q ua n ti ty."
U n fo r tun a tel y, they n eg l ec t to tel l us ho w w e a r e to d ec i d e w hen tw o d i ffer en t
exp er i m en ts m ea s ur e the s a m e q ua n ti ty o r ho w to d ea l w i th m o d el p a r a m eter s
tha t l a c k a n y n a tur a l i n ter p r eta ti o n in ter m s o f p hys i c a l q ua n ti ti es . M o r e
o v er , in v i r tua l l y a l l o f thei r exa m p l es , the s et is un i n ter p r eted a n d
m er el y s er v es to i n d ex the s et o f d i s tr i b uti o n s {P
0
}, w hi c h s ug g es ts tha t i n
thes e c a s es they a r e thi n k i n g a b o ut in the s en s e o f i n ter p r eta ti o n b ) .
It i s ha r d to ta k e the LP s er i o us l y a s a fo un d a ti o n a i n s tr um en t i f w e m us t
a l w a ys i n ter p r et the el em en ts o f a s " r ea l " p hys i c a l q ua n ti ti es , un l es s w e
a r e g i v en s o m e g ui d a n c e o n w ha t c o n s ti tutes r ea l i ty a n d ho w r ea l i ty is ti ed
to m a them a ti c s b y the m o d el s w e s el ec t.
It m a tter s w hi c h o f the thr ee i n ter p r eta ti o n s w e g i v e to the
el em en ts o f . T hey l ea d to w e r y d i ffer en t c o n c l us i o n s a b o ut the v a l i d i ty o f
the d er i v a ti o n o f the L P . I n ter p r eta ti o n a ) g i v es n o s c o p e fo r the m i xtur e
p r i n c i p l e: o n l y exp er i m en ts w ho s e s a m p l i n g d i s tr i b uti o n s a r e i d en ti c a l s ha r e
" the s a m e . " A S s uc h, the LP i s r ed uc ed to the s uffi c i en c y p r i n c i p l e a n d ,
fo r exa m p l e, the s to p p i n g ti m e p r i n c i p l e d o es n o t fo l l o w fr o m the L P .
I n ter p r eta ti o n b ) , o n the o ther ha n d , g i v es tr em en d o us s c o p e fo r
m i xi n g . A n y tw o exp er i m en ts w i th the s a m e i n d ex s et c a n b e m i xed . C o n s eq uen t
l y, i f ther e a r e a p a i r o f o b s er v a ti o n s , o n e fr o m ea c h exp er i m en t, tha t yi el d
the s a m e l i k el i ho o d fun c ti o n o n the i n d ex s et , the LP then d ec l a r es tha t the
" ev i d en c e" o r " i n fer en c e" d er i v ed fr o m the tw o exp er i m en ts w i th thes e tw o
o b s er v a ti o n s m us t b e i d en ti c a l . T hi s is a s ta r tl i n g l y un Ba yes i a n c o n c l us i o n .
F o r exa m p l e, m us t m y p r ed i c ti v e i n fer en c e fo r the n ext o utc o m e in a n y s eq uen c e
o f Ber n o ul l i tr i a l s in w hi c h I ha v e s o fa r o b ta i n ed thr ee s uc c es s es a n d o n e
fa i l ur e b e the s a m e? But w ha t in the m a them a ti c s o f the LP p r o o f p r ec l ud es
i n ter p r eti n g p ur el y a s a n i n d ex s et a n d s o d er i v i n g a v er s i o n o f the LP
tha t c o n fl i c ts w i th Ba yes i a n p r a c ti c e?
T he fo un d a ti o n a l s ta tus o f the LP c a n n o t b e d eter m i n ed un ti l is
i n ter p r eted . D ep en d i n g o n w hether o n e a d o p ts i n ter p r eta ti o n a ) , b ) o r c ) , the
LP is d ev o i d o f i n ter es ti n g c o n s eq uen c es , w r o n g , o r s ev er el y a n d a m b i g uo us l y
r es tr i c ted in i ts d o m a i n o f a p p l i c a b i l i ty.
2 . T he p r o o f o f the LP is c o n v i n c i n g o n l y in s o fa r a s the s uffi c i en c y a n d
w ea k c o n d i ti o n a l i ty p r i n c i p l es a r e i n tui ti v el y c o m p el l i n g . W hi l e Ba yes i a n
p r a c ti c e r es p ec ts b o th p r i n c i p l es , o n l y w ea k c o n d i ti o n a l i ty s eem s un a r g ua b l e
o n i ts fa c e. I s ha r e I. J . G o o d ' s r ea c ti o n to the s uffi c i en c y p r i n c i p l e, a s
r ep o r ted i n hi s d i s c us s i o n o f Bi r n b a um (1 9 6 2 a ) . D es p i te F i s her ' s g i ft fo r
D I S C U S S I O N BY L A N E 1 7 9
s ug g es ti v e n a m es (w ha t m o r e c o ul d yo u p o s s i b l y n eed tha n s o m ethi n g tha t i s
s uffi c i en t? ) , the fa c t tha t the d i s tr i b uti o n o f X g i v en the v a l ue o f a
s ta ti s ti c T i s fr ee d o es n o t i m m ed i a tel y i m p el m e to b a s e m y i n fer en c e o n l y
o n the v a l ue o f T .
S up p o s e, tho ug h, tha t the o b s er v a ti o n x is g en er a ted b y fi r s t
g en er a ti n g a v a l ue fo r T a c c o r d i n g to a d i s tr i b uti o n i n d exed b y s o m e el em en t
o f the p a r a m eter s et a n d then a n extr a n eo us r a n d o m i z a ti o n m ec ha n i s m is us ed
to p i c k a n x o n the o r b i t o f the o b s er v ed v a l ue o f T . In s uc h a c a s e, it is
c l ea r tha t T is s uffi c i en t a n d tha t i n fer en c e a b o ut s ho ul d b e b a s ed o n l y o n
T . (T he s uffi c i en t s ta ti s ti c tha t a p p ea r s in the d er i v a ti o n o f the LP d o es
n o t b ea r thi s p o s tr a n d o m i z a ti o n r el a ti o n to the o b s er v a ti o n x.)
N o w , fo r a n y s uffi c i en t s ta ti s ti c T d efi n ed o n a s ta ti s ti c a l
m o d el (X, , { P
a
} )
9
ther e is n o w a y to tel l fr o m the i n fo r m a ti o n en c o d ed in
(X> , { P Q } ) w hether the o b s er v a ti o n x is o r is n o t g en er a ted fr o m T b y
p o s tr a n d o m i z a ti o n . S o , i f yo u d o n o t fi n d the s uffi c i en c y p r i n c i p l e c o m p el
l i n g exc ep t in the p o s tr a n d o m i z a ti o n c a s e, yo u m us t a g r ee w i th Ba r n a r d a n d
F r a s er tha t n o t en o ug h i n fo r m a ti o n is en c o d ed in (X, , {P
}) up o n w hi c h to

b a s e a g en er a l p r i n c i p l e o f i n fer en c e. A n d I b el i ev e tha t thi s c o n c l us i o n
i s c o r r ec t. A fter a l l , the i n fo r m a ti o n in (X, , {P
Q
}) s a ys n o thi n g a b o ut ho w
the m o d el r ep r es en ts r ea l i ty, a n d it is ha r d to s ee ho w a p r i n c i p l e o f
i n fer en c e c a n d i s r eg a r d the d eta i l s o f thi s r ep r es en ta ti o n . T ho ug h w e us e
m o d el s to g ui d e the w a y w e fo r m ul a te i n fer en c es , the i n fer en c es them s el v es
ha v e v a l ue to us o n l y i f they yi el d us eful s ta tem en ts a b o ut the w o r l d .
3. E v en tho ug h " i n fer en c e" is un d efi n ed in the LP fo r m ul a ti o n , the v a l i d i ty
o f the LP s eem s to d ep en d o n tw o p r em i s es a b o ut the n a tur e o f i n fer en c e in
the (X, , { P
p a r a d i g m :
a ) T he p ur p o s e o f i n fer en c e is to m a k e s o m e s ta tem en t a b o ut the " tr ue" v a l ue
o f a n un o b s er v a b l e p a r a m eter o n the b a s i s o f a n o b s er v ed q ua n ti ty x;
b ) exi s ts i n d ep en d en tl y o f the " exp er i m en t" E tha t p r o d uc es x, a n d i n fo r m a
ti o n a b o ut c a n b e s ep a r a ted i n to tw o c o m p o n en ts , o n e d er i v i n g j us t fr o m
E (to w hi c h the LP r efer s ) a n d " o ther i n fo r m a ti o n " p r es um a b l y p r eexi s ti n g
E.
I believe t hat these premises are r ar e l y t rue in real sit uat ions to which
st a t i st i c a l inf erence is applied. I f I am r i ght , the scope of the LP as a
f oundat ional instrument is narrow.
Except f or measurement e r r or problems, the real aim of inf erence
is usually t o generate a predict ion about the value of some f ut ure observables;
see Geisser (197 1 and 198 4 ) and Ait chisonand Dunsmore ( 197 5 ) f or ext ensive
discussion of t hi s proposit ion and f ur t he r ref erences. This is especially
t r ue in sit uat ions where t he model parameters do not represent real physical
qua nt i t i e s, the t ypi cal case in regression and t ime series analyses. Est i
mating model parameters is in general a "half way house" on the way t o pr e dict
ing some relevant f ut ure observat ion, and much can be lost by focusing founda
t ional discussion on t he half way house inst ead of t he ult imat e dest inat ion.
For example, the relevant uncert aint y f or a pat ient wit h a pa r t i cul a r c l i ni c a l
condit ion undergoing a pa r t i cul a r therapy is not a confidence band f or an
est imat ed survival curve; r at he r , t he pat ie nt and his physician should be
concerned wit h the pr e dict ive di st r i but i on f or t hat pat i e nt ' s f ut ure l i f e t i me .
The i nf e r e nt i a l question of i nt e r e st to t he pat ie nt is how t o generate t hi s
pr e dict ive di st r i but i on.
The LP does not address t his question di r e c t l y. Berger and Wolpert
claim t hat pr edict ion can be embedded i n the LP framework by including t he
f ut ure observable as part of t he unknown parameter. But then appears as a
nuisance parameter t hat is cl e ar l y not "noninformative" in the sense of Berger
and Wolpert. LP ideas provide no guidance on the t reat ment of inf ormat ive
nuisance parameters. On t he ot her hand, de Fi ne t t i ' s subject ive Bayesian
theory is direct ed towards t he problem of predict ing f ut ur e observables, and
t he not ion of coherence derived from t hat theory provides a f oundational basis
f or pr e dict ive inf erence; see Lane and Sudderth ( 1 9 8 4 ) . In t hi s t heory,
models may be used to help generate predict ions about the f ut ure observable y
based upon observed x, but t he models merely provide a convenient st ruct ure
D I S C U S S I O N BY L A N E 181
a n d n eed c a r r y n o m eta p hys i c a l b ur d en o f " r ea l i ty" fo r the p a r a m eter s they
c o n ta i n .
P r em i s e b ) c i ted a b o v e i g n o r es the fa c t tha t m o d el p a r a m eter s a r e
fr eq uen tl y i n s ep a r a b l e fr o m the " exp er i m en t" w ho s e p o s s i b l e d i s tr i b uti o n s they
i n d ex. E s p ec i a l l y i n a p p l i c a ti o n s a r i s i n g i n n o n exp er i m en ta l s c i en c es l i k e
ec o n o m etr i c s o r r es o ur c e m a n a g em en t, the m o d el is s c up l tur ed ei ther fr o m d a ta
a l r ea d y in ha n d o r p er ha p s fr o m a r ea l i s ti c v i ew o f w ha t d a ta a r e p o ten ti a l l y
o b ta i n a b l e. In s uc h c a s es , ther e i s n o w a y to s ep a r a te w ha t (E ,x) s a ys a b o ut
fr o m " p r i o r " i n fo r m a ti o n a b o ut ; in fa c t, c a n n o t b e s a i d to exi s t p r i o r to
the fo r m ul a ti o n o f E , ev en tho ug h ther e m a y b e m uc h p r i o r i n fo r m a ti o n a b o ut
w hi c h x m i g ht b e o b s er v ed . In thes e s i tua ti o n s it is ha r d to c r i ti c i z e
" o b j ec ti v e" Ba yes i a n s w ho v i o l a te the LP b y l etti n g thei r " p r i o r s " d ep en d up o n
the s tr uc tur e o f the exp er i m en t E .
D I S C U S S I O N B Y P R O F E S S O R L U C I E N L E C A M
( U n i v e r s i t y o f C a l i f o r n i a a t B e r k e l e y )
P r o f e s s o r s B e r g e r a n d W o l p e r t a r e t o b e t h a n k e d a n d c o n g r a t u l a t e d
f o r g i v i n g u s a c l o s e l y a r g u e d v i e w o n t h e f o u n d a t i o n s o f s t a t i s t i c s . T h e i r
a r g u m e n t s in f a v o r o f t h e L i k e l i h o o d P r i n c i p l e a r e y e r y p e r s u a s i v e i n d e e d . O n e
m a y s u s p e c t , h o w e v e r , t h a t s o m e r e a d e r s w i l l b e c o n v i n c e d a n d c o n v e r t e d w h i l e
s o m e o t h e r s w i l l h o l d f a s t t o t h e i r m i s g u i d e d b e l i e f s , i n s p i t e o f a l l t h e
e v i d e n c e .
I s h a l l t r y h e r e t o i n d i c a t e w h y t h e p r e s e n t w r i t e r b e l o n g s t o t h e
l a t t e r c a t e g o r y .
T h e r e i s a b o d y o f s t a t i s t i c a l t h e o r y , c a l l i t " t y p e 1 " , t h a t d e a l s
w i t h t h e f o l l o w i n g k i n d o f s y s t e m s . W h e n c o n t e m p l a t i n g a p a r t i c u l a r u n r e s o l v e d
q u e s t i o n , o n e d e v i s e s e x p e r i m e n t s t o a s c e r t a i n w h a t t h e f a c t s a r e . T h e m a t h e -
m a t i c i a n w i l l a b s t r a c t t h e i d e a o f " e x p e r i m e n t " , u s i n g a n o b j e c t f o r m e d b y a
f a m i l y o f p r o b a b i l i t y m e a s u r e s o n a s u i t a b l e f i e l d . T h e c o n s e q u e n c e s o f u s i n g
p a r t i c u l a r p r o c e d u r e s t o a n a l y s e t h e " e x p e r i m e n t " a r e t h e n d e s c r i b a b l e i n
p r o b a b i l i s t i c l a n g u a g e . O n e c a n a t t e m p t t o s i n g l e o u t p r o c e d u r e s t h a t h a v e a
r e a s o n a b l e p e r f o r m a n c e i n t h i s p r o b a b i l i s t i c w o r l d . T h a t i s a b i t l i k e s e l e c t -
i n g t o o l s : w r e n c h e s a r e o f t e n , b u t n o t a l w a y s , s u c c e s s f u l a t u n s c r e w i n g b o l t s ;
p a i n t b r u s h e s o f t e n fa i l i n t h e s a m e a c t i v i t y .
T h i s k i n d o f e n d e a v o r h a s g i v e n u s t h e N e y m a n - P e a r s o n t h e o r y a n d
W a l d ' s t h e o r y o f " s t a t i s t i c a l d e c i s i o n f u n c t i o n s " . O n e c a n r e a d i l y c l a i m t h a t
t h e w h o l e e n t e r p r i s e i s m i s g u i d e d , b u t i t d o e s s e e m t o h a v e a r o l e t o p l a y i n
c e r t a i n e n d e a v o r s , l i k e p l a n n i n g e x p e r i m e n t s , s e t t l i n g a r g u m e n t s t h a t i n v o l v e
s e v e r a l s c i e n t i s t s a n d o d d q u e s t i o n s s u c h a s " i s m e t h o t r e x a t e e f f e c t i v e i n t h e
1 8 2
D I S C U S S I O N BY LE CAM 1 8 3
t r e a t m e n t o f c o l o n c a n c e r . "
T h e r e is a n o t h e r b o d y o f t h e o r y , c a l l it " t y p e 2 " , t h a t d e a l s w i t h
a x i o m s o f c o h e r e n t b e h a v i o r a n d p r i n c i p l e s o f e v a l u a t i o n o f e v i d e n c e . S o m e o f
i t , a n d p e r h a p s m o s t o f i t , h a s t o do w i t h w h a t " o n e " s h o u l d " t h i n k " a f t e r t h e
r e s u l t s o f t h e e x p e r i m e n t s h a v e b e c o m e k n o w n . C o m p a r a t i v e l y l i t t l e h a s b e e n
w r i t t e n on h o w " o n e " c a n t r a n s m i t t h e " e v i d e n c e " t o a n o t h e r p e r s o n , e v e n i n t h e
B e r g e r - W o l p e r t t e x t , t h i s c o m m u n i c a t i o n p r o b l e m t a k e s s e c o n d p l a c e t o t h e " o n e
s h o u l d t h i n k " q u e s t i o n .
B e r g e r a n d W o l p e r t s e e e v i d e n c e o f c o n t r a d i c t i o n b e t w e e n t h e
" t y p e 1" a n d " t y p e 2 " a p p r o a c h e s . In a s t r i c t l y m a t h e m a t i c a l v i e w o f t h e p r o b -
l e m , t h e r e is n o o v e r l a p b e t w e e n t h e t w o a p p r o a c h e s b e c a u s e " t y p e 1" doe s n ot
h a v e a n y p r o b a b i l i t i e s t o p l a y w i t h o n c e t h e d i c e h a v e b e e n c a s t .
C o n s i d e r f o r i n s t a n c e a n e x p e r i m e n t i n v o l v i n g t w o c o n t a i n e r s , o n e
w i t h 5 0 % r e d o b j e c t s , t h e o t h e r w i t h 2 5 % . A c oi n is t o s s e d t o s e l e c t a c o n -
t a i n e r . T h e n o n e e x t r a c t s a ba l l fr om t h a t c o n t a i n e r . It t u r n s o u t t o b e
b l u e . W h e n a l l o f t h a t h a s b e e n p r o p e r l y c a r r i e d o u t t h e r e a r e n o p r o b a b i l i t i e s
l e ft s i n c e t h e c o n t a i n e r h a s b e e n s e l e c t e d . It i s e i t h e r t h e f i r s t o r t h e
s e c o n d a n d n ot a p r o b a b i l i s t i c m i x t u r e o f b o t h . A n y a s s i g n m e n t o f p r o b a b i l i t i e s
a t t h a t s t a g e r e q u i r e s a m p l i f i c a t i o n o f t h e m o d e l , w i t h t h i n k i n g a b o u t p o s s i b l e
r e p e t i t i o n s o f t h e e x p e r i m e n t s o r d e g r e e s o f b e l i e f , o r b e t t i n g s t r a t e g i e s o r
w h a t e v e r .
B e r g e r a n d W o l p e r t t r y t o c o n v i n c e us t h a t i n s u c h a s i t u a t i o n o n e
s h o u l d f o l l o w t h e l i k e l i h o o d p r i n c i p l e . T h e a r g u m e n t is t h o r o u g h . L. J .
S a v a g e ' s a r g u m e n t w a s a l s o y e r y t h o r o u g h , b u t I h a v e y e t t o fi n d a s c i e n t i s t
w h o w o u l d b e c o n v i n c e d by a p o s t e r i o r d i s t r i b u t i o n on t h e m e t h o t r e x a t e a n d
c o l o n c a n c e r q u e s t i o n i f t h e p r i o r h a s b e e n s u p p l i e d by a p h a r m a c e u t i c a l
c o m p a n y . T h e p o i n t i s t h a t o n e c a n e a s i l y a r g u e o n e s e l f i n t o a c o r n e r .
In t h e p r e s e n t c a s e , h o w e v e r , I t h i n k t h a t t h e a r g u m e n t h a s o n e
m a j o r f l a w . It is b a s e d o n t h e a s s u m p t i o n t h a t g i v e n a n e x p e r i m e n t E , a n d t h e
r e s u l t x o f t h a t e x p e r i m e n t , t h e r e is a w e l l d e f i n e d o b j e c t Ev ( E , x ) . T h e
n a t u r e o f t h e o b j e c t Ev ( E , x ) is n ot d e s c r i b e d e x p l i c i t l y . T h i s is n ot t h e
problem. What matters is t he assumption t hat t here is such an obje ct , or more
spe ci f i ca l l y a f unct ion ( E, x) Ev ( E, x) . St ar t i ng from such an assumption,
and adding a few ot her "pr i nci pl e s", one can prove t hat the f unct ion Ev must
have ce r t ai n pr ope r t ie s.
( I am reminded here of the standard "proof" t hat 1 = ( 1 ) , assuming
t hat J is a f unct ion: ce r t ai nl y / l / ( l ) = / ( 1 ) / 1 and by mul t i pl i ca t i on
( / T) = (/ ) . I am also reminded of Spinoza's "Theologia more geometrico
demonstrata") .
The very exist ence of t he f unct ion Ev is not cl e ar t o t hi s wr i t e r .
Even i f i t exist s in a s t r i c t l y mathematical abst ract ion of "experiments" and
r e sul t s, the relevance t o pr act i cal applicat ions is not di r e ct l y' e vi de nt .
Several years ago a problem of t his nature was raised during the
conference on the Future of St a t i st i cs held at Madison. Someone asked t he
panel how they would report the evidence i n a cl i ni ca l t r i a l of a drug intended
t o suppress renal ca l cul i . The answer, given by G. Barnard, was "report t he
l i ke l i hood f unct ion". That may be, but one should also report the age,
ancest ry, healt h st at us of the pa r t i ci pa nt s, the presumed mode of act ion of the
drug, i t s manufacturer, ideas about whether cal cul i occur in clust ers or
bunches, t he i r size di st r i but i on, whether t he i r formation may be spurred or
hindered by nut r i t i ona l f a ct or s, e t c . , e t c . , including whether the randomization
used ( or unused) led t o apparent disbalance.
There i s no short cut t o report ing what was act ually done and
observed. In sit uat ions involving games of chance wit h de f i ni t e r ul e s, one
might simplif y t he evidence r e por t . I t is also t r ue t hat Savage could argue
t hat anyone playing games according to Savage's rules need only report ( t o
himself ) the r e sult ing post er ior di st r i but i ons.
I t does not f ollow from such mathematical theorems t hat one must
necessarily frame pr act i cal questions in terms of Savage's games or i n terms
of t he Berger Wolpert rules of evidence, even i f these authors event ually
argue themselves int o a Bayesian framework.
D I S C U S S I O N BY L E C A M 1 8 5
H e r e t h e s i t u a t i o n i s c o m p l e x b e c a u s e " t y p e 1 " t h e o r i e s h a v e g i v e n
p r o o f s t h a t " e x p e r i m e n t s
1 1
a r e c h a r a c t e r i z e d b y t h e d i s t r i b u t i o n s o f t h e i r
l i k e l i h o o d f u n c t i o n s . A l s o it is a s t a n d a r d r e s u l t o f " t y p e 1" t h e o r i e s t h a t
B a y e s p r o c e d u r e s , o r t h e i r l i m i t s f o r m c o m p l e t e c l a s s e s . A m a i n d i f f e r e n c e is
t h a t t h e " t y p e 1 " t h e o r i e s i n s i s t t h a t t h e y a r e a b o u t r i s k f u n c t i o n s , n o t
p o s s i b l e i n t e r p r e t a t i o n s o f s i n g l e p o s t e r i o r d i s t r i b u t i o n s .
T h e p a s s a g e f r o m a d v o c a t i o n o f t h e l i k e l i h o o d p r i n c i p l e t o B a y e s i a n
t h e o r y is d e s c r i b e d b y B e r g e r a n d W o l p e r t b u t n o t a s a s t r i c t l y l o g i c a l
c o n s e q u e n c e o f t h e L . P. p r i n c i p l e a n d o t h e r e x p l i c i t l y s t a t e d a x i o m s . It is
w e a k c o m p a r e d t o t h e r e s t . H o w e v e r , in t h e p r o c e s s , t h e y a l s o d e m o n s t r a t e t h a t
t h e y d o n o t a b i d e b y t h e i r o w n L . P . p r e s c r i p t i o n s .
T h i s o c c u r s in t h e d i s c u s s i o n o f a n e x a m p l e o f C . S t e i n . T h e
a u t h o r s s a y
" n o t e t h a t it w a s a s s u m e d t h a t x = y = d in t h e a b o v e c o n d i t i o n a l
a n a l y s i s , a n d s i n c e i t c a n b e s h o w n t h a t Y i s a l m o s t c e r t a i n t o b e e n o r m o u s . . . . "
( e m p h a s i s a d d e d ) .
T h a t s e e m s t o b e a v e r y d i r e c t a p p e a l t o a f r e q u e n c y e v a l u a t i o n o f
t h e s i t u a t i o n , a n d n o t e v e n a c o n d i t i o n a l o n e a t t h a t . S u c h a n a p p e a l d o e s n o t
f i t w i t h t h e l o g i c o f t h e r e s t o f t h e p a p e r .
T h e r e a r e o t h e r m a t t e r s t h a t s h o u l d b e d i s c u s s e d , b u t it w o u l d t a k e
t o o m u c h s p a c e . O n e o f t h e m h a s t o d o w i t h a p p r o x i m a t i o n . A s s u m i n g t h a t t h e
f u n c t i o n E v e x i s t s a n d t h a t i f ( E j , xj) a n d ( E
2
, x ^ ) g i v e t h e s a m e l i k e l i h o o d
f u n c t i o n , t h e n t h e e v i d e n c e s a r e t h e s a m e , is o n e e n t i t l e d t o p r e s u m e t h a t i f
t h e l i k e l i h o o d f u n c t i o n s a r e ' a p p r o x i m a t e l y ' t h e s a m e t h e n t h e e v i d e n c e s a r e
a l s o " a p p r o x i m a t e l y " t h e s a m e ?
H e r e w e h a v e t w o " a p p r o x i m a t e l y " w i t h u n d e f i n e d , b u t p e r h a p s d e f i n
a b l e m e a n i n g in t h e f i r s t i n s t a n c e a n d a n a p p a r e n t l y u n d e f i n a b l e m e a n i n g in t h e
s e c o n d o c c u r r e n c e s i n c e E v ( E , x ) i t s e l f is a n u n d e f i n e d o b j e c t .
F o r i n s t a n c e i t is a c l a s s i c a l r e s u l t t h a t i f o n e t a k e s a y e r y
2 1
l a r g e s a m p l e ( x
1
, . . . , x
n
) f r o m t h e s t a n d a r d C a u c h y { [ l + ( x ) ] } " , f o r " m o s t "
s a m p l e s t h e l i k e l i h o o d f u n c t i o n w i l l b e " c l o s e t o " o n e o b t a i n a b l e f r o m a s i n g l e
o b s e r v a t i o n y f r o m f t ( , ) . D o e s t h a t h a v e a n y " e v i d e n t i a l
1 1
m e a n i n g f o r t h e
L . P . ? M u s t o n e n e c e s s a r i l y i n t e r p r e t it o n l y t h r o u g h a c o m p u t a t i o n o f p o s t e r
i o r d i s t r i b u t i o n s ? I f s o , f o r w h a t p r i o r s ?
T o s u m m a r i z e B e r g e r a n d W o l p e r t h a v e g i v e n us a v a l i a n t d e f e n s e o f
t h e L . P . H o w e v e r it d o e s d e p e n d o n a b a s i c a s s u m p t i o n o f e x i s t e n c e o f t h e
e v i d e n c e f u n c t i o n E v . T h i s f u n c t i o n , i f it e x i s t s , d o e s n o t c o n f o r m t o t h e
t r a d i t i o n o f r e p o r t s in s c i e n t i f i c j o u r n a l s . T h e t h e o r y d o e s n o t a c t u a l l y
c o n f l i c t w i t h t h e s o c a l l e d " c l a s s i c a l " o n e b e c a u s e t h e i r d o m a i n s o f e x i s t e n c e
a r e s e p a r a t e a n d t h e i r a i m s d i f f e r e n t .
T h i s a u t h o r p r e s u m e s t h a t t h e r e is s o m e v a l u e in s o m e o f " c l a s s i c a l
s t a t i s t i c s " a n d a l s o in t h e l i k e l i h o o d p r i n c i p l e , b u t f e e l s t h a t o n e c a n n o t
s u p p o r t t h e p r a c t i c a l a p p l i c a t i o n o f e i t h e r ( o r o f o t h e r t h e o r i e s ) o n p u r e l y
m a t h e m a t i c a l g r o u n d s . O n e s h o u l d k e e p a n o p e n m i n d a n d b e a b i t " u n p r i n c i p l e d " .
D I S C U S S I O N O F T H E S E C O N D E D I T I O N B Y P R O F E S S O R L E C A M
In o r d e r t o a v o i d a n y m i s u n d e r s t a n d i n g , l e t m e r e p e a t t w o o f t h e
c r i t i c i s m s m a d e a b o v e : U s i n g " e v i d e n c e " a s a f u n c t i o n o f a p a i r ( E , x ) a n d
u s i n g " a p p r o x i m a t e " l i k e l i h o o d f u n c t i o n s . T h e t w o p o i n t s a r e h i g h l y i n t e r
c o n n e c t e d .
F o r s i m p l e e x p e r i m e n t s P r o f e s s o r s B e r g e r a n d W o l p e r t u s e a m a t h e
m a t i c a l e n t i t y E t h a t c o n s i s t s o f a s e t a n d a f a m i l y { P
Q
, } o f p r o b a b i -
l i t y m e a s u r e s on a g i v e n f i e l d .
" E v i d e n c e " is u n d e f i n e d , b u t it is s u p p o s e d t o b e a f u n c t i o n o f t h e
p a i r ( E , x ) w h e r e x is t h e o b s e r v e d v a l u e . T h i s m a y s e e m i n n o c u o u s , b u t it is
d e f i n i t e l y n o t . It r e j e c t s a p a r t o f t h e d a t a u s u a l l y c o n s i d e r e d a s p a r t o f
t h e e v i d e n c e in t h e c o m m o n l a n g u a g e u s e o f t h e t e r m , n a m e l y t h e t h i n k i n g t h a t
w e n t i n t o t h e s e l e c t i o n o f t h e m a t h e m a t i c a l e n t i t y c a l l e d E .
E x c e p t p e r h a p s in t h o s e c a s e s w h e r e t h e r a n d o m i z a t i o n is m a n m a d e
o n p u r p o s e a n d p e r h a p s a l s o in t h e P o i s s o n f o r m u l a s f o r r a d i o a c t i v e d e c a y , t h e
s e l e c t i o n o f t h e " m o d e l " E is b a s e d o n r a t h e r l o o s e a r g u m e n t s t h a t a r e n o t
t h e m s e l v e s r e p r e s e n t a b l e b y p a i r s ( E
1
, x
1
) . H e r e , b y " m o d e l " I d o n o t m e a n
D I S C U S S I O N BY L E C A M 1 8 5 . 2
s o m e t h i n g l i k e " l i n e a r m o d e l s " bu t m o r e a m a t h e m a t i c a l c o n s t r u c t t h a t a t t e m p t s
t o c a t c h t h e i m p o r t a n t f e a t u r e s o f a p h y s i c a l o r b i o l o g i c a l p h e n o m e n o n . I
h a v e , o f c o u r s e , n o o b j e c t i o n t o a t h e o r y o f " e v i d e n c e " b a s e d o n a f u n c t i o n o f
p a i r s ( E , x ) . It j u s t f a i l s t o c o n n e c t p r o p e r l y w i t h m y o w n i n t u i t i v e n o t i o n
o f w h a t e v i d e n c e i s . T h e r e f o r e I d o n o t fe e l b o u n d i n p r a c t i c e by t h e t h e o r e m s
d e r i v e d f r o m s u c h a t h e o r y .
E v e n i f o n e t r i e s v e r y h a r d t o p u t t h e i n f o r m a t i o n i n t h e f o r m
( E , x ) o n e w i l l a l m o s t a l w a y s p u t i n E c e r t a i n f o r m u l a s f o r t h e s a k e o f c o n v e n -
i e n c e , s i m p l i c i t y o r p l a i n l a z i n e s s . T h u s ( E , x ) w i l l o n l y be o u r a p p r o x i m a -
t i o n t o a " b e t t e r " ( E
1
, x
1
) . It i s m y f e e l i n g t h a t , i f o n e w a n t s t o t a k e i n t o
a c c o u n t t h e f a c t t h a t s u c h a p p r o x i m a t i o n s a r e t h e r u l e , o n e m u s t a l s o e x p l a i n
w h a t d i f f e r e n c e s t h e y m a y m a k e in t h e u s e o f t h e u n d e f i n e d " e v i d e n c e " Ev ( E , x ) .
I a m n o t t o o s u r e t h a t t h i s c a n be a c c o m p l i s h e d w i t h o u t i n t r o d u c i n g i n t h e
s y s t e m a v a r i e t y o f c o n c e p t s t h a t g o b e y o n d p a i r s ( E , x ) .
In s u m m a r y I r e m a i n o p p o s e d t o t h e a p p a r e n t n o r m a t i v e a s p e c t o f a
t h e o r y t h a t s a y s t h a t I m u s t a b i d e by t h e L P w h e n I a m u n a b l e t o p u t m y e m o -
t i o n s a n d v a r i o u s b i t s o f k n o w l e d g e , o r l a c k o f k n o w l e d g e , i n t o i t .
REPLY TO T H E D I S C U S S I O N
W e are v e r y g rat e f u l t o t he d i s c u s s an t s f or t he i r s t i m u l at i n g c om
m e n t s . B e s i d e s d e s c ri b i n g i n t e re s t i n g l y d i f f e re n t p e r s p e c t i v e s , t he c om m e n t s
s e rv e t o hi g hl i g ht a n u m b e r of i m p ort an t i s s u e s w e i n ad e q u at e l y d i s c u s s e d in
t he t e x t .
R E P L Y _ TO P R OF E S S OR S B A Y A R R I A ND DE G R OOT
It i s i n d e e d a p l e as u re t o t han k Prof e s s ors B ay arri an d De g root f or
t he i r c are f u l re ad i n g of ou r m a n u s c r i p t an d t he d e e p i n s i g ht re f l e c t e d in t he i r
d i s c u s s i o n . In t he m an u s c ri p t we t ri e d t o e x p l ore t he i m p l i c at i on s of t he LP
an d t he i s s u e s it rai s e s wi t hou t e n d ors i n g an y p art i c u l ar m od e of i n f e re n c e
( u n t i l t he f i n al c h a p t e r ) ; in p art i c u l ar we t ri e d hard n ot t o l e t ou r B ay e s i an
p oi n t of v i e w c ol or t he b as i c arg u m e n t s e n ou g h t o m ak e t he m u n p e rs u as i v e t o
f ol l owe rs of t he f re q u e n t i s t t r a d i t i o n . Thu s ou r e m p has i s was n ot on " what i s
t he l i k e l i hood f u n c t i on ?
1 1
R a t h e r , we t ook t he l i k e l i hood f u n c t i on as g i v e n ,
an d arg u e d t hat t he LP wou l d f ol l ow n o m a t t e r what re as on ab l e d e f i n i t i on of t he
l i k e l i hood f u n c t i on i s u s e d . The d e f i n i t i on s in ( 3 . 5 . 1 ) an d ( 3 . 5 . 2 ) are b ot h
r e a s o n a b l e , an d s e rv e d i f f e re n t p u r p o s e s .
B u t we are B a y e s i a n s , an d are in e s s e n t i al l y c om p l e t e ag re e m e n t
wi t h t he b as i c i s s u e s rai s e d b y B ay arri an d De G root . W e ag re e t hat t he re i s n o
c l e ar d i s t i n c t i on b e t we e n " p aram e t e rs " an d " v ari ab l e s " , an d t hat d e f i n i t i on of
t he l i k e l i hood f u n c t i on i s a m b i g u o u s . As B ay arri an d De G root o b s e r v e , an y
p art i t i on of t he p aram e t e rs an d v ari ab l e s i n t o t wo d i s j oi n t s e t s s, an d Sp ,
wi t h Sj c on t ai n i n g t he ob s e rv e d q u an t i t y x , l e ad s t o an ac c e p t ab l e l i k e l i hood
f u n c t i on
( s
2
) = f ( s j | s
2
) ( p rov i d i n g t hi s f u n c t i on i s ac c e p t e d as " k n o w n " ) .
As l on g as on e al s o k e e p s t rac k of al l k n own m arg i n al an d c on d i t i on al i n f orm a
t i on ab ou t t he v ari ab l e s an d p a r a m e t e r s , an y s u c h p art i t i on l e ad s t o a l i k e l i
hood f u n c t i on whi c h c on t ai n s al l e v i d e n c e f rom t he e x p e ri m e n t ( at l e as t t o a
1 8 6
R E P L Y T O T H E D I S C U S S I O N 1 8 7
B a y e s i a n ) . B u t t h e n e e d t o k e e p t r a c k o f t h i s m a r g i n a l a n d c o n d i t i o n a l i n f o r -
m a t i o n , a n d t o t r e a t u n k n o w n s i n s , d i f f e r e n t l y f r o m u n k n o w n s i n s
?
, s h o u l d b e
s o u r c e s o f c o n c e r n t o n o n - B a y e s i a n s .
B a y a r r i a n d D e G r o o t s u g g e s t t h a t t h e c h o i c e s
1
= " o b s e r v e d " a n d
s
2
= " u n o b s e r v e d " ( w h i c h t h e y a n d P r o f e s s o r K a d a n e c a l l L F . i n B D K ) h a s l o g i -
c a l p r e e m i n e n c e a s a d e f i n i t i o n o f l i k e l i h o o d ; t h e n i ( s
9
) r e p r e s e n t s p r e c i s e l y
w h a t w a s l e a r n e d f r o m t h e o b s e r v a t i o n o f x , u n c o n f o u n d e d b y a n y g i v e n i n f o r m a -
t i o n a b o u t S . W e a g a i n a g r e e ; i t w a s o n l y t h e s o c i o l o g i c a l c o n c e r n s m e n t i o n e d
i n o u r f i r s t p a r a g r a p h a b o v e t h a t k e p t u s f r o m s o d e f i n i n g t h e l i k e l i h o o d
f u n c t i o n i n g e n e r a l .
F u r t h e r r e p e t i t i o n o f t h e i n s i g h t s o f B a y a r r i a n d D e G r o o t w o u l d b e
u n n e c e s s a r i l y d u p l i c a t i v e . S u f f i c e it t o s a y t h a t w e a g r e e t h a t n o n - B a y e s i a n s
c a n h a v e a v e r y d i f f i c u l t t i m e d e f i n i n g a n d i n t e r p r e t i n g t h e l i k e l i h o o d
f u n c t i o n ; a n d o n c e t h e y p a s s t h i s h u r d l e , t h e y s t i l l m u s t c o n t e n d w i t h t h e L P .
R E P L Y T O P R O F E S S O R H I L L
It w o u l d s e e m r a t h e r f o o l i s h o f u s t o q u e s t i o n P r o f e s s o r H i l l ' s
i n t e r e s t i n g d i s c u s s i o n a t a l l , b e c a u s e h e s e e m s t o f e e l t h a t w e d o n o t g o f a r
e n o u g h i n o u r s u p p o r t o f t h e L P . F i r s t w e w o u l d l i k e t o c l e a r u p t h a t
m i s i m p r e s s i o n ( w e a r e f u l l y a s e n t h u s i a s t i c a s h e i s c o n c e r n i n g t h e a p p l i c a -
b i l i t y o f t h e L P ) , a n d t h e n p r o c e e d t o t h e d e e p i s s u e h e r a i s e s c o n c e r n i n g u s e
o f i m p r o p e r , o r p r o p e r b u t f i n i t e l y a d d i t i v e , p r i o r s .
F r o m P r o f e s s o r H i l l ' s c o m m e n t s ( a n d a l s o t h o s e o f P r o f e s s o r L e C a m )
i t i s c l e a r t h a t w e d i d n o t e x p r e s s o u r s e l v e s c l e a r l y i n t h e M o n e t t e - F r a s e r ,
S t o n e , a n d S t e i n e x a m p l e s , w i t h r e g a r d t o t h e r o l e o f f r e q u e n t i s t m e a s u r e s
a n d o u r o w n c o n d i t i o n a l p e r s p e c t i v e o f s t a t i s t i c a l a n a l y s i s . ( T h i s l a c k o f
c l e a r e x p r e s s i o n w a s p r i m a r i l y d u e t o o u r c o n c e n t r a t i o n o n u s i n g t h e e x a m p l e s
t o i n d i c a t e t h e n e c e s s i t y f o r s o m e t y p e o f B a y e s i a n p r o c e s s i n g o f l i k e l i h o o d
f u n c t i o n s . ) O u r d i s c u s s i o n o f f r e q u e n t i s t m e a s u r e s w a s m o t i v a t e d p a r t l y b y t h e
f a c t t h a t t h e e x a m p l e s w e r e h i s t o r i c a l l y d e v e l o p e d i n t h a t f a s h i o n , a n d p a r t l y
1 8 8 THE LIK ELIHOOD PRINCIPLE
t o i n d i c at e t hat c on d i t i on al ( p rop e r) B ay e s i an s wi l l n at u ral l y ov e rc om e t he
d i f f i c u l t i e s i n v ol v e d , e v e n f rom t he f re q u e n t i s t p e rs p e c t i v e . W e al s o d o
b e l i e v e t hat t he re c an b e v al u e i n f re q u e n t i s t m e as u re s as p os s i b l e warn i n g
s i g n al s t hat c are m u s t b e t ak e n i n t he B ay e s i an a n a l y s i s . H o w e v e r , w e b y n o
m e an s m e an t t o i m p l y t hat ( b e c au s e o f a B ay e s i an f re q u e n t i s t c on f l i c t ) on e
m u s t n e c e s s a r i l y c han g e t he B ay e s i an an al y s i s .
The s e p oi n t s a r e , p e r h a p s , b e s t i l l u s t rat e d b y t he St e i n e x a m p l e ,
i n t hat i t i s we l l re c og n i z e d t hat ( f or d at a f rom u n i v ari at e n orm al m o d e l s ) t he
u n i f orm i m p rop e r p ri or i s t y p i c al l y q u i t e s at i s f ac t ory . It i s t y p i c al l y s a t i s
f ac t ory , h o w e v e r , o n l y b e c au s e is u s u al l y s m al l e n ou g h t hat t ru e p ri or b e l i e f s
c an b e ap p rox i m at e d b y t he u n i f orm p ri or. The b ad f re q u e n t i s t p e rf orm an c e of
t he u n i f orm p ri or i n t he m od e l ( 5 . 3 . 3 ) s hou l d b e a warn i n g t hat t he ad e q u ac y of
t he u n i f orm ap p rox i m at i on t o p ri or b e l i e f s s hou l d b e i n v e s t i g at e d , an d i n d e e d
s u c h an i n v e s t i g at i on wou l d u s u al l y i n d i c at e t hat t he ap p rox i m at i on was b a d ;
t hi s w o u l d b e t he c on c l u s i on u n l e s s y was \ / e r y s m al l . T h e f re q u e n t i s t m e a s u r e ,
h e r e , is ac t u al l y s u p e r f l u o u s , howe v e r. T h e c on d i t i on al B ay e s i an wou l d
n at u ral l y u s e a u n i f orm p ri or ( as a g ood a p p r o x i m a t i o n ) whe n y / d was v e r y s m al l ,
an d w o u l d re c og n i z e ( i f he had an y p ri or i n f orm at i on w h a t s o e v e r ) i t s i n ad e q u ac y
f or t y p i c al l y l arg e y / d , s i m p l y b e c au s e t he l i k e l i hood f u n c t i on wou l d t he n b e
m u c h m ore d i f f u s e t han e v e n y e r y v ag u e p ri or i n f orm at i on . No k n owl e d g e of
f re q u e n t i s t p r o p e r t i e s , or o f d i f f e ri n g p rop e rt i e s of s c al e an d l oc at i on
p a r a m e t e r s , i s n e c e s s ary t o b e hav e s e n s i b l y . A l s o , w e i n n o s e n s e re c om m e n d
c han g i n g t he p ri or as t he m od e l c han g e s . If on e ' s p ri or op i n i on s t ru l y are
d i f f u s e o v e r t he ran g e of t he l i k e l i hood f u n c t i on , b y al l m e an s u s e t he u n i f orm
p r i o r , n o m a t t e r what t he m o d e l . W e s i m p l y d o n ot f e e l t hat t hi s wi l l b e t he
c as e f or t he m od e l in ( 5 . 3 . 3 ) , h o w e v e r , u n l e s s y hap p e n s t o b e e x c e p t i on al l y
s m al l . ( Li k e wi s e , w e j u d g e t hat t he u n i f orm p ri or wi l l u s u al l y b e i n ap p rop ri
at e f or n orm al m od e l s w h i c h hav e m on s t rou s l y l arg e v ari an c e s . ) The re is n o
i n c om p at i b i l i t y wi t h t he l i k e l i hood p ri n c i p l e h e r e , s i n c e t he " ad e q u ac y of t he
ap p rox i m at i on " c an b e j u d g e d s i m p l y b y l ook i n g at t he l i k e l i hood f u n c t i on .
REPLY TO THE DISCUSSION 189
The major issue raised by Professor Hi l l concerns the need f or
conglomerabilit y, or a l t e r na t i ve l y the concern t hat need be f e l t when the
f r equent ist answer completely cont radict s t he post erior Bayesian answer.
Spe ci f i ca l l y, i n the Monette Fraser example Professor Hi l l argues t hat the
uniformly bad f r equent ist performance of analysis based on t he f i ni t e l y addi
t i ve uniform pr ior is not operat ionally meaningful, because the sample space i s ,
in r e a l i t y, always bounded. The issue here is not di r e ct ly r e lat e d t o the
like lihood pr i nci ple , but is another aspect of t he possible problems caused by
t he use of i nf i ni t e models t o approximate r e a l i t y. I f t he sample space in the
Monette Fraser example is bounded by N, then ce r t ai nly t he uniform pr i or be
comes permissible, since one can act ually simply choose t he proper discret e
uniform pr i or on 0 to 2N. We do f e e l , however, t hat t he subject ive assessment
of unif ormit y on 0 t o 2N would r ar e ly be reasonable in pr act i ce , precisely
because t he use of t he i nf i ni t e model as an approximation would t ypi ca l l y be
due to the be l i e f t hat no X, so large as to be unmeasurable, would act ually
occur; t his implies a pr i or be l i e f t hat could not be extremely lar ge . I n
general, we would view a uniform conf li ct between f r e que nt ist and Bayesian
measures as an indicat ion t hat e i t he r the approximation of an i nf i ni t e model
was inappropriat e, or the use of the f i ni t e l y addit ive pr ior was inappropriat e.
We do, of course, f e e l t hat a l l sample spaces are act ually f i n i t e ,
and t hat ( f or vi r t ua l l y any problem) one could act ually provide a (perhaps
overly large) f i ni t e sample space. Do examples of the type we are discussing
e xist f or f i ni t e sample spaces? I f so, such would seem t o provide a counter
example to Professor Hi l l ' s argument. I f not , one could indeed not obje ct ,
philosophically, t o the use of f i ni t e l y addit ive measures. There would, how
ever, remain pragmatic questions concerning the pr a c t i c a l i t y of using f i ni t e l y
addit ive priors (as opposed t o countably addit ive pr ior s) t o approximate pr i or
be l i e f s, but t hat is an issue f or another time and place.
We have long been admirers of Professor Hi l s caref ul t reat ment
of the random e f f e ct s analysis of variance model, and do not r e a l l y disagree
wit h his comments here. I f we observed a li ke li hood f unct ion, over the range
o f w h i c h o u r p r i o r w a s v e r y d i f f u s e , w e w o u l d h a v e n o q u a l m s a b o u t u s i n g t h e
u n i f o r m i m p r o p e r p r i o r . I f , h o w e v e r , t h e u n i f o r m p r i o r l e a d s t o a p r o c e d u r e
w i t h b a d f r e q u e n t i s t p r o p e r t i e s , w e w o u l d i n f e r t h a t t h e u n i f o r m p r i o r w a s a
p o o r a p p r o x i m a t i o n t o o u r p r i o r b e l i e f s f o r m o s t o f t h e l i k e l i h o o d f u n c t i o n s
t h a t c o u l d b e e n c o u n t e r e d , a n d w o u l d b e l o a t h e t o i m p l e m e n t i t i n , s a y , a
" r o u t i n e " c o m p u t e r p a c k a g e .
O u r v i e w o n t h i s m a t t e r i s p a r t l y t i e d t o t h e d i s c u s s i o n s s u r r o u n d -
i n g E x a m p l e s 1 6 a n d 3 7 . G o o d f r e q u e n t i s t p e r f o r m a n c e w i l l o f t e n g i v e s o m e
a s s u r a n c e t h a t a t y p e o f c o n d i t i o n a l B a y e s i a n a n a l y s i s i s m o d e r a t e l y r o b u s t ,
w h i l e b a d f r e q u e n t i s t p e r f o r m a n c e o f s u c h a n a n a l y s i s i s o f t e n a n i n d i c a t i o n o f
n o n r o b u s t n e s s . S u c h i m p l i c a t i o n s a r e b y n o m e a n s c e r t a i n , a n d u s e o f f r e q u e n t -
i s t v e r i f i c a t i o n m a y o f t e n b e a n i n e f f i c i e n t w a y o f i n v e s t i g a t i n g r o b u s t n e s s ,
b u t w e s h o u l d n o t d i s m i s s a n y a v a i l a b l e a i d s . In t h i s w e a l s o p e r c e i v e a t
l e a s t p a r t i a l a g r e e m e n t w i t h P r o f e s s o r H i l l , a s w i t n e s s e d b y h i s n u m e r o u s
p a p e r s o n t h e m a t t e r ( r e f e r e n c e d a n d d i s c u s s e d i n B e r g e r ( 1 9 8 4 e ) )
R E P L Y T O P R O F E S S O R L A N E
Before considering the two deep issues raised by Professor Lane,
we would raise one minor quibble. His second paragraph consists of a l i st i ng
of "bail-out options" for statisticians who choose not to follow the LP. A
major purpose of the monograph was to argue the inadequacy of such bail-outs.
Professor Lane does not make his views on such bail-outs clear, although
presumably, as a Bayesian, he does not accept thei r val i di ty (for perhaps
reasons other than those given in the t ext ) .
The two main issues raised by Professor Lane are ( i ) the adequacy
of the model paradigm and usefulness of the LP within i t , and ( i i ) the fact
that the LP ignores the Basic Tenet of Bayesianity, namely that inference
should consist of the quantification of uncertainty. In our analyses of these
issues i t is particularly important to realize that we perceive l i t t l e , i f
any, disagreement between us and. Professor Lane concerning the correctness of
the Bayesian pardigm for st at i st i cs. We do di f f er, however, in our opinions
REPLY TO THE DISCUSSION 91
c on c e rn i n g t he m os t c on v i n c i n g an d p rac t i c al l y u s e f u l way i n whi c h t hi s
p arad i g m s hou l d b e p re s e n t e d . W e e m p has i z e t he b as i c ag re e m e n t b e c a u s e , al l
t oo of t e n , n on B ay e s i an s u s e t he s e rat he r m i l d d i s p u t e s b e t we e n B ay e s i an s t o
re j e c t t he e n t i re B ay e s i an p arad i g m .
Prof e s s or Lan e on l y b ri e f l y m e n t i on s t he s e c on d i s s u e , t hat
q u an t i f i c at i on o f u n c e rt ai n t y s hou l d b e t he g oal o f i n f e re n c e . Thi s t e n e t
s e e m s al m os t s e l f e v i d e n t ( e v e n t hou g h i t i s n ot ac c e p t e d b y t he b u l k o f t he
ad v an c e d s t at i s t i c al c o m m u n i t y ) , an d i n d e e d t he LP d oe s n ot d i re c t l y i n c orp or
at e i t . An al t e rn at e p hras i n g of t he t e n e t , h o w e v e r , i s t hat s t at i s t i c i an s
s hou l d t re at k n own q u an t i t i e s as f i x e d an d t re at u n k n own q u an t i t i e s p rob ab i l i s
t i c al l y . T h e LP d oe s d e al wi t h t he f i rs t hal f of t hi s p hras i n g , t re at i n g t he
k n own d at a, x , as f i x e d f or i n f e re n c e , whi l e at l e as t t re at i n g as v ari ab l e
( i f n ot as a ran d om q u a n t i t y ) . He n c e t he LP e m b od i e s a m aj or p art o f t he B as i c
Te n e t of B ay e s i an i t y .
W e hav e s e v e ral re as on s f or ap p roac hi n g t he B ay e s i an p arad i g m
t hrou g h t he LP, rat he r t han t hrou g h ac k n owl e d g m e n t o f t he B as i c Te n e t . T h e
f i rs t i s s o c i o l o g i c a l , an d i s p art l y d u e t o t he c u rre n t s t at e o f s t a t i s t i c s .
Two p re v e l an t n ot i on s in t hi s " c u rre n t s t at e " are t hat t he f re q u e n t i s t p arad i g m
p rov i d e s a s at i s f ac t ory u n d e rp i n n i n g f or s t a t i s t i c s , an d t hat B ay e s i an an al y s i s
i s u n ac c e p t ab l e b e c au s e of i t s p ri or i n p u t s . It i s b e c au s e o f t he s e n ot i on s
t hat t he m aj ori t y o f s t at i s t i c i an s wou l d re j e c t t he B as i c T e n e t , an d t hat
d i re c t arg u m e n t s f or B ay e s i an i t y of t e n m ak e l i t t l e he ad way . N o t e , h o w e v e r ,
t hat t he LP d i re c t l y i m p u n e s t he f i rs t n ot i on , whi l e av oi d i n g t he b i as e s of
t he s e c on d n ot i on .
O f c ou rs e on e c an arg u e t hat t ran s i e n t s oc i ol og i c al c on c e rn s s hou l d
n ot b e t he b as i s f or j u d g e m e n t , b u t e v e n f rom a s t ri c t l y s c i e n t i f i c p e rs p e c t i v e
t he re i s s om e d ou b t as t o t he c orre c t rou t e t o t ak e t o t he B ay e s i an p arad i g m .
Di re c t arg u m e n t s f or t he B as i c Te n e t i n v ol v e s om e v ari e t y of c ohe re n c y arg u
m e n t s , b as e d on ax i om s o f rat i on al b e hav i or. Su c h ax i om s are b y n o m e an s
ab ov e c ri t i c i s m . F or i n s t an c e , t he arg u m e n t s l i s t e d in Se c t i on 3 . 7 , t hat hav e
b e e n rai s e d ag ai n s t t he c om m on " b e t t i n g s c e n ari o, " are n ot e as y t o d i s m i s s ( s e e
a l s o L e C a m ( 1 9 7 7 ) ) . A l s o , e v e n i f a l l s u c h a x i o m s a r e a c c e p t e d , t h e f a c t
t h a t t h e o n l y " r a t i o n a l " a n a l y s e s a r e t h o s e c o m p a t i b l e w i t h s o m e B a y e s i a n
a n a l y s i s d o e s n o t l o g i c a l l y i m p l y t h a t t h e o n l y a c c e p t a b l e w a y t o d o s t a t i s t i c s
i s t o w r i t e d o w n a p r i o r d i s t r i b u t i o n ( w h i c h c a n n e v e r b e m o r e t h a n a n
a p p r o x i m a t i o n t o p r i o r b e l i e f s ) a n d p e r f o r m a B a y e s i a n a n a l y s i s .
A l t h o u g h t h e L P i s a l s o s u b j e c t t o a x i o m a t i c a n d o p e r a t i o n a l
c r i t i c i s m , i t h a s s e v e r a l a d v a n t a g e s i n t h e s e r e g a r d s . T h e f i r s t i s t h a t i t s
a x i o m a t i c b a s i s i s c o m p e l l i n g t o m o s t p e o p l e . T h e W C P i s c o m p e l l i n g t o a l m o s t
e v e r y o n e , a n d t h e S P i s a n i n t e g r a l p a r t o f m o s t e x i s t i n g s t a t i s t i c a l p a r a d i g m s .
W e d o a c k n o w l e d g e t h a t t h e S P i s n o t r e a l l y " o b v i o u s , " a n d i n d e e d w e n t t o
c o n s i d e r a b l e e f f o r t i n S e c t i o n s 3. 6 a n d 3 . 7 t o j u s t i f y t h e p r i n c i p l e ( a n d n o t
j u s t f o r t h e " b e t t i n g s c e n a r i o " ) . T h e s i m p l e f a c t r e m a i n s , h o w e v e r , t h a t y e r y
f e w s t a t i s t i c i a n s w i l l r e j e c t e i t h e r o f t h e s e a x i o m s , w h i l e m o s t s e e m u n m o v e d
b y t h e c o h e r e n c y a x i o m s .
A s t o t h e o p e r a t i o n a l c r i t i c i s m , t h e L P w o u l d a g a i n s e e m t o h a v e
a n e d g e , p r e c i s e l y b e c a u s e i t d o e s n o t p r o v i d e a f i n a l a n s w e r a n d c a n h e n c e b e
m o r e s p e c i f i c i n i t s p a r t i a l a n s w e r . T h e c o h e r e n c y a p p r o a c h p r o v i d e s o n l y t h e
v a g u e g e n e r a l r e q u i r e m e n t t h a t s u b s t a n t i a l i n c o n s i s t e n c y w i t h s o m e B a y e s i a n
a n a l y s i s s h o u l d b e a v o i d e d . T h e L P i s , o n t h e o t h e r h a n d , s p e c i f i c i n i t s
r e c o m m e n d a t i o n t o u t i l i z e o n l y t h e o b s e r v e d l i k e l i h o o d f u n c t i o n , e v e n t h o u g h i t
d o e s n o t a d d r e s s t h e q u e s t i o n o f h o w t h i s i s t o b e d o n e . A n d f r o m a p u r e l y
p r a g m a t i c v i e w p o i n t , t h i s f i r s t s t e p m a y w e l l b e t h e m o s t i m p o r t a n t s t e p o f
a l l . T h e r e a s o n i s t h a t , i n p r a c t i c e , o n e o f t e n s p e n d s t h e g r e a t e s t e f f o r t i n
m o d e l s e l e c t i o n a n d v e r i f i c a t i o n ; t h e k n o w l e d g e t h a t o n e n e e d o n l y c o n s i d e r
t h e o b s e r v e d l i k e l i h o o d f u n c t i o n c a n s i m p l i f y t h i s t a s k e n o r m o u s l y . I n d e e d ,
i t i s n o t u n u s u a l f o r t h e c h o i c e o f a p r i o r o n m o d e l p a r a m e t e r s t o b e o f s u c h
s e c o n d a r y i m p o r t a n c e t h a t o n e n e v e r g e t s b e y o n d " p l a y i n g w i t h l i k e l i h o o d s . "
P r o f e s s o r L a n e d o e s p o i n t o u t t h a t t h e " s t a n d a r d " d e c o m p o s i t i o n i n -
t o m o d e l a n d p r i o r i s o f t e n a r t i f i c i a l , a n d s o s h o u l d n o t b e a p a r t o f s t a t i s -
t i c a l f o u n d a t i o n s . W h i l e s y m p a t h e t i c , w e v i e w s u c h d e c o m p o s i t i o n s a s
REPLY TO THE DISCUSSION 1 9 3
e s s e n t i al p rac t i c al s i m p l i f y i n g d e v i c e s , n e c e s s ary t o ac hi e v e p rog re s s . On e
u s u al l y p rog re s s e s on a hard p rob l e m b y i d e n t i f y i n g s i m p l e c om p on e n t s t hat c an
f i rs t b e an al y z e d s e p arat e l y , an d t he n c om b i n e d t og e t he r. Al t hou g h w e ag re e
t hat s u c h d e c om p os i t i on s are n ot al way s ap p rop ri at e , t he i r p rag m at i c al l y
c e n t ral rol e t o s t at i s t i c s i s hard t o d e n y . An d t he f ac t t hat t he LP p rov i d e s
s o m u c h i n s i g ht i n t o what i s p rob ab l y t he m os t c ru c i al c om p on e n t o f t he d e c om
p os i t i on , g i v e s i t c on s i d e rab l e ap p e al .
In s u m m ary on t hi s i s s u e , t he c ohe re n c y ap p roac h t o t he B ay e s i an
p arad i g m has m an y ad m i re rs ( ou rs e l v e s am on g t he m ) an d c an p e rhap s c l ai m a
l og i c al as c e n d an c y ov e r t he LP ap p roac h, b u t ( f or t he re as on s m e n t i on e d ab ov e )
we f e e l t hat t he LP ap p roac h has had , or at l e as t c an h a v e , a l arg e r i m p ac t .
The q u ot at i on on p . 2 f rom L. J. Sav ag e i s re v e al i n g in t hi s re g ard , c om i n g
f rom an ard e n t ad m i re r o f c ohe re n c y . ( M ore c om p l e t e d i s c u s s i on s of t hi s
i s s u e c an b e f ou n d i n B e rg e r ( 1 9 8 4 b ) an d B e rg e r ( 1 9 8 4 e ) . )
It was p e rhap s u n f ai r t o s p e n d s o m u c h t i m e on t hi s i s s u e , g i v e n
t hat Prof e s s or Lan e on l y b ri e f l y m e n t i on s c ohe re n c y . H o w e v e r , w e f e e l t hat i t
i s i m p ort an t t o v i e w Prof e s s or Lan e ' s ob j e c t i on s t o t he LP in t he l arg e r p e r
s p e c t i v e of al t e rn at i v e ap p roac he s t o t he B ay e s i an p arad i g m .
Le t us n ow t u rn t o Prof e s s or Lan e ' s s e c on d i s s u e , s p e c i f i c al l y
t he c ri t i c i s m s ab ou t t he " m od e l " p arad i g m an d t he ap p l i c ab i l i t y o f t he LP
wi t hi n i t . The f i rs t i s s u e Prof e s s or Lan e rai s e s is t hat of t he i n t e rp re t at i on
of , an d t he q u e s t i on of ap p l i c ab i l i t y of t he LP u n l e s s i s a " re al "
p hy s i c al p aram e t e r. W e wan t e d t o av oi d t he p hi l os op hi c al p rob l e m s i n he re n t i n
an y d i s c u s s i on of t he m e an i n g of p aram e t e rs , b u t in re t ros p e c t s hou l d hav e
s p e n t m ore t i m e on t he i s s u e . T h e re as on is t hat , whi l e of c ou rs e t he LP wi l l
ap p l y i f is a re al p hy s i c al p aram e t e r ( i n s om e s e n s e ) , it al s o ap p l i e s in
t he m u c h m ore c om m on s i t u at i on w h e r e i s on l y d e f i n e d b y s om e as p e c t of t he
e x p e ri m e n t . F or i n s t an c e , a v e ry l arg e p art of s t at i s t i c s d e al s wi t h s i t u a
t i on s i n v ol v i n g a s e ri e s o f ( ap p rox i m at e l y ) i . i . d . ob s e rv at i on s X ^ The
p aram e t e r i s of t e n i m p l i c i t e l y d e f i n e d b y t he as s u m e d d e n s i t y ( s a y ) , f
0
( x ^ ) >
f or t he ob s e rv at i on s , an d i s n o t , as Prof e s s or Lan e i m p l i e s , n e c e s s ari l y
1 9 4 THE LIK ELIHOOD PRINCIPLE
i d e n t i f i e d wi t h t he ov e ral l { P
> , whi c h c ou l d al s o i n v ol v e ot he r as p e c t s of t he
e x p e ri m e n t s u c h as t he s t op p i n g ru l e , p os s i b l e c e n s ori n g , an d s o f ort h. W e
c ou l d c on s i d e r an y n u m b e r o f e x p e ri m e n t s wi t h t he s am e i m p l i c i t e l y d e f i n e d ,
b u t wi t h d i f f e re n t { P
} , an d ap p l y t he W CP an d SP t o d e d u c e t he LP. In d e e d a
m aj or p u rp os e of t he LP is t o s how t hat f e at u re s of { P. } whi c h are i r r e l e v a n t
t o the i mpl i ci t de f i ni t i on of are ignorable in the analysis. Professor

Lane's t hree int e r pr e t at ions of do not cover t his case, which we would ca ll
t he case of major pr act ical i nt e r e st .
Of course, we did not mean the LP t o apply in Professor Lane's
case b ) , where is j ust an index se t , and spe ci f i ca l l y warned against t hi s on
several occasions. (The e nt i r e mixing setup makes no sense i f the parameters
in the two experiments can d i f f e r . ) Our f a i l ur e t o car e f ully def ine in
examples was admit t edly sloppy, but was based on the desire t o avoid complex
philosophical issues t hat are of uncert ain pr act ical import. (Convincing a
pr act icing st a t i st i c i a n, who rout inely uses models, t o base his analysis on
the observed li ke li hood f unct ion is a si gni f i cant pr act ical st ep. Informing
him t hat his model parameters r e a l l y have no meaning is unlikely t o cause much
improvement i n his st a t i st i c a l pr act i ce . )
Professor Lane next questions t he value of inf erence about model
parameters, arguing t hat pr e dict ive inf erence about f ut ure observables is of
most concern. We do not dispute t his point , but , as Professor Lane acknow
ledges, we do handle pr e dict ive inf erence by incorporat ing the f ut ure observ
able in . The complaint t hat t he LP does not then say how t o eliminat e is
one of t he arguments we use f or Bayesian implementation of the LP, but the
complaint in no way li mi t s or casts doubt on the LP. Professor Lane may pr ef er
the de Fi ne t t i approach, which allows di r e ct dealing wit h pr edict ive inf erence,
but , as discussed e a r l i e r , we f e e l the model based "half way house" is general
ly a pragmatic necessit y. I t is enormously d i f f i c ul t t o attempt di r e ct ly to
ascert ain such complicated things as pr edict ive di st r i but i ons. Even invent ing
crude models and a r t i f i c i a l l y creat ing model prior separations w i l l , we f e e l ,
s e r v e p r e d i c t i v e s t a t i s t i c i a n s b e s t i n t h e l o n g r u n . P r o f e s s o r L a n e d o e s
r a i s e t h e v a l i d p o i n t t h a t o u r e m p h a s i s t h r o u g h o u t t h e t e x t o n m o d e l p a r a m e t e r
i n f e r e n c e , i t s e l f , m a y b e m i s l e a d i n g . O u r o n l y d e f e n s e i s t h e e s s e n t i a l
i m p o s s i b i l i t y o f s e n s i b l y d i s c u s s i n g p r e d i c t i v e i n f e r e n c e o u t s i d e a B a y e s i a n
f r a m e w o r k , c o m b i n e d w i t h o u r d e s i r e t o m i n i m i z e B a y e s i a n i n v o l v e m e n t ( f o r
a l r e a d y m e n t i o n e d s o c i o l o g i c a l r e a s o n s ) .
W e h a v e t r i e d t o d e s c r i b e a c c u r a t e l y t h e r e a s o n s f o r o u r p r e f e r -
e n c e f o r t h e L P a p p r o a c h t o B a y e s i a n i t y . A d m i t t e d l y t h i s p r e f e r e n c e m a y b e
d u e t o o u r t r a d i t i o n a l p r o b a b i l i s t i c a n d s t a t i s t i c a l b a c k g r o u n d ( w i t h i t s
m o d e l o r i e n t a t i o n ) , b u t , o n t h e o t h e r h a n d , t h e a l t e r n a t i v e d e v e l o p m e n t s
h a v e n o t m a n a g e d t o p r o d u c e a n y b r o a d l y u s e f u l n e w p r a c t i c a l m e t h o d o l o g y .
T h e r e i s re a l d a n g e r i n l e t t i n g p h i l o s o p h i c a l g a m e s o b s c u r e t h e p r a c t i c a l
r e a l i t i e s o f t h e s i t u a t i o n . ( F o r i n s t a n c e , t h e c o h e r e n c y g a m e o f " b e t t i n g "
s e r v e s t o g i v e v a r i o u s s o u n d m e a n i n g s t o p r o b a b i l i t i e s , b u t i t s e e m s c o m p l e t e l y
b a c k w a r d s i n i t s a p p l i c a t i o n : p e o p l e d e c i d e h o w t o b e t b y f i r s t d e t e r m i n i n g
p r o b a b i l i t i e s , u s u a l l y t h r o u g h s o m e c o m p a r a t i v e l i k e l i h o o d m e t h o d . ) A
p h i l o s o p h i c a l g a m e t h a t c a n b e p l a y e d t o s u p p o r t t h e L P , f o u n d a t i o n a l l y , i s
t h e " f i n i t e s a m p l e s p a c e " g a m e ( s e e S e c t i o n 3 . 6. 1 ) . R e a l i t y a l w a y s h a s a
f i n i t e s a m p l e s p a c e , a n d t h e L P a l w a y s a p p l i e s t o t h e i m p l i e d " m o d e l . " T h i s
f o r m u l a t i o n h a s l i t t l e o p e r a t i o n a l s i g n i f i c a n c e , h o w e v e r , a n d s o w e d o n o t
v i e w i t a s a s e r i o u s a r g u m e n t f o r t h e L P a p p r o a c h .
In c o n c l u s i o n , a l t h o u g h w e c e r t a i n l y s u p p o r t , a n d i n d e e d f i n d
p h i l o s o p h i c a l l y e n l i g h t e n i n g , a p p r o a c h e s t o B a y e s i a n i t y b a s e d o n c o h e r e n c y ,
o u r o w n p r e f e r e n c e i s f o r t h e L P a p p r o a c h .
R E P L Y T O P R O F E S S O R L E C A M
T h e m a j o r a n d p r o b a b l y m o s t i m p o r t a n t p o i n t m a d e i n P r o f e s s o r
L e C a m ' s i n t e r e s t i n g d i s c u s s i o n i s t h a t w e s h o u l d b e "a b i t u n p r i n c i p l e d . " H e
s e e s v a l u e i n b o t h c l a s s i c a l m e t h o d s a n d t h e L P . A s " t o o l s " i n t h e s t a t i s t i -
c i a n ' s t o o l k i t , w e a g r e e t h a t t h e r e i s p o s s i b l e v a l u e i n c l a s s i c a l m e t h o d s ,
a l t h o u g h w e w o u l d t e n d t o p r e f e r B a y e s i a n t o o l s , i f a v a i l a b l e . T h e c h o i c e o f
a t ool i s , however, not r e a l l y t he question addressed in t he monograph. The
purpose of t he monograph was t o attempt t o c l a r i f y t he more fundamental
quest ion: What should t he st a t i st i c i a n be using his t ools f or? We believe
t he vast majorit y of st a t i st i c a l users want t o know "t he evidence about from
E and x, " and indeed wi l l l i k e l y be unable t o assign any ot her meaning t o a
st a t i st i c a l conclusion. Because of t he demonstrated c onf l i c t between t hi s
goal and t he f r e que nt ist goal of procedure performance ( except , of course, f or
t he various discussed except ions, such as experimental design) , we f e e l t hat
t hi s question of purpose can not be ignored. And while a var i e t y of t ools
may be useful in reaching our st at ed goal of t he det erminat ion of condit ional
evidence (even f r e que nt ist t ools may be useful see Section 5. 4 and also
Berger (1984b) and Berger ( 1 9 8 4 e ) ) , we would argue t hat t he value of t he t ool
must be r e lat e d t o t hi s ult imat e goal. The big stumbling block in t he long
running controversy in st a t i st i c s has been the lack of separat ion of purpose
and method.
A recurring theme in Professor LeCam's discussion is the issue of
communication of st a t i st i c a l evidence. Indeed, because we br i e f l y indicat e
in Chapter 5 t hat we f e e l i t necessary t o be Bayesians (and hence produce
priors and post e r i or s) , Professor LeCam int imat es t hat we have "argued. . . int o
a corner. " Our i nt e r pr e t a t i on, however, is t ha t , even i f communication of
evidence through Bayesian measures is deemed unappealing, i t is a scientific
necessity, unless one is wi l l i ng t o sa cr i f i ce t he goal of communicating the
act ual evidence obtained about . Of course, the Bayesian si t ua t i on (as
regards sc i e nt i f i c communication) is not nearly as bad as many non Bayesians
t hi nk; t he spectre of being f orced t o accept someone's unreasonable pr i or
di st r i but i on i s not r e a l l y an issue. Good Bayesian report ing can be done
wit h a var i e t y of strategems involving t he present at ion of t he conclusion f or
a wide var i e t y of pr ior s ( c . f . Dickey ( 1 9 7 3 ) ) . And simply present ing l i k e l i
hood f unct ions or, perhaps somewhat be t t e r , post erior dist r ibut ions f or
noninformative priors can be viewed as a reasonable condit ional communication
device. Of course, such are not t r a di t i ona l in sc i e nt i f i c j our nals, but
w e a l l k n o w o f a n u m b e r o f " t r a d i t i o n s " c o n c e r n i n g s t a t i s t i c a l r e p o r t i n g i n
s c i e n t i f i c j o u r n a l s t h a t w e w o u l d a l l g l a d l y r e t i r e .
A s t o P r o f e s s o r L e C a m ' s f e e l i n g t h a t o n e s h o u l d r e p o r t a l l p o s s i b l y
r e l e v a n t d a t a a b o u t a n e x p e r i m e n t , n o s u b j e c t i v i s t w o u l d t h i n k o f d i s a g r e e i n g .
A f t e r a l l , a s u b j e c t i v i s t i s ( t h e o r e t i c a l l y ) r e s p o n s i b l e f o r p r o d u c i n g h i s
l i k e l i h o o d s ( a s w e l l a s p r i o r s ) , a n d a l l d a t a a b o u t t h e e x p e r i m e n t c o u l d b e
r e l e v a n t t o t h i s e n t e r p r i s e . O f c o u r s e , t h e L P d o e s s a y t h a t i n p r o c e s s i n g
a l l t h i s i n f o r m a t i o n t h e c o n d i t i o n a l v i e w p o i n t s h o u l d h a v e p r i m a c y .
P r o f e s s o r L e C a m f e e l s t h a t a m a j o r fla w" i n t h e L P a x i o m a t i c s i s t h e
a s s u m p t i o n t h a t E v ( E , x ) e x i s t s . S i n c e w e a l l o w e d E v ( E , x ) t o b e a n y t h i n g * a n y
c o l l e c t i o n o f c o n c l u s i o n s o r r e p o r t s , w e a r e u n c l e a r a s t o t h e e x a c t o b j e c t i o n .
( O n e s u r e l y m u s t m a k e s o m e r e p o r t . ) A l l t h e a x i o m a t i c s s a y i s t h a t i f o n e
p r o c e s s e s i n f o r m a t i o n i n v i o l a t i o n o f t h e L P , p e r h a p s b y r e p o r t i n g f r e q u e n t i s t
e r r o r p r o b a b i l i t i e s , t h e n o n e i s b e h a v i n g i n v i o l a t i o n o f e i t h e r t h e W C P o r S P
o r b o t h . It i s , p e r h a p s , c o n c e i v a b l e t h a t , f o r e a c h e x p e r i m e n t , o n e c o u l d
p r o c e s s i n f o r m a t i o n i n a c o m p l e t e l y n e w w a y , s o t h a t o n e ' s E v ( E , x ) w o u l d b e
c o n t i n u a l l y c h a n g i n g , a n d s o t h a t n o v i o l a t i o n o f t h e W C P o r S P c o u l d b e
e s t a b l i s h e d . T h i s , h o w e v e r , i s n o t r e a l i s t i c : a s s t a t i s t i c i a n s w e a r e b o u n d
t o s t a n d a r d i z e m a n y o f o u r a n a l y s e s , o r a t l e a s t p a r t s o f m a n y o f o u r a n a l y s e s .
T h e t e x t a r g u e s t h a t a n y s u c h s t a n d a r d i z e d m e t h o d s o f p r o c e s s i n g i n f o r m a t i o n
s h o u l d b e i n a c c o r d w i t h t h e L P .
P r o f e s s o r L e C a m i s c e r t a i n l y c o r r e c t i n h i s c o m m e n t t h a t o u r
p a s s a g e f r o m t h e L P t o B a y e s i a n i t y i s m u c h w e a k e r t h a n t h e a r g u m e n t f o r t h e
L P . W e f e l t l i t t l e n e e d t o r i g o r o u s l y j u s t i f y t h i s f i n a l s t e p , m a i n l y b e c a u s e
w e f e e l t h a t i t i s b e l i e f i n t h e L P t h a t i s t h e m a j o r h u r d l e ; i t i s h a r d t o
a v o i d b e c o m i n g a B a y e s i a n a f t e r f u l l y a c c e p t i n g t h e L P .
P r o f e s s o r L e C a m f e e l s t h a t w e m a k e a d i r e c t a p p e a l t o f r e q u e n t i s t
i d e a s i n o u r a t t e m p t t o r e s o l v e t h e S t e i n e x a m p l e . W e c l e a r l y d i d a b a d j o b
i n t h e e x a m p l e , o f e x p l a i n i n g o u r p o s i t i o n , b e c a u s e P r o f e s s o r H i l l l i k e w i s e
s e e s u s a s r e s o r t i n g t o f r e q u e n t i s t r e a s o n i n g . T h e p a s s a g e t o w h i c h
P r o f e s s o r L e C a m r e f e r s w a s a n a t t e m p t t o e x p l a i n t o f r e q u e n t i s t s w h y , a s
1 9 7 . 1 THE LIK ELIHOOD PRINCIPLE
B ay e s i an c o n d i t i o n a l i s t s , we wou l d n ot s u f f e r f rom a f re q u e n t i s t p e rs p e c t i v e .
The c on d i t i on al B ay e s i an an al y s i s we d i s c u s s e d i n n o way d e p e n d e d on f re q u e n t
i s t e v a l u a t i o n s , howe v e r. F or a m ore l e n g t hy d i s c u s s i on of t hi s p o i n t , al on g
wi t h a b ri e f d e s c ri p t i on of t he rol e c on d i t i on al B ay e s i an s c an as c ri b e t o f re
q u e n t i s t m e a s u r e s , s e e ou r re p l y t o Prof e s s or H i l l .
The f i n al i s s u e rai s e d b y Prof e s s or Le Cam i s t hat of ap p l i c at i on of
t he LP whe n on l y " ap p rox i m at e l i k e l i hood s " are av ai l ab l e . W e hav e s e e n n o
e v i d e n c e t o i n d i c at e t hat t he n e e d t o u s e ap p rox i m at i on s wi t h t he c on d i t i on al
ap p roac h c au s e s an y m ore p rob l e m s t han t he u s e of ap p rox i m at i on s wi t h an y ot he r
ap p roac h. In t he e x am p l e of n i n d e p e n d e n t Cau c hy o b s e r v a t i o n s , we wou l d of
c ou rs e p re f e r u s e of t he e x ac t ob s e rv e d l i k e l i hood f u n c t i on , b u t i f n we re
e n orm ou s an d we had t e c hn i c al p rob l e m s in c al c u l at i n g an d u s i n g t he e x ac t
l i k e l i hood , we wou l d c e rt ai n l y c on s i d e r u s i n g t he 7((, ) ap p rox i m at i on . B u t
we wou l d u s e t he o b s e r v e d l i k e l i hood f u n c t i on f rom t hi s ap p rox i m at i on as t he
e x p e ri m e n t al i n p u t t o e v i d e n c e , n ot f re q u e n t i s t m e as u re s c al c u l at e d b y
av e rag e s ov e r t he n orm al ap p rox i m at i on . W i t hou t k n owi n g t he s p e c i f i c p rob l e m
on e c an n ot s af e l y re c om m e n d s p e c i f i c p r i o r s . W he n n i s l arg e , howe v e r, p ri or
i n f orm at i on wi l l t y p i c al l y b e v ag u e c om p are d wi t h t he l i k e l i hood f u n c t i on , s o
u s e of t he n on i n f orm at i v e u n i f orm p ri or wou l d b e a re as on ab l e f i rs t ap p rox i m a
t i on .
R E P L Y TO P R OF E S S OR L E C A M ' S S E C OND E DI TI ON C OM M E NTS
W e are s y m p at he t i c t o Prof e s s or Le Cam ' s p os i t i on , t hat at t e m p t i n g t o
s u m m ari z e a c om p l e x s i t u at i on b y t he p ai r ( E, x ) m ay om i t m u c h t hat i s re l e v an t .
Thu s we hav e al way s b e e n i n t e re s t e d in at t e m p t s t o d e p art f rom t he u s u al s t at i s
t i c al f ram e work of p rob ab i l i s t i c al l y m od e l l e d e x p e ri m e n t s ( t hou g h we hav e y e t
t o s e e an al t e rn at i v e f ram e work t hat work s b e t t e r ) . N o t e , howe v e r, t hat v i rt u
al l y al l of c l as s i c al s t at i s t i c s i s b as e d on c on s i d e ri n g p art i c u l ar n ot i on s of
Ev ( E, x ) . Thu s Prof e s s or Le Cam ' s ob s e rv at i on s wou l d s e e m t o ap p l y e q u al l y
we l l t o al l s t an d ard s t at i s t i c al c on c e p t s . He d oe s m e n t i on t he p os s i b l e n e e d
f or " i n t rod u c i n g in t he s y s t e m a v ari e t y of c on c e p t s t hat g o b e y on d p ai rs
( E, x ) " ; t he arg u m e n t t o ab an d on ( or at l e as t e x t e n d ) t he u s u al s t at i s t i c al
REPLY TO THE DISCUSSION 1 9 7 . 2
f ram e work i s t oo b i g f or u s .
Pe rhap s Prof e s s or Le Cam i s m ak i n g t he s m al l e r l og i c al p oi n t t hat
p ri n c i p l e s ( e . g . , t he L P ) t hat are d e d u c e d wi t hi n a t oo n arrow f ram e work are
n ot n e c e s s ari l y v al i d in t he c orre c t f ram e work . The c on s t ru c t i v e s i d e of t he
LP U
v
( ) s u m m ari z e s what i s n e e d e d f rom ( E, x ) ) m ay t hu s b e q u e s t i o n e d ; b u t
t he d e s t ru c t i v e s i d e of t he L P , t hat m e a s u r e s b as e d on ( E, x ) whi c h are
i n c om p at i b l e wi t h t he LP ( s u c h as f re q u e n t i s t m e a s u r e s ) are c o n t r a i n d i c a t e d ,
s e e m s i n t ac t . Af t e r a l l , a f re q u e n t i s t m e a s u r e b as e d on ( E, x ) s hou l d c e rt ai n
l y b e ab l e t o p as s an e v al u at i on in i t s own d om ai n . If it f ai l s t h e r e , it i s
hard t o i m ag i n e t hat i t wou l d b e g ood in an e n l arg e d d o m a i n .
W e hav e b e e n a b i t ov e rl y d og m at i c t o e m p h a s i z e ou r b as i c v i e w s .
At t he s am e t i m e , ou r p o s i t i o n , s t at e d in Se c t i on s 5 . 4 an d 5 . 5 , b e ars a c e rt ai n
s i m i l ari t y t o L e C a m ' s , in t hat we al s o d o n ot f e e l t hat al l ou r ac t i on s " m u s t
ab i d e b y t he L P . " Ou r own s u m m ary p o s i t i o n , h o w e v e r , i s t hat ab i d i n g b y t he LP
i s a g e n e ral l y g ood g u i d e l i n e , an d t hat m a j o r d e v i at i on s f rom t he LP are hi g hl y
s u s p e c t .
ADDITIONAL REFERENCES
IN TH E DISCUSSION
Listed here are those references in the discussion that are not in
the regular bibliographies (pp. 143-159 and 1 6 0 - 1 6 0 . 2 ) .
DICK EY , J. M. ( 1 9 7 3 ) . Scientific reporting. J. Roy. Statist. Soc. B 35, 2 85-
3 0 5 .
G EISSER, S. ( 1 9 8 4 ) . On the prediction of o b s e r v a b l e s : a selective update. In
Bayesian Statistics II, J. M . B e r n a r d o , e t . a l . ( e d s . ) . North-Holland.
H ILL, B . ( 1 9 6 5 ) . Inference about variance components in the one-way m o d e l .
J. Amer. Statist. Assoc. 58, 9 1 8 - 9 3 2 .
H ILL, B. ( 1 9 6 7 ) . Correlated errors in the random m o d e l . J. Amer. Statist.
Assoc. 62, 1 3 8 7 - 1 4 0 0 .
H ILL, B. ( 1 9 8 0 ) . Robust analysis of the random model and weighted least squares
r e g r e s s i o n . In Evaluation of Econometric Models, J. Kmenta and J. Ramsey
( e d s . ) . Academic P r e s s , New Y o r k .
H ILL, B . ( 1 9 8 5 - 8 6 ) . Some subjective Bayesian considerations in the selection
of models (with d i s c u s s i o n ) . Econometric Reviews 4, 1 9 1 - 2 8 8 .
H ILL, B . ( 1 9 8 7 c ) . DeFinetti's theorem, induction, and A ( n ) , or Bayesian
nonparametric predictive inference. To appear in Bayesian Statistics III,
J. M . B e r n a r d o , et. a l . ( e d s . ) . Oxford University P r e s s , Oxford.
H ILL, B . ( 1 9 8 8 ) . Bayesian data a n a l y s i s . Technical Report, Departments of
Statistics and Management Sc i e n c e , University of M i c h i g a n .
198
ADDITIONAL R E F E R E N C E S IN THE DISCUSSION 199
H I L L , B. and L A N E , D. ( 1 9 8 3 ) . C o n g l o m e r a t e ! i t y and c o u n t a b l e a d d i t i v i t y . To
appear in Essays in Honor of Bruno de Finetti* P. Goel and A. Zellner
( e d s . ) . N o r t h H o l l a n d , A m s t e r d a m .
J A M E S , W. ( 1 8 9 6 ) . The will to b e l i e v e . Reprinted in Essays in Pragmatism*
A l b u r e y Cas tell ( e d . ) . The H a f n e r Library of C l a s s i c s , N u m b e r 7 ( 1 9 6 6 ) .
L A N E , D. ( 1 9 8 1 ) . C o h e r e n c e and p r e d i c t i o n . Proceedings of the 43rd Session of
the ISI
%
8 1 9 6 .
L A N E , D. and S U D D E R T H , W. ( 1 9 8 4 ) . Coherent p r e d i c t i v e i n f e r e n c e . Sakhya.
S A V A G E , L. J. ( 1 9 6 2 ) . The Foundations of Statistical Inference, A Discussion.
M ethuen and C o . , L o n d o n .
S T O N E , M. ( 1 9 7 9 ) . Review and analys is o f s ome i n c o n s i s t e n c i e s related to
improper priors and finite a d d i t i v i t y . In Proceedings of the 6th
International Congress on Logic, Methodology, and Philosophy of Science,
L. J. Cohen et. a l . ( e d s . ) . N o r t h H o l l a n d , A m s t e r d a m .
I N D E X O F E X A M P L E S
E x a m p l e 1: 5 , 1 1 , 1 3 , 1 4
E x a m p l e 2 ( C o x ) : 6, 1 1 , 1 3 , 1 5 , 1 6 ,
4 5
E x a m p l e 3 : 7 , 1 5 , 17
E x a m p l e 4 a : 8
E x a m p l e 4 b : 8 , 17
E x a m p l e 5 ( Stein e s t i m a t i o n ) : 9
E x a m p l e 6 ( S t u d e n t ' s t ) : 1 2 , 13
E x a m p l e 7 : 1 3 , 1 5
E x a m p l e 8 : 1 7
E x a m p l e 9: 2 0 , 2 1 . 2 , 5 6
E x a m p l e 1 0 : 21
E x a m p l e 1 1 : 3 7
E x a m p l e 1 1 . 1 : 4 1 . 1 , 4 1 . 6
E x a m p l e 1 2 ( R e g r e s s i o n ) : 4 1 . 4
E x a m p l e 1 3 : 4 4
E x a m p l e 14 ( M a u l d o n ) : 1 4 , 4 8 , 4 9 ,
5 5 , 57
E x a m p l e 1 5 ( B l a c k w e l l , B u e h l e r ,
R u b i n ) : 61
E x a m p l e 1 6 : 6 9 , 1 3 8 , 1 3 9
E x a m p l e 1 7 : 7 0
E x a m p l e 1 8 : 7 2
E x a m p l e 1 9 : 7 3
E x a m p l e 1 9 . 1 : 7 4 . 1
E x a m p l e 2 0 ( A r m i t a g e ) : 7 5 , 7 7 , 7 8 , 8 0 ,
8 1 , 8 2 , 8 3
E x a m p l e 2 1 : 7 6 , 8 0 , 8 2 , 8 3 , 8 5
E x a m p l e 2 2 : 7 9
E x a m p l e 2 3 : 8 8
E x a m p l e 2 4 ( P r a t t ) : 9 1 , 9 3 , 9 5
E x a m p l e 2 5 : 9 2 , 9 5
E x a m p l e 2 6 : 9 3
E x a m p l e 2 7 : 9 3 , 9 5 , 9 6 , 1 0 2
E x a m p l e 2 8 : 9 4
E x a m p l e 2 9 : 9 5 , 9 6 , 9 9 , 1 0 2 , 1 0 3
E x a m p l e 3 0 : 1 0 6 , 1 0 8 , 1 0 9 . 1
E x a m p l e 3 0 . 1 : 1 0 7
E x a m p l e 3 0 . 2 : 1 0 9 . 1
E x a m p l e 3 1 : 1 1 3 , 1 1 4 , 1 1 6 , 1 1 7
E x a m p l e 3 2 : 1 1 6
E x a m p l e 3 3 : 1 1 7 , 1 1 8 , 1 1 9
E x a m p l e 3 4 ( F r a s e r - M o n e t t e - N g ) : 1 2 7 ,
1 3 5 , 1 6 7 - 1 7 1 , 1 8 9
E x a m p l e 35 ( S t o n e ) : 1 2 9 , 1 3 5 , 1 3 6 ,
1 6 5 - 1 6 7
E x a m p l e 3 6 ( S t e i n ) : 1 3 3 , 1 3 5 , 1 6 2 - 1 6 4 ,
1 8 5 , 1 8 8 , 1 9 7 - 1 9 8
E x a m p l e 3 7 : 1 3 9
200
A U T H O R I N D E X
A I T C H I S O N , J . 4 0 , 1 8 0 B E R N A R D O , J . M . 6 8 , 1 3 8
A I T K I N , M . 4 1 . 4 , 1 2 5 B E R R Y , D . 6 7 , 7 4 . 1
A K A I K E , M . 5 6 B H A V E , S . V . 2 6
A M A R I , S . 1 4 B I R N B A U M , A . 1 , 2 , 2 1 . 1 , 2 2 , 2 4 , 2 5 ,
2 6 , 2 7 , 2 9 , 3 0 , 4 2 , 4 6 , 4 7 , 6 6 ,
A N D E R S E N , E . B . 1 1 , 1 4 , 1 2 3 7 1 , 7 4 , 9 1 , 1 2 3 , 1 2 4 , 1 2 5 , 1 7 4 . 2 ,
1 7 5 , 1 7 8
A N S C O M B E , F.J. 7 4 , 7 8 , 1 7 4 . 1
B L A C K W E L L , D . 61
A R M I T A G E , P. 8 0
B O N D A R , J.V . 1 3 , 6 0 , 1 3 3
A S P I N , A . A . 12
B O X , G . E . P . 6 7 , 6 8 , 1 0 9 . 4 , 1 2 2 , 136
B A H A D U R , R . R . 3 8 B R O W N , L . D . 1 2 , 1 4 , 17
B A R N A R D , G.A. 1 , 1 0 , 1 4 , 1 8 , 2 2 , 4 7 , B R O W N I E , C . 1 4
4 8 , 4 9 , 7 1 , 7 4 , 8 3 , 1 0 5 , 1 0 6 ,
1 0 9 . 4 , 1 2 3 , 1 4 0 B U E H L E R , R . J . 1 2 , 1 3 , 1 4 , 6 0 , 6 1 , 62
B A R N D O R F F - N I E L S E N , 0 . 1 1 , 1 4 , 2 6 , B U N K E , H . 4 8
4 1 . 3 , 5 2 , 123
B U T L E R , R. 4 0 , 4 1 . 2
B A R N E T T , V . 1 4 , 1 8 , 2 2 , 1 0 5 , 1 2 3
B A R T H O L O M E W , D . J . 7 4 C A S E L L A , G . 9, 1 0 9 , 1 0 9 . 4
B A R T L E T T , M . S . 2 2 C A S S E L , C M . 117
B A S U , D . 1 4 , 2 2 , 2 5 , 2 6 , 2 8 , 2 9 , 4 2 , C O R N F I E L D , J. 6, 1 2 , 6 0 , 7 4 , 1 1 7
4 7 , 7 4 , 8 0 , 1 1 5 , 1 1 7 , 1 1 9 , 1 2 0 ,
1 2 6 , 1 3 5 , 1 7 5 , 176 C O X , D . R . 6, 1 1 , 1 4 , 2 2 , 2 6 , 4 1 . 3 ,
7 1 , 1 0 5 , 1 0 6 , 1 0 7 , 1 2 3
B A Y A R R I , M . J . 2 1 . 2 , 3 9 , 4 1 . 2 , 1 6 0 . 6 ,
1 8 7
D A W I D , A . P . 1 4 , 1 8 , 2 2 , 2 5 , 2 6 , 4 1 . 3
B E R G E R , J. 1 4 , 1 6 , 1 7 , 1 8 , 2 4 , 2 6 , 4 7 , 4 8 , 1 0 4 , 1 7 5 , 1 7 7
3 7 , 4 1 . 1 , 4 7 , 4 9 , 5 2 , 5 6 , 6 7 , 6 8 ,
6 9 , 7 0 , 7 1 , 7 4 , 7 4 . 1 , 1 0 5 , 1 0 8 , D E F I N E T T I , B . 5 5 , 6 8 , 7 1 , 1 7 0 , 172
1 0 9 , 1 0 9 . 2 , 1 1 6 , 1 2 1 , 1 3 3 , 1 3 4 ,
1 3 6 , 1 3 7 , 1 3 8 , 1 4 0 , 1 9 0 , 1 9 3 , 196 D E G R O O T , M . H . 2 1 . 2 , 3 9 , 4 1 . 2 , 1 0 9 ,
1 0 9 . 4 , 1 6 0 . 6 , 1 8 7
B E R G E R , R. 1 0 9 , 1 0 9 . 4
D E L A M P A D Y , M . 1 0 5 , 1 0 9 , 1 0 9 . 2
B E R L I N E R , L.M. 3 9 , 1 3 8
2 0 1
202
A U T H O R INDEX
D E M P S T E R , A . P . 1 1 , 1 0 5 , 1 0 6 , 1 1 0
D I C K E Y , J.M. 1 0 4 , 1 0 9 , 1 0 9 . 4 , 1 3 1 ,
1 3 6 , 1 9 6
D U R B I N , J. 4 5 , 1 7 5
D U N S M O R E , I.R. 4 0 , 1 8 0
E D W A R D S , A . W . F . 2 2 , 7 1 , 1 0 6 , 1 2 3
E D W A R D S , W . 7 4 , 7 8 , 1 0 5 , 1 0 7 , 109
E F R O N , B . 1 1 , 1 4 , 1 3 8
E R I C S O N , W . A . 1 1 7
E V A N S , M . 2 6 , 4 7 , 1 2 7
F E D D E R S O N , A . P . 12
F I S H E R , R . A . 1 , 1 1 , 1 2 , 1 3 , 1 4 , 1 8 ,
1 9 , 2 2 , 1 1 7 , 1 2 2 , 1 2 3
F O L K E S , J . L . 1 1 7
F R A S E R , D . A . S . 1 2 , 1 8 , 2 6 , 4 7 - 5 0 , 6 1 ,
6 2 , 1 0 9 . 4 , 1 2 7 , 1 3 1 , 1 3 2
F R E E D M A N , D . A . 6 0
G E I S S E R , S . 4 0 , 1 8 0
G I E R E , R . N . 71
G O D A M B E , V . P . 4 2 , 4 7 , 7 0 , 7 1 , 1 1 7
G O O D , I.J. 4 7 , 1 0 5 , 1 0 9 , 1 2 0 , 1 2 2 ,
1 2 4 , 1 3 7 , 1 3 8 , 1 7 4 . 3
G R A M B S C H , P. 1 4 , 123
H A C K I N G , I. 1 8 , 7 1 , 1 0 5 , 1 0 9 . 3 , 1 2 3
H A J E K , J. 1 4 0
H A L L , P. 1 0 5
H A R T I G A N , J. 1 6 0 . 2
H E A T H , D . 6 0 , 6 4 , 1 6 5
H E N K E L , R . E . 1 0 6
H I L L , B . 2 1 , 2 1 . 1 , 5 2 , 5 5 , 6 2 , 1 2 0 ,
1 2 4 , 1 2 9 , 1 3 0 , 1 3 1 , 1 3 2 , 1 6 4 -
1 6 6 , 1 7 3 - 1 7 4 . 4
H I N D E , J. 4 1 . 4 , 1 2 5
H I N K L E Y , D . V . 1 1 , 1 4 , 2 2 , 4 0 , 4 1 . 3 ,
7 1 , 1 0 5 , 1 2 3
H W A N G , J . T . 9
J A M E S , W . 9, 57
J A M E S , W i l l i a m 1 7 4 . 4
J A Y N E S , E . T . 6 8 , 7 1 , 1 2 7 , 1 3 8
J E F F R E Y S , H . 1 1 , 2 1 , 2 3 , 5 6 , 6 8 , 7 1 ,
8 2 , 1 0 6 , 1 0 8 , 1 0 9 , 1 0 9 . 4 , 1 2 9
J E N K I N S , G . M . 2 2 , 4 7 , 7 4 , 1 2 3 , 1 4 0
J O S H I , V . M . 2 9 , 4 2
K A D A N E , J. 2 1 . 2 , 3 9 , 4 1 . 2 , 1 6 0 . 6 , 1 8 7
K A L B F L E I S C H , J . D . 4 0 , 4 1 . 3 , 4 6 , 4 7 ,
1 1 7 , 1 2 3 , 1 2 4
K A Y , R. 4 1 . 3
K E M P T H O R N E , 0 . 1 1 7
K I E F E R , J. 1 4 , 1 6 , 1 7 , 71
L A N E , D . A . 6 0 , 6 4 , 1 6 4 , 1 6 5 , 1 7 2 , 1 7 6 ,
1 8 0
L A U R I T Z E N , S . L . 4 0
LE C A M , L . 7 1 , 1 9 2
L E V I , I. 6 0 , 1 2 4
L I N D L E Y , D . V . 1 0 , 2 1 , 5 5 , 8 2 , 1 0 9 , 1 2 0
L I N D M A N , H . 7 4 , 7 8 , 1 0 5 , 1 0 7 , 1 0 9
M A C K A Y , J. 1 0 9 . 4
M A U L D O N , J . G . 4 8
M I L N E S , P. 1 3 3
M O N E T T E , G. 2 6 , 4 7 , 1 2 7
M O R R I S O N , D . F . 1 0 5
N E Y M A N , J. 4 5 , 6 6 , 7 1 , 7 4
A U T H O R INDEX 203
N G , K .W. 1 2 7
N O V I C K , M . 1 2 0
O L S H E N , R . A . 12
P E D E R S E N , J . G . 1 3 , 1 8
P E I S A K O F F , M . 1 3 3
P H I L L I P S , L . D . 21
P I C C I N A T O , L . 1 0
P I E R C E , D . A . 1 3 , 6 0
P L A C K E T T , R . L . 1 8
P L A N T E , A . 1 2 4
P O C O C K , S .J. 7 4 . 1
P R A T T , J.W. 8 , 1 0 , 1 1 , 1 4 , 1 5 , 2 2 ,
7 0 , 7 1 , 7 4 , 8 8 , 9 1 , 1 0 5 , 1 0 9 ,
1 0 9 . 3 , 1 0 9 . 4 , 116
P U R V E S , R . A . 60
R A I F F A , H . 9 0
R A O , C . R . 1 1 5 , 1 1 6 , 1 1 7
R O B I N S O N , G . K . 1 0 , 1 3 , 5 9 - 6 2
R O S E N K R A N T Z , R . D . 6 8
R O Y A L L , R. 1 1 5 , 1 1 7
R U B I N , D . B . 1 1 5 , 1 2 0
R U B I N , H . 61
S A R N D A L , C . E . 1 1 7
S A V A G E , L . J . 2 , 1 0 , 4 6 , 6 6 , 7 4 , 7 8 ,
8 3 , 9 1 , 1 0 5 , 1 0 7 , 1 0 9 , 1 2 0 - 1 2 1 ,
1 2 4 , 1 3 9 , 1 6 3 , 1 7 4 . 2
S C H L A I F E R , R . 9 0
S E I D E N F E L D , T . 1 4
S E L I N G E R , B . 1 0 5
S E L L I A H , J. 5 7
S E L L K E , T . 1 0 8 , 1 0 9
S H A F E R , G . 1 0 9
S M I T H , A . F . M . 1 0 9
S M I T H , C . A . B . 121
S M I T H , T . M . F . 1 1 7
S P I E G E L H A L T E R , D . J . 1 0 9
S P R O T T , D . A . 1 8 , 4 1 . 3 , 4 8 , 1 1 7 , 1 2 3 ,
1 2 4
S T E I N , C . 9, 1 0 , 1 3 , 5 7 , 1 3 3 , 1 3 4 ,
1 6 2
S T O N E , M . 1 8 , 4 8 , 1 2 9 , 1 3 0 , 1 3 1 , 1 3 2 ,
1 7 2
S U D D E R T H , W . 6 0 , 6 4 , 1 6 5 , 1 7 2 , 1 7 6 ,
180
T H O M P S O N , M . E . 7 0 , 1 1 7
T I A O , G. 6 8 , 1 2 2 , 136
T J U R , T . 1 2 3
W A L L A C E , D . L . 1 3 , 6 0
W E L C H , B . L . 1 0 , 1 2 , 1 4
W I L K I N S O N , G . N . 1 8 , 4 7
W I N S T E N , C . B . 2 2 , 4 7 , 7 4 , 1 2 3 , 1 4 0
W R E T M A N , J . H . 1 1 7
Y A T E S , F. 12
Z E L L N E R , A . 6 8 , 1 0 9 , 1 0 9 . 4
S U B J E C T I N D E X
A d h o c k e r i e s 1 2 4
A d m i s s i b i l i t y 4 7 , 5 0 5 9 , 8 9 8 3 , 1 2 8 ,
1 7 4 . 4
A n c i l l a r i t y 1 3 , 1 4 , 1 6 , 1 7 , 2 5 , 4 1 . 3 ,
4 1 . 4
B a y e s i a n s 1 1 4 , 1 1 5 , 1 6 0 . 7 , 1 7 4
Us e o f f r e q u e t i s t m e a s u r e s 6 9 , 7 0 ,
1 6 2 , 1 6 4 , 1 7 4 , 1 8 5 , 1 8 7 1 9 0 ,
1 9 7
In s e q u e n t i a l e x p e r i m e n t s 8 1 , 8 2
R o b u s t 1 3 6 1 4 0
s e e a l s o L i k e l i h o o d m e t h o d s
N o n i n f o r m a t i v e p r i o r s
B e t t i n g 5 9 , 1 6 8 1 7 1 , 1 9 1 , 1 9 2 , 1 9 5
C e n s o r i n g 9 0 1 0 4
F i x e d 9 2 9 5
R a n d o m 9 5 1 0 4
N o n i n f o r m a t i v e 1 0 3 , 1 0 4
C e n s o r i n g m e c h a n i s m s
E q u i v a l e n t 9 3 9 5
N o n i n f o r m a t i v e 9 1 , 9 6 1 0 3
C h i s q u a r e t e s t 1 0 9 . 1 , 1 0 9 . 2
C o h e r e n c y 4 7 , 4 8 , 5 0 , 5 9 6 4 , 1 2 1 ,
1 2 2 , 1 2 8 , 1 6 5 , 1 6 8 , 1 7 2 ,
1 9 2 1 9 3
C o n d i t i o n i n g 3 , 5 1 8 , 2 2 , 2 4 , 1 7 4 . 1
C o n d i t i o n a l f r e q u e n t i s t 1 4 1 7 , 2 2
C o n d i t i o n a l c o n f i d e n c e 1 4 , 1 5 , 17
E s t i m a t e d c o n f i d e n c e 1 6 , 17
C o n f i d e n c e s e t s 5 , 6, 9, 1 5 , 1 6 , 5 5 ,
6 1 , 6 2 , 6 6 , 6 9 , 7 0 , 7 2 ,
1 1 3 1 3 5
D e s i g n o f e x p e r i m e n t s 3 6 , 4 0
E v a l u a t i o n g a m e 5 2 5 9 , 6 1 , 6 2
E v i d e n c e 2 5 , 3 1 , 3 7 , 4 1 , 4 1 . 5 , 4 5 , 6 6 ,
6 8 , 1 6 0 . 7 , 1 7 4 1 7 4 . 1 , 1 8 3
1 8 5 . 2 , 1 9 6 1 9 7 . 2
E v ( E , x ) 2 5
E v
v
J E , x ) 37
y >
E x p e r i m e n t 2 4 , 2 9 , 3 1 , 1 7 6 1 7 8 ,
1 9 3 1 9 4
N o n i n f o r m a t i v e 2 1 . 2 , 4 2 , 8 9
(X, , { f
Q
} ) 2 4
F i d u c i a l i n f e r e n c e 1 8
F i n i t e p o p u l a t i o n s a m p l i n g
s e e R a n d o m i z a t i o n a n a l y s i s
F r e q u e n t i s t B a y e s i a n 2 3 , 2 4
F r e q u e n t i s t v i e w p o i n t 5 1 8 , 6 5 7 4 . 2 ,
7 7 8 3 , 1 0 5 1 0 9 , 1 2 7 1 3 6 ,
1 8 2 1 8 3
204
S U B J E C T INDEX
205
G o o d S a m a r i t a n 1 3 2
H o a x 7 8
s e e a l s o F r e q u e n t i s t v i e w p o i n t
I n v a r i a n c e 3 0 , 1 3 3
L i k e l i h o o d f u n c t i o n 1 9 2 2 , 3 0 , 3 8 , 3 9 ,
4 0 , 4 3 , 4 4 , 6 7 , 7 6 , 8 2 , 9 7 , 1 1 3 ,
1 1 4 , 1 1 6 , 1 2 2 1 2 6 , 1 3 3 , 1 3 5 , 1 3 6 ,
1 4 1 , 1 6 0 . 3 1 6 0 . 7 , 1 8 6 1 8 7
M a r g i n a l 4 1 . 1 , 4 1 . 2
P a r t i a l 4 1 . 3
R e d u c e d 4 1 . 2
L i k e l i h o o d m e t h o d s
B a y e s i a n 2 3 , 4 1 . 2 , 4 1 . 5 , 1 2 4 1 4 1
N o n B a y e s i a n 1 1 , 1 4 , 1 8 , 3 9 , 4 1 . 3 ,
4 1 . 4 , 1 2 2 1 2 4
L i k e l i h o o d r a t i o s 2 1 , 2 2 , 3 3 , 1 0 7 , 1 2 3
W e i g h t e d 1 0 8 , 1 0 9
M i x e d e x p e r i m e n t s 6, 3 1 , 4 6 , 4 7 , 5 1 ,
5 7 , 6 3 , 8 4 , 8 5
E * 31
M o d e l s 4 3 , 6 7 , 7 2 , 1 1 4 , 1 1 5 , 1 2 0 ,
1 2 6 , 1 2 7 , 1 6 0 . 5 , 1 6 0 . 6 , 1 7 6 1 8 1 ,
1 8 5 . 1 , 1 8 5 . 2 , 1 9 3 1 9 4 , 1 9 7 . 1
N o n i n f o r m a t i v e p r i o r 2 1 , 4 1 . 3 , 5 2 , 5 6 ,
6 5 , 6 7 6 9 , 8 0 , 1 2 9 , 1 3 0 , 1 3 3 , 1 3 5 ,
1 3 6 , 1 4 0 , 1 6 1 1 7 4 , 1 8 8 1 9 0
N u i s a n c e p a r a m e t e r 4 , 1 3 , 3 6 , 3 7 , 4 1
4 1 . 6 , 1 6 0 . 5
N o n i n f o r m a t i v e 4 1 . 5 , 8 9 , 9 0 , 1 1 5 ,
1 2 5
O b j e c t i v i t y 6 7 , 6 8 , 8 0 , 1 7 4 . 3
O p t i o n a l s t o p p i n g 7 7 8 3
s e e a l s o S t o p p i n g r u l e s
P v a l u e , s e e S i g n i f i c a n c e t e s t i n g
P i v o t a l and S t r u c t u r a l i n f e r e n c e 1 8 ,
2 5 , 4 7 5 1 , 5 4 5 9
P r e d i c t i o n 3 6 3 9 , 4 1 . 1 , 1 8 0 1 8 1 , 1 9 4
P R I N C I P L E S
A d e q u a c y 8 , 15
C e n s o r i n g 1 ( C P 1 ) 9 4 9 5
C e n s o r i n g 2 ( C P 2 ) 9 7 1 0 3
C o n d i t i o n a l i t y ( C P ) 1, 2 5 , 3 0
W e a k ( W C P ) 2 5 , 2 6 , 2 8 , 3 1 , 3 3 ,
3 5 , 3 6 , 4 5 , 4 6 , 5 1 , 5 7 , 6 3 ,
8 4 , 8 5
C o n f i d e n c e 6 6 6 7 , 7 1 7 4
Formal 7 1 , 7 2
D i s t r i b u t i o n ( D P ) 47
L i k e l i h o o d ( L P ) 1 9 , 1 7 4
F o r m a l 2 6
N u i s a n c e v a r i a b l e 41
R e l a t i v e ( R L P ) 3 2 3 6 , 4 4 , 6 3 ,
6 4 , 8 7 , 9 8 1 0 1 , 141
M a r g n a l i z a t i o n 4 1 . 1
N o n i n f o r m a t i v e n u i s a n c e p a r a m e t e r
( N N P P ) 4 1 . 5 , 9 0 , 97
S t o p p i n g r u l e ( S R P ) 3 3 , 4 0 , 7 4 , 7 6 ,
8 7 , 1 7 4 . 1
G e n e r a l 8 6
S u f f i c i e n c y ( S P ) 1, 2 9 , 3 0 , 3 2 ,
3 5 , 4 6 5 0 , 1 7 8 1 7 9 , 1 9 2
W e a k ( W S P ) 2 6 , 2 8
M a t h e m a t i c a l e q u i v a l e n c e 2 6
P r o c e d u r e s 4 1 , 6 8 7 0
R a n d o m e f f e c t s 4 1 . 1
R a n d o m i z a t i o n a n a l y s i s 1 1 0 1 2 0
F i n i t e p o p u l a t i o n s a m p l i n g 1 1 1 1 1 7
206 S U B J E C T INDEX
R a n d o m i z a t i o n t e s t i n g 1 1 7 - 1 2 0
R e l e v a n t s u b s e t s 1 1 - 1 6
R e p e a t a b i l i t y 7 0 , 71
R i s k
B a y e s i a n 3 2
F r e q u e n t i s t 5 2
R o b u s t n e s s 1 0 , 1 1 , 4 4
S e q u e n t i a l a n a l y s i s 3 7 - 4 0 , 7 9 - 9 0 ,
1 7 4 . 1
S i g n i f i c a n c e t e s t i n g 7 , 7 4 . 1 , 7 4 . 2 ,
1 0 4 - 1 1 0
S t e i n p h e n o m e n o n 9, 1 7 4
S t o p p i n g r u l e s 2 0 , 4 0 , 7 4 - 9 0
D e f i n i t i o n s 7 5 , 8 6
I n f o r m a t i v e 8 8 - 9 0
S t r u c t u r a l i n f e r e n c e
s e e P i v o t a l and S t r u c t u r a l
S u f f i c i e n c y 7 , 1 8 , 1 9 , 2 4 , 3 1 , 3 2 ,
3 5 , 3 8 , 4 5 , 57
U n o b s e r v e d v a r i a b l e s 3 6 - 4 1 , 4 1 . 1
207
E R R A T A A N D C L A R I F I C A T I O N S
T h e n o t a t i o n " 1 7 " m e a n s p a g e 1 7 , l i n e 3 f r o m t h e t o p ; " 1 7 " m e a n s
p a g e 1 7 , l i n e 4 f r o m t h e b o t t o m .
Q
p . 2 : i m p u n e > i n p u g n
p . I I
1 1
: I ( ) "
1
. x ) "
1
p . I I
1 2
: I ( ) "
1
I ( ) "
1
p . 1 6 ^ : . . . f o r g e t a b o u t x . . . > . . . f o r g e t a b o u t x i n e v a l u a t i n g t h e p r o c e d u r e . .
p . 1 8
4
: P
1
P
Q
2
p . 4 5 : p r i n c i p a l + p r i n c i p l e
p . 4 9 : N o t e t h a t a s t r u c t u r a l i s t o r p i v o t a l i s t w o u l d n o t c o n s i d e r S t o b e
s u f f i c i e n t ,
p . 2 7
7
: SP + W S P
p . 4 5 : p r i n c i p a l + p r i n c i p l e
p . 5 3 : f o r a n y , > f o r e v e r y
p . 7 3
8
: . 01 . 0 1 0 1
p . 7 5
1 Q
: c h a n g e t o " A^ ( A. x 2
m
~
J
) = 0 "
p . 7 7
1 1
: l o o k s + l o o k
p . 8 4
4
: A d d " f o r n = n * a n d x
n
* 6 A * "
p . 8 6
9
: b y > b e
p . 8 6 5 : IN d e n o t e s t h e n a t u r a l n u m b e r s { 0 , 1 , 2 , 3 . . . } .
p . 8 8
1 0
, 8 9
7
, 8 9
1 0
: a u x i l i a r y > a u x i l i a r y
p . 9 3 : l a r g e s t * s m a l l e s t
2 0 8 ERRATA A N D C L A R I F I C A T I O N S
p . 9 7
8
: Se c t i o n 3 . 5 . 5 > Se c t i o n 3 . 5 . 3 .
p . 9 7
1 5
, p 9 7
n
, p . 9 7
y
: C h a n g e t o " E v
( E
9
' \ y ) "
P . 9 8
C
: R e p l a c e b o t h o c c u r r e n c e s o f " Ev " w i t h "Ev "
D
p . 9 9
7
: c ( ( x , ) ) c ( g ( x , ) )
p . 1 1 9
1 3
: c h a n g e t o " e x p e r i m e n t e r i n d u c e d "
p . 1 2 5
1 0
: A i t k e n > A i t k i n
p . 1 2 6 , 1 3 6 . , : N o t e t h a t
( ) h a s t o b e p r o p e r l y n o r m a l i z e d t o b e a n a c t u a l
p r o b a b i l i t y d e n s i t y w i t h r e s p e c t t o .
p . 1 3 4 : i n a p p r o p r i a t e > a p p r o p r i a t e
p . 1 3 6 . 0 ^ i n e x o l o r a b l y > i n e x o r a b l y

Berger The Likelihho Principle 1988 Book

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Berger The Likelihho Principle 1988 Book

Загружено:

Авторское право:

Доступные форматы

Institute of Mathematical Statistics

LECTURE NOTES-MONOGRAPH SERIES

Conditionally, and Sufficiency 24

corrects t hi s gl ar i ng problem, but the point is that a procedure which looks

wi l l be assumed t o be Borel measures. Of course, X of t en arises as

example, i n case a (v almost everywhere) continuous density exists we could

= {open neighborhoods of x %} and put

a ppl i e s t o t h i s compl e t e l y nonpar ame t r i c se t up, as we l l as t o any s i t u a t i o n

wit h P^( A) > 0 f or a l l ,

A c ondi t i ona l Bayesian ( f o r s i mp l i c i t y ) would, f o r a p r i o r d i s t r i b u t i o n on ,

stopping rule, , which can be r epr esent ed by a sequence of set s,

f or a l l . Examples 2 0 and 2 1 are of t hi s t ype, since the

, respect ively, are decision rules

. I f the WCP is f ollowedf or E* and ( 4 . 2 . 5 ) holds, then

, . . . , E , whet her or not t o pe r f or m E , , . As usua l ,

i nst e a d of X , one observes Y = g ( X ) , where g i s a known f unc t i on f rom % i nt o

i nt o t he measurable r e ct a ngl e s ( or product se t s)

> y j >. . . >y i )> w h i c h i s a s u b s e t o f % h e r e s = {ij. , i

(y,W,,) = f(x|,) f(y,w|,).

(y,,,) = f(x|,) f(y,w|x,,)

requiring e i t he r more st ruct ure (as do st r uc t ur a l i st s, pivot eers, and, perhaps,

c ) is a p o s s i b l e v a l ue fo r s o m e " r ea l " p hys i c a l p a r a m eter , a n d P

b e r eg a r d ed a s the d i s tr i b uti o n o f the r a n d o m q ua n ti ty X s ho ul d b e the

t o the i mpl i ci t de f i ni t i on of are ignorable in the analysis. Professor

Вам также может понравиться