ICOTS 2, 1986: Terry Speed

QUESTIONS, ANSWERS AND STATISTICS

T e r r y Speed
CSIRO Division of Mathematics and Statistics
Canberra, Australia

A major point, o n which I cannot y e t hope for universal agreement,

i s that o u r focus must b e 'on questions, not models. ...
Models can
- and will - g e t u s in deep troubles ifwe expect them t o t e l l us what
the unique proper questions are.
J.W. Tukey (1977)

.
1 Introduction

I n my view the value o f statistics, by which 1 mean both data and the tech-
niques we use t o analyse data, stems from i t s use i n helping us t o give
answers of a special t y p e t o more or less well defined questions. This is
hardly a radical view, and not one with which many would disagree violent-
ly, y e t I believe t h a t much of the teaching of statistics and not a l i t t l e sta-
tistical practice goes on as if something quite different was the value of
statistics. Just what the other t h i n g is I f i n d a l i t t l e hard t o say, b u t it
seems t o be something like this: t o summarise, display and otherwise ana-
lyse data, o r t o construct, fit, test and evaluate models f o r data, presum-
ably i n the belief t h a t if t h i s is done well, all (answerable) questions i n -
volving the data can then be answered. Whether t h i s is a fair statement o r
not, it is certainly t r u e t h a t statistics and other graduates who f i n d them-
selves working with statistics i n government or semi-government agencies,
business o r industry, i n areas such as health, education, welfare, econom-
ics, science and technology, are usually called upon t o answer questions,
not t o analyse o r model data, although of course the latter will i n general
be p a r t of their approach t o providing the answers. The interplay be-
tween questions, answers and statistics seems t o me t o be something which
should interest teachers of statistics, f o r if students have a good apprecia-
tion of this interplay, they will have learned some statistical thinking, not
just some statistical methods. Furthermore, I believe t h a t a good under-
standing of this interplay can help resolve many of the difficulties common-
ly encountered i n making inferences from data.

My primary aim i n this paper is quite simple. I would like t o encourage you
t o seek o u t o r attempt t o discern the main question of interest associated
with any given set of data, expressing this question i n the (usually non-
statistical) terminology of the subject area from whence the data came, be-
fore you even t h i n k of analysing or modelling the data. Having done this,
I would also like t o encourage you t o view analyses, mode.ls etc. simply as
means towards t h e end of providing an answer t o the question, where
again the answer should be expressed in the terminology of the subject
area, although there will always be the associated statement of uncertainty
which characterises statistical answers. Finally, and regrettably t h i s last
point is b y no means superfluous, I would then encourage you t o ask y o u r -
ICOTS 2, 1986: Terry Speed

self w h e t h e r t h e answer y o u gave r e a l l y did answer t h e question o r i g i n a l l y

posed, a n d n o t some o t h e r question.

A secondary aim, w h i c h I cannot hope t o achieve i n t h e time p e r m i t t e d t o

me, w o u l d b e t o show y o u how many common d i f f i c u l t i e s experienced in a t -
t e m p t i n g t o d r a w inferences from data can b e resolved by c a r e f u l l y f r a m i n g
t h e q u e s t i o n o f i n t e r e s t a n d t h e f o r m o f answer sought. A f e w remarks o n
t h i s aspect a r e made i n Section 6 below.

2. W h y speak on this topic?

O v e r t h e y e a r s I have h a d many experiences w h i c h have lead me t o t h i n k

t h a t t h e i n t e r p l a y between questions, answers a n d s t a t i s t i c s i s w o r t h y o f
consideration. L e t me b r i e f l y mention f o u r , each o f a d i f f e r e n t t y p e .

T h e f i r s t experience i s a common one f o r me. Someone i s d e s c r i b i n g an

application o f s t a t i s t i c s i n some area, say biology. T h e speaker u s u a l l y
b e g i n s w i t h an o u t l i n e o f t h e b a c k g r o u n d science a n d goes o n t o g i v e an
o f t e n detailed d e s c r i p t i o n o f t h e data a n d how t h e y were collected. T h i s
p a r t i s new a n d i n t e r e s t i n g t o a n y statisticians listening, most o f whom w i l l
b e unfamiliar w i t h t h a t p a r t i c u l a r p a r t o f biology. Sometimes t h e biologist
who collected t h e data i s p r e s e n t a n d c o n t r i b u t e s t o t h e explanation, b u t
a t a c e r t a i n stage t h e statistician s t a r t s t o e x p l a i n w h a t she/he did w i t h
t h e data, how t h e y w e r e "analysed". B y now t h e b i o l o g i s t i s quiet, d e -
f e r r i n g t o t h e s t a t i s t i c i a n o n a l l matters statistical, a n d terms l i k e main e f -
fects, regression lines, homoscedacity, interactions, a n d covariates fly
a r o u n d t h e room. Sooner o r l a t e r I find myself t h i n k i n g "Here a r e t h e
answers, b u t w h a t was t h e question?" A l l too f r e q u e n t l y in such p r e s e n t a -
t i o n s n e i t h e r t h e statistician n o r t h e biologist has posed t h e main question
o f biological i n t e r e s t in non-statistical terms, t h a t is, in terms w h i c h a r e
i n d e p e n d e n t o f analyses o r models w h i c h may o r may n o t b e a p p r o p r i a t e f o r
t h e data, a n d I can c e r t a i n l y remember occasions when t h e analysis p r e -
sented was seen t o b e i n a p p r o p r i a t e once t h e f o r g o t t e n question was f o r m u -
lated. O f course many s c i e n t i f i c questions can b e t r a n s l a t e d i n t o state-
ments a b o u t parameters i n a statistical model, so t h a t 1 am n o t condemning
a l l instances o f t h e above practice.

A similar s o r t o f experience i s s u r e l y familiar t o a l l who have helped people

w i t h t h e i r statistical problems. T h i s time a scientist, say a psychologist,
comes t o me w i t h a set o f data and one o r more questions. She/he knows
some statistics, o r a t least some o f t h e jargon. A f t e r b e i n g b r i e f e d o n t h e
b a c k g r o u n d p s y c h o l o g y a n d t h e mode o f collection o f t h e data 1 u s u a l l y say
something l i k e "What questions d o y o u w a n t t o answer w i t h these data?",
i m p l i c i t l y meaning "What psycholoqical questions . . .
?" Not i n f r e q u e n t l y
t h e answer comes b a c k " I s t h e d i f f e r e n c e between such a n d such s i g n i f i -
cant?" meaning, o f course, s t a t i s t i c a l l y s i g n i f i c a n t . [I-n m y p e r v e r s i t y I
o f t e n think t o myself: "Well, you should know; it's y o u r data a n d y o u a r e
t h e psychologist! "1 A n o t h e r similar q u e r y m i g h t concern interactions, o r
r e g r e s s i o n coefficients o f covariates etc. What t h i s has i n common w i t h t h e
p r e v i o u s example i s t h e unwillingness o r i n a b i l i t y o f t h e psychologist t o
state h e r / h i s questions o f i n t e r e s t in nonstatistical terms. We s h o u l d all b e
familiar w i t h t h e idea t h a t s c i e n t i f i c (e.9. psychological) significance and
ICOTS 2, 1986: Terry Speed

statistical significance a r e n o t necessarily t h e same thing, b u t how many o f

u s keep in m i n d t h e f a c t t h a t t h e l a t t e r involves an analysis o r a statistical
model, a n d t h a t t h e r e may b e as many answers t o t h i s question as t h e r e
a r e analyses o r models? S u r e l y much o f t h e blame f o r s u c h t h i n k i n g r e s t s
w i t h us, t h e teachers o f statistics, who n e v e r fail t o popularize t h e rigid
formalism o f Neyman- Pearson t e s t i n g t h e o r y .

M y third t y p e o f experience concerns r e c e n t graduates i n statistics, s t u -

dents I a n d m y colleagues have t a u g h t and whom we believe should b e able
t o operate i n d e p e n d e n t l y as statisticians. Many o f these graduates g o i n t o
jobs i n big p u b l i c enterprises: railways, a g r i c u l t u r e bureaux, m i n i n g com-
panies, government departments a n d so on, a n d a few - f a r too many f o r
comfort - g e t in t o u c h w i t h u s when t h e y meet a d i f f i c u l t y i n t h e i r new
job. It i s n o t t h e fact t h a t t h e y g e t i n t o u c h w h i c h i s discomfiting, b u t t h e
questions t h e y ask! For we t h e n learn how l i t t l e t h e y have grasped. T h e y
have questions in abundance, o f t e n important p o l i c y questions, access t o
lots o f data, o r a t least t h e p o s s i b i l i t y o f collecting a n y data t h a t t h e y
deem necessary, b u t t h e y a r e q u i t e u n s u r e how t o proceed, how t o answer
t h e questions. O u t t h e r e i n t h e w o r l d t h e r e a r e "populations" o f real
trains, f i e l d plots, c u b i c metres o f o r e o r people, a n d even t h e simplest
question r e l a t i n g t o a mean o r a p r o p o r t i o n o r a sample size can b e f o r -
b i d d i n g . Perhaps t h e y should standardize something t o compare it w i t h
something else, p e r h a p s i n c l u d e t h e v a r i a b i l i t y o f one f a c t o r when ana-
l y s i n g another, o r something else again, all t h i n g s w h i c h we feel t h a t a
g r a d u a t e o f o u r course should b e able t o cope w i t h unaided. B u t how well
did we t r a i n them f o r t h i s experience?

Finally, a n d b r i e f l y , l e t me castigate m y professional colleagues -

and m y -
self, since I am n o exception - f o r allowing ourselves t o f o r g e t t h e f u n d a -
mental importance o f t h e i n t e r p l a y o f questions, answers a n d statistics, f o r
in so many o f o u r professional interactions we act as if it is i r r e l e v a n t .
How many times have we p r e s e n t e d new statistical techniques t o one a n -
other, i l l u s t r a t e d o n sets o f "real" data, d r a w i n g conclusions a b o u t those
data concerning questions no one e v e r asked, o r i s e v e r l i k e l y t o ask? A n d
how o f t e n do we d e r i v e statistical models o r demonstrate p r o p e r t i e s o f
models w h i c h a r e u n r e l a t e d t o a n y s e t o f data collected so far, a n d c e r t a i n -
ly n o t t o a n y questions f r o m a substantive f i e l d o f human endeavour. We
are, so we t e l l ourselves, simply a d d i n g t o t h e stock o f statistical methods
a n d models, f o r possible l a t e r use. I s it a n y wonder t h a t we o r o u r co-
w o r k e r s t h e n f i n d ourselves u s i n g these models a n d methods i n practice,
regardless o f w h e t h e r o r n o t t h e y help u s t o answer t h e main questions o f
i n t e r e s t . For a discussion o f some closely r e l a t e d issues o f g r e a t relevance
t o teachers o f statistics, see t h e t w o excellent articles Preece (1982,
1986).

3. Why this audience?

I don't t h i n k I w i l l b e v e r y wide o f f t h e m a r k if 1 assume t h a t most o f y o u -

a t least t h e a c t i v e teachers o f statistics amongst y o u - have come f r o m a
b a c k g r o u n d o f mathematics r a t h e r t h a n statistics, a n d t h a t f e w o f y o u have
a c t u a l l y been statisticians b e f o r e y o u s t a r t e d teaching t h e subject. I would
f u r t h e r guess t h a t many o f y o u s t i l l teach mathematics, a n d perhaps a t t h e
ICOTS 2, 1986: Terry Speed

school level, statistics within a mathematics. curriculum. It is on t h i s as-

sumption that I have chosen t o focus on non-mathematical aspects of our
subject, ones with which I feel you will generally be less familiar. As I
said i n the introduction, I hope t h a t my talk will encourage you t o give
more attention t o the non-mathematical aspects of statistics i n y o u r teach-
ing, in particular t o spend more time considering real questions of interest
with real sets of data.

It is a curious t h i n g t h a t interest i n t h e teaching of statistics i n schools,

colleges and universities has sprung u p worldwide as an extension of math-
ematics teaching, because I certainly feel t h a t t h e practice of statistics is
no closer t o mathematics than cooking is t o chemistry. Both mathematics
and chemistry are reasonably precise subjects i n t h e i r own ways, and i n
general what goes on i n them both is repeatable; perhaps they are t r u e
sciences. On the other hand, statistics and cooking are as much arts as
t h e y are science, although both have strong links t o t h e i r corresponding
science: mathematics i n the case of statistics, and chemistry i n t h e case of
cooking. Who would recommend t h a t a chemistry teacher with no cooking
experience be appointed as cooking teacher as well? If I can convey t o you
some of the enjoyment and intellectual challenge t h a t lies i n my particular
variety of cooking, and encourage you t o try it yourself, I will have suc-
ceeded i n my aims.

4. Two further examples

I n this necessarily too b r i e f section I offer two more concrete illustrations

of interplay of t h e questions, answer and statistics. The f i r s t bne is a
v e r y simple paraphrase of Neyman's classic illustration of hypothesis test-
i n g involving X - r a y screening f o r tuberculosis, and I refer you t o Neyman
(1950, Section 5.2.1) f o r a fuller background and f u r t h e r details.

You have a single X - r a y examination and, after the photograph has been
read b y the radiologist, you are given a clean b i l l of health, t h a t is, you
are told t h a t there is no indication t h a t you are affected b y tuberculosis.
With Neyman we will assume t h a t previous experience has led t o

pr(clean b i l l ]no TB) = 0.99

pr(cleanbillpB) =0.40

You now ask the radiologist "What are the chances t h a t I have T B ? " She
says "I can't answer t h a t question b u t I can say this: Of the people with
T B who are examined i n t h i s way, 60% are correctly identified as having
TB, and of . . . " You i n t e r r u p t her. "Doctor, I know the procedure is
imperfect, b u t you have j u s t examined my X-ray . . . What are the
chances t h a t I have TB?"

If your radiologist is sufficiently flexible and well informed, she will

answer "Well, t h a t depends -on t h e prevalence of T B i n your population,
t h a t is, on the proportion of people affected b y T B i n the (a?) population
from which you may be regarded as a typical individual". Indeed a simple
application of Bayes' theorem yields:
ICOTS 2, 1986: Terry Speed

p r ( T B Iclean bill) = p r l c l e a n bill I T B ) ~ ~ ( T B )

pr(clean bill)

A t last you see how t o get an answer t o your question. It may not be easy
t o obtain a value f o r p r ( T B ) : your smoking habits, the location of y o u r
home, your occupation, your ancestry . .
. may all play a p a r t i n defining
"your population", b u t this is what is needed t o answer the question and it
is f a r better t o recognise this than t o fob you o f f with the answer t o an-
other question not of interest t o you.

If this example smacks of Bayesian statistics it is not entirely accidental,

f o r there are many occasions where the Bayesian view (which is certainly
not necessary i n this example) helps answer t h e question of interest,
whereas classical statistics refuses, frequently answering another, u n -
asked, question instead. For a more complex, explicitly Bayesian example,
see the v e r y fine paper Smith and West (1983) concerning the monitoring
o f renal transplants.

My second example concerns the determination of the age of dingos, Aus-

tralia's wild native dogs. A statistician was given a large body of data r e -
lating the age of a number of dingos t o a set of physical measurements i n - -
cluding head length. The data concerned both males and females, a number
of breeds and animals from a number of locations, b u t for t h i s discussion
we will r e s t r i c t ourselves t o a single combination of sex, breed and loca-
tion. The question, or at least the task, t o be addressed was the follow-
ing: produce an age calibration curve f o r dingos based upon t h e most s u i t -
able physical measurement, t h a t is, produce a curve so t h a t the age of a
dingo may be predicted by reading o f f the curve at the value of the p h y -
sical measurement. This curve was f o r use i n the field and it was taken f o r
granted t h a t an estimate of the precision of any age so predicted would al-
so be obtained.

It was found that a curve of the general form h = a + b [ l - exp(-ct)],

where h and t are head length and age, respectively, and a, b and c are
parameters of the curve, f i t t e d the data from each dingo extremely well
over the range of ages used. This was an exercise i n non-linear regression
with which the statistician took great care, special concern being given t o
t h e different possible parametrizations of the curve, the convergence of
t h e numerical algorithm used, the residuals about t h e fitted line and t o the
validity of the resulting confidence intervals f o r a, b and c. The param-
eters estimated for different dingos naturally differed, although, not s u r -
prisingly, the values of a (head length at b i r t h ) showed less variation
than those of b (ultimate head length -a) and c (a growth rate parameter).

A l l this seems fine, and you might wonder why i am mentioning t h i s

example a t all in the present context. My answer is as follows. The statis-
tician in question knew, or knew where t o find, lots of information about
t h e f i t t i n g of individual growth curves, and so he focussed on this aspect
of the problem. To answer the original question, however, his attention
ICOTS 2, 1986: Terry Speed

should have been pointed i n quite a different direction, towards: t h e cal-

culation of a population o r group growth c u r v e f o r t h e calibration proce-
dure; features o f the sample of dingos measured t h a t may affect t h e use o f
t h e i r measurements as a basis f o r the prediction of t h e age of a new dingo;
properties of the parameters which are relevant t o t h i s question; and,
finally, towards obtaining a realistic assessment o f the prediction e r r o r
inherent i n t h e use of t h e curv e i n the field.

I n summary, he was willing and able t o spend a lot of time on t h e i n d i v i d -

ual animals' curves; he was
demanded b y t h e question,
lation variability, problems
issues including the use of
fined populations.
less willing and less able t o focus on t h e issues
those concerning population parameters, popu-
of selection, u n representativeness, and other
normal theory, with real b u t not v e r y well de-

5. What is t h e problem?

Let me oversimplify and p u t my message like this. I n the beginning we

taught mathematics and called it statistics; much of t h i s was probability, a
quite d i f f e r e n t subject. Then, with the help of computers, we started t o
teach data analysis and statistical modelling; t h i s was fine apart from one
feature: it was largely context-free. The real interest (for others and
many statisticians), the important difficulties and t h e whole point o f statis-
tics lies i n t h e interplay between the context and t h e statistics, t h a t is, i n
the interplay between t h e items of my title.

Let me offer a few similar views. A.T. James (1977, p. 157) said i n the
discussion of a paper on statistical inference:
The determination of what information in the data is relevant can only
be made by a precise formulation of the question which the inference
is designed to answer. ...
If one wants statistical methods to prove
reliable when important practical issues are at stage, the question
which the inference is to answer should be formulated in relation to
these issues.

Cox (1984, p. 309) makes the following characteristically b r i e f contribution

t o o u r discussion :
It is t r i t e that specification of the purpose of a statistical analysis is
important.

Dawid (1986) is even more t o the point:

Fitting models is one thing; interpreting and using them is another,
... If the model is correct and we know the parameters, how ought
we to compare [schools]? ...There is in fact no unique answer; it
all depends on our purpose. ...there remains a strong need for a
careful prestatistical analysis of just what is required: following
which i t may well be found that i t is conceptually impossible to esti-
mate it!

Tukey and Mosteller (1977, p. 268) offer seven purposes of regression,

or, as I would paraphrase it, seven types of questions which regression
analysis may help answer. Summarized, these seven purposes are:
ICOTS 2, 1986: Terry Speed

1. t o get a summary;
2 . t o set aside the effect of a variable;
3. as a contribution t o an attempt a t causal analysis;
4. t o measure the size of an effect;
5. t o try t o discover a mathematical o r empirical law;
6. f o r prediction;
7. t o get a variable o u t of the way.

Similarly, T u k e y (1980, pp. 10-11) gives the following six aims of time
series analysis;
1. Discovery of phenomena.
2 . "Modelling".
3. Preparation f o r f u r t h e r inquiry.
4. Reaching conclusions.
5. Assessment of predictability..
6. Description of variability.

Similar numbers of aims, purposes, o r types of questions could be given

f o r t h e analysis of variance, the analysis of contingency tables, multi-
variate analysis, sampling and most other major areas of statistics. Yet
how often, do our students meet these techniques i n context with even one
of these aims, much less the f u l l range? And how else are t h e y going t o
learn t o cope with the special difficulties which arise when questions are
asked of them i n context whose answers require statistics? This is the
problem.

6. Some General Comments

I n this section I will mention a few difficulties which I believe can be r e -

solved i n a given case when the relation between the questions asked, t h e
form of t h e answers desired and the statistical analysis t o be conducted
are carefully considered. A f u l l discussion of any one of the difficulties is
o u t of the question, and even if t h a t had been given, there would probab-
ly remain an element of controversy, something which would be o u t of
place i n a talk like this. The section closes with some f u r t h e r general com-
ments about questions.

Some elementary difficulties which I t h i n k arise include

? What is t h e population?

When are population characteristics (e. g . proportions) relevant?

ICOTS 2, 1986: Terry Speed

What is the "correct" variance t o attach t o a mean o r proportion?

0 When should we standardize (for comparison)?

I have found t h a t t h e relations between statistical models and analyses on

the one hand, and populations and samples on t h e other, with parameters
playing a role i n both, are something which puzzle many students of our
subject. The former play a b i g role i n standard statistics courses whereas
t h e latter are prominent i n applications. J u s t how t h e y connect is not a
t r i v i a l matter.

A few somewhat more advanced difficulties include

0 Which regression: y on x, x on y o r some other?

0 When should we use correlation and when regression analysis?

When can/should we adjust y f o r x?

9 Which e r r o r terms do we compare (in anova)?

Should we regard a given effect as fixed o r random?

0 Which classifications (of a multiway table) .correspond t o factors and

which t o responses?

More subtle difficulties are associated with general questions such as

9 Should we do a joint, marginal or conditional analysis?

I believe t h a t i n all of t h e above cases the difficulties arise because insuf-

ficient attention has been given t o the nonstatistical context i n which the
discussion is t a k i n g place, and t h a t when t h e question of interest is clari-
fied and the form of answer sought understood, the d i f f i c u l t y either dis-
appears completely o r is readily resolved. Of course doing so takes some
experience. Note t h a t many of the difficulties listed involve, implicitly o r
explicitly, the notion of conditioning, or i t s less probabilistic forms, stan-
dardizing o r adjusting. Just what we regard as being "held fixed" and
what we "average over" i n any given context is crucial, and here our
questions and answers determine everything. The simplest form of this
issue is usually: "Are we interested i n j u s t these units (the ones we have
- seen), o r i n some population of units from which these may be regarded as
a (random?) sample, o r both?" Models are no help here.

A simple b u t easy t o forget aspect of the use of a statistical method is t h a t

not all questions which could be asked and answered b y that method, are
necessarily appropriate i n a particular context. Lord's paradox, see Cox &
McCullagh (1982) and references therein, provides a good example here.
ICOTS 2, 1986: Terry Speed

7 . What can/should be done?

It hardly needs saying t h a t t h e best way t o promote interest i n t h e i n t e r -

play between questions, answers and statistics is t o p u t trainee statisti-
cians into situations where they are required t o provide answers t o clearly
stated questions on t h e basis of real data sets. Note t h a t t h i s can be a
v e r y different t h i n g from "illustrating" a statistical technique on a set of
data. I n particular, much more background t o t h e data is usually r e -
quired, and this is r a r e l y available i n data sets presented i n statistics
texts. Indeed technical journals are now so t i g h t with t h e i r space t h a t it is
rare t o f i n d f u l l data sets published together with analyses and conclusions
i n scientific articles. This means t h a t the best sources o f suitable material
of the k i n d being discussed, t h a t is, of questions and data, are often one's
colleagues o r clients: teachers and researchers i n other disciplines who
make use of experimental o r observational data i n t h e i r work. Seeking out
such material can be a way of f o r g i n g links with the users of statistics and
of course sandwich courses are designed with this general aim i n mind.

One practice which I believe is valuable is the conduct of regular practical

statistics sessions where students are asked t o help answer specific ques-
tions on the basis of sets of data supplied together with background mater-
ial. This is much more like the situation t h e y will meet after their t r a i n i n g
is over. Two objections which are often expressed t o me when I recommend
t h i s approach are (i) Surely it is unrealistic, except with the most ad-
vanced students, f o r unless t h e y have learned a wide range of techniques,
they will not be able t o begin attacking "real" problems with any likelihood
of success?; and (ii) Surely it is unrealistic, because real problems are so
complex and real data sets so large, or even ill- defined, t h a t nothing like
what happens in practice can be presented i n the classroom?

Both these objections have some validity, b u t let me make a few observa-
tions concerning them. Firstly, it is not necessarily a bad t h i n g f o r a s t u -
dent (or anyone!) t o attempt t o answer a particular question (solve a p a r -
ticular problem) without knowing of the tools o r techniques t h a t may have
been developed t o answer just t h a t t y p e of question (or problem). This
goes on all the time i n the real world: parts of the wheel are rediscovered
time and time again, and locomotion is even found t o be possible without
the wheel! And of course there is v e r y seldom a single "correct" way t o
answer a question; an approach using less knowledge of techniques may
well be better than one which uses greater knowledge. I n the hands of a
good teacher, such experiences can provide valuable object lessons, and,
at the v e r y least, valuable motivation f o r techniques not y e t learned. Sure-
ly nothing could be more satisfying than hearing a student say: "What 1
need (to answer t h i s question) is a way of doing such and such, under the
following circumstances (e.g. errors i n t h i s variable, t h a t factor misclas-
sified, these observations missing o r censored, t h a t parameter chosen i n a
particular way, etc.)? Group discussions, where ideas are shared and
knowledge pooled, are also most appropriate f o r t h i s sort of work, and
most enjoyable. The teacher can then play a subsidiary role, at times
. focussing the discussion back on the questions, perhaps at other times
supplying a sought-for technique.
ICOTS 2, 1986: Terry Speed

'It would seem t o me t h a t t h i s is j u s t t h e s o r t o f statistics which should be

t a u g h t i n secondary schools, not t h e watered-down and frequently sterile
mathematical material which is often found a t t h a t level.

T h e second objection, t h a t real problems are often v e r y complex and r a r e l y

amenable t o t h e s o r t o f trimming t h a t would b e necessary before t h e y could
be used i n a classroom, is harder t o dismiss. It i s certainly t r u e t h a t many
(most?) problems are like this, b u t s u r e l y t h i s highlights even more t h e
difference between "illustrative" data sets, taken o u t o f context, w i t h no
realistic questions o r idea what would b e satisfactory answers, and what
we expect students t o be able t o cope w i t h upon graduation. There is cer-
t a i n l y a b i g gap here - between "pseudo-applied" statistics involving
context-free sets of numbers, t o illustrate arithmetic, and fully-fledged
"warts and a l l " consulting problems - and I can o n l y state t h a t i n my e x -
perience it is possible t o f i n d problem data sets which can be presented i n
t h e way I am suggesting. It certainly takes a l i t t l e e f f o r t t o f i n d such
material, particularly if you are not in t h e habit o f meeting people w i t h
data and statistical problems. B u t as teachers o f t h e subject, t h a t is not
such an unreasonable t h i n g f o r me t o expect of you is i t ?

A teaching strategy which could provide a means of p u t t i n g these ideas

into practice might be t h e following: pair yourself (the statistics teacher)
with a teacher i n an empirical field o f enquiry, e.g. biology, agriculture
o r medicine, and also pair your statistics students w i t h students i n t h e
corresponding class, r e q u i r i n g them t o w o r k together on a practical p r o -
ject which will enrich t h e i r understanding o f both disciplines, and how sta-
tistics helps t o answer questions. Many variants on t h i s suggesti,on could
be devised; t h e important t h i n g is try something along these lines. Statis-
tics students must meet more than mathematicszand sets of numbers i n
t h e i r training, and it is t h e teachers o f statistics who must arrange f o r
t h i s t o happen.

Acknowledgement

I am v e r y grateful f o r t h e discussions and comments on t h i s topic offered

t o me by my CSlRO colleagues Peter Diggle, Geoff Eagleson and Emlyn
Williams.

8. References

Cox, D. R. (1984). Present position and potential developments: Some p e r -

sonal views. Design o f experiments and regression. J.R. Statist. Sac.
Ser. A., 147,306-315.

Cox D. R. & McCullagh P. (1982). Some aspects o f t h e analysis o f covari-

ance. Biometries 38, 541-553.

. Dawid, A.P. (1986). Contribution t o t h e Discussion of: "Statistical mod-

elling issues i n school effectiveness studies" by M. A i t k i n and N.
Longford. J. Roy. Statist. Soc. Ser. A, 149, 1-43.
ICOTS 2, 1986: Terry Speed

James, A.T. (1977). Contribution t o the Discussion of: "On resolving the
controversy i n statistical inference" b y G. N. W i l kinson. J. Roy. Statist.
Soc. Ser. B, 39, 157.

Mosteller, Frederick & Tukey, John W. (1977). Data analysis and reqres-
-
sion. Sydney: Addison-Wesley Publishing Company.

Neyman, J. (1950). F i r s t Course i n Probability and Statistics. New York:

Henry Holt & Company.

Preece, D.A. (1982). t is f o r trouble (and textbooks): a critique of some

examples of the paired samples t-test. The Statistician, 91, 169-195.

Preece, D. A. (1986). Illustrative examples: illustrative of what? The Sta-

tistician, 35, 33-44.

Smith, A.F.M. & West, M. (1983). Monitoring renal transplants: An appli-

cation of the multiprocess Kalman f i l t e r . Biometrics, 39, 867-878.

Tukey, J.W. (1977). Contribution t o the Discussions of "A reformulation of

linear models" b y J.A. Nelder. J. Roy. Statist. Soc. Ser. A, 140,72.

Tukey, John W. (1980). Can we predict where "Time Series" should go

next? Directions i n Time Series Eds D.K. Brillinger & G.C. Tiao. IMS.
pp.1-31.