Yesearch Design

nnru;ddc /$zrtc hnr ftwrvr1 nrrv nfl!?r, hr FI ~ r n ~ nwr r!r o n h nlt,rltodr Irr ~ h o r ~ k r
unwrt(i . T/w ~ ~ ~ r ! ~ ~ ~ /,r tor / ,, cr,*ot , r ~ rh~nrt
~ t t ~ r~~ t r~z i i i - r q < z r { t 3 f tI!! t t $ ~ c \~, n i / t ~ d t r ~ g 0 A
~ ! , O ~ ~ ~ I ~,,%U. F
t r < / t n : th,,07ff,q. t . f , i l ~ o t i otlnr<l F III\I,.
~ I ~ I ~ I~ I I- ~o ~ l ~ l ( J 4 . r tnnrti7f f~~rf ~ rql~t t. d Vr
lrrrne l / l p I / ! # ~ T ~ , I ~ i ! ~ , d innd
r f t - . r ~l~~I~r r>~~ v - r r ~H/ ~. r f ~ v ~tJ!r~r t ~ rn~trrrrrl>r~ ~t~tlJ 0
-find tlrrr rrn P I rrr,mr.ly ~rnttmhle.IPPII~ ~ ' r r r f i -borlk.
rt
P R ~ ) F F C , ~A~t Rks RR\HNt. I . N l v F R ~ l T Y <)I L r l l l C 1 t f f O R O i ' I ; H
Y in Social Research
n ~ r ~ n rDd ~?r ; g r~n~ S~~rraI IS onr-stop rritlral guide that d~monstratrs
Rr~~rflrrJr CI
&
the impc~rtanwr,F attend in^ to design issr~csIn sorial r ~ ~ e a r r h .
~ ~ rrollrctinn strat rm and statistical
thlr Fll'ld p r i c > r ~ tclnfa
H David de Vaus
W l ~ ~ r ne tal l~~ t~ooks
r
analysis, de Vaus arjiur4 that, a- s ~ g n l f i r a n tas t h r w Isslirs arv, rnost p p r t l n ~ n nf
t
all ro r ~ ~ r ' n r rtl l~ r h l i l q 15
~ tthe T ~ drsign of thr rewarrh.
~ ~ S I T U C ~ ~ Iand
1'111~bnnk:
* Prvirles t h t~~ a d sit of the irn1)orranrr nf' rcsparrh
~ r11 an ur~rl~r~rarirI~~i~
dr+i am1 it< place in tire researcll pmww
Drscrtbes. thp maln r ) ~ of
s r ~ s ~ a r rrlrstgt15
h 1 1 1 socinl rrwarrh

Erplarns thp Ioglr a n d IIurpSeF of d e ~ to

~ canatde
p students to rvatuatr

equip^ studrnu wil h t h clr~ipzi

~ sk ~ l tol rjprrdtr
~ in rral wnrlrl

Divided into 5 partc. part I rvplorrs iqstrrq ;lro~lndr ~ s ~ a r tvpps, c l ~ ronwpts;.

y tlas~crssirps ln t ht' ~ I P S I K I I ~ ~ O C T S ST. h v terna~nina4 perL.i
qurst inns, t , a u s ~ l ~ tarld
focus on diff~rrnttypes of rrsearch design. Exprrmental, lon+tr~dinal,rroas-
. m t o n a l and rw.stutly rnetllodz arr clearly artd qstrrnat~callur~amtnprl, at~d
thrir strengths ant! wraknm.~snre descsibrd.
'This bonk will b~ invaluablr to hoth sturlerl~sarid r c s ~ a r c l l ~ r sil~forming
, and
p i l l :Ing t hrm tawarrlq s ~ ~ r ~ ~ swt-1~1
s f u l rc~r*;lrc11.

n rl!r~rmrrronhv.4, rrl1,nr Rtrr

Cnrrr d ~ < r cand
I S B N 0 - 7 6 1 9-5346-9

frE Pulblications
r,onnon J troricatrd Oak$ ~ P I I L ~
 T i n t puhlr.;hril ?t?(!l

Apart h r n a n v fair dralrng for lhr purpowq of research or

CONTENTS
yrr\ ate ~tn<l\., o r crihcrzrn or rc*vit.trv. a5 pernii tted under the
Cvpvrigh'ht, I\-~psand rnientc Act, 19W. 1ht5 publicatlvn
mav be reproduced, slnrc~1or tr~~nsniltted In any fom, or try
anv means. cmlv rvlth t11(* prior pcsrrnlrs!r)nin wrltln): r ~ the t
yuhllqherc. or In the l a s t k 1 1 f reproqraphlc rrpnxluctbnn, in
arctlrdilncr w ~ t hthe fchrmsuf I l c e r ~ c eL~S ~ L I C L !bv the
Copk right I ~ct+~lsing A ~ r n c y Inquirres
. cmrernlng List uf F ~ , ~ I I T P P xi
t ~ c n ~ thew tprrnz 4inuld ht, cent to the>
r c ~ ~ ~ r n d u c otltclde
pul7lr~hers List trf Totdrs xiv

PWfnct' xvi
SAGI' I'ublicati[>ns Ltd
h nnnhill S t r r ~ t
h(>ndon FCZ.1 .FI1U PART I: WHAT IS RESEARCI-E DESIGN?
1 T h e Context of Design
Description and ekplanation
Uescripfive research
SACX I'ublications l n d i , ~IXvt I td Explanatory resen rch
32, C1-Rlnck Market Thcory tsting and theoq constructinn
Greatc~rIGilfazh - 1 Theory bullding
New Tlelhr 1 1 0 048
Theory testing
Eritirih Library Cataloguing in Publication data What is rrsearch d r z i p ?
Desigm versus method
A catillopir r c ~ o r dfor I ! i ~ sbonk 1% availahlr
from the Brltlql~Library Adopting a sceptical approach to cxplana tions
Summary
TSRN O 7619 5?46 9
ISHN 11 7619 53.17 7 (pbkj 2 Tools for Research Design
Before design
Lihran, of Congress catalog record available Focusing and clarifying the research question
Identifying plausible rival hypothcqes
Tvprrr3t by M;avht=wTvyrscthnq, Rhavades. I'owr,.i
Operationalization
In C;t*j~t Rrrtaln I,\ rltddl~sZ td, GtrLl,ltilrd,Surrr-l
Pr~ntc~tl
Concepts for research design
Internal validity
External validity
Measurement error
Summary
=3 Cawsation and the Logic of Reqearch Design
Inferring causal relrlt;ionshipq
criteria For infprring cause
T,.v~v
,,fF ,,.-I . 7+*,.l-n7
 C o r n y a r ~ n gtirnc points Summary
hlaLrn7 m c ; l n i n ~ f rcomparisons
~l Notes
Int~rvcntinnl:a n d indepcntlent variables
Dirnrn.;iclns crf a rrc;c.lrch ~ i c - 1 ~ ~ Analysing Experimental Data
A wngc of rrsearch designs 5 c l c c t i n ~the mcthod of ~ r ~ a l v s i s
ExpcfirnrntaI cle.;ip \Vhat kind of annlvsis is rcqlti rcd: dcscriptinn o r
I,~>nqih!dinaE dec;r~n infcrrnc~?
C r c ~ s - ~l(mal
c t dv\ien W h a t w r t of .;ample do we hL3vc7
C w t stud~rs \'\'hot Icvel of data d o w e ha\.c'
Summary Typc of cornp,lrisons rt7quired'
Vrltr,c Mrhnt type of displav i.: reyt~irtd?
f-lorv m a n y )!,mItFS art? to bc cornpasrd7
P A R T 11: EXPERIMENTAL DESIGNS Arc thc c ~ m p ~ i r i s ogroups
n indrpmdcnt samples?
1s thc dcpendcnt vnr~~lblc nnrmalIy distributtgd?
4 Tvppg of Experimental Design Is thc vnrianct nn thc rtependcnt variable similar
TIw classic c ~ \ p ~ r i m c n t dt*sign
al b ~ t w e e neach comparison grc~up?
3:upcrirnr~tlal ctlntcuts How lnanv indtpenctmt vari,lbles?
Simplct c\prr~rncnt~iE de~igns Summ,lry Froce\s for selecting the right rneastlre
I ' o ~ t - t ~ t only
rt w12hcontrol grnLlp (%her ~ S S L S ~ ~ S
i l m e< > i p ~ r i r nl,lli ~design
I<t~lrusyci-l ~i Changr scorc3.:
\.lnrc, crmlpll-. expt>rrmcnt,~l Jei;iqns (flrwr ~ n i !ci*ilinq cfftctl;)
Trunc,.ilim eff~*cts
hluLt1plc p,st-tt.5t.. Trend analysis
MuItipEc g r o u p Summary
50lorn4m tour-croup d e ~ i c n Nntc
Factnrial ~lcsigns
Summary PART 111: LONGITUDINAL D E S I G N S
Volr\
5 Issues in Experimental Design 7 Types of Longitudinal Design
I ' u r p ~ ~oft lon~itudinnl
~ rlesi~n
Methndological i.;<~t*.:
Dwcrihing patterns of change and stabilitl:
The prnI7lem ot rxplanatorv narrownesq
EstnLlli\hing trmpow8 order
l'mblems with i~itrmmlvalidity
Estatll ishing develr~pmcntal( ~ g ecffccts)
I'n)bIclnq 12-lth r%ulcrnalvalidi?
Est;tbli s h i n ~historical (period ) effcct.:
I'mcticaI i ~ s u c ?
I-low rnz~clld n y i ~ utell participnnts? I , i k course 'career' analysis
I-lori. mnnv past icipantq' Tvp.p.~s crf longitl~dinalr l ~ q i p
l - l c l w 5 h o l t l ~ ipartic~pantqI-r r~cruitvd?
Procptbsti~~ pane! dtwqn.;
Retro.;ppcti\ c designs
i ;aps b r t ~ r e c ntcyst.: and intcrvcntir~ns
Il'llicll nirthc~d(tf Jat,? ccnllcthun7 Qunki-longitt~dinaI designs
S~trnn1.1~1
I'rthlems x\-ith r,~ndornimcl a s q ~ r i r n e n t
knte
Unrveneic~~.; of I ntcrvent ions
T'ht wlf-fulEiIlinr: prnphec~. R Iswes in Longidutinal D e s i p
Ethical i s s i ~ w Mctho~iolog~cal Iqsueq
Vol untarv pdrticip,~tion Is.;ur\ n t ~ntrrnalvalidih/
I n f n m r d ronstlnt 1$4111'< of eutt>rnal vnlrdie
Nn harm t n pnrtlcipant5 Practical Issue.;
,In1f r - ~ ) l ~ f i ( i r ~ ~ ~ f i l l i t ~ *
,+inon\,~~iitx~ <I ,,,, I ,,.,4;.,I;,,., , , I ; + * ,
b>..,n>lm>#7!,
 I'n~elattrition Practical is.;uc.;
R~spondc.11 t Zlurden Method clf collecting data
Rcspunrlpn t rr.cal l Samplc si?es
Cnst Sufficient variation in the sample on kc!
Method uf data collection variables
hTumhernf iravcs Infnrmation for statistical controls
Gap betwtcn waves Length
S.irnple error Types of data
Sample s i m Ethical imurs
In~trumenbdesign Summa?
5taifin~ Y otes
I;thzcal issue<
32 Cross-Sectinnal Analysis
Voluntary participation
Informed conscnt
Descriptive nnaly sis
Harm to participants
HOW~ J R V ?
Lei~elnf iictnil
Cnnfidentialih. and anonvmity
Summary
Form of data
Who?
9 Data Analysis in Longitudinal Design Factor structures and scale structures
Misqinq data Haw ~ t . n r r a l STON
l CIOSC~
hwtrccs of missing data Explanatory a n a I y i s
~ ~ ' a1 ~TX&Il'fil'
1 ~ The locic ot stahstical cimtrels
Iiicntifying miwing data bias Multiplc statistical cnntmk
Ikaling with miwing data The elabnration techniquc
M~asuringc h ~ n ~ c Basic approach
A ~ ~ r e g a Icvcl
te versus inctividual level change Interpsrtation
Qual~tativrv c n u s quantitative change Problems with the elaboration model
Measuring change in panel d c s i p s Multivariablr analysis
Dclscrihjng c h a n g ~ Cohort analysis
Tablcs Constructing and reading cohort tablr.:
Graphical Problms with cohort analysis
Summary Summary
Nrlti.
PART V: CASE S m Y DESIGNS
PART IV: CROSS-SECTIONAL DESIGNS
23 Case Study Design
10 Cross-Sectional D e q i p \*at is a caw'
\ ( r time dimcn\ir>n Units of ilna!v~is
Rrliancc on existing diffewnces Moliqtic ar~dembedded units of analvcis
Thl. nature nf ' ~ r n ~ r pin
s ' thc cross-sectional i i ~ s i g n Case studies and theory
Obtaining a timc cl imcnsion: rc*pcnted csnsq-sclct~onal Explanatory case studirh
s h d ips Descriptirtc case 5tud1c.;
Other elrments of case qtudv design<
II Issues En Cmss-Sectional Design Singlc nr multiple case\'
h.lrtlit~dological w u e s Partlllcl nr sequential?
Intrrnal validity Rctrosprctive or prospcctivc?
External valtcli t i ) Types of caw study d e s i p s
 CONTENTS

W i a t a case study is not

Not the one-shot case shtd y
Not a data collection n~ethod
Summary
Note
14 Issues in Case Study Design LIST OF FIGURES
Methodological issues
Internal validity
External validity
Practical issues
Sampling
Number of investigators: getting consistency Three types of causal relationships
When to go in to the field Theory building and theory testing approaches to
Prescnhng case studies research
Ethical issues The relationship between divorce and parental conflict
Summary The logic of the research process
Notes Kelaticmship between research dcsign and particular data
collection methods
15 Case Study Analysis Causal and non-causal explanations of the relationship
Statrstical analysis betwt.cn school type and acadcnlic achievement
Meaning and context An intervening variable
Analysis in descriptive case studies An extraneous variable
I Theoretical dimension of descriptive analysis Starching for causes
Explanatory case studies Searching for effects
I Theory testing analysis
A specific causal proposition
I Literal and thcnreticaI replication
Analysis for theory building: a n a l y ~ cinduction
A more complex causaI model for increases in divnrce
Dimensionalizing the concept of chiId's wellbeing
Summary A spurious relationship
Note An indirect causal relationship
More complex indirect causal relationships
Types of relationships between three variables
Propositions and observations without a comparative
frame of reference
Using propositions with a n explicit comparative frame of
reference
Multiplc comparison groups m a t e d by combining two
independent variables
CIassic experimental design
Simple longitudinal design
Simple cross-sectional design
Simple causal proposition
Structure of the classic experimental design
Classic experimental design: effect of difficulty of joining
a group by perceived desirability of group membership
Post-test only experimental design with a control group
~ e t r o s ~ e c t i experimental
r~b design
 1.157 01' FIGURES ~ 1 1 1

Cau.;at model of indirect causa t rela tionship bet-ween

Expc~.imcntaldcsign wrtf? multiple grtnlps and one
p r r - t ~ sand
t one post-ti'st
Snlnmon four-group drsign
I I\,pnthptical e x a m ~ l co f thc %I@rnnn four-group d e s i p
D t t ~ n i n ggroups in a 1 2 factorial dc-ip
Elrmcnts nf a 2 x 2 f,rrtnrial design
UFrm-t and indirect i ~ f k ~ cof t s gender on inctmrc
Pri,blt~mswith comparing averages
Graphical teprewnt.itionq nf productivitv changes bv
S u m m ~ r y~tatistics:productivity chanac bv bonus
sclicrnc
Srlccting statistical methods of analysis, part 1
Scli.cting statisticai mr~hodsof analysis, part 2
Sclrcting statistical nlcthcds OF analysis, part 3
Sc,lccting statistical rncthods of analysi~,part 4
k r n c nutcomes of an c~perirnentalvnrinblc on a time
writ,+
~ c n d c rand hcomr
MocEcl indicating indirect rrla tionshp h e t w c ~ nschool
t p t * ~ n student
d re!igiousness
Thcorcticallv derivd typology nf deviants
Variatinns of case 5tudy dcsrqs
h c - s h o t case study d e s i ~ n( m e group, pmt-test onlv
design}
Pattcrn matching for two variables each with two
categories
Pa ttcrnq for two variables each with 'three categories
Pa ttcrn matchine for parental sty Ie and child b~haviour
Logical patterns with two independent variables and one
dependvnt variable
L O R I Cppatterns
~ with one independent variable and five
dcpcndcnt variables
Prrd ictcd patterns d ou tcomcs on five dependent
varlablrs by level of worker autonomy

MultipIc prospcctrve p i e l de5ih.n

Sin~lrnpanel dtsign withnut replacemtant
Single panel desikn with replacement
Rotating pnnel d e s i p
C'r>liort s~quentialpancl design
Sirnplc retrospective pant4 design
Multiple point retrwpcctiue panel design
Simtrlnled before-afttsr rlccip
Crudc divorce rate, At~stral~o, 1970-95
Rmplo~mentrate of pcwple aged 55-59 by gender,
.Ai~qtmlia,1975-201 0 act~laland predicted
Emplovmcnt rate i y f people aged 55-59 by gender,
huqtrafia, 11975-2010 actual and predicted: (a) flattened
S C J ~ P . ( h ) c x p a n d ~ dscalc
Mali* su~cidedeaths and unemployment rate ratios, 20-24
vclas old^, Australia, l Vhh-sO
Fu-nuptial births and scrious crime rate, Australia,
1Y l ) i - L ) l

Cro--.-.cctional design: mnr~tai happinc5r; bv parenthood

status
G T T H Iin
~ Scross-scct~nnnldesigns
Rcpcntcd cross-sectional design
7 hrce causal models ot child's academic acliiev~plent
A \~.~riabIe bv case matrix
 12.6 %ern-order and conditinnal tnblcs illustrating a spurious
rcla tionship 209
1 . 7 Interpretshon of conditional rela tionship+c~rmparcdto
~ e r m r c i rdatinnships
~r 110
12.R Cnhort table of percentaE.c attending church a t least
LIST OF TABLES monthly hy agr cohnrt and ycar, 197U-20011: ageing effect.; 213
12.9 Cohort table 17f pcrccntagc , ~ t t ~ n d i nchurch
g at least
mnnthlv by agc cohort. a n d ycar, TY70-2030: period effects 714
12.10 Cohort table rrf percentaRe attending church at leaqt
mnnthly bv a c r cilhnrt and year, I9TD-?I]I10: cnhor! cff~cts 214
12.F 1 Cclhnrt table, lor lrreplarlv <paced surveys: ageing anid
m f i n i n ~groups in a 3 x 2 factorial design cohort effects fnr percentagr regular church ,ittendanc~(at
Dctvrnlin~ngthe IcvcE of measurement of varia hlcs least monthly) 216
Pr~n111ctiv~ty changv according tn incentive schernc 12.1 2 Cohort table for irreplarlv spaced survevs: period effect?
Mcan percentage c h a n ~ in c productivity by bonus scheme For percentage regular churcll .Ittendance (at Icast
Age cohorts over t i m ~ monthly) 217
Development effects: changes in pdi tical conserva tkrn
from 1 L W l to 2000 by aRe cohort
P e r i d r~ftcctq:changr.: In politic~lcnn.;en.atisrn from 1980
to c,;l(l[ril aKc cohort
Creattng groups from panel varintinn
Method.; of analysis I'cw panel data
Wav.; o f examining aggregate change over twr, waves
Adlusting for inflaticrn
C h a n q r ~In :-score< of individrtals
Chan~c.:measured bv changes in rank
Mean :-score for mcn and women, 1490-2000
Femalr carnings a s a proportion of male earnings
Changrs in proportions: bottom income quartilc as a
proportion of thc top income quartilc
Evamplrs of t n b u l ~ rwavs of prcsmting data about
aggrqylc chang~
ExarnpErs of tabular wayq of pres~ntingdata about
individual change ( l i v i n ~standards)
A d v a n t a ~ e sand disadvantages of tlircc methods sf
qucrtrt~tit~n~re adrniniqtratim
Sarnpli*S ~ J requirrd
~ S f o r ~rariouqsampling error.; at 4;
per ccnl cnnfidence Ic*\fel{simple ra~idomsampling)
.4n il lustratiiln of recorl ing masking a relationship
1lIu~tr;ltionof dix-idinl: a ~ a r i a b l erntn three ervlrp.; rt5ing
substant~vca n d dirfrit~~ttional recorlrn~:method..
Zcrwnrdrr and cond~tlc~nat tabTcq ind~cating
speCific;lt !on
Zero-ordcr and condi tic~naltablcq illir.;trating replication
Zero-order and conc! itiilnal tables il tuqtrating an ipd irect
causaf rt~lntionship

PREFACE
In 1"63 Campbell and Ttanlev q u ~ r r ~McCall's l BQ??! statemrnt that
'Thiw ari. t*xcelleni books mrl courstBsof insiruction dealing w i t h thc
statistical manipulation of c x p e r i m ~ ~ i t data, al but there is little help to
be Found i>n the m r t h d l ; of securing adequate and proper data on
wliicl~to apply statistical prtlccdure.' -1 here art* now cvcellent books on
snmc of t h r technical aspccts of collecting quality dnta (e.g, sampling,
qutwticm dt-s~gn,ob.;rn7ational technrqlres). However. it remain5 true
that there is a lack t>t attention ~ i v e nto 'securing adrquate and proper
ds t , ~ ' in tlic. spnsc III the lop(. of the type of ~ l . l tthrlt ~ a r r collt.drd ancl
thr cnpaci tv to m;l kc useft11 and cnnvinrinl: comparisons b c k e e n
c r t ) ~ !Tfi~< ~ ~ may 2 - r ~ 1 ~ 5 5+I\ in psvchologv, and t c l scme c~tcnfin
cducatinnnY r~search,hut in the other cncial sciences it remains true that
studpnts arc. typlc.-~llvpoorly prrparcd to think about thts most hasic of
se.;rarch I~.;UFS. C O M ~ U ~ Pand P t;ophfrticaM statistical analvsis pro-
gramr hatrr llclped In some of the dnalv.;is questions, but analysis
without first designing the structure clF the data coPIection will he flawed
and Iimited. Thew flaws arc all too nften c\+ident in a considerable
arnnunt of pmtgr;radu.~terescarch and in the work of too many estab-
1ishr.d rescarchcrs.
Many texts and averview coursm teach different data collmticm
c hi ~ i t ~ r ~ i e questionnaires
s t r a t e ~ e q~ ~ t as ws, and nbsemation, This book
is designed around tllc belief that these are subsidiary issues and that the
fcrcuq on dnta cnllcction strategv and r n e t h d s of statistical analvsis
neglects thc core i s s u ~ qnamely , Fhe smcture and d m i p of the wsearch.
T h t l focus on data cnllcction techniqul-s leads to the n e ~ I e c tof appro-
prialc comparison grciups anci pays too little nttcntion t o the notion of
rt'a lua tin^, a [t~~snatil-t. c ~ p l a n a t ~ c >;.onf sa phenorncnon.
rlic n e d tn attend t i , Issues of research design - thr structure nf the
rtasrarch - a p p l i ~ srqi~nllyto 'cl~ralitativc'resen rchers a s ti1 'q~~antitntive'
rescarcher~.[ t npplte~r~+eardtcs.;of tht. rarticular methtdq that art1 used
to collect d a t a .
T l ~ cbook i.; desim*lr~lto show social wirnce students thr impcrrtance of
artcnding tn rdcsiy i s w r s rvhl-n undcrtaklng scrcial rescarch. Ont=0 6 the
hrnr.lan1cnt~12fl;l!c.; nf ;l grPat deal nf wcial stlrnce sewarch is that its
d~si~ and
n structurt. arc inapprnpriatr f<lrthp u.;rs to which i t IS being
put and often d o not supyurt tlrc c o n c l ~ ~ < i rthat
~ ~ l,ire
s Frcqi~crltlydrawn
from i t .
Technology has assisted greatly in the conduct r ~ qocial f rrqearch. Put
some ch the basic thinking about the lr)<v~r o j rr~mrt.li is <till missing in
research, and no arnor~ntof technology will rcplac~the nec~lfor a clitar-
sigl~trdtlnder~tandinjiof the princ~plcsof research design.
Tlie aims of thr book arc.:
1 to prm-ide social scicnce students with a clear understanding of the
importance of research design and its place in the res~archprwcss
2 to dcscribcn thr m a i n tvpas nC rcsearch J~rignsin s w b l r*rscarc!n
3 ti) rliqcuss thc.;c dcqigns in terms of thtarr I o ~ t c40 that 5tudents a n d
resmrchers will be ablc to evaluate partlrular ~ l r s i g nstrategies d n d to
sefcct stratcgics apprrlpriate to the problem a t h a n d
4 to providc students a n d rcwarchen with the principlcu; of d c s i p
such that thvy will L-rr. ahle to adapt dcsignr. efifrctix~elvtrl meet the
contingencicq o f real world rescnrcli s r t ~ ~ ~ ~ t i o n s
5 tn providc <ti~dents .>nil rcscarchers rvith the tmd.; by which they can
n t esltrcn~thsand ~vcakneweq[if vannllc design <trateciis.i
c ~ ; l l ~ ~ 5hp
h t<o highlight the mat11 t e c h n i q i ~ ~tor
s an,!lysing d ~ t acnfllccted i l h ~ t ~ ) :
the various drbqip str.ltc~icsa n & t ~ provide
1 a n u n d c r s t a n ~ d i nof~ how
to sclcct the moqt appropriatt- rnethnil n l anal\-+i<.
The b(wk is divided intrl five parts. Part I consider5 types of seWc~rch,
research concepts, research que.;tionr;, cawality and some crf the cnnccp ts
that arf l-rasic to under5tanding rcwarch destgn. Each of th~.other four
parts focuses on n broad typc of research design: cxperimcntal, I o n ~ i -
tudinal, crr~ss-wctionaland case .intdv. Within each part the bmk:
m de~critresthc. hasic dcqign and its variants and provldcs cxamplrs of
studies using these designs
identifies thc strengths and \rcnknesr;cr uf i l i ~design and I t r a y of
maximizing the strpn~tlisand minimizing the weaknesses, centre~i
around the three cow x e a s of methodological, practical and ethical
cnn~idrrationq
a ouhines thc main ways of a n a l y s i n ~the bpr of dain that arc
collected uqin): the particular dcsign.
In preparing a book vich ar: thi%them art, man!. proplc to tvhorn one
owes tll~nks.1 w , ~ n tto , ~ c k n o w l i ~ r lrnv
g ~ hpecial appreci~tionto Chi.;
Roiek, Kcn Dernp~cl;,[an Winter .ind lunc r j e Vauc for thtir rnccrumqc-
rnent, advice a n d timc.
 PART I

WHAT IS RESEARCH DESIGN?

THE CONTEXT OF DESIGN

Before examining types of research designs it is important to be clear

about the role and purpose of research design. We need to understand
what research design is and what it is not. Wc nccd to h o w whcrc
design fits into the whole research process from framing a question to
finally anaIysing and reporting data. This is the purpose of this chapter.

Description and explanation

Social researchers ask two fundamental types of research questions:

1 What is going on (descriptive research}?

2 Why is it going on (explanatory research)?

D~scrfptiveresearch
Although some people dismiss descriptive research as 'mere descrip-
tion', good description is fundamental to the research enterpristr and it
has added immeasurably to our knowledg~of the shape and nature of
our society. Descriptive research encompasses much government spun-
sorcd research including thc populahon census, the coElection of a wide
range of social indicators and economic informatinn such a s household
expenditure patterns, time use studies, employment and crime statistics
and the like.
Descriptions can be concrete or abstract. A relatively concrctc. descrip-
tion might describe the ethnic mix of a community, the changing age
profile of a population or the gender mix of a workplace. Alternatively
tFlr dc.cri~~trc,nrnrght a& morr ahshacf qiicsshon\ quih a5 'I.; the 1evt~Io f
inequality incrtx,i+il~g
sr,~.~,il or ~ l ~ c l i n i n ~'Hnw
", sccular i5 societv?' or a] Dtrect causal retationshb
'Hcv\*tnz~clip o t . ~ r h 'i~ there in th!\ cun?mttnih7'
Icc~lratpdtlqcripti~rns thtb l ~ v e lot ~lncnaployrnt~nt o r potet'rty have
hi\tnrically pl,~vcda ktlv rnlr in sr~rialpol~cyreforms (Mar\li, 1487). Ey
rlc*rt~rlmstratinthe er~*tcncenf scrc~alproblems, competent description bj Indirect causal refationsh~p,a musat chain
can c h a f l e n ~ accep'tt~~i
t~ ass~lrnptronsatlout the tvav t h ~ n parc and c,ln
prc,vtrke actit~n.
Good d e w r ~ p t i o n pmvokcs tlic 'wlly' quel;tions of c ~ p l a n a t o r v
rrst.arch. I f we1 deter! p e a t c r S O C I prrlar17ation
~ ~ trvcr the last 20 vmrs
cl A more complex causal r,?od~l
sf drect and rndrrecr causal hnks
(I.(*the rlcti are sett~nt:rtcher and the pnnr arc. grtSinq pnrrnlr) rvr arc
ic~rcrtdto ;ltL '\Yhv i~ th15 liaplx~n~iig?' Rut hcfnre asklng 'why"we muqt
bt* sure about thc fact a n ~ dlnirn.;ions
i 01 the phenomenon of tncreasing
pcll,~ri/ntirln.I t 13 all vrry \t.cll to develop etabnratc tlimrics as to n h \ -
sc~cicbtyrnighl m e n b polari7cd nniv than in the rccent past, but i f the
hisic prcrni.;c is w r o n c ( I c. socltlty is not bccnming nlore p7nl,lrizcd) thcn
atlthnip&tc, criplain a non-eu~stcntphcnclrnenon arc vllv. Part flme or full
Of cnursc i[csmipticm can tIcll;rnerattt to rnindrrqs Fact g'lthering i.rr Ch~ldcare
what C.tV Mills ( l 0 5 U ) callcd 'ab.stractrd p~npiricrsn~' fhrrcl arc plrnty
~ ~ c ~ c l and castn s t u d ~ c sthat rciport trivial
13f twniplt>s o t ~ ~ n f c ~ c t si~l.v<~vs

inf~rrnai1r3n,~ncifail tn pmvrrktl anv '12.ht ' ytrestic>n*.or pro\,lcli, an!. b ~ r ~ s

for gt.rreral~rat~on.F h)nfcvcr, 2 h ~ s1s a tunction nt ~ n c r ~ i i s e y ~ r c n t 1 ~ 1
dc.;cripbinns rclther tlian an incl~cimentof descriptive rcqearch itqelf.

occuyntic.nn1 opEinn4, \.r.hich are linkcct to npporbunihcs fnr promotion,

which in turn affrct income Icvcl [Flg~rrt.1-lb).O r u7eco~rldpmit a more
Eupl,l~~atorv re~~earch Cncuses on i l ~ qi~cqtions.
l ~ ~ Fnr rxarnplr, it is one cornplcv model i n v o l v i n ~a number of interrcl;rtcd caulinl chains (Figure
th~nc:to dcwrrbc the cr~nlrrate in n cntlntry, to e ~ a r n i n etrend5 over time 1.1~).
nr tcr cnrnpsrr thc ratrs in diffcrcnt countrrc.;. It is quite a ditfrrent thin^
to devplnp r\planabon.; about why the crime rate E l i SI; high ~1s~tis, why
PREDICTION,
CORRELATION AND CAUSATION
sornt. type.: of crime arr increasing or why the rate is htght-r in some
cn~orit~ics than in ntIapr5. People o f t m confuse c o r r e f ~ h o nwith carrqntic>n. Simp!\. because one
'tht. way in which rpsearclicr~devcl.clp rescarch designs is funda- event f{~llawsanother, or two factor< cn-vary, does ncrt mean that one
men t~ lly afftbctrd bv wllether the research questton iq descriptive o r causes thc other. The link bctween t w o cvents may be cnlncidental rather
euplanatory. I t affect.; ~ \ ~ h ,information
it 1s cclIlwted. For example, if tvtX than can.;aP.
want tcr c ~ p l ~ ~r n v some h ~penplc are mortbl i krlv to b~ a p p r r h ~ n d p danif Thew 1.; a cctrrclntion k ~ t t v c r nthc number of fire engincs at a fire and
convoctrd i d crimes rtzr nccci to hnve hunchus about why this is so. lVr the arntn~ntof r i a m n ~ ccnuwd by tlic fire (the more fire cngines thr more
mas havc m~invpnwihlv ~ncurnlp~~tible htint-l~csand will nivd to collrct ciamap*\. Is it thcrerore rcacc3nable to cr~nrlitdtthat thc numbcr ot fire
infrrrmatic~n! h ~ rnablc<
t 115 tn \cvt. s.r'l-\ich!iiznches ~r.c~rk best t.rnpiricall\l. en~inescauses thc arnnrtnt of damage7 CTcnrlv thc number of hrc
~2n\1vcrin,: !litb 'M-111' qilestion., 111~01v1>s d e ~ c l o p ~ nsnrrzr?!
g cxrcplann- rnpj;mr< d n ~ the j amount c,t damage will boilr 1,tb due tcl some thlrrt factor
tiotls. Causal c.xplancition\ argur that plitnoinenc~nY ( e . ~incume
is atfrctcd b! tactnr S ( t . . ~ .~t.nc4t.r). %rnc causal t.kplanatlc7ns will 21r.
. l~vcl) - such a s the serio~rsness(if the flrc
Similarly, a s t h dtvorcv
~ rate changed over t h e twcnticth century the
s i i n p l ~rcliilt~o1ltt.r~13 i l l be rnc,rts cr3mplcx. For c.iamplc, 1z.p might argur crime rate ~ncreasr.il R ~ C M ' 1~ear5later. But this dws not rntBnn that
that lllcrc a rforerf rrflrct of pbndrr on incomc (i.r. siniplc genJr*r divorctl causes crime. Rathcr than divorcc calrsing crrnlc, divc~rcrand
drscriminatinnl (Figurt- I la). Wr*might arb:ur for a c,llrsal ch- in, such as crime riltrs m ~ g h tboth thr clur to cjIl~r*r social yrocessc.; such as srcular-
th;lt ccnder ,ittt~ikc h t ~ r i rof held of t r a i n ~ nxv~trc~; ~ in hlrn affrcl5 i ~ a t i o n greater
, i n ~iv~dt~alisrn
l or pot'rrtv.
 Sti~dentsaf fer paving pri\.atc5schonlq kprcallv perform better in thvir
final rrar nf schnolin~than tlic~sca t govcrnrnpnt fundtd u~cst~cwls. R u t th15
nced not be bcral~seprivate scliooEs prndlrt-1, better pcrformancc. I t mav
be t h ~ attending
t n private school and bc~ttc*rfinal-yrar prrformancc arc
I v f l r tht- ouutcomc of wme othcr cause (sec later discuwon).
C o n f u < i n ~caus;l tion with corseFation akn confuses p r ~ d i c t i o nn0ith
causation and prrdichon with rxplnnation. U'hcre hn.o ~ \ ~ c nor t schsrac-
terist~csare corrc1atc.d we can predict o n r from the oth1.r. Knowing the
b y e elf schwl nttrnded improves our capacity to prcdict acadcmic
t . this does nnt mean that the whod t ~ affects
a c h i ~ v ~ r n e nRut ~ c ;ICJ-
demic nchievrmmt predict in^ pcriorntancc on the ba+ cchor~lt ~ p o
doc.; not tell uq n71ry private school studcnts do better. Gnori pr.t~dirtinrrrilrt.5
not ilr7pr3rrd on rnt1~17Irrlntirvzslti/~sNor doc's (hr, nbllrt!~to prrdict accrrmr"rl!l
rirnrt>ri~trrrlenn!rtlrity n l x ~ i r fm l ~ ~ l t l i f ! ! .
Rclcognizing thnt causation iq more than correlation highlights a
~ r o l d r m IVhzle
. wr can ohsewc cnrr~lahonirlr mnlmf r>flsr+rzgtvCflll5t'. Wc
h a w tn irrfrr cause. These infcrmces howtvcr are 'neccqs,~rily falliblr . . .
[tliryj nrc onlv indirectly linkrd to o b w r v a h l ~ ~ '
(Cook and Camyhr>ll,
1 W u : t i ) ) . Becnuw n t ~ rinfcrtlntcs are fallibl~we r n m t rninimi~ctht>
c h a n c r ~of incclrrcctlv 5;lrT1n);fIr.lt ~3 rcIntion.;l~lpis cawal t\ hen In tn<t i t
is not Ortc @! fhr ftl~?,fi7~rren!#/ ~ I I ~ ~ W Cd W r 1 ~ ~ 1 ~ 1 t1ir~c h
q~qn ~sp!~~r~/it~rt/
is to I~zJ(I;TJ
~~'SI,RI.C\I I H I J R infirr~wt:s.
~ ~ ~
D ~ F R M I K I S ATNI DC I'RORABT1.15TIC COYCTII.\ OF CAu5?\'l l0\i
'Illere arc two ways of thinking about causes: deterministical/v and
prot-rabilistically. Thc smoker who denies that tobaccr, causes cnncrr
becausc he srnok~sheavily h11t has nut ccantracted c~incerillrlqtra tcs
deterministic cawatinn. Prob~bilisticcausation is i l l ~ i ~ t m t c bv r l hral th
authorities t ~ h pnin o t tn the incrttased chances. of cancer nmong s m o k c r ~
D ~ t ~ r m i n i s tcauwtion
ic is whcrc variable X is said tn cnuw Y it, and
onlv if, X rnvnrinhly p r d u c c s Y. That is, wlicn X is prvstlnt then Y will
'neccwarilv, inevital-4~and infnl lihly' occur (Cook and Campbell, I Y79:
14). This approach to e<itat,fi<hcausal In;llf quch as: r\*hrnevcrw n t t r
is hratcd to 100 O C ~t always boils.
In reality Iaws arc never this simple. Tliry will always specify p;Er-
ticular rtlr?dific??~suncdcr which that law i>pcmtcs. Indt~t~cl a grPdt dral of
scientific investigation ~ ~ V O I Y P S < ~ C C I ~ $ R F ; the condl t1c3ns m d e s which
particitlar Paws crprrabc. Th11s. tvc. might 5av that nf G r n Irr+r.lheatme tlrrrtj
zrattlr t o 100 *C wiIl illwavs C ~ I I + Cwater tc, hc+iE.
Altcrna tively, t l ~ clarv might he stated in tht. form ot 'othcr thing-,
being equal' then ?( u~11always p m ~ - l ~Y~.cAed~terminrsticversion o f the
relnt~onshipbetwrcn race and inclrme let-el 1v0l1ld5 . 3 ~thnt other thin$:.:
being [,qua1 {age, c=rlucation, pcrsonali t y , c\pc*rience c"tu.1 thrn ;I xvhr t c
perscm tvill [alwayql earn a li~ghcrincomc tlian a black prrstm. Thai 15,
I
racc (X) causes income level (li ).
Starcd like thi5 the notion of det~rnministiccausation tn the < t l c ~ n l
scienc~ssound< o d d . I t is hard to c o n c ~ i v rnf a charact~risticor w P n t
that will invart;lhly result in a given outcornc even i f n fairly ti#[ sct uf
conditions is spccificd. Thc rnnr(?/r7x~h[ of human social bchaviour a n d thc
sr~bjt*<t~i'c, ntzr! ~ r r d l ~ ~ l l n r l scomponents
n~m)rin~qfrrl hc of human behaviour
mcan that it will never be po.;qiblc to arri1.r at causal statements of the
h ~ 'Ifc 31, and A and R, thrn Y will alw,~yqf~llow.'
Most causal thinking in the social scicnccs is pral~nI~ilis~ic rather than
fitrrrttiuistic (Suppcs, 1970). 'That is, wc work at the level that a Riven
factor Incseawq Iur drcreascsl the probabil i tv oi a partictl far crutcnmr*,for
cxam ylc: bcing tcmnle incrmws the prt>b.~ t i l i p of 1%-orkingpast time;
racc affects thc p r t h a b i l i ~of having a high status jnh.
W r can impmvr probabilistic explana t ~ o n sby spwi fying condi t ~ o n s
undtar which ?( i.; less Iikelv and more likely .to affect Y. Rut we will ncvrr
a c h i ~ v ecomplete or detcrrninistic esyl.>nsticms.Hun~nnhcha\+iot~ris
both ; r 7 i ! l P d and mrrcril: there i s a double-qi Jrd character to human social
behaviour, Pet~plt.co~zstrri~tilieir social world and tlirr~are creative
aspccts to hurn'ln action but t h i ~freedurn ,qncl agcncv will alwavs be
constrnincd bv !Fir qtructurc.; within which people l i ~ c Reca~rse . bc1hhr7v-
icl~~ I < r not sirnpl! dcttrrn~ncdrre cannot achlcvc dctrrnl~nlstic explarla-
ticlnq Hmvevt~r, because hrhavinur is ctlnstrainccl w c can achirvc
probah~listicexplanations. Wc can say that a given Factor will inrrcase
thc liklihond ot n given c l ~ ~ t c o m butr tlicri*will n w e r b r certainty about
otttcnrnc%.
Dcspite thc ~robabilisticnature of c~u.;al statements in the cocial
science';, much popular, ider~lo&al and pr~liticaliliscnurse translates
these into deterministic statcrnmts. Findings about thr causal effccts of
c I ~ s s gender
, or rthnicity, for cxnmplc, arr nften read as i f these factors
invarinbly and ccrrnpleteIy produce particufnr outuornrs. One could be
forriven for think~ngh a t social science has demonsttatcd that gender
completely and invariably dctrrmincs positinn in ~ c r i c t y , roles in
famili~s,values and ways nf relating to crthcr pcoplc.
Theory testing and theory construction
Attt~rnpkto a n s w l r the ' w l i ~ 'clue st ion^ in social scirncc arc. theorirs.
Thesc tl~cnrieqI nr\- in their c - , l t r r y l l r - . ~ ~ e(hcmr
r mnnv var~ahle<and I rnks),
i l f ~ ~ f r , ~ t and
. l ~ ~ ~.;t-opr.
~r To understand t h r role of thnlrv En empirical
1.csenrc11 it i~ u.;cful to dlbtingt~iqh bctwc.ran twn diCft-rent sh)lc.; of
r ~ s r a r c h theory
: tcsting and theory building (Figure 1.2)
Thenry build in^ is a process in which rrwnrcll begins xvith i>bser\-iltinnq
and uccs irrdrrrfrrv' rcnsoning tn derive a tlirnry from thrw obscrvatirrns.
 T H E COYTEXT 01:I3ESIGN

Empirical Stan
level here

Low (a) Ib)

Conceptual-abstract
level Parental conflict
H!qh (cl Id)

Theory testing approach

Conceptual-abstract Start
~PVPI here
~f Elri* Ihuo~!, is Irttr t h t r crrfnrn Ilri)~<qs f;>f/oii*in t h ~
_slri~til~i real worFd. MIp
Empirical
thcn a s s ~ ~whctlicr
s: these yrc*dictions arc correct. I f tF~cyare correct thc
level theory i s supported. rf they do nut hold up then the thcory nccds to be
eithtr rejected or modified.
, may wish to tcqt the tt~ro?'that it 1102 divorce itself
Fnr c r ~ m p l e we
that nffccts the ~ f c l l b e i n gof children but the le~~el of conflict trctw~rrt
parrnts. To test this idea we can make predictions abnu t the u~tllbcit~)= of
c h i l d r ~ nunder different familv conditions. For the simple throrv that it
Thcrta t l ~ c u r ~ .lttcrnpt
es to mnkra .censc of otlrcrvations. l<rc.~usethc thcsorv is p~rcntalccintltct rather t h ~ ndivurcc that affects a child's ~vrtl'tlring
iq ~ T C K ~ I d I tCi - ~r i>I-t*tnration':are madc i t is o f t ~ ncnflcd p o ~ t,(;7(*1141?1 thert are four hn.;rc 'conditrons' (see Fiq~trt*1.1). For ~ c 'ccmditinn' h the
theory (Merton, TWK) or ex / ~ o fnrlo ~ f theori~ing. thcwry would make different prediction.; about thc Irt~elof childrcn's
W i i z form of thcory kruildini: r n t a i l s askiny: whether tl~cobservation is wellbeing t h t wc can examinc.
a pnr.ficrrlnr cnsr, of'n Iwrr g~~rr,mlfni7ctor, or t ~ n wthe r~bsrrvation-fitsrrtto a I f the theory that it is p a r ~ n t a lconflict rather than pnrcnbl divorce is
pttr.rtr or n stcrnr. For e ~ a m p l c Uurkhcim
, obscrvecl that tht. suicide rate correct the following propoqitions should be qupporttd:
was higher arnonfi Prfrtcstaxltr thnn Catholics. Rut is rvligious affiliation a
partict~lnr case c)f something niorc general? Of what more gcncrcll
phenomenon might it be an indicator? Arc there other ohscrvatinns that Prupositio?r 7: children in srlrvttions (n) n t ~ d(b) ujnrrlri 1~ cqtml[v 717t711
shed light on this? He also ob.;cn~cdthat men were more suicidal than off That iq, where parental conflict law, chi1drt.n with divnrcrd
is;

ivornt3rt,urban dwrllers morc than mral drvellers and thc wciaE11~mobile
mow than the sot-ialiv stable. Hc argued that the comrntm factor behind
all t h ~ s eobservations was that those groups who wcrp mast suicidal
'
wcre also less wcll socially integrated and crpcrienced ~ r c n t c ar r n b i ~ t ~ i t v
about how to behave and what i q right and wrong,. He tlrrorized that one
of the c*xplanation.; for suicidal behazrinur w a s a wnw of norrnlesr;ncsq -
a d isconncctedncw rrf indi\o~rlt~als frnm tht*ir w c i a l ~vnryd.Of course,
thcrt>may Iiavc b c c ~ i ir~therways nf accounting for thcsr observations but
a t Ica.;t D ~ ~ r k l i ~ i rrrplanatinn
n'+ was cunsi.;tt*nt with tlic f.lct5.
In contrast, a thcory testing apprnach 1~r;yi~tswith a theorv and u w s
t h e u n to ~ u i d crvliich o b f ; e r v a ~ ~ nto
n sm a k r i t move< from the fit'ntral
tcs t l ~ cp>nrtrcuFar. Tht. nbstanatiun5 ~ h c r ~ ~ lprovidecf a tmt of the worth
parents will do just a s weIl as those whose par~nt..;are married.
rn I'rclprrsifirrn 2: cf;rldren In ~rftmfion.: ( r )~ r r i l( d ) sltn~rl~l
be eqrrtrlllt rworfy
vff That is, children in conflictual couple 6amiEies will do just as
badly as children in post-divorce familirr where parcnks sustain high
conflict.
rn Proposrticrrl 3: rlrrfdrnz if; sift~ntt(lrlC ~ rvill J do ;~vfit'tltnrt t l m v rn s r t t m t i ~ ) n
(nl That is, those with married pnrcxnts in high crmflict will dcr
worse than those who h ~ v c married . pnrtvts who n r r not in conflict.
Srr~position4: 1.llrldrrbnirr .srtrrrlhon l d ) nivll i l r ~7rrorstut l t l r r l H F D 171~ s i t ~ r n f t n l i
(b) T h a t i.;, those with divorced p r c n t s in h r ~ hconflict r v i l l d o
MFrrr<cB l a n thore who have divorccd parrtits who .~rt-not in conflict
Pri?osilion 5: rhrlriml 111 sihrntro~(bJ ~ i r r l lrln brffer f l i n ~Hiosr ~ in t t r ~ n l r o n
(c) That is, children with divorced parunts who art. not in conflict
will do better thnn those rvith married parents rvhn are in conflict.
* ~ c ! t ~ ! ~ i rium < f / t m t l o r( ~r )~ ~(*i!! bP!frr ~TI~III / h ~ ti11
~ ' r o p p s i t i o tti- > ~rf11(8l1t)rt

of thr theow. Usrnx tk?drllrcfir-vrrnsoning to cFcrivc a syt of propoqitinns (iil That is, children t v i t l ~married parcnts who art. not in ronflict
from thtl theory dolb.; t h i s . We nr.rd to dcvrlnp ihcse propositions st1 that will d o betlcr than thmc with divorcrd parent< rvlln are in cnnflict.
 THE CTJVTt-X 1 O F DESK;\ r~

Similarlv, ~ ) ~ i ra~ ls e a r c hntvdk design or n <tructure before dGitn

culEection or tlnL~lysis can conlrni.ricc. A rewarch d c ~ i ~ rrnt just a wnrk
is n

J
I Theoy ,1 plan. A ~vcri-kplnn rlctails what ha< tr, be done to cumplctc the project but
tire wnrk plan will flow, E r ~ mthc proiirt'5 rcwarch d r s i ~ nT.T l r p firrtrtrorr of
Inference Deducrion
R S C ~ ~ * R T CrEr*s:,cl~
!? 14 t(7 rf:siIrt7 !ffr?t I ~ R ~~~ii-inrct*
' IFIF? IF?('^^ 1'1111!7!~'.; ?Iff~ Rl15illt'l' / I l l '
r r r r n ~ ~ z b i ~ r w17.~ yws.;~\~/r.
l ~ l r t r n lql~rsfjtwros r~h Obtaining r ~ l c v n n te17idt-ncc
entails syccifying thc type cif ct'idct~ccnwded to a n s w e r the wscarcli
question, to l r s t a thevry, tc) c~nlunte a propammc o r to accurntcly
describe somc phmomenon. fn other words, when designing researcl~
Develop measures.
w e need tn a&: gtvrn this reqtarch q u e ~ t i o n(nr thtclr~),what hpc of
samplp etc. evidence ntlt.rltd to anqlvcr tht. qllc~tion( c r tc5\t nhr theov) r l f 17
crvrr~itrri~;~
rr>rlr/'
Resrarch d~,.;ign 'deals witli A Ioqit-nl problem and no2 a Ioxis!~mE
1 Collect data I problem' (Yin, 1iIH9: 24). Bcf~>rca httilclcr nr arcli~tectcall deveIop a work
plan nr ordcr materials they mtist First establisli thp tvpe of building
Start~ngpoint of
theory burfd~nq required, it.; II+PS and the nrrds u f the occupant.; The* wnrk plan flow.;
From thiq. Sin1il,~rlv, in sucial rtl\tnrch the isques o f qnrnpling, method of
data collcctir\n It..$, qurstitmnnlirtx,nbscrvat~on, dtwl~rnent analvsiq),
p quc.;tit~n-,are all s~rb.;~drarv
d ~ s i of to the rnattcr of ' l i ' h a t evidenre do
1 need t r ~cr~llcrt"
Yo ';in+> ~-rrnposition1~1-nuIt1prtlvide a comprlling test of thv original Too oftcn n~\cnrchcr~ design ( ~ i ~ ~ ~ \ t i i i n n aori rbrl~ill
cs int~rviewingfar
theory. In~lccd, taken crn it.; ozvi~proposition 3, tor cxamplc, would ton early - brfurc thinking Lhrou~h what informn I ion hey r e q ~ ~ i rto r
reve,~lnothing bout t h impart ~ of d i v n r c ~ .F-lowrver, taken a pack- answer t11t.i r ri1c;tlnrch queshclns. \Y ithou 1 attendi nx ti] these research
age, thc ~ r - foi proposihon.; prtlvidcs a stronscr tcst of the theory than anv design rnattcr.; ;it tlic beginning, tlic ctmilusions drawn will norma2ly hc
single prclpnsition. weak and unconvincin~and f;l I l to answer the rficbarch question.
r ' i l t h ~ ~theory
~ ~ h testing ntld theory building arc often yrest.nted A S
c l i sllould be part of one ongoin!: process
alternativr, rnodcs of r ~ s ~ ~ i l rtlic'y
(Figure 1.31. Typicallv, tlieorv I7uilding will prcjduce a p l a u s i h l ~account
or explanntfon uf a set of ohsrrvations. Hnwtqvcr, w c h explanations are
f r ~ q u e n t l yillst one of a number of possible cxplanntions that fit khc datn. Research dcsign is different fr~jnithe method hy which data arc
Whilt plausible they arp not necessarilv compelling. Tkev r c q ~ ~ i r c collected. Mnnv r~searchrnttl-lorlc tcvts confuse rrwa rch designs with
svsternntic ttlr;tln:: 5%-hertdafa arc coIlwtcd kn spcrifically evaluatc tlnw methods. 11 15not nncomrntm to scc research design t r c a t d as a rnoclc of
\\.ell the crplanntion holds wllrn slrbjectrd to a r<lnKcof crucial tfit.;t.;. data cullerti~~n rather than as a Ic7ctcal ~ t r u c t u r eot thc i n q u i v . But therc
1s n o t h ~ nEntrinwc
~ about any rr5tlarch design that rc.qllrres a particular
method of data ctlliection. Al thougIi cross-section~l st1n7eys arc frv-
quently equattd with questioiinai r r s ,111d case ~ t u ic.i d arc often eqirattd
What is research design?
with p;lrtrcilrant olwcrvatinn ( c . ~ W . h yte's Sfrr*r'! Cor'rrr'r Sncicfi/, 1443,
datn for anv dt,.;icn can S r t ct7llrrtc.J with any datn ccrllcction methot!
How ic thtx tcrm 'research rIrn.;iqm' to bc rtqerl i t 1 th~?; book' A n analnqv (Figure I .sl. t l r l \ v the data arc crlllrc.td is irrclel-ant t o the l t l ~ ~nfc thc
rn1cl7t h i p ii-hcn cnn5trircfri1!: a b u ~ l d
in< !Iic~rv1s no pn~ntorilcrunr: JP~F~T
matcrj~lqo r +ctting crit~catdate.; for c o m y l ~ t i u nrli project s t a ~ c l~n!il
s mrt' Failing trl cFl<t~ngulshtret~r-ern r l c s i p and mcrhtx-l leads to prnw
know {what sort of building 1s bring c o n s t n ~ c t r d YIP , first decision i.: e ~ ~ a l u a t i oof
n dt%.;qns. Equating crr~rs-sectionald ~ s i h ~with .; questinn-
whethcr w r nccd a high risr, cltficc building, a factory Ior manufactt~ring mires, vr cnsc s t u d ~ ~with s parlicipant obsc.rv,~trtlr~, mcans that thc
t home or an ,~p;lrttnmtblock. Until this
machintry, 3' school, a r ~ s i d r ni;ll designs arp clftcn ~ ~ ' r l l ~ again<!
~ a t ~ dt h strengths
~ ~ and wi~aknesscsof thtl
1s dolit* rrttl r-annot sketch n plnn, c>l>tain pr,rmits, wurk o r ~ t;1 ~ v c ~ r k methot3 rathcr lhan their ability tc, d r a w r c l a h v c l ~i ~ n , l m b i ~ t l o uconclit-
s
5cheJtilr~cjr clrdcr rnatcri;ilq. won< o r to <t.lcct I-rct\r,een t i v ~ lplnusil-rlc h v ~ ~ t l i ~ ~ t , * .
 D@slgn
tvpe 777 Lonq~tvdrnaF Cross-sectronal
a p i n t of c n n t ~ r s ~ .~.m I ~ a sbeen thr t ~ n f o r t u n a l rI l n h i n ~1-rt.t~
study m e t l i o ~and

t k ~ o ~ that
l

~ h t the case ~ t ~ t dr nv c t h d re~llrited thcm

collcchon rnethcds . . . (31 the contrArv, the rnrthcxl dm:: not 1rnp1~.
~ r l i c u l a form
(lcW3: 32)
r
:

o n qua11tativv methods, ettrriograyhv, o r p a r t i c i p n t nbsen-At Inn. I'cnplc 11av~-

1at.n the c a w

ccrtnln typw 01 date> r t ~ l l e c t i o n- hlr i~xamplclhose t ( x u s ~ l ~ ~

embracr t l ~ r s edata

of data cid!echon - which can hr q i ~ a l t t a t r v rnr quantitatrvr.

5imilarly, Marsh (1982) argue5 that quantitative 5unreys can prwidc

an\'

Ousst~unnaire Quest~onna~re Questionnaire Questtonnaire information and explanations that arr 'adequate at thc l t v d o f mean in^'.
Whilc recognizing that rurvey rt.sc;lrch has not ,llwavs hrcn goorl at
1 Mefhod lntphr~ew lntew~ew Interview Interview
1 data tapping the suhicctive dirnt~nsionof brhaviour, *hc argue< that:
(structured or (structured or IstrucTuredor (structured or
COlleCT10n loosely I loosely loosely 1 loosely
structured] structur~d) structured) structured) Making sensr o( social action . . . is . , . hard and sur\rcys have not traditionallv
h c ~ nw r y good a t ~ t TIw . cnrlicst sllrvrbv rfiearchvr.: started n tradition . . . nF
b r i n g i n g thr rneanmg frtrm nutside, ~ i t l ~ bv c r makin): u<e of thv researcher's
Observat~on Observation Obse~atlon Observanon c t t ~ l rnt p l . ~ t ~ ~ i lcxplan.~ttons
rIc . . . or hv htinging ~t trom w b < i d l a n - rn-deptl~
intrnriews <prinkling quts~m. . . Pibernllv nn thr r a w cnrrelahon%drrived frcrnl
the survey, Silrvey res~nrchhrcamc much mirre ixxciling. . , when it h c ~ n n
Analys~s01 Analys~sof Analysis of Analysis of
Clocuments docvm+nts documents
~ n s l u d i n qmr-anincful d~mlw.;ic,ns vn l l ~ cqtr~dvdcki\:ri. [Thi? ha< Ilc,rn donr~1n1
documents
I hvn rvavs, f ~ r < t l vIbyJ ashinl;. the actor cither for kit-r reastin% dirPctlv. or 111
I
<upply ~nfnmatuonabout ~ h tc- ~ n t r a vl a l u r s in ht-r I ~ t t .around ~ t l u i hrye m a r
a w l m e she i s ortenting her life. IThl.;l involvrp collect~n): a .;uffici~ntlv
Unobtrustve Unobtr~slv~ Unobtrusive Unobtrusive t which an actor fiilds h e r w l l Ilint n team ol
c o m p l e t ~ptclure of the c o l ~ t c x in
methods methods methods methods ou biders mnv mad off the mcaninpful dimensinn\. (1 Q82: 223-4)

-
-

The need for rcsearch design stems from a sccpticnl approach ti) research
and a view that scientific knoruled~r.must alrvnvs be prwisional, 7 h c
purpose of rcsrarch dedm is to r~dziccthe ambiguity o f much research
evidcncc.
We can always find sornc cvidenct* consistent with alrnort any theorv.
QUANTTTA'I.EVF AND QuAl.tl.AITVt KPSCARCH
Howeves, we should bc sceptical nf the evidcmcc, and rather than
seeking evidence that ic ron.ci<tnlf with our theory w e should seek
Similarly, d c s i ~ arc ~ s oftcn equated with qualitative and cluantitativta evidence that provides a m r r i p t * I I i ~tcst
~ ~ ~of the tlirory.
rcqcnrch methods. Social rurveyq and pupcrimcnt.; are fr~clucntlyviewcd
There arc two related rtrategie.; tor doing thi.;: elirnina ting rival
a.; piirne cuamplcs nf c l l l a n h t a t i ~ rrst.;lrch
~r and ~ r trvaluatt*d
. a ~ a i n s tht>
t
euplanationc of the er.idcnce and dcliberatrly sccking evidcnce h i t
strmgthq and tt-eaknf-cc of statiqtlcal, q t ~ a n t i t a t ~ \ .reccarch e meth0134
cnu l d rJisymi~.thc theory.
~ n anal\-<I%.
d Case ?turltt9.;,on the ~ ~ t h hand,
cr arc oftcn qrcn a s prime
rauample:: of y i ~ nita l tivc rrsr.nrch - w l i r c h acfopt5 .~ni n t c r p r r - t ~ v approach
e
to data, st~ldres'things' \rf~thintlicir ctjntext and considers thr subjective PLALSIRLE
RIVAL IIYPOTIT~SI:$

mcnnings that pmplc bring to their situatinn.
I t is ermncclus to equate a p t r t i c u l a r r e s r a r c h deqign with eithpr
qunntitahvc o r qualitativca methods. Kin /199?), s respected autllority nn
castL slud y drvsign, has shcssed thr irrelevance nf the quantitativt*/
q t l n l i t a t i v e distinction for case studic.; He point< out that.
A fundamental . ; t r a t e ~nf social research involve.; cvnlun'tin~'plausibtc
rwal h v p o t h r < c ~ ' .We ncrd to cxaminc and cvaluatr alternat~\~e wFa!.q tut
cxpl~ininga particular phenomenon. This applic..: rcgardEcsc ot whetlirr
thc data arc- quantitative or qualitntivr; rcgardl~ss of thtl particular
r e s r a r c h d e ~ i q n (experirnmtat, crvz\-wcfional, lono,ituc!inal nr casta
 IVI TAT I5 RESFARCI I DFSIGY'
Causal relationsh~p
Academic
ach~evemenf
on child's ~ n ~ tab~l~ty
Alternatrve explanallon*s~lect~vity ~al
1 achievement 1
Alternative explanation lamily resources
resources ach~evernent
Atternaf!ve explanatron educational values
I, Child's I On I ? c w many c a w %,I re the conclusions based? Could the pattern simpIy
Figure 1 h Ctrrrsnl ond n t r t ~ - r i ~ l t s at~xplnnnlirrtl.:
f of Ill'
r~lntin~r~lrip
sr!:ou! hflrrnn ~ i j iur7t1rfer?rrrna'lrtt.r'rv)rtv!/
study); and regardless crt the method of data collectinn (p.g, observation,
questionna irc). Out rn indsct needs to anticipate altcrnativc ways of
in terprcting findings and trr regard anv intcrpreta tion of these findings
as prori.iion~!- sutriect t o further tcst~ng.
Ihe i d r ~nf evaluatlnf; plau3ible rrvn l h y p n t h e ~can ~ ~be illu.;kratrii
using tlic crample of tht. cnrrclation b r l w ~ e nFFC crt >ch~>ol attenci~dnnd
academic acliirucrnent. Many parents a c c ~ p the t causal propos~tionthat
attendance a t f ~ e payin? private scl~nc>lstmprovcs a child's acadtsmic
pcrfnrrnanc~~ {Flhwrc 1 . f ~ ) .%-hnuls thrmcc.l\-es pmrnc~tt~ thc same nolion
h v prclrnincntly a d \ . e ~ i < i n gtheir PI<': rates and comparing them with
those of otfirr qchools or with natimnl averages. Pv irnphcat~nnthey
prnposc a cat~.;al connrcl ion: 'Send your child to our sclinnl and thcy will
pass (or K F ~grades to gain cntry intc~prt%qti*ous institutions, courses).'
The data t hpv provide artb consistent rvitli their pmpp~lsition that tht=sc
qchonls producr trcttpr rc.;i11tq
Rut t h w e data art* not cornpclliiig. There arc a t lrnst three o ~ h r ways r
of accoi~ntingfor this correlation rtrithout accrpting thc. caliral link
1 bttn.t.u.n SCIICWI kpr*and achievt*mr.nt (Figure 1.6). There is thc ~r,lt,rtrritu
ex~lanat~rm : more able student< may bc srnt to fee paving private
khe
school< In t h firs1 ~ place. Thew is t h e j l ~ n l f rrsnr~ri-r7s
y explanation: parents
who can afford to send their childrrn to fee pa, i ~ r gprivate schnnIs can
also afford other hclp (c.g. books, private tutoring, quiet studv space,
computers}. It is this hclp rather than the type of school that p r d ~ r c e the s
bcttcr pcrformanc~nF psivatc. schml sh~detlts.Finally, there is lhc family
irnllrc.~ cuplanatinn: parcnts who value ~ducaticmmoqt ate prtlpared to
send t h c ~ rchildrcn ti) fee paying private schouls and I E is this family
emphasis on ed ucntion, not t h schools ~ themsclvcs, that produces the
trcttvr academic prrfr~rrnancr,All these explanat ions are equally con- I
sistent th the obwrv.~tionthat prt va te school .;tudents do bctter than
go\.ctnment schoor ~rudcnts.Wi tl~cjut colIecHng Curther ev~dcncewc
cannot choose betwccn these explanations and therefore must remain
open niinded abnut which one ma k t ~ smost empirical sense.
Thrrr- m i ~ h also t bi>rnetlrodologrcal euplanation.; for the Fitiding that
pricatcl ~ c h o n l5hrdt.nt.; pcrfclrrn t,t~~lt.racadcrnical lv. Thew rnrthosfolo-
gicn l 1\+1rt=3might ilntltvnine A n \ a r ~ w m e n tt h ~ ta causal conncrlt~nn
the rc.;ultq due to q~r~hlirm.~ble
~ ~ 1 5 1 t\ .l r ~ w , ~ y <of rncnsurin): arhlcve-
merit? From what mngc aird numhcr uf schools were the data obtained?
be a functirm of chance"ese arc*all possible altcrnati\~eevplnnntions
for thc finding that prii-ate schwl students perfrjrm L~tfer.
Good re5carcl-r dwign will anticipate cornpetin# explana ticlnq Ivfnre
collecting data so that relevant in forma tion for evaluating the rclalive
lvt7~wn
merit4 uZ thew cumprting explanntions is obtained. In this example of
schot,lc and academic nchievement, thinking about alternative plau~ible
h p c l t h w e s beforehand would I ~ a dus to find out about thta parcnts'
financial resource.;, the study rr.;r>urces available in the home, the
parent<' and child's attitude? about ducati ion and the child's acndcmic
abilities b ~ f o r cnt~ring
e the schnnl.
Thr fidlirrit of nftirtttr~tv!lttb ior?crq~tt+rrf Althnuqh evid~nccma\* be con-
sistcnt rzmlthan i n i t ~ proposihr~n
~l ~t might he rqunlly consktcnt r\.ith a
ranginof alternative yrnpositionq. Too oftcn p~uplt' in not even think of
thc allernrttive hhvpcltlicses and simply conclude that since the eviclcncc is
con~iqtvnt with tlirir ihcon. thcn the thcorv 1;. tnw. Thik form of
r9lsnning commit5 t l ~ cI(yical fi7!brcl, of iifirrrrrriy t\rra c-(vrs~t71irrrt.This form
nf rc;lqoning has thr follrnving lo~icalsbucturr:
a If A i.; true thcn R should follow.
a Wc i3bwn.e R.
'T'l~rrrioreA i< truc
 !\'I1A.S I5 RESEARCH VTS1CY3

If wr apply this logic to ~ I I Pl y e of school and achievement prt~pcwitinn,
tlic logicLil struchlrc of the scl~ool tvpc and achieveme11t arEurncnt
becnrnt..; clearer.
Ini ti,! prnpos~tion:
a Privntc schools pruducp better students thnn do government schools.
ilclthing !+ill chnnge the mind n f thc true bclic\'cr. Fxchangp ~llcnrv
(Homans, 1Yh1; f3lnii. 1964) ir hrgt-lv non-falsifinblr. I t assurntas t h d i wc
alwavs r n a x i r n i ~o~u r gains a n d a\vid costs. RII! MY can see a l r n r ~ s t
n n y t h i n ~;r* .I gain. Great sacrifict-s to care for a disabled r~latircecan bc
interpreted as a p i n (satisfactinn of helping) rathrr ihan a lvss (incomtl,
time for self rtc.). We need to fr;lmr our prc~~cwitrrrns and define (rur
terms in s ~ r c hs way that thcy art* capable of bcint; diqprnven.

The trst:
THEPRDVTGlONAL NATURE C)F SUPI'ORT FOR TI 1 E O l t l I.\

ICA tjvtt R If privntc wlrnols p r o d i ~ c ebcttcr students ( A ) tl~critheir
~titdcntqshould get bcttcr final mark.; than thoqc from gnviv-nrnen t
funded schools (B).
F 1s: Irrrrl Private school qtudcnts dc, achic3ur better final marks than
gcwrrnment school st11dr11ts(observe. H).
A is t r l t ~ 'l'hc.rcforc private, schools do p r o d ~ i c cbcttcr
'I'lrr'vt~fi~ur
str~dcnts(A is truc).
Rut I hat.? ~lwactyar~lrctl,bhc b e t t ~ rpt~r(cjrrnnnceof privi1fr ~ c l i o a l
t htr effect of othcr f.ictors. The problcrn hcrc is
shldcnts might aIso r ~ f l c ~
that ,iny number of cxplanntions may br ccrrrrct and tlir cvidcncr does
not hclp rule out many nf these. For thr sncinl scientist thiq Itwrl of
indctcr'minacy is quite ~tn.;ntisf~~ctory. Tn rCfcact we are only in a position
tn say:
* If A lor C, or D, or F, or F, o r . . .I then R.
Cl'tcrbsenle B.
v Theriifcrrc A [ox C, or 13, or E, o r F, or . . .] is tnre.
Although explanation (A) i q still in the running because it i s conststcnt
with thc nbsenlations, we cannot say that i t is the moqt plausible
explanation. We need to t r ~ to u r prupositinn more thororrghly bv
eva lunting the worth of the ~ l t c r n a t i v epropo<itinn+.
FALSII:IC'ATION:
LOOKING FOIZ F V l n E N C E TO IIISI3RC)VE THE TllFORY
As wcll as c v a l u a r i n ~ and ?liminatin~ nltrrnative explanations we
shot1 lcl rigorc~uslveva luatc o u r vwn theol-iw. Rather %ban asking 'Wh'lt
evrdcncc M ' I I L I I ~ coni;titlitt' Supp~Jrtfor t h tli~~r!'?',
~ ask 'What evidence
W ~ I I L cr>n\.ince
~ m e that thc t h c n y ic ; t v m r ~ I~t iq npt difhcitlt tc, hnd
evidrncv consistent with a tlieoy. It ir; nli~cl?tougher for a Ihrnry to
suruivc thc f c ~ of t pcnplr trving tci disprove it.
Unforl u n n t ~ l ysome thctirlcs nre closcd svstrmh in which a n y cvidence
call bc intwprcted as support fur the fheorv. Such theories a r c <aid to be
non-lnlsifinhle. Many religions or belief ~vstcrnsgan brcarntl c1ust.d
wst~rn.;tv her+ all e\birlc*nci*can be accnrnrncid,~ tcd bv the tlimrl- and
E v ~ nwhtrr thr. theory is cclrrt>hc+mtcdand ha< sur\,irtd attcrnpt.; ~ L I
d i s p r c r ~it,~ thc thcnry remains prt~visiona!:
fnlsific.lhnni.;rn stresses the arnbi1;uity of conhrmatrt~l~ , . . i.orrnhoration ~ i v t n . :
only Ihr c~nitortthat the tllerwy has hrrn testcd allrl <iir\ ived the test, t h a t
even altrr t l r t * mo4 irnprcq<~vv ct>rrnbt~ratinn< o f prr~dictrunsit ha'; rlnlv
7 hic . . . 1s t,>r trnm thr <tatus of
nchicvcr! !lrt, ctattz< crf 'not vrt ~Iisct~nhrrncd'.
'b~inq trrrr, (C~ujknnd Csm(-trr-ll, I L ~ ~?OF~ l -
These always may be a n unthcru~lit-ofcxplanatinn, Wi* cannot anticipatr
The
or evaluatc r1\?cry p n s ~ i b lc~ j > l . ~ n a t i ~ > n rnorr, dltiwiative explcina-
ticrns that havr b ~ e nelirninatcd and the more wc h t i v t ~t r i d to disprin~t*
our thwrv, thr. more confidence wtl will have in it, but rue should avr7ic-I
thinking that i t is /7roi7cli.
H o w t ~r r ivt3can dlsprclrlr ;l throrv. The logic of this ic:
If t h ~ n r vA is true then f l sliould filllow
R docs r ~ r ~follow.
l
T h e r ~ f o r cA is not true.
So long ar; D is A valid test of A tlit- ahsence of B should make tls rcicct cw
rel'isc ihc Ilicnry In rcality, wt) w i ~ u l dnot reject <I tli~.nr!! simply becau.;tl
a single facl o r observation cFoc.5 nett fit. Befort. rt=jr*ctinga plausihl(b
t h e n 6 wc w n i ~ l d requirc rnultiplc disconfirrnnt~~~n.; ~iqingcliff~miit
mcasurr.;, dittcrcnt samples and d~ffcrentmethod.: ot data cc,licction and
an~lys~\.
In surnrn,lry, \vc ~ l i o u l dadopt a ~cepticalapprcrach to euplanat~t~n.;.
We should nnticipabe rival intrrprctatir~nsand c o l l t ~ tdata to ennblt thr
winnowing t r t t t o f the weakcr 11xplnnntion< and Lhr ~drr~tificatinii of
w hicli altcrna t ivc theories nla kc* 11lnstempirical st*nsc3.Wc also nivd to
ask what data wro~lld challenge tfic explanation and cdlect data tc,
evalrlati* thra thrr>rv from this nirlrr d ~ m n n d i n f prr.;pcctiif~.
i
 W H A I' IS RESEARCIH DESIGN?

Summary

This chapter has outlined the purpose of rcscarch design in both dcscrip-
live and explanatory rcscarch. In explanatorv research the purpose is
to develop and evaluate causal theories. The probabilistic nature of
causation in socia1 scicnccs, as opposed to dctcrrninistic causation, was TOOLS FOR RESEARCH DESIGN
J i scussrd.
Rcsearcli design is not related tr) any particular method of collecting
data or any particular tvpe of data. Any rcsearch design ran, in principle,
U ~ anyP type of data collection method and can use either quantitative ur
quaIitativc data. Research design refers to the strucflarr of an enquiry: it is
a logical matter- rather tl-rdn a logistical one. Tn achicvc a rcasonabl~research deqign we nerd to attend to a number
It has been argued that She central role of research design is to uf matters before wc arrivc a t Lhc final design. The first section of this
minimize t11c chancc nf drawing incorrect c a u ~ a linferences from data. chapter outlines these preliminary steps that prccede design. IL then
Design is a logical task undertaken to ensure that the ~vfdencccollected expands on the idea of alternative rival hypotheses that was introduced
enables us to answer questions or to test theories as unambiguously as in Chapter 1. Thc second section introduces a number nf concepts that
po~siblr.When designing re.;earch it is essential that we identify the type are f~~ntlamentalto designing good rcscarch - internal validity, external
of evidence required to answcr the research qucslion in a co~lvincing validity and measurement error.
way. This means that we must not simply collect evidence that is con-
ststunt with a tmrticuldr thcory or expianaticln. Research n c d s to be
qtructured in such a way that the evidence also bears on alternative rival Before design
explanations a n d enables us to identify which of the competing explana-
tions is must compelling cmpiricallv. It also means that we must not In the same way that an architect needs to know the purpose of tllp
simply look for cvtdence that supports our favourite theory: we should building before designing it (is it an office building, a factory clr a home?)
also look for e~ridenccthat has the potential to disprove our preferred social researchers must be clear about their research question before
explanations. develop~nga research design.

Fncr~sil?~
at7d clurrfih~~q
the rt"~earchquesti071
The first questinn to ask is, 'Wl~ntquestion am 1 trying to answer?'
Specifying a question is more than identifying a topic. It's not enough to
say, 'I'm interested in gctting some answers about family breakdown.'
What answers to what questions? Do you want to know the extent of
family breakdown? Who is most vulnerable to family brcrlkdown?
Changing rakes of breakdown? Over what period? Where? Or are you
really looking at the causes of brcakduwn? The effects of family
breakdown? AIF tht cffectq or just particular ones?

'1'0 1iarrow thp t o c ~ t sof descriptive rcscarch W P need to specify the scope
of what is to bc described. The following guidelines, using farnilv
breakdown as an example, help narrow down a descriptive ~esearch
topic into a researchable question.

1 What is the .scopr5nf th(5 rorr roncryts? What is to be inclucled in the

concept f n r ~ ~ i l r ,llrcnkdo~c~n?Do wc mean divorce? What about
 .t.pxabion1 d4r~awc r e t v r r i l ~on!! ~ to ttic b r c , ~ k t I o ~ rIn
n rn;lr~tal I i l e . ~z, l n ~ r n p l ~ ~ yICVP!,
other infurmaticm about t ~ a ~ *pear ~ n t ~int1.1
~ t t~on
h t pw~ e' mran thc tolnF brt-nLdnwn nr \ I I I I F E ~ FOI.,~
r c ~ l , ~ t ~ t , n ~ I>o ttaTd- r,~tc,chance+ t r r Inws) that \%a<rel~v.mtt v thc h\*pothews.
t ~ o n ~ l l i p \Yha!
s? ,iboult relatinnships bcttrivn p r e n t 5 and chifdrm o r
brtwcen children? Until wc specify what we mean by our corc con-
crpts rt is RurnR to he imynssiblc to t r r ~ i nthc de~cription,
F L J ~ V G I EXIV
N C ;A N A TORY RFSFAIZCH QUESTIONS

2 What is the tttrrr* f r i r n ~for ~ the descript~nn?Is our 1ntt1re5t in c l i a n ~ c In frnrnin~cxplanntory qnccti(>ns tve need ttr furthcr spcc~fyo u r f~jrus.
nvcr timc or jukt about ro!rh,trrpc~m?flevrl< of famiti7 breakdown? If ~t Explanatory rcrt~.~rch explvri.5 cause< and /or constc-prnces of a php-
3s ~ h n u chanp*,
t over what pcrind? nilmt*iirrli, s o t l i ~ ' researcli q u r s t ~ o nnittst hp clear n b o l ~ tthe stvlc c-rt
3 What is the 8,-cymphiatl location for the descripticm? Is the intercst in explan,ltory rcsrarch and icltvltify which causes nr consrquences ~t will
I
Family brcakdnwn in a particular community, in d i f f t w n t region5 or in\-{=stigate.
tllr suhole n a t ~ o n ? I< i t rrm-tparativc, Zooking at breakdown in Pt4tlrc ilutlln~ncsomr difftlrrnt t y p ~ +o f c\~lan.ltirrv research i t i.;
Jifftarent hpcs ot co~lntrirs(t>.g.l u ~ h l v1ndu5trialirrJrcrws rnpi~Ftl* ~ I C C ~ to
I I intrrwdclunb
~ wrne tprni.;.
industrraltzin~lFrrsusirnp(>vrrished cnunkries)?
4 I low 'yenenrl 1s thc de.;criptirJn to bc? 1jo you want tn be ablc to
Jcscribe pattrrnq for aprr.iFic subgroups (e.g. arncmg those who i t 1 This i.: the vanahltb that is trc,itr.d ns the r*ffi,r.t In
married as tecnngers, amcmg those who arc in i1v,ht-!o relationqhipc, thc* causal rnt~dcl:it is dt-pendent o n tfrc inflticncc r l l somv othcr
\tu-tjnd nrarr1aKes ctc.)? t ,~ctor.The {lcprndent \.nr~a blc is a1c.c~referred t13 a4 the clutcornc
5 What n.qpect o l I he topic a r c vou inter~stcdin? Is thc inicrcst in ra lcs of \rnriablp and ~ncausal d f ~ g m r n ~t s is mnvontionall\>d c s i p a t e d ns the
I~rc,ikdnwn?'Tlrc cxperirncr? Laws? Attitiidcs and hc*lleFs? Y variable.
6 I l o ~ \~dr.;trnt.l
. i\ vcmr intcrll5t' 15 your inlt,rta.;t in f,~rn~Ev 1~roakdc)it'ti or Irr~!cb~rrrrrl~,tll ;nrttdl!r ni\ i* Ihc vCiri;lt~tc th;tt 75 tht, yrta<urncdc ' r 7 / h c L . i t
irr fnrn~EybrcnLclnwn as a rr>flrrhonnf sljmethinq morr abstract (c.g. l < a l w callid tt~clpredictor vdriablc, t l ~ c \ p ~ n m c n t a l\ ~ ~ ~ nor a bthel ~
'it~.ly?l frilgrn~ntat~~ n~, C I ~ .~(lnflrct,
SL I~ r n , role (11 the
~ n d ~ i i d u a l ~ s{hc c,xplar~at~x-y ~ ~ n r r n hand
l c I, { l p s i p ~ ~ l tin
i d cauwl ~ i i a g r a m sas thr> 1
stntc in the private lives of citizens)? v n r i ~ h l e(as In tsducntinn (X') 4 incnrnc level (Y)).
7 What is the ~rnittifa~mlws~s? Thc unit of .~nalysisis Iht~'thing' about I~rh~t-rwzi~r~ r v r k i l d r ~ I w r c vnriablcs come between Il~cindependrnt
which we collcct ~nformation and from which we drnrv cnnclusion~. i.,iriahle anti tlrr dependent variable in a causal cliain. Tliey arc. the
Often this is a person (e.g divorced pcrqcm) bwt ~t may h~ ' t h ~ n ~ . ; ' nln7n.;by n.hic11 carise S prt~ducese f f ~ c l1'. lntrrtominr:\.ariablcs are
such as organi7ations (divorcc courts), a family ax a whole, cvcnts rcbpwsentcd in causal d l a ~ r a r n sby the sl~mhoI%, as illustratcd In
( c . ~d.i v o ~ c ~ s periods
), (divorce in diffcrcnt years), places (cnm- Figure 2.1.
mt~nities,countries). F~fmr~mrr.; rwrr(1llEcs Two vnriables can bc. corrrlatr*d withor1 t bclng
Thtl questic~nsrvc can a n w e e r will dcpcnd on the unit nf analysis. c s u s a l l ~rdtited. The corrc!;ltion r n d p Irt. d u e to tht- two F~ctr~rx bcin):
Wc co~zldcompare divorcrd individuak with nnn-rlri~orced ~nrli- r~utcomesof a third variahlc (see C h a p t ~ rI). This third variable i s
vidlir7t~(indiviii~~al as the unit of analysis). We could studv a serirls of called an exttancous variable and is also symbolirud as Z in C ~ I ~ S ~ I I
~1ivorct.sand ctxarnine whai the process nf b e c o m i n ~divnrced was dingrams, and the form o f this reIatinnship is illustratcd in Figi~rc.
(rvcnt as unit of analysis). Wc might itqr vear as the unit of anajvsiq 2.2.
and track changt.s in d i t - o r c ~rates sincr*. Iq45. Uking cnuntric5 urc
might examine the different divorce rates in differ~rlltvpcs nf mun- , ~ T I F I ~ W or r f f r c f ~ T l l i ~is the Irast focused typt*r-rf cxy7an;ltnrv
Smrr I r r ~ r for
tricl.: w i t h rl vivw to comparing the p a t t r r n s in d i f h w n t typts of r~si*rlrcIi.I t ilivoIv~*< idcnti Fy ing the corc pht~nomcntln(P R. c l i a n ~ cEn~ ~
colrntrics. Allcrnatively familics might br tlic unit5 of analvsis and wc divorce rate dncc World War II) and then searcliinl: for caizhc.; or
may w a n t to Inok a1 the char.~ctrristrcqo f d i v n r c ~ nFantiIleq ~ ( e . ~~. 1 7 1 ~ . constaquc.ncw r l f thiq. Searcliin~for catl.;r.: would invol~.ridentif~jing
fC~milv incomt>, farnil!. ~ p c ,ndhllri* (71 sclatir,n\hrp in tarnill) p c l s ~ i l ~causal
l ~ ~ f,ictors ( c . ~ihanging \'alue5, rlc*clinc~in rcl~gron,
compmed wbtli those of nun-di ~ ~ o r c i nfami g lies. chanpnji population mix, crrsnnmic chang~..;, I c y 1 rt%forms,changr.; In
Thinking bcvclnd individuals a s UII~ t sof anal y s i ~broadens thc welfar~.suppnrt for lonc parc=nts). Wc would then drsign research tn
ranKc nf resuarch question.; we ask ~ n broaden.. d the rangc atld ~vnl~l.itc> which ol tFie~ecau\is< hc*lpq e ~ p l a i nchangrs in divorce mtc.;.
crrllrccs OF d ~ nvailablr. t ~ l'or r.rarnpl~*,ti YeaF wtatrnthe untt of nil<form of r ~ ~ r a r cqhu e ~ t i o ni k ~llustratcdIn F i q i r r 7.7
analvsis we ~%'ould obtain st.ltistlcs f~clmthe rrr[e\.ant nntirrtial collt*r.- rVlrrnafivt.lv normight focn~, nn the rcmqilqucncw rn tlirr than cauwq
tann agencicq ri=l:arcIing d i ~ u r c cfor each yc~lr.We t z r t ) ~Id~ also col lcct of cli,in\:cs in dii.nrcc sate (Fl~ui'c2.4).
 f ducatjon Occupat~on * Income
1eve14~) I Changes In l @ v ~ofl
welfare orovisron ( x ) I

I Rise ot I
Workforce
particlpabon

hgllre 2 Jf1 ~ ~ l f ~ 1 7 1 1 r ~I ~~ tI tI 5di/t.

~ I Economic
independence

ChalFengo of
IXP
Increase !n
conll~cl~n divorce rate
marrlaqe
Fi+.urc 2 <,*lr, lr~rr=,, 6~ I nlr<r..
Rrlinio~~s

Changes In d~vorcerate
slnce WWll ( X ) - ?(YJ Rejection of
wrongness

absolutes

E x I I i A mnrc fncir~cdresearch q~tcstion

will 5pr.cify a povficr4lnr cntawl proposition lo cvaluatc (Figurc 2.5). [t
might prop(Iw an impact of particular factor o r examine n spccific 4 W ~ a lpn.;sihlr. mechanism.: connect the prcsumcd causes to IIIP
con~equc*nt~r. Fnr example, we might propow that changes in grwcrn- presumcd r'fCr*ctq (what arc* thr i n t ~ r v e n i nv ~a r i ~ b l ~ s ) ?
ment benefit.: t r p lone parvnts have led to an incrcase in divorcr ratp<
since \Yorld \Tar IT.
Another I V A ~ OF framing rc~cv~wh qtjestions is to formulate d i f f ~ r r n !
Milri, rolliplr,r i-nirsrll mo~it~l<5ttcIi propositions ,ire qimpIistic in that thcy ways of undprstancling a phenomenon and then crrrnprp which of thew
do no1 .sl>call out the rncch;rni~mshy w h ~ c htlit* two factors m i ~ l i tbc t t l r r facb. For examplrl, wc might cnmparr
c n m p e t i n ~apprrrnchcs b e ~ f~ts
related. Wt) might d e ~ p l n pnmrc cornpIclr nindrk that spell out qclrnt. of threc diCtcrtwt w.iys of ~indcrsl,inding change< i r l r j i v n r c ~ rat^^: nn
t I ~ ernech,lrli~rnsin thr causal chain. Th15 t u l l t ~rntldcl then hcurn~.; rconomlc appnmcl~,A social valu~asapproach a n d a lr~i<latir,e appro;lcli.
thy facuq t ~ the
t research nnrl prnl life5 the frarnc\v{>rk ~ \ ' i t h ~wnh ~ c hthe The econrrnit~.,~ppn>ach might ;lrprc that chanecq in ~ltvnrccrate.; .;Irhrn
r e w ~ r ~rltk\lcn
lt 2t.111 lv frarnryl (Fikwre 2.6). from econrvnlr t;lslor< such ;I.; tht- I r a \ t.1 of afiltrenct., '3t.cr.5~ of rromi,n t r l
12 ht- clL>r~tvinq a r ~ s c a r c hrllrcqticln i t i 3 h d y f ~ 1tt)
1 draw- diagrams like y o r k and Icl L~I. of welfare . ; ~ ~ p p rfor t lone parent5 T l ~ cwcinl VJ 1111%~.
thaw in F l s ~ i r c - 1.1-2.5 I t I < ,i/<o hrlyfull to four key qurstions: cxplrrnalion rnifilit .;et-k tn expl,>~nthta illcreased dtvorcr, rate in tcrrn* r b i
incrcasinfi Icvrl.; nf inctivrclu,ili~rn a n d greater qo~'ia1 acccptancr Of
1 Wh;tl atn T trving to cuyl,~iii(I.c. what is Ihv dcrendent vari;rblr)7 Jivorcc. A I c y i s l a trvc approach rn~gl>t focus on thr iinpnc! of I c g i ~ l ~ ~ t i v r
2 nrh,7t arc the pnssitrlr caust..: trt-hat arr t l ~ cFnclrp~ndentvari~tllt..;)? changes srarli ,IS no-fault rli\*orcr fir ne141 rnlc.: rr*l:ariling propcrtv
7 IVhich o111cr.cwill 1 ~,\plon.' divlrion.
 rrl trilrlrtraq ot thi, ucmplc - thrir pcar*oiiality, tl-re~r\,,I l ~ r c ~ belirts a n ~ l
,ind
T17c a b o ~ , tr.t~rnples
, tocu.; on pnrticu lar causcs nnd consrqu t-nccq nnd thcir ~ntcrpt~r~cliinl sh~lls.I t c o u l ~ Iocir5
i on ccclrlrvrrrt. t.ictoi-s arid intcrprrt
marital ha~pinr*q.; in terms c d tlrr ~-ostsand I-cntlSlt< tcl each pnrtntqr.
att~.nlpt t c l provi~fconlv a ~lrrrtttrlcvplanatirln c j t c l ~ ~ ~ n in j i rds~ v n r c rak't.
t~
r\lternati\tllv, thc rrplanation ctruld adopt a !if(+fL\;r?ir' !l~'r5{lcrfl~'r* th~t
Tlic.;~ ~ - \ p l ~ n " t ' o n a~r e ~ . . 1 1 1 t ~ l !~ot?!i?fI~~tli
cxpIanations - partial
c ~n~aritalharpiness accorrling to $\-herethe c(~t~pIc-
i n t c ~ r cdt ~~ i i r r m c in
eupIanattc\n< of a class of CJSCS rathcr than a 'full' explanation 01 a
i q in the lifc coirrw (e.g.nelvlv r n a r s ~ ~beiore d, clrilrlrcn, young rhililri~n,
y a r t ~ c t ~ l ,caw.
lr The\- in\-nh.r nn examinatinn of r~lativel?;ICM. c n ~ i ~ r l l
adolescent chilifrt%n,ernptv nest, I ~ t c rTifr). 11trnllnist approach rneglit
factor.. and a larger n u m b r r nf cnses. In contrast, ~ i i i u ~ ~ ~ ncuplanatiuns
yd~ic
focus 1711 p ~ t i c u I a cases
r and develop as cnmplrtr a n explanation of rnch
trl, to ewplnir~ ~ n ~ ~ r iliappiiic~>.;
tal in tcrms of gctldt-r rnlcs, division t r f
casc as pc~rqihle.They invnlvc cxnrnining as many factors a s possihlc that
labour and pr~wi-rwithin the rcla t i n ~ ~ s h i A p . d~rnc~grapher migli t focu.:
on tlie birth cclhnrts ( r . ~ .
dcprcqsion marriages, poqtwilr marriages, 1Qullq
c o n t r i t ~ ~ r~l rI~thc
I casc i n c l u r l ~ nunique
~ f~ctnrc.
:\ nt7mothrtic explanatirrn o i divorce might fc>cirson the impcwtancc n l
rnarriagr~r4c.j Other resrarclit~rsmight wek tcl iaupl.i~n marital happi-
a 5i11ylcitlntributing factor s~rch,75 the intcr~cnt~rational tran5nit\<ion of ness in ~tru~-tirr;l! tvrnrs ~ u c l ,T<
i thc c~~itple's IC,YL'F e l f c c ~ 1 ; 1 1d i s a d ~ i l n t ~ ~ ~ .
Social n~tl\.clrktheorists might concentrate m thta futent to ~ \ ~ h i cah
divorce, Arc couplcs mnrc Iikrlv to di\'orcc I( onc oi t h e ~ rpnrc*nt<liave
dil-nrcrd7 Tlit- study might in\vcll\rcc o m p a r i n ~pcnrrple who had divnrcud couple is intcgmtucl ~ n t othe widr*r fnmilv and cornni~~nlt? a n d is n b l ~i t 3
rcccivc .;tippnrl from these nrtw nrL.;.
with tliow wIi(l h a d not 1 0 stira if khe divorcvrs wcrc more likrly to have a
faniilv histcrrv of divorcc. III contrast, an idinglnpliic a p p r o a c h might Tliis list i b not rxhdl~stivc.l'hc point is t11~itdiffcrcnt approaches ( c . ~ .
focr~ron ;I pnrticular cJivt>rcrd couple and hullil .I t ~ ~explanation
lt o f w11y
p ~ y c I x ~ l o g i ~life
; ~ IC(NI~SP,
, t . w l i n n ~ ~structural,
, lrrnillist) will have a
T l i ~rkplar~ahnnwcruld ce~n<idcrthe family histvrv
f f r ~ sc o ~ t p l L~I\-urccit.
t~
particuldr 'angle' which alert.; 21.i to diffcrcnt w x . : r,f 11lt3kjn~ a t an i w l r .
When thinkrnq ahout a yrcrbl~%n~, a 4 yourself q~rc\titrn.;quch as 'Flon'
of this ccrttple a n d cxarnlnt* t h i s alonq wrth nianv other cnntrih~tirl<
might a h.in~n~\f,Iirnunl for t h ~ ~ ' 'I-I{v\. ', r n i ~ h t,I 1 r t a i r r l ian p s v c ! ~ n l r ~ ~ ~ ~ t
factc~r+ 1 liv ~dlngrapliicc.\plLlnatinn ~ v c r r ~ l ipl r n ~ ~ ~u3d ervitl~ ,I g t ~ r d
uncIc.r.;l.~n~li~i~ of thc c.,lcr* tvli~lct11c nom~lth~tic-i~xplanatiimwrwld prtr- cxylain thiq", 'l-luw rvnuld a n rxcli,~ligetheorist ;Iccrrunl inr his?', 'M hat
vide an ilndcrstanding o f thc influence of a f n u ~ n r . might a conl1lc.t t l ~ t ~ n rsay?'~st

on the topic can I)c a rich source oT

Oilfcr restnt.r.irl'r.: Prcvious rt~sr-<~rcti
cornpct~nc v\planations. Rrad tFrr literature in jnzlmals and starch
In Chsptcr 1 F stressed ttic rmvc*rt;lnccof identifying plauqible rival rvays electronic ~latnb.wcs.Kevie~sarticle..; a n d intrnclt;ctnry c~\rervit.~j.s
can tit.
of accounfilig for the phtzntrmt-non being studied. ObviorrsIy i t maker; c~fremelvlicrl7tirl.
sensc tn anticipate thesc altcrnativcs before cnrrving nut the rc.;carch. Rut
how d o wtl identify tliesc nInrrnative explanatii~nslThere a r e two marn r t ,- i s , , J i t 'Insiders' with prac-
types of rival hypcrthrses a n d these suggest w;lys of anticipating altcrna- ticaI knowltdgc of a field can hr invalurlblc. Tn n ~ t i ~ don y divorce,
tive cuplan.itit>ns. marriage co~~n.;rllors, married c t ~ ~ p l c divorcPd
q, p c ~ ~ p and
l c advocscy
g r o u p can provide valuablc i n s ~ ~ h tLs ~ t c r a t u rcucli
~ as novclt; and
plays can a l w prt~videkeen idc,~.; rhat r a n be Z C S ~ C.;yqternaticalFy
~ Itx.g.
Tolstoy's nnvd Alr~inKRrr~rzrrnp r o ~ ~ i d eones way c.rf intcrprcting n ~ a r i t ~ ~ l
There 1 5 nrr magical wav of prcxiucing a s r t of alternative substaniivc or un happincqs).
thct~rcli r ~ larplanations.
l 111 t hc end the re<~arclicr murt f o r r n ~ ~ lt~ht~c m .
Thc niclria familiar thc rtqcbarciicv. is with thc particular substantivr l o p i c h not igiir)ri*yoiir own pkpt~ri-
arrd i i r t ~ i r l i ~ ) ~DO
I ) i r l l ~ t7Tl)r'tt t h ~ f ~ . lIrrrrrt-!IPS,
b,
and w~fll~ C ~ C ' I J Jsciencr rnoJt~I5and thcoricr, thc nlore likely thcat. .lrtl to enct, VOLIr 11\1~11intutt~onsand Iiun~lic*~. In thc cntl a l l t~cplanations<tart
antlstp,?tr9diltermt way.; o t interpretin< a givtan ~t ot result\. The lt-ith hrBnc!%rwthat q ~ r i front r ~ ~~ n d i r i d u ~w l sh o liar11~~lc.al;
and n17starrr
folI~v5In!: prt)\*rc!t. .;ousrcs o i ~ l t c m a t i c\ c r ~ l , ~ tticin<.
~,l
t hlncs atr)utl~lt h t w V3c thrcc ~n.ighh, rxpcricnct.* and r)bscn.at~cjn.
and t t ~ nthem \\..-.tcmaticall?.
71zr ~ S ~ t , o l r l r r - nEttrrrrtrircb
l Rrrwrl approaches In a rli5clpline can yrta.;mt
diffcrcnl ways rrt viewinr: L I I ~ question. Supptv;cb, lor examy lc, t h a ~we T1ic.r~is nn r i ~ h way
t of dc\,cfop~iig ideas. Do nclt linrit y~111rsclfto fornml
wantrd t r l tlnderstancf wliv sornc propIc seem have happicr marriagrs social scipnct- rrscarch or to rc*.icarch t - m the very ~p~ncific topic you arc
than ntht.1. lx=oplcdo. A n c\plnn,~ ticln coulcl conc.tln trd te on the /lrvr.;crr~nl working nn tl.;r. c f ~ v c r ~s ne u r r w ot ideas. Brainstrlrrn w ~ t h;1 flotlp tltld
24 WHAT IS RESEAKCH DESIGN?
dcbate the topic. Think latcrallv: if your topic is marriage breakdown,
look b~yclndthe marriage and d ~ v a r litcraturt..
c~
Tf findings are likely to bc due to poor measurement (see below) then any
theoretical interpretation of these results will be unconvincing.
Throughout this book 1 will identify many techilical and methodological
factors that can undermine the cnnclusiomq we draw from our research.
Good rtsca~chdesign will minimize the threat from these sources.
1 will iiot a t this point go into these methodological issues in detail.
I-lowever, it will be helpful to highlight the types of methodological
rivals that will bc cxnmincd. Thcw arc outlined in Goldenberg (1992).
Derwurrd rlrlz~ar/eristir.sof the sihration,
Tronsirnf prrstvlnl c!iarnct~~i~frcs such as the respondent's mood,
attention span, health etc.
Si~rmtiunn!ffiifctarssuch as anonymity, people present when data were
being collected, gender, dg-e ~ n cd[as5 of i nrestipitor and respnndcnt.
S n n ~ p l i q ynf 1i~nl5 Arc lhc conccpts w ~ l measured?
l dictionaries ( e . ~ .R dictionary of sociology), encyclopaedia5 ( e . ~an
N n t ~ r e[iJ IIzr sflntple Can we geiicralize froin the sample?
I , ~ c ko f rlnrity r$ flw inqfrurnmt Are the questions clear and unam-
biguv~is?
Forrnnf of dotn uoll~ction Are the results an artifact d the data
collection method? Would different patterns be found if a different
method was used (e.g. cibservabon rather than questionnaire)?
Varratiun in the ndrnrnistwatiorr of the instrum~ti!in studies tracking
change over time.
Prnc~ssir~~/n~zalysis crrrws.
13pcmfion~lizntii1n
Most social science research invol~~es making observations that we
presume tap concepts. I f w7e were cunducting a study on the effect of
marital breakdown 011 thc wcllbeing of children wc would need first to
~ w i - kout what is mcant by marriage breakdc>wn, wellbeing and children.
This invoIves defining these conccp ts, which in turn requircs developing
a rrortlii~nldffinifin~land an o p r ~ a f i o ~ l dcfirritiolr
al of each concept.
Concepts arc, by their nature, nnt dircctly obsmablr. We cannot see
social class, marital happiness, int~lligence etc. To use concepts in
research we need to translate concepts into something observable -
something we can measure. This involves defining and clarifjling abstract
coiicgts and developing indicators of them. This process of clarifying
abstract concepts and transrating then1 into specific, observable measures
is called nperatii~nali7ationand involves drsccr~dir~g flw ladder of ubstmcfiun.
Befnre developing indicators of concepts we must first clarify the con-
ccpts. This involves developing both nominal a n d opcmtional defini tiuns
of the concept.
I
Noirzirrnl ~!c,iirzitio~zs Concepts d o not have a fixed or correct meaning.
Marriagc Freakdmvn could be defined in terms of the law (such as
whether the decree nisi has been granted), thc quality of thc relationship
I
I or practical arrangements (living apart). Similarly, wc need to define
children. Is a child defined by a blood relationship, a legal relationship
(incIudes adopted ), d social rcla tionship (& f i ~ d oparents), chrt~nological
age, level nf dependency or some other criterion? By deciding c>n the
type of definition we provide a nominnl definition: it sptcifies the meaning
of thc coi-rcept but remains abstract.
Different dcfinitions produce different findings. Consequently, defin-
rilg concepts is a crucial stage of research. It needs to bc done deliberately
and to he sy c,tcmnticall y and carch~ll y ju5tified. There are three steps in
developing and narrcwving d i w n a nominal definition.
~ < . a t review articles, d isciplinc
Firqt, obtrrin 17 vorr,p3 c!f t r ! ~ f i w i t i n ~ Louk
.
ellcyclopaedia of swial =,rieruces) and journal articles. Look for both
explicit and implicit definitions.
Second, d~ciiIrun rr defirrition. Fmm your list either select one definitirm
or. create a better definition from the common element? nf several
dcfinitions. Explain and justify your approach.
Third, drlmede She dimensions of thp concept. Many concepts have a
number of dimmsions and it is hclpful to spell these out as they can help
tc> furlher refine your definition. This can be illustrated using tl-re concept
of the child'< tvrllheing. W e can think of a nurnbtr of dimensions of
wellbeing: emotional, psychological, physical, educational, financial,
social, environmental, legal etc. 16 ure are arguing that marriage break-
I down affects a child's wellbeing, what sort of wellbeing are we talking
I about? All of these? Just one or two? Having delineated various dirnen-
sions you will need to decide which are of interest in the present study.
You map examine all aspects or limit yourself to one or two.
A concept may have s;~bdfm~'17sil~n5. Suppose MT focusrd m qnci~7E
wellbeing. This broad concept could incorporate subdimensinns such
a s the level of qafety in the neighbourhnod, the nature of the child's
rela hvnships ancl hcr expel.ience5 of social diqcrimination . T l ~ csub-
d.irnension of 'relationships with others' could be hrthcr divided intu
relationships with particular people such as mother, father, pecrs,
siblings and grandparents. Having settlcd on the particular relationships
with which we are going to deal, w e wotild ncud tn identify what raspf7rfs
I
05 the relahonships to measurc nnd decide how to measure them (see
below). In Figl~rc2.7 relationships are measured according tn the lcvel of
WHAT 15 RESEARCH DESIGN! TOOLS FOR RESEARCH DESIGN

contact, cnnflict, closeness, helping and type of shared activities. These

measures or indicdtors would then provide the core around which to
frame specific questions or to fncus an observatic~nalstudy.
In this example I have focused on one dimension at each level but I
could have been exhaustive and developed measures for each
conceivable dimension and subdimension. The decision of whether to
adopt a focused or an exhaustive approach will depend on what you are
interested in. Thc advantage of adopting a systematic approach as out-
lined in Figure 2.7 is that it helps focus and refine the research queqtion
a n d forces you to make deliberate decisions about how to measure a
concept.

Opevotionr~ldrfin~fions Having defined t h ~concepts you will develop

an nperafinnal defftzitiov~- the obsen~ationsto measure the concept. What
irtdicnftlrs o f marriage breakdown will you use? We might operationally
dcfine marriage breakdown according to the quality of the rclationship.
Using this definition wc might muasurc breakdown according ro level of
conflict, type of communication, s i p s of lack of affection and level of
cooperation or lack of cooperation.
Idow well s ~ t c hindicators tap thc cr~nccptas dcfintld will have a criticdl
bcaring on the value of the cc>nnclusions drawn from the study. If the
indicators tap something other than what we claim they do then our
cmclusinns can be easily challenged. For example, if our measures of
marital breakdown simply tap social class differences in marital style
rather than breakdown then our conclusions about marital breakdown
will bc suspect.
Once the operational definition of the concept has been developed we
come to the final stage of operationalization. This entails the precise way
in which thc indicators will be measured. This might involve developing
queqtions for a questionnaire or identifying what and how observations
will be made. Articulating the mechanics of these strategies is beyond the
scope of this bnok.

Concepts for research design

Twr) concepts, internal and external validity, arc. fundamental to

developing researu1.r d e s i p s . Ideally research designs shrjuld be hnth
intcrnally and externally valid.

Infprnal zlnIidity
We need to be confident that the research design can sustain the causal
cunclusions that we claim for it, The capacity 05 a research design to do
this reflect? its internal validity.

Imagine a rcscarch project that compares the emotional adjustment of
children from divorced families and intact families. It finds Hiat children
from divorrrd families are less lvcll adjusted than children from intact
families. Can we conclude that divorce caused emotional maladjust-
ment? Not on the basis of these rewlts. The design does not enable us to
eliminate alternative explanations. The poorer adjustment of children
with divorced parents might be due to adjustment differences that
predated parental divorce.
A different research shdy may deal with this problem by tracking
children before parents divorce and for some years afterwards. It may
find that these children do show a significant decIine in emotional
adjustment after thcir parents' divorce. Docs the research design s l ~ o w
that divorcc is producing this decline in adjustment? No. The decline in
adjustment may simply rcflcct a general decline in emotional adjustment
as children get older. The same decline may be cvident among children
from intact kamilies.
Yet another shldy might try to overcome this problem by tracking
changes in adjustment c ~ chikdren
f bufure and after thcir parents divorcc
and changes among children tram lntact families as well. If children
troni inlact families show less drteriuration in adjustment than children
from divorcing famflies, this must surely demonstrate the effect of
divorce. No We would need to bc sure that the two groups of children
were romparahle in othcr relevant respects ( e . ~age).
. The different rates
of change in adjustmerlt could be because those from divorcing families
wcrc younger on average, Maybe younger childrrn show greater
changes in adjustment over a particular period than older children. It
may be age differences rather than having divorced parents that account
for the adjustment changes of the two groups of children.
Internal validity is thc cxtent to which the structure of a rcsearch
design enables us to draw unambiguous conclusions from our results.
The way in which the study is set up (e.g. traclung changes over time,
ma king comparisons between cornparabIe groups) can eliminate
dlternative explanations for our findings. The more the structure of a
study eliminates these alternative interprets tions, the stronger the
internal validity of the s h d v . A central task of research design is to
s h c h ~ r ethe study so that the ambiguities in the abovf examples are
minimi7ed. It is impossible to climinate all ambiguities in social rcsearch
h u t we can certainly reduce them.
External validity refers to the extent to whicli results from a study can be
generdlized beyond the particular shtdy. h study may I-rave good
irlteinal validity but its value is limited i f the findings unly apply to the
peoplc in that particular investipltion. The critical question is whether
I
the results are likely to apply rntrrc widely. The mosE cnmmnn threat to
TO01,S FOli RESEARCH DESIGN 29
our capacity tr, gcncrnlizc m t m widely from a rcscarch study is tl-rc use
of unrepresentative samples. This, and other threats tcl rxtcrnat valid~ty,
will be discussed more fully in Chapters 5, 8, 11 and 14.
A further threat to the concIusions that can be drawn from any study is
measurcmcnt error. This occurs when we use flawed indicators to tap
concepts (see Chapter 1).
Indicators must meet two fundamental criteria. They must be both valid
and reliable. A valid indicator in this context means that the indicator
measures the cnncept we snv it does. For example an 1Q test is used to
measure intelligence. If it really measures intelligence the test would be
valid. If the lQ test measured son~ethingtlsc instcad, such as cducatitm
level or cultural background, thcn it would be an invalid measure of
intclligc~~cc.
Reliabilitv means that the indicatnr ~'o)rsistcnll~tcomes lip tvlt1-r thc
same measurement. For examplc, if pctlylc consistently obtain the same
[Q score on repeated IQ tcsts, then the test would he reliable. Iftheir
results fluctuate wildly depending on when they take the test, then it
would bt. unreliable.
V~liclrf!/ The earlier discu~sionof internal validity related to the validity
of the research d ~ s i p i It
. addressed the question: is t h research
~ design
delivering the cnnclusions that we claim it delivers? In addition we need
to cxamine the validity of the nzemures uscd in any piece of research.
The validity of a measure depends both on the use to which it is put
and on t h sample
~ for which it is used. For example, the validity of using
frcgucncy of arguments hetween partners to measure marital happiness
turns on what we mean by marital happiness. The validity of this
measure may vary for different cultural groups and for the same cultural
group in different histnricaI periods. Measures of children's emotional
adjustment will vary according to thcir age and thcir cultural group.
There arc three basic ways of assessing validity. Criferrt~rlvnlidity is
best suited to situations where there are well-c.;tahlished measures of a
cnncept that nctd adapting, shortening or updating. It ~nvolvcscom-
paring the way people rate on the new measure with how they rate on
yell-established rncasurcs of the concept. If ratings on the new measure
match those of an established measure we can be confident of ib validiiy.
Criterion validity has two limitations. First, it requires that the
established benchmark is valid. If the benchmark is invalid then i t is of
littlc valuc in assessing the new measure. Second, there arc no estab-
fished nleastlrrs for many social science concepts.
 hrrnt,t~tncscrrtcriun Krtwys c i n 1.~2 u.rvcl t o a>-*c\s ct~tr.rirvn <'oli~lit!. ~ J L ' ~ t ~ t t ' ~\ V\ .I l I C r l III,..--
In<tcnd (IF cnmp.lrmg n rnt.a<urt. a ~ a ~ nan c t e\i.;t ~ r 1)runchmark
l ~ mcaqurr, changr c,r just rncasurr~nmt'nt>ise'?
thc. nrw rneaqtlrt can hi1 tri~lfed.T.or eramplc, ;l nrw rncawrc ot manta1 Unrelinh~litvcan stern from manv svtrrccs. Prior question rvt,rrl~ngmnv
happinew could be tnaPIctl nn couples w h n seek rnarit.11 counseTlin~.We cause a r e ~ p o n d e n tto unrlerstand k h ~questicln drtfcrrntlv nn diitervnt
would cxpcct that t h i ~g r t ~ u puE cottplt.~~ , o u I dn n r n ~ a l l vobhin low occasions. IXffcrcnt inttrviewers can clicit ~ l i f f e r ~ nanswers t from a
scores on a I alitl mt-asurt. of marital happiness. I f thc5e couple\ ~ c t u ; l l l ~ ~ person: thc match nf agc, ~ e n d r r class , and rfhnrcitl; of an interv~crr+rr
obtain high %cormon tl>r.mar-tal happiness mensure rue ~l-rrlrldprohabl y a n d interviciver can intlucncc. rctsprmses. A q k i n ~cl~ir~tion.; about which
\ran1 to rj~,eqtion whether thc rnrasllrv w6w rc~FIy tapring tnari~al prople have nt? opinion, 1m.e inrufficicn t ~nforrnatin11 or requirt. too
happincr;s. precise an answer can lcad tn unreliable data. Thc answers to somr
i c l ~ trbnf
r i~lirlit!(cvalua trls h(jw well tht* mrasrircs tap fhc differcxnt q u r ~ t i o n *can be affpcted hv mood and bv tlie particlilar context in which
aspt*cth 01 thc. ctlnccpt a\ sve haw* drfinc-d t t . h Lest of arithnlc~ticskills tliclj nrts askcd.
thEitonly Ierted st~htractinn?kill$ would cli~arlvnot ht- a valnl rncasurr~)f Mca.iurt-s ncvd tn bc both t,alicI and rcli;il-If. . I \ E t l ~ n r ~the50 ~h two
arithmrtic skills. sirnil.arl\~n meakurc o f marital happiness [hat only concepts are related they arc not the qarnc. A mcasurr can bt- rc*lbable
nskcd about t h r frequency of arguments hetwccn partner< would pmb- withotit being v a l ~ d Thnf . is, a measure can be cnns~stt-ntlyrurnng For
ahlv lach cont~nt valrdity unlesq we had deflncd marital h a p p i n ~ ~ q cxarnpic, people cnnsistentlv undcrcstinlate thcir Icvel nl ,~lcnhoIcon-
simp34 as lthc abscnce tlf arg~irnents.Fvleasi~r~s of marital happiness wmpticln in q ~ ~ e s t ~ o n nsuwevs. a ~ r c IIEcnhol consumption rnvasures arp
cottld a l w incltrdr tlic nahlrc o f t l ~ careumcnts, Ieisurt, ;lctivities shared rt.liable but do not accuratclv tell us al-rnut thr~ t r u e Ic.\.tl n! nlcolrnl
by partners, cnmrnunicat~cm,method': cd reqr,l\-ing crmfl~ct,the quality crf ctlnsurned.
thv <c.u~~al rclation~hipetc. Mea.;lirc< rvitl never be p~rfcctlyrclial-rlc anti pcrft~rtl~f valid. ?IIc'~c ilre
C11.thn ~ l i ~ a ~ t ~ ~t tabr3r1i n 'ctmltbt~t' r l t manv snil;l l tcicmctb cnn-
~ m r . the ~ I r~'Ii,ll~ilitv
nut ,ill o r ntrth~liyconcepts ;lnJ thr g-{r.ll iq tcl ~ I ~ I X ~th~b J C and
r-rpts i t C J I I bc. diffirul~to drvclnp rncaqurr15 that havr ncr~w-lvaliditv. valid th.If !hew aspects of measurtnmilnta r c then tltr rcsu Its nf the
r-:vu.n il i r e can agnx*iin the ronrrpi ~ n rnpn<urc. d tt ti sin^ a whn!t- bnttrry s t u ~vi that u ~ e sthrm might plnitsibly 2 7 ;i!trillr~tcr!
~ to poor mra>tlrernent
vf qucstinnq, w t then face t h c prrrtrlcm o f thc ~r'!nt!-r*r ~rtlplrtnnr-co f t h r rathrr than tellinfi u s a n y t h i n ~abortt social real~ty.
vnsitru.; cornyrlncnts of tlw rneaqurc. TJilr ruarnple, s l i t ~ l Mmtxapilws of
thc lrequcncv of argument< bc as important a.: Lhe nature crl t h r arKu-
m m l r , the method of conflict rcsolut~on,the qtvlp nT cornmunicnt~onor
statumcnts about level uf subicctivt. marital satisfnctmn? Error can take different forms and the conspquences of error will vary
Corish.rrct zanlirfify rclirs on seeing how WCII thc rmlilts rvr n h t ~ i nwhen depencllng nn i t s form. These forms of crror art, rnlrdrwr, r-on4nllt a n d
~ ~ q i nthe g rncasurch f t t with theorcticai mpecfatinns. To validatr n ru rrrlnted.
men\urc of manta1 happinpm we might nntic~yatt.,nn the hasi-=of kheo?, R R I I ~ J ~ I I~'rrnr
II i~ that which har no s y t e m a h c form I t meam that in
that happiness will vary i n prcdictnhle wavs accnrJing to sbgc in thc 11fe some case5 a rncawrcrncnt for a variable might be too l o w whiIc in
cvclc. I t the result5 of a studv p r n v i ~ i ctlnhrmatir~n,
t thib coulcl rimflect the others it is too high. The mcasurcmcnt nf somennta's w e i ~ h tmight
validih. of o u r rnpasurc of marital happine~q.HonPevcr,Ihiq approach to display random error. %metimes peoplc~undcresf matr thc,i r w c i ~ h t
a ~ w s s i n g\ralidiv rrfies on the correctness nf n u r cxpectakions. I f o u r while others may overestimate it, but il: thme errors are randnm thew
thcrvy i.: not supportrd this could be for nnc of two reasnns: tlie measure will b~ thc samc number ol ovcr- and undcrestim,~tesand the s i ~ of e tlw
of marital happinew cnuld bc wrong nr thta t h e o v a p i n \ t which thcb s he thc same on avcrngr as the ~indcrwtirn~lte.:.Whcn
o ~ c r c s t i m a t ~will
rnc.isurc i< h r i n ~ btanchrnarked mav bc ivrnng. the avcrAgp (me,ln) ih calculated Fclr the ~ ' h o ql c~~ l l pit wtll b ~ ,~cc~tratt.
.
TIicrta is n o ideal way of asstsss1nK v a l i ~ l i t ~1f. a rncnsurr passi,.; ~ 1 1 bccaure tl~cut-crcstirnntfi , ~ n dthe undcrestirnatrs c,incel each other ottt.
threr, tt3stl; i t f > more Eikclv to bc. \-,hid hut \vtn cannot bv c r r t ~ i n .In thc Furthermnrc, tliesta mis-estimatet arp not cr>rrclatrd i v ~ t hany other
hnal ;~n;llvqr~ ~ t h r r.,~ltditr~f our rnc;lsurc.:
lrc, ~villnrr-rl In ! ~ t . y r rfur characterirtic t e . ~ ~. c n d r r 'l~c)
, but ,isc i r u l ~ random.
. I3rcauw random
emur doe< nnt di+tort mcanq and is ~rncorrelatrd1z71tf1other Factr>rs, i t 1s
R t n l m l l i l i f ~ t A rrl iahle rnca.;lirr 15 nnc that wfit-s the wrnc 'rending' whrn !u.;s sersotis than other fnrms of crror.
u.;ed nn rtlpca tmi occ-astons. Tor cxamplt%, r7<< t ~ r l l i rtl~r.rt-
~ ~ r r r r r c !TO ( 7 ~ - fti01 Cnn.;tnl~trrrnr occurs rvhcrc t h ~ r cis the sdmC error for cvtlry caw. For
t,lmrr,v(-, r~liat.rlt> rnr.1wrc3 of rnarit~3happine.;.: sl>nuld !*ic.lrl tlic 5anw rxamplr, i f Pveryonc ~~ndcrrcprrrtedtheir wcighl bv ;ki l ~ ~ r ~ l l m s
. . . , . -- -*-.-'
111fiermt c x c a r i ~ t w .A thrrmometcbr that rnpas- would have constant errnr. Such error is unccx-n.latcd zvith ibther ~-lisrtic-
I"='c bhp n v x t wrr~ald tcristic.;. Although purely constant rrrrbr will he rarc* thtv'c wiil he
 WHAT IS RFSFARCJ I I3ESlGN7 TCK)LS FOI3 RESEARCH DFSIGN 33

variables for which thrre will bpcally be n crwiponenl of constant et.ror Finally, thrw corc concept< that are a t the heart of guod design wcre
(c.g.ovcrstaicmcnls of frequencv id sexual intercourse and undcr5ta te- discilssed . These were the concepts of internal validity, external validity
rnrnts about the amount nf alcohol ct~nsumcd).Because such error is and measurement error. Later chapters will evaluate the various designs
constant it Jws not canccl out but has an effect on sample estimates. using time concepts.
Thus the averagc weight of the sample would be a n underestimate to thc
extent of the constant error.
Corret.lotcd error takcs place when the amount and direction of error
vary systematically according tn other characteristics of respondents. For
example, if women tend to c~vercstirnatctheir weight while men under-
estimate thcirs then this error wouId bc correlated with gender. If the
format or lang~iage in a questionnaire is ditficult then m i s t a k ~ sin
answering rl~iestinnsmay well hc correlated wit11 education. This wou Id
produce results that make it appcar tliai people with diffrrenl levels of
education hehdve or think differently wliilc in fact i t is only their
capacity to undcrstand the question that differs.
A crucial gcml of the design and administration of survcy instruments
is the minirn~zationof the various forms c~frnc.asurempnt error. Achicv-
ing this cn tails paying carcf 111 attcn lion to question wording, indicator
quality, interviewer and observcr training, and to ways of identifying
social dcs~rabilityreqponses ancl ~ t h t ' forms
r of dclibcrate misrrpresenta-
tinn bv respondents. rn many cases it is difficmlt to identify the extent to
which such errors actuclIl!! occur. However, this does nut reduce the need
to d o all that cmc can to minimim their likclihoc>d and to have built-in
checks to identify soinc sources of error. Such checks includc looking for
inconsistenc~csin answers, tising multiple questions rather than single
question? to tap concepts, identifying social desirability response S P ~ S ,
ma king intcr-interviewer chccks and careful field work sapcrvision.

Summary

This chapter ha? empIiasized the importance of clarifying rcscarcll

qutstions and concepts beforc developing a research design. Lack of
clarity regarding the research question and the central research concepts
wiIl severely compromise an); research design. Guidelines were
provided tn help focus hot11 descriptive and explana tosv research
questions and to clarify the concepts tliey cmploy.
It i q also unwise to devclop a rescarch d ~ s i b vunless alternative wav?
of underqtandit~gthc matter a t the heart of thc sc<earch qucslir~nhave
been identified. Sincr one nf ttre PtirposeS of resr,lrch design is to help
identify w~hichof a rangy of alternative explanatinns work best it is
desirable that thcse al t&;t ti1.e explanatinns be identifled before the
research design is developed. Thc design can then be structured in such
a wag that relevant d a b arc collected to cnable u s to chonse between
these alternative< G~lidelineswere providccf in asqist in identifying these
altcrna l ~ v eexplanatiuns. #
 Sector (publ~c
or prlvate)
A Q ~

CAUSATION AND THE LOGIC OF Achrevemenr

onentatton
(low or h q h )
RESEARCH DESIGN

It3nst, necd to demonstrate t h ~ tprivate sector wr lrkrrs had a higlirr

E s t a b l i ~ h ~ ncaitsal
g relation%htpq is at the heart of cuplanatory rcw-arch
d e s i p . I lowrrver, it is not a 5imple mattcr trr c5tahlish that one cvcnt
causes another (Blalock, 1904; Hage and Meeker, IYHS). The ma in rc3sson
why it xh diftic~1Etto establisl~causal relationship.; is becausc wr. cannnl
actually crlr<rnn~r, one phent~rncnnn producing changc it1 anothtr. Even
though rlnc cvcnt might always follow anoth?r rvc rlrl not know that this
is licc~tr~t' rlnp went c a n w . thc clthcr. Ca~t.;aI r ~ ~ l a t i n n s l ~muGt
fore b c x inft~rrcd rather than vbwrved. TJlp Irir ryshr* a! rrst,n.rrrr-ilt dt+*iyr~I r l
v ~is~ ~fo ~ I F T J ~ f h~ t lI q) ~~ m l i f yof iw r i flrt<t~l? ~ ; f i r ~ f k - [ ~ > .
r r { ~ l a ~ ; i ~ trt (~br i+tdr
Inferring causal relationships
In Chaptt-r 11 1 distingui.;l.rd 'trchvwn probabilistic and deterministir
concepts o f sausa tion. Prc7babil istic approaches tcr causation a r r those
that argue Illat a given factor incrcases (or d e c ~ c ~ ~ s the
e s ) probability o f a
particular c>i~tcorne.For cxamplr we may a s p w that Ithere is a causal
g time - that g~ncndcrafft-cts
relationship between gender and ~ ~ c r r k i npart
the probability OF working part timc.
Tn ortlr r to infer that a prohabili5tic causal rcla tianship .exist< between
trr-o variabrr'q, m e baqic critcr~nmust be met. F I ~ sthere ~ , mu5t h r cm
variatinn o f causal and outconip variahlcs ( c . ~ ,between genclcr and
hcing a l'i111 timc or part time workct); and second, thc asscrtlon tha l one
variabltl nffrbcts the othcr m u ~ make
t sense.
Ff hvn factnm are causall\~r r l a t r d thev muqt ni I r . ~ c ! br corrrlatctl: tF1r.v
must co-v;rsy. IfS causes 1' ihcn pcople ~ v h or l i f f c r Erom onc anr~tlit*ro n
X sl~ouldtcnd to differ frvm onc another on Y . I:or pxamyle, if w r rvrre
to a r p c that working in tht* private sector ratlici. t h a n the public qrrtnr
make5 prnplc morc achi~igt*rnmt oricnted at wurL w r drould, nl t l ~ r 'r 1 ( , t ~ t
achievement nrirntation than public sector w o r k ~ r q .I f two factors did
not co-vary - that is public and private sector workers had identical
levels of wnrk achievement - thcn we would bc harti pressed to arguc
that they arc causally related.'
Howevcr, while co-variation i<a precondition it i s not cnuugh for 11.; to
t variables are c a i ~ s ~ lrelated.
assert t h ~ thc lv Cortcla t ions can also reflcct
i p s thcrrn- ncm-rauwr n-latic*n<h~ps.12'lit~ntwci variables rlr t5\,vnts ;Ire scirrrl ,I tcit
hut not cawal ly rclated the rclaticm\hip between thc two variables is s a d
to bc spurtoll< (rcc. Chapter 1 ). I or rxnmplc, tlir corrt.lation bctrz-ccn
sector of crnploymcnt and e m p l n p m ~ tnnrienta tion m i ~ hbe t due entirr-tv
to a third variable such as agr (Figure 3.1). Youngrv. peoplc might bc
more likelv than older people t c ~work in the privatr srctor and vounErr
people rniqht a l m have a higher dchicvement orrcnt~tinnthan olclcr
people. Thtls~-two patterns will mean h a t sector and achievement
orientation arc c~?rrrlated(rnnre young people in thc* p s i ~ ~ a tstytor,
e so
therefore t h private
~ sector is asscrciated with a c h i t ~ v ~ m c orientation).
nt
Howevcr thc link between stctnr and achicvrmi*nt uricntatinn is
I
coincidental rather than causal.
Any assertinn that c ~ v a r i a t i o nrc.fIc.ctq a causal retatinnship mrrs! be
plausibye I t rntw make sense a t three levels.
Timr or&r I f two variables arc. cr~rr~lnted the causc m 11qt come hr:fi,n-fhc
c f f ~ ~Caitsal
t. rc.irclnirig has no t1mt1 for the assertion that a fuh~rervcnt
can h a w a prcscnt effect (ti=lci~Iogicalexplanatinn). Our calisnl
propnsition muct be such that rhr causal variablis t m u r s trtforc t h r
Twsurncd effect. The fime gap trctwcm cause and rff'~3ctcan be mrnnt~s
tlr may be yrvrs ( e . ~the . ~ t f t ' t tof education nn InctmP can take many
years to show i t r ~ 1 f ) .
Even though two variabler might bc causally rclatrntl i t can sornetirncr
Ilc difficult t o work nut which v , t r i n b l ~comes first ,ind thercforc t o
36 WHAT IS RLSEAKCI-I I )FSIGN7 CAUSATION AND T H E I.OCIC OF RESEARCH DESIGN

establish which variable i s thc c;ll~st.and M'IIICII i> t h ~rftect.

' For example,
docs .;cctor vf emplnymcnt ,?ffPct achievement orientation or is i t the
other way around? E v m wherc we assert that vne v ~ r ~ a b lcomes t. first
the causal relationship may be two-way. That is, sector of employment
may affect achievement orientation which in turn influences Future
decisions about the sector of ernpfoyment in which one works. Causal
relationships can be reciprocal (two-rvay) rather than one-way.

D r p ~ ~ i d c wrmrinblc
i ~ l ~ u bc. t ~ say that a corrrlation
~ we
s t ctapnhlf of ~ ~ l l o n If Figure 3.2 An indirect calrsnl relnfionship
hehvccn two ~ariablesi s because one is causing the other, we must make
sure that the dependent \ nriable (the effrct) iq cnptrblc o f being changcd. I f
it cannot bc changed then a cauqal accrmnt c ~ tht. f r ~ l tionship
a makes no a) Long causat chain
sense. For example, any causal relationship between sex and income
X--tZ--tW+V--tP--tY
coutd only be in thc. direction of sex affecting income. The opposite
prr)position (income-sex) makes nn sense.
b) Multiple indirect paths
Tl;~orctiinlplnuqihilify Thc causal assertion must make scwso. We should
b r able to tell a story of how X affects Y if we wish to infer a causal
relationship behveen X a n d Y. Even i f we cannot cmpiricallv demonstrate
! ; D T ~X
F nffecls Y we nccd to p r o v ~ d e
d piat~sibleaccount of the connection
(plausible in terms of other research, cu r r ~ n throrv
t rtc.). Fnr cxan~plr,to
support the assertion that sector o f crnploymcnt affects achievement
oricntat~onwe might arguc that the private sector fosters the dcvcl(~p-
ment of an achierement orientatinn by strategi~ssuch as paying per-
formance bonuses, developing a culture of Fligl-rcr expectations, providing
better resources and creating less job security. When backed up by this
t y p ~of rcasrlning, any cnrreIatinn between crnployment sector a n d
achievement oricntatinn can bc plausibly interpreted in causal terms.
TYPESOF RELATIONSHIPS I N A THREF-VARIARLE MODEL
Types qf cnrrsaf pdterns
Anv relationship between trvn variables will consist of two components
~ I R E C TA N D INDlRECT C h U S A I REl.ATlONSHJPS
- a cdusal component and a non-causal {spurious) component, The
causal component can consist of a direct component, an indirect
Cauqal relationships can be cithcr direct or indirect. A direct relationship component or both.
is one wherc we assert that the cause affects the outcnme directly rather I t follovvs then that any relationship between two variables can be
than tFiaother variables. An indirect causal relationship is one whcrc tlie int~rpretedas:
t operating via its lnflucncc on another variable that,
cause has its ~ f i e c by
in turn, produces the cffcct. 'P'l~evariable through which the hvo val-i-
ables arc. rulatcd is called the i~lterrtrrllripvariable: it comes in time a n d in a direct causal relationship
n causal sequence bt.tcuvcn t11c inikini cause and Il~c.rffcct. For examplc,
an indirect causal relationship
n spurious relationship
we might argue that the way the priv,lte sector produces higher achieve-
ment orientation is by making e J t l ~ ! o yfear ~ ~ for their jobs (the intcr- r any combination of these.
vcning variable) (Figure 3.2).
Tndtrect causal relationships may be simple (as in Figure 3.2) nr consist Figure 3.4 il lustratcs the possibiIities where we have three variables
of an extcndcd causal chain or a number of diffyrent causal paths ( F i p r e which, for the purposc of the example, T will call X, Y and Z. The
3.3). I relationship between X and Y could be any of the following:

/-CLJ
I te) Dlrect and lnd~rectcausal
relahon5h1pand a sp~lrlouscomponent
{,I 1 LJ~nlcl~ r r r r c r r l \ 6r1llr)zv.;
11-1 ti tnt,, I I\ c a p d l ~ l tot
~ Iltl~ngc h . l n ~ c dand
,
i r jk plau~ibtet l ~ a tN c i l ~ ~ tydr t d r ~ c cch;lnqt.s in I In lIip ab.;i*nse t v (
finrttng anv nthrr \.ariablr that is ri5ponsibI~for this rrlation41ip ~ v t .
may cmtinue to arguc that thr vbwrvPd ctjrrclGlt~on IS dircct n i i d is
causal ( F i ~ u r 3e 421.
(%) firrJlrrr! r f i ~ ~ s n lY fcrllo\v<X in time, 1' 1s cLip~blr nf being changetl,
and i t is pla~~sible that X cnuld producr shan~t-sIn Y. I-lclwt.\.cr, In
this case we .Irtl syt*lliiiqout the n ~ e c h n n ~ vbvn which X affcctq 1'. Wc
m;ly think of Z a< a 5inclc mcctianisnr, or a wt~nlclot of i n t e r v m i n ~
varlablcr; (Figure 3.4L-r).
I I ~~ n dY arc not causallv r r l a t ~ dto nrlr annther. Even
(c) G ~ I I I T I ~ I ,\C
t h m ~ ~Y'hmight follc~ir.Y in t i m ~nncl bib sapn'tllc of l l c ~ nc h~~ m y c d ,
both t' and I a r e joint c f f t ~ t sof somp third variable Z.X ;lnJ ) (-0-
I l d i Direct and nblrecl causal r@lat~onrn~p] vnrv purr*ly hecauw Z Iias a simultnneorrq c*ffeck on both S artd S
(Figure 3 4c).
(ct) E,)fI, rf!r,rloitcl i t ; d i r c ~ - t The cffrctof X may bt. partly vi.1 its cffect cm
an i n t e r v r n i n ~variable and pnrtlv d~rcct( F i ~ u r .3.+Ld). c
Eta) Prrt-rt nr;d zlidrvt7c.f ciirlwl nrlri sljrrrrtltr.: TIIV rvlationship betwervn S
and I'coutd cnnsiqt of threc mrnpclncnts:,I dircct 1-at13alp;rrt ( S - . Y ) ,
-111 intl~rcctc-ausal pJrt -1 - Y ) . ~ r l < l >, \purriril< part (Y. Z .: )
[Frgtrrc 3 . 4 ~ ) .
( f ) L)lrcLt (t11(.<01 ri-hti(v14ti~) ~' {r
t * o r r t l > ! t t f ~:it1111 ~ / ~ ~ lL ~. Oi ~( I v~ ~ ~~(F:iqt~re
[ ] I I ~ ~ / I !

't.4f).
( 1 ) Splrnous (nnn-causal) and r11r~cT { g ) Imlirr,cf crltr,wl ridntw~nshrprorr;brrit+rl'ilr~fiin ~p~irlo~i': c t ~ ~ p ~ i r r ' r (Figure
li
sal relal1on5hlp of X w ~ t hY
3.4~)

T h i s s r t of pos.;~bilitics rc~sultsfrom situations In which rzrc ha\,e only

t1irt.t. vnrial.rlcs. The more variabI~5wc takt. intc~.nccnunt, tlic more ccrrn-
p1t.y matter< l ~ c u m e .
k17hcn wr collect and nnalysc data i t w n hp hclpfu! to d r a w rfitngr~rn.;
t o ~ p r l out
l the ways in wh~chwe ptc~poyevariable< arc inttarrclatrd. W P

-
nred to 5pecify:
1 ~ gS~urtous
) (non-causal)and rndlrect causal relat~onshtpof x W I ! ~v J
1 whethrr selatinnships arp presumed to he causal or spurious
2 whether cauwl ri*laSicmshipsart- expecttd to be direct or indircct
? the mcchanr~ms (inten-rning \-,lr~;rble.;) unrlcrl!.fng anv indirect
ca~rsalrc1ationship.i.

Providing a frame o f reference

 WI-l A T I5 RESEARCH DESIGN?
[divorce -
Propositron Dlvorce leads to emot~onalproblems In young children.
emotional problems of children]
Observation We find that many ch~ldrenwith divorced parents have
emotional problems
Propusrtion Private schools produce high academic performance among
therr students.
[pnvate schools -. high performance]
Observal~onThere are many instances of studems from private schools
achlevlng h~ghly.
[unemployment -
Praposrtron Youth unemployment IS responsible for youth suicide
hrgh suic~derate]
Obsen/airon 50% of young people who commit suicide are unemployed
Figurt* 3.5 Prop>sittnrzs nnd otisrwwt~onsn-itl~orrtrr n ) ~ ) r p r n t r rf~r vr i ~ ~ofr
~~y~~rt'it~>r
research study is providing a compamtivc frame of reference. Dfffcrcnt
research designs go about the task of providing compnrisnns in different
ways.
By making comparisons we provide a frame of referencc within which to
try to make sense of particular findings. 'me importance of this can be
seen in khe iIlustratio~~s in Figure 3.5. In these examples the obsesvatinn
in each case hardly provides convincing support of the propositii~~.
'reason for this is that there is no frame of reference within which to make
sense of the observations. Thcre are n o groups wit11 which to cornparc
and contrast the observations. It is only hy making rnvrpnrisor!s that
o u r observations take on much meaning and we are able to clirninnte
al tcrnative explanations.
The propclqition that rii-rlorce leads to e ~ n u f i t ~ r?lnlhl~rns
r~! in ! / n r i 1 7 ccl~il~~rrrr
wcluld encourage us to expcct lo f ~ n d emrjtional problcms among
children whose parents a r c divorced. But finding such evirlence will not
get us very far. To go further down the path Jt explatlatian we must
make compasisnns. Arr emotional problems greater than, the same as or
less than those among chilcfren from intact farnilics? Do the obscrvahorls
among children of divorced parcnk simply rcflcct that ~ ~ l i i cwe
find among any group of cliildr~n? 1 I
CAUSAl'IC3N AND TFTE I,OGIC 01;RESEARCH DESIGN
Propostiton Dlvorce leads lo emot~onalproblems m young ch~ldren.
Restated pruposr!ion Children whose parents have dlvorced are more
likely to d~splayemotronal problems than are children whose parents
have not d~vorced.
[parental rnar~talstatus -level of emotional problems]
Proposrbon Private schools produce h~ghacadern~cperformance
among their students.
Restated proposition Private schools will have hrgher levels of acadern~c
-
performance among their students than wrlfgovernment schools.
[school type level of academ~cperformance]
Proposdion Youth unemployment is responsible for youth sutcide.
Reatatedproposrtion Unemployed youth are more likely to comrnft su~c~de
than employed youth
[employment status -su~crde rate]
I t will be recallcd that when we say that two variables are related it
means that variation or difference on one variable is linked to differences
on the other variable. In this itlstancc we have two variables: (I) parents'
marital status (married versus divorced) and (2) child's emotional
a d j u s t m ~ n t(low vcrsus high). Tf our research design incIuded only
divorced parents we could say nothing about the impact of divorce on
children. Even if 100 per cent of the children from divorced parents had a
low level of emotional adjustment we could draw no conclusions abont
the impact of parental divorce on childrrm's emotional adjustment. We
I would need to compare the adjustment of the children with divorced
I parents with the adjustment of children From intact families. The
One d$krmcr in adjustment levels in the two groups will provide strong
evidence regarding the alleged effect of divorce on children's emotional
l
adjushnent.
'The propositions in Figure 3.5 only mtlntirln one particular group
(children of divorced parents; private school students; suicidal youth).
It can be helpfu! to restate thc propositions to highlight the implicit
~ cornparisnns. Figure 3.6 restates these propositions using explicit com-
parisons.
MULTIPT~E
COMPARISON GROUP5
I
The independent variables above (parcnts' marital status, school type,
h would youth crnyloyment stahls) have all been treated as two-category vari-
ables (dicIintomies). However, we are not restricted to comparisons of
hnt~r\.~ r h ~ cuth the ~ i i l t c r ~ ~ itr..l\ ~ c t ~rt,~pon+il,!r
< I t v an!, cliltclrct~sc\ In .I+*. Ii'e could ihcn ctmtp,lrc t h t ~ s rgroup.; and sec i f t11c.r~rt3r,rr differ-
,~c,~~lt.rni,- a ~ - l i i t - ~ c t'~ n c ~ ~ i r n i v s in their I ~ v c lof acadrrnfc ncliirv~rncnt.
C'on~pari\cm\ 11f chi Id rt*n frcrnl ci i\-orcc*ii ,lticl 1nt.7~tfiimilrcs ,Ire. T!icre .lrc many prilblcrns with thiq approach The trbviotts clnv is Lhnt
cnmplicnte~lbv tllc lac! t l l ~ tt~picilllvc F i i l ~ t ~ (1I . ~ nd ~\~orccil yarcmtk arc rr7c must have thr r r l e ~ ant information rln nrhich to niatch. 5incte thiq
olcler than chlldrcn from intacl f n r n ~ l ~ eAny s difference< bckwren the will alwavf; btl Itmitrd, t h t groilpq arc. likclv to remain i~nrnatchedon
lcvcl of cmtrtiunal a d l u . ; t m ~ i ~olt thc two s ~ t . ; of c h ~ l d r e ncol~!dbtl diic to important t w t u n k n t i ~ ~taclor.;.
n A n t ~ t l ~prol-rl~m
~r i s that many casrs
.igta diffVrcncc<rathcr thdn t l ~ cm a r ~ t a lstatus uf t h e ~ rpnrrant.;. Crnm both group\ s~rnplvwill not have matches frrrrn thr% other Rroup
rhr s;lmt, prohlrrn can arisc. 1vht.n lor kin^, st tl~rstimc grcwp or and will have tn I*c ~liscarded.1f WL- match nn more than a smnll numbcr
cntrgny of peopIc o\.er time. Icicnl l y rvr th comparisons over hmc thc c~nly of factor\ wc r ~ i loften
l end with only a very small number of people
d i i l ~ r t ~ n cbetst-vcn
~.; thc pre-tc5t 2nd the p o q t - ~ t ~\houlclt b r thc ci7cntwe in thr s t u r l ~slncta the nurntrcr nf cilse.: from both group.; that match c.,in
artb pr~jpurinqa* thr r;iu.;c nt ;InV c11an):c. M o r v t - ~ ~ man!. r, crt~ntsc,in Lit, cluitt. small.'
c i < U r r v l i i t - t ~<.In aiiillint frlr Ihc changc ~ n d tI1c.r ma). conf uqc oilr
Cilnl~TlWtlq2nd ntir m~tirib~ttir)n of 1%hcit lies behind Ihc chnngr i r i lie
rr~~tcrvnt. v;in,~blb.[wc Chnptt-r -1 for Furt hcr di>c~rs<iuri)
Since corn!>ari.;rlnh arc. ccili rl71 t(l g(,nEi r c s ~ a r ~dckicn I1 si0cmuqt cn511re A simpler dnd m o w ~>fft-ctivc wav ~ ) ma f kin^ ~roup:: comparrzblt1 i s to
th,it tfir1v art1 mraningful r l ~ t y rnorr- MT car) rt'rnoIrv u t i ~ n t u n ~ l rand d randornlv nllocntp ppnplc to diftenwt ~ r ~ u at p sthe h e g ~ n n i nof~ n study.
unknown clilfcrtbnctaqtlet~v\.ct*ngroupr, tlits n1tx-t. n f c rcdllcta tho rtqk ot Rv randomly assignjng inditidtrals t r ~each g s ~ i u pany d ~ f f ~ r c n c c s
m i s t , i h i n ~. ; y u r r t ~ urt~lation.;liips
~ for cnus;ll rrlaticliishi~.;- tli,it i?, thc h t * t ~ t ~ uKroup.;
n l d random rather than svstrmatic. 5 3 Ion)= a s
s l ~ n t ~ be
mrjrc wrc3t'linl~nnte~ l t t m , ~ t i pIiiti~iblc \~e cal>lat~ationr;. grotlps ;lrchI.?rgc r t i o u ~ h random , assrgnmvnt s h ~ u f datit~>mntically pro-
r!~c.c' g r { ~ ~ ~ p cr i i hcrvny;lrrlbTt~p+ufilc.c: clil Lvth Irnor\.n mil 11nLnown
l-kit3rt-C I ~ L * tt~urT I I ~ I ~\ I IrLitt3~it*\
for T T I ~ ~ \ I ~ ~ I / t~ hI c> ~K~ ~ m i p ~ i r ~ ~tjlb i l ~ t \
S"""3. t,ritcrn. Fron~a qtn!ist i c ~ ~! ~ ~ r ; p ~ ~ t~3nCjo1n i 1 * t , , a<.;ignment of ycrrplt, hr
qrotl p%12111 111~1kv iht' grt,t1~,1;lilcl~t~cill for rll E intents anti pt~r\'oqt.s .ln~J
yrol idc*s 13,hat Dnvi.i call< '7hc all purpow kpuricwitv in5uranrc of
rCinrlnlni7atic~n' (IQK4: 95). \Wt> cotitrrd tor .In ~ n f i ~ i i nt ~uqm b e r of plsnciblr~
W1~t.nrwruitinq thc xrt)tlp< w v c.ln dcIil-rt.ratvl\. match then1 on rclel'ant ~ P ~ C R ) Iw of t l i i 7 1 1 l nrib(Camphell, 1'W.
rll-al h v p o t h r s r ~rr*ithour S J I P T E S I ~7l*lrai
c h a r a c t e r r ~ t i c ~TIIIIY
. wlicn cc*mp;lsing sludrnt.; a t t ~ n d r n gpj\.ernnlml Thiq 15 thi* npproach takcn in d r i ~ et r i ~ l s .Individual4 are randomlv
qchool.: on lllr onc hand ,ind fvc paying prir.,itc ~ ~ 1 1 1 7 0(in 1 ~ thta ~ ~ t l i r r n s s i ~ ~ cto i of severaI Eroups and cach g r t l r ~ pis thcn R ~ V P R;1 d ~ f f e r -
~ ionc
11rl11ij we s h o u l ~ lenwtre Ilint the two st44 ~ l .;luclPnls f are similar in tl.1.m.; t>n! drug. Sincc the g r o u p shoi~lcih,lvc v l r h ~ n l l yidentical profi1r.s tu
IBF intvlligcnl*c, ; ~ s p + a t i r ~ n sp,lrcntaI , rcquurctbs,t-d ucntinn vnl~res,fatnil v hegin WI th, any difftlrencps in outcnmes bcttvecn the groups should be
Ilistr~ry,gentler and agc. R v comparlnf; like with I ~ k ewr. stiuuld be ablo Jut1 trr the d~ftcrcntt r ~ ~ i h n e n ndrninistcred
ts to rach group. kkI(>~'t'vt'r,
to ~'inlatcthr ~ I f e c tnf tlie tvt* o f ssl-rt,clf on acndc.m~cachirvrmcnt. this approach rs r ~ f t c nnot n p p l i r a b ! ~in social res~nrchhefause practical
7'he ptntrltm in i o m p a r i r ~ glikr with lihtl is tcl e<tahfisl~the idtantit]; of and cthicrrl considerations prr-cludc 11.; a r s i ~ m i n gprr3plc !I>groups and
all thc* t*asiahli** on whE1.11 M'C nvecl b match the grnttps. \Ye can match thcn dc~ingsontetlring tn one p o u p to sec what effect i t ha< (wc Chaptcr
lor fhc charactt.ri>ticc ttmt w r k f ~ o i cnirght ~ rcmtattlinat~~ o u r restillc btlt 5). 4~lrantagc.;and dra1vback5 of thiq appronch tn social rtsearch nre
Ihcrc hc o t h t ~I S C ~ C J T ' ~ that 1t.e have 1101 t h t l u ~ h of.
t d i q c u ~ s e di n C'haptrrs 4 and .'

FI / ~ o %Ctc!,i f T I I , ~ ? ~!!ij!s: icdtafi!!~j:rot!p > ! ~ ( ~ T1~~I rL-nI~ t c h ~ c~ti !li<, b t a y ~ q -

nInq of thtb - 1 1 1 ~ 1 i lwfdlrt* r-ritical P \ t>nI< (1b.g dii.orit', iiffr+r~~fin?: 11ie I f ' l ~ i Irandrmiz,itit,n
t nlinirnileq thr chancesuf initial rli tfercncw Ilettct.cn
q ~ h t ~ r >:Intrthi.r
l) ,~ppr('*lchrt-ferred t13 bt. Spcrtcw (l"81: 4 4 ) a\ ,T 'p~it-17- Errwtys t h ~ r erc stlF1 t h r clinnc'~that thrrc u-ill he d~ffcrtnnct*~ ~~~~~~~~~n the
\ \ - r d , p - t , c c t l ~ ~ ~i nr
. t 'r)!\.rtc r t q t i n ~~r~aicllctf
r r i w 17s frotn a ~ c h o h~. r > t \ of
l qnluys - t.+pt,ci~llyin .;rn;lllcr RrrlrlFq 12:h~.re tt-nnt to Ilr certarn thnt
.;tud\ p.~rticip,int.:nl/t7r. all thr. datn hn\mP Ivt-n r r ~ l l ~ ~ c tFor ~ . dc\arnylc,
. we tlw ~ r o i ~ art.p s cc~rnpar,?blenn .I partjcular chnractrrrstic ~ c can c take an
rn~clit ha\^ a l , ~ r <niunb(~r r nl41irclrnls frnnl qcr~w-nmcntand fr(b p,>vin:: additional \ t ~ pto plnranlcc t l i ~ the t gn,ups arc ctlrnpamblt* a t least on
privalt' rc111101~.1 r i m pcrril rut*c~ruldr ~ t t r ~ l ac tjirotlp of p w r r n m t ~ n t q-rccific \7L~rialllc*. 'I his i<achtrvprl zirllh u har i<called the randcllniscrl
rcIrotr[ +furlcnl< ant1 a crrrup nt fcr plyirl): pr!\.ntta .;ch~,ol st~~clt*nt< ~vlrt~ block dt,5tKn - ,I ctunbinatirm t>f h(lt11 rnr~ijomi;r,~iicrii and rnatchitlj:
arta c.ornpar,~bIe111 ttltnls of ~ntc!lif:ence, a+piration$, p,?r~ntnlrcsrntrrcs, 511pposc-wr- wanted to c ~ ~ , l I i ~thr a t e~ F f c ' t011 a r a d ~ m i cperfornianctnof
... I . .
- -'.+-*-* Eh.3 imntrrt,in~tnr i l ~ ~ I u c , i t ~ fo, n ~ ,t n ~ Il v~ I ~ I V,
~ o gcn11~r,
I ;1ni1 thrrta nirtiro~-!'; r + F t l r l r v e r ~ n~rnix*rrsitv
~ courstxs: (I ) d(~livcr~.
 46 WHAT IS RESEARCH DESIGN?
of lecturcs and tutorials, (2) correspondence using written materials and
(3) electronically using t h intcmet
~ and e-mail. Further, suppose that we
could assign individuals to one of these three modcs. Random allcratinn
would be the obvious way. But suppose we want to be absolutely certain
that the students in each of the three modes are of comparable academic
abiIity.
To achieve this we war~lddo the following:
1 Obtain a measure of academic ability at the pretest.
2 Rank all the students from highest to lowest ability.
3 Select the three students with the highest abilitv (because we have
three learning modcs). This group of tl-rrcc students is block 1.
4 Randomly allocate each of the three students to a group (one to
face-to-face; one to written correspondence; and one to elcckonic
learning}.
5 Select the next three most able students (block 2) and repeat the
random allocation to groups.
h Repeat this until all students have been allocated to ont? of the three
groups.
An alternative way of making groups comparable is to do so at thc data
analysis stage after data have been collecfed. It involves multivariate
analysis that, in one respect, matches woups on specified variables (see
Chapter 12 for a discussion of the logic of some nf these techniques).
Although the procedures of multivariate analysis can be very complex,
the essential logic is simple. Suppose we want to compare the emotional
adjustmcnt of childrcn from divorced and intact families but we believc
that any comparison would be confounded by the fact that, on average,
children whose parents divorce are older than those from intact families.
'
h y greater maladjustment amnng children xvhnse parents have
divorced cnuld occur because they arc older, a s older children display
' > greater maladjustment.
Multivariate analysis rcmoves any effects that might be due to age by
comparing like with like - by selecting, say, preschoolers and comparing
those from divorced dnd intact farnilics tn see if, despite similarity of age,
the children show different levels nf adjustmcnt. The same comparisons
could be repeated among say 5-8 year olds, 9-12 year olds, 13-15 year
olcls and so on.
The obvious shortcomings of this approach are similar to that of
marching. We can remove the influence of variables t h a t we have
thought of and on which we have data but we cannot remcwe the cffccb
of unknown variables, or thwe for which we have no data. Since we can
never know what factors we have missed thcre is always the danger that
factors we have not thought of may be contributing to group differences.
I CAUSATION AND THE LOGIC OF RESEARCI F DESIGN 47
1 Throughout this chapter I llarc used the terms 'intervcniion', 'treatment',
'independent variable' and 'groups' more or tess interchangeably.
Research designs vary in terms of the type of independent variable
employed and in the number of independent variables built into the
I study.
Whcn thinlung in causal terms (X+Y) X is thc independent variable.
When we conduct the research and do the analysis we compare the
I outcomes (Y) for different groups. The groups are defined according to
which category of the independent variable they belong to. In some
studies the investigator introducing an active inferv~ntiondefines the
independent variable (e.g. allocate different people to one of three modes
I of course delivery - face-to-face, correspondellce or electrnnic). In other
situaticms the indepcndent variable is d e f i e d by s nnfurnlly c~cc;rrrin~y
,.
n ' a person retires or remains employed). In other
~ ~ ~ f e r z l e n f i n(e.8.
situa tiuns the indepcndcn t variable is determined by the rclativcly j x e d
I nitrib~ites of participants (e.g. sex, education, race etc.) rather than by
! intttrvc-tions of any snrt.
1 R~searchdesigns can also differ in terms of the number of interventions
that are madc. Where the independent variable involves either active
or a passive intenrention we can examine the effect of single versus
repeated interventions. We can see whether a singlc intervention has a
different impact than the cumulative impact of repeated interventions.
For example, we might be evaluating the impact of teachers giving
students ncgative feedback on their work. Initial criticism of the work
may boost a student's effort and performance but repeated negative
feedback may lower performance as the student loses confidence. A
multiple intervention design helps refine our understanding of t h way
~
in which interventions affect participants and enables us to rule out
alternative rva ys of interpreting results.
Qimensions of a research design
I
The abovc discussion has ida~tifiedsix core elerncnts uf a research
design. The particular mix of the elements in any study will yield
different designs.
Thcsc six main elements in producing a research design are:
48 WHAT IS RESEARCH DESIGN? CAUSATION hVD THE LLKIC Of: RESEARCI I UFSIGN 49

1 T k r rr11tr~br7r of pn~rrpsi l r tlrr desi,y~r Designs will vary from those with
Method of
no curnparisons ( r . ~single . case d c s i p oi casc study) to those wit11

2
many different cornparison groups.
T/rrb nunrbf>rof 'prr>-tr,5t' r n i ~ a ~ r r ~plrnsrs ~ ? ~ ~ r Designs
)~/ vary from those
1 allocation to
groups

with no 'prr-test' (e.g. cross-scctlonal designs some experimental Random

dcsi,ps) to thuse with a series of 'pre-tests' which cstablish prc- outcome varrable outcome vanable
existing trends before an event. (experimental
3 Tlrr ~17117rber. qT 'post-t~st' mcastirewrmt phnses All designs s q u i r e at
least one 'post-test' - the measurement of an outcome variable. In
some designs (e.g. cross-vctional) there will be onf 'post-test', while
o t I l ~ rdesigns can have many post-test? to help distinguish between
Measure an
short and long term outcomes outcorns varrabie outcome var~able
4 nre ~ ~ ~ r i hif onllncfltiun
d of c n s ~ 5fn grnlrfls In multiple group designs (control group)
groups call bc made comparable by allocating people to diffe~ent
groups by random allocation, 'pre-tcst' matching, post hoc matching or
block matching or by sing statistical controls in the analysis phase.
5 T l i p nat~lrPof t l z ~rr~frr:~rrttiunStudies that rely on existing variation
(cross-sectional designs and tlwse wit11 'fixed' ~ndependentvariables)
I~ave1 - r ~in tc.r\-cntion5. Other designs rely un intcrventions between a
prc-test and a post-test. Thcsc 'interventions' may be either active or 3 Random allocation to the groups brforc thc prc-tcst.
~iah~ral.' 4 One inter\ ention (test/ treatment).
6 T!rc t ~ l ~ r r l h tI$ L ~ I I o with
~ t '~~ ~ ~ ~ ~ ~ ~ ~ vDesigns ~ T s an intervention can have 5 One post-intervcntir~n(pust-test) measure on the oufcomc variable.
t.it11c.r a single ~ntrrvention os multiple interventions. Multiple inter-
ventions can bc used to identify thc effect of cumulative 'treatments'. This d e s i p is illustrated in Figurc 3.8.
The analysis of any effect of the intervention focuses on changes in the
experimental group before and after the interventio~~ and a comparison
A range of research designs with the rate of change in the control group. If the change is greater in
thc cxperimcntal group than in tlie control pinup the researcher will
The carnbinatiim o f the possibilities created by these six elements of a attribute this to the impact of the independent variabIe (the treatment).
research design yields a large number of possible designs. A useful
summary discussion of many of these designs is provided by Spector
(1981).To impose some order on this range of possibilities it is helpful to
think in terms of four broad types of design. For each design type, The basic form of this design involves:
decisions taken by the investigator will produce variations within the
type. These four types providp the structure for the rest of t11c book. The
7 One group.
four brorlJ lvpes of design are expcriment~l,longitudinal, cross-sectional
2 One pre-'intrrven tion' moasurrment on the nutcornc variable.
,jnd cast qtudy. 1 zvill examine each uf thcsc in turn.
3 Onc 'intervention' wl~ereeverynnc ruceivcs the 'heatmen t'.
4 One post-'intervention' mensurement on the outcome variable.

The class~cversion c ~ the

f t.xperimenta1 dcsjgn has the following elements: In uffcct, this dcsign is similar to the experimental design cxccpt that
there is no control grnup and typically only one 'experimental' group.
1 Onc pre-inter'! ention (prc-test) measure (In the outcome variable. The design is ~llustratedin Figure 3.9.
2 Two groups: one gmup that i s rxposcd to the intervention (the The analysis in this design cornparcs the pre-intervention measures
d group that is not e x p s e d to the inter-
expenmcntal group) ~ 1 1 nne with tlie post-intervention measures. Change in these scores ran reflect
vcnlion (the control group). the influence of X on Y. Mowe~rer,the nhscnce of a randomized control
50 WHAT IS RESEARCH DESIGN? CAUSATION A N D THE LCK;TC OF RFSEARCH DESTGN 51

I Method of 1 Pre-test Infervention Post-test 1 Method ol Pre-test Intervention Post-test

allocation to
groups
I
11 i
allocation to
groups

----:
(XI

Random

(one group)
11 'Treatment' Measure on
Outcorns variable

-- I
group makes it difficult to know whethcr the intervention or some other
I "on-random 1 None No 'Ireatmen Mea- on 1
outcome varlable

factor prcxluces any change. - -.

. ..
.--

TIlc basic elements of thc cross-sectional d e s i p arc as lallows:
1 Instead of interventions the cross-sectic?nal design relies on existing
variations in the independent variable(s) in the sample.
2 At ledst one independent variable with at least two categories is
present.
3 Data are collected at une point of time,
4 There is n o random allocation to 'groups'.
This design mirrors the post-intervention phase of the classic cxperi-
mental design but without any random allocation to 'groups' being
made. The data for this design are collected at one point of time and are
analysed by examining the extent to which variation in the outcome
variabIe is linked with group differences. That is, to what extent do those
in diffcrcnt categories of the independent variable differ in reIation to
the outcome variable? Causal relationships are established by utilizing
statistical controls rather than by random aIlncatinn of people to groups.
This design is illustrated in Figure 3.10. Tn this casc thc 'inten~cntion'is
simply being in a different category of the independent variable.
Case study designs rely less on comparing cases than on exhaustive
analysis of individual cases and then on comparing cases. A distin-
guishing characteristic nf case studies is h a t contcxtual information is
cnllected ahout a case so that we have a context withiq which to under-
stand causal processes.
Case study designs might consist of n single cd5e study (e.g. a com-
munity study, a study of an organization) or a series of c a w 5hrdit.s with
perhaps cach cnsc testing a theory from a difftxnt angle. It is useful to
think of a case study in a similar way to an experiment. We do not finally
reject or accept a theory on the basis of a single experiment; we try tw
replicate an experiment and conduct it under a varicty of conditions.
Similarly, a case strldy project that entails a single case study is ana-
logo~oustn a single experiment. If similar results are found in repented
case studies, or predictable differences in results are found for particular
cases in thc study, then we develop greater confidence in the findings of
the cases in the same way that we gain confidence in experimental
rcsults that are found in repeated experiments.
Summary
This chapter has ~ x a m i n r dways of strircturing rcscnrch designs to help
draw convincing causal inferences from the research. Since causes
cannot be 01~t)rxed they must bc infcrred from obw-vations. However,
rncorrtct Inferences can casilv he made. The chapter has considered
ways of structuring research to improve the quality of these infercnccs. A
rangc of ways of interpreting correlations between variables was orrt-
lined and criteria required for inferring that a correIation reflects il causal
connection were provided.
Tl~cchapter emphasized the importance of making meaningful corn-
pnrisons behveen groups as a core clcrn~ntof drawing causal inferences.
 W H A T tS RESEARCH DESTGhr?
Onc of the tasks of research design is to structure the research so that
meaningfu! cnmparisons of outcomes between groups can bc made.
Finally, tlie chapter outlined s i x core elcrncnts of research designs:
nurnbcr of gror~ps,number of pre-tests, number of post-tcsb, nature nf
allocation to groups, type of interventions and number of interventions.
Research dts~gnsvary in the way these eIerncnts are dealt with. Four
main types of research design - experimental, longitudinal, cross-
sectional and case study - were then briefly described. These broad
categories of design provide the framework for the remainder of this
book.
Notes
1 R c r c can be exceptions to thi5 where a suppressor variable may he
operating to mask a causal relatinnship- See I I o s ~ i ~ b ~(1968:
r g Chaptcr 4).
2 We sl~allsee later that this pmhtem of Ioss uf cases can be overcome with
cprtarn fcorrns of multivarlatc analysis.
3 A nalurnIly o c c u ~ring ~nlervcntion1s an event. llint takes place b~twccnpre-
tust and pocl-ic.;t w ~ t l ~ to utl-te invcsiigd tor initialing a n y intervention. An
example mtght be a study of fnrn~l~es over tilnc In which wrne fam~liescxperi-
ence parental divorce. Divorvc is the 'natural intr wcntlon'.
PART IT
I
1 EXPERIMENTAL DESIGNS
TYPES OF EXPERIMENTAL DESIGN
I
1
I
I
This ch(1ptc.r builds on the elements uf rcscarch dcsign outlined in
Chapter 3 to intrt~duccn variety of types of experimental designs. It.
outhncs thc diffwen t environm~ntsin which experiments can be con-
I
ducted and describes both simple and more complex variations of
experimental designs. This chapter does not seek to evaluate experi-
mental designs: that is the task of Chaptcr 5. The discussion also pro-
vidcc; a useful framework within which to undcrstand some of the
strengths and weaknesses of thc research designs discussed later in this
kmok.
The classic experimental design
The classic experimental design focuses on two variables: the inde-
pendent variable (the cause/intervention) and the dependent variable
(outcome). 'll-rcpurpose of the design is tn remove the influence of other
~~ariablcs 50 that the effect of the intervention can be clearly seen. Since
the cIassic cxperimmtal dcsign was outlincd in the previous chaptcr
thew is n o need tn repeat i t here. I fowever, an example will help cIarify
its elements.
Suppose we wanted to test the proposition in Figure 4.1, narnply thr'
I H I I T P diffic~lltif is to joitt n gro~rptlrr more clcsirable tlre gronp will serrn.' In
order tn test this proposition it is necessarv to devclop measures of
desirability of membership and to dcfine what is meant by 'difficuTties nf
joining'. The expression could refer to things such as restricted member-
hip, long waiting periods, high fees, harsh initiation, Iligll time com-
mitments. For the purpose of this exercise we will use 'severity of
 Perretvd
Dilticultv of
rornlng group
belong~ngto

Random
atlocallon

(expenmental
I
rre-rest a v ~ r a g e
desrrab0l.f~.SC-P
'Severe
rnrt~atlon' 'Po5T-tr>r ~vareuc
des~raol:,'ys:o?e
Erhr~nqs= €2 - El

, group)
initi~tion'inkc) tlit, grnnp as thc indicator of diflicultv oF join in^. Tlie
cla%%icexp~rimmtal design for such an invttstigation e n t a ~ l s ti111
i r j l low in^ qteps: Random NOnttaton C, Cchalrqe ' -~ 1 1

P ~ t e saVerRgP
I Post-test averaop
1 RandornEy n l l n c . ~ t i npeople
~ to t n w group: a n ctpertmental group (f~~rr; I
and a contri~lgrnup,
2 Ubta~ningrnen<tll.cs of prrccivrJ ijrsirability of gmup rncmbi.r<liip.
3 Subjecting i h rrp~rimenta
~ l groir p to severp inittnlinn reqi~irt*nicnts
and the control group 50 ntl initi.~tic>nrcquircmtwts.
4 Obtaining rntncurl- nf desirabtlitv of group nlcrnbcrqhip from both
?:mtrp aftcr thtb t n i t i a t ~ u n t * r tlic erpcri~ncnbal grouy ha5 I ~ < ~ ~ ~ I I
~ornplcted
C'nEculrltin~the i.lrrr~;~r in dc~irnhilityof mcmbvrsh ip for thc r.~ijr,ri-
r ~ gmtrp bcfort- a n d aftcr tlir initiation and nt the control gmup at
r ~ r t , tnl
Experimental
T I and TZ. inlt~atron'
h Comparing t t ~ Ic1.t.lr of c h a n in~ ~the experimcnta! group with that in I Mean
Z ~ Pcontrol Ernup. I

i
1
-
I

I f chatigcs in the prrccived desirability o f rnembersliip arc diffi*rt*nf I

I
b c t w c ~ l ithe ~ Y L firoltps
I (MOTC' t11aii wot~ldbe duc to chance) this should I
Control group No initlation
br1 bccnuw o n e grnilp had ecpcbricnced sevrrt. initiation. 'This is I
rrprtwnted in Figurt. 4.2. Here thc qvmbol TI reprtwnts time 1 or the I MPT
yrp-tmt. E, reprvwnt.; the expctirn~ntalgroup at timp 1 and C, r ~ p -
rrqcnts the contrnl Krirup at time 1. Thr amount of cf~angcm the cxperi-
rnpntal group rs symhol~zedas C,.h,lnF,y and 1s thc d ~ f f r r e n c ebetween Ihe
grinip'~desirability score a t TI compared with T ,. 'l'he comparable
change for the ctmtml group can bc ~ymbnlizeda < C, h,lnp,.. The ~ ' f f of~ ~ t
thc intervention ic the* difference 171,ttv~enthe arntrunt 13t change In t l ~ e
c*\yr*r~mc.nta! ernlzp bcfort. and aftclr the inter\.rn!tcli~( Cihanrc I A ~ C I~ II C
~ r n o ~ l nrrtt c h a n p , In i he control qrotlp (Ch,k, bc7t1vt.m 7,and T- l .P. T < thi* .;arnta luvcl n i d t r ~ r , > l - ~ l ibefort%
Rcrth ~ r t + u phad t\ thr initiation
T ' , I ~ . , ~-~ ~C,-l,Anl:,
, ). I f t h i s i~ jirratcbr Ihnn zcro t11t'~t M'I, wnuld p r < d ~ ~ h l y (tht>rr3cull o f ranclclnl allcr,~tion).
cnnclucle that thc ~niti~ltion had a n i m p c - t on rlcb~rahtlitv. 2 TIic dcqira bility sctlrcb nl the experrrncntal F O L I ~ incrca;lsrs (hy 25
If the rncasurc of desirability was mcasureci qr~antitativclyon a scnlc i point').
vt, sav, 0-100 with a t ~ i ~ score
h ~ n d i c a t ~ nhigh
g c l c q ~ m b i l i w~
~ , ctlulJ 3 Tht. rlt~\iwbilitv _scnrc of thc control srirtip a l w incrr,.iws (by 10
cj~tantif~. the e f f c c ~nf the intcn,rnt~on(st!-cr:tir o t initiation). Thi.; ir p"linIsl.
~1Itwtr~itrd wlfli hvpothctical data In r - i p r e -2.3 Vking these fip1rr.k w r 4 Thcrrftrrt, wme cllnn~t. in the c.upt-rrrnt.nt;ll grtirlp I < not due t r ~
J
ctln <t,rA Illat: initiahon ( I n points tztrrtli ~ i n c ethiq Ic-\.rl crcc~arqu , i t l l r ~ t ~initiatinnl.
l
56 EXPFRIh4ENTAL DESIGNS
5 The change in thc cxpcrin~entalgroup is greater d ~ a nin the control
group thy 1.5 points).
6 'l'herefilre scvcritv o f inihation leads on average to a 15-point increasc
in percei\,ed desirability nf group membership.
The logic of this experimental design is based on two conditions:
1 Thc groups are the same in all relevant respects before any inter-
vention (TI).
2 The groups cxpcricncc the same conditions between TI and Tz except
for the specific intervention/heatment.
I f either of these conditions is not met then there are a l f ~ r n a t i vplnusiblp
~ mented in the 'field' where a rcal life environment is trcated as the
h,ypothcses that cnuld account for the different changes between the
experimental and conlrol groups. The different 1~velsof change could he
due to initial differences bctwcen the two groups rather than the severity
of initiation. If the two groups differ in other experiences between TI and
Tz ( p . g . mixing with diffcrcnt people in thc group) these differences
rather tI1n11 sev~rityof initiatiun co111d account for the different rates of
change in thc h ~ groups.
u 5000 low income families partic1patc.d in the experiment in thc 1970s. The
Experilr~entnl r-ontex!~
Experimental designs can be implemented in three different ways: in a
laboratorv, in the field a11d by utilizing natural clccurrences.
LABORATORY
EXPERIMENTS
The ldboratory experiment is designed to ensure that experimental and
control groups are exposed to exactly the same environment except for
the experimental intervention. This is achieved by standardizing and
controlling the environment and all the events between TI and T2.Maxi-
mum control over the environment is designed to ensure that the ~ n l y
feasible reas011 for group differences at Tz is the differential treatment.
An example of a laboratory experiment is a study designed to test the
effect of extremely graphic images of alcohol caused road accidents on
vicws abnut drink-driving. Participants could be randomly nllncated to
experimental and control groups and asked abnut their views regarding
appropriate penalties for d ~ ~ n kriving.
-d The experirncntal group could be
shown a graphic film of alcohol causecl accidents. Members of the other
g-ruup would nnt be shown thc film. At a specified time aftcr showing the
fiIm both groups would again bc asked about their views regarding
drink-driving penal ties. If the views of the experimental group changed
more than the views of the control group i t wc~uuId bc reasonable to
assume that viewing the film caused the cliangc. The laboratory setting
mabltd the rescnrcher to control everything, so the, nn!y difference
bctween the gi-oups was h e exposure or lack o f exposure to the film.
TYPES (IF EXPERIMENTAL DESIGh' 57
Labnratorv cxpcriments have been used to great effect in tl-tc physical
a n d biological sciences. I -lnwever, for a variety of methodological, prac-
tical and ethical rcastms their use in social research has been limited (scc
Chapter 5).
Because of the artificiality and impracticality of lalwra torv experiments,
social scientists have established experimental designs in real world
environments. In a field cxpcsiment the investigator creates cxpcrimental
and control grnups through randomized allocatirm of participants. The
experiment also involves a n interventir~n.The intervention is implc-
laboratory. This approach has become especially pnpular among those
evaluating policy initiatives and pilot p r ~ g r a m m c s . ~
In order to illustrate the nature of ficld cxpcriments 1 will recount t11c
major features of an American shdy of income maintenance among
families of Inw socio-cconornic standing. The study is known as the
%a ttlc /Denver Tncornr Maintenance Experiment (Hakim, 1986). Almnst
purpow of t l ~ cstudy was to establiqh the effect r ~ thc.
f introd~ictiono i a
guaranteed minimum income on the work effort and maritaE stability ilf
participants. Families were randomly alltlca ted to diflcrcn t versions of
the scheme (no income support, short ttrm support, throt~ghto long
term guaranteed income). Among the many variables measurcd were
work effort and marital stability. They were measured at the beginning
of the scheme and then at various points throughout the experiment.
While participating in the experiment these families went about their
normal lives and were subject to a11 the varying experiences and
additional changes (e.g, political dnd social changeq) that took place over
the period of the study.
I participated in a field experiment conducted by the Australian
Department of Socia! Security (DSS). ?t was designed to see i f the
participation of welfare recipients {DSS clients) in self-help groups
increased their living standards. Various self-help groups were esta b-
lished in 80 locations. Client5 of the BSS living in those locations could
choose whether or not to participate in the groups. Extensive measures of
living ytandards were obtained before participation and 12 months later.
In each of the 80 locations a random sample of DSS rEients who did not
participate in the self-help groups were also qu~stioncdabout their living
standards. Thcsc people acted as a control or comparison group.
Clearly this was an attempt tc) implement thc l o p c of the experimrntal
dcsign in a ficld situation. Ilowever, the design had a serious flaw.
Participants were not randomly allocated to groups, so wc d o not know
whether those who participated in the self-help groups wvre initially
similar to the non-participants. In addition u7e d o not know whether
 EXPERlMENTAL DESIGNS
participants and tlnn-participants wcre exposed to comparabfe influ-
cnces over thc 12-month period. We observed that self-help group
participants dici b e t t ~ over
r the 12 months compared to mcrnbcrs of the
cnntrol Emup. However, wc could nnt be sure whcther this was due to
participation in the self-help groups nr whckher thc sorts of people whu
volunteered to participate were more self-help oriented prior to the
cornrnr.ncement of I ~ cxperimen
F t. These initial differences might have
accounted for h e i r greater improvement in living standards.
While laboratory experiments suffer from problems of artificiality,
field experiments encounter problems horn inadequatc control of events
between T , and T2.Thurr is always the danger that uncontrrrlled events
rather than the experimental intervention are producing any obscrr.cd
changes.
Howevcr, other factors (i.e, uncontrolled events) should nnt be a major
prob!em if hofll thp exprrimentnl a ~ llzc
d cntllrol groups ore eq~ially~xpusedEO
tlrrsc ollrrr influenres. Total change in the experimental group between TI
and TZ consists of two efements: that produced by the intervention and
that produced by other intervening events.' In the control p u p , total
change n d l be due entirety to the othcr intervening events. I f the same
intervening events apply equally to both groups N e can shll jdcnlity
experimental change by lr~okingat the diffcre?~cc~ in the total change
between the two groups. Ln the carlier examplc (severity of initiation) the
total change in the experimental group was 25 points compared to 10
points in the control group. The difference in total change between the
two groups was 15 points - the experimental effect.
We do run into problems, however, when any of these external factors
influenctl one group more than the other. This can occur in at least two
ways:
I One group may be selectively exposed to additional factors more
than the other group.
2 The external factors may infmgct with [he intervention. That. is, when a
person is cxposed to both the experimental intervention lrnd the external
has a unique effect that neither the experimental
factnr this ro~nhi~nztion
nor the ext~rnalfactur has on i t s own. An example of an interaction
effect is evident in the treatrncnt nf depression. Drug treatment on its
nwn mav have some limited effect on dcprcssion, and psychotherapy
on ~ t own
s may have snme effcct But when the two apprt~achcsare
used together they can have a dramatic impact on depression. The
conibincd impact is much greater than the simple cumulativ~impact of
each approach: it is the mir that has the most effect.
The natural experiment relies on naturally occilrring cvcnts as inter-
I
ventions rather than on controlled cxperimenter-induccd in tervm tinns.
TYPES OF EXPERIMEN'TAL DESIGN
I f WP can antic~patethat an c-r~cntwill occur we can collect information
before and after to monitor its impact. To help intcrprct thr rcsults we
ct~uldtrv to identify a comparison group that will not be exposed to thc
event.
A natural experiment is well suited to s t t ~ d y i nthe ~ impact of
Ic~alizingpokcr machines on family wellbeing. For such an experiment
we would obtain mcasurcs of family wellbeing before and after the
legalization of poker machines. We w t ~ u l dobtain a comparison group by
measuring family wellbeing in a comparable regon, state or country in
w h ~ c hpoker machines remained illegal. Similarly, a nahlral experiment
is well suited to examining tlic impact of the introduction of no-fault
divorce on divorce rates. We would examine the divorce mtes in a
country before and after the introduction of no-fault divorce and
compare the rate of change with a cnmparable country without no-fault
divorce.
Annther form of natural experiment can be achieved by constructing
experimcntal and control groups (Per the 'intervention' has occurred. Let
us assume, for examplc, that wc want tr) establiqh how women's return
to the workforce impacts on the division of housework and child-care
between marriagc ppartners. We would collect baseline information about
the d ~ ~ r n e s tdivisicm
ic of lnbnur from couples where khe wife is not in the
paid workforce. We could rehlrn to the same sample sc'17cral years lntcr
anticipating that some wives will have returned to the workforce. These
could be treated as thc exp~rimcntalgroup while tllose couples where
the wifc remained out of the labour force would constitute the cnntrol
group. Changes in who did what around the home would be compiired
in the two groups to assess the impact of the return to the workforce.
The difficulty here is that the groups are self-forming rather than being
created by random allocation. We might be able to match repons,
countries or farnilics to some extent but matching has its obvious limits
(see Chapter 3). These limits mean that the groups we are comparing
may be dissimilar in important ways. This necessarily compromises om
ability to s a y exactly what is producing any change or any differences
between the groups. In the gambling example the problem will be
finding a truly comparable region in which gambling remains illegal;
with divorce the problem will be finding truly comparable countries. The
housezvork examplc would encounter the problem that the women
returning to work may bc different from those staying at home.
Simpler experimental designs
In m a n y circumstances wc cannot obtain all the measurements required
for thc classic cxpcrimental design (pre-test and post-tcst fur both the
control and the experimental grnups). In some circumstances valid
inferences can be drawn without a71 these measurement po~nts.
 TYPES C1F LXPERIMENTAL DESIGN 61

- - -
I
Method of I Tlrnel(i,) Inletventton Time 2 (T,) T ~ m e1 (TI) I ntervenfion T ~ m e2 (T,)
allacat~onto

-
allocation to (XI (X)
groups Pre-test Post-test groups Pre-test Post-test

-- - .-
1
Random None 'Treatment' Retrospect~ve MeasurB on 'Treatment' Measure on
allocatton outcome vanable matching outcome var~able outcome variable
1,
(experimental (experimental
group?

-- -
Random
allocation

(control group)

Figurc 4 4
None No 'treatment' j

Post-t~stonly rsprvrrnrntnl tf~signuuth n corrtrol Xmup

Measure on
outcome var~able

-- -
Retrospective
matching

(control group)

-
I

I
I
Measure on
outcome v anabls
. ..
No 'treatment'

I Measr~reon
outcome variable

groups are not Formed through random alli~ation any differences

This J c s ~ g nis illustrated in Figure 4.4. So long as groups are large enough,
random allocation to cxpcrimental and control groups means that any
diffcrcnct.~between expesimontal and control groups a r e random and
will not account for group ciiffesences in outcr~mes.Therefore, where we
fi~rmgroups using randon-1 allocation, the pre-test measurement p h a s e
can be ornittcd.
Camphell and Stanlcy argue that:
While the pre-test is a concept deeply cinbcdded in the thinking of research
workers . it is not actualiy essential trr true experim~ntaldesigns. For
psychological reasons it is difficult to give up 'knuwing for sure' that the . .
p s 'equal' brfore the differential cxpcrimental treatment. None-
~ ~ o u were
theless, the most adequate all-purpose assurance of lack of initial biases
behuceti groups is randomization. (1 963: 25)
When this design is utilized analysis cannot be undertaken by examining
t s r f l n l i p between
different a m r ~ r ~ nof ~ the groups. Rather, analysis focuses
on post-tcst differences between group?. Because participants have bccn
randomly alloca tcd to their groups, post-test di ffcrcnccs (In the outcome
measure should bc thc same a s thc difference in change scores of
ehpcrimental and control groups in the classic experimental group.
Thiq design entails constructing expcrimen tal and contr?I groups @el* an
' i n t e r v e n h o n ' (see earlier dacussion of natural expcrimcnts). Since such
hetween thp groups cou Id simply reflect 'pre-intervention' differences.
This design can be improved in two ways using retrospcctivc data
(Figure 4.5).
7he two groups could be nmtch~dso that t h e y are comparable in a t
least some rclcvant ways. Some of the information for matching wnuld
relate to past events. This mfomation would have to be collected by
asking about these past events. The design can also be improved hy
refining the measurement of c/ia~z~yeon thc outcome variable. If we
simply have 'after' measures we cannot see whether, following the
'intervention', one group has changed more than t h e other. To measurc
change we could obtain 'before' mcasures retrospectively and then
. assess the amount of change. So even if the two groups were different to
start with we can see whether the amount of change in one group IS
grrater than in the other.
For example, we might want to see whether having children encour-
ages p c o p l ~so be more politically conscnrative."To achieve this md we
could compare the level of political conservatism nf a group of familits
w ~ t hchildren and a group without children. Since any differences
between groups could be d u e to factors other than parenthood w e would
cc~lfcctntlier infornmt~cmabnrlt the familits and match the two groups so
that we compare like with like. However, we still cannot be surc that
having children has produced changes in political views. To test this
prnposi tion we could ask participants to describe their political vicws a t
some earlier point before they had children. This stratcgy provide.; us
with a pseudo 'before* measure. We could use this measure to assess
whether parents had changed any differently from thc nun-parents.
 "' 2 li , l
-

'
Time 3 (T,i Tine 4 / T , /
nlultiple variation.;. Thcrc may L ~ P qc\cr'~l* ~ l t r r n a t i ~intcrvrntlon.:
t%rallld
tc.
For euarnplc., t\.r may be intrres'led in t h cffcct
~
t.

ot compuI<r)rv votinc
to

qroups Post-IF., 1 Pos!-tesf 2 POSI-test

J
I on electoral turnout 1Yc might con<idi*r tivc tcmditions (1) c o r n ~ l r t ~ b v
voluntary vnbin~,(2) compulsion without sanctions for non-compliancc,
(3) compulsion with a warning only for t11r first offence, (4) compulsion
Random 1 Mrsstrr~ Mealuro with a fine of $50 for non-compliance, ( 5 ) compulsion with a fine of $250
le on ou~cone
allmatron
1 I
vnr~abla
for non-complrancc. hftcr randomlv assigning individuals to one o f f i ~ c
1
(expenmentaf
group)
I
I iYj
grnupswe could ask about their intention t o vote in the next election.
i\sr;uming that voluntary voting is the qtatlrs quo, four of thc fivc ~rt,upq
will be told that voting is gokg to be made compulsor!+ a t tlrc n r x t
rlrction. Each Rroup is to be told a diffprcn1 khing about the ~anctiuns
, Random

1
I:

allocat~on . . >... . , .; 'Treatment' , .. . .,. , ,.. . - , .<. , L : u , r , v r , ~ d non-voting. The control gsorip will tlclicvi, !hat
that will he i r n p t ~ ~for
voting will remain voluntary. After bcing ~ i v e nthis ~nformationprtlpIc
icontro (! in each group will hc nskcd about thcir voting intention. Diffprcnc~sin
, QrouP) thc T? voting intrntions of the five groups s h o ~ ~ reflectld thr c f f t ~ of
t
11
ct~rnpulsionkin thc .;tatid intention to vote. T h i s & s i p 1s d i a p a m m r d in
F1jiure 4.7'.

More complex cxperi mental designs

Wc can add to thr classical experimental cfc5ign by i n b o d u c i n ~more
than two grnilp<,c l d ~ l i additional
n~ prc-te~tand post-test rneasrr rcs a n d
tncflzding morc than one independent variable.
Since the effects of an intervention might not show u p imrnediatcly or
might only last for a short time, multiplr p < ~ ~ t - t e can
s t s help idrntify the
short and longcr trrm outcomes. Fnr ruarnplc, divorce law rrform
designed to simp1i fy divorce may prod~tcca sharp increase in divorces
initially (pent-up dc~rnand,formalizatinn of farnil? breakdown) hut over
the longer term thr divorce rate may d r ~ l i n and
r skabili7e to prc-reform
lev~ls.
.1, multiple pt+xt-tcqtcrperimenial dc\ign i* represented in F~!:urt*4.6.
'The qame dcqign could be modificd tn l>ro\'idc rnulti~lc~ ~ n x - t r ~ .to
rstimatc the evt~lnt20 which c h a n ~ cwmir1~1kingplace brfc~rc.f l l r ~ntcr-
vention to 5cc In whnl rvtrnt the ~ntcrvt~ntion arcclcrat~sor alclws the
1.a tc of change.
Sn far 1 have onlv d t w r i b c d des~cns~ v h r r rthe independent i~ariahlrnr
intcmcntion ha.; twib ra t~eorirs.Hoivc\.~lr,in tcnmen tiunk frc=qumt lv Iiavc=
One problem whcn d ~ s i g n i n ge~perim~nt:: which have befnrc and a f t ~ r
rncnsurement phases i~ that making rn~aaurtmentscan producp chnngc
on its own, regard lrss of anv int~mention.Thiq is the problem elf r!rstrrr-
rrrrnt rt.nctiz~ier.To thc clxtent that change rnnv bc attributed to thr ctfcct nf
measuring rather than the cxpcr~rncntnl 'tr~atrnent', the rt..;ultq will
rcmain arnbiguou~.
The Solomon four-gmup d e i p helps rvalrla tc whcther changc in the
outcome variablr is due to instrument rcai.tivit?; {Figure 4.8). In this
cxarnple I describe this dcs~g-nfor an i n d c p n d r n t variable with only two
categories, b u t t l ~ logic
r of the d ~ s i g ncould casiky be extended to inde-
pendent variable$ with more than two cntrgor~esor conditions.
Using this d r . s i v the experimental gTOLtF is split into W o scrbgrnups,
as is the control group. &e experimental Rroilp im.o!ves a yrc-tcst ~%'hlie
the other docs not. VIP same conditions npply to the ccmtrtd group. I f
thcre is no instnlni~nt rcach.rrlty then t l i ~hvn cupenmental Ertrup::
~t.: 41uuld scnrc identically on the outct,mc mtwasurrsa t TZ,a5 5lloul~d~FIc'
two control group.;. Any difference brtwcrn the T2 scorcs of thtl 1 ~ 1 1
~ \ p ~ r ~ m e ngro~ips
taI or of the two C O I I ~ ~ O I groups S ~ O U ~heC n~ t t r i b ~ ~ t , ~ b l c
to instrument rcrlctivtty.
The example In FEcurr 4.9 illustrates thiq dp.;ign 11singf i p ~ r r s For . tlie
cake of the cxarnple thr outcome vari.tblt* 7.; measured on a ~ c ~r l f~ 11-l r
1L10. In the ewperirncntal group3 the pw-tt~t~ r i ~ n obtained
p I wnrr i j t
;
compared wlth n sccrrc ot 711 for t h e e x p e r i m m t ~gmupl n-ithout tl~cprc-
tcst. lf we r a n assump Ithat tlww werv n o i3tIi~~r d~ffcrcnccsbchvccn the
 I

(14 IIXPERIMEN'l'Al, DESIGNS TY13E5 OF' EXPFRTMFN'l'Al DESIGN

r -- I -
' !
M~vhodof allocation I T lme 1 ll.) Intervenllon Tlme 2 ( f,l Intervent~on Time 2 (Tzh
lo groups 1X)

Random aflocat~on O.
Pre-test

~nt~ndmnn Compuls~on
wrthout
-- - ";
Post-tesl

lntendim Random atlocat~on Mertqurn oo

o u t ~ n w v;ir
* :bin ,
'Treatment'
nil!
Post-test

on
MPRSU~P
Fe
twperlmentaf group 1 I sanctrons (e~penmenlalgfoupl I Y I

I Sb inter Random allocat~on Nnn 'Treatment' Measure on

outclme vsrbabk
(exper~mentalqroup 21
I

I
I0 VC
I 1 (exoeivnanfal group)
1
(Yi
I
- 1 1
Random atlocatron "'a inter Cornpuls~onwllh
to VC modest frne as ) out~~nin I
{expsrlmental group 3) 1 sanctron (control group) IYI
I

--
- -

Ranrlnm ~llocahon
*-I
".ntclr CornpvTsion wlth
I
",~ n t ~ n & n t l
I , P a n - l o ~n!lr,r7tlon

(con110qmup)

- - --
I

I -
, AS, ,I Nr! 'fr~atrnent' b . l n x l ~ r - 02

Random allocatron
Figure 4.X Solr)~trclrrhrrr-qro~q',lr.;rtv~
(control group)
-- I - I. I

Figure* 4.7 I P I I / Iwith

E , Y ~ I P Y ~ I IrEr:<i(p~ I! t ? l f ~ l l ~ ~6)rl lr' o ~ da11d
p ~ nric' !rrrb-lest nnd ottr I behvccn thrl two groups and we may wish to cond~rct prc-tests to
1 7 ~ -l13\t
4 measurp In such c a s w nrc may ntx=rl this more cnrnplcx design.
clliityc.

twn ~rtwtpqthen thc rfiscrcpancy of 10 points bctxwn tlrc hvo ~ 0 1 1 pi sq

onlv nther difference hetween thc. two
attril~utalde~ C Itlitl c f f 4 ~ ~oft t h ~ AFI the r~p~v-imental deqigns so far have dralt with just one independent
groups - being cxposrd tn a prc-tmt. The same logic can be applied to I
variable. Elnwever, wc will nftpn want to cvnluate the effects of a number
the hvn control grniips. 1
I of independent variahlps. W r could, i f wc had the timp and r~sourcps,
Thtl rtvl effect c ~ thr f intcrven tion would be the d i f f ~ r ~ n behvc.cn cc th~. contluct a separate evpcrin~rnt for tach independent variable and
c\pllri nlental qrntlp , ~ n dthe contrrd ~ m u p t inshumi3nt
tvith thc ~ f t t - c of !-ropefulT~rt.rrrk out rvhic11 inrl~pendenti ,~r~nl.rlc had t h t ~rcatcsteffect
rcacti\.~hrcmtl~'ctl.I hi< ~\.nuld~ T V I I- I il~ Score of 70 for ~ I I C e ~ p e r ~ m r n t , a ! But +cpar<~tc' cuperirncnl.; ~ v u u l dbe both ~nt-fiicienta n d impractical.
g r o ~ i pa n d a sctjrc t j ( 513 for the c o n t r ~ dgroup. W r v,'r~zllJconclurlc that Vnre irnycwtantlv, separ,>tp ~ ~ u p r r i m e would nt~ not allow 11% t u w e hoxt'
thc ~nt{\i-\,e~~tirm h . ~ t Ian cttect of 20 poillt~. thc vatlorl5 inclcpendpnt varinhlcs t\.cv-k tr~):kbilicrIn particular minbina-
Yiw might ask, whv htrthcr using \11ch n complicatt~~i d e q i p kc) remove I lions to p ~ o d ~ un1quc1 s c ~ cffractq. An i n d ~ p c n d e n tvariable can have an
thc rfft,ct of pre-tt-\trn~' Could wc3not hnvc achievnd the same outcomrl cffect on it< own (called marn ct'fects or d~rtacteffects) or in ctrmbrnation
b ~5, i n ~ pdI ~ r o p p ~ ntlw ~ pre-test altogctllcr and ct~mparingthe T: Gcnrih.; ~vitfo i t h r r r ~ ; l r i , i h f s( ~ n t c r ~ c t i neffects).
n :\ ni~lrneffect !~,IIc~I, r e ~ a r d -
n rl ~ ) ~ zi pi ~ ~CIdsingle ctrntrnl > ; r n ~ i pt\'e
nt J --ir~cEt.~ x r c r i m t - r ~ t g ~ ~vnutd Icss u t an\-!!jinq else, tlw \,nrj;lblr afttact\ the irutcomc. An tntrtaction
1ia1.r arl-It.cd at e u r i ly the snmr anslztrr Flvrvcvrr, w h w c ha^^ .;mall effect crccurq when the effrct of nnc inclcprnrlrni variable i s at Icast partl!*
Erolly.: mndomi~atirmmay be inruffir-itwt to rernnvr 5~ n. ~. t l nI ll.~ f f ~ r c ~ i i v r ~ c o n h g r n t on c~therhctorx also being prrsrant,
 - -

i Method
,o J
of
groups
allocation
l l ; ; ;-; ; tT 1ntey;rlo
Control (XI )
High la\ l o w lbl

Post-fes~

-.
Random allocation # I Mwn-5rl Treatmen
Male (a) Males Mates

(experimental
II I exper~m~ntal qroups
1s 10 IRO mlnuc 7 0 ) Gender
J
I Thrs IS ~nnllut7hlelo (Xzl
I I 1
I lnstrumenr rrnrtlvtty
Randsm allaal~on *Inm , Treafrnenl' Mean=iO 1
(expenmental
!3rou?J) II
Random allocatron Moan-50 No 'treatment' Mean=60 D~fferenceb~!wc.en
control group< IS 10
(control group) (60 mmus 50) J h ~ s
I rs attr~hulahlalo
1
(Figtrrc 4.10). TIIPdcytlndt.nt variable (1')i < jnh snti<facbi*n r n ~ a * l ~ r1311 t~~l
I tnslrument r~ac11vlb a ycale of 0-100, uvith 100 indicahnq vrrIr hrzh sati<f,~clion. 13nc
Random allccation No Treatmen1 I M~an=50 I i n d ~ p e n d m wriabl~k
t 1.; h ~ r e ocf control o~.t?s (me'<~ t w r kv hh>litin t 2't)
rl<~t\rfi~ <imply
d ~ l crintrol Illr* {ither indtbytbn~lr.ntk , l r r -
ac. I u ~ c~ n high
[control group) i nblr i5 gender (?i7).
I A factorial dc*sign entails forming cupcr~niental grcrupq f(*r c-ach
possible conrIli~;~ltio)~ nf thc two indcpr>ntlc.n! variables. ln this casr the
lour R ~ O U Q Sart' ~IIIKC* defined in Figurr -1,lO.Fvrnbolically thc tnro indc-
pcndent v a r i a b l c ~c a n he rcpresentpd a s )ib[contrcll) and X ? (grncicr).
The categories of vacf-r can be repreqcnicd ns 'a' and 'b'. Thus ma 1t.c tvith
For cxarnplc, J lctd~ol.on its own, nffc-crk mood. On their cr\trn, .myhe- high cnntrol can ht> .;\mboli7ed a5 .Y,,Y:, while lonu control I t m a l ~ s
tarnines afsi) affect mnnd. The effect.; arc ninln ~ffcts:Ithey cnch Iiavc an 5vol1ld be symhnli7cci a s xlbS?h.
effect a n tllcrr own lxut the rnixturp (>f ;~lcc>holand arnphctarninc.; can In t h i ~part~culnrcAsr, ~ i n c egender i~ an independent \.arialllt. hlil is
have unique efkcts .ibovc and b e ~ o n dcithi-r alcnhol or ampht~tamines Iinrdly a n 'intervcntinn' we would start with a randomly st~lcctciigroilp
independentty. This will be more than juqt a n additive effect: it will hc a of males and a randomly selected group of fcma les (Figure 4.1 F 1. '1 ( 1 rach
u n i q u ~effect that i.; only produced by tnixiny: tile twn drugs. Sim~larlv, c-rf these gmups wp would tnboduc~tht* 'trpatrnent' of high and l i l ioh ~ ~
giving workers nlortb contrcbl over t h c ~ rwtirk might improve wnrk ati is- control. Half the r n a l ~ swould bc randomlv assigned tn a low cnr~lrnl
laction for cvcrvnnc (main effect) h t it rni~l11have rmlrr. cffcct nn work environment and half to a high crrntrr~l ~ v o r kenviriinmcnt. The
particular type.; of p w p t e such as pmlr.;siclnslq or older pcr~plr{lntcr- same would apply to fcmaleq. 1%'~ w o ~ i l dtlit~shave fmrr g r o u p fvs
action effcc!). tz-hich we would thvn obtain before and aftcr rncawres.
VIE'~ U T L W t ~ tf.lctt~rr;h euperinit*ntal drstgns is simultant~rru\lt. to The number of grotlpc grnrvs i f tit1it.r vLir~al7!c.has motc lli,in !\to
tl\.~rnine t h i~l i r v ~ ~k'ffect5
t trnaln citvcts) ot a nl~rnherut indcptlntlt1nt categories. It ts ptarl1,~psmure diific~tltto v i s l ~ ~ l l w ~ hr ca t \v~)ulcihappan
vnriahles and tcr b t r * how various c.c>mb~n,ition< OF charactcrrrtit:< wclrk with three or mrlrtxvariables. We c a n c l o thi.; ~vrnhnlicnllvwhprt>wi, h n ~ r
togcthcr to product. ,In effect Thc sinlplrst vrbrsiun of thiq design I.; calIt~d three indeprndcnt variahlcs with n v ; l ~ . v ! nnitnlher ~ oi cati*jirlrEcs:
tht. 2 x 2 factor1.1l dzlsi~n,thdt is one b s ~ c on i twn indepcni4tlnt \.arlablcs I
(two categories c,~llcda ,ind b), X 2 (two i a i t ~ ~ o rc~lIect i t ~ ~ a ,Inti b) nnrl N1
cnch with iuqt twrn cat(-xclrics. Thiq st~nplr.? Y 2 d e s l ~can t ~ . i ~ i be l\~ (t1i.n catcgoric-s called a and b). Tkc firqt catcaxor\- (if the first vsri~l,I(ac~1n
cttcnded tn mrirta than t ~ v ovariablcc. cnch (?I which might I l n v v trvr, t,r bc cvmbolr7ed a s S,", and the second cat(-!:(try nq Sib: the h-I s.ittni:t,n. of
marc cate~oriczrir conditions. tiit' C . C ' T C ~ rndey~nrlt*nt
~ variable i s rcprrwlntt~rl.I\ s7,,; ~ n sod on 'f,lblt.
The simyle 2 x 2 d c s i ~ ncan be ilt~r.;tratrd bcqt with an cu,lmplr* in 3.1 i1lur;trates thr n~lrnbcrof groups that cnuld bc proclliccd by c-nch n f
< nne deikndcnt r.;lri;lhlc
whlch ~ v chtir.c htrn i n d ~ ~ e n d c vn at r ~ ~ l , l catill Zlip p o ~ l ; i b rorn2~inaticln~
l~ nf these tl1rt.c- inrll-ycnrlcnt v r l r ~ a h l r ~
 Summary
M~~ho 0' d?llncatinn fbme f IT,) 'Interv~nfrons' T ~ m p2 (T?I
70 groups
I Pre-fesi
i ( X , and X,)
Posl-lest Thrq chapter has c1ull1nt.dA v ~ r i e h .of c*\pc.rlm~ntnldesign< af v a v i n g
ci,rnplcrity. Ric 10pc o i drawing c~~i.;nl~ n t t ~ r c n c cfrom
wi tli experimental rlr.;igr~swas out1it ~ ~ d
s data obtalnccf
. various es ycri rntantal
Tllc
Random allocat~on X I , x?, Measure on designs vary in term< clt their use uf prc-ti>.;lr,the number of prc-twt and
i
I
I job sat~sfact~on Males w~th tron p n ~ t - t e s trncn.;t~rcrncntpoints and the nu mbcr of groups cc~mpnred
I 4Y I hroh control
I rvitl~lnthe Jcs~qn. 1\11 cxperirnental r l t ~ s i ~ n wilI s ha1.e ccrmp,lrl<on
I -I Rroupr: and all , ~ t t ~ r n p tot make h c grnrips sq similar as pos~iblr,cvcept
Random allocation 1 ~ e a s XI., X1 h M ~ a s u r eon t r l relation tn t l ~ ct=\p,rimrntal inter\ cntion. r\ltlinugh random ~ l l t r c ~ ~ t i o n
Malcs with lob s;r:rc!actron to groups is fhra t;lvclt~rr*dmethod, tht* rnc,In.; bj. n.hic11 Rrt-lLryq tviFE bc
I ICIW control / (Y) I malie romp-rrahlc w i l l vary dcpcnd in^ t ~ ntht. circumstancr< of tlir
I rt%search(see Cliapirr 3). The riguur of r x p r r i m ~ n t a designs
l will vary
d c p c n d i n ~o n t hc ctmlr1uts within which thc cxpcriment 1s conducted.
Random allocation Measure t
Females w~th job satisfac Lnborntosv setting.: provirlc maximum cnntrol of confound in^ cxti>rnal
I 1' . h~qhcontrol ,Y\ influences whilt* n,ltural crpcrimcnts prtn'idc. the least ci~nkol.

--
s I
Notes
, Ranlnn? ~1loc:ttron yli, X,I,
I
MPRFUVon
job satwtac
1 7111s is <I tncnlifit-,?tion 01 an arpr~nt~nt t ~ a < l t . ( E 111 ,In experiment bv Arttnson
(vl
, ~ n dMi115 (1959) a n ~ lrl~\ruh\c~l 111 Kerlingc*r (E'17')' ' ) I )
2 Untorhindtriv Ihr-sr. p>nFs arr not nlways spi'cifi~da t the hrcinning of thta
~mplrmcntationr l f tht* pilot prngr;lrnmr.; snd t~vaEuations t u d ~ r qarta all too
frt.quently cornmi+ tr>n~*tlrr!tr.r tlitl p~totprngrarnmt. has been de\,f,lc)ptd and
~mplrmcntcd.
7 There i~ also r n t . n ~ i r r 4change that 1s dr~tnto various type5 of rrror (see
C'liapter 1).
4 T l ~ targument h c ~ t ilhaf
~ ns pwplc havt. Oi~ldr~n their ccmc1.m It)r thp
wellbeing nt thtair tarnil\ ii~,~L.rc
them brccmir~morr ronc~rnedahnut sr>rialand
pcll~ticalstability.

Ft y l ~ l c k l vbcir~rnt%\ .Jplwrcnt that tht>n~tmhcrot groups rrquirtld can

n cach atlclitir>nal rranabIc and a* ruch of
t y l o d e ~ v l f l it l i t a ~ d i l ~ t l onf
thcmt. 1.arinble5 hri* ;l(ltl trnnnl catcqr)r~tn\.I l f c-o~r r.;c we dn nnt nvrd to
d r a ~ ~the ~ nr ~ c c a r t - h{vrth ctcn. ycls.;~l,lca ctrmb~natinn ~ n c l u J r * ~Fl .r ~ r
thec~rctical and pr;lt-tlcal rtusonG tvc niight juqt wlect ~ a r t ~ c u l . <ct.; j r nl
crmbiriations ~ n l t..raminc. l tlip e f f ~of
' ~tht*<c*
~ ~ t ' l ~ c ct i~) rdn l i r ~ . ~ t ~ o n ~ .
 ~ ~ D C ~ V I / A TM
I 4
CW ~ !T D I F F I C UIT TO
I F r< I A A L I S H TWF KO1 1 OF
OTHrR FAC'TOItS
5
L
The <trcn>:ili of randon11;ratizm is also it.; wcakne-: in rtlnlorlng thta
effect of numerous var1,1bfi1.; i t also rrmoves too much inf(>i+mation.
ISSUES IN EXPERIMENTAL DESIGN While tht. control that mndi~mitatinnprt~vtdrshelps isrrlato the impact
of the ~ . r p ~ r i m e n tvariablt.,
al i t prevents u~ tcqting morp cr~rnplc~causal
rnndcls. nnrking lilt ~ ~ ;lnhintrrvent~on
r has an impnct.
Ihi, d r ~ ~ f b r l ccan
k bc illusttated witli the earlier pxamplc of the
impact uf school type cln student acadrmic nchievemrnt. lrnaginc an
experi nicnt where wc ctrulcl randomly 1' loc;l!ra stude~llstn ~ o v e s n m c n t
fundrcl schon19 and other.; to fee p a y i n ~privtitc school\ qtlch n random
Euyerirnental J e s t ~ n khave bevn u*~drIyused in the n.~turalsclenctls and I

in ysvchnlodcal ;~nilleducational rtlwsrclr and have prc>vidcd ;I ~ r c n t a l l u c tion

~ sl~ould rcrno\*r1 nnv differenctac t r r t w ~ r n thr* twil sets of
students in key variablw such a< initial qtudent abilih. and home
dt-nl OF vaIuablc. rr.~t*arch data. W(lwever, eupcrimt=ntal destgn* Ilnvt.
I backgrnund. A t the end of the stud!! p * r ! r d w e caulrf compare the
br.cn IPSS zvidrlv IISC'LI in the soc~a!sc~rncrs.This lack nf 11scin tht>wlcial
scicnccs sterns irnm rncthodolo~~cal, practical and ctliicnl considcratic>ns. acadcrnlc achicvemcn t of siudcnts in t11r Kovernmen t fundcd schools
with that nf students in the private schools. Any difft.rcnctlx should be
Thi.: is not to say, howevcr, that cxpcr~mcr~tal d r s i ~ n rcannot be zrscd in
onc fnrm or anothrlr in social qcir~nct*research or that the logic of tlw
I attributablt>to tht school.
I Hnrv~vxsr,what we Icnm sbnut actual lift, qituatFc~nsL n . i n v o k i n ~thiq
cvpcrinlental d c ~ ~cannot m bc invaluable in dcl-clop~n.: other rck~aarc.h
* limited. In t h t ~r t v l tj.urlcf rn;lnv ~rthcrfactors ,iyart from t l ~ c
~ ~ L ' C C ~ I I I Ii<
cl~,slcn\that ~7t-I, I ~ ~ ~ p l ~ in
i t tht*
~ L \1wi111
> l ~ sclcnceq. T h r h chapter r a y l ~ ~ r t - s
c the s t r m ~ f h 5and xveaknt~.;.-.r~~
w n ~ nf nf e.cpcrimrntnl d e q i p and ihc
tvpe nf .;chtrc~litself mav contribute ti) tht- ,~ctuaEdiffcrcncrs u7t7see. Yet
issuvs that muht Ilr ilenlt with in J r v r l ~ l p i n ga n y rrwarch that uses thts we hc~\?i* eliminated such rt~lcvanti n f l ~ ~ ~ , l i-c ~e.g, . s homth backgrt)und -
rlt*\i~n.
by randoniization and ccm\ctluently wc carinnt identify t l i r ~ rimpact.
Ever1 if wv dcrnonstrntt>that, despitr qtnrting off thr. same, private
schcwll s t u d ~ n kperform bcttcr than govt-mmen t schovl students, ruc d o
not know ir?lru the former perform butkr. lt'hat i q i t abcl~rtthe private
school< t h,lt produces bctter perfnrmancc7 1% i t trctter tcaclicrs, superior
resources, an environment that emp1insi;lrl; academic yrrh~rrnsnceor
what? A11 that we know is lhat privatc qrhnol students prrfc~rmbetter
7'h~ qkcngth of cxpr*l+imcntalrescarcli i.; the ability it prtwides us with to acad~mic~illy.
sav how much dirrct causal impact a rarinble has. Fxpcrirnenk allow irq Ln summarv, the expcrimcntal design can be effective in identifying
to iudate the impact o f the e.rpcnmcnta1 variatllc. 7 hi< is achlci.ctl tw w h a t i n t c * r ~ ~ c n will ~ cffectice but i t doe5 not help L I understand
t ~ m be ~
randomly allrwating ~ ~ c n p ltoe cxpr:rimental and control groups - a rrhy an intervention is c f l t ~ t l v e .
process that rnakcs the h~oupsccrmmpiirable in aII rc3spccts except with
scp't-d to the expcrimcntat v a r i a b l ~ Hakim
, argues that: R A N ~ ~ M I Z A T I OCAM
N UNI)FlZFSTIMATE TIIT: T O T A L C A U S A L r f F F C T

f'uptlnmcnb a r r appmpriate whm kfit,rr iq wrnc ar~:trmpnt as 20 thr. Randomization can rps~lltIn the undcrcstimasion of thc tnhd uarrsnl
t.\r-tcnce nf n i a u c , ~ Irtal;i!ir>n5htnb~.t~vt,r.nh' and I tnr.! o t clther canfi.\tt~,~l trr bphvefn thc c\ptrimental i~ r i a b l c~ n tht
rplntln?rrl~r~l d vi~lctlmrvariable.
rt,l.lttlcl h c t o r ~ l tir . l v l ~ r nthe F T P C I L ~ ~nlt4rlct!rcof tht. rl/r .111t1 ij~rectv(\not l f i ~ h the rra l world the rota! c n ~ ~ ctiect
~ n l bctlvcaen hvo \.aria14 t- consists of
~ . t t a = t =I \~ n w d v d I lor(, l O F 1 two rrrm~oncnts:

'I hc. kitrcngth o f cxptlrin~cntsgaincd through randnmizntinn, I~owc*vt,r, m dirwr causal effect
cornrs ,at a cnnsidcr,~blccost. Exprsirncmtnl resvarch 1.; not urell witcad t c l 0 i n d i r t ~ causal
t effect.
~ r o \ * i r l ian ~ r l the
n ~i , . ~ ~ l r ~ r r r ~ l of rl r~<ult.i- to identltv~nr,the mechani\nw
171~ \ r l > ~ i one
h rarr;rt~lt~ ,?ffects anr)tt~er.Yor dcw5 i t allow us to b u t l ~ la Ranclomii.aticrn ensure.; that 111e g r o u p I ~ Vcompare ,Jrcm alikc in aV1
rict~rrilof the crrmplcx ~ r tof factorc t h ~ Ft ~ ~ U C ,given P outcnrnr. relevant rcqprcts. \ T r can only look at thc direct eftcct r l f t h r c F r r l -
E\pvrlrncnts fixu.; ltn the imyart nl' ji~rtone or two fnctrlrs.' rntmtal variable. We cannot lorlk a t the rt3nl world cFfrr1.; crf a v ~ r ~ ainb l ~
 v a r ~ a b l envr ~ r ,lnt
, to I?(, \ I I ~ C = t h ~ ilt 1 % d ~ i t t-o tvhat \\.I. t h ~ n ki t is r,ithcr
thin < o r n e t l i ~elw.i ~ ~ 1Vlit.r~the I ~ ~ and I c Gtnlcture (I( a de<ign art. f ~ i ~ t t y
anti f;lil tcr climinnte c-ampcting t'\plnnatiuns nf reqult'i then tkir d c q i ~ n
lack.; ftrkprrinl ?vilrrlit~~.
At1 desiqns !ace thrr,~t<to infvrnal \ratf~i~tv. Carnphcll and StanIcv
( 14t~7)h a w ul~mtifiellthr. thrwt.; disrusscii Iycknw.

I t , in additton to tht* cxperirnontnl intcrvrntion, ~ ~ v c n ttakta s place

1 7 ~W1 C ' C ~prtb- t t * 4 and t>n\t-trst thcr~Ihcsc ckrcnts can thrt*atm the inl(*rnat
i,,ll~drty171 thtb reseal-ill. 'I F~csr. 1.1cnts, r.at11cr tllnn tlw erpc.rrrncntal
inttmrent Ion, could prod ~rccany cliange r\.r. ub~cn.e.The prrlblrrn of
~ u t r ~ ~ n e ovvcntsus is ~ r t a ~ ~ k Ins tl i ~ l danrl natural cuprrim~nt.; in a n
thoqe circum.;tancc+ in 1vhic.11 'in intewt+iition or t~.cprs~mciit,~l \rnriatllc. ~ i z ~ k n c ~ wav.w n In such expcrirntbnt.; wr l ~ n v clesq r-oiitrol ovvr what
works inii ~rcactly h a y p n ' i to ptv~ple.T I I I lack ~ vt r-c~tilrolis a ):rcat dancpr to the rta.;rarch
This st~~lrtcarnrng <,In be ilfustratcd bv rnnsidcring the rr>l,itic>nshrp n q thc timr- I,t-tr~~tcn tht* pfc-te-t .~titf po<t-lt3+tIncrt';lsl'\.
~ I - ~ M . E ~j;tanilcr
II , l r ~ l l rncornt1. lllc ~~~~~~~~t ~ f f c c tt r f gcnilcr rln in corn^. f',\prr~nlc,nl;llJesi!:n> deal wr 1 h such p r o b l e n ~ s17)' using cnntrnl
c~rnsists111 tht~inzcmit- ~ftffercnct,! ~ r h z . ~mrn l ~ n and Ivtlrncn ~vhclarc alikt. qrvups 50 l<>tig t l ~ 1-t?i7ti-~?I , croiip ,IIIL~ t hi> e-xp~~rinit~n!alX ~ ( > L Iarp
~
rtl, ,111 rchbj .ir>trekpri I \ r.\icrt 1ht.r t. 5cn~lr.r I t i.; tht- ct 1,-rt t l ~ gc-nde~r ~ t l~.i* cLp~,!ll~c\pir<o'rl t v f t ~ t . k r , ~rnctrntrc~l[t-r3 c\ tc.tn;ll c\.tan!x thcn tht. cxprri-
111 , I ~ L ] t ~ i it-tnl f - not l ~ + c ~ i td it < l ~~ ~t*ndt-rrnich! I-v 11nLtd
t r A ! 111 <vliicl~ r n t v t , ~ l in ttv-1.1mtion rtbrn,iiti<t ha* trnlv cfiflvrcnce lit-lnvt~rn the. crilups.
M ' I I ~~I ) t l i eInriors
~' s~isli, ~ stEic I , V I N ~ of job, Iravcl clt wcrrkforoe pirticipatiim Altl~olighi l i ~ boccurrLsIlc.tbof su1.11 c*ttcrnal t,vk>nts mean.; thal wtnc a n n i ~ t
t>r cd L P T , I ~1evc.E. I L > ~ I \ y c r ~ n i { ~ n tclcsi~n.;,~l ,lrc gclnd at iclr-nt~t \ring L l i ~ ~ a t t r i l ~ u k~rlr'I lit, chnii\:t~ivt.. c>t?<r-rvr>to thc ~ntrrvcntir~ri luc can still attn-
c-xlcnt c h a direct t - f ' t t ~ t l-~ttc. ~lii+i.r~,rrtIn thc , ~ n l o ~ol
( 3 ~ ~ ni ltl ~ n q rrn ~ h c\pcrirn~*iital
c and controI
The pn4>FV~rr, T~oi~tb\.c~r., i.; t l ~ , ~~ tr n d e <-an
r a l w l1.11.r. a n inclirtt-2 effect - grotilw to tht- ~ntcr\.trntit~n. For caiample cvca rnigI~tc,t-;eme -50 unit4 i l f
via o t l ~ c r~~1rinblt.s. l'or c.tnrnl,It>, gendcr n f l r c k t h t ~tvpc nC crl7~1patfon.: change rn t l r t a trperimcntal Emup ; ~ n d30 1111itu; vf cl~.inplin kt111 cnntrnl
people c h t l t or ~ WIIICII t h t J~i-1' atloweli to 4.11 tcr. Thr tvpe of cu-cupation grotip. If ~ ~ s t c ~ r nC n\ rlc a r ~ dre l ~ clp-ratinq n i ~ dproduct* the 30 unit< nf
In t ~ l mr ~ l t ~their ' ~ t 11,vt'l
~ inct~rnc.50, .is ~melln5 nien e n r n i n ~rnnrc. i h o n ~ in c twtli e \ y r s i r n t v ~ t ~and
l control grtwps the rt%rnainin~ amount
pl~rclyI,rt-ailr<t n t thtbir gendtlr (kirnplt- ~liscriniin~itirm), thcy also cam ot cllsnge in the c ~ ~ ~ t ~ r i m cgroup n t a l (211 u n ~ t s )is ;lttritrritahlr, t o the
more bccau!-t* of isiclors such n.: the tvpr7s of jobs ilirv I~;lvc.,tlic greater c~xprr~rncnt~ll intervention.
likclihoo~l o l wnrLbnq FuFf t~tnc.and n ~ r c a t e rnppc~rtunitv t c l accert I:rcn ~ v h c ~ tcontroli Ernup5 nrru w e d tl~eqe ui~ctmtrvllcd rxtcmal
nr-rrtirnc I f l i p r e 5 1 ) inIl1itnct7~c;rn cause prnl~lcmsIclr c ~ p c r i m e n t sIVe . cannnt be cert.itn that
li we v~l~rnatecl the* impact of p . n d ~ rtin incomt>Iy imly Inc>kingat its both thc t.rp>rimen~.il. ~ n duontrtll Kroups are equnll y cxpnsrd tn thc
direct cflr3r-t wc M T ~ ~wriii~~sl! IM undcrc+!irnnte tlic clv~rallinfl~it-ncpof 11nknow n nrld ~ ~ n tmr ~nlrd ~ l cxtc~rnalinflucl~ces.If, tor qomr rcnqon, the
gthnder o n iniomt.. Iqnilrlng indirect r f t r a r t < has ~ r ~ ~ i t i imylrsations. cal t > \ p ~ i m ctal n crc~upr\.nc mare tlkpoxcd this could accrwnt for ant. pnqt-
I,cndcr d~f!tbrt.ncr. r r i Incornto i5.111 not t-r Irrercnrntn .;lrnplv h i ~ a ! , i n ~ tckt diFfcl.rcwct.+bct~x.r.i.n the ctlti tscrl and r \ y e r i r n c n ! ~1 srnup.; I:urther-
[lien an41 itrc>rnent111- qamc mrlntrv fns t h r , .;amp I L ~ Hccnuqc J. plnclcr I + nlorlT, thcrtz al+o ma1 I-ta ;rn ~rltr~rnt-tr~l~r 11ff~t-t [hat mistsktm for a
litikcd to ~nr-onit,~t~~ljrr>ctl!l, arlIr policv rlt><igntg~l~ L I incrr~ctagenclrr .;~mpledircct cffect L I tlic ~ ~nttn~ vcv~l~im. That iq, tlic tntpr~enticrnmight
r . ~ l ~ t r~~t y~ l ~ alkrl
~ l l i tltv~rl
l trl ;lildrl-.s tht- z ~ i r * c h ~ n i ~ that
r n . I ~ n hzrnrier to h,i\.t. 1t5 t ~ f ! c - only
~t 2~rb~,1uw ~ n ntwr t unknr,\\,n or unintended tavctit n l ~ n
Incrrrnl, I + , .1: t h t ~ IlnL I,c!u.~r-n !:cnrf~.r ,\nd C I C ~ E Itiunrll ~ I ~ 1 1 ~~ n t~ l 1 ~ ~ totqA r ! ~ l i ~ .
bctxvccn t:c,n~lrr ant1 hours t j t t\rt+rk).

In a 5hrdr. r \ f chance wrme of t l ~ c~hangtbut)til~lbc ~lurnto tht* pa-lng o i

l'hr p u r p v w of cltasi):n I < tc- qtrurtllre t h cnllcctinn~ nnd ~ n , ~ l y OF
\ ~ data
> tirnr rathtar tliC\nthc c~\p~*rirncnt;ll ~ n t e r v ~ n l i oFor
n , c*unmple, I 1 wc ~visli
f .
tr, rcacli ~ ~ n ~ ~ r n b i ~ c oi n~ coltur~\ ~ c lI nt ~\vt3 ob<er\.ctyh,lnqr>In ,in ~ > ~ r t c o n l c t o t w t thc ~ ~ m p ~nF
a c~1
t It..1ching nitltf~nclr,n m;lthem.ltrr*.il performance of
74 F.XPERI34FINTA I . DESIGNS
cliil~lrcnw e wotll~i~LIYC'tn be surv tlrat an!' i r n p r t r ~ c n ~ rin
n l perf(7rmancc
is J u r to tlic 1nlcs1cntic~n( t e a c h l n ~method) r;ltl~rrthan b ~ a u s ctl\c
chlldrt-n are firoi\.rng rrldcr. \Ve nctd to c'nsurt8 that w c do not confu~r
thc impact of all intcr\,pnSion for dcvcFnprn~nta[or maturational c h a n ~ c s
that tn k e place in thc coursc of a n cxpcrl ment. Ma turd tional effccts rnigl.12
occur in a study of orwnizational change. In such studies we n w d to
diqtinguish behwwn thc Impact OF the inkolluctinn of nc.w ways of do in^
thin^, t ~ u r ntlic ih;lncrc that arcruc clrnply becauw, vvCr time, peopltb are
in the organization longer a n d bccnmc. more experienced. In a Idboraiory
cxpcrimcnt hungrr, tiredness and t-rorcdom providc further cxamplrs nf
maturation effccis.
S ~ n c cboth the ~*\pcrimentaland control groups should be erluatlv
\lulneratrFc to matur,ition, these c f f c ~ t scan he largcttr removed by using
rL~ndnmi7ed control groups. The l o ~ i cfor this is thc same as nutlir~rdin
t l i t drscussicln of thi, influence of liist~ry.The usc of multiple prr-ttlsts
and pst-tests can also lielp detcct maturation as tht-v allr~wu s to srta i f
c l i a n g ~in the exprrrmrntal pericld ih simply part of an ongoing tiatand
that befrrrp thr intcnrention.
'I'csting participants s ~ v e r a ll i r n c ~ccln contarnina tc rilsults. A pt3rson
may remember prcwious answcr.; ,and subseq~iently answer clucastinns
In ~ c a hway as 10 appear con5istcnt. CompEcting qlrc.;tinns can srnsi-
ti/c people to iqwt-s nhout which thcv had previou.;lv ~ i v e nlittle thnr~ght
and thi.;, rather than any ~xperimcnralintcn-ention, cJn producc change.
Similnrly, darnillaritv with a keqt ( c ~ g .an IQ t ~ s t )can lead to bcttrr
pcrformnnce.
Assuming t h a t tcqting will havc Ihp same effects for both experimental
and control group5 the effect of tcqting can be rcmt>vcd by usrnR a
control group, in thc same manner a< described ahnve. The d a n ~ c rnf
interactron effect< (trsting and the internention combined have a unlquc
effect) is a danger t h a t the Solnmnn four-group desim (Chapter 4) can
identify and enahlc us tn removc statisticalSg. The trsting effects in n
randomized expfrimcn t can also bc clirnfnated by dropping thc prc-tttst
pharr altogeth~r.T h ~ rapproach Is cvidcnt rn the randnmized post-twt
onlv Jeq~gn(Chaptt.r $1.
Mi.as~~rt.d changc c o ~ ~ he
l d due to changes in the way prc-te4t a n d p t ~ l -
teqt nlcasures are mnsle. C h a n ~ i nt l~~ cquestinn wt~rding,the rcr;pon.;tt
alttlrnatiucs (e.g. add in^ a 'dm't k n t w ' uption t t l attitude qir~~tir,n.;),
thc qurqtion sequt3ncr o r the data c+oIFcction mc~rir.( e . facetrl-fnce
~
inti*n-1t.w at 7, and telephone tnt~rx-icwa t T,) ctruld all account fot
I
mcawrcbdchange. A different person collecting thr infijrmntion a l ~ uc.ln
ISStiFS IN EXPBRTMVIFN 1 AI. DESIGN
contarnina tc the proced urc, as can thr s,lmc pcrwn bt-cnm tng morc
experi~nctdand lamiliar with collecttng thv data.
Providing careful training and s p t t a r n n t ~ c instructic~n.; for t h n v
collecting data can reduce this problcm. The introduction of conhol
groups and the use of the same instruments fnr botIr expcsimcntal and
control grnups can su k t a ntiall y reduce the prLihlcm (31 instrument
decav. Fvtn rf there is instrument decav, its ~ f f e c tcan bta rcrnoved by
compartnK rilfiri7nc~s hetwrcn lthc exycrimrntnl and ctrntrtd groups in
the amount ot change nn the outcornc variable.
Grnup5 or people who register c.;tremc scortbson a n c7utcnmtbmpasure at
. o r IOWIF)artx likely to cchngc to a more rnrrdt~rati=score
TI ( e . ~high by
T7.T ' I S~t,~tisticaI tcndcncy r ~ t h e trh a n an cxpcrirnental intcrvcntion can
account for some changc.
Statistical regression occrirs hecause ,111 xcorrts are LIP i l l part 20 chance
variation.;. An t*xtrernelvh i ~ or h low TQ hcorc is morc likrlv to contain an
element of r.~ndomcrrcrr than a moder,itr ~ c r r r The . odd< art. against a
pcrwn aillrc\ rnq a n rutrt~mt.5ctjrc ~ > I !v,rr I clccasipn.; dut, 20 r,~ndi)m
error, Th(-rcfrrrc,the odd< ,trt- tlint a fair prnporhon of tho.;r people who
initially nchii.ued a h i ~ hsil>rr due to wnltx ran~lurncrror will nnt be
subjcct to thr~same randonr crror a scccrncl timrb. Accnrdin~lyin a second
test t h ~ i scnrc
r 1s likeIv to bc less extremr th;ln it initially was.
Stat~st~cal regression cc,ln alsn he d u e to the fact that rneasurtm~enterrrrr
can only change extreme .;cores in one dircclit~n.On a r;calt. of I to 10 a
score of 10 can onEv get In;r+*rdue to measurement crror. A qccrre of I can
onIy gct Iti,ylrt,r. The bia.;, thrrriore, of rneast~sementerror bnwd change
wlll br to ,I more moderatu wore.
By u t i l i ~ i ncontrol
~ groups the researcher can deal with this problem.
So long a< people with cutreme score< arc randomIy allncatt-d to the
control and crperimental ~ r t l u pthe ~ , Eevcl of statistics t re~rc.;sionshould
be t h e samc tor both grntlps
rf hvfr grrllrps are initialIv di ff~rent,pmt-trqt differcncc.: bctwern groups
could hc clur to this in~tfnldifft=rcncc r;ltllrr than to thc rewarcher's
i n t e n c n t i n n . Rtnsonatrlc 4 r c d randcrmi;lcJ cnntrol ~ o u p k
sllcluld elimi-
nate -clrdinn effects. \Vhcrc randomlzatinn is not Seasihlc, rnalclling can
clirninatc ~pccifiedselection effects but thr' pcfisibili t!, ol ~ i n n ~ a t c h r d
criffercnccq influencing rcsult.; rcrnains (scc Chapter 3).
If an rxprriment is cclnductcd over bimc a low of certain t~pr.qnf parti-
cipant<,miller than an c x ~ r r i m c n t a lint~rvrntinn,may br r~3qponqihtefor
 any m ~ ; l * u r t - ~c lh a r ~ ~ Tt>r
c . elarnrlt., w c mislit mc.;lsurcl thc acntftmic ~vorld.li thc crpcrin~cntalp < ~ ~ t i c i p , ~ n t sdifft.sr~*ntin irnpr~rtmltways
are
qradrs nt n z r o u p ot qttldcnt~a t TI <ind c a l c ~ i l , ~ thr. t t ~ ~ r t ~ u~ yv c r , ~ ~ c . tram the 'rral ~ v ~ ~ r prriplt.
lil' fiw wt~rmitllc intcnentjtm i$ i ~ ~ t ~ m w(# ~lcd
\Ifc t h t v ~ n ~ y l r m t m a nt cuyr~rfrnrntalintcr\.t.ntion nvcr a pcr~odc?f vrar cannot l7c surc n'hat thc outcome 1rr11l be. Fnr e\arnplr, w t l rniyht usr
~ n dt t ~ nrtrneasurt. ~ ~ r h m n a n lrl '~e . m i ~ th clb.;erv~ a n i n c r ~ a s pIn
I f~rst-yesru n i ~ ~ r s l qtudents
ty to twt tht* eSfcct of s t r i n ~a film portraying
1
,3Vcr,ljip ~rarfe5.11'011ld thi\ Iw due tn Zhc in ttbnornttr~n? Pt~sqrbly.But j t ~ r a p l l r croad accrdcnb nn d r i v i l r ~bc~haviour.Can w r be ct~ntldcntthat
cc~ulda l w ht>dur* In thtb Icss atlltn people dropping nut nf thc prclqramme. anv effect on thi.: voung and prcdominantlv middle clasr ~ r t will ~ ~ p
li Iht. It.;< abh* ~lrclyptndoirt then thtl i ~ i s r t > ~rn, ~ saver,?cc
r mark< c o u l ~ l apply to other typpr; of prlrplr7
s i m ~ l vb r b c c , ~ ~ lhe ~ s r! ~ q hc3tllc ~ c c r rnot be in^ included in the calcu- I rliere are two approachtls to d e ~ l i n gwith this prrrhlerrl. Onc i 5 tu
Int~rbns;it T*. ct~nductexperiments ii.;ing large rrprcstlntative sarnplrs and to analvsp
7hca lv\t t\,<~v of Jt'illlnq w i t h the impact c p f tlir low of participant< i c to rt.wlt.; tu wtLif thcv ll~lldP ~ ~ I for E ~ alE ~ ~ V nf ~ c c ~ p lIn
~ type< e . the ah5tlnc~of
minimif? ~;t,ltact~r.~c d~t)pr>~jI. 'lrhr n n t bts! t l i i n ~is tcr rtnnlrri-t.11, toffprt , ~ t r;lnrlnm l y selected rvprrsrn t,iti\'c <Girnplcst hr* Iv.;t htrategy is to rr+jlt9n!
the n n a l ~ q i s.;tap-. In the alx)vt~~.xnrnplc*wc colrld rcralcul,itr tF1c~ 7 , I
the crpvriment manv times with diftcrcnt sarnplt>s anrl in diHercnt
niftlr;l~c's I L W ~ for tliv s t u J ~ * n t st h t tcmained t h r o ~ ~ g l i o ~t hl tr s h ~ d v . 1 crlnteutq T h i ~i~ cnllcd rr~dit.ntrn11.[f we koep g c t t i r l ~tht. same ri*sult.;
?\lternnti~*t~l!~, ~ v h c r c~hi.rc*i~ t.ibidcncc of ~ e l r c t i ~dmpclut, c~ w e cnit!d with d i f f ~ r c n tvyrs
t elf ss rnplc*~ w e can be n w r c ionfidr~ntthat thc rcqtillq
t v e i ~ h t h ~nrnpEc5
~ to alljird for qclrctivr dropout (C'haptrr 'j), intrtd~lce apply ntclrt. ivirlrly. If,however, wc fail ta rt*plicatr. tl-rc rcsuits with
qtnh.;tic.al conhnT+(Chal>ttrr123 ur cnndtict an.~lvsiswparateEv for v,iric~us p a r t i u i a r lvpe\ nf pc.opIi- or under particular conditlonq I! rlnr.; not
ql~hqrcrup.; I rntm;ln that the prm-erlurc is fi~ndarncnfallyf l a ~ , c d .R,lthcr, i t ~ n ~ ~ b luqe ' to
:
spccif! t!ic l ~ m i t snt our ~mc~rali/atirin and the i!'pes nf p13ilyl~to whom
thib FCCIL(I!+ do an<! ~ l 1n1 0 f , 7 p p ! ~

The notion 01 cittarnnl i m a l ~ d i tr~tt'i-1;

v f ~ the
) e\tc,nt to ~ v l i i c lr~t r L-JII
I
p ~ n c . r n Ethe i ~ ~re<ult<oi t h t,\perm~cntnl
~ .;ample to t l i r p c ~ p ~ ~ f ~ ii lt ii~
on
mean1 tn rcprcbstltlt.Wr~uldt h e cxpcrirntnntal ink-neent~onwork In a rcwl
world qituatlon nr zs ~ t l ;etfect only p i n < to occur i l l a n erpcrimcntal In ~ n ~ i e a ~ o u r to
i nachieve
g intr.maf ~.,ilirlitt-and trl attcnd to etl~icaland
svtting t ~ i t ha p~Ttic111.1rS C ' ~of participant<?Cnmptirll and S t ~ n l c y(1W3:
C practical m a t t t w we can encl up ~ 7 1 tan h crpcrirnihnt h a ! i.; cleganl ar~cl
5 4 ) rdcntih a nuniht~rof factors that can u n d p r m ~ n rour ab~litvto ethical hut is .;o artificial thnt w~ learn n o l l ~ i nof~ any u.;e ahout real
I

g c n e r , ~ l i zexperimental
~ find i n ~ bc.vnnd
s thc spccihc c~xpcrirnrnt. PCRPIC in n*al srxial contcuts. T;ljft,l (11W3) r~mrnii*115 that the ~ . p i c a l
s r r ~ a psycholog!l
l expcrimcnt is hnscd on

a t ~ m p o r x ycollt-ct~onof late , ~ d o l ~ w stmnp=rs

~nt glrrcn a puz7le to wlj8e
I f pa-tcstiiig itqclf clr,~nj:esbehaviol~ror nttiti~dcqor ~;cn.iiti~es pct3plc so
~tntierbi7nrrr cnnJi!rnn< in r7 I t r n i t r d frmr <luring their first rncrting while
that ~ h r yarc mc>rra\.ulnclr;lble to the eff~ctsof the intcwentinn, h r n any being p t ~ e r at
~ l from trvhii~da rnlrrnr (In Hnhirn, IqSh. 11 1 )
crperimtmtal finding.; may trc. .in artitact rrf the r~prrimental~itlration
and not apply to p c n p l ~who do not participntc in ilir c~xperlmwit.If the
prr-test ra111t.r than the intrn~rtitionprrldr~crsthe cl'Ccct then . l h ~rffrrt I Simplzr he in^ s c l t ~ t e dInr a < f ~ i dcan i ~ makc propltl fvel special and pro-
\TITI nclt rlcr-~lr111 spa1 w j r l d irnplc~meli~~itic~nof a n intewtan til>n\lnct5 lhc duct* , ~ r t i f i c i ~rcsnfts.
iI : If~rnotiq ~ t ~ r cof
i y the \Ve<tern E l r ~ t r i ci - l a ~ \ e t h o r ~ ~ r
~ P J Ir\8r~rldcotltrvt trill not includc t>upr,rimc,ntal prc-tvst.; \\'arks. rn U h i c a ~ oin the t.,>rll; lC'.30< Hrst d~scovercrltht' irnpnrtancc of
In s~tit~ltlcms cvhrrr, tht.rl, 15 07 rhancc ui prp-ttst l~ffwts,i t i~hcst t c b what I1;15 L-r~.crhrncknown ,I< the Hnwtliornc effect (r\nt.thl~sht-r~er and
~ t i t ~ pa t~ 7 w 1 c nwifh(?utthta prt7-f~%si rdias~-, t~ 1 1 s ~ ~ ~ ~ I " I P !"~i~r-
t l ~O !I Dickscln, 1970). TIM. rc.;rarcherl: rtllecttd a erc7up o f worhtr.: and cha1icr.d
g r o ~ ~tIr<lcn
p 5n t h d t thc tnipact t ~ !th!.; tl pc l r f ~nt~r;lctlnt? can Ilr, the16 working rrn irc~nrnr~nt in an cffr>rt t r ~irnpro\*ra~sr,ductiiit\.. T!WY
m d tnh-n Into account.
p ~ t i r n c ~ hand ~ l r t y n I l e s s of which rntc~r\cnt~or~s
f 6 ~ 1 n that tht=v made, thr c r t ~ ~ r p
becrlmc more pro(!L I C ~ I V ( ~ . cone!uded that it was not the intcrvcn-
TIICV
tioils that rtJerC cf!cctir.e - it W A G being ';ycciall\. selvcted tlint produrt3d
tht* cFt\>ct.f iacl tire sarntbc h a n ~ r sbwn nl,lrle r v i ~ ! r o r ~glr t Ing people (he
tctaling that they w e r e qpecial we cantlot bc ronfidtlnf that sat~sfartiona n d
,, ..A ....&...;I.. ..,....1.4 L...,...,.,.,,.+
Practical issues
Thcrc arc fCc.rv simple ans\trers to thc many practical qucstiom h a t .iri.;c
~ v h c ndesigninq sclci~lcrperiments In most cases the answer tn thc
rlwstion i<, 'It depend$.'
Sht~uldwe tdl participants they arc in the control group? HOIVm ~ ~ c l i
sltotdd wc tcll them about the experimental hcatment7 Should thpy
know whnt thc research hypnthcsi:. is7 W l ~ i l ctllc ethical principll- c;f
~nforrncdct>nstlnt (scc Eattcr in thiq chapter) i;ug~eststhat wc should trll
pnrtrcipant\ r v e r y t h l n ~clning so could makr the studv worthEes5. I f
prticipants know w l ~ a tw c are lorlking for they might bchavt. in such n
way as 10 cither confirm c11. sabotage our exprctations. T~lIingpcoplr
t h t ~are in a control group or crplaining tllr range of c~perimcntal
intt.rvmtinn+ ( c . ~the . rLingeof income support 1 ~ 1 ~ ~in7 thr
s parantrcd
income prr~jcct)may make them ftubl that thcy a r e missin): out on t l ~ r
rlcsirable in tc*rv~ntronsand creak rtlwntments ~ n non-c<,opera
d tion.
'T1~t.r~a r r n o hard a n d fast rulr.: about h n ~ much
. inlormation t o
prtn.ide part icipmts, 1711t within thc rcquiremcn tr of informcrl conspnt i t
1.; hafest to nlit11mi7et h r ;ln~onnt of dctailed information prrJi.ided bta!r>rc
llic experiment is complctrd. This minimize5 the chances of con tam in^ t-
~ exprrimmt. Snmctfrnes an
i n the involves cxplici t deception
to avoid sabotage. fn such cases ethical guidefine.; and cart-tul debriefing
nt~t-dto f i ~cr~~pu10ut;lv
~ adhered trr.
Thr importance rve attach t ~ ~ i c x t e ~ validity
nnl nnil t h t apprnnch we takt* to
external validity will have a Iargc bearing on thc* si?e of fhc snmpte w c
chot~se.If wc want to gencrali7e staiiqticalfy to A wider populahon we
n t r d to use a s;lrnple that is both randomlv srlccted and b i enough
~ fmm
rvhich tn gtrncralize confidently. Many bonk.; provide Formulae for
wnrking out tlir sample qi7c r e q u i r ~ dfur statistical generdi7a tion (MRHT
and Kalton, 1W1; Kaltnn, 19R3b). If, however, wc use a replication (sce
above) to achieve generalization the critical factor is whcklier we h.lvr
rt*plrcnted t h t ~rvperirncnt a sufiiclrnt nurnllcr of timw ,~nrf under n
qlrfficient range n i conditlc~nc;.Thrwrctical cnn.iidtrationh a n ~ icornmcln
scnsc a s to tlitl rangc rrt conditions under whicli we attempt rrpl~cations
muqt guidcl 11s.
i t our main c m c c r n 15 iteith intrrnal ~ a l i d i t vnrc can manage :e\.ith
much 5rnalIer .;nrnpIeq than are rcqt~trcdfor ~tnt~.;tical gcncsalization but
thcy must st111hi1 large rnnufih to dctrrt an cff~ct.That 15, thr sample s i ~ c
musl be largr rnough to ~ i v ethP cxper~mentstatistical 'pnwer'. Pro-
crd irrcq for rvnrking out tht* 5arnpTc q i w requircd tu defect t*ff~cts can he
lr)trnd ~ p J ~ r \ v ) ~(Cohrn,
rre IW7). Thv <ample .;l?r must a l w take into
account the IY,IV in which w e \rill arl.llyse thc dnla. difft,rt*nt n ~ e t h n r l q
rt~~1uii.e
diffcrtwt ran~plpsi/.cs.
Extcrnal validitv is best nchicved by choosing a snrnple thnt i.; rcpresen-
tatijbrof thc iv~clcrpopliiation 10 which we wiqh ko extrapolate. A rcp-
rr.;mtati~esnmplc is bcqt gained by u.irng random s ~ r n p l i ntrchniqut-=-. ~
T h i s requires u.; to select p,?rticipantr. in such a w;ly that each persnn In
the wider pnpi~lationhas ,In equal or a known prnbabilitv of bcinfi
.;tllt=ctcd. Prnl?;thilitv thcnrv provldrs IIG rt~ithn wnv of estimating how
Ilkrlv our rehlil!~are t o I~clkJfor the* \\.idrr pclct~l;it~irn fnjm \\.htch thtb
4,lmpIk' MIRG \t'lt'Ct~d.
Unlortundtrly, even i f wc use random sampling we will not necest;-
a l+ilv end up wltll a reprcsc~ntatir,csrrmpI~.It can bc biased btlcaus~w~ go
, ~ b o r ~our
t rcqt~archin a n ethical i v , ~ \ ~kt. is uncthicnl ~ i t l i c rtu prcswrrl
I potential sub~~r-tq 211 par!icipatc or to try to prcvrnt them withdrawing
from the study nt any stage during thi. researrh. TIIF ~ x t r n tof the bia.;
thnt rcsults will depend on how manv potential particip,lnt\ rcfusc to
participate, 01. cirbsequctifli \\,~thdra\v.
Hnrv long st7ould the ~ l bc p hehvccn ~ntrodt~cin,:an intrrvr~itinnand
the post-test? T ~ lPe n ~ t hof the gap muqt depend on \\+hat our thenrip4
I and previous rttsrarch 1c.d uq to expcct about the time it ~vnuldtakr for
an cffcct to hr dVtcctab~e. The length r d the gap bctwcen intcrvcntion and
poq-test will "st' depend cm whether we arp looking for shnrt term nr
Ennqrr term c f f t ~ t s .
The gap h ~ t w c c na prc-trst nnd an ~ntrrventionshould bc as shorl 3.;
pohsiblc. A hhillrt gap is dmirable bccai~scit minimizes the c l ~ a n c( ~~ f
cvcnts taking place befnrr the intewrnlion that might makc the cxperi-
rncntnl and control groups different in some rvav betore tltc intervention.
Horcever, wc may sornetinlcs need to I ~ a v ea g;lp ~f n7es u s p t ~ tthat the
p r c - t ~ s t might interact with the intcrvention (c.g the pr~=-testha.;
sen<il~zcdpnrticfpants to iqsues).
In t-qtablishing the 5 i 7 t b rrf thc tcnip\ral gap hct\rven the prc-tc~tanvf
thc pnqt-teqt \ Y V must bc mindful c7C thc impact r3f thc Eoss nf participants
1 and its thrcnt ti3 both intcrnnl and c,rtrrnal ~ a l i d i t y .Sincr drnpnut.;
incrt~osethe Iongcr a studv I,ists. wc ntscd to sct thc gap will1 ,iview t o
n~~ nmizing
i t h r ~ roccurrcXncc.
I-lcr~\,ever,tcx) short n time gap can prodt~ctbn h e r mct h(rdnloqical
prot>ltms, IThrrra the gay Eiiltween prca-test anrl prlst-trst I < ~ h c ~ ~t r liq
J li kt-lv that participant.; will remcmb~artheir prr%-trstrcsponscs. Thcir
rntlrnt,ry mav ~nfluencethth wav thrv respond in thc pn+t-tc+.;t phaw nf
thy (.uperimt.n!.

Such rncrnory effects can he dealt with in a number of ways. Pilot
twting might cstablrsh the amount of time that has elapsed before
parhcipants forget their pre-test responses. Alterna tivcly, if we have
allncatcd participants randomly and have a sample of sufficient size thc
pre-test can be eliminated altogethct (Chapter 4). A third alternakive is to
use the Solomon four-group desipjw. This design overcomes the problem
a s it enables us to assess and statistically remove pre-testing effects.
Practical matters will also determine time gaps between stages of the
study. The amount of time availabIe to complete the study and the
availability of funds at our disposal to keep track of p e o p l ~will influence
the way we finally design the experiment.
Which mr!hod of data cnllectic~n?
The particular research design we adopt does not dictate a particular
method of data collectinn (e.g, observation, interview or questionnaire).
The data collrct~onmethod we adopt will depend on the nature of the
ubservations wc need to make (Chapter 1). What is critical is that the
method of data collection produces reliable, valid and meaningful data.
Since the expcrimen tal design relics on detecting differcnccs over time
and dlffercnces between groups it is critical that we have measures that
accurately and reliably reflect change. We must be confident that meas-
ured change is real rather than just an artifact. of poor methods of
informabion collection.
The need to achieve high reliability does not point us to a single data
collection method. The reliability of a method will d e p ~ n don what data
are being cnllcctcd and on the skill of the person collecting them. In some
cases we will achieve much better information by watching people than
by asking them highly structured questions. Sometimes a less structured
questionnaire that enables an interviewer to probe and follow up
responses might be more effective than a highly stnicturcd one.
AS well a s yielding reliable data our data collection method must
produce valid information: we must be measuring what we think we are.
There is little use in collecting mliablc data tl-rat are wrong. There is little
merit in cnllecting data using very reliable instruments on, say, level of
racial prejudice i f thc method of data collcction is getting people to
express socially desirable views rather than their real views.
Tht. practice of randomly assigning participants is one of the distin-
guishing characteristics of experimental design. For a number of reasons,
however, random assibpnent docs not necessariIy achieve its goal of
ensuring that the exprrimentrtl and control groups ,are initially com-
parable.
ISSUES IN FXI'ERIMENTAL DESIGN
LIMITS OF RANDOM AI.I.OCATION
Choosing a small sample can impede achieving comparability of expcri-
mental and controI group^. For example, randomly assigning a sample of
10 people into two gruups of five people each may easily result in
dissimilar groups. As the size of the total sample increases the averagc
charactcristics of the experimental and controI groups should converge.
Wl-tere the sample size cannot be increased, the ramdnmized block design
can be used to ensure equivalence on specified charactcristics. However,
with large groups there can still he important differences between the
groups on variables with low variabiIity despite randomized assignment
(Cook and Campbell, 1979: 342).
GATEKEEPERS
AND THRFATS TO R A N D O M ALT.OCATION
Random assignment can be difficult to achieve in field scttinp where
practical realities may require negotiating with authorities and other
'gatekeepers' who control access to potential participants. Among the
gatekeepers the researcher is likely to come across are school princ~pals,
manngcrs of organizations, and professional welfare workcrs. Gate-
keepers can cause problems by insisting on allocating people to groups
ti~cmsclvpsnr by undermining the research by poor implementation.
They might, for examplc, allocate khe 'most deserving' to desirable
treatment groups. PdeaFly the researcher should allocate people to groups
but, w h ~ r cthis is not possible, close scrutiny of those doing the alloca-
tion is required.
If people believe that they are in a control group they can fccl badIy done
by. Tile resentment that results can lead to a high dropout and produce a
control group that differs in important ways from the experimental
group (e.g. more negative). These perceptions, rather than the effect of
any interventions, could be responsible for any final outcome differences
between the p u p s .
One solution is to disguise the fact of being in the control group by
using a 'placebo' - i.e, an irreievant treatment. In dnig research the
placebo might be a sugar pill. In social research i t might bc a treatment
that is known not to affect the imtcorne variables and is thus safe to use.
Alternativelv, wc can assure control group participants that they can
have the deqirab3e treatments, sl~ouldthey wish, at she cnnclusinn of thc
expcriment.
A comparable problem can arise when people allocated to a given
tr~atrnent refuse to participate because the treatment is seen to bc
82 E X P E R I M E V A t DESIGNS
undesirable. Drnpaut a t tl~lspoint would mean that despite initial
mnJom allocation the groups could be different 17efurf the in tervcntion.
This prrjblcm can bc avoided by poqtponing random allocation to
groups (Cook and Campbell, 1979: 358). Having recruited participants
we can ascertain their willingness tn participate in the control ar any one
of a ~iurnber of different experimental trcahnent~. We could then
eliminate from the study participants who were unwilling to participate
in any particuiar group. Then we could randomly allocate the remaining
willing participants to the various groups.
Since experimental designs examine change over time they face the
danger of differential dropout from diffcrcnt groUFS ( c . g the less desir-
able interventions). This differentidl dropout could be responsible For
any post-test group differences (see earlier in this chapter). This call be a
serious problem in long term, field based social research. Furthermore,
even where the dropout rate i s much the same across all groups, the
drc~pc~uts may differ significantly from tl-rost whn remain in the experi-
mcnt, thuq tlrrcatening the external validity of the studv
A kev thing to check for is whcther dropouts are dist~nctivcin relation
to their initial scores on the outcome variable. For example, are those
with low initial s c r r c s on thc outcnme variable more pranc than others to
drop out e~therbefore, during or after the experimental intervention? rf
they are mnre like1y to withdraw then differential dropout rate between
groups could be rcsponsiblc for post-test differences. If participants with
Inw scores drop out this will automatically boost the average score of the
group, even i f the intervention itself did not produce the boost in scores.
Apart from minimizing dropout the next best thing is to identify the
characteristics of thosc who drop out and then adjust for these charac-
teristics a t the data analysis stage. We can use the information collected
at the ptc-test stagc to help identify these dropout characteristics and
then inctude these variables in a multivariate analysis to see if group
diffcrcnccs would have emerged if there was no differential dropout.
In field based expcrimcntal sncial rcscarch both political and practical
considerations reduce our degree of contrid over the intervention. M1hcn
we rely on a variety nf people to implement a n intervention (c.g, a
di ffcscni person might be responsible for carrying out intervention in
different orcanizations) we may not be able to ensure that h e inter-
ventinn 1s consiqtently or properly imptemented. When the intervention
is uncvcnly carrlcrd out we dn no! necessarily have a good test of the
impact of the intended intervention.
As far as possible the investigator should maintain Sight control of the
wav In which intertren tions arc implcmcnted. Without t h ~ we
s cannot be
ISSUES IN GXPERI MENTAL DESIGN 83
confident what interxention has actually taken piace and we may thus
have difficulties making sense of the data.
In anv rcscarch there is always the danger that the researchcr's expec-
tations and values will inadverkcntty distort the way he or she collccts
and interprets information. For example, i f we cxpcct that a partictllar
teaching method might improve tlie level of children's classroom
cooperation we might kste' more cooperation than if we did not hold this
bellef. A researcher's expectations can affect what they see, how they
~nterpret it and h o i ~l~rrrdthey look for cvidcncc contrary to their
exprctations.
In drug cxperin~entsthis problem is minimized hy using a 'double-
blind' approach. Nci thcr thc parfirfpan t nor t h ~investigator knows
which drug or placebo is being administcr~dto which participants. But
this solution is rarely availabie in social research. Because of the opera-
tion of thc ethical principle nf informed consent (see later in this chapter)
it can be difficult to usc placcbo trcatmtntq. Similarly, it1 many social
cxpcri~nentsit iq vrrv difficult to hide i ~ h i c hgroup5 arc suhicct to which
intervention.
About tlie closest cvr can get to thc 'dr~ublt-bImd'approach in social
research 1s to cnsure that the person collecting the data is unaware of
the researcl~ hypothesils. But cvcn if this condition is achieved the
rcsearchcr's values and beliefs can distort what they see and how they
report what they sce.
We can take two approaches that can reduce the danger of only seeing
what wc want to see. One is to develop clear operational definitions and
research protocols that reduce distortion of ubscrvations and interpreta-
t~ons.'The secol~dis to use multiple 'judges' and observers so that the
different nbservers can act as a check on one another. By using multiple
r~bserverswe can identify the extent to which the patterns are observer
dependent.
Ethical issues
t~egarcllcsso f the restarch dcsign, social research shnrrld conform to four
broad ethical principles (Kirnrnel, 1988; Hornan, 1991).
A well-established princ!pIe of SIK~RI r~scarchis that people should not
be requircd nr led Lo believe that the!! are requircd tr, partic~patcin a
study. Furthermore, participanks should know they can withdraw from
the study at any p i n t . We shuuld avoid people gaining the impression

from the c o n f ~ din which the request is made that participation is
required. In situation.; where there is a 'captive audience' and where there
is a power differential hehvcen the researcher and the potential cuperi-
~nentalparticipants (e-g. first-year psychology class, welfare recipients a t
a welfare agencv) the researcher needs to be especially careful to stress
that participation is truly voluntary.
The surest way nf ensuring that people understand that paltiripatinn
is voluntary is to explicitly tell them. For example, at thc beginning of a
study participants can be told
',2ltho~~ghyour pdrticipat~onin this study will be greatly valued, you ,>re not
required to p,trhctpate Ynu can stop a t any point or choose not to answcr any
particular questitm. lust let the intcrr~icwerki~ow.'
Voluntary participatioil can produce a number of problems. It can
thrcatcn the external validity crf the experiment. Since certain types of
people (e.g. those with lower levels of education, those from non-
English-speaking backgrounds, oldcr pcople) arc more likely than others
to dt.cline ti) participate in studies, vol~lntaryparticipatio~~
can p r n d ~ ~ c e
biased samplcs. It can also threaten the internal validity if we get differ-
eiitia l dropout in the experimental and contrt~lgroups (see eaxl~ersection
on problems with randomization). In addition, if voluntary participation
leads to high dropout rates it can lead to too few cases for meaningful
analysis.
Howcvtr, compulsion is not the answer. Although cornpul.;ion might
reduce sample bias, unwilling participants can undermine the quality of
data in a study. Since we can statistically adjust for known sample biases
it is best to maximize khc quality of responses and encourage voIuntary
participation.
Tt is better to cncuurage rather than require people to participate in
studies. There are two main ways of doing this. One is to appeal to
people's altruism by pointing to the study's possibIe benefits. PeopIe are
remarkably generous and willing [n participate in studies where they
believe it will d o some good. However, in our enthusiasm to recruit
participants we must avoid 'false advertising' which overstates the
benefits. Second, we can appeal to pec>ple's self-interest - hnlv parti-
cipation can benefit them. We might offer rewards fnr participating or at
least compensate for their time and anv costs they rnEght incur by
pnrlicipating. Somc studies provide token gilts and offer to provide
participants with a siirnrnary uf the main findings.
Infnrmcd consent is a close cousin of v d u n tary participation. Typically
participants should be informed about:
ISSUES rN EXPERIMENTAL DESIGN
3 the purposc of the study and its basic procedures
2 thc identity of the researcher and the sponsor
3 the use to which the data might be put.
They should also be provided with the following:
4 an outline of reasonably foreseeable r i s k , ~mbarrassmentor dis-
comfort
5 a description of the likely benefits of the study
15 a description of how they were selected
7 an offer to answcr any questions
8 a statement that participation is voluntary, that each participant is
free to withdraw a t any time or to decline to answer any particular
queshon.
While the issue of informed consent seems entirely reasonable and
desirable it is not always straightforward. How much infnrma tion should
we provide before a participant can be considered informed? How firlly
informed should participants he? What does it mean to be informed?
Providing more information need not mean that people wilI bc
better informed. Simply providi~lgdetailed descriptions of the study
does not mean that respondents will be a n y mom enlightened as a result.
indeed, detailed technical information may confuse, distract and
overwhelm rather than inform.
Furhermc>re, providing details about the study - especially detailed
information about the study d c s i p , about the hypotheses and thcories
we are testing - can distort the way people answer questions and
undermine the validity of the findings.
One solution is to provide baqic information and to offer to answer
further questions. If this information could distort the way people
b ~ h a v ewe can tactfully explain that this is a concern and offer to deal
with the questions nftw the experiment is over. h some psychological
research participants are deliberately deceived since accurate knoudedge
would invalidate the study (Homan, 1991: 9tiff). Where this is warranted
it is critical that participants be fuIly debriefed after the study.
'This raises the question of when consent should bc obtained. Most
ethics ctmunittccsz require that consent be obtained hPfnre participation.
However, a case can he made that participants do not really l u ~ o wwhat
they are agreeing to until they have participated, and that they can only
provide irtfi?~wrrrlconsent affu they have participated.
- w!10 should give consent? Who should give permission in research of
young children, the intellwh~allydisabled and others who may not be in
a pnsitinn to fully understand the implications of participating in a
survey? Participation still ought to be voluntarv but consent may need to
be obtained from other pcop,plc such as parents and guardians as well a s
the particip~nt. In hierarchical organizations gatekeepers may grant
pcrmiq.;i(m tcrr illnior p r t ~ p l eto yarticipatr and t11115,111 w m c srn<c, t h l i g ~
rtlnior p i ~ o p l rkc7 ~ n r t i i i p a t c In
. .;~tch si!uation.; b i . ~~ h r w l t lrln.;ilrrn thal
~ . l i nct,n*ctit frtm both rhc qatck(,eper.: anci thc ;lctu;ll pnrticip,~nt\. :'in o h v r o u ~w n l 117 r\.l~icl~
participanl< onn Ilc- hnrnic.cl i.; h i faitlrrr t,
hunclttr prrlrniwc of C-~niidcntialilvPC-oplrp r t i c i p ~ t i n111 ~ ,r ~ p t ~ r i r n ~ n t s
~ l c u p t ~ t h ~ they
,ire r r ~ t i t l c tti t cannot bth idcntifred a< the ~ r u r c t of . any
yarticu1nr infflrrnation. Frcqucntlv, rm-inl \cicncv tbkl-rurimmts 2,ntalF thp
collecting of personal informatinn which, t f rnLidr rublic, souId be
~ . m b a r r ~ i < sor i nhurnilintin~
~ o r carz<e liarm to p;lst~crpant+In one way o r
In sc>rnc t.xpcrimcntal studie., particirpants arc pcltcntiallv evp~lscd tn another.
Iiarm. !'or k>karnpic, in medical rlipl*rrmcnt.; in which a new d r u ~ic; It 1% r<s~-ntial tlicrtafore t h , ~ infnrmnl~nn
t bc collrcte~lin ruch ,1 w a y that
tria1lr.d. plrticipants at'c pi~tcntlally~ n d ~ l n j i c r r [n d . ~syrhrdirricalruprri- c t ~ r l f ~ ~ l r n t ~coin l i f vg ~ ~ a r a n t ~ . cI!d .i.: ,~l+trtasqcntial l l ~ l tthc pcbol-rlc
~ ~ lw
r n t m t s parlicip,inl+ r n i ~ h !be given stimuli (11. bc iniillcrd 10 L-rc=I~n\re in invt3l r c d ui ctdlrnctin~, an;llvsin~ and rrbportitig tllc find in^.; r w p c t
I \ ~ ; I ~ Gtfial thy? later rt*grr%t ;~ndfind di+trr>ssing.' s u r u p u l ~ u s l vtlic c i ~ n f ~ d t . n t ~ nrcq~~ircmtantq.
l~tv
14'here thvrr- ic ati! d,injict' uf harm trl pnrtic~pantqilic p r i n c i ~ l caC In clrdrr In pn>tcct partlcipat~tc,data muqt be '.tortd in *i!ch 1' way d 4 t13
inklrmt~c!cc,nstxnl rrqtrlrc>s that partiripan15 \-retol~l( r f thzb dancer< hefort, Frcclutlc ;rnv (rnatlthiwi7ed ni+cc.;s. This may rncxan tori in^ data in I r ~ k e d
p,3rticiyatin?; in [hi. etpclrimrnt. 12'htnrtn thc.rt5 is ~@teI?ti~l ior h.rmi , ~ n d c . ~ b ~ t i c(lnd
t s rrvnovin): ,111 ~ d c n t i f v i ninfomatinn. ~ If thc data an., <torell
tl-11. p ~ ~ i - ~ tst111l n agrLbtJ:. t o particiy.itr tll~re is the nccll f(3r c;rrrt~ll elcctrt3nic'dllv thrn ~~a>st\.rrrds and r t ~ s r ~ , p t i canc ~ nhcIp t.nl;ilrr p r i ~ a c ! . I. t
d r b r ~ c h n,lftcr ~ thc c\pt,rimcnt. D ~ l p ~ n r l i non g thr- typc o f pntt.ntlal Ii,irm is h ~ g h dt-siralllt*
l~ that prrsonal ~dt-ntikyingtl~tai3qGinill h ~ rtlspnsr\
thrrc. m.iv hr the n t w l for Irlnl; trVrrn ( C I I I O W - L I ~ t o t*nqtirtA~ h , T~R V t rt*l;lt~ns10 p s r h i i p ~ n l shc kcpt s ~ p a r a t i , l etlforrnat~on
~. about s pt8rs(1n's
p ~ t c ~lallv n t li,~r'n~iul ctfrac-15a r c n~on~trrrcrl. natnc ;lnd .~iliirr~.;\ s h o u l ~ l1-c 3torctl in i r t i c lltt, ,inti 6i'11 Ih(brr ,jtv-wer. la
Lnlikt> o t f ~ r dp.!gns,
r nianv t~~pertmtlntal dc-s~~n.: ~ n ~ . n l \tht~
. c invt>sti- cl~~t'ctir>n> <lioultl I-re .;tot.~tlin n ~ i i i t i ~ n ~tilt,. n t Idcnllficaticrr~nurnb(*r\ ftbr
gGitt>r a c t ~ \ ~ v ldclin):
v w n i t > t h ~ n potcnl~allv
q hnrrnlul to thtl p r t ~cipantq. c r n \ ~ - r t 5 t c r ~ c n cm~uns~t bo ~ t n r e t li l l quch a WJ\J thtir onlv ar~tharizt\d
IZ~li~lr ~t <hrlulrl IT ob\.irru.: t h a ~n1ci n l ~ rriol
t s u b l t ~ pcvplc
t to p ~ ~ f r n t i a l l ~ ~ pcrlplc ran rn,lkc tht* m , ~ f . r h i n ~ .
11.1rrnl~rlinter1 entions. i t <ecrn.; rrsnw~rableto ~ c up t rl\ytr.inicnts ivhcre I'rtn'iilinq .w.;urnncc\ c ~ fcnnfiduntialitv ir irnpc>ttar~ttor mett~ndolcrgi-
thc inlcrvc~itinns~ ' 11 i lwnefi t the participants. F L t~rc~~nlp!t> ~ t ht. g11~r- c-al a5 ~vtlEas ctllical rt-as~~nk. Ii p r t i c i p ' ~ n t s arc. confident that t h r ~ r
antrxetl income c ~ p e r ~ m e n tcli~scril-red r; in Chapter 1 ct>uld bc just i f i d respnnws are* truly corifrdenti,~l (or et0en better i f thcy arc a n o n v m o i ! ~ )
1>ec'd11r~ Ihtx intrr~~cnticm of prrlr-tding a ~ u a r a n t w dincr~rncworllrl rmlv \vt, cnn expect that pcoplc art- m o w likrly tr) p r l i c l p a t c in tlrc + t t ~ d y-
benefit parti(ip,~nt<(Fl.lktrn, Icr8h: 102). ilspcc-ially 11 i t IS al>otrt private matter.. 14't. car1 atso p*pt~ctthal ~f a
t lolvc~rr,thc a r p ~ m e n tt h ~ ,in t inten.cntion w ~ l lbr lrtnpf'icial pre- pt3rstv-i fei*ls that thtir answer.; ;ire trulv confiil~ntia!thrv xvi ll be mtrrc*
*umes that we know thc outcomes of the intenfention. We rn,y trrltrrrr I~hctvto provide k ~ n and k honrst answer?.
that rnconlc maintenancp w ~ l l at , wnrst, bc harm!css, but ra7do not A ~ T L J ~ I J one .;hould not milkc prclrni<lbs nf ct>nfldcntialitv un1i.s~they can bt*
t h i s rrl~t-~rd clt tlwrcn. Even wiih the t,r*st o l intentions thcrc c a n be kvpt. In a stud7r nf 5clioc1l chi2iirc.n v i l r l might Icarn ot thc idpntitv o f a
irn~ntendrrl harmful r.ffi>ct.;. 'Therr it; albo thc isst~eof 'beneficial to drug puslrrr nr a trachcr tvhn i\ 5cw11allv rnolcstiny: s t ~ d m t s In . a qtudv
whom?'. l-lakitn ( 3986) ncltcs thnt thr. income maintenance e r p c r i m c ~ i t s nf pri~cmcrsyc>u might Irarn of plan.; fur ,ln c s c a ~ cnr of thr identitv OF ;I
led to an Incrcaqc i t 1 t 4 i $ r n ~ ~a<e ,SI)MC' wclmrn bt.cam~.financialt!~ morp F ~ S Wwho I ~ murdered an inmatc. In such castt.; vnu fL)c{.thp. pr~>blcm nf
secure. 1-hi+ ) r r n t ~ 1ia1.c I~t>nt.fittdthr wclrncn but thc husbantls (.jnd ~ v c ~ i q h ~Imcturn g prorniw OF confidrntialitv n~;lin<tthc harm olu<ctl to
ti IIL!r ~ n rn;r\.
) 11n1.ehtrld i.cbrl.il iffr~rmtI irrvc c3tl1i~rst-r! r c < p c r t i n ~that From1.t:. 'il;lLr. qurc !ll,it \ - c ~ td<r ~ ntlt rn,lhr
\2%i1n thtare artv perrri\.cad bt-r~cfit\t i ) parhcipatiriq in a n t.uprirnc.nt ~ ~ n r t v l r s l iprttrni~t.+
c of sr~nhrEc.ntiali~
I P . ~ Jn . incr\mrnmaintcnnnce ru~vrimcn!, rnvdical trial\) t v r m;ly FRCC t l ~ t b In promi\ing cnnfirlrntialil\ bc ,1\varr o f the I c p l rami t i c .rticlns r vr>trs jf

I r V n - i t c I : I>t'~~Cfici,il trtbdtnlcbnt frrrni yrornlw. Drytwding nn tt14. ~ r ~ n ~i lcl 9% \ lh ~ l claf.1

i are rirllectr~l,r l ~ t mar.
- ~
t l l ~C O I I Z T ~ ~ ~ EToup. IS it tdif 10 dr'nv w>nie ptlcryle Zhrn polcntkal brnrfit tw w b i ~ ~ t (t ~fret.~lorn ~ l ii n ' t o r n ~ ~ ~!t?fii~l,it~i)ti.
t ~ i l ~ ~ In nianv ci1untr1t.i
u f ttic trr-atmr~nt' Ran~lurn . ~ l I t ~ c a t ~can o n rrducc sum<, 11f tht* ~athrcal: rc<~>arch d,~tac t ~ l l t ~ c twith
t ~ I tht. g u a r a n t c ~o f confi~let~tti.~lity dn not r n p y
prot~lrm.;in thls rt*~3rrl.Wc can ~ l w undrrtaki~t o prnvi~lethta crln~tbl Icg,~lprlrilt.~t,,inti crrufcl bv st~l~piii=n,icd cimrts, C;IVPR*.~tch ~ i t u - LI

pruup M.I'~II ~ C C ' P < Sto th{' tre~trncnttmcr wtnknc~wit h a s brbcn succesqli'r~l: a tlon 11 15 ~mpc>rfant to clLlritvvnur r i ~ h t >to rnaint,iin thc rontidc~ntialitv
(or ;l m o ~ l i f i c a t ~ uofn tht\ trtvlmtnnton the h,~qi.:d wlint wnc: lrarnetl from of d ~ t awhrn wiIrk~ngIn Rnvrrnmctnt agcnclcs or under n ctlntmct
lhr c ~ v c r i r n e r ~ t ) Contracts Frr-q~lcntlvmakc cup!~ c i trtnt13rncntq nticnrt tho owner.;hrp ot
 , ~ it 14 irnpnrt~tlt tc, encure tIi,?t tlirasr- prtlr Fsintl- dl3 not corn-
d ~ t anil
promise ~ O I H .~111dvrt;lking'i
r e ~ a t d i r usonfidrntialitv.
~
6
ANALYSTNG EXPERIMENTAL DATA
F.rpctimc.nfat tlrsigns arc valiiablic for isolating the cau5al impact of
qlv*cific x . l r i ~ h l ~1-f~rs
r . 1s rkpcclally thr case n-hcn the!' are cot~~luctcd in
hl~V11vcnntrollcd environinent~.The structure. of the dr.;ign, ~specinlly
tht. t ~ s cof ~ n t d cc~ntrcllgroup--, help< nlaxirni~ethe int(=rnalt.alrditv of
crpuriinr>nt~l rr,.;enrrh. Hnwc\~ci.,the nrttficinlity of s i ~ c hrcsearch and the
il,lrrorvnr~+r l t t f ~ dtlsigrlr
r often rne,ln that, fnr w c i a l w-tencr researil~, T11t.r~.lrr6 I n r ~ e~ l u r n b ~t i tr ~ t a t i q t i c a ltcchn~qurkthat a n 111. used for
~ u p p r i r n ~ ~ ndo51gn\
tal y1ul~donly Timitc.il informatinn th,~tfrequently has a n a l v s i n ~expr~r~rnciital dnh. Thc purpose of this chap1i.r i s to p r o v ~ d c
poor c~tilrnalvalid~tv.T h i s chapter ha.; disrussed come of thc threat< tn some g~liilancrin stalcctinq from amcmg klir.;e mnnv diflertxnt M-a\.< of
~ntcrnalJ ~ ruternal J validiv 111 rcscarch and ha5 rlutlincd thc w a y s in , ~ n , ~ l y s ~expt-rimet~tal
nx dntn.
which c~xperimentaldesims wcrcclrnc t l i c ~ cthreats. In ~ w i scirnce ~l -1-he c-rvre r r f euprlrimcntal arr;llvsi< is thc c i 7 t r r j ) t ; r r q r l r i of grrl;rp<. Dn
scwarch p r n c t ~ ~ a. ~ l n d rlfhical rnattvrq nl-ck mtaan l h , ~ tcuperimcr~tal grtwy.; that rllffcr In tcrm- t,f t h r indrpendi~ntvarl,~hlealso dl t t ~ or1 r the
rlclqi~ns(,Iten ranntbt be used. T l i ~ schnptcr lids crttt1inl.d t h c s ~practical l ~ ~ n t no grcruyk that hnve diffcrcnt ?~sc,-ltrncnth'prrltrrnr
~ l t ~ y c n ~v;lr1dtrlr.7
, ~ n detl~icalis\tres. J ~ i t c r r n t l tnn the rrfitcomil varlnE1!c7 In a rrrll-cltls~cnerlriuprr1rnpnt the
rinly t i c r t . n ~ bc!n,tacn
.~~ Illc g r i n ~ p s~ I I u ~ I ~\lt>
L ~ thtb diFt~'ri'~lt*~~t'~tl7ll'llt5'
4(, rvhzt-h Ihtby a r e t~\yost~tI Tht. morr IVP can be hi1 rc t F i ~I nrtr !;roup ?re
Notes similar cxcrpl in tcbrmq of thc indcpt*ndcllt variilble {tn'tervt*ntion)thc
mnre Lr't, can be crlnfidrnt that thy grtwp d i f f ~ r t - n c e4m ~ thrl nutrtlmc
1 Tlit+c irr.n;Cnt--~c~:,d r a n ~ l r ~ n ~ i t , ~ t.lrtT
r n n rnt~sl evitlrnt with s ~ m p l e variablr are duc tcl the infcrvent~on.
tauperinuhnltnlt l r s l ~ n srn whrr-13 Ihtbrr I\ onlzo nne rndcpendtbnt varlablr 'Tlivv In m.lkrn~comynrii;on~thrcrx thine.; nwO krr hr + - ~ c w c .
can br rr.~luctl~l .;ornPr\.hat In marc cnmplrv twp~rimriitnl dr+im<that 1nt.1ude
\twcral indrpt.nrlcnt variables (sets d~scus~irm nn factrbrrill d p r ~ ~111i sChapfrr 4)
1 Thc comp.~risonfcq-u<~.l;nn ,yrLvrp - nut indi~'idual.;.
mrnt dt.partnwnts) I ~ . ! v c~ r i - t ~ l a l ~ o nctlrics
at rornm~ttrrsM ll~cfimuqt appr0rre 3 M7t*nt)edurayrpf workingri~zthow r n ~ s u l i d i f f t * r ~thercnceds
nc~ tobc
pmject~lrnm a11 etY71r~:per*pvcti~,~~ b ~ f i > rtunding
*> iq rtblcawd for thr str~tty hctwpr-n c~sperimcntal and control grtwps bt*forc i v e attrtbu It, a n v
? n l r \l~lgr,~rnt,xper~rnt.nbcln r~bcdrmcearta c l a ~ ~ rexample< c of thr\. Fee irnprrrtanctl to thc cliff(-rencc* For instance, ~ 2 . rm i ~ h ohsr~rvr
t qrnall
TXn~rrnrrnd(1 Oh3) and Milgram (lW14) bul inlportant dilfcrc%nccf;ur we might rrhscrvc substantial differc~iice<
that might be q~rnplvtFue to sampling and rnr<asrarcrnentrrror.
3 Wr compare grnirps in terms of thc outcome variable.

s~letfingthe method of analysis

s ancllv~illqr,spt>rirnrntal data. Rir rcingra

'Tlicrc ,Irk1str itrnnv t ~ . c h n ~ q u elor
nt tt>cl~n~q~itb< is bt%cau*rlof thta ran!:t. of type< nf \.arialde< .~ticlrli\'crw
tt>rni< t h t ~h t a lliaf ,arr1 cc.lltbctctI To ;llialy.c e\pCrirnvntal C I ~ In'ra ~ ~ ~
nc~d t o match tht= mcthral uf ;l.nalysic to tfic ch,>ractcr~.;tic~ c l t thc tlata
Lj~l FXI'ERllwPI'N'I'rl I. DESIGNS

rt,ttibtic%cstablt~li11 t'tic tind~ncclor n sample can bv cxtrapolaterl t o thc
i r r m n.Ii~chthe s,irnple is drarrn. For c*uample,1r.c might
i t i d i /iol!~ilnf/oclli
~ best expta ~ n r dtzfith thrctl c+nr~rples.
f i n ~ l;Irnt>ngrnernl>cr\ of the samplr that peclplt n . l ~ rarc ~ unenipln\.~d
h n \ ~ nrnrse
i~ hvalth than employed pi*oplc=.How liktnly is it that thr < a m p
link L-lotween t~ncmpluymcntand hralth would hc fnund in thv widcr
o n tllr <.~mpleis meant ti) rcpresent? Inferential statiqtics
~ ~ o p l ~ l a t ithat
prcn'ide a ivay of t-.;t~mntingwhcthtxr %amplepattcrns will hoSd in tlie
r\.lrfcr population.
Idrally, anal?+ s h u ~ ~ lin~.oF\~c
d both descripti\fe and inferential
aiinly-;is. We shollld carefully cicscribr thc c x l ~ n to f any diff~rcnrcs
bratwccn ~ o u and p ~tl~l>nwork out w h ~ t l i e rany di tfcrcnceq arc Ei k r l y to
Ioi~ndin the widtbr population.
Infrrcntial statistics can imIy be uwc.d to generalize finilings if a ~ ~ o l r n h l i ~ ~ ~
.WIII/I/~, 2s used. A probability sarnplr is one nf two hl+oad types of saniplr-s.
'Tlic ~jthcris a nun-probability sarnplr. A probabrlity sample is nnr in
which t ~ pcrscln h h,lr an cqual or at Icaqt known c l ~ ~ i n (probabi!it\.)
ce (>f
. nbt.11nlngsuch J -,1rnp1e 15 to r ~ ~ i i l t ~ ~ n l ~ ,
I>r.inq \clrxctrd Jhc- % ~ r r c . ittv a ~ ot
~ r a l t dsnmpIc rntmhcrk from a knt)wll and dciincd pcrpl~lation.
1Ylit-n w r hnvc probability snrnplt* we can U R Z ~ Ir*gitirnatelyKcncr-
illi7i' I11 the spt~ciCicpopulation I r t m which the sample is d r a w n . A
pmhability s a m p l ~o f a country which systematics llv cxcIuded prnplc
.~gt-clunder 18. thaw withnut telepl~nn~q or thosc living in institutinns
rt,n\uld yietd findrngs that couId onlv bta gvneral17cd t c ~the populofic~nof
nt>n-tn$tih1tirmali7cdadults rvith a tcycphnne. Tf an cvperiment I < bawd
cln n prohabilip samplc of students in a f i ~ t - y e a rpsyclinlogy class In ,3
particular irniversitv then we can only confidcntyy ge~~eralize to this
pop" la tion (unless wc can demonstrn tc that t h ~ s rf~rsi-yearpsychology
studcnts at the particular unir~rsityare r~presentntivcof some ividrr
pop11Inti~~n).
11 12.t. d o no! hn111- n probability .;ample thew iq little point in w i n g
inlcrrntial statistlcs t r l generalize to snmc wider populntic>n. We <lio~~lcl
limit n ~ ~ r s e l v ctos iicscriptive statistics that sumrnari~c.patterns in that
sarnpl~.Many rcsrarcliei.~are unablc t o recruit ra~irlornsample%.Npvcr-
thrlcv- they continirc 10 irse infcrmtial statistic7 inappropriatelv in thrir
. I ~ , It ~ t psample=< arr not fea5iblt~thc m w t apyroyrl-
I < i t . IZ'herr prt~'t~al.rl
; ~ t vrsrnl 117 test for ~cn~'r~ili7ab111tv is to rcpcat the tl\pt~rimentin diftrrc~nt
rcrntibut% a n d wit11 rl'rftcrcnt sarnplrks.
I t 1 \tatiqE~calanaf!,.;ih thr. l~-i,cl of ? t r r , , r ~ ~ / r r ? ? of IS crltical in
l r ~ lvilri,~!llt'~
I 6.1.
wlc.chng particul;lr .;tatisticar analiw~q.There are thrW main Ic1r.1.; a t
n'h lcll ~ ~ a r i a b l arc
w rncnq~~red: int{.rvnl /ratio (alkn c a f t ~ dcontinuc~u.;),
nrdinal arid nurninal (a'tscl called catc.grv+lcalrrr rlunlitatii.~).t 1it.w can he
An interval/ratio \rariablo is one in which thc categories can be ranked
tmm IF;(* far Irryl~in some* meaningful npay.In add~tiontcr ordering the
cntcgorit\ Isom Iaw to I~i~ch it i.; p s i b l c to sprcliy the ot~lotrrrf~irtl~fiert-rrrr.
hhzqeenthc cntcgyvirs. Age, when mea4urc.d in ycars, is, an r r a m p l t of
an inkerval variable. Wr. cclla rank order the catc~orimof agc ( y ~ a r sfrom )
voung~qt(loweat) to nldeqt (highest). Purthrbrmorr, we c.ln specify the
a r n o ~ ~ nzrft d iffcrence brtwccn the catcgoric..;. 'rhe d ifftwncc bchz7cc.n
the ciltcbgory '20 years old' and the cat~gor!. '25 yens< old' 1s 5 years.
i'ariablt.~ \uch a s the wceklv number caf litwrs of paid rvork, I@, height,
weight, Income [in dollam) Arc. all variahlr.: where the c n t q n r i ~ sare
numcric and ~ I i r r thp e ~ntcrvaIs bvhvrcn thc cntegories can bc \pccificd
psecis~l!~.
An ordlnsl \..iriaI,le is c~ni,n.hcrc eve can rank order cateqrrrr~..;irnin low
tn high. I-lnwcvcs, WP cminnt specify in numcric terms Irtliv ) l r r i l - l t diffcr-
cncc thrrr i'i l~c'kwecnt'nc c,itr);nries. For crcrl~npfc,
when thc variable age
has catcgorirs quch a s 'child', ',~dolcsccnt','young adult', 'middle aged',
and 'elrlt~rlv7tht.variable i.; m~asrrredat thc* 0rcl inal level. Thc catcgclrics
can be nrdcrt=d from \-ounpa.;t to eldest but n v ccannot sp~cEl\~ prt.ciseI!*
the age ~ a hptween
p proplr in different cate~tjries.If we mva~tlredl e \ ~ I
of workforctl participation as full hmc, part tirne and not in tlrr labour
force this vnriablc would bc a n ordinal variahlc. If we aqkcif I l l r precisf
numbcr of hot~rsworkd tach wcck the varial3le (hours of wnrk) woufrl
he m e a q ~ ~ r catd the interval level.
A nominal variable is onv whcrc the different categories hnvc no set rank
order. With a nnminal ~arialllei t makes no setwe to sav tIi.lt the cate-
gorieq cnn t3c ordcrcd tronl tcnv ti1 high 117 .;clmc qenw. I<vligiu~ts affrli-
ation is a non~innlv a r i a b k wFit=r~i v e can d i\tin~uish t r t t ~ u v ~cnt~goric.;
*~i
oi afhlintir~n( c . ~Ic~visli,
. I<ninan Catholic, /Irthrxln\, Prt)t~,.;tn~it, I4arnic.
n o rrliginn). TI1c.s~catc~trrii.* ha\^ n o nnttjr,~lrank rrr~frr.
_ When sclr\cting a mdhod c>F data analysi.: we must first j~ltbntifythe
level of mcaruwment of c,lch of the variablrns in our analpis 'TO work
out the Icl L'! 05 mcasurtlmpnt rrf an!. variill,lta, ask the c~utstirrri..in Tablr
In sel~ctirlgapproyriatt>mt*thoclsof a n a l v i < the level of mt*nsurcmfnt
nf IatJr tht. grotipmg v;lria'l.!r ( ~ n d c p c n d ( ~ onrt i n t e n ~ c n t ~ ~ m > ) the
and
92 EXIIEKTMENTAI, DESIGNS

Aw tlirre ditfesenl calegnrres' Ye$ Yrs Yes

Can I rank the catt.gortc\
from Iow t c ~h i ~ h ' Nn Yes Ye.;
Can I spectfy the amount
nf difference between
t h cat~gories'
~ No No Yes

nutcornc variable are relevant. Thc wlcction nf particuIar methods u~ill

depend (in part) on the combination of levels of measurement across the I ~ i m 2e Score on outcome variable
two variables and on which variable (dependent or independent) is a t
which Ievel of rncasurcment. Figures ti.5 to 6.8 demonstrate how this
affects the choice of statistical methods.

Grnups are compared 111 tcrms of the outcon~evrlriilbIc. These corn-

parisuns might examine the amount uf prc-test and post-test a31angt in
Lhr experimental and control groups or the absolute differences between
the groups at the post-test stage. There are different aspects of the
outcome variable that can form the focus of the cnmparisons.
~ F F E K E N C F SIN CENTRAL TENDENCY
Gmups can be compared in terms of their averages on the outcome
variablc. Diffcrcnt typcs of avcrages (mean, median or mode) can be
used depending on the Ievel of measurcmcnt of the outcome variable.
The most widely used comparison technique is to compare the means
of groups, but this rcquircs that the outcnmc variablp is measured at the
interval level {see below). A t the descriptive level this simply requires
comparing the means. However, thew differences in the sample may not
hold in the population: thcy may simplv bc thv rcsult of sampling error.
We could use an inferential statistical technique to work out whcther
thc difference between the means is like1y tco reflect a real difference in the
popuFa tion or is simply due to chance (sampling error). To do this we
couId use a technique such as a /-test, an F-test or analysis of variance.
Unforhnatelv, avcragcs can hide ~mportantin fcr m a tion. For ~xarnple,
we can get idenhcal averages between twn groups yet the two groups
can differ sharply. The example in Figure 6.1 ~llustratesthis point. In this
example the experimental and control group5 are identical a t time 1.
Each person in both groups ~chicvedan identical score on the outcolne
variable ( 5 ) and the average for each group was 5. At /ime 2 the control
group remains unchanged and maintained an average score of 5 . How-
ever, thc experimental group has changed a great deal. A11 individuals
liavc chdnged: half havc much lower scores and half have higher scores.
However, the average score for this experimental group remains at 5.
The averagcs of the two groups at kime 2 remain the same. Consequently,
a simple comparison nf the two averages without tooking at other
elements of tht. distributions wnuld lead to a false conclusion that thc
intcrvention had n o effect, since the averages reflected neither change
nor differences hetwccn experimental and control groups.
DIFFERENCES
IN V A R I A B I L I T Y
An alternative way of comparing groups is to measure thc degree c ~ f
diversity within cac11 of the grnups. Arc some groups morc uniform than
others? Do stlrnt. gr~mpschange to become morc uniform nr less unifc>rm
than others7 The impact of an intervention may be not to increase the
scores overall but tn m a k ~a grnup more uniform nr more Jivcrse (a5 in
Fig~irt'6.1). Wicrc arc statistical measures such as sangc, variance and
~ t a n d a r ddeviation that prnvidc sumrnarics of the degree of uniformity
Ihnrnogencity) in groups and therc are teqts which measure whcther the
lcvcls of homo~eneitvin groups differ. Graphical techniques such a s
histograms and box and whiskcr plots can also be used tn examine
whether groups arc equally diverse, Which trsts nf variation arp used in
part dcpends on thc lcvcl of measurement nl the variables concerned.
 EY I'EIIIMEN'I'AL DESIGNS

w ModoIify Distr~buhonscan bc unimodnl, bimodal, nr multrrnsdal

(Figure 6.2a). An experimental intervention might change a
unimodal distribution to a multirnodal distribution.
Symmefry Distributions of ordinal or interval variables can be
symmetrical or skewed ( F i p r e 6.2b). Skewed distributions can
be positive (casts clustering around the low end of the outcome
variable} or negative {cases clustering around the higher end of the
outcome variable). An intervention might change the symmetrv of
the distribution.
Kurtosis Distributions can vary in terms of how clustered thcy are
around thc ccntrc of thc distribution. This is called the kurtosis
(Figure 6 . 2 ~ ) .The distribution can be very clustered or peaked
[a) Modal~ty ( b l Symmetry (leptnkurtic), bell-shaped (mesokurtic) or fairly flat (platyk~rrtic).

Thcsc cliffwent ways of thinking about distributions all provide different

ways of examining the impact of an intervention on a n outcome variable.
High Rather than Focusing narrowly on the impact nn the average score of a
High group we can look at other dimensions of the outcome variable to detect
an effect.
,,-..,
Platykun~c ,;.:': 1 *,
. ', '..~esokurt~c

.. Another approach to comparing groups, best used for categorical

LOW Hlgh Low High (nominal) outcome variables or variables with few categories, is to
compare thc proportion or the percentage from each group giving a
particular response.

(c) Kurtosis Id 1 Spread
The i m p a d of an intervention may be seen in its cffect on thc slrnpc of the
scares on thc niltcomc variable. Comparisons of group homogeneity arc
one form of comparing shapes ul distributions. Otlicr aspects of the
shape nf a distr~butioncan also form thc basis of gmup comparisons and
reveal thc impact of an intervention.
Apart from comparing thc. degree of sameness or difference within a
group we can also compare the distributions using other dinlensions of
the shape of a distribution. These different dimensions arc illustrated in
Figure h.2.
!
For example, imagine that we wanted to examine the effect on uni-
versity course complcsion r;l tes of different methods of delivery of a
course. We examine completion ratcs of those who are taught in face-to-
face situations with those who take their course over the internet. We
might find that in the face-to-face group 75 per cent successfully com-
pleted compared to 60 per cent of the mcmbtrs o f the intemct goup.
These percentages provide a description of the different outcomes in the
two groups (descriptive statistics). We could then undertake an analysis
that would tell us whether these differences behvecn the groups (75 per
cent and 60 per cent) are likely to be due to chance or whether thev are
likely to reflect real differences in the pnpulation from which the sample
was drawn (inferential statistics). In the example just provided, the twu-
samples test for difference bchveen proportinny u,ould Fulfil this
function (Loether and McTavish, 19741: 189ff).
Groups also can be compared usefully by examining thc association or
i correlation between variables. StatisticaP measures of association test the
42 WHAT IS RESEAIKI-F VFSIGNT CAUSA I ION AND THE LOGIC OF KE5EARCI I IIESIGN 43

just two groups. Multiple ccrn~pariso~~

groups are possible and will arise
from at least two circumstances. Male Female

Ir~rft.pi,ndrn!varinbles with moviJ than tu7u cate~urics Where independ~nt Young Group 1 : Group 2:

r"; 1 1 1
variables have more khan two categories we can compare more groups
and provide stronger and subtIer tests of theories. For example, in
examining the effect of divorce on children we could simply classify
rnarriagcs into divorced or intact, or we couId classify them as single Group 3: Group 4:
(never married), Intact, separated, widowed, divorced (repartnered) and old males old females
divorced (not repartnered). If we took the latter approach we could make
multiple comparisons. [n doing so we can get a clearer picture of what is
going on. lf u7e Find that it is the children of divorced parents who have
repartnered who havc the most problems we achieve a more focuscd
understanding of the impact of parental mari taI status on children. We
have learned that it is not. divorce per sr but the arrangements that follorv
divorce that are critical. Had we simply compared divorced and intact remeasurement at the end is called t h ~ 'post-test'. Between the pre-test
marriages we might ncver have identified this. and the post-test certain critical events occur - in this case parental
divorce (or non-divorce as the case may be). These events scrvc as th.c
CnwrhEwni effech of clcffeewnf i n d ~ p ~ n d e vnrinbles
nt It is possible thdt it is independent variables in the study - i.e. the presumed causal variable
onIy when people havc a particular combination nf characteristics that an behind the observed change, In some designs thcsc intervening events
effect is produced. For example, it may be the joillt effects of gender and are called the intervention or the heatment (see Chapter 4).
age rather than cach independently that is linked to suictde. Wc may We can collect data about more than fwo time points rather than being
hypothesize that when a person is both male and young, the l j keli hood restricted to simple 'before' and 'after' data collections. We might have
of suicide is at its highest. If, for h e purpose of this example, we think of repeated measurements over an extended period to track 'ups' and
age as a dichotomaus variabPe (15-35 = y o q ; 35+ = oId), then we can 'downs' and to track /rends before and after any critical event. Multiple
think in terms of four groups based on the combination of these two 'pre-tests' and multiple post-tests can help distinguish between short
independent variah'les. These are illustrated in Figure 3.7. term and long term trends. They also help identify the effect of the
'intervention' or independent variable. For example, a simpfe measure of
Comparing tirr?r puinrs emntional adjustment before and after divorce might show a decline in
adjustment. But what if there was strong evidence of the declinc having
A finding that children from divorced families are less well adjusted
commenced well before the divorce? Only multiple measures before
emotionally than those from intact families prestimes that divorce is
divorce would show that a trend had already begun. Similarly, a simple
resyonsibfe for this difference. The problem with a simple comparison
post-test might show that adjustment is lower after divorce but mulliplc
between groups (divorced and intact) is that it does not tell us whethcr
post-tests might show poor adjustment immediately after divorce but a
divorce has actually produced any chnnge in the emotional adlustmcnt of
steady improvement over time (see Chapter 6).
children. Any conrlusions wouId be much more convincing if we could
track the emotional adjustment of children both before and after divorce
to see if thcrc was any change in the emotional adjustment of children
following divorce.
Idcall!! fhe Xrotrps 7uc are conrpnring sr~nulifhp the sarnr in ull rfler~nritrespt7cts
By tracking children over time we could see if thcrc wa5 a change in
except in regard to the indcpendcnt variable. For cxamplc, il wc want to
emotianal adjustment of the children and evaluate whether h i s was
attributable to their parents' divorce.
test the idea that non-government schools produce students that achieve
better academically than government schools we would need to be
confident thal both types of schools contained comparable studcnts. We
MULTIPLE
PRE-TESTS A N D POST-TESTS
would need this so that we could bc confident that the only relevant
Tn the above example the measurement of emotional adjustment a t the difference between the two sets of students is in the type of school
beginning of the study is often referred to as the 'pke-tt.st' and the attended. If the studcnts differed in additional ways, how would we
extent to wli~chtwo v,~rinblr~I-0-vary. Wllct~we say that gcndvr and
income nrc a ~ ~ o c i a t wc ~ r l nlcarl that hcnowinq someone's g ~ n ~ i hclps cr "b changr i n Fixcd ratr C;n>~rp bonu.; Cntl~vidualbonus
prcdzct f l ~ e income
~r bcttcr than i t we did ntit know their ~ c n d r r . ~~LIL~!I~-II\*I~V ~ r o u pIn11l ~ r t l u p(no \ ~:rr\up(no.)
A <in~plrexperimental d ~ w i ~has n hr-o v i l r ~ a b l c-~ the intcr\.vnt~nn
variabltb {independent \.arlabIrl 2nd the outcome variable. If the inter-
rcntilm hnq an effect on thc outcome variablc thcn the two t'sr~nbles-
the intrr\,cntinn and thc otitcornc - will be nsqociatcd statisticnllv. That
is, i f tht~\r, prople in the Rrotrp that recrivcnd rhc intervention (i*xpt*l.r-
mental grollp) scored dif fcrtan tlv on the .oittcornr v a r i a b l ~than thc15r.who
did not rt.st*ive the intcrvcntir>n, the two variables - inten.cntir,n and
outcrrmts - rvtwld be qtatistic~llv associat~d.5i1cli an a s w c ~ a t i n ~can i b~
d i < p l ~ v r dtr.;lng cro.;r;-t;lbulali(~n~, graph.; o r +tahqtical summarit*.: quch
as ~ o r r ~ l a t r ncoefficient.
n
In atl~litlonto rlescrihiny: ~ h extent r to which t h e is an arsocintion
bc-twccn Ihc cxperirnenta l intervention and tl~routcome, wc shou Id use
a n infrrcntiat statistic dcicrmi~wwhether thc as4ociation is likcly to
When deciding 10 display data wi tli a graph we must also decide which
hold in t l ~ c&\rider poy~rlation.Different rnfrrpnt~al tests havc L3ccn b ~ nfe graph lo use. rn Figurt* 6.7 just three of many possible graphical
developrd fnr different sr~rm*lntic~n coeificient~. displays arc pfiw~dcd:histogram?, stcm and I ~ a fpl~lt.; and ho\ . ~ n d
wliisker plnb 4II three grayhq arc based on tht. 5amr d ~ t and a yrocrdr
d~ffcsentl v t ~ y q01 thinking abou I and ~ ~ i s u a l i z ~thr.
n gdistrib~rtinns.

Since the* purposc of analysis in cxptrimcnts is to compare groups, we

must ~lrcirlchow to s S l i ~ i inny
~ group differencrs. O u r choicc is l?r>twc*cn
usinq ta blrq, graphs or summary statistics. Which nne(s) we choow tvill
deprntd partly on the ntrdlrncc and the cornpleuity of the data.
Thc t h r c ~forms of dt.;pI~l;ing resultr; cnn h illustratd w ~ t han
mampIc. Suppose we had an c x a m i n ~ n gthe trnpacl c l t diCf~r-
ent forms ot rvwnrds on wclrlipl,~ceproductivity. The three exy~rirxr~ntaf
condition< would be: (1) standard fixed ratcs of pay rcgardlcqs of
productivitv, (2) a productivi ty bonus for the w h r d ~group a t thccntl of the
year drpcnding on g r o u p nutput, (3) indir~idl~al
dependin): rrn the ind~~idual'.;
productivitv trnnuseq
productivitl;, Thc nu tcome vnriablv rrcjulcl
be the pcrscntrtge change in thc prductivi& rrf tbach individual.
Ftarnplrs cd tabular, graphical and statistical methods of rllsplaving
result%a rtl illustrated belowr.
T a b k h 2 pro\ ides a n exarnptc n i hnxv rr<trlt. f r t m a stud\' nlz!:lrt bc
prewnttbil rn tabular fc~rrn.In thiq exarnplc thi. clegrw clf p r n d ~ l ~ t i r i t v
c h a n ~ cI S rcported for cazh of the three bnnz~ss c h ~ a r n ~Thy
s ctnllh in the
table iniltcatr t h e numbcr of p~cjplefrom each bonus s c h e m ~cxhihitlng
varinmr lc-velq O F productivity change. To Frwk for differences in tht* r * f f ~ ~ - t
Alternativ~ly,wc* could avoid the z~scnf tables and cr,~phsaltogether and
I
I
simply Jernonqtr~tegroup differonce%by using avcraqcs, percentages or
some othcr form of statistical summary. ~ i & w(3.4 provides a set o f
e the three group';.
surnrnarv stntiqtlcq In relalion 10 protduchvity c h a n ~ in
I This is a con~prrhensic-estatisticar description of thc changes in thc tl1rf.c
~ r o u p s ,Htiowcvcr, much of tht* same plrture can bc gleaned from thc
graphs and table.
I
I
Ifuzr? rt?nr;v prrvrp.; nrr to bc cnnrpr~rcil~
Another defcrminan! of the rvay w e analvse our data i q the nrr~nlrrvof
grorrps we cvnipare. Some sta tistical techniques arc limited to compari~nnq
nf h r o t;mirps (c.g. f-test) wh11eothcrs are a p p r ~ p r i ~ ifor
t c multiple groups
I (e.g F-ratio.; nncl analysis of varinncr). Some mrrvlntion coefficients a r r
dcsigned For ti<m;lTl number nf ~ r u t t p s(Kendall's tau, rta) while other.;
are apprclprintr for i n d e p e n d ~ n tvnriablrs with a Isrcta number of catr-
I
qorics. Sirnil.~rlr., the appropriatenr~sof tabular a n d graphical diqpla~
1
tt~hniquel,1\+11Tt l ~ p e n dnn YIIY ntlmhpr OF Ernufs in t h t - analyrs.
I
I @YF-GROUI' COW PAIIISONS W111.I A KNOWN VALUE

The simpli*st lorm of comparison i q where we h ~ v cwly c one group ancl

of thtt t h r ~ vbonus schcmc.: we need to look at thi*pattern and syrturl uf we wish to conlparc. Zht. reqults in thi.: xruup with ,i known value. Thest*
I
nurnbrrs for each of thc thstr* \chemcs. methods OF anal,v.;is are referrc-d tn a< c~rlr-~nr?rplt'
drqiqn.;.
 EXPERIMENTAI. DESIGNS

(a) Histograms
Bonus system Statlst~c Std. Error
% productiv~tychange Flxed rate pay Mean -.5455 2 3769
Median .OOOO
Var~ance 186.443
Std Devlat~on 13.6544
Minimum -30.00
Maximum 30.00
Range 60 00
lnterqwartile Range 7 5000
Skewness -.I61 409
(b) Stem and leaf plots Kurtosis 1 078 798

k groductiviry change Stem and L e a f Plot f o r : Group bonus Mean 45.4848 5 9012
GROUP= Fixad rate pay Median 39.0000
Frnqucncy S t e m & Leaf Varlance 1149.195
5 . U O Fxtremes (=<-IT)
I
Std. Deviation 33.8998
1.00 -0 . 9 Min~mum -5.00
-90 -0 . Maxlmurn 130 00
2.00 -0 . 45 Range
3.00 -0 . 223 I 135 00
1. 0 0 -0 . 1 lntwrquartlle Range 53 0000
13.00 3 . 0000030001 Sltewness ,567 409
1. 0 0 I) . 23 KU~~OSIS -.261 ,798
3.00 3 . 445
. iio L' .
lnd~vidualbonus Mean 31.1 51 5 9.071 2
2.00 n . 8s I
Median 14.0000
4. n 0 E x t rPves (>=17) Var~ance 27 1 5.445
GROUP- B r o u g bonus Std. Deviat~on 52.1099
Frequency S t e m & Leaf M~nimum -45.00
1-00 -0 . 0
29.00 0 . 0000111222233333444
Maxlmum 125 00
11.00 S . 55667778899 Range 170.00
.
I
2.30 1 03
GROUP- Individual boaus Skewness
Frequency S t e m & Ledf Kurtos~s
7 . CO -C . 0123334
16.00 0 . DOClUl100111112222
1 3.00 0 . 569 Figure 6.4 Summary staliqtirs: productivity ch~npeIry hnnws scheme
7.00 1 . 0111122

(c) Box and whisker plots

Say for example we wanted to estabIish what connectinn there was, if
any, between having a child with a disability and the likelihood of the
-1 child's parents divorcing. Let us suppose that we know that the national
divorce rate in anv onc ycar for couples with a child in the household
is 12 per 1000 married couples. We then find from our research that in
a sample of families in which here was a child with a serious disability
the divorce rate was 14 per 1000. The rate of 14 for the disability sample
is certainly higher than the national overalI figure, but is the difference
likely to reflect a real difference or simply bc due to sampling error?
An inferential statistic can answer tIis questinn. The same logic o f
comparing a single sample group with a known comparison h p r e can
be applied to analysis in which we focits on the mean, proportion or
variance of a single gruup (see Loether and McTavish, 1 9 7 4 ~Chapter 7).
 EXPERIMENTAL DESIGNS
TWOGROUPS
The classic experimental design cornpsiscs two groups - the experi-
mental group and the control group. In experimental dcsiws with an
interval level outcome variable thc /-test is used to establish the stahstical
significance of intergroup differences. It indicates whcther the difference
behvepn the means of the two groups is likely to be due to chance or
w h ~ t h e rit rcnccts true differences in the population. Thc use of this test
depends somewhat on the distribution of the variable (see below). Othtr
tests, such as the critical ratio, can be used to establish the significance of
group diffcrcnces for large sarnptcs whcre the assumption uf normaliiy
is not satisfied (Mueller et al., 1977: 422).
There is nothing about the logic of the experimental design that
restricts u s to interval level outcome variables. Where therc arc only t-cvo
groups and the outcome variable is either nomind or ordinal we might
use a measure of association as a summary statistic to describe the l ~ n k
bctwwn group membership and outcome.
Alternatively, we might use cross-tabulations nr graphs to display
the differences between hz70groups. 1f we needed to generalize about
wliethcr any such rrli~tiunshipshown in tahlcs or graph< was likely to
hold in the wider population we coulcl use an appropriate inferential
statistic. Chi qqrmvf is one widely used statistic that is cspccially useful
when outcome variables arc measured at the nominal level. This tells
us whethcr the hnro variables are independent of one another - that
is, whether group membership makes any difference to the oi~tcorne
variable.
THREEOR MORE GROUPS
Some experimtntal designs require three or more comparison groups.
For example, we may want to compare a range of interventions a s well as
having a control group.
When the outcome variable is nominal nr nrdinal we will largely be
restricted to tables, graphs, measures of association and other ways of
comparing di5tributions. These methods are satisfactory so long as there
are sufficient numbers of people in cach Ernup and there are nnt ton
many groups to compare. When there are more than six or seven groups
to compare the comparisons can become very d~fficult.
If the outcome variablc is an interval level variable t h analysis
~ is most
effect~velymade by cnmparing the means of the different groups. A nue-
7lay nlanlysrs cf variflncr is frequently used for this purpose.
A one-way anaTysis of variance will tell us whether there are differ-
ences within a set of means. For cxample, in the earlier illustration of the
impact of bonus schemes on productivity there were thrcc groups: fixed
rate of pay, group bonus and individual bonus. Let, u s suppose the
resuIts outlined in Table 6.3 wpre obtained in this study.
ANALYSING FXPERIMEN'TAI, DATA
Table 6.3 Maru pcrct+uhl~c~'hnlrxeI??
Iry ilntlus SCJICHPI,
;im~Juctiriil~/
Honus group Mean "A change
in productivity
Fixed rate of pap -0.55
Gruup bnnus 45.48
lndiridual bonus 31-15
An analysis of variance may show that, overalI, productivity change
varies according to group membership. However, the problem with this
analysiq is that we do not know zuhiclr group differences are important.
Fairly clearly both the group bonus and individual banus groups do
better than the fixed pay group, but is the difference between the voup
and individual bnnus systems statistically significant! Could the
diffcrencc between the 45.48 per cent improvement and 31.15 per cent
improvement be simply due tn chance?
To hnid out which groups have means that are statistically different
from one anothcr we need a multiple comparison procedure. In such a
procedure all possible pairs of groups are cornpnrcd and i t can be estab-
lished which pairs of ~ ~ o u pifs any,
, have means that are significantly
different from one another. One widely used multiple comparison
technique is the Scheffe test.
Thc Schcffi. test gives us an estimate. It does so by looking at the
differences in means between the following pairs of groups:
1 fixed pay versus group bonus
2 fixed pay versus individual bonus
3 Group bonus versus individual bonus.
It can be shown that the differences in the first two pairs (fixed pay
versus group bonus and fixed pav versus individual bonus) wcre
unlikely to be due to chance. However, the difference between the pre-
ductivity increases in the group and individual bonus schemes was
probably due ta chance. This means that if the observed differences in
productivity imprnvernent of those with individual bonuses and those
with group bonuses is due to chance then essentially the two schemes
are equally effective. It does not matter which bonus scheme is used.
Are the conlparistln gmtrp75 i n d q e n d ~ n sanryles?
t
~ D F P F N D E N TSAMPLES DESrGNS
The simplest and most effective way of making groups initiatly com-
parable i s to randomly assign individuals to expertmental and control
groups. This is called an ~ndcprndellfsarnples dmign. Random allwation to
IU? EXPERIMENT.4L DESIGNS
cxrcrimental and control groups should ensurt hidcpcndcncc betawen
the groups. There should also be independence between cases rvfflrin
groups. Srnce the sel~ctionof one sample element should be independrnt
of the scIcction of Pverv other dement, a gruup consisting of, say,
husband and wife pairs docs not produce an independent samples
design.'
FAIREDSAMPLES DLSrCNS
In a prlirtld snmp1t.s design groups are made comparable by pairing sample
elernenis rather than sin~plcrandom allocation. Here the selection of
people in one group is not independent of selecting pcoplc for other
gruups. For examplc, wc might have a group of identical twins and
assign one twin to the experimental group and the other to thc control
group. Alternatively, we might simply administer different experimental
treatments to the s u m group at different times. By using the same group
wc theoretically remove any other differences between the groups
(except of course later treatmcn ts might be contaminated by earlier
treatments to the same gmup). In effect people in one experimental
group arc paired wit11 themselves for the next trcatmcnt.
Some ir~ferentialstatistic5 need to take into account the way in which
peoplt. were allocated to the different experimental condi tiuns. Werc
thev pniwd in somc wav or wcrc they randurnly (independently) selected?
I,q t h d~~ ~ n d e rvarznblp
zf normally dislribzjfed?
Some inferential statistics, caIled parametric statistics, require in tt.riral
level ou tcnme variables that are normally distributed in the population.
Unfortunately often either nutcome variables are not. normally distri-
buted in the population (in the statistical sense) or their population
distribution is unknown.
When the outcome variable is not normally distributed or we cannot
reasonably assume a normal population distribution we have three main
options (Loether a n d McTavish, 1Y74a: 2hXff).
1 We can 'normalize' the distribution by transforming i t (Marsh, 1988:
Chaptcr 11)
2 M.Tt ran violate t h a~sumption
~ of normality dncl use the paramctr~c
statisbc anyway. rt is argued that some of these statistics are robrrst
and vield very similar results regardless of whether their assump-
tions arc met.
3 Wc can u?e a notr-pnmr~ictrrcstatistic that docs nnt assume a normal
distribution.
How can wc tcll whether tl-rc outcome variable iq normqlIy distributed In
the pnpuIatinn? Given a sample of sufficient size a line graph or a
A N A LYSING EXPERTMFNI AL I I A T A 1 U3
histogram can give a visual idea of whckhcr Ihc distribution is a bell-
shaped, unirnodal distribution. A normal curve will also have a skew-
ness of O (i.c.symmetrical) and a kurtosis of 3 (i.e. mesokurtic) (Loether
and McTavish, 1974b: 162). When the sampIc is small (e.g. less than 50)
and is not randomly drawn from the population, and ~ t spopulation
distribution is unknown, the qafest strategy is to use either non-
parametric statistics or a robust parametric statistic.
1s the vnriunue an the d~pe~tdent
varinblr similar betz~wnrack
c o r n p n ~ i f o~ ~r o~l r y ?
Group comparisons can be usefully executed by comparing the typical
response for each group. However, comparisons ot what is typical can bc
invalidated if the groups have very different variances. Comparing the
means nf groups with very different variances make5 it difficult tc>
establish whether any differences in means are due to differences in
central tendency or not (Loether and McTavish, 1974a: 176).
Some statistical techniques require that wc establish that the variance
in thc (mu)grtwps is similar. This is particularlv important for t-tests of
differences bctween the means. In the case of /-test5 there are diffcrcnt
forms of the test d~pendingO n whether there is cqualiv of variances
between the groups. Most statistical packages include statistical
techniques to test for this condition.
How many irrdqlrndent vnriablfs?
The simplest experimental designs have just nnf independent variable.
However, many designs include several independent variables. Factorial
!i
and Solomon four-group designs examine thc impact of each of the
experimental variables on the outcome variable. They also identify
the impact of interactions between these independent variables on the
nutcome variable. In field experiments where random allocation is not
possible we need to estimate and statistically remove the effect of known
I group differences so that we can examinc the 'pure' effect of the cxpcri-
mental variable.
When mu1tiple independent variables are used the analysis strategy
will depend on:
I
I - a the level of measurement of all the variables (both independent and
dependent)
whether we want to examine the rrlrrfilte inlpact of each independent
variable (including interaction effects) or
e whether we want to look a t the effcct ot a s i ~ ~ gindepcndent
le vari-
able with h e effect of other variables controlled.
1D4 EXPERIMENTAL UESIGNS ANRLYSINC: EXPEIIIME!+JTAL DATA

When a design uses multiple independent variables and an intcrrlal level

outcome variable, two-way or fac/ori~la~lalyqis?f rwrinrruc is a common
method of analysis This method enables the investigator to look at the
impact of each independent variable w ~ t hthe effects of the other inde-
pendent variables removed. That is, the analysis can isolate the 'pure'
effect. nf each ind~pendentvariable and each possible interaction effect.
This approach can be illustrated by employing a chemical analogy in
which there are three chemicals A, R and C. The question is, 'What is the
corrosive effect of these three chcrnicals on steel?' We could look a t the
effect of chemical A alone, chemical B alone and chemical C alone. We
might also want to know what happens w h ~ nwe combine the three
chemicals in different ways: A with R; A with C; R with C; and A, I3 and
C combined. Factorial anaIysis of variancc cnablcs us to answer these
sorts of questions.
Multiple regression is another approach employed for analysis
involving a n interval level outcome variable and multiple independent
variables. Multiple regression provides a way of examining the effect af
cacl-r independent variable minus the effect of every other indepencient
variable, and of the interactions between the independent variables.
W ~ e r t .the tlzd~prtdenfvariables are not measured at the intcrval Icvcl
(normal multiple regression analysis assumes that all the variables are
measured at the interval level) we can use a modified form of regression
analysis called dlrmmy zwriahl~s(see de Vaus, 19%).
If thc drrpendent variable is nominal or ordinal in character, analysis of
variance or standard regression analysis cannot be used. There are,
however, other forms of regression analysis such as logistic regression
that can bc used far these situations,

If we are interested in the effect of just one independent variable we may

need to statistically remove the effects of okher group differences on the
outcome variable. Analysis of variance and regression achieve this.
However, with nominal and ordinal outcome variables we can also use
cross-tabulatiuns to rcrnove the effect of 'contarnina ting' variables. To do
this we would uqe ehboration analysis (see pp. 203-2101. There is
insufficient space to describe this approach hcrc, buk clear explanahons
are readily available (Rosenberg, 1968; Johnson, 1977). Another wav of
removing thc confounding effects of othcr variables is to use an average
partial table (Loether and McTavish, 1974b: 290).

S u n r ~ i r n ~process
!~ for sele~-fil~g
fllc right Inemure
F i p r r s 6.5 through 6.8 are designed tn help you select, an appropriate
form of slatrstical analysis given different types of rxperimcntal situations
 ourwme vanable

......................... . . . . . . . . . . .

N u m k r of groups

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.,-
HOWmany '
Indcpmdofll
1

_ (urou~~nq)
variabl* are there7

I'
. . . . . . . . . .
,
,

, Is the nutcoma

. .

, . . .
Graphs 1e.g. rlns grspr.3. stwn and l m f plms for each ', Descdpltve
' L W 8
~1:
'Skewness '
, group, hlstcgrams for Each qroup erc ) hsmlprlon ' Multlp'elpartlat cormlm~on
, l ~ law
' Crws-tabulatons (only w h o ~ o u a r l a bhave ' Gmphs 1p.g llne graphs, sfem and
' Kuflcs~s
' catcqonos) leal p,ms 40, 8 8 m group. hxlugrams ln'e-lbl I
' Ccnlml IOndenv Two-way tiacforlnll
. Knlmogolo u--Sm,rnw . ' Comp~nsonorihe groups rn termsol cerltral tendenv, for ~ a c group
h ptc )
' Crops-tabvlairona ( ~ n l y
where analys~s01 variamje
v a l h i on, shape
' H~sloq'am
I ' Mcasufe of assoclatlon- Pnarson's r, eta I varlablos h a w few categories]
' Stem and leal g o t
'.
,-
~nfererrce
' 07s-way analysis of vaflanoe (oomwnrqmearsl
I -
Commnson ol Iho groups In rsrmf ol
cer>trillfendeqw. varstlofl. shape
. .

-- -. ...... .- .-.
' Kwskal-Wallis K t e s t
.
.
.
..
.
..

Figure 6.6 Sclecfing statlsfical methods of analysis, part 2

0 Level of measuromant of ttw

outcome varlabla

. . . . . . . . . . . . .

0- Palmd or ~ndepend@nl
groups 1 Palred groups

'. ! Cornman arnuos uslna: I

- .

Compfe groups using:

Descriplbn
' Graphs
' Cross-tabulauom (11s m l l vumkrolcategorles @I mlt var~ables)
1
1
hrnpreg, o;p9"31"g
fescnphwr
Descrlpilon ' Summav s t n t ~ s t l asuch it5 centra'tendmry unrlal nn qhane 01 Grap?~
' Glapvs
C~cinbvt~on * Crws iabuaettloqs(n m a l l n u m w r 0I
Cross.rabulations [II small numbar d
' Propr: ~ n s categome<01 l l a n varlablss)
categoves or boib variables)
* Moasufes cf assoclat'on (whm companmn groups can be 'ank Surrrray slaI8sl cssuch as aentral
' S u n ~ r n a ~ s t r t t ~ ssuch
t ~ c s as centml tendency,
areredl gamma Kendell's rau, 5++ermal'4 rho Iendcrcy vorla! on s h a w ol
varlatlun, rtinpe uf dl>tr:@ullon
' Pmuo't~o~s
- Measofesol a s s a ~ a l ~ n(when
n comparlscn qmdpscannol be rank d~s~rlbutlon
ordered) lamoaa Cramer's V ' Prnwn8ons
'Odds rntlw
' Measu-es ot assoc~al~oh
' C?,squdre based measures of a s m ~ a t l o n . Inbmncs I
p h ~Crarners V , contingency mefhcleni Any oumber of grwps
' P n burell measures dassw~aumlambda. , " Cbl w v o r e twf
Go@dmal a i d Kruslal s mu. Yule's 0 ' approy>r,ats lest c r gamma Kenailll s Iau or
of s ~ g n ~ l l c a nIm - McNe-nar lest (dchomous varlaWes
Inference
* Ch1 muare 1851
' Marg~rlnlhornog?ne~t) ,'lest (only lor
'
Spes'man's rh3
Twu Groups
Kntmogoro&!3m~r,lov2-rest (2 rndependent srnples, ron.earamoInc)
I :$:; IPSI
W.lmxon marcbed palrs tesi
mn'par8son of 2 g r o u p ) Watt-Wollow~tz tuns t a s l ( 2 Indnmndont somples. non-parametric) Thrse gmvps
- Tcslsof slqnmf~wr;cef o r a c h prc bassd : * Mann-Whltney N e s t (2 independent wrnrles, nnrrparamelv~c)
' Two-Sawple rnedlan t e ~ i
' Cocoran's 0
meqsure
-- -- Three or more gmups
I
I
' KwskeI-WBIIIS Hr~qde~enden
sarnpte..
t 3+ QICI~US.110n-parametric)
k-sample msdlan
-tssi I

ststistical methods of analysis, part 3

Figure 6.7 Selecti~l~q
EXPERIMEWAL DESIGNS ANALYSING EXPERIMENTAI, DATA 109

and different types ot data. These figurcs cannot coves all pnssibilitics but
provide guidance for a fair range of typical situations. Most of these
statistrcs are easily computed using packages such as S f i S or SAS.

Selecting the appropriate method of analysis essentially involves asking

a series of questions:
What sort of sample do you have?
Probability.
a Non-probability.
What IS the level of measurement of the outcome variable!
a Nominal.
w Ordinal.
w Interval.
How many group4 are you comparing?
m One (agninst a known standard).
Two.
Tnrce or mo-re.
How were pcnplc allocated to each group?
Are samples (groups) independen t7
Are the samples (groups) paired or matched in some way?
How is the outcome variablc distributed in the population?
a Normal.
a Not normal or cannot assume normality.
Do t l ~ egroups have similar variance on the o~itcomevariable?
r Equal variance.
m Unequal variance.
W h a t group comparisons are required?
Ccntral tendency.
Variability /shape.
Proportions.
m Association.
What type of way do you want to display your comparisons
between groups?
Tabular.
Graphical.
0 Use r ~ summarizing
f statistics.
Mixture.
How manv independent variablps do vou have'
One.
e Two nr more.
Are you mainly concerned with description or inference?
m Describe patterns in the sample?
Generalize from the sample?
m Both.
Ntit all t h t qutu.;tir\nr
~ ,Ire . ~ p p m p r i a tIn~ ,111 circt~mstnncrs.5ornt9 arta ? or 4 I t t l ~ L , ~ n a l v +n~lqr a ~ i l r t tJtagr~ata
b~ ol chtit~qc,r<~thcrIh,~n\lrnyIv
npprnprh~ted c p t - d i n g cln h i ~ wt3al.l itxr quv5tion%arr sn<t.vi%rt,~l ( c . ~ I.I t.hangc / ) r h r ' . i t ' t11t~ 'iltrur' (11 tl~c=w a l l , c a n Itaarl u> 1 1 7 ~ r n d t a r c s t c m ~i h~tt,
y o u r t r ~ ntlt ha1.p n yrirh~lilltvsample then VOII c a n n o t usc qtatiqttcal. clt.qrot* c + l ch,tnp.
inftlrcrlcv . ~ n d~ v i l lthuq bt- rtwtricted to drscriptirm) Thrq ik ,I prcrlllem with many rnrtarures fhat have ilpprr and Irwt-r
limii\. I t ~ n t r t d ~ r c cn s bias in tc1.m~ot the tvpc nt c h ~ n p Illat , WP C J ~
InPa+1trc . ~ tthe indlvidual Itnvcl (w.r Chaytrr qj and t l i c~ a ~n under-
I
Other issues e<tirn;ite the cxtcnt of c h a n ~ cand w r t j n ~ l vt..;tirnatc tl-tc dircctinn (16
d ~ a n c co w r a I!.
It i < diffic~tlito dpnl with this prot7lcm of dnt,i dm6i-n front scales wrth
qtric't tiprer and lo\vVr llrnits at t!ie analvsis <t~iqr.Thtn pn)blem of
Tht. lo):lc t d thl* cI,l<sic rxpr~simentnlJ t . s l ~ nM.~'IPIIL~1~,1d115 to ~151-chang~. truncatlclrr rfftbch 1s best ~1aii~I1ecl at the 11eqi~n<t;lqcawlirrt> wr nctbd tc*
sclrrt'k. I\-licmn 17 c hart, II pre-tt,ht \\.c (-,Inra\amlne hot79 milch c a c l ~jirt?lip en+urtntli,~tmlr mt5;ls11rcs;ITC Je\~gnt,iIin s ~ t c h way t h ~ pt*oylr.
CI l artnntlt
chanec. on tht. nutsonlt. nl'asure l,rfore and ;~Ctcrthc inttan'cr~ticm,5 n c c
cfu<tc*rrcl, ~ \>lie t t-nd cd ,I ~ c ~ l\2'e c . 5hni1Fd tamplux rnt.xrlrch.; [hat har r s
WI, rccoRnlze thal thr ci>ntrrrl group mtIv at50 cxllibit change dt11. to
h i ~ l rccilirl~c,imJ l ~ u floorr ant1 n h m , ~ t w p l (do ' xit3t cluqtpr tcl~vdr~l
t ~ c t ~ j qwch
rs a< matur;lSion ~ v ctvoulr.l s~rnpll;h ~ * p o t l i e ~ itli;lt
l t . thangc PI 1 her mJ.
neill it.lrn vcatcr in tllc cxprrimt~ntalgroup than in thr conkol grt~lry.rclr
exarngilv'

In rrrrI1.r t r t icl(-rltli\. .;Irt\rt tt>rnlanti b t y tl.rm cmffimctc ~ * t . C'ha~tt.r

t~ 4 1wmr-
v \ \ l e r i ~ ? ~ r ~ ncle.si<n\
tdl rtlclrlir(, r n ~ ~ l t i pbtafnri>
l ~ < anrf a f t r b r mcr7curt3+.TIle\th
di,$rgns c.ln yrc<tbnt p c i ~ ~ t ~ c t iprtdilcrn\ lar ul ,3ii;lly51~atrd can ~ n \ ~ t ) l ~ i b
I\'c can thcn ttndrstakt. ;In ~n,ilyqiccot vntiancc t o i f thr. I ~ P ~ T E( 3~i - ~ l l ~ i t~i lo n i p I ~,in;ll!.\t-.
\ i f ~ t cCIIV try in;: !t r c.ctirnp,~rct1z.o trrndk (a<
d iff&-renceIn chanp. score.; 17-0 compared tn 5 ) could h a v r oc~-ilrrr*d 14 i ~ e t ~ p p ~ h t 'tnd S<T\ tw11 171edn\) Thc~rta t~ a r ~ l n e ec r t r!*l?at are callcd tin~r*
to chancta (wmpling crror) If llic ~lifft*rcncec-o~ilrltrc, dirr to s a n ~ p l i n g .;crlcs .lnalvsi< mcthorls tlrtit arc I ~ ~ , y n ntl~c d scoycn (11 thls bwlk
rrror w t ~zlijulclns\llrntS t h , ~ tthc.rP rv,.crliId be nir differcnc~.in thib pnpll- 1 IOM'CI PT,a I I S P ~ L I I i i i r t i . ~ l apprnnch to a n ~ l p i thekc n ~ sort< I,! d,ita IS ti3
latirm from r\.hich the qnlrnplewas drawn. d r ~ w5impEr line ~ r a y l ~Ts h. ( w may cir m a y not rcvral ,I clear Innq or
71erc arc seibcral protllcms wilh using simple c l i a n ~ cscor~.:to detect short t r a mcff~ctor intEicatc th,~tthc. intcrv~hntit~n 31~1sn o rlistit~cliveeflcct
change , ~ n dthe<(>will bc d~scu.;seJ in Chapter 9. (Fi~un> 6,s).The patttsrn 111 tht- e r y e r i ~ n ~ n t a~$(711p(h) l can tl1r.n h r cr)m-
partd rrith that ol a cr,ntri>! crolrp.
I The ciifficlll tv, hl~tvcvcr,i.: th,it L V I th r n ~ltlplt- l o l x ~ r \ , n r i o n s[he patt~rr~:.
I ,ire frequently no1 a4 ~ l e ~ ~ r - a5 c u tlinse
l ~n Fih~lrc.h.'). We can ~ r l ~ s c r v c
In dc.;ip.; zcheri*scores for a gstrwp cluqter around the top rlr bottom end
of a scalr ( i t , t h r mcnasureof the outcome varlablr) wc m ~ t s tremrrnbrr
lit hvrcnct5 ~t di t f ~ v = ~time n t ptbint- s~rnplydill= to ch.>ncr n n ~ lmcasltrc-
n~entermr. The prrlbltrn ' t h ~ nb t ~ c o n one ~ ~ l i~r f what con<titute%a rral
that changc can PR~! bp detc.cted in one dirpction. This tact can intrrrducc
a bia.; in I he nature of the nt.rserved changt'. I diffr-rence I ~ c t w ~ cditlcrent n time ct.ric.; Iinrs. 'l*lii< isqtlc is ton I n r ~ cto hc
For ertarnpI~,we may use a 0 in 10 scalp. 10 rnrasure happiness wlierc a I
cfeart with ~lrfinitiicry hcrc. I f you 17la11 to d o this lypc of .inaIy<i5 cnsure
si-nrt- of I F ~ndicstcqirnhnppine.;s. 5llppo<e that wc had p r o ~ l e111 nur
that !.oil c>bt.vn r a t d s t ~ i t i ~ t ~aJ\bicr c a l beftlsc. undt~taLincthv analy31.;.
croup< that qccrrvrt 11, i.r. \*.ere ~t thv ekt rprnr. PI unhappint.<s 0.. r r r i , , ~ < r r r l . l !
, thew pcoplt- bcbca rnr rwen ~nh~7pplc.r
I ~ Ir Iv r t ~ i r r f r .I i n w ~ v t r if nvcr hmr
thcrrl is no way of dcatccting t h ~ ,rvrth clus Incasurc.. Such pcoplc wt~ulil Summary
bt. itlcntrticd as nr3t c h . ~ n ~ i n g\ I r k * t o ~ i l i l n n l ~ .idenill\. peclplt. n ~ h n
btbcamr Ii,ippicr 'Thlr- .;Arne IIiinq ~vt,uId a ~ l y l rti? thost, wllr, .;r.(lrtvl TI) 'Tbi.. chsytcr hac cx;lnlintd ~ t l l t ~, ~ r l a I v ~ofi +~b\ptbrrrncntalcl.it.1.
the I r r p of
c i l ~ ~ lvnlvd dr*tecl thnsr rl;ho hcc;lmi%1e.s happv. Iwr fhc most pnrt t l l r ~h a w logic I\ thdt of c.omparinK ,:roItps In rralatlon
Fvtm t h l ~ ~vhc) r art1 not a t tht. ta\trcrnt- encj of a +calt. !,lit art3 nrdr thi* t o h t m . t>ncli FOIIF ~lcrtnrnmt c t ~t h ~ o . ~ r t c ~ mL nri;ll~It*
c lllic b a ~ 11f ~ sthe
end prcq~ant .I prr+bIcln T r v t-\;lrnylt~,nltl1ou\:li thr ptlrsr\n \\rho obtain< ;1 cq~)rnl-ranw~n can frc averazrc. ~.r.lit~lc. ~liqtrib~i tion.;, v~lrrancer v m,Tnv oththr
score of 1 in thc prtX-tesl can lower (thc~vcan cc~t 0 ) thcy cannrrl rc)~amctrarl?tic.;(11 di<frihi~tir>nsrhrrc i + a h ~ * ~ i ~ l l c f ~;Irr;lv r ~ n )crl: st,itistics
~ l ~ a tc3 n the ~ r sdmc r + j t r , l i f as. \omctmia whir ol7ta1nt.d ari initial w r r e u i qav t l ~ ,Ire
~ t .iv;l~l.~hli-
t(1r t h t ~an;lI\.+l+01 e x p c ' r ~ r n e n t ll,lt,~
~ l In tbrdrbr ti) ,ls.;r.;t
112 EXPERIMENTAL DESIGNS

14 -
I
PART PI1
12 -

LONGITUDINAL DESIGNS
--

a
7
-- - TYPES OF LONGITUDINAL DESIGN
+F

-I -G

You will find a wide vaiiety of IongitudinaI designs in the literature.

However, they a l l share the common goals of measuring change over
time and of doing so by collecting data concerning at least two time
points. In both respects they are similar to the classic experimcntal
01 02 03 04 05 06 design that has a prr-test and a post-test. However, longitudinal designs
Observation time do not typicauy have a randomized control group.'
Developing longihdinal designs involves a number of critical deci-
Figure C7 9 Sonle otrtcornes of nil rx;rrini~ntnlvnriablr on a filnp sfrim sions. The decisions taken will determine just what type of longitudinal
design evolves.

with thc seIection of the appropriate statistics this chapter h a s outlined
the main considerations that need to be taken into account when select-
ing the particular method of comparison. Finally, since experirncntal
designs normal1y involve the analysis of change the chapter discussed a
number of issues about the measurement of change: the usc of change
scores, truncation effects and trend analysis. Further problems witli
measuring change will be considered ill Chapter 9.
Note
1 This ts t r u e if ~ndlvldualsare the units nf analysis in h e ~xperiment.Ef the
cnuple M'3S the unit of analysis then having the couple would not violate Ule
reqt~isrnrrrits for indcpendencr.
1 h7iIlthe same cases be followed over time? Here the choice is between
trend studies and panel studies. Trend studies (or repeated cross-
sectional sbdies) entail collecting information from comparable
samples over time but not from the same people. Annual surveys
such as the General Social Survey in the US, the British Social
Attitudes Survey, the Eurobarorneter surveys and the International
SociaI Science Program Surveys are examples of trend studies in
which similar questions are asked of a new sample each year. This
design allows the tracking of changes in attitudes and behaviour over
time. A panel design involvcs repeated surveys of the samc people.
This design allows the tracking of change at both the aggregate level
- and the individual level (see Chapter 9).
2 Will dafa be collpcted at onr poznt of time? Although panel designs
involve collecting data concerning at least two time points, they d o
nut necessarily q u i r e that these data be collected on two different
nccasions. The prospective longitudinal design entaiIs collecting data
on several different occasions. In such a design a group of people will
 11y I , , t k I 1 i ~ t v t . ~ l lrr1L
, ht.13
, I I I ~~!L ~ I I ~ I ~~ ~
> Ci \I ~ t b
s ti ! ! L * , I < ~ wi~.t-,il sonit,
Itrh>r!?tinic. 1 !I<, rr7frrl-:'h.il r;?. yrlnt.! ~lc,<i~m =nL6ir-r c.nt,li 1 caT!c.r-ting i1~t.i
un r7nI~r>tlla tlccxiot~. rti~ ~ * changia ~linietw~c>n.:
l i ~ t i and to tlic 5 t u ~ l y
;Irt> rdlt,~ined171,wL~nl:p t ~ t y ! ~to. recall w h ~ tt h ~ n wrm
~ s likt* a t sonic
~vrllttrp0111t "'I h m r ,IF, IVCII <I< af the prficnt. In !hi\ \\'a\. rve can
nl-rtnin cclmc ~ndicabonof thc cvtent iu which the pLarson ha\ c h a n j i d
t n ~ Irime C,tu.;al anntvqit; ~ n t ~ i el sql ; r b l ~ ~ h ~thc n r :trmpor~lorder of rvents T'llis i<
nrt-i-+wr\.\ltbiau>c3 a llasii ttmet ot i , l u s a l rr;lson!nq I < t h t ~a cali+r r n w t
I+ongittirlitla1cttl\i~nswri vnrv OII manv other ~iirnenstonsas wrll. Thcqth prcc(.de ~t.;cltf~ctin time In crtlss-scct~nnal r e s ~ ~ i r cwh1.i-t. h all d a t a art-
~ n c l u ~ it t! ,i ~ lf t b l l t l t t ~ n g * colltcted at cmc.p i n t of time it can be Jiftict~ltto rvctablisli the i~rilcrin
r~-Iiichr t c n t s occur (we C hapirr !(I) f lt~it.rvcr,It>neit~rtlrnaldta.;i~m.
5rtti11,: Thc, logic t h ~ \lonh.itu~li~ial rlt'.;i~n c.ln trr ,~pplii,d in ~ariir.ulnrlv prn.;pt~c~~vt~ ~ltqsignz,i~nablr. trackiilg the rwJcr i t ) which
qc!t~ncs r,~ri::inc from t l h~ 1 5 h l ~controllctl lal,trsntor\. crtztnticm. cvrb~it.;take* place I : Iex,~mplc, ~ \trtdy miqht c ~ t a l l l ~ sa h
(I ~-rtrhs-ztbrt~tmaf

sm,~llt,r.;cntc grnL1p.i (z*.g.srhocr! l ' f . i ~ + ~p>; ,l r t ~nf an rrrg~ni;.atron)t o c o r r t ~ l ~ ~ t I3ctwtvn

ian tinemplovn~c~rt , ~ n dmental hralth hut wo~rlcf have
1ari:t~ wale r i , ~ I ~ o nsurvevh. ,~l di 1 Ficultv t,.;lnhli<hinj: whtathcr unt~mpln!~nient<~flt>rted mcntal h t . ~1111 or
m T i - 7 The g a p wrer~vhti-hcti,~nxe(art be m t , ~ q r t r ~ can *d \'an' vict. vt3rw .\ Fongr ~ l r d i n ~ lF e ~ i g Irmvri n, rr. could m e a s ~ ~ rthc
r rnent~l
from a \.er\' hhclrt p~,riod( r , a~buton- and a f t r r mtnasurt.r t l nt2ihldta.; Iit~althot ,T \nrnpltb cli cmplr>);e~lpcnplc and thew qec ~ v t l ch.~ngcs ~~t to
aftrbr sccinq .> f~lrn)to nlanv \,pnr\. rlrt>h'nrqonal Child Dclrtdrrp~nt~nt mvnlal lit~.llth,1 1 nnv, nccllr among thna. that cubwqucntlv hccr~mc
C;tuJ\ 1% a n t.\,3n1pIt- r r t a l t ~ n skrrn panel stln elr. I-lit# ~ t u r l vllegan In u n t ~ n i ~ l t ~ v r~~Irllt.t ~ m a t i \ ' .t ~nee
I t cl%uldc t u d ~ .untmplor tad ~ m p r t and *
IqLH ,\nrI int Iride~t,111 chilrirran born in R r i t ~ ~%jehrcrt.n n "
7 .lncl Mart-h. ol>+~arve i.h.incrs 117 rnrntal l~cnlth,irni\nfi t l i ~ ~ ctli,lt ' get .I lob.
Thrstt, c h ~Jrrbn l l~avrbIven trnc ked n n r l in1nl\*rrlin +uliqcqut-ntfnIIo\t,-
L I PL I A ~ ~ I titon\ i r i TQfi;,
c(jll~-a lL't79,It374, I L + 5 i , leu1 , I ~ I L ?~ II(~~I
a i l ' r r t r r l ~ r ~ 11f
r . f / r l r r q p c l r t l t - L o n ~ i ~ u d i n5tctdic.s al will irnrvin trxrmsof hnw
nian! trmw ~nforrn,tl~tm i<C ~ I I I L T ~ P L ~ Irom tlic *;an>tb cct nf p c t ~ p l ein n In ctutain ,ilid~c<of c h , l n ~ i ,i t l.;l~ibe c I l i f t i ( - ~ ~ lto t t=itatrli~~I~ ~ ~ hw~rt
t ~ oft
pantrl > t u J ~crr , ilitftart~ntset>~ r pf ~ o p l ns t in n trend + t t ~ dT~h .r t;7mtrtrl; change rlccrrrs. Fc~r~unrnyllc,age and pilltical c-tmserv.l!icm art, rorrl*-
'7-t~p'TV prt~grnmnit~, cv11ic.t~h a s ftdloweif ..I p u p OC cllildrrn ovi3ra Enttd: p t > l i t r ~ a cl n ~ s ~ r v n t ~ is < nnlore
i ~ ~ ~ I otdrr people
p r r j n ( > ~ l nantrlnj:
loiig prlr~otfit(r .;ec \\-hat changcs ha\.c takVn place in their I T I ~ P ~ , i.; an tlr,~narnonK vounscr pctlrlr. Hrriz.r\.et, tlljk rorrvlntinn in itrelf d o r ~nclt
euc+~llpnt c x . ~ r n ~136 l e a pancll tlrstgt~.I t ccmn~mc~nced i\.~thchrlclrcn ivho e\plt3in thr. link. 14 it, tor tx%nrnylc,bcu-ausc ~eOl7kbtncc?rnern(brt5con-
W ' ~ I . V\t3ven vcnrs L ~ L I,~nclha. rcvisi 1r.d t I ~ c chi ldrcn cvery scvcn ycarq srarv,itivtl ,I< they grow c~ldrr,rtr il/lcs tht> corrcl,it~onr~4l~pct surnethinfi
tinct. Tht*scb rhll~lrr,narc nt7w in !heir 40.; The I3ritit;Ii 1 lrrusehr~ld abritit thv rra In \v111ch Ihc ol~1t.r petlyl~.yrriv lip 311~1formi~il t h c ~ r
Panrl 5urvt.v pruvidcq an c ~ a m p l rnF a pane1 sturlv with more frc- pc>ltttcaI\ . I ~ ~ wFor s ? exarnplc, tlir t>ctmomicinscc~rritvc~prricnctlrlbv tbr
qut.tik data c n l l w t i c ~ n It ~ . r t ~ ' i 4 i tits
~ ~ i a t i n ~.;amplv
~ ~ i l e w r v yrar. jirricra tian growing u p d i ~ r i n gtht, Great Ilcpresqion of Ih(1 1 9 3 0 ~nlav hc
' * I Lrrnqitudinn I survey* rvilF atco 1 arv in term< I r f rcspnnsiblr inr Iht. political cnn.;crvn tism nf thc c,lcler gtw~ratirrn.
whvthcr th1.y inx~c)lvt.plannrd 'fntt-n cnticw,' or whrther tht-v reFv on I hen again, thr. c c n w n atism nt older prople mnv be d u e to ,I t ! ( ~ ~ E ( > l l -
~ b s r r \ ~ i l rtllc g resul Is elf nat11mHv ocrurriny: rrrent<.W h t w thwc nr13 ~ l r r - l ~ t cfitbct:
nl pcoplc mav brcnmc more c t ~ r s e r v a t ~ as \ ~ cthrv grow nldcr.
pTsnilccl ~nkt~rl.enticm.; the lon~itu~fin,rl dt-ilen is qimilar f r l rhe c!a*cic Tht-v ma\. hecornc~It-ss a Jt.pntun%us,nerd grca t ~t ccrt.a~nt\.or b ~ c n m c ~
c ~ p v r i r n e n t de5iqn ~~l rlurcpt ~ l i r 7 ttt~tnrtawill l,tl nu r,i1lCjt~rni7t*CI contrnl niclrc cct*pfical ot thost. w h r ~grrtlrnlst. t l l , ~ t rh.ltik:t~ will brin): ;r btatr~ar
'gi-CjLI 12. ~ ' 1 1 ~ lTIN- d . r>nlirrr.1 i,rblrh IY,IV tcl t ~ ~ t ; l b l ~ivlictllr~r
+li pc>lit~c;rt cnii~i.rt.nti~n1
i- ?tit. n.cr~lt a1 ;l tlc,\.c.I~rpn~c.ntd t h i i ~ c t to t r ~ i kthc s a m t intli\,idtr;ll.-
ovcbr I~rnt* ,ind s c ~ .li t h c ~~ l i a n c t a<
, thcv ai;tl.
Vurpoqek of loneitudinal design

Tlit'rc iq a prc>blc~ii with ;~ttc,ptirll:.i trackin< Gtralrcv. IF wt. track -1 Crtwp

Tht3FLtr1'PW ot [ ~ v ' k ~ i t ~ ~ rdtb+renc
f i n a l c , ~ tt arv
~ frt>m tht, drscripbiv~to thc 01 ifnrrng p t * o p l ~C ~ P T11nw ant3 trbct)rvt. that 1 1 i t - i . btac-l~rnt.rntlrc con-
~ , ~ p l d n ; l l t ~r\<
r y .;I dcwripti\.r> tclul the. Ialue 111 the I ( , ~ { ~ i h ~ ~dcklgn
l~n;ll F C ~ N J ~ ~ V C w c s CLInllothe stlrc th,it lhv ct~.l~i!:c in . ~ t t l t l l t l t * c .1 5 thc' i-ri<tllt tli
 5-

~ f ~ i ~ z ~older.
i i i g Fveqcinc in the ~ ~ ) c i c t yrep>rc?less
, of J ~ P ,m ~ g h tbe
-2
-1
+
5 2
5:
ir rr
7
bccon1111gmore c o n s ~ r vtivv. a ,1 f
M'l~at~t is nccc-wary 50 do tu a n w c r our quc+~tion i s to track diffcrlant ;> fA

agcmgroups nvtlr tinlc. Dr~ingthis ~.n~ilrE enable us to Fnak at ageing or

dcvc~lt~prnenkaI rffccts. T~;ltllc7.1 reprcscntq thrh w a y in rvliich w7c could
track a mngc of age cohort.: cwer timc. For each n p = cohort (those born in ...
a .;pcciflcd peritid) M-cc0~11dImk a t any changes in pr~litical conwr- .
;
\,ati<rn CIVET tht, ?/)-year ptlriod ( 1 YHO-2000). If r511t-hcohort ~ T P I \ . more .- , z-
7

cnnwi.vahve crvrr the 70-year per~odand if thev did qo at thc same rate
we roi~Fdconcluclc that thc c h n n g ~tn more conwrvativc posihon is '3
pff~bct.That iq, srorzing older re.;llltq in pci~plcbecnrning
rlr.;~c,l~~rrr?rr~t~tn/ -.
-
-- .d-
mvrc pc~litjcalfvcrlnservat~vi..
In t cxampl~,, l'rtbl~. 7.2 show\ Iliat each cc>liort tl{$corne>morr i L

cons~rvntiveby 10 poin tr rach dccade. F~~rthrrmort, r e ~ a r d l c s s(if the

:J zg 2
2 kE I
cl~cndc.,pec>pTr , l ~ e dSF-60 have a conwn,ati.;m <core of 50. In othcr --- s.5 C
C
\t+ort&,rrgardlrss ot the pibrind nr timc in history, ptwplr of a given age -.

obtan a particu lnr level OF consewn tisrn. Furth~rrnnr~,all coliurt 5 '"

1 E
display A similar Ipvel of change that is indcptlndent of thc particular "2
;g z
'I.

<: :
histortcal periru-i. Quite <imply, timr, not historical puriod, prod~lce.;
chan~c,
-:.: -
j

:,. G
-.
k l n ~ only
i onc cohort hctln backed (e.g, tliohr Imrn In thc 1950s) we
would h a w observed that as tlwy p e w older they bc-came more LL

con3rn.ative. Howe\.cr, 5r.c M-ouZdnut know whcthcr this \\.as beca~~sc* 06

L< zs 2z 7
agcing nr becausc this parkict~larcaliort was sc>rnehow di.;finctive and :
..-
&,C
bccainr lnorc conqrrvakivr brcause of some pecttliar effect of growing u p
+
h.,
<
ar,

, .
in thr* 14hC)s and 19705. To d istinpiqh behveen dcvelopmrntal (ageing) .-.
?

effect5 and hrr;tnrical (cohort /pcrrtrd 1 effect5 we need to examine 3;

Pl I 5
rnnllipfi= cohorts over timr In some ~ v n ha\,ing ~ ~ s mu1 tiplt cohort.; iq k.

cornp~ rahlc. to ucing an rxprariment,~l design w hcrc thc control jiroup

prm idraq a point of ccrrnpari~or~ to help interpret change$ in the cxprrl-
mental S r U U F
t < nnt bt~rtlnicrnr)rthconser\.,~!lvca as tlic,\ 1:rew oldtar
I f I l l r a ~ . u h ~ rdid
t h i s w o l ~ l dindicatc ihe at7hc~nct.of n dt~veloprnrntnleffect. Ilower~ci; In
Tablr 7.3 cnnscrv,ati.;rn 5cnrc\ deptln~lcln when pccrpir wenBborn. Thnsc
born tn IQY-q 02ltained a cnnwn.atrqm scc>reo! 70 in lQW and rt~rtr~rr~rt-if
that conservat~vc~tl~rnugh 1Q40 a n ~ l 2000. T l ~ c ~born e tn the 10(10.;
obtainr-d a con.w-vntisrn s c r w of 1 0 Irr 1980 and rctnn!~lr,ilat that lrvcl
 LONGITUDINAL DESlCNS
tE~rough1930 and 2000. Differences in the cnnsesvatisrn of thc agc groups
in, say, Zfl[)O is becairse nf period effects rather than because some people
are older. The nldcr group in 2000 has 'always' becn more consewative.
Longitudinal strzdies can hcIp researchers study 'carc~rs'(Menard, 1991:
15). At a purely dcscripti~lelevel we frequently do not know how various
'careers' develop. Obtaining an accurate sequence of the way 'careers'
unfold can bc crucial in developing sensible explanations of these
carccrs. For example, in the 'career' of a drug user what is tlw sequencc
of events? Do people start off using soft drugs and go on to hard drugs,
or is there no progression? Do people mix with drug users and then
hcgin to use drugs, or does drug use lead to participation in a drug
subculture? Understanding the order of events is critical to understand-
ing the pathways in to drug use and developing intervention strategies.
Work histories can bc tracked with longitudinal studies. We might
look at the agc a t which peopEe move in and oul of the workforce, and
the types of p b changes they make, and plot this history onto other
cvcnts such a s life stase events including marriage, having childr~n,
divorce.
Types of longitudinal design
This simple design (Figure 7.1) requires the collection of data at two
poinls of time from the same sample. We may make an active inter-
vention or we might simply observe the effect of naturally occurring
events between time 1 and time 2.
The simple prospective panel design is very much like the top row of
the classic experimental design (Figure 4.2). That is, the design has an
'cxpcrimcntal group' in which change can be measured. However, the
design lack5 a randomized control group, which produces the problem
of not knowing whether any change i s due to any 'intervention' by the
rcscarcher, thc elapse of time or somc other influence.
The prnblcrn af not having a conlrol group can be alleviated somewhat
where a wffiriently large and d~vcrsesample is used in the study. Such a
sample rtllorvs us to drtt~rmincif, over timc, somc pcopIc experience
'interventions' while others do not. Adopting this approach can produce
comparison gro~lpsthat provide a useful, but limited, way of examining
thc effvct of specified 'interventions'. For example, we may be interested
in establishing whether or not having children leads to an increase in thc
gendered division of labour in thr home. We wnuld begin with a sample
TYPES OF LONGTWDINAL DESIGN
T ~ m e1 Intervention Time 2
Measure variables ?
1
Measure variables
Figure 7.1 Sjlllplf ~ ) m ~ p " t i t l{el n t f ~ ldCs;gr~
of childless couples and observe the extent to which men and women
share the domestic work. Wc could track changes over a period of time
during which some couples would have children while others would
not. Wc could then group the sample into those who became parents and
those who remained childless and see whether those who became
parents showed a greater move towards a gendered dividon of labour.
We may find that the parents did adopt a more gcndcred division nf
labour.
The problem with this approach, however, is that it does not take into
account the fact that the two groups may bc different in manv ways - not
just parentl~ood(e.g. age, ethnicity, level of workforce participation).
Beci~useM ~ PI I ~ V C not controlled for thcse possible differences w e cannot
be sure what is respnnsible for the final d~ffcrencesin t h division~ of
labour. Howcvcr, if our sample 1s sufficiently largc and diverse and wc
collect the relevant Information we can statistically remove the impact of
known differences betwwn the groups and at [east eliminate some of the
alternative cxplanations for the group differences.
MULTIPLE
POINT P R U S F E C I ~ I V EP A N E L DESIGN
This design is similar to the simple two-point d e s i p but simply involves
more data collection points (Figure 7.2).
We might h a w spccific interventions between any set nf data collec-
tion points or simply collect information about relevant intervening
evcnts between each measurement point. Theoretical considerations and
previous research would affect what are considered 'relevant' interven-
ing events.
Like the simplc p a n d design this multiple point design reqlifses
sufficient variation in the initial group and wfficicnt diver5ity in
intervening experiences to enable ur to construct comparison groups in
the dnalvsis stage.
'nit purpose of multiple data collection points is to:
examine long and short term effects
o track when changes occur (e.g. in 'career' analysis)
plot the 'shape' of any change
tdentify factors that precede any change ((or nun-change)
 * Dropouts
I I

Inlt~at I + Dropouts
sample size
I
For cuample, f i t * rnrtv wish 10 (rack the wnv in which marriages change
nvvr time. Wc could lonk rlt these ch~ngesalong any nurnbrr uf
rl in~cncinnqsuch sr level.: o f intimacy, t r h o eucrci.;es ptnvcr, the wav in
r v h i r h ilomeqtic Ial7r~rrr is .;h,lrt.d, happin~*5s,Eer el< n t ctmflict and st,
forth. \Vc cn~ilrlIrask chnr~gt,sIn t l ~ e s casp-ct.; of thr rc.l;ltionship trlrm
cvhrn n ccluple b r ~ l n sto livr t o s t h c r a n d at regular it~trrvaIsthrrralter
- SJV t v e r v two ycar5. WP W O L I I ~ a l find ~ out about otl~ersliai-rgcs
nccu;rinR <ctr<cm o u r tracking \.int5 Ir c. Icvel of ivnrktorce partrci-
1 Final
sample size
1
patlcln, birth of i h ~ l d r ~ nPmV n , pcnple ~ u t -azl ~a parvnl nr p a r c n t - ~ n - l n l v
lolnlrig the h ( ~ ~ l ~ l i oorl dcllnngrd
, econnrnic circurn.;t,incc.;).
II w c wantct-l ti) cxplnrt. !tic effect rrl having n child on marital
l i . ~ p p ~ n\vtn ~ wC O ~ I ~ Lby
~ , hnckin~ couflcs rt8gularly rwcr a long prrtod,
rl~\t~n::rri*hIwttvt-rnn <hurt, rntl~li~irn and Eunq term t ~ l f ~ ~ cIZfe
t \ c ~ l t l r .;PC
i
thth t t i 7 5 , ~ n ddii\rnk that n two-point dr\rt.,n simplv ~ r u r ~ not l d pcrmit TI J2 T? T4 T5
+ Dropouts
--I - + Dropouts
ln~~ral lcDrOpouts
\.l-ht.thcr o r not ;r simpIe trzt)-point (prc-tcqt, pclqt-ttl.tl cjr a rnztltiplr.
Dropouls-

I
p i n t design i q trscx1, a deci<ion will nrcd to be madc about what fo tin

I
+

cunccrning dropcmts. The morr points a t w l ~ i c hdata arc* collected and

thr longcr the ~ l ~ a rion a t (3F a \tuCIv, the mnrrl dropnuts hrcr~rnca pro1,ltm. .......... -----
Fignrc 7.3 illurtrates a panrl c l ~ s i prvithout replacement. T1.terr. .Ire
h z ' r l kc? prc~ldcm. jrvth thi5 rir*5ih~.C h e iq that in a nllil~rpointqt~rrlvt h ~
curnulativc drnpnut nf rcspondcntq can rrksult in an unacceptabl!! +malt
f ~ n la sample. Thv other is that the final s a r n p l ~can become unreprcwn-
Aecrurts + New --
Recruits -, , Recruits 4 ,
tativr <ind Slit].: (hrvnten ~utc*rnaIvaliditv. Unrepresentntiveness qtcms
frnm t ~ + . i ,.;c>urc.t3\:h r s t thc pctlpte who drop out arc normally diffrrcnb
Recruits +
u!
R e ~ w i l s+ i
In prtrcuTar ~ Y ; I I - . from thmc I ~ * I I O remain in the stud^,; .;tcond, in mnnv
p n p ~ ~ l . l t ~ oRC'I*' n s rntlmbcrq inin and old members dcpart. E11t.n if thc
ni-i~:in;rl panel rrrnnin:, r c p r ~ ~ l ~ n t ~of~ttl il tv~pnpulation
c a< i t was ~ti11t,11 t 1 1 ~
j ~ , ? t 1 1 ' 1i ~ ' r rfrr;t
~ 1 ri r!rrhTri,a < t h ~ ,populati~ln~ t \ r lcIlan~t3.
f it ma17 \?r,cr)n~r
rtnrr7prta.;mtati\ t b . Ovt-r time, nntl depending on the i r i . ~ Iof dropout, the panrl t t r l l
becrwnc. cornpcrqrd increaslnqlv i7f d i f f ~ r c n tindir.~dnnl.;to hthoqflwhir
wt.rta ~ n ~ t ~ a lsrlected.
lv IY h t t h p r the pnrlcl maintains thp initial
rcprcsentativencss- depends on the metl~odof replacement. Given that
Onc. I\*ar of nrlrli-t-<sing thtw. prnblern. I.; tt, rrnylov s o m r tclrrn a i peoplt. do not drup o u t randomlv, r e c n ~ i t sshould bc rn,~tchcd,wlwre
rrplaccrnent for sample dropolrt.; and rcl rnclttdc net\. tvpc.; ot rn~mbcrs pnss~bFe.with drrlpnuts. T h i s can be done a t a broad It.rcl by ensuring
o t thrf prjpula111tn.In this t r a y s,?rnple <17t1 and, h n p r f i ~ l l ~repreccnta-
~, that rcpIaccment5 are of the Fame age, sex, race and cia.;<. However, $11i'Il
t~\~t*nc.\:.,Irc rna~ntaincd.TIII. lypc nf dtbsi~ni h illus!ra!t.rl In FigurP 7 4. matching will nlrvays be Ips.; than perfect and will Imd to a Ins? of
r.~n~loiiinr.;<, tIlu5 niaking it dlihcult tn cxtr~polatrwith conf~rlrlnccfrom
thr, \,irnplc.
I<c~pl.ic~.nimt alqn intrt,d~lctww r n e difticzlltic.5 with analysis. Sincc, in
Fiq~ln,7.4, many r.rf tlic wst+ a t T,, arr difftwnt rases from those it1 the
k.~nlplt.at T I , rvc r l c i ncl! h.77 c measures nrt,r lime for m a w 1ntlivi~lunl5.
-r]tl\ .~listncep i data wC7r t ~ n wfor individun!~rnmak~sanalysik of clianqe
nf X I I I ~r?rtfrivrltrirl It.i'r*l difficult. \%'emay be rt*strictrcl to l o n k i n ~at change
a t an ;lt;jircgatc rathcr th.ln J I a n indiiriclu;ll It\ el (see Cliaptcr 'I). 1 1 1 s
can br t,rrrr.nme 111 pirt rtdlecting sornt- rctrosp~ctivcinftrmn tinn for
snmc \ ari,~blcs(see br*low). We can nlso I I W certain prnrrrlurr~s tt)
rst~ni.itt*wllnt we arc3 Irhc'i,~tci 11nt.e bccn Itkr at earlier stage*. To chiev eve
th~s c7rlcl rvt9 r\.auE~lilsc lhc inh~rnlatinnwc hart. nbout similar caqrs that
I ~ acl l ~ t b t , prcsent
~ in tht. *ample from thc btyinning.

This is ,I further d r s i ~ nth,lt provides ,I mnrr systcrnatic and ranrlomizcd

w a y ( 1 1 tlc.aling with %anipltl dropout and In so doing rnaximi/es the
~ytt-rn,>l \.aljdity of thtl d c . ~ ~ nI.t 15 called a rot;ltinc panel rTc\ign.
-1 171. <l(.<rqti I - 1% 1 c k ~ 1 1 ~ I + I > In<qhxdiC<
! t h G i trtaqillrp f r e q ~ ~ ~ n
, n~It ~ a ~ k ~ r ~ = s . ,ln~I!,~i- ( W Y , 1'.irt 1'1') The p,~rtti.ul,~r;I& antayc r,f tt~i.; ,~pproachis th,lt it
Lal~trtll*ttlr-4-r ~ L I T Y CL7rcll ~ . . ;iilc such an ru.implt~.I'anel data nf this kilrd cnablcs ihcb rt~\rarcherto I7icrcaJ<rthe samplc ~ i ~ ac n. d thus rninrmi~t*
rn,ibltb lh<. ~.cs~;lrcl~t.r tcr r t - t nwrli3t happens I t ) t
h t b cn~ploymenlpatt~*rns of s a m r l c t3rl-t,r I:\! providing ~1 I,arqrg poolc~l S I I I I ~ I ~ 111~' d e ~ i g n~ l s r ~
nn i/rrtrr~rrllrol(e.g. how long i t takcs to find work, efforts tn find work, provides rnr>rtacascs of hard t c ~find Kroups (t.g,lonc mothers). In 1~ho11r
r n o v t ~ ~ l ~ i in ~ nand
t s o u t trl ~ub';ctc.) at frrtlitcnt interval< ( e . ~rnt~nthlv). . iorcc 511rv~~v.; lhc. precision id sample estimatt.5 can be partic~rlarl y
Tlica r r r h t i n ~panel clr~ign<ilsominimize< the problcrn of d r n p o l ~ (and t tlie important. h).the advantagr* of pooling t h r pGantll. for cross-sectional
conqrq~tr~n problems
t ~\.ithrpprcsentntivenczs) and burnout (drrnsnding analysis cnnnot be o v e r v m p h a ~ r ~ c i l .
tno ml1sF1of the -me yroplc). Rr ~TTlplm'i"~ a wries nt ~>rctl,~pping pane\<the rr~+cl.~rchcr can smooth
\.t:rtli ,I rotahng pant4 t l ~ ~ q i gan serieq of pnneys is establi3hc*d. Fach out-thc impact nf factors quch a* tlropout and parli.1 conditioning (.;t~
p ~ n r l Irta~ins
: at a diffr~rrnttime but ovcrlapr t~mporarilywit11 at Tcnst Chapter 8). I:t.en though p;lnc*l 1 ~ ~ 1 1suffer 1 snmc dropout and crltl-
somc tvt Ihc other pnncl.;. t3sst~ntiallya sct nf pnncls is created !ha1 have ditiontiig LIFtrlrIhc first data cnllrctinn, the impact L > ( thcsc tnctnrs can 1c.
sknggi'ribd starting and f~nisliing times. Rut Ihp same information is a s s e s s ~ dbv comparing estimakcs from h e scconil panel that will not
~oll(~c.tc~d from each ~ r l n c lThc . logic of this d r q i ~ nis i l l ~ ~ t r a t tInd f'igurc be sublrvt t o thr cffccts ot d r t y w l ~ tor conditicrntng. Thc same logic
7.5 a~plit.>at T 7 ~ n thrcru~linrit.
d Srncc fresh .;arnplrs ,ire being obtainid
In thv. ~'\arnpIf the F ~ b pilncl t (mrv 1 ) i.; rcttcqted each rnontl? fvr 5lu continu~~Tlv the rffycts of burntrtlt and d r o p r ~In
~ tthc oldcr panr.~scan b~
mnnthc. niter this panrl lin.: conrmenced anoth~xrpanel begins a rnrrnth adju<trd (crr. lwrthermore, in any cross-sectional analysiq of the prmlcd
~ r 1 5 followrd !hrt)ugh for six month<. Tcqting of the stkcclnri panel
I ~ t t ntiA saml.7Tc< thr 111clusionof n c ~ . ~ rpnncls
r can rnoderntc the hiasing efterl.; o f
wrll 171, cc%mplcteda mot1111 after Ihe first pnn(1l. A numbcr ol ht,lgp,ert.d dropout
parli,l.; <*,Inbe run s~rn [IItanctn~sly.
F'lch 1 j l 1 t t t ~ 1 ha? a Irrnitvrl Irtr, ru h u p c f ~ ~ l lpartioiyanl.;
v arc, not
c.\IiL~~~--lt.rI petlyte. Horvta\ r r , thc ~frrrlt/ can rvtrnd indcfinitr*lv. HI
tral.R~n): ,i p , ~ r t ~ v u l p3nt.l
ar 1ncl1~1dual uIi;lnr;r~r,ln trc studrrd o\ tar t h ~ t
ptar~oil. Ct~ii~equentlv. wtJ t~litalna <erir\ c ~ \llurt f tcrm panel qtudics. Rv In all tht, p ~ n Clt.5ihm5
~ l l a s ~ n n l ccrllurt
d i s i ~ ~ c \ ; t ~ iar r could bc u*i*ii.
marntajning the <hid\. c x 4 L . r a \'cry long ~ r r ~ o t, ~i l, b ~ a~i t - i t hcii f f t ~ t ~ t cascs,
it Using a sinl:ltk cuhori inx.ol\.c< st*lrctinga grouk3 oi .;~tnilarlyaged pct,plr.
aggnagntc% change ovpr tirn13 can be shcllcrl. at one pwnt of tme and h ~ l l n w ~ rit~ over g a n ib~trv~~.ltbd period. l-hihy
I lic nclvnntage 0oC 11,1vrn): cwcrlappin~:panrls i q tliat at an): point of might ct?~i.;isloC people b o r l ~in a p7rticular ytur, tlip ciass nF 2000 o r
tinlr t h y ~ n l n r m a t i ofrtlni
~ ~ nil thc panels car, he1 pcrclledFforcrow-bcct~onnl Show wlln t ~ i tlwir
l first child in 1001.
 'Thp pn*blt%rn w ~ t hii'-lnq a .;inqltn cthort, ~ 1 %~ d c , n ttl~t ~ r lrbar!~cr,i5 th,lt ~t I\ Years o l surveys
irnyt~sc~hle t hc ~r Is h a n ~ cIn the qrrlzl p i.: bccall.;~, the K r n u p
to trill ~ v h ~ a l l ~
i< c c t t ~ n crlldcr or I ~ i a l l w clrcxrvcmrI < changing. S ~ r p p that ~ e a <tud\#on
thta \cxual bcliavici~rrand krrnwlt3d~i= nf ,r Erouy 111 ?[I vcar old* cclm-
mrncecl in I Lrq(F The .;tudv rllllcctcd information about r n ~ t t t ~S Ls I C ~a%
thc It1vt.l of SCY LMI,~ctivitv,numhcr o f partnrrc, a n d knowledgr and
pmcttcr of safe qck t-lad thcu>20 vcar rrlds bwn t r a c k ~ dover r hc npxt 15
vt8;Ir\ the itr~d!. probabIv wclul J I1a1.e found tha! thcqc ynltnq pcople
bccnmc mrjrc nindernte anif 'cnnscrvativc' 5~;rr uallv itit11 lhc rJl;snge nnf
time. For 1-~amy-rlc,the study tvould prohablv find tFrnt over tinw thcxse
\-c~unr:people had Itb\< .;CY wilh Iexvcr pcnllle and adopted .;.ifpr \c\
/T~c~~C'C<
'1'0 lvhat is this cl~anct-tcl btt attrit-utcd7 I.; i t .;irnpTv a matter of ~ c t t i n ~
rdJt.r7 I t might prrtfy Ire that: ;ls penpl11grow r1ldc.r fhcv c~hktlctlorvn rvitl?
onta partner. Ht3t 11 1 l o ~ 1 I 1 - i also br ifuty to histc~ric~l effects Zl~iringt h ~
Iiiqtnrisal pc*rioil in ~ . l i i t + I the\.
i are growin,: IIIJPF,AEDS cli+c(tvt~rt~~E
'rliih ~iissc3~ervgin>.; riw Zn nurncmuG rncdin cnrnp.lign5 d c . ; ~ ~ n c dto
cI7angta qlx\i~alpr.lcticc.;. It i.; in~pr~*itrle to t ~ l horv
l much the cll;tni:e tr
rluc to a q e i n ~eifects a n d Iluw mucll i* r j u ~ .ti1 pcrlcxi e!lec7tr u n l e ~ q
cli~nct" tlrnonq .I ranee of ace g o u p < ,Trr ~ i r n ~ ~ I t , ~ n t c~uonum~~lnve i i
Multrylc colmrt rlcsiqv, ovtlrcornc this prnhlenl

The main feature t r f thr. multiple cohort ~ L ' S I ~isT t h >prt>,~rl~ rif cuhortq
T l ~ i c cprcad permit5 cornpatiqons hutt\.~cn cohurts and t h u ~hrllp.;
a g ~ i n gand p c r i ~ dcffccts.
d i ~ t i r ~ q ~ t-rct~zw-n
li~h
It i. not nccessi? tn est;lbll~hqeynmte cohorts at thc beg inn in^ of a
v . lotlg as the sample h ~ . ;a suffrclent divrr.;~tvnf aKc grclupq the
s t ~ ~ d 50
colrort~rat1 bc c..;tt~bli~ht~rl sutrst~q~ientl!. at t l ~ edata a n a l y + ~+:aKc
<
the year 2001, thow who were 50 vear old< in 2000 are now q5 years old.
T/ic sample can tell us nothing about the new 50 year old.;: thaw who
h~rn 511 i n 2005.
A Ic5* common but prltc.nti,~ll\-pn\verful drsign i s onc that usta.; nlulsiple The distinctive aspect cjf the cohort ~equentialiief;iq is that twcr limp
cohort4 and ha< a form of panel rcncrval elver hmcn(t~gurr,7 6 ) .One of cohort< drop out of thy studv. This 1s similar in way3 tcr the p a n c l ~in the
th? sllortccrminp I ~ F thc p n r l r l e ~ i pi~ that i f the panet laqh for a n rcitnting panel desim cxcept that it iq coho* that arr rr%placrdrathcr
~.utt>rntlc(fpt.rirlt? tlie 1vIic4t~~ a n t l lajir.: ,ind the pnt.1 tviF1 ntlt ~nclude than whole panels. In tht* etarnple in Figure 7.6 tve drop a rof~orta f c r
\'ounccr pt-ode. Con.;cr(zic~t-ttl~,,ah thc. panel ilxe. nt+thinq I.Ivarncif they rpach the agt. of 75.
nb(wt czmt11rnporL3r~ vcwnc pt~opltb For euarnplc~,r\.e r n l ~ h tcnrld~lct.I ttre oltlcr;t rthurt drops out oT the 5tudv thcv arc rcplncc*d with A
I ( m ~ i t u d r ~ i .ai ll u d ~(71 ;1gt.ini:. I\'ta w a n t tr, knr3~rw h a t i s 11 tikc, t t r Crutv nrr\., vclunc (70 y a r old) cohnrr. This newbcphnrt In t'ffpct \lrllst~tute>f ~ r
rdd H~IW do old*-rp c ~ p l rthink
~ al-rortt ,rqrlnt: and rn~>rt,~lit\.' 13rl pcorlc'k ~ y h at~ tthe f~rqtd n g r w a s the y o ~ ~ n cohort.
~e~t
V I ~ \ V exycriencc,\,
~ , ptrcrptitin* and [car>ch;rnge as the\ gro\\ older' 1%'~. This design rnablrs 11s tn do three things:
11avtl plenty sf time Zn track c h a n j p and decidc tn incIi~rlt= cs>hr>rt<rrmho,
tti thc \,e,$r20fl13, arc (isrd 51). 5 5 . 1d1. h i , 70 and 75. ~ V can P at trrrtl to h , ~ c k 1 It enable< ur, in each tire-war sur\*ey,tu hak'r J. nerv crocs-qectlonal
p . c 7 l > l t l throl~ghto t hc year 207;. 5 n c e n c have- d ~lFt*rent

ctrrnpart. chang~ncattitudrs Iltt<vrcntht. coh(~r!c..l.htl r!r>tllcm 1s t h a ~by

cr~hortcrse can
-
sarnpl~ni ppople aged 50,55,hO. 6 . 7 0 and 7 5 . Fach column in Fleurt1
I h reprewntq n .;cp~ra te cms+scftional rample. Wc could a n a l v ~ c

tl~isin the same way a s w e wor~ldanalvsc any cross-sectional survey
(see Cl~apter12).
2 Ol,er timc w e would build rip panels for each cohort. We could track
through those who were 50 in the yeas 2000. By thc ycar 2025 we
wor~ldhavc trackcd thiq cohort right through. We can do the samc
thing for those who were older except that the panel for that cohort
would run for a shorter time. These panels are rcprcsented by each
row in Figure 7.4. These pancEs allow for the standard analysis of
individual change. The multiple ~ n e l also s prnvide multiple cohorts
to explore thc c f f w t of ageing as distinct from wider societal changes
(see carlier d iqcussion on the advantages of multiple cohort studies).
?I It enables u s to plot thc changing experience of an age group over
timu. Because wc have rolling cohorts (i.e. we havc information on 5n
year olds in tlie year 2000, 2005, 2010, 2015, 2020 and 2025) we can
track how the cxpcrience of being a 50 year old changes over time.
Sirn~larlv,we can track what it is like to be 75 years old over time.
Does the expcritance of this age gmup change over historical time?
So far all the designs have involved following a sample forward ovcr a
period of time. This approach has many advantages as will be shown in
Chaptcr 8. But it also I ~ a sdisadvantages to do with matters such as cost,
sample attrition and the delay in obtaining useful data abnut changes in
the sample. The primary advantage of longitudi~~al designs is that they
enable 11s to establish the sequence of events which, in turn, enables us to
make strnnger asscrtic~nsabout causal order and causation.
In some studies it is possible to reconstruct data over time by collecting
all the information a t one point of time. It is possible to reconstruct the
degree of change and the sequence of ~ v c n t sby asking people to rccnll
what happened, and when it happened.
We might, for example, ask people about their current situation or
attitudes and then ask them tn provide similar material about an earlier
point in their lives. We might, for argument's sake, want to know how
the birth of a first child affects marital happiness. ,l\ retrospertive design
would ask a samplr of people who have Jiad their first child about their
current level nf marital happinew and ask them to rccall how happy
they were beforehand. This d w i p is reyresented diasrama tically in
F~hqire7.7.
Thc same approach could bc applied in the study of the experience of
growing oldcr described earlier. We might take a group of 75 year nlds
and ask thcm to recall how optimistic thcv felt about the future when
they were 50, 55, 60, 65, 70 and how thev currently feel. This design is
r~prescntcddiagramatically in Figure 7.8.
TYPES OF LONGITUDINAL DESlGN
TI 'Intewentbontavent' Tz
: Recollection of marital
happiness then
: Birth of first child
yl
Level of marital happiness
Figure 7.7 Slr~tpler t ~ t r o s p e ~ +prtlel
t i ~ ~drszgrr
~
: Recollect~onj Recollection j Recollection : Recollect~on : Recollect~on Opt~mism
I of opttrnlsm : of optim~srn : of optlm~sm j of optlmlsrn j of optlmlsm now at 1
j at age 50 : at age 55 at age 60 : at age 65 : at age 70 age 75
The problems tvith this sort of deqign for thesr sorts ( ~ mntturs
f arc
obvit>us.There will be a g r ~ a dcal
t of distortion (deliberate and other-
wise) in those recollections. Not only wiIl pcoplc not bc ablc to rccall
accurately how they felt, but we know that recollections of past experi-
ences are interpreted in the light of sr~bscqucntcvcnts and cxpericnces.
All that these sorts of designs wo11Id tell us is how people construct their
past
But this problem is not grounds to condemn all retroqpeckive studies.
When dealing with certain events, quite good and reliable information
can be nbtahcd abnut the scqucnce of past ~ v c n t s- espccialIy if they are
significant events in the person's life. From these accounts nf the
sequence of past rvents wc can r~construct'careers' and in so doing
solve some of the problems of temporal order so necessary to iievclopin~
causal explanations,
For cxample, we might want to examine the influence of family life
stage on the workforce participation of women. We can ask womcn
retrospectively about the timing of particular rvents. We would ask abnut
the timing of their marriage, birth of first child, birth of subsequent
children, divorce etc. We could also ask about any changes in lr=velsof
workforce participation such as when they entered the workfnrce,
whcthcr and \+-hen they lelt thc workforce, any changes in workforce
status (part t i m ~ / f ~ t ltime/casual)
l or the hrning of work (weekend,
evening etc.). We couId then plot changes in family life stage and changes
in wnrkforcc. participation to help identify the sequencc of cvmts. Given
that such tvcnts arc significant ~ ~ e nfor t smost pcoplc wc can reconstruct
the temporal order reasonably wcll and build up a picture of work and
family careers for women. The study could also ask these women to

rcrulllcct WIIY thtlir ~vt)rkfnrc~ p~rtlcrpntioi~
st~nnzedat ~ i \ . e npt~nt*.This
rcould pro1 rrlc furthtar insight inti3 a n y c , ~ ~ l s nlinks
l betrvccn I t k r qtagr
and workforcr p r t i c i p t i n n amnnK wnrncn.
This retrospcctivc approach is m w t rt.llab!e when we arc dcaling with
rnemorablc cvcntq. Various devices can Ile used to hclp peoplc rpcon-
s t n r c t thc ordcr in which things tcwk plncc. Wc can work hack from the
present to the pact o r focus on 'anchor cvtwtq' and thcn g u ~ s t i c ~nrnund
n
khrsc. For ~ u a r n p l crvc might aqk thc qliwtions: '?\'hen I~TIS vcm~rf i r s t
child born?', 'k'crt. vou working at thc hrnr", 'HOWrnt~chbelnre having
the baby did vou sthr wnrk?', 'Did v o ~ go i hack to work after !hc bnbtr? ,
'Hnw long a f k r w a it?' ~
Although a r c m r ~ llln kage design i.; ncrt ncccssarily a rc*trr~rppctive
design it usct~llyi.;. It is a design ~ v h c r rdata are collntcd from official
rwords. W h ~ r rthcsc* records have been ctlllcu-tcd over time ( ~ ~ ~ l ~for ally
official purpmrs) tvta can construct .I tr-rnpnral qequencc of t.\.r.li t.;. In
order to implen?r>ntthir type of de5irn \r.ra rvtwld need to hnvt. prr+clnnI
identifiers an t hc rrcords so that we c,ln makc, sure that llnk ri*cord<
for thc same yi>r<nnover time. FrL~mpFcsn l rucli rcctrrdh might hr
income tau rrstilrnx, social security rr,rords, medical records a l i i l yoli1.e
rccnrds. ny linking thcse records wp can build up a life history ilr kcrcer'
for that person.
The main shortcoming with a rect>rd linking design is the difficult!, of
obtaining stlch, d ~ t a .In many countricc confldentialih provisions pre-
clude accessing sltch dntn. Thc other main difficult\. is rhnt <itch rtrorclq
tl..picaUy cmlv cnllt.ct limited infnrmntinn and m3v not contain key
information nlhcc<l;arvfor develoyin~a n c.rplnriation for the acVqurnccof
events. Ncv~r! hclrass record link~ngcan htdp us accurately map n~rta
bcmporal seguencp elf c ~ r t a i ncvcnts and thc nature and extent 171 changes
over time. I11 this respect these rrcord.; can help teqt whcthcr the
scquence i q at I c ~ s consistent
~ with our t htwry.
AH the IongitttrFinnl desims d e r r i b r d nbclve are panel dmicns and
invnlve fi-arhlng the 5ame p ~ o ~over I e timr. R V so doing, c h ~ n q r 5; ~ both
t
an aggregatr atid an individsral Irvcl c.in bc plotted to ru5olvr iswe5
about sequcncc r l f cvcnts.
However, panrl based longitudinnl drsign~arp not the only w n v of
obtaining data o l c r t i n e - a t least a t tlic a~jiregrltelerel. 111rw <iItema-
tive types of Inngitudinal d c s i p s enfail cr~Tlr.ctin~,data a l difft~rr*ntpoints
nf time 5(1 that change.; can be plotted, but thew data are crdlvctrd from
different indivtd~~al.: at cach time point. \l'licn, at each timc po~nt,the
wnlples artmrcprt-.;tantntir.c of thp pclyi~laIicr11 the ap roach 01 uqirrg
drlicrent sarnplri a l cvch lime pclint c.111 i * i b ~ t i ~ ~b ralry1 changrs ak fllc
1YI'I-S OF LONGlTUT)IU hl. L31:SIGN
TI Interventlon7 T?
universtty course)
n g ~ r e g a klevcl. Howt*\*rr,since the d ~ t Artx
, ~ collected from diifrrent
individuals this tvpr of design cannnt b~ used to track changr* at the
ind i ~ i d u a llevpi.
TIic simple prnspcctivc panel design dcrrrihcd ~ a r l i e rent,~ilrd men+
uring a group hctr~rtbanintervention and :i~;linafterwards. Thi.: design is
himilar to thc classic t-xperimental design hilt withnut a cn~itmlRrnup.
In the _si~?rrrlntr-tlhctnrc-after cteqlgn, nlrasurt..; art. obtainrd fnrm clnc
group a t Ti and thcr-1 frrlrn a cnrnparablr grrltrp nt T? atter w m c rvmt or
~ntrrkmtion ha5 tnkrn place. cctrnprint: the twn grtwtpq ,?ftpr tlic
'.entmcntic~n'thc impact of the intrn.i,nticm i.; cstimatrrl.
Ior examplt*, nsv il-c,~y bc interested in tlrt. inlpact on mlig~nukbral1i4sof
attending univi>rsitv. WP cnuld estahli\h Ihc impact bv rn~a.;l~rin!:tlic
religious bclicfs crf a sarnplc of first-ycnr anrl final-year studrlith a t the
onc point n l timr (i.t*. n cross-sectional s f i ~ d v )Rut we would nni kni~w
whether any dif fcrcnccs between fir5t-yc~rand final-year shdcn t.; were
clue to initid1 d ~ t f r r ~ n c ' c(i.c.
s the final-yra r s h ~ d e n t smav havp been
different even i v h r r ~they were first-!car ~tudcnt5).
The sirnulatcd h c f o r ~and after d r - ~ g n would entail r n c a . ; ~ ~ r i the
n~
religious belicfs of sample of first-year .;t~~dcntswhen they cnmmr~nccd
un11-ersity.Tl~m,say thwe or four v c a v latrr, the religrn1;ui;bc*lic1fqnf a
sample of linal-ymr studcnts woulcl br nicasurcd. The t w t ~samples
would i n c l u d ~ri iIfcrpnt people. Vic palfern of beliefs of the firsl-vcar
students cou!d bc compared wit11 t l i ~bclirf patterns of the samplim of
final-year students takcn four years later.
The simulated b~fnrt--after design i s rc1pr~<entedin Fipirc 7 9.
data over timi>that ;~IIc>ws us tn tmck ~ll;ln):e
Anuther wav i3f r~bt,7i1iin~
,?t an aggrrgat~ Ievcl is the repeated crow-~cctinnald e s f ~ n .'Thi.: 1s
discussed fully in Clinptpr 10.
Summary
Longitudinal rlchignf prtiv~dra mean.; of tracking c h a n g ~nvcr tlmr and
of p.;tatrlishing the* sctlucnce in w h ~ c hrf.rntc t.lhr placr - an 1rnportm"t
mattcr t o r ,lrlr ca~rcal;lnalv<ic. Ihlf cti;lpt~-rhas rwtl~nr.d a r . i n ~ t( i t
~lilfcrcrrtw;tiJ<1.11 cr~t-rstrttcting.I I t ~ i ~ p t u ~ l i ~tud!'
n a l ,I< n rncdns i d [rack-
i11~ ch,?nge. 7 h r w irlrtudr, yrosprctik e and rctro%pwt~ve Jt.~rqns.t l r s i ~ s 8
that Fallow t h ~samr. peclplr ovrr t i m ~(panel ~ t u i l i c and ~ ) llrtrsc that Intjk
nt difttbrcnt prloylr , ~ dilfcrent
t time prlrnt5 (trend ~tudicq),\Vithin each
rvpe of rtudv thrrc are a number crf d ~ r l + t o nthnt < ncl*d tu bc madr about ISSUES IN LONGTDUTINAL DESIGN
tlir struclurc oC the ctudv. Tf~esc includl>matttxrs such as thc sethng, t l ~ c
n~trnbt-rof t1rnt.z ,I c t o ~ ~i k pf o l l t ~ r v ~try,
d the- ovt*r.~lld~rraticrnu i tliy
qttidy, thc gap bchvrrm WLII~CS,tht* ustl of active interventions, tht. way in
which ~ttritirlnis d t > ~ ln.ith.
t anrl the L I . ; ~ OF mtrltiplc panels. Thr rvav rn
t\fh~zhIllrsc Irtlrlous,r n a t t ~ r *art> r r s ~ l \ ~ c m
~ c'inv
l particwlar shtdv rcsutk
in ,-I i,aricty ot diffc*rr*ntfornis at longitt~~linal d e i q n Tht- main type5 rrt 4 n u r ~ i t v rt ~ the
f ~ n ~ ~ t h v r l o l o ~ ipr.~vtical
caI, .rnd ctl~lcali-ut~q thnt appli
Irmgituclinal design\ ~vcrc.outlined. Chaptrr F cc)rv.ide~ comv o f th(. t o rxyc.rimcntal dr7si~nscan dl.;ij appl! to Ion~ittidin~il ~ l c > ! ~ nTh15
<.
i ~ weaknrqse\ of thcst. dcsignt ant3 tlic is+ueq [lint nr.1v-I to br
h t r c n ~ t l ancl clisplcr fuci~sias rln r n a t t t ~thnt arlcl* m w t shnrplv i t 1 lon~iiudinar
itrmq~dcr~d when imylcrnt-nting Irmg-ituilinal ~lc.;ip~.;. dtbc~gn\,alth1111~l1 Inan\ 111 thekc rnattr,r.; arc not t ~ n r y uto~ thi< J e s i p .

Methodological issi~es

In the prcviou. c h a r t c r 1 itirl~cati-rlthat p,~nt.ldesigns art. rnLlrrrtl bv khrir

failurc tn include rnncf(\nlizeiI control gmups. Illis slinrtcnming I*
nianact~J,howrver, IIV ~ ~ t ~ l ivariatitln l i n ~ ~uitliinlhc pant4 to ctmstrr~ct
g r ~ w p \ ., ~ thr
t anal! .;I+ s t a ~ cfor , ctlrnparlson purprltes. Fnr ~.uamplr,thi*
p u ~ c ~ of * t a~ I n n ~ i t u d i n ~ i<tudy
l might hr lo exelmine Ihr i ~ n p n c tof
i ~ n c m p l u i m m nn t llic mental hcalth of \.ourlc p t o p l r IGravt7, 19Q7). Tht3
rnitial Irra\,e (h the +tilcly could ~ n s l ~ rtmth anrl ~lnrrnployi~d
i i ~ ~~~nrplnvc\el
vtrllne people. Ry the h m r subst>quent ivalrcs occur n number clf cle\,cl-
r1prne1it.i 12711l Iinl-P rilken place: .;rime of ihose rrhn wen. cmplo!lrd rn tlic
CIrqt wave will Iiavc brcc~rnruncmployrd; Porn<. unrmplrryr*~lwill rcrnain
iincrnylovcd; some crnpl(~\,edrv i l l rcrnain cniplovc~d;and w m r of the
~ r i i e r n p l o v ~will
d h,we b P c o r n ~cmplnvcd. 7 - l ~ c sfnur ~ pclrsibilititlq arrb
ill~rstratr=din T,~blc5 1. Tht* man* ~ \ . a i . t ~tree s hnvt), tht* mnrt. \ ,~ricdr\?ilT bt.
khc ratigc uf ~ v l < s ~ b ~ l t t i u s .
i;o Ionc a\ mcaFtlscs of mcntal hcaltli .IVC ,iv;lilabli~for b ( ~ t h w . ~ \ . ~IVC
s
can +c'lrntl idra I,! thc ~ m p s i f nf t-tr~plovnicntiunc.rnpltrvmr~nt and
rli,~ngt~.;tl-tcretjf nn mental 11er1Ftli.T h o rhht ic~ir\prot~lcrnI + that L ~ ~ ~ C R U S I *
t h v(+up.;
~ ,?rtnnot forrnetf bv r , ~ r r t l c ma..;ign,nmcnt thi*v rn,i\. be iirtlcren?
In criticnil tvay< oth1.r than kheir cmplt~ymcnthtatus. Thest>o t h ~ ditfer- r
cbncesnrav bc W.;~<*n~!blp lttr a n t di ffert-nceq in rnentaF hca Ith c ~ i tCilmP'..
r
Thr.; prt'blcrn i;ln tic rcd uced ~ o r n c w h n twith mu lti\.nria tc , i n a l v s i ~
bcacnus~.it ;ll2rwv\ the rt~scarchert o TC>TI trol for kliow~i~li t fert~r~ces t~tatwcrn
tllc grotIpS.
 LONGITUDINAL DESIGNS
rablu 8.1 Cn,rrtfry X ~ O I j Ir ~~ i ~pn?ieI
r~
irnr~t?tiol~
'Gmupq' Waxcl Wave 2 in ways that are influenced by their previous exposure to the same
t Zrnp1uyt.d I-'rnplnyed
2 Unemplc>yt.d Un~npluyed
3 Ernploved Un~rnplnved
4 Unemployed Employed
In any study that extencis over time, many events may take place in the
world in which people live, and it can be difficult in panel studies to
isolate which events are having an impact.
For example, we might want to examine the impact of introducing a
scheme where unt-rnployrn~nlbenefits arp only available to unemployed
people who engage in rlnpaid community projects for two days a week.
W e could measure the work seeking behaviour before the introduction of
the scheme and again, say, 22 months later. The problem with this design
is that any changcs in work seeking behaviour could bc d u e to other
events taking place in the 12-month pcriod. For example, the economy
might improve arid incrcastd job prospects couid increase work secking
behaviour. T h e weather might be better. Unlcss t h impact~ of all these
sorts of factors can be isoIated it is difficult to tell how much change in
work seeking behaviour is due t o the introducticm of community work
requirements.
,Changes can also be d u e to people simply growing older (developmental
effects). For example, a study o f the effect of unemployment on the
mental health of ynung people will encounter the problem that late
adolescence is a period when mcntal health problems such as depression
and schizophrenia arc more likely to develop anyway. I f we simply track
the incidence nf mental health problcrns among a group of young people
who become unemployed we risk overestimating the effect of unernploy-
ment on mental health. It is prubable that some of the incidence in
mcntal health problems would be due to a common increase in problems
in this age group regardlesq of employmmt status.
In order to identify anv distinctive contribution that becoming unem-
ployed madc to mental health we would nccd to track several groups
of young penple: thosc who remained employed; those w h o bccame
I
employed; and those who reniained unernploycd.
ISSUES IN LOl\rCIDUTINAl, DESIGN
Panel conditioning occurs when panel participants respond to questions
questions. When people remember how they previo~rslyanswcrcd ques-
tions it may influence the way they respond on s~ihsequentoccasions.
Participants may respond so that they rtppcar consistent over time.
Alternatively, they may respond in a particular way because thcy have
Learned !o avnid an extended series of follow-up questions. Panel con-
ditioning can contaminate the responses o f panel participants and either
undere5timate or overestimate the degree of real change over kime.
Experimental shiciies can minimize testing problems by avoiding the
prc-tcst stage altogether nr by using designs such as the Solomon four-
group design (Chaper 4). Unfortunately i t is difficult to eliminate this
problcm whcn using a prospective panc.1 design since, by dchi~ition,at
least two tests are required. However, there arc some ways of eqtirnating
the effect of testing.
One way is to employ either the rotating panel design or the cohort
sequential design or other designs that have replacement stratepl;.
Thcsc al tcnlative designs provide a means of estimating testing effects.
Using these alternative approaches we can crlmyare the responses of new
participants or panels with responses of panel membrrs who arc similar
in all other relevant respects. With thcn;c comparisons we can evaluate
whether the 'experienced' members are answering questinns in a dis-
tinctive way.
h some studies that have many waves wc also can experiment to
estimate the effect of testing. For example, we can vary the frequency
n s askvd. For example, we could ask a
with which particular q ~ ~ c s t i o are
particular set nf questions of some people in t~er!! wave but ask the same
questions less frequently of other people. By comparing the responses of
people who have answcred the questions in each wave with those who
have been asked the questions irregularly we can estimate the extcnt of
testing effects.
h a panet survey changes over simc can also be d u e to changes in the
way data are c~lllectedin each wave (Chaptcr 5). Varying question
wording, ordcr, interview length, interviewer, methods of collecting data
(e.g. face-to-face a t wave 1 and telephone a t wave 2) can produce
apparent but not actual changes in attitudes.
-The more the data coIlection method i~ open to the srrbjective inter-
pretations of the people collecting the data, the grcatcr the danger that
apparent changes could reflect observer differences rather than changes
in that which is being observed. Even where the same obscrvc.r is
involved that person is subject to changes and can be affected by
historical and maturational factors. Cher time observers may see the
 1 2 0 h r G I ~ D I N A DESIGNS
L ISSUES IK LLONGlDUTlNAL DESIGU
same things differently becnusc thcjl, a s obqervers, have changed in some
way. Thtrc arc famous examples in anthropology where diffcrcnt
anthmpnlogists have 'seen' the same community in very different ways.
Red field ( 7 930) and Lewis (1451) both conducted ethnographic studies of
a Mexican village 20 years apart and 'saw' the community totally differ-
ently. Similarly, Mead (1943) and Frteman (1984)both studied Samoans
some years apart but nbscrved the culture totaIly differently. Givcn the
unstructurcd iiature of the anthrupolugist's methods in these cases it is
difficuSt to be certain to what extent thc different obscn~atinnsreflect
changes in the communities or simply reflect the different ways the
observers interpreted their nbscrvations.
Since a key reason for conducting longitudinal strrdies is to measure
change it i s crucial that we observe real change rather than simply
measurement error. Using unreliable measurement instruments will
cause us to detect 'change' whew h e r e really is no change. For example,
we might measure a person's level of prejudice at TI and again at TI. If,
l~owevrr,the measure of prejudice is unskabIe, how can we tell whether
any change is due to the unreliable instrument or real change? Equally, i f
thc measure .of prejudice is insensitive it may fail to dctect change evcn
h u g h there has been changc.
Measurement error can lead to serious overestimates of the amount of
change at the individual level (i.e. identifying individuals that change).
Using unreliable instruments cause many individuals to appear to
change at least marginally. In descriptive analysis the problem will not
be as pronounced a t the aggregate level. IS we look at the mean scores of
a panel ur of subgroups of the panel then the effect of mndnwr measure-
m e n t error will be muted. Those who appear to change in one direction
will cancel out error caused by penple who appear to change in the other
direction. For example, in a panel of 1000 people the average score on
n measure of prejudice that ranges from O to PO might be 5 . If, due to
,measurement error, 200 people show u p as becoming less prejudiccd and
200 show up as becoming more prejudiced (and the degree of change in
one direction is the same as the other) then the average prejudice score
will remain unaffected.
BI! g~aphingthe change scores one can see if the ubserved change i s
due to random measurcmcnt error. Tf thc observed d ~ a n g cis due to
random measurement error the change scores should form a norma1
distribution each side of no change.
The concept of statistical regression of extreme scores to lcss extreme
scores over several measurement waves was discussed in Chapter 5.
Kcgression is no less a problem with lnngitudinaI dt!signs than with
experimental designs. In other words individuals with extreme scores
on wavp I will tend, a t wave 2, to 'regress' to the mean. I-Iigh scores
will tend to get lower and low scores will tend to get higher. When
analvsis of change is conducted at the irrdividual level rather than a t the
aggrrp~televel wc will mistakenly attribute some individual changes to
a n intervention or some other factor when it is simply an artifact of
shtistical regression. At the aggregate level we should not make this
mistake since movement From low to moderate scorcs will balance
movement from high to moderate scores (see discussion of truncation
effects in Chapter 6).
in experimental designs it is relatively easy to distinguish change due
to statistical regression from change due to otl~erfactors. Since both the
experimental and the control groups will be equally liable to statistical
regression, it will he the drfrrence In thc 1cvc.l of change (rather than the
absnlute amount) that will reflect the influence of other factors. In panel
studies the absence af strictly comparable control groups makes it more
difficult to measure the extent of regression.
The concept of mortality - thc differential drc3ppout in the experimental
and control groups - was discuqsed in Chapter 5. Different outcomes For
the two groups can be due to diffcrenkial dropout at some stage betwee11
pre-test and post-test rather than the intmn,ention.
The same problem arises in panel studies since dropout will not be
consistent for all categories of people. The importance of the prnblem
depends on the particular analysis (comparisons) undertaken. Identify-
ing the biases introduced by the dropout and then statistically removing
the effcct of such biases can minimize the impact of differential dropout.
PANELATTRITION
A major potential threat to external validity in panel studies is the loss of
cases over time (attrition). Thc longer a panel study exists and the more
waves it has, the greater the danger of attrition. Attrition is a threat to
external validity because it is not random. Some types of people are more
likely to drop out tltan arp others. As dropout is compounded across
waves the sample becomes increasingly unrepresentative.
There are several ways of responding to attrition and its threat to
external validity. One way is to replace dropouts with people with
similar characteristics so that the known biases are redressed. This is
more easily said than done. We may not know what biases are intr*
duced by dropout. Also it may not be possible to recruit the appropriate
 ncw <nrnpItx n.tmll,rr~. I f wta 115crcal~l~icc-mcnts thcn the data for thew tht- .;tl~dy7ni):ht nnlt bc* cornparabl~tn Ihosc nrnorlji ~.Fliti1 rcn n t tnarri,lgt*
nr*w rtacruils r \ . ~ l l bc ~ncnrnylt~tc,
thu\ Ilrnit~nl:~ i n . ~ ~ to
v szg~rt'gdte
r~ ~vI'I brca kdown more gcntbr;ilIy
anal! ~ 1 . i ( C ~ I Chaptcr
- 4). t-lcnrr\-tr, tlrere I* no teaqtln rzqhr rrinditiorlin~shnulrf aFfcct the r(*ltl-
I'his probltm m n he a d d r c s s ~ db v w t - ~ g h t i nthc ~ samplt* to take t ;r,rr~nl*lt*.;.
tic?n.;lfrl~b~,l.tir't+c.~ M'r m ~ f i h w
t , ~ n ttn .;tu>if h i ~ hcon t l i d tl~vorci.:.
a c c t ~ t ~ noft .rttrittm. Wt.i,shting mtailc ~tati~ticstlly boo.;ting tht. propor- produce pcwrrr adimtrnent nnionc chi tdrm Illan Itir\, conflict el ivorcc.;.
tion ot ntndtxr-reprr~riitc~fSrrvlpF In thta sarnplt* so that t h e ~ rprt~pnrtion WhiZr prticiyatinn in the \ t ~ l d y rnw rnoilrrntc tt>c IrvcF nf c r m f l ~ c t
reflt>ctq thpir pmpnrtirm in tlrr p n p u i ~ i l ~ o(C'hnpter n 'j). I f tlilc srlmplr i.; k h u c m di\*orcrdparrnts and may r 7 f f i ~tlie t ~~flustmcn ofchilrlrrn
t thcrt*
\vt.i<htpd ~ r c ~ i r a t r lthcn v i t i n n still prn~~iiit*
a uscfuI bask tor gentlr- is nil rc.;lrm why y a r t ~ c i p a t ~ o nill ,iltcr tlri, l l r t b b c t w c ~ nconllict aird
aIi~,?tion. adl~~strntmt. F r m thr>~ighthr absnltrtc levclf; of conflict and ndlustrncnt
Thc probl(*nis cauwd bv snrnpl~n t tritiun n l w can bc addrcsqed hy may not r~~fll>ct thi> Icavcl.; in the ~ r i i i e rpupulatlrm of d i v t r r c ~ sthc
acknt31vledgin~thr I71awq crist and nctqepting that tlici* will aftect s~t;- relnt~onsliiptlt*tw~cn1-onflict and adlustrnmt +fioul~lremain t~~iaftcctcil
n i t ~ c , i n t lu~u t i~paClt!' tn genrralizc ~t rhc ~ l ~ ~ cca I~C Ti C~~I. l~o w h ~rv e rn. i Cot~.;cqi~entl\, fIt15 n,l,rttcvrd~lrjcnn br ct-ncralii.cvl ttl the, ~ d e pt)p~tlatinn.
r
hhc I * X ~ ] R I I R ~ O ~Irl7rl
V b ~ n sr n ~ ~ not h t rnattcr. 11, for cxamplr, a panrl tn tt~osec i r c u m s t ~ t ~ cwlicrr
cs panc.1 cnndjtioning 15likely to I7ecomr a
studv is bin.;cd bec(1i ~ wof ttii. diqpr~jportionatedrnpout of voungcar serlou< probltm for thtx rxtrrnnl vafirlitv uF a .;luJy i t m,Iv be appropriate
peoplc (the! m o r e ;lrr)i~ncFmarc nn~tart. thcrdorc harctcr to ktytbp track t r f tc) E)IW diffctcnt dr.;i~m.A rc*pt.attrf cmqs-stnct~cmal d i , ~ l ~mav w nfkr a
nIocrtinic), thi5 age bznq only mnttcrs i f R E P is linked ~ G the J variables t h a t suit,~lllca3tcrnntivc. rotating pnnrml cle5i~n ~ l s op r r r ~ ~ d eas way n i
are Ilcing tnu;lrninrd. If' X affracts Y, rtynrillc.;s rlt agp, tlwn the age bins estimating thcb cvtent to which ccrndltit~ri~ng might bc sffrctins r c s u l t ~I t
L ~ U P I(> attritlnn doe\ not mLittt3r. 1 % "can ~ cclnildentlr ekpcct t h a t thtl dm=c this by comparing r c ~ ~ l l in t . ncn. panel.; w r h tt~o.;~, of 'c~pt-rlenccd'
rcyal Ionship br~trvemX nnd Y izrill PX i.;! in thta w11lc.rpopul,~iinn,clespitr. pat~t~I.;
2 1 ciiflc~r~ncc,~.
~ in the* ~ i g cpro'filc~of t h t , ramplc ,~nclp c l p ~ i l ~ ~ l i l ~ t ~ .
In thwe circrrmstan<cq in { v h ~ c ha \,irnpIe Ilia< ictr~~-ti is proriuced trv
attrition ma\ attect t h t h v a t t r r r ~ sWP &jrl>pxarnlnlng, \z,tA nccd tu rcmc.rr.c
strttiql.ic.~llythe effcct of thiq b i x and wta if llic rclatinnqhip btatween X Panclq can br.cclmc un rrprcwntative i ( thr population From which tllc
and \ rchrnalns.Iree Ch,?ptefi Lp and 12). I t the ri*lation4wptwtwcrn 4: a~lci pancl i.; clr,~wnchances. Ccrt'lin h-pth$of pt-uplc nj,>\ Ieavc the arcv
1' rcm;lltls aftt+r removing thc cffect nt any variable cln w h i c t ~there I;. wlicrc the r;tucly i.s k i n g cnncluctcd. As a n*<uttthc pancl mnv orFcr-
snmplr hia.;, wc can w k l y grnerali7r to tht. wider pnpulatinn despi tr reprcscn t thrw leaves.. (outmigmnts). Simi larlv, n r w tvprs of prt~plemnv
sample bia.;. movta rn to tlic area and tht. pnnel wilE con~c*i[~rcntlv r~ndcr-rtbpremt
thew inmipatlts.
Whrrc. pcyjtilation changes arc c r e a t i n ~a substantial problcnl for tlir
prcjtbctthe srdtrtion lc to 113t*r j ~ n a m i ccarnplt-. Dvnarnir sarnp1t.r can bc
The csrlier discc~s<ioncin the i.ff~cts rd test in^ ~ n it< d irnpIir;ltinn~for a c h r c u ~ din ~ r ~ w - ruavs.al Yhcy are: rotating ppant.1 c J ~ ~ r ~ decign<
ns; thnt
internal validitv ( a t t r i h u t i n ~chnngc to a n inlcrrpentitin or othrr facttjr aIlrrw for r~placemenlcrr sarnplc si~pplcmcntLit ion; ancl repca It'd crow-
whpn I I is ~ ~ a l drte l v to s i m i l v kin^ qtlrdiedl nl~rrha.: implicatinns irlr sectional ~ ~ r n - c ~ ~ ~ s .
ekterr~al\,af~dlty.J i j~nrticr;mttr.rt in a l o n g i t u ~ l m ~ l
study p r i > d u c ~change.
* I f w r use dvnarnic panels wc* need to hu clrar about the scmrcc of nnv
(because i t alrarts participants to ntattcr%the!, w o ~ r l dnthrrwise nnt think changc in crtrr depenili1nt vnriahles. Changes nn the a g ~ r c p i t elcvel in ;l
about) i t is diffici~ltto <;1rthat c l i a n ~ t wc?b~er\,cdin thth .;tudv trill take C I I R J I ~ ~ C pant1 mat- rrfltyt rinr of ~ \ Y tI l ~ i n pc-lianc(, In trrdrivri~rnls,tor
p l a c r In the xvldcr pnpl~lation. c h a n ~ c qto the, c.clrnpo~iti.nnof thr panct. l-tjr t~x;lmplr~,~ v cr n ~ g h thi-
H o w c \ r r , th~.; is mnrc ol a problrm ,it th~.descriptive thin nt thta t r a c h ~ nlevcnl.;
~ of tife hnilrfacticin in a p a r t i c ~ ~ l local a r rcgion nncl ohscrvr
P \ ~ ! N L I ~ L )Icvcl.
T~ TF ~ ; l r t i i i p n t i t ~inr : ;I pnnsl *tudv afftact.: the brahavicr~ls that, 01 rLr tirntg,thy i l ~ ~ t ~ dIVVC~I q p of 11 tv +,I tast,~~-titln
dc*cliiltw. m h ~ cilcd inc
< ~ n ;~ttjt~rdc-
d r l t p a r t ~ c i ~ n n ~t t s i \ r i l l n1,rI.e it d,fi~cnlltto j:clncrall/r. from ma!' br Jut, t o i n d i ~ i t l t ~ , i l <bt#comin,gIr-\ satiqtlt=tlrlr tv ~lt'uplc\v110 h~c1
thr. p i n ~ tc,
~ l thta pcjpu [ ~ F arm.
I H r ? ~ c \ ~ ccrn, n d i t i n n i n ~sh{bil not affect thr high I t v e l 5 c1I life sati\tlcticln Ii%.i\'ing[lit> ;lrrn;l( ~ n dt h ~ ~ r c ' 1t I~l v~ h panrll
tvav lb~ctclrs ,>rt>linked to nnc .~nnther.For euamyle, in c1rdt.t to rxplvrc At thtl same time ptt>ple lvitli low I t ~ 1 7 r 3 5 r r f < a t i ~ f ~ ~ z t irn~l!, o n rno1.c
the rnlpact ot thr 1~rcaLclnu.nid nlarriaj:r irn thc wrllbclng ~ i c h i l r l r c nn o t * into ~ h area c (and thu. ;Ire rikcruited into o u r d v r ~ a r n ~y cn n ~ l ) Ij~.clinint:
.
miglir t r x k fnmilies w h r ~are t1\pcrienclng marriagv Lbr~akijown.li being aggrcnKate Ic\.c,lq nf li fr .;atisftiction ~ ( 3 1Id1 bc 11111' entnribl v to changcs tn
pnrt elf the sturtv makr.: parcntk and ~-1i1ldr~n morp nlsrt to mnttcrs r l t thc crmrnyo~tirt>nof thc panrl , ~ n dnirt tu c h a n ~ t ~i n\ 1I1c lettbl.: r ~ Iilr i
adiuqtrnc~ntat711 ~ ' ~ l l t l c , ~t hn tq~ nthe lc>v~I< o f a t i l r r s h t ~ n !we c)bwnreIn s a t i ~ l a c t l ~c nx p ~ r i ~ n c!.I,~ ~icnlr l r ~ ~ t d ~ t a i ~

Practical issues.
In both the clrsicn and rutuuhon stage4 of R longitudinal study thcrc arc
I tlicw clthcr backlip cuntacts to find the* rcbcpnndent. Public rcctlrd.; can
m a n s practical rnattcrs to scrl~re.Somc of these arr to d o with ma\irnizin~
internal and cxt<*rnalvalidity. Othrr.; h a v r more to do with tlir cost OF
implerncntin~~ i i f f ~ r c idesigns
it and thr practicalities of gettiill: tlic siudy
completed.
I
S ~ n c echange i< thi. core focus of Inn~itudinalstudies rve n w d trr be sure
that obscncrl c h a n p ~reflects real change rstlrcr than chanqc- in tht*
data arc c n k l ~ c t ~ Idt .iq eqsential thcrcfnrc to ~tnndardizcthc way data are
cchlccted for thi. ~ ~ i r i o uwave5 s of thc study. I t is necessary to stan-
dardiw tlic rncthod hv which data a r cnllccted, ~ the way qitcstic~nsare
asked, the order in wiiich thcy are asktad etc. Wc need to clcvclop clear
and cnns~stcntmlc.; n h u t what to d o in particular s i b a t i o n s and rcli-
giously f~llorvthcm. The time nf vear at which data are crjllcctcd may
nccc! tn tv ct)n.;r~h=nt fc3r each wave. Bus\- times, glmmv t1nrt.s n i the
\'ear, tirncc (?I .;!rtxsr { r . ~ Christmac,
; ~ n rdj t financ~alyear), \.ac,ltilrr~timp
Inay all atfect thc k ~ n d sof rcf;pon<r<wr PI.
Jt is not nlwavs pos5ible to a c h i r v t ~ t ~ i n d ~ l r d i z a t iThe
o n . Ivss .;tructurcci
the data collcciioii method the more difficult i t is to stan~iardizrpro-
cedures. Whl*rt* more rluaIitahve and subjective techntqucr; S L I C I ~ ds
ohscn~aticmand ~rnstntcturedintervlcws are called for, thc ~ r c n t c rthe
~ossibilitvthat changcs may be dup to thc FPOFIC collect in^ thr. data and
the rnrtlmdq thcm~r~11.e~.
C o m p l ~ t eqtandardi7ahon is not alwavs dmirablc. For examylc, it can
interfere with our r p ~ t a r c hgoals in lifc crrurse studie5 in tzthich pt~ople
arc tracker) as Ihry grow older. In longit~ldinalstudies peuplc's clrcum-
stances chanjir 417 we may necd to aciilpt our indacato~sto reflt=ctsuch
changes. In thtb same way that tlic indicatnrs of i n t e l l i g e t i ~nrc ~ age
specific, sc) our ind~catorsof change may also need to be l i f r stage
specific. Mcaqi~rc.;elf family w e f l b e i n ~and cnhesion mav well nccd to he
adapted according to the srape in ithp farnilv life cycle.
In o n ~ n i n qv n v 1 studies there rq nftt3n a cnre oi standard rncazllrtSs
cr~llectedin t ~ n c rz7,;x.c
l~ ot the 5hdv. In ,~clditittnmodules ~vhichd l ~ n g e
frcorn time t r ~timr rErts.IISO f+equrntlv i n c l i ~ d ~ d .
Sample attritioi-r prod uccs problems with both sample size and rcprcwn-
tatir,cness and tf~tlrt*fnremust be rninirni f e d .
To rnin~rnirrattrihcm we muqt kecp track of panel r n e m b ~ r sTrackrng
dernnnds considcrablc resources. A rangr of t c r h i q u e s is ttwd tn a s ~ i q t
w t h track in^ prclp!c. Additional infnmmntion is collect$d a b u t parents,
15SUES IN LONGIDU I'I N AI. I3ESIGN 139
thrldren o r wc~rkplnctw that it thc rt-qpondcnt mores rvr can ctwtact
also be uwd T ~ l c p h ~ directvrie~,
~ne e l t ~ t o r a rolls
l and ~ u b l i c l va\ratlabl~
records from IIK.~II cr~uncils can help IOC'R~C r ~ s p o n d ~ n\vhn t ~ Ii;1ve
moved. Of course, i t is easier if panrl participants tell 11s t11c.v hnvc
rnovcd. A t each intrrview and any othcr cnntnct i t is helpful to provide
change of address notification cards to participants to mail back tcl the
.;tudi manasrr.; i f thcy move. Keeping u p c o n t ~ c hetwtvn t suntcv wJves
is anither helpfrll strategy. A useful tracking device is to send n birthda!,
card to participants. I f tl-tis card i> 'rcturncd to sender' we arc atcrtcd to
the need to st,lrt tracking them down ~ n ~ m r d t a t e l Thc y . card dl.;(l pro-
\.ides a n opprlrt~~ntty 'to scnd a c h a n ~ tof address card and ;tct.; n s nn
acknou,ledgrmmt of appreciation fnr part~cipntingm the httrdy.
To srducc n ttrilticln i t is essential to maintain the respondent.;' motiva-
tion to continuc to participate in vrdcr to rcduce attriticln. !'mvid~ng
incentives such CISrnnney for the t i m ~pcople qpend on the s t i d y, .;malt
tcrkens of thank< or summaries of key wrvcy rcsdts can hclp achieve
t h i ~goal. WhiEr rnnt~rlnl incentivw can hrlp maintain participation,
Frobdb!v thy rnrrqt citcchrre ~ncentivc I < for pcuple to f w l t h ~ ttheir
~nrtisiya!ion r< both important and cnn<bttclivc. The morc pvoplt. !PC!
lhev are participant.; ~ n pda r h e r s in rc*sc;lrcli ra tlier than 'sublcct.;' the
better. T ~ nvcrridillg
E ~ ( 1 should
~ 1 bc ta nmkc parhc~pationin t l ~ t study , a
positive exp~ritlncc~ rt>t1-trrthan an unwrlcnme burden and intrusion
(Dillman, 1YTH).
Attrition is a t - i i ~ g ~problem
r in somt- kinds of st~tdiesthan othtars.
Sh~dicsof peoplc w-hcr are highlv mobile, marginal or membcr.: o f
'extrerne"roup5 can cxpcricnce high attrition m tes. For e~arnplc.stutlieq
tlf parenk aaftcr clivcirccc inrariablv crp;lte the probPern of m a ~ n h ~ n i n g
contact rvith non-rcqiclent parent< - especially non-resideni f ~ t h c r < .
Longitudinal s t l ~ d i c sof potip': such as criminals, drug USPrF arid some
S O ~ ~ ofSuncrnptcrycd people havc very high attrition rates induccd by
high levels of niubility, illness and dcath. Panel studics of prnple
e~periencinglifr transitions can also havc highcr khan average attrition
rates, Researchc*rs investigating young, pcnple becoming ad (11 t, ynung
pec~pleentering Ihe labour market, per~plcafter divorce, or Itho<e p r o p l ~
who leal-P the IsElnur force all face thr prrrblrm that people expcrtcnctng
theqe transition.; a r c hiqhly likely tn c h n n ~ nddrt-5.
c As a r c ~ u l tihcv s;ln
be lost to tlic panel studv.
Panel surveys can impose a considerable h u r d ~ non participants. Thi.: is
especially so i f rach wnve uf the shrdy i s closta togcthpr or i t inttkrvitaws/
qir~5ticmnatreqarc- Icnqthv or mtnis~vt*.I-1~a1.yrespondent Ptlrdrn rs n
n a Ims of data quality
problem b~caustli t can prtduce hirh a r t r ~ t ~ oand
as the burden increases.
 140 LONGITUDINAL DESIGNS
Therc are a number nf ways of reducing rcspondent burden. Some
questions d o not need to be asked at every wave: every second ur third
wave may L7c sufficient for some questiims. Alt~matively,a set of qucs-
%ionsmight be asked of only half the panel and a different set of
questions put to the other half. Although we have f ~ w e people
r answcr-
ing the questions we map stiIl havc sufficient nrtrnbers for meaningful
analysis.
Rchuspective panel design? rely on respondent recall. This has two
problems. The first i s that pcople will recimskruct past evcnts in the light
of subsequent cvcnts. This can result in distortion and reinterpretation
by which recalled events bear little resemblance to actual events. For
example, a study that asks people who 11ave divorced to describe aspccts
of their marriage will be affected by the divorce - by the nature of the
divorce, who initiated it ctc.
The second problem with studies requiring respondent recall is tele-
scoping and reverse telescoping (Menard, 1992). People can rnisremem-
bcr and think that events l-tappenrd morc sccen tly than thcy actually did
(telescoping) or longer ago than they really did (rcvcrse telescoping). A
number of techniques can be used to reducc this problem. Getting people
to Focus on certain rn~mclra'tllecvcnts and constructing a calendar of
thcse key evcnts can providc anchor points to help remember the
sequence o f events more accurately. Fur txarnple, we might be interested
~ between family life stage and the work patterns of mothers. To
in t h link
provide anchor points we cor~ldfirst ask mothers to provide key dates
such as the year of their first job, the date of birth of each child and so
forth. Since thcse are likely to be dates that can be accurately remem-
bcred we can then, piecc by piece, reconstruct how long after these
memorable events other, perhaps leqq memorable, events took place.
Panel studieq are typically rnt~chmore expensive than a single cross-
sectional study. 'Thr expense of panel surveys sterns from the use of
multiplc data collections and the cost of tracking and maintaining the
panel.
However, if the choice is between say a five-wavc panel study and f i v ~
repeated cross-sccrinnal surveys thc cost disadvantag~of the panel
design is less obvious. Tncre is some evidence (Buck et aS., 1995) that the
cost of panel maintenance may he no more than the cost nf recruiting
new samples. Furthermore, the cost of collecting the same background
data in each cross-sectional surrey will limit the amount of other
I
information that can bc collerted. Ln a panel design most background
ISSUES IN LONGIDUTINAL DESIGN 141
information needs only to be collected once (in thc first wave} thus
enabling us eithcr to collect extra informatinn in suhsequcnt waves or to
reduce the length (and cost) of subseqtrent data collections.
UltimateIv the cost efficiency of the panel design will depend on the
difficulty of maintaining the panel - how much time is needed for
tracking and the cffnrt rcquircd for 'call conversions' (convincing panel
membcrs who initially refuse to continue).
The cost will also depend on the method of collecting the data. Of the
three main methods of collecting data in panel studies - tace-to-face
interviews, telephone interviews and self-administered postal ques-
tionnaircs - face-t(>-faceinterviews are the most expensive. However,
they tend to yield the best response rates. National panel surveys
typically rely on some form of cluster sampling (Moser and Kaltun, 1971;
de Vaus, 19%) where scts of interviews are concentrated in a number of
geographical areas. By this mean5 it is financially feasible to conduct a
number of face-to-face interviews in the one area at the same time as
maximizing response rates and minimizing attrition.
Chrcr the course of a panel study face-to-face interview5 m a y became
less financiall~~feasible. Despitc cIustcring inter~iewsinitially, in parti-
cular areas high rdes of geographical mobility can lead to 'dccIustcri11g'
or scattering of the sample, thus increasing interviewing costs con-
siderabIy. Fortunately, deveIopmcnls in tclcpllonc based interviewing
(see next section) can alleviate decIurtering problems (Buck ct al., 1995).
However, changing from tacc-to-face to telephone interviews can intro-
duce instrumentation problems as d i s c ~ ~ s s cearlier.
d
There is nothing intrinsic in any research design that dictates that the
researcher use a particular method of data collection. Tn principle, data
can be collected using structured qucstionnaircs, observation, structured
or unstructured interviews or any other methcd.
However, when it is proposed to utilize a panel design there are real
advantages in using computer assisted interviewing techniques. Here
thew are two a1ternatives available: computer assisted telephone i n t r -
viewing (CATI} and computcr assisted personal interviewing (CAPI).
Therc are tWn advantages to using computer assisted methods in
panel designs. First, these designs can involve extremely complex sets of
questions, with many qucstions being relevant to only small subsets
of respondents (depending on their charactcrisbcs or the patterns of
change we have observed). For exampl~,wc may require a special
module of questions for families that had experienced divorce between
waves and require diffcrcnt questions for r e s i d ~ n t and nor?-resident
parents. With cornputcr assistcd in t ~ riewing
t m e t l i ~ d sall the possible
qucstions arc programmed inlo the interviewing package. On the basis
of answcrs to earlier questions ihc cornputcr selects which questions
 1.UNGITUDINAL DESIGNS
the interv!c\vt.r necds to ask. This simplifies the task of administering
con~pIcvquestionnaires and reduces recording and interviewing errors.
A further advantage of computer assisted inknriewing i5 that in the
secmd and subsequent waves the answers from previous waves for that
respondent can be fed into the intenricw. These can serve as excellent
prompts that provide respondents with reminders as to how they have
previously rcsyonded. For example the CAT[ or CAPT interviewer can
say, 'Last time you said . . .' or 'Last time you indicated that vou were
planning to retire. Have you actually retired yet?' Not only does this
provide respondents with a reminder of how h e y were feeling East time,
but it personalizes the interview and makes the respondent feel that their
previous answrrs ~ R V Cbccn taken seriously.
Nu nr bur nJ wavcs
The number nf waves utilized will partly dcpend on the purpose of the
study. [f we are interested simply in 'before' and 'after' a particular cvent,
two waves should be sufficient. If we are trying to track a process of
change, multiple waves may be needed. For example, if we simply
wanted to knnw how well people have adapted to retirement, say two
years a Ftcr retirement, h ~ waves
o may be all that is required. If wc wan tetl
to track stages of adaptation to retirement we might require multiple
waves to detcrt the ups and downs of the process (see Chapter 7).
The number of waves will also depend on funding since each wave
adds comideralsly to the cost of the study. Respondent burden must also
be c~msideredwhen planning t h number
~ of waves.
Many large scalc prospective panel surveys adopt an open ended
appro ad^. Funding is available for an initial set of waves and decisions
about how Ic>ngto continue with the pane! will depend on the capacity to
sccure hrther funding for subsequent waves.
Setting the gap hetween waves will bc affected by 'political' considera-
tions such as the pressure to come up with some results. The nature and
content of the study will also be crucral in setting the gap between waves.
Studies of high frequency events or processes of reIatively rapid change
may requit-c only small gaps between waves. The theoretical model
being tested will also dictate the gaps. For example, in a study of adap-
tation to retirement that anticipated different phases in the retirement
transition le.g honeymoon in first 3 months; distress by 6 months;
rebuilding new- identity by 72 months; stabilization by 18 months) thc
model would dictate the gaps between waves.
Where changes are expected to be gradual there i s IittIe point in
f'reqiient revisits. The famous '7-UP' scries, which tracked a group of
people every seven years from the age of seven, is an example of a long
term study with large gaps between waves. I
ISSUES IN LONGlDUTINAL DESIGN
In establishing the gap between waves, respondent burdcn and panel
must bc considered. Short gaps can exacerbate these
problems. On the other hand long gaps can create tracking problems and
increase attrihon.
Sample error is, in part, a function of sarnpIe size. The smaller the sample
the greater the chance of sample error. In prospective panel surveys
(without replacemcnt) sampIe error will occur when the initial sample is
selected. Subsequent attrition may produce bias and will increaw sample
error as a rcsult of reduccd samplc size. If the sample is replenished
sample error due to overall sample si7e will be controlled. I-tow~ver,
~eplacemcntwill be based on new mini samples and these replacement
mini samples will be subject to sampling error and can therefore aifecr
the precision with which we can confidently gmeralize our result
beyond the panel.
Sample sizc creates a problem with any type of desip. 5mall samples
encounter problems with sampling error and with insufficient cases in
special subgroups t c ~allow meaningful analysis.
In studies of change and transitions, for which lon@tudinal shdics arp
cspwially useful, the actual number of people in a panel who experience
a particular change or transition can be quite small. For example, we
might want to use a panel design to examine the effect of divorce on
children. If we used a general population panel of households the
number of households that actually cxperiencc divorce in a given period
will be yuite small. For example, on the basis of the Australian annual
divorce rate of 1 2 per 1000 couples (i.e. in any given year 12 out of cvery
1000 married couples will divorce) we would require a very large panel
indeed to have enough couples divorcing in a year to allow meaningfu1
analysis. Even if we had a panel of 5000 couples nnly hO couples would
divorce in a year. I f we followed the panel over a five-year period this
wouId increase tn something approaching 300 couples. But even this
number may ccmstltute too small a sample if we wanted to look at
divorcing couples with young children. The numbers would be even
smaller if we wanted to look at fathers of such children whu had primary
reqponsibili~for caring for the children.
-Where uur interest is in small groups we may need to refine the initial
panel more (e.g. just couples with young children rather than a random
s a n ~ p l eof hnuseholds). Alternatively it mav be possible to oversample
certain tyres of people (e.g. couples with young children) to maximize
our chances of obtaining sufficient numbers rrf pcoplc who subsequentll~
experience the changes in which .rut arc interestcd.
 Thc qi/c r,f thc iniiiol G J ~ Fir~ill
I P ,11.;t> ilct,d to taktb t ~ r c o n nof
t tht. likclv implcmi~ntcd.Sincr th1.s~ principlthq wcrc discu.;scrl in Chapttsr 5 this
attrition %in(-rattrition tv11I Jepcnd tn p , ~ r ton the dumtion of thtn .;tudy section simply hiyhlrglits matter< th.~t arise particulntlv with Iunqitu-
and tF~cnt~mhtlrand irt,<ltrvniv at IV.~\I..;. the i n ~ t ~ pa l3 ~ r slzel uv~llntrd dinal dt,.;igns.
to takv ,iccciunt of cqflRT,TI(YIiittriti~nr.lttT.;.

G i w n the need to minimize panel attritir~nthere is thr danger that ~ r n d u e

On tht' fiwt ~ ~ e 2L I >,~% ~>r(~c;pvctil-c. pinrE <tin-ey c ~ crvill typically qpmd t ~ unrth~c,ilnleans arc crnpltlyed to crmvjncc ptwprl~lto ron-
p r c 3 ~ i 1 ror
quit? ,I t ~ of t t i r n t abtaining inir7rrnatlon such a s ~ t ~ r c w i e r n n q r a ~ h ~ c , tinuc to participatr in each wave t l q the 'inveqtment' in cach pewon
housr!lnTd , ~ n dhi<tc+rrc,llin tormation. Thik M-auld incli~desuch thing:. g r n w ~3' they participate in mtlrl. warmes,She ' c c ? ~nf~ 'penplc d r ~ r p p i n g
yc;lr 01 nl~lrrlagc,act3 J I ivl'tainin~, l~rstj ~ r ' t letc. Qn .;uh.;cq~icnt wavcn\ we out incrr;lse~.N ~ v r r t l i c l c ~ srnlunt,irv
, participn~icln must rrmaln the
wiPI nyt*~lto c l ~ c c kwhthth1.r romp things l-tave cl1,ingcd ( c . g . nl,~rilnl bar;is on which pcrrplc continue to pariicrpatc.
stattr*, anv ~-h,lngc<to hrrusc~lioldcornpositinn, ~h~1ngc.i In inh etc.). Tht clumtinn also ariscs as to ~ v h ~to l t do with ptwple whrr rcblu~cto
The ~ l r q ~ qot n in\trumi3nts \\-ill {,at-1. rfcpending rln the mcthorl of participate In clnc walre. Should t h a t p r ~ c l u dtr?.inq ~ tn seurr~it thcm
cnllectiti~~l;lta.Ccrrnpufrri7ed methrrdk alloi\. for rt~rnplr,:queqtlonnaire again ~n ,I sub~equcntw i l ~ c ?At li.hr1t point doi.s r p t u m i n ~t o prior
structurr,< n ith s u b ~ n l u ymodulrs ( w c rfiscuqsinn ot CAW a n d CA 1'1 re!uwr.; bccomc. Iiara~sment?
nbovc) r ~ h i l t . wlS-adm~nrstered qt~c*rtionnairesnt>rcf to Ec ~ i r n p l c .
Instrumrnt dcsign will cal.;o bc afkcctc~ltry dccislnnq about the us(* of
core clur-~tirm<, rotatin$ rnnijt~lr<ant! .t*t.; r>i yuezticln* K1t.t.n tci tnrllv haFf
thc ~;lrnl71i-l h t ' t ~edrl t ~ <r ) I , c I I + ~ c \ ~ ) r\t rvh,lt pe*int >hc~tllrlparticipant.; 1.1.,ickcd for lita air ct+nccnt7 In a <tutFy
that c,in crtcnd iclr rn,ln\ years, d r ~ *strnbent
c at thr Z>t*ginnrng1,151 lor the
rtmainrltv of the 5t ud\ ri~gnsdlrs:. cvt hot\. many u v . l \ ra.; ~t c v c n t ~ ~ ahas?
ll~
O r is informed consent rctluitcd a t ravh wave?
A I o ~ ~ ~ i ~ u i studv
l i t ~ n ilt ill h ~ n r f i gtCLlt
t lv trom thc i~mployrncntof rirll- A n ~ntcrc<tlngqiicsticvn ariqes in ticsigns ~nvr~ltring rectrrd litlkngp.
trained st,itf who ha\(, ;rn ongoin< involvement w i t h the shturlv. II' a
Althougli privacr. law.; In many countrres prevrnt quch Ilnk,>cr, thew
person I < rr~tcr\,rc~i,cd the qamc r n t m l e w e r ovrr t~rnc*there* I < niore
devcn5 nFso patentiallrr contrarrrnc the principlra ot infomeil crrnwnt.
chance of 1111%respondt.nt c t m h n u l n ~in tlic study h.causc nt a f e ~ F l nof~
Even ~ l i n u ~a hprrson may havr- givcn informcd conscni ~ v h they r ~ ~
cnrnm~tmt>nttn the pcrsorl as much a.; to the study (IZose et al., lL)YI).
providrd tbach srparatr. set of ~nforrnation,this 1s not tlrc snmc a s
A wcll-coiiducttd p,~ncblstudy will rrcl~rirrpeople w h o arc particulnl'l~
agre-wng that the dL1tnbe linked srncc thiq is a lrcrv different u~ of the
45lled at w n \ . ~ n c i n gp'1nt.l members ~ v h om a v be inclincd to drop cvut of
data ant! is unlikclv h) be what the inslividuat rt,n.;cnted to.
the study t ~ continue.
l I'crst~adingpanel rnembcfi ~ I i ~on ~ t i a l Irefuse y to
continur, in !hr study I < c~IIJPCI ' c t l n ~ c r t i n ~ '
refuwlq. I t take5 p a r t ~ c ~ ~ l a r
skills tcl i~ndcrtnkeh 1 . s task ~ l ~ c c e s s f ~ wrthout illv contravening thc cthic;ll
principle.; (h voluntary and informed conqent. In txpcrimcntal d ~ s i v in s which thrlrc is an activtb interventrun t h t w is
Data rnnllagrrncnt i~ nlqo crihcn! to the succeqs clF r7 large r;ralr. pancl an ~ t h i c a lsequirernct~tthat bhis intcn~t7ntiond(w<not evpose participant.;
s t ~ ~ d rSF;~Ilr.li
.. data rnantlCt8rs r4,ho .tort* d a t a , p r o p c r l ~match data f r t m to h a m . T h e same cnnccrn can arj.;r in panel dc'i~qnsthat ~ n t . ~ d van c
kucrrs.l\ I, ;1\.c5 and rtt*l.~lup t r a n ~ p . l r ~ ndata
t manaKrrnent yrr1tnct~Ts act1r.r ~ntcrventicv~. I-l~~tvcver,panc.1 \ t u d m mt>ri>typ1c~T1\-rrl\*on the
r et i Finall\-, ~ l ~ .innl!,sis
c= 171 Ir rnqrti~rfinald a t , ~c ~ l nbwnnit* \.t>rtJ e f i ~ r of
t 'naturallv crcci~rring'event.: #tnd thu5 d o I I r n~t i f r ~ n ttht, pr1.b-
romplcx and project mt,mllcrs who mrl prnpprly h a n ~ j l rt h e analvsis w ~ l l Icm (11 Flarrnful intcrvcntions in t l i r * samc waV Ilia1 t l ~ rc u p c r ~ n l c ~ i l c r
h e inipcrrt,~nt. does.
Flnrvvirr, the rc<rarr.htbr ma\ xvt,lt <t>ntrc>nt\ltrl.il~onc in ~ v h i r hthrv
Ie&-n of individuals M,IIO nrc c~pclqt-rllto harm l'rw tam-npl~.t t l l . c t ~ ~ d v
Ethical issues ma\. i d m t i h a pt.mnn w h o iq ~ u i r i d ~ l l lrleprewcd
v or fhey nItl\, uornt.
acros3 caws of cl~ilclal7usc. The rr.;ranrchcr must dccidt~whnt I t ) i l o in
Longitudinal d ~ ~ i pc{)~it m r(mt tlic G ; l r n t X ctliical i \ c ~ t l \ s A< a n y t1tI1t.r such c;r.;c.;.There mav hr the temptation to do nothin): qince nnv onward
deqi~w.T h a t 16, the matttar<o f ~ ~ o l u n t a l,.~rticiyation,
n, thformed cclnsi>nt, referral for other hclp mav jmpartli~r-thp ongo1nj: part~cipationnf the
nn h a r m 2 1 1 p a r t i c ~ p a n t s and c o n f 1 t l t ~ n t i a l i ~ / i l 1 1 ~ v n v n 1 irntt\t
t1' be p e n o n rn flrc pancl. l'urthcrrnorc, \ ~ t i - l 1 intervcntrrjr~mav 'cnn~.~rninate'
 146 LONGTTUDW AL DESTGNS

My own view i q that the wcI1being

the research by changing L-rehavicr~tr.
of participants shrr~~ld take priority and that, by inviting people to
participate in a study, the researcher takes some sesponsibility for the
safety and wellbeing of the participants - even where it is people other
than the investigator causing the harm.
DATA ANALYSIS IN LONGITUDINAL
DESIGN
Since wc necd to be able to match cases between waves and to track
cases it is impossible to assure participants of anonymity. However, it is
necessary tu cnsurc confidentiality. Apart from normal guarantees not to
divulge information that would compromise any person's privacy, there
is a need to ensure that data are securely stored. Since we need to be able
to match rcspnnses with information that can identify respondents, it is Many of thc matters discussed in Chapter 6 in relation to the analysis of
important that data arc stored in such way that no unauthorized person experimental data also apply to Inngitudinal analysis. Rather than repeat
can match this identifying information with other responses. This can be that discussion, this chapter focuses on a number nf analysis issues that
achieved by storing the information in separate files with only a very are particularly pertinent to longitudinal analysis.
limited numhcr of people who have the information necessary to match
thc hun files.

Summary
Wlulc experimental designs are ErequentIy not Feasible in social science
research, Iongitudinal designs provide an important way of tracking and
understanding change a t the individual and societal level. Whilc longi-
tudinal designs can encounter some problems with internal validity
because of the absence of randomized control groups, there are ways of
minimizing these effects. These have been discussed briefly and will l-re
deveEoped more fully in Chapter 9, Other potential problems with inter-
nal validity have been discussed and ways of structuring longitudinal
designs to overcome thesc problems have been outlined in this chapter.
i, As il general rule, longitudinal designs tend to have bettcr external
validity than experimental designs but the threats to externaI validity
remain and must be efirninated. The chapter outlined these threats and
indicated strategies for minimizing them.
I,ougiludinal designs can be quite compIex to implement and involve
a widc variety of practical deciqions. The practical issues and ethical
issues that arc particularly pertinent to panel designs wcrc outlined.
Missing data
Missing data arF a problem for any s t ~ r rof analysis but are particularly
problematic in panel analysis. There are two types of missing data: item
non-response and unit non-response.
Sourcijs of missinx data
I T ~ M NON-RESPONSE
Ttem non-response occurs where individuals do not respond 50 a parti-
cular question. In somc cases this is because they are not required to
respond to thc. item (e.g. questions about a spouse for people who are not
married) while in otl~ercases some people may refuse to respond to a
question (e.g, income).
This is a problem that is peculiar to panel studis.' Since panel designs
jnvolve collecting data from the same individuals on several occasions
=mdmeasuring change over that time, we encounter difficulties if wc fail
to collect data frvm the case on the second or any subsequent occasion.
There are turn korms of unit nnn-response. One is where people drop
out of the study altogether. The other occurs in multiwave panel studies
where some people miss some waves but are not permanently lost to the
panel. This is called wavr non-rpstxl~~se
(Rose et al., 1991).
 LONGITUDl l\lA L DESIGNS
Missing data represent a prol-hcm for two main reasons.
Item non-response can lead to a substantial loss of avaiIabIe sample size
for data analysis. This is especialIy a problem when constructing scales
or doing multivariate analysis where rnlssing data on any one of a set of
variables leads to the loss of that case for the analysis. This problem may
quickly compound to produce an unacceptablc loss of cascs. For
example, suppose that we arc conducting analysis u5ing only three
variables and that for cach of the t h r c ~variables 10 per cent of people
had not snswcrcd the question. Suppose that for each variable a difruent
10 per cent of people had not answered. We would then have a loss of 30
per cent of cases.
The problem of item nun-response can compound over waves. This
can he illustrated with an extreme cxarnpIe. Tmagine a five-wave panel
study in which an individual participated in each wave. We may have a
set of 10 question? that form a prejudice wale administered in each wave.
I f R person tailed to answer just onc item in one wave we map be unable
to construct a scale scvrc for that person in that wave. We ~ ~ o u l d
therefore have incomplete data for that person which could force us to
eliminate that case from the analysis. Because of one piece of missing
data a great deal of data about a particular person woulrF need to be
discarded. Attrition of this extent can lead to a sc.vpre loss of cases and
daka. As a panel matures the problem created by missing data can
worsen sincc the chancc nf an individual having missing data on a
relevant variable a t some point increases as thc number of waves
increases.
, 1991: 19).
Missing data can also introduce bias into the sarnplc. Where iten1 non-
response is systematic rather than random (i.c. certain sorts of pcople are
more likely than others to refuse to answer questions) unrepresentative
samples can bc. the outcome. For example, i f higher income earners are
more likely than Iow'r income earners to rcfusc to reveal their income
we a r c left with inaccurate income eqtimates. ConsequenMy, any analysis
in which inctlme is a variable will be affected by [he loss of cases due to
item nvrn-rcspnnsc.
Unit non-response will introduce other sampIe biases. Those peopIe
who drop out of a panel are unlikely to be the same as those who remain.
Loss of panel members may be due to death, gcograpliical mobility,
changed family status, or some other type of dtanqe in a person's
circumstances.
DATA ANALYSIS TN LONGlTUDlNAL DESIGK
1dt.n tifying nrissil~gh t n bias
En dealing with the problem of item non-response we can check whether
non-responders arc different from responders simply by comparing
them on other variables. For example, we could chwk whether non-
responders to a particular question were any different from responders
in terms of age, income, education, sex, ethnicity or their attitudes on
particular issues. We can also check fur bias due to unit non-response.
Une of the advantages of panel studies is t h a t when a person does not
respond in a particular wave or drops out altogether we still know a
great deal about that person from previous waves. We can use this
information to see if those who drop out are different Frnm those w h o
continue. If they arc we can ernpluy various strategies during the data
analysis to make adjustments.
Even if wc ldentify bias it is nor necessarily a problem. Bias is a
probIem i f we wish to makc generalizations and arrivc a t populabon
estimates for the variables on which there is bias. However, in cxplana-
tory research the bias wiIl nnt always matter. If thc characterishc on
which there is sample bias is not related to the variables that makc up the
~xplmatclrymodcl then the bias will not affect the patterns in the
explanatory model (Chapter 8).
Missing data create a problem because nf their effect on sample size and
on sample hias. The strat~giesfor dealing with missing data will depend
partly on which problem we are trying to minimize.
Of course, we should try to minimize the problem a t the data collec-
tion stage, but there are three ways of addressing the problem during
data analysis: imputation, weighting and statistical controls (Rose et al.,
Essentially imputation involves constructing responses for the questions
which people do not answer. There are a number of forms of imputation,
some of which are discussed by H e r t ~ l(1976) and by KaEton (1983a;
1986). These include the following.
Samplr nzwi? flpjlroach Tf we do not know the value of a person on any
pivcn variable, a reasonable guess for that person is the average respclnse
by other sample members for that variable. With interval levcl variables
we could replace missing values with the mean for the variablc. We
could use the median for ordinal variables and the mode for nominal
\,ariables. Although we will make errors in our guesses this method
produces fewer errors than guessing any other value. Where the sample
is relatively homogeneous bhis approach may result in few bad 'guesses'.
7 hv prtjhlcm t ~ 1 1 1 1~ r l c ~ p t t ltlliq
~ c apprrwch, hrliccavrr, i < t h , ~~t~redl~iths
sample \ .~ria't-rTtt1ern !hta \,+r~ablt. rtlr tvhrch t h t ~mlc.;ing ~ l a t aart. I-+t*ltig
tlrtirn.~h+rl. Thl.;, ~n turn, rr~illccr;tht* ctlrr'lntion t%ctwccnfhfs v a r ~ n b l r
and r ) t l ~ \,ar~alllta.;.
r This i s a rnori* cnrnpl~~x
I < i - , q r t ~ ~ ~rrrrrl~,.;is:
~~ou metllnd rtf impirtntir~nthat
intnlvcs the 115enf rc!:rr-ion t o yrdicb thc i alue of tht, F e r w n (In thc
I I I : C h 7 ~ivav of cnt-rcnrning fhi.; probll-rn IS tc, u w nliw,lnc 3 al~lcqkarial7lth. C7n thc l,n.;i~ ni a \\.hole set r l t charactt~rr~tirs we
group mcLinsmth1.r than t-ht*overall <ample mcan, In du !hi+ we would c,tn cshrnatr how, on ;IirrraRc, a person with those c't~nrarteri~tic.; nrc~ulcP
l o S ~ I I T I ~ ~ I ~into
~ l i v ~ ~1171- . croup.; on thr basis 11f a harkgr~jilnd varinlllr* a r l r n r e t t h r question nli rvhicli tlicv have rnisqing data. In this rrqpccl i t ia
Ic E. ~;I.~Y. aze ant1 cthnic~tt.)that corrclntes rvrll rvitl-t thc nli<l;ins c a l u t c o r n p x ~ b l tt>c thc
~ grrwp means approach. Thc prnhlt*m with thc mcthod
t.ar1.2bTv \\e wu~tlilthen obtain the mtvn tor tlir- missinq d , ~ t a\~asi;rfilr i\ that 5111ccr e q , ~ ~ < sI <~ u t ~ o n a ctim~lationnlatri\ in rvl~lrhthp
bnsctf
for c.i<.h u t thp.;~,sclhgn)llp+. l-nr c x n m p l ~ ,if nrv ~ v a n t c dto cactinratt, the p.i~rwisrarnibtlind of I i ~ n d l i n gr n w i n g data (st117 below) I.; ~1st.d.thl.ri. v\,1I1
inctr~il~b Inr p~%rjl,lt'w h o cit>clrnrd to ntiswcr t l ~ ( r jncornta qllp~tion i\*l> nt)rmnllv I-rtl home lo%% of caqcs.
micht rilvidt. tl11. <ample itlttl vcrup\ according tn their chtlt~cationIrrcl
and t h l . ~ ,within taach cduca hon !r~,t*l, Furthcr 14 ivide t h t ~5;lrnplc lntn t t i i i For <om? stnblr*charaztcr~sticsrvca can 'carry
mallas ,?nd fen~;tlt*s.For cach cclucat~cinletpelatid ~ e n d c r.;trhfir0~1pw v ci~~cr\nbut%s trnm ~ l nrarlier w.ivr (Kmc rt dl., 1YQl) or check [lack from
cnul~i1ll7t;lintht. rnt-an incomr Ievel for Illnqc ruho did dnwvC'r 1 1 7 ~income ' k~11,~t.qucnt w.~vcs.'Hti~i~cvc.r, flrr variable\ that a r c Irkcl y to t l c n subject
q t u ~ ~ t i r wW. c r n i ~ l i tfind thaf thy Incnme Irvcl I i l r tc.rtjnrt edt~salr\iI t o c h a n c ~ ,lnd
. for nunrrric \ . ~ r ~ ~ ~ in [ d p;rrt!ck~lat
c=< t t can be incnrrccl to
fcrnti It-. Eq VO,O131) F a. rvhilr trrtiarv c.ti~tcatedrn,~lesenrncd. on avcw>:t-. 'c;trrb nrcgr' rerponwdc from ;I p r e r - i o ~ ~w\ a r e . Ovt-r time i t may be
<75,0111);lnd 30 tc?rtl~.U.;ing thy ~ 0 ~ 1 1~ 7i ~ a approach n s w r n'111lld slrnplv pnwlble ttl fill in sonlrl gaps CAI~SI.YI by M ~ ~ V non-rilqp{rnse
I* l-rv Inrjking at
~ ~ ~ b \ l i thew*
( ~ l t 111~~111
t ~ ~.11111-.for rni.;.;ili~value#\ Ior pct,pl{. ,ir-cordln< i ~ ~ a n ~ t!he q ~ r h r - q u c nwt a l m eand r r n y i r ~ ~ nrn~tlpoinl
1111, r ~ r c v i r rrvavca ~
the Crrrrrp inhi tvhich t l i t * ~tcT1. r,,ltae fnr the- rnisqinq uvar-c.
Rich glr-ad\~;lnt,~yt~ of th[\ approach 1% that it t3\.E~gerntr.. t i ~ Jcgrt>r
~ a r,t
hnrnop~t~rlity wlfhin grt>up-- and t l i ~ ~ ccan ; ~ 1 v t ~ t ~ ' ~ t i rthc
n ; l vananre
t~~ r r : f~J.ss.inl:~ i t l t i l Thi5 is not a n imputation L ~ p v r o ~hut
I g ~ r o l ~ i ~/lrru c h pro-
b~t13~rbr~n g r o u p and tor ihr *arnplc nvt-rail. Thi.; c5rror,in turtr, can 1nHntt1 ~ ~ d ac W.IV s nf rnatmgin): with miwing data when ct>n.tt~rctinc scales. I t
the cc~rnal,ltion.;ir*hcn using tfic r-ariablr t ~ which r t h e mi- in^ data 11.ir.t- rntailh c.alculatin): q c n l ~ScorcG bv calculatin< ,I perwn'5 mean <illre h r
bccn cstimatrd tjtd- ;-l~ril~/l? h cr- wItiL.11 !lft*t/ I i ~ i ~ /?~,r,i > : v d r i ftn17Ti7i-r.q.5 ~ 1 p p 0\vta
~ i v t w con-
qtrurting a scale from 10 qucsticms cacll h,iving pn\<lble scorc.: ranging
I f I i t This approach i s rimilar 11) the f i n j i ~ p trl>~ii0 to 1. : Ip e r w n who c-rbt~inrda ~c4,rt. nf 2 oil all I0 itcrnq tvoul J
rncarl.; npproaclr 111 that i t rc-lit*~ on ~livldingtht, +arnpl~ i r ~ t t rL T O U ~cln a c h r e ~t* a total qcnrc of 7n ovcs ,711 10 itpnl.; and a mran SCOT^^ of 2. ,I
the I,J\I< n t o t l ~ c characti.r~\trcs
~r that a r c Itlel! tr, bc corrtll~tcrlwit11 t t ~ c ptJrson who n b t a i n d a scar? t r f 2 on cucli nf c ~ ~ ttcms h t l ~ did t not
r n i s s i n ~data variable. But ~t d ~ f f r r sin that i t does not involve sutl.;ti- ~ i n r w e rthy rtvnaining two items would ohfain a Scltnl wore of l h but a
h ~ t i n gI E W g r ~ l ~
nlcan
p for any rniqsirlfi clata. In<tr*nd,wF~l.nr\,e Incattb a nlcan of 2 Calculatinr: a scnFc <core in t h i ~w a y avoids the prot~lern(11
casc iz.rth m i w v n ~data a n ,I particul,w vasiablc we ~vntaldrook .lt thc rniqsing d a t ~whcn ctrn5huctinc .;caleq.
vnlur on the qamc variablc of the noarcBhtprpr-ding casc In [hat gat7up
and give that snmr ~ ~ I L to I C thy caqe w ~ f hthe m i w n g valur, l'his nlcan.:
that thr* missint: lire5 ;Irt7 rcplacrrl w t h n t.3rit-k of d ifh-mnt ~ r , ? l t r t a <
rantlonllv chrv.*r~from tt.lt7~1nt h ~ t+irtl~rl'r\ipt > t c ~ s cl ~Firc ayprc>acl~ \\'eiqhtrnq I < ;I \\.a\ <,i ,~Rluqtint:a sarnytt- to aHo\ro for yo5sit.lltb Iliac d u c
doc..; 1 1 c ~ t atrcvt ,,~rnplror gri)uT ~ . ~ i r ~ a t , ~ift ~hast v . n o t n f f ~ r ton thc t o ttrut ncv\-ri7';pcln\ta.W c i g h t ~ n,I ~+amplt, chcruld m,ihi. I! rnclrta rl-prcs~n-
stret~gtli of rol.rt~lativns ,~ncl avoid.; nnv Inss of c a s w I h p i t c btxing l . ~ t i \ eof 11it. populatlc~n it is rlcsignpd to reprt*srtlt st3 that rclialllr
~crrnt~it'l~at morenrr,mylcx to taut-cutr, t h ~ >1s a h ~ q h l vd r ~ l r ; h ~rnctlrt>rl
lt~ r>t vktlrnatt~(,in be m,idt* from l l i ~ a<,~rnp!t'to the pc>yi~laticlnFor I-uamplc,
hantllltlg rni-lni; d a t a r l n , l ~ i n c;I wftlatliin trl i\*hicll per cell1 or the p o p i ~ l a t ~ oarta n male and
50 per c t ~ i t.Ire icrnalt~.Hnwt*~.t,r, when ,I <amplo ot 1111spoytllallon
nt qpnrr A relatrd approach rrqi~iresthat wc cnlculnte the tnlbnn
Arjr7r?~,q13 ~ l r a w ni t 1% tnund tli,if, cithcr I i r ~ c n ~ ~ofs tnun-rcqpimw
~ or bccause of
vr thr r n c d ~ a n(;1.: approprt,lt~~) ot IFI . case< tr-~tliina gi\i*11<pan t r t flit t i r n p u t , (dl yrr cent of camplc ~nt~rnbcrq art, tcmale , ~ n t llil pcr ct-nt mnlc
c;lhc \vltli mlwlnl: 14dtd d n ~ lwhstittltt, this v;llur for thr rn1wnq d ~ t . 3 %nee yt-nt11~ 17itl rr,lnted t c 1 m,lnv otlm.r vnr1ab1t.c fhis b i o ~ir.111 affrr-t
I
valtlt-. Thiq ayprtv;rrh is r t ~ l . ~ t ~ v cslrnplc
lv and ,~rclidstlct7,iting varlarlre r.\tirnrrttbs rrthrr varl.~t.rleqtsrm the s a m p l ~tn thc p c ~ y r i l ~ t i r ~ t ~

To adirt.;! tor tl~rI)i,14 IYC ~x,nllldnrrtl to rvclght each of thc trrnnlcs to
~ ca hl ~would bp ~ ~ ~ ' i g tc)
count for lpsc 111,111 0 1 1 ~ ~ P T S C I HI % ~ ~ I I C #t ~ ~ m
count for rnnrl>t h a n oiir person. In thw ~ ~ their a v pryrrrt~on.;in the
<ample are adjustc~lto equatc with ihcir proportions in the prjyulation.
Tu achievc this rand wc weight each casc hy a spccific weigh^. A wcight
is achieved hy dividing the population pcrrcntage for a category by the
<amplep~rccnt"~r In this uaqe the formi~lamtirlFor males thrc~ulclbc FiO/
40 which gi~~r.; .I nrt>ight of 7.25. For ftmalcs the ratio M,OLIIJ br 50/h(l
whish gi\.rl;a tr~eichtc ~ 0.53 f (de Vaus, 2olrl).
In wave 1 of a pllnel study we ~ ~ ~ on ~w dl ltod rzpeightthe samplr on khe
bnqis of tvhat w t a know about the ~ h a r ~ ~ c t c . r i s t01~ cthe s pnp~~l,liir>n and
l l i ~chilracI~ri<tic~ o f tfinqe \%.hi) rcspnnrf ;II the first yh;l\c (r E. sex,
rcgion, ethnlclty). Tliis rcquires that wtt I~ilvrreliable knuwlrdgt. 1)I the
populatiim cliaractcristics.
In sitbsequent waxFpswe will need to niakr further weighting ad just-
ments to takr ncctwn! of biases that ma), crccp in hecausr of .r ttsiticln.
Given tIip amount of' ~nfc~rrnaticln available from the lirqt 1var.c rtrt* \.\,~lj
!lave a grc.dt drat of infnrrnation that can tlr lrscd to calclilatr t c c i ~ ! ~ t In
these cases thr rvciglit IVL>II~LI be a c h i ~ w c dbv dividing thr t\..l\.r 1
ycrccntagr 11!. i l i i~z , I \ Y 2 (or 7, 4 ) ycrccril.lgt. Thi5 wilI enLlblt,lht, IV.I\ r.;
after wavc 1 to rrnl.rin rrpresentativr o l wnvc 1. So Iring an w n v ~I is
represel~tativcc l t I hi# populntlon then tl~twtbwrightings cnn hill p rnch
wave remain rrprrsc~ntnti\~e of the pryml"ion.
Analysis can bfcomc quite complcx w11t.n mtllhple waves arc analyseif
a t once ~ i n c reach waiee mav nerd to bc wcigf~ted difft'rcn tlv. Thcsc
strategies for wrighting ~ v i t h i npanel <unrrvG are discussed In dttail In
Kalton (19S3a: IqSh), 1,~'pkowski(lQS9),I.vlin et a!. (1994) and Ruck et a1.
(1 495).
Vot only d n c ~bias ~ t i our ~ atr~lity
~ t tt, n ~ ~ rcliahlp
k c estimatt,.: frrrm a
%ampleto the pcrp~~lat~rln but it ran .11+0 d~qttrrtthe patterns of rt*lation-
ships ilnd I-rrtrz~trnvariables ~ 1 1 1 i 1 1 101 sample. Tt 15 not alv,.al;s
pm5ibIe to rt-wcil:ht samples reliahlv. tlowl.vrr, we can en5urc th,it the
pattern of rr.l;ltion.ihips wtl find betrvcrll varinblc5 is not cllw tn thc rffcct
of the variablt* It~rwhtch we have bia<rd information.
To do this we cuntrtd qtatisticallv tcsr var1.1bIc4where t\.r rithrr know
o r believe thertl m a r be a bias. Bv z t s i n ~<tnti<tlca! controls [ c k ~ . ~ p l c17)
~r
tt-c can Inuk a t r<=latinnshipswith the r t t t ~ ut' t other rariabfr. rcnm\?rd.
Rv r c r n ~ \ ~ i nthc t ~ cfftlct of a
g cficcts ot other .r.ariablt,\ wtac;ln e l ~ m i n q tkc
~.ariab!ewJit3r~sarnplt~hias is evident
D A T A AKhl,YSIS IN t~CFY(;ITUI7IYAI.DESIGN
Measuring change
lit~~l
In studics that txxtcnd over time we can think uf change at two Icvcls:
aggregate and individual.
Change at the agErrrgate level is al?io scfr+rrcd to as net c h n n ~ cor
niacro c l i a n ~ etvh1lr5, change a t the initividual Ir\,el is also rt.fcrrr\d tc-r a<
g r r r s ~change oor rnicrr~chance. The meaning id the two ways of t l i i n k l n ~
,3t>outcElange can bc moqt enwly described W Ith a n example. In WJI r 1 of a
.;n~dvabout ppoplr in thc paid wutkfirtcc~\ v t rnlght identih~thc prrctXnt-
age of peoplc worhlng full time and thc pi>rcmtcigrorki king p.lrt t t n i ~At ~
wave 2 we c o l l ~ r tthr same informarion a n d find that exactly thr w m r
pt?rccntages c ~ propl(- f work full time and part time as in the F~rstwiivc.
While it would b~ correct to conclude t t ~ att the aggregate Ic\lrl lllrm
has been no changc hctwccn wave 1 and w n v r 2, it does not fnl low thaf
ntr individuals lia\,c changed their level of rvorkfurce parhci~.$tir~n. If 70
s. pcr cent of full t i m w~nrkrrs ~ had t h a n x ~ dto part time and prr CL-nt~f
part timers h;ld hrct>mc*titlP t~merswc 1 \ ~ 7 t l l C ( have a large number OF
t ~ ~ t l ~ ; ~ ~ rMTJ~CI
f r r n ill~l ~ ' ! t c ~had
f cli.~ngecl 111t)irI c ~ c luf workic~rcrp r l i c i -
p k i o n , but at an ; ~ ~ ~ T c ' Rlcvel J ~ c 11 wnuld appear that t h ~ r ~~ . ~ 110 1 %
clrangc. At the aggrtal;.alr l ~ v e lchangw t y En~li\.ldualsin onc tf~ri.clinn
can be cancelled rl~ltby changes in thc opposrtr direction.
Distinguishing Lbctw~cnaggregate and Individual change is irnpnrtant
both at a theoretical Icvcl ;lnd at the rtwarch dcqign level. it 1% easy to
miqinterpret IOI\? Icvrls of aggregate c h a n p , I f tve interpret I Q ~ + ~ re-
gate change as thc ,lb.ience ot individu,?l changc rvc will arriipcat ~ l ~ r l c r -
c m c l u s i o n ~ahnu t the phenomenon. Fur rtample, the ~ercr.ntagt. of
people who rccimivc welfare and the charactcrivtics of such p ~ o p l carc
Fairly stahlc ovcr tiinc.. This has Icd snmc prople to believe that wrl (arc.
r~ciplcntsare a stnhlc and unchanging group. In reality, there is a large
h~rnoverof thosc (In w ~ l f a r ewith rela hvcl y Icw continuouslv d c p ~ n d m t
on welfare over timr (FIakirn, 1986). Stabilitv at the aggregate lrvcl will
indicate snrnethin~:about the nature 0 1 tlir macro system (i.c. thcrr 1s
something nbnut tF-tc str~~cturc of the s t ~ r n land wclnornlc s y t r m that
produces a given 1ct.d of pcrrple on ~\.i*li,~rtb) Fln~vp\,pr,it d n ~ nots t r ~l l ur
.ihout the indrviduals and the naturc of changes thev experlcncr.
The type of rr>t7;lrcll design we adopt urill dcpcnd nn whctlir~rour
Sncu~is on the aggrr*gn!c or the individual It%vrlof changt.. h srrir* nf
rcpea ted rros<-sccr rclnal ~ r v c v is s qil ite appropriate for backi tly: ~~~~~c-
gat' change. Sincv qilch n s u n e y mea\urta.; c . h , ~ na t~a~grclup Ik=rr,l tvc
need ccrrnpa~abl~ groups river time. Hc~wcvcr,11 we are intcrc~trd in
change a t the ~ndivrduallevel wc have to u\c n pane! d e s i q bt~;tt~.;c lllf
Game rndividual~arc inrcrlvcd in cach %\'.~I.L*.
D a t a will he analvsrd diffcrentfy dt.pr,nd~nc on ~VhrthrrWP a r c t - ~ l m -
ining aggregate o r individual level c h a n ~ rI4'twn draling with agcrth}:ate
I ~ , \ , t h l change MY' ;trc reqtricttd t r j nggregatc rntLasurc.;. I-.or e ~ a m p l t tviicn
,
c ~ , ~ n ~ ~rtqys~jiate
ninz rhangt. i v c ~ 7 i l FI-umpnrc yrrltrp nlcans nvcr timc,
rhancc.; i n Rrrllry vnrlancc, nvrrall percvnt,rces i n particular cntc:~nrli*ia t
rl i l ((*rent timc points ( e . pewcntagc
~ unemployed, ptlrcrntagc on wcl-
(,in. txtc.). Whcn adoptinn a n aggregat~change stratcgv we can compare trmr scri#% .inalvs~c;latcnt
whgrnupr; to ~ c icf the ertcnt and direct~rmof ag~rcr:atrc h a n ~ cdiffer ~ a r i ~ b4ru1-tural
lr rn(dt.1-
bchvwn sul,eri~~!ps. For example, we rnifiIit track c h a n c t r , at an agere- htl\turc t>f nominal and A Z ~ C ~ r* I AVCI )\ 4,
' Jltlr
ink-rva! (r#+nhnuotrs.~nt! rvpcwrln I\ llh Jummr
1 ; ' 1 1 ~ Icvel, in tlw 1ci.cl of rlfrtirnism of b i ~ + i n ~otvnrxr<
ss and empln\.ccr. categtrn .I11 vnr~ablc+
bcfor~and aftcs the election n f a labour oriented go\)cmment. Ry corn- Wnrnlnal (L-alegorical) A N O V A , nun-parantetric
r,?ring the difftmrences in ~ g ~ r t . g a tch.ingc e for thc rlifftvent group< we AKCIi74,rlrirnrnv varialdt.
can brgin to ~dcntit!. qnmc r j f tlic fiic~orsthat affiact c~ptimtsm 1 1 : Ilrt+
il~vK,'~l?tl' IL+:~,>~, Diwriniin~~i~l L1n21y<i5:I ~ ~ ~t ~~ r, I !
probit .~n.ilvsis, Eoh~stic
f-lowcver, nonca irf thesc htrrltegies ill lows US fn identify the cximt r e p s c i ~ u i ;h n r a r d / s u n ~ l v : i l /
and multidirectional n a h ~ r cof changc n t khe individual Icvcl. Wc event hictr lry ,~nalysis
cannnt identifv thc d e g c c tn which ind~viduals changc, the numbt*r wIln V i x t ~ r r r -1 1 1 nominal dncl LLV-lincw .inalv~is;k o ~ i ~ t i t ,
cl~nncc in nnc rl~rcction or annther. or the mwlt~riirectionaS nature ~ntewslit-vntini~uus.~nrl reFrclw>n. lnzard /.i~~wlr.aI I
of change (infliws a n d ryutflorvr;). Whcn we can ~cltantih i~idrrvrlrrni< cateE(.clrlcvI) rl-en! 111~lorv
annlvcis
Non! tnal (catcmrical) Log-lint'.~~.analysis; mulli.;laltn~
whtl cliange thcn our analyq~rc~lnfw~rso n identifying t h charactcriqtics
~ haznrcl/
life t,lhtr n~o~lcls;
who thaiip' to small and large eutr-ntk, and tlio.;t*who
01 rlinngerq: t h o ~ c sun:ivnl /#>writ histon' an.hv.;ir
~-li,irr~cin oppo.;ifr dircctiijil, To uondisvt dnalvqic nt t h i s 1t.t t.! lye
rl*qlltrt3pane1 c3altl For Indi\,~iI~rnls.

Thr. tvpe of analvqis undertaken rvilE depend on the Irtpr* of rlro~i~pr

bring
This chapter zznill discuss only rneastlring changc in panel dc-ignq.
considcrcrl Rliq relates to thc icvel of mcasuremcnt of the dcpendcnt
Chapter 12 will ct~nsiderwav.; tn which change i s analycd in mpcated
variabl~.
cross-sectional dttsips,
If the depcndcnt variablc is nominal (cn tegorical, ilunlitative) thcn
I n panel dcsi+:ns we havc to work out how change will bc mca<ltred.
change will simply consist of whether pcople have c h a n ~ e dfrom crne
Thi.; could 1-r~limited to examining agFegate chanec, in which caw
slatr to some otlrcr \tatc (p.c.nlanied to dlvnrced; crnplor-ed tn rlnrrn-
a~grrgatemenwres of the dcpcndent variablc would be ccimpar~da t
plovcd). With rlualitatrve ~lrpendcnt \~artahles WT rrill uqe nnalysls
r,qc'Ii wave. For cxnn~ple,wrc rzlould ctbmpare means, v~rianceqor pro-
m ~ t l i o d sthat can copc with this t y p e of dcpendent vnriahre. 711r parti-
portions at each wave and srt, haw much these diffcrrd hetwecn wavcs
cular form of analysis will also dcpend nn thc level of measuremcnt of
t h r inclependcnt variables. Mcnard ( l ~ c ~ has l ) summarized the main (rablc 9.2).
Iiowrver, i f this is thr only way in which panel data were annlv.;c.d
form.; clf analtp~iq appropriatr tn different ctrcurnstancw and t h r . ; ~arp
tlirrc wn~rldbr little point in tising a panel deign. .A repeatrd c r o w
proviclcd in Table 4' 1 X dc5cription of cach of thew frrrnrr of analvqrs i s
wcticlnal deqijin n4ould do jt15t as rz.ell.
br,vt~ntlthe scope uf this bot)k.
WFivn pant-l data are avsilahle thcv will norrnrtlly be analysibd to
Whrw dcpentl~.ntvariablrs are r n e a w r t d at the in t~vvnllevel ivr c.1"
idtv.rljlv changcb n l the ind~vidunllevel a c wtall as t h a~~ c r e S a t eIrvcl. Thc
rnl>a<tlrcthe alnarr~?tor quantitv of changc rather th.~nstrnpl~trrht.thcr
qi~c*<Iinn thcn I.; holv cl-ran!:c~ . ~ tthe ~ndli-iduallevel i-- lo be mc;rsurt*d.
I not. 1Z'r can do thi\
tl1r.v ~ J V PC ~ J R C ~ ~ C or l7uth thr a ~ ~ r c c a It ~e rt~t~p)
and ~nrl~vidual 3 1 ~ ~ ~ For
1 5 . rxample, r t 7 r r n ~ q h trne;l+lirrbthe c f ~ g r r , l a to
whlcli cconon~icw ~ l l b e i nch;anges ~ nrrer time. Assliming that wtmcan
accur.~telymeasure cconornjc wellbeing (c.g. income IPvr~l,net wtrrth nf
a.;.;tat4 rn dollar.;) !zFe can qirant~fvhow rnltcl~tlctter (or wcrrse) off pcnplc I</til' r l ~ o i l ~sd-ntrb.:
t~ One ~ i m p l eway nf measuring ch.rnge whcn the
artm rrircr timv Approyrratr mcthndr; cbl nnalysln~:chanrc whcn thc dcprndcn! vdr~nblt.is quantitative i s to c.~l{>ulate clttnrv~,~ ~ v r for
f c tvch
chnnjit> \barialslc rs quantitative nrp outlintd in Tatrlc 11.\ prrsrjn. Riis i.; dcine bv suhtr,>ctingw7a\'c 1 \cores fr11rn tva1.c 2 scnrt-s (nr
 LONGTLJDINAL DESIGNS
l'able 9.2 t%r!/i 17f ~ I I I > )~Il ~t r~rf~~i l ' ~w ~ ~ z ~ L ~ ~
e~nnrtrrriigi~gryrrr,qufc~
Wave t Wavc 2
hlean 25 35
Variancc 62 5U
I'ercentage In catpgory X 45% 49")"
whatever waves arc being cumparcd) and treating the difference as
reflecting change. However, there are several probIems with this
approach.
'The first is [hat any rnpasure will have a degrec of error. By using a
measure twice (wave 1 and wave 2) and computing a change score, the
unreliability of cach test will be compounded in the change score. Great
l d taken in using change scorcs unless the measure of the
care s h n ~ ~ be
outcome variable is known to hc highly reliable.
Another related problem is that extreme scores are more likely to
change due to the pheliomena of rcgrcssion to the mean and tnrncation
be difficult tu tell to what extent
effects (Chapters 5, 6). 11 can tl~erefc>re
thc chnngc score reflects reat chaiige and how much it rcflects measurc-
mfnt errnr.
Residual uhi1nge scores A problem with raw change scores is that the
amount of change may not be independent of t h e initial scorcs. For
example, we might expect more change among thosf who had extreme
scores initially or, for some other reason, we might anticipate greater
change among people who score m the middle of some scalp. Certainly
the dirrction of change is likely to be different depending on where a
person ties on some scale. Tl~oseat the ex trcrne ends of a scale can only
change in one direction.
O n e way to remove the effect nf the initial score is to use residualized
or regressed change scores (Kerlingcr, 1973: 337; Menard, 1991: 45).
,Theseidentify cases where a person has changed more than would have
been expected on the basis of their initial scorc. Using this procedure we
use regression analysis to predict wave 2 scores from wave 1 scores on
thc basis of tlic correlation between wave 1 and wavc 2 scores. We then
subtract these prrdirted rvave 2 scorcs from the l ~ ~ t l l (wave
lr 2 scores. What
remains a the residual gain score - the amount of gain that is F F ( ? ~ UP tu
the influence of the initial wave 1 score.
P~rrrntngt,clrang~ Another way of measuring change at She individual
level is to calculate the percentage change in the initial scnre. This
approach is only appropriate for ratio scale variables where there is a
non-arbi Wary zpro (this includes most continuous va iables in sncial
wlence ~esearch).I-Iere we subtract the wave 1 score r t m the wave 2
DATA ANALYSIS TN I,OYGr'TUISINA I . DESIGN 157
scorc and divide the answer by tRc wavc 1 score. To convert i t to n
Change percentage change we would multiply the answer by 1UU.
For example, a perqon's wavc 1 scorc might be 30 and their w a v e 2
+lo
-12
score 4U. %n:
+4%
wave 2 score - wavc 1 score 40 - 30
x 100 = -
wave 1 score 30
10
=- i n o = 33%
30
1 The wave 2 score represents a 33 per cent increase over the wave 1 score.
~ lack of ~ t l i n t ~ i l i f yAll these ways of
L'listing~iishinpbetween rral c h n n , ~ond
measuring changc have a common problem: they all rely un possessing
seIiable measures of t h e dependent variables. Meeting with this condition
can be espec~allyproblematic when usmg numeric dependent variables.
One approach that can be used to help distinguish between real
change and 'change' produced by unreliable instrumenk is to replicate
the analysis using different measures. If the same p a t t ~ r n spersist when a
number of different measures of the underlying concept are used it is
less likely that the mcasurcd change is due mainly to n1~asurcmei1terror.
Thc achievement of consistent patterns when different rneasures are
employed s~rggeststhat the pattern is not dependent on the particular
measures and 1s not simply an artifact of unreliable measures.
Honr much c h a ~ l ~i sr change? Given that there will always be some
degree of measurement error, how is change to be defined? The problem
of specifying how much measured change has to occur before we are
prepared to h a t it as real change is a difficulty regardless nf whether
raw change scores, residualized change scores or percentage change are
used. If, for example, an a scale of 0-20 a person changes b y 0.5 of a
point, or if a person changes by 1 per cent, do we treat this as change, or
is this degree of change likely to be due to measurement error? Should
we employ a minimum cutoff point and only define change of more than
a certain predefined level as constituting changc?
There is no simple or clear-cut answer to these questions. The basic
rulc shouId bc that the more reliable the measures the more we can treat
small changes as real changes. Sincc many var~ablusin social science are
prone to measurement error we should be cautious in what we define a s
change.
f Adjlistinp.for 'illflation' Some measures (e.g. income) have to be adjusted
ovcr time before we can measure change. In snme cases the actual
 LONC17'U13INAI. DESIGNS

txpwss'd In year 71K)O d i ~ l l f i rThus

~ . 51IJOO vi8nr 71Kll) dvrll,~rcrvcwld L-uv
51673 wr~rtlinf goods IF tIr;it 51011f) was syrnt in 190.
L~ ~t~i//rf~<
f PlF'Lj+l
/ . T / ~ ~ - L ~ ~i .f G
StnlrJni-di:rlhn~t: z-scorcs { ~ n d{vrsmtil~rnrrk Another form r r t adjusting
scores is to cxprrss all scores in terms of whcrp they lie on a di.;tri'hutinn.
T h a t is, ~ , r can
, sinrrtLrdh9 scores and, rt7thr.r than looking at change in
SlW3 11"' ( r e TI)(1-X",.) nbsoltitr values, we look at changes in a pcrson's scnre ~ i ~ l n l i vtor clthfr
5UZII x I1 "2 ( 1 c lW-8" ,) p f o p l ~( w bl,~rsh,
~ 19% for an accessible d i \ c l ~ w i o nOF a ranKimnf method.;
5MA IlCj! ( l f l [ Y l - ~ ,)
Y
of stanclard~/ation). We look a t changcs trr whew a persrln Iics in a
$796 Y 4) Q?
$712 Y (1 L G
distrihuti~)n.Whcn Iuoking a t change, in rrla tivr rather than absolute
$W5 Y 0 Vfl terms, wc would only count n person at. c h a n ~ i n gif they changed their
EhhY Y 11 44 positinn relative to others, tliat is the!! mnvcd more (or 1~1s.;)than some
5f7Y x 0 kt7 clthcr penplr. To eyprew c l i a ~ on ~ ~nten~alIc\.el depmdcnt tbariablc
~ gan
<hl)J I! rr:
in relat~vrtc-rms we can ruprws each pcrson'5 score at r v a v r 1 and wave
SWl v ll U H
2 as a :-<cnrcl. With an ord ~ n a dcpendvnt
l variablc we can use a person's
perccntilc rank. We W O L I ! thcn~ measure chanjie by look in^ ;I!changes in
Eupreccrd rij ? ~ ~ f Nth~llirrs
) z-scorcq or pcrcmtile rank,
\car $ Inflation rat+. 1",,l ?OPE
For e\nrnplc, i f w7e wrrtl lrloking at cha~igcsin incorn? rrtpcrtime wc.
could convrrt each person's income into a :-score which ri.i~ulcltell us
about tll,lt pcr+rmrsir~cnrnt,rz~l~itive to otl7t.r pcople's incomi, If we had
t h e w z-svc-lrcs Cnr khe samr inslrviduats ovrr two waves rvc ctluld identify
those whnsr :-score incrcnqed and thosf for whom it dpcri*ased. Wc
coulcl also quantify how milch the pcrqon's relativc inuornc position
changed (Table 9.4). Altcarila tiuely, we I-(ruld calculate Ihrir percentile
rank In tlic lncnme stake.; and see how thcv chang-ed over timc in rank.
This mcthod of measuring change in relative terms is \aluablc when wr
need to ~ d l u for ~ t changes in the wav in which the dep~ndcntvariable is
measurcd over time. H(>wcvcr, it 15 a mcthod that can br used even
when wc d o not necd to adjust for thc way in which thc variable is
measusrd.
Since thc --woreindicntcs tlic number of qtandard dcviatlonq a prrwn
inqtn~rnentmny change [e.g. questions may need to bc changed to match I lies abovc o r belor*, the .;ample mean we can usefuliv compare changcs
tht- age or life 3 t a ~ eof respondcntq) For other variables, such as income, I in their paqition over timr. In Table 9.4, c a w 1 remains below the sample
where the rncnning of the categories changes (P.K. income need.; to he 1 mean a t wave 2 but has drawn close. Casc 2 has deterinratcd relative to
adjusted for inflation) the vnr~,lhl.r*itself neccds to be rescalcd to makc others by 0.4 of a standard deviation. Casc 7 has improved ller relativc
mtms>urescumparable over timr, 1 posit ion cijnsiderahl y. Rv using z-score< we autornnticallv adjust for
I'i~rcxamplc, w o may be Innkinq af changes in incnrnc among mrn and cliffetencck in actual inconlt. due to inflation and the twpral l rncrenw in
tvornen betr\.cm l Y Q 0 and 2i)rl~I. Ilrfore we can rn.~kerneanin~fulcorn- mean income. This approach enabltr crs to f c u s on chnnccl; in how
~ x w ' i ~ obehzrren
ns income levels in EL@9n and ?(1110 wc ntvd to know h t ~ v people ar? doing rcIntivc to others.
much the s a m c cjc-hlar amount is tvorth over t ~ m c ,attcr aJju5ting for rlltilrnativcty, we could examine the rrlaticc change of i l t t l ~ v d r r n lbv
~
intlntion. Bccnust- c ~ finflation $1000 in 1990 is not thr cquivalen t of B IOE30
in the year 2000. In Table 9.3 incornc ha5 been adjusted bv the inflation
ralli>.Thr incomr in m c h year ha< bren expreswd In 1Y90 dollar<. T l i u ~ ,
I examining change in r ~ n k('l'able 9.5).This is comparnblc tn Iclnking at
changes In T-scnrts hut dopa not qunntifv the amount of difference
behvcen tndividuals. It i~ thrrefore useful wit11 ordinal levcl dependent
tsr adfustmrnt~ic>r ~nflation,4F(H)IS in 2Dnn :.i the cqui\,alent ( y f only variahlrs. In T,3ble 9 5, ttlc rrlative po\itinn rrf rase 1 has irnprurl-ed, that
q.7'8 in lwli dollaw after inflntion. In the second rrample, thy rjpprl.;rtc of case 7 Ira.; detenuratcrl ~ h a r p l ywhilr that of cnsf has irnproroed
r.alc~ilation~ tlren rnadc ic~ththe worth of 510011 in each ycar bring
hdl\~z. markccllv.
I
160 CONGITUDINAI. DESIGNS DATA ANALYSF IN LONGlTUDINAL DESIGN

'Table 9.4 Cllarlge.; irr z-scorcs of rndrcidrtals Tablc '4.6 ~Mrnnzscnrr filr 1 1 1 ~ 1 1~ r t r l fT L W ~ I ~ P I T ,
7 9?0-2OllU
Case Raw score 2-wore Raw score ;-senre Change in
IVY0 (income) 1YYU 2000 (income) ZWO :-score 1W ZOO0 Change

1 (female)
2 {male)
3 (female)
4 (male)
5 (fernalp)
b (female) Table 9.7 Fmnle rnrnrnp as a pmprf~ilnof male e#r?iings
7 (male)
8 (malc) Year Colrrinn 1 Col~rrnil2 CuIumn 3 Cnlumn 4
4 (female) Averape FT Rv~ragcFT Female as prnportion r i f Abwlutc
10 (malc) male $ pa female $ pa mnlc (crdumn Zlcolumn I) $ gap

1970 $5.ODO S3,400 n.68 -$1.600

Mean
1980 $1 i,OOR $10,500 0.7U -$4,500
Standard deviation
1990 S25,OOO $1R,OUU 0.72 -$7.0UO
20011 S?~.OOO $2h,qi11 0.77 -$%,a05

Table 9.5 Cf~nvrgcsmrnsur~dhy rhnngrs In rank

Case Raw score Rank Raw scrwc
1400 [ incomc) 2000 I ~ n c o m p )
We can also use z-scores (or percentile rank) to see if some groups
change more than others. For example, if we were examining whether
'the income gap between men and women had narrowed between 1990
and 2000 we cnuld obtain the z-scores of all men and all women as wave
1 (from a pooled sample of both men and women) and do the samc at
wave 2. Using aggregate lcvcl analysis we could then see i f the mean 2-
scorc of mcn at wave 2 has changed more than the mean z-score of
women a t wave 2. i n Table 9.6, women, On avcragc, havc an income of
0.75 (1990) and 0.57 (2000) standard deviations belnu~the mean ~ v h i l e
men, on average, have an income of 0.75 (1990) and 0.57 (2000) standard
deviations ~ h o r vthe mean. Men continue to do better b t ~ tthe gap has
narrowed slightly between 1990 and 2000
Anothcs approach to adj~rstingfor 'inflation' or other change over time
is to c h a n g ~values into proportions (Table 9.7). Absolufe figures might
rndicatc a widcning of the income gap between men and women. For
Rank example, the average income gap in w e ~ k l yincomc between men and
women might have been $25 in 7970. l n 2000 this might havc increased
to $150 per week. However, 1970 dollars are not cornparabie lo 2000
dollars. We could use any of the above approaches to make the necessary
adjustmcnk to facilitate comparison. Alternatively, we might express
female earnings as n proport~orrofmnlr comings in each year. In this way
we automatically remove the effect of inflation. Thus although the dollar
gap between the average incomes of men and women is increasing in
Table 9.7, the relative pwition of wnmcn is improving.
We can also approach the problem of standardization by adapting
some other base. For example, we might want to see if the income gap is
widening between the lower and higher income earners. Wc could take
the bottom 25 per cent of income earners and express their average
income as a proportion of the average income of the lop 25 per cent.
Tn this example we see in Table 9.8 that in 1970 the top 25 per cent of
income earners, on average, earned five times more than the average
of the lowest ZFj per cent of income earners. In this hypothetical example
the disparity increasts over the years so that by 2000 income earners
in the top 25 per cent earned 7.7 times more than the average income
earner in the bottom 25 per cent.
EXPLAINING
CIIANCE
Assuming that the issues of how to measure change have been dealt with
and we are reasonably confident that real change is being measured, wc
will probably also want to explain the change.
The theories and propositions around which the research is designed
will provide guidance as to how to go about testing explanations.
 Describing change

C3ncc changp has hrcn conccphlall7c*d a n ~ lmcasuscd, rlr*cisiiln~nlirct I7e

made about hnw to reprt-scnt this changc. 'rlicse a r r \.anuus math1.-
matical and stnhstical w 7 n y s of doing thlh ( M ~ n a r d1991:
, Wff) bur IIL-re1
will only mention hricfly some ways nf d i s p l n y i n ~changt* in tabular and
~ r a p h i r a !form.

Change r>xrertimr can Llc p r c ~ c ~ n tcficctivcly ~d in tnbu2;lr (ornr. The

I-lowt~vcr,the prcll9lrm In a n b cuplan~tivnis that w e cannot be 1-rrtain
what F,~ctari havr procluced thc ch;lnge. Bccausr w e ldck tht= lrvcl OF
control of an cupcrimcntnI d e i g n , any changr could b~ duc to ~ ~ n k n o w n
fact orb that m-curred behvrrn the waee.i.
Whtn c~plninlngchange tl~ranalysis ivill be bawd on con~pasing
different jii-vupc:of proplc. We wiFl we whether the cvtcnt and nature of
ch,i~i~ int one
~ gnrup arc diffvrent from ~ t l i r sgrtr-trps Anv diffcreiic~s in
ch,lnge yattc.rn< of grcwps i5 ntlrrbutcd to cllli~rditt~*rcnccs that mav c x ~ s t
hctwcen ~ h poup\.r Ihkr., for t.xarnplt, a shidy cxaminlnq thtl impact nf
p,lid cmptovment on the sclf-estrcrn clt young people Using n pancl
dc>i~wHbecnulci rneasurc scllc5tcwn at t ~ v oor mow w.nrcs. 'rc~tcxsk the
rftcd ot crnployrncnt on self-t*sttlem we would identthr thosc young
pnrticul~rf ~ r mivilZ depend on the n~trnhl*ro f tEnw points oktar ~\.llich
ch,lngc is being trackrd and ivhctl~rrchnn~rais L l ~ i r l consider~d ~, ~t t h ~
aggregate or thr ~ndi\,fdu.tl levt~l.
T i c c*arlFcr table.: prnvidc illustrsticlns nf w m e of t h c w a y i l l wliicl~
tablrs might displav clinnge. Tlicse tablcs will usu;lllv h a r ~ c!cam c+none
axic - r i t l i ~ racross t l i ~tnp (e.g. t a b l ~ son cohort annlvsis In Chnptcr 12)
o r d ~ w nthe .;idt%(e.g. Ta1,fes q.3, ').7 and q-8). The wav in wllirh c h a n ~ e
i s indicattrl will vary dc.pendin): m ivhcther agpt-gnte or inclivi~l~~al
ch.~n~:t> is being c,on<iderpd 51rntn of thc V J S ~r,anp pi pcic~iblr(rat.5 r,f
rt*pnrtiilK change is p r o v l ~ l ~rnd l ~ ~ i b lUr Q.
'
12'11cn dcLllinqwith change ~t the 1nd11~ld11aJ I<,\~rl,
.;~ni~l.ira~~rtrsrlir.;
applv. I'mods ovpr w h ~ c h~ncl~vldual clranqc i:. h t + ~tr.lc-ked
~ ( c g . vrar)
acmsq thy top i i f thc t,~ble1lr trn thc *id(,. Thc
\s 131 bc i n r l i c a t t x i eitV~r>r

people who gained ernplrrymcnt Getrvccn wavcs, thnse who becamp othrr axis will represent tllr particular {ray in which ~nrlividualchangt. is
t~nernploved, those who remained cmploved, a n d thosc wlrn have being cmccptunli7ed. A ~ a i na, \vide rangy of poq.;il-le ~v;lt.'; of rcprcsent-
rcmninctl uncrnplovcd W e might find thnt thosc who eainrd ~rnplov- in^ ~iidividualclian~c*i<pc5rsilrlc. %me commtm approachez arc ill~is-
rnent showed a real irnprcl~rrnrntin sclf-rsfrrrn, and those who hlst thcir tratrd In Table Cb.10.
jvh shnrv~c1a fo.;~in self-cstet.m, while thr stabfc emylaycd and ilnem-
yloyrd showed no sclt-cst~cmc21nngt~.It tvou Id br tempting to ctmclude
that it i q tlic diffprcnceq in emplnvnient that account tor the d~fterc~nt Graph5 a r c a g00d way to rtlprcscn t chnngc. The most ccmjmnn h-pe crf
s self-eqtcvm of the p+c~ups.
c h , ~ n ~ ein p a p h for $ispl;ly~ngchan~e;ciq Ihc trenii graph, This stytion \rill p r o ~ i d t
Thc problem, howevcr, is that whe du not know that ctiangrs in a br~cfintrcldttction tcl trend graphs A filllcr dircusrion of thc.;t* ~raphli
emplrrymennt s t a h ~ sare the only changes t h e people a r c e ~ p c ~ i c n c i n g . and other w ~ y of s d i s p i a v j n ~chnn~tngraplilcal'lv can bc found it1 Hrnrv
Nor d ~ we 1 know that the four p o u p r are comparable En otlicr wavs ( r . ~ . ( 3 99,s).
gpnder, age , ~ n deducation). In experimental deqijins this prclbleln is The sirnplc trend b ~ ~ i p1%-ill h h a \ e mo ayes: tlir X-nxi~ar-ros< thtb
l~anillrd 1.y random nllocstion to RrtuFs. Tvplwlly In panel ~1csim;nsthis Iyclttcln~ant1 the Y-a.;i> on the ~ i d eThv . S-axi:, ~ v i l lrrprcsent time rvliilr
I.; not \x)w~trlc. thc Y-ari.; will rt.flwt ~iq ~ ~ ~ t i ton i t vthe vari,~bltthat iq b ~ i n ) ex,~rnined
:
n l c maln i\rav of r m p n n d i n ~tn the problem of compamhilih i s to i15c for cl-ranct*.Figure 0 1 ~lluslratesa srmplc twnd gmpli.
st;ltistic-al controls to cr;~ithli<hcnrnparnbil~t\' among the );snup%. Thic T ~ L)'-axil;
' c~3nbc r n ~ a r u r e c u.;ing
j a w ~ d ranee
e of mraxurcrnenl t l r ~ i h
gi?,ll cat1 be achievtbd bv u.;gng one form or anilthcr of rnr~lt~vnri;ltc am- ! Fig~tri.u.1 it 15 m r a s u r t d as a ratc - t h t ~nitnlbcr crf r i i v n r c ~I,clr
n ~ ~ 111013
Iy'is This s t r a t e p ~~inablcs
trs trl rcmclve thc. effwtq u l kntl%zbn d iftertnnct,? pn~~i~Entirm aged 15 or nver. Flnwri.cr, the Y-axi5 cnuld Iiave denoted the*
h~twt.r.11the ~ T L ~ I&CP F ~ Sit. i~ estaitrlish~dthat t l i ~p u p ' ; are qirn11~1rIn trri~~tllrrof Ji\ nrcc.5, err snmc other rvay t r t lndrcatin~tlir prc\.alenrtb t r f
the> .;pcc~ht-tl rc..;pccts i t is thpn pa.;siblc to cstabllsh hvw much the d i vnrce.
~ o i t p qCl~itcrtn regard to self-estwm ch.~nges.Thr log^ and tccf~niques Figurta 9 1 prllvicics trend data f-r A -
of thjs approach will bc dizcii.;<ed more full\. in Chayt~.r 12. @j-,31>17~ ""- '
 DATA ANALYSIS IN LONGITUDINAL DESIGN
164 LONGITUDINAL DESKNS

Table 9.9 Exnrrlyle.: uf tabrtlnr n<n!c;of ~ ? l r w z t i ) dtrta

r ~ nbou!
nsg r.rgtitr clrnnp 5 .- -

Year
1998-9 201113-1 2002-3 '

Mean 5 10 15

Varianw 80 8tl HU

9G change ( ~ mean)
n 1on./, 5o0O
horn previous per~od
rat^ 12 pcr lOOO 12 8 pcr 1000 13.6 per IORO
Gaps (c.g bctrveen 510,000 $7,5110 ?%5,5,0Cf
male and ferndie
avcrdgc income)

1970 1975 1980 1985 1991) 1995

prnprlrtiun nf male
Year

mtr, Aartra!in, 1970-95 (de Vatrs and Wolcotl,

F i p r e 9.1 Crzrdr ~fi~wwrr
19971

% in~pmved[from
+Males --t Females All 1 ,
. .- - - - -
prcvious period) 5% 15% 2% go .

% stable 85% 80"/0 7SV/" 4

"10 decl~ned tW 5"& 20% 80 -
\ -

"dj

"<)
change
cliang~ngby X amrn~nt
"'%1
10%~
2wyb
5Ys
22';'"
7.5%
.
70 -
Vean improvement -u
t
* +
of improvers 35 prr~ntq 40 p o ~ n b K
Mean dect~nenf decliners
~5 ppinnh
10 points 10 points 10 points
-
:so --
- -

-k5 50- - - - --

s different tmnds in employment IeveIs fnr men and

Figure 9.2 s h o ~ r the
a
z
women aged 55-59 over the years 1975-2010 and compares these with
the overall trmds for the age group. By reporting the trends separately / / - -
for the different subgroups (men and women) wc can obtain a more fine- 1
30 -
graintd picture than the trend line for khe age group as a whole provides. I
The way in which the X-axis and the Y-axis are scaled can distort the 20
trend in the data. Each graph in Figure 9.3 has the same information a s f 975 1980 1985 1990 1995 2000 2005 2010
Figure 9.2 but stretching or contracting the scales provides a very
Year
different impression of the bend in the employment levels of older men
and women. In Figure 9-33 the changes in employment levels appear
modest because the X-axis has been stretched and the Y-pxis contractcd. , 1975-
Figure 9.2 Ernp10,ynvel~t mte uf p e q ~ l en x ~ d55-53 h!j g m t d ~ r A~rst~n-nl~n,
This has the effect of flattening out the trend line, In Fibwre 9.3b the X- 2111 0 (arctrml and predicted) (de Vaus and Wolcntt, 1997)
166 I,ONGITUDINAL DESIGNS DATA ANALYSIS IN LONG! F'U131NA1, DESIGN I67

(a)
$00 -
1 - ~ a l
-
e s m -
---.
s 2 4 year old malo sutrae rate a$ tat1001 male age standnra.rpd so~carlefa!-
-24 war old male u n e r n p l m l ralp ns rutlo 01 uvprall unemploymrrll raw
- -
Bk soh-
p- 6, I~
1
L-

w
E d 0 1 .
g 20'
;
I
- . - --

1-7
2 ' -- - -

1975 1980 1985 1990 1995 2000 2005 2010

Year
e
z
[II

(b) -Males -Females All -

- ---

-
0.5

0 , - 7 - - 1 , I-T--l I , I I I , ,-7 1 I

."
h9 '6
24 ,GO ,$ $
$ ,' \d6 4
,' 8 ,,# &
,' @
,' $
0' @
,'

Year

'u
z n t ratios, 20-24 yenr d d s ,
Figure 9.4 Malc s u i r i d ~duntlrs nrrd n r ~ m p l o ! / m ~ mfr
% 60- - - - -- -
A~tsfralio,7966-90 (DHSH, 1995)
E
d

.. 5 0 ,
a)
U

a"
axis has been contracted and the Y-axis stretched which has the effect of
, exaggerating the trend rate of change.
I Trends for multiple variables can be plotted on h e same graph a s
illusrrated in Figure 9.4. The figure plots two variables: suicide rate and
40 - unemployment rate of young people. In this figure the trend line for
suicide of 20-24 year olds has been plotted alongside the unemployment
rates for the same age groups. Plotting trend llnes from different vari-
ahles on the same graph can highlight the way in which trends on
--
different variables co-vary. However, one must take care not to assume
thrt t this co-variation of trends demonstrates a causal relationship.
In F~gure9.4 each of the variables is measured on the same scale, so
each variabIe can be plotted against the same Y-axis. Sometimes wc may
20
1975 1980 1985 7990 19952000 2005 2010 wish to comparc trends on two different variables, each of which is
Year measured in different units of measurement. Tn such a case we can use
two Y-axes - each with a diffcr~ntscale of measurement. In Figure 9.5
Figure Y 3 Etrlplqrlment o f p r o ~ l unged 55-59 lry ,yeyellcIL.r,Aw5trnlia, 1975- the left hand Y-axis (primary Y-axis) is the seric>us crime rate per 500,000
2010 trrr'tttal n ~ r dpredicted) fIattmed scale, {b) e . ~ r r d dscale (de Vaus and
[i~) populahon, while the right hand Y-axis (secondary Y-axis) represents the
Wolcntt, 1997) percentage of ex-nuptial births (birtlls to unmarried mothers). Again,
168 LONGITUDINAL DESIGNS DATA ANALYSTS I N LONGITUDINAI. DESIGN 165

3.5
+
2
3
Xm
m
c 2.5
g 8,000. .-wa
m
-
9 2 2
0 7,000 0
51
$ g s,ow X 1.5
rn = r 7

j 5,000 r
n al
ur

g
U
4,000-
3,000
w
2
.-
0.5
0
-5- I.-
0
5 1970 1995
1,000- Year
< - 0
Flgure 9.1, Crude drirorce rntr, Austndin, 1970 niid 1495 (de Vaus and
Wdcott, 1997)
Year

When measuring change it is important to distinguish between chmgr

Figure Y 5 Ex-nupfnol brrths nnd serious m'mp rate, Ausfufllia, 19133-93
(Sullivan et al., 1997)
great care must be taken not to confuse similar trends with causal
connections.
Where a longitudinal study provides data for just two time points
rather than long term trend data it will normally be more appropriate to
use bar graphs to represent the two time points (Figure 9.6). To simply
have two time points and join them togcthcr with a line to form a trend
graph could be highly misleading since this could hide h e variations in
the intervening years (sec Figure 9.1 for these variations).
at the aggregate level and change at the individual level. This distinction
has been discussed and its implications for analysis strategies have been
considered. Prnblerns of reliably detecting change have been identified
and some ways of reducing these problems have been outlined. Finally,
the chapter has d~scussedthe need to standardize some measures over
time so that real rhangc can be identified. Different standardization
strategies were described.
Note
1 It can also occur in expenmental designs but can be avoided by adopting a
randr~mizeddesign without a pre-test measure.

Summary

Since a basic aspect of lnngitudinal designs is to describe and explain

c h a n g ~ ,the focus of this chapter has been on analysis issues that arise
with the measurement of change. One important problem is the problem
of missing information which is caused b y incomplatc data collection at
some point En the multiple data colIcction waves of longitudinal studies.
The different types of missing data were deqcribed an$ strategies for
dealing with missing data during data analysis were evaluated.
 .

Groups T ~ m e1 ( T I ) 'Intervention' Time 2 (T,)

PART IV
I
(XI
Initial test F~veyears later

CROSS-SECTIONAL DESIGNS 1 1

I
1 Level d marital No children Level of marital

CROSS-SECTIONAL DESIGN
2 1I LWBI 01 manta!
happiness 1
1 Haw Orst child
wrthln last 12
months

lave first chrld

1 LWI sl of marital
h,appiness
('4

Lewl of marrtal
1
ween 12 and 60 happhesq
rnths previously , (Y)

Cross-sectirjnal dcsigns have three distinctive feature?: n o time

dimension; reliance on existing differences rather than change following
intervention; and groups based a n existing diffcre~~ccs ratl~erthan
random allocation.
No time dimension
In the standard cross-sectional dcsign data are collected at one point of
time. Tn this ruspwt this design differs from standard pancl designs and
pre-test and post-test experimental designs where data are collected at a
number of different points of time. As such the cross-sectional design can
only measure d f i r r n c e s between groups rather than change.
, The distinction between measuring difference and measuring change
i s important and may be cIarified with an example (Figure 10.1). Suppose
we want to Ieam abnut the impact of children on marital happiness. A
panel design would track people over time. It would begin with a group
of d~ildlessmarried couplcs and measure their marital happiness. Five
years after the initta! wave of the study we might revisit the sample and
rerneawre their level of marital happiness. Bv this time some couples
will have had children and some will h a w remained childless. Snme
couples will have had their first child recently (say within the last year)
while others will have had their first child earlier (behveen I and 5 years
ago) Wc could then think in terms of three groups: the childless, the new
parents (first child within last year) and the established parents (first
child between one and five years of age}. The analy~is~could consist of
looking at the degree of change irr marifaf l~aypiness05 the three groups
between wave 1 and wave 2. Different amounts of change in marital
happiness of the three groups cuuld reflect the effect of parenthood.
A cross-sectional design for the same research question would involve
a one-off study in which we measured the levels of marital happiness
of couples, whether they had any children, and, if so, the age of their
eldest child. Using this information we could create three groups: those
without children, those whose eldest child was less than 12 months old,
and those whose eldest child was between one and five years old. We
could then compare the level of marital happiness of the three groups to
see if there is any difference. If there was any difermce we might be
tempted to attribute this te the effect on marital happiness of I~nving
children.
Essentially the difference between the two designs is that thc panel
design enables us to compare groups in terms of diJferrnf anlnlan ts of chnfzg~
on the dependent variab4e while the cross-sectional dcsign involves
comparing the groups simply in terms of diJfcre?lc~on the dcpendent
variable.
Reliance on existing differences
Because the cross-sectional design entails collecting data at and corl-
ccrnin,y one point of time, all analysis relies on differences in the sample
at that point of time. Unlike the panel design the cross-sectional design
d0t.s t ~ o tallow for difft~r~iiccsto emerge over timc. Unlike the ~hxperi-
n the researcht,r t ~ j
mpnral d l - s t p where thcsc 1s an active ~ n t e r ~ t n t i oby
prodrlcc chnnge or tn crcntp d i ftcr~nces,thc cn>s~-.;cctiona!design rtblicq
on elistlny: difirrences rathcr than examining thc Impact of erpcr~mental
intemc*ntsclns.
Craw-wctinnal dezigni; pmpfny a relati~.cly larwit*e approach to
making causal inferenccq (Marsh, 1982: 6) and can be distinguished
from cxp~rimcntsin this regard. J4i ke crosr-scctic~naldesibvs, t>xperi-
mcnh ruplain variation in the dependent variable by seeing if tjiis
\win tinn is svstcmalicall y linked to \?ariatinn in rile indepenclcn t t~ari-
ahlr 1.. variation in rn;lnt;ll:hapyincsq rnatchtd bv variations In parrbnt-
hnod stntlrs? Howel-er, cxpcrirnents activclv crcnte variation In the
indepcnticnt variable tzuhcn*.lscross-sectinn,~l drqipw rely 071 ckisting
diffcrcnccs.
The nature of 'groups' in the cross-sectional design
The ~rpcrimenta!desirn rcllkv nn compariiil: groitpc. Tndivirl~ta!~ ~rc'
ranrfun~l\,I Elocnted to Rrtwps that prior tc, .my rxyenn~riita 1 Iiitcr-
vcn tioil f lic L~vn groups 41nuld be identical. Tlie rcsearch~rart i ~ ~ r l ) ~
pmduccs JiiCrrences bctwcr n thrr groups by a dcIi bcrate and con trt9lrd
intcrvm tion. This i n t e r v ~ lion
n is the indepcndtxn t variable in the cxpvrl-
mental design. Ideally, this differential treatment is the only difference
behvrrln Sroups and therrfnrp any group differences in regard to the
uutcorn~\ ar~ableshould bc attributable to !hi< d i fierential int~nrcntion.
In tho cro~q-sectionaldesign the 'groups' art1 cnnstructed on thv ha+
of rxirting diffc.mces in khr* <ample. The sample is divided irp intn
groups acctlrding to the catrgnry of the indepr*nilenlvariable to which
thcv happt-n to belong. In thc cnrlier exanplv tIic comparison grcnips
wroulri br thnsr without anv chi tdren, thosc whnw elc?er;tchild was IPSS
than 12 months old, and thnse whnsc eldest child waq between ontaand
five ?car< olcl. The critical p i n t , howev~r,i.: that 'because we mtrct rclv
on euiqtinq differences there can be no random allocation to groups. The
groups nlnv very well hr dlffcrent in other rcspcck apart from t h ~
category of the independcnt variable to which ~ h c yhefong. This, in turn,
can m,ikr i t difficult to work olit whether a11v Kroup difference.; on thc
outccvtw varic~t-llc are duc t r l tlir independent vnr~nblcor to other rrlated
d ~ f f c r ~ n c tbrt\rreen
>s thc crzmps.
Lqin:: thra framerr-ork crl t ? ~ t lclaqcic exycrimcntal d r s i v ai; ?. ftamc of
ref~rt-ncrbt Vie cross-sectional clc5ign can bc rppr<..;cntcd diaerama t icallv
as in Figtirc 112.2. In this c a w the 'intcwentinn' 8s simpIy "irelng in a
diffcrcn l cntcKnry of thc indr.prwlmt variable. Thc information about
thiq and thp outcome mcastrn: iq cnIlected at rhc same point of timp.
Cmw-srctional d e s i ~ n sarc not l~mitedto c u m p a s i ~ o yb ~ t w c ~illst n ~ , repeated ern<<-rcctiilnal design nlitnins cornparablr
ht70or ~ l ~ r cgroups.t. Thc number of cat~piric.; of the indcprniicnt
to 'groups'
1
Non-random None Measu-e on
Non-random None Different existing
cx,,
I
I I
Figure 10.2 Grorr!)~i t ! cmss-srctit~)mld,x5isy~i5
variable ~f~trrrnineq tht. nurnbcr elf 'groups'. Oftrn we will Iook at the
c~rre2atinnbctuvccn continztocr~r'ariafiles such as, sav, age and Incnmr
Icvcl. \\'hen lt~cA.jn): .lt such scrrrcl,~lion,tve r7rv . ~ \ L t i i f i ivhctlicr incrmi,
vanes according tn agc. That 15,w r cnmpare all the a c r IrtreIs lor group<)
and see whct11t.r income varie~qysttln~atically across ngc levels.
h surnmdry, cross-sectionill d v s i ~ n sdiffcr from t>uperimental and
panel desih~sin that they do not have a time dimk1n.;ion. Consequrntl~,
they face difficul tie4 in unarnbrg~~c~uslv i d m ti f ~ ng
r the timc scqucncc
ot events, H c r a u ~ eof this they tncc the prcrtrrern of ~dentifyingcausal
direction. Furthemarc, brrcausc criws-sectional dp.;ips re1y im existing
differences rathcr than random allocatic>n, mv apparcnt effect of the
independcnt variable may in f.tct bc due to other, i~ncontrolleddiffer-
cnccs betwccn the groups. The problrrnq created by this will be discussed
more fu fly in thc rncthodolngicnl problcms section o I Chapter 11.
Obtaining a time dimension: repeated cross-sectional studies
Cross-section,~l designs are frcqucntl y critici7ed for lacking a timc
dimension. Rrcausc. of this shnrtcclming they are lrnnhlc to unarnbijiu-
~ u s l vcstabli~hthc time sequcncc rn which vents occur. This is parti-
c~llarlya prnh!rm with one-off crow-sectional stird~ck.H o ~ ~ e \ ~thr er,
repeated cmsc-svctionnl deGm ad(! tesses thiq yrrlblcm to a llmi trd
er tcnt.
This d e s i ~ ninvalr,rs collecting ~nfnrrnationat a n~imberof diffcrcn!
time pointh E ~ r r l from difilr7;l ~ n f ~ pRIl r ench Ij~rrr 110111t. Rather t h a n
tracking n Ernup of indivtduals cwcr time and sccing how thow indi-
viduals c l i n t ~ g the
samples at t><~cli timc point. 50 Inng as the sarntx rllrr..;ti~~n.; are aqkeii i l l
 ~1qt.dfor cc~Vlurkanatvsic. T h ~ allow v tiit, in\ . ~ . s t i ~ a t /tco
i r both twt-k change
and get sumc Iln<lcr+tandini of the ~ i ; ~ ! u rc)tt ~clinngr. Cuhurt ~n.al~<15
will be discrw~tadI ~ IChapter 11.
Repeated mca5urrs designs can take twc) t 11rms:prospecti\)c n nd rptrr7-
spcctlve. A pro<ppctivc d c ~ i g nis onr w h t ~ r cwc bcgin now and rcprat the
study a t various puints in the futurc. TIic rctrospertive r e p ~ a t r drneila-
ili-eq design is onc tlwt draws on exiqting data scatsto exarninc pattcrns of
change up to tht. prc3rcbntpoint. Right mcv wc could use tlic C;cntlral
Srxinl Suweys of tliil laqt 25 years tn exarninv patterns of change. Census
data cnIlcctcd nvrr thc pact ?car+ can br an,~lvwdin a s ~ r n i h manncr r Zrr
rnnn!. other sctc ot data that are ro~ltiiirli'ut>llccttld le.g. annual I ~ I V I F T C ~ ~
\tnhshcs, r n a r r i n ~ r~ l d t i l ,tcrhl~lyhtilti>ti~+E.

each of thew tndtlycndcnt cross-sectional samples rvc can build u p a

picture over tirnt* nf whatever v a r i a b l t ~n7c ~ Irave.
Thc naticinal c~lisuk~n racll count? an r>b\.iotrs e\;lrnplv o f thic h.pe
of 'design'. T(1 thc extent that the wrnr q ~ ~ c s t i i marc s <~<ked in cach I
census, changc*s can be tracked uvcr l o n ~pertnds. Many cr\un!rirs have
regular national hwirll surveys that track a far wider ranjy of matters
than any national ccnsus. Fn the United Stntcs the General Social Survey
has been conducted annually since 1972 and provides an erccllcnt way
nf Itwking at changrs in hchaviour, belief<, values and attitudrs over the
last quarter cenhlsy. In Britain the Blitish Social Attitudes Sttwrv is 11st.d
to achieve a rirntlas md. The General Hntrsehold Sun-ev provld- data
for hvn dccadcs and thc Family Erypnrliturc St1rr.e~vi~'IJ5timr scries
data for over 33 ypnrs. The International %rial 5ience Survry lnvolves
1
asking a similar core nf questions each year to new sarnpres in rach of the I
countries that arc1 members of the Int~rnntional Social Science Pro-

I
, gramme (35 countrics in 2000). Rcpeatcd croqq-sectionat survcvh provide
an rxcclIent snapshot of values at different points in a pcriml id rapid
change. '
Dia~ramrnaticnllvthe repedtcd cross-scrtional design can be repre-
w n t d as 3 ?;rricu;cri independent <arnplc.s IFipre 10.7). Each timr point
d r a w cm a nC*w samplc. Tvpically, car11 <ample will provide a wide
cross-scctinn of rastas so that useful crtlss-wcticonal analvqiq r a n Irc con-
. can track changes by compi~ringpattcrt~s
ducted witllin rnch s a n ~ p l eWF
in each sarnplc with those in previcn~sor subsequent sarnpTps ( c . ~rather ,
than just <)TIC aEtl group khev include a widr range of ages 5 0 that at any
gilVenpoint n t ttmp the age differences c a n be examined).
M'hile this itt-sikm does not prmit thc trackins ot individual change
o%.erhme i t enable-< l o n ~ i t u d i n a lanal!'siq at the aggregate Ic~~ck Ise
Chapter 0).Fwr!hrrmorc, these r~pca tcd cn)ss-srctinnal q h r d i ~ scar1 bc
 Minr, ever, mn*t 01 t1ic.w thscat? Ihictt>r\, m , tcrr;ltron, ~ instrulni3nf d v c n ~ ,
rV<rLv.\rtln,
+tL~ti<tlc~1 inort;llilv ant1 tc*tini: ctloct.;) a r r w bi,ca~l<t, of
thc n ~ r r - t ~ ntvtt*rnc*nt
~~b ttt thcsc dt><ig~~+. 7 I ~ L >il\ re not pr(~L~1~-111\ wvtli
srtlr5-scctioiial ilc.;i~n.;. 'l'hr main bhrt.,~r\ t i ) ~ h i r~ t c r n avalidif\ l of cross-
~ r c t ~ o tdwigns
~al .ilcani frr>it~h v o ~ o u r c c q :~ ~ r o b I ~in ~ restabli3liinji
n< c.luse
ISSUES IN CROSS-SECTIONAL w i tlic~uta tirnc. Jirn~~tision; and problem\ a t thc I c t ~olf meaning (Mnrqh,
1482). Problen~s.lt thc Ievcf of cal~sc.arca, i l t cclune, an ~ s s ~ t~c\ ~ 1 1 any
li
DESIGN ci~-.i~m b u t cnrc+st~c.~lcru;rld ~ s i v .ire s p r ! i i ~ r l . i r l v prone to problrnis ~t
thw level. !'roblrn,s a t tlic Icr.cl of mi,sntnq a r p also common t o ill!
dc*i~msand ~ r i c t -\ v ~ t fequal ~ furct. for t.\prrlni~-ntat. I o n g ~ t i ~ J ~ ,n~anl t l
~ r ~ ~ r ~ - s c cdt--i):ti\
~~onnl
h.l;lr~h(1 L)F;2) ]\.I\ 1s.l-~tfcnclearl! anrl fort-t*f~rl I\ about thcsc trt.t~13rrlll-
1i.m.; Shc takcs I.;\ilr> r v i t l ~those who t i in i.;.; cross-s~ctit~nalc i r~v r y
Cross-scction~ld r 4 ~ n sarc probatrlv thc most widely urrd d r \ i g n r i n
J ~ r i g n sh e c a ~ l wof thcii. ,~llcgcdprobl(~rnswill1 causality and r n t w n i n ~ .
wcial researvh. C ~ I I P r ~ a s o nfor this prlpularlty is that the!, cnablt- the
Shc argucq that Iliic d t ~ q i j i nis not nt.arly a + flawcd as i t i s fr~~c1~scwtlv
rc~enrclierro r\llt,iiii r c ~ z l t s ~ l a t i v c ~quickly.
ly 5ince data arc ct~ll~,rtr-J ;~t
p u r t r a y ~ dto bt, Thc f r ~ l l ~ b w i ncliscuwion
g Jr,rt%.son Mdrsh'\ ~ r p ~ r n t ~ n t .
one point nb titilt1 t h t w is no need In .cv,.,littrlr various tollow-up stngcs or
intcrcenti~m+hr.tt)rr* a n n i v s ~ n qthe d.it;t. I t iq aJqci h i e t h ~ t tjt11t.r , t hlngs
being r c ~ u a (l c K. 5,implc s i x , p n y ~ ~ l , ~ t r osna,n i ~ l ctype), crclss-<r.itronaF
(lc-ignq a r ~ . nlrwr- r o \ t rtfcctivr th,ln c t > n ~ y , ~ ~ a tcl l\ty. c r ~ m ~ w t . land l
I n n ~ ~ t u d i nral rll + i ~ n +'Tl-rls
. 1~ becauhcn crti.;s-scctir>n;il des~q~i.; d o not r n t n i l 'fl~crci s nt?den\ i r l th,~!
~ irn5s-.;ection.11 51rltlic. I , ~ c cprnbIcm\ ~rl~aritilvin,:
t l i t ~cusi 17i r1~[11~~ift~cl iltlt;l collections, (rar-A~ngrcspnndenb r>r t>f c r p r r i - cn~~.;nl >~ivI m i ~ l i1t ~ crrrrlh-
vari~l,l(-+.F\,t.n I l i o ~ ~ gh lvio t ~ r n t ~ - c ~ l . d t\r,i~~iai)lrs .
liicntal intervcntio~is. I,itcd cir gruz~psnligl11~ l i f f t lon r an o u tcnnitb v~ri;lbIcwc cnnnc~tIjr L;.I rc that
C:ross-scctit~n~lltlcsign~can bc irlra l for d c d p t i v e ar~alv.;i.: I f we thcsc dil!crencc\ art- r l ~ l ct c ~a mri,si~llilrk bi,lwc.c,n ~ h vari~blt.5 r (RlCilurrk,
sirnyIy want tcl ~ l t ~ s c r i l lthe t . c h ~ r a c t ~ r i ~i )tt t ac ~p0puldtf~1n. thtbir a t t i - 19h-l).In CI.r,~ptt~r 1 4 ) 1 ~ n t r o d ~ ~ can
c ccl r a r n p l r r ) f the effect af parc~nthr~oit
h~dcs,t h r i r v c ~ t i n grntcntlon or thtbrr bu! inK Fatterns thtxn Il~rcnlss- .tnt~is (have nc-,children; plslpst child ~ 1 n t 1trnp t ~ v e x old and eliIcst chiltl
.;cctional stir\ PI 15 n nlnGt satisfactcln i v ~ ovf obta~nln:: 11734 c f r ~ ~ r ~ p t i v c bct.c\wn 7 and 7 r.r>r.: tdd) and marital h,>ppint*s<.Fcen i f tht*thrrr*gmup.;
nntc~rnlation.nut crt)<~-sechonaldesign* .Irr not restricted tcr di-.;cr~ptf\~e 11 i ttered in tcrrnh t ~ nzarltal
t happinex\ rvv i ~ o u l dharrc difi~cultvi r ~ c > r k ~ n g
nnalyqis :I<~ 1 1 t~1 ;lrgued below, yrt3pt.r analysis that uses qtatl.;tical out whether t h i ~ dil trrtsncc M'RS CIUC' t v t l ~ t i cliift3r~nt
r parental +iaf11.;. 'l'hc
controls c n ~ b l c *crtwq-s~cticmaldnl,i 10 p r r r v ~ d cr.aliz;lblc itifc~i+rn;l tion 1I1i-PPp o u p ~ w ' r i ,likoly tn differ i n w,iy< tvlhcl. 117i117 just thc~rpart~nthnod
J bout causal prt)crsscs and for testiiig cn t ~ s a lmadrls. h t a t ~ ~-s 1 1 0 ~
I ~ ~ i ti hg y \lave been tc?gcthrlr, their age profllt., Irbvcl uf
'There arc., I i o ~ ' ~ va~number r, oi mrt h n d o l o ~ c a land p r q ~ ical
c t iqsues wtrrkfnrce partrciyntiiin and so forth. 71rf,<t, ~iifferenccs,rathcr than
ot w h i c h wp need tri hr aware when ~ i s i n xcmss-secticln~~l designs. An parcr~thoodshtli\, c r l i ~ l daccount for diffc>rt.na*hin marital htippintlsa.
;EztBarencss of thtvsr* 1s5uc.sshould help m i n i m i ~ ethe ~ h o r t c o m i n pof this
clr5ien.
Thi, prnbtrm r r f c r v i f r > ~ ~ n r l i n~g. a r i ~ > t lir; In c m ~ . ; - ~ ~ ~ c t ~ r r n n l
l ~ t,lcklcd
\
dc*kigns a t thy rlrrliro~rr~lir.;~.; r ~ ~ tli,~n at tlit. data c v l l r c t ~ o nrt,Ij:tb
~ t i rathrar
as it i s in cwpc~rinic~r~l,il r l ~ ~ s i 'Thc
~ ~ s.inlirllt>n
. 15 nchieved lv m a h i n ~
t l ~ rgroups a\ \itni!ar 4% pmstble by ~ t , ~ f r ~ t r r - , 1.crno~ing ~iE~/ ~liffrrt.iicr~s
s rfirln IIII;'~'bt,rvl cmthl,'r'h'r/W hcn jvc cmnpart- ):rtvu ?I;.
hctwccn j i r o ~ ~ priflr4r
Wc encnr1ntt.r p r r ~ t ~ l t ~ with ms intcrn,~l \>,ll~dih.rvhcn llit, I O ~ I L - nncl MT need t n hnou, Iiirw l t i l l ( . I ~~lifferrncc41 II~J h r to ~ Y t l f l / r ~I I I I ~ I CI*> i ' ~ / / y
;friiitlrrt' OF t111. ~irv5ti:n docs n o t enatllv 11s tc? clitjc~scunarnbigunu4v unp rrr!rurl. 5inct. rrthtsr I h r n ~ ,trial. nut bc cqual n t v d to remnvc n1.Tnv nt'
c'rpl.inatlon oi r w r rcsitlts (31 er anctt h r a r t.\planation. C'arnphr~TI and tf~rditfer~nccta \ pr>\-r27ly ~ n cnrnp;rrt7 d !]kt- rv1tF1 like.
C;t,lnlcr- tlLhr7l) h.n tx ~dcntlfiedA n u m b v r c v t t;lctors that threatr-n intcsna! I n outllnc c ~ fhrviv I.; dnnc \ r - t l l 1 ~ .tli+ru<wd in Chapter 11.
~ t ! - thr<ta Im\.c b w n dt>cusst~lI~Idt%!;liE in Chaytcrs 7 and S in
~ , . ~ l ~ r l dnd I lt~iz,cvcr,tlic h;lkic IticEc can bc 1ntrnJ11~t.tEnow For the c l i i l d r t ~ n.~nd
relntron tcr I o n c r t u l l ~ t ~and
~ i l experinrcnt~ilrli-.ignq. r n a r ~ t nli;lppirat.;s
l t ~ ~ r o u l dcr~n~l,.irr~
r u a r n l ~ l \uc tllr marital h.1ppinr.t.; of
 tlirtbt> t ; r o u p . 1 1 i c 1 ~ 1\vitliot~t
~ *In{ ~ - t ~ i I d r ~t l -i t~~ ci t, -\Y!I~-vtbldtv\tc11iTd [ \ Chrlrl's acaOPmlc
Model 1 ach~euem~nt
11ntlt'r tlnp !par ~ t .lpl t n n ~ Ilio\tb
l r\,htr.;v t<lclt.\l c h ~ l d1.. bcluvcrn 1 a n d 5 school
\.i..trs (111.1. %rrnvl\nc r n r ~ h %a\. t th,\t ,>n\ rliibt.rt~nc~b~ rn ~n;lrlfdlIia~j~jnc+.
3r(' nt\t ~ L I L .to c l i ~ l t l r cIwt ~ ~ r i t i r to t h c d ~ i f c r ~ ngc n t proiilt..; 43f tlit. thrr-t*
gn'~ip\ pm't~ll'.. I h r ~.lrgllr , h ~ i flrt~sc
t r t ' l ~ uha\ r trtal'n 1larcnt.s Iongar
Parental value
art,, r?n arcr;lgc, oldcar than !hr~.;c w ~ t h u u tch11clrt.n Thcby ~ i r q u t h~, ~ tit I \ placed on
tht, aytb ot partners, not thc prcqrnce of c h i l d r ~ ~ n~,v l i i r ~ infl~rt.nutl< h ~ducat~on
r n , ~r t t l~happintacs. ach~eveinent
Il'e idn It,<t thr+ pos.;il~il~l\hv r c m c > v i n ~the, actv J It trrtncc..: from ottr
cornptiri5t>ns. Wc c.ln make a nurnhcr rlt nbsfrictc~lcninparlconc. Firhi wcm
171 1<11tt t v b l . c \ t p(-t>p11+ L ~ ~ l>t>t\vet*n
c ~ ~ l 20 *ln{T2 1 , i r i k I \ t y S \ v i ~ ~l1far , t ;(sft!~~)r !hi>
p a r l ~ c u l ~ii:~.
~ ~ r grrb11l-rlhtb mnrital h.ipprntn\% ( > + Ihc thrt*t, Errwry, d i f f t . r ~ ~ l Parenlal
l ' h a j t IS, !vt3 rc,slrlct o u r cumparlsclri arruqc the t h r t ~ t ~- ~ I \ p3 L I to f l ~ t ? \Z ~ ~h ,o Model 3
rP.sources
art, !nrit-11 Ihv 5.1rnc J ~ L ' .1 4 ' ~ 'C(TIII J r r \ p ~ ~ dtliic. t ch3rnparrcon +cparattl\ Ch~ld'sacadem~c
within caich ag<*h+rcli~p(1, E. 25-2'). 311-11, 74-W). 5tlppc)sc w e l ~ n dt h a ~ , achievement
;rvll!~?r t*.~c.h Srl-rttpinK, thtyrP.lrc strll dilft'rc~ncr\In rnarilal 1 ~ a p p ~ n c r ; s
hctkvcrti Ihr- SrtrupG ( I e bt,tzvt~t.ntlic clitldlt-w, I he nt,zt. parl>ntsGindtIic>st,
with (llilcl Eigctl 1 10 I; ~cdr.;)dr+pitt, ~lirnrrc o ~ n p a r a l ~agt3 l c ~ prt,irlu.;. M'r
(-an thcn *it It.;l~t$ b ~ i t thiq rtal;lt~~>nchir
thn Ix~t\vt~t*r~ tnar~t,ilh , ~ ~ p t ~;lnCj css
r d r ~ ' r ~ t h o t )+!a1115
d i- Itri! tlllr' t v the ~f t l ttbrcnt JCI. prtrt ~ l r + c l t l h t ~thrrt, ,~r!:t~v<,'I~nrling,I c c ~ ~ ~ r c l , ~ b! irot ni v c ~ rkrntlk~nq
~ ,inrE lung c.ancc>r I11 nil
~ r ( ~ k 1Tlit, p . IOZI( nt fhiq i \ ~ i n i i l , ~tor fht> it~i:i~- o t i ? r i ~ t , - ! t ~!:~L>LII?, ~tc tllal I < w-,ay FrI%vPsthat smoking cause3 tung sanccr, 11~1i f ~ : r l r . ; mran !hat thc
~ o n l t s t i ~~l\c'rf
v t ~ ~in c-\p*rrrntan tal d r s j ~ n s . l i v ~ n t h t camnot
- ~ ~ ~ btl rul1.d n u t ' ( l W 1 721
I-hrbnlclrcb~,.rrialllc. Ihat art1 ~.c>t~lrr>llt-~l for <t.~li~tir.all\, lhc rnrlr.taw t bt-an
I,e conirelcnf thnlf t h r * fiii,>l rr,lati~>nslirpw r l ~ r ~ cI,ctr-,.~.cn l N arid li i h
r ~ fwtinql a t h , 7 I r~*i~jh(m,h 111 b t l t n . ~Y ~c ~~ ~n\'. ~~ l Crt~v.-<rt-tio~r;II cis til can h ~ l p~b~f,lbTi<I~ t t t t~C.FLIS~I r e l , t~ron5hip.: clo v n t
TIM-prohlr~ln will1 till\, htlwtwtnr, ir that wit c.ln rinlr~ ct)~itrcll for cuiqt. L%'hrle i! is truv that "I c ~ ) r r ~ l ~hctrvci~n ~ t ~ o i tlvn
~ ~~;lriahlc< doe.: not
r a r i ~ b l c <!hat 1 ~ 1 I~a\,r. % thottp,lit oi , ~ n d,~l,o~it 1\,111ihnvt.li,~\.r ~nforrnat~un. ~~.;tal~lisli caurntion it is a prc-rr>qlrisit<, for r < t a b l i ~ l , ~,Incat1\a3 ~ rt*la!ton-
Rnntlrrm aFl(>r,7tionin ,In c~\pt~rirnt~rltal de.;tgn r4ft.c titmclvconkrr3lq Ft~rall s h i p If thcrv i s nn rrrrrt4,ltitln rva* cart br, cont~dc+ntt h ~ tthcrt. i.: ~ r r r t ;I
1 nr1,117Tc\ - Lnnivn ancl ~ ~ n k n n w\l;ht>n n wt. c t ~ l i t r t ~fob.! vari,~L>l<>.; st,it~~- cxrqa1 rcl,lti~>nshi(> I I I r r ~ l r r ~ r i t r r rv.lrinb!c+
y a.; cauqcs can ht,clf jiist ar much
ticallr at tht, dst,i an,1!?.<1\ qt.1gp t r y t a n ncvrr br, .;tlr~%t l l j t \tmc11~1\-p + c i c n t ~ f ~importance
c as I(~callngC L I U S C l y t r r l ~ , ~ c cmanu-
~ t~%~r.t . ~ ~ nthi* c
c-cmtrtlllvrl f ( v all rclc\.;lnt 1~ari.ll7lcs 'rhcrc nq alwav.; th(* p(>wibilit\r thal t,lctrircrs ut+ru~fdconqidcr n fintlinr: of ) r r l ctrrtrl~tit\nbrt\\.tr~n.;lnokin<
,In> rtb!;ltitlndirp Lrrc h ~ v fnunll r brh\,rt1nS nnll k' cc>liI~1 be d ~ ~ cL sr7inc I
and lung cdnccr lo Lli>an imporla11 f i ~ r d i n ~
zrncnnkr,! lud t ariablt*.
But t l r i ~is ilot to ctmtlenln crn~s-+-t~ctic~t~aI i i c > ~ c n$1.;s bt~tngof n o u+t. in h'f0111 IS: A PKIOItf R L A s O M l V G A K I ? 4.0 110(' R E A + V V l N G
cb\~lc~rinq c L ~ ~ ~rrala~lt~nd1ipr.
\aF In~lc~d .;inc.t*
, ira sosi;ll +cier~sc.rrse.in-h
t l i ~ c.1pat.1
r ty ttb ~ r ~ l t ~ n .t3iIl1ert v c p r ~ c i ~ ~ ~t>r i lt~t111c~ITv
lv < i ~ i%~v~i ~ i a n i p ~ ! I ~ ~ t e Thtl moqt ~ ~ s c f tuvl , i ~ti3 $(I al,nrrt tht- cau.;.~l nn,ilv.;i< of crtws-3r,ctirmnl
wI~ti111on~ ,I< r q ~ i ~ r t ' lby i 1~xpt~t.imtvlS111 t l ~ < ~ g n~cc ..11111ltt.1i Ilit~rei h c~Xlc.n d.1tcl i s tal d w t v ninelt.ls, o r tlo~r-rhart, c>r -path dicigrLin~i. of tlw tvpe
t t - ~ vii, C T T T ~, a l ~ c r ~ ~ , i ttt3 i ~ <.rm--\tb~
t*\ lit>~i,i! r{t>\iqri+ elluhtr.~t~,tl in I rqrrta I 1 1 & ~ l r ir%l,ctt
l hcrtl In tt71.; Elr~uk 'rhc t a 4 o! !I>cd8?ta
RJ t l i ~ rI han rejty-[in\: crtl--vv~:l lon,i I tltb<iqti+nc heqn!: t i - t - T c b k + for ~-aut,ll .in<ilvct it thrn lo cvaluattb l i n ~ \ n,cll ' t h a w t~iodr~lq Cif tl-11. Jnlti.
an,iF\.<r+ 1%c n i ~ ~yrvctvd sl -\ qtc.~n,>trc,illt, . ~ t i ~~-arcfuII\~
i !\ 1111 tlw , i n . ~<lI ~ \, Tht. Itc.rn1 4 tfl~.clrir+ ,ind nlod~bls2 % a ):uidc. t c ~,inalt.~i\i + ,it1 ~ m l ~ r , r t a n t
, ~ n dsoilti-!>! ctati\ticnll~ ttjr t,>~ttirkt?i.it mrchf yl.lu.;~lrl\- t , \ ~ l , i r l l the t+>,rl tt>r t 4 i t g Ftar\oii at1.11y5it1t: crt>c+.;e<l~otr,lId a t ~ '.I t ~s tiit, rntwlr.1 !halt

rtlrrr~T,itic?i~s n r r , ol~srr\.c~. Fir r t l i r ~ r n t r etvc iii~lslc n r l ~ t , ~ n t lrrrnind v ollr- ~ t n n c f sbclwctan t l ~ ri\+r>archer

r ,~trili~nhrtdlrtltlnp~r~'ir;inin the n t t c . m ~ ~ t
~ 4 \ u r<~ !lit, t tmtnti\ c naturt. 01 #in!. rt>nrliih~tw? i t tm d r . 1 1 ~In ;Ins s r ~ t ~ n t i t ~ c t c dr,!~v
~ c-,?u<.il~ n t t > r e n r t yr\larsli
' 1Q42: 7 2 ) It i s imprlrt.~nIt ~ ~l ! t ~ ~ . tthe
~lop
t ~ l l ~ l t ~L)llr
.l\ n1tfdt4 /rr,'orra ~ ~ \ i ncrtr\.-crctic>n,~l
q 1.1.1td ti, ds,~t.r*C ~ L I L corlclusior?.;.
. ~ ~

A l t h ~ ~ i ~Findin):
gli th,l 1 t ~ v,i w riahlt?q ,I rc ~ r l r r t d ~ ~ tdut>+ r r l ntyt ez!,ihl i\h rht,rc I.; a dif t ~ r ~ ~be!rvcr.n r ~ c ~ .n J V rorr reahonrilg a n d rrrl Irut r c a + n n i r ~ ~ .
carlbt, I! t1tw.; nltun that ,i i . l t l s ~ 1 ~ ~ I ~ ~ ~ \ R i\J ~~I~Ir w > R . ~ E ~\ bl c fTtlrkV~ ,4 r)rprt rt*n%onrnc in\.nlrcs yroptr.;im:, r ~ n the twsi.: nf thc.c~r~tik.al

cunsirlcration~ and prcvious rcscarch, that X will cause Y and will
thcrcforc be cnrselatccf. This reasoning is ~ndependcntof the data we
may have. Ari hnc reasoning, nn the other hand, is the process whereby,
nn the basis of an observed correlahvn in nur data, urc make up a causal
story as to why the correlation exists.
Thc psoblcm wilh nrl Iloc redsoning is that we can always make LIP a
story r e p r d less of the correlation that we find. 1t iq not hard to t h ~ n kup
a qtory that is consiste~itwilh a correlr~tioll - it is quit^' a different matter
to anticipate, on h e basis of rcasnning and theory, that a correlation
should xis st and then find that it does.
I n silmmary, cross-sectinnal data can be quite effective in dcmnnstrat-
irlg where a cai~salrelaticlnsliip probably does not exist. F~lrthermosc,it
can hclp evaluatc which, of various competing theoretically derived u
priuri models fit the data best. Finally, wc can use cross-sedional data to
evaluatc and modify a given R prrori model.
Prc)perlv d ~ s i p c d,~nticiyaling
, rivdl causal explanations a s far as powiblc and
building in wavs to test t h ~ m [crc>rs-sect~onal]
, surxbt.ys can provide evidence
for arid against different causal ~nodels.The t.\.rdence i s not prouf, huwever; it
I S only as goud <I\ L 11c mnclr~lis. ( t I c ~ r s 1982:
l ~ , 117)
Another prnblern with cros<-sectional data is that Even i f we estabIish
that two variablcs are correlated this does not establish their causal
(Iircctio?~.Tn order to illustrate this point let us use the parental status and
marital happiness example again. Suppose (hat the cross-sectional study
showecl thd t the chiIdless couplcs arc less liapyy than those with babies
and that corrples with the oldfr cIiildren are t l ~ chappiest. The problem is
that evcn i f we assume that this 14eflectsa cai15aI relationship between the
two varial-4es we know nothing about the direction of the relationship.
What comcs first - childrcn or marital happiness? Do children increase
marital happiness nr are happjlv marricd people more likely than the
unhappily married to have childrcn?
A panel survey could resoIve this questiun. We would know the level
of marital happiness before children and again afterwards and scc if thcre
was any clerrig~.We could also check whether thc initial level nf happi-
ness was linked wrth the likeliliond of subsequentiy having children.
However, t h c r ~is nu simple way of doing [hi? when all data a i e collected
n t and aliorlt the one point (if time
Onr way of dppronching this dilemma i s tc3 dcvelnp a priuri rnodcls ta
tcst. WhiIc support for a n 17 priuri mociel docs not prove causal direction
it a t least prmidcs a kheorcticnl basis for arguing a case and provides
empirical data that a t least arc consiqtent with this. Hairing found data
consistent with thc thesis o f rr partic~tlarcausal directipn the next step
would hc to adopt a dcsign that can establrsh for causal direction.
ISSUES 1?J CI<CXSSECTIOVAI IIFSI(;N 181
Evcn in thcrsc c~rcum~trlncrs in wliicl~we cannot develop a new study
to establish causal direction wu.c can shcd stmw I ~ g h lon the rlircctic~~~
by mllcchng rrtrnspectivu infc>rniation and 23v asking people dirrctly
about any effect childrcn may have had on their marital l~appiness.This
approach does not yield unambiguous data hut, bit by bit, i f builds a
picture regarding which causal direction is the more likely.
Establishing causal dirrctinrl, however, is not always pmblcmatic. 5)mc
independent variablcs arc fixed (r.g. ge~ider,race), others are not r ~ a d i I v
subject to our rnan~pulation( e . age, ~ religinus denomination) and for
others the time nrdcrings are straightforward (e.g parent's education
prrccdcs rcspimdent's educn tion, cduca tinu l r v ~ lubually prececlrs the
tvpe of job we gct).
By establishing causat explanations on thc basis of cilrrelations, cross-
sectional designs a r c nftcn accused elf treating human action as dcicr-
mined by cxtcrnal forces and of neglecting thc rule crf huinan agency.
, Marsh puts this vicw most cIedrly:
The aim of c x p l n n n t i o ~i s~not just lo show high cr,rrelatir)ns between v ~ r i a b l ~ s ;
sociological cxplnm tirjns IYLIII~L~ i1Imo5t i ~ l t v a yturn
~ (wt to be r~ flop R I I V W 011
~ ~
thrse grounds. It must also show how the actions of the peuplr involved were
the actirms of ctlnsciuus hurndt~hcings, rcwting Lo a n environment, trying tr,
makc sensc c-rf it and pursuing vasiuus goals anrl actions with morc or lcss
success. Only cxplanatinns thal take cognizance of the meaningful aspect of
social action will satisfy us as human observers. (IqH2: Y 8 j
Herbert Blumer (19Fih) c o i n ~ dthe term 'variable analysis' to describe the
type of analysis and understanding providcd by cross-sectic~nalsurvey
designs {hewould also apply ii to longititdinal and experimental designs
. arglles that ai~alysisusing discrete,
of the type already c o n s i d r r ~ d )Me
quantifiable variables is inadcquntc for at least thrct rcascms. First,
It l e a w s oul the actual compluxiher of acllvity , ~ n dthe actual prorrsscs 05
internchon In which h u m a n 11fehas ilq bring. (195b: 101)
!%cond, i t ignores the actor's ii1tcrprct;ltion of khr r ~ b u tion
a and b ~ h n v -
lour and tlic way in which thcse intcrprctatinns impact on their actions:
The i~-tdepc.ndcnt vnriablr ih put , ~ tt l ~ rhrginni118 of the prucew uf
' tntcbrprctdhun anti the dependent varlahlc. at the tcwn~lialpart of thr process.
Tilt, inten~eningprocess is ignored . . . as sumethii~g that need not b r
considcrcd. (lq56. 47)
By ignoring the i n t e r \ , m i n g process one cannnt ~ u ~ d c r s t a ntuhy
d the
independent and dependent variables are or arc not related. Where
 I H2 CROSS-SBCTIONAL DESIGNS
ht~rnaii hehaviour is intentional and based on l~ndcrstand~njis of a
situation thcn thesc. intentions 2nd uiider~kandingsqt~ouldbe part of
the explanation of thc beliaviour, L'lumer argues that variable analysis
usually fails to do ihis.
Third, Blurner argues that variable analysis w wnches people and thcir
b~haviourouk cjf their contexts. By trying to work with discrete, quanti-
fiable variables onc loscs touch with the 'whole'. By dealing nnlv with
artjh'clalr); isolated hits of behaviour the full meaning ot the behaviour
will be misscd. By looking a1 particular aspects of proplc's bcliefs and
actions without looking a t the context in which thpy take place i t is
pmhLiblethat we will niisundcrstnnd lhc meaning of the RcI~aviour.
In the cnd the qucslion should b e n o t whether I l ~ cmeaningful and
intentional dimension of behaviour should constittlte part c ~ fsncial
explanations, but whcthcr this aspect of bel~aviourcan L7c captured using
the data coll~ctinn methods usually associa tcd w it11 cross-scctir>nnl
designs.
Thcrc Ilavc been a variety of ways in which thosc uskng cross-sectional
survcys I ~ a v gone
e about {'roviding 'lncnnit~gful*
rxplnnations. As Marsh
observes, thcse
a p p r n ~ r l l c sdiffer in whnm t h r y bclirvr thc n>mtjlrig ~ h u u l dbe s~rpplredby
ant! how rni~chand what tvpc. of e\,idencc is required before nnr can r~alidly
nttribuir mcaning to action (1982: 1111)
She identifies a number of approaches. The first, and least satisfactory, is
that which she calls 'facesheet sociology'. Tliis consists of correlating a set
of sncio-demographic characteristics (c.g. sex, class, ngc, rcligion and
ethnicitv) with the phenorncnnn one wants to explain. Tlie more one can
dctect cori.elations the more we have 'explained' variancc in that phe-
nomenon. However-, this approach doe? not provide sociologicalIy
meaningful explanations. 'To find that gender is corruIated with
rcliginusness is not to u n d e r s t a ~ ~why
d gender is linked to religiousness.
We netad to ~znderstalidwhat it is about gpnder and religion that explains
why women are more religious than men. Typically, as Marsh points out,
this approach to supplying mean~ng1s brought from outside the data
whcrc, in an nrl lint way, t h ~ rcscnrcher
' si~ppIiesthe explanation horn
thcir stoc-k of plausible sociolngical idpas. While these explanations
ma!; be consistent wiih the data they arc rarelv ct~mpellingsince many
'cxplanrltiorls' that arc equally consistcnt with the data can bc dc.vt.loped.
A second a p p r o a c h i5 to ask indi\,idu;lls tn s u p p l ~the
, meaning. We
c-a11 ask them w h y they acted a r they did. Tliis approach is criticized
h!, thnse who argu13 that uftcn people d o not know why they acted a s
they did. Thc response to thiq objection 1s that if people d o not know why
thcy acted a s thry did then these 1s not a pmblern sincc wr cdnnot s a v
that that bchavtour is in any sense mcaningiul A s e y n d objtction 1.0
asking people tn give their own explanations is that t h ~ ymisunderstand
ISSUES EN CROSS-SECTIONAL DESIGN 183
thetr behavio~ir.Experiniental ~ v t d e n c cshows h a t in certain euperi-
mental situatiims we can drmc~nstrateen~piricallywhat causal factor led
to given behnvlour but thc indi\liduals themselves are unaware of what
this factor is. This objection has same merit for somc behaviour in some
situations but it does not mean that individuals, in many situat~ons,
cannot supply at lenqt some reasons w h y they have behaved as they did.
To say that they do not have access to the full range of factors that
affected their actions is not to say that they have 110 understanding rrf
their bchaviour. If we were to ask a mother with young children why she
wanted to work part time she woulci no doubt be able to identify a
number of thc rp;lson3 behind her decision. That she might not be aware
of or he n b l ~tn artrculatc all the intluenccs is not to invalidate the
irnpnrtancc of those she has identified. A third objection is that indi-
viduals may indeed bc aware of the reasons frjr their acticms but, because
the reasons may not he all that nc~hle,social desirability factors wilT
prevent them fmm reporting these reasons.
Whilc these objection5 havc some merit, it can still be useful to ask
people for the masons for their behaviour. Marsh argues that 'actors . . .
do have privileged access Lo their own experience but not to all the
drtcrminanks of their o w n brhavinur' (1982 107). Hcrwever, M a r s h says,
w e shnuld nr)t cxagscratc what we will discover and we must b e a t the
explanations provided by people thcn-rselves with circumspection.
A final approach to includi~lga meaningful dirnmsiun to explanatinns
IS for khr rcsearcher to supply the meanings (1982: 111). Thcrc will be
many situations where we simply cannot reliably ask individuals for
their reasons. Often we can only identify reasom by Icmkinp, at a number
of peopEe in similar stti~ationsand reading off commonalties b e t w ~ e nthe
cases. The indtvidual actor will normally not havc acccss to the addi-
tional insights available to thc researcher who can ohserve from a range
of cases.
Burkheim's (1970) study of suicide provides such an cxamplt.. R v
studying thc statistical regularities he identified the social groups that
were most 'suigenic' - the groups with the Iiighcst suicide rates. By
asking what these suigcnic groups had in common he sought to provide
the reasnns for suicide. He did so by proposing that in thew groups
people were most likely to experience anomie - the feeling of normless-
ncss and a sense of not belonging. He argues that thrs subjective feeling
is distresqing to the extent that penplc commit suicide. Durkheim sup-
pIics this rncanlng mthcr than i t corning from the data. rt is Durkheim
rvho rirylrrs r a t h e r than (~tnylirir.rlll!td ~ m a ~ l s t ~ nthat
f ~ , qthc suigenic groups
suffer more anomie. However, he ncvcrtheluss attempt5 to lnakc sense of
the empirical correlatio~~sb! supplying a sublectively meaninghi1
tf t I ~ cl~nksbthvcen rnernhership nf certain social categories and
a c c n ~ me
s~~icide.
Marsh (1982) discusses a t lungth a study of the social basis of
depression by Bmwn and Harris ( 1 978) where i k was bclicved that those
 ISSUES IN C'KCX-SECTltJYrZ1. IlESIGN 165

ctlltcrlng From ~lcyrt~c..iunr \ r l [ not bc full\ ,Ir\,artAo! all tht- tnctors d a l th,it
~ .Irka rcflccti\ i3nf thi. p o p t ~ l a t i ~rm
t i t h r rvt3rc.c f c ~ ~ to ~ rcprewnt.
n ~ ~ !
i l > n t r , b u t ! nto
~ lhr-rr rlt:prrsqic>r~. I;rn\tTn a ~ i ~I4;lrr1% ! adopt an ,ipproa<.h For thi\ rcmnn c r c w ~ - ~ c r l ~\tnrv~ys
r ~ ~ l ~ lim'1,
~ l ! ? C * C C ) ~ Ct l i t x d i v p ot
~ ch~~it*
nlrf ~znllkcthat uttl~;.c.d by Uurkhr*ini in his shtdv (Ti suicide - but w ~ t h For h t l ~ d i ~zvli~re
"i acctlr,llcn ricscripticln is r c q ~ ~ i r r(r d K. pol^ tical pnllq,
ant* diffcrcncc. I'licv cuIlwtvci ~nft~rrndhon a b o ~ ~what
t tlrr!! thought househnlil taxpend~turc . i ~ ~ t v r y xWliilv). csir\r-srctio~ral rlra51~msIiavc
W O L I I L be~ ~ h ~ c n c i a lp r e ~ i p i t ~ ~ t ifactors
nq behind dt-pression ( l i k tvcnts potential wt*aknesws in unambigunu4 y irlc.ritiiying cari.;tl+ (a matter of
rvhlch acted 3.; p m ~ o k i n g aplnlc such it< famtlv problems; poor internal \'alirli?) the!. src* qtrilng at descr~ptiun.
) ~ t > ~ t s l nepr n; p l ~ i ~ mtc strcswrq;
n .;cvcrih; of qtrr*ssom; access to quppnrt
t ~ ~ t w o r k cThcv;) tnfrrrrna tion about I he\(. tac tors becarl5r t liry
tlio~#ghtrt would Fwlp make qcnsr, a t a sublectivc* level, of why some
piwplr were drprcqccd and why othrrs uterc not. I3,v planning ahc.ad and Practical issues
,~r?ticipatingthr pr>s.;ihle rcawnc that might lic brhind depreqsinn thrv
trvrc able to dt?\clup a n accn~rnlof drpre3sion t l i ~ tried t SO h u ~ l din the Many c ~ thtl
f practical i.;\u~-\ Ji.;cusscd in thi.; srctinn a r r no1 prculiar trr
t,upt>rit.nces uf !ti[, .ictcws. It is not n f~3olproofw a v c ~ ftapping thr .;t~b- cro5.;-sc~-lirjnal dcr;igns I w t a r e impnrt.~nt is5u.c~ for ; 1 2 ! research
~rcti\.c.l!+ m c a n i n ~ f ual y x c t of brhaiuiour but I! dr*n~un~tr;ltes a n attrmpt rcgard lccq of the d e l ~ n .
111 n cross-s~ctron~il surLbevdcqign tcl Include hi:: form of explanation.
I r i summarv, tlliarc i s nnthil~):vnlit-rcnt in cross-st-ctitlnal designs that
prc=cludes t r y ~ n gto a c c ~ s sthe r n r ~ n ~ i i f idirnenh~or~s ~f~~l of human brliav-
lour and b u ~ l d ~ nthesr g into t ~ r p l a n a l ~ o nIt
s . rnaktx:, qmsc to emplov 3 Thcrc ic nc~tliingabout tlit* It~grcof cm~+srartic~nal desicn.: that r q u i r e s J
r,irigc i l f stratqqcv5 .;trice no stnclt. approach i s w ~ ~ h c n~i tt qsliortcotnrnq~. partir~~F,lr method of coll~.r-!in): the d , ~ t a I'hc c.;scnt~al thin,, in crcvsk-'

bcctltrnal <lra,irni, i<t o rvlli,i~n ,I \truc:ur~*<j +(*t 41t ~ l a t , it I r , ~ l tztiab!e 5v\-

tematic ct~nlpirison5bctrz4t-~,n cases, nr )I;rt~iipkut cnr;c.;.
Tlic m~n,~lv.;i+ iorlar data rctltllrra.; data fmm m i~ltiplccssrs
of cri>s.~-.;t*t?t
us in^ I lit1 s.lIne variahli..; Anrhvsis rcl1t.s on cornp;lrlrlc c a w s and
l y :varlntjnli hr.twri~r1cases o n onc variablv in trrrns of vari-
a c c o i ~ n t ~ ~tor
Rr0presentatir.c 5arnples a r e nccr.;.iarY i f \we are trl ~ c n c r a l i z efrom I+~*'('sLI~~s ation on anrlthcr variat-It.. For r ~ a m p l t\ri, ~ , t v ~ l loh~ervt,th.~t there is
oI*tained m a samplc to the \\-tdc*r population that the sarnpEc ik mmnt vanation In terms of inct>mc: wrne penplr r a m morc tI~nnothers. Wc
tu rppresent. Tlir surc.st way o f achithv~ngrcprc.;cnta tivc qarnplt-s 1.1 to wilI tlicn srr i f these variaticriis correspr>nd with \.ariation< i t 1 qornethin~
crnplov probabilit!! hampling metl-rods. Thew arc m ~ t h o d sw)icrt~rach else: for ru,?inplc,d o prnplv w i t h highrr inr-r>mealso tend to h ; l l ~higher
p1r5on in t I ~ epclpi~lntiun to which wc want to ~ ~ n r r a l ihas r e an cquaF o r educatrr111Ihan those wrlh low incnrne? Dtl tljc high incnmt- 1.a rncrs tend
kt~rlr++n chancrc o f bring selrctcd in thc sample. tn be malc ~ v h i l cthe Ioru incon~taearncr.; tr~nrlto be fVmEllr7
Cross-sectional r'lrbiygs can liarP more success th.ln okher drrqiqns in Tl-11.; .;or1 nk cntwlation.il nnnlt~qisspy~tirt- what is wmcffmcs called a
a c t ~ i r r ~ i ntepwwn
g ta tlveness. ~+ari;lblc tw c.ire rnatrlr - a ~ r l din ~\.hichwtl have the sanlt, ~nftvnlation
!:xperiments encounter problcrn.; with repr~.;c~nltati\~eness for two tor each caL;t>.In the examplc in Figier~3 1.2, cnse 1 13 a 44 year old,
main reasons. 'l'hcy often aqk rntrrc of people than do one-off cross- middlc class male with ari annual invomr cri $15,0011. I lc. votcxs Conscr-
srrtional studicr. 'I hcv also invnlvc. active inkrvcn tions and tlietcfnrc vativc and ~JRrccswith attil.~tdcA but h a s n o opiriion on nttit~rdcB.
h.lvc trj rely on r,ctlunt~ersm d av;lilnbility samplcq. l'hev conqcqucntly The d a t a 111 a varial7ltl bv cnse matrix can he collvctcJ i~.;ing anv
lack rcpresentativtnt=,\ Even ivl~c~rc rcpresentat~vrqnrnples arc rJit.~~ned rnethnd: t~ura+t~onnaircs, f~cc-to-face i n t t ~ r v ~ r w (>175~'r~,lf1o11<,
s, L~~c'hiv~l
i n ~ t ~ a l lthis
i c;fn hv lost a< F C ' O ~ I P p vvtxr t h t cmrrw ot the
d r ~ ~cjut rccnrdc, itblc*rt~onpintr3rt Icrl.; am4 <%ti r h c r c 15 no ri%.1+nn\.r.hv a
tn*ytnr~mrnt. rniut~zrrrl! ~ n t ~ t l i o dcanno(
s Srt=uqed tcr cotiipli*tr the m n t r i ~ 11 15, ncvcr-
.I\ similar yrrlhlrm nf d r n p r ~ t ~c,in t cornprc~mi*r,panel shldics. T h r thclcs.;, import,~ntthal tht, s,lrncXmix c l C n i t l t h ( ~be ~ / ~11ki.rl lor mcfi c a w .
morra wavcs a patlt,l s t ~ ~11as ~ l t11.t.
y \:t.r,aitcr the c ~ r r n t ~ l n tdropout.
i\#~ 'T'rr thc~ FRTCYJIIIIIIC~ WC' sh~tl!d c ~ v ~ci>II~rl!ti~
i~l in tclrnlnlion on inc+r>mlrfroln
t*\lrnt that thrc drop17~1 t lnirnd ilcc~s~tiipnrtant sarnlllr binses, reprcqrlnta- some pt,npl~% sing ~n < ~ t ~ o n v r n osrlt-~=ctmplctinn
us qtlr<ticlnn,iiru whilt.
tivt-nt.ss can br prr~gr~r>tvcIr. ~lndcrrn~nrd. r d y r n ~o n ,I f.7ce-tcrfac~rntclrr!crz. for r>thr.r p~*c>ple, and rPlv!nr:on p r -
q r t ~ ~ .c ~ r ~ a l t l r l ntlt havc R timty dtrnrmqjon the!' a v c ~ d
t ,r c l q ~ - s w t ~ ~Jcsign.; sonncl rrrcltd. for nthcr%,nr t,rxation rcturn.: Ftlr a hrthc.r si%tr j t pcoplc
tlrr, problem nf I,ia.; bt~ung~ n t r n d t ~ c rbv c l 5arnpltbatlrjtion. So long a4 t h y Sincc thr rnt-thnd b\* whit-11 data a r c c~dlt~ctt,ci can infl~rt-tlzcthe rva\,
I
in! ll.1 E .;ample is w d l rclectcd thc. cross-sectlr~natsurvcv qhnr~lrl yirFd peoylt, .In<wt3rcluestifrn5 M'tl a! Erast rnwt ran<urccnnqi.;tclnc-v.
 7 3 i i ~s h o ~ c rL I ~anv ynrt1cul~1rrl~t,icnllr*ctio~rmtathtld tt.111 ci~,pc.nd on
lnnnv f ~ c t r ~ r'1s hcsi,. ~ n c l ~ SIN~ d c( ~ It hc %amplca,thc cautcnt to whtch they
art8 i l u ~ t t ~ r~t i i tll ~ i lclntn plncv nr , ~ t ewidrlv .;c;ltEt~ril, bh(. timc ~ r t d
rcs041rcr5 nvailablc, the Jcgrtbr of .;cnsitivitv of tlir rnal(arint lwinx col-

Case
I
I I Mate

I
Mldqle S4SK , 44 , Agree No

I openion I
Consewalrve

I
Iestt.ii, ant1 thc c ~ m ~ p l e x 01

i h r ~mcist widcllv ustd mt~lhotl.Stru~.t~irrd

~ t vthe m,~tcrinl.
While ,I varirtv of methods can h.used to ct~llectitlformaticm to form a
~ t n i ~ t z l r cdatn
~ r l wt i t rern,~insthe r.l.;e that s t n l c t u r ~ dq~it~sticrnna~rt~\
rlutn.;tionnnirc.; can be a ~ l m i n -
istcrrd in n number crf dltfercnt rvnvG, thrm must ccmmon of wl>ith;irc3:(I 1
art.

61- I 4-
Female Workrng1 - 52% . -
56
I
-c
D~saqrne D~sagr~r
- A -
Uon- 1 facr-tn-facc hv kai ncd ~ntcrvlewerr, (2) over the ti*l(&phr>ne
inttbr\.ier\.~~r~, (7).;tllf-adn1ini5trrcclr by tht. rr<pun~lcmtivllerta the q u e -
with tr,iincd

I '
I I I I
Conservnrtve
I ti(v111air1,i\ nr~rmallvrt~ci.~\le~I
Fach rncthud 13f ,>drnini5terrng qucqt~r~nnairpz
'qnd returned IIirt~~r):hthr>mail.
l i , ~ *i t \ a d v w a t ~ * anii
<
dt'ii*;lct~rsan(+ thi, mt.tliod\ havtb Ihcir <trcncths a n d wcakr~cssc.:
(Dillman, 79%; Ro~crs,107h; Crorta., l't70; C;rovv.; a n ~ fKniin, 7'4:ct). In
s c l r ~ . i i na~p a ~ - t ~ c u mclhod
l,~r of nrltrlinistmticw vrltl n c c ~ lto hv aware of
th~-+i.rt3lklti~ia ~trcn):thsanrt wt-akntkssrq and huw t h ~ vrntght applv to thc
pasticul.il- slir\.r,v in which vrw arc1 t ~ n g + ~ ~'rl~c.;t% c c I . strt-ncthr and weak-
rtLw-p< c,ln btb p-~>upcrlinto f ~ v cb t o ~ dc..~teg~rriec. T a b l ~i I. 1 prtn.irltr5 ,I
! i (I>INV
W I - , I ~ A V \ * tl\ a ~ l ~ ~ a of v ~ P J C ~(if flit, tlirc,c*r n p ~ l i t I~I l~~~d r i ~ t n i k t r ~ ~ L i ( ~ i ~
r;1tta4cm ear11 nt thc critlaria. \t fttllt>r~frccuwcini s ; l ~ ~ ~ l a binl t 13illmnn
-
(ItVS) , I T > Liltz ~ V c ~ (~l ~~~\~ J 1 1 ) .

Tlitl sample <we dcpenil< 017 fund.;. timi*,acitb.;s ttl pottant ~ a parhclyants,
l
pl.innt~cl method.; i l t aniilyii\, and Iht- clugrtv' nt p r e c i . ; ~ ~arid
~ n acr.urncy
rrquirt*rl. 111 ~ c n p r a l ,rllc Earger thc qamylc the bPttt*r, Iluf br\.rmd 3
ctarttlin point ~ n c r c , ~ s i nthc
c s;lrnplr silt, ha.; .;mnllcr and rnrjrc marginal
bt3nrfit~.

The Inr):rr thr, sanlplr v r r thc anore it can I-(.i;uhdEviilcd so thn! mcnn-
i n ~ f u lwbgrnuy cnrnparistir~scan br mad(,. If, 111 a gcner,~lsrirvev OF
h r ? ~ ~ w h ( ~ we
l d s ,uPantt>rlto comparta thv wellbeing nf ~hlldscwin couplc
h t ~ i ~ s ~ h n~l d1 s1 t hthat (1C chilrlrcn in 5olc p-trcnt Iiuuw*hold~,I\.rV wnuld
1lt.t.d t c ~cniurc Ii1;lt thr urnplr 1%tn~lrll ~ c lai-!:c cntluqh !(t \,~~alrl \ ~ l Cistont
f
nun~bc>l.+ IT( suit, p n w t hriu.;e't~t)ltls. I f wi, wantrd 1 0 ronlparta Iatht~r
ht-,~drdm i l rnut1w.r h ~ . , ~ d c5171r. ~ ! prt'nt ht\r~\rh.rllkl<I\.+: \vtrt~ldt ~ l r dtn
<c.crurt a vcrv largt3 tniti,il \amplc. simplv t ~ ttnd t c n t n t ~ hI,~tlirr1 i ~ a ~ l r d
5ole parpnf hc*u<chold>.
Adopting thy .;tratcgv of ~ncri,,isin):the ~rvernllsample s i x tn ol7tairl
.;~rfirc~cnt hard to t ~ n dKrrluyr; c,in b r ineif~cirr~t1%-c <-an t - i d up \v"h
more of thc ' t _ v p ~ ~rr~spoilrlent*
l' t h a ~ iwe a c h t a l ! ~I I ~ V ~,An
. alltarnalivc
appronch i~ t(t cwrrsatnpl~.flrc hard t r l h n ~ lpoupq I t 1 en<urp \t~tflc~c'r~t
 Sat!sl;uc.ton
['or-r
53tu4.l!..hTv
I.rxd
Coatq i l
t rrl
C,lr,d

that \rimt-rarr,I .;,irnplv \\'L* ill r v , ~ i i ttr\ Lntw h ~ n v~ l o ~r tl ~* ertw! F ( ~ N I -

I,itlnn I ~ q u r t1~5 I~lichly !cr btb h>f h ~ hs,?n?plt. c*cliin,ltt.. L l \ i n ~pr02~ahilitv
GIrod tl~i,orc t' 2~11t3\timntc* 1v1tl1r7 h i ~ hI ~ a v t a l o f collfit4vnc~th,it the* real
C.1~311 pup~dlaticmI ~ ~ u~ rt ' lcl lIIC' ~ ~ ~ ~ t$a$l i i t1.0
n , y1.r c c t ~ nf
l l l x n rample e.;tirn,ite
['vnr (i.e. In this cldrnplc., sr~mtbrr,hcrc\\'!thin tht. rarlgr 01 55 p a r cvnt to h?. pcr
Giw.>tl
cr.nt).
I;{~x!
U t ~ v0.l If-tv kc\ factors tliat affect< h o ~ vyrrcistal~.\i-ca c a n specify thv
pnpulatfrtn t i g ~ ~ r c1 %<t l ~ c.;ifr nf the <,lrnplc. I'hc 1ar):c.r Ihc .;ample t h ~ a
Iiari.ox\rr>r thc 13.ind (callrd thc cunhdrncr inter\ all w ~ t l l ~~n{ h i c nbc
h can
t><tilnt~it* tht. y r ~ y ~ r l ~ ~ilgtlr~'
t i n n wtlE f~c.In tlic al-rol-t~i ~ ~ a i n pri-c l i ~rn~ght
tiavc- had n qaniple nf a0011t ?4-IC)(J pt.01~11~ Wi tli 13 ~ ~ i mt lp~ l~4~ l 7i e w e cnn
say tIi,?t thr pnpulntirln tvilF ltc \vi thin n rclativrly n;lrro14, band of +2,0
pcr cent arrwnll thr* sampfe e>tfmato. If wc wanted to narrtrw ~ h band c
IVP wo~lldhil\.e to ~ R C T C ~ I S V thv s a m p ! ~sizt,. 1 ' 1 1 ~~ c n r r a tn ~ l vi s that t o
nt1mht.r.;. Thiq can a f f ~ c tthe reprc.;entnttvl>ner;s of the qarnple, but this I i 4 t l ~ ~ cth~,
1 conf~rlriiceinterval we nctbd to quadruple t l ~ t ,.;arnplc sizc.
cnn bc adlu.;trlc!hlr bv r ~ w e i g h t i n gthc sample to take aucounf of un~ler- Tliub to reilrrc~.Ihta sarnplc cmcv froni +2 n ptlr cmt tn ? I 0 ptnrccmt nrr
and m crwrnpltnl: (Ch,lptcr '1). M J O I I T ~ ~ 11~t?dlo irlcrr,tce the s , ~ r n p qI ~ z ct r ~"2011 caqrlt. n b ~ ~ l o l ~ stllcrC l r ' 1..
,I ptlirit at ~ ' h i ~tlw - h c?tra Ic\cl o f prtt-~\icrni\ tlot \vi)rth thr' tb\trn cnqt.

I t htbrt. th.it prrtnt 1s will rl~~pcnd on the h~n~l.: * l n ~tlie

32'.111;11~1t~ i dcxr~,eof
yrcii-:inn nbqtrivecl
Il'hvn w.ins c:t,lmylr+thrrta is nit%,it G 111r. IiLrliIii~~d rjf c a r n p l i n ~error: thtl -T,~bltl1 1.2 pran id<#\yst~rnatc~s nt' tlir .;,~n,plt. +I/L~\ \.nu t z n ~ ~ frt~tluii-c
d
hamplc d{*t.\ ntlt L i r ~ + ~ ~rcllt~zt
r . ~ t ttlic
~ l pilp~~lah'nrl
~ Irom rvlilch zt i q dra\vn. tlt.pc.niJ~n z {>TI hl lrv m ~ ~ c .snrnpltn~li tt.rror you 1vottl~4Iw prt)parc.rl to
I1'ht.n r\,tU~ i ~ r<~ndo~n
r . s,~niplr,<~ v rca n c\tirnatc, rvithtn a sptacif~r~d Ic.\,eF ~ o l c r ~ i7~hl. c . tnblc, ~ n d i c ~ ~whatt c s thtb conf~dcnrcint~lrvatrr~vrltlht- ,lt
(11 prroFlnb~litv,how cTnw t l ~ t *(real but i1nL17nzr.n)fi~;urch in t l i ~~rot?lrlnirrvi hat i s rutrrrccl trr nq tlit. 4$ ptarcrnt ctinf~dtanctI~.\.cl.TI7u.i. tvta can Elr (15
artmliLt.li. tr, br- to thc fip~rt>q{vt. gfntf r ~ j r nthr~~ a t n p 17~h. t ~M.P
+ ni~):htf ~ n d FYI. ctnt conf~dtantthat if I ~ P had ,I r . ~ n d t ~ n\ample
i of 100 c n w s the
th;lt hll prr ckcritof n sarnplc <a\- tircv 5i1ppost cnpital pudshrnen( <<itrcn p(>pulationy\ljnlatt, W<MIId I-"~*'""-:*' "'
1YO CKC7SS-SFCTIONA L DESIGNS
find in the sarnptc. IC 50 per cent of the sample supported a particular
polit~cianthen wc cur~ldbc a l m u ~certdin
t that the real level of ?upport in
the population would bc somcwhcrc bctwccn 40 and 60 per cent. If the
sarnpIe was increased to 400 then the hand of seasonable certainty would
bc 45-55 per cent (50 r 5 per cent).'
Since the analysis of cruss-sectional data requires that we anaEysc
variation in one variabEe in terms of variation in annther variable, our
measurements must bc sufficiently scnsit~vcto tap the vaiiatioi~that
exists. For example, i f we were trying to explain variations in income and
sirnpIy collcctcd incomc datn using just two broad categories, SO to
$7 n0,UOO and $1 00,000+, wc. would mask most of d-te variation that exists
and have litllc variation in 'explain'. By including mast pcople within the
one category (under $100,000) we would be treating them a s though they
are the same. Conscqucntly, we would bc unable to explain the variation
nlifhi~;this category. Similarly, if we warrted to measure marital satis-
faction and this was done in such a way that everyone dppeared highly
sahsfrcd we w t d d bc ~rnablcla 'explain' variations In marital sa tic;-
faction. Ef our it~rtrun~ent is too blunt to pick u p variations then we
cannot explain variation since we have not measured it i l l the first place.
The same applies with the key exp1,anator~variables: we must ensure
variation. For cxarnple, if wc thought that variations in incomc might
he due tn gender it is extremely tlnlikely that, in a sample consisting
mainly of males, much of the variation in income will be due to gender
differences. I f we thought thal incomc diffvrtmces are partly due to
age differences but then tested this using a sampIc that was fairlv age
homogeneous, wc would find that age was not very important in
explaining income variations in thiq samplc.
for statistical cu~rfmls
I~~futurr~aiion
Because of the nat~rrcnf crnss-scctio~~al designs we must br ahlt to
introduce statistical controls into our analpsis (Chapter 7 2). This is
essential if we arc tu get any clarity regarding causal prcresses using
cross-scciinnal data. I f we are to intrnducc statistical controls a t the
s i s these must be anticipated brfi~rrcollecting d a t a sincr we
a ~ ~ a l ~stage
cat1 nnly conhilf fur factors about wh~cliwe liave information. This can
only br achieved by first reading the rclcvnnt li tcrnture and i.hinking
thsclu~htthc thcorctical mc>dpls h r are plarnning to evaluate ~ n antici-
d of pit.ct~of ~nformatiniito identify the type of relationship I thought
pa ting what variables will requlrt. coi~.trollingfor statistically.
Whcn data arc collected directly from respondents wedneed to be mind-
ful of the burden placed nn respondents. This hurdcn can take many
forms including iucr~nvc.iiirnce,invasion nf privacy and time. How long
should inlcrvicws las! and how long can a qi~estinnnairebe?
rt is impossible to he definitive. The optimal lengtli of an interview or
qucstionnairc will vary widely among different types of people. Some
people \till be busier and unable to spend time. Others will encounter
language difficulties and find the interviewlquestionnairc more
demanding. It will depend on how easy the questionnaire is tn follow
and how skillcd thc interviewer is. Even more important is how interest-
ing people tind the topic. Undoubtedly, the length of a market rcsearch
questionnaire about something of little interest to people will need to be
shorter than one dealing with rnattcrs of direct concern and about which
they have first-hand experience.
Types of dntn
There is natl~ingabout thc logic of cross-sectional designs thal rcquires
the use of quantitative or qi~alitativcdnia. Thc kcv thing i s that data are
systematic - that we have information on thp same matters for each case.
Whether these data are in thc form of quantitative, numerical or coded
data, or uhrther they are quote< frnm intcrvicws, cxtlacts froin diaries or
c>bbscrvations,is not irnpurtant FLIT the logic of t h e design.
Analysis is simpler (but not ncccswrily better) if data are collected
using predefined variables and categors~s.This is wliy q ~ e s t i o n n a i rRTC'
~'~
so widely used in cross-sectional designs. But even in structured ques-
tionnaires we must make choices between open ended and forced choice
questions (see Fodd y , 1993: 126-52 for a ful l ~iiscussionof the purpnrtcd
advantages and disadvantages of vpen and closed questions).
Structured questionnaires are one of the simplest ways of obtaining a
structured data set. Howcver, we might use lcss structured data collec-
tion techniques and frnm this construct a structured datn sct. Using in-
depth interviews, for example, we migl~ t identify from the interviews a
set of variables of interest to us. or each of thcsc variables we would then
construct a set of categories and then classify each case. For exampfc, I
conducted a set of in-depth interviews with adults that examined the
nature oi relatianships with their wider family. As part of the interview
we talked about relationsliips with parents. After completing the
interviews I constructed a set of variables, a number of which were about
adult-parent relationships. Prom my interviews I had idmtified what r
thought were a number of different type5 of rclationsl7ips. I u s ~ da range
applied in a particular case. 1 (hen dassificd each case r)n the basis of a
whole set of clues scattered thrnugh Lhc intcrvicw. I rcpeated the same
process for many other aspects of the interview ( e . ~ relationship with
siblings, nature of marriage, degree of involvement in local community,
ct.iih;?liiy of work in life). In other words, 1 war: able m rlsc a range of
q~~alitativc information t~.?produce variables and to classify individual
 I ISSUES IN CI<ClSS-SECTIONA I. UFSIGN
192 CROSSSEUTIC7NAL DESIGNS
I
1
I
cases. I was then able to systematically analyse pattcrns in the data (de
Vaus, 1994).
Ethical issues
As with a n y rcscarch design the researcher using a cross-sectional de5ign
will need to attend to matters of confidentiality, privacy, avoidance of
harm In participants, and infnrmed consent.
Tn some respects cross-sectional designs can minimize some of the
ethical problem5 that can arise with experimcnta1 and longitudinal
designs. Since no tracking of participants is required it is much easier to
ensure anonymity. Thcrc is no nwd to kccp l~stsr ~ names f or to match
names a t all to responses. Furthermore, since the data collection is a 'one-
off', cross-sectional studies ran be less intrusive and, in this respect, can
be less of an imposition on a person's privacy.
In some survfys cenfidcntiality issues can arise even when information
is provided anonyrnrrusIy. In a survey of an organization or a specific
region or in a national census it is conceivable that individuals could be
iden tifird by examining a scl of charactt.ristics. For i n d i l iduals with
rcIatively unusual characteristics tt may be possible to idcn tlfv individual
people. For example, In a national census data will be collected for each
person that lists their geographic locatio~~ (down to relahvel y small
geographical arcss called collector's districts), their ngc, gender, occu-
pation, income, education, household characteristics, marital status and
so forth. With detailed information of this type it would be possible for
someone with access to tht. data to identify some individuals such as the
doctor in a particular small town, the school teacher, the clergyman ctc.
Sincc some survey data and census data are made publicly available
to researchers for Further analysis it is important that no individual is
identifiabre by means of cross-classifying detailed information in indi-
vidual cases.
, To guard against this problem, data from census callcctions and
surveys wilP be 'anmymized ' before they arc released to other pcc,ple for
further analysis. This is achieved in various ways. One way i5 to omit
certain key inlormatinn, such as any geographic identifiers, from any
public release 06 the data. Another way is to reduce the precisic~nof the
data by collapsing categories. For example, instead of providing data
u ~ t specific
l~ ages o r birth datcs, only age g o u p s (e.g. in five-year age
gruups) will bc reyeased. Another stratcgv is to omit any linking infor-
ma tion between cases. For example where informatit,n is collected from
each member nF a household the publicly released data may makc i t
impossible tn identifv which indiviclual cases belong t c ~the same house-
hold. By controlling the deta~land amount of puhlicly released infoma-
tion in these ways it is possible to prevent the iclentificatjon d particular
inrlividuats from an anonymnus data set.
193
Bccause cross-sectional designs rely on existing variahons rather than
intrr>ducing il~terventionsthey avoid manv of the potential ethica! con-
cerns about harm to participants occurring as a result of the inten~cntinn.
I t is also an advantage that they avoid randcrm allocation tn experimental
and control groups. Randurn allocation can give rise to serious ethical
problems (see Chaptcr 6).
Summary
Sincc cross-sectional designs lack a time dimension and randomized
control groups thev encounter potential shortcomings for causal ana-
lysis. 'These shortcomings represent threats to the internal validity of
cross-sectional studies. This chapter has outlined thtse shnrtcomings and
identified some ways of minimizing their impact. While cross-sectional
designs present some peculiar weaknesses in relation to internal validity
it has been pointed out that they also have some particular strengths.
Because they do not have a time dimension, they do not suffer from
many of the other threats to internal validity t h a t we have encountercd
with cxpcrimental and Ioiigih~dinaldesigns. Cross-sectional designs are
a simple and cost effective design and consequently are often able to
employ exccllcnl s n m p l ~ sand avoid thc problems caused by sample
attrition. As such cross-sectionnI designs can have strong external
validity.
The absence of randomized control groups means that the analysis of
data from cross-sectional designs relies on considerable sample variance
and on the creation of post hoc comparison groups. This has implications
for sample sizc, sample type m ~ dthe practical strategies of implementing
cross-sectional studies. The chapter has outlined these practical and
cthical considerations.
Notes
1 n i s needs to be q~ialificdto the extent that snppmsor variables may mask a
causal relationship. Strictlv speaking we would n w d to be able to eliminate the
influence nf all suppressor variables before we could bc ccrtaln that the absence
of a correlation b ~ l w c c nhvo vnriablt-j really rrflccts the absence RE a C ~ U S ~ I
relatiunship. For a fuller cfiscussion of suppressor var~ahlessee Rosvnbcrg (1 968:
Chaptcr 4).
2 These estimatrs assume thdt we are dealing with variabfes where there 1% a
W/30splil. 'Thc cstimatt'd sample sizes will be different In more hnmogcncous
pnpula t~ons.
 levels in particular locationr: and , i t a pnrticular \tag{, nl Ihr Fanj~lvlifil
cycle.
Since thy.;(. sirrvets are nnt primarily designcd ti) rlc\,rlnp cxyl;lnn-
tinns c,f bcha\,loi~r,problcrnc (11 causation, of ~a~1*;12 o r d ~ or
r the inflw-
ence of unciintrr>llcd \-ariabl- do not matter. TIlc ki~vthing is to rn.;rlvca
CROSS-SECTIONAL ANALYSIS that sarnplr%,irr Inrgc enorrgl~5cr that rve can p a t sufficient ntrmbcr.. En
the varicius si~hi:roups to prrrvidtb rt-liable descriptions. The 0tht.r kcv
thing is ti) ohtaiii qarnplcs In .;llcli a way that wr3i a n l?;t~ncraI~ze bryclnrl
the samplr ttr thc population frtrrn which thc. sa mplr w a s drawn.
?hc an,ilvsis i ~ w then
c bvcnrnrs a matter of t171* npprnpriate wnv': tct
~lc.scriPethr data It 1;. beyilnJ th~b+ctrpr of this I l i ~ > tt,
k go through nll thi*
Crtr5h-.;tu.trrrnaI Jeqigns arc r ~ d.suited
l t r b 14t~~cr1ptive anaivis - at Itust a be i i r ~ r r i l ~ c r l .
warTsIn tv111t h d ~ i can
for dcscriptionq uF thing< ; t ~ thev are at n glvcn point of time. Flowrvcr,
crii~q-st-ct I ~ a v cbrrn cri t i c i l ~ t t tor their weakncss n t the
~ n n a ld~s~g115
level nf t*rplanatory, causal analysis. Somc rt*scnrc-lier? argue that, for
scvcral rr*n.;ons, cross-sectional designs are ~ ~ t ~ r t l i lat c sthe
s c.xplnn;rtory C n u n t i n ~i.; 3 Ilnsir aspect of dcsrrjptive analyqi.;. Tlic aim i s to eqtnblisli
q t n of~ ~'1 ~ t u d v .To support t h ~ i rl u d ~ e m e n tIhry cite thc diflicirlties liow m a n y pcople have pnrticul,lr cliaractrri~ticr, hnve n particular
c m ~ s - w cI (tma1 data presrv~ilor t ~ t ; t a b l i s l ~ ithe*
n ~ c a u ~ a 1order ot c t . m ts, opinion, f r c l ~ s ~inc a g i ~ e ni \ r a v ;lnJ $0 forth. h,l;rrk(,t x\lrvPyG wilt find
for cun troll l n for ~ all thta variablrs that miell t 1-c inflz~cncin~ corrcln ticwq, out hnn, man!, pcnple like a particrlldr product, a pr)litical poll rvifl c011~lt
nncl tcrr r%\tnbli<hing,~ z n , i r n % ~ r ~ u ~cntv.,lli u ~ l y , clrder. .-\-\ltIiou~h t l ~ c t ~ o w111.111\. pi,opIr ,"tend to 7 r ) t ~tor n spccifir p;rrtv clr can~ttd;ltca, ,I
j ~ ~ d g ~ mIt~vst~ eon c l the- I-r~tic~smsis too hnrqli I1it.v arc ~-alnab!t* hc~ca~~~c census w ~ l lctrunt the nt1mhr3r of pcople in tl~cn l,c*ptll;ltiun, tvhilst .I
they C ~ ~ J M ~ l*l c tion
n tc1 inl~tlrcn
t weakncssrs uf tV~i% cross-sectional dcsign hcluse!mld i*xp~.nd i ture SiIrvLay ~ v il l dcscriht. h o ~ vmany pcople 1iai.r-
that murt 271' addressccl. parhcular patflmrnsof e x p e n d i t ~ ~ r v
As a result wc rnunt how many pcoplv gi\re p a r i ~ c ~ ~answer5 lar brcausc
we are intcri>stcd in the t i r . ; l r r l l l r l r r l r i nf cases. Mrhat is tlic h.pical rt.spon<t.
Descriptive analysis

Cross-sect innal design< arc iJcnll y suited tn ~icbscriptiveanalysis and arc.

I H c' '~ wppicat i < ~ l r rttvpical r ~ s p r v ~ ~Art-
to the q u ( ~ ~ t ~ { m
over a Inrgc, number of c;rtccnrirs' Are they ciu+trrc*d in lust a f t w
categnrics' I-low ~ i r n i l aor r drssvrni lnr are people in rlw sample? Art. tlwv
' people qprt.nd

widt.1 y ~ i s r dfor these ptlryoscq. Thcir wideaprtud use in markct rcqmrcli clustered lowards one end ot n ~ i ~ s t r ~ b u f( op n. ~ low . inrnnle a d ) .
and pol~ticsl polling b ~ a r this out. 'l'hcsc .irtl contexts in which In othcl. wt)rrls in addition tu nimplc counting wtTwill look at thc sirnpr,
rt.scnrchrr< want simply to dcscribe such tI11ng5 as: who u s w o r likcs of any distribution and try to summari7e in rnmningful wavs what tllr
particular products; aspcct'i af consumer brlinvinur; voting intcntinn; distributinn Irx+kslike. Thew ~ n ; l l smay be acliirvrd with varini~s
level of rtittlrt.>t in politics; trt-lings toward< plrtictrlar politicians; and m a y st~tisticq.'To stlmmarire wlir?f i~ ~ I I J J ~ C RIn! a ~iistr!b~tiiun w e liunuld
attih11k.s tcwards a rnngc of iqsues. Typicallv, this FTe elf rnqcarch use a n appropriate measurc. of ctmtral tendency (mi*nn,rned~ano r rntrc"l1).
fc~itsc.;on question? of 'Irr>w m,lny' and 'which tvprs" of pctlpZc do or 'To sum marirc thc spread and i r n t t ~ ~ l t i l iin l ! ~a d ibtrili~~tion we would I I ~
t h ~ n krn pirticular ways. an npprt~priatcmcasurc of d ~ s p ~ r s i o(ne . ranpa, ~ intcrquartile rangc,
'Fhc ciln.;IiG i h anothcr 1-xnnlplc of a desvriptivtn crclss-sectional tfc~ign. variancc and rlt~ndnrddevtntltrn) Other aspect5 of Ihr shape of n clislrl-
I t s pfinlarv f~mction is it1 prt~vidc <In accur.llc description c,f fhc
bution call IV .;llinmarizetl cvitli qlatiqtic~sucli a.: skrwncsr; and kurtrwi.;.
pnp~l!ntic~n of a cotlntr~f: hurr mJny pcnplc lhttn~arc> in tllr p~rpulatiun, G r a p h ~ c drrcpr~sentations rli.;tribi~tions can offtlr~c o i i v e ~mrlrc- ~ and
d n ~hmvf man\ people p o c \ c - ~ parhcular chLlmct~ * r ~ c f i(e.9.
t > svnrfrr, arc,
be mmo i~nliritl~clt. u n d ~ r s h n ~ i , ~than l ~ l cf;iimn?ar\, .tatr.;ttcal mc;lwrL*>.
e d ~ r c ~ l t ~ rIrl.ing
m, in p , l r t ~ ~ = u Itration.;
t.~r chtc 1, lloucehold c x p c ~ ~ ~ l i t u r c
Tic c h ~ r t k ,Ivr chart5, 11nc gr;lyIi\, h ~ ~ t n q a r n lieu s , and tvhiqkcr ylllt\,
survcv* provide a filrthrr t*uarnptcof c m r ~ - w c t i c ~ nclata a ! bcinz II~C'LJtnr stern and I c ~ fplot.. are ius! somc nf the graphtcal mr;rn<c>f reprecrntinfi
dcwript i v r purposes. In thrsr. \11~vcys ~ o ~ ~ ~ r n snei~m ktot mnp
~ thp
distributinnh. Mo.;t cvrnputer b ~ s c d<t*~tisticnl p;ick;ly;.rs and sprcndsht~c~tc
cxpcnd iti~rv patterns of indrviduals snit I ~ u u ~ r I ~ r > lSuch d s . .;urvrys producc n rangc3of such praplis. AltcanlativeIy, wv may choose to pn3hrntit
ennbli~.(int>-graineddc.;crip~ir~nsof the expcnd i t ~ i r cpatterns of cl i fft-rent
the data in tabular form. Frcqiiimcy distribuhon. arta lllc normti! upavof
tvp~':of Iamilicb and Iwia~c~h~drls and for pivvjlll~1231th different it~comc
doing this.
 CROSS-SECTICINAI, IIESTGNS CROSSSECTIONAL AN AI.YSIS

i i rm>dincy~iro<kril,y
Table 12.1 ,411~ l l r t ~ t r f l t mof rr rrlntto~tsl~iy
Recoded version
M't~endescribing distrtbutions we have to decide how much detail to Male Fcmnle
provide. Continuous variablcs such as age (in years), incnrnc (in dollars)
and hours worked each week can be analysed as cnntinuous variables, or Shonglv agree 5W" 60% M1%
Agree I0 45
collapsed into groups. Altcrnrltivcly, age might bc collapsed into cate-
g o r i e ~S U C as
~ under 20, 20-29, 30-39, 40-49, 50-59, 60-69, 70-79 and Disagree
Skrmgly disagrw
30
113 35
' 1- Dirnpree 40 40
go+; income might be collapscd into income groups (Iess than $5000,
$50[10-10,000, $10,001 -1 5,000, $15,001-20,000) and analysed in thcst.
grouping5. Manv ordinal variables might be collapsed into fewer cate-
g~jries.For example, respondents in a survey are asked liow strongly they
agree or disavcc with a particular attitude statement (strnngly agree,
agree, can't decide, d~sagreeor strclngly disagree). We would need to
make decisions about whether to collapse down to thc cat~gories'agr~c', Using this approach we focus on the distribution of a variable as a basis
'can't decide' and 'disagree' or, alternatively, if we are only interested in for recoding and let the distribut~onrather than the substantive meaning
agreement wc could simply use the categories of 'agree' and 'other'. of the categories dctcrminc how we combine categories. For example,
Nominal variables, such as country of birth, may havc a largc number of rather than collapsing income in the way described above we might
categ-nriics - many of which can be combined for the purpose of simple simply have two categories: people below the median incornc and those
and cTear presentation. Wc might group countries into regirms (e.g. above the median it~come.If t h median
~ income was 535,000 then the
Western Furope, Cpntral arid Eastern Eumpc, North Arncrica, Other hvo incoinc categories would be helow $35,0011 and abovc $35,000. Or we
America, Sor~thAsia, South East Asia, Oceania) or according to levels of might divide thc sample into deciles: the bottom 10 per cent of income
development, or whether or not thcy arc English-spea king (Chapter 6). earliers, the second bottom 10 per cent ~iincome carriers, through to the
The d e p e of detail we retain will depend on factors such as sample r o f income earners. The specific dividing Iines would be
top 10 p ~ cent
size, the number of people in particular categories, and the way in which determined by looking at the actual distribution. Using this approach,
data are to he analysed and presented, and on the main points that need definitions of low and high income are not imposed arbitrarily by the
to be highlighted. researcher but are defined mlafiz~e[yon the basis of the distribution.
The way In which variables arc collapsed can have important impli- Table 12.2 illustrates liow the same variable might be collapsed differ-
cations for patterns of results a n d care must be taken to not to simply ently depending on whether substantive or distributional methods of
rerode to cithcr crcatc or mask a pattern. For example, Tahle 12.1 recoding were used.
ilIustra tes t h way~ in which coding can mask a relationship.
There am two broad ways in which coliapsing of categories of a Form of datn
variahle will be done.
As well as deciding on how to present and summarize distributions and
how to group values, there may be a need to attend to theform in which
the data are analysed. Sometimes it is desirabfe to change the way in
Using this approach, categories are combined because nf thc similarity of which our measurements are expressed.
the categories. Thus combining the categories 'agree' and 'strongly agree' We may need to convert variables so that thcy are a11 measured on a
bvould be done because thcv botli reflcctcd asreemen t. A1terndtively, uniform base. This can be necessary when analysing data in repeated
depending on the purpose of thr analysis we might corntlint. 'strongly crosssectiona1 surveys where the value of, say, lncomc needs to be
agree' and 'stron~lvdfsagree' as t h y both indicate people who feel adjusted fnr inflation. In cross-na tional studies some variables, such a 4
strongly about t l ~ cissuc. k i t h a variable such as incorn?, substantive income, may need to be converted into a commun currency (say US
recodlng would combinc categories c>ri the basis of the actual income dollars). This was Jiscuss~din Chapter 9.
values. Tl~uqwe rnigI~tcollapse income into categories such as below Data may also be represented by exprwsing individual scores, not in
5113,000, 51 0,000-19,994, S20,000-29,999, $30,0130-39.999, $40,000-49,999, tern~sof the units of the original variable ( e . ycars
~ for age or dollars for
950,000-59,999, $60,000-69,999. 970,nOO-79,994, $8U,Ol)ll-fi9,9'dY, $90,000- income), but in terms of how the score of an individual (or group)
99,949, $100,000 and over. compares with the vpical vaIue for that variable. This might be done by
 do1F.l r+, Il ~ . i ~ t < l - h r n . ~vtln
r k ~ .and fr;inil;. Instca~lrrvt, I\ Iruld Iw tit-al rtiq ~n
1 hv cclmmrm 't+tsrrtrr~c!' nt dc\.iatinn< Frrlt~ltht, miv,ln
A v ~ r i a b l c15 <tL~ndtirdi/l+d \lrnpl.r: 1v. s i l l ~ t r a i t i n gthc typrc,il \.,iIuc
Irnran or mw'rnnl frclnl c;tch pr>rsonrsacrual ccl3rt. on Ihdt x . ~ r ~ ~ b lTilt' tb,
rtb.;~rltinfi vn1ur for cnch ycr+cm I* thr-n d i i . i d t d bv n rntlasurc c ~ \.nrintinii l
(5tanriar~1dcvintlrrn 1 1 n?t5;rn rvas itsccl a n ~ lintcrqliartilr, range - nr
~\.linlc*rcr- if [he rncdi;ln trrn< 1 1 4 )

Tle~(-rihtn c l~~ + t r ~ l l i ~ t ~nocir-trw < ;l ~vli.olc~ .;tirrfr; I+ t\~prc,~fl\ t)nlv thra hr,t
part I ~ Fa Jrst+ripti\.c. n n a l <i+ ~ florr, clftrn th.111 ii(+l\\.I., \tm.int stat. bur\*
r l ~ s ~ r i h t ~ t riit icm frar~ anlong lit C~lrcnt.;ub.;rls i j t tli<bpctrplc rmnl3rj<ing tht*
<t~i<F\..Fttr r \ < ~ r n p l ccf41 , tllr incurnr pattt~rnkt ~ rnvn f ; ~ n drvtmicn ~ l l f t t > r '
By hnw nluch? I n rvhal dircction7 Art- Ihtarc rlifftlrt-nctl.; ,lmclnr: tilt*
yclurig, r n ~ r l c l,1l1:~td ~ n rthr. l rldcrl!7 thtav dibtcr asrordrng to rr>gion7
C~vnp,~sini; ~ ~ ~ l ~ on q ra oy t ~ m t~wvar,,rblc
p ~ linctrtnc, c t ~ ~ q c r ~ ~ l t i 5 n 3 ,
rclic~nu\nps\,l i a r ~ p i n r ~ <i\s f";\r~ntlnll,v l n d ~ \ c r i p l l v ecbuerc15ethat rntail.;
r<t,~l,li~li~ rl-hirli
n ~ tvpcq ptv,pTtl ha! c p,?rtiuul~irrIi;lrLictcr~~hcs, att i -
rc~-c.rprc~siiiq 1*,1ch pcrkrrrl's inctrmc If~lrt . \ , ~ ~ n ~ ~it1 l t "tr7rtii<
l t r l tie\\ m~rcli tuilr... cir I)ch.~~~crtir.: It \trrp+ ~ l l r ) ~rrft ,iftcrtjptin!: t r y c,\~l,rrn r v h ~snnlt9
I o n - r r c>r 111):hc.r i t i c th,In thta avtLrngr \r;llt~c.O n t ~rz-av I n \\.h~rhthlc is >:nrtrp ,ire cliFfr>rcbnt Irvm r~tlirars(,il t l u r ~ ~ gthat h IVCIIIIC~ IJC'tht' o l 1 v i ( ~ 1 \
dtlnt. 14 b~ {,\yrr,\+rnq a pcr\on'+ ind lvidunl w r r r Iir tcrn-rs ot t h ~iumtirr ~ next +try) 12'11arta .;irnlllv ~l~,st-rltlin:: F;Flrrbrn%, \\ Fll~h n~,t\,in tk~rn.![-ad r ~ b
(I( <tilli~IilrCId c \ ~ i i l t i ( ~~t t ~lit'<
s ~ ~ O Vor P~ P I C I W t h t ~IT~CYII a u c o ~ ~ l i(31 t * why p-nul,c ~Clffer.I'Eir, I , Y P T C ~ ~of drascrilrin~I+, lur~v.yt-ttv+, tl~c
lZrc can go one ~ t r pt i 1 rthr-r anif stnrtrft~rrit:t>~ . i l r i , i l d p ~ Standnr~l
. i~atton flr.;t rnd ~ ~ ~ t ~ n5t.l-p c aITrltlr
l ~ l h~1 t!tb\~clup~ng
~ p ~ l ~ l ~ ~ n a t ?Tor o n c~.x a t n p l v ,
11.1.; thv effrct of cxprcssing c~,ichyrrscm's ccnrc rt,lt~liiirto that c3f other.: in brfvrc . ; I T ~ I e ~ \ yIl ~a ~ n~ ~ n t l rdifft*r(.ii~cc
l. i t 1 it~cornr> M ~ L mu<t
~ flrkt
the dl.;trihutinn I t has t l i cftrct ~ t ~ cllrninatlng
t difltkren~-rsrn thc 11n1t.; of ~ ~ t ~ i f ~tliv l i st*xtrllit
li ; ~ t l dnaturr 05 tlic LI i l tpc.r<.nct...
m c a s ~ ~ r c nt ~ acrosk
r ~ n cniln hlc*\, nc-ms+trrnt. anti air(?\.; d l ffert-nt iv.~riLillles. l ? c ~ r - r i p h t r at-rrrs\
n~ * u l > g r o u p cat^ bi*cwrn l n v ~t~krrig l a ~\.idt* ~ a r i e rrf
t~
I t mah'lcs LIK tu cornpart, ;~pylcsw i t h oranges. U y s t n n t l ~ r d i ~ i nincome, g data analvs~\I c c l i i i i ~ l i ~ bu ~ ts~ l i ( n rtnhcarr-her w i l l a l w a v ~rmplu!. F-riv~riatc
tor curnplr, wt. cc>illd then t>xarnlnihchance+ in !lie incornr* po?\lhon c d or n ~ ~ l l t ~ \ * ater !mrthuils
,l o t ontl w r t n r nt-lother. Tlic tt~cliniclirr*s\ \ r i l l varv
rncn and wornrn (3vt.r firm- i n diffCrt*nt c t n i ~ l l r i ~If+ w e c'(lt~ld~ 7 ~ p r t ' s s hrlnr siniplc crt~<+=knbulatlrrnq and s w n l ~ a r i ~ o cltnrnc.rnt; to rncjrc. c n r n ~ l t l \
c,~chman'\ anrl each w c m a t i ' ~i n stniidar~lizcdtr~rrn,t1ri.n for any ycnr or n ~ k itivaria
l t r ~ r l i n i q u c ssuch as niuJt ~ p l crcgrt~ssici;lnlld d ~<crrrnin;~n t
country rr.I3 ctr~llrfcalsulatc thc a\.tbrage : r t ~ n i l a r J.;crrrc t ~ tmrn arlrl of anal\-si<. Thr analr~l.,m i ~ hrtr l v t>n wrnm.lry .;tatisfirs ( r - g. rrrnly;lr1.;rm
wonlcn. We c t ~ ~d~o lthi< ~ l c3vt.r a largca ntirntlcr of year.; and nnt b r of rncvtl.;, corn>l,?tinncntbfficirnk, annlv5i.; of \~nriance),uatl ~ l a p h i c a l
w n r r i t ~ dahout thc nvcr,lll tnfftuf of inflalron. rrprfit>ntatitlns ( c . ~ wattrr~mrns, . co1nplt.x I ~ n cgwph.;, sirtstcrt*d I w r
M'r MFI>LIJII stlt' thlft, 011 nvcrajic, hvrntat? ~ ~ o u l ibc 4 rclci.i% infi incnrnvs r l i n rtql nr rrly tin t.~bular prt.w~itaticm ( t ~ t - t ~ ~ vrra<c-tnbtrlatirm ay to
t.cblow thc ;IvtaraKtl and m,~le.; nbovc lhc ,jveraKe. I ~ v htnndardiring m u ltiwf,i\r crc,s~-t.lbulatiotix)
c s i n n rr.,~dilvstm(,
t t ~ r ~ a l l l 1l.e r\.hctht~rthta cap bt"tt\~rcnrncr a ~ i dxvtrrnvn Tht. I-htlir-t* l ~ t . t r t
c9t.11 t h c ~ rV J rtrjkab .~prnml~.I~c\ I\-111 <1ef~-nCIrtn thc
,
I n a r r n ~ u i t rerr
~ \ i * i d r r ~ i ~ i )(>v(*r
: tin~t.. I n ttrc un+t;liitla rtj i ~ t x i l f~lrtv 11 c l ~ ~ > ~ ~ n c t r , r i thr- t - ~ It...:. Il~~t.1mc~a<~irrtnt-nt)
~ t ~dnl<i ,init 11ie tvpi. of
L\ IW Id be rnclrt- diitlcu It tri r%ht,lblish$In\ Insrtqki<th o r drcllnt, bt-ca~rwbtht, atttl~cnt-ct ~ ir,li1t'F1 ) thl* an,~lysl. 1s d i r e c t ~ dChapter . rl p r r ~ v i ~ f ian
rk o u t l ~ n ~
' \ aIur,' of rncc)mc.: c t l n n ~ c 5'Iht* avtxr,iqc E~CIII~IT ~T~>~I\.CC'II IIICinctlrnlx.: nl a mric,.p i l f alr,>Tvcrc trnrhnrtluv< ;lntt t l i t , d a t , ~~on-.~d~.r,~tfo~i~ Ih;lt tviI1
c l b r n r b r l ,rnd wurntan m,iv h ~ v ~ricrt*,i~ecI c I n i1t~c)luIc~ l o l l tei.ms ~ ~ r ' t l ~ In~t aill-ct clnc's c h o ~ c vI ~ t ~ t w t ~ t lc i~L~Itl~rn.r
~n t j x (15.

tcnTatitc *I-rm.:tht. c.1~nlav ha? t n nnrrorz'td

StC~ndarJi7tnl:incomca a l w allor\'+ ztc to clrnlFarP t l w d i f f t r t * t r c ~ ~ ~
Ivtrvrbcn ic>lrntrit*\ Rr. st,1nd,>r~l17117~ lye cciuEd cuniyarL1 t f ~ t -r t ~ l ~ t i \ . ~ b
~~OSI!IO~ ofR nltm ,~ndi\.onlLhnrrr Erlfaln, 3rmt~ica.Gtv-plsrlr., lapan anrl
I r a n w withollt t-tainj: t r 0 ~ 1 l d 1 ~ 1~1 tlitb d ~t p t r~~ri~ - ee~~ ~ vp rct l~~ i n~ l c . ,
l ~
21Hl CRC7SSSFC I'ION A 1 , DESlCNS
penplc resptmd to sets of questions. I t is an approach tr) analysis that is
frequently used in s t ~ ~ d i cofs ntiitl~dcsand valtles and personality
characteristics. Essentially it involves looking for pattcrns in sets of
attitudes people hold. It seeks to drscover whcther there is an underlying
structure to tlie pattern of responses to questions. For cxarnple. we might
ask people to tell u s what they ronsidcs to be the more important attri-
butes for children to possess (e.g. good manners, obedience, neatness,
! particular political party. Wc can use interval estimates to estimate
imagination, independence and self-controI). By observing the pattern of
answers, wc might find that stme people emphasize good manners,
obedience and self-controf and place littlc emphasis on imagination and
independence or vice versa. In other wolds ccrtain variables cluster
together. %me people will select charactcristics that reflect the under-
lying factor of uorifo~mit!/whilc others select charactcristics h a t reflects
an ralltunnrny dimension.
Similarly, we might hc intercstcd in the types of things pcoplc look fur
in a job. We might ask about the importance uf the following things in a
job:
1 good pay
2 opportunities to use initiative
3 having rvspc~nsfbility
4 the feering of achieving sornerhing
5 absence of too rnuc1-1pressure
6 gcncrous holidays
7 g o d hours.
We might analysc answers to see if there is a structure or a pattern in the
way individuals respond to each of these job characteristics. We might
find that there is a structure: sump people tend to list the intrinsic job
characteristics (npportuniti~sto use initiative, responsibility and feeIings
of achieving something) while others list t t ~ cextrinsic characteristics
(pay, holidays, hours and no pressure).
By looking for a pattern in the way people respond to sets of questions
we are able to develop a more sophisticated understanding of the
structure of attitudes and behavioilr.
As well as describing patterns in n sample we will frequently want to
generalize to the wider population From which the sample is drawn. We
1viI1 want tn know if the pitterns we have described in the sa~npleare
likely to hnId in the wider population and, if so, how close the popu-
lation figures are likely to be to thosp we havc foi~ndin the sample.
When we use a probabiIity sample (Chapter 6 ) we cap use inferential
stntishcal tcchnjqt~esto answer these sorts of questions. Two main types
CROSS-SECTION A L ANALYSIS 2I31
of inferential statistics are cc>mrnonlyused in descriptive analysis: inter-
val cstimatcs and tests c ~ significance.
f
Interval estimates allow ws to estimate within what range of the
sample figures the population figures are likely to lie. For example, in
a sarnpIe we might find that 60 per cent say they intend to vote for a
within what range of the 60 per cent the population figure is likcly to lie.
Using a statistic called the standard crror we would be able to say, f0.r
example, that we can be 95 per cent confident that the real population
figure will be within a certain rangp of the sampIe cstimate of 150per cent
~ per cent or within the range of 58 per cent to 62 per cent).
( e . *2
Another way of using inferential statistics is to use tests of significance.
Tl~cscarc frequently used in conjunction with correlation coefficients.
They provide an eshmate of the likelihood that a correlation at least as
strong as that i~bscrvcdin the sample will also bc found in the popu-
lation. For example, a correlation in a sample could reflect a real correla-
tion in the population or it could simply bc thc result of sampling crror.
Tests of significance provide an estimate of the likelihood that the sample
correlation could be due simply to sampling error. If there were a
rcasonalrlt chancc that it could bc due to sampling error W E wnr~ldnot
feel confident in predicting that the sample correlation would be found
in the wider population.
Explanatory analysis
When trying tr, make statements about causal relationships, cmss-
sectional designs must rely on static comparisons between groups or on
correlations hehvcen variables where measurements are made at the
same point of time. The problems with this have already been discussed
at length (Chapters 3 and 11). The basic strategy by which these prob-
lems are handlcd is by introducing statistical controls into the analysis.
TIre logic af statistical controls
The logic of statistical controls is best understood within the context of
the logic 01 experimmtal dcsigns. The experimental design relies on
comparing change over time in two or more groups, which were initially
idenhcal in all relevant respects, Different groups are exposed to cxpesi-
mental treatments and a c o n t r ~ Igroup is E X P Q S E ~ to no treatment.
Following the treatment, any differences between the groups on the
outcome variable is taken to be d t ~ eto the effect of the treatment. How-
ever, ii ! h ~p n u p s were different to begin with we would not know
whetf~erany post-treatment difference5 between groups were due tn the
treatment or tto the initial differences.
 I n rror.-w.c.rt i,?tla\ rft.ciqns tht, l o ~ l r1, to I-tvnparc crclllps \Y hicli art*
~ ~ ~ ~ (1t1r bt/,l.r bwis
l ul e\~.;tinq ~iiflrrcnces(the indi.p~nrit'nt variatll~:)
\t'p t ) ~ ~ Cc>mpJTta
r i OLIt<t>m~** ( t lie C I I ' P C ~ L ~ C I I I \)arrt~blc)In t hcsi* grr ii~pq. Thc s t r t m ~ t . an.ilv<i\
~t rxcusc whrn ~t 1%. f ~ ~ l ' r r L >fo * .7 tch on a tvhvlr 5t.t t ~ i
l ~m
\ l ~ r .difft,rcncr< rnav be interprettd .+ b c ~ n qduv t t l thc influence of variablrls ~ r ) ? r l r l ! r t r ~ ~ - ( l r r 1~ ht*
f ~ t . tirnrtnvnri,~blt,.;M.V C C I I troJ
~ a t once, ~ I T c ' more
dlftcrmt grm111 rnt~rnbt*r3k1p. Thus diff~rrncc.;in lhc incornc (drprnden! s ~ m i I a rthe grcrul,q wt5vtrrnparc \+,ill his. I-nr rx,~n~pl(b, wtnniiglit think !hat
\.asinl~le)tlf nralc< and femaler [indc~pcnclcnti ~ n r ~ ~ brni>:ht l t * ) I3c due t~, incomc J ~ l t ~ r c n s klct\\.t.cn cs men .lnd \\,nnj<.n .Ire ttue tn thta curnula t ~ e\
the cftect of ~ c n d c rorr incnmp H ~ M ~ ~ the V I male-fi.rnalc
.~, difference.; efft3ctq of gi-nclcr drfCprcnncc.; rn hcrurs o f rvt~rk,tvyt*. t > f job<, atnoun1 of
might trc J u t , not to ~ c n d r p7t.rr Gr but ZII oihcr dilfert>nucs bc.lv\rcrn thr wrrrkforct. ~ ~ u ~ r r i c n and c c * 19 li~ca~ion,ilF,rckgtnund. I\'(, .~rnulrlh!. to
g r o u p . T11c nialr p o u p rn~gfithi* (11d1,r. br bcttcr educated, includi- rcmtrvc3all tliehtb fartors frcm uur cornpari.;on.; of rncn and women. We
nlorc t'ul l ttmr* workers, bc rlre1\vn from d iftt-rent c r c ~ ~ ptiom ; l than thr m i ~ l i dt i v i ~ l co u r qnmplil into m a n y grnupG so ~ h a in n r Kroup rn1g1-1'be
tr-n~alegrrltlF. Tn i\wrk t n ~ t ~.hekht*r m.ilc-fcn~alc rltffYrr.ncr.< In xvorner? ~\.nrkrn);h1)1 tinlrl in yrofr.s51rm~Ii c h c \viit> ! l ~ \ ' ctrrtiary ~ quali-
irlcornc ~ l r cdi~tatv ~cnt1c.rwe r ~ t w !to ct)mp,ire curnp.~ra~~lr* nl.iIe5 ,111d fications ant! 11or.c a l Icnst 1-1 irrar.; worktorcc rb\perrclnrt%.Th~.;t> rcrulll be
tt~malt~s. compnrcci w i t h ,I c o n i p w h l c group o f men. ttnr~thtl.FaTr I I F KriIup n l l ~ h t
?litb c.xyc.rimt.n tall cit-.to,n norrnallv achievcb.; ctrrnynrat,~l ~ t yhehzeen bc < i r n ~ f , ~ rthct o ahrn,c. in 211. r r ~ p e ~ t s . e \ c c pthat I t l i ~nwn nnd w n m m h,lve
CrnuFG by lantium a q s ~ g i ~ m t lr ~t igro ~ n a l stink to
~t i ~ p sC r o r ~ - ~ c c l i ~rlt~.;ign.: trntlc jobs. M . I I ~r~tticrpair5 01 grtwps c-clulJ hc comparctl, R y ct>ntrcill~n~
d c h i ~ v ccrlrnprlrt7b1l1 h- h v I ~ I ( I ~ C I I F (Chapter
I~,~ 3) t w t , slrlctl Ihrt' r t . 1 ~nn for or rc>mt>vinginal~a-fcrnali. dit1t.i-encus we [.,in ,in.;ivcnr tht* q i ~ c s t l r ~ n .
ctiqting diffen-ncr<and all rlatn arc* ctdlccted . ~ at single time pcr~nl,tliv 'M'lrrrl r7llrt.r Ilrlny. nrtut-,~rlt~l, d(x&cgcndrr*mahr*a n v di Fft,rt.rir.t*to i n c o t i ~ rlf, ~'
mntch~ngis dtlne at tht, dala analv\is s~;lgc - ~ l t c rrlata CITC collcctcd ~%.hi,na11 irthcr t h i n ~ sark- ctlir;ll, rncn ant! wonrcn *till havta d ~ t f c r ~ * r i t
s , t~her than bt-lr r e . ~nc~lnic*r, !hr*t~ rvt. h a t t~evt~lenc-i~t>f gcntlt*r~li*crirnin,it~or~ Shr\ nlea,n$ that
Th~krnatrhing will alzva];? he s I~niittlJ, ~ t ~.I dIrs, . i d ( ~ q ~ ~mt*,ltls a t r ~ Illkin gt311~Ec.r iirtlf il: rt>spc~t~rlhlta ftrr tllr ~rh.o~iit,111lfcrr-nccs If,~ r ~ h ccz-c i r r-tlnlnd
r,indc>inalh~cat~cm o f achic\.jng grtwp corny;lr,~l)il~tv IZtbcnn on11 maitch !r>rGI sci c)t \,anahlcs, gcntlcr d lfertanctb- 111 rnctrmc 11 ~+alywar, 1k.e \rcwlrl

in r~l;ltionto .iriaI~lcs~ b n i ~\.l,ich ~t ~trtc>rnrntit,nhn-- brtan i-t>flect~d.7'his bax t47i1( I h c > i rri 1i,11 i t ~ i . o n ~~ i~i,( t ~ > r ~ ,tt*L\rta
~ ~ ~ tt!l{d4, $ (!I(, v'i r~~iL-l~~<\VL> h ~ v t -

nican5 wc n~listanticipate rc,lt.vont r;lriabl~l.;on w'lli~ll+t:rrnips bl10~11C1 ht* conlrnl lcif. I'lit 11 ~ffcrr-nilv,wtb wc,ulcl s a y tha b t hc. c{lnll.n'flctl iF rrabltbh
;l

n~,~tchcd. Our ,111ticipatirln r \ , l l l aluva~=,be I t ~ sthan pcrti3ct. crpl(ir?~ t h t x irliti a I c 1 I' ttt,ren~-~s.
-
I f M'I* wt1ri3e ~ a m i i i j n g!he i r n p ~ c nf
t gender nn incorntb wc would ncc.4 C I I c o i ~ r s rrt, is not pcxqil7le to control for c.1 erv posh~blev;lriaihlc, so I h i s
to con1p;lri. mcrl and tvonicn ~ ' I 1 ( workcd 1 ,I sirnildr number of h r ~ f per i yrr-ibili~!~ atwal,s rc,main* that any gcandcr r ~ l ~ i t cincc~~nr- d di ffrrcnct.~
~ v r t = kIt wnrilcl be inappropriate tn I-ornpre a grrxlp crf W I ~ C I Iw l i ~ r e cor~ldhtn dut. to thc<t* rrncuntrt1lletl vari~bltn+.
m a n y worked part timc ivitll a R ~ I ~ L I F )rnt-n w h ~ 3Wcre mainly full time Ilirria a r r .I vari~t\.of lvily\ in w h ~ c h.;tal~sticalcontrols A r c marJc a n d
wnskcr<. It maLcs m n ~ wnsc t to comparc. mvii ~ v h otvnrkrd f u l l titl~e fhprc. art- a I ~ r i c . l \OF wav\ of interprei-tng tht* ~ . a r i o t ~ pattern\
< of rrsullq
wi tfl u.cirncn who workrcl (~113 timtl and crb~t~n~t, incomc dif lrrcncrs il~dtc i l ~ i1~ t~tltaincd.Ejnv cmmrnon nlctliod o f ct>ri~rrrll in!: for v;lriablr\ ih
brtwccn !Itrsc ITVCI FroilFs. We ro11IiI thrn c(>mp,lre men and r%,orncsn the ilrabomti~tntcchniqlrc.
w h o wnrkcrf say 2q-75 hour< a zveek, and then th(3sc that work 75-21
hours, and so forth. Senqiblt. a n a l y t s rill seek to C ~ ) H I J I R IIikr P z1)!/1t
The elahoratinn technique
I f income d ifit-rt.nct=spcrsi-t hctwcfn rnvn ant1 warnen ichrn wr match
for (control) hot~rqof ~ ' ~ tlit'n r k wc h r i t 7 ~tllaf~ flle initial inctlmr, difftbr-
cnccs an3 1 r r 1 l dllr~to dkffrwnf t~o~rr.; r>f wvrk trf mptl anti ivlrrncn, T h ~ r tih, insufficitwtspace licrc to cxplorr all the Jcatall* nf t h i ~trachniqrlr.
Ilthrla~ghthrrt, m<iv Ilr' nthcr %actr,r, ttwt ma\. ,iftrot ink-r>mrrlifttxr- An ruccllr~ntImt~kt h ~ fullv t dt.\cril.rs thv mrthc~tli\ !hat Atr<~~~t~er~.
ent-tl~l i t ~ t w ~ t 'nlcn
n ,ind wotncn, rIur ,1n,rly5i< will a t Ir>,lstt i . l ~ . ( ~ elilni- Us. 1 tl,yrr I$ S I ~ ? ~ A~ J~ I+ /TI J! IFL!~~S).
II~~
natthrl hours of \x,nrL. a 5 t h r t-\pf,qt~ntit>t~ t<rr the incc-tme d 1fitwnrt3s \Ye
micht iv;lnl t t r cl>rnp.lrr tncmnitl drt it.rtwc.rs .Irnlmq rncv and wclrncn ~ T I ~ R I
sirn~lar~rcc~lpatltm.;. Wc r4t)ulcJco~itrollor (rnntc.11) t)c-cupation aritl cnm-
part- profe-~c~naE mr1n \v ~ t hprcrtcs~~nnalwclrnen, clerical men with
clertcal women, and 50 on. If men nt~tlwiinien in tliu same tvpr t ~ fjob
ha\ i> di l f ~ r r n rncr)mtas
t thcn \vc r%mcrul~l h a v r tcr say that inrclmr J if!t*r- 1 C'rvrilrrt t tl~r' I~rr~rrrrnlt'nijr~l!/.;r~ This will cx;lrnlnin tht. rcla tir>nship
cnrr>+hchs-xn m r n and \rfomrtl art. r r t ~cluv tn the rliffr~t'nttvpes (if jotw bt9trvt.r.n tht. rnrlrptmdtv~tand dt.pcn~lent x.ar~;lfilr.( c . ~ ~ t w d c s rand
helrl by rncn and wnrnen. iilcnrn~,).'l tiis . ~ n!aI s i s might be i r l t h ~ fnrm
, nl corrtalat ~r,n+,t;llll(a.;or
204 CIIOSS-SECTIONAL DESIGNS

Table 12.3 Zero-ordv [?nilcol?rlttronnl t a b l ~ sindlcnliri~spr,cjfrratiol~

comparison of means (Chapter 6).The rclationship between these
two variables is called the zrro-ordt7r relatronship. (a) Zcrrj-ordrr tahle zero-order relationship ni gendcr and incnme
2 Mvlfifil r p ! p i U ~ l l t rmltrnl -r~nrinblr,ltltr third iwrinhir~) This is the vari-
able on w h r h you are trying to match thc groups (as defined by the
categories of thc indeppcndent variable). It is often callcd the test
variable or the Z variable. Your selection of the third variable will I Male Female
be on the basis of theoreticaI modeEs and what you expect could be Low
responsible for the zero-order relationships. This will depend on Inct~rnr
whether you are proposing that the zero-order relationship is High
spurious (sec below) or arc propusing an indirect causal relationship

3
(see below).
Drdw a r~rodrlof the rclatinnship you are proposing between the three
1 fi) Cnnd~tionalhblcs rt.lat~onr;hirof g ~ n d r rand Income cot~trollingfor occupition

variables (see discussion below).

4 Undeutaki> HIP ~ n a l y s i s This may be in the form of tables, and/or
Male Fernare
correlations, or comparison of means. Essentially the analysis involves
looking a t the rclationship bytween the initial two variable? (X and Y ) Low
separately for each category of the control variabIc (1). These
rela honships are called conditic~nalrela tionships.
5 Infcrprrf the rcsrrll< Ways r ) l interpreting differell t results are
discussed below.

There are four main ways of thinking nbout and interpreting the results
of elaboration analysis, as follows.

Ln this approach we begin with a zero-order relationship and then

specify the particuIar suhgrnups to which it applies and does not apply.
For example, there is a general relationship behvren gcnder and income
as illustrated in the zero-order cross-tabulation in Table 12.3 there is a
greater percentage of malcs than fcmaaIes who are high income earners.
Mowcv~r,when we control for occupation (blue colla~and white collar),
and look at both occupational groupings separately, wc find that thc
zcrv-order relationship docs not hold consistently within cach occu-
pational group. The separate tables for bIuc coJlar and white collar dre
called coniiif imml tnblcs: they show the relatiomhip bctwcen gcndcr and
incomc rindcr different conditions (blue collar condition a n d white collar
conditinn). The initial pattern of women doing worst pcrsish within the
blue collar workers but not ainong thc white collar workers. TIIE zero-
order relationship seems to he specific to particular cwcupational group4
rather than general to all groups.
Whert. thr initial relationship holds for sume categories of the contrn!
variable (occupatirm in this case) but not others, the independent vari-
l : is, &c rumhinotion of
able and the control variablc a r e said to i n t ~ r i ~that
the two variables can creatc a particular effect ( i r.. being female a ~ r dbkue
concept of interaction has been
collar) that neithcr on its own creates. Tl~e
discussed earlier in this book {Chapter 4).
Replication occurs where the zero-order rclationship persists in ench
category of the control variable. Where this occurs then the control
variable is not responsibIe for the initial 7ero-ordcr relationship. In thc
exampre in TabIc 12.4 location (rural and urban) has been cnntrolled.
When comparing males and females in general (the zero-order table)
there is n 30 per cent gap betwccn males and females (60 per cent males
vcrsus 313 per cent females have a high income). The same pattern
persists among rural men and women and also among urban men and
women. In other words the zero-order relationship is replicated within
each category of the control variable.
When ihis pattern occurs the zero-order reIationship is said not to be
due to the control variable.
206 CROSS-SECTIONAL DESIGNS

Controlling for hours

Zero order relationsh~p Hours of Income

Gender
, work level
Gender lnctlmc lcvcl

(2) Zero-clrdcr tnble: ~ e r n - o r d e rrdat~rtn+hipof gciirlcr and income

RII Gender initial relationship was due to the hour5 of work variable as represented
Male Female
in the figure.
This is also illustrated in the cross-tabulations in Table 12.5.In this case
Low tlie zero-order relationship shows that a greater percentage of men than
Inconir women are high income earners (55 pcr cent versus 40 per cent). The
Iliglr
cross-tabulation for part timers shows that virtually the same percentage
of men as women are high income earners (31 per cent versus 33 per
(h) Conditional rablcs: rrlatinnship uf ~ n d e snrl
r income contmlling fur lt~c~ticm cent). Similarly, among the frdl timers, the same percentage :ef men and
uf women are high income earners (65 per cent versus 65 per cent). In
0t11~rwords, in this hypnihctical example, when men and women have
comparable hor~rsof work the income differences between men and
women disappear. T1-2.czero-order relationship is due to thc relationsh~p
betwecn gci-rder and hours of work and t h e relationship behnre~nhours
lncnmc of work and income, The fact that there were far more part time w o m m
High than part time men in thc sample generated the correlation between
gender and income. Had the sample had the same number of past time
men and women then the initial relationship between gunder and
Gender
income would not have appedred.
Male Femalc

I..ow
Income
High
TND~RECT CAUSAL RELATIONSHIPS
Elaboration analysis can also be used to clarify the nnturc. of the xero-
order relatinnqhip. In this case we might be asserting that gender
casually affects income. But we might want to know by what mech-
anisms it has this effect. Dofs gender affect income because gender
affects hours of work, which, in turn, affects income (Figure 12.1)? Herc
we are controlling for hours of work in order to see if the relationship
between huurs of work and gender is due tu the relationship hrh~rcn
gender and hours OF work alld the relationship between liours of work
and income. If the relationship between gendcr andfincome disappears
anlnng those with cnrnpar~lbl~ hours of work [hen we would say that the
A spurious relationship is one whcrc the correlation b e t w ~ c ntwo
variables is not because of any causal relationship between them hut
because both the variables are related to, or outcomes of, some third
variable (Chapter 3). For example, let us suppose that people who attend
religious scl~oolsare more religious than those who attend secular
schools. If this reiationship bccause religious schools make people more
religious or is it because both factors are produced by some third factor?
Ln tliis exarnplc the modcl is proposing that the reason for the rcla-
tionship between type of school and religiousness is trot because one
cailscs thc othvr but becausc religious parents arc more likely than non-
religious parents to send their duldren to reli*ous schools (Figure 12.2).
Religious parents are also more likely to raise religious cl~ildren.Thus
the reason whv young pecrple in religious schools are more religini~sthan
those in secular schools is because religious parents sent their children tu
such schools nnd raise religious children.
Resulk such as those in Table 12.6 wauld be cnnsistcnt with this
interpretation. The initia! relationship disappears when .controlling for
zoe CROSS-SECTIONAL DESIGNS CROSS-SECTIONAI. ANALYSIS 209

rable 12.6 Zcm-nrdpr nlrrt a splrrntIrs rrlrztronchtp

nlr~drtlunnltnbll,s ~llrrstrrrlti~g
(a) Zcrtrt~rdcrtahle zero-ordcr rekitirmsh~pof schovE t>pea n d rcllpo~rsn-s
(a) Zero-ordrr [able. z~rc-ordcrrchtlonsh~pof gcndcr and inctJrnp
All Type of s c l ~ w l
Gender
lleliginus Secular
Male Female

Low Child's
Income religionsness High
High

(a) cnndltional tdhIcs: reIat~an.;hipof school type and religinulncs.; ctmtrolling for parental
(b)Condltlonal tables relalionshlp of gcndcr and lncome contrulllnp, for liours of work rcliginusnms

I .ow Law
Incotnc Cliild's
High rcligic)usnesc High

Ftrll i ~ n r r Gender ~ ' O M - W ! I ~ Iprlrcnts

OS lype of school
Male Female Religious Sccular

Low 6 5'%, 822

Incr)mc Child's
I31gh 153, 18"h

N N
-

Controlling for parental religiousness parental religiousness. Among those with religious parents we see the
qnme level of reli~inusnessamong chi4dren regardless of thc type of
School type school attended: 65 per cent of those children in religious schnols were
(religious or highIv religious compared to 63 per cent of tllose in secular schools. The
Parental
same patteyn 1s repeated among those with non-relipjous prlrcnts: 15 prr
cent of children attending religious schnols were rcligieus compared to
18 pcr ccnt of thosc in secular schools.

religiousness
SLIMMARYOF INTERI'KE'CATION OF DIFFERENT PATTERNS

r r l n l ~ o n ~ l ; ktalren
F i p r e 12.2 MorJr,I~irlircnfir(q~l~drrect tp sdrod /ypc and The strategv in elabnrdtion analysis is to compare the pattern in the wro-
stridr~rtrc.lrgror~rnt~~p order relationship with that in the conditional rela tionships. 711e
 Multivarjahle analysis

I-Tabnratinn annly\r< f~ a 5irnplt. form of multivarinte anah <I<. I t ukuallv

f t ~ t ~ %cjn e qthr. rolr of a third z ~ . ~ r i a lin ~ l rilnder.;tandinq an initial bivatl-
r t i X-+Y
ate rc2ation41ip. Othcr lornls nf rn~rltiv~iri,lkc a n a l v s ~. ;~L I ~ ~n.;I partial
15 not L ~ U P to t t . 4 varr,ll?lr J2 X-.l' corrclatiun, rnzaltiple regresrion, dixcr~minantanalyqiq, path nnalvsi.: and
log-ltnc,is ,lnalys15 allow us ti3 corltrnl ftlr a n r ~ m h e rnf nthcr var~ablt=s
Inter,irtic ~ r rb!tu-n-n 7 , S-.)' s l n l ~ ~ l t a n e o ~ i r liTariclu.;
y. forms elf rnulti\bariaSt! a n ~ l y s i scan tlnal-rlc
indrfrmrii-nl imari4\blr and 7 Y Y po~rrerftt!and ~ n f n r m ~ t i vc ~ e p l a n a t o r vanalysis of crnss-sectional datn
!ch?t u.iri;rblt.; X nnlv . ~ t t r r ' f >
I under hptr-iiic t - ~ n % t i lk; w
1 \%'ccan rcmo1.t the confot~ndin:: cfftact.:of GI $14c r f \~.lriahle.; at k3iire
3 L' 1 t i I Spur~r>r~cncl;+ ; ~ n dfocus 1j11 tlitb 'ptrre' et'lec~of nn indrpcndrnt variablc I ht. mtre
nnrl t-t t ariahl~
variah1t.s we can control i ~ r ra t oncc the more valid our cimprr.;on of
prt,ctutt-c h > t h Y and
in !!me
gruup.. bt*comrq,and thc* morc- confident rvc can hc that ,In> uhscn.cd
t~fft-ctof an lndvpcndent \arrablc is duel to that vnsi~bltsand not the
ccrnfcrr~ndinqefl~hct.;of r ~ t l i ~\.ariabl~s.
r
7 Thew rn~~ltrvari~itcr analysis tcchniclut~.: can tell u.; abi>ul thi* jrrrfli
rfh>ct tlf n wt nf i n ~ l c p c n ~ l mvariables.
l
~ ;l set c ~ varirlhlcs
3 13y r o n t r r ~ l l i nfor f ~ r m u l t n n m ~ u s thew
lv rnrrltit a r ~ a t r
t e c h n i q ~ r r(wable
~ 11st o ta\antlnr bl~c~l~sc.rctr> ebtrct ot t * x h .irr.ilili.
rvtthin a ret c ~ frndtaprndcnt varrst-rlt~.12'hilc ~ u n d c r t,d u c a t i o n ancl
hi~ui-3nS ~vtrrkma)' each h;l\.c sunlc rlffect 011 ilicnrnc, rnultfvariatr~
nnalvsis isn;lPlc<us to cxplnrr. how mrlch c,lcli rmr ha5 indt-pendt-ntly
cnmparistsn p r o ~ i d c *the kry tcl interpreting thc wav in which tlic ccmtrol of tht* others. This str,itt.gy mahles us to ~ d r n t i f ythe rrlillrz~ij~rrr/)r>rl-
variablr affects the n n ~ i n a irt'Idlion~h~p. Table 12.7 prnirirjes n simple ntrrr* n!' each o f a st*t o f indcpcndcnt ~,ariahlt.%.
s u m m a n nf M - R ~ Sto interpret the' cornpa~isonbl't~vn~~i ~ ~ T C ~ - C ) Trcla-
~CT 1 h411lti1.ariattb techniqut- t,na'tlle the t~stingof c a 1 1 ~ r?;rrrlel- 2 as t\.t>ll a.:
tionships a n d cclnditional relntionships. thr impact o f s!nglt. ~ ~ ~ r i a b Rl gebst t > ~ t i nnq i d e l thtwe
~ t r ~ c h n i q i #can
~%
bt* cffcct~vc.in Iiclpfn): idcnlify c r l u ~ ~~~~~~~~r.:~l or at Ic.a.;l in evalu-
ating how well a m o r l ~ lf ~ t as siht nf cri~ss-sectinnalrlntn.

Thr ~ l ~ b n r a t i nmodel
n ha%a nurnbrr elf prrs2llcm%that can limit its u w - Thcrt* is in.;ufEicir*nt qpace hpre to descriht* tht. rnultiv,iriatc
fulness a4 a tz7av of contsnlling ft,r varinbli~qin crosh-wctional analv5iq. fcshniqtrt><.XLin]: simpltl yct sr~phi5ticatcdintroduction< are available
h rnajnt prol~lcin is that this sort uf malcliing rcquirrs very Inrgr ( K e r l i n ~ c and
s Pedfta7izr. I(T3; T~lrachnickand Fidell, 1Q89, Grimnl and
samplt>s. Bccause it involvt*~subdividing the snrnplc ~ n t osrparntr Yamold, lLW5).
groups for each catrgnry of tlie test varrabIc rve can quit-kly t n d lrp with
quite small ~ ( w for p an.?ly.;is. If thba te%t \*arrat.l~has, <a? fivr
caleqoric., caili subcroup c;En t-e ~m~11. I f wc rzant to match fnr h v o
\.ariallle\ a t once thi.; probFem is compcwnde~d.7 he iliificullics crc,ated bv Cohort analysis
runnrng nut of c a w s ;IrcL so wriot!q thal this form t r f nnalvsrs iq lim~trri
Frt~qucntZvto r n a t c h i n ~cln just onc v a r l ~ h l ent a time. Cross-~c.iticlnal rlcsii:n5 art, Ijmitcd in thr extent t 1 5 rvhiih they can
\ < rvtall .I< rttnnrrlg ntrt oi cnq= thc rr<trlts o b t ~ i n e dfrom t*labr)r.itir>n pruv~dr~nfnrmationabout c h r ~ 1 g trx e r t ~ m t I~lm\,evcr,
. reptattad ~-ro--
~n;llyst'h can tlrcnme vrrjmcomplex and cfifficult to ~ntcqlret.\Then 1r.r <cctinnnl < t l ~ d i e scan yrm-idc valuahlt. in<iellt.~a b i l ~ t tchanqe Cohurt
Iia\-rn man! diflcrrilt cr~nditicmal tatlle.. thc logic invnlvcd in ~nterprt*tinc annlvsi'i can I I ~ CJ ~ ~ ~ i ofi ' rrpeated
< cros+\ecticinal .;tutlic.; to drscrrhe
tliv vt~ryin):pa t t ~ r n s(,in I,ert-lmr \*crycornp2vu. It unn heccsmi. ex treniel v o,ygctrp~rfrc h a n ~ colfer trmr and to Fdrntif\p the cxlmt tu which this
dhfticillt to taupla~nand prtvqtlnt tlie resirltq of anvthing but the quitc change is attribtttablc to pt'ri~d cttrcts, agclnj: ,~nd cohort c*ffrcts
I
q~rnplctlYnhrrrat~onanaly\e< [C'hnptvr 7).
212 CROSS-SECTIONAL DESIGNS CROSS-SECTIONAL ANALYSIS 213

For example we might want to study changes in church attendance.

This can be achieved by constr~ictingwhat Glenn (1977) calls a sfnndn~ii
ccrhurt Inble.
Year nf data colIection
Construdinx und rendins cohort tables

When a series of cql~allyspaced surveys (e.g.a population census every

10 years) has been conducted, a standard cohort table can be constructed.
In order to construct the table wc would establish an age cc/iorf spun that
is the same as the interval between each survey. Thus i f data were
collected every 10 years we might cstablisl~age cohorts of 20-29, 30-39,
40-49 etc.
The table would consist of age cohorts down the side and data collec-
tion years across the tnp. In each cell of the table we would insert the
relevant information that applied to the particular age ccnhort in the
particular year. If the dependent variable was church attendance, each
cell would contain some measure of church attendance (e.g. percentage
attending church a t least montl~ly,the average numbcr of times people m
that age group attend church each year, a measure of variance regarding
church attendance or some other way of summarizing church attend-
ance) for the relevant agc cohort for the particular study year (sec Tables
12.8 to 12.10).
We can then read the standard cohort table in three different ways to
detect different patterns of change.

Ageing effects or developmental effects are thosc where change is
attributable to people growing older. These can be detected by reading
thr standard cohort table diagonally-from top left to bottom right to identify
lntr~coho~t trends. Reading down the diagonals of the table enables US to
see how a particular age cohort changes as it ages (Table 2.8). Iiow have
people who were in their 20s in 1970 changed as an age group as t h y
havc moved into their 30s in 1980, into the 40s by 1390, and into their 505
by the turn of the century?
Table 12.8 shows perfect ageing cffccts. Tcn per cent uf the cohort of 20
year olds in 1970 was attending church a t least monthly. Ten years later,
when this c o h c ~ would
t have been 30 or so years of age, the church
attendance of this cohort increased to 15 per cent. Another 10 years later
(1990) this cohort - now in i t s 40s - displays a further increase in church
attendance. The same ageing pattern can be seen in any left to right
dnu~nwarddiagonal.
The concluston one would draw is that as people get older they
increase their church attendance. This is consistent regardless of the
1
years one examines, or the age group examined.
Clear period effects are evident if, within any particuIar year, there are
no differences between the age groups (i.e. within a column) but there
are differences between each column. In Table 12.9 we see that in 1Y70 all
agc groups had the same proportion of r e ~ J a church
r atkenders. Age
groups were indistinguishable. This irrelevance of age persists in every
decade. Period effeck are evident by looking a t the trend across cnlumns.
In cach decade the proportion of regular atlenders changes, and it
changes in an identical way for each age group.
A n examination of these p e r i d effects maables us to see how people
within the same age group differ at different periods. Thus in Table 12.9
we can see that people in their 20s in 1970 are quite different from those
in their 20s a t the turn of the century. The age group is constant: the
penod in which this age group is living is different.
Cohort effects arc cvident in Table 12.10. That is, wit hi^ a l l y grzlun vpnr tlie
a ~ groups
e have different proportions of regular attendcrs: the age
group makes a clifftrencc to the Ievcl of frequent attcnders.
214 CKOSS-SECTIONAL DESIGNS CROSS-SECTIONAL ANALYSIS 215

nl I ~ m trwu?i!lrly by ngc
Tablc 12.9 Colrcrrl table ofpercrntlrfi, flttc~rdirrxrl~~rrcl~
~ d 1 L?711-2110!1: p e r i ~ ~E ~~JtC Z S
culiort a ~ Ilcnr, I Cohort effects are evident where there is change within a column and
within the rows, and the direction of change in CI column (c.g. from low
I to high per cent) is opposite to the direction OF change w i t h ~ nthe rows
(e.g. from high to low per cent).

i M i l e a cnhort table can be very heIpf~tlin identifying ageing, period

and cohort effects there is an important difficulty with this approach. It is
impossibIc to completely isolate a 'purehaging, period or cohort effect
(Glenn, 1977: 130). Any table that demonstrates one kypc of effect
invariably display? one other type of effect. For example, Table 12.8,
which was designed to iIlustrate ageing effects (change in the diagonal),
also di~playscohort effects (change in cnlumn). TabIe 12.9, designed to
display period effects (change across columns), also displays ageing
effects. Tablc 12.10 was designed to display cohort effects but aIso
displays period effects. While i t is not possible to disaggregate these
c flbs~ncrof particular effects and the
effccts we can, ncverthelsss, s ~ the
ycjssihility of other effects.

Unfortunately, we do not always have tlie luxury of studies being

conducted at cvui~Iyspaced or cotlvcnicnt time Lntcrvals. Whcn these
conditions do nnt prevail we need to modify the way in which tlie cohort
tabEe and the age cohorts are constructed. This is illustrated in Table
12.11.
TabTe12.10 Cn~iort~nbleofperrmtn~eutt~ndzngchurrhatIm~tmnthl~~h~~~~t.
cohort and ypau, 2970-2000. cohort tjftjcls To construct this table we first establish our cohort widths for the first
year, 1979 (in this case 10-year age bands). For the ycar of the next data
colIection (1984) wc wiIl construct age cohorts of the same age space (i.e.
3 0 years) but the lower and upper limits of the cohort will be adjusted for
the interval between the data collcrtions. Thc same procedure would be
used to adjust coharts between each data collection phase.
For cxampIe, cohort 1 in 1979 comprised the 20-29 age group (10-y~ar
span). The next data collection was 1985 (five years later). To track the
I initial 20-24 year old cohort five years later wc identify Lhc 25-34 age
I
I
group in 1984. The comparable cohort two years later in 1986 will be 27-
36. T h w years later, in 1989, the comparable cohort will be 30-39 years.
The basic strategy entails establishing a fixcd agc span for thc cnhuri and
then adjusting tl~clower and upper ages for f ~ cohort
e equivalent to the
gap hc!wcen the data collections.
This cnliort table lias to be read a little diff~rentlyfrom thr standard
cohort table:

1 To identify ageing effects we read orross rows rather than down a

cnhort diagonal. Tn this case we can see an ageing effect as the per-
centage of regular attenders increases as w e track the agc cohort
across successive surveys.
Summary

l'hlq ch,~l.trr ha.; providcd guic~elmras fnr thc dcscripti\'e and prplanatory PART V
, ~ n a ~ y ~ i i-f, datGi gcncralrd by croqs-?t>clinnal Arsigns. It has not
sctnsirlcrcd the spclclfic tcshniqur;, but manv of thc .;tahstica! $;.l~~dcliiics
pr~ivirlcdIn Ch,~ptes6 apply to croqc-sectioiial analvs~s. CASE, STUDY DESIGNS
The ctzapter consid~rcd the particular problems of using crom-
.t.ctiunnl data for causal nn;llysi.; - pmblems- creattd bv the absence nf J
tirnc dimension to the d e * j p and the lack of rnndomi7ed control groups
The genera! wav nf dealing with thcsc prohlcms is a t the analysis s t a ~ c
r a h r ttian at thr r c s x ~ r c l rlrsip
i .;tag?. Thc chapter otltlined thc. loqic uf
ilsing ~ t ~ l t ~ s t icontrols
cal and rnultiv;lr~ateana1ysis r;o that cm.;~-wct~clnnl
data c.in bc uwCul for ci-tuqal analysis. tt tllrn described on(-particulnr
form rrF rnultit~arintean,llvsis - elahoratinn analyqis - to prilvidc an CASE STUDY DESIGN
itnderct,lndin~ctt the \ r a w in which the pmblrms of cauqal annlvqi~cnn
1~ tacklta~l.Finally, the chapter d ~wussedn rnetlioil oi a n a l y i n g d a t a
f rmn rrgptba tcd crow-section~ls t l ~ ips. d In p.irticular r t fc~uscilon cohrwt
nnalvsr> - how to conqkr~ictand intcrprct ct~hclrttahl~.<and tn be awnre
c ~ Ihc
f 111nihctl t h i ~typc of analys~z.
For rn~inv~ ~ ~ 1 1t . h5 c~a w *iudy h a s been Ihc rlglv ducklling of rt.sr>arrh
d e s i ~ nMust rtwurch rncthudu t r r t 5 eithtbr ignore c6i*csturli~~+ or cotifuse
[hen1 \ \ , Ith ntlit-r tvpcs o t 5 ~ ~ ; i rt-*tvrcI~.
al \I Iwn cawa *tud! JL,+~LW~ 11~1t.e
been disrussed thry havr generally been scrn - trom a r n r t h u d o l o ~ ~ c a l
p i n t nl view - as 'soft'options %me curnmrntatca< believe* cnse 5 t u ~icq l
+auld bc w e d rmly for cxplora tcsv research: ke genera tc Iir.pothesrs tnr
futurc tcqting ~ 'th i rnLJrv r1gor011rrcsearch designs.
The rnrthodoloby of c a w study design has not bccn neglcctrd because
c a w stud ics h,>\rt, bccn ~ ~ n i m p o r t nin r ~~t n c i a scicnccq.
l Indcrd they have
been f~uldarnt*nt,llto thy <tlbstantirfc2nd mcthodolo~icaldct'clnprnent of
the st~cial scicr~ct~s. In hcrial anlhropology studies of trihrs linve bfcn
c a w studies prrr t,.rrr/lrrlcin.Community .itudies sirch as the Lvnds'
h4iddlr,to:l*~1/lu?ql and small GrOUF s t ~ ~ d i ~like . ; M'hyte'r; 5trr7t.t Corrrrr
S o o r f v (1943) have rnacir major crmtributions to lllc development of
ii(7ci~~Iogy as R d iscipIiiir. P~yc'l~onnrlTysi~: ha4 thrivcd on a case shtdy
approach (fr e d , l Q 5 ) Educativnal r e c a r r h , cvalnahon rcwarch and
organi~ntionalrtksearch haire all made ~ r t m s i v e11qr of caw studim to
instcr f h ~ i de~cli>prnen
r t.
The I ~ c ki3f a - \ sbernatic- rll~cu.;.;ifmof w.t. studr. d r > i v q 11.1q hequn to
be a d ~ l r t w e dIn rtacent trrnr>\. I'in ( 1 'J89; 7Qq3)h a < provided ,I pnrticuIarl\-
useful tl-catrn~ntof casc \ ~ L I C I V Jrasigns ;lnd .I porvi,rful dcfrwce of their
value.
This rhaptcr rramincc n range of w ~ v sof dcvel~ryinc:case ~ ~ I I J V
d e s i ~ q Thi5
. i< a difftnrrmt task to that ~xndcrtakcrl for cuprrimental,
loneihlclinal nncl crc>ss-\r~ction~l ~lrqignsI~iyn~r.;e,ns Yin points nut,
'unirkc 4,tht.r sr.;parch strr7tept,s, the prjtcntial "c,rtalog" r r f rewnrch
desi~n.;has n d !,t*t been dtnc.elript>d'(1Q.q'): 27).
 CASE '4 UDY 171.51GNS
A caqc I K tht 'i~blect't ~ fstudy. I t is thc unit ~ n fanalysis (we Chapter 2 )
about tvhich tvc collect inforrn~tion.In c~lsestlady desiqnc it is the unit
that r \ r seek trr ~nderstanda s a w h i d ~The . unlt of analvsiq mav he a
person about whom we trv to build up an understanding that is
~nfnrmra~i by thc ~.rrntcr!in which the wholc caxr ruistq.
Rut c a w studicr arc. not rcstrictd to 111Ai15irlrrnl~. \ V r cam scltct man!,
other Ivpes nf c~lscs- units cbf analysis. A marriagp, a famiFv or R
I1oust4iofd m.qv serve a 5 a caw. Plnrvs si~cha.; a black of linl~ses,n
resldcntlal cornmunrh.. ;l region o r a cotlrltn- can all scnrc.I.; caw>,a< can
argnn~:i~tiaprsSLECII a s a busincs~,R school, government dppartmrnt or a
union A case rntght bc an r ~ ~ ~quch i r fas a rl~vorcrrather thnn t l pcrsnn ~ ~
I\-ho divnrce5. 12ltcrnntivrly, a rl~~c.rsior~ ( c . dcci~ictns
~ a b m t dcrrunqi7ing
a n organization) mzght bc the r m 1~nf an,llysis for the casc xtudv. Tlir caw
~ t u d vccruld lnvol\,e understanding thc clccision a3 a wholc, eiarnining
thc prnccss b\. i \ . h ~ t hi t \\.as rnadc, the particlyanfq. the conqequcncw etc
'T~nlr~ x ~ ~ might
m i q b r thc unlt 4)F analyis. Wc might s t ~ ~ dtlw y lYfiO~.
o r thv !()YO.; t1ar.l1 n i t h t w c ~ l nbc tlrr '!!lint;' .~llouttvh~chwc c,-r>llect
infom,~tinn.
It is htblpful to disting~~ish betwcen ca*eq as ;Y whole a n d rasps h a t
cnnsist nf varinus levcls or components. Yin (1'189) ~rscs thc terms
'l~olistic' and 'embedded' d e s i m s to refer to this Jiqtinctinn.
Some cases cnnsist ni multiple lcvels or components. For example, a
school as a case includps teacliing staff, administrative staff, staff at
different level5 of seniori? and experience, students, students differ-
cnt year levels, parents, go\.ernrnent and community members and so
forth. A school can be coiiceived of at thc 'holistic' level where wtLfor115
on charncterir;tic~of the school that apply to that l e ~ c lThr . school exists
as an c n t ~ t !and~ has schrml le\.ct characteristics ( t . . ~size,
. type of school,
locahcln, culturc nf school, a s ~ of t rules, a struchlrc, a svstrm of
rnana~crncnt,a qchool phi losophv, qtrmgt lis and weaknc~wsi.RLIt thew
;Ire a l v ) man\, .;tlbler-ch or elempnks ttl, ihc- sclrool. A full p i c h ~ r rirt t h r
cchool in all ~ t \cornplcxitv wclt~ld snly be uhi.~ined uf we collcctcd
a from a w ~ d ranee
~ t ~ f o r r ntitm r of the constiturnt elemrnts (ernbcd~led
ttnib) nf the largrr untt.
: rnasri;lRe proc idcs ,inother rxamplr. A t thp linlistic Ii~vcIMT would
%
treat thc m ~ r s i a i :a~s a whole in trrms of things \llch as i t + length, stage.
type, Ir)\.clof cnnil~ctand intimar\', equality. rncdt-s o i int~ractinnrtc. \Ye
could a l w treat Oir huqhand and the w~tt*as rrnbt>tlded u n ~ t sand hutld
tip a ~ L I C I Ifulltnr and diftcrent plrturc nf the m a r r i , ~ ~
bve incorporating
the c-xplarrcncecand p ~ r ~ p c c tofi \the ~ ~11~1.bnntl
~ and ;he t ' v t f r (szrt.~tn~ts).
CASE q1'UUY DIISIGN
A wt*II-desigiird caw s h d y will av(11tI examining lust somc nt thc
constit~~c,nt t.lcm~*nss. It will build up a picturc rd the c;l<ixbv taking lntc,
account ~nformnt~im p i n t 4 horn many Icvels. The final CAW sZ~~il\. will
tcll us nlclre than, and snrncthlng qualit,~tive\ydifferent froin, that which
a n y ct>nslituentelement of the c,~.;ccould tell 11s. 111 the c a w of a school,
the i n 4 ; ~ h t ws i n d from shrdcnts, teachcrs. parcnts, a d n l ~ n i s t r ~ t rand
w
communi& mrmtrers ~ t ' i l l probahl!r differ and, when taken trrgcther,
proviclr a much fuller, rntjrc complex understantling of tlir who]!>than
would tlic pempcctlvc prnvided bv any particular element of thc c a w .
Ihc ~ ~ I i ( >I Sl egrixater than the <urn ot its parts.
Sinct- many c,i\cs will consist of diffrrc~ntelcmtank, diffrrent n~t-thot.ls
u i d a t a collectitm mav br requircd for thc t-! ifierer~telement.;. A siir\.cv of
ktudentq might be apprtlpriatr; c>bst,rv;ltion of classror~msand qtaff
nicetin$'; might nlso bp wc~rthwhi!e;wliilc interviews rntght. bp n ~ o o d
way nt KninIne ~ n f n m n t i o nfmm t~achcrq.An a n ~ l v sof~ ~ &ool
; nrnrds
and archives ctn~ldprnvide utcful infomatrt>rl about thc Iiiktorical
cnntext within whrch thc schotd operatrbs.
Case studies and theory
,\ l thnugli somv case study rescarcIicr3 ccmduct c-asc sturl i r s a.: 111uugh
ihey o ~ i l vhave tn colleri tlie facts about the caw and writ^. abnut thcse in
an c n y n l n g wav, the task of t h r case studv reqenrchcr 1.; fundamcntallv
theor~tiinl.Coll~=cting and anal !sing informa tion from case stud i r q must
r l thrrwy.
bc g ~ ~ i d rby
Case studv c l r s i ~ n sdiffcr from tlic desihws discuwed earlit>rin this book
In that thy\ seck to achic*v~ botll more cumplex a n d fuller t~uplanazion~ of
phenomena. Thry scck to achicvc idiogmphic 3 s well nq nnmt,tllptic
rvplanations (SPP Chapttbr 14). 'Tl~cvalso differ in the wav in which they
~0 abn~ltt y i n g to achicvc c a u ~ a cbxylanaticms
l (5cr Chapters 14 and 15).
Caze \ t l ~ d yrcwarch in socia! ~ i e n c e smust have a thcorctical d imen-
smn. W ~ t h n ~ai ttheor~ticnldimcnqion a casc s t u d y will br of Iittlr, value
tor ~ v ~ d c~enrral17a
,r ticm - one c-nt tliv goals of social ccienitl rcwarch. T h i ~
w c h o n vr)nsidrr%Ihscr I, nys in tlehich tlrrilry i s used in ~lt..;igrl~n~ cast.
5ludies.
This approach, which ic spcn by Yln 11'18q1 a s bring at Ihv hcast r r f cnsc
~ t u d i r b~ c, ~ i n sivith a thecrrt,, clr a set ot rival ~hcorri>\,rcgardlng a
particul,lr phennrncncm. On biw basis ot a theor!# w r p r ~ , J ~ that c t n case
with a ~ a r t i c u l ~set ~ r of rharactcr~stics ill h a w .I particular ntltc'ome.
T!lr t l i t ~ r 3 r vma\ br sirnylt, or cnrnl.leu.
111
L-- CASE 57 U13t DESIGN5 C'ASF STUDY I1I:SIC;N 27.3

f'or ruarnplc~,tht- c a w ?hzcly mrglit begin ueith tht. talloxc~ngqutb>t~on: RegLtr~ITt-+of the c r , r i ~ l i l t v *of~ ~t l w t l ~ c t ~ r vthis , ; ~ ~ ~ r oto~ icase
cl~
is the cjfr3it ~ l de\-crlvcrl,
'\\'?l;lt i c i h r ~ base2
~l icmtrnl of staffing on tlic studit- bt,crri.; with il C . C ~1+k ~ - \ ~ ) ~ r t a t ri l~r~. ri~i \*, ~Fr~lrn
d prr.\ trru5 rwcarch
qt~alityof e d u c ; ~ t ~ nin
n a scIio~17'Dt'\~oll,rd or ~cI11>idhasrd staffing and /nr thc~rricq.lye approach t h e c a w y t t ~ d vwith thc purptr.;c nf test in^
cc~ntsulinvolvch 13~ich<chon1 appili nting, dism k i n g ,tnd promoling stalf OUT tl~txor! in a reaj l i l t > <ituation. Clc,?rlv the selection t ~ fthr real life
a s well as scttlng pay levels ant1 Icnchtng conditiitns. Devolvtd qtaffing situation (tlw cast) wnuld nr.cd to match thc c a n d i t ~ o n sundtxr which the
svstcms can br contrasted wit11 cmtraFi7ed systcrnq where public service theory priipcnes particular r.rutcomus (Chapter 74).
burt=aucrats managc these staff in^ rnJttcrs at head ofiice l e v ~ l Wc . rn~ght
I7ccln rvith thc prrrpr~Gtiont h t ~Jr\'nl~.edcontrol will yir.Id much Iwttpr THEDRY
Iil'll n f Y G CASE V r t r n I F 5
r d ilea tional nutcumcs than c e n t r a l v d contm!. T l ~ crcnqoning \vnitld be
i lint Iclcal conlrol makes peoplc h r more accorrntablc, enables a schnnl to
us in^ ;I thc=tjrvbuild in^ appr<x~chtn c a w rturiics we seltxct cnscs to help
Jevclop and refine tllc prr~p.,m~tir>ns and d ~ v c l o pa tllcor!. that fits the
build a staff prr)Filc appropriate to its necds, and makes it much ~ n s i c rto
case5 wiL\ttrtly. For exaniy-rl~.,cvc migh l .;larl with thc silnplt. pi.oposition
rrward qtaff wht, nrc a c h i e v i n ~the- oiitcornes dcsirrd bv the schord. The
that this intmductirln drvnlved svstc\nlr wi I t irnprrwc. cd iicationnl
~ n ~ p l i cthconp
it hcrc is that edizcatinnal clualih iq fundn&entallr. a rcsult
nutcom~.;. LVc might .;c*Ft~-t a schmd rvlwrc 5uch a .;vstcm has Lwcn
of t ~ n c h e quality
r Ins opposed to thc nature of studcn ts, school rP\cllr re[.<,
intrcducrJ and find that t*rl~~caticmal outcrvncq did, in f;lcf, improve. We
nr~,lnitationals t n ~ c t u r eparcntal
, ini.olvemmt) ,-inti !hat teachcr q u a l i b
iz a function of tmchcr accoz~ntnhilityand teacher 'fit' with t h schtjol. ~ rn~glitthm find a scl~rx)lrn which [fir inttndrrctivn (]I fht. de\~ol\'ed
staffing \vstcm did t ~ o f~>rt>duce better educat ic~naln ~ ~ l c n m rIl'hy r . did i t
The selectinn oC a case to test this theory would 11.i111 u s kr, find a sclioul
tlrnt has intrtld~icccl'I cievolvcd staffing system. 1x1, thnroughlv invt~qti- producc grind otltcomr7.i i l l wmt. cascs and not others?
Our an,~lvsi.;of cnch c,l\r would aim tt, highli~litclitfrrt~n~+t~s bt*hvt.cn
g d ~ n f the i school (including thc 'cmhrdded elcrnrlit?' such a s tpnc'hcrq,
cases 5%-hcrci t did and Jiii nnt work. I h r analvqiq wntild , ~ l < oi d ~ n t i f v
man~tycmcnt,partlnt.;, past and prthwntstlrdrnf5) \rfc 1vnu2d buqF(1 u p ?,
cornrnt~nal~ti~ls among r,twh+ \\ here rlcz.r>l\.t-dqt,+ttingna* ~ u c z c w f u Tand
p i c t l i r ~of t11~rlu;lltty irf edur.-ltioii In the schcrol (houueverdcfintrll ; ~ n d
commn~r.~l itit,.; arnnng c,l.tn\ whcrc ~t 1x.n.; u n ~ ~ r c c c ~ s f( tt, t. l~ . t hc wav in
n i ~ out l ~ ~any l111L.s [hat exist bc*twt-ci~the cducatinnal q u a l ~ l y, ~ n %hc
~l
which tlic -y.;tem was irnplrmtmt~d).Tlirsca case shidivs coi~ldthen bc
staffing systpm 11 pi~qsibIew~ wtn~lrlcollect histnric,ll data re la tin^ to the
used tcl dcvttlop a sct o f 1>ro1~o~i tion5 r ~ l > to lthy
~ conditions and con tcxt
timtl before tlic dt*vnlved s y ~ t c n lwa5 introductld lo see if tlicrc w , ~ s
rmder wliicl~dcvolvt.rl .rtakfing system.;, Ir,i~lt o impmvrmr>nt~ (Chapter
c\r~rlcnccof improvement after it\ introductic~n Wtn ivcluld also qcck to
~dcntifyw h a t r.l.;r% happen in^ In the schnol that might hay[* Icd to 15).
Rie Jiffcrcnce b~t\vrtln thc t h ~ o r y trsting and thc~rrv building
irnprt~r.erncn!s In thr yualihf of education err factom that r n i ~ l i thave
a p p r c ~ a c h cis~ that in thr fr~rrnerw e Irr4~r)l with a set c)f qt~itpspecif~c
prc\,cntcd imprnvements dcqpitr the introduction of the rlevolibcd
propc~qitic>n.;and thm qrc if thcsc wnrk in rrlal world .;it~tntions.Ln the
systpm.
theory building model wr begin with t ~ n l vn question and perhaps a
Tlie point nf tlic care s h ~ d vr n ) i ~ l dbe to qrc i f thc thenry actti;lllv
basic propor;itinn, lnnk a t rt=al cases and rwri trp with '1 m ~ r wspecific
worked in a r ~ a lil fc. situation. I f i t did work then thc theory i s supprwtcd
theon rrr s r t of propnsition.; it< n rcsrrlt r)! rtarnining ; r c t ~ a cascq. l
(nvt proven) 1 f i t did not work thrn we wouId <ct,k tn understand, f r t m
il c < ~ r c f anafvv.;
~tl (if the case, w ! ~ ythc predicted outcome f i e t t ~ rcifrtca-
tic~n)did not evcntuatc. Is it trccalise the theory 1s completely wrnng?
IJclrs the thct,ry r r q t ~ i r esome rc?finernt.nt? Is lhc I hmry applicable only Whilc tlic t w o above c a w stud y approachrs ,Ire rlrrory ~*c,ffirr*ri in that thc
under specific c~rc~tmstsnces? goal is to i ~ s cthe case to trst, rchnc and ~+cvclopthenr~ti'al gcn~rali7a-
In.;tead of thi+ wrnptc prnpnqftinn ive mie;ht dcvclclp a rnnre complcu tions, fh~lclrnical ca5r >tudv use:: theoricc: vrrv differentlt.. Clinical c a s t
modrl that spco flrlt what wc+ultl h ~ p p c nrr-ill1 t h t . intrcldrlctmn of n s t u d i ~ ;lrcx
s r i ~ ~ it e. ~ ~ t r ~Ttl !~ t USP
' ~ ~ther~r~tah to understand a c.;lst.
dt~\ol\-cdstakting stern. The rtrnd4.I rnlght a n t i c ~ p n tpo5itil.e ~ effect< in Thc w.lrr in xvhich a r l ~ n i c ~ ndeals n wlt'fi .I iiicnt illu5h.1ti.c t l i i q s ~ l of e
p,irticnlar ar;pcch of educatior~alnutcomes and ncg.11ir.c e f f e cin ~ r,tht=r% cast. sttidy. 4 child mar V I ~ I aI p ~ v c h o l o ~ ~because .;t Irc is prrf(mning
It inight specify pnritive effvcts c~nlyafter a givrn ptriod of lirnt*, and poorly at srlinol and h,i+ ht,~.nmed ~ s r u p l ~ The v ~ ~ tnqk
. nf thc cl~nical
only when thc d c v u l v t d systcm ;1. i r n p l ~ r n e n t ~ind n particular wnv (r..g, psycl1c71uci<t is to work olrt w h a t is going c~nin this casc ,~ndw h y it is
! r ~ t hbu!l teachrr and cnrnrnunitv cor~sultahon,wrth appropriate. cliccks h a p p ~ n i n < .tZ goor3 clit~ici~ln ~ 1 1 sfart
1 ~ ' t t 1 1~vmptams.wlrat is happen-
anrj balance?). In t)tller weord<,thz= thtw? xve spvcitv might nnttciy.~tc ing a t <chr?()!' In {\.hat nril,i.; of schm~liq tht~clitldperfr3rniin.: ptrv-lyTThc.
I h n ~when cond~tinn. ;i + H + C + D are met thvn thc devolved ?v.;tt=rn clinician rr.il! undt~trbttdlvti.~vt.a rancp ot pi)<~ib!c e ~ ~ I i l n a t l t ,tor
i i ~Forv
1 ~ 1 1 1~ r h i c v c t s aspcctq f and l' aftpr a p'crincf ot V ZivrArq.
I n ~ p r r l r ~ r n e nin schcml pt%rfrrrmanceanrl d ~ ~ r u p t i v~ eE I I ; F I . I On !I Ischncll.
~ 5hr1 \\,ill gather
 CASE 5-I'Ci131'DF.SlC;U

~nfc\rmatitbnt t j l ~ u l l ~lipl ti p~clured what I \ going tm.%hi,will J ~ * t . c l ~ ~ p Acceptance of cultural goals

liunchc\ and collect fiirth(~rinformation t o tcst thrsc hit~icli~?s. Thla p s t -
rlholoeir-t m.lv give tftv child variou.: tt9.;t< to establi.:h hi< intelliqc~ncc
q~~otlcn!. a n d .>ny G P P C I ~ ~ IC t ~ m i nJi<abilitis.
~ Thr ps~*clioEo~istd(w4
to scc ii a cngnitivc ta\plnnatinn crt thc learning dtll~cuftiesh t ~ Shc . Accepts means of Accepts
ach~evrngthe goals
may have thc r h ~ l d ' seyesight or hearinq trsted to src if sight or nuditorv i
problems arc. contributing ta his learn~ny:disabilities. Thr psychologist Rejects
rnavcvaluatcl h e child's rclatinnship sk~llhto establid~if pmblems in this
qPhere c n n t r i h t c to thr Ernrnlng difilc~rltic3.A I t ~ r n a t ~ v c lshc
y rni~ht
probe into thc child's familv rrlafir~n.;hipi, to sec i f st~mctfiingthat i <
p i n g on ,it home IS creating probIerns at .;chooI. P e r h a p the child ~ n t l
r nnt get otz , i n ~the
l l ~ ct e a c h ~do l prol3lcn1s 5pring from ilicir rclatinn.;liip
~litficultit*~. It is also pcwihle that thc child is sufkring from anuirty, but thi.4 cannot pc)ssitrly mean that the case study ccmsists of r.rlr,twf/~i~~cp
d~presGionor some other clinical svndrtlmr. abo~ttthc case. To dcscribe e ~ e r v t h i nis~ irnpossibtc: there must t
xa
The pvint I \ that the clrn~cianwill havr ;l battcy id pt)tcntial e\plann- ~CKU
5.
tlons for thca child\ prclhll*mq.Her task iq to build up a full picturc nl fhr~ Urscription must nrjt, indwd cannot, be athmrchcal. ll!~ always
rase so that she can c ~ a l u a t rwhich explanation best fits the Carts of tlrr selccl 2nd organizc t h ~ which t wc dtlscrib. Dcwriptions will h i ~ h l i g h t
case. Having correctly dtngnrrsed thta n n ~ u r eo r cnli\cl n f the cliilrl'r aspcctq of the c a w . I t will be more likc a paint in^ of A landscape than a
problem4 tht* clinician can hrgin trcatnwnt. h , btl a n interpretation rathtnr than a mirr~,r image.
p t ~ u t c ~ q r ~ ~itpwill
ln th15ta\.lrnplc the ~i>a'i i< try undcr~tanrlthe caw ;lnd 41dvca prtilyl~*m lYrn r n ~ ~ hdo Z [hi< ~r+inyPYFITCFI tl~ct>ricsvr p r t ~ - t \ i s t i n crmc~ptual
~
for this ca\c. I he puq7nGr ls not to tt*<! or d t ~ v - l o jt~l i t w r ~ ebut~ to lt+t2 catcp~r~(:\,or u<lnji implicit thrllrr~.<nf what i.; rclcvant and what
rbuisting t111~)rics. Thc clinician works m,ith p l a ~ ~ s i b rilL1l
l ~ hypr>thcsi,s l ~ jrnpwtant. For cx,iml~le, whcn drqcribing n ptar.;nn 1
~ i l t ~ p v i care
a n d prngrc.;aivcly collects information ti3 hclp sort crut which fits k t . might, a s sociologist, highlight thri r personality, rthnicitv, agc, ~ e n d e r
The samr logic can ayplv In consultancv case shidic*. An org-ani~~tirm and the like becauw. 1 str these as thc critical clcmcnts that cc~nsrltutea
r n i ~ h bet sufferins a g t a t drat ofconflicl and FO@rrnnralr. A c c ~ n ~ u l t a n t social pcrsnn. Anothrr person, a binlogist, m i ~ h tFocus much more on
may have ~1r.lnyc? of pnwihle cxplanation~for h i s and should g o a l w l ~ t physical, ge-enehc dttributes. Each of the descr~ptionsrcBects a dfffcrcnt
the task of ci~llectinginformation about the organizaiiirn that can hrlp focus of ~ n t c r e ~and t a differt9nt nssewment rrf what i s r c l r v ~ n tin
establish wllich of the rxplanahons fits t l i ~ particular
~ rirpnizatinn. 'Thc describing n person.
goal it: not tri dcvelup a thcnry of organiratir~nalconflict and moralr It~rq
fyut 10 idcntih. tvhich o f .;c~vcralaIternnti\.ctheories rnakr- most 5enw in
this pasticul;lr case.
Dcscripti1.e case atudic.; m a y consiqt of qingle or mill tiple cases. One way
of reporting multiplr case studics is to use typolngics and idral typcs.
Typologies may bc thcnretically derived or cn~pirically dcrivcd . A
Although t h ~ qchapter and the fc~llowingtwn chapters foctrs on explnna- thc~rcticallvderiv~rf t \ . p o l o g ~i4 nne that is lo~icallvor thenr~ticallv
tory case ~ t t ~ d i ei -t ;is important to di~ctt-;sillt~scriptivc \ b d v dc<~rn<. pnwthlc*. Fnr ex;irnplt=. CIcrton 11Qh31 d c v e ! n ~ dYIIW p o l o ~of tvpcs of
t cmrsirlt-r descriptive in\(. qtudles for two rm-ons T'hr
I t is i r n y n r t ~ n to de~,ianl.;bawd on thc notions ~ r fcultural cqtjd~E; and institl~!ionalized
t~rstis t h ~ unlt's~
t we can )=and dc<cr~ptionsof cn<cn5 wt. will he In nil iq a
tJrr4ofrGid a c h i e u i n ~tl~rr.;t. goals (Fijir~rr13.2). 'Tlitls the ct~~~fi)rtr~r.;l
pusition tci 'lchieve good p ~ p l a n a t i i mtising ~ those cast%%. The sccnlid theorr.trca1 constri~ctdcfined as somt~oncwho acctlpts the g(lnS::of culturc
rcarcm is thal .I disccrssic~nt ~ dcscriptirr
t casr r;h~ditl.:further highliclith a n d thr tt:r7trpr_i by whicli the cultrtrt*pr~scribest h t k nchievement o l thcsc
I l ~ eirnporta:irt~uf tfienr\. In c ~ . ; cq.t u ~ l vrtb<tanrch. ~ndF<.rhc fllTro:lrlfc,t a~.c.t*pts the c r i a l ~but not thr. prescriber1 rnc6ans for
aclrirvjng them. T l ~ crrIrr(ll15t reject< L-uttural ~rralqhut neverthvfcz.; goes
along w ~ t hthe prcwr~trcdcultural brhaz.~our.Thc rintrrutisf rtaitbctq both
thc gnals and thc mcans of n c h ~ e v i nthose ~ goal<.
Thc p r o b l m ~in any dercription is w h c s ~to~begin and whcrc tn end. This Typologies construdcd in this tlcductive wav rt*yl.c~rnta sct nl ideal
i~ r.\pccia[ lv kc. ~ . i t h P Sc i l <
cact1 ~ ~ U L ~ I A ~ t u d deal<
~ i.r,1t14 the il'l!o!t, c,~st. epc. (Cowr, 1977: 117-4). Ideal tvptnsarc a thenrt-hc~ldescripticrn c + f srts

ut chararlteri.stics we e ~ p e c !trill
t grr together tn a c a w . Thcv represent a
p i r r or theoretical tvpp t h ~ i~t ilcot necessarily found in that purr fc~rrnin
empirical reality. For example, while Wcbcr identified the charactenqtics
of a ' p w ' form oi bureaucracy ( G ~ r t h and Mill$, 1946: 1Yh-204) we may
find no cxan~plcsof aciuaI bureaucracies that function in exactly the way
Wehcr postulates.
Deductive typologies and ideal types can providc a useful way of
analysing casc studies. The ideal tvpe can serve as a yardstick a~ainst.
which tn compare actual cascs (Chapter 15). The template pmvided by
the ideal type can guide the way in which we investigate the casc; i t can
guide what we Pook for.
An inductivelv derived typologv is one in which we start with a cluesticln
and then examine cases in the light uf thc question. A comparison of
cases can then highlight rlusfrrs nf similar cascs. For example, 1 com-
menced a set of casc sttldies with thy question, 'How d o aduIts pet on
with their cldcrly parents?' I s h ~ d i c da mnge of cases and found that
cascs diffcred along a rangc of dlme~isions(u.g.lcvcl of conflict, tension,
drpendencc, balance, r n j o y m c i ~t, inh~nacyand intrusit cness). I then
noticed that cases tended to cuhibi t diqtinct cl t~stersof ~Iiaracteristics.On
thc basis of thesc conimnn elements I d-l~rlract~rized particlllar cascs as
heing of a particular type. Mrurking from a question and actual cases 1
then deveioped a fourfold case based typology of adul t-parent relahcm-
ships that T called parent centred, child c e n t r ~ d ,remote, ~nterdcpendent
(de Vaus, 1494).
The particular characteristics of each type nted not concern us here.
The point is that typologies can bc developed in different ways - tither
deductivtly nr inductivclv. R~girdlessof the way in which they are
developed they can provide a helpful template for conducting, arralysing
and reporting (Chapters 14 and 15) descriptive case studics.
Other elements o f case study designs
WIlcn designing case studies tlierc, are n numhes of elements, apart from
!how discusscd abnve, that can he built intn thr design.
A case study desigr ran bc based on slnglc or multiple cascs. A single
casc design will norrn~llybc less cornpcll~ngthan muEhple casc d e s i p s .
Using the logic of rep!ication (Chapke~14) a single caw represents only
one rcplicatfnn and does not necessarily provide a tough test of a thcorv.
However, W E mav have little choice. Limited access to cases or the
extreme nature of the c a w may mean that w e ran nnlf study a single
CASE STU I I Y DESIGN 227
example. In other sltuatron? we mnv rely on a single cr-rtrcnl case. This can
be appropriate when rve have a clear tlieorv with well-Filrmulated
propositions and wc haire a single cdse that rnects all the requirements of
thc theory. Such a case can provide a rnodcrately convincing test elf a
complex theory (Yin, 1989).
Multiple cases, strategically sel~ctcd,c;ln providc a rnrrch tougher tcst
of a theory and can help specify the different conditions under which a
thcorv may nr may not hold. Fusthcrmorc, lnultiplc cascs a r c essential if
the case studies are heing r~scdfor inductive FuToses (see Chapter 15 O n
andlytir induction). Given sufficient resot~rces and access to cases,
multiple casc designs will norrnnllv bc more powerful and cnn~incing
and providc rnorc i n s i ~ h t sthan single case rlesi~ws.
I Iowever, when using a multiple case dcsikqi tvc should endeavnur to
treat each case as a single cnsc so that we are able to establish a full
account of that case before engaging in cross-case ctmparisons. The
unity of thc singIc casc shauld bc respected (Stake, 1994; Yin, 1989: 56-77,
h paral1c.l dcsign IS one where all the case st~rdies arr clone at once (e.g.
ddtcrent in17rstigators eC3chdoing a cast.). On1y ( ~ thc
t ct)nrplct~r>nuf thc
investigation of tach r a w s h ~ d varc cornparisnnq madc bctwren the
cases. Thls strategy can be appropriate when the project adopts a simple
theory testing apprc~ach.
A sequential d e ~ i ~i.; nonc where case studies follow one another.
Using this approach one investigator could, in princ~ple,conduct cach
case study. An advantage of the sc.qucntial approach is that the selectinn
of each casc and some of the issues examined can be infc~rrncdby
putzlcs identified in earlier cases. One case can ihrow up idcaq t h a t can
influence the selection of subscqucnt casts and that can be followed up
in these later cases (Chapters 14 and 15). When s d n p t i n ~a more induc-
tivc, theory building approach a scquen tfal d e s i p is mnre appropriate
than a parallcl approach.
Most case shrdics, a n d all explanatory caw studies, will incorporate a
time dimension. Without tliis dimcnqion anv adcquatc causal cxplana-
tinn is not possible. Casc studics can provide a p w d wav of carrfulfy
mapping t11c scqlrcnce of events, which is the basis of c n u s d explana-
tions. It has to be decided whether this t i m ~dimension is obtained
rctrospectivel y nr pro5pectively.
A rctraqprctirlc design involves collcciing, on thc one occasion, infor-
mation relating to an extended period. This r ~ q u i r e sthe reconstruction
of the history of the case This rnlght be done ttirough the use nf archival
records and drvcurnents, or interv~ewswith peuplc whn participated in or
chserved past events.
 T'lriq J t , \ i ~ n)la\ lht*clbvit~usprut7lcnls as\r~crali,d with 11)~s01 c ~ ~ r l c t l c ~ ,
rec~rnstruchr,nr l t thc pact rn t l i ~Ihgllt of t h v prra?en!, a n d m i + t a k l n ~he
%pqupnnn in rr,hnt=heiprnt+rvc~~rreii. l-lt)~vc\~cr, in manv qitualions tf~rrc,is - --- - -- ------ ----- - -
l ~ t t l ccl~oiccll~rtt o dr,lw on pcopltb'.; ahiliiv to rccall thr p + t With <.ilse I E rplannlny"
- - -- --
I
.-
Ll~rrnct~vn
- - _-- --_I -- --- rlp'Annlnry
- ,I
+tuifrc<tlrc ilsc ut rnliltiplc sourccc of rvicfmcc can t-educe t h t ~ r n b l c r n s
that qcl 1vit11 t h i ~approach.
c - - --
T~mp
-1-
/ Ravosp~fw~
-- - -
h s m w e
-- .-- J - - -1
Relm4~~-1w.-
-
P~FIIPE~IV~
- -I
A ~ ~ r n ~ ~d uc ~y n s ~involvcs
~ t ~ r t s ~ c k i n gc h a n ~ e sforward over t i r n ~ .t t 1- - C a r crde,

I I I 1 1 I

- 7:-1
-
I I I
ha.; thc obx tolls atl\'nntayr of enatlFin~,thc inur.;h-q,ltcw to I(wk at e\.rnts ?
) ; ; I ?
thev tnccur rather lhan relying rm partial and rcron.;tructcd nccoui~ts.
I3t*pc*ndlngon the i<srrc.; uncfcr consirleratic~nand thtl t y p s 17f c,rses, I--,-
1 3 1 pwatta I wJwb8ql.-
*-
phml,om , s w , w n a
- -1- -
I w l m a 8I
parfilb

I- t --i - I I -1-1- I
pri)<pcrtt\.rcast* \tud1e~might la+t for l;r.lrs. 'The u b ~ i t , u s~Fisailvan~agc
r a n bc I l ~ c t i lnc and rc<ourct's rryuirrd. 'I hew c u n ~ t m i n t scan <cvt>relv
l i n ~ the~ t nrtmber . ~ n drangeof ca.;~.q that can bv5h1~lied.Thi\, in turn, can
rcqult In \'rl.v I1rnrtc.d teql. ol a tlicor\e.
lipgarrl lcrs nf w l i ~ t h ~ar r e t ~ t ~ s p v c t ior v ~ pr(xpt=ctivc approash i~
d . gi>:clis trl build 11p J d e a r ant1 reaqnnnbly ~letailcd picbrirc, r > f
~ ~ d o p t rthr.
thc qetlilcncr In whfcll cvr-nt.; took place a n d nl thr conlcxt In which thcv
i ~ r ~ t ~ r rTu ~ i ?b.( ~ i l dmeaningful c a ~ ~ + t1xplanationq
aI we must grt tlic
serlucncc and thc ct~ntextr ~ ~ h/It .con\-incing c a u ~ a r\planatrt~n l ivjll bt,
~ b l t .to frack t l ~ i b 'ql(vy' bv which Ijnc (or a sct t 1 0 c'vcn t(3) rb17tl1nrtlip
producing .I particular rjutcomr.

Type5 of case study d ~ c i g n s

UnliCt- 0tht.r types OF r c ~ n r c hdesign there is, as ycat,n u invcntrvy of

5pc.s 13f ca-e studv d r ; ~ g n (Yln, IcES<)). l Ir~wrl-er,t h r ilisci~ssrtmr ~ clc-
f
rnvnt.; r l l caqe .;ludy d ~ s i ~ ndiscus'irdl
.; in thiq chapter prn\ridrl%a wnv i l f
t h l n k ~ n g.~houtthe difftrent ways in which casv s t ~ t d i t \can be strut'-
hrcd. ' I ht*w cltlrncnts .,arc:

Jr5criptilfe o r rxpfanatnr).
a tlirory testing or thi.nnl building
r singlc case or rnultipie casr
= hrd~sticor cmhcddt-d units of analysis
pr~11'1 or ~ e q u e n t i ~case
il <t~~dicl<
rtltrthycct~veo r prorprctivr.

A cm~~-clci.;sif~ratrtln ot thew cltlmc.nts rr.;r~lts in thc Ir~yic~rl ~ o + s i b i l i t vof

h.F dlttcrt3nt ~ a r i a t ~ n nof< c a w .;tud\ clr.;igns ( f ~ ~ u r1.7p 1). T~II.;iiqsrc
i.; ~ r ~ c c n t t . dnut tc, rnerruhrltn \t.lth the ranee of pr)~qilrilit~r>q, nrtr
t t > dt~vrloph4 diffr*rent names for ditfcrent ca*ta s t t ~ d yd e s i ~ n s ,but to
yrovirfe a rvav of t h t n k ~ n gabout different altrrn~tir.rnkfor <haping a c a w
+ i ~ t d vdcricn. The d ~ i s i o n smadc In rillatinn to thc+y six rlrmrnls \\,ill
rta~ultin d~ffvrcntwas.; of conducting kt~crase +tt~dv,nnd r j t analyqing
I
the d a t a crrlltrtcd in the stitd~
What a case study is not 1 Method of Tlme 1 ( T,) Intewent~on I Time 2 (T,)

B t . f n r ~cnding this chapter i t is nwrth saying what case study designs are groups Post-test ,
1111k.
'Experemental' Group rs untfom >re on
In terms of the variable
Not thr onr-shut casc 5trd!1
proposed causal /)
vanable
The taxonomy of r e s ~ n r c hd c s i p s outlined by Campbell and Stanley
(lqh3) and hv Cook and Campbell (1979) has led to misunderstandings
of what a case study design can be. Using the template of the classic
experimental design they identify a design which they refer to as the onr-
slrtrt msi3sf~r~-2r!.This is rcprespntrd cliagramat~callyin F~gurc13.3.
As a simple example of this type of d e s i k ~we could consider a group
of children whnsc parents divorced w~thinLhc last year and for whom
wc obtained a measure nf emotional adjurtment. In effect this design
consists nf members o f an 'experimental' group only and relies only nn
'post-in tervention' rnfnrmation.
The earlier discussions of the logic of txperlmcntal, longih~dinaland
cross-sectional designs indicate how this deqign tells us nothing of value
ahout thc impact i ~ fparental divt~rccon the eniotiot~~l adjustmrnt of thc log~crvI7~reby~ n i t i allypotlles~s
l can be s~rbju~tcd
clr research qur<t~nn\ Lo
children. We cannot tcll from this design whcthcr children with divorced empiric~ltcsting. Dccidin~betwccrl single and multiplc casr sturlit.s [st,?
partanthare in any way ~ l ~ s t ~ n r t frum
i u p othrr ct~ildrcn,and whct1lt.r hle1.e latrrl, srlecting s ~ r c r f cnscq
~ c 10 he .;tu~lied,dcvclopin~a case study prutcjml
and dehl-ring r~alcvantdata collection stratrgres 1e.g. the period to b~ covered -
is any tl~njtgr!in emotional adjustment follt~wingdivorce.
rrtrospech've, prospectirc] are all part o f case study d e ~ i v (1993
. 33)
Representing this type of design as an ekamyle of case study design is
unfortunate and provides a rnislcading picture of thc ~ o t e tial n of a well-
Any method of data collection can be used wikhin a case study design so
cot~ceivedcase study design. lndccd Cook and Campbcll (1979) haxfe
long as ~t is practical and cthical. Tndeed, nnp of the distinguishing
acknoruledged this and have renamed thc design as a L I I I E - ~ ~ U I Ip~1st-Ir5t
P
features of thc case study method is that multiple methuds of data
only design, and now stress that it should not bc confused wit11 true case
colltrtion will often be employed (see Chapter 14). Case studies seek to
study design.
build up a ELITI picture of a casc, its subunits (see earlicr discussion on
units of analysis and embedded designs in this chapter) and ib contcxt.

Casc studies have frequently been cqua ted with qualita t ~ v emethods,
e<peclally participant nbscrva tion and unstructured, i n - d ~ th p intcr-
views. This identification is no doubt partly because some of the early,
influential case stt~diesused participant obseriratinn rncthcjds (e.g. Sfnxct
G ~ r ) l rSorir.t!l
r and Middlefuua),Sirnilarly, most ethnograpllic casc studies
used in sncial ,~nlt~rupologp employ participant obser~.atior~,and ethno-
graphieq and case stiidies have sornctimrs been lakcn to be the same
tliing.' I n other contexts case studies are cqrlated wlth unstructured in-
dppth intervlrws that enable the rcscarcher tn bii~ldup a much fuller
picture of a rase than is possible with more quantitdtive based methods
of data collection. In general there k a tendency to equate 'thick dcscrip-
ticln' (Ccertz, 1973) with case studies.
I f WL' equate case s t u d ~ e with
s a particular data cnllcction method we
misunderstand case study d e s i p . Yin argtles that casr s'tudy dp5z4yn iq
Summary
Case study deqigns constitute a major deqign for social r w ~ a r c h Thcy
.
offer a flrxiblr approach, which can result in an extensive variety nf
particular designs. They make usc of a wide variety of data collccticm
methods and they are particularlv suited to using a di\*erw range of
units of analysis. C a w studies In social science rcscarch should bc
fundernen tally theoretical.
Case study designs are particularly suited to situations involving n
small number of cases with a large number of variables. Thc apprc>ach is
appropriate for the invcshgatlon of case5 when i t is necessary to under-
stand parts of FI case withln the conkpxt of thr whole. Casc studies are
dc.~ip,wcRudto qtudy vvholcs rather than parts. They are ;lFst~partic~rlarly
232 CASE STUDY DESIGNS
appropriate when we need tn inlFestigatr phenomena wherc it is not
possible to introduce interventions. While experimental1 y based designs
allow us to exclude the inflr~cnccof many variables, case study designs
are particularly u s c h i when we do not wish or are unable to screen out
the influence of 'external' variabres but when we wish to examine their
effect an Hie phenomenon we arc investigating.
Note
1 Yin (1993:h0ff) careh~IIvdistinguishes ethnographies from case studies. One
reason Yin provides for this distinction is that cthnoPaphies are not suited to
theory testing wllrreas case studies arc. A further basis lor the distmction is that
cthnopaphies do not assume a s~ngleobjective reality that can be invcsttgated
following Ihe traditional mles of scientific inquiry.
14
ISSUES TN CASE STUDY DESIGN
A11 research d c s i p s shnuld be internally valid and cxtcrnatly valid,
should produce reliable results and should hc amenable to replica tion.
Case study designs are often secn to be deficient in all these areas.
However, careful attention to theqe matters at the design stagc can deal
with many of these criticisms.
Methodological issues
Thr intcrnal validity of many of the research designs examined in Parts 1r
to IY of this book depends on having groups that arc comparable e x c ~ p t
in relation to specific key variables. Internal validity in these earlier
designs relies on screening out the influence of variables other than the
key causal variables. They involve focusing on a small number nf vari-
ablm and removing the influence of other variables by cnutrols of one
sort or another. Threats to internal validity stem from the danger that
factors other than our key variable am producing any changes we
observe
IDIOGRAPHIC AND NOMOTHETIC EXPLANATIONS
By focusing on a very restricted range of variables the earlier designs
achieve a narrow or nomofhetic explanation. That is, h e y achieve partial
explanations of a class of cases rather than a 'full' explanation of a
particular case. They involve an examination of fewer causal [actors and
a larger number of cases. For example, a nornothetic explanation might
cxarninc. divorces as a class of cases and identify key factors that
contribute to divorce overall ( c . ~age of marriage, external family
stressnrq etc.). The nomnthetic approach contrasts with iditynrphir
expldnation. Idiographic explanation focuses on particular events, or
cases, and seeks to deveIop a compIcte explanation of each case. For
cxampIe, an idiugraphic cxplanntinn of a divorce would examine a
particular divorce and develop a frill picture of why flmt divorce
occurred.
231 CASE STUDY DESIGKS
Case sti~dicscan be utllizcd for both types of explanation. A case qtudv
adopts an idiographic approach wIicn a full and contextualized
understanding of a case is arhicvecl. But case study designs can adopt
R nomothetic approach. Thiq nccurs whcn particular cases are used to
achicve a more generalized understanding of broader thcwretical
propositions (see drscussion of case studies and theory in Chapter 13).
By dcvcloping a full, well-rounded causal account, case studies can
achieve high internal validity. By seeing particular causal factors in
combination with other causal factors we call assess bath the relative
importance of particular causes and the way in which various cazlseq
interrelate. As such, case studies can achievc a sophisticated and
b a l a n c ~ daccount of causal processes and, in so doing, avoid mistaking
cause with cvrrcIation.
WIIOLES,NUT IUST PARTS
The designs discussed in earlier parts of this book Fwus on 7torinblw
rather than cusps. They examine how different haits nf cases arc associ-
ated with one another (Mitchc.11, 1983: 192). Tvpically, howcver, thesc
trails are not considcrcd within the contexl of the case cd rvhicll thcy are
part. By wrenching traits out of the context in which they occur we strip
them of much of their meaning and ccrnsequcntly risk misreading therr
meaning and sipificance and thus misunderstanding their causes.
Dlumer (1956)uses the term 'variabIe analysis' to describe analysis that
f0cusc.s on variables rather than cases. HE argues that although such
analysis can establish statistical relationships between variabIes they do
not providc much insight into causal processes. Hc argues that:
Thc i~ldrpendentvariablr i s put a t the beginn~ng part of the proccqs of
interpretation and the dependent variable IS put at the terminal part of the
prccss. The intervening prncess i s ignored . . . as sometliing that need not be
considered. (IY56: 97)
Although this is a somewhat simplistic version of the statistical analysis
uf relationships between variables, it contains some truth. Whilc many of
the deslg~iscovered earlier in this bonk can isolate variables that produce
part~cularoutco~nes,they are not so gi>i>da t telling LIS 71jh!/ they produce
these outcomes.
Case sti1ciie3,on the crther hand, en~phasiyean understanding nf the
whole cnse and swing t h r case within its widcr context. Goudc and Hatt
(1 952) describe cnse studies as
a w ~ uf v organinng soc~alrlatn - . to preserve the rrnrtnry cknmctrr n f t h r socml
.
olyn-t IY'IJTX S ~ I E ~ I R. !. il is a11 approach whrch vlews a n y socyl unit as n whole
(1952. 331, their rtalics)
ISSUES 1N CASE STUDY IDF.SIGN
They stress thc importance of looking at parts w i t h ~ nthe contcxt of
the whole.' Thp casc also must bc seen within the contcxt in which it
exists. By examining this context fully the rcscarcher can gain a fuller
and more rounded picture of the causal processes surrounding a parti-
cular phenomenon. Yln argues that
a major rational? (or using [case studies] is whcn your investigation must
cover both a particular phmontpnoi; and the roiltcxf within which the phenom-
enon is occurring eithcr bccausc (a) the context is hypothesized to cmtajn
important rxplanatory ~nformatiunabout the phenomenon or (b) khe boru~~d-
aries hehveen phtnomennn nnd contcxt are n n t cle~rlyevident. (1993: 31 )
The study of context is important because behaviour takes place within a
context and its meaning stems largely from that context. The same
behavrour can mean very different things depending on its coi~text.
Furthermore, actions have meanings to pcnple performing those actions
and this must form part of our understanding of the causcs and meaning
of any behaviour. To simply look a1 bckaviour and give it a meaning
rather than take the meaning of the actors is to miss o u l on an important
source of ~tnderstandingof human bcllaviour.
An example can illsistrate this point. 'rl~lc research li tera turc on
extended families assumes [hat cxchange of help and carc between the
ge~cncrations(e.g, adults and older parcnts) reflects the quality and
strength nf intcrgcncrationrtl ties. However, if we examine the meaning of
intergenera tional help within its context we might interpret its meaning
differently. Within a particular Family thcrc may be a great deal of
intergenerational help: t h adults
~ in the family are attentive to the elder1y
parents, visiting often and helping nut where they can. Rut is this a sign
of family solidarity, closeness and caring, or is it a sign of control,
dependence, manipulation and blackmail (adults help out of guiIt, con-
ccrn about hhcritaiice, worry that siblings will be favoured unless they
help)? b the help reciprocal? What wcrc the patterns of help in the past?
What is the history of parent-child relatinnships in this family? What
particular forms of help arc given? What type of help is withheld? Why
do people give the help? How happiIy is it accepted? What hnppcns if
help is not offered? What are the norms regarding family loyalty and help
within thc community, class and ethnic gmup in which this family 1s
situated? What are the rrrles nf inheritance in this society? What are the
legal obligations, if any, of children tn carc for elderly parentq? W e would
need to address thcse and other questinns b e f ~ r cwc could build u p a
picture of the rneantng uf intergenerational help and the yossihle causcs
of help in a particular casc. To isolate the behaviour from this broader
context and to strip it of the meaning p v e n to it by actiws is to invitc
misunderstanding, and thus threaten the internal v a l i d ~ qOF the study.
To take a further example: our research goal may bc to gain an undcr-
standing of drug addiction. To make sense of adclickion in a particular
236 CASE STUDY DESIGNS
case we would need to understand its social and institutional context
Was addiction the outcome of medical treatment to contrul pain (addic-
tion from trcatrnent)? Did it originate wrth emotional distress following
relationship breakdown (addiction as escape)? Was drug use a taken for
granted part of the immediate social context in which the person lived
{addiction as conformity)? Did the person comc from a very anti-drug
background (addiction as rebellion)? The context is all-important for
understanding the phenomenon (addiction) and presumably is crucial in
shaping appropriate ways of managing or treating the addiction,
In summary, explanations bastd on case studies involve much more
than explaining variation in one variable in terms of variation in another
variabfe. Case sh~dydesigns are devised ti> yield a sensible, plausible
account of events and in this way achieve internal validity. They achieve
explanations by building a full picture of the sequence of events, the
context in which they occur, and the meaning of actions and events a s
interpreted by participants and their meaning as given by a context. In
the end an adequate causal explanation is onc that makes sense. It
invnlv~stelling a plausible, convincing, and logically acceptable story of
how events unfold and how thcy are linked to onc another.
HISTORY
AND MATURATrON
The way in which 'l~istory'and maturation can threaten the validity of
causal explanations has been discussed in previous chapters. Exptri-
mental designs seek to deal with the problems of hiqtory and maturation
by using randomized control p u p s and focusing on d$ferenc~s in
change between these groups rather than on absolute chang~(Chapters 4
and 5).
Case study designs try to deal with the 'problem of history' and
mattrration by looking duscly at the wider context and exploring the
extent to which these sorts of concurrent cvcnts contribute to observed
outcomes. That is, rather than diminatinx historical /cnnt~xtual and
maturational factors from the analysis (by contrnlling them out) the case
study designs incllide them in order to enhance our understanding. In
this way we can arrive at a fuller and richer understanding while, at the
same time, avniding the threats t h a t history and maturation present
to the internal validity of our case study conclusions. The quality of the
case stucly will largely rest on how well i t identifies these liisto~ical/
contextual and maturational factors and includes them in any
explanation.
Simply doing a case qtudy can produce changes in the case and we can
confuse the effects of doing the study with the effccts of pther variables.
Whether or not this will be a problem in a particular case study will
ISSUES IN CASE STUDY DESIGN 237
drpend, in part, on the type of case, the data collection methods, and the
nature of the outcome variables.
A life history of a dead person, which relies on documentary evidence,
is unlikeIy to suffer From reactive effects. Retrospective designs will be
less prone to reactive effects than prospective studies. But a study of a
tightly knit group in which the researcher uses participant observation
and interrrifws will inevitably affect the way in which the group
operates.
We can try to reduce the threats of reactivity by using unobtrusive
data collection methods (Kellehear, 1993). For some types of cases this
can work well, bl others it may not be possible to use unobtrusive
methods for c i t h ~ rpractical or ethical rcasons. If data can only be
ohtained by interviews or 0bsEr~ationit is very difficult to avoid the
reactivity of being a 'foreign object' (Denzin, 1978: 200).
While caw studies may achieve excellent internal validity by providing a
profound understanding of a casc, they havc been widely criticized as
lacking external validity. A profound understanding of a casc, it is
nrgucd, provides no basis For generalizing in a wider population beyond
that case. A case is just that - a case - and cannot be representative of a
larger univers~of cases.
If is correct to say that case study designs cannot provide a basis for
making statistically valid gmcralizatinns beyond that particular case (see
Chapter 15 on statistical analysis). A case study may consist of just one
case (one community, one organization, one pcrsnn) and we cannot be
confident, in any statistical sense, that the case represents a wider class of
cases. Even if we conduct multiple case studies, say 10, our n remains too
small for credible statistical gencralizatinn.
Ijowever, case studies do not strive for this type of external vaIidity.
There are two types of generalization: statistical and theoretical.
Sfufisfic111 genmnliznti~~zis achieved by itsing represents tive random
sampleq. Oh the basis of statistical probability we generaIizc our findings
to a wider population that our sample i s designed to represent.
Theoretical generalization involves generalizing from a study to a
t h ~ ~Rather
y. than asking what a shtdy tells u s about the wider pupu-
lation (statistical gcneralizntion) we ask, 'What does this case tell us
about a specific theory (or theoretical prc>position)?' In Chapter 13 1
argued that casc study d e s i p s are fundamentally theoretical. They are
des~gnedto heip develop, refine and test theories. They d o this rising the
logic of repfication.
r"'
238 CASF STUDY DESIGNS
To understand the ccrnccpt of theoretical generalization it is necessary to
tlnderstand the logic of rqdicnfinn in research design (Yin,1989). Th~sis
the logic that lies a t the heart of generalizing from experiments. Since
experiments typically do not use representative probability samples they
provide nn basis for statistical generalization to a wider population. An
experiment might employ an excellent design that ensures a high degree
of internal validity but unless it is based on a probability sample wc do
not know if those findings will occur in the wider population. There may
he something about the particular sample that means that the findings
wilI appIy only to the people who participated in the experimcnt.
Exprrimo~tsusing non-probability samples argue for external validity
on thc basis of replicntion !ugic. That is, if the experiment can bc repeated
again and again under thc same conditions and produce the s i l m ~results
we can be confident that the experimental results will hold up more
generally. If the experirncntaI results hold up under different conditions
and with different types of experimental participants our confidence in
the generalizability of the r e d ts gou-s. If the experimental results
cannnt be rcplirated under particular conditions then we will need to
specify conditions to which thc cxperiment cannnt be generalized. We
gain cnnfidencc in the value o f and robustness of experimental findings
by repeating the experiment. Mrhcre the experiment is repeated with
different samples and under different condi tinns we get morc of a sense
of the limits of generalizability. Repeated experiments provide some idea
of the range of generalization. If an experiment only works once with a
particular group of people and the results cannot be reproduced we
would have little confidence in its findings nr its applicability more
generally.
Experimental science is based on the Togic of replication. No finding is
accepted on the basis of a single experiment. The same logic applies to
case studies. Indeed a single case stcrdy can be thought of in much the
same way as a single experiment. In Chapter 13 1 argued that the
findings of a case study are tested by repeating case studies (more of this
'
later). We can examine different types of cases under different conditions
and maybe even using different methods. 'She more the cases behave in a
way that is consistent with thc way we would expcct on the basis of our
theory, the more confident we are about our theory. 1f we find that some
rases do nut behave in the way WE would have predicted we need to
modify our theory to acconlmodate that case in lhc same u7av that
thcories wilI be modified to account for the experiment that docs not give
thc expectcd results.
STRATEGIC
SELECTION OF CASES
The external validity of cdqe sh~diesis enhanced bv h e strnfe'yic selection
of cascs rather than by the statistical selection of cases. The strategic
ISSUES IN CASE STUDY IIFSIGh 239
selection of cases contributes to Iit~nlI and tlrrorctict~lreplica tion (Yin,
1989).
Case study d ~ s i g n involve
s sciccting cases for thenrctical and targeted
purposes. We select a case because it tests whether a theory works in
particular, real wnrld situaticms. Alternatively, we select a case because
we tliink it might disprove a proposition, or because we want to scc if the
theoretical proposition works under particular conditions (like repeating
an experiment under different conditions).
The strategic selection of cascs means that we know something of the
chnracteristics of a case before the case study proper bcgins. Since cases
aw selected because they meet particular requirements (as opposed to
bting randomly selected) we muqt first know sumcthing about each case.
This may rcquire extensive case screening before actual cases are finally
selected (see later in this chapter).
How cascs are selected strategically can be illustrated by using the
earlier example nf devolved s~qffir~g systems. If we begin with the pro-
pnsihon []-tat 'the introduction of devnlved staffing systems in schools
inlproves thc qualit-?;of education' our first case study will target a
school in which wrclr a system had bccn intrt~duccd.We w,ould predict
that in this school there would be evidence of e d ~ r c ational impro\~ement.
Further~ncre,we would predict that an intensive s h d y of the school
wcmuld show the following:
1 The improvement in education could be traced back to the ncw
systcn1.
2 The reasons the new system was effective wcre thosc that were
anticipated in the theory (better fit of teachers with school, greater
fcrlings of accountability and greater belief that effort was rewarded).
If we found that thc case matched what the theory predicted, a titern1
replication of the theory would have been achieved (Yin, 1989: 54). Since
this outcome may br due to chance we could seek out some other schools
in which the devolved system was introduced and cxarninc its impact on
the quality of education in those schools. If the same patterns occur we
have further literal replications.
We could then attempt to tesl the theory in a difkrcnt way. Sincc we
had concluded that the imprnvement in education was due to the intro-
duction of the devolved system we would expect, on the basis nf cHlr
theory, that where a centralized system was rctaincd there would be no
improvement in education. W e w n ~ ~ deliberately
ld seek out instances of
schools still functioning with a centralized system to see whethrr our
theoretical predictions were supported.
A s a tougher test wc might look for cases that mn counter to the hasic
proposition: that is, cases of a failed dcvulvcd system. Alternatively, we
might Eook for cascs of centralized systems that, over the samc period,
also delivered irnprovcmcnts in educational qualiv. However, our
thcor] nlaj Ilr ~ z ~ thb7r ~t' It m i ~ h t
c h ~t sLlnarcou~rtftw tlic5t' ' ~ l r v ~ n rcaw< c,wc, ~ j w i ~ n !L>e ~cf;<[?7'tjipr'
l mlr p r o p t ~ ~ i t i o nlvil!
, 4 ~ ! w i ~ bthv n ~ ~ r n l v r
kp~riF\ z l i ~ t tht basic rclatinnship will tjnly \uor'Cc i1ndt.r pnrtiw lar o f cnsv ~ h l d t e srcr can crrnduct.
t hawc propclritirln.; do not hold up and thlq
ccviditiorir. I f we find t h ~ tht* Thc cnmple\itv ol o u r prt*dictinn< m,iv jliflurnce tht. c t ~ n f i d c n cwc ~~
i. yredjctablt- lrn tlie bawr of thr theon' M'C hrlve acfdcd furthcr support havc in utrr rrplic.~t~nns Thc rntlrc ccwnplcx our prediction< the rntrrc
l t = ~r t i- d rt$it-o?io~~(Yln 1USq, 54).
to our thevry. .Ibis i q ~ ~ ~ l!{1trt?rt+f cmfidi*nt n7c c,ln bc in the 51nglc c a v where nur prcd~ctions arc
replicated. Wherc* rlur prcrliction.; .lrt3 qirnple ( e . ~wlicrl . A taxistq then 7
ruill tucur) n . ; i n ~ l ecase study ma\. lint reprissrnt a t o u ~ htmt and rve nm!.
Practical issues want multiplc rekll1cahinn5befnre wc fpcl confident In the rcqlilt5, Illhcrca
our p w d ~ c t i o n sarp more cornprrx and demanding ( c . ~wlrrn . A+l;+C+D
exist\ thm \I'J,Y+\'+Z t\'ilZ orcur) i t is wlakivc~ly tsnlikvl). that t l ~ i ~
c o n , p l ~ rp t t r r r t ivil l crccur ~ l ~lol c~F~nr~ct.. 14cci~rd in):ty w c rnav itwl rllr~tc
ct3nhrlri~tin tlic tt~iyhnlwsof thta tcst that Ihl, ~ i 1 1 ~cast> l e f t t ~ r l vprnuidtxs.
How mjny castn~ t ~ t d i crul.cnnduct s rvrll nl.;od~pc.nd un o u r knrrwlcilge
5incc c a w 5 arL1 u w d for thrortttical rather than s t a t i s h d ~cneralwatinn nf cutc.rnal far~orsthat rni5:ht .ittcrl results. That 15, i! t4.e expect that our
tlir*rihiq littit> p i n t in srlccting caws b c ~ a u . ;they ~ arc ul stTrnc scnsp rcbsuli.; will hrdd u p i ~ n d c ln wide ranqc of c.utcrnnI cnntliticin< wc nlav
reprt*~r,n ta t ivc of w m c ~virlerpnylifatitm. ntTd letvckr casc \hldie<. \Y licrt- w=,1rtx rtnctxrta~nabnut the ~xtt.rn,ll
Nor d n lvtUnerd h, IlIr>k For ' t ~ ~ p ~ ccnsr>s2 nl' W c h a w no sttrr. wav nf cnndit~c>li+ tnirler which our prtdict~int~.i will hold wtamsv ncr~lnlclrt3c;lsr
k n t w i n q i\*llt~thcr'1 c a w i.; truly typical a n d no n r a v OF rqtirnatin~is5 ~ h l ctc,l ~ ~ n r l ctl>c<rr rl iff~rerltct $nrlll~on.; !o see how c~ln.;i.;tt~nllyocrr
h ~ ~ c ~ l7'11~ ~ i jt ~~\ t. r t I\\
~ +irrnt5 pr3cy'1kh to find t \ ~ i c a l c.isi~s rcflcrtq findinj:s hllltl IYin, 1"s" tTc>ri % \ , ~ n ~ pr\.c
l e ,may be un<itrtsrtmIri-th~r thl'
contllsirrn abrlut w h ~ thlitthcll / I W ? ) isill< ianuntar;lhve nntl nnnlvtlc Fink Iltmk txrrr.nJcvol\.tvl ht,i t l In:: Lv5tl.m~~ n c cd l itcatit~nalpc.rtorn~,-lnct~ wil!
matlcr (T! ~ n ~ i ~ ~ co tr iwh'it c ~ n Ytn ( 1 q H Q 1 c,llT.; .;tati.;tical ~ r ~ ~ e r a l i ; l a.111d
l~on linld rtyartlltnk.; of rckfii(~nIc.g ii,nl:r c ~ t v~, t ~ k i ~ ~ rur*al r h . ~anti
n , r t - n ~ o t r Or,
)
thccrrrtical ~enc.r;lli/aticm. 5incr t hc. purpwe crf rase 5tutf it? I \ ~ r t ot a p m , IYL' mav not knari, it thrpo\rtivt&impact wiII hnlil ,rt ~litftar~nt Ic,\.rls
gcncwl17r t c ~a widrr sample of m s r s (r~nunicrativc~ndllclirrnor rtahs- (pnmilr!, <el-ondarv or t r r t i ~ w l ,vr In the d ifferpnt t\.pcas n i .;chnot
tical gencral~znt~nn) t h t w IS little point in t y i n g to find a typical c a w for s v ~ t e r n sfpr~vatc,r P l i ~ i o l ~(1r$ sinti>).In thr tacc of unccrta~ntita.: w r m,ly
a cnqt- shldy. T1ir nwrl i.: to fin~icaws thal will prnvide valid and necd to inrPiitl~c.lrt, studit- that accommrdatr* scrrnc of thrw=cJ ifferencc~
challenq~ngtc.;t< of .itlicorv. ti3 s ~ ict tlic prediction5 can bc rPpllcatcd in thcsc rlitfrrrnf cr~nditinns.
Cllnwr and Strauss (l'jh71 hn\,c uqed thi. term lilrarrfii.nl w t ~ i p l i r i ytci A furthr%rtartclr that may affcc-t the n u r n h ~ rof cnqe.; Ihr ct~ncr~ptual it.

de%cribe tlie normal methud o f selezt~ngc a w s in case sttrdv dtbs~gns. knn~ctvnrkwith n~hicl.rw r Art, ~ v n r k i n For ~ . example, i f qtwial cia+\ $\.a<
Flakim 11 9x6) uses thr tcwn frrrlrsrd .ca~rrydili,pto dcscribc thc snme prooeG3. ;Irr in>pr,rtnnt concept in tllr shrrlv ~t r v w l d hc neccqsarv to ~ n c l t ~ ct~sta.; de
T h c ~ cterms r ~ f c rtc) tlic strattyv uf scltacting caw? that wi31 prciviilc frnm vach cyf the drffrwnt s o c ~ a lclasses. Tlie number of cnscXs would
illuminating examples of a tr-pe of caw (x In de~rriphvrcasc qtllciy dfpi-nd tvi hntv .;wial clnsc wras concephiali~ecfand (HI ]low many cln\<
d r s r ~ n s n) r that will provide nppsnpriate t c ~ t snf ,t t h r o y . cat~goricsthi:, concrp~tralizntion included ( c . ~ .jlrst rniddlc~L - I J . ; ~ and
working clasc, or a more r ~ f i n o dclaw c l , ~ s ~ i f ~ c n tthat i ( ~ ntnclude< upper
c l a w and \.arton.; cla.;~iSicntinnq rvitlirn thc middlc ant1 working cla<ws).
Wr co~ililgo on fortwt-r and chrck t h ~ n g smat under anrr nlrmbcr ot
T h e i > no ctrrrcct rrl~rnbcrnf caw< to i n i l ~ r d ein a c.l\cl .;t~td\-di*'.iqi. rlifi~rentcontliticln* in the end we mu<! m a k e ~ l ~ d ~ c r nat)ouf t ~ n tI~~ k r l v
C,IGL~ ktudt rlt..;isn< can conqi~lof a sinelc E ~ P v. ~r mt11t1yIr~
~ c w t - (Uh;lytcr
~ \ a r i a t i c ~ n;rnd
~ hallc good rt.a<rm\ for t.atx,cttnq [lint thrsca difft.rt~n~
121. I4'ith r n ~ t l h y lcaw ~ ~ *tlldic< tl~cnlimbcr nf is a nlntt1.r r ~ f ci~ndrhc~ns h t the p ~ t t c r n s W
r n ~ ~ *3ffect . r nlstl nimt I7e gi~rdedb) n halt ih
judgcm~nt()'in, I t r R q ~.X). IZqisnlfrcant factor in d r t c r m ~ n ~ nthc*
) : number prnctlcnl w ~ ~ t ht ~hnc-rinrtraintq
~ of time, mnncv and .rccr.;s tin relevant
r p t co'itn.;~ ~ hc l h-lel r i ~ o r l rwith whrch t h e prtlpositions a s r t r l 1-c Icqttld. C J ~ L ~ ' ; . l'hc critic-al thin: ic t t ) 5trlrct the mo<t ~ t r a t e ~ c-;lscs
ic trr twt 4 ~ 1 r
Ll<inr: the Ior;ic o f repllcation n singlc rcp2icatrun tctl5 r l s ~c-rrnttIi!t~g but proprrhrhon\ rathcs than aiming ftir a l a r ~ cnumbtar of r a w < .
n ~ p m t t dr c p 1 i ~ d t ~ ogi\ n ~c' marc corltidrlncc* in trnd ingq l i w f r seck
tlict~rctic~il replication5 wrbr v i l l npetl to cnndl~ctaddf tinnrtl caqe s t u r l i r ~
Whcrtl lvr fii~dcase* tIi:lt not fit with nur tbxprct.ltions thi\ rn,I!.
p~crri\~cr thra r~rri!to rr~nductaddit~cln~ll c;lsta sltt~riib'; to enable c r q trl
r,,tl..... 1..... .,L-- . . I . ...I 1 S T -. . c
govt.min1: sd~rrcrT\I l . 1 ~bc'ttcr ~ t.ci~icntlnrlal ou tcnm~.;w e nerd to knnw
<chlcl-rschur>l<h a v e ~\*hicllsytcrn. IYtb r n i ~ h t2150 need to kn(~r%d, for
~nfc~rrnnticm. I.
G a i n i n ~rclia1,lc informat~rm a p r r r c q ~ ~ i + i tfnr
rrsc;lrch. Thi* mc;ln5 that tlic infurn~ntlonnjuct nvt s i r n ~ l vbe .In artifact
e ~ood

.;rlerliun pr~rl-rosr.;,how l o i ~ ga ? C ~ O O I 1135 113d ;I particuhr system. Wc t p n r l i c ~ ~ l c1rcumst;lnces

r ~ thtb ar of itq c~llt'ct~on. I4.c n i ~ ~ 11111 bt, c o n l ~ d ~ n t
nilght w a n t 10 srlect schnol5 that had good and poor n l t ~ c ~ t i c ~ n n l that the s~irnt*~nfr~rmation rvauld cnTerKc i f ~t were s o u ~ h again t in com-
rrutc*crmP.;.Tn do this WC' I \ ~ ( I u ! ~ to havc this informatron I-efc>nx the par'jble circumstances. Wc certainlv d o not wan1 cnrnpdrison< bctwccn
c a s t ~tll~cficln prortacq t r d placr*. castk<t c ~'t7c in\,;lliclnred bv sloppv collrction o r dl ffrrent approache< twin):
S t r , ~ t c c ~wh;e
c .;~lcction can r ~ q ~ l i rcr~nsiclcrnl.rl~*
r gmttnJ\rclrk t r i applied in each caqe ~ t u d vb e c . 1 ~ 1that ~ rnakcs cases ~ncnmparablc*For
itfent~fyilrc. L-liaracteri~lics of a largr n u m b r r rbt casrs 111 ordtnr to scrB r*xarnpl~,it wnulri be cornplctrly i n c ~ p p r o p r ~ a to t p Cump,ire n1nrrrngc.s
which unrs thc critrria for wlrrtlcm. Thy.; process can br timt. wherc in onr cdse marital sati1;f,3ctmn was ascertained b\. using a
con<rtrni~i~: but is critical ancl must bt. built intn t ~ n l cline< anti b~,dp=ts rtnlcturcd hatterv nf r~t~e<tionnairr kern4 while in anothcar satisfaoticln
C ~ s cscrcvnin!: m a v br don~lin n u m h r r of rv;lvs, Jepl-nd~ngon t1lr was n i c a s ~ t r ~by ~ d ~n\cstic.Ttor chscarvatlon, rlr in ;I thirrl ~~~tiqf'lrtion
t1.p~ case. sun cy ~ l u ~ % t i o n n a iannirah n>, tcprrrb clt a n rrrgsniration, itpa!: detcrminrd t,n tlic b;i.~s r,f rrvpclrtt ol othr.r pc~>plc* ( e . ~kiibnd.;,
ilrclliv~lrccords, d,~tab,iaeso r a wide range nF other sourct7s nray yield c h ~ l drcn, parmts). A1 thrn~gliearl1 c a w s i i ~ d ywould provirlr in lr~rrnation
1 1 1 ~rrfwant screcnlng ir~tormattnn. ilhc>~it'm;lrit,>l sati~factinn, thp w a y this conccrt i s mrasured us in^ thrb
Evcn when .;crtvninq i> intrt>ducvd we mav still cnd up .;clectiny: differcnt tcd~niqrrt..:mivansthat, in rFfpct, it i* betng cctncep!uali;.erl very
inapprrrpria tt. cases, Ni~vesthelr';s, thesr cases mav bc u s r h l l for other d iffcrtxntlv in vnch casc3.
yrrrpvsc\. /\lt~mat~veSy, rvv mav d ~ c i d ctn disccrntin~lcthnt parh-cular Swond, it is imperative that a n v >ttld\ 1% capal-rlc of rcplic.~ticm.
c;tse rlildv .;inrr i t docs not 11c.l~answer the rwiginnl r1.searl.h questions h l e e t i n ~[his c~,nditic>nr t q ~ ~ i r vthai a if crihcr invr';sil;att~>cont1ucti.d a
s~niilar3ttidy ~l~e!' ~ t , o l ~ find
l d thr same psttprns. Succ{-.\fr~! repjic-atiil~~ i.;
.I +altyudrcl a ~ ~ ~ ifrazld. n t t I t JISO cnahlcs r>thtnr- t 1 5 ~ c 1 1r tl1tb pattcm r>n!lm
orcur5 wI1r.n a yarl icul,ir person cnn~llictkthc ol-~;rrvatlon+ or intcrvtcws.
'niib ntlrnl7t.r of cast.'; sclc.ctt-d, and the partic~trarcases scli-ctetl, will bc I1 finrfinp arcx onlt. sap,~blcof b v i n ~ f c m r d El\. ontLpcrwn thcv prt)~idtn
little I ~ a s ~for
c ~ent>rali/ing trl thcarctical proyositinns.
afkctrd 1.j. the , ~ r n n ~ tOFn t tinlr and mt,nc.v ailabltx. IYtlll-t.uectrtcd t-ase
s l ~ ~iwr l can I?(- verv time con<i~minh+.Cnnrt~clucntly, only a limiteil
numbi*r can brt cnnductcd. Thy ~ ~ 1 sfactor t l l ~ g h l ~ g h evtm
ts mure the
iniportnncc of the qh-atry$c selcctim~of caw.;: since ~ r n t va fcw case
studit..: can be cnmplctcd, each {me hsr to 'earn it.: kccp'.
.hccc.;s to caw.; iq the ofhcr critical factor that affect! both t h number ~ Thc trmptation is t r ~'grt your hands dirty' carlv on - to ~ c otu t thert* ~ n d
of casr.; shldied m d thr par!icuj,~l-c a w 5 .;i%lertcd.Certain cases nlav be
It-,~rnFrclni thr caci.4 in full confrdcnce that the truth of thc c + ~ s crviFl
~clmcl~orv cnrrrge. Tc~c*d t r n pi*r>pYt*cornmpnce cast. stud ie3 rvithou t
idt*ntif~cdas bcing stratqica'l lv irical frlr tlic dcqim but pmdicai rnat1r.r~
mav mcan that w c h c,ises arc. not ; ~ v a i l a b lfor ~ rcsr-arch plirpows. knowing what their re.;c;lrr+li q~lc.sbrlni s o r what p r o p o s i t ~ i ~ nthev
s art.
G,~lniny:acccss tu cases that h n v ~been idcnkihcd a< strdregicn I1 y appro- el alunting
priate n w d ~to bc approached 1 ~ 1 t cirnsidtv-ablr
h care and rffort. ,
Thiq is recipe for disaster. lt i q wrcntinl that wc h a w a clrar r r v n r c h
question brforr heginning a casc stutlv.: l n d ~ c t wr i cannot i v c n hegin to
sclrct case\ until r v r ha1.r a clear ~ t a t r r n e n of
t the r e a r c h q u ~ q t i o nWe
.
ntvd to go bevi~nrlhaving a clucrtinn. Wc slio~tldbe ablc to fr,rmulat~
sonle ~ n i h a lpropositinn+ - some Initial an.;rtmcrst r ~the question that the
c.1.;~~ttlcliv4 m.Ty i n v o l ~J~ nl~rnbr-rot ditfcrt-nt irn*e.;ti~att)rs. It1 cast. ~ t ~ r c l i c~ <' 1 1 1h t . 1 ~L I C ~test.
'p,lmlTc!' de.~gns rt is alnrnqt inevrtablt- that a numhcr of i n t e s t i ~ a t o r s Shere will br thnsv who urKr that fEcSdlvnrh 41n11ld b r a frr=(,of ~ l i e n r i c ~
75,111 ~ n v r l l ~ ~-e rprrliap.;
l 1 ~ 1 t3h ifi~tt'rent i n l - r . t i ~ ~ t r condrrctinq
v ench since thr1r;c irnpo.;~nkrr p r r c n r ~ c c ~ t i o na kn ~ yrt.-t.x~\tinq
l t-atccorics on t47e
studv. In rntlltiplt. c a w d~m.;ignse ~ c l cast i sttldv .;hot~ldhi, condtrctt~di r a~ clala. %mc will urgtmthat WP.L;IICIU I L ~g o intu t h ~i~elcl ' trct, f r ~ ~ itlrrt,sctt~.~l!
ii
cnnqiGtrnt iva!' slncc rnconsist~ncybctwern cnqeq can i n v ~ l i d a t rcnsc cncurnbranccr a.: tliwe sirnrlv bltnrl u.; to ii-hat r\.C r n i ~ h r~therrvlsc t we
camparlwn.; Yin (lcjqu) tlrg,,, t h , ~ 1 of 5 wsc
~ C . ~ C J V protoc~1T~ hrlp and make 115 dtanf to wliat prkr\pli&arr I r v i n ~to tcZl rl<. CJrlr jr~b,the): will
acliir,vc this son51stcncy. say, i < to ,lllnw clthcrs t o tell u< their story rather than u> irnprlsin); a
1'1,-1nntnq t o encurc conq~stcncv,icrtn< c a s ~~ttsrjicsam1 ncrtKs inve5tl- thrclr~tical~ n t p r p r ~ t a t i oon n their <tory, or uqtnq their \tory ttjr thtbclr-
~ a t n r isr ~ n ~ y r ) r t nFclr
n l Iwcl rea.;cln.;. hrst, 11 i*~rlinncc~ tlrc rt~liabilitv( > f o u r eticnl pilrycrres.
244 CASE STUDY DESIGNS
In mv view this is not thc task of thc social scientist. rt might be thc
role of a biographer, a novclist or an activist. The role of a social scientist
is tc> develc~pand evaluate theoretical generalizations that enable us to
understand whole dasses of cases - nut, in the final analysis, individual
cases.
We do not have to have a welL-fo~mulatcdtheory to tcst. Our questions
may be such that there a r e no obvious theories to test. l3ut this dws not
release u s horn the nrcd to be well read and well prepxed theoretically
beforc going into the field. Gasteur observed that, 'Where observation is
concerned chance only favours the preparcd mind' (quoted in Mitchell,
1983: 203) If wc are unaware of rclcvani theories, concepts, dcbates ancl
thc Iikc, we will probably miss the significance of much of what we
might come across. Without having some idea of what we are looking for
we will not know what we have found.
Case s t ~ l d i ecan
~ involve the collection of a vast amount of information.
This ~nfrsrmatic~n must be carefully prcrccs5ed and distilled bcfnre it can
he yrescntcd. W e cannot simply describe the casc and somehow let the
facts speak for d~ernsclvcs.Any attempt ti) prcsent all the facts will result
in an indigestible mess that is unreadable and unread.
The facts do not speak for themselves and we should not pretend
othenvise. Describing a case always involves selection and ordering and
at least implied construction of causal sequences and interpretations.
Therc are many different ways in which case studies might be pre-
sented. We might describe each case as a whole s n that the reader builds
up a clear picture nf the casc. Alternatively, we might describe good
examplcs of particular types of cases and then cornpatc thcsc different
types of cdses and draw general conclusions regarding our theories.
Hochschild's The Second Shiff (1989) prnvides a good example of this type
of reporting. She describes a series of marriages and from each of her
cases she extracts ~ C S S O ~about
S the domestic division of labour and the
procrsses by wluch marital inequaIity in this sphere is maintained.
Alternatively, we might repnrt the results of a series of case studies by
extracting ilrern~ffrom the case studies and reporting the findings of the
case studies thema ticall y. For example, Vnughn (IY84) cnnd~~cted rrtru-
spectivc case studies of coupIes w hoqe Intimate relat~onshipshad cnded.
From thcse casc studies she Identified a set of underlyng patterns and
stages beneath cach disinfegrahng relationship. Another approach is to
focus on a series of qlrfstionl; and rcport the cases in terms of how they
shed light nn thesc questions. For example, Dempsey (1990) conducted a
single case shldy of a small country town The guiding questions in the
case study were questions such as, 'I-low unequal arq people In this
comrn~tnity?','What are the key bases of social clifferentiation?', 'How, in
ISSUES IN CASE, STUDY DES1C;S 245
the face o f cnnsidcrable social incyuality, does the communily maintain a
strong sense of cohesion?' These questions guided the study a n d pro-
vided the framework around which the study was reported.
Regardless of which approach is adopted it is highly desirable that
the logic of the research design remains clear. The reader shcnlld bc
clear about the prnposition(s), and why a particular case was selected
to trst the proposition. Tht reader should be able to see that there is a
structure and a logic to the selection of cases - that they have bcen
sdected for a purpose rather than simply because they happen to be
available.
Any report that merely provides a number r)f stories is incomplete. In
reporting ctlse studies it is critical [hat wr relate tl~ecases to propositii>ns
and that we seek to draw comparisons between cascs and arrive at
generalizations or more refincd yropoqitions as a requit of the caw
studies. W c need to tell the story of the set of cases - the generalizations,
thc propositions and the questions they answer and the propositit>ns we
end up supporting.
Ethical issues
'Thr s a m e ethical principles apply Lo case study designs as to the designs
discusscd in previous chapters. The particular ethical iqsues confronted
will vary, depending 011 the typc of case study design adopted and t l ~ r
particular form of data collection used for that design.
For cxarnple, the ethical issues w ~ l lbc cliffereit if the casc study
involves an active intervention or a passive intervention, or requires that
W E obtain a retrospective account of an event, intervention ur change.
The ethical issues will also differ according ta whether data are cnllccted
by interviewing individuals, using informants, analysing official records,
passive nb3cnration, conducting a survey of casc elements, or using
participant observation. Since case studies frequently employ a range of
diffcrcnt data collection techniqucs fur the one study it is likely that a
greater range of ethical issues will arise when using a case study design
than with other designs
TIicre is no point In repeating the ~tlncalmatters identified earlier in
this book. You sliould read thesc earlier sections carefully and you
should see how they apply to the specifics of the part~crrlarcase study
Jesim you adopt. H o w e v ~ rsince, case study designs frequent1y involve
~k--trliningmore in-dcpth information and Crcqllcntly employ participant
nbscrvation techniques, some ethical issues arise that are less of an ~ s s u e
with other designs.
I argued earIier that reactivity can be a significant thrcat to the intcmal
validity nf casc s t ~ l d ydesip;ns: thc pwsence uf a r~searchcrcan alter thc
dynamics of the cascs being nbserved. This can be partlcuIarl y so where
participant observation techniques arc used. One way in which some
 I
I'
246 CASF STU13Y DFSlGNS ISSUES IU CASF STUUY I3ES1GN 247
il
I

mlgl~th y to reduce the effects nf rractivitv is to disguise their I

1;
~ d e n t ~ tays rcscnrchers to avoid their r r s ~ a r c l ~ cidentity
r intruding. For
example, an rnvestigator may want to examine sexual harassment in the
workplace. If he went into a companv revealing his idcntity and purpose
schnnl and you learn of the identity of a person sell~ngdangerous drugs
to students? What do you d o if you discover, during your rcscarch, that a
group participant is about to commit a csirnc? Is tlre betrayal nf trust of
thnse around you more unethical than one's responsibility to those in

i
!i
it is likely that people in the nrganization would behave differently while danger?
1; the investigator is present and thus undermine the study. As an alterna- What is the responsibility of the researcher to othcr rcscnrchcrs? If one
I tive the investigator might undertake the research by obtaining a job
with the company and covertly observing what goes on.
rcsearchcr betrays the trust of a community, even if for ethical reasons,
does this thm undermine the capacity of other researchers to study in
I The ethical issues of deception, and failure to obtain informed conscnt, that community? Once a group has learned tcr distrust social researchers

1 i

I
obviously a r i ~ in e this situation. The dilemma it higl~lightsis the tradeoff
that occurs between ethics and internal validity: keeping to cthical
guidelines can threaten the internaI valldity of the study.
There can be little doubt that revealing one's identity and gaining
informed consent can undcrmine a great deal of what Punch (1994) calls
it can be very difficult to conrfuct any further research with that group,
and this may havc further ethical consequences. The behaviour of some
researchers w ~ t hindigenous groups has led to profound mistrust by
some indigcnaus co&unitic.s of outside researchers.
How do you resolve the nbjcctivt. of feminist research to promote
II
'strcct style' ethnography. rn some cases revealing one's identity can identification, trust, empathy and non-exploitative rcla tionships between

I effectively kill the research prnjcct. In these situations careful considera-

tion 1111ist be given to the merits of some form of dcccption in the light of
the likely benefits of tlie research and thc potential for harm to the
participants. Whilc it is easy for a researcher to rationalize that the
the investigator and thnse participating in the research on the one hand,
and tu expose unethical and harmful belraviour on thc othcr?
We lack clear-cut answers tu these dilemmas. They highlight the fact
that ethical principles can often conflict with une ai~olher,and that
11
bencfits will oi~tweighany potential fur harm, thcrc 1s a Iot of sense in ethical issues can compete w i t h rnrthndological and practical
11 obtaining advice or even in having dccisions made by third parties such principles.
,!I1 as ~nstitutionaI ethics committees.
lnvestiga tors will need to resolve cornpcting ethical cr~nsideratinns.To
1'1 what extent should the ethics of informed consent and voluntary par- Summary

I ! iI
I
ticipation take priority over tlie value of research exposing I~ighly
unethical behaviour?
Even if a researcher revealed his/her identity and gained permission
'I to undertake the study, the question arises of how to get informed
I
II consent, and From whom to obtain that consent. If we were conducting a
study of sexual harassment in a workplace, would we obtain pcmisslc~n
I
from the gatekeepers in the organization, or from all people whom we
I might observe in the coursc n i thc study?
Do people need to he informed that their public behaviour is being
observed for the purpose of research? Clearly m many cases this is
simply impractical (u.8. observing the behaviour of people at a party, in a
dernonstratron, shopping at sales). In my view abservtng public behav-
This chapter has emphdsized that thc intcrnaI validity of case studies
relies on a Ingic that sets this design apart from the other designs
discussed in this book. Case studies focus on a large number of variables
and thc way in which they intcrrclate. In contrast, the other designs rely
on cnntroIling out the influence of other variables and on focusing on the
influence uf a very limited number of variables. One of the strcngkhs of
case studics is that they atternplt to understand the significance of
particular factors within the context of the whole case rather than by
screening out this context. As such case studies havc the pc~tcntialfor
good internal validity based on a more thorough understanding of the
meaning of partic~~lar behaviour and events than the other designs

I I
iour raises few cthical problems. So long a s individuals arc not identi-
fiable there should be n o danger of harm to participants. After all, if the
prclvidc.
The external validity of case studirs is based on the logic of replication

:I
li
I
I
bchaviour is public there is l-rardlv anv invasion of privacy.
Anothcr ethical problcm that is more likelv to arise whcn using nbser-
vat~clnin a case sh~rlyis the problem of what to d o when we observe
illegal or i2arrnful bchavirrur. Should the cthical isstres of informed
rather than on sampling logic. Case qtudics are used to generalize to a
theory rather than to a populahon. Cases in casc study designs are
selected strategically rather than statisticaIly and a r c selected as critical
tests of theoretical propositions. Iswes in selecting cases, the seTectinn of
1 consent and voluntary participation take priority over exposing highly research methods to collect information about the case, and condtlcting
I
I

I ,
unethical h ~ h a v i n u r ?For cxample, what wuuld be your ethical respon- case studies were discussed. Finally, particrilar ethical questions that can
sibility if your case studies of families revealed cascs of dpmestic v~olence arise in casc studies, particubsly Lhosr employing participant obscsva-
or c h ~ l dabuse? What should y c d~o when conducting a case study of a tion and ethnogapliic methods, wcrc raised.

-- -- A
 t Thr. crlriccpt nf n wholt- IS, .is C;c?c>rlr dlld tlntt (1Y51) point oul, n ctmqtrflct
1tqt.lt It 1.; r l q I! Iin <It-hntbqi\har the '~r,lu>le I.' 7'11eill-lincat~rmI,~*twt>rn t h ia5t1
~
It h ~wlx7Te)
. and I!< context 1%. intlwti arl~~tr,ir>.
2 'Tvyical' casr.: qhr~illdnot bt, ct~nlu.;rclwith selt*cting '~rlcalhpical' c a w < - CASE STUDY ANALYSTS
C.-X(-: th.it mn! n,prr<ent ail idral tvpe (Chapter 13)

7 It ma\ br. arlrrtyrlatt. to cnndut t c . 1 ~ ~ t l ~ d iw.ithnt~t

e claur rr1~c;rrch
qut~clion<tlr prrlpr><rtionsw h m w r art. ~iolnr:erplntatorv case fttrd,t,\. I1 wrL.lrr
iiclng fht\ c a w 5ti1d\ In clarilv cnrrcpfts, Ihtn lo l ~ ~~t dpe n l t ~l ~u ~~ ~ \ ~ !l.lml-m~cht (HIs
t-nttv thr held b~.Cort= a n,~t.;lrrhtluccllon l i n t berw iitiall/t~l R u t fht.; t y p ~ L ~ ~C . l s t >
~ dt**~clleJ
41 L I V, I I C J~ ~r ( w v r ~ t qttrxstitln,
i n 1 1 1 . 1 ~pr~x1 l I + I t.rv ttrfft,re~~t
trom a chirp
s t k ~ J \ .d r w ~ n 4. 1-GIW chlifv dr.vyrr IS Ihc \lrurtllnxr~fa <tudv lhat M 111 cbnat,le 11, Ir, XlrbthuLl.s I nr nnalvsing r.a.;t. stt~ilit+ rc lc<.; ~ v ~ t ~ ~ m n t i cdr\.cIopccl!
a~lIv
. ~ n ~ . ~ril -' ta ~r crh qur*\lltm.: It reftarnt o tht- loclc and qmrcturtnof thc ktul!\. ~ canalv51ng
than ,Ire thc t ~ . c h ~ i i q ~tljr s data ct~llccttdwith orher tvpta.;of
rewL~rch dtlqiFns Analvstq wiih thtw othcr. dc.~gns typicnIlv use.; slatis-
iical tt1chn1q~rcb.;that rclv ciri I ~ ~ i v i rmanv
~ g r,ises ;lr~drrvolve ,irclund
cc>rnp,lrlng grutrp* nt cases. Furilirrrnort., ,ittc.~npt.; ti) ~ r n c r a l i r rfrom
ihesc Jc.;~i:ns ~lrcb a s ~ don prn!lL?tl~!lh tht.~lr\.rvhich. in turn, rt-li~qnn
havinq ~ u t l i c i r ~ [Ci.tI \ I . ~ tram nblrrc'lltn gentsrali7e irritli a y v c n Ievrl of
prcclslnn , ~ n dccrlnnitv.
~ , i ( l t , ~ t t\\,ITII v , ~ r I ~ 3v11r rclv 071 c t ~ ~ n p ~ ~ r !l~-ta>t,f?i
Tiit= r ~ ~rrli~ ~dt,,~gn\ iv+~is
;:rtillp< u<tng a I~m~ttarln i ~ ~ n l ot ~ r rI ariallles. Caw \ t u ~ l t e s , h a w c ~ c r ,
,~ch~cx\t. euplanattnr~sb! intt-n.itvc1 .inalvqi< of m a n y \'arl,lblr.; i ~ ~ i t l r t r.Ir
casts ~ ~ P I I ~ cI a~wI s c ~ i ' tI' I S ~ Cto~ replicate rrsults - z ~ c l t a 5 n mcans of
locat my: cn11w1c \ y l d ~ ~ a t i c ~ nIns .acld~tlnn rn~tltiplcc a w s h d i e s are usrf111
tc~rmnLt nq cnmparxwn< h r h v t ~ inn w c and t hu.; t.\trapula t ~ n to g bro~cler
tlicoric.;

Statistical analvsis

AltIinugIi the logic nf thth e,~rlicrd c s i ~ n sd m . ; nijt rtlcluirc statistical

analysis, s~rt-hanalysi.: is I$*IIIPIV ~ i w d M . ~ t h o d sw c h a s cnrrclahl-rn~,
c r i ) w - t d b ~t ~im, l ~ sr.grrssit,n and conlparlsvn of mean< rtvly on splc of
r;l.;e< and o n Zlllildmq a e ~ i c a lp ~ t t u t . rc>r dc=ccribin~trenik acrnw
~ l i r r f t ! p f rcast+^
~ r,~ihcrkh.ln rrflcrting tlie charactcrist~cso i any particular
CrlhC.
Be)(auct* CAW qtudv ~ l e c ~ q nfix-~sr: < un inrl i v i ~ual l c a w q t hct. ernflov
dikfcrt*nl mcthcldz 11t analrsi+ A numhcr ( d s ~ ~ cmethodc h h w e I7crn
dcvelopcti cjr a r r In thc prtlcc.;%uf I w i n ~rlcr~clopc,d.U;l?c study nnnlyq~s
r r \ ~ o l v c,~rountl
~ a s w s z l n q tlir ' f i t b t ~ t ~ v r ~acnn ~ n !\.id
d ~ r a icasta and tlrc
tht,or\ or theories b r i n ~rr.;tr.il. Tht. glral nf c,ast. ct~tdvanal!*.;iq I <
bh~*nrctical~ r ~ n c r a l i 7 a t i nrcjthvr
n than ~tnti.;tical~ p n t ~ a l i 7 a t i c(Cliaptrr
~n
1 4 ) . T l t ~ sgoal ,~itrlctsthl. wnv In M~IIICII W ~ gI r ~aL~t>u!our analysiq.
Rcc;luw thk. ftwus i< on ~ndiric!uaI c n ~ 5I ! I + innppr~7yrlafi., I\-hcn
a n , ~ l ~ c r nIracc
t : rlata, to t o l ~ t r t the tli~rnbcrof cases zhar hari particular
cli.rrac~cr~~tic-< crr beIi,lvt*d in J parlic~il.irwav Tht*<mall number of cnsc.;
and till> a t v p ~ c dn a t l ~ r cof Zht* c&isc.ssrlt.rtorl rn,~kt>it rncnnin~lcs<trr
,in;llv.t* wbc studies b\. c n u n t ~ l l ~ .
Ti115i < nclt ttt sav t h ~ casi.
t \tudv analwis cannot hnve a guantitati\.v or
statiqtlcnl clrrnmt. "4 ca.srb can be dtv5urr/rt-if qtati5hcally. .A p r t t c u l a r
rt'ginn ~ n i ~ be h ta case In a r r w ~ r c hp m i ~ tEt. can bc dcscribc~llrn tcrrns
of ~ t s.;i/c, wvalth, avtrJgL3 ,Ige, nncmploymcnt ratc, crime rate ,>nd t h ~ '
like. Howcvcr, the f o c u ~nf the analysis is t o d ~ s c r i b cthe rcgicin - to
b ~ i ~ ludp .T pictitre of tll~ltCSW.
F~lrthrrmrirc,where cn.;t7.; include emtreddrd units of nnalv<is ( s r r
Chaph.r 11) thc analysiq of qcrrne levels may wcll include the sort rrl
a n a l v + ~ <~vp~caTIy a~srrci,~todtvith other rltvsns. For crmmplc, n ewe
shidv ot ;! rrglon rniyht includt. a sumev ot rt,.;iJt>ntsin the rcgiull (thi,
reqlclcntq t-tjnfi an crnbcdcl~d unit thin thp case). The surl.ty of
resident.; wnuld include thc tvpr of analysts tvpically used in ,I crtrsq-
sectional c-lcqrgn. Howcvcr, tht. ftwus of the ovt-rall analyws w n u l ~ lhi- nn
the cnsc ns a whole - the region. The survey r c ~ i ~ l twould .; simply hr rmr
piccc 1r-t t11c j~gsawto builrl i ~ pa pcture of tlic hole case.
In Chaptlar 13 1 empha.;i~i*d that case srudicc arc h~nrlamclit.>lly
thcurvtic,~].Tlicir analyk~sis a l w Fnndament.~tlytl~i>orrtical.The annlrf<~s
nwd+ t o 1.c built aruuntl thr' theoretical ~ r t ~ p a c l t i u nbctng s tt.ctt.rl nr
de\~eItrp~~l.
Meaning and context
In Ctiaptvr 13 I she~sec!!h.it caw studies arc particularlv suitcd to
rescnrch problem5 rvhvn the phenomenon In which we are in tcrvstrd
either be distingui~lit~il from its contcvt irr mu5t bc seen wirhin its
contrut Many social scientists believe that s d ~ q i l a t rcausal cxpl;lnntic>ns
neccl lakt1 account nf the meaning that partrcip~ntsin a social sitijr7ti0n
attributr to t h d r b ~ h a \ ' r m ~Thev
r. n r p e that tlir meaning of br.lia~.iour
helps 11.; 'makc sense' of n.hy cTnp event pmducir particular outcomes.
Thcrcforcb when conductirl~caw studips it i q useful tr) ~ o l l ~ t
informaticrn about the <utljccti\-e meaning rrf brh;lvinur for pnrticiyantr;
and build this subjechve datil into our analvsiq of the case.
Analysis in descriptive case studies
C a w dta.;cr~pttonss h n ~ ~ lbv
t l structured around theoretical idt*as. 'T'hcre
are <rhohrs, however, w h o ,irgue that castx dcwriptinns shoi~ld~ i n i p l v
dc~csiht>thr case and not \trek tn interpret it. Thrs view argllcs thnt we
should lct t t ~ e'facts' 5pc;lk f t ~ rth~rnwlvcsa n d a\.ntd lqpoqing our own
~ntcrpr~,t.llicinnn 'the t.ict\'. %tch an approach i.; both u n d r + ~ r ~ band
l~-
impcrssiblc. Any dcscsiption nl J I I V cilw ~ I W H V S
inv~rlvcs a ~rlrcfrl~rs (IF
facts. Ihih .;clract~ou will bc bnscrl 011 what wr ~;vL- a s relevant and
important T h i ~7 tery act of .;~It~rtinq nlcans that tvil art1 niaking decisions
about wl~iltI $ rtllcr-ant. This s~*lrctic~n process tvill lllr I ~ c a v ~ Iintlt~cnctbd
y
by o u r ~ r n p l i c ~theories.
t Fi~rthrrmore,anr- mporlinq of a cast. W I ~ I
involve onitbrrll,' tlic selected fact+.'T'hc ~ n e \ . i ~ a b.;c*lrct
l e ~ v i hand
. ordering
will mcan tlrat all descriptions arc. our .rl~scripticm.-.,r~lther t h ~ ntlrt,
description OC tlir case.
The incvi tahlr strbjective elcmcnt o f case s hrdv nnalvsis does not mcan
that such an<~lv<is I < thercforc nut rr.c>rth~vhil~. Wl~rlti t ~413~s mean is that
\r7e.iho~ilcEbl* r*\plicit abntat thr hn\it nn ~ c h i c h~ v c .a r c c o n s t n z c h n ~l h ~ =
description.
One \vav conktructing ric..;criptiun r* tn zl5c rtlc,il t \ yt.5 ((see Chnptrr
13).This i n v o l r t ~conqtrucfing
~ tlicc>rcticallyJrrivrd Jcscriphcms o f h ' p ~ s
of cases. On thv b ~ > irlf s thct)n,trca~lrtvr<idcrahnn* , ~ n o~t l~ Ln~lorvlcJ~c
r nf
tl?e exiqting rtlwarcli I~ternt~rrc M'C m n l i ~ the t clcnli*nh OF a type ( p . ~an ,
authorilnrinn pcrson, a perfectly l~urcaucraticorg~nixalion, n democratic
~Eectclralqvqtrm, an cga1itari;ln m a r r i a g ~ ) .
The anal~rtics t r ~ t e g yis tn u s r this icjcla1 type as a t~v-nplateti1 5uitJt' tlir
a n a l y q i ~n l at1 actual case. Wc can list the templntc. tn srp how cln<cly nur
actual case fit.; the template. M'c. might have good reasons for eupecting
that p a r t i c ~ l ~ciar w s will reyrescmt the characteristic.: of a particular Vpv.
For exarnplr, wc might cxprct, for various rea<ons, that a government
departrncnl in ,I Weqtrninst~rform uf government might bc close to a11
ideal type of n bt~reaucracya'i delincd b y Wcbcr {Ccrth and Mills, 1Y4h).
Using o u r trrnplnte rrf an deal type bureaucracv wu can examine n
selected d r p ~ r t m m (case) t to WT 1 1 1 1 ~CIOSPI~
' i t appr(~xirnnte<the ideal
Fpe. AS ~vt)ll;li; ~dcntifiinf:rvhrrc the case !its t ! I ~ C , I ! rve would ~ l w
use the ternplatr to hclp h i ~ h l l ~ where ht the c a w dtvcsgi- from the ideal.
Rcpeated ~ ~ 1 qtudieq s t ~ { e . n~r ~ l n g cof government dl-pastrnents) might
.;how uq thnt empirical ekamplc.; ~+cinsi<tently divcrgi>from the i d r a l in
st) far as mini-lcrial intrrirtanlion.: might nvtxrridr, tlic ideal rrf rulra-
~ o v e r n e dcfcci<inn r n , ~ k i nwmliich
~ i:. a charactpri.;tic of thc idcal tvyr
L~ureaucr;lc7*.I t qhotrld be r ~ m r m b r r r d Rorrci.cr,
, Ihat the ideal type .tnTI
qtands a < an ii1tnaT t?pe r e ~ a r i i l t , - of t h ernpiric~l
~ tiiirlinp from J %rtnf
cases. Thr ~ d ~ +t al ;l p rernd~n..;r 'purr' ideall7ed , ~ h \ t r , ~ c t ~ c ~ n .
The uw nT tht* Idea1 type prtrv~cfic<a rvay nf Ir>tiL.ing a t and o r p n i n n ~ :
the annlysis Ft~rthe cfescript~onso f actual cast>$. U.;lng this appmacli wnx
avoid dc'icriptinn that simply dcrcribcq w h a t ~ v wtb ~ r l~appento tind irl~t
about thc c ~ s co r sirnplv reports the features t h , ~ tcatrlz our attention.
Instead, the dt..;cripticw i c < t t ~ t r t i ~ r c dtanned
, nnd p l ~ r p o s t h ~ l .

252 CASE STUDY DFSIGNS
A typologv consists of a set of types. Thus we might develop typologics
of ditferent sorb of personality, forms of government, tvpcs of organ-
izational structure or types of marriage. This set of types may he based
on a set of ideal types or he derived empirically (see Chaptcr 13).Typo-
logies and ideal types provide cxccllent methods of analysis in which the
'wholeness' uf n case is preserved. Rather than just concentrating on
traits, rt typology attempts to build up an overall picture of a case taking
a wide range of characteristics into accnunt.
Where we ernplc~ya theoretically derived typology the task of analysis
is simply an extension of the ideal type analysis described abovc.
Where we use our case studies to derive empirically grounded typo-
logies ( i t . those based on our actual cascs) we nccd to adopt a different
appronch to analysis. These approaches are only possible if we have
multiple cases.
Clrrstrr n l ~ n l ~ t t ~pprcmches
lc Tt~cseapproaches involve identifying a sct of
variables we want to use as the basis for our typology. POT example, in a
study of an organization we might have collected from i w r case studies
information abntrt the way deciqions arc made, the way in which rules
a n d rcgulahons are used, the degree of h~erarchy,dcgrec of rationality
rtc. rn a set nf case studies wc could then group case5 that had similar
constellations of chardcteristics.
I f we have a small number of cases that we know well we can do [his
manually. I f we have a larger number of cases we can use a statistical
technique called cluster analysis to a c h i ~ v cLlic same end. Eqsentially,
this technique involves identifying the characteristics on which we want
the typology to be based, coding cach case in terms nf cach of thnse
characteristics, and then feeding that information into a clustcr analysis
computer program. The program will then identify which cases are mnst
,similar to one another. It will cluster cases into gruugs that have
'relati~belysimilar sets of characteristics.
Ratlicr than presenting cases in terms of types we can represent a case as
a history of some sort or another. Tf we are dcaling with individuals the
case description might be prcscnltcd as a biography - an account of a life.
The biogmphy will necessarily be selective and will represent thc persun
in a particular light. It i5 likely that the biography will go beyond sin~plc
description and will include either implicit or expliclt thcorirs of why the
perscm's life went in particular directions.
Histories of events, nrganizat~ons,policies, or whatever thc unit 06
analysis might be, reprcqent a way of describing a case whcrc the
emphasis is on the sequencr of events. The sequential a'ccount may seek
Ln cx\dnin the course nf events and may bc d e s i ~ n e dto support a
CASE STUDY ANALYSIS
particular explanation. Altcrnativcfy, it may simply bc designed to pro-
vide a record of events without an cxpl~citattempt to explain or inter-rct
those events.
Explanatory case studies
As in descriptive analysis, in explanatory analysis theory plays a key
role. Thcorctical propositions will either direct the analysis or bc the goal
of ~nalysis.
When multiple case studies are being conductecl the aim of the first
stcp in the anaIyqis is to understand each cast. 11s 11 alllulr. Only after a
picturc of each case has bccn constructed is i t appropriate to ennipare the
cases (Yin, 1989; Stake, 1994). The analysis of each case must be organ-
ized around questions and theoretical coi~cepts.Each case should be
used either to test or to build th~oreticalpropositions. When each case is
ana tysed aro~~ncl a co~nrnonframework it is thcn possible to comparc the
cascs to arrive a t higher levd generalizations, or to provide tougher tests
nt thpnries. If cach caw is evclrnir~edin an idiusyncratic way then i t will
be very difficult to compare cascs meaningfully.
Thr way in which we approach the analysis of rxplanrrtury case
studies will vary somewhat depending on whcther we are using them
for theory testing or theory building.
Thcrc am a number of ways of using case studies when testing theor-
etical propositions. Kin (1989) outlines two approachcs: pattern matching
and time series analysis. In my view time series analysis is simply a y p e
ol pattern matching but in the discussion below r wit1 treat them
separately.
Pattern matching is a form of theory testing analysis that esbblishes a
dctailcd set of predictions before the cnsc study is conducted. These
prediction5 stem From a theoretical model and thercfnrc represent a clear
theory besting approach. Thc analysis could proceed by e5tabIishing a .Get
of altprnative pattern4 we wnuld predict on the basls of c o m p ~ t i n g
tIleorics.
Thr form and cornplcxiiy of pattern matching can vary considerably.
The basic principle is that t l ~ emore elaborate the predicted pattern (so
long as it still follows logically from the theory) the tnughcr the test of a
theory. Having predicted a particular pattern we would {hen conduct the
case study to see if the case does, in fact, match the predicted pattern. If
the case matches the preclicted pattcm then the casc supports the thcorl~

Independent variable
I
Dependent variable
Yh
in thc same way that a successfuI experiment supports a thenry. If,
however, the caw docs not match the predicted pattern the theory
requires modification.
Yin (1989) describes pattern matching that varies in complexity
according to the number of independent and dependent variables
included in the predicted patterns.
SinipEi, pntfer~ls At its simplest level pattern matching involves one
independent variable with huo values (e.g. male and female) and one
dependent variable with two possible values (bchnvus in one of two
ways in a particular situation). In this case there are four possible difier-
cnt patterns (Figure 15.1). For a givcn case with a given d~aracteristicwe
could predict nnt of hvo patterns. Which pattern we predicted would
depend on theoretical considcrations.
The pattern wc might predict for a particular case that we know has
value X, on the indepcnd~ntvariable is that the case will have value Yb
on [he dependent (outcome) variable. The prediction takes the form: if X,
then Yb.
For example, our theory of work perfomancc in schools might lead us
to expect that when staffing appointments, dismissals and promotions
* are managed at the local school level rather than by a highly bureau-
cratic, centraliled and remotr system (X variable), staff teaching com-
mitment ( Y ) will be high. If devolved staffing systcrns are symbolized as
X, and high level of teacher commitment is symbolized as Yb, nur
prediction is:
When X , (local based staffing system) exi4ts then Yh [high levels of tenchcr
cornm~tmcnt)w1l1 finllllow.
We would also cxpcct that when Xb (centralized system) exists then Y,
(low levels of teacher commitment) will follow.
Our test, even in this very simple examply of pattern matching. would
be more interesting and more powerful if we were tTting alternative
theories that predicted different patterns. One theory might predict
CASE STUDY ANALYSIS
Independent variable
Pattern 1 Pattern 2 Pattern 3
Dependent vanable Y, Pattem 4 Pattern 5 Pattern 6
Pattern 7 Pattern 8 Pattern 9
pattern I while a competing theory might predict pattern 3. The case
study would enable us to see which pattern emerged and therefore
which of the rival theories was supported in this case.
For examplc wc might propose the following two rival theories:
Thtvry A !,oral control will lead to higher commitment to work
bccausc effort and 'fit' is swn and rewarded (and lack of effort and
not fitting in with school needs is punished).
a Theory R 1,ocal co~itrolmakes people feel more demoralized, and
vr~lncrahlcto local politics and prejudices, and dues not recognize
wider professional development etc. This leads to a lack of commit-
ment and a lack of professinnalism and to plaving politics to win
Favour rather than fostering performance.
A slightly more complex version of ~ssentiallythe same situation would
arise if either of the variables had more categories fe.g, medium leveIs of
performance and a hybrid of local and centralized staffing modeIs). The
more values either variable or construct possessed, the greater the
number of possible patterns we could predict. For example, if both
variabIcs had three possible values there are nine possible patterns we
could find in any particular case (Figure 15.2). Tn this situation we would
still predict a particular pattern for a givcn cast.: if they were X,we might
predict Y.,
Fnr example, we might have a theory about the impact of parenting
styles on child behaviour. Each of these two factc~rsmight havc thrcc
c a t e p i c s a s illushatcd in Figure 15.3. Using this example we might
predict that authoritarian families will produce children who are
rebellious (pattern 1) A case study of an authoritarian family will help
address whether this proposition receives support. Repeated case studies
of autl~oritarianfamihes wilI provide a firmer test. A further proposition
map be that permissive families will produce children who constantly
engage in limit testing brhaviour (pattern 9).
 Parenfal style of aulhoritv

---

ae&rl~ous -1 Pattern I I Panern 2

7 - -

Panem 3
Low 1
I
Hfqh , 101 1 Hlqh 1

The bcqt test of the theory would hc a writ-\ of c a w stud~csyredictin~

1, 2 I Ctr~ldsets I pattern , pallem Pattern
particular ypes OF child hhaviour for each parenting style nntl for r~rt.ll 0
2 I
of the rnlztuaIlv cscIusivc prcrpnqit~nnsto h c confirnlcd by thc rdevant l 7: l
own t~rnfts

c~wesltld ius.

rrrrri,rl~lr~s rt more ~.[,mpleas r l

~ /rUvr;s: ~ ~ r r r l t ~ /irilir,pr,~~rlrn!
Mclrc c t > v r / l l :pnl ~lr'
Tests ' Pattern I Patfern I Pnltnrn I P vrrr*
I
Pmnm Palloin

19 70 7 , 24
~ r fpattrrn.. can hc prcrlictc~c!if h.rmoindcpcndent rari~blcqa r p i~scd.a
diftcren t outcom t1 is prcd~cted for each a ~ ~ ~ ~ b r ~ r of ~ r ~indgendcnt
rirn
v,~riablrs(Fiprc 15.4).
Here wc might make different prediction<, depending on the corn-
bination of parcntat authc~rityspir combined with the child's leve! (if
anxiety. We might predict a diffr~rentoutcomc pattcrn of pcrmisqivc
parenting d c p c n d i n ~on whether we a r e looking at n farnil11 where thr
child IS ,inxinu< rrr s~?ct~rr/cunfident. We rniglit predict that t h anxiouq
~
child w ~ t hpermissive parents will exhibit the ' t ~ s t sparental l i r n ~ t r ' is not present we will nnt get n spcciflr outcome (see I.itt*r for ft~Ilcr
responw {patt.cn 24) while the confident /r;ccure chiFil wi tll pcrmissivc discuwion).
parents may set her own limits (pattern 17). Another approach to cnllecting and anaIvsing case studv data i i s to
We cnutd extpnd this example tu include more indryendent variabEcs lonk for cases with particular outcomes (c.g. Itiglllv corny l lank children)
and thus anticipate even more pl~sriblepnttcrns. and predict !hat thesp children wilI be anxinus and hnvc authoriiarian
Figurr 15.4 ill Listrates ihtt logit-nl possibi litie- - thr different patterns parent<. In term.; of the prediction5 in F i ~ l ~ 1r5.4 c ruc have predictctl this,
one cclttld conce~vabIyfind. Of course, our thmretkat rcasnniny: will I ~ v d but t f we find case<of f i ~ ~ hcompriant ly childr~nwho are n r ~ t h e anxious
r
u s to prtdict pnrtrcrrlar pattwns i n particul,ar circurn~tnnces,!hat is, for nor h ~ v au e thnri tarian parents thcn o ~rt t htwry 11rt.d~ refining-
particul.7r h-pcs of casw ;I< illustralrrl in tl~cs h a d ~ dcclls in Figi~rc15.4. 0 1 1 ~ .way of tc?tin~, a theoretical propns~tionis. t i i demnn\tmte. t h r o u ~ h
Harjnt: cstabli<hcd tht* pmsitrlt*pattern* ,lnJ the. prediitl-t! pattern+ c a w %tudic<,t f r d t ~vhcncertain crmditlclns apply (e.g. at~tl~ori!;lr~an
nTcurould then wck 011 t C ~ S that ~ S rnahl~ u s to test the pmprw~tions.For pmrcnting and child anxiety) tllvrc will J I bc~ ;l sprcific c~~rtcr~rnr (h~t;hly
taliarnplt., we wrluld conrtz~cta c.,isc stud\, cm a familv in wliiclt authcwi- ctlmplinnt child). The prohleni with this w r t of r n , ~ t c h i ni q~that i l only
trrrian parentint: and child anuiec occur. O u r p r e d ~ ~ + t ~I\-ot~ld
r , n be that ~ 1111 cwtcorne. It dtw\ not It-I! U G ~ v l l t ~ f h ~ ~
sptu-ifies sttfhc lr'tlf c o r l ~ f r l t ofor
the ch~li4~roulclhc v c y ctmlpl~ant.if khis proved t o be h ~ c w . e would the<c ;ire the ~)!tl~/conditinns under which the outcome occllr5. As hitch i t
hnve achieved a Iltrrnl rr,j~l~cntinrr.We m i ~ l i tthen Iwk for a casr of does ilot rcprta.;cnta rlrmandln); t ~ q ol t n propositfnn. A nmorc. dpnl<~t1rl-
authnritasian parrmntln:: ~ ' l t ha cliilil I \ not an~iou5.Our predrcfitm ing tc+t tvnulrl involve Innkinq Inr caws !rehere tlii* particular outcrvnt3 is;
here mij:lit be that the chitd twirl not bc c n m p l i a n t ~This would hc a prrwnt and tlien a5king whrthrr thiq cltitcomc rlirl!, mcilrs unrft*r the
tlrc*orr'irti~lrrgplirr~lrnrr.TIl.lt is, whcrv tl-re prticular combinatinn of factnss khcorc*tically prcdictcd ci~nditions.
 IndependPnT vnnables Independen! vanahles

--
---r 11

I
Condlt~on1 Condition 2

' Values of dspendenl vanables

I H~ghlevel of worker
auTonomy
Hkqh degree d dlrect control
over worker

Work
l qual~ty

Staff

A further test nf n propositinn is whcthrr nPc.find cases whcrc all the

condition.; are met hut tht. outcome does rrnf occur. For cxnrnplt., d r ~we
find c a w s of authoritarian parenting and anxious childrrn where
childrcn are not compliant? IF we do ihcn clitr proposition is not. sup-
ported and our theory will n w d to be revised to accommodatt this tvpe
of cart*.
Alone ca~~rldrr p t t e r ~ t sr r r ~ r l f i y !d~~ y ~ n c i P t rnrmblm-
rf The prcvintas section
focu.;t%don situations in which we prcdict a ~ i n g l eoutcomc variabIe
undrr a pcrticular srt of ctlnditions (rnultip lc independent variables). We
can tanvisage situations wlirrc the oppt>s~tc applics - wherc wc have a
qlx\cif i c cunclition (onc indcpcndent vnrrnblc) and predict that Iliis cun-
dititrn wiF1 producc a particular set. of nutcnmcs. T h a t is, we rvill yrcd ict a
p f l l l r . ~ r fo f or~tcomcsl~nclrra ),ileen cnnditinn (Fiqurr I5+.5), 12'c ivrll then
match that prcd~ctc~l pattern ~ v i t hcase5 that mect the condition.
follow in^ characteristics on the fivc outcome variables (A, B, C, D, E): A I,
H7, C , , LI7, El. For cases that m r t t condition 2 wr predict a diffc*rc*nt
pattern c ~ fc,utcornes: A t , R1,Cq, L& and E l .
Thi%cnn I 7 r illushated rvi th an cxarnple ( F i ~ u r c1 5.h). Ttie theory h i n g
used hew is one that prcd~ctsthat autonomy lends to better outcomrss
than h ~ g hIcvrls of supen~isiun.While there may bc nn payoff in terms of
cfficien~y,worker autonomy hns pusitive hcn~fit\ in terms of rnoralc,
work ct~ntnitment,~nitiativrand improved work quality.
I n (II~PI' words wc predict that c~rganizations ( c n w s ) that permit high
Ic~relsrrf nit tc~nomyarncln): mcrnbcrs of tlicir wc*rkfrrrce will d isplnv a
parbcular 1rr7ifr~rrtof outconlc.; hie11 marale + 111~17qlinllh' of worI\ + IIIW
~ t a fkirnnxbcr
i + modetalc Ir\.cl<(jt efhaenu?~ A h!gh le\-cis of innov;~!~nn.

firrr EVP prcd~cta pattrrn of outcomtXsin c a w < that meet condition 1
ancl n cliffwent pnttcrn tnr cases that mcrt condition 2. Thc pnttcm is a
p~rticitlarrornI~i?rnfifl)lr b l ~>l~l.comes. In t l i i q c a w wc prcdict t h ~ tfor cases
rnr.rat~tigcondition 1 (on the independrnt vrrriabld) we will f ~ n d the
0rg~ni;tatirlnz u-ith high dc~rr*csof supewl+lr,n and direction arc
p ~ e ICICCJ
d to bchave differrnttv. fhcy are prcdic?r=dto have a drffcrrnt < t a t
of outcolnr chnrrlct~ristics:Pow morale + low qualrty of work +high staft
turnover + moderate Ievelc nC cfficitncy + low Fcvcnls of ~nnovatiun.
 1561) CASE STUl)\r 1 )I!SIC;N';

H I7 1 t I ! Tlic mo<t rtlmpfcr x.cn~cjnt i t p t t t 8 m Trend nnalv..!c ~ t ~ be l d far n l ~ r tctirnplr\

m a t c h ~ nr \ ~ill ( x - t l ~n.hc.n
r tvc 11ai.v a i t ' ! of ~r~dtxpcndent
i r ~ tof dcpcndrnt ~.ariableq.In this <rtllnrlon rtotQ rndy bc a b h tcl. prc~lrut
that ~i.llcn c a w has a yartic11l;lr GI,! o f c ' . J ~ Iattr~butes s~~
matched ko a pnrticlilar srr of outcornr.;.
For cxampIe, wc might predict that whcn an organization is fairly
<mall, and pc~<t\t..;~c.s ,I ' f l a t ' org;lni;.nt l(rnt1l s l r u r t u r ~ ,r r s p ~ c l c drons~ll-
tJtixre mechnni.;m.; and workcr rcwnrd pliln<, then there will hc a vn1ur.d
set nf outcornt~~. Thc.;c might be high r\,ork~rcnmmitmmt, tx\iclIent
qiialit!' o u p u t , +lrt*riqtic.\ betryeen ~\,orkrv-\t~ut<irlcthe rcrrrLplnct~,rc.,~iFv
acucptance vt innrrr.mitic-m,low level.: 0 1 a!~v~ntceisrn and n ~ )lntlu\tr~;tl
ii~<pi~taticln.
I t i St r1ciIy speaking tht.ory t c ~ t i n ginvc)lvcs specify 1iig a
lthctjretical prc>pn<ition, t r s t i n ~i t and i+itlicr ncccpting or r t > ~ r c t i nthe
~,
hlnvevcr, i f a propt,\itii~n I < not sl~pyt,rtt=ilhv ;l
p r ~ p o ~ i t ~Ino nrt~~ilih+,
(-n.;t+ stud! tFirn tht. 11Fht qtep is to rlliint, f l ~ ctlicon. so tlia t t t can ~ a k p
aiiount of the, t~kcrptiony r o \ . i d ~ i jtr, fhr c,I*e. In this iw\. tht. Iv+r>pt>kl-
tlcw cover< a tv~dt,r,ind ~ v l d t 5ct r c r f c,v.c. .intf I~ect~meq rnllna~wnvt*r!til
This psocrsc is callt~frrrlndtfrc i ~ r t l ~ ~ r iand
r r ) ~will
~ be outl1nc.d mnrt. tttl.lr.
Iatcr in the s c ~ t i n n111at ~Ical::with tlicnory tl~tildrng.
TIMESERTES ANAl.I'SI5
:llthough Yin (1WQ)treats this methikt c>I analj.qi.i as qcyarato Iriini thy
pattern matching dl-ccribcd dl-ovc, thc POSIC iz Identical and i t % h ( t ~ ~ l d
tl~ereforer;implv hc treated a s a fomi ni p t t r m matching. Inqkrad nf
predicting a particirl~rp;lttt1m for a ~ r otf v.~riabIesit prctlist.; n p.irt1-
cular trend or ~ q u r n c t ,of event5 The* nn.ilytic strategy ~ n i , o l v rprc-
~
cl~ctfnga p a r t i ~ u l ~pattern
~r of c h a n ~ cnvcmrtime. This type OF p a t l c r n
annlysiis can takt\ rrnr of two forms: trrjfrl an,llvsis and cfrrorrt)lt!~~inrl
(cl\,t>nt
~t7rluencc)aiinlv.;~~,
~ R a t h e r thein . ~ n i ~ ~ I v
1.arinblt.5 and a y r t d ~ r t i na ~rliftt.ri-nt t r t d tin the wrnc vnrvalylc tor rllftPrpnt i . ~ > t - 5 .,I
rcm~plexvrr5inn r l t ~rtur~d analys~swoul~lprta~ltct trend< tr>r a ~t ( i t
tht'v tvill iie t.ciriablt>:: in any g i v m r.15e. For exarnplc, w r might prrcltct t'ti,it In a
gixfcn cmrrse tllc cnrnlmvnts will incr~nsr,tht* ratio of females to ninli,.;
will improve with thr* increased cnrnlrnrnt Icvcls, there rvill t-c an
inrrtwsc in rnattlrc aEp sturlrnts, and t l i p r r wilt t ~ cnn rcdiictinn in thc
q ~ ~ ~ ~ofl studcilts.
itv In azintlier cnursc wc might p r c d ~ c nt dlftcrcnt corn-
tvnatinn of trend<. ,I <tc,ldv enrolment, a rlrcljnc in qtandardz, n ~lvclint?
in mature agc ctudrntc, ,mcI fewer fcrnalc stl~llcnts.
Trend analyst-\ can b t cithcr ~ ~ l t c r r l r l ~ lrir Intrrr~~lltt.~i
r - ~rrrtrr~~~*rr~r~~fr.i1.
t
time series ;~nalvsisrtafpr.;to a situation rvficrr ,I \pcrific event tL1h.cpl,~cr'
.;onrcwher~within the srqrlcnce of evrnts. This rnahles u s tv cxnrni nc. t hr
p a t i e r n of cvcnts hratrlrr>and after the intrrri~ptinil(or intcrvcntron). It1
thiq respect an interruptrd sime serics analysis I < ~ i m i l a rto n bt*lr>rt*and
,iftcr design. Althi1~1~;11i f dt,cs not invthvtb a control group, tvv wtluFcI
cclnduct stuclics ot c a w \ tt~herethc intPrrlipticirl did , ~ n ddid riot t,rkr%
I ~ l , i c c\I'hilr tvta c.>~rnritcnntrol for r r t l ~ c rJr il~arrncc~ bctrrccn c.l.tn.
collect dcta~lracl~tllorrnattnnabout all cn\c%\ to mnble uq to c o i i ~ ~ < l t ~ r
M t ~ c t h r rthesc c7lllt.r r i l l tr,rcwces Irathcr tki,~n Iht. ~nterruytit%n) rnl<lll
rt.spcm.;ilde for [Ilk* rrllp,ircnt rffect nf thc ~ntc~rrupt~r~n.
Ititerruptcd tsrrlci .~n,lly.;iq u,ould invnlvc l ~ r t d i c t ~ na gprr.2 al~cl1 ~ + 1 -
inlrrrupkioil pL?ltCrn(r.g. dccline prc and lncrcahe post, fcrlloivctl I>\'
longer term s t ~ h i l i t v and
) matching t h i ~prcdictrd pattern with th,lt fnr
actual cascq.
Not ajl trend qtudic< w ~ l lhave n n intr*rrliption. %me will s i n l p l ~ .
prpdict a trend t h a t $\+ill be anticipated in .I yartlcular ctlnh3ut l or
cwrnple, we r n i ~ h tanttcipste from what IVP knaw about lht. rit.rnt3-
~ r a p h r cand ernnnmic prtifile of a r ~ g i o nIthc* case) that it 1.; FiLc*l). to
t.xperience l u n ~ term pc~pulakinndccl~ne.Ano1llc.r region with n rl I ft~>n>nf
inix of denrogmyhic ,inti ectmomic chamctcri.;tic~ mrght be prrtlictcti to
c x y ~ r i c n c cgrowth w h ~ yet
l ~ another rvgion with a different prtjfilr
might be pred ictcd to stasnate.

7rt.rtrl nrmlrr~rs 7'rrnR analysis i< a n cuaniin~tinn of thc directinn nf Cl;rtvmlii~yicnl nrtnli/rl< Chrnnolo~ical ,analv\i< involves prt=d~t-tinl: n
c!i,lnpc in partirufar 1-ariable or w t nf \.,lrlablc.;. Mre addrr- I ~ I rllic-q-
?
L * of e ~ ~ c n tI qor rl-en a sequencc nt trt'ndq) in\.ol\ins a n?irn1.rzr
.;r7r!~trvi~t-~7
trrvn ol rvhctlwr tlic Irrsnrl i< i~pr\,ard(htt,cp tIr gr,-1ilu.71),shot\'* r l r ) ch,tncc. (if dlitcrcnt evtant.; or i,,iriL~blcs That is, \\T w c ~ l r lprcdict ~ r . h , ~t \ tr~i!lrf
variable ( u p ant? clo\+,n) or drrwnwar~l(htc~bptrr graduall. clinngc, nt what t-vr-litc \teould take plactl, in w h ~ turclttr, 111~1
Prcd icted trc*nrl.;c;lii rangy f r u n ~tlic srrrrplt, t c ~the higlily t - o t ! r / l f r . ~ .TFir .;txqllcancenf evcwt* might bc a corrst7nrrrl rbfl'fTc.r srqucnct. nr it rnrght Ilc ,I
simplest fnrnlr p n d i c l a trend in tlnc ili*pc~nJrritvari;lblt~.Wta ~ n ~ i : thrn l~l J c s c r i p t i r ~S~P L ~ ~ I I V ~ C -tl1,lt
L' proposes partic~rlnv<tr7"yrS~in a prorr%+
prcd ict a d it'ferviit par lcrn of change t ) i i t311s\,a la tllc in dificrcnl crlnttlrtq. l'xnrnplcs of n s t . ~ ~ vcvioii r~l of chrr)nt~loj:lrill;111,11~si<
might 111, rm~t1~,lc
Ior e ~ a m p l c ,w r nrkli-anticipate that cnrt~lmrr)t< in a partluttlar t v p , c j f llrat prcrpmc predirtnhlc Gtazeq in b t ~ o n i ~ r l ap , marijuana w r r t l < t - ~ - L c ~ r ,
tli111-erqih.cotlr+LaIthta c a w ) ~ v i l l irwrcn.;c
. In ~lirchnrF tern hut d+-c-l~rlc in ICkhhk.~ t d q e =~n t l l t a rlrvntpgsatinn vf int !mato rc~latiniwhiy.: (\'auqhn.
f h c Imger t t = m ~rlnc,tht*r course m ~ g h t ~ r ~ l d ~ cto t umarntaln
d ~tabtc I W h ) , stages in thv procrw ot adiustmcnt to rt-tlrcrn~nt(Xtchltav. TLr7rlh,cbr
en rdrnent !cvt4c ~ r ~ l ~,~tir~thcr
ilc tteprc ) C r turr+r i c prcdictkd t o txtyrar!tlrlcc chnngt+ in the rcl=ltton.;hip~ beh\.een ,~rlultsavJ thcir pnrcntk .I\ pEirtant+
'J-l.irp Incrcaqeq in [.ti rt~/nientIc\.cls. ilKe l Massdcn ant1 1 2 t ~ a m <1947). ,
2hZ CAST: STUDY IIPSIGIGS

R r g ~ r dlcb~s ot whether thC analvsi.; involvec a causc and effcct nnt find case% ~vht'teMT 1?,1\.~'thc c f t t ~ t swifhclt~tour prt>\t~rned
aeqi~wicvof evenis, or a set c l t predictcd stages, the analytic q u ~ s t i o ni.; cause."If we ,foil tn find any huch caws we havc achieved lurther
wr!l~fh~.r t l ~ cpredictcd p n l l ~ ~ l 'rlfclra71,iy
h occurs in the casts w h ~ r etheory thcnret~cslrepl~cntic~n.
rvt3uld rxpcct it to occur. Or does t h c pattern nccur rvht~nw e would
predict it .;hould ttrrr c ~ m r ?
T l i e Corms of the* predicted patterns can vary. Yin (1989) indicates four Analysis for theory building: analytic induction
tvpt's of w;iy5 in wli~c-lrevent< might br predicted ttl change in r ~ l a t i o nin
~ a c hntl~rr.We can predict chmnolo~ie.; in which: At ib heart, analytic induction ic 'a s t r a k c ~ vof analvsis that dircrts the
~ apply to all instances ot
inrtrs tigator to iorrnulatc g e n ~ r a l i x a t i o nthat
.ir>mc must always occur ht-fore othrr cv~nts,with tlic ntvrr<cb tl1c1 prckblc~n' (Denzin, 1978: 191). It is a rncthod that can bc 11st.d to
+,rf~/r,r~~t' bt.~ncimpmsibl~.. rirliicve d c ~ c r i p h v rgmcrali7ation.; o r tn arrl1.e at ca~rcal explanatinn5. I t
wmtx event? nitlst ahvnvx h t followrd bv other r\ ents, on n m n t i r r ~ r . ~ ~ ~ t / ;.E n q t r a t c p that move< from i n d ~ v ~ d ucaal w s and ~ c h tos identifv what
t,n51.,
tlic cases havc in common. Thc common talerncnt prclvidcs tht- basis ot
v somc events can only follow other evpntp after n spccifled pt.;qn,~t+nf timt*;nr
tli~oretica2~eneraliz~ition (see Iyiguru 1.2).
rrrtain f i n i l * / r 7 r l m lIn a cnsP study mav be marked I-y clawe.; t > f events that
diftrr ~ u b = , t a n t ~ a l from
lr t h w e of u t h r r time prnani.. [stages]. ( 1 999: 17'3)
Analytic inductinn Y G a stTafPg~that s r ~ k . :In arrive a t genrral~zationq
that appZv tn d l ca<i3s,bn t h i ~rr.+pect it tI4ffers from the s t r a t r g i e esf
nri.~lysisdisc~~ssed In previous wctions of this hook. In Chapter 3 1
dislinfiii<htd hetwecn deterministic and prnbahilistic notion5 of causn-
I n Chaptt*r 13 1 n r g u ~ dthar gi>neraIizin): from caw ~tudie*;rrdit.s on t h r t~rm.rJw I T I I > ~ V Gof ilrldl\-if: ~ I ~ S C L in S ~ ' ~ wctions a w bawd on
I Cprcviouq
repiication logic 01 cxperimrnts rather than the .;tatistical 317gic of sur- p n ~ b a b i l ~ ~causat~c,n:
tii r4.e cst~rnatc1~1irtht.rone group is mclrc Zikely
veys. M'r gain mntidcncr in e ~ p ~ r l r n v n t arcsu
l It? not ji~st from thc tll.in 0 t h ~~T ~O Y P S to I ~ t h d v cin a part~culnrway. Analytic ind~~ction,
elegai~cilof the cxpcriment but from our capacity to predictably replica tc hnwcuer, st.ckq to achirvc univprsal gencr;lli;lations.
! replicate results (i.c. mr anticipate that
r ~ s u l band to pwtlictably f ~ i to Denzin ( l Q i R : I i ) 3 sunimnri7cs c i kc!~ ~tcpqin thr process of analytic
the intcmcnhon wll ha1.e its effect ttnder spcci fic conditirms hut not indirction:
under other conditions).
Similarly we gain confidpncc in case study findings wlicn we can 1 Specify what it is you are s c r k i n ~to explain (the d ~ p e n d c n variable).
t
accuratclv predict which k p c s of caw.; rvill d i ~ p l a vparticular patterns 2 Formulatr an initial and prc~\,isionalpc>r;sible c\planatinn of thc
and ~vhictrcasee will ~ i o fd~\plays ~ ~ f fpatternq.
ic phentln~rnnnvou are seekin< to explain (your thcory).
Where. we have a predfcterl .wt of outctxnes and a singIe causal factor a 3 Conduct a shtdy of a caw scnl~lctedto tcst your thi~ory.
similar logic holds. We would ask the following <or% of qu~stions: 4 Review (and revisv i f necessary) your pro~isionilltheory in the Iight
of the caw o r exclude thc casc as tnap~mpriatr.
1 DOCSthe full set of outcomc charactrristics occur when thr prcsurneJ 5 Conduct further case studiw 10 teqt the (seviwd) propositinn and
c a u w 1 factor is present? IT so we have confirmation of our theory. reformulate the proposition as requirrcl,
7 Mlc wc~uldthen find another case where the prr.;urned causal factor i~ h Continuc with caw studies ( i n c l u d i n ~l m k i n ~for cases thal might
PPPSIVI~ and stc whether the full wt of outcomtSs is also prt-sent in that disprnvc the proposition) nnd revl\t. the prnposition until you
cart1. Zi so rvc h a v e a litcral replic~tionof thc previntrs case a n d ~chievcaa causal proposition that accounts for all the casrs.
furtht-r confirma tion uf o u r theory.
3 D{JwrJ get caw< whew the presumcrf causal factor is prcscnt but nnlv 'nie prncr,w might bc illuatr,ltt.*d uqinc the hvp.~"th~tical eiampIe in
w r r r . of the pr~bdictedcrutcime characteristic+ are preq~nt7t f we F~nd Cftr7ptt'r 13 about tlita impact (rn the qtralin' of c d ~ ~ c a t i nr)!n d ~ \ ~ c > l v c d
surli I-ases t h r n rc-e hn1.c Ialled to rraplicate thr throry and we would \.rr.;ur; ccntralizcd sclionl st,^ tfin): sy5tcrn5.
c11hcr rqect or modify tlic theory. I F wr could find no c ~ s c swhere thc In this c*xample tlic quality ot ~ d u c n t i o nis the pl~mnrncnonwe arc
c a u $*.a<~ prcwnt and the full qet of nutcornrs was rrrrt present tlicn wrking tcl csplai~i(.;try 1). O~rrprnvisinnal (partial) cxpfanatinn iq t h a t
I\.(, hazre a t hccrrchcal rt>rlication. ~Fr~,nI\-cd .;tnffing s\..;tems 1 ~ 1 1 1~7rnducvgreater i r n p r o w ~ ~inn thr ~~
4 1Z'r rvould thvn seek to dtnd a caw in which the presumed cai~\al ~ U J I iry o f t.rIlrcaZ-ton in ,I schoo[ Ihan will ~~t~r-rt~~lizcrl
.;v~tern$(%trap 2 ) . 1%'~
factor ES !lo! prtxscnt. Wi* would rxpcct that thc fullI set of outcc~mrs would thprcforc anticipate that cases [st-hnols) w i tli devolvtd +ystcms
~ i l u l dnot 01-stir whet1 the, catlsu M3,r.. not prc-rcnt T h a t i.;, we shn~rld ~vouFclhn1.1. bcttrr ~ ~ u ; leducaticmli~ tllnn thore W Ith ccnSrali7rif svstcm.:
 ( ( ~ t h c rt h ~ n ~k si n g e q ~ l a l ) ./\ltcrn,~hvt~Fv,we might anticipntc that
schools th;lt intrcduced a dc\,ol\,cd st,~ffrngsystr-rn wcluld exhibit an
lmprcwemcnt In thr qlralrty c ~ feducatitm and thnt this improvcmcnr
M*LIUICI b~ greater than in schools that did not intmducr such a ~ s t e m .
WP w o i ~Jl thtln sclcct CI caw to tcst our prol.713~1tinn(~tt7p3), W r might
find a schcml that ha< rect~ntl!. ~ntrodlsceda Im-alfy hilsed staffing .;vstenl.
Having d c v c l o p ~ da detinition of what c o n s t i t u t ~ seducational quality
and worked ntEt horv tn measure thiq, we could then conduct a caqr
rtud v to src if the introd i~ctionof the ncw staffing systrm had led to tl~r
prcdtcted Irnprtm.crnentk. Let tts auppi).~ that thc prrndictclri improvc-
nients had rn fact tnkcn place. This lends support to our pwpositicm hut
the support is t~ardlvovc*nuhclming and probably wonld no1 convince a
.;ceptic (stcp 4) Thc sceptic ct~uldsav that one case hardlv proIVrsthe
point, that there rndy have h r ~ n a gencra! Irnprtlvclm~.ntin c*ducationrrl
qualitv in all --chonl?: over thc sarnt. prriod, and that qchonls that had
kept the old system m i ~ h thave also exhitlitcd an improvement in
c d u c a t i ~ n qzlalih*.
~~l At tl~c*vcrv Pen\t thcy m r ~ h .;,iv t that thrv c~>uld
a schnol with a ct*ntr,ll svrtcm Illat had t m p n ~ v c d thcir qu,lIity of
cld~iiahonOVPT the <amc perltld. They might a k t , itg gut^ th;lt t h c ~c't7~tltl
find cxan~plcsof schnnl+ whcre the lot-al .;v<tem ruaq introducrd w ~ t h
d isnqtrnus conqequcnces for cducarional qua lie.
Wc wnuld ntrc=d to d n more work - further casc st~ldtes(step 5 ) .
Initially we rn~ghtlook for furthcr casw o f n r w t y ~ntrt>duccdTordl
staffing svstems to check that the initial casc studv was not ju4t good
luck.
Let us imagine that we then ccrme acmss a qchool thnt o u r critics
a l l i t d d tci - one wherr the ~ n t r o d i ~ c t i oofn the ncw .;=tern had heen a
disaster. I t had led to tcachcr, sturient , ~ n dparent disratisfaction, I o s ~of
morale atrd a ctear decline in many aspccts of educational qt~altty.How
do we make scnsr of this caw? What is dilfert-nt about this case that
shouId makc it behave d i f f ~ r m t l yfrom that which w r predicted pravi-
sionnlly, and so diffcrentlv from t h e other caw< wc havr alrcndv 5tudicd7
Since our study should protluce a compl~trpicture OF each casc and its
f o c i ~ and
l rultrrml: context we rvill prubahlv pick up some initial clues.
How is the 'drviant' sd~ooldifferent from the others? W c might notice
~ m fvaturt.q c ahnut thc pnlcrs.; by \vhlch the ~ l e v n l \ . ~scheme d
~ntroducc*rlin the school in which it ~ 7 ~ a1 sInilurc. Fur cxampIe, tcachrrs
report t h ~ rtt svas impn.;tld nn thcm Tlruy wPrr not c(>nsultcdand thrre
arc nrr p r o c r s r s 01 appcal a~ain.;! thc principal'.; decrsions. O u r linrnch
thcn 15 t31;11 t h r success of thrs type of ~chcrncdepends cln thr waV in
tvhich it i q irnplemrqtp~-l
M'e tllcn Inok back s t our cnrlier caws to qcr i f this c ~ ~ l a i nt ls~ c
diftcrent outcnrnc.; Dn the s~~ccc..;.;ful cnseq ,tan4 11uta.: hal47ngdiffcr~nt
tmplc.mt3ntationpsnccsw.; tn the clisa<trnu<ctlse? I-et 11.; s l p p o s e that the
s u r c e s ~ f u lc~weq irnplr*m~nt~,d t h r scheme nnlv affcr 3 great deal ot
.-?"-' fl'tltlc~n ,lnlj qlrr~rt-! \ . i s w~drl+~,rcn~l cnn<lbn.;llrnl,nut tht3 c l c ~ ~ r ~ b l l v h
of thc ~ i l i c m e thc process bv which i t \\lac tn c > p r r a t ~M'c
~ wo~tlrithcn
Ilr in a position (step 6 ) tcr mollify o u r initinl, pro\,~sionalprt>powtion ti)
read ~ o m c t h i n xI ~ k r :'Whtn ~mplcrnenfcdin a clirnatil of con.;ul tatit~n
and ctmsrnsus, de\.nlr,ed staffing svqtcmc wrll prtxlirce imprcrr.ernr*ntqin
cducntion;ll quality.'
Mnr\+evr,r, our ncrt caw CII~~SCS problems. Ills. have Iwated ~ n o t h c r
casc ~ ~ h c tht. r c l n c ~ l l ybaked staffing s y t e r n also h a s brcn a d~s;~.;tcr.In
thi5 case, t-etlwcver, thr 5yskern w a s irnplrmentcd only after cascf 111
const~ltaiiunnrld w ~ t hthr full agrrt8rncnt of ~ t a f f,lnd ollicr.;. mi.; is not
\chat ~ v expcitcd.
c ll'hy tines !I115 caw not fit' rtlrther r u m Ination ot the
case rt.r.c*,ilsthat t l ~ t~chr,nl
* hw had a nrbwprincipal appointed sincc l lie
.;ystrarn IVJS ~ntrmiuced.This principal i< not t n ~ s t e da n ~ il< bcl~cvetito
plav faunuritr.; H r is a l w bel~errcdto victimize particular popular staff.
Mavbe thcse aw the rcnsons whv the systrrn d t ~ lnot work. Chrcking
back nn t*arlirr cascs, the succew stories all had princip.lt.; rvhn wrbrc
widcly liked and trnstc~i.
hnd ntr thp two failurcs h a v e anvthing in c n r n r n ~ n ?Althr~rlghthe arc
dilfrrent in ccrtain ways, what they h ~ v in e common i s th,ll teacl~er\1117
~ not IIJVC L ~ ntv. ( i t i t j r f ~ ~f~rc t~fit,+
+ s!t,td7i~;. 111 oiit, C J ~ I *~t t v ~ q
f ~ m - ~ t>n
c l tlitvn
withoul ~.onsuttatiim.In thc uthcr, t h ~ .prrncipal has trndi~rmini>dtl11.1r
ccmi~denccin the t'airntls o f t h system. ~ IVe might tE7c.n marl I @ rwr
revised prnprlsition to srlrndhing like: ' Whrrc tt*ilciwrshavt. confidence
in thr falrnw- of a devolved ~ t a f t i n gsytern ~t will Imd to in1pr4n*cmrnts
in the qualitv of rducation,'
We could gn nn with additional casc st~rdicsto c h t ~ ortt k how well this
yrtrposition Ilnlds. En additicm lo looking at caw5 whcre n local q t a f f ~ n g
yyqiprn was inttoduced we should look at some cases whcrp the central-
i7rd syqtcrn was rctainrd. We would ~ v p e c tthat r 4 would ~ not sec the
s a m e Icvcl of improvement in edt~cationin thew s c h o n l ~ I. f this prr~vcd
tn bc sn it wi7t1ld strengthen our ctlnfidrnce in our initinl throry about thc
inipact of local st,iffin~systcms.
We r n i ~ h look
t for cases that could di;pro~vo u r thenry. For e u ~ m y l r~f ,
wi> coulcl find a school that had kept thc centraliwd syqtcrn and had
shnwn an irnprcrvrmmt in educatinnal qualitv wc rn~ght2x1 a b l ~ ,to
f ~ ~ r t l l eimprove
r our theory. WP might shrdy a schnol w h ~ r t we * hnvc
~vas idt~ntificdthat there Il,i< h e n nn Irnpravrmtnt I n cducatitln btst thc
ctntltrali;rtd ~ t a f f i n gs)rstpm had b c w rt\taincii. Thiq wt>~lldtvll a ~ a i nour ~l
r c v i ~ c dprcrp{~rh~rn. Hr~rvevcr,further in\ c..;tijiation of thc <cI?tlol rn rsllt
~ wlien w e Irwk a t thv qchtml in i t s widt\r contcvt ~ h e r c15 J
~ I i o r *that
l~o<titerclaticmship bctwecn thc school ~ n Ithc d Ivs,il cc,nrmr~n~tv. 1Z'c
I ~ a mthat thrre had b c ~ nn previous attempt b7. t h e schnnl r{n~ncil(on
which thrrc. werv manv local cornmun~tl,rcprescntattves) tc, ~ r l t r o d ~ tar e
Iucai stnffinq cchrmc. TcacFlcrs had sl~cce+<fullv rc>+i\trdt h i q ant1 Fc~llthat
tl~tlccntrali/~dst0iihngsyqtrm was protest~ngthem.
Mfhtli M.P It.ai-n t h a w thing.; a b n t ~ trr-hat nt firkt 5 i ~ l l t1 ( ~ k Itkc s n
'tltw~arilcaqr*' th,il dt+prn\rc*c:o u r propositirm, in facl it t 11rthr.r ctinl~rni.;
266 CASE STUDY DESIGNS
o u r propositinn that strfsses the notion that teachers must have con-
fidence in the system. Perhaps this case would cause us to further
broaden our proposihnn to emphasize the importance of teachers having
confidence in the system and to identify t h e factors that produce that
cnnfidcnce rather than strcss the importance of a local or centralized
systenr in itself.
Summary
This chapter has stressed the pi)iiint that case study analysis should he
fundamcntally a theoretically informed undertaking. This applies
equally to descriptive and explanatory caw studies. The use of ideal
types as a method of analysing descriptive case studies was di.sfrrssed.
Two broad methods of analysing casc studies for explanatory purposes
were outlincd. Where case studies are used to test a theory, pattern
matching provides an appropriate way of analysing and comparing
cases Pattern matching can range from matching very simple predicted
patterns to highly complex patterns involving mrlltiple independent and
dcpendent variables. Where case xtudics are used to build a theory,
analytic induction provides a uscful wav of analysing and using each
case study.
Note
1 Of course this Iogic dssumes that our causal variable is the only factor that
can produce a given imtcnme. If we find caws where the effects occur but our
favoured cause is absent then we have alternative explanations of thc phenom-
enon. O u r research design should he de5igned to elim~natesuch alternative
explanations
REFERENCES
Aronson, t a n d Mills, j. (1954) 'me effcct of severity of initiahon on liking for a
group', Joirrrlrll r l f Ablrr>r8trrllni7d Sacrol Ps!/c!t~/~gyr 5% 177-81.
Alchley, R.C. (1976) Tlrr S o c r ~ / r ) nf ~ qXpt~r~tnenl. Cambridge, MA: Schcnkman.
Baumrind, U. (1964) 'Srlme thoughts on ethics of research: after reading
Milgram's "Behavioral study of ubedicnce"', A~ntlrfrflnP~.vc!zo~o&'~s~, I$. 421-3.
Becker, H.S. (1966) Orlfsi'ler~Stlrrit~,~ 01Dmiance. New York: Free
Irl tile Socii~lo~ey
Press.
Blalock, I I , (1964) Corrsal I n f e r c ~ ~ c cins Non Expi~uii~rrntalRe.~wrcli.Chapel Mill, NC:
Universitv of North Carulind Press.
Iqlnu, P. ( I Q h 4 ) Sxcllnrfgr nnd Pr~tlli,rit1 Sorrn! L ~ f cNew York Wlley
Rlurner, H 11956) ' S o c ~ n l o g ~ canalysis~~l. and thp "vnriable"', A m r r ~ c n ~Sorir~logicfll
i
Kr'url.rtt, 2 1 683-40.
Rrutvn, C;.W and Harris, 'r. (1978) TIie Socml C7rr~if1sr ~ fDt-prcssinr>:a Sfrrrly of
Fs!jrhintrrc Dl~ardrr111 M7onre1r.I.ondom: Tavistock.
Ruck, N., Ermisch, J a n d j~nkrns,S. (1995) Choosing n Lvngitudinal S u m q Design:
Tlic lssrt~'s,ESRC Rosearch Centre on MicrwSoclal Change, Univers~tyof Essex.
Campb~11,D.T. (1989) 'Foreword', in R.K. Yin (rd.), Carse 5tud.y R~rsenrch:D~sign
and MriholJs. Rcvcsly Hills and Imndon: Sage.
Cdmpb~ll,D.T. and Stanley, J.C. (196.7) E x p ~ r r t ~ ~ r t ~nnd t f l l Q~rasi-Experimentfll
Ursr~~trs,!i~r R ~ w r r c h .Bor;tc>n'Houghton Mifflin.
Cohcn, J. (1987) Statistjcnl Poxcr Annlysiclfor the Brlra7)iourrll Scrc!~c~s. Ncw York:
Academic Press.
Cook, T.D. and Campbell, D.T. (1479) Quasi-Exp~ri~tzmtat~on Dbipu und Annlysis
Issurs for Fidd Srttinp. Boston: Huughton MiMin.
Cnser, L.A. (1977) Mnslrrs of So~-inlng~~-i~l Thought: Idens Sn Historical and Socrnl
Glnl~xt New York: Flarcourt, Brace, Jo~,anovich.
Davrs. 1.A (1985) Tlre Loxic of Cnirsnl Order. Beverly I-Iills, CR: Sagc.
Dempsry, K. (1990) Smnllfuictt: n Stlrrdrt o f Snrinl I ~ ~ r q l m l r t Cohrsinri
y, nnil 6clorrapii~,q
Melbournr. Oxford Unikcrsity Press.
L1~n7rn,N . (197s) Tliv Rr..sp~truhAct- a I'l~rvr+,ticnlrtrfrr~iilrctrnnlo Rrseflrcl~Mcthods.
U e w York McGra~v-I-lill.
I l ~ , p a r t m r ~of~ tf l u m a n %;ertmicesnnd I3ralth (1995) Ynl/th S z ~ i ~ i d1t1 t ' A r ~ ~ t r n l i n0:
Br~cl;qm~/i~d Mrr~~ryrnl?h. Canberra: Australian Government Publishing Servirc.
dc V a t ~ s ,D.A, (lq9.l) Lurtlng Go: Rrl~rt~nrrsf~r~~:: b~fior'rlrAifrrlts und thcrv Pnrewls.
Melbourne: Oxfnrd Universib Press
de Vaus. [).A. (2001 ) S A L Y V C rI iIl~ Social Rt,srurcllr l..ondnn: Routlrdge.
d e Vaus, U.A. and LVolcott, r. ( e d s ) (1997) Arrsfr~lrlrrrF ~ n ~ i I'roftlr~: ly Sncrnl nnd
Drrrlnyrrtyl~rr Pnlterns- Melbnurnr: Australian Instih~teof Family Stndics.
 2hH REFERENCES

Uillman, D (1978).2117il r~ndTC~PIIII(IIII' ~ F I P ? T I / S flrr

. T ~ t r dS ~ P S I , ~ I I Mtatlrod. New York: Krrl~ngrr. 1.R (1973) Frlrrndnt~n~~s Krsrarrlr New York Hott,
(if Br~ltlr~lrc~rir,~l

Wiley. R~nehartand Winston

Durkheim, F. (1970) S~liclrif.London: 12outlcdgc and Kegan Paul. Kerliilpr, t- h (1'179) Br'lrnrrror~unlRcssn~rrlzr a CunctVptunl A l r p ~orrr /r N c r ~York
Foddv, W. (19931 Cor~sfr~ictrn,y Questinn<fi?r Tr~tt,mic-;cls f l r d Qurstiorrnn~res Thmq Holt, Rtnrhart and Winston.
17nd Flucllcr it! Soclnl Res~~orcl;.Canibrtdge: C a m b r ~ d g cUnrversity Press. Kerllngt=r, F.N and Pedhazur, E.J (1973') Mrrlhylr Rrgrt,r~m)r $17 Lir'l~arj~tlrrrn!
Freeman, D. (1984) Mnr~rrrrtMmd find Sntlion: flrr Mnkiug niid Un~nokinxof (in Rr,~cnrcl~. New York: Holt, Rinehart and W ~ n s t o n
A~rt!~rr~pulo,qit-ol Myth. l+armnndsworth: Penguin. Kimmel, A.J. (1988) Eth~rsnud Vulurs rn Applied Snctnl Jirst,arc!r Reverlv Hill< and
Freud, 5. (1955) T ~ VCasp O H~st[?rrt~ (Lritlr Hnns and Rot M111iJ. Londun: Hugarth London: Sage.
Press and the Instih~teof Psycho-Analysis. I rpkowsk~,J 14.(1989) 'Treatment of wave non-response In pancl surveys', in 13
Geertz, C. (2973) 'Thick descriphon: towards an interpretive theory of culhre', in Kasprzyk, G. Duncan, G Kalton and M P S ~ n g h(cd?), PrfnfulSIITTII,~F New
C. Geert;. (ed.), Tlru I~tt.r;~r~totion of Cult1rn:c: Sl,lt>ctrdFsqalrs. New York: Basic. Ynrk iq~ley
pp. 3-32. Lewrq, O (1951) L ~ f il r r n Mrxrcnrt Vllln~r. Trprrztlnn Rrsttrrl~~d. Urhana, IL:
Gerth, M.H. and Wright Mills, C. (1946) Fm~nMnx W r h ~ r :Essatl:: it1 Sociolo,y!/. Universlty of Fllinois Prcss.
Oxford Oxford Universitv Press. Loether, H 1. and McTavish, U G (1974a) Itlfcrrnl~nlStattstrcs for Soric~lug!~ts. nn
Glaser, U.G and Strauss, A.L. (1967) Tht. Disr-oorrrn/ of Crour~dedTl~cluoy:Stratcpes Inlrod~rctiot~. Boston. AElyn and Bacon
for Qlmlitutrrw Research. Chicago: Aldine. F .nether, F1 J and ,McTavirh, D G (1974b) Ursrripltur~Statistrrq Jor Socfo!o~ylst~nit
Glenn, N.V. (1977) Coljorf Atrrrlvsrs. Uebe~ly H~lls,CA: Sage. Iiitrodrtctron I3ostnn: Allvn and Bacon.
Goldenberg, S. (1992) U~rnkirrgMet/~odolo,yrcally.New York: HarperCullins. I ynd, R.S. Meriell-Lynd, H (1g29) M~~Jdlufoilln: n Strirly 1r1Modcr~rAntrrrcnn
Goode, W.J. and Hatt, 1' K. (1952) Mr,tlrorls rtr Socirr! Rt-srori-11. 1,ondnil. McCraw- Uzr!tlric. New Ynrk: Harc~rurt,Bractl and World
Hill I.vnn, P., Purdon, S., tlcdges, B dnd Mcaleese, 1 (199C) 'An asws3mcnl of
Graetz, 13. (1993) 'Health consequences of ~ r n p l o v r n t ~ natn d unrmploymenl: nlterna!rve ~ ' c i g h t l n g+trdttbg~rs',Yolith Colinrt Surr cy Report, Fcf iicd h o r ~
longitudindl cvidcnct. for voung mcn and women', Sarml Scrcilce and Medlcirre, Dt.partmrnt, Sl-reff~ld
36. 715-24. Mardm, D. and Abrarns, 5. (1E)87) ' L ~ b c m t n ~compnntolls, , t n t r u d ~ r sand
C r ~ m m ,I G, and Yarnold, R.R. 11995) Rrnding nrld LFnderstn~~rlrng Xru!t~rlar!ah, cuckoos in t h r nest: a soc~ologyof caring relatlr>nshlp$over the I ~ f cycle', c in P.
Allatt, T Ked, A Rr\)nlan and R Byihewaj ( c d ~ )Wolnrr~ , orld t/tr L l f i Tyi!~
Slntrctir,q. Wadlington: American Psychological As<nriatinn.
Tmnstfit>n::ni~dTurnrng-Pt>117tsLondon Macrn~lldn
Grnvcs, R.M. (1979) 'Actors and questions in telephone and personal interview
S U ~ I ' P ~ SPnblrc
', Opri~ionQnnrtrrly, 43, 190-20.5. Vvlarsh, C (19B2) T~ICSatnjcy M ~ t h o d the Co?;tnhntrotr o f S u n ~ r ~ , fu c Si~rnlo~rcfll
Eup1n)ratEun. London: Georgc Allen and U n w u ~ .
Groves, R.M. and Kahn, R.L. (1979) Sur~reysI?y Trlrphonr. u N ~ ~ t ~ oConipnrisorl ~ial
u~rHrPerso~~al Intrruiml~.New York: Academic Press.
Marsh, C. (1988) txplorrng Uatfl. nn Introdurhan fo Uotn Rrlalqstc for Socrul
Screntr~tsCambrrdge. IJolzty
Hage, 1. and Meeker, B.F. (1988) Sorinl Crru~nl~ty. London- George Allen and
Mmd, M. (1947) Covtlil,qof Agu rn Sanron o S t ~ r i i of j A d u l ~ ~ ~fwd i ~ nScu
c ~111 Prltrrrtirlp
Unwil~.
Surrttrt? Fhrrnundswurth' P c t i p ~ i n .
Hakim, C. (1986) Rusedrrh Des~gn:Stnttugrus and Clroiut.~In the Design rf Social
Menard, S (1991) LOngitz~CIr~lol Re~tvrch.hrerubury Park, CA. Sage.
12rswrrJt. London: Allen and Unw~n.
1 I-lmry, G.T. (1995) Grirphrng Data: Trchrrr~ytr~s of Disflh!/ and At~alysis.Thousand
Merton, R.K (1968) Socral Throry and Sarol Slrrrttr~vr Ncw Ynrk F r w Press
M ~ l g r a n ,S. (19M) 'Issues in the studv of obedience a reply to Rsurnr~nd',
I'
! Odks, CA: Sage. Arnrurc(ulrll Psyd~ol~)grst, 14 848-52.
Hkrtel, B. (1976) 'Minimizing error ~arinnceintroduced by missing data analysis

I
Mjlls, C W. (1959) TIlr Socrologri-01 Imnglrraflon New York- Oxford University
in survcy analysis', Soc~nlo,~ica! M ~ t h o d snrld ResmrcIt, 4: 459-74 Press
Hochschild, A 11989) TIw S~,co?ldShift New York: Avon. Mi tchrll, J C. (1983) ' C a ~ eand situation analks~s',Socrt~/ryrrolRnllnrl, 31 : 1x7-211.
Holnnn, R. (19Y1) T l t ~Ftllrr~nf Sllcrnl Rrsenrch. London I.ongrnan. Moser, C A and Kaltun, C (1 971) S I I M~Hrodlr ~ ~ 111 SOCIIIII~f~~strxnt1011 London
i
I
t!clnlans, C;. (1961) Sncrnl Rrltavici~rr:[IFElrmtbnhory l ortrr?. Yew York Harcorirt,
Brdce, luvanovich.
Hrlnemann
Mlreller, I H., Schuessler K r and Cosmcr, H.L. (1977') Strrhsf>cn/ K I ' I ~ S O I I I I!II ~
Tnhnsnn, h, (1977) Sac:,ll Sfnt~strc5r t < r f / ~ c ~f r Tt3~?rs.
r N e w Y n r k McGraw-Hill.
:I Kalton, G. (19833) 'Compensating for miss~nfis u n e y data', Research Report
Srrc~olo~~r Ibston Houghtrjn h'liifl~n
n t i v c In N Denzin ntld
II Scr~cs,lnstih~teof Soc1a1 Rcscarch, University ol kllchzqan
I'unch, M (1W4) 'l%I~tlc+and etll~csIn q ~ ~ a ~ ~ t ~ trr<enrch',
Y. L~ncoIn(cds), Hiri~iflmoF.ot Qtralifoti~rReqarch Thou5and Oaks, CA S d g ~
Kalton, C; (1983b) ltrtrod~tcfiurtto SItn~e!/So)npli?r,g.Beverl' HIITS,CA: Sage. pp 83-97
Kalton, G. (1086) 'Ciandling wave notl-response in pancl surveys', Jorrrrrfll ~ r f Rrdftcld, R. (1'130) Tl~puztlnn,o M1,rrcolr Yrllyiy 17 5tlrti1, of Frdk l @ Chicago
C?fir>nlStof7sf1rs, 2: 303-14 Unrvi.r<~tyrrf Ch~cagoP r e s
Kt'llehear, A. (1947) 7 . l ~ililclbtr~r.iuc Rmei~rt-lwr: Grr~dcto ~ u f h d d hSt . Leonnrds: Riwthlrsbcrgrr F.l and Dicksnn, W T (1919) M ~ r r f a ~ ~ ~ rrurd ~ l ~ iIht'
i t Wnrhrr.
Allrn nrtd Unwin Carnbridgp, MA: Harvard Universlh, Prrs5.
 IZEFERENCES

Rogers, T.F. (1976) 'Interviews hy telephone and in personmquality ot responses

atid firld prrfnrrnancc', Ptrhlir Opitrton Q~tnrtesll/,4 Q 51-65,
liose, O., Duck, N.A. and Corti, L. (1941) 'Design !<sues in the British Household
Pallel Study', Working Papers of thc FSRC Research Centre nn Micro-%cia1
Changc, Papcr 2, Universitv OF Essex, Colchcst~r.
Rosenberg, M (1968) Tlre L n ~ r cnJ S u n ~ t yAAn,y~ts.New Yurk: Basic.
Sprctor, P I . (1981) Rrsrnrcf~Dr51~~1s. Beverly Hills, CAmSage. NDEX
Stake, R.E. (lqY4)'Case studies', in N.K. b n 7 i n and Y. Lincoln (eds), Harrdhnok of
Qtinlitnfr-tw Rcsenrch. Thmsand Oaks, CA: Sage. pp. 236-47.
Sullivan, L., Maley, B. and Warbp, M (19Y7) Sfafe of tl~rNat~nn: Stotiqficnl
Irrditsrtrbrs rifAu.;tnrlia'c We11 Rr2ing.Melhorlrne:Crntrc for hdepcndcnt Stucl~es.
Supprs, P. (1470) ,4 I'roballrEistrc TI~rnr!,of Catisnlilu. Amsterdam: Nnrth Holland.
Tilbachn~ck,B.G. and Pidell, L.S. (1983) U s i q M~ultivnriateStntislics. New Yo&: A prlnrl reawning Bias, TI
Harprr and Row. crrjsc wct~onalL I L * ~178-9
I~S, attrltlnn, 75-h
1a jfeI, 1-1. (1984) Tt~cSncinl Ditnrnsiorr ' European Wev~loplnerltsi~ Socral PsyrJro10~~. Abstrncted ernplrlcrcm, 2 c f k ~ ont descriptic~n,136
Cambridge: Cambridge University Press Ad hnc rPd5Clnln): effm-t on exylanatic~n,I?h
cro.;s seft~onaldcs~gns,174-80, 182 ~dcntifvingmissing data bins. 149
Vaughn, D. (1986) U)7corrplmniy:l'rrr~zrtlgPoirrts in Il~tinmtrR~iatrotrsh~ps. New Ynrk:
Agc ct,lir~rt \pan, 71 2, r~mol~irrg stnt?shcnHy, 1.75, 136, 152
Oxford Un iversitv Press. tcstinx for effects, 77
Ageing efiectc, 115, 124, 211,212-17,
h'hytt*, W.F. (lq43) S t r ~ e lComtxr Snrirty. tltt, Soclol S ~ ~ I I C ~ofI on
I M Itnlimi S l ~ t r n . 215 r w i ~ lt I ng,
i 15 1 -2
Chicago: University of Chicago Press Inn#iludmal designs, 172
Yin, R , K . (198q) Ct75r. Strid!/ Rr.~cnrrl~: D P S ! , ~rind
F Mrfhods. Rcverly Hllls and hclltmr~n4 FrOUF 11t.slgn.173 CAI'I, 1.11-2
London: Sage A g ~ r e g a i ecllanpe CASC bSL1dlCh
o f Casr S t ~ t d yRrsmrcJz. Beverly HiIIs and London:
Yin, R.K. (1493) Applirnhvr~~ Ag~rcjiatcchdnge, 122, 128-9. 131, 135, 137, 2 ,2 3 4
cl~nlc~d
s<1ge 1h7-54, 211 d c ~ c n p t n ~a cn ~ l y s i s25n-3
,
~rdplis,163 rxplana torv nnalvc~s,253-6A
methods of analy<lq, 151-4 prcwntlng, 214-5
repeated crosssrrtlorl dw~gns,153 report!ng, 244-5
h h l ~ 167
~ , 16-4 thcorv building, 267-6
Alltxation to grotips, 44-h,48 I henry tesllnR. 25-67
hnalvhc mdurhnn, 263-6 Casc study d c ~ i .50-1,21$-%
~ ,
six steps. 263 fypPS, 728-')
Anr>nyrn~v a n , 141-2
experiments. 87-8 Ceuyal direct~nn
cross s ~ t i n n ad
l e s i ~ ~192
s, cross scctiunal dcsikms, 180-1
cw also ronfida~tinl~tv C~usaleffec!
nnonvmizing data dirt~t,65, 71
c r t ~ ssectional designs, 192 indirect, 71
Arhficiality, T', total, 71
hssoc~ation,measures of Caufal explanation, 2-4
cr~'crvrdatinn, ctr-variation diffcrcnt tvpcs, 37-Y
Attrilrnn dirrct effects, 2-4, 36-4
asscssin~cffects, 123 ~ndirect.3 - 9
bias, 75-h g 4,344
~ n f t ~ r n nc;l~~sc,
i n ~ t i nsample
l s17e.14.1 mle of cornparisnn groups. 79-47
niinimizing c i f ~ ~135-6
h, Cau-al rids t i n n s h ~ p ~
rnlntm171ng,I T - q crltrrla fnr l ~ f t ~ r c n c9
c , -.(>
nori-r,jtidr\m, 135 direct and ~ n d ~ r ~76-9 rl,
panr.1, 138-4 Causat~on,4-5, 3 - 5 1
136
rrplarem~nt~ , a prlrti-I reasoning. 178-9
rrspondrnt hurdrti, 139-411 ;rd hex rensonlng, 17Y-HI). Tll2
tmckirl~panel m ~ m b e r s 138-9
. correlation and cause, 178
n n ~ non
t response, 147 dctcrminirtlc, 3-5
Avcragc pnrl~alt,iblc, 104 ~ < t n b l i < h ~ causc,
n~, 7 4 4 7 , 178-81
 INDEX INDEX

Uausatlon (cottt rnultipt~c a w studitr, 22h Defmi tions seplrcatron, 205-h

cmss sectional d~stgns,177-81 means, 1YQ norn~nal,24-7 q p c ~ f ~ c a t i o204-5
n,
matching. 178 on? shot rase slurly designs, 230 upfrahonal, 24,27 spunous rclahonehiy, 204. 207
model f ~ t t ~ n17Y-8ng, time points, 42-3 Demand characteriqtics, 24 steps, 203-4
probahllrstic, 4-5 Computer assistcd dota crdlcction, Dependent variable, 1936 types of causal pattern, 36-9
rcc~procal,36 142-2 Description, 1-2, 114-5 zcro-order r~lationqhips,2114-10
statt~ticalcontrols, 177-9 Conccpb cross-sectional, 194-2MF set also d~rectrelationship, causal effect,
type? of causal patterns, 36-9 clnrifyirig, 24-7 deductive typologies, 225-6 causation
S P a~ l ~ nsaucal dirirhon dimensions, 25,Zh descriptive. research questions, 17-19 Ernbddcd units uf analysis, 220-1
Ctnhal tendency, measures of, 92-3 nature of, 24 empirical typologes, 225 Equallty of vanance, 103,
Change graph? CATI, la 1-2 idea1 type$, 225-6 Error
SL,C graph" UAPI, 141 -2 inducttve typologiec, 226 SPP measurement error; sampling
Change measurement, 153-63 a d v a n t a ~ e sin I o n ~ i t u d i n ~ l
studics stanstics, 89-90 ermr
calcuIating raw change scr>rrs,155-6 Cnnditinnal tahl~s,204-5 theoretica1 typologies, 225 Ethics
cross-sect~o~%al, 171-2, 17?-5 Confidence interval, scc sampling theory, 224-5 experiments, 83-8
experiments, 53-6 Confidence Iwel, srr sampling lypologi~s,2 2 4 6 anonym~ty,192
measurement error, 156 Cunfidcntialitv w ~ t hcase studies, 224-5 anonymr71ng data, 192
multrd~rect~onal, 154 data storage, 87 hwriphrre analysis ct~nfidcntialityand anmymlty, 146,
perccntagc change, 156-7 expcrim~nts,87-8 cross sectional data, 194-218 192
problems of raw change scores, 1% cmss sectiorral designs, 192 Distr~butions,195 covcrt observa tinn, 246
qualitative change. 154 Inn~ihtdinaldesigns, 146 ccntral tendpncy. 195 data security, 146
quantitative changc, 154 1cg.tl contcxt, 87 frequency distributinn~,195 d~ceptlon,246
raw change scores, 155-6 limits, 87 graphs, 145 dtsp~ismgidentitv,216
reliability and raw chanxe, 156 s~r,lrlsll anonymity [deal typ! analysis, 251 harm lo participank, 145, 193
r e s i d u a l ~ ~ echange
d scores. 156 Context irnplicrl theories, 251 illegal/harmful behav!our,246
s nlsu ag-grepitcl rhange, indtvrdual
'
t c a w study nnaly\is, 234-6, 250 kurtosis, 195 cave studies, 246-7
change Control groups shape of distrlbuhons, 195 crmss sechnnal designs, 192-3
Change scorec, 1 10 strengthening rntcrnal vaIidity, 73-5 skeu-nesq, 195 informcul conwnt, 145,246, 85-6
Chanh~,descrthmg, 163-8 Correliition, 95, spread, 195 longitudinal designs, 144-6
graphs, 1h3 Cost suhjectiv~ty. 250-1 partrcipant observation, 246
tables, 163 cascstudies, 24% typologim, 252 respons~bilityto other researchers, 247
Chronotog~calanafys~s,261-2 declustering. 141 Developmental effects voIuntary participabon. 79.145
Cohort analvsis, 115-18, 21 1-17 1ongihidinA designs, 140-1 see ageing e f k k see informed consent, voluntary
age cohort span, 212 Co-va ria lion, 34-li Dirtct relaltinnships, 36-9, 65,71 participation, anonymity, harm to
ageing cffectc, 11 5, 21 1 Covert t>hsc.rvntion,246 SPP nlso causal cffect participants
aggregate change, 211 case studies. 246 D~stributions,descrhing Experiments, 53-1 12
cohort effects, 21 1,217-15,216-17 Critical cases, 227 centra1 tendency, 92-3 analysis, 89-111
constructing tables, 2112 Cross sccfior~aldesigns, 50. 170-218 proportions, 95 and explanation, 70
perir~defi~cts,115-lh,Z11
8 cf lr)n$udinal, 1711-1 shape, 94-5 artificiality, 77
problemq, 217 group differences, 170-1 var~abllity,9 3 4 clas~icdesign, 48-9, 53-9
reading cohort tables, 212-17 intcrvcntinns, 172 Double-blind, 83 ethics. 83-8
repeated cross-sectional dmign, 21 1 nature of 'groups', 172-3 Dropout field, 57-8
standard cohort table, 212 rcpeatcul stud it!^, 173-5 w e atli-liton Iaboratorv, 56-9
unevenly spaced surveys, 215-17 tlmr rlimension, 173 n y n a m ~ csamples, 137, natural, 58-q
Cohort scquent~aldes~gn,124-6, 133 aggregate change, 137 5eattEc/Dcnver lncome Maintenance
panel agetnE, 124-5 Data serurily E x p r ~ r n e n t57
,
Cohort tahlrs [ongitudinal d c s i k ~ s146
, Effects types of dc-ign, 53-69
st*^ cohort anaivs~s .;PP c~l>iidp~itialt!v short and long term, 79 Explana han, 1, 2-5
C~7llapsingvar~ableh Dcucptiun, 246 Elaboration analysis, 1IM.203-10 cf prediction, 3-4
ser recod ing cast= R ~ I ~ ~ 246
P s , mndlbona I tables, ZD4-5 cxp1anatc)ry narrowness of cupcrimcnts,
Cornpanson" disguis~ngidentity, 236 control variables, 2M 711-2
dtffercnt methods, 40-7 Srt, c'nl.~oirnfrvmcd ctjnscnt indirect causal relationships 204, ~diographrcrxplanation. 221, 233-4
group?, 09-47 Declusteri~rg,141 206-7 nomuti~chccxplanahon. 221, 273-4
meaningful, k7-B DPductive rpasoning, 6-#, 253-62 interaction effects, 204-5 roie of expcrimmts, 70
multiple, 41 -2 typnlosirs, 226 problrms, 210 w ~ t hcase studies, 221-4.253-63
274 INDEX
tl.1wthon1c ctfcct. 77
c r r w wrtionaE ct t.xp~.rimcnts,IK4 History, 132
rrt~)rssccti(tnaE c f pallel dudips, 184 ~mxpvrimcnts,73
casc studl: clesip, 237-411 CJ\C s t u ~ l ~ t236
s.
cross scctlcmnl design.;, IH4-5 c r w s scctir~naI d~sikm5,171
cxperimmt5,7h-8 [ongittdinal designs, 132
longit~rdin~rl rl~~sigps, 135-K Mnli?Iir units o l annl\rsis, 22U-1
inln~gratlcmand outrnigratron, 137
panvl attrition, 135-h
renctiviiy, 7h
repwx.nt;ltivcnL%s, 1 P4-5
5t;l trstrcal cpnrralixalirm, 237, 2111, 249
iesting for bins cffcctl;, 77
thcorr~ticnl~ m r m l i f a l i n n227,
, 240,
Z4Y
tlirents tu. 76-7
unrr-pr~wntativfsanlplecl, 76-7
Ertrancoiir variahlcs, 19-,20, 37-4 arv nlslo 0 p t - rtirlnali~atinn
~ Instrument d ~ c a y ,7-1-5,1334, 177'
lndlrect causal relationships. 36-9, 71
Factor ,~nalysis,lL)B t.laboration ~nalyi.;,204, 2Dh-7
1:;lclnriat a n ~ l y s i ui i vnrianr~ lndlvidual level chc~nge. 122, 129-9.134. 175,
~ L > L -I r w - \ v a ~ an;llyc~k
- of vari~ncc 157-54
r:;lctori,>[designs, fji-H, 103 grapt15, Ib3
I',~llacy o f aiiii-minfi tlw ron.;rqutqt, 1npthr3d.; of nnalvsic, 1 5 1 4
13-14 ~nulttdircctionalct~~angi-,
F;rl.;iiira lion, 14- 15, 267, 2h.5 panel d r s ~ g n s 157 , unlly of casw, 234-5
closed thcr~rieu,14- IF; t<lt)lt.h,163, 1M
Fcminlst rprrarch ethics, 217 Induction
]:loor and cciljnfi cffccts ,inalytic, 2411,263 -6
.,itrtruni-alim ~ i l m - l s mumeratlvc, 240
I:OCUSW~ s,?mpling, 240 lndtlcti vc typologies, 225-6
Framcb nf refcrenw, 3 9 4 7 Inductive reafonlng, 5-h
5m lrlst~cornpiris~~ns st? nlso nnalvhc ~nducticln,
F r c q u ~ n c yiiistril~\itiont;,145 Inference
srr causal rrlatii)nsh~ps,thcnry
htt~td~n~
Inferential stati~tich,89-90, 99
Informed consent, 7H,P7, Zlh
rasp studleq, 24h
d e c ~ p t i o n 7,R
how much infin-n~ntiun,R.4
erperunents, 64-6
l(m3rttldinal dc~ikms,145
IirnlnK, H?
who frrlnl, 8.5-6
lnm~~rntiom, 120, 1:7
I t ~ ~ t r u n i r nc tl ~ r a v
ckpvrrmcntal, 71-5
SFPSS srrtronal d ~ s ~ p nI77
I a ~ n p t t t d ~dnc~slr p \ , 1W,131-4
Intcr.~ct~rln cfftrt.;, 42, 55. 65-6. 73, 74. 103, M,i>kin~relationfhips. I<>(>-7
1114 Ma(rlicd fluup<, 44-5, 75
clabr~ration~ n ~ l y s i20.1-5
s, r x p m t fdctn, 44-5
Internal vnltdity, 27-8 cross sectional d ~ r ~ g n171; s,
ad hr>c t*xplanaho~rf,182 ~labnratlnnanalysis, 203-1 R
cause and correlation, 234 matchcd block d e s l g c , 47-6
uar~ableanalvsis, 3 - 6 post test, 33, 4H.hl
crlntext in cace studies, 234-h p r p - k t , 14
cross sectiorral dcsibm, 177 sandomisa tion, 45
pffectof no randoln~7eJcontrol group, 131 statislical controls, 202-3
cqtahlishtng rauw, 177-81 Matur.it~nn
pft.lbl~shlngmvaning, 18 1-4 wsc sfudics, 23h
!11510l y, 77-4, 132, 177, 276 cross seciiunal des~jins,177
i d e o ~ a p h i cexplanations, 233-4 cxpcrirnents, 73-4
race s t u d ~ e s233-7
, Iurigrtud~naldesigns, 132, 133
cxpcnmcnk, 72-6 Meaning, 5, 11
long~tudlnaldeslgna. 131-5 establish~ngmeanins, 181- 4
Facesl~wtscxrolt>gy, 182
rnnttrrat~on,73-4, 132 case snldv annlyhis, 2 3 4 4 , 250
n?r*asurt,ment frror, 134 cross swllnnal designs, 1814
mnrtnlitv, 75-6, 1.14-5, 177 pro\~dinfimcaning, 1H2
nnmntlietic explanat~nns,273-4 variable analvs~s.181
sclc~hr>n, 73 Mmsr~rt.menlprnlr, 24-71, 84, I DJ, I Sb, 157
stalislical regressron, 75, 134, 177 conqtant error. 31
151 tcstlng/pand cnnditionlng. 71, 17.7. 177 corrt4a led PrIvr 72
term* of, 31 -2
Interval estlmntrs, 2tH Inn~~tudunnl drsipis, I34
Itjtewals random crrrlr, 71
attrition, 79 Measuring changt,
hetwwn t.xpcrimt.nta1 s f a ~ e s79 , bcr rhange mmPa5uwrnent
memory rflert~,80 Micro c l ~ a r ~ g e
~ n ~ 19,3h
I n t e r ~ ~ e nvnriablf, qt-r indlv~dualc l i a r ~ ~ r
mdwctlo~i s , 172
I n t ~ n ~ e n t r o n47. Missing data, 117-53
nctivc, natural, 47 alprage o l span rmprltatlon, 150
types ol, 48 bias, 148
uneven, 82-3 dealing with, 14s
Ilem non-response, 147 group means ~mputatinn,150
ignoring missrng data, 151
I-adder of abstractton, 24 imputatirm Fmm earllr=rwaves, 151
Level of rnpacuremrnl, 40-2 ltem non-response, 147
interv,~l,91 prrwise, 151
nnminnf, 91-2 randnm wstgnmcnt within ~ O I I P F
ordin.11, 91 i r n p i i t a t ~ o ~150
~,
Lifc ct,ur\r analvsi.;, I lt; regrrssion haqecl i r n p ~ t ~ l b o151 n,
I .~teraEr~pTlcatior1,262-3 wmplp mean< Irnpul.ltion, 1.19
Lon~itudlnaldc\irm\, 4g-50, 1 1 3-hq sample s v e , 148
c f cvpcrlrnrnral d e s i ~ n 113
, scalc ct?n.;tructinn, 14X
dlnieiisions rrf, I l ? l Gnurrpg, 14;
s, p n f l cf trend, 113 unit IIU~ rt3sponsc,I47
prospectwe cf r c t r r ~ ~ p c c ~113-14
rc, w a v r nnn rpsponse. 137
purpr~o r f, 11;-lH CIY R / W nnn resptmfe
Mortdlih, 134-i
Macro changr cross x ~ t i ~ n idl ~l s i ~ m177 5,
srr aggreaatt? change e k p t i m c n t s . 75-6
Mnun effcvtb, h5 l o ~ l g i t ~ ~ dd+l+it;ns,
i ~ ~ . ~ t134-5
 INDEX 277

Multiple jiroups s~thst<>nkial arrJ theoretical types, 22-4 RPactivity hard kn frnd groups, 123
multiple c a w study design, 22h tedinical and rnctlio~icllu~,ical, 21 case studies, 236 pooling panels, 127
multtple coI>orl dcstgn. 124 Post test only wirh conlrul group Recall purpose, 122
multtplc comparison test?, 101 e x ~ ~ r i n t n ~MI
ts, anchor events, 128
multtple group cnpcr~rnentald w p , Prcdictton, 3-4 memosahle events, 128 %mplc size
62-3 I'r~vacy problems with, I40 attrition, 143
Multiple time points 5r.r' confidcntiality recall problems, 127 crms h~chonaldeslgns, 187-90
multiplc potnt panel d e s g n , 119-20 Probability sampl~ng,79, 90 rel~ospectiwdesips, 140 dctcrrninants of, 187-188
multipkr yre test experimental dfijgn, I'roof, 15 r c v e s c telescoping, 140 elaboration analysis. 210
62 Provisronill nature of suppnrt, 15 telescoping, 140 experiments, 78-9
Multivar~nteanaTvsi5, 211 Proportions, 95 Rpcoding, 196 missing data, I 4 8
mu1 tiplc. regrcsslon. 104 Propositions. bH dishlbutionaI approach, 197-8 oversampling, 143, 187-8
Pruspcrti\ c d c r i ~ m masktng relabonships, 196-7 random, 79
Net change cnw studv dec~gnq,227-8 substantive approaches, 196-7, 198 single and multlple case stttdy dcsignq,
sw aggregate change disndvantagcs, 127 Rword I i n k a ~dcsips, 128 22h-7
Nomlnal d ~ f ~ n i t i n24-7
n, rnult~plrpoitll panel, 119-20 confidet~tialitv,128 statistical power, 78
Nomothctic explanation, 22,22 1, 233-4 simple pinrl, 118-19 Jteltabllity, 29, 30-1, fK subgroups, 187-8
Yon-pararnetr~cstatist~cs.1U2 changc scares, 156. 157 Sampling, 14
Yon-probabilily snmplmg, 90 QuaI!latlrre, IZepeated cmsy-sectional d~siw, 129, 137, access to cases, 242
Yorrnal distrrbutiun, 1UZ-3 cf quanhtativc. 10 21 1 caw s c w n i n g , 24 1
assuniption of. l(12 change, 154 Replacement for attrillon, 137 case qtudy designs, 240
crttical ratio, 1W data, 191 analysis irnplicatirlns, 122 confidence tnlerval, 189
t-tPStS. 1110 casc stud ips, 230- 1 strategies, 121 cnnfidenrp Ievc.1, 189
Y u r m a l l ~ e ddistributions. 102 ~ ~ ~ e r i m m83t s , hplicahtnl, 77, 78, 226 external valrditv, 29
Quantitative case studies, 226, 237-8 focuswd, 2-11
One grrl~tpcornparirons with known value, ct qual!lativp, 111 elabora t ~ r manalpcis, 203-6 number of cases, 2411-1
97.99 change, I54 exper~menh,90 oversampltng, 143, 187-8
One shot case s k d y design, 230-1 data. 191 literal, 279, 262-3 prvcislon of estimates, 188
Operational definlt~on,24, 27 rnsc stltdtcs, 249-50 theoretical, 239,262-3 s a m p l ~ n gPrror, I #
Operationalization, 24-27 methods, 1U Rppnrting case stud~es,2 4 - 5 selection in case studies, US-40
Clutmigration, 120,137 Quahr longitudinal designs, 12H-4 Research design theoretical, Z40
Oversampt~ng,143, 187-8 aggregate change, 12R-9 cf method, 9-10 typical caws, 240
indivtdual change, 128-9 cf work plan, 9 unrepresentative, 29
P a i d sample design, 102 Qwstronnaire length, 190-1 definition, 8-9 variance, 190
Panel agcing, 124-5 dimension5 of, 47-8 weighting, 188
Panel condit~oning,143 Random error, 31 Research question, Sampl lng error, H9,92,99, 188,
assessing cffi~k,123 I<andrjm sampling- descriphve, 17-19 longh~cllnaldeslgm, 143
Panel data, analysing, 155 scc prob,ttrll~tysampling explanatory, 19-21 Scale constr~rction,148. 151
Parallel casc study d c s i p s , 227,242-3 Iiandomizcul assignment, 45 focuss~ng,clarifying, 17-21 Stepticism, 11-15
investigators. 242 attrition, 8l,1(2 Rcsidualizcd change scorns, 156 Scheffe test. 101
research protwds, 242 control v o u p resentment, 81 Respondent burden, 13940,148, 140-1 %,lection of cases, 2313-41
Parametr~cstat~stics,102 gatekeeperp, HI lictrmpectivc design4 access to cases, 242
Part~clpantobccrvat~tm,246 group S ~ J P ,81 anchor ~ v e n h 128, case screening, 239, 241 -2
case s h l d i ~246
~, s random alltwation, 81
I ~ m j t of casc studics, 227-8 case studies, 238-40
Pattern matching, 2553-60 problems, 71 -2, 80-2 experiments, h0-1 number of cases, 240-1
P?r~Wit;lg~ change, l i h - 7 r c f t r ~ l s 81-2
, memorable cvenh, 128 sdrnpling, 240
calculal~ng,156-7 tinderestiinatin~~ , i u s , cffprt,
~l panelf, 126-H I;electmn, 75
rnPnsuremcnt crioc, I57 71 -2 recall problems, 127 %;eft-fulhll~ngpropl~ecy.83
probpems, 157 R a w diange Ccrjrc5, 155-6 re%erse telpscnp~ng,140 '+qucntial case study dcs~gns.227
reliabil~v,157 ~alcul~itin)?;. 155-h telescoping, I40 Shape nf distribution$, 94-5
Pcnod efferts. 121, 115-6, 221, 21 1-14,216, cxtwrne scorph, 156 Rotating panel design, 122-5, 133 S ~ m p l epanel design
Placebo, 81,83, mpasurpment frrnr, 1 % attntlun, 122, 12.3 attrlhon, 120.121
l'!ous~blc rival hypotheses, 9, 1 1-14. pmblems. 156 avording burnout, 122 h a s , 120
case shldies, 224 condihoning effects, 123 inmigmhun and outrnigratiun. 120
sourcc nf* 21,22-4 external vnltdity. 137 samplc size, 120
 S i n ~ p t tp~iriel
* tlcc1i:n ( t - m ~ l , ) llir.or!., 5 Validllv (cr~rt) ektcrn.~lv a l ~ ~ i i t34
v,
wit h ~ C ~ I J C P T I I N ~ I , 1 3 - 2 cu po.;L fnrlo, h crit'ric~n groups, 30 intcmal i7alidit\.%4
$viIlinul rrplacrmm~t.120 post tnclun1, h rrit~rinnv a l i d i l ~2Y
, lon~iludinaldcsijinc, 145
S~rn~rlatcd bcforr-.liter d r s i ~ n ,l?Q c~se study .~nalvsis,240-2, 753-hl) rxterndl, 28-'1 probleml: frnm, %.I
i i n ~ l ca5p
r shldy dcsi~m.2 2 ~ case study drsihm, 121 -h internal, 27-H
S i n ~ l cculinrl cle~ign,177-4 ~ t . 1 . nl.40 tl1ecv-y h~tildrng,thmry testing natui-c of, 28 W a w non rc\pon\r, 147
a ~ t - i ncif~~crt.;,
fi 124 Thcorv building, 5-6 thwats, 24 U'3vr'l
pcr~ud~ j i l ' c l ~124 . annlylic induction, ?6D, 263-6 sirbnlslr internal validity, external validity inh?nf;rl\ htween, 142-3
124
pr~ihlcn~>, 227,227, 2h3-3
crlht' ~iudivs, Variability, mcasurcs r)i, 43-4 nilmber In long~tudrnnlstudzes, 1 4 2
Solomon .I-grtwp desijin, h3-5, HI). Theory testing, 6-3, 11, 253 Variable analys~s,1H1,234-6 pinpl r o n d ~ h o n l n143
~
103 rase sludy tlesisms, 221 -3, 227, Variahlc. bv casc matrix, 185-6 rcspclndent hurdcn, 143
lestin~ ell<brls, 71,fh 253-hll Variance, samplc, 100 clletghts, 7h.3 7h. kHH
Sprri ficatitw, 2114-5, Tl~ick~lcscripti~~n,
2.711 iQoluntay p>rticipilivn mi>sing ci,?t,? bid., 151-2
Spur~oul;rcl,ilinnshipx. 33, 37-'1. 44 'I'irnr orderetl an,~lysis,35, 11: bins, 84
vlahnwt~nnnnatysis, 2LI-I. ZU7 biogmyhy, 251 experiments, K - 4
Sti~ffi~ig cnsrnl;t~rdivs, 752-.3
lnn~itudlnaldesigns, 144 chmnnln~ical,~nnl ysis, 261-2
Standariiiratltin, 157-hl ct~ronoIr)girs,2.52
inllalinr~adl~~slmcnt. 157-0 hislories. 252
inhtruincnth, 17s i ~ i t r - r n i ~ t tinw
c d s~ries,Zhl
~ * ~ c ~ mnk, ~ l t I~V l- hc 1 linir ~ t ~ r iannl\sis.
~'. 2M1-2
prtiynrlion.;, Ih(V-1 twnd onalysi\, 2MI
SC.I I IIS, 157 -h 1 Tmc klng paricl rncrnbl,crs. 13S-? I - l t ~
vnriablt,~. ILK-s cvht, 1111
~-5cnrcs,159 - h l dccl~r%tcrrng, 14 t
Statistical cnntrol5. 162, IW Tr.insk~rntt~d distrjhut~on';, 102
atlju5tinp ftlr mi.~hingdata bias, 152 nor~nnlizcrlcli<hibutiom, 102
cC ~xpeririieritalcontrols, 201 -2 Translorrnrrl variables, 1'17-8
1~1,lhr)ralionanalysis, 313-1 [I Trvilhn~nb
crms scctiunal cicsihms. 1 7 7 4 s r t 7 intervenkions

trlgic of, 201 Tr'nd andysis, 111, 2hn

mi>tct~ing,202 T'ruiication etlects, I 10- I
nu~TIiylr,200 Tu.rrway an.iIysis r,f vasi.inw, 104,
Stnt~stiwlgcncr.~lizatir>n,237, 210, 110
24Y Typcho~ies,225-6, 252
St,~tisficnlinfcrencc, 2110-1, cluskr analysis, 252
i n t ~ r v aestirnnles,
l 201 deductive, 225-h
trsb of significance, 2111 rmpirical. 225, 252
: Shhstical pwrer, 78 ind~ictivc,226
Stntistic,ll mgrc~sim,75, 134, fhenr&ic,~l, 225
cro'rs wctifmal designs, 177
Irrngitudinal dr.;ips, 134 Unit non-rrcpon.;~, 137
St<~ti.sticaE curnmnrics, 97, '19 Units r 3 f ani~lvsix,18
Srlhgrrrup an,~ly.;ih, 1.V-H rrnbrdd~clunit.;, 22i1-1
holi5tic ~ l r ~ iZZn-
k. I
c a w sludirs. 2 0
Unobtr~rsir,~ tnctlr~xlb.237
L!jirt~llal~~litv