Ch.

6: Reliability and Validity

in Measurement and Research

Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall

 Validity and Reliability
 Validity: How well does the measure or
design do what it purports to do?
 Reliability: How consistent or stable is
the instrument?
 Is the instrument dependable?

Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall

 Random and Systematic Error
 Random Error: Chance fluctuations
 Tend to cancel out over repeated
measurements.
 Systematic Error: Fluctuations that are
slanted in a particular direction.
 Also known as “bias”

Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall

 Types of Reliability
 Test-Retest Reliability: Degree of temporal
stability of the instrument.
 Assessed by having instrument completed by
same people during two different time periods.
 Alternate-Forms Reliability: Degree of
relatedness of different forms of test.
 Used to minimize inflated reliability correlations
due to familiarity with test items.

Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall

 Types of Reliability (cont.)
 Internal-Consistency Reliability: Overall
degree of relatedness of all test items
or raters.
 Also called reliability of components.
 Item-to-Item Reliability: The reliability
of any single item on average.
 Judge-to-Judge Reliability: The reliability of
any single judge on average.
Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall
 Reliability, Replication, and
External Validity
 External Validity: The generalizability of
an inferred causal relationship.
 The dependability of causal
generalizations is based on replicable
findings.
 But how know if have replicated a finding?
 Role of the effect size.

Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall

 Validity in Test and Instrument
Construction
 Content Validity: Is the relevant
material adequately sampled?
 Criterion Validity: How well does the
test correlated with outcome criteria?
 Concurrent Validity
 Predictive Validity

Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall

 Construct Validity in Test
Development
 Construct Validity: What does the test
really assess?
 Does the test have the “ability to
discriminate”?
 Establishing construct validity
 Convergent validity
 Discriminant validity

Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall

 Validity and Causal Inference
in Experimental Design
 External Validity
 Construct Validity: The validity of the
hypothetical idea linking the IV and DV.
 Statistical Conclusion Validity: Are the
statistical conclusions well-grounded?
 Internal Validity: Ability to rule out
plausible rival hypotheses.

Rosnow, Beginning Behavioral Research, 5/e. Copyright 2005 by Prentice Hall