Академический Документы
Профессиональный Документы
Культура Документы
Overview
Content validity Criterion-related validity
Issues of Validity
Does the test actually measure what it is purported to measure? Do differences in tests scores reflect true differences in the underlying construct? Are inferences based on the test scores justified?
Types of Validity
Content Criterion-based Construct
Construct (general evidence gathering)
Example: Depression
(Modified from the DSM IV)
A complex of symptoms marked by: Disruptions in appetite and weight Insomnia or hypersomnia Loss of interest or pleasure in activities Loss of energy Feelings of worthlessness Feels sad or empty nearly everyday Frequent deathrelated thoughts
Describe the content domain Determine the areas of the content domain that are measured by each item Compare the structure of the test with the structure of the content domain
Challenges:
Difficulty in defining the domain Categorizing the content domain and map items to the categories Ensure representativeness
Measure Contamination
What do we want?
A measure that samples from all important domains or aspects (Low Deficiency) A measure that does not include anything irrelevant (Low Contamination) That is, a measure that adequately captures all of the domains of the construct that it is intended to measure. (High Content Validity)
Job Performance
Job Performance
Job Performance
Measurement Error
Reliability Index of the presence of measurement error (1.0 reliability = No error) Unreliability in the predictor and criterion serves to reduce (attenuate) their observed correlation Researchers are often concerned about attenuation in predictor-criterion associations
Assume that measures of X and Y have alphas of .60 and .70, respectively. The observed r between X and Y is .40. However, we might want to know how much this correlation is depressed by measurement error.
.90
.90
.40
.44
Predictor-Criterion Overlap
Same items on both measures bad!