CHAPTER 6

EVALUATING SELECTION TECHNIQUES AND DECISION

CHARACTERISTICS OF EFFECTIVE SELECTION TECHNIQUES

 RELIABILITY- is the extent to which a score from a selection measure is stable and free
from error.
-the extent to which a score from a test or from an evaluation is consistent and free from
error
-reliability is an essential characteristic of an effective measure
Test reliability is determined in four ways:
 TEST-RETEST RELIABILITY- the extent to which repeated administration of the
same test will achieve similar results.
-each one of several people take the same test twice. The scores from the first
administration of the test are correlated with scores from the second to
determine whether they are similar. If they are, then the test is said to have
TEMPORAL STABILITY-the consistency of test scores across time
*typical time intervals between test administration range from three days to three months.
Usually the longer the time interval, the lower the reliability coefficient.
Test-retest reliability coefficient= .86
 ALTERNATE-FORMS RELIABILITY- the extent to which two forms of the same test
are similar
o COUNTERBALANCING- a method for controlling order effects by giving half
of a sample Test A first, followed by Test B, and giving the other half of the
sample Test B first, followed by Test A.
-designed to eliminate any effects that taking one form of the test first may
have on scores on the second form. The scores on two forms are then
correlated to determine whether they are similar, if they are the test is said
to be FORM STABILITY (the extent to which the scores on two forms of a
test are similar)
*time interval should be as short as possible
 INTERNAL RELIABILITY- the extent to which responses to the same test items are
consistent.
FACTORS THAT CAN AFFECT INTERNAL RELIABILITY:
o INTERNAL CONSISTENCY- the extent to which similar items are answered
in similar ways and measures
METHODS USE TO DETERMINE INTERNAL CONSISTENCY
1. SPLIT-HALF METHOD- a form of internal reliability in which the
consistency of item responses is determined by comparing
scores on half of the items with scores on the other half of the
items.
-easiest to use as items on a test are split in to two groups. The
scores on the two groups of items are then correlated. Because the
 number of items has been reduced, researchers have to use
SPEARMAN-BROWN PROPHECY FORMULA to adjust the correlation
2. CRONBACH’S COEFFICIENT ALPHA- a statistic used to
determine internal reliability of test that use interval or ration
scales
3. KUDER-RICHARDSON FORMULA 20 (K-R 20)- a statistic used to
determine internal reliability of tests that use items with
dichotomous answers(yes/no, true/false)

o ITEM STABILITY- the extent to which responses to the same test items are
consistent
o ITEM HOMOGENIETY-the extent to which test items measures the same
construct
 SCORER RELIABILITY- the extent to which two people scoring a test agree on the
test score, or the extent to which a test is scored correctly.
-scorer reliability is specially an issue in projective or subjective tests in which
there is no one correct answers, but even test scored with the use of keys suffer
from scorer mistakes.
 VALIDITY- the degree to which inferences from test scores are justified by the evidence.
*if a test has poor reliability, it cannot have high validity
FIVE STRATEGIES TO INVESTIGATE VALIDITY OF SCORES ON TEST
 CONTENT VALIDITY- the extent to which test or test items sample the content that
they are supposed to measure.
*one way to test the content validity of a test is to have subject matter experts
rate test item on the extent to which the content and level of difficulty for each
item are related to the job in question.
 CRITERION VALIDITY-the extent to which a test score is related to some measure
of job performance.
-the extent to which a test score is statistically related to some measure of job
performance called CRITERION (a measure of job performance such as
attendance, productivity, or a supervisor rating)
CRITERION VALIDITY IS ESTABLISHED USING TWO RESEARCH DESIGNS:
1. CONCURRENT VALIDITY- a form of criterion validity that correlates
test scores with measures of job performance for employees
currently working for an organization.
-a test is given to a group of employees who are already on the job.
The scores on the test are then correlated with a measure of the
employees’ current performance
2. PREDICTIVE VALIDITY- a form of criterion validity in which test
scores of applicants are compared at a later date with a measure
of job performance
 -a test is administered to a group of job applicants who are going
to be hired. The test scores are the compared with a future
measure of job performance
o VALIDITY GENERALIZATION (VG)- the extent to which inferences from test
scores from one organization can be applied to another organization
-the extent to which a test found valid for a job in one location is valid for the same
job in a different location
BUILDING BLOCKS OF VALIDITY GENERALIZATION ARE META-ANALYSIS AND JOB
ANALYSIS
o SYNTHETIC VALIDITY- a form of validity generalization in which validity is
inferred on the basis of a match between job components and test previously
found valid for those job components
-is based on the assumption that test that predict a particular component of one
job should predict performance on the same job component for a different job.
 CONSTRUCT VALIDITY-the extent to which a test actually measures the construct
that it purports to measure
o CONVERGENT VALIDITY
o DISCRIMINANT VALIDITY
o KNOWN-GROUP VALIDITY- a form of validity in which test scores from two
contrasting groups “known” to differ on a construct are compared.
 FACE VALIDITY- the extent to which a test appears to be valid
-the extent to which a test appeared to be job related
BARNUM STATEMENTS- statements so general that they can be true of
almost everyone.

MENTAL MEASUREMENTS YEARBOOK- a book containing information about the reliability and
validity of various psychological test

ESTABLISHING THE USEFULNESS OF A SELECTION DEVICE

 TAYLOR RUSSEL TABLES- a series of table based on the selection ratio, base rate
and test validity that yield information about the percentage of future employees
who will be successful if a particular test is used
-provide an estimate of the percentage of total new hires who will be successful
employees if a test is adopted
 SELECTION RATIO- the percentage of applicants an organization hire
 BASE RATE- percentage of current employees who are considered successful
 PROPORTION OF CORRECT DECISIONS- a utility method that compares the
percentage of times a selection decision was accurate with the percentage of
successful employees.
 LAWSHE TABLES- tables that use the base rate, test validity, and applicant
percentile on a test to determine the probability of future success for that
applicant
 UTILITY FORMULA- method of ascertaining the extent to which an organization
will benefit from the use of a particular selection system
  TENURE- the length of time an employee has been with an organization

DETERMINING THE FAIRNESS OF A TEST

 MEASUREMENT BIAS- group differences in test scores that are unrelated to the construct
being measured
 ADVERESE IMPACT- an employment practice that results in members of a protected class
being negatively affected at a higher rate than members of the majority class. Adverse
impact is usually determined by the four-fifth rule
 PREDICTIVE BIAS- a situation in which the predicted level of job success falsely favors one
group over another
o SINGLE GROUP VALIDITY-the characteristic of a test that significantly predicts a
criterion for one class of people but not for another
o DIFFERENTIAL VALIDITY- the characteristic of a test that significantly predicts a
criterion for two groups, such as both minorities and nonminorities, but predicts
significantly better for one of the two groups
*REMEMBER: with single group validity, the test is valid only for one group. With differential
validity, the test is valid for both groups, but it is more valid for one than for the other.

MAKING THE HIRING DECISION

 MULTIPLE REGRESSION-a statistical procedure in which the scores from one criterion-
valid test are weighted according to how well each test score predicts the criterion
 TOP-DOWN SELECTION- selecting applicants in straight rank order of their test scores
 COMPENSATORY APPROACH- a method of making selection decision in which a high score
on one test can compensate for a low score on another test.
 RULE OF THREE- a variation on top-down selection in which the names of the top three
applicants are given to a hiring authority who can then select any of the three
 PASSING SCORE- the minimum test score that an applicant must achieve to be considered
for hire
 MULTIPLE CUTOFF APPROACH- a selection strategy in which applicants must meet or
exceed the passing score on more than one selection test
 BANDING- a statistical technique based on the standard error of measurement that allows
similar test scores to be grouped
 STANDARD ERROR OF MEASUREMENT- the number of points that a test score could be
off due to test unreliability