Human Performance

ISSN: 0895-9285 (Print) 1532-7043 (Online) Journal homepage: http://www.tandfonline.com/loi/hhup20

Order Effects in Personnel Decision Making

Scott Highhouse & Andrew Gallo

To cite this article: Scott Highhouse & Andrew Gallo (1997) Order Effects in Personnel Decision
Making, Human Performance, 10:1, 31-46

To link to this article: http://dx.doi.org/10.1207/s15327043hup1001_2

Published online: 13 Nov 2009.

Submit your article to this journal

Article views: 154

View related articles

Citing articles: 15 View citing articles

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=hhup20

Download by: [Mr John Track] Date: 07 August 2016, At: 05:33
 HUMAN PERFORMANCE, 1O( l), 3 1 4 6
Copyright Q 1997, Lawrence Erlbaum Associates, Inc.

Order Effects in
Personnel Decision Making
Scott Highhouse
Department of Psychology
Downloaded by [Mr John Track] at 05:33 07 August 2016

Bowling Green State University

Andrew Gallo
Department of Psychology
Indiana University-Purdue University Indianapolis

Research on primacy versus recency effects in personnel decision making was

reconsidered within the framework of the belief-adjustment model of Hogarth and
Einhorn (1992) in this article. The model posits that primacylrecency is a function of
the interaction of various features of the judgment task. On the basis of this model,
we predicted that recency effects will predominate in a simulated work sample,
regardless of whether ratings are made after each work-sample exercise or only at the
end of the process. One hundred seventeen university students received rater training
and observed a candidate's performance in 2 role-play exercises. The conditions
differed in terms of exercise order (positive-negative performance vs. negative-posi-
tive performance) and response mode (step-by-step vs. end-of-sequence). Results
were consistent with predictions. In addition, evidence was found for (within-target)
contrast effects in the work-sample ratings. Implications of the model for selection
and performance evaluation judgments are discussed.

When a person's judgment is sensitive to the order in which evidence is presented,

an order effect has occurred (Asch, 1946). O n e type of order effect, known as a
primacy effect, occurs when judges are disproportionately influenced by evidence
presented earlier rather than later. Another type of order effect, known as a recency
effect, occurs when judges are disproportionately influenced by evidence presented
later rather than earlier.

Requests for reprints should be sent to Scott Highhouse, Department of Psychology, Bowling Green
State University, Bowling Green, OH 434034228, E-mail: shighho@bgnet.bgsu.edu.
 32 HIGHHOUSE AND GALL0

Little is known about when and if either primacy or recency effects are likely to
occur in personnel decision making. A sampling of introductory textbooks in
industriaYorganizational (UO) psychology reveals that there is little consensus on
this issue. For example, some U 0 texts argue that primacy effectspredominate (e.g.,
Dipboye, Smith, & Howell, 1994, p. 388; Smither, 1994, p. 83), another only
mentions recency effects (Miner, 1992), and still another concludes that suscepti-
bility to primacy versus recency is an individual difference factor (Muchinsky,
1993, p. 117). Landy (1989, p. 206) concluded that because some studies show
primacy effects and others recency effects, information order does not really matter.
The confusion surrounding this issue is not surprising given the lack of research
Downloaded by [Mr John Track] at 05:33 07 August 2016

and mixture of findings in the research on order effects. For example, Farr's studies
on order effects in the employment interview (Farr, 1973;Farr & York, 1975)found
that ratings of a transcribed interview showed recency effects when participants
used a repeated-judgment response mode (i.e., when multiple evaluations were
elicited during the presentation of performance information),and showed primacy
effects when participants made only one final-overdl judgment about candidate
suitability. London and Hake1 (1974) also found recency effects for interview
information using a repeated-judgment response mode. Steiner and Rain (1989)
showed participants videos of teaching performance and found recency effects on
performance evaluations regardless of whether the videos were presented all at once
or over a period of days.
The confusion about order effects has not been unique to personnel research.
Hogarth and Einhorn (1992) reviewed 76 order-effect studies from several areas of
investigation including jury decision making, attitude change, psychophysics, and
others. Of the 76 studies identified by the authors, 36 found primacy effects, 35
found recency effects, and 5 found no effect from order of presentation. Clearly, ~t
would be incorrect to conclude that one effect (i.e., primacy or recency) predomi-
nates. or that order effects are irrelevant.

BELIEF-ADJUSTMENTMODEL

In response to the inconsistencies in findings on order effects, Hogarth and Einhorn

(1992) developed a belief-updating model. Their belief-adjustment model is an
anchoring and adjustment model that recognizes the role of task characteristics as
moderators of the effects of order on judgment. The three task characteristics
considered by Hogarth and Einhorn's model are task length (i.e., number of pieces
of evidence), task complexity (i.e., amount of information in the evidence items),
and response mode (i.e., step-by-step [Sbs] vs. end-of-sequence [EOS]).' To under-
stand the distinction between complexity and length, for example, one can imagine

h model also makes a d~stlnctronbetween discrete (e g , true-false) and contmuous

' ~ l t h o u ~the
appraisals, we only considered continuous-ratingjudgments in this investigation
 ORDER EFFECTS 33

a trial in which the jury is presented with a few (vs. many) pieces of evidence in
which some pieces are more complex (e.g., DNA evidence) than others (e.g.,
eyewitness evidence).
According to the belief-adjustment model, an individual's current belief serves
as an anchor that is adjusted on the basis of new evidence. This revised belief then
becomes the anchor for the next adjustment, and so on. This is the typical SbS
judgment process that occurs in belief updating. When evidence is inconsistent(i.e.,
positive and negative), more weight is attached to a piece of evidence that comes
second in a sequence rather than first. This results in a tendency toward recency in
judgments. Occasionally,however, the task requires that judgment be reserved until
the end of evidence presentation. Whether primacy or recency occurs under this
Downloaded by [Mr John Track] at 05:33 07 August 2016

EoS response mode depends on the length and complexity of the evidence series.
Table 1 shows the belief-adjustment model's order-effect predictions based on
the interaction of response mode (i.e., SbS vs. EoS) and task characteristics. Note
that, for the SbS response mode, recency is predicted until an individual is faced
with a task involving a long series of evidence. In this case, individuals can become
overloaded with information and less sensitive to the impact of new information.
Thus, information presented earlier is disproportionately weighted in the ultimate
judgment. Table 1 also shows that the predictions for the EoS response mode differ
only in the case where a task is short and simple. Here, individuals accord more
weight to the initial anchor in making an ultimate judgment. As information
becomes more complex, however, EoS individuals resort to a SbS or on-line
judgment strategy because of the demands on information processing and memory.
Thus, for a short series of complex information, recency is predicted regardless of
response mode.
It is useful to examine previous research on order effects in personnel decision
making within the framework of the belief-adjustment model. For example, Farr
(1973; Fan & York, 1975) found recency effects for interviewers making SbS
responses and primacy effects for those making EoS responses. In these studies,

TABLE 1
Belief-Adjustment Model Order Effect Predictions

Response Mode

EoS SbS

Short series
Simple Primacy Recency
Complex Recency Recency
Long series Force toward primacy Force toward primacy

Nore. Predictions for continuous-rating tasks only. EoS = end-of-sequence;

SbS = step-by-step.
 34 HIGHHOUSE AND GALL0

participants were presented with between 8and 16items from Hakel and Dunnette' s
(1970) Checklistfor Describing Job Applicants. Items in this checklist range from
one-word adjectives (e.g., "enthusiastic," "manipulative") to short behavioral state-
ments (e.g., "Says he prefers to work alone"; '"Talked too d o ~ l ~ "This ) . ~is very
similar to the prototypical short series of simple information described in Hogarth
and Einhorn (1992, p. 7). London and Hakel (1974) used a similar method that
involved reading three-part candidate self-descriptions. All participants made SbS
responses, rating the candidate after each "set" of (positive or negative) information.
London and Hakel found recency effects on overall interview ratings using this
procedure. Both F a d s (1973; Farr & York, 1975) and Landon and Hakel's (1974)
findings are consistent with the bellef-adjustment model's predictions for judgment
Downloaded by [Mr John Track] at 05:33 07 August 2016

tasks having these characteristics (Table 1). However, many selection decisions are
based on performance in tasks different from those utilized in these studies.

BELIEF UPDATING AND

WORK-SAMPLE PERFORMANCE

Many personnel professionals utilize work samples as a part of the employee

selection process. In many cases, applicants engage in work-related role plays used
to measure a set of personal qualities and abilities associated with successful
performance on the job. For example, role plays are used in assessment centers
(Thornton & Byham, 1982), individual assessment programs (Ryan & Sackett,
1992), and even in some structured interviews (e.g., Pursell, Campion, & Gaylord,
1980).
Considerable research on the assessment center has found that ratings on one
ability in one work-sample exercise are related less than would be expected with
ratings on the same ability in another work-sample exercise (e.g., Bycio, Alvares,
& Hahn, 1987;Sackett & Dreher, 1982).Many authors have argued that this finding
has more to do with candidate performance variability from exercise to exercise
than with rater unreliability (e.g., Highhouse & Harris, 1993; Lance, Newbolt, &
Gatewood, 1995; Neidig & Neidig, 1984; Schneider & Schmitt, 1992). Given that
candidate performance appears to vary from exercise to exercise in work samples,
it would seem worthwhile to understand the impact of exercise order on evaluations
of candidates. For example, is negative performance more damaging to a candi-
date's overall rating when it occurs in the first role-play exercise or when it occurs
in the last? The order in which exercises are seen or discussed is often arbitrary and
should be irrelevant to the ultimate outcome of the evaluation process.
Most work samples present evaluators with the task of integrating complex
behavioral information obtained from a short series of work simulations. This
classification of work samples as a short series of complex evidence is consistent

'we are grateful to Marvin D. Dunnette for sending us samples of this work
 ORDER EFFECTS 35

with Hogarth and Einhorn's (1992) classification criteria. According to Hogarth

and Einhorn, the decision to classify a task as simple or complex is based on the
amount of information contained within each piece of evidence, and the decision
to classify a task as long or short is based on the number of pieces of evidence.' We
classify work samples as complex because they typically involve making judgments
about attributes from a large amount of behavioral evidence and integrating these
judgments into an overall rating. This involves much more than making a rating
based on some experimenter-presented attributes. Moreover, we classify work
samples as short because the number of work samples involved in a typical
assessment center or individual assessment program does not put the demands on
attention and memory that a long series of evidence, such as that involved in a 6-
Downloaded by [Mr John Track] at 05:33 07 August 2016

or 1Zmonth performance evaluation, does.

Note from Table 1 that the belief-adjustment model predicts recency for this
type of judgment task, regardless of response mode. That is, recency is always
associated with complex, short tasks. Hogarth and Einhorn (1992) pointed out that
their predictions for this type of evidenceare the most novel of the belief-adjustment
model's predictions. This type of evidence has also been studied seldomly in the
past. With regard to personnel decision making, the predictions of the belief-ad-
justment model are inconsistent with models that predict primacy effects. These
include confirmation-bias models where assessors are said to disproportionately
attend to information consistent with initial impressions (e.g., Sackett, 1982), and
attention-decrement models where raters are thought to pay less attention to later
information unless a repeated-judgment process is employed (e.g., Farr & York,
1975).The belief-adjustment model prediction is also inconsistent with models that
predict an over-weighting of negative information (e.g., Constantin, 1976; Rowe,
1989). According to these negative-attention models, information order alone
should have no effect on ratings.
In sum, past research on order effects in personnel decision making has resulted
in a mixture of findings and has lacked an overall guiding framework. Moreover,
no previous research has examined order effects on work-sample performance
judgments. Hogarth and Einhorn's (1992) belief-adjustment model presented a
conceptual framework that predicts primacy or recency as a function of the
interaction of various features of the judgment task. The model appears to predict
recency main effects for exercise order on work-sample ratings, but no interacting
effects for response mode. The present experiment was designed to test these
predictions by crossing response mode with work-sample exercise order.

'studies classified by Hogarth and Einhom (1992) as shodcomplex included an experiment that
used two 600-word messages, and one that used two long audio tapes (i.e., one of the defense and one
of the prosecution) of evidence in a mock trial. Studies classified as long/complex included one that
was conducted over a period of 12 days, and one presenting 17 arguments containing approximately
175 words each.
 METHOD

Design

In groups of 3 to 14, participants were trained as raters for two role-play work-sam-
ple exercises taken from a commercial assessment center. The participants were
also shown video tapes of the same candidate performing in each of the two
exercises, and were asked to evaluate the candidate's suitability for a middle-level
management position. Performance in one of the taped exercises was positive, and
in the other exercise it was negative. The &sign was a 2 x 2 (positive-negative vs.
Downloaded by [Mr John Track] at 05:33 07 August 2016

negative-positive order x SbS vs. EoS response mode) between-subjects factorial.

Participants

Participants were 117 undergraduate studeats recruited from psychology classes in

a large, midwestern, urban university. The number of participants per cell ranged
from 27 to 33. Statistical power to detect a .SO standard deviation difference
between groups was 72% (Cohen, 1988).Fifty-two percent of the participants were
women. The participants ranged in age from 18 to 44 years, with an average age of
23. Eighty-three percent were currently employed either full- or part-time.

Materials

Role-play exercises. Two exercises were chosen from an assessment center

used by a large international corporation to select internal and external candidates
for a middle-level management position. This assessment center is not unusual in
that it was commercially produced and has been shown to produce ratings exhibkting
the typical exercise-factor solution (Highhouse & Harris, 1993). The role-play
exercises included a fact-finding exercise and a sales-call exercise.
In the fact-finding role play, a candidate assumes the role of a newly appointed
customer service advisor. The candidate is given 5 min to review circumstances
surrounding a customer's attempt to return a piece of merchandise. The candidate
then meets individually with a resource person to inquire about any facts that might
be relevant for making a decision concerning the return. The candidate then reports
an ultimate decision to the assigned assessor, along with a rationale.
In the sales-call role play, a candidate assumes the role of a new salesperson for
a food-product manufacturer. As part of this role, the candidate is responsible for
preparing and making a simufslted sales phone call to a buyer for a retail chain. The
candidate's objective is to have the buyer accept the pilot program. The candidate
is given time to prepare for the assignment, after which a sales call is made to an
assessor playing the role of a buyer.
 ORDER EFFECTS 37

Stimulus tapes. The actor who played the target candidate was a 30-year-old
male undergraduate student whose regular job was as a recruiter for a large
transportation company. The actor was instructed to perform poorly in the fact-
finding exercise and to perform as well as he could in the sales-call exercise. He
was paid for his performances.
As part of his negative performance in the fact-finding exercise, the actor leaned
forward in his chair, spoke with his hand covering his mouth, and asked the resource
person inappropriate questions (e.g., "What do you think I should do?')).Also, the
final decision of the actor was to refund all of the money requested by the customer,
despite the fact that the customer bought the product for a lesser amount of money.
As a rationale for this decision, the actor argued that the customer would otherwise
Downloaded by [Mr John Track] at 05:33 07 August 2016

complain and "it's gonna end up coming back on me, and I'm gonna end up losing
my job." The work sample lasted approximately 7 min. In piloting this video tape,
Scott Highhouse showed it to students in an undergraduate psychology class (n =
12) as an example of work-sample performance. The studentsrated the performance
as below average to poor (M = 1.6 on a 5-point scale), but none admitted knowing
that the person in the tape was acting.
For the sales-call exercise, the actor was given the role-play materials well in
advance of the taping. Also, he had all of the relevant sales points in front of him
(out of the sight of the viewer). During the taping, the actor assumed a very
professional tone and hit all of the relevant product information during the sales
pitch. The actor completed the exercise in 8.5 min. Pilot ratings were obtained from
undergraduate psychology students (n = 17) asked to rate the overall performance
on a 5-point scale. This pilot group rated the performance in the sales-call exercise
as above average (M = 3.9).

Training tape. We developed three assessor training segmentson tape to help

standardize the training sessions across experimental conditions. A 55-year-old
male graduate student in the school of education was paid to deliver the training
sessions on tape. The exercise training segments were developed to provide a
common frame of reference for the novice assessors. The first segment included a
discussion of dimensions, definitions, and rating anchors. The second and third
segments focused on the individual exercises. Assessors were provided with
behavioral checklists to use during the observation of the focal exercises. Training
for the fact-finding exercise described the background for the role play and
concentrated on the facts that the candidate could uncover from the resource person.
Training for the sales-call exercise described the background for this role play and
concentrated on the selling points the candidate could use in the presentation.
Participants had the training materials in front of them to which the trainer referred
periodically. Each of the segments ended with a review of the dimensions and
definitions and emphasized the need to rate the candidate on the dimensions and
not on the number of facts uncovered or selling points utilized.
 38 HIGHHOUSE AND GALL0

Dimensions and rating form. We chose only dimensions that the develop-
ers of the role plays indicated were measured well by both exercises. These included
Tolerance for Stress, Oral Communication Skill, Sensitivity, Analysis, and Judg-
ment4 The form also included an Overall Performance rating. Ratings were made
on a 5-point scale ranging from 1 (poor)to 5 (excellent).

Procedure

Upon arrival, participants were alternately assigned to one of two rooms. All
participants in one room received the negative-positive order condition, whereas
Downloaded by [Mr John Track] at 05:33 07 August 2016

all participants in the other room received the positive-negative order condition.
The response mode conditions (i.e., SbS vs. EoS) were alternated between experi-
mental sessions, with the first session being determined by a coin flip. Experiment-
ers were alternated among the order conditions.
Participants received a handout describing the purpose of the research and the
training materials. They were told: "Your task is to observe the video-taped
performance of one job candidate who participated in two role-play work samples.
This candidate wants to be hired by a large beverage company as their Wholesaler
Inventory Coordinator." The participants were presented with the following job
description:

The Wholesaler Inventory Coordinator (WIC) is a middle-level manager employed

by a beverage manufacturing company. The WIC is responsible for interacting with
independent wholesalers who depend on him or her to take orders and ship the product
as soon as possible. The WIC must ensure that inventory is at the appropriate level,
deal with wholesaler complaints, and occasionally convince wholesders to purchase
additional or different product lines. Therefore, the WIC must possess tolerance for
stress, oral communication skill, sensitivity for wholesaler needs, ability to analyze
information, and good judgment.

Participants then watched the first two segments of the training video (i.e., discus-
sion of dimensions and specific instructions for rating one of the exercises). The
training video prompted participants to follow along by reading the summary of
the background information and a checklist of points that could be (un)covered by
the candidate.
After observing the candidate's performance in the first exercise, participants
were shown the third segment of the training video, which provided specific
instructions for rating the other exercise. After observing the first exercise, partici-
pants in the SbS conditions (only) were instructed to evaluate the candidate's overall

4 ~ h e s dimensions
e and definitions can be found in Highhouse and Harris (1993)
 performance on each dimension and then place the completed form in an envelope
provided to them. After both exercises were shown, all participants completed a
final evaluation of the candidate's performance on each dimension (i.e., across both
exercises) and the assessor reaction questions. For this final evaluation form, the
experimentersemphasized the importanceof rating the candidate on the dimensions
across both exercises. Thus, the SbS participants rated the candidate's overall
performance twice, whereas the EoS participants rated his overall performance
once. Participantsin the SbS condition were also asked to make ratings of the second
work-sample exercise alone. The reason for doing this was because this procedure
is commonly used in making assessment-center ratings, and because it provided
within-subject ratings of the exercises that could be used in later analyses.
Downloaded by [Mr John Track] at 05:33 07 August 2016

RESULTS

Manipulation Check

It is possible to establish that the positive and negative performance videos were
evaluated as expected by examining the effects of order on the first-step ratings of
participants in the SbS rating condition. The results of a multivariate analysis of
variance (MANOVA) on the overall performance rating and five dimension ratings
revealed that the target's performance in the sales-call exercise (i.e., positive-per-
formance exercise) was seen as significantly different from performance in the
fact-finding exercise (i.e., negative-performance exercise), F(6, 55) = 19.10, p <
.01. Table 2 presents the mean (first) ratings by exercise, along with univariate
statistics. Inspection of Table 2 reveals that performance in the sales-call exercise

TABLE 2
First Ratings of SbS Subjects by Exercise

Exercise

Sales Call Fact Finding

Dimension M SD M SD FI
Tolerance for stress 3.9 ,233 2.4 .70 56.51
Oral communication 3.1 .96 2.0 .73 26.13
Sensitivity 3.7 .90 2.9 .94 11.87
Analysis 4.0 .78 2.2 1.05 54.25
Judgment 3.7 .76 2.0 1.02 55.71
Overall performance 3.7 .66 2.0 .64 101.28

Note. SbS = step-by-step.

*All univariate Fs are significant at p < .001
 40 HIGHHOUSE AND GALL0

was rated higher on all dimensions than performance in the fact-finding exercise.
Moreover,all of the mean ratings for the sates-call exercise were above the midpoint
of the scale, whereas all of the mean ratings for the fact-finding exercise were below
the midpoint. Thus, it is reasonable to conciude that performance in the exercises
was perceived as expected.

Impact of Order and Response Mode on Ratings

MANOVA was used to determine order, response mode (SbS vs. E d ) , and
interaction effects on the final set of dimension ratings and overall performance
Downloaded by [Mr John Track] at 05:33 07 August 2016

rating. These results indicated a statisticdly significant effect for information order,
F(6, 108) = 2.32, p < .05. This effect accounted for 11% of the variance in the set
of ratings. No effect was found for response mode, F(6, 108) = 1.7,p = .12, nor the
Order x Response Mode interaction term, F(6, 108) = 0.77, p = .60. Table 3 shows
means and standard deviations by information order, along with univariate statis-
tics. Table 3 shows that the multivariate order effect was due primarily to the effect
of order on ratings of Oral Communication Skill and Overall Performance. This is
not surprising, given that oral communication is such an important component of
performance for both exercises. Inspection of the cell means in Table 3 reveals that
participants presented with the exercises in the positive-negative order rated the
candidate significantly lower on oral communications and overall performance than
participants presented with the exercises in the negativepositive order. This is
consistent with a recency effect. Note also that the trend across each of the
dimensions is in the direction of recency. It is also notable that mean overall
performance ratings are above the midpoint (i.e., "average") far the negative-posi-
tive group and below the midpoint for the positivenegative group.

TABLE 3
Final Rating Means and Standard Deviations by Exercise Order
and Univdate statistics
- -

Exercise Order

Dimension

Tolerance for stress

Oral communication
Sensitivity
Analysis
Judgment
Overall performance
 ORDER EFFECTS 41

Exploratory Analyses

Assimilation versus contrast. The design of this study allowed for the
opportunity to add another data point to the assimilationlcontrast debate (cf. Foti
& Hauenstein, 1993). Specifically, our design allowed us to contrast ratings (i.e.,
for the SbS raters) of the stimulus performance in the absence of prior performance
impressions with ratings of the same performance in the context of inconsistent
prior performance. That is, we had a no-previous-performance control group.
Previous assimilation/contrast research in personnel decision making has been
criticized for failing to include a true control group (Kravitz & Balzer, 1992).
Figure 1 shows the three SbS overall performance ratings by exercise order.
Downloaded by [Mr John Track] at 05:33 07 August 2016

Note that the two overall-performancerating points on the final assessment repre-

-0- Negative-Positive

+ Positive-Negative

1- -- I I

Position #I Position #2 Final Assessment

Rating Step

FIGURE 1 SbS overall performance ratings by exercise order

 42 HIGHHOUSE AND GALL0

sent the recency effect. Ass~milationor contrast is identified by examining the

distance between the mean rating of the positive video in the first versus second
position, or the distance between the rating of the negative video in the first versus
second position. For the positive video, this difference is statistically significant,
t ( l , 6 0 )= 2.10,p < .05. That is, the positive video was rated more positively when
it followed the negative video (M = 4.09) than when it was presented in the first
position (M = 3.69).For the negative video, however, the difference does not reach
statistical significance, t (1.60)= I .61, p = .lo, although this difference is in the
direction of a contrast effect. That is, the negative video was rated more negatively
when it followed the positive video (M = 1.69) than when it was presented in the
Downloaded by [Mr John Track] at 05:33 07 August 2016

first position (M = 2.03).

DISCUSSION

This study adds to a growing body of research successfully applying Hogarth and
Einhorn's (1992)belief-adjustment model to real-world problems (e.g., Adelman,
Tolcott, & Bresnick, 1993; Ashton & Ashton, 1990; Johnson, 1995; Tubbs, Gaeth,
Levin, & Van Osdol, 1993).Consistent with the predictionsof the belief-adjustment
model, we found recency effects on work-sample ratings, regardless of response
mode. Participants who observed the negative-performance exercise second rated
the candidate's oral communication skill and overall performance lower than those
who observed the negative-performance exercise first. This recency effect was
observed for those instructed to make repeated overall evaluations (i.e., after each
exercise), as well as those instructed to make only one overall evaluation. The
failure to find evidence for an Order x Response Mode interaction is not consistent
with Fan- and York's (1975) finding of recency for SbS participants and primacy
for EoS participants. We argue that this is because the information presented in our
work-sample simulation is more complex than the "paper-people" interview simu-
lation used by Farr and York. According to this argument, both the present results
and Fan- and York's results are consistent with the belief-adjustment model.
However, we would hesitate to conclude that the recency effects found for a
work sample are different than order effects that would be found in an employment
interview. That is, most interviews involve more than evaluating short, externally
presented descriptions of candidates. In fact, some interviews are quite elaborate
and occur more than once before an employment decision is made. Thus, it 1s
possible that, like work samples, most interviews are complex and subject to
recency effects regardless of response mode. Along these same lines, our results
are very similar to the results of Steiner and Rain (1989).Steiner and Rain showed
participants four 7-min videotapes of instructor lectures.They found recency effects
regardless of whether the videos were shown all at once or over a period of 4 days.
Although Steiner and Rain generalized their results to performance appraisals, we
believe that their rating task was actually more similar to that which is encountered
 ORDER EFFECTS 43

in an assessment center or work sample. Most performance evaluation tasks require

the collection of a greater number of performance incidents over a much longer
period of time. Under such circumstances, the belief-adjustment model predicts
primacy effects. Thus, with the emphasis placed on task characteristics by the
belief-adjustment model, the application of the model to personnel decision making
requires close attention to ecological validity.
The SbS versus EoS response mode distinction is also particularly relevant to
assessment centers. In the traditional AT&T-style assessment center (Bray & Grant,
1966), assessors make overall dimension ratings after all exercises have been seen
and discussed (i.e., EoS). An alternative to this method has been to provide a rating
for each dimension after observation andlor discussion of each exercise (i.e., SbS).
Downloaded by [Mr John Track] at 05:33 07 August 2016

This is followed at the end with overall dimension ratings (e.g., Sackett & Dreher,
1982). The results of our investigation suggest that neither practice is likely to
eliminate recency effects.
From an applied perspective, it is interesting that participants presented with the
negative-performance exercise last rated the candidate's performance below aver-
age, whereas participants presented with the positive-performance exercise last
rated the candidate's performance as above average. Thus, the order in which the
exercises were observed influenced not only the magnitude of performance ratings
but also the judgment of the suitability of this candidate for the job. We suspect
that assessor training, if it mentions primacy versus recency at all, gives equal
(minimal) attention to the potential effects of both primacy and recency on assessor
judgments. Given the propositions of the belief-adjustment model and the results
of this experiment, we believe that assessor training should pay relatively more
attention to the potential for recency effects on work-sample assessments. In
addition to assessor training, another method of potentially reducing the effects of
recency on judgments of work-sample performance is to videotape candidate
performance so that assessors can review earlier performance evidence. Alterna-
tively, the use of mechanical, rather than intuitive, methods for combining individ-
ual exercise ratings into an overall performance rating would likely be the most
effective way of eliminating order effects on overall ratings (cf. Kleinmuntz, 1990).
With that said, it is important to point out the limitations of our investigation as
representative of work-samplejudgments. It is possible that our findings are limited to
work samples with a small number of exercises because only two role plays were used.
Also, the training, observation, and evaluation all took place within a I-hr block of time.
Most assessment centers, for example, are spread out over a longer period of time
(Thornton & Byham, 1982).However, it is notable that Steiner and Rain (1989) found
the same recency effects using more pieces of evidence spread out over 4 days.
Perhaps the greatest limitation of our study was the lack of incentives for raters
to provide accurate ratings. That is, the raters were not involved in an actual hiring
process and, therefore, experienced little risk for making a poor evaluation. How-
ever, we suspect that order effects are not likely to be as motivationally driven as
 44 HlGHHOUSe AND GALL0

other rater errors such as leniency. Moreover, just as one might argue that lack of
motivation may lead to recency, one might alternatively argue that lack of motiva-
tion could lead to primacy effects via attention decrement.
Another interesting finding in this study was the contrast effect observed for the
positive-performance video. Participants who viewed the positive video after the
negative one rated the target higher than participants who viewed the positive video
first. This result is consistent with the contrast effects observed in Gaugler and
Rudolph's (1992) investigation of assessmentcenterjudgments. It is also consistent
with within-ratee contrast effects observed in performance evaluation (Murphy,
Balzer, Lockhart, & Eisenman, 1985). The finding is not, however, consistent with
Downloaded by [Mr John Track] at 05:33 07 August 2016

studies finding evidence for assimilation (e.g., Foti & Hauenstein, 1993;Steiner &
Rain, 1989).It is notable that the examplesof performance that we used were clearly
identified by participants as representing either good or poor performance. We
agree with Foti and Hauenstein (1993) that the confidence in which one holds an
initial judgment has a lot to do with wh&r assimilation or contrast will be found.
According to Hogarth and Einhorn (1992), extremity of contrast effects is directly
related to the extremity of the initial anchor, or "the bigger the anchor, the harder
it will fall" (p. 14).

CONCLUSIONS AND FUTURE RESEARCH

This study used the belief-adjustment model to make sense of previous research on
primacy and recency in personnel decision making. The recency effects observed
for the role-play work samples were consistent with the model's predictionsfor this
type of evidence. We believe that future research on order effects in personnel
decision making would benefit from using this model as a theoretical framework.
For example, the model predicts primacy effects for a short series of simple
information only when an EoS response mode is utilized. Consistent with this
prediction is research finding evidence for primacy effects on job evaluations
(Smith, Benson, & Hornsby, 1990). We might similarly expect primacy for pre-
screening of resumes or applicant files. The belief-adjustment model also predicts
a "force toward primacy" when the task involves a long series of evidence. Thus,
the model would appear to predict primacy for a typical performance evaluation or
promotion decision. Clearly, research is lacking that examines order effects on
judgments based on multiple pieces of evidence (e.g., critical incidents) spread over
a long period of evaluation.

ACKNOWLEDGMENTS

We are grateful to John Hazer for helpful comments and to Beth Huff, Michael
Johnson, Sarah Melner, Shirley Sullivan, Eric Vincent, and Payam Yiice for their
assistance in stimulus development andlor data collection.
 ORDER EFFECTS 45

REFERENCES

Adelman, L., Tolcott, M. A., & Bresnick, T. A. (1993). Examining the effect of information order on
expert judgment. Organizational Behavior and Human Decision Processes, 56. 348-369.
Asch, S. E. (1946). Forming impressions of personality. Journal of Abnormul and Social Psychology,
41,258-290.
Ashton, R. H., & Ashton, A. H. (1990). Evidence-responsivenessin professional judgment: Effects of
positive versus negative evidence and presentation mode. Organizational Behavior and Human
Decision Processes, 46, 1- 19.
Bray, D. W., &Grant, D. L. (1966). The assessment center in the measurement of potential for business
managers. Psychological Monographs, 80, (17, Whole No. 625).
Bycio, P., Alvares, K. M., & Hahn, 1. (1987). Situational specificity in assessment center ratings: A
confirmatory factor analysis. Journal of Applied Psychology, 72, 463-474.
Downloaded by [Mr John Track] at 05:33 07 August 2016

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ:
Lawrence Erlbaum Associates, Inc.
Constantin, S. W. (1976). An investigation of information favorability in the employment interview.
Journal of Applied Psychology, 61,743-749.
Dipboye, R. L., Smith, C. S., & Howell, W. C. (1994). Understanding industrial and organizational
psychology: An integrated approach. Fort Worth, TX:Harcourt Brace.
Farr, J. L. (1973). Response requirements and primacy-recency effects in a simulated selection
interview. Journal of Applied Psychology, 57, 228-233.
Farr, J. L., & York, C. M. (1975). Amount of information and primacy-recency effects in recruitment
decisions. Personnel Psychology, 28, 233-238.
Foti, R. J., & Hauenstein, M. A. (1993). Processing demands and the effects of prior impressions on
subsequent judgments: Clarifying the assimilation/contrast debate. Organizational Behavior and
Human Decision Processes, 56, 167-1 89.
Gaugler, B. B., & Rudolph, A. S. (1992). The influence of assessee performance variation on assessors'
judgments. Personnel Psychology, 45.77-98.
Hakel, M. D., & Dunnette, M. D. (1970). Checklist ,for describing job applicants. Minneapolis,
University of Minnesota Industrial Relations Center.
Highhouse, S., & Harris, M. M. (1993). The measurement of assessment center situations: Bem's
template matching technique for examining exercise similarity. Journal of Applied Social Psychol-
ogy, 23, 140-155.
Hogarth, R. M., &Einhom, H. J. (1992). Order effects in belief updating: The belief-adjustment model.
Cognitive Psychology, 24, 1-55.
Johnson, E. N. (1995). Effects of information order, group assistance, and experience on auditor's
sequential belief revision. Journal oj'Economic Psychology, 16, 137-1 60.
Kleinmuntz, B. (1990). Why we still use our heads instead of formulas: Toward an integrative approach.
Psychological Bulletin, 107, 296-3 10.
Kravitz, D. A., & Balzer, W. K. (1992). Context effects in performance appraisal: A methodological
critique and empirical study. Journal of Applied Psychology. 77, 24-31.
Lance, C. E., Newbolt, W. H., & Gatewood, R. D. (1995, May). Assessment center exercise factors
represent cross-situational specifcity, not method bias. Paper presented at the Tenth Annual
Conference of the Society for Industrial and Organizational Psychology, Orlando, FL.
Landy, F. J. (1989). Psychology ofwork behavior (4th ed.). Belmont, CA: Wadsworth.
London, M., & Hakel, M. D. (1974). Effects of applicant stereotypes, order, and information order on
interview impressions. Journal of Applied Psychology, 59, 157-162.
Miner, J. B. (1992). Industrial-organizational psychology . New York: McGraw-Hill.
Muchinsky, P. M. (1993). Psychology applied to work (4th ed.). Belmont, CA: Wadsworth.
 46 HIGHHOUSE AND GALL0

Murphy, K. R., Balzer, W. K , Lockhart, M. C., & Eisenman, E. J (1985). Effects of prevlous
performance on evaluations of present performance. Journal of Applied Psychology, 70.72-84
Neidig, R. D., & Neidig, P. J. (1984). Multiple assessment centerexercises and jobrelatedness. Journal
of Applied Psychology, 69, 182-186.
Pursell, E. D., Campion, M. A., & Gaylord, S. R (1980). Structured interviewing: Avoiding selection
problems. Personnel Journal, 59,907-9 12
Rowe, P. M. (1989). Unfavorable information and interview decis~ons.In R. W. Eder & G. R. Ferris
(Eds.), The employment interview. 7'heory, research, andpractice (pp. 77-89). Newbury Park,CA
Sage
Ryan, A. M., & Sackett, P. R. (1992). R e l a t i o h p betweeo graduate training, professional affiliation.
and individual psychological assessment practices for personnel decisions Personnel Psychology,
45, 365-387
Downloaded by [Mr John Track] at 05:33 07 August 2016

Sackett, P. R. (1982). The interviewer as hypothesis tester: The effects of impressions of an applicant
on interviewer questioning strategy. Personnel Psychology, 35.789404.
Sackett, P. R.,& Dreher, G. F. (1982). Constructs and assessment center dimensions: Some troubling
empirical findings. Journal of Applied Psychology, 67,401-410.
Schneider, J. R., & Schmitt, N. (1992). An exercise design ~pproachto understanding assessment center
dimension and exercise constructs. Journal of Applied Psychology, 77.32-41.
Smith, B. N., Benson, P. G., & Homsby, J. S. (1990). The effects of job description content on job
evaluationjudgments. Journal of Applied Psychology, 75, 301-309.
Smither, R. D. (1994). The psychology of work and human p e t f o m c e (2nd ed.).New York: Harper
Collins Colkge Publishers
Steiner, D D., &Rain, J. S. (1989). Immediate and delayed primacy and recency effects in performance
evaluation. Journal of Appl~edPsychology, 74, 136- 142.
Thornton, G. C ,& Byham, W C. (1982). Assessment centers and nuutagenalperformance. New York
Academic
Tubbs, R. M., Gaeth, G. J., Levm, I. P., & Van Osdol, L. A. (1993). Order effects In belief updlttLng
with consistent and inconsistent evidence Journal of Behavioral Decision Making, 6, 257-269