Evaluation of Student Learning:

Test Construction &

Other Practical Strategies

Faculty Professional Development

Fall 2005
Dr. Kristi Roberson-Scott

Constructing the Test

Types of Test Questions:

Multiple-Choice Items
True-False Items
Matching Items
Fill-In, Completion or Short-Answer Items
Essay Questions

Multiple Choice Items

Advantages:
Extremely versatile-can measure the higher level
mental processes (application, analysis, synthesis
and evaluation)
A compromise between a short answer/essay and
T/F item
Can cover a wide range of content can be sampled
by one test

Disadvantages
Difficult to construct plausible alternative responses

Types of Multiple Choice Items

Four Basic Types

Question Type
Incomplete Statement Type
Right Answer Type
Best Answer Type

Which Type is Best?

Question Type vs. Incomplete Statement
Right Answer vs. Best Answer Type

Multiple Choice Items

1. Writing the stem first:
A. Be sure the stem asks a clear question
B. Stems phrased as questions are usually easier to
write
C. Stems should not contain a lot of irrelevant info.
D. Appropriate reading level/terms
E. Be sure the stem is grammatically correct
F. Avoid negatively stated stems

Multiple Choice Items

Writing the correct response
Use same terms/reading level
Avoid too many qualifiers
Assign a random position in the answer sequence

Read the stem and correct response together

Generate the distractors/alternative responses

Multiple Choice Items

Other Tips for Constructing MC Items:
Items should have 3-4 alternatives.
Stem should present a single, clearly formulated
problem
Simple, understandable, exclude extraneous words
from both stem and alternatives
Include in the stem any word that are repeated in
each response
Avoid all of the above (can answer based on partial
information)
Avoid none of the above

Multiple Choice Items

Alternative responses/distractors should be plausible
and as homogeneous as possible
Response alternatives should not overlap
Two synonymous terms (arithmetic average/mean)

Avoid double negatives

None of the following are part of the brain except which one?

Emphasize negative wording

Each item should be independent of other items in the
test
Information in the stem of one item should NOT help answer
another item.

True-False Test Items

Best suited for testing 3 kinds of info.:
Knowledge level learning
Understanding of misconceptions
When there are two logical responses

Advantages:
Sample a large amount of learning per unit of
student testing time

Disadvantages:
Tends to be very easy
50-50 chance of guessing
Tends to be low in reliability

Tips for Constructing True/False Items

Tips for constructing True-False Items
Avoid double negatives
Avoid long or complex sentences
Specific determiners (always, never, only, etc.) should be
used with caution
Include only one central idea in each statement
Avoid emphasizing the trivial
Exact quantitative (two, three, four) language is better
than qualitative (some, few, many)
Avoid a pattern of answers

Objective Test Item Analyses

Evaluating the Effectiveness of Items..
Why?
Scientific way to improve the quality of tests and test items
Identify poorly written items which mislead students
Identify areas (competencies) of difficulty

Item analyses provided info. on:

Item difficulty
Item discrimination
Effectiveness of alternatives in MC Tests

Short-Answer Items
Two Types: (Question and Incomplete Statement)
Advantages:
Easy to construct
Excellent format for measuring who, what, when, and where
info.
Guessing in minimized
Student must know the material- rather than simply recognize
the answer

Disadvantages:
Grading can be time consuming
More than one answer can be correct

Short Answer Items

Tips for Constructing Short Answer Items
Better to supply the term and require a definition
For numerical answers, indicate the degree of
precision expected and the units in which they are
to be expressed.
Use direct questions rather than incomplete
statements
Try to phrase items so that there is only one possible
correct response
When incomplete statements are used, do not use
more than one blank within an item.

Essay Questions
Types of Essay Questions
Extended Response Question
Great deal of latitude on how to respond to a question.
Example: Discuss essay and multiple-choice type tests.

Restricted Response Question

More specific, easier to score, improved reliability and validity
Example: Compare and contrast the relative advantages of
disadvantages of essay and multiple choice tests with respect
to: reliability, validity, objectivity, & usability.

Essay Items
Advantages:
Measures higher learning levels (synthesis,
evaluation) and is easier to construct than an
objective test item
Students are less likely to answer an essay question
by guessing
Require superior study methods
Offer students an opportunity to demonstrate their
abilities to:
Organize knowledge
Express opinions
Foster creativity

Essay Items
May limit the sampling of material covered
Tends to reduce validity of the test

Disadvantages
Subjective unreliable nature of scoring
halo effect good or bad students previous level of
performance
Written expression
Handwriting legibility
Grammatical and spelling errors

Time Consuming

Essay Questions
Give students a clear idea of the scope & direction
intended for the answer
Might help to start the question with the description of the
required behavior (e.g., compare, analyze)

Appropriate language level for students

Construct questions that require students to
demonstrate a command of background info, but do not
simply repeat that info.
If question calls for an opinion, be sure that the
emphasis is not on the opinion but on the way its
presented or argued.
Use a larger number of shorter, more specific questions
rather than one or two longer questions so that more
information can be assessed.

Essay Questions
You might
Give students a pair of sample answers to a question of the
type you will give on the test.
Sketch out a rubric (grading scheme) for each question before
reading the papers OR randomly select a few to read and make
up the grading scheme based on those answers
Give students a writing rubric
Detach identifying information and use code numbers instead
to avoid letting personality factors influence you.
After grading all the papers on one item, reread the first few to
make sure you maintained consistent standards
Be clear to student the extend to which factors other than
content (e.g., grammar, handwriting, etc.) will influence the
grade.

Essay Questions
Tips for constructing Essay Questions
Provide reasonable time limits for each question
thinking and writing time

Avoid permitting students a choice of questions

Will not necessarily get a representative sample of student
achievement. Only be requiring all students to answer all
questions can their achievement be compared

A definite task should be put forth to the student

Critical words: compare, contrast, analyze, evaluate, etc.

Scoring Essay Items

Write an outline of the key points (use outline to design
a rubric)
Determine how many points are to be assigned to the
question as a whole and to the various parts within it.
If possible, score the test without knowledge of the
students name
Face Sheet

Score all of the answers to one question before

proceeding to the next question
Consistent standard

Scoring Essay Exams

If possible, score each set of answers within the
same time frame
Handwriting, spelling & Neatness
Two separate grades?
Mastery of material
Other

Cheating
Preventing Cheating

Reduce the pressure (multiple evaluations)

Make reasonable demands (length/content of exam)
Use alternative seating
Use alternative forms
Be cautious with extra copies

Using Assessment & Evaluation to

Improve Student Learning Outcomes
Providing feedback to student
Closing the assessment & evaluation loop
Maximizing student learning