Questionnaire Design

A survey is only as good as

the questions it asks

Questionnaire Modes

Face-to-face interview

Telephone survey

Self-administered survey (i.e. mail

survey)

Types of Questions

The background question

used to obtain demographic characteristics of the group being
studied, such as age, sex, grade, level of assignment, and so forth.
This information is used when you are categorizing your results by
various subdivisions such as age or grade. Therefore, these questions
should be consistent with your data analysis plan.
Multiple choice or closed-end question.
It is used to determine feelings or opinions on certain issues by
allowing the respondent to choose an answer from a list you have
provided .
The intensity question (Attitude Measurement)
a special form of the multiple-choice question, is used to measure the
intensity of the respondents feelings on a subject. These questions
provide answers that cover a range of feelings.
The free response or open-end question.
This type requires respondents to answer the question in their own
words .It can be used to gather opinions or to measure the intensity
of feelings.

Questionnaire Construction

Keep the language simple.

Keep the number of questions to a minimum.
Limit each question to one idea or concept.
Do not ask leading questions.
Allow for all possible answers.
Avoid emotional or morally charged questions.
Include a few questions that can serve as checks on the accuracy
and consistency of the answers as a whole.
Organize the pattern of the questions
Place demographic questions at the end of the questionnaire.
Have your opening questions arouse interest.
Ask easier questions first.
To minimize conditioning, have general questions precede specific
ones.
Group similar questions together.
If you must use personal or emotional questions, place them at the
end of the questionnaire.

Questionnaire design
The Effect of One Word
How important do you think it is
to have a hotline telephone number
people can call to get help when they
have problems with their health plan?

To have a hotline
Very Important

Somewhat Important

How important do you think it is for the

GOVERNMENT to set up a hotline
telephone number people can call to get
help when they have problems with their
health plan?

For government to set up a hotline

Not too important

Not at all important

Kaiser Family Foundation/Harvard University National Survey of Americans Views on Consumer Protections in managed Care, January 1998

Leading Questions
Do

you think the food in the hotel

made you sick
Did the hotel staff seem unhiegenic to
you?
Do you agree that hospital staff were
close to exhaustion

Questionnaire design
Balance in question wording
Whats wrong with these questions?

Do you support the United States taking military

action against Iraq?

If you heard that Candidate X supported stricter

gun control laws, would that make you more likely
to vote for her?

Did you happen to vote in the presidential election

in 2000?

Questionnaire design
Balance in question wording
Bad questions, improved:

Do you support or oppose the United States taking

military action against Iraq?

If you heard that Candidate X supported stricter

gun control laws, would that make you more likely
to vote for her, less likely to vote for her, or
wouldnt it make much difference?

Did you happen to vote in the presidential election

in 2000, or not?

Questionnaire design
Open-ended or Closed-ended?
What do you think is the most important
problem in health or health care for the
government to address? (top 7 answers)

Which of the following do you think is the

most important health care issue for the
President and Congress to deal with?

Cost of health care

Helping seniors pay for

prescription medicines

Cost of Rx drugs

Increasing the number

of Americans covered
by health insurance

Coverage for uninsured

Senior citizens care
Other medical conditions
Rx coverage for seniors
Cost of insurance

Making Medicare more

financially sound for
future generations
Lowering the cost of
medical malpractice
insurance for physicians
Helping states with the
cost of Medicaid

Kaiser Family Foundation/Harvard School of Public Health Health News Index, April 2003 (conducted April 3-6, 2003)

Questionnaire design
Dont Know Answers offered or volunteered?
Do you favor or oppose the government
offering parents money or "vouchers" to
send their children to private or religious
schools, or public schools outside their
district?

Do you favor or oppose the government

offering parents money or "vouchers" to
send their children to private or religious
schools, or public schools outside their
district, or havent you heard enough
about that to have an opinion?

Oppose

Favor

Oppose

Favor

No opinion
NPR/Kaiser Family Foundation/Kennedy School of Government Education Survey, June 1999 (conducted June 25 July 19, 1999)

No
opinion

Types of questions

Types of fixed alternative questions

Single dichotomy or dichotomous-alternative

questions
Are you currently registered in a course at the
University of Lethbridge?
Yes____ No____

Respondent chooses one of two alternatives

(yes/no; male/female)

Types of fixed alternative

questions
Multi-choice

alternative

Respondent chooses from several

alternatives
Many types

Multi-choice alternative
questions

Determinant

choice

Choose only one from several possible

responses

Which faculty are you currently registered

in at the University of Lethbridge?
Management ___
Education ____
Arts/Science____
Health sciences____
Combined degree____

 Frequency

determination
Asks for an answer about
frequency of occurrence
In a typical week, how often do
you purchase chocolate chip
cookies?
__never
__ once
__ 2 or more times

 Check

list

Provide multiple answers to a single

question
Should be mutually exclusive and
exhaustive
What brands of chocolate chip cookies
have you, to the best of your
memory, purchased in the past month
(check all that apply?)
__ Dare
__ Chips Ahoy
__ Presidents Choice Decadent etc. etc.

Attitude

rating scales

Attitude:
An enduring disposition to
consistently respond to various
aspect of the world, including
persons, events and objects

Attitude Scales: Scaling Defined

The term scaling refers to procedures for
attempting to determine quantitative measures of
subjective and sometimes abstract concepts.

Types of attitude scales

Simple

attitude scales
Most basic form respondent responds to a
single question
Do not allow for fine distinctions or
placement on continua
You are at a company party and are feeling
nervous, but you are obligated to be there. Do
you:
__ find someone you know to buddy up with
__ take it as an opportunity to meet new people

 Category

scales

More sensitive; provides more information

Overall, how satisfied are you with the high
speed performance of your Mercedes:
__ very satisfied
__ somewhat satisfied
__ neither satisfied nor dissatisfied
__ somewhat dissatisfied
__ very dissatisfied
If you could choose, how long would each term
be?
___26 weeks __ 13 weeks
__ 6 weeks ___4
weeks

 Summated

scale

rating scales the Likert

Respondents indicate their attitudes by

checking how strongly they agree or
disagree with statements
Chocolate chip cookies are my preferred
variety of cookie
Strongly disagree
Strongly Agree
(1)

Disagree
(2)

Uncertain
(3)

Agree
(4)

(5)

Likert Scale
Kids apparels provided by the retailers do
not address the safety issues of babies
No, I strongly disagree
No, I disagree quite a lot
No, I disagree just a little
I am not sure about this
Yes, I agree just a little
Yes, I agree quite a lot
Yes, I strongly agree

 Semantic

Differential Rating scale

An attitude measure consisting of a

series of seven-point bipolar rating
scales allowing response to a
concept
Think of your favorite type of cookie. Rate it
on each of the following continua:
Hard----------------------------------------------------Soft
Lots of chips----------------------------------Fewer
chips
Crispy-----------------------------------------------chewy

Example of Semantic Differential

Scaling Form
Service is discourteous ___:___:___:___:___:___:___ Service is courteous
Location is convenient ___:___:___:___:___:___:___ Location is inconvenient
Hours are convenient ___:___:___:___:___:___:___ Hours are inconvenient
Loan interest rates are high ___:___:___:___:___:___:___ interest rates are
low

Figure 5: Contrasting Profiles of Banks A and B

Service is discourteous ___:___:___:___:___:___:___ Service is courteous

Location is convenient ___:___:___:___:___:___:___ Location is inconvenient
Hours are convenient

___:___:___:___:___:___:___ Hours are inconvenient

Loan interest rates are high ___:___:___:___:___:___:___interest rates are

low

Bank A
Bank B

 Numerical

Rating scale

Similar to a semantic differential

except that it uses numbers as
response options to identify response
positions instead of verbal
descriptions
Think of your favorite type of cookie. Rate it
on each of the following continua:
Hard----------------------------------------------------Soft
8
7
6
5
4
3
2
1

 Constant

Sum Scales

Attributes based on their importance to the

person. Respondents are asked to divide a
constant sum to indicate the relative
importance of attributes
Example: Suppose the photocopy budget per
professor was $100 per month. How much
should be allocated to the following. Divide
the $100 according to your preference:
____ photocopying for student needs;
____ photocopying for research needs;
____ photocopying for committee needs.
====
$100 TOTAL

 Stapel

Scales

An attitude measure that places a single

adjective in the center of an even-number
range of numerical values

Example:
Research Methodology
+3
+2
+1
Exciting
-1
-2
-3

Example of a Stapel Scale

-5 -4 -3 -2 -1 1 2 3 4
5
Service is courteous

Location is convenient

Hours are convenient

Loan interest rates are high

 Graphic

Rating Scales

An attitude measure consisting of a

graphic continuum that allows
respondents to rate an object by
choosing any point on the continuum

 Rank-Order

Scales

Scales in which the respondent

compares one item with another or a
group of items against each other and
ranks them.
Example: next slide

Most important skills

Adaptability to change
Problem identification
Listening skills
Written communication
Leadership
Informal Oral communication
Analytical thinking/problem solving
Time management
Coping with stress/job pressures
Interpersonal relations
Formal oral presentations

Most important skills

Adaptability to change
Problem identification
Listening skills
Written communication
Leadership
Informal Oral communication
Analytical thinking/problem
solving
Time management
Coping with stress/job pressures
Interpersonal relations
Formal oral presentations

s
s
s
r
e
e
n
i
g
s s
a
u
n
B ad
a
M
gr

8 9
6 6
1 1
2 4
4 2
3 3
5 5
7 10
11 7
9 8
10 11

 Paired

Comparison Scales

For each pair of sports car, place a

check beside the one you most prefer if
you had to choose between the two.
- BMW Z3
- Chevrolet Corvette
- Porsche Boxter
- Porsche Boxter

- Chevrolet Corvette
- BMW Z3

 Sorting

Respondent indicates their attitudes

or beliefs by arranging items.
Example: Please sort the following cards with
pictures of cookies into the following
categories
Like
Dislike
Neither like nor dislike

Examples of Poor Survey

Questions
Which of these services would you be interested in?
____ loans ____ mentoring ____ business counselling ____
information referral
(This question should have an other category)
What beverages do you drink?
____ Milk ____ coke ____ non-cola drink ____ coffee ____
tea
____ juice
(This question is too broad. Most of us will have drunk
some of
these at some time. Is the respondent to check a
number of boxes or only one?

Sample Questionnaire
(developed by a woman who was interested in selling
southwestern jewelry made by Native Indians)
Q1. Have you ever purchased or received southwestern jewelry?
Yes ______ No ______
Q2. Have you ever purchased or received southwestern jewelry
made by native Indians?
Yes ______ No ______
If Yes, what type of jewelry?
Necklace____ Ring _____ Bracelet _____ Earnings _____Other _____
Q3. Would you be interested in purchasing the above mentioned
jewelry made by native Indians?
Yes ______ No ______
Q4. Do you know where to shop for such jewelry?
Yes ______ No ______
Q5. When buying jewelry, what do you value the most? On a
scale of 1 through 5, list in order according to your
preference. One represents your most valued choice.
Craftsmanship_____ Cost _____ Uniqueness _____ Other_____