26.

which of the following three areas do you think is most important for the schools to
concentrate on? (start at designated point and read choices)
140- 1. teachingmathematics and reading
2. preparing students for college or a career
or 3. teaching a code of values and moral behavior
9. don’t know

27. which is second most important? (read remaining two choices)

141- 1. teachingmathematics and reading
2. preparing students for college or a career
or 3. teaching a code of values and moral behavior
9. don’t know
28. how much attention do you think the schools pay to people like you when they decide
what to teach-a great deal of attention, or no attention at all?
142- 1. a great deal of attention
2. some attention
3. no attetion at all
9. don’t know
29. turning to the matter of discipline, are the schools in your community too strict, not
strict enough, or just about right?
143- 1. too strict
2. just about right
3. no strict enough
9. don’t know

Obtained in a minimun of telephone time. One of the questions is response-keyed. All of

them are specific, and all responses are precoded in scaled, categorical or checklist from.

Preparing questionnaire items

The procedure for preparing questionnaire items parallel those for Preparing
interview schedule items. Again, the relationship between the items and the variables as
operationally defined is of critical importance. Thus, you must constantly ask about your
items : is this what I want to be measuring? Three sample questionnaires appear in
figures 9,4,9,5, and 9,6 on pages 217-224.
The questionnaire in figure 9,4 was used in a follow-up study of community
college graduates and high school graduates who did not go on to college. The
researchernwas interested in determining whether the community college graduates
subsequently obtained higher socioeconomic status (that is, earnings and job status) and
job satisfaction than a matched group of people who did not go to college. The items on
the questionnaire are designed to (1) determine earnings, job title, and job satisfaction
(the dependent variables) (items 1-17): (2) determine subsequently educational
experiences in order to eliminate or reclassify those going for additional education (a
control variable) asvwell as to verify the educational status distinction of two-year
college versus high school only (the independent variable) (items 8-15, 23): (3) determine
background characteristics in order to match samples (items 16-20, 24, 25): and (4)
determine health in order to eliminate those whose job success chances were impaired
(items 21, 22).
All of the items are to be completed by all respondents, except item 7 which is
response-keyed to the preceding item. (items 12-15 also have response-keyed parts). The
result is a reasonably simple, easy to complete instrument.

Figure 9.4 Sample questionnaire I (from E. W. Butler, A comparison of the

socioeconomic status and job satistaction of male high school and community college
graduates. Worcester, mass, unpublished, 1977. reprinted by permission of the author)

The sample questionnaire in figure 9.5 (page 220) uses the scaled response and is
an attempt to measure attitudes toward ones’s work, or job satisfaction (forsyth 1976).
Note that for each item an eight-point scale is available for responding and that the
statement format is utilized. Note further that some of the items have been reversed (for
example, 18, 22). That is the questions have been written in two directions so that
agreementon some items indicates positivity while other have been written so that
disagreement indicates positivy. Agreement with an item such as number 19, for
example, reflects a positive attitude towars one’s job while disagreement with item 18
also reflects a positive attitude. This technique is a protection against the form of
response bias in which an individual simply selects the same response number for each
item. Under this system, respondents will not delude you into thinking they are extremely
positive or extremely negative, but they will come out neutral since half of the items are
written in each direction. One feature that this questionnaire lacks is items not
specifically related to its objective. Items that deal with an attitude area that is unrelated
to the one being measured, that is, filler items, serve to make the true purpose of an
instrument less visible. Filler items are particularly userful in test situations where the
potentially obvious purpose of the test and social desirability considerations may be
biasing factors.

Figure 9. 5 sample questionnaire II. ( from p. b forsyth, isolation and alienation in

educational organizations, new brunswick, n.j., rutgers university, unpublished
doctoral dissertation, 1976, pp. 182-184. reprinted by permission of the author.)

The next series of questions conntains a set of alternative answers for each
questions. These alternative answer from a continuum from one extreme at the left end to
the other extreme at the right. A series of descriptive terms is used to define, broadly,
four positions along the continuum. Two boxes under each positiongive eight choices for
each question. Please indicate your choices by completely filling in one box in the
category that best describes your view of that question.
18. I really don’t feel a sense of pride or accomplishment as a result of the type of work
that I do

19. If I had it to do again, I would choose the same work.

20. My work is interesting nearly all the time.

21. My work gives me a feeling of pride in having done the job well.

22. My work does little in the way of tapping my expertise and know-know

23. I very much like the type of work that I am doing.

24. If it became necessary to move to another city in order to continue the work I da, I
would do so willingly

25. My work is almost always challenging

26. My work rarely gives me a chance to do the things that I do best.

27. My work is my most rewarding experince.

28. My work never gives me a sense of accomplishment.

A third sample questionnaire is shown in figure 9.6. developed by bryan (1963),

this student-bopinion questionnaire (SOQ) is used to determine teachers behavior as
perceived by students. Items 1-10 are scaled items, each utilizing a five-point scale as the
response mode. Note the format includes both a statement and a question (the statement
appearing first in block letters and the corresponding question following in parentheses).
No attempt has been made to counteract for response bias by reversing the direction of
some of the items: each item has been written so that agreement indicates positivity.
Obviously, these itemsa are susceptible to biased response based on considerations other
than respondent’s judg.

Figure 9.6 Sample questionnaire III. (from R.C. Bryan, Reactions to teachers by
students, parents, and administrators, western michigan university {U.S office of
Education Cooperative research Project No. 668}, 1963, P, 53. Reprinted by permission
of the author)

STUDENT-OPINION QUESTIONNAIRE (SOQ), Form 6

Please answer the following question honestly and frankly. Do not give your
name. to encourage you to be frank, your regular teachers should be absent from the
classroom while these questions are being answers.
The person who is temporarily in charge of your class during this period will
collect all report and seal them in an envelope addressed to university.
Your teachers will receive from the university a summary of the answers by the students
in-your class. The university will mail this summary to no one except your teacher unless
requested to do so by your teacher.
After completing this report, sit quietly or study until all students have completed
their reports. There should be no talking.
Underline your answer to each question on this page. Write your answers to
questions 11 to 14 on the other side of this page.

WHAT IS YOUR OPINION CONCERNING:

1. THE KNOWLEDGE THIS TEACHER HAS OF THE SUBJECT TAUGHT?

( Has he a thorough knowledge and understanding of his teaching field?)
Below average average good very good the very best

2. THE ABILITY OF THIS TEACHER TO EXPLAIN CLEARLY?

(Are assignments and explanations clear and definite)
Below average average good very good the very best

3. THIS TEACHER’S FAIRNESS IN DEALING WITH STUDENTS? (Is he fair

and impartial in treatment of all students?)
Below average average good very good the very best

4. THE ABILITY OF THIS TEACHER TO MAINTAIN GOOD DISCIPLINE?

(Does he keep good control of the class without being harsh? Is he firm but fair)
Below average average good very good the very best

5. THE SYMPATHETIC UNDERTANDING SHOWN BY THIS TEACHER?

(is he patient, friendly, considerate, and helpful?)
Below average average good very good the very best

6. HOW MUCH ARE YOU LEARNING IN THIS CLASS?

(Are you learning well and much? Are you really working?)
Below average average good very good the very best

7. THE ABILITY THIS TEACHER HAS TO MAKE CLASSES INTERESTING?

(does he show enthusiasm and a sense of humor? Does he vary teaching
procedures?)
Below average average good very good the very best

8. THE ABILITY OF THIS TEACHER TO GET THINGGS DONE IN AN

EFFICIENT AND BUSINESS-LIKE MANNER?
(are plans well made? Is little time wasted?)
Below average average good very good the very best
 9. THE SKILL THIS TEACHER HAS TO GET STUDENTS TO THINK FOR
THEMSELVES?
(Are students’ ideas and opinions worth something in this class? Do students help
decide how to solve problems and how to get their work done? Do they get at the
real reasons why certain things happen?)
Below average average good very good the very best

10. THE GENERAL (ALL – ROUND) TEACHING ABILITY OF THIS

TEACHER?
(All factors considered, how close does this teacher come to your ideal?)
Below average average good very good the very best

11. PLEASE NAME ONE OR TWO THINGS THAT YOU ESPECIALY LIKE
ABOUT THIS TEACHER.

12. PLEASE GIVE ONE OR TWO SUGGESTIONS FOR THE IMPROVEMENT

OF THIS TEACHER

13. PLEASE NAME ONE OR TWO THINGS THAT YOU ESPECIALLY LIKE
ABOUT THIS COURSE

14. PLEASE GIVE ONE OR TWO SUGGESTIONS FOR THE IMPROVEMENT

OF THIS COURSE

PILOT TESTING AND EVALUATING A QUESTIONNAIRE

It is usually highly desirable to run a pilot test on a questionnaire and to revise it

based on the results of the test. A pilot test, which uses a group of respondents who are
part of the intended test population but will not be part of the sample, attempts to
determine whether questionnaire items possess the desired qualities of measurement and
discriminability.
If a series of items are inteded to measure the same variable (as would be true of
the ten SOQ items in figure 9.6), it is desirable to determine whether these items are
measuring something in common. To determine this, the scale would be administered to a
pilot sample, and, based on responses, correlations would be run between the score
obtained by each person on each item and the scores obtained by each person across the
whole scale (see the discussion on item analysis in the previous chapter). The large the
correllation between an item score and the total score, the greater the relationship
between what the item is measuring and what the total scale is measuring. Following the
completion of this item analysis, the researcher can select those items having the highest
correlations with the total score and use them to make up the final scale. Consider ten
items to measure a person’s attitude toward some object and the following correlations
between each item score and the mean score across all ten items.
 Item Number Correlation
1 .89
2 .75
3 .27
4 .81
5 .19
6 .53
7 .48
8 .72
9 .33
10 .60

Based on these data, the decision would be to eliminate item 3, 5, 7, and 9 and use the
other six items as the final scale, confident that the remaining items were measuring
something in common.
Item analysis of items inteded to measure the same variable in the same way is
one important use of the data collected from a pilot test. However, item analyses are not
as critical for the refinement of questionnaires as they are for the refinement of tests.
Questionnaire items are usually reviewed for clarity and distribution of responses without
necessarily running an item analysis.
A pilot test can uncover a variety of failings. For example, if all respondents reply
identically to any one item, that item probably lacks discriminability. If you receive a
preponderance of inappropriate responses to an item,examine it to see if it is ambiguous
or poorly worded. Poor instructions and other administration problems become apparent.
If respondents refuse to answer certain items, tryto desensitize such items by rewording
them. Thus, pilot runs enable researchers to “ debug” their questionnaires by diagnosing
and correcting the above kinds of failings.

9.5 SAMPLING PROCEDURES

Random Sampling

A researcher administers a questionnaire or interview to gain Information about a

particular group of persons, such as high school graduates, school administrators in new
england, or home economics teachers in new jersey. This target group is termed the
population of the study, and the first step in sampling is to define the population. Once
this Has been done you must then select a sample or representative group from this
population to serve aas respondents. One way to insure that this sample will be
representative of the larger population is to draw a random sample because random
selection plimits the probability that you choose a biased sample. For example you are
interested in obtaining information about presidents of two year colleges. The population
is 2800 presidents from which you want a sample off 300. which 300 should you choose?
To draw a random sample, you might write the names of all the twoyear colleges in
alphabetical ordert giving each a number in the sequence. Three hundred numbers
would then be randomly selected using a table of random numbers (see appendix B). the
resulting list of 300 colleges, each yielding one presidents, is a random sample of the
population from which it was drawn. No systematic biases in selection or selectees need
be expected as a result of this procedure. When certain sample variables are of special
interest to the researcher (for example, age) stratified sampling is employed with the
variables of interest termed sampling parameters (see page 228)

Defining the population

The population (or target group) used in a questionnaire or interview study is that
group about which the researcher is interested in gaining information and drawing
conclusing. If the researcher were unterested in the educational aspirations of theacher,
for example, the population of the study would be teachers. The term defining the
population refers to the establishment of boundary conditions that specify who shall be
included in or excluded from the population. In the above example, the population could
be defined as elementary school teachers, or public school teachers, or all teachers, or
some other choice.
Specifying the groyup that is to constitute the population is an early step in the
sampling process hat affects the nature of the conclusions that may be drawn from a
study. If the populationis broadly defined (like “ all teachers” in the above example),
external validity or generalizability will be maximized, although sucha broad definition
may make obtaining a representative sample difficult and require a large sample size.
Conversely, defining the population narrowly ( for example, as “female, elementary
school teachers”)may facillitate the selection of a suitable sample but will restrict
conclusions and generalizations to the specific population used, which may be
inconsistent with the intent of the study.
The definition of the target population in a study is most reasonably based on the
independent, moderator, and control variables in the study design along with practical
considerations such as availability of subjects or respondents. When a control variable in
a study deals with a population characterisric, the researcher must systematically include
or exclude this characteristic in defining the population. (see also 107) for example, if a
researcher were interested in a comparision between academic high school graduates and
vocational high school graduates with the limita