DATA GENERATION TECHNIQUES IN RESEARCH METHODS

TABLE OF CONTENT
INTRODUCTION-----------------------------------------------------------------------------------2
OVERVIEW OF DATA COLLECTION TECHNIQUES--------------------------------------3
Using available information---------------------------------------------------------------3
Observation----------------------------------------------------------------------------------5
Interviews------------------------------------------------------------------------------------6
Questionnaires-------------------------------------------------------------------------------7
Focus group discussions (FGD)-----------------------------------------------------------9
Projective techniques----------------------------------------------------------------------10
Mapping and scaling----------------------------------------------------------------------10
ETHICAL CONSIDERATIONS IN DATA COLLECTION---------------------------------11
CONCLUSION-------------------------------------------------------------------------------------12
REFERENCES--------------------------------------------------------------------------------------13

Page 1 of 13

DATA GENERATION TECHNIQUES IN RESEARCH METHODS

INTRODUCTION
Data-collection techniques allow us to systematically collect information about our objects of
study (people, objects, phenomena) and about the settings in which they occur. Data Collection is
an important aspect of any type of research study. Inaccurate data collection can impact the
results of a study and ultimately lead to invalid results. Data collection methods for impact
evaluation vary along a continuum. At the one end of this continuum are Quantitative methods
and at the other end of the continuum are Qualitative methods for data collection.
The Quantitative data collection methods, rely on random sampling and structured data
collection instruments that fit diverse experiences into predetermined response categories. They
produce results that are easy to summarize, compare, and generalize. Quantitative research is
concerned with testing hypotheses derived from theory and/or being able to estimate the size of a
phenomenon of interest. Depending on the research question, participants may be randomly
assigned to different treatments. If this is not feasible, the researcher may collect data on
participant and situational characteristics in order to statistically control for their influence on the
dependent, or outcome, variable. If the intent is to generalize from the research participants to a
larger population, the researcher will employ probability sampling to select participants.
Qualitative data collection methods play an important role in impact evaluation by providing
information useful to understand the processes behind observed results and assess changes in
peoples perceptions of their well-being. Furthermore, qualitative methods can be used to
improve the quality of survey-based quantitative evaluations by helping generate evaluation
hypothesis; strengthening the design of survey questionnaires and expanding or clarifying
quantitative evaluation findings. These methods are characterized by the following attributes:
a)

they tend to be open-ended and have less structured protocols (i.e., researchers may
change the data collection strategy by adding, refining, or dropping techniques or
informants)

b)

they rely more heavily on interactive interviews; respondents may be interviewed several
times to follow up on a particular issue, clarify concepts or check the reliability of data
Page 2 of 13

c)

they use triangulation to increase the credibility of their findings (i.e., researchers rely on
multiple data collection methods to check the authenticity of their results)

d)

generally, their findings are not generalizable to any specific population, rather each case
study produces a single piece of evidence that can be used to seek general patterns among
different studies of the same issue

Regardless of the kinds of data involved, data collection in a qualitative study takes a great deal
of time. The researcher needs to record any potentially useful data thoroughly, accurately, and
systematically, using field notes, sketches, audiotapes, photographs and other suitable means.
The data collection methods must observe the ethical principles of research.
OVERVIEW OF DATA COLLECTION TECHNIQUES
Various data collection techniques can be used such as:
a)

Using available information

b)

Observing

c)

Interviewing (face-to-face)

d)

Administering written questionnaires

e)

Focus group discussions

f)

Projective techniques, mapping, scaling

a) Using available information

Usually there is a large amount of data that has already been collected by others, although it may
not necessarily have been analysed or published. Locating these sources and retrieving the
information is a good starting point in any data collection effort. In order to retrieve the data
from available sources, the researcher will have to design an instrument such as a checklist or
compilation sheet. In designing such instruments, it is important to inspect the layout of the
source documents from which the data is to be extracted. The advantage of using existing data is
that collection is inexpensive. However, it is sometimes difficult to gain access to the records or
reports required, and the data may not always be complete and precise enough, or too
disorganised. e.g.

Page 3 of 13

Document Studies
Existing records often provide insights into a setting and/or group of people that cannot be
observed or noted in another way. This information can be found in document form. Lincoln and
Guba (1985) defined a document as "any written or recorded material" not prepared for the
purposes of the evaluation or at the request of the inquirer. Documents can be divided into two
major categories: public records, and personal documents (Guba and Lincoln, 1981).
Public records
Are materials created and kept for the purpose of "attesting to an event or providing an
accounting" (Lincoln and Guba, 1985). Public records can be collected from outside (external) or
within (internal) the setting in which the evaluation is taking place. Examples of external records
are census and vital statistics reports, county office records, newspaper archives, and local
business records that can assist an evaluator in gathering information about the larger community
and relevant trends. Such materials can be helpful in better understanding the project participants
and making comparisons between groups/communities.
For the evaluation of educational innovations, internal records include documents such as
student transcripts and records, historical accounts, institutional mission statements, annual
reports, budgets, grade and standardized test reports, minutes of meetings, internal memoranda,
policy manuals, institutional histories, college/university catalogs, faculty and student
handbooks, official correspondence, demographic material, mass media reports and
presentations, and descriptions of program development and evaluation. They are particularly
useful in describing institutional characteristics, such as backgrounds and academic performance
of students, and in identifying institutional strengths and weaknesses. They can help the
evaluator understand the institutions resources, values, processes, priorities, and concerns.
Furthermore, they provide a record or history not subject to recall bias.
Personal documents
Are first-person accounts of events and experiences. These "documents of life" include diaries,
portfolios, photographs, artwork, schedules, scrapbooks, poetry, letters to the paper, etc. Personal
Page 4 of 13

documents can help the evaluator understand how the participant sees the world and what she or
he wants to communicate to an audience. And unlike other sources of qualitative data, collecting
data from documents is relatively invisible to, and requires minimal cooperation from, persons
within the setting being studied (Fetterman, 1989).
The usefulness of existing sources varies depending on whether they are accessible and accurate.
In the hypothetical project, documents can provide the evaluator with useful information about
the culture of the institution and participants involved in the project, which in turn can assist in
the development of evaluation questions. Information from documents also can be used to
generate interview questions or to identify events to be observed. Furthermore, existing records
can be useful for making comparisons (e.g., comparing project participants to project applicants,
project proposal to implementation records, or documentation of institutional policies and
program descriptions prior to and following implementation of project interventions and
activities).
Key Informant
A key informant is a person (or group of persons) who has unique skills or professional
background related to the issue/intervention being evaluated, is knowledgeable about the project
participants, or has access to other information of interest to the evaluator. A key informant can
also be someone who has a way of communicating that represents or captures the essence of
what the participants say and do. Key informants can help the evaluation team better understand
the issue being evaluated, as well as the project participants, their backgrounds, behaviors, and
attitudes, and any language or ethnic considerations. They can offer expertise beyond the
evaluation team. They are also very useful for assisting with the evaluation of curricula and other
educational materials. Key informants can be surveyed or interviewed individually or through
focus groups.
b) Observation
Observation is a technique that involves systematically selecting, watching and recording
behaviour and characteristics of living beings, objects or phenomena. Observation of human
behaviour is a much-used data collection technique. It can be undertaken in different ways:
Page 5 of 13

i.

Participant observation: The observer takes part in the situation he or she observes.
(For example, a doctor hospitalised with a broken hip, who now observes hospital
procedures from within.)

ii.

Non-participant observation: The observer watches the situation, openly or concealed,

but does not participate.

Observations can be open (e.g., shadowing a server with his/her permission during routine
activities) or concealed (e.g., mystery clients). They may serve different purposes.
Observations can give additional, more accurate information on behaviour of people than
interviews or questionnaires. They can also check on the information collected through
interviews especially on sensitive topics such as alcohol or drug use, or stigmatising diseases.
Observations of human behaviour can form part of any type of study, but as they are time
consuming they are most often used in small-scale studies. Observations can also be made on
objects. For example, the presence or absence of a public toilets and its state of cleanliness may
be observed. Here observation would be the major research technique. If observations are made
using a defined scale, they may be called measurements. Measurements usually require
additional tools. For example, in nutritional surveillance we measure weight and height by using
weighing scales and a measuring board. We use thermometers for measuring body temperature.
c) Interviews
An interview is a data-collection technique that involves oral questioning of respondents, either
individually or as a group. Answers to the questions posed during an interview can be recorded
by writing them down (either during the interview itself or immediately after the interview) or by
tape-recording the responses, or by a combination of both. In Quantitative research (survey
research), interviews are more structured than in Qualitative research. In a structured interview,
the researcher asks a standard set of questions and nothing more. (Leedy and Ormrod, 2001)
Face -to -face interviews have a distinct advantage of enabling the researcher to establish
rapport with potential participants and therefor gain their cooperation. These interviews yield
highest response rates in survey research. They also allow the researcher to clarify ambiguous

Page 6 of 13

answers and when appropriate, seek follow-up information. Disadvantages include impractical
when large samples are involved time consuming and expensive. (Leedy and Ormrod, 2001)
Telephone interviews are less time consuming and less expensive and the researcher has ready
access to anyone on the planet who has a telephone. Disadvantages are that the response rate is
not as high as the face-to- face interview but considerably higher than the mailed questionnaire.
The sample may be biased to the extent that people without phones are part of the population
about whom the researcher wants to draw inferences.
Computer Assisted Personal Interviewing (CAPI): is a form of personal interviewing, but
instead of completing a questionnaire, the interviewer brings along a laptop or hand-held
computer to enter the information directly into the database. This method saves time involved in
processing the data, as well as saving the interviewer from carrying around hundreds of
questionnaires. However, this type of data collection method can be expensive to set up and
requires that interviewers have computer and typing skills.
d) Questionnaires
A questionnaire is a research instrument consisting of a series of questions and other prompts
for the purpose of gathering information from respondents. Although they are often designed for
statistical analysis of the responses, this is not always the case. Questionnaires have advantages
over some other types of surveys in that they are cheap, do not require as much effort from the
questioner as verbal or telephone surveys, and often have standardized answers that make it
simple to compile data. However, such standardized answers may frustrate users. Questionnaires
are also sharply limited by the fact that respondents must be able to read the questions and
respond to them. Thus, for some demographic groups conducting a survey by questionnaire may
not be practical.
As a type of survey, questionnaires also have many of the same problems relating to question
construction and wording that exist in other types of opinion polls.

Page 7 of 13

Questionnaire construction
Question types
Usually, a questionnaire consists of a number of questions that the respondent has to answer in a
set format. A distinction is made between open-ended and closed-ended questions. An openended question asks the respondent to formulate his own answer, whereas a closed-ended
question has the respondent pick an answer from a given number of options. The response
options for a closed-ended question should be exhaustive and mutually exclusive. Four types of
response scales for closed-ended questions are distinguished:
a)

Dichotomous, where the respondent has two options

b)

Nominal-polytomous, where the respondent has more than two unordered options

c)

Ordinal-polytomous, where the respondent has more than two ordered options

d)

(Bounded)Continuous, where the respondent is presented with a continuous scale

A respondent's answer to an open-ended question is coded into a response scale afterwards. An

example of an open-ended question is a question where the testee has to complete a sentence
(sentence completion item).

Question sequence
In general, questions should flow logically from one to the next. To achieve the best response
rates, questions should flow from the least sensitive to the most sensitive, from the factual and
behavioural to the attitudinal, and from the more general to the more specific.

Questionnaire administration modes

Main modes of questionnaire administration are:
a)

Face-to-face questionnaire administration, "where an interviewer presents the items

orally."

b)

Paper-and-pencil questionnaire administration, "where the items are presented on paper."

c)

Computerized questionnaire administration, "where the items are presented on the

computer."

Page 8 of 13

d)

Adaptive computerized questionnaire administration, "where a selection of items is

presented on the computer, and based on the answers on those items, the computer selects
following items optimized for the testee's estimated ability or trait."

e)

Focus group discussions (FGD)

A focus group discussion allows a group of 8 - 12 informants to freely discuss a certain subject
with the guidance of a facilitator or reporter. Focus groups combine elements of both
interviewing and participant observation. The focus group session is, indeed, an interview
(Patton, 1990) not a discussion group, problem-solving session, or decision-making group. At the
same time, focus groups capitalize on group dynamics. The hallmark of focus groups is the
explicit use of the group interaction to generate data and insights that would be unlikely to
emerge without the interaction found in a group. The technique inherently allows observation of
group dynamics, discussion, and firsthand insights into the respondents behaviors, attitudes,
language, etc.
Focus groups are a gathering of 8 to 12 people who share some characteristics relevant to the
evaluation. Originally used as a market research tool to investigate the appeal of various
products, the focus group technique has been adopted by other fields, such as education, as a tool
for data gathering on a given topic. Focus groups conducted by experts take place in a focus
group facility that includes recording apparatus (audio and/or visual) and an attached room with
a one-way mirror for observation. There is an official recorder who may or may not be in the
room. Participants are paid for attendance and provided with refreshments. As the focus group
technique has been adopted by fields outside of marketing, some of these features, such as
payment or refreshment, have been eliminated.
When to use focus groups
When conducting evaluations, focus groups are useful in answering the same type of questions
as in-depth interviews, except in a social context. Specific applications of the focus group
method in evaluations include

identifying and defining problems in project implementation;

identifying project strengths, weaknesses, and recommendations;

assisting with interpretation of quantitative findings; 5

Page 9 of 13

obtaining perceptions of project outcomes and impacts; and

generating new ideas.

Recording focus group data.

The procedures for recording a focus group session are basically the same as those used for indepth interviews. However, the focus group approach lends itself to more creative and efficient
procedures. If the evaluation team does use a focus group room with a one-way mirror, a
colleague can take notes and record observations. An advantage of this approach is that the extra
individual is not in the view of participants and, therefore, not interfering with the group process.
If a one-way mirror is not a possibility, the moderator may have a colleague present in the room
to take notes and to record observations. A major advantage of these approaches is that the
recorder focuses on observing and taking notes, while the moderator concentrates on asking
questions, facilitating the group interaction, following up on ideas, and making smooth
transitions from issue to issue. Furthermore, like observations, focus groups can be videotaped.
These approaches allow for confirmation of what was seen and heard. Whatever the approach to
gathering detailed data, informed consent is necessary and confidentiality should be assured.
Having highlighted the similarities between interviews and focus groups, it is important to also
point out one critical difference. In focus groups, group dynamics are especially important. The
notes, and resultant report, should include comments on group interaction and dynamics as they
inform the questions under study.
f)

Projective techniques

When a researcher uses projective techniques, (s)he asks an informant to react to some kind of
visual or verbal stimulus. For example: An informant may be provided with a rough outline of
the body and be asked to draw her or his perception of the conception or onset of an illness.
Another example of a projective technique is the presentation of a hypothetical question or an
incomplete sentence or case/study to an informant (story with a gap).
g)

Mapping and scaling

Page 10 of 13

Mapping is a valuable technique for visually displaying relationships and resources. In a water
supply project, for example, mapping is invaluable. It can be used to present the placement of
wells, distance of the homes from the wells, other water systems, etc. It gives researchers a good
overview of the physical situation and may help to highlight relationships hitherto unrecognised.
Mapping a community is also very useful and often indispensable as a pre-stage to sampling.
Scaling is a technique that allows researchers through their respondents to categorise certain
variables that they would not be able to rank themselves.
For example, they may ask their informant(s) to bring certain types of herbal medicine and ask
them to arrange these into piles according to their usefulness. The informants would then be
asked to explain the logic of their ranking. Mapping and scaling may be used as participatory
techniques in rapid appraisals or situation analyses.
ETHICAL CONSIDERATIONS IN DATA COLLECTION
As we develop our data collection techniques, we need to consider whether our research
procedures are likely to cause any physical or emotional harm. Harm may be caused, for
example, by:
i.

violating informants right to privacy by posing sensitive questions or by gaining access

to records which may contain personal data;

ii.

observing the behaviour of informants without their being aware (concealed observation
should therefore always be crosschecked or discussed with other researchers with respect
to ethical admissibility);

iii.

allowing personal information to be made public which informants would want to be kept
private, and

iv.

failing to observe/respect certain cultural values, traditions or taboos valued by your

informants.

Several methods for dealing with these issues may be recommended:

i.

obtaining informed consent before the study or the interview begins;

ii.

not exploring sensitive issues before a good relationship has been established with the
informant;

iii.

ensuring the confidentiality of the data obtained; and

Page 11 of 13

iv.

learning enough about the culture of informants to ensure it is respected during the data
collection process.

If sensitive questions are asked, for example, about family planning or sexual practices, it may be
advisable to omit names and addresses from the questionnaires.
CONCLUSION
When discussing different data collection techniques and their advantages and disadvantages, it
becomes clear that they can complement each other. A skilful use of a combination of different
techniques can reduce the chance of bias and will give a more comprehensive understanding of
the topic under study. Researchers often use a combination of flexible and less flexible research
techniques. Flexible techniques, such as; loosely structured interviews using open-ended
questions, focus group discussions, and participant observation are also called qualitative
research techniques. They produce qualitative data that is often recorded in narrative form.
Qualitative Research Techniques involve the identification and exploration of a number of often
mutually related variables that give insight in human behaviour (motivations, opinions,
attitudes), in the nature and causes of certain problems and in the consequences of the problems
for those affected. Why, What and How are important questions.
Structured questionnaires that enable the researcher to quantify pre- or post-categorised answers
to questions are an example of quantitative research techniques. The answers to questions can be
counted and expressed numerically.
Quantitative research techniques are used to quantify the size, distribution, and association of
certain variables in a study population. How many? How often? and How significant? are
important questions. Both qualitative and quantitative research techniques are often used within a
single study.

Page 12 of 13

REFERENCES
Fetterman, D.M. (1989). Ethnography: Step by Step. Applied Social Research Methods Series,
Vol. 17. Newbury Park, CA: Sage.
Guba, E.G., and Lincoln, Y.S. (1981). Effective Evaluation. San Francisco: Jossey-Bass.
Lincoln, Y.S., and Guba, E.G. (1985). Naturalistic Inquiry. Beverly Hills, CA: Sage.
Lofland, J., and Lofland, L.H. (1995). Analyzing Social Settings: A Guide to Qualitative
Observation and Analysis, 3rd Ed. Belmont, CA: Wadsworth.
Patton, M.Q. (1990). Qualitative Evaluation and Research Methods, 2nd Ed. Newbury Park, CA:
Sage.
Schalock, H.D., Schalock, M.D., and Girad, G.R. (In press). Teacher work sample methodology,
as used at Western Oregon State College. In J. Millman, Ed., Assuring Accountability? Using
Gains in Student Learning to Evaluate Teachers and Schools. Newbury Park, CA: Corwin.

Page 13 of 13