Вы находитесь на странице: 1из 36

BU I L D I N G RE S E A R C H CA P A C I T Y

Scales Series: Chapter 6




FoQus on Scales:
Data Analysis and
Application


PSIs Core Values
Bottom Line Health Impact * Private Sector Speed and Efficiency * Decentralization, Innovation,
and Entrepreneurship * Long-term Commitment to the People We Serve
P
S
I


R
E
S
E
A
R
C
H
&

M
E
T
R
I
C
S


T
O
O
L
K
I
T



Research & Metrics
Population Services International
1120 Nineteenth Street, NW, Suite 600
Washington, D.C. 20036












FoQus on Scal es: Data Anal ysi s and Appl i cati on


PSI Research & Metrics
2008












Population Services International, 2008


Contact Information
Hibist Astatke
1
, Linda Kaljee
2
, and Kate O'Connell
3

1. Senior Researcher,
PSI/Washington
2. Associate Professor, Wayne
State University, Pediatric
Prevention Research Center
3. Deputy Director, PSI Research
& Metrics
For more information, please contact:
Hibist Astatke
1120 19
th
St. NW.
Washington, DC 20036
Telephone 202-572-4623
Email hastatke@psi.org


PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


1

LEARNING OBJECTIVES

By the end of this chapter, the reader will be able to:

1. Adapt the 'core' coding dictionary for a specific health issue and target audience;
2. Code qualitative data;
3. Synthesize coded qualitative data and generate summary statements;
4. Generate and adapt scale items


BACKGROUND

OVERVIEW

FoQus on Scales is one component of Project FoQus (A Framework for Qualitative Research in
Social Marketing), which incorporates qualitative methods to improve steps within the social
marketing research process. More specifically, FoQus on Scales is designed as an enhanced
method for scale development and adaptation within the PSI Behavior Change Framework. In
this chapter, we will discuss the tools and methods necessary to analyze the FoQus on Scales
qualitative data from focus groups and free-listing in order to adapt or develop determinants and
their definitions, and adapt or develop multi-item scales to measure those determinants. The
methods for collection of these focus group data are available in Toolkit Chapter 5.

DATA ANALYSIS AND MANAGEMENT

Developing and adapting a coding dictionary

The first step in the analysis of qualitative focus group data is to develop a coding dictionary (see
Appendix A). The coding dictionary provides a defined and finite list of terms used to identify themes
within portions of the transcribed texts. Steps toward the development of the coding dictionary include:
1. Development of the coding dictionary is a team effort in order to maximize on experience and
knowledge of different members of the project;
2. A coding dictionary is developed only after a portion of focus group data have been
transcribed and read by project team members. This allows for the development of content-
specific codes;
3. The codes reflect both the respondents and the researchers perspectives;
4. Codes include both broader determinant-related (e.g. self-efficacy) and content-specific terms
(e.g. casual partner);
5. Codes must be short and self-explanatory;
6. All codes must be clearly defined and examples provided as necessary;
PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


2

7. Determinant-related codes must be used consistently with the core definitions provided
through PSI. This allows for the possibility of cross-site data analysis;
8. The number of codes needs to be limited to between 15 to 25 terms. Too many codes will
result in too few comparable portions of text. Too few codes will result in texts which are too
broad to analyze;
9. The coding dictionary is a dynamic tool and codes can be added during the coding process as
new topics or themes emerge during analysis;


Coding Focus Group Data

Once a coding dictionary has been developed, the focus group data is coded by project personnel. The
purpose of the coding is to organize the textual data under specific topics and themes. The coding is a
text management tool to support data analysis it alone does not constitute data analysis. Steps for
coding include:

1. All coders must be familiar with the codes and their definitions. Coders will meet prior to
coding to review the dictionary;
2. Meetings are important during coding to review the coders interpretations of texts, refine
definitions, and/or add new codes;
3. New codes are added as themes or issues emerge from the text. New codes need to be
defined, and previously coded texts reviewed to include the new code(s) as applicable;
4. Portions of texts can be coded with more than one code. The purpose of the coding is to
provide a means of sorting texts for analysis. Therefore, accurate analysis depends on
complete and detailed coding;
5. To facilitate development of scales, statements generated during free listing will include at
least two codes: one content specific code (e.g. number of partners) and one determinant-
related code (e.g. social norms);
6. Transcribed texts from each focus group is coded independently by at least two coders. The
purpose of this double coding is to increase inter-coder reliability. In some instances, both
coders will work on-site, and in other circumstances one coder (regional researchers) will be
on-site. More than one set of two coders can be used to speed up the coding process.

a. After a selected number of texts (e.g. 3 or 4) are double coded, all the coders will meet or
have a conference call (e.g., if a regional researcher is involved) to discuss their decisions
regarding the coding;
b. Any significant discrepancies are discussed, and a decision made regarding how to code
that portion of text;
c. Decisions made about coding during these meetings are incorporated into the definitions
of the codes in the dictionary;
d. If these decisions possibly effect other portions of the text, the transcriptions must be
reviewed and changes made as appropriate
e. Remaining transcribed texts can be either double-coded or single-coded depending on
time and resources. If double-coded, meetings are conducted between coders at regular
intervals.

Using Microsoft Word for Coding Text
While several software programs are available for coding texts, because of the many PSI sites
utilizing FoQus on Scales and the need for training on these software, an alternative method has been
PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


3

adopted, which utilizes Microsoft Word. Microsoft Word is readily available across sites and does
not require any special training. This method will permit the numbering of the texts to more easily
identify the portion of text allocated to any one code, as well as simplifying the searching and
synthesis tasks. [see Sample in Appendix B]
The steps for this process are:

1. Number Lines of Transcribed Text.
a. Go to File menu, and click Page set-up
b. Click Layout tab
c. Click Line Numbers
d. Select Add line numbering
e. Select Continuous option
f. Hit ok on line numbering and layout screens
g. Text will be numbered starting with 1 and continuing through the end of the
document.
2. Create Coding Table.
a. Go to a blank document
b. Under Table go to Insert Table
c. Create a one row and one column table
d. Hit ok
e. Under Table go to Table Properties
f. On Table tab
i. Size Preferred Width = 4 cm
ii. Alignment = right
iii. Text wrapping = around
g. On Row tab
i. Specify Height = 24 cm
h. Hit ok single page
i. Click Table tab and Select Table
j. Use Control C to copy table.
3. Copy Coding Table into Text Document
a. Place cursor at the beginning of the first line of text;
b. Use Control V to paste table into first page of document;
c. Continue to paste the table on each subsequent page.
4. Entering Codes
a. Place cursor within coding table
b. Reading through the text determine portion to be included for a specific code;
c. Enter code into the table preceded by * [the * provides a means of identifying
codes rather than words within the text for search purposes];
d. Enter lines of text matching the code;
e. Continue coding.

PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


4

Focus Group Data Analysis

The FoQus on Scales focus groups are specifically tailored to: 1) adapt existing determinants
and/or identify new determinants; and, 2) modify existing scales and/or develop new scales.
Therefore, analysis of these data will be directed toward defining or redefining determinants
within a particular cultural context and modifying items in existing scales or generating items to
develop new scales.

The following steps are used for this analysis process:

1. Systematic searches are conducted through all of the data for same-coded texts. If the
Microsoft coding system is used, these searches can be conducted through the Microsoft
program (Edit Tab, Find Option);
2. A separate compilation document is developed for each of these same-coded texts (See
Figures 1 & 5). Using copy and paste, each identified portion of text is copied into this
new document from the original data. The numbering of the text and reference to these
numbered texts in the coding column makes this process more accurate;
3. Other information copied into this document include the identification number for the
focus group discussion where the data originates , type of participants in the group (e.g.
male youth ages 15 to 19), and content codes used to code the portion of text;
4. After all data have been searched, and relevant coded text separated into each of the
compilation documents, the research staff reviews these documents for themes within the
coded texts as well as across the coded texts. The research staff also reviews the data to
make sure that all text fall under the determinant code and make changes as appropriate
after a discussion among all the coders;
5. Research staff will develop a summary statement for each compilation document
whenever new scales are being generated (See Figure 1). This statement provides a
succinct overview of the primary themes and issues .
6. For purposes of item generation, within each of the compilation documents, free listed
statements as well as supporting information from the general text are reviewed to
determine possible items for inclusion in the scale.
7. The frequency and saliency of determinants are also assessed through the compilation
documents to identify the top six to eight determinants for which scales will be
developed. This is done by counting the number of focus groups in which certain
determinants are mentioned as well as making note of the determinants that appear to be
more relevant, important or significant than others (e.g., potential for high risk,
immediate influence of the determinant on the behavior);.
8. Determinant definitions are updated or produced and scale items are generated or
modified only for the top six to eight determinants in terms of frequency and saliency.

PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


5


FIGURE ONE: COMPILATION DOCUMENT AND SUMMARY STATEMENT





Determinant Code=social norms
Content code Focus group data Identifier
PARTNER
TYPE





But with what type of partner does one use a condom?
Response: My husband, for example.
Youre saying that you use a condom with your husband?
Yes, because when he goes out, I dont know what hes doing. I dont trust him at
all.

Now what are the factors that compel young people to use condoms?
When they frequent prostitutes.
When you know that the girl has several partners (li nan chen li nan chat).
If one doesnt know the other person, one might not have unprotected sex with that
person.

When one thinks that the girl might have several partners, one is forced to use
condoms.

FGD1f2024





FGD5M2024






FGD6M2024

SITUATION




People dont use one when its the first time.
That depends on the person.

Are there occasions when young people dont use condoms?
When one is very familiar with their partner (everyone needs to have sex).

When do young people use condoms?
When they go to cafs (or night clubs).
They dont use them when theyre at home

If the person is afraid, if she doesnt want to be, or if he or she has only just met the
person and wants to have sex with them right at that moment, the person uses a
condom

If one doesnt trust their partner.

Tell me gentlemen-why is it that you dont use a condom when you make love?
If one is faithful to their woman, you dont need to.

FGD4F2024


FGD6M2024


FGD8F2024




FGD7F2024



FGD2M1519


FGD2M1519

Summary
Statement
Youth use condoms with prostitutes, their spouse, with people that have several
partners. Youth dont use condoms with partners that they are faithful to. Youth use
condoms when they go to a caf/night club, when they are afraid, when just met
someone and when they are afraid of infection.. Youth dont use condoms the first time
they have sex and when they are familiar with their partner.

FGD1-8 (unique identifier for focus group discussion)
M=male
F=female
1519 (youth ages 15 to 19)
1924 (youth ages 20 to 24)

PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


6

ITEM GENERATION

STEPS FOR GENERATING MULT-ITEM SCALES

Step One Adapt the existing determinant definition
Step two Update the existing determinant definition
Step Three Define any new determinants
Step Four Establish the level of specificity
Step Five Decide on appropriate response options
Step Six Generate items that get at the same idea in different ways
Step Seven Match items with definition (add/delete items as necessary)
Step Eight Up-date determinant definition as needed


Once data have been synthesized this information is used to generate items for new scales or modify
existing scale items. If the determinant is in the PSI Behavior Change Framework and/or defined in peer
reviewed articles (See Chapter 1, Literature Reviews), the first step is to adapt the existing definition to
health area, target group and behavior. In the second step, the summary statement, compiled data, and the
synthesis are used to further specify and up-date the determinant definition. If the determinant is not
included in the PSI Behavior Change Framework and peer review articles, a new definition is developed
through the summary statement, compiled data, and the synthesis. . Determinant definitions will explain
what is being measured by the determinant and describes different aspects or dimensions of the
determinant.

In the first example below (Figure Two), the definition in the PSI Behavior Change Framework for social
norms is first adapted for a specific health area, target group and behavior. It is then modified or updated
using data from focus group discussions. Two context specific dimensions or aspects of social norms are
identified including types of partners with whom condoms are used and situations when condoms
are used.

In the second example, a new definition is generated for a new determinant, pregnancy intention, which
emerged during the study. Thus, for step three in cases when a determinant is not included in either the
PSI Behavior Change Framework or existing literature the first two steps are skipped and the summary
statement and compiled data are used to generate the determinant definition.

Once determinants are defined and dimensions identified, in step four the level of specificity for the
scales is determined (see Toolkit Chapter 2). With sparse data across different dimensions or aspects of a
determinant, a one dimensional scale is developed that broadly measures all aspects of that determinant
(see Example Two). When there is a significant amount of data about one aspect of a determinant one
dimensional scale is developed which measures this aspect of the determinant in detail. If rich detailed
information about different aspects of a determinant is available, a multi-dimensional scale with subscales
is developed. In example one, data are available about condom use with different types of partners and
in different situations. Therefore the decision was made to develop a scale that measures two different
aspects of social norms for condom use.


PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


7

FIGURE TWO: EXAMPLES FOR ADAPTION AND DEVELOPMENT OF DETERMINANT
DEFINITIONS



In Steps five and six a pool of items is generated and response options determined. These steps typically
take place at the same time to ensure that response options and statements match. A four point Likert
scale is recommended for use in PSI TRaC studies. A Likert scale consists of statements each followed
by response options indicating levels of agreement or disagreement. Detailed information on the number
of response options that can be used including situations where a neutral point may be appropriate for a
Likert scale can be found in Toolkit Chapter 3 (Response Options). Prior to generating items, criteria for
good items and guidelines for generating items are reviewed (See Toolkit Chapter 4, Scale Development
& Adaptation). Items are reviewed for clarity and to ensure that each item contains only one idea.

Statements to be included in multi-item scales are developed using determinant definitions that have been
generated or modified and the compiled raw data, including the free-list statements within the compilation
documents for each determinant. If the determinant definition includes many dimensions, focus on
generating items for one dimension at a time.

Generally at least one item is developed for each concrete expression of the determinant found in the
compiled data. These items should closely reflect the ideas expressed in the raw data as well as any minor
variations (See Figures Three and Four)
EXAMPLE ONE: Adapting a PSI definition (Steps One, Two, and Four)

PSI Behavior Change Framework Definition for Social Norms: Perceived standards of behavior
that are accepted as the usual practice.

Adapted Definition for Social Norms: Perceived standard for condom use that are accepted as the usual
among youth in Haiti.

Summary statement: Youth use condoms with prostitutes, their spouse, with people that have several
partners. Youth dont use condoms with partners that they are faithful to. Youth use condoms when they go to
a caf/night club, when they are afraid, when just met someone and when they are afraid of infection.. Youth
dont use condoms the first time they have sex and when they are familiar with their partner.

Updated Definition for Condom Use Social Norms: Perceived standards for types of partners with
whom youth use condoms and situations when condoms are used.


EXAMPLE TWO: Developing a definition for a new determinant (Step Three)

Summary statement: Youth want to have a child with a sexual partner for financial gain, to increase
their social status and because of physical attraction to a partner.

Definition for Pregnancy Intention: An individuals desire to become pregnant/have a child. This
desire can be motivated by personal, social or economic benefits.

PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


8


FIGURE THREE: EXAMPLE ITEMS FOR PARTNER RELATED CONDOM USE SOCIAL
NORMS

















Text data types of partners and
condom use social norms
Items Generated


When they frequent prostitutes.


Statement 1 : Young people use
condom when have sex with
prostitutes.
When you know that the girl has
several partners (li nan chen li nan
chat).

When one thinks that the girl might
have several partners, one is forced to
use condoms:
Statement 2: Young people use
condoms with a partner who has
multiple partners.


But with what type of partner does
one use a condom?
Response: My husband, for
example.
Youre saying that you use a
condom with your husband?
Yes, because when he goes out, I
dont know what hes doing. I dont
trust him at all.


Statement 3 : People use condoms
with their spouse.


PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


9

FIGURE FOUR: EXAMPLE ITEMS FOR SITUATION RELATED CONDOM USE SOCIAL
NORMS

Text data situations & condom use social
norms
Items Generated
People dont use one when its the first
time.
That depends on the person.


Statement 1: People dont use condoms
the first time they have sex.

If someone is faithful, they can make love
without using a condom.

Tell me gentlemen-why is it that you
dont use a condom when you make love?
If one is faithful to their woman, you
dont need to.


Statement 2: When partners are faithful to
one another, they dont need to use
condoms.

Are there occasions when young people
dont use condoms?
When one is very familiar with their
partner (everyone needs to have sex).

Statement 3 : Youth dont use condoms
with partners they know well.

When do young people use condoms?
When they go to cafs (or night clubs).
They dont use them when theyre at
home.

Statement 4 : Condoms are used when
having sex with someone met at a caf
(night club)



If the person is afraid, if she doesnt want
to be, or if he or she has only just met the
person and wants to have sex with them
right at that moment, the person uses a
condom
Statement 7 : Young people use condoms
with someone they have just met.

Statement 8 : A condom is used when one
of the partners doesnt trust the other.



Once items have been generated review items and reword any that dont accurately reflect the
determinant definitions. Items are then reviewed and revised based criteria for writing items (see Toolkit
Chapter 4: Scale Development and Adaptation).

In steps seven and eight review the items to ensure that they reflect the determinant that you are
measuring. If not, decide if the item should be dropped, or if perhaps there is a need to modify the
determinant definition.

PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


10

ITEM MODIFICATION FOR EXISTING SCALES

STEPS FOR MODIFYING MULT-ITEM SCALES
Step One Review and compare existing scale items and compiled data
Step Two Modify phrases/words in existing items
Step three Modify response options
Step Four Identify and add new items
Step Five Drop contextually or conceptually irrelevant items
Step Six Determine level of specificity for scale
Step Seven Update the existing determinant definition
Step Eight Adapt the existing determinant definition


Existing scales are modified for use in PSI TRaC studies whenever they are available and in-
country researchers and programmers elect to adapt them to local contexts. A PSI scales bank
has been created to make the process of identifying scales easier for researchers and marketers.
The scales bank contains PSI and Non-PSI scales for determinants found in the PSI Behavior
Change Framework (see Appendix C).

Non-PSI scales were gathered through a comprehensive literature review. PSI scales were
compiled from HIV/AIDS SMRS reports. Currently PSI scales included are from HIV/AIDS
study reports in the 2005 and 2006 Social Marketing Research Series. PSI scales in other health
areas will be available as reliable multi-item scales in SMRS reports in these areas are reviewed.
The scales bank includes a fact sheet which contains information on the intended use of the
scale, recommended modifications, and a reference for the source article or report.

Once an existing scale has been identified and any recommended changes have been made for
those found the PSI scales bank, the first step in modifying existing scales is to review and
compare the data and themes in the compilation document with the items in the scale.

Below is compiled focus group data on attitudes about condoms followed by items from an
existing attitude scale from the Scales Bank. This scale has been modified based on these data..
To be consistent with the definition of attitudes from the PSI Behavior Change Framework,
items that referred to buying condoms were dropped. Generally, items about condoms were
reworded into phrases about using condoms..


PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


11


FIGURE FIVE: COMPILATION DOCUMENT









Determinant Code=attitude
Content code Focus group data Identifier
TIME
"We are living in a country where the tendency is to protect ourselves in everything
we do, and it's the same thing when we make love. Using a condom takes too much
time.

"We're on our guard; it's something we do secretively and in that case we don't have
time to use a condom."

"Condoms waste too much time during sex."

When they start having sex, young people think they are going to use a condom, but
then they feel like it takes too much time and they lose interest.
FGD1f2024




FGD5M2024


FGD6M2024


FGD6M1519


COMFORT It's more comfortable without a condom. How do you explain that? It means that
with a condom you feel uncomfortable, and that you prefer direct skin-to-skin
contact."

When sex is protected, we don't feel comfortable. It shows a lack of sensuality, of
feeling, of sexual enjoyment."

There are girls who feel uncomfortable when they use one.
FGD4F1519



FGD4F2024

PLEASURE




"When sex is protected, we don't feel comfortable. It shows a lack of sensuality, of
feeling, of sexual enjoyment."

"People refuse to use a condom to have much greater sensation (the condom
decreases the sensation)."

"You say that you can have condoms handy and not use them. What can that mean?

It means that you're going to change the rhythm and that coitus won't be as
effective."

Without condoms the feeling isn't the same. When you use condoms you don't feel
good. Without a condom it's more pleasurable.

In my opinion, condoms aren't effective.
They often say that the condom takes everything out of it."

FGD4F2024


FGD6F2024



FGD6F2024





FGD6M1519

FGD6M2024

FGD1-8 (unique identifier for focus group discussion)
M=male
F=female
1519 (youth ages 15 to 19)
1924 (youth ages 20 to 24)
PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


12







Modified Condom Attitude Scale

Source: Otto-Salaj, L. L., Kelly, J. A., Stevenson, L. Y., Hoffmann, R. & Kalichman, S. C. (2001).

1. Using condoms can make sex more stimulating
2. Using condoms is uncomfortable.
3. Using condoms can be pleasurable.
4. Using condoms ruin the "mood."
5. Using condoms interrupt foreplay.
6. Using condoms is unreliable.
7. Using condoms is an effective method of preventing sexual diseases.

The second step is to modify phrases/words in existing items. In the above example condom
attitude scale modifications are not made since the wording/phrases in the existing scale items
are consistent with the data and do not require modification for use among the target group.

In step three, modifications are made to response options as needed. In the original scale a six
point likert scale was used. A decision is made to retain this number of response options and
conduct a pre-test to see if participants are able to distinguish between all categories and if the
difference between the categories can be translated into the local language. (See Chapter 9 for
pre-testing multi-items scales and response options).

In the fourth step, information in the compilation data that is missing from the existing scale is
noted. In the above example most of the themes (e.g., time, pleasure, comfort) in the compilation
data are already included in the scale. However, additional items are generated for time, pleasure
and intimacy to more accurately reflect the data (see Figures 6 and 7). Also, in the existing scale
Determinant Code=attitude
Content code Focus group data Identifier
INTIMACY





"Condom's prevent intimacy. So the feeling's in the condom."

"There is more intimacy when you don't use a condom" (already discussed)
Some people tell me that condoms decrease sensation and that they prefer to have
unprotected sex even if they are aware of the risks they are taking."




FGD6M2024


FGD6M2024

PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


13

there were no items regarding sexual intimacy, and therefore new items are generated to reflect
this aspect of the determinant (see Figure 8).



FIGURE SIX: EXAMPLE ITEMS FOR PLEASURE RELATED CONDOM USE ATTITIDES

Text data Pleasure and condom use
attitudes
Items Generated
Wncn scx is prciccic!, uc !cni jcc|
ccnjcria||c. |i sncus a |ack cj scnsua|iiq, cj
jcc|ing, cj scxua| cnjcqncni.


Without condoms the feeling isn't the same.
When you use condoms you don't feel good.
Without a condom it's more pleasurable.
1a. Using condoms decrease sexual
enjoyment.
1b. Using a condom decreases sexual
pleasure

Pccp|c rcjusc ic usc a ccn!cn ic natc nucn
grcaicr scnsaiicn (inc ccn!cn !ccrcascs inc
scnsaiicn).


3a. Using condoms decreases sexual
feelings

|n nq cpinicn, ccn!cns arcni cjjcciitc.
Tncq cjicn saq inai inc ccn!cn iakcs ctcrqining
cui cj ii.

3b. Using condoms diminished sexual
satisfaction

Ycu saq inai qcu can natc ccn!cns nan!q an!
nci usc incn. Wnai can inai ncan?

|i ncans inai qcurc gcing ic cnangc inc
rnqinn an! inai cciius ucni |c as cjjcciitc.

4a. Using a condom changes the rhythm of
sex.


FIGURE SEVEN: EXAMPLE ITEMS FOR TIME RELATED CONDOM USE ATTITUDES
Text data Time and condom use attitudes Items Generated
. Using a ccn!cn iakcs icc nucn iinc.

Ouring scx, cnc cxpcricnccs cncrncus
p|casurc, scnsaiicns, an! cjicn uc !cni natc
iinc ic usc a ccn!cn.

8. Using a condom takes too much time
Wcrc cn cur guar!, iis scncining uc !c
sccrciitc|q an! in inai casc uc !cni natc iinc
ic usc a ccn!cn.
9. Using a condom takes too much time
when you are having sex secretively. .

PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


14

Ccn!cns uasic icc nucn iinc !uring scx. 10. Using condoms waste too much time
Wncn incq siari nating scx, qcung pccp|c inink
incq arc gcing ic usc a ccn!cn, |ui incn incq
jcc| |ikc ii iakcs icc nucn iinc an! incq |csc
inicrcsi.



FIGURE EIGHT: EXAMPLE ITEMS FOR INTIMACY RELATED ATTITUDES

Text data Intimacy and condom use
attitudes
Items Generated
Ccn!cns prctcni iniinacq. Sc inc jcc|ings in
inc ccn!cn.

11. Using condoms interferes with sexual
intimacy.
Tncrc is ncrc iniinacq uncn qcu !cni usc a
ccn!cn (a|rca!q !iscussc!)
Scnc pccp|c ic|| nc inai ccn!cns !ccrcasc
scnsaiicn an! inai incq prcjcr ic natc
unprciccic! scx ctcn ij incq arc auarc cj inc
risks incq arc iaking.


12. Using condoms decreases sexual intimacy

In the fifth step, contextually irrelevant aspects of the scale are identified. In the above example,
ideas related to reliability and foreplay are not mentioned in the qualitative data. However, the
decision is made that since there is no data indicating these aspects are not important, those items
will be retained prior to pre-testing and piloting the scale (see Chapters 9 and 10).

A scale with a total of 17 items was developed for the pilot study. Some if these items address
new themes discovered in the data and others build on existing themes. Items are number so that
new items that address existing themes have the same number used in the original scale item.
New items within these themes are identified by letters (e.g. 1a, 1b).

1. Using condoms can make sex more stimulating
1a. Using condoms decrease sexual enjoyment.
1b. Using a condom decreases sexual pleasure
2. Using condoms is uncomfortable
3. Using condoms can be pleasurable.
3a. Using condoms decreases sexual feelings
3b. Using condoms diminished sexual satisfaction .
4. Using condoms ruin the "mood."
4a. Using a condom changes the rhythm of sex.
5. Using condoms interrupt foreplay.
6. Using condoms is unreliable.
PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


15

7. Using condoms is an effective method of preventing sexual diseases
8. Using a condom takes too much time
9. Using a condom takes too much time when you are having sex secretively. . .
10. Using condoms waste too much time.
11. Using condoms interferes with sexual intimacy.
12. Using condoms decreases sexual intimacy

In step six, the level of specificity for the scale is determined. In this case a decision is made to
retain a one dimensional scale. In this example, it is difficult to conclude if any of the themes are
more important than other themes and there is insufficient data to create subscales for all
possible dimensions (e.g. comfort, intimacy).

After the scale has been adapted, in step seven the PSI determinant definition for attitude is
updated based on the new items that have been included and/or excluded and the modifications
of existing items. In step eight, the up-dated definition is adapted for the specific health area,
target group, and/or behavior.

Example PSI definition

PSI Behavior Change Framework Definition for attitude: An attitude is an individuals
evaluation or assessment about the promoted behavior

Updated Definition for Attitude about Using Condoms: An attitude is an individuals evaluation
or assessment related to time, pleasure, intimacy and comfort associated with using condoms.

Adapted Definition for Social Norms: An attitude is an evaluation or assessment made by youth
living in Haiti related to time, pleasure, intimacy and comfort associated with using condoms.

DATA ANALYSIS WORKSHOP (OPTION)

This chapter has presented the tools and methods for analyzing qualitative data for FoQus on
Scales. The previous sections have included tasks which are done by a team of researchers over a
period of time. Another option for the data analysis phase of FoQus on Scales is the use of a
three-day workshop during which data coding and compilation are completed, as well as
identification and defining of determinants and associated scale items. This data analysis
workshop can be conducted separately, or it can be combined with the researchers and local
researchers and programmers interpretation workshop (see Chapter 7). In the case of the
combined analysis and interpretation workshops, the overall length will be approximately five
days with the introductory materials presented in Day One, the analysis workshop in Days Two
and Three, and the interpretation workshop in Days Four and Five. An outline of the curriculum
for the analysis workshop is presented in Figure Nine, and a full curriculum is included in
Appendix D.

PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


16

FIGURE NINE: OUTLINE OF ANALYSIS WORKSHOP CURRICULUM
DAY ONE
Session 1: Background and introduction
Activity 1: Introductions, Overview of the Training, and Training Objectives [Large Group 30 MINUTES]:
Objectives: 1) share some personal information about themselves with the group; 2understand the purpose of the training
and 3) know the various activities to be completed during the training;

Activity 2: Research Background Review [Large Group-30 MINUTES]
Objective: know about the rational for the study and know and understand the research objectives.

Activity 3: Overview of Coding [Large Group-30 MINUTES]
Objectives: 1) be familiar with the FoQus on Scales Behavioral Change Framework; 2) understand the role of coding within
qualitative data analysis (as conducted in FoQus on Scales); 3) be able to distinguish and understand the two types of codes
determinants and content codes; and 4) understand terms used as a part of the coding process.

Session 2: Developing a coding dictionary
Activity 1: Reviewing Transcripts and Identifying Themes [Small Groups and Large Group-90 MINUTES]
Objectives: 1) start to be familiar with the content of the transcribed data; 2) start to identify themes within these data;3)
share identified themes and start the process of determining themes for which codes need to be developed.

Session 3: Developing a coding dictionary
Activity 1: Coding Transcript and Refining Code Definitions [Individual and Large Group-90 MINUTES]
Objective: 1) Be familiar with the process of coding so that can complete the coding of the transcripts and 2) developed a
more refined coding dictionary to facilitate coding.

Session 4: Establishing inter-rater reliability
Activity 3:Double coding- Reliability Coding Session I [Small Group-90 MINUTES]
Objective: know that the coding dictionary is being used consistently by all workshop participants (inter-coder reliability)

DAY TWO
Session 1: Second coding meeting, updates and revisions
Activity 1: Group discussion and agreement- Reliability Coding Session I [Small and Large Group-90 minutes]
Objective: 1) Review, discuss and reach consensus regarding decisions made during small group work; 2) Update the coding
dictionary and 3) Finalize coding of transcripts from day one.

Session 2: Begin coding
Activity 1: Code Transcripts Session I [Individual-90 MINUTES]
Objective: Complete coding one more transcript each.

Session 3: The third coding meeting, updates and revisions
Activity 1: Review of Coding [Large Group-30 MINUTES]
Objective: Review any new codes or changes to definitions.

Session 4: Continue coding
Activity 10: Code Transcripts Session II [Individual-150 MINUTES]
Objective: Continue to code transcripts.

DAY THREE
Session 1; Finalize coding
Activity 1: Code Transcripts Session III [Individual-90 MINUTES]
Objective: Complete coding transcripts and entering codes into the electronic version of the transcripts.

Session 2: Data compilation
Activity 1:Compiling Coded Texts [Individual 90 MINUTES]
Objective: Conduct a search and compile texts for individual codes. All codes will have been searched and compiled.

Session 3: Review of steps for writing summary statement
Activity 1: Write summary statement for one determinant [Large Group Activity 90 MINUTES]
Objective: Review purpose and procedures of writing summary statements for compiled texts [determinant codes only].

Session 4: Writing summary statement
Activity 14: Write summary Statements for all Determinants [Individual-90 MINUTES]
Objective: To complete summary statements for all determinant codes


PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


17

QUALITY IMPROVEMENT CHECKLIST

Data analysis
Create succinct and well-defined codes prior to beginning coding procedures. The
level of analysis is dependent on good coding. The coding provides the means for
searching and summarizing data during the analysis phase;
Have at least two people code the first three to four transcripts. After double coding
hold meetings to discuss any discrepancies in coding, make decisions about these
discrepancies, and incorporate changes into the coding dictionary
revisit codes on a regular basis throughout the coding process to ensure that changes
in definitions or new examples are incorporated into the coding dictionary
Recode previously coded transcripts as appropriate when changes are made to the
coding dictionary

Item generation

Update existing determinant definitions using summary statements
DEFINE NEW determinants using summary statements
Generate items using updated or new determinant definitions AND compiled data
FOR each determinant

Item adaptation
Modify existing scales using compiled data for each determinant
Update determinant definitions for modified existing scales to reflect the changes made
to these scales
PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


18

CASE EXAMPLES AND LESSONS LEARNED


CASE EXAMPLE: 2006 YOUTH HAITI FOQUS ON SCALES STUDY

STUDY PURPOSE

Scale development for TRaC studies involves modifying existing scales and developing new scales. In
the summer of 2006, PSI Haiti conducted a qualitative study to develop multi-item scales. The purpose of
this study was to identify determinants of condom use relevant to Haitian youth ages 15 to 24 and develop
multi-item scales to measure these determinants.

DATA COLLECTION

A semi-structured focus group guide was developed with in-put from the country PI. This guide used
free-lists to generate statements on determinants of behavior and used pile sorts to have participants group
and define relevant determinants. Prior to data collection, the interview guide was piloted on two groups
and further revised. . Separate focus groups were conducted for younger and older and male and female
youth with a total eight focus group discussions. Data from the free-listing exercise were used to identify
relevant determinants of behavior and generate scale items. Data from the pile sorts were found be of
poor quality and not used.

DATA ANALYSIS

A coding dictionary was developed based on the definitions for the determinants in the PSI Behavior
Change Framework. New codes were added for determinants such as pregnancy intentions that were not
in the PSI Framework. The data from the free listing were coded by the research staff in Haiti and the
research backstop separately. Research managers and the research backstop met for one day to review the
codes and reconcile any differences. Next, these coded data were examined for frequency of determinants
mentioned across focus group discussions. Eight of the most frequently mentioned determinants were
identified for inclusion in the questionnaire for the TRaC study. A data management method similar to the
one described earlier in this chapter was used to compile data on each determinant identified.

DATA INTERPRETATION AND ITEM GENERATION

Five PSI program and research staff and three peer educators from a partner organization participated in a
two-day data interpretation and item generation workshop. This workshop was co-facilitated by the
research backstop and the director of the research department for PSI Haiti.

The first morning of the workshop was devoted to providing participants with background information
and knowledge they needed to generate multi-item scales. The workshop began with a presentation on
TRaC studies and multi-item scales. Participants were then introduced to criteria for writing statements to
be used in multi-item scales and definitions for the relevant identified determinants. Individual and small
group exercises were used to increase participants skills and understanding in these areas. By the end of
that first morning, participants were exposed to the entire process of scale development.

PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


19

During the afternoon, participants worked in small groups consisting of at least one researcher, one
programmer and one peer educator to generate items. Participants interpreted data and generated items in
these groups. Large group discussions were used to share items and challenges.

The entire second day was spent on data interpretation and item generation. At the end of the workshop
all items generated were compiled by the research staff and reviewed. Duplicate items were deleted.

SUCCESS, CHALLENGES AND LESSONS LEARNED

Successes of the Haitian study included:

Reliability of multi-item scales developed using the focus group data ranged from .78 to .96.
This is within the good to excellent range for scale reliability;
A number of significant determinants of condom use were identified through the
segmentation analysis suggesting that the scales used were of good quality
Scales used to measure these determinants were rich in contextual detail. Therefore, these
data provide information that can be used to design interventions that are uniquely tailored to
Haitian youth.

Challenges of the Haitian study were primarily in relation to data collection. These challenges included:

The interview guide required highly skilled moderators who were familiar with the PSI
Behavior Change Framework.
o Based on this experience the interview guide was simplified.
The data collected by the agency selected to conduct the focus group discussions reflect poor
moderating skills and lack of understanding of the purpose of the study.
o Different implementation options are provided so that the first implementation option
requires less resources and capacity, e.g., through free listing (see FoQus on Scales
concept paper).
o Pile sorting that requires higher moderating and analysis skills should be used in
subsequent implementation options (see FoQus on Scales concept paper).

Several important lessons were gained from PSI Haitis experience analyzing and interpreting qualitative
data and generating items.
Data should be coded at several levels to allow for identification of dimensions within scales
(e.g. self-efficacy for using condoms, self-efficacy for negotiating condom use).
When data with similar codes are compiled this information should be used to write a
summary that can be used to define new determinants or modify definitions for determinants
in the PSI Behavior Change Framework.
A decision about the level of specificity for a scale (a very specific one dimensional, a broad
one dimensional or multi-dimensional) should be decided prior to generating items.
The content validity of scales should be checked and improved by comparing the items
generated against the determinant definition and either revising the definition or moving items
to other scales. Following this process allows for determinant definitions and items that are
consistent with one another and based on findings from the qualitative study.

PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


20

REFERENCES
Otto-Salaj, L. L., Kelly, J. A., Stevenson, L. Y., Hoffmann, R. & Kalichman, S. C. (2001). Outcomes of a
Randomized Small-Group HIV Prevention Intervention Trial for People with Serious Mental Illness.
Community Mental Health Journal 37(2): 123-144.
PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


21

Appendix A: Example coding dictionary

Definition-Primary Codes (to identify determinants)

Threat (adapted PSI) An individuals perception of the magnitude of harm posed by HIV/STIs. Threat
is first comprised of severity, which is an individuals perceived magnitude of the harm of HIV/AIDS (i.e.
significance or seriousness of HIV/AIDS AND susceptibility is an individuals perceived likelihood that
HIV/AIDS negatively harms him/her (i.e. degree of vulnerability, personal relevance, risk of experiencing
the public health problem).

Social norms (adapted PSI) Behavioral standards (cultural and religious etc) which exist in the
community for an individual to follow.

Locus of control (adapted PSI) Is the external or internal site of control of involvement in concurrent
relationships. An external locus of control suggests that an individuals behavior related to concurrent
partnerships is under the control of powerful other or is determined by fate, luck or chance. An internal
locus of control suggests that an individuals behavior in this area is directly controlled by him/herself.

Beliefs (adapted PSI) Is a perception about an object/behavior, which may or may not be true e.g. spirits
can make you have concurrent partners

Outcome expectations Positive or negative expectations or consequences of being in monogamous
relationship.

Pressure Strong negative or positive influence from significant people to engage in concurrent
relationships

Transaction Exchange of sex for material, financial gains and other favours.
Sexuality Perceived inherent drives to have more sex, need for different types of partners.
Quality of Relationships Satisfaction or dissatisfaction in a recurrent sexual relationship.
Media exposure The influence of media on sexual practices
Revenge Involvement in another relationship with intention to hurt another person.
Recreation Sex as a form of entertainment.
Outcome expectations -Ve or +Ve consequences of engaging in current relationships.
Free time Time that is not allotted for a specific activity
Perceived benefits Positive outcomes specifically material or financial gain that someone gets as a
result of being involved in concurrent relationships.

PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


22

Definitions-Secondary codes (content codes to capture contextual information)
Culture Any references to traditions and belief systems. A set of learned beliefs, values and behaviours
the way of life shared by the members of a society
Religion Beliefs and practices regarding supernatural beings, power, and forces. These beliefs and
practices can be general or specific to concurrent partnerships.
Expected roles/responsibilities duties expected of a partner including the manner in which one partner
relates to another.
Age-differences Difference in age between sexual partners
Emotions Feelings about a situation, person or object
Finances/money The amount of money someone has or needs. The ways in which individuals raise,
allocate, and use monetary resources.
Status/prestige The relative standing or position of a person in society or a community. A high
standing achieved through success, wealth or influence.
Sexual desire/drive Inherent sexual drive or desire
Alcohol use Use of alcohol generally or in relations to involvement to concurrent partnerships.
Sexual desire enhancing drugs Drugs taken to increase sexual desire or stamina
Partner characteristics Intrinsic factors such as being argumentative or being a good listener that are
unique to a partner .
Sexual fulfilment Sexual satisfaction or gratification
Partner availability Physical or emotional availability of a partner
Partner behaviour Any behaviours exhibited by a partner
Main partner The primary, married or official partner (dropped-not enough codes)
Partner communication The way in which a partner talks or listens to another partner
Seductive behaviour The way that a person moves, talks or dresses in an attempt to sexually attract
another person
Abuse Verbally/psychologically or physical hurting someone including physical violence
Situation/social setting Places where or conditions under which a particular act takes place
Roles/responsibilities Expected behaviours from a spouse or other sexual partners
PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


23


Material goods Anything of value such as food, housing and products (e.g. cell phones) that people
barter, obtain/purchase or sell.
Place/residence -a specific location or area where a person lives or spends time
Peers A group of people who are like one another or have something in common (e.g youth who live in
the same community and are the same age). The group can be composed of people who are friends but
this is not always the case.
People A group of persons/individuals,
Parent(s) Can be the biological father and/or mother of a child or one or more people who have
nurtured and raised a child without any biological relationship. Includes biological or step father and/or
mother and common-law mothers and/or fathers or adoptive fathers and/or mothers.
In-laws A spouses parents.
Love A strong positive emotion of regard and affection for a partner. It can include having strong
positive emotions for a partner despite any faults they may have.
Dislike A feeling of aversion or antipathy about a partner.





PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


24

Appendix B: Sample coding

Focus group discussion 1-male youth ages 15 to 19 (FGD1M1519)
I: man and a woman? 1
R: A good relationship between a man and a woman is that they have the same point of 2
view and the same idea. When discussing something, they have the same ideas and the 3
same point of view. They understand each other. In class, if the girl does not understand 4
lessons the boy explains to her the lessons. In general, they have to take care of each 5
other in studying. 6
I: Anything else? 7
R: In general, they talk many things to one another to understand each other more, and 8
closer to one another. 9
I: Can you have a good relationship that is also sexual? 10
R: No. There were no cases in my class. Girls and boys just sit and talk to one another, 11
help and take care of one another in studying or go out with other friends. There were no 12
cases like that. 13
I: Why is good relationship not a sexual relationship? 14
R: In general, we are grown up and we know it is not right and it is wrong. And the 15
school also provides necessary information to young people and students in school know 16
the information and avoid it. In general, students now know the information. They are 17
not like students in the past who did not understand it and unexpected things happened. 18
I: Under what circumstance might a young man and a young woman have sexual 19
relationship? 20
R: In general, there are sex workers standing on streets, young men often find a pleasure 21
and it happens. Or people who lack knowledge go with their boyfriends, they can not 22
control themselves and they are attracted to engage in activity that is unfortunate. It is 23
because their point of view is not strong enough. It will not happen if they are strong. 24
I: Would that couple use contraceptives? 25
R: Im not very clear. Contraceptives are usually condoms. Im not very clear. In general, 26
it is up to us, ourselves. Many people can not do it. 27
I: How would young people here get contraceptives? 28
R: About contraceptives, there are many. For example, when I was in school, the school 29
provided me about information that young people needed to know, and protect 30
themselves. In general, as a girl, she has to protect herself without being reminded by 31
anyone. She has to protect herself when she has a boyfriend. Usually, condoms are used 32
more. 33
I: Are there other kinds of contraceptives besides condoms? 34
R: Contraceptive pills. They are used for those who have given birth and they do not 35
want to have more children. They will use contraceptive pills. But as I know, young 36
women often use condoms more. 37
*relation 1-20
*student 1-20











*sexrelation 11-27
*CWS 21-27
*pleasure 21-27
*control 21-27


















*contraceptive 32-37
*healthed 32-37
*condoms 32-36





PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


25

Appendix C: Scale Bank

Existing scales are modified for use in PSI TRaC studies whenever they are available and
in-country researchers and programmers elect to adapt them to local contexts. A PSI
scales bank has been created to make this process easier for researchers and marketers.
The scales bank contains PSI and Non-PSI scales for determinants found in the PSI
Behavior Change Framework (see links below).

Non-PSI scales were gathered through a comprehensive literature review. PSI scales were
compiled from HIV/AIDS SMRS reports. Currently PSI scales included are from
HIV/AIDS study reports in the 2005 and 2006 Social Marketing Research Series. PSI
scales in other health areas will be available as reliable multi-item scales in SMRS reports
in these areas are reviewed. The scales bank includes a fact sheet for each determinant
which contains information on the intended use of the scale, recommended modifications,
and a reference for the source article or report.


PSI scales:

http://www.psi.org/research/toolkits/bank_psi_scales_hiv_aids.doc


Non-PSI scales:

http://www.psi.org/research/toolkits/bank_non-psi_scales_a&b.doc

http://www.psi.org/research/toolkits/bank_non-psi_scales_c.doc

PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


26

Appendix D : Curriculum for Focus Group Data Analysis Workshop

DAY ONE

Session one: Background and introduction

Activity 1: Introductions, Overview of the Training, and Training Objectives
Facilitator(s)
Objectives:
After these activities, participants will
1. have shared some personal information about themselves with the group;
2. understand the purpose of the training and know the various activities to be
completed during the training;
3. know and understand the training objectives.
Total Activity Time: 30 minutes

Time Step Method Materials
10 min Introduction





Large Group activity: Facilitator welcomes
everyone and discusses the purpose of the
workshop and topics that will be covered.
Participants light a match and say as much as
they can about themselves before the match
burns out.
10 min Overview of
Training
Presentation: Facilitators will present the
schedule for the training activities
10 min Review of
Training
Objectives
Presentation: Facilitators will present the
overall training objectives.
1. Learn how to develop a coding dictionary;
2. Develop a coding dictionary for the
current project;
3. Learn how to code transcripts;
4. Complete coding of transcripts from the
current project;
5. Learn how to compile coded text for
determinant codes;
6. Complete compiling texts for determinant
codes for current project;
7. Learn how to write a summary statement;
8. Complete summary statements for all
determinant codes for the current project.
PC and Projector

Power point
presentation




PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


27

Activity 2: Research Background Review
Facilitator(s):
Objective: After this activity participants will know about the rational for the study and
will know and understand the research objectives.
Total Activity Time: 30 minutes

Time Step Method Materials
20 to
30 min
Presentation of
Study Design &
Rational
Presentation: Explain rational for the
existing study (based on current
literature and/or contract with donor).
Review the research objectives.
PC and Projector

Power point
presentation
10 min Discussions &
Questions
Large Group activity: Opportunity for
workshop participants to ask questions
regarding the research rational and
objectives



Activity 3: Overview of Coding
Facilitator(s):
Objective: After these activities, participants will
1. have reviewed the PSI Behavioral Change Framework;
2. understand the role of coding within qualitative data analysis (as conducted in
FoQus on Scales);
3. be able to distinguish and understand the two types of codes determinants and
content codes
4. be familiar with terms used as a part of the coding process.
Total Activity Time: 30 minutes

Time Step Method Materials
20 min Presentation Presentation:Overview of FoQus on
Scales, review of the PSI Behavior
Change Framework including definition
for all determinants and explanation of
the role of Coding within FoS.


PC and Projector

Power point
presentation

10 min Discussion Large group work: Opportunity for
workshop participants to ask questions
regarding the PSI Behavioral Change
Framework, FoS, and coding


PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


28


Session 2: Developing a coding dictionary

Activity 1: Reviewing Transcripts and Identifying Themes
Facilitators(s):
Objective: After these activities, participants will
1. have started to familiarize themselves with the content of the transcribed data;
2. have started to identify themes within these data;
3. have shared identified themes and started selecting themes for which codes need to
be developed.
Total Activity time: 90 minutes

Time Step Method Materials
60 min Review
Transcripts
Small group work: Workshop
participants are divided into groups of
two.
Each group is provided with a copy of a
transcript. Participants will read
through the transcripts and take notes on
themes within the transcripts
Copies of Transcripts
30 min Free Listing and
Code Term
Identification
Large group work: Workshop
participants will come together as a
large group.
Each group will present their themes
[note: If participants in one group have
more experience, they should be the
first to present].
The themes are recorded on a board or
on a projector.
After the first presentation, subsequent
presenters note those themes which they
also have identified by marking next to
that theme. They should then add to the
list any other themes they identified.
Workshop participants and facilitators
then give a code word for each theme
starting with those themes identified
most frequently.
Workshop participants and facilitators
discuss the need to merge themes under
a single code and/or divide a single
theme into multiple codes.
Once all themes given codes, the
facilitators will divide the codes into
two categories: determinants and
content codes
PC and Projector

PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


29

Session 3: Coding practice and the first coding meeting

Activity 1: Coding Transcript and Refining Code Definitions
[Individual and Large Group]
Facilitator(s):
Objective: After this activity, participants will be more familiar with the process of coding
needed to complete coding all the project transcripts. Participants will also know how to
refine a coding dictionary to facilitate coding.
Total Activity Time: 90 minutes

Time Step Method Materials
45 min Code Transcript Individual work: Each participant is
provided with the same portion of a
transcript. He/she will individually
code the transcript.
Copies of Transcripts
45 min Review Coding
and Refine
Definitions
Large group work: The participants
come back together as a large group.
The facilitator goes through the code
words and asks for examples of when
that code word was used (participants
should read the text coded and if
necessary explain their decision to use
that code).
During this process, code definitions are
refined to more accurately reflect how
they are interpreted and used in the
coding process.
If necessary, codes can be modified,
e.g., combined or separated, and/or new
codes can be identified and defined.
PC and Projector with
Coding Dictionary





Session 4: Establishing inter-rate reliability

Activity 1:Double coding
Facilitator(s):
Objective: After this activity, participants and facilitators will know that the coding
dictionary is being used consistently by all workshop participants (inter-coder reliability)
Total Activity Time: 90 minutes

Time Step Method Materials
90 min Double Coding
Transcripts
[Note: Facilitators
should aim to have
Small Group work: Workshop
participants divide into groups of two.
Each of those two participants will code
the same transcript independently.
Copies of Transcripts
PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


30

25% of transcripts
double coded]
After coding, the two participants will
discuss their coding decisions.
Participants will take notes of
discrepancies in the way in which they
each coded.
Participants will reach consensus on
how to best code the transcript to reflect
the code definitions.


PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


31

DAY TWO

Session 1: Second coding meeting, updates and revisions

Activity 1: Group discussion and agreement

Objective: After this activity, participants will review, discuss and reach consensus
regarding changes made during small group work on inter-rater reliability, update the
coding dictionary and finalize coding of transcripts.
Total Activity Time: 90 minutes

Time Step Method Materials
30 min Double Coding
Transcripts

Small group: Workshop participants
will continue the coding process started
in Day 1 session 4 activity 1.

Copies of Transcripts
30 min Review of Double
Coding
Large group: Workshop participants
will come together as a large group.
Participants will discuss any
discrepancies experienced in the
reliability coding activity.
Participants will review and agree upon
the changes in the definitions (as
determined in the small groups).
PC and Projector with
Coding Dictionary
30 min Revise &
Complete Coding
of Transcripts
Small groups: Workshop participants
will reconvene into the same small
groups (2 participants each).
They will finalize the coding of their
transcript based on the decisions made
in the large group.


Session 2: Begin coding

Activity 2: Code Transcripts
Facilitator(s):
Objective: After this activity, each participant will have completed coding one more
transcript each.
Total Activity Time: 90 minutes

Time Step Method Materials
90 min Coding Transcripts
[Note: facilitators
should aim to have
50% of transcripts
coded by the end
of this session]
Individual work: Each participant will
individually code a single transcript.
Codes will be entered into the electronic
version of the transcript.
Facilitators will move around the room
to assist with questions and review
coding procedures with workshop
participants
Paper & Electronic
Copies of Transcripts
PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


32


Session 3 The third coding meeting , updates and revisions

Activity 1: Review of Coding
Facilitator(s):
Objective: To review any new codes or changes to definitions.
Total Activity Time: 30 minutes

Time Step Method Materials
45 min Review of Coding

Individual work: Participants will
reconvene in a large group to discuss
any changes in code definitions or use
of additional codes.
Changes will be made to the master
coding dictionary.

PC and Projector with
Coding Dictionary
45 min Revise coding [Note: if changes are made to the
dictionary, participants go back and
review their previously coded
transcripts including those used for
reliability testing, and make appropriate
changes in the coding.]
Paper & Electronic
Copies of Transcripts


Session 4: Continue coding

Activity 2: Code Transcripts
Facilitator(s):
Objective: Participants will continue to code transcripts.
Total Activity Time: 90 minutes

Time Step Method Materials
90 min Coding Transcripts
[Note: facilitators
should aim to have
75% of transcripts
coded by the end
of this session]
Individual work: Each participant will
individually code transcripts.
Codes will be entered into the electronic
version of the transcript.
Facilitators will move around the room
to assist with questions and review
coding procedures with workshop
participants

Paper & Electronic
Copies of Transcripts

PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


33

DAY THREE

Session 1: Finalize coding

Activity 1: Code Transcripts
Facilitator(s):
Objective: After this activity, participants will have completed coding transcripts and
entering codes into the electronic version of the transcripts.
Total Activity Time: 90 minutes

Time Step Method Materials
90 min Coding Transcripts
[Note: facilitators
should aim to have
100% of
transcripts coded
by the end of this
session]
Individual work: Each participant will
code a transcript.
Codes will be entered into the electronic
version of the transcript.
Facilitators will move around the room
to assist with questions and review
coding procedures with workshop
participants
Paper & Electronic
Copies of Transcripts

Session 2: Data compilation

Activity 1: Compiling Coded Texts
Facilitator(s):
Objective: Participants will learn how to conduct a search and compile texts for individual
codes. All codes will have been searched and compiled.
Total Activity Time: 90 minutes

Time Step Method Materials
90 min Compile Text for
Each Code
Individual work: Participants are each
assigned codes. For each code, the
individual will go through and identify
the text associated with that code. The
identified text will then be cut and
pasted (electronically) from the coded
transcript into the compilation forms.
Codes are compiled as outlined in the
FoQus on Scales Toolkit.
Electronic Copies of
Coded Transcripts
PCs for each
participant

PSI RESEARCH & METRICS TOOLKIT


FoQus on Scales: Data Analysis and Application


34

Session 3: Review of steps for writing summary statements

Activity 1: Write summary Statement for one determinant
Facilitator(s):
Objective: To review purpose and procedures of writing summary statements for compiled
texts [determinant codes only].
Total Activity Time: 90 minutes

Time Step Method Materials
15 min Review procedures
for writing
summary
statements
Presentation: Facilitators will present a
short review of the purpose and
procedures for writing summary
statements for the determinant codes.
This presentation will build on the
review of procedures from session 1
activities 1 and 3.
PC and Projector
Power point slides
45 min Identification of
Themes in
Compiled Data
Small group work: Participants will
break into small groups. Each group
will review a copy of data compiled for
a single determinants (all groups will
review the same data).
Group members will list themes within
the compiled data
Paper Copies of a
Single Determinant
Compiled Data Form
30 min Review Themes
and Write
Summary
Statement
Large group work: The participants will
reconvene into a large group. Small
group members will provide theme
identified from the compiled data.
Facilitators will incorporate these
themes into a summary statement
PC and Projector

Session 4: Writing summary statement

Activity 1: Write summary statements for all determinants
Facilitator(s):
Objective: To complete summary statements for all determinant codes
Total Activity Time: 90 minutes

Time Step Method Materials
90 min Write Summary
Statements
Individual: Each participant will be
assigned compiled text data for
determinants (each participant works on
separate determinants).
Participants will write summary
statements for the determinants.
Facilitators will move around the room
to assist with questions and review
summary statements.
Paper Copies of
Compiled Data for All
Determinant Codes.

Вам также может понравиться