0 оценок0% нашли этот документ полезным (0 голосов)
12 просмотров36 страниц
FoQus on Scales is designed as an enhanced method for scale development and adaptation within the PSI Behavior Change Framework. This chapter will discuss the tools and methods necessary to analyze the qualitative data from focus groups and free-listing. The reader will be able to: 1. Adapt the 'core' coding dictionary for a specific health issue and target audience; 2. Code qualitative data; 3. Synthesize coded qualitative data and generate summary statements.
FoQus on Scales is designed as an enhanced method for scale development and adaptation within the PSI Behavior Change Framework. This chapter will discuss the tools and methods necessary to analyze the qualitative data from focus groups and free-listing. The reader will be able to: 1. Adapt the 'core' coding dictionary for a specific health issue and target audience; 2. Code qualitative data; 3. Synthesize coded qualitative data and generate summary statements.
Авторское право:
Attribution Non-Commercial (BY-NC)
Доступные форматы
Скачайте в формате PDF, TXT или читайте онлайн в Scribd
FoQus on Scales is designed as an enhanced method for scale development and adaptation within the PSI Behavior Change Framework. This chapter will discuss the tools and methods necessary to analyze the qualitative data from focus groups and free-listing. The reader will be able to: 1. Adapt the 'core' coding dictionary for a specific health issue and target audience; 2. Code qualitative data; 3. Synthesize coded qualitative data and generate summary statements.
Авторское право:
Attribution Non-Commercial (BY-NC)
Доступные форматы
Скачайте в формате PDF, TXT или читайте онлайн в Scribd
PSIs Core Values Bottom Line Health Impact * Private Sector Speed and Efficiency * Decentralization, Innovation, and Entrepreneurship * Long-term Commitment to the People We Serve P S I
R E S E A R C H &
M E T R I C S
T O O L K I T
Research & Metrics Population Services International 1120 Nineteenth Street, NW, Suite 600 Washington, D.C. 20036
FoQus on Scal es: Data Anal ysi s and Appl i cati on
PSI Research & Metrics 2008
Population Services International, 2008
Contact Information Hibist Astatke 1 , Linda Kaljee 2 , and Kate O'Connell 3
1. Senior Researcher, PSI/Washington 2. Associate Professor, Wayne State University, Pediatric Prevention Research Center 3. Deputy Director, PSI Research & Metrics For more information, please contact: Hibist Astatke 1120 19 th St. NW. Washington, DC 20036 Telephone 202-572-4623 Email hastatke@psi.org
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
1
LEARNING OBJECTIVES
By the end of this chapter, the reader will be able to:
1. Adapt the 'core' coding dictionary for a specific health issue and target audience; 2. Code qualitative data; 3. Synthesize coded qualitative data and generate summary statements; 4. Generate and adapt scale items
BACKGROUND
OVERVIEW
FoQus on Scales is one component of Project FoQus (A Framework for Qualitative Research in Social Marketing), which incorporates qualitative methods to improve steps within the social marketing research process. More specifically, FoQus on Scales is designed as an enhanced method for scale development and adaptation within the PSI Behavior Change Framework. In this chapter, we will discuss the tools and methods necessary to analyze the FoQus on Scales qualitative data from focus groups and free-listing in order to adapt or develop determinants and their definitions, and adapt or develop multi-item scales to measure those determinants. The methods for collection of these focus group data are available in Toolkit Chapter 5.
DATA ANALYSIS AND MANAGEMENT
Developing and adapting a coding dictionary
The first step in the analysis of qualitative focus group data is to develop a coding dictionary (see Appendix A). The coding dictionary provides a defined and finite list of terms used to identify themes within portions of the transcribed texts. Steps toward the development of the coding dictionary include: 1. Development of the coding dictionary is a team effort in order to maximize on experience and knowledge of different members of the project; 2. A coding dictionary is developed only after a portion of focus group data have been transcribed and read by project team members. This allows for the development of content- specific codes; 3. The codes reflect both the respondents and the researchers perspectives; 4. Codes include both broader determinant-related (e.g. self-efficacy) and content-specific terms (e.g. casual partner); 5. Codes must be short and self-explanatory; 6. All codes must be clearly defined and examples provided as necessary; PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
2
7. Determinant-related codes must be used consistently with the core definitions provided through PSI. This allows for the possibility of cross-site data analysis; 8. The number of codes needs to be limited to between 15 to 25 terms. Too many codes will result in too few comparable portions of text. Too few codes will result in texts which are too broad to analyze; 9. The coding dictionary is a dynamic tool and codes can be added during the coding process as new topics or themes emerge during analysis;
Coding Focus Group Data
Once a coding dictionary has been developed, the focus group data is coded by project personnel. The purpose of the coding is to organize the textual data under specific topics and themes. The coding is a text management tool to support data analysis it alone does not constitute data analysis. Steps for coding include:
1. All coders must be familiar with the codes and their definitions. Coders will meet prior to coding to review the dictionary; 2. Meetings are important during coding to review the coders interpretations of texts, refine definitions, and/or add new codes; 3. New codes are added as themes or issues emerge from the text. New codes need to be defined, and previously coded texts reviewed to include the new code(s) as applicable; 4. Portions of texts can be coded with more than one code. The purpose of the coding is to provide a means of sorting texts for analysis. Therefore, accurate analysis depends on complete and detailed coding; 5. To facilitate development of scales, statements generated during free listing will include at least two codes: one content specific code (e.g. number of partners) and one determinant- related code (e.g. social norms); 6. Transcribed texts from each focus group is coded independently by at least two coders. The purpose of this double coding is to increase inter-coder reliability. In some instances, both coders will work on-site, and in other circumstances one coder (regional researchers) will be on-site. More than one set of two coders can be used to speed up the coding process.
a. After a selected number of texts (e.g. 3 or 4) are double coded, all the coders will meet or have a conference call (e.g., if a regional researcher is involved) to discuss their decisions regarding the coding; b. Any significant discrepancies are discussed, and a decision made regarding how to code that portion of text; c. Decisions made about coding during these meetings are incorporated into the definitions of the codes in the dictionary; d. If these decisions possibly effect other portions of the text, the transcriptions must be reviewed and changes made as appropriate e. Remaining transcribed texts can be either double-coded or single-coded depending on time and resources. If double-coded, meetings are conducted between coders at regular intervals.
Using Microsoft Word for Coding Text While several software programs are available for coding texts, because of the many PSI sites utilizing FoQus on Scales and the need for training on these software, an alternative method has been PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
3
adopted, which utilizes Microsoft Word. Microsoft Word is readily available across sites and does not require any special training. This method will permit the numbering of the texts to more easily identify the portion of text allocated to any one code, as well as simplifying the searching and synthesis tasks. [see Sample in Appendix B] The steps for this process are:
1. Number Lines of Transcribed Text. a. Go to File menu, and click Page set-up b. Click Layout tab c. Click Line Numbers d. Select Add line numbering e. Select Continuous option f. Hit ok on line numbering and layout screens g. Text will be numbered starting with 1 and continuing through the end of the document. 2. Create Coding Table. a. Go to a blank document b. Under Table go to Insert Table c. Create a one row and one column table d. Hit ok e. Under Table go to Table Properties f. On Table tab i. Size Preferred Width = 4 cm ii. Alignment = right iii. Text wrapping = around g. On Row tab i. Specify Height = 24 cm h. Hit ok single page i. Click Table tab and Select Table j. Use Control C to copy table. 3. Copy Coding Table into Text Document a. Place cursor at the beginning of the first line of text; b. Use Control V to paste table into first page of document; c. Continue to paste the table on each subsequent page. 4. Entering Codes a. Place cursor within coding table b. Reading through the text determine portion to be included for a specific code; c. Enter code into the table preceded by * [the * provides a means of identifying codes rather than words within the text for search purposes]; d. Enter lines of text matching the code; e. Continue coding.
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
4
Focus Group Data Analysis
The FoQus on Scales focus groups are specifically tailored to: 1) adapt existing determinants and/or identify new determinants; and, 2) modify existing scales and/or develop new scales. Therefore, analysis of these data will be directed toward defining or redefining determinants within a particular cultural context and modifying items in existing scales or generating items to develop new scales.
The following steps are used for this analysis process:
1. Systematic searches are conducted through all of the data for same-coded texts. If the Microsoft coding system is used, these searches can be conducted through the Microsoft program (Edit Tab, Find Option); 2. A separate compilation document is developed for each of these same-coded texts (See Figures 1 & 5). Using copy and paste, each identified portion of text is copied into this new document from the original data. The numbering of the text and reference to these numbered texts in the coding column makes this process more accurate; 3. Other information copied into this document include the identification number for the focus group discussion where the data originates , type of participants in the group (e.g. male youth ages 15 to 19), and content codes used to code the portion of text; 4. After all data have been searched, and relevant coded text separated into each of the compilation documents, the research staff reviews these documents for themes within the coded texts as well as across the coded texts. The research staff also reviews the data to make sure that all text fall under the determinant code and make changes as appropriate after a discussion among all the coders; 5. Research staff will develop a summary statement for each compilation document whenever new scales are being generated (See Figure 1). This statement provides a succinct overview of the primary themes and issues . 6. For purposes of item generation, within each of the compilation documents, free listed statements as well as supporting information from the general text are reviewed to determine possible items for inclusion in the scale. 7. The frequency and saliency of determinants are also assessed through the compilation documents to identify the top six to eight determinants for which scales will be developed. This is done by counting the number of focus groups in which certain determinants are mentioned as well as making note of the determinants that appear to be more relevant, important or significant than others (e.g., potential for high risk, immediate influence of the determinant on the behavior);. 8. Determinant definitions are updated or produced and scale items are generated or modified only for the top six to eight determinants in terms of frequency and saliency.
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
5
FIGURE ONE: COMPILATION DOCUMENT AND SUMMARY STATEMENT
Determinant Code=social norms Content code Focus group data Identifier PARTNER TYPE
But with what type of partner does one use a condom? Response: My husband, for example. Youre saying that you use a condom with your husband? Yes, because when he goes out, I dont know what hes doing. I dont trust him at all.
Now what are the factors that compel young people to use condoms? When they frequent prostitutes. When you know that the girl has several partners (li nan chen li nan chat). If one doesnt know the other person, one might not have unprotected sex with that person.
When one thinks that the girl might have several partners, one is forced to use condoms.
FGD1f2024
FGD5M2024
FGD6M2024
SITUATION
People dont use one when its the first time. That depends on the person.
Are there occasions when young people dont use condoms? When one is very familiar with their partner (everyone needs to have sex).
When do young people use condoms? When they go to cafs (or night clubs). They dont use them when theyre at home
If the person is afraid, if she doesnt want to be, or if he or she has only just met the person and wants to have sex with them right at that moment, the person uses a condom
If one doesnt trust their partner.
Tell me gentlemen-why is it that you dont use a condom when you make love? If one is faithful to their woman, you dont need to.
FGD4F2024
FGD6M2024
FGD8F2024
FGD7F2024
FGD2M1519
FGD2M1519
Summary Statement Youth use condoms with prostitutes, their spouse, with people that have several partners. Youth dont use condoms with partners that they are faithful to. Youth use condoms when they go to a caf/night club, when they are afraid, when just met someone and when they are afraid of infection.. Youth dont use condoms the first time they have sex and when they are familiar with their partner.
FGD1-8 (unique identifier for focus group discussion) M=male F=female 1519 (youth ages 15 to 19) 1924 (youth ages 20 to 24)
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
6
ITEM GENERATION
STEPS FOR GENERATING MULT-ITEM SCALES
Step One Adapt the existing determinant definition Step two Update the existing determinant definition Step Three Define any new determinants Step Four Establish the level of specificity Step Five Decide on appropriate response options Step Six Generate items that get at the same idea in different ways Step Seven Match items with definition (add/delete items as necessary) Step Eight Up-date determinant definition as needed
Once data have been synthesized this information is used to generate items for new scales or modify existing scale items. If the determinant is in the PSI Behavior Change Framework and/or defined in peer reviewed articles (See Chapter 1, Literature Reviews), the first step is to adapt the existing definition to health area, target group and behavior. In the second step, the summary statement, compiled data, and the synthesis are used to further specify and up-date the determinant definition. If the determinant is not included in the PSI Behavior Change Framework and peer review articles, a new definition is developed through the summary statement, compiled data, and the synthesis. . Determinant definitions will explain what is being measured by the determinant and describes different aspects or dimensions of the determinant.
In the first example below (Figure Two), the definition in the PSI Behavior Change Framework for social norms is first adapted for a specific health area, target group and behavior. It is then modified or updated using data from focus group discussions. Two context specific dimensions or aspects of social norms are identified including types of partners with whom condoms are used and situations when condoms are used.
In the second example, a new definition is generated for a new determinant, pregnancy intention, which emerged during the study. Thus, for step three in cases when a determinant is not included in either the PSI Behavior Change Framework or existing literature the first two steps are skipped and the summary statement and compiled data are used to generate the determinant definition.
Once determinants are defined and dimensions identified, in step four the level of specificity for the scales is determined (see Toolkit Chapter 2). With sparse data across different dimensions or aspects of a determinant, a one dimensional scale is developed that broadly measures all aspects of that determinant (see Example Two). When there is a significant amount of data about one aspect of a determinant one dimensional scale is developed which measures this aspect of the determinant in detail. If rich detailed information about different aspects of a determinant is available, a multi-dimensional scale with subscales is developed. In example one, data are available about condom use with different types of partners and in different situations. Therefore the decision was made to develop a scale that measures two different aspects of social norms for condom use.
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
7
FIGURE TWO: EXAMPLES FOR ADAPTION AND DEVELOPMENT OF DETERMINANT DEFINITIONS
In Steps five and six a pool of items is generated and response options determined. These steps typically take place at the same time to ensure that response options and statements match. A four point Likert scale is recommended for use in PSI TRaC studies. A Likert scale consists of statements each followed by response options indicating levels of agreement or disagreement. Detailed information on the number of response options that can be used including situations where a neutral point may be appropriate for a Likert scale can be found in Toolkit Chapter 3 (Response Options). Prior to generating items, criteria for good items and guidelines for generating items are reviewed (See Toolkit Chapter 4, Scale Development & Adaptation). Items are reviewed for clarity and to ensure that each item contains only one idea.
Statements to be included in multi-item scales are developed using determinant definitions that have been generated or modified and the compiled raw data, including the free-list statements within the compilation documents for each determinant. If the determinant definition includes many dimensions, focus on generating items for one dimension at a time.
Generally at least one item is developed for each concrete expression of the determinant found in the compiled data. These items should closely reflect the ideas expressed in the raw data as well as any minor variations (See Figures Three and Four) EXAMPLE ONE: Adapting a PSI definition (Steps One, Two, and Four)
PSI Behavior Change Framework Definition for Social Norms: Perceived standards of behavior that are accepted as the usual practice.
Adapted Definition for Social Norms: Perceived standard for condom use that are accepted as the usual among youth in Haiti.
Summary statement: Youth use condoms with prostitutes, their spouse, with people that have several partners. Youth dont use condoms with partners that they are faithful to. Youth use condoms when they go to a caf/night club, when they are afraid, when just met someone and when they are afraid of infection.. Youth dont use condoms the first time they have sex and when they are familiar with their partner.
Updated Definition for Condom Use Social Norms: Perceived standards for types of partners with whom youth use condoms and situations when condoms are used.
EXAMPLE TWO: Developing a definition for a new determinant (Step Three)
Summary statement: Youth want to have a child with a sexual partner for financial gain, to increase their social status and because of physical attraction to a partner.
Definition for Pregnancy Intention: An individuals desire to become pregnant/have a child. This desire can be motivated by personal, social or economic benefits.
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
8
FIGURE THREE: EXAMPLE ITEMS FOR PARTNER RELATED CONDOM USE SOCIAL NORMS
Text data types of partners and condom use social norms Items Generated
When they frequent prostitutes.
Statement 1 : Young people use condom when have sex with prostitutes. When you know that the girl has several partners (li nan chen li nan chat).
When one thinks that the girl might have several partners, one is forced to use condoms: Statement 2: Young people use condoms with a partner who has multiple partners.
But with what type of partner does one use a condom? Response: My husband, for example. Youre saying that you use a condom with your husband? Yes, because when he goes out, I dont know what hes doing. I dont trust him at all.
Statement 3 : People use condoms with their spouse.
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
9
FIGURE FOUR: EXAMPLE ITEMS FOR SITUATION RELATED CONDOM USE SOCIAL NORMS
Text data situations & condom use social norms Items Generated People dont use one when its the first time. That depends on the person.
Statement 1: People dont use condoms the first time they have sex.
If someone is faithful, they can make love without using a condom.
Tell me gentlemen-why is it that you dont use a condom when you make love? If one is faithful to their woman, you dont need to.
Statement 2: When partners are faithful to one another, they dont need to use condoms.
Are there occasions when young people dont use condoms? When one is very familiar with their partner (everyone needs to have sex).
Statement 3 : Youth dont use condoms with partners they know well.
When do young people use condoms? When they go to cafs (or night clubs). They dont use them when theyre at home.
Statement 4 : Condoms are used when having sex with someone met at a caf (night club)
If the person is afraid, if she doesnt want to be, or if he or she has only just met the person and wants to have sex with them right at that moment, the person uses a condom Statement 7 : Young people use condoms with someone they have just met.
Statement 8 : A condom is used when one of the partners doesnt trust the other.
Once items have been generated review items and reword any that dont accurately reflect the determinant definitions. Items are then reviewed and revised based criteria for writing items (see Toolkit Chapter 4: Scale Development and Adaptation).
In steps seven and eight review the items to ensure that they reflect the determinant that you are measuring. If not, decide if the item should be dropped, or if perhaps there is a need to modify the determinant definition.
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
10
ITEM MODIFICATION FOR EXISTING SCALES
STEPS FOR MODIFYING MULT-ITEM SCALES Step One Review and compare existing scale items and compiled data Step Two Modify phrases/words in existing items Step three Modify response options Step Four Identify and add new items Step Five Drop contextually or conceptually irrelevant items Step Six Determine level of specificity for scale Step Seven Update the existing determinant definition Step Eight Adapt the existing determinant definition
Existing scales are modified for use in PSI TRaC studies whenever they are available and in- country researchers and programmers elect to adapt them to local contexts. A PSI scales bank has been created to make the process of identifying scales easier for researchers and marketers. The scales bank contains PSI and Non-PSI scales for determinants found in the PSI Behavior Change Framework (see Appendix C).
Non-PSI scales were gathered through a comprehensive literature review. PSI scales were compiled from HIV/AIDS SMRS reports. Currently PSI scales included are from HIV/AIDS study reports in the 2005 and 2006 Social Marketing Research Series. PSI scales in other health areas will be available as reliable multi-item scales in SMRS reports in these areas are reviewed. The scales bank includes a fact sheet which contains information on the intended use of the scale, recommended modifications, and a reference for the source article or report.
Once an existing scale has been identified and any recommended changes have been made for those found the PSI scales bank, the first step in modifying existing scales is to review and compare the data and themes in the compilation document with the items in the scale.
Below is compiled focus group data on attitudes about condoms followed by items from an existing attitude scale from the Scales Bank. This scale has been modified based on these data.. To be consistent with the definition of attitudes from the PSI Behavior Change Framework, items that referred to buying condoms were dropped. Generally, items about condoms were reworded into phrases about using condoms..
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
11
FIGURE FIVE: COMPILATION DOCUMENT
Determinant Code=attitude Content code Focus group data Identifier TIME "We are living in a country where the tendency is to protect ourselves in everything we do, and it's the same thing when we make love. Using a condom takes too much time.
"We're on our guard; it's something we do secretively and in that case we don't have time to use a condom."
"Condoms waste too much time during sex."
When they start having sex, young people think they are going to use a condom, but then they feel like it takes too much time and they lose interest. FGD1f2024
FGD5M2024
FGD6M2024
FGD6M1519
COMFORT It's more comfortable without a condom. How do you explain that? It means that with a condom you feel uncomfortable, and that you prefer direct skin-to-skin contact."
When sex is protected, we don't feel comfortable. It shows a lack of sensuality, of feeling, of sexual enjoyment."
There are girls who feel uncomfortable when they use one. FGD4F1519
FGD4F2024
PLEASURE
"When sex is protected, we don't feel comfortable. It shows a lack of sensuality, of feeling, of sexual enjoyment."
"People refuse to use a condom to have much greater sensation (the condom decreases the sensation)."
"You say that you can have condoms handy and not use them. What can that mean?
It means that you're going to change the rhythm and that coitus won't be as effective."
Without condoms the feeling isn't the same. When you use condoms you don't feel good. Without a condom it's more pleasurable.
In my opinion, condoms aren't effective. They often say that the condom takes everything out of it."
FGD4F2024
FGD6F2024
FGD6F2024
FGD6M1519
FGD6M2024
FGD1-8 (unique identifier for focus group discussion) M=male F=female 1519 (youth ages 15 to 19) 1924 (youth ages 20 to 24) PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
12
Modified Condom Attitude Scale
Source: Otto-Salaj, L. L., Kelly, J. A., Stevenson, L. Y., Hoffmann, R. & Kalichman, S. C. (2001).
1. Using condoms can make sex more stimulating 2. Using condoms is uncomfortable. 3. Using condoms can be pleasurable. 4. Using condoms ruin the "mood." 5. Using condoms interrupt foreplay. 6. Using condoms is unreliable. 7. Using condoms is an effective method of preventing sexual diseases.
The second step is to modify phrases/words in existing items. In the above example condom attitude scale modifications are not made since the wording/phrases in the existing scale items are consistent with the data and do not require modification for use among the target group.
In step three, modifications are made to response options as needed. In the original scale a six point likert scale was used. A decision is made to retain this number of response options and conduct a pre-test to see if participants are able to distinguish between all categories and if the difference between the categories can be translated into the local language. (See Chapter 9 for pre-testing multi-items scales and response options).
In the fourth step, information in the compilation data that is missing from the existing scale is noted. In the above example most of the themes (e.g., time, pleasure, comfort) in the compilation data are already included in the scale. However, additional items are generated for time, pleasure and intimacy to more accurately reflect the data (see Figures 6 and 7). Also, in the existing scale Determinant Code=attitude Content code Focus group data Identifier INTIMACY
"Condom's prevent intimacy. So the feeling's in the condom."
"There is more intimacy when you don't use a condom" (already discussed) Some people tell me that condoms decrease sensation and that they prefer to have unprotected sex even if they are aware of the risks they are taking."
FGD6M2024
FGD6M2024
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
13
there were no items regarding sexual intimacy, and therefore new items are generated to reflect this aspect of the determinant (see Figure 8).
FIGURE SIX: EXAMPLE ITEMS FOR PLEASURE RELATED CONDOM USE ATTITIDES
Text data Pleasure and condom use attitudes Items Generated Wncn scx is prciccic!, uc !cni jcc| ccnjcria||c. |i sncus a |ack cj scnsua|iiq, cj jcc|ing, cj scxua| cnjcqncni.
Without condoms the feeling isn't the same. When you use condoms you don't feel good. Without a condom it's more pleasurable. 1a. Using condoms decrease sexual enjoyment. 1b. Using a condom decreases sexual pleasure
Pccp|c rcjusc ic usc a ccn!cn ic natc nucn grcaicr scnsaiicn (inc ccn!cn !ccrcascs inc scnsaiicn).
3a. Using condoms decreases sexual feelings
|n nq cpinicn, ccn!cns arcni cjjcciitc. Tncq cjicn saq inai inc ccn!cn iakcs ctcrqining cui cj ii.
3b. Using condoms diminished sexual satisfaction
Ycu saq inai qcu can natc ccn!cns nan!q an! nci usc incn. Wnai can inai ncan?
|i ncans inai qcurc gcing ic cnangc inc rnqinn an! inai cciius ucni |c as cjjcciitc.
4a. Using a condom changes the rhythm of sex.
FIGURE SEVEN: EXAMPLE ITEMS FOR TIME RELATED CONDOM USE ATTITUDES Text data Time and condom use attitudes Items Generated . Using a ccn!cn iakcs icc nucn iinc.
Ouring scx, cnc cxpcricnccs cncrncus p|casurc, scnsaiicns, an! cjicn uc !cni natc iinc ic usc a ccn!cn.
8. Using a condom takes too much time Wcrc cn cur guar!, iis scncining uc !c sccrciitc|q an! in inai casc uc !cni natc iinc ic usc a ccn!cn. 9. Using a condom takes too much time when you are having sex secretively. .
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
14
Ccn!cns uasic icc nucn iinc !uring scx. 10. Using condoms waste too much time Wncn incq siari nating scx, qcung pccp|c inink incq arc gcing ic usc a ccn!cn, |ui incn incq jcc| |ikc ii iakcs icc nucn iinc an! incq |csc inicrcsi.
FIGURE EIGHT: EXAMPLE ITEMS FOR INTIMACY RELATED ATTITUDES
Text data Intimacy and condom use attitudes Items Generated Ccn!cns prctcni iniinacq. Sc inc jcc|ings in inc ccn!cn.
11. Using condoms interferes with sexual intimacy. Tncrc is ncrc iniinacq uncn qcu !cni usc a ccn!cn (a|rca!q !iscussc!) Scnc pccp|c ic|| nc inai ccn!cns !ccrcasc scnsaiicn an! inai incq prcjcr ic natc unprciccic! scx ctcn ij incq arc auarc cj inc risks incq arc iaking.
12. Using condoms decreases sexual intimacy
In the fifth step, contextually irrelevant aspects of the scale are identified. In the above example, ideas related to reliability and foreplay are not mentioned in the qualitative data. However, the decision is made that since there is no data indicating these aspects are not important, those items will be retained prior to pre-testing and piloting the scale (see Chapters 9 and 10).
A scale with a total of 17 items was developed for the pilot study. Some if these items address new themes discovered in the data and others build on existing themes. Items are number so that new items that address existing themes have the same number used in the original scale item. New items within these themes are identified by letters (e.g. 1a, 1b).
1. Using condoms can make sex more stimulating 1a. Using condoms decrease sexual enjoyment. 1b. Using a condom decreases sexual pleasure 2. Using condoms is uncomfortable 3. Using condoms can be pleasurable. 3a. Using condoms decreases sexual feelings 3b. Using condoms diminished sexual satisfaction . 4. Using condoms ruin the "mood." 4a. Using a condom changes the rhythm of sex. 5. Using condoms interrupt foreplay. 6. Using condoms is unreliable. PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
15
7. Using condoms is an effective method of preventing sexual diseases 8. Using a condom takes too much time 9. Using a condom takes too much time when you are having sex secretively. . . 10. Using condoms waste too much time. 11. Using condoms interferes with sexual intimacy. 12. Using condoms decreases sexual intimacy
In step six, the level of specificity for the scale is determined. In this case a decision is made to retain a one dimensional scale. In this example, it is difficult to conclude if any of the themes are more important than other themes and there is insufficient data to create subscales for all possible dimensions (e.g. comfort, intimacy).
After the scale has been adapted, in step seven the PSI determinant definition for attitude is updated based on the new items that have been included and/or excluded and the modifications of existing items. In step eight, the up-dated definition is adapted for the specific health area, target group, and/or behavior.
Example PSI definition
PSI Behavior Change Framework Definition for attitude: An attitude is an individuals evaluation or assessment about the promoted behavior
Updated Definition for Attitude about Using Condoms: An attitude is an individuals evaluation or assessment related to time, pleasure, intimacy and comfort associated with using condoms.
Adapted Definition for Social Norms: An attitude is an evaluation or assessment made by youth living in Haiti related to time, pleasure, intimacy and comfort associated with using condoms.
DATA ANALYSIS WORKSHOP (OPTION)
This chapter has presented the tools and methods for analyzing qualitative data for FoQus on Scales. The previous sections have included tasks which are done by a team of researchers over a period of time. Another option for the data analysis phase of FoQus on Scales is the use of a three-day workshop during which data coding and compilation are completed, as well as identification and defining of determinants and associated scale items. This data analysis workshop can be conducted separately, or it can be combined with the researchers and local researchers and programmers interpretation workshop (see Chapter 7). In the case of the combined analysis and interpretation workshops, the overall length will be approximately five days with the introductory materials presented in Day One, the analysis workshop in Days Two and Three, and the interpretation workshop in Days Four and Five. An outline of the curriculum for the analysis workshop is presented in Figure Nine, and a full curriculum is included in Appendix D.
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
16
FIGURE NINE: OUTLINE OF ANALYSIS WORKSHOP CURRICULUM DAY ONE Session 1: Background and introduction Activity 1: Introductions, Overview of the Training, and Training Objectives [Large Group 30 MINUTES]: Objectives: 1) share some personal information about themselves with the group; 2understand the purpose of the training and 3) know the various activities to be completed during the training;
Activity 2: Research Background Review [Large Group-30 MINUTES] Objective: know about the rational for the study and know and understand the research objectives.
Activity 3: Overview of Coding [Large Group-30 MINUTES] Objectives: 1) be familiar with the FoQus on Scales Behavioral Change Framework; 2) understand the role of coding within qualitative data analysis (as conducted in FoQus on Scales); 3) be able to distinguish and understand the two types of codes determinants and content codes; and 4) understand terms used as a part of the coding process.
Session 2: Developing a coding dictionary Activity 1: Reviewing Transcripts and Identifying Themes [Small Groups and Large Group-90 MINUTES] Objectives: 1) start to be familiar with the content of the transcribed data; 2) start to identify themes within these data;3) share identified themes and start the process of determining themes for which codes need to be developed.
Session 3: Developing a coding dictionary Activity 1: Coding Transcript and Refining Code Definitions [Individual and Large Group-90 MINUTES] Objective: 1) Be familiar with the process of coding so that can complete the coding of the transcripts and 2) developed a more refined coding dictionary to facilitate coding.
Session 4: Establishing inter-rater reliability Activity 3:Double coding- Reliability Coding Session I [Small Group-90 MINUTES] Objective: know that the coding dictionary is being used consistently by all workshop participants (inter-coder reliability)
DAY TWO Session 1: Second coding meeting, updates and revisions Activity 1: Group discussion and agreement- Reliability Coding Session I [Small and Large Group-90 minutes] Objective: 1) Review, discuss and reach consensus regarding decisions made during small group work; 2) Update the coding dictionary and 3) Finalize coding of transcripts from day one.
Session 2: Begin coding Activity 1: Code Transcripts Session I [Individual-90 MINUTES] Objective: Complete coding one more transcript each.
Session 3: The third coding meeting, updates and revisions Activity 1: Review of Coding [Large Group-30 MINUTES] Objective: Review any new codes or changes to definitions.
Session 4: Continue coding Activity 10: Code Transcripts Session II [Individual-150 MINUTES] Objective: Continue to code transcripts.
DAY THREE Session 1; Finalize coding Activity 1: Code Transcripts Session III [Individual-90 MINUTES] Objective: Complete coding transcripts and entering codes into the electronic version of the transcripts.
Session 2: Data compilation Activity 1:Compiling Coded Texts [Individual 90 MINUTES] Objective: Conduct a search and compile texts for individual codes. All codes will have been searched and compiled.
Session 3: Review of steps for writing summary statement Activity 1: Write summary statement for one determinant [Large Group Activity 90 MINUTES] Objective: Review purpose and procedures of writing summary statements for compiled texts [determinant codes only].
Session 4: Writing summary statement Activity 14: Write summary Statements for all Determinants [Individual-90 MINUTES] Objective: To complete summary statements for all determinant codes
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
17
QUALITY IMPROVEMENT CHECKLIST
Data analysis Create succinct and well-defined codes prior to beginning coding procedures. The level of analysis is dependent on good coding. The coding provides the means for searching and summarizing data during the analysis phase; Have at least two people code the first three to four transcripts. After double coding hold meetings to discuss any discrepancies in coding, make decisions about these discrepancies, and incorporate changes into the coding dictionary revisit codes on a regular basis throughout the coding process to ensure that changes in definitions or new examples are incorporated into the coding dictionary Recode previously coded transcripts as appropriate when changes are made to the coding dictionary
Item generation
Update existing determinant definitions using summary statements DEFINE NEW determinants using summary statements Generate items using updated or new determinant definitions AND compiled data FOR each determinant
Item adaptation Modify existing scales using compiled data for each determinant Update determinant definitions for modified existing scales to reflect the changes made to these scales PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
18
CASE EXAMPLES AND LESSONS LEARNED
CASE EXAMPLE: 2006 YOUTH HAITI FOQUS ON SCALES STUDY
STUDY PURPOSE
Scale development for TRaC studies involves modifying existing scales and developing new scales. In the summer of 2006, PSI Haiti conducted a qualitative study to develop multi-item scales. The purpose of this study was to identify determinants of condom use relevant to Haitian youth ages 15 to 24 and develop multi-item scales to measure these determinants.
DATA COLLECTION
A semi-structured focus group guide was developed with in-put from the country PI. This guide used free-lists to generate statements on determinants of behavior and used pile sorts to have participants group and define relevant determinants. Prior to data collection, the interview guide was piloted on two groups and further revised. . Separate focus groups were conducted for younger and older and male and female youth with a total eight focus group discussions. Data from the free-listing exercise were used to identify relevant determinants of behavior and generate scale items. Data from the pile sorts were found be of poor quality and not used.
DATA ANALYSIS
A coding dictionary was developed based on the definitions for the determinants in the PSI Behavior Change Framework. New codes were added for determinants such as pregnancy intentions that were not in the PSI Framework. The data from the free listing were coded by the research staff in Haiti and the research backstop separately. Research managers and the research backstop met for one day to review the codes and reconcile any differences. Next, these coded data were examined for frequency of determinants mentioned across focus group discussions. Eight of the most frequently mentioned determinants were identified for inclusion in the questionnaire for the TRaC study. A data management method similar to the one described earlier in this chapter was used to compile data on each determinant identified.
DATA INTERPRETATION AND ITEM GENERATION
Five PSI program and research staff and three peer educators from a partner organization participated in a two-day data interpretation and item generation workshop. This workshop was co-facilitated by the research backstop and the director of the research department for PSI Haiti.
The first morning of the workshop was devoted to providing participants with background information and knowledge they needed to generate multi-item scales. The workshop began with a presentation on TRaC studies and multi-item scales. Participants were then introduced to criteria for writing statements to be used in multi-item scales and definitions for the relevant identified determinants. Individual and small group exercises were used to increase participants skills and understanding in these areas. By the end of that first morning, participants were exposed to the entire process of scale development.
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
19
During the afternoon, participants worked in small groups consisting of at least one researcher, one programmer and one peer educator to generate items. Participants interpreted data and generated items in these groups. Large group discussions were used to share items and challenges.
The entire second day was spent on data interpretation and item generation. At the end of the workshop all items generated were compiled by the research staff and reviewed. Duplicate items were deleted.
SUCCESS, CHALLENGES AND LESSONS LEARNED
Successes of the Haitian study included:
Reliability of multi-item scales developed using the focus group data ranged from .78 to .96. This is within the good to excellent range for scale reliability; A number of significant determinants of condom use were identified through the segmentation analysis suggesting that the scales used were of good quality Scales used to measure these determinants were rich in contextual detail. Therefore, these data provide information that can be used to design interventions that are uniquely tailored to Haitian youth.
Challenges of the Haitian study were primarily in relation to data collection. These challenges included:
The interview guide required highly skilled moderators who were familiar with the PSI Behavior Change Framework. o Based on this experience the interview guide was simplified. The data collected by the agency selected to conduct the focus group discussions reflect poor moderating skills and lack of understanding of the purpose of the study. o Different implementation options are provided so that the first implementation option requires less resources and capacity, e.g., through free listing (see FoQus on Scales concept paper). o Pile sorting that requires higher moderating and analysis skills should be used in subsequent implementation options (see FoQus on Scales concept paper).
Several important lessons were gained from PSI Haitis experience analyzing and interpreting qualitative data and generating items. Data should be coded at several levels to allow for identification of dimensions within scales (e.g. self-efficacy for using condoms, self-efficacy for negotiating condom use). When data with similar codes are compiled this information should be used to write a summary that can be used to define new determinants or modify definitions for determinants in the PSI Behavior Change Framework. A decision about the level of specificity for a scale (a very specific one dimensional, a broad one dimensional or multi-dimensional) should be decided prior to generating items. The content validity of scales should be checked and improved by comparing the items generated against the determinant definition and either revising the definition or moving items to other scales. Following this process allows for determinant definitions and items that are consistent with one another and based on findings from the qualitative study.
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
20
REFERENCES Otto-Salaj, L. L., Kelly, J. A., Stevenson, L. Y., Hoffmann, R. & Kalichman, S. C. (2001). Outcomes of a Randomized Small-Group HIV Prevention Intervention Trial for People with Serious Mental Illness. Community Mental Health Journal 37(2): 123-144. PSI RESEARCH & METRICS TOOLKIT
Threat (adapted PSI) An individuals perception of the magnitude of harm posed by HIV/STIs. Threat is first comprised of severity, which is an individuals perceived magnitude of the harm of HIV/AIDS (i.e. significance or seriousness of HIV/AIDS AND susceptibility is an individuals perceived likelihood that HIV/AIDS negatively harms him/her (i.e. degree of vulnerability, personal relevance, risk of experiencing the public health problem).
Social norms (adapted PSI) Behavioral standards (cultural and religious etc) which exist in the community for an individual to follow.
Locus of control (adapted PSI) Is the external or internal site of control of involvement in concurrent relationships. An external locus of control suggests that an individuals behavior related to concurrent partnerships is under the control of powerful other or is determined by fate, luck or chance. An internal locus of control suggests that an individuals behavior in this area is directly controlled by him/herself.
Beliefs (adapted PSI) Is a perception about an object/behavior, which may or may not be true e.g. spirits can make you have concurrent partners
Outcome expectations Positive or negative expectations or consequences of being in monogamous relationship.
Pressure Strong negative or positive influence from significant people to engage in concurrent relationships
Transaction Exchange of sex for material, financial gains and other favours. Sexuality Perceived inherent drives to have more sex, need for different types of partners. Quality of Relationships Satisfaction or dissatisfaction in a recurrent sexual relationship. Media exposure The influence of media on sexual practices Revenge Involvement in another relationship with intention to hurt another person. Recreation Sex as a form of entertainment. Outcome expectations -Ve or +Ve consequences of engaging in current relationships. Free time Time that is not allotted for a specific activity Perceived benefits Positive outcomes specifically material or financial gain that someone gets as a result of being involved in concurrent relationships.
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
22
Definitions-Secondary codes (content codes to capture contextual information) Culture Any references to traditions and belief systems. A set of learned beliefs, values and behaviours the way of life shared by the members of a society Religion Beliefs and practices regarding supernatural beings, power, and forces. These beliefs and practices can be general or specific to concurrent partnerships. Expected roles/responsibilities duties expected of a partner including the manner in which one partner relates to another. Age-differences Difference in age between sexual partners Emotions Feelings about a situation, person or object Finances/money The amount of money someone has or needs. The ways in which individuals raise, allocate, and use monetary resources. Status/prestige The relative standing or position of a person in society or a community. A high standing achieved through success, wealth or influence. Sexual desire/drive Inherent sexual drive or desire Alcohol use Use of alcohol generally or in relations to involvement to concurrent partnerships. Sexual desire enhancing drugs Drugs taken to increase sexual desire or stamina Partner characteristics Intrinsic factors such as being argumentative or being a good listener that are unique to a partner . Sexual fulfilment Sexual satisfaction or gratification Partner availability Physical or emotional availability of a partner Partner behaviour Any behaviours exhibited by a partner Main partner The primary, married or official partner (dropped-not enough codes) Partner communication The way in which a partner talks or listens to another partner Seductive behaviour The way that a person moves, talks or dresses in an attempt to sexually attract another person Abuse Verbally/psychologically or physical hurting someone including physical violence Situation/social setting Places where or conditions under which a particular act takes place Roles/responsibilities Expected behaviours from a spouse or other sexual partners PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
23
Material goods Anything of value such as food, housing and products (e.g. cell phones) that people barter, obtain/purchase or sell. Place/residence -a specific location or area where a person lives or spends time Peers A group of people who are like one another or have something in common (e.g youth who live in the same community and are the same age). The group can be composed of people who are friends but this is not always the case. People A group of persons/individuals, Parent(s) Can be the biological father and/or mother of a child or one or more people who have nurtured and raised a child without any biological relationship. Includes biological or step father and/or mother and common-law mothers and/or fathers or adoptive fathers and/or mothers. In-laws A spouses parents. Love A strong positive emotion of regard and affection for a partner. It can include having strong positive emotions for a partner despite any faults they may have. Dislike A feeling of aversion or antipathy about a partner.
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
24
Appendix B: Sample coding
Focus group discussion 1-male youth ages 15 to 19 (FGD1M1519) I: man and a woman? 1 R: A good relationship between a man and a woman is that they have the same point of 2 view and the same idea. When discussing something, they have the same ideas and the 3 same point of view. They understand each other. In class, if the girl does not understand 4 lessons the boy explains to her the lessons. In general, they have to take care of each 5 other in studying. 6 I: Anything else? 7 R: In general, they talk many things to one another to understand each other more, and 8 closer to one another. 9 I: Can you have a good relationship that is also sexual? 10 R: No. There were no cases in my class. Girls and boys just sit and talk to one another, 11 help and take care of one another in studying or go out with other friends. There were no 12 cases like that. 13 I: Why is good relationship not a sexual relationship? 14 R: In general, we are grown up and we know it is not right and it is wrong. And the 15 school also provides necessary information to young people and students in school know 16 the information and avoid it. In general, students now know the information. They are 17 not like students in the past who did not understand it and unexpected things happened. 18 I: Under what circumstance might a young man and a young woman have sexual 19 relationship? 20 R: In general, there are sex workers standing on streets, young men often find a pleasure 21 and it happens. Or people who lack knowledge go with their boyfriends, they can not 22 control themselves and they are attracted to engage in activity that is unfortunate. It is 23 because their point of view is not strong enough. It will not happen if they are strong. 24 I: Would that couple use contraceptives? 25 R: Im not very clear. Contraceptives are usually condoms. Im not very clear. In general, 26 it is up to us, ourselves. Many people can not do it. 27 I: How would young people here get contraceptives? 28 R: About contraceptives, there are many. For example, when I was in school, the school 29 provided me about information that young people needed to know, and protect 30 themselves. In general, as a girl, she has to protect herself without being reminded by 31 anyone. She has to protect herself when she has a boyfriend. Usually, condoms are used 32 more. 33 I: Are there other kinds of contraceptives besides condoms? 34 R: Contraceptive pills. They are used for those who have given birth and they do not 35 want to have more children. They will use contraceptive pills. But as I know, young 36 women often use condoms more. 37 *relation 1-20 *student 1-20
Existing scales are modified for use in PSI TRaC studies whenever they are available and in-country researchers and programmers elect to adapt them to local contexts. A PSI scales bank has been created to make this process easier for researchers and marketers. The scales bank contains PSI and Non-PSI scales for determinants found in the PSI Behavior Change Framework (see links below).
Non-PSI scales were gathered through a comprehensive literature review. PSI scales were compiled from HIV/AIDS SMRS reports. Currently PSI scales included are from HIV/AIDS study reports in the 2005 and 2006 Social Marketing Research Series. PSI scales in other health areas will be available as reliable multi-item scales in SMRS reports in these areas are reviewed. The scales bank includes a fact sheet for each determinant which contains information on the intended use of the scale, recommended modifications, and a reference for the source article or report.
Appendix D : Curriculum for Focus Group Data Analysis Workshop
DAY ONE
Session one: Background and introduction
Activity 1: Introductions, Overview of the Training, and Training Objectives Facilitator(s) Objectives: After these activities, participants will 1. have shared some personal information about themselves with the group; 2. understand the purpose of the training and know the various activities to be completed during the training; 3. know and understand the training objectives. Total Activity Time: 30 minutes
Time Step Method Materials 10 min Introduction
Large Group activity: Facilitator welcomes everyone and discusses the purpose of the workshop and topics that will be covered. Participants light a match and say as much as they can about themselves before the match burns out. 10 min Overview of Training Presentation: Facilitators will present the schedule for the training activities 10 min Review of Training Objectives Presentation: Facilitators will present the overall training objectives. 1. Learn how to develop a coding dictionary; 2. Develop a coding dictionary for the current project; 3. Learn how to code transcripts; 4. Complete coding of transcripts from the current project; 5. Learn how to compile coded text for determinant codes; 6. Complete compiling texts for determinant codes for current project; 7. Learn how to write a summary statement; 8. Complete summary statements for all determinant codes for the current project. PC and Projector
Power point presentation
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
27
Activity 2: Research Background Review Facilitator(s): Objective: After this activity participants will know about the rational for the study and will know and understand the research objectives. Total Activity Time: 30 minutes
Time Step Method Materials 20 to 30 min Presentation of Study Design & Rational Presentation: Explain rational for the existing study (based on current literature and/or contract with donor). Review the research objectives. PC and Projector
Power point presentation 10 min Discussions & Questions Large Group activity: Opportunity for workshop participants to ask questions regarding the research rational and objectives
Activity 3: Overview of Coding Facilitator(s): Objective: After these activities, participants will 1. have reviewed the PSI Behavioral Change Framework; 2. understand the role of coding within qualitative data analysis (as conducted in FoQus on Scales); 3. be able to distinguish and understand the two types of codes determinants and content codes 4. be familiar with terms used as a part of the coding process. Total Activity Time: 30 minutes
Time Step Method Materials 20 min Presentation Presentation:Overview of FoQus on Scales, review of the PSI Behavior Change Framework including definition for all determinants and explanation of the role of Coding within FoS.
PC and Projector
Power point presentation
10 min Discussion Large group work: Opportunity for workshop participants to ask questions regarding the PSI Behavioral Change Framework, FoS, and coding
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
28
Session 2: Developing a coding dictionary
Activity 1: Reviewing Transcripts and Identifying Themes Facilitators(s): Objective: After these activities, participants will 1. have started to familiarize themselves with the content of the transcribed data; 2. have started to identify themes within these data; 3. have shared identified themes and started selecting themes for which codes need to be developed. Total Activity time: 90 minutes
Time Step Method Materials 60 min Review Transcripts Small group work: Workshop participants are divided into groups of two. Each group is provided with a copy of a transcript. Participants will read through the transcripts and take notes on themes within the transcripts Copies of Transcripts 30 min Free Listing and Code Term Identification Large group work: Workshop participants will come together as a large group. Each group will present their themes [note: If participants in one group have more experience, they should be the first to present]. The themes are recorded on a board or on a projector. After the first presentation, subsequent presenters note those themes which they also have identified by marking next to that theme. They should then add to the list any other themes they identified. Workshop participants and facilitators then give a code word for each theme starting with those themes identified most frequently. Workshop participants and facilitators discuss the need to merge themes under a single code and/or divide a single theme into multiple codes. Once all themes given codes, the facilitators will divide the codes into two categories: determinants and content codes PC and Projector
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
29
Session 3: Coding practice and the first coding meeting
Activity 1: Coding Transcript and Refining Code Definitions [Individual and Large Group] Facilitator(s): Objective: After this activity, participants will be more familiar with the process of coding needed to complete coding all the project transcripts. Participants will also know how to refine a coding dictionary to facilitate coding. Total Activity Time: 90 minutes
Time Step Method Materials 45 min Code Transcript Individual work: Each participant is provided with the same portion of a transcript. He/she will individually code the transcript. Copies of Transcripts 45 min Review Coding and Refine Definitions Large group work: The participants come back together as a large group. The facilitator goes through the code words and asks for examples of when that code word was used (participants should read the text coded and if necessary explain their decision to use that code). During this process, code definitions are refined to more accurately reflect how they are interpreted and used in the coding process. If necessary, codes can be modified, e.g., combined or separated, and/or new codes can be identified and defined. PC and Projector with Coding Dictionary
Session 4: Establishing inter-rate reliability
Activity 1:Double coding Facilitator(s): Objective: After this activity, participants and facilitators will know that the coding dictionary is being used consistently by all workshop participants (inter-coder reliability) Total Activity Time: 90 minutes
Time Step Method Materials 90 min Double Coding Transcripts [Note: Facilitators should aim to have Small Group work: Workshop participants divide into groups of two. Each of those two participants will code the same transcript independently. Copies of Transcripts PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
30
25% of transcripts double coded] After coding, the two participants will discuss their coding decisions. Participants will take notes of discrepancies in the way in which they each coded. Participants will reach consensus on how to best code the transcript to reflect the code definitions.
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
31
DAY TWO
Session 1: Second coding meeting, updates and revisions
Activity 1: Group discussion and agreement
Objective: After this activity, participants will review, discuss and reach consensus regarding changes made during small group work on inter-rater reliability, update the coding dictionary and finalize coding of transcripts. Total Activity Time: 90 minutes
Time Step Method Materials 30 min Double Coding Transcripts
Small group: Workshop participants will continue the coding process started in Day 1 session 4 activity 1.
Copies of Transcripts 30 min Review of Double Coding Large group: Workshop participants will come together as a large group. Participants will discuss any discrepancies experienced in the reliability coding activity. Participants will review and agree upon the changes in the definitions (as determined in the small groups). PC and Projector with Coding Dictionary 30 min Revise & Complete Coding of Transcripts Small groups: Workshop participants will reconvene into the same small groups (2 participants each). They will finalize the coding of their transcript based on the decisions made in the large group.
Session 2: Begin coding
Activity 2: Code Transcripts Facilitator(s): Objective: After this activity, each participant will have completed coding one more transcript each. Total Activity Time: 90 minutes
Time Step Method Materials 90 min Coding Transcripts [Note: facilitators should aim to have 50% of transcripts coded by the end of this session] Individual work: Each participant will individually code a single transcript. Codes will be entered into the electronic version of the transcript. Facilitators will move around the room to assist with questions and review coding procedures with workshop participants Paper & Electronic Copies of Transcripts PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
32
Session 3 The third coding meeting , updates and revisions
Activity 1: Review of Coding Facilitator(s): Objective: To review any new codes or changes to definitions. Total Activity Time: 30 minutes
Time Step Method Materials 45 min Review of Coding
Individual work: Participants will reconvene in a large group to discuss any changes in code definitions or use of additional codes. Changes will be made to the master coding dictionary.
PC and Projector with Coding Dictionary 45 min Revise coding [Note: if changes are made to the dictionary, participants go back and review their previously coded transcripts including those used for reliability testing, and make appropriate changes in the coding.] Paper & Electronic Copies of Transcripts
Session 4: Continue coding
Activity 2: Code Transcripts Facilitator(s): Objective: Participants will continue to code transcripts. Total Activity Time: 90 minutes
Time Step Method Materials 90 min Coding Transcripts [Note: facilitators should aim to have 75% of transcripts coded by the end of this session] Individual work: Each participant will individually code transcripts. Codes will be entered into the electronic version of the transcript. Facilitators will move around the room to assist with questions and review coding procedures with workshop participants
Paper & Electronic Copies of Transcripts
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
33
DAY THREE
Session 1: Finalize coding
Activity 1: Code Transcripts Facilitator(s): Objective: After this activity, participants will have completed coding transcripts and entering codes into the electronic version of the transcripts. Total Activity Time: 90 minutes
Time Step Method Materials 90 min Coding Transcripts [Note: facilitators should aim to have 100% of transcripts coded by the end of this session] Individual work: Each participant will code a transcript. Codes will be entered into the electronic version of the transcript. Facilitators will move around the room to assist with questions and review coding procedures with workshop participants Paper & Electronic Copies of Transcripts
Session 2: Data compilation
Activity 1: Compiling Coded Texts Facilitator(s): Objective: Participants will learn how to conduct a search and compile texts for individual codes. All codes will have been searched and compiled. Total Activity Time: 90 minutes
Time Step Method Materials 90 min Compile Text for Each Code Individual work: Participants are each assigned codes. For each code, the individual will go through and identify the text associated with that code. The identified text will then be cut and pasted (electronically) from the coded transcript into the compilation forms. Codes are compiled as outlined in the FoQus on Scales Toolkit. Electronic Copies of Coded Transcripts PCs for each participant
PSI RESEARCH & METRICS TOOLKIT
FoQus on Scales: Data Analysis and Application
34
Session 3: Review of steps for writing summary statements
Activity 1: Write summary Statement for one determinant Facilitator(s): Objective: To review purpose and procedures of writing summary statements for compiled texts [determinant codes only]. Total Activity Time: 90 minutes
Time Step Method Materials 15 min Review procedures for writing summary statements Presentation: Facilitators will present a short review of the purpose and procedures for writing summary statements for the determinant codes. This presentation will build on the review of procedures from session 1 activities 1 and 3. PC and Projector Power point slides 45 min Identification of Themes in Compiled Data Small group work: Participants will break into small groups. Each group will review a copy of data compiled for a single determinants (all groups will review the same data). Group members will list themes within the compiled data Paper Copies of a Single Determinant Compiled Data Form 30 min Review Themes and Write Summary Statement Large group work: The participants will reconvene into a large group. Small group members will provide theme identified from the compiled data. Facilitators will incorporate these themes into a summary statement PC and Projector
Session 4: Writing summary statement
Activity 1: Write summary statements for all determinants Facilitator(s): Objective: To complete summary statements for all determinant codes Total Activity Time: 90 minutes
Time Step Method Materials 90 min Write Summary Statements Individual: Each participant will be assigned compiled text data for determinants (each participant works on separate determinants). Participants will write summary statements for the determinants. Facilitators will move around the room to assist with questions and review summary statements. Paper Copies of Compiled Data for All Determinant Codes.