JOURNAL CLUB

Journal club 2: qualitative research

Jennifer Reids series aims to help you access the speech and language therapy literature, assess its credibility and decide how to act on your findings. Each instalment takes the mystery out of critically appraising a different type of journal article. Here, she looks at qualitative research.

oons.co.uk www.francart

rather like numbers. It seems obvious to me to count things to see if there is a pattern, evidence of progress or a useful comparison to be made. At this point, you will not be alone if you are thinking anorak! Many of you may feel that as soon as you convert something to numbers it loses any kind of real meaning. However, in research design, its all about horses for courses; using numbers (quantitative methods) works for studying some kinds of phenomena and sticking with words (qualitative methods) works for others. Here is a story to illustrate the different yet potentially complementary approaches of quantitative and qualitative methods. Amy leads a small team of speech and language therapists working in the community in quite a sparsely populated area, so she and her colleagues spend a lot of their time travelling, and timetabling can be quite a juggling act. She notices that referrals are rising quite rapidly in one part of their patch while elsewhere they remain pretty constant. As she may need to change how the service is deployed to make best use of the teams resources, she discusses this with her manager, and they agree they need to know a bit more about what might be going on. What background research would help them? Some sources of quantitative data that might be available are:

1. The local authority publishes regular bulletins on population trends within age bands in NHS areas, parliamentary constituencies, council wards, secondary school catchments areas and so on. 2. The speech and language therapy service has an electronic database of referrals and their sources going back over 12 years. 3. Most of the teams referrals come from practitioners in either primary/community care (40 per cent) or education (45 per cent). 4. Amys not sure but feels there may have been more inappropriate referrals over the past year, and she is certain they are seeing a lot more 2-3 year-olds than they used to. 5. Four of the biggest schools they cover have new headteachers and there have been quite a few meetings with them to discuss roles and responsibilities and to go through the procedures for referral. 6. Several health visitors in the area retired recently and Amy feels that the team has not yet had the opportunity to develop strong working relationships with their successors. Two community paediatrician posts have also been vacant for more than a year now, and the health board has been unable either to fill the posts or to secure locum cover.

The numerical sources 1 and 2 should give Amy and her manager an indication of whether the trends in referrals reflect (a) changes in the population demographics of the area, or (b) changes in the referral rate (the proportion of the population being referred). Items 3 and 4 provide figures for investigating whether increasing referral numbers appear to be associated with particular sources of referral rather than across the board. Items 5 and 6 probably reflect some of Amys hypotheses about potential sources of temporary fluctuation in referral rates. They could use their database to check numbers of referrals over time from these schools and healthcare practitioners to see if the pattern of fluctuations over time in referral numbers is associated with personnel change. However, knowing who is referring more, or fewer, clients than is typical for their practitioner group wont necessarily help Amy and her manager understand why this is happening. They need to elicit information on the perspectives of these referrers. Here is where the qualitative techniques of document study, observation and interview come into their own. In this case, exploring the content of referral forms (document study) and / or exploring referrers decisionmaking (using individual or group interview

18

SPEECH & LANGUAGE THERAPY IN PRACTICE WINTER 2010

JOURNAL CLUB

Criticial appraisal for speech and language therapists (CASLT) Download the qualitative research framework document from www.speechmag.com/Members/CASLT. Use it yourself or with colleagues in a journal club, and let us know how you get on (email avrilnicoll@speechmag.com).
techniques) should help Amy and her manager understand, and subsequently manage, the reasons underlying the changes in their referral numbers. Qualitative methods are the gold standard (the method of choice) for exploring and creating meaning from participants subjective experiences, and for gaining insight into phenomena that are ill-defined or poorly understood. The strength of these methods rests in their ability to reveal the underlying truth of a phenomenon in research terms, its validity. It may be true only for a single individual at one point in time and in one particular context, but listening to or observing individuals is much more likely to elucidate their subjective experience as they perceive it than any amount of counting. As Greenhalgh puts it: The strength of qualitative research lies in validity (closeness to the truth) that is, good qualitative research, using a selection of data collection methods, really should touch the core of what is going on rather than just skimming the surface. The validity of qualitative methods is greatly improved by the use of more than one method in combination ( triangulation), by the researcher thinking carefully about what is going on and how their own perspective might be influencing the data ( reflexivity), (Greenhalgh, 2006, pp.168-9). However, you do need to bear in mind that the strength of qualitative methods may also be the source of a potential weakness. Generalisation of the findings of qualitative research to other contexts (people, setting) may not be justified, or at least not without appeal to a wider evidence base. It is the reliability of quantitative methods which give us confidence that the same study would produce roughly the same findings if it were to be repeated elsewhere with different participants. This reliability allows us to generalise with a degree of confidence beyond the immediate context in which a study has been conducted. Authors of qualitative studies may not help you much with this, so you do need to think carefully about the context of the research study and how this may differ from your own. From a service development point of view there is no point in searching for meaning in studies whose context or setting will not illuminate your own service issues. A fantastic research team and top-notch methods wont do you much good if you go looking in the wrong place. As the story goes: Late one evening, a passer-by notices someone searching about under a lamppost. Have you lost something, says the publicspirited passer-by, Can I help you? The searcher replies that he has dropped his car key and accepts the offer of assistance. Some time elapses while both search the ground all round the lamppost. Are you sure you dropped it here? queries the passerby, I cant see it anywhere. Eh, no, says the searcher, I dropped it down the road but its so dark down there I couldnt see properly so I came up here where its brighter. There is a plethora of literature on qualitative methods should you wish to read about this more widely (Barbour, 2008; Immy, 2010). I also recommend a useful short summary article that is a free download from the British Medical Journal (Kuper et al., 2008).

READ THIS SERIES IF YOU WANT TO yy BE MORE EVIDENCE-BASED IN YOUR PRACTICE yy FEEL MOTIVATED TO READ JOURNAL ARTICLES yy INFLUENCE THE DEVELOPMENT OF YOUR SERVICE

methods are expected to have their research question clearly formulated at the outset. In qualitative designs, however, it is acceptable for the aims to be influenced by the data analysis (the so-called iterative approach, in which results influence questions, which then influence results, and so on in a cyclic fashion). The authors should therefore explain if and how their aims have been shaped during the study. Question 2: Was the choice of qualitative approach appropriate?

QUALITATIVE RESEARCH FRAMEWORK

Use the following questions adapted from Greenhalgh (2006) and CASP (PHRU, 2006) to assess the rigour, credibility and relevance of the study and whether there are implications for your service. This is set up at www/speechmag. com/Members/CASLT as a document for you to download, print off and use as an individual or with colleagues in a journal club. Question 1: Did the paper describe an important clinical problem addressed via a clearly formulated question?

Was a qualitative method appropriate in the first place? Look for evidence that the study sought to interpret or illuminate the actions and / or subjective experiences of the research participants. Was the research design appropriate to address its aims? Consider whether the authors discussed selection and justification of their methods. Question 3: How were the participants selected?

This may be broken down into whether the aims of the research were clearly stated, what the point of the study was, why the problem is important and how the issue is relevant for clinical practice. Studies that use quantitative

Have the authors explained how the participants were selected? You may be less familiar with qualitative sampling methods who you select to interview or observe, or which texts you choose to analyse as they are different from the representative sampling

SPEECH & LANGUAGE THERAPY IN PRACTICE WINTER 2010

19

JOURNAL CLUB
(and variations thereof) used in quantitative designs. Kuper et al. (2008) have a helpful list of qualitative sampling methods for interviews or focus groups in healthcare settings. These include self-evident ones like: typical or deviant case sampling, to less familiar ones like: maximum-variation sampling sampling from a range of perspectives that is wide enough to include all the factors that might influence the nature and quality of peoples experiences, or snowball sampling each recruit to the study generates more participants through personal contact, often used for research with hard to reach or stigmatised groups. Have the authors explained why the participants they selected were the most appropriate to provide access to the type of knowledge sought by the study? Were there any discussions around recruitment, such as why some people chose not to take part? Question 4: Were the data generated in an appropriate way?
Qualitative design methods involve exploration and interpretation via data generation. They are stronger on validity (closeness to the truth). They are the preferred methods for poorly understood or relatively unexplored phenomena. Documents Passive observation Participantobservation Semistructured interview Narrative interview Study of documents produced by real people in real situations (for example, casenotes). Systematic recording of behaviour and talk in naturally occurring settings. The researcher takes part in the setting as well as observing. Face-to-face (or telephone) conversation with the purpose of exploring issues or topics in detail. Uses a pre-set list of questions or topics but is not restricted to these. Interview undertaken in a less structured fashion, with the purpose of getting a long story from the interviewee (typically a life story or the story of how a condition has unfolded over time). The interviewer uses only general prompts to tell me more. Method of group interview which explicitly includes and uses the group interactions to generate data.

Focus Groups

Table 1 Qualitative methods

Was the setting justified the right place to look rather than the easiest? (Not just looking under lampposts!) Peoples experiences of your service may be fresh in their mind on the way out of the door from their first appointment but perhaps they might provide more meaningful information once they have experienced some intervention. In my local setting, we recently conducted a series of focus groups with people with aphasia. The participants were all folk whose Samantha Paula stroke had occurred relatively recently but who were now discharged from speech and language therapy. We could be reasonably confident that: a. the groups views provided a unique perspective on their whole care pathway, which reflected our (and other healthcare practitioners) current practice, b. the impact of communication difficulties had been minimised, and c. there was no undue influence from a desire to maintain current therapeutic relationships. The findings are being used as an evidence base for our current service strategy - for example, that we invest effort and resources in raising awareness of aphasia and health promotion for people living with aphasia in our local community.

Is it clear how data were generated in the study, and have the researchers justified the methods chosen? Table 1 is extracted from the first article in this series to remind you of the most common sorts of qualitative methods. Authors should make their methods very explicit so that you can judge whether bias might have crept in. For example, for an interview method, is there an indication of how interviews were conducted? If different interviewers were involved, how did they minimise differences in personal interviewing style? Did they use a topic guide or a standard set of questions and / or prompts? A lot of qualitative research is exploratory and so researchers are expected to reflect on their data as it is being generated. This means that methods may be modified during the study. Check whether this happened in this study, and, if so, do the authors provide enough explanation of how and why? Is the form of data clear? A good test is whether you can you visualise what the data looks like (tape recordings, video material, notes)? Illustrations of field diaries, transcription frameworks and so on are helpful. Now we come to a bit of qualitative technospeak. Did they discuss data saturation? This is the point at which your method is generating no new information. Question 5: Was the study conducted within an appropriate ethical framework?

Participants in a research study need to understand what they are letting themselves in for, why the study is important and how their contribution will be used once the study is completed and disseminated. This is true of all research. However, in qualitative studies the ethical issues can be trickier, since the personal experiences of the participants will not necessarily be reduced to a set of anonymous numbers. There are potential consequences when peoples personal experiences are exposed both to themselves and to others. Have the authors discussed the ethical issues raised by the study, such as informed consent (and how they achieved this where children or participants with communication difficulties are concerned), confidentiality and how they have handled the effects of the study on the participants during and after the study? Its not enough just to know that approval was gained from a recognised ethics committee (but they should mention this as well). Question 6: What perspective is the researcher coming from, and how has this been built into the study? Another bit of qualitative technospeak is reflexivity the extent to which the researchers critically examined their own role, potential bias and influence during the course of the study. You need to look at reflexivity in the formulation of the research questions, in the data generation, including how they recruited their sample and in their choice of location. It is also critical that they explain how they responded to events during the study and considered the implications of any changes in the research design.

20

SPEECH & LANGUAGE THERAPY IN PRACTICE WINTER 2010

JOURNAL CLUB
Question 7: How have the data been analysed? it one or two dangers of its own. Ignoring data that doesnt fit into any of your categories or themes is one of them. To what extent have contradictory data been taken into account? Question 8: How credible are the findings? Question 10: What impact does this study have? Do the study findings resonate with my own experiences (and / or those of my colleagues) and therefore have a potential impact: for my practice for my colleagues or care group for the service as a whole? Is there a further question to be asked, or a local study to be carried out, that would allow me to generalise the findings to my own context? And finally... In another recent journal club we used this framework to appraise an article on the perceptions of clinicians and parents on the outcomes of speech and language therapy with children aged 2-6 years (Thomas-Stonell et al., 2009). The study had been conducted in Canada but we looked very carefully at how they selected their participants and at the sorts of provider organisations included. We concluded that there seemed no reason to think that the participants were significantly different from the sorts of families we see in our own paediatric service. Their findings on the disparity between parents and therapists perceptions and perspectives of change also rang true for us, providing support for our current service-wide focus on outcome measures that are clientcentred, and related to impact rather than being impairment-based. Parents, apparently, are more aware than therapists of how speech, language and communication disorders are affecting their childrens social participation. SLTP Now theres a surprise! Jennifer Reid is a consultant speech and language therapist with NHS Fife, email jenniferreid@nhs.net.
Barbour, R.S. (2008) Introducing qualitative research: a students guide to the craft of qualitative research. Los Angeles:Sage. Graves, J. (2007) Factors influencing indirect speech and language therapy interventions for adults with learning disabilities: the perceptions of carers and therapists, Int J Lang Comm Dis, 42(S1), pp.103-121. Greenhalgh, T. (2006) How to read a paper: the basics of evidence-based medicine (3rd edn). Oxford: Blackwell. Holloway, I. & Wheeler, S. (2010) Qualitative Research in Nursing and Healthcare. Chichester:Wiley-Blackwell. Kuper, A., Lingard, L. & Levinson, W. (2008) Critically appraising qualitative data, BMJ 337, pp. 687-90. Available at: http://www.bmj.com/content/337/bmj. a1035.full (Accessed: 8 November 2010.) Public Health Research Unit (2006) Critical Appraisal Skills Programme. Available at: www.phru.nhs.uk/Pages/PHD/ CASP.htm (Accessed: 8 November 2010.) Skeat, J. & Perry, A. (2008) Grounded theory as a method for research in speech and language therapy, Int J Lang Comm Dis, 43(2), pp.95-109. Thomas-Stonell, N., Oddson, B., Robertson, B. & Rosenbaum, P. (2009) Predicted and observed outcomes in preschool children following speech and language treatment: Parent and clinician perspectives, J Comm Disorders 42, pp.29-42.

Did the authors critically examine their own role, potential bias and influence during analysis and selection of data for presentation? Remember that qualitative research often uses an iterative method where data generation and analysis proceed hand-inhand. Each may influence the other in a cyclic fashion during the course of the study. This means that it is essential that the report includes an in-depth description of the analysis process. Was thematic analysis used? If so, is it clear how the categories or themes were derived from the data? Bottom-up categories are derived from the raw data itself whereas some studies will start with some of their categories pre-set (top-down), perhaps derived from previous research. Horses for courses, but the authors should have described how they arrived at their categories. Beware of bland statements about use of, for example, grounded theory, without any kind of explanation. Do not take on the responsibility for not knowing exactly what they mean! In a journal club earlier this year we were reviewing an article on factors influencing indirect interventions for adults with learning disability (Graves, 2007), which we found helpful for supporting our current practice. The term grounded theory was new to most people in the group so we found her brief explanation invaluable its a bottom-up approach to building categories but I also found another article (Skeat & Perry, 2008) which actually explained the grounded theory approach in a speech and language therapy context. We could do with a few more accessible reviews of methods, illustrated from the speech and language therapy field, to help us with our appraisal of methods. Its common to include in the text of the article examples from the data set to illustrate categories and themes. Do they explain how the data presented were selected from the original sample to demonstrate the analysis process? Are sufficient data presented to support the findings? In qualitative research, the researcher is included as one of the study components, rather than being seen as some sort of objective, reliable, impartial and unbiased observer from the planet Zog. This brings with

First of all, consider whether the findings are explicit enough, then whether there is adequate discussion of the evidence both for and against the researchers arguments. Do they discuss how they have attempted to increase the credibility of their findings using methods such as triangulation (using different sources to elicit and compare information), respondent validation (using participants that are qualified to speak about the issue in question), more than one analyst? Do you believe the authors narrative? Finally, are the findings discussed in relation to the original research questions? Its important to know if their results did not in the end really illuminate the issue they set out to explore. After all, youre probably only appraising the article because you think the study aims were related to some aspect of your own clinical work. Question 9: How valuable are the findings? Do the authors discuss the contribution the study makes to existing knowledge or understanding? We have already established that its more difficult to generalise from qualitative studies so its important that the findings of this one study are plugged into the bigger picture, however fuzzy it might be. For example, do they consider the findings in relation to current practice or policy, to other relevant researchbased literature or to theory? A theoretical motivation is considered particularly appropriate for exploration of issues with no substantive research base. Qualitative findings sometimes provide at best an incomplete or inconclusive story. Do the authors identify new areas where research is necessary? Do they discuss whether or how the findings can be transferred to other populations or consider other ways the research may be used?

References

SPEECH & LANGUAGE THERAPY IN PRACTICE WINTER 2010

21