Qualitative methods in combination with quantitative methods (Handout to presentation) Banff May 2-5 2003 Introduction My name is Karin

Johansson Blight. I am a registered nurse educated at the Red Cross School of nursing and health in Sweden and have got a MSc and diploma in Public Health from the London School of Hygiene and Tropical Medicine (LSHTM) in England. I am currently undertaking research at Karolinska Institutet as part of my doctoral thesis in transcultural psychology. Overhead 1:
Qualitative methods in combination with quantitative methods Karin Johansson Blight RGN, MSc & Diploma Public Health, Doctoral student Research carried out in Sweden Main academic advisor associate prof. Solvig Ekblad, Karolinska Institutet, Sweden Second academic advisor prof. Jan Ekberg, Vxj University, Sweden

My presentation today will concern the advantages and disadvantages of combining qualitative and quantitative methods. However, before embarking on the presentation I would like to briefly describe my own view on science, knowledge and methods. For me, science is a learning process, which involves a view that knowledge has a historical or maybe better described as having a cumulative component. This does, however, not necessarily mean that knowledge gained in the past consists of the one and only truth or alternatively that past knowledge is less worth than current. Instead, scientific knowledge may be rejected or seen as one way out of many to describe reality at one period in time and in one context. The different between scientific knowledge and other types of knowledge is, in my view, that one has a duty to describe the process from theory to conclusion and it involves critical thinking. I also believe that humans strive to categorise the world to understand it and tend to want to

generalise experiences. Finally, for me there is no hierarchy or contradiction between qualitative or quantitative research, in stead it is a matter of different tools used to understand and learn about aspects in the surrounding world.

To enable the discussion regarding advantages and disadvantages of a combination approach, I will use examples from my current research project Is mental ill health a barrier to employment, a study conducted amongst persons of refugee background from Bosnia Herzegovina. The initiator of this project is my second academic advisor, a professor of economics at Vxj University in Sweden, called Jan Ekberg. Jan was working investigating access to employment amongst persons from Bosnia Herzegovina who came to Sweden in 1993 and 1994, when he came in contact with my main academic advisor and chair of this session, Solvig Ekblad. Jan had found that amongst these 48.000 persons included in the study and coming from Bosnia Herzegovina during those years there were large differences in access to employment depending on where these persons lived. What he discussed with Solvig was whether these observed differences could have any links to health, especially mental health. From these discussions a research project was formed to investigate the relationship between employment and mental health using the observations drawn from the Jan Ekbergs register study. Clarifying concepts Before explaining the aims and hypothesis underlying the project in more detail I would like to briefly discuss aspects concerning the concepts: qualitative and quantitative. Quantitative analysis focus, according to Starring (1994), on the quantity of phenomena and according to Barbosa da Silva and Wahlberg (1994) is the quantitative approach generally associated with for example something measurable and something objective and empirical.

Quantitative research often use pre-set questions and response alternatives that are given in the same order and terms used are structured or semi-structured. Research often aims at testing hypothesis and to investigate associations or cause/effect relationships. Overhead 2:
Clarifying concepts Quantitative analysis focus on the quantity of a phenomena (Starrin, 1994) Qualitative is the type of research that produces findings not arrived at by statistical procedures or other means of quantification (Strauss and Corbin, 1998)

Within the filed of quantitative research, there is a sub-categorisation used to describe the type of data collected and these sub-categories are, in turned, called quantitative and qualitative data. In this context, quantitative refers to numerical data with a mathematical value such as discrete (where only certain numerical values can be used. For example, number of children) and continuous (where the restrictions of the observations is made through the accuracy of the measuring instrument. For example, blood pressure. Information on blood pressure may contain several decimals but is usually restricted to the nearest 2 or 5 mm Hg). Another word for qualitative data, within quantitative research, is categorical data, which includes for example dichotomous/binary data (data with two categories such as male/female) and ordered categorical/ordinal data (several categories with a ordered structure such as categories used in the research I have carried out namely, no problem, a bit of a problem, moderately serious, a serious problem and a very serious problem). These types of data are often given a numerical label, although, this numerical label does not represent a mathematical value. Ordered categorical data are often used to measure subjective assessments of things that (using Altmans words) cannot be measured such as level of experience of pain (Altman 1991, page 11).

Qualitative research, according to Strauss and Corbin (1998), is the type of research that produces findings not arrived at by statistical procedures or other means of quantification (page 11). However, they argue, that some of the data within qualitative research may be quantified such as census data but with this Strauss and Corbin (1998) do not refer to the quantifying of qualitative data but rather to a nonmathematical process of interpretation, carried out for the purpose of discovering concepts and relationships in raw data and then organizing these into a theoretical explanatory scheme (page 11). In qualitative research, questions and order may vary and it is possible to use probing. Terms used include semistructured or unstructured, interview guides and themes. Research often aims at for example, investigating sensitive topics and explore research topics and processes.

To sum up, I will in this presentation use the concepts qualitative and quantitative when I refer to the broad research areas. If referring to data collected within the field of quantitative research I will use the concepts categorical and numerical. References: Altman, D.G. (1991) Practical statistics for medical research. London: Chapman &Hall. Barbosa da Silva, A., Wahlberg, V. (1994) Vetenskapsteoretisk grund fr kvalitativ metod. kapitel 2, i Kvalitativ metod och vetenskapsteori, Starrin, B., Svensson, P-G. (red). Starrin, B. (1994) Om distinktionen kvalitativ-kvantitativ i social forskning, kapitel 1, i Kvalitativ metod och vetenskapsteori, Starrin, B., Svensson, P-G. (red). Strauss, A., Corbin, J. (1998) Basics of qualitative research: techniques and procedures for developing Grounded Theory. Sage Publications: California.

Description of project aim and hypotheses Now, to enable the discussion on advantages and disadvantages concerning the combination of the two fields I will, as I mentioned before, use the study I am presently working on. This study concerns a sample of persons coming from Bosnia Herzegovina to Sweden in 1993 and 1994. The aim of the study is to increase knowledge about the relationship between perceived mental health, amongst newly arrived refugees, and access to the Swedish labour market. Underlying the aim are two hypotheses that we want to explore. These hypotheses are well known within employment research and they are: that poor mental health is a barrier to employment and that there is a reverse relationship, which means that prevention of access to labour market creates psychosocial stress and this in turn leads to mental ill health even if the person originally felt healthy. Overhead 3:
Combining methods: illustration

Title: Is mental ill health a barrier to employment? (year: 2001-2004) Aim: To increase knowledge about the relationship between perceived mental health, amongst newly arrived refugees, and access to the Swedish labour market Hypotheses to explore: 1) poor mental health is a barrier to employment, 2) the reverse relationship; prevention of access to labour market creates psychosocial stress, which leads to mental ill health even if the person originally felt healthy

This project is part of a larger project called health promoting introduction and is commissioned by the National Integration Office in Sweden and is co-financed by the European Refugee Fund. It started in 2001 and is planned to finish in this year (2003). If you are interested in more detailed information regarding the overall project I have an information leaflet with me that you are welcome to take part of. Reference: Health Promoting Introduction- information leaflet. Version 23Jan03.

Research design Overhead 4:

Research design

Register Postal questionnaire In-depth interviews

The research approach we decided on to meet the aim of the project was a combination of quantitative methods: register and postal questionnaire (sent to 650 persons) and a qualitative method: interviews (26 persons). The choice of this combination approach was largely due to the fact that the origin of the main research question was based in a quantitative study and with this we had access to data from the register that my second academic advisor had set up. This had many advantages as we, through the register, had access to 50 variables containing information dating back since the persons arrived in Sweden, regarding their access to employment, economic status, education level in Sweden and indications on mental health from for example number of days on illness benefits and so on. However, to further understand the relationship between mental health and employment we were interested in the perceptions of the persons coming to Sweden as refugees themselves. Overhead 5:

Choice of methods Interested in:

Individuals history Integration process Perceptions of health and integration aspects

Context Had access to: Register variables of relevance to research aim and hypotheses

So, through a postal questionnaire, sent to a sample within the register, we could explore questions such as What is the prevalence of perceived mental ill health in this population? How does long term unemployment in Sweden relate to mental health in this group? The register variables was then also used in combination with the postal questionnaire to explore certain background variables (e.g. age or income) in the register in relation to items measuring perceived mental health in the postal questionnaire. Through the register we also had the opportunity to gain an understanding of the non-respondents of the postal questionnaire, at group level.

A purely quantitative approach would, however, not provide us with sufficient information. We know from previous research as well as from agencies such as the UNHCR (United Nations High Commissioner for Refugees) that the mental health of refugees is affected by the events experienced during for example war and forced migration. The register data would not provide information regarding this type of pre-migration and migration experiences. Furthermore, in our view, it is not ethical to distribute postal questionnaires asking questions about pre-migration experiences, which could include for example, experiences of ethnic cleansing. Moreover, the theory we are using to understand and explain the mental health of refugees includes an understanding of the context within which the persons are exposed to a potentially traumatic event and also within which the recovery after a traumatic event takes place. The latter is especially relevant when wanting to understand the relationship between mental health and employment. Of importance here is also research, which has shown that post migration factors (experiences or life circumstances in the host country) such as being worried for relatives outside Sweden, have an impact on the experience of mental health amongst refugees (Sndergaard, 2002; Steel, Z et al 1999). So, in other words we needed tools to provide more information regarding context. Additionally, in our view, we cannot see

access to employment as a static situation. It is instead a matter of a process that is part of a larger process- that of integration. For this the complex matter of the integration process a more in-depth approach is needed to enable the highlighting of important aspects. References: Statistiska centralbyrn (2001) Invandrare frn Bosnien- registerstudie. Steel, Z., Silove, D., Bird, K., McGorry, P., Mohan, P. (1999) Pathways from War trauma to Posttraumatic Stress Symptoms Amomg Tamil Aylum Seekers, Refugees and Immigrants. Journal of Traumatic Stress 12(3):421-435. Sndergaard, H.P. (2002) Post-traumatic stress disorder and life events among recently resettled refugees. Department of Public Health Sciences, Division of Psychosocial Factors and Health. Karolinska Insitutet. Doctoral thesis. UNHCR (2001) The State of The Worlds Refugees 2000. Oxford University Press.

Lessons learnt From the experiences of using a combination approach there are firstly a few aspects I would like to share with you regarding what to think of before you embark on a combination of qualitative and quantitative research.

One of the most important aspects is whether your model allows this sort of combination. Professor David Silverman defines model as: an overall framework for how we look at reality (Silverman 2000, page 77) and means that in social research examples of models are functionalism and behaviourism. Other concepts describing similar to models are, according to Silverman, paradigm or idiom(Silverman 2000). Secondly, it is also important to think through your aim of your research. Does it include a possibility of using

both methods? Finally, in relation to these two aspects it is important to consider whether it is necessary to use both fields of research methods. Overhead 6:
To think of

Appropriateness of:

Modell (paradigm, idiom) - Aim Translation and interpretation Representativeness of groups Choice of methods within each approach

Another important aspect to consider when combining approaches is whether there are several language groups within sampled population. If so, one has to consider that a thorough translation procedure is necessary when using questionnaires and this may take months. At the same time interpreters might be needed for conducting interviews. The question is can you afford and do you have time for both?

It is important to consider how to chose participants for the different groups of investigation. In our case, one sample was drawn from a register to investigate quantitative research aspects and another sample was gathered using snowball/convenience-sampling methods to investigate qualitative research aspects. To be able to use the information gathered through the different approaches the two samples were matched as closely as possible with regards to certain inclusion criteria. The qualitative collected information will be used to provide reflections in relation to the quantitative collected information. However, it is important to highlight that just because the two samples (the 650 receivers of a postal questionnaire and the 26 interviewees) are similar in inclusion criteria does not mean we can generalise the experiences in the interviews to the whole postal questionnaire sample. Likewise, it is

important to remember that care must be taken in generalising the results from the postal questionnaire to other refugee populations.

Finally, what methods within each approach (qualitative and quantitative) do you want to use, and in what order will you collect data? In our study, within the postal questionnaire we had one page at the end where the respondents could provide comments freely. In the interviews I used opened questions, however, at the end of the interview I asked three structured questions but with space given to discuss the interviewees understanding of the questions. The data collection using postal questionnaire started about tone and a half month earlier than the data collection using interviews. For a few months data collection was simultaneously and then the data collection using postal questionnaire ended about two and a half month earlier than the data collection using interviews. Reference: Silverman (2000) Doing qualitative research; A practical handbook. Sage publications: London. Advantages of combining To begin on the main aim with this presentation, I will now discuss the advantages and disadvantages that I can identify using a combination approach in the way we have using examples from our research. Overhead 7:
Advantages of combining

Sensitive or difficult questions Validation of information Missed questions, clarification of questions A face to the numbers, keep the closeness to the research population Research fields can learn from each other Correct data available Generalising but with information on context

Sensitive or difficult questions: When using a combination approach you are given the opportunity to ask questions appropriate to the setting. In our research, questions that were to sensitive to ask about in the postal questionnaire such as the pre migration experiences, I could explore in the interviews. This included for example experiences of being beaten or witnessing beatings or periods with out food or having to live in the forest. Because of the nature of the war in former Yugoslavia we decided not to ask the respondents of the postal questionnaire about their religion. Instead, we explored this in the interviews. This provided further knowledge that would most certainly not have been captured using only quantitative methods, which consisted of experiences presented by the interviewees that they previously had not been thinking of themselves in terms of religion but because of Milosevic and the war they now identifies themselves as belonging to a certain religion.

Validation of information: Using a combination approach it is also possible to validate information. In the postal questionnaire we are using a list of problems experienced in relation to living situation in the last month such as have you had a feeling of loss of respect and status in society? or have you felt unsafe? The answers to these questions can be compared with how interviewees describe their life, are similar aspects brought up? If so, are they attributed similar importance? Overhead 8:
3-item Sense of coherence 1) Do you usually see a solution to problems and difficulties that other people fins difficult to solve?- manageability 2) Do you usually feel that the things that happen to you in your daily life are hard to understand? - comprehensibility 3) Do you usually feel that your daily life is a source of personal satisfaction?- meaningfulness Response categories: 1) Yes, usually; 2) Yes, sometimes; 3) No (Lundberg & Nystrm, 1995)

Another example is that of the understanding of the three questions of Sense of Coherence. This is 3-item measurement a revised version of Antonowskys Sense of Coherence developed by Lundberg & Nystrm (1995) and includes the following questions: 1) Do you usually see a solution to problems and difficulties that other people fins difficult to solve?- manageability 2) Do you usually feel that the things that happen to you in your daily life are hard to understand? - comprehensibility 3) Do you usually feel that your daily life is a source of personal satisfaction?meaningfulness The response categories are: Yes, usually, 2) Yes, sometimes, 3) No In the interviews there was a possibility for the interviewees to give feedback on their understanding of the questions. Here, some interviewees found difficulties in understanding the question regarding comprehensibility (see above). Comments were given that it was difficulties to understanding the war it self or what Milosevic did. Also, the question regarding meaningfulness seemed to some extent difficult and my impression is that the concept personal satisfaction was too abstract. Perhaps this is due to the attempts to comprise SOC too much (into three items) or perhaps it is to do with equivalence

Missed questions, clarification of questions: The combination approach also provides the opportunity to ask questions that have been missed in one form of data collection. In our case data, collection in the quantitative study and the qualitative study overlapped. However, the postal questionnaire was set and could not be changed. The interviews on the other hand provided the opportunity to raise aspects that had been missed in the questionnaire. One example is the importance of, what the interviewees

expressed as, a goal in life to feel well. Another example is the clear focus on children that the interviewees expressed. This included aspects such as the difficulties experienced by the parent when the parent did not have the possibility to provide his/her child with food so that the child/children had to starve or, in post-migration experience, worries about what could happen to the child/children when him/her grow up in Sweden (for example the childs ability to cope with school or possible exposure to drugs etc).

When it comes to clarification of questions, issues of equivalence- something that is very important in transcultural research- may be found during the research process. One example here is the use of the word family. In the postal questionnaire family was used in relation to a question concerning how the respondent is currently living. The response alternatives were: living on his/her own, together with husband/wife/partner, together with family/relatives, together with a friend/s. Some persons filled out the option living together with husband/wife/partner and added and children to that option. This could in my own view be categorised as the option living together with family/relatives. However, in the interviews the issue of what constitute a family could be furthered explored. Here, it was explored in relation to a map of networks and three concepts were used for identifying contact with persons in relation to home, family and relatives. It turned out that the interviewees had very varied understanding of who is included in family and the interpretations also differed in relation to how family differ in comparison with home and relatives.

A face to the numbers, keep the closeness to the research population Qualitative data can help in giving a face to the figures and to keep the closeness to the research population. The first is, in my view, especially important in migration research. By

keeping a face to the figures, there is a greater possibility to remember the persons/humans behind the data collected, including their reactions and expressions in relation to positive or negative experiences. This is, I believe, of great importance in transcultural research in order not loose the humanitarian aspects and in this way to work against dehumanisation and distance of groups or individuals of. For researchers, including myself, who are not currently undertaking any clinical work, this is of probably of even greater importance.

Research fields can learn from each other My main thought on this aspect is that I will, in the presentation of the results stemming from a combination approach, use the initiative to scrutinise my role as a researcher, and the effect I have, on the collected data. This approach is, as understand it, more thoroughly used in qualitative research. But by using the combination approach I also make it applicable to the data collected using the quantitative approach. There are most certainly other aspects that the approaches can learn from each other too

Correct data available Another advantage with a combination approach, as I see it, is that the researcher is able to collect the right data from the right environment. Aspects of process that are difficult to explore in a quantitative way may be found using the qualitative methods. If quantitative data is available methods can be used to minimise bias and in this way increase representativeness. Furthermore, to be able to capture structures in society quantitative methods can be very useful. In the research I am undertaking, which is based on prof. Ekbergs research involving 45.000 persons from Bosnia Herzegovina, we know that there are very large regional differences between region 1 (10%) and region 2 (80%) in access to employment. Because of the large number of people this research includes, we can presume that the observed

differences are not due to individual characteristics within the group. Instead, it is more possible that there are larger structures involved in allowing persons to access employment or keeping them in unemployment. This could be, for example, discriminatory structures in different regions, differences in work opportunities or that, like our research projects hypotheses, which states that persons who are excluded from the labour market may suffer mental ill health because of the exclusion. But also that persons experiencing mental ill health might also experience difficulties in accessing employment. This, in turn, may have the effect on mobility in the sense that to those persons who feel well actively move to areas where they can get jobs. Quantitative data is in this way helpful in highlighting trends and present proportions. Qualitative methods, on the other hand, can help us explore whether there are other phenomena contributing to these differences than those we already know of (such as discrimination or poor employment opportunities in one region).

Generalising but with information on context As mentioned in the introduction, I believe that humans tend to want to generalise experiences and we, as researchers, have to work actively not to do so in many situations. Additionally, in the research project I am working in we are using theories where context play an important role in understanding health and especially mental health of refugees. I mentioned the understanding of in relation to trauma where the context regarding trauma exposure and recovery is of vital importance. One context-specific aspect I noticed when conducting the interviews were that some of the persons I interviewed who lived in smaller towns in region 2 in Sweden expressed that it was so quiet and empty and no people outside as compared to the same-sized towns they were living in Bosnia Herzegovina prior to migration where members of the town used to take evening strolls and visit cafs and such. When analysing quantitative data on loneliness and boredom, these aspects should be incorporated as one explanatory

factor for any regional differences observed with regards to this might be that in the smaller towns in Sweden socialising outside the houses in public places is a less frequent phenomena than in big cities such as the one in region 1.

Disadvantages of combining Overhead 9:

Disadvantages of combining

Lots of data, can and will all be used? Risk of shallow interpretations Keeping up-dated Risk of confusion Time influence on data collection Time consuming and expensive

Combining approaches can easily produce a lot of data and too much untreated data could be viewed as unethical. Considering this is a time consuming and expensive approach this must be balanced with the opportunity to make interpretations of quality and not be tempted into shallow interpretations. For a single researcher might be difficult to be up-dated with models, theories and methods used within both approaches. Additionally, it is easy to be confused about what rules of work are applicable in relation to what type of method. This process and a separation on what is what may, however, be helped through collaboration across disciplines as well as through meeting researchers regularly in research groups where methods are being discussed. The aspect of time might influence data collection. In our study, there was a major factory providing employment for several hundreds of people in region 2 (consisting of several small towns) in Sweden. Shortly after the data collection through the postal questionnaire started this factory closed down. The qualitative interviews in the region were undertaken once the factory was closed and this will most possibly affect the answers given.

Prerequisites Overhead 10:

Certain prerequisites needed to be able to combine A belief in the usefulness of both designs (this includes model and theory) Access to study participants Ethical approval Time and economy Final advice: Rather carry out a non-combined research approach as well as possible than compromising quality in a combined approach

The certain prerequisites needed to be able to combine approaches are, in my view, first and foremost a belief that this is a valuable way to collect data. This includes understanding and permission in terms of models and theories used but also an understanding in relation to sources of funding of the study.

As with any other type of study participant must be available. In many cases in migration research it might be easier to access participants using qualitative methods as, especially, refugee populations quite often is mobile. Another aspect is that of trust and for this reason it might be more feasible to conduct a qualitative study (less participants are needed and if using, for example, a snowball sampling technique access might be enhanced).

Finally, ethical approval from ethic committees should, as with any other study, be given prior to embarking on this type of study and a through judgement on aspects of time and economy should be adhered to. As in my view, the choice for the researcher should always be quality (even when conducting quantitative research) and this means that one should rather carry out a non-combined research approach as well as possible than compromising quality in a combined approach.