Criteria for evaluating case studies

Criteria reflecting sensitivity to readers needs.

Criteria reflecting thoroughness of data collection and analysis.

Criteria reflecting use of sound research methods.

Criteria reflecting sensitivity to readers needs. (Kriteria yang mencerminkan sensitivity terhadap keperluan pembaca)
Strong chain of evidence
Quasistatistics Participatory models

Truthfulness

Usefulness

Strong chain of evidence.

Researcher able to provide a clear documentation on a) The sources and methods of recording raw data. (interview and audiotaped) b) Process notes. (write a journal on the process of the data collection and analysis. c) The development of the instruments and procedures used to collect data. (interview) d) Data reduction and analysis products. (select important documents) e) Data reconstruction and synthesis product. (build up the finding)

Truthfulness
Researchers descriptions are faithfully representations of the phenomena that were studied (verisimilitude- palpably felt) Ex: Criticisms of the trustworthiness of qualitative research have included researcher bias and subjectivity and the fact that the researcher served as the interpreter of data. The use of multiple data collection methods contributes to the trustworthiness of the data.

Usefulness
Researcher can enlighten readers need. Ex: The findings can be applied to the readers settings and used to make meaningful changes in how they deal with phenomena that the case study involved.

Participatory models
Researchers involve participants in all phases of the research from conceptualization of the study to write the final report. Ex: The researcher continued to communicate by email until after the research project was completed.

Quasi-statistics
Researcher able to provide simple numerical results from the data to support conclusions of findings.

Criteria reflecting thoroughness of data collection and analysis. (Kriteria yang mencerminkan keseluruhan pengumpulan data dan analisis)
Triangulation Contextual completeness

Rich data

Representativeness check

Long-term observation

Triangulation of data
Time- researchers collect data about a phenomena at different points in time. Ex: The initial interview was followed 2 months later with a second interview. Space- researchers collect data at more than one site. Ex: Ambers room and her parents. Person- researchers collect data from more than one level of person, that is either a set of individuals or groups.

Contextual completeness
Researchers able to interpret (i)Multivocality- participants have diverse points of views and interests. (i)Tacit knowledge- participants cannot finds the words to express or they take so much granted that they do not explicate them either in

research interviews.
(i) History, physical setting & environment, the number of participants, specific activities, the schedules & temporal order of events, divisions of labor, routines and variations from routines, significant events and their origins and consequences, members perceptions and meaning, social rules and basic pattern of order.

Long term observation

Researchers able to gather data (i)Over a long period of time and (ii)Making repeated observations of the phenomenon.

Ex: The researcher had collected and analyzed data over a 3-month period, working persistently to discover pattern and themes.

Representativeness check
Researchers able to compare the sample with others to avoid overreliance on accessible or elite informants. Ex: Compared with the other sample, Reed Ball who died by suicide.

Rich data
Researchers able to show interview transcripts and detailed notes or recordings of observations.

Criteria reflecting use of sound research methods. (Krtiteria yang mencerminkan kesempurnaan penggunaan kaedah penyelidikan)
Member checking Disconfirming case analysis Coding checks Peer examination Intervention & pattern matching Researchers reflection

Comparison

Member checking
Researchers able to ask participants to review statements in the report for accuracy and completeness to correct factual errors and if necessary, collect more data to reconcile discrepancies. Ex: Members check with the family included sending interview transcripts to Ambers mother so that she could check for accuracy and perhaps think of additional topics that she like to add or clarify.

Intervention and pattern matching

Researchers able to check whether the patterns in the data correspond to predictions drawn from the intervention.

Ex: The researcher finds the commonalities of suicide (Shneidman,1985; Stillion et al, 1989)

Researcher reflection
Researchers able to discuss role relationship, researchers assumptions, world view, theoretical orientation and positionality toward the phenomenon being investigated on their study. Ex: Researcher is a data interpreter only, bias and emotion are excluded in the study.

Comparison
Researcher able to compare the same setting at different times to address the counterfactual of what would have happened without the presence of the pressured course.

Peer examination
Researcher able to ask colleagues to comment on findings as they emerge and to review a draft of the case study report.

Coding checks
Researchers able to use low-inference descriptors and mechanical means to record data. Thus, to check the inter-rater reliability of the coding process.

Disconfirming case analysis

Researchers able to find the outlier (an individuals or situation that differs greatly from most other individuals or situations that are studied. They may even ask the skeptical colleagues to look at the raw data and independently come up with their own conclusions.

Positivist conceptions of the validity & reliability of case study research

Validity is the strength of our conclusions, inferences or propositions. In short, were we right?
1. Construct validity
2. Internal validity

3. External validity

Construct validity
occurs when the theoretical constructs of cause and effect accurately represent the real-world situations they are intended to model. Ex: The Suicidal Mind (Shneidman, 1996) listed 10 commonalities of suicide where Amber has fitted to some of the commonalities such as feeling of hopeless and escaped from difficulties (being rejected and wont go to a good college).

Internal validity
occurs when it can be concluded that there is a causal relationship between variables being studied.

Ex: Is there a relationship between the variables (being bullied, rejected & misunderstood, suicidal ideation & collaborating with peers, perfectionism, lack of trust) with Ambers death, is it a causal relationship?

External validity
occurs when the causal relationship discovered can be generalized to other people, times and contexts. Ex: Could we generalize our results to other suicide of a gifted female adolescent?

Reliability
Reliability is the consistency of the measurement, or the degree to which an instrument measures the same way each time it is used under the same condition with the same subjects. Ex: The researcher has carried interviews for twice.

Determining the applicability of case study findings

Applicability here means to which extent the studys finding can be applied to the individuals or situations in the practitioners setting? 1. Sampling strategies a. Typical or extreme case, the result should be applicable to other similar cases. b. Multiple-case design, a cross-case analysis will be conducted to help readers determine whether there was a generalizability of findings among the cases that they studied. The presence of crosscase generalizability is strong evidence that the findings are applicable to other situations and individuals than those studied by the researchers.

2. Judgement on readers If readers find the similarity of the cases that were studied, the finding are more applicable.

Reporting case studies

There are two ways to report case study. (i)Reflective reporting The report is characterized by literacy devices and the strong presence of the researchers voice. (ii) Analytic reporting The report where the researchers voice is silent or

subdued and other conventions typical of quantitative

research reporting.

Strength of case study research

1. Able to catch unique features They catch unique features that may otherwise be lost in larger scale

data (survey); these unique features might hold the key to

understand the situation. Ex: Search in depth personals documents (writings, letters, and videotape) 2. Provide insights They provide insights into other similar situations and cases, thereby

assisting interpretation of other similar cases.

Ex: A female was needed to fill a gap in the research, as precious psychological autopsies of gifted adolescents had focused on males.

3. Can be conducted by one researcher only. They can be undertaken by a single researcher without needing a full research team. Ex: I was the only researcher who collected and analyzed the data. So, I choose to include some relevance documents and to exclude others.
4. Recognize the complexity and embeddedness of social truths. By carefully attending to social situations, case studies can represent something of the discrepancies or conflicts between the viewpoints held by participants. The best case studies are capable of offering some support to alternative interpretation. Ex: Even though school officials knew about bullying, they did nothing to intervene.

5. A step to action They begin in a word of action and contribute to it. Their insights maybe directly interpreted and put-to-use for staff or individual self-development, withininstitutional feedback, formative evaluation and in educational policy making. Ex: Implications from the findings include the need for changes in the education of gifted children and adolescents.
6. Thick description A richly detailed report that re-creates a situation and as much of its context as possible, along with the meanings and intentions inherent in that situation. Ex: The process of collecting data and analyzing data.

Weaknesses of case study research

1. The result may not be generalizable. It is because a case study is focused on a person or a small group only. 2. They are not easily open to cross checking. Hence they may be selective, biased, personal and subjective. 3. Ethical problems can arise. Identity of the organization or individuals that were studied must be disguised.

Who wants to be a case study researcher?

Win

Lose

The term emic perspective refers to

A. A positivist researchers view of the phenomenon being studied. B. An interpretive researchers view of the phenomenon being studied. C. The researcher participantsviews of the phenomenon being studied. D. The readers view of the phenomenon being studied.

The case study finding that there is an association between the amount of structure in teachers lessons and how teachers think about students learning progress is an example of a A. B. C. D. Construct Theme Thick description Pattern

The cases in case study research typically are selected by

A. B. C. D. Researcher interpretation Convenience sampling Replication logic Purposeful sampling

Experts generally agree that researcher bias in case studies is best handled by A. Honestly revealing ones possible biases and being willing to have them disconfirmed. B. Using data-collection methods that rule out the possibility of researcher bias. C. Using several researchers and seeking consensus in their conclusions. D. Not studying phenomena in which the researcher has a personal interest.

Reflective reporting of a case study tends to rely heavily on

A. Presentation of structural data analyses. B. An objective writing style. C. Formats developed for use in reporting quantitative research studies. D. The use of literary devices.