An introduction to Critical Appraisal for Psychiatry

Wednesday 17th Aug 2011

In an Academic Meeting at Department of Psychiatry, Eastbourne District General Hospital

Akira NAITO MD PhD CT1 to Dr. A. GALEA

Background
Psychiatry and neurology only became distinct medical specialities about a century ago. Antidepressants and antipsychotics were first evaluated in the 1960s. The efficacy of treatments (psychotherapies and biomedical treatments) could not be rigorously evaluated until the invention of RCT. The EBM movement appears to offer some solutions, but requires adequate support.
2

Evidence-Based Practice (EBP)

Definition of Evidence Based Medicine:
The conscientious, explicit and judicious use of current best evidence in making decisions about the care of patients (Sackett et all 2000)

The goal of EBP is the integration of:

1. Clinical expertise 2. Scientific evidence, and 3. Client/patient/caregiver values

Information solutions
Useful sources of information for clinical practice
The Cochrane Library: http://www.thecochranelibrary.com/ The Evidence-Based Mental Health (EBMH):
http://www.rcpsych.ac.uk/publications/journals/ebmhinfo1.aspx

Other internet resources

Clinical Evidence: http://clinicalevidence.bmj.com PubMed (MEDLINE): http://www.ncbi.nlm.nih.gov/pubmed National Institute of Clinical Excellence (NICE):
http://www.nice.org.uk/

Why doing critical appraisal?

Uncertainty is universal in clinical medicine. It takes practice to routinely identify areas of uncertainty and to ask clinical questions in a productive way. Such questions will be brief summaries of the clinical scenario and the information required, constructed in a way that makes searching for evidence likely to succeed.
5

Different types of question require different study designs

Aetiology:
What caused this illness? Cross-sectional surveys RCTs

Diagnosis:
What does this test result mean?

Prognosis:
What is likely to happen? Inception cohort studies RCTs

Harm:
Is being exposed to it likely to do harm?

Effectiveness of therapy/treatment:
Is this treatment likely to help? RCTs
6

Qualitative:
What are the outcomes that are most important?

Evidence levels (from NICE)

Ia. Systematic review / Meta-analysis of RCTs Ib. Randomised Controlled Trial (RCT) IIa. Non-randomised controlled study IIb. Quasi-experimental study III. Non-experimental descriptive study
Comparative studies, Correlation studies Case controlled studies

IV. Clinical experience

Expert-committee reports or opinions Evidence from expected authorities
7

Main types of bias

Selection bias
Sampling / Performance bias
non-representative identification/recruitment of subjects Unequal provision of care apart from treatment under evaluation

Response/ Attrition bias (non-/late- responders)

unrepresentative participation differs from responders Biased occurrence and loss to follow up

Observation / Detection bias

Interviewer bias (blind to subject status or not):
differential data recording in subject groups by researcher

Recall bias
historical data is selectively filtered

Confounding correlations/associations (not causal links)

8

What is critical appraisal?

The process by which a reader can assess whether a written material possesses:
Validity (Is it close to the truth?), and Applicability (Is it clinically useful?).

The process of carefully & systematically examining research to judge its:

Trustworthiness, and Value and Relevance in a particular context.
9

Common ground inquiries

1. Is the study/research/guideline valid?
Validity / Trustworthiness (Internal validity = free/minimum bias)

2. What does it show? & Is it important?

Value and Relevance

3. Can I use this in caring for my patients?

Applicability (for ones own decision making)
10

Structure of a paper
Introduction/Background
Clinical importance (Why do the study?) A review of previous work in the subject area Hypotheses to be tested: a null hypothesis

Methods Results Discussion / Conclusions

11

Methods
Research subjects
Subjects: Representative sample? (who? where from? Inclusion & exclusion criteria?) Measures: Validated/published measures? (reliability and validity in manner and process) Statistical tests + Power calculation

Types of a paper
A hierarchy of evidence (see the previous slide)
12

Results
Check whether original research question or hypothesis was properly assessed?
Appropriate tables and figures? Logical manner? Key findings?

Any inconsistencies in the data?

13

Discussion / Conclusions
Value and relevance of the study results Limitations of the study Implications of the studys findings
Generalisability of positive findings Implication of negative results

14

Any question so far?

(References for writing a paper) Writing a case report
http://careerfocus.bmjjournals.com/cgi/content/full/327/7424/2153-a

Tips and Check-points for academic writing in English

http://homepage2.nifty.com/akira_naito/english.doc

Using English for Academic Purposes

http://www.uefap.com/writing/writfram.htm
15

Example questions for therapy/treatment articles

16

What question did the study ask? + PICO Patients

Intervention Comparison Outcome(s)

17

Are the results of the trial valid?

Was the assignment of patients to treatments randomised? Were the groups similar at the start of the trial? Aside from the allocated treatment, were groups treated equally? Were all patients who entered the trial accounted for? and were they analysed in the groups to which they were randomised? Were measures objective or were the patients and clinicians kept blind to which treatment 18 was being received?

What were the results?

How large was the treatment effect?
Relative Risk (RR) Relative Risk Reduction (RRR) Number Needed to Treat (NNT) = 1 / ARR Coefficient of Intra-Cluster Correlation (ICC)

How precise was the estimate of the treatment effect? (confidence intervals for each estimate)
19

Will the results help me in caring for my patient?

The questions that you should ask before you decide to apply the results of the study to your patient are:
Is my patient so different to those in the study that the results cannot apply? Is the treatment feasible in my setting? Will the potential benefits of treatment outweigh the potential harms of treatment for my patient?
20

References
EBM Tools for Critical Appraisal
http://www.cebm.net/index.aspx?o=1157

What is critical appraisal?

http://www.medicine.ox.ac.uk/bandolier/painres/download/whatis/W hat_is_critical_appraisal.pdf

EBP tutorials - Critical Appraisal Skills

http://www.brighton.ac.uk/ncor/tutorials/EBP_Tutorial_intro_Critical_ Appraisal_Skills.pdf

Clinical Study Design and Methods Terminology

http://www.vetmed.wsu.edu/courses-jmgay/glossclinstudy.htm

Critical Appraisal for Psychiatry (MRCPsych Guides)

http://www.amazon.co.uk/dp/0443070172
21

Thank you!

22