Prof BV Moses Centre for

Evidence-Informed Health Care

and Health Policy
Christian Medical College, Vellore

SUMMARISING AND
Prathap Tharyan, MD, MRCPsych
INTERPRETING FINDINGS USING Adjunct Professor, Clinical
Epidemiology Unit
THE GRADE APPROACH
Grading of Recommendations: Assessment,
Development and Evaluation
http://www.gradeworkinggroup.org
ADDRESSING THE GAPS BETWEEN EVIDENCE AND PRACTICE

GRADE Summary of GRADE process for

Findings (SoF) Tables guideline developers
from
Systematic Reviews

Getting the evidence right Getting the evidence used

BVMC |
CMCVellore
 BVMC | 4
CMCVellore
FOREST PLOT OF COMPARISON: PRIMAQUINE 5 DAYS VERSUS PRIMAQUINE 14 DAYS
OUTCOME: 1.1P. VIVAXPARASITAEMIA > 30 DAYS AFTER STARTING PRIMAQUINE.

BVMC | 5
CMCVellore
RISK OF BIAS

BVMC | 6
CMCVellore
 BVMC | 7
CMCVellore
FOREST PLOT OF COMPARISON: 1 PRIMAQUINE: 5 DAYS VERSUS 14 DAYS,
OUTCOME: 1.1P. VIVAXPARASITAEMIA > 30 DAYS AFTER STARTING
PRIMAQUINE.

BVMC | 8
CMCVellore
 SUMMARIZING THE RESULTS OF A SYSTEMATIC REVIEW:
• In order to evaluate the results of a systematic review to inform policy
decisions, we need to know:
1. Do the pooled effect estimates (with 95% CIs) for each clinically
important outcome indicate appreciable advantages?
• Efficacy | Safety | Adherence | Costs | Quality of life | Other
2. How confident are we that these effects are likely to be seen in
practice? How certain are we that further research will not
change these estimates?
• A separate summary of findings table is created for each comparison
(if more than one)
• For each outcome (maximum 7 clinically important outcomes) the
GRADE quality (confidence / certainty) assessment is presented along
with effect estimate and absolute difference (with 95% CIs)

BVMC |
CMCVellore
 CONFIDENCE IN THE RESULTS DEPENDS ON THE QUALITY OF
EVIDENCE
Can we trust the evidence?
The risk of bias of the included trials: (Study Limitations)
Does the evidence apply to my setting?
The characteristics of the individual trials:
Population/comparisons/outcomes/health systems (Directness)
Are the results consistent in all the trials?
If not, can heterogeneity be explained? (Inconsistency)
Are the results precise?
Do the results clearly indicate statistically significant and clinically important
effects ? (Imprecision)
Would your treatment decision change if the effect estimate was at either
end of the 95% CI?
Are the data sufficient based on populations covered and number of events?
Is there evidence complete?
Did the review include all relevant trials or only easily accessed ones?
(Publication bias)

BVMC |
CMCVellore
 BVMC |
CMCVellore
 BVMC |
CMCVellore
 Linking numerical results to our confidence in the effect
estimates
Levels of certainty or uncertainty Terminology
4 = High We are very confident that the true effect lies close to that
RCTs of the estimate of the effect; It will
Further research is unlikely to change this estimate

3 = Moderate We are moderately confident in the effect estimate: The

true effect is likely to be close to the estimate of the effect, It probably
but there is a possibility that it is substantially different;
will
Further research may change this estimate

2 = Low Our confidence in the effect estimate is limited: The true

Observational effect may be substantially different from the estimate of It may
the effect; further research is likely to change this estimate

1 = Very low We have very little confidence in the effect estimate: The
true effect is likely to be substantially different from the
estimate of effect; further research is needed and is very
We don’t
likely to alter these estimates know

BVMC |
CMCVellore
 USING THE EVIDENCE: TO INFORM POLICY DECISIONS

The overall confidence in the Strength of the

result recommendations
• Strong recommendation: most
informed patients would choose the
recommended management
• Conditional recommendation:
patients’ choices will vary according
to their values and preferences

Balance of benefits/harms/ cost/burdens/preferences

Panel of stakeholders

BVMC |
CMCVellore
RATING QUALITY OF
EVIDENCE

Study design
 STUDY DESIGN
• In the GRADE approach to quality of evidence:
• randomized trials without important limitations provide
high quality evidence
• observational studies without special strengths or
important limitations provide low quality evidence
• Limitations or special strengths can, however, modify
the quality of the evidence of both randomized trials
and observational studies.

BVMC |
CMCVellore
 STUDY DESIGN

• Nonrandomised experimental trials (or quasi-RCT) will

automatically be downgraded for limitations in design (risk of
bias)
• Case series and case reports are observational studies.
Source of control group results is implicit or unclear, thus, they
will usually warrant downgrading from low to very low quality
evidence
• Expert opinion is not a category of quality of evidence. Expert
opinion represents an interpretation of evidence in the context
of experts' experiences and knowledge

BVMC |
CMCVellore
RATING QUALITY OF EVIDENCE

RCTs
For RCTs:
5 criteria to downgrade
• Limitations (of study design)
• Inconsistency (between results)
• Indirectness (of evidence)
• Imprecision (of measure of effect)
• Other (publication bias)

Quality of Evidence = HIGH

BVMC |
CMCVellore
5 criteria
• Limitations (of study design) No serious limitations
Serious limitations (-1)
Very serious limitations (-2)
• Inconsistency (between results)
• Indirectness (of evidence)
• Imprecision (of measure of effect)
• Other (publication bias)

Quality of Evidence = HIGH

BVMC |
CMCVellore
 LIMITATIONS IN STUDY QUALITY
• RCTs
• Improper generation of randomization sequence
• Lack of allocation concealment
• Lack of blinding leading to performance or detection bias
• Incomplete accounting for patients and outcomes
• Selective reporting or publication bias
• Other limitations
• stopping early for benefit observed in randomized trials, in particular in the
absence of adequate stopping rules
• use of un-validated patient-reported outcomes
• carry-over effects in cross-over trials
• recruitment bias in cluster-randomized trials

BVMC |
CMCVellore
The COCHRANE RISK OF BIAS TOOL
• Risk of bias table
• Risk of bias graph
• Risk of bias summary figure

BVMC |
CMCVellore
5 criteria
• Limitations (of study design) No serious limitations
• Inconsistency (between results) No serious inconsistency
Serious inconsistency (-1)
Very serious inconsistency (-2)
• Indirectness (of evidence)
• Imprecision (of measure of effect)
• Other (publication bias)

Quality of Evidence = HIGH

BVMC |
CMCVellore
 STATISTICAL HETEROGENEITY
• Statistical heterogeneity exists when the results of individual
studies are not “consistent” among themselves.

Clinical diversity
Methodological diversity Statistical
Biases heterogeneity

Chance

BVMC |
CMCVellore
 IDENTIFYING IMPORTANT INCONSISTENCY
• Non overlapping confidence intervals
• Chi-squared test with P value < 0.1
• I2 : I2 of 25% = low; 50% = moderate and 75% = high (tentative)
Higgins et al, BMJ 2003; 327: 557-560
Current interpretation:
• 0% to 40%: might not be important;
• 30% to 60%: may represent moderate heterogeneity*;
• 50% to 90%: may represent substantial heterogeneity*;
• 75% to 100%: considerable heterogeneity*.
• *The importance of the observed value of I2 depends on :
• magnitude and direction of effects and
• strength of evidence for heterogeneity (e.g. P value from the chi-
squared test, or a confidence interval for I2).
BVMC |
CMCVellore
 IMPORTANT INCONSISTENCY
• Unexplained heterogeneity may be a reason to downgrade
• If heterogeneity is explained (like in sub-group analyses),
then one need not downgrade
• Unexplained heterogeneity may or may not be a reason to
downgrade when results are variable but the differences are
between small and large beneficial effects

BVMC |
CMCVellore
 Is there important heterogeneity ?

SHOULD WE DOWNGRADE? BVMC |

CMCVellore
SHOULD WE DOWNGRADE?

BVMC |
CMCVellore
SHOULD WE DOWNGRADE?

BVMC |
CMCVellore
5 criteria
• Limitations (of study design) No serious limitations
• Inconsistency (between results) Serious inconsistency (-1)
• Indirectness (of evidence)
• Imprecision (of measure of effect)
• Other (publication bias)

1 Downgraded for serious inconsistency because……….

Quality of Evidence = MODERATE BVMC |

CMCVellore
5 criteria
• Limitations (of study design) No serious limitations
• Inconsistency (between results) Serious inconsistency (-1)
• Indirectness (of evidence) No serious indirectness
Serious indirectness (-1)
Very serious indirectness (-2)
• Imprecision (of measure of effect)
• Other (publication bias)

Quality of Evidence = MODERATE

1 Downgraded for serious inconsistency because……….

BVMC |
CMCVellore
 INDIRECTNESS

• Population: All studies includes only adults when children are also
affected in practice
• Intervention: Given in doses or ways not used in practice
• Comparator: Given in doses or ways not used in practice (E.g.: Newer
antipsychotics compared against high dose haloperidol and not
variable/low dose)
• Outcome: Not defined in similar ways, not assessed in all trials;
surrogate outcomes used. (E.g.: Rosiglitazone produces tight glycaemic
control but no evidence on preventing complications)

BVMC |
CMCVellore
5 criteria
• Limitations (of study design) No serious limitations
• Inconsistency (between results) Serious inconsistency (-1)
• Indirectness (of evidence) Very serious indirectness (-2)
• Imprecision (of measure of effect)
• Other (publication bias)

Quality of Evidence = VERY LOW

1 Downgraded for serious inconsistency because……….
2 Downgraded for very serious indirectness because…….

BVMC |
CMCVellore
 GRADING IMPRECISION: I- (95% CIS)

* appreciable benefit equates to

RR>1.25 for adverse effects or
RR<0.75 for benefits
$ non-appreciable benefit equates to

1.0>RR<1.25 for harms or

0.75>RR<1.0 for benefits

No serious imprecision Both limits of the 95% CI of the pooled estimate suggest
appreciable benefit* with one drug
Serious The 95% CI of the pooled estimate includes appreciable
benefit* and non-appreciable benefit$ with one drug

Very serious The 95% CI of the pooled estimate includes appreciable

benefit* with one drug and no significant difference BVMC |
CMCVellore
between the drugs
 NO SERIOUS IMPRECISION
Quality assessment Summary of Findings
No of Design Imprecision AL6 AQ+SP Relative
studies (95% CI)
Efficacy: Total Failure (P. Falciparum) Day 28 adjusted -
3 RCT no serious 8/460 67/459 RR 0.12
imprecision* (1.7%) (14.6%) (0.06 to 0.24)

* No serious imprecision: Both limits of the 95% CI of the pooled

estimate suggest appreciable benefit with AL6.

BVMC |
CMCVellore
 SERIOUS IMPRECISION

Quality assessment Summary of Findings

No of Design Imprecision AL6 AQ+SP Relative
studies (95% CI)
Transmission potential: Gametocyte carriage at day 14
4 RCT serious 8/761 19/775 RR 0.46
imprecision* (1.1%) (2.5%) (0.21 to 0.90)

*Serious imprecision: The 95% CI of the pooled estimate includes

appreciable benefit and non-appreciable benefit with AL6.
Some may not downgrade

BVMC |
CMCVellore
 VERY SERIOUS IMPRECISION

Quality assessment Summary of Findings

No of Design Imprecision AL6 AQ+SP Relative
studies (95% CI)
Harms: serious adverse events (including deaths)
5 RCT very serious 16/1319 18/1365 RR 0.98
imprecision* (1.2%) (1.3%) (0.56 to 1.10)

*Very serious imprecision: The 95%CI of the pooled estimate includes

appreciable benefit as well as appreciable harm with AL6 (and no
significant difference between the drugs).

BVMC |
CMCVellore
 GRADING IMPRECISION I: (95% CIS)
Exception:
• When event rates are very low:
• 95% confidence intervals around relative effects
(RR) can be very wide, but
• 95% confidence intervals around absolute effects
(RD) may be narrow.
• Under such circumstances:
• One need not downgrade the quality of evidence for
imprecision if the sample size is large (2000 in each
arm)

BVMC |
CMCVellore
 GRADING IMPRECISION: II
SAMPLE SIZE AND OPTIMAL INFORMATION SIZE
(OIS)
• Other reasons to downgrade for imprecision:
• Rule of thumb: Total number of events is less than 300
OR
• Total (cumulative) sample size is lower than the calculated optimal
information size (OIS)

BVMC |
CMCVellore
 OPTIMAL INFORMATION SIZE
• Optimal information size (OIS): represents the
number of patients generated by a conventional
sample size calculation specifying a particular
alpha and beta error, relative risk reduction, and
baseline event rate

Pogue and Yusuf, Controlled Clinical Trials, 1997;18:580-593

BVMC |
CMCVellore
 BVMC |
CMCVellore
OPTIMAL INFORMATION SIZE GIVEN Α OF 0.05 AND Β OF 0.2 FOR VARYING CONTROL EVENT
RATES AND RELATIVE RISKS

BVMC | 42
Journal of Clinical Epidemiology 2011 64, 1283-1293DOI: (10.1016/j.jclinepi.2011.01.012) CMCVellore
 IMPRECISION:

• For continuous outcomes, down-GRADE if:

• 95% confidence interval of the Mean Difference includes no effect and
the upper or lower confidence limit crosses the minimal important
difference (MID), either for benefit of harm
• if the MID is not known or use of different outcomes measures required
calculation of an effect size (ES), we suggest downgrading if the upper
or lower confidence limit crosses an effect size of 0.5 in either direction.

BVMC |
CMCVellore
5 criteria
• Limitations (of study design) No serious limitations
• Inconsistency (between results) Serious inconsistency (-1)
• Indirectness (of evidence) Very serious indirectness (-2)
• Imprecision (of measure of effect) (No serious imprecision)
• Other (publication bias)

Quality of Evidence = VERY LOW

1 Downgraded for serious inconsistency because……….
2 Downgraded for very serious indirectness because…….

BVMC |
CMCVellore
 Reporting biases

Statistically significant, ‘positive’ results are

• more likely to be published: publication bias
• more likely to be published rapidly: time lag bias
• more likely to be published in English: language bias
• more likely to be published more than once: multiple
publication bias
• more likely to be cited by others: citation bias

BVMC |
CMCVellore
We would expect less precise studies (with fewer participants and events) to be
more affected by the play of chance, and so more widely scattered about the
pooled estimate.
As studies get bigger with more events, we expect them to be closer to the
pooled estimate.
Overall, this should produce a triangular shape, or inverted funnel (depending on
how the axes are plotted)

BVMC |
CMCVellore
5 criteria
• Limitations (of study design) No serious limitations
• Inconsistency (between results) Serious inconsistency (-1)
• Indirectness (of evidence) Very serious indirectness (-2)
• Imprecision (of measure of effect) (No serious imprecision)
• Other (publication bias) (No other biases)

Quality of Evidence = VERY LOW

1 Downgraded for serious inconsistency because……….
2 Downgraded for very serious indirectness because…….

BVMC |
CMCVellore
 RATING QUALITY OF EVIDENCE
Observational studies
• Controlled before and after studies
• Interrupted time series
• Cohort studies
 4 levels of ‘Quality’

• 4 = High

• 3 = Moderate

Observational
• 2 = Low

• 1 = Very low

BVMC |
CMCVellore
 OBSERVATIONAL STUDIES
• Observational studies start as low quality
• Those that have been downgraded for any reason, as well as,
• Uncontrolled case series starting as very low quality
• Cannot be upgraded.

BVMC |
CMCVellore
 LIMITATIONS TO QUALITY IN OBSERVATIONAL
STUDIES: 1
• Failure to develop and apply appropriate eligibility criteria
(inclusion of control population)
• under- or over-matching in case-control studies
• selection of exposed and unexposed in cohort studies from different
populations

Observational

BVMC |
CMCVellore
 LIMITATIONS TO QUALITY IN OBSERVATIONAL
STUDIES: 2
• Flawed measurement of both exposure and outcome
• differences in measurement of exposure
• recall bias in case- control studies
• differential surveillance for outcome in exposed and unexposed in
cohort studies

BVMC |
CMCVellore
 LIMITATIONS TO QUALITY IN OBSERVATIONAL
STUDIES: 3
• Failure to adequately control confounding
• failure of accurate measurement of all known prognostic
factors
• failure to match for prognostic factors and/or adjustment in
statistical analysis
• Incomplete or inadequately short follow-up
• Downgrade 1 or 2 levels depending on extent to which
limitations may bias results

BVMC |
CMCVellore
3 additional criteria: to upgrade evidence from
Observational studies
• Strong association
• Confounders all act to reduce observed effect
• Dose-response effect

BVMC |
CMCVellore
 UPGRADING OBSERVATIONAL
STUDIES
• For magnitude of effect:
• Large effect:
• A meta-analysis of observational studies (with no threats to validity)
showed that bicycle helmets reduce the risk of head injuries in cyclists
(OR: 0.31, 95%CI: 0.26 to 0.37).
• This large effect suggests the evidence could be upgraded one level
• Very large effect:
• A meta-analysis of observational studies (with no limitations) showed that
warfarin prophylaxis reduces the risk of thromboembolism in patients with
cardiac valve replacement (RR: 0.17, 95%CI: 0.13 to 0.24).
• This very large effect suggests the evidence could be upgraded two levels

BVMC |
CMCVellore
 UPGRADING OBSERVATIONAL
STUDIES
• For plausible confounding:
• if only sicker patients receive an experimental
intervention or exposure, yet they still fare better,
it is likely that the actual intervention or exposure
effect is larger than the data suggest.
• Upgrade 1 level

BVMC |
CMCVellore
 UPGRADING OBSERVATIONAL
STUDIES
• A dose-response gradient:
• In observational studies (with no study
limitations) in patients receiving anticoagulation
with warfarin, higher levels of the international
normalized ratio (INR) leads to increased risk of
bleeding
• This increases our confidence that supra-
therapeutic anticoagulation levels increase
bleeding risk.
• Upgrade 1 level

BVMC |
CMCVellore
 BVMC |
CMCVellore
SOME VARIATIONS
 BVMC |
CMCVellore
DIFFERENCES FOR CONTINUOUS OUTCOMES

▪ typical effect taken from control group mean

scores

BVMC |
CMCVellore
DIFFERENCES FOR CONTINUOUS OUTCOMES

▪ mean difference, direct from forest plot

▪ standardized mean difference must be converted
to a representative scale

BVMC |
CMCVellore
DIFFERENCES FOR CONTINUOUS OUTCOMES

▪ no additional relative measure

BVMC |
CMCVellore
Part ii:
Primaquine to
prevent relapse
with vivax
malaria
Summary of findings

BVMC |
CMCVellore
FIVE DAYS VERSUS 14 DAYS OF PRIMAQUINE + CHLOROQUINE

BVMC |
CMCVellore
FIVE DAYS VERSUS 14 DAYS OF PRIMAQUINE +
CHLOROQUINE

BVMC |
CMCVellore
 STRENGTH OF RECOMMENDATIONS

• Strength of recommendation is determined by the balance

between desirable and undesirable consequences of alternative
management strategies, quality of evidence, variability in values
and preferences, and resource use
• Systematic reviews end with grading the quality of
recommendations and do not make recommendations- can
discuss the implications of the overall quality of evidence

BVMC |
CMCVellore
Part 3: Exercise-
Grading
evidence for
guidelines
Falciparum Malaria

BVMC |
CMCVellore
 BVMC |
CMCVellore
 BVMC |
CMCVellore
 BVMC |
CMCVellore
 BVMC |
CMCVellore
 PICO QUESTION
In patients diagnosed with severe falciparum malaria does treatment
with Artesunate (i.v.) reduce death compared to treatment with Quinine
(i.v.)?

BVMC |
CMCVellore
 BVMC |
CMCVellore
ARTESUNATE VERSUS QUININE IN SEVERE MALARIA
OUTCOME: DEATH

BVMC |
CMCVellore
Artesunate versus quinine in severe malaria
Outcome: Death

Effect estimate:
Relative = RR 0.62 (95% CI 0.51 to 0.75)
Absolute = ARR = 84 fewer per 1,000 (95% CI 56 to 109)
NNT = 1/ARR = 12 (95% 9 to 18)

BVMC |
CMCVellore
PART III: CAN WE TRUST THESE
EFFECT ESTIMATES?

What is the overall certainty of

evidence?
STUDY LIMITATIONS

BVMC |
CMCVellore
 BVMC |
CMCVellore
Artesunate versus quinine in severe malaria
Outcome: Death

BVMC |
CMCVellore
5 criteria
• Limitations (of study design) No serious limitations
• Inconsistency (between results)
• Indirectness (of evidence)
• Imprecision (of measure of effect)
• Other (publication bias)

Quality of Evidence = HIGH

BVMC |
CMCVellore
 INCONSISTENCY

• Is there inconsistency in the results?

• Is this inconsistency due to chance?
• If not due to chance, how much is chance and how much is due to true
inter-trial differences?
• If inconsistency is important, can it be explained?
• If it cannot be explained, then our confidence in the effect estimate is
reduced
BVMC |
CMCVellore
 Artesunate versus quinine in severe malaria
Outcome: Death

3 tests for heterogeneity:

• The eye ball test: Do the confidence intervals overlap?
• Chi² p-value: The probability that the observed differences between trials
occurred by chance
• I²: The percentage of the observed differences between trials that is not due to
chance
BVMC |
CMCVellore
AL6 VERSUS AQ+SP IN UNCOMPLICATED MALARIA
OUTCOME: TREATMENT FAILURE AT DAY 28

RR 0.50 (95%CI 0.43-0.58)

AL6 is superior to AQ+SP in treating uncomplicated malaria

BVMC |
CMCVellore
AL6 versus AQ+SP in uncomplicated malaria
Outcome: Treatment failure at day 28

BVMC |
CMCVellore
 Pneumococcal vaccine versus Placebo
Outcome: X-ray defined pneumonia

Effect estimate:
RR 0.79 (95% CI 0.73 to 0.85)

BVMC |
CMCVellore
 Pneumococcal vaccine versus Placebo
Outcome: X-ray defined pneumonia

Effect estimate:
RR 0.82 (95% CI 0.68 to 1.00)

BVMC |
CMCVellore
5 criteria
• Limitations (of study design) No serious limitations
• Inconsistency (between results) No serious inconsistency
• Indirectness (of evidence)
• Imprecision (of measure of effect)
• Other (publication bias)

Quality of Evidence = HIGH

BVMC |
CMCVellore
INDIRECTNESS

BVMC |
CMCVellore
Artesunate versus quinine in severe malaria
Outcome: Death

Population: Only 2 out of 6 trials included children.

Intervention: 5 out of 6 trials used IV artesunate, one used IM
Control: Only 4 trials gave the loading dose of Quinine

BVMC |
CMCVellore
Artesunate versus quinine in severe malaria
Outcome: Death (subgrouped by age)

BVMC |
CMCVellore
Artesunate versus quinine in severe malaria
Outcome: Death (subgrouped by IV/IM artesunate)

BVMC |
CMCVellore
Artesunate versus quinine in severe malaria
Outcome: Death (sub-grouped by loading dose Quinine)

BVMC |
CMCVellore
5 criteria
• Limitations (of study design) No serious limitations
• Inconsistency (between results) No serious inconsistency
• Indirectness (of evidence) No serious indirectness
• Imprecision (of measure of effect)
• Other (publication bias)

Quality of Evidence = HIGH

BVMC |
CMCVellore
IMPRECISION

BVMC |
CMCVellore
Artesunate versus quinine in severe malaria
Outcome: Death

Artesunate is superior to Quinine in Severe malaria

RR 0.62 (95% CI 0.51 to 0.75)

A precise result?
BVMC |
CMCVellore
 Artesunate versus quinine in severe malaria
Outcome: Neurological disability at discharge

Artesunate results in more neurological sequelae than Quinine

RR 2.21 (95% CI 0.64 to 7.63)

A precise result?
BVMC |
CMCVellore
Pneumococcal vaccine versus Placebo
Outcome: Clinical pneumonia

Pneumococcal vaccine reduces clinical pneumonia

RR 0.95 (95% CI 0.92 to 0.98)

A precise result?
BVMC |
CMCVellore
 Pneumococcal vaccine versus Placebo
Outcome: X-ray defined pneumonia

Pneumococcal vaccine reduces x-ray defined pneumonia

RR 0.82 (95% CI 0.68 to 1.00)

A precise result?
BVMC |
CMCVellore
DHA-P versus AS+MQ in uncomplicated malaria
Outcome: Serious adverse events

RR 0.90 95%CI 0.38 to 2.15

An imprecise result?

Absolute effect: 1 fewer per 1000 (95% CI from 4 fewer to 8 more)

BVMC |
CMCVellore
5 criteria
• Limitations (of study design) No serious limitations
• Inconsistency (between results) No serious inconsistency
• Indirectness (of evidence) No serious indirectness
• Imprecision (of measure of effect) No serious imprecision
• Other (publication bias)

Quality of Evidence = HIGH

BVMC |
CMCVellore
OTHER

BVMC |
CMCVellore
Standard Error 1

0.5 1.0 2
Risk ratio
BVMC |
CMCVellore
Artesunate versus quinine in severe malaria
Outcome: Death

BVMC |
CMCVellore
EVIDENCE PROFILES

BVMC |
CMCVellore
Artesunate compared to Quinine for Adults with severe malaria
Patient or population: Adults with severe malaria
Settings: India
Intervention: Artesunate
Comparison: Quinine
Outcomes Illustrative comparative risks* (95% CI) Relative effect Participants Quality Comments
(95% CI) (studies) (GRADE)
Assumed risk Corresponding risk

Quinine Artesunate

Death 222 per 1000 84 fewer per 1000 RR 0.62 1938 ⊕⊕⊕⊕
(from 56 fewer to (0.51 to 0.75) (6 studies) high1,2
109 fewer)
Neurological 5 per 1000 6 more per 1000 RR 2.21 1190 ⊕⊕⊝⊝
sequelae at (from 2 fewer to 35 (0.64 to 7.63) (2 studies) low3,4,5
discharge more)
Coma recovery The mean The mean Coma 231 ⊕⊝⊝⊝
time coma recovery recovery time in the (2 studies) very low
time in the intervention groups 6,7,8

control groups Was 2.11 hours

Was 55.80 Longer (from 19.17
Hours hrs quicker to 23.4
hrs longer)
Time to hospital The mean time The mean Time to 113 ⊕⊝⊝⊝
discharge to hospital hospital discharge (1 study) very low
discharge in in the intervention 5,9,10

the control groups was

groups was 0.10 days longer
5.00 Days (1.34 days quicker BVMC |
to 1.54 days
CMCVellore
Artesunate compared to Quinine for Children with Severe Malaria
Patient or population: Children with Severe Malaria
Settings: India
Intervention: Artesunate
Comparison: Quinine
Outcomes Illustrative comparative risks* (95% CI) Relative effect Participants Quality Comments
(95% CI) (studies) (GRADE)
Assumed risk Corresponding risk
Quinine Artesunate

Death 222 per 1000 84 fewer per 1000 RR 0.62 1938 ⊕⊕⊝⊝
(from 56 fewer to (0.51 to 0.75) (6 studies) low1,2
109 fewer)
Neurological 5 per 1000 6 more per 1000 RR 2.21 1190 ⊕⊝⊝⊝
Sequelae (from 2 fewer to 35 (0.64 to 7.63) (2 studies) very low3,4,5
more)
Coma recovery The mean The mean Coma 231 ⊕⊝⊝⊝
time coma recovery recovery time in the (2 studies) very low6,7,8
time in the intervention groups
control groups Was 2.11 hours
Was 55.80 Longer (from 19.17
Hours hrs quicker to 23.4
hrs longer)
Time to hospital The mean time The mean Time to 113 ⊕⊝⊝⊝
discharge to hospital hospital discharge (1 study) very low5, 9,10,
discharge in in the intervention
the control groups was
groups was 0.10 days longer
5.00 Days (1.34 days quicker BVMC |
to 1.54 days CMCVellore
Would you recommend Artemisinin in adults?
Would you recommend Artemisinin in children?

What other factors might you want to consider?

BVMC |
CMCVellore
 BVMC |
CMCVellore
 THE ROLE OF EVIDENCE
Local evidence
Systematic reviews
Judgements about modifying
Judgements about
factors, needs, values,
the impact of
resources
policies

Judgements about the

expected benefits,
harms, and
opportunity costs of
policies

Judgements about trade-offs

Desirable impacts Undesirable impacts

• Health benefits • Harms
• Less burdens’ • More burdens
• Savings • Costs

Good health policy

BVMC |
Support tools for Evidence Based Decision Making: Health Research Policy and Systems 2009, 7(Suppl 1): CMCVellore
 CONTEXTUALIZING EVIDENCE FOR GUIDELINES:
GOVERNMENT OF GHANA
• Government of Ghana: Reviewing essential medicines list for drugs to sue for malaria
• Consultative process with the Cochrane Infectious Diseases Group via WHO: Dave Sinclair
• World Health Organization recommends
• IV Artesunate in preference to Quinine in treating severe malaria due to P.
falciparum in children (Strong recommendation; High Quality evidence)
• Cochrane Systematic Review: 4 RCTs; 5764 children: artesunate resulted in appreciably
fewer deaths than quinine
• Government policy: Use quinine in preference to artesunate:
• Artesunate may have relative benefits over quinine but quinine is also effective (local &
global data)
• Quinine is cheap and easily available; artesunate is more expensive, and difficult to
ensure continued procurement
• Example of Evidence informing (not dictating) the decision

BVMC |
CMCVellore
“When you know something, to hold
that you know it, and when you do
not know something, to allow that
you do not know it; that is
knowledge.”
Confucius (551 BC - 479 BC)

BVMC |
CMCVellore