Critical Appraissal

(Diagnostic, Therapy, Prognosis, Etiology)

Validity
Precision
Aplicability

UPEP FK UNSRI

What is EBM?

Implementation of the best evidence

obtained from clinical research to
clinical practice

UPEP FK UNSRI

What is in the real world?

Haynes (An Intern Med 1986; 309 :105) ;
800 research articles in 4 famous journals
valid only 19%
Reid (JAMA 1995; 274: 651); 1300 research
articles on accuracy of diagnostic tools from urine
dipstick to MRI and CT scan valid only 6%
Cohrane Collaboration (1996): out of 16,000
studies on mild hypertension valid only 22

UPEP FK UNSRI

What is in the real world?

The case of gastric freezing machine
In the 70s, 2500 gastric freezing
machines for treatment of gastric bleeding
were sold
Until a randomized trial showed this
machine was not better than conventional
treatment

UPEP FK UNSRI

Why EBM?

Increasingly new evidence (1 million new

publications/year) should lead to major changes in
patients care
Traditional CME does not improve clinical performance
EBM can keep the physician up to date
(Sackett 1999, Geyman 2000)

UPEP FK UNSRI

How to do an appraisal ?
By asking questions:
Is it Valid ?
Is it Important ?
Is it Aplicable ?

UPEP FK UNSRI

PROGNOSTIC STUDY

Is this evidence about prognosis valid ?

1. Was a defined, representative sample of patients

assembled at a common (usual early) point in the
course of their disease ?
2. Was patient follow-up sufficiently long and complete ?
3. Were objective outcome criteria applied in a blind
fashion ?
4. If sub groups with different prognosis are identified:
- Was there adjustment for important prognostic factors?
- Was there validation in an independent group of test-set
patients?

UPEP FK UNSRI

Valid ?
1. Was a defined, representative sample of patients assembled at a common

(usual early) point in the course of their disease ?

Ideally : entire population who ever live who developed the

disease

How Close the report approaches to Ideal ?

How the disease was defined ?
How the participants were assembled ?
From what point in the disease should patients be followed ?
Inception cohort
Exception if only learn about late stage in the disease

UPEP FK UNSRI

Valid ?
2. Was patient follow-up sufficiently long
and complete ?
Ideally : every patient in the inception cohort would be followed
until they fully recover or develop one of the other disease
outcomes
Study prognosis 100 patients, 4 die, 16 lost to follow up
A Crude case-fatality rate 4,8 % (4/84 x100%)

UPEP FK UNSRI

Valid ?
3. Were objective outcome criteria applied
in a blind fashion ?
Diseases affect patients; some are easy to spot and some are
more subtle.
Extreme outcomes ; death or full recovery. (easy to detect)
More difficult between them; rediness to work, intensity of
residual paint.

UPEP FK UNSRI

Valid ?
4. If sub groups with different prognosis are
identified:
- Was there adjustment for important
prognostic factors?
- Was there validation in an independent
group of test-set patients?

UPEP FK UNSRI

Is This valid evidence about prognosis

important ?
How likely are the outcomes over time ?

UPEP FK UNSRI

Important ?

How precise are the prognostic estimates ?

The text, tables, graphics of a proper

prognostic study include the confidence
intervals for estimates of prognosis

UPEP FK UNSRI

Can we apply this valid, important evidence

about prognosis to our patient?
Are the study patients similar to our own ?
Are the study patients so different from ours that we should
not use the result at all in making prediction for our
patients?

Will this evidence make a clinically important impact on

our conclusions about what to offer or tell our patient ?

UPEP FK UNSRI

Diagnosis
Was the test compared blindly with a gold standard?
Was there an adequate spectrum of disease?
Was the referral pattern described?
Was the description of the tests clear enough to
reproduce it?
Was the test reproducible (observer agreement)?
Was contribution of the tests to overall diagnosis
assessed?

UPEP FK UNSRI

Therapy
Was the assignment really randomized?
Were clinically important outcomes assesses
objectively?
Was the treatment feasible to your practice?
Were their at least 80% follow up of subjects
Were both statistical and clinical significant
considered?
If the study was negative the power assessed?

UPEP FK UNSRI

ETIOLOGY
Was the type of study strong? (RCT>Cohort>casecontrol>cross-sectional)
Was the assessment of exposure and outcome free of
bias (blinded assessors)?
Were both association statistically and clinically
significant?
Was the association consistent across studies?
Was cause shown to precede the effect?
Was there a dose response relationship?

UPEP FK UNSRI

Evidence-Based Guidelines
Effectiveness of Clinical Intervention
Level Type of evidence
Ia
Ib
IIa
IIb
III
IV
V

Meta-analysis of randomized trial

At least one randomized trial
Well-designed, controlled study
Well-designed, quasi-experimental study
Descriptive and comparative studies.
Non random study.
Case serial, expert panel / committee

UPEP FK UNSRI

Recommendations On Clinical Intervention

Grade
A
B
C
D
E

Nature of Recommendation

Ia + Ib / ( > I )
Ia / Ib
IIa / IIb
III
IV / V

UPEP FK UNSRI