The GATE frame:

critical appraisal
with picture
Dr. Saryono, MKes.
sarbiokim@gmail.com
Medical evidence increasing at epidemic rates: we
all need EBP skills to keep up-to-date

Bastian, Glasziou, Chalmers (2010) 75 Trials and 11Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med 7(9)
Medical evidence increasing at epidemic rates: we
all need EBP skills to keep up-to-date

approx 75 new
trials published
every day

Bastian, Glasziou, Chalmers (2010) 75 Trials and 11Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med 7(9)
Medical evidence increasing at epidemic rates: we
all need EBP skills to keep up-to-date
MEDLINE 2010
2,000 articles / day

approx 75 new
trials published
every day

Bastian, Glasziou, Chalmers (2010) 75 Trials and 11Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med 7(9)
About 10% of published
evidence is worth
reading =bernilai

About 1/3 of
worthwhile evidence is
eventually refuted or
attenuated = lemah

About 1/2 of
relevant evidence is
not implemented
The Problem
Vast and expanding
literature.
Limited time to read.
Different reasons to read
mean different strategies.
Keeping up to date.
Answering specific clinical
questions.
Pursuing a research interest.
6
 Akurasi uji diagnostik
Kekuatan prognosis
Efikasi
Keamanan intervensi

Validity of this evidence

Critical appraisal Alat telaah grafis

Graphic Appraisal Tool for

Epidemiological studies (GATE)
GATE

Merupakan alat telaah graphis untuk penelitian

epidemiologi
Manfaat: untuk mengkonseptualisasikan keseluruhan
penelitian dalam bentuk komponen2nya
Merupakan rangka visual yang mengilustrasikan desain
umum dari penelitian epidemiologi
Graphic Appraisal Tool for Epidemiological studies

Graphic Approach To Evidence Based Practice

Graphic Approach To Epidemiology

 the GATE frame

the shape of every epidemiological study 11

11
 British doctors

smoking status measured

smokers non-smokers

yes
Lung cancer
5 years
no

Longitudinal (cohort) study12

 British doctors

smoking status measured

smokers non-smokers

normal
Lung function
abnormal

Cross-sectional study 13
 British doctors

Randomised to aspirin or placebo

aspirin placebo

yes
Myocardial infarction 5 years
no

RCT
14
 Middle-aged US women

Test applied

Mammogram positive Mammogram negative

yes
Breast cancer
no

Clinical use of a diagnostic test

15
 Middle-aged US women

Breast cancer no Breast cancer

positive
mammogram
negative

Diagnostic test accuracy study

16
GATE: Graphic Appraisal Tool for
Epidemiological studies

1 picture, 2 formulas & 3 acronyms

17
17
 One picture: the GATE frame

every epidemiological study hangs on the GATE frame

18
GATE Frame
Population
 Grouping

Exposure / Intervention
Comparison

Grouping
Outcome
 Diwakili oleh panah
Time horisontal dan vertikal
Panah horisontal
untuk hasl penelitian
yang diukur saat satu
waktu
Panah vertikal
digunakan untuk
menggambarkan
pengukuran outcome
setelah beberapa
waktu
Kerangka GATE
PECOT/PICOT
Population
Exposure/intervention
Comparison
Outcome
Time
The 1st acronym = PECOT : the 5 parts of
every epidemiological study

Participants P

Exposure Group E C Comparison Group

O Outcomes
Time
T
25
Lewis RT et al. Should antibiotic prophylaxis be used
routinely in clean surgical procedures: A tentative yes.
Surgery 1995;118:742-7.

26
Background. The incidence of surgical site infection (SSI) after
clean surgical procedure is regarded as too low for routine
antibiotic prophylaxis. But risk of SSI can be as high as 20%. We
assessed the value of prophylactic cefotaxime in patients stratified
for risk of SSI in a double-blind RCT.
Methods. Patients having clean elective operations were stratified
for risk & randomized to receive IV cefotaxime 2 gm or placebo
before operation & followed for 4-6 weeks for SSI.
Results. The 378 of 775 patients who received cefotaxime had
70% fewer SSIs than those who did not --Mantel-Haenszel risk
ratio (MH-RR) 0.31; 95 % CI 0.11 to 0.83. Benefit was clear in the
616 low risk patients--0.97% versus 3.9% SSI (MH-RR 0.25, CI
0.07 to 0.87, p = 0.018), but only a trend was seen in 136 high risk
patients--2.8% versus 6.1% SSI (MH-RR 0.48, CI 0.09 to 2.5).
Conclusions. The results indicate clear benefit for routine
antibiotic prophylaxis in clean surgical procedures.
27
High risk
patients need more study. 27
28
1st critical appraisal task: describe studys design by
hanging on GATE frame using PECOT acronym

E C

O
T 29
 Participants
Study Setting

Eligible Participants

P
Participants
30
Lewis Trial
Participants
Study Setting: patients admitted to QE Hospital,
Montreal, Canada (1992-5?)

Eligible Participants: undergoing clean surgery or

simple cholecystectomy

Participants: 633 (?consecutive eligible patients)

31
 Exposure & Comparison Groups

Exposure or Comparison or
Intervention Group EG CG Control Group
(EG) (CG)

32
 Exposure & Comparison Groups:
low risk group

633

316 317
Exposure or Comparison or
Intervention Group Control Group
(EG): (CG):
2g cefotaxime IV pre- Identical placebo
op IV

33
 Exposure & Comparison Groups:
low risk group

633

316 317
Exposure or Comparison or
Intervention Group Control Group
(EG):
308* 308* (CG):
2mg cefotaxime IV Identical placebo
pre-op IV

* With complete
34 follow-up
 Outcomes (O)

yes a b
Dis-ease O Outcomes (O)
no c d
35
 633

Outcomes (O)
316 317

a= 3 b=12
yes
Surgical site
infection (SSI)
O Primary Outcome
no c d (O)
36
Time (T)

incidence
T
prevalence
37
 Time (T)
316 317
Outcome: SSI

incidence
T= time from
initiation of treatment
to end of follow-up
38
 Study design: GATE frame & PECOT

Participants P

Exposure Group E C Comparison Group

O Outcomes
Time
T 39
Lewis

Setting: QE Hospital, Montreal

Eligible: clean surgery or cholecystectomy
Participants 633

316 317
Exposure Group: 308 308 Comparison Group:
IV cefotaxime IV placebo

Time: 3 12
Outcomes:
Up to 6 wks post-op SSI
40
Questions?

41
The 1st formula: study analyses

Occurrence (risk) of disease

= Numerator Denominator D

42
All epidemiological studies involve measuring
the OCCURRENCE of outcomes

D Denominator (Participants)

N Numerator (Outcomes)

Occ = ND 43
All epidemiological studies involve measuring
the OCCURRENCE of outcomes

D Denominator (Participants)
During what period of time (T) was N
T measured? (incidence)

N Numerator (Outcomes)

Occ = ND (T?) 44
All epidemiological studies involve measuring
the OCCURRENCE of outcomes

D Denominator (Participants)
T
At what point in time (T) was N measured?
(prevalence)
N Numerator (Outcomes)

Occ = ND (T?) 45
The 1st formula:
Occurrence (risk) = Numerator Denominator

Exposed
P Comparison
Group Group

DE DC

T NE NC T
O
46
2nd appraisal task: describe analyses by hanging
numbers on the GATE frame and calculating
occurrences in exposure & comparison groups

Denominator 1: Denominator 2:
Exposure Group EG CG Comparison Group
(EG) (CG)

Numerator 1: a a b Numerator 2: b
O
c d 47
Occurrence = N D
P

Denominator 1: Denominator 2:
Exposure Group EG CG Comparison Group
EG CG

Numerator 1: a b Numerator 2:
a O b
c d

Exposure Group Occurrence: Comparison Group Occurrence:

EGO = a EG CGO = b CG
48
Occurrence = N D
P

Denominator 1: Denominator 2:
Exposure Group EG CG Comparison Group
EG CG

Numerator 1: a b Numerator 2:
a O b
c d

Exposure Group Occurrence: Comparison Group Occurrence:

EGO = a EG CGO = b CG
49
Occurrence = N D
P

Denominator 1: Denominator 2:
Exposure Group EG CG Comparison Group
EG CG

Numerator 1: a b Numerator 2:
a O b
c d

Exposure Group Occurrence: Comparison Group Occurrence:

EGO = a EG CGO = b CG
50
Calculate EGO & CGO for SSI in low risk group
P

Denominator 1: Denominator 2:
Exposure Group EG CG Comparison Group
EG = 604 CG = 604

Numerator 1: a b Numerator 2:
a = 87 O b = 101
c d

EGO = 87/604= 144,04/1000 CGO = 101/604 =

167,22/1000
51
Calculate EGO & CGO for SSI in low risk group
P

Denominator 1: Denominator 2:
Exposure Group EG CG Comparison Group
EG = 308 CG = 308

Numerator 1: a b Numerator 2:
a=3 O b = 12
c d

EGO = 3/308= 9.7/1000 at 6 CGO = 12/308 =39/1000 at

weeks 6 weeks
52
OT (on treatment) or per-protocol analysis
 Describing differences between
occurrences
Relative difference or Relative Risk = EGO CGO

Absolute Difference or Risk Difference = EGO - CGO

Number Needed To Treat (NNT) = 1 RD

53
 Describing differences between
occurrences (SSI in low risk patients)
Relative difference or Relative Risk = EGO CGO
= 144,04/1000 167,22/1000 = 0.86
Absolute Difference or Risk Difference = EGO - CGO

= 144,04/1000 - 167,22/1000 = -23.18/1000

Number Needed To Treat (NNT) = 1 RD

= 1 (- 23.18 /1000) = - 1000/23.18 = 43.14

if 35 patients were given IV cefotaxime pre-op, there would be 1 fewer SSI up
to 6 weeks post-op 54
Study analyses

its all about EGO & CGO

55
Questions?

56
Kunci validitas dalam GATE
RAAAMBo
Represent: keterwakilan partisipan
Allocation: alokasi kelompok terpajan dan
pembanding
Adjustment: penyelesaian confounding
Accounted: pencatatan
Measured: pengukuran akurat
Blinding: pembutaan
Objective: objektif
RAAAMBo
The 2nd acronym = RAMBO* : assessing bias
strength of study

P P Recruitment

E Allocation

C Maintenance
E C

O
Blind or
T O
T Objective outcomes
measurement
59 59
* Paul Glasziou
The 2nd acronym = RAMBO* : assessing non
random error (i.e. bias)

P P Recruitment

E Allocation

C Maintenance
E C

O
Blind or
T O
T Objective outcomes
measurement
60 60
* Paul Glasziou
3rd appraisal task: assess the degree of bias by
applying the RAMBO acronym

P P Recruitment

E Allocation

C Maintenance
E C

O
Blind or
T O
T Objective outcomes
measurement
61
 Study setting

Eligible people
RAMBO
were Recruitment processes
P appropriate to study goals?
P
Study setting & eligibility criteria well
described?
e.g. Recruit random/representative sample
E C OR consecutive eligibles OR volunteers
from advertisements
Participants representative of eligibles?
Prognostic/risk profile appropriate to
study question?
O
T
62
RAMBO: A is for Allocation

Was Allocation to
EG & CG
successful?

RCT: Allocate randomly by Cohort: Allocate by

investigators (e.g drugs) measurement (e.g. smoking)

were EG & CG
EG CG similar at EG CG
baseline?

O O 63
T T
 P
RAMBO
were Participants Maintained as
allocated?
did most participants remain in
allocated groups (EG & CG)
EG CG Participants &/or investigators blind to
exposure (and comparison exposure)?

Compliance high & similar in EG & CG?

Contamination low & similar in EG & CG?
O Co-interventions low & similar in EG & CG?
T
Completeness of follow-up high & similar in EG
& CG? 64
 P RAMBO
Were outcomes measured
Blind or Objectively?

EG CG

If outcome measurements not

Objective (eg. automated or definitive)
O were investigators Blind to exposure
T (and comparison exposure)
65
The 4 (GATE) study biases
P Recruitment bias

Allocation bias

E C Maintenance bias

Outcomes
O
T Measurement bias
66
Questions?

67
Excel CATs & paper Gate-lites

There is a GATE for every study design

www.epiq.co.nz 68
68
Final appraisal task: search for & appraise
SRs / meta-analyses using 3rd acronym (FAITH)

Find appropriate studies?

Appraise selected studies?
Include only valid studies?
Total-up (synthesise) appropriately?
Heterogeneity adequately addressed?
69
69
Systematic Reviews

There are 4 Cochrane SRs on this topic and

the findings are not consistent

70
Using GATE
as a framework for evidence based practice
 The first 4 steps of EBP
1. Ask a focused question.
2. Access (systematically search for) epidemiological
evidence to help answer question.
3. Appraise evidence found for its validity, effect size,
precision (ideally all the relevant evidence)
4. Apply the evidence:
a. amalgamate the valid evidence with other relevant
information (patient/community values, clinical/health
issues, & policy context) and make an evidence-based
decision; and
b. act (implement) the decision in practice
 EBP Step 1: Ask- turn your question into a
5-part PECOT question

Participants (the patient problem)

Exposure (e.g. a therapy)
Comparison (there is always an alternative! - another therapy or
no treatment
Outcome (e.g. a disease you want to prevent or manage)
Time frame (over which you expect a result)
 EBP Step 2: Access the evidence use
PECOT to choose search terms

Participants (the patient problem)

Exposure (e.g. a therapy)
Comparison (there is always an alternative! - another therapy or
no treatment
Outcome (e.g. a disease you want to prevent or manage)
Time frame (over which you expect a result)

74
74
 EBP Step 3: Appraise the
evidence

using the best evidence from epidemiology to help inform

decisions

more critically (using GATE)

more systematically (using FAITH)
EBP Step 4: APPLY the evidence by: a.
AMALGAMATING the relevant information & making
an evidence-based decision: the X-factor

 X-factor: making evidence-based decisions

Evidence

Clinical / health
Patient / community considerations
preferences

Policy issues

Xpertise: putting it all together the art of

practice
Aplikasi evidence dalam praktik
X factor / expertise factor
Hasil riset harus
dipertimbangkan oleh
pembaca ketika membuat
good health care decisions
Kesimpulan
GATE frame mewakili struktur umum dalam penelitian
epidemiologi
GATE frame memfasilitasi telaah kritis terhadap
penelitian epidemiologi