Knowledge Translation:

The steep path between evidence

generation and application

Brian Haynes
Health Information Research Unit
Dept of Clinical Epidemiology and Biostatistics
McMaster University
KNOWLEDGE IS THE ENEMY OF
DISEASE

The application of what we know

will have a bigger impact on health
and disease than any single drug or
technology likely to be introduced
in the next decade.
Sir Muir Gray, UK National Library for Health
 Knowledge Translation

the organization, retrieval,

appraisal, refinement,
dissemination, and uptake of
knowledge (eg, important new
knowledge from health research)
 Generalizable knowledge for better
clinical practice and healthcare

knowledge from research

(sometimes called evidence)
knowledge from the analysis of
routinely collected and audit data
(sometimes called statistics)
knowledge from the experience of
clinicians and patients.
 Cost-effectiveness of warfarin*

Warfarin for atrial fibrillation

$25CDN saved per stroke averted

Aspirin for atrial fibrillation

$65CDN saved per stroke

*Gustafsson C, et al. Cost effectiveness of primary

stroke prevention in atrial fibrillation: Swedish
national perspective. BMJ. 1992;305:1457-60.
What proportion of patients with
atrial fibrillation do not receive
anticoagulants?

50%

Bradley BC, et al. Frequency of anticoagulation for atrial

fibrillation and reasons for its non-use at a Veterans
Affairs medical center. Am J Cardiol. 2000 Mar
1;85(5):568-72.
 In Hamilton, Ontario,
The Clot Capital of the Universe,

the proportion of medical inpatients receiving

clot prevention according to guidelines is

33%
Current guideline adherence for diabetes

Intervention:
Ophthalmology assessment 46% - 80%
Proteinuria assessment 35% - 82%
Foot assessment 30% - 72%
HbA1c 16% - 87%
Cholesterol assessment 55% - 68%
Smoking status assessment 25% - 87%
In all, 73% of microalbuminuric
patients were not on ACE-I/ARB.
Hypertensive type II diabetic
patients were often left untreated
and only a minority of those treated
were optimally controlled. The
importance of an elevated systolic
pressure is underestimated and the
number of antihypertensive drugs
prescribed, insufficient. Screening
and treatment of albuminuria are
inadequate.
 The routine application of what
we know can prevent or minimise:

unknowing variation in clinical practice

errors of commission and omission
unsatisfactory patient experience
Evidence (from research) is necessary
but, of course, not sufficient

...it has to be combined with

the circumstances of the
individual patient and the values
of each patient. But without
evidence it is improbable that
patients, professionals, and
those who manage resources, will
to make good decisions.
 researchers

decision makers
 application a
generation synthesis policy
5
c b

4 decisions
3
1 2
Knowledge Translation
MRC
Steps
CIHR from evidence generation to clinical application
Steps: 1. generation of evidence from research; 2. evidence summary and
synthesis; 3. forming clinical policy; 4. application of policy; 5. individual
clinical decisions, including a) patients circumstances, b) patients wishes,
and c) evidence from research
Step 1. Generating Research Evidence

Barrier Solutions

too little research large, simple

addressing real randomized trials
world problems head to head
comparisons
Step 2. Synthesizing Research Evidence

Barrier Solutions

size and noise of research into

the research rating, abstracting,
enterprise and synthesizing
research
How much synthesis do we need?

..at least 10000 Cochrane reviews

are needed to cover a substantial
proportion of the studies relevant
to health care that have already
been identified

Susan Mallett & Mike Clarke

ACP Journal Club. 2003 Jul-Aug;139:A11.
When will we have our 10,000 reviews?
Growth of Cochrane Reviews and Protocols
2003
Non-Cochrane reviews: >50% of all reviews
2500 completed mid-2005 protocols
2000 completed mid-2004

1995 reviews

between 2010 and 2015.

Mallett&Clarke, ACPJC 2003
 Step 3. Developing Policy

Barrier Solutions
problems in national drug and
developing technology assessment
evidence-based agencies
clinical and health local leadership
policy
Step 4. Applying evidence in practice

Barrier Solutions
poor access to development and
current best testing of information
evidence and systems that integrate
guidelines evidence and guidelines
with patient care
(eg Diabetes In-
CHARGE)
 The McMaster PLUS project

only a tiny proportion of all research is

ready for application
only a tiny fraction of the ready
research is relevant to the practice of
a given clinician

only a tiny proportion of the relevant

research for a given practitioner is
interesting in the sense of being
something new, important, and actionable.
 Evidence-Based Journals
Critical Appraisal Filters

~2,500 articles/y
50,000 articles/y meet critical appraisal
from 120 journals and content criteria
(95% noise reduction)
 McMaster PLUS Project
Clinical Relevancy Filter (MORE)

~20 articles/yr for

clinicians (99.96%
~2,500 articles/y noise reduction)
meet critical appraisal
and content criteria
~5-50 articles/y for
(95% noise reduction)
authors of evidence-
based clinical topic
reviews
Dear Dr. Jones,

We want to alert you to NEW articles in the PLUS system.

These articles that have received very high relevancy and newsworthiness scores:

1. Bohlius
J, et al.
Erythropoietin for patients with malignant disease.
Cochrane Database Syst Rev 2004;(3):CD003407.

Rated by: IM/General (patients Relevance: 6 of Newsworthiness: 6

referred from Primary Care) 7 of 7

2. Gourlay S, et al. Clonidine for smoking cessation.

Cochrane Database Syst Rev 2004;3:CD000058.

Rated by: IM/General (patients Relevance: 6 of Newsworthiness: 6

referred from Primary Care) 7 of 7

We hope that you will find these articles of value in your clinical practice.

Best wishes from the PLUS Team

 PLUS Trial Northern Ontario Physicians

134 non- 344 consent eligible

7 refused
respondent consent
203 randomized: 10 communities

6 small clusters 4 large clusters

Group 1 (3) Group 2 (3) Group 1 (2) Group 2 (2)

2 left study
 Intervention
Randomization to 2 different trial interfaces

SelfServeVersion FullServeVersion
Ovid Ovid
Stat!Ref Stat! Ref
Pyramid of Evidence Pyramid of Evidence
PLUSEmailAlerts
PLUSSearchEngine
 PLUS Preliminary Findings:
% of Participants Using PLUS by Month
Baseline (5 mo) Self-serve vs Full-serve Full-Serve

70
Percentage Using PLUS

60
50
40
30
20
10
Relative increase 58.7%, P=0.001
RCT begins Control cross-over begins
0
Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun
03 03 04 04 04 04 04 04 04 04 04 04 04 04 05 05 05 05 05 05
Month
Self-serve Full-Serve
Free EBM literature updating service
http://www.bmjupdates.com

Free at www.bmjupdates.com!
(sponsored by BMJ Publishing Group)
 Step 4. Applying evidence in practice

Barrier Solution
ineffectual continuing effective continuing
education education and quality
improvement programs for
practitioners
Step 4. Applying evidence in clinical
decisions

Barrier Solution
ignorance about shift a portion of health
barriers and their investment from services to
solutions quality improvement
 WHO estimates US$100B/yr for
health-related research

not enough is for application research

the balance is shifting slowly
should there be a Nobel Prize for
applied research?
 Step 5. Making better
clinical decisions

Barrier Solutions
not having the Computerized decision
right information at support
the right time
Effects of Computerized
Clinical Decision Support Systems
on Practitioner Performance and
Patient Outcomes
A Systematic Review

Amit Garg MD, Neill Adhikari MD, Heather McDonald MSc,

Patricia Rosas-Arellano MD,PhD, Phillip J. Devereaux MD, , Joseph Beyene ,
PhD

Justina Sam, R. Brian Haynes MD,PhD

DepartmentsofClinicalEpidemiologyandBiostatistics,McMasterUniversity
DepartmentsofMedicine,McMasterUniversity,UniversityofToronto,and
UniversityofWesternOntario
DepartmentofBiostatisticsandEpidemiology,UniversityofWesternOntario

Ref: Garg et al. Effects of computerized clinical decision support systems on

practitioner performance and patient outcomes: a systematic review. JAMA
2005;293:1323-38.
Context Computerized Clinical Decision Support
Systems

Software designed to directly aid in clinical decision making in

which characteristics of individual patients are matched to a
computerized knowledge base for the purpose of generating patient
specific assessments or recommendations.
Rules /
INPUT
Algorithms
Patient characteristics
Automated through EMR
By extra research staff
By existing health care staff
Computer
By the patient
By the practitioner

OUTPUT
Recommendations
delivered to health
Outcomes care provider
Directly by computer
Provider performance
integrate into By pager
Patient outcomes
workflow By extra research staff
By existing health care staff
 Are CDSSs
clinically effective?
Did CDSS improve practitioner
performance?
100 studies
counting positive results on 50% outcomes measured

Examined in 97 studies,
63 cited improvement (65%)

In 16 of 21 (76%) reminder systems

In 24 of 37 (65%) disease management systems

In 19 of 29 (66%) drug dosing or prescribing systems
In 4 of 10 (38%) diagnostic systems
Did CDSS improve patient
outcome?
Update 100 studies

Examined in 52 studies,
7 cited improvement (13%)

most had inadequate power to detect important difference

none proven to improve definitive outcome such as mortality

surrogate outcomes such as BP and HbA1C not meaningfully

improved in most studies
Reminder Systems
40 studies Improved Improved
Practitioner Patient
Performance Outcome
- 76% - - 0% -

Screening, counseling, vaccination, testing, medication use, or

the identification of at-risk behaviors

CDSS successes were typically demonstrated in ambulatory care,

although one successful system was used in hospitalized patients
 Disease Management Systems
37 studies Improved Improved
Practitioner Patient
Performance Outcome
- 62% - - 19% -

Most are RECOMMENDATIONS.

Range of problems, for example:

- diabetes care
- cardiovascular prevention
- incontinence in the elderly
- advanced directives
- ventilator support
- infertility
- corollary orders
- reduce unneeded health care utilization
 Step 5. Improving health care
decisions

Barrier Solutions
low patient adoption of effective
adherence to strategies to assist
treatments patients to follow
evidence-based health care
 The weakest links
Policy - especially at the local level
Coordination - 4P
Helping practitioners to recommend
effective treatments
Helping patients to follow effective
treatments
 The strongest link

Organization of health care

knowledge according to the
hierarchy of evidence (evidence-
based medicine)
The evolution of Evidence-Based Examples
information systems
Computerized decision
support
Systems

Evidence-based textbooks
Summaries

Evidence-based journal
Synopses abstracts

Systematic reviews
Syntheses

Studies
Original journal articles
KNOWLEDGE IS THE ENEMY OF
DISEASE

The application of what we know

will have a bigger impact on health
and disease than any single drug or
technology likely to be introduced
in the next decade.
Sir Muir Gray, UK National Library for Health