EVIDENCE-BASED MEDICINE

USydMP HANDBOOK

Compiled by: Evidence-Based Medicine Resource Group University of Sydney Medical Program (5th Ed.)

Contents

Evidence-based Medicine Resource Group Objectives......................................................... 2 Checklist of Evidence-based Medicine Skills......................................................................... 5 Evidence-based Medicine Course Overview ......................................................................... 7
What is EBM and why do we need it? ..................................................................................................................... 7 How EBM is taught in the USydMP.......................................................................................................................... 8 Assessment of EBM................................................................................................................................................ 11

Evidence-based Medicine Glossary .................................................................................... 13 A humorous look at EBM .................................................................................................... 18 Asking the right questions and literature searching ............................................................. 19 Study types ......................................................................................................................... 30 Causality............................................................................................................................. 33 Frequency and Prognosis ................................................................................................... 35 Intervention studies............................................................................................................. 39 Diagnostic tests .................................................................................................................. 43 Meta-analysis and systematic reviews ................................................................................ 49 Additional resources ........................................................................................................... 51

Evidence-based Medicine Resource Group Objectives

The teaching objectives of the EBM Resource Group focus on the identification, critical appraisal and application of evidence from clinical epidemiological research. Students should have covered each objective at the end of the year indicated in brackets. Students should find opportunities to revisit each objective in subsequent EBM theme sessions and PBL discussions in years one and two, and to reinforce and apply these objectives in the clinical years three and four.

Overall Goals of Evidence-Based Medicine

Our aim is that graduates make decisions about health problems on the basis of the best available evidence. As part of their role in health care, graduates should have the willingness and ability to: 1. Define a problem in a way that can be addressed by research evidence in the form of epidemiological data. (Year One) 2. Identify and critically appraise research evidence. (Years One and Two) 3. Integrate this evidence with the details and preferences of individual patients in the form of a management plan and then determine how the success of the plan will be assessed. (Years Three and Four) 4. Interpret information on the benefits and costs of health interventions. (Year Four)

A. Critical Appraisal of the Evidence To achieve the above aims, students will need to be able to critically appraise the literature. This will include primary research studies, review articles, including systematic reviews and meta-analyses (Years One and Two) and clinical practice or management guidelines.

By the time of graduation, students should be able to critically appraise the literature on the following: interventions for treatment and prevention of health problems diagnostic tests screening programs causes of health problems (aetiology) natural history of health problems (prognosis) prevalence and incidence of health problems

And be familiar with some introductory principles of decision analysis and economic evaluations.

B. Skills Required For Critical Appraisal The following skills are needed for critical appraisal: a) Conduct computerised literature searches to identify relevant high quality research evidence and information from Medline, other data bases (eg. Cochrane Library) and Internet sites. b) Identify the following study types and be able to explain their strengths and weaknesses: randomised controlled trials, cohort studies, case-control studies (population and hospital-based), cross-sectional analytic and other descriptive studies. c) Identify major sources of bias in medical research: selection bias, confounding, measurement bias, lead time bias and length bias (studies of screening). d) Calculate and interpret the following measures of disease frequency and effect: prevalence, incidence, relative risk, absolute risk reduction (risk difference), number needed to treat and odds ratio (interpret only). e) Interpret confidence intervals and P values. f) Explain the difference between statistical significance and clinical significance.

g) Understand and apply criteria for establishing causality in the study of health problems. h) Understand the importance of pre-test probability for the interpretation of test results. i) Calculate and interpret the following features of diagnostic tests: sensitivity, specificity, likelihood ratios, post-test probability. j) For economic evaluations, identify the study question and the viewpoint from which costs and outcomes have been measured. opportunity cost, the margin and discounting. Note: By interpret we mean understand and be able to explain in your own words. The critical appraisal enabling skills are concerned with understanding research and using results. Only those students interested in actually doing research will need to be able to perform tests. Explain what is meant by

C. Application of Evidence to Decision Making By graduation students should be able to interpret the results from clinical and population research and decide how to apply these results to individuals or groups of people. This is introduced in years one and two as results from critically appraised papers are considered in relation to the PBL problem of the week, and will be reinforced in the clinical years.

Checklist of Evidence-based Medicine Skills

Critical appraisal of literature Interventions for treatment and prevention of health problems Diagnostic tests Screening programs Causes of health problems (aetiology) Natural history of health problems (prognosis) Prevalence and incidence of health problems

Comments

Skill 1. Framing appropriate clinical questions 2. Literature searching Medline Cochrane Library Internet sites 3. Identify the following study types and explain their strengths and weaknesses Randomised controlled trials Cohort studies Case-control studies (population and hospital-based) Cross-sectional (analytic and descriptive)

4. Calculate and interpret measures of disease frequency Incidence Prevalence 5. Calculate and interpret measures of treatment effect Relative risk Absolute risk reduction (risk difference) Odds ratio (interpret only) Number needed to treat 6. Interpret confidence intervals and P-values 7. Explain the difference between statistical significance and clinical significance 8. Understand and apply criteria for establishing causality in the study of health problems 9. Understand the importance of pre-test probability for the interpretation of test results 10. Calculate and interpret the following features of diagnostic tests: Sensitivity Specificity Likelihood ratios Post-test probability 11. For economic evaluations, identify the study question and the viewpoint from which costs and outcomes have been measured. Explain what is meant by: Opportunity cost Margin Discounting

Evidence-based Medicine Course Overview

What is EBM and why do we need it?

Evidence-based Medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.1 By best evidence we mean patient-centred clinical research into (for example) the accuracy and precision of diagnostic tests (including the clinical examination), the power of prognostic markers and the efficacy and safety of therapeutic and preventive interventions. The practice of EBM entails a process of life-long self-directed learning in which caring for patients creates the need for clinically important information and we then Convert these information needs into answerable questions Track down efficiently the best evidence with which to answer them Critically appraise that evidence for its validity (closeness to truth) and usefulness Apply the results to patient care.

EBM is new and rapidly growing. There are many reasons why EBM has developed now. These include: The explosion of medical information. There are over 2 million biomedical papers published every year -- textbooks can become out-of-date before they are available in bookstores. Patients have much greater access to information (eg on the Internet) and patients are becoming increasingly aware of their rights and responsibilities as consumers of health services.

7 Sackett DL et al. Evidence-based medicine: what it is and what it isnt. BMJ 1996;312:71-72.

How EBM is taught in the USydMP

Evidence-based Medicine is part of the Personal and Professional Development Theme. It is taught in all four years of the USydMP. In Years 1 and 2, students acquire basic skills in EBM (see EBM Objectives). In Years 3 and 4, these skills are refined and applied in clinical practice. In Years 1 and 2, EBM is part of the integrated learning in PBL tutorials. Students are progressively encouraged to consider the kinds of questions patients might ask of their health care provider in each case, to search and to critically appraise the research literature for evidence on which to base answers to these questions. Students are particularly referred to the Cochrane Database of Systematic Reviews and Best Evidence. In general, EBM encourages students to think critically about their learning and to consider the evidence base for clinical decisions. EBM teaching in Years 1 and 2 is supplemented by EBM theme sessions. There are two EBM theme sessions in each block. The first session of each pair is a large group session in which there will be a presentation of EBM concepts and skills. The second session of each pair is a small group session in which students practice the skills demonstrated in the previous session. The small group sessions are facilitated by tutors with high-level EBM skills and students are expected to spend some time in preparation before the session. Over time, there is an increasing expectation on students to think of relevant clinical questions, to search the literature, and bring worthy papers for appraisal and discussion in their small group sessions. In between the formal EBM theme sessions, skills in EBM are reinforced and consolidated by participation in EBM online activities. There are one to three EBM activities in each block, each of which is linked to a specific PBL case. The activities are self-contained, computer based activities and often involve framing searchable clinical questions followed by a computerized literature search for 8

relevant articles. Participation in the activities is prompted by PBL tutors.

EBM Theme Sessions Years 1 and 2

Block 1 Block 2 Block 3 Block 4 Block 5 Block 6 Block 7 Block 8 Asking Questions I Asking Questions II Study Designs I Study Designs II Causality I Causality II Clinical Schools Frequency and Prognosis I Frequency and Prognosis II Intervention Studies I Intervention Studies II Diagnostic Tests I Diagnostic Tests II Meta-analysis and Systematic Reviews I Meta-analysis and Systematic Reviews II Clinical Schools Large Group Session Small Group Tutorial* Large Group Session Small Group Tutorial* Large Group Session Small Group Tutorial* Large Group Session Small Group Tutorial* Large Group Session Online Tutorial Large Group Session Small Group Tutorial* Large Group Session Small Group Tutorial*

Block 9

*Indicates that student preparation is required for the session.

In years 3 and 4, the EBM curriculum is integrated with the clinical attachments that make up the medical program. The aim is for students to apply the results of clinical research to the problems of individuals. Each clinical attachment has its own integrated EBM activity tailored to its characteristics and timing. The EBM activities are supported by clinicians with high-level EBM and content expertise. The clinical attachments and EBM activities are summarized in the tables below. Details of each activity are available on the GMP website.
Attachment
Integrated Clinical Attachments (ICA) Psychological Medicine (PM) Children & Adolescents Health (CA) Perinatal & Womens Health (PW) Community Practice (Comm) Pre-Internship (PRINT)

Activity

Aim

Assessment
Summative rating of presentation + report on audience feedback Participation. Required formative assessment Participation. Integration in exams. Participation Integration in exams.

Student PEARLS To experience using the tools of EBM to answer real clinical questions Journal Club To enhance wider understanding of the evidence on which the practice of psychological medicine is based. To incorporate high quality evidence from guidelines and systematic reviews in management discussions with patients and their families. To integrate and apply information from different sources into clinical decision-making.

Integration in PBL Integration in PBL Community PEARLS - case commentaries Pre-Intern PEARLS

To apply research findings from groups to the Summative rating of problems of an individual. 1000 word report To apply the principles of EBM within the constraints of routine clinical practice Patient rating of discussion. Required formative assessment.

Student PEARLS is the model used in the integrated clinical attachments of GMP3. Students prepare and give a short, evidence-based presentation addressing a focused clinical question raised by contact with a specific patient seen during their integrated clinical attachments. Preparation of the presentation is facilitated by a series of 3 tutorials that the students attend in the preceding 3 weeks supported by a clinician with expertise in EBM. Each PEARLS presentation lasts 15 minutes with an additional 15 minutes for questions and feedback.

10

A similar model is used in year 4, except that students develop their presentations more independently. PEARLS in Psychological Medicine uses a journal club format. PEARLS in the Community Practice Rotation and PreInternship focus on explaining evidence from research to patients. EBM in Children & Adolescents Health and Perinatal & Womens Health focuses on integrating evidence in decision making.

Specific objectives and their corresponding clinical attachments

(see appropriate column)
Students will demonstrate their ability to ICA Rotating Practice Placements Comm CA PW PM PRIN T

1. Translate real people's problems into answerable clinical questions 2. Find, appraise, and interpret pertinent clinical research 3. Discuss the generalisability of research to clinical practice 4. Discuss the applicability of research to an individual 5. Discuss evidence with patients, relatives, doctors, Dr Pt Rel All Pub Pt the public 6. Consider the preferences of patients, relatives, Dr Pt Rel All Pub Pt doctors & the community 7. Integrate clinical data, evidence and preferences in decision making 8. Appraise and use systematic reviews and evidence based guidelines 9. Cope with clinical time constraints 10. Appraise and use imperfect evidence 11. Dealing with conflicts between evidence and prejudice (hash - all attachments; black - that attachment)

Assessment of EBM
EBM is a component of Personal and Professional Development. It is necessary to achieve satisfactory performance in EBM in order to achieve satisfactory performance in PPD and thus to be able to progress from Years 2 to 3, Years 3 to 4 and to complete Year 4.

11

In Years 1 and 2, EBM is assessed summatively in a critical appraisal exercise which is integrated into the year 2 summative assessment. Critical appraisal exercises are also provided in the formative assessments. In addition there will be EBM multiple choice questions (SBAs) in the block formative assessments and in the summative assessment. EBM questions are also included in the online voluntary formative assessments. In Year 3, the Student PEARLS presentation is assessed summatively by EBMedicos and formatively by fellow students. In Year 4, there are discrete required formative assessments in Psychological Medicine (PEARLS Presentation) and the Pre-Internship (written report); and a discrete summative assessment in Community Practice (written report). EBM assessment is integrated in the summative assessments at the end of the rotations in Children and Adolescent Health, and in Perinatal and Womens Health. Please see reference: Sackett DL, Rosenberg WM, Gray J A M Haynes R B, Richardson W S. Evidence based medicine: what it is and what it isnt. BMJ 1996;312:71-72.

12

Evidence-based Medicine Glossary

Absolute Risk Reduction (ARR) is the difference in the event rate between control group (CER) and treated group (EER): ARR = CER EER Case control study involves identifying patients who have the outcome of interest (cases) and control patients without the same outcome, and looking back to see if they had the exposure of interest. Case-series is a report on a series of patients with an outcome of interest. No control group is involved. CER Control Event Rate: see Event Rate Clinical Practice Guideline is a systematically developed statement designed to assist practitioner and patient make decisions about appropriate health care for specific clinical circumstances. Cohort Study involves identification of two groups (cohorts) of patients, one of which did receive the exposure of interest, and one of which did not, and following these cohorts forward for the outcome of interest. Cost-Benefit Analysis converts effects into the same monetary terms as the costs and compares them. Cost-Effectiveness Analysis converts effects into health terms and describes the costs for some additional health gain (e.g. cost per additional MI prevented).

13

Cost-Utility Analysis converts effects into personal preferences (or utilities) and describes how much it costs for some additional quality gain (e.g. cost per additional quality-adjusted life-year, or QALY). Crossover Study Design is the administration of two or more experimental therapies one after the other in a specified or random order to the same group of patients. Cross-Sectional Study the observation of a defined population at a single point in time or time interval. Exposure and outcome are determined simultaneously. Decision Analysis is the application of explicit, quantitative methods to analyse decisions under conditions of uncertainty. Ecological Survey is based on aggregated data for some population as it exists at some point or points in time; to investigate the relationship of an exposure to a known or presumed risk factor for a specified outcome. EER Experimental Event Rate: see Event Rate. Event Rate is the proportion of patients in a group in whom an event is observed. Thus, if out of 100 patients, the event is observed in 27, the event rate is 0.27. Control Event Rate (CER) and Experimental Event Rate (EER) are used to refer to this in control and experimental groups of patients respectively. Evidence-Based Health Care extends the application of the principles of Evidence-Based Medicine (see below) to all professions associated with health care, including purchasing and management.

14

Evidence-Based Medicine is the conscientious, explicit and judicious use of current best evidence in integrating individual clinical expertise with the best available external clinical evidence from systematic research. See also the article on EBM: What it is and what it isnt. Likelihood Ratio is the likelihood of a given test result in a patient with the target disorder compared to the likelihood of the same result in a patient without that disorder. Meta-analysis is an overview which uses quantitative methods to summarise the results. N-of-1 Trials The patient undergoes pairs of treatment periods organised so that one period involves the use of the experimental treatment and one period involves the use of an alternate or placebo therapy. The patients and physician are blinded, if possible, and outcomes are monitored. Treatment periods are replicated until the clinician and patient are convinced that the treatments are definitely different or definitely not different. Negative Predictive Value (-PV) is the proportion of people with a negative test who are free of disease. See also SpPins and SnNouts. Number Needed to Treat (NNT) is the number of patients who need to be treated to prevent one bad outcome. It is the inverse of the ARR: NNT = 1/ARR. Odds are a ratio of events to non-events, e.g., if the event rate for a disease is 0.2 (20%), its non-event ratio is 0.8 (80%), then its odds are 0.2./0.8 = 0.25 (see also Odds Ratio). Odds Ratio is the odds of an experimental patient suffering an event relative to the odds of a control patient.

15

Overview is a systematic review and summary of the medical literature. Positive Predictive Value (+PV) is the proportion of people with a positive test who have disease. Also called the post-test probability of disease after a positive test. See also SpPins and SnNouts. Randomised Controlled Clinical Trial a group of patients is randomised into an experimental group and a control group. These groups are followed up for the variables/outcomes of interest. Relative Risk Reduction (RRR) is the percent reduction in events in the treated group event rate (EER) compared to the control group event rate (CER): RRR = (CER EER)/CER*100 Risk Ratio is the ratio of risk in the treated group (EER) to the risk in the control group (CER): RR = EER/CER. RR is used in randomised trials and cohort studies. Sensitivity is the proportion of people with disease who have a positive test. See also SpPins and SnNouts. SnNout when a sign/test has a high sensitivity, a negative result rules out the diagnosis; e.g. the sensitivity of a history of ankle swelling for diagnosing ascites is 92 per cent, therefore if a person does not have a history of ankle swelling, it is highly unlikely that the person has ascites. Specificity is the proportion of people free of a disease who have a negative test.

16

SpPin when a sign/test has a high specificity, a positive result rules in the diagnosis; e.g. the specificity of fluid wave for diagnosing ascites is 92 per cent. Therefore, if a person has a fluid wave, it is highly likely that the person has ascites.
Glossary taken from: http://www.cebm.net/glossary.asp

17

A humorous look at EBM

Please see references: Down End Research Group. Polythenia gravis: the downside of evidence based medicine. BMJ 1995:311;1666-8. Isaacs D, Fitzgerald D. Seven alternatives to evidence based medicine. BMJ 1999;319:1618.

18

Asking the right questions and literature searching

Objectives i. Define a problem in a way that can be addressed by research evidence in the form of epidemiological data ii. Describe the types of questions patients and doctors ask: background versus foreground, diagnostic, frequency, aetiology, prognosis, and intervention questions iii. Use PICO to develop questions: problem/population+intervention/exposure+comparator+outcome iv. Appreciate the importance of asking questions in terms of PICO as it is patient-centred and focuses the learning need. The question type will suggest the form and (study designs) answers take, and shapes the literature searching strategies.

The objectives of this unit are to introduce students to formulating clinical questions and the principles of searching computerised databases for valid and applicable information that will assist them in their clinical decision making. This section includes information about journal searching and we start with a reminder that there are other resources that already summarise the results of such searches, incorporating quality filters and critical appraisals. An excellent example is the Cochrane Library (for systematic reviews of interventions). An alternative approach is to limit searches to studies that have met strict EBM criteria by using the EBM reviews limit. And finally searches can also be done using "in-built" methodology filters in Medline, either the Ovid or Pubmed versions.

19

The aim of this material is not to turn you into expert literature searchers but to introduce you to the principles of searching and provide a few lists of useful terms that you can apply to any search you may need to do in the future. When trying to find a street in a street directory you wouldnt think of opening the pages and hoping you will find the right street, you would use the index and key. Finding the right journal article is the same. Think of this material as your guide to the medical literature equivalent of the street directory.

Steps of an EBM Literature Search 1. Decide on the question type: Intervention/therapy, diagnosis, aetiology/causality, prognosis. 2. Frame the question in PICO format: 3. Decide on the best feasible study type to answer the question 4. Select the most appropriate database/search method

Formulating Clinical Questions using PICO Patient population /clinial problem Intervention (study factor) /exposure Comparator (intervention or study factor) Outcome Who is the question about? Which clinical problem? What is being done to the patient? Is an exposure the cause of a problem? What is being compared? What outcomes are we trying to achieve?

Why is using PICO important? Frames clinical questions into a more specific question that is searchable using online databases They are patient centred, help communication and focus learning needs 20

Literature Searching Strategies Important note: When searching for articles on treatment (intervention) start at Search A, however when searching for articles on prognosis, aetiology or diagnosis start at Search B. This is because at the present time the Cochrane Database of Systematic Reviews only includes systematic reviews of interventions. Advanced search: The new Ovid gives you a choice of doing a basic search or an advanced search. For the purposes of EBM searching it is more efficient to use the advanced search feature which searches using subject headings. You will find that when you use the basic search feature you will be given a large number of unnecessary articles that can be time consuming to go through! Demonstration search topic: Is Glucosamine sulphate an effective agent in the short-term treatment of osteoarthritis? Structure of clinical question: Population/Disease Intervention (Study factor) Comparator (Often the control) Outcome (Outcome factor) = People with osteoarthritis = Glucosamine sulphate = Placebo = e.g pain, function

N.B. The words in bold in the above PICO format are the ones which you will use to do your search. Note that most commonly only the population and intervention are used to search and there is no need to put in terms such as placebo or any outcome factor. This is because if you put in too many terms it will limit your search too much.

21

Types of EBM searches (in order of preference) Search A Search in EBM Reviews: the Cochrane Database of Systematic Reviews (CDSR) (Ovid version). CDSR is a full text database of the regularly updated systematic reviews of the effects of healthcare prepared by the Cochrane Collaboration. Step by step instructions: 1. On the opening page, click on the tab Advanced Search. 2. Enter search term osteoarthritis. 3. Enter search term glucosamine. 4. Click on search history to see these two searches. 5. Combine these searches by ticking the boxes next to them and then clicking AND. 6. Click on limits to show you what limits are available. Tick the Systematic Reviews limit (this will remove any protocols for studies yet to be done from the list) and then click search again. Please note that you will find the full-text systematic review by clicking on the EBM Topic Review button. Search B If you dont find any systematic reviews in Cochrane (also remember Cochrane only has systematic reviews of intervention studies), you can look for a systematic review in Medline by doing the following: Step by step instructions. 1. Change database to Medline 1950-. Choose open and clear so that you do not re-execute the previous search. 2. Again click on the Advanced Search tab.

22

3. Enter 1st term Osteoarthritis (population/disease). Medline will map your term to a MeSH term (Medical Subject Heading). 4. Check the explode box to the right of osteoarthritis to widen your search then click continue. 5. On the next page, do not choose any subheadings and click continue. 6. Enter 2nd term Glucosamine Sulphate (treatment/intervention). Once again Medline will map your term to an appropriate MeSH term. 7. Check the explode box to the right of Glucosamine and click continue. 8. Do not choose any subheadings. Click on continue. 9. Combine searches 1 and 2 by ticking the boxes next to them and clicking AND. 10. Click on limits and then additional limits. Scroll down to publication types and select meta analysis. 11. Click on limit a search. An alternative to the last two steps is to type in meta analysis.pt. This will select all articles referenced as a meta analysis in Medline. pt stands for publication type. You can then combine this with your other results by using AND. Search C If search B did not produce useful results, move on to search C in which you limit your search with a Medline Clinical Queries filter. Filter terms that limit the search differ for different clinical questions i.e. diagnosis, aetiology, therapy, or prognosis. Emphasis can also be placed on specificity, sensitivity, or optimization. A specific filter will retrieve most relevant articles but will probably omit a few. A sensitive filter will retrieve mostly relevant articles but will probably include some less relevant ones as well.

23

An optimized filter is a trade-off between the above two. The question we are searching is one about treatment so use the therapy filters. Use the most specific filter first and if this does not retrieve useful results try the optimized and then finally the sensitive filter. Step by step instructions: 1. Steps 1-9 as above, then click Additional Limits. 2. Select the search that you need to limit (it should be the one in your search history that combines osteoarthritis and glucosamine). 3. Scroll down to the Clinical Queries limits. 4. Choose the appropriate filter by highlighting the filter name, i.e. Therapy <specificity>. 5. Click Limit a search. 6. The results of the limit will appear in the search history box. Evaluate the results of the search. If you would like to broaden the results try applying the optimized or sensitive versions of the therapy filter. Quick search Perform a comprehensive search of Medline (Ovid version) using subject headings then apply the EBM Reviews limit. This is a quick search that finds good evidence to answer questions, but may not always turn up results. A limit to EBM Reviews will restrict your retrieval to those articles or studies that have been included by the Cochrane Collaboration when creating a Topic Review, articles that have been reviewed in the ACP Journal Club or in the Database of Abstracts of Reviews of Effectiveness (DARE). By their inclusion in these publications, these studies have met strict Evidence-Based Medicine criteria. Use this limit to narrow a large search to only those articles or studies which are considered Evidence-Based by experts.

24

Step by step instructions. 1. Steps 1-9 as above. 2. Click on limits, choose the EBM Reviews limit and click search. PubMed Search This search is an easy alternative to Search C, and is useful if you are off campus and dont have access to the library databases. PubMed is the US National Library of Medicines version of Medline and is available free of charge worldwide. Search PubMed Clinical Queries using keywords. The software automatically combines search terms with AND. You do not need to type Boolean connectors unless you wish to use the OR connector. Step by step instructions: 1. Select PubMed from the Databases list or go to http://www.ncbi.nlm.nih.gov/entrez 2. Click on Clinical Queries (on the blue sidebar). 3. Under Search by Clinical Study Category select Therapy and narrow, specific. 4. Enter keywords osteoarthritis glucosamine sulphate. 5. Click on Go.

The two tables over the page show you the search strategies that OVID Medline uses in its clinical queries limits.

25

Search strategies used for OVID Medline Clinical Queries limits

Question type Best study type Optimised search Most specific search More sensitive search

Therapy Randomised controlled trial randomized controlled trial.pt OR randomized.mp OR placebo.mp

randomized controlled trial.pt OR randomized controlled trial.mp

clinical trial.mp OR clinical trial.pt OR random$.mp OR tu.xs

Diagnosis

Cross-Sectional Analytic

sensitiv$.mp OR predictive value$.mp OR accurac$.tw specificity.tw

sensitiv$.mp OR diagnos$.mp OR di.fs

.xs pre exploded subject heading - Sometimes multiple MeSH Subheadings can be logically grouped together. Such related groups of subheadings are "pre-exploded" under one broad subheading. This means that all the grouped subheadings can be retrieved together by searching on the "pre-exploded" version of the broad subheading. .fs floating subheading these are searches on subheadings, but they are not attached to a MeSH term.

26

Question type

Best study type

Optimised search

Most specific search

More sensitive search

Aetiology Cohort study Case control study risk.mp OR mortality.mp OR cohort.tw Prognosis/ OR diagnosed.tw OR Cohort$.mp OR predictor$.tw OR death.tw OR exp models, statistical

relative risk$.tw risk$.mp OR risks.tw OR cohort stud$.mp OR exp cohort studies OR between group$.tw Incidence/ OR exp mortality OR follow-up studies/ OR Mortality/ OR Prognos$.tw OR predict$.tw OR Course$.tw

Prognosis

Cohort study

prognos$.tw OR first episode.tw OR cohort.tw

Filter terms on this table are taken from OVID Medline Clinical Queries and these are based on the work of R. Brian Haynes MD, PhD et al. of the Health Information Research Unit (HIRU) at McMaster University. http://www.ovid.com/site/products/ovidguide/haynes.htm

27

Bibliography Haynes RB. Wilczynski NL. Optimal search strategies for retrieving scientifically strong studies of diagnosis from Medline: analytical survey. BMJ 2004: 328(7447):1040. Available from: http://www.bmj.com/cgi/content/full/328/7447/1040 Haynes RB. McKibbon KA. for the Hedges Team. Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytical survey. BMJ 2005: 330(7501):1179. Available from: http://www.bmj.com/cgi/content/full/330/7501/1179 Wilczynski NL. Haynes RB for the Hedges Team. Developing optimal search strategies for detecting clinically sound prognostic studies in MEDLINE: an analytic survey. BMC Med 2004; (1):23. Available from: http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=441418. Wilczynski NL, Haynes RB for the Hedges Team. Developing optimal search strategies for detecting clinically sound causation studies in MEDLINE. Proc AMIA Annu Symp 2003:719-23. Extra Information Medline classes: For students who do not feel confident searching Medline the Medical Library offers hands-on tutorials. For more information and to register go to http://www.library.usyd.edu.au/subjects/medicine/medIlitregistration.html Online Tutorials: Medline (Ovid version) tutorial:

http://www.library.usyd.edu.au/subjects/medicine/tutorials/medlinetut/index.html EBM Literature Searching Guide:

http://www.library.usyd.edu.au/subjects/medicine/tutorials/ebmtut/index.html

28

Free on the Web: PubMed

http://www.ncbi.nlm.nih.gov/entrez Cochrane EBM Reviews databases

http://www.thecochranelibrary.com/

Link to other useful EBM sites: http://www.library.usyd.edu.au/subjects/medicine/links/ebp.html Centre for Evidence-Based Medicine Netting the evidence (provides links to many useful websites) PICO Tutorial

http://www.cebm.net/searching.asp http://www.shef.ac.uk/scharr/ir/netting/ http://www.cwml-tutorials.blogspot.com/2008/01/formulating-your-questionsusing-pico.html

Many thanks to Dianne van Sommers, Jeremy Cullis and Kathy Thornton for their contribution to this chapter.

Please see references: Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence-based decisions. [editorial] ACP Journal Club 1995;123:A-12. Oxman AD, Sackett DL, Guyatt GH. Users guides to the medical literature. I. How to get started. JAMA 1993;270:2093-2095.

29

Study types

Objectives i. Identify the following study types and be able to explain their strengths and weaknesses: randomised controlled trials, cohort studies, case-control studies (population and hospital-based), cross-sectional analytic and other descriptive studies
ii. Identify major sources of bias in medical research: selection bias, confounding,

measurement bias RANDOMISED CONTROLLED TRIAL

START HERE

R andom isation Exposed to Study Factor

OUTCOME FACTOR

STUDY FACTOR
N ot exposed to study factor

OUTCOME FACTOR

COHORT STUDY
START HERE

N o R andom isation Exposed to Study Factor

OUTCOME FACTOR

STUDY FACTOR
N ot exposed to study factor

OUTCOME FACTOR

CASE-CONTROL STUDY
START HERE

Exposure to STUDY FACTOR

C ase

Yes

OUTCOME FACTOR Exposure to STUDY FACTOR

C ontrol
No

30

Steps for determining study type

1. Identify the study question (ie the aim of the study) 2. Identify clinical question type (eg therapy, diagnosis, prognosis, causality, frequency) 3. Identify study factor 4. Identify outcome factor 5. Did the study begin with the study factor? Was exposure to the study factor randomised? IF YES-Randomised controlled trial IF NO-Cohort study 6. Did the study begin with the outcome factor? IF YES- Case-Control study 7. Were the study factor and outcome factor measured simultaneously Was an attempt made to find an association between study factor and outcome factor? IF YES-Cross-sectional analytic IF NO-Cross-sectional descriptive (Prevalence only) 8. Was it comparing a diagnostic test to a gold standard? IF YES-Cross-sectional analytic (Diagnostic) 9. Was the study type appropriate/best for the question type?

31

Study Types Summary

Study Type RCT Clinical Question *Therapy/ Intervention *Harm/aetiology (? unethical) Evidence Closest to scientific experiment STRONGEST EVIDENCE Possible sources of BIAS (YEAR 2) *No allocation concealment *Loss to follow up *No analysis by intention to treat *Non-blind assessment of outcome Cohort study *Therapy/ Intervention *Harm/aetiology *Prognosis Case-control study *Therapy/ Intervention *Harm/ aetiology Observational Observational *Loss to follow up *Non-blind assessment of outcome factor *Confounding *Recall bias *confounding *low response rates *non-blind assessment of study factor *controls not chosen from the same population as the cases Crosssectional Analytic (Diagnostic) Diagnosis Best study to assess diagnostic test accuracy (YEAR 2) *non-independent blind comparison with reference std *Verification bias *inappropriate spectrum of disease *Compares a diagnostic test to a gold standard *Begins with outcome factor *Begins with study factor-exposure non-randomised Time *Begins with study factor-exposure randomised

Crosssectional Analytic

*Therapy/ intervention *Harm/Aetiology

Association/ Hypothesis generating

*Selection of participants nonrandom (volunteers) *confounding *low response rates *Small sample size

*Study and outcome factors measured at one time point

32

Causality
Objectives i. Understand the difference between a cohort and a case-control study

ii. Interpret odds ratios iii. Understand and apply criteria for establishing causality in the study of health problems Odds ratio An odds ratio is the odds of a person with the disease being exposed divided by the odds of a control (person without the disease) being exposed. The odds ratio is usually a good approximation of the relative risk. ie OR=RR Interpreting odds ratios and relative risks For questions of harm where the outcome is disease or death, an RR>1 means that the risk factor is harmful and an RR<1 means that the risk factor is protective. For example, if a study looking at the risk of heart attack in people with diabetes has an RR of 3.0, this would mean that diabetics are 3 times more likely to have a heart attack than non-diabetics. Similarly if the RR for a low fat diet is 0.8, then this would mean that those consuming a low fat diet are 0.8 times or 20% less likely to have a heart attack.

Bradford Hill criteria for causality Strength of association Consistency across all studies Temporality of exposure Dose response Biological plausibility Analogy Reversibility Experiment Coherence of evidence

33

Please see reference: Levine M, Walter S, Lee H, Haines T, Holbrook A, Moyer V. Users guides to the medical literature. IV. How to use an article about harm. JAMA 1994;271:1615-1619.

34

Frequency and Prognosis

Objectives i. Interpret confidence intervals and P values significance iii. Critically appraise cross-sectional and cohort studies Incidence No. of persons experiencing a new event during a time period/No. of persons at risk at beginning of time period. Therefore, incidence can only be calculated by following a group of people over time (i.e. in an RCT or cohort study).

ii. Understand the difference between statistical significance and clinical

Prevalence Total no. of persons with attribute at a given time/Total population at a given time. This can be assessed by a cross-sectional study but to be representative the study participants need to be selected at random (not volunteers). Cohort studies on prognosis do not need a control group as the study follows diseased people over time to determine their outcome.

P Value A P value refers to the statistical significance of the results. Another way of thinking of it is that it refers to the probability that the observed results are due to chance rather than an actual effect. The role of chance in the results is assessed by what is known as the null hypothesis. This is the hypothesis that there is no difference between the two groups. The statistical test looks at the probability of getting the observed result (or one even more extreme) by chance, if the truth is that the two groups are actually

35

the same. For example, if we do a study looking at drug A compared to placebo, the null hypothesis would be that there is no difference between drug A and placebo. If our observed result was a RR=0.5 (i.e drug A decreases the chance of disease/death by half) then the statistical test would look at how likely it would be to get this result by pure chance. In most studies a result is considered to be not due to chance or statistically significant if the P value is less than 5% (p<0.05) i.e. that there is only a 5% probability that the results observed in the study are due to chance. *Remember that a cut-off of 5% is arbitrary and the difference between 6% (p=0.06) which would be considered not statistically significant, and 4% (p=0.04), which would be considered significant, is in fact quite small. There is a further explanation of P values with examples in the attached paper Primer on Statistical Significance and P Values.

95% Confidence Intervals The confidence interval refers to the range in which the true effect is likely to be. Studies will usually give a point estimate (e.g. RR=1.10) and then a 95% confidence interval (e.g. 0.46-1.74). The relative risk of 1.10 is only an estimate of the true underlying relative risk. The confidence interval means that we are 95% confident that the true effect will be somewhere between a relative risk of 0.46 (i.e. a 54% decreased risk) and a relative risk of 1.74 (i.e. a 74% increased risk). NB: A 95% confidence interval (0.46-1.74) does not refer to 95% of the population. It does not mean that 95% of people have a RR in the range 0.46-1.74, nor does it mean that 95% of people have a RR of 1.10. Confidence intervals refer to the precision of the estimate If the confidence interval is narrow, it means that your estimate is precise. If the confidence interval is wide, it means that your estimate is not precise.

36

How are 95% CIs and P values related? If a confidence interval includes no difference then it means that the null hypothesis (i.e. that there is no real difference between groups) is possible. As the P value is a measure of the null hypothesis being disproved, this would mean that the results would not be statistically significant when the CI includes no difference. NB: When looking at results such as a relative risk which is a ratio, an RR of 1 is equivalent to no difference between the two groups and therefore if the CI crosses 1 the results will not be statistically significant. However, if the results do not refer to a ratio but subtraction (e.g. the difference in mean weight loss between 2 groups), then a value of 0 would indicate no difference and hence a CI crossing 0 would not be statistically significant. What P values and CIs dont tell you These two statistical measures do not tell you how believable the result is. If the results come from a study that is heavily biased then even statistically significant results could not be relied upon. This is why it is important to assess a study according to the JAMA validity criteria before looking at the results. Statistical significance also does not tell you anything about whether the effect is important i.e. whether the results are clinically significant. For example, it is possible that a study looking at a new antihypertensive medication discovers a statistically significant effect but that the difference in mean BP between the treatment and control group is only 0.5mmHg which obviously is not clinically important. There is a further explanation of confidence intervals in the attached paper Primer on 95% Confidence Intervals.

37

Please see references: American College of Physicians-American Society of Internal Medicine. Primer on statistical significance and p values. Effective Clinical Practice 2001;4:183-184. American College of Physicians-American Society of Internal Medicine. Primer on 95% confidence intervals. Effective Clinical Practice 2001;4:229231. Laupacis A, Wells G, Richardson WS, Tugwell P. Users guides to the medical literature. V. How to use an article about prognosis. JAMA 1994;272:234-237.

38

Intervention studies

Objectives i. Calculate and interpret the following measures of disease frequency and effect: prevalence, incidence, relative risk, absolute risk reduction (risk difference), number needed to treat (NTT). ii. Discuss potential sources of bias in randomised controlled trials

Allocation Concealment Allocation concealment is a way of protecting the randomisation process from bias. If allocation is concealed it means that the people involved in the study (investigators, clinicians, patients) cannot identify which group the patient will be allocated to (i.e. treatment or control) during randomisation and can therefore in no way influence the group the patient will be put into. For example, a study might decide to allocate patients to groups using non-opaque envelopes. Clinicians may hold the envelope up to the light, see what the allocation is, decide that the allocation is not the best for the patient (e.g. they are very sick so should get the treatment not placebo) and then pick the next envelope. Obviously if this were to happen then the study would not be truly randomised. Allocation concealment differs from blinding in that blinding refers to lack of knowledge about which group patients are in after the randomisation process.

Analysis by intention to treat Analysing patient outcomes based on the group into which they were randomized regardless of whether they actually received the planned intervention. This analysis ensures randomization is preserved. In the diagram below, the 8 patients lost to follow up (5 in the treatment group, 3 in the control group) cannot be included in the analysis as they will have no available data. The 95 left in the treatment group should all be analysed in the treatment group

39

i.e. it would be wrong to only analyse the 87 that took the drug and/or shift the 8 who didnt take the drug into the control group.

200 Participants
Randomisation
Treatment Control

100

100

5 lost to follow up

3 lost to follow up

95
87 took the drug 8 didnt take the drug

97
95 took placebo 2 actually took the drug

Relative risk and relative risk reduction Neither of these measures take into account baseline risk. Drug companies most often quote these measures.

Absolute risk reduction and number needed to treat Both of these measures take into account a patients baseline risk. Varying a patients baseline risk will give you different Absolute risk reductions and different Numbers needed to treat.
Baseline risk (CER) 0.4 0.04 0.004 Risk in treated group (EER) 0.3 0.03 0.003 Relative risk (EER/CER) 0.75 0.75 0.75 Relative risk reduction (%) ((1-RR)x100) 25 25 25 Absolute risk reduction (CER-EER) 0.1 0.01 0.001 Number needed to treat (1/ARR) 10 100 1000

40

Calculation of estimates of effect

Problem 1. WHEN RELATIVE RISK<1 Of the 750 people randomized to receive treatment, 350 people developed the outcome. Of the 800 people randomized to no treatment 500 developed the outcome.
Experimental event rate =EER (incidence in exposed) =350/750 =0.47 ie 47% of treated people experienced the outcome May be expressed as 47 events per 100 Control event rate exposed) =CER (incidence in non-

Problem 2. WHEN RELATIVE RISK>1 17% of people exposed developed the outcome whereas only 5% of people who had not been exposed developed the outcome.
Experimental event rate =EER (incidence in exposed) =17% =0.17

=baseline risk =500/800 =0.625 ie 63% of controls experienced the outcome May be expressed as 625 events per 1000 etc Relative Risk=RR RR =EER/CER RR =(350/750)/(500/800) =0.467/0.625 =0.75 From the study it is estimated that people who undergo treatment have 0.75 times the risk of the outcome compared to those who did not receive the treatment. (ie treatment IS PROTECTIVE) Or It is estimated that people who undergo the treatment have three quarters of the risk of the outcome compared to those who did not receive the treatment. Or It is estimated that people who undergo the treatment have 75% of the risk of the outcome compared to those who did not receive the treatment Relative Risk Reduction=RRR (expressed as a percentage) RRR =(1-RR)x100 =(1-0.75)x100 =0.25x100 =25% It is estimated that people who undergo the treatment have a 25% relative reduction in the risk of the outcome compared to those who did not receive the treatment. Absolute Risk Reduction=ARR (or RD) May be expressed as a risk reduction or an event rate reduction ARR =CER-EER =500/800-350/750 =0.625-0.467 =0.16 =16 events per 100 If a person undergoes treatment he reduces his absolute risk by 16 events per 100 Treatment prevents 16 events per 100 Number Needed to Treat=NNT NNT =1/ARR =1/0.16 =6.3 7 (6.3) people need to be treated to prevent one outcome.

ie 17%=17/100 ie 17% of exposed people experienced the outcome (given) May be expresses as 17 events per 100 etc Control event rate =CER (incidence in nonexposed) =baseline risk =5% =0.05 ie 5% of controls experienced the outcome (given ) May be expressed as 5 events per 100 etc Relative Risk=RR RR =EER/CER RR =0.17/0.05 =3.33 From the study it is estimated that people who were exposed have 3.33 times the risk of the outcome compared to those not exposed. Or It is estimated that people who have been exposed have three and a third times the risk of the outcome compared to those not exposed. Or It is estimated that people who were exposed have 333% of the risk of the outcome compared to those not exposed.

Do not calculate RRR when RR>1 ie it is a relative risk increase

When RR>1 better to change the name from Absolute risk reduction to Risk Difference (RD) RD =CER-EER =0.05-0.17 = - 0.12

Take the absolute value and interpret it with respect to your results. =12 events per 100 If a person is exposed she INCREASES her absolute risk by 12 events per 100 Exposure results in a risk difference of 12 events per 100

41

Please see references: Guyatt GH, Sackett DL, Cook DJ. Users guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? JAMA 1993;270:2598-2601. Guyatt GH, Sackett DL, Cook DJ. Users guides to the medical literature. II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients. JAMA 1994;271:59-63.

42

Diagnostic tests

Objectives i. Understand the importance of pre-test probability for the interpretation of test results ii. Calculate and interpret the following features of diagnostic tests:sensitivity, specificity, likelihood ratios, pre-test probability, post-test probability.

Gold Standard Disease Yes Diagnostic Test Positive Negative TOTAL True positive (TP) False negative (FN) TP + FN No False positive (FP) True negative (TN) FP + TN TOTAL TP + FP FN + TN

Sensitivity and Specificity of tests Sensitivity is the proportion of all truly diseased persons (TP + FN) who test positive using the diagnostic test. i.e. SN = TP/(TP+FN) Specificity is the proportion of all truly non-diseased persons (TN + FP) who test negative with diagnostic test. i.e. SP = TN/(TN+FP) Numerical example:

Gold Standard Disease Yes Diagnostic Test Positive Negative TOTAL 90 10 100 No 20 80 100 TOTAL 110 90 200

43

SN = TP / (TP + FP) = 90 / (90 + 10) = 90% i.e. of those with the disease 90% will test positive (and 10% will test negative) SP = TN / (TN + FP) = 80 / (80 + 20) = 80% i.e. of those without the disease 80% will test negative (and 20% will test positive)

Predictive values Predictive values tell you the probability of the test result being correct. Positive Predictive value=PPV If you have a patient with a positive test result, this will tell you the probability of that patient actually having the disease. As PPV refers to the proportion of people testing positive that are true positives, it is calculated by: PPV = TP/(TP+FP)

Negative Predictive Value=NPV If you have a patient with a negative test result, this will tell you the probability of that patient actually not having the disease. As NPV refers to the proportion of people testing negative that are true negatives, it is calculated by: NPV = TN/(TN+FN) Numerical example: If we look at the previous numerical example for sensitivity and specificity: PPV = 90 / (90 + 20) = 82%

44

i.e. of those with a positive test result 82% will have the disease NPV = 80 / (80 + 10) = 89% i.e. of those with a negative test result 89% will not have the disease Notice that in the above example the prevalence of disease is 50% (as 100 people have the disease and 100 people do not have the disease). If we change the prevalence of disease to 10%, we would get the following table (note that the sensitivity and specificity of the test do not change).

Gold Standard Disease Yes Diagnostic Test Positive Negative TOTAL 9 1 10 No 20 80 100 TOTAL 29 81 110

This would mean that the predictive values would now be: PPV = 9 / (9 + 20) = 31% NPV = 80 / (80 + 1) = 98% Notice how affected predictive values are by the prevalence of disease. So for example, the probability of a test result being correct might be very different when you conduct the test in a situation with low prevalence (e.g. screening test at a shopping centre) compared to one with a high prevalence (e.g. specialist clinic).

45

Likelihood Ratios Likelihood ratios are a way of looking at test results that are not dependent on the prevalence of disease. They demonstrate how well the test differentiates between diseased and non-diseased people. Likelihood ratios always compare the chance of a particular result (positive or negative) in diseased people versus the chance of the same result in non-diseased people. If the LR = 1 then this would mean that the test cannot differentiate between diseased and non-diseased people. Positive likelihood ratio=LR+ Likelihood of getting a positive test result in the diseased (TP / TP + FN) divided by the likelihood of getting a positive test result in the non-diseased (FP / FP + TN). i.e. LR + = [TP / (TP + FN)] / [FP / (FP + TN)] This is equivalent to: Sens / (1 spec) In the original numerical example the positive likelihood ratio would be: LR+ = [90 / (90 + 10)] / [20 / (20 + 80)] = 0.9 / 0.2 = 4.5 This means that you are 4.5 times more likely to get a positive test result if you have the disease than if you are disease free. For a positive LR, the higher the value is above one the better.

Negative Likelihood ratio=LRLikelihood of getting a negative test result in the diseased (FN / FN + TP) divided by the likelihood of getting a negative test result in the non-diseased (TN / TN + FP). i.e. LR - = [FN / (FN + TP)] / [TN / (TN + FP)] This is equivalent to: 1 sens / spec In the original numerical example the negative likelihood ratio would be:

46

LR-

= [10 / (10 + 90)] / [80 / (80 + 20)] = 0.1 / 0.8 = 0.125

This means that you are about one tenth as likely to get a negative test result if you have the disease than if you are disease-free. For a negative LR, the lower the value is below one the better. NOTE: When calculating LRs, people with the disease are always in the numerator, regardless of whether you are looking at the proportion with a positive or negative test result. How are likelihood ratios used? LRs can be used in combination with a nomogram to calculate the post-test probability of disease. Looking at the example below, the cross on the left hand side refers to the pre-test probability of disease. This is the prevalence of disease in the particular patient group. A line is then drawn through the LR (the middle line). The value that the line hits on the far side is the post-test probability.

Prevalence of disease

= pre-test probability = 20%

LR + = 10 Post-test probability = 72%

47

Please see references: Jaeschke R, Guyatt G, Sackett DL. Users guides to the medical literature. III. How to use an article about a diagnostic test. A. Are the results of the study valid? JAMA 1994;271:389-391. Jaeschke R, Guyatt G, Sackett DL. Users guides to the medical literature. III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? JAMA 1994;271:703-707.

48

Meta-analysis and systematic reviews

In all cases systematic reviews of the stated best study type would be better than any individual study.

Meta-analysis Meta analysis refers to the quantitative methods for combining the results of several studies into a single pooled or summary statistic.

Systematic review
Systematic review is a critical evaluation of research that attempts to address a focused clinical question using methods designed to reduce the likelihood of bias. It may include a meta analysis (summary statistic).

Narrative review
Narrative Review is a good place to start when learning about a topic however it is subject to substantial biases and limitations so is insufficient for clinical decision making.

Publication bias Publication bias refers to the preferential publication of studies with a statistically significant positive result.

Duplication bias Duplication bias refers to the inclusion of the same study more than once in the systematic review.

49

Please see references: Meta-analysis and systematic reviews Oxman AD, Cook DJ, Guyatt G. Users guides to the medical literature. VI. How to use an overview. JAMA 1994;272:1367-1371.

Other JAMA Users Guides Richardson WS, Detsky AS. Users guides to the medical literature. VII. How to use a clinical decision analysis. A. Are the results of the study valid? JAMA 1995;273:1292-1295. Richardson WS, Detsky AS. Users guides to the medical literature. VII. How to use a clinical decision analysis. B. What are the results and will they help me in caring for my patients? JAMA 1995;273:1610-1613. Hayward RSA, Wilson MC, Tunis SR, Bass EB, Guyatt G. Users guides to the medical literature. VII. How to use clinical practice guidelines. A. Are the recommendations valid? JAMA 1995;274:570-574. Wilson MC, Hayward RSA, Tunis SR, Bass EB, Guyatt G. Users guides to the medical literature. VII. How to use clinical practice guidelines. B. What are the recommendations and will they help you in caring for your patients? JAMA 1995;274:1630-1632. Drummond MF, Richardson WS, OBrien BJ, Levine M, Heyland D. Users guides to the medical literature. XIII. How to use an article on economic analysis of clinical practice. A. Are the results of the study valid? JAMA 1997;277:1552-1557. OBrien BJ, Heyland D, Richardson WS, Levine M, Drummond MF. Users guides to the medical literature. XIII. How to use an article on economic analysis of clinical practice. B. What are the results and will they help me in caring for my patients? JAMA 1997;277:1802-1806. Dans AL, Dans LF, Guyatt G, Richardson S. Users guides for the medical literature. XIV. How to decide on the applicability of clinical trial results to your patient. JAMA 1998;279:545-549. Barratt A, Irwig L, Glasziou P et. al. Users guides to the medical literature. XVII. How to use guidelines and recommendations about screening. JAMA 1999;281:2029-2034. EBM cards from Sackett, D. How to practice and teach EBM

50

Additional resources
Some useful Internet sites
Cochrane Library: http://www.thecochranelibrary.com PubMed: http://www.ncbi.nlm.nih.gov/entrez Oxford Centre for Evidence Based Medicine: http://www.cebm.net/ Netting the Evidence: http://www.shef.ac.uk/scharr/ir/netting/ Users guides to evidence based practice: http://www.cche.net/usersguides/main.asp Sydney University Library links to useful EBM sites: http://www.library.usyd.edu.au/subjects/medicine/links/ebp.html Duke University Introduction to EBM: http://www.hsl.unc.edu/services/tutorials/ebm/welcome.htm University of Colorado: A students guide to the medical literature: http://grinch.uchsc.edu/sg/

Some useful references

Evidence-based medicine: how to practice and teach EBM. 3rd ed Sharon E Strauss, W. Scott Richardson, Paul Glasziou and R Brian Haynes. Churchill Livingstone, 2005. Evidence-based healthcare. J.A. Muir Gray. Churchill Livingstone, 2001. Clinical Epidemiology; a basic science for clinical medicine. 2nd ed David Sackett et al. Little, Brown, 1991. A Dictionary of Epidemiology. 5th ed. John Last. Oxford University Press, 2008 Clinical Epidemiology: the essentials. 4th ed. Robert Fletcher and Suzanne Fletcher. Williams and Wilkins, 2005

51