11/10/2016 EBM,HTA,andCER:ClearingtheConfusion

MilbankQ.2010Jun88(2):256276. PMCID:PMC2980346
doi:10.1111/j.14680009.2010.00598.x

EBM,HTA,andCER:ClearingtheConfusion
BryanRLuce,MichaelDrummond,BengtJnsson,PeterJNeumann,JSanfordSchwartz,UweSiebert,andSeanDSullivan
TheInternationalWorkingGroupforHTAAdvancementUnitedBioSourceCorporationUniversityofYorkStockholmSchoolofEconomicsTuftsMedical
CenterUniversityofPennsylvaniaUniversityforHealthSciences,MedicalInformaticsandTechnologyUniversityofWashington
Addresscorrespondenceto:BryanR.Luce,CenterforHealthEconomicsandSciencePolicy,UnitedBioSourceCorporation,7101WisconsinAvenue,
Suite600,Bethesda,MD20814(email:bryan.luce@unitedbiosource.com).

Copyright2010MilbankMemorialFund

ThisarticlehasbeencitedbyotherarticlesinPMC.

Abstract Goto:

Context:Thetermsevidencebasedmedicine(EBM),healthtechnologyassessment(HTA),comparative
effectivenessresearch(CER),andotherrelatedtermslackclarityandsocouldleadtomiscommunication,
confusion,andpoordecisionmaking.Theobjectiveofthisarticleistoclarifytheirdefinitionsandtherelationships
amongkeytermsandconcepts.

Methods:Thisarticleusedtherelevantmethodsandpolicyliteratureaswellasthewebsitesoforganizations
engagedinevidencebasedactivitiestodevelopaframeworktoexplaintherelationshipsamongthetermsEBM,
HTA,andCER.

Findings:Thisarticleproposesanorganizingframeworkandpresentsagraphicdemonstratingthedifferencesand
relationshipsamongthesetermsandconcepts.

Conclusions:Morespecificterminologyandconceptsarenecessaryforaninformedandclearpublicpolicy
debate.Theyareevenmoreimportanttoinformdecisionmakingatalllevelsandtoengendermoreaccountability
bytheorganizationsandindividualsresponsibleforthesedecisions.

Keywords:Evidence,policy,decisionmaking,evidencebasedmedicine,healthtechnologyassessment,
comparativeeffectivenessresearch

Manyjurisdictionsarebecomingincreasinglyinterestedinevidencebasedhealthcaredecisionmakingbecauseof
theirdesiretoimprovethequalityandefficiencyofcareprovidedtopatients.Suchactivitiesaregivenvarious
names,suchasevidencebasedmedicine(EBM),healthtechnologyassessment(HTA),ormorerecently,
comparativeeffectivenessresearch(CER).Thesetermsarenotusedconsistently,however,whichhasledto
confusioninthemedicalandhealthpolicycommunities.Inthisarticle,weproposeanorganizingframeworkthat
relatestheseevidencetermstothreebasicquestionsregardingevidencegeneration,evidencesynthesis,and
decisionmaking:Canitwork?Doesitwork?Isitworthit?

Thelackofconsistencyandclarity,andeventhemisuseofbasicwordsandtermsofevidencebasedactivities,
leadstounnecessarydisagreementsamongkeystakeholdersconcerningtheirappropriateroleinhealthcare
decisionmaking.Bykeystakeholders,wemeanmedicalprofessionalsandotherhealthcareproviders,health
carepayers,legislatorsandregulators,privateandpublichealthagencies,manufacturers,patients,patientadvocacy
groups,and,finally,taxpayers.Theimplicationscanbesignificantbecausetheevidenceprocessestowhichwe
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2980346/ 1/11
11/10/2016 EBM,HTA,andCER:ClearingtheConfusion

referincludethedevelopmentandapplicationofclinicalguidelines,insurancecoveragedecisions,drugformulary
placement,andreimbursementdecisions,thatis,thepaymentforandaccesstohealthcareitself.WhenEBM,
HTA,orCERisinvoked,whatismeantbythetermevidence?Isitlimitedtoevidencefromrandomizedclinical
trials?Ordoesitalsoincludeevidencefromwelldesignedobservationalstudies?Whichelementsofevidenceare
inandwhichareout?Whataboutevidenceofcoststohealthplansorpatientsoutofpocketcosts?Howaboutlost
orgainedproductivity?Patientspreferences?Orhealthrelatedqualityoflife?

AmongthemanyexamplesofthisproblemthatweaddresshereisthatofDr.DavidSackett,whopopularizedthe
conceptandapplicationofEBM(Sackettetal.2000):Thepracticeofevidencebasedmedicinemeansintegrating
individualclinicalexpertisewiththebestavailableexternalclinicalevidencefromsystematicresearchand
individualpatientspredicaments,rightsandpreferencesinmakingclinicaldecisionsabouttheircare(Sackettet
al.1996,71).ThisdefinitionisverydifferentfromEddy's(1997,2005),asheadoptsabroaderdefinitionofEBM
thatalsoconsidersthedevelopmentofevidencebasedpoliciesandguidelines,theviewtakenaswellbythe
InstituteofMedicineRoundtableonEvidenceBasedMedicine(IOM2009).AccordingtoEddy,thebroader
notionofEBMalsoencompassescosteffectiveness.

Asaconcept,healthtechnologyassessmentwasinitiallyestablishedbytheOfficeofTechnologyAssessmentin
1978(OfficeofTechnologyAssessment1978)andcontinuestobedefinedbytheInternationalNetworkof
AgenciesforHealthTechnologyAssessment(INAHTA)asamultidisciplinaryfieldofpolicyanalysis,studying
themedical,economic,socialandethicalimplicationsofdevelopment,diffusionanduseofhealthtechnology
(INAHTA2009).Inthiscontext,technologyisdefinedbroadlytoincludedrugs,devices,procedures,and
systemsoforganizationofhealthcare,althoughinpractice,itiscommonlyappliedmuchmorenarrowly.In
Canada,Europe,andmanyotherpartsoftheworld,agenciesdesignatedasHTAorganizationshavesocial
legitimacytheycontinuetoflourishandtheassessmentsalmostalwaysincludecosteffectiveness.TheUnited
States,however,hasshiftedawayfromusingthetermHTAandseemstohavesubstitutedtermslikeoutcomes
research,effectivenessresearch,EBM,and,mostrecently,comparativeeffectivenessresearch.Unfortunately,the
sequentialsubstitutionofthesetermsforHTAhasnotbeenaccompaniedbyclarityofmeaning,otherthanit
usuallydoesnotincludeeconomicevaluationintheUnitedStates(LuceandCohen2009).

ThetermCERalsoisuseddifferentlybydifferentindividualsandorganizations.Oneexampleisarecentpaper
(Chalkidouetal.2009)thatreviewedtheCERexperiencesofagenciesinthreeEuropeancountriesthatundertook
arangeofwork,includingactivitiesthathavetraditionallybeentermedHTA.IntheUnitedStates,whereCER
becameafocalissueinthe2009/2010healthreformdebate,itsdefinitionhastakenonmultipledimensionsand
meaningsandhasnotbeenclearlydifferentiatedfromeitherHTAorEBM.AtypicaldefinitionofCERisthatof
theInstituteofMedicine(IOM):

CERisthegenerationandsynthesisofevidencethatcomparesthebenefitsandharmsofalternativemethods
toprevent,diagnose,treatandmonitoraclinicalcondition,ortoimprovethedeliveryofcare.Thepurposeof
CERistoassistconsumers,clinicians,purchasers,andpolicymakerstomakeinformeddecisionsthatwill
improvehealthcareatboththeindividualandpopulationlevels.(IOM2009)

Thisdefinitionissufficientlybroadtomeanentirelydifferentthingstodifferentstakeholdersandpolicymakers.
SomewouldconsiderCERtobeasingleheadtoheadcomparativetrialorothersinglecomparatorstudy,and
otherswouldconsideritasynthesisofexistingevidenceofalternativeinterventions.TomostEuropeans,theIOM's
definitionreferstoHTAandincludeseconomicevaluation.

SomeanalystsintheUnitedStates,mostnotablyGailWilensky,whosparkedthecallin2006foramajornational
commitmenttoCER(Wilensky2006),speakofitlargelyinregardtoinvestingincomparativeeffectivenesstrials
(Wilensky,personalcommunication,2010).Manyothers,however,seemtoconsiderCERmainlyasanextension
ofandadeepercommitmenttoimprovingcomparativeevidenceusingtraditionalhealthservicesresearchmethods,
forexample,vialiteraturesynthesis,observationalmethods,andimprovedhealthinformationtechnology.Some
analystsadvocateincludingeconomicevaluationinCER(AmericanCollegeofPhysicians2008),whileothers
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2980346/ 2/11
11/10/2016 EBM,HTA,andCER:ClearingtheConfusion

wanttoexcludeit(Wilensky2008)orhavenotcommentedonitsinclusionorexclusion(FederalCoordinating
Council2010IOM2009).

Morepreciseterminologywillnotbesufficienttoensurewiseinvestmentinimprovingtheevidencebaseandthe
optimaluseoftheevidence.Butitisnecessarynotonlytoenablepublicpolicydebatetobeinformedandclearbut
also,andmoreimportant,toimprovedecisionmakingatalllevelsandtomaketheorganizationsandindividuals
responsibleforthesedecisionsmoreaccountable.

Thisistheobjectiveofourarticle.

Webeginbyproposingageneralorganizationalframeworkandthendepictinsomedetailourconceptofthe
interrelationshipsofEBM,HTA,CER,andrelatedterminology.

AnOrganizingFramework Goto:

InordertobetterillustratethecurrentconfusionandtohelpderivemoreprecisedefinitionsofEBM,HTA,CER,
andrelatedconcepts,weproposeathreebythreematrix.Alongoneaxisarethethreequestionsthatevidence
basedprocessesinhealthcareseektoansweraboutanintervention,namely,Canitwork(i.e.,efficacy),Doesit
work?(i.e.,effectiveness),andIsitworthit?(i.e.,economicvalue).Inthecaseofeconomicvalue,wealso
distinguishbetweenvaluetothepatientandvaluetothepayer/society.Alongtheotheraxisarethethreekey
functionsofimplementingevidencebasedactivities,namely,EvidenceGeneration,EvidenceSynthesis,and
DecisionMaking.

Asfigure1shows,ifwedepictedthecurrentnotionsofEBM,HTA,andCERasrectanglesonthematrixbasedon
definitionsimpliedbycommentsinthecurrentliterature,wewouldimmediatelyseewhythereissomuch
confusion.ThespaceoccupiedbyEBMcoversallthreeevidencequestionsplusthefunctionsofbothevidence
synthesisanddecisionmaking.ThespaceoccupiedbyHTAissimilartothatofEBM,exceptthatitstretches
furtherintothedomainofvaluetosocietyandisperhapslessinfluencedbyefficacy.AlthoughEBMisnot
depictedasincludingprimaryclinicalevidencegeneration,theboxforHTAdoescrossintothisspace,sincein
somecasesHTAstudiesmayincludeprimaryeconomicevidencegeneration.ThespaceoccupiedbyCERis
depictedasresidingmainlyinevidencegenerationandsynthesisbutalsooverlappingintodecisionmaking,
notwithstandingthatsomeU.S.commentators(e.g.,Wilensky2008)andlegislation(e.g.,U.S.Congress2003)
havemadeclearthattheprocessofmakingcoverageandotherpolicyrelateddecisionsshouldorwillremain
separate.

Figure1
CurrentConfusionoverViewsofEBM,CER,andHTA.

ThisparticulardepictionofconfusedbutrelatedconceptsmaybesomewhatU.S.centric,butwewouldarguethat
ourgeneralassertionofconfusionexistsoutsidetheUnitedStates,too.

MorePreciseDefinitions Goto:

Todefinethethreekeyconceptsmoreprecisely,weneedtodescribeinmoredetailthecontentsofeachofthecells.
Wediscusshowevidenceisgenerated,synthesized,andusedindecisionprocessesforeachofthethreeevidence
questions.Inthemedicalfield,evidenceofsafetyandefficacyisgeneratedprimarilyfromclinicalstudiesofvarious
designs,asdescribedlater.Fordecisionmakingpurposes,however,alltheavailableevidenceonagiventopic
needstobesynthesized.

Efficacy:CanItWork?

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2980346/ 3/11
11/10/2016 EBM,HTA,andCER:ClearingtheConfusion

Ahealthcareinterventionisconsideredefficaciouswhenthereisevidencethattheinterventionprovidesthe
intendedhealthbenefitwhenadministeredtocarefullyselectedpatientsaccordingtoprescribedcriteria,oftenby
expertsinaresearchsetting.Theevidenceofefficacytypicallycomesfromwellcontrolledrandomizedclinical
trials(RCTs),astypicallyconductedonnewmedicines.BecauseRCTsuserandomizationandotherresearch
designfeaturesthatminimizepotentialbiases(e.g.,confounding,information,selectionbias),theyareessentialto
helpdemonstrateacausalrelationshipbetweenaninterventionandanoutcome(i.e.,theyhavehighinternal
validity).Efficacy/RCTtrialsarecommonlyandappropriatelydescribedasexplanatorytrials.Thatis,
althoughtheyareoftendesignedtoevaluateCanitwork,theyalsoareoftenusefulinexplainingwhyitworks
(SchwartzandLellouch1967/2009).Thesetrialsareusuallydesignedtoanswerpreciselyquitenarrowclinicalor
biologicalquestions.

EvidencesynthesisrelevanttotheCanitworkquestionisconductedbyasystematicreviewoftrials(SRT),in
whichathorough(systematic)searchofthetrialliteratureisconductedandsummaryestimatesofthekeyclinical
parametersareproduced,oftenthroughaformalprocesssuchasmetaanalysis.Themainandmostappropriate
decisionprocessusingefficacyRCTsisthatoflicensing/marketapprovalforhealthtechnologies,ascarriedoutby
theFoodandDrugAdministration(FDA)intheUnitedStatesandsimilarbodiesinotherjurisdictions,thatrequire
evidenceofefficacy,aswellasthesafetyandqualityofmanufacture.Thesetrialsalsooftenformthebasisfor
clinicalguidelinesandclinicaldecisions,althoughasnotedinthenextsection,guidelinesanddecisionsshouldbe
alsoinformedbytheDoesitworkquestion.

EvidenceofefficacyfromRCTsisappropriatelyconsideredtobethegoldstandardstudydesignintheevidence
hierarchy,butonlyfortheCanitwork?question.Conversely,itisnotnecessarilythegoldstandardfortheDoes
itwork?question,thesubjectofthenextsection.Unfortunately,aswepointoutlater,efficacytrialsarealltoo
commonlymisspecifiedasleadingtoevidenceofeffectiveness,whichcanresultinoverinterpretingtheirvalueand
relevancetoroutinepracticesettingsandtheresultingdecisions.

Effectiveness:DoesItWork?
Ahealthcareinterventionisconsideredeffectivewhenthereisevidenceofbenefittopatientswhenadministered
byphysiciansinroutineclinicalpracticesettings.Factorssuchassurgeonslearningcurvewithanewprocedureor
patientsadherencetodrugtherapiescancausetheeffectivenessofahealthcareinterventiontodiffersubstantially
fromitsefficacy.Byitsverynature,astudythatproducesevidenceofeffectivenessinrealworldpracticesettingsis
moregeneralizable.Ittendstotradeaspectsofinternalvalidityforhigherexternalvalidityandtypicallyaddresses
broaderquestionsthandoefficacytrials.

RCTsthatareconductedinroutinepracticesettingsarecalledeffectivenessRCTsor,morecommonly,pragmaticor
practicalclinicaltrials(PCTs)(SchwartzandLellouch1967/2009Tunis,Stryer,andClancy2003).Asthereisno
hardandfastdistinctionbetweenefficacyandpragmatictrials,itisprobablybettertothinkofaspectrumoftrial
designsandconduct(Thorpeetal.2009).ItisalsotruethatsomeRCTsareconductedinroutinepracticesettings,a
trendthatwehopecontinues.Thedistinguishingfeaturesofpragmatictrialsarethattheyarealwaysconductedin
routinepracticesettingstheynormallycomparethenewinterventionwithanacceptedcurrentpracticeandthey
tendtobemorefocusedonendpointsofdirectinteresttothepatient(e.g.,functionalstatus,finalhealthoutcome),
asopposedtosurrogateendpointssuchasviralloadorbloodpressure.Inaddition,theprotocolsforsuchtrialsare
usuallymuchlessintrusive(i.e.,fewerprescribedvisits,fewermeasurementsofclinicalparameters,andlesspatient
monitoring),andtheactualratesofadherencetotherapymaybeofstudyinterest,ratherthanadherencebeing
specificallyencouraged,asistypicalinanefficacyRCT.Forinstance,inapragmatictrialcomparingthenewer,
moreexpensiveantidepressant(fluoxetineProzac)withinexpensivegenericalternatives(imipramineand
desipramine),thepragmaticresearchquestionwasWhatwillhappentothehealthofourpatientsandtoour
plan'sbudgetifweinitiatetreatmentwithdrugXorYintheroutineprimarycaresetting?(Simonetal.1996).
Thatis,almosttheonlyclinicalprotocolfeaturewasinitialrandomassignmenttotherapy.Allpatientswhomthe
primaryphysiciandeemedqualifiedforanantidepressantdrugwereeligible,andclinicaldecisionsandpatients
behavior(e.g.,adherencetotherapy)werecompletelynaturalistic.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2980346/ 4/11
11/10/2016 EBM,HTA,andCER:ClearingtheConfusion

Inothersituations,aclinicaltrialcannotbeconductedbecauseoflogisticalorethicalreasons,oratrialmaynotbe
themostefficientwayofgatheringthedata.Thelatterisoftenthecaseforraresideeffectsoroutcomesinthe
longerterm.Insuchcases,itoftenisnecessarytouseuncontrolledstudiessuchasregistries,administrative
databases,orclinicalcaseseries.SuchstudiesareusedbybodiesliketheFDAtoassesssafety,butthese
observationalstudiescanalsobeimportantsourcesofevidenceforeffectiveness,eventhoughtheydonothavethe
highinternalvalidityofRCTs.

Onereasonthisdistinctionbetweenefficacyandeffectivenessisimportantisthatthemisapplicationoftheterms
maybecontributingtoafalsesenseofwhatconstitutesbestsynthesizedevidenceforinformeddecisionmaking
abouttheeffectsofhealthinterventionsinroutinepracticesettings.Decisionsbasedonsuchmisconceptionsmay
adverselyaffectpatientsaccesstocareandinnovativeproducts,thequalityofcare,andthecostsandcost
effectivenessofdeliveringsuchcare.AsnotedbyLohrandcolleagues,Thefundamentalbasicscienceimperative
maybetogenerateinformationabouttheefficacyofhealthcareinterventions,butthepracticalrealitiesofpolicy
andeconomicdecisionscallforknowledgeabouteffectiveness(Lohr,Eleazer,andMauskopf1998,16).

Therefore,synthesisofeffectivenessdata,whichwetermsystematicreviewofevidence(SRE),typicallyincludes
datafromwellconductedobservationalstudiesaswellasRCTs.Asarule,efficacyRCTdataandrealworld
effectivenessdatacomplementoneanotherandarebestusedintandem.AlthoughtheliteraturesearchforSREis,
orshouldbe,asrigorousasthatinSRTs,theinclusioncriteriaforstudiesarelessrestrictive,andthemethodsof
summarizingevidenceonkeyclinicalendpointsaretypically,andoftennecessarily,lessformalinordertoaddress
abroaderrangeofquestions.

Decisionprocessesrelatingtoeffectivenessevidencereviewsarehighlyvariedandmayincludethedevelopmentof
clinicalguidelines(e.g.,aspartofEBM),individualpatientphysiciancaredecisions,orinputstothedecisions
abouteconomicvalue,whichweaddresslater.Here,wegenerallyplacedecisionsabouteconomicvalueinthe
HTAcategory.

Severaloftheworld'smostprominentevidencebasedmedicinerelatedorganizationshavereliedalmost
exclusivelyonevidenceofhealthcareinterventionsderivedfromrandomizedclinicaltrials(CenterforEvidence
BasedPolicy2010CochraneCollaboration2010).ThesestudiesareprimarilyefficacyRCTs,particularlyinthe
caseofnewmedicinesforwhichtrialsareconductedmainlyforlicensingpurposes,althoughoccasionallythe
evidencebasemayincludetrialsthatareclosertothepragmaticendofthespectrum.Aheavyrelianceonefficacy
RCTsisnotamajorproblemintheworkofaregistrationauthoritysuchastheFDA,sinceitsroleistoestablish
whetheranewtechnologycanworkforthepurposeofmarketapproval.

Theprincipalproblemariseswhenorganizationsseekingtoinformrealworldchoicesoftherapiesrelysolelyon
efficacyRCTstoinformtheDoesitwork?question.Here,itisimportanttoknowhowinterventionsworkin
practice,asopposedtowhethertheycanworkinprinciple.Attheinternationallevel,organizationsseekingto
informsuchrealworldchoicesincludeGermany'sInstituteforQualityandEfficiencyinHealthCare(IQWiG
2009a),CommonDrugReviewattheCanadianAgencyforDrugsandTechnologiesinHealth(CADTH)
(CADTH2009),andtheUnitedKingdom'sNationalInstituteforHealthandClinicalExcellence(NICE)(NICE
2009).WithintheUnitedStates,theyincludetheDrugEffectivenessReviewProject(DERP)attheOregonHealth
andScienceUniversity(CenterforEvidenceBasedPolicy2010),theMedicareEvidenceDevelopmentand
CoverageAdvisoryCommittee(MEDCAC)(CMS2009b),andBlueCrossBlueShield'sTechnologyEvaluation
Center(TEC)(BlueCrossBlueShield2009).

Itisimportantthattheseorganizationsconsiderawiderrangeofavailable,germaneevidence,includingPCTsand
observationaldata,becausethegoldstandardevidencebaseforthesedecisionswouldbeonethatemphasized
studiesconductedinroutinepracticesettingsor,asRawlins(2008)wouldsay,arefitforpurpose,takinginto
accountbothexternalandinternalvalidity.Arecentstudy(Neumannetal.2010)evaluatedanumberofHTA
organizationsaroundtheworldastowhethertheyconformedtoasetofpublishedprinciples(Drummondetal.
2008),oneofwhichaddressestheinclusionofallrelevantevidence.WithintheUnitedStates,severalmanaged
careentitieshaveadoptedtheAcademyofManagedCarePharmacy's(AMCP)FormatforFormularySubmission
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2980346/ 5/11
11/10/2016 EBM,HTA,andCER:ClearingtheConfusion

(AMCP2009),whichencouragestheconsiderationofabroadrangeofevidence.Thesameappliestothe
methodologyguidelinesdevelopedbyNICEintheUnitedKingdom(NICE2008).

Distinguishingefficacyfromeffectivenessandemphasizingitsimportancetodecisionmakingdatesbacktoatleast
1978(OfficeofTechnologyAssessment1978),soitisfarfromanewconcept.Butitissomewhatdisconcerting
thatconfusionstillexists.SomeofthisconfusioncanbetracedtotheFDA'sauthorizinglegislationandregulations
inwhichthetermsefficacyandeffectivenessareoftenusedinterchangeably,soeffectivenessisusedwhenefficacy
isintended.1Thismisapplicationoftermshascarriedovertootherorganizationsaswell.Forexample,theDrug
EffectivenessReviewProject'sstatedmissionistoobtainthebestavailableevidenceoneffectiveness[italicsadded]
andsafetycomparisonsbetweendrugsinthesameclass,andtoapplytheinformationtopublicpolicyandrelated
activities(CenterforEvidenceBasedPolicy2010),yetDERPhastendedtorelyexclusivelyonevaluationsbased
onRCTs.Similarly,thehighlyregardedCochraneCollaborationdescribesitsreviewsasexploringtheevidence
forandagainsttheeffectiveness[italicsadded]andappropriatenessoftreatmentsinspecificcircumstances
(CochraneCollaboration2009),thoughevenacasualexaminationoftheCochranereviewsindicatesanearly
completerelianceonRCTliterature.ThesameistrueofsomeoftheGermanIQWiG'searlybenefitassessments,
suchasthatontheinsulinanalogues(IQWiG2009b).Basedonthesearguments,thesebodiesarerelyingmainly
onevidenceofefficacy,asopposedtoeffectiveness,unlessthebodyofRCTevidenceincludesasubstantial
proportionoftrialsadoptingapragmaticmethodology.

EconomicValue:IsItWorthIt?
Forahealthcareinterventiontobeworthit,thevalueoftheclinicalandeconomicbenefitsneedstobegreater
thantheclinicalharmsandeconomiccosts,sincetheresourcesthatthetreatmentconsumesarenotavailablefor
otherhealthcarepurposes.Althoughthetermvalueiscommonlyusedbothinandoutofhealthcare,ithasnot
beendefinedinawaythatresonateswellwithinthehealthcareevidencecommunity.Valueisusuallysaidtobein
theeyesofthebeholder,whichisavalidwaytoconsidertheterm,althoughitisnothelpfulforpolicymaking
purposes.However,wedistinguishbetweenvaluetothepatientandvaluetothepayer/society,sincethesemaybe
different.

Theprincipalwaytodeterminewhetherthebenefitsobtainedfromagiveninterventionjustifytheeconomiccosts
andpotentialclinicalharmsistocompareaneconomicevaluationofatherapywithrelevantalternativesinthe
settingconcerned.Althougheconomicevaluationsmayincludegeneratingevidence,theyfrequentlyconsistof
synthesizingtheavailableclinicalandeconomicdata.Sincetheobjectiveistoassesscostsandbenefitsinroutine
practicesettings,economicevaluationsuseeffectiveness,asopposedtoefficacy,data,oncethelatter(efficacy)is
establishedandwhentheformer(effectiveness)isavailable.Economicevaluationsoftenareconductedaspartofa
coverageorreimbursementprocessor,forourpurposeshere,withinanHTAthatinturnmayinformacoverageor
reimbursementprocess.Insuchcases,thevaluequestionistypicallyapproachedfromanaggregateperspective,for
instance,thatofahealthplanoranationalhealthbudget.Alsoofnoteisthatsomeorganizations,suchastheU.S.
CentersforDiseaseControlandPrevention's(CDC's)AdvisoryCommitteeonImmunizationPractices(ACIP),
includecosteffectivenessconsiderationsintheirclinicalguidelines(Smith,Snider,andPickering2009).

Requirementstogeneratenewevidencearenowbeginningtobeembeddedincoverageschemes.Fromthe
payer's/society'sperspective,thecoveragedecisionforatechnologymaybeconditionedonfurtherevidence
regardingthecomparativeeffectivenessandpossiblecostsofthehealthcareinterventionconcerned(Levinetal.
2007).Claxton,Sculpher,andDrummond(2002)refertothisasconditionalreimbursementinSweden,theterm
restrictedreimbursementisused(TLV2009).TheU.S.CentersforMedicareandMedicaidServices(CMS)has
coinedthetermcoveragewithevidencedevelopment(CED)(CMS2009a),butinkeepingwiththecurrent
Medicarecoveragepolicy,theCEDprocessdoesnotincludeeconomicevaluation.Othercountrieshaveroughly
analogousprocesses,forinstance,theUnitedKingdomhasaformalconditionalcoveragelikeprocesstermedOnly
inResearch(see,e.g.,Chalkidouetal.,2009).

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2980346/ 6/11
11/10/2016 EBM,HTA,andCER:ClearingtheConfusion

Anelementofeconomicvaluealsocanbe(and,weargue,shouldbe)containedintheEBMdecisionprocess
specificallyasitrelatestopatientsoutofpocketexposuresuchascopaylevels.Althoughseldomconsidered,
whetherinclinicalguidelinesorliteratureaddressingEBMprocesses,patientspreferencesareundoubtedlybased
atleastinpartonoutofpocketcostsandsometimesmayaffecttheirpersonaldecisionsaboutalternativetherapies
and,perhapsmostimportant,affecttheirpatternsofadherence.

RedefiningEBM,HTA,andCER Goto:

Armedwiththeinsightsprovidedbythisorganizingframework,weproposeclarifyingthetypology,nomenclature,
andinterrelationshipsoftheevidencetermsdiscussedinthisarticle.Figure2depictsEBM,HTA,andCERagain
asrectangles.Thekeyrelatedconceptsareincirclesandovals,andthemostimportantdecisionmakingprocesses
areshownasdiamonds.Thearrowsillustratetheprincipalrelationshipsamongthevariousconcepts.Thedotted
lineslabeledAthroughCindicaterelationshipsaboutwhichthereisconsiderabledispute,aswediscusslater.

Figure2
RedefinedRelationshipsofEvidenceProcesses.

ConsideringfirstthecolumnheadedCanitwork?wecanseethatevidencegeneratedfromtraditionalefficacy
RCTsisusedfordecisionsaboutthemarketapprovalofnewinterventions.TheseRCTsarealsocurrentlythe
majorinputtosystematicreviewsoftrials.SRTsinclude,too,pragmaticclinicaltrialsthatarebecomingmore
common,andtheyalsoareinputstoclinicalguidelines,whichweviewasstraddlingthefirsttwocolumnsinthe
diagram.

ComparativeeffectivenessresearchisfirmlysituatedinthecolumnDoesitwork?coveringbothevidence
generationandevidencesynthesis.Thatis,weexpectatwowaylinkbetweenCERandpragmaticclinicaltrials,
andweexpectasimilarlinkbetweenCERandSREs.Inturn,SREs(systematicreviewsofevidence)consider
evidencefromRCTs,PCTs,andobservationalstudies.WealsodepicttheoutputsofCERactivitiesasinfluencing
EBMeitherdirectlyorindirectlythroughclinicalguidelines,whosedevelopmentweconsidertobeoneactivityof
EBM.TheoutputsofCERactivitiesalsoareanimportantinputtoHTAactivities(e.g.,theresultsofnationally
fundedCERstudiesmaybeanimportantinputtoHTAsconductedbydifferentpayers).Oneimportantaspectof
thisdefinitionofCERisthatithasnodirectlinktoanydecisionprocess,althoughitspurposeistodosoandthusit
ideallyinfluencespatientsandphysiciansdecisionsthroughEBM,orcoveragedecisionsincludingthosethrough
HTA.

EBMischaracterizedasadecisionprocess,focusingondecisionsbyindividualpatientsandphysicians,butitdoes
crossintotheevidencesynthesisspace.Inourrepresentation,EBMfocusesmainlyonthequestionDoesit
work?althoughitalsocontainsanelementofthequestionIsitworthit?specificallyfromthepatient's
perspective.Asmentionedpreviously,weacknowledgethatthedevelopmentofclinicalguidelinesisanimportant
aspectofEBM,butwethinkofitfromapatient'sperspectiveratherthanasocietalperspective.Weregardthe
productionofguidelinesfromasocietalperspectiveasbeingclosertoHTA.

Healthtechnologyassessmentstraddlesthelasttwocolumnsinthefigure,andisviewedasamethodofevidence
synthesisthatreceivesinputsfromCER,economicevaluation,andtheconsiderationofsocial,ethical,andlegal
aspects.Aswedepictit,HTAisamaininputtocoveragedecisions,whichmayalsobeinfluencedbybudget
implications.

Finally,conditionalcoverage(depictedinthediagramasCED)isrepresentedasbothadecisionandanevidence
generationprocessinvolvingthecommissioninganduseofobservationalstudiesand,occasionally,pragmatic
RCTs.

Althoughwebelievethatourorganizingframeworkandaccompanyingdefinitionsdohelpdistinguishconcepts
anddepictrelationshipsofEBM,HTA,andCER(andotherrelatedterms),somerelationshipsremainthesubjectof
debate.Theseareidentifiedbythebrokenlinesinthefigure.LineAindicatesthattherecouldbealinkbetween
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2980346/ 7/11
11/10/2016 EBM,HTA,andCER:ClearingtheConfusion

clinicalguidelinesandHTA,openingupthepossibilitythatclinicalguidelinesmayandsometimesdoexplicitly
includeaconsiderationofcosteffectivenessand,throughHTA,belinkedtocoveragedecisions.Weconsiderthis
tobeentirelyappropriate,anditistheapproachfollowedbyNICEintheUnitedKingdomandothercountries
(see,e.g.,ValueinHealth2009).Ourdiagramreflectsthecurrentmajorityviewthatclinicalguidelines,as
practicedthroughEBM,areprimarilyconcernedwithimprovingthequalityofcareforthepatientratherthan
increasingvalueformoneyforthepayerorsociety.

LineBsuggeststhattherecouldbealinkbetweenEBMandHTA.First,asweacknowledgedearlier,somehealth
technologyassessmentsmayfocusontheclinicaleffectivenessandqualityofcare,ratherthanonvalueformoney.
Second,aswearguedearlier,someoftheobjectivesofqualityofcareandvalueformoneyarecloselyaligned.For
example,removinginterventionsthatcontributeonlylittletoimprovingpatientsoutcomesandareinconvenientto
thepatient,areexpensive,orareriskywillincreasethequalityofcareandsaveresources.Nevertheless,westill
feelthatitispreferabletodistinguishbetweenactivitiesthatconcentrateprimarilyonbenefitstothepatient(EBM)
andthosethatconcentrateprimarilyonbenefitstosocietyatlarge(HTA).

LineC,goingdirectlyfromEconomicEvaluationtoCER,isprobablythemostcontroversialofall.This
relationshipimpliesthatCERstudiesshouldincludeconsiderationofcosteffectiveness,asuggestionthathasledto
considerabledebateintheUnitedStates(AmericanCollegeofPhysicians2008Wilensky2008).Atthetimeof
thiswriting,however,U.S.legislation,forinstance,thatassociatedwithhealthcarereform,doesnotincludecost
effectiveness(U.S.Congress,Senate2009).Wecanseethemeritsofbothsetsofarguments.Butourbasic
diagram,withoutlineC,hasbalance,inthatCERstudiesareviewedasanimportantsourceofinformationfor
thoseconductingbothEBMandHTA.

ThosewhobelievethatcosteffectivenessshouldberoutinelyincorporatedintoCERmaketwoarguments.First,
CERstudiesrepresentanimportantopportunityforcollectingdataonresourceuseandcost,whichinturnare
usefulforsubsequentHTAs.Second,ifcosteffectivenessisnotpartofCER,subsequentHTAsmaynotbefully
informed.Forexample,individualhealthplansmaynothavetheexpertise,time,resources,oreveninclinationto
conductpropercosteffectivenessanalysis.Insuchcases,theargumentisthatcoveragedecisionstendtofocuson
theacquisitioncostofanewtechnology,ratherthantoconsiderfullyitseconomicvalue.

ThecounterargumentisthatthedirectinclusionofcosteffectivenessconsiderationsinCERstudieschangesthe
overallbalanceintheemphasisanduseofthesestudies,concentratingontheiruseinHTAandpossibly
disregardingtheiruseinEBM.ThismayleadtoobjectionsthatwouldhampertheoverallCERmovement.We
firmlybelieve,though,thatbecausealldecisionshaveresourceimplications,costeffectivenessconsiderations
shouldplayaroleincoveragedecisions,througheitherinclusioninCERstudiesdirectlyoradequatelyconducted
HTAs.

Finally,wenotethepotentialforinherentconflictsthatinevitablyresidesinsome,ifnotall,theconceptsthatwe
havetriedtoclarifyanddistinguishfromoneanotherinthisarticle.Forexample,considerEBMandHTA.Ifwe
takeasgiventhattheproperandfullexpressionofEBMincludesindividualpatientsvalues,whichitselfcan
includepatientsoutofpocketcosts,thesocietalaggregateofthesevalueswouldquicklyconflictwithsocietal
HTA.Thisisbecausetheformerassumesthatindividualpatientsaretobesatisfiedonebyone,whereasHTAmay
takeasocietalapproach,oftenwithinbudgetconstraints,meaningthatnoteveryonemaygeteverythingheorshe
desires.Thus,althoughweseektodifferentiate,bind,andrelateEBMHTACER(plusotherrelatedevidence
basedprocesses)withinoneholisticgraphic,wearenotcontendingthattheyallhaveacentralunifyingaspect.

Bearingallthisinmind,thefollowingareourpreferreddefinitionsofthethreekeyterms:

1.Evidencebasedmedicine(EBM)isanevidencesynthesisanddecisionprocessusedtoassistpatientsand/or
physiciansdecisions.Itconsidersevidenceregardingtheeffectivenessofinterventionsandpatientsvalues
andismainlyconcernedwithindividualpatientsdecisions,butisalsousefulfordevelopingclinical
guidelinesastheypertaintoindividualpatients.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2980346/ 8/11
11/10/2016 EBM,HTA,andCER:ClearingtheConfusion

2.Comparativeeffectivenessresearch(CER)includesbothevidencegenerationandevidencesynthesis.Itis
concernedwiththecomparativeassessmentofinterventionsinroutinepracticesettings.TheoutputsofCER
activitiesareusefulforclinicalguidelinedevelopment,evidencebasedmedicine,andthebroadersocialand
economicassessmentofhealthtechnologies(i.e.,HTA).
3.Healthtechnologyassessment(HTA)isamethodofevidencesynthesisthatconsidersevidenceregarding
clinicaleffectiveness,safety,costeffectivenessand,whenbroadlyapplied,includessocial,ethical,andlegal
aspectsoftheuseofhealthtechnologies.Theprecisebalanceoftheseinputsdependsonthepurposeofeach
individualHTA.AmajoruseofHTAsisininformingreimbursementandcoveragedecisions,inwhichcase
HTAsshouldincludebenefitharmassessmentandeconomicevaluation.

Conclusions Goto:

Evidencebasedprocessesforhealthcaredecisionmakinghaveattractedincreasedinterest.Whilethisisawelcome
development,itspotentialimpactmaynotbefullyrealizedowingtoconfusionovertermssuchasevidencebased
medicine,healthtechnologyassessment,comparativeeffectivenessresearch,andotherrelatedterms.Webelieve
thatmorepreciseterminologywillbenefitthepublicpolicydialogue,generalhealthandclinicalpolicydecision
making,andspecificclinicaldecisionsbypatientsandphysicians.Itwillalsopromoteaccountabilitybythe
organizationsandindividualsresponsibleforthesedecisionsandultimatelyenhancethequalityofpatientcare.

Inthisarticle,weoffermoreprecisedefinitionsofkeytermsbasedonanorganizingframeworkthatclarifiesthe
differencesandrelationshipsamongEBM,HTA,CER,andrelatedconcepts.AllthreetermsaddresstheDoesit
work?question,butnoneaskstheCanitwork?question.Healthtechnologyassessmentistheprimaryactivity
thatconsidersIsitworthit?althoughaswepointout,EBMshouldalsoaddressthemorelimitedquestionIsit
worthittothepatient?takingintoaccountthecoststothepatient.Ofthethreeconcepts,onlyEBMisadecision
process,althoughbothHTAandCERareappliedspecificallytofeedintodecisionmaking.

Acknowledgments Goto:

TheInternationalWorkingGroupforHTAAdvancementwasestablishedinJuly2007withunrestrictedfunding
fromtheScheringPloughCorporation(nowMerck&Co).TheWorkingGroup'smissionistoprovide
scientificallybasedleadershiptofacilitatesignificantcontinuousimprovementinthedevelopmentand
implementationofpractical,rigorousmethodsintoformalhealthtechnologyassessment(HTA)systemsand
processes,byfacilitatingthedevelopmentandadoptionofhighquality,scientificallydriven,objective,andtrusted
HTAtoimprovepatientoutcomes,thehealthofthepublic,andtheoverallqualityandefficiencyofhealthcare.We
aregratefultoEmilySargentforheroutstandinggraphicswork,formatting,andgeneralmanuscriptpreparationand
forthemanyhelpfulcommentsbyseveralanonymousreviewersaswellasBradfordGray.Theviewsexpressedin
thisarticlearethoseoftheauthorsanddonotnecessarilyreflecttheopinionsofanyoftheseindividuals.

Endnote Goto:
1
Forexample,seeGuidanceforIndustry:ProvidingClinicalEvidenceofEffectivenessforHumanDrugsandBiological
Products,availableat
http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM078749.pdf(accessed
November24,2009).Inthisdocument,note2states:Asusedinthisguidance,thetermefficacyreferstothefindingsinan
adequateandwellcontrolledclinicaltrialandthetermeffectivenessreferstotheregulatorydeterminationthatismadeon
thebasisofclinicalefficacyandotherdata.However,itiswellunderstoodthatFDAdecisionsarebasednearlysolelyonsafety
andefficacy.ThisisseenintheFDA'sdocumentsreferringtoregulatoryapprovalrestingonmeetingthesubstantialevidence
requirement.Substantialevidencewasdefinedinsection505(d)oftheFood,Drug,andCosmeticActasevidenceconsisting
ofadequateandwellcontrolledinvestigations,includingclinicalinvestigations,byexpertsqualifiedbyscientifictrainingand
experiencetoevaluatetheeffectivenessofthedruginvolved,onthebasisofwhichitcouldfairlyandresponsiblybeconcluded
bysuchexpertsthatthedrugwillhavetheeffectitpurportsorisrepresentedtohaveundertheconditionsofuseprescribed,
recommended,orsuggestedinthelabelingorproposedlabelingthereof.This,however,istheappropriatedefinitionof
efficacy,noteffectiveness.

References Goto:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2980346/ 9/11
11/10/2016 EBM,HTA,andCER:ClearingtheConfusion

AMCP(AcademyofManagedCarePharmacy)TheAMCPFormatforFormularySubmissions.2009.Version
3.0.Availableathttp://www.fmcpnet.org/index.cfm?
c=news.details&a=wn&id=C604C25D_2_1~Final_Final.pdf(accessedNovember25,2009).
AmericanCollegeofPhysicians.InformationonCostEffectiveness:AnEssentialProductofaNational
ComparativeEffectivenessProgram.AnnalsofInternalMedicine.2008148:95661.[PubMed]
BlueCrossBlueShield.TechnologyEvaluationCenter.2009.Availableathttp://www.bcbs.com/blueresources/tec
(accessedNovember25,2009).
CADTH(CanadianAgencyforDrugsandTechnologiesinHealth)CommonDrugReview.2009.Availableat
http://www.cadth.ca/index.php/en/cdr(accessedNovember25,2009).
CenterforEvidenceBasedPolicy.TheDrugEffectivenessReviewProject(DERP)2010.Availableat
http://www.ohsu.edu/ohsuedu/research/policycenter/(accessedFebruary24,2010).
ChalkidouK,TunisS,LopertR,RochaixL,SawickiPT,NasserM,XerriB.ComparativeEffectivenessResearch
andEvidenceBasedHealthPolicy:ExperiencefromFourCountries.TheMilbankQuarterly.200987(2):339
67.Availableathttp://www.milbank.org/quarterly/8702feat.html(accessedMarch26,2010).[PMCfreearticle]
[PubMed]
ClaxtonK,SculpherM,DrummondM.ARationalFrameworkforDecisionMakingbytheNationalInstitutefor
ClinicalExcellence.Lancet.2002360:71115.[PubMed]
CMS(CentersforMedicareandMedicaidServices)CoveragewithEvidenceDevelopment.2009a.Availableat
http://www.cms.hhs.gov/CoverageGenInfo/03_CED.asp(accessedNovember23,2009).
CMS(CentersforMedicareandMedicaidServices)MedicareEvidenceDevelopmentandCoverageAdvisory
Committee(MEDCAC)2009b.Availableathttp://www.cms.hhs.gov/FACA/02_MEDCAC.asp(accessed
November25,2009).
CochraneCollaboration.CochraneReviews.2009.Availableat
http://www.cochrane.org/reviews/clibintro.htm#intro(accessedNovember25,2009).
CochraneCollaboration.2010.Availableathttp://www.cochrane.org(accessedFebruary25,2010).
DrummondMF,SchwartzJS,JonssonB,LuceB,NeumannP,SiebertU,SullivanS.KeyPrinciplesforthe
ImprovedConductofHealthTechnologyAssessmentsforResourceAllocationDecisions.InternationalJournal
ofTechnologyAssessmentinHealthCare.200824(3):24458.[PubMed]
EddyDM.InvestigationalTreatments:HowStrictShouldWeBe?JournaloftheAmericanMedicalAssociation.
1997278:17985.[PubMed]
EddyDM.EvidenceBasedMedicine:AUnifiedApproach.HealthAffairs.200524(1):917.[PubMed]
FederalCoordinatingCouncil.DraftDefinitionofComparativeEffectivenessResearch.2010.Availableat
http://www.hhs.gov/recovery/programs/cer/draftdefinition.html(accessedMarch16,2010).
INAHTA(InternationalNetworkofAgenciesforHealthTechnologyAssessment)HTAResources.2009.
Availableathttp://www.inahta.org/HTA/(accessedNovember25,2009).
IOM(InstituteofMedicine)InitialNationalPrioritiesforComparativeEffectivenessResearch.Washington,DC:
NationalAcademiesPress2009.Availableat
http://www.iom.edu/Reports/2009/ComparativeEffectivenessResearchPriorities.aspx(accessedNovember25,
2009).
IQWiG(InstituteforQualityandEfficiencyinHealthcare)2009a.Availableathttp://www.iqwig.de/institutefor
qualityandefficiencyinhealth.2.en.html?random=5458e2(accessedNovember25,2009).[PubMed]
IQWiG(InstituteforQualityandEfficiencyinHealthcare)LongActingInsulinAnaloguesintheTreatmentof
DiabetesMellitusType2.Cologne:2009b.Availableathttp://www.iqwig.de/index.558.en.html(accessed
March26,2010).[PubMed]
LevinL,GoereeR,SikichN,JorgensenB,BrouwersM,EastyT,ZahnC.EstablishingaComprehensive
ContinuumfromanEvidentiaryBasetoPolicyDevelopmentforHealthTechnologies:TheOntario
Experience.InternationalJournalofTechnologyAssessmentinHealthCare.200723:299309.[PubMed]
LohrKN,EleazerK,MauskopfJ.HealthPolicyIssuesandApplicationsforEvidenceBasedMedicineand
ClinicalPracticeGuidelines.HealthPolicy.199846(1):119.[PubMed]

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2980346/ 10/11
11/10/2016 EBM,HTA,andCER:ClearingtheConfusion

LuceBR,CohenRS.HealthTechnologyAssessmentintheUnitedStates.InternationalJournalofTechnology
AssessmentinHealthCare.200925:3341.[PubMed]
NeumannPJ,DrummondMF,JonssonB,LuceBR,SchwartzJS,SiebertU,SullivanSD.AreKeyPrinciplesfor
ImprovedHealthTechnologyAssessmentSupportedandUsedbyHealthTechnologyAssessment
Organizations?InternationalJournalofTechnologyAssessmentinHealthCare.201026(1):7178.[PubMed]
NICE(NationalInstituteforHealthandClinicalExcellence)GuidetotheMethodsofTechnologyAppraisal.2008.
Availableathttp://www.nice.org.uk/media/B52/A7/TAMethodsGuideUpdatedJune2008.pdf(accessed
November30,2009).
NICE(NationalInstituteforHealthandClinicalExcellence)2009.Availableathttp://www.nice.org.uk(accessed
November25,2009).
OfficeofTechnologyAssessment.AssessingtheEfficacyandSafetyofMedicalTechnologies.1978.September.
NTISorder#PB286929.Availableathttp://www.fas.org/ota/reports/7805.pdf(accessedNovember25,2009).
RawlinsM.Detestimonio:OntheEvidenceforDecisionsabouttheUseofTherapeuticInterventions.Lancet.
2008372:215261.[PubMed]
SackettDL,RosenbergWM,GrayJA,HaynesRB,RichardsonWS.EvidenceBasedMedicine:WhatItIs,and
WhatItIsnt.BritishMedicalJournal.1996312(7023):7172.[PMCfreearticle][PubMed]
SackettDL,StrausS,RichardsonSR,RosenbergW,GrayJA,HaynesRB.EvidenceBasedMedicine:Howto
PracticeandTeachEBM.London:ChurchillLivingstone2000.
SchwartzD,LellouchJ.ExplanatoryandPragmaticAttitudesinTherapeuticalTrials.JournalofChronicDiseases.
196720:63748./2009.ReprintedinJournalofClinicalEpidemiology62:499505.[PubMed]
SimonGE,VonKorffM,HeiligensteinJH,RevickiDA,GrothausL,KatonW,WagnerEH.InitialAntidepressant
ChoiceinPrimaryCare:EffectivenessandCostofFluoxetinevsTricyclicAntidepressants.Journalofthe
AmericanMedicalAssociation.1996275(24):18971902.[PubMed]
SmithJC,SniderDE,PickeringLK.ImmunizationPolicyDevelopmentintheUnitedStates:TheRoleofthe
AdvisoryCommitteeonImmunizationPractices.AnnalsofInternalMedicine.2009150(1):4549.[PubMed]
ThorpeKE,ZwarensteinM,OxmanAD,TreweekS,FurbergCD,AltmanDG,TunisS,BergelE,HarveyI,
MagidD,ChalkidouK.APragmaticExplanatoryContinuumIndicatorSummary(PRECIS):ATooltoHelp
TrialDesigners.JournalofClinicalEpidemiology.200962:46475.[PubMed]
TLV.WelcometoTLV.2009.Availableathttp://www.tlv.se/inenglish/(accessedNovember25,2009).
TunisSR,StryerDB,ClancyCM.PracticalClinicalTrials:IncreasingtheValueofClinicalResearchforDecision
MakinginClinicalandHealthPolicy.JournaloftheAmericanMedicalAssociation.2003290:1624632.
[PubMed]
U.S.Congress.MedicarePrescriptionDrug,Improvement,andModernizationActof2003.2003.Pub.L.no.108
173,117Stat2066.
U.S.Congress,Senate.ThePatientProtectionandAffordableCareAct.2009.111thCong.,1stsess.,H.R.3590.
Availableathttp://frwebgate.access.gpo.gov/cgibin/getdoc.cgi?
dbname=111_cong_bills&docid=f:h3590eas.txt.pdf(accessedMarch112009).
ValueinHealth.SpecialIssue:HealthTechnologyAssessmentinEvidenceBasedHealthCareReimbursement
DecisionsaroundtheWorld:LessonsLearned.200912(s2):S1S53.
WilenskyGR.DevelopingaCenterforComparativeEffectivenessInformation.HealthAffairs.200625(6):w572
w585.[PubMed]
WilenskyGR.CostEffectivenessInformation:Yes,It'sImportant,butKeepItSeparate,Please!AnnalsofInternal
Medicine.2008148:96768.[PubMed]

ArticlesfromTheMilbankQuarterlyareprovidedherecourtesyofMilbankMemorialFund

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2980346/ 11/11