~~~..

~~~~~~~~~~~~~~~.~.
-.--. ... .. . ,....

WORLD BANK TECHNICAL PAPER NO. 334

\WrP33L0
Work in progress
for public discussion
9

Performance Monitoring
Indicators Handbook

AGRICULTURE
4 - i',
3g1, AIUSTMUENT
I
EDUCATION
ENVIRONMENT
FMANCIAI
SECTOR
- -x.t - : @HOUSING
AND
UR_AN
DEVELOPMENT
_.USTRY
AND
MINING
} - _ t ~~~~~~~~~~~~~OIll
AND
GAS
POPULAnON,
HEALTH.
* a _ i: ANDNUrRMON
? POVERTYREDUCTION

DEVELOPMENT
_ SECTOR

-1 ',Wiiun MANAGEMENT
TECHNICAL
ASSISTANCE
TELECOMMUNICATIONS
TRANSPORT
- AND
WASTEWATER

RobertolIosse
Leigh Ellen Sontheimer
RECENT WORLD BANK TECHNICAL PAPERS

No. 256 Campbell, Designand Operationof SmallholderIrrigationin SouthAsia

No. 258 De Geyndt, Managingthe Qualityof HealthCarein DevelopingCountries
No. 259 Chaudry, Reid, and Malik, editors, Civil ServiceReformin LatinAmericaand the Caribbean:Proceedingsof a Conference
No. 260 Humphrey, PaymentSystems:Principles,Practice,and Improvements
No. 261 Lynch, Provisionfor Childrenwith SpecialEducationalNeedsin the Asia Region
No. 262 Lee and Bobadilla, HealthStatisticsforthe Americas
No. 263 Le Moigne, Subramanian, Xie,and Giltner, editors, A Guideto theFormulationof WaterResourcesStrategy
No. 264 Miller and Jones, Organicand Compost-Based GrowingMediaforTreeSeedlingNurseries
No. 265 Viswanath, BuildingPartnershipsforPovertyReduction:The ParticipatoryProjectPlanningApproachof the Women's
EnterpriseManagementTrainingOutreachProgram(WEMTOP)
No. 266 Hill and Bender, Developingthe RegulatoryEnvironmentforCompetitiveAgriculturalMarkets
No. 267 Valdes and Schaeffer,SurveillanceofAgriculturalPricesand Trade:A Handbookforthe DominicanRepublic
No. 268 Valdes and Schaeffer,SurveillanceofAgriculturalPricesand Trade:A HandbookforColombia
No. 269 Scheierling, OvercomingAgriculturalPollutionof Water:TheChallengeof IntegratingAgriculturaland
EnvironmentalPoliciesin the EuropeanUnion
No. 270 Banerjee,RehabilitationofDegradedForestsin Asia
No. 271 Ahmed, Technological Developmentand PollutionAbatement:A Study of How EnterprisesAre FindingAlternativesto
Chlorofluorocarbons
No. 272 Greaney and Kellaghan, Equity Issuesin PublicExaminationsin DevelopingCountries
No. 273 Grimshaw and Helfer, editors, VetiverGrassforSoil and WaterConservation,LandRehabilitation,and Embankment
Stabilization:A Collectionof Papersand NewslettersCompiledby the VetiverNetwork
No. 274 Govindaraj, Murray, and Chellaraj, HealthExpendituresin LatinAmerica
No. 275 Heggie, Managementand FinancingofRoads:An Agendafor Reform
No. 276 Johnson, Quality ReviewSchemesfor Auditors:TheirPotentialforSub-SaharanAfrica
No. 277 Convery, Applying EnvironmentalEconomicsin Africa
No. 278 Wijetillekeand Karunaratne, Air Quality Management:ConsiderationsforDevelopingCountries
No. 279 Anderson and Ahzned, The CaseforSolarEnergyInvestments
No. 280 Rowat, Malik, and Dakolias, JudicialReformin Latin Americaand the Caribbean: Proceedingsofa WorldBankConference
No. 281 Shen and Contreras-Hermosilla, Environmentaland EconomicIssuesin Forestry:SelectedCaseStudiesin Asia
No. 282 Kim and Benton, Cost-BenefitAnalysisof the Onchocerciasis ControlProgram(OCP)
No. 283 Jacobsen, Scobie and Duncan, Statutory Interventionin AgriculturalMarketing:A New ZealandPerspective
No. 284 Valdes and Schaeffer in collaboration with Roldos and Chiara, Surveillanceof AgriculturalPriceandTradePolicies:A
HandbookforUruguay
No. 285 Brehm and Castro, TheMarketfor WaterRights in Chile:MajorIssues
No. 286 Tavoulareas and Charpentier, CleanCoalTechnologiesfor DevelopingCountries
No. 287 Gillham, Bell,Arin, Matthews, Rumeur, and Hearn, CottonProductionProspectsforthe Next Decade
No. 288 Biggs,Shaw, and Srivastiva, Technological Capabilitiesand Learningin AfricanEnterprises
No. 289 Dinar, Seidl, Olem, Jorden, Duda, and Johnson, Restoringand Protectingthe World'sLakesand Reservoirs
No. 290 Weijenberg,Dagg, Kampen Kalunda, Mailu, Ketema, Navarro, and Abdi Noor, StrengtheningNationalAgricultual
ResearchSystems in Easternand CentralAfrica:A FrameworkforAction
No. 291 Vald6s and Schaeffer in collaboration with Errazuriz and Francisco, SurveillanceofAgriculturalPriceand Trade
Policies:A HandbookforChile
No. 292 Gorriz, Subramanian, and Simas, IrrigationManagementTransferin Mexico:Processand Progress
No. 293 Preker and Feachem, MarketMechanismsand the HealthSectorin Centraland EasternEurope
No. 294 Valdesand Schaeffer in collaboration with Sturzenegger and Bebczuk, Surveillanceof AgriculturalPrice
and TradePolicies:A HandbookforArgentina
(List continues on the inside back cover)
WORLD BANK TECHNICAL PAPER NO. 334

Performance Monitoring
Indicators Handbook

RobertoMosse
Leigh Ellen Sontheimer
The WorldBank
Washington,D. C.
Copyright © 1996
The International Bank for Reconstruction
and Development/THE WORLDBANK
1818H Street, N.W.
Washington, D.C. 20433,U.S.A.

All rights reserved

Manufactured in the United States of America
First printing September 1996
Technical Papers are published to communicate the results of the Bank's work to the development
community with the least possible delay. The typescript of this paper therefore has not been prepared in
accordance with the procedures appropriate to formal printed texts, and the World Bank accepts no
responsibility for errors. Some sources cited in this paper may be informal documents that are not readily
available.
The findings, interpretations, and conclusions expressed in this paper are entirely those of the
author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations, or
to members of its Board of Executive Directors or the countries they represent. The World Bank does not
guarantee the accuracy of the data included in this publication and accepts no responsibility whatsoever
for any consequence of their use. The boundaries, colors, denominations, and other information shown on
any map in this volume do not imply on the part of the World Bank Group any judgment on the legal
status of any territory or the endorsement or acceptance of such boundaries.
The material in this publication is copyrighted. Requests for permission to reproduce portions of it
should be sent to the Office of the Publisher at the address shown in the copyright notice above. The
World Bank encourages dissemination of its work and will normally give permission promptly and, when
the reproduction is for noncommercial purposes, without asking a fee. Permission to copy portions for
classroom use is granted through the Copyright Clearance Center, Inc., Suite 910, 222 Rosewood Drive,
Danvers, Massachusetts 01923, U.S.A.
The complete backlist of publications from the World Bank is shown in the annual Index of Publications,
which contains an alphabetical title list (with full ordering information) and indexes of subjects, authors,
and countries and regions. The latest edition is available free of charge from the Distribution Unit, Office
of the Publisher, The World Bank, 1818 H Street, N.W., Washington, D.C. 20433, U.S.A., or from Publica-
tions, The World Bank, 66, avenue d'Iena, 75116 Paris, France.

ISSN: 0253-7494

Both authors work in the Bank's Operations Policy Group in the Operations Policy Department. Robert Mosse is
operations adviser; Leigh Ellen Sontheimer is a consultant.

Library of Congress Cataloging-in-Publication Data

Mosse, Roberto, 1944-
Performance monitoring indicators handbook / Roberto Mosse and
Leigh Ellen Sontheimer.
p. cm. - (World Bank technical paper ; no. 334)
Includes bibliographical references.
ISBN0-8213-3731-9
1. World Bank. 2. Economic development projects-Evaluation.
1. Sontheimer, Leigh Ellen, 1964- . II. World Bank. III. Title.
IV. Series.
HG3881.5.W57M685 1996
338.9'0068'6-dc2O 96-27346
CIP
CONTENTS

FOREWORD V

ACKNOWLEDGMENTS VI

PERFORMANCE MONITORING INDICATORS

Whatare performance monitoring indicators? 1
Why are performance monitoring indicatorsimportant? 1
PortfolioManagementTaskForcefindings 1
OEDstudies 2
Next StepsAction Plan 2
Whatare theadvantages of performance monitoring indicators? 3
Howare performance monitoring indicatorsdeveloped? 4
Thelogicalframework 4
Antecedentsto the logical framework 5
Importanceof clarifyingassumptions 7
Hierarchyof objectivesand the link to performcnceindicators 8
Organizing the hierarchyof objectives 8
Typesof performanceindicators 11
Resultsindicators 11
Riskindicators 14
Efficacyindicators 14
Waysof measuringperformance 15
Directmeasures 15
Indirectmeasures 15
Earlypointers:intermediateand leading indicators 15
Quantitativeand qualitativemeasures 16
Measurementscope 17
Specialstudies 17
Generalprinciplesfor selectingindicators 18
Problemswith definingindicators 18
Problemswith measuringimpact 19
Indicatorbenchmarks and internationalcomparators 19
Howdo performance monitoring indicators
affecttheBank'swork? 19
Economicand sectorwork 19
Projectcycle 20
Projectidentification,preparation,and preappraisal 20

iii
 Appraisalandnegotiations 21
Implementation andsupervision 21
Evaluation 21
Performance indicators
andprojectmanagement 21
Field-level
implementers 22
Implementation unitmanagers 22
BorrowerofficialsandBankstaff 23
Somerelatedissues 23
Goodpracticein monitoringandevaluation 23
Datacollection and management 24
Institutional
arrangements
andcapacitybuilding 25
Feedback frommonitoringandevaluation
effortsandinterpreting
indicators 25
Relationtoloancovenants 26
Workingat crosspurposes 26
Lastword 27
Notes 27

SECTORAL NOTES ON INDICATORS 28

Whattopicsdo thenotescover? 28
Howwerethenotesdeveloped? 29
Howare indicators
selected
fromthenotes? 29
Wherearethenotesavailable? 29
Howto learnmore 29

EXAMPLES OF INDICATORS 3 1
Table1 HondurasBasicEducation Project
(SAR13791-HO, March8, 1995) 32
Table
2 Indonesia
RuralElectrification
Project
(SAR12920-IND, February3, 1995) 33
Table
3 Indonesia:
Second Agricultural
ResearchManagement
Project
(SAR13933-IND, April21, 1995) 35
Table4 Lithuania
SiauliaiEnvironmentProject
(SAR14981-LT,November 9, 1995) 37
Table5 PeruRuralRoadsRehabilitationandMaintenance
Project
(SAR14939-PE, November 6, 1995) 38
Table6 ThePhilippines
Women's HealthandSafeMotherhood
Project
(SAR13566-PH, January27, 1995) 40
Table7 VenezuelaAgricultural
ExtensionProject
(SAR13591-VE,March7, 1995) 42
Table8 Chad:Structural
Adjustment Credit
(President's
ReportP-6785-CD) 43
Table9 Morocco:FinancialMarketsDevelopment Loan
(President's
ReportP-6633-MOR) 45

iv
FOREWORD

As part of ongoingeffortsto improvethe qualityand impactof itswork, the World

Bankis placingnew emphasison theuseof performancemonitoringindicators.These
indicators,which are basedon a logical frameworkof projectobjectivesand end-
meansrelationships,help generatemorethoughtful,logicallyconstructedproject
designs.And becausetheyserveas benchmarksagainstwhich to measureproject
progresstoward developmentobjectives,theyresultin moremeaningfulprojectmoni-
toring and evaluation.
Over the posttwo yearsBankstaffhavedevelopednoteson suggested
performancemonitoringindicatorsfor eachof the mainsectorsin whichthe Bankis
active.Thesenotesoffer a frameworkfor useby taskmanagers,borrowers,and pro-
ject implementationunitsin analyzingthe relationshipbetweenobjectivesand moni-
torable outcomesand impacts.Theyalsooffer a menuof possibleindicators.
Thishandbook,which introducesand supplements the sectornotes,is divided into
threesections.Thefirstsectionexplainswhy menusof indicatorswere developed;
providesthe backgroundon the logical frameworkand typologyof indicators;
describeshow indicatorsare developedand applied in projectdesign,supervision,
and evaluation;and discussesimportantissuesrelatedto the meaningfuluseof indi-
cators.Thesecondsectiondescribesthe sectornoteson indicatorsand their useand
explainshow to get copies.Thethird sectionprovidesexamplesof performanceindi-
catorsdevelopedfor Bank-financed projectsand showshow the indicatorswere
developedon thebasisof each project'sdevelopmentobjectives.

Myrna Alexander
Director,OperationsPolicy
May 1996

V
ACKNOWLE DGME NTS

Thishandbookwas edited by PatriciaRogersof the World Bank'sOperationsPolicy

Departmentand Meto de Coquereaumont and PaulHoltzof AmericanWriting Corpo-
ration. LaurelMorais and ChristianPerez(AmericanWriting Corporation)laid out the
text. Theauthorsare gratefulto Gerald Britan(U.S.Agencyfor InternationalDevelop-
ment),MosesThompson(Teams Technologies,Inc.),UlrichThumm(OperationsEvalua-
tion Department,World Bank),and DeborahWetzel (Development Economics,World
Bank)for their valuablecontributionsand comments.
Theauthorsalsothankthe followingBanktaskmanagersfor preparingobjectives
and indicatorsmatrixes:Maria Dalupan,HennieDeboeck,DelyGapasin,JoseLuis
Irigoyen,RamaLakshminarayanan, DavidNielson,Arun Sanghvi,Anna Sant'Anna,
StanleyScheyer,Sari Soderstrom,and ChristopherWalker.

vi
PERFORMANCE MONITORING
INDICATORS

Whatareperformance
monitoring
indicators?

Performanceindicatorsare measuresof projectimpacts,outcomes,outputs,and

inputsthat are monitoredduring projectimplementation to assessprogresstoward
projectobjectives.Theyare also usedlater to evaluatea project'ssuccess.Indicators
organize informationin a way that clarifiesthe relationshipsbetweena project's
impacts,outcomes,outputs,and inputsand help to identifyproblemsalong the way
that can impedethe achievementof projectobjectives.

Whyareperformance
monitoring
indicators
important?

In recentyearsseveralimportantstudies-mostnotablythe PortfolioManagement
TaskForceReportand reviewsby the OperationsEvaluationDepartment(OED)-
havefound that the monitoringand evaluationof Bank-financed projectsdid not focus
adequatelyon the factorsthat are criticalfor positivedevelopmentimpact.Boththe
taskforce and OEDconcludedthat performanceindicatorsshouldbe integratedwith
the monitoringand evaluationproceduresusedby the Bankand itsborrowers.

ThePortfolioManagementTaskForceReport(alsoknownas the WapenhansReport) Porifollo

analyzedthe factorsthat affectthe developmentimpactof Bankoperations. 1 As part Management Task
of this analysisthe taskforce focusedon how evaluationmethodologies,includingthe Farcefindings
calculationof economicrate of returnand the projectrating methodology,were used
during projectappraisaland supervisionto enhancethe qualityof Bank-supported
projects.Thetaskforcefound that projectratingswere not providing implementation
units,borrowers,and Banktaskmanagerswith adequatefeedbackabout progress
toward developmentimpactfor severalreasons:

* Toomuchemphasiswas placed on the mechanics(physicaland financial)of pro-

ject implementation.
* Therisksand factorsthat mostinfluenceprojectoutcomeswerepoorly identified.
* Objectivecriteria,transparency,
and-since the methodologydependedon the
judgmentof individuals-consistencyacrossunitswere lacking.
* Ratingstendedto be overlyoptimistic.

1
 Withoutappropriate feedback, noneof thepartiesconcerned with projectoutcomes
couldmakeappropriate, informed decisionsaboutwhetherandhowto adjustproject
designor implementation arrangements to betterachievea project's
intended
objectives.A relatedproblemisthattheobjectives themselves areoftennotwellthought
outor clearlyarticulated,cloudingthedevelopment of appropriateperformance moni-
toringindicatorsandmakingmonitoring andevaluation evenmoredifficult.
Further,thetaskforcepointedoutthattheBank'straditionalmethodof appraisal
and evaluation of development impact-thecalculation of economic rateof returnor
netpresent value-cannotbe appliedto all projectsandthatforsomeprojects,a
meaningful cost-benefit
analysisis notfeasible.Evenfor projectsfor whichnetpresent
valueor economic rateof returnis calculated,
Bankpracticeis to calculateit onlya
fewtimes-duringappraisal,duringa midtermreview,uponrestructuring if
necessary, andat thetimeof theImplementation Completion Report.Neitherthecal-
culationsnorthecriticalvariablesthataffectthemaremonitored explicitlythroughout
implementation.Moreover, thereportpointedout,in manyinstances costsandbene-
fitscouldbe moreeasilyidentifiedandmeasured withperformance indicators.
Thetaskforceconcluded thattheBank'sprojectratingmethodology andsupervision
reportingsystemshouldbe adaptedto include theuseof projectperformance monitor-
ing indicators
derivedfroma project's development objectivesandimplementation plan.

OEDstudies TwoOEDstudiesreviewingtheBank'srecordon themonitoring andevaluation of

projectsreinrorced 2
thefindingsof thetaskforcereport. In fiscal1994 a studyassess-
ing twentyyearsof Bankprojects foundthatBankguidelines anddirectiveson moni-
toringandevaluation hadnotbeenfollowedadequately, eitherduringproject
appraisal(whenmonitoring andevaluation are planned)orduringimplementation.
Butthestudyalsofoundevidence thatthesituationwaschanging.
A follow-upstudyof monitoring andevaluation plansin a sampleof fiscal1995
projects suggests thatthequalityof suchplanninghasimproved. Theimprovement is
evidentin therisinguseof performance monitoringindicators-theshareof projects
with at leastsomeindicators rosefrom72 percentin fiscal1993 to 77 percentin fis-
cal 1995.Nevertheless, theexpandinguseof indicatorshasnotbeenmatched by
arrangements for datacollection or monitoringandevaluation capacity-building
effortsin eithertheBankor borrowingcountries. Relatively
fewprojects(14 percent
of thesamplereviewedby OEDin fiscal1995)achieveoverallgoodpracticein com-
prehensive designor useof monitoring andevaluation. And performance monitoring
indicators,althoughmorewidelyused,areweakin structure and usuallydo notfol-
low a logicalframework or a typology,andthereis notalwaysfollow-through on
datacollection.

NextSteps Inresponseto theseconcerns, Bankmanagement hasmadeeffortsto fostertheuseof

ActionPlan performanceindicators.
In theNextStepsActionPlan,whichwasdesigned to imple-
menttherecommendations of theWapenhans taskforce,management calledfor incor-
poratingperformance monitoring indicators
in theprojectratingsystemusedfor project
monitoring
(through Form590 andtheAnnualReporton Portfolio Performance), to

2
bettermonitorprogresstowarda project'sdevelopmentobjectives.Managementalso
recognizedthat the Bankneededto developsector-specific indicatorsto help borrow-
ersand Bankstaffdefineprojectobjectivesmorenarrowlyand to derivelogicalmea-
suresof projectoutcomesand impactin orderto measureachievement of project
objectives.TheBankalso neededto supportthe useof indicatorsin the revisedproject
rating system.Thereforethe actionplan calledfor the sectordepartments withinthe
CentralVice Presidencies indicatorsthatare mostrele-
to developsetsof sector-specific
vantto projectdesignand monitoring-the sectornoteson indicatorsdiscussedin the
secondpart of thishandbook.Staffwouldthenbe requiredto ensurethat key sector-
specificprojectimpactindicatorswere identifiedin projectappraisaldocumentation
and that progresswas monitoredagainstthesebenchmarks.

What are theadvantages

of performance indicators?
monitoring

Performance monitoringinvolvesperiodicallymeasuringa project'sprogresstoward

explicitshort-and long-termobjectivesand givingfeedbackon the resultsto decision-
makerswho canusethe informationin variouswaysto improveperformance(box 1).

Box1. Uses indicators

of performance
PtANNING.For any program or activity, Froma developmentproject to a sales plan, incorpo-
STPATEGIc
ratingperformance
measurement of the criticalassump-
intothe designforcesgreaterconsideration
indicatorshelp
and causalpaths.'Thusperformance
tionsthat underliethatprogram'srelationships
clarify the objectivesand logic of the program.
PERFORMANCE Performanceindicators can
ACCOUNTING. if they
helpinformresourceallocationdecisions
are usedto direct resourcesto the most successfulactivities and thereby promote the most efficient use
of resources.
FORECASTING
ANDEARLY DURING
WARNING PROGRAM Measuring progressagainst indicators
IMPLEMENTATION.
providingfeedbackthatcanbe usedfor planning,identifying
maypointtowardfutureperformance,
areasneedingimprovement,
and suggesting
whatcan bedone.
MEASURING
PROGRAM Good performanceindicators measurewhat a program has achievedrela-
RESULTS.
tiveto itsobjectives, thustheypromoteaccountability.
not justwhatit hascompleted;
MARKETING
PROGRAM ANDPJBUC Performanceindicators can be usedto demonstrateprogram
RELAnONS.
thevalueof a
datacan beusedto communicate
resultsto satisfyanexternalaudience.Performance
programor projectto electedofficialsandthe public.
Performance
BENCHMARKING. data againstwhichto measureotherprojectsor
indicatorscan generate
identifyinggood
Theyalso providea wayto improveprogramsby learningfromsuccess,
programs.
performers,and leaming from their experienceto improve the performanceof others.
QUALITY Performanceindicatorscan be usedto measurecustomer(beneficiary)
MANAGEMENT. satisfac-
tion, and thereby assesswhether and how the program is improving their lives.

Lessons
LeslieStewort,andJamieAdrian,'PerformanceMeosurement:
1. ThomasJ. Cook,JerryVanSant, Learned
Management,'WorldDevelopment
for Development 23181:1303-15
(19951.

3
 Forthepurposes of theBankanditsclientsthemostsignificant benefitsof performance
indicators accruein projectdesign(strategic planning), projectsupervision andmonitor-
ing (forecasting andprojectevaluation
results), (measuring resultsandqualitymanage-
ment).Theneededinformation anddatacollection egfortsbecome evidentasproject
objectives areformulated. Ina performance monitoring system, indicatorsserveastools
for measuring theflow ofchange.Baselines arethevaluesof performance indicators at
thebeginning of theplanningperiod;targetsarethevaluesat theend.
Thebenefits of indicators comefromtheirmeasurability andfromtheirdirectderiva-
tionfromprojectobjectives, whicharegroundedin sector, economic, risk,andbenefi-
ciaryanalysis.Indicators linka project's
specifically inputsandactivities withquantified
measures of expected outputs and impact.Beforeselecting indicators,theborroweror
projectimplementation unitandtheBankmustconsider whichmeasures of
performance will tellthemwhetherandhowa project's proposed objectives arebeing
achieved andwhowill benefit-thereby contributingto a moreprecise definition of the
objectives.Borrowers andtheBankmustalsoaskwhetherthenecessary dataareavail-
ableanddecidewhatusersshoulddo in response to theindicatoroutcomes.
With indicators, monitoring andevaluation is morecompelling because it isobjec-
tive,notbasedon personal judgments or puredescription. Furthermore, indicators
holda project'sultimate goalsclearlyin viewthroughout implementation. If designed
and usedcorrectly, indicatorsmeetthespecificinformation needs and scope of
authorityof all thepartiesconcerned withimplementation-field staff,implementation
unit,borrower,taskmanager, and Bank management. The implementation unitandits
subunits are mostinterested-andin a betterpositionto respondto-indicatorsof
inputs,riskfactors,andoutputs.TheBankanditsborrowers are mostinterested in
indicators of output,outcome, anddevelopment impact.Thustheindicatorshelpall
partiesfocuson theareasof greatest concern them.
to

indicatorsdeveloped?
monitoring
Howare performance

of individualprojects.3
Performance indicators mustbe basedontheuniqueobjectives
Butanysetof performance shouldalsobe basedon an underlying
indicators logical
framework thatlinksprojectobjectiveswith project components and theirrespective
implementation
andoutputsat different
inputs,activities, stages.Theframework is
objective-driven,sinceanyaction undera project should be aimed at achieving its
objectives.Giventheproject'sdevelopment objectives, thebestmixof outputs to
achievetheseobjectives andcomponents that will yieldthese outputs are derived. A
generalschematic of thepointofviewof thelogicalframework is shownin box2.
Understanding howto deriveperformance indicators from a project'sobjectives and
components requires someunderstanding of theconcept of thelogicalframework.

TheIogiclThelogicalframework isa methodology projectsandan analytic

for conceptualizing
frmework toolthathas thepower to communicate a complex projectclearlyand understand-
ably on a singlesheetof paper.It isa participatoryplanningtoolwhosepower
dependson howwellit incorporates thefull rangeof viewsof intendedbeneficiaries

4
 Box2. Pointofviewofthelogicl
framework

DEvELoWaq
OSECUnE
Describes
the proiect'srealoutcome-theimpacthattheproject'soutputswill hae on the
beneficiary,institution,
or system
in termsof changedbehavioror improvedperwformrc.
Thedevelopment
objectivedefinestheprojectssuccess.

PROJECT
OUuws

Definewhattheprojectcan6e helddirectlyaccountable
for producing-theprojeds deliv-
erables thegoodsand services
it will produce.Typically,
outputsare indepwdnt, synergis-
tic, andintegrated.

Clusters
of ac6ivifies
that rdefine
howvtheproductsand services 6e deli-YOO(hchnical
Wfill
ossistance,
physicalinfrastructure,
and the lilke).

Source:TeamTechnologies,Inc.

and otherswho have a stakein the projectdesign. It is bestusedto help project

designersand stakeholders:

e Setproperobjectives
* Defineindicatorsof success
Identifykey activityclusters(projectcomponents)
* Define critical assumptionson which the projectis based
* Identifymeansof verifyingprojectaccomplishments
* Defineresourcesrequiredfor implementation.

As an up-frontplannerthe logical frameworkcan be usedto helpdesigntoolsfor

projectimplementation and evaluation.Knowingits advantagesand limitationshelps
in assessingthevalue of the logical frameworkmethodologyat variouspointsin the
projectcycle (box 3).

Antecedentsto the logical framework

Thelogical frameworkcan improvethe identification,preparation,and performance
appraisal processby clarifying a project'sdesignand makingit transparentto the
borrower,the lender,and beneficiariesand otherstakeholders.Theframework
assumes that projectsare instrumentsof changeand that they are selectedfrom

5
 Ibx3. Advantages
and
lihitations
ofusing
thelogkal
framework
method
ADVANTAGES LMTA11ONS
* Ensuresthatfundamental queshonsareasked * May giveriseto rigidityin projectadministra-
andweaknesses are analyzedin orderto provide tionwhenobjectives andextemalfactorsspecified
decisionmakerswith betterand morerelevant at the outsetare overemphasized.
Thiscon be
information. avoidedby regularprojectreviewsat whichthe
keyelements canbe reevaluated
and adjusted.
* Guidessystematicandlogicalanalysisof the
interrelated
elementsthatconstitute
a . As a generalanalytictool,is policyneutralon
well-designedproject. questionsof incomedistribution,employment
opportunities,accessto resources,local
* Improves
planning6y highlightinglinkages participation,costandfeasibilityof strategies
and
technology,
andeffectson theenvironment. betweenprojectelements andextemcal factors.

* Facilitatescommonunderstanding andbetter . Isonly oneof severaltoolsto be usedduring

communication betweendecisionmakers, managers,projectpreparation,implementation, and
andotherpartiesinvolvedin the project. evaluafion.It doesnot replacebeneficiary
analysis;fimeplanning;economic,financial,and
* Usedalongvith systematic monitoring, ensures cost-benefitanalysis;environmentalimpact
continuityof approachwhenoriginalprojectstaff assessment; or similartools.
are replaced.

* Mayfacilitatecommunication
between
govemments anddonoragenciesonceit has
beenadoptedby moreinstitutions.

. Makesit easierto undertakesectoralstudiesand

comparativestudiesin generol,if usedwidely.

amongalternativesas the mostcost-effective

way of achievinga desiredoutcome.It
bringstogetheras its antecedents
severalprojectmanagementperspectives:

* Results-oriented
management.Projectsbeginwith a setof objectivesratherthan
components,and successis measuredby the degreeto which developmentobjec-
tives havebeenmet.Managementis heldaccountablefor results.
* Basicscientificmethod.Projectsare experiments undertakenin an uncertainworld.
Fromthispoint of view a projectis a structuredprocessfor learningabout what
producesresults.Thisperspectiveassumes that projectsare learningsystems.
* Systems analysis.A project,like a system,is notdefineduntilwe havedefinedthe
largersystemof whichit is a part.Projectstakeplacenot in a vacuumbut in a rela-
tionship with an external environmentof organizations, institutions,and other projects.
* Contractlaw. Everycontract(project)hasthesamebasicfeatures:
. A setof deliverables(outputs)
. Circumstances thatconstituteforcemajeure(assumptions)
. A meetingof the mindsabout what thedeliverableswill produce
(development objective).
* Causeand effect.Thecoreconceptunderlyingthelogicalframeworkis causeand
effect.Thebetterthecauseand effectlinkagesbetweenobjectives,the betterthepro-
jectdesign.Bydefinition,eachprojecthasthisif-thenor causeand effectlogic
embeddedin it. If theprojectproducescertainoutcomes undercertainconditions,

6
 thenit canbe expected to achieve certainotheroutcomes. Forexample, if theproject
suppliesfarmers withimproved seedandputsa creditsystem in place,andassuming
thereisadequate rainfall,thenproductionwill increase.
Thelogicalframework forces
projectplanners to makethiscauseandeffectlogicexplicit,butit doesnotguarantee
a gooddesign.Thevalidityof thecauseandeffectlogicdepends onthequalityand
experience of thedesignteam.

Importance of clarifyingassumptions
Assumptions are riskandenablingfactors-external conditionsthatare outsidethe
directcontrolof theproject(figure1).Achieving objectives
candependon whether
assumptions holdtrue.Assumptions aremadeaboutthedegreeof uncertainty
(degreeof risk)betweendifferentlevelsof objectives. Thelowertheuncertainty that
certainassumptions will holdtrue,thestronger theprojectdesign.Failingassumptions
canderaila projectasoftenaspoorlyexecuted outputs.
If causeandeffectrelationships are thecoreconceptof goodprojectdesign,nec-
essaryand sufficient conditions are thecorollary.Thecauseand effectrelationships
betweenlevelsin theproject'shierarchy of objectives(inputsto outputs,outputsto out-
comesand impact,impactto relevance) describethenecessary conditionsfor achiev-
ing development objectives (fordefinitionof termsseepages12-13). Thisis the
internallogicof theproject.
Alsoimportant arethesufficient conditions at eachlevelfor achievingthenext
higherlevel(conditions in whichthenextobjectivecanbe met).Thesufficient condi-

Figure external
1. Assumptions/risks
obout factors
must
also
bemade
expliet

Program jectve Assumptions

caegory/Couny Risks
Assislonce
Slrabgy Risks

Development Assumptions/
objective Risks

/ Otput s tAssumptions/

Components t Assumplios/\
~Ri
spks

IIF | >AND \

Note: Thisis a simplifiedrepresentationof the logical framework

(shownlater in box 4), highlightingthe
importanceof assumptions and risks.Thetwo centrolcolumns(blankherelwould normallycontain
performanceindicatorsand monitoringand supervision for verifyingeachindicator.
measures
Source:TeamTechnologies, Inc.

7
tionsbetweenlevelsin thehierarchyof objectives Thisis the
are theassumptions.
hierarchy(necessary
externallogicof theproject.Theobjectives plusthe
conditions)
assumptions together
conditions)
(sufficient giveonea muchclearerideaof thepro-
ject'sdesign.Thisiswhatthelogicalframework graphicallyrepresents.

Hierarchyof objectives and thelinkto performance indicators

Usersof thelogicalframework havecometo agreeona common setof terms(out-
comes,results,targets,indicators, outputs,goals,achievements, development objec-
tives).Thesetermsareessentially waysof referringto or classifying objectives.
Thelogicalframework assumes thatthereareseverallevelsof objectives in a pro-
ject(ahierarchyof objectives). Thoughthereisno logicallimitto thenumberof lev-
els,mostprojectteamsfind it difficultto managemorethanfourlevelseffectively. The
logicalframework typicallyspecifies four.TheWorldBankusesindicators predicated
on a framework thatspecifies threelevelsof objectives: inputsfor projectactivities,
outputsof projectactivities,andoutcomes and impacts.FortheWorldBank's
purposes, inputsandoutputs correspond strictlyto projectmanagement, while
outcomes andimpactscorrespond to theproject'sdevelopment objectives.
TheCountryAssistance Strategy reflectstheagreement betweentheBankandits
borrowers on a country's overalldevelopment goalsandtheanticipated means-
projects,economic andsectorwork,technicalassistance-fortheBank(andother
financiers)to helpachievethem.Projects mustshowhowtheirobjectives are relevant
to therealizationof overallcountrydevelopment goals.
Forany projecta setof performance indicators shouldbe designed withinthelogi-
cal framework. The development of performance indicators beginswith theproject's
objectives andreflects theassociated hierarchy of activities andtheiroutputs and
intendedoutcomes for each project component. The activities
conducted and the
resultsachieved at lowerlevelsof objectives are inputstowardtheachievement of
higher-levelprojectobjectives, at theinstitutional, sectoral,program,or countrylevel. 5

Thedefinitionof indicators at eachlevelthushingeson theproject'sultimate objec-

tive,whichcanbe modestIforexample,to reducetheincidence of preventable dis-
easewithina givenpopulation by increasing immunizations) or moreambitiousin
scale(toreducechildmortality).

Organizingthehierarchyof objectives
Thelogicalframework presents schematicallythehierarchyof projectobjectives, the
performance for measuring
indicators theachievement of eachobjective,themeans
of verifyingeachindicator,andtheassumptions (riskandenablingfactors)criticalto
achievingthenextobjectivelevel.Box4 illustrates a samplelogicalframework for a
nutritionandchilddevelopment projectin Uganda.
Therelationshipsamongprojectobjectives-andtheneedfor performance informa-
tion-can bealsobe clarifiedbygraphically depicting theoverallprogramlogicand
performance expectations tree(figure2).Thetreebeginswiththeover-
in an objectives
archingdevelopment objectives of a project(consistent of theCoun-
withtheobjectives
try Assistance liststhelower-level
Strategy), outputs throughwhichtheseobjectives are

8
Box4. Logkaifromework
fortheUganda
Nutrition
andChildDevelopmen
Prsct
NARRA suMRY
I PEWORMANCNDICATORS MONTOING AND RSUbSIO4N ASSmwPT AmD mS6
IMPACT (impactto relevance)
* Improvedhealth,nurition, * Reduced
prevalence of protein * Noainal anthropomeiric * Oiler nationalprogramsin heath
andcogniivestatusof young energymalnutrition-measured by survey (Immunizoaion, primaryhealhcore),
children underweight
preschoolchildren- food production, primaryeducaionare
from25%to 13%by 2003 sustained; synergy with then programs
will achieve higherprojedimpoct
Reduced micronuirient
deficiency * Demogrophicandhealth
problems(IDMprevalence reduced survey
by 50%in endemicareas,anemia
in childrenreducedto 33%,wormlood
reducedto 50%,andxeophthalmia
reducedio 2%by 2003)
* Netenrollment in grode 1 * Schoaenrollment
records
increased to 80%by 2003

OUTCOMES (outcometo impoct)

* Improvedchildcare,nutrition, . Numberof childrenreachedby . Annualprojec report * UNICEF maintainsinterestin
andhealthservices services partnership
withprojed

* Improvedinoomegeneraion * NumberoFwomenreached * Annualprojedreport . Governmensignschildbill into w

copocityforwomen by services * Partnership
withotherdonors
L sustained

OUTPUTS (outputtooutcome)
* Established
community-based * Numberof childrenandwomen * Projedc
monitoringsurvey, * Decentralizoaion
policyimplemented
childcareprogram who participatedin program household surveys
* Numberof healthday * Project
monitoring
outreacheffortslounched reports
* Numberof ECDdaycorecenters * Projedc
monitoringreports
established

* Established
micronutrient * NumberoFvitaminA capsules, . Projecmonitaringreports * Distridgovernments
continuesupport
program iron tabletsdistributed
. Percentgeof saltiodized * Surveysompleof swtsold

* Established
women'sincome * Numberof womenwho received * Projedmonitaringreports
component trainingin skillsfor livelihood/
childcareenterprise

ACTIVITIES INPUTS (octivityto output)

* Community-6asedgrowth * btal $42 million,including . Adiviies will 6everifiedthrough * Sufficient interestfromcommunities
a monitoringprograms operations
and maintenance and regularprojedmonitoring
monitoringand evaluation
* Heoah-dayoutreochprograms
* MatchinggrantsFr oommunity
ECDcenter
* Massmediacampaigns

* Grontsfor women'slivelihood * Mothersapplynewskillsand

capocitybuilding knowledge

* VitaminA capsules,
irontablet
distributed
* Iodizingof salt
a Deworming compoign

TaskManager: Marib Garcia.

9
Figure
2.Partial troeforahypothetkal
oblectlves chiM proFgm
health

OBJECTIVE Reducechildmortality

Reoding downansvers
IMPACTS Reduced Improved Improved family Expanded thequestions whyand
incidenceof nutrition healthpractices availoble how? -whoat dowe
needto do to achiev
preventable care thisresult?
disease

Increased polio
OUTCOMES Increased
other Increaseduseof
immunizations immunizations oral rehydration

Reading
across
answers
OUTPUTS Expanded Increased Improvedpublic the eshonwhoatese?
school availability
of awareness =at dditional
things

I=
immunizations vaccines
=
doweneed
achieve todo to
thenext
=aobjecthve?

ACTIVITIESIncrase Improve Expand

supplies training facilities

upanswers
Reading the
question
'sawhato?
-what is the
INPUTS Provide Enforce Provide significonce
af our
funds regulations vaccines accomphdshments?

Notethat thisis a partial objectivestree.Only the elements in bold are expandedin thefigure to showthe
variousactivitiesand the intendedoutcomes, outputs,and inputsneededto achievethe projectobjective.
Thesometreatmentcan be applied to all elements in the figureto completethe objectivestree.
Source:AdoptedfromGeraldM. Britan,'Measuring ProgramPerformance for FederalAgencies:Issues
and Opfionsfor Performance Indicators'(Washington,D.C.: U.S.GeneralAccountingOffice).

achieved,and showsthe specificinputsrequiredto undertakeprojectacfivities.Using

an objectivestreeto representa project'sgoals helpsverifythe logic of projectdesign
and confirmthatthe right indicatorsweredefinedto measureperformance.
An objectivestree shouldincludeall the lower-evelresultsthat are necessarycon-
ditionsfor achievinghigher-levelobjectives.Thatincludesexternalfactorsand
assumptions about conditionsin the wider projectenvironmentthat do not actuallyfall
underthe controlof the project-the externalrisk and sensitivityfactors(seepages7
and 14).6 Theseaspectsmustbe monitoredthroughoutthe projectalong with
impacts,outcomes,outputs,and inputs,and testedduring implementation to ensure
that the assumptions remainvalid.
In theexampleof the partialobjectivestreeshownin figure 2, eachof the lower-level
resultsis dependentnot only on the factorsand assumptions shownin the figurebut
alsoon othersthat are not shown.Thesecould be the responsibility of the projectshown
in the diagram,with the detailssuppressed herebecauseof spacelimitations.Butthey
couldalso be the responsibilityof anotherprojector projectsor of the government,
makingthema partof the assumptions and externalriskfactorsunderlyingthe project.

10
Forexample,"expanded schoolimmunizations" depends, asshown,on "increasing
supplies,""improving training"and"expanding facilities"all,
according to thefigure,
responsibilities
of theproject.It alsodepends on "increased availabilityofvaccines"
and"improved publicawareness," whichcouldbe partof theproject(though thefigure
doesnotshowthat)or partof anotherprojectandtherefore partof theproject's
assumptions andexternalriskfactors.
A toolto conceptualize, design,andappraiseprojects, suchasan objectives tree
or similaranalyticaltool-and theperformance indicators derivedfromit-should not
be considered unchangeable. 7 It shouldbe continually reevaluated duringeach
phaseof projectpreparation, implementation, andevaluation. If theresultsanalyzed
duringprojectimplementation pointto a needto changethemeansof achievingpro-
jectobjectives,themixof inputs(activities to be undertaken) anddefinitionof outputs
shouldbe adjustedandnewindicators derivedto measure thenewlydefinedtargets.
Thegeneralstepsthatgo intoformulating projectobjectives andconstructing the
logicalstructureof a projectare important aspects of theidentification andprepara-
tionstagesof theBank'sprojectcycle(box5).8

Therearedifferentwaysto measure performance for anygivenvariable(objective, Typesofperformance

impact,outcome, output,input).A systemof indicatorsanchoredin thelogicalframe- indicators
workapproachis modeledin figure3. Whilefigure3 showsa comprehensive system
of indicators
in orderto providea complete frameof reference, theelements
of the
system thatare notcoveredin detailby thishandbookareshownin dottedlines.
These elements areincludedin OED'sevaluations of projectperformance.

Results
indicators
Results
indicatorsmeasure projectresults
relativeto projectobjectives.
Resultsare
measured at theleveldeterminedby a project'sobjectives.Remember thatfollowing
thelogicalframework approach,theyshouldbedefinedstartingwiththeimpactand
outcome indicators(andworkingbackwards to theinputindicators).

Inputindicators
INPUTINDICATORS. measurethequantity(andsometimesthequality)of
resourcesprovidedforprojectactivities.
Depending
on theproject,thesecaninclude:

* Funding-counterpartfunds,Bankloanfunds,cofinancing, grants
* Guarantees
* Humanresources-number of person-yearsfor members of theimplementation
unit,
consultants,
andtechnicaladvisers
* Training
* Equipment,materials,
andsupplies,or recurrent costsof theseitems-forexample,
textbooks,
syringes,
vaccines,classroom facilities.

Outputindicatorsmeasurethe quantity(andsometimes
OUTPUTINDICATORS. the qual-
ity)of thegoodsor services
createdor providedthroughtheuseof inputs.Depending
on theproject,thesecanincludesuchelements as:

11
 Box5. Processes
withinthelogical
framework
STEPI PROBLEM
AND BENEFICIARY
ANALYSIS
ThisanalysisaHtempts to answerseveralquestions: What is the problemat stake,and who is involved
in it?What are the needs,expectations, intentions,andmotivationsof thevariousstakeholders,disag-
gregoted6y genderand otherrelevantsocialcriteria?Doestheproblemrequireextemaldevelopment
assistance,
or couldit be resolvedsomeotherway2Thisstepis normallypart of economicand sector
workor of the projectidentificationand preparationexercisewhenthereis noprior economicand
sectorwork, and it contributesto theformulationof the CountryAssistanceStrategy.
STEP2 OBJECTIVES ANALYSIS: CONSIDERATION OF ALTERNATIVESAND PERFORMANCE EXPECTATIONS
Thisstepidentifiesneedsor problemsand transforms themintosolutions-specific, quantifiedobjec-
tives.Eachobjectivemaybebrokendowninto subobjectives and outputsto be achievedat different
stagesof the project.Therearealmostalwaysalternative waysof meetinga projectsobjectives, and
themethodchosenwill affectthe configuration of outputs.Thusconsideration of alternatives
and selec-
tion of themostappropriateis part of thisstep.Consideration oFaltematives entailseconomic,riskand
sensitivity,
cost-effectiveness,
environmentaland socialanalyses,andbeneficiaryconsultation.
Basedon theobjectives analysis,themainprojectelements-comprising thewholelogicalchain,
includingsectorand projectobjectives, plannedoutputs,necessary activities,and theirinputs-can be
graphicallyrepresented in thelogicalframeworkor an objectives tree (seefigures1 and 2).
Eachelement-inputs,activities.,outputs,outcomes, and impacts, and theriskand enablingfactors
thataffectthern-is measuroble. Thisis whereindicatorscomein. Indicatorsthatdefineand measure
eachof theseelements are identifiedduringtheobjectivesanalysisandprojectplanningstage.For this
reasonit is essential
to formulateandexpressprojectobjectives andoutputsin a waythatallowsmea-
surement of bothshort-term and longer-term results.Definitionof anoutput,outcome,or impactobjec-
tiveshouldinclude:
* Targetgroup(for whom)
* Quantityor level(howmuch)
* Quality(in termsof designstandards and environmental or otherimpact)
* Time(whenit shouldbe accomplished)
* Location(whereit shouldhappen)
STEP3 FINALIZATION
OF PROJECT
DESIGNAND INDICATORS
In thisstepplannerscarefullyexaminethe projectto ensurethatall itselements are logicallyrelated.
Planners alsoassesstheintegrityof indicators
and realismof targets at thisstage,takingintoaccount
all projectassumptions
and baseline data, andfinalizetheir plans.

* Clientsvaccinated(by a healthproject)
* Farmersvisited(anextensionproject)
* Miles of roads built (a highwayproject)
* Electricitygenerationand transmission facilitiesinstalled(a rural electrification
project)
* Pollutioncontrolmeasuresinstalledor incentivesor regulationsenforced(a pollu-
tion controlor air or waterquality improvement project).

OUTCOME
ANDIMPACT
INDICATORS.
Outcome and impact indicators measurethe quan-
tity and qualityof the resultsachievedthroughthe provisionof projectgoodsand ser-
vices.Dependingon the kind of project,thesecan include:

12
igure
3. Asystem
ofindicators

RISK
INDICATORS DIRECT
INDICATORS EFFICACY
INDICATORS

Relevance
Riskor enabling factors -- - --------- Sustainability

Riskor enabling factors Outcomesand impacts Effectiveness

Riskor enabling factors Outputs Efficiency

Inputs

------ Explicit
development
of thesekindsof indicators
is notrequiredforWorldBankprojects.

* Reducedincidenceof disease(throughvaccinations)
* Improvedfarmingpractices(throughextensionvisits)
* Increasedvehicleuseor trafficcounts(throughroad constructionor improvement)
* Increasedrural supplyand consumptionof electricity(throughexpansionof elec-
tricity network)
* Reducedmortalityor lower healthcosts(throughimprovedfamily healthpractices
or improvednutrition,or cleanerair and water)

INDICATORS.Someprojectshaveintendedimpactson higher-orderobjec-
RELEVANCE
tivesthat are not capturedby directoutcomeindicatorssuchas the onesdescribed
above.For instance,someprojectshavenationalor sectoralobjectives,and for them
impactmustbe measuredat thoselevels.Projectsmayalso haveunintended-often
negative-impacts.Thesetoo can be capturedby evaluationstudies.Relevanceindi-
catorsmeasuretrendsin the wider policy problemsthat projectimpactsare expected
to influence.If appropriateto the project,theseindicatorsshouldbe used.Depending
on the project,thesemayinclude:

* Improvednationalhealthas measuredby healthindicators(throughimproved

healthcare, healthsystemperformance)
* Increasedfarmprofitsand reducedfood costs(throughimprovedfarmingpractices)
* Reducedtransportationcostsand expandedeconomicdevelopment(throughroad
constructionor improvement)
* Improvedeconomicgrowthand enhancedconsumerwell-being(throughexpanded
electrification,pollutioncontrols,and othernew technology).

A systemof indicatorsdrawn fromthe logical frameworkmeasuresthe relevance

of projectresultsas well as outcomesand impacts(seefigure 3). However,such
effectscan be very difficultto attributeto individualprojectresults.For the Bank's
monitoringpurposes,therefore,measurement of resultsstopswith project impactrel-
ative to projectobjectives.Thatsaid, as mentionedabove, relevanceindicatorsare
appropriatefor objectivesthat are sectoralor national in scope.

13
Riskindicators
Riskindicatorsmeasurethe statusof the exogenousfactorsidentifiedas critical
throughthe riskand sensitivityanalysis(riskand enablingfactors)performedas part
of a project'seconomicanalysis.Theseare the factorsthat are determinedto be the
mostlikely to havea directinfluenceon the outcomeof variousaspectsof the project
(for example,economicpricesfor power or competitivesalariesfor projectstaff)-the
assumptions that are madeabout conditionsexternalto the project.A project'sobjec-
tivescan only be achievedif the logical means-endrelationshipof the project
elementsis secureand the externalriskfactorsare favorable.

Efficacyindicators
Efficacyindicatorsshow how well the resultsat one level of project implementation
havebeen translatedinto resultsat the nextlevel: the efficiencyof inputs, effective-
nessof project outputs,and sustainabilityof project impact. They measurea pro-
ject'sefficacy in achieving its objectives,ratherthan its results.The logical
frameworkapproach sometimesusesthesemeasuresin addition to direct indica-
tors of results.
For the Bank'smonitoringpurposes,the measurement of resultsusuallystopswith
impact: Bankstaffare not expectedto measuresustainability,effectiveness, or
efficiencyas part of projectsupervisionor Implementation CompletionReportsbut
ratherto stickto "direct results"indicators.However,efficacyindicatorscan be rele-
vant indicatorsof resultsif efficiency(ofan institution,for example)is a projectobjec-
tive.Moreover,OEDusesthesemeasuresin its evaluationsof projectperformance.It
is for thesereasonsthat thesedescriptionsof indicatorsare includedhere.

EFFICIENCY Efficiencyindicatorsusuallyrepresentthe ratio of

INDICATORS. inputs
neededper unit of output produced-for example,physicalinputs,dollars, or
labor required per unit of output. Accountabilityindicators(whichare the central
focusof muchproject and financial auditing) can be considereda subsetof effi-
ciency indicators.They measurethe extentto which resourcesare available for
and appropriately applied to the activitiesfor which they were targeted.

EFFECTIVENESS Effectiveness
INDICATORS. indicatorsusuallyrepresentthe ratio of
outputs(or the resourcesusedto producethe outputs)per unit of project outcome
or impact, or the degreeto which outputsaffect outcomesand impacts.For
example:

* Numberof vaccinationsadministered(or their cost)per unitdeclinein morbidity

rate (illnessprevented)or per unitdeclinein mortalityrate
* Numberof farmersvisitedper measuredchangein farm practices(numberof farm-
ers adoptingnew practices),or numberof farmersadoptingnew practicesper unit
increasein agriculturalproductivity
* Miles of read built per unit increasein vehicleusage,or new road usageper unit
decreasein trafficcongestion.

14
 indicatorsrepresent
INDICATORS.Sustainability
SUSTAINABILITY thepersistence
of pro-
jectbenefits
overtime,particularly
afterprojectfundingends.Theycouldinclude,for
example:

* Diseaseincidencetrendsafterexternalfundingfor a vaccinationprojectends
* Persistence
of changedfarmingpractices afterextensionvisitsare completed
* Maintenance anduseof roadsafterhighwayconstruction ends
* Persistence
of institutions
(programs,
organizations, relationships,
andsoon)cre-
atedto deliverprojectbenefits.

Information
on performanceasmeasured by impact,outcome, output,input,efficacy, Waysof measuring
andriskindicators
canbe expressedandgatheredin differentways.Thechoiceof performance
indicatorandmeansof collection
dependon dataavailability,timeconstraints,
and
cost-benefit
considerations
aswellastherelationshipsbetween thevariables.

Directmeasures
Directmeasures correspond precisely to results
at anyperformance level.Forinstance,
quantities
of goodsdeliveredorcountsof clientsservedaredirectmeasures of output,
instances
of changein beneficiary behaviorarea directmeasure of projectoutcome,
anda decrease in infantmortalityisa directmeasure of projectimpact.A givenvari-
ablecouldpossibly serveasan indicatorof results at anyof variouslevels(input,out-
put,outcome, or impact)depending on theprojectobjectives.

Indirectmeasures
Indirectmeasures correspond lesspreciselythandirectmeasures to theperformance
sought.Theyare oftenusedwheredirectmeasures aretoodifficult,inconvenient,or
costlyto obtain.Indirectmeasures arebasedon a knownrelationship betweenthe
performance variableandthemeasure chosento express
it-for example,using
lowerfarmgatepricesasan indirectindicatorof increased agriculturalproductivity,
usingdecliningfreightor taxitariffsasan indirectmeasureof decreasing trafficcon-
gestion,or usingreducednumbers of consumercomplaintsasan indirectindicatorof
improvedcustomer processing.

Earlypointers:intermediate andleadingindicators
At timesinformation on likelyprojectresultsis neededbeforefinalperformance data
are available.At othertimesit isimportant to gaugewhethera projectis on track
eventhoughfinalresultshavenotyetbeenachieved.Inbothcasesintermediate or
leadingindicators canprovidean earlyassessment of performance (figure4).
Intermediateindicators measure intermediate resultsor intervening stepstoward
projectobjectives.Theyusuallymeasure changes associated with theultimateimpact
soughtbutfor whichinformation canbe obtainedearlier.Forexample, fertilizerpur-
chasescouldbe usedasa preliminary indicatorof changedfarmingpractices, or
increased nutritional
knowledge asan indicatorof improvedeatingpractices. Two
pointsmustbe keptin mind,however. First,intermediateindicators oftenrepresent

15
Figure
4. Early
pointers

Developmentobjective
Sustainable
timberproduction

-4 CL~ ~ ~~C

0 3 5 25 years
I I
Lifeof project Actual impact

Source:TeamTechnologies,
Inc.

preliminarylinksin a causalchain, so their usefulness dependson the validity of the

hypothesisthat linksthosemeasuresto final performanceresults.Second,intermedi-
ate indicatorssometimes representresultsfrom initial or selectedprogramsites,so
their usefulness dependson the extentto which theyprove to be representative.
In addition,it is sometimes possibleto identifyleadingindicators(or indexesof indica-
tors)that areclearlylinkedwith longer-term results.While similarto intermediateindica-
torsin concept,leadingindicatorsgenerallyhavemultipleapplicationsand a statistically
valid recordof reliability(for example,theU.S.indexof leadingeconomicindicators).

Quantitativeand qualitativemeasures
FormanyBank-financed projects,indicatorsof impacts,outcomes,outputs,and inputs
are easilyquantified,that is, measuredby definednumericalvalues.Theseare typi-
cally the basisfor calculationsof economicrateof returnor net presentvalueduring
appraisal.Duringimplementation the monitoringand evaluationsystemof the project
or entitybeingfinancedcan providethesedata by recording,for example,the number
of studentsmatriculatedand the rate of repetition,the numberof farmersvisitedby
extensionworkers,or the quantitiesof fuelconsumed,electricitysold,or technical
capacitylost.Dataon kilometersof road maintained,tollscollected,volumeof traffic,
and so on can easilybe collected.In fact, thereis a dangerof collectingtoo much
information.In someinstancesthecostof collectingdata on projectoutcomesis high-
for example,a new systemmayhaveto be developedto measureeducationalattain-
ment.It is importantto determinehow preciseand timelyinformationneedsto be, and
who will collectit and at what cost,beforeinformation-gathering effortsare initiated.

16
 A project'soutcomesand impactsmay not manifestthemselvesas directly mea-
surablenumericinformation,however.Theseprojects' effectsmay be felt more in
termsof the attitudesof beneficiaries.For example,do parentsnow have more
input into their children'seducation?Is teachermorale higher?A purely narrative
descriptionof theseeffectsmaybe insufficientto measureresults,however.It may
be necessaryto deviseways of measuringoutcomesand impactsin quantitative
terms,convertingqualitativedescriptionsinto quantitativeinformation.Thisconver-
sion can be achievedusing surveytechniquessuchas beneficiaryassessments,
rapid rural appraisals,or focusgroup interviewswith structuredquestions.The
informationgained throughthesetechniquescan be usedto calculatenominal mea-
sures,rank orderingsof categories,and frequencycounts.Ratioand interval scales
can also be used.Thepotential for quantifying informationon projectresultsshould
not be underestimated.

Measurement scope
Measurementscoperefersto the useof samplepopulations.Performanceindica-
tors sometimesmeasureresultsdirectly for an entire target population(of individu-
als or organizations)throughadministrativerecords,observations,or census
surveys.Often, however,the scopeof measurementis limitedto a sampleof tar-
getsor sites.Thisapproach raisesan additional technicalissue:how reliably can
overall project performancebe statisticallyinferred on the basisof this sample?
Sometimesperformanceis measuredin only one project setting,or in a very few,
as casestudiesinsteadof statisticalsampling.While case studiescan provide use-
ful informationon how projects work (or why theydo not work as expected)and
how theycan be improved,care mustbe taken (evenmore than in the useof sam-
ples)not to assurnethat resultsfrom one site necessarilyrepresentproject perfor-
manceoverall. Accordingly,suchcase studiesare usuallyconductedin the context
of speciol studies(seebelow) ratherthan as a replacementfor broader
performancedata.

Special studies
Sometimesa project's routine monitoringand evaluationdata do not provide suffi-
cient information.If an unexpectedproblemarises,additional in-depthanalysis
through specialstudiescan guide the way toward solving it---and avoiding it in the
future.Specialstudiesare formativeevaluationsof the fundamentalsof problems
and their origins, and in that way differ from monitoring indicators,which are part
of an early warning system.For instance,project managersmight need to learn
moreabout the causal linksamong project outputs,outcomes,and impacts,espe-
cially when indicatorsrevealthat the broader purposesof a project are not being
achievedeventhough its planned outputsare being delivered.In this case some-
thing clearly is wrong: the project logic may be faulty,assumptionsabout risk and
enabling factorsmay be invalid, or somenecessaryinput may be lacking. Special
studiesoften provide importantfeedbackfor project redesignand higher-levelpol-
icy debates.

17
Generalprinciplesfor Threegeneralprinciplesshouldguide the selectionof performanceindicators:indica-
selectingindicators tors mustbe meaningfuland relevant,a reliable systemfor collectingthe requisite
data mustbe developedin a timelyfashion,and the borrower'sinstitutionalcapacity
for usinga monitoringand evaluationsystem-and itswillingnessto do so-must be
takeninto account.Thesegeneralprinciplesimply the followingconsiderations: 9

* Relevance.Theindicatorsselectedmustbe relevantto the basicsectoraldevelop-

mentobjectivesof the projectand, if possible,to overallcountryobjectives.
* Selectivity.Theindicatorschosenfor monitoringpurposesshouldbe few and mean-
ingful. It is recommended that the Bankmonitorno morethana dozen indicators,
at leasthalf of themimpactindicatorsthat explicitlymeasureprojectimpact
againsteach major development objective.
* Practicalityof indicators,borrowerownership,and data collection.If performance
indicatorsare to meaningfullyreflecta project'sobjectives,theyshouldbe selected
jointly by the borrowerand the Bankduring participatoryprojectpreparation,and
the data they measureshouldbe usefulto both projectand country.Thedata
requiredto compilethe key indicatorsmustbe easilyavailable; if collectingthe
data will requirea specialeffort,this needshouldbe identifiedearly in the project
cycleand includedin the projectdesign.Datacollectioneffortscan thenbe pro-
grammeddirectlyinto projectimplementation or contractedto institutionsthat do
surveywork at the sectoral,regional,or nationallevel.
* Intermediateand leading indicators.In the absenceof moredefiniteimpactindica-
tors, early pointersof developmentimpactmaybe usedduring projectimplementa-
tion to indicateprogresstowardachievingprojectobjectives.In manycases
outcomeindicatorstogetherwith indicatorsof riskfactorscan serveas suitable
intermediateor leading indicatorsof impact.
• Quantitativeand qualitativeindicators.Tothe extentpossible,performanceindica-
torsshouldallow for quantitativemeasurement of developmentimpact.For some
projectobjectives(for instance,capacitybuilding)it may be necessaryto develop
qualitativeindicatorsto measuresuccess,which shouldstill allow credible and dis-
passionatemonitoring.(Fordetailson convertingqualitativeindicatorsto quantita-
tive measuresof the qualitativeaspectsbeing investigated,seepages 16--17.)

Problemswith defining indicators

A previoussectiondescribedthe differenttypesof indicatorsusedto measurelevelsof
performance.However,it is sometimes difficultto distinguishbetweena project'sout-
putsand outcomes,for example,or betweenoutcomesand impacts.When defining
indicators,it is importantto thinkof the typologyof indicatorsas a continuummirror-
ing the logical means-endrelationshipof the project:inputsto variousactivities,which
yield outputs,all of whichcontributeto outcomesand impacts.A particularmeasure's
logical relationto the project'sstrategicobjectivewill definethetype of indicatorthat
it represents.
Consideran agriculturalextensionproject.Twopossibleindicatorsare the number
of farmerstrainedand the numberof farmersadoptingthe recommendedtechniques.

18
Thesecondindicatoris obviously an outcome,butwhataboutthefirst?Trainingisan
outputof theproject,butit isalsoan inputintoimproving agricultural
practices.This
exampledemonstrates theneedto thinkof indicators in termsof a continuum,with
inputsleadingthroughactivities to certainoutputsthatleadto desiredoutcomes and
impacts.It isnotalwayseasyto distinguish betweenoutputs andoutcomes or
impacts,whereasinputsare usuallystraightforward. Consider a projectthathopesto
improvegirl'semployment prospects by improving theiraccessto formalandvoca-
tionaleducation. An outputindicatorwouldbe thenumberof girlsgraduatingfrom
secondary andvocational schools, an outcomeindicatorwouldbe thenumberof girls
employed fromprojectschools, andan impactindicatorwouldbe theirearningsrela-
tiveto earlierlevelsor relativeto averagecohortearnings.

Problems withmeasuring impact

Impactindicatorsare themostdifficultto measure andcollect,mainlybecause of lags
betweenprojectimplementation andimpactor,putanotherway,between thetimeof
impactandthetimeit isfeasibleto collectdatarelatingto impact.Butthemonitoring
of projectimpactduringimplementation is oneof themainmotivationsfor usingper-
formance monitoring indicators.
Usingleadingindicators andintermediate indicators
asproxiesfor impactis a way to tacklethemeasurement problem.Beneficiary assess-
ments,rapidruralappraisals, andfocusgroupinterviews are usefulwaysof collect-
ing qualitative
impactdata.Beforeandafterhousehold andcommunity surveys are
excellent
toolsfor collecting
comprehensive impactdata,butthetimeandcost
requiredmakethemimpractical for regularprojectmonitoring.Theyare moreoften
usedduringprojectidentification,intermediatecheckpoints(suchasmidtermreviews),
implementation completion,andex postevaluation.

Whenselecting indicators
duringprojectpreparation andappraisal,theborrower Indicator
benchmarks
(withtheBank'sassistance,asneeded) shouldusebaselinedataandcomparative andinternational
datafromotherprograms to settargetsfor theindicators
it will monitor-thatis,the comparators
minimum valuesthatit expectsto aimfor.Someof theindicatormenusissuedby the
Bank'sCentralVicePresidencies providecomparator valuesasan indicationof what
is highandlow fora givenvariable.Bankstaffshouldconsult therelevantsector
departmentsof theCentralVicePresidencies on theselectionof indicators
for a pro-
jectandshouldseektheiradviceon targets.

Howdo performance
monitoring
indicators
affecttheBank'swork?

Indicators
playan integralrolein theBank'swork,fromeconomic
andsectorwork,
throughtheentireprojectcycle,to ex postevaluation.

Indicators
of sectorperformancecanbe bothderivedfromandusedin theanalysis Economic
and
doneaspartof economic andsectorworkandusedlaterto informprojectdevelop- sector
work
ment,projectimpactmonitoring,andsectormonitoring.Besides
providinga frame-
workfor evaluatingsectorperformance,
sector-specific
performance
indicatorscan

19
 helppolicymakers, taskmanagers, andBankstaffrationalizea countryassistance
and lendingstrategy, selectprojectstrategies,
andcreatea frameworkfor monitoring
projectactivities(seebelow).Sectoral indicators,
obtainedfromsectorworkor from
generallyavailablesectordata,areespeciallyusefulduringprojectidentification
and
appraisalto clarifyprojectobjectivesandlaterto assess
impact.

Project
cycle A schematic
representation
of howperformance
monitoring
indicators
fit intothepro-
jectcycleisshownin figure5.

Projectidentification, preparation,andpreappraisal
Projectplanningstartsby definingobjectives thatreflecta project'sdesireddevel-
opmentimpact;theseobjectives maybehierarchically arrangedbuttheyhaveto be
definedin termsof themeasurable targetsonwhichperformance indicatorswill be
based.Theproposed objectives
guidetheselection of a first-roundsetof indicatorsdur-
ingprojectidentification anddesign.
As partof theproblem,beneficiary, andobjectives analyses, severalquestions
shouldbe discussed withtheborrowerandotherstakeholders: Whatarewe tryingto
achieve? Howdo we measure whatwe are achieving? Whattypesof indicators or
measures will we needto develop? Whattargetvaluesshouldwe use?
Theprocess of selectingperformance indicators
helpsborrowers andBankstaff
defineprojectobjectives moreclearly,setmeasurable goals,andensurethata pro-
ject'sactivitiesleadlogicallyto therealizationof itsobjectives. Duringprojectprepa-
rationit is important to developa clearideaof thebaselinevaluesof impact
indicators andtheplannedtargetvaluesfor thedevelopment impactof theproject,as
wellasthemeans for measuringprogress(thelogicalchainof indicators).
ThisactivityhelpstheborrowerandtheBankassess therealismof projectobjec-
tivesanddetermine dataavailabilityandcollection methods for measuring indicators.
(Thenextsection discusses issuesassociated withdefinitionof objectives anddata
measurement.) Concentrating on howto quantifyandmeasure impactat thisstage
helpstheborrowerandtheBankimprovethedesignandconsistency of projectcom-
ponents to achievethedesiredresults ascost-effectively
aspossible. Thiseffortmay
Figure
5. Performance
indicators
andtheproject
cycle

PROJECT
IDENTIFIEATION,
PREPARATION,
PREAPPRAISAL
Performbaseline
studies
Analyzesectoral
doat
Idenlify
riskandenabling
factors

PREPARATION/ IMPLEMENTATION/ SUPERVISION/ COMPLETION

APPRAISSAL SUPERVISION OOMPLETION ONWARDS
Identify
risk,input, Useindicators Evaluate and
output Evoluate
outcome
and
output,
outcome, and lomonitor outcome
indicators impact
indicators
of
impact
indicalors implemenitlion project
success

20
also point to the needto developmonitoringcapacity beforeimplementingthe-project
or to buildmonitoring
andevaluation
capacitydevelopment
intotheproject.

Appraisalandnegotiations
Asprojectplanningisfinalized,projectobjectives aretranslated
intospecific
project
components. Indicators
for measuringprogresstowardeachobjective andfor monitor-
ingtheprovision of projectinputsandthestateof riskandenablingfactorsidentified
in
theeconomic andrisksensitivity
analysesare alsofinalizedduringappraisal.
Theimplementation plandeveloped by theborrowerandappraised bytheBank
mustincludetheperformance monitoringindicators-inputs,
outputs, outcome,and
impactindicators-foreachaspectof projectimplementation. Duringappraisalthebor-
rowerandtheBankagreeon no morethanabouta half-dozen of themostimportant
inputandoutputindicators andaboutthesamenumberof outcome andimpactindica-
tors-keyperformance indicators
thattheBankwill monitor.These indicators-as
agreedperformance benchmarks, notcovenanted targets-aresetoutin theproject
appraisaldocumentation andin theBank'slegalagreement withtheentityresponsible
for projectimplementation (seeBankProcedure 10.00,AnnexB andrelated
Operational Memorandums). Incaseswheretargetvaluesarecriticalto theattainment
of projectobjectives,
theborrowerandtheBankmayalsowantto makethema
covenant of theloanagreement (seepage26).

Implementationandsupervision
Duringprojectimplementation theborrowerandprojectimplementation unitmonitor
theindicatorsdeveloped duringprojectdesign,followingthescheduleof theimple-
mentation plan.Duringregularsupervisionmissions andthemidtermreviewtheBank
monitors thekeyindicatorsdefinedandagreedwiththeborrowerat appraisal.These
indicatorsformthebasisfor theBank'smeasures of implementation
performance (IP
ratings)andimpact(development objective,or DO ratings)
on Form590. If theindi-
catorsoriginallydevelopedfortheprojectbecomeinappropriate because external
conditions changeor projectdesignisrestructured,
theBankandborrowerdevelop
new indicatorsto reflectthe changedcircumstances.

Evaluation
Afterimplementationiscompleted,theachievementof projectobjectives
isassessed
usingmeasurable indicators-theoutcomeandimpactindicators developed duringpro-
jectdesign-thatcompare theproject's
actualimpactwithitsintended impact.Theuseof
indicators
removes subjectivity
fromevaluations,givingthema moreobjective basis.

Thelogicof projectdesigngovernsthelogicof monitoring:theimplementation unit Performance

monitors inputsandoutputsin greaterdetailthandoestheborrowinggovernment or indicators
and
theBank.Implementation managers areprimarilyconcernedwith thetacticsof pro- projectmanagement
gramimplementation, whileborrowers andBankstaffare moreconcerned withthe
strategicimplications
of projectimplementationandalternatives
for realizingproject
objectives.

21
 Tomeetthedifferentinformation needsof thevariousstakeholders, performance
indicatorsusuallyneedto be hierarchicallystructured.
Thisstructure
can beaccom-
plishedthroughnestednetworks of partiallyoverlappingperformance informationsys-
temsin whichlower-level performance indicatorsare summarized, or selectively
skimmed, asa basisfor higher-level
information.Thefollowingsections describethe
differentperformanceindicatorsandinformation needsof thevariousactorsresponsi-
blefor projectimplementation.(Thisinformationis summarized in box 6.)

Field-levelimplementers
Implementationmanagers are primarilyconcerned with themethods usedto imple-
menta particularprojector programcomponent. Withintheirrelativelynarrowareas
of responsibility,
theyneedto knowthatnecessary inputs(human, financial,andphys-
ical resources)
are availableandthatplannedoutputs(goodsandservices) are being
produced.Theyalsoneedto knowthatresources are beingusedlegitimately,that
financialaccountability
isensured,andthatinputsarebeingefficiently transformed
intooutputs.Implementationmanagers in thefieldalsoneedat leastsomeinformation
on programoutcomes andimpacts;thatis, theyneedto knowtheextentto whichpro-
gramoutputsare achievingtheirintended resultssothattheycanfine-tune project
implementationandmotivateperformance.

Implementationunitmanagers
Themanagers of an implementation
unit(orof whatever
organizational
unitisresponsi-
bleforoverallprojectmanagement)
are usuallyresponsible
for implementing
an entire

Box
6. Project
management
andperformance
information
needs
IMPLEMENTERS
IN THEFIELD
NEED
* Inputindicators
a Outputindicators
* [Efficiency
indicators]
e Riskindicators
* Some
outcome
andimpactindicators
THEIMPLEMENTATION
UNITNEEDS
* Summary inputandoutput including
indicators, site-comparative
indicators
asappropriate
* Outcome indicators,
including
site-comparative asappropriate
indicators
* [Effectiveness
indicators]
* Riskindicators
a Impact
indicators
AND THEBANKNEED
THEBORROWER
* Summary
inputindicators
* Summary
output indicators
a Riskindicators
* Keyoutcome, impact,
[andrelevance]
indicators
e [Sustainability
indicators]
Note: Indicators
inbrackets
arenota required monitoring
partofBank orproject
supervision.

22
programor a majorprojectcomponent.Thisresponsibility involvesa rangeof activities,
oftenacrossa numberof sites,aimedat achievingsomestrategicobjective.Because
thesemanagershavea primaryresponsibility theyshould
for projectimplementation,
play a supportingrole in definingstrategicobjectives,choosingamongthemor articulat-
ingthe underlyingpoliciesfromwhichtheseobjectivesderiveduringprojectpreparation.
Implementation unitmanagersare concernedwith managingtheir programbetter,
selectingand fine-tuningprojectactivitiesto improveoutcomes,and enhancingproject
impact.Thusthey needsummaryinformationon projectinputsand outputs,particularly
comparativeinformationacrosssites,so thattheycan identifyemergingproblemsand
directmanagerialattentionand resourcesto solvingthem.More important,theyneed
comparativeinformationon projectoutcomes.Implementation managersalso needat
leastsomeinformationon projectimpactto validatethe significanceof their project
strategy.

Borrowerofficialsand Bankstaff
Representativesof the borrowerand Bankstaffare primarilyconcernedwith strategic
management. Theyhaveparticipatedin articulatingthe policyassumptions thatunderlie
the projectand the strategicobjectivesthe projectis intendedto achieve.Moreover,
theymustcontinueto pushtheseobjectivesin thecontextof a dynamicexternalenviron-
ment.Thustheyneedsummaryinformationon projectinputsand outputsto be surethat
any emergingimplementation or outcomeproblemsare beingidentifiedand addressed
at appropriateprojectmanagement levelsand to be able to answerquestionsaboutthe
project.Tomanagestrategically,however,borrowersand Bankstaffmostneedcompar-
ative informationon projectoutcomes and impactsand riskfactors.When impactis
measuredin relationto thecostof projectinputs,thisinformationcan alsofeeddirectly
intoa performance-based budgetsystem.Inaddition, in the contextof the CountryPort-
folio Performance Reviewor CountryAssistanceStrategyupdate,the borrowerand
Bankmanagersneed informationon the nationaltrendsto whichprojectsare expected
to contribute.

Somerelatedissues

Severalfactorsthatare part of good practicein monitoringand evaluationare integral

to the effectiveuseof projectperformancedata, in particulardata collectionand man-
agement,the institutionalarrangementsfor managinginformation,and the useof feed-
backfrom monitoringand evaluation.In addition, usersneedto be awareof the
differencebetweenloancovenantsand indicatorsand potentialpitfallsin the definition
of indicators.For performancemonitoringindicatorsto work, a management structure
and incentivesthatvalueresultsmustbe in place.Performance indicatorsare a tool:on
their owntheycan do nothing,but in the properenvironment theyinspireaction.

It is criticalthat managers-Bank,borrower,and projectmanagers-regard perfor- Goodpracticein

mancemeasurement as an integralpart of their institution'smissionand strategic monitoring
and
10
plan. Oftentheydo not; instead,theyseeit as an adjunctto the plan, in the same evaluation

23
way thattheymayseeevaluation asa requirement afterthemore
to be satisfied
important workof projectimplementationisdone."lTobe effectivemonitoring and
evaluationmustbe addressed duringprojectdesign.Onceprojectplanners have
carefullydefinedthestrategic andselected
objectives logicalindicators,theirnext
consideration for datacollectionandmanagement,
shouldbe therequirements institu-
tionalarrangements andcapacitybuilding,andtheuseof feedbackfromthedata.

Datacollectionandmanagement
Thenatureof a variabledetermines thelogicalsourceof dataaboutit. Indicators of
inputsandof someoutputs of projectactivitiesare derivedprimarilyfromproject
records;however,indicatorsof someoutputs,outcomes, andimpactsmayrequirethat
data becollectedusingsurveys or specialstudies,includingthosethatuseparticipa-
torymethods.Whereit is possible, it is almostalwaysbetterto piggybackregularsur-
veysontoexistingnationallyor internationallysupportedsurveys (suchastheLiving
StandardsMeasurement Study)than to create
a datacollectionfacility.
Specialstud-
iesmaybe managed by theprojectunitdirectlyor subcontracted to a privateentity.
Anyproposalto collectdataforan indicatorrequiresa discussion of:

* Objectives of thestudyor survey

* Sources of data
* Choices and proposed methodof collection
* Likelyreliabilityof thedata.

Collectionof someindicators,particularly outcome and impactindicators (suchas

morbidityand mortalityrates,educational achievement, or cropproduction) may
dependon theexistence andqualityof nationalcensus or surveysystems. Before
choosingsuchprojectindicators, theborrowerandtheBankmustconfirmthatthe
necessary systems are in placeandreliableandthattheavailabledataarestatisti-
callyvalidfor thepopulationorareacoveredbytheproject.Thecomplexity of statis-
ticsandtheproblems of attributingcausalitymeanthatin manycasesit isbetterto
useservicedeliveryandbeneficiary response asproxymeasures thanto attempt to
measure impactdirectly.
Manyoutputindicators arederivedfromrecordskeptbytheparticipating agen-
cies,oftenat projectfieldsites.Forthisreason,for purposes of projectmonitoring
andevaluation design(including projectplanners
indicatorselection), shouldexamine
theimplementer's record-keeping andreportingprocedures to assessitscapacityto
generatedata.
Theessential pointsare thatdatashouldbecollectedandusedcloseto thesource
andthatdatacollection andreliable.It is important
be cost-effective notto createa
separatemeasurement bureaucracy withina projectstructure. Havingsucha bureau-
cratichomefor dataproduction isnotusuallycost-effective and presents theriskthat
thoseresponsible for producing thedatamayhavelittlecontactwiththoseresponsi-
blefor usingit. Thusthedatausersdo notfullyunderstand whatis behindthenum-
berstheyare givento use,andthedatacollectors havelittleappreciation for the

24
issuesthat stakeholdersand seniorproject managersface, for which performance
data would be useful.Justas performancemeasurement shouldbe fully integrated
with projectdesign,informationgeneratedshouldbe integratedwith the project's
managementstructure.As with any otherprojectcomponent,the benefitsof usingper-
formancedata mustat leastequalthe costof collectingthem.A projectcan provide
the bestinformationat the leastcostby usingexistingdata, samplingtechniques,
rapid appraisal methods,and othercreativecollectionmethods.
A few otherconsiderations about data collectionshouldguidethe designof indica-
tors. Aboveall, thedata shouldmeasureresults,not just processes. Theperformance
measuredby thedata shouldfocuson what the projectis accomplishing,especiallyin
termsof its impacton people.Thepointis not only to knowwhat projectsare doing,
but to knowwhethertheyare doing any good.
Selectivityis desirable.Performance analysisshouldbe limitedto the few areas
that are directlyrelevantto the project'sstrategicobjectives,as definedby the
borrowerand Bank.
Finally,performancemeasurement systemsshouldusedata that are not construed
as threateningby thosewho are reportingit. Simplytelling projectmanagers-
whetherborrowers,Bankstaff,or thoseworkingfor them-to reportdata on their pro-
jectswithoutactivelyinvolvingthemin the performancemeasurement process,without
explaininghow and by whom thedata are going to be used,and withoutassuring
themthat the data will not be usedto judge thempersonallyis likely to be viewedas
threatening.Relyingon sucha compliancemodeof performancemeasurement is
likelyto backfire,not only minimizingmanagerownershipbut also likely producing
bureaucraticresistanceand possiblydata corruption.

Institutionalarrangementsand capacitybuilding
Datacollectionarrangements haveimplicationsfor the institutionalarrangements
betweenthe implementation unit,the borrower,and the Bank.Therationalefor indica-
torsdemandsthatthe indicatordata be of valueto the borrower,not justto the Bank.
A project'soverallmonitoringand evaluationdesignshouldbuild on the reporting
arrangements alreadyusedby theimplementation unitand the borrower,while further
developingthe technicalskillstheyneedto plan informationneeds,designdata collec-
tion, executestudiesand surveys,analyzedata, and reportresultsin a formatthat is
relevantto projectmanagement.If necessary, the Bankcan financetechnical
assistance and equipmentrequiredto designand implementmonitoringand evalua-
tion systemsand to meettrainingneeds.

Feedbackfrommonitoringand evaluationeffortsand interpretingindicators

Care shouldbe takento timedata collectionactivitiesso that informationwill be
availablewhen it is needed,for exampleat the timeof the Bank'sportfolioor
midtermreviews.Projectsfor which operatingperformancestandardsare specifiedas
an objective,or for which decentralizedprocessescall for local capacityto plan and
managework programsand budgets,will need specialconsiderationof how indica-
tor findingsare usedto inform decisions.

25
 Projectindicatorsare bestinterpretedthroughcomparison.Ideally,impactis evalu-
ated by comparingbaselinedata with projectoutcomes,to providean ideaof what the
indicatorswouldbe if the projecthad not beencarriedout. Butbecausesuchanalysisis
sometimes not practical,analysisof indicatorscan useotherkindsof comparisons:
actualresultsrelativeto targets,beforeand afteranalysis,timeseries(moreillustrative
than simplebeforeand aftercomparisons), or comparisonsof controlgroups.
Trendsor fluctuationsare commonin the valuesthat are usedto measureoutputsor
outcomesand impacts.If thisis the case,timeseriesor controlgroupsshouldbe
used.Timeseriesdata capturetrendsor fluctuations,and controlgroupshelpverify
attributionof causality.For example,the controlgroup chosenis critical for compar-
ing the increasein averageeducationalattainmentor incomeamong a project'sben-
eficiarieswith the increasein a nonprojectarea: thecontrolgroup mustbe identical
to the projectsamplein all respectsotherthan the presenceof the project.Finally,any
interpretationof indicatorsmustconsiderthe relationshipbetweenexogenousfactors
and the indicatorstheymay influence.Thisis one reasonwhy the monitoringof risk
factorsis so important.

Relation
to loan Indicatorsare factsabout projectimplementation results,notactions.Thusstandard
covenants practiceby the Bankand borrowersis to considerperformancetargetsas indicative
and not legallybinding.Theunderstanding is thatthe agreedtargetsare usedprimarily
to gaugeprogressin projectimplementation and in realizingdevelopment objectives.
However,in instances whereattainmentof certainindicatorsor targetsisconsidered
essentialto the attainmentof a project'sobjectives(forexample,in the caseof certain
financialindicatorsof a profit-makingentitysupportedby a project),theseindicators
(targetvalues)shouldbe incorporatedin the legaldocumentsnotonly as performance
benchmarks butalso as a loancovenant(seepage 21).

Workingat cross Projectssometimeslack well-definedobjectivesbecausethe borrowerand the Bank

purposes cannotagreeon what a project'spurposesare or shouldbe. Forexample,it is some-
timesunclearwhethera project'sultimateobjectiveis to transferresourcesor to truly
alleviatepoverty.A project'spurposesand strategicobjectivesmustbe agreedon
early,and performanceindicatorsprovidethe clearestguide for managementaction
when theylogicallyreflectclear programpriorities.In practice,however,programs
mayencompassmultipleand evenconflictingobjectives.While it is difficultto portray
multiplestrategicpurposeswithin a singleobjectivestreeor logical framework,it may
sometimes be possibleto defineparallel objectivestreesreflectingthesemultiplepur-
poses.Theperformancedata could thenbe usefulin makingdecisionsthat resultin
tradeoffsamong competingobjectives,especiallyif suchobjectivescan be prioritized
or weighted.12
Anotherdifficultyis thatclearlydefinedobjectivesbring a measureof accountabil-
ity that may makesomeofficialsuncomfortable.In somecasesambiguousproject
objectivesmay,in fact, be politicallydesirable.Delineatingusefulperformanceindi-
catorsmay provedifficultif thisis the case;and in the absenceof willingnessto com-
mit to clearly definedobjectives,the Bankshoulddissociateitselffromthe project.

26
In theendperformance monitoring indicatorsandthefeedbacktheyprovideareonly Lastword
asgoodastheunderlying analysis(economic andfinancialanalysis,economic and
sectorwork,socialandenvironmental assessment) supportingtheprojectdesign,and
thedatato beassessed overtime.Thelogicalframework approachto projectformu-
for projectdesignandevaluation,
lationis onlya structure notthefull extentof project
designor evaluation. Noneof thetoolsdescribed in thishandbook can replace
soundeconomic, financial,social,environmental,andriskandsensitivity analysisor
comprehensive monitoring andevaluation. Together, theanalysis,thelogicalframe-
work,andtheindicators forma system for continuous analysisanda holistic
approachto projectdesign,monitoring, andevaluation.

Notes
1. Effective Implementation: Key to Development Impact, September 1992.
2. An Overview of Monitoring and Evaluation in the World Bank, OED Report 13247, June 30, 1994,
and Monitoring and Evaluation Plans in ProjectsApproved in Fiscal Year 1995, OED draft report, October
6, 1995.
3. The logical framework approach and indicator typology described in this section draw extensively
from material prepared by R. Moses Thompson,Team TechnologiesInc.; Gerald M. Britan, 'Measuring
Program Performancefor Federal Agencies: Issuesand Options for PerformanceIndicators,' prepared for
the U.S. General Accounting Office, 1991; and 'Guidelines for Result-BasedPlanning, Management and
Monitoring," preliminary version, from the Swedish International DevelopmentAgency, January 1995.
4. Several formal techniques for collaborative project planning and managementare variations on the
overall logical framework concept described here: Logical frameworks, ZOPP,and TeamUP,for example.
Appendix 1 of the World Bonk Porticipation Sourcebook (February 1996) describes ZOPP(objectives-
oriented project planning), TeamUP,and other methods and tools that support participatory development,
and provides additional references.
5. The relationship among project objectives may also involve aggregation across spoce, with resultsin
different project locations added to yield regional objectives and achievements. Hierarchically linked and
spatially aggregated objectives often exist in tandem. For instance, a notional child health project whose
overall objective is to reduce child mortality may have a subobjective of reducing the incidence of
preventable disease by educating and inoculating a certain number of individuals nationwide-that is, the
sumof the individuals reached by individual clinics throughout the notion.
6. See also Handbook on EconomicAnalysis of InvestmentOperations (Washington, D.C.:World Bank,
1996), Chapter 10, "Risk and SensitivityAnalysis."
7. Various formal logical framework techniques use tools suchas problem and objective trees, situation
and SWOT (strengths-weaknesses-opportunities-threats) analyses, and project planning matrixes to graphi-
cally representthe problem, beneficiary, and objectives analysis.
8. Even if a borrower is not explicitly aware of or does not follow the logical framework approach in
designing a project, the Bank can follow the approach to appraise the project.
9. This discussion draws heavily on a report of the EastAsia Working Group on Project Performance
Monitoring Indicators prepared for consideration by the EastAsia Regional Management Group. The work-
ing group included staff from the EastAsia Region, the Asia Technical Department, Human ResourcesSer-
vices, OED, and Operations Policy.
10. For additional discussion, see Monitoring and Evaluotion Plansin Staff Appraisol ReportsIssued in
Fiscal Year 1995: A Follow-up to OED's Report 'An Overview of Monitoring and Evaluation in the World
Bank,' OED Report 15222, December29, 1995.
11. ThomasJ. Cook, Jerry VanSant, LeslieStewart, and Jamie Adrian, "Performance Measurement:
LessonsLearned for DevelopmentManagement," World Development23(8): 1303-15 (1995).
12. Briton, p. 20.

27
SECTORAL NOTES ON INDICATORS

Tohelp Bankstaffand borrowersselectand useperformanceindicators,the Bankis

developingsectoralnotesthat discussthe useof indicatorsin relationto the major
objectivesor categoriesof problemsnormallyaddressedin Bank-financed projects.
Eachnoteincludesmenusof possibleindicators.

Whattopicsdothenotescover?

Eighteensectoralnoteswill eventuallybe prepared.Theywill covereachsectorin

which the Bankworksand areasof emphasisthat crosseconomicand socialsectors,
suchas environmentalconcerns,povertyreduction,publicsectormanagement,and
technicalassistance.Of the sectorperformanceindicatornotesplanned,seventeen
havebeen issued:

Agriculture Povertyreduction
Economicadjustment Power
Education Privatesectordevelopment
Environment Technicalassistance
Financialsector Telecommunications
Housing Transport
Industryand mining Urbandevelopment
Oil and gas Water and wastewater
Population,health,and nutrition

Theonly notethat remainsto be issuedis publicsectormanagement(expectedin

June 1996). All of the notesexceptone are in theirfirstedition, that is, theywill be
revisedas the Bankand its clientsgain experiencewith theuseof indicators.(The
educationnote is in its secondedition and is consideredto be an exampleof best
practice.)Furtherwork on thesenoteswill:

• Standardizethemto makethemmoreconsistentwith the methodological

frameworkdescribedearlier
* Incorporatemorebestpracticesand lessonslearned
* Incorporateinternationalcomparatordata at the global and regional levelsto facil-
itate benchmarkingamongcountries,provinces,regions,and so on, to appreciate
what is high or low in a given instance.

28
Howwerethenotes
developed?

As part of the Next StepsAction Plan,the CentralVice Presidencies,sectordepart-

ments,and Development Economics Vice Presidencywere askedto developsector-
specificperformancemonitoringindicators.TheOperationsPolicyDepartment(OPR)
coordinatedthe effortsto developthe indicatornotes,facilitatingthe exchangeof
ideasamong sectordepartmentsand reviewingthe product.After several
departmentshad produceddrafts,OPRformeda qualityreviewgroupcomprising
seniorstafffrom the Regions,the CentralVice Presidencies, and OED.Thegroup
reviewedall of the notesand developeda frameworkfor consistencyin approach.
Fromtheseinitial efforts,the Bankdevelopedthe frameworkfor performanceindi-
catorsthat hasbeen applied in mostof thesenotes.Mostof the sectornotes:

* Followa typologyof indicatorsbasedon a logical frameworkapproachto project

design (indicatorsof projectinputs,outputs,outcomes,impacts,riskand enabling
factors,efficiency,effectiveness,
and relevance);and
• Providean executivesummary,a menuof recommended key indicators,and real
examplesof indicatorsusedin Bank-financed projects.

Howare indicators
selected
fromthenotes?

Thenotesare referencematerialsto specificsectorsand to overarchingconcerns

suchas povertyreduction,macroeconomicadjustment,and environmentalissues.
Theyare meantto guide taskmanagersand borrowersin applying performance
monitoringindicators.They are not, however,intendedto replacethe judgmentor
knowledgeof task managersor borrowers.The notesprovide menusof indicators
that are neitherexhaustivenor mandatory-indicators are project-specificand must
be customizedby the borrowerand taskmanagerto project,sector,and country
circumstances.Sinceevery projecthas its own uniqueobjectives,task managers
and borrowersmustdevelop indicatorsthat correspondto theseobjectives,and not
restrictthemselvesto the menusprovided in the notes.Conversely,care shouldbe
taken not to "order the entire menu;"rather,it shouldbe treatedas an indicativelist
from which to choosethe mostappropriate selections.

Whereare thenotesavailable?

Thenotesare issuedby theoriginatingCentralVice Presidencies to all SeniorOpera-

tionsAdvisers,directors,projectadvisers,and sectordivisionchiefs.Staffmembers
shouldhavereceivedcopiesof the notespertainingto the sectorsthattheir department
coversor havebeen notifiedthat the notesare available.Additionalcopiesare avail-
able fromthe originatingdepartments. Thenoteswill also be availablein electronic
form throughthe Bank'sEnterpriseNetwork(contactthe relevanttaskmanagerlisted
below to confirma note'selectronicavailability).Thedepartments,taskmonagersfor
the notes,and personsto contactfor a copy of the noteare

29
Sector Department Task manager Contactpersonand extension

Agriculture AGR Cornelisde Haan JoyceSaboya 38959

Economic adjustment DEC Deborah Wetzel DeborahWetzel 31698
Education* HDD SverrirSigurdsson Joe-Shin
Yang 81418
Environment ENV JohnDixon JohnDixon 38594
Financialsector FSD DavidScott/ HediaArbi 34663
MonikaQueisser
Housingond urban
development TWVU PatriciaAnnez LauraO'Connor 37009
Industryandmining IEN FelixRemy ElisaTorre 80323
Oil and gas IEN VVilliamPorter KyranO'Sullivan 32722
Population,
health,
andnutrition HDD TomMerrick VivionOctran 33639
Povertyreduction PSP SoniyoCarvalho SoniyaCarvalho 35705
Power IEN Jean-Pierre
Charpenlier KyranO'Sullivan 32722
Privatesectordevelopment PSD SyedMahmood ShirleyWallace 38 13 1
Publicsectormanagement PSP MichaelL.O Steven.s MichaelStevens 37493
Technicalassistance OPR NimrodRaphoeli NimrodRaphaeli 84015
Telecommunications IEN RogatiKoyani RogatiKoyani 34515
Transport TWU ColinGannon Colin Gannon 85784
Waterand wastewater TWU GuillermoYepes RosePoole 33749

* Secondeditionnoteissued.

Howto learnmore
Besides developing theirsecond-editionnotes,theCentralVicePresidencies are dis-
seminating thefirsteditionnotesand advisingRegionalstaffon theuseof
performance monitoring indicators.TheCentralVicePresidencieswillcontinue to
sponsor workshops on theirindicatornotesand, whenrequested, adviseprojectstaff
on all aspectsof theuseof performance monitoringindicators.
Inaddition,the Bank's
Learning and Leadership Centeris providing
trainingon performance indicatorsas
partof itsregularcourses on projectpreparation,
appraisal,and supervision.

30
EXAMPLES OF INDICATORS

Thefollowingexamplesof key performanceindicatorswere developedfor several

Bank-financed projects.Theseprojectsrepresentcurrentbestpracticein the useof
performancemonitoringindicators;mostwere highlightedin OED'sfiscal 1995
follow-upreviewof monitoringand evaluation(Monitoringand EvaluationPlansin
StaffAppraisalReportsIssuedin FiscalYear95, Report15222). Foreach project,a
matrixpresentsthe project'sobjectives,with indicatorsof inputs,outputs,risk factors,
outcomes,and impacts.Thematrixesdo not list all the detailedindicatorslistedin the
borrower'sProjectImplementation Plan;theyonly showthe key indicatorsthat the
Bankwill monitoras the basisfor projectsupervisionand evaluation.
Thesematrixesweredevisedby the project'staskmanagerswith OPR'sguidance.
Theinformationsetout in thesematrixeswas alsogiven in the StaffAppraisalReport
for eachof the projects,althoughnot in thisformat.In the futurethe mostimportant
projectoutcomeand impactindicators,suchas thosepresentedin thesematrixes,
shouldbe listedin everyprojectappraisaldocumentand monitoredusingForm590.

31
Table
1. Summary
ofobjectives
andkeyperformanceindicators,
Honduras
Basic
Education
Project
(StaffAppraisal
Report13791-HO,March
8, 1995)
RISKSANDCRITICAL
INPUTS OUTPUTS ASSUMPTIONS
(Resources provided (Goodsandservices (Theoutcomeis OUTCOMES AND IMPACTS
OBJECIIVES for projectactivities) producedby theproject) dependenton...) (of projectactivities)

Improve quality * IDAcredit($30 million) * Strengtheningthe * 10-20%increase in

of lecirning
andstudent * Government
funds institutional
capacity students'
testscores
performance in the ($9.8 million) of theMinistryof Education* I 0%reductionin repetition
primaryeducationcycle * GermanlKfW)funds maytake longerto anddropoutrates
($13 3million) achievethananticipated * 5-10% increasein future
Inadequateprovision earnings of primary school
Fundswill be uisedto~ of counterpartfunds graduates from poor families
* Trainprimary and *30,000 trained teachers, by the government
preprimaryteachers, principals,and supervisors
principals,and supervisors

* Supply textbooks., 4.3 million new textbooks

didactic materials,and and 20,000 library books,
~ibrorybooks 8,500 poor rural studentswith
didactic materialseach year

*Encouragebilingual - 60% of indigenous

education primary studentsreceive
bilingual education

* Financeexternal - 8 roundsof math aind

evaluationsof student Spani-shte5stapplied to
academic achievements 20% of primary students

* Improveschoolsinipoor * 290 expanded or

ruralareas and appoint rehabilitatedand furnished
additionailteachers rural schools; 1, 200 new
rural preschoolprograms

Strengthencapacity *IDA, bilateral, and * ReorganizeMinistry of *30% reductionin

of the Ministry of governmentfundswill Education administrativecosts
Educationto deliver finance technical * Decentralizeservicesto * 40% reductionin
basic education services assistance,equipment, 18 departments centralstaff
office improvements5, staff * Municipalities maintain * Budgetdeficitsavoided
training, monitoringand schools * Lowerteacher absenteeism
evaluationefforts, and * Greater efficiencyin the use
teacher performance of public resourcesfor basic
incentives educaitionservices,as
mneasured
by loweroverhead
costs (by X%)

Taskmanager.Anna Sant'Anna.

32
Table
2. Summaryofobjectives
andkeyperformoance
Indicotors,
Indonesia
Rural
Electrification
Prolect
(StaffAppraisal
Report
12920-IND,
February
3, 1995)

RISKSAND CRITICAL
INPUTS OUTPUTS ASSUMPTIONS
(Resources provided (Goodsand services (Theoutcomeis OUTCOMES ANDIMPACTS
OBJECTIVES for projectactivities) producedby theprolect) dependenton... iof projectoctivities)

Expand coverage and * IBRDloan($398 miliion) Expand electricity None.WellwithinPLN's * Supply electricity
to 7,000
supplyof electricity
to * Government/PLN funds network implementation
and additional ruralvillagescovering
ruralareas ($442.8million) * 28,000 kmcof MV lines financialcapacity 2. 1 millionnewcustomers
* Ministryof Cooperotives* 35,000 kmcof LVlines customers * Meetelectricity consumption
($0.5 million) * 1.3 millionpoles targets
of 35-45 kilowotthours
* 833 MVAdistribution a month, depending on region
Funds will be usedto transformers * Increase penetration of
procure equipment, televisions,radios,andother
works,consultants,and appliances
trainingandtechnical * Increase ruralhousehold
assistance for capacity electrification
from32%to more
building than40%by 1998

Establishincentivesfor * Publish
smallpower * PLNmaybeslowto * Strengthen theenabling
privatesectorandlocal purchase tariffandstondard enterintopowerpurchase environment forprivatesector
cooperatives to providea powerpurchase contractfor contracts andcooperatives to generate,
largershareof rural saleof electricity by private * Delayedimplementationdistribute, andsellwholesale
electricity
distribution
and generators to PLN of thepolicyandregulotoryandretail-level powerfrom
renewable energy * Esiablish bulksupply framework to overseeretail renewable energysources.
generation for rural tarifffor saleof bulkelectricity pricingandservice aspects * Increase megawaits and
powersupply of privatedistributors megawatt hours of non-PLN
generation providedby
renewable energyfrom0 to 75
megawatts and 300 megawatt
hoursa yearby 1998
e Increase thenumber of
cooperatives engagedin
electricity
distribution
who buy
bulkpowerfromPLN
* Reduce powergeneration
requirements fromhigh-cost
dieselplantsto 75 megawatts
and200 megawatt hours
by 1998

Tablecontinues
on thenextpage.

33
Table
2 (continued)
RISKSAND CRITICAL
INPUT OUTPUT ASSUMPTIONS
(Resources provided (Goodsandservices (Theoutcomeis OUTCOME/IMPACT
OBJECTIVES for projectactivitiesl producedby theprojectl dependenton... (of projectactivities)

Advance theefforts
initiatedundertheFirstRural
Electrification
Projectto
placetheruralelectrification
program onanefficient
andsustainable footingby

* Enhancing theefficiency * Expand

PLNraining
of ruralelectricity
delivery program to enablevillage-level keepingpacewithtargets
by increasing and technicians to undertake
strengtheninglocal selecteddistribution
and
participation customer servicefunctions
Itroining
targetnumbers to be
determined)
* Training
programnot * Increase
from2,000 to
4,000 thenumber of village
cooperatives engagedin
distribution-related
operations
andmaintenance andcustomer
servicefunctions,
increasing
coverage from7 millionto 12
millionruralcustomers
* Enhance PLN'sabilityto
extenditscustomer base
withoutproportionately
increasingitsorganizational
size
a Improve staffproductivity
indexfromalreadyhigh250
customers peremployee

Maximizing theeconomic * Convert30,000 small * Weekmanagement * Target36 millionkilowatts

benefits
of ruralelectricity
by ruralbusinesses
to electricity by PIN peryearincrease inelectricity
promoting itsproductiveuses forproduction
purposes consumption by rurolbusinesses
* Createthepotential for
15,000 newjobsinrural
businesses
andincreased
human capitaldevelopmeni
and participation
benefits to
villagecooperatives

Initiatepilotprogramsto Completefourpilotprojects: * PLNdelaysin timely * Generate dataon

testnewdesigns and * SingleWire EarthReturn completion of pilotprojects adaptabilityof lechnicaloptions
construction management systems * Improper implementationandconstruction management
methods withthepotenlial * Reinforced
concretepole limitsvalueof operational techniques andsavings potentiol
forfurther lowering
the * Low-cost
substations experience inruralelectricity
delivery
costsof ruralelectricity * Contract
administration * 10-25%reduction incosts,
extension andintensifioation company depending on program
* 15-20%reduction incostsof
ruralelectricity
deliveryin next
stageof government's rural
electricity
program

Taskmanager:ArunSanghvi.

34
Table
3. Summary
ofobjedtives
andkeyperformance
Indicators,
Indonesia:
Second
Agrickutural
Research
Management
Proled
(SAR13933-IND,
April21,1995)
RISKSAND CRITICAL
INPUTS OUTPUTS ASSUMPTIONS
(Resourcesprovided (Goods andservices (Theoutcomeis OUTCOMES AND IMPACTS
OBJECTIVES Forprojectactivities) producedby theproject) dependenton .) (ofprojectactivities)

Establishondoperate * IBRD loan($63 Eightagricultural

technology * Sustainability of local * 25%increase inadoptionof
eightagricultural
tech- million) assessmentinstitutes
withnew counterpart funding locahon-specific techinologies and
nologyassessment * Government counterpart andrehabilitatedfacilities * Relocation of MRD bestpractices tested by
institutes
in 12 provinces
to fundsJ$38.8 million) andequipment offering. staffto regionallocations agricultural institutes
serveas regionalcenters * Dotabases onAEZ * Strengthening of * Decentralize agricultural
forfarmingsystems Funds will financecivil characteristics
and extension andsupport research anddevelopment
research andtech- works,equipment, books, regionalinformation services at locallevels system, focused onlocalneeds,
nologytransfer training,research,and * Supportservices for farmers in placewithinfiveyears
technical assistance andextension workers * 30%increase innumber of
on-farm triols,demonstrations,
Theeightinstitutes
willcreate andtechnology tronsfer
fiveyearresearch and activitiesinvolving agricultural
development master plans institutes'
clients
andreceive stafftraining * Train75%of agricultural
andtechnical assistance institutes'
staffin research and
underthiscomponent extension methods, technical
fields,andfarmingsystems
* Increase arlention to social
andgenderissues inresearch
anddevelopment planning,
projectdesignand
implementation, and
monitoring andevaluation
a Ensure thato moreresponsive
anddecentralized research and
development system isin place
withinfiveyears
e 10%increase in productivity
of majorfarmingsystems
* Improve dissemination of
information andnewtechnology
to farmers andusers
Reform research and * Introduce
standard * Increased
commitment* Adoptslandard research
development management researchanddevelopment of localgovernments to anddevelopment management
at theregionallevelto management procedures supportdecentralized procedures for planning, priority
ensure relevance of research testedby theagricultural researchand seiting,monitoring andevalua-
anddevelopment to users institutes development tion,finance,andadministrolion
andto improve research * Develop databases on * 20%increose in useof
qualityandeffectrveness AEZcharacteristics in nine databases byMRD, agricultural
additionalagricultural institutes,
localgovernments, and
institutes universitiesfor national
andlocal
* Setupinformation and planning andpolicymaking
accounting systems in * Implement researchand
17 agriculturalinstitutes development master plansand
* Createfiveyearresearch annualplansandprojects in
anddevelopment master plans 17 agriculturalinstitutes
for nineodditional * Increase technology transfer
agricultural
institutes support forfarmers, extension
* Establish
threepilotsoil workers, andclientsby the
labsforfarmers agriculturalinstitutes
andtheir
* Providestafftrainingand colfborators
technicalassistance on research * Increose AARDstaffl(bout
anddevelopment management 70%in agricultural institutes
for nineadditional agricultural andprojectimplementation
institutes,
theMRD Secretariat, units)trainedin research and
andreseorch institutes development toolsandpractices

Tablecontinues
onthe nextpage.

35
T . 3 (citued)
RISKS
ANDCRMCAL
INPUTS OUTPUTS ASSUM?P1ONS
(Resources
provided (Goodsandservices (Theoutcomeis OUTCOMES AND MPACTS
OBJECTIVES forproject
octvities) producedby the project) dependen on... (of projectactivities)

* Manageinsitutsandstaff
beher
* Increase gavernment
allocationforagricultural
researchanddevelopment
(to 1%of agricukural
GDP)
Fundresearch inpriority * 25%increase in reseorch * L.intedgovernment * 5%increase in governmrent
areasat national
reseorch outputsfromprojects on courhrpartfundsfor allocation foragricultural research
institutes
tosupportregional pnority
areas(livestck, priorityareasand anddevelopment; 10%increase
resech anddevelopment hsheries horticuiture,
tree disciplines infundsfor priorityareosand
programs craops)
andprioritydisciplines disciplines
fbioiechnolpg, agribusiness, * 25% increase innumber of
morketing, sociobogy) technologies generated by
* Rehobilitatespeciolized nationalresearch institutes
for
loboratoriesinselected national on-farm testinganddemonstration
research insritutes by agncultural institutes
* Trainscientisbondprovide * Rehabilitate specialized
tachnicol supportin priority loboratories in national research
areasanddiscipines institutes
to support priorityareas
anddisciplines
* 10%increase in number of
trainedscientists (posigroduate) in
notionalresearch institutes
vwvrking
on priorityareasanddisciplines
* Generate high-quolity research
in nraional research insfHtutes
and
agriculuralinstitutes
* Secure increased and
sustained fundingforpriority
research areasanddisciplines
Improve accessto * Colloborating
inslituions'* Improve
technologies
and * Iaeased language * 30%increase in number of
eenmalygenerated counterpart
funds newinformation fromjoinl capacityof AARD staff colobortiveprojects and
technobgiesand projects
undertheUniversity * Icreasedcommiment activihes vith intemationaland
strengthen
collaboration GrantProgram of ensemnlgroupsto Asio-Pocific researchoenters
withAsia-Pocific
and * Inroduce nevwechnolo- colaborate andfund localuniversities, andtheprivote
international
centers,
the gies,monagement tools,and joird
octivilies sector
privatesector;
and stondard methods fromjoirt * increase exposure of
universihes projects
vwith and
Asia-Pacific Indonesian scientists
ond
international
centers,NARS, monagers to international
and
universities,
andtheprivte regionalforums and20%increase
sectr in number of scientific
papersin
* Trin staf andjointpopers referredjournals
fromtheScientificEchange * 25% increase in scientists,
Program managers, andagriculturolinstitute
stafftrainedoutsideIndonesia
* Increase exposure of
Indonesian scientists
and
managers to globalresearch
system
* Foster collaboration among
NARSandbetweenNARS
and IARCs

Task
manager:DefyGaposin.

36
Table ofobectives
4. Summary ii keyperfonrmce
hWktos,
Uthuonla
SlaulalEnvlunntProedt
(Stuff
Appraisal
Reort1498l-IT,
Nvovmbw
9, 1995)
RISKSANDCRITICAl
INPUTS OUTPUTS ASSUMPTIONS
(Resourcesprovided (Goods
andservices [Theoutcome
is OUTCOMES ANDMPACTS
OBJECTIVES for project
activities) produced
by theproject) dependent
on.. I (ofproject
activitiesl

Reduce pollutant * IBRDloan($6.2million) * Rehabilitate

sewernetwork * Problems
withsecuring * Increase
treated
waste-
loadsfromtheSiouliai * Bilateral
gronts($8.54 * Rehabilitate
wastewater locolfunding waterfrom40,000cubic
areaintotheUpper million) treatment
plant metersa dayto 50,000
LielupeRiverBasin * Government funds * Construct
newwastewater cubicmeters a day
1$7.6millioni treatment
plant * Reduce nitrogen,
M
Municipalityfunds * Implement pollutioncontrol phosphorus, and other
($0.4 million) measuresat pig farms pollutionat thetrectment
* Implementpollutioncontrol plants'outlets,at the mouth
Fundswill be usedto measuresfor agricultural
runoff of theLielupeRiver,and at
procureequipment, works, thewastewater treatment
consultants,andtechnical plant,fromX tonsa yearto
assistanceitrainingl Y tonso year
* Reducepollutionlevelsat
selectedpointsdownstream
fromagricultural pilotsites
and pig farms
9 Lower(byXX}healthcare
costs
* Increase (by X%)tourism
revenues
* Increose international
politicalgoodwill

Improve quality, * Rehabilitate

equipment
reliability,andcost * Providenewequipment
efficiencyof water * Restructure
waterutility
supplyand wastewater * Troinpeople
services in Siauliai
* Abilityto adjusttariffs * Improve drinkingwater
* Revenue collection quality(lowerironcontent
difficulties andsofterpotablewater)
* Politicaldifficulties * Ensure fewerbreaksand
withorganizational troublecallsonthe water
restructuring(staff supplyanddistributionsystem
reduction) and thewastewater
collectionandconveyance
system
* Provide an adequate
operatingratio(lessthan
85%)andodequateworking
ratio(lessthan70%Xfor
thewaterutility

Improve localand * Providemonitoring

and * Potential coordination * Ensure
regularand
regionalenvironmental laboratoryequipment difficulties
between accuralemonitoring of water
qualitymonitoring
and * Provideotherequipment concerned parties quality
enforcementsystemin * Trainpeople * Ensure regularenforcement
theUpperLielupeRiver * Developmanagement plans visitsat pollutionsources
Basin to reduceindustrial
pollution
andsludge
* Developemergency
management plan

Taskmanager:SariSoderstrom.

37
Table
5. Summary
ofobledives
andkeyperformance
indicators,
PonRural
Roads
Rehabilitation
andMaintenacne
Project
(StaffAppraisal
Report
14939-PE,
November
6, 1995)
RISKS
ANDCRITICAL
INPUTS OUTPUTS ASSUMPTIONS
(Resources
provided (Goods andservices (The
outcome
is OUTCOMES ANDIMPACTS
OBJECTIVES forproject
activities) produced
bytheproject) dependent
on... (ofproject
octivities(
Reduce
tronsport
costs * IBRD
loan($90million)
* Bring
5,000kmofrural * Favorable
macroeconomic
* 80%
ofcommunities
in
ond increasereliability ' Inter-American roadsup to adequate conditionsand termsof programareaslinkedby a
of vehicularaccessin Development Bankloan
standardsfor traffic trade reliableand affordable
ruralareasto expand ($90 million) * Bring2,500 kmof * Favoroble natural publictransportation system
markets forogricultural * Government funds
connecting roadsup to conditionsin projectarea * 30%lowerfreighttoriffsfor
andnonfarmproducts by: ($70.2 million) goodcondition * Securitysituation
doesnot transport of agricultural output
* Maintain7,500 km preventexecution of works * 30%lowerpossenger
* Integrating
inaccessibleFundswill be usedto of roadsroutinelyaccording * Localconstruction
industry tariffs for transportin rural
zoneswithregional procuretechnical to guidelinesdeveloped developscapacityto areas
economiccenters assistance,equipment, underthe project undertake largevolume * 30%increase in
and works of works commercial traffic(buses,
pickups,and trucks) within
threeyearsof completion of
roadimprovements
* Alleviateruralpovertyin
the 12 poorestdepartments
(X%changein poverty
measure)

* Improvingtransport * Improve220 kmof streets * Communities

willingto * Reduce
dustpollutionand
conditions
in rural in at least300 villages participoteandcommit increased trafficand pedes-
villages * 20%of iniegrated rood resources triansafetyin 200 villoges
projectsincludea non- * Successfulcoordination * Improve trafficconditions
motorized transport withotherprogramsto and 25 villagesdevelop
component enhanceaccessto inter- capacityto managethe
mediatemeansof transport unclassified neKworks of
trocks
* Raiselivingstandards for
.5 millionruralresidents
throughincreased Iby Y%)
infrastructure access

Increaseemployment * Contractoutto local * Continued government * Generate35,000 one-

throughrehabilitation communities andcontractors supportfor poverty yearequivalentnonskilled
andmaintenance of at least$250 million eradicationpolicyand seosonaljobs
ruralroads invested in works theproject * Generatemorethan
4,000 nonskilled
permanent
jobs

Tablecontinues
on the nextpage.

38
Table
5(confloed)

RISKSAND CRlTICAL
INPUTS OUTPUTS ASSUMPTIONS
(Resources provided (Goodsand services (Theoutcomeis OUTCOMES AND IMPACTS
OBJECTIVES for projectactivities) producedby the projectl dependenton... I (of projectactivities)

Increase
institutional Provide technicalassistance
for: * Government commitment to * Establish ruralroadsunit
copocityat thelocallevel * Planning andmanogement empowering municipalities withinMTCto planand
andincrease capacityfor of ruralroadswithinMTC, * Municipalities'
porticipation monitor rurolroodinvesinment
smallandmedium-size PERT-PCR, andlcol in thetechnical assistance andtransporafion performance
enterprises
to manageand governments program * Revise fundional and
carryoutsustainable * Developing microenterprises* System of intergavernmental jurisdictional
classification
maintenance andupgrading andsmallandmedium-size iransfers andlocalrevenue of roadsandinventory
of roods enterprisesfor roadmaintenance sources in placeasa resuh of all publicroods
* Developing thelocal of thedecentralizationagenda * Streamline localgovernment
constructionindustry. procurement proceduresin
Studieson: accordance withtheaction
* LocalRoadAdministration plandeveloped underthe
Practices LocalRoods Administration
* RuralRoadsFunding Practices study
* Implement mechanism to
securefundsfor rural
maintenance agreedbetween
centraland localgovernments
in accordancewiththe
actionplandevelopedunder
the RuralRoadsFunding study
* Encourage morethan 300
local contradors and
consultants to registerwith
PERT-PCR andparticipotein
projecactivities
* Ensure that100 microenter-
prisesand smallandmedium-
sizeparticipatein routine
maintenance
* Ensure that60 lof 117)
provincialmunicipalities
adoptinstitutionalprograms
designedundertheproject

Taskmanager:
JoseLuisIrigoyen.

39
Table
6. Summryofobjedives
andkeyperformace
Indacators,
ThePhilippires
Womn'sHotho Prolet(StaffAppraisal
andSaf Motherhood 13566-PH,
Report 27,1995)
January
RISKS
ANDCRITICAL
INPUTS OUTPUTS ASSUMPTIONS
iResourcesprovided iGoodsandservices (The
outcomeis OUTCOMES ANDIMPACTS
OBJECTIVES for project
activities) bytheproject)
produced on...j
dependent (ofproject
activities)

health
Improve status * IBRD
loan($18million) * Increase
proportion
of
women
oflow-income of * ABBloan($54million) women receiving
prenatal
and
reprodudive ageby * KfW cofinancing postnatal care,family
substantially ($14.24million)
increasing planning, and RTI/STD and
and * ECcofinancing
theiruseof effective ($13 cervical cancermanagement
sustainablehealthcare million) fromX%to Y%
services.
Specific * AIDAB cofinancing * Lowerfrom60%to X%
development objectives: ($10.55 million) theshare of lowerincome
* LGUcounterpart funds womennotattendedby a
($26.66 million) trainedphysician
* Increase fromX%to Y%the
shareof pregnant and
lactating womencompleting
micronutrient supplementation
regimes
* 25%dropin shareof
wvomen contracting puerperal
infections(among thetotal
number of diagnosed
pregnancies)
* 10%dropinshareof low
birthweight newborns
* Reduce matemal mortality
ratefrom208 per 100,000
livebirthstoX per 100,000
by 2005
* Lower fertilityratefrom
4.3%to X%by 2005

qualityrange,Micronutrients,
* Improving food * IncreasefromX%to Y%the * LGUshavethemanagerial* Ensure that90%of
andaccessto women's fortifioation, immunizations,shareof womenwithaccessto andtechnical capocityto womenreferred with
health
services deliverykits,drugsand motemal andprenatal implement theproject obstetric
complicationsore
supplies, healthfacilities care,familyplanning, RTI/STD monaged correctly
renovation, technical andcervicolcanceroore,and * Adequate cashflowto * 25%increase inwomen
assistance,andmedical counseling support projectacfivities andappropriately
referred
andlaboratory * EnsurethatX%of the * MostLGUs decideto managed for RTls
in theten
equipment to: population resides withinY km porticipateintheproject provinces
* Strengthen maternal of a healthfacility andeffective * 20%increase
* Sufficient inwomen
andprenatal core * EnsurethatX%of women coordination by thevarious detectedand managed for
and services
facilities whosoughtcarethrough a donorsisdeveloped and cervicalcancerinthe 15
* Strengthen family heolthfacilityweresatisfied maintained provinces
plOanning services withtheservices
* Implement RTI/STD
prevention management
program
* Design andimplement
cervicalcancerscreening
program

on thenextpage.
Tablecontinues

40
Thbl6 (continu.d)
RISKSAND CRITICAL
INPUTS OUTPUTS ASSUMPTIONS
(Resources provided (Goodsand services (Theoutcomeis OUTCOMES AND IMPACTS
OBJECTIVES for projectactivities) producedby the project) dependenton...) (of projectactivities)

* Strengthening
LGU Technicalassistance, * Increase to X%onthe-job * Skillsleamedin training * Ensurethatlessfhan25%of
capocityto manage health workshops andtraining, competency levelsof staff programs by midwives healthfocilities
reportstock-outs
services
andDOHcapacitybroadcasttime,office trainedinclinical,diagnostic, andbarangay health of essential
dwugs
to providetechnical,
finon- andotherequipment, counseling, interpersonal, workers areused * Reducecostof logistics
to
cial,andlogistical
support travel,salaries honoraria,communication,historr effectively
ascommunity lessthan8%of valueof
incremental operating taking,and referral outreochtools commodities
costs,supplies,computerpractices * Midwivesandbarangay
hardware andsoftware, * Increase to X%the healthworkers arenot
warehouse construction timeliness
andaccuracy tooaverburdened with
andrenovation, vehicles,of procurement
anddistributionadditionalpatients
to
logistics,
andmaterial of drugs,reagents, andother deliverqualityservices
productionpublication, medicalsupplies * Localcontractorsand
anddistribution contracts consultingfirmshavethe
to: technicalexpertise
to conduct
* Designandimplement studies
in-service
training
program
forpublichealthcare
workers
* Establish
andoperate
projectmonagement office
* Strengthennotionol
publichealthandlogistics
system

* Increasing
local * Designedandimple- * FormXcommunity * Ensure
thatat least80%of
andNGO involvement in mentedhousehold-evelpartnerships toempower thosesurveyedareawareof
women'shealthprograms communityoutreach womenandtheircommunities onepreventiveintervention
for
programs to improvetheirhealth RTls,twosymptoms of RTIs,
and
* Designedandimple- * Provincelevel LGUsconduct onecomplication dueto RTI.
mentedintegratedinfor- Xtrainingprograms * Ensurethatat least90%of
mation,education,and * Fund X proposals womensurveyed knowabout
communicationstrategy submiffed by NGO antenatal
Tr injectionsandiron
* Designedandimple- partnerships at community folatesupplements
mentedcommunity level
partnerships
program

* Improving
knowledge * Consultants,
studies, * Complete X studies
bosefor healthpolicy monitoring,
and on thecosteffectivenessof
formulation evaluation servicedeliveryoptions
* Improve healthpolicy
framework leadingto increased
cost-effectiveness
of various
optionsto deliverhealth
servicesto women
* Establish protocolsfor
betterdetection anddiagnosis
of RTIs
andeorlycervicalcancer

Taskmanagers:
StanleyScheyer,
Maria Dalupon,andRamaLakshminarayanan.

41
Tab7. Sommnry
ofobletWvs
_I keyperfomance
idiators,
Venezuela
Aglcultural
Extension
Proec(Staff
Appraisal
Report
13591-VE,
March
7,1995)
RISKSAND CRITICAL
INPUTS OUTPUTS ASSUMPTIONS
(Resources provided (Goodsand services (Theoutcomeis OUTCOMES AND IMPACTS
OBJECTIVES for projectactivities) producedby theproject) dependenton...) (of projectactivitiesl

Helppoorfarmers raise * IBRD loon1$39million)* Establish extensionoffices * Availabilityof counterpart * Farmers learnaboutand
theirproductivity
and * Government funds ineachstateandmunicipality funding adoptnewtechnologies
incomes ($21 million) (18 statesand 180 * Sufficient
administrative (measure: Xbeneficiaries adopt
* Stategovernment funds municipalities) coordination
in at leastonetechnology
Helpsmallfamiers ($11 milionl * Provide50 trainingprogramsestablishing
a new recommended by the
improvetheenvironmental* Municipalgovernmentforextensionists eachyear nationwideprogram extension servicel
sustainability
of their funds($6 millionl * Providefourextensionvisits * Extensionists learnnew
agricultural
activities * Farmers funds to eachformeachyear, technologies fromuniversities
($2million) covering90,000 farmers andspecialists (measure:X
* Fundat leasttwmannualvisits trainingevents andfielddays
Funds will beused bysubjectmotter specialists eachyear,withYextensionists
to trainagricultural to everymunicipality's
farms participating)
extensionists,establish andextension offices * Appliedresearch conducted
newextension offices, * Sponsor onefielddayeach to address farmer's expressed
andprovideextension yearin everystate needs(measure: Xfieldtrials
visitsto farmers * Sponsor twofieldtrialseach conducted by specialists
yeorin everymunicipality together withextensionistsl
* Establish
Farmers associations * Higherincomes for90,000
ineverymunicipality beneficiaries(measures: X°h
0 of
* Coordinate montlymeetings participatingfarmers haveyield
of extensionists
andsubject increases overprevious
matterspecialists
ineachstae period,average yieldincrease
Y/ of participating farmers
increase theirnetincomeover
previous period)
* Economic growthin rural
communities (measure: average
netincomechangeover
peviousperiodof participating
famers)
* Lessenvironmental domage
(soildegradation, erosion,
runoff)
causedby poorfarmers
(measure: Xpotson
participatingfarms with
environmental improvement
overprevious period

Taskmanager:DavidNielson.

42
Table
S. Summary
ofobledives
andkeyperfonmance
indiktors,
Chad:
Strudural
Adjustment
Credit
(President's
ReportP-6785-CD)
RISKSANDCRITICAL
INPUTS OUTPUTS ASSUMPTIONS
(Resources
provided IGoods andservices iTheoutcome
is OUTCOMES ANDIMPACTS
OBJECTIVES forproject
activities)produced
bytheproject) dependent
on...) (ofproject
activities)
Resbore
publik
sector
capacity
andimprove
pubik
secor
finances
Civilservice
reform * $30million * Completeacivilservice * Norenewalinpolitical
unrest* Reducecivilservice
vmge
* Capocity-buildingcensus * Sufficient
institutionol
capocitybillfrom
5.8%ofGDP in
project
($9.5million) * Abolish
automoaicwage * Newadministration follows 1995to5.1%in 1998
increases through
onprogram
* Prepare
anorganizational
chort
forcivilservice
Public
finance * Introduce
single
taxpayer * Increase
revenue
from9%
rehabiliiation code ofGDP in 1995to 11.5%
* Reformspecial
enterprise in1998
regimesinlinewithregional
economic agreements
* Renegotiatecustoms
treatment
forprivate
firms
and
public
enterprises
e Centralize
collection
of all
international
trade
taxes
e Establish
exoneration
quotas
forimports
bydiplomats
Increase
development
andpoverty
alleviation
impact
ofpublic
spending
Demobilization * Reduce
armysize,
with * Cutarmy
from47,000
to
assistance
forresettlement 21,000
soldiers
Public
expenditure * Abolish
off-budget
management operations
* Eliminate
arrears
to
domestic
suppliers
Development
ofpublic * Reach
agreement
with In1996budget:
expenditure
program IDAon1996budget
and * Limit
wages
andsalaries
for 1996andcapital 1996-98publicinvestment to5.7%of GDP
expenditure
program program * Limit
goodsandservices
for 1996-98 to 2.6%of GDP
* Lmit
overall
current
expenditures
tolessthan
13.5%ofGDP
* Limit
capitol
expenditure
toless
than14% ofGDP

Table
continues
onthenext
page.

43
Taleb
8 (cont0Ined)
RISKSAND CRITICAL
INPUTS OUTPUTS ASSUMPTIONS
[Resourcesprovided IGoodsand services (Theoutcomeis OUTCOMES AND IMPACTS
OBJECTIVES for projectactivities) producedby theproject) dependenton...) (of projectactivitiesj

Improved resource * Increase

resources
for In 1996budget:
allocation
to social primaryhealth,primaryeduca- * 26%increase inallocation
sectors tion,welfareof womenand to educotion,heolth,ond
children,
androadmaintenance womenandsocialaffairsover
1995 level(20%real
increase(
* 43%increase in road
maintenance, fromCFAF2.8
billionin 1995to CFAF4.0
billionin 1996
Createfavomble
nvironmient
for
prisvte
seor gwth

Strengthened
judicial * Simplifyprocedures * Increase
pivateinvestment
andregulatory
environment fr licensing businesses from1.4%of GDPin 1995to
* AmendArticles116 and 4.5%in 1998 (excluding
142of laborcode impactof Dobaoil)
- Simplifyinvestment
code
* Amendthebusiness code,
includingcompony and
bankruptcy laws
* Ratify
OHADA

Financial
system * Sellpublicsharesof BTCD
restructuring andBDT
* SellMeridien's
shareinBMBT
* Reducepublicsector
membership on theboard
of CNPS
* Preporeactionplanto settle
government'sdebtto CNPS
andCNRT

Stateenterprise
divesiture * LiquidateSONACOT, BICIT
CoisseSucre,andFIP
* Prepore HotelduChari,BIEP,
INT,andONHPVforprivatization
* OfferSONAPA,SOTEC,
BDT,BTCD, andBMBTfor
privotization
* Initiate
divesiture
strategy
for
STEE ONPT,TIT,COTONCHAD,
andSONASUT

Cottonsectorreform * Launch
COTONCHAD * Increaseproducerpricefor
human resourcedevelopment firstgrade
cottonfromCFAF
strategy 120 perkilogramto CFAF140
* Improveproducerpricing perkilogram
* Adopttwopartcotton * 10%annualincrease in
producerpricingmechanism cottonexports
between1995
and 1998

Taskmanagers:
AmadouCisseandEmmanuel
Akpa.

44
Table
9. Summary
ofobledives
andkeyperformace
Indiators,
Morocco:
Finandal
MarketsDevelopment
Loam(President's
Report
P-6633-MOR)

RISKSANDCRITICAL
INPUTS OUTPUTS ASSUMPTIONS
(Resourcesprovided (Goodsandservices (Theoutcomeis OUTCOMES AND LMPACTS
OBJECTIVES for projedactivities) producedby theproject) dependenton...) lof projectactivities)

Treasury
financing
EnobleTreasury to finance * IBRD loon($250 * Eliminate
mandatory
Treasury * Nonagricultural
GDPgrows* Increase
ratioof Treasury
itselfat market
terms million) bondto sightdeposits
ratio from2. in 1995 to 3.2% bondauctions to totalTreasury
in 1996, to 3.6%in 1997 domes6c debtfrom25%in
1995 to 30%in 1996, to
35%in 1997
Increaseliquidityof * Technical
assistance* Adoptlw on negotiable * Ratio
of debtoutstanding
to
publicsecurities
and * Debtmanagement--securities GDPfallsfrom221%in 1995
strengthensecondary $50,000 (France; * Issue CentralBankcirculars to 200% in 1996, to 182%
marketsfor Treasury bonds on Treasury
bondtenders in 1997
Eliminate
remaining * Accounting- * Eliminate
fiscalincentives * Current
account
deficit/GDP
taxbias $75,000 (Froncel onTreasury
bonds ratiofallsfrom3.5%in 1995 to
2.9%in 1996,to 2.8%in 1997
Strengthen
publicdebt * Complete
trainingprogram * Budget deficit/GDPratiofalls
managementsystem from3.5%in 1995 to 3.0%in
1996, to 2.5%in 1997
* FixedcapitalFormation/GDP
ratioincreasesfrom21.2%
in 1995t 22.0 in 1996,
to 22.8 in 1997
* No deteriorationinbanks'
Finoncialcondition
Indirect
monetory
conto
Implement
indirect * CentralBanksetsboserates * Limitratioof Central
Bank
monetary
controlpolicy * Issue CentralBankcircularestob- advancerateto interbonk
lishingterms
andconditions of its ratemaximum spreadto 2%
interventions
inmonetary market through1997
* Calculate
reserverequirement
ona dailybasis
Liberalize
interest
rates * Eliminate
ceilingon lendingrates
Capitalmarket
developrmnt

Diversify
financing * Technicalassistance * Issue decrees on stack * Increase
corporate bond
sourcesforprivatefirms, for a centraldepositoryexchange issuesfrom0 in 1995to
strengthenaccounting anddeliveryand * Implement accounting DH 2 billionin 1996, to
fromework, anddeepen poyments system- framework DH4 billionin 1997
capitalmarkets $1 million(Francel * Present draftlow oncorporations * Incrrease
ratioof stock
to Houseof Representatives marketcapitalization
to GDP
* Approve rulesgoverning from17%in 1995to 19%
nationalcouncilforaccounting in 1996,to 23%in 1997
* Presentdrahlaw on
preporation, publication,
and
cerfification
of consolidoted
financialaccounts
* Eliminote government
guarantees ondomestic bond
issuesby publicenterprises
* Develop occounting rules
forinsurance companies

Tablecontinues
on thenextpoge.

45
Te 9(Cethied)
RISKSAND CRmCAL
INPUTS OUTPUTS ASSUMPTIONS
(Resources provided (Goodsand services (Theoutcomeis OUTCOMES AND IMPACTS
OBJECTIVES for projectactivities) producedby the project) dependenton...) (of projectactivities)

Increase
institutional * Technical
assistance
for * Competestudyof and Sameas above(increase
savings contractual
savings- achonplanon potential role corporate
bondissues,stock
$600,000 (Francel of institutional
investors market
capitalization)

Encourage investors
to * Technical
assistance * Definetradingfees
trodesecurities
onrfloor onstockexchange
andincrease transparencyoperations,
upgrading,
of stockexchange andtroding(CIDA)
operations

Strengthen
securities * Implement
actionplanto
commission
andstock strengthen
securifies
commission
exchange andstockexchange

Bknmng
syslen

Accelerate
prvatization * Sellstate's
sharesinBCP,
program BNDE,andCIH

Establish
foreign * Announce
creationof forex
exchangemarket market
* Issue circularestablishing
forexmarket andmodalities of
operafions
* Issue accounting standards
forforeigncurrency operations
by banks
* Eliminateinitialminimum
IDH 100,000)required to
opennonresident Moroccan
accounts in foreigncurrencies
* Increase foreigncurrency
limitsof exporters of goodsand
srvicesfrom5%and 10%
to 20%

Strengthen
bonk * Develop
offsiteandon-site
supervision supervision
* Submit
regularreports
preporedby independent
external
auditorsoertified
by
theCentralBank

Taskmanager:Emmonuel
Forestier.

46
 Distributors Oi ~ F.wCHNA P&aarroc SA L CosrruTocnau Sansor SPA Navesan hronrnabnCenter SINGAPORE,TAIWAK VanDiarrms E r WerM
DiShibUtors of Chra Frralciat6Eronox 35 StnamStr. VlaDu D Carasbr 1/1 BookDepnmarnte tYANMAR BRUN Ch.de Laa 41
W orld Bank PuthAngHoLsce 10682AEhenri Ctle PosWeS52 PO.Bo er.ad
v". n DaFoSiDonsgJA
8~~~~~~~~S Tel.(1)%4-IM2 50125Frrene N48D020dso6
A Pa12tMeEtsahaP
W"Asia
PacificPie.Lid
CH18D7Ebn
Tel(021)943267
Publications Beyaqo Fx: (1)364-8254 Tel:(55)645-415 Tel (22)57-3300 41 KA"rPdds Road#04-03 Fax.(021)94338S
Prircs Prices
redtcutersvry.
ndcredit
and terms wry Fac (56)10-401-7345
Tel (56)
10-333-8257 HONG KONG,
MtACAG FaGKNdAL
F(55)641U257 Fax (22}68-1901 Swngapore
G 349316
aok Whee
Biiding TANZANIA
fToam country to country. Asia
25 Ltd JAMAICA PAKISTAN Tel.(65)741-5166 Ordod
UreausitPress
Consult your local distributor COLOMBIA Seles&C4culabonDepeatrnt Ian RardtePLbtshs Lid MArzaBookAgasey Fax (65)742-9356 mottia Stre
beforeplacing an order kdocanwe Ltea. SeabrdHouse.unil1101-02 206OldHopeRoad 65,Shrlerah-ea e-Azemn a-mae.arhgaleOas riaors cOn PO Box5298
Apetado Aersa34270 22-28Wrdne Streea Cenate Kinslon6 P.O.BoxNo 729 Dares Saealra
Tel (1)205-2790 HorgKong Tel:809-927-2095 Laxare540D SLOVAK REPUBLIC Tel:(51)292D
ALBANIA Fax (1)285-2798 Tel:952253D-1409 Fu 809-977-0243 Tel (42)7353661 SovalrG.TG.Lid Fax(51)46922
AdnmLid. Farx8522526-1107 Fax (42)7565263 pnttWea4
Plerl Rexheip
Slr. COTEDW1OIRE URL:ht0pJAr.saie@seasia2DDD.oo.hk JAPAN POBox152 THAIAND
Pal 98StLI Ap14 CantecEdism atdeDdilom Easterrn
Bok Servce OntordUrw ty Press 852996bsdevaS Carel Batooks
L3b6tn
Team Kte s (CEODA) HUNGARY Hrsgo3-CSane, Bangoa lor"n Tel:(7) 39472 306SilomRoad
Ta (42)2741922172 04B.P.541 Founat tort4aet Burkyo-kul3 SharaeFaisat Faxa (7639485 Bwtkok
Fax (42)27419 Abeae04 Platu Ecorny Tokyo PCBox13633 Tel.(2)235-540
Tel:225-24t610 D)ardbsal ( 17-19 Tel (03)3918-1966 Karaehi-75350 SOUTH AFRCA, BOTSWAANA Fea (2)237-8321
ARGENTWA Fee 225-25-0567 H-ll BII7 iBudvpt Fax (03)3018-0664 Tel (21)446307 For me eas.
OticinadelLie trternrtorat Tel:36 12042951or UFL hItpiMwbekko&.or.p(/-svI-ebs Fax:(21)454-7640 OxtordUnserenyPress TRtNIDAD& TOBAGO,JAIL
AvCordoalSi7 CYPRUS 3612042948 E0-mad:
oapO p kIeccmcootpk SnAham Atrnc Syl ah SaedisUInd
Il2DBurmnAkes Canrl orAppbed Farch Fu: 3601 2042953 KEYA PO Box1l41 tl9WadsSbreel
TtL (I) S1543156 Cyprus Carge AtricaBeakSamoc(EA) Ud. PERU CapeTossnW Curepe
Fax (1)8154-4 6,DiogesesStt Engo tIA CUrs House, Wna o Sueet EddonitDesamrcoSA TexI(21)45-7266 Tbi, West lAn
P.O.Boo2DD AbedPutus Ltd. PO Box45245 Apest 3824 Fa- (21)45-7265 Tel W6-68-5654
AUSTRALIA, RJ, AMPUANEWGLKNEA, Naesi 751MourilRad Natob Lman I Fax: 80966625654
SOLOMON SLAND, VAt41ATU, AND Tl: (2)44-1730 Madas-50002 Tel: (2)23641 Td4 (14)2830 Forebtc n orders.
WESTERN SAMOA Fax (2)46-2051 Tat:(44)852-393 Fax:(2)330272 Fax:(14)28626 SArdcnpbon
ntnMeralitDa Serve UGANDA
DA IlotrukalonSrces Fax (44)852-649 P.O.Bax41095 GLAso Ltd
6486WtalhoRoad CZECH RE BLtC KOREA, REUBLICOf PItIJltES Crall" MaCrar Bi*dg
lcartn 3132 NaIselotrsmaori CantWr tNDONESSA Deo TdhivgCo.Ltd. hlerninla BODIsotare
Calmeroc Jarrebg 2024 POBox9897
Vire POd*i, K rska 5 p1 ksdeLiniLed P.O. Br 34 SLAG 720 iyland10 Tet.(11)88D 844 PM 16/4Jqa Fd.
Td. (61)39210 77T7 CS- 11357PragueI 20
JalanBomabudLr YaDida Caxmorerhn Tower2 Fax:(11)880-6248 Krrab
Fax (61)3921O 7788 TeL(2)2422-9433 PO. Boa181 Seiu H.VdedaCos"acorner TeV/Fax:
(41)254763
URLtlinpAtwaradret.corLeu Fax (2)2422-1484 Jeana 10320 Tel:(2)735-1631/4 VaeroSI. SPAIN
URLtipJAwMwMW.r2 Tel:(21)390-4240 Fa (2)784-0315 Matab,Mlts Mus Musni-Prrrea
Lienos.
SA UN1TED KV3 WM
AUSTRIA Fax: (21)421-4289 Tat:(2)817-9676 CaslaloS37 Maods Ltd
Garld ansdCo DENMARK MALAYSIA Fax (2)817-1741 28DD1Maidd PO.Box3
Graben 31 SwnkAram tRAN Unrery at Maya Coopereli Te: (1)431-3399 AMorlHwalne GU342PG
A-lOll Wb1 RoWF rerInr4 th11 K
Kra Putibs Rookiop,Ulred POLAND Fax (1)575-3998 England
Td (1(51247-31-0 OK-1970FradbrnrergC PO.Boo19575-511 PO.Box1127 kieandiobrm
Pddeg Serse ht9pJA BalasiArPesa Tel (1420)86840
Fax (1)512-47-31-29 Tel:(3l-351942 Talrs JaenPPbuamBr Ul.PoleAs311/37 Fax (1420)888M
Fax (31(357822 Te: (21)258-3723 59700Kuaea Luirpur 00-577Werzrwe Matrd-PraenBardona
JIGLADEBH Fax98(21)258-3723 Tel:(3)756-50D Tel.(2)62S-6089 CoaredeCerl, 391 ZAMBIA
Moo AstatO Deoeepmenwl EGYPTARABREPUBLIC OF Fax (3)755-24 Fax (2)621-7255 0O9 BarO Uarsty Bookshop
AsAna SDiely(MIDAS) AtAhrrneu
DOibm Agencry KOWSae Ca.Puths Tat:(3)4-30DM GrCt EaSRoodCUT
Hoa S.RoPd 16 AlGae Sas KhaledEslasLbAve- MEXCO PORTUGAL Fax (3(487-7659 PO.Box32379
DlwOs t FVArea Cairo 6tnSbeel WFOTEC LAmed Poulpt Luab-k
Dhake1209 Tel-(2)57-0603 KuarhehDeelmo ND.8 Apauloo Postal22-660 RoLDoCasino70-74 SRILANKATHE MALDES Td (1)213221
EaL482
T. (2)326427 Fax:(2(576-633 Te*M 14060Tiper, 120 LUsbo LakeHorse Bookshp
Fua (2(81118I Td: 871719or8716104 MetxioD.F. Tel:(1)347-4982 PO.Box244 ZIMBAt1E
Thebddt EastOter Fax 8682479 TdW:
(5)60-011 Fax:(1)347-024 155,Se9 rirtkA LA.rrnWTFZimabwe0.)aW
BELOWI 41,StartSIre E-rca.tralatseagneOdocdhr Fax:(5(624-2822 Gaedinw Maretr TrouleRoad.Asbnar
JaantDe Lmb Cairo x ROMAIA Cobnibo2 PO.Bo.STl25
A. duoir202
l2) Tel:(3973 tELAUND NETHERLANDS Coewd DeLbmrrBucuresiSAk Tat.(1)32105 Sorenon
10968Bet_ Faxr(2)393-9732 Gvae Suppiess
Agency De LnetboomtnOr-Pudbtkalad, Str.LcaO no.26, ndra3 Fax (1)432104 HarMu
Tit 2)5385169 Oftigan tSoM P.O.Box22 Baucdrst Tel:(4)6216617
Fax (2)531141 FAD 4-SHaro4r Road 7480AEH-ksbaerge Ted:(1)6139645 SWEDEN Fax:(4) 621670
Akeerwan Keka;;ppa DrtAn2 Te1:(53)574-604 Fax(1)3124000 Fdza Cua m Service
IlAZL PO.Box120 Tel:(1)461-3111 Far (53)572-9296 Regaking l2
Putae cnlariciYatkfrr
mis FSN40101 HeSsnli Fax (1)475-2670 RUSSIAN FEDERATION S-10647Shddcir
WrL It. (0)12141 NEWZEALAND dweabeovs Mr, let )69D9089D
Rem Patb Gre,, 209 Fax:p) 121-4441 RAIEL EBSCO N12
Ld. go,Krovh* PereubL Fax () 214777
01409 SaoPW, SP. URLttpkoolinelztatln bal YoresolLIa e Lbi Prva5bMelBag89914 Moscw 101031
Tt (11)2z94 P.O.Box 6056 New el Tel:(95)9178749 Wuweeal-WituirAS
Fac III) 258-60 FRANCE TelAviv61560 Aucland Fax (95)91762 59 P.O.Box1306
UFLht.IpJawoLbk WortBan Preddte Tet:(3)5281397 Tet:(9)524-8119 S-171 25 Soae
66, ensu lire Fax (3)5285-307 Fax 9)524-8C67 SALDARABIA, QATAR Tet: (8)70F-7-50
CAIIADA 75116Paw Jari BookSUore Fax (8)27-071
Rard Pawt"igC. it Tet (1)40-JI14-56V7 FO.Y.Ineren NIGERIA PO.Box3196
1294 AJRomFed Fax (1)40-88-30-56 POBox13056 Uriiseady
Pre LUmed Riyadi11471 SW6YZENLAO
Onms0ChnsKRB3W8 TelAvri61130 TlseCwOwBuildingJetldro Tat:(1)477-3140 L4x iiePayol
Tdt61-74143 GERMANY Te()9 5461423 Priate ModBg 5095 Faxx(1)477-2940 Serniickub trSw9
F&ac 613-741-5439 UNO-Vs Far (3)541442 budan Ce4-brte 30
URL,htMii1wabxrLC/-eufi Poppledler Ai 55 Tat:(22)41-13506 1002LauttxW
53115Brm Pattie Aedspib ES Fasx(22)41-2856 iat 028441-3229
Tt MO 212940 tdresbirrr enSie Fax (021(3414251
Fax (228)217492 P0.B 19562Jerral
Tel- 21271219
RECENT WORLD BANK TECHNICAL PAPERS (continued)

No. 295 Pohl, Jedrzejczak, and Anderson, CreatingCapitalMarketsin Centraland EasternEurope

No. 296 Stassen, Small-ScaleBiomassGasifiersforHeatand Power:A GlobalReview
No. 297 Bulatao, Key IndicatorsforFamilyPlanningProjects
No. 298 Odaga and Heneveld, Girls and Schoolsin Sub-SaharanAfrica:FromAnalysis toAction
No. 299 Tamale, Jones, and Pswarayi-Riddihough, Technologies Relatedto ParticipatoryForestryin Tropical
and SubtropicalCountries
No. 300 Oram and de Haan, TechnologiesforRainfedAgriculturein MediterraneanClimates:A Reviewof WorldBankExperiences
Department,Africa Region,July 1987 to April 1995
No. 301 Mohan, editor, Bibliographyof Publications:Techinical
No. 302 Baldry,Calamari, and Yameogo, EnvironmentalImpactAssessmentof Settlementand Developmenitin the UpperLeraba
Basin
No. 303 Heneveld and Craig, SchoolsCount:WorldBank ProjectDesignsand the Qualityof PrimaryEducationin
Sub-SaharanAfrica
No. 304 Foley,PhotovoltaicApplicationsin RuralAreasof the DevelopingWorld
No. 305 Johnson, Educationand TrainingofAccountantsin Sub-SaharanAngloplhone Africa
No. 306 Muir and Saba, ImprovingState EnterprisePerfornance:The Roleof Internaland ExternalIncentives
No. 307 Narayan, TowardParticipatoryResearch
No. 308 Adarnson and others, EnergyUse,Air Pollution,and EnvironmentalPolicyin Krakow:Can EconomicIncentivesReally
Help?
No. 309 The World Bank/FOA/UNIDO/Industry Fertilizer Working Group, Worldand RegionalSupply and DemandBalances
for Nitrogen,Phosphate,and Potash,1993/94-1999/2000
No. 310 Elder and Cooley,editors, SustainableSettlementand Developmentof the Onchocerciasis Control
ProgrammeArea:Proceedingsofa MinisterialMeeting
No. 311 Webster,Riopelleand Chidzero, WorldBankLendingfor SmallEnterprises1989-1993
No. 312 Benoit, ProjectFinanceat the WorldBank:An Overviewof Policiesand Instruments
No. 313 Kapur, Airport Infrastructure:TheEmergingRoleof the PrivateSector
No. 314 Valdes and Schaeffer in collaboration with Ramos, Surveillanceof AgriculturalPriceand TradePolicies:A Handbookfor
Ecuador
No. 316 Schware and Kirnberley,InformationTechnologyand NationalTradeFacilitation:Making the Most of GlobalTrade
No. 317 Schware and Kirnberley,InformationTechnologyand NationalTradeFacilitation:Guide to Best Practice
No. 318 Taylor,Boukambou, Dahniya, Ouayogode, Ayling,Abdi Noor, and Toure, Strengtlening NationalAgriculturalResearch
Systems in the Humid and Sub-humid Zonesof West and CentralAfrica:A FrameworkforAction
No. 320 Srivastava, Lambert and Vietmeyer,MedicinalPlants:An ExpandingRolein Development
No. 321 Srivastava, Smith, and Forno, Biodiversityand Agriculture:ImplicationsforConservationand Development
No. 322 Peters, TheEcologyand Managementof Non-TimberForestResources
No. 323 Pannier, editor, CorporateGovernanceof PublicEnterprisesin Transitionial
Economies
No. 324 Cabraal, Cosgrove-Davies, and Schaeffer,Best PracticesforPhotovoltaicHouseholdElectrificationPrograms
No. 325 Bacon, Besant-Jones,and Heidarian, EstimatingConstructionCostsand Schedules:ExperiencewithiPowerGeneration
Projectsin DevelopingCountries
No. 326 Colletta, Balachander, Liang, The Conditionof YoungChildrenin Sub-SaharanAfrica:The Convergenceof Health,
Nutrition, and EarlyEducation
No. 327 Valdes and Schaeffer in collaboration with Martin, Surveillanceof AgriculturalPriceand TradePolicies:A Handbookfor
Paraguay
No. 328 De Geyndt, SocialDevelopmentand AbsolutePoverty in Asiaand LatinAmerica
No. 329 Mohan, editor, Bibliographyof Publications:TechnicalDepartment,AfricaRegion,July 1987to April 1996
No. 332 Pohl, Djankov,and Anderson, RestructuringLargeIndustrialFirmsin Central andEasternEunmpe:An EmpiricalAnalysis
No. 333 Jha, Ranson, and Bobadilla,Measuringthe Burdenof Diseaseand the Cost-Effectiveness of HealthInterventions:A Case
Study in Guinea
 i~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

i%P~~~~~~

II I

I . ,i!I S

-~~~~~~a -

:~~~~~~~~~~~~~~~~~~u.

I~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Ill ''ll, l