Participatory Monitoring and Evaluation

Detailed Outline

TABLE OF CONTENTS

Section I: Introduction and Overview

• Welcome
• Shifts in Development Thinking
• Learning Objectives
• Methodology
• Target Audience
• Expected Outcomes
• Lesson Plan
• Exercise

Section II: Concepts and Principles of Participatory Monitoring

&Evaluation
• Definitions
• Indicators, Outputs, Outcomes And Impact
• Quantitative, Qualitative and Participatory Methods
• Principles of PM&E
• Role of Gender
• Value-Added of PM&E
• Exercise

Section III: The PM&E Process Cycle

Section IV: PM&E Case Studies

• Micro or Project Level
• Meso or Local Governance Level
• Macro or Policy Level

Section V: Exercise: Group Action Planning

Section VI: Additional PM&E Resources

Appendix A: Monitoring Mechanisms

References

1
SECTION I: INTRODUCTION

Welcome
Most of us would agree that in theory, evaluation is a good thing. But putting this ideal into
practice is often seen as too complicated, conflict-ridden, and/or costly. This Module is designed
to introduce diverse development stakeholders to Participatory Monitoring and Evaluation in
different contexts – what it is and how it adds value to development work at both the project and
the policy level.

Participatory Monitoring and Evaluation (PM&E) is a tool for bringing diverse stakeholders
together, giving them an opportunity to negotiate regarding what success should look like and
what indicators should be used. This process of negotiating the definition of success is a first
step in institutionalizing participation and voice in development. Whose information is used to
assess the success of a policy or project is important if we aim to make projects and policies
more grounded in the realities of poor and disenfranchised people. PM&E ensures that all
stakeholders have a voice in what indicators are used and how the results are analyzed. Multiple
sources of information guarantee the greatest accountability and triangulation of results, while
allowing for the monitoring of monitors. There are many potential positive spillovers from
PM&E -- for example, experiential learning through working in diverse groups builds trust and
the ability to cooperate across sectors/stakeholders for future development endeavors.

Shifts in Development Thinking

PM&E encompasses some of the major conceptual shifts in development work today. In
accordance, this Module is based on the following assumptions:
1. Participation and ownership of all stakeholders is essential for sustainable development.
2. Monitoring and evaluation are critical for holding development actors (all members of
society) accountable, measuring our successes and learning from our mistakes. This means
going beyond measuring inputs and outputs.
3. Expertise is not the monopoly of technocrats; people at all levels hold important relevant
information.
4. Working across groups in an atmosphere of trust (with some healthy skepticism) is
important. Experiential learning prepares people to work cooperatively across groups and
organizations; people often need spaces and opportunities to build healthy social capital
across institutions.

Learning Objectives of this Module

By the end of this module, participants should be able to:
Articulate the importance of monitoring and evaluation – who measures; what goals; and
how should it be done;
Highlight the value-added of participatory monitoring and evaluation versus traditional
approaches;
Describe the process cycle for designing a participatory monitoring and evaluation program
applicable to various projects, policies and processes;
Appreciate the wide variance in evaluation approaches; each approach needs to be tailored to
specific context and capacity;
Recall model practices and case studies on PM&E in different settings and refer to additional
on-line resources;

2
 Collaborate with other groups of stakeholders on PM&E activities, based on practice during
the module;
Call upon a community of practice to support one another post-training.

Methodology
In order to meet the above objectives, this Module is delivered in a participatory, interactive way,
and is intended to be adapted to your local needs. Follow-up activities may involve Distance
Learning (computer-based/internet training and video-conferencing) where appropriate. The
minimum time required for delivery of this module is three hours, however several face-to-face
and/or Distance Learning activities can be added to deepen understanding and skills. Case
studies will also be used to help participants apply their learning during the course itself.

Target Audience
The primary audience for this Module is community-based groups. Other target audiences
include representatives from NGOs, local and national government, private sector, media, think
tanks and donors organizations. Ideally a diverse group of organizations will be represented
during the training and have the opportunity tot work together on some of the practical
applications during and after this module. Trainers for any of the above mentioned
constituencies will also be targeted.

Expected Outcomes
By the end of this Module, participants will hopefully view PM&E as a doable, collaborative and
worthwhile investment for designing and delivering programs and policies that meet the needs of
a wide range of stakeholders, especially poor people. Participants will be familiar with the
PM&E Process Wheel and will have practical experience working through some of its stages.
Participants will also know where to find additional resources and technical assistance via the
internet and through the community established during the training. Note: this module will offer
sufficient background and experiential learning for stakeholders to participate in on-going or
future PM&E activities. However, it will not provide sufficient technical skill-building to enable
all participants to design and implement their own monitoring and evaluation. Additional
learning will probably be necessary.

Lesson Plan
• Section II presents a Conceptual Framework including Definitions and Principles of PM&E.
• Section III introduces the PM&E Process Cycle which can be modified and applied in a
variety of different contexts.
• Section IV provides short Case Studies of PM&E at the micro, meso and macro levels with
Learning Activities which allow you to practice different stages of the Process Cycle.
• Section V offers space for Action Planning regarding potential PM&E relevant to your own
context.
• Section VI refers you to additional resources (some of them on-line) and opportunities to stay
engaged in a PM&E Community of Practice in order to be involved in follow-up learning
activities and to request and/or provide technical assistance to and from your fellow
participants.

3
Before You Begin…
Before you begin, we would like you to reflect a little on your own situation by answering the
following questions.

1. What project or policy in your local community or region would you like to see
monitored and evaluated using participatory techniques?
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________

2. What are two expectations you have for this course/module?

___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________

3. How will we judge success at the end of this module? How will we know if this course
has been successful and if it has met your expectations?
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________
___________________________________________________________________________

=====================================================================

SECTION II: CONCEPTS AND PRINCIPLES OF PARTICIPATORY MONITORING

& EVALUATION

=====================================================================
Definitions

Monitoring examines whether the project has been implemented as planned, made good
progress, and faced any problems. It also considers how to solve such problems. Monitoring
helps all stakeholders to recognize problems and obstacles, and to be able to solve them
immediately. It thus plays an important role to ensure that activities are performed on schedule
and within the allocated budget, that all concerned are satisfied, and that the project outputs and
outcomes correspond to the objectives. It enables a more responsive, accountable and
transparent system of operating.

4
Evaluation systematically judges the value of changes (planned and unplanned) resulting from
project outputs and outcomes. Evaluation also reveals the state of changes, the persons such
changes affect, and the extent of the benefits from such changes. Evaluation helps to target
project outcomes, and the extent and features of project benefits. This makes sure that we
measure not just inputs and outputs but what significant changes have taken place.

Monitoring and evaluation yield the greatest benefits when they are performed in a
participatory manner. In this process, all stakeholders discuss and plan the project together
from the outset, jointly setting the objectives, targets, and work process. The evaluation should
take into consideration the impact upon individuals and the community in terms of the
development of new ideas and quality of life, resource mobilization, income distribution, self-
reliance, and environmental and natural resource conservation.

Participation effects monitoring and evaluation in the following ways:

• The purpose of the evaluation
• The criteria and indictors identified
• The design and implementation of the evaluation
• Who participates in the monitoring and analysis
• How the information is used.

PM&E is a systematic management tool designed to reveal the degree of effectiveness and
efficiency in the achievement of objectives according to the perspectives of various stakeholders.
The mission of the monitors is not to find fault, but to give advice and assistance so that the
project can fulfill its objectives efficiently and effectively.

=====================================================================

Citizen participation in monitoring the activities of local administrative organizations is an

important method for creating good local governance. Participation ensures four things:

• Responsiveness: objectives are met according to the priorities of all stakeholders;

• Transparency: clarity and openness regarding activities and relationships;
• Accountability: actors are held responsible for their contributions by other
constituents;
• Equity: inputs and benefits reach the various stakeholders fairly.

Monitoring and evaluation offer not only tools for effective and efficient project implementation,
but also a learning process for all concerned. Notably, all interested parties and civil society can
play a role themselves and take part in making decisions and planning policies for development
and problem solving.

Evaluation can be conducted at several levels. For instance, the evaluation of a project designed
to change unfavorable citizen behavior can be done at three levels, as follows:

2) Primary evaluation or phase 1 is to assess the output or immediate outcome. It can be

done immediately after the training, by measuring and comparing the trainees’

5
 knowledge before and after the training.

3) Secondary evaluation or phase 2 is to assess project effectiveness. Output is compared

with objectives by following the trainees’ behavior to assess whether and how they
change their behavior after a certain period.

4) Tertiary evaluation or phase 3 is to assess the outcome and impact, by assessing how such
behavioral changes affect society.

Indicators, Outputs, Outcomes And Impact

Four key terms tell a monitoring and evaluation group the effects of a project: indicator, output,
outcome, and impact.

Indicator: a variable to measure the progress of activities or changes resulting from the project
activity, which assesses whether a change is in the desired direction, and whether the objective
will be achieved. A good indicator reflects both quantitative and qualitative changes.

Quantitative indicators can be displayed as a ‘single figure’ (such as the amount of produce in a
given year), or a ‘cumulative figure’ (such as the produce over several years combined together.)
This figure can be in the form of percentages and proportions.

Qualitative indicators measure either a behavior or an indirect representation. These indicators

often require open-ended responses. For instance, qualitative indicators may:
• Measure behavior in food consumption from weight and height gained
• Measure the health situation by the availability of clean drinking water and toilets in a village
• Measure poverty by the condition of houses and construction materials

Output: the product or service produced in order to achieve a target.

For instance, to achieve a target to enable 75% of women to understand laws and the
Constitution, the outputs include training documents, materials, trainers, workshops, and so on.

Outcome: a result after the use of outputs.

For instance:
• The availability of clean drinking water in the village results in an outcome of a lower
incidence of diarrhea.
• School construction results in an outcome of higher enrollment.
• Provision of loans without collateral results in an outcome of more capital for women to
invest.

Impact: the final result of the outcome.

The impact might be seen by the end of the project or several years afterwards. It may change
the living condition of project participants. For instance,
• Lower diarrhea results in a lower infant mortality rate, and healthier children.
• Higher enrollment in a school might have the impact of increased literacy.
• Women’s having more capital to invest results in this impact: higher income and better

6
 nutrition for children.

=====================================================================

Exercise:

Based on the local project or policy you proposed to monitor and evaluate, identify one of each
of the following:

Indicator: _____________________________________________________________________

Output: ______________________________________________________________________

Outcome: _____________________________________________________________________

Impact: ______________________________________________________________________

====================================================================

Monitoring and Reporting

Principles or Characteristics of Participatory Evaluation

The Role of Gender

The Value-Added of PM&E

SECTION III: QUANTITATIVE, QUALITATIVE AND PARTICIPATORY METHODS

• ‘Quantitative’ methods often refer to random sampling for survey research, structured
individual interviews for data collection and subsequent statistical analyses which allow
the results to be considered representative, comparable and ‘generalizable’ to a wider
population.

• ‘Qualitative’ methods often refer to a spectrum of data collection and analysis techniques
where sampling is not necessarily random and more in-depth, unstructured, open-ended
efforts are incorporated. This allows for in–depth analysis of social, political and
economic issues. 1

1
Krishna and Shrader. 2000

7
Given the imprecise divide between quantitative and qualitative approaches, it may be more
useful to distinguish between contextual versus noncontextual methods of data collection and
analysis (Hentschel 1999). However, neither of these approaches are guaranteed to be
participatory. And both methods can be carried out in participatory ways. Participation requires
that various stakeholders have a voice and some leverage in deciding how a policy, project or
evaluation is designed, implemented, monitored, and assessed.

SECTION IV: PARTICIPATORY MONITORING & EVALUATION PROCESS CYCLE

Participatory Monitoring and Evaluation Process Cycle 2

a. Conduct a readiness assessment;
b. Decide who participates, establish principles for working together;
c. Establish goals;
d. Develop indicators;
e. Gather info (quantitative and qualitative);
f. Construct baseline;
g. Set realistic targets;
h. Gather info again (quantitative and qualitative);
i. Analyze results including all stakeholders;
j. Share info/results with all relevant parties;
k. Take action – use the findings!;
l. Start over (making any needed modifications in process…)

SECTION V: THREE CASE STUDIES OF PARTICIPATORY MONITORING AND

EVALUATION

Case Study 1: Haiti3

This case study is divided into three parts:

1. Introduction and the case of the CCIC in Haiti
2. A few myths about PM&E
3. Activities for learning more about PM&E

1. Introduction and the Case of the CCIC in Haiti

Evaluations classically exist to ensure accountability, so that funds are spent as they are intended
and so that we learn from our mistakes and build on successes. The two underlying assumptions
of most evaluations are that the people judging a project are a neutral and independent third
party; and that an evaluation without these neutral, independent evaluations has no claims to

2
References: Gaventa IDS, Kusek and Rist, Narayan, and PRSP Sourcebook):
3
The information in this case study is drawn from Françoise Coupal and Marie Simoneau, “Participatory
Evaluation: A Case Study of CCIC Humanitarian Fund Projects in Haiti” (http://www.mosaic-net-
intl.ca/pehaiti15.html).

8
“objective” effectiveness. This approach is mainly quantitative because it chiefly measures
inputs and outputs, and it seeks quasi-scientific evidence for judging project success.

Many observers have called for a new approach to evaluation, one that is qualitative and
participatory. The advantage of participatory evaluation can be summarized by this diagram,4
which illustrates its added benefits:

Accountability to
Build capacity of Stakeholders,
stakeholders to Managers, and
reflect, analyze, and Donors
take action.

Various functions Development

of Participatory of lessons
Evaluation learned which
Development of help program
lessons learned staff improve
for corrective
action by Improve rapport
stakeholders between
government and
civil society.

4
The diagram is adapted from (and slightly changed) Coupal and Simoneau, 3. Adapted from Who Are the
Question-Makers? A Participatory Evaluation Handbook (New York: OESP, UNDP, 1997).

9
Recently, the Canadian Council for International Cooperation (CCIC), an aid organization, took
the principles of participatory M&E and tested them, with notable success, in a somewhat
volatile and unstable setting: Haiti, in the wake of a political coup.

Haiti is the poorest country in the Western hemisphere. In 1990, Haiti held its first-ever
democratic elections, and elected Father Jean-Bertrand Aristide as President. After seven
months there was a military coup, ousting President Aristide and stirring a wave of violence and
unrest. The United States invaded Haiti in 1994 and President Aristide was restored to power.
As Haiti began to rebuild its fragile civil society, NGOs and grassroots organizations began to
implement development programs. The CCIC, a coalition of volunteer groups, funded thirty-six
Haitian development projects, ranging from agriculture, to nutrition, to credit, to
communications.

As part of its broad program, the CCIC made a point of using participatory monitoring and
evaluation. Its intent was to go beyond the simple accountability function. Accountability only
documents inputs and outputs instead of impact. Instead, the CCIC aimed to build capacity
among Haitian and Canadian NGOs by training local participants in participatory methods.

The CCIC trained approximately thirty personnel from NGOs in Haiti in participatory methods
of monitoring and evaluation. Some of these personnel were reluctant to participatory methods
at first. After conducting a four-day workshop and an evaluation process, the participants
concluded that the success of any development project depended on several factors:

• It must respond to real priorities in the community

• Training should create and build capacity of organizations and groups
• Group dynamics should enhance feelings of belonging and of honesty
• Community participation from the onset is important
• Good governance and leadership of the project is key
• Good project structure and follow-up is essential
• Strategies must be flexible
• Sufficient resources must be available

The program also found that project aims can be limited or blocked by the following factors:

• The centralization of power, or inadequate sharing of information on a project

• Poor management and lack of transparency
• Poor leadership and paternalistic attitudes
• Egotism
• A lack of community participation and communication between project managers and
project beneficiaries
• Poor monitoring and follow-up

2. A few myths about participatory evaluation and monitoring

10
The CCIC found that its NGO personnel participants brought with them four reservations about
PM&E, which their experience in the workshop significantly changed:

1. Participatory evaluations can only be done with projects designed in a participatory

manner.
2. Participation takes time and is costly.
3. Project participants cannot “objectively” evaluate projects.
4. Evaluations require “experts.”

The first of these, that participatory evaluation only works in a project devised with participation
as its basic principle, failed to hold water. Though many of the CCIC projects were designed
traditionally, to respond to donor priorities, the inclusion of a participatory M&E approach “did
not disrupt the status quo.”5 In fact participatory approaches to M&E allowed NGO personnel to
appreciate its effectiveness in all aspects of project design, and they became better skilled in its
methods.

Time and cost objections were also reversed when the participatory M&E was revealed to (1)
cost the same as traditional evaluation; (2) take the same amount of time; and (3) purchase, for
its time and money investment, a greater impact than a traditional evaluation. Facilitators
declared after the participatory M&E training that “it really changed the way that [they] saw
development.”

Finally, the concerns about objectivity and expertise were also found to be unsupported;
participatory M&E facilitators found that a core of committed and skilled practitioners can train
participants to evaluate projects fairly and effectively.

Field Insight: Lesotho

Community people know well the relative wealth and poverty of other community members. In
Lesotho, a way was found to put community knowledge to use without resorting to
socioeconomic surveys.

Based on information from earlier studies, a set of thirty small picture cards was drawn,
signifying indicators of local wealth and poverty, such as maize porridge, rice, cattle, chickens,
and a nice house. A few blank cards were provided so that any missing indicators could be
added.

Village women sorted the pictures into three categories of items owned by the rich, average,
and poor people. They then picked three indicators for each category of wealth. Feeling no
need for secrecy, the women took only a few minutes to rate themselves and neighboring
families in the appropriate categories. In total agreement, they then recategorized the cards
cased on the gender of the head of household, thus identifying wealth differences with
gender.
Source: Deepa Narayan, Participatory Evaluation, (World Bank Working Paper, No. 207), 64.

5
Coupal and Simoneau, 15.

11
3. Activities.

Activity 1. What project do you want to monitor and evaluate? What knowledge or expertise of
this project do you bring to the table? What knowledge and expertise would you like to see
involved in monitoring and evaluation, and who else might you engage in your M&E process to
add this?

Activity 2.
What are three words that come to mind when you think of evaluation? List them here.

Here is a list of evaluation characteristics. Mark “T” next to the ones you consider typical of
traditional evaluation, and “P” next to the ones you consider participatory.

_____Conducted by experts
_____Empowers local people to take action
_____Focuses on scientific objectivity
_____Open results
_____Frequent small evaluations
_____Conducted by community members
_____Results have limited access
_____Indicators set by inputs and outputs
_____Procedures simple and adapted to local culture
_____Conducted to determine accountability
_____Takes place at the end of a project
_____Procedures are uniform and complex

Source: Narayan, 12.

12
Case Study 2: Porto Alegre, Brazil6

This lesson is divided into two parts:

1. Introduction and the case of Porto Alegre
2. Activities to become acquainted with participatory monitoring

1. Introduction and the case of Porto Alegre

Setting policy on a broad scale, such as deciding the budget of a region, is a complicated and
lengthy process; participatory practices and approaches prevent a local administration from
designing a policy, at great effort, which may ultimately fail because it does not reflect or
respond to local priorities.

Citizen participation in monitoring and evaluating the activities of local

administrative organizations helps to ensure four things:

Transparency: the clarity and accessibility of local administration

Accountability: tasks are done and money is spent as planned
Responsiveness: objectives are met
Equity: benefits reach the locality fairly

The example of Porto Alegre suggests that participatory monitoring not only relieved local
communities from some of the severe effects of the macroeconomic crisis experienced during
Brazil’s process of decentralization in 1988, but also that it improved local conditions.

Porto Alegre implemented participatory budgeting in 1988, during the first challenging year of
the new decentralized administration in Brazil. One of the best examples in terms of
participatory budgeting, Porto Alegre’s system is an instrument that combines direct democracy
with the empowerment and promotion of parliamentary representation.7 Now over ten years old,
participatory budgeting consists of a series of open meetings held March through June by the
city, the community, and by associations and groups such as trade unions. The purpose of the
small-scale meetings is to elect representatives for regional meetings, and to set the budgetary
agenda to be promoted by the municipality at the regional meeting. After local meetings are
completed in all the regions, representatives meet as a Council to set the budget.

Porto Alegre reports the following successes since the inception of participatory budgeting:

6
The information in this case study is drawn from Zander Navarro, “Participation, Democratizing Practices, and the
Formation of a Modern Polity: The Case of ‘Participatory Budgeting’ in Porto Alegre, Brazil (198901998)”
(Development 41: 3 [Sept ‘98]: 68-71).
7
Participatory Budget, Municipal City Hall of Porto Alegre (Municipal Department of Culture, Coordination of
Social Communications, 1995).

13
 • Considerable reduction in widespread corrupt behavior and administrative
malpractice. Open meetings mean open books. By holding open meetings and making
the budget every citizen’s business, the processes of allocation and distribution are less
susceptible to bribery, graft, or other “backstage” arrangements between civil servants
and private interests. Reduced corruption also increases public confidence in local
government, which encourages even greater levels of civic engagement and participation.

• Considerable reduction in clientelism (or patronage). Confronting a better-informed

and more demanding citizenry, well versed in the new “rules of the game,” city
councilors and would-be candidates notably improved in unfair practices of awarding
patronage.

• More equitable, “progressive” distribution of taxes and resources. Property tax, the
main source of revenue in the region, was reassigned on a progressive scale. In effect,
wealthier neighborhoods are now taxed at a higher rate than poorer neighborhoods.
Correspondingly, poor neighborhoods are allocated a greater share of resources, in
keeping with their greater neediness.

The participatory budget practice in Porto Alegre offers a way to redistribute resources to the
poor, to allocate funds more fairly overall, and to reclaim political relations in a formerly corrupt
system.

The process, in brief, is this:

• The region is divided into 16 areas for a more representative system

• The Participatory Budget Council includes two representatives from each area
• Municipal meetings, intermediate meetings, and thematic meetings, organized by the
community, are held to discuss and decide public works decisions and budget priorities,
and other topics raised by community leadership
• Local delegates attend the Participatory Budget Council to discuss and decide the
priorities of the region, i.e. where the municipality should invest
• The Council delegates attend the Forum of Delegates for the region, to discuss and decide
public works, to inform the community about their actions, and to receive new requests
• The Council establishes the next year’s budget and it is approved by elected city
councillors

Citizens in the region were able to identify and prioritize their needs; co-manage municipal
resources; control and evaluate the mayor’s commitments; and create opportunities for
participation, accountability, and transparency.8

8
Participatory Budget, Porto Alegre.

14
2. Activities

Activity 1. What is the quality of public services in your area? Give each one a letter grade
from A to F.

___Roads: Paved, no potholes; traffic lines recently painted; trees plentiful and in good
condition
___Buildings: No abandoned buildings; building and other permits are issued without difficulty
___Lighting: Working street lights
___Sanitation: Streets cleared of litter; garbage collected regularly; recycling pick-up available
___Motor vehicles: No abandoned cars; parking regulations enforced; traffic lights work
___Other: Public telephones, mailboxes, newspaper machines available and in good working
condition
___Safety: police and fire departments come when they’re called; stray dogs collected regularly;
crime not severe
___Resources: food and other stores are within reasonable distance
___Amenities: libraries, cinemas, museums, or parks within reasonable distance

Which public service received the lowest grade?_______________________

Which public service received the highest grade?______________________

What specific things, in your opinion, caused the worst public service to perform poorly? What
caused the best public service to perform well?

Using the M&E Learning Wheel, how would you describe the “baseline” of one of these two
public services?

15
Activity 2. Do you think a participatory budget process could work in your community? Why
or why not?

Consult the M&E Learning Wheel. Which steps of the Wheel would present the greatest
challenge for doing participatory budgeting? Which steps would be the most useful?

16
 Field Insight: Nigeria

In Nigeria, a workshop was held to evaluate the budgetary process for a large externally-financed rural
water and sanitation project. In attendance were the project coordinator, technicians, and local
government management and extension staff. In the informal atmosphere, the participants were each
asked to list who made decisions about resources for different project activities and how those decisions
were reached.

The subsequent discussion revealed that while the project was trying to support greater autonomy on the
part of local governments in the planning and implementation of rural water and sanitation systems, in
effect the main decisions regarding disbursement of funds were made by the project coordinator. This
was done to ensure accountability and to avoid mismanagement of funds, but it also had the unintended
effect of reducing the sense of ownership and control over the project by the local government
administration.

Similarly, while the project strove to encourage community participation and self-determination, the
schedule of project activities was also beyond the control of community committees. Although this
situation was a disincentive for more committed involvement in the project by both government and
communities, the need to account for externally provided public funds and the pressure placed by donors
to achieve construction targets in a timely fashion made it difficult for project managers to contemplate an
alternative approach.

Source: Narayan, 57.

17
Case Study 3: Sri Lanka9

This lesson is divided into two parts:

1. Introduction and the case of Sri Lanka

2. Participatory planning and evaluation activities

1. Introduction and the case of Sri Lanka

Rural development includes projects such as equipment repair and replacement, the expansion of
credit services to farmers, and the introduction of subsidies for certain crops. It can also include
investment in local health care, education, water supply, transportation, and electrification.

To make rural development sustainable, a key component of project design is participation by

beneficiaries. Those on the receiving end of rural development see the greatest benefit when a
project is planned around their contributions and involvement and they are participants in the
whole process. Such participation develops a sense of ownership among stakeholders, who take
a greater interest in the continuation and maintenance of the assets and processes that the project
seeks to improve.

The example of Sri Lanka offers a lesson in how participatory monitoring and evaluation--which
is typically added onto development projects at their conclusion--could have significantly
transformed project results and impacts if it had been integrated into the project from the start.
This case study is about a project that had limited success, according to its traditionally-set goals,
indicators, and methods. Importantly, during participatory evaluation, project managers learned
that participatory methods would have improved the and outcome of the entire project. The
disappointment of the rural development aims, alongside the successes of participatory M&E,
offer strong evidence of the potential this approach to project design, especially in setting
indicators.

In Sri Lanka in the 1970s, the World Bank funded two major rural development projects (the
Kurunegala Rural Development Project, or KRDP; and the Second Rural Development Project,
or SRDP). At that time, rural development projects were generally not designed with
participatory approaches in mind. The two Sri Lankan projects enjoyed mixed successes, and the
evaluation of these projects suggested that the difficulty arose from the use of a “blueprint,” or
top-down, model for project design. The project did not necessarily offer useful tools, nor did
beneficiaries necessarily want what was offered. Local beneficiaries (farmers, community
organizations, households, government workers) were consulted only during the evaluation
process, when the defects of the projects became clear. (The rates of return on agricultural
products were optimistically estimated to be 24 to 34 percent, but were actually only 10 percent.)

9
The information in this case study is drawn from John English, “Sri Lanka: Kurunegala Rural Development Project
and Second Rural Development Project”
(http://wbln0018.worldbank.org/oed/inter.nsf/htmlmedia/publications.html).

18
The Sri Lankan experience provided valuable lessons about project design, which have since
guided rural development in that country. Here are some of the findings of the Sri Lankan
KRDP and SRDP:

• Project planners did not consider how to sustain benefits after projects closed, and so the
assets created (irrigation systems, roads) were not regularly maintained. Beneficiaries,
who were left out of the planning process, did not take on a sense of ownership of these
assets.

• User groups, which are crucial to operation and maintenance in rural development, were
not part of the project. Farmers saw user groups promoted by the government as outside
impositions, and did not participate. User groups succeed when they are initiated by the
users themselves.

• Management practices in irrigation, again recommended “from outside,” were not

followed by farmers, who perceived a limited benefit in doing so.

• When farmers were involved in all stages of irrigation work, structures were properly
maintained, and conflicts with contractors were minimal. When they were not, quality
control suffered.

• The project had one objective, and unintentionally provided another. The latter turned
out to be far more useful to the community. Although the project sought to diversify
crops in the district and increase agricultural production, farmers reported that the
greatest benefit of the project was the construction of roads, replacing footpaths. Travel,
job opportunities, and access to markets, medical facilities, and schools significantly
benefited the community, and raised the average household income some 30 percent—a
far more significant impact than the income increases from improved farming output (10
percent).

The Sri Lankan project evaluators drew two principles from these data:

• Participation by stakeholders at all stages, especially monitoring and evaluation. It

is essential that M&E be conducted using truly significant variables, rather than those
imposed from outside. Variation between the intended outcome and actual outcome (the
modest increase in agricultural output, versus the significant impact of building roads)
demonstrates the importance of relying on local values for M&E.

• Flexibility of design. Differing conditions in a variety of rural areas call for a process-
oriented, learn-by-doing approach, following one- to two-year rolling plans. In this
approach, participatory monitoring is a management tool, to keep the project in line, and
prevent it from wasting resources on assets and procedures that are not useful to the
community.

19
 Field Insight: Indonesia

Lack of information, confidence, trust, and clarity about who owns facilities can prevent corrective action
from being taken. In a rural water project in Indonesia, a pump remained broken for six months because
the government officials in the provincial capital assumed that the well had run dry. When a participatory
evaluation team came to the village, Pak Kake, an elderly man whose love of tinkering was well known,
was asked why the well was dry. He said, “I have measured with a rope and the water level has fallen to
52 meters, whereas the cylinder hangs at 45 meters. We have not repaired the pump because we are
waiting for the chief administrator to give us permission.”

When this information was conveyed to the senior government officials in the provincial capital, a
technical team visited the site, verified Pak Kake’s observations, clarified ownership rights, and worked
with the water group to fix the pump. Within hours, the water was flowing again.

Source: Narayan, 62.

2. Activity: A tool for participatory planning

The Participatory Monitoring and Evaluation Learning Wheel, described

earlier in this Module, includes these important steps: Establish goals;
Develop indicators; and Take action. When a group establishes shared
goals for M&E, it knows what it is measuring. When a group develops
shared indicators for M&E, it knows what success looks like. When a
group takes thoughtful and realistic action as a result of M&E, it addresses
problems more effectively.

This section offers an activity for learning and enhancing the common
thread of these three steps in the PM&E Wheel: determining and setting
priorities.

Given that all activities have their own costs including money, time, and
labor, the implementation of one project or goal means that there are fewer
resources left for others. Finite resources mean that it is necessary to

20
prioritize your goals and your indicators.

Important factors in setting priorities include:

• the urgency and severity of the problem

• the possibility the problem could be solved with existing resources
• the number and types of stakeholders who will be either positively or
negatively affected by the project or activity
• the community’s participation
• the budget (amount and source)
• the sustainability of the development which would result from the
project/activity

The aim of this section is to provide trainees with basic knowledge and
skills in prioritizing, with emphasis on the participation of the community
and stakeholders.

The following table can be used to compare the merits of various goals of
the group, and to prioritize them:

Goals Beneficiari Urgency Resource Impact of Ranking

(fill in es of goal s required not of this
what your (positive pursuing goal
group is and this goal
prioritizing negative)
)

21
(The table can be adapted to take action, by inserting “actions” into the first
column.)

22
A table for developing indicators might look like this:

Indicators Who can How much Resource Is it Ranking

and can't participati s required quantitive of this
use this on does or indicator
indicator? indicator qualitative
call for? ?

23
24
Activity 7.4: Prioritization of a project/activity
Take a copy of today’s newspaper. Make a list of some of the projects in
your community that are reported there. In a small group, choose two
projects from that list. Use the chart above to evaluate those two projects.
What did each small group discover about its chosen projects? Report
your findings to the class.

25
SECTION VI: EXERCISE -- ACTION PLANNING

Here you have some time to work through the stages of the Participatory Monitoring and
Evaluation Process Cycle in the context of a project or policy which affects your community. Jot
down some ideas for steps you could take to complete each phase of the process cycle. Consider
ideas you have drawn from case studies and adapt them to your own needs/context.

• Conduct a readiness assessment

• Decide who participates; establish principles for working together
• Establish goals
• Develop indicators
• Gather info (quantitative and qualitative)
• Construct baseline
• Set realistic targets
• Gather info again (quantitative and qualitative)
• Analyze results including all stakeholders
• Share info/results with all relevant parties
• Take action – use the findings!
• Start over (making any needed modifications in process…)

SECTION VII: ADDITIONAL RESOURCES

• On-Line
• Other

26
APPENDIX A: MONITORING MECHANISMS

• Report Cards (Ukraine, Philippines)

• E-Gov (India, Brazil)
• Household Surveys (Guatemala, Tanzania)
• Citizen Dialogues/For a (Brazil)

27
• REFERENCES:

1. Michael Bamberger
2. Soniya Carvalho, Indicators for monitoring poverty reduction World Bank Discussion
Paper 254, Washington D.C. 1994.
3. Guidance note on indicators for measuring and assessing primary stakeholder
participation. ODA/DfID. July 1995
http://www.dfid.gov.uk/public/what/advisory/group7/pdf/sddstak2.pdf
4. David Fetterman, Stanford University. Empowerment Evaluation: Collaboration, Action
Research, and a Case Example. The Action Evaluation Research Institute. Go to:
www.aepro.org/inprint/conference/fetterman.html
5. John Gaventa, IDS.
6. Jesko Hentschel
7. Sam Kaner et al. 1996. Facilitator’s Guide to Participatory Decision-Making. New
Society Publishers.
8. Kusek and Rist 2001. Course for Establishing M & E Systems in Government. World
Bank.
9. Mosaic Intenrational Training in PM&E Ottawa, Canada
10. Deepa Narayan. 19993. Participatory Evaluation: Tools for Managing Change in Water
and Sanitation. World Bank
11. Deepa Narayan, 1995. The Contribution of People’s Participation: Evidence from 121
Rural Water Supply Projects, World Bank.
12. Lyra Srinavasan et al. 1994. Community Participation Strategies and Tools: A Trainers’
Manual for the Rural Water Supply and Sanitation Sector in Pakistan. UNDP and Govt
of Pakistan.
13. Lyra Srinavasan. 1990. Tools for Community Participation: A Manual for Training
Trainers in Participatory Techniques. PROWWESS/UNDP Technical Series.
14. PRSP Sourcebook Chapter on Monitoring and Evaluation. 2001.
http://www.worldbank.org/poverty/strategies/chapters/monitoring/moneval.htm
15. World Bank Participatory Monitoring and Evaluation Website:
http://www.worldbank.org/participation/partme.htm
16. Evaluation of Programmes Promoting Participatory Development and Good Governance:
Synthesis Report. DAC http://www.oecd.org/dac/htm/pubs/p-pdggev.htm
17. Evaluation Paradigms: four generations of evaluation (short review of Guba and
Lincoln's Fourth Generation Evaluation charting the evolution of evaluation thinking)
http://www2.deakin.edu.au/dcad/FTL/evaluation/paradigms/paradigms.htm
18. Guidance note on Increasing the Participation of the Poor in the Assessment of the
Impact of Development Interventions. DfID/Actionaid.
http://www.mande.co.uk/guidance.htm
19. Participatory Evaluation in programmes involving governance decentralization. UNDP.
20. Participatory Monitoring, Evaluation and Reporting: Online Manual, PACT
21. Larry Salmen

28