MONITORING AND

EVALUATION
Discussion points for the 2nd Global Forum on
Incubation in Hyderabad, India
6 – 10 November 2006

Steve Giddings
(sgiddings@ananzi.co.za)
Overview

1. What is “monitoring and

evaluation” (M&E)?
2. Some common M&E methods
3. Considerations for incubator
M&E
 What is M&E?
Monitoring: a continuing function that uses systematic
collection of data on specified indicators to provide
management with indications of the extent of progress and
achievement of objectives and progress in the use of
allocated funds

Evaluation: the process of determining the worth or

significance of a program to determine the relevance of
objectives, the efficacy of design and implementation, the
efficiency or resource use, and the sustainability of results.

M&E are synergistic… monitoring is a necessary, but

not sufficient, input into evaluation
M&E

Tools, methods and approaches for M&E

(from Monitoring and evaluation: some tools, methods

and approaches. The World Bank, 2004)
Performance indicators

 Measures of inputs, processes, outputs, outcomes,

and impacts
 When supported with sound data, indicators can

assist track progress, demonstrate results, and take

corrective action to improve service delivery
 Important to include key stakeholders in defining

indicators - they are more likely to understand and

then use indicators for management decision-making
Performance indicators
USED FOR:
■ Establishing performance targets and then evaluating
progress
■ Indicating whether an in-depth evaluation or review is needed

ADVANTAGES:
■ Effective means to measure progress toward objectives

DISADVANTAGES:
■ Poorly defined indicators are not good measures of success
■ Tendency to set too many indicators, or those without
accessible data sources - costly, impractical… and then
underutilized
■ Often a trade-off between selecting the best indicators and
accepting those which can be measured using existing data
The Logical Framework
 Helps to clarify objectives of any project,
program, or policy
 Aids in the identification of the expected
causal inputs, processes, outputs outcomes,
and impact
 Leads to the identification of performance
indicators as well as risks which might impede
the attainment of the objectives
 Useful for engaging partners in clarifying
objectives and designing activities
 Useful tool to review progress and take
corrective action
The Logical Framework
USED FOR:
 Improving quality of programme designs by requiring the
specification of clear objectives, performance indicators & risk
assessment
 The preparation of detailed operational plans
 Objective basis for activity review, monitoring, and evaluation

ADVANTAGES:
 Sets out key questions and analyses of assumptions and risks.

■ Engages stakeholders in the planning and monitoring process

■ A management tool to guide implementation, monitoring and
evaluation

DISADVANTAGES:
 Can restrict creativity

■ Tendency to be static
■ Training and follow-up often needed
Theory-based evaluation
 Similar to Log Frame but more detailed in
understanding programme logic
 Seeks to identify causal or determining factors

seen as important for success and then what

should be monitored
 Ultimately leads to the determination of critical

success factors (CSFs)

 Evaluation of CSFs used to inform likelihood

of programme success
Theory-based evaluation
USED FOR:
■ Mapping the design of complex activities
■ Improving planning and management

ADVANTAGES:
■ Early feedback about what is or is not working, and why
■ Allows early correction of problems as soon as they emerge
■ Assists identification of unintended side-effects of the program
■ Helps in prioritizing which issues to investigate in greater depth
■ Provides basis to assess the likely impacts of programs

DISADVANTAGES:
■ Can easily become overly complex if the scale of activities is
large or if an exhaustive list of factors and assumptions is
assembled
■ Stakeholders might disagree about which determining factors
they judge important, which can be time-consuming to address
Formal surveys
 Formal surveys can be used to collect
standardized information from a carefully selected
sample of people or households
 Surveys often collect comparable information

for a relatively large number of people in particular

target groups
Formal surveys
USED FOR:
Providing baseline data against which the performance of the

strategy, program, or project can be compared

■ Comparing actual conditions with the targets
■ Providing key inputs into a formal evaluation of the impact of a
programme or project

ADVANTAGES:
■ Findings from the sample can be applied to wider target group
■ Quantitative estimates for the size and distribution of impacts

DISADVANTAGES:
■ Processing and analysis can be a bottleneck for larger surveys
■ Many kinds of information are difficult to obtain through formal
interviews
Rapid appraisals
 Rapid appraisal methods are quick, low-
cost ways to gather the views and feedback
of beneficiaries and other stakeholders, in
order to respond to decision-makers’ needs
for information
 Can include: key informative interview;

focus group discussion; direct observation;

mini survey etc.
Rapid appraisals
USED FOR:
 Providing rapid information for management decision-making,

especially at the project or program level

 Providing qualitative understanding of highly interactive social

situations, or people’s values, motivations, and reactions

 Providing context and interpretation for quantitative data

collected by more formal methods

ADVANTAGES:
 Can be conducted quickly and at low cost

 Provides flexibility to explore new ideas

DISADVANTAGES:
 Less valid, reliable, and credible than formal surveys

 Difficult to generalize from findings

Participatory methods

 Participatory methods provide active involvement in

decision-making for those with a stake in a project,
program, or strategy and generate a sense of
ownership in the M&E results and recommendations
 Can include: stakeholder analysis; beneficiary

assessment etc
Participatory methods
USED FOR:
 Identifying problems and trouble-shooting problems during

implementation
Providing knowledge and skills to empower people

ADVANTAGES:
 Examines relevant issues by involving key players in design

■ Establishes partnerships and local ownership of projects

■ Enhances local learning, management capacity, and skills
■ Provides timely, reliable information for decision-making

DISADVANTAGES:
 Sometimes regarded as less objective (domination and misuse)

 Time-consuming if key stakeholders are involved in a meaningful

way
Cost benefit evaluation

 Cost-benefit and cost-effectiveness analyses are

tools for assessing whether or not the costs of an
activity can be justified by the outcomes and impacts
 Cost-benefit analysis measures both inputs and

outputs in monetary terms

 Cost-effectiveness analysis estimates inputs in

monetary terms and outcomes in non-monetary

quantitative terms
 Cost benefit evaluation
USED FOR:
 Making decisions about the most efficient allocation of resources

 Identifying projects that offer the highest ROI

ADVANTAGES:
 Good quality approach for estimating the efficiency

 Makes explicit the economic assumptions that might otherwise

remain implicit or overlooked at the design stage

 May convince policy-makers and funders of benefits

DISADVANTAGES:
 Fairly technical: requires skills

 Data for cost-benefit calculations may not be available, and

projected results may be highly dependent on assumptions made

 Results must be interpreted with care (benefits difficult to quantify)
Impact assessment

 The systematic identification of the effects –

positive or negative, intended or not – caused
by a program or project
 Impact evaluations can range from large scale

sample surveys to small-scale rapid assessment

 Impact assessment
USED FOR:
 Measuring outcomes and impacts of an activity and distinguishing

these from the influence of other, external factors

 Helping to clarify whether costs for an activity are justified

 Drawing lessons for improving the design and management of

future activities

ADVANTAGES:
 Provides estimates of the magnitude of outcomes and impacts

 Provides answers to: what extent are we making a difference?

What are the results on the ground? How can we do better?

 Rigor gives managers added confidence in decision-making

DISADVANTAGES:
 Can be expensive and time-consuming which reduces utility when

decision-makers need information quickly

Incubator M&E
 Many ways of doing M&E…is there any one
“correct” method ?
 Central question for M&E is “what do we want

to achieve and how do we best measure it?”

 Need to ensure sound M&E in place to

measure progress AND convince stakeholders

of value (especially “E”)
 Approach must match pragmatism with need

 Experience shows hybridization needed to

overcome:
 Situational issues
 Resources (skills, time, money…)

 Ability to find what can “really” be measured