Вы находитесь на странице: 1из 19

fiinovation

Opportunity Solutions, Simplied.

MONITORING &
EVALUATION
FRAMEWORK

Index
1 Monitoring and Evaluation & its importance
02 Monitoring and Evaluation framework in the Project Program Cycle
03 Monitoring and the inclusion of RBM
04 Types of Monitoring
06 Tailor-Made Evaluation Studies
08 Evaluation Questions & RBM
09 Six key steps for M&E Planning
10 Minimizing Bias and Errors
11 CSR in India - Industry Overview
12 Fiinovation Expertise
13 Some of our Initiatives
14 Organization Brief
15 Awards & Recognition

Monitoring & Evaluation

01

What Does Monitoring and Evaluation Mean?


Monitoring

Evaluation

Evaluation is about making judgements about the


Monitoring is a routine and systematic
value of any component part of an organisation
collection of information against a plan. The
or its products, services or benets, or
information might be about activities, products
about the organisation as a whole.
or services, users, or about outside factors
affecting the organisation or project.

Why is Monitoring and Evaluation Important?


Organisations use monitoring and evaluation for two key purposes:
To learn about their own activities and results, and to support internal planning and development
To be accountable to their stakeholders.
Organisations need evidence of their efciency and effectiveness for funders, commissioners and
investors. They also need to communicate achievements to a wider public. Charities are now
specically required to report the achievements of the charity over the year. However, the role of
monitoring and evaluation is providing learning and improving the organisation is of equal importance.
The following eight questions shall help in creating a better understanding on the need of
monitoring and evaluation:

Are monitoring data regularly used during


M&E meetings or other events?

1
Does your project
have an M&E plan?

7
Are staff and partners trained
on data entry and analysis?

2
Does your project
have an M&E binder?

6
Have you created a
monitoring database?

3
Have you developed
all monitoring forms?

4
Are staff and partners trained on
using the monitoring forms?

Have you conducted a quality


check of the monitoring data?

If your answer is No' to any of the questions above, the project team should work to complete the
step(s). This review tool has not been designed to be used in isolation and refers to the package of
practices being offered at Fiinovation, and this handbook is for further guidance at each review stage.

Monitoring & Evaluation

02

M&E Framework in the Project


Program Cycle
This is done to determine whether a
project/program is needed and, if so, to
inform its planning.
This informs ongoing programming.
Although this has been put in the
last section of the diagram but,
reporting, reflection and learning
should occur throughout the whole
project/program cycle.

This involves the operational


design of the project/program
and its objectives, indicators,
means of verification and
assumptions

Initial Needs
Assessment

Dissemination
and use of
lessons

Final
evaluation
Midterm
evaluation
and/or reviews
This occurs after project/program
completion to assess how well the
project/program achieved its
intended objectives and what
difference this has made.

These are important reflection events


to assess and inform ongoing
project/program implementation.

Log frame and


indicators

M&E planning

Baseline
assessment

This is the practical planning for


the project/program to monitor
and evaluate the log frame's
objectives and indicators.

This is the measurement of the


initial conditions (appropriate
indicators) before the start of a
project/program.

There is no magic 10-percent sampling rule. It is important to note that the sample size is not related to the size
of the popula on being sampled. A frequent mistake is to conduct surveys among 10 percent of a given
popula on; in fact, it is likely that 10 percent of the popula on is either too many or too few households. With
too many households, the survey is using excessive resources and me; with too few households, the sample
will not adequately represent the popula on.

Monitoring & Evaluation

03

Monitoring and the Inclusion of RBM


Result Based Management (RBM) is an approach to project/program management based on
clearly dened results, and the methodologies and tools to measure and achieve them.
RBM supports better performance and greater accountability by applying a clear, logical
framework to plan, manage and measure an intervention with a focus on the results you want to
achieve.
By identifying in advance the intended results of a project/program and how we can measure their
progress, we can better manage a project/program and determine whether a difference has
genuinely been made for the people concerned.
Monitoring and evaluation (M&E) is a critical part of RBM. It forms the basis for clear and
accurate reporting on the results achieved by an intervention (projector program).
In this way, information reporting is no longer a headache, but becomes an opportunity for critical
analysis and organizational learning, informing decision-making and impact assessment.

Monitoring Questions and RBM


Impact

Measuring changes at impact level requires a longer time frame, and is therefore dealt with
by evaluation and not monitoring.

Outcomes

Are the outputs leading to the achievement of the outcomes?


Is there anything happening that should lead management to modify the operations
implementation plan?

Output

How do beneciaries feel about the work/intervention?


What is causing delays or unexpected results?

Activities

Are activities leading to expected outputs?


Are activities being implemented on schedule and within the budget?

Inputs

Are nances, personnel and materials available on time and in the right quantities and
quality?

A mistake common for M&E systems is to rely solely on either observa on data or par cipant responses.
Observa on data alone does not provide an explana on of prac ces or behaviors and o en requires large
assump ons on the part of the M&E team. Focus group data (an example of par cipant responses) may not
capture important prac ces that par cipants do not see as relevant and may record instead what par cipants
think data collec on teams want to hear.

Monitoring & Evaluation

Types of Monitoring
1. Results Monitoring
It tracks effects and impacts.
This is where monitoring merges with evaluation to determine if the project/program is on target
towards its intended results (outputs, outcomes, impact).
It measures whether there may be any unintended impact (positive or negative).
For example, a psychosocial project may monitor that its community activities achieve the outputs
that contribute to community resilience and ability to recover from a disaster.

2. Process (Activity) Monitoring


It tracks the use of inputs and resources, the progress of activities and the delivery of outputs.
It examines how activities are delivered the efciency in time and resources.
It is often conducted in conjunction with compliance monitoring and feeds into the evaluation of
impact.
For example, a water and sanitation project may monitor that targeted households receive septic
systems according to schedule.

3. Compliance Monitoring
It ensures compliance with donor regulations and expected results, grant and contract
requirements, local governmental regulations and laws, and ethical standards.
For example, a shelter project may monitor that shelters adhere to agreed national and
international safety standards in construction.

4. Context (Situation) Monitoring


It tracks the setting in which the project/program operates, especially as it affects identied risks
and assumptions, but also any unexpected considerations that may arise.
It includes the eld as well as the larger political, institutional, funding, and policy context that
affect the project/program.
For example, a project in a conict-prone area may monitor potential ghting that could not only
affect projects success but, endanger the projects staff and volunteers.

04

Monitoring & Evaluation

5. Beneficiary Monitoring
It tracks beneciary perceptions of a project/program.
It includes beneciary satisfaction or complaints with the project/program, including their
participation, treatment, access to resources and their overall experience of change.
Sometimes referred to as Beneciary Contact Monitoring (BCM), it often includes a stakeholder
complaints and feedback mechanism.
It should take account of different population groups, as well as the perceptions of indirect
beneciaries (e.g. community members not directly receiving a good or service).
For example, a cash-for work program assisting community members after a natural disaster may
monitor how they feel about the selection of program participants, the payment of participants and
the contribution the program is making to the community (e.g. are these equitable?).

6. Financial Monitoring
It accounts for costs by input and activity within predened categories of expenditure.
It is often conducted in conjunction with compliance and process monitoring.
For example, a livelihood project implementing a series of micro-enterprises may monitor the
money awarded and repaid, and ensure implementation is according to the budget and time frame.

7. Organizational Monitoring
It tracks the sustainability, institutional development and capacity building in the project/program
and with its partners.
It is often done in conjunction with the monitoring processes of the larger, implementing
organization.
For example, a National Society's headquarters may use organizational monitoring to track
communication and collaboration in project implementation among its branches and chapters.

05

Monitoring & Evaluation

Tailor-made Evaluation Studies


Evaluations conducted during the life of the project should provide actionable recommendations to
improve the quality of the project in the time remaining. The final evaluation findings must be
incorporated into the strategy design for subsequent programming. The evaluation exercises can be
categorized under three broad heads as enumerated below.

According to evaluation timing

According to who conducts the


evaluation

According to evaluation
technicality or methodology

Formative evaluations occur


during project/program
implementation to improve
performance and assess
compliance.

Internal or self-evaluations are


conducted by those responsible
for implementing a
project/program. They can be
less expensive than external
evaluations and help build staff
capacity and ownership.
H o w e v e r, t h e y m a y l a c k
credibility with certain
stakeholders, such as donors,
as they are perceived as more
subjective (biased or one-sided).
These tend to be focused on
learning lessons rather than
demonstrating accountability.

Real-time evaluations (RTEs)


are undertaken during
p r o j e c t / p r o g r a m
implementation to provide
immediate feedback for
modifications to improve
ongoing implementation.
Emphasis is on immediate
lesson learning over impact
evaluation or accountability.

Summative evaluations occur at


the end of project/program
implementation to assess
effectiveness and impact.

External or independent
evaluations are conducted by
evaluator(s) outside of the
implementing team, lending it a
degree of objectivity and often
technical expertise. These tend
to focus on accountability.

Meta-evaluations are used to


assess the evaluation process
itself. Some key uses of metae v a l u a t i o n s i n c l u d e : t a ke
inventory of evaluations to
inform the selection of future
evaluations; combine evaluation
results; check compliance with
evaluation policy and good
practices; assess how well
evaluations are disseminated
and utilized for organizational
learning and change, etc.

Midterm evaluations are


formative in purpose and
occur midway through
implementation.

Participatory evaluations are


conducted with the
beneficiaries and other key
s t a ke h o ld e rs , a n d c a n b e
empowering, building their
c a p a c i t y, o w n e r s h i p a n d
support.

Thematic evaluations focus on


one theme, such as gender or
environment, typically across a
number of projects, programs
or the whole organization.

06

Monitoring & Evaluation

According to evaluation timing

According to who conducts the


evaluation

According to evaluation
technicality or methodology

Final evaluations are summative


in purpose and are conducted
(often externally) at the
completion of project/program
implementation to assess how
well the project/ program
achieved its intended
objectives.

Joint evaluations are conducted


collaboratively by more than one
implementing partner, and can
help build consensus at
different levels, credibility and
joint support.

Cluster/sector evaluations
focus on a set of related
activities, projects or
programs, typically across
sites and implemented by
multiple organizations (e.g.
National Societies, the United
Nations and NGOs).

Ex-post evaluations are


conducted some time after
implementation to assess longterm impact and sustainability.

Impact evaluations focus on the


effect of a project/ program,
rather than on its management
and delivery. Therefore, they
typically occur after
project/program completion
during a final evaluation or an expost evaluation. However,
impact may be measured during
p r o j e c t / p r o g r a m
implementation during longer
projects/programs and when
feasible. It highlights some of
the challenges in measuring
impact.

Proper management of an evaluation is a critical element for its success. There are multiple resources
to support evaluation management which identify the key criteria and standards that guide how we
plan, commission, conduct, report on and utilize evaluations. The framework mentioned in the
beginning is to be applied to all evaluation activities. It draws upon the best practices to ensure
accurate and reliable evaluations that are credible with stakeholders.

07

Monitoring & Evaluation

Evaluation Questions and RBM


Impact
Are the benets likely to be maintained for an extended period after assistance
ends? (sustainability)
What changes did the project bring about?

Outcomes
Were the operations objectives consistent with beneciaries needs and with
agency policies? (relevance)
Were there any unplanned or unintended changes.?

Output

Were the operations objective achieved? (effectiveness)


Did the outputs lead to intended outcome?

Activities

Were activities implemented on schedule and within budget? (efciency)


Were outputs delivered economically?

Inputs

Were stocks of items available on time and in the right quantities and quality?

08

Monitoring & Evaluation

Six Key Steps for M&E Planning


1.

Identify the purpose


and scope of the M&E
system

Review the project/program's operational design (RBM/LFA)


Identify key stakeholder informational needs and expectations
Identify any M&E requirements
Scope of major M&E events and functions

Plan for data collection


and management

Develop an M&E plan table


Assess the availability of secondary data
Determine the balance of quantitative and qualitative data
Triangulate data collection sources and methods
Determine sampling requirements
Prepare for any surveys
Prepare specic data collection methods/tools
Establish stakeholder complaints and feedback mechanisms
Establish project/program staff/volunteer review mechanisms
Plan for data management
Use an indicator tracking table (ITT)
Use a risk log (table)

2.

3.

Develop a data analysis plan, identifying the:


i.
Purpose of data analysis
ii.
Frequency of data analysis
iii. Responsibility for data analysis
iv. Process for data analysis.

Plan for data analysis

4.
Plan for information
reporting and utilization

Follow the key data analysis stages:


i.
Data preparation
ii.
Data analysis (ndings and conclusions)
iii. Data validation
iv. Data presentation
v.
Recommendations and action planning.
Anticipate and plan for reporting:
i.
Needs/audience
ii.
Frequency
iii. Formats
iv. People responsible.
Plan for information utilization:
i.
Information dissemination
ii.
Decision-making and planning

5.
Plan for M&E human
resources and capacity
building

6.
Prepare the M&E
budget

Assess the projects/program's human resources capacity for M&E


Determine the extent of local participation
Determine the extent of outside expertise
Dene roles and responsibilities at each level of the M&E system.
Plan to manage project/program team's M&E activities
Identify M&E capacity-building requirements and opportunities
Itemize M&E budget needs
i.Human Resource
ii.Capital Expenses
Incorporate M&E costs into the project/program budget
Review any donor budget requirements and contributions
Plan for cost contingency

09

Monitoring & Evaluation

Minimizing Bias and Errors


Minimizing bias helps to increase accuracy and precision. Accuracy means that the data measures
what it is intended to measure. For example, if you are trying to measure knowledge change following a
training session, you would not just measure how many people were trained but also include some type
of test of any knowledge change.
As much as we would like to eliminate bias and error in our measurements and information reporting,
no research is completely without bias. Nevertheless, there are precautions that can be taken, and
the first is to be familiar with the major types of bias we encounter in our work:

1. Selection Bias
Selection bias results from poor selection of the sample population to measure/study. Also called
design bias or sample error, it occurs when the people, place or time period measured is not
representative of the larger population or condition being studied. It is a very important concept to
understand because there is a tendency to study the most successful and/or convenient sites or
populations to reach (which are often the same). For example, if data collection is done during a
convenient time of the day, during the dry season or targets communities easily accessible near paved
roads, it may not accurately represent the conditions being studied for the whole population.

2. Measurement Bias
Measurement bias results from poor data measurement either due to a fault in the data
measurement instrument or the data collector. Sometimes the direct measurement may be done
incorrectly, or the attitudes of the interviewer may influence how questions are asked and responses
are recorded. For instance, household occupancy in a disaster response operation may be calculated
incorrectly, or survey questions may be written in a way that biases the response, e.g. Why do you like
this project? (rather than What do you think of this project?).

3. Processing Error
Processing error results from the poor management of data miscoded data, incorrect data entry,
incorrect computer programming and inadequate checking. This source of error is particularly
common with the entry of quantitative (statistical) data, for which specific practices and checks have
been developed.

4. Analytical Bias
Analytical bias results from the poor analysis of collected data. Different approaches to data analysis
generate varying results e.g. the statistical methods employed, or how the data is separated and
interpreted. A good practice to help reduce analytical bias is to carefully identify the rationale for the
data analysis methods.

10

Monitoring & Evaluation

11

CSR in India - Industry Overview


1

CSR Expenditure Among 10 Major Industries


HEALTH
RURAL DEVELOPMENT

22%

33%

2012-13
ENVIRONMENT

22%

EDUCATION

23%

Expense on CSR in the Private and Public Sector


95

RD Projects

65

35
18

Technical Incubators
PM Relief Fund

82
85

15
30

Sports
Forced Army Veterans

70
95

National Heritage

42
42

Environmental Sustainability
Reducing Inequalities

58
58
63

37

Education, Vocational Skills

Private Sector

75

25

Hunger, Poverty, Healthcare

78

22

Public Sector

Slum Area Developent

20

40

60

80
1
2

100

As per the annual reports of the top 200 rms for the year 2012-13
FICCI and Nextgen

Monitoring & Evaluation

12

Fiinovation Expertise
A vast bank of expertise gained over the years across a wide range of sectors and research
methodology. And also, the flexibility to serve a wide range of projects in the sectors of: Livelihood,
Education, Health and Environment.
Defined M&E standards that help improve program quality and positively impact the people we
serve. The standards followed at Fiinovation reflect the key characteristics of high-quality
programs and agency culture that promotes better learning and strengthens accountability to
stakeholders. These are critical elements of a high- performing, dynamic learning organization.
Triangulation of data is another key step practiced at Fiinovation, which involves collecting data
from multiple sources, sometimes using multiple tools, to identify and reduce bias. If triangulation
of qualitative data is not carried out, a risk of biasing or distorting the collected data exists,
resulting in incorrect or incomplete information which might result in skewed understanding of the
circumstances.
By collecting data from multiple sources or with multiple tools, we identify and address
discrepancies or inconsistencies in the data. Triangulation often leads to additional questions or
clarifications, which can be answered through follow-up interviews, discussions or exercises.

Confidentiality

We ensure secrecy for all the


research projects that we
carry out and maintain
confidentiality as per the code
of conduct.

ty

Core
Values

ali

Qu

Au
the
nti
cit

We guarantee authenticity,
reliability and credibility of the
data, facts and figures that we
gather and disseminate
through primary and secondary
research.
We ensure excellent quality of
the actionable reports which is
intended to provide absolute and
detailed insight into the
markets, products, competition
and the overall perspective at an
affordable price.

Monitoring & Evaluation

Some of our

Initiatives
The Monitoring and Evaluation Team of Fiinovation has been involved in helping its partners to
carry out eld research studies for monitoring, evaluating and assessing the impact of their
proposed and ongoing/complete projects.

A few of our successfully completed assignments in the current year are mentioned below:

Fiinovation partnered with IL&FS Skills to carry out an integrated on-site assessment study
for assessing the current status of the community in terms of their readiness and
acceptability for skill based trainings. The study was carried out to explore the avenues that
shall be available at the proposed site for Power Plant in Nana Layja (Mandvi), Gujarat. Based
on the research ndings of Fiinovation, the project aims to develop skill, employment and
entrepreneurial opportunities for the population of Mandvi Block in Kutch district with
respect to the Nana Lajya Power Plant being put up by IL&FS Group. Fiinovation gathered
adequate information through extensive research, both primary and secondary sources for
the purpose of the study. The Need Assessment of the project was carried out for 3 months
with assessment tools intrinsically focused on the need and acceptance of the community.
Fiinovation has also conducted a summative evaluation study of the skill training institute
(Cairn Enterprise Center - CEC) of Cairn India Limited (CIL) at Barmer (Rajasthan). The study
was carried out for the CSR project of CIL being implemented by IL&FS Skills with a view to
evaluate the program performance and its effectiveness by collecting information that can be
used for further evaluation. Fiinovation developed tailor-made evaluation parameters for
assessing the outcomes of the skill training intervention. The framework used empirical
evaluations aimed at elucidating and substantiating the linkages between the trainings
provided and the intended or observed benets. The project required unbigoted opinion along
with technical expertise which was provided by Fiinovation for assessment and
recommendations for future course of action.
With its team of experts, Fiinovation has successfully conducted evaluation and assessment
of Adult Literacy Centers (ALC) run by M/s JK Tyre & Industries Ltd. at Kankroli (Rajasthan),
Morena( Madhya Pradesh) and Mysore( Karnataka). For this purpose, a detailed research
was carried out to assess the efcacy and effectiveness of the ALCs.

13

Monitoring & Evaluation

Organization Brief
Established in 2008, Fiinovation (Innovative Financial Advisors Pvt.
Ltd.) has created a niche as a multi disciplinary research
organization in Corporate Social Responsibility (CSR) and
Sustainability domain. Headquartered in New Delhi, it has been
forging partnerships between credible NGOs and corporations that
are looking to implement and initiate community based interventions
and initiatives. With a pool of over 4000 credible NGOs spread
across the country, the motive has been to create awareness,
inspire innovative thinking and action with sustainable development
at its core. These NGOs are empanelled post a thorough and
stringent due diligence check through secondary research and on
ground evaluation.
Fiinovation is known in the development sector for enhancing quality
across organizational value chain through its in-house Proposal
Design & Research Laboratory for designing innovative CSR
initiatives. Fiinovation also supports corporations through
conducting need assessment, monitoring and evaluation as well as
impact assessment of their programs.
The distinct practices of Fiinovation include CSR Portfolio
Management (CPM), CSR-CSO Partnership, Initiative Design,
Initiative Management, Monitoring & Evaluation, Impact
Assessment and CSR audit across verticals of health, environment,
education and livelihood, which contribute to INR 15 crores in the
top line of nancial statement.

Afliations
Fiinovation has been a knowledge partner with various industrial bodies such as Confederation of
Indian Industries (CII) and Institute of Directors (IOD). With CII, the partnership emphasizes on
research for promotion of afrmative action along with CSR and social enterprises. The organization
has deliberated with various industrial leaders at numerous conferences conducted on corporate
governance, environment management and CSR & sustainability during its path breaking partnership
with IOD. Being a research organisation, Fiinovation also contributes to IOD's Master class for
Directors that has a wide spectrum of subjects such as corporate governance, sustainability, HR
practices, nance, corporate social responsibility, environment management and board
effectiveness. Other than this, the organization has been closely associated with other forums such
as NGO BOX and World CSR Congress.

14

Monitoring & Evaluation

15

Awards & Recognitions


Fiinovation has been conferred with the CSR Team of the Year &
Caring Company Award at the World CSR Day 2015. It was also the
recipient of the 'Best Social Innovation: Product or Service' at the World
CSR Day 2014. Apart from this, the company was bestowed with the
"Manager of the Year" & "Best Enterprise of the Year - in the eld of
Health, Education, Environment and Livelihood with a focus on CSR &
Sustainability" Awards by the European Business Assembly at the
Socrates Award Ceremony, Summit of Leaders in 2014 at Oxford Town
Hall, United Kingdom.

Mr. Soumitro Chakraborty addressing the gathering


at CII conference - Bridging the Gap: Fostering
growth through Education & Entrepreneurship

Mr. S. Chakraborty along with Smt. Sheila Dixit at the


24th World Congress on Total Quality and Leadership

fiinovation

Media Links
24/30, Ground Floor, Okhla Industrial Estate, Phase III, New Delhi - 110020, India
Phone: 011-42332200 | Fax : 011-42332205
Email : info@inovation.co.in, | Website : www.inovation.co.in

Вам также может понравиться