Impact Measurement

What ‘value’ do our programs create?

 The idea of ‘Impact’

have an effect on,

influence,

alter.

Example - Looking at the impact of a teacher training program

have an effect on - teacher effectiveness
influence- learning levels
alter- quality of education in schools
 Principles of ‘Development’

Inclusive

Participatory

Sustainable
 1 Purpose of
Participation

Experiences
2 with impact
assessment

What role do you

3 play in achieving
your organisation’s
social goals?

INTRODUCTIONS
Setting the Tone

“The best part of

learning is sharing
what you know”
- Vaughn K. Lauer
 1. What to Measure? Setting performance indicators

An indicator is a variable, the purpose of which is to measure change

in a phenomena or process.

PERFORMANCE INDICATORS ARE MEASURES OF

Inputs Processes Outputs Outcomes Impacts

Participation of key stakeholders in defining indicators is important because they are then more likely to
understand and use indicators for decision-making.
 S M A R T
Specific Measurable Achievable Relevant Time-bound

Example - Providing computers in schools

Short Term Indicator Medium Term Indicator Long Term Indicator

100% access to computers (technology) among Increased knowledge and usage among 40% Improvement in learning levels over
students of Std. 10-12 students three years
 2. Why Measure?

Provide answers to some of the most central development questions –

To what extent are we making a difference? What are the results on the
ground? How can we do better?
Analyse design, implementation, Provide information that is credible and
and results to understand, manage useful, enabling the incorporation of lessons
and communicate the social value learned into the decision making process and
that your work creates fuel cross learning

!
Compare the effectiveness of Help to clarify whether costs for an activity
alternative interventions are justified.

Inform decisions on whether to

expand, modify or eliminate
projects, programs or policies.
 EVALUATION IS NOT A ONE BALANCING THE CALL TO PROVE
TIME EVENT WITH THE NEED TO IMPROVE
Effective evaluation is not an “event” that Do not view evaluation only as an
occurs at the end of a project, but is an accountability measuring stick imposed on
ongoing process which helps decision projects, but rather as a management and
makers better understand the project. learning tool for projects.

SET APPROPRIATE
IMPACT AS EVERYONE’S
INDICATORS
RESPONSIBILITY
Be careful not to delegate all evaluation
3. How to Measure? Indicators should be developed based on
conversations with specific stakeholders
decision making to your evaluator. Stay
and an M&E system should encourage
involved and encourage teamwork.
stakeholder feedback at regular intervals.

IMPACT LIES IN THE EYES

INNOVATE AND
OF THE BEHOLDER
EXPERIMENT
Focus on creating successful replicable Value perceptions differ according to
models which are focused on a smaller context and situations. It is important to
group of individuals to be scaled to cover recognise different perspectives and
a larger populace and area different values.
 DETERMINING 

A BROADER
CONCEPT OF
VALUE

as appreciated by
the perception
1 and experience of
SOCIAL varied
stakeholders 
2 ENVIRONMENTAL

3 ECONOMIC

SOCIAL RETURN ON INVESTMENT

 SROI OBJECTIVES

IMPROVE COMMUNICATE PLAN MAXIMISE VALUE

Evaluative SROI Forecasting SROI

• Retrospective! • Prospective and introspective!
• Outcomes already achieved! • Outcomes to be achieved !
• Reporting purposes ! • Planning purposes
 SROI PRINCIPLES
Value things that matter

Do not over-claim

Only include what is material

Be transparent

Involve stakeholders

Verify the results

Understand what ‘changes’

SROI PROCESS

Identify 3
Evidence 5 Calculate
1
stakeholders outcomes SROI

Map Establish 6
Use
2 4
outcomes impact results
 INPUTS OUTPUTS OUTCOMES IMPACT

Regular and
effective
engagement
Interested youth Availability of
Case Study 1

inputs at school
level

1. MENTORSHIP

(ACADEMICS - 100 VOLUNTEERS ENGAGE ONE Volunteer satisfaction Improved learning
+CAREER TO ONE WITH 100 YOUTH TWO Improvement in career outcomes and lower
COUNSELLING) HOURS A WEEK goals drop-out rates among
2. GRADUATION - CAREERS DISCUSSED Academic improvements youth
EVENT

Government
Reduced budgets
policies change
over time
Other existing
programs

Evidence underpinning the key assumptions/ external factors

Turn to your worksheet
 ESTABLISH SCOPE
1 Audience and relevance
of the SROI results

2 Background of the
project

Type of project
3 and geographic
area

Duration of project and

4 timeframe of conducting 

the SROI
IDENTIFY Importance and Influence matrix

High
STAKEHOLDERS
A: the ‘primary B: the ones that
Beneficiaries’ make a difference

LEVEL OF IMPORTANCE
D: the ‘bystanders’ C: the ‘risk group’

Low High
LEVEL OF INFLUENCE
 Theories of Change Link outcomes and Theories of Change
are best when starting activities to explain require justifications at
with a goal and used to HOW and WHY the each step – articulate
decide program desired change is the hypothesis about
interventions expected to come why something will
about. cause something else

Design a complex initiative and

1 have a rigorous plan for success

Evaluate appropriate outcomes at

2 the right time and the right sequence

Explain why an initiative worked or did

3 not work, and what exactly went wrong

Theory of Change
A Theory of Change shows the big, messy “real world” picture, with
all the possible pathways leading to change, and why you think they
lead to change (do you have evidence, or is it an assumption?)

CLEAR HYPOTHESES OF
CHANGE
Does the narrative highlight
and describe the overall logic
of the intervention and the key
hypotheses which the
programme is based on?
ANALYSIS OF THE
CONTEXT ASSESSMENT OF THE
Does the theory of change
EVIDENCE
make sense as a response to
analysis of the context, the Is there a narrative assessment
problem and the changes of the evidence for each key
needed? Is there one hypothesis?
statement that sums up the
theory of change?

PROJECT SUMMARY
10% increase in the number
of Grades 5-6 primary
GOAL
students continuing on to
high school within 3 years.
Improve reading proficiency Reading proficiency
OUTCOME among children in Grades 5-6 among children in
by 20% within 3 years. Grades 5-6
1. 500 Grade 5-6 students
with low reading proficiency
complete a reading summer
camp.
OUTPUTS
2. 500 parents of children in
Grade 5-6 with low reading
proficiency help their
children read at home.
1. Run five reading summer
camps, each with 100 Grades Number of summer
5-6 students who have low camps run.
reading proficiency.
ACTIVITIES
2. Distribute 500 “Reading at
Home” kits to parents of
children attending summary
camps.
Logic Model Case Study
MEANS OF
INDICATORS RISKS / ASSUMPTIONS
VERIFICATION
Percentage of Grades
Comparison of primary
The Outcome is
5-6 primary students
continuing on to high
and high school N/A sometimes called the
enrolment records.
school. Purpose or Objective.
Six monthly reading Improved reading
proficiency tests using proficiency provides self
the national assessment confidence required to If you have more
tool. stay in school. than one output they
can be numbered
Number of students Children apply what they
completing a reading
Summer camp
learnt in the summer
sequentially.
attendance records.
summer camp. camp.
Number of parents Survey of parents Children are interested The number of the
helping their children conducted at the end of in reading with their
to read at home. each summer camp. parents.
activity should match
the number of the
Parents of children with output that it
low reading proficiency
Summer camp records.
are interested in them
corresponds to 

attending the camps. (e.g. Activity 1 leads
to Output 1).
Parents are interested
Number of kits
Kit distribution records. and able to use the kits at
distributed.
home.
 INPUTS OUTPUTS OUTCOMES IMPACT

Better
quality inputs
lead to
- New textbooks in all subjects improvement in Increased non
- Schools and ABE centres are learning and salary spending Availability
Theory of Change

teaching for quality in of inputs are

receiving capitation grants schools school level
- Strengthened in-service and pre- impacts national
service teacher training targets
- Improved school planning and
leadership Improved learning
Funding for - Improved community Teachers have skills and
improvement outcomes and lower
involvement resources to help
in quality of drop-out rates among
- Increased harmonisation and students learn.
education boys and girls
alignment
- Increased momentum behind the
Governments drive to improve Increased Quality
quality accountability of improvements
schools leads to attract and retain
- Strengthened education Support leads students
better teaching
management information to better and learning
availability of
inputs

Evidence underpinning the key assumptions

 INDIVIDUAL 

QUANTIFYING
1 individual contributions to

INPUTS
the project, or what the cost
to individuals has been

COMMUNITY

2 what collective resources
were required or mobilised
for the project

ORGANISATIONAL
3 money and resources brought to
the project
1 TANGIBLE QUANTIFYING
OUTPUTS
EASILY
2 MEASURABLE

3 PRACTICAL

4 IMMEDIATE

5 INTENDED
 EVIDENCING
1 Stakeholders formulate OUTCOMES
the indicators

2 Putting a value on
outcomes

Establishing how long

3 outcomes tast

4 Collecting outcomes
data
 ESTABLISHING IMPACT
BEFORE AND AFTER
1 Resulted cost saving or increased
income over time

RELATIVE VALUE
2 Put the obtained result into
perspective by comparing it to
something else.

HEDONIC PRICING
3 Builds up a value from the market
values of constituent parts of the
service or good being
considered.
Stakeholder input
experiments/ tools
1 Priority Ranking 4 Tree diagrams
Helps in understanding the
Multi-purpose, visual tools for narrowing and prioritising
programs which
problems, objectives or decisions. Information is organized
stakeholders think have a
into a tree-like diagram. The main issue is represented by
high impact and the
the trunk, and the relevant factors, influences and outcomes
programs which have low
are shown as roots and branches of the tree.
impact.
5 Contingent Valuation
2 Changes Observed In the contingent valuation method, the were asked to
place values for each interventions. The question was
This tool gives an answer to the posed in two ways to each group: How has the intervention
perceived benefits as helped save costs ? (monetary benefits) How will you place
understood by the stakeholders. a value on the benefit derived from each intervention ?
(giving financial values to social benefits) Then we
summed over all the values that they were willing to pay.
3 Resource Allocation This is known as the Contingent Valuation method.

Gives the stakeholder the freedom to decide how they will

spend the hypothetical money for the next phase of the project.
This helps in stakeholders for issues/ projects.
Stakeholders Outcome Indicator Proxy Average value Total Yearly
Value

Volunteers Volunteer How they value Average hourly (50,000/160)**2*52=32,500 inr 32,50,000
satisfaction their experience earnings

Self-reported 20,000 inr 20,00,000

meaning derived
from volunteering
Youth Academic grades tutors 200 inr/hr*2 hr/wk*52 wks/ 20,80,000
improvement yr=20,800 inr

Youth Sense of Response to Career counseling 500 inr/session*52 sessions/ 26,00,000

direction/ career question about year=26,000 inr
goals career aspiration
Corporation Consumer Consumers’ Survey consumers 5,00,000 inr (Estimate of 5,00,000
perception attitudes about loyalty in impact of increased revenue
towards response to based on survey results
corporation knowledge of CSR
Corporation Community Community’s Cost spend by 10,00,000 10,00,000
relations perception of other companies
company on community
relations
NGOs Resource pool Work hours of Value of 2 hrs/week *52 weeks/year * 10,40,000
volunteers volunteers’ work 100 inr/hour (cost of NGO
hours employee)=10,400
NGOs Name Additional funds Difference in 5,00,000 per NGO 15,00,000
recognition and raised fundraising over
publicity previous year
Total 1,69,70,000
 CALCULATING SROI
!
DEAD WEIGHT 

1 (‘what would have happened anyway’)

ATTRIBUTION
2 (‘who else helped’)

DISPLACEMENT
3 (‘how much of the outcome displaced
other outcomes’)

4 DROP OFF
(‘what will happen over time’)

SROI ratio = Total (adjusted) value of results 


--------------------------------------

Total value of inputs
Stakeholde Outcome Attribution Dead-Weight Displacement Drop-
r off
Volunteers Volunteer satisfaction 10% 20% (May have sought out 20% (May cause 25%
other opportunities employees to
elsewhere) not have enough
time for work
Volunteers Sense of purpose 10% 20% (May have used the 0% 25%
time to pursue other
passions or interests)
Youth Academic 20% (Contribution 20% (NGOs maybe would 0% 25%
improvement of schools, have created other
teachers, and programs)
families)
Youth Sense of direction/ 20% (Contributions 10% 0% 25%
career goals of families and
Corporatio Consumer school staff) 10% 20% (May have launched 0% 25%
n perception other marketing
Corporatio Community relations 10% campaigns) 10% 0% 25%
n
NGOs Resource pool 10% 20% (Could have sought 0% 25%
out other sources of
NGOs Name recognition 10% volunteers) 10% 0% 25%
Benefits Year 1 Less Less Less Year 2 Less Less Less Less drop
Deadweight Attribution displacement Deadweight Attribution displacement off (25%)

Volunteer 32,50,000 26,00,000 23,40,000 18,72,000 32,50,000 26,00,000 23,40,000 18,72,000 14,04,000
satisfaction

Sense of purpose 20,00,000 16,00,000 14,40,000 14,40,000 20,00,000 16,00,000 14,40,000 14,40,000 10,80,000

Academic 20,80,000 16,64,000 13,31,200 13,31,200 20,80,000 16,64,000 13,31,200 13,31,200 9,98,400
improvement

Sense of direction/ 26,00,000 23,40,000 18,72,000 18,72,000 26,00,000 23,40,000 18,72,000 18,72,000 14,04,000
career goals

Consumer 5,00,000 4,00,000 3,60,000 3,60,000 5,00,000 4,00,000 3,60,000 3,60,000 2,70,000
perception

Community relations 10,00,000 9,00,000 8,10,000 8,10,000 10,00,000 9,00,000 8,10,000 8,10,000 6,07,500

Resource pool 10,40,000 8,32,000 7,48,800 7,48,800 10,40,000 8,32,000 7,48,800 7,48,800 5,61,600

Name recognition 15,00,000 13,50,000 12,15,000 12,15,000 15,00,000 13,50,000 12,15,000 12,15,000 9,11,250
and publicity

Total 96,49,000 96,49,000 72,36,750

 Prasuti Foundation

CSR wing of HealthWise Ltd.

Focus areas are health and education

Programs

• Provision of computers
• Health camps
• Tailoring training
• Building toilets

Case Study 2
 THEMATIC
DOMAIN
Economic Outcome
1. Savings on computer installation and
maintenance costs due to free accessibility
Education
2. Savings on software acquisition
3. Savings on external training
1. Decrease of travel expenses
to get first aid treatment from
outside
Health 2. Savings on medicines for
basic ailment
3. Decrease of medical expenses
due to disease prevention
1. Increase in household incomes
Women 2. Savings in availing skill training
Empowerment
1. Savings in Consrtuction Costs
Sanitation 2. Savings in health costs due to decreased
illness
IMPACT INDICATORS
Social Outcome
1. Value addition of extra skill/subjects

!
2. Improvement of teaching methods
3. Access to better Job opportunities (long term)
4. Improvement in Learning Levels
5. Increase in teacher effectiveness
1. Improvement of health conditions in the village
2. Creation of awareness about health and hygiene
3. Increase in school attendance due to lesser
sick days
4. Decrease in waterborne diseases
5. Improvement of quality of health
1. Empowerment of Women
2. Utilisation of skill of the women
1. Better sanitation facilities for households
2. Improved long-term health condition in the village
During SROI experiments, people’s allocation calculated are given below

People’s allocation 

Program Allocation of the company (INR lakh)
(INR Lakh)

Education 15 5

Health 10 20

Women empowerment 10 5

Sanitation 15 20

During SROI experiments, values as expressed by stakeholders are given below

Average Value 

Total value addition
Program Relevant stakeholder (per person) No. of people
(INR lakh)
(INR)
Education Teachers 1200 1,000 children 12

Health Community 1500 2,000 people 30

20,000 

Women empowerment Women 100 women 20
(for 3 years)

150 household = 750

Sanitation Low-income families 4000 30
people
THEMATIC DOMAIN

Deadweight Attribution Drop-off

Education
15 20 5

Health 10 10 15

Women
10 10 5
Empowerment

Sanitation 5 0 0

Calculation of final benefits

Value (–)
Program Value Value (-) attribution Value (-) drop-off Final value of benefits
deadweight

Education 12 lakh 10.2 lakh 8.16 lakh 7.75 lakh 7.75 lakh

Health 30 lakh 27 lakh 24.3 lakh 20.66 lakh 20.66 lakh

Women empowerment 20 lakh 18 lakh 16.2 lakh 15.39 lakh 15.39 lakh

Sanitation 30 lakh 28.5 lakh 28.5 lakh 28.5 lakh 28.5 lakh