Effective

Monitoring&
Evaluation

Monitoring & Evaluation

Evaluation
Monitoringandevaluationshouldbebuiltinto
g
theprojectattheplanningstage.Itshouldnot
pp
,
beanaddonthathappensattheend,but
shouldbewoventhroughouttheproject.
Evaluationisawayofseeing,ofreflectingon
practice (VanDerEyken1993)
(Van Der Eyken 1993)
Judgingmeritagainstsomeyardstick
(Phillipsetal,1994)
Monitoring & Evaluation

Do you know?
Doyouknow?
Howyoucanshowthataprojectisworth
continuing
p j
p p
Whattheprojecthasdoneforthepeople
livinginyourcommunity
Why grant providers should continue to fund
Whygrantprovidersshouldcontinuetofund
yourproject
Thisiswhatevaluationandmonitoringcan
h l
helpyoufindoutandgiveyouevidenceto
fi d t d i
id
t
backupyourfindings
Monitoring & Evaluation

What is Self Evaluation

WhatisSelfEvaluation
Selfevaluationiscarriedoutbythe
Self
evaluation is carried out by the
organisationthatisdeliveringa
project.
Th
Theevaluationisinternallydevisedand
l i i i
ll d i d d
delivered.
delivered
Monitoring & Evaluation

Self Evaluation
SelfEvaluation
..isusuallyseenasinextricablylinkedwith
i
ll
i
i bl li k d i h
organisationaldevelopment.Evaluationthus
becomesanintegralpartofanorganisation
workingoutwhatitistryingtoachieve,
collectingevidenceofprogress,reflectingon
howitisdoing,andexploringtheimplications
forfutureplanninganddevelopment
(Charitable Evaluation Services, 1998)
(CharitableEvaluationServices,1998)

Monitoring & Evaluation

Self Evaluation Advantages

SelfEvaluation
Theevaluationismoreintegratedwithother
organisationalprocesses
p
g
y
Cantapintointernalknowledgemorereadily
Isflexibletotheneedsoftheorganisation
Controlofevaluation&otherdataremainsinternal
Control of evaluation & other data remains internal
Theresultsaremorelikelytoleadtochange
Theprocessislessthreateningtostaff&volunteers
Usuallycheaper
Enablesstaffandvolunteerstolearnnewskills
Monitoring & Evaluation

Self evaluation challenges

Selfevaluation
D
Demandstimeandresourcesfromtheorganisation
d ti
d
f
th
i ti
Maybeviewedaslessobjective
Internalstaffmaybetooclosetotheprojectcreating
conflictsofinterest
Participantsmaytalklessfreelytointernalstaff
Staff&volunteersmaynothavethenecessary
evaluationskills
Maybedifficulttolinkevaluationfindingstobroader
issues

Monitoring & Evaluation

Project&EvaluationDiagram
What problem do you want to work with?
Needs Assessment

Funding

What must you

do differently to
increase your
success?

Partnership

What changes do
you want to achieve
in yyour community?
y

What must you

do to achieve
this change?

Outcomes

Aims & Objectives

Programme
development

Have yyou succeeded

in producing the
required change?
Evaluation of organisation &
project
Monitoring & Evaluation

How will you know when

thi change
this
h
h
has occurred?
d?
Performance indicators, outcomes,
quality standards

Thebenefitsofevaluation
For Funders

For Organisations

Ensures good use of public funds

Ensures that resources are used efficiently

Highlights good practice

Helps clarify aims & objectives

Identifies gaps in provision

Provides evidence of impact

Provides a basis for selecting among competing

applicants

Sets standards and provides quality control

Informs policy

Uncovers unexpected consequences

Provides information for planning and
campaigning
Helps validate new approaches

For Staff
Provides feedback on performance
Challenges assumptions and confirms
impressions
Suggests areas to develop further
Offers access to the views of others
Helps staff (& volunteers) to see their
Helps
work in a wider context
Monitoring & Evaluation

F Users
For
U
Demonstrates usefulness or limitations of
project
Provides an overview of the project
Offers a chance to have views heard
Represents an opportunity for more active
participation
Publicises any gaps

Starting Point
StartingPoint
Tobeabletocarryouteffectivemonitoringand
y
g
evaluationyouneedtohavesomethingto
measureagainst
Whenyouareplanningaprojectyouneedtoset
someaimsandobjectives

Monitoring & Evaluation

Setting Aims & Objectives

SettingAims&Objectives
Aims
Thesearegeneralstatementsofintent,
p p
usedtoindicatetheoverallpurposeofa
projectorprogramme.
FormorehelpseetheProjectPlanningmodule
http://www3.hants.gov.uk/communitysupportservice/howto
d /
do/projectplanning.htm
l
h

Monitoring & Evaluation

Setting Aims & Objectives

SettingAims&Objectives
Objectives
Thesestatewhatwillbeachievedatspecificpointsandare
h
h
ill b
hi d
ifi
i
d
moredowntoearthactionstatements.
Theyprovidestructureforaneventorproject
Theyactassteppingstones whatskills,knowledgeor
behaviourareyouhopingtochangeasaresultoftheproject
Theyhelpwithplanning
They help with planning
Ensureyouuseappropriateverbs,avoidwordlikeunderstand
unlessyoucanproveitinyourevaluation
Needtobe
Need to be measurableas
measurable astheywillactasabasisfor
they will act as a basis for
appropriatemonitoringandevaluation
NeedtobeSMART(seenextslide)

Monitoring & Evaluation

Setting Aims & Objectives

SettingAims&Objectives
S=Specific
M=Measurable
A = Attainable/ Achievable
A=Attainable/Achievable
R=Realistic
T=Haveatimetable

Monitoring & Evaluation

Whatsortofdata?
QuantitativeorQualitative
Quantitative
Numerical or statistical data
Numericalorstatisticaldata
e.g.thenumberofattendees

Thistypeofinformation

Givesobjective,accurateresults
Gives
objective accurate results
Canbeusedtomakecomparisons
Does not identify opinions/ feelings
Doesnotidentifyopinions/feelings
Unlikelytomeasureimpact(longtermchanges)

Monitoring & Evaluation

Quantitative or Qualitative
QuantitativeorQualitative
Qualitative
Observationsthatdonotinvolvemeasurements
ornumbers e.g.descriptionsoropinions

Thistypeofinformation

Canproducemoreindepth,comprehensiveinformation
Can
produce more in depth comprehensive information
Collectsopinionsthathaveaholisticview
Canhaveproblemswithreliabilityifnotmanagedwell
I
Issubjecttoresearcherbias
bj t t
h bi
Canbetimeconsuming
Cantbegeneralisedtothewidercommunity
Cangiveanindicationofanychangestheprojecthasmade(impacts)
Canmeasuresoftoutcomesthatcannotbemeasuredinnumbers

Monitoring & Evaluation

Summative or Formative
SummativeorFormative
Summative gathertheinformationtowards
p j
theendoftheproject
FFormative
i
G h i i f
Gatheringinformationrightfrom
i
i h f
thebeginningoftheplanningstage,Focuses
ontheprocessaswellastheoutput,probably
the most useful for you.
themostusefulforyou.

Monitoring & Evaluation

Whendoyouthinkabout
When do yyou think about
monitoring
and
evaluation
monitoringandevaluation?
Shouldfeatureinthefirstplansforanyproject
not an add on at the end
notanaddonattheend
Shouldensureyouhavethetimeandresources
needed to implement (you can add this cost
neededtoimplement(youcanaddthiscost
toafundingapplication)
Monitoring & Evaluation

Guidance
Makeitparticipative
Make sure you are gathering the information you need
Makesureyouaregatheringtheinformationyouneed
ifyoudontneeditwhyask!
Don
Donttleaveittothelastminute
leave it to the last minute
Planitandincludethecostsintoyourproject
Ensureallstaff&volunteersareinformedandinvolved
Makeitfun&interesting

Monitoring & Evaluation

Methods
Observations
A lid
Avalidmethodgatheringinformation,haveathinkabouthowthisisrecorded
h d h i i f
i
h
hi k b
h
hi i
d d
andevaluated.Photographycanbeusedasonetoolforthis.
Questionnaires
Q
i
i
Triedandtrustedmethod,remembertomakethemuserfriendly consider
equalitiesanddiversitywiththismethod,willeveryonebeabletocontribute
equally through this method
equallythroughthismethod.
Interviews
G d
Goodwaytocollectqualitativeinformation,but1:2:1methodwillbetime
ll
li i i f
i
b 121
h d ill b i
consuming.Alsoneedtoensureinterviewersarewelltrainedtoreduce
interviewerbias.
Telephoneinterviews
Asabove butanopportunitytospeaktopeopleaftertheyhavehadachanceto
reflect on the event or activity
reflectontheeventoractivity.
Monitoring & Evaluation

Methods

Participantlogbooks
Goodmethodforlongertermactivitiese.g.playschemeortrainingcourse
Groupdiscussion
Useful method, but make sure it is facilitated carefully to allow everyone to have
Usefulmethod,butmakesureitisfacilitatedcarefullytoalloweveryonetohave
theirsay.Youmaywanttoputgroundrulesinplaceorevenuseatalking
stickwhereonlythepersonholdingthestickcanspeak.
Focusgroups
Bringing68peopletogethertohaveaguideddiscussioncanbeaveryeffective
methodoffindingoutmoreindepthinformationaboutwhatpeoplethink.
TakealookattheFocusGroupProjecttoolkit
http://www3.hants.gov.uk/communitysupportservice/howtodo.htm

Datacollection
D
ll i
Thereareseveralwaystocollectingnumbers,ethnicity,ageetc (seenextslide),
butonlycollectthisinformationifyouaregoingtouseit
Monitoring & Evaluation

Interactive Evaluation Methods

InteractiveEvaluationMethods
Using sticky dots, this form of data
collection
co
ect o can
ca be used to gather
gat e a wide
de
range of information.
In this example it is asking how people
travelled and has this in picture and word
form to make it more accessible
You could also use this to gather age
groups ethnicity or any other information
groups,
that falls into groups.
Have this at the beginning of your event
when
h participants
ti i
t are arriving
i i
it gives
i
you
a chance to talk to people and an
opportunity to gather information in a non
threatening way
way.
N.B. You could also mount this on a piece of
polystyrene and use map pins if you wanted
t reuse it for
to
f severall events
t
Monitoring & Evaluation

Interactive Evaluation Methods

InteractiveEvaluationMethods

Putt the
P
th three
th
faces
f
on the
th wallll and
d give
i each
h participant
ti i
t
a sticky dot to show how they feel about the event.
To make this even more effective yyou can do this at the
beginning and end of the event to track changes
Hopefully you will end up with a face like the one shown
on the left
Monitoring & Evaluation

Measuring Change
MeasuringChange
This sheet can be used for a fairly
small group to find out how people
feel at the beginning of an activity
and again at the end.
It is a good ice breaker (in an
appropriate situation) as you can
ask p
people
p to explain
p
why
y they
y have
chosen a particular face. This is
then repeated at the end.
This could then indicate how
participants feel at the end of the
process and with careful facilitation
will give you an idea of how
effective the activity has been

Monitoring & Evaluation

Balls in Buckets
BallsinBuckets
Simple voting Systems
Think laterally about the
way you collect data ask
a question and have a
b k t (or
bucket
( some other
th
receptacle) for each of the
possible answers. Give
each
h off the
th applicants
li
t a
ball (or something similar)
to put in the bucket that is
showing the statement they
agree with
Tip if you want accurate
results make sure there are
no toddlers around playing
ball!
Monitoring & Evaluation

Evaluation Checklist
EvaluationChecklist
Issues to consider when embarking on evaluation
Do stakeholders have differing views about the purpose of evaluation and
the approach adopted
Who will be involved in the self evaluation
What form of support will be available to the staff undertaking the evaluation
g in evaluation skills
Do Staff or volunteers need training
Is the process of self evaluation integrated with other organisational
processes
Have realistic time scales been made of the time needed to carry our the self
evaluation
Have steps been taken to increase the credibility of the data gathered
Do you have frameworks or other resources to guide the evaluation

10 top tips for ensuring your evaluations are sound

http://rodlaird.hostinguk.org/default.asp?page=398
Monitoring & Evaluation

Tick

Assessing Impacts
AssessingImpacts
Thesearethelongertermeffectsthatyourproject
h
h l
ff
h
mayhave howhasparticipatingimpactedon
thelifeofpeoplewhotookpart.
Recordexpectationsatthebeginning
Carryoutamidpointreview
Completeanendofsessionreview
Complete an end of session review

Monitoring & Evaluation

AssessingImpacts
whatyoumightask
Willtheycomeagain/takepartinfuture
opportunities thinkaboutsustainabilityforthe
future
Didtheeventmeettheirexpectations
Whatdidtheyenjoy/notenjoy
Thingstheygained
Things they gained
Isthereanythingtheywanttodoasaresultof
attending this event
attendingthisevent
Askforsimplewordstosummariseevent
Monitoring & Evaluation

Completing the Loop

CompletingtheLoop
Ensuretheinformationisfedbacktotheright
Ensure
the information is fed back to the right
placeattherighttimeandintherightformat
to
to
Fundingbody
Workers(staff&volunteers)
W k ( ff & l
)
Participants(includingany
partnerorganisations)
Community(users)
Community (users)
Otherstakeholders
Monitoring & Evaluation

Finallyy
Evaluationandmonitoringgivesyouachancetobe
creativejustmakesurethemethodyouuseis
appropriatetothesituationoractivity.
Makeitfun,butplanitcarefullyaspartofyourproject
toensureyougettheinformationyouneed
Dontforgettousetheresultstohelpyouplanfuture
eventsandfeedbacktoanystakeholders
y
Dontthinkofmonitoringandevaluationasachore
look for ways to make it fun and you will get better
lookforwaystomakeitfunandyouwillgetbetter
reactionsandresultsfromtheparticipants
Monitoring & Evaluation