PEN International Guidance Notes for PEN Centres: 6.

Monitoring and Evaluation

Introduction:
These Guidance Notes are designed to assist Centres in monitoring and
evaluating their projects and programmes. They provide information for Centres
to implement systems as they plan and implement activities to help the Centre
measure if the project has been successful and report to donors.

You might feel that, at this stage, you do not need all the information that is
provided in these Guidance Notes. It is useful however for all Centres to be
familiar with relevant concepts and terminology so that, in the future, you will be
able to develop more complex Monitoring and Evaluation (M&E) systems which
may be required by new and different donors.

What is Monitoring and Evaluation?

Monitoring and evaluation form an essential part of all project work. Specifically,
they are key components of the project cycle and Centres Annual Plans. Briefly:

- Monitoring is a system for collecting information which tracks what you

have done in a project;
- Evaluations assess what happened as a result of these activities, and
answers the questions To what extent did your project achieve what it set
out to achieve? What have we learned as a result of this assessing the
effectiveness of our work?

Evaluations rely heavily on information collected in the monitoring systems for

assessment and analysis of progress towards agreed aims and objectives.

Why monitor and evaluate?

It is important that Centres monitor and evaluate their work and projects for the
following reasons:
- Accountability: PEN Centres are accountable in two directions: to their
donors, their members and to their beneficiaries. The donors want to know if
their funds have been used effectively and efficiently.
Beneficiaries have a right to be involved in decisions and actions which
affect them. Centres therefore have a responsibility to be clear about what
they are doing, why they are doing it; and what results have been achieved.
By monitoring and evaluating their projects, they can be accountable both to
donors and to beneficiaries and their Centre members.

1
 - Improving performance: At the end of a project, Centres will need to
reflect upon what worked well, what was not so successful and how they will
plan for the next piece of work. The evaluation process directly supports this
process. Having completed these reflections, Centres should be in a
position to improve performance and results in subsequent projects.

- Learning: As well as improving performance, monitoring and evaluation can

also provide valuable lessons for other projects either within the Centre or in
relation to similar projects that are being carried out by others. The process
of monitoring and evaluation can also help staff and others develop new
skills.

Additionally, donors usually require organisations to provide evidence of how

they have monitored and evaluated their projects.

There are three levels of results that come from projects being carried out:

- Outputs: the products or services, which are delivered on completion of

the project activities. In other words: What was done?
Examples of outputs that PEN might use:
o Reading Circles: monthly meetings for PEN members and the
general public, with book presentations, readings and discussion.
o Writers workshops where published authors will support
participants with ideas and guidance
o Story competitions in schools

- Outcomes: immediate and observable changes in relation to the project

objectives, which were brought about as a direct result of project activities
and outputs. In other words: What happened?
Examples of outcomes that PEN Centres might use:
Greater interest and enthusiasm for literature
More and better quality of materials produced
Changes in amount and quality of press coverage
Improved levels of literacy in schools
Improved exam results

- Impact: this concerns longer-term changes that have come about as a

result of a project or programme. Impact can be either positive or negative
both are equally important. What changed?
Higher attendance at school
More girls going into work after completing school
Girls having their first babies later than previously

2
Monitoring and evaluation systems are designed to report on these different
levels of results.

When and how to use it?

It is important to plan for monitoring and evaluation at the beginning of the project.
In order to do this, you need firstly to be clear about what is involved in and different
about the two processes. Then you will need to plan how to build these in to your
work. The last section in these Guidance Notes provides ideas about how to gather
the information that you will need in order to carry out your monitoring and
evaluation plan

In terms of monitoring, you will see (from the table below) that this is an on-going
activity, which should be documented as activities take place. It is often a good idea
to develop forms or report books in order to make the collection of this information
as routine as possible

All the monitoring information will be needed for any evaluations that are
conducted. The following table, adapted from Sharpening the Development
Process, Oliver Bakewell, INTRAC, provides an overview of what needs to be
considered when planning for M&E:

Monitoring
Timing Continuous, throughout
the project
Scope Day to day activities
Main participants Project staff and project
users
Reporting formats Regular reports and
updates to project users,
management and donors
Evaluation
Periodically at significant points in
the project: mid-term or end of
project are most common
Assess overall delivery of activities
and progress towards achieving
aim and objectives
External evaluators/facilitators,
project users, project staff, donors
Written report with
recommendations for changes to
project

Key Steps
Once you know what needs to be done, by whom and when, the next task is to
devise key questions and indicators that will be used to collect and analyse the
information you need.

3
 What are indicators?
Essentially, they are what they sound like: illustrations or pointers that something has
happened or is happening.
- When the car in front of your indicates that it is turning left, you understand what
is happening
- When you see very heavy dark clouds in the sky, you have an indication that it
may rain very soon
Lets take an example of an objective and a few activities and develop some key
questions and indicators:

Objective (outcome): To have promoted greater participation with reading and

writing in .. region
- Activity 1: Reading Circles: monthly meetings for PEN members and the
general public, with book presentations, readings and discussion.
- Activity 2: Organise writers workshops where published authors will
support participants with ideas and guidance
- Activity 3: Story competitions in schools
-
Activity Key Question Indicators
Reading Circles: Are the Reading Numbers and locations of Reading circles
monthly Circles popular and Numbers of people (men/women) attending
meetings for building on local Topics covered
PEN members interest? Feedback from members
and the general Press reports
public, with book Book sales
presentations,
readings and
discussion.

Organise writers To what extent are Numbers of workshops

workshops the writers Numbers of people ( men+women) attending.
where published workshops effective Results: participants materials being published
authors will in enabling new on web, newspapers, anthologies, books.
support writers to get
participants with published?
ideas and
guidance
Story Are the competitions Nos of schools involved
competitions in popular and building Nos of children (girls+boys) taking part
schools on interest in Publicity
reading and writing?

4
Indicators at outcome level - checking progress towards achieving the objective
would include the following which would be measured over time ( to indicates levels
of change)
Changes in attitude to literature
Quality of materials produced
Changes in amount and quality of press coverage
Levels of literacy in schools
Exam results

In other words, the outcomes (results) can be seen as the sum of the parts
(activities)

A note on indicators
Indicators may be:
- Quantitative the change in indicators can be shown through numbers. Eg the
number of people submitting stories to a competition over time
- Qualitative the change is shown through description. Eg. the changing level of
interest in literature
- Direct: Something you can measure directly eg. the number of meetings held by a
committee
- Indirect : For example the measurement of increasing womens involvement in a
committee could be indicated by the number of decisions made which support
issues raised by women.

5
Gathering the Information that you need to monitor and evaluate:

As stated at the beginning of this Note, you will need to plan how to gather information needed for monitoring and
evaluation. The following table provides some methods that you might consider using, as well as an indication of the
strengths and weaknesses for each. When making you monitoring and evaluation plan, it will be wise to decide which
methods you will use at each stage, so that you also plan the time, costs and the resources that you will need. Many
plans will use some elements of the following techniques.

Definition and Use Strengths Weaknesses

Technique
Case Studies Collecting information that results in a story
that can be descriptive or explanatory and
can serve to answer questions of how and
why
Holding focussed discussions with members
Focus Groups of target population who are familiar with
issues that are being explored . The
purpose is to compare the beneficiaries
perspectives with concepts in the
evaluations objectives
- Good case studies are
- Can deal with a full variety of difficult to do
evidence from documents, - Cant generalise findings
interviews, observation
- Time consuming
- Provides insights that are not
easy to collect in more formal
processes
- Similar advantages to - Can be expensive and
interviews time consuming
- Particularly useful where - Cant generalise findings
participant interaction is
desired

The interviewer asks questions of one or
Interviews more persons and records the respondents
answers. Interviews may be formal or
informal, face-to-face or by telephone, or
closed- or open ended
- People and institutions can
explain their experiences in
their own words and setting
- Flexible to allow the
interviewer to pursue
- Time consuming
- Can be expensive
- If not done properly, the
interviewer can influence
the interviewees

6

Observing and recording situation in a log or
Observation diary. This includes who is involved; what
happens; when, where, and how events
occur. Observation can be direct (observer
watches and records), or participatory
(observer becomes part of the setting for a
period of time).
Developing a set of survey questions whose
Questionnaires answers can be coded consistently
Reviewing documents such as records,
unanticipated lines of enquiry
or to probe issues in depth
- Particularly useful where
language difficulties are
anticipated
- Greater likelihood of getting
input from senior officials
- Provides descriptive
information on context and
observed changes
- Can reach a wide sample
simultaneously
- Allows respondents time to
think before they answer
- Can be answered
anonymously
- Impose uniformity by asking
all respondents the same
things
- Make data compilation and
comparison easier
- Can identify issues to
response
- Quality and usefulness of
data highly dependent on
observers observational;
and writing skills
- Findings can be open to
interpretation
- Does not easily apply
within a short time-frame
to process change
- The quality of responses
highly dependent on the
clarity of questions
- Sometimes difficult to
persuade people to
complete and return
questionnaire
-
- Can be time consuming
7
Written administrative databases, training materials investigate further and
Document and correspondence provide evidence of action,
Analysis change and impact to support
respondents perceptions
- Can be inexpensive