Civil Society Challenge Fund Final Evaluation: Lessons

Contents
1) Introduction
2) Methodology
3) Civil Society Challenge Fund Management
4) UK/Southern Civil Society Partnerships - Partnerships for
Change
5) Capacity Development: Pathways to Change
6) Civil Society Challenge Fund impact: emerging trends

Summary of Evaluation Findings

The Civil Society Challenge Fund continues to be relevant to the local context of Southern CSOs
by supporting civil society to empower poor and marginalised people to increase their voice in
decisions that affect their lives. In addition to widespread evidence of civil society engagement in
policy dialogue and development, a large number of projects contributed to the adoption of
policies likely to bring lasting benefits for poor, vulnerable people by influencing the policy,
professional and normative frameworks relevant to their targeted populations. This was often
achieved through a wide range of formal and informal capacity development approaches. The
Evaluation found many positive examples of projects benefiting very diverse groups of poor and
marginalised people.

7) The Evaluation Conclusions

Introduction

By the end of the fund 526 projects will have

been funded across 31 countries: worth an
approximate total of 164 million.

The Civil Society Challenge Fund was one of DFIDs longest

running challenge funds. It was created in 2000 to support
UK based Civil Society Organisations (CSOs) to strengthen
the capacity of civil society in the Global South to effect policy
and practice change that benefits poor and marginalised
people. In 2010 the fund was closed to new applicants and
the last projects were completed in March 2015.
As the fund drew to a close, IOD PARC was asked to conduct a
Final Evaluation of the Civil Society Challenge Fund with an
emphasis on learning. The objective of The Evaluation was to
Understand how the Civil Society Challenge Fund has
contributed to improved capacity of southern civil society to
engage in local and national decision making processes.
This summary outlines some key learning from the Civil
Society Challenge Fund theory of change, and some lessons
for future challenge funds.

Eastern Province Womens Literacy Project in Rwanda -Adventist

Development and Relief Agency

Methodology
The IOD PARC team developed an implicit
theory of change for the Civil Society Challenge
Fund consisting of seven pillars, each of
which is based on a working hypothesis. The
team developed an evaluation framework and
used the hypothesis from pillars two to five to
guide the enquiry:

CSCF Evaluation
Intervention Logic

Data collection
methods
Data was gathered from:
Document review of

Hypotheses

63 .projects approved
since 2010.

Pillar 2: Outsourced fund management

provides an effective means through grant
management and programme support,
systems and processes of delivering the
funds objectives.

In-depth document

review of 17 projects.
Interviews with 5 DFID

Staff, 7 Fund
Management staff, 16
Grantholder staff and 17
staff from Partner
Organisations.

Pillar 3: UK & Southern CSO partnerships

with Southern CSOs provide an effective means
of channelling funds to strengthen Southern
Civil Society.

Pillar 4: The capacity development provided

through these partnerships will be relevant to
and effective in strengthening civil society
capacity at individual, organisational and
systemic levels.

Pillar 5: The resulting improvements in

capacity will lead to increased engagement in
decision-making and policy and practice
changes that impact positively on poor women
and men.
This summary will look at some key learning
associated with each pillar and for future
Challenge funds.

Online partner survey: 38

of the 63 organisations
responded (56%).
Capacity development: three levels of change
The team looked at capacity development at three different levels:
The individual level e.g. the development of individual skills and expertise
through participation in training courses, workshops, communities of practice,
South-South learning initiatives, mentoring, coaching and other methods.
The organisational level e.g. the development of an organisations capacity in
terms of its procedures, systems, policies and culture. This in turn refers to its
organisational sustainability.
The systemic/societal level e.g. changes in societal values, customs, laws,
policies and system of governance. This level is sometimes equated with the
development of an enabling environment for civil society.

Online discussion

platform: 11 partners
participated.

A Learning
Approach
The evaluation took a
learning approach with an
iterative approach to
analysis and testing the
hypothesis.

Civil Society Challenge Fund Management

Hypothesis - Outsourced fund management
provides an effective means through grant
management and programme support
systems and processes of delivering the
Funds objectives.
ConclusionThe evaluation found the outsourced fund management arrangements for
the Civil Society Challenge Fund to be effective
although was not in a position to assess their
cost effectiveness. The Fund Manager managed
the grant cycle efficiently and provided good
quality support to Grantholders to enable them
to comply with changing monitoring and
reporting guidance.
DFIDs changing policy priorities have largely
driven the evolution of the Fund and have led
to a somewhat extractive approach to
learning. A more collaborative approach by the
Civil Society Challenge Fund with Grantholders
and Implementing Partners would have helped
to ensure the relevance and effectiveness of
Funds approach to learning and capacity
development.

Civil Society Challenge Fund Management: some key learning

While donor policy priorities will shape the priorities of fund management, the professional expertise of the

Fund Manager should be enabled to add value to the fund.

Challenge Funds require some financial flexibility subject to e.g. Treasury boundaries, to respond to the

unpredictability of projects working in challenging environments.

Diligent risk management helped weaker projects but may have inhibited learning in reporting.
Challenge Funds should continue to combine quantitative aggregate reporting with more qualitative insight

in monitoring portfolio performance.

Lessons For Challenge Funds

1) Balance donor
priorities with
partnership

2) Retain Open Calls but

minimise wasted
effort

3) Include a formal
Inception Phase

Grantholders and partners

can play a role in ensuring the
relevance and effectiveness of
the Challenge Fund support.
A Steering or Advisory
committee(s) might help
moderate the different
priorities and perspectives of
stakeholders.

Open Calls enable smaller

CSOs to play to their
strengths and avoid the
temptation of mission drift.
DFID should retain an Open
Call facility for smaller CSOs.
The grant approval process
should try to minimise
disappointment and wasted
effort at proposal stage.

The early involvement of

stakeholders in project
design helps to build
ownership and ensure the
relevance of the project. A
formal inception phase
for example, to consult on
appropriate M&E processes
and systems -would help
set the foundations of a
successful project.

Key Points of Influence

4) Focus on improved
reporting and
performance
A focus on better
reporting does not
necessarily correlate to
better programmes. A
Challenge Fund learning
strategy would ensure
capacity development and
learning are targeted to
improve project
performance as well as
reporting.

UK/Southern Civil Society Organisations - Partnerships for Change

Hypothesis - UK CSO partnerships with Southern
CSOs provide an effective means of channelling
funds to strengthen Southern civil society.
Conclusion - The evaluation found that Southern
Partners valued their partnerships with UK
Grantholders, and the role they played in
channelling funds, as long as they added value to the
project. The Evaluation identified a number of
value-adding roles that Grantholders played, as
seen in the following figure.

Indirect funding:
Grantholder roles
with Implementing
Partners

The principle role of Grantholders is to offer a level

of project oversight that would be difficult for a
UK-based Fund Manager to replicate at portfolio
level. Grantholders played less of a role than
anticipated in scaling up the influence of the project.
The evaluation noted that, as small to medium sized
Civil Society Organisations, Grantholders required
capacity development support from the Fund
Manager to meet the growing expectations of their
fiduciary role.
Understanding the role of the Grantholder
The activities of Grantholders tends to be
under-reported in Annual and Project Completion
Reports. There is no specific field in reports for the
Grantholder to report on its work in partnership with
the Southern CSO unless this is specifically included
as a logframe output such as support to the
organisational development of a partner/s.

Lessons For Challenge Funds

5) Demonstrate the added value of the UK CSO
Southern CSO partners on balance favoured the indirect nature of Civil Society Challenge Fund
funding as long as UK CSOs added value to the project. Challenge Funds supporting Southern
civil society through UK CSOs should consider how their added value can be better defined,
assessed and reported on so as to provide the rationale for indirect funding.

Capacity Development: Pathways to Change

Hypothesis - Capacity development provided
through UK NGO/ Southern CSO partnerships will
be appropriate to and effective in strengthening
civil society capacity at an individual,
organisational and systemic level.

A ChildHope UK
project on violence
against children in
Uganda illustrates
the way in which
capacity cascades via
different actors,
approaches and
target groups in this
pathway to change.

Conclusion - Most project capacity development

activities were focused at the point of delivery with
target groups, involving Collaborative Partners and
community networks. This involved a wide range of
formal and informal approaches.
The evaluation found evidence of these leading to
successful civil society engagement with decisionmakers but more guidance and support should be
offered on appropriate methodologies to gather
evidence of attitude and behaviour change as a result
of capacity development.

Three contributory factors to success

Early investment in building relationships to
establish a conducive environment for
subsequent engagement.

A clear focus helped capacity development

activities to be mutually reinforcing with
regard to project objectives even when they
involve diverse approaches and multiple
actors.

Capacity development approaches should be

targeted to the needs and learning
preferences of different groups and
adjusted, if necessary.

CSCF capacity development: some key learning

CSCF capacity development is a complex, multi-faceted process of learning and change. Measuring this
change in attitude and behaviour presents methodological challenges.
Grantees and partners need support and guidance on the better use of data collection tools e.g. o for baseline
and end-0f project assessment, in order to provide a robust evidence base for change.
Successful capacity building may need to adopt diverse approaches with different stakeholders which can be
mutually reinforcing.
The use of local staff and/or volunteers who spoke the relevant language, knew the context, and in some cases
the actors involved, was key to building trust, confidence and credibility.
Targeting both rights holders and duty bearers is a key feature of many successful projects.
Peer group learning was prioritised by partners but did not feature prominently in CSCF learning documents
and projects.
Involving beneficiaries in project planning & implementation develops capacity through learning by doing.

Civil Society Challenge Fund impact: emerging trends

Hypothesis: Improved capacities of target groups will
result in increased civil society engagement in local and
national decision-making leading to pro-poor policy and
practice changes.
Conclusion - The evaluation found considerable evidence of
capacity development leading to civil society engagement at
local and national level with decision-makers, and in policy
dialogue and development. A large number of projects
contributed to successful policy adoption although there were
fewer cases of following through on policy implementation
(not surprisingly in a three-five year funding period).
In particular, projects working in a policy niche area with a
well-defined operating environment and target population
were able to influence the professional and regulatory
frameworks to influence policy and practice.

Hospice UK: Building an enabling environment for child palliative care (CPC) in India
and Malawi
Hospice UK aimed to improve the quality of life for children living with life-limiting illness and their families and carers in
India and Malawi. The project formed a coalition of collaborating partners at different levels to influence the relevant
professional bodies, training institutions and legislation in both countries. For example, in India:

CPC services are integrated into the routine practices of 5 of 6 hospitals involved in the project.
The Indian Association of Palliative Care (IAPC) included CPC in all of its strategy documents and established a working
group to ensure that it is integrated into its on-going plans and policies.
CPC was integrated into standard training and education for health care professionals in the state.
The government of Maharashtra passed a resolution on palliative care that commits to the provision of CPC and
includes CPC in the policies of the relevant health and family Ministries and an amendment to the Narcotic Drugs and
Psychotropic Substance Act. (NDPS) Act.

By embedding CPC in national legislation; the policies of the relevant Ministries; the strategy of the national professional
association; the training of health care professionals; and the routine practices of public health services the project has
contributed to creating a enabling legal, policy, and professional environment for the delivery of child palliative care.

Lessons For Challenge Funds

Lessons for engaging with decision makers

A prior track record (in expertise/partnership) helps
kick start project activities.
Projects with a well-defined sphere of influence can
shape the enabling environment that affect specific
groups of poor and vulnerable people.
Technical expertise/advocacy skills combined with a
broader representative platform can facilitate access to
and influence with decision-makers.
Effective implementation of existing policy
commitments e.g. re. service delivery, can deliver
tangible benefits at local level when embedded in public
sector.

6) Empowerment and
advocacy still relevant.
Partners affirmed the
relevance of the Civil society
Challenge Fund to their local
context. DFID should retain a
facility to fund civil society
engagement with decision
makers, reinforced by a
public theory of change of
how an independent, vocal
civil society contributes to
pro-poor outcomes.

7) Lasting impact within

spheres of influence
A number of projects
influenced the policy,
professional and regulatory
frameworks relevant to
their targeted beneficiaries.
DFID should retain a
facility to support smaller,
more specialised UK CSOs
to promote policy/ practice
change within their area of
competence.

8) Demonstrate capacity
development as a
change process
Demonstrating the
effectiveness of diverse,
often informal, capacity
development approaches is
a challenge. A Challenge
Fund for smaller CSOs
should provide support and
guidance to Grantholders on
the use of appropriate
indicators and data
collection tools to provide a
robust evidence base for
change.

Conclusions - Relevance, Effectiveness and Sustainability

Relevance
The activities of the Fund Manager and
the projects supported by the Civil
Society Challenge Fund were generally
relevant to the Funds Objectives.

The focus on empowerment and advocacy

was relevant to strategic and programmatic

priories of Implementing Partners and needs
of beneficiaries.
The fund management approach was
responsive and relevant to donor demands,
although to a lesser extent to the needs of the
Implementing Partners.
Projects targeted poor and marginalised
populations with specific needs.
Working with local staff and community
activists familiar with local socio-cultural
context ensured that the capacity
development approaches adopted by projects
were relevant to the target populations.
A collaborative approach to working with
decision makers at the local level encouraged
ownership and contributed to relevance.

Effectiveness
The Evaluation found evidence of
projects contributing to the improved
capacity of Southern civil society to
engage with decision makers at a
local level.

The Fund was effective at reaching women

and girls through targeted approaches.
The existence of a well-defined operating
environment aligned to the focus and
expertise of the project enabled a coalition
of actors to influence decision-makers
within their sphere of influence.
National level civil society engagement
was more common at the earlier stages of
the project cycle e.g. policy development,
and to a lesser extent policy adoption.
Few projects participated in international
policy fora.

Meeting The Needs of Poor

& Marginalised
Civil Society Challenge Fund projects
supported specific needs of poor and
marginalised groups such as: TB
control, HIV/Aids, children with
terminal illness, people affected by
leprosy, mental illness and epilepsy
and gender inequality.

Sustainability
Evidence of the Civil Society Challenge
Fund contributing to sustainable
Southern CSOs was inconclusive.
However there was evidence of
outcomes at project level continuing
beyond the project.

Amongst Implementing Partners there

was evidence of increased organisational
capacity in systems and processes as well
as increased profile and credibility
associated with DFID funding.
Evidence of success in leveraging funds
was however inconclusive and
organisations were vulnerable to losing
skilled staff on project closure.
The evidence of lasting achievements as a
result of a number of the Civil Society
Challenge Fund projects was found most
likely where the project brought about
mutually reinforcing changes in the
policy, professional and regulatory
frameworks relevant to the needs of target
populations.