“SOME PEOPLE MET AND SPOKE TO EACH OTHER...

... SO WHAT?”

A PROPOSED APPROACH TO MONITORING AND

EVALUATING AN UNOFFICIAL PEACE INTERVENTION

Lucian Harriman

Post Graduate Certificate in Conflict Resolution Skills

University of Coventry – December 2008

6,318 Words
 CONTENTS

Introduction .............................................................................................................. p3

Accountability – formal and informal, external and internal .............................. p5

Learning .................................................................................................................... p9

Method and rationale ............................................................................................. p10

Constraints and challenges to implementation ................................................... p14

Investigating and understanding impact and effectiveness ................................ p21

Conclusion ............................................................................................................... p28

Bibliography ........................................................................................................... p29

2
Introduction

Monitoring and evaluation relates to understanding, justifying, accounting for and

assessing an intervention and its relationship to the context that it engages. Adopting

a framework for monitoring and evaluation can provide a systematic means of

supporting learning and accountability in the implementation of a peacebuilding

initiative.1 It requires an agency to explain a project’s methodology and its

underlying rationale. It also involves the agency planning and articulating how it will

set out to understand the impact of the intervention, and determine whether or not, or

to what extent, it has been effective or successful. This all relates to how the agency

will obtain, document, interpret and share information related to the project, and also

how information obtained during the course of the project will be used to make

decisions related to the conduct of the work.

This paper proposes an approach to the internal monitoring and evaluation of an

unofficial dialogue project which will be conducted in Sudan over two years, starting

in January 2009, by Concordis International (“Concordis”, “the agency”), a small

British NGO.2 The objective of the project will be to facilitate dialogue addressing

sticking points and challenges to the implementation of the Comprehensive Peace

Agreement of 2005, seeking to contribute to the prevention of a recurrence of large

scale armed violence, particularly in the areas of transition between Northern and

Southern Sudan. The primary component of the project will be the facilitation of a

series of low-profile consultations between influential individuals from a range of

1
OECD, 2008, p26
2
The author of this paper is the Research Manager of Concordis International.

3
Sudanese constituencies. The consultations are intended to encourage cooperative

relationships across conflict divisions whilst developing consensus on potential means

of addressing contentious issues.

The paper suggests that Concordis uses the research process that will determine how

it applies its method during the course of the project as the basis for monitoring. This

means that the collection of information described as “monitoring” will not simply be

undertaken to facilitate a retrospective evaluation. Rather, its primary purpose will be

to determine and document the decisions the agency makes regarding who it will

bring into dialogue, and on which issues. Therefore the “monitoring” process will

itself be an inherent element of the project’s implementation. It will involve planning

how information will be gathered, documented, interpreted and used to make

decisions.

Making explicit the project’s method and rationale, along with any values or

principles which are supposed to guide the work, and documenting how these have

been applied during the course of the project, particularly how they have determined

the making of specific decisions, will provide a basis for internal accountability and a

means of making the work structured and systematic, as opposed to being simply ad

hoc or intuitive. This will be especially important working in a dynamic, largely

unpredictable context, where the agency is accountable formally and informally to

numerous external actors.

Implementing the project will be far from a straightforward, predictable, technical

exercise and planning and evaluation should take into consideration potential

constraints and challenges and possible limitations of the method. It should also be

4
recognised that the project will likely face unanticipated challenges and that there may

be unanticipated effects of the work on the situation in Sudan. An overall objective of

the monitoring and evaluation process should be, therefore, to identify strengths and

limitations of the method, and lessons to inform future work. The extent to which the

project is “successful” will be identified by tracing “outwards” how interventions

have impacted upon the broader Sudanese context.

Accountability – formal and informal, external and internal

Evaluation is usually seen primarily as a means of “accountability, control or

documentation [to] find out whether an activity has been performed as intended

and/or whether the expected results have been achieved”3. This is accountability for

something; it relates to whether an organisation has done what it has said it would do

and, also, whether it has achieved what it has said it would achieve. Accountability,

and accordingly evaluation, can be understood in a short-term and functional sense

related to the use of resources and the immediate impact of a project but they can also

take a longer term and “strategic” approach, related to the broader impact of an

intervention.4

Accountability has been defined as “the means by which individuals or organisations

report to a recognized authority (or authorities) and are held responsible for their

actions.”5 This is accountability to somebody. It can be argued that an organisation is

more likely to perform well if it is held accountable and its activities are evaluated. In

3
OECD, 2008, p86
4
Conradi, 1998
5
Edwards and Hulme 1996 cited in Conradi 1998

5
turn, the way in which an organisation is held accountable, can define standards of

performance.6

There is a difference between formal, official accountability, as defined by contracts

and laws, and informal accountability, which may be defined by the relationships that

an organisation needs to sustain in order to continue to operate.7 NGOs are said to

have “multiple accountabilities” and, therefore, “[d]emonstrating accountability for

NGOs is a process of promoting, bargaining, and negotiating among the various

stakeholders in the NGO work”.8

Concordis is accountable to many different entities in both the formal and the

informal sense. What the agency is accountable for is determined by the specific

relationship.

Formally, the agency is accountable to its donors, explicitly by means of contract. It

is also accountable to governmental authorities in Sudan, with whom it will have to

sign formalised agreements, and to whom it will have to provide regular reports on

project implementation. There are rules and guidelines which Concordis has or will

undertake to follow in order to be able to participate in forums for International NGOs

in Sudan. Formalised agreements are also necessary with UN agencies who provide

transport, security and other infrastructural support in Sudan. Naturally, Concordis is

accountable to the law in all of the countries where it operates.

6
Conradi, 1998
7
IBID
8
IBID

6
The agency is also dependent on relationships with many other groups and actors,

each of which needs to be managed slightly differently. This could be described as

informal accountability. Concordis is to some extent dependent on informal

relationships with sympathetic individuals in Sudanese organisations close to

government for indigenous political support and facilitation of its activities. This

includes individuals working in governmental and quasi-non governmental

organisations in Sudan. Concordis also relies on relationships with other international

NGOs working on similar issues, in order to access information essential for the

success of its work, and also for other forms of support. Ultimately, for Concordis to

operate at all, it relies on people agreeing to participate in its consultations. There is

therefore a degree of informal accountability to participants, on whom the agency is

dependent if it is to function.

Overall, Concordis aspires to be acting in the interest of all Sudanese, or at least those

who are negatively effected by violent conflict. Whether or not there is a mechanism

of accountability to the broader population is questionable, although Concordis is

formally accountable to governing authorities. However, one could argue that out of

security necessity Concordis is informally accountable to the people located in all of

the areas where it operates. “Acceptance” is recognised as one of the main

components of NGOs’ security strategies when operating in violent environments.9

Concordis’ relationship with its neighbours and the way it is perceived as it carries out

its activities will affect the agency’s security and its overall ability to operate.

Therefore it is in Concordis’ interest to account publicly for what it is doing, in order

to allay suspicions, and maintain positive public relations. This involves being aware

9
Van Brabant, 2000

7
of how local communities or specific actors perceive Concordis, and not only to how

Concordis conveys itself to them. It involves interaction, listening and reciprocation.

In order to be able to manage so many different and complex strands of formal and

informal accountability to external actors, Concordis could benefit from ensuring that

it is first and foremost accountable to itself. Representations of the work may need to

vary in emphasis and detail when provided to external actors, but Concordis should

also have its own internal, consistent and frank understanding and narrative of the

principles which have guided its decisions and the factors which have constrained or

enabled its operations. This should be the basis from which accounts to external

parties, and external evaluators, are drawn. This paper, therefore, concerns itself with

how Concordis is accountable to itself - how it conceives and applies its method, how

it makes decisions and how it seeks itself to understand the impact of its work.10

To facilitate and substantiate this kind of internal accountability, information should

be recorded in a way that enables Concordis to account for its decisions. The agency

should record, for example, why it has chosen to approach a certain individual to

participate in a consultation. The record should note the principles that determined

the decision as well as the pieces and sources of information that informed it. This

will encourage Concordis to be systematic and principled in its decision-making and it

will enable the agency to trace and show how and why it has acted. This will form a

resource for retrospective evaluation as well as a means of disciplining decision-

making.

10
This is for ethical as well as practical or methodological purposes. There is a real and significant
pressure to represent all of one’s activities as successful, in order to assert credibility, attract funding
and continue to be able to operate; the income and reputation of an organisation and its staff depend
upon it.

8
Learning

As well as being a source of accountability, evaluation can be a means for “learning

and improvement [to] systematise knowledge of results and performance, which can

help improve [the specific project in question] or similar activities.”11 In this latter

sense it may be used to develop knowledge and theory,12 to serve “internal learning

and external sharing”.13

Developing theory involves assessing the method applied in a given intervention. A

peacebuilding project may be deemed to have failed as a consequence of poor

implementation of a method, but failure may also be a result of a misconceived

methodology. Therefore, a distinction can be made between “implementation failure”

and “theory failure”.14

The evaluation of a peacebuilding initiative should therefore seek to determine to

what extent a method is being consistently applied, as well as whether the applied

method is appropriate.15 Articulating the methodology to be implemented during a

peacebuilding initiative is necessary to ensure accountability and consistency in

implementation, determining if the agency has done what it has said it would do.

Explaining the underlying rationale of the method is necessary to justify why it is

being used in a specific context. Once the rationale is articulated, one can observe

11
OECD, 2008, p86
12
Lederach et al, 2007, p2
13
Church and Rogers, 2006, p179
14
OECD, 2008, p40
15
Conradi, 1998

9
and compare whether in practice the project has had the effect that was anticipated

and if the logic behind the method was accurately conceived.

Method and rationale

Concordis’ method involves organising unofficial discussions between individuals

from the various constituencies involved in an ongoing, or potential conflict. These

discussions, called “consultations” can include up to approximately 30 participants

and last up to five days. They are intended to take place out of the public and media

eye and are supposed to be attended by senior individuals, who can influence policy,

invited in a personal capacity in order to relieve them from pressure to maintain a

particular party line. Social interaction is encouraged during the course of a

consultation and participants, as much as possible, are given equal status within the

consultation’s ‘microcosm’. Participants, well connected with their respective

leaders, and respected within their constituencies, are seen to have the potential of

“building bridges” across conflict lines on the basis of shared interest. The

atmosphere of the consultations is intended to encourage trust and develop personal

relationships across conflict or social divisions, both of which are hoped to contribute

to the success of negotiations and more general peaceful relations.

Concordis’ method and rationale correspond to its theoretical understanding of the

dynamics of conflict. Three overarching analytical perspectives from which conflict

and violence tend to be understood, as delineated by Tilly,16 help to inform the

agency’s approach. One perspective understands social activity, and by extension

violence and conflict, as driven by self-interested responses to incentives. Another is

16
Tilly, 2000

10
concerned with the role of ideas and perceptions in causing violence, and the third

conceives violence and conflict as a consequence of relationships and social

structures. The latter, relational perspective emphasises that both incentive structures

and perceptions are shaped by and exist in the context of social relations. “In this

view, restraining violence depends less on destroying bad ideas, eliminating

opportunities, or suppressing impulses than on transforming relations among persons

and groups”.17

Concordis explicitly takes such a relational approach and it uses the definition of

peace, and peacebuilding, as “the facilitation of non-exploitative, sustainable and

inclusive relationships free from direct and indirect violence and the threat of such

violence”.18 The agency recognises nevertheless that a relational approach to

reducing violence depends to some extent on changing perceptions and on creating

incentives, or at least facilitating the perception of incentives, for parties in conflict to

change their individual relationships and for change in broader structures of social

relations to be possible.

Lederach et al, identify four dimensions of conflict related change: the personal

dimension, which refers to individuals’ attitudes and behaviours; the relational

dimension concerns patterns of interpersonal communication and perceptions between

individuals and communities; the structural dimension concerning how conflict

“impacts systems and structures—how relationships are organized, and who has

access to power”; and the cultural dimension, which relates to “the norms that guide

17
Tilly, 2000
18
Mac Ginty, 2006, p10

11
patterns of behaviour”.19 Curle indicates one way in which these dimensions might be

connected, explaining violence as a response to a sense of social and personal

disconnection and an accompanying “alienation from our common humanity”.20 This

understands personal and relational dimensions of conflict to be largely a

consequence of social and cultural systems and structures.

Concordis’ approach responds explicitly to the personal, relational and structural

dimensions of conflict related change. The agency facilitates a process designed to

encourage personal reflection and changes in participants’ interpersonal relations. At

the same time, it engages participants in analytical discussions concerning how to

facilitate structural change in their society. Participants are selected on the basis of

their position within social structures, and it is intended that any potential changes in

relationships and attitudes developed during consultations will be “transferred” into

the working practices and into the explicit policies of their broader groups and

institutions.

The method corresponds very closely, if not identically, to “Interactive Conflict

Resolution” as described by Ronald Fisher.21 The method also draws on Lederach’s

concept of “levels of leadership” adopting a “middle-out approach” seeking to engage

a “set of leaders with a determinant location in the conflict who, if integrated

properly, might provide the key to creating an infrastructure for achieving and

sustaining peace.”22

19
Lederach et al, 2007, p17
20
Curle, 1995, p54
21
Fisher, 2005, pp2-3
22
Lederach, 1997, ch4

12
In its consultations, Concordis seeks to apply thinking about the dynamics of

interpersonal relationships as developed by the Relationships Foundation, which

argues that developing and experiencing “relational proximity” in five domains,

creates an enhanced quality of “relational experience”, which can contribute to

“outcomes such as trust, understanding, support, accountability or belonging”.23 To

elaborate, in the domain of communication, proximity in directness – “reducing the

extent to which presence is mediated or filtered” – is said to produce a relational

experience characterised by connectedness. In the domain of time, proximity is

achieved via continuity, which is said to produce a relational experience of shared

story. In the domains of information and knowledge, proximity is characterised by

multiplexity, a term used to describe the “breadth and quality of information” shared

between the parties, which allegedly leads to an experience of being known and

having a mutual understanding. In the domain of power, proximity is achieved

through parity, which is said to be the “fair use of power”, said to produce the

experience of mutual respect. Finally in the domain of purpose, proximity is

characterised by commonality, the “building of shared purpose”, which can create the

experiences of synergy and unity.24

Concordis sets out to encourage “relational proximity” amongst the participants in its

consultations by framing discussions towards a common purpose, establishing

continuity of participation is a series of meetings over time, and providing a different

environment to usual for participants to interact, face to face, not only during

facilitated discussions but also during opportunities outside of the sessions to speak

and socialise. The agency seeks to produce “parity” during consultations by

23
Ashcroft et al, forthcoming
24
IBID

13
according all participants equal opportunities to contribute, regardless of their relative

social positions. This is partially achieved by inviting participants to attend in their

personal capacities and also by asking participants not to attribute statements made

during consultations to specific individuals or parties.

Constraints and challenges to implementation

Any realistic attempt to evaluate the Peace Building Initiative should recognise that

implementing the method described above will not be a straightforward, technical,

predictable process. The project’s character and success, and whether it is even

possible for it to begin to be implemented, will all be contingent on the extent to

which Concordis is able to negotiate and manage a number of challenges, many of

which will be beyond the agency’s control. As much as possible, these challenges

should be anticipated in plans to conduct and monitor the project, and considered in

any evaluation.

Authorization of the process by government

For the project to go ahead at all, it will need to be authorised, politically accepted and

not obstructed by the governments in North and South Sudan. Concordis has

succeeded in registering as an NGO with the Southern Sudan Relief and

Rehabilitation Commission (SSRRC) and with the Ministry of Humanitarian Affairs

in the North. However, a lengthy bureaucratic process remains to be completed with

the Government in the North, including signing a Technical Agreement and obtaining

work and travel permits. Political obstruction and bureaucratic delays could severely

slow down, disrupt or prevent project implementation. In the South, although

14
Concordis has registered with the SSRRC, it is not clear whether further registration is

necessary and if so which ministry should be approached.

The challenge for Concordis in negotiating these issues will be to represent and

describe to government authorities its planned work in a manner that will not arouse

suspicion and will encourage them to allow it to happen. This is not to suggest that

the nature of the project be misrepresented, rather that elements of the project which

would be most appealing to the respective authorities be emphasised as it is described

to them. A further challenge will be to manage effectively and appropriately

Concordis’ relationships with the relevant individuals within the government

institutions in question. The design, documentation and evaluation of the project

should take these challenges into account and record the approaches that Concordis

takes and the responses of the respective authorities and how this has affected activity.

Recording this will enable the agency to account for how it has negotiated

government constraints.

Acceptance of the agency in its role as facilitator

Although it has nine years of experience in doing so, Concordis should not assume

that it is automatically entitled to mediate or facilitate in the Sudanese context. The

agency will need to establish, or re-establish, relations and credibility and earn the

acceptance and trust of the parties it wishes to engage. Influential individuals will

have many demands on their time and they live in a politically tense environment,

where there is a history of suspicion, animosity, violence and loss. The project will be

trying to addresses politically sensitive issues. Concordis will need to establish

interest in and acceptance of the process, and respond to potential participants’

15
questions and concerns, encouraging them to participate themselves. The agency has

existing contacts and relationships with a number of influential individuals from

relevant Sudanese civil society groups, political parties, armed movements and other

individuals, some of whom participated in past series of consultations. Former

participants and other contacts will be approached and included in the forthcoming

initiative. However, Concordis’ network will still need to develop new relationships.

The processes of identifying relevant actors, approaching them and establishing or

resuming bi-lateral relationships with them should be recorded and considered in the

documentation and evaluation of the project, not only so Concordis can account for

how this has been conducted but also because these meetings will be a source of

initial, ‘baseline’ data regarding the various parties’ perspectives and positions.

Consent to the guiding principles of the process

The key characteristics of Concordis’ approach to peacebuilding are articulated in the

section on method and rationale above. It should be recognised, however, that the

agency has no real power to enforce the principles of the process either inside or

outside of the facilitated meetings. Participants’ consent is required for the process to

go ahead according to the terms that Concordis intends. The agency cannot guarantee

to participants that others will uphold the rules of the process. They will need to be

confident themselves that this will be the case, or, at least, be prepared to accept the

risk that the rules guiding the meetings might not be upheld.

Non-attribution of statements

16
A good example of a rule exposed to such a risk is the principle of non-attribution of

statements made “off the record” during the course of consultations. Individuals are

invited to attend consultations in their personal capacity and discussions are supposed

to be conducted under the Chatham House Rule.25 This is supposed to provide

participants with flexibility to explore ideas and positions that they might not be able

to in a highly pressurised public discussion. However, Concordis is not able to

guarantee that participants in the meeting will not report publicly or attribute what

they have heard said in the meetings. The agency can express the intention that

statements are not attributed, and it can undertake to refrain from attribution itself, but

ultimately the broader implementation of this rule rests with participants themselves.

Participants will need to trust Concordis, and more importantly each other, to respect

the principles of the process. Concordis will need to ensure that all participants

understand what the principles are, but also that participants understand the limits of

Concordis’ ability to enforce them, beyond its own conduct as an organisation. If a

participant feels that they are at risk of being reported and facing negative

consequences for a particular statement, he or she may refrain from articulating it in

the consultation environment.26 This is in a context where arbitrary arrest is a

systematic practice of the national security services.27 How Concordis conveys the

principles of the process and its limited means of enforcement, as well as how

participants react to this, should be documented and reflected upon in the evaluation

of the project.

25
“When a meeting, or part thereof, is held under the Chatham House Rule, participants are free to
use the information received, but neither the identity nor the affiliation of the speaker(s), nor that of
any other participant, may be revealed” - http://www.chathamhouse.org.uk/about/chathamhouserule/
26
This was raised to a member of Concordis staff by a participant of a past consultation, who said that
the presence of members of state security meant that participants were cautious about what they said
for fear of potential difficulties in the future.
27
UNHCHR, 2008

17
Inclusion and exclusion

Concordis will need to restrict the number of participants in each consultation, not

least for practical reasons. However, the agency may encounter pressure to include

individuals or groups who it might not have determined to be relevant or suitable.

Furthermore, Concordis may be unaware of, or choose not to engage, certain groups

or constituencies who have become aware of the process and may wish to participate.

A challenge will be to limit participation without alienating some constituencies or

appearing biased. Those denied the opportunity to participate may become frustrated,

suspicious or even hostile to the process. This could pose a risk to the organisation’s

reputation, and pose potential political difficulties in conducting the work. At the

furthest end of the spectrum it could pose security risks both to Concordis and to

others participating in the initiative. Planning, documentation and evaluation of the

process should address how Concordis will deal and has dealt with this issue and what

the implications are for the project’s implementation and impact.

Maintaining a low-profile

A related challenge is that of managing and minimising publicity associated with

consultations to maintain the low-profile nature of the initiative. Deliberately

minimising publicity or public awareness of the process may arouse suspicion and

compromise some people’s acceptance of the project’s legitimacy. Again, how this

issue is managed and the implications of the approach taken, should be addressed in

project planning, documentation and evaluation.

18
Material and political pressures on participants

Concordis’ consultations seek to provide a neutral, non-pressurised environment for

participants to explore rationally and analytically their common challenges and

consider potential non-violent solutions to their problems. However, even if such an

environment is achieved, it does not eliminate the real material, political, social and

cultural pressures experienced by participants outside of the meetings. It is worth

noting that participants’ social status, political influence and means of livelihood may

depend upon their position within systems of patronage or their conformity to the

political agendas of their benefactors or constituencies. Therefore one should not

underestimate the significance of such pressures in shaping participants’ decisions.

This may constrain their opportunities to promote, implement or otherwise “transfer”

what they may have agreed to be an appropriate policy within the confines of the

consultations.

Producing parity

A central component of Concordis’ relational approach is that it seeks to promote

parity between participants; that they are able to engage with each other on equal

terms. This is likely to be uncharacteristic of their relationships outside of the

consultations. Within consultations Concordis may be able to afford equal

opportunities to participants to express their views and engage in discussions.

However, there will be power dynamics between participants that Concordis

nevertheless will be unable to redress and some of these may be unknown to the

19
agency. This may relate to potential threats that some participants could pose to

others. Concordis should seek to identify such power-dynamics within its

consultations, if it is to get a sense of the nature of the interaction that it is facilitating.

Such information will likely need to be conveyed by participants themselves, although

they may not feel secure enough to express it openly or even to Concordis on the

promise of confidentiality.

Safety and security of staff and participants

As an overall goal, the project seeks to contribute improving of the security situation

in Sudan, or, at least, preventing its deterioration. Ensuring staff security may on

some occasions limit where staff are able to travel and therefore reduce their access to

some groups or individuals whom the project should ideally be seeking to engage.

The documentation and evaluation of the project should convey if and how the

agency’s security policy has affected the implementation of the project.

Ensuring the security of participants is a different challenge, arguably more difficult.

Concordis should try to ensure that it does not expose participants or others to security

threats that they would otherwise not incur. Partially this relates to how the initiative

is perceived and the implications for those involved. It also relates to the choice of

locations of consultations and the means of transport that participants use to get to

them. It may also relate to how participants understand the information they disclose

in consultations will be used. There is a risk that consultations be used as a

surveillance exercise by some parties or even as opportunities by some groups to

attack specific participants. Concordis should at the very least establish a system

whereby it seeks to prevent the disclosure of any information it has recorded that may

20
make participants vulnerable. This should be documented and considered in

evaluation.

Potential lessons and overall limitations

The discussion of these challenges provides an insight into some of the complexities

of implementing the method. The monitoring, documentation and any evaluation of

the project should recognise these complexities, and identify how Concordis has

approached them, to what effect.

All of these challenges indicate the limitations and fragility of the process that

Concordis will be offering. They highlight a defining characteristic of the planned

intervention, that it seeks to provide an opportunity for something potentially

constructive, but it cannot guarantee how it will be used by those engaged.

An outcome of an evaluation of the project may be to draw lessons about how the

agency was able to negotiate challenges successfully and which elements of

Concordis’ approach were problematic or impossible to implement. Monitoring and

evaluation should also identify unanticipated challenges which arose during the

course of the project. This may produce some generally applicable lessons about

how to manage such challenges and it may also point to elements of the organisation’s

methodology which might need rethinking or further development.

Investigating and understanding impact and effectiveness

21
Whereas, the section above has described some of the challenges to the

implementation of Concordis’ method, the following section addresses how the

agency should investigate and indentify the impact of its activities.

There a number of challenges to identifying the impact of a peacebuilding

intervention. Relationships and trust are difficult to measure objectively or

quantitatively,28 primarily because they are based in intangible in attitudes and

perceptions.29 Furthermore, “[v]arious actors may have diverse or even contradictory

interpretations of an intervention’s impacts (positive/negative) or relevance, based

upon their own position within the conflict.”30

Peacebuilding interventions take place in an “open system” with a broad range of

factors and actors effecting the context. Therefore, the context is dynamic. “Change

in the conflict patterns occurs within a changing environment”.31 It has been

suggested that in peacebuilding, long-term “[s]trategic accountability is nearly

impossible in part because NGOs operate in contexts in which so many factors are

beyond their control.”32 This relates to the challenge of determining causality between

an intervention’s “inputs” and a broader change in context. For work intended to

prevent violence there is the additional challenge of proving the counterfactual

situation that would have occurred if an intervention had not taken place.33 Such

challenges make it all the more necessary to be clear about how an agency will seek to

understand the impact of what it is doing.

28
Lederach et al, 2007, p2
29
Church and Shouldice, 2002, p2
30
OECD, 2008, p38
31
Lederach, 1997, p136
32
Conradi, 1998
33
Lederach et al, 2007, p2

22
When seeking to determine how an intervention has affected a situation, one can look

at a number of different levels of analysis, each a further degree of separation away

from the implemented activities. Investigations at the “micro” level relate to how

individual participants’ positions, perceptions or actions have changed as a result of

the activity. Impact at the “meso” level refers to how participants’ reference groups

and constituencies have been affected. Identifying impact at the “macro” level relates

to seeing how the intervention has affected the broader society and context.34

A sensible approach to investigating the impact of Concordis’ consultations would be

to start at project implementation and the “micro” level and investigate “outwards”,

seeking to trace how the “meso” and “macro” levels have been affected. It should

also be recognised that changes in the broader context will affect the possibilities and

priorities for project implementation and the characteristics of change at the “mico”

and “meso” levels.35 Therefore, it is necessary also to monitor the “macro” level

situation and try to trace how that affects the situation “inwards”.

A convention in evaluation is to establish a “baseline” prior to an intervention in order

to determine how a situation has changed following the activity. This can be done at

the “micro” level by “pre-testing” the attitudes of participants and at the “macro”

level by analysing the characteristics of the context prior to an intervention.36

For Concordis’ Peace Building Initiative “establishing a baseline” and investigating

and monitoring the overall context, as well as the project’s implementation and

outcomes, will all be integral elements of the implementation and development of the

34
USIP, 2004
35
Lederach et al, 2007, p59
36
USIP, 2004

23
work. They will be necessary to determine how the agency decides to apply its

method, and so, more than just being part of a retrospective assessment of the project,

these activities will actually determine the direction the work takes.

Concordis will carry out a research process to determine what should be the specific

issues addressed in the consultations. The agency will also need to investigate and

identify which individuals would be relevant and significant participants in the

dialogue process.37 Bilateral engagement with potential participants will be necessary

to establish, or re-establish, credibility with them and also to secure their interest and

commitment to participating in consultations. These initial phases of the project’s

implementation will be useful in collecting “baseline” data, but they would be

necessary in any case.

The initial thematic development and participant identification process will involve a

review of relevant analytical literature and a series of consultations with individuals in

donor, diplomatic, peacebuilding, analytical and other relevant communities. These

will contribute to the project’s “baseline” analysis of the overall “macro” context.

Identification of potential participants will also include a review of Concordis’

archive and database in order to identify with whom the agency already has a

relationship.

The “baseline” at the “micro” and “meso” levels - concerning the positions,

perceptions and actions of specific participants and their constituencies - will be

developed during initial bi-lateral discussions prior to consultations, as well as from

the statements they make during the first consultation.

37
This process is already underway.

24
Decisions regarding who should participate in subsequent consultations and which

issues should be their focus, will be based on conclusions articulated by participants

during consultations, further bi-lateral discussions with participants and others, and on

analysis of the overall context and political priorities in Sudan.

Over the course of the project, data for “micro” and “meso” analysis will primarily

come from meetings and interviews with participants, before, during and after

consultations. These will also inform context analysis, but context analysis will also

be informed by consultations with other international actors and by reviewing relevant

media and publications.

As mentioned above, the primary reason for monitoring and analysing the context,

will be to inform decisions about the conduct of the project. The primary logic for

recording this information, will be to discipline decisions and ensure a record of how

decisions have been made. Records from specific meetings, and records of relevant

points of reviewed literature, will be stored in a way in which Concordis can retrieve

them in order to provide evidence of the process and sources which informed its

decisions. This information will at the same time provide a means for the agency to

attempt to determine the relationship between the project and the context, both

whether the project is affecting the context, and, how the context is affecting the

project.

Identifying success

25
Rather than seeing “success” as something absolute, it would seem more useful and

appropriate to understand success in terms of degrees or increments. “Success is an

arbitrary determination of progress and can be set at any point along the continuum in the

desired direction of change.”38 Considering success in terms of the extent of a project’s

impact enables an observer to identify its achievements and its limitations, possibly

tracing which factors constrained the work. This can help draw lessons about the scope

and overall utility of the method in question.

A degree of success could be attributed to the project’s activities simply being conducted

as planned. If consultations happen and are felt to have conformed to recognised

principles of good practice, this could be identified as evidence of successful, or at least

“correct” implementation.39

However, this would not verify whether consultations had produced what was intended.

As mentioned above, Concordis broadly hopes to build relationships and develop

consensus. Observations of proceedings, any concluding statements, and participants’

reflections and feedback on consultations could all shed light on whether, within the

meetings, either of these objectives were achieved.

This still, however, would not indicate whether the project had made a difference at the

broader level of society. As noted above there are different types of conflict related

change which Concordis’ seeks to induce. The agency should therefore investigate if any

of these types of changes have taken place, at which levels, and if they can be attributed

to the intervention. As mentioned above, this could be achieved by working “outwards”

from the project, starting with observations of proceedings and discussions with

38
Church and Rogers, 2006, p12
39
Berry Associates, 2006

26
participants, and investigating how a broader impact has followed, if at all. This could be

described as tracing “transfer”: the transmission of developments generated in the

unofficial sphere of the intervention into official negotiations and agreements, which can

includes “effects” of unofficial processes such as “attitudinal changes” and “new

realizations” as well as “outcomes” such as “frameworks for negotiation” or “principles

for resolution”.40

Transfer may be achieved via participants’ communication to their broader constituencies

of new insights and understandings developed during unofficial consultations; the

development of groups of individuals who, having participated in unofficial

consultations, become willing to engage constructively in official negotiations, and,

following on from this; the introduction into official negotiations of proposals and

frameworks for resolution which were developed in the more flexible context of informal

consultations. These achievements would on changes in attitudes and perceptions made

possible in unofficial contexts, which may facilitate “more open and accurate

communication, more accurate and differentiated perceptions and images, increased trust

and a cooperative orientation”.41 According to this model, transfer can occur across

“decision and policy making interfaces” between leaders, official and unofficial

diplomats, negotiators, governmental-bureaucratic constituencies and public-political

constituencies.42 It will be up to Concordis to trace such processes, and identify, if they

exist, the extent of their impact.

However, the notion of “success” should not be limited to the project’s influence on

official negotiations or agreements. One could more broadly investigate the project’s

“peace impact”: how it supports “sustainable structures and processes which

40
Fisher, 2005, p3
41
IBID, p4
42
IBID, p6

27
strengthen the prospects of peaceful coexistence and decrease the likelihood of the

outbreak, occurrence/recurrence or continuation of violent conflict”. One could also

assess it in terms of its “violence impact”: how it effects the likelihood that conflict

will be dealt with violently.43

Conclusion

This paper has set out some parameters for how its author intends to conduct and

comprehend his and his colleagues’ work over the coming two years. It is a proposal,

the final approach will have to be agreed by his colleagues and be workable in

practice. Producing systematic and consistent documentation of how Concordis has

made decisions, including the principles and evidence that have informed them, it is

hoped will provide an internal resource that should be useful for the purposes of

internal, or external, evaluation of the Peace Building Initiative. Evaluation could

assess to what extent the agency was able to implement its methodology, how it dealt

with anticipated and unanticipated challenges and how its work impacted on the

situation in Sudan. Evaluation could also produce lessons about the limits of

Concordis’ methodology and the validity of its underlying rationale.

43
Fisher et al, 2000, p162

28
 BIBLIOGRAPHY

John Ashcroft and others, The Relational Lens: Focussing on Solutions, (forthcoming).

Geoff Berry Associates, Evaluation of WMTTS Mediation Project in Birmingham, May 2006.

Koenraad Van Brabant, Operational Security Management in Violent Environments,

Humanitarian Practice Network, Overseas Development Institute, June 2000.

Cheyanne Church and Mark M. Rogers, Designing for Results: Integrating Monitoring and
Evaluation in Conflict Transformation Programs, Search for common Ground, 2006.

Cheyanne Church and Julie Shouldice, The Evaluation of Conflict Resolution Interventions:
Framing the State of Play, INCORE International Conflict Research
(University of Ulster & The United Nations University), 2002.

Cheyanne Church and Julie Shouldice, The Evaluation of Conflict Resolution Interventions,
Part II: Emerging Practice and Theory, INCORE International Conflict Research (University
of Ulster & The United Nations University), 2003.

Loramy C. Conradi, Accounting for Success?: Improving Performance Evaluation and

Accountability of Non-Governmental Organizations Engaged in Conflict Resolution,
International Studies Association March 18-21, 1998.

Adam Curle, Another Way: Positive Response to Contemporary Violence Oxford: Jon
Carpenter, 1995.

Ronald J Fisher, ed., Paving the Way: Contributions of Interactive Conflict Resolution to
Peacemaking. Lanham, Maryland: Lexington Books, 2005.

Simon Fisher et al, Working with Conflict. Skills and Strategies for Action. London, Zed
Books 2000.

John Paul Lederach, Building Peace: Sustainable Reconciliation in Divided Societies

Washington DC: US Institute of Peace (1997)

John Paul Lederach, Reina Neufeldt & Hal Culbertson, eds. Reflective Peacebuilding: A
Planning, Monitoring and Learning Toolkit, The Joan B. Kroc Institute for International
Peace Studies, University of Notre Dame, 2007.

Organisation for Economic Cooperation and Development (OECD), Guidance on Evaluating

Conflict Prevention and Peacebuilding Activities – Working Draft for Application Period,
Development Assistance Committee, 2008.

R. Mac Ginty, No war, No Peace. The Rejuvenation of Stalled Peace Processes and Peace
Accords. Houndmills and New York: Palgrave Macmillan, 2006.

C. Tilly, ‘Violence viewed and reviewed’, Social Research, Vol. 67, No. 3, 2000.

UNHCHR, Tenth periodic report of the United Nations High Commissioner for Human
Rights on the situation of human rights in the Sudan. Arbitrary arrest and detention
committed by national security, military and police, 28 November 2008.

United States Institute for Peace, What Works? Evaluating Interfaith Dialogue Programs,
USIP Special Report 123, July 2004.

29