You are on page 1of 3

This article is about evaluation use.

It focuses on the well-known paradox that evaluation is


undertaken to improve policy, but in fact rarely does so.
The key argument is that in order to explain all
types of evaluation uses, including non-use and justificatory uses, the focus needs to be on
the
evaluating organization and its conditioning factors, rather than the evaluation itself.

Administrators, politicians and citizens


have an interest in knowing to what extent evaluations are used to improve policies.
Despite
this great interest in evaluation use, it is well described that evaluations rarely do
change policies
First, the article argues that the paradox described above is inherent in the nature of
evaluation
and that this has ramifications for our understanding of types of evaluation uses such as
legitimizing- and symbolic use as well as non-use.
Second, the article argues that the literature on evaluation use insufficiently addresses
evaluations
organizational context and therefore is unable to resolve the paradox.
Finally, to remedy this gap in the literature concerning the lack of contextual
explanations
of evaluation use, the article argues along the lines of Dahler-Larsen (2012) and
Sanderson
(2000) that it would be beneficial to complement core assumptions of causality and
rationality
with a theory of organization. A model is proposed that: 1) explains evaluation use by
factors
that are contextual to the evaluating organization rather than contextual to the
evaluation; and
2) integrates non-use (as well as justificatory use types) to resolve the paradox of nonuse
explained above.
A definition of evaluation
use is provided by Johnson et al. (2009: 378): any application of evaluation processes,
products, or findings to produce an effect.
The literature on evaluation use converged early on four main types of evaluation use:
instrumental-, conceptual-, process- and symbolic use.
They
recognized that process use (use during the evaluation process) could be instrumental,
conceptual
and legitimizing (e.g. evaluation is legitimizing the organization). Moreover, the
evaluation
result (typically an evaluation report), could also be used instrumentally, conceptually
and
symbolically (e.g. evaluation legitimizes the evaluated programme). In this article,
legitimizingand symbolic use are referred to as justificatory uses.

The oftenstated
purpose and idea behind evaluation is rooted in assumptions of rationality and causality.
Evaluation is commonly understood as a tool informing policy-makers and civil servants
of what works and what does not.
the
actor is perceived to be rational and in pursuit of goal-attainment and utility-maximizing
behavioural patterns of action (Sanderson, 2000). These assumptions contain an
underlying
positive and evolutionary assumption of progress and betterment, whether of the
bureaucracy
or the policy.
Evaluation itself was born in a time of belief in a better world through rational
interventions
and social engineering (Vedung, 2010) and therefore evaluation is inherently rationalist,
causal and evolutionary in nature.
Evaluation is
commonly defined as a systematic inquiry leading to judgements about program (or
organisation)
merit, worth, and significance, and support for program (or organisational) decision
making
Ideally, evaluation improves policy through the instrumental application
of an evaluations results (conclusions and recommendations) to improve policy.
However,
evaluation does not deliver on its promise to inform policy-makers.
It is very hard to argue that symbolic
and legitimizing use of evaluation lead to social betterment; for example, water quality
is not
improved because an aid organization uses evaluation to legitimize itself. The argument
of
rationality would need to be stretched very far to argue that symbolic and legitimizing
evaluation
use can actually improve living conditions for human beings (i.e. social betterment).
The problem might be that the evaluation use literature often focuses on the evaluation
itself its implementation, its outputs, its conditioning factors, etc. and largely ignores
the
organizational context. It is
argued that an organizations concerns of its external legitimacy are likely to have
priority
over evaluation findings and thus in certain cases lead to non-use of these findings no
matter
how relevant they might be to the organization.
A large number of later studies observed that organizations act
contrary to the rational utility-maximizing behaviour that is expected and contrary to
the
objectives organizations set themselves. Instead, organizations mostly act according to
norms

and values in their environment in order to legitimize themselves. Organizations have a


need to legitimize themselves in order to survive as organizations.
Evaluations are almost
exclusively embedded in organizational contexts one way or the other. Typically,
evaluations
are procured by organizations, carried out by organizations (enterprises or teams of
individuals),
may be read and acted upon by a third organization, etc.
Definitie institutii - First, institutions should
not be confused with organizations. Rather they are the frames for action of
organizations or,
simply put, the taken-for-granted formal and informal rules of the game. From this
definition it is important to stress that institutions are resilient to change and that they
consist of cultured-cognitive, normative,
and regulative elements that provide meaning to social life
While the institution of evaluation is commonly believed to produce instrumental and
rational outcomes (Sanderson, 2000), it is also a result of the legitimacy-seeking
behaviour of
organizations (Dahler-Larsen, 2006, 2012). Hence, according to institutional theory, the
institutionalization
of evaluation depends on the pressure from the field.

External pressure
The core idea of organizational institutionalism is that organizations are fundamentally
concerned
with securing legitimacy by meeting societal expectations about their structure,
practices,
rhetoric or outputs (DiMaggio and Powell, 1991). Organizations do this in order to reduce
uncertainty and stabilize social relations