American Journal of Evaluation http://aje.sagepub.

com/

Dealing With Complex Causality in Realist Synthesis : The Promise of Qualitative Comparative Analysis
Fritz Sager and Cline Andereggen American Journal of Evaluation 2012 33: 60 originally published online 14 June 2011 DOI: 10.1177/1098214011411574 The online version of this article can be found at: http://aje.sagepub.com/content/33/1/60

Published by:
http://www.sagepublications.com

On behalf of:

American Evaluation Association

Additional services and information for American Journal of Evaluation can be found at: Email Alerts: http://aje.sagepub.com/cgi/alerts Subscriptions: http://aje.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations: http://aje.sagepub.com/content/33/1/60.refs.html

>> Version of Record - Feb 1, 2012 OnlineFirst Version of Record - Jun 14, 2011 What is This?

Downloaded from aje.sagepub.com by guest on July 3, 2013

Dealing With Complex Causality in Realist Synthesis: The Promise of Qualitative Comparative Analysis
line Andereggen2 Fritz Sager1 and Ce

American Journal of Evaluation 33(1) 60-78 The Author(s) 2012 Reprints and permission: sagepub.com/journalsPermissions.nav DOI: 10.1177/1098214011411574 http://aje.sagepub.com

Abstract In this article, the authors state two arguments: first, that the four categories of context, politics, polity, and policy make an adequate framework for systematic review being both exhaustive and parsimonious; second, that the method of qualitative comparative analysis (QCA) is an appropriate methodical approach for gaining realistic results that are useful for political practice. Instead of providing unsatisfactory monocausal explanations, the approach identifies different combinations of conditions leading to a given outcome. The authors illustrate their points with a two-step multi-value QCA (mvQCA) of 17 transport policy cases in Switzerland.

Keywords realist synthesis, evidence-based policy, complex causality, multivalue qualitative comparative analysis, two-step qualitative comparative analysis, transport policy

Systematic Review and Complex Causality

The evidence-based policy movement aims at improving government effectiveness by developing and utilizing a more rigorous base of information and scientific evidence to guide decisions about program design and implementation (Heinrich, 2007, p. 255). As Pawson (2002a, p. 157) states, The policy cycle revolves quicker than the research cycle, with the result that real time evaluations often have little influence on policy making. As a result, the quest for evidence-based policy has turned increasingly to systematic reviews of the results of previous inquiries in the relevant policy domain. However, it is a generally acknowledged problem of systematic review that traditional statistical analysis leads to unsatisfying results for the political decision-makers. It is argued that the

1 2

Center of Competence for Public Management, University of Bern, Bern, Switzerland Federal Parliamentary Control of the Administration, Bern, Switzerland

Corresponding Author: Fritz Sager, Center of Competence for Public Management at the University of Bern, Schanzeneckstrasse 1, P.O. Box 8573, CH-3001 Bern, Switzerland Email: fritz.sager@kpm.unibe.ch

Downloaded from aje.sagepub.com by guest on July 3, 2013

Sager and Andereggen

61

established correlations between variables cannot be understood in a causal manner without considering the singularity and wholeness of each case at hand (Pawson, 2002a; Simons, 2004). Pawson (2002a, p. 163) notes that the problem with this way of arriving at conspectus is that it squeezes out vital explanatory content about the programs in action, in a way that renders the comparisons much less rigorous than the arithmetic appears on paper. To capture this wholeness, a scheme is needed that embraces in an exhaustive manner the various factors which influence policy making. In order to empirically mirror the wholeness of the cases in the findings of systematic review, a method is required which is able to deal with complex causality, that is, which does not focus on monocausal explanations. In the following, we seek to address these two needs in a coherent manner. As for the analytical framework, various studies refer to the fundamental trichotomy of governance: polity, politics, and policy (Sager, 2007a; Treib, Ba hr, & Falkner, 2007). However, these categories neglect context which Pawson and Tilley (1997) have shown to be a crucial antecedent for realistic evaluations. Accordingly, we develop a comprehensive scheme integrating both the context and the three dimensions of governance. As for the adequate method, Befani, Ledermann, and Sager (2007) have identified the method of qualitative comparative analysis (QCA). This method shares the same logic as the approach of realistic evaluation (Pawson & Tilley, 1997) in that it presumes that social phenomena have multiple and conjunctural causes. Causal complexity is thus fundamental to QCA. We present a two-step QCA to capture the difference between context or remote factors and proximate conditions. We then apply this analytical and methodical design to the systematic review of 17 transport policy evaluations in Switzerland. In the next section, we will develop our comprehensive framework. In the section that follows, we will present our method in more depth and specifically discuss its suitability for realist synthesis. In the section Illustration: The Acceptance of Transport Planning Projects in Switzerland, we illustrate our proposition by the empirical example of Swiss transport planning. In the concluding section, we both discuss our design and summarize the main findings from our empirical illustration.

A Realist Synthesis Framework of Analysis

The realistic approach to evaluation (Pawson, 2002b; Pawson & Tilley, 1997) emphasizes the notion that, depending on different contexts, a political program can lead to different outcomes. This gives rise to different context-mechanism-outcome configurations, briefly referred to as CMO configurations. Realistic evaluation is about conceiving and testing these CMO configurations in order to provide insight on what works for whom and under what circumstances (Van der Knaap, Leeuw, Bogaerts, & Nijssen, 2008). Accordingly, not only do we have to conceptualize mechanisms triggered by policies but also contexts under which this happens as well as the respective outcomes. Realist synthesis adopts this logic for systematic review (Pawson, 2002b). As for the selection of mechanisms triggered by public policies, we refer to Treib et al. (2007) who define governance by the three dimensions of politics, polity, and policy. First, governance has a politics dimension, which includes player constellations, power, and conflicts in the political process. Second, the polity dimension of governance concerns the structures and rules that influence the players. Finally, the policy dimension refers to the instruments and content of policies. Dividing governance into the three dimensions of political action can serve as a useful framework for policy analysis (Sager, 2007a). Indeed, a political measure can be successful if it is planned and implemented within particularly suitable structures. Another factor for success may be that the content of the project is convincing. Finally, a political measure may be successful, if there is no significant conflict or opposition during its planning and realization. However, a policy-planning and decision-making process always takes place in a specific context (Pawson, 2002a, 2002b; Pawson & Tilley, 1997). This context is defined by factors that exist
Downloaded from aje.sagepub.com by guest on July 3, 2013

62

American Journal of Evaluation 33(1)

Context - Federal level - Financial situation - Urbanization - Language region - History

Polity factors - Planning coordination - Degree of professionalization in the administration - Administrative discretion - Influence of external experts

Policy factors - Service providers - Benefits - Costs - Inclusion of environmental aspects - Change in behavior

Politics factors - Conflict among politicians and within the administration - Participation of affected parties - Participation of societal players - Importance of communication - Decision makers

Outcomes Acceptance Success of implementation

Figure 1.

Analytical framework.

exogenously and cannot be altered (or only with great difficulty). To a certain degree, these factors also define the structure within which the planning process under examination takes place (Sager, 2008). Ultimately, the consideration of contextual factors serves the purpose of differentiating the success factors according to the framework of the project, for example, for our illustration, according to each federal level in a decentralized polity as Switzerland. In our example, the two outcomes to be explained using the model are the acceptance (Outcome 1) and the success (Outcome 2) of the implementation. On the one hand, conditions under which a transport policy decision achieves a high level of acceptance will be listed. On the other hand, the success factors for the realization of a project will be identified. Here, the assumption is central that a high level of acceptance does not necessarily lead to the actual realization of a transport project. It is thinkable, for example, that official actors are able to implement a project in spite of opposition from affected parties and environmental groups. As displayed in Figure 1, the polity factors examined relate primarily to the organization and structure of the administration, which both play a central role in the policy-planning process. The role of the administration and actions taken in the course of the decision-making process can influence public opinion. The administration also bears the main responsibility for the implementation of a transport project and can, if necessary, support, accelerate, slow down, or prevent its realization. Furthermore, the content of a projectthe policyhas a direct effect on its level of acceptance. Considerations of costs and benefits, for example, are decisive for whether a new bypass or a new public transport route is approved and built. Finally, the planning and realization of transport projects can be blocked by affected parties or other interest groups. To realize a broadly accepted project and to prevent obstructions, it is very important that various players and, potentially, their respective interests can be integrated into the planning and decision-making process. These processes,
Downloaded from aje.sagepub.com by guest on July 3, 2013

Sager and Andereggen

63

conflicts, and power relationships are also reflected in the politics factors. Contextual conditions shape patterns of polity, policy, and politics. Accordingly, the framework of analysis is divided into three strands (see Figure 1): the first examines the influence of polity factors on the outcomes; the second strand shows how policy factors determine the degree of acceptance as well as the success of the implementation; the third strand analyzes which political factors prove to be beneficial for a high degree of acceptance and successful implementation. The factors enumerated in Figure 1 will be explained in detail in the section Illustration: The Acceptance of Transport Planning Projects in Switzerland with the illustrative empirical example. In the next section, we argue that for realist synthesis, the method of QCA is an appropriate methodical approach to empirically implement the proposed framework of analysis.

The Promise of QCA

For a long time, two research methods dominated comparative study. On the one hand, researchers compared a small number of qualitative case studies (small N, case-oriented). On the other hand, quantitative research examined as many cases as possible together using various statistical methods (large N, variable-oriented). The latter approach has been criticized because quantitative studies were often unable to support conclusions about the individual case. Meanwhile, the former approach has been criticized because it was usually impossible to derive any widely applicable, general explanations from the individual cases (Brady, Collier, & Seawright, 2006; Mahoney & Goertz, 2006). QCA, presented for the first time in 1987 by Charles Ragin, seeks to close the gap between qualitative and quantitative approaches (for the most current overview cf. Rihoux & Ragin, 2008). QCA represents a mixture between the qualitative interpretation of individual cases and a mathematical logical comparison of these cases. It has proven to be very useful, particularly when a moderate number of cases (550 cases) are to be considered (Ragin, 2007, p. 14). In the following, we will first discuss analogies between realist synthesis and QCA before outlining the method in more detail.

Realist Synthesis and QCA

Befani et al. (2007) argue that QCA and the realistic approach to evaluation, as adopted in our analytical framework, share various features, namely a complex view of causality, a generative perspective, a theory-driven approach to empirical observation, and a limited claim to generalization.

Complex causation. The most important analogy between realist synthesis and QCA is their similar notion of causality. They both presume that social phenomena have multiple and conjunctural causes. The way in which a certain condition X affects a specific outcome Y depends on the presence or absence of other conditions. Similarly, a specific outcome can be caused by different combinations of conditions (Befani et al., 2007, p. 173). In terms of realistic evaluation, this notion of complex causation means that the relationship between causal mechanisms and their effects are not fixed, but contingent. Context matters because it turns (or fails to turn) causal potential into causal outcome (Pawson & Tilley, 1997, p. 69). Similarly, the key to QCA is the insight that cases have to be understood and examined as configurations (Ragin, 2007, p. 14). This implies that causal factors are not seen as independent conditions. Quite to the contrary, these individual conditions can also influence one another and together determine the case as a whole. The importance of a condition is linked with the context of the case. In this respect, the notion of multiple conjunctural causation is crucial. Different causal pathseach path being relevant, in a distinct waymay lead to the same outcome (De Meur & Rihoux, 2002). The term multiple refers to the number of paths, while the term conjunctural conveys the notion that each path consists of a combination of conditions. Thus, multiple conjunctural causation contains the notion of equifinality, which simply means that different paths can lead to the same outcome (Berg-Schlosser, De Meur, Rihoux, & Ragin, 2008,
Downloaded from aje.sagepub.com by guest on July 3, 2013

64

American Journal of Evaluation 33(1)

p. 8). Hence, by emphasizing the importance of constellations of causes instead of monocausal explanations, QCA clearly differs from quantitative methods. Statistical procedures assume that a certain condition always takes effect in the same direction. QCA, in contrast, presumes that the effect of a condition depends on context. Therefore, it is not monocausal conditions but combinations of conditions that need to be examined.

Generative perspective. Pawson and Tilley (1997) argue that due to the sectioning of cases into mere sets of independent variables, statistical methods miss the singularity and wholeness of each case. With this so-called successionist explanatory logic, correlations can be established but cannot be understood in a causal manner. Instead of simply establishing regularities in the relationship between variables, realistic evaluation demands a closer look at the way in which the variables are connected. For that reason, realist synthesis utilizes a generative approach to causation. According to this perspective it is not programmes that work: rather it is the underlying reasons or resources that they offer subjects that generate change. Causation is also reckoned to be contingent. Whether the choices or capacities on offer in an initiative are acted upon depends on the nature of their subjects and the circumstances of the initiative (Pawson, 2002b, p. 342). Similarly, QCA as a method follows a generative perspective in that in a first step, it focuses on the case as such. This is to say that the trajectory, the peculiar path of each case is interpreted individually, thereby taking into account its historicity and its specific features (De Meur & Rihoux, 2002, pp. 2627). In a second step, however, the cases need to be reduced to a limited set of variables. QCA demands that the researcher, at several points in the analysis, acts with transparency in his or her choicesselecting variables, processing them, choosing tools for the analysis, intervening during the analysis, and so on. During this process, the researcher regularly refers back to the cases with all their richness and specificity (Berg-Schlosser et al., 2008, p. 14). The generative perspective thus lies in the constant mirroring of the results of QCA in view of the individual cases. Theory-driven observation. Pawson (2002b, p. 347) emphasizes the importance of building on theory in systematic review: Realist synthesis assumes that the transmission of lessons occurs through a process of theory building rather than assembling empirical generalizations. There is an obvious affinity here with the theory-driven approaches to evaluation (Bickman, 1987; Chen & Rossi, 1992; Connell, Kubish, Schorr, & Weiss, 1995; Pawson & Tilley, 1997). Each of these begins with the notion that programmes are conjectures taking the form, if we apply programme X this unleashes process Y, which will result in Z. The task of evaluation by these lights is to gather evidence to see if the process occurs as planned and, if it should not, then to amend the theory to account for the divergent outcomes. As Berg-Schlosser et al. (2008, p. 7) points out, theory plays a crucial role for QCA at various stages. First, theory is a tool to select the important variables. As mentioned above, the cases are first observed in a holistic manner. Afterwards, however, they are reduced to a certain number of variables. These variables are not themselves real causes but mere representations of them, while the knowledge about the causes must be grounded in theoretical frameworks and is subject to revision with new evidence. Second, during the analysis, theoretical knowledge, as well as a deep knowledge of the empirical field, will help researchers make decisions regarding several practical QCA operations (Berg-Schlosser et al., 2008, p. 7). Limited generalization. As mentioned above, the realist synthesis approach to generalization is also different. The policy community is not offered a best buy (approach x or case y seems to be the most successful) but a tailored, transferable theory (this programme theory works in these respects, for these subjects, in these kinds of situations) (Pawson, 2002b, p. 342). For Pawson and Tilley (1997, p. 124), the task of empirical research is to test, refine and adjudicate the middle-range theories. According to Merton (1968, p. 39), middle-range theories are theories that
Downloaded from aje.sagepub.com by guest on July 3, 2013

Sager and Andereggen

65

lie between the minor but necessary working hypotheses that evolve in abundance during day-to-day research and the all-inclusive systematic efforts to develop a unified theory that will explain all the observed uniformities of social behaviour, social organization, and social change. Thus, realist synthesis has a limited claim to generalization. So does QCA. In contrast to statistical methods, which are based on a large number of cases, QCA is capable to compare a limited number of cases (small N) and factors. Under these circumstances, it is not possible to test grand theories but rather middle-range theories (Berg-Schlosser et al., 2008, p. 12). This is to say that as far as empirical research is concerned, QCA and realistic evaluation touch the same, limited level of generalization. Overall, realist synthesis and QCA share a number of conceptual characteristics. QCA, therefore, seems to be an appropriate method to test and refine CMO configurations. In the following, the technique is presented in more detail.

QCA as a technique
Based on the considerations above, we employ QCA as the adequate method for our realist synthesis model. Before doing so, the original version of the method will be outlined. Subsequently, we discuss two-step multi-value QCA (mvQCA) in more depth. In the original QCA variant (Crisp-Set QCA, csQCA), each individual condition variable is given a binary code. Then, a so-called truth table is compiled in which the combinations of factors observed are brought together. Subsequently, the conditions are summarized from the formalized data in the truth table using a minimization algorithm. This procedure serves the purpose of finding specific combinations of factors that are sufficient for the occurrence of the outcome. The method of minimization is based on Boolean algebra, that is, the logic of the binary system with the two logical operators or () and and (). The basic idea is that if an outcome D is found in a case displaying A, B, and C as well as in another case displaying A and C, but not B, it obviously does not make a difference for the occurrence of D whether B is present or not. The respective formula is: Af1g Bf1g Cf1g Af1g Bf0g Cf1g ! Df1g which is then minimized accordingly to Af1g Cf1g ! Df1g 2 1

QCA identifies such different constellations of conditions, that is, the causal complexity behind a given phenomenon (Rihoux & Ragin, 2008). The number of cases for each configuration of causes observed is only of a secondary importance. As our framework of analysis has a hierarchical order in which the context is placed at a higher level than the three sets of conditions of policy, polity, and politics, we employ a so-called two-step QCA for our analysis. The basic idea behind this methodological expansion is that various conditions can be causally relevant to an outcome; they display varying distances to the outcomes (Schneider & Wagemann, 2007, p. 256). According to Schneider and Wagemann (2006), the more distanced factors (referred to hereinafter as remote conditions) are conditions that remain relatively stable over time. For this reason, they are also known as structural factors. In contrast, the closer factorsthe proximate conditionschange fairly easily over time or can, in principle, be altered and influenced by players. The proximate factors are also more closely connected with the outcome in terms of both time and space. Technically, this means that the analysis is split up into two individual analyses. First, a QCA is carried out examining only the remote factors. The analysis results in various combinations of factors that promote the outcome. It is only with the second analysis that the potentially causal connections are specified. To this end, each factor, or combination of factors, resulting from the first analysis is examined together with the proximate factors, separated from the other relevant context factors, or combinations of factors. In the present
Downloaded from aje.sagepub.com by guest on July 3, 2013

66

American Journal of Evaluation 33(1)

case, this means that a QCA is first carried out for all contextual factors. The resulting factors or combinations of factors are then each analyzed together with the polity, policy, and politics conditions. The purpose of the analysis is to clearly identify the context and polity, policy, and politics conditions under which an outcome is achieved. A second expansion of the original version regards the dichotomization rule. A central problem in csQCA as described above is that all data have to be dichotomized. Such a dichotomization is unlikely to be problematic in many cases when a certain feature simply is present or absent. In other cases, however, there are clear limits to the binary codification of the data. In our example, among other aspects, the federal level is of interest. This means that the factor to be codified depends on whether it is the federal government, a canton, or a town or municipality that bears the main responsibility in a transport policy-planning and decision-making process. Dichotomization is difficult and would lead to a loss of information. In recent years, Lasse Cronqvist (cf. Cronqvist & Berg-Schlosser, 2008) has developed a new QCA variant, namely mvQCA, which makes it possible to codify the causally relevant conditions in a multinominal fashion (the outcomes still have to be dichotomous). Accordingly, in mvQCA, indices have to be used for the conditions. To stay with the aforementioned federal levels, the communal level was coded {0}, the cantonal level {1}, and {2} represents the level of the federal government in the analysis. Subsequently, the minimization rule of the dichotomous QCA variants can equally be applied with mvQCA. In the following section, we employ a two-step mvQCA for the implementation of our realist synthesis framework. The conditions for acceptance of transport planning projects in Switzerland serve us as an example.

Illustration: The Acceptance of Transport Planning Projects in Switzerland

This section, by way of an example, examines only the first strand of analysis in Figure 1 and the outcome acceptance. Accordingly, it will be examined, under which contextual and polity conditions a transport policy decision meets with a high level of acceptance. Figure 2 summarizes the part of our analytical framework that will be examined by way of a two-step mvQCA. The two steps of the QCA are indicated by the arrows. In the following, we first discuss the outcome and the different conditions, before we turn to the results of our analysis.

Result to be explainedacceptance
In political-planning processes, it is desirable that the final decision meets with the greatest possible degree of approval and acceptance. Acceptance of the decision can greatly ease the subsequent realization process, while a lack of acceptance, that is, continued opposition to the transport policy project, can hinder or even prevent the implementation entirely. Accordingly, the outcome we will treat in our example is the degree of acceptance of a transport project, measured by the number of complaints and the strength of the opposition to the project.

Discussion of the Individual Conditions

The individual context conditions and the polity conditions will be described in more detail and discussed in the following. What matters here is primarily a theoretical explanation of why the factors are included in the analysis, and not the formulation of expectations and hypotheses. This is because, due to the QCA method used, the focus lies not on explaining the influence of the individual factors but rather on the development of constellations of causes. Again, this means that, depending on the
Downloaded from aje.sagepub.com by guest on July 3, 2013

Sager and Andereggen

67

Context conditions - Federal Level - Financial situation - Urban/rural - Language region - History 2nd Step: Relevant remote and proximate factors Remote factors 1st step: Polity conditions - Planning coordination - Degree of professionalization in the administration - Administrative discretion - Influence of external experts

Outcome Acceptance

Figure 2.

Two-step QCA procedure.

context, a condition can have varying influence. Nevertheless, the selection of the factors included is based on empirically proven or theoretically plausible expectations of their effect.

Contextual conditions
The contextual conditions describe the framework within which a transport policy decision-making process takes place. These conditions are relatively stable over time and cannot be influenced by the players involved, or only with difficulty.

The federal level. On the basis of 62 case studies, Vatter, Sager, Bu hlmann, and Maibach (2000, p. 168) examined the conditions under which transport policy measures have been effectively implemented in Switzerland. While this study showed that a project is not necessarily realized more successfully if it is situated at a higher federal level, a measure is more likely to be effective, if it is implemented at a higher federal level. Accordingly, we will include the federal level as a first contextual condition. Financial situation. The financial position of the players mainly responsible for a project affects the acceptance of a transport project. It is reasonable to assume that in times of scarce state finances, a transport project is more likely to lead to distribution conflicts. It thereby results in a lower degree of acceptance than in years when the budget is balanced and revenue from taxes is high. In their study about the acceptance of Swiss transport policy, Vatter et al. (2000) showed that financial considerations play an important role: submissions promising a restructuring of state finances were accepted every time. The financial situation of the player mainly responsible for a project, taking into account joint-financing, is the second contextual condition. Urbanization. In an area with a high degree of urbanization, where buildings and transport routes tend to already be very close together, a new transport project affects more people, resulting in greater potential for conflict. This is because a new road or a new public transport route has to be integrated
Downloaded from aje.sagepub.com by guest on July 3, 2013

68

American Journal of Evaluation 33(1)

into numerous existing structures. In a rural context, a bypass, for example, is easier to plan, because the number of persons affected is small and the spatial conditions are more generous. Thus, it can be assumed that transport policy decision-making processes in less urbanized areas involve less conflict than those in urban areas. Accordingly, the degree of urbanization is the third contextual condition.

Linguistic region. Vatter et al. (2000, p. 65) attribute the varying acceptance of transport plans to a large degree to language group membership. This line of conflict is, however, founded in part on more fundamental cultural causes, including individualism and attitude toward environmental protection. Linder (2006, p. 20) argues, among other things, that the French-speaking areas in Switzerland (Romandie) vote more individualistically on transport issues. It can be assumed that transport policy projects that limit personal freedom will meet with greater acceptance in the German-speaking areas than in Western Switzerland. Accordingly, we have selected the dominant language group as the fourth contextual condition. Project history. Transport problems tend not to occur suddenly; they are the result of long-term changes in transport behavior. Often, a problem is acute for a considerable time and initial solutions are discussed over a long period of time. In the case of transport problems which have been known for some time and for which various solutions have already been discussed or even decided, a new project can build on the earlier approach or learn from the failure of earlier projects. Potential hindrances are more likely to be identified early and possibly even avoided in the same way as conflicts can be anticipated and defused. Thus, it can be expected that in transport policy projects with a prior history, the level of acceptance will be greater and implementation is more likely to be successful than in completely new projects. The existence of a prior history is the fifth contextual condition that we will take into account.

Polity conditions
The polity conditions examined are intended to show the organization and structures of the administration that bears the main responsibility in transport policy-planning processes.

Planning coordination. Coordination within the administration can have a major influence on the planning process (Sager, 2005). While cooperation between the various competent administrative bodies (such as the construction office, the transport office, and the environment agency) can occur on a voluntary basis, it can also be mandatory, on account of external pressure, for example, from politicians, or because one of the players involved has a dominant position and is able to obligate the others to participate. However, even more important than the distinction between mandatory and voluntary negotiations is the rationality of the coordination. Whether the joint interest in the best possible cooperative solution or the separate interests in the respective negotiating position (Scharpf, 1993, p. 34) is at the fore plays an important role. The presence of coordination in the planning process is the first polity condition. Professionalization of the administration. A professional administration with plentiful resources in terms of staff and expert knowledge at its disposal is more likely to be able to effect a broadly supported and well-founded planning process than an administration that is less well-positioned in financial and professional terms (Sager, 2005, pp. 247248). The degree of professionalization is thus the second polity condition. `-vis politics. A transport policy-planning process, which is The administrations discretion vis-a dominated by politicians, may lack technical knowledge. The decision may, in factual political terms, not be the best solution, but it is nevertheless liable to meet with a higher degree of acceptance
Downloaded from aje.sagepub.com by guest on July 3, 2013

Sager and Andereggen

69

than a decision taken by an administration that lacks democratic legitimacy (Sager, 2007b). On the other hand, it can be argued that power-motivated considerations play a larger role in the political decision-making process. This may therefore lead to an increased incidence of conflict or the need to reach a compromise, which is not well-founded in factual terms, meaning that a project ultimately meets with only a very low level of acceptance or is rejected. For this reason, the administrations discretionunderstood as autonomy vis-a ` -vis political influenceis included as the third polity condition.

The influence of external experts. In principle, the inclusion of external experts makes sense, above all in those areas where the administration does not itself possess the necessary knowledge. The experts do not, however, exert any influence merely by supplying purely technical information; nowadays, the experts function goes far beyond this and experts are actively involved, accompanying and influencing the planning and decision-making process, for example, as mediators. Thus, a project that was developed under the influence of experts and which may, in terms of the policy issues, be correct, may nevertheless meet with only low levels of acceptance. Ultimately, it may not be implemented due to the lack of democratic legitimacy or due to the experts ignorance of what is politically feasible (Fischer, 1990). The importance of external expertise is therefore the fourth polity condition. The operationalization of the condition and outcome variables, the selection of cases, as well as the collected data are presented in the Appendix A. In the following paragraph, we turn to the empirical analysis.

Results
For reasons of space, various choices in terms of the breadth of the actual analysis had to be made. Concretely, we limit the presented analysis as follows: First, while QCA allows for both necessary and sufficient conditions of an outcome, we only present the solutions for sufficient combinations of conditions and neglect the necessity aspect. While both aspects are essential, we argue that for the present purpose of contextualized lessons for political practice, sufficiency is more important than sheer necessity. Second, due to the basic QCA assumption of asymmetric causal relationships, the analysis has to be run for both the occurrence and nonoccurrence of an outcome. The latter is omitted in this article for space reasons and due to the primarily illustrative purpose of the analysis. The full results are reported in Schenkel et al. (2009). Two problems of QCA are contradictory cases and limited diversity. Contradictory configurations are configurations that in some observed cases lead to Outcome 0 and to Outcome 1 in others. The limited diversity problem denotes the fact that the observed data are far less rich than the potential combinations of the factors under examination. In the following QCA, we excluded contradictory configurations,1 whereas we allowed for logical remainders to address the problem of limited diversity.2 We used the Tosmana software (Cronqvist, 2006). Following the two-step approach, we will first present the solutions for the context conditions. We then integrate the proximate conditions at the polity level leading to various settings in which a project found acceptance.

Context: remote conditions. A transport project in Switzerland is characterized by what federal level carries the main responsibility in the planning and decision-making process, by the financial situation of the responsible federal players, by the language region, by the degree of urbanization, and by whether the project is new or has a prior history.
Downloaded from aje.sagepub.com by guest on July 3, 2013

70
Table 1. Truth Table for the Context Conditions for Acceptance Fed 2 1 0 1 1 2 1 1 0 0 1 1 1 1 Finan 1 1 1 1 0 0 1 0 0 1 0 0 1 1 Ger 1 1 1 1 1 1 0 1 0 0 1 0 1 1 Urb 0 0 1 1 1 0 1 1 1 1 0 1 0 1 Hist 0 1 1 0 1 0 1 0 1 1 1 1 0 1 Accep 1 1 1 0 1 0 1 1 0 1 1 0 1 0 0 0 1

American Journal of Evaluation 33(1)

Project Bypass Lungern Pfynwald, Vereinatunnel Berntor 1, Berntor 2 Claragraben 1, Gruene Linie 1 Supportive measures: West Zurich Claragraben 2 Gruene Linie 2 Loetschberg Raron Locarno bypass RER La Plaine Seftigenstrasse Wabern LEB 1 LEB 2 T10-bypass Gampelen lemont, TCSP Tramline Bypass De Geneva-Annemasse Wagen-Eschenbach-Schmericon bypass Schulhausplatz Baden

Fed Federal level; Finan Financial situation; Ger Language region; Urban Urbanization; Hist Prior history.

The following QCA will be used to examine what factors or combinations of factors encouraged a positive outcome, that is, a high level of acceptance of the project. Table 1 shows the combinations of context conditions for the outcome acceptance of the projects examined. The QCA results in the following solution: The degree of acceptance of a transport project was high where . . .
fed{2} finan{1}hist{1} finan{0}hist{0} Federal government level (Bypass Lungern, Loetschberg Raron) or Positive financial situation and prior history (Loetschberg Raron, Seftigenstrasse Wabern) or Negative financial situation and no prior history (Pfynwald, Vereinatunnel, Berntor 1, Berntor 2, LEB 2, Schulhausplatz Baden) Context 1 Context 2

This result means that there are two different contexts that influence the level of acceptance of a transport project. One context is defined by the federal level (Context 1), the other by the financial resources and the prior history of a project which in combination are either both present or absent (Context 2). What is interesting is that the degree of urbanization and the language region do not appear in the solution and thus do not seem to be a relevant context for the acceptance of the transport projects examined. As explained above, the conditions resulting from the analysis do not necessarily have to be causally relevant; they only create a structural framework, which, together with additional conditions, influence the acceptance.

Polity: proximate conditions. The polity conditions include the quality of the coordination within, and the degree of professionalization of the responsible administrative bodies, as well as the extent of the administrations discretion and the degree of influence of external experts.
Downloaded from aje.sagepub.com by guest on July 3, 2013

Sager and Andereggen

71

In the following, acceptance is viewed to be dependent on the polity conditions. This analysis must be carried out for both of the contexts resulting from the first stage of the analysis. However, again for reasons of space, we limit our illustration to the second context.3 The second context that can encourage a high level of acceptance refers to the financial situation and the prior history of a project. If one carries out the QCA using these two conditions and the polity conditions, the following results are achieved:
Positive financial situation and strong coordination (Bypass Lungern, Pfynwald, Berntor 1, Berntor 2, Schulhausplatz Baden, RER La Plaine) or negative financial situation and broad discretion and low influence of external experts (Gruene Linie 2, Loetschberg Raron) or no prior history and high influence of external experts (FlamaWest, Seftigenstrasse Wabern) finan

finan{1}coord{1}

finan{0}disc{1}expert{0}

hist{0}expert{1}

hist

Accordingly, one can say that a transport project met with acceptance, if the financial situation was positive and the responsible administrative bodies at the same time coordinated well. If the financial situation of a player was not so rosy, however, it was important that the administration was granted far-reaching discretion in the planning process (i.e., that politicians did not become too involved) and that external experts were also able to exert only a small degree of influence. The project was also well accepted in a transport policy decision-making process without any prior history in which external experts influenced the planning process. It should be noted with regard to these solutions that either of the two context factors appears in every solution. The first two solutions relate to the financial situation (finan). In the projects examined, which were planned in a favorable financial context, it was important that coordination between the administrative bodies was good. It is likely that this is due to the fact that where the financial situation is good, there is a great deal of greed or projects are more over ambitious. This can lead to conflicts, blockades, and low acceptance levels. If, however, the administrative bodies coordinate well with one another and possibly also set priorities, then a project meets with a high level of acceptance. On the other hand, in times of scarce finances, broad discretion on the part of the administration, that is, if neither politicians, nor external experts were able to have a major influence on the planning process, encouraged a high degree of acceptance. In times of financial hardship in particular, political (power) conflicts can make planning more difficult or prevent a decision. Furthermore, spending money on external experts in times of financial austerity may seem inappropriate. In the Gruene Linie 2 planning process, a small task force within the administration took the lead with a lot of discretion. In the Loetschberg project, the federal administration also had great room for maneuver and finally presented a balanced project which was accepted despite its huge costs and the tense financial situation. The third solution shows that the influence of external experts in projects without prior history can encourage acceptance. In planning processes that are still new, the involvement of external experts is likely to be perceived less critically than in projects that are already encumbered and which may also feature hardened lines of conflict. An inclusion of professional consultants from outside of the administration from the very beginning may help to avoid mistakes and to put a planning
Downloaded from aje.sagepub.com by guest on July 3, 2013

72

American Journal of Evaluation 33(1)

process on the right track. Despite the fact that both FlamaWest and the Seftigenstrasse were rather complex projects with lots of stakeholders involved, they were accepted at the first try. This may be due to the involvement of experts. In the Seftigenstrasse project, great emphasis was put on communication. The administration mandated an external communication expert to inform residents and other stakeholders professionally about the planning process, its decisions, and the implementation. In the FlamaWest case, experts played a role in the technical as well as in the communicational sectors.

Conclusion
In this article, we have made two arguments: first, that adopting a realistic approach to research synthesis, the four categories of context, politics, polity, and policy make an adequate framework for systematic review, being both exhaustive and parsimonious; second, that the method of QCA is an appropriate method for gaining realistic results useful for political practice as it does not provide monocausal explanations but different combinations of conditions leading to outcomes. The empirical application with a two-step mvQCA of 17 transport policy cases in Switzerland has shown the adequacy of this design. The two-step QCA conducted showed that the influence of the polity conditions on the acceptance of transport projects is dependent, above all, on the federal or the financial context, while urbanization and the language region constitute less important framework conditions. It is not the intention of the present article to imply that there are no problems with QCA. They exist at various instances and can also be found in the presented analysis. Limitations of the method regard the codification of the conditions, the case selection, the limited generalization, the treatment of contradictory configurations, and the application of simplifying assumptions, among others. These problems are subject to a vivid debate that will last for another while (De Meur, Rihoux, & Yamasaki, 2008). However, the presented approach has advantages that, to a certain degree, may make up for its deficiencies. The main value-added of QCA is its achievement of the goals of realist synthesis in a systematic and comparative manner by providing context-sensitive conjunctural explanations for outcomes, while preserving the substance and the explanatory richness of the cases. An increased number of cases than used in the presented example would allow for the integration of more conditions as well as less limited generalization. The challenge with a high number of cases is to preserve the advantages of the method by in-depth knowledge of each case.

Downloaded from aje.sagepub.com by guest on July 3, 2013

Sager and Andereggen

73

Appendix A Operationalization, Selection of Case Studies, and Raw Data

For the data collection, the conditions and results described above have been operationalized as described in Appendix Table A1. The labels for the conditions and results are shown in parentheses.
Table A1. Indicators and Codes for the Condition and Outcome Variables Used
Condition Context Federal level (fed) Explanation Federal level, which carries the main responsibility in the decision making and implementation process. Financial situation of the players with the main responsibility for the project (i.e., if the project is privately financed and/or greatly subsidized, the financial situation can, despite budget deficits, be classified as good). Sociostructural location of the project Cultural location of the project 2 1 0 1 0 Code Federal government Canton Town/Municipality Financial situation good Financial situation bad

Financial situation (finan)

Urbanization (urban) Language region (ger)

1 0 1 0

Prior history (hist)

Polity

Planning coordination (coord)

Professionalization (prof)

Prior history: the transport problem has been known for some time and possible solutions were previously discussed or, as the case may be, measures were planned. Coordination of planning within the various administrative bodies involved. The coordination is considered strong if it is action-motivated and takes place on a voluntary basis or alternatively is encouraged by one player. In contrast, if coordination is characterized mainly by power motivation and if it is dominated by one actor, it is considered to be weak. Professionalization of the administrative bodies with main responsibility.

1 0

Urban Rural German-speaking Switzerland French-speaking Switzerland Prior history No prior history

1 0

Strong coordination No or weak coordination

1 0

Administrations discretion (disc)

Influence of external experts (expert)

Outcome

Acceptance (accep)

` -vis politics: Does Administrations leeway vis-a politics have a considerable influence on the planning or does the administration have the main responsibility? Influence of external experts: Does administration rely mainly on own expertise or are external experts (consultancies, mediators, etc.) granted a great deal of influence on the contents or in the process of the decision-making process? What degree of acceptance does the project enjoy following the final factual decision (without court decisions) or at the time of the decision to stop? Acceptance following factual decision: Operationalization by number of complaints and/or ongoing opposition

1 0

High degree of professionalization Low level of professionalization Broad discretion Limited discretion

1 0

Great influence No influence

high

low

As for the cases, a selection was made for the QCA analysis following an initial appraisal of the existing case studies on transport policy projects in Switzerland. The main requirement was that the authors of the case study had a scientific background (social sciences, geography, or economics), the case study contained information on the condition and outcome variables under examination here, and it followed common scientific standards.

Downloaded from aje.sagepub.com by guest on July 3, 2013

74

American Journal of Evaluation 33(1) The studies on the following projects fulfilled these conditions:

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17.

Motorway N8, Lungern bypass, OW: a medium-sized MIT (motorized individual transport) infrastructure project. Motorway and railway line in the area of Pfynwald, VS: large MIT and PT (public transport) infrastructure project, emphasis on MIT. Multistorey car park Berntor, in the town Solothurn, SO: 2 Subcases, communal MIT infrastructure project. Supportive measures West Zurich bypass, ZH: major infrastructure and operating concept with emphasis on MIT. Tramline extension Claragraben in Basel, BS: 2 subcases: medium-sized PT infrastructure project. Gruene Linie (Green Line) regional commuter train in Basel, BS: 2 subcases, medium-sized PT infrastructure and operating project. Alptransit Lo tschberg, basic tunnel in Raron, VS: major PT infrastructure project. Locarno bypass, TI: medium-sized MIT infrastructure project. Seftigenstrasse, crosstown link Wabern, BE: communal MIT infrastructure project. LEB commuter train line extension Lausanne-Echallens-Bercher, VD: 2 subcases, mediumsized PT infrastructure project. RER La Plaine in Geneva, GE: medium-sized PT infrastructure and operating project. T10-Bypass in Gampelen, BE: medium-sized MIT infrastructure project. Vereina tunnel near Closters, GR: major PT infrastructure project. De lemont bypass, JU: medium-sized MIT infrastructure project. TCSP Tramline Geneva-Annemasse, GE: medium-sized PT infrastructure and operating project. Wagen-Eschenbach-Schmericon bypass, ZH: medium-sized MIT infrastructure project. Restructuring Schulhausplatz Baden, AG: medium-sized MIT infrastructure project (including improvements for nonmotorized traffic).

The coding of the cases leads to the raw data presented in Appendix Table A2.

Downloaded from aje.sagepub.com by guest on July 3, 2013

Table A2. Raw Data

Projects

Factors

A8 Bypass Lungern OW Locarno Bypass

Pfynwald VS

Multistorey Car Park Berntor, Solothurn, 1. Project Tramline Claragraben, Basel, 2. Project tschLo berg, Basic Tunnel in Raron VS RER La Plaine, Commuter Line, Geneva Seftigenstrasse, Crosstown Link Wabern BE T10 Bypass Gampelen BE LEB, Commuter Train Line, Lausanne, 1. Project LEB, CommuterTrain Line, Lausanne,2. Project

Multistorey Car Park Berntor, Solothurn, 2. Project

Supportive Measures West Zurich

Tramline Claragraben, Basel, 1. Project

GrueneLinie, Regional Commuter Train Basel, 1. Project

Gruene Linie, Regional Commuter Train Basel, 1. Project

Vereinatunnel, Closters GR

leDe mont Bypass

TCSP Tramline GenevaAnnemasse

WagenEschenbachSchmericon Bypass

Schulhausplatz Baden

2 1 0 1 0 1 1 0 0 0 0 1 1 0 1 0 1 0 0 1 1 1 0 1 1 1 0 0 1 0 0 1 1 0 1 1 1 1 1 0 0 1 0 0 1

1 1 0 1 1

0 1 1 1 1

0 1 1 1 1

1 1 1 1 0

1 1 1 1 0

1 0 1 1 1

1 1 1 1 0

1 0 1 1 1

2 0 0 1 0

1 1 1 0 1

1 0 1 1 0

0 0 1 0 1

0 1 1 0 1

1 1 1 0 1 1 1 1 0 1

1 0 0 1 1 0 1 0 1 0

1 1 0 1 1 0 1 0 0 1

1 0 1 0 1 1 0 0 1 0

1 0 1 0 1 0 1 0 0 0

1 1 0 1 0 0 1 0 0 0

1 1 1 1 1 1 1 1 1 1

Downloaded from aje.sagepub.com by guest on July 3, 2013

Context Fed Finan Urban Ger Hist Polity Coord Prof Disc Expert Outcome Accep

1 0 0 0

1 1 1 1

1 1 1 1

1 1 1 1

0 1 1 1

0 1 1 0

Fed federal level; Finan financial situation; Ger language region; Urban urbanization; Hist prior history; Coord planning coordination; Prof professionalization; Disc administrations discretion; Expert influence of external experts; Accep acceptance.

75

76 Declaration of Conflicting Interests

American Journal of Evaluation 33(1)

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The presented research was funded by the Swiss Association of Road and Transportation Experts (VSS) and the Swiss Federal roads office (FEDRO) (Project SVI 2004/004).

Notes
1. Before doing so, we aimed to find ways to lift these contradictions by going back to the cases. Due to the limited information found in the case studies at hand, however, this strategy was not successful. Thus, we opted for exclusion. 2. Logical remainders are logically possible combinations of conditions that have not been observed among the empirical cases and do not contradict them. For a thorough discussion of the problem of limited diversity, see Schneider and Wagemann (2007, pp. 101115, 256262). 3. As for Context 1, the inclusion of the federal level showed that the two projects for which the federal government was responsible were accepted, regardless of the characteristics of the polity factors. At the municipal level, projects were received positively and implemented if politicians had a central influence on the planning and decision-making process. The results for the cantonal level are somewhat more complex. There, projects were accepted mostly when the planning was more technocratic rather than decisionistic, that is, if the administration orin the case of a lack of coordination within the administrationexternal experts influenced the planning and decision-making process. So, while the administrative organization for federal government projects was secondary, cantonal projects were accepted above all if the administration was powerful and the bodies involved coordinated well with one another, or where coordination within the administration or its discretion was restricted, but instead external experts had a considerable influence on the planning and decision-making process. Conversely, a communal project was more likely to be accepted and realized, where the administration did not have too much discretion, and, instead, politicians were involved in the planning process.

References
Befani, B., Ledermann, S., & Sager, F. (2007). Realistic evaluation and QCA. Conceptual parallels and an empirical application. Evaluation, 13, 171-192. Berg-Schlosser, D., De Meur, G., Rihoux, B., & Ragin, C. C. (2008). Qualitative comparative analysis (QCA) as an approach. In B. Rihoux & C. C. Ragin (Eds.), Configurational comparative methods. Qualitative comparative analysis (QCA) and related techniques (pp. 1-18). London, Thousand Oaks, CA, and New Delhi: Sage. Bickman, L. (Ed.). (1987). Using program theory in evaluation, new directions for program evaluation (Vol. 33). San Francisco, CA: Jossey-Bass. Brady, H. E., Collier, D., & Seawright, J. (2006). Toward a pluralistic vision of methodology. Political Analysis, 14, 353-368. Chen, H., & Rossi, P. (1992). Using theory to improve policy and program evaluations. Westport, CT: Greenwood. Connell, J., Kubish, A., Schorr, L., & Weiss, C. (1995). New approaches to evaluating community initiatives. New York, NY: Aspen Institute. Cronqvist, L. (2006). Tosmana Tool for small-N analysis. (Version 1.254) Marburg: University of Marburg. Cronqvist, L., & Berg-Schlosser, D. (2008). Multi-value QCA (MVQCA). In B. Rihoux & C. C. Ragin (Eds.), Configurational comparative methods. Qualitative comparative analysis (QCA) and related techniques. London, Thousand Oaks, CA, and New Delhi: Sage.
Downloaded from aje.sagepub.com by guest on July 3, 2013

Sager and Andereggen

77

e (AQQC-QCA): Approche, techniques De Meur, G., & Rihoux, B. (2002). Lanalyse quali-quantitative compare et applications en sciences humaines [Qualitative Comparative Analysis (QCA): Approach, technique, and applications in the Humanities]. Louvain-la-Neuve: Bruylant-Academia. De Meur, G., Rihoux, B., & Yamasaki, S. (2008). Addressing the critiques of QCA. In B. Rihoux & C. C. Ragin (Eds.), Configurational comparative methods. Qualitative comparative analysis (QCA) and related techniques (pp. 147-165). London, Thousand Oaks, CA, and New Delhi: Sage. Fischer, F. (1990). Technocracy and the politics of expertise. Newbury Park, CA: Sage. Heinrich, C. J. (2007). Evidence-based policy and performance management. Challenges and prospects in two parallel movements. The American Review of Public Administration, 37, 255-277. Linder, W. (2006). Politische Kultur [Political Culture]. In U. Klo ti, P. Knoepfel, H. Kriesi, W. Linder, Y. Papadopoulos, & P. Sciarini (Eds.), Handbuch Schweizer Politik [Handbook of Swiss Politics] (pp. 15-34). Zurich: Verlag Neue Zu rcher Zeitung. Mahoney, J., & Goertz, G. (2006). A tale of two cultures: Contrasting quantitative and qualitative research. Political Analysis, 14, 227-249. Merton, R. K. (1968). Social theory and social structure. New York, NY and London: Free Press. Pawson, R. (2002a). Evidence-based policy: In search of a method. Evaluation, 8, 157-181. Pawson, R. (2002b). Evidence-based policy: The promise of realist synthesis.. Evaluation, 8, 340-358. Pawson, R., & Tilley, N. (1997). Realistic evaluation. London, Thousand Oaks, CA, and New Delhi: Sage. Ragin, C. C. (1987). The comparative method. Moving beyond qualitative and quantitative strategies. Berkeley, CA, Los Angeles, CA, and London: University of California Press. Ragin, C. C. (2007). Foreword. In C. Q. Schneider & C. Wagemann (Eds.), Qualitative comparative analysis (QCA) und Fuzzy Sets. Ein Lehrbuch for Anwender und solche, die es werden wollen [A Textbook for Users and those who want to become it] (pp. 13-18). Opladen and Farmington Hills: Verlag Barbara Budrich. Rihoux, B., & Ragin, C. C. (Eds.). (2008). Configurational comparative methods. Qualitative comparative analysis (QCA) and related techniques. London, Thousand Oaks, CA, and New Delhi: Sage. Sager, F. (2005). Metropolitan institutions and policy coordination. The integration of land use and transport policies in Swiss urban areas. Governance, 18, 227-256. Sager, F. (2007a). Making transport policy work: Polity, policy, politics and systematic review. Policy & Politics, 35, 269-288. Sager, F. (2007b). Habermas models of decisionism, technocracy, and pragmatism in times of governance. The relationship of public administration, politics, and science in the alcohol prevention policies of the Swiss member states. Public Administration, 85, 429-447. Sager, F. (2008). Planning, power, and policy change in the networked city: The politics of a new tramway in the city of Bern. In G. Pflieger, L. Pattaroni, C. Jemelin, & V. Kaufmann (Eds.), The social fabric of the networked city (pp. 169-187). Lausanne and Oxford: EPFL Press/Routledge. Scharpf, F. W. (1993). Legitimationsprobleme der Globalisierung. Regieren in Verhandlungssystemen [Legitimization problems of globalization. Governing in negotiation systems]. Ko ln: MPIFG Discussion Paper. Schenkel, W., Oetterli, D., Sager, F., Andereggen, C., Ru efli, C., Hu gli, E., & Blumenstein, A. (2009). Verkehrspolitische Entscheidfindung in der Verkehrsplanung. Eine systematische Prozess- und Kommunikationsanalyse [Political decision-making in transport planning. A systematic analysis of processes and communication]. Bern: SVI. Schneider, C. Q., & Wagemann, C. (2006). Reducing complexity in qualitative comparative analysis (QCA): Remote and proximate factors and the consolidation of democracy. European Journal of Political Research, 45, 751-786. Schneider, C. Q., & Wagemann, C. (2007). Qualitative comparative analysis (QCA) und Fuzzy Sets. Ein Lehrbuch fu r Anwender und solche, die es werden wollen [A Textbook for Users and those who want to become it]. Opladen and Farmington Hills: Verlag Barbara Budrich. Simons, H. (2004). Utilizing evaluation evidence to enhance professional practice. Evaluation, 10, 410-429.
Downloaded from aje.sagepub.com by guest on July 3, 2013

78

American Journal of Evaluation 33(1)

Treib, O., Ba hr, H., & Falkner, G. (2007). Modes of governance: Towards a conceptual clarification. Journal of European Public Policy, 14, 1-20. Van der Knaap, L. M., Leeuw, F., Bogaerts, S., & Nijssen, L. T. J. (2008). Combining Campbell standards and the realist evaluation. The best of two worlds? American Journal of Evaluation, 29, 48-57. Vatter, A., Sager, F., Bu hlmann, M., & Maibach, M. (2000). Akzeptanz der schweizerischen Verkehrspolitik bei Volksabstimmungen und im Vollzug [Acceptance of Swiss Transport Policy in Public Votes and in Implementation]. Bern: BBL/EDMZ.

Downloaded from aje.sagepub.com by guest on July 3, 2013