Tolin, David F, Is cognitive-behavioral therapy more effective than other therapies?

:
A meta-analytic review. Clinical Psychology Review, 08 2010, vol./is. 30/6(710-720),
0272-7358 (Aug 2010)

Abstract:
Cognitive-behavioral therapy (CBT) is effective for a range of psychiatric disorders.
However, it remains unclear whether CBT is superior to other forms of psychotherapy,
and previous quantitative reviews on this topic are difficult to interpret. The aim of the
present quantitative review was to determine whether CBT yields superior outcomes to
alternative forms of psychotherapy, and to examine the relationship between differential
outcome and study-specific variables. From a computerized literature search through
September 2007 and references from previous reviews, English-language articles were
selected that described randomized controlled trials of CBT vs. another form of
psychotherapy. Of these, only those in which the CBT and alternative therapy condition
were judged to be bona fide treatments, rather than "intent-to-fail" conditions, were
retained for analysis (28 articles representing 26 studies, N =1981). Four raters identified
post-treatment and follow-up effect size estimates, as well as study-specific variables
including (but not limited to) type of CBT and other psychotherapy, sample diagnosis,
type of outcome measure used, and age group. Studies were rated for methodological
adequacy including (but not limited to) the use of reliable and valid measures and
independent evaluators. Researcher allegiance was determined by contacting the principal
investigators of the source articles. CBT was superior to psychodynamic therapy,
although not interpersonal or supportive therapies, at post-treatment and at follow-up.
Methodological strength of studies was not associated with larger or smaller differences
between CBT and other therapies. Researchers' self-reported allegiance was positively
correlated with the strength of CBT's superiority; however, when controlling for
allegiance ratings, CBT was still associated with a significant advantage. The superiority
of CBT over alternative therapies was evident only among patients with anxiety or
depressive disorders. These results argue against previous claims of treatment
equivalence and suggest that CBT should be considered a first-line psychosocial
treatment of choice, at least for patients with anxiety and depressive disorders.
(PsycINFO Database Record (c) 2010 APA, all rights reserved) (journal abstract)

Mansell W, Core processes of psychopathology and recovery: "Does the Dodo bird
effect have wings?", Clinical psychology review, 25 June 2010, 1873-7811

This editorial proposes that the task of identifying common processes across disorders
and across psychotherapies will be the most fruitful way to develop efficient, easily
trainable and coherent psychological interventions. The article adapts the concept of the
'Dodo Bird Effect' to argue for a mechanistic, testable account of functioning, akin to
other unified approaches in science. The articles in the special issue complement this
perspective in several ways: (1) three articles identify common processes across disorders
within the domains of anger dysregulation, sleep disruption and perfectionism; (2) one
article emphasises a case conceptualisation approach that is applied across different
disorders and integrates theoretical approaches; (3) three articles focus on the utility of a
control theory approach to understand the core processes of maintenance and change.
Critically, there is a consensus that change involves facilitating the integration within the
client's awareness of higher level, self-determined goals (e.g. insight; cognitive
reappraisal) with their lower level regulation of present-moment experience (e.g.
emotional openness; exposure). Taken together, these articles illustrate the benefits of a
convergent rather than divergent approach to the science and practice of psychological
therapy, and they strive to identify common ground across psychotherapies and across
widely different presentations of psychopathology.

Berman, Jeffrey S,Reich, Catherine M, Investigator allegiance and the evaluation of

psychotherapy outcome research. European Journal of Psychotherapy and Counselling,
03 2010, vol./is. 12/1(11-21), 1364-2537;1469-5901 (Mar 2010)
Author(s):
Abstract:
Considerable evidence has demonstrated that the beliefs of researchers can inadvertently
influence research findings. The possibility of this type of bias is of special concern in
studies evaluating the outcome of psychotherapy, where investigators frequently have
marked allegiances to particular therapies and these allegiances have been found to
correlate substantially with the pattern of results. In this article we discuss the evidence
concerning investigator allegiance in psychotherapy research, emphasize the need to
distinguish between this factor as a potential confound and a proved causal effect, and
outline strategies that have been suggested for researchers to minimize the potential for
bias both when designing future research and drawing conclusions from existing
evidence. (PsycINFO Database Record (c) 2010 APA, all rights reserved) (journal
abstract)

Budge, Stephanie,Baardseth, Timothy P,Wampold, Bruce E,Fluckiger, Christoph,

Researcher allegiance and supportive therapy: Pernicious affects on results of
randomized clinical trials. European Journal of Psychotherapy and Counselling, 03
2010, vol./is. 12/1(23-39), 1364-2537;1469-5901 (Mar 2010)
Author(s):
Abstract:
Allegiance effects have been discussed, debated, and tested over the past several decades.
The evidence clearly shows that allegiance affects the findings and representation of
therapies that are considered efficacious. We argue that allegiance effects are evident in
randomized controlled trials of supportive therapy. Supportive therapy is an established
and bona-fide therapy but its implementation as a control group for non-specific elements
is a treatment that does not resemble supportive therapy as would be used therapeutically.
Allegiance effects in the use of supportive therapy are caused by the design of supportive
therapy controls, the therapists who deliver supportive therapy, and the patients who are
enrolled in the trials. (PsycINFO Database Record (c) 2010 APA, all rights reserved)
(journal abstract)
Botella, Luis,Beriain, Diana. Allegiance effects in psychotherapy research: A
constructivist approach. European Journal of Psychotherapy and Counselling, 03 2010,
vol./is. 12/1(55-64), 1364-2537;1469-5901 (Mar 2010)
Author(s):
Abstract:
This paper examines the concept of allegiance effects in psychotherapy research from a
constructivist approach. After considering their role in outcome and process research, a
constructivist explanation of them is proposed. It is also suggested that traditional ways to
control them, while necessary and sound, may not be enough. Alternatively, a call for
methodological pluralism in psychotherapy research is made, especially regarding the
inclusion of qualitative, hermeneutic, phenomenological and discovery oriented case
studies that privilege the voice of clients and not only the researchers favoured constructs.
(PsycINFO Database Record (c) 2010 APA, all rights reserved) (journal abstract)

Voracek, Martin,Tran, Ulrich S,Fisher, Maryanne L, Evolutionary psychology's notion

of differential grandparental investment and the Dodo Bird Phenomenon: Not
everyone can be right. Behavioral and Brain Sciences, 02 2010, vol./is. 33/1(39-40),
0140-525X;1469-1825 (Feb 2010)
Abstract:
Presents open peer commentary on an article in the current issue by Coall and Hertwig
(see record 2010-08500-001), who addressed the question of whether the help that
grandparents provide, which may have benefited grandchildren in traditional and
historical populations, still yields benefits for grandchildren in industrialized societies.
The current authors note that integration of different lines of research concerning
grandparental investment appears to be both promising and necessary. However, it must
stop short when confronted with incommensurate arguments and hypotheses, either
within or between disciplines. Further, some hypotheses have less plausibility and
veridicality than others. This point is illustrated with results that conflict previous
conclusions from evolutionary psychology about differential grandparental investment.
(PsycINFO Database Record (c) 2010 APA, all rights reserv

Pence, Steven L Jr.,Sulkowski, Michael L,Jordan, Cary,Storch, Eric A, When exposures

go wrong: Trouble-shooting guidelines for managing difficult scenarios that arise in
exposure-based treatment for obsessive-compulsive disorder. American Journal of
Psychotherapy, 2010, vol./is. 64/1(39-53), 0002-9564 (2010)
Author(s):
Abstract:
Cognitive-behavioral therapy (CBT) with exposure and ritual prevention (ERP) is widely
accepted as the most effective psychological treatment for obsessive compulsive disorder
(OCD). However, the extant literature and treatment manuals cannot fully address all the
variations in client presentation, the diversity of ERP tasks, and how to negotiate the
inevitable therapeutic challenges that may occur. Within this article, we attempt to
address common difficulties encountered by therapists employing exposure-based
therapy in areas related to: 1) when clients fail to habituate to their anxiety, 2) when
clients misjudge how much anxiety an exposure will actually cause, 3) when incidental
exposures happen in session, 4) when mental or covert rituals interfere with treatment,
and 5) when clients demonstrate exceptionally high sensitivities to anxiety. The goal of
this paper is to bridge the gap between treatment theory and practical implementation
issues encountered by therapists providing CBT for OCD. (PsycINFO Database Record
(c) 2010 APA, all rights reserved) (journal abstract)

Duncan, Barry L, Some therapies are more equal than others?

PsycCRITIQUES, 2010, vol./is. 55/37, 1554-0138 (2010)
Author(s):
Abstract:
Comments on Thomas L. Rodebaugh's review (see record 2010-12182-001) of Barry L.
Duncan, Scott D. Miller, Bruce E. Wampold, and Mark A. Hubble's edited book, The
heart and soul of change: Delivering what works in therapy (2nd ed.) (see record 2009-
10638-000). In his review, Rodebaugh candidly admits his allegiance to empirically
supported treatments, which perhaps explains the myopic lens used to examine the book.
The dodo verdict ("Everybody has won and all must have prizes") still perfectly describes
the state of affairs in psychotherapy--all bona fide approaches, in spite of vociferously
argued differences, appear to work equally well. Rodebaugh's assertion that one must
examine specific treatments for specific disorders to uncover differences between
treatments ignores the many direct comparisons that have not yielded any differences for
specific disorders, such as the Treatment of Depression Collaborative Research Program,
Project Match, and the Cannabis Youth Treatment Project, to mention a few (see these
program descriptions in The Heart and Soul of Change). No wear in the book is there any
suggestion that the dodo verdict implies that we should "leave well enough alone"
regarding research, that (perhaps the most egregious comment) anything goes in the
consulting room, or that there is little point to training. Quite the contrary. The book
advocates for a shift toward research and training about what works and how to deliver it,
and away from a sole reliance on comparative, "battle of the brands" clinical trials.
Dismissing the book on the basis that some therapies are more equal than others is
reminiscent of another set of animals in another classic story. It's time to transcend the
polemics and instead focus on what works with the client in my office now. (PsycINFO
Database Record (c) 2010 APA, all rights reserved)

Rodebaugh, Thomas L, The heart and soul of the dodo. PsycCRITIQUES, 2010,
vol./is. 55/28, 1554-0138 (2010)
Abstract:
Reviews the book, The Heart and Soul of Change: Delivering What Works in Therapy
(2nd ed.) edited by Barry L. Duncan, Scott D. Miller, Bruce E. Wampold, and Mark A.
Hubble (see record 2009-10638-000). In this book, considerable attention is paid to
establishing that Saul Rosenzweig was the original articulator of the dodo bird
hypothesis: All psychotherapies work about equally effectively. The dodo bird's
statement is not meant to be a hypothesis: It is meant to quiet the animals. Taken literally,
the declaration regarding winners and prizes is clearly intended as nonsensical. The dodo,
otherwise best known as a dead bird, is thereby made immortal as a purveyor of
nonsense. The dodo is a strong force in The Heart and Soul of Change. The book is a
series of chapters by different authors but maintains a structure largely focused on the
dodo bird hypothesis, its historical context, the research that can be taken to support it,
and its implications for practice. Much of the rest of the book consists of further
demonstrations that the dodo bird hypothesis is the most sensible interpretation of the
data, set alongside critiques of empirically supported therapies (ESTs) and policies that
support their adoption. Some later chapters focus primarily on what should be the next
steps given that the dodo bird's viewpoint is better supported than is a viewpoint that
emphasizes ESTs. (PsycINFO Database Record (c) 2010 APA, all rights reserved)

Johansson H, The effectiveness of psychologically/psychodynamically-oriented,

pharmacological and combination treatment in a routine psychiatric outpatient
setting. Internet Journal of Mental Health, 01 January 2010, vol./is. 6/2(0-9), 15312941
Author(s):
Abstract:
Background: This was an outcome study in a routine psychiatric outpatient unit in
Sweden where the treatment the patients received was determined by normal routines at
the unit and performed by members of the staff. The aim of the present study was to
study the effectiveness of psychologically/psychodynamically-oriented-, pharmacological
treatment and its combination. Method: Newly admitted patients were diagnosed
according to the ICD-10 and completed questionnaires regarding symptoms and
interpersonal problems at the beginning and termination of their treatment (n = 76).
Follow-up assessments were conducted 18 months after treatment began. An ANCOVA
was used to calculate differences between groups. Effect sizes and clinical significance
were also calculated.Results: The results showed that there was a significant treatment
effect for all treatment conditions in many outcome variables. Further, all three treatment
groups showed equal effectiveness. However, the combination treatment used
significantly more treatment sessions than the other two groups. The results also showed
that many of the patients had considerable problems after the treatment.Conclusion: The
results indicate that the patients were offered treatment, and achieved what they required
in order to reach a positive outcome. It appears that the treatment was determined by
responsiveness and regulatory processes of both the staff and patients, and that the
patients acquired what they needed to accomplish with a sufficient outcome.

Budd R,Hughes I, The Dodo Bird Verdict--controversial, inevitable and important: a

commentary on 30 years of meta-analyses. Clinical psychology & psychotherapy,
November 2009, vol./is. 16/6(510-22), 1099-0879
Author(s):
Abstract:
In this article, the assertion that different psychological therapies are of broadly similar
efficacy-often called the 'Dodo Bird Verdict'-is contrasted with the alternative view that
there are specific therapies that are more effective than others for particular diagnoses.
We note that, despite thirty years of meta-analytic reviews tending to support the finding
of therapy equivalence, this view is still controversial and has not been accepted by many
within the psychological therapy community; we explore this from a theory of science
perspective. It is further argued that the equivalence of ostensibly different therapies is an
inevitable consequence of the methodology that has dominated this field of investigation;
namely, randomised controlled trials [RCTs]. The implicit assumptions of RCTs are
analysed and it is argued that what we know about psychological therapy indicates that it
is not appropriate to treat 'type of therapy' and 'diagnosis' as if they were independent
variables in an experimental design. It is noted that one logical consequence of this is that
we would not expect RCTs to be capable of isolating effects that are specific to 'type of
therapy' and 'diagnosis'. Rather, RCTs would only be expected to be capable of
identifying the non-specific effects of covariates, such as those of therapist allegiance. It
is further suggested that those non-specific effects that have been identified via meta-
analysis are not trivial findings, but rather characterise important features of
psychological therapy.
Language:
ENG

The Dodo Bird Verdict--Controversial, inevitable and important: A commentary on

30 years of meta-analyses.
Citation:
Clinical Psychology & Psychotherapy, 11-12 2009, vol./is. 16/6(510-522), 1063-
3995;1099-0879 (Nov-Dec 2009)
Author(s):
Budd, Rick,Hughes, Ian
Abstract:
In this article, the assertion that different psychological therapies are of broadly similar
efficacy--often called the 'Dodo Bird Verdict'--is contrasted with the alternative view that
there are specific therapies that are more effective than others for particular diagnoses.
We note that, despite thirty years of meta-analytic reviews tending to support the finding
of therapy equivalence, this view is still controversial and has not been accepted by many
within the psychological therapy community; we explore this from a theory of science
perspective. It is further argued that the equivalence of ostensibly different therapies is an
inevitable consequence of the methodology that has dominated this field of investigation;
namely, randomised controlled trials [RCTs]. The implicit assumptions of RCTs are
analysed and it is argued that what we know about psychological therapy indicates that it
is not appropriate to treat 'type of therapy' and 'diagnosis' as if they were independent
variables in an experimental design. It is noted that one logical consequence of this is that
we would not expect RCTs to be capable of isolating effects that are specific to 'type of
therapy' and 'diagnosis'. Rather, RCTs would only be expected to be capable of
identifying the non-specific effects of covariates, such as those of therapist allegiance. It
is further suggested that those nonspecific effects that have been identified via meta-
analysis are not trivial findings, but rather characterise important features of
psychological therapy. (PsycINFO Database Record (c) 2010 APA, all rights reserved)
(journal abstract)

Siev, Jedidiah,Huppert, Jonathan D,Chambless, Dianne L, The Dodo Bird, treatment

technique, and disseminating empirically supported treatments. the Behavior
Therapist, 04 2009, vol./is. 32/4(69, 71-76), 0278-8403 (Apr 2009)
Author(s):
Abstract:
The aim of this article is to provide some historical context in terms of previous attempts
to respond to these contentions and to present an update on recent research bearing
directly on the Dodo Bird verdict and the assertions regarding variance accounted for by
active ingredients (e.g., technique). Evidence for the claim that all psychotherapies are
equally efficacious derives from meta-analyses that combine various treatments for
various disorders. Therapist effects have been discussed on and off for over 30 years.
More recently, some have shown that differences between therapists in treatment
outcome may be decreased with manualized treatments. However, the question of what
makes therapists different from each other remains, and one answer may be technique.
Some therapists are likely more adept than others at using some techniques, formulating
treatment plans, encouraging their patients to do difficult exposures, etc., even within
CBT. (PsycINFO Database Record (c) 2010 APA, all rights reserved)
Language:
English

Overholser, James C,Braden, Abby,Fisher, Lauren You've got to believe: Core beliefs
that underlie effective psychotherapy. Journal of Contemporary Psychotherapy, 12
2010, vol./is. 40/4(185-194), 0022-0116;1573-3564 (Dec 2010)
Author(s):

Abstract:
A mixture of core beliefs may lay the foundation for effective psychotherapy. Sincere
trust in these beliefs may help to promote therapeutic change. The therapist must have
faith in the power of words to promote change. Clients usually change in a gradual
manner, and the initial plan for therapy can be simplified by focusing on strategies for
changing actions and attitudes. Also, therapy can help to improve various aspects of
clients' intimate relationships. However, before attempting to promote therapeutic
change, it is important for the therapist to begin by understanding the client's life
situation, current distress, and natural tendencies. Clients benefit from emotional
tolerance of stressors by recognizing that many negative life events turn out better than
initially expected. A tendency to dwell on past events can perpetuate problems, while it
can be more helpful to accept and grow from negative events. Therapists are encouraged
to view a client's emotions as natural reactions, not deviant dysfunctions that need to be
blocked or suppressed through medications. In a similar manner, most labels, including
many psychiatric diagnoses, pose a danger through societal discrimination and self-
stigma. When therapists adopt these core beliefs, they can more effectively help clients
move forward, making adaptive psychological changes. (PsycINFO Database Record (c)
2010 APA, all rights reserved) (journal abstract)
Language:
English

Cooper, Mick, The challenge of counselling and psychotherapy research.

Counselling & Psychotherapy Research, 09 2010, vol./is. 10/3(183-191), 1473-3145 (Sep
2010)
Author(s):
Abstract:
Aims: The purpose of this commentary is to argue that the value of counselling and
psychotherapy research lies, not only in what it teaches us as therapists, but also in its
ability to challenge us and our assumptions. Method: The paper identifies eight beliefs
that may be prevalent in sections of the counselling and psychotherapy community, and
presents evidence that challenges them. Findings: While many of our beliefs may hold
true for some clients some of the time, the research evidence suggests that they are
unlikely to be true for all clients all of the time. Discussion: By questioning and
challenging therapists' a priori assumptions, research findings can help counsellors and
psychotherapists to be less set in their beliefs; and more open to the unique experiences,
characteristics and wants of each individual client. (PsycINFO Database Record (c) 2010
APA, all rights reserved) (journal abstract)
The challenge of counselling and psychotherapy research.
Citation:
Counselling & Psychotherapy Research, 09 2010, vol./is. 10/3(183-191), 1473-3145 (Sep
2010)
Author(s):
Cooper, Mick
Abstract:
Aims: The purpose of this commentary is to argue that the value of counselling and
psychotherapy research lies, not only in what it teaches us as therapists, but also in its
ability to challenge us and our assumptions. Method: The paper identifies eight beliefs
that may be prevalent in sections of the counselling and psychotherapy community, and
presents evidence that challenges them. Findings: While many of our beliefs may hold
true for some clients some of the time, the research evidence suggests that they are
unlikely to be true for all clients all of the time. Discussion: By questioning and
challenging therapists' a priori assumptions, research findings can help counsellors and
psychotherapists to be less set in their beliefs; and more open to the unique experiences,
characteristics and wants of each individual client. (PsycINFO Database Record (c) 2010
APA, all rights reserved) (journal abstract)
Copyright 1987 by the American Psychological Association, Inc.
0022-006X/87/J00.75

John R. Weisz , Bahr Weiss, Mark D. Alicke, M. L. Klotz, Effectiveness of

Psychotherapy With Children and Adolescents:A Meta-Analysis for Clinicians, Journal
of Consulting and Clinical Psychology, !987,Vol. 55, No.4. 542-549

University of North Carolina at Chapel Hill

University of Florida at Gainesville
University of North Carolina at Chapel Hill
How effective is psychotherapy with children and adolescents? The question was addressed by metaanalysis
of 108 well-designed outcome studies with 4-18-year-old participants. Across various outcome
measures, the average treated youngster was better adjusted after treatment than 79% of those
not treated. Therapy proved rnore effective for children than for adolescents, particularly when the
therapists were paraprofessionals (e.g., parents, teachers) or graduate students. Professionals (with
doctor's or master's degrees) were especially effective in treating overcontrolled problems (e.g., phobias,
shyness) but were not more effective than other therapists in treating undercontrolled problems
(e.g., aggression, impulsivity). Behavioral treatments proved more effective than nonbehavioral treatments
regardless of client age, therapist experience, or treated problem. Overall, the findings revealed
significant, durable effects of treatment that differed somewhat with client age and treatment method
but were reliably greater than zero for most groups, most problems, and most methods.
In the late 1970s and early 1980s, experts raised diverse concerns
about child and adolescent psychotherapy research. Some
complained of insufficient information about therapy outcomes
(e.g., Achenbach, 1982). Others suggested that outcome studies
revealed few or no effects of therapy (e.g., Gelfand, Jenson, &
Drew, 1982). Still others (e.g., Barrett, Hampe, & Miller, 1978)
argued that researchers were too preoccupied with the global
question of psychotherapy effects per se and should instead
study outcomes as a function of treatment approach, type of
child, and therapist characteristics. In recent years, the prospects
for addressing these concerns have improved considerably.
With the development of meta-analytic techniques (Smith,
Glass, & Miller, 1980), it is now possible to aggregate findings
across multiple studies and to systematically compare findings
across dimensions such as treatment approach and client characteristics.
The basis for analysis is the effect size, which is computed
separately for the treatment group versus control group
comparisons of interest. The effect size is an estimate of the
magnitude of the treatment effect (treatment group versus control
group scores on measures of psychological functioning) adjusted
for sample variability.
Meta-analyses of adult outcome studies (Shapiro & Shapiro,
This project was facilitated by support from the North Carolina Division
of Mental Health, Mental Retardation, and Substance Abuse Services
(Department of Human Resources) and by National Institute of
Mental Health Grant R01 MH 38240-01. We are grateful to David
Langmeyer for his support and suggestions, to Gary Bornstein, Larry
Crum, and Lynn Fisher for their assistance in data collection and computation,
and to Thomas Achenbach for his thoughtful review of an
earlier draft of this article.
Correspondence concerning this article should be addressed to John
R. Weisz, Department of Psychology, Davie Hall 013A, University of
North Carolina, Chapel Hill, North Carolina 27514.
1982; Smith & Glass, 1977) were recently complemented by
Casey and Herman's (1985) meta-analysis of studies with children
aged 12 years and younger. In the 64 studies that included
treatment-control comparisons, the average effect size was ,71,
indicating a reliable advantage for treatment over no treatment.
Although the percentage of boys in the samples was negatively
correlated with outcome, Casey and Herman found no substantial
differences as a function of age or of group versus individual
therapy. Even initial findings showing the superiority of behavioral
over nonbehavioral treatments were judged to be artifactual,
the result of confounding type of treatmeit with outcome
measure characteristics. The findings suggested that child therapy
is demonstrably effective (and about equally so) across age
groups and types of therapy. However, before we can conclude
that age level is unrelated to therapy effects, we must sample
adolescents as well as children. Moreover, as Parloff (1984)
noted, findings suggesting that different therapies work equally
well deserve close scrutiny because their implications are so significant.
Might therapy effects be different for adolescents than for
children? Various cognitive and social developments (see Rice,
1984), including the advent of formal operations (see Piaget,
1970), make adolescents more cognitively complex than children,
less likely to rely on adult authority, and seemingly less
likely to adjust their behavior to fit societal expectations (see
Kendall, Lerner, & Craighead, 1984). Thus, it is possible that
adolescents may be more resistant than children to therapeutic
intervention. On the other hand, adolescents are more likely
than children to comprehend the purpose of therapy and to understand
complex, interactive psychological determinants of behavior,
which could make them better candidates for therapy
than children. Because most child clinicians actually treat both
children and adolescents, a comparison of therapy effects in
542
EFFECTS OF CHILD AND ADOLESCENT PSYCHOTHERAPY 543
these two age groups could have implications for their work. To
effect this comparison, we reviewed studies of youngsters aged
4-18 years.
We also reexamined the question of whether therapy methods
differ in effectiveness. Casey and Herman (1985) found that an
apparent superiority of behavioral over nonbehavioral methods
largely evaporated when they excluded cases in which outcome
measures "were very similar to activities occurring during
treatment" (p. 391). Our own review suggested that many of
the comparisons that were dropped may actually have involved
sound measurement of therapy effects. Consider, for example,
interventions in which phobic children are exposed to models
who bravely approach animals that the phobic children fear.
The most appropriate outcome assessment may well be one that
is similar to therapy activities-behavioral tests of the children's
ability to bravely approach the animals. Certainly, such tests
may be less than ideal; for example, they may involve contrived
circumstances that differ from the clinical situation of primary
interest. However, in many such tests, children who perform
the target behavior may well be displaying not a measurement
artifact but real adaptive behavior. On the other hand, some of
the Casey and Herman (1985) exclusions seem quite appropriate.
For example, researchers who use the Matching Familiar
Figures Test (MFFT) to teach reflectivity should not use the
MFFT to assess outcomes because the ultimate goal is broader
than improved MFFT performance. In this case, the similarity
between treatment activity and outcome measure is inappropriate
because it is unnecessary.
To further examine the issue of behavioral-nonbehavioral
differences, we distinguished between two kinds of comparisons
in which outcome measures were similar to training activities:
(a) comparisons in which such similarity was necessary for a
fair and valid assessment and (b) comparisons in which such
similarity was unnecessary and posed a risk of artifactual findings.
In a key behavioral-nonbehavioral contrast, we included
comparisons in the first group but not those in the
second.
Our third objective was to examine treatment effects as a
function of treated problem. However, we focused not only on
the specific problem categories used in other meta-analyses
(e.g., phobias, impulsivity) but also on the two overarching,
broadband categories most often identified in factor analytic
research (Achenbach, 1982; Achenbach & Edelbrock, 1978):
overcontrolled (e.g., phobias, shyness) and undercontrolled
(e.g., aggression, impulsivity).
Finally, we questioned whether therapy effects differ with
level of therapist training. The evidence thus far is mixed but is
often discouraging for trained therapists in general (see Auerbach
& Johnson, 1977; Parloff, Waskow, & Wolfe, 1978). However,
there is little summary evidence on therapists who work
with young clients in particular, which is an unfortunate gap
given the substantial time and resources invested in the professional
training of child clinicians. We attempted to fill this gap.
Recognizing that the effects of training might differ depending
on client age, on the treatment method used, or on the problem
being treated, we also explored interactions of training with
each of these factors.
Method
Defining Psychotherapy
We defined psychotherapy as any intervention designed to alleviate
psychological distress, reduce maladaptive behavior, or enhance adaptive
behavior through counseling, structured or unstructured interaction,
a training program, or a predetermined treatment plan. We excluded
approaches involving drug administration, reading material
only (bibliotherapy), teaching or tutoring only to increase knowledge of
a specific subject, moving youngsters to a new living situation (e.g., a
foster home), and efforts solely to prevent problems to which youngsters
were deemed at risk.
We did not require that psychotherapy be conducted by fully trained
professionals. Some schools of thought hold that extensive professional
training is not required for effective interventions but that parents,
teachers, or siblings may function as effective change agents. This suggested
to us that, rather than prejudge the issue, we should treat the
impact of therapist training as an empirical question in the analysis.
Literature Search
Several approaches were combined to identify relevant published
studies. A computer search was carried out using 21 psychotherapyrelated
key words and synonyms, and the resulting data were crossed
with appropriate age group and topic constraints.1 This helped us to
rule out the myriad of articles that merely described or advocated certain
therapy methods. The initial pool of 1,324 articles was reduced in
stepwise fashion by using title, abstract, and method-section information
to select only those studies that met our inclusion requirements.
We used three other procedures to enhance the comprehensiveness of
our survey. First, all articles cited in the meta-analyses by Smith, Glass,
and Miller (1980) and by Casey and Herman (1985) were surveyed and
included if they met our selection criteria. Second, Psychological Abstracts
entries from January 1970 to September 1985 were searched by
hand. Third, four journals accounting for the majority of appropriate
articles in the preceding steps were searched, issue by issue, for the same
time period. These were Behavior Therapy, the Journal of Abnormal
Psychology, the Journal of Consulting and Clinical Psychology, and the
Journal of Counseling Psychology. The result of the search was a pool
of 108 controlled studies of psychotherapy outcomes among children
and adolescents that met our criteria.2 The psychological literature accounted
for more of the studies than the psychiatry, social work, or nursing
literature. Of the 108 studies, 24 (22%) were also included in the
Smith et al. (1980) analysisand 32 (30%) were also included in the Casey
and Herman (1985) analysis.
Subject Population
The studies focused on the prekindergarten through secondary school
age range (i.e., 4-18 years). Across the 108 studies, mean age was 10.23
(SD = 4.00). Approximately 66% of the youngsters sampled were male.
1 The 21 psychotherapy key words and synonyms were client-centered,
contract- (ing, systems, etc.), counseling, cotherapy, dream analysis,
insight-, intervention-, model-, modinca-, operant-, paradox-, psychoanaly-,
psychodrama-, psychothera-, reinforce-, respondent, roleplaying,
therap-. training, transactional, and treatment. The age group
constraints were adolescen-, child-, juvenile-, pre-adolescen-, and
youth-. The evaluation-oriented topic constraints were assess-, comparison,
effect-, efficacy, evaluat-, influence, impact, and outcome.
2 A list of the studies included in this meta-analysis is available from
the first author for a $5 fee to cover printing, postage, and handling.
544 WEISZ, WEISS, ALICKE, AND KLOTZ
To be consistent with other reviews (e.g., Casey & Berman, 1985; Smith
& Glass, 1977), we included studies focusing on a broad variety of psychological
or adjustment problems. However, we excluded mental retardation;
underdeveloped reading, writing, or knowledge of specific
school subjects; problems involving seizures; and physically disabling
handicaps. Problems that have been attributed to physiological causes
but for which etiology has not been well-established (e.g., attention
deficit, hyperactivity, and learning disability) were included provided
that a behavioral or psychological problem (e.g., impulsivity) was actually
addressed in treatment.
Design and Reporting Requirements
We required that a study compare a treated group with an untreated
or minimally treated control group and that the control condition provide
little more than attention to the youngsters. We classified control
groups that had provided alternate treatment or one element of a full
treatment package as treatment groups. If such treatment constituted
the only control condition included, the study was dropped. We also
excluded studies that used subjects as their own controls in single-subject
or within-subject designs. Such studies generate an unusual form
of effect size (e.g., based on intrasubject variance, which is not comparable
to conventional variance statistics) and do not appear to warrant
equal weighting with studies that include independent treatment and
control samples
Classification and Coding Systems
Studies were coded for sample, treatment, and design characteristics,
with some coding systems patterned after those of Casey and Berman
(1985).3 Coding and effect size calculation were carried out independently
to avoid bias. One fourth of the studies were randomly selected
for independent coding by two judges.
Treatment approaches. Treatment methods were classified using the
three-tiered system shown in Table 1. Tier 1 included the broad categories
of behavioral and nonbehavioral. Tier 2 included the subcategories
(e.g, respondent procedures) grouped within each Tier 1 category. Tier 3
included fine-grained descriptors (e.g., extinction); only the behavioral
studies could be classified this finely. Despite what were often quite limited
descriptions of treatment methods, the two raters achieved kappas
of .74, .71, and .78 on Tiers 1, 2, and 3, respectively. Two of the 163
comparisons (1%) were described too vaguely to be coded. We also
coded treatment approaches as group-administered or individually-administered
(« = .92).
Target problem. Treated problems were coded using the two-tiered
system shown in Table 3. At the most general level, problems were
grouped into the two broadband categories most often identified in factor
analyses of child and adolescent behavior problems: undercontrolled
(e.g., aggressive, acting out, or externalizing behavior) and overcontrolled
(e.g., shy or withdrawn, phobic, or internalizing behavior; see
Achenbach & Edelbrock, 1978). Problems not fitting either category
were coded as other. The second tier consisted of descriptive subcategories
(e.g., shy, withdrawn; phobias, specific fears). The two raters
achieved kappas of .94 and .86 for Tiers 1 and 2, respectively. In 9 of
the 163 group comparisons (6%), the problems were described too
vaguely to be coded.
Outcome measures We used Casey and Herman's (1985) system to
code whether outcome measures were similar to treatment activities
(our K = .82). As noted previously, we also carried out one further coding
of outcome measures that were rated similar to treatment activities:
We coded for whether the similarity was necessary for a fair assessment
(given the treatment goals) or unnecessary (K = .81). We also used Casey
and Herman's (1985) category systems for coding outcome measures
Table 1
Mean Effect Size for Each Therapy Type
No.
treatment
Therapy type groups
Behavioralb
Operant
Physical reinforcers
Consultation in operant methods
Social/verbal reinforcement
Self-reinforcement
Combined physical and verbal
reinforcement
Multiple operant methods
Respondent
Systematic desensitization
Relaxation (no hierarchy)
Extinction (no relaxation)
Combined respondent
Modeling
O'Connor film0
Live peer model
Live nonpeer model
Nonlive peer model
Nonlive nonpeer model
Social skills training
Cognitive/cognitive behavioral
Multiple behavioral
Nonbehavioral "
Client-centered/nondirective
Insight-oriented/psychodynamic
Discussion group
39
8
16
6
2
3
2
17
8
3
3
2
25
4
6
3
92 5
10
10
20
3
4
Effect
size
.78
.92
.77
.78
.33
.75
.72
.75
.65
.43
1.46
.43
1.19
2.90
1.25
.62
.79
.29
.90
.68
1.04
.56
.01
.18
P"
.0001
.07
.0001
.002
.40
.01
.07
.0002
.01
.05
.14
.39
.003
.27
.02
.26
.0001
.21
.04
.0004
.0002
.0001
.98
.18
Note. Because some descriptions of treatment methods were too vague
to be classified, not all Ns sum as expected.
a The probability that a particular effect size is greater than zero reflects
the variability of effect sizes across the category of comparisons being
sampled in addition to the number of studies and the mean effect size.
Thus, where effect sizes are quite variable across a category, the p value
may be low despite a high mean effect size and a substantial pool of
studies. A Bonferroni correction applied to the tests in this table indicated
that probability values ^ .002 should not be regarded as statistically
significant.
bN= 126; effect size = . 88; p<. 0001.
c The O'Connor film shows a child engaging in social entry behavior
One study using this film reported an unusually high effect size (9.5);
with this study dropped, effect size = .77.
d N = 28; effect size = .42; p < .0001.
into type and source. Seven types were included, and five types occurred
with sufficient frequency to warrant kappa calculations. The categories
were (a) fear/anxiety (K - .98); (b) cognitive skills (« - .87); (c) global
adjustment (K = .83); (d) social adjustment (K = .91); (e) achievement
(including school grades and achievement test scores; * = .98) (f) personality
(including scales measuring attitudes and beliefs); and (g) self-
3 We used Casey and Berman's (1985) coding schemes for (a) whether
outcome measures were similar to activities occurring during treatment,
(b) type of outcome measure, and (c) source of outcome measure.
In addition, we distinguished between behavioral and nonbehavioral
therapies, as did Casey and Berman, but we also distinguished among
subtypes within each broad category.
EFFECTS OF CHILD AND ADOLESCENT PSYCHOTHERAPY 54S
Table 2
Mean Effect Size for Behavioral and
Nonbehavioral Treatments
Nonbe-
Behavioral
Analysis M
All comparisons .88
Omitting therapy-like
outcomes .6 1
Omitting unnecessary
therapy-like outcomes .93
N
126
34
94
havioral
M
.44
.51
.45
Nt
27 2.14
22 .64
24 2.09
P
<.05
.52
<.05
Note. All six effect size means were significantly different from zero (all
ps < .0006).
concept. The kappa calculation for the full system was .90. Casey and
Berman coded eight sources of outcome measures in their system: (a)
observers (* = .90); (b) therapists; (c) parents (« = .96); (d) subject's own
performance (K = .9 l);(e) expert judges; (f) peers (a = . 90); (g) teachers
(>: = .97); and (h) self-report by subjects (* = .96). The kappa calculation
for the full system was .92.
Therapist training. We classified therapists according to level of training.
Levels included (a) professionals who held a doctor's or master's
degree in psychology, education, or social work; (b) graduate students
who were working toward advanced degrees in psychology, education,
or social work; and (c) paraprofessionals who were parents, teachers, or
others lacking mental-health-related graduate training but trained to
administer the therapy.
Calculation of Effect Sizes
Effect sizes were estimated using the procedures of Smith et al. (1980,
Appendix 7). In each calculation, the mean posttherapy treatment
group-control group difference was divided by the control group standard
deviation.* Most studies in our pool included multiple outcome
measures, and a number of studies included more than one treatment
condition. Thus, in most cases, a study initially produced numerous
possible effect sizes. To retain all effect sizes in our analysis would have
resulted in disproportionate weighting of those studies with the most
measures and groups. Several solutions to this problem are available
(e.g., Glass, McGaw, & Smith, 1981). We chose to collapse effect sizes
across outcome measures except in analyses comparing such measures.
However, the comparison of treatment procedures was central to the
overall analysis, and separate treatments within studies appeared sufficiently
independent to warrant separate attention. Consequently, for
each study we computed one effect size estimate for each of the treatment
conditions included. The 108 studies included an average of 1.54
therapy conditions for a total of 163 effect sizes over the entire pool.
Some studies included both follow-up and posttreatment assessments
of therapy effects. We describe follow-up findings separately in a final
section.
Results
Overview of Procedures for Analysis
Two problems often present in meta-analyses are that most
group comparisons involve tests of main effects alone and that
key variables (e.g., treatment method and treated problem) are
confounded (see Glass & Kliegl, 1983; Mintz, 1983). The few
cases that have attempted more controlled comparisons have
often dropped substantial portions of available data. Here, we
selected an approach intended to use all the data available while
avoiding undue risk of either Type I or Type II error. Minimizing
Type II error is particularly important in meta-analyses
given its potential heuristic, hypothesis-generating value.
The first wave of analysis focused planned comparisons on
our four factors of primary interest; age group, therapy type,
target problem type, and therapist training. For these analyses,
we tested the four main effects. We then tested each main effect
for its robustness, using (a) general linear models (GLM) procedures
that eliminated (i.e., controlled for) the effects of each of
the other three factors (see Appelbaum & Cramer, 1974) and
(b) tests of whether any of the main effects were qualified by
interactions with any of the other three factors.5 We also tested
the robustness of the therapy type effect using the Casey and
Berman (1985) procedure followed by our own revised procedure.
For all other group comparisons, we applied a Bonferroni
correction (Neter & Wasserman, 1974), which set the alpha at
.01. Bonferroni corrections were also applied to each family of
tests to compare obtained effect sizes to the null hypothesis of
zero (see Tables 1, 3, and 4).
Overall Effect Size
Across the 163 treatment-control comparisons, the mean
effect size was 0.79 (significantly different from zero, p < .0001).
The average treated youngster was placed at the 79th percentile
of those not treated. Of the 163 effect sizes, only 10 (6%) were
negative, indicating an adverse effect of treatment. The mean
effect size of 0.79 was comparable to those reported in earlier
meta-analyses of therapy effects among children (0.71 in Casey
& Berman, 1985), adults (0.93 in Shapiro & Shapiro, 1982),
and mixed age groups (0.68 in Smith & Glass, 1977).
Preliminary Check for Sex Effects
Before proceeding to the group comparisons of primary interest,
we checked the relation between effect size and gender
composition of treated groups. For 99 of the treatment-control
comparisons, information was sufficient to reveal whether a
majority of the treatment group comprised male (A' = 72) or
female (A' = 27) participants. Effect sizes averaged 0.80 for the
male majority groups and 1.11 for the female majority groups
(p value for difference = .33). Studies that did not report gender
composition averaged an effect size of 0.55.
4 Some researchers (e.g., Casey & Berman, 1985; Hedges, 1982) favor
the use of a pooled treatment and control group standard deviation as
denominator. If one consequence of therapy is an increase in behavioral
variability, as some researchers have suggested (e.g., Bergin & Lambert,
1978), such pooling can cause interpretational and statistical problems
(see Smith et al., 1980), which we sought to avoid.
* We considered using full factorial model ANOVAS, but cell sample
sizes were too unbalanced to yield interpretable results and some were
so small that tests of 3-way and 4-way interactions were impossible.
546 WEISZ, WEISS, ALICKE, AND KLOTZ
Age Level
A question of primary interest was whether the impact of
therapy differed for children and adolescents. The mean effect
size for the 98 treatment-control comparisons involving children
(ages 4-12 years) was 0.92 (82nd percentile), which was
significantly larger than the mean of 0.58 (72nd percentile) for
the 61 comparisons involving adolescents (ages 13-18 years),
((157) = 2.17, p < .05. The correlation between age and effect
size was -0.21 (p < .01) over the entire sample; the coefficient
was —0.17 (p < .10) for effect sizes involving children and 0.15
(ns) for those involving adolescents.
We next tested the robustness of the age group differences by
using eliminating tests. The age effect was reduced slightly when
therapy type (behavioral vs. nonbehavioral) was controlled
(p = .084) and when problem type (overcontrolled vs. undercontrolled)
was controlled (p = .086). Both reductions were
caused partly by reduced sample size because not all treatments
or target problems could be coded. However, the age group
difference grew more reliable statistically when therapist training
was controlled (p = .013).
A series of 2 X 2 analyses of variance (ANOVAS) testing interactions
of age with therapy type, problem type, and therapist
training, respectively, revealed no significant effects (all Fs <
1.8; all ps > . \ 5). To provide the most thorough assessment, we
carried out GLM ANOVAS on the same interactions, with age
entered as a continuous variable. Neither therapy type nor
problem type interactions were significant (both ps > .30), but
the Age X Therapist Training interaction was significant, F(2,
102) = 3.49, p < .05. Age and effect size were uncorrelated
among professionals (N = 39, r = 0.11, p < .50) but were negatively
correlated among graduate students (N = 43, r = —0.31,
p < .05) and paraprofessionals (N = 26, r = —0.43, p < .05).
Trained professionals were about equally effective with all ages,
but graduate students and paraprofessionals were more effective
with younger than with older clients.
Therapy Type
Behavioral versus nonbehavioral. Table I shows that mean
effect size was higher for behavioral than for nonbehavioral
treatments, r(152) = 2.14, p < .05. The difference remained
significant after eliminating tests controlled for age (p < .05),
problem type (p < .05), and therapist training (p < .05). Interactions
of therapy type with problem type and with therapist
training were not significant (both h's < 0.50, both ps > .65).
Similarity of therapy procedures and outcome measures.
Next, we examined the main effect for therapy type, followingup
on Casey and Herman's (1985) analysis. We first excluded
comparisons involving an outcome measure similar to treatment
procedures. Consistent with Casey and Herman's (1985)
finding, the procedure reduced the behavioral-nonbehavioral
difference to nonsignificance, ;(55) = 0.64, p = .52. As we noted
earlier, the Casey and Herman procedure may rule out some
carefully designed studies in which measures similar to the
training procedures are appropriate and necessary for a fair test
of treatment success. To correct for this apparent limitation,
we again compared behavioral and nonbehavioral methods and
Table 3
Mean Effect Size for Each Target Problem
Target problem
Undercontrolled
Delinquency
Noncompliance
Self-control (hyperactivity,
impulsivity)
Aggressive/undisciplined
Overcontrolled
Phobias/anxiety
Social withdrawal/isolation
Other
Adjustment/emotional
disturbance
Underachievement
No. treatment
groups
76
19
9
31
17
67
39
28
18
9
9
Effect
size
.79
.66
1.33
.75
.74
.88
.74
1.07
.56
.69
.43
If
.0001
.0004
.005
.0001
.0002
.0001
.0001
.002
.0002
.001
.046
Note. Because some problem descriptions were too vague to be classified,
not all Ns sum as expected.
' A Bonferroni correction applied to this table indicated that probability
values s^ .005 should not be regarded as statistically significant.
restored to the sample all treatment-control comparisons in
which similarity of training and assessment methods was
judged by our raters to be necessary for a fair test. In this analysis,
as Table 2 shows, behavioral-method comparisons showed a
significantly larger effect size than nonbehavioral comparisons,
r(117) = 2.09,p<.05.
Specific therapy types. The ANOVAS focused on Tier 2 of the
therapy-type coding system (see Table 1) revealed no significant
difference between the behavioral subtypes (e.g., operant, modeling)
or the nonbehavioral subtypes (e.g., psychodynamic, client
centered), all Fs < 1.9; all ps > . 15. The comparison of nonbehavioral
subtypes should be interpreted with caution: The
majority of these studies used client-centered therapy and only
three used insight-oriented psychodynamic therapy. The
ANOVAS focused on Tier 3 revealed no significant differences
between the therapies within each subtype. As Table 1 shows,
effect sizes for most of the categories within each tier were significantly
greater than zero.
Target Problem
Overcontrolled versus Undercontrolled problems. Next, we focused
on the problems for which the youngsters were treated.
There was no significant difference between the broad categories
of overcontrolled and undercontrolled problems (p = .46;
see Table 3). This continued to be true when eliminating tests
were used to control for age level (p = .59), therapy type (p =
.67), and therapist training (p = .67).
However, problem type and therapist training did interact,
F(2, 90) = 2.93, p = .059. Tests of simple effects revealed no
significant differences in effect size between overcontrolled and
undercontrolled problems at any of the three levels of therapist
training. Nor did the three therapist groups differ in their success
with undercontrolled youngsters. The groups did differ,
however, in their effectiveness with overcontrolled youngsters:
As amount of formal training increased, so did effectiveness,
EFFECTS OF CHILD AND ADOLESCENT PSYCHOTHERAPY 547
Table 4
Effect Size as a Function of Source of Outcome Measure
Source
Observers
Parents
Teachers
Subject performance
Subject report
Peers
N
62
19
48
128
61
18
Effect
size
1.08
.66
0.68
0.65
0.49
0.33
If
.0001
.0001
.0001
.0001
.0001
.01
" A Bonferroni correction applied to this table indicated that probability
values > .008 should not be regarded as statistically significant.
linear trend F\ 1, 35) = 4.68, p < .05. Professionals achieved a
mean effect size of 1.03, graduate students of 0.71, and paraprofessionalsofO.
53.
Specific problem types. The categories in Tier 2 of the problem
coding scheme (e.g., delinquency, specific fears) did not
differ significantly in effect size (p = .41). However, as Table 3
shows, effect sizes were reliably greater than zero for most problem
categories.
Therapist Training
Although therapist training entered into two interactions, its
main effect was not significant (p = .43). This continued to be
true when eliminating tests were used to control for age (p =
.56), therapy type (p = .65), and problem type (p = .51).
Other Findings
Individual versus group therapy. Does it matter whether
youngsters are treated individually or in groups? Our data revealed
somewhat larger effect sizes for therapy that was individually
administered rather than group administered (M = 1.04
and M = 0.62, respectively), but the difference did not attain
significance under our Bonferonni correction procedure
(/- = .03).
Source and content of outcome measure. Casey and Herman
(1985) found significant differences as a function of the source
of outcome measures (e.g., observers, teachers). Measures derived
from observers revealed the largest difference between the
treated and control groups. Using their category system (with
two low-frequency categories dropped), we also found a main
effect for source, ^5, 330) = 4.00, p < .005. Newman-Keuls
tests indicated that observers reported more change than any of
the other sources (all ps <.05), none of which differed significantly
from one another (see Table 4).
Casey and Berman also reported significant differences as a
function of the content of outcome measures (e.g., fear and anxiety,
cognitive skills). We used their category system but failed
to find a significant main effect (p = .61). We dropped two low
frequency categories, but the main effect remained nonsignificant
(p = .42).
Clinic-referred versus analog samples. Outcome research,
and meta-analyses of such research, have been criticized for excessive
reliance on analog samples, that is, samples that have
been recruited by researchers specifically for treatment studies
rather than samples that have been spontaneously referred by
clinics (e.g., Parloff, 1984; Shapiro & Shapiro, 1982). Combining
outcome results from analog and clinic samples makes it
difficult to gauge the relevance of findings to actual clinic practice.
For this reason, we separated the 126 comparisons involving
analog samples from the 37 comparisons involving true
clinical samples. Mean effect sizes were 0.76 for analog samples
and 0.89 for clinical samples: The difference was not significant
(F< 1).
Follow-up findings: Do therapy effects last? The preceding
findings suggest that therapy does have positive effects that are
measurable at the end of therapy. To have real practical value,
however, the effects must be durable and must persist beyond
treatment termination. To assess the durability of therapy
effects, we analyzed the follow-up treatment-control comparisons
contained in our sample; these follow-ups averaged 168
days subsequent to termination of treatment. Average effect size
at follow-up (0.93) was actually larger than effect size immediately
after treatment (0.79), although the difference was not significant
(p = .45). When we included in the posttreatment
group only those studies that also had a follow-up assessment,
the means for posttreatment and follow-up were identical
(0.93). Thus, the effects of therapy in this pool of studies appear
to be durable.
Discussion
Is psychotherapy effective with children and adolescents? The
findings reviewed here suggest that it is. After treatment, across
multiple measures of adjustment, the average treated youngster
was functioning better than 79% of those not treated. However,
the benefits of therapy depended to some extent on the age level
of the youngsters treated, with children profiting more than adolescents.
This is consistent with the idea, suggested earlier, that
cognitive changes such as the advent of formal operations (Piaget,
1970), and other quasicognitive changes (Perlmutter, 1986;
Rice, 1984), may make adolescents less responsive than children
to therapeutic influence; enhanced powers of reasoning
may strengthen adolescents' convictions regarding their own
behavior and may also make them adept at circumventing or
sabotaging a therapist's efforts. Of course, it is also possible that
outcome measures used with adolescents are less sensitive to
change than most child measures or that adolescents' problems
are more entrenched (see Kendall et al., 1984, for further ideas
about age group differences and their sources).
Only paraprofessionals and graduate student therapists were
more effective with younger than older children; trained professionals
were about equally effective with younger and older clients.
Taken together, the age main effect and the Age X Training
interaction suggest an intriguing possibility. It may be that adolescents,
for cognitive or other reasons, are generally more
difficult than children to treat successfully, but formal training
may provide professionals with sufficient therapeutic acumen
to override age differences in initial treatability.
Our findings on therapy type do not support the often noted
summary of adult psychotherapy findings that different forms
548 WEISZ, WEISS, ALICKE, AND KLOTZ
of therapy work about equally well (for a critical discussion, see
Parloff, 1984; see also Frank's, 1973, "nonspecificity hypothesis").
Instead, we found that behavioral methods yielded significantly
larger effects than nonbehavioral methods. This finding
held up across differences in age level, treated problem, and
therapist experience, and it was not qualified by interactions
with any of these factors. The behavioral-nonbehavioral
difference was reduced to nonsignificance when we excluded
all therapy-like outcome measures (following Casey & Herman,
1985) but was readily revived when we excluded only the unnecessary
therapy-like measures that seemed likely to produce artifactual
findings. Overall, the findings make a case for the superiority
of behavioral over nonbehavioral approaches.
On the other hand, we found comparatively few controlled
studies assessing nonbehavioral methods; it might be argued
that these studies do not represent the best of nonbehavioral
approaches. In fact, many would argue that nonbehavioral approaches
are best suited to the only partly scrutable process of
unraveling complex causal dynamics and of stimulating insight
over months or even years of treatment. By contrast, most controlled-
outcome research focuses on relatively specific target
problems, directly observable outcome measures, and brief
treatment. Do such studies miss the point of nonbehavioral intervention?
Perhaps, but the present findings seem to place the
burden of proof on those who make that argument.
Those familiar with the evidence on the long-term stability of
undercontrolled behavior and the relative instability of overcontrolled
behavior (reviewed in Robins, 1979) may have been surprised
to find that therapy effects were no more pronounced
with the latter than the former. Note, though, that the evidence
on long-term stability concerns the tendency of problems to
persist or dissipate over time, independent of therapy. Our review,
by contrast, focuses on the persistence of various problems
in treated groups relative to control groups. When the natural
dissipation of problems over time is thus held constant, our
findings suggest that undercontrolled problems may be no more
intractable than overcontrolled problems.
Our failure to find an overall difference in effectiveness between
professionals, graduate students, and paraprofessionals
might be disquieting to those involved in clinical training programs
(see also Auerbach & Johnson, 1977; Parloff, Waskow,
& Wolfe, 1978). A simplistic interpretation might suggest that
training does not enhance therapeutic effectiveness. A more
thoughtful evaluation, though, suggests otherwise. First, it
should be noted that the therapeutic work of the graduate students
and paraprofessionals did not take place in a vacuum: In
nearly every instance, these therapists were selected, trained,
and supervised by professionals in techniques that professionals
had designed. Thus, the success enjoyed by the two less clinically
trained groups might actually have reflected the judgment
and skill of professionals working behind the scenes.
Moreover, the finding of no overall difference was qualified
by two important interactions, both suggesting possible benefits
of training. A Training X Age interaction suggested that professional
training may enhance therapist effectiveness with older,
more difficult-to-treat children. And a Training X Problem
Type interaction suggested that training may enhance therapist
effectiveness in treating overcontrolled problems. On a more
negative note, this interaction suggested that training may have
little impact on therapists' effectiveness with undercontrolled
problems. Why? One possibility is that youngsters with undercontrolled
problems are responsive to interventions that are relatively
easy to learn; some of these interventions may be similar
to natural, naive, parent-like responses that arise in situations
requiring discipline and rule enforcement.
Two findings were particularly encouraging. The first revealed
that therapy studies with analog samples yielded results
quite similar to studies with true clinic-referred samples. Thus,
we found no evidence that the positive findings generated by
analog therapy studies presented an inflated or otherwise distorted
picture of therapy effects among children and adolescents.
A second source of encouragement was the finding that
the impact of therapy assessed at immediate posttest was not
reliably different from the impact assessed at follow-up, which
occurred an average of 6 months later. This is consistent with
other findings across a broader age range (reviewed by Nicholson
& Herman, 1983) indicating that therapy effects may be relatively
enduring.
Here, and in most of the other findings, there is reason for
optimism about therapy effects with children and adolescents.
On the other hand, the number of available outcome studies is
still much too modest to permit a definitive analysis. A welldeveloped
understanding of therapy outcomes within the broad
range sampled here will require the continued efforts of our best
researchers.
References
Achenbach, T. M. (1982). Developmental psychopathology (2nd ed.).
New York: Wiley.
Achenbach, T. M., & Edelbrock, C. S. (1978). The classification of child
psychopathology: A review and analysis of empirical efforts. Psychological
Bulletin, 85, 1275-1301.
Appelbaum, M. I., & Cramer, E. M. (1974). Some problems in the nonorthogonal
analysis of variance. Psychological Bulletin, SI, 335-343.
Auerbach, A. H., & Johnson, M. (1977). Research on the therapist's
level of experience. In A. S. Gurman & A. M, Razin (Eds.), Effective
psychotherapy: A handbook of research (pp. 84-99). New York: Pergamon
Press.
Barrett, C. L., Hampe, I. E., & Miller, L. (1978). Research on psychotherapy
with children. In S. L. Garfield & A. E. Bergin (Eds.), Handbook
of psychotherapy and behavior change: An empirical analysis
(pp. 411^135). New York: Wiley.
Bergin, A. E., & Lambert, M. J. (1978). The evaluation of therapeutic
outcomes. In S. L. Garfield & A. E. Bergin (Eds.), Handbook of psychotherapy
and behavior change: An empirical analysis (pp. 139-
189). New York: Wiley.
Casey, R. J., & Berman, J. S. (1985). The outcome of psychotherapy
with children. Psychological Bulletin, 98, 388-400.
Frank, J. D. (1973). Persuasion and healing. Baltimore, MD: Johns
Hopkins University Press.
Gelfand, D. M., Jenson, W. R., & Drew, C. J. (1982). Understanding
child behavior disorders. New "York: Holt, Rinehart, & Winston.
Glass, G. V., & Kliegl, R. M. (1983) An apology for research integration
in the study of psychotherapy. Journal of Consulting and Clinical Psychology,
51. 28-41.
Glass, G. V., McGaw, B., & Smith, M. L. (1981). Mela-analysis in social
research. Beverly Hills, CA: Sage.
EFFECTS OF CHILD AND ADOLESCENT PSYCHOTHERAPY 549
Hedges, L. V. (1982). Estimation of effect size from a series of independent
experiments. Psychological Bulletin, 92, 490-499.
Kendall, P. C., Lerner, R. M., & Craighead, W. E. (1984). Human development
and intervention in childhood psychopathology. Child Development,
55, 71-82.
Mintz, J. (1983). Integrating research evidence: A commentary on metaanalysis.
Journal of Consulting and Clinical Psychology, 51, 71-75.
Neter, J., & Wasserman, W. (1974). Applied linear statistical models.
Homewood, IL: Irwin.
Nicholson, R. A., & Herman, J. S. (1983). Is follow-up necessary in
evaluating psychotherapy? Psychological Bulletin, 93, 261-278.
Parloff, M. B. (1984). Psychotherapy research and its incredible credibility
crisis. Clinical Psychology Review, 4, 95-109.
Parloff, M. B., Waskow, I. E., & Wolfe, B. E. (1978). Research on therapist
variables in relation to process and outcome. In S. L. Garfield &
A. E. Bergin (Eds.), Handbook of psychotherapy and behavior change:
An empirical analysis (pp. 233-274). New \brk: Wiley.
Perlmutter, M. (Ed.). (1986). Cognitive perspectives on children's social
and behavioral development: The Minnesota symposia on child psychology
(Vol. 18). Hillsdale, NJ: Erlbaum.
Piaget, J. (1970). Piaget's theory. In P. E. Mussen (Ed.), Carrmchael's
manual of child psychology'(3rd ed., Vol. 1, pp. 703-732). New York:
Wiley.
Rice, F. P. (1984). The adolescent: Development, relationships, and culture.
Boston: Allyn and Bacon.
Robins, L. N. (1979). Follow-up studies. In H. C. Quay & J. S. Werry
(Eds.), Psychopalhological disorders of childhood (2nd ed., pp. 483-
513). New York: Wiley.
Shapiro, D. A., & Shapiro, D. (1982). Meta-analysis of comparative
therapy outcome studies: A replication and refinement. Psychological
Bulletin, 92,581-604.
Smith, M. L., & Glass, G. V. (1977). Meta-analysis of psychotherapy
outcome studies. American Psychologist, 32, 752-760.
Smith, M. L., Glass, G. V., & Miller, T. I. (1980). Benefits of psychotherapy.
Baltimore, MD: Johns Hopkins University Press
Received July 31, 1986
Revision received December 1, 1986
Accepted December 1, 1986 •
Low Publication Prices for APA Members and Affiliates
Keeping You Up-to-Date
All APA members (Fellows, Members, and Associates) receive — as part of their annual dues — subscriptions to
the American Psychologist, the APA Monitor, and Psychology Today.
High School Teacher and Student Affiliates receive subscriptions to the APA Monitor and Psychology Today, and
they can subscribe to the American Psychologist at a significantly reduced rate.
In addition, all members and affiliates are eligible for savings of up to 50% on other APA journals, as well as
significant discounts on subscriptions from cooperating societies and publishers (e.g., the British Psychological
Society, the American Sociological Association, and Human Sciences Press)
Essential Resources
APA members and affiliates receive special rates for purchases of APA books, including the Publication
Manual of the APA. the Master Lectures, and APA's Guide to Research Support.
Other Benefits of Membership
Membership in APA also provides eligibility for low-cost insurance plans covering life: medical and income
protection: hospital indemnity: accident and travel; Keogh retirement; office overhead: and student/school.
professional, and liability.
For more information, write to American Psychological Association.
Membership Records. 1200 Seventeenth Street NW. Washington. DC 20036

Kazdin, Alan E., Evidence-based treatment and practice: New opportunities to bridge
clinical research and practice, enhance the knowledge base, and improve patient care.
American Psychologist, Vol 63(3), Apr 2008, 146-159.
Abstract
The long-standing divide between research and practice in clinical psychology has
received increased attention in view of the development of evidence-based interventions
and practice and public interest, oversight, and management of psychological services.
The gap has been reflected in concerns from those in practice about the applicability of
findings from psychotherapy research as a guide to clinical work and concerns from those
in research about how clinical work is conducted. Research and practice are united in
their commitment to providing the best of psychological knowledge and methods to
improve the quality of patient care. This article highlights issues in the research- practice
debate as a backdrop for rapprochement. Suggestions are made for changes and shifts in
emphases in psychotherapy research and clinical practice. The changes are designed to
ensure that both research and practice contribute to our knowledge base and provide
information that can be used more readily to improve patient care and, in the process,
reduce the perceived and real hiatus between research and practice. (PsycINFO Database
Record (c) 2010 APA, all rights reserved)

Albert J. Bellg, Belinda Borrelli, Barbara Resnick, Jacki Hecht, Daryl Sharp Minicucci,
Marcia Ory, Gbenga Ogedegbe, Denise Orwig, Denise Ernst, Susan Czajkowski
Treatment Fidelity in Health Behavior Change Studies: Best
Practices and Recommendations From the NIH Behavior Change Consortium
Appleton Heart Institute
Brown Medical School
University of Maryland
Brown Medical School
University of Rochester
National Institutes of Health
Cornell University
University of Maryland
University of New Mexico
National Institutes of Health
(For the Treatment Fidelity Workgroup of the NIH Behavior Change Consortium)
Treatment fidelity refers to the methodological strategies used to monitor and enhance the reliability and
validity of behavioral interventions. This article describes a multisite effort by the Treatment Fidelity
Workgroup of the National Institutes of Health Behavior Change Consortium (BCC) to identify treatment
fidelity concepts and strategies in health behavior intervention research. The work group reviewed
treatment fidelity practices in the research literature, identified techniques used within the BCC, and
developed recommendations for incorporating these practices more consistently. The recommendations
cover study design, provider training, treatment delivery, treatment receipt, and enactment of treatment
skills. Funding agencies, reviewers, and journal editors are encouraged to make treatment fidelity a
standard part of the conduct and evaluation of health behavior intervention research.
Key words: treatment fidelity, health behavior, translational research, reliability, validity
Treatment fidelity refers to the methodological strategies used to
monitor and enhance the reliability and validity of behavioral interventions.
It also refers to the methodological practices used to ensure
that a research study reliably and validly tests a clinical intervention.
Although some strategies to enhance treatment fidelity in research
may be quite familiar (e.g., the use of treatment manuals, videotape
monitoring of therapist adherence to research protocols, and testing
subject acquisition of treatment skills), there is inconsistency in their
use, particularly in health behavior intervention research. Methodological
procedures for preserving internal validity and enhancing
external validity in studies, though critical to the interpretation of
findings, are not emphasized in research-training curricula, and their
relative lack of perceived importance is also evidenced by the scant
reporting of treatment fidelity practices in journal articles. By comparison,
procedures for evaluating the reliability and validity of questionnaires
and other measurement instruments are well understood.
Our purpose in this article is to provide a useful conceptualization of
treatment fidelity, describe specific treatment fidelity strategies, and
offer recommendations for incorporating treatment fidelity practices
in health behavior intervention research. We believe that adopting
these practices will contribute to the continued development of innovative,
credible, and clinically applicable health behavior interventions
and programs.
The concept of treatment fidelity has evolved over time. Although
treatment fidelity was mentioned in a few social and
behavioral studies in the late 1970s and early 1980s (e.g., Peterson,
Homer, & Wonderlich, 1982; Quay, 1977), Moncher and Prinz’s
Albert J. Bellg, Appleton Cardiology Associates, Appleton Heart Institute,
Appleton, Wisconsin; Belinda Borrelli and Jacki Hecht, Center for
Behavioral and Preventive Medicine, Brown Medical School; Barbara
Resnick, School of Nursing, University of Maryland; Daryl Sharp Minicucci,
School of Nursing, University of Rochester; Marcia Ory, National
Institute on Aging, National Institutes of Health (NIH); Gbenga Ogedegbe,
Weill Medical College, Cornell University; Denise Orwig, School of
Medicine, University of Maryland; Denise Ernst, Department of Family
Practice, University of New Mexico; Susan Czajkowski, National Heart,
Lung, and Blood Institute (NHLBI), NIH.
Marcia Ory is now at the Department of Social and Behavioral Health,
School of Rural Public Health, Texas A&M University. Gbenga Ogedegbe
is now at the Department of Medicine, College of Physicians and Surgeons,
Columbia University.
Senior authorship is shared equally between Albert J. Bellg and Belinda
Borrelli. Funding for this multisite project was provided by NIH/NHLBI
Grant R01 HL62165 to Belinda Borrelli (principal investigator). We would
like to thank all the principal investigators and staff members of the
Behavior Change Consortium who contributed to this article by identifying
treatment fidelity practices used in their studies.
Correspondence concerning this article should be addressed to Albert J.
Bellg, Appleton Heart Institute, 1818 North Meade Street, Appleton, WI
54911. E-mail: albert.bellg@thedacare.org
Health Psychology Copyright 2004 by the American Psychological Association
2004, Vol. 23, No. 5, 443–451 0278-6133/04/$12.00 DOI: 10.1037/0278-6133.23.5.443
443
(1991) article was the first to formally introduce a definition and
propose guidelines for the enhancement of treatment fidelity. Prior
to Moncher and Prinz’s article, treatment fidelity was generally
considered as treatment integrity—that is, whether the treatment
was delivered as intended. Moncher and Prinz added the concept
of treatment differentiation, or whether the treatment conditions
differed from one another in the intended manner (Kazdin, 1986).
Subsequently, Lichstein, Riedel, and Grieve (1994) argued that
two additional processes needed to be assessed in order to properly
interpret the results of studies: (a) treatment receipt, which involves
both assessing and optimizing the degree to which the
participant understands and demonstrates knowledge of and ability
to use treatment skills, and (b) treatment enactment, which involves
assessing and optimizing the degree to which the participant
applies the skills learned in treatment in his or her daily life.
They considered treatment delivery, receipt, and enactment to
constitute a full treatment implementation model (Burgio et al.,
2001; Lichstein et al., 1994).
Lichstein and colleagues (1994) used a medical example to
illustrate these different components. Assessment of whether a
physician wrote a prescription (delivery) is inadequate for ensuring
that the treatment has been implemented as intended. To receive an
active dose of the treatment, the patient must then fill the prescription
(receipt) and take the medicine as prescribed (enactment).
Although enactment is identical to treatment adherence in their
example, there are numerous situations in health behavior research
in which enactment is distinguished from adherence. For instance,
in a Behavior Change Consortium (BCC) study on smoking cessation
for parents of children with asthma, smokers motivated to
quit were given strategies that would help them do so (delivery),
and the strategies were discussed with them to verify that they
understood and could use them (receipt). However, the strategies
may or may not have actually been used (enactment), and if they
were used, they may or may not have led to smoking cessation
(adherence to the treatment recommendation to stop smoking). In
both examples, assessment and potential intervention with therapist
behavior (in relation to treatment delivery) and with patient
behavior (in relation to treatment receipt and enactment) are integral
to maintenance of a study’s reliability and validity.
Rationale for Considering Treatment Fidelity
Treatment fidelity influences a variety of study issues. Questionable
internal and external validity may make it impossible to
draw accurate conclusions about treatment efficacy or to replicate
a study. For example, in evaluating a new intervention, if significant
results were found but fidelity was not monitored and optimized,
one does not know whether the outcome was due to an
effective treatment or to unknown factors that may have been
unintentionally added to or omitted from the treatment (Cook &
Campbell, 1979). If, however, nonsignificant results were found
and the level of treatment fidelity is unknown, one does not know
whether the outcome was due to an ineffective treatment or to lack
of treatment fidelity (Moncher & Prinz, 1991), because internal
validity and effect size are highly correlated (Smith, Glass, &
Miller, 1980). In the latter case, new, potentially effective treatments
may be prematurely discarded, whereas in the former case,
unsuccessful treatments may be implemented and disseminated in
clinical and public health settings at a high cost to patients,
providers, and organizations.
By assessing treatment fidelity, however, investigators can have
greater confidence in their results. If they go a step further and use
quantitative methods for assessment, they can use treatment fidelity
measures in data analyses to determine the extent to which their
results are actually due to the study intervention. For instance, one
might use a measure of nonspecific treatment effects associated with
different therapists (a treatment delivery variable) as a covariate to
better define the effects of the intervention apart from the effects of
the therapists. Treatment fidelity may also be assessed with the goal
of improving the design of a study (Kazdin, 1994). For example, in a
study with poor treatment adherence among participants, if measures
of treatment receipt are found to be associated with adherence, the
study procedures may be redesigned to improve receipt and thereby
provide a better test of the intervention.
By reducing random and unintended variability in a study, improving
treatment fidelity can also improve statistical power. Monitoring
and optimizing treatment fidelity over a series of studies may increase
effect sizes and reduce the number of subjects required in later studies,
thereby decreasing costs and improving the efficacy of an intervention
research program. Even during a single study, optimizing treatment
fidelity increases the chance that investigators will find significant
results. For instance, evaluation of treatment delivery over time might
reveal a drift in interventionist adherence to a smoking cessation
treatment protocol, perhaps warranting retraining of those providing
the intervention to minimize the problem’s impact on the internal
validity of the intervention.
Procedures to maximize treatment fidelity also have implications
for research focusing on theory development, comparison,
and application (Nigg, Allegrante, & Ory, 2002b). Only when
there is a high degree of awareness and control over factors
associated with a study’s internal validity, such as the impact of
nonspecific treatment effects and unintended clinical processes on
an intervention (e.g., a treatment provider’s inadvertent use of a
cognitive procedure in a behavioral protocol) is it possible to
evaluate the efficacy of a theory-based intervention, test a theoretical
question, or compare the impact of two or more theoretical
processes on an outcome. Unless treatment fidelity is explicitly
maintained, the extent to which the theory-based intervention
being tested is the primary mechanism for the observed changes in
the dependent variables of interest will remain unclear.
Finally, treatment fidelity is also a potentially important component
of successful research dissemination. Behavioral medicine practitioners
are often in the position of attempting to implement new procedures
in medical settings where medical and nursing staff have clinical
expertise but limited familiarity with behavioral change research.
Translating effective behavioral change interventions from research
settings to clinical practice can be facilitated when investigators employ
and describe treatment fidelity strategies that can be used as
guidelines for implementing the new interventions in the clinic.
Addressing Treatment Fidelity in the BCC
In July 1999, the National Institutes of Health (NIH), along with
the American Heart Association, established the BCC to provide
an infrastructure to foster collaboration among 15 projects that had
been funded under a request for applications calling for proposals
to test innovative approaches to health behavior change in diverse
populations. The studies either test two theories of health behavior
change or the effectiveness of one theory across multiple health
behaviors such as diet, exercise, or smoking (Ory, Jordan, &
444 BELLG ET AL.
Bazzarre, 2002); details of the studies are available in a special
issue of Health Education Research (Nigg et al., 2002a). The BCC
consists of principal investigators and coinvestigators on these
projects, key staff, program representatives from the NIH who
were involved in the projects, and representatives from the American
Heart Association and foundations such as the Robert Wood
Johnson Foundation that provided additional support.
Because of the complexity of research designs, the diversity of
populations, and the greater than usual need to maintain credibility
when testing innovative interventions, issues of design and implementation
in the BCC studies were particularly challenging. This resulted
in formation of a set of BCC work groups to address these issues
across studies. As part of this effort, the Treatment Fidelity Workgroup
was formed and charged with advancing the definition, methodology,
and measurement of treatment fidelity both within the BCC
and, more generally, for the field of health behavior change (Belinda
Borrelli, Albert J. Bellg, and Susan Czajkowski were the cochairs). In
pursuing that mission, the Treatment Fidelity Workgroup developed
new recommendations for treatment fidelity that expand upon the
Lichstein et al. (1994) model and increase the relevance of treatment
fidelity for health behavior change studies. A detailed survey of all 15
BCC studies was also conducted to identify the strategies the studies
used to address their particular treatment fidelity issues. From the
responses, a list of “best practices” in treatment fidelity was created to
provide examples of how the BCC recommendations may be used in
health behavior intervention research.
BCC Treatment Fidelity Recommendations
The BCC treatment fidelity recommendations intend to link
theory and application in five areas: study design, training providers,
delivery of treatment, receipt of treatment, and enactment of
treatment skills. The five areas (with examples from BCC studies)
are intended to provide behavioral health investigators with a
comprehensive way to conceptualize and address treatment fidelity
issues in their studies.
Design of Study
Practices. Treatment fidelity practices related to study design are
intended to ensure that a study can adequately test its hypotheses in
relation to its underlying theory and clinical processes. Ensuring that
interventions are congruent with relevant theory and clinical experience
involves operationalizing treatments to optimally reflect their
theoretical and pragmatic roots and precisely defining independent
and dependent variables most relevant to the “active ingredient” of the
treatment (Moncher & Prinz, 1991). The active ingredient of a treatment
may vary substantially depending on whether an intervention is
designed to influence cognitions, behavior, or a subjective motivational
state. In addition, the effect of an intervention can only be
adequately assessed when the research design does not confound
treatment effects with extraneous differences between treatment
groups or treatment and control groups. Therefore, treatment fidelity
goals in this category (see Table 1) include establishing procedures to
monitor and decrease the potential for contamination between active
treatments or treatment and control, procedures to measure dose and
intensity (e.g., length of intervention contact, number of contacts, and
frequency of contacts), and procedures to address foreseeable setbacks
in implementation (e.g., therapist dropout over the course of a
multiyear study).
For example, a BCC study looking at dietary change controlled
treatment dose by using group sessions of the same length for both
Table 1
Treatment Fidelity Strategies for Design of Study
Goal Description Strategies
Ensure same treatment dose within conditions. Ensure that treatment “dose” (measured by
number, frequency, and length of
contact) is adequately described and is
the same for each subject within a
particular treatment condition.
Use computer prompts for contacts; ensure fixed
length, number, and frequency of contact
sessions; ensure fixed duration of intervention
protocol; record deviations from protocol
regarding number, length, and frequency of
contacts; ensure fixed amount of information
for each treatment/control group; use scripted
curriculum or treatment manual; externally
monitor sessions and provide feedback to
providers; have provider self-monitor or keep
log of encounter; monitor homework
completion; give specialized training to
providers to deal with different types of
patients equally.
Ensure equivalent dose across conditions. Ensure that treatment dose is the same
across conditions, particularly when
conditions include multiple behavioral
targets (e.g., exercise, smoking).
Have equal number of contacts for each
intervention; use equal length of time for each
intervention; use same level of informational
content for each intervention. When dose is
not the same, stipulate the minimum and
maximum amount of treatment provided and
track number, frequency, and duration of
contacts.
Plan for implementation setbacks. Address possible setbacks in
implementation (e.g., treatment
providers dropping out).
Have pool of potential providers so that new
providers need not be trained in a hurry; train
extra providers beyond those needed; have
human backup for computer-delivered
intervention; track provider attrition.
SPECIAL NIH REPORT: TREATMENT FIDELITY IN RESEARCH 445
treatment and control conditions, with attendance at all sessions
encouraged by a reward at the end of the study. A study providing
a smoking cessation intervention to individuals, however, could
not reasonably control the length of contact with subjects as
closely and so encouraged treatment providers to stay within a
certain range of time and had them record the exact amount of time
spent delivering the intervention so that the possible effect of this
variable could be examined.
Addressing possible setbacks in implementation at the outset is
important to ensure consistency throughout the course of the study.
For example, unanticipated provider dropout may result in hurried
attempts to recruit and train new providers, which may lead to
performance differences between the new and existing providers.
The majority of the BCC sites reported that they were taking
measures to prevent setbacks in implementation, such as training
extra providers or, when the intervention is delivered by computer
and the study design permitted it, training humans as a backup for
the computerized intervention.
Recommendations. Strategies for enhancing treatment fidelity
related to study design should be well defined and thoroughly
described prior to study implementation. We recommend that
researchers consider the following questions during the design
phase of their study: How well does the intervention itself reflect
its theoretical foundations, and in what specific ways does it do so?
What are the areas where it might not do so? How does the study
ensure that each participant receives the same “dose” of the treatment
or treatments? How does the study ensure that treatment dose
is the same across multiple interventions or multiple behavioral
targets? How does the study anticipate and address possible implementation
setbacks?
Training Providers
Practices. An important area of treatment fidelity is assessing
and improving the training of treatment providers to ensure that
they have been satisfactorily trained to deliver the intervention to
study participants. Training in a specific intervention often requires
the acquisition of new skills, which may interact significantly
with a clinician’s existing clinical training and experience.
The adequacy of training to implement the intervention needs to be
evaluated and monitored on an individual basis both during and
after the training process. General strategies in this category include
standardizing training, measuring skill acquisition in providers,
and having procedures in place to prevent drift in skills over
time (see Table 2).
The first strategy in Table 2, standardization of training, involves
ensuring that all providers are trained in the same manner
in order to increase the likelihood that the intervention will be
delivered systematically across providers, decrease the likelihood
that there will be Provider _ Treatment interactions, and prevent
differential outcomes by provider. Standardization, however, does
not preclude individualization of training, which includes accounting
for different levels of education, experience, and implementation
styles. Some methods of standardizing training include using
standardized training materials, conducting role-playing, and observing
actual intervention and evaluating adherence to protocol.
Standardized training of providers to criteria also needs to be
viewed as an ongoing effort rather than as a one-time event. This
is especially important when it is likely that there will be turnover
of staff throughout the intervention period. When multiple training
sessions are required, it is helpful to have the same trainers
conducting training workshops in order to maintain and reinforce
standards across providers and throughout the study period. Certification
or recertification of providers is another way to enhance
and document adequacy of provider training and standardization of
training procedures. Using standardized and pretested training
materials and manuals can also increase the likelihood that all
providers are receiving similar training. Setting performance criteria
and documenting that all providers meet those standards
before delivering interventions also help to ensure the required
skill level of all providers.
Measuring provider skill acquisition both during and after training
is necessary to ensure that training has been successful. Nearly
all BCC sites measured skill acquisition either by direct observation,
written pre- and posttests, or some combination of the two
methods. However, although initial skill acquisition may be adequate,
such skills may be vulnerable to deterioration over time.
Intervention components may be unintentionally omitted or extraneous
components unintentionally added, thus contaminating delivery
of the intervention. It is essential that procedures be put in
place to address provider deficiencies throughout the study. “Drift”
from the original protocol can be minimized in a variety of ways,
such as by scheduling periodic training “booster” sessions with
providers or having regular supervision with providers. All but one
BCC site systematically evaluated provider skills and implemented
measures to prevent skills drift over time. The sites that reported
using layperson providers used many of the same training strategies
outlined previously but also made training more intensive and
took professional experience into account when evaluating the
intervention’s effectiveness.
Recommendations. Most researchers make sure that provider
training is addressed at the beginning of studies. There is less
focus, however, on monitoring and maintaining provider skills as
the study progresses. We recommend that researchers be able to
answer the following questions: How will training be standardized
across providers? How will skill acquisition in providers be measured?
How will decay or change in provider skills be minimized?
How will providers of differing professional training or skill levels
be trained to deliver the intervention in a similar way?
Delivery of Treatment
Practices. Treatment fidelity processes that monitor and improve
delivery of the intervention so that it is delivered as intended
are essential. Even well-trained interventionists may not always
deliver an intervention protocol effectively when clinical circumstances
or their training or involvement in other types of interventions
interfere with their doing so. General goals in this category
include using procedures to standardize delivery and checking for
protocol adherence (see Table 3).
The gold standard to ensure satisfactory delivery is to evaluate
or code intervention sessions (observed in vivo or video- or audiotaped)
according to a priori criteria. Requiring providers to
complete process evaluation forms or behavior checklists after
each intervention session may remind them to include the requisite
skills and content appropriate for each intervention and minimize
contamination from comparison interventions. Checklists, however,
are less reliable correlates of what actually happens in a
446 BELLG ET AL.
session (W. Miller, personal communication, March 22, 2002).
Alternatively, creating forums or case conferences where providers
can discuss intervention cases and review skills required for each
intervention can help ensure that interventions are standardized
across providers and are being conducted according to protocol.
Whether the treatment is being delivered in the way in which the
intervention was conceived may be affected by providers not
having enough time to implement the intervention, by having
unforeseen obstacles to intervention delivery, or by nonspecific
treatment effects such as the warmth and credibility of the provider.
Behavior Change Consortium sites reported using audiotapes,
videotapes, in vivo observation, or behavioral checklists to
ensure that providers adhered to the treatment protocol. Most
research sites used more than one method. All but one site reported
that their providers used a treatment manual or an intervention
protocol or script to aid in standardization of delivery. Several
studies reported that they were using the same provider to deliver
both treatment and control interventions. These sites reported that
they were taking steps to reduce cross-contamination between
treatments by using direct observation, audiotape monitoring, or
subject exit interviews to ensure that control participants did not
receive any of the intervention components.
To control for subtle expectations on the part of interventionists,
one BCC study emphasized to treatment providers that it was
important to give both treatment and control interventions the
same emphasis because a primary outcome was long-term dietary
adherence, and the posttreatment baselines needed to be similar for
both groups. Behavior Change Consortium studies reported measuring
other nonspecific treatment effects by self-report questionnaires
completed by study participants, or in some cases, rating
audiotaped intervention sessions for therapist–provider nonspecific
effects.
Recommendations. Verifying the extent to which treatment
was delivered as intended (and having a mechanism to improve
delivery as needed) is crucial to preserve both internal and external
study validity. We recommend that researchers be able to answer
the following questions: How will the study measure and control
for nonspecific treatment effects? How can you ensure that pro-
Table 2
Treatment Fidelity Strategies for Monitoring and Improving Provider Training
Goal Description Strategies
Standardize training. Ensure that training is conducted similarly
for different providers.
Ensure that providers meet a priori performance criteria;
have providers train together; use standardized
training manuals/materials/provider resources/field
guides; have training take into account the different
experience levels of providers; use structured practice
and role-playing; use standardized patients; observe
intervention implementation with pilot participants;
use same instructors for all providers; videotape
training in case there needs to be future training for
other providers; design training to allow for diverse
implementation styles.
Ensure provider skill acquisition. Train providers to well-defined performance
criteria.
Observe intervention implementation with standardized
patients and/or pilot participants (role-playing); score
provider adherence according to an a priori checklist;
conduct provider-identified problem solving and
debriefing; provide written exam pre- and
posttraining; certify interventionists initially (before
the intervention) and periodically (during intervention
implementation).
Minimize “drift” in provider skills. Ensure that provider skills do not decay
over time (e.g., show that provider
skills demonstrated halfway through the
intervention period are not significantly
different than skills immediately after
initial training).
Conduct regular booster sessions; conduct in vivo
observation or recorded (audio- or videotaped)
encounters and review (score providers on their
adherence using a priori checklist); provide multiple
training sessions; conduct weekly supervision or
periodic meetings with providers; allow providers
easy access to project staff for questions about the
intervention; have providers complete self-report
questionnaire; conduct patient exit interviews to
assess whether certain treatment components were
delivered.
Accommodate provider differences. Ensure adequate level of training in
layperson providers or providers of
differing skill level, experience or
professional background.
Have professional leaders supervise lay group leaders/
paraprofessionals; monitor differential drop-out rates;
evaluate differential effectiveness by professional
experience; give all providers intensive training; use
regular debriefing meetings; use provider-centered
training according to needs, background, or clinical
experience; have inexperienced providers add to
training by attending workshops or training programs.
SPECIAL NIH REPORT: TREATMENT FIDELITY IN RESEARCH 447
viders deliver the intended intervention? How will you ensure that
providers adhere to the treatment protocol? How will you minimize
“contamination” across treatments when they are implemented
by the same provider?
Receipt of Treatment
Practices. The last two treatment fidelity categories shift the
focus from the provider to the patient. Receipt of treatment involves
processes that monitor and improve the ability of patients to
understand and perform treatment-related behavioral skills and
cognitive strategies during treatment delivery. If the intervention
seeks to increase motivation for change or alter other subjective
states conceptually related to motivation (e.g., readiness to change,
self-determination, self-efficacy), receipt refers to the extent to
which the patient’s speech or behavior endorses the increased level
of motivation. Note that treatment receipt specifically relates to the
ability of patients to demonstrate during the intervention that they
understand and can perform the behavioral skills (e.g., relaxation
techniques, completing food diaries) or cognitive strategies (e.g.,
reframing, problem solving) that have been presented to them or
that they are able to experience the desired change in subjective
state induced by the intervention. If a patient does not understand
or is not able to implement the new skills, then an otherwise
effective intervention may be incorrectly deemed as ineffective.
For receipt of treatment (see Table 4), most BCC sites reported
that they verified that participants understood the intervention
during treatment sessions. Methods of measurement included administering
pre- and posttests, structuring the intervention around
achievement-based objectives, and reviewing homework assignments.
A majority of sites also reported employing strategies to
verify that participants were able to use the cognitive, behavioral,
and subjective skills provided in the intervention. For instance, a
BCC study that focused on changing exercise behavior used
monthly review-of-goal forms and activity calendars to confirm
that participants were able to perform the treatment activities
during training sessions.
Recommendations. It is important to choose measures of receipt
that take into account the specific types of information and
skills that are part of the intervention. We recommend that researchers
be able to answer the following questions: How will you
verify that subjects understand the information you provide them
with? How will you verify that subjects can use the cognitive and
Table 3
Treatment Fidelity Strategies for Monitoring and Improving Delivery of Treatment
Goal Description Strategies
Control for provider differences. Monitor and control for subject perceptions of
nonspecific treatment effects (e.g.,
perceived warmth, credibility, etc., of
therapist/provider) across intervention and
control conditions.
Assess participants’ perceptions of provider warmth
and credibility via self-report questionnaire and
provide feedback to interventionist and include in
analyses; select providers for specific
characteristics; monitor participant complaints;
have providers work with all treatment groups;
conduct a qualitative interview at end of study;
audiotape sessions and have different supervisors
evaluate them and rate therapist factors.
Reduce differences within treatment. Ensure that providers in the same condition
are delivering the same intervention.
Use scripted intervention protocol; provide a
treatment manual; have supervisors rate audioand
videotapes.
Ensure adherence to treatment protocol. Ensure that the treatments are being delivered
in the way in which they were conceived
with regard to content and treatment dose.
Provide computerized prompts to providers during
sessions about intervention content; audio- or
videotape encounter and review with provider;
review tapes without knowing treatment
condition and guess condition; randomly monitor
audiotapes for both protocol adherence and
nonspecific treatment effects; check for errors of
omission and commission in intervention
delivery; after each encounter, have provider
complete a behavioral checklist of intervention
components delivered; ensure provider comfort
in reporting deviations from treatment manual
content.
Minimize contamination between conditions. Minimize contamination across treatment/
control conditions, especially when
implemented by same provider.
Randomize sites rather than individuals; use
treatment-specific handouts, presentation
materials, manuals; train providers to criterion
with role-playing; give specific training to
providers regarding the rationale for keeping
conditions separate; supervise providers
frequently; audiotape or observe sessions with
review and feedback; conduct patient exit
interviews to ensure that control subjects did not
receive treatment.
448 BELLG ET AL.
behavioral skills you teach them or evoke the subjective state you
train them to use? How will you address issues that interfere with
receipt?
Enactment of Treatment Skills
Practices. Enactment of treatment skills consists of processes
to monitor and improve the ability of patients to perform
treatment-related behavioral skills and cognitive strategies in relevant
real-life settings. In the case of an induced motivational or
subjective state, enactment is the degree to which the state can be
adopted in the appropriate life setting. This treatment fidelity
process is the final stage in implementing an intervention in that it
involves patients’ actual performance of treatment skills in the
intended situations and at the appropriate time.
Enactment of treatment skills may seem to be confounded with
treatment adherence or treatment efficacy, and making clear distinctions
between these three concepts is useful. Enactment specifically
relates to the extent to which a patient actually implements
a specific behavioral skill, cognitive strategy, or
motivational state at the appropriate time and setting in his or her
daily life (e.g., fills a pill organizer at the beginning of the week,
uses a cognitive strategy to deal with a craving for cigarettes, or
tries out new recipes to identify healthy and appealing dinners). In
contrast, treatment adherence relates to whether the patient performs
the tasks definitive of medical treatment or a healthy lifestyle
change (e.g., actually takes medications, avoids smoking, or
eats a healthy diet). Treatment efficacy relates primarily to whether
the intervention influences the research or clinical endpoint of
interest (e.g., whether a cholesterol-lowering medication lowers
cholesterol or reduces acute medical events or hospitalization,
whether stopping smoking reduces asthma severity, or whether
eating a low-salt diet results in lower blood pressure).
It is therefore possible to have a study with adequate or excellent
enactment of treatment skills that has poor treatment adherence or
treatment efficacy (e.g., someone who fills a pill organizer but
never takes his or her medications or gets the health benefit of
taking them, deals with cravings for cigarettes but does not stop
smoking or have fewer asthma symptoms, or tries out healthy
low-salt recipes but does not keep eating them or achieve a
reduction in blood pressure). Such a study would provide a good
Table 4
Treatment Fidelity Strategies for Monitoring and Improving Receipt of Treatment
Goal Description Strategies
Ensure participant comprehension. Ensure that participants understand the
information provided in
intervention, especially when
participants may be cognitively
compromised, have a low level of
literacy/education, or not be
proficient in English.
Use pre- and posttest process and knowledge
measures; have providers review
homework or self-monitoring logs; have
providers ask questions/discuss material
with subjects; use scripts that prompt
providers to paraphrase/summarize
content; complete activity logs; structure
intervention around achievement-based
objectives; conduct structured interview
with participants; have providers work
with subjects until they can demonstrate
the skills; have providers monitor and give
feedback on practice sessions.
Ensure participant ability to use cognitive
skills.
Make sure that participants are able to
use the cognitive skills taught in the
intervention (e.g., reframing,
problem solving, preparing for highrisk
situations, etc.).
Conduct structured interviews with
participants; have providers review
homework; have providers work with
participants until they can demonstrate
skills; use measures of mediating
variables; have providers monitor and give
feedback on practice sessions; measure
participant performance and completion of
training assignments; have providers
assess cognitive skills; have participants
provide feedback on ability; use
questionnaires; use problem-solving
structured interview that sets up
hypothetical situations and asks
participants to provide strategies for
overcoming obstacles to changing their
behaviors.
Ensure participant ability to perform behavioral
skills.
Make sure that participants are able to
use the behavioral skills taught in
the intervention (e.g., relaxation
techniques, food diaries, cigarette
refusal skills, etc.).
Collect self-monitoring/self-report data
(participants verbally confirm
competence); observe subjects; use
behavioral outcome measures; complete
training assignments; monitor
(electronically/objectively) behavioral
adherence; follow-up telephone contacts/
counseling.
SPECIAL NIH REPORT: TREATMENT FIDELITY IN RESEARCH 449
test of the intervention, because treatment skills are being used by
patients but are not effective at changing their health behavior or
their health outcomes. In a study with poor enactment, however,
neither treatment adherence nor efficacy is likely to be high, but
the researcher will be unable to state whether this is due to poor
enactment or to an ineffective intervention.
It should be noted that in psychological intervention studies in
which the outcome is incorporation of a set of psychological,
social, or behavioral skills into daily life (e.g., mental health or
psychotherapy outcome studies) and in biomedical studies that
involve routine use of medication or medical devices, treatment
goals may be defined in such a way that treatment enactment may
be the same as adherence to treatment. For example, in a study
examining ways in which to train patients with heart failure to care
for a ventricular assist device, the patient’s proper response to
warning alarms may be defined as both enacting the skill the
patient is trained in and adhering to the health behavior outcome of
interest. However, for behavioral change studies in which behavioral,
psychological, or social treatments are used to alter behavioral
risk factors such as diet, physical activity, or smoking behavior,
enactment is appropriately distinguished from adherence, as in
the previous examples.
As for enactment of treatment skills in real-life settings (see
Table 5), most BCC sites reported assessing whether participants
actually used the cognitive skills that are part of their intervention.
Enactment assessments and interventions included questionnaires
and self-reports, structured follow-up interviews, and telephone
calls. All but one study site also reported assessing whether subjects
actually used the behavioral skills in the intervention. Along
with the above strategies, enactment of behavioral skills was
monitored with activity logs, participation in social-learning games
that provided a record of the desired activity, electronic monitoring
of behavior (engaging in exercise or pill taking), and measurement
of biological markers associated with the desired behaviors. For
example, a BCC smoking cessation study measured enactment by
tracking the use of nicotine patches, and a study intervening with
diet and exercise tracked participants’ reports of using problemsolving
and emotional expressiveness skills taught during treatment
with their spouse or partner.
Recommendations. Enactment is one of the most challenging
aspects of treatment fidelity, both conceptually and pragmatically.
Even so, we believe that an important distinction needs to be made
between what is taught (treatment delivery), what is learned (treatment
receipt), and what is actually used (enactment). We recommend
that researchers be able to answer the following questions:
How will you verify that subjects actually use the cognitive,
behavioral, and motivational skills and strategies you provide them
with in the appropriate life situations? How will you address issues
that interfere with enactment?
Discussion and General Recommendations
The following are our general recommendations to the research
community for improving the current state of the art in treatment
fidelity and making it a practical and useful part of health behavior
research.
We recommend that plans for enhancing and monitoring treatment
fidelity be conceptualized as an integral part of the initial
planning and design of health behavior intervention studies. This is
particularly important for studies venturing into less wellunderstood
areas. The needs of each study are different, and
ideally the components of the treatment fidelity plan are selected
Table 5
Treatment Fidelity Strategies for Monitoring and Improving Enactment of Treatment Skills
Goal Description Strategies
Ensure participant use of cognitive skills. Ensure that participants actually use the
cognitive skills provided in the
intervention in appropriate life
settings.
Use process measure; assess with questionnaire;
use self-report regarding achievement of
goals; provide contact form to monitor
participant interaction with staff; use
structured interview with participants; use
exercises, goal sheets, and other printed
material to foster adherence; assess mediating
processes periodically; record telephone
contacts; discuss ongoing use of skills with
subjects; conduct follow-up discussions with
participants.
Ensure participant use of behavioral skills. Ensure that participants actually use the
behavioral skills provided in the
intervention in appropriate life
settings.
Assess with questionnaires; observe participants’
in vivo interactions; assess during provider
encounter; use social-learning game,
providing record of behaviors; conduct selfreport
or self-monitoring and maintain activity
log; measure objective biological or
physiological markers; maintain longitudinal
contact (telephone, mailed information, etc.)
to encourage adherence; record time spent at
facility; monitor frequency of sessions; use
specific behavioral skill use measures;
electronically monitor behavior; follow up
discussions with participants; conduct followup
discussions/telephone calls/counseling with
participants.
450 BELLG ET AL.
on the basis of the theoretical and clinical framework for each
intervention. For example, a participant’s demonstration of certain
behavior- and knowledge-based skills in his or her life might be an
appropriate indication that the participant is enacting an educationally
based intervention but may not accurately reflect enactment of
a motivational intervention. Enactment of a motivational intervention
may be better indicated by a participant’s self-statements
reflecting confidence in being able to make changes to improve his
or her health. With multilevel interventions, it is also important to
assess treatment fidelity issues at both the micro and the macro
levels, examining, for instance, whether interventions both incorporate
specific behaviors and achieve broader behavioral
objectives.
We also recommend that investigators not only institute treatment
fidelity plans at the outset of the study but also maintain
consistent efforts to adhere to a comprehensive treatment fidelity
plan throughout the study period. We recognize, however, that
such plans may need to be modified to accommodate practical
needs and other study demands. In studies where intervention
providers work exclusively for the study, for example, it may be
possible to use numerous strategies to maintain high standards of
treatment fidelity. However, in situations where intervention providers
are integrating the intervention into their current clinical
practice, it may not be feasible to use all desirable treatment
fidelity strategies. In these situations, a more pragmatic and limited
plan may be necessary and should be documented. Therefore, it is
important to consider the setting, other study demands, and provider
and participant burden in order to design a plan that is
practical, achievable, and effective for monitoring and improving
treatment fidelity.
Overall, we believe that having a specific plan to enhance and
monitor treatment fidelity concerns addressed in all five areas
covered by the BCC treatment fidelity recommendations will help
counter threats to the study’s internal and external validity and
therefore enable investigators to draw more accurate conclusions
about the validity and effectiveness of study interventions. It also
will help guide future researchers and program developers in
testing and selecting intervention components that have the most
positive impact on behavioral and treatment outcomes. For clinicians,
it will make it possible to identify interventions appropriate
to the available resources and implement them with the reasonable
expectation that the results will be similar to those achieved in
clinical trials.
It is particularly important that funding agencies, reviewers, and
journal editors who publish behavioral change research consider
treatment fidelity issues. It is our hope that funding initiatives (e.g.,
Requests for Applications and Requests for Proposals), reviewer
guidelines, and publishing requirements will include an explicit
focus on the methods used by researchers to monitor and enhance
treatment fidelity in health behavior intervention studies. As is the
case with current efforts to ensure adequate representation of
women and minorities in clinical research studies, those charged
with oversight of the funding, review, conduct, and reporting of
behavioral change research need to take the lead in encouraging
researchers to address treatment fidelity issues. By asking researchers
to address this issue in funding applications and by
making report of treatment fidelity methods a part of journal
editorial policy, methods to enhance and measure treatment fidelity
are more likely to become standard features in health behavior
intervention studies. Ultimately, this will lead to increased credibility
for the field of behavioral medicine research.
Some researchers may be concerned that such efforts will be
time-consuming and costly. It is not our intention to add to the
work and cost of health behavior intervention studies but to make
them more efficient and effective in identifying useful interventions.
Each study deals with unique circumstances, and there is no
fixed set of treatment fidelity practices that must be added to the
budgets of research projects and the burdens of researchers. Indeed,
our list of “best practices” compiled from BCC studies
represents existing assumptions and strategies for the use of treatment
fidelity practices in research. However, it is our hope that the
BCC treatment fidelity recommendations will play a role in identifying
and organizing treatment fidelity practices so that they may
be more easily and regularly applied by the research community.
Our contention is that not devoting resources to treatment fidelity
is ultimately more costly in time, financial resources, and
credibility than doing so. Moreover, with the current focus on
translation of research findings into real-world settings, treatment
fidelity issues become all the more important. Funding agencies
and researchers clearly have an interest in minimizing the chance
that the studies they are involved in produce equivocal results or
cannot be replicated in the laboratory or the clinic. Health behavior
intervention research and behavioral medicine as a whole can only
benefit from studies that are more reliable, valid, and clinically
applicable. Our final recommendation is that treatment fidelity
should become an integral part of the conduct and evaluation of all
health behavior intervention research.
References
Burgio, L., Corcoran, M., Lichstein, K. L., Nichols, L., Czaja, S.,
Gallagher-Thompson, D., et al. (2001). Judging outcomes in psychosocial
interventions for dementia caregivers: The problem of treatment
implementation. The Gerontologist, 4, 481–489.
Cook, T. D., & Campbell, D. J. (1979). Quasi-experimentation: Design
and analysis issues for field settings. Geneva, IL: Houghton Mifflin.
Kazdin, A. E. (1986). Improving the quality of research: Reflections and
commentary. Counseling Psychologist, 14, 127–131.
Kazdin, A. E. (1994). Methodology, design, and evaluation in psychotherapy
research. In A. E. Bergin & S. L. Garfield (Eds.), Handbook of
psychotherapy and behavior change (4th ed., pp. 19–71). New York:
Wiley.
Lichstein, K. L., Riedel, B. W., & Grieve, R. (1994). Fair tests of clinical
trials: A treatment implementation model. Advances in Behavior Research
and Therapy, 16, 1–29.
Moncher, F. J., & Prinz, F. J. (1991). Treatment fidelity in outcome studies.
Clinical Psychology Review, 11, 247–266.
Nigg, C. R., Allegrante, J. P., & Ory, M. (Eds.). (2002a). Behavior Change
Consortium [Special issue]. Health Education Research, 17(5).
Nigg, C. R., Allegrante, J. P., & Ory, M. (2002b). Theory-comparison and
multiple-behavior research: Common themes advancing health behavior
research. Health Education Research, 17, 670–679.
Ory, M., Jordan, P. J., & Bazzarre, T. (2002). The Behavior Change
Consortium: Setting the stage for a new century of health behavior
change research. Health Education Research, 17, 500–511.
Peterson, L., Homer, A. L., & Wonderlich, S. A. (1982). The integrity of
independent variables in behavior analysis. Journal of Applied Behavior
Analysis, 15, 477–492.
Quay, H. C. (1977). The three faces of evaluation. Criminal Justice and
Behavior, 4, 341–354.
Smith, M. L., Glass, G. V., & Miller, T. I. (1980). The benefits of
psychotherapy. Baltimore: Johns Hopkins University Press.
SPECIAL NIH REPORT: TREATMENT FIDELITY IN RESEARCH 451

Journal of Counseling Psychology

2001, Vol. 48, No. 3, 251-257
Copyright 2001 by the American Psychological Association, Inc.
0022-0167/01/S5.00 DOI: 10.1037//O022-OI67.48.3.251

Hyunnie Ahn, Bruce E. Wampold, Where Oh Where Are the Specific Ingredients? A
Meta-Analysis of Component Studies in Counseling and Psychotherapy, Journal of
Counseling Psychology, 2001, Vol. 48, No. 3, 251-257

University of Wisconsin—Madison
Component studies, which involve comparisons between a treatment package and the treatment package
without a theoretically important component or the treatment package with an added component, use
experimental designs to test whether the component is necessary to produce therapeutic benefit. A
meta-analysis was conducted on 27 component studies culled from the literature. It was found that the
effect size for the difference between a package with and without the critical components was not
significantly different from zero, indicating that theoretically purported important components are not
responsible for therapeutic benefits. Moreover, the effect sizes were homogeneous, which suggests that
there were no important variables moderating effect sizes. The results cast doubt on the specificity of
psychological treatments.
It was established in the 1980s that counseling and psychotherapy
are remarkably efficacious (Lambert & Bergin, 1994; Wampold,
2000); now on center stage is the controversy about whether
the beneficial effects of counseling and psychotherapy are due to
the specific ingredients of the treatments or to the factors common
in all therapies (Wampold, 2000). On one side are the advocates of
empirically supported treatments who claim that treatments are
analogues of medical treatments in that efficacy is attributed to
their respective specific ingredients, which are usually presented in
treatment manuals (see, e.g., Chambless & Hollon, 1998; Chambless
et al., 1996; Crits-Christoph, 1997; DeRubeis & Crits-
Christoph, 1998; DeRubeis et al., 1990; DeRubeis & Feeley, 1990;
Task Force on Promotion and Dissemination of Psychological
Procedures, 1995; Waltz, Addis, Koerner, & Jacobson, 1993;
Wilson, 1996). Specificity (i.e., attributing outcome to specific
ingredients) is one of the hallmarks of the medical model. On the
other side are the advocates of models that stipulate that the
common factors, such as the healing context, the working alliance,
and belief in the rationale for treatment and in the treatment itself,
are the important therapeutic aspects of counseling and psychotherapy
(see, e.g., Frank & Frank, 1991; Garfield, 1992; Luborsky,
Singer, & Luborsky, 1975; Parloff, 1986; Rosenzweig, 1936;
Strupp, 1986; Wampold, 1997, 2000, 2001; Wampold et al., 1997).
From a scientific perspective, the specific ingredient versus common
factor polemic should be settled empirically rather than
rhetorically.
Demonstrating that the specific ingredients of a treatment are
responsible for the benefits of counseling and psychotherapy is
Hyun-nie Ahn and Bruce E. Wampold, Department of Counseling
Psychology, University of Wisconsin—Madison.
We thank Nancy Picard and Dongmin Kim for volunteering to rate
articles for this study. This meta-analysis was conducted as part of the
dissertation of Hyun-nie Ahn under the supervision of Bruce E. Wampold.
Correspondence concerning this article should be addressed to Bruce E.
Wampold, Department of Counseling Psychology, 321 Education Building—
1000 Bascom Mall, University of Wisconsin, Madison, Wisconsin
53706. Electronic mail may be sent to wampold@education.wisc.edu.
complex (see Wampold, 2001, for a discussion of research strategies
for establishing specificity). There are many research strategies
that can be used to demonstrate the specificity of psychological
treatments. Of such designs, component studies come
closest to the "gold standard" of experimental designs and, as such,
should show evidence for specificity, should specificity exist.
Component studies attempt to isolate the effects of ingredients by
comparing treatments with and without those ingredients. Component
studies contain two similar designs, dismantling designs and
additive designs.
The dismantling design involves a comparison between the
entire treatment and the treatment without a given specific ingredient
that is hypothesized to be critical to the success of the
treatment, as shown in Figure 1. Provided the treatment has been
shown to be efficacious, the logic of the design is to "dismantle"
the treatment to identify those ingredients that are responsible for
the benefits that accrue from administration of the treatment. In a
dismantling study, if removing the specific ingredients results in
poorer outcomes vis-a-vis the complete treatment, evidence accrues
for the specificity of those ingredients. Borkovec (1990)
described the advantages of the dismantling study:
One crucial feature of the [dismantling] design is that more factors are
ordinarily common among the various comparison conditions. In
addition to representing equally the potential impact of history, maturation,
and so on and the impact of nonspecific factors, a procedural
component is held constant between the total package and the control
condition containing only that particular element. Such a design
approximates more closely the experimental ideal of holding everything
but one element constant. . . . Therapists will usually have
greater confidence in, and less hesitancy to administer, a component
condition than a pure nonspecific condition. They will also be equivalently
trained and have equal experience in the elements relative to
the combination of elements in the total package.... At the theoretical
level, such outcomes tell what elements of procedure are most
actively involved in the change process. . . . At the applied level,
determination of elements that do not contribute to outcome allows
therapists to dispense with their use in therapy, (pp. 56-57)
251
252 AHN AND WAMPOLD
Group I
Complete Treatment
• All specific
ingredients,
including critical
specific ingredients
• All incidental aspects
Group II
Treatment without
Critical Specific
Ingredient
• All other
specific ingredients
• All incidental aspects
Groups for Dismantling Study
I ~\ Effect due to critical
" specific ingredients
Complete TX without
TX Ingredients
Effect for Specific Ingredient
Figure 1. Dismantling study illustrated. Tx = treatment.
In the additive design, a specific ingredient is added to an
existing treatment (Borkovec, 1990). Typically, there is a theoretical
reason to believe that the ingredient added to the treatment will
augment the benefits derived from the treatment:
The goal is ordinarily to develop an even more potent therapy based
on empirical or theoretical information that suggests that each therapy
[or component] has reason to be partially effective, so that their
combination may be superior to either procedure by itself. In terms of
design, the [dismantling] and additive approaches are similar. It is
partly the direction of reasoning of the investigator and the history of
literature associated with the techniques and the diagnostic problem
that determine which design strategy seems to be taking place. (Borkovec,
1990, p. 57)
A prototypic component study was used by Jacobson et al.
(1996) to determine what components of cognitive-behavioral
treatment of depression were responsible for its established efficacy.
Jacobson et al. separated cognitive-behavioral therapy into
three components: behavioral activation, coping strategies for
dealing with depressing events and the automatic thoughts that
occur concurrently, and modification of core depressogenic cognitive
schemas. Participants were randomly assigned to a behavioral
activation group, a treatment involving behavioral activation
combined with coping skills related to automatic thoughts, or the
complete cognitive treatment, which included behavioral activation,
coping skills, and identification and modification of core
dysfunctional schemas. Generally, the results showed equivalence
in outcomes across the groups at termination and at follow-up.
This study illustrates the logic of the component design. As well,
the results failed to produce evidence of the specificity of ingredients
of cognitive-behavioral therapy.
If specific ingredients are indeed responsible for the benefits of
counseling and psychotherapy, then component studies should
consistently demonstrate an effect when a treatment condition is
compared with a condition not involving a theoretically stipulated
component. Bearing in mind that a few component studies could
demonstrate such differences by chance (i.e., Type I errors), it is
important to determine whether the corpus of component studies
produces specificity effects. Meta-analysis has been shown to be a
powerful method to review literature and bring clarity to disputes
in education, medicine, psychology, and public policy (Hunt,
1997; Mann, 1994). The purpose of this study was to metaanalytically
examine component studies to determine the degree to
which these studies produce evidence that supports the specificity
of psychological treatments.
Method
Procedure
Because this meta-analysis involved a methodological feature (viz.,
component studies), determining a keyword for an electronic literature
search was not possible. Therefore, a comprehensive search of journals that
publish outcome research was undertaken. Wampold et al. (1997) reviewed
the research included in Shapiro and Shapiro's (1982) meta-analysis of
comparative studies and found that the preponderance of such studies were
published in four journals: Behaviour Research and Therapy, Behavior
Therapy, Journal of Consulting and Clinical Psychology, and Journal of
Counseling Psychology. Stiles, Shapiro, and Elliott (1986) noted that
detecting the relative efficacy of treatments depended on sophisticated
research methods and that more recent studies, involving improved methods,
would be more likely to reveal differences between treatments, should
they be present. Accordingly, we searched for component studies published
in the most recent decade (i.e., 1990 to 1999) in the four identified journals.
This strategy eliminated dissertations, presentations, and other unpublished
studies. However, given that studies with statistically significant results are
more likely to be published (Atkinson, Furlong, & Wampold, 1982),
omitting unpublished studies would tend to overestimate the effect of
specific ingredients; consequently, the present analysis yields a liberal test
of specificity.
In identifying the studies for this meta-analysis, Hyun-nie Ann examined
every study published in the four journals just identified from 1990 to
1999. To be included in this meta-analysis, a study had to (a) involve a
psychological treatment intended to be therapeutic for a particular disorder,
problem, or complaint and (b) contain the necessary statistics to conduct
the meta-analysis. To determine that a treatment was intended to be
therapeutic, we used the criteria developed by Wampold et al. (1997);
specifically, a treatment had to involve a therapist who had at least a
master's degree and who met face to face with the client and developed a
relationship with the client. Moreover, the treatment had to contain at least
two of the following four elements: (a) The treatment was based on an
established treatment that was cited, (b) a description of the treatment was
contained in the article, (c) a manual was used to guide administration of
the treatment, and (d) active ingredients of the treatment were identified
and cited. Finally, the study's research design had to involve a comparison
of one group with another group, and one of the following two conditions
had to be satisfied: (a) One, two, or three ingredients of the treatment were
removed, leaving a treatment that would be considered logically viable
(i.e., coherent and credible), or (b) one, two, or three ingredients that were
compatible with the whole treatment and were theoretically or empirically
hypothesized to be active were added to the treatment, providing a "super
treatment." A study was excluded when treatment A was compared with
WHERE ARE THE SPECIFIC INGREDIENTS? 253
treatment B, where B was a subset of A but both A and B were established
treatments in their own rights.
Initially, all studies were gathered that compared one treatment group
with another group that had components added or removed, although the
study may not have met the inclusion and exclusion criteria. Two raters
(both doctoral students in counseling psychology) were then asked to
determine the suitability of each study for this meta-analysis using a rating
sheet listing the inclusion and exclusion criteria. A study was retained if
both raters agreed on its inclusion in the study. When the raters disagreed
on a study, Bruce E. Wampold rated the study, and the study was included
if he determined that it met the criteria. The resulting meta-analytic sample
included 27 treatment comparisons derived from 20 studies (see Table 1).
Analytic Strategy
For each study i, an estimate of the effect size dt for study i that reflected
the effect of a given component or components, as well as an estimate of
the variance of this estimate—that is, SP-(,d,)—was calculated in the following
way. First, for each dependent variable, a sample effect size was
obtained by calculating the difference in the means of the two conditions
and standardizing by dividing by the pooled standard deviation: (morecomponent-
group M - fewer-component-group M)/SD. This value was
adjusted to yield an unbiased estimate of the population effect size; as well,
the standard error of estimate was calculated (Hedges & Olkin, 1985). To
determine a single estimate of the effect size for each study, we combined
the effect sizes for each dependent variable under the assumption that the
correlation among the dependent variables was .50, a reasonable value for
this correlation in psychotherapy studies (see Hedges & Olkin, 1985, pp.
212-213, for the method and Wampold et al., 1997, for a justification and
application in the psychotherapy context). This procedure yielded, for
study i, the desired estimates dt and a2^,); it also provided a more precise
estimate of d{ (i.e., reduced the standard error of estimate) than would the
estimate for any single dependent variable (Wampold et al., 1997).
To aggregate the effect sizes over the 27 comparisons, we weighted each
dj by the inverse of the variance, in the standard fashion, to yield the
aggregated effect size estimate d+ (Hedges & Olkin, 1985). As well, the
Table 1
Component Studies of Psychotherapy
Study
Appelbaum et al. (1990)
Barlow et al. (1992)
Baucom et al. (1990)
Blanchard et al. (1990)
Borkovec & Costello (1993)
Dadds & McHugh (1992)
Deffenbacher & Stark (1992)
Feske & Goldstein (1997)
Halford et al. (1993)
Hope et al. (1995)
Jacobson et al. (1996)
Nicholas et al. (1991)
Ost et al. (1991)
Porzelius et al. (1995)
Propst et al. (1992)
Radojevic et al. (1992)
Rosen et al. (1990)
Thackwray et al. (1993)
Webster-Stratton (1994)
Williams & Falbo (1996)
Disorder
Tension headache
Generalized anxiety disorder
Marital discord
Tension headache
Generalized anxiety disorder
Child conduct problem
General anger
Panic disorder
Marital discord
Social phobia
Depression
Chronic low back pain
Blood phobia
Eating disorder
Depression
Rheumatoid arthritis
Body image
Bulimia nervosa
Parenting effectiveness
Panic attack with agoraphobia
More
components group
CT + PMR
CT + PMR
CT + PMR
CR + BMT
EET + BMT
EET + CR + BMT
CT + PMR
CBT
CMT + Ally
CRCS
EMDR
Enhanced BMT
CBT
BA + AT
BA + AT
CT + PMR
BT + PMR
Applied tension package (BT)
Applied tension package (BT)
OBET
CBT-Religious
BT + social support
CBT + size perception
training
CBT
GDVM + ADVANCE
CBT
CBT
Fewer
components group
PMR
CT
PMR
BMT
BMT
BMT
PMR
AR
CMT
RCS
EFER
BMT
Exposure only
AT
BA
CT
BT
Tension technique
only
Exposure in vivo
only
CBT
CBT
BT
CBT
BT
GDVM
BT
CT
Component(s) tested
Cognitive component
Relaxation skills
CR
CR
EET
EET + CR
Cognitive component
Cognitive component +
self-control desensitization
Social support
Cognitive component
Eye movement
CR + generalized training
+ affective exploration
Cognitive component
BA
Modification of automatic
thoughts
Relaxation skills
Behavioral component
Exposure in vivo
Tension techniques
Advanced CBT with a focus
on coping skills and
cognitive interventions
Religious content modified
to fit CBT
Family support
Size perception training
Cognitive component
Cognitive social learning +
group discussion
Cognitive component
Behavioral component
Note. CT = cognitive therapy; PMR = progressive muscle relaxation; CR = cognitive restructuring; BMT = behavioral marital therapy;
EET =
emotional expressiveness training; CBT = cognitive-behavioral therapy; AR = applied relaxation; CMT = child management training;
CRCS = cognitive
and relaxation coping skills; RCS = relaxation coping skills; EMDR = eye movement desensitization and reprocessing; EFER = eye
fixation exposure
and reprocessing; BA = behavioral activation; AT = automatic thoughts; BT = behavioral therapy; OBET = obese binge eating
treatment; GDVM =
videotaped parent skills training program; ADVANCE = cognitive training social learning program.
254 AHN AND WAMPOLD
standard error of this estimate (d+), which is used to calculate the confidence
interval of d+ and to test the null hypothesis that the population
effect size is zero, was calculated according to the methods developed by
Hedges and Olkin. Finally, a homogeneity test was conducted to determine
whether the 20 effect sizes were drawn from the same population.
Results
Using the aggregation strategy just described, we obtained the
following estimates: d+ = -0.20 and o2^,) = 0.176. The negative
value for d+ indicates that the treatment conditions with fewer
components outperformed the treatment conditions with more
components, a result in the opposite direction from that anticipated.
In any event, an effect size of magnitude 0.20 is considered
small (Cohen, 1988).
The 95% confidence interval for the population effect size,
given a normal effect size distribution, was as follows: lower
bound, d+ - 1.96 d<<f,.) = -0.541, and upper bound, d+ + 1.96
d(dj) = 0.149. Because this confidence interval contained zero, the
null hypothesis that the population effect size is zero was not
rejected.
To determine whether the effect sizes for the 20 comparisons
were drawn from a single population, we conducted a test of
homogeneity using the methods described by Hedges and Olkin
(1985). The Q statistic is a goodness-of-fit statistic, as follows:
where k is the number of studies aggregated. The Q statistic has
approximately a chi-square distribution with k - 1 degrees of
freedom. If Q is sufficiently large, the homogeneity hypothesis is
rejected. In the present case, Q was 33.34, which, when compared
with a chi-square distribution with 26 degrees of freedom, was
insufficiently large to reject the null; therefore, it was concluded
that the effect sizes were homogeneous. Thus, it appears that there
were no variables that would moderate the overall effect size,
which was not different from zero. However, this conclusion must
be tempered by the fact that the power of the homogeneity test can
be low when various assumptions are violated and the sample sizes
of the studies are small in comparison with the number of studies
(see Harwell, 1997).
Discussion
The present meta-analysis of component studies produced no
evidence that the specific ingredients of psychological treatments
are responsible for the beneficial outcomes of counseling and
psychotherapy. For example, the aggregate effect size for comparisons
was not significantly different from zero. Moreover, the
effect sizes from the 27 comparisons were homogeneous, ruling
out rival hypotheses that a missing variable would moderate the
relationship between components and outcome.
It should be recognized that the studies reviewed in this metaanalysis
examined treatments that have been found to be efficacious.
Moreover, the component removed or added was hypothesized
by the researchers to be efficacious according to the
theoretical tenets of the respective treatments. For example, in the
component study described in the introduction, Jacobson et al.
(1996) clearly described the theoretical basis of the study:
Beck and his associates are quite specific about the hypothesized
active ingredients of CT [cognitive-behavioral treatment], stating
throughout their treatment manual (Beck et al., 1979) that interventions
aimed at cognitive structures or core schema are the active
change mechanisms [for treating depression]. Despite this conceptual
clarity, the treatment is so multifaceted that a number of alternative
accounts for its efficacy are possible. We label two primary competing
hypotheses the "activation hypothesis" and the "coping skills" hypothesis.
. . . If an entire treatment based on activation interventions
proved to be as effective as CT, the cognitive model of change in CT
(stipulating the necessary interventions for the efficacy of CT) would
be called into question, (pp. 295-296)
In the Jacobson et al. (1996) study, the authors were examining the
most validated psychotherapeutic treatment in existence, namely
cognitive-behavioral treatment for depression, and testing
whether the cognitive ingredients were indeed necessary to produce
benefits.
A criticism could be raised that included in the corpus of studies
examined were some ingredients that are important and others that
are not and that aggregating across diverse studies yields spurious
conclusions. This is a familiar criticism of meta-analysis. First, the
homogeneity finding suggests that there are not two classes of
comparisons, those with efficacious specific ingredients and those
without. Second, an occasional study demonstrating that a component
was related to the outcome must be considered, in light of
the present results, a Type I error. The argument that a given
specific ingredient is efficacious would need to be supported by
replications, a situation not evident in the studies reviewed. Third,
it is important to note that Jacobson et al.'s dismantling of the
empirically supported cognitive-behavioral treatment of depression,
probably the most established psychological treatment in
existence, failed to demonstrate that the components of the treatment
were responsible for the benefits.
The evidence produced by this meta-analysis casts suspicion on
the specificity of psychological treatments. Although some of the
treatments contained in the studies reviewed were designed for
disorders that are not prevalent (e.g., blood phobia), all of the
treatments contained discrete components that lend themselves to
detecting the efficacy of specific ingredients. That is, if the specific
ingredients of treatments are responsible for the benefits of psychotherapy,
then the expected effects should appear in the studies
reviewed. As well, it would not be expected that specific ingredients
of treatments with less well-defined components would be
responsible for the benefits of such treatments.
Other research evidence tends not to support the benefits of
specific ingredients of psychological treatments. If specific ingredients
were remedial for a problem, then it would be expected that
some treatments (viz., those containing potent specific ingredients)
would be superior to other treatments. However, the outcome
research conclusively has shown that all treatments produce approximately
equal benefits generally (Wampold, 2000; 2001;
Wampold et al., 1997) as well as in particular areas, such as
depression (e.g., Elkin et al., 1989; Robinson, Berman, & Neimeyer,
1990; Wampold, Minami, Baskin, & Tierney, in press) and
anxiety (see Wampold, 2001). Attempts to demonstrate specificity
by examining mediating effects have failed to show that specific
WHERE ARE THE SPECIFIC INGREDIENTS? 255
treatments work through the theoretically hypothesized mechanisms
(Wampold, 2001). For example, in the National Institute of
Mental Health Treatment of Depression Collaborative Research
Program, cognitive-behavioral treatment and interpersonal treatments
did not operate uniquely through the intended respective
cognitive and interpersonal mechanisms, as hypothesized (Imber et
al., 1990). Finally, specificity predicts that certain treatments will
be particularly effective with clients with certain deficits, for
example, cognitive treatments for clients with irrational thoughts
and interpersonal treatments for clients with maladaptive social
relations. However, theoretically predicted interactions between
treatments and client characteristics of this type have never been
demonstrated (for laudable attempts, see McKnight, Nelson-Gray
& Barnhill, 1992; Project MATCH Research Group, 1997; Simons,
Garfield, & Murphy, 1984).
The results of the present meta-analytic study are not an anomaly
in an otherwise uniform field of research results supporting
specificity; rather, the preponderance of the research evidence is
not supportive of the benefits of specific ingredients. This suggests
that the benefits of treatments are probably due to the pathways
common to all bona fide psychological treatments, such as the
healing context, the belief in the rationale for and the efficacy of
therapy by the client and by the therapist, the therapeutic alliance,
therapeutic procedures consistent with the client's understanding
of his or her problems, the development of increased self-efficacy
to solve one's problems, and remoralization (Frank & Frank, 1991;
Garfield, 1992; Wampold, 2001). The research evidence supports
the notion that the benefits of counseling and psychotherapy are
derived from the common factors. For example, it has been shown
that the therapeutic alliance, measured at an early stage, accounts
for a significant portion of the variability in treatment outcomes
(Horvath & Symonds, 1991; Martin, Garske, & Davis, 2000).
Moreover, the variance due to therapists within treatments is
greater than the variance between treatments, lending primacy to
the person of the therapist rather than to the particular treatment
(Crits-Christoph et al., 1991; Wampold & Serlin, 2000). Indeed,
the common factors account for about 9 times more variability in
outcomes than do the specific ingredients (Wampold, 2001).
Rejecting the specificity of counseling and psychotherapy has
implications for training, practice, and research. Training models
should focus on the common factors as the bedrock of skills
necessary to become an effective practitioner. The importance of
interviewing skills, establishment of a therapeutic relationship, and
the core facilitative conditions in the training of counselors and
psychologists is supported by the empirical evidence. Omitting
these vital components and training students to conduct solely
various empirically supported treatments is contraindicated. Nevertheless,
counselors and therapists need to learn techniques, a
position well stated by common factor advocate Jerome Frank:
My position is not that technique is irrelevant to outcome. Rather, I
maintain that, as developed in the text, the success of all techniques
depends on the patient's sense of alliance with an actual or symbolic
healer. This position implies that ideally therapists should select for
each patient the therapy that accords, or can be brought to accord, with
the patient's personal characteristics and view of the problem. Also
implied is that therapists should seek to learn as many approaches as
they find congenial and convincing. Creating a good therapeutic
match may involve both educating the patient about the therapist's
conceptual scheme and, if necessary, modifying the scheme to take
into account the concepts the patient brings to therapy. (Frank &
Frank, 1991, p. xv)
The use of treatment manuals in practice is not supported by the
research evidence. Although standardization of treatment appears
scientific and may be required for experimental control in the
research context, there is no evidence that adherence to a treatment
protocol results in superior outcomes; in fact, slavish adherence to
a manual can cause ruptures in the alliance and, consequently,
poorer outcomes (Wampold, 2001). As well, use of manuals restricts
adaptation of treatments to the attitudes, values, and culture
of the client, a necessary aspect of multicultural counseling.
A common factor perspective places emphasis on the skill of the
therapist. There is compelling evidence that a large proportion of
variability in outcomes is due to therapists, even when therapists
are "experts" in a particular approach and are supervised and
monitored (Wampold, 2001, chap. 8). Thus, emphasis should be
placed on the therapist or counselor rather than on the particular
therapy. Consequently, those who control access to therapy (e.g.,
health maintenance organizations) should refer clients to counselors
who have demonstrated efficacy rather than mandate particular
services. Indeed, it would be in the best interest of agencies to have
therapists of various orientations so that clients could receive the
type of therapy that best accords with their worldview.
Combined with the evidence that all bona fide treatments are
equally efficacious (see Wampold, 2001, chap. 4), the results of
this meta-analysis suggest that comparative outcome studies will
yield nonsignificant differences and therefore are costly experiments
in futility. It is safe to say that hundreds of millions of
dollars have been spent on outcome research that has shown that
bona fide psychological treatments are efficacious but that all such
treatments produce about the same benefits. Continued outcome
research will only support that general pattern of results and yield
little informative evidence about counseling and psychotherapy.
Rather, the focus of counseling research should be on the process
of counseling and on the common factors that have historically
interested humanistic and dynamic researchers and clinicians.
References
References marked with an asterisk indicate studies included in
the meta-analysis.
*Appelbaum, K. A., Blanchard, E. B., Nicholson, N. L., Radnitz, C,
Kirsch, C, Michultka, D., Attanasio, V., Andrasik, F., & Dentinger,
M. P. (1990). Controlled evaluation of the addition of cognitive strategies
to a home-based relaxation protocol for tension headache. Behavior
Therapy, 21, 293-303.
Atkinson, D. R., Furlong, M. J., & Wampold, B. E. (1982). Statistical
significance, reviewer evaluations, and the scientific process: Is there a
(statistically) significant relationship? Journal of Counseling Psychology,
29, 189-194.
*Barlow, D. H., Rapee, R. M., & Brown, T. A. (1992). Behavioral
treatment of generalized anxiety disorder. Behavior Therapy, 23, 551—
570.
*Baucom, D. H., Sayers, S. L., & Sher, T. G. (1990). Supplementing
behavioral marital therapy with cognitive restructuring and emotional
expressiveness training: An outcome investigation. Journal of Consulting
and Clinical Psychology, 58, 636-645.
*Blanchard, E. B., Appelbaum, K. A., Radnitz, C. L., Michultka, D.,
Morrill, B., Kirsch, C, Hillhouse, J., Evans, D. D., Guamieri, P.,
Attanasio, V., Andrasik, F., Jaccard, J., & Dentinger, M. P. (1990).
256 AHN AND WAMPOLD
Placebo-controlled evaluation of abbreviated progressive muscle relaxation
and of relaxation combined with cognitive therapy in the treatment
of tension headache. Journal of Consulting and Clinical Psychology, 58,
210-215.
Borkovec, T. D. (1990). Control groups and comparison groups in psychotherapy
outcome research. National Institute on Drug Abuse Research
Monograph, 104, 50-65.
•Borkovec, T. D., & Costello, E. (1993). Efficacy of applied relaxation and
cognitive-behavioral therapy in the treatment of generalized anxiety
disorder. Journal of Consulting and Clinical Psychology, 61, 611-619.
Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported
therapies. Journal of Consulting and Clinical Psychology, 66, 7-18.
Chambless, D. L., Sanderson, W. C, Shoham, V., Johnson, S. B., Pope,
K. S., Crits-Christoph, P., Baker, M., Johnson, B., Woody, S. R., Sue, S.,
Beutler, L., Williams, D. A., & McCurry, S. (1996). An update on
empirically validated therapies. The Clinical Psychologist, 49(2), 5-18.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences
(2nd ed.). Hillsdale, NJ: Erlbaum.
Crits-Christoph, P. (1997). Limitations of the dodo bird verdict and the role
of clinical trials in psychotherapy research: Comment on Wampold et al.
(1997). Psychological Bulletin, 122, 216-220.
Crits-Christoph, P., Baranackie, K., Kurcias, J. S., Carroll, K., Luborsky,
L., McLellan, T., Woody, G., Thompson, L., Gallagier, D., & Zitrin, C.
(1991). Meta-analysis of therapist effects in psychotherapy outcome
studies. Psychotherapy Research, 1, 81-91.
*Dadds, M. R., & McHugh, T. A. (1992). Social support and treatment
outcome in behavioral family therapy for child conduct problems. Journal
of Consulting and Clinical Psychology, 60, 252-259.
*Deffenbacher, J. L., & Stark, R. S. (1992). Relaxation and cognitiverelaxation
treatments of general anger. Journal of Counseling Psychology,
39, 158-167.
DeRubeis, R. J., & Crits-Christoph, P. (1998). Empirically supported
individual and group psychological treatments for mental disorders.
Journal of Consulting and Clinical Psychology, 66, 37-52.
DeRubeis, R. J., Evans, M. D., Hollon, S. D., Garvey, M. J., Grove, W. M.,
& Tuason, V. B. (1990). How does cognitive therapy work? Cognitive
change and symptom change in cognitive therapy and pharmacotherapy
for depression. Journal of Consulting and Clinical Psychology, 58,
862-869.
DeRubeis, R. J., & Feeley, M. (1990). Determinants of change in cognitive
therapy for depression. Cognitive Therapy and Research, 14, 469-482.
Elkin, I., Shea, T., Watkins, J. T., Imber, S. D., Sotsky, S. M., Collins, J. F.,
Glass, D. R., Pilkonis, P. A., Leber, W. R., Docherty, J. P., Fiester, S. J.,
& Parloff, M. B. (1989). National Institute of Mental Health Treatment
of Depression Collaborative Research Program: General effectiveness of
treatments. Archives of General Psychiatry, 46, 971-982.
*Feske, U., & Goldstein, A. J. (1997). Eye movement desensitization and
reprocessing treatment for panic disorder: A controlled outcome and
partial dismantling study. Journal of Consulting and Clinical Psychology,
65, 1026-1035.
Frank, J. D., & Frank, J. B. (1991). Persuasion and healing: A comparative
study of psychotherapy (3rd ed.). Baltimore: Johns Hopkins University
Press.
Garfield, S. L. (1992). Eclectic psychotherapy: A common factors approach.
In J. C. Norcross & M. R. Goldfried (Eds.), Handbook of
psychotherapy integration (pp. 169-201). New York: Basic Books.
•Halford, W. K., Sanders, M. R., & Behrens, B. C. (1993). A comparison
of the generalization of behavioral marital therapy and enhanced behavioral
marital therapy. Journal of Consulting and Clinical Psychology,
61, 51-60.
Harwell, M. (1997). An empirical study of Hedge's homogeneity tests.
Psychological Methods, 2, 219-231.
Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis.
San Diego, CA: Academic Press.
*Hope, D. A., Heimberg, R. G., & Bruch, M. A. (1995). Dismantling
cognitive-behavioural group therapy for social phobia. Behaviour Research
and Therapy, 33, 637-650.
Horvath, A. O., & Symonds, B. D. (1991). Relation between working
alliance and outcome in psychotherapy: A meta-analysis. Journal of
Counseling Psychology, 38, 139-149.
Hunt, M. (1997). How science takes stock: The story of meta-analysis. New
York: Russell Sage Foundation.
Imber, S. D., Pilkonis, P. A., Sotsky, S. M., Elkin, I., Watkins, i. T.,
Collins, J. F., Shea, M. T., Leber, W. R., & Glass, D. R. (1990).
Mode-specific effects among three treatments for depression. Journal of
Consulting and Clinical Psychology, 58, 352-359.
*Jacobson, N. S., Dobson, K. S., Truax, P. A., Addis, M. E., Koerner, K.,
Gollan, J. K., Gortner, E., & Price, S. E. (1996). A component analysis
of cognitive-behavioral treatment for depression. Journal of Consulting
and Clinical Psychology, 64, 295-304.
Lambert, M. J., & Bergin, A. E. (1994). The effectiveness of psychotherapy.
In A. E. Bergin & S. L. Garfield (Eds.), Handbook of psychotherapy
and behavior change (4th ed., pp. 143-189). New York: Wiley.
Luborsky, L., Singer, B., & Luborsky, L. (1975). Comparative studies of
psychotherapies: Is it true that "everyone has won and all must have
prizes?" Archives of General Psychiatry, 32, 995-1008.
Mann, C. C. (1994). Can meta-analysis make policy? Science, 266, 960-
962.
Martin, D. J., Garske, J. P., & Davis, M. K. (2000). Relation of the
therapeutic alliance with outcome and other variables: A meta-analytic
review. Journal of Consulting and Clinical Psychology, 68, 438-450.
McKnight, D. L., Nelson-Gray, R. O., & Barnhill, J. (1992). Dexamethasone
suppression test and response to cognitive therapy and antidepressant
medication. Behavior Therapy, 23, 99-111.
•Nicholas, M. K., Wilson, P. H., & Goyen, J. (1991). Operant-behavioural
and cognitive-behavioural treatment for chronic low back pain. Behaviour
Research and Therapy, 29, 225-238.
*Ost, L.-G., Fellenius, J., & Sterner, U. (1991). Applied tension, exposure
in vivo, and tension-only in the treatment of blood phobia. Behaviour
Research and Therapy, 29, 561-574.
Parloff, M. B. (1986). Frank's "common elements" in psychotherapy:
Nonspecific factors and placebos. American Journal of Orthopsychiatry,
56, 521-529.
*Porzelius, L. K., Houston, C, Smith, M., Arfken, C, & Fisher, E. Jr.
(1995). Comparison of a standard behavioral weight loss treatment and
a binge eating weight loss treatment. Behavior Therapy, 26, 119-134.
Project MATCH Research Group. (1997). Matching alcoholism treatments
to client heterogeneity: Project MATCH posttreatment drinking outcomes.
Journal of Studies on Alcohol, 58, 7-29.
*Propst, L. R., Ostrom, R., Watkins, P., Dean, T., & Mashburn, D. (1992).
Comparative efficacy of religious and nonreligious cognitive-behavioral
therapy for the treatment of clinical depression in religious individuals.
Journal of Consulting and Clinical Psychology, 60, 94—103.
*Radojevic, V., Nicassion, P. M., & Weisman, M. H. (1992). Behavioral
intervention with and without family support for rheumatoid arthritis.
Behavior Therapy, 23, 13-30.
Robinson, L. A., Berman, J. S., & Neimeyer, R. A. (1990). Psychotherapy
for the treatment of depression: A comprehensive review of controlled
outcome research. Psychological Bulletin, 108, 30-49.
*Rosen, J. C, Cado, S., Silberg, N. T., Srebnik, D., & Wendt, S. (1990).
Cognitive behavior therapy with and without size perception training for
women with body image disturbance. Behavior Therapy, 21, 481—498.
Rosenzweig, S. (1936). Some implicit common factors in diverse methods
of psychotherapy: "At last the Dodo said, 'Everybody has won and all
must have prizes.' " American Journal of Orthopsychiatry, 6, 412-415.
Shapiro, D. A., & Shapiro, D. (1982). Meta-analysis of comparative
therapy outcome studies: A replication and refinement. Psychological
Bulletin, 92, 581-604.
WHERE ARE THE SPECIFIC INGREDIENTS? 257
Simons, A. D., Garfield, S. L., & Murphy, G. E. (1984). The process of
change in cognitive therapy and pharmacotherapy for depression. Archives
of General Psychiatry, 41, 45-51.
Stiles, W. B., Shapiro, D. A., & Elliott, R. (1986). "Are all psychotherapies
equivalent?" American Psychologist, 41, 165—180.
Strupp, H. H. (1986). The nonspecific hypothesis of therapeutic effectiveness:
A current assessment. American Journal of Orthopsychiatry, 56,
513-519.
Task Force on Promotion and Dissemination of Psychological Procedures.
(1995). Training in and dissemination of empirically-validated psychological
treatments: Report and recommendations. The Clinical Psychologist,
48(1), 2-23.
*Thackwray, D. E., Smith, M. C, Bodfish, J. W., & Meyers, A. W. (1993).
A comparison of behavioral and cognitive-behavioral interventions for
bulimia nervosa. Journal of Consulting and Clinical Psychology, 61,
639-645.
Waltz, J., Addis, M. E., Koerner, K., & Jacobson, N. S. (1993). Testing the
integrity of a psychotherapy protocol: Assessment of adherence and
competence. Journal of Consulting and Clinical Psychology, 61, 620-
630.
Wampold, B. E. (1997). Methodological problems in identifying efficacious
psychotherapies. Psychotherapy Research, 7, 21-43.
Wampold, B. E. (2000). Outcomes of individual counseling and psychotherapy:
Empirical evidence addressing two fundamental questions. In
S. D. Brown & R. W. Lent (Eds.), Handbook of counseling psychology
(4th ed., pp. 711-739). New York: Wiley.
Wampold, B. E. (2001). The great psychotherapy debate: Models, methods,
and findings. Mahwah, NJ: Erlbaum.
Wampold, B. E., Minami, T., Baskin, T. W., & Tierney, S. C. (in press).
A meta-(re)analysis of the effects of cognitive therapy versus "other
therapies" for depression. Journal of Affective Disorders.
Wampold, B. E., Mondin, G. W., Moody, M., Stich, F., Benson, K., &
Ahn, H. (1997). A meta-analysis of outcome studies comparing bona
fide psychotherapies: Empirically, "all must have prizes." Psychological
Bulletin, 122, 203-215.
Wampold, B. E., & Serlin, R. C. (2000). The consequences of ignoring a
nested factor on measures of effect size in analysis of variance designs.
Psychological Methods, 5, 425-433.
•Webster-Stratton, C. (1994). Advancing videotape parent training: A
comparison study. Journal of Consulting and Clinical Psychology, 62,
583-593.
•Williams, S. L., & Falbo, J. (1996). Cognitive and performance-based
treatments for panic attacks in people with varying degrees of agoraphobic
disability. Behaviour Research and Therapy, 34, 253-264.
Wilson, G. T. (1996). Manual-based treatments: The clinical application of
research findings. Behaviour Research and Therapy, 34, 295-314.
Received July 14, 2000
Revision received September 26, 2000
Accepted October 24, 2000 •
ORDER FORM
Start my 2001 subscription to Journal of
Counseling Psychology! ISSN: 0022-0167
$38.00, APA Member/Affiliate
$76.00, Individual Non-Member
$170.00, Institution
In DC add 5.75% sales tax
TOTAL AMOUNT ENCLOSED $.
Subscription orders must be prepaid. (Subscriptions are on
a calendar basis only.) Allow 4-6 weeks for delivery of the
first issue. Call for international subscription rates.
SENDTWSORDER FORM TO:
AmericanPsychological Association
Subscriptions
750 First Street, NE
Washington, DC 20002-4242
Or call (800) 374-2721, fax (202) 336-5568.
TDD/TTY (202)336-6123. Email: subscriptions@apa.org
AMERICAN
PSYCHOLOGICAL
ASSOCIATION

Send me a Free Sample Issue •

• Check Enclosed (make payable to APA)
Chargemy: Q VISA QMastaCard QAmerieanExpress
Cardholder Name
Card No. Exp. date
Signature (Required for Charge)
Credit Card
Billing Address
City State Zip _
Daytime Phone
SHIPTO:
Name
Address
City State. Zip.
APA Customer #
GAD01
PLEASE DO NOT REMOVE - A PHOTOCOPY MAY BE USED

Abstract

The number of sessions required to produce meaningful change has not been assessed
adequately, in spite of its relevance to current clinical practice. Seventy-five clients
attending outpatient therapy at a university-affiliated clinic were tracked on a weekly
basis using the Outcome Questionnaire (Lambert et al., 1996) in order to determine the
number of sessions required to attain clinically significant change (CS). Survival analysis
indicated that the median time required to attain CS was 11 sessions. When current data
were combined with those from an earlier investigation (Kadera, Lambert, and Andrews,
1996), it was found that clients with higher levels of distress took 8 more sessions to
reach a 50% CS recovery level than clients entering with lower levels of distress. At a
six-month follow-up, CS gains appeared to have been maintained. Other indices of
change also were examined (reliable change, average change per session). The
implications of these results for allocating mental-health benefits, such as the number of
sessions provided through insurance, are discussed. © 2001 John Wiley & Sons, Inc. J
Clin Psychol 57: 875–888, 2001.

Wampold, Bruce E.; Mondin, Gregory W.; Moody, Marcia; Stich, Frederick; Benson,
Kurt; Ahn, Hyun-nie, A meta-analysis of outcome studies comparing bona fide
psychotherapies: Empiricially, "all must have prizes." Psychological Bulletin, Vol
122(3), Nov 1997, 203-215.
This meta-analysis tested the Dodo bird conjecture, which states that when
psychotherapies intended to be therapeutic are compared, the true differences among all
such treatments are 0. Based on comparisons between treatments culled from 6 journals,
it was found that the effect sizes were homogeneously distributed about 0, as was
expected under the Dodo bird conjecture, and that under the most liberal assumptions, the
upper bound of the true effect was about .20. Moreover, the effect sizes (a) were not
related positively to publication date, indicating that improving research methods were
not detecting effects, and (b) were not related to the similarity of the treatments,
indicating that more dissimilar treatments did not produce larger effects, as would be
expected if the Dodo bird conjecture was false. The evidence from these analyses
supports the conjecture that the efficacy of bona fide treatments are roughly equivalent.
(PsycINFO Database Record (c) 2010 APA, all rights reserved)

Sol L. Garfield Washington University, USA, 2005 - Electicism and integration in

psychotherapy
Several of the important developments in the field of psychotherapy and behavior change
are discussed, including the relative decline in popularity of psychodynamic orientations
and the increase in electic preferences. The variation in operational meanings of an
electic approach is described, and possible commonalities among diverse forms of
psychotherapy are suggested. In terms of patients' views of the important factors in
psychotherapy, characteristics of therapists and some common aspects of therapy appear
to be emphasized over differences in techniques. Finally, some of the recent emphases on
convergence and integration in psychotherapy are discussed.

Comments on the State of Psychotherapy Research (As I See It)

David Orlinsky

University of Chicago

Note: This essay was written in response to an invitation by Chris Muran, North
American SPR regional chapter president, to contribute my views on the current state of
psychotherapy research for the past-president’s column of the NASPR Newsletter. It
appeared, sans references, in the January 2006 issue. Comments on the essay are
welcome at <d-orlinsky@uchicago.edu>.

I must start by confessing that I don’t really read psychotherapy research when I

can help it. Why? The language is dull, the story lines are repetitive, the characters lack

depth, and the authors generally have no sense of humor. It is not amusing, or at least not

intentionally so. What I do instead of reading is scan or study. I do routinely scan the

abstracts of articles as issues of journals arrive to assure myself there is nothing I need or

want to know in it, and if the abstract holds my interest then I scan tables of results. Also,

at intervals of years, I have agreed to study the research on psychotherapy systematically,

usually with a specific focus on studies that related process and outcome (Howard &

Orlinsky, 1972; Orlinsky & Howard, 1978, 1986; Orlinsky, Grawe & Parks, 1994;

Orlinsky, Rønnestad & Willutzki, 2004). I have been doing this for 40 years more or
less, and on that basis (for what it is worth) here is what I think about the state of

psychotherapy research.

I think in recent years that psychotherapy research has taken on many of the

trappings of what Thomas Kuhn (1970) described as “normal science”—meaning that

research by and large has become devoted to incrementally and systematically working

out the details of a general “paradigm” that is widely accepted and largely unquestioned.

The research paradigm or standard model involves the study of (a) manualized

therapeutic procedures (b) for specific types of disorder (c) in particular treatment

settings and conditions. This is very different from the field that I described three decades

ago (Orlinsky & Howard, 1978) as “pre-paradigmatic,” and in some ways it represents a

considerable advance. However, I refer above to the “trappings of normal science” as a

double entendre to suggest that the appearance (trappings) of normal science with its

implicit paradigmatic consensus may also represent entrapment (trapping) in a

constricted and unrealistic model.

The paradigm is familiar. It holds that psychotherapy is basically a set of specific

and specifiable procedures (“interventions” or “techniques”) that can be taught, learned,

and applied; and that the comparative potency or efficacy of these procedures in treating

specific and specifiable psychological and behavioral disorders defines more or less

effective forms of psychotherapy—if patients are willing and able to comply with the

treatment provided by a competently trained therapist.

In this process, therapists are assumed to be active subjects (agents, providers)

and patients are assumed to be reactive objects (targets, recipients). Researchers may well

believe theoretically that patients as well as therapists are active subjects, and that what
transpires between them in therapy should be viewed as interaction, but in practice the

paradigm or standard research model that they typically follow implicitly defines

treatment as a unidirectional process.

Evidence of these implicit conceptions of the patient, therapist, and treatment

process is to be found in experimental designs that randomly assign patients to alternative

treatment conditions, just as if they were ‘objects’ (rarely bothering to inquire about their

preferences) whereas they never assign therapists to alternative treatment conditions,

randomly or systematically (because it seems essential to consider their subjective

treatment preferences). The consequence is that comparisons between treatment

conditions reflect treatment-x-therapist interaction effects rather than treatment main

effects—as Elkin (1999) and others have made clear—but it is an embarrassment that is

conveniently ignored by all (as in the tale of the emperor’s new clothes).

In addition, the dominant research paradigm constricts our view of the phenomena

that psychotherapy researchers presume they are studying by focusing on certain

abstracted qualities or characteristics of patients and therapists. The target of treatment is

not actually the patient as an individual but rather a specifically diagnosed disorder.

Other personal characteristics of patients are presumed to be “controlled” either through

random assignment (another embarrassing myth, since the effectiveness of random

assignment depends on the law of large numbers, and the number of subjects in a sample

or of replicated samples is rarely large enough to sustain this), or controlled statistically

by using the few characteristics of patients that are routinely assessed in studies as

covariates. The covariates most typically are atheoretically selected demographic

variables assessed for the purpose of describing the sample—age, gender, marital status,
race/ethnicity, and the like—since there are no widely accepted theories to guide the

selection of patient variables. (More recently, “alliance” measures have been routinely

collected from patients, reflecting the massive accumulation of empirical findings on the

impact of therapeutic relationship.)

Psychotherapists are likewise viewed in terms of certain abstracted qualities or

characteristics. The agent of treatment studied is not actually the therapist as an

individual but rather a specific set of manualized treatment skills in which the therapist is

expected to have been trained to competence and to which the therapist is expected to

show adherence in practice. The few other therapist characteristics that are routinely

assessed—professional background, career level, theoretical orientation, and perhaps

gender and race/ethnicity—are used largely to describe the sample or, occasionally, as

covariates. Again, this is because there are no widely accepted theories, or extensively

replicated empirical findings, to guide the selection of therapist variables.

The constricted and highly abstracted view of patients, therapists, and the

therapeutic process in the dominant research paradigm is supported by cognitive biases in

modern culture that all of us share. One of these was well-described by the sociologist

Peter Berger and his colleagues as componentiality. This is a basic assumption that “the

components of reality are self-contained units which can be brought into relation with

other such units—that is, reality is not conceived as an ongoing flux of juncture and

disjuncture of unique entities. This apprehension in terms of components is essential to

the reproducibility of the [industrial] production process as well as to the correlation of

men and machines. … Reality is ordered in terms of such units, which are apprehended

and manipulated as atomistic units. Thus, everything is analyzable into constituent

components, and everything can be taken apart and put together again in terms of these

components” (Berger, Berger & Kellner, 1974, p. 27).

This componentiality is reflected in the highly individual and decontextualized

way that we think about persons. We tend to think of individuals as essentially separate,

independent and basically interchangeable units of ‘personality’ that in turn are

constituted by other internal, more or less mechanistically interacting components—

whether those are conceptualized as traits that may be assessed quantitatively as

individual difference variables, or more holistically but less precisely as clinical

components of personality (e.g., ego, id, and superego). Thus when researchers seek to

assess the (hopefully positive but sometimes negative) impact of psychotherapy on

patients, they routinely focus their observations on componential individuals abstracted

from life-contexts, and on the constituent components of individuals toward which

therapeutic treatments are targeted—symptomatic disorders and pathological character

traits. They do not generally assess individuals as essentially embedded in sociocultural,

economic-political and developmental life-contexts. A componential view of

psychotherapy and of the individuals who engage in it is implicit in the dominant

research paradigm, and produces a comforting sense of cognitive control for researchers

—but does it do justice to the realities we seek to study or does it distort them?

Another widely shared bias of modern culture that complicates and distorts the

work of researchers on psychotherapy and psychopharmacology (and medicine more

broadly) is the implicit assumption of an essential distinction or dichotomy between

soma and psyche (or matter and mind), notwithstanding the efforts of modern

philosophers like Ryle (1949) to undo this Cartesian myth. Because of this, findings that
psychological phenomena have neurological or other bodily correlates (e.g., using MRI

or CT scans to detect changes in emotional response) are viewed as somehow amazing

and worthy of note even in the daily press. The materialist bias of modern culture also

fosters a tendency to view this correlation in reductionist terms, so that the physiological

aspects of the phenomena studied are assumed to be more basic, and to cause the

psychological aspect.

Thanks to a conversation at the recent SPR conference in Montreal among

colleagues from different cultural traditions (Bae et al., 2005), I became aware of how

unnatural the body-mind dichotomy (with its consequent distinction between ‘physical

health’ and ‘mental health’) appears from other cultural perspectives, and of how grossly

it distorts the evident psychosomatic continuity of the living human person. When this

basic continuity is conceptually split into ‘psyche’ and ‘soma’, a mysterious quality is

created as the byproduct (much as energy is released when atoms are split)—a mysterious

quality that is labeled (and as much as possible viewed dismissively) as “the placebo

effect.” This effect, mysteriously labeled in Latin, is viewed as a “contaminant” in

research designs—but, struggle as researchers do to “control” it (rather than understand

it), they typically fail in the attempt because the ‘effect’ reflects an aspect of our reality as

human beings that cannot be eliminated.

The reality, as I see it, is that a person (a) is a psychosomatic unity, (b) evolving

over time along a specific life-course trajectory, and (c) is a subjective self that is

objectively connected with other subjective selves, (d) each of them being

active/responsive nodes in an intersubjective web of community relationships and

cultural patterns, a web in which those same patterns and relationships (e) exert a

formative influence on the psychosomatic development of persons.

The reality of psychotherapy, as I see it, is that it involves (a) an intentionally-

formed, culturally-defined social relationship through which a potentially healing

intersubjective connection is established (b) between persons who interact with one

another in the roles of client and therapist (c) for a delimited time during which their life-

course trajectories intersect, (d) with the therapist acting on behalf of the community that

certified her (e) to engage with the patient in ways that aim to influence the patient’s life-

course in directions that should be beneficial for the patient.

Neither of these realities seems to me to be adequately addressed by the dominant

paradigm or standard research model followed in most studies of psychotherapeutic

process and outcome. Instead, the dominant research paradigm seriously distorts the real

nature of persons and of psychotherapy (as I see them). Why then does this paradigm

dominate the field of psychotherapy research, and why do researchers persist in using it if

it is as uncomfortably ill-fitting a Procrustean bed as I have claimed?

The answer is partly cultural, as the paradigm neatly reflects the componential,

psycho/somatically split, materialist cognitive biases of Western culture. It is also partly

psychological, with supporters of the paradigm becoming more militant as a result of

cognitive dissonance generated by the incipient failure of the paradigm’s utopian

scientific promise (see, e.g., Festinger, Riecken & Schachter, 1956). It is partly historical

too, as the field of psychotherapy originated and initially evolved largely as a medical

subspecialty in the field of psychiatry—as well as the field of clinical psychology that

overlapped with, imitated, and set out to rival psychiatry. Again, the answer is partly
economic, since it is necessary to please research funding agencies (the real ‘placebo’

effect) in order to gain funding for research and advance one’s career by contributing

publications to one’s field and reimbursement for “indirect costs” to the institution where

one is employed.

It may be ironic that the paradigm adheres so closely to the medical model of

illness and treatment at a time when the psychiatric profession which historically

represented medicine’s presence in the field has largely (and regrettably) withdrawn from

the practice of psychotherapy (Luhrmann, 2000). The apparent solidity of the paradigm

that survives is based (a) on the fact that psychotherapeutic services still are largely

funded through health insurance which had been politically expanded (after much

lobbying) to include non-medical practitioners, and (b) on the fact that psychotherapy

research still is largely funded through grants from biomedical research agencies.

Although there is no for-profit industry promoting psychotherapy and supporting research

on it as Big Pharma does with the psychopharmacologic treatments of biological

psychiatry, most of the money that can be had in psychotherapeutic practice and

psychotherapy research comes from sources that implicitly support a medical model of

mental health. As ever “they who pay the piper call the tune,” though perhaps it is more

subtle and accurate to say that pipers who need and seek financial support (therapists and

researchers) play their tunes in ways that they hope will be pleasing to potential sponsors.

Necessity drives us (always), but we (all) have an uncanny ability to persuade ourselves

that advantage and merit coincide.

A sociology-of-knowledge confession: I know full well that I can say these things

mainly because I am privileged by having an old-fashioned, tenured, hard-(but small)-

money position in an arts and sciences faculty, and because I am not really in the

competition for funds. As a producer of psychotherapy research, I am free to go my own

way through my work as participant in the SPR Collaborative Research Network; but as a

consumer of psychotherapy research, I have serious misgivings about the state of the filed

stem from a perception that the prevailing paradigm which permits research to pursue

their studies in the manner of “normal science” represents a risky premature closure in

understanding the actual nature of psychotherapy and the people who engage in it. If it is

not overtly corrupting (as may be true of some research on psychopharmacological

treatments funded by pharmaceutical firms), it is nevertheless constricting in ways that

seem to me highly problematic.

If we are indeed to have evidence-based psychotherapies grounded in systematic,

well-replicated research (e.g., Goodheart, Kazdin & Sternberg, 2006), and evidence-

based training for psychotherapists (e.g., Orlinsky & Rønnestad, 2005), then it would be

very nice—in fact, I would think essential—for that research to be based on a standard

model or paradigm which more adequately matches the actual experience and lived

reality of what it presumes to study. I don’t know what that new paradigm or model for

research will turn out to be. Constructing it is the task of the next generation—but from it

will come the sort of psychotherapy research I think I would like to read.
 References

Bae, S. H., Smith, D. P., Gone, J., & Kassem, L. (2005). Culture and

psychotherapy research-II: Western psychotherapies and indigenous/non-western

cultures. Open discussion session, international meeting of the Society for Psychotherapy

Research, Montreal Canada, June 22-25, 2005.

Berger, P., Berger, B., & Kellner, H. (1974). The homeless mind: Modernization

and consciousness. New York: Vintage Books.

Elkin, I. E. (1999). A major dilemma in psychotherapy outcome research:

Disentangling therapists from therapies. Clinical Psychology: Science and Practice, 6,

10-32.

Festinger, L., Riecken, H. H., & Schachter, S. (1956). When prophecy fails: A

social and psychological study of a modern group that predicted the destruction of the

world. New York: Harper.

Goodheart, C. D., Kazdin, A. E., & Sternberg, R. J., Eds. (2006). Evidence-based

psychotherapy: Where practice and research meet. Washington, DC: American

Psychological Association.

Kuhn, T. S. (1970). The structure of scientific revolutions (2nd edition). Chicago:

University of Chicago Press.

Howard, K. I., & Orlinsky, D. E. (1972). Psychotherapeutic processes. In Annual

review of psychology, vol. 23. Palo Alto, Cal.: Annual Reviews.

Luhrmann, T. M. (2000). Of two minds: The growing disorder in American

psychiatry. New York: Knopf.

 Orlinsky, D. E., Grawe, K., & Parks, B. K. (1994). Process and outcome in

psychotherapy—noch einmal. In A. Bergin S. & Garfield, Eds., Handbook of

psychotherapy and behavior change, 4th ed. New York: Wiley.

Orlinsky, D. E., & Howard, K. I. (1978). The relation of process to outcome in

psychotherapy. In S. Garfield and A. Bergin, Eds., Handbook of psychotherapy and

behavior change, 2nd ed. New York: Wiley.

Orlinsky, D. E., & Howard, K. I. (1986). Process and outcome in psychotherapy.

In S. Garfield and A. Bergin, Eds., Handbook of psychotherapy and behavior change, 3rd

ed. New York: Wiley.

Orlinsky, D. E., Rønnestad, M. H. (2005). How psychotherapists develop: A study

of therapeutic work and professional growth. Washington, DC: American Psychological

Association.

Orlinsky, D. E., Rønnestad, M. H., & Willutzki, U. (2004). Fifty years of

psychotherapy process-outcome research: Continuity and change. In M. Lambert, Ed.,

Bergin and Garfield’s Handbook of Psychotherapy and Behavior Change, 5th ed. (pp. ).

New York: Wiley.

Ryle, G. (1949). The concept of mind. New York: Barnes & Noble.