Research Methods for Evaluating Treatment Efficacy- course III

Several research methods can be used to evaluate which therapeutic approaches are the most beneficial
under which circumstances.

LEARNING OBJECTIVES
Describe the research methods and criteria that are used to determine the effectiveness of therapy

KEY TAKEAWAYS
Key Points
Researchers who assess therapy are interested in areas such as whether or not counseling is effective,
under what conditions it is effective, and what outcomes are considered effective—such as symptom
reduction, behavior change, or quality-of-life improvement.
Topics commonly explored in the study of the counseling process and outcome include therapist
variables, client variables, the counseling or therapeutic relationship, cultural variables, process and
outcome measurement, mechanisms of change, and process and outcome research methods.
When assessing the effectiveness of therapy, researchers often rely on mixed methods, or both
quantitative and qualitative designs. Each type of data provides different forms of information, together
providing a fuller evaluation of the therapy.
Quantitative methods include randomly controlled clinical trials, correlational studies over the course of
counseling, or the use of inventories to track a client’s progress throughout the course of treatment.
Qualitative methods may involve conducting, transcribing, and coding interviews or therapy sessions;
analysis of single counseling sessions or counseling cases; or reported observations made by the
therapist.
A randomized controlled trial (RCT) is a type of scientific experiment that is often used to test the
efficacy or effectiveness of various types of medical intervention.
Meta-analyses can be thought of as “conducting research about previous research” in order to gain a
better understanding of the effectiveness of different therapeutic approaches.
Key Terms
quantitative: Of a measurement based on a number or numerical value, rather than on a quality.
qualitative: Of a measurement, description, or distinction based on a characteristic, rather than on a
quantity.
attachment style: Based on Bowlby’s attachment theory, which describes the dynamics of long-term
relationships between humans; there are four main styles of attachment—secure, anxious-preoccupied,
anxious-avoidant, and fearful-avoidant.
subjective: Formed, as in opinions, based on a person’s feelings or intuition, not on observation or
reasoning; coming more from within the observer than from observations of the external environment.
meta-analysis: Any systematic procedure for statistically combining the results of many different studies.
The Importance of Evaluation
Assessing the effectiveness of therapeutic interventions is important for determining which therapies
are the most beneficial, and for which types of disorders and/or individuals. Researchers who assess
therapy are interested in areas such as whether or not counseling is effective, under what conditions it
is effective, and what outcomes are considered effective—such as symptom reduction, behavior change,
or quality-of-life improvement.

Variables That Influence Treatment

Topics commonly explored in studies of counseling processes and outcomes include therapist variables,
client variables, the counseling or therapeutic relationship, cultural variables, process and outcome
measurement, mechanisms of change, and process and outcome research methods.

Therapist Variables
Therapist variables include the characteristics of the therapist, the techniques of the therapist, therapist
behavior, theoretical orientation, and training. The characteristics of the therapist can include the
therapist’s gender, race, sexual orientation, culture, biases, personality, etc. The techniques of the
therapist include the ways in which the therapist approaches the therapy sessions.

Client Variables
Client characteristics, such as help-seeking attitudes and attachment style, have been found to be
related to seeking out counseling, expectations of counseling, and outcomes. Educating clients about
expectations of counseling can improve client satisfaction and outcomes.

The Therapeutic Relationship

The therapeutic relationship is defined as the feelings and attitudes that a client and therapist have
toward one another and the manner in which those feelings and attitudes are expressed. The amount of
trust or distrust that a client has toward the therapist can have an impact on the outcome of therapy.
The therapeutic relationship has been found to predict treatment adherence and outcomes across a
range of client diagnoses and treatment settings. A positive relationship between the therapist and the
client based on trust and confidence often leads to a good working alliance, in which the therapist and
client can agree on tasks and goals for therapy.

Approaches to Measuring Therapeutic Outcomes

When assessing the effectiveness of therapy, researchers often rely on mixed-method designs, which
means using both quantitative and qualitative designs. Unfortunately, a number of theoretical models
used in therapy, such as interviews and observations, lack quantitative data to support their
effectiveness and rely solely on qualitative data. Ideally, therapies should use mixed methods to provide
both quantitative and qualitative data. Each type of data provides different forms of information,
together providing a fuller evaluation of the therapy.

Quantitative vs. Qualitative Methods

Quantitative methods include randomly controlled clinical trials, correlational studies over the course of
counseling, and laboratory studies about specific counseling processes and outcome variables. One way
of gathering quantitative data is through the use of inventories. For example, the Outcome
Questionnaire-45 is a 45-item self-report measure of psychological distress; the Beck Depression
Inventory specifically measures depression; and the Quality of Life Inventory is a 17-item self-report
measure of life satisfaction. These types of inventories can be given at the beginning of treatment when
a client enters therapy and then again at some point near the end. The differences in scores can then be
examined to determine if the quality of life has improved, if the distressing symptoms have decreased,
and other factors that indicate the effectiveness of the therapeutic approach.

Qualitative methods may involve conducting, transcribing, and coding interviews; transcribing and/or
coding therapy sessions; analyzing single counseling sessions or counseling cases; or using observations
made and reported by the therapist. Qualitative data can provide subjective information that cannot be
measured or effectively captured by quantitative methods. Many psychotherapists believe that the
nuances of psychotherapy cannot be captured by quantitative, questionnaire-style observation, and
prefer to rely on their own qualitative clinical experiences and conceptual arguments to support the
type of treatment they practice.

Randomized Controlled Trial

image
Flowchart of a randomized controlled trial : In a randomized controlled trial, people are randomly
assigned to different groups that are receiving different treatment or no treatment at all, in order to
study the effects of various treatment interventions.

A randomized controlled trial (RCT) is a type of scientific (often medical) experiment, where the people
being studied are randomly allocated to one or another of the different treatments under study. Often,
people will be randomly allocated to an “intervention” group (for example, those that receive the
medication being studied) and a “non-intervention” group (those that do not receive the medication, or
receive a placebo instead). The RCT is often considered the gold standard for a clinical trial. RCTs are
often used to test the efficacy or effectiveness of various types of medical intervention and may provide
information about adverse effects, such as drug reactions. Random assignment of intervention is done
after subjects have been assessed for eligibility and recruited, but before the intervention to be studied
begins.

Meta-Analyses
A meta-analysis comprises statistical methods for contrasting and combining results from different
treatment-focused studies in the hope of identifying patterns among study results, sources of
disagreement among those results, or other interesting relationships that may come to light in the
context of multiple studies. Meta-analysis can be thought of as “conducting research about previous
research” in order to gain a better understanding of the effectiveness of different therapeutic
approaches. Many meta-analyses have been used to explore the effectiveness of psychotherapy. For
example, one large-scale study that examined 16 meta-analyses of cognitive behavioral therapy (CBT)
reported that it was as effective as, or more effective than, other therapies in treating post-traumatic
stress disorder (PTSD), generalized anxiety disorder, depression, and social phobia (Butlera, Chapmanb,
Formanc, & Becka, 2006). Another meta-analysis found that psychodynamic therapy was also as
effective at treating these types of psychological issues as CBT (Shedler, 2010).