Вы находитесь на странице: 1из 15

Why Use Experimental Designs?

Experimental research is used to establish cause and effect relationship.

internal validity is at the center of all causal or cause-effect inferences. When you want to determine
whether some program or treatment causes some outcome or outcomes to occur, then you are
interested in having strong internal validity. Essentially, you want to assess the proposition:

If X, then Y
or, in more colloquial terms:

If the program is given, then the outcome occurs

Unfortunately, it's not enough just to show that when the program or treatment occurs the expected
outcome also happens. That's because there may be lots of reasons, other than the program, for why
you observed the outcome. To really show that there is a causal relationship, you have to
simultaneously address the two propositions:

If X, then Y

If not X, then not Y

Or, once again more colloquially:

If the program is given, then the outcome occurs


If the program is not given, then the outcome does not occur

Experimental Designs
Two categories

Lab experiments
An artificial or contrived environment.

2.Field experiments
Natural / non-contrived environment where activities regularly take place.
When the causal effects can be tested. Where controls and manipulations are introduced to
establish cause and affect relationships in an artificial setting. It is possible in lab. When we postulate
cause and affect relationships b/w two variables x (Ind.) & y (dep.). Another factor Z may influence Y.
Here it is not possible to see effect of X or Z or Y to control Z effect is called control. Z is a
contaminating/nuisance factor. e.g. Training Improve performance but some have previous
experience, they may perform well so excluding experience is control & providing training to fresh
Lab experiments involves controlling the contaminating variables through the process of either
matching or randomization and manipulation of the treatment.
In order to examine the causal effects of an independent Variable on a dependent variable, certain
manipulation are tried. It means create diff levels of the independent variable to assess the impact
on the dependent variable. For example treatment of depression (CBT, SSRI, CBT+SSRI) is
independent variable then level of depression can be dependent variable, We measure level of
depression before & after treatment.

Nuisance Variables/Factors
Controlling the contaminating exogenous or nuisance variables is an important task. Two ways can
be used matching and randomization.
One way of controlling nuisance variables is to match the various groups by taking their
characteristics and deliberately spreading them across groups.
e.g. Students

20 each group (04) Members

05 (02 Boys 02 Girls)

Gender, Age, Experience Characteristics or Contaminating or nuisance variables. Motivation &

Grades ? of 20 students.
Another way of controlling contaminating variables is randomization No predetermination to 5
groups every member has a known & equal chance of being assigned any of there 5 groups. So the
characteristics or nuisance variables (gender, Age, Experience) are also randomly and equally
distributed. Here each group is comparable And all variables are controlled. Now we cannot say
cause and effect relationship is affected by some nuisance variables. They have been controlled by
randomization. So here we have more internal validity or confidence in the cause are effect
1st draw
2nd draw

1st group (more age, Gender, less exp.)

2nd group (more exp. less age, Gender)

3rd draw

3rd group (more exp. less Age, Gender.)

4th draw

4th group (and so on)

Internal Validity
It refers to the confidence we place in the cause and effect relationship or To what extent does the
research design permit us to say that the independent variable A causes a change in the dependent
variable B?
Low internal validity less causality or here cause & effect relationship not substantiated
Seven (7) factors or threats to internal validity.
1. History,
2. Maturation
3. Testing
4. Instrumentation
5. Selection
6. Statistical regression
7. Mortality
8. Design contamination

1. History: Did some unanticipated event occur while the experiment was in progress and did these
events affect the dependent variable? History is a threat for the one group design but not for the
two group design. In the one group pre-post test design, the effect of the treatment is the
difference in the pre-test and post-test scores. This difference may be due to the treatment or to
history. Is not a threat for the two group (treatment/experimental and comparison/control) design
because the comparison is between the treatment group and the comparison group. If the history
threat occurs for both groups, the difference between the two groups will not be due to the history
event. E.g.
In a short experiment designed to investigate the effect of computer-based instruction, Some missed
some instruction because of a power failure at the school.
2. Maturation: Were changes in the dependent variable due to normal developmental processes
operating within the subject as a function of time? Is a threat to for the one group design. Is not
a threat to the two group design, assuming that participants in both groups change (mature)at
same rate.
e.g. The performance of first graders in a learning experiment begins decreasing after 45 minutes
because of fatigue.
3. Statistical regression: An effect that is the result of a tendency for subjects selected on the
bases of extreme scores to regress towards the mean on subsequent tests. When measurement of
the dependent variable is not perfectly reliable, there is a tendency for extreme scores to regress or
move toward the mean. The amount of statistical regression is inversely related to the reliability of
the test.
e.g. In an experiment involving reading instruction, subjects grouped because of poor pre-test
reading scores show considerably greater gain than do the groups who scored average and high on
the pre-test.
4. Selection :Refers to selecting participants for the various groups in the study. Are the groups
equivalent at the beginning of the study? If subjects were selected by random sampling and
random assignment, all had equal chance of being in treatment or comparison groups, and the
groups are equivalent. Were subjects self-selected into experimental and comparison groups? This
could affect the dependent variable. Selection is not a threat for the one group design but it is a
threat for the two group design.
e.g. he experimental group in an instructional experiment consisted of a high-ability class, while
the comparison group was an average ability class.
5. Experimental Mortality: Differential loss of participants across groups. Did some participants
drop out? Did this affect the results? Did about the same number of participants make it through
the entire study in both experimental and comparison groups? Is a threat for any design with
more than one group.

E.g. In a health experiment designed to determine the effect of various exercises, those subjects
who find the exercise most difficult stop participating.
6. Testing: Did the pre-test affect the scores on the post-test? A pre-test may sensitize participant
in unanticipated ways and their performance on the post-test may be due to the pre-test, not to the
treatment, or, more likely, and interaction of the pre-test and treatment. Is a threat to the one
group design. Not a threat to the two group design. Both groups are exposed to the pre-test and
so the difference between groups is not due to testing.
e.g. In an experiment in which performance on a logical reasoning test is the dependent variable, a
pre-test cues the subjects about the post-test.
7. Instrumentation: Did any change occur during the study in the way the dependent variable
wasmeasured? (Is a threat to the one group design; not to the two group design. )
e.g. Two examiners for an instructional experiment administered the post-test with different
instructions and procedures.
8. Design contamination: Did the comparison group know (or find out) about the experimental
group? Did either group have a reason to want to make the research succeed or fail? Often,
investigators must interview subjects after the experiment concludes in order to find out if design
contamination occurred.
e.g. In an expectancy experiment, students in the experimental and comparison groups compare
notes about what they were told to expect.
Some other threats are:
Compensatory rivalry. When subjects in some treatments receive goods or services believed to be
desirable and this becomes known to subjects in other groups, social competition may motivate the
latter to attempt to reverse or reduce the anticipated effects of the desirable treatment levels.
Resentful demoralization. If subjects learn that their group receives less desirable goods or services,
they may experience feelings of resentment and demoralization.

External Validity
External validity refers to the degree to which the results of an empirical investigation can be
generalized to and across individuals, settings, and times.
How representative is the sample of the population? The more representative, the more confident
we can be in generalizing from the sample to the population. How widely does the finding apply?
Generalizing across populations occurs when a particular research finding works across many
different kinds of people, even those not represented in the sample.
Ecological validity is present to the degree that a result generalizes across settings. Types include:
Interaction effect of testing

Interaction effects of selection biases and experimental treatment

Reactive effects of experimental arrangements
Multiple-treatment interference
Experimenter effects

Quasi-Experimental Designs
quasi(meaning having some but not all of the features) preceding the term experimental
indicates that we are dealing with a design that resembles an experiment but is not exactly an
experiment. How does a quasi-experimental design differ from an experimental design? 1)
Sometimes the difference is the lack of a control group or a comparison group, that is, only one
group is given a treatment and then assessed. 2) At other times the independent variable is not a
true manipulated independent variable; instead, it is a participant variable or a nonmanipulated
independent variable. And 3) finally, some designs may be considered quasi-experimental because
participants were not randomly assigned to conditions, that is, they were already part of a group
and the researcher attempted to manipulate a variable between preexisting groups.


Single-Group Post-test-Only Design

simplest quasi-experimental design.

As the name implies, it involves the use of a single group of participants to whom some
treatment is given.
The participants are then assessed on the dependent variable.
there is neither a comparison group nor a comparison of the results to any previous
For example, a new educational technique such as interactive learning, outcomes learning,
or computer-assisted learning is proposed, and school systems begin to adopt it. Post-test
measures are then taken to determine the amount learned by students.
This design is open to so many criticisms and potential flaws
Chances of maturation effect history effect,





Single-Group Pretest/Posttest Design

measures are taken twice: before the treatment and after the treatment.
The two measures can then be compared, and differences in themeasures are assumed to
be the result of the treatment
For instance, if a single group of depressed individuals wanted to receive treatment
(counseling) for their depression, we would measure their level of depression before the
treatment, we would then have them participate in the counseling, and finally, we would
measure their level of depression after the treatment.
With no comparison group, we do not know whether any observed change in depression is
due to the treatment or to something else that may have happened during the time of the
study ie chances of history, testing and instrumentation effect.
Maybe the pretest depression measure was taken right after the holidays when depression
is higher than during the rest of the year for many people. Consequently the participants
might have scored lower on the posttest depression measure regardless of the counseling.





Single-Group Time-Series Design

involves using a single group of participants, taking multiple measures over a period of time
before introducing the treatment, and then continuing to take several measures after the
multiple measures allow us to see whether the behavior is stable before treatment and how,
or if, it changes at the points in time at which measures are taken after treatment.

Cannot eliminate statistical regression, or regression to mean.





Post test 2

Post test n..


Non-equivalent Control Group Posttest-Only Design

similar to the singlegroup posttest-only design, but a non-equivalent control group is added
as a comparison group.
This design is still not a true experimental one because as with the previous designs participants are
not randomly assigned to the two conditions.
e.g. The researchers found a small Canadian town that had no television reception until 1973; they
designated this town the Notel group. Life in Notel was then compared to life in two other
communities: Unitel, which received only one station at the beginning of the study, and Multitel,
which received four channels at the beginning of the study. A single channel was introduced to
Notel at the beginning of the study. During the 2 years of the study Unitel began receiving three
additional stations. The researchers measured such factors as participation in community activities
and aggressive behavior in children in all three groups, both before and after the introduction of
television in Notel. Results showed that after the introduction of television in Notel, there was a
significant decline in participation in community activities and a significant increase in aggressive
behavior in children.

Sources of Invalidity for Basic Research Designs

single-case design
In certain types of research researchers use methods that minimize the number of participants in a
study. This procedure may sound contrary to the basic principles of design discussed so far.
However, in these methods often referred to as single case designs, only one person is measured
repeatedly. Frequently the research is replicated on one or two other participants. Thus we
sometimes refer to these studies as small-n designs. Such studies can also be thought of as a
variation of the pretest/posttest quasi-experimental design.
Researchers may choose a single-case design for several reasons. They may want information on
only the single participant being studied. They may not be interested in trying to generalize the
results to a population, but rather they may only be interested in how the one participant reacts to
the manipulation. Single-case research is often used in clinical settings. In clinical studies many
researchers believe that it is unethical to use traditional experimental methods in which one group
of participants receives the treatment and the other group serves as a control. They believe it is
unethical to withhold treatment from one group, particularly when the participants may really need
the treatment. In such instances single-case or small-ndesigns are more ethically appealing because
they involve providing treatment to all who participate in the study.
One problem with group designs, is that they do not allow for adequate replication of results
whereas singlecase designs do. Consequently single-case designs are better at demonstrating a

reliable effect of an independent variable. A second problem is that group designs contribute to
error variance in a study. Error variance is the random differences in scores found within the
conditions of an experiment. Using many people in a group design increases error variance resulting
from individual differences. The increase in error variance may make it difficult to identify a
relationship between the variables in the study.
A third problem that Sidman notes is that when using group designs, we typically look at the mean
performance in each group. However, a mean score for a given condition may not accurately
represent the performance of all the participants in that condition. Once we have drawn conclusions
based on the mean performance within a group, we then attempt to generalize the results to
individuals. Psychologists thus draw conclusions about individual behavior based on studying the
average performance of a group of people.
Single-case and small-ndesigns address each of these problems. To determine the reliability of the
effect, we can either repeatedly manipulate the independent variable with the same participant or
perform replications with a few other participants. Further, error variance resulting from individual
differences is eliminated because only one participant is used. Finally, rather than looking at group
means and conducting the appropriate statistical analyses, we look at only the performance of the
single participant in the study to determine the relationship between the independent and
dependent variables. Most commonly we graph the performance of the single participant and
examine the resulting graph. The effect of the independent variable is determined by how much the
participants behaviour changes from one condition to another. Also because the findings are based
on an individuals performance, it makes sense to generalize the results to other individuals.

Single-subject design or single-case research design is a research design most often used in
applied fields of psychology, education, and human behavior in which the subject serves
as his/her own control, rather than using another individual/group. Researchers use
single-subject design because these designs are sensitive to individual organism
differences vs group designs which are sensitive to averages of groups. Often there will be
large numbers of subjects in a research study using single-subject design, however
because the subject serves as their own control, this is still a single-subject design.[1]
These designs are used primarily to evaluate the effect of a variety of interventions in
applied research.[2]
The following are requirements of single-subject designs:[3]
Continuous assessment: The behaviour of the individual is observed repeatedly over the
course of the intervention. This insures that any treatment effects are observed long
enough to convince the scientist that the treatment produces a lasting effect.
Baseline assessment: Before the treatment is implemented, the researcher is to look for
behavioural trends. If a treatment reverses a baseline trend (e.g., things were getting worse
as time went on in the baseline but the treatment reversed this trend) then this is powerful
evidence suggesting (though not proving) a treatment effect.

Variability in data: Because behavior is assessed repeatedly, the single-subject design

allows the researcher to see how consistently the treatment changes behavior from day-today. Large-group statistical designs do not typically provide this information because
repeated assessments are not usually taken and the behavior of individuals in the groups
are not scrutinized; instead, group means are reported.
Phases within single-subject design
Baseline: this phase is one in which the researcher collects data on the dependent variable
without any intervention in place.
Intervention: this phase is one in which the researcher introduces an independent variable
(the intervention) and then collects data on the dependent variable.
Reversal: this phase is one in which the researcher removes the independent variable
(reversal) and then collects data on the dependent variable.
It is important that the data are stable (steady trend and low variability) before the
researcher moves to the next phase. Single-subject designs produce or approximate three
levels of knowledge: (1) descriptive, (2) correlational, and (3) causal.

Between-participants design

participants in each group are different, that is, different people serve in the control and
experimental groups
most basic ideas behind an experiment is that there are at least two groups to compare
the control group serves as the baseline, or standard, condition. The experimental group
receives some level of the independent variable.
By randomly assigning participants, we are trying to make the two groups as equivalent as

posttest-only control group design

Group (random)






pretest/posttest control group design,

Group (random)



within-participants design the same participants are used in all conditions. Within-participants
designs are often referred to as repeated-measures designs because we are repeatedly measuring
the same individuals. A random sample of participants is selected, but random assignment is not
relevant or necessary because all participants serve in all conditions. Within-participants designs are
popular in psychological research for several reasons. within-participants design to minimize the
number of participants needed is advantageous. Second, within-participants designs usually require
less time to conduct than between-participants designs. The study is conducted more quickly
because participants can usually take part in all conditions in one session; the experimenter does not
use a participant in one condition and then wait around for the next person to participate in the next
condition. Further, the instructions need to be given to each participant only once. If there are 10
participants in a within-participants design and participants are run individually, the experiment
need only be explained 10 times. If there are 10 participants in each condition in a betweenparticipants design in which participants are run individually, the experiment needs to be explained
20 times.
Third, and most important, within-participants designs increase statistical power. When the same
individuals participate in multiple conditions, individual differences between the conditions are