Pre-Experimental Design - loose in structure, could be biased

Aim of the Research

Name of the Design

Notation Paradigm

Comments

To attempt to explain a
consequent by an
antecedent

One-shot
experimental case
study

X O

An approach that prematurely

links antecedents and
consequences. The least
reliable of all experimental
approaches.

To evaluate the influence of

a variable

One group pretestposttest

OXO

An approach that provides a

measure of change but can
provide no conclusive results.

To determine the influence

of a variable on one group
and not on another

Static group
comparison

Group 1: X O

Weakness lies in no
examination of preexperimental equivalence of
groups. Conclusion is reached
by comparing the performance
of each group to determine the
effect of a variable on one of
them.

Group 2: - O

True Experimental Design - greater control and refinement, greater control of validity

Aim of the Research

To study the effect of an

influence on a carefully
controlled sample

Name of the Design

Pretest-posttest
control group

Notation Paradigm

R--[OX
O
[O-
O

To minimize the effect of

pretesting

Solomon four-group
design

R--[OX
O
[O-
O

Comments

This design has been called "the old

workhorse of traditional
experimentation." If effectively
carried out, this design controls for
eight threats of internal validity.
Data are analyzed by analysis of
covariance on posttest scores with the
pretest the covariate.
This is an extension of the pretestposttest control group design and
probably the most powerful
experimental approach. Data are
analyzed by analysis of variance on
posttest scores.

O
O
To evaluate a situation that
cannot be pretested

Posttest only control

group

[- X
[--

R--[ XO
[-O

An adaptation of the last two groups

in the Solomon four-group design.
Randomness is critical. Probably, the
simplest and best test for significance
in this design is the t-test.

Quasi-Experimental Design - not randomly selected

Aim of the Research

Name of the Design

Notation Paradigm

Comments

To investigate a situation
in which random selection
and assignment are not
possible

Nonrandomized
control group pretestposttest

OXO

To determine the influence

of a variable introduced
only after a series of initial
observations and only
where one group is
available

Time series
experiment

OOXO
O

If substantial change follows

introduction of the variable, then the
variable can be suspect as to the
cause of the change. To increase
external validity, repeat the
experiment in different places under
different conditions.

To bolster the validity of

the above design with the
addition of a control group

Control group time

series

OOXO
O

A variant of the above design by

accompanying it with a parallel set of
observations without the introduction
of the experimental variable.

O-O

OO-O
O
To control history in time
designs with a variant of
the above design

Equivalent timesamples

[X1 O1] [X0

O2] [x1
O3]

One of the strongest and most widely

used quasi-experimental designs.
Differs from experimental designs
because test and control groups are
not equivalent. Comparing pretest
results will indicate degree of
equivalency between experimental
and control groups.

An on-again, off-again design in

which the experimental variable is
sometimes present, sometimes
absent.

http://www.okstate.edu/ag/agedcm4h/academic/aged5980a/5980/newpage2.htm

Pre-Experimental Designs
Pre-experiments are the simplest form of research design. In a pre-experiment either a
single group or multiple groups are observed subsequent to some agent or treatment
presumed to cause change.

Types of Pre-Experimental Design

One-shot case study design

One-group pretest-posttest design

Static-group comparison

One-shot case study design

A single group is studied at a single point in time after some treatment that is presumed to
have caused change. The carefully studied single instance is compared to general
expectations of what the case would have looked like had the treatment not occurred and to
other events casually observed. No control or comparison group is employed.

One-group pretest-posttest design

A single case is observed at two time points, one before the treatment and one after the
treatment. Changes in the outcome of interest are presumed to be the result of the
intervention or treatment. No control or comparison group is employed.

Static-group comparison

A group that has experienced some treatment is compared with one that has not. Observed
differences between the two groups are assumed to be a result of the treatment.

Validity of Results
An important drawback of pre-experimental designs is that they are subject to numerous
threats to their validity. Consequently, it is often difficult or impossible to dismiss rival
hypotheses or explanations. Therefore, researchers must exercise extreme caution in
interpreting and generalizing the results from pre-experimental studies.
One reason that it is often difficult to assess the validity of studies that employ a preexperimental design is that they often do not include any control or comparison group.
Without something to compare it to, it is difficult to assess the significance of an observed
change in the case. The change could be the result of historical changes unrelated to the
treatment, the maturation of the subject, or an artifact of the testing.
Even when pre-experimental designs identify a comparison group, it is still difficult to
dismiss rival hypotheses for the observed change. This is because there is no formal way to
determine whether the two groups would have been the same if it had not been for the
treatment. If the treatment group and the comparison group differ after the treatment, this
might be a reflection of differences in the initial recruitment to the groups or differential
mortality in the experiment.

Advantages and Disadvantages

Advantages
As exploratory approaches, pre-experiments can be a cost-effective way to discern whether
a potential explanation is worthy of further investigation.

Disadvantages
Pre-experiments offer few advantages since it is often difficult or impossible to rule out
alternative explanations. The nearly insurmountable threats to their validity are clearly the
most important disadvantage of pre-experimental research designs.