Chapter 9: Basic Issues in Experimental Research

PSYC 3F40

Experimental Research
Descriptive studies allow us to describe how people think, feel, and behave. Correlational studies examine whether variables are related to one another. But neither of these approaches allows us to study the causes of behavior. An experiment must be conducted to test explicitly whether one variable causes or causally influences another variable.

Three Essential Properties of a Well-Designed Experiment

The researcher must: vary or manipulate at least one independent variable to assess its effects on participants responses have the power to assign participants to experimental conditions in a way that assures their initial equivalence control extraneous variables that might influence the outcome of the experiment

Independent Variables
In an experiment, the researcher varies (or manipulates) one or more independent variables. An independent variable must have two or more levels (different values). These levels can reflect either quantitative or qualitative differences in the independent variable.

Types of Independent Variables

Environmental manipulations modifications of the participants physical or social environment

E.g., temperature, color, presence of others, interaction with confederates Priming Activating a concept

Instructional manipulations vary the independent variable through the verbal instructions that participants receive Invasive manipulations create physical changes in the participants body through surgery or the administration of drugs

Subliminal priming Activate a concept outside of conscious awareness Supraliminal priming Activate a concept by (for instance) reading a paragraph

Experimental and Control Groups

Experimental group participants in an experiment who receive a nonzero level of the independent variable

Control group participants in an experiment who receive a zero level of the independent variable (or the absence of the variable of interest)

Assessing the Impact of Independent Variables

Many studies fail because the independent variable was not manipulated successfully. Pilot test a preliminary study that examines the usefulness of manipulations or measures that will be used in an experiment Manipulation checks questions designed to determine whether the independent variable was manipulated successfully

Participant Variables

Participant (or subject) variable a personal characteristic of research participants, such as age, gender, self-esteem, weight, or extraversion. Although they are sometimes called independent variables, subject variables are not true independent variables because they are not manipulated by the researcher.

(Subject variables may be used in a QUASIexperiment)

Dependent Variables

Dependent variable the response being measured in a study, typically a measure of participants thoughts, feelings, behavior, or physiological reactions

Most experiments have several dependent variables.

Simple Random Assignment

Simple random assignment participants are placed in experimental conditions in such a way that every participant has an equal probability of being in any condition

Random assignment is used to make the conditions roughly equivalent at the start of the study.

Matched Random Assignment

Matched random assignment participants are matched into homogenous blocks, and then participants within each block are assigned randomly to conditions

Matched random assignment helps to ensure that the conditions will be similar along some specific dimension, such as age or intelligence.

Repeated Measures Design

Repeated

measures design an experimental design in which each participant serves in all conditions of the experiment; also called a within-subjects design measure designs eliminate the need for random assignment because every participant is tested at every level of the independent variable. in which each participant serves in only one experimental condition are called between-subjects designs.

Repeated

Designs

Advantages of Within-Subjects Designs

Within-subjects designs are more powerful than between-subjects designs because every participant serves in every condition

Power the ability of a research design to detect effects of the independent variable

Within-subjects designs require fewer participants.

Disadvantages of Within-Subjects Designs

Order effects occur when the effects of a particular experimental condition are contaminated by its order in the sequence of experimental conditions in which participants are tested.

Order Effects

Practice effects participants responses are affected by completing the dependent variable many times Fatigue effects participants become tired or bored as the experiment progresses Sensitization participants gradually become suspicious of the hypothesis as the experiment progresses

Counterbalancing

To protect against order effects, researchers use counterbalancing. Counterbalancing involves presenting the levels of the independent variables in different orders to different participants.

Carryover Effects

Carryover effects occur in within-subjects designs when effects of one treatment condition are still present when the participant is tested in another condition.

Systematic Variance

Systematic variance (also called between-groups variance) is that part of the total variance in participants responses that reflects differences among the experimental groups.

If the independent variable has an effect on behavior, we should see systematic differences between the scores in the various experimental conditions.

Two Sources of Systematic Variance

Treatment variance (primary variance) is the portion of the systematic variance that is due to the independent variable. Confound variance (secondary variance) is the portion of the systematic variance that is due to extraneous variables that differ systematically between the experimental groups. Confound variance must be eliminated.

Error Variance

Error variance (or within-groups variance) is the portion of the total variance in participants responses that remains unaccounted for after systematic variance due to the independent variable is removed. It is unsystematic variance that is unrelated to the independent variable(s) under investigation in an experiment.

Internal Validity

Internal validity is the degree to which a researcher draws accurate conclusions about the effects of the independent variable on a dependent variable. To have internal validity, researchers must eliminate all potential confounds.

Threats to Internal Validity

1.

Biased assignment of participants to conditions effects are due to initially nonequivalent groups rather than to the independent variable; this can occur when random assignment fails Differential attrition participants drop out of experimental conditions at different rates, making the experimental groups no longer equivalent.

1.

Threats to Internal Validity

3.

Pretest sensitization completing a pretest leads participants to react differently to the independent variable than they would have reacted had they not been pretested

4.

History effects extraneous events occurring outside of the research setting have an effect on participants responses

Threats to Internal Validity

5. Miscellaneous

design confounds Something other than the independent variable differs systematically between the experimental conditions Threats to internal validity (confounds) must be eliminated so that we can be certain that differences among conditions are caused by the independent variable.

Expectations
Experimenter

expectancy effects occur when a researchers expectations about the outcome of a study influences participants reactions. Demand characteristics occur when aspects of a study indicate to participants how they should respond. To avoid these, researchers use double-blind procedures in which neither the participants nor the experimenter who interacts with them know which condition the participant is in.

Placebo Effect

A placebo effect is a physiological or psychological change that occurs as a result of the belief that an effect will occur. A placebo control group may be used in which some participants are administered an ineffective treatment (placebo). If there is a difference between the true control group and the placebo control group, we know that a placebo effect is present.

Sources of Error Variance

1.

Individual differences pre-existing differences between people; this is the most common source of error variance Transient states at the time of the experiment, participants differ in how they feel (mood, health, fatigue, interest, etc.)

2.

Sources of Error Variance

3.

Environmental factors differences in the conditions under which the study is conducted (noise, time of day, temperature, etc.) Differential treatment treating different participants in slightly different ways Measurement error unreliable measures contribute to error variance

4.

5.

External Validity

External Validity is the degree to which the results obtained in one study can be replicated or generalized to other samples, research settings, and procedures.

The Experimenters Dilemma

The Trade-off Between Internal and External Validity

The more tightly controlled an experiment, the stronger its internal validity. However, tight experimental control makes the experiment more unique and less like other settings, thereby lowering external validity.
Experimenters almost always opt for internal over external validity.

Any Questions ?