CHAPTER 9: EXPERIMENTS EXPERIMENTS are studies involving intervention by the researcher beyond that required for measurement.

. The researcher manipulates the independent or explanatory variable and then observes whether the hypothesized dependent variable is affected by the intervention.

3. 4. -

Advantages: 1. 2. Ability to uncover causal relationships; researchers ability to manipulate the IV. Contamination from extraneous variables can be controlled more effectively than in other designs; provisions from controlling extraneous and environmental variables. Convenience and cost of experimentation are superior to other methods. The ability to replicate findings and this rule out idiosyncratic or isolated results. Replication repeating an experiment with different subject groups and conditions. Ability to exploit naturally occurring events; can use naturally occurring events to reduce subjects perception of the researcher as a source of intervention or deviation in their everyday lives.

3. 4. 5.

5. 6. 7.

Disadvantages: 1. 2. 3. 4. 5. The artificiality of the laboratory. Generalization from nonprobability samples. Applications of experimentation outrun the budgets for other primary data collection methods; disproportionate costs in select business situations. Focus is restricted to the present and immediate future. Ethical issues related to the manipulation and control of human subjects.

Controlling the experimental environment Environmental control is holding constant the physical environment of the experiment. [ BD ] Blind: subjects do not know if they are receiving the experiment treatment. Double Blind: experimenters do not know if they are giving the treatment to the experimental or control group. Choosing the experimental design Researchers apply their knowledge to select one design that is best suited to the goals of the research. Serve as positional and statistical plans to designate relationships between the experimental treatments and the experimenters observations or measurement points in the temporal scheme of the study. Strengthens the generalizability of the results beyond the experimental setting. Selecting and assigning participants Systematic sampling may be used if the sampling frame is free from any form of periodicity that parallels the sampling ratio. If not, randomization is used: random assignments, matching (employs a nonprobability quota sampling approach) and quota matrix. [ RMQ ] Pilot testing, revising, and testing Pilot testing is intended to reveal errors in the design and improper control of extraneous or environmental conditions. Pretesting the instruments permits refinement before the final test The experiment should be times so that subjects are not sensitized to the independent variable by factors in the environment. Analyzing data

Validity in Experimentation: INTERNAL VALIDITY - refers both to how well a study was run (research design, operational definitions used, how variables were measured, what was/wasn't measured, etc.), and how confidently one can conclude that the observed effect(s) were produced solely by the independent variable and not extraneous ones. Do the conclusions we draw about a demonstrated experimental relationship truly imply the cause? Threats: [ THIMESS ] 1. 2. Testing refers to how a pretest can affect subjects' performance on a post-test. History refers to the effect external events has on subjects between the various measurements done in an experiment.

Conducting an Experiment: 1. Select relevant variables Variables must be operationalized (how concepts are transformed into variables to make them measurable and subject to testing). Number of variables in an experiment is constrained; selection of measures must be adapted to the unique needs of the research situation without compromising their intended purpose or original meaning. Specifying treatment levels Experimental treatment is the manipulation of the independent variable. Treatment levels of the IV are the arbitrary or natural groups Control group (composed of subjects who are exposed to the IV) provide a base level for comparison.

2. -

3. 4. 5. 6. 7.

Instrumentation refers to the objectivity, reliability and validity of the research measurements. Maturation refers to how subjects naturally can change over the passage of time (rather than due to the treatment) Experimental Mortality/Attrition refers to the potential bias that occurs depending on who stays or drops out of a study. Selection refers to the effect of nonequivalent groups on a study's validity. Statistical Regression is the phenomenon whereby retest results tend to regress toward the mean. When subjects in a study are selected as participants because they scored extremely high or extremely low on some measure of performance.

Experimental Research Designs: 1. a. PRE-EXPERIMENTAL DESIGNS weak in controlling the various threats to internal validity. [ Os-Og-S ] The One Shot Case Study a single group is given a treatment and then tested. X O

Example: A third grade class is provided a special Bible study course on Paul, after which their knowledge of Paul is tested. b. One-Group Pretest/Posttest a single intact group is tested before and after a treatment. O1 XO 2

Additional Threats to Internal Validity: [ DCCRL ] 1. 2. 3. 4. 5. Diffusion or imitation of treatment those in the control group may learn of the treatment. Compensatory equalization reluctance to deprive the control group members Compensatory rivalry members of the contr0ol group know they are in the control group. Resentful demoralization of the disadvantages control group members may become resentful of their deprivation and lower their cooperation and output. Local history

Example: A group of third graders is tested on knowledge of Paul before and after a special study on the life of Paul. c. Static-Group comparison two intact groups are tested after one has received the treatment. X O1

EXTERNAL VALIDITY - represents the extent to which a study's results can be generalized or applied to other people or settings. Does an observed causal relationship generalize across persons, settings, and times? Threats: [ RIO ] 1. 2. 3. Reactivity of testing on X refers to sensitizing subjects via a pretest so that they respond to the experimental stimulus (x) in a different way. Interaction of selection and X - an interaction between how the subjects were selected and the treatment can occur. Other reactive factors: Artificial setting can produce results that are not representative of larger populations. If subjects know they are participating in an experiment, there may be a tendency to role-play in a way that distorts the effects of x.

----------------O2 Example: Two classes of third graders are tested on their knowledge of Paul after one of them has had the special Bible study. 2. TRUE EXPERIMENTAL DESIGNS experimental designs are considered true experiments when they employ randomization in the selection of their samples and control for extraneous influences of variation on the dependent variable. [ PpPo ] Pretest-Posttest Control Group two randomly selected groups are measured before (O1 and O3) and after (O2 andO4) one of the groups receives a treatment (X). R R O1 O3 X O2 O4

a.

Example: Third graders are randomly assigned to two groups and tested for knowledge of Paul. Then one group gets a special Bible study on Paul. Both are then tested again.

b.

Posttest Only Control Group subjects are randomly selected and assigned to two groups. Due to randomization, the two groups are statistically equal. No pretest is given. One group receives the treatment. R R X O1 O2

c.

Group Time Series establish a baseline measure of subjects by administering a series of tests over time (O1 through O4 in this case). Expose the group to the treatment and then measure the subjects with another series of tests (e.g., O5 through O8). Useful where regularly kept records are a natural part of the environment and are unlikely to be reactive. O1 O2 O3 O4 X O5 O6 O7 O8

Example: Third graders are randomly assigned to two groups. Then one group receives a special study on the life of Paul (no pre-test). Both are tested on their knowledge of Paul at the conclusion of the study. 3. FIELD EXPERIMENTS: QUASI- OR SEMI-EXPERIMENTS - are used when true experiments cannot be done. Without randomization, there are no true experiments. [ NSG ] Nonequivalent Control Group Design subjects are tested in existing or intact groups rather than being randomly selected. The dotted line in the diagram represents non-equivalent groups. Both groups are measured before and after treatment. Only one group receives the treatment. O1 X O2

Example: A class of third graders is given several tests on Paul before having a special study on him. Several tests are given after the special study is finished.

a.

-----------------------------O3 O4

Example: Two intact third grade classes (no random selection) are tested on their knowledge of Paul before and after one of them receives a special study on the life of Paul. Note: Intact Equivalent design members of the experimental and control groups is naturally assembled. Self-selected experimental group design weaker because volunteers are recruited to form the experimental group, while non-volunteer are used for control. b. Separate Sample Pretest-Posttest Design most applicable when we cannot know when and to whom to introduce the treatment but we can decide when and whom to measure. R R O1 (X) means irrelevant X O2