Experimental Research Designs

Proposed New Course # for 02 Spring: EDLD 656

from Winter 2000

Instructor: Russell Gersten, Ph.D.

Phone: 342-4268 E-mail: rgersten@OREGON.UOREGON.EDU
Office: 132 E. Broadway (off-campus main office)

A. Course Objectives

The major objective of the course is to develop skill and facility in:
• designing experimental and quasi-experimental designs that can be
implemented in schools and classrooms and other educational settings;
• interpreting data from experimental and quasi-experimental studies.

A major focus throughout the course is:

• translation of research questions into research design, and how each aspect of
the research design subtly shapes and re-shapes the questions asked, and the
"answers" obtained.

By the end of the course, students should be capable of designing a study -- or a series
of experimental/quasi-experimental studies -- to examine a research issue or question
of interest to them or those they work with. This includes an evaluation of the pros and
cons of various designs, determining a reasonable sample, selecting or designing a set
of valid measures and other related issues.

This ability is essential for developing a dissertation proposal if your interest is in applied
classroom research, and should be extremely useful for comprehensive examinations.

The course will stress concepts and applications rather than computations. Thus, for
example, in utilizing a text such as Keppel and Zedeck, the main thrust will be on the
theoretical rather than the computational formulae. Virtually all the computations
required (for example on planned comparisons, Tukey post-hoc tests) should be basic
enough that they can be done with a hand calculator.

Because no textbook stresses the approach I will use, the major source will be
handouts. The text will serve primarily as a source book. A wide range of examples will
be used in class. Two new statistical topics (not covered in the winter statistics course)
-- mixed designs and analysis of covariance -- will be covered.

The following pages outline a list of topics. A rigid scheduling of topics to days does not
seem appropriate. The course will be set up so that you can feel free to ask questions,
expand on ideas, raise questions about applications to your own work or interests, etc.
We will stress applications and discussions of application exercises.

Note that some of the topics have been covered in previous statistics courses you have
taken. This is totally intentional. The goal now is to revisit these concepts in terms of
experimental designs, to begin to understand the utility of these concepts.

Experimental Research Designs 00 Winter Page 1

In many ways, learning statistics is like learning another language and repeated
exposure to the same concepts from different vantage points is likely to increase
comprehension. As part of the course, you will be asked to read a series of studies, in
part, to immerse yourself in the language of experimental design.

There will also be occasions when you will be asked to critique a research study in
terms of the concepts and issues covered in class.

B. Assignments to be collected

The following will be collected and serve as a basis for grading:

1. Two quizzes
2. Two article critiques
3. One draft research proposal and presentation
4. Group project on research design standards

If appropriate, may overlap with research topic that was developed for Grant Writing
Seminar.
More information regarding assignments will be provided in class.

C. Student Support

A student with a documented disability and in need of accommodations in this course

should meet with the professor as soon as possible. Please request that Hilary Gerdes,
Counselor for Students with Disabilities (346-3211), send a letter to the professor
verifying the disability.

D. Prerequisite
Prerequisites include two terms of statistics. If you have any concerns about the
prerequisites, please talk to the instructor.

E. Presentations

Two students working on their doctoral dissertations will present to the class, Sr. Mary
Karen Oudeans and Lana Edwards.

There will also be an opportunity for several students in this class to present their
designs to the class during the final two weeks. Often this is most useful for those who
are getting ready for a dissertation proposal meeting.

Experimental Research Designs 00 Winter Page 2

D. Topics [Dates subject to change because my goal is comprehension of the material]
For each topic, sections of two texts will be suggested for review:
1. Gall, Borg and Gall (1996) Educational research: An introduction (6th ed.)
and
2. Keppel and Zedeck (1989) Data analysis for research designs . Emphasis on
conceptual understanding

Topic Gall, Borg and Article Keppel and

Gall (1996) Zedeck (1989)

1. True experiments vs. Quasi- p. 463-501 Dimino et al. 1990. p. 379-389

experiments
• Random assignment
• Matching with random
assignment
Review of threats to internal
validity for quasi-experiments
and experiments
Date: January 7
(Ch.12) Read the article for
p. 505-508 second class. Read it
as a story, and a
study to understand,
not (yet) as a study to
critique.

2. The language of Analysis of p. 392-394 Woodward, Carnine p. 69-80 (Ch.

Variance p. 508-519 and Gersten (1988) 6)
• Between and within (Skip the Secondary • For basic
subjects variables Analyses p.81) statistical
• Types of factors concepts
(personological,
treatment, other (e.g.
time/setting))
Review of 4 major types of
experimental designs
Date: January 7 - 10

3. Critiquing articles-- basic issues p. 142 (Fig. 4.9)

and guidelines p. 741-744
Date: January 10 (Appendix F)
p. 745-748
(Appendix G)

Topic Gall, Borg and Article Keppel and

Gall (1996) Zedeck (1989)
p. 508 Darch and Gersten p. 213-236
4. Linking main effects and (definitions) (1986) (Ch. 14)
interactions to research questions p. 510-511 (an p. 239-259
• Post hoc tests: review example) (Ch. 15)
and application p. 232 (Fig.
• All Four Cases of 14-1!)
Designs
• Relating interactions to
research questions
Date: January 21

5. Data Analysis and Interpretation • Reread Woodward

Date: January 28 et al.

Experimental Research Designs 00 Winter Page 3

 et al.
• Reread Dimino et
al. (skip ATI)

6. Effect size calculation p. 194-196

Date: January 28

7. Core Concepts in Experimental p. 463-501

Design (Ch.12)
• Power and error P. 505-537
Date: February 7 (Ch.13)

Topic Gall, Borg and Article Keppel and

Gall (1996) Zedeck (1989)
p. 215-242 • Review Gersten, p. 15-20
. Core Concepts in Experimental • sampling Baker, and Lloyd p. 32-37
Design (con) p. 245-367 (in press)
• Power • data • Review the
Brief discussion of Darch-Gersten collection handout
study p. 485-490 “Designing A Field
• random Study”
. Designing an Experimental Field assignment •
Study p. 537-538
• Sampling, blocking common
• Specification of measures mistakes
(transfer, maintenance
etc.)
Experiments vs. quasi-
experiments
Issue of random assignment
(with and without matching)
Guest Lecture: Lana Edwards
Date: February 14

. Designing an Experimental Field p. 394-395 Baker, Gersten and p. 453-482

Study (con) Keating (in press) (Ch. 22)
Example of a current research study • Emphasiz
at the middle school level e
conceptua
l rather
Analysis of Covariance than
• The link between numerical
regression and ANOVA understan
• Residual scores ding
• Realistic issues
Date: February 18

Gersten (1999)
Qualitative Interlude

Experimental Research Designs 00 Winter Page 4

Guest Lecture: Sr. Mary Karen
Oudeans

Date: February 25

Topic Gall, Borg and Article Keppel and

Gall (1996) Zedeck (1989)
Read Greenwood
12. Quasi-Experiments p. 505-508 (1991) p. 379 - 389
• Problems/issues in using
covariance with quasi-
experimental studies
Date: February 28

13. Mixed Designs: Their Basic p. 514-515 p. 294-299

Statistics and Their Unresolved, p. 312
Hidden Secrets
Why they are used so much in special
education research?

Aptitude treatment interactions -- a

modern view
Date: March 3

• Lyon and Moats

14. Introduction to Growth Curve (1997)
Analysis • Baker et al. (in
• Concept of linear vs. press)
quadratic growth
Date: March 10
p. 58-59
15. Planned Comparisons p. 2-6
Date: TBA p. 179 (Fig.
12-1)

Experimental Research Designs 00 Winter Page 5

E. Schedule and Assignment Due Dates

Monday Friday

January 7

January 10 January 14

January 17 M.L.K., Jr. Day January 21 CRITIQUE 1 DUE

Focus on Design and
Questions (not Analysis)

January 24 January 28

January 31 February 4

February 7 February 11 EXAM #1

February 14 February 18

February 21 February 25 MINI-PROPOSAL DUE

February 28 CRITIQUE 2 DUE March 3

Focus on Design and Analysis

March 6 MINI-EXAM #2 March 10

Experimental Research Designs 00 Winter Page 6

F. Readings

Sections of the following beginning research and data analysis textbooks will be suggested to the student
to review:

Gall, M.D., Borg, W.R. & Gall, J.P. (1996). Educational research: An introduction (6th
ed.). New York: Longman Publishers.

Keppel, G. & Zedeck, S. (1989). Data analysis for research designs. New York: W.H.
Freeman and Company.

The following journal articles will be used by the instructor to illustrate various research themes.

1. *Dimino, J., Gersten, R., Carnine, D., & Blake, G. (1990). Story grammar: An approach for
promoting at-risk secondary students' comprehension of literature. Elementary School
Journal, 91(1), 19-32.

2. *Woodward, J., Carnine, D., & Gersten, R. (1988). Teaching problem solving through
computer simulations. American Educational Research Journal, 25(1), 7-28.

3. Darch, C., & Gersten, R. (1986). Direction setting activities in reading comprehension: A
comparison of two approaches. Learning Disabilities Quarterly, 9(3), 235-243.

4. Greenwood, C. R. (1991). Engagement, and achievement in at-risk versus non-risk students.

Exceptional Children 57(6): 521-535.

5. Gersten, R., Baker, S., & Lloyd, J.W. (in press). Designing high quality research in special
education: Group experimental design. Journal of Special Education.

6. Gersten, R. (1999). Lost opportunities: Challenges confronting four teachers of English-

language learners. Elementary School Journal, 100 (1), 37-56.

7. Baker, S., Gersten, R., & Keating, T. (in press). When less may be more: A two-year
longitudinal evaluation of a volunteer tutoring program with minimal training. Reading
Research Quarterly. (note: utilizes analysis of covariance).

8. Lyon, G. R. and L. C. Moats (1997). Critical conceptual and methodological considerations in

reading intervention research. Journal of Learning Disabilities 30(6), 578-588.

For article critique #1

Johnson, L. and M. Pugach (1991). Peer collaboration: Accommodating students with mild
learning and behavior problems. Exceptional Children 57: 454-461.

For article critique #2

Johnson, R.T., Johnson, D.W., & Stanne, M.B. (1986). Comparison of computer-assisted
cooperative, competitive, and individualistic learning. American Educational Research Journal, 23(3),
382-392.

*Based on dissertations completed at the University of Oregon.

Experimental Research Designs 00 Winter Page 7

 General Issues to Address in Article Critique

1. Importance of research question/clarity of research question.

2. Sampling.

3. Nature of comparison group/Assignment of subjects to treatment

4. Quality of the intervention/monitoring of fidelity of treatment – is the intervention feasible in a

classroom setting?

5. Measurement – quality and adequacy of instruments used.

6. Were data analyzed appropriately?

7. Were results interpreted properly?

8. Generalizability of findings.

9. Clarity of writing style and any other salient issues.

This critique should be between 6 and 10 pages. Not every issue raised above will be particularly
germane to the article.

Equal Opportunity / Affirmative Action / Americans with Disabilities Act Statement

The University of Oregon is an equal opportunity, affirmative action institution committed to cultural
diversity and compliance with the Americans with Disabilities Act. If you have a documented disability and
anticipate needing accommodations in this course, please make an appointment with the primary
instructor during the first week of the term. Please request that the Counselor for Students with
Disabilities send a letter verifying your disability. The current counselor is Molly Sirois 346-1155.
Disabilities may include (but are not limited to) neurological impairment, orthopedic impairment, traumatic
brain injury, visual impairment, chronic medical conditions, emotional/psychological disabilities, hearing
impairment, and learning disabilities.

Experimental Research Designs 00 Winter Page 8

EDLD 607: Experimental Design
Reaction Form

Name Date

√ A few things I learned today:

? Questions, problems, or areas of confusion:

Comments:

Experimental Research Designs 00 Winter Page 9