Acceptable SOE Research Methods and Research Designs

Last updated 10/05/2012

Introduction

This document will identify and describe research methods and research designs that are
acceptable for dissertation research in the School of Education.

Quantitative Methods
In quantitative research designs educational researchers confirm or verify general statements or
predictions. Lodico, Spaulding, and Voegtle (2010, p. 10) refer to this type of research as
employing “the hypothetic-deductive method, which begins by forming a hypothesis – a tentative
explanation that can be tested by collecting data.” The hypothesis is formed from theory or a
synthesis of the results of related previous research. Quantitative methods are frequently placed
in three main categories:

True Experimental Designs – The assignment of control and treatment groups and the
manipulation of selected variables while controlling all others. Acceptable research
design examples include:

 Pretest-Posttest Control Group Design.

 Solomon Four Group Design.
 Posttest-Only Control Group Design.
 Within Subjects Design.

While true experiments, which require random selection of participants, are not typical in
educational settings, there might be instances when random selection is possible.

Quasi-experimental Designs – An experimental approach (control and test groups)

where participants are not randomly assigned or groups such as classes are assigned
either randomly or not randomly. Acceptable research design examples include:

 Non-randomized [Non-equivalent] Control Group Pretest-Posttest Design.

 Solomon Four Group Design.
 Simple Times-Series Design.
 Control Group, Time-Series Design.
 Reversal Time-Series Design.
 Alternating Treatment Design.
 Multiple Baseline Design.

The most common quasi-experimental design done in educational settings is the non-randomized
control group pretest-posttest design.

Doc. Reference: soe_diss_m03_acceptable_research_methods_designs.docx

 Non-experimental designs. The researcher examines the phenomenon under
investigation to explain it or describe it with observation or measurement without
manipulating or controlling the environment or variables.

 Descriptive Designs (Survey research).

 Observational Designs.
 Ex Post Facto Designs [e.g., causal-comparative design, factorial design,
combined experimental and ex post facto design all of which require
partial controls].
 Correlational Designs [e.g., bivariate analysis, simple linear regression,
simple logistic regression, multinomial or multivariate logistic regression,
multiple linear regression, discriminant function analysis, canonical
regression, path analysis, factor analysis].
 Comparative Descriptive Designs.
 Secondary Data Analysis (Records-based Research).

The most common non-experimental designs conducted in educational settings are survey
research, correlational designs and comparative descriptive designs.

Note 1: Quantitative descriptive research designs might put the learner-researcher at risk of
adding little to the existing body of knowledge and thus are rarely appropriate for SOE
dissertations. Descriptive research designs should be considered in light of the criterion provided
the Capella Dissertation Manual (2011):
The dissertation is not just descriptive; it has a sound extant basis or a well-developed
conceptual basis that leads to the question(s) under investigation. This basis serves as the
origin for conclusions and inferences that lead to further research, to enhanced theoretical
understanding, and to recommendations for organizational improvement in cases of
action science. (p. 8)
Thus it would be unusual or unlikely that a sound extant or conceptual basis would lead to an
examination of simple descriptions or opinions or that simple descriptions or opinions will
advance theoretical understanding.

Note 2: Pre-experiments [e.g., one-shot experimental case study, one-group pretest-posttest

design, and static group comparison] that are done alone; that is, without being a preliminary
component of either a true experimental or quasi-experimental design are unacceptable
research designs for dissertations. This unacceptability is because “In pre-experimental
designs, it is not possible to show cause-and-effect relationships because either (a) the
independent ‘variable’ doesn’t vary or (b) experimental and control groups are not comprised or
equivalent or randomly selected individuals. Such designs are helpful only for forming tentative
hypotheses that should be followed up with more controlled studies” (Leedy & Ormrod, 2005, p.
223).

Note 3: If questions arise as to the applicability of a particular research design, then the Mentor
should contact the SOE Research Chair for direction.

2
Qualitative Methods

Qualitative research designs in the SOE typically focus include exploratory studies attempting to
describe, explain, or better understand social or educational phenomena. Such exploration is
necessary when there is not enough knowledge regarding the particular phenomenon that a
quantitative instrument cannot be developed or that the exploration seeks an in depth
understanding or insights that cannot be obtained via a quantitative study with existing
instruments. Qualitative research designs primarily focus on the participants’ experiences, the
meaning participants ascribe to those experiences, and/or understanding a process from the
insights of a process participant. Merriam (1998) provides a broad definition of qualitative
research “Qualitative research is an umbrella concept covering several forms of inquiry that help
us understand and explain the meaning of social phenomena with as little disruption of the
natural setting as possible”(p. 5). The centrality of meaning appears again and again in the
qualitative methods literature. Acceptable research design examples include:

Action Research – “The practice of action research is a fairly common mode of

investigation in educational research especially among those researchers interested in
classroom teaching practices” (Berg, 2004, p.195). Action research is “intended to
uncover or produce information and knowledge that will be directly useful to a group of
people (through research, education, and sociopolitical action) . . . that provides people
with the means to take systematic action in an effort to resolve specific problems” (Berg,
2004, p. 197). Action research will always involve developing, implementing, and
evaluating an intervention.

Basic Qualitative Design – Merriam (2009) describes a basic qualitative research study
as having philosophically been derived from constructionism, phenomenology, and
symbolic interaction and is used by researchers who are "interested in “(1) how people
interpret their experiences, (2) how they construct their worlds, and (3) what meaning
they attribute to their experiences. The overall purpose is to understand how people make
sense of their lives and their experiences" (p. 38).

Case study – “A qualitative case study is an intensive, holistic description and analysis
of a single instance, phenomenon or social unit” (Merriam, 1988, p. 21); thus the case
occurs in some form of bounded context. For example a case could be a classroom, a
program, a process, etc. A strong case study is often the end product of field work
research and thus relies heavily upon observations over an extended period of time.
Interviews are typically used to clarify and verify observations and artifacts. Although
case studies do not need to have this longitudinal aspect and could be done without
observations, what they have in common is multiple data collection methods to develop
thick rich holistic cases. “Understanding the case in its totality, as well as the intensive,
holistic description and analysis characteristic of a case study, mandates both breadth and
depth of data collection. Data collection in a case study is a recursive, interactive process
in which engaging in one strategy incorporates or may lead to subsequent sources of
data” (Merriam, 1998, p. 134).

3
Ethnographic Study – An ethnography focuses on society and culture attempting to
uncover and describe beliefs, values, and attitudes that structure behavior in a group. Its
analysis is unique from other forms of qualitative research in that a sociocultural
interpretation is applied to the unit of analysis (Merriam, 1998). Ethnographies are rare
as dissertations in the SOE because of the length of immersion in the studied culture.
Although methodologists there is some disagreement on the length of immersion and
observation, six months to a year or longer appear to be the standard.

Grounded Theory – The researcher derives a substantive theory grounded in the

experiences and views of participants. Substantive theory is not formal or grand theory
but rather “has is referent specific, everyday-world situations such as an innovative
middle school science program, the coping mechanisms of returning adult students, or
stages of late-life development. A substantive theory has specificity and hence
usefulness to practice often lacking in theories that cover more global concerns”
(Merriam, 1998, p. 17). A salient feature of grounded theory is the application of
theoretical sampling to reach theoretical saturation, which could lead to a sample size of
20 to 50 participants.

Phenomenological Design – SOE qualitative researchers focus on the essence or

structure of an experience or phenomenon. Patton (1990) describes this type of research
as based on:
the assumption that there is an essence or essences to shared experience. These
essences are the core meanings mutually understood through a phenomenon
commonly experienced. The experiences of different people are bracketed,
analyzed, and compared to the identity of the essences of the phenomenon, for
example, the essences of loneliness, the essence of being a mother, the essence of
being a participant in a particular program. The assumption of essence, like the
ethnographer's assumption that culture exists and is important, becomes the
defining characteristic of a purely phenomenological study. [p. 70, emphasis in
original]
Phenomenology is focused on individuals' meaning making as the quintessential element
of the human experience (Patton, 2002). The important findings derived from
phenomenology are an understanding of a phenomenon as seen through the eyes of those
who have experienced it. Phenomenological inquiry holds the assumption that there is an
essence or essences to shared experience (Patton, 2002). The focus of a
phenomenological study is in uncovering and interpreting the inner essence of the
participants’ cognitive processing regarding some common experience. Thus meaning is
central to the experience. In general “a phenomenological research is well suited for
studying affective, emotional, and often intense human experiences” (Merriam, 2009, p.
26).

In general “a phenomenological research is well suited for studying affective, emotional,

and often intense human experiences” (Merriam, 2009, p. 26).

4
The most typical qualitative research designs done in educational settings are the basic
qualitative research design and the case study design.

Note 4: Whatever qualitative research design the learner proposes it is imperative that the learner
follow, cite, discuss, and reference an authoritative guide for the particular research design and
corresponding data analysis [e.g., Action Research – Carr & Kemmis, Glanz, Kemmis &
McTaggart, Schön; Basic Qualitative – Merriam, Miles & Huberman; Case Study - Merriam,
Stake, Yin; Grounded Theory – Charmaz, Corbin, Corbin & Strauss, Glaser & Strauss;
Ethnography – Denzin, Fetterman, LeCompte & Preissle; Phenomenology – Colaizzi, Giorgi,
Moustakas, van Manen].

Mixed Methods

Mixed methods research is “research in which the investigator collects and analyzes data,
integrates the findings, and draws inferences using either qualitative and quantitative approaches
or methods in a single study or program of inquiry” (Tashakkori & Creswell, 2007, p. 4).
“While mixed methods have an intuitive appeal, they also demand that the researcher be expert
in both approaches to research or work with a team that has such expertise” (Mertens, 2010, p.
293). Therefore a mixed methods study requires the support of a mentor or a committee member
who is familiar with either mixed methods research, or has sufficient knowledge and skills in
both quantitative and qualitative research. Examples of acceptable mixed methods designs or
approaches include:

 Sequential Explanatory
 Sequential Exploratory
 Sequential Transformative
 Concurrent Triangulation
 Concurrent Embedded
 Concurrent Transformative
 Dominant-Less Dominant

Other Acceptable Research Designs

This category includes research designs that might be either quantitative or qualitative or blended
as in mixed methods or mixed modes.

 Action Science Research (ASR) – ASR is research into the structure and
processes of systems at every level for the purpose of bringing about change and
improvement. Systems in ASR may be individuals, small groups such as classes,
organizational systems, and large-scale social systems. The conceptual framework
of ASR is drawn from systems theory, action science theory, or change theory.
Essential to ASR is a carefully designed intervention for bringing about change at
a specific site (often the workplace of the researcher). The intervention is
followed by assessment of the effects of the intervention on the system, and a

5
 sharing of those results with stakeholders of the system. While continuation of the
cycle of improvement is typical of ASR projects in the real world, dissertation
research limits the study to a single cycle and uses the last chapter of the
dissertation to describe plans for later cycles. Generalizations can be made from
the results at a particular site to theory and/or to practice at other similar sites.

In education, ASR is often called participatory action research (McIntyre, 2008),

or planned change (Schein, 2011).

 Delphi Technique – The Delphi technique is “a method for structuring a group

communication process so that the process is effective in allowing a group of
individuals, as a whole, to deal with a complex problem” (Linstone & Turoff,
1975, p. 3). Delbecq, Gustafson and Van de Ven describe it as “a method for the
systematic solicitation and collation of judgments on a particular topic through a
set of carefully designed sequential questionnaires interspersed with summarized
information and feedback of opinions derived from earlier responses” (1975,
p.10). Also called a Jury of Executive Opinion, it is a process that integrates the
judgments of a group of experts to produce a product such as a strategic plan or an
assessment of technology innovations, policy impact, or social change. Key
features of this technique include forecasting, involving a group or panel of
experts, the use of remote group communication, the use of an iterative research
process, and a goal of reaching or attempting to reach group consensus.

 Program Evaluation – Lodico et al. (2010) describe program evaluation as

follows: The field of education is filled with programs designed to improve both
learning and teaching. Examples of these programs include a reading intervention
program design to help struggling readers or a teacher-training program designed
to help teachers integrate technology into lessons. Program evaluation is designed
to make decisions about such programs. Although program evaluation uses
quantitative and qualitative methods, its overall purpose is different from most
other types of research. Whereas quantitative and qualitative researchers study
programs, findings from such studies typically are slow to change or improve the
programs themselves. In program evaluation, however, findings are often used
for ongoing or short-term decision-making purposes, and programs can be
changed or “improved” based on the results of a single evaluation. (p. 41)

Other Research Designs that Require Permission on an Exceptions Basis

Other possible research designs not typically done in the School of Education are listed below.
These designs, if proposed by a learner, would require permission on an exceptions base and the
learner would need to provide evidence of doctoral level knowledge of and expertise in the
design. The mentor and specialization chair would need to pre-approve any of these research
designs if proposed by a learner. Most learners are encouraged to use a more traditional design
from the list of acceptable research methods above.

6
Quantitative

 Futuring Studies.
 Meta-analysis.

Qualitative

 Analytic induction.
 Anthropological research.
 Autoenography.
 Biography.
 Content/document analysis.
 Conversation analysis.
 Critical qualitative research.
 Discourse analysis.
 Ethnomethodology.
 Event analysis.
 Metaphorical analysis.
 Hermeneutics.
 Hermeneutic Phenomenology.
 Historical research.
 Historical Social Science research.
 Historiography.
 Narrative analysis.
 Natural experiment.
 Phenomenography.
 Photography research.
 Postmodern Research.
 Secondary Analysis of Qualitative Data.
 Sociometry Study.
 Semiotics.
 Unobtrusive Measures.

Other

 Appreciative Inquiry.

7
References

Berg, B. L. (2005). Qualitative research methods: For the social sciences (5th ed.). Boston, MA:
Pearson.

Capella University (2011). Dissertation manual (vers. 2.3). Minneapolis, MN: Retrieved from
http://www.capella.edu/iGuidePA/PDF/academics/DissertationManual.pdf

Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for
research. Chicago, IL: Rand McNally & Co.

Carr, W. & Kemmis, S. (1986) Becoming critical: Education, knowledge and action research.
Falmer.

Colaizzi, P. (1973). Reflection and research in phenomenology: A phenomenological study of

learning. Dubuque, IA: Kendall-Hunt.

Colaizzi, P. F. (1978). Psychological research as the phenomenologist sees it. In R.S.

Valle & M. King (Eds.), Existential-phenomenological alternatives for psychology (pp.
48-71). New York: Oxford University Press.

Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative

analysis. Thousand Oaks, CA, Sage.

Corbin, J. M. (2008). Basics of qualitative research: Techniques and procedures for developing
grounded theory (3rd ed.). Thousand Oaks, CA: Sage.

Corbin, J. & Strauss, A. (2008). Basics of qualitative research. (3rd ed.). Thousand Oaks, CA:
Sage.

Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods

approaches (3rd ed.). Thousand Oaks, CA: Sage.

Delbecq, A. L., Gustafson, D. H. and Van de Ven, A. H. (1975). Group Techniques for program
planning: A guide to nominal group and Delphi processes, Management Application
Series, Glenview, IL: Scott Foresman and company.

8
Denzin, N. K. (1986), ‘Postmodern Social Theory’ in ethnographic practice. Sociological Theory
4, 194-204.

Denzin, N. K. (2010). Moments, mixed methods, and paradigm dialogs. Qualitative inquiry
16(6), 419-424.

Denzin, N. K. & Lincoln, Y. S. (Eds.). (2011). The SAGE handbook of qualitative research (4th
ed.). Thousand Oaks, CA: Sage.

Fetterman, D. M. (1989). Ethnography: Step by step. Thousand Oaks, CA: Sage

Fetterman, D. M. (ed.) (1984). Ethnography in educational evaluation. Beverly Hills, CA: Sage.

Gall, M. D., Borg, W. R., & Gall, J. P. (2006). Educational research: An introduction (8th ed.).
White Plains, New York, NY: Longman.

Giorgi, A. (1985). The phenomenological psychology of learning and the verbal learning
tradition. In A. Giorgi (Ed.), Phenomenology and Psychological Research. Pittsburgh:
Duquesne University.

Giorgi, A. (1988) Validity and reliability from a phenomenological perspective. In Recent

Trends in Theoretical Psychology (Baker W.J., Mos L.P., Rappard H.V. & Stam H.J.,
eds). Springer-Verlag, New York, pp. 167–176.

Giorgi, A. (1997). The theory, practice, and evaluation of the phenomenological method as a
qualitative research procedure. Journal of Phenomenological Psychology, 28(2), 235–
260.

Giorgi, A. (2006). Concerning variations in the application of the phenomenological method.

Humanistic Psychologist, 34 (4), 305-319.

Giorgi, A. (2008). Concerning a serious misunderstanding of the essence of the

phenomenological method in psychology, The Journal of Phenomenological Psychology,
39 (1), 33-58.

Glaser, B. G. & Strauss, A. L. (1967). The discover of grounded theory. Chicago: Aldine.

Kemmis, S. & McTaggart, R. (1982) The action research planner. Victoria, Australia: Deakin
versity Press.

LeCompte, M. D., and Preissle, J. (1992). Toward an ethnology of student life in schools and
classrooms: Synthesizing the qualitative research tradition, In M.D LeCompte, W.L.
Millroy, and J. Preissle (Eds.). The handbook of qualitative research in education.
Orlando, FL: Academic Press.

9
LeCompte, M. D., and Preissle, J., with Tesch, R. (1993). Ethnography and qualitative design
research (2nd ed.) Orlando, FL: Academic Press.

Leedy, P. D. & Ormrod, J. E. (2005). Practical research: Planning and design (8th ed.). Upper
Saddle River, NJ: Pearson.

Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage Publications.

Linstone, H. A. and Turoff, M. (eds) (1975). The Delphi Method: Techniques and applications.
Reading, MA: Addison-Wesley.

Lodico, M. G., Spaulding, D. T., & Voegtle, K. H. (2010). Methods in educational research:
theory to practice (2nd ed.). San Francisco, CA: Jossey-Bass.

McIntire, A. (2008). Participatory action research. Vol. 52. Thousand Oaks, CA: Sage.

Mertens, D. M. (2010). Research methods in education and psychology: Integrating diversity

with quantitative and qualitative approaches (3rd ed.). Thousand Oaks, CA: Sage.

Miles, M. B., & Huberman, A.M. (1994). Qualitative data analysis (2nd ed.). London, UK: Sage.

Merriam, S. B. (1998). Qualitative research and case study applications in education. San
Francisco, CA: Jossey-Bass.

Merriam, S. B. (2009). Qualitative research: A guide to design and implementation (2nd ed.)
Francisco, CA: Jossey-Bass.

Moustakas, C. (1994). Phenomenological research methods. Thousand Oaks, CA: Sage.

Myers, L. S., Gamst, G., & Guarino, A. J. (2006). Applied multivariate research: Design and
Interpretation. Thousand Oaks, CA: Sage.

Neuman, W. L. (2006). Social research methods: Qualitative and quantitative approaches (6th
ed.). Boston, MA: Allyn & Bacon.

Patton, M. Q. (1990). Qualitative evaluation methods (2nd ed.) Thousand Oaks, CA: Sage.

Patton, M. Q. (1997). Utilization-focused evaluation: The new century text (3rd ed.). Thousand
Oaks, CA: Sage.

Patton, M. Q. (2002). Qualitative & evaluation methods (3rd ed.) Thousand Oaks, CA: Sage.

Schein, E. (2011). Kurt Lewin's change theory in the field and in the classroom: Notes toward a
model of managed learning. Retrieved from http://www.solonline.org/res/wp/10006.html

Schön, D. (1983) The reflective practitioner: How professionals think in action. New York, NY:
Books.

10
Stake, R. (1995). The art of case research. Newbury Park, CA: Sage.

Stake, R. E. (1981). Case study methodology: An epistemological advocacy. In W. W. Welsh

(Ed.), Case Study Methodology in Educational Evaluation. Proceedings of the 1981
Minnesota Evaluation Conference. Minneapolis, MN: Minnesota Evaluation and
Research Center.

Tashakkori, A., & Creswell, J. (2007). The new era of mixed methods. Journal of Mixed Methods
Research, 1 (1), 3-8

van Manen, M. (1990), Researching lived experience: Human science for action sensitive
pedagogy, Albany, NY: State university of New York Press.

Yin, R. (1993). Applications of case study research. Newbury Park, CA: Sage
Publishing.

Yin, R. K. (1998). "The abridged version of case study research: Design and method."
In L. Bickman and D.J. Rog, Handbook of Applied Social Research Methods.
Thousand Oaks, CA: Sage Publications.

Yin, R. K. (2009) Case study research: Design and methods (4th ed.). Thousand Oaks, CA: Sage.

11