DATA COLLECTION AND ANALYSIS

TCUP Fundamentals of Education Research Workshop

Mercy Mugo
October 31, 2014

Quality Education for Minorities (QEM) Network

The Research Linkages

Identify the
Research
Question(s)

Determine
the Research
Methodology

Collect the
Data

Data
Analysis
and Report
of Findings

What is Data Collection?

A detailed plan of procedures that aims to

gather data for the purpose of answering a
research question(s)

Quantitative Data
Collected in standardized manner
Presented in numerical format
Analyzed using statistical techniques
Results more generalizable

Qualitative Data
Collected in natural setting
Presented in narrative format
Thick in detail and description
Analysis often emphasizes understanding
phenomena as they exist

Data Collection Methods

Quantitative Methods
Surveys
Tests
Rubrics
Checklists

Qualitative Methods

Observations
Interviews
Focus Groups
Document Review/Analysis
Case Studies
Photographs, Videos

Cultural Responsive Data Collection

Methods
Talking Circles
Visiting
Performance-based Assessment
Appreciative Inquiry

Pros and Cons of Quantitative Methods

Method

Advantages

Disadvantages

Surveys

Inexpensive
Self-report may lead
Good for gathering
to bias
descriptive data
Data lack depth
Cover a wide range of No control for
topics
misunderstood
A variety of software for questions, missing
analysis
data

Pros and Cons of Quantitative Methods

Method Advantages
Tests
Objective information on
what the test taker
knows and can do
Can be constructed to
match level of skills
Easy to administer/score
Provide hard data
Accepted by public

Disadvantages
Time consuming
Biased against some
groups
May be subject to
corruption via
coaching or cheating

Pros and Cons of Qualitative Methods

Method

Advantages

Interviews Yield richest data,

insights
Permit face-to-face
contact
In-depth exploration of
topics
Allow interview to
explain or clarify
questions

Disadvantage
Expensive and time
consuming
Need well trained
interviewers
Interviewee may distort
information
Large volume of information
may be difficult to transcribe

Pros and Cons of Qualitative Methods

Method Advantages
Focus
Groups

Useful to gather
different viewpoints
and new insight
Less time required
Subject matter is not
sensitive

Disadvantage
Not suitable for
generalization
Require qualified
facilitator
Peer pressure may
inhibit responses

Pros and Cons of Qualitative Methods

Method
Advantages
Observations Provide direct
information
Permit observer to
enter into and
understand context
Exist in natural, flexible
setting
Good for identifying
unanticipated outcomes

Disadvantage
Expensive
Time consuming
Need well qualified
observers
Selective perception of
observer may distort data
Behavior observed may not
be representative of a
group/situation

Pros and Cons of Qualitative Methods

Method

Advantages

Disadvantage

Document
Analysis

Inexpensive
Available locally
Grounded in setting
and language in
which they occur
Provide information
on historical trends

May be incomplete
May be inaccurate
Challenges locating
suitable documents
Time consuming

When Choosing Methods, Consider

Purpose of study (research questions)
Respondents/data sources
Resources available
Type of information needed
Value of using multiple methods
Importance of ensuring cultural appropriateness

Other Considerations..
Understand the community
Involve community
Allow time to establish relationships
Take care in constructing and asking questions
Respect cultural protocols
Provide incentives

DATA ANALYSIS

What is Data Analysis?

Summarizing data into manageable format to
communicate its meaning
Reflecting on the data and searching for
patterns
Seeking out the story in the results

Types of Statistics in Quantitative Research

Descriptive Statistics

used to
organize,
describe, and
summarize a set
of data.

Inferential Statistics

used to draw
inferences about
characteristics of a
population based on
what is known about a
sample drawn from
that population.

Types of Statistical Analyses in Quantitative

Research
Descriptive Statistics

Measures of central
tendency
Mean,

variance,
standard deviation

Parametric Tests

median, mode

Measures of variability
Range,

Inferential Statistics

t-tests, Analysis of variance

(ANOVA), Regression
analysis

Non-parametric Tests

Chi-Square test; the sign

test

Data Analysis in Qualitative Research

Read all data, get a sense of the whole
Code data, tag items with same meaning using
using unique codes
Identify patterns/themes among the codes
Represent themes (writing, visual, etc.)
Interpret and make meaning out of the data

Qualitative Data Analysis

Mixed Methods Designs

Utilizes both quantitative and qualitative data
collection methodologies
Major designs:
Convergent Design
Explanatory design
Exploratory design
Embedded Design

Convergent Design
Collect

qualitative and quantitative data

concurrently
Analyze the two datasets separately
Mix the two databases by merging results
during interpretation
Creswell. J (2012): Borrowed from Abraham S. Fischler, Presentation Mixed Methods

Explanatory Design
Starts

by collecting and analyzing quantitative

data
Uses quantitative results to inform subsequent
qualitative inquiry
Uses quantitative results to shape the qualitative
research questions, sampling, and data collection
Creswell. J (2012): Borrowed from Abraham S. Fischler, Presentation Mixed Methods

Exploratory Design
Starts

by collecting and analyzing qualitative

data
Utilizes qualitative results to build the quantitative
phase
Connects the phases by using qualitative results to
shape the quantitative research questions,
variables, and instrument
Creswell. J (2012): Borrowed from Abraham S. Fischler, Presentation Mixed Methods

Embedded Design
Quantitative (or Qualitative)
Design

Quantitative (or Qualitative)

Data Collection and Analysis
Qualitative (or quantitative)
Data Collection and Analysis
(before, during, or after)

Interpretation

Validity of Results

Validity
Most

important characteristic of the study/assessment

results
Concerned with the appropriateness of the
interpretations made from assessment results
Specific to the interpretation being made and to the
group being assessed

Reliability of Results

Reliability
Concerned

with how well the results can be

replicated
Would a particular technique (or survey) yield
the same results each time?
Reliability does not ensure accuracy

Working with Identifiable Data

Maintaining confidentiality

Presenting accurate information

How can we reconcile these two conflicting

dynamics?

Ways to Address Small Sample Size

Report results in aggregate --- across several
samples to maintain confidentiality
Allow access of raw data to user who may need it
for e.g. evidence-based decision-making, policymaking, budgeting, proposal development, etc.

IRB

approval
Participants permission to release information/data

Example: Impact of Suppression of

Small Data Cells
Negatively affect underrepresented groups
(loss of information)
Value of data significantly diminished
Prevent access to information essential to
providing highly needed opportunities

A report on the series of outreach meetings on the Impact of the Suppression of Small Data
Cells in the Survey of Earned Doctorates (SED) Report (2009).
Prepared by QEM for NSFs Science Resources Statistics (SRS) Division.

Impact of Suppression Continued

Harm diversity-focused initiatives and minorityfocused programs
Difficult finding role models
Difficult designing intervention strategies

A report on the series of outreach meetings on the Impact of the Suppression of Small Data
Cells in the Survey of Earned Doctorates (SED) Report (2009).
Prepared by QEM for NSFs Science Resources Statistics (SRS) Division.

Questions

Thank you!