Chat 1

1. Educational Research and the Scientific Method (SM)

used to guide empirical testing of various propositions and

theories in education or any field of inquiry

SM is how good researchers go about learning, in a systematic

way, what does and does not work in education.

Example:

Theoretical Mapping for Performance from Self-efficacy Perspective

Also useful for classroom teachers so they can assess the viability
and utility of classroom practices and instruction.
What are the basic steps of the SM?

Steps vary by author, but consists generally of the following:

(1) identify problem

(2) set expectations of findings to help develop and direct study to

be executed

(3) collect data

(4) analyze data

(5) draw conclusions

Research reports and published articles follow format of SM.

What sections of a research report correspond with the SM steps?

(Recall the format of a research report:

http://www.bwgriffin.com/gsu/courses/edur7130/content/format_of
_a_research_paper.htm )

Scientific Method Sections of Research

Report
Identify problem = ?

Set expectations of findings = ?

Collect data = ?

Analyze data = ?

Draw conclusions = ?

Scientific Method Sections of Research

Report

Identify problem = Purpose

statement/introductory
section of report;
literature review

Set expectations of findings = Hypotheses or Research

Questions

Collect data = Method section of

report

Analyze data = Analysis and Results

section of report

Draw conclusions = Conclusions/discussion

Big Picture of this Course what will we learn?

learn how educational researchers execute, carefully, the

various steps of the SM

and how those steps are reported in research reports.

Throughout this course we will cover different aspects of

research reports.

For example, soon we cover the literature review, then move to

hypotheses/research questions, methods of collecting data,
methods of analyzing data, and so on. All this ties into the SM
and the systematic approach researchers use to test various
educational interventions and outcomes of interest.

2. Purpose Statement (or Problem Statement)

Most research reports contain an introductory section with purpose

statement or problem statement.

What role do these statements play in the report?

Purpose statements:
 sometimes can be several statements or a paragraph,

function as the theme of the report --- they provide the reader
with a brief introduction explaining the intent/purpose of the
study which follows,

and may often highlight the theoretical or practical

importance of study.

Details of the study will be revealed later in the report (e.g.,

methods used to execute the experiment will be provided in
the Method section; findings from the study will be reported in
the Results section).

Purpose statements differ across disciplines.

May be a statement, for example:

The purpose of this study was to compare differences

between high and low achievers in the ninth grade on
measures of originality and divergent thinking.
In this study we investigated whether the ability to
discriminate among parts of speech increased with
chronological age and educational level.

Or may be communicated through questions:

Is the relationship between age and dropping-out behavior

correlated with students perception of the utility of the
program to their future needs?
 How do three counseling techniques differ in their
effectiveness in decreasing test anxiety in high school
juniors?

What is the relationship among leadership skills, intelligence,

and achievement motivation of high school seniors?

Other examples can be found in the course notes entitled

Purpose Statements (sometimes call Problem Statements)

3. Literature Reviews

(a) Why literature reviews, who benefits who?

Two perspectives: researcher and reader.

How does researcher benefit?

(a) Generation of research ideas by seeing gaps or weaknesses in

field

the more familiar one becomes with this literature, the better
one will be at locating potential research topics because
weaknesses in some studies or gaps in our understanding of
certain behaviors become more apparent.

(b) Historical overview of field

Lit. review helps a researcher better understand what was

done in previous studies. The review presents a history/story
of past endeavors by other researchers in this area.
(c) Prior methods learn how difficult studies executed

This can be very helpful because it shows how others solved

difficult design issues and study problems.

For example:

Academic and recreational reading interest among young

readers

How would you measure academic and recreational reading

interest among kindergarten or 1st grade students who are
weak readers?

Possible solutions:

Could I simply give these students statements such as the

following?

"I enjoy reading recreationally from books found in the library"

1 = strongly disagree

2 = disagree

3 = somewhat agree

4 = agree
 5 = strongly agree

Can children of this age handle complex statements and rating

scales?

Literature review could help us find a measure of reading interest

for young readers.

Use Google or Google Scholar:

https://www.google.com/

http://scholar.google.com/

Use Google - What search terms would you use?

academic and recreational reading inventory scale kindergarten

students alpha

In summary, a good literature review can help one uncover

solutions to problems that may be faced by the researcher,
provides a historical overview, and also helps one see potential
studies and weaknesses in the field under investigation.

(b) Does the reader of your research report benefit

from the literature review in any way?

literature review is the story behind your research -- it

explains the logical development of your study and places
your study in the context of other studies on the same topic.
After reading the review, the reader should then be able to
understand why you are conducting the current study (the
 reader should understand the theory/logic driving your study,
should see the need for your study);

at the end of the literature review, readers should understand

your research questions and hypotheses and see the logic,
based upon the review, of why you formed the particular
research questions and hypotheses you present.

(c) Boolean Logic

What effect does having a search with OR vs. AND have on search
results? This is an issue of Boolean logic --- how does inclusion of
OR and AND filter search results?

OR broadens the search, includes additional results; OR tells

search to find A or B so if either A or B appear the results are
displayed.
AND restricts the search so both A and B must appear
otherwise results are not displayed.

To see these two filters in action, do a search in Google Scholar

http://scholar.google.com

Google responds to capital AND/OR differently from lower case

and/or; type exactly what I have below:

Search 1 =
 Dropouts OR (minimum competency testing)

How many pages hit?

About 125,000 results (0.03 sec)

Search 2 =

Dropouts AND (minimum competency testing)

How many pages hit?

About 20,600 results (0.17 sec)

In summary,

use of AND in the search means a filter is set in the search

and finds only those articles that contain mention of
dropouts and minimum competency testing (both phrases
must be present),

whereas use of OR produces a search that seeks for

dropouts or minimum competency testing --- it is not
 necessary in this search for both terms to be present in the
same article.

For research purposes,

if I am interested in reading literature that reports the possible

relationship between high school dropouts and results from
minimum competency testing --- that is, does ones
performance on a competency test influence ones decision to
drop out of school --- which is the better to use, AND vs. OR,
and why?

Yes, use of AND will provide better results because those hits
obtained will be more focused on the specific search conducted,
the possible link between competency testing and dropping out of
school. Otherwise use of OR will result in many studies completely
unrelated to that you wish to conduct.

(d) Characteristics of Written Reviews

Most of this is self-explanatory so you can read this on your own.

Will focus on a few characteristics.

Primary vs. secondary references how different?

Primary references show that you have found the source firsthand.
Secondary references mean that you found it in someone else's
work but you didn't actually find the source yourself. Also, a
primary source is one in which the author is the one who collected
or observed the data (if the report is of empirical research), while a
secondary is one in which the author did not collect data, but
instead relied upon others work. In a secondary source, the author
is removed from the observations -- the author did not perform the
experiment, study, etc., but simply summarizes what was found by
others.

If I cite a study from the author who collected and analyzed data,
that is a primary source. If I cite someone who summarized others
research, that is a secondary source.

An autobiography should be primary source, and biography

secondary.

The journal Review of Educational Research specializes in

summary literature reviews---would this be a source for primary or
secondary work?

Empirical studies these are usually preferred in reviews,

what is meant by empirical?

An empirical study is simply one in which data are collected to

address questions or hypotheses. If no data were collected to
address an issue or question, then that is not an empirical piece of
work.

If I say I believe this will work or Logic suggests this will work
or Theory suggests this will work I am making non-empirical
statements. If I say, Evidence from data suggests this will work
then I am making a statement based upon empirical support --
data driven.
How can one identify whether an article or report is empirically
based --- that is, the authors collected data to address a question?

Yes -- one can usually identify an empirical work by the inclusion of

a Method section; publications without some description of
Method are usually not empirical. The Method section explains
how data were collected (usually includes sampling or
participants, measurement, and procedures).

What are data?

For the most part data are recorded bits of information. Data can
range from numbers (such as ITBS scores) to words (such as
transcribed conversations). Contrary to some beliefs, data do not
have to be limited solely to numbers (quantitative information),
data can also be words, pictures, etc. (qualitative information).
This chat session, once posted on the Internet for all to view, will
be a source for data.

4. Variables

What is a variable?

Sometimes folks write that a variable is something that changes:

Does your race change? Does your sex, based upon your
chromosomes (xy or xx), change?

A variable is anything that varies and has more than one unique
category.

Examples:

Age in years = 1, 2, 3, 4, 5, 6, , 48, 49, 50, etc.

 Test Scores = SAT verbal ranges from 200 to 800; 200, 201,
202, 203, etc.

Sex: Biological distinction with Female and Male being most

common categories of sex

Gender: Psychological state often measured on scale that

ranges from feminine to masculine with multiple steps
between these anchor points

If only one category is present, then is it a variable?

For example, if everyone in this class is female, is sex a variable in

this class?

If only one category present, then it is a constant.

What are the variables in these hypotheses?

1. There is no difference in Body Mass Index (BMI) between

females and males?

Two variables: Sex and BMI

Why are female and male not variables?

These are categories of the variable sex. It is easy to confuse

categories with variables, so watch for this when identifying
variables or writing hypotheses.

2. The higher ones level of academic self-efficacy, the lower will

be ones test anxiety.

(Note that academic self-efficacy and test anxiety are measured

on a 20 point scale ranging from 1 = low to 20 = high.)
Also two variables: Academic Self-efficacy and Test Anxiety

5. Measurement

What is measurement?

Process of assigning labels to categories of a variable.

Is quantification required for measurement to occur? Must

numbers be assigned in order for measurement to occur?

No, labels for variables may be non-numeric. See examples below.

Questionnaire Item:

What is your sex?

Female ____
Male ____
Etc.

What is your age in years? _____

What was your pretax income last year? _____

6. Scales of Measurement

Nominal: has only unranked categories (i.e., no inherent rank to

categories)

Examples: sex, race, type of flower

Ordinal: categories with inherent rank, i.e., ranked categories

(i.e., this makes it easy to sort categories from high to low, more
to less, etc.)
 Examples:

Questionnaire Item rate instructor on the following

dimensions

(a) The instructors content was well organized:

Strongly disagree
Disagree
Somewhat agree
Agree
Strongly Agree

(b) The instructor presented material in a clear manner.

Use scale above

(c) The instructor was open to student questions, comments,

and concerns

Use scale above

SES socio-economic status (originally measured by three

indicators: educational level, income, and occupational prestige)

High
Middle
Low

Interval: ranked categories with equal distance between

measuring categories or measuring device

Examples: time, distance

Ratio: same as interval, but also has a true zero point (a true start
or end point); allows for formation of ratios
 Examples: time it takes to complete a task; number of test
items answered correctly

What is the key distinction between nominal variables and all

other types of variables?

The key distinction between nominal variables and all other

variables is that the categories of nominal variables have no
natural rank to them. So, nominal categories cannot be ranked,
but the other scales of measurement, including ordinal, interval,
and ratio, all have categories that can be ranked from high to low,
from more to less, from best to worse, etc. For example, identify
flowers by type (e.g., roses, pansies, etc.) --- most would agree
there is no natural ranking to these types, just qualitative
differences among flowers. However, if the variable is frequency of
occurrence of certain types of flowers in ones yard, then we take
a count of types of flowers. This variable, count of flower types,
does have categories that can be ranked (0, 1, 2, 3, etc.), and is
therefore not nominal.

7. Types of Variables

(a) Qualitative Variable

Nominal or categorical (i.e., no inherent rank to categories), or

ordinal variable with limited number of categories (e.g., SES with
three categories of low, middle, high --- treat this as qualitative or
nominal for convenience of statistical analysis)

Examples

Sex
Race
Types of flowers

(b) Quantitative Variable

Ranked categories (ordinal, interval, or ratio, assuming the ordinal
measure contains many ranked categories

Examples

Number of test items answered correctly

Weight in lbs.
Number of pages read over the summer
Age
Weight
Score on FRER/EDUR 7130 Test 1

(c) Practice Exercise

Example 1: If everyone buys a bag of apples, and we each

count the number of whole apples in our bags, is the count of
apples a qual or quan variable?

Yes, number of apples in a bag is quantitative.

Which scale of measurement is the count of whole apples in our

bags (Nominal, Ordinal, Interval, or Ratio)?

Remember the four criteria that define scales of measurement:

CATEGORIES:
Does the count of apples in a bag have distinct categories,
and if yes, what are those categories?

Categories; RANK:
Does the count of apples in a bag have distinct categories
that can be ranked? For example, are 7 apples more than 6
apples? Are 4 apples less than 5 apples?

Categories; rank; EQUAL INTERVALS:

 Does the count of apples form equal intervals? For example,
the difference between 2 apples and 1 apple is 1 apple. The
difference between 15 apples and 14 apples is 1 apple. Does
this 1 apple represent the same amount of difference no
matter when on the scale (on the count of apples) we place
this 1 apple?

Categories, rank, equal intervals; TRUE ZERO:

Is there a true zero point with the count of apples?

So what scale of measurement is count of apples?

Since count of apples has all four criteria present it is a ratio

variable.

Recall that one way to determine if a variable is ratio is to consider

whether one can form a RATIO between two quantities. For
example, if I have 5 apples and someone else has 15 apples, then
the other person has three times as many apples as I have --- 15/5
= 3. Ratios can only be formed with ratio level data.

Example 2: What about the time it takes individuals to

complete a given task, is this variable quantitative or
qualitative? Also, what scale of measurement is this variable?

Quantitative and Ratio -- Since the categories of time are rank-able

(i.e, 4 seconds is longer than 3 seconds, and 3 seconds is longer
than 2 seconds, etc.), the time it takes one to complete a task
represents a quan variable. This is also a ratio variable since
differences in seconds is precise and equal (e.g., the difference
between 12 and 13 seconds is 1 second, and the difference
between 115 seconds and 116 seconds is 1 second, and the 1
second difference across this range is equal).

What about: The classification of people into student groups in

high school (such as nerds, athletes, and losers). Quan or qual
variable? Also, what scale of measurement is this variable?

These groups represent categories of a qual variable and therefore

nominal (no natural rank; they are simply different groups).

What about this, we take the classification used above (nerds,

athletes, losers) and provide a different label now with
numbers, like this: group 1, group 2, and group 3? Is this now
a qual or quan variable? Also, what scale of measurement is
this variable?

This variable represents a simple label transformation (nerds = 1,

athletes = 2, losers = 3). Does changing the name of the group
labels make this a quan variable now instead of a qual variable?

Despite using numbers 1, 2, and 3, this is not a quantitative

variable. If there is no inherent ranking to these categories,
changing the labels will not change the scale of measurement. My
point here is to illustrate that use of numbers does not necessarily
make a variable quan. So one should not focus on whether a
number is used to decide whether a variable is qual vs. quan.
Rather, one should ask whether the categories have a natural rank
to them. We simply do not know what the labels group 1, group 2,
etc. represent. If these groups are formed by the random
assignment of students to group, then we have only a qual
variable, for example.
 What about computer chip makers AMD, SIS, and Intel. Is
this variable (computer chip maker) qual or quan?

Since these represent names of different companies, with no

natural rank to the names of the companies, this would be
qualitative.

(d) Independent Variable (IV) and Dependent Variable

(DV)

That which precedes the dependent variable in time and is

expected to predict or influence the dependent variable

That which follows the IV in time and is expected to be predicted

or influenced by the independent variable

Examples find the IV and DV in these hypotheses and determine

whether the variables are Quantitative or Qualitative

I. There will be a difference in math scores between males and

females.

IV = sex (categories of sex are male and female)

DV = math scores

Is sex, qual or quan?

Is math score, qual or quan

sex (categories of sex are male and female) qual

math scores quant

II. The more time spent studying, the greater will be ones final
exam test score.

IV = time spent studying

DV = final exam scores
Time spent studying, qual or quant?
Final exam test score, qual or quant?

time spent studying (how measured in hours) quan

final exam scores quan

III. Students taught on-line will score at a higher level on the

terminal examination than students taught in the classroom
format. Which is IV and DV?

IV = format of instruction (on-line vs. classroom)

DV = terminal examination [scores]

Format of instruction, qual or quant?

Terminal examination scores, qual or quant?

Format of instruction qual

Terminal examination scores quan

IV. There is a negative association between academic self-

efficacy and test anxiety.

(Academic Self-efficacy is ones confidence in their academic

ability to engage in academic activities.)

What are the variables in this hypothesis?

Two: academic self-efficacy and test anxiety

Which is IV and which is DV?

 (a) Is this possible the more anxious one about a test, the
lower will be their confidence (and hence efficacy) in doing
well on the test? If yes, which is IV and DV?

(b) Is this possible the more confidence one has about a

topic, the less anxious that person will be about upcoming
tests? If yes, which is the IV and which is the DV?

This is an example where it is not clear which is IV and DV, so

more information would be needed; we would need to know the
theory driving the study or the study design in order to
determine IV and DV.

V. Student ratings of the instructors teaching and student

perceptions of amount learned in the course both vary
according to student level of pre-course motivation for the
course and instructor presentation of the course content.
Which are IV and DV?

There are four variables here: (a) ratings of the instructors

teaching; (b) student perceptions of amount learned; (c) level of
pre-course motivation; and (d) instructor presentation of the
course content.

Which are IV and DV?

The IVs are: (c) level of pre-course motivation; and (d) instructor
presentation of the course content, and the DVs are: (a) ratings
of the instructors teaching; (b) student perceptions of amount
learned.