COMPARING TRADITIONAL AND ONLINE ASSESSMENTS 1

Comparing Traditional And Online Assessments

Mark M Miyashita

Castle View High School

COMPARING TRADITIONAL AND ONLINE ASSESSMENTS 2

Introduction:

My name is Mark Miyashita, I currently teach United States History since 1865 to

sophomore students at Castle View High School in Castle Rock, Colorado. Castle View is a

reform school that is based on an Academy Model that helps students find additional instruction

in subject areas that are avenues of potential interest for further study at the post-secondary level

and beyond. Part of the reform emphasis when the school opened was to utilize 21st Century

technologies as part of instructional methods. This has been implemented with the use of

Promethean Boards and a wide variety of technology classes, but many teachers continue to use

traditional methods. The piloting of a Course Management System (CMS) in the form of

Moodle is being conducted and my teaching partner Michael Schneider and I want to see how we

can expand beyond the walls of our classrooms in working with our students. Currently I teach

in a team setting two Humanities II classes, which is a yearlong course, comprised of English

and History.

Problem Statement:

Increasingly there has been an emphasis on the implementation of technology in

education with the development of online and hybrid classes being offered to students, what

needs to be examined is the performance of students on assessments that are delivered online

versus the traditional face-to-face method. Do students prepare better and perform better overall

on assessments when given the opportunity to do so online versus the traditional method.

Purpose and Intended Audience:

The purpose of this research is to study the performance of students completing their

assessments online versus those students that are given the same assessment in a traditional

setting. One group will complete quiz assessments online while the other will complete them in-
COMPARING TRADITIONAL AND ONLINE ASSESSMENTS 3

class with large unit assessments being completed by everyone in class. This data will be used to

compare the performance of students on formative assessments that are delivered through

different methods and how it relates to student preparation. This will be further extended to

compare this data with student performance on a summative unit assessment that will take place

in the traditional setting to determine gaps in student learning, retention, and application of

course material.

The intended audience of this research includes: teachers, administration, students, and

parents as each group is a stakeholder in the process. Teachers will use this to help refine and

craft their practice as it relates to delivery of instruction and assessment of student learning. It

will allow for a review of student study habits and allow teachers to reinforce good prep habits.

Administration will have an interest in the effectiveness of student learning through the online

model and the value it provides for the school and its parents and students. Parents will want to

understand the value of allowing their student to participate in these courses and gain insight to

student study habits. Students will be able to reflect upon their study habits for assessments and

the impact these habits have on their approach and performance.

Research Questions:

• How do students’ scores vary between the two different delivery models for quiz

assessments?

• What relationship/differences exist between the assessment quiz scores and unit

assessments?

Context of the Study

This study will serve as a field experiment to relate the performance of students when

provided an alternative environment for completing an assessment. The two classes being used
COMPARING TRADITIONAL AND ONLINE ASSESSMENTS 4

to complete this research are Humanities II courses that students earn credit for both English and

History that is team-taught. There are 112 students in these two classes that are fully inclusive

with Honors, General Education, and Special Education all grouped together. Each class has

roughly the same number of students from each of these groups. Castle View High School is

located in Douglas County School District RE-1 in Colorado, which is an affluent area.

According to US Census Bureau (2008), “92.3% of the population is white, with an average

household income of $94,042. Those that live in poverty is 2.4%”

As one of the highest performing school districts in the state there is high demand for

student achievement. With budget crunches teachers are being asked to do more with increased

numbers in the classroom, being able to find alternative methods to assessing student learning is

increasingly needed. The use of a CMS like Moodle allows teachers to extend/expand beyond

the traditional four walls of the classroom. With this program we are able to hold online forum

discussions, ensure a digital drop box for student work and provide direct feedback on their

work, and set up assessments to be completed outside of the classroom. Thus it allows a teacher

to maximize instructional time and provide opportunities for student participation outside of the

physical classroom.

However, academic dishonesty, effort, and development of study habits are some

potential problems that are being faced. In completing this research it would it is intended to

discover the impact of using an online assessment has on these areas. Having already seen

questionable practices by students regarding the amount of time spent on assessments and when

they have been logged into the CMS has raised eyebrows about academic dishonesty and

students working with one another on these. Another point of interest is the amount of time

some students put forth on these assessments as time limits exist and the duration a student
COMPARING TRADITIONAL AND ONLINE ASSESSMENTS 5

spends on the assessment is logged for the teacher as well. Often there are times that a student

may have twenty minutes to complete a fifteen-question quiz and uses fewer then three minutes.

Scores have shown a link between potential cheating and lack of effort.

Procedures:

This research will be a field experiment as the two classes will be taught all the same

material, but will be assessed using different methods to help determine the effectiveness of the

online assessments. From this data conclusions will be developed as to how students prepare for,

approach, and complete these assessments and its effect on unit assessments that are taken in

traditional classroom. Determining if there is a difference in how students approach these

assessments is important in looking at the expanded use of CMS platforms within our building

for other teachers.

Selection of these two classes was done on the basis that both classes are the same

subject and are taught by the same teachers using the same methods. Each class is roughly

comprised of the same demographics relating to ethnicity, classification, and enrollment.

Collection of data will take place online for one class through the use of Moodle, which

will record and log student activity. For those completing assessments in class, myself will time

students or teaching partner Michael Schneider to determine the length of time students took to

complete assessments so this data can be used later.

Date Activity
February 1 World War II and Holocaust Pre-Test
February 19th Assessment Quiz 1
February 25th Assessment Quiz 2
March 3 Unit Assessment

Student performance will first be looked at, followed by a comparison with the amount of

time that was spent completing the assessment. Looking at the relationship and results of this
COMPARING TRADITIONAL AND ONLINE ASSESSMENTS 6

data with the class from the traditional setting will follow this. Student performance from the

quiz assessments will then be compared with the Unit Assessment to determine the relationship

and performance of student from each delivery method.

Data Collection:

Date Timeline and Collections

February 1st- Design, Proctor, and Grade Assessments
March 5th
March 5th-12th Collect and Analyze Data using a T-Test to determine if there is a p≤ . 05
March 12th- Complete Final Report
31st

The impact of this research will be the ability of students to complete the assessments at

their convenience versus the structured school day. Some students will feel that it is unfair that

one group has the ability to complete this at home while others will be required to do so in class.

All students are able to use the Internet for schoolwork and have not signed the waiver that

exempts them from using technology as part of teacher instruction. Despite these being two

different sections of the same course an instructor is required to make changes to their delivery

methods based upon the class and allows for differentiation of instruction to further the

development of best teaching practices. The potential for negative results will help me look to

develop better methods for assessing student learning and developing learning skills and

strategies that students need to be successful. Negative results will require an effort of staff to

reconsider its deployment of Moodle as a school wide CMS and the use of technology in

delivering student assessment.

Conclusion:

Potential outcomes for the research being conducted include the following:

• Minimal difference in student performance between those completing the online delivery

compared with the traditional method

COMPARING TRADITIONAL AND ONLINE ASSESSMENTS 7

• Performance of students completing online assessments decreases on the Unit

Assessment.

• Students completing assessments in class use more time to complete the assessments.

• Students completing assessments online have more absences or do not complete

assessments compared with students completing them in the traditional model.

This research is important to education as the push for increased use of technology has many

benefits, but the drawbacks are often not considered. Increasingly issues of academic dishonesty

are becoming commonplace as students lack skills that make them better students. Often the

solution for a student is to simply “google it” and take someone else’s work directly or to simply

find an answer for the purpose of a grade. Students need to develop learning skills that will help

them beyond the purpose of getting a grade. In completing this research it will be interesting to

learn about the study habits of students and the difference in their performance when technology

is not readily available and the assessment environment is equal for all students.
COMPARING TRADITIONAL AND ONLINE ASSESSMENTS 8

References

Douglas County Quick Facts. (2008). Us census bureau. Retrieved (2010, February 15) from
http://quickfacts.census.gov/qfd/states/08/08035.html
COMPARING TRADITIONAL AND ONLINE ASSESSMENTS 9

Appendices:
COMPARING TRADITIONAL AND ONLINE ASSESSMENTS 10

Appendix 1
Raw Data Collection

Student Pre-Assessment Score

COMPARING TRADITIONAL AND ONLINE ASSESSMENTS 11

Student Quiz #1 Score Quiz #1 Time

COMPARING TRADITIONAL AND ONLINE ASSESSMENTS 12

Student Quiz #2 Score Quiz #3 Time

COMPARING TRADITIONAL AND ONLINE ASSESSMENTS 13

Student Unit Assessment Score Unit Assessment Time

COMPARING TRADITIONAL AND ONLINE ASSESSMENTS 14

Appendix 2
T-Test p≤ .05

Class 1 Class 2
Time Avg Avg
Score Avg Avg