Running Head: Evaluation of Marathon Gear Module

Evaluation of Marathon Gear Module

Ryan Quesenberry

Tyler Provost

California State University Monterey Bay

IST 622

Dr. Bude Su

June 18, 2019

EVALUATION OF MARATHON GEAR MODULE 2

Table of Contents
● Section I…………………………………………………………………………………..3
○ Introduction………………………………………………………………………..3
● Section II……………………………………………………………………………….....3
○ Methodology……………………………………………………………………....3
○ Prototype………………………………………………………………………......3
○ Learners…………………………………………………………………………....4
○ Expected Outcomes……………………………………………………………….4
○ Psychomotor Domain……………………………………………………………..5
○ Affective Domain………………………………………………………………….5
○ Tryout Process…………………………………………………………………….5
○ Pre-Test…………………………………………………………………………....6
○ Training…………………………………………………………………………....6
○ Post-Test…………………………………………………………………………..6
○ Testing Survey…………………………………………………………………….6
○ Usability Survey…………………………………………………………………..7
○ Observations………………………………………………………………………7
○ Tryout Conditions…………………………………………………………………7
● Section III………………………………………………………………………………...8
○ Results……………………………………………………………………………..8
○ Entry Conditions…………………………………………………………………..8
○ Instruction………………………………………………………………………....8
○ Summary of Data………………………………………………………………….8
○ Outcomes………………………………………………………………………...18
○ Recommendations………………………………………………………………..19
● Section IV………………………………………………………………………………..19
○ Summary………………………………………………………………………....19
● Section V………………………………………………………………………………...20
○ Appendices……………………………………………………………………….20
○ Appendix A: Instructions………………………………………………………...20
○ Appendix B: Pre and Post-Test Questions……………………………………….20
○ Appendix C: Link to Marathon Gear eLearning Module………………………..22
○ Appendix D: Marathon Gear Testing Survey…………………………………... 22
○ Appendix E: Marathon Gear Usability Survey…………………………………..22
○ Appendix F: Instructional Module Storyboard…………………………………..24
EVALUATION OF MARATHON GEAR MODULE 3

Section I

Introduction

This project will evaluate the effectiveness of a training module developed for a capstone

project. The project, and this module, hope to address knowledge gaps encountered by first time

marathoners that can result in pain and discomfort while participating in their first marathon.

The Los Angeles Road Runners (LARR) is the official training team for the Los Angeles

Marathon and each year trains dozens of amateur runners to complete their first marathon. This

is primarily done through hosting weekly “long runs” of varying distance between 6 and 22

miles. As someone who recently completed training for my fifth marathon with LARR I have

been able to both observe and experience some of the shortcomings of their program, specifically

that they are not able to successfully convey information to novice marathoners that will make

their 26.2-mile journey less of an ordeal.

Section II

Methodology

Prototype

The prototype being evaluate is one module developed for a capstone project being

developed by Ryan Quesenberry using mAuthor to create an interactive multimedia presentation.

The module contains narration to guide learners through the module, and information is

presented via video informational diagrams and photos. The information contained in this

module focused on appropriate gear selection for distance runners. This included information on

choosing the correct gear for different weather conditions, and information to guide their

purchase of appropriate running shoes.

EVALUATION OF MARATHON GEAR MODULE 4

The prototype module is designed to be accessed via a web browser. Depending on the

learner’s reading speed and amount of time spent answering questions, the module can be

completed in approximately seven to ten minutes. A storyboard of this module can be viewed in

Appendix F.

Learners

This capstone for which this module was developed had a target audience of members of

the Los Angeles Road Runners and the Los Angeles Running Club who are training for the Los

Angeles Marathon. This evaluation is taking place outside of the L.A. Marathon training

window which occurs between October and March annually. Because of this, we will test

primarily with learners that have some running experience but little, if any marathon experience

to approximate the base knowledge of a first time marathoner. Similar to the members of the

L.A.R.R. & L.A.R.C. all learners have at least basic computer skills and familiarity with using a

web-browser to access the internet.

A total of 10 test subjects were asked to participate, a majority of whom have a

background in law-enforcement along with two retirees, and a librarian. Ages range from early

20’s to late 60’s. While not quite as diverse in background and age as members of the L.A.R.R.

the amount of base knowledge and experience should be similar.

Expected Outcomes

Upon completion of the instructional module learners will be able to demonstrate

improved knowledge of running gear and how to select the appropriate gear for the given
EVALUATION OF MARATHON GEAR MODULE 5

weather conditions. This includes detailed knowledge on running shoes to allow learners to

make informed choices when shopping for their next pair of running shoes in which to train.

Psychomotor Domain

While there is not direct physical practice included within the learning prototype, when

presented with the overall L.A.R.R. training there will be opportunities to apply the content

learned within the module during weekly training runs. Additionally, the learned knowledge can

be applied when shopping for appropriate running gear; while training for a marathon, a runner

will go through two or more pairs of running shoes which will in turn need replacement.

Affective Domain

The instructional prototype implements Gagne’s nine events of instruction, as such it

begins by getting the learners attention. This is done by sharing the fact that the information

contained within the instructional module will help them avoid unnecessary discomfort while

working towards completing their first marathon.

Tryout Process

The process involves the following: pre-test, training, post-test, testing survey, and

usability survey. All links to the tests, training and surveys were presented on a Google

document developed for this process. Surveys were developed using Google forms. All learners

have some basic computer knowledge, and technology is not expected to be an impediment to

the learning process.

EVALUATION OF MARATHON GEAR MODULE 6

Pre-Test

Pretesting is accomplished via a Google survey. This survey will be accessed by learners

via the Google document created for this project. The survey asks questions gauging their

baseline knowledge on the material presented within the prototype. The survey is viewable in

Appendix B.

Training

Training, which was hosted on CSUMB’s “myspace” server, was accessed via a link on

the Google document created for this project. Once the user has accessed the link to the training

module, it will take the learners approximately seven to ten minutes to complete the training. A

link to the training module is listed in Appendix C.

Post-Test

After completion of the training module, the learns will be instructed to complete a

post-test. The post-test will contain identical questions to the pre-test and will be used to

determine if the instructional module was effective. The questions are viewable in Appendix B.

Testing Survey

The testing survey gathers basic information from the learners. This includes the

collection of information on the learners e-mail address, approximate age, their computer skills,

and how they felt regarding the content of the training module. The full survey can be viewed in

Appendix D.
EVALUATION OF MARATHON GEAR MODULE 7

Usability Survey

This survey was included to help evaluate the overall intuitiveness of the instructional

module. The focus of the questions was on the ease of use of the instructional module. There is

also a section for comments. Information learned from this survey will be used to improve this

module, but also in the design of future models in this capstone project. The questions for the

survey can be viewed in Appendix E.

Observation

The learners had two choices to participate in the training. The subjects were either

observed in person or virtually. An email was sent out containing links to all of the tests,

training, and surveys. A Zoom link was provided to the users who wished to participate in their

own location. The participants who did not wish to participate virtually were monitored at a

nearby police station due to many participants being active law enforcement officers.

Observations were recorded for both virtual and in-person learners.

Tryout Conditions

All of the trainings were conducted on a computer or similar technology that had access

to the internet. All of the links to access the tests, module, and surveys were provided to the users

in a Google document. The participants either completed the training in the location of their

choice (observed via Zoom meeting) or at a nearby police station (observed in person) with

either Ryan or Tyler as the facilitator. Each activity was completed by the participants

individually. Each test and survey was completed once by the subjects.
EVALUATION OF MARATHON GEAR MODULE 8

Section III

Results

Entry Conditions

There was little difference between the intended and observed conditions. Similarly there

was little difference between the test audience and the target audience. Membership to the Los

Angeles Running Club and the Los Angeles Road Runners is accomplished primarily online.

Because of this nearly all learners would be expected to have some familiarity with a computer

and using a computer to access resources. With the exception of one learner who’s computer

necessitated a reboot mid prototype, learners navigated the prototype, pre-test and post-test

without incident.

Instruction

Instruction went according to stated direction. One learner did comment that he was hard

of hearing and even at max volume he had trouble hearing everything in the prototype clearly.

There was close-captioning on the video portions of the module but the portions that were only

narrated lacked close captioning or links to the content script. (This is something that should be

addressed in future modules.) After completing the training module all but one learner improved

their score, with a majority of learners receiving 100%.

Summary of Data

Ten learners were asked to complete the prototype including pre-test, post-test, and

survey. Nine learners responded during the allotted testing window. One candidate experienced
EVALUATION OF MARATHON GEAR MODULE 9

technical difficulties with his computer part-way through the process and restarted using a

different e-mail address. The data from this incomplete test is included with the raw data, but

was not used for any final calculations. The results of the pre and post tests can be seen below.

Pre-test Scores Post-test Scores

3 4
2 4
3 3
4 5
3 5
4 5
4 5
3 5
4 4

All but one participant increased their scores from the pre-test to the post-test. The

subject who did not increase their score maintained the same score and did not decrease.

After the true and false questions on the pre-test and post-test, we asked the subjects

multiple questions asking them to rate their knowledge in certain questions related to the topic.

We provided them five levels to choose from ranging from strongly agreeing to strongly

disagreeing.

During the pre-test, we discovered that only one subject agreed they were above average

in running gear knowledge, half of them said they were neutral, and the remainder said they were

not very knowledgeable regarding the topic. After completing the training and taking the post
EVALUATION OF MARATHON GEAR MODULE 10

test, our users were either neutral in their knowledge or agreed they were knowledgeable about

the topic.

During the pre-test, our users said they strongly disagreed or were neutral when asked if

they were familiar with different types of running gear for different types of weather conditions.

After completing the training and post-test, only one of our users had a neutral response while

the rest either agreed or strongly agreed they were familiar with different types of running gear

for different types of weather conditions.

EVALUATION OF MARATHON GEAR MODULE 11

During the pre-test, our users ranged from strongly disagreeing to having a neutral

response when asked if they were familiar with the different features of running shoes. Once

completing the training and post test, our users ranged from having a neutral response to strongly

agreeing with being familiar with the different features of running shoes.
EVALUATION OF MARATHON GEAR MODULE 12

During the pre-test, our users ranged from strongly disagreeing to having a neutral

response when asked if they understood the purpose of most running shoe features. Once

completing the training and post test, our users ranged from having a neutral response to strongly

agreeing when asked if they understood the purpose of most running shoe features.

During the pre-test, our users ranged from strongly disagreeing to agreeing when asked if

they would be comfortable picking out shoes in which to run a marathon. Although during the

pre-test some subjects agreed with being comfortable picking out shoes, over half of them

disagreed or strongly disagreed. Once completing the training and post test, our users ranged

from having a neutral response to strongly agreeing when asked if they would be comfortable

picking out shoes in which to run a marathon.

EVALUATION OF MARATHON GEAR MODULE 13

During the pre-test, our users ranged from strongly disagreeing to agreeing when asked if

they were familiar with different types of moisture wicking fabrics. Although one subject agreed

during the pre-test, over half of them either disagreed or strongly disagreed. Once completing the

training and post test, our users ranged from having a neutral response to strongly agreeing when

asked if they were familiar with different types of moisture wicking fabrics.
EVALUATION OF MARATHON GEAR MODULE 14

During the pre-test, our users ranged from strongly disagreeing to agreeing when asked if

they understood the topic of heat management while running. Once completing the training and

post test, our users ranged from having a neutral response to strongly agreeing when asked if

they understood the topic of heat management while running.

When reviewing the testing survey, we asked our learners to rate themselves on different

questions. We provided them five levels to choose from ranging from strongly agreeing to

strongly disagreeing.

The first question we asked was what they would rate their overall skills are in using

technology. 44% of the users said they were average or below average when using technology.

55% of the users said they were above average or strong when using technology. None of our

users said their skills were poor.

EVALUATION OF MARATHON GEAR MODULE 15

We asked the users if the content covered in the module was relevant and easy to follow.

We also asked them if the time of the module was fair and if they learned new things throughout

the module. The users agreed or strongly agreed to each of those statements we provided.

When asking our users about how easy it was to navigate through the eLearning module,

their replies ranged from agreeing it was easy to strongly agreeing.

EVALUATION OF MARATHON GEAR MODULE 16

When asking our users about the information being appropriate in the eLearning module,

their replies ranged from agreeing it was appropriate to strongly agreeing.

When asking our users about the length of the eLearning module being appropriate, their

replies ranged from agreeing it was appropriate to strongly agreeing.

EVALUATION OF MARATHON GEAR MODULE 17

When asking our users about the organization of the eLearning module, their replies

ranged from agreeing it was organized to strongly agreeing.

When asking our users about our eLearning module being visually pleasing, their

answers ranged from disagreeing to strongly agreeing. Over half of the users strongly agreed

with the eLearning module being visually pleasing. Only 11% disagreed it was visually pleasing.
EVALUATION OF MARATHON GEAR MODULE 18

Outcomes

The working hypothesis for this project was that the instructional module would improve

learners understanding on the subject matter presented (in this case information on running gear

and shoes.) With the understanding that a positive outcome to this training would result in an

improved test scores, the null hypothesis would be no statistical difference between the pre and

post-test scores. The data would be evaluated with a t-test for dependent samples. The degrees

of freedom for these calculations was determined to be 8.

EVALUATION OF MARATHON GEAR MODULE 19

The test statistic was 4.264 which is greater than the critical value of 1.860. The p-value was

.0014 for our one tail test, this is less than our alpha of .05 so the results are statistically

significant and we can reject the null hypothesis. The instructional prototype resulted in a

statistically significant improvement in test scores among learners in this study.

Recommendations

This prototype had a small test audience. After reviewing the pre-test and post-test

answers, we achieved our goal of expanding the user’s knowledge with the eLearning training

module. In the future, we plan on having a larger audience. This will allow us to have a better

understanding of how effective the training actually was.

One of the recommendations one of the users had was that they had difficulty hearing the

prototype while participating. This comment encourages us to be conscious of the volume during

production and ensuring options additional accessibility options are included on future

prototypes

Another recommendation that was made regarding our training was that the users believe

the training would be more effective with more usage of videos and images. This is something

that will be incorporated more into the prototype in the future.

Section IV

Summary

The hypothesis for this prototype was that the Marathon Gear training module would

improve the test scores and knowledge of the learners. After reviewing the pre-test and post-test
EVALUATION OF MARATHON GEAR MODULE 20

results for the training, the results proved that the learners had higher test scores and knowledge

regarding marathon gear after completing the training module. As we continue to work on the

Marathon Gear training module, we will use these results and feedback to help create a more

effective and efficient training module for the future.

Section V

Appendices

Appendix A: Instructions

1. Take the ​pre-test​ to see what knowledge you already have regarding the topic

2. Complete the​ ​eLearning training module​ regarding Marathon Gear.

3. Take the ​post-test ​to see what knowledge you gained from the eLearning module.

4. Fill out the​ ​testing survey​.

5. Fill out the​ ​usability survey​.

Appendix B: Pre and Post-Test Questions

1. The ankle collar helps hold the heal down in place.

a. True
b. False
2. Running clothes are available with reflective elements to enhance safety when running in
low-light conditions.
a. True
b. False
3. Cotton clothes will help you stay dry.
a. True
b. False
4. The upper portion of your running shoe works with the laces to hold your foot like a
puzzle piece.
a. True
b. False
5. Merino wool fabrics absorb sweat and distribute over a large surface area to speed
evaporation.
EVALUATION OF MARATHON GEAR MODULE 21

a. True
b. False
6. I consider myself very knowledgeable when it comes to running gear.
a. Strongly agree
b. Agree
c. Neutral
d. Disagree
e. Strongly disagree
7. I am familiar with the different types of running gear available for different weather
conditions.
a. Strongly agree
b. Agree
c. Neutral
d. Disagree
e. Strongly disagree
8. I am familiar with the different features of running shoes.
a. Strongly agree
b. Agree
c. Neutral
d. Disagree
e. Strongly disagree
9. I understand the purpose of most running shoe features.
a. Strongly agree
b. Agree
c. Neutral
d. Disagree
e. Strongly disagree
10. I would be comfortable picking out shoes in which to run a marathon.
a. Strongly agree
b. Agree
c. Neutral
d. Disagree
e. Strongly disagree
11. I am familiar with different types of moisture wicking fabrics.
a. Strongly agree
b. Agree
c. Neutral
d. Disagree
e. Strongly disagree
12. I understand the topic of heat management while running.
a. Strongly agree
b. Agree
c. Neutral
d. Disagree
EVALUATION OF MARATHON GEAR MODULE 22

e. Strongly disagree

Appendix C: Link to Marathon Gear eLearning module

​http://myspace.csumb.edu/~rquesenberry/gear/

Appendix D: Marathon Gear Testing Survey

1. Email address:
a. ___________________
2. Age:
a. 20-30
b. 31-40
c. 41-50
d. 51-60
e. 61-70
3. Please rate your level of agreement with the statements listed below:

Strongly Agree Neutral Disagree Strongly

agree disagree

The content covered

was relevant to the
module

The content in this

module was easy to
follow

The time to complete

this module was fair

I learned new things

during this module

Appendix E: Marathon Gear Usability Survey

1. Email address:
a. ____________________
2. It was easy to navigate through this eLearning module.
EVALUATION OF MARATHON GEAR MODULE 23

a. Strongly agree
b. Agree
c. Neutral
d. Disagree
e. Strongly disagree
3. The graphics and videos used in this eLearning module were helpful.
a. Strongly agree
b. Agree
c. Neutral
d. Disagree
e. Strongly disagree
4. The amount of information in this eLearning module was appropriate.
a. Strongly agree
b. Agree
c. Neutral
d. Disagree
e. Strongly disagree
5. The length of this eLearning module was appropriate.
a. Strongly agree
b. Agree
c. Neutral
d. Disagree
e. Strongly disagree
6. This eLearning module was organized.
a. Strongly agree
b. Agree
c. Neutral
d. Disagree
e. Strongly disagree
7. This eLearning module was visually pleasing
a. Strongly agree
b. Agree
c. Neutral
d. Disagree
e. Strongly disagree
8. What did you like most about this eLearning module?
a. ______________________________________
9. What can be improved in this eLearning module?
a. ______________________________________
10. Additional comments:
a. _______________________________________
EVALUATION OF MARATHON GEAR MODULE 24

Appendix F: Instructional Module Storyboard

Slide #1 Title: Title Screen Time: 0:30 Seconds

Media Notes Screen Text Media Script

Paused state Marathon Gear: Hello this module

will familiarize you
Text in Center Screen
What you need to with the gear you’ll
Buttons are a right arrow in bottom right
train and compete need to train for
corner linking to next slide.
in a marathon. and compete in a
Media marathon

Image None

Audio Narration1.mp3

Music/Sound None
Effects

Video None

Interaction Quiz

None none

Branching Advance

Next Slide 2 User selection of

button
Prev N/A
EVALUATION OF MARATHON GEAR MODULE 25

Slide #2 Title: Introduction Time: 30

Media Notes Screen Text Media Script

Begins with audio paused. none Would you like to

complete your
marathon while
Media
avoiding
unnecessary pain
Image StartingLine.jpg and discomfort?
Audio Narration2.mp3
A little knowledge
Music none on selecting the
correct running
Video none gear can go a long
way in helping you
Interaction Quiz reach that goal

none

Branching Advance

Next Slide 3 By user

Prev Slide 1
EVALUATION OF MARATHON GEAR MODULE 26

Slide #3 Title: Introduction Time: 2:30 minutes

Media Notes Screen Text Media Script

Video A Runner’s Like other sports

Uniform runners have their
own “uniforms.” A
Media
marathoner can be
in the elements
Image none from 2.5 to 10
Audio Narration3.mp3 hours depending on
their speed.
Music Embedded with Because of this it is
video important what you
wear. Watch the
Video https://youtu.be/sFa2 video below for
KHhZsYk more information
on selecting your
Interaction Quiz gear.

none none

Branching Advance

Next Slide 4 By user

Prev Slide 3
EVALUATION OF MARATHON GEAR MODULE 27

Slide #4 Title: Quiz Time: 1:00

Media Notes Screen Text Media Script

Audio auto plays Quiz Time Now let’s check to

see you were able
take away from the
Media
video.

Image None

Audio Narration4.mp3

Music None

Video None

Interaction Quiz

Matching Quiz

Branching Advance

Next Slide 5 By User

Prev Slide 3
EVALUATION OF MARATHON GEAR MODULE 28

Slide #5 Title: Shoes Time: 0:45

Media Notes Screen Text Media Script

Audio paused none Perhaps the most

important piece of
gear you will
Media
purchase are your
shoes. A properly
Image Shoes.jpg fitted pair of
Audio Narration5.mp3 running shoes are
critical for both
Music None marathon
performance and in
Video None helping to prevent
blisters or more
Interaction Quiz serious injuries.

none none

Branching Advance

Next Slide 6 By User

Prev Slide 4
EVALUATION OF MARATHON GEAR MODULE 29

Slide #6 Title: Reading Time: 3:30

Media Notes Screen Text Media Script

Article will open in separate none. Using information contained in

window. the following runners world
article, the key aspects of a
Media running show and how to select
are described to learner.
Image None
https://www.runnersworld.com
Audio None /gear/a20842305/how-to-buy-th
e-right-running-shoes/
External Article

Video None

Interaction Quiz

none Matching Quiz

Branching Advance

Next Slide 7 By User

Prev Slide 5
EVALUATION OF MARATHON GEAR MODULE 30

Slide #7 Title: Quiz Time: 1:30

Media Notes Screen Text Media Script

Audio auto plays Quiz Time Now let’s check to

see you were able
take away from the
Media
article.

Image None

Audio Narration7.mp3

Music None

Video None

Interaction Quiz

On wrong answer, provide helpful hint. T/F & Multiple

On correct answer, provide reinforcement. Choice

Branching Advance

Next Slide 8 By User

Prev Slide 6
EVALUATION OF MARATHON GEAR MODULE 31

Slide #8 Title: Thank You Time: 0:30

Media Notes Screen Text Media Script

Audio auto plays Good Job You now know

what you should be
wearing for your
Media
upcoming training
runs, and your
Image FinishLine.jpg marathon. If you
Audio Narration8.mp3 have any further
questions on gear
Music None feel free to ask your
coach or
Video None pace-leader.

Interaction Quiz

none none

Branching Advance

Next Slide NA By User

Prev Slide 7