Running head: CRITICAL ANALYSIS 1

Paper 1: Critical Analysis

Justin Baffico

California State University Monterey Bay

IST 520

Donald Fischer, Sarah Tourtellotte, PhD

CRITICAL ANALYSIS 2

Introduction

Using Interactive Multimedia to Teach Pedestrian Safety: An Exploratory Study (2005)

by Ann Glang, PhD, and John Noell, PhD, is a quantitative experimental research study. The

purpose of the study was to evaluate an interactive multimedia (IMM) program that teaches

young children safe pedestrian skills. The research did not provide a direct hypothesis to weather

or not the researchers believed that the IMM program would be an effective training. However,

the researchers did another study that claims children can learn pedestrian behaviors through a

simulated digital environment (Mccomas, Mackay, & Pivik, 2002). Based on their knowledge of

similar methods of delivery yielding positive results, and their descriptions of the versatility of

IMM programs in effective learning environments, the researchers show a positive bias for the

effectiveness of an IMM program teaching proper pedestrian behaviors. The researchers made no

attempt to state that a hypothesis was necessary for conducting this research. Proposing a

hypothesis in this type of study could lend to even more of a positive bias on the effectiveness of

an IMM program.

Research Procedures (Methods)

A flyer for the study was distributed to approximately 700 students in a suburban

neighborhood school district in the Pacific Northwest. The first 36 students who responded to the

flyer were accepted for participation in the study, ranging from kindergarten to fourth grade. The

researchers claim that a tight timeline for the study impacted their ability to have a randomly

selected group of participants. The fact that these participants were the first to respond to the

study flyer indicates that the children, and their families may be more alike than the people who

responded to the study flyer at a later time, weakening the argument that the group of participants
CRITICAL ANALYSIS 3

fairly resembled the target population of children. The study was conducted over two

consecutive Saturdays at a local church. Half of the participants were observed on the first

Saturday and the other half on the second Saturday. The participants were compensated ten

dollars after the study.

The procedure began with the participants taking the ten minute introductory portion of

the IMM program, and completing the mouse practice portion. The students were then asked to

complete both a real-life traffic simulation pretest, and a computer based traffic simulation

pretest, in random order. The pretest and posttest are identical, allowing the observers to measure

the gains of each student after they completed more of the IMM program. Both the real-life and

computer based pretests measured the participants ability to identify the number of potentially

dangerous vehicles based on their placement in the simulated traffic environment. Following the

pretest, the participants were then escorted to a classroom, where they completed a forty-minute

portion of the IMM program that taught the learner how to identify potentially dangerous

vehicles. Immediately following the IMM program, the participants took the posttests (in the

same order that they had taken the pretests), and completed a social validation measure of their

satisfaction with the program, concluding their involvement in the study.

The procedure accurately measured the participants ability before and after the

experimental variable. However, the research lacks the ability to answer a variety of questions

about the effectiveness of the IMM pedestrian safety program. The research does not indicate

that the participants will retain the information for a substantial amount of time after the program

is completed, since the posttest was administered immediately after the instruction. The research

does not include a control group that takes both the pretest and the posttest without the IMM

program portion. Posttest results could have increased due to a familiarity with the test. The
CRITICAL ANALYSIS 4

research was limited in its observation of learned pedestrian behaviors, only focusing on

identifying hazards. The researchers could have included a more comprehensive pretest and

posttest to gather this information. The research method was explained in enough detail to

replicate. Future researchers could look into improving upon this research by addressing the

concerns above, and by scaling the participant number to a higher level.

Research Results

The research showed conclusive evidence that the students improved the desired behavior

after completing the IMM program treatment. The research study utilized a within-study pretest-

posttest design. Treatment effects were evaluated by examining subjects gains on the pretest and

posttest scores. Results showed that subjects had gained an average of 40 percentage points on

the computer based posttest. The researchers used a paired T-test to indicate that this

improvement was highly significant (t (35) = 6.525, P<.001). The mean score at posttest

represented an increase of more than one standard deviation (SD=.33), yielding an effect size of

d=1.20 (Glang & Noell, 2005) The researchers cited Muller and Cohen (1988) who defines an

effect of d=.80 as a large effect. The real-life posttest also produced a large effect of d=1.12. The

research indicates that IMM is a promising approach to pedestrian safety education through

though the differences in their pretest and posttest variables. Since the deviation is significantly

larger than 1, the findings support that their measured variable was indeed changed by the

treatment of the IMM program. The posttest satisfaction questionnaire also showed that the

children thought the training administered was important and should be given to their friends.

Discussion of Results

The researchers conclusion reflects an accurate depiction of the impact IMM pedestrian

safety programs can have. It uses their evidence to provide an argument that the immersive
CRITICAL ANALYSIS 5

multimedia learning environment can yield positive behavior changes in simulated

environments. The researchers also acknowledge that their research was limited. They attribute

family-selected participants as their only source of information, and that the studys participants

did not fully master the proper skills needed to be a safe pedestrian. Their conclusion also refers

to IMM pedestrian safety trainings as an efficient means, rather than the most effective means of

pedestrian safety training. Based on their reflection in these areas, the researchers did provide

reasonable explanations, ones that did not assume more than the study evaluated. The

implications are also reasonable and meaningful. They support a continued research and practice

in the area of pedestrian safety through improving IMM programs.

Summary

Recommendations stated by the researchers for improving upon the research include studying the

cost-effectiveness and efficiency of IMM pedestrian safety programs compared to other

programs, as well as the ability to identify dangerous vehicles in comparison to the ability to

cross the street safely. Both of their recommendations can be added to a study that follows the

same procedure. This study should also include a greater focus on random sampling, and more

current versions of IMM programs like virtual reality (VR). Fortunately, pedestrian safety is not

an area that can be overstimulated by too many different types of training environments (in-

person, IMM, VR, video instruction). Any research done in this area will more than likely

contribute to better, and more available pedestrian safety training programs, a training program

that can save countless lives in the future.

References

Glang, A., Noell, J., Ary, D., & Swartz, L. (2005, 09). Using Interactive Multimedia to Teach

Pedestrian Safety: An Exploratory Study. American Journal of Health Behavior, 29(5),

CRITICAL ANALYSIS 6

435-442. doi:10.5993/ajhb.29.5.6

Mccomas, J., Mackay, M., & Pivik, J. (2002, 06). Effectiveness of Virtual Reality for Teaching

Pedestrian Safety. CyberPsychology & Behavior, 5(3), 185-190.

doi:10.1089/109493102760147150

Muller, K., & Cohen, J. (1989, 11). Statistical Power Analysis for the Behavioral Sciences.

Technometrics, 31(4), 499. doi:10.2307/1270020