Safety Science 115 (2019) 176–187

Contents lists available at ScienceDirect

Safety Science
journal homepage: www.elsevier.com/locate/safety

Serious games for learning prevention through design concepts: An T

experimental study
Zia Ud Din , G. Edward Gibson Jr.
⁎

Del E. Webb School of Construction, Arizona State University, USA

ARTICLE INFO ABSTRACT

Keywords: Many accidents occur during construction and maintenance of facilities. Logically, decisions made during the
Prevention through design design and planning of a project can influence workers’ safety. The approach of considering safety in design and
Engineering education planning to control workplace hazards is called Prevention through Design (PtD). Many fresh graduates from
Construction education architecture, construction, and civil engineering programs do not learn about PtD. Hence, the organizations
Serious games
which hire these graduates have to train them in the area of PtD. In most university curricula, there is no room
Innovative pedagogies
for a new course focused on PtD. Then non-traditional approaches such as computer games and simulations can
be implemented to teach PtD in universities. Computer games are routinely considered as the most important
and influential medium by today’s college students. The present work aims to measure the effectiveness of
computer games to train and educate students for safe design thinking. Therefore, three interventions—a
computer-based serious game, a paper-based game (the paper version of the serious game) and a traditional
lecture—were developed and implemented to measure their pedagogical value. The serious game was found to
be more effective as compared to the lecture. The paper-based game failed to motivate the students to learn. This
paper discusses the possible reasons for the success and failures of these pedagogical approaches.

1. Introduction several organizations have developed consensus standards such as ANSI

The U.S. construction industry employs 4.5 percent of the entire
non-farm workforce (BLS, 2017) and encounters about 19 percent of the
total fatalities (BLS, 2016). Over the past decades, concerns about
safety have received focus from many stakeholders (e.g., OSHA, NIOSH,
ASCE, ASSP), which has resulted in a decrease in the incident rate of
fatal and disabling injuries. With all the progress in engineering and
safety science, still, construction is prominent for its poor safety per-
formance. Often the efforts to improve safety focus on the construction
phase of a project while safety in the design and pre-construction
planning is typically overlooked. Researchers have recognized a strong
link between workplace fatalities and the absence of safety considera-
tion in the design process. For example, when a researcher examined
more than 200 fatality reports, he found that 42 percent of the fatalities
reviewed were linked to the absence of safety consideration in design
(Behm, 2005). Hence, it is vital to identify potential risks and develop
solutions to prevent work-related injuries, illnesses, and fatalities
through design and this approach is called Prevention through Design
(PtD) (Manuele, 2008).
To address worker safety in design and pre-construction planning,
Z10, Occupational Health and Safety Management Systems, and ANSI
Z590.3-2011, Guidelines for Addressing Occupational Hazards and
Risks in Design and Redesign Processes (ANSI/AIHA, 2005; ANSI/ASSE,
2011). These standards can be utilized by professors while teaching the
topics of structural design, electrical and mechanical design, site layout
planning and other related areas to prepare students for preventing
workplace hazards before they occur. Realizing the importance of
finding and designing out hazards in various products such as tools,
equipment, processes, and buildings, the National Institute of Occupa-
tional Safety and Health (NIOSH) partnered with several professional
organizations, including the American Society of Safety Professionals,
the Center for Construction Research and Training, and others to launch
the National Prevention through Design Initiative in 2007 (NIOSH,
2014). The purpose of the initiative was to facilitate the creation and
distribution of business tools, case studies, model projects, and best
practices focused on design solutions that decrease worker health and
safety issues and associated costs. Still, a major obstacle in developing
safe designs is that most newly graduated architects and design en-
gineers are not knowledgeable in safety and construction processes
fundamental to “design out” hazards (Gangolells et al., 2010). Often

⁎
Corresponding author at: Del E. Webb School of Construction, School of Sustainable Engineering and the Built Environment, Arizona State University, 660 S.
College Avenue, Tempe, Arizona 85281, USA.
E-mail address: zziauddi@asu.edu (Z.U. Din).

https://doi.org/10.1016/j.ssci.2019.02.005
Received 26 November 2018; Received in revised form 18 January 2019; Accepted 5 February 2019
Available online 14 February 2019
0925-7535/ © 2019 Elsevier Ltd. All rights reserved.
Z.U. Din and G.E. Gibson
university architecture and engineering programs do not teach students
about PtD at all or introduce briefly in their courses (López-Arquillos
et al., 2015). Thus professionals, typically, learn about PtD once they
enter professional practice. Ideally, the architects and engineers who
are involved in designing portions of the permanent and temporary
structures needed for a successful project should have the knowledge of
construction safety to develop safe designs. In addition, personnel in the
supervisory position such as project engineers, superintendents, and
project managers should understand the value of safety inclusion in
decision making during the project design and pre-construction project
planning process.
1.1. Statement of the problem
The challenges of teaching PtD concepts to future designers, en-
gineers, and construction managers have not been fully addressed due
to several reasons, including the lack of room in the existing curricula
for an additional required course (Behm et al., 2014; Mann, 2008),
shortage of the trained faculty to teach the topics of PtD, shortage of the
PtD teaching resources, and not fully recognizing the value of PtD
education of engineers by some professors (Zarges and Giles, 2008).
While the development of teaching resources such as textbooks and
educational modules is being encouraged under the NIOSH PtD in-
itiative (Popov et al., 2013); the teaching and learning potential of in-
novative teaching strategies such as digital game-based learning should
not be overlooked.
The use of digital games in different disciplines is on the rise. Mostly
serious games are the predominant application type of digital game-
based training systems (Ritterfeld et al., 2009). A serious game is a
game developed for the main purpose of teaching other than only en-
tertainment (Susi et al., 2007). Serious games use computer game set-
tings to train, educate, motivate and change the behavior of users in an
engaging and entertaining way (Ritterfeld et al., 2009). Today’s “mil-
lennial” college students appreciate playing video games more so than
past generations (Gale, 2011). Therefore, by incorporating character-
istics of games with the instructional material of PtD, the potential for
motivating students to learn new concepts in less time or outside the
traditional classroom may increase. Students may also improve the
retainage of knowledge through experiential learning in the game en-
vironment. These improvements need to be assessed. In the absence of
convincing empirical proof supporting the implementation of games for
learning and teaching of a particular subject, serious games can always
be rejected as an overstated, optimistic solution.
The main contribution of this paper is empirical evidence for the use
of serious games in PtD education. This empirical evidence focuses on
whether the use of serious games can address some shortcomings of
traditional and alternative approaches of teaching. To obtain empirical
data, an evaluation framework was developed to measure the effec-
tiveness of the SafeDesign game which is a prototype serious game
prepared for this study. The paper also explores the relationship, if any,
between variables (gender, relevant work experience, computer use
frequency, and learning-game playing experience) and student test
performance among each of the instructional techniques.
2. Literature review
2.1. Prevention through design
Rinehart et al. (2009) emphasized that business leaders expect de-
signers, construction professionals, and engineers to identify and con-
trol safety risks early in design development, rather than making
modifications once workers get injured. Hence, this process of hazard
identification and development of solutions to control those hazards
should be performed during the design and pre-construction activities
of facility development. This approach is comparable to “green chem-
istry or sustainable chemistry” which requires designs of chemical
177
Safety Science 115 (2019) 176–187
products and processes to consider pollution prevention and reduction
of usage of non-renewable resources (Anastas and Warner, 2000). For
many decision-makers, prevention of hazards through design is a lo-
gical requirement to save resources and get a competitive advantage in
the business environment, and they see it as crucial for lean production
and waste elimination. Many businesses worldwide have started prac-
ticing prevention through design as a part of their management ap-
proach for eliminating the costs linked to workplace fatalities, injuries,
and illnesses (Rinehart et al., 2009). Hence, designers and managers are
expected to control safety risks in the early design process, and this
approach is simply better than controlling hazards or protecting
workers from hazards after construction work starts (Manuele, 1997).
Although about 25% of construction hazard cannot be identified during
the early project life cycle, still the majority of the risks and hazard can
be identified and controlled in design (Hallowell and Hansen, 2016)
which is a very strong reason to teach and learn PtD to university
students.
Behm et al. (2014) conducted a four-year-long longitudinal study in
an engineering program to incorporate safety in design concepts in
various courses throughout the degree curricula. The results of the in-
tervention indicate an overall improvement in the students’ safe design
thinking. However, such efforts can only be successful when concerned
faculty and school management are interested and engaged focusing on
and teaching PtD concepts. Equally important is the availability of the
room in curricula for adding up PtD related topics to existing courses.
2.2. Front end planning process
Ideally, the consideration of safety should start with the front end
planning (FEP) process when owners allocate resources to maximize the
prospects for a successful project (Gibson et al., 1995; Gibson and
Bosfield, 2013). Fig. 1 shows the overall PtD implementation approach
in design and pre-construction planning.
Front end planning occurs before the design and construction of the
project and consists of feasibility, concept and detailed scope phases.
During the feasibility and concept phases, the project alternatives
should be evaluated from a health and safety perspective along with
other constraints. In the next phase, the health and safety concerns
identified during the previous phases should be assessed, and pre-
liminary design from a health and safety perspective should be ana-
lyzed. Then the design developed after the FEP process meets the ex-
pectations of safe design practice, paying particular attention to both
construction hazards and operational hazards during the facility life
cycle. Similarly, making safety-conscious decisions in pre-construction
planning such as material storage, visitor control, fall protection plan-
ning will help prevent many safety hazards. Fig. 1 shows the overall PtD
implementation approach in design and pre-construction planning.
2.3. Serious games
Some feel that education is in crisis because the traditional ap-
proach of education with one teacher and many pupils is failing to
motivate students to learn (Svinicki, 1999). Others feel that online
education, mostly limited to a set of online reading homework and a
final test, is not delivering the knowledge necessary for effective edu-
cation (Prensky, 2003a). Therefore, when education professionals see
the enthusiasm and long-hours people spend playing challenging
computer games, they imagine using this medium to teach educational
content. Hence, the use of serious games to motivate learners has much
potential for educators (Van Eck, 2006).
Theories addressing the benefits of games in the classroom have
emerged along with the rise in popularity of video games in con-
temporary culture. Prensky (2003b) has repeatedly argued that the
characteristics of younger generations indicate that students would
thrive in learning environments that include serious games. For in-
stance, research conducted by Squire (2004) suggested that students in
Z.U. Din and G.E. Gibson
Modify
requirements
Define Work Scope Feasibility Analysis
Identify Hazard &
Evaluate Potential Risks
Design Analysis
Establish PtD Controls
to Design-out Hazards
Construction
Monitor PtD Control Process Design
Adoption
Analysis
Preconstruction planning
Fig. 1. PtD consideration in design and pre-construction planning process.
Adapted from Kamardeen (2013)
a history class were more engaged in learning when using the game
Civilization III than with the traditional teaching methods. Other find-
ings from research focused on the use of serious games include in-
creasing social interaction (Oliver and Carr, 2009), benefiting from
experiential learning and constructivism (Dieleman and Huisingh,
2006; Saunders, 1997; Wagner, 2008), and increasing cognitive
learning achievement (Chuang and Chen, 2007).
Serious games for safety training are in use in different industries. In
the aviation sector, the effectiveness of the mobile virtual reality (VR)
and serious games was tested to deliver the safety briefing for aircraft
passengers, and the results were compared with the training using
traditional briefing card and found the interactive training approach
yielded a higher level of efficacy of the training (Chittaro, 2016;
Chittaro et al., 2018). In another study, Johnson (2008) suggested the
use of computer simulation techniques to model a broad range of po-
tential hazard scenarios in the design of stadiums and in training the
officials for security planning of large events such as the Olympic
Games. Dickinson et al., (2011) reported that the students at a college
teaching construction trades were interested in learning safety through
serious games.
Realizing the potential of serious games in training, the
Occupational Safety & Health Administration (OSHA) of the United
States has developed an interactive, online, game-based training tool
for teaching the main concepts of hazard control (OSHA n.d.). The in-
tended users are small business owners, workers, and others interested
in learning. The authors reviewed this tool and found that it is very
useful. However, in order to fully benefit from this tool, the players
need prior safety know-how and understanding of various construction
processes.
As of now, no study offers empirical evidence to suggest the use of
serious games to teach Prevention through Design principles to
178
Safety Science 115 (2019) 176–187
Concept
Modification Concept Design
Alternatives
Preliminary design
modification Detailed Scope
Schematic design
Detailed design
modifications
Detailed designs Detailed Design
Preconstruction
planning
Pre-Construction
Design
modifications
Finalized Detailed Design
and Construction Plan
construction engineering and construction management students at the
undergraduate level. Although various studies report the teaching po-
tential of serious games for different disciplines in higher education, it
is not suitable to generalize these findings. For example, effectiveness in
the field of military or medical sciences (Lateef, 2010) does not guar-
antee that similar results can be achieved in other fields such as con-
struction or agriculture. In each discipline, students benefit from in-
novative pedagogies differently. For examples, at one university the
researcher did not find the use of serious games helpful in improving
the learning of students in the area of agriculture when compared
against two interventions namely audio lecture and text reading (Gale,
2011). Another study contradicts a popular belief about the teaching
effectiveness of innovative pedagogies such as immersive virtual reality
(Mikropoulos and Natsis, 2011), a recent study found that the use of
PowerPoint had the similar effect on learning as VR (Leder et al., 2019).
Hence the effectiveness of serious games should be tested for each
discipline before investing resources in deploying serious games.
In this research, the goal is to conduct an empirical investigation to
determine the effectiveness of serious games by comparing the differ-
ences in learning performance of students taught the same material
using either a serious game, traditional lecture or a paper-based game.
This effort may advance the research in the use of serious games in
construction management and construction engineering education.
Additionally, demographic information will be collected to determine
whether differences exist in the test results of different types of students
when using serious games. The ultimate goal of this research is to
identify whether or not serious games should be considered effective
instructional tools for PtD education.
Z.U. Din and G.E. Gibson Safety Science 115 (2019) 176–187

Relevant work them to see drawings from a safety perspective as well. Other than
Experience teaching strategies, the process of measuring the effectiveness of ped-
agogies using the pre- and post-tests method was held as constant as
In-class Lecture possible among all the treatments.
Posttest
(Pedgogy)
Gender 3.2. Dependent variable

In this research, the dependent variable is the measure of learning

improvement. The learning improvement is also called the gain score,
Learning and this was calculated by deducting pre-test scores from post-test
Effectiveness
scores. The post-test score and pre-test scores were collected by totaling
Paper-based game Gameplay
(Pedagogy) (Posttest-Pretest) Frequency the test score of content understanding questions. These tests were in-
tended to find the total improvement in learning the contents delivered
through the three interventions.

3.3. Independent variables: Pedagogies

Previous
Serious Game
Knowledge Computer useage
(Pedagogy)
(Pretest) The pedagogies and their learning significance were studied. Based
on learning theories and scholarly work of past researchers, this study
Fig. 2. Pedagogical interventions and other covariates that affect learning. describes the following as independent variables: in-class lecture,
paper-based game, and serious game. Details of each of these are pre-
sented below.
3. Methodology

The authors developed and implemented two versions of the 3.3.1. Prevention through design serious game—SafeDesign game
SafeDesign game—a computer-based serious game and a paper-based One of the treatments, as described earlier, was the computer-based
game (the paper version of the serious game). Both types of games had SafeDesign serious game. The first step in developing this learning tool
feedback and gameplay features. Learning, storytelling, and user ex- was to establish a goal for the effort and then to develop a set of edu-
perience were held constant for both versions of the game. Fig. 2 il- cational objectives to meet the goal. The goal was to design an engaging
lustrates a theoretical model for measuring the effectiveness of peda- serious game that educates students about hazard identification, and
gogies, variables and the relationship of different variables to learning moreover, control measures that could be addressed during pre-con-
(Din, 2017). struction. This goal was achieved through the selection of several sce-
narios for the game adapted from the Construction Industry Institute’s
tool for the design that enhances construction worker safety, which was
3.1. Sample
developed by Gambatese and colleagues in 1997 (Gambatese et al.,
1997). The following topics were used to create scenarios: Site location
The trends in the literature review suggested that millennial stu-
and access, material storage options, housekeeping, pedestrian safety,
dents would be one of the prime beneficiaries of serious games.
overcrowding, trenching and excavation safety, formwork erection and
Therefore, the population for this study is construction engineering and
removal decisions, use of personal protective equipment, laying un-
construction management students with a significant majority being
derground utilities, parapet adequacy for fall protection, fragile roofing
digital natives (Prensky, 2001) and millennial students (Elam et al.,
(skylights, corrugated fiberglass), material storage and overhead power
2007). Due to cost and time constraints, a sample of the population will
lines, and excavation and underground power lines. The gameplay was
be used in the study. The sample consists of all sophomore construction
demonstrated at the start of implementation with the students, and then
engineering and construction management students from Arizona State
they were allowed to “play” the game as Treatment 1.
University—a public university, during the 2017/2018 academic year.
Courses with 30 or more registered students were identified to parti-
3.3.1.1. Examples of the SafeDesign game scenarios. Fig. 3 presents a
cipate in this research. One of the courses chosen to take part in the
typical scenario in which material is stored under power lines. The
study was in the computer lab because the serious game required
player is given four options of possible hazards and can identify the
computer resources, while the other two classes were in regular class-
hazard based on his/her understanding of the scenario. Figs. 4–6
rooms because computers were not required for the lecture and paper-
represent three important gameplay options: feedback and selection
based game treatments.
The proposed interventions were given to students in different
courses in Fall 2017 and Spring 2018. This research was set up with a
quasi-experimental arrangement to isolate the various formats of
treatment activity. Proposed courses to implement the treatment ac-
tivities were CON/CNE 271- Construction Safety and two sections of
CON 244 - Working Drawing Analysis courses. The enrolled students in
the construction safety course were from the construction management
and construction engineering disciplines. Whereas, students in Working
Drawing Analysis were from the construction management discipline
only. The reason for the selection of the construction safety course was
to see the effectiveness of teaching PtD topics, along with standard
construction safety topics covered under OSHA 30-hour course curri-
culum requirement. The Working Drawing Analysis course was a nat-
ural choice for teaching safety in design to construction management
students because they learn for the first time how to read drawings
developed by designers and engineers. Safety knowledge may help Fig. 3. Typical scenario of the SafeDesign game.

179
Z.U. Din and G.E. Gibson
Fig. 4. Typical SafeDesign game scenario for providing feedback.
Fig. 5. Typical SafeDesign game scenario presenting hazard control options.
Fig. 6. Typical SafeDesign game scenario presenting hazard control feedback.
of controls that limit hazard exposure (Din, 2017).
For each scenario, the player selects a hazard from the options on
the first screen as shown in Fig. 3. When the selected option is correct,
then the player receives feedback on the next screen as shown in Fig. 4.
However, if the selected option is incorrect, the player receives feed-
back and is directed to try again. In case of a correct answer, the
180
Safety Science 115 (2019) 176–187
feedback screen directs the player to the next screen where the player
can select one of the control options shown in Fig. 5. If the selected
option is correct as shown in Fig. 6, the player can move to the next
scenario, or is prompted to try again. This way the player navigates
through four screens to complete one scenario. At the end of all 25
scenarios, the player is given an overall score for their efforts.
3.3.2. Paper-based game
To better understand the effectiveness of an alternative pedagogy, a
paper-based approximation of the serious game was developed for
Treatment 2. In this study, the paper-based game offers visualization
through pictures and cartoons. The paper-based game included game-
play and visualization of gameplay that enabled players to see what is
occurring in the game.
After delivering necessary instructions to students on how to use the
paper-based game, this treatment was implemented with one section of
Working Drawing Analysis. The workflow for the development and
implementation of this activity includes some of the same tasks that
were completed in the SafeDesign game, but without the use of a
computing device.
3.3.3. In-class lecture
The instructional method of the lecture is believed to have started
from the ancient period around 1000AD and 1500AD. Even today,
students learn in the classroom through instructional materi-
alsdisseminated through lectures. The effectiveness of lectures has al-
ways been questioned, but so far, this is the most dominant form of
instruction method in adult education in most parts of the world
(Atanga et al., 2015).
To understand the benefits of the traditional teaching method, an in-
class lecture was delivered in the construction safety class in Spring
2017 and again in Fall 2018 semesters. This helped to illustrate the
inherent behaviors of construction engineering and construction man-
agement students who might not be interested in a computer game
experience.
Students enrolled in the class were delivered a 50-minute lecture,
including discussions about the PtD concepts in construction. During
this lecture, an overview of safe design considerations was covered. A
PowerPoint lecture was prepared to deliver the content such as facts,
statistics, and graphics of the main four reasons for the construction
workers’ deaths, which are falling from a height, struck-by falling or
flying objects, caught-in or between two surfaces and electrocutions
(OSHA, 2017). Moreover, the topics presented were related to the top
four hazards in construction, the sources of these hazards and ways to
control them through design. In this lecture, the authors tried to address
the learning needs of all students through including facts, graphics, and
interactive discussions during and at the end of the lecture presentation.
3.4. Confounding variables
Classic experiments are often performed in a controlled environ-
ment, such as laboratories, to establish the effect of the independent
variable on the dependent variable only, and no other factors are al-
lowed to undermine the impact of independent variables on the de-
pendent variables. However, in measuring the pedagogical effective-
ness of teaching methods, it is often not possible to experiment in a
completely controlled environment. Therefore, in this study, the fol-
lowing variables probably influence the gain score, which is the mea-
sure of learning effectiveness and the dependent variable of the study.
3.4.1. Gender
Historically, the literature on digital games shows that there is an
inconsistency between male and female students in their interest in
using computer games. Hartmann and Klimmt (2006) found in a survey
that females are less interested in video games than males, and when
they like to play, they frequently choose different games. Nevertheless,
Z.U. Din and G.E. Gibson
the latest studies on gender and games present an entirely different
picture. For example, a study by Klisch et al. (2012) and Chang et al.
(2014) found that female learners scored higher than males in term of
learning performance score and engagement. In addition, the En-
tertainment Software Association (ESA) reports that the number of fe-
male video game players has escalated in the past few years. Forty- two
percent of all game players are female.
Interestingly, 41 percent of total game players are adult females,
whereas 17 percent of video gaming players are boys of age 17 or
younger (ESA, 2016). These figures indicate that an increasing number
of adult females are interested in video game playing in general.
Therefore, females might also be interested in learning through serious
games.
This study examined whether both male and female students benefit
to the same extent from all three pedagogies. Therefore, gender is
treated as a covariate and is marked as 1 = Female; 2 = Male.
3.4.2. Learning-game playing experience
Learning-game playing experience may increase the learning per-
formance of students in the case of serious game intervention.
Therefore, the survey, which is part of the pre-test, asked students if
they ever played video games for learning. The learning-game playing
experience question gave respondents two options, yes or no and a
follow-up question asks participants, their perception of the usefulness
or learning value of the game if they played.
The authors wanted to understand the effect of the confounding
variables on gain scores and interaction effects confounding variable on
the primary independent variables. The research design comprises a
dependent variable (gain score), three independent variables (pedago-
gies), and five confounding variables (gender, learning-game playing
experience, computer use frequency, and relevant work experience).
The participant data were gathered to understand how the independent
variables and confounding variables affect the learning performance.
3.5. Design of study
The dependent variable was the gain score. The authors collected
data concerning the students' knowledge about hazard recognition and
hazard mitigation before and after the implementation of three inter-
ventions. The change in the knowledge, the gain score, was measured.
In order to find a statistically significant effect on learning, two popular
tests were considered. First, an analysis of covariance (ANCOVA) uses
post-test scores as the dependent variable, and pre-test as a covariate.
Second, an analysis of variance (ANOVA) that uses gain score as a de-
pendent variable and any other variable such as interventions, gender,
learning-game playing experience, and computer use frequency as an
independent variable. In this study, the authors applied ANOVA to the
gain score instead of ANCOVA. A large number of research publications
(Cribbie and Jamieson, 2004; Dimitrov et al., 2003) suggest the use of
ANOVA on gain score over ANCOVA. ANOVA finds if there are any
statistically significant differences between gain score means of the
interventions.
This study has only one dependent variable and therefore is called a
univariate study. Hence, a hypothesis testing procedure called ANOVA
is suggested. ANOVA is a statistical technique that compares the sig-
nificance of mean differences of a dependent variable between two or
more samples (Agresti, 2017; Amr, 2012). To determine the main ef-
fects of pedagogical interventions, univariate analysis was conducted.
The mean difference of gain score among the pedagogical interventions
is called the main effect of the pedagogies (Price et al., 2017). Similarly,
when the learning effect of pedagogies may also be due to confounding
variables, this is called an interaction (Price et al., 2017). In this study,
factorial design determines the pedagogical intervention’s effect on the
learning performance, and it assesses the collective effect of gender,
learning-game playing experience, relevant work experience, and
computer use frequency on the gain score.
181
Safety Science 115 (2019) 176–187
3.6. Procedure
To measure the effectiveness of pedagogies, a survey/questionnaire
methodology was used to compare the differences in student knowledge
improvement following an instructional session. Previous literature was
used to establish an inquiry tool that was used to investigate the ped-
agogical nature of the serious game and other pedagogical strategies
being implemented. Specific comparisons were made between the in-
structional techniques of the lecture, paper-based game reading, and
serious game.
The intervention evaluation studies used a pre-test and post-test
approach. Since research interest in the field of teaching Prevention
through Design topics is rather recent, no relevant surveys were found
in the literature which could be amended for this study. However, a few
studies in the in the discipline of education (Amr, 2012; Gale, 2011)
were consulted to learn from their findings. Therefore, the authors
prepared survey instruments (pre- and post-test), and these tests were
sent to the Office of Research Integrity and Assurance at Arizona State
University for review and approval. The survey instruments were fi-
nalized based on the comments obtained from the Office of Research
Integrity and Assurance at Arizona State University. The analysis
techniques for the data collected were descriptive statistical analysis
and inferential statistical analysis.
In the subsequent section, quantitative analyses of the data are
described. The effect of the three independent variables—serious game,
paper-based game, and in-class lecture—was determined using a one-
way ANOVA. To find the learning effect of the confounding varia-
bles—learning-game playing experience, computer use frequency,
gender, and relevant work experience —factorial ANOVA’s were con-
ducted.
4. Limitations of study
This study was implemented with a relatively small sample size of
five classes (180 valid participants); therefore, the results of this re-
search cannot be generalized. The subjects of this study are construc-
tion management and engineering students who are mainly involved in
the pre-construction and during-construction planning; therefore their
influence on the prevention of hazards through design is mostly limited
to these two phases. The authors included only those scenarios in the
tests which were pertaining to the construction engineering and man-
agement disciplines. Further work in this area needs to be done focused
on students who are learning early or conceptual design and detailed
design hazard recognition; this was not part of this study. This exercise
of measuring the pedagogical value of interventions to teach Prevention
through Design topics is a pilot study. Since the time distance between
the pretests and post-tests was approximately two weeks, it is hard to
infer whether the serious game would help long-term knowledge re-
tention.
Two other issues may have influenced the results. This study focuses
only on testing the proof of concept. Therefore, the researchers them-
selves created all aspects of the game, including content development,
coding, graphic design, audio design, etc. The quality is not on par with
commercially available games. There are numerous aspects of a serious
game such as better audio and visual effects, and 3D navigation cap-
abilities, which are required for a game to be commercially successful.
Therefore, a professional game designer and a team of developers could
make a high-quality game that most are used to seeing in the com-
mercial world.
The graphic design quality, storytelling, game-playing can have a
significant impact on learning. In this study, the researcher designed the
paper-based game-based on the serious game scenarios, and probably
this had some weakness, which influenced the results of the study. In
order to deliver an effective lecture, teaching experience plays a vital
role. The duration of the lectures was only 50 min, and the researcher
had not interacted with students before this activity, which might have
Z.U. Din and G.E. Gibson
Table 1
Paired sample T-Tests of the in-class lecture’s pre- and post-test scores (N = 59).
Paired Differences
Mean Std. deviation Std. error mean
Pair 1 Post-test – Pre-test −4.781 23.083 3.005
a negative effect on learning and teaching. The researcher did, how-
ever, have five years of experience in the classroom instruction in the
past.
5. Analysis of data
The authors analyzed the data obtained from the study using IBM
SPSS Statistics for Windows, Version 23.0. The total number of parti-
cipants in the interventions across the two semesters was 246, but 61 of
these students did not fill either pre- or post-test, or both, as the activity
had to be considered voluntary per Institutional Review Board man-
dates. Five more responses were not considered for further analysis
because these students participated in more than one intervention;
therefore, this left 180 valid responses, which formed the sample and
the basis of conclusions in this study.
The authors conducted a paired-samples t-test to find if the peda-
gogical interventions influenced students’ learning through the differ-
ence in the pre-test vs. post-test scores. The findings of the paired
sample t-tests for each intervention are presented in Tables 1–3.
The results of the paired-samples t-test results indicate that there
was a statistically significant difference in the mean pre-test and post-
test scores of the SafeDesign game intervention. The average difference
between post-test and pre-test scores are M = 18.165, SD = 21.484 for
the SafeDesign game. These results propose that the SafeDesign game
positively affects learning performance. When students played the
serious game, their gain score improved. Furthermore, a positive cor-
relation exists (r = 0.435) that shows that the participants who per-
formed better on the pre-test also performed better on the post-test.
These findings are in congruence with earlier studies in this field
(Andreu-Andre and Garci, 2011; de-Marcos et al., 2016; Dieleman and
Huisingh, 2006; Kebritchi et al., 2010; Kwon and Lee, 2016;
Papastergiou, 2009) which have examined the influence of educational
computer games on learning performance.
The results also show that there was no statistically significant
learning when the paper-based game and lecture were implemented.
The scenarios were presented through illustrations, very similar to
those used in the serious game. One explanation for the poor perfor-
mance is that perhaps these illustrations and the storyline in the paper-
based game did not help student learning; rather, diverted students’
attention. Researchers like Weidenmann pointed out that there is a
good reason to doubt the benefits of pictures in the educational text
(Weidenmann, 1989). He said learners would often consider illustra-
tions superficially and inadequately, failing in achieving any con-
tribution to learning. Consistent with the study by Caffaro et al., 2018
in which they found that out 12 safety illustrations affixed on agri-
culture machinery to guide users seven were comprehended poorly.
Table 2
Paired sample T-Tests of the paper-based game’s pre- and post-test scores (N = 34).
Paired Differences
Mean Std. Deviation Std. Error Mean
Pair 1 Post-test – Pre-test 1.79 27.27 4.68
182
Safety Science 115 (2019) 176–187
t df Sig. (2-tailed)
95% confidence interval of the difference
Lower Upper
−10.796 1.234 −1.591 58 0.117
Due to the inability of paper-based intervention to motivate students,
the authors decided to drop the paper-based intervention from the next
implementation cycle (e.g., Spring 2018).
When the lecture was delivered in Fall 2017, gain score was 7.57%,
but the second time in Spring 2018 the gain score of lecture dropped to
−14.48%. The content of the lecture and the lecturer were the same in
2017 and 2018, the only observed difference was the primary class
instructor was not present in the class during the second time im-
plementation, and many students left the post-test survey incomplete;
this was probably due to the absence of extrinsic motivation. In a lec-
ture-based pedagogy, students generally participate due to external
values and demands such as “external pressures from parents and tea-
chers, competition for grades and the consequences of failure” (David
et al., 1998). In the lecture-based intervention, no extrinsic motiva-
tional factors were present as this was a voluntary activity that did not
affect grades.
5.1. Analysis of variance
To measure the effectiveness of the treatments, the researchers used
the dependent variable, which is the post-test score minus the pre-test
score, known as the gain score.
Eighty-seven participants in the serious game intervention group
had an average gain score of 18.19 (SD = 21.48); 59 participants in the
lecture intervention group had an average gain score of −4.76
(SD = 23.08) and 34 participants in the paper-based game group
earned an average gain score of 1.79 (SD = 27.27) as shown in Tables
1–3.
The combined sample showed significant effects of the interventions
on learning at the p < 0.05 level for three interventions, F (2,
177) = 18.435, p = 0.000 (Table 4).
Post hoc comparisons using the Tukey HSD test (Table 5) showed
that the mean gain score for the serious game intervention (M = 18.19,
SD = 21.48) was significantly different from the paper-based game
(M = 1.79, SD = 27.27) and the in-class lecture (M = -4.76,
SD = 23.08). Taken together, these results suggest that the serious
game has a significant effect on learning for this sample. Particularly,
for this sample, these results hint that when a serious game is used
learning improves.
However, presenting scenarios on paper perhaps confused the stu-
dents, causing only a slight rise in learning performance. In this study,
when the lecture was delivered, the overall score did not improve. The
current results suggest that the approach of using lecture only for
teaching PtD does not improve learning.
A one-way ANOVA is used to check the equality of three or more
means of samples (Chernick and Friis, 2003). There are three
t df Sig. (2-tailed)
95% confidence interval of the difference
Lower Upper
−7.72 11.31 0.384 33 0.704
Z.U. Din and G.E. Gibson Safety Science 115 (2019) 176–187

Table 3
Paired sample T-Tests of the serious game’s pre- and post-test scores (N = 87).
Paired differences t df Sig. (2-tailed)

Mean Std. Deviation Std. Error Mean 95% Confidence interval of the difference

Lower Upper

Pair 1 Post-test – Pre-test 18.195 21.484 2.303 13.617 22.774 7.900 86 0.000

Table 4 assumptions for the test, and these are: (1) the distribution of gain
Analysis of variance. scores follows a normal distribution, (2) the samples must be in-
Sum of Squares df Mean Square F Sig.
dependent, and (3) the variances of gain scores must be equal. The first
condition was tested through plotting a histogram and found that the
Between groups 19868.924 2 9934.462 18.435 0.000 data were normally distributed. The second assumption was met be-
Within groups 95383.861 177 538.892 cause the participants in each intervention group are independent. To
Total 115252.786 179
satisfy the third assumption, the authors conducted Levene’s test of
homogeneity of variances to confirm the equal variance assumption.
Table 5 The result of Levene’s test shows that the significance is 0.736, which is
Multiple group comparison-post hoc test. higher than the 0.05 significance level, and it can be interpreted as the
variances are almost equal. If variances are not equal, other tests would
Dependent variable: Gain score
have been used.
(I) (J) Mean Std. Error Sig. 95% confidence Fig. 7 provides a chart of the ranges of mean gain scores of student
Intervention Intervention Difference interval learning in the three pedagogies.
(I-J) The box plot shows that the students in paper-based game inter-
Lower Upper
vention performed very different as compared to the other two inter-
bound bound
ventions that resulted in extreme values and four mild outliers.
Lecture Paper-based −6.559 4.998 0.390 −18.37 5.26 The results of ANOVA tests for the interventions were compared
game with α = 0.05, the P-value of the serious game is notably less than
Serious −22.925* 3.915 0.000 −32.18 −13.67
alpha (α), it is safe to infer that serious game positively affected
game
Paper-based Lecture 6.559 4.998 0.390 −5.26 18.37
learning in this sample. When the P-value of the paper-based game is
game Serious −16.367* 4.695 0.002 −27.46 −5.27 compared with alpha (α), it can be deduced that the paper-based game
game intervention did not enhance the gain score of the participants.
Serious Lecture 22.925* 3.915 0.000 13.67 32.18
game Paper-based 16.367* 4.695 0.002 5.27 27.46
game 5.2. Factorial design analysis

* The mean difference is significant at the 0.05 level. In order to find the impact of the confounding variables—gender,
learning-game playing experience, computer use frequency, and

Fig. 7. Gain score range of Pedagogical interventions.

183
Z.U. Din and G.E. Gibson Safety Science 115 (2019) 176–187

Table 6
Descriptive statistics gain score for male and female participants.
Dependent variable: Gain score gain score

Intervention Gender Mean N Std. Deviation

Lecture Female −3.33 8 30.139

Male −4.99 51 22.137
Total −4.76 59 23.074

Paper-based game Female 5.75 8 27.296

Male 0.58 26 27.690
Total 1.79 34 27.274

Serious game Female 28.07 14 17.499

Male 16.26 73 21.838
Total 18.16 87 21.555

Total Female 13.75 30 27.083

Male 6.32 150 24.930
Total 7.56 180 25.375

relevant work experience—on learning performance of any of the

pedagogical interventions, individual factorial design analysis was
carried out and the outcome of analysis helped to examine the main
effects and the interaction effects of these confounding variables.
Fig. 8. Chart of mean gain scores of three interventions based on gender.

5.2.1. Gender
A two-way ANOVA, which is also called Factorial Analysis, was
performed to find the effect of interventions and gender on the gain
score and the combined effect of gender and intervention on the gain
score. Thirty female participants and 150 male participants completed
all the intervention sessions. The mean gain scores and standard de-
viation of male and female participants in all three pedagogies are
shown in Table 6.
Fourteen female participants in the serious game intervention group
had average gain scores of 28.07 (SD = 17.5); 73 male participants in
the same intervention group had an average gain score of 16.26
(SD = 21.83). These results show that for this sample female partici-
pants’ performance was very high. In all three intervention groups, 30
female participants earned an average gain score of 13.75
(SD = 27.08), and the 150 male participants scored 6.32 (SD = 24.93).
The effects of the gender on the gain score, therefore, were not statis-
tically significant.
Table 7 presents the interaction effects of gender and the pedago-
gies. The result indicates that the interaction effect between gender and
pedagogical interventions on the gain score was not statistically sig-
nificant, F (2174) = 0.425, p = 0.637. The main effect of gender, F (1,
174) = 1.645, p = 0.201, was not statistically significant.
Fig. 8 is a chart showing the mean gain scores obtained by male and
female participants for three interventions.
From the chart, it is evident that the female participants performed
better than the male participants in all three intervention groups.
Because of the small number of female participants (30 vs. 150), the
results are not statistically significant. With a P-value of 0.201 (shown
in Table 7) which is not less than the designated alpha (α) value,
however, for this sample gender has no impact on gain scores.
The average age of the participants was 21.8 years.
The majority of students (93%) participated in the study are digital
natives which mean their age is 25 years or below.
5.2.2. Learning-game experience
A two-way ANOVA was performed to measure the influence of
previous learning-game experience and interventions on the gain score.
Table 8 contains descriptive information.
Thirty-eight participants who had played learning games before
earned average gain scores of 15.87 (SD = 21.72) in the serious game
intervention group; 12 participants who had played learning games
before in the paper-based game intervention group had an average gain
score of negative 4.67 (SD = 28.166). In all three intervention the 75
participants who had played learning games before earned an average
gain score of 4.45 (SD = 24.99) and the 105 participants who never
played games for learning scored 9.77 (SD = 25.52). The effect of in-
teraction between previous learning-game experience and intervention
on the gain score, therefore, were not significant.
Fig. 9 is a chart of the mean gain scores obtained by those who have
the learning-game playing experience and who have no learning-game
playing experience for the three intervention groups. The plot indicates
that the participants with no learning-game playing experience per-
formed well in all intervention groups. Whereas the mean gain score of
experienced learning-game players was low, particularly low in the case
of the paper-based game intervention group and in the lecture

Table 7
Tests between-subjects effects: gender and intervention’s effect on gain score.
Dependent variable: Gain score
Source Type III Sum of Squares df Mean Square F Sig. Partial Eta Squared

Corrected model 21690.576a 5 4338.115 8.068 0.000 0.188

Intercept 4559.974 1 4559.974 8.480 0.004 0.046
Gender 884.548 1 884.548 1.645 0.201 0.009
Intervention 13641.052 2 6820.526 12.684 0.000 0.127
Gender * Intervention 486.260 2 243.130 0.452 0.637 0.005
Error 93562.210 174 537.714
Total 125526.830 180
Corrected total 115252.786 179

a
R Squared = 0.188 (Adjusted R Squared = 0.165).

184
Z.U. Din and G.E. Gibson Safety Science 115 (2019) 176–187

Table 8 Table 9
Descriptive statistics of learning-game experience. Descriptive statistics of the gain score and relevant field experience.
Dependent variable: Gain score Dependent variable: Gain score
Intervention Previous learning-game Mean N Std. deviation
experience Intervention Relevant field experience Mean Std. Deviation N

Lecture No −2.00 34 24.844 Lecture No experience 0.14 23.862 32

Yes −8.52 25 20.310 Construction safety −5.19 17.242 14
Total −4.76 59 23.074 Engineering design −50.00 1
Construction −13.58 22.589 12
Paper-based game No 5.32 22 26.768
Total −4.76 23.074 59
Yes −4.67 12 28.166
Total 1.79 34 27.274 Paper-based game No experience 5.90 29.227 21
Construction safety −6.80 24.397 5
Serious game No 19.94 49 21.478
Construction −3.63 24.272 8
Yes 15.87 38 21.722
Total 1.79 27.274 34
Total 18.16 87 21.555
Serious game No experience 21.03 20.576 31
Total No 9.77 105 25.527
Construction safety 20.40 14.639 5
Yes 4.45 75 24.999
Engineering design 11.00 1
Total 7.56 180 25.375
Construction 15.13 26.845 16
Total 19.00 21.843 53

Total No experience 9.29 25.655 84

Construction safety −0.19 20.632 24
Engineering design −19.50 43.134 2
Construction 1.39 27.494 36
Total 5.39 25.808 146

Table 10
Descriptive statistics of computer use frequency.
Dependent variable: Gain score

Intervention Computer use frequency Mean Std. Deviation N

Lecture More than 8 hrs. a wk. −4.98 23.467 25

6–8 hrs. a wk. −13.76 18.526 16
4–6 hrs. a wk. −0.22 24.066 11
2–4 hrs. a wk. 4.44 28.932 5
0–2 hrs. a wk. 21.95 0.354 2
Total −4.76 23.074 59

Paper-based game More than 8 hrs. a wk. 1.96 29.487 25

6–8 hrs. a wk. 7.50 3.536 2
4–6 hrs. a wk. 4.60 26.140 5
2–4 hrs. a wk. −13.00 19.799 2
Total 1.79 27.274 34

Serious game More than 8 hrs. a wk. 17.72 22.377 39

Fig. 9. Chart of mean gain scores of three interventions based on a learning-
6–8 hrs. a wk. 19.43 23.010 23
game experience.
4–6 hrs. a wk. 17.35 19.730 20
2–4 hrs. a wk. 25.00 27.404 3
0–2 hrs. a wk. 10.00 7.071 2
intervention it was worse. The hypothesis was that the learning-game
Total 18.16 21.555 87
playing experience would have a positive impact gain score. With a P-
value of 0.439, not less than the designated alpha (α) value, it is safe to Total More than 8 hrs. a wk. 6.91 26.504 89
6–8 hrs. a wk. 5.90 26.083 41
reject the hypothesis because the previous learning-game playing ex- 4–6 hrs. a wk. 10.21 22.863 36
perience has no impact on gain scores for this sample. This is perhaps 2–4 hrs. a wk. 7.12 27.982 10
due to the sophistication of the serious game as developed. 0–2 hrs. a wk. 15.98 8.019 4
Total 7.56 25.375 180

5.2.3. Relevant field experience

A two-way ANOVA was carried out to explore the influence of re-
levant field experience and interventions on the gain score. Table 9
shows the mean gain scores of each participant in the experience ca-
tegory for different pedagogies.
Eighty-four participants with no experience in all three intervention
groups had average gain scores of 9.29 (SD = 25.655); 31 participants
with no work experience in the serious game intervention group had an
average gain score of 21.03 (SD = 20.58).
It is evident that the students with no experience got a higher gain
score in all three groups. This shows that, for this sample, relevant work
experience does not significantly improve learning for any intervention.
5.2.4. Computer use frequency
The results of a two-way ANOVA show that there is not a
statistically significant interaction between computer use frequency and
intervention. Table 10 indicates that the students who use the computer
per week performed very well in the serious game group and the mean
gain score for those who use computer between 6 and 8 h per week was
19.43 (SD = 23.01) and the students in the lecture intervention group
who use the computer for 6–8 h per week their performance was
−13.76 (SD = 18.53).
In the table below, “N” indicates the number of students who use
computer weekly for a particular amount of time. For example, 89
students out of the total 180 participants use the computer more than
8 h every week.
The results are not statistically significant, but the authors observed
that the effect of the computer use frequency on learning performance
in all three interventions was positive. The students who use more than

185
Z.U. Din and G.E. Gibson
8 h per week and 6–8 h per week performed better than others.
Another interesting finding is that 22 students who reported com-
puter use for the least amount of time (0–2 h per week) scored higher in
lecture intervention. This indicates that for non-computer users, the
lecture was an effective method of learning for this sample.
5.3. Student feedback
To understand the students’ perspective, the authors engaged a
small group of ten students in discussing their experience of playing the
paper-based game and serious game and unfortunately, no students
from the in-class lecture group participated in the discussion. The
comments of the participants can be divided into four groups. First, the
three students who had never played computer games did not believe in
their effectiveness in teaching. They said they consider these games “for
kids” and they are not interested in playing games. This indicates that
not all students love to play games.
Students were not familiar with some of the terminology about
safety and construction. One common concern was the use of jargon
such as safety hazards, controls, and formwork, etc. The second group
of comments was related to the terminology used in the game.
Therefore, those students who had difficulty in understanding a few
basic terminologies did not enjoy playing the computer game, while
others decided to search on Google to understand the meanings.
The third group of comments was regarding the game development
process, and five students enjoyed playing the game. The fourth group
of comments was about the paper-based game. Three students liked the
colorful pictures, but they were not interested in reading “25 pages” to
solve the “puzzle.”
5.4. Faculty feedback
Two instructors graciously helped the authors in data collection
from their students. One instructor mentioned that the consent form
was given to students before the data collection that clearly mentioned
that the activity was completely a voluntary exercise (as required by the
Institutional Review Board) and students were free to stop their parti-
cipation, anytime; that is why some students did not complete the tests
or even left in the middle of the test. Overall, both instructors who were
present during the intervention in the Fall of 2017 mentioned that the
majority of the students liked the activity and asked follow-up questions
the next week in succeeding classes, especially regarding game devel-
opment.
6. Conclusions
Well-designed serious games have the potential to turn learning into
a fun challenge through the right blend of instructive and entertaining
elements. This relatively small sample showed that computer-based
serious games (in this case SafeDesign) provided a platform that en-
hanced learning as opposed to in-class lectures and paper-based games.
This finding corroborates with several studies which have investigated
the effect of serious games on learning (Coller and Scott, 2009;
Kebritchi et al., 2010; Lateef, 2010; Ortiz et al., 2015). For example,
many studies report that students enjoy learning more when they use
computer-based games (Andreu-Andre and Garci, 2011; Dickinson
et al., 2011; Dieleman and Huisingh, 2006; Kwon and Lee, 2016). The
findings of this study are insightful and show the promise of this
technology in improving student performance, because serious games
reported to have positive impact on young users’ motivation and their
performance related to content understanding (Connolly et al., 2012).
Therefore, the use of serious games in other fields such as in the avia-
tion sector for safety training of aircraft passengers was found useful
(Chittaro, 2016; Chittaro et al., 2018).
Obviously, lecture-based teaching and learning can help students
when the lecture is based on adult education principles such as meeting
186
Safety Science 115 (2019) 176–187
a student’s need, is interactive, and considers student’s background
knowledge and experience. Perhaps combining lecture with computer-
based serious games would enhance both treatments when focusing on
a specific topic. This is a possible future study topic.
Among the various factors or confounding variables studied in this
sample, the authors did not find any of the essential elements of
learning, which positively augment learning, for example, gender, re-
levant fieldwork experience, and learning-game playing experience did
not seem to help toward learning for this relatively small sample. There
seemed to be some hints of impact in the sample that with perhaps
more data collection to prove some of these variables are important.
Despite a large number of studies in literature that support the use
of illustrations and graphics in teaching improve learning (Stokes,
2002), the authors did not find statistical support for the same in this
study. Most likely the illustrations used in the paper-based game dis-
tracted students from the educational features of the game. To improve
the effectiveness of a paper-based game, cognitive load theory should
be considered (Kirschner, 2002; Paas and Sweller, 2014) in the design
using visuals in teaching. According to this theory, a learner can only
process a certain amount of information due to one’s short-term
memory capacity. The lesson here is that a paper-based serious game
should probably not mirror a computer-based serious game as in this
study.
More work needs to be done about whether the SafeDesign game
will have a positive effect to retain that knowledge in the long-term.
Researchers found the degree of realism is important to keep the par-
ticipants interested (Mendonça et al., 2006). Therefore, future ex-
ploration may focus on what features of a game are necessary for a
positive impact on learning. The effect of features such as better audio,
visual, and navigation functions should also be studied. One should not
consider serious games as a “magic wand” to improve learning per-
formance, but serious games have the potential to be a great instrument
for effective learning.
Acknowledgements
This research was supported by the Arizona State University Global
Center for Safety Initiative (ASU GCS).
References
Agresti, A., 2017. Statistical methods for the social sciences. Pearson.
Amr, K., 2012. Learning through games: essential features of an educational game.
Syracuse University.
Anastas, P.T., Warner, J.C., 2000. Green chemistry: theory and practice. Oxford
University Press.
Andreu-Andre, N., Garci, M., 2011. Perceptions of gaming as experiential learning by
engineering students. Int. J. Eng. Educ. 27, 795–804.
ANSI/AIHA, 2005. American national standard: occupational health and safety man-
agement systems (ANSI/Z10-2005). American Industrial Hygiene Association,
Fairfax, VA.
ANSI/ASSE, 2011. Prevention through Design guidelines for addressing occupational
hazards and risks in design and redesign processes (ANSI/ASSE Z590.3-2011).
American Society of Safety Engineers, Des Plaines.IL, USA.
Atanga, M., Abgor, N., Ayangwo, J., 2015. Criticisms of the ‘lecture’ method in the
teaching of nursing students: the case of nurse tutors in Bamenda. Cameroon. Br. J.
Med. Med. Res. 6, 397–403. https://doi.org/10.9734/BJMMR/2015/13223.
Behm, M., 2005. Linking construction fatalities to the design for construction safety
concept. Saf. Sci. 43, 589–611. https://doi.org/10.1016/j.ssci.2005.04.002.
Behm, M., Culvenor, J., Dixon, G., 2014. Development of safe design thinking among
engineering students. Saf. Sci. 63, 1–7. https://doi.org/10.1016/j.ssci.2013.10.018.
BLS, 2017. Employment by industry, 1910 and 2015. [WWW Document]. The Economics
Daily: U.S. Bureau of Labor Statistics (accessed 7.10.17). https://www.bls.gov/
opub/ted/2016/employment-by-industry-1910-and-2015.htm.
BLS (U.S. Bureau of Labor Statistics), 2016. Current Population Survey, Census of Fatal
Occupational Injuries [WWW Document]. URL https://www.bls.gov/iif/oshwc/cfoi/
cfch0014.pdf.
Caffaro, F., Schmidt, S., Murphy, D.J., Cavallo, E., 2018. Comprehension rates of safety
pictorials affixed to agricultural machinery among Pennsylvania rural population.
Saf. Sci. 103, 162–171. https://doi.org/10.1016/j.ssci.2017.11.021.
Chang, M., Evans, M., Kim, S., Deater-Deckard, K., Norton, A., 2014. Educational video
games and students’ game engagement. In: 2014 International Conference on
Information Science & Applications (ICISA). IEEE, pp. 1–3. https://doi.org/10.1109/
Z.U. Din and G.E. Gibson
ICISA.2014.6847390.
Chernick, M.R., Friis, R.H., 2003. One-way analysis of variance, in: introductory biosta-
tistics for the health sciences. John Wiley & Sons, Inc., pp. 295–307. https://doi.org/
10.1002/0471458716.ch13.
Chittaro, L., 2016. Designing serious games for safety education: “Learn to brace” versus
traditional pictorials for aircraft passengers. IEEE Trans. Vis. Comput. Graph. https://
doi.org/10.1109/TVCG.2015.2443787.
Chittaro, L., Corbett, C.L., McLean, G.A., Zangrando, N., 2018. Safety knowledge transfer
through mobile virtual reality: a study of aviation life preserver donning. Saf. Sci.
102, 159–168. https://doi.org/10.1016/j.ssci.2017.10.012.
Chuang, T.-Y., Chen, W.-F., 2007. Effect of computer-based video games on children: An
experimental study. In: 2007 First IEEE International Workshop on Digital Game and
Intelligent Toy Enhanced Learning (DIGITEL’07). IEEE, pp. 114–118. https://doi.org/
10.1109/DIGITEL.2007.24.
Coller, B.D., Scott, M.J., 2009. Effectiveness of using a video game to teach a course in
mechanical engineering. Comput. Educ. https://doi.org/10.1016/j.compedu.2009.
05.012.
Connolly, T.M., Boyle, E.A., MacArthur, E., Hainey, T., Boyle, J.M., 2012. A systematic
literature review of empirical evidence on computer games and serious games.
Comput. Educ. https://doi.org/10.1016/j.compedu.2012.03.004.
Cribbie, R.R.A., Jamieson, J., 2004. Decreases in posttest variance and the measurement
of change. Methods Psychol. Res. Online 9, 37–55.
David, T.J., Dolmans, D.H.J.M., Patel, L., Van Der Vleuten, C.P.M., 1998. Problem-based
learning as an alternative to lecture-based continuing medical education. J. R. Soc.
Med. 91, 626–630. https://doi.org/10.1177/014107689809101204.
de-Marcos, L., Garcia-Lopez, E., Garcia-Cabot, A., 2016. On the effectiveness of game-like
and social approaches in learning: Comparing educational gaming, gamification &
social networking. Comput. Educ. 95, 99–113. https://doi.org/10.1016/j.compedu.
2015.12.008.
Dickinson, J.K., Woodard, P., Canas, R., Ahamed, S., Lockston, D., 2011. Game-based
trench safety education: development and lessons learned. Electron. J. Inf. Technol.
Constr. 16, 118–132.
Dieleman, H., Huisingh, D., 2006. Games by which to learn and teach about sustainable
development: exploring the relevance of games and experiential learning for sus-
tainability. J. Clean. Prod. 14, 837–847. https://doi.org/10.1016/j.jclepro.2005.11.
031.
Dimitrov, D.M., Rumrill, P.D., Rumrill Jr, P.D., 2003. Pretest-posttest designs and mea-
surement of change. Work 20, 159–165. https://doi.org/10.1017/
CBO9781107415324.004.
Din, Z., 2017. Teaching prevention through design (ptd) principles using a non-traditional
pedagogical strategy. Arizona State University.
Elam, C., Stratton, T., Gibson, D.D., 2007. Welcoming a new generation to college: The
millennial students. J. Coll. Admiss. 195, 20–25.
ESA (Entertainment Software Association), 2016. Essential facts about the computer and
video game industry.
Gale, M.T., 2011. Gameplay in higher education: the use of serious games vs traditional
instructional methods in learning. Auburn University, ProQuest Diss. Theses.
Gambatese, J.a., Hinze, J.W., Haas, C.T., 1997. Tool to design for construction worker
safety. J. Archit. Eng. 3, 32–41. https://doi.org/10.1061/(ASCE)1076-0431(1997)
3:1(32).
Gangolells, M., Casals, M., Forcada, N., Roca, X., Fuertes, A., 2010. Mitigating con-
struction safety risks using prevention through design. J. Safety Res. 41, 107–122.
https://doi.org/10.1016/j.jsr.2009.10.007.
Gibson, G.E., Bosfield, R., 2013. Assessment of effective front end planning processes,
research summary 268–1a. The Construction Industry Institute, Austin, TX.
Gibson, G.E., Kaczmarowski, J.H., Lore, H.E., 1995. Preproject-planning process for ca-
pital facilities. J. Constr. Eng. Manag. 121, 312–318. https://doi.org/10.1061/
(ASCE)0733-9364(1995) 121:3(312).
Hallowell, M.R., Hansen, D., 2016. Measuring and improving designer hazard recognition
skill: Critical competency to enable prevention through design. Saf. Sci. 82, 254–263.
https://doi.org/10.1016/j.ssci.2015.09.005.
Hartmann, T., Klimmt, C., 2006. Gender and computer games: exploring females’ dislikes.
J. Comput. Commun. 11, 910–931. https://doi.org/10.1111/j.1083-6101.2006.
00301.x.
Johnson, C.W., 2008. Using evacuation simulations for contingency planning to enhance
the security and safety of the 2012 Olympic venues. Saf. Sci. 46, 302–322. https://
doi.org/10.1016/j.ssci.2007.05.008.
Kamardeen, I., 2013. Design-for-Safety analysis support system for building designers. In:
Proceedings of the 13th International Conference on Construction Applications of
Virtual Reality, CONVR 2013, pp. 56–63.
Kebritchi, M., Hirumi, A., Bai, H., 2010. The effects of modern mathematics computer
games on mathematics achievement and class motivation. Comput. Educ. 55,
427–443. https://doi.org/10.1016/j.compedu.2010.02.007.
Kirschner, P.A., 2002. Cognitive load theory: implications of cognitive load theory on the
design of learning. Learn. Instr. 12, 1–10. https://doi.org/10.1016/S0959-4752(01)
00014-7.
Klisch, Y., Miller, L.M., Wang, S., Epstein, J., 2012. The impact of a science education
game on students’ learning and perception of inhalants as body pollutants. J. Sci.
Educ. Technol. 21, 295–303. https://doi.org/10.1007/s10956-011-9319-y.
Kwon, J., Lee, Y., 2016. Serious games for the job training of persons with developmental
187
Safety Science 115 (2019) 176–187
disabilities. Comput. Educ. 95, 328–339. https://doi.org/https://doi.org/10.1016/j.
compedu.2016.02.001.
Lateef, F., 2010. Simulation-based learning: Just like the real thing. J. Emerg. Trauma.
Shock 3, 348–352. https://doi.org/10.4103/0974-2700.70743.
Leder, J., Horlitz, T., Puschmann, P., Wittstock, V., Schütz, A., 2019. Comparing im-
mersive virtual reality and powerpoint as methods for delivering safety training:
Impacts on risk perception, learning, and decision making. Saf. Sci. 111, 271–286.
https://doi.org/10.1016/j.ssci.2018.07.021.
López-Arquillos, A., Rubio-Romero, J.C.C., Martinez-Aires, M.D.D., 2015. Prevention
through Design (PtD). The importance of the concept in Engineering and Architecture
University courses. Saf. Sci. 73, 8–14. https://doi.org/10.1016/j.ssci.2014.11.006.
Mann, J.A., 2008. Education issues in prevention through design. J. Safety Res. 39,
165–170.
Manuele, F.A., 2008. Prevention through design: addressing occupational risks in the
design and redesign processes. Prof. Saf. 53.
Manuele, F.A., 1997. On the practice of safety. Occupational Health and Safety Series,
Van Nostrand Reinhold.
Mendonça, D., Beroggi, G.E.G., van Gent, D., Wallace, W.A., 2006. Designing gaming
simulations for the assessment of group decision support systems in emergency re-
sponse. Saf. Sci. 44, 523–535. https://doi.org/10.1016/J.SSCI.2005.12.006.
Mikropoulos, T.A., Natsis, A., 2011. Educational virtual environments: a ten-year review
of empirical research (1999-2009). Comput. Educ. https://doi.org/10.1016/j.
compedu.2010.10.020.
NIOSH, 2014. The state of the national initiative on prevention through design [WWW
Document]. Dep. Heal. Hum. Serv. Centers Dis. Control Prev. Natl. Inst. Occup. Saf.
Heal.,. DHHS (NIOSH) Publication No. 2014–123. https://www.cdc.gov/niosh/
docs/2014-123/pdfs/2014-123.pdf.
Oliver, M., Carr, D., 2009. Learning in virtual worlds: using communities of practice to
explain how people learn from play. Br. J. Educ. Technol. 40, 444–457. https://doi.
org/10.1111/j.1467-8535.2009.00948.x.
Ortiz, S.A., Bowers, C.A., Cannon-Bowers, J.A., 2015. Video game self-efficacy and its
effect on training performance. Int. J. Serious Games 2.
OSHA (Occupational Safety and Health Administration), 2017. Commonly Used Statistics
- OSHA [WWW Document]. Commonly Used Stat. URL https://www.osha.gov/
oshstats/commonstats.html (accessed 1.1.17).
OSHA (Occupational Safety and Health Administration), n.d. OSHA’s Hazard identifica-
tion training tool [WWW Document]. URL https://www.osha.gov/hazfinder/index.
html (accessed 1.9.19).
Paas, F., Sweller, J., 2014. Implications of cognitive load theory for multimedia learning.
In: Mayer, R. (Ed.), The Cambridge Handbook of Multimedia. Learning. Cambridge
University Press, Cambridge, pp. 27–42. https://doi.org/10.1017/
CBO9781139547369.004.
Papastergiou, M., 2009. Digital game-based learning in high school computer science
education: Impact on educational effectiveness and student motivation. Comput.
Educ. 52, 1–12. https://doi.org/10.1016/j.compedu.2008.06.004.
Popov, G., Blunt, L.A., Mcglothlin, J., Young-Corbett, D., Zey, J.N., Heckel, P., Popov,
B.G., Blunt, L.A., Mcglothlin, J., Young-Corbett, D., Zey, J.N., Heckel, P., 2013.
Integrating PTD Into undergraduate curricula. Prof. Saf. 58, 44–49.
Prensky, M., 2003. Digital game-based learning. Comput. Entertain. 1, 21. https://doi.
org/10.1145/950566.950596.
Prensky, M., 2001. Digital natives, digital immigrants part 1. Horiz. 9, 1–6. https://doi.
org/10.1108/10748120110424816.
Price, P.C., Jhangiani, R.S., Chiang, I.-C.A., Leighton, D.C., Cuttler, C., 2017. 9.2
Interpreting the Results of a Factorial Experiment – Research Methods in Psychology.
In: Research Methods in Psychology. The Saylor Foundation, pp. 161–174.
Rinehart, R., Heidel, D., Okun, A., Barsan, M., 2009. Defusing prevention through design
principles through engineering textbooks. In: 2009 Annual Conference & Exposition.
American Society for Engineering Education, Austin, Texas, pp. 14.489.1–14.489.12.
Ritterfeld, U., Cody, M., Vorderer, P., 2009. Serious games: mechanisms and effects.
Routledge, New York, USA.
Saunders, P.M., 1997. Experiential learning, cases, and simulations in business commu-
nication. Bus. Commun. Q. 60, 97–114. https://doi.org/10.1177/
108056999706000108.
Squire, K.D., 2004. Replaying history: learning world history through playing civilization
III. Indiana University, Indianapolis, IN.
Stokes, S., 2002. Visual literacy in teaching and learning: a literature perspective.
Electron. J. Integr. Technol. Educ. 1, 10–19.
Susi, T., Johannesson, M., Backlund, P., 2007. Serious games: an overview. School
ofHumanities and Informatics. University of Skövde: Skövde.
Svinicki, M.D., 1999. New directions in learning and motivation. New Dir. Teach. Learn.
1999, 5–27. https://doi.org/10.1002/tl.8001.
Van Eck, R., 2006. Digital game-based learning: It’s not just the digital natives who are
restless. Educ. Rev. 41, 16–30.
Wagner, C., 2008. Learning experience with virtual worlds. J. Inf. Syst. Educ. 19,
263–266.
Weidenmann, B., 1989. When good pictures fail: An information-processing approach to
the effect of illustrations. In: Advances in Psychology. Elsevier, pp. 157–170. https://
doi.org/10.1016/S0166-4115(08)62153-6.
Zarges, T., Giles, B., 2008. Prevention through design (PtD). J. Safety Res. 39, 123–126.
https://doi.org/10.1016/j.jsr.2008.02.020.