International Journal of Game-Based Learning

Volume 6 • Issue 3 • July-September 2016

The Impact of a Racing Feature on

Middle School Science Students’
Performance in an Educational Game:
The Effect of Content-Free Game-Actions
Marilyn Ault, University of Kansas, Lawrence, KS, USA
Jana Craig-Hare, University of Kansas, Lawrence, KS, USA
Bruce Frey, University of Kansas, Lawrence, KS, USA

ABSTRACT

Reason Racer is an online, rate-based, multiplayer game designed to engage middle school students
in the knowledge and skills related to scientific argumentation. Several game features are included
as design considerations unrelated to science content or argumentation. One specific feature, a
competitive racing component that occurs in between challenging tasks, is the subject of this analysis.
The effect of two conditions on 72 ninth grade students’ performance was analyzed: game play with
a competitive racing component (Race) and game play without a competitive racing component (No-
Race). A counterbalanced design was used with two randomly assigned groups playing the game
using two different science scenarios. When students played with a racing component interspersed
between challenging tasks they completed the tasks more quickly and accurately than when they did
not experience the racing component. These findings are discussed in terms of game design and the
use of game features not related to academic content.

Keywords
Casual Game Features, Competition, Educational Game, Engagement, Flow, Middle-School Science, Scientific
Argumentation

LITERATURE REVIEW

Introduction
The overall effectiveness of educational games in engaging students in difficult subjects has been well
demonstrated (Plass et al., 2013; Shin, Sutherland, Norris, & Soloway, 2012; Squire & Jenkins, 2003;
Van Eck, 2006; Vogel et al., 2006). Games motivate learners and may make them more receptive to
learning (Plass et al., 2013). Educational game developers utilize the concept of flow as one design
consideration used to engage players and establish the motivation to play. What is needed, however,
are examples of how this construct is implemented within educational games.
This study examined the effect of a particular feature, a competitive car-racing experience
unrelated to the academic content, in a game designed to engage students in the difficult skill
of scientific argumentation. Reason Racer is a rally-race, arcade-style casual educational game
specifically designed to engage students by including many features that have been identified as

DOI: 10.4018/IJGBL.2016070102

Copyright © 2016, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

18
 International Journal of Game-Based Learning
Volume 6 • Issue 3 • July-September 2016

creating a sense of flow. A separate study demonstrated that middle school students who played the
game as a part of science class improved in knowledge and skills related to scientific argumentation as
well as improved in their confidence and motivation to engage in science, when compared to students
who did not have access to the game (Ault, Craig-Hare, Frey, Ellis, & Bulgren, 2015). This study
examined the impact that the competitive racing component, one game feature designed to create a
sense of flow, had on students’ performance.

Creating the Experience of Flow in Games

The concept of “flow” is an accepted construct describing intense engagement in an activity
(Csíkszentmihályi, 1975, 1997). Flow describes the experience of feeling totally involved in an
activity such as during a game when the player achieves a state of total focus, complete immersion,
and limited awareness of time. It is generally assumed that the experience involves intense
involvement and concentration, as well as enjoyment. The experience of flow draws players into
playing a game over and over, seemingly “compelling” them to play (Garris, Ahlers, & Driskell,
2002; Koster, 2005; Lazzaro, 2004). Authors of a series of studies, summarized by Hoffman and
Novak (2009), suggest that flow is a complex process, influenced by many elements. Features
demonstrated to elicit a sense of flow include focused goals, ease of learning, simplicity of play,
immediate feedback, quick rewards, and fast-paced game-play (Tams, 2006; Wallace & Robbins,
2006; Waugh, 2006), reward systems and challenges (Evans et al., 2013; Hamlen, 2013), interactivity
(Huang, 2003, 2006; Skadberg & Kimmell, 2004; Choi, Kim, & Kim, 2000), ease of use and
perceived usefulness (Agarwal & Karahanna, 2000; Hsu & Lu, 2004; Sanchez-Franco, 2006); and
attractiveness, novelty, and playfulness (Agarwal & Karahanna, 2000; Huang 2003, 2006; Skadberg
& Kimmel, 2004; Choi, Kim, & Kim, 2000).
Game developers strive to create a sense of flow during game play because when a player achieves
a state of total or compete focus, complete immersion, and limited awareness of time there is also
created a strong desire to repeat or extend the experience. This is identified as a compulsion to play,
the drive to play a game over and over (Garris, Ahlers, & Driskell, 2002; Koster, 2005; Lazzaro,
2004). Educational game developers focus on creating a sense of flow during play in order to provide
an environment in which students strive to improve their performance.

Flow as a Model for Game Design

Flow models have been applied to educational game design and development. For example, Fu
and Yu (2008) suggest there are four flow factors in games: skill, challenge, concentration, and
pleasure. Sweetser and Wyeth (2005) propose a game flow model that consists of eight core
elements. They include elements requiring concentration, providing a challenge, requiring use of
specific skills, and providing control, clear goals, and feedback. They also suggest there should
be a sense of immersion and social interaction. The activites that elicit a sense of flow must
be intrinsically rewarding and autotelic, or meaningful or fulfilling in and of itself (Sweetser
& Wyeth, 2005). Kickmeier-Rust and his colleagues (Kickmeier-Rust, et al., 2007) argue that
educational games successful in creating a sense of flow need to provide clear goals and rules,
a meaningful learning context, an engaging storyline, immediate feedback, high levels of
interactivity, challenge and competition, random elements of surprise, and rich and appealing
learning environments (Presky, 2001). The early work of Malone (Malone, 1980, 1981; Malone
& Leppert, 1987) in proposing the motivational aspects of endogenous, versus exogenous, factors
being critical for learning in games have been re-interpreted as an effect of the experience of
flow (Habgood, Ainsworth, & Benford, 2005).

19
International Journal of Game-Based Learning
Volume 6 • Issue 3 • July-September 2016

Competition as a Contributor to a Sense of Flow

While not a part of Csíkszentmihályi’s original proposal for the creating a sense of flow, many
developers have identified the social context of a game as a critical component of establishing
game behavior (Hsu & Lu, 2004; Sweetser & Wyeth, 2005). The social context of the game, both
collaboration and competition, have been demonstrated to be important in creating the motivation
to play and a significant contributor to the play experience (Choi & Kim, 2004; Hsu & Lu, 2004;
Koster, 2005).
Hsu and Lu (2004) hypothesize social influence to be an informational influence (when a user
accepts information from other users) and normative influence (when a person conforms to the
expectations of others). Lazzaro (2004) proposes “The People Factor” as one of the four key game
factors that elicit an emotional attachment. This utilizes the social experiences of competition,
teamwork, as well as the opportunity for social bonding and the personal recognition that comes from
playing with others. Others agree that competition is a significant contributor to the play experience
(Koster, 2005) and is seen as a key factor in increasing the compulsion to play (Plass et al., 2013).
In a study comparing individual play in a variety of conditions, students who played in competitive
situations increased in-game learning over students who did not experience competitive play. In
addition, competition elicited greater interest in and enjoyment of the game, resulted in a stronger
mastery goal orientation, and stronger intentions to play the game again and to recommend it to
others (Plass et al., 2013). Generally students experienced greater enjoyment and improved skills
when playing against others.

Operationalizing a Game Feature to Create a Sense of Flow

While the list of features that should be included in game design to create a sense of flow is informative,
actual examples of game-features are needed to inform both developers and educators. These are the
core mechanics of a game, the procedural mechanisms that provide the essential interactions required
to create a meaningful gaming activity (Habgood, Ainsworth, & Benford, 2005). This study explores
a game feature, a core mechanic, which includes high levels of interactivity and competition in a
car-racing component of the game that is unrelated to the academic skill or content being addressed
in the game. The consideration is whether all educational game actions designed to engage players
should include the academic content, or can the opportunity to engage in competition unrelated to
academic content effect overall performance. The issue is addressed within the context of a game,
Reason Racer, designed to provide middle school students with experience in knowledge and skills
related to scientific argumentation.
The development of Reason Racer builds on lessons learned from a three-year study funded by the
National Science Foundation (Bulgren & Ellis, 2012). That research examined the ability of teachers
of grades six through nine to incorporate scientific reasoning into their regular instruction at least
10 times throughout a year, and to assess their students’ abilities to acquire the skills of analysis of
arguments. While teachers found that the argumentation and evaluation instruction enhanced students’
understanding of inquiry and critical thinking, both teachers and students had difficulty engaging
in the process and putting it into practice. They found both the content of argumentation and the
process proposed in the study difficult to implement, time consuming, and not engaging. The skill
of developing an argument based on evidence continues to be a challenge, and additional resources
are needed to support instruction (Purcell, Buchanan, & Friedrich, 2013).
We proposed the use of a casual, rate-based, competitive game to overcome the problem of
engagement. We hypothesized that students would actively participate in a highly engaging game and
that repeated play would provide them with sufficient practice in the skills and knowledge related
to scientific argumentation to support learning the skill. We also hypothesized that a game would
engage students in a way that made the process of making decisions about science fun. Among the
many game genres demonstrated to be effective in both engaging students and supporting learning,
we chose a casual game format (Squire, 2005). This format has a low production cost, is easily

20
 International Journal of Game-Based Learning
Volume 6 • Issue 3 • July-September 2016

expanded to include a variety of content, and engages students. The decisions and resulting actions
that students encounter in the game include identifying, matching, sorting, ranking, and racing in a
fast-paced competitive environment, similar to many arcade-styled games.

Research Question
The Reason Racer game uses a rally race format to engage the players in a rate-based, competitive,
multi-player game, alternating between challenges, or PitStops, and car racing segments across a
variety of racecourses. The PitStops contain the educational content of the game, requiring students
to make decisions about science concepts by applying components of scientific argumentation. During
the PitStops, students attempt to move through the challenge quickly, with as few errors as possible,
while receiving feedback on each answer. A competitive car-racing component is completed after
each PitStop, where students navigate through a variety of racetracks around obstacles and turns,
similar to many racing games, as quickly as possible to move to the next PitStop. The accuracy of
a player’s performance in the PitStop affects the speed with which his or her car can move through
the next racing segment, providing a disincentive for guessing. The racing components apply design
features intended to create a sense of flow: challenging, easy to learn, simple to play, high levels of
interactivity, and access to feedback, clear goals and rules, and competition. The race-car’s speed
and direction is controlled by the player through key-strokes. While racing the players can see the
relative positions of their competitors on the racetrack, in real-time, and whether they are gaining on
or falling behind other players. Students, therefore, alternate completing challenging tasks as quickly
and correctly as they can with rapid-play car-racing segments of the game.
This study addressed the primary question: Does the inclusion of the content-free car-racing
component as part of game-play effect students’ performance during the challenging PitStops?
Secondly, a brief analysis was conducted to determine whether students reported having a sense of
flow during the game?

METHOD

Subjects
Players for the study included 72 ninth grade students, 51% female and 49% male, from one high school
in Kansas City, Kansas during Spring 2012. Of this group, 39% described themselves as White, 37%
as Hispanic, 23% as African American, and 1% as Asian. Fifty-one percent of the participants qualify
for free or reduced lunch and 10% receive services for English as a second language. In terms of the
achievement level for the whole school: 19% do not meet the metric of at-or-exceeding proficiency
in science, 30% do not meet proficiency in reading, and 25% do not meet the standard in math.

Materials
The application used in this study was the Reason Racer game. The game is accessed through the
internet and individual student performance is captured in a database on the server that hosts the game.
The Reason Racer game is designed to provide students with practice in the use of the components of
argumentation as applied to science scenarios. This involves the players in a competitive, multiplayer
rally-type race, alternating between challenges, or PitStops, and racing segments across a variety of
racecourses. The PitStops engage students in decisions about evidence, claims, and reasoning using
40 scenarios developed in the areas of physical, life, earth and space science. Other research found
that frequent play of the game as part of instruction across a two-month period increases scientific
argumentation skill and increases student confidence and motivation to engage in science compared
to students who received traditional instruction only (Ault, Craig-Hare, Frey, Ellis, & Bulgren, 2014).
As students play through a game session in Reason Racer they address challenges and questions
about a single science scenario. Multiple play sessions could repeat the same scenario or access others

21
International Journal of Game-Based Learning
Volume 6 • Issue 3 • July-September 2016

scenarios assigned by the teacher. In this study, the students played the game using two scenarios
in the area of life science that dealt with real science claims, evidence, and arguments. The Worm
Glue scenario addressed the finding that sand worms create their own glue as they construct sand
tunnels. The scientific claim presented is that this glue may help in the healing of broken bones. The
Panda Poop scenario dealt with the finding that enzymes in a panda stomach may have potential
for producing biofuels in a carbon-neutral manner. Both scenarios involve decisions about claims,
evidence, and reasoning in the PitStops.
The PitStops require actions that are common to fast-paced games, such as matching, ranking,
sorting, and discriminating, all within a rate-based game interface. Figure 1 shows the PitStop
challenges from another scenario That Shrimp Packs a Punch!”. The PitStops require students to

Figure 1. A sample of the PitStops requiring students to make decisions about evidence, claims, and warrants and engage them
in actions that are common to fast-paced games such as matching, ranking, sorting, and discriminating, all within a rate-based
game interface

22
 International Journal of Game-Based Learning
Volume 6 • Issue 3 • July-September 2016

identify components or make decisions about the claim, evidence, or reasoning. Students attempt
to move through each PitStop as quickly, and with as few errors, as possible. Each PitStop requires
between three and ten correct responses, depending on the type of challenge, before the player is
allowed to move on in the game. Incorrect responses slow down the presentation of items in the PitStop,
providing a disincentive for guessing. Even though all the players are engaged in the same scenario,
each experiences the items in the PitStop in a different way. This is accomplished by pulling items
randomly from a database, ensuring that each play experience is unique and that a student can play
the game multiple times without repeating the same items or order of items in a PitStop. The PitStop
questions assess the students’ ability to engage in these six key processes which define scientific
argumentation: distinguishing fact from opinion, identifying claims, recognizing qualifying language,
ranking the relative strength of evidence, and recognizing the various types and relative strengths of
“warrants” (patterns of reasoning).
The competitive racing component, Figure 2, is completed between each PitStop. During this part
of the game the students navigate various racing tracks with turns and obstacles as quickly as possible
to move to the next PitStop. Players use keyboard arrow keys to navigate their car and control their
speed throughout the racecourse. Some features of the racing component were specifically designed

Figure 2. Sample of the car-racing components, unrelated to content, in which students move their car as quickly as possible
across a variety of racecourses to the next PitStop

23
International Journal of Game-Based Learning
Volume 6 • Issue 3 • July-September 2016

to promote flow (Fu & Yu, 2008). They were control, clear goals and rules, feedback, high levels of
interactivity, ease of learning, simplicity of play, quick rewards, and competition. Up to 20 players
can be engaged in a race at the same time and, as is shown at the bottom of both figures, each player
can see the relative, real-time position of the other players.

Procedure
The study was conducted in the school computer lab during 9th grade biology instructional time.
This experience was the students’ first encounter with the game and with instruction specifically
addressing components of argumentation. Students were randomly assigned to either Group 1 or 2,
as shown in Table 1, and received a URL to a web page giving them access to the appropriate game
conditions, race or no-race condition, and a follow-up survey. After a brief introduction on how to
play the game and how scientists use claims, evidence, and reasoning, each student logged onto the
game with a unique login ID. The whole process of introducing the game-play process, discussing
argumentation, playing the game across two scenarios, completing a survey, and participating in a
question & answer period lasted about 35 minutes. This process was repeated four times in one day
for each of the 9th grade biology classes.

Design
The design of the study, as reflected in Table 1, was counterbalanced with two randomly assigned
groups playing the game using both scenarios, Panda Poop and Worm Glue, in one of two conditions.
The conditions of interest were 1) game play with racing between each PitStop and 2) game play with
no racing between each PitStop. Both groups played the Panda Poop scenario first and the Worm Glue
scenario second. The Race and No-Race conditions were alternated with Group One experiencing the
Race condition first and Group Two experiencing the No-Race condition first. The only difference
in the game experience for the students was whether they engaged in competitive car-racing between
PitStops. The challenges in the PitStop, in both scenarios were the same for both groups. Students
in each condition were separated so no cross-contamination of the game condition occurred, the
students were not aware that the other group was having a different game experience, with or without
a car-racing component, until the discussion took place after the surveys were completed. No student
required more than eleven minutes to play an entire game, with an average duration of six minutes.

Flow Measurement Instrument

A project-developed 13-item attitude scale was developed and used to collect student feedback
following both game sessions (shown in the Appendix, Table 4). The survey was primarily designed
to provide a preliminary analysis of components of the construct of flow using items similar to Hsu
and Lu (2004) and those reviewed by Hoffman and Novak (2009). Items addressed a sense of flow
and reflected, in a brief assessment, its multidimensional quality (Huang 2006; Koufaris 2002). Items
asked about ease of use, sense of fun, social nature, and usefulness. Additionally, one item related
to perception of time, as suggested by Pace (2004). The questions used to form a flow subscale were

Table 1. Students’ race and no-race play conditions with two game scenarios

Game Play Session by Condition

Group Number Scenario
1st 2nd
Panda Poop Race
1
Worm Glue No-Race
Panda Poop No-Race
2
Worm Glue Race

24
 International Journal of Game-Based Learning
Volume 6 • Issue 3 • July-September 2016

1, 3, 4, 7, 10, 12 and 13. The other items asked whether students felt the game helped them learn
about science or argumentation, whether the game was easy or hard, and whether their friends played
online games.

Data Collection and Analysis

Two types of data were collected during this study. The web-based game was connected to a database
that captured player-metrics associated with each unique login in a log file. The log files captured
metrics for each player including duration, in milliseconds, for each component of the game, PitStops
and race, as well as the number, frequency, and type of incorrect responses, and overall duration of
the game. Each participant also completed the survey instrument. The survey was hyperlinked and
accessed after the games were played.
Paired samples t tests were conducted to compare the duration of play and number of errors in
each PitStop across two conditions. Data from the survey were analyzed based on descriptive statistics
and single-sample t tests.

RESULTS

Game Performance
Figure 3 presents the average amount of time spent in the PitStops. PitStops are where students utilize
argumentation skills within the context of one scenario to answer challenge questions. In the figure,
the six types of PitStops are labeled by the particular scientific argumentation skill being addressed.
Overall, students spent less time in a PitStop when they were engaging in a car-race than when there
was no car-race opportunity. As indicated in the figure, this difference was significant for three of
the six PitStops, but consistent across all six PitStops in that when the students had an opportunity to
race, less time was spent answering questions. Paired sample t tests were conducted to evaluate student

Figure 3. Average number of seconds per PitStop in the two conditions of race and no-race

25
International Journal of Game-Based Learning
Volume 6 • Issue 3 • July-September 2016

duration of play per PitStop comparing the Race and No-Race Conditions. The descriptive statistics
used for these analyses (and the comparisons of errors between conditions) are shown in Table 2.
There was a significant difference in the duration of play for these PitStops: Selecting the Best Claim
Statement (M = 13.26 seconds, SD = 33.90 seconds), t(64) = 3.15, p = .002; Ranking Evidence (M
= 9.62 seconds, SD = 30.10 seconds), t(64) = 2.58, p = .012; and Identifying the Warrant Type (M
= 6.67 seconds, SD = 21.09 seconds), t(64) = 2.55, p = .013. The standardized effect size index, d,
was .4, .3, and .3 respectively and indicates a small effect.
In most cases, however, the increased speed in the PitStops for those in the Race condition also
resulted in fewer errors for four of the six PitStops. Figure 4 displays the average number of missed
questions per PitStop in both the Race and No-Race conditions. When playing the game without the
racing component of the game, students missed more questions overall and in four of the six individual
PitStops. These differences, however, were not statistically significant.

Survey Results
Survey data were collected after both play conditions were completed. Results are presented in Table
3. Overall the students confirmed the first assumption of this study, that the game engages students
in a “flow” experience. Using a Likert-type strongly disagree to strongly agree scale of 1 to 5, the
mean score on the Flow scale was 3.30 (SD = .52), which is above a the neutral score of 3, t(69) =
11.28, p ≤ .001.
Two survey questions addressed whether students thought they could learn about both science
and argumentation while playing the game. While students felt they learned “about science” while
playing the game (M = 3.76, SD = .97, t(69) = 6.53, p ≤ .001.), they were neutral on the question
of whether they learned “about arguments” (M = 3.04, SD = 1.16).

Table 2. Difference in students’ performance when comparing race and no-race conditions by PitStop

PitStop Decision Race No-Race

t p
Speed Mean SD Mean SD
1. Identify Fact v. Opinion
42102.85 23374.91 43713.77 24561.78 0.54 0.591
Statements
2. Identify Best Claim Statement 36720.75 24768.37 49979.94 27064.31 3.15 0.002*
3. Identify Qualifiers 18869.80 15930.43 25662.15 32146.87 1.54 0.130
4. Rank Evidence 29949.83 21672.40 39567.80 25148.97 2.58 0.012*
5. Identify Type of Reasoning 41276.83 15268.48 47951.08 19866.65 2.55 0.013*
6. Determine Quality of Reasoning 24775.35 11642.77 27820.05 11019.62 1.77 0.082
PitStop Decision Race No-Race
t p
Errors Mean SD Mean SD
1. Identify Fact v. Opinion
2.05 2.77 1.49 2.05 1.54 0.129
Statements
2. Identify Best Claim Statement 1.63 1.99 2.20 2.34 1.43 0.158
3. Identify Qualifiers 0.45 1.41 1.08 3.59 1.39 0.170
4. Rank Evidence 2.51 2.57 2.91 3.22 0.80 0.426
5. Identify Type of Reasoning 3.66 1.95 4.02 2.53 1.07 0.290
6. Determine Quality of Reasoning 2.94 1.20 2.86 1.22 -0.41 0.683
* Indicates significant difference between the Race and No-Race conditions.

26
 International Journal of Game-Based Learning
Volume 6 • Issue 3 • July-September 2016

Figure 4. Average number of errors per PitStop in the race and no-race conditions

Table 3. Descriptive statistics of survey results

Items M SD
Flow Subscale (7 items, coefficient alpha = .83) 3.30 .52
I learned about science when I played this game. 3.76 .97
I learned about arguments while I played this game. 3.04 1.16

DISCUSSION

Strengths
The major finding of this study suggests that mechanics of a portion of a game can be unrelated to
content yet influence performance during content portions of an educational game. The procedural
mechanism that provided interactions and created a meaningful gaming activity was a competitive,
car-race experience. When the car-race opportunity was present in between the challenging portions
of the game students had a tendency to complete the challenging portions of the game more quickly
and with more accuracy than when there was no racing experience. Even though this difference was
significant in the speed with which students completed three PitStops, the results do reflect faster
completion rates in all the PitStops and more accurate completion in five of the seven PitStops when
the car-race experience was present.
The features of the car-race experience are consistent with the elements that are considered to
create a sense of flow, or total engagement, in the game. These include, for example, goals and rules
(Kickmeier-Rust, et al., 2007; Malone, 1981), feedback and fast paced play (Tams, 2006; Wallace &
Robbins, 2006; Waugh, 2006), interactivity (Huang, 2003, 2006; Presky, 2001; Skadberg & Kimmell,
2004; Choi, Kim, & Kim, 2000), and control, cooperation, and competition (Habgood, Ainsworth,
& Benford, 2005). These experiences were design considerations for an experience that occurred
between academic content portions of the game yet maintained students’ engagement and excitement
for play. Students drive their cars (which they selected) and compete against others both in the game
and on the racetrack. Students can control their speed and direction, use game-incentives to purchase

27
International Journal of Game-Based Learning
Volume 6 • Issue 3 • July-September 2016

strategies to pass other players, and compete to reach the next PitStop as quickly as possible. The speed
of the player’s car during the car-race is dependent on his or her accuracy in the previous PitStop.
The car-race actions, however, do not involve any consideration about the content of the game. The
experience is the rally race that occurs between two challenges and is designed to be fun and engaging.
The effect of the care-race on students’ performance in the PitStops suggests that a mechanism that
provides interactions and creates a meaningful gaming activity does not have to address the content
of the game in order to positively affect students’ engagement in the game and improved outcomes.
The survey data support the hypothesis that students experienced a sense of flow while playing
the game. Students reported that time seemed to pass quickly, peers would enjoy the game, and
they had good feelings while playing the game. These are all indicators of “flow” or an emotional
attachment that facilitates learning. Students in this study were only given the opportunity to play
the game twice (once with and once without the racing experience), yet their responses in the survey
indicated that they did play for both a good score and to improve their score. During our iterative
development process, we had several students independently play earlier prototypes multiple times
just to improve their scores.
It is also interesting that students felt the game helped them to learn “about science”, but were
not convinced that it helped them learn “about argumentation”. The game was designed with an
intrinsic definition of science as the process of scientific argumentation, so one might have expected
the perceptions to be the same. Clearly, the 9th grade science students in our sample view the two
terms as different from each other. It may be that they perceived the content of the scenarios- facts
about the natural world, theories about biology, research reports- as science stuff, but did not view
argumentation as the central activity of scientists. Further research needs to address whether playing
the game affects both an understanding of science concepts presented in the scenarios and the ability
to make decisions related to the process of argumentation.

Limitations
This analysis was limited in that the students only played the game with the racing component once. While
a mild effect was evident, the experience of only one game-play session was not enough to determine the
magnitude of the effect of the car-race component on students’ overall performance. A more complete
analysis of the effect of the racing component would involve multiple game session with one group having
access to the racing component and the other group playing the game without the racing component. This
study does, however, provide some evidence to suggest that the racing component does make a difference.
In addition, while the students did indicate a sense of flow while playing the game, the survey
did not request students to differentiate the experience of the car-race from the overall experience
of the game. This lack makes it difficult to determine whether the car-race experience sustained
engagement by providing an experience of flow.

Implications and Suggestions for Future Research

This study represents an attempt to connect the effect of a game feature designed specifically to engage
students in the experience of “flow” with performance in tasks associated with science content and
higher-order thinking skills. While strategies to create engagement, or flow, have been a goal of game
research, the use of features such as competition and racing, aligned with other elements such as rapid
and high rates of responding, focused goals, ease and simplicity of play, and quick rewards, represents
a new line of investigation and game development in the area of science education. Casual games,
such as Reason Racer, can be developed quickly and with relatively low production costs, so they
are a promising area of development of resources to address the Next Generation Science Standards
(Achieve 2013). This is supported by the indication that students thought they learned science while
playing the game. Further research is needed to validate this assumption.
None of these findings are surprising to commercial game developers, because the engagement
potential of a game is the top development priority, with the goal of establishing a compulsion to

28
 International Journal of Game-Based Learning
Volume 6 • Issue 3 • July-September 2016

play. Typical educational game developers, however, do not take advantage of the potential to engage
students through the creation of a sense of flow with game mechanics that are distinct from the content
addressed in the game. The implication for educational game development is whether similar types
of activities, ones that are not related to content but specifically designed to heighten emotional
attachment and a sense of flow, should be considered an integral part of games that are designed
for students to practice complex and difficult skills. This component reinforces what Von Ahn has
observed: “Some tasks are inherently unenjoyable -- until you make them a game” (Thompson 2007).

ACKNOWLEDGMENT

The authors wish to thank the students and teachers from Bishop Ward High School in Kansas
City, Kansas who participated in the research. This research was supported by the National Science
Foundation under Grant #1019842. Any opinions, findings, and conclusions or recommendations
expressed in this material are those of the authors and do not necessarily reflect the views of the
National Science Foundation.

29
International Journal of Game-Based Learning
Volume 6 • Issue 3 • July-September 2016

REFERENCES

Achieve, Inc. (2013). Next Generation Science Standards. Retrieved from http://www.nextgenscience.org/
Agarwal, R., & Karahanna, E. (2000). Time flies when you’re having fun: Cognitive absorption and beliefs about
information technology usage. Management Information Systems Quarterly, 24(4), 665–694. doi:10.2307/3250951
Ault, M., Craig Hare, J., Bulgren, J., & Ellis, J. (2012). Analysis of a game feature designed to heighten
engagement and learning. In P. Resta (Ed.), Proceedings of Society for Information Technology & Teacher
Education International Conference 2012 (pp. 2462–2466). Chesapeake, VA: AACE; Retrieved from http://
www.editlib.org/p/39952
Ault, M., Craig-Hare, J., Frey, B., Ellis, J., & Bulgren, J. (2015). The effectiveness of Reason Racer, a game
designed to engage middle school students in scientific argumentation. Journal of Research on Technology in
Education, 47(1), 1–21. doi:10.1080/15391523.2015.967542
Bulgren, J., & Ellis, J. (2012). Argumentation and Evaluation Intervention in Science Classes: Teaching and
Learning with Toulmin. In M. S. Khine (Ed.), Perspectives on Scientific Argumentation (pp. 135–154). Springer
Netherlands. doi:10.1007/978-94-007-2470-9_8
Choi, D. H., Kim, J., & Kim, S. H. (2007). ERP Training with a web-based electronic learning sSystem: The
Flow Theory perspective. International Journal of Human-Computer Studies, 65(3), 223–243. doi:10.1016/j.
ijhcs.2006.10.002
Csíkszentmihályi, M. (1975). Beyond boredom and anxiety. San Francisco: Jossey-Bass.
Csíkszentmihályi, M. (1990). Flow: The psychology of optimal experience. New York: HarperPerennial.
Csikszentmihalyi, M. (1997). Finding flow: The psychology of engagement with everyday life. New York: Basic
Books.
Fu, F. L., & Yu, S. C. (2008). Three layered thinking model for designing web-based educational games. In
Advances in Web Based Learning-ICWL 2008 (pp. 265-274). Springer Berlin Heidelberg.
Garris, R., Ahlers, R., & Driskell, J. (2002). Games, motivation and learning: A research and practice model.
Simulation & Gaming, 33(4), 441–467. doi:10.1177/1046878102238607
Habgood, M. P. J., Ainsworth, S. E., & Benford, S. (2005). Endogenous fantasy and learning in digital games.
Simulation & Gaming, 36(4), 483–498. doi:10.1177/1046878105282276
Hoffman, D. L., & Novak, T. P. (2009). Flow Online: Lessons Learned and Future Prospects. Journal of Interactive
Marketing, 23(1), 23–34. doi:10.1016/j.intmar.2008.10.003
Hsu, C. L., & Lu, H. P. (2004). Why do people play on-line games? An extended TAM with social influences
and flow experience. Information & Management, 41(7), 853–868. doi:10.1016/j.im.2003.08.014
Huang, M. H. (2003). Designing website attributes to induce experiential encounters. Computers in Human
Behavior, 19(4), 425–442. doi:10.1016/S0747-5632(02)00080-8
Huang, M. H. (2006). Flow, Enduring and Situational Involvement in the Web Environment: A Tripartite Second-
Order Examination. Psychology and Marketing, 23(5), 383–411. doi:10.1002/mar.20118
Kickmeier-Rust, M. D., Peirce, N., Conlan, O., Schwarz, D., Verpoorten, D., & Albert, D. (2007). Immersive
digital games: the interfaces for next-generation e-learning? In Universal Access in Human-Computer Interaction.
Applications and Services (pp. 647-656). Springer Berlin Heidelberg. doi:10.1007/978-3-540-73283-9_71
Koster, R. (2005). A theory of fun for game design. Scottsdale, AZ: Paraglyph Press.
Koufaris, M. (2002). Applying the Technology Acceptance Model and Flow Theory to Online Consumer
Behavior. Information Systems Research, 13(2), 205–223. doi:10.1287/isre.13.2.205.83
Lazzaro, N. (2004). Why we play games: Four keys to more emotion without story. Retrieved from http://www.
xeodesign.com/xeodesign_whyweplaygames.pdf

30
 International Journal of Game-Based Learning
Volume 6 • Issue 3 • July-September 2016

Malone, T. W. (1980, September 19). What makes things fun to learn? Heuristics for designing instructional
computer games. Paper presented at theAssociation for Computing Machinery Symposium on Small and Personal
Computer Systems, Palo Alto, California. doi:10.1145/800088.802839
Malone, T. W. (1981). Toward a theory of intrinsically motivating instruction. Cognitive Science, 4(4), 333–369.
doi:10.1207/s15516709cog0504_2
Malone, T. W., & Lepper, M. R. (1987). Making learning fun: Ataxonomy of intrinsic motivations for learning. In
R. E. Snow & M. J. Farr (Eds.), Aptitude, learning and instruction: III. Conative and affective process analyses
(pp. 223–253). Hilsdale, NJ: Lawrence Erlbaum.
Plass, J. L., O’Keefe, P. A., Homer, B. D., Case, J., Hayward, E. O., Stein, M., & Perlin, K. (2013, September9).
The Impact of Individual, Competitive, and Collaborative Mathematics Game Play on Learning, Performance,
and Motivation. Journal of Educational Psychology, 105(4), 1050–1066. doi:10.1037/a0032688
Prensky, M. (2001). Digital game-based learning. New York: McGraw-Hill.
Prensky, M. (2005). Complexity matters. Educational Technology, 45(4), 5–20.
Purcell, K., Buchanan, J., & Friedrich, L. (2013). The impact of digital tools on student writing and how writing
is taught in schools. The Pew Research Center, National Writing Project. Retrieved from http://pewinternet.org/
Reports/2013/Teachers-technology-and-writing
Sanchez-Franco, M. J. (2006). Exploring the influence of gender on web usage via partial least squares. Behaviour
& Information Technology, 25(1), 19–36. doi:10.1080/01449290500124536
Shin, N., Sutherland, L. M., Norris, C. A., & Soloway, E. (2012). Effects of game technology on elementary
student learning in mathematics. British Journal of Educational Technology, 43(4), 540–560. doi:10.1111/j.1467-
8535.2011.01197.x
Skadberg, Y. X., & Kimmel, J. R. (2004). Visitors’ flow experience while browsing a web site: Its measurement,
contributing factors and consequences. Computers in Human Behavior, 20(3), 403–422. doi:10.1016/S0747-
5632(03)00050-5
Squire, K. (2005). Game-based learning: Present and future state of the field. x-Learn Perspective Paper, Maise
Center eLearning Consortium, Saratoga Springs: NY
Squire, K., & Jenkins, H. (2003). Harnessing the power of games in education. InSight, 3(5). Retrieved
from http://66.102.1.104/scholar?hl=en&lr=&q=cache:21N8iW3kLEAJ:www.edvantia.org/products/pdf/
InSight_3-1_Vision.pdf+games+education
Sweetser, P., & Wyeth, P. (2005). GameFlow: a model for evaluating player enjoyment in games. Computers in
Entertainment, 3(3), 3-3. ACM. Retrieved from http://dl.acm.org/citation.cfm?id=1077246.1077253
Tams, J. (2006). Online casual games Q&A. Minna Magazine, 2-5. Retrieved from http://mag.casualconnect.
org/MinnaMagazine_Summer2006.pdf
Thompson, C. (2007). For certain tasks, the cortex still beats the CPU. Wired Magazine, 15.
Toulmin, S. (2003). The uses of argument. Cambridge, UK: Cambridge University Press. doi:10.1017/
CBO9780511840005
Toulmin, S., Rieke, R., & Janik, A. (1984). An introduction to reasoning. Upper Saddle Ridge, NJ: Prentice Hall.
Van Eck, R. (2006). Digital game-based learning: It’s not just the digital natives who are restless. EDUCAUSE
Review, 41(2), 16.
Vogel, J. J., Vogel, D. S., Cannon-Bowers, J., Bowers, C. A., Muse, K., & Wright, M. (2006). Computer gaming
and interactive simulations for learning: A meta-analysis. Journal of Educational Computing Research, 34(3),
229–243. doi:10.2190/FLHV-K4WA-WPVQ-H0YM
Wallace, M., & Robbins, B. (Eds.). (2006). IGDA 2006 casual games white paper. Retrieved from http://www.
igda.org/casual/IGDA_CasualGames_Whitepaper_2006.pdf
Waugh, E. (2006). GDC: Casual Games Summit 2006: An introduction to casual games. Retrieved from http://
www.gamasutra.com/features/20060322/waugh_01.sht ml

31
International Journal of Game-Based Learning
Volume 6 • Issue 3 • July-September 2016

Marilyn Ault is a Senior Research Associate and Director of ALTEC at the University of Kansas Center for Research
on Learning. Her research interests focus on the instructional use of technology, including the design and use of
educational games, to affect teaching and learning.

Jana Craig-Hare is an Assistant Research Professor and Associate Director of ALTEC at the University of Kansas
Center for Research on Learning. Her research interests focus on the instructional use of technology, school-wide
interventions, and program evaluation.

Bruce Frey is an Associate Professor in the Department of Educational Psychology in the School of Education at
the University of Kansas. His research interests focus on classroom assessment, instrument development, and
program evaluation methodology.

32
 International Journal of Game-Based Learning
Volume 6 • Issue 3 • July-September 2016

APPENDIX

Rally Game Survey

Thank you for helping us. We want to know what you think about the Rally Game.
We do not know your name so your name will not be linked with our research findings. With
the Internet it is possible that some other person could see your scores. Since your name is not linked
with your answers, it should be as safe as we can make it!

Table 4. Rally game survey

Strongly Agree ↔ Strongly Disagree

I like playing this game with my friends. :-D :-) :-| :-[ :-(
This game is easy to play. :-D :-) :-| :-[ :-(
My friends probably like this game. :-D :-) :-| :-[ :-(
It was a lot of fun to play this game. :-D :-) :-| :-[ :-(
Most of my friends play online games. :-D :-) :-| :-[ :-(
I learned about science when I played this game. :-D :-) :-| :-[ :-(
I felt good while playing this game. :-D :-) :-| :-[ :-(
This game was too hard to play. :-D :-) :-| :-[ :-(
I learned about arguments while I played this game. :-D :-) :-| :-[ :-(
I was bored while playing this game. :-D :-) :-| :-[ :-(
I tried to improve my score. :-D :-) :-| :-[ :-(
I don’t want to play this game again :-D :-) :-| :-[ :-(
Time passed quickly while I was playing. :-D :-) :-| :-[ :-(

Your number: _______________________

Is there anything else you’d like to tell us about the game?

Thank you for your help!

33