Computers in Human Behavior 74 (2017) 63e71

Contents lists available at ScienceDirect

Computers in Human Behavior

journal homepage: www.elsevier.com/locate/comphumbeh

Full length article

BYOD or not: A comparison of two assessment strategies for student

learning
Pao-Nan Chou a, *, Chi-Cheng Chang b, Ching-Hsin Lin c
a
Department of Education, National University of Tainan, Tainan, Taiwan
b
Department of Technology Applications and Human Resource Development, National Taiwan Normal University, Taipei, Taiwan
c
Shan-Hua Junior High School, Tainan, Taiwan

a r t i c l e i n f o a b s t r a c t

Article history: This study investigated the effects of the BYOD (bring your own device) approach on student language
Received 28 December 2016 learning. In the research scenario, junior high school students brought their Android-based smart phones
Received in revised form to class in order to engage in language evaluation activities, where an application (app) called Socrative
22 March 2017
was used as a learning assessment tool. A quasi-experimental pre-test and post-test control group design
Accepted 11 April 2017
Available online 14 April 2017
was used to answer the research question. 46 eighth-grade students from two classes at a public junior
high school in Taiwan participated in the four-week educational experiment. Student participants
received 6 quizzes as a formative evaluation during the experiment. A learning achievement test
Keywords:
Bring your own device
(summative evaluation) was developed to measure the students' understanding of English vocabulary
Mobile learning and grammar. One month after the completion of the summative evaluation, the same achievement test
Instructional media evaluation (delayed summative evaluation) with different item numbers was administered to assess the long-term
Experimental study transfer of learning in the students. A self-report questionnaire and an informal interview were used to
Assessment strategy ascertain the students' learning experiences. The formative evaluation results showed that students in
Language learning the traditional instruction class outperformed those in the BYOD instruction class because several BYOD
students were not familiar with the BYOD approach. Although the BYOD and the traditional instruction
approaches yielded similar summative evaluation and delayed summative evaluation results, the BYOD
approach demonstrated a valuable benefit on the students' long-term transfer of learning. Students in
the BYOD instruction class exhibited a steady growth on learning outcomes and subsequently scored
higher on the learning retention segment of the study. In addition, the BYOD teaching practice advanced
the students’ second language learning experiences, particularly regarding learning motivation and in-
terest. The results of the study suggested that schools and teachers should explore creative ways to
integrate traditional and BYOD approaches.
© 2017 Elsevier Ltd. All rights reserved.

1. Introduction Report (New Media Consortium, 2015), Ke12 schools worldwide

Because of the rapid growth of reliable mobile technologies,
BYOD (bring your own device) has become a feasible instructional
strategy for promoting students’ active engagement during the
learning process (Nortcliffe & Middleton, 2013). In the educational
setting, the phrase “bring your own device” refers to students
bringing their mobile devices into classrooms for learning purposes
(Nelson, 2012). When students are asked to bring their own devices
to class, their mobile devices (e.g., smart phones) must be autho-
rized by the schools (Hockly, 2012). According to the 2015 Horizon
* Corresponding author.
E-mail address: pnchou@gm2.nutn.edu.tw (P.-N. Chou).
have begun adopting the BYOD policy as an innovative approach to
support traditional teaching practices.
Although the trend of BYOD continues to grow in popularity, the
practice has created a number of behavioral and security problems
on campus. First, students may use mobile devices to access inap-
propriate materials and websites; this, indirectly creates a class-
room management issue for instructors (Mittal, 2014). Second,
mobile devices that students bring into the classroom may pose a
security threat to school network systems (Dickerson, 2013).
Finally, because some students use more technologically advanced
(or “better”) mobile devices than other students, the BYOD method
contributes to learning inequality in classrooms (Hockly, 2012).
However, perceiving the learning potential of the BYOD approach,

http://dx.doi.org/10.1016/j.chb.2017.04.024
0747-5632/© 2017 Elsevier Ltd. All rights reserved.
64 P.-N. Chou et al. / Computers in Human Behavior 74 (2017) 63e71
educators express a willingness to address and overcome these
potential hurdles (The Journal, 2012).
This study investigated how a public junior high school
attempted to transform the learning process by implementing a
BYOD policy. Prior to the study, the research team and the school
administration collaborated to create a pilot test designed to assess
the feasibility of the technology infrastructure and instructional
planning. During this three-month preparation period, the research
team and school administrators discussed the logistics of the study,
considering the issues of classroom management, cyber security,
app (application) selection, and device hardware. A preliminary
analysis indicated that soliciting support from parents was a crucial
aspect of the BYOD adoption since parents are the ones that provide
their children with the required devices. On the basis of data ac-
quired during a pilot project, a formal experimental evaluation in
an English course was conducted to verify the effect of the BYOD
approach on student learning.
2. Literature review
2.1. Rationale for BYOD in the classroom
The rationale for implementing a BYOD policy can be explained
in economic terms. Although schools lack the necessary funds
required to purchase information technologies, they nevertheless
aim to achieve a 1:1 computing initiative (Hockly, 2012; Nelson,
2012; Song, 2014). In other words, most Ke12 schools expect to
establish a “robust access ration of one computer to one student”
(Bebell & O'Dwyer, 2010, p. 6) in classrooms because 1:1 computing
enables each student to experience the benefits of information
technologies on learning, but they lack the financial resources to
implement this plan. Because the BYOD approach allows students
to use their own mobile devices, it solves this economic/resource
problem. However, while previous studies have shown that the 1:1
computing initiative increases student engagement and improves
learning achievement in the classroom (Bebell & Kay, 2010; Looi
et al., 2011), further research is required to verify the educational
benefits of the BYOD approach.
2.2. Perceptions of BYOD in the classroom
According to a 2013 survey conducted by Bradford Networks
(Bradford Networks, 2013), the BYOD approach had a higher
acceptance rate at colleges and universities than it did at Ke12
learning institutions. The survey results revealed a connection be-
tween the BYOD approach and student in-class participation and
assignment completion. However, from a pedagogical perspective,
several teachers perceived BYOD as a barrier in the classroom. For
instance, O'Bannon and Thomas (2014) reported that Ke12 teach-
ers expressed negative views regarding the efficacy of the BYOD
approach; these teachers noted that class disruptions, cheating, and
cyber bulling (caused by BYOD) impeded the learning process.
O'Bannon and Thomas (2015) also surveyed pre-service Ke12
teachers: while acknowledging the same potential educational
barriers cited by the Ke12 teachers, the pre-service teachers
expressed support for mobile phone features in school-related
work. In the current study, to avoid potential classroom manage-
ment problems, the BYOD approach became a short-term formative
evaluation activity.
2.3. Learning outcome of BYOD integration into teaching practices
BYOD is a variant of “mobile learning” (Huang & Tsai, 2011).
Previous scholarly work on mobile learning has focused on school-
provided mobile devices (and not on BYOD). Only a few empirical
studies have investigated the practice of integrating BYOD into
pedagogical practices (Song, 2014). One report from the Cisco
Company showed that levels of learning achievement, particularly
math and science scores, were significantly increased for BYOD
students in one school district (The Journal, 2012). However, this
report did not implement a research procedure. In Song's (2014)
study, one group of students using their own mobile devices in a
science class was examined. The findings indicated that BYOD
adoption strengthened students' content knowledge and sup-
ported the student learning process in various ways. While Song
reported some positive outcomes, she (like the Cisco researchers)
failed to employ a formal experimental method whereby future
researchers could verify the results.
2.4. BYOD as an assessment tool
In some instructional cases, the BYOD teaching model serves as
a student response system (SRS) used for measuring student con-
tent knowledge or offering student practice. Traditionally, SRS is
equipped with many handheld remote controllers (or clickers) that
transmit the students’ answers to instructors (Caldwell, 2007).
Although previous studies have confirmed the learning benefits of
traditional SRS in different learning environments (Draper &
Brown, 2004; Stuart, Brown, & Draper, 2004), a recent study con-
ducted by Stowell (2015) did not find a noticeable difference be-
tween BYOD and SRS in improving student learning. Therefore, to
offer students a simpler learning experience, Wang (2015) sug-
gested that the BYOD approach could replace traditional SRS
assessment tools in schools.
2.5. Moving BYOD into MALL
MALL (mobile assisted language learning) refers to using mobile
devices to support language learning activities (Kukulska-Hulme &
Shield, 2008). Mobile devices have proven effective in teaching
vocabulary (Godwin-Jones, 2011). For example, Lu (2008) exam-
ined how students used the short message service on mobile
phones to learn vocabulary, and found that students using mobile
phones recognized more vocabulary words than their (paper-based
reading) counterparts. Wang and Smith (2013) investigated the
feasibility of developing students’ English skills through mobile
phones, and reported that students preferred using mobile phones
to access English reading and grammar materials. However,
because MALL related studies did not focus on the concept of BYOD,
more research is required to determine whether a combination of
BYOD and MALL may yield different learning outcomes.
3. Significance and rationale of the study and research
question
Plass and Jones (2005) proposed that integrating second-
language acquisition methods and multimedia technologies
would provide a useful instructional tool to facilitate student lan-
guage learning, particularly for vocabulary comprehension. In
Taiwan, English as a second language (ESL) is a core curriculum that
students must take in junior high school. Adopting BYOD teaching
practices may advance students' second language learning experi-
ences. In addition, the cognitive theory of multimedia learning
suggests that technologies enhance knowledge transfer from short-
term memory to long-term memory (Clark & Mayer, 2011). In the
current study, students employed smart phones to support their
language learning during the BYOD instruction. The instructional
use of mobile devices may enhance students' ability to learn En-
glish vocabulary, which will indirectly reinforce knowledge transfer
in memory. According to Ferreira, Klein, Freitas, and Schlemmer
 P.-N. Chou et al. / Computers in Human Behavior 74 (2017) 63e71
(2013), mobile learning can also serve as a motivation-
enhancement tool to stimulate students' learning attention. The
innovative BYOD approach may arouse students’ interest in
learning and inspire them to actively participate in the language
learning process.
On the basis of literature review, compared to the traditional
teaching method, past research (e.g. Lu, 2008) have reported that
students using mobile phones significantly improved their English
skills, particularly for vocabulary comprehension. In addition, pre-
vious studies (e.g. Caldwell, 2007) have also verified that students
in SRS teaching model outperformed the students in traditional
instruction. While combining the advantages of both mobile
learning and SRS adoption for student learning (Wang, 2015), the
BYOD instruction perhaps provides positive learning experiences
for student participants, which in turn may increase more content
knowledge than those in the traditional instruction. In this study,
paper-based assessment was a commonly used traditional teaching
method for learning activities (i.e. formative evaluation) in the
classroom. In contrast, the BYOD approach as a new assessment tool
was used for supporting language learning evaluation during class.
Since none of previous related studies were identified in the liter-
ature, whether those two assessment strategies may significantly
differ in student learning outcomes needs further investigation in
the current study.
This study employed an experimental design to investigate the
effect of the BYOD approach on student learning performance. To
minimize potential classroom management problems, the BYOD
approach in the study became a formative assessment tool during
class, where an application (app) called Socrative was used in
school-authorized smart phones. Junior high school students
brought their Android-based mobile devices to class and used them
in language evaluation activities. In order to gauge the students’
immediate responses and their long-term transfer of learning, the
researchers conducted formative, summative, and delayed sum-
mative evaluation. The study was designed to answer one key
question:
RQ: Does the BYOD instruction approach yield different learning
outcomes than the traditional (paper-based) instruction
approach among junior high school students?
4. Research method
4.1. Research design
This study adopted a quasi experimental pre-test and post-test
control group design to examine the effect of the BYOD approach
on language learning. The educational experiment lasted for four
weeks. The independent variable was the type of instructional
strategy (BYOD or traditional instruction). The dependent variable
was the student learning outcome in English (ascertained using
Table 1
Quasi-experimental study design.
Group Pre-test Experiment
Experimental group (Class A) O1 X1
Control group (Class B) O2 X2
X1: Students receiving BYOD instruction.
X2: Students receiving traditional instruction.
O1~Q2: Achievement test on English learning (pre-test).
O3~Q4: Summed scores on class quizzes (formative evaluation).
O5~Q6: Achievement test on English learning (summative evaluation).
O7~Q8: Achievement test on English learning (delayed summative evaluation).
65
formative, summative, and delayed summative evaluation). Table 1
presents the research design of this study.
Prior to the study, an achievement pre-test on English learning
was administered to measure the students’ knowledge of the
course content. During the implementation of the educational
experiment, two groups of students completed quizzes (formative
evaluation) following the class lessons. Upon the completion of the
experiment, the students received the same achievement test
(summative evaluation) with different item numbers. One month
after the summative evaluation was completed, another achieve-
ment test (delayed summative evaluation), comprised of different
item numbers, was administered to assess the long-term transfer of
learning in students.
In the experimental group, students used their smart phone
app (Socrative) to access required tests (Fig. 1). In contrast, stu-
dents in the control group only received sheets of test papers. Both
groups completed formative quizzes before the end of the class
sessions. After completing the quizzes, the students were given
the correct answers for each question from the app tool (experi-
mental group) or from the answer sheets provided by the teacher
(control group).
Before the implementation of the BYOD instruction (for the
experimental group), the researchers re-examined the classroom
infrastructure and smart phone hardware. The Wi-Fi connection
in the classroom was adjusted to a higher level to ensure
strong network communication. The smart phones that the stu-
dents brought to class were verified for specification re-
quirements. The instructor also implemented a learning rubric
for smart-phone use during class. In addition, the students were
instructed on how to use the Socrative app to complete the
formative tests.
To avoid potential threats to the internal validity of the experi-
mental design (Creswell, 2009), several experimental controls were
used during the implementation of the study. Table 2 reports the
experimental controls used in the study.
4.2. Research instrument
4.2.1. Quiz (formative evaluation)
10 min before the completion of each class, a quiz was
administered to the students in both the experimental and
the control groups. Each quiz contained 6 multiple-choice
questions and 4 short-answer questions, which directly
related to the class lessons. During the course of the experiment,
student participants completed 6 quizzes developed by the
instructor. The formative result was the summed score of the 6
quizzes.
Post-test
O3 O5 O7
O4 O6 O8
Fig. 1. Students using their smart phone app to access quizzes.
66 P.-N. Chou et al. / Computers in Human Behavior 74 (2017) 63e71

Table 2
Strategies for experimental control in the study.

Control Factor Description

1. Class instructor The same instructor taught both English classes.

2. Class time Both the experimental and the control groups received the same class time (45 min per class).
3. Learning contents The same learning material was employed for language instruction.
4. Class setting All learning activities occurred in similar physical classrooms.
5. Test implementation The pre-tests, quizzes, and post-tests were administered on the same day of the week. The pre-tests and
post-tests were not returned to the students.
6. Teaching method The instructor used the same teaching method to impart English knowledge in both the experimental and the control groups.
7. Initial behavior The pre-test results and the students' learning achievements in the previous academic year were listed as covariance
variables to ascertain the students' initial learning behaviors.

4.2.2. Learning achievement test (summative evaluation and
delayed summative evaluation)
A learning achievement test was developed to measure the
students’ understanding of English vocabulary and grammar for a
specific lesson. The test was comprised of 37 different types of
questions, including multiple-choice, missing-word, figure-
answering, word-matching, and paragraph-reading. The score
range of the test was between 0 and 67. Higher scores represent the
higher learning achievement in English. Fig. 2 shows a test item
from the figure-answering section.
To ensure the validity of the test questions, two professors of
English, one professor of education, and five English teachers in
junior high schools collaboratively reviewed the content of the test.
Subsequently, modified content of the test was administered to 25
ninth graders who were already familiar with the test content.
During this stage, some test questions were deleted because the
discrimination index (in the item discrimination analysis) was
below 0.25 and the difficulty index (in the item difficulty analysis)
was below 0.2 (Aiken & Groth-Marnat, 2006). In addition, the result
of the KR-21 reliability test revealed that the reliability coefficient
was 0.94.
4.2.3. Self-report learning questionnaire
A self-report learning questionnaire was developed to assess the
students' learning experiences regarding BYOD teaching practices.
The questionnaire adopted a 5-point Likert scale design and con-
tained 10 survey questions. Prior to the study, two professors of
English and one professor of education collaboratively reviewed
the questionnaire, verifying the accuracy and validity of the ques-
tions. The final edition of the questionnaire was administered to 15
junior high school students who participated in a BYOD pilot study.
The reliability test was performed to assess the reliability of the
questionnaire. The reliability coefficient (Cronbach's alpha) was
0.89.
interview consisted of a casual conversation between students and
the researchers. This interview lasted approximately 15 min.
4.2.5. Socrative
Socrative is a web-based assessment tool that allows teachers to
create multiple-choice, true and false, and short-answer questions.
Upon the completion of test creation, the teachers share unique
room number of quizzes to students, and subsequently launch
quizzes by their desktop or laptop computers. Through the Socra-
tive app (Fig. 3), students use their smart phones to answer ques-
tions by clicking different buttons in a given time-slot. In the study,
the time limit for each quiz was set at 10 min. Fig. 4 shows a
question listed in the Socrative app.
4.3. Participants
This study used the convenience sampling technique to select
two classes at a public junior high school in Taiwan. 46 eighth-
grade students (Class A: 24, Class B: 22) participated in the four
week educational experiment. The male to female ratio (almost
1:1) in the two classes created a gender balance percentage. Table 3
presents a profile of the research participants.
4.4. Instructional scenario
This study focused on one course entitled English Learning for
Eighth Graders. The learning unit was “Lesson 7: How can we get to

4.2.4. Informal interview

The informal interview supplemented the quantitative-based
survey. The qualitative data also enabled researchers to obtain in-
formation regarding BYOD adoption in the classroom. The informal

Fig. 2. A test item from the figure-answering section. Fig. 3. A screenshot of the Socrative app.
 P.-N. Chou et al. / Computers in Human Behavior 74 (2017) 63e71 67

Table 3
Profile of the research participant.

Group Class n Male Female

Experimental Class A 24 13 11
Control Class B 22 11 11

Table 4
ANCOVA results of formative evaluation.

Source SS df MS F Sig.

Covariance
1. Pre-test 1930.08 1 1930.08 40.75 0.00
2. Prior English learning achievement 558.17 1 558.17 11.79 0.00
Instructional Method 1046.10 1 1046.10 9.83 0.00**
Errors 4364.91 41 106.46

Note: Experimental (M ¼ 35.39/S.D. ¼ 2.11), Control (M ¼ 44.98/S.D. ¼ 2.21).

**
p < 0.01, Control > Experimental.
Bold row represents independent variable.

6. Results of summative evaluation

The ANCOVA results of the summative evaluation (learning

achievement) and the t-test results between the pre-test and the
post-test are presented in Tables 5 and 6. The findings indicate that
no significant difference (F ¼ 2, p > 0.1) on summative evaluation
Fig. 4. A question listed in the Socrative app.
was found for the two groups of students. However, the results of
the t-test indicated significant differences between the pre-test and
the post-test for the experimental (t ¼ 3.41, p < 0.01) and control
Chih-Kan Tower” in which several new vocabulary words and groups (t ¼ 6.15, p < 0.01). Therefore, regardless of the instructional
grammar rules were taught to students. In the educational exper- model, the two groups of students both significantly enhanced
iment, students in the experimental and control groups received six their learning performances.
classes (45 min per class) of instruction. During the classes, 5 min
was used for test-preparation and 10 min was reserved for 6.1. Results of delayed summative evaluation
formative evaluation (BYOD or paper-based tests).
The ANCOVA results of the delayed summative evaluation and
4.5. Data analysis the t-test results of the two post-tests are listed in Tables 7 and 8.
The study found that no significant difference (F ¼ 0.003, p > 0.1)
In the study, t-test and analysis of covariance (ANCOVA) were between the experimental and control groups in the summative
employed to evaluate the effect of the independent variable on the evaluation stage. However, the results of the t-test indicated a
students' learning achievements. The Pearson correlation was used significant discrepancy (t ¼ 3.42, p < 0.01) between the post-tests of
to compare the relationship among dependent variables. The the two test groups.
rationale for using ANCOVA was to exclude the influence of stu-
dents' prior knowledge in English learning as covariance variables
6.2. The change among pre-test, post-test, and delayed post-test
on experimental results (Tabachnick & Fidell, 2007). Traditionally,
the pre-test often served as a covariance variable to remove the
Because the same questions were asked in the pre-test, the post-
effect of the prior knowledge in the experimental study (Creswell,
2009). However, Chou, Chang, and Lu (2015) suggested that stu-
dents' prior learning achievement might be a potential covariance Table 5
variable. To rigorously control the effect of students' prior knowl- ANCOVA results of summative evaluation.
edge, two covariance variables were used in ANCOVA: students’
Source SS df MS F Sig.
pre-test scores and English learning achievement in the previous
Covariance
academic year. If a significant value was identified in ANCOVA, a
1. Pre-test 2433.77 1 2433.77 44.00 0.00
Scheffe method was performed to ascertain the post-comparison 2. Prior English learning achievement 306.12 1 306.12 5.36 0.02
difference. Instructional Method 110.65 1 110.65 2.00 0.17
Errors 2267.64 41 55.31
5. Results Note: Experimental (M ¼ 38.14/S.D. ¼ 1.52), Control (M ¼ 41.26/S.D. ¼ 1.59).
Bold row represents independent variable.
5.1. Results of formative evaluation
Table 6
Table 4 reports the ANCOVA results of the formative evaluation t-test Results between Pre-test and Post-test.
(class quizzes). The findings showed that a significant difference
Group Post-test ePre-test (M) t Sig.
existed on formative evaluation (F ¼ 9.83, p < 0.01) for two groups
of students. The post-comparison results also indicated that stu- Experimental 5.88 3.41 0.00**
dents in the control group outperformed the students in the Control 9.05 6.15 0.00**

experimental group (mean difference ¼ 9.59, p < 0.01). **

p < 0.01.
68 P.-N. Chou et al. / Computers in Human Behavior 74 (2017) 63e71

Table 7 Table 9
ANCOVA results of delayed summative evaluation. Results of self-report learning questionnaire.

Source SS df MS F Sig. Item S.D. Mean

Covariance 1. BYOD aroused my learning interest 0.15 4.25

1. Pre-test 1930.08 1 1930.08 40.75 0.00 2. BYOD enhanced my learning attention 0.15 4.21
2. Prior English learning achievement 558.17 1 558.17 11.79 0.00 3. BYOD improved my learning attitude toward English 0.18 4.00
Instructional Method 0.12 1 0.12 0.003 0.96 4. BYOD created a joyful learning atmosphere 0.13 4.58
Errors 1847.12 41 47.36 5. BYOD supported my English learning 0.17 4.00
6. BYOD facilitated learning interaction 0.16 4.04
Note: Experimental (M ¼ 37.38/S.D. ¼ 1.44), Control (M ¼ 37.49/S.D. ¼ 1.51).
7. BYOD catered to different levels of English learners 0.22 3.92
Bold row represents independent variable.
8. BYOD enhanced my English learning achievement 0.17 3.54
9. Overall, I was satisfied with BYOD approach 0.16 4.29
10. I prefer to use BYOD in the future 0.18 4.33
Table 8 All 0.17 4.12
t-test Results between Two Post-tests.

Group Delayed-Post-test ePost-test (M) t Sig.

Experimental 0.91 0.44 0.6 Table 10

Control 3.71 3.42 0.00** Correlations between survey results and post-test by gender.
**
p < 0.01. Gender n r Sig.

Male 13 0.30 0.32

test, and the delayed post-test (but using different item numbers), a
change occurred in the learning outcomes of the two study groups,
as identified in Fig. 5. The findings indicated that the learning
performance for students in the control group fluctuated more
considerably than their counterparts (experimental group). How-
ever, regarding long-term transfer of learning (from the post-test to
the delayed post-test), students in the experimental group out-
performed the students in the control group.
6.3. Survey results
Tables 9 and 10 summarize the quantitative results of the self-
report learning questionnaires and the correlations between the
survey results and the post-test by gender. Regarding survey items,
only items 7 (3.92) and 8 (3.54) were below 4. The mean score in
each survey item registered at a high level (more than 4.00),
revealing high learning satisfaction for the BYOD approach. How-
ever, a negative relationship was observed (r ¼ 0.67, p < 0.01)
between female students’ self-report learning scores and learning
achievement. Female students with higher-level learning achieve-
ment tended to yield lower scores in the self-report learning
questionnaire.
Female 11 0.67 0.02*
*
p < 0.05.
6.4. Informal interview results
Table 11 presents the results of the informal interview con-
ducted with students in the experimental group. The positive
comments from 1-1 to 1e6 regarding BYOD adoption were
consistent with survey results. Regarding the negative comments:
the participants reported, problems related to mobile phone
hardware (2-2 and 2e4), unique feature of BYOD assessment (2e3),
and personal attitudes toward BYOD instruction (2e1, 2e5, 2e6,
and 2e7). The higher-achievement female students listed the
specific problems they encountered while taking the BYOD test:
these problems were related to hardware (or mobile-specific)
issues.
7. Discussion
In the formative evaluation segment of the study, students in
the traditional instruction class performed significantly better than
their counterparts in the BYOD instruction class. From a statistical

45

40

35

30

25
Experimental
20
Control
15

10

0
Pre-test Post-test Delayed Post-test

Fig. 5. The change among pre-test, post-test, and delayed post-test. Note: one month elapsed between the pre-test and the post-test, and between the post-test and the delayed
post-test.
 P.-N. Chou et al. / Computers in Human Behavior 74 (2017) 63e71 69

Table 11
Informal interview results.

Type Representative Quotes

1. Positive 1-1 “BYOD was fun for learning.”

1-2 “BYOD allowed me to pay more attention to class activity.”
1-3 “BYOD was an innovative and creative teaching style.”
1-4 “I never thought the mobile phone could support my learning.”
1-5 “I wished that other courses could follow BYOD model.”
1-6 “During BYOD learning process, I was more curious to English learning”
2. Negative 2-1. “Compared with traditional instruction, BYOD adoption required more time and energy to prepare quizzes.”
a
2-2. “Due to the level of hardware, my mobile phone sometimes could not perform very fast, which directly influenced the pace of quiz-taking.”
2-3. “Unlike the paper-based test, when I completed the quizzes in the BYOD model, I could not go back to previous items to change answers.”a
2-4. “Because of the small screen in the mobile phone, I often pressed the wrong buttons to answer the questions.” a
2-5. “Taking paper-based quizzes was much easier than BYOD approach.”
2-6. “I was not sure if BYOD could enhance my learning achievement.”
2-7. “Sometimes, I could not complete BYOD assessment within 10 min”
a
Higher-achievement female students' complaints.

perspective, the students that participated in the traditional in-
struction class registered higher test scores and exhibited a more
complete understanding of the course material than the students in
the BYOD class. However, upon reexamining the test scores, the
researchers noticed a number of spelling errors (see Fig. 6) and
missing answers on 6 separate BYOD quizzes. This result was
consistent with Stowell's (2015) study of smart phone test takers.
As delineated in Table 11, these results can be explained by the
BYOD students unfamiliarity with app interface system (causing
them to press wrong buttons) or hardware issues (causing a delay
in the testing process), which in turn influenced the students'
learning performances.
In the summative evaluation (post-test), the traditional in-
struction students outperformed their BYOD counterparts; yet, the
data gathered from this segment of the test does not indicate a
significant quantifiable difference between the two test groups.
From a learning improvement perspective, the students in the
traditional instruction class learned more than students in the
BYOD instruction class. Paper-based tests proved more effective in
reflecting student English learning than the BYOD tests. In addition,
a slower learning growth rate was identified in the BYOD instruc-
tion group. However, because previous studies did not employ an
experimental design to evaluate the BYOD instruction for Ke12
students (e.g., The Journal, 2012; Song, 2014), the results of this
study require further examination.
The delayed summative evaluation (delayed post-test),
which we administered one month after the summative evalu-
ation (post-test), did not reveal a significant difference in the
learning outcomes achieved by the two instructional approaches.
In other words, both the BYOD and the traditional instruction
produced nearly identical results regarding the long-term
transfer of learning. However, a comparison between the post-
test and the delayed post-test yielded a significant result: the
learning retention for students in the BYOD instruction surpassed
the learning retention of the traditional instruction students.
Only a marginal decrease from the post-test to the delayed
post-test was identified for students in the BYOD instruction.
This finding can be attributed to the novelty effect (Burke &
James, 2008) and the multimedia effect (Clark & Mayer, 2011;
Plass & Jones, 2005): two theories that explain how a
technological tool (smart phone app), stimulates students to
actively engage in language learning, leading to better learning
retention.
Although students in the BYOD group did not significantly
improve their language learning, their attitude toward BYOD
adoption in the classroom was extremely positive. Students
noted that the BYOD integration into language learning created a
joyful learning environment. Subsequently, a higher level of cu-
riosity drove students to pay more attention to the learning ac-
tivity. Students also expressed a desire to take BYOD courses,
preferring this model to traditional instruction. These results
supported Song (2014), Wang and Smith (2013), and Wang
(2015): studies that identified positive learning experiences for
mobile device students engaging in educational activities. How-
ever, because students only focused on perceptions of BYOD use
experiences, they often expressed uncertainty regarding the ul-
timate efficacy of the BYOD instructional approach in enhancing
learning achievement. In addition, some students commented
that extra effort was required for BYOD test preparation.
Regarding higher-achievement female students, their lower
scores in the self-report learning questionnaires can be explained
by the qualitative results. These female students expressed
dissatisfaction with the app features and phone hardware, stat-
ing that these mobile-related issues negatively affected their
quiz-taking performance.

8. Conclusion

8.1. Contribution of the study

Fig. 6. Spelling errors on BYOD quizzes. This study investigated the effect of the BYOD instruction
70 P.-N. Chou et al. / Computers in Human Behavior 74 (2017) 63e71
approach on student language learning. The results of the sta-
tistical analysis showed that a significant difference was only
identified in the formative evaluation between the traditional
and the BYOD instruction. Two instructional approaches yielded
similar learning outcomes in the summative and delayed sum-
mative evaluation. However, regarding learning retention, the
BYOD instruction students outperformed their traditional
counterparts. In addition, the results of the self-report learning
questionnaires and the informal interviews indicated that
most of the BYOD participants expressed positive learning ex-
periences, particularly in relation to their motivation and interest
levels.
8.2. Research limitations and future works
Because a unique experimental design was adopted to examine
the BYOD approach, this present study may be difficult to replicate
in the future. Further studies examining the BYOD approach
should consider the research limitations inherent in this study.
First, to minimize potential classroom management problems, the
BYOD instruction in this study was only used for short-term
formative evaluation during limited class time. Allowing stu-
dents to use their smart phones for the entire class period (and for
all learning activities) may yield different learning outcomes.
Second, for teaching and learning purposes, the Socrative app
used in this study was fully appropriate to the instructional sce-
nario; however, future studies may employ other available
assessment apps (e.g., Kahoot) to obtain student learning perfor-
mance results. Different learning apps may create a different
learning experience and yield different learning outcome results.
Third, because the study was conducted in a short period (four
weeks), the findings might not provide an overall picture on long-
term use of the BYOD teaching model. Future studies may extend
the study duration to evaluate the potency of the novelty effect or
potential long-term behavioral problems. Finally, the mobile de-
vices students brought into the classroom in this study were
Android-based smart phones. The small screen design on smart
phones may adversely affect students' app engagement and
knowledge acquisition, leading to a slower pace of quiz taking.
Future studies may attempt to use another mobile device, such as
a tablet computer, to advance the students’ language learning
experience.
8.3. Implications for teaching practice
Although this study was conducted at a junior high school, the
results of the study may be used to inform instructors teaching
other educational levels. First, because of the uneven pace of app
operation in the BYOD class, BYOD students misspelled their test
answers or failed to answer test questions. The BYOD-specific test
errors may serve as an analysis tool to better understand students’
in-class learning experiences. Second, students using mobile de-
vices for learning purposes may also attempt to use these devices to
access non-learning websites, such as social networking or gaming
sites. In addition, a more sophisticated and comprehensive
instructional rubric (specifically related to mobile device learning)
needs to be developed in order to make the learning experience
easier and more fluent for BYOD students: this could dramatically
improve BYOD learning outcomes. Third, schools that plan to
implement the BYOD policy need to motivate their students to
embrace this new instructional approach: providing a syllabus or a
lecture wherein they delineate the benefits of (and the rationale
for) BYOD instruction. Finally, because BYOD instruction and
traditional instruction both support student learning, schools
should explore creative ways to integrate these two effective
instructional approaches.
9. Statements on open data, ethics and conflict of interest
a. The data is kept by the researchers and can be accessed by
contacting with the researchers.
b. The research approval from school administration officers and
parents was obtained at the junior high school where the study
was carried out. Once the assessment data was collected, all
indenters were removed.
c. There is no conflict of interest.
References
Aiken, L. R., & Groth-Marnat, G. (2006). Psychological testing and assessment (12th
ed.). Boston, MA: Allyn and Bacon.
Bebell, D., & Kay, R. (2010). One to one computing: A summary of the quantitative
results from the berkshire wireless learning initiative. Journal of Technology,
Learning, and Assessment, 9(1). Retrieved from Wired http://www.jtla.org.
Bebell, D., & O'Dwyer, L. M. (2010). Educational outcomes and research from 1:1
computing settings. Journal of Technology, Learning, and Assessment, 9(1).
Retrieved from Wired http://www.jtla.org.
Bradford Networks. (2013). The impact of BYOD in education. Retrieved from Wired
http://th-ebooks.s3.amazonaws.com/The_Impact_of_BYOD_in_Education.pdf.
Burke, L. A., & James, K. E. (2008). PowerPoint-based lectures in business education:
An empirical investigation of student-perceived novelty and effectiveness.
Business Communication Quarterly, 71(3), 277e296.
Caldwell, J. E. (2007). Clickers in the large classroom: Current research and best-
practice tips. CBE Life Sciences Education, 6(1), 9e20.
Chou, P.-N., Chang, C.-C., & Lu, P.-F. (2015). Prezi versus PowerPoint: The effects of
varied digital presentation tools on students' learning performance. Computers
& Education, 91, 73e82.
Clark, R. C., & Mayer, R. E. (2011). E-Learning and the science of instruction (3rd ed.).
San Francisco, CA: Pfeiffer.
Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods
approaches (3rd ed.). Thousand Oaks, CA: Sage.
Dickerson, M. G. (2013). Bring your own device (BYOD) in K-12 classroom: A discourse
analysis on the impact of BYOD in teaching and student learning. Retrieved from
Wired http://www.aace.org/conf/site/submission//uploads/SITE2013/paper_
3053_38862.doc.
Draper, S. W., & Brown, M. I. (2004). Increasing interactivity in lectures using an
electronic voting system. Journal of Computer Assisted Learning, 20, 81e94.
Ferreira, J. B., Klein, A. Z., Freitas, A., & Schlemmer, E. (2013). Mobile learning:
Definition, uses and challenges. In L. A. Wanket, & P. Blessinger (Eds.), Increasing
student engagement and retention using mobile applications: Smartphones (pp.
47e82). skype and texting technologies (Published Online: Emerald Insight).
Godwin-Jones, R. (2011). Emerging technologies: Mobile apps for language
learning. Language Learning & Technology, 15(2), 2e11.
Hockly, N. (2012). Tech-savvy teaching: BYOD. Modern English Teacher, 21(4), 44e45.
Huang, G.-J., & Tsai, C.-C. (2011). Research trends in mobile and ubiquitous learning:
A review of publications in selected journals from 2001 to 2010. British Journal
of Educational Technology, 42(4), E65eE70.
Kukulska-Hulme, A., & Shield, L. (2008). An overview of mobile assisted language
learning: From content delivery to supported collaboration and interaction.
ReCALL, 20(3), 271e289.
Looi, C. K., Zhang, B., Chen, W., et al. (2011). 1:1 mobile inquiry learning experience
for primary science students: A study of learning effectiveness. Journal of
Computer Assisted Learning, 27(3), 269e287.
Lu, M. (2008). Effectiveness of vocabulary learning via mobile phone. Journal of
Computer Assisted Learning, 24, 515e525.
Mittal, R. (2014). Use of cell phones in classroom teaching: Boon or bane. Interna-
tional Journal of Research, 1, 1253e1257.
Nelson, D. (2012). BYOD: An opportunity schools cannot afford to miss. Internet@
Schools, 19(5), 12e15.
New Media Consortium. (2015). 2015 Horizon report (Ke12 version). Retrieved from
Wired http://www.nmc.org/publication/nmc-horizon-report-2015-k-12-editio
n/.
Nortcliffe, A., & Middleton, A. (2013). The innovative use of personal smart devices
by students to support their learning. In L. A. Wanket, & P. Blessinger (Eds.),
Increasing student engagement and retention using mobile applications: Smart-
phones (pp. 175e208). Published Online: Emerald Insight: skype and texting
technologies.
O'Bannon, B. W., & Thomas, K. (2014). Teacher perceptions of using mobile phones
in the classroom: Age matters! Computers & Education, 74, 15e25.
O'Bannon, B. W., & Thomas, K. (2015). Mobile phones in the classroom: Preservice
teachers answer the call. Computers & Education, 85, 110e122.
Plass, J. L., & Jones, L. C. (2005). Multimedia learning in second language acquisition.
In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp.
467e488). New York: Cambridge.
Song, Y. (2014). Bring your own device for seamless science inquiry in a primary
 P.-N. Chou et al. / Computers in Human Behavior 74 (2017) 63e71 71

school. Computers & Education, 74, 50e60.
Stowell, J. R. (2015). Use of clickers vs. mobile devices for classroom polling. Com-
puters & Education, 82, 329e334.
Stuart, S. A., Brown, M. I., & Draper, S. W. (2004). Using an electronic voting system
in logic lectures: One practitioner's application. Journal of Computer Assisted
Learning, 20, 95e102.
Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Boston,
MA: Allyn & Bacon.
The Journal. (2012). Schools plug into BYOD: Mobile devices transform learning at Katy
ISD. Retrieved from Wired http://thejournal.com/whitepapers/2012/cisco_12a/
byod/schools-byod-mobile-devices.aspx?sc_lang¼en.
Wang, A. I. (2015). The wear out effect of a game-based student response system.
Computers & Education, 82, 217e227.
Wang, S., & Smith, S. (2013). Reading and grammar learning through mobile phones.
Language Learning & Technology, 17(3), 117e134.