Running Head: Annotated Bibliography

Using Alice to Integrate Computer Programming and Science: An Annotated Bibliography

Erica Carrillo
Educational Research- ITEC 7470
Kennesaw State University
Spring 2015

Annotated Bibliography

BISHOP-CLARK, C. V. (2006). PROGRAMMING IN PAIRS WITH ALICE TO

IMPROVE CONFIDENCE, ENJOYMENT, AND ACHIEVEMENT. Journal Of
Educational Computing Research, 34(2), 213-228.

The authors conducted an experiment to investigate the impact of using a graphics

programming environment (Alice) and pair-programming to increase confidence,
enjoyment, and achievement. There were two objectives that included getting
students excited about computing and programming while the other objective was
that students would have better understanding of basic programming principles.
Alice provides students with image-rich programming language that lends itself to
working in pairs to create positive attitudes about learning programming. The
authors state, Pair-programming is a practice in which two programmers work
side-by-side at one computer, continuously collaborating to create a single
solution (Bishop-Clark, C.V., 2006, p.214). The research method included 64
participants enrolled in an undergraduate introductory computer science course at
a medium-sized, mid-western university. Students were divided into two groups:
those who worked with Alice in pairs and those who worked with Alice
individually. Two of the five sections completed the project individually and three
completed the project in pairs. Data was collected from five sections of the same
course on two different campuses at the same university. There were a variety of
disciplines and academic backgrounds. The procedure included an introduction to
a one-week (3 contact hours) unit in programming using Alice in week six of a

Annotated Bibliography

semester-long course. The procedure also included all students taking an online
questionnaire that took around 15 minutes to complete and a pre and post- test
that was a five-point scale that evaluated their confidence and enjoyment with
programming. The only difference between the two groups was that one group
had to complete the tutorial and assignment individually while the other group
had to complete the tutorial and assignment in pairs. Next, students were given a
brief overview of programming terminology, and spent the rest of class learning
the Alice programming language and environment via three on-line guided stepby-step tutorials included with the Alice package. At the second session, students
were given an assignment to create a project on their own with set requirements
designed to expose students to fundamental constructs. A paper tutorial was used
to help with basic actions required to create the new world, populate it with
objects, and animate them. The results indicate that students in both groups did
have more confidence in programming after the one-week session of Alice
programming than before. The results indicate that those working in pairs showed
more growth in confidence and enjoyment, as well as, improved knowledge of
programming concepts at the end of the one-week session. This finding suggests,
the combination of both Alice and pair-programming may help create an
environment that students very much enjoy (Bishop-Clark, C.V., 2006, p.224).
Some of the weaknesses of the study are that it was a fairly restricted sample used
which limits the possibility of generalizing the findings. Another weakness is that
the authors did not capture the qualitative differences of working with Alice both
individually and in pairs. The on-line multiple choice quiz was limiting because

Annotated Bibliography

observation and writing programs can be used to provide more insight in

understanding concepts of individuals and pairs. The authors also did not measure
the quality of projects that the two groups created, and it may be the groups in
pairs created projects that were more innovative and exciting. It might make a
significant difference in data if quality was measured. In comparison with the
other articles, the authors did provide a clear procedure, but data was very
subjective. As the authors stated, the data might have more of an impact if
projects were evaluated based on quality of projects versus completion and
questionnaire.

http://proxy.kennesaw.edu/login?
url=http://search.ebscohost.com/login.aspx?
direct=true&db=cph&AN=22606939&site=eds-live&scope=site

Bishop-Clark, C. V. (2007). A Quantitative and Qualitative Investigation of Using Alice

Programming to Improve Confidence, Enjoyment and Achievement Among NonMajors. Journal Of Educational Computing Research, 37(2), 193-207.

The authors, in this investigation, use Alice programming language to study from
a qualitative and quantitative perspective. Participants included 154 students
where 95 participated in the spring and 58 in the fall. In total the data was
collected from nine different sections (four in the fall and five in the spring) with
three different instructors on two different campuses. Although three instructors

Annotated Bibliography

were included in the study, all instructors used identical handouts, assignments,
and class notes. Students were introduced to a five-session unit in programming
using Alice in the third week of the semester. The first session included the
questionnaire, brief introduction to programming terminology, and spent the
remainder of the session learning Alice programming language via three on-line
guided tutorials included with the Alice package. The second session, students
were given an assignment to create using multiple objects and actions in
conjunction with a paper tutorial to allow independent individual work. The third
session was did not include Alice, but was a brief introduction to algorithms. It
included guided notes, the basics of algorithms, and an in-class lab. The fourth
and fifth class, the focus was Alice where students had to class sessions to
complete an in-class lab where students used storytelling to create a story from
predefined story titles. The quantitative measure included a pre and post-test
about programming experience and knowledge using a 5-point scale, as well as,
an 11-item multiple choice quiz that largely measured the students understanding
of programming concepts. Qualitative measures included focus groups with three
of the classes (one from each instructor) that were held immediately after the
Alice programming unit and lasted thirty minutes. Spring semester students
completed an essay reflecting on their Alice experience and this was a graded
assignment to motivate students to complete it. Essays were coded for the
variables of the three hypotheses: confidence and lack of confidence, enjoyment
and disliking programming process, and knowledge/understanding. The results
indicate that in both spring and fall, students were more confident and had more

Annotated Bibliography

enjoyment in programming after their experience using Alice, more so during

their fall experience. Students also performed significantly better on the concepts
post-test after their Alice experience. There are weaknesses that include
extraneous variables such as students talked to each other, some sections worked
in pairs, different sections had different student questions so all students did not
hear the exact same explanations, and students had varying motivations and
expectations. Also, the fact that subjects were in the same course with primarily
freshman and sophomore non-majors limits the ability to generalize findings. The
multiple-choice quiz also did not allow much insight of differences in
understanding of concepts before and after the Alice unit. According to the
authors, the primary strength of the study is that it provided a comprehensive
look at using Alice in a short unit for non programmers (Bishop-Clark, 2007,
p.205). In comparison to the 2006 article presented by the same group of authors,
this article seems to address some of the issues posed in the previous article
concerning all the subjective data. This study has a content knowledge quiz where
the previous primarily focused on completion versus quality.
http://proxy.kennesaw.edu/login?url=http://search.ebscohost.com/login.aspx?
direct=true&db=cph&AN=26369493&site=eds-live&scope=site

Chang, Chih-Kai. (2014). Effects of Using Alice and Scratch in an Introductory

Programming Course for Corrective Instruction. Journal of Educational
Computing Research, 51 (2), 185-204.

Annotated Bibliography

The author conducted a study comparing the effects of two introductory computer
programs, Alice and Scratch. The research study explored the relationship
between learning anxiety, learning engagement, and learning playfulness which
corrective instruction. The research method in this study consisted of a
purposeful sampling that targeted subjects recruited from university freshman in
the Information Technology course in southern part of Taiwan. The Scratch group
consisted of 45 freshman in an Introduction to Computer Science Course while
the Alice group was reduced to 35 people. Both Scratch and Alice were
introduced as supplementary information, and they had 3 hours to play with it in
the 10th week of the semester. After midterms students were asked to take part in a
remedial program for 3 hours. Those ranked in the bottom quarter were assigned
the coursework and participated in a posttest. In the final-term test, questionnaires
on learning anxiety and learning engagement were distributed to the students
using a 5 point Likert-Scale, and informal interviews were conducted after
corrective instructions. Data was collected and compared, and the author
concluded that Alice is enjoyable, pleasant, and fun. Overall, this study found
that both Alice and Scratch could be used in corrective instruction in a way that is
playful and effective, which ultimately improves students learning performances
and ought to be considered by programming instructors (Chang, 2014, p. 202).
The author also found that Scratch VPL could be used for increasing
programming ability where Alice VPL is a great tool for grabbing the interest of
learners in producing creative writing in program design when implementing a
remedial program. Some limitations and issues do exist with this study such as

Annotated Bibliography

low achieving students studying other materials to improve achievement. One

suggestion that the author has is that instructors should, adapt the blended
learning of combining project-oriented or problem-based learning strategies with
VPL to focus on computer programming learning on practical task development
(Chang, 2014, p. 202). A final suggestion, and this is one Ive read in other
research articles and studies, is to have students work with a peer sharing in the
creative process to increase growth. Compared to other articles, this article did
offer a clear comparison of Scratch and Alice, but other research studies offered
more data-driven conclusions.
http://proxy.kennesaw.edu/login?
url=http://search.ebscohost.com/login.aspx?
direct=true&db=cph&AN=99408064&site=eds-live&scope=site

Dann, W., & Cooper, S. (2009). Education Alice 3: Concrete to Abstract.

Communications Of The ACM, 52(8), 27-29.

The authors address the need for women, Hispanics, and other
traditional underrepresented groups to enroll and gain in
computer programming courses. There is a need for innovative
teaching and learning strategies to engage students, and Alice 3
is a perfect tool to transition from concrete to abstract. There are
basic skills that students must know, and that is the concrete.
The second issue posed in the article is the importance of

Annotated Bibliography

context. Through Alice 3, students use animation, storytelling,

and game authorship as a context to which to entice students
toward computing. The third issue is the most challenging
because students must transition from concrete to more abstract
computing concepts. All three of these issues are the driving
force behind the creation of Alice 3. To investigate the
effectiveness of using the Alice 2 and Learning to Program with
Alice, a pilot study was conducted targeting pre-CS1 audience.
Students were divided into two groups: control group not
exposed to Alice and a group exposed to Alice. It was found that
at-risk computing majors who were exposed to Alice were nearly
twice as likely to continue to CS2 as their control-group peers. A
limitation to this study is that too few control-group students
remained in CS2 courses to meaningfully measure how they
performed in later courses as compared to those exposed to
Alice. The Alice 3 will provide teachers with the choice regarding
how close to Java their students will begin. Alice 3 also provides
two galleries with a richer set of animations. The final additional
feature would a transition option that allows students to open
the hood and type Java code, edit, and run it in the NetBeans
text-based Java integrated development environments. The
article explained the development of Alice 3, and the issues it

Annotated Bibliography

10

addresses. Although the authors address a pilot study, this article

was more about how and why Alice 3 was developed.
http://proxy.kennesaw.edu/login?url=http://search.ebscohost.com/login.aspx?
direct=true&db=bth&AN=43479956&site=eds-live&scope=site

Ernst, J. V., & Clark, A. C. (2012). Fundamental Computer Science Conceptual

Understandings for High School Students Using Original Computer Game
Design. Journal Of STEM Education: Innovations & Research, 13(5), 40.

The authors designed a research study to investigate and identify the impacts, if
any, that using gaming as an instructional approach has on the development of
computer science competency. Past research has indicated that access to learning
tools and integrating them into the course work engages and enhances classroom
discovery. The supplemental unit included viewing content demonstrations,
complete tutorials to learn about gaming and programming, independently
creating an original game using design brief research-based format and rubric, and
participate in cognitive and performance-based testing for feedback. The entire
sequence was designed to take approximately 8 hours to complete, and it was selfcontained. Once completed teachers submitted student work for research and
evaluation. The participants included three CTE instructors and classes selected
and trained for the first pilot testing of materials, and the participant data for the

Annotated Bibliography

11

sample size was less than 50. Some limitations to this study were that the
heightened course requirements, finding time to implement the supplement unit,
and the freeware presented technical issues. The authors found that, the use of
gaming as an instructional tool supports computer science competency
attainment (Ernst, J. V., & Clark, A. C., 2012, p. 40). The study supports gaming
as a pedagogical process and the need for technology literacy. The authors
determined more research is needed on the use of gaming as a pedagogical tool in
STEM disciplines as professionals in education prepare 21st century learners. In
comparison to other studies, I found this article was very difficult to understand. I
had difficult locating the specific steps of the methodology including subject
groups. The article organization was sporadic, and I couldnt really identify data
results and implications.
http://proxy.kennesaw.edu/login?
url=http://search.ebscohost.com/login.aspx?
direct=true&db=aqh&AN=89166317&site=eds-live&scope=site

Hoffman, B., & Nadelson, L. (2010). Motivational engagement and video gaming: a
mixed methods study. Educational Technology Research & Development, 58(3),
245-270.

The authors of this research study sought to determine factors associated with
motivational engagement in video gaming. They had 189 video game players self
report on goal orientation, affect, need for cognition, and perceptions of

Annotated Bibliography

12

engagement in flow. They also interviewed a sub-set of 25 participants to identify

patterns that influenced need for gaming. The authors wanted to see if
motivational engagement for gaming would transfer to an educational setting.
They did find that engagement had a definite relationship between individual skill
and game complexity. Gamers also sought pleasure with gaming and advancing
through challenges because gamers received immediate feedback. Qualitative
methods to collect and analyze interview data in order to create a coding structure,
taxonomy of domains, and comparative conclusions to determine gaming
motivation and engagements. Quantitative analysis was used to identify unique
variance among individual difference factors and motivational variable, success
factors, need for cognition and the affective factors related to gaming.
Quantitative and Qualitative were equally weighted. Instruments used were:
1) Video Game Play, which a 7-point scale to assess habits and attitudes toward
gaming
2) Goals Inventory (18 item), which is a 5-point scale that measures subject
motivation and performance
3) Need for Cognition (18-item), which evaluates subject interest and propensity
towards engagement in cognitively demanding activities
4) Success for Gaming Scale (13-item), which is a modified form of the Task and
Ego Orientation in Sport Questionnaire (both intrinsic and extrinsic)
The procedure included students completing a demographic survey, the four
instruments, and then 25 were interviewed for 30 minutes using a 12 open-ended
questions to learn about participant descriptions, perceptions, and behaviors
associated with video game play. There was a three-phase process to categorize
and analyze:

Annotated Bibliography

13

1) Centered exclusively with content of the interview protocol

2) Smaller categories from Phase 1 set by cluster coding
3) Coding to generate the main themes for analysis identifying patterns that
triangulated the self-report data
The results indicated that participants participate in video gaming to fulfill
recreational, social, and esteem needs, however, evidence suggested that it wasnt
very likely that games would transfer to instructional expectations. This is due to
the need for gamers to socialize and not learning, but learning would not be the
reason. The authors encourage researchers to, test new methods and models of
pedagogy that replicate the video gaming experience described by our
participants (Hoffman, B., & Nadelson, L., 2009, p. 267). This would include
promoting collaboration, instant feedback, and controlling choices. Some
limitations to the research include general view of video gaming, exogenous
variables like gender, and the subjects were student volunteers who wanted extra
credit. Compared to other research studies, this study did include comprehensive
instruments, but it did not provide evidence that it would transfer to the
educational setting.
http://proxy.kennesaw.edu/login?url=http://search.ebscohost.com/login.aspx?
direct=true&db=tfh&AN=49157000&site=eds-live&scope=site

Kelleher, C., & Pausch, R. (2007). USING STORYTELLING TO MOTIVATE

PROGRAMMING. Communications Of The ACM, 50(7), 58-64.
doi:10.1145/1272516.1272540

Annotated Bibliography

14

The authors investigated using the Storytelling Alice programming environment

to create computer-animated movies to inspire middle school girls interest in
learning to program computers. More than 200 middle school girls (mostly Girl
Scouts, ages 11 to 15, from Pittsburgh area) participated in the formative
evaluation that informed the design of Storytelling Alice. They tested 15 different
versions of Storytelling Alice and were guided by the girls storyboards of what
they wanted to build, observed problems, and listened to questions that came up
during user testing and recorded logs. Three major changes were made: Added
high level animations and support for creating multiple scenes, created a library of
3D characters and scenery that helps spark story ideas, and created a story-based
tutorial we present through a new interaction technique they developed called
Stencils. The study showed that girls did start with simple sequence of
instructions, but storytelling provided a transition to more complex
programming concepts and constructs. During this study it was found that, All of
them created a sequential program in Storytelling Alice; 87% created a program
with multiple methods; and, in their first two hours of programming, several
were using loops and/or creating methods that take parameters (Kelleher and
Pausch, 2007, p. 63). It was also found that girls using this program were more
motivated to program and the evidence was that they were 3 times more likely to
sneak extra time after time was called. The authors cited that storytelling works
for three main reasons: It gives girls an opportunity for self expression, time to
think through issues they encounter in their own lives, and allows girls to share

Annotated Bibliography

15

stories with friends and classmates. The author stated, As we search for games
that are a reasonable scope for beginning students in a game world that is not
gender-neutral, we risk choosing games (such as first-person shooters) that
disproportionately appeal to male students (Kelleher and Pausch, 2007, p.63).
It is important that when choosing games we dont alienate or cause a reduction
of diversity in computer science. In comparison to other articles Ive read, this
article did include some research, but research for developing an improved
version of Alice to appeal to girls and motivate them to participate in computer
science programming courses.

http://proxy.kennesaw.edu/login?url=http://search.ebscohost.com/login.aspx?
direct=true&db=bth&AN=25690090&site=eds-live&scope=site

Kose, U., Koc, D., & Yucesoy, S. A. (2013). Design and Development of a Sample
"Computer Programming" Course Tool via Story-Based E-Learning
Approach. Educational Sciences: Theory And Practice, 13(2), 1235-1250.

This study introduces a story-based e-learning course that was designed and
developed for using within computer programming courses. The authors believe
that using this approach and the story-based environment, it can be easy, fast, and
entertaining for students to learn the fundamentals of computer programming.
According to the authors, It is important that story-based e-learning tools enable
students to perform learn-by-doing sessions in the context of experiences

Annotated Bibliography

16

provided by the events of designed stories (Kose, U., Koc, D., & Yucesoy, S. A.,
2013, p. 1236). The methodology is a solutions approach where students can
complete an experimental application or student survey to provide evidence. This
tool includes a typical story and feedback, as well as, allowing students to create
their own characters on the related environment and take part in story flow. It also
allows students to perform the story-based activities, exercises, quizzes, and
learn-by-doing approach. The experimental application group and the control
groups both included 50 students each for a total of 100 students. The
experimental group used the story-based e-learning tool with all its features and
functions while the control group took the traditional sessions formed by
theoretical and applied teaching approaches provided by the teachers. The results
indicated that the experimental group had a significantly high percentage in the
program. There are some points that should be addressed as to why the
experimental group may have performed better than the control group. These
included but are not limited to the following:
1) Students enjoyed the self-learning process and performed more course
activities
2) Students were faster in the related learning process due to the story-based elearning tool
3) Students were encouraged to be active participates in the learning process
4) Students were allowed to perform more applied activities via computer
5) Student personal features and distinguishing features may also affect the
results.
All attempts were made to make equivalent groups to get more accurate results.
Students also had the opportunity to take a student survey to express his or her

Annotated Bibliography

17

experience. The authors concluded that the tool provided an effective solution
approach on the problem of learning computer programming. It led students from
concrete to abstract concepts. They stated, As general, this study is a successful
work on showing the usage of computer-software technology to develop more
effective tools for improving understanding and learning levels for especially
technical and applied courses including abstract and complex subjects (Kose, U.,
Koc, D., & Yucesoy, S. A., 2013, p. 1249). Due to the positive feedback, it
encouraged others to create newer versions of the tool to achieve more successful
results. In comparison to other studies, I felt that this study was very thorough and
did provide evidence that using a story-based e-learning approach showed
significant growth in computer programming both concrete and abstract
application.
http://proxy.kennesaw.edu/login?url=http://search.ebscohost.com/login.aspx?
direct=true&db=eric&AN=EJ1017269&site=eds-live&scope=site

SUNG, K. (2009). Computer Games and Traditional CS Courses. Communications Of

The ACM, 52(12), 74-78. doi:10.1145/1610252.1610273

The author in this article examined the ongoing efforts to integrate computer
video games in existing traditional CS courses. Due to decline in computer
programming courses the author feels that there needs to be a push for presenting
CS concepts using real-world applications. There are three general categories of
video game integration into CS courses. These include game development classes,

Annotated Bibliography

18

game programming classes, and game development client. There are three broad
approaches used in Introductory Programming classes: Little or no game
programming but playing custom games, Pre-assignment game development and
these are designed around technical topics, and then Extensive game development
where faculty design programming assignments based on the custom library.
These approaches have evidence that integrating computer gaming in these
courses is a promising strategy for recruiting and retaining potential students.
These projects in these courses are student-centric where the main goals of study
are student engagement and various learning outcomes. The author cites
considerations that must be taken when designing materials and include:
1) The materials should not demand knowledge in computer games or graphics
2) The materials should include independent modules that are limited in
curriculum scope
3) The materials should support selective experimentation by individual faculty
members in small-scale pilot demonstration projects in existing courses
It is also very important that when adopting new materials that the selective
gradual approach is used. Also, it is important factors are taken into consideration:
Institutional oversight, faculty background, gender and expertise neutrality,
infrastructure support, conceptual integrity, and textbook availability. I found this
article very useful in understanding the methods and steps to adopting a new
material or program. In comparison to other articles, this one provided much
insight that was specific to the benefits of integrating video games into the CS
programming courses. One important aspect into the adoption of new curricula or

Annotated Bibliography

19

components is the need to make the adoption gradual and requires more than just
me but the whole village.

http://proxy.kennesaw.edu/login?url=http://search.ebscohost.com/login.aspx?
direct=true&db=bth&AN=45580313&site=eds-live&scope=site
Sykes, E. R. (2007). Determining the Effectiveness of the 3D Alice Programming
Environment at the Computer Science I Level. Journal Of Educational
Computing Research, 36(2), 223-244.

The author of this study sought to determine the effectiveness of the 3D Alice
programming environment at the Computer Science I level. This program
provides the ability to build virtual worlds where objects and their behaviors are
in a real context. After the world is created, students are able to write the code
to support the execution of the interactive game using a drag-and-drop fashion.
Studies have shown that involving gaming where students learn from designing
their own games have promising results. According to the author, The Alice
programming language is a distinct overlap with these studies since Alice is a tool
that empowers the learner to design their own interactive games and multimedia
programs while attempting to generate interest in the field of computer science
(Sykes, E. R., 2007, p. 228). There were two comparison groups used in this
research that were Computer Science I course in the summer of 2004 and the
summer of 2005. The experimental group consisted of the students in Computer
Science I during the fall of 2005. There are some factors that exist that could

Annotated Bibliography

20

impact the research. These are that the course is considered an elective and
academic levels are not consistent. The Comparison Group 1 consisted of 23
students, the Comparison Group 2 consisted of 11 students, and during the fall of
2005, the Alice group consisted of 72 students. There was 1 professor who taught
all three groups for the entire terms. C1 and C2 were taught in traditional format
using C programming language where the Alice Group conducted hour long
sessions approximately every week to elicit specific information about their
experience using the Alice programming environment. This study utilized both
qualitative and quantitative techniques in a quasi-experimental design. The
qualitative focus was observation, surveys, and personal interviews of the Alice
Group and input from professors while the quantitative was based on performance
scores on the pre and post-tests. All tests were a combination of knowledge-based,
skill-set-based, and problem solving-based programming problems. Qualitative
results provide an insight to the student and professor perspective. Although
students enjoyed working with Alice, students felt that it experienced instability,
quirks, and heavy resource demands. It leads one to believe that Computer
Science I level students would like to learn more established languages such as
C/C++, C#, Java, etc. instead of Alice (Sykes, E. R., 2007, p. 236). The
quantitative data provides support for the Alice program in that the Alice Group
outperformed students in both C1 and C2 Groups. The benefits of Alice included
the elimination of syntax errors, highly visual programming environment, enables
students to focus on problem-solving skills, and students found Alice enjoyable.
Some disadvantages included instability of Alice, not all students were happy with

Annotated Bibliography

21

a game programming environment, often led to long, verbose code, not suited
to solve real-world problems, and students do not develop any skills in typing
code. The author stated, Qualitative feedback from students and professors,
indicate that Alice is a good environment for novice programming students. It is
interesting to note that, despite the numerous technical problems, the Alice group
persevered and rose over these problems to outperform the comparison groups
(Sykes, E. R., 2007, p. 241). The author also states that there has been an increase
in enrollment, and he attributes it to word of mouth that the course is enjoyable
and uses interactive and game-like programming. The author also indicated that
the overhauled Alice would be able to transition to conventional programming
languages. In comparison to other articles, I find that this article was clearly stated
and organized and provided a comprehensive explanation and was data supported.
It is clear that Alice is enjoyable, but this study supported the idea that it could be
used to, provide an interesting, rewarding experience and to increase the chances
that students will continue to succeed in Computer Science with careful
adjustments (Sykes, E. R., 2007, p. 242).

http://proxy.kennesaw.edu/login?url=http://search.ebscohost.com/login.aspx?
direct=true&db=cph&AN=24660527&site=eds-live&scope=site