Вы находитесь на странице: 1из 14

G Model

EARCHI-978; No. of Pages 14 ARTICLE IN PRESS


Early Childhood Research Quarterly xxx (2018) xxx–xxx

Contents lists available at ScienceDirect

Early Childhood Research Quarterly

Reliability and validity of a measure of preschool children’s theatre


arts skills: The Preschool Theatre Arts Rubric
Amy Susman-Stillman a,∗ , Michelle Englund a , Chloe Webb b , Amanda Grenell a
a
Center for Early Education and Development, Institute of Child Development, University of Minnesota, 1954 Buford Avenue #425, St. Paul, MN, 55108,
United States
b
Department of Educational Psychology, University of Minnesota, 250 Education Science Building, 56 East River Road, Minneapolis, MN, 55455, United
States

a r t i c l e i n f o a b s t r a c t

Article history: The increasing attention to arts integration in early childhood education necessitates assessment of arts
Received 1 February 2017 skills in order to measure the impact and the mechanisms by which the arts can affect early childhood
Received in revised form development. The purpose of this study is to describe and provide initial validation of a newly developed
23 November 2017
observational measure of preschool children’s theatre arts skills, the Preschool Theatre Arts Rubric (PTAR),
Accepted 9 December 2017
which measures five preschool theatre arts skills: independence in role play, use of face and gesture,
Available online xxx
focus/persistence, collaboration, and theatricality. We present findings regarding internal consistency,
inter-rater reliability, construct validity (convergent and divergent), and sensitivity to change of the
Keywords:
Creative drama
PTAR using a sample of 158 ethnically and linguistically diverse, low-income preschoolers participating
Storytelling/storyacting in an early childhood theatre arts program using storytelling and storyacting (ST/SA) in their preschool
Observational assessment classrooms. Findings indicate that the PTAR demonstrates acceptable internal consistency and interrater
Measurement reliability; convergent and divergent validity with a norm-referenced measure of expressive language, an
authentic measure of narrative expression, a teacher-rated measure of learning related social skills, and
a measure of storytelling quality; and significant sensitivity to change. While more research is needed,
results suggest that the PTAR can be reliably and validly used to observe preschool children’s theatre
arts skills in research, classroom, and programmatic contexts. Findings suggest that theatre arts skills are
valid indicators of expressive language skills and to a degree, learning-related social behaviors.
© 2017 Elsevier Inc. All rights reserved.

1. Introduction (Colorado Preschool Drama and Theatre Arts Academic Standards,


2009; Minnesota Early Childhood Indicators of Progress, 2017). Use
Theatre arts, or creative drama, traditionally part of best prac- of dramatic techniques in preschool classrooms, such as improvised
tices in early childhood education (Furman, 2000; Wee, 2009), is storytelling, or storytelling/storyacting (ST/SA; Paley, 1990) is being
experiencing a rebirth as an integral component of early child- promoted around the country (Cooper, 2009; McNamee, 2015;
hood education. The National Coalition for Core Arts Standards Sachs, Mardell, & Boni, 2014). An intentional emphasis on integrat-
recently developed a set of core theatre arts standards to promote ing theatre arts into early childhood education provides promising
coherence and quality in arts education for preK to 12th grade, opportunities to enhance early education program quality, engage
which focus on creating, presenting, responding, and connecting early learners, improve children’s school readiness skills, includ-
(National Coalition for Core Arts Standards, 2014). State early learn- ing those of special populations of early learners (English Language
ing standards, which describe the expected knowledge and skills Learners; Greenfader, Brouilette, & Farkas, 2015), and reduce the
of young children as they enter kindergarten, include critical skills achievement gap (Mardell, 2013).
that are fostered through theatre arts activities, such as listening, Currently, however, measurement of children’s skills in the con-
speaking, imagining, inventing, creating, and critically responding text of theatre arts programming is generally of non-arts skills,
such as oral narrative skills (Lee, Patall, Cawthorn, & Steingut, 2015;
Mages, 2008). There is a dearth of sound measurement tools avail-
able to appraise relevant arts skills, which hampers the ability to
∗ Corresponding author.
understand the processes by which theatre arts may affect chil-
E-mail address: asusman@umn.edu (A. Susman-Stillman).

https://doi.org/10.1016/j.ecresq.2017.12.001
0885-2006/© 2017 Elsevier Inc. All rights reserved.

Please cite this article in press as: Susman-Stillman, A., et al. Reliability and validity of a measure of preschool children’s theatre arts
skills: The Preschool Theatre Arts Rubric. Early Childhood Research Quarterly (2018), https://doi.org/10.1016/j.ecresq.2017.12.001
G Model
EARCHI-978; No. of Pages 14 ARTICLE IN PRESS
2 A. Susman-Stillman et al. / Early Childhood Research Quarterly xxx (2018) xxx–xxx

dren’s development, and maximize the potential benefits of arts Their pretense becomes more decontextualized and distanced from
integration. In this study, we describe and provide initial reliability reality and they begin to substitute objects that are different from
and validity data for the Preschool Theatre Arts Rubric (PTAR), an the original object in function and appearance (Carlson & Zelazo,
observational tool developed to assess children’s preschool theatre 2008; Fein, 1981). During the preschool period, pretend play is
arts skills in the context of storytelling/storyacting (ST/SA; Paley, in “high season” (Singer & Singer, 1992) since children begin to
1990). The PTAR represents a first effort to measure children’s the- engage in complex social pretend play, or sociodramatic play, in
atre arts skills in the context of early childhood classroom-based which they act out roles with others (Howes & Matheson, 1992).
theatre arts programming. Children’s executive function (EF) or self-regulation is also rapidly
improving; children are able to practice their EF skills in the con-
1.1. The development of preschool theatre arts skills text of pretend play, such as being flexible enough to pretend that
one object stands for another, taking turns with a play partner
Preschool theatre arts skills emerge in the context of play, or during role-play, and following certain rules based on the pre-
“drama in its natural state” (Mages, 2008, p. 127). Children’s dra- tend context (Berk & Meyers, 2013). Similarly, their language skills
matic activities have been conceptualized on a spectrum with become increasingly complex; pretend play provides opportuni-
children’s pretend play at one end and theatre or “communica- ties to enhance vocabulary, tailor speech to a particular role, and
tion between actors and an audience” on the other end (Way, practice conflict resolution skills (Ervin-Tripp, 1991). They are also
1967, as cited in Mages, 2008, p. 127), and theatre techniques as developing a sense of physical presence, such as where their bod-
tools that enhance children’s natural proclivity toward creative ies are in space. The cognitive, social-emotional, executive function,
play (Youth Stages, 2016). Most researchers agree that all types and motor skill changes occurring during the preschool period facil-
of play are intrinsically motivating, enjoyable, have an element of itate the development of both theatre arts skills and early learning
imagination, and are flexible in that pretend actions can be dif- skills.
ferent than real ones (Krasnor & Pepler, 1980). In early childhood Engaging in dramatic play also promotes early childhood skills
classrooms, opportunities for dramatic experiences are actualized (Fleer, 2014; Lillard et al., 2013; Vygotsky, 1978). Vygotsky (1978)
through play-based and drama-based activities and described with posited that dramatic or pretend play is a major source of devel-
multiple terms, including creative drama (Furman, 2000; Mages, opmental growth for preschool age children; through dramatic
2008), sociodramatic play (Elias & Berk, 2002), guided play (Gupta, activity, children use imagination, learn to appreciate the function
2009), make-believe play (Berk & Meyers, 2013), drama-based ped- of rules, and develop self-regulation skills. Furthermore, through
agogy (Lee et al., 2015), storytelling drama (Wright, Diener, & Kemp, play children become more deeply engaged, reflective, socially
2013), and ST/SA (Hynes-Berry, 2012; Mardell, 2013; Paley, 1990). skilled, and mature in terms of their interactions and personal skills.
In this paper, our focus on dramatic play within the context of “In play it is as though he were a head taller than himself. Action in
ST/SA aligns most closely with the concept of guided play. Guided the imaginative sphere, in an imaginary situation, the creation of
play “incorporates adult-scaffolded learning objectives but remains voluntary intentions, and the voluntary formation of real-life plans
child-directed” (Weisberg, Hirsh-Pasek, & Golinkoff, 2013, p. 105). and volitional motives—all appear in play and make it the high-
The collaboration between adult and child during guided play is est level of preschool development” (Vygtosky, 1978, p. 102). More
an important difference between spontaneous pretend play or free recently, cultural-historical perspectives that build on Vygotskian
play and guided play. When we refer to theatre arts skills, we are theory (Fleer, 2014) argue that role-play is central to understand-
referring to a set of skills that children use to participate in what ing children and contemporary societies, and is viewed as a cultural
is commonly referred to as creative drama or “improvised guided tool promoting children’s development. Children’s role play actions
enactment” (Mages, 2008, p. 127) and include language, cognitive, serve as a window into their understanding of their world, a strat-
social, emotional, and motor skills. egy for adults to cultivate particular ways of thinking and behaving,
The National Coalition for Core Arts Standards Board defined and a strategy for children to practice rules and behaviors and
what preschool children should be able to know and do when they explore ideas.
participate in theatre arts activities, such as engaging in dramatic Indeed, play and its dramatic underpinnings have been used in
play experiences, recalling an emotional response, telling a short the early childhood classroom to promote an array of key early
story, and understanding that imagination is fundamental to dra- childhood skills ranging from literacy (Mages, 2008) to creativity
matic play/guided dramatic experience (National Coalition for Core (Yeh & Li, 2008) to social and self-awareness and self-regulation
Arts Standards, 2014). Demonstrating these standards requires (Barnett et al., 2008; Blair & Raver, 2014; Bodrova & Leong, 1996,
children to use preschool cognitive, language, social, emotional, 2001, 2007; Diamond, Barnett, Thomas, & Munro, 2007). There is a
and motor competencies. The skills of a preschooler engaging in nascent literature examining the impact of preschool theatre arts
theatre arts overlap with the skills of a preschool early learner. on young children’s development. Findings are promising, with
With the emergence of pretend play during the early toddler a handful of studies demonstrating the impact of guided drama
period, children show the beginnings of theatre arts skills. These training and activities (including ST/SA) on oral narrative abili-
first pretend actions involve children acting “as if” with objects ties of English-speaking preschoolers (Cooper, Capo, Mathes, &
such as bringing an empty spoon to the mouth to pretend to eat Gray, 2007; Lee et al., 2015; Mages, 2008; Nicolopoulou, Cortina,
(Fein, 1981). Around 24 months of age, children begin to understand Ilgaz, Cates, & de Sá, 2015; Phillips, Gorton, Pinciotti, & Sachdev,
the symbolic relation between objects and their substitutions (e.g., 2010; Podlozny, 2000) and non-English-speaking preschoolers
using a spoon as a phone) and start to combine object substitutions (Greenfader, Brouillette, & Farkas, 2015; Krueger, Orton, & Bays,
while playing (Carlson & Zelazo, 2008; Sachet & Mottweiler, 2013). 2013a; Krueger, Orton, & Bays, 2013b). A few studies also find posi-
They also begin to engage in role-play through acting on dolls and tive effects of theatre arts education on social competence (Menzer,
stuffed animals (Sachet & Mottweiler, 2013) and begin to under- 2015; Nicolopoulou, Barbosa de Sa, Ilgaz, & Brockmeyer, 2009;
stand the pretend actions of others and the imaginary outcomes Nicolopoulou, Cortina, Ilgaz, Cates, & de Sá, 2015). These studies
that these pretend actions can have (Harris, Kavanaugh, Wellman, suggest that the integration of theatre arts and early childhood
& Hickling, 1993; Lillard & Witherington, 2004). activities may be fruitful for supporting preschool children’s devel-
During the preschool years, children become better able to opment.
reflect on the relation between symbols and their referents and
begin to understand irony and sarcasm (Carlson & Zelazo, 2008).

Please cite this article in press as: Susman-Stillman, A., et al. Reliability and validity of a measure of preschool children’s theatre arts
skills: The Preschool Theatre Arts Rubric. Early Childhood Research Quarterly (2018), https://doi.org/10.1016/j.ecresq.2017.12.001
G Model
EARCHI-978; No. of Pages 14 ARTICLE IN PRESS
A. Susman-Stillman et al. / Early Childhood Research Quarterly xxx (2018) xxx–xxx 3

1.2. Integration of preschool theatre arts skills and early and personal skills (Lee et al., 2015; Podlozny, 2000; Rinne, Gregory,
learner/school readiness skills Yarmolinskyay, & Hardiman, 2011; Winner, Goldstein, &Vincent-
Lancrin, 2013).There is evidence that arts education for children,
Despite the natural proclivity of young children to pretend and when it engages sustained interest, results in improvements in
engage in creative drama, and studies demonstrating relations developmental functioning, such as attention and cognition (Posner
between preschool theatre arts skills and children’s develop- & Patoine, 2009). Other studies, conducted with older children
ment, there has been little intentional interdisciplinary integration and a growing number with preschoolers, find that drama training
between the fields of theatre arts and early childhood development. has an impact in particular on oral narrative skills (Winner et al.,
Yet there are multiple reasons to integrate these two disciplines. 2013). The field of early education currently struggles with how
Theatre arts offer innovative and diverse opportunities to promote to most effectively educate an increasingly diverse population of
a range of early childhood skills. For example, Cooper (2009) dis- early learners, and there are tensions surrounding the developmen-
cusses the close linkages between fantasy play and storytelling, tal appropriateness and effectiveness of child-centered, play-based
both requiring the use of imagination and pretense. Mardell and curricula and didactic, academic educational approaches. Using the
colleagues (Mardell, 2013; Sachs et al., 2014) discuss how the ST/SA arts as strategies to cultivate non-arts skills may bring innovative
practice offers an authentic opportunity to build literacy, social- and additionally effective strategies to early childhood settings and
emotional, and self-regulation skills by providing a bridge between potentially lead to improved developmental outcomes for children.
children’s contextualized speech and decontextualized language of There exists a significant amount of complementarity between
books, using children’s own stories and group expression of them, the theatre arts and early childhood skills; although described with
and requiring children to wait and defer gratification. different vocabulary, the kinds of practices that enhance preschool
Theatre arts also offers promising practices to reach different theatre arts skills also enhance early childhood skills. A cross-
types of early learners. For example, theatre arts can be used to walk (a comparison of content from one discipline to another;
support the oral and written language skills of English language Gross, 2012) comparing theatre arts skills and early childhood
learners (ELLs). Creative drama activities, which include both verbal skills was conducted to examine the potential intersection between
and non-verbal communication (Greenfader et al., 2015), and link- the two (Susman-Stillman, Rader, Asp, & Beckstrom, 2012). The
ing of words and emotions (Kruger, Orton, & Bays, 2013a, 2013b) resulting crosswalk document describes the close alignment in
offer experiential opportunities to increase breadth and depth of the goals and practices of six categories of theatre arts and early
vocabulary, speaking, listening, and written language skills, regard- childhood education skills: voice/focus, emotional expression and
less of the multiplicity of languages being spoken by children in understanding, physical awareness, ensemble experience and col-
a classroom. They also encourage high levels of engagement and laboration, characterization, and creativity (see Table 1 for an
connect the curriculum to children’s lives, which are recognized abbreviated version of the crosswalk). For example, theatre artists
pedagogic principles for teaching ELL students (Haneda & Wells, talk about the use of voice to express themselves while acting,
2012). Brouillette (2012) notes the efficiency of drama activities reflecting important early childhood skills such as receptive and
and observing peers for second language learning, as teachers can expressive language use. Theatre artists also emphasize emotional
more quickly gauge the learning of multiple children and peers expression and understanding, which aligns with preschool social
can observe multiple children as they seek to comprehend. Krueger and emotional competence. Review of state early learning stan-
et al. (2013a) emphasize the enhanced language engagement that dards also reinforces the notion of overlap between theatre arts
occurs in dramatic activities as opposed to simple exposure or and early childhood skills, as many include one or more categories
drilling. Rather than posing a barrier for children who are learning of arts skills, such as “demonstrate emotions and feelings in dra-
English, theatre arts can serve as a facilitator of second language matic play” (Colorado Preschool Drama and Theatre Arts Academic
learning. Standards, 2009).
Theatre arts can also enhance early childhood teaching tech- Further evidence for an integrated conceptualization of theatre
niques across diverse early childhood education settings. There are arts and early childhood skills is present in literature document-
multiple similarities in the practices of preschool theatre artists and ing relations between theatre arts skills or dramatic play and
preschool teachers (e.g., encourage students to express emotions other important early childhood academic, social, and personal
verbally and non-verbally, ask questions to inform storytelling outcomes. For example, pretend play in preschoolers is related
and story comprehension, foster positive peer interactions), and to skills such as children’s self-regulation or executive func-
use of theatre arts can reinforce quality early childhood teaching tion (Carlson, Davis-Unger, & White, 2014; Elias & Berk, 2002;
practices in engaging ways. Reports of theatre arts programs in Thibodeau, Gilpin, Brown, & Meyer, 2016), language (Ervin-Tripp,
early childhood settings describe unique professional development 1991; Levy, Schaefer, & Phelps, 1986; Lovinger, 1974), social com-
opportunities specifically designed to incorporate theatre arts into petence (Colwell & Lindsey, 2005; Connolly & Doyle, 1984; Howes
early childhood teaching practices, regardless of classroom curric- & Matheson, 1992), theory of mind or perspective taking skills
ular model, and that use a range of teaching strategies including use (Astington & Jenkins, 1995; Schwebel, Rosen, & Singer, 1999; Taylor
of video, role play, and pantomime (Brouillette, 2012; Mages, 2017; & Carlson, 1997), and emotion regulation (Fantuzzo, Sekino, &
Sachs et al., 2014). They also help teachers develop approaches for Cohen, 2004; Galyer & Evans, 2001; Lindsey & Colwell, 2003). Thus,
handling challenging situations that reflect deeper issues, such as the theoretical and practical integration of theatre arts and early
dealing with violence in stories (Mardell, 2013). Preschool class- childhood skills represents a natural step toward an innovative per-
rooms that have thoroughly incorporated use of ST/SA have used spective from which to understand and promote young children’s
it not only to promote language and literacy, but also to guide development. To build on this integrated perspective and gauge
their authentic assessment practices and behavior management the benefits of a theatre arts-early childhood integrated lens, it is
(McNamee, 2015) and create a democratic classroom environment necessary to develop measurement strategies for preschool theatre
(Cooper, 2009). Boston Public Schools, with its Boston Listens pro- arts skills.
gram, made ST/SA a cornerstone of the curriculum and teaching
practices in their preschool and kindergarten classrooms, even as 1.3. Measurement of theatre arts skills
a component of family engagement (Mardell, 2013).
Furthermore, arts education is recognized as a vehicle for the Documenting the potential benefits by which theatre arts may
transfer of skills developed within the arts to other academic, social, facilitate attainment of academic, social, and personal development

Please cite this article in press as: Susman-Stillman, A., et al. Reliability and validity of a measure of preschool children’s theatre arts
skills: The Preschool Theatre Arts Rubric. Early Childhood Research Quarterly (2018), https://doi.org/10.1016/j.ecresq.2017.12.001
G Model
EARCHI-978; No. of Pages 14 ARTICLE IN PRESS
4 A. Susman-Stillman et al. / Early Childhood Research Quarterly xxx (2018) xxx–xxx

Table 1
Crosswalk of Theatre Arts Skills, Early Childhood Skills, and PTAR Validation Measures.

Theatre Arts Skills Examples of Early Childhood Skills PTAR Scale PTAR Validation Measures

Voice/Focus – speaking, posing questions, Receptive and expressive language Vocalizations BDI-II: Expressive Communication
modulating voice, listening and
comprehending
Focus/Persistence PLBS: Preschool Learning Behaviors
Emotional expression and understanding – Social and emotional competence Vocalizations BDI-II: Expressive Communication
ability to recognize, express, and understand
emotions with accuracy; respond
appropriately to others’ emotions; and
self-regulate
Gestures & Movements PSSR: Storytelling Quality
Overall Theatricality
Physical Awareness – use of body to convey Motor development, including large and Gestures & Movement BDI-II: Expressive Communication
information and emotion small muscle control and movement
Ensemble experience and collaboration – Social competence, such as successfully Collaboration PLBS: Preschool Learning Behaviors
working together developing and participating in group
experiences, building positive
interactions among children, and group
conflict resolution skills
Characterization – ability to adopt the Cognitive development, such as symbolic Independence in Role Play BDI-II: Expressive Communication
personality and perspectives of a character representation, the ability to role play,
and perspective-taking
Vocalizations PLBS: Preschool Learning Behaviors
Gestures & Movement PSSR: Storytelling Quality
Overall Theatricality
Creativity and imagination – invites new, Approaches to learning, such as making Independence in Role Play NAP: Grammatical Complexity
different, and potentially unusual ways to unique connections among differing
express and act ideas or experiences, generating multiple
ideas, considering alternatives, and
reflecting on possibilities, as well as
safety to push boundaries
Overall Theatricality PLBS: Preschool Learning Behaviors
PSSR: Story Telling Quality

Note. BDI (Battelle Developmental Inventory); NAP (Narrative Assessment Protocol); PLBS (Preschool Learning Behaviors Scales); PSSR (Preschool Storytelling Skills Rubric).

in young children requires the ability to accurately and expedi- knowledge, no reliable and valid measure of preschool children’s
tiously measure theatre arts skills (Winner et al., 2013). This has acting skills currently exists. The descriptions of programs using
been a challenge in the research on the benefits of pretend play, ST/SA (Mardell, 2013; McNamee, 2015) do not report measures of
as the literature does not yet definitively demonstrate its unique the growth of the acting skills themselves. Given the increasing use
impact on children’s development due to methodological chal- of theatre arts in early childhood settings, developing preschool
lenges in previous studies (Lillard et al., 2013). theatre arts measures is necessary for high-quality research on the
Measuring pretend play is a complex endeavor (McNamee, impact of theatre arts in early childhood and for refining best prac-
2015) and is conducted in multiple ways. Some assessments tices in preschool theatre arts and early childhood education (Lee
of play are conducted individually (Child-Initiated Pretend Play et al., 2015; Mages, 2008; Winner et al., 2013).
Assessment, Stagnitti & Unsworth, 2004) or in the laboratory
(Nicolopoulou et al., 2015). Others are either via independent
1.4. Present study
observation in the authentic context of play itself with small groups
(The Smilansky Scale for Evaluation of Dramatic and Sociodra-
The present study is an initial assessment of the psychomet-
matic Play, Smilansky & Shefatya, 1990) or large groups (Mature
ric properties of the Preschool Theatre Arts Rubric (PTAR), a new
Play Observation Tool, Gemeroth, 2017), or through teacher rat-
observational measure of preschool children’s theatre arts skills
ings (Penn Interactive Peer Play Scale, Fantuzzo, Coolahan, Manz,
developed for use in the context of an early childhood theatre
Canning, & Debnam, 1995). Different aspects of pretend play are
arts outreach program that emphasizes storytelling and storyacting
also assessed; for example, affect and emotion in play (Affect in
(ST/SA; Diener, Wright, & Kempf, 2016; Mardell, 2013; Paley, 1990).
Play Scale, Kaugars & Russ, 2009), or social interaction during
Our goal was to fill an important measurement gap by creating
play (Doyle & Connoly, 1989; Howes, 1980). When considering
an observational measure of preschool children’s theatre arts skills
the appropriateness of these tools for measuring theatre arts skills
that would demonstrate necessary psychometric properties of reli-
in the context of ST/SA, however, there are virtually no measures
ability and validity. We wanted to develop a measure that would be
which appear to be appropriately aligned with the skills that the-
easy for researchers, early childhood teachers, and theatre artists to
atre arts programs are cultivating (Lee et al., 2015) and that can
use to gauge the growth of preschool children’s theatre arts skills,
be implemented in an ST/SA context, which is increasing in its
study the impact of arts participation on children’s development,
prominence in early childhood settings (e.g., Boston Listens).
and conduct program evaluations.
There are a few, recently developed measures of theatre arts-
relevant skills specific to preschoolers (Lee et al., 2015; Mages,
2008, 2015), such as pretend abilities (Nicolopoulou et al., 2015), 1.5. Developing the PTAR
artfulness, or the imagination children use to tell original stories
(Glenn-Applegate, Breit-Smith, Justice, & Piasta, 2010), and creativ- As we developed the PTAR, we took into consideration the
ity (Garaigordobil & Berrueco, 2011; Yeh & Li, 2008). However, these importance of the process of validating the measure (Chan, 2014),
measures do not index preschool children’s acting skills, and to our and integrated support for both face and content validity. Especially
important to this effort was our close work with early childhood

Please cite this article in press as: Susman-Stillman, A., et al. Reliability and validity of a measure of preschool children’s theatre arts
skills: The Preschool Theatre Arts Rubric. Early Childhood Research Quarterly (2018), https://doi.org/10.1016/j.ecresq.2017.12.001
G Model
EARCHI-978; No. of Pages 14 ARTICLE IN PRESS
A. Susman-Stillman et al. / Early Childhood Research Quarterly xxx (2018) xxx–xxx 5

researchers, educators, and theatre arts professionals, and our use 1.6.2. Use of face/body/movement/gesture
of the National Core Theatre Arts Standards (National Coalition This category refers to the appropriate use of face, body, and
for Core Arts Standards, 2014) and the theatre arts-early child- gesture to indicate a particular action or behavior, or to express
hood development crosswalk as the conceptual bases for selecting emotion. This category is captured in the following pre-K National
theoretically and practically relevant theatre arts-early childhood Theatre Arts Standard: “With prompting and support, contribute
skills (Table 1). ST/SA (Paley, 1990), a well-known practice in through gestures and words to dramatic play or a guided drama
early childhood education to build language, literacy, and narrative experience (e.g., process drama, story drama, creative drama).” For
skills (Mages, 2008; McNamee, 2005), and classroom community young children, gestures are thought to play “a role in causal learn-
(Wright et al., 2013) is the context for which the PTAR was created. ing, perhaps by giving learners an alternative, embodied way of
Since ST/SA encourages children to tell a story, interact, use actions representing ideas” (Cook, Mitchell, & Goldin-Meadow, 2008, p.
in a story, collaborate, and demonstrate individual expression, it 1047). Furthermore, preschool-age children improve their abil-
offers an ideal opportunity to cultivate children’s theatre arts skills ity to identify, label the emotions of others and understand the
(Wanerman, 2009). We developed the PTAR in order to describe cause of an emotion (Denham, Zoller, & Couchoud, 1994; Lagattuta,
those skills and thus enable the ability to reliably and validly mea- Wellman, & Flavell, 1997), and gain in large and fine motor dexter-
sure them and facilitate study of dramatic arts in early childhood ity, all which undergird the use of face and body in acting. This
development. category is intended to capture use of gesture or movements to
A first step in creating this new observation tool was to select appropriately portray, or enhance the portrayal of a character.
the skills to be measured. Working with theatre artists and early
childhood development experts, we began this process with a set of 1.6.3. Focus/commitment to the play
theatre arts skills for elementary-age students (Children’s Theatre This category gauges children’s maintenance of attention during
Company, 2012). We referred to the National Core Theatre Arts the acting process, and ability to independently maintain or regain
Standards (National Coalition for Core Arts Standards, 2014) and focus if moments of distraction or inattention occur. This category
state early learning standards (Colorado Academic Standards for is captured in the following pre-K National Theatre Arts Standard:
Drama and Theatre Arts: Preschool and Kindergarten, Minnesota “With prompting and support, transition between imagination
Early Childhood Indicators of Progress) to identify overlapping and reality in dramatic play or a guided drama experience (e.g.,
skills that both the theatre artists and early childhood experts process drama, story drama, creative drama)” (National Coalition
deemed important and that were precursors of the identified ele- for Core Arts Standards, 2014). Maintaining attention requires
mentary theatre arts skills. We also confirmed the alignment of the development of executive function skills or “neurocognitive
those skills with the theatre arts-early childhood crosswalk. We skills involved in goal-directed problem solving, including working
then reviewed videotapes of ST/SA sessions to determine which memory, inhibitory control, and set shifting/flexibility” (Carlson,
skills were observable and measurable. An initial seven skill cate- Zelazo, & Faja, 2013, p. 706). Executive function skills develop
gories were selected and subsequently defined and definitions with rapidly during the preschool period (Carlson, 2005) and are asso-
examples of behaviors within each category were developed. We ciated with school readiness and academic achievement (Blair &
then piloted the tool during videotaped and live ST/SA sessions, Razza, 2007; McClelland et al., 2007). Therefore, the preschool
and conducted group reviews. Through that process, we selected period is a good time to promote children’s executive function
six of the initial seven skills representing each of the domains of and pretend play is one context in which children can practice
the crosswalk, with substantial face validity amongst the group and use their developing executive function skills (Berk & Meyers,
of developers: independence in role play, use of face/body, vocal- 2013). Children who are able to maintain their focus on the play,
ization, focus/commitment to the play, collaboration/awareness of regardless of the size of their role and other potential distractions,
others, and overall theatrical quality (Appendix A). We also recon- demonstrate their attention during ST/SA.
firmed with the theatre artist and early childhood experts these
skills and the definitions. During the piloting process, we contin- 1.6.4. Collaboration/awareness of others
ued to refine the category definitions and accumulate illustrative This category captures the degree to which a child participates
examples. We also decided to eliminate the category of vocaliza- as a member of the group while acting, cooperates and compro-
tions because they were occurring at a very low incidence, likely mises with others, and contributes ideas, listens to others’ ideas,
because the stories the children were dictating tended to include and supports others’ ideas as they tell their own stories. This cate-
very little, if any, dialogue. Thus, we included five preschool theatre gory is captured in the following K National Theatre Arts Standard:
arts skills in the final version of the PTAR. “With prompting and support, interact with peers and contribute
to dramatic play or a guided drama experience.” Positive peer rela-
1.6. PTAR skills tionships in early childhood are critical to both concurrent and
future psychosocial adjustment (Ladd, 1989). Successful collabora-
The final PTAR scales and the rationale for their inclusion is tion also increases children’s motivation during challenging tasks
offered below (also see Appendix A). (Butler & Walton, 2013). In addition, group social skills are included
in various state early learning standards, so we have included it here
1.6.1. Independence in role play as a key preschool theatre arts skill on the PTAR (New Mexico Early
This refers to children’s need for prompting or support from Learning Standards). Children who work together to accomplish
the adult (s) facilitating the storyacting, with the goal of child acting out the story demonstrate collaboration.
independence in role playing. All of the pre-K National Common
Core Theatre Standards begin with the phrase “with prompting and 1.6.5. Overall theatrical quality
support” which aligns with the ideas of agency and independence In this category, we provide a global rating of the child’s perfor-
that are important to early childhood educators. For preschoolers, mance, taking into account their entire role and performance and
the development of a sense of agency, akin to Erikson’s notion of considering the demonstration of a particularly dramatic expres-
initiative, is a key developmental task (1959). Children who need sion or action (s) (e.g., the dragon flinging himself on the floor
few, if any prompts, or are able to move into or sustain their role after being slayed by the bad guy, or portraying an ocean by act-
after prompting are viewed as showing greater independence in ing as waves crashing on the beach). This category is captured in
role play. the following pre-K National Arts Standard: “With prompting and

Please cite this article in press as: Susman-Stillman, A., et al. Reliability and validity of a measure of preschool children’s theatre arts
skills: The Preschool Theatre Arts Rubric. Early Childhood Research Quarterly (2018), https://doi.org/10.1016/j.ecresq.2017.12.001
G Model
EARCHI-978; No. of Pages 14 ARTICLE IN PRESS
6 A. Susman-Stillman et al. / Early Childhood Research Quarterly xxx (2018) xxx–xxx

support, express original ideas in dramatic play or a guided drama elementary schools, one an arts-based school program) in an urban
experience (e.g., process drama, story drama, creative drama)” Midwestern school district received a preschool theatre arts pro-
(National Coalition for Core Arts Standards, 2014). Emerging evi- gram developed and delivered by a nationally renowned children’s
dence suggests that creative drama instruction is a factor that can theatre company. The preK programs, typical of school-based
increase children’s use of creativity (Yeh & Li, 2008). Children who preschool programs, followed the district pre-academic curricu-
embellish or bring an innovative twist to their role demonstrate lum, which is aligned with both district standards and state early
creative, theatrical qualities. learning standards. To participate in the preschool theatre arts pro-
gram, preschool teachers merely replaced an hour once a week of
1.7. Validating the PTAR their normal routine with the program.
In each year, four of the five classrooms offered full-day
There are not yet any other valid measures of which we are
preschool and one classroom offered a half-day program. All class-
aware that assess preschool theatre arts skills. Given this, and rooms received 19 sessions between October and April of the school
because we were interested in demonstrating linkages between
year with a professional teaching artist (TA) trained in interactive
theatre arts skills and preschool language and learning-related
and improvisational storytelling and storyacting and experienced
social behaviors, we chose to validate the PTAR against a measure in working with preschool-age children. The goal of the theatre
of preschool learning-related social behaviors and measures of oral arts program is to help children transform into storytellers of their
narrative skills—expressive communication, narrative microstruc-
own lives and build their oral narrative and theatre arts skills.
ture, and storytelling quality. There is substantial literature
During the program, TAs provide one hour-long weekly sessions
documenting the significance of oral narrative skills for success in
where they lead children through: (a) a series of theatre warm-
literacy (August & Shanahan, 2006; Mages, 2015; Snow, Burns, &
up activities, (b) an interactive storytelling experience based on a
Griffin, 1998), and ST/SA can enhance oral narrative skills using
fairytale, (c) a small group manipulative activity based on the inter-
story dictation, voice, gesture, and movement as experiential
active story being told, (d) ST/SA using stories that were dictated
strategies to cultivate language use, vocabulary, expression, and
to the classroom teacher before the day’s theatre arts session, and
understanding (Mages, 2008, 2015; Mardell, 2013; Vasquez, 2004).
(e) a closing acting activity. All TAs followed the same format for
Importantly, the oral narrative measures we chose are aligned
each session. Classroom teachers are trained by the TA to obtain
with the context of ST/SA. Expressive verbal communication is
children’s original, dictated stories used in ST/SA by using open-
a key component of story dictation, and a feature of standard-
ended prompts, hand-writing the children’s stories verbatim so
ized language assessments. Narrative microstructure, assessed by
the child can see his or her words on the page, and re-reading it
examining the grammatical features of a narrative story, again akin
out loud to the student for any clarification (Appendix B). Sepa-
to the context of ST/SA, provides an authentic example of expres-
rately, classroom teachers meet one-on-one with the TA to review
sive and functional language, and is predictive of later language
the lesson plans and select fairytales for the TA-led interactive
skill (Justice, Bowles, Pence, & Gosse, 2010). Storytelling quality
storytelling portion of the session. They also assist the TA during
provides an indication of the dramatic aspects in narratives, and
the small group manipulative activity and provide general behav-
may provide opportunities to display acting skills. For instance, the
ior management support if needed. Four of the five classroom
higher the quality of the story, the more likely a higher quality of
teachers participated both years; two teachers participated for one
acting can be observed.
year.
There is literature demonstrating relations between preschool
learning-related skills and children’s achievement in school
(McClelland & Morrison, 2003; McClelland, Morrison, & Homes, 2.2. Participants
2000). ST/SA has the potential to cultivate preschool learning-
related social behaviors, such as attention, confidence, persistence, Across the two years, 158 (74 males, 84 females;
and positive attitudes toward learning, because it is inclusive of Mageattime1 = 55.70 months, SD = 3.31) low-income preschool-
children’s ideas and suggestions and provides them with oppor- ers (determined by the school district’s program eligibility criteria
tunities to engage in guided social interactions (Cooper, 2009). based on eligibility for free and reduced price lunch) were
For example, ST/SA can facilitate children’s confidence because study participants. Children participated in the program for one
children’s ideas are actively accepted and positively reinforced, school year. The sample included both English language learners
thus fostering internal motivation (Lee et al., 2015). ST/SA, with identified by the school district based on their home language
the requirements of staying in character and following the sto- (ELL; n = 120) and native English speakers (non-ELL; n = 38). All
ryline, creates a context that promotes attention (e.g., needing to children in the classrooms participated in the year-long theatre
know when it is time to play your part) and self-regulation (e.g., arts program activities regardless of whether families consented
controlling the impulse to continue play fighting after the bully to participate in the study; however, only those children with
apologized). We also aligned our measure of preschool learning- parental consents were included in analyses. Since there was a
related behaviors with the expectations of the impact of ST/SA, large subset of school-district identified ELL students, we admin-
measuring attitudes toward learning, attention, and confidence. istered the pre-LAS 2000 (Duncan & De Avila, 1998), a language
Below, we provide initial data to establish the psychometric screener used in other studies of low-income preschool age
rigor of the PTAR. We present findings on internal consistency, children (e.g., Head Start FACES), to all participants, both ELL and
inter-rater reliability, and construct validity (with measures of non-ELL, to ensure that they possessed sufficient proficiency to
preschool-learning related behaviors, expressive communication, complete the expressive language portion of our assessment. A
grammatical complexity, and storytelling quality). We also present subset of 128 preschoolers successfully passed the pre-LAS 2000
data demonstrating the PTAR’s sensitivity to change. and were included in analyses validating PTAR skills in relation
to expressive communication. Sample size varied by assessment
2. Method since there were consented children who did not participate in all
assessment activities at both time 1 (pre-intervention) and time
2.1. Program
2 (post-intervention) based on language constraints, attendance
In each of two consecutive years, five preschool classrooms (10 during program sessions, or willingness to engage in program
classrooms total) in four elementary schools (three conventional activities at the time of assessment (e.g., storyacting).

Please cite this article in press as: Susman-Stillman, A., et al. Reliability and validity of a measure of preschool children’s theatre arts
skills: The Preschool Theatre Arts Rubric. Early Childhood Research Quarterly (2018), https://doi.org/10.1016/j.ecresq.2017.12.001
G Model
EARCHI-978; No. of Pages 14 ARTICLE IN PRESS
A. Susman-Stillman et al. / Early Childhood Research Quarterly xxx (2018) xxx–xxx 7

2.3. Procedures are acting out their original, dictated stories (Appendix A). Each of
the scale points for each of the scales has examples provided in
We employed a pre-post short-term longitudinal design to the scoring criteria. The PTAR is designed to be used over time, and
examine the reliability and validity of the PTAR. Each year for two scores can be used to gauge a child’s beginning proficiency with
consecutive school years, data were collected prior to the children these theatre arts skills, as well as their increasing proficiency over
receiving the theatre arts program (time 1) and again when they time.
completed the program (time 2), approximately 5 months between
assessments. 2.4.2. Battelle Developmental Inventory, Second Edition
Data from four sources were collected in this study. First, prior (BDITM -2)
to the beginning of the theatre arts program, trained graduate stu- The BDI -2 (Newborg, 2004) is a developmental assessment tool
dents in child or school psychology with assessment experience for preschoolers. Trained graduate students used the expressive
individually screened each consented child for minimal English communication subscale at time 1 to assess children’s verbal skills
verbal skill competence and assessed expressive language skills. prior to program delivery. Normative data using 2500 children
Children who did not meet the English language screening crite- between the ages of birth through 7 years, 11 months indicated
ria were excluded from the expressive language assessment, but adequate test-retest reliability scores for the Expressive Commu-
were included in other data collection activities (e.g., storyacting nication subscale (␣ = .92).
observation). Second, as part of the theatre arts program, students
were asked by their classroom teacher to dictate an original story.
2.4.3. Preschool learning behaviors scales (PLBS)
The student stories dictated at the beginning and end of the 19-
Classroom teachers completed the PLBS (McDermott, Green,
week program were collected for the storyacting analyses reported
Francis, & Stott, 2000) for each child at time 1. The PLBS consists
herein. The research team coded those stories separately for sto-
of 29 items that correspond to three learning-related behavioral
rytelling quality and grammatical complexity. Students dictated
dimensions: (a) competence motivation (interest in learning-
additional stories that were used for storyacting during the course
related activities); (b) attention/persistence (attention and focus);
of the program. Third, trained observers used the PTAR to code
and (c) attitude (ability to cooperate and accept help, as well as
students acting out their stories with classmates during the first
tendency to express frustration when faced with difficult tasks).
three and last three sessions to facilitate observing as many chil-
Teachers rated items on a three-point scale. Internal consistency
dren as possible acting pre-and-post-program. A maximum of three
estimates indicated adequate reliability for the full PLBS scale
children were coded at one time by two observers for reliability
(␣ = .88) and each of the three dimensions (␣ = .85, .83, and .75
purposes. Each class had approximately 13–15 sessions (approxi-
for competence motivation, attention/persistence, and attitude
mately 4–5 months) between pre- and post-story data collection.
toward learning, respectively), and adequate test-retest and inter-
Fourth, classroom teachers rated children’s approaches to learning
rater reliability (McDermott, Leigh, & Perry, 2002). Findings from
at the beginning of the program via surveys.
additional studies also support the use of PLBS with minority
The dictated student stories from both time 1 and time 2 were
children and children from low-income backgrounds (Fantuzzo,
coded for their grammatical complexity and storytelling quality by
Bulotsky-Shearer, McDermott, & McWayne, 2007; Fantuzzo, Perry,
trained, reliable coders (N = 273 stories). To ensure story coders
& McDermott, 2004; McDermott et al., 2002). In this study, the
were unaware of the timing of the story collection or the child
overall mean was 57.24 and SD was 7.13.
author, stories were blinded and randomly assigned to coders at
the conclusion of the program.
Training for the PTAR consisted of coders first learning cate- 2.4.4. Narrative Assessment Protocol, Short Form (NAP-SF)
gory definitions and examples, and then coding the PTAR scales Trained coders assessed children’s stories obtained at time 1 and
individually using videotaped sessions of ST/SA as practice. The time 2 using the NAP-SF (Pence, Justice, & Gosse, 2007). The NAP-SF
coders then met as a group to discuss codes; reliability was assessed is used to assess a sample of language skills in an ecologically valid
throughout the training and proceeded until the coders met an context–a narrative—from which narrative microstructure, the
acceptable level of reliability (␣ = .70) on each of the scales. We syntactical, morphological, and lexical structures, is coded (Justice
assessed reliability four times, before each data collection period; et al., 2010, p. 219). The NAP-SF includes 12 items organized into
at time 1 and time 2 in year 1 and at time 1 and time 2 in year five domains: (a) sentence structure, (b) phrase structure, (c) mod-
2. Particularly in the first year, we continued to add examples to ifiers, (d) nouns, and (e) verbs. Inter-rater reliability for the NAP-SF
ensure clarity for each scale and scale point. This facilitated our ranged from .75 to 1.00, with the majority being .90 or greater
reliability training such that by the second year our team achieved (Pence et al., 2007). Sufficient inter-rater reliability was also estab-
reliability on all scales consistently after 2 reliability sessions. Thus, lished by the current research team (range = .77–1.00). Individual
we are confident that obtaining reliability can be achieved within domain scores were summed and averaged for each story, yield-
a reasonable period of time. ing a summary score used in analyses. The NAP-SF demonstrated
Once reliability was achieved, two coders rated the same chil- sufficient concurrent validity with key language assessments (Wiig,
dren during the live observational assessments to obtain reliability Secord, & Semel, 2004) and is strongly correlated with the NAP Long
estimates reported herein. Coders rated up to three children at one Form (NAP-LF; Justice et al., 2010). Coders were trained to accept-
time. The coders then discussed each child’s scores directly after able levels of reliability (␣ = .70–1.00) before coding stories each
the observation and consensus codes were used in analyses. year.

2.4. Measures 2.4.5. Preschool Storytelling Skills Rubric (PSSR)


The PSSR (Susman-Stillman & Asp, 2013), developed specifically
2.4.1. Preschool Theatre Arts Skills Rubric (PTAR) for this project, is a coder-rated measure used to gauge the quality
Trained observers used the PTAR (Susman-Stillman, Asp, & of student storytelling. The PSSR measures six aspects of student
Beckstrom, 2013) to rate children’s theatrical skills – independence stories: (a) the use of emotion language (e.g., presence of identified
in role play, children’s focus, gesture and movement, collaboration emotional state(s) in the story); (b) creativity (e.g., story demon-
with others, and overall theatrical quality of the acting–on a 3-point strates imagination rather than merely repetition of a familiar story,
scale, from “emerging” to “developing” to “achieved” when children unique descriptions, or examples, instance of problem solving);

Please cite this article in press as: Susman-Stillman, A., et al. Reliability and validity of a measure of preschool children’s theatre arts
skills: The Preschool Theatre Arts Rubric. Early Childhood Research Quarterly (2018), https://doi.org/10.1016/j.ecresq.2017.12.001
G Model
EARCHI-978; No. of Pages 14 ARTICLE IN PRESS
8 A. Susman-Stillman et al. / Early Childhood Research Quarterly xxx (2018) xxx–xxx

Table 2
Bivariate Correlations of PTAR Scales Time 1 (lower diagonal) & Time 2 (upper diagonal).

Target Skill Indep. in role play Gestures & Focus/Persistence Collaboration (T1: Overall Theatrical Time 2: Mean (SD)
(T1: N = 126; T2: Movement (T1: (T1: N = 126; T2: N = 125; T2: Quality(T1: N = 126;
N = 141) N = 126; T2: N = 141) N = 141) N = 141) T2: N = 141)

Indepen-dence in role play – .57*** .29** .65*** .56*** 2.17 (.69)


Gestures & Movement .55*** – .21* .53*** .63*** 2.46 (.74)
Focus/Persistence .20* .15 – .33*** .27** 2.57 (.59)
Collaboration .47*** .33*** .22* – .63*** 2.05 (.70)
Overall Theatrical Quality .60*** .53*** 0.11 .42*** – 1.71 (.71)
Time 1: Mean (SD) 1.83 (.70) 1.99 (.78) 2.80 (.46) 1.85 (.57) 1.37 (.55)

Note: *p < .05; **p < .01; ***p < .001.

(c) reflective thought (e.g., insightful thinking or emotional under- time 1 would be significantly correlated, as follows: (a) the PTAR’s
standing of character, such as bowing their heads and removing focus/persistence in commitment to play scale with the PLBS’ atten-
their hats when the dinosaur died, or explaining that the character tion/persistence scale, and (b) the PTAR’s independence in role play
was afraid because she didn’t like sleeping in the dark); (d) narrative with the PLBS’ competence motivation. We also expected that those
structure and detail (e.g., story sequence with a beginning, middle, children who have higher expressive communication (BDI) and nar-
and end, mentions characters, setting, time); (e) story coherence rative language skills (NAP) would be more likely to have higher
(e.g., story makes sense); and (f) overall length (word count). Chil- levels of independence in role play, more expression in their body
dren’s written stories are rated in each category using a 3-point gestures and movement, and higher overall theatricality scores.
scale with 0 indicating not observed in the story and 2 indicat- Furthermore, we expected that children who had greater devel-
ing clearly observed in the story. Sufficient inter-rater reliability oped storytelling abilities, especially as it relates to their narrative
was established for all categories by the research team each year detail and creativity, would be more likely to have higher levels of
(range = .68–1.00). independence in role play and theatricality.
Fourth, demographic data (ELL status) obtained at time 1 and
expressive language scores from the BDI (adjusting for child’s age
2.5. Data analysis plan
in months) were included as control variables to examine the ability
of the PTAR to assess sensitivity to changes from time 1 to time 2
Data analysis to examine the psychometric properties of the
independent of language skill. We examined the PTAR’s sensitivity
PTAR proceeded as follows. First, we examined the internal con-
to change by conducting repeated measures anovas of each of the
sistency of the PTAR based on Cronbach’s Alpha and patterns of
PTAR scales controlling for the children’s expressive language at
intercorrelations.
time 1.
Second, we assessed interrater reliability using two-way mixed,
single measures, consistency intraclass correlation coefficients
(ICC) conducted separately for the PTAR observations for each time 3. Results
point in each year. As indicated by Hallgren (2012), the two-way
mixed, single measures, consistency ICC is interchangeable with 3.1. Internal consistency
a weighted kappa with quadratic weights for ordinal scales and
can accommodate more than two coders. The ICC is commonly Based on Cronbach’s Alpha, internal consistency for the PTAR
used to assess inter-rater reliability when multiple coders rate a was in the acceptable range for the full sample at time 1 (␣ = .74)
subset of participants; whereas a weighted kappa can only accom- and good internal consistency (␣ = .81) was established at time 2
modate two coders. The single measures ICC is an appropriate when including all five scales in the analyses.
statistics given that we intended to generalize the reliability of In addition to examining internal consistency via Cronbach’s
the ratings to those scores rated by one coder based on a sub- Alpha, we also explored the pattern of correlations among the
set of participants rated by multiple coders. The single measures scales. Bivariate correlation coefficients for the PTAR observational
ICC, however, results in lower estimates than the average mea- rating scales are presented in Table 2. All of the scales were sig-
sures ICC or weighted kappa. Cicchetti (1994) provides guidelines nificantly correlated with each other at time 1 with the exception
for evaluating the intraclass correlation statistic based on guide- of the focus/persistence scale, which only significantly correlated
lines developed by a number of biostatisticians. These guidelines with independence in role play and collaboration (Table 2). At time
state that correlation coefficients of .40–.59 are considered fair, 2, all of the scales were significantly correlated with each other;
correlation coefficients of .60–.74 are considered good, and corre- these associations were also stronger at time 2 than they were at
lation coefficients of .75–1.00 are considered excellent inter-rater time 1. These patterns of intercorrelations suggest that children
reliability estimates. who demonstrate greater independence in their role play are also
Third, construct validity was examined by considering both more likely to use gestures and movement, collaborate with oth-
convergent and discriminant validity. We calculated partial cor- ers, and demonstrate theatricality when they act, and that those
relations of the PTAR scales with measures of similar and divergent relations are stronger when the children have more experience
constructs, controlling for gender, ELL status, and age in months. We engaging in role play and storyacting. This is further confirmation
included measures of early childhood skills similar to the theatre that the theatre arts skills in the PTAR relate in a meaningful way.
arts skills we were interested in measuring (e.g., learning-related
social skills). These partial correlations were run between the PTAR 3.2. Reliability
scales and (a) learning-related social skills as measured by the
PLBS subscales, (b) expressive language as measured by the BDI- Results from the ICC suggest that we were able to obtain accept-
2 expressive communication subscale, (c) narrative language skills able reliability for all of the scales coded at almost all time points
as measured by the NAP, and (d) the quality of students’ storytelling (r = .23–.93, with the majority of the scales at 0.70 or above—good
as measured by the PSSR. We expected that the PTAR scales from to excellent reliability; Table 3). At year 1 time 1, reliability
time 1 that measured similar constructs as the PLBS subscales at for focus/persistence and collaboration were low (r = .40 and .23,

Please cite this article in press as: Susman-Stillman, A., et al. Reliability and validity of a measure of preschool children’s theatre arts
skills: The Preschool Theatre Arts Rubric. Early Childhood Research Quarterly (2018), https://doi.org/10.1016/j.ecresq.2017.12.001
G Model
EARCHI-978; No. of Pages 14 ARTICLE IN PRESS
A. Susman-Stillman et al. / Early Childhood Research Quarterly xxx (2018) xxx–xxx 9

Table 3 Table 4
PTAR Interrater Reliabilities (Two Way Mixed, Single Measures, Consistency Intra- Partial Correlations controlling for gender, ELL status, and age in months between
class Correlation Coefficients) for Year 1 and Year 2 at T1 and T2. PTAR and BDI-2 Scaled Score and Narrative Assessment Protocol Time 1.

Year 1 Year 2 PTAR Target Skills Battelle Developmental Narrative Assessment


Inventory Scaled Score Protocol Summary
T1 T2 T1 T2 (controlling for gender & Score (N = 108)
Independence in role play .61 .64 .75 .81 ELL status only)a (N = 99)
Gestures & Movement .72 .82 .85 .93 Independence in Role Play .20* .26**
Focus/persistence .40 .55 .76 .80 Face/Gestures .12 .23*
Collaboration .23 .74 .70 .81 Focus/Persistence .12 .05
Overall theatrical quality .61 .76 .89 .79 Collaboration .00 .08
Theatricality 0.11 .34***

respectively), but we worked with our research team to refine our Note: *p < .05; **p < .01; ***p < .001.
a
The BDI Scaled Score incorporates the childs’ age at the time of assessment into
definition of the scales by adding examples for each of the scale
the score, therefore, we only controlled for gender and ELL status.
points to clarify coding. The focus/persistence and collaboration
scores improved to a higher level at time 2 in Year 1 (r = .55 and .74,
respectively; fair to good reliability) and reliability for these scales role play on the PTAR, also measured at time 1. This finding, that
were good to excellent at both time points in Year 2 (r = .70–.81). children’s expressive communication is related to their abilities to
These findings are positive and suggest that the PTAR is a tool that be independent in their role play provides partial support for the
observers can be trained to use reliably in a live coding situation. convergent validity of the PTAR in measuring expressive commu-
nication and supports our hypothesis regarding independence in
3.3. Construct validity role play. These results also suggest divergent validity given that
the constructs of focus/persistence and collaboration are reflective
The PLBS and the BDI were only collected at time 1; therefore, of theatre arts skills related to attention and social relations rather
correlations between these measures and the PTAR are only pre- than expressive communication.
sented for time 1. However, the NAP and PSSR were collected at
both time points, so correlations between these measures and the 3.3.3. Grammatical complexity
PTAR are included for both time 1 and time 2. The partial correlations between the PTAR scales and the NAP
controlling for gender, ELL status, and age in months are also shown
3.3.1. Preschool learning behaviors in Table 4 for the full sample. As expected, the summary score,
We examined partial correlations of the PLBS scales with the which reflects the complexity of children’s narrative microstruc-
PTAR scale scores controlling for gender, ELL status, and age in ture (e.g., sentence structure, phrase structure, nouns, and verbs) is
months. Four individuals who were outliers on the PLBS total scale significantly correlated with their ability to independently engage
score (more than 2 standard deviations below the mean) were in role play; their use of their face, body, or gestures to enhance
removed from analyses; three of these individuals were non-ELL the story; and theatrical quality. The NAP did not correlate with
students and one was an ELL student; two were girls and two were focus/persistence or collaboration. These results provide further
boys. The only significant correlation for the full sample is that, as support for both the convergent and discriminant validity of the
expected, the PTAR focus/persistence scale was correlated with the PTAR. Children’s expressive language skill is indicative of role play-
attention/persistence scale on the PLBS (r = .21, p = .03). Although ing skills and theatrical quality when acting, as well as the use of
we also expected the PTAR independence in role play scale to be non-verbal actions to communicate, but children’s expressive lan-
correlated with the PLBS competence motivation scale, this partial guage skill does not necessarily relate to their ability to focus their
correlation was not significant. These results not only provide initial attention or their ability to collaborate with others.
support for the convergent validity of the Focus/Persistence scale
of the PTAR, but also provides support for the discriminant valid- 3.3.4. Storytelling quality
ity of the PTAR, demonstrating that the focus/persistence scale is Table 5 shows the results for the concurrent partial correlations
measuring the construct that it is intending to measure, but is not for time 1 between the PTAR skills and the PSSR scales. Table 6
measuring other related constructs. shows the results for the concurrent partial correlations for time 2
between the PTAR skills and the PSSR scales. As expected, at both
3.3.2. Expressive communication time points, children’s independence in role play and theatricality
Table 4 shows the results of the partial correlations between were correlated with features of storytelling quality, most consis-
the PTAR scales and the BDI expressive communication scale at tently narrative detail and creativity. Children’s independence in
time 1. Given that the BDI scaled score takes into account the role play while acting out stories was significantly related to their
age of the child, we only controlled for gender and ELL status in narrative detail, creativity, and word count in their narrated stories
the partial correlations. For the full sample, expressive communi- at both time 1 and time 2. In addition, at time 2, children’s indepen-
cation at time 1 is significantly correlated with independence in dence in role play was also related to narrative structure, reflective

Table 5
Partial Correlations controlling for gender, ELL status, and age in months between PTAR and PSSR Time 1.

Preschool Storytelling Skills Rubric (N = 97)

PTAR Target Skills Emotion Language Narrative Structure Narrative Detail Reflective Thought Creativity Story Coherence Word Count

Independence in role play .05 .14 .31** .07 .23* .09 .20+
Face/Gestures .07 .09 .15 .08 .26* .12 .12
Focus/Persistence .07 .06 .16 .09 −0.00 .05 .13
Collaboration −.06 .13 .25* .07 .21* .12 .17
Overall Theatrical Quality −.06 .17+ .35*** .11 .30** .13 .19+

Note:+p < .10; *p < .05; **p < .01; ***p < .001.

Please cite this article in press as: Susman-Stillman, A., et al. Reliability and validity of a measure of preschool children’s theatre arts
skills: The Preschool Theatre Arts Rubric. Early Childhood Research Quarterly (2018), https://doi.org/10.1016/j.ecresq.2017.12.001
G Model
EARCHI-978; No. of Pages 14 ARTICLE IN PRESS
10 A. Susman-Stillman et al. / Early Childhood Research Quarterly xxx (2018) xxx–xxx

Table 6
Partial Correlations controlling for gender, ELL status, and age in months between PTAR and PSSR Time 2.

Preschool Storytelling Skills Rubric (N = 133)

PTAR Target Skills Emotion Language Narrative Structure Narrative Detail Reflective Thought Creativity Story Coherence Word Count

Independence in role play .08 .24** .26** .16+ .22* .20* .23**
Face/Gestures .07 .14 .18* .11 .04 .18* .01
Focus/Persistence −.14 .03 .02 −.08 −.02 −.00 −.05
Collaboration .04 .07 .17+ .11 .23* .12 .13
Overall Theatrical Quality −.02 .16+ .20* .17* .24** .12 .17+

Note: +p < .10; *p < .05; **p < .01; ***p < .001.

thought (marginally), and story coherence. Children’s use of their The data presented in this study indicate promising findings
face, body, or gestures to enhance their acting out stories and chil- related to both reliability and validity. Our measures of internal
dren’s collaboration during the observations are both related to consistency and inter-rater reliability met or exceeded reliability
the creativity and narrative detail shown in their narrated stories standards, indicating that the PTAR is a measurement tool that can
at time 1; however, only collaboration was significantly related be used reliably by observers. This is a strength of the PTAR; it is
with creativity at time 2. Furthermore, children’s theatrical quality relatively easy to use and obtain reliability.
observed during storyacting was significantly correlated with chil- Bivariate inter-correlations of the PTAR items indicate mod-
dren’s narrative structure, narrative detail, reflective thought, and erate, positive relations among them. Children who were more
word count of children’s narrated stories at time 1; and with chil- independent in their role play were more likely to demonstrate
dren’s narrative detail, reflective thought, and creativity at time 2. higher levels on a number of theatrical skills. Children who demon-
Theatricality was correlated at both time 1 and time 2 with narra- strated higher levels of independence in their role play were more
tive detail and creativity. likely than those who had lower levels of independence in their
role play to use gestures and their bodies, collaborate with their fel-
low actors, and demonstrate a sense of theatricality in their acting.
3.4. Sensitivity to change These relations grew stronger with more exposure to the theatre
arts program, although the lack of a control group makes it difficult
As indicated in Table 7, significant change was observed from to sort out whether maturation or exposure to the theatre arts pro-
time 1 to time 2 for all of the PTAR skills. Of note, however, is that the gram contributed to the increased strength of the relations over
children’s focus/persistence decreased from time 1 to time 2 rather time. We obtained similar patterns with and without controlling
than increasing as is shown in all of the other scales. Nonetheless, for expressive language skill, suggesting that language proficiency
these results provide preliminary support for the abilities of the does not affect the relations among the acting skills, and that we
PTAR to identify change across time. are measuring a set of skills that are not solely reliant on verbal
expression.
We examined two aspects of construct validity, convergent
4. Discussion
validity and divergent validity, between the PTAR and measures
of learning-related social behaviors, expressive communication,
With the resurgence of attention to arts integration in early
and storytelling quality. The analyses established limited construct
childhood education, and its potential to improve the quality of
validity between the PTAR and learning-related social behaviors,
early care and education, engage early learners, and improve chil-
with significant positive relations between focus/persistence on the
dren’s developmental outcomes, a new generation of research on
PTAR and attention on the PLBS. This finding, which demonstrated
the impact of the arts is dawning. However, research examin-
convergent validity, is noteworthy because the ratings of attention
ing the developmental processes and potential benefits of theatre
on the PLBS and persistence on the PTAR were conducted indepen-
arts/creative drama on young children’s development suffers from
dently (one by a reliable observer, the other by classroom teacher
a lack of psychometrically rigorous measures of arts skills (Mages,
report using a validated paper and pencil measure) and under dif-
2008, 2015). This measurement gap limits the ability of researchers
ferent circumstances (e.g., during storyacting and over the entire
to document the extent to which theatre arts affect children’s
school day). Furthermore, the focus/persistence scale on the PTAR
development and examine potential mechanisms by which they
did not relate to other PLBS subscales, indicating divergent validity.
may support children’s outcomes. The work described herein builds
Although we expected additional relations between the PTAR and
on the unique integration of the fields of theatre arts and early
attitudes toward learning and competence motivation on the PLBS,
childhood education. It is the first study of which we are aware to
we obtained a higher mean and a smaller standard deviation than
develop a measure of children’s preschool theatre arts skills, thus
McDermott et al. (2002) did with their validation sample, which
contributing to a growing literature on early childhood develop-
may have made it more difficult for us to detect other potential
ment and the arts, and facilitating continued research in this area.
relations. Furthermore, due to resource constraints, we obtained a
We introduced the Preschool Theatre Arts Rubric (PTAR), an obser-
single sample of each child’s acting, which may have also limited
vational tool assessing five preschool theatre arts skills, and present
our ability to detect relations with the PTAR and the PLBS. As theatre
initial results documenting its internal consistency, reliability, con-
arts skills may positively impact learning-related skills such as con-
struct validity, and its sensitivity to change. Psychometrically sound
fidence, attitudes toward learning, and self-regulation (McNamee,
measures that are easy to use, like the PTAR, can help early child-
2015), continued research is needed to explore this potential rela-
hood researchers, teachers, program developers, and theatre artists
tion with other validated measures of those skills and with multiple
explore the potential impact of early childhood theatre arts pro-
samples of acting skills per child.
gramming on young children’s school readiness and gauge program
The analyses validating the PTAR with measures of expres-
effectiveness. In addition to being easy to use, the PTAR does not
sive communication (a standard and authentic measure) suggested
require specific training in theatre arts, so those with other areas
convergent validity between expressive communication and inde-
of expertise, for instance, in early childhood development or edu-
pendence in role play, use of face and gesture, and, and divergent
cation, can use it.

Please cite this article in press as: Susman-Stillman, A., et al. Reliability and validity of a measure of preschool children’s theatre arts
skills: The Preschool Theatre Arts Rubric. Early Childhood Research Quarterly (2018), https://doi.org/10.1016/j.ecresq.2017.12.001
G Model
EARCHI-978; No. of Pages 14 ARTICLE IN PRESS
A. Susman-Stillman et al. / Early Childhood Research Quarterly xxx (2018) xxx–xxx 11

Table 7
Repeated Measures Anovas: PTAR Scores Pre-Post Differences (controlling for expressive communication and ELL status).

Support for Role Play Face/Gesture Focus/Persistence Collaboration Overall Theatrical quality

T1 T2 T1 T2 T1 T2 T1 T2 T1 T2

Full sample (N = 95) 1.87 (.69) 2.25*** (.67) 2.06 (.77) 2.51*** (.73) 2.82 (.44) 2.63** (.55) 1.87 (.59) 2.09* (.70) 1.41 (.57) 1.80*** (.74)

Note: *p < .05; **p .01 < .01; ***p < .001.

validity with focus/persistence and collaboration. Thus, these con- 4.1. Future directions
sistent findings document relations between theatre arts skills and
expressive communication skills, indicating that in their own right, These initial reliability and validity results offer indications of
theatre arts skills are related to important developmental skills next steps to continue to establish sound psychometric proper-
such as expressive communication. Given that most of the stud- ties of the PTAR. One important consideration is the representative
ies of narrative skills have not included aspects of drama (Wright nature of the sample. Our sample included a large proportion of
et al., 2013), and we were unable to locate a measure of story- ELL students. It is important to include ELL students in a valida-
telling quality that was well-aligned with the theatre arts program tion study, as there is a growing literature illustrating that theatre
and features of the stories we were interested in measuring, we arts provides innovative strategies for teaching young children lan-
used an author-developed measure of storytelling quality. With our guage and preliteracy skills (Greenfader et al., 2015). Thus, there is
measure of storytelling quality, we were able to demonstrate pos- a continuing need to document how participation in theatre arts
itive relations between theatre arts skills and not only narrative affects skills of ELL students. However, with the increasing presence
detail, but also creativity and to a lesser extent, reflective thought. of theatre arts in early childhood education more generally, con-
Thus, there is a relation between the quality of the acting and the tinuing efforts to validate the PTAR with larger samples of non-ELL
quality of the story, such that the quality of the story may pro- students and students with varying demographic characteristics is
vide more opportunity for children to demonstrate acting skills, warranted, and would allow for exploration of potential differences
or alternatively, children with stronger acting skills may also tell in the validity of the tool by language status or early childhood
higher quality stories. experiences.
Our last test of validity, sensitivity to change, resulted in changes A second consideration in continuing the validation of the PTAR
in all five of the theatre arts skills measured while controlling for is the choice of measures for validation. To continue construct
expressive communication skills. Changes for four of the scales validation, testing with some of the newer arts-related measures
were positive, and changes for one of the scales, focus/persistence, for preschoolers that may align appropriately with the PTAR (art-
was negative. Focus/persistence is a scale where children initially fulness in stories; Glenn-Applegate et al., 2010; creativity, Yeh
tended to score higher, and the slight decline, although statistically & Li, 2008) appears to be a promising next step. To continue
significant, still resulted in a relatively high score. Initially higher to examine predictive validity, it may be useful to think about
scores may be due to their unfamiliarity with, but strong interest the evolving conceptualization of the relations of arts skills to
in, learning ST/SA, and the slight decline may be due to a greater other relevant developmental outcomes, such as socioemotional
level of comfort or familiarity with the ST/SA practices such that development/learning-related social behaviors, or executive func-
there were some minor lapses in attention. This negative change tions. The current body of work investigating the impact of arts
indicated that both positive and negative change can be captured on preschool socioemotional outcomes is limited and offers mixed
by the tool. Overall, the sensitivity to change analysis indicated that results about the benefits (Goldstein, 2015; Menzer, 2015). Further
the PTAR is sensitive to change over time. Therefore, the PTAR can research to document predictive validity and potential relations
be used to document change in children’s theatre arts skills. between theatre arts skills and relevant social-emotional outcomes
Here, we describe initial research conducted to establish relia- is recommended.
bility and validity for the PTAR using measures of oral narrative A third consideration is exploring the contexts in which the
skills and one measure of learning-related social behaviors. The PTAR can be used. Most reports of children’s growth or progress in
majority of work examining the impact of the arts on young chil- the context of ST/SA highlight the success of children’s storytelling
dren’s development has focused on documenting change in oral and storyacting anecdotally, qualitatively, or through the use of
narrative skills without attention to the development of the arts case studies (Paley, 1990; Wright et al., 2013) or document the rela-
skills themselves; limited work with preschoolers has connected tions between ST/SA and standardized test measures (Cooper et al.,
arts skills with relevant early developmental skills. Importantly, 2007). The PTAR allows for a quantitative way to measure theatre
we documented that a measure of theatre arts skill is linked to arts skills in the context of ST/SA. However, as skills indexed by the
expressive communication skills, indicating that there is value in PTAR are core theatre arts skills, exploring the viability of the PTAR
the measurement of theatre arts skills themselves as well as docu- to observe these skills during other kinds of early childhood theatre
menting transfer of skills from arts to a non-arts contexts. activities, such as drama-based pedagogical techniques (Lee et al.,
As theatre arts continues to become more intentionally inte- 2015) or literature-based dramatizations (Goldstein, 2015), would
grated into early childhood education, having measures that can be informative. Furthermore, because the core theatre arts skills
document the presence and growth of theatre arts skills will be are also key early childhood skills, another potential extension of
helpful in understanding which theatre arts skills are being effec- the use of the PTAR to explore is during dramatic playtime in early
tively cultivated by specific programs. Furthermore, they afford the childhood settings.
opportunity to learn about the developmental trajectory of theatre Variations in the intensity of the exposure children receive to
arts skills in young children and the similarities and differences in theatre arts or creative drama activities in those contexts are also a
those trajectories in comparison to early childhood skills, such as consideration. The PTAR initially evolved from a need to document
pretend play, as well as explore factors that may affect those trajec- drama skills expressed during ST/SA, implemented via a TA coming
tories, such as child temperament or child self-efficacy. This kind into the classroom. It will be worthwhile to see if the PTAR can effec-
of information can also be used to support increasing use of theatre tively capture skill growth in other kinds of drama programs similar
arts in early childhood education settings. in intensity. Furthermore, as theatre arts or ST/SA practices become

Please cite this article in press as: Susman-Stillman, A., et al. Reliability and validity of a measure of preschool children’s theatre arts
skills: The Preschool Theatre Arts Rubric. Early Childhood Research Quarterly (2018), https://doi.org/10.1016/j.ecresq.2017.12.001
G Model
EARCHI-978; No. of Pages 14 ARTICLE IN PRESS
12 A. Susman-Stillman et al. / Early Childhood Research Quarterly xxx (2018) xxx–xxx

incorporated alongside different preschool curricula (Cooper, 2009; Blair, C., & Razza, R. P. (2007). Relating effortful control, executive function, and
Mardell, 2013; McNamee, 2015; Nicolopoulou et al., 2015), it will false belief understanding to emerging math and literacy ability in
kindergarten. Child Development, 78(2), 647–663. http://dx.doi.org/10.1111/j.
also be important to test the success of the PTAR in gauging the 1467-8624.2007.01019.x
growth of theatre arts skills in programs where children’s expo- Bodrova, E., & Leong, D. J. (1996). Tools of the mind: The Vygotskian approach to early
sure is more intense We encourage colleagues to use the PTAR, childhood education. Upper Saddle River, NJ: Prentice-Hall.
Bodrova, E., & Leong, D. J. (2001). Tools of the mind: A case study of implementing the
which is available free of charge from the first author, to continue Vygotskian approach in American early childhood and primary classrooms.
its development and expand our knowledge about its usefulness. (UNESCO innodata monographs: Educational innovations in action no. 7). Geneva,
Switzerland: International Bureau of Education, UNESCO. Retrieved from.
http://files.eric.ed.gov/fulltext/ED455014.pdf
5. Conclusion Bodrova, E., & Leong, D. J. (2007). Tools of the mind. Upper Saddle River, NJ: Pearson.
Brouillette, L. (2012). Advancing the speaking and listening skills of K–2 English
Increasingly, there is interest in integrating theatre arts language learners through creative drama. TESOL Journal, 3(1), 138–145. http://
dx.doi.org/10.1002/tesj.8
practices into early childhood education. States are describing age- Butler, L. P., & Walton, G. M. (2013). The opportunity to collaborate increases
related expectations for skills in the arts, and moving toward preschoolers’ motivation for challenging tasks. Journal of Experimental Child
accountability for those skills in addition to the more tradi- Psychology, 116(4), 953–961. http://dx.doi.org/10.1016/j.jecp.2013.06.007
Carlson, S., Zelazo, P., & Benson, J. B. (2008). Symbolic thought. In M. M. Haith (Ed.),
tional early childhood skills. Improving our understanding of the Encyclopedia of infant and early childhood development (pp. 288–297). San
impact and mechanisms by which theatre arts can affect children’s Diego, CA: Academic Press.
developmental outcomes is critical. As a result, there is a strong Carlson, S. M., Zelazo, P. D., & Faja, S. (2013). Executive function. In P. D. Zelazo
(Ed.), The Oxford handbook of developmental psychology, vol. 1: Body and mind
need for reliable and valid measurement tools. The PTAR, a new (pp. 706–743). New York, NY: Oxford University Press.
observational measure of children’s preschool theatre arts skills, Carlson, S. M., White, R. E., & Davis-Unger, A. C. (2014). Evidence for a relation
contributes to this growing arena of work with this initial exam- between executive function and pretense representation in preschool children.
Cognitive Development, 29, 1–16. http://dx.doi.org/10.1016/j.cogdev.2013.09.
ination of its reliability and validity with measures of expressive 001
communication and preschool learning-related behaviors. While Carlson, S. M. (2005). Developmentally sensitive measures of executive function in
more research is needed, the PTAR is a promising tool to observe preschool children. Developmental Neuropsychology, 28(2), 595–616. http://dx.
doi.org/10.1207/s15326942dn2802 3
preschool children’s theatre arts skills in research, classroom, and
Chan, E. K. H. (2014). Standards and guidelines for validation practices:
programmatic contexts. Development and evaluation of measurement instruments. In B. D. Zumbo, &
E. K. H. Chan (Eds.), Validity and validation in social, behavioral, and health
sciences, social indicators research series (vol. 54) (pp. 9–20). Switzerland:
Acknowledgements Springer International. http://dx.doi.org/10.1007/978-3-319-07794-9 2
Children’s Theatre Company. (2012). Neighborhood bridges acting, collaboration, &
The authors would like to acknowledge the contributions of public speaking rubric. Minneapolis, MN.
Cicchetti, D. V. (1994). Guildelines, criteria: and rules of thumb for evaluating
Maria Asp, Emily Beckstrom and Rachel Austin Bernstein to the normed and standardized assessment instruments in psychology. Psychological
PTAR, and the helpful assistance of Alyssa Meuwissen and Jessica Assessment, 6, 284–290.
Koolick in data collection and coding. Colorado Department of Education. (2009). Colorado academic standards for drama
and theatre arts: Preschool and kindergarten. [Retrieved from]. https:// www.
This project was supported in part by an award from the cde. state. co. us/ coarts/ statestandards#Drama
Research: Art Works program at the National Endowment for the Colwell, M. J., & Lindsey, E. W. (2005). Preschool children’s pretend and physical
Arts: Grant # 14-3800-7009 and matching grant funds from the play and sex of play partner: Connections to peer competence. Sex Roles, 52(7),
497–509. http://dx.doi.org/10.1007/s11199-005-3716-8
Children’s Theatre Company.
Connolly, J. A., & Doyle, A. B. (1984). Relation of social fantasy play to social
The opinions expressed in this paper are those of the author(s) competence in preschoolers. Developmental Psychology, 20(5), 797–806. http://
and do not represent the views of the Office of Research & Analysis dx.doi.org/10.1037/0012-1649.20.5.797
or the National Endowment for the Arts. The NEA does not guar- Cook, S. W., Mitchell, Z., & Goldin-Meadow, S. (2008). Gesturing makes learning
last. Cognition, 106(2), 1047–1058. http://dx.doi.org/10.1016/j.cognition.2007.
antee the accuracy or completeness of the information included in 04.010
this paper and is not responsible for any consequence of its use. Cooper, P. M., Capo, K., Mathes, B., & Gray, L. (2007). One authentic early literacy
practice and three standardized tests: Can a storytelling curriculum measure
up? Journal of Early Childhood Teacher Education, 28(3), 251–275. http://dx.doi.
Appendix A. Supplementary data org/10.1080/1090102070155556
Cooper, P. M. (2009). The classrooms all young children need: Lessons in teaching
from Vivian Paley. Chicago, IL: University of Chicago Press.
Supplementary data associated with this article can be found, Denham, S. A., Zoller, D., & Couchoud, E. A. (1994). Socialization of preschoolers’
in the online version, at https://doi.org/10.1016/j.ecresq.2017.12. emotion understanding. Developmental Psychology, 30(6), 928–936. http://dx.
001. doi.org/10.1037/0012-1649.30.6.928
Diamond, A., Barnett, W. S., Thomas, J., & Munro, S. (2007). Preschool program
improves cognitive control. Science, 318(5855), 1387–1388. http://dx.doi.org/
References 10.1126/science.1151148
Diener, M., Wright, C., & Kemp, J. (2016). Storytelling and storyacting. SAGE
Astington, J. W., & Jenkins, J. M. (1995). Theory of mind development and social Encyclopedia of Contemporary Early Childhood Education, 1298–1300. Thousand
understanding. Cognition & Emotion, 9(2–3), 151–165. http://dx.doi.org/10. Oaks, CA: SAGE Publications. http://dx.doi.org/10.4135/9781483340333.n391
1080/02699939508409006 Duncan, S. E., & De Avila, E. A. (1998). Pre-Las 2000. CTB: McGraw-Hill.
August, D., & Shanahan, T. (Eds.). (2006). Developing literacy in second-language Elias, C., & Berk, L. E. (2002). Self-regulation in young children: Is there a role for
learners: Report of the National Literacy Panel on language-minority children and sociodramatic play? Early Childhood Research Quarterly, 17, 216–238. http://dx.
youth. Mahwah, NJ: Lawrence Erlbaum Associates. doi.org/10.1016/S0885-2006(02)00146-1
Barnett, W. S., Jung, K., Yarosz, D. J., Thomas, J., Hornbeck, A., Stechuk, R., et al. Ervin-Tripp, S. (1991). Play in language development. In B. E. Scales, M. E. Almy, A.
(2008). Educational effects of the tools of the mind curriculum: A randomized E. Nicolopoulou, & S. E. Ervin-Tripp (Eds.), Play and the social context of
trial. Early Childhood Research Quarterly, 23(3), 299–313. http://dx.doi.org/10. development in early care and education (pp. 84–97). New York, NY: Teachers
1016/j.ecresq.2008.03.001 College Press.
Berk, L. E., & Meyers, A. B. (2013). The role of make-believe play in the Fantuzzo, J., Sutton-Smith, B., Coolahan, K. C., Manz, P. H., Canning, S., & Debnam,
development of executive function: Status of research and future directions. D. (1995). Assessment of preschool play interaction behaviors in young
American Journal of Play, 6(1), 98 [Retrieved from]. http://www.journalofplay. low-income children: Penn Interactive Peer Play Scale. Early Childhood Research
org/sites/www.journalofplay.org/files/pdf-articles/6-1-article-the-role-of- Quarterly, 10(1), 105–120. http://dx.doi.org/10.1016/0885-2006(95)90028-4
make-believe.pdf Fantuzzo, J., Bulotsky-Shearer, R., McDermott, P. A., & McWayne, C. (2007).
Blair, C., & Raver, C. C. (2014). Closing the achievement gap through modification of Investigation of dimensions of social-emotional classroom behavior and school
neurocognitive and neuroendocrine function: Results from a cluster readiness for low-income urban preschool children. School Psychology Review,
randomized controlled trial of an innovative approach to the education of 36(1), 44–62. Retrieved from. http://repository.upenn.edu/gse pubs/124
children in kindergarten. Public Library of Science, 9(11) http://dx.doi.org/10. Fantuzzo, J., Perry, M. A., & McDermott, P. A. (2004). Preschool approaches to
1371/journal.pone.0112393, e112393 learning and their relationship to other relevant classroom competencies for

Please cite this article in press as: Susman-Stillman, A., et al. Reliability and validity of a measure of preschool children’s theatre arts
skills: The Preschool Theatre Arts Rubric. Early Childhood Research Quarterly (2018), https://doi.org/10.1016/j.ecresq.2017.12.001
G Model
EARCHI-978; No. of Pages 14 ARTICLE IN PRESS
A. Susman-Stillman et al. / Early Childhood Research Quarterly xxx (2018) xxx–xxx 13

low-income children. School Psychology Quarterly, 19(3), 212–230. http://dx. Lillard, A. S., & Witherington, D. C. (2004). Mothers’ behavior modification during
doi.org/10.1521/scpq.19.3.212.40276 pretense and their possible signal value for toddlers. Developmental Psychology,
Fantuzzo, J., Sekino, Y., & Cohen, H. L. (2004). An examination of the contributions 40(1), 95–113. http://dx.doi.org/10.1037/0012-1649.40.1.95
of interactive peer play to salient classroom competencies for urban head start Lillard, A. S., Lerner, M. D., Hopkins, E. J., Dore, R. A., Smith, E. D., & Palmquist, C. M.
children. Psychology in the Schools, 41(3), 323–336. (2013). The impact of pretend play on children’s development: A review of the
Fein, G. G. (1981). Pretend play in childhood: An integrative review. Child evidence. Psychological Bulletin, 139(1), 1–34. http://dx.doi.org/10.1037/
Development, 52(4), 1095–1118. http://dx.doi.org/10.2307/1129497 a0029321
Fleer, M. (2014). Theorising play in the early years. New York: Cambridge University Lindsey, E. W., & Colwell, M. J. (2003). Preschoolers’ emotional competence: Links
Press. to pretend and physical play. Child Study Journal, 33(1), 39–52.
Furman, L. (2000). In support of drama in early childhood education, again. Early Lovinger, S. L. (1974). Sociodramatic play and language development in preschool
Childhood Education Journal, 27(3), 173–178. http://dx.doi.org/10.1007/ disadvantaged children. Psychology in the Schools, 11(3), 313–320. http://dx.doi.
BF02694231 org/10.1002/1520-6807(197407)11:3<313:AID-PITS2310110315>3.0.CO;2-L
Galyer, K., & Evans, I. (2001). Pretend play and the development of emotion Mages, W. K. (2008). Does creative drama promote language development in early
regulation in preschool children. Early Child Development and Care, 166, childhood? A review of the methods and measures employed in the empirical
93–108. http://dx.doi.org/10.1080/0300443011660108 literature. Review of Educational Research, 78(1), 124–152. http://dx.doi.org/10.
Garaigordobil, M., & Berrueco, L. (2011). Effects of a play program on creative 3102/0034654307313401
thinking of preschool children. The Spanish Journal of Psychology, 14(02), Mages, W. K. (2015). Educational drama in early childhood: Promoting language
608–618. http://dx.doi.org/10.5209/rev SJOP.2011.v14.n2.9 development and supporting literacy transitions. Open Online Journal for
Gemeroth, C. (March, 2017). Play it high, play it low. Examining the reliability and Research and Education, 2, 1–5. Retrieved from. https://journal.phnoe.ac.at/
validity of a new observation tool to assess children’s make-believe play. Paper index.php/resource/article/view/200
presented at the Society for Research in Child Development Biennial Meeting, Mages, W. K. (2017). Practice makes perfect: The impact of training and rehearsal
Austin, TX. on program fidelity. Youth Theatre Journal, 31(1), 7–22. http://dx.doi.org/10.
Glenn-Applegate, K., Breit-Smith, A., Justice, L. M., & Piasta, S. B. (2010). Artfulness 1080/08929092.2017.1298542
in young children’s spoken narratives. Early Education and Development, 21(3), Mardell, B. (2013). Boston listens: Vivian Paley’s storytelling/story acting in an
468–493. http://dx.doi.org/10.1080/10409280903440620 urban school district. New England Reading Association Journal, 49(1), 58–67
Goldstein, T. R. (2015). The effects of a RCT of dramatic pretend play on social [Available for download at]. https://www.highbeam.com/doc/1P 3-
cognition and self control. In Poster presented at cognitive development society 3088477281.html
biennial meeting. McClelland, M. M., & Morrison, F. J. (2003). The emergence of learning-related
Greenfader, C. M., Brouillette, L., & Farkas, G. (2015). Effect of a performing arts social skills in preschool children. Early Childhood Research Quarterly, 18,
program on the oral language skills of young English learners. Reading Research 206–224. http://dx.doi.org/10.1016/S0885-2006(03)00026-7
Quarterly, 50(2), 185–203. http://dx.doi.org/10.1002/rrq.90 McClelland, M. M., Morrison, F. J., & Homes, D. L. (2000). Children at-risk for early
Gross, K. (2012). Crosswalks. Retrieved from. https://sites.ed.gov/ous/2012/05/ academic problems: The role of learning-related social skills. Early Childhood
crosswalks/ Research Quarterly, 15(3), 307–329. http://dx.doi.org/10.1016/s0885-
Gupta, A. (2009). Vygotskian perspectives on using dramatic play to enhance 2006(00)00069-7
children’s development and balance creativity with structure in the early McClelland, M. M., Cameron, C. E., Connor, C. M., Farris, C. L., Jewkes, A. M., &
childhood classroom. Early Child Development and Care, 179(8), 1041–1054. Morrison, F. J. (2007). Links between behavioral regulation and preschoolers’
http://dx.doi.org/10.1080/03004430701731654 literacy, vocabulary, and math skills. Developmental Psychology, 43(4),
Hallgren, K. A. (2012). Computing inter-rater reliability for observational data: an 947–959. http://dx.doi.org/10.1037/0012-1649.43.4.947
overview and tutorial. Tutorials in Quantitative Methods for Psychology, 8(1), McDermott, P. A., Green, L. F., Francis, J. M., & Stott, D. H. (2000). Preschool learning
23–34. http://dx.doi.org/10.20982/tqmp.08.1.p023 behaviors scale. Philadelphia, PA: Edumetric and Clinical Science.
Haneda, M., & Wells, G. (2012). Some key pedagogic principles for helping ELLs to McDermott, P. A., Leigh, N. M., & Perry, M. A. (2002). Development and validation
succeed in school. Theory Into Practice, 51(4), 297–304. http://dx.doi.org/10. of the preschool learning behaviors scale. Psychology in the Schools, 39,
1080/00405841.2012.726059 353–365. http://dx.doi.org/10.1002/pits.10036
Harris, P. L., Kavanaugh, R. D., Wellman, H. M., & Hickling, A. K. (1993). Young McNamee, G. D. (2005). The one who gathers children: The work of Vivian Gussin
children’s understanding of pretense. Monographs of the Society for Research in Paley and current debates about how we educate young children. Journal of
Child Development, i-107. http://dx.doi.org/10.2307/1166074 Early Childhood Teacher Education, 25, 275–296. http://dx.doi.org/10.1080/
Howes, C., & Matheson, C. C. (1992). Sequences in the development of competent 1090102050250311
play with peers: Social and social pretend play. Developmental Psychology, McNamee, G. D. (2015). The high-performing preschool: Story acting in Head Start
28(5), 961–974. http://dx.doi.org/10.1037/0012-1649.28.5.961 classrooms. Chicago, IL: University of Chicago Press.
Howes, C. (1980). Peer play scale as an index of complexity of peer interaction. Menzer, M. (2015). The arts in early childhood: Social and emotional benefits of arts
Developmental Psychology, 16(4), 371–372. http://dx.doi.org/10.1037/0012- participation: A literature review and gap-analysis 2000–2015.. Retrieved from.
1649.16.4.371 https://www.arts.gov/sites/default/files/arts-in-early-childhood-dec2015-rev.
Hynes-Berry, M. (2012). Don’t leave the story in the book: Using literature to guide pdf
inquiry in early childhood classrooms. New York, NY: Teachers College Press. Minnesota Department of Education. (2017). Minnesota early childhood indicators of
Justice, L. M., Bowles, R., Pence, K., & Gosse, C. (2010). A scalable tool for assessing progress: Introduction to the arts domain.. Retrieved from. http://education.
children’s language abilities within a narrative context: The NAP (Narrative state.mn.us/MDE/dse/early/ind/
Assessment Protocol). Early Childhood Research Quarterly, 25(2), 218–234. National Coalition for Core Arts Standards (2014). National Core Arts Standards.
http://dx.doi.org/10.1016/j.ecresq.2009.11.002 Rights administered by the State Education Agency Directors of Arts Education.
Kaugars, A. S., & Russ, S. W. (2009). Assessing preschool children’s pretend play: Dover, DE. www.nationalartsstandards.org.
Preliminary validation of the Affect in Play Scale-Preschool version. Early Newborg, J. (2004). Battelle developmental inventory (2nd ed.). Itasca, IL: Riverside
Education and Development, 20(5), 733–755. http://dx.doi.org/10.1080/ Publishing.
10409280802545388 Nicolopoulou, A., Barbosa de Sa, A., Ilgaz, H., & Brockmeyer, C. (2009). Using the
Krasnor, L. R., & Pepler, D. J. (1980). The study of children’s play: Some suggested transformative power of play to educate hearts and minds: From Vygotsky to
future directions. New Directions for Child and Adolescent Development, 9, Vivian Paley and beyond. Mind, Culture, and Activity, 17(1), 42–58. http://dx.
85–95. http://dx.doi.org/10.1002/cd.23219800908 doi.org/10.1080/10749030903312512
Krueger, A. C., Orton, J., & Bays, R. B. (2013a). Story enactment enhances language Nicolopoulou, A., Cortina, K. S., Ilgaz, H., Cates, C. B., & de Sá, A. B. (2015). Using a
acquisition in English language learning kindergarteners. In Poster presented at narrative-and play-based activity to promote low-income preschoolers’ oral
Biennial Society for Research in Child Development Meeting. language, emergent literacy, and social competence. Early Childhood Research
Krueger, A. C., Orton, J., & Bays, R. B. (2013b). Enhancing written expression in Quarterly, 31, 147–162. http://dx.doi.org/10.1016/j.ecresq.2015.01.006
English language learning kindergartens. In Poster presented at Biennial Society Paley, V. G. (1990). The boy who would be a helicopter: The uses of storytelling in the
for Research in Child Development Meeting. classroom. Cambridge, MA: Harvard University Press.
Ladd, G. W. (1989). Toward a further understanding of peer relationships and their Pence, K., Justice, L. M., & Gosse, C. (2007). Narrative assessment protocol. Columbus,
contributions to child development. In T. J. Berndt, & G. W. Ladd (Eds.), Peer OH: Preschool Language and Literacy Lab.
relationships in child development (pp. 1–15). New York: John Wiley. Phillips, R. D., Gordon, R. L., Pinciotti, P., & Sachdev, A. (2010). Promising findings
Lagattuta, K. H., Wellman, H. M., & Flavell, J. H. (1997). Preschoolers’ understanding on preschoolers’ emergent literacy and school readiness in arts-integrated
of the link between thinking and feeling: Cognitive cuing and emotional early childhood settings. Early Childhood Education Journal, 38(2), 111–122.
change. Child Development, 68(6), 1081–1104. http://dx.doi.org/10.1111/j. https://doi.org/10.1007/s10643-010-0397-x
1467-8624.1997.tb01986.x Podlozny, A. (2000). Strengthening verbal skills through the use of classroom
Lee, B. K., Patall, E. A., Cawthon, S. W., & Steingut, R. R. (2015). The effect of drama: A clear link. Journal of Aesthetic Education, 34(3/4), 239–275. http://dx.
drama-based pedagogy on preK–16 outcomes: A meta-analysis of research doi.org/10.2307/3333644
from 1985 to 2012. Review of Educational Research, 85(1) http://dx.doi.org/10. Posner, M. I., & Patoine, B. (2009). How arts training improves attention and
3102/0034654314540477 cognition. Cerebrum. Retrieved from: http://www.dana.org/Cerebrum/2009/
Levy, A. K., Schaefer, L., & Phelps, P. C. (1986). Increasing preschool effectiveness: How Arts Training Improves Attention and Cognition/.
Enhancing the language abilities of 3-and 4-year-old children through planned Rinne, L., Gregory, E., Yarmolinskyay, J., & Hardiman, M. (2011). Why arts
sociodramatic play. Early Childhood Research Quarterly, 1(2), 133–140. http:// integration improves long-term retention of content. Mind, Brain and
dx.doi.org/10.1016/0885-2006(86)90024-4 Education, 5(2), 89–96. http://dx.doi.org/10.1111/j.1751-228X.2011.01114.x

Please cite this article in press as: Susman-Stillman, A., et al. Reliability and validity of a measure of preschool children’s theatre arts
skills: The Preschool Theatre Arts Rubric. Early Childhood Research Quarterly (2018), https://doi.org/10.1016/j.ecresq.2017.12.001
G Model
EARCHI-978; No. of Pages 14 ARTICLE IN PRESS
14 A. Susman-Stillman et al. / Early Childhood Research Quarterly xxx (2018) xxx–xxx

Sachet, A. B., & Mottweiler, C. M. (2013). The distinction between role-play and Thibodeau, R. B., Gilpin, A. T., Brown, M. M., & Meyer, B. A. (2016). The effects of
object substitution in pretend play. In M. Taylor (Ed.), The Oxford handbook of fantastical pretend-play on the development of executive functions: an
the development of imagination (pp. 175–185). New York, NY: Oxford intervention study. Journal of Experimental Child Psychology, 145, 120–138.
University Press. http://dx.doi.org/10.1016/j.jecp.2016.01.001
Sachs, J., Mardell, B., & Boni, M. (2014). Storytelling, story acting, and literacy in Vasquez, V. (2004). Creating opportunities for critical literacy with young children:
Boston public schools: An interview with Jason Sachs, Ben Mardell, and Marina Using everyday issues and everyday text. In J. Evans (Ed.), Literacy moves on:
Boni. American Journal of Play, 6(2), 173–189. Using popular culture, new technologies and critical literacy in the primary
Schwebel, D. C., Rosen, C. S., & Singer, J. L. (1999). Preschoolers’ pretend play and classroom (pp. 78–96). New York, NY: David Fulton Publishers.
theory of mind: The role of jointly constructed pretense. British Journal of Vygotsky, L. S. (1978). Mind in society: The development of higher psychological
Developmental Psychology, 17(3), 333–348. http://dx.doi.org/10.1348/ processes. Cambridge: MA: Harvard University Press.
026151099165320 Wanerman, T. (2009). Using story drama with young preschoolers. In D. R. Meier
Singer, D. G., & Singer, J. L. (1992). The House of make-believe: Children’s play and the (Ed.), Here’s the story: Using narrative to promote young children’s language and
developing imagination. Cambridge, MA: Harvard University Press. literacy learning (pp. 27–38). New York, NY: Teachers College Press.
Smilansky, S., & Shefatya, L. (1990). Facilitating play: A medium for promoting Wee, S. J. (2009). A case study of drama education curriculum for young children in
cognitive, socioemotional and academic development in young children. early childhood programs. Journal of Research in Childhood Education, 23(4),
Gaithersburg, MD: Psychosocial and Educational Publications. 489–501. http://dx.doi.org/10.1080/02568540909594676
Snow, C. E., Burns, M. S., & Griffin, P. (Eds.). (1998). Preventing reading difficulties in Weisberg, D. S., Hirsh-Pasek, K., & Golinkoff, R. M. (2013). Guided play: Where
young children. Washington, DC: National Academy Press. curricular goals meet a playful pedagogy. Mind, Brain, and Education, 7(2),
Stagnitti, K., & Unsworth, C. (2004). The test–retest reliability of the child-initiated 104–112. http://dx.doi.org/10.1111/mbe.12015
pretend play assessment. American Journal of Occupational Therapy, 58(1), Wiig, E. H., Secord, W., & Semel, E. M. (2004). Clinical evaluation of language
93–99. http://dx.doi.org/10.5014/ajot.58.1.93 fundamentals—preschool (2nd ed.). San Antonio, TX: Harcourt Assessment.
Susman-Stillman, A., & Asp, M. (2013). Preschool student story rubric. Minneapolis, Winner, E., Goldstein, T., & Vinent-Lancrin, S. (2013). Art for art’s sake? the impact
MN: Center for Early Education and Development and Children’s Theatre of arts education. In Educational research and innovation. OECD Publishing.
Company. http://dx.doi.org/10.1787/9789264180789-en
Susman-Stillman, A., Rader, S., Asp, M., & Beckstrom, E. (2012). Crosswalk between Wright, C., Dierner, M. L., & Kemp, J. L. (2013). Storytelling dramas as a community
theatre arts skills and early childhood education practices. Center for Early building activity in an early childhood classroom. Early Childhood Educational
Education and Development: University of Minnesota. Journal, 41(3), 197–210. http://dx.doi.org/10.1007/s10643-012-0544-7
Susman-Stillman, A., Asp, M., & Beckstrom, E. (2013). Preschool Theatre Arts Rubric. Yeh, Y. C., & Li, M. L. (2008). Age, emotion regulation strategies, temperament,
Minneapolis, MN: Center for Early Education and Development and Children’s creative drama, and preschoolers’ creativity. The Journal of Creative Behavior,
Theatre Company. 42(2), 131–149. http://dx.doi.org/10.1002/j.2162-6057.2008.tb01291.x
Taylor, M., & Carlson, S. M. (1997). The relation between individual differences in Youth Stages. (2016). What is creative drama? http://www.youthstages.com/
fantasy and theory of mind. Child Development, 68(3), 436–455 [10]. CreativeDrama/WhatIsCDrama.asp

Please cite this article in press as: Susman-Stillman, A., et al. Reliability and validity of a measure of preschool children’s theatre arts
skills: The Preschool Theatre Arts Rubric. Early Childhood Research Quarterly (2018), https://doi.org/10.1016/j.ecresq.2017.12.001

Вам также может понравиться