http://aut.sagepub.

com/

Autism

Social Stories to improve social skills in children with autism spectrum disorder : A systematic review
Mohammad Karkhaneh, Brenda Clark, Maria B. Ospina, Jennifer C. Seida, Veronica Smith and Lisa Hartling Autism 2010 14: 641 originally published online 5 October 2010 DOI: 10.1177/1362361310373057 The online version of this article can be found at: http://aut.sagepub.com/content/14/6/641

Published by:
http://www.sagepublications.com

On behalf of:

The National Autistic Society

Additional services and information for Autism can be found at: Email Alerts: http://aut.sagepub.com/cgi/alerts Subscriptions: http://aut.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations: http://aut.sagepub.com/content/14/6/641.refs.html

>> Version of Record - Dec 13, 2010 OnlineFirst Version of Record - Oct 5, 2010 What is This?
Downloaded from aut.sagepub.com by guest on April 17, 2013

Social Stories to improve social skills in children with autism spectrum disorder
A systematic review
MOHAMMAD KARKHANEH
Edmonton, Canada University of Alberta,

autism 2010 SAGE Publications and The National Autistic Society Vol 14(6) 641662; 373057 1362-3613(2010)

BRENDA CLARK MARIA B. OSPINA

University of Alberta, Edmonton, Canada University of Alberta, Edmonton, Canada University of Alberta, Edmonton, Canada University of Alberta, Edmonton, Canada University of Alberta, Edmonton, Canada

JENNIFER C. SEIDA VERONICA SMITH L I S A H A RT L I N G

A B S T R AC T

Over the past 20 years a variety of treatments have been developed to remediate decits associated with autism. Since the early 1990s, Social Stories have been suggested to positively affect the social development of children with autism spectrum disorder (ASD). Despite much research, there remains uncertainty regarding the effectiveness of this modality. We conducted a systematic review of the literature using pre-dened, rigorous methods. Studies were considered eligible if they were controlled trials evaluating Social Stories among persons with ASD. Two reviewers independently screened articles for inclusion, applied eligibility criteria, extracted data, and assessed methodological quality. A qualitative analysis was conducted on six eligible controlled trials. Five of the six trials showed statistically signicant benets for a variety of outcomes related to social interaction. This review underscores the need for further rigorous research and highlights some outstanding questions regarding maintenance and generalization of the benets of Social Stories.

K E Y WO R D S

autism spectrum disorders; systematic reviews; Social Stories

ADDRESS Author to whom correspondence should be sent: L I S A H A RT L I N G , Department of Pediatrics, University of Alberta, Aberhart Centre One, Room 9424, 11402 University Avenue, Edmonton, Alberta, T6G 2J3, Canada. e-mail: hartling@ ualberta.ca

Copyright The Author(s), 2010. Reprints and permissions: http://www.sagepub.co.uk/journalspermissions.nav DOI: 10.1177/1362361310373057
Downloaded from aut.sagepub.com by guest on April 17, 2013

641

14(6) Autism spectrum disorders (ASD) is a class of life-long neurodevelopmental disorders characterized by impairments in reciprocal social interactions, verbal and nonverbal communication skills, and restricted, repetitive and stereotyped patterns of behaviour, interests and activities (American Psychiatric Association, 2000). In addition to these core features, a range of other nonspecic behaviour problems are common, such as anxieties, depression, sleeping and eating disturbances, attention issues, temper tantrums, and aggression or self-injury (World Health Organization, 1993). Although symptoms often persist through adulthood, timely detection and appropriate treatment may be important factors in improving both shortand long-term outcomes (Bryson et al., 2003; Marnie, 2005). Over the past 20 years, a variety of treatment approaches have been developed in an attempt to remediate the decits associated with ASD. Interventions vary extensively in terms of their underlying theoretical framework, mode of delivery, intensity, degree of parental involvement and comprehensiveness. Since the early 1990s, Social Stories (Gray and Garand, 1993) have been suggested to positively affect the social understanding and behaviours of children with ASD. A Social Story describes a situation, skill, or concept in terms of relevant social cues, perspectives, and common responses in a specically dened style and format (Gray, 1994). The goal of a Social Story is to share accurate social information with a patient in a reassuring manner that is easily understood by its audience (Gray, 1994). As such, it is a short story written for an individual that describes a specic activity and the behaviour expectations associated with that activity. Social Stories can be implemented through different methods: they can be read, either independently or by a caregiver, or presented through audio or video equipment or through computer-based programs (Charlop and Milstein, 1989; Gray and Garand, 1993; Hagiwara and Myles, 1999; Rogers and Myles, 2001; Thiemann, 2001; Wetherby et al., 1997). One important component, irrespective of how the social story is presented, is monitoring individual progress once the intervention has been introduced (Gray, 1995; Gray, 1998). Progress is usually monitored by the caregivers based on the change in frequency and intensity of the targeted behaviour. Three recently published reviews have summarized the evidence for Social Stories interventions among persons with ASD (Sansosti et al., 2004; Reynhout and Carter, 2006; Ali and Frederickson, 2006). The reviews and their component studies are described in Table 1. Overall, these reviews captured 22 studies, 7 of which were common to all reviews. The earliest review (Sansosti et al., 2004) included published studies identied from two electronic databases. The review methods were not described, therefore it is considered methodologically awed based on
AU T I S M

642

Downloaded from aut.sagepub.com by guest on April 17, 2013

Table 1 Reynhout 2006 Ali & Frederickson 2006 Research design Other intervention

Summary of included studies in three previous reviews, their designs and populations N of participants

Studies included in the reviews

Sansosti 2004

Downloaded from aut.sagepub.com by guest on April 17, 2013

Adams 2004 Barry 2004 Bledsoe 2003 Brownell 2002 Cullain 2002a Feinberg 2002a Hagiwara 1999 Ivey 2004 Kuoch 2003 Kuttler 1998 Lorimer 2002 Moore 2004 Norris 1999 Pettigrew 1998a Rogers 2001 Romano 2002a Rowe C 1999 Scattone 2002 Smith 2001 Staley 2002a Swaggart 1995 Thiemann 2001

ABAB ABCD-M-B ABAB ABAC ABA pre- post-test M-B ABAB ABA-ACABA ABAB ABAB DCS AB pre- post-test AB pre- post-test DCS M-B post-test M-B AB M-B

NR Visual timetable Using photos NR NR Verbal prompting Verbal prompting Language therapy Verbal correction Prize reinforcers NR Positive reinforces Adult structure Scaffolding activities Comic strip NR Support assistant Verbal prompting NR Verbal prompting Verbal prompting Verbal prompting

1 2 1 4 5 34 3 3 3 1 1 1 1 69 1 10 1 3 15 5 3 5

A REVIEW OF SOCIAL STORIES

a. Dissertation. Note. M-B = multiple-baseline; DCS = descriptive case study; NR = not reported.

643

14(6) standard criteria for assessing the quality of reviews (Oxman and Guyatt, 1988). The authors described eight single-subject design studies published between 1995 and 2002. While all included studies reported positive results of Social Stories for different behavioural targets, the authors urged readers to consider the ndings with caution due to the lack of experimental control, weak treatment effects, confounding treatment variables, cointerventions (Sansosti et al., 2004) and evidence that gains in skill acquisition were not maintained (Sansosti et al., 2004: 199). Further, the authors highlighted general weaknesses with this body of literature including lack of measurement of treatment integrity and consistency in applying the intervention (Sansosti et al., 2004: 201); unblinded outcome assessment; lack of consideration of generalization and social validity (Sansosti et al., 2004: 201) of the intervention; and comparisons with typically developing peers. Reynhout and Carter (2006) conducted a comprehensive review of the literature that included electronic database and manual searches up to December 2003. The authors clearly described their inclusion criteria; however, their methods for study selection and data extraction were poorly described. Furthermore, there was no systematic attempt to evaluate the methodological quality of the included studies. The authors combined data statistically; however, this approach is questionable given the large heterogeneity across studies in terms of their design and methods, the development and implementation of the intervention, and the characteristics of participants in the studies (Thompson, 1994). In fact, the authors acknowledge that the pooled results should be interpreted cautiously due to caveats in the primary studies. Overall, this review included 16 studies, 12 of which were singlesubject designs. While most studies reported appropriate (Reynhout and Carter, 2006: 461) changes in targeted behaviours, the authors highlighted the following methodological limitations in the primary studies: confounding by other variables, including co-interventions (in particular the use of reinforcers); non-standardized development and implementation of the Social Stories intervention; lack of data on maintenance of observed changes in behaviour and generalization to other behaviours; and, questionable use of reversal designs that may be inappropriate if the intervention is not fully reversible. Furthermore, the reviewers commented on the inadequate reporting of study methods, interventions, and participants, thereby limiting the generalizability of the ndings beyond the participants studied. The authors concluded that the effects of Social Stories are highly variable and the results may be highly confounded. The third and most recent review (Ali and Frederickson, 2006) identied published studies using a limited search strategy in three electronic
AU T I S M

644

Downloaded from aut.sagepub.com by guest on April 17, 2013

A REVIEW OF SOCIAL STORIES

databases. The review methods were not fully described. The authors reviewed 16 studies, published from 1994 to 2004, which agreed with the ndings of the previous two reviews. The authors stressed the positive ndings reported in the individual studies; however, they also echoed limitations cited in earlier reviews, including inappropriateness of reversal designs, inconsistent implementation of the interventions, the role of confounding and co-interventions, lack of controlled and blinded experimental conditions, and the need to consider generalization and long-term maintenance. The authors raised the question of what should be considered clinically signicant in this body of literature and stressed the importance of functional outcomes that are relevant to the individual children. As highlighted in these reviews, most of the evaluated studies (16/22, 73%) reported that some additional strategies were applied along with the Social Stories intervention. These other interventions, such as paired and visual timetables, token economy, verbal and physical prompts, photos of peers, positive reinforcement chart, written text cues, video feedback, and relevant corrective verbal feedback, can be considered as confounders of the effect of Social Stories (Table 1), hence the true effect of Social Stories is difcult to isolate and estimate. Moreover, previous reviews have identied the predominance and shortcomings of single case designs in this body of literature which further precludes denitive conclusions regarding the efcacy of Social Stories compared to other interventions (Smith et al., 2007). Our objective was to build on previous work through a rigorous and systematic approach to identify and synthesize all available controlled trials evaluating Social Stories for ASD. To this end, we followed established guidelines for systematic reviews (Higgins et al., 2006), searched for both published and unpublished evaluations, and restricted our inclusion to controlled trials, as these studies provide the most unbiased estimates of treatment effectiveness.

Method
Search strategy A medical research librarian conducted comprehensive searches of 22 relevant electronic databases (e.g., MEDLINE, EMBASE, ERIC, CINAHL, Cochrane Central Register of Controlled Trials, Dissertation Abstracts, PsycINFO, and Web of Science) to May 2007 as a part of a broader systematic review on the behavioural/developmental continuum of interventions for autism (search strategy is available upon request). We also searched Current Controlled Trials, an international registry of ongoing
645

Downloaded from aut.sagepub.com by guest on April 17, 2013

14(6) and completed trials. Reference lists of relevant studies were reviewed to identify other potentially relevant studies. Hand searches of the Journal of Speech and Language Pathology Applied Behaviour Analysis (2006), and the Journal of Early and Intensive Behaviour Intervention (20042006) were also conducted. We contacted experts to identify any unpublished research.
AU T I S M

Study selection Studies were to be randomized controlled trials (RCT) or controlled clinical trials (CCT), published in English, which evaluated a Social Story intervention versus any other intervention in individuals with ASD. Two reviewers independently evaluated the title and abstract (where available) of each study, using a predefined set of broad criteria, in order to select references potentially relevant to the review. Then two reviewers independently assessed potentially eligible studies for inclusion using criteria that were established a priori. Disagreements were resolved by consensus between two reviewers, involving a third person if necessary to adjudicate. Data extraction and assessment of study quality Trained research assistants extracted data using a comprehensive and pretested data extraction form. One reviewer veried the accuracy and completeness of the data. Discrepancies were resolved by consulting the primary study. An individual components approach, in which specic aspects of design, execution, and analysis are judged as adequate or inadequate, was used to assess the methodological quality of studies in the review. Two assessment tools that have well-established face validity and for which a relationship with bias has been proven in empirical studies were used. The Jadad Scale (Jadad et al., 1996) is a validated scale that includes questions related to bias reduction: randomization, double-blinding, and description of dropouts and withdrawals. The concealment of allocation approach (Schulz et al., 1995) is based on the evidence of a strong relationship between the potential for bias in the results and allocation concealment. Individual components of other quality scales that were relevant to this body of evidence were also evaluated (Chalmers et al., 1981; Detsky et al., 1992; Smith et al., 2007) and included questions regarding description of selection criteria, therapeutic regimens, therapists or intervention providers, blinding of investigator/care providers and outcome assessors, treatment delity, statistical analysis (sample size calculations, missing data management, testing of randomization), and source of funding. Quality was assessed independently by two reviewers. Disagreements were resolved by consensus.

646

Downloaded from aut.sagepub.com by guest on April 17, 2013

A REVIEW OF SOCIAL STORIES

Synthesis of the evidence A narrative approach was used to synthesize the evidence. In this synthesis, we focus on the results as analyzed and presented by the authors of the primary studies. We also calculated standardized mean differences (SMD) for key outcomes in each study in order to quantify the potential magnitude of effect (Cohen, 1995). If a study collected data at a number of time points we calculated the mean difference from baseline and the last time point reported. We did not pool results across studies due to heterogeneity in outcomes.

Results
Characteristics of the included studies Six trials (four RCTs [Andrews, 2004; Bader, 2006; Feinberg, 2002; Quirmbach, 2006] and two CCTs [Ricciardelli, 2006; Romano, 2002]) were included (Figure 1). Study characteristics are presented in Table 2. The studies were published between 2002 and 2006. All of the studies were dissertations and conducted in the United States. The studies included a total of 135 participants, with a median sample size of 20 (inter-quartile range [IQR] 12.5 to 30.5).The median age of participants was 120 months (IQR 114 to 122); the range was 4 to 14 years. Only two studies (Feinberg, 2002; Quirmbach, 2006) specied the ethnicity of their samples: in addition to Caucasian participants, Feinberg included Black, Asian, and Hispanic participants, while Quirmbach included Asian and Hispanic participants. Four of the studies described the diagnostic criteria that were used; however, none of the studies reported on whether the participants were diagnosed independently. The time to outcome assessment in the majority of studies was short, ranging from the same day (i.e., immediately after the intervention) (Feinberg, 2002) to six weeks (Romano, 2002). Treatment intensity ranged from two trials during a single session (Ricciardelli, 2006) to 10 readings a day for 30 consecutive school days (Romano, 2002). The comparison groups varied across studies: regular school instruction (Ricciardelli, 2006), no intervention (Romano, 2002), regular stories (Andrews, 2004; Bader, 2006; Feinberg, 2002), and a Social Story unrelated to the targeted behaviour (Quirmbach, 2006). Five of the six trials reported statistically signicant results that favoured the Social Stories intervention for the outcomes under study. Methodological quality and reporting of the included studies A summary of the methodological quality of the included studies is provided in Table 3. In terms of the individual components of the Jadad
647

Downloaded from aut.sagepub.com by guest on April 17, 2013

Table 2 Design Sample size (% males) RCT (parallel) 20 (NR) 812 (10) Severity not reported; children were verbal and could read and understand written words at or above rst grade level Severity not reported; children were verbal and could read and understand written words at or above kindergarten grade level Moderately severe symptoms; children had at least phrase speech Children could read at or above rst grade level; 42/45 children diagnosed with autism and 3 with ASD Age range, years (mean) Description/severity of autistic symptoms Sample characteristics Diagnostic criteria (independently conrmed)

Characteristics of studies included in the review

Study

Objective(s)

648
RCT (cross-over) 20 (NR) 613 (9) RCT (parallel) RCT (parallel) 45 (93%) 714 (9) 34 (74%) 813 (10)

AU T I S M

Andrews 2004

To increase game playing skills, story comprehension, and social skills comprehension

DSM-IV-TR, ADOS-G (NR)

14(6)

Bader 2006

To increase emotion recognition and labeling skills (in abstract)

DSM-IV-TR, ADOS (NR)

Feinberg 2001

To increase four specic social skills during game playing

DSM-IV, ADOS-G, ADI-R, GARS DSM-IV-TR, ADOS

Downloaded from aut.sagepub.com by guest on April 17, 2013

Quirmbach 2006

1) To increase game playing skills and story comprehension; 2) to compare different story formats; 3) to assess childrens ability to generalize and maintain skills CCT (parallel) CCT (parallel) 10 (50%) 6 (100%)

Ricciardelli 2006

To improve ve specic pro-social behaviours

1013 (11) Cognitive levels average to low average; some verbal skills present 48 (6)

NR Children able to communicate NR (express wants and needs) either verbally or non-verbally (i.e., augmentative device, manual communication board, sign language, modied sign language, gestures)

Romano 2002

To reduce inappropriate communication, aggressive behaviour, and inappropriate socialization behaviours

Note. ADOS = Autism Diagnostic Observation Schedule; ADOS-G = Autism Diagnostic Observation Schedule-Generic; ADI-R = Autism Diagnostic Interview-Revised; CCT = controlled clinical trial; NR = not reported; RCT = randomized controlled trial; GARS = Gilliam Autistic Rating Scale.

A REVIEW OF SOCIAL STORIES

Figure 1

Flow-diagram for study selection

Total number of citations retrieved from literature a searches N = 12,509

Articles retrieved and evaluated in full for inclusion N = 1,136

rtr

Number of unique articles evaluating behavioural or developmental interventions for ASD N = 99 - 97 articles each reporting one unique study - 2 articles each reporting two studies Reason for exclusions -Applied Behaviour Analysis = 31 -Communication-Focused Interventions = 10 -Contemporary ABA = 12 -Developmental Approaches = 12 -Environmental Modification = 1 -Integrative Programs = 14 -Sensory Motor Interventions = 15

Unique studies evaluating behavioural or developmental interventions for ASD N = 101

Excluded N = 95

Studies evaluating Social Stories Interventions N=6

RCT = 4

CCT = 2

a. Search strategy was designed for a broader review evaluating all behavioural and developmental interventions for ASD [see Ospina et al., 2007, The Behavioural and Developmental Continuum of Intervention for Autism Spectrum Disorder: A Systematic Review]. For requesting a copy you can contact Alberta Centre for Child, Family and Community Research;Tel 780-944-8636 (URL: http://www.research4children.com).

scale, only one trial (Andrews, 2004) described the use of an appropriate method for generating the sequence of randomization. None of the trials were described as double-blind, although two trials (Andrews, 2004; Quirmbach, 2006) reported that blind or independent outcome assessment was conducted. None of the trials provided an adequate description 649

Downloaded from aut.sagepub.com by guest on April 17, 2013

650
Blinding Participants No No No No No No No No No No No No Yes Unclear No Yes Unclear No Investigators Outcome assessors Unclear Unclear Unclear Inadequate Unclear Unclear Concealment of allocation

AU T I S M

14(6)

Table 3

Methodological quality of included studies

Downloaded from aut.sagepub.com by guest on April 17, 2013

Author, year

Jadad score (out of 5)

Andrews 2005 Bader 2006 Feinberg 2002 Quirmbach 2006 Ricciardelli 2006 Romano 2002

2 1 1 0 0 0

A REVIEW OF SOCIAL STORIES

of withdrawals and drop-outs. Furthermore, none of the trials reported whether allocation to study groups was adequately concealed. Only one trial (Bader, 2006) mentioned how representative the sample was in terms of the study setting, the selection criteria for enrolling participants, and the operational denition of ASD used for the study. One trial (Romano, 2002) provided a complete description of ineligible patients before treatment allocation took place. Half of the trials (Ricciardelli, 2006; Quirmbach, 2006; Romano, 2002) provided a complete description of the intervention delivered to the treatment group. None of the studies described the characteristics of the intervention provider. Only one trial (Andrews, 2004) fully reported details on monitoring the delity of intervention implementation. None of the studies provided details on the methods for calculating sample size or reported the use of an intention-to-treat approach in the analysis. One trial (Feinberg, 2002) reported how the study managed missing data (i.e., participants were excluded from the analysis).

Study results In the following paragraphs we describe the included studies individually. More details are presented in Table 4. Andrews (2005) compared the effectiveness of Social Stories versus a story without social loading in an RCT involving 20 children, aged 8 to 12 years, with a diagnosis of autism and a minimum rst grade reading level. The authors reported signicant ndings with large effect sizes for each of the three outcomes examined: game playing skills, reading comprehension, and social skills comprehension. Despite some methodological weaknesses (Table 3), this study had several strengths including the use of independent and blinded outcome assessors (i.e., minimizing the risk of detection bias), the inclusion of a homogeneous sample with respect to diagnosis, and the attempt to control for co-interventions, such as reinforcement or behaviour modication strategies. Another strength was the validation of the stories, as well as the reading and social skills comprehension tools, among typically developing peers. While the study found benets of Social Stories, generalizability is limited to the clinical setting of the study and to children with a minimum rst grade reading level. Moreover, the author commented on factors that may result in variable response to Social Stories including IQ and reading level. Finally, the author recognized that the study provides no data on maintenance of the acquired skills. Bader (2006) evaluated the effectiveness of Social Stories compared to a social story without loading to increase emotional recognition and labelling in a cross-over RCT of 20 participants with autism ranging in age from 6 to 13 years. Three computer-administered visual perception tasks
651

Downloaded from aut.sagepub.com by guest on April 17, 2013

Table 4
Comparison Description Implementation Outcomes Results* Conclusions

Overview of study comparisons and ndings

Study

Intervention

652
Story without social loading Game playing skills 5 trials on a single day; received Social Story on second day, 1 week later Story comprehension Signicant improvement for intervention group (p < .025, ES = 1.53) Signicant improvement over time for all children (p < .05, ES = 1.39) Signicant (p < .05) time group interaction for 3 visual perception tasks Signicantly higher scores overall favouring intervention group (p < .05, ES = 1.37) Social skills comprehension Story without social loading 3 trials, then given social story for 3 trials; over 2 sessions Emotion recognition based on 3 visual perception tasks: affect discrimination, emotion matching, affect choice

AU T I S M

Target behaviour

Implementation

Andrews 2004

Game playing skills

14(6)

5 trials on each of 2 days which were 1 week apart

The Social Story enhanced game playing skills, reading comprehension, and social skills comprehension. Individual variability may be associated with IQ and reading level.

Downloaded from aut.sagepub.com by guest on April 17, 2013

Bader 2006

Emotion recognition

3 trials, then given control story for 3 trials; over 2 sessions

The Social Story was effective in increasing recognition and labeling of four emotions. Results demonstrate some short-term maintenance of learning from the Social Story. Social skills score based on 4 components: greeting behaviours; requesting to play a game; asking what to play; accepting suggestions Signicant condition trial interaction for overall score (p < .001) and for two behaviours: greeting behaviour (p = .002) and asking what to play (p < .001) A Social Story was effective in increasing game playing skills. The learning effect was maintained over a 1-week period. Autism severity was identied as the most important predictor of social skills scores.
Continued opposite

Feinberg 2001

Game playing skills

5 trials on day 1 and day 2, which were 1 week apart; parents read story daily in between

Regular story on day 1 then social story in day 2

5 trials on each day 1 and day 2

Table 4
Comparison Description Implementation Outcomes Results* Conclusions

Continued

Study

Intervention

Target behaviour Social story unrelated to game playing skills Comprehension All groups showed signicant improvement with no differences between groups No signicant differences observed for any groups when games were switched No signicant differences between last trial on day 1 and rst trial on day 2 No signicant differences between groups post-intervention in four of the ve skills; signicant difference post-intervention only for paying attention, favouring curriculum without Social Stories (p = .04) 4 trials on day 1; on day 2 control group was randomized to one of the social stories groups Game playing skills Signicant difference between two intervention groups and control group (p < .05)

Implementation

Quirmbach 2006

Game playing skills

4 trials on day 1 and day 2, which were 1 week apart (divided into two groups of standard and only directive story)

Generalization: whether children used new skills with different games Maintenance

Social Story intervention was effective in improving game playing skills and comprehension; the acquired skills were generalizable to other games and maintained over a 1-week period; Social Stories with only directive sentences may be as effective as those with a variety of sentence types as advocated by Gray. Social Stories may be most effective among children with higher verbal IQ scores. Minimal signicant differences between groups suggest that Social Story intervention had little impact. In fact, standard care group showed greater improvement in paying attention.
Continued over

Downloaded from aut.sagepub.com by guest on April 17, 2013

A REVIEW OF SOCIAL STORIES

Ricciardelli 2006

Social skills

Stories read twice a day at same time each morning andevening (2025minutes) in addition to regular curriculum

Social skills curriculum

55 min a day Social skills (1 month for each target behaviour including 1 week observation time)

653

654
Comparison Description Implementation Outcomes Results* Conclusions No social story Signicant difference (p < .05) Signicant difference (p < .01) Signicant difference (p < .05) The same Aggressive behaviour program as intervention group Echolalic except social communication story component. Social isolation Social Stories had a meaningful impact in reducing inappropriate behaviours among intervention group. Changes were observed in control group but were not maintained at the six-week follow-up.

AU T I S M

14(6)

Table 4

Continued

Study

Intervention

Target behaviour

Implementation

Downloaded from aut.sagepub.com by guest on April 17, 2013

Romano 2002

Pro-social behaviours

Stories read 10 times a day for 30 school days

a. Results are in favour of the Social Stories group unless otherwise indicated. Results based on 13 participants; 7 removed from analysis who scored 100% on measure at baseline. These included the following:greeting behaviours,requesting to play a game,asking what another person wants to play,accepting another persons suggestion for a game,turn taking, enjoyment in game play, and,continued desire for game play. These included the following:introducing oneself to a new person;initiating a conversation with peers;asking an adult for help;paying attention; and,saying nice things to others when they have done something nice.

A REVIEW OF SOCIAL STORIES

were used as a measure of emotion recognition and labelling skills. Among the experimental group, the scores for the three tasks increased signicantly after implementation of the Social Story intervention and were maintained after crossing over to the control story. The control group showed no change in the three tasks after receiving the control story but showed statistically signicant changes after crossing over to the Social Story intervention. The authors observed a potential mechanism for the effect of Social Stories in that children more often imitated the emotions after exposure to the Social Story. This study validated the Social Story among typically developing children, included a homogeneous sample with respect to diagnosis, and controlled for co-interventions such as reinforcement and behaviour modication strategies. Generalizability of the ndings is limited for different IQ and reading levels and outside of a clinical setting. Further, the authors commented on the potential confounding resulting from previous exposure to Social Stories among some of the participants. While this study demonstrated short-term maintenance of learning outcomes, they suggested further research regarding maintenance over extended periods of time. Feinberg (2002) conducted an RCT involving 34 children, aged 8 to 13 years, diagnosed with autism and required to have at least phrase speech (Feinberg, 2001: 27). The authors compared the benets of a Social Story intervention compared to a regular story on social skills. The authors found that the Social Story intervention increased overall social skills scores in the context of game playing; signicant differences were found for two of the four component social skills (greeting behaviour and asking another what they want to play). Exploratory analyses showed that the effects were maintained over a one-week interval between the two training sessions and that the learning continued for the intervention group during the second session. The author demonstrated that receptive language, verbal IQ, and autism severity were predictors of social skills score, although autism severity was identied as the best predictor. The study included a homogeneous sample with respect to diagnosis; controlled for other co-interventions; and used two independent outcome assessors who were unaware of the study hypotheses. Limitations of the study were related to a lack of evaluation of maintenance over extended time periods; and the lack of generalizability of the ndings beyond children with phrase speech and to other settings and interaction with peers. Quirmbach (2006) evaluated the effectiveness of two different formats of a single Social Story to increase comprehension skills and to improve, generalize and maintain game playing skills among 45 children between the ages of 7 and 14 years, diagnosed with ASD. The children were described as high functioning based on at least a rst grade level on reading recognition 655

Downloaded from aut.sagepub.com by guest on April 17, 2013

14(6) and reading comprehension subtests. Children were randomly assigned to one of three arms: one group received a standard Social Story for game playing skills, a second group received a Social Story for game playing skills that included only directive sentences, and the third group or control received a Social Story unrelated to game playing skills. Compared to the control group, the two intervention groups demonstrated signicant improvements in game playing skills, as well as generalization to other games and maintenance over a one-week interval. All groups demonstrated improved comprehension over the intervention period. Exploratory analyses suggested that a Social Story intervention may only be effective among children with moderate to no verbal comprehension impairment (i.e., greater than 68 on the Verbal Comprehension Index of the WISC-IV). This is the largest controlled trial to date that examines the effectiveness of Social Story interventions; however, the results may not be generalizable to other social skills, to more natural settings with peers, and to children with more severe symptoms or lower verbal comprehension. This study provides some evidence that a single Social Story can be used for a large cohort of children. The author identied the need to examine maintenance beyond a one-week period. Ricciardelli (2005) conducted a controlled trial evaluating the use of individualized Social Stories in addition to a formalized school-based social skills curriculum. The trial involved six children with high functioning autism between the ages of 10 and 13 years. The Social Stories were written to coincide with the monthly skill being taught by the teacher within the school curriculum. Five social behaviours were targeted in both the school curriculum and the Social Stories intervention. No signicant differences were observed between groups either following the training month or at the end of the school year for four of the ve targeted behaviours. For the fth behaviour (paying attention), no signicant difference was found at the post-training month assessment; however, a signicant difference favouring the curriculum-only group was observed at the end of the school year. An important methodological limitation of this study was that the outcome assessors were not blind to study group. The small sample in this study precludes any denitive conclusions. Romano (2002) examined the effects of a Social Story intervention on three targeted behaviours among 10 autistic children between 4 and 8 years of age with some ability to express needs and wants, either verbally or non-verbally. The Social Story and no-treatment control groups were matched on age and baseline data for the targeted behaviours. Signicant differences in favour of the Social Story group were observed for seven of the nine comparisons across the three targeted behaviours including greater reduction in inappropriate or aggressive behaviours and less recidivism at
AU T I S M

656

Downloaded from aut.sagepub.com by guest on April 17, 2013

A REVIEW OF SOCIAL STORIES

six-week follow-up. One of the main limitations of the study was the potential for contamination of the comparison group, as they may have listened to the stories and imitated the behaviours demonstrated by children in the intervention group. This would have resulted in underestimating the effect of the intervention. The authors highlight limitations with respect to generalizability, including the higher level of functioning among the study sample and the use of trained professionals in a structured educational setting. This trial measured maintenance of behavioural changes beyond one week after the intervention period. While the study group demonstrated some maintenance of behavioural changes at six-week follow-up, the author suggests that sustained behaviour modication requires intense daily instruction (Romano, 2002: 64). Table 5 presents effect sizes, or standardized mean differences, for key outcomes in each study. The effect sizes give an indication of the potential
Table 5 Author Effect sizes for key outcomes by study Outcome Standardized mean difference (95% CI) 2.55 (1.30, 3.79)* 1.04 (0.10, 1.99) 0.20 (0.71, 1.10) 0.06 (0.93, 0.82) 0.11 (0.99, 0.76) 1.34 (0.59, 2.09)* 0. 99 (0.23, 1.76)* 0. 86 (0.11, 1.61)* 0.21 (0.51, 0.93) 0.73 (1.47, 0.01) 1.14 (0.80, 3.09) 0.00 (1.60, 1.60) 0.01 (1.61, 1.59) 0.90 (2.72, 0.92) 1.08 (0.83, 2.99) 1.24 (2.67, 0.18) 1.93 (3.57, 0.28) 1.82 (3.42, 0.21)*

Andrews 2004 Bader 2006

Game playing Social story comprehension Affect discrimination Emotion matching Affect choice Total social skills Feinberg's game playing score Andrews' game playing score Story comprehension of playing games Story comprehension of getting ready for school Introducing oneself to a new person Initiating a conversation with peers Asking an adult for help Paying attention Saying nice things to others when they have done something nice Inappropriate communication Aggression Social isolation

Feinberg 2001 Quirmbach 2006

Ricciardelli 2006

Romano 2002

* Statistically signicant at p < .05 Results presented are for standard social stories versus control. Calculations were based on post-intervention data (i.e., intervention had been completed for all ve targeted areas).

657

Downloaded from aut.sagepub.com by guest on April 17, 2013

14(6) magnitude of effect that could be observed with this intervention, but are not intended to be viewed as precise estimates of effect. The effect sizes reect substantial variation across outcomes and studies in terms of magnitude of effect.
AU T I S M

Discussion
This systematic review of controlled trials evaluating Social Stories for children with ASD complements previous reviews that highlight the positive effects of this modality for higher functioning children with autism. We identied six studies, four of which had not been included in previous reviews. Our ndings support and highlight the effects of Social Stories for short term improvements of the social decits among school-aged autistic children (mean age across studies 8 to 11 years). Five of the six studies (four RCTs [Andrews, 2004; Bader, 2006; Feinberg, 2002; Quirmbach, 2006] and one CCT [Romano, 2002]) reported statistically signicant differences between the study groups in favour of Social Stories. These ve studies showed benets for a variety of outcomes related to social interaction, such as game playing skills, story comprehension, generalized social comprehension, facial emotion learning and labelling, social skills, aggressive behaviour, and communication skills. The duration of the intervention and time to outcome assessment was short-term in the majority of studies. One study (Ricciardelli, 2006) found no signicant difference between groups in four of the targeted social skills but found a signicant difference for maintaining attention in favour of the control group. This trial (Ricciardelli, 2006) was the smallest trial (N = 6) in the review and integrated the use of Social Stories with a long-term formalized schoolbased social skills curriculum; all students received a daily schedule of 55 minutes devoted specically to social skills instruction and the intervention group received additional 2025 minutes of Social Stories training. The ndings raise the possibility that Social Stories add little benet alongside another intense, structured modality. As evident in previous reviews, there has been some uncertainty regarding the real effect of Social Stories on children with ASD due to weak treatment effects, confounding factors, inadequate participation, co-interventions, and poor study design and implementation. With this in mind, we restricted our inclusion to study designs (i.e., comparative trials) that provide the most unbiased estimates of treatment effectiveness. Despite the poor methodological quality of the included studies, it must be recognized that these trials represent an important change from previous studies in a eld which has concentrated mostly on single subject designs and reviews 658

Downloaded from aut.sagepub.com by guest on April 17, 2013

A REVIEW OF SOCIAL STORIES

based on that body of literature. All six studies were dissertations which represent an important source of information not previously considered for the most part. While these dissertations have not undergone peer review, their inclusion in the review limits the potential for bias that may result from including only studies that are published in peer-reviewed journals. Two questions not yet adequately addressed in this literature pertain to the generalization and maintenance of skills acquired through Social Stories. Most positive outcomes have occurred in settings that are highly structured and predictable, and in controlled environments engineered for success with highly trained personnel (Andrews, 2004; Bader, 2006; Feinberg, 2002; Romano, 2002). Further research is required to delineate to what extent benets can be achieved in other, less-structured settings (e.g., home) and with administration of the intervention by non-professionals (e.g., parents). Further, there is little information on how long the acquired skills can be maintained and the level of support or other circumstances required for maintenance. Two studies measured maintenance of skills over a one week period (Bader, 2006; Feinberg, 2002; Quirmbach, 2006) and one over a six-week period (Romano, 2002). The latter study found some maintenance of skills at six weeks but suggested the need for long-term and intensive treatment in order to achieve maintenance. One of the studies that evaluated game playing skills found a generalization of skills learned to other games (Quirmbach, 2006) but none of the other studies attempted to measure this dimension. The studies included in this review followed guidelines set out by Carol Gray for developing the Social Stories interventions. The intensity of treatment in the included studies varied from 2 trials during a single session to 10 trials a day and intervention period ranged from 2 days (sometimes 1 week apart) to 1 month. Gray indicated that success usually comes quickly within a two-week period. A lack of response suggests that the story needs to be re-written or re-introduced. Included studies all referred to the treatment frequency and duration but did not investigate the optimal treatment dose for the immediate response and long-term maintenance of skills. Grays intended target for this intervention is described as high-functioning individuals with autism. High-functioning autism was dened in some of the included studies as a minimum of grade one reading level, the ability to communicate either verbally or non-verbally, and a minimum of phrase speech. One study noted that a verbal comprehension index of at least 68 or greater on the WISC-IV was associated with better outcomes but had not dened the severity of autism based on verbal IQ. All of these children would represent those with the greatest potential to learn. One criticism of these studies is that the same story was often used for a group of children. Grays intent was for stories to be developed on an 659

Downloaded from aut.sagepub.com by guest on April 17, 2013

14(6) individual basis and personalized to specic situations. The results from this review demonstrate that the same story targeting a specic skill or behaviour can be used effectively among different children; however, different effects may be observed where stories are individualized for each participant. We followed guidelines for the development of methodologically rigorous systematic reviews (Higgins et al., 2006). One limitation of this review was our pragmatic decision to restrict inclusion to English language studies due to a lack of resources for translation. The other limitations of this review pertain primarily to the limitations of the existing studies. Additional unbiased controlled trials with larger sample sizes will serve to enhance the evidence base for Social Stories and bring greater clarity to outstanding questions.
AU T I S M

Conclusion
This rigorous systematic review of six controlled trials demonstrates that Social Stories may be benecial in terms of modifying target behaviours among high functioning children with ASD. Long-term maintenance, effectiveness of the intervention in other, less-controlled settings, and the optimal dose/frequency is unknown and requires further research.

Acknowledgement We are grateful to Lisa Tjosvold for the literature searches and reference setup for the manuscript. We thank Ben Vandermeer for assistance with the statistical analyses. Janine Odishaw helped with selection of studies for the review. We also thank Karalee Ratzlaff and Marie-Claire Saint-Jacques for assistance with article retrieval. This project was supported by a grant from the Alberta Centre for Child, Family and Community Research. References
Ali, S. & Frederickson, N. (2006) Investigating the Evidence Base of Social Stories, Educational Psychology in Practice 22(4): 35577. American Psychiatric Association (2000) Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR) 4th, Text revision edn. Washington, DC: American Psychiatric Association. Andrews, S.M. (2004) Increasing Game Playing Skills and Social Comprehension in School-Aged Children with Autism Using Social Stories. PhD, Alliant International University California School of Professional Psychology, San Diego. Andrews, S.M. (2005) Increasing game playing skills and social comprehension in school-aged children with autism using social stories. Dissert Abst Int B: Sciences and Engineering 65(10-B): 5386. Bader R. (2006) Using Social Stories to Increase Emotion Recognition and Labeling in School-Age Children with Autism [dissertation], PhD, Alliant International University, San Diego, CA.

660

Downloaded from aut.sagepub.com by guest on April 17, 2013

A REVIEW OF SOCIAL STORIES

Bryson, B., Rogers, D. & Fombonne, E. (2003) Autism Spectrum Disorders: Early Detection, Intervention, Education and Psychopharmacological Management, Canadian Journal of Psychiatry 48(8): 50616. Chalmers, T.C., Smith, H., Blackburn, B. & Silverman, B. (1981) A Method for Assessing the Quality of a Randomized Controlled Trial, Controlled Clinical Trials 2: 3149. Charlop, M.H. & Milstein, J.P. (1989) Teaching Autistic Children Conversational Speech Using Video Modeling, Journal of Applied Behavior Analysis 22(3): 27585. Cohen, J. (1995) Statistical Power Analysis for the Behavioral Sciences. New York: Academic Press. Detsky, A.S., Naylor, C.D., ORourke, K., McGeer, A.J. & LAbbe, K.A. (1992) Incorporating Variations in the Quality of Individual Randomized Trials into Meta-analysis, Journal of Clinical Epidemiology 45(3): 25565. Feinberg, M.J. (2001) Using social stories to teach specic social skills to individuals diagnosed with autism. Dissertation Abstracts International: Section B: The Sciences & Engineering, 62(8-B), Mar 2002, p. 3797. (UMI NO. AAI3024504) Feinberg, M.J. (2002) Using Social Stories to Teach Specic Social Skills to Individuals Diagnosed with Autism [dissertation], PhD, California School of Professional Psychology, San Diego, CA. Gray, C.A. (1994) The New Social Story Book. Arlington, TX: Future Horizons. Gray, C.A. (1995) Teaching Children with Autism to Read Social Situations, in Teaching Children with Autism, Strategies to Enhance Communication and Socialization, pp. 21941. New York: Delmar. Gray, C.A. (1998) Social Stories and Comic Strip Conversations with Students with Asperger Syndrome and High Functioning Autism, in E. Schopler, G.B. Mesibov & L.J. Kunce, (eds) Aspergers Syndrome or High-Functioning Autism? (pp. 16798). New York: Plenum Press. Gray, C. & Garand, J. (1993) Social Stories: Improving Responses of Students with Autism with Accurate Social Information, Focus on Autistic Behavior 8(1): 110. Hagiwara, T. & Myles, B.S. (1999) A Multimedia Social Story Intervention: Teaching Skills to Children with Autism, Focus on Autism and Other Developmental Disabilities 14(2): 8295. Higgins, J.P.T., Green, S., et al. (2006) Cochrane Handbook for Systematic Reviews of Interventions 4.2.6 [updated September 2006], in The Cochrane Library. Chichester, UK: John Wiley & Sons, Ltd. Jadad, A.R., Moore, R.A., Carroll, D., Jenkinson, C., Reynolds, D.J.M. & Gavaghan, D.J. (1996) Assessing the Quality of Reports of Randomized Clinical Trials: Is Blinding Necessary? Controlled Clinical Trials 17: 112. Marnie, S. (2005) Special Education for Children with Autism, Report to the chairman and ranking minority member, subcommittee on human rights and wellness, committee on government reform, house of representatives. Washington, DC: United States Government Accountability Ofce. Oxman, A.D. & Guyatt, G.H.(1988) Guidelines for Reading Literature Reviews, Canadian Medical Association Journal 138(8): 697703. Quirmbach, L.M. (2006) Social Stories: Mechanisms of Effectiveness in Increasing Social Skills, Social Skill Comprehension, Generalization and Maintenance of Newly Acquired Skills in School-Aged Children Diagnosed with Autism [dissertation], PhD, Alliant International University, San Diego, CA. Reynhout, G. & Carter, M. (2006) Social Stories for Children with Disabilities, Journal of Autism & Developmental Disorders36(4): 44569.

661

Downloaded from aut.sagepub.com by guest on April 17, 2013

AU T I S M

14(6)

Ricciardelli, D. (2006) A Social Skills Program Evaluation: Will Social Stories Combine with a Traditional Social Skills Curriculum Increase Pro-Social Behavior in Autistic Children? [dissertation], PhD, Fairleigh Dickinson University, Teaneck, NJ. Rogers, M.F. & Myles, B.S. (2001) Using Social Stories and Comic Strip Conversations to Interpret Social Situations for an Adolescent with Asperger Syndrome, Intervention in School & Clinic 36(5): 310. Romano, J. (2002) Are Social Stories Effective in Modifying Behavior in Children with Autism? PhD, Fairleigh Dickinson University, Madison, NJ. Sansosti, F.J., Powell-Smith, K.A. & Kincaid, D. (2004) A Research Synthesis of Social Story Interventions for Children with Autism Spectrum Disorders, Focus on Autism and Other Develeopmental Disabilities 19(4): 194204. Schulz, K.F., Chalmers, I., Hayes, R.J. & Altman, D.G. (1995) Empirical Evidence of Bias: Dimensions of Methodological Quality associated with Estimates of Treatment Effects in Controlled Trials, Journal of the American Medical Association 273: 40812. Smith, T., Scahill, L., Dawson, G., Guthrie, D., Lord, C., Odom, S., Rogers, S. & Wagner, A. (2007) Designing Research Studies on Psychosocial Interventions in Autism, Journal of Autism and Developmental Disorders 37(2): 35466. Thiemann, K.S. (2001) Social Stories, Written Text Cues, and Video Feedback: Effects on Social Communication of Children with Autism, Journal of Applied Behavior Analysis 34(4): 42546. Thompson, S.G. (1994) Systematic Review: Why Sources of Heterogeneity in Meta-Analysis Should Be Investigated, British Medical Journal 309(6965): 135155. Wetherby, A., Schuler, A. & Prizant, B. (1997) Enhancing Language and Communication Development: Theoretical Foundations, in D. Cohen & F.Volkmar (eds) Handbook of Autism and Pervasive Developmental Disorders. New York: Wiley. World Health Organization (1993) The ICD-10 Classication of Mental and Behaviour Disorders: Diagnostic Criteria for Research. Geneva: World Health Organization.

662

Downloaded from aut.sagepub.com by guest on April 17, 2013