Evalua&on

of Xtreme Reading Implementa&on

Sean Murphy North East Leadership Academy Cohort 1 Spring 2012, NCSU

Problem of Prac&ce
STRATEGIES
A: Train teachers on the 13 instruc&onal stages used year long as core instruc&onal strategies

OBJECTIVES

ULTIMATE GOAL

Teachers know and use research-based strategies eec&vely

B: Train teachers on research based literacy strategies through Xtreme Reading Program

Teachers dieren&ate instruc&on

Teachers know how to reach struggling learners At-risk students are more condent and successful readers.

C: Train teachers to use Possible Selves strategy

Students collaborate during class Students use literacy strategies prociently

D: Provide on-going support for Xtreme reading teachers and students

STRATEGIES

Evalua5on Ques5ons
How many hours were spent training teachers on 13 stages of instruc9on? How many hours were spent suppor,ng teachers on the 13 stages of instruc9on? Did the teachers think the session was of high quality? On how many strategies were teachers trained? How many professional development training sessions were given? To what extent did teachers think the trainings were useful for their instruc9on?

Data
Training agendas Support logs Professional Development Ques&onnaire

A: The 13 instruc&onal stages provided an instruc&onal rou&ne of pedagogic prac&ce (the how).

B: Xtreme Reading included 8 research-based literacy strategies explicitly taught to students (the what). C: Possible Selves was a mo&va&onal strategy. D: On-going support for Xtreme Reading teachers and students included coaching, planning assistance and modeling.

Training agendas Support logs Professional Development Ques&onnaire

For the purpose of the project, results were insignicant . How many hours of support were provided for the strategies? How supported did teachers feel during the process? Likert Scale Focus Group

Objec5ves

Evalua5on Ques5ons
How condent do teachers feel using research-based strategies? How oEen do teachers use research- based strategies?

Data

1: Teachers can explicitly teach literacy strategies to students (the what).

Professional Development Ques&onnaire Teacher Self-Report Survey

2: Teachers dieren&ate instruc&on, as laid out in the 13 Instruc&onal Stages (the how).

How oEen do teachers dieren9ate instruc9on?

Learning Strategies Walk Through Rubric Retrospec&ve Pre/Post Student Survey Pre-Post Assessments for Each Strategy

3: Students collaborate during class, as laid out in the 13 instruc&onal stages (the how).

How oEen do students collaborate during class?

4: Students use literacy strategies prociently

How many students used the strategies prociently?

Results
A: The 13 instruc&onal stages provided an instruc&onal rou&ne of pedagogically prac&ce (the how)

According to training agendas, teachers received approximately 4 hours of training on the 13 stages of instruc&on at the beginning of the year. According to the support logs, teachers received 7 hours of support specically regarding teacher prac&ces (13 stages of instruc&on). Teachers did not receive support for 13 stages of instruc&on (as a whole model) a`er ini&al training. However, there was specic support for dieren&a&on and student collabora&on. For the former teaching strategy, I provided 3 hours of support (mainly through direc&ve support and modeling) and curriculum coaches provided .5 hours. Teachers received 1 hour of support on student collabora&on from myself and 2.5 hours of support from coaches.

Results
B: Xtreme Reading included 8 research-based literacy strategies explicitly taught to students (the what)

Teachers were trained on a total of 3 strategies during the rst semester of implementa&on, conducted over 4 training sessions. Teachers received a total of 12 hours of training each during the Fall Semester. Teachers were trained on 2 strategies from January through March in 3 training sessions, totaling 8 hours for each teacher. Over the course of the semester teachers were trained on literacy strategies. Their reac&on was mixed. During the rst session, 1 out of 3 teachers strongly agreed that training was useful for their instruc&on, 2/3 disagreed. For the second session, 1/3 strongly agreed and 2/3 agreed. With the third session, all teachers strongly disagreed that it was useful. All agreed the fourth session was useful.

Results

Results
C: Possible Selves was a mo&va&onal strategy.

Ques&onnaires showed that 3/3 teachers felt condent and prepared to teach the Possible Selves strategy a`er training. According to interviews, teachers were introduced and discussed how to use Possible Selves strategies in the classroom. Sharing of Possible Selves lessons was facilitated by administra&on during weekly PLC sessions. Only 1 lesson was shared, student ar&facts were collected but not reviewed.

D: On-going support for Xtreme reading teachers and students included coaching, planning assistance and modeling.

Results

Support logs showed administra&on provided 26 hours of support either individually or with Xtreme teachers as a PLC, over the course of the school year. Support came in the form of coaching, modeling and facilita&ng planning. University of Kansas coaches provided 6 hours. Support included coaching and facilita&ng planning. According to self-report likert scales, 3 out of 3 teachers strongly agreed that administrators prepared them for implemen&ng strategies. They also strongly agreed that administra&on supplied adequate on-going support to eec&vely teach strategies. However, the feelings about curriculum coaches were less enthusias&c. 2 teachers agreed that the coaches prepared them for implemen&ng strategies and 1 disagreed. Regarding the on-going support from curriculum coaches, 1 teacher felt neutral about the level of support and 2 felt inadequately supported. In a 30-minute focus group, teachers voiced nega&ve feelings toward support 18 &mes and posi&ve feelings toward support 6 &mes. It was stated that more on-going support and coaching was needed a`er ini&al training. Specically, teachers felt neglected, abandoned, and used by the curriculum coaches a`er training. They generally agreed that administra&on provided on-going support (4 of 6 posi&ve comments were directed toward administra&on).

1: Teachers can explicitly teach literacy strategies to students (the what)

Results

Based on survey data, 66% of teachers they strongly agreed that they were condent teaching three out of four of the strategies they received training. They were least condent teaching the Word ID strategy. In comments, teachers said this training was ineec&ve because it was conducted virtually through Face Time link. 26 copies of the Learning Strategies Walk Through rubric showed that teachers were using research-based strategies on 12 occasions; 4 in January, 6 in February, 2 in March.

Results
2: Teachers dieren&ate instruc&on, as laid out in the 13 Instruc&onal Stages (the how)

According to 26 copies of the Learning Strategies Walk Through Rubric, teachers dieren&ated instruc&on 18 &mes; 2 in January, 6 in February and 10 in March. Dieren&a&on consisted of purposeful organiza&on of group work, leveled texts, task variety and individual support.

3: Students collaborate during class, as laid out in the 13 instruc&onal stages (the how)

Results

According to a retrospec&ve pre/post survey, Xtreme Reading students said the number of &mes they collaborated in the classroom increased from the rst days of school to the middle of spring. Specically, in response to a prompt asking, how o`en did you collaborate the beginning of the school year, 33% responded about once a week, 29% responded once or twice a week and 14% collaborated everyday. In the same survey, a prompt asked student, how o`en do you collaborate now? 20% of students marked that they collaborated once a week, 42% reported once or twice a week and 25% said they collaborated everyday. In 26 copies of the Learning Strategies Walk Through Rubric taken from December-March, students were collabora&ng 18 &mes. Collabora&on consisted of student paired-reading 8 &mes and collabora&on on projects 10 &mes. 2 in January, 6 in February, 10 in March.

Results

Results

Recommenda&ons: Strategies
Provide More Spiraled Support for 13 Stages of Instruc5on: One major lesson learned, in reec&on, was the necessity for clarifying the need for instruc9onal change. One of the reasons Xtreme Reading was chosen as an interven&on program was that it required dieren&ated and collabora&ve work. At the &me of ini&al training it was assumed that this was already a rou&ne, to some extent, in the classrooms. Thus, teachers did not have ample training on eec&ve pedagogy, as presented in the 13 instruc&onal stages rou&ne. Informal and formal observa&ons showed that South Johnston teachers relied heavily on direct instruc&on and assessment. This was true both before and a`er the program, proving the need to measure the amount of dieren&ated instruc&on and collabora&ve work. In preparing for next years implementa&on, much considera&on and focused support should be given on instruc9onal change. Oer more support. Three major changes should be made in suppor&ng methods: 1) Frontload support, 2) Oer more ongoing support, 3) Make support more direc&ve. First, a`er consul&ng with the KU coaches, it is clear that amount of support should be more frontloaded next year, with consistent visits and updates during the rst quarter. Second, while the amount of support might decrease in the second semester, open and regular dialogue should persist throughout the implementa&on (including analysis of data collected). The third point will be discussed in more detail below. At this point, though, it is worth men&oning that support should be more rooted in the materials and include specic follow-up about how instruc&on can be improved. Manage the Transi5on. In hindsight, it is clear that the transi&on from should have been planned and managed more eec&vely. Many of the teachers made it clear in support sessions that they were unfamiliar with specic strategies. They voiced apprehension with teaching reading strategies explicitly and had a hard &me lenng go of tradi&onal instruc&onal prac&ces (predominately lecture and test). This could be expected, but plans should have including pathways forward, including methods for celebra&ng accomplishments and acknowledging areas for improvement.

Recommenda&ons: Objec&ves
U5lize the programs resources. This was alluded to above in the oer more direc&ve support bullet. The Xtreme Reading program has a vast amount of resources, including lesson plans, pacing outlines, and forma&ve assessments that can inform instruc&on. Yet, almost all of the support we oered (especially in the beginning of the year) did not put these materials to use. On a regular basis, administra&on and/or instruc&onal coaches should sit down with the teachers and the materials. This can be used to ground conversa&on. Teachers should reect on how closely their planning and pedagogy follow the guidelines. Of course, teachers should have the autonomy to make modica&ons; however, they must know what the standard looks like and try adhering to the program as it is designed. Make Goals Clearer and More Prominent: In retrospect, the outcomes for the program shouldve been framed in both teacher and student terms. We were trying to change instruc&onal habits as much as content. However, this is a more delicate process, as former teaching habits were already in place, demanding apen&on to the transi&on. Whats more, the en&re instruc&onal cycle should have been emphasized, so that teachers were constantly reminded how dieren&ated teaching and collabora&ve learning t into the framework for teaching. Once the goals were stated more clearly, systems should be in place to regularly monitor progress. The walk through form could have been used more consistently with teachers, anchoring coaching conversa&ons.

Evalua&on Reec&ons
Dont ask so many ques9ons. To be embarrassingly honest, I didnt mean for this PowerPoint to be this long. However, by asking so many ques&ons, I le` myself with liple choice. I remember Dr. Corn saying, Keep it simple, youll have to collect data on everything you ask. Well, I guess we learn from experience. Stop and smell the data. I waited too long to really start analyzing the data I was collec&ng. If I had started earlier, I wouldve no&ced a papern (content not changing, pedagogy changing) earlier on. This could have shaped student interven&ons and teacher support. Sense of ecacy from understanding the process. In general, I didnt have the slightest idea about how to conduct an program evalua&on. This projects helped me take the long-view. By seeing how the en&re cycle is completed, Ill be more prepared every step of the way, from planning to reec&on.