Evaluation of

Blended Literacy Intervention (BLI)

Submitted to Ms. T, Teacher of Freshman BLI* Class
Submitted by Kathy Strickland, MET Candidate, Boise State University

*Fictitious names for the program and the teacher implementing it are used throughout this report.

Summary
Blended Literacy Intervention (BLI) is a program designed for students in grades 5-12 who are
reading two or more years below grade level. The program blends online instruction in
foundational literacy skills with teacher-led instruction in grade-level reading. By engaging
struggling adolescent readers and writers in relevant, age-appropriate content, BLI aims to
accelerate their learning and give them the knowledge and skills they need to be successful in
core ELA and content-area classes.
The purpose of this report is to inform stakeholders of the effect BLI implementation is having
on the attitudes and skills of students in Ms. Ts freshman intervention class. Students answer
survey questions (see Appendices A and B) regarding whether and to what extent they are
achieving the programs stated goals. This exercise can be beneficial to these individuals, as it
helps them clarify their personal goalswhat they want to get out of the program before the year
is through and what changes they want to see in themselves. Data from the surveys presented in
this report provide information that should prove useful to teachers and administrators as BLI
implementation continues in this and six other classrooms at the same high school.
Major steps in this program evaluation included:
1. Initial teacher interview to gather context and background; set teacher goals (early Jan.)
2. Initial question posed to students early Feb.: What do you want to get out of this class?
3. First online survey administered early March, followed by analysis and teacher interview
4. Second online survey administered mid-April, followed by analysis and teacher interview
5. Data compiled and results reported to teacher (end of April)
This evaluation found that the majority of students surveyed believed they were making progress
toward the programs goals and increasing their skills/knowledge in the areas of vocabulary,
comprehension, fluency, spelling, reading, and content areas (see Results section for details).
Evaluation results imply that the students and the teacher perceive the BLI program as effective.
Description of the Program Evaluated
Blended Literacy Intervention (BLI) is a program designed to accelerate the literacy growth of
students who are reading two or more years below grade level. The program is most often used
in high schools, and this report evaluates its implementation in a classroom of freshman students.
There are two levels of BLI, Level 1 and Level 2. Students in the same class can be at different
levels, although the implementation in this report is a Level 2 only classroom. The course length
varies for each student, depending on how quickly they progress through the program. The
program is an intervention, which means students are placed in a context apart from their peers
who are reading on grade level.
Students can exit the program and return to a grade-level ELA class upon successful completion
of Level 2, provided they also demonstrate understanding through achievement on the states
Standards Based Assessment (SBA) and/or BLIs Progress Assessment for Reading (PAR).
Program Objectives
The BLI program aims to promote at least a two-year gain in grade-level literacy over four
semesters or fewer of daily lessons, depending on the students entry point.

Overall goals of the BLI program include:

Improving vocabulary
Improving reading comprehension
Improving reading fluency
Improving spelling
Improving foundational skills and knowledge needed in grade-level content areas
(See Appendix C for more detailed goals from the programs Research Foundation.)
Program Components
BLI is a blended literacy intervention consisting of two major components: Word Training and
Text Training.

Word Training is conducted online, at the students own pace, via video tutorials
focused on phonological awareness and word recognition. Students also read online
independently, timing themselves and watching their progress. A sound library allows
students to record themselves reading and provides hints for correct pronunciation.
Interactive online activities, including encoding and decoding activities, reinforce
learning and provide teachers with ongoing progress monitoring.

Text Training is the teacher-led component of the program. It focuses on developing

reading comprehension and grade-level literacy skills through guided/close reading of
increasingly complex text in small groups or as a whole class. This instruction
emphasizes content-area and high-use vocabulary; grammar and syntax; comprehension;
writing skills, including sentence building, paragraph writing, basic essay structure, and
revising; and online practice activities assigned by the teacher.

Word Training and Text Training ideally happen in consecutive 45-minute sessions during a 90minute class period, although this is not always possible due to set schedules (see Procedures
section for specifics on this implementation). The philosophy behind the program is that
simultaneous instruction in foundational skills and grade-level reading will yield optimal results.
Other components of the program include the following assessments:
Word Training:
o Fluency Checks at end of each online unit
o End-of-Unit Assessments
Text Training:
o Content Mastery at end of each unit
o Power Pass text-readiness questions
Benchmark Assessments (administered online three times per yearbeginning, middle,
and end):
o Progress Assessment for Reading (PAR): measures reading comprehension;
yields a Lexile score
o Test of Silent Contextual Reading Fluency (TOSCRF): assesses the speed with
which students can recognize the individual words in a series of printed passages
that become progressively more difficult in content, vocabulary, and grammar
o Test of Written Spelling, Fourth Edition (TWS-4): uses a dictated word format to
measure spelling proficiency

Evaluation Method
An initial survey conducted in class by the teacher at the start of this evaluation (early February)
asked students what they would like to get out of the program. Students wrote down goals for
themselves, which aligned well with the programs stated objectives as well as with goals Ms. T
set for them during her initial interview (late January). These were the common threads:
Read better/faster
Learn big vocab words
Spell better
Learn things needed to do better in content-area classes
Feel better about reading ability
(See Appendix D for how students goals directly align with the programs objectives.)
After this initial question was asked by the teacher, students took two online surveys in class:
one in early March, before spring break, and the other in mid-April. In-class surveys were the
main evaluation method, followed up by phone and e-mail interviews with the teacher. Survey
questions aligned with the programs objectives to determine whether students believed those
objectives were being achievedand thus perceived the program to be effective.
Students were asked to rank their knowledge, skills, and confidence using a 5-point Likert scale,
yes/no and multiple-choice questions, and some open-ended short-answer questions (see
Appendices A and B for links to the complete online student surveys).
Participants
Students participating in this class have been identified as needing literacy intervention. They
were selected based on low ELA performance in earlier grades and low SBA and/or PAR scores.
Students in the evaluation classroom are all freshmen who were reading at approximately a
fourth to fifth grade level (average Lexile scores between 770-880) upon entry. They all tested
into Level 2, Unit 1, of the program (see table below).
Lexile Level Entry Points for BLI:
Spring
Grade
Level

Spring
Lexile

9 to 11

< 625

Fall
Grade Level

Fall
Lexile

Blended Literacy
Intervention Level
Placement
Recommendation

< 635

Level 1 Unit 1

625725

635740

Level 1 Unit 5

730920

745930

Level 2 Unit 1

925985

935995

Level 2 Unit 7

10 to 12

For most (approximately three-quarters) of the 22 students initially enrolled, this was their first
year in the program. About one-quarter of the students had previously received one year of BLI
instruction in middle school/junior high.
The instructor of this class, Ms. T, is a fourth-year teacher who has been implementing the
program since February of 2015. She switched from the traditional print version of the program
during second semester of the 2014-2015 school year after receiving approximately 2 months
of district training in BLI, provided by certified trainers from the company that created and
published the program. There are seven total classes implementing BLI at the same high school
during this 2015-2016 school year, and the print version of the program is no longer in use.
Procedures
The BLI class is four days a week: 50 minutes on Monday/Friday and 80 minutes on
Tuesday/Wednesday. One 50-minute period per week (either Monday or Friday) is dedicated to
Text Training, and the other to Word Training (see Description of the Program Evaluated
section above). Both 80-minute periods are split into half Text Training and half Word Training.
For Text Training, approximately 80 percent of BLI texts were read as a whole class, with the
other 20 percent read in small groups or pairs. According to the teacher, students rarely, if ever,
read a text for the first time independently.
The two online surveys given to students during this evaluation did not disrupt instruction. Ms. T
carefully selected times when students had just finished a unit and were receptive to selfevaluation. Surveys were given at the start of the class period introducing a new lesson or unit
(see Appendices A and B for the two online surveys conducted in this evaluation).
The teacher answered questions on her own time, including an initial phone interview describing
the background and context for the programs implementation. Later conversations, focused on
evaluation goals, were conducted before and after student surveys over the phone or via e-mail.
Data Sources
Teacher interviews and student surveys were the instruments used to collect data on a variety of
factors indicative of the programs impact and effectiveness. These factors include:
Participants attitudes about the program and what they are learning
Participants perception of their progress/achievement in BLI and other classes
Participants attitudes about school in general and their future
Participants confidence in their reading, writing, and communication skills
Participants perception of the impact of the program (whether it was stimulating change)
Teachers perception of students attitudes, progress, and achievements
Teachers perception of the impact of the program (whether it was stimulating change)
Results
A total of 15 students took the first online survey, and 12 students took the second online survey.
This sample was out of a total population of 22 students enrolled in the course at the start of the
evaluation and 17 at the time of the second survey.

Presented here are key results of the online student survey questions directly related to the
programs goals, followed by data related to students confidence and attitudes toward reading,
school, and the future.
Vocabulary Development
On Survey 1, 100 percent of participants said BLI is helping them learn bigger/harder words.
When asked on Survey 2, closer to the end of the school year, 75 percent of participants said BLI
helped them learn bigger/harder words. Teacher interviews confirmed improved vocabulary.
The first survey presented students the opportunity to select from a list which activities or
lessons were helping them build their vocabulary. Students were instructed to select as many as
apply from this list:
Online independent reading
Online video lessons
Online activities and quizzes
Recording myself as I read
Lessons my teacher gives to the class
Group reading with my classmates and teacher
Class discussions
Writing about what I read in class
As illustrated in the following pie chart, approximately one-third of participants indicated that
online independent reading was helping, while 27 percent selected group reading. Online
activities/quizzes and self-recording were other choices selected by 13 percent of the students
surveyed. (Numbers have been rounded throughout this written explanation.)

The second survey offered students the opportunity to type (in a short-answer field) which lesson
or activity helped the most with their vocabulary development. Two of these answers involved
writing down the words, and there were no other commonalities in response.

Reading Comprehension
In Survey 1, students were asked whether or not this class is helping them understand what they
read. All of the students surveyed answered Yes, compared with 75 percent in Survey 2. When
asked in Survey 1 to select from a list which activities or lessons were helping them understand
what they read, 40 percent chose group reading with my classmates and teacher, and 20
percent selected class discussions. Again, two other choices were picked by 13 percent of
respondents: online independent reading and online activities/quizzes.

The second survey offered students the opportunity to type (in a short-answer field) which lesson
or activity helped the most with their reading comprehension. The mode of these answers
mentioned vocabulary as a contributing factor (three out of seven responses).
Reading Fluency
On the first survey, 73 percent of respondents answered Yes to Is this class helping you read
faster? compared with 58 percent on the second survey answering yes to Did BLI help you
become a faster reader? When asked to select from a list activities or lessons that were helping
them read faster, 36 percent picked recording myself as I read, and 27 percent chose class
discussion. The following four choices were evenly distributed with 9 percent of respondents
selecting: group readings with my classmates and teacher, lessons my teacher gives to the class,
writing about what I read in class, and online independent reading.

On Survey 2 - where students were given the opportunity to fill in the activity or lesson that
helped them get faster at reading - timed readings, testing, and repetition were among the replies.
Spelling Skills
When asked on Survey 1 whether or not this class was making them better spellers, 87 percent
answered Yes. When asked on Survey 2 if BLI had helped them become better spellers, 75
percent responded Yes. On Survey 1, students selected these activities as helping them become
better spellers: 31 percent picked writing about what I read in class, while group reading with
my classmates and teacher and online activities and quizzes received 23 percent each.

In the short-answer format on Survey 2, there were no clear trends regarding which activity or
lesson students filled in as helping the most with their spelling, but a few responded that writing
the words helped. This may be an indication that these students perceived the TWS-4 assessment
as helpful (see Program Components section above). The teacher could not provide test scores.
Other Classes
On both surveys students were asked whether what they are learning in BLI is helping/helped
them in their other classes. On Survey 1, 80 percent said it was helping, citing improvement in
social studies, science, ELA, and math (see below). On Survey 2, 75 percent said it had helped,
and ELA (55%), science, and social studies (46% each) were selected most from a subject list.

Attitudes and Outlook

This evaluation also examined students confidence in their ability to communicate effectively
and their attitudes toward reading, school, and the future.
On Survey 1, in the middle of the semester, 100 percent of students said this class was helping
them read better. When surveyed again toward the end of the year, 100 percent of students said
they were better readers now than when they started the program. All but one of those students
indicated that BLI had helped them become better readers. On both surveys, group reading in
class was cited by the most students as an activity that helped them improve.
Half of the Survey 2 respondents gave themselves a 3 out of 5 when asked, How would you rate
yourself as a reader? (5 = great). None of those students gave themselves a score lower than 3.

Even though confidence in their reading ability increased, students attitudes toward reading did
not necessarily improve (see below). Responses to other survey questions showed no clear
connection between attitudes toward school and feelings about or performance in BLI.
On a scale of 1 to 5, how did you feel about reading when you started BLI (Pre-BLI), and
how do you feel about reading today (Midyear and End of Year)? (1 = hate it; 5 = love it)

10

Discussion
The purpose of this evaluation was to determine whether and to what extent students receiving
BLI instruction believe that the program is achieving its stated objectives, in order to inform
future implementation in this high school. These objectives align with the goals students
indicated they wished to achieve by taking this class (see Evaluation Method section).
Data in the Results section above demonstrate that the majority of students believe that BLI is
helping them improve in every stated objectivevocabulary, comprehension, fluency, spelling,
and foundational skills/knowledge needed for grade-level achievement in content areas (see
Program Objectives section). Typically 75 percent of respondents on Survey 2 and more on
Survey 1 answered Yes when asked if the program was helping them achieve these goals.
The highest rates of Yes responses (100 percent) were in the areas of vocabulary development
and comprehension on Survey 1. The lowest perceived impact BLI had on any of its stated goals
was in the area of reading fluency. Only 73 percent of respondents to Survey 1 said this class
was helping them read faster, while 58 percent responded Yes on Survey 2. On both surveys
these were the lowest Yes responses on any question related to whether the program was
helping them achieve key goals.
When students were given a list of activities and lessons to choose from on Survey 1, there was
often a large distribution of responses. The highest percentage any choice received was on the
question regarding reading comprehension. A record 40 percent of students said group reading
with my classmates and teacher was a factor contributing to their increased ability to understand
what they read. This indicates that the Text Training component of the program has been
effective with these students. Ms. T agreed that this is some of the most valuable instruction.
Student response was generally positive on survey questions not directly related to program
goals as well, including those intended to measure students confidence in their academic
abilities and attitudes toward reading, school, and the future.
A mean of 85 percent of the total survey population (Survey 1 and Survey 2 combined) said they
were looking forward to graduating from BLI. This could be because they were unsure whether
they would be exiting the intervention at the end of the year. Exit data was dependent on end-ofyear assessment and not yet available at the time of this report.
Encouragingly, all but one student surveyed toward the end of the semester (Survey 2) said they
were glad they took this class. On the same survey, all but one student also indicated that they
felt being in the BLI program this year would make them better students in 2016-2017.
In short, the programs effectiveness in the eyes of the students is clear. Although scores from
student assessments were unavailable for this report, interviews with the teacher also
demonstrated that most students in her intervention class were improving as a result of BLI
instruction. This was only a small sample of the schools population of BLI students, and it is
hoped that the results of this evaluation will inform and inspire other evaluations so that more
data can be collected. The more informed students and teachers are about the impact BLI is
having, the better equipped they will be to get the most out of its implementation.

11

Appendices
The following appendices provide links to the online surveys used in this evaluation of BLI as
well as additional information on program goals.
Appendix A
Here is the link to the first in-class online survey, given to students in early March:
https://docs.google.com/a/u.boisestate.edu/forms/d/1J0brus9KDy_zGCE3fBiUC_2ULqzm6lWugE4ULkMswI/edit?usp=drive_web#responses
Appendix B
Here is the link to the second in-class online survey, given to students in mid-April:
https://docs.google.com/a/u.boisestate.edu/forms/d/1Dn9HtdZJmtCKv0DAUueHCLHZkpqcksH
Zz2TWI__sAGs/edit?usp=drive_web#responses
Appendix C
The following list presents more in-depth program goals taken from the Research Foundation of
the actual program referred to in this report as Blended Literacy Intervention (BLI).
The program aims to remediate gaps in the foundational language skills that enable reading,
including:
Conscious awareness of speech sounds in spoken words (i.e., phonological awareness)
Analysis and spelling of written words by phoneme-grapheme correspondence, syllable,
and morpheme (i.e., word recognition)
Achievement of sufficient reading fluency to support comprehension
Recognition and use of word meanings (vocabulary)
Understanding of grammar, syntax, and usage for speaking, reading, and writing
Comprehension of informational and narrative text
Written expression
Appendix D
This is the evaluators program description (EPD) used to align program goals with student
activities, evaluation questions, and evidence of effectiveness. Note how program goals align
with the goals students identified for themselves in written responses to the initial question asked
by their teacher at the start of this evaluation: What do you want to get out of BLI? Student
goals are presented under the main program goals as Student Speak in the table below.

12
Program Goals

Program Activities

Evaluation Questions

Evidence of Program Merit

Master foundational
literacy skills as needed
on an individual level

Online video tutorials

To what extent have

students mastered
foundational skills they
did not previously grasp?

Students successfully
complete Word Training
lessons

Students successfully read

and use vocabulary words
taught in the program

Direct preteaching and

teaching of vocab words

Do students demonstrate
understanding of
academic, content-area,
and high-use vocabulary
words in their reading
(independent recordings
and whole-class readings),
responses, and
discussions?

Whole-class directed
reading of ageappropriate text that
increases in complexity
over time

To what extent has

students comprehension
of grade-level text read in
this class and others
increased?

Increase in Lexile score

Student Speak: Fix

learning gaps

Build academic, contentarea, and high-use

vocabulary
Student Speak: Learn big
words

Comprehend grade-level
text
Student Speak:
Understand reading in
other classes so I can get
good grades

Online educational game

(Sight Words)
Self-paced online lessons

Individual online lessons

Whole-class close
reading of informational
text containing domainspecific content

Graphic organizers and

comprehension exercises

Student Speak: Read

faster and better

Self-record reading
passages online
Read together as a class

Students no longer skip as

many words while reading;
can read and discuss words
in class and apply them to
content areas

Students demonstrate
understanding of text read
in lessons
Meaningful group
discussions and written or
oral answers to
comprehension questions

Videos to build
background/content
knowledge
Improve reading fluency

Students build confidence,

feel better about their
ability to read and succeed
in school

Improved performance in
content-area classes
To what extent has
reading fluency
increased?

Read with a partner

Activities to measure
number of words correct
per minute

Increase in percentile rank

on Test of Silent Contextual
Reading Fluency (TOSCRF),
which assesses the speed
with which students can
recognize the individual
words in a series of printed
passages that become
progressively more difficult
in their content,
vocabulary, and grammar
Progress indicated on
evaluation of students selfrecordings and group
readings

Improve spelling of gradelevel words

Student Speak: Be a better
speller

Online videos and

activities revealing
spelling patterns and
Greek/Latin roots
Writing prompts and
activities

To what extent has

students spelling
improved?

Increase in percentile rank

on Test of Written Spelling,
4th Edition (TWS-4)
Spelling improvement in
written responses to
activities and prompts