Академический Документы
Профессиональный Документы
Культура Документы
Abstract
The study addresses the extent to which subtests on the Dynamic In
dicators of Basic Early Literacy Skills Reading Assessment (DIBELS;
Good & Kaminski, 2002) predict student success on a measure of read
ing comprehension and if prediction is consistent for native and second
English Language Learners. 2,649 elementary students were assessed
on a reading comprehension measure, of which 29.7% were English
Language Learners. Descriptive and analytic statistics were generated
including bivariate correlation analysis split by language proficiency.
Critical measures and suggested cutoff values (Good, Simmons, et al.,
2002) were evaluated for predictive utility by visualization of Receiver
Operating Characteristic (ROC) curves (Swets, Dawes, & Monahan,
2000), and comparison of the area-underthe- curve (AUC) values. DI
BELS better predicts children who are at low risk than those at
risk; however, DIBELS correctly classifies children at risk better
for ELL than non-ELL students in third grade.
Key words: English Language Learners (ELL), DIBELS, Reading,
Sensitivity, Specificity
87
88 / Reading Improvement
Introduction
The Dynamic Indicators of Basic Early
Literacy Skills (DIBELS; Good & Kaminski,
2002) is a measure designed to assess 3 of the
5 big ideas of early literacy espoused in the
National Reading Panel report (National Insti
tute of Child Health and Human Development,
2000): Phonological Awareness, the Alphabet
ic Principle, and Fluency with Connected Text.
Measures of Phonological Awareness include
Initial Sound Fluency (ISF) which assesses a
childs skill to identify and produce the initial
sound of a given word, and Phonemic Segmen
tation Fluency (PSF) which assesses a childs
skill to produce the individual sounds within a
given word. A measure of the Alphabetic Prin
ciple is Nonsense Word Fluency (NWF) which
assesses a childs knowledge of letter-sound
correspondences as well their ability to blend
letters together to form unfamiliar nonsense
(e.g., fik, lig, etc.) words, and a measure of
Fluency with connected text is Oral Reading
Fluency (ORF) which assesses the number of
correct words a child can read per minute in
grade level connected text.
The authors of the DIBELS claim that its
subtests are reliable predictors of reading under
achievement and thus may be used to identity
students in need of intervention and to reliably
determine student progress (Good, Simmons
6 Kameenui, 2001). DIBELS is being used in
thousands of schools across the nation, often to
provide formative data to schools accountable
for increasing student achievement on end-ofgrade-level state reading achievement tests. In
spite of its widespread use, some question its
utility in assessing reading comprehension, the
undisputed goal of reading (Good et al., 2001).
Samuels (2006) has been particularly critical
of fluency measures on the DIBELS assert
ing that fluency involves both decoding and
comprehending texts simultaneously, whereas
the DIBELS Oral Reading Fluency measure
focuses on decoding speed and does not assess
comprehension. This may have particularly
Participants
Participants in the study were a cohort of
2,649 elementary school students in one of
the western states in the United States. These
students attended schools which received grant
monies as part of a national literacy reform
initiative targeting schools with themost chal
lenged students in terms of low achievement
and socioeconomic status. These students
were assessed at the conclusion of the 2006
school year for reading achievement on the
state assessment, and had received reading
interventions for two or more years previ
ous. 33.1%of the sample were Caucasian,
Table 1 2006 CSAP Grade 3 Reading Means, Standard Deviations and Percent of Total
by Subgroup
Subgroup
Total Tested
Mean SS
SS STD
% of Total Tested
Native American
627
450.44
48.26
1.16%
Asian/Pacific Islanders
1981
477.63
54.81
3.65%
African American
3206
443.91
54.05
5.91%
Hispanic
14689
442.34
52.14
27.08%
White
33942
480.83
49.48
62.58%
Female
26747
475.53
53.78
49.31%
Male
27698
460.35
52.66
51.07%
Not ELL
46933
562.59
72.15
86.53%
ELL
7304
499.44
83.18
13.47%
Total Tested
54239
554.09 7
6.80
90 / Reading Improvement
Percent of Total
Sample
White
840
33.1%
1.4%
98.6%
Hispanic
1557
61.3%
48.7%
51.3%
92
3.6%
95.7%
4.3%
Demographics
African American
Percent ELL
Students
Percent Non-ELL
Students
Asian/Pacific Islander
24.
9%
41.7%
58.3%
Native American
26
1.0%
11.5%
88/5%
No Ethnicity Label
110
4.2%
33.6%
66.4%
Male
1337
50.5%
30.9%
69.1%
Female
1312
49.5%
30.9%
68.7%
Free/Reduccd Lunch
Eligible
1903
75%
38.7%
61.3%
Not Free/Rcduced
Lunch Eligible
636
25%
8.0%
92.0%
Total Students
2649
29.7%
66.1%
Table 3 Distribution of Language Proficiency for Total Sample and for ELL Students
Language Proficiency
Total Sample
1752
66.1%
No English Proficiency
54
2.0%
6.9%
399
15.1%
50.7%
334
12.6%
42.7%
No ELL Label
110
4.2%
Not ELL
Total
2649
92 / Reading Improvement
indicates that the states third grade reading
assessment has strong internal consistency and
that the test produces relatively stable scores.
For 2006, the third grade assessment consisted
of 32 multiple choice items and 8 constructed
response items. The mean p-value for the mul
tiple choice itemswas .70 and themean p-value
for the constructed response items was .51.
Descriptive statistics for the third grade
reading assessment are in Table 1. Table 2
contains descriptive statistics for the study
sample, and Table 3 contains language pro
ficiency information about the study sample.
The DIBELS Monitoring Assessments
The DIBELS assessments are asserted
to be reliable predictors of reading under
achievement that may be used to identify stu
dents in need of intervention and to reliably
determine student progress to provide schoolbased data to inform instruction and to review
school level outcomes.
Table 4 lists the six tests used for bench
marks and/or progress monitoring. Good,
Simmons et al. (2002) used evaluation of the
predictive capacity of each test in the suite to
recommend a process for monitoring the de
velopment of literacy skills.
Table 5 summarizes the reported reliability
statistics for the DIBELS suite. The reliabil
ity and validity of the Oral Reading Fluency
tests have been the focus of most evaluations
reported in the literature. Table 6 summarizes
the reliability and validity tests that have been
reported for the Oral Reading Fluency (ORF)
For evaluating student proficiency, the
third grade scale scores are categorized as:
Proficiency Level
Advanced Proficiency
Proficient
526 to 655
Partially Proficient
466 to 525
Unsatisfactory
Grades
Kindergarten
Phoneme Segmentation
Fluency (PSF)
Specific Test(s)
Coefficient
Cronbachs alpha
.92
Test-retest
.92 to .97
.80
.66
McKnight (2001)
Inter-rater reliability
High 80s
Elliott (2001)
Inter-rater reliability
.82 to .94
Tcst-retest
.74 to .93
Hintzc (2003)
Equivalent forms
.64 to .91
Cronbachs alpha
.49 to .69
Table 6 Results of Comparable Studies Relating DIBELS ORF Scores and State
Assessments of Reading
Study
.43 to .44
.62 to .78
Wilson (2005)
.74
VanDerMecr (2005)
.61 to .6 5
.73 to .80
Barger (2003)
McGlinchcy & Hixon (2004)
Current Study
.73
.49 to .81
.604 to .628
94 / Reading Improvement
were evaluated for predictive utility by vi
sualization of ROC (Receiver Operating
Characteristic) curves (Swets, Dawes, &
Monahan, 2000) and comparison of the areaunder-the-curve (AUC) values.
Results
Students in the sample for the current
studywere by definition at risk by attending
schools that qualify for specific grant monies.
As a result, the distribution of students in the
sample is not typical of the Colorado population
at large as can be seen by comparing Tables 1
and 2. Project schools have a higher proportion
ofininority students, students that qualify for
free and reduced lunches, and students who are
English Language Learners (ELL).
The Pearsons correlation coefficients for
the relationship between D1BELS Oral Read
ing Fluency (ORF) scores for Fall, Winter, and
Spring of the third grade and the CS AP reading
scale score appear in Table 7. The correlations
DIBELS Winter
Non-ELL
ELL
ALL
DIBELS Spring
Non-ELL
ELL
DIBELS Winter
0.912
0.906
DIBELS Spring
0.877
0.873
0.879
0.912
0.908
0.912
CSAP Spring
0.582
0.558
0.587
0.607
0.588
0.602
ALL
Non-ELL
ELL
0.909
0.623
0.603
0.628
% At Risk Correctly
Identified
.58 to .86
.52 to .88
.41
.90
.81
.91
Wilson (2005)
.93
.82
VanDerMcer (2005)
.96
.72
.77
.72
.51 to .64
.92 to .93
This Study
H
01
200
100
100
200
300
96 / Reading Improvement
ALL STUDENTS
A t Risk
Some Risk
Low Risk
(< 8 0 )
(80 -1 0 9 )
(>110)
Below Proficiency
298
171
73
542 (26%)
168
554
836
1558 (74%)
Total
466(22%)
725(35%)
909(53%)
2100
Total
ALL STUDENTS
A t Risk
Some Risk
Low Risk
(< 80 )
(80 -1 0 9 )
(>110)
Below Proficiency
369
125
57
551 (26%)
316
521
736
1572 (74%)
Total
685(32%)
646(30%)
793(37%)
2124
Total
ALL STUDENTS
At Risk
Some Risk
Low Risk
(< 8 0 )
(8 0 -1 0 9 )
(>110)
Below Proficiency
385
118
57
560 (26%)
366
549
659
1574 (76%)
Total
751 (35%)
667(31%)
716(34%)
Total
Note: Sensitivity and Specificity calculations did not include students whose ORF scores were in the
midrange (i.e., students at some risk)
ALL STUDENTS
Sensitivity
Specificity
% At Risk
% Low Risk
correctly
correctly
Overall
classified
classified
Accuracy
0.80
0.83
0.64
0.92
0.82
0.87
0.70
0.54
0.93
0.75
0.87
0.64
0.51
0.92
0.71
0.77
0.77
0.58
0.89
0.77
0.76
0.78
0.58
0.89
0.77
0.82
0.69
0.54
0.90
0.73
0.54
0.92
0.77
0.81
0.80
0.53
0.90
0.71
0.81
0.79
0.71
0.64
0.54
0.79
0.67
0.76
0.86
0.59
0.93
0.84
0.85
0.77
0.70
0.89
0.80
0.83
0.74
0.49
0.93
0.76
0.91
0.58
0.60
0.91
0.71
0.83
0.70
0.47
0.93
0.73
0.92
0.50
0.57
0.89
0.67
0.75
0.81
0.57
0.91
0.79
0.81
0.64
0.61
0.83
0.71
0.72
0.83
0.59
0.90
0.80
0.81
0.64
0.58
0.84
0.70
0.80
0.75
0.53
0.92
0.76
0.85
0.51
0.57
0.82
0.66
0.47
0.94
.77
00.82
0.81
0.64
0.86
0.76
0.77
0.77
0.43
0.93
0.70
0.81
0.79
0.74
0.82
0.71
0.83
0.79
0.63
0.72
0.55
0.78
0.69
0.86
0.44
0.53
0.81
0.61
98 / Reading Improvement
References
Barger, J. (2003). Comparing the DIBELS oral reading
fluency indicator and the North Carolina end o f grade
reading assessment (Technical Report.) Asheville,
NC: Carolina Teaching Academy.
Buck, J. & Torgcnson, J. (2003). The relationship be
tween performance on a measure o f oral reading flu
ency and performance on the Florida comprehensive
assessment test. FCRR Technical Report #1. Florida
Center for Reading Research.
Elliott, J., Lee, S., & Tollefson, N. (2001). A reliability
and validity study o f the dynamic indicators o f basic
early literacy skills - modified. School Psychology
Review. J0(l):33-49.
Good, R. H., Gruba, J.& Kaminski, R. (2001). Best
practices in using dynamic indicators o f basic early
literacy skills (DIBELS) in an outcomesdriven mod
el. In A. Thomas & J. Grimes (Eds.), Best Practices
in School Psychology IV (pp.679-700). Washington,
DC: National Association o f School Psychologists.
Good, R.H., Simmons, D.C., & Kameenui, E. J. (2001).
The importance and decision-making utility o f a con
tinuum o f fluency-based indicators o f foundational
reading skills for third-grade high stakes outcomes.
Scientific Studies o f Reading, 5, 257-288.
Good, R. H., Kaminski, R. A., Smith, S., Simmons, D.,
Kameenui, E., & Wallin, J. (In press). Reviewing
outcomes: Using DIBELS to evaluate a schools core
curriculum and system o f additional intervention in
kindergarten. In S. R. Vaughn & K. L. Briggs (Eds.),
Reading in the classroom: Systems fo r observing
teaching and learning. Baltimore: Paul H. Brookes.
Good, R. H., Simmons, D., Kameenui, E., Kaminski, R.
A., & Wallin, J. (2002). Summary o f decision rules
Copyright of Reading Improvement is the property of Project Innovation, Inc. and its content
may not be copied or emailed to multiple sites or posted to a listserv without the copyright
holder's express written permission. However, users may print, download, or email articles for
individual use.