PD 360 Impact Assessment:

A Study of the Impact of Participation in PD 360 on Student Performance in Hawaii

Prepared by Steven H. Shaha, PhD, DBA Spring 2011

Gains in Proficiency Rates

Improvement in Reading Proficiency
57.00

55.00
Proficiency Rate

53.00
51.00 49.00 47.00 Schools Districts

45.00
Pre Year Post Year

Consistent pattern:
Despite schools beginning at a disadvantage, by year 2 they not only close the gap, but surpass their districts.
Pre Year Post Year Improvement Pct Change
Statistical Significance Contrasting Schools with Districts

Schools Districts 46.98 51.67 56.12 55.12 9.15 3.45 19.5% 6.7%
p<.001 p<.001 p<.001

Data imply that improvements in schools boosted performance for the districts within which they were nested. School improvement outpaced the districts within which they were nested.
2

NOTE: Statistical significance is affected by sample size, wherein significance is achieved with smaller increases given larger samples or numbers of students.

Gains in Proficiency Rates

Improvement in Math Proficiency
58.00 56.00

Proficiency Rate

54.00
52.00 50.00 48.00 46.00 Schools

Districts

44.00
Pre Year Post Year

Consistent pattern:
Despite schools beginning at a disadvantage, by year 2 they not only close the gap, but surpass their districts.

Pre Year Post Year Improvement Pct Change

Statistical Significance Contrasting Schools with Districts

Schools Districts 48.23 51.66 56.46 53.25 8.24 1.59 17.1% 3.1%
p<.001 p<.001 p<.001

Data imply that improvements in schools boosted performance for the districts within which they were nested. School improvement outpaced the districts within which they were nested.
3

NOTE: Statistical significance is affected by sample size, wherein significance is achieved with smaller increases given larger samples or numbers of students.

Top 750 PD-360 Analysis: Expanded Findings

Steven H. Shaha, PhD, DBA July 2011

Reading Advantages
Districts gained 21.2%, while PD-360 Schools gained 30.5% for a 9.3% advantage (p<.001)

Schools Districts Advantage

Change 30.5% 21.2% 9.3%

Statistical Significance p<.001 p<.001 p<.001

Reading Advantages
The Districts rose 0.1%, while PD-360 Schools gained 4.8% for a 4.6% advantage* (p<.001)

* Figures do not sum perfectly due to rounding

Pct Change PD-360 Schools 4.8% Districts 0.1% Advantage 4.6%

Sta t Signif
p<.001 ns p<.001

PD 360 Impact Assessment:

Updated Findings regarding the Impact of PD 360 on Student Proficiency Rates
Prepared by Steven H. Shaha, PhD, DBA

For questions please contact Steven H. Shaha, PhD, DBA, steve.shaha@att.net, 801-879-5543. Dr. Shaha is an independent program evaluator, Professor at University of Utah and lecturer at Harvard University.

Improvements in Math Proficiency for PD 360

Math Proficient - 4th Grade

58.0 57.5 57.0 56.5 56.0 55.5 55.0 54.5 54.0 53.5 Prior Year District Benchmark Post-PD 360 Year

High Utilizing Schools improved by 4.9% over the District benchmark (p<.001) and Low Utilizing School improved by 2.0% (p<.01)
High Users Low Users

Improvements in Reading Proficiency for PD 360

Reading Proficient - 4th Grade

72.0 70.0

High Utilizing Schools significantly outperformed the collective district benchmark by 10.3% (p<.001) High Utilizing Schools significantly outperformed the Low Utilizing Schools by 2.9% (p<.01)
High Users Low Users

68.0
66.0

64.0
62.0 60.0 Prior Year District Benchmark Post-PD 360 Year

Low Utilizing Schools significantly outperformed the collective district benchmark by 7.2% (p<.001)

Improvements in Math Proficiency for PD 360

Math Proficient - 8th Grade

56.0 55.0

54.0
53.0

High Utilizing Schools improved by 4.9% over the District benchmark (p<.001) and Low Utilizing School improved by 2.6% (p<.01)
High Users Low Users

52.0
51.0 50.0 Prior Year District Benchmark Post-PD 360 Year

10

Improvements in Reading Proficiency for PD 360

Reading Proficient - 8th Grade

70.0 68.0 66.0

PD 360 Utilizing Schools significantly outperformed the collective district benchmark by 8.3% (p<.001) While the district fell by 3.2%
(p<.001)

64.0 62.0 60.0 58.0

56.0 Prior Year District Benchmark Post-PD 360 Year PD 360 Users District Avg

High Utilizing Schools significantly outperformed collective district change between years to create a performance gap of 11.9%
(p<.001)

11