Zeros or No-zeros?

Stats from 6 EPSB high schools

--No Zero Policy-Evidence It Does NOT Work
An initial investigation into the relationship between a no-zero policy and 5-year longitudinal data for school performance, diploma completion rate and other factors.
While the 8 graphs could stand alone as reason to question the practice, explanatory notes have been added so parents, students and the general public may gain a better understanding. EMAIL: critically.analyzed@gmail.com

Zeros or No-zeros? Stats from 6 EPSB high schools

A colleague challenged me to research and consider the effectiveness and soundness of a reformed grading policy as presented by Ken OConner AND others. Finding little more than anecdotal evidence and OConners own admissions on this The Grade Doctor site that he is not aware of any hard-core research, I concluded I had to do my own study. Please consider it as only a starting point for a reevaluation of the this new assessment model that is apparently being imposed on EPSB high schools.

Zeros or No-zeros? Stats from 6 EPSB high schools

My Research Evolved Into Three Realizations:
I was surprised by my findings. Some of the reformed practices have been conducted by caring, master teachers for decades and agree with OConnors Grading for Learning type of philosophy. But searches reveal that the policy is certainly not accepted by all educators and is clearly controversial. Evidence is not hard evidence, only anecdotal. Although I am frustrated, Im trying to get beyond the no-zero/zero argument to find middle ground here and identify root causes so we could work together at restoring a healthy educational environment. (The 8 charts included are clearly very telling--so whats wrong?) A new operating system of management proposed by Daniel Pink (quoted as an authority on motivation by supporters) suggests autonomy be given to students to promote mastery and sense of purpose.

(Shouldnt this autonomy be given back to teachers as well for the same reason?)

Zeros or No-zeros ? Stats from 6 EPSB high schools

There was a total vacuum when I tried to find some real evidence that supported the use of codes vs. zeros as motivators. Did using zeros as placeholders discourage students and make them drop out? Anecdotal evidence was all that I could get, and it was typically standards based stuff at elementary and may not be best suited to high schools--especially in Math and Sciences. While monitoring his blog called The Grade Doctor, even OConner himself admits several times that he knows of no research to support the movement and when he did mention empirical evidence he does not offer links either in his blog or on his website.

Zeros or No-zeros ? Blog Comment 1

Zeros or No-zeros ? Blog comment 2

Zeros or No-zeros ? Blog comment 3

Zeros or No-zeros ? Stats from 6 EPSB high schools

How did the known hard-core no zero or code policy Edmonton schools rate province-wide on the standardized provincial diploma exams? I decided to check the Fraser Institutes Alberta Report Card .
http://alberta.compareschoolrankings.org/ChooseReport.aspxf

(Provincial Diploma exams are a very good mirror for students, parents, teachers and secondary institutions since it the ONLY authentic, trustworthy mark indicating that students are being marked consistent with provincial standards. The great inflation of marks of the past 4 years are making the diploma exams LOOK like they are of too high a standard, but the difficulty level of these exams has been fairly consistent. Its the school mark worth 50% that is questionable sometimes due to principals changing marks or teachers being told to do so.) (The linked document explains the Fraser Institutes methodology and how, in their final rating, actually compares schools only with schools in the province of the same socioeconomic level and similar course offerings).

Zeros or No-zeros ? Stats from 6 EPSB high schools

I used the Frazers comparison tool to produce the following charts for the 5 yr span 2006-2010 for 6 schools only (a limit of the comparison tool):

One school where teachers were instructed by an administration implementing a strict interpretation of the reformed grading philosophy for the 5 year period (they do NOT use
any zeros but use codes instead -- Eastglen H.S.)

One school where the administration openly professes that they leave such decisions up to teachers since they are in the best position to evaluate the students (Some teachers there claim
that the use of zeros builds responsibility and citizenship -- Strathcona H.S.)

Ross Sheppard and 3 other high schools with varied implementation of the new philosophy. (Ross Shep had JUST started
the policy in the spring of 2011 and many teachers were not using it--or were confused)

Zeros or No-zeros ? Stats from 6 EPSB high schools

NOTE 1: I do NOT criticize the teachers they may well be the victims, having had their autonomy and professional discretion removed. NOTE 2: As you review the charts, consider the trend over the 5 years. Eastglen, being a school from a lower socio-economic area, may have different statistics. However, if there is a significant change in values over the 5 years, one has to consider the possible reasons for the change. NOTE 3: Definition of terms and more detailed data for each school are on the Fraser Institute Website and in the Appendices.
http://alberta.compareschoolrankings.org/ChooseReport.aspx

Which school uses OConner non-zero?

(and which encourages zeros?)

Zeros or No-zeros ? Stats from 6 EPSB high schools

Notice that the course load is significantly smaller for Eastglen. Likely more students may have jobs because of its socio-economic location. But shouldnt the boost in motivation from the no zero code policy have had students enjoying school more and taking more, not fewer, classes? No wonder it shows that MORE students there are in school for much longer as shown in the Delayed Advancement Rate chart on the following slide. Is that frustrating for students? Is it expensive for taxpayers? (Side issue MacLeans Raising the dropout age wont
work)

Which school uses OConner non-zero?

(and which encourages zeros?)

Zeros or No-zeros ? Stats from 6 EPSB high schools

Perhaps the next slide is the most unsettling of the lot. I concede that one doesnt expect as many students from a lower socio-economic school to get their High School Diploma. But after 5 years of code policy and supposedly facilitating success, note that the next slide shows a significant trend. There is a DROP (red line) in the High School Diploma Completion Rate. (more dropouts with the no-zero policy?)
(Interesting I wonder what Jasper Place (dark purple line) has done to greatly INCREASE their Diploma Completion Rate? I understand they have a modified/moderate zero policy rather than a rigid, extreme no-zero policy.)

Which school uses OConner non-zero?

(and which encourages zeros?)

Zeros or No-zeros ? Stats from 6 EPSB high schools

The next two slides show the % of Diploma Exams failed and Diploma Mark over 5 yrs.

The no-zero schools stats went from 18% failure rate to 33% of the students failing the Diploma Exams. (Province was about 15%-17%)

The school diploma exam averages plummeted. When using the codes, teachers have found that considerably MORE effort goes into tracking down, multiple times, only 5% to 10% of the students that always are absent on exam days, or fail to turn in assignments etc. It has been reported by many teachers that putting in Zeros As Placeholders (ZAP) greatly reduces this task and more time for creativity.
Could the other students be suffering since the teacher is pre-occupied?

Which school uses OConner non-zero?

(and which encourages zeros?)
(diploma)

Which school used OConner non-zero?

(and which encourages zeros?)
(diploma)

Zeros or No-zeros ? Stats from 6 EPSB high schools

The next slide is of the most concern to me. It compares the school-awarded mark with the diploma exam mark (currently weighted 50/50). This is an indicator of totally unjustified mark inflation by the schools. The extreme upward trend is disturbing because it distorts the blended mark. You cannot trust it on its own. Is the use of codes inflating the marks?
(Unfortunately there is talk of reducing the value of the diploma to a 25/75 % blend making it even a more distorted, non-level playing field. Perehaps the diploma should be worth more than now because of rampant inflation of school-awarded marks. That would present a more authentic mark.)

Which school uses OConner non-zero?

(and which encourages zeros?)

Zeros or No-zeros ? Stats from 6 EPSB high schools

The Overall School Rating only compares schools with similar schools with the same average family income and programs. The rigid, strict non-zero / code policy school appears to be losing considerable ground. So who is to say that noticeable drop in ratings are due to a non-zero or a code policy? Maybe yes; maybe no. But something is sure affecting student performance. Inflated, non-authentic marks are unfair to students--to those that DO turn in all their assignments.
(Post-Secondary Institutions really have a problem with this. Your mark shouldnt depend on which school you attend ! Alberta Learning, ATA and school boards have to hold the principals and superintendents accountable and deal with this.)

Which school used OConner non-zero?

(and which encourages zeros?)

Which school used OConner non-zero?

OK so this was only ONE reformed grading policy school. How do I compare others? Clearly something has been driving the school-awarded marks upwards resulting in severe marks inflation. Recently the Calgary Herald offered an interactive database that allows one to list and rank the mark differential for all 200 high schools. I was SHOCKED to see 20%-40% inflation rates. On the next slide is the inflation ranking of Edmonton Catholic and Public Schools for Mr. Dorvals subject (Physics 30). Do your homework and find out which schools have been rigidly adhering to the philosophy. Then looking at the next slide, you can decide for yourself if the marks are inflated for those schools.

Zeros or No zeros? Marks Inflation ?

The Calgary Herald Making the Grade database has an excellent user interface to compare schools in the province. At the top of the page there is this excellent descriptor (see below). Ross Sheppard (including Mr. Dorval) 3.3% discrepancy between the teacher awarded mark and the diploma exam mark indicates that students are being marked consistent with the provincial standards. (Hed been doing
that for 35 years and didnt want to compromise that by ignoring the fact that some

Sadly I present to you my final chart, showing the travesty of marks inflation happening in this province. (Why doesnt Alberta Learning step in and sanction those schools?)
students were not doing all of the work).

Zeros or No-zeros ?

Is there great inflation in the school awarded marks of the schools known to be rigidly adhering to the reformed assessment philosophy? Ross Shep had barely started its policy by June 2011 and teachers were (are?) resisting.

Calgary Herald Database

Zeros or No-zeros? Rumours ?

Some Edmonton high schools have been using the reformed assessment policy for a number of years. Recurring and disturbing rumours include a new initiative for cheaters get to Do-Over (the same exam with scrambled answers) instead of getting zero (Schoolhouse Consulting). (Cheating is in epidemic proportions already and this would not provide a
deterrentthere are no consequences for cheating.)

At a north Edmonton school, students only get marked on attempted questions so if they only bubbled in 20 of 25 multiple choice questions teachers are told to consider that students mark is to be based out of 20 instead of 25! (with this logic a student can choose 5 questions they know
out of 25 on a test, leave the rest blank, and get 100%! (I would request the board and media investigate these allegations)

Zeros or No-zeros ? Possible Solution

Those advocating the reformed grading philosophy often quote Daniel Pink (of TedTalks renown - a business management philosopher often quoted by educators) when it comes to the motivation of students and the move to make marks less important to learning. Pink's basic premise is: research has shown rewards are not the best motivators. Pink suggested a new operating system of management--shown to work-includes three elements: Autonomy , Mastery and Purpose. Autonomy meaning that individuals (students AND teachers) are given the opportunity to solve problems in their own unique way. Mastery refers to the phenomenon that autonomy leads to even better performance, with individuals continually working to improve. Purpose refers to the feeling that a person is working for a cause that really matters on a larger scale.

Zeros or No-zeros ? Possible Solution

Under some successful EPSB administrations in the past, Pinks philosophy has indeed produced some master teachers. Now this is being undone and nullified ironically, one of these master teachers was suspended for not following orders by an administration that claims to be implementing a philosophy which quotes Pink when it comes to motivation !

Zeros or No-zeros ? Possible Solution

Is it possible that heavy-handed management in schools where administrations are making the reformed grading philosophy mandatory is the REAL cause for declining student performance shown by the graphs? Perhaps the moderate administrative style at Strathcona school is what is EMPOWERING their teachers and students to do their best.

Zeros or No-zeros ? Possible Solution

After much controversy over evaluation reform, the Ontario Minister of Education has returned the issue of grading to teachers and their professional judgment. Texas school districts would no longer be able to force teachers to inflate students grades under a 2009 law.

Implement a proven management strategy and give the teachers back their autonomy to use their professional discretion when assessing students

EPSB Board Members TAKE YOUR POWER BACK !!!

On the Surprising Science of Motivation

Daniel Pink TedTalks

Appendix URLs to Linked References

The linked references in the PowerPoint version will not work in this video version. They are listed below
Fraser Institute Alberta Report Card Fraser Institute Comparison Tool
http://alberta.compareschoolrankings.org/ChooseReport.aspx http://alberta.compareschoolrankings.org/pdfs/Fraser_Institute_Report_Card_on_Albertas_High_Schools_2011.pdf

The Calgary Heralds Data Centre Making the Grade

http://www.calgaryherald.com/news/data-centre/school-marks-vs-exams/index.html

http://www.ted.com/talks/dan_pink_on_motivation.html

Daniel Pink TedTalks On the Surprising Science of Motivation

MacLeans - Raising the dropout age wont work

http://www.calgaryherald.com/news/data-centre/school-marks-vs-exams/index.html

Schoolhouse Consulting

http://educhatter.wordpress.com/2011/11/01/do-over-student-evaluation-what-lesson-does-pardoning-cheaters-teach/

Appendix Contact Information

While I would prefer to come out of the closet and identify myself, I do have a great sense of self-preservation. Unlike the brave Mr. Dorval, I have a family to support and still have to teach. I love my subject and sharing it through my interactions with my students. Its supposed to be fun. If left alone to motivate my students as needed--it is fun. I hope that you do not minimize what I am trying to convey because I have remained anonymous. I care deeply about this issue and re-instating the autonomy of the teacher. EMAIL: critically.analyzed@gmail.com

Appendix Case 2
(Which school uses non-zero and which encourages zeros?)

I criticized my own data saying that although there is a great negative correlation between nozero policy and school performance etc. doesnt mean that there is causationbut one certainly cannot ignore the fact that something is going terribly wrong in the strict no-zero schools. Before anyone charges that the data is only for ONE non-zero school that I have singled out (Eastglen), I now have produced the same charts for a SECOND known, strict no-zero school (M.E. LaZerte)
Four comparison charts follow without commentary

Appendix Case 2
(Which school uses non-zero and which encourages zeros?)

Appendix Case 2
(Which school uses non-zero and which encourages zeros?)
(diploma)

Appendix Case 2
(Which school uses non-zero and which encourages zeros?)

Appendix Case 2
(Which school uses non-zero and which encourages zeros?)

Appendix - Sample Report

Appendix - School Report

Appendix - School Report

Appendix - School Report

Appendix - School Report

Appendix - School Report

Appendix - School Report