Discussion Summary

About the student writing sample

As shown in Appendix A, the writing sample is written for a college level L2 course:

Writing for Proficiency.​ Given 50 minutes, the student instruction was to read the article by

Terry Lee Goodrich, ​Lies are so commonplace, they almost seem like the truth.​ Afterword they

are to write an essay to the following prompt: Is lying always wrong? Why or why not? In

addition, they are to consider both sides of the issue and reference the article.

As shown in Appendix A and B is Becca’s scoring using the holistic rubric with a score

of 76%. In Appendix C and D is Becca’s scoring using the analytic rubric with a score of 73%.

As shown in Appendix E and F is Mariah’s scoring using the holistic rubric with a C+. In

Appendix G and H is Mariah’s scoring using the analytic rubric with a score of 81%. Mariah’s

grade is higher than Becca’s grade. Casanave (2017) explains that it is unnatural for any two

raters to agree about the quality and grading in writing, therefore raters should attempt to come

to a consensus (pp. 230).

Discussion Holistic Essay Scoring

If you disagree, arrive at a consensus evaluation

Mariah mentioned that “Teachers may fear making grading because judgments may be

bias, unfair and possibly damage student writers” which is part of why she graded so high

(Crusan, 2010; Glenn, et al., 2007 as cited in Ferris & Hedgcock, 2014). We agree that we do

not want to discourage the writer by giving an unfair grade. We agree that the student had

enough rhetorical features in the writing that we understood what the student was trying to

convey. Our final grading decision is a C+.

Suggestions for using holistic scoring rubrics

While using a holistic rubric, an instructor should be open to using a (+) or (-) instead of

just whole grades, in addition to the percentage grade, to more accurately access students’

writing. Highlighting either the problems that led to their grade and/or identify what would

make their paper better is a necessary process while using a holistic rubric. This should be done

in the students’ writing as well as on the rubric itself. Also, we agreed on the type of holistic

rubric that we used to grade, shown in Appendix B. Ferris & Hedgcock (2014) explain that there

are different types of holistic rubrics, some are for expository text and others are for advanced

writing courses such as our student sample shown in Appendix A.

Discussion Analytic Essay Scoring

If you disagree, arrive at a consensus evaluation

Despite the fact that we graded each skill through our own lens for each one we came up

with the same grade level in a couple of skills. After our discussion we agreed on these scores:

Content - 21, Rhetorical skills – 17, Grammatical form – 15 as we agree that while there were

errors in some sentences, it did not obstruct the message, Dictation and tone – 15 because we

both consider the student’s message came across well, and for mechanics, Mariah would agree

with Rebecca's grade of 8 if it was typed, but we will assume it was handwritten and give it a 10.

This gives us a total score of a 78%, this brought Mariah’s grading down 3 points and Rebecca’s

up 5 points.

Suggestions for using analytic scoring rubrics

While using an analytic rubric the instructor should consider if it is timed, typed or not,

and if the students have familiarity with the topic. We are not sure of the specific context of the
class and because this could affect how the rater would score (beginner, intermediate or

advanced ESL class); The rater would need to know exactly what was covered in order to use the

analytic rubric effectively. A rater may grade more or less harshly depending on how much they

know of the context of the class. The use of the analytic rubric for both the beginning and end of

the semester assignments would help the student focus on each skill according to the rubric as

well as help the student and the teacher see any progress made.

Summary of Discussion

What do you think is the cause of the similarities and differences?

Interrater reliability is how closely the assigned scores of two or more raters agree

(Crusan, 2010; Fulcher, 2010; Weigle, 2002, as cited in Ferris & Hedgcock, 2014). We used the

numeric grade conversion scale, shown in Appendix I, to identify our grading. Our grading

ranged from a 73% to a 83% and were at the same point on the grade scale. According to Ferris

and Hedgcock (2014), when ratings are the same or differ by only a single point of a five-point

scale, they are thought to agree (pp.208).

According to Crusan (2010), Rebecca would be a left-brain assessor because her focus is

on mechanical aspects of writing and emphasize the logical continuity of thought. On the other

hand, according to Crusan (2010), Mariah would be a right-brain assessor because she places a

high value on creative processes and the use of novel ideas and imagery. Rebecca’s grading

focused more on the mechanical elements in the writing and Mariah’s grading focused on how

well the message was conveyed. Rebecca’s assessor’s personality trait can be described as a

prescriptivist, while Mariah is a descriptivist assessor.

 Mariah took into consideration that the assignment only allowed 50 minutes to complete

in class. We don't know if they had access to a dictionary nor if it was to be typed or

handwritten. Mariah says if it was typed, then we should expect no spelling issues at the least

and that is why she was a little bit more lenient on the mechanics portion in the analytic rubric.

​While using the rubric she had to remind herself of these points. We both agree that the

conclusion was short and the student should expand the writing on the things that were listed in

the introductory paragraph, such as what kind of party. In addition, we would have liked to have

seen more written about why lying is not harmful. We both debated about punctuation issues.

In the conclusion of our conversation, we discussed that there may be an issue where

Rebecca gave back too much written information in the rubric. This may lead the student to not

be able to understand what was written (message conveyed and handwriting) due to using

high-level vocabulary, such as rhetorical and syntax. Mariah may have done the opposite by not

providing enough feedback. We both struggled with how much or little to write in the feedback.
 References

Casanave, C. P. (2017). ​Controversies in second language writing: Dilemmas and decisions in

research and instruction ​(2nd ed). University of Michigan Press.

Crusan, D. (2010). Assessment in the second language writing classroom. ​Assessment in the

second language writing classroom (pp.87-114).​ University of Michigan Press.

Ferris, R., D. & Hedgcock, S., J. (2014).Classroom Assessment of L2 Writing. ​Teaching ESL

composition: Purpose, process, and practice​ (3​rd​ ed, pp.196-236). Routledge.

 Appendix A

Attempt using a Holistic Rubric

 Appendix B

Holistic Rubric
 Appendix C

Scoring Attempt using an Analytic Rubric

 Appendix D

Analytic Rubric
 Appendix E

Scoring Attempt using a Holistic Rubric

 Appendix F

Holistic Rubric
 Appendix G

Scoring Attempt using an Analytic Rubric

 Appendix H

Analytic Rubric
 Appendix I

Numeric Grade Conversion Scale