Вы находитесь на странице: 1из 4

CHAPTER 34+12

MARKING ASSESSMENTS: RATING SCALES AND RUBRICS


GRADING AND STUDENT EVALUATION
Coombe, C., Davidson, P., O'Sullivan, B., & Stoynoff, S. (Eds.) (2012) The Cambridge guide to
second language assessment, Cambridge: Cambridge University Press.
Brown, H. and Abeywickrama, P. (2010) Language assessment –Principles and classroom
practices. New York: Pearson Longman, 2nd edition.
Group 3:
1. Phạm Trần Nhật Huỳnh - huynhptn.188t@ou.edu.vn
2. Phan Thị Huê - huept.188t@ou.edu.vn
3. Nguyễn Hữu Cường - cuongnh.188t@ou.edu.vn
4. Trần Minh Hà - hatm.188t@ou.edu.vn
SUMMARY
1. Key Chapter 34: Marking Assessments: Rating Scales and Rubrics
concepts  Rubrics are a set of guidelines that explain the criteria by which performance will be judged
and or rated (may include a rating scale). Rubrics outline performance standards.
meanings  Analytic – individual aspect of the task is judged and used to determine overall score
 Holistic – the performance or product is judged as a whole, compared to models or exemplars.
Rubrics typically provide a description of how the rater should determine the quality of various
performances at specific levels.
 Rating Scales: They provide a convenient recording method, common frame of reference,
and focus the raters’ attention on specific important aspects of the performance.
Used for (limited to) make quality judgments. Requires additional information regarding performance
expectations. These take many forms (numerical and descriptive) but are used to provide a uniform
way to score performances along a continuum (at least ordinal, preferably interval).
Chapter 12: Grading and student evaluation
 Grading criteria : Establish grading criteria can make the process of grading more efficient
and consistent. Providing these criteria to students can help students perform better and prevent
confusion or frustration about their grades. The syllabus is a useful location to include the
information.
2. Main - Chapter 34: Marking Assessments: Rating Scales and Rubrics
points + All scoring systems require careful management to minimize mistakes.
+ The open-ended questions make them more challenging to score the test fairly.
 Rating Scales
+ Make a qualitative judgment about the extent to which a behavior is presented consisting of a set of
characteristics or qualities to be judged by using a systematic procedure numerical and graphic rating
scales are used most frequently
 Types of Rating Scales Numerical Rating Scales:
+ A sequence of numbers is assigned to descriptive categories; the rater marks a number to indicate
the degree to which a characteristic is present.
 Graphic Rating Scales: a set of categories described at certain points alongthe line of a
continuum; the rater can mark his or her judgment at any location on the line.
 Advantages of Using Rating Scales
+ Used for behaviors not easily measured by other means. Quick and easy to complete. User can apply
knowledge about the child from other times. Minimum of training required. Easy to design using
consistent descriptors (e.g., always, sometimes, rarely, or never). Can describe the child’s steps
toward understanding or mastery.
 Disadvantages of Using Rating Scales: Reliability
+ Highly subjective (rater error and bias are a common problem). Raters may rate a child on the basis
of their previous interactions or on an emotional, rather than an objective, basis ambiguous terms
make them unreliable: raters are likely to mark characteristics by using different interpretations of the
ratings (e.g., do they all agree on what “sometimes” means?).
 Developing Quality Rating Scales
1. Identify the learning outcomes that the task is intended to assess.
2. Determine what characteristics of the learning outcomes are most significant that are directly
observable; and scale clearly define points on the scale.
3. Select the scale that is most appropriate for the purposes of the assessment.
4. Provide between three and seven ratings to the scale (the number of points will depend on how
many levels of accomplishment are needed).
 Rubrics
+ Qualitative instruments used for assessing student progress in development and learning, or scoring
student work. Provide guidelines and descriptors to distinguish performance from one level to
another. Criteria for scoring or indicators of performance also describe dimensions of performance.
 Holistic Rubrics:
+ Assign a single score to a student’s overall performance. Usually have competency labels that define
the level of performance. The quality of work or performance at each level described by a number of
indicators.
 Analytic Rubric:
+ Describes and scores each of the task attributes separately. Uses limited descriptors for each
attribute. Uses a narrow and broad scale. Allows for specific diagnostic feedback.
 Advantages of Using Rubrics
+ Provide guidelines for quality student work or performance. Flexible; can be designed for many
uses and ability levels. Easily modified: Can be used by both teacher and student to guide the student’s
efforts prior to completion of a task. Can be translated into grades and can be used to discuss with
parents and students.
 Disadvantages of Using Rubrics
+ Some difficulty in determining scoring criteria. Teachers may focus on excessively general or
inappropriate scoring criteria. Teachers might use predetermined criteria, rather than basing scores on
examples of student work. Teachers might inappropriately focus on the quantity of characteristics,
rather than the indicators of quality work. Holistic rubrics might lack validity and reliability.
 Developing Quality Rubrics
1. Do descriptions focus on important aspects of the performance?
2. Does the rating match the purpose?
3. Are the traits directly observable?
4. Are the criteria understandable?
5. Are the traits clearly defined?
6. Is scoring error minimized?
7. Is the scoring system feasible?
- Chapter 12: Grading and student evaluation
+ Grading and Student Evaluation : Challenges, Choices, and Consequences
+ Grades Scores & Grades vs. Ability & Social meaning? Reliability vs. Validity? Tests vs.
Alternatives in assessment Philosophy of grading: What to include? (See and do p. 320) Importance
of triangulation (using multiple measures)
+ Sound assessment and grading practice : help teachers to improve their instruction, improve students
motivation to learn, and increase students level of achievement. Brookhart, 1999)
 Roles of grading (Walvoord, Anderson, 1998)
1. Evaluation: the grade claims to be a valid, fair and trustworthy judgment about the quality of the
students work
2. Communication: the grade communicates the teachers judgment of the students work
3. Motivation: because it affects the time and effort students spend, grading is a powerful part of the
motivational structure of a course (for better and for worse)
4. Organization: a grade helps mark transitions, bring closure and focus efforts for both teachers and
students .
+ The teachers job is to render an informed and professional judgment to the best of their ability.
+ Teachers need to : establish clear and thoughtful criteria and standards to student’s work, exercise
that professional judgment within the context of their institution, department and the students they
deal with.
 Grading and teachers roles : Grades matter greatly for teachers who often experience the
tension of performing two conflicting roles :
1. Coaches - instructing, guiding student writing
2. Judges - evaluating students work
+ Teachers themselves are often judged by the grades they give.
 Guidelines for selecting grading criteria :
1. Be consistent with institutional philosophy & regulations
2. Provide grading criteria and weights of each component
3. Recognize own subjectivity and do not change rubrics or policies at the last minute
4. Focus on main elements for learning or achievements
 Calculating Grades points :
+ Percentile Absolute vs. relative grading
Posteriori relative grading: adjust grade distributions after performances to complement levels of
difficulty or your own philosophy of grading (e.g., content, mastery) Refrain from ‘massaging’ or
‘bend’ grades
 Toward Appropriate Grading Teacher perception:
+ Bias toward ‘good’ class & Grade inflation Student perception Working with/against institutional
expectations & constraints: admission & grading policies on theoretical/research basis vs. practicality
or hunches/unwritten assumptions
 Toward Appropriate Grading Cross-cultural factors & Issue of difficulty -be mindful
cultural variables and ‘work around’ them for fair assessment (see p. 329-330); issue of (c)overt
cheating (recycling of old exams, etc.) -Construct and reflect your own educational philosophy for
better and fair practice in class! -Consider factors for tests not piloted
 Toward Appropriate Grading -Consider factors for tests not piloted: own experience with
intuition, ability to design feasible tasks, framing clear and relevant items, mirroring in-class tasks,
variations of test types, referring to prior tests, reviewing and preparing for the test thoroughly,
understanding your students’ abilities, and, of not, mostly luck!!!
 Toward Appropriate Grading Meaning of grade -descriptors of letter grades -issue of
holistic assessment by grades? -be aware of meaninglessness of letter grades and USE 1) a carefully
constructed system of grading, explicitly stated criteria, the criteria established based on course
objectives and assessment procedure shared with students!
 Toward Appropriate Grading Calculating grades: consider web tools (see p. 332) Use or
complement with alternatives to letter grading for both formative & summative assessment (See p.
333) Use summative alternatives -Self-/peer-assessment, narrative evaluations, checklist evaluations,
Conferences (see p. 334-337)
 Some principles & guidelines for grading & evaluation understand that grading -is not
universal, subjective & context- dependent, and done on a ‘curve’ -reflects your & institutional
philosophy(power?), cross-cultural variations, and your expectations rather than students’ real
abilities. Testing does not reflect the level of difficulty intended unless carefully designed!
 Some principles & guidelines for grading & evaluation Grades from the test are different
in meanings or interpretations! Use alternatives to grades for additional indicators of achievement
Guidelines of grading & evaluation (See p. 337) Focus more on LEARNING, not TESTING!!!
 Grading schemes for a course
Grading schemes - the series of assessment tools (exams, tasks, projects, etc.) that are scored and used
to arrive at a final grade for students.
(Brown & Abeywickrama, 2010) Standards for assigning grades are extraordinarily variable across
teachers, subject matter, courses, programs, school systems, and even cultures
3. 1. In your opinion, how to reduce the issues that make the results of impression scoring unreliable?
Group’s (page 300, Coombe et al)
questions 2. What purpose do grades serve?
3. How to make grading more efficient?
4. What is the trouble with evaluation of students?