Assessing Writing

91.12.04 by Cristina, Crystal, David, Rachel,

Roy I. Principles for assessing writing II. Existing writing assessment methods III. Comparison of traditional and alternative assessment IV. Sample elicitation and evaluation V. References

Principles

Reliability Give writing assignments before real tests Avoid unclear or too difficult instructions Give instructions as specific as possible Validity Testing only writing ability, and nothing else Set as many tasks as is feasible Ensure samples long enough Practicality--ensure ease of access (online writing) Positive washback--give authentic writing tasks Scoring Holistic scoring Analytic scoring

Holistic Scoring
NS NSNative speaker standard Close to native speaker standard

MA
MAA

Clearly more than adequate

Possibly more than adequate Adequate for study at this university

D
NA FBA

Doubtful
Clearly not adequate Far below adequacy

Existing writing assessment methods

Traditional
1.Grammar and structure 2.Control Writing 3.Free Writing

Alternatives
1.Diary () 2.Online discussion board 3.Dictocomps 4.Think-aloud writing 5.Portfolio 6.Peer evaluation

Traditional
I. Grammar

and structure

a. Multiple-choice items b. Error-recognition items c. Re-arrangement d. Changing words e. Blank-filling

Error-recognition
a. choose the underlined words or phrases which are incorrect. e.g. 1. It was a terrible accident at an air show in west a b c d Germany yesterday. 2. One of the aircraft was crashed into two other a b c aircraft during the display. etc d

Error-recognition
b.students are told that there is a mistake in grammar in each sentence and instructed to write the letter of that part of the sentence in which it occurs. e.g. 1. It was/ a terrible accident/ at an air show / a b c yesterday. d 2. One of the aircraft/ was crashed into/ two other a b c aircraft /during the display. etc d

Re-arrangement

e.g. 1.She was wearing a . new/ jacket/ leather/ red / lovely

2.Susan the phone/ when already/ rang / suddenly/ finished/ had

Changing words

e.g. I said if I (1) to discover the fire earlier,I (2) to succeed in (3)to extinguish it before it (4)to catch hold. etc.

(1)________________ (2)________________ (3)________________ (4)________________

Traditional
I. Grammar

and structure

a. Multiple-choice items b. Error-recognition items c. Re-arrangement d. Changing words e. Blank-filling

Traditional
II. a. b. c. d. e. f.

Control Writing
Transformation Broken sentences Sentence and paragraph completion Notes Summary writing Chinese-English translation

Transformation Suitable for intermediate and advanced tests than elementary tests. a. re-writing sentences according to certain patterns e.g. 1.No other living plant grows so tall and fast as bamboo. Bamboo ______________________. 2.A stalk at Japan s commonest bamboo has grown over a meter in only 24 hours. It has taken_____________________. etc.

Transformation b. giving them a word in brackets and use the word to rewrite the sentence. 1. It is necessary to answer all the questions.(need) _______________________________________ 2. His angry out burst was completely unexpected.(took) _______________________________________

**weakness: lack of context.

Broken sentences
Dear Cindy, Thanks/ lot/ your letter/ which/ arrive/ yesterday. 1._________________________________________________ Congratulations/ pass/ your entrance exams. 2._________________________________________________ You/ work/ very/ hard/ past/year/ but/now/ it/ all/be worthwhile 3._________________________________________________ I wonder/ if / you/ like/ stay/ me/ few days. 4._________________________________________________ I look forward/ hear/you/ very soon. 5._________________________________________________ Best wishes Cathy.

Sentence and paragraph completion

a.

Most of the students in my class were rather lazy and did not enjoy the course. Pauline, however,___________ __________________ __________________ __________________ __________________

b. A:__________________ B:So do I . I generally watch for an hour or two every morning. A:__________________ B: Immediately after Ive finished my homework about eight.

Traditional
II. a. b. c. d. e. f.

Control Writing
Transformation Broken sentences Sentence and paragraph completion Notes Summary writing Chinese-English translation

Free Writing

Using Picture for Writing Essay Writing Journal

Choosing subjects : (Avoid very general abstract topic) Realistic writing task : (Think about the likely real life contexts in which your students ) Writing for purpose Specific instruction Clear, simple, and realistic

Using Picture for Writing

You may want to use pictures to do any of the following tasks in your test: 1. to describe a scene, object or person 2. to compare two scenes, etc. 3. to tell a story 4. to give instructions or directions 5. to describe a process Notes: 1. Cultural difficulty 2. AGE

Essay and Journal Writing

1. The error count method 2. The analytic method 3. The impression (or multiple-marking) method

The Error Count Method

The error count method involves counting up the number of errors which a student has madeand deducting this number from a possible maximum total Flaws: 1. Completely ignores content 1. fails to take into account that some errors are more serious than another

The Analytic Method

5 Fluency Grammar Vocabulary Content

O O

Spelling

Comp 1 Marker A 2

Comp 2 1

Comp 3 4

Comp 4 2

Marker B
Marker C Total

3
3 8

2
1 4

5
3 12

4
1 7

Alternative
Portfolio: Dictocom Discussion board Think aloud writing Peer Evaluation Diary

Monday

I have an exam today. I did not study

Tuesday

Go swimming!

Wednesday The exam result was better than I thought! I guess Im just lucky Thursday Friday Saturday Sunday Visit Uncle Ronny I went to the see Harry Potter with my GF

Comparison

Strengths

Weaknesses

Traditional

1.With many different kinds of tasks 1. Possibility of negative 2.Students practice a lot of similar tasks washback and exercise before the real test 3.The objective of the assessment is easy to be focused on 4.Both holistic scoring and analytic scoring can apply to traditional assessment

1.The stress on individual difference and creativity can stimulate students

washback more
Alternative

1.Time consuming 2.Lack of validity (ex. Dictocomps) 3.Lack of practicality (ex.Online discussion) 4. Difficulty in scoring

Comparison of Two Assessments

Traditional

Alternative

Set as many tasks as is feasible

Ensure samples long enough Give writing assignments before real tests Avoid unclear or too difficult instructions Give instructions as specific as possible (restrict test-takers)

Test only writing ability, and nothing else Practicalityensure ease of access (online writing)
Positive washback

Holistic scoring
Analytic scoring

Sample 1

--Translation 20

Sample 1

1. You need not wonder any longer. A hotel in Japan will give you a preview. The name of this hotel is the Capsule Inn; its rooms rent for eleven dollars a night. 2. 3. But here any resemblance to a twentieth-century hotel ends. The rooms are small plastic capsules, and each capsule is about five feet high, by five feet wide, and by seven feet deep. Guests have to crawl into bed through a large porthole entrance. 4. 5. perhaps because the price of the rooms is as small as the rooms themselves.

Strength: A. Reliability Students can practice many similar tasks before the real test B. Validity Students know what to write from Chinese sentences Weakness: Validity Not only writing process but also concept transference are involved in translation tasks

Sample

Portfolio in assessing writing

Portfolio assessment is a cumulative and ongoing collection of entries that are selected and commented on by the students, the teacher and/or peers, to assess the students progress in the development of a competency

1. Positive Washback
A. Teachers' individual teaching B. Students individual improvement (1) Writing to learn (2) Writing for self-realization

2. Reliability 3. Validitypossibility of lacking validity 4. Scoringdifficulty in scoring 5. Time consuming

References
Brown, H. Douglas. Teaching by Principles: An Interactive Approach to Language Pedagogy. 2nd ed. New York: Longman, 2001. Harris, David P. Testing English as a Second Language. USA: McGraw-Hill, 1969. Heaton, J. B. Classroom Testing. New York: Longman, 1990. Hughes, Arthur. Testing for Language Teachers. New York: Cambridge, 1989.