Thesis of MA Linguistics

TESTING
ENGLISH
GRAMMAR
AT
INTERMEDIATE LEVEL AND ITS EFFECT ON TEACHING AND LEARNING.
By: PIR SUHAIL AHMED SARHANDI
THESIS
Presented to the department of English university of Karachi In partial fulfillment of the requirement For the degree of Masters in Linguistics
UNIVERSITY OF KARACHI JULY 2005
This document was created by the trial version of Print2PDF. ! Once Print2PDF is registered, this message will disappear.! Purchase Print2PDF at http://www.software602.com
ABSTRACT
This research study is conducted in order to know how testing of English grammar at Intermediate level effects the teaching and learning process. After discussing the test and its types thoroughly in section two, I think I have provided the readers a well understanding of testing and our testing system especially grammar testing at Intermediate level. The question grammatical items in papers (I & II) of English compulsory at Intermediate level were evaluated and analyzed with their impact on learning and teaching. The results show that the testing system of grammar is not so effective because they are devised for testing not for learning that lacks their validity. And at last I have proposed sample items that are devised by me on the communicative approach of testing grammar. The sample test items, if introduced, in the testing system of grammar at Intermediate level would help to achieve the beneficial backwash effect on teaching and learning.
AKNOWLEDGEMENT I would like to thank Mr. Muhammad Ali Khan, my research supervisor, to the core of my heart who has supported me and encouraged me throughout my research work. Because of his kind supervision and beneficial suggestions, I became able to complete my thesis. I am also thankful to my grammar teachers: Ms.Dilshat Bano, Mr.Ghulam Hussain Manganhar and Mr.Ahsan Ahmed Alavi, who taught me grammar in such a way that paved me the way to work on my research on grammar testing. I am also grateful to my teacher as well as a sincere friend Mr.Zubair Ahmed Mangrio for his well wishes and precious suggestions throughout my linguistics program. At last but not least, I am extremely grateful and obliged to my family members especially to my father, Pir Khalil Ahmed Sarhandi who always encourages me throughout my education.
CONTENTS
ABSTRACT ACKNOWLEDGMENTS Page No. SECTION ONE: Introduction to Research 1.1 1.2 1.3 1.4 1.5 Introduction to My Research Proposed Research Methodology My Research Methodology Purpose of My research Literature Review 1 1 3 4 4 5
SECTION TWO: Introduction to Testing 2.1 2.2 An overview of Testing What is a good Test?
8 8 8
2.3 2.4 2.5 2.6 2.7
Why do we need to Test? Why do we need to Test Grammar? Approaches to Language Testing Types and purposes of Tests Test Classification (a) Kinds of Testing
9 11 11 14 17 19 22 23 27 27 28
2.8 2.9
Subjective and Objective Testing Criteria for the evaluation of a Test
SECTION THREE: Backwash effect 3.1 3.2 Backwash Effect on Teaching and Learning Harmful backwash effect
SECTION FOUR:
Evaluation of English Compulsory Papers (I & II) and some suggestions.
29 29 38 39
4.1 4.2 4.3
Evaluation of Grammatical items. Advantages of Proposed Grammatical items Conclusion
REFERENCES
40
BIBLIOGRAPHY APPENDICES
41
SECTION ONE: Introduction to Research
1.1 Introduction to my Research
The thing that encouraged me to work on the Testing Grammar at Intermediate Level was the inability of our students who after being in touch with English since sixth standard or even theres a good number of those learners who have been taught English since their Montessori class. The reason to work on only this topic was to know what is lacking that our students are still unable to communicate in English. After pondering over and discussing this problem with my friends and teachers I came to know that our testing system is no so effective to produce the communicative competence in the learners.
According to Arthur Hughes:
Too often language tests have harmful effect on teaching and learning; and
too often they fail to measure accurately whatever it is they are intended to measure.
(2003:1)
One of the major problems, faced in English language teaching and learning is the inappropriate administered tests at the end of the course. In our education system, we have not been taught the language for the sake learning but for testing.
We are now in a communicative era, a time when we emphasize evaluation of language use rather than language form. I have tried my level best to expose those reasons that are in the way to communicative learning.
The reason to choose the topic on grammar testing is that in our society there is a n impression that grammar has no concern in communicative language so it should not be taught and tested. This idea has overwhelmed our society. I think the major reason behind this idea is ineffective teaching and learning system in our society.
.testing has been one of the greatest single beneficial forces in changing the direction of the greatest single beneficial forces in changing the direction of language in many areas and in encouraging the more responsive teachers to examine not only their own teaching methods but also the language they are teaching.
Heaton (1988: 171)
The above words of Heaton inspired me to work on a research that deals with testing in order to bring a change in the way grammar items are taught and learnt. Teachers dont teach the grammatical items according to their function and use in real life communication so they dont test these items accordingly.
Widdowson describes the importance of grammar in these words:
A communicative approach, properly conceived, doesnt involve a recognition

of its central mediating role in the use and learning of language.
(1990:88)
Little-Wood (1986) while stressing the importance of grammar writes:
Communicative language use is only possible by virtue of the grammatical

system and its creative potential. ( Little Wood 1986 quoted in Lock, 1996:66)
1.2 Proposed Research Methodology
Methodology is a general approach studying a research topic. It establishes how one will go about studying any phenomenon. In social research, examples of methodologies are positivism (which seeks to discover laws using quantitative methods) and, of course, qualitative methodology (which is often concerned with including hypotheses from field research). Like theories, methodologies cannot be true or false, only more or less useful. (Interpreting Qualitative Data by David Silverman 1993)
Definition
Qualitative is the collection and analysis of extensive data in order to gain insight into a situation of interest. It involves intensive data collection that is the collection of extensive data on many variables over an extended period of time in a naturalistic setting. It is much broader in scope.
Qualitative research assumes that all knowledge is relative, that there is a

subjective element to all knowledge and research, and that holistic ungeneralisable studies are justifiable (an ungeneralisable study is one in which the insights and outcomes generated by the research cannot be applied to contexts or situations beyond those in which the data were collected.
Reinhardt and Cook (1979)
1.3 My Research Methodology
My methodology for conducting this research is Qualitative because I think the best to do the research on my topic Testing English Grammar at Intermediate Level and its impact on Teaching and Learning is to evaluate and observe the data extensively in informal way.
For this research study I have selected the Intermediate level in which I will analyze and evaluate the papers (I & II) and the impact of testing grammatical items on teaching and learning. The purpose of choosing the Intermediate level for my research work is that, in my opinion, these two years of study (first year and Inter) are the foundation stones for a learners coming up professional life. As English is the utmost need for a bright future in our country and the major problem in learning this language is mostly caused by grammar so I have decided to evaluate the grammatical items and try to give some effective suggestions to improve testing system of grammar which would automatically change the teaching system as well.
For this study only those grammatical items are chosen which are found constantly in the Intermediate papers of English for some years. These items include: articles, preposition, English tense system, idioms, phrases and translation.
1.4 Purpose of my Research: The main purpose of my research work is to discuss the possible implications testing grammar may have for the teaching and learning of English grammar at Intermediate Level. In Intermediate examinations, grammar is tested in isolated sentences, which are not connected to each other. They are mechanical in a sense that the learners can do them without really understanding the meaning of the sentences.
J.B.Heaton defines the purpose of tests of grammar and usage that
these tests measure students ability to recognize appropriate grammatical

forms and to manipulate structures. (1988:9)
The testers of grammar at Intermediate level here are quite contrary to the purpose of these tests defined above. It is my fundamental aim to analyze the Intermediate grammar papers in the light of the above definition and derive the attention of teachers to the usage of grammar in English language.
My research study would help the testers to set the effective grammatical items so that the testees focus would remain on the practical usage of grammar.
1.5 Literature Review Testing has been a burning question since long different writers have different opinions about its purpose.
John Bright Heaton defines the purpose of testing as
devices to reinforce learning and to motivate students or primarily as a means

of assessing.
(1988:7)
In his book Writing English Language Tests (1988), J.B.Heaton has given a brief introduction to testing. He discusses almost all the language skills to be tested in English language. The book analyses the methods and techniques used in assessing the English tests. Criteria and types are also given importance in the book. Heaton has also provided proposals for the better testing in all skills.
Harold.S.Madsen in his book Techniques in Testing has noticed the other purpose of testing and says: Good tests can sustain or enhance class morale and aid learning. (1983:4)
In his book he gives a thorough introduction to testing and its types very clearly explained with numerous examples. The book contains a good number of activities and techniques that help the teachers to focus on teaching and evaluating in the class.
The newly re-edited book Testing for Language Teachers by Arthur Hughes (2003) may be said to be the most current publication on Testing.
In his book Hughes analyses and differentiates what makes a good test. This book considers the different purposes of testing, discusses how testing encourages good and effective teaching and learning.
In my opinion, testing cannot be fully assessed without focusing on the teaching process. Both, testing and teaching are interrelated fields, which cannot be separated from each other.
Heaton defines the purpose of tests of grammar and usage that
these tests measure students ability to recognize appropriate grammatical

forms and manipulate structures.
(1988:9)
Properly made English tests can help create positive attitude toward instruction by giving students a sense of accomplishment and a feeling that the teachers evaluation of them matches what s/he has taught them. Heaton
(1988:4)
In the light of the above statements I would analyze and observe the impact of Testing English grammar at Intermediate level on teaching and learning.
SECTION TWO: AN OVERVIEW OF TESTING
2.1 What is Testing?
Testing is not an easily definable term. Since long, it has been a burning question if Teaching and Testing are two different fields. Its obvious that we cannot separate these terms from each other. Both are interrelated fields for without focusing on teaching process or methods we cannot think of proper or better assessment.
Testing is a multifaceted process so one cannot stick with its only one definition. WE have to go through many definitions given by different scholars in order to grasp this term. Carroll (1968) defined the term test, cited in Bachman (1990:20) as;
A psychologist or educational test is a procedure designed to elicit certain behavior from which one can make inferences about certain characteristics of an individual. (Carroll 1968:46)
2.2 What is a good Test?
Hughes has given the characteristics of a good test, which facilitates our understanding of the nature of the test. They can be enumerated as followed:
1. It should attempt to measure students actual ability to communicate in language, pointing out that a good test is never a mechanical test only. 2. It should have a beneficial influence on syllabuses and teaching strategies. In other words, the test should enable the teacher to locate which part of the course has been found difficult and problematic by the class. In this way the teacher can evaluate the effectiveness of the course as well as his teaching strategies. (Arthur Hughes 2003: 8) Harold.S.Madsen says:
Good tests can sustain or enhance class morale and aid learning.
(1983:04)
Considering the above points, I will measure the efficacy of Intermediate Test papers. There is no denying the fact that teaching and testing are related and this is best explained by the term BACKWASH EFFECT. This term is further defined in details in Section Three. Things start becoming clear when we identify different purposes and types of tests.
2.3 Why do we need to test?
Testing as already discussed is a great source of teaching and learning especially because of its backwash effect. It is a tool which encourages the teachers and up to date them with their teaching process. For students it acts as an achievement of their struggle while learning.
According to Madsen:
Properly made tests can help create positive attitude towards instructions by giving students a sense of accomplishment and a feeling that the teachers evaluation of them matches what he has taught them. Good English tests also can help the students learn the language by requiring them to study hard, emphasizing course objectives and showing them where they need to improve. (1983:5)
John Bright Heaton defines the purposes of testing as:
devices to reinforce learning and to motivate students or primarily as a means of assessing students performance in the language. (1988: 06)
It is essential to give a list of the aims and purpose of the tests before conducting the test.
Language testing both serves and is served by research in language acquisition and in language teaching. Language tests, for example, are frequently used as criterion measures of language abilities in second language acquisition research. Similarly language tests can be valuable sources of information about the effectiveness of learning and teaching.
Bachman (1990: 2- 3)
According to Hughes: The first thing that testers have to be clear about is the purpose of testing in
any particular situation. Different purposes will usually require different kinds of tests. This may seem obvious but it is something which seems not always be recognized.
(Hughes 2003:7)
2.4 Why do we need to test grammar?
These tests measure students ability to recognize appropriate grammatical

forms and to manipulate structures:
says Heaton (1988:9)
Arther Hughes (1989) while discussing the testing of grammatical items he says that if it is language skill which is to be tested then they should be tested directly instead of the abilities that seem to underlie them. Firstly because by underlying skills one cannot accurately predict the mastery of a skill and secondly, the backwash effect of direct testing would be preferable to that of indirect testing.
As it is essential to have a good command of particular grammatical structures to communicate, thats why its also necessary top test these grammatical items.
2.5 Approaches to Language Testing
Language tests may be classified according to four main approaches to testing:
a. b. c. d.
The essay-translation approach; The structuralist approach; The integrative approach; The communicative approach.
A useful test will generally have features of several of these approaches. Actually, a test may have certain innate weaknesses simply because it is limited to one approach, however attractive that approach may appear.
a) The Essay-translation approach:
This approach is commonly referred to as the pre-scientific stage of language testing. No special skill in testing is required. Tests usually consist of essay writing, translation, and grammatical analysis.
b) The Structuralist approach:
This approach is based on the view that language learning is chiefly concerned with the systematic acquisition of a set of habits. All the four skills are separated from one another as much possible because it is considered essential to test one thing at a time.
c) The Integrative approach:
This approach involves the testing of language in context and is thus concerned primarily with meaning and the total communicative effect of discourse. This approach of testing tests the language skills integratedly. The language is tested in context, chiefly
concerned. Integrative tests do not seek to separate language skills into neat divisions in order to improve test reliability. Oral interviews, translation and essay writing are also included in many integrative tests.
d) The Communicative approach:
According to Heaton
This approach is sometimes linked to the integrative approach. However, both

approaches emphasise the importance of the meaning of the utterances rather than their form and structure, there are nevertheless fundamental differences between the two approaches. Communicative tests are concerned primarily (if not totally) with how language is used in communication.
(1988:19)
This approach focuses on the function and the meaning of the utterances rather than their form and function.
B.J.Caroll says: The ultimate criterion of language mastery is therefore the teachers
effectiveness in communication for the setting he finds himself in.
Defining in a single sentence I say: Communicative approach is mainly concerned with how language is used in real life communication.
2.6 Kinds of tests and their purposes.
Lyle F. Bachman has suggested five features to classify tests
.language tests can be classified according to five distinctive features: the purpose, or use, for which they are intended; the content upon which they are based; the frame of reference within which their result are to interpreted; the way in which they are scored, and the specific technique or method they employ.
Bachman (1990:70)
There is some confusion regarding the terminology used to denote the different types of language tests in use. Most test specialists, however, agree on the following types of test divisions:
a)
Achievement/ attainment tests,
b)
Proficiency tests,
c)
Aptitude tests, and
d)
Diagnostic tests.
a) Achievement tests:
This group can be further subdivided into progress achievement tests and (standardized) achievement tests.
According to Hughes:
Progress achievement tests, as their name suggests, are intended to measure

the progress that students are making.
(2003:14)
(Standardised) Final achievement test are administered at the end of the course indicating the extent to which an individual has mastered in the specific skills.
Achievement tests are directly related to language courses, their purpose being
to establish how successful individual, groups of students or the courses themselves have been in achieving objectives.
Hughes (2003:13)
b) Proficiency tests:
Madsen defines it as:
Proficiency tests can measure overall mastery of English or how well prepared
one is to use English in a particular setting.
(1983:9)
The proficiency test is concerned simply with measuring a students control of the language in the light of what s/he will be expected to do with it in future performance of a particular task. According to Arthur Hughes:
Proficiency tests are designed to measure peoples ability in a language, regardless of any training they may have had in that language.
(2003:11)
c) Aptitude tests: Heaton defines it as: A language aptitude test is designed to measure the students probable
performance in a foreign language, which s/he has not started to learn: i.e. it assesses aptitude for learning a language.
(1988:173)
Aptitude tests try to tell us how well a person will do in learning a foreign
language:
says Madsen (1983:189)
d) Diagnostic tests:
The teachers in the classroom conduct these tests in order to know what is still lacking in teaching and learning.
Diagnostic tests are used to identify learners strengths and weaknesses. They
are intended primarily to ascertain what learning still needs to take place.
Hughes
(2003:15)
2.7 Test Classification:
Here are some of the ways tests can be classified given by Harold.S.Madsen in his book Techniques in Testing.
Table 1. CONTRASTING CATEGORIES OF ESL TESTS
Knowledge Tests Performance (or Sills) Tests Subjective TestsObjective Tests Productive TestsReceptive Tests Language Subskill TestsCommunication Skills Tests Norm-referenced Tests.Criterion-referenced Tests Discrete-point Tests. Integrative Tests
Proficiency Tests..Achievement Tests
Review of the contrasts shown in Table 1. 1. Tests of Knowledge are used in various school subjects, from math and
geography to literature and language. While ESL knowledge exams show how well students know facts about the language, ESL performance exams show how well a student can use language. 2. Subjective tests, like translation and essay, have the advantage of measuring language skills naturally, almost the way English is used in real life. By contrast Objective tests such as multiple-choice or matching tests , can be scored quickly and consistently. 3. Productive test measures, like speaking exams, requires active or creative answers, while receptive test measures, like multiple-choice reading tests, tends to rely on recognition, with students simply choosing the best answer. 4. Tests of language subskills measure the separate components of English, such as; vocabulary, grammar, and pronunciation. Communicative skills tests, on the other hand, show how well the language in actually exchanging ideas and information. 5. do. 6. In discrete-point exams, each item tests something very specific as a preposition or a vocabulary item. Integrative tests are those like dictation that combine various language subskills much the way we do when we communicate in real life. 7. The final classification is proficiency and achievement tests. Proficiency tests can measure overall mastery of English or how well prepared one is to use English in a particular setting, such as; an auto mechanics course or a university. Achievement tests, on the other hand, simply measure progress-gains for example, in mastery of countnoncount noun, use or mastery of the skills presented in an entire language text or course. Norm-referenced tests compare each student with his/her classmates but criterionreferenced exams rate students against certain standards, regardless of how other students
It is obvious from the above discussion that several labels can be applied to any one test. But more often, we apply only one pair of labels at a time.
(The above words under the title Test Classification are the abstract of the topic Test Classification from the book Techniques in Testing 2003: 8 by Harold .S. Madsen)
2.7 (a) Kinds of Testing.
(i)
Direct versus Indirect Testing.
According to Hughes,
Testing is said to be direct when it requires the candidates to perform precisely the skill that we wish to measure. If we want to know how well candidates can write compositions, we get them to write compositions. If we want to know how well they pronounce a language, we get them to speak. The tasks and the texts that are used should be as authentic as possible.
Direct testing is easier to carry out when it is intended to measure the productive skills of speaking and writing.
Direct testing has a number of attractions. First provided that we are clear about just what abilities we want to assess, it is relatively straightforward to create the
conditions, which will elicit the behavior on which to base, our judgments. Secondly, at least in the case of the productive skills, the assessment and interpretation of students performance is also quite straightforward. Thirdly, since practice for the test involves practice of the skills that we wish to foster, there is likely to be a helpful backwash effect. Hughes (2003:17-18)
Hughes also goes on defining indirect testing,
Indirect testing attempts to measure the abilities that underlie the skills in which we are interested.
The main problem with indirect tests is that the relationship between performance on them and performance of the skills in which we are usually more interested tends to be rather weak in strength and uncertain in nature. Hughes (2003:18)
(ii)
Discrete Point versus Integrative Testing.
Definition given by Hughes would best define it, before him :
Discrete point testing refers to the testing of: one element at a time; item by item. This might, for example, take the form of a series of items, each testing a particular grammatical structure. Integrative testing, by contrast, requires the candidates to combine many language elements in a completion of a task. This might involve writing a composition, making notes while listening to a lecture, taking a dictation, or completing a close passage. Discrete point tests will almost always be indirect, while integrative tests
will tend to be direct. However, some integrative testing methods, such as the cloze procedure, are indirect. Hughes (2003:19)
(iii)
Norm-referenced versus criterion-referenced testing.
According to Bachman (1990).
Norm-referenced (NR) tests are designed to enable the test user to make normative interpretations of test results. That is, test results are interpreted with reference to the performance of a given group, or norm. The norm group is typically a large group of individuals who are similar to the individuals for whom the test is designed. Bachman (1990:72)
Criterion-referenced (CR) tests are designed to enable the test user to interpret a test score with reference to a criterion level of ability or domain of content. An example would be the case in which students are evaluated in terms of their relative degree of mastery of course content, rather than with respect to their relative ranking in the class. Bachman (1990:74)
(iv)
Objective testing versus subjective testing.
It would be better to define this topic according to Hughes, he writes:
The distinction here is between methods of scoring, and nothing else. If no judgment is required on the part of the scorer, then the scoring is objective. A multiple-choice test, with the correct responses unambiguously identified, would be a case in point. If judgment is called for, the scoring is said to be subjective. Hughes (2003:22) 2.8 Subjective and Objective Testing:
Subjective and objective are the terms used to refer to the scoring of tests. Students are always required to exercise a subjective judgment while attempting a test paper irrespective of how the tests are devised. In an essay test, a candidate is to think what to say and in which manner s/he expresses the ideas; in a multiple-choice test, a candidate has to be rationale in selecting the proper answer from the various alternatives given.
Hughes defines:
The distinction here is between methods of scoring, and nothing else. If no

judgment on the part of the scorer, then the scoring is objective. A multiple-choice test, with the correct responses unambiguously identified, would be a case in point. If judgment is called for, the scoring is said to subjective.
(2003:22)
It is only the scoring of the test that can be described as objective because all the tests are devised subjectively by the test developer, who decides which areas of language to test and how to test those particular areas.
Unfortunately, in our education system the students are taught by a person who follows the syllabus and an unknown person, who is not in touch with the teaching and learning process throughout the course, is the test developer and the third one is required to check it. How can it be expected to achieve the beneficial backwash effect on teaching and learning in such a situation? This issue will be discussed later on in Section Three.
Effects of subjective and objective testing:
In his book
Writing English Language Tests (1988) Heaton writes two main points of criticism on Objective testing:
1.
Objective tests are frequently criticized on the grounds that they are simpler to
answer than subjective tests.
2.
Objective tests of the multiple-choice type encourage guessing.

In subjective testing, according to J.B.Heaton (1988) the tests are not reliable because the scoring depends on the personal impressions of the teachers, while there is no personal impression or favour on the part of teachers in objective tests so they are more reliable.
2.9 Criteria for the Evaluation of a Test]
Issues of validity:
It is essential for a good test to respect the issues of validity. The test must aim to provide a true measure of the particular skill, which it is intended to measure: to the extent that it measures external knowledge and other skills at the same time, it will not be a valid test.
Madsen defines:
A valid test is one that in fact measures what it claims to be measuring. (1983:178)
According to Heaton:
The test must cover an adequate and representative section of those areas and skills it is desired to test.
(1988:12)
Content validity:
A test is said to have content validity if its content constitutes representative

sample of language skills, structure, etc with which it is meant to be concerned.
Hughes (2003:22)
What is the importance of content validity? First, the greater the tests content validity, the more likely it is to be an accurate measure of what it is supposed to measure. i.e. to have construct validity. Secondly, a test, lacking in content validity, is likely to likely to become areas have harmful backwash effect. Areas that are not tested are ignored in teaching and learning. Hughes (2003:26-27)
Face validity:
Heaton defines it as:
If a test item looks right to other testers, teachers, moderators, and testees, it
can be described as having at least face validity.
(1988:159)
Construct validity:
This term can be well defined in the words of Heaton. He defines:
It (construct validity) is capable of measuring certain specific characteristics in

accordance with a theory of language behaviour and learning.
(1988:161)
Empirical validity:
This validity is obtained as a result of comparing the results of the test with the
results of some criterion measure.
(1988:161)
Issues of Reliability:
When we construct, administer and score tests in such a way that scores
actually obtained on a test on a particular occasion are likely to be very similar to those which would have been obtained if it had administered to same students with the same
ability, but at different time. The more similar the scores would have the reliable, the reliable the test is said to be.
Hughes (2003:28)
For a reliable test what we have to do is construct, administer and score tests in such a way that the scores actually obtained on a test on a particular occasion are likely to be very similar to those which would have been obtained if it had been administered to the same students with the same ability, but at a different time. The more similar the scores would have been, the more reliable the test is said to be. Hughes
(2003:36)
Madsen defines it as:
A reliable test is one that produces essentially the same results consistently on
different occasions when the conditions of the test remain the same.
(1983:179)
Practicality:
A test being reliable and valid is not positive or effective if it creates problems for the administration and practicability of performance required by learner.
A test must be practicable: in other words, it must be fairly straight forward to

administer.
Heaton (1988:167)
SECTION THREE: Backwash Effect
3.1 Backwash Effect on Teaching And Learning:
The effect of testing on teaching and learning is known as backwash, and can be harmful and beneficial. If a test is regarded as important, if the stakes are high, preparation for it can come to dominate all teaching and learning activities. And if the test contents and testing techniques are at variance with the objectives of the course there is likely to be a harmful backwash. Hughes (2003:1)
The term is used to account for how the tests effects on teaching and learning process. It is required that the test techniques and the test content should meet the objectives of the course, only then the beneficial backwash effect can be achieved.
Arthur Hughes defines the backwash effect in the following words:
The effect of testing on the teaching and learning is known as backwash. (2003:1)
That is the reason why the national examinations or tests of much high value for the learners have a major impact on the teaching. And teachers often teach to the test as phrased by James Popham (1989)
Teaching to the test means teaching only what is to be tested. It is quite clear that in order to bring about the changes in teaching one must bring about the changes in testing. According to Hughes:
Davies (1968) has said that the good test is an obedient servant since it
follows and apes the teaching. I find it difficult to agree. The proper relationship between testing and teaching is surely that of partnership.
(2003:2)
Anjum Saleemi says:
Good backwash means that the test and the course of instruction influence each other reciprocally, and that testing is beneficial to teaching-learning process. The test objectives and the course objectives, ideally speaking, should be the same, or at least should correspond very closely.
(1988:261)
3.2
Harmful Backwash Effect:
According to Hughes:
It should hardly be necessary to point out that where a test that is important to students is multiple-choice in nature, there is a danger that practice for the test will have a
harmful effect on learning and teaching. Practice at multiple-choice (especially when-ashappen-as much attention is to paid to improving ones educated guessing as to the content of items) will not usually be the best way for students to improve their command of a language.
(2003:78)
SECTION FOUR:
Evaluation of English Compulsory Papers (I & II) and some suggestions.
4.1
Evaluation Of Grammatical items:
Grammatical items are tested in isolated and decontextualised sentences in Intermediate English compulsory papers. The sentences, which are given in grammatical items, are not connected with each other and form an artificial use of language. The items that are tested are mechanical. They are mechanical in the sense that the students can do them with understanding semantically and sometimes they do the activities without understanding meaning. The grammatical items are tested on the basis of rote learning as well as teaching.
Hughes criticizes the idea of rote learning as:
In the view of some testers, the content of a final achievement test should be based directly on a detailed course syllabus or on the books and other material used. This has been referred to as the syllabus content approach. The disadvantage is that if the syllabus is badly designed, or the books and other materials are badly chosen, the result of a test can be very misleading. Hughes
(2003: 13)
After going through the Intermediate papers(I&II), I have come to know that the paper developers have set the paper not on the modern approach of testing grammar but they still stick to the traditional approach of grammar testing. However, they are using
modern techniques of testing grammar but the purpose of testing grammar is not achieved because of their approach to testing.
Heaton defines the purpose of grammar tests in the way that:
These tests (grammar and usage) measure students ability to recognize

appropriate grammatical forms and to manipulate structures.
(1988:9)
Marianne and Hilles (1988) suggested that grammatical items should be tested in such a way that learners would have to prove that how the grammatical structure is used meaningfully in the context.
Unfortunately in our system grammar is taught in isolated, unconnected sentences and that is how it is tested which gives a fragmented, unrealistic picture of English and makes it difficult for students to apply what they have learnt.
Madsen (1983) says that grammatical items should be tested in such that they would result in achieving positive backwash.
According to my observation of some papers of Intermediate level, all the grammatical are being tested with the support of disconnected, isolated and decontextualised sentences, which with the change of context would have meant something entirely different.
Heaton says:
Note that context is of the utmost importance in all tests. Decontextualised

multiple-choice items can do considerable harm by conveying the impression that language can be learnt and used free of any context. Both linguistic context and situational context are essential in using language. Isolated sentences in a multiple-choice test simply add to artificiality of the test situation and give rise to ambiguity and confusion.
(1988:28)
Hence in the case of articles and prepositions it is required to have the knowledge of when and why of their use which is out of question in these papers. The grammatical items of these papers are discussed below one by one. Question on the use of correct tense form of verbs: In the Intermediate paper (I) there is a question that requires the testees to put the verbs given in the brackets in their correct tense forms. In these types of questions are required the appropriate forms of verbs and the sentences were provided with the neutral form. These types of questions only provide the students to transform the tense in a sentence but the actual purpose of the grammar in a language (its usage and function according to the context) is lacking. Another drawback of these are, in appropriate test techniques there is a possibility of more than one possible answer. E.g., In English paper I (2003) (see appendices paper (1) 2003 P.no)
Questions on the correct use of prepositions and articles:
The questions on the appropriate use of articles and prepositions are imperative in the Intermediate English compulsory paper I. The candidates are asked to fill in the blanks with articles, where necessary and fill in the blanks with appropriate prepositions.
E.g.; In Intermediate compulsory paper (I) 2004 (see appendices paper I 2004 P.no ii)
Q.6 a)
Insert a, an, the in the following blanks:-
i)
______ honest man is _____ noblest work of God.
ii)
He is _____ only person I trust.
iii)
______ copper is ____ useful metal.
Context is very important in the case of article, for many of the cases it is context, which determines whether to use the indefinite article (a/an) or the definite article (the) and the absence of context makes them totally form focused.
The context provides students with enough background knowledge and details to avoid ambiguity and alternative interpretations.(1988:36)
This type of test also shows invalidity so it would be better to avoid practicing these tests in the test papers.
Q.6
b)
Fill in the blanks with appropriate prepositions:-
i)
I came _____ an old friend yesterday.
ii)
He was observed_______ deep thinking.
iii)
He went _____ college ____ noon.
iv)
I brought a gift ______ my father.
To use the prepositions in the sentences in the absence of context, the potential knowledge of meaning remain unrealized but as students are to use them in the sentences so they memorize them because prepositions, that are questioned in the papers, are limited in numbers.
Q.6
(c) Put the appropriate forms of the verbs in the following sentences.
(ii)
If they (miss) the flight, they (not reach) Islamabad to attend the seminar.
Possible answers:
(i)
If they will miss the flight, they would not reach Islamabad. OR
(ii)
If they miss the flight, they would not reach Islamabad. OR
(iii)
If they have missed the flight, they would not have reached Islamabad.
By evaluating the above questions it is quite clear that these questions lack content validity and face validity.
The greater the tests content validity, the more likely it is to be an accurate
measure of what it is supposed to measure. Hughes
(2003:27)
Question On Translation:
The Intermediate English compulsory paper I contains a question that is:
Q.8
Translate the following into English:-
OR
Translate the following passage into Urdu or Sindhi.
(See appendices Intermediate paper I on p.no.3)
The question on translation directly drives my mind to the essay-translation approach (defined in section two), which is now referred to as the pre-scientific stage of language testing.
Tests of translation, however tend to be unreliable because of the complex

nature of the various skills involved and the methods of scoring. In too many instances, the unrealistic expectations of examiners result in the setting of highly artificial sentences and literary texts for translation. Students are expected to display an ability to make fine syntactical judgments and appropriate lexical distinctions- an ability which can only be acquired after achieving a high degree of proficiency not only in English and the mother tongue but also in comparative stylistics and translation methods.
Heaton (1988:18)
This traditional approach in language teaching and testing has its limitations, because we use the sentences/utterances to perform a function and purpose in real life, not just merely complete, make a translate the sentences.
Translation attempts to test multi items at the same time which is not effective
and suggestive in learning and teaching a language.
(1988:44)
Question on the use of Idioms and Phrase:
Idioms and phrases are also given in the Intermediate English Compulsory Paper I &II, the candidates are asked to make sentences using them.
(See appendices Intermediate English Compulsory Paper I&II.)
The purpose of these tests is to test students vocabulary and their ability to use them in writing correctly. But this test item fails to serve its objective because the idioms e.g. at sixes and sevens, to look down upon, crocodile tears, etc are repeated so often that the students can almost guess the ones they will be required to make sentences in the papers. Hence, this test item only tests candidates memory i.e. how well they can utilize it in exams. So these test items cause a negative backwash effect. I think it would be
suitable to apply other testing techniques like; asking the students to match the idioms with their meanings or the idioms should be used in the sentences and students would be required to paraphrase or rewrite the sentences in simple words, etc. These techniques would enable the learners to infer the meaning of the idioms and phrases from meaningful contexts.
Question on Narration:
Question on narration only appears in Intermediate English Compulsory Paper II.
Q.6
Change the following into indirect speech:-
Certainly, cried little Hans, I take quite as a compliment your coming to me, and I will start off at once. But lend me your lantern, as the night is so dark that I am afraid I might fall into the ditch. (See appendices Intermediate English Paper I 2004.)
I think this is the only question that needs no change. The question Change the following into indirect speech is according to communicative approach of testing. This item is not in isolated or unconnected sentences. Context is used in order to develop semantics and pragmatics sense in the candidate. In my opinion, this technique of testing narration is a fit in the Intermediate English Compulsory Paper II.
In support to this item I must quote the Madsen here. He says:

The best exams are those that combine various skills as we do when
exchanging ideas orally or in written.
(1983:7)
After analyzing and evaluating the papers I here recommend that the testing system on the traditional approach should be replaced with the communicative language testing approach which requires that the grammar be tested in a meaningful context, preferably using authentic material (newspaper articles etc). I design a sample test on the communicative approach that would, I think, better serve to test the grammatical competence of the students in general and especially to the college students.
4.2
Advantages of my proposed grammatical items:
The technique that I have chosen to test grammatical items is Cloze Procedure. The reason to choose this vary technique of testing is that it provides ease to the paper developers in preparing test and to checkers in scoring it. It is based on integrative testing (defined in details in section two)-that requires the students to use the grammatical components of language as much like what happens when people communicate.
Madsen has given some of the advantages of this technique as:
1. It is easy to prepare and quite easy to score.
2. It is a good measure of integrative English skills.
3. Standard cloze is a good measure of overall ability in English.
(1988:52)
Madsen has mentioned some of the limitations of this type of test in the words below:
1. It is not a sensitive measure of short-term gains.
2. It is difficult for teachers who are non-native English speakers to choose acceptable equivalents. (1988:52)
From the above given advantages and limitations of cloze, it is quite obvious that this technique is productive for both, the teachers and the students. The limitations that this technique has are not applicable in our testing system because our tests are achievement tests (defined in section two) we are not concerned with short-term gains. The second limitation is also of no use because in our system teachers are not required to prepare a test but the test developers do this task. The grammatical competence of the students in cloze test is required to be used rationally as in real life communication. The teachers and test developers need to be more skillful to teach and test the communicative competence of the learners respectively. Hence, it crystal clear that this type of grammar testing technique achieves the beneficial backwash effect on teaching and learning.
4.3 Conclusion:
After having a thorough look at the Intermediate English Grammar papers I&II, I have come to the conclusion that these types of grammatical item tests only test students knowledge about the language such as; asking students to write a correct form of verb, suitable prepositions or articles which only focus on the knowledge of English grammar but not their ability to use these components in actual communication.
According to Madsen:
Tests today are mainly concerned with evaluating real communication of grammar in second language.(1983:6) Here I would like to quote some lines of J.B.Heaton about language testing. He says:
Language testing constantly involves making compromises between what is

ideal and what is practicable in a certain situation. Nevertheless this should not be used as an excuse for writing and administering poor tests: ideals and goals, constantly trying to devise a test which is as valid as and reliable as possible- and which has a useful backwash effect on the teaching and learning leading to the test.
(1988:24)
Our testing system of grammar needs to be revised in order to achieve beneficial backwash on both teaching and learning.
(See proposed sample items 1 & 2 in appendices)
REFERENCES
1.
Bachman, Lyle. F. 1990. Fundamental Considerations in Language Testing. Oxford University Press.
2.
Bright, J.A. 1982.Teaching English as Second Language. ELBS
3.
Heaton, J.B. 1988. Writing English Language Tests. (New Ed) London and New York
4.
Hughes, Arther 2003. Testing for Language Teachers. (2nd Ed). Cambridge University Press.
5.
Madsen, Harold. S.1987. Techniques in Testing. Oxford University Press
6.
Murccia, M.C and Hills, S 1988 Techniques And Resources In Teaching Grammar. London. Oxford University Press.
7. .
Saleemi, Anjum. P. 1984. Language Testing: Some Fundamental Aspects. (Extracted from A Forum Anthologyp. 260) United States Information Agency, Washington, D.C.
8.
Silverman, David 1993. Interpreting Qualitative Data SAGE Publications, London.
Bibliography
1.
Bachman, Lyle. F. 1990. Fundamental Considerations in Language Testing. Oxford University Press.
2.
Bell, Judith 2001. Doing Your Research Project (3rd Ed) Open university Press. Buckingham-Philadelphia
3.
Batstone, R.K. 1994 Grammar London: Oxford University Press.
4.
Bright, J.A. 1982.Teaching English as Second Language. ELBS
5.
Brown, H. Douglas 2000. Principles of Language Learning and Teaching. (4th Ed) Longman.
6.
Candlin, C.N. Murphy, D.F 1987. Language Learning Tasks. Lancaster practical Papers in English Language Education. Prentice-Hall International
7.
Cunningsworth, Alan 1987. Evaluating and Selecting EFL Teaching Materials. Heinemann Educational Books. London
8.
Ellis, Rod. 2003. Task-Based Learning and Teaching. Oxford University Press
9.
Haycraft, John 1988. Getting On In English: The BBC Intermediate Course. BBC English.
10.
Harmer, Jeremy. The Practice of English Language Teaching. A.I.O.U. Islamabad
11.
Hayes, Andrea. Ryan, Jo. 2001. Wide World of English. Longman
12.
Heaton, J.B. 1988. Writing English Language Tests. (New Ed) London and New York
13.
Howe D.H, Kirkpatrick T.A, Kirkpatrick D.L 2004 English For Undergraduates Oxford University Press.
14.
Hughes, Arther 2003. Testing for Language Teachers. (2nd Ed). Cambridge University Press
15.
Husain, Abbas. Testing Testing. Daily Dawn Newspaper/Internet
16.
Husain,
Abbas.
Private
Exam
Boards:
Rebuttal.
Daily
Dawn
Newspaper/Internet
17.
Lock, G. 1996 Functional English Grammar, Cambridge. Cambridge University Press.
18.
Leech, G and Svartik, J 1975 A Communicative Grammar Of English. Singapore: Longman Group VL Ltd.
18.
Madsen, Harold. S.1987. Techniques in Testing. Oxford University Press
19.
Murccia, M.C and Hills, S 1988 Techniques And Resources In Teaching Grammar. London. Oxford University Press.
20.
Mueen, Aysha. Mrs. 1992. English Language Teaching in Pakistan. National Book Foundation. Islamabad
21.
Nunan, David 1992. Research Methods in Language Learning. Cambridge University Press
22.
Pamela, J Sharp 2003 Barrons TOEFL 13th Edition. Barrons.
23.
Quirk, R and Green Baum, S 1973 A University Grammar Of English. Hong Kong. Common Wealth Printing Press.
24.
Rowntree, Derek 1988. Assessing Students: How Shall We Know Them? Nichols Publishing Company, New York
25.
Saleemi, Anjum. P. 1984. Language Testing: Some Fundamental Aspects. (Extracted from A Forum Anthologyp. 260) United States Information Agency, Washington, D.C.
26.
Sayyid Sadaat Ali Shah 2004. Exploring the World of Grammar. Ilmi Kitab Khana, Lahore.
27.
Silverman, David 1993. London
Interpreting Qualitative Data SAGE Publications,
28.
Stannard Allen, W 1960 Living English Structure. Longman.
29.
Thomson, A.J and Martinet, A.V 1997. A Practical English Grammar (4th Edition). Oxford University Press.
30. Wray, Alison 1998. Projects in Linguistics: A Practical Guide Researching Language. Arnold, London.
Proposed Sample Items #1
Q.
Fill in the passage with suitable articles, where necessary:
My father has _____ big house. We all live together. My _______ eldest brother lives on _____1st floor with his family. He has ______ son and two daughters. _____ son goes to ____ local college, while______ daughters stay at home. Rest of ____ family lives at ground floor. My father is ____ engineer. He gives _____ pickn drop to my sister. I go to ____ evening school.
Q.
Fill in the passage with suitable prepositions:
That was the first time I was expected ____ the harsh realities of life. I came ___ know how selfish everyone was, how no one cared _____ anyone. I realized that this world is not worth good people. I learnt not _____ blindly trust anyone, not ___ be dependent.
Q.
Paraphrase the sentences into simple words: 1. 2. Please turn that radio down. Alis teaching brings out an improvement in the class.
3. 4. 5.
He always stands by his friends in their difficult times. A fair weather friend will forsake us in adversity. Returning to my room, I found that all my things were at sixes and sevens.
Q.
Fill in the gaps with the suitable grammatical usage:
Today (be) my last day in the school and I am (plan) out my future. I (visit) my relatives tomorrow because there (be) a get together. My friends have (advise) me to take admission in computer courses but I will prefer (take) English language classes. May be, I (visit) different places but it (depend) on my parents. If I (visit), I will call my friends for a combined study.
Proposed Sample Items #2
Q.
Correct the use of underlined articles in the following passage, if necessary: Mr. Ahmer is the doctor. He is dedicated and hardworking. He always arrives at
his clinic in time. He usually sets off from the house at 8:30 and starts examining his patients at 9:30. he often travels around a world to attend an International seminars on medicine and new technology in a field of surgery.
Q.
Look at the picture and use the given prepositions in the gaps below: in, on, behind, near, under.
1. 2. 3. 4. 5.
The boy and the girl are standing ______ the table. Theres a butterfly _______ the painting. The legs of boy and girl are seen ______ the table. The pen is ______ the boys hand. The girl is standing ______ the boy.
Q.
Write in each space the best word(s) to replace the idiom(s) or phrase(s) underlined in each sentence.
1. 2. 3. 4. 5.
Sooner or later, everyone must kick the bucket. What are you looking for in my room? When our cousins come to our home, we receive them with open arms. We all want to talk on this subject, but no one is willing to break the ice. It was plain that she was shedding crocodile tears on the death of her mother-inlaw.
Q.
Find the errors and correct the underlined words from the passage, if necessary:
Self-imposed and luxurious livings beyond lawful incomes are mainly responsible for all the evils that have make our nation beyond what it ought to be. Everyone want mighty dollars by any mean. We know better that we have blinked the eyes and dont got at the bottom of this burning question. To rooting out corruption, a complete change in political, social and economical system is necessary. This is the time when every individual should felt his/her active participation in efforts to eradicate evils from our society.

Thesis of MA Linguistics

Загружено:

Сведения о документе

Исходное описание:

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Thesis of MA Linguistics

Загружено:

Авторское право:

Доступные форматы

TESTING

INTERMEDIATE LEVEL AND ITS EFFECT ON TEACHING AND LEARNING.

By: PIR SUHAIL AHMED SARHANDI

UNIVERSITY OF KARACHI JULY 2005

2.3 2.4 2.5 2.6 2.7

Subjective and Objective Testing Criteria for the evaluation of a Test

Evaluation of English Compulsory Papers (I & II) and some suggestions.

4.1 4.2 4.3

Evaluation of Grammatical items. Advantages of Proposed Grammatical items Conclusion

SECTION ONE: Introduction to Research

1.1 Introduction to my Research

According to Arthur Hughes:

Heaton (1988: 171)

Widdowson describes the importance of grammar in these words:

A communicative approach, properly conceived, doesnt involve a recognition

Little-Wood (1986) while stressing the importance of grammar writes:

Communicative language use is only possible by virtue of the grammatical

1.2 Proposed Research Methodology

Qualitative research assumes that all knowledge is relative, that there is a

Reinhardt and Cook (1979)

1.3 My Research Methodology

J.B.Heaton defines the purpose of tests of grammar and usage that

these tests measure students ability to recognize appropriate grammatical

John Bright Heaton defines the purpose of testing as

devices to reinforce learning and to motivate students or primarily as a means

Heaton defines the purpose of tests of grammar and usage that

these tests measure students ability to recognize appropriate grammatical

SECTION TWO: AN OVERVIEW OF TESTING

2.1 What is Testing?

2.2 What is a good Test?

2.3 Why do we need to test?

John Bright Heaton defines the purposes of testing as:

2.4 Why do we need to test grammar?

These tests measure students ability to recognize appropriate grammatical

says Heaton (1988:9)

2.5 Approaches to Language Testing

Language tests may be classified according to four main approaches to testing:

a) The Essay-translation approach:

b) The Structuralist approach:

c) The Integrative approach:

d) The Communicative approach:

This approach is sometimes linked to the integrative approach. However, both

2.6 Kinds of tests and their purposes.

Lyle F. Bachman has suggested five features to classify tests

Achievement/ attainment tests,

Aptitude tests, and

Progress achievement tests, as their name suggests, are intended to measure

Madsen defines it as:

says Madsen (1983:189)

2.7 Test Classification:

Table 1. CONTRASTING CATEGORIES OF ESL TESTS

Proficiency Tests..Achievement Tests

2.7 (a) Kinds of Testing.

Direct versus Indirect Testing.

Hughes also goes on defining indirect testing,

Discrete Point versus Integrative Testing.

Definition given by Hughes would best define it, before him :

Norm-referenced versus criterion-referenced testing.

According to Bachman (1990).

Objective testing versus subjective testing.

It would be better to define this topic according to Hughes, he writes:

The distinction here is between methods of scoring, and nothing else. If no

Effects of subjective and objective testing:

__ honest man is _ noblest work of God.

__ copper is useful metal.

He went _ college noon.