99

TOPIC 6

Testing and Evaluation of Reading Skills




FRAMEWORK OF TOPICS










Session notes: During this module it is expected that course participants will
self-study and be prepared to do these activities in the tutorial in groups or pairs
to construct additional meaning with classmates.



Discrete feature test for reading

This test is mainly concerned with testing vocabulary. This is because it is thought
that vocabulary tests can be a good predictor of pupils reading ability. Another
reason is that the syllabus for KBSR and KSSR spells out vocabulary items that
pupils need to master at each stage of their reading development. By including
vocabulary items in tests of reading can also be intrinsically motivating for pupil to
recognize they are learning a second language, and secondly, vocabulary tests can
encourage them to learn even more words. However, we must remember that test of
vocabulary for reading are essentially test of recognition. Most of us can recognize
many more words in our listening and reading tests than we can use in our speaking
and writing. Samples of discrete feature reading tests are as follows.


discrete feature tests

holistic reading tests

Types of Reading Tests

100
Holistic reading tests

When choosing texts we must select texts that are familiar to the pupil and truly test
reading comprehension and not just prior knowledge. To do so authentic texts are
recommended that provide a more realistic and reliable means of assessment and
also help to motivate pupil by demonstrating how the target language is used in real-
life situations. We should not focus our questions on information contained in
individual sentences. Rather, we ask questions which require pupils to look at ideas
from different parts of the text together, for example, by asking questions about main
points, cohesion and predication of outcomes. Finally, we should make sure that our
test really tests reading and not something else, for example, the pupils general
knowledge.

There are various holistic testing formats that a teacher can design to assess the
pupils reading skill.

Below are some common tests.

1 Multiple choice tests

This is a form of assessment where the pupil is given a choice of a few options to
select the best answer to the question. It is an effective and efficient way to test
reading comprehension. The example below illustrates multiple choice testing.






Taken from Tan et al. Reference PMR English, 2005.

1. Where was the victim when the fire broke out?

A Sleeping with her baby.
B Going to a sundry shop.
C Working at a sundry shop.
D Walking home from the sundry shop.
Mother burnt while saving baby

Kuala Lipis : A mother suffered burns on her body and legs when she rushed into her
burning house to save her sleeping baby. Rahimah Salam, 37, was walking to a
nearby sundry shop when she heard cries of fire coming from the direction of her
house.


101
2. Which of the following statements is TRUE?

A The victim was burnt all over her body.
B The news was reported in a local newspaper.
C Someone started the fire in Rahimahs house.
D The mother was buying food at the sundry shop.


Task : With a partner use the text below to create 3 multiple
choice questions.












Taken from Tan et al. World of English Easy Link, 2004

2 Text completion

This form of assessment requires the learners to understand of the content of the
stimulus. The pupil may be expected to complete the sentence with a word, a phrase
or a sentence. It might test the learners overall comprehension of the stimulus,
specific area or the learners attitudes, beliefs, motivations, or other mental states.

Example 1
Read the short dialogue and complete the sentence.
Lisa : Do you want to go cycling with me after school?
Devi : No I cant. I dont really like that form of exercise.

Devi does not enjoy _______________ .

This is a story of a special four-year-old girl Loke Kim Hee who wants to follow
her neighbours children to the nearby kindergarten. But she can only do so if
she undergoes a hole-in-heart surgery.
Kim Hee suffers from a heart problem from birth. Her condition was
detected when she was two weeks old. According to her mother, Lee Sook
Fong, her daughters condition is getting worse. She is often short of breath
and her fingernails are now slightly blue. Kim Hee needs to undergo a
corrective heart surgery to close the hole in her heart as soon as possible. But
Sook Foong who earns RM800 a month as a seamstress is unable to raise the
RM56 000 needed for the operation. Sook Foong is appealing for donations
from the public so that her daughter can lead a normal life. Those who wish to
donate can do so by sending crossed cheques, money order or postal order
payable to New World Publishing, 48 Jalan Murni, 59100 Kuala Lumpur.
Please write Kim Hees Fund on the reverse side of your cheque.


102
Example 2

Read the text and complete the sentences that follow.








Taken from Tan et al. World of English Easy Link, 2005

1. The main purpose of the car wash was ____________________________ .

2. ________________ and _______________ benefitted from the proceeds of
this event.


Example 3
Read the following advertisement and complete the sentences that follow.
Write one word or phrase in each space.















New Vogue Theatre
Forthcoming Attractions
Monday 8 January for 2 weeks
MY FAT FRIEND
Charles Lawrences popular comedy
^^^
Wednesday, 24 January
Saturday, 27 January
Shanghai Festival Ballet
Presents
SWAN LAKE
^^^
Monday, 29 January for one week only
Tim Danbys thrilling mystery
RUN AND KILL
Beauties in charity car wash

Penang: Eighteen young ladies taking part in the Miss Teen Princess 2001
converged at the car park of the Tanjung Pinang Shopping Complex for a
charity car wash which raised RM2,300.
The event, which was jointly organized by the RodaAct Club and the
management of the Tanjung Pinang Shopping Complex, was to raise funds
for the Handicapped Children Welfare Society and the Bakti Orphanage.




103
1. The Shanghai Festival ballet will perform on ________________ evenings.

2. ______________will be the most amusing play.

3. If you like dancing, you should see_________________.

4. The Shanghai Festival ballet will perform on ________________ evenings.

5. ______________will be the most amusing play.

6. If you like dancing, you should see_________________.

7. The play written by _________________ is very exciting.




Task : Use the theatre advertisement to create four text
completion questions below.



Example:
The Shanghai Festival ballet will present ______________.

1.

2.

3.

4.


104
3 Cloze Test

In cloze tests, the words are deleted systematically. The interval at which words are
deleted is usually between every fifth and every ninth word. However, if every
seventh word has been deleted in the first few sentences, then every seventh word
must be deleted for the rest of the text. The most common purpose of the cloze test
is to measure reading comprehension. When designing a cloze text, always leave the
first sentence intact to introduce the context. This text can be turned into a multiple
choice question.

The example below uses the 7
th
word deletion.









Taken from Tan & Tan, PMR Kertas Model Cabaran Bahasa Inggeris 2000























Slowly the train pulled out of the station and disappeared around the corner
taking Swee Ching along to pursue her dreams.
I was glad to send Swee Ching off. It had been her burning desire to
continue her (1) _____after she completed her Form Five. But (2) ____ had
not been kind to her.
As soon as she had finished her SPM, she enrolled at a private (3) ____
to study accountancy. Unfortunately her (4) ____ were shattered mid-way
when both her (5) ____ were killed in a road accident.


105

4 Open ended questions

This type of question should tests a whole range of reading comprehension skills. It
may range from identifying main ideas, supporting details, inferring skills, cause and
effect to the pupils personal reaction or interpretation of the text.

Example


Read the text and answer the questions that follow.

Late one evening Sarah saw a mouse running into the cupboard in the kitchen.
She wanted to catch the mouse. So she looked for a mouse trap and some food.
She found the trap but she couldnt find anything to put inside it. It was late and
all the shops were closed. So Sarah drew a picture of some cheese and put it in
the trap.

The next morning the picture of the cheese was gone but there was another
picture. The picture showed a mouse.

Why did Sarah need to catch the mouse?


What word would you use to describe the mouse in the story? Why?









5 True/False questions

When we construct such a test we must be sure that our statement is clearly true or
false and not partly true and partly false. To make the test more challenging we do
not lift (copy and paste) statements from the text. This is to ensure that pupils do not
merely match the words of the statements with the words of the text. We should
paraphrase clearly, otherwise pupils can answer with the correct item without
understanding the text. Also there should be more true statements than false ones.









106

Example

Read the text and answer the questions that follow. Put a tick () in the right
column, T for true and F for false.


It is often easy to recognize Muslim countries because of their beautiful buildings. These
often have domes, arches and minarets with decorations on them.

There are no priest in Islam. An imam leads the prayers. Worshippers stand in rows and
follow the movements of the imam. They all face the direction of Mecca.

Most mosques have a minaret, a tall slender tower. The call to prayer is made from this
tower. Once the minaret was also used to act as a lighthouse to guide travellers in the
dark.


1. Domes, minarets and arches are beautiful
parts of Muslim buildings.

True False

2. An imam is a Muslim priest.



3. The call to prayer is made from the
minaret.



4. Minarets no longer act as lighthouses.




Answer key:
1. T
2. F
3. T
4. T





That is a brief introduction on the two types of commonly used tests to assess pupils
reading comprehension: discrete feature tests and holistic tests.

Next, we move on to an important section that we need to adhere to when designing
tests.




107

Designing test questions

What should we test? How should we test? What types of test should we administer
and what levels of thinking should our questions be? These are important questions
we have to ask ourselves so that our questions have validity and reliability.

The Table of Specifications is essentially a blueprint for designing reading test. As
the name implies, it specifies the content of the test. Its basic purpose is to ensure
that all intended outcomes are measured and that the test includes the appropriate
number of items for each measured item.

The table of specifications allows the test objectives to be identified quickly and
explains the reading skills, context and level to be tested in the reading classroom.
1


Below is an example of a table of specifications for testing the reading skill.

Section Skill Format No. of Items Marks
A



Reading for main
ideas



MCQ

5
10

Reading for
details

5

B




Reading to make
inferences

open
ended

5

10

By looking at the Table of Specification, we can tell that this test is made up of two
sections and each section carries ten marks. Then the two sections test different
reading skills and the types of questions will also vary. There will be 15 questions all
together in this test.

The table above is a very simple Table of Specification. In more comprehensive
ones, you may find the levels of thinking order questions specified, whereby there will
be a spread from lower order to higher order thinking questions and the number for
each item specified. The scope of the questions may also be determined.







1
Ridwan Mohamed OSMAN (2012). Educational evaluation and testing. African Virtual University, downloaded from the World
Wide Web August 2012.

108


Task : My table of test specifications

You are asked to prepare questions for a monthly test for the reading section in your
school:
Draw a table of test specifications.
Come up with the objective of reading items that you seek to test.
Be sure that your test items correspond to the table of specifications that
you have set.


This ends the section on designing questions using discrete feature test and holistic
testing.

The following section is an example of an assessment which may be used informally
and as a formative type of assessment that can be utilized in the primary school
classroom. It is essential that the teacher adapts and organizes these suggested
activities so that they are appropriate to the second language reading and vocabulary
level of pupils in their classroom.


Five Reading Powers
The Five Reading Powers is a type of holistic and formative assessment to be used
after the pupils have become familiar with reading comprehension strategies taught.
Here, the pupils are to apply metacognition skills to their reading.

According to Gear (2011) in order for our pupils to become better readers they must
first become better thinkers. The Reading Powers teach pupils not what to think
rather how to think. The Reading Power approach respects pupils thinking by
teaching them that their thoughts, their connections, their images, their questions,
and their insights are the most important things when it comes to constructing
meaning in reading. By teaching pupils that reading is not just words on a page, but
about our thinking, feeling, and imaginations we can bring those words and books to

109
life in our lives. For teachers, the Reading Power templates provide the language
and format for pupils to learn how to think and talk about their experience with
reading, all of which inspires pupils intrinsic motivation and metacognition in our
reading classroom.

Using the five Reading Powers
2

Below is an excellent example to have pupil practice using metacognitive strategies
while they read. The reading powers include five critical moments of metacognition:

1. Connect: What does this story remind me of?

The ability to make connections between the story they are reading and their own life
experiences. This increases what they know and allows them to assemble new
information in a logical way.

2. Question: What am I wondering about this story?

The ability to ask both literal and deep thinking questions while they read allows
pupils to explore the story, deepen meaning, comprehension and enhance the
learning experience.

3. Visualize: What pictures can I make in my head from this story?

The ability to use words in a text to create mental images, or a movie in the mind
while they read.

4. Infer: What am I thinking about this story that isnt actually written?

The ability to look for clues within the text and picture them while they read and to
use these clues to fill in what is not written directly in the text.

5. Transform: How has my thinking changed because of this story?

2
This section has been adapted from Adrienne Gears book: Reading Power, chapter 8, page 115.

110
The understanding that books have the capability of transforming the way they view
themselves, others, and the world; the ability to identify the things in a book that
matters most to them.

111
Lower proficiency pupils and the five Reading Powers
The five Reading Powers are designed to teach metacognition skills to pupils with an
already well-developed lexicon of English vocabulary and reading proficiency skills.
For pupils with a lower proficiency, the teacher may decide to initially introduce only
ONE or TWO of the reading powers in class. For instance, over the course of one
month, the teacher may focus on two reading powers: making connections and
asking questions.

Two months later, the reading class can use four reading powers: making
connections, asking questions, visualization and inferences. In the third month, the
pupils focus on integrating the five reading powers: making connections, asking
questions, visualization, inferences and transformation of thought. In this way, the
pupil with lower proficiency can develop the five reading powers over time and not
feel overwhelmed with the number of new skills to be learned during the English
class. It is critical that teachers be sensitive to their pupils needs to implement the
Reading Powers strategy effectively.



Reading Powers Questionnaire
The following questionnaire is a good indicator for teachers to determine if your
pupils have developed on awareness of their thinking and metacognition. Secondly,
you can adapt and hand out this questionnaire in class with your pupils as a form of
self-assessment of their own reading skills (powers). The questionnaire will provide
you with insight to guide your teaching, and will help you establish which strategies
you may need to focus on. For example, if most of your class is regularly making
connections, then you may not need to spend much time on the connection strategy.

112
Reading Powers Questionnaire

1. What parts of the body do you use when you read?

2. What things did you need to learn in order for you to be able to read?
(E.g: I need to know the alphabet)

Circle the best answer for you:
1. When I read, I make connections between what I am reading and my own
experiences, other books and the world around me.

Always Often Sometimes Never

2. While I am reading, I ask myself questions about the story.

Always Often Sometimes Never

3. While Im reading, I make pictures in my head about what is happening in the
story.
Always Often Sometimes Never

4. While Im reading, I fill in words or pictures in my head that the author didnt
include.
Always Often Sometimes Never

5. While Im reading, my ideas, thoughts, and opinions about what Im reading
will change.
Always Often Sometimes Never







113
Assessment of Reading Powers
In order to engage in a formal assessment of each of the reading powers, the teacher
can use the Comprehension Assessment. This can be used as an individual
interview with each pupil after a strategy is taught, or as a whole-class assessment.
You may want to choose one story to read aloud and then have the pupils write their
responses. For lower primary pupils, you may need to read the questions aloud and
leave time in between for the pupils to write their responses.

























114
Comprehension Assessment

Name: Grade:
Book: Date:

Reading Power Question Pupils Response
Connect



Can you tell what connecting is?
Can you tell me some different ways you
can make a connection (thinking feeling,
prior knowledge)
Give me an example of a connection you
made to this story.

Question Can you tell me the difference between a
quick question and a deep-thinking
question? Give me an example of each
from this story and then try to answer
them.

Visualize Can you tell me what visualization is?
Can you tell me about a part in this story
where you visualized? Tell me about
some other senses you used besides
what you saw.

Infer Can you tell me what inferring is? What
are you doing when you infer? What
things help you to infer? Give me an
example of inferring from this story. Give
me a maybe thought you had on this
page.

Transform Can you tell me what transforming is?
When does thinking change when you
are reading? Give me an example of
how your thinking. changed when you
read this story.




115
The Reading Power criteria enclosed on the next page outlines a five-point scale that
ranges from Exceeding to Not Yet Meeting levels of achievement according to the
reading powers comprehension strategies. This will also help you determine what
level pupils are reading at in your classroom




116
Reading Powers Rubric

Exceeds Fully Meets Meets Minimally Meets Not yet Meeting



C
O
N
N
E
C
T

- fully understand
the strategy of
connecting to text
and how to use
these personal
connections
cognitively, orally
and in written form
to enhance
understanding of
what he/she is
reading

- connects to self,
other text, and the
world without
prompting.







- understand the
strategy of connecting to
text, and is easily able
to make personal
connections to the
stories he/she is reading
to enhance his/her
understanding

- connects to self, other
text, and the world with
prompting.


- is learning how to make
personal connections
with the text his/she is
reading in order to make
better sense of the story;
can express simple
connections orally and in
written form with no
prompting

- makes connections that
are relevant and enhance
comprehension

- has been introduced to
the during-reading strategy
of connecting to text and is
beginning to understand
how connections can help
enhance understanding of
the story; can make a
connection when
prompted, but it is not
always meaningful to the
story.

- makes connections that
may be more literal than
inferential

- is not yet able to make
(or requires support when
making) personal
connections to stories
he/she is reading in order
to better understand the
text.

117


Q
U
E
S
T
I
O
N

- is easily able to
ask him/herself
thoughtful
questions, both
literal and deep
thinking, during the
reading process,
and understand fully
how these
questions, answered
or not, can lead to a
greater
understanding of the
text


- is able to ask
thoughtful questions
during the reading
process, and
understands how the
process of asking and
answering questions
while reading can help
him/her better
understand the text
evidence of both literal
and inferential (deep-
thinking) questioning is
present

- is learning the strategy
of asking him/herself
questions while reading,
and is demonstrating a
beginning understanding
of how the answers to
these questions can help
enhance understanding
of the text; questions are
generally literal with
some evidence of
inferential

- is learning to ask
him/herself simple
questions about the story
he/she is reading, and
demonstrates minimal
understanding of how this
strategy can be used to
enhance comprehension of
the text

- is not yet able to ask
him/herself questions
about a test while reading,
nor to demonstrate an
understanding of how this
reading strategy may help
him/her understand the
text better


V
I
S
U
A
L
I
S
E

- can easily and
naturally combine
the words in a text
with his/her won
background
knowledge to create
mental images while
reading, as
demonstrated in
both oral, written
and drawn

- is able to use the
words in a text to create
mental images while
reading, demonstrates
this in both written and
drawn responses
-incorporates some of
the senses to mental
images
-fully understands how
visualizing while reading

- is learning to use the
words in a text to create
mental images while
reading and is attempting
to use this strategy while
reading independently as
demonstrated by oral,
written and drawn
responses
-attempts to incorporate
more than one sense

- is beginning to use the
words in a text to create a
movie in the mind while
reading, and demonstrates
a beginning understanding
of this strategy as
demonstrated in oral and
drawn responses
-is not yet able to utilize
more than one sense
unless prompted

- is not demonstrating
competence or
understanding of
visualizing
(creating mental images of
what is happening in a
story while reading) unless
there are pictures or
illustrations to support
him/her

118
responses.
-is easily able to
incorporate the
senses to mental
images
-fully understands
that visualizing while
reading will help
enhance
comprehension and
bring reading to life
can help to enhance
understanding of the
text
when visualizing.



I
N
F
E
R

- demonstrates
complete
understanding of the
difference between
a fact and an
inference. While
reading, is
competent in looking
for clues within the
text and pictures,
and combines these
clues with his/her
own background
knowledge to help
fill in and interpret
the text

- understands the
difference between fact
(something that is
written explicitly in the
text) and an inference
(something that is
implied in the text);
while reading, is able to
look for clues in the text
and in pictures and is
learning to infer
meaning form them by
reasons with I think
or Maybe

- is beginning to
demonstrate an
understanding of the
difference between fact
and an inference, and is
learning how to look for
clues in both pictures and
text what will help him/her
to make these inferences
and make better sense of
the text.

- demonstrates a
beginning understanding of
what an inference is and,
with support is learning to
look for clues within the
text and pictures to help
him/her make these
inferences.

- has difficulty
distinguishing the
difference between a fact
something that is written
explicitly in the text) and an
inference (something that
is implied in the text) and is
not yet able to make
inferences indepentlty
while reading




119


T
R
A
N
S
F
O
R
M

- understands fully
that books have the
ability to change the
way we think about
ourselves and our
world, and is able to
look beyond the
pages of the text
towards the
implications and
affects the book
may have on his/her
own life as
demonstrated in
written and oral
responses.

- understands that
books have the ability to
change the way we
think about ourselves
and our world, and is
able to look for and
identify things in a book
that matter most to
him/her as
demonstrated in written
responses.


- is learning that a story
has the power to change
the way we think about
ourselves , others, and
the world around us and
is learning to identify
things in a book that
matter most to him/her

- is demonstrating minimal
understanding of what it
means to be transformed
by a book and, with
teacher support, is learning
to identify important issues
in books that may change
the way he/she thinks.

- is not yet demonstrating
an understanding of
Transform, and is unable
to identify important issues
in books which may
change the way one might
view themselves or the
world.



120