Malaysia Standards Referencing and School Assessment (Adj)

Jim Tognolini
International Experience on Standards

Referenced Assessment in the Context of
School Based Assessment
Why are standards important?

Political and media issues centre around
questions such as: are educational standards
improving or falling?
Qualifications frameworks are being
developed around standards e.g. European
Framework for Lifelong Learning
They enable the comparisons of education
systems e.g. PISA and TIMSS
Empower students and teachers in the
learning process to understand what is
required of them.

Oxford University Centre for Educational
Assessment. www.education.ox.ac.uk/assessment/
What are Standards?

Standards describe what it is that students should
be able to know and do.

The US New Standards Project defines standards
as: what students should know and be able to do.
This definition is also used by the American
Foundation of Teachers (see AFT, 1999).

Victorias CSF, on the other hand, uses standards
as a synonym for learning outcomes which are
described as: benchmarks or standards against
which student achievement can be measured.

Types of Standards
Commonly a distinction is made between
curriculum standards and performance standards.

Curriculum standards are the knowledge, skills
and understanding expected to be learned by
students as a result of studying a course; while,

Performance standards are defined as the levels
of achievement of the knowledge, skills and
understanding

Curriculum Standards
Curriculum standards: (Statements of what
students should be able to know and do at
specified stages (grade levels) along the
developmental continuum.)

Syllabus Standards
Content Standards
Grade Level Standards
Core Standards
Outcomes
Competency Standards (VET and Professional)

Performance Standards
Performance standards: (Statements of how
well students are expected to be able to know
and perform the skills included in the syllabus
standards.)

Achievement standards
Benchmark Standards
Proficiency Standards
Reporting Standards
Accountability/Target Standards
Performance Indicators

Examples of Performance Standards
Standards for a subject are generally partitioned
into levels of achievement or performance for that
subject.

New South Wales, for example, has 6-performance
levels (bands) for each subject in the New South
Wales Higher School Certificate (HSC).
The GCE (General Certificate of Education) in the
UK has an 8-point scale A* to G and U. Performance
descriptors (performance standards) have been
written for the A/B boundary and the C/D boundary
for each subject.

Standards for a subject are generally partitioned
into levels of achievement or performance for that
subject.

In Queensland results for each subject are reported
on a 5-point scale: Very High Achievement (VHA),
High Achievement (HA), Sound Achievement (SA);
Limited Achievement (LA); and Very Limited
Achievement (VLA).
In New Zealand, each achievement standard is
composed of 4 categories (equivalent to bands or
achievement levels): students do not achieve the
standard (NA); students achieve the standard (A):
students achieve the standard with merit (M); and,
students achieve the standard with excellence (E).

Standards for an outcome are generally
partitioned into one or more levels of achievement
or performance for an outcome or a criterion.

In some US states each standard (there may be 5 to
10 for a subject across grades) is partitioned by
benchmarks at key locations at the end of a segment
of work (e.g. Colorado has benchmark descriptors for
Grades k to 4; 5 to 8; and, 9 to 12)
ACARA uses an achievement standard for each
Grade Level and the standard states what students
should be able to know and do by the end of the
nominated grade.

Performance Standards and Benchmarks Colorado
Mathematics
Benchmarks
Standard Description Grades K - 4 Grades 5 - 8 Grades 9- 12

1

Students develop number
sense and use numbers and
number relationships in
problem-solving situations and
communicate the reasoning
used in solving these problems
1. Demonstrate meanings
for whole numbers and
commonly used fractions
and decimals; and
represent equivalent
forms of the same number
through the use of
physical models, drawings
calculators and
computers.
1.Demonstrate meanings
for integers, rational
numbers, percents,
exponents, square roots,
and pi use physical
materials and technology
in problem-solving
situations.
1. Demonstrate
meanings for real
numbers, absolute
value and scientific
notation using physical
materials and
technology in problem-
solving situations.
2. Read and write whole
numbers and know place-
value concepts and
numeration through their
relationships to counting,
ordering and grouping.
2. Read, write and order
integers, rational numbers
and common irrational
numbers such as 2
2. Develop, test and
explain conjectures
about properties of
whole numbers and
commonly used
fractions.
3. Use numbers to count,
to measure, to label, and
to indicate location.
3. Apply number theory
concepts to represent
numbers in various ways.
3. Use number sense to
estimate and justify the
reasonableness of
solutions to problems
involving real numbers
4. Develop, test and
explain conjectures about
properties of whole
numbers and commonly
used fractions.
Performance Standards and Benchmarks Colorado
Mathematics
Benchmarks
Standard Description Grades K - 4 Grades 5 - 8 Grades 9- 12

2

Students use geometric
concepts, properties and
relationships in problem-
solving situations and
communicate the reasoning
used in solving these problems
1. Recognise shapes and
their relationships using a
variety of materials
1. Construct two- and
three-dimensional models
using a variety of materials
and tools.
1. Find and analyse
relationships among
geometric figures using
transformations in
coordinate systems.
2. Identify, describe, draw,
compare, classify and
build models of geometric
figures.
2. Describe, analyse and
reason informally about
the properties of two-and
three-dimensional figures.
2. Derive and use
methods to measure
perimeter, area and
volume of regular and
irregular geometric
figures.
3. Relate geometric ideas
to measurement and
number sense
3. Apply number theory
concepts to represent
numbers in various ways.
3. Make and test
conjectures about
geometric shapes and
their properties,
incorporating
technology where
appropriate
4. Recognise geometry in
their world
4. Solve problems using
coordinate geometry.
Australian Curriculum Kindergarten Mathematics
Number and Algebra Statistics and Probability Measurement and Geometry
1. Counting
Say understand and reason with number
sequences, initially to and from 20 and
then beyond, moving to any starting point
1. Data Representation
Collect, represent and interpret data from
simple questions with objects and
drawings where one object or drawing
represents one data value
1. Geometry
Sort, describe, name and represent
familiar two-dimensional shapes and
three-dimensional objects in the
environment
2. Numeration
Understand numbers to 10, including
matching number names, numerals and
quantities, and work fluently with small
numbers including subitising and
partitioning
2. Data Investigation
Solve problems by collecting data and
answering questions about obvious
attributes of themselves and familiar
objects and events.
2. Comparison
Use direct and indirect comparison to
decide which is longer, heavier and holds
more and explain reasoning in everyday
language
3. Comparing Collections
Compare and order collections, initially to
20, and then beyond; and explain
reasoning.
3. Time
Read time on the hour on digital and
analogue clocks, and make connections
between common sequences such as
days of the week
4. Addition and subtraction
Model, represent and solve problems
concerning additive and sharing
situations involving combining, change
and missing elements.
4. Location
Describe the position and movement of
objects, including themselves.
Australian Curriculum Kindergarten Mathematics
Achievement Standard (Kindergarten)

By the end of Kindergarten, students are able to confidently recall the
sequence of numbers to 20, matching names and numerals and find the total of
small collections of counting. They subitise small quantities, partition numbers
to 10 and use one-to-one relations to share and count out quantities. Students
collect data from straight forward questions about themselves and familiar
events and, with assistance, can organize this data. They already use language
to describe measurements found by direct comparison and sort and classify
familiar shapes.

Performance Standard
The Malaysian Primary School Performance Standard;
Band 6
Master and apply mathematical knowledge and skills in
solving complex mathematical problems creatively and
innovatively
Band 5
Master and apply mathematical knowledge and skills in
solving complex mathematical problems using different
strategies
Band 4
Know and understand mathematical knowledge to do
mathematical computational procedure, conversions with
guidance, and solve simple mathematical problems.
Band 3
Know and understand basic mathematical knowledge to do
basic mathematics operations, direct conversions and solve
word problem.
Band 2
Know and understand basic mathematics knowledge
Band 1
Know basic mathematics knowledge
Band 6 Biology Description
The typical performance in this band:
demonstrates an extensive and detailed knowledge and superior understanding of
biological concepts, including complex and abstract ideas
demonstrates an extensive understanding of the historical development of biological
concepts, their applications and implications for society and the environment, and the
future directions of biological research
communicates succinctly, logically and sequentially using a variety of scientific formats,
including diagrams, graphs, tables, flow charts and equations relating to biology
analyses and evaluates data effectively, identifying biological relationships, quantifying
explanations and descriptions, synthesising information to draw conclusions
uses precise biological terms extensively and correctly in a wide range of contexts
designs valid experimental processes using appropriate technologies and incorporating the
thorough knowledge of the use of a control, variables and repetition to solve biological
problems
applies knowledge and information to unfamiliar situations and designs an original solution
to a biological problem

Band 5 and 4 Biology Descriptions

Band 5
demonstrates thorough knowledge and understanding of most biological concepts
demonstrates a thorough understanding of the historical development of biological
concepts and their applications and implications for society and the environment
communicates effectively in a variety of scientific formats including diagrams, graphs,
tables, flow charts and equations relating to biology
explains qualitative and quantitative biological relationships and ideas coherently; identifies
patterns in data to draw conclusions
uses precise biological terms frequently and correctly in a range of contexts
identifies the correct application of scientific experimental methodology to solve biological
problems

Band 4
demonstrates sound knowledge and clear understanding of some biological concepts
demonstrates a sound understanding of the historical development of biological concepts
and their applications for society and the environment
communicates using clear written expression and incorporating diagrams of biological
structures
provides qualitative and quantitative descriptions of biological phenomena and explains
straightforward biological relationships
uses general biological terms frequently and correctly in a range of contexts
identifies the correct components of the experimental scientific method in biology

Setting Standards
If As are easy they lose their power to inspire effort.

If Cs become rare, then they are more likely when given
to discourage students and crush their desire to try
harder.

Initial standards setting needs to be in achieveable rather
than aspirational.

Requirements for Standards in Standards-
Based Education Systems
Standards should describe performance expectations
and proficiency levels in the context of a clear
conceptual framework, and are built on sound models of
student learning (developmental continuum).
Standards should be clear, detailed, and complete;
reasonable in scope; and both rigorous and well- grounded
in the knowledge domain.
Standards must be elaborated so that curriculum,
teaching and assessment are aligned.
Standards are derived from the curriculum and not
developed independently from mandated curriculum
requirements.

A Standards-referenced Approach
Curriculum Standards
Intentions and
Expectations
Clarified by
Aims
Objectives
Outcomes
Content
Levels of Achievement
Clarified by
Performance Descriptors
Exam Papers and Tasks
and Marking Guidelines
Student Responses
ASSESSMENT
including
EXAMINATIONS
What is to be learned What is reported
Content and Standards
Less formal More formal
Unstructured
chance meetings
conversations
Most structured
examinations
standardised tests
published aptitude
tests
Slightly structured
questionnaires
observation
student self-
assessment
More structured
classroom tests
checklists
practical work
project work
case studies
Assessment involves professional judgement
based upon an image formed by the collection of
information about student performance
Oxford University Centre for Educational Assessment.
www.education.ox.ac.uk/assessment/
Why Assess Students?
To inform prospective employers or admissions
committees
To distinguish between the quality of performance of
students in a given class programme or school
To report to parents and the students themselves
To motivate students
To assign standards of performance

Oxford University Centre for Educational Assessment.
www.education.ox.ac.uk/assessment/
Assigning Standards
Student achievement is assessed using either external
examinations, internal assessments or a combination of each.
Student images are constructed. These images are extracted
from and modified by information from marks, portfolios, etc.
(categorical judgement) Each student can then be assigned to a
band or level (category) based upon an on-balance judgement
using rating scales (similar to marking extended response type
items with an analytical marking scale where the marking rubric
shows an ordered developmental progression).
Standard Setting
Alternatively, teachers can construct a distribution of marks by
aggregating the results from a range of different tests, term
examinations, assignments, etc. during the year; order the
students in the class or school for a subject on the basis of their
total marks (added across all tests etc.); and, then come down
the list until the image that they have for the borderline Grade
A/B student corresponds with the student in the school list. This
student is thus the Grade A/B borderline student and the mark
that the student obtained is the cut-off mark. All students with
scores above this student are in Grade A. The process can be
repeated to obtain all the other cut-off marks for all the other
bands.
Standard Setting
More formal judgemental models can be used with distributions
of examination results. In this case one of a range of techniques
(e.g. Angoff Method; Bookmark Method; and, statistical equating
methods) uses professional judgement to locate the marks
accorded to borderline students (students at the Grade A/B
border; etc.) along the performance scale and then the
distribution of results is transposed onto the performance scale.
These methods are referred to as standard setting techniques.

Enables the performance of cohorts of students to be
monitored against pre-determined standards.
Empowers the student in the teaching and learning
process.
Provides all students with a meaningful record of their
achievements.
Provides a mechanism to recognise and reward prior
achievement at school within broad-based
qualifications frameworks.
Provides a mechanism to bring together curriculum,
pedagogy and assessment in a way that has not been
possible in the past.

Value of Standards-referencing
Changing Role of Examinations

Traditionally used for high-stakes reporting

Increasing importance of school-based assessment

Rationale for School-based Assessments
What counts is what is valued
Fairness
Empowerment of teachers
You cannot have reliability without validity
Relationship between student and teacher
Issue of comparability

Issue of Comparability
Need something in common
Could be examination
Could be generic tests
Could be common assessment tasks
Could be standards

Conditions for Comparability
Teachers need time to make good assessments
Teachers within a school get consensus
Across-school comparability can be achieved with
different models
Statistical
Social or consensus
Mixed model

Challenges of School-based Assessment from a
Policy Perspective
Moving teachers, parents and the public from what they know
and understand to something new - meaning of marks,
comparability (perception), trust
communications strategy getting the correct messages out
about standards and school based assessment
reduce the stakes
Reporting to parents using standards can standards serve the
dual purpose of student monitoring and reporting to parents
international issue
System accountability address the key issue facing systems
around the world that are reporting against standards: Is the
performance that we as a system are reporting correctly aligned
to standards?
validation of performance against standards

Challenges of School-based Assessment from
an Operational Perspective
Designing, articulating and implementing standards making
sure that they are achievable and not aspirational
linking the standards to the curriculum
helping the teachers internalise the standards
producing calibrated work samples (standards packages)
System wide monitoring
is there a need and how do you do it?
Teachers ability to write good quality, higher order items linked to
curriculum and performance standards
validity and reliability
System wide monitoring how do you do it? is there a need?
Validation how do you do it?
Interpreting Results Over Time
How should we interpret variation in the numbers
achieving the top grade over time?
Time series data often show incremental creep with
more students achieving the top levels of
performance each year.
This result then leads to debate about whether or not
standards are falling or whether the education system
itself is delivering some consistent improvement
(Wikstrom, 2005).

Assessment www.education.ox.ac.uk/assessment/
0
5
10
15
20
25
2000 2001 2002 2003 2004 2005 2006 2007 2008
%
t
o
p

g
r
a
d
e

Year
JCQ
SQA
NSW
QLD
Linear (JCQ)
Linear (SQA)
Linear (NSW)
Linear (QLD)
English Percentage Top Grade
Biology Percentage Top Grade
0
5
10
15
20
25
30
2000 2001 2002 2003 2004 2005 2006 2007 2008
%

t
o
p

g
r
a
d
e

Year
JCQ
SQA
NSW
QLD
Linear (JCQ)
Linear (SQA)
Linear (NSW)
Linear (QLD)
Media Lobbying By Head Masters
Media Lobbying By Head Masters
The Need to Validate
Clearly a fundamental question which arises in
education systems is how do we validate standards-
referenced results?
Public examination authorities are expected to
maintain standards.
There are a number of procedures and tools to
assist the process in systems where results are
referenced to standards.
They all ultimately rely on professional judgment to
some degree.
A Scenario
An education system introduces a new professional
development programme aimed at increasing student
outcomes.
The high stakes exit examination is reported with
respect to standards. Much effort is put into ensuring
that standards are being maintained.
Student performance on the exit examination
appears to be much improved.
Is it an easier examination, therefore cut-scores to
determine grades need to be raised?
or
Is it evidence that the professional development
programme is bearing fruit?
How would we know?
VET Courses 2001-2008
The next slide shows scaled academic achievement
scores averaged across VET courses to determine if
there has been any change in general ability of the
cohorts taking VET subjects over time.

The figure shows three points in the distributions : the
90
th
percentile (P90), the mean, and the 25
th
percentile
(P25). For each of these points scaled scores are quite
stable over time.
0
5
10
15
20
25
30
35
2001 2002 2003 2004 2005 2006 2007 2008
U
A
C

s
c
o
r
e
s

P90
Mean
P25
General Academic Ability of VET Candidature
Over Time
Percentage in Band 6 VET cf Other Courses
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
2001 2002 2003 2004 2005 2006 2007 2008
P
e
r
c
e
n
t
a
g
e

Other
VET
Percentage in Band 5 VET cf other courses
0.0
5.0
10.0
15.0
20.0
25.0
2001 2002 2003 2004 2005 2006 2007 2008
P
e
r
c
e
n
t
a
g
e

Other
VET
Some Reflective Questions
1. Are assessment outcomes significant in the accountability agenda
in your system?
2. To what extent is gaming an issue in your system?
3. Are issues of reliability and/or validity given prominence in public
debates about education in your system?
4. Has there been a discussion of validation of your reporting system?
5. Is the multiple meaning of standards a cause of confusion across
and within education systems? If yes, as educators how should we
deal with the confusion?
6. What role do international tests play in helping to define common
core standards?
7. Is there anything wrong with the statement Standards are rising or
standards are falling? If so, what?

Assessment
Developmental Continua

A Conceptual View of Standards
Developmental continuum for showing growth in an area of learning

Grade 4 Grade 5 Grade 6

Description Description Description

Description Description Description
A B C D F A B C D F


F
A B C
D
Progress Map for Bands (NSW)
Relationship between Curriculum and
Performance Standards?
What is the relationship between curriculum
standards and performance standards? This will
be considered in the context of a developmental
continuum as a major conceptual organiser of
curriculum and assessment.

Distinction between curriculum and performance
standards doesnt really apply in a competency
based education and training system where
there is no formal curriculum apart from
competency standards. In this context the
relationship between assessment and learning
can be described as assessment as learning or
learning through assessment.

Band 6 Biology Description
demonstrates an extensive and detailed knowledge and superior understanding of
biological concepts, including complex and abstract ideas
demonstrates an extensive understanding of the historical development of biological
concepts, their applications and implications for society and the environment, and the
future directions of biological research
communicates succinctly, logically and sequentially using a variety of scientific formats,
including diagrams, graphs, tables, flow charts and equations relating to biology
analyses and evaluates data effectively, identifying biological relationships, quantifying
explanations and descriptions, synthesising information to draw conclusions
uses precise biological terms extensively and correctly in a wide range of contexts
designs valid experimental processes using appropriate technologies and incorporating the
thorough knowledge of the use of a control, variables and repetition to solve biological
problems
applies knowledge and information to unfamiliar situations and designs an original solution
to a biological problem

Band 5 and 4 Biology Descriptions

Band 5
demonstrates thorough knowledge and understanding of most biological concepts
demonstrates a thorough understanding of the historical development of biological
concepts and their applications and implications for society and the environment
communicates effectively in a variety of scientific formats including diagrams, graphs,
tables, flow charts and equations relating to biology
explains qualitative and quantitative biological relationships and ideas coherently; identifies
patterns in data to draw conclusions
uses precise biological terms frequently and correctly in a range of contexts
identifies the correct application of scientific experimental methodology to solve biological
problems

Band 4
demonstrates sound knowledge and clear understanding of some biological concepts
demonstrates a sound understanding of the historical development of biological concepts
and their applications for society and the environment
communicates using clear written expression and incorporating diagrams of biological
structures
provides qualitative and quantitative descriptions of biological phenomena and explains
straightforward biological relationships
uses general biological terms frequently and correctly in a range of contexts
identifies the correct components of the experimental scientific method in biology

Performance Achievement Levels or Bands

Malaysia Standards Referencing and School Assessment (Adj)

Загружено:

Сведения о документе

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Malaysia Standards Referencing and School Assessment (Adj)

Загружено:

Авторское право:

Доступные форматы

Jim Tognolini

International Experience on Standards

Вам также может понравиться