Академический Документы
Профессиональный Документы
Культура Документы
This pamphlet results from the Assessment Systems for the Future project, funded by the
Nuffield Foundation. The project was set up by the Assessment Reform Group in
September 2003 to consider evidence from research and practice about the summative
assessment of school pupils, and to propose ways in which such assessment can benefit
their education.
The role that assessment by teachers can take in summative assessment was the
project’s particular focus.
The project was directed by Wynne Harlen and based at the Faculty of Education,
University of Cambridge. On the final page is a list of members of the project’s Core
Group, which has overseen all the projects activities and publications. Further
information about the project and the Assessment Reform Group’s other activities and
publications can be found at www.assessment-reform-group.org
Summary
Assessment and testing have a strong effect on the lives and careers of young people.
Decisions taken within and by schools influence the prospects and opportunities of their pupils
and of even greater importance are their results of national tests and examinations. When the
results of tests and examinations are used to pass judgments on teachers and schools, they also
affect the ways in which pupils are taught. Given their importance, it is essential that results of
summative assessment should reflect and influence school learning in the best possible way.
Further, to avoid the negative consequences of using high stakes summative assessment to
evaluate teachers and schools it is argued that systems of school accountability should not rely
solely on the data derived from summative assessment of pupils and that the monitoring of
standards of pupils’ achievement should be derived from a wider base of evidence than test
results from individual pupils.
Implications for those responsible for making assessment policy and for those responsible for
implementing it are drawn out.
2
Summative assessment by teachers can be Although the words ‘pupils’ and ‘schools’ are
problematic, and is certainly no panacea, but used here, the arguments and implications
in many respects it is superior to an external have relevance for learners in other
test-based system. educational institutions.
Individual pupils:
Uses internal to the school/college – for keeping records and giving reports on progress to
other teachers, parents and pupils.
Uses external to the school/college – including certification, selection and meeting
statutory requirements.
Groups of pupils:
Evaluation – of teachers, schools and local authorities. The types of evidence gathered are
usually determined by national and local policy rather than by individual schools.
Monitoring – for year on year comparison of pupils’ average achievements at the regional
or system level. The procedures for doing this are also determined outside the school.
4
becomes particularly clear when one Summative assessment by teachers is the process
considers the qualities that effective by which teachers gather evidence in a planned
summative assessment should have. and systematic way in order to draw inferences
about their students’ learning, based on their
In common with assessment for other professional judgment, and to report at a
purposes, summative assessment should have particular time on their students’ achievements.
the following qualities:
Improving the reliability of
Validity: the assessment must cover all teachers’ summative assessment
aspects, and only those aspects, of pupils’
achievement relevant to a particular T IS FAIR to point out that the research
purpose.
Reliability: it should be designed so that
users can have confidence that the results
I identifying the deficiencies of teachers’
assessment comes from studies where no
attempt was made to prepare teachers for a
are sufficiently accurate and consistent major role in assessment. Other research4,
for their purpose. and experience from countries where
Impact: it should not only measure teachers’ assessment is used for external
performance but have desirable summative purposes, identifies the
consequences for teaching, learning and conditions that affect the dependability of
pupils’ motivation for learning. the assessment and the steps that can be
Assessment generally has a strong impact taken to increase it.
on the curriculum and on pedagogy, so it
is vital that any adverse effects are The following five points were brought out
minimised. in consulting those with experience of
Practicability: the resources required to implementing summative assessment by
provide it – teachers’ time, expertise and teachers5 in the UK and in Australia and the
cost, and pupils’ learning time – should be United States (Queensland and California).
commensurate with the value of the The project has also considered how pupils
information for its users. working towards national vocational
qualifications are assessed in the workplace
How successfully does summative or college6.
assessment by teachers meet these four
criteria? The table opposite weighs evidence
1. Teachers should have clear criteria
for and against. However, before setting out
the conflicting points it is important to be describing levels of progress in various
clear about the meaning given here to aspects of achievement and, ideally, they
summative assessment by teachers. The 3 See Appendix on costs of assessment in Working
definition that emerged from discussions the Paper 3 on ARG website.
teachers and other professionals in education 4 Summarised in Working Paper 2 (ASF, 2004) on
restricts the meaning to situations where ARG website.
5 See report of seminars 2 and 3 and summary of the
teachers assess their own pupils: main points in Working Paper 1.
6 See report of seminar 5 and Working Paper 4.
5
Pros Cons
Validity
• Teachers can assess a wider range of achievement • The validity of teachers’ assessment depends on
and learning outcomes than formal tests and the learning activities and opportunities that
examinations. Teachers’ assessment can provide schoolwork provides.
information about learning processes as well as • Bureaucratic moderation procedures for quality
outcomes. assurance could constrain the operation of
• Freedom from test anxiety and from practice in teachers’ summative assessment so that only
test-taking means that assessment by teachers “safe” and routine approaches are used.
gives a more valid indication of pupils’
achievement.
Reliability
• With appropriate training and moderation • Teachers’ assessment is often perceived as being,
teachers’ assessment can reach satisfactory levels of and indeed can be, unreliable and biased due to
reliability. varying standards being applied.
• There are many examples in other countries and in • Some external tests or tasks may still be needed to
further and higher education of teachers making supplement teachers’ judgments.
crucial summative assessments of pupils’
performance.
Impact
• Teachers have greater freedom to pursue learning • Public confidence in the system may be low due to
goals in ways that suit their pupils. teachers’ assessment being perceived as inferior
• When teachers are gathering evidence from pupils’ to external tests, particularly for children aged 11
on-going work, information can be used and over.
formatively, to help learning, as well as for
summative purposes.
• Moderation procedures provide valuable
professional development for teachers
• Pupils can share in the process through self-
assessment and derive a sense of progress towards
“learning goals” as distinct from “performance
goals”.
Practicability
• Financial resources are released at the school level • Responsibility for summative assessment can
by reducing the number of commercial tests increase the workload for schools and teachers.
purchased, which increased dramatically when • Teachers can find that the process of moderating
national testing was introduced and used to judge their judgments is time-consuming.
schools. • Training in the interpretation and use of
• Teachers can spend more time teaching rather than assessment criteria is needed.
preparing for and marking tests.
• Pupils’ learning time is increased by at least two
weeks per year by using classroom work rather
than tests to assess progress3.
6
should help to develop these criteria. As use, if needed, to ensure that all intended
well as providing a common basis for goals are taken into account in their
interpretation of evidence, such criteria assessment.
should also spell out the learning 5. Where teachers and users of summative
opportunities that are required. This assessment have become dependent upon
makes it easier for teachers to assess pupils external tests it will take time to increase
dependably on the basis of regular teachers’ competence in using on-going
classroom work. assessment and to build confidence in
2. Professional development is needed so that teachers’ judgments. It is important that
teachers follow procedures that assure all involved have time to trial and
dependability. Training should also focus evaluate new practices and to be clear
on the criteria to be applied and the about the procedures and the safeguards
sources of potential bias that have been that are built in to protect dependability.
revealed by research.
3. A system of moderation of teachers’
Using tests for summative
judgments through professional
collaboration benefits teaching and
purposes: some problems
learning as well as assessment. Moderation T IS CLEAR that using teachers’
that affects the planning and
implementation of assessment, and
consequently teachers’ understanding of
I assessment for summative purposes is
not without its problems, some of which
are shared by any procedure for summative
learning goals and of the criteria assessment, particularly when the result is
indicating progress towards them, has used for external high stakes purposes.
more than a quality assurance function. However, these problems should be set
4. The provision of a bank of well-designed against those caused by the alternative of
tasks, with marking criteria, can do more depending on tests.
than help teachers to make judgements
about their pupils’ achievements. Such There is ample evidence that a system based
tasks exemplify activities through which on tests is flawed for the following reasons:
pupils can work towards important goals,
such as critical reasoning and the • It fails to provide information about the
application of knowledge in new full range of educational outcomes that
situations. As assessment tasks they can are needed in a world of rapid social and
provide pupils with interesting and technological change and therefore does
relevant learning experiences. They should not encourage the development of these
not be allowed to dominate the assessment skills. These outcomes include higher-
process and certainly should not be seen as order thinking skills, the ability to adapt
separate measures to be set beside teachers’ to changing circumstances, the
judgments. Nor are they intended to understanding of how to learn, and the
confirm teachers’ judgments. Rather they ability to work and learn collaboratively in
are part of the evidence that teachers can groups as well as independently.
7
• It inhibits the development of formative individual pupil test results for a range of
assessment (or assessment for learning) purposes, from target setting to league
which is proven to raise achievement tables, is too simplistic. Information
levels and reduce the gap between higher gathered for one purpose does not
and lower achieving pupils. necessarily serve others, nor do the methods
used to collect evidence of some types of
• The data it provides are less reliable than learning suit all. The negative consequences
they are generally thought to be. For
example it has been estimated that the key of summative assessment may be minimised
stage (KS) tests in England result in the by giving teachers a greater role in assessing
wrong levels for at least a third of pupils at individual pupils9 and using different
the end of KS2 and up to 40 per cent at approaches for evaluating and monitoring
the end of KS37. teacher and school performance, as discussed
later.
The weak reliability of tests means that
• unfair and incorrect decisions will be
Before considering the implications of this
made about some pupils, affecting their
course of action, it is important to
progress both within and between
schools8 and beyond school. substantiate the claims that much
summative assessment does not measure
• There is no firm evidence to support the some key outcomes of modern education
claims that testing boosts standards of and that it is having a negative impact on
achievement. pupils, on the curriculum, on teaching and
on the use of assessment to help learning
• It reduces some pupils’ motivation for (AfL).
learning.
that this can result in pupils being taught to ‘students cannot learn in school everything
pass tests even when they do not have the they will need to know in adult life’13. School
skills that are supposedly being tested. A must therefore provide the skills,
study commissioned by the Department for understanding and desire needed for lifelong
Education and Skills (DfES) concluded that learning. Since what is assessed has a strong
while drilling 11-year-olds to pass national influence on what is taught and how it is
tests is likely to boost results it may not help taught, we must look critically at what is
pupils’ longer-term learning11. The narrow assessed. If the required outcomes are not
range of learning outcomes assessed by tests included, then alternative methods of
contrasts with the broad view of learning assessment are needed.
goals reflected in the DfES Every Child
Matters policy document12. The impact on pupils, teachers and
teaching
What ought to be assessed? As already emphasised, current testing
It is crucial that assessment covers the regimes have considerable consequences for
learning that will be essential for young pupils’ motivation and for their learning
people who will live and work in a rapidly experiences. Research14 indicates that
shrinking world and changing society. Two
key sets of goals in any subject are: • valued
test performance can become more highly
than what is being learned;
The second set of goals relates to the These negative effects can operate both
development of awareness of the process of directly in the preparation of pupils for
learning. It is widely recognised that
13 OECD, 1999, Measuring Student Knowledge and
11 http://www.dfes.gov.uk/research/programmeof Skills. OECD Programme for International Student
research/projectinformation.cfm?projectId=14390&t Assessment (PISA). Paris: OECD, p9.
14 See Testing, Motivation and Learning (2002),
ype=5&resultspage=21.
12 DfES 2004 see www.everychildmatters.gov.uk available on the ARG website.
9
which a wide range of evidence is judgments and a further similar period for
collected for small samples of pupils. dissemination to users and training of
teachers’.
assessment policy in schools and colleges, for • Set up a system of sampling pupils’
advisers and inspectors and for teacher performance for national monitoring,
educators. Whilst implementers are thereby reducing the overall test burden
dependent on policy decisions, it is also the whilst increasing the breadth and
relevance of the evidence.
case that changes require understanding and
appropriate action by those who must
operate them in practice, particular, as here, Implications for school management
where trust and responsibility are involved. • Establish a school policy for assessment
There is, therefore, no hierarchy in the that supports assessment for learning at
implications for policy-makers and policy all times and requires summative
implementers set out below. assessment only when necessary for
checking and reporting progress.
Implications for national and local • Arrange quality assurance of all
policy-makers summative assessment, including any tests
• Recognise that the financial and time given by teachers, so that decision made
burdens at national and school levels of within a school about the progress of
current summative assessment policies pupils are based on dependable
based on testing are not justified by the information.
value of the information gained.
• Ensure that parents understand how
• Replace national testing, where it exists, assessment is helping learning and how
by a requirement for reporting moderated criteria are used in reporting progress at
teachers’ judgments of pupil performance, given times during the year.
and divert some of the time and money
saved into quality assurance that enhances • Resist pressure for “hard” data from tests
teaching and learning. and encourage use of a range of types of
evidence of pupils’ learning.
• Review the role of teacher assessment in
examinations for 16 and 18-year-olds. • Provide protected time for quality
assurance of teachers’ assessment through
• Promote open discussion of why and how moderation.
changes in the system are being made.
Implications for teachers
• Allow at least two years for the trial and
evaluation of any new summative • Ensure that assessment is always used to
assessment system based on teachers’ help learningand that, when a summative
14
assessment report is needed, the best and the extent to which they benefit
evidence is reliably judged against relevant teaching and learning.
criteria.
• Help schools to develop action plans
based on self-evaluation across a range of
• going
Involve pupils in self-assessment of on-
work and help them to understand measures rather than only on levels
the criteria used in assessing their work achieved by pupils.
for reporting purposes and how
summative judgments are made. Implications for initial and professional
development course providers
• Take part in moderation of summative
judgments and other quality assurance • Ensure that courses allow adequate time
procedures. for
o discussion of the different purposes of
• Use tests only when most appropriate, not assessment and the uses made of
as routine.
assessment data;
Implications for inspectors and advisers o trainees and participants to identify,
sample and evaluate different ways of
• Review school policies and practices to gathering evidence of pupils’
ensure that assessment is being used performance;
formatively and is not overshadowed by o giving experience of generating
summative tests and tasks. assessment criteria linked to specific
learning goals;
• ofEncourage the use of a range of evidence
pupils’ achievements. o considering evidence of bias and other
sources of error in assessment and how
• Ensure that continuing professional they can be minimised.
development in assessment is available for
those who require it.
Members of the Assessment Systems for the Future project Core Group
Mr David Bartlett Co-ordinator for Assessment, Birmingham City Council Children and
Young Person Services
Professor Paul Black King’s College, University of London
Professor Richard Daugherty University of Wales, Aberystwyth
Dr Kathryn Ecclestone University of Exeter
Ms Janet English Head Teacher, Malvern Way Infant and Nursery Schools, Croxley Green,
Hertfordshire
Professor John Gardner Queen’s University, Belfast
Professor Wynne Harlen University of Bristol, ASF project director
Ms Carolyn Hutchinson Head of Assessment Branch, Scottish Executive Education
Department
Professor Mary James Institute of Education, University of London
Mr Martin Montgomery Assessment Development Manager, Council for the Curriculum,
Examinations and Assessment, Northern Ireland
Dr Paul Newton Principal Assessment Researcher, Regulation and Standards Division,
Qualifications and Curriculum Authority
Dr Catrin Roberts Assistant Director, Nuffield Foundation
Mr Jon Ryder Teacher, Lord Williams’s School, Thame, Oxfordshire
Professor Judy Sebba University of Sussex
Dr Gordon Stobart Institute of Education, University of London
Ms Anne Whipp Curriculum and Teacher Assessment Manager, ACCAC
(the Qualifications, Curriculum and Assessment Authority for Wales).
Further copies can be obtained from:
CPA Office
Institute of Education
University of London
20 Bedford Way
London
WC1H 0AL