Supporting your team:

Guidance for principal examiners and

team leaders
Version: May 16
Welcome to the session

Thank you for supporting the IB mission by acting as a senior examiner for the May 2016 session.
Aspects of senior examiner roles have changed since the introduction of eMarking. The emphasis is
now on collaboration and consensus building in standard setting, and clear communication of this
marking standard to assistant examiners so that they can understand and apply the same marking
standard. This document is aimed at helping you to confidently fulfil an eMarking principal or team
leader role. As senior examiners you are in a unique position to influence examiner behaviour.

Subject expertise and reliable application of markschemes remain critical to the senior examiner
roles. We have also learned that the quality of advice and feedback you provide to your team
members is critical to the successful completion of marking.

The shift to eMarking has not been without its difficulties but we see significant improvement
between successive sessions, as the technology recedes and refinements are made in light of
experience. This document aims to address one of the needs we have identified to assists senior
examiners and team leaders in particular with their mentoring role. We hope that with your ongoing
commitment, and with this guidance, we can continue to improve the quality and facility of
eMarking.

Thank you again and best wishes for the session.

Richard Penrose

Head of Diploma Programme Assessment

Contents

The purpose of this booklet

Other documents available

An overview of the session

Outline of roles

The senior examiner pool (approval management)

Exemplar communications
The Purpose of this document
The purpose of this document is:
to help team leaders and principal examiners understand their role in supporting examiners
during the session
to provide some context in terms of the structure of the marking session during which this
work takes place
to provide guidance and examples of good support communication.

Other documents available

In addition to the marking instructions available on IBIS the following documents are available.
Please note that the RM Assessor documents at the bottom of the list are for reference only should
you need technical help or want to learn more about the functionality of RM Assessor.

Name of document Purpose of the document Notes

Standardization: All the information you should need to Sent by email to
Guidance for principal successfully carry out standardization including examiners who are
examiners and subject specific roles and deadlines. involved in the
standardization team standardization meeting
members

RM Assessor Examiner A detailed technical booklet explaining more of Available at IBIS > My
Guide* the marking functionality available in RM Marking > Emarking
Assessor. This should be used for reference only.
RM Assessor Principal A detailed technical booklet covering Available at IBIS > My
Examiner and Team standardization and the monitoring of teams. Marking > Emarking
Leader Guide* This should be used for reference only.

*Please note that if you contact our office for help with using RM Assessor on +44 29 2051 7114 or
by emailing emarking@ibo.org, our technical support team will refer to these documents.

1
An overview of the session
The following is a general outline for all subjects. Actual dates can vary due to the requirements of
the session or of particular marking teams. In particular, the start of group 3-6 script marking in
Spanish, French or German may be later than for scripts in English due to the time it takes to
reproduce and exemplify the marking standard in all response languages.

Date of The exam is sat by candidates, senior examiners prepare for the standardization
examination meeting by analysing and discussing the examination paper and markscheme.

4 UK The standardization meeting begins; the markscheme is amended in light of

working candidate responses; the principal examiner (assisted by other standardization team
days later members) produces the definitively 1 marked scripts for quality assuring the marking
of other examiners.

7 10 days The standardization meeting concludes; the markscheme is released and eMarking
after the begins. Team leaders are expected to go through qualification in the first two days
exam and are given priority in support; examiners then follow and should be encouraged
through qualification within five days of the release of the markscheme.

Three Examiners should now have marked at least half of their marking target. Examiners
weeks after who have not made sufficient progress or have not responded to communications
the exam from the IB assessment centre may have their targets reduced.

Four weeks The deadline for mark entry can be viewed in RM Assessor, it is usually 28 days after
after the the exam date, to ensure sufficient marks are in IBIS in time for grade awarding.
exam

1
The marks given by the principal examiner and team are definitive in that they define the standard to which
all other examiners need to adjust their marking standards and against which grade boundaries will be set. The
marks are not definitive in the sense that no other mark could be reasonably defended or that they are the
only true or correct marks.
2
Outline of roles
Each examining team usually comprises a principal examiner, team leaders and examiners. Sufficient
team leaders are appointed to make teams of 6 10 examiners. Where the number of examiners is
such that the principal examiner can support them all directly, team leaders are not appointed.
The principal examiner
The principal examiner leads the standardization meeting, working closely with the subject manager.
The principal examiner defines and embodies the marking standard and ensures that all examiners
for the component understand and apply that marking standard. Standardization encompasses:
establishing the standard and approaches used in marking for the session with a group of
team leaders and/or examiners
amending the markscheme or marking notes to accommodate session specific issues and
real candidate answers
communicating the discussion, decisions and approaches agreed on in the standardization
meeting to all team leaders and to the examiners for whom they are directly responsible
production of definitively marked scripts, marked to the agreed standard and annotated for
use in RM Assessor.
The production of practice, qualification and seed scripts is a requirement of IB assessments quality
assurance model but does not achieve accurate and consistent marking in itself. Reliance on
definitively marked scripts without carefully establishing and communicating the standard, results in
disagreements about marks and examiners failing to qualify for marking. The principal examiner
should be consultative and consider all advice from the standardization team when establishing the
standard and producing marked and annotated scripts that define the standard. The principal
examiner should work closely with the subject manager to establish effective communication with
the standardization team and then with the rest of the examining team.
Once the set up of the component has been completed and the markscheme is released on IBIS, the
principal examiner should contact the team leaders with a message that can be cascaded to
examiners (see example at the end of the document). This communication informs examiners that
they can begin practice and qualification, and provides an opportunity to remind them of key
information. If there has been a change in approach of the marking team for example which has
been agreed with the subject manager this can be highlighted in the message by referring to the
markscheme or practice scripts in which this is displayed. The message must not duplicate or replace
the markscheme as a means to inform examiners of how to mark but it can highlight information
already expressed in the markscheme. Follow up of the message with a phone or Skype call is
especially reassuring for examiners.
Once marking is underway the principal examiner should remain in close contact with team leaders,
examiners and subject manager. They should be available for grade awarding, at risk marking and
for enquiry upon results marking following publication of results to candidates.

The team leader

A team leader is responsible for the support of examiners in his/her team. Examiners depend on
their team leader to help them understand and apply the marking standard defined by the principal
examiner and standardisation team. Through good communication, explanation and discussion the
team leader effectively conveys the marking standard established at the standardisation meeting.
In addition to this general supporting role the following specific tasks are expected of team leaders.
1. Contact team members about 1 week before the examinations
This communication should be used to:

3
 Welcome examiners to the team and the session
Ensure lines of communication are open
Create an expectation of caring support of team members by the team leader
Create a supportive professional environment
Promote confidence in the team members both in the use of the technology and the
qualification/seeding process
Encourage participation in RM Assessor familiarization and simulation modes
Encourage engagement with the practice scripts when they become available
Encourage team members to discuss the practice scripts in detail with the team leader
before attempting qualification. Examiners must clarify any points of doubt that result
from the analysis of the markscheme and practice scripts.
To alert examiners that the automated message from RM Assessor when a qualification
or seed script is marked outside of tolerance can seem harsh, and in that scenario, team
leaders are there to support and guide. All interactions with examiners in the marking
session should be viewed as useful training opportunities to get the examiner back on
track.
To establish expectations for the date of the start of marking and the deadline for
completion of marking. For large entry subjects team leaders should emphasize that
examiners can only receive support once their team leader has qualified for marking and
therefore team leader guidance is unlikely to be available for the first two days after
release of the markscheme, practice and qualification scripts. Examiner teams should be
encouraged to qualify within the first five days of marking.

2. When requested by the subject manager, to participate in some or all of the following
standardization processes:
Standardization discussions prior to the main meeting by reviewing scripts with
reference to the markscheme, identifying issues that are not covered sufficiently by
the markscheme.
Supporting the principal examiner in the production of practice, qualification and
seed scripts and in communications to all examiners.
To ensure all team leaders can confidently support their teams, it is essential that they are
actively engaged with the principal examiner, either in the above processes, or where the
number of team leaders is large, through discussion of the outcomes of standardization.
3. Communicate again with the team when the markscheme is released.
This communication should include any advice sent from the principal examiner about
marking in the session as well as to remind examiners of the main points from the first
communication.
Team leaders who are not members of the standardization team should qualify for marking
promptly once the component is live on RM Assessor. By ensuring proper engagement with
the principal examiner this should become a straight-forward process and ensure that team
leaders are well placed to support the members of their team as they attempt qualification
and throughout the marking session.
4. Support team members when they query the markscheme.
Please encourage detailed discussion with examiners at this point. Just as our quality model
is front-loaded in terms of the work invested by senior examiners, it is anticipated that the
first week after the initial marking is underway will be the busiest time for team leaders also.
Encouraging your team to scrutinize the practice scripts and to take advantage of the

4
 training you can offer them at this stage will result in more examiners succeeding at
qualification and a smoother session overall.
5. Support team members who have marked qualification or seed scripts out of tolerance.
Feedback for examiners should be prompt, encouraging and friendly in tone. The emphasis
should be on specific discussion of what is required to award marks and on the principles of
positive marking (i.e. mark the work that is there, not look for things that are not) rather
than a general discussion of marks being too low or too high. Examiners who are told to
generally adjust their standard are likely to over-compensate; specific guidance is much
more effective. Reasons for specific marking differences can be uncovered through probing
questions over a number of email exchanges or a Skype conversation. A framework for
providing feedback is:
Start with a positive encouraging tone, note that for example... you got three out of
the five pretty much spot-on
Begin a discussion about the causes of disagreement such as can you explain to me
why you thought that answer fulfilled the criteria for a mark of 4? Or Are there any
parts of the markscheme and feedback for question 7 that you need help with or dont
understand?
Provide feedback including some detail focusing on the divergent marks. Ideally, the
analysis should refer directly to the script (... you say that sentence structure is poor,
but have a look at the final paragraph ...)
Give succinct general guidelines e.g. So, you need to ensure you are giving full credit
for explanations that involve one example only as directed in the markscheme, even
though the marks are more easily achieved with two....
Please try to maintain examiners engagement with the process. Keep communications
positive, informal but professional in style. Sympathize with the examiners but do not
discourage them by criticising the process. Under no circumstances should information be
supplied that undermines the quality model.
6. Be available for the whole of the marking period (i.e. up to the grade award meeting) but
particularly for the first week following markscheme release to ensure that examiners are
well supported through the qualification process.
7. Write a report about the marking of the examination paper, to send to the principal
examiner. The report should be based on your own observations about the paper and
candidate answers but should also represent the feedback of your examining team whose
reports are available to you in IBIS.
8. Be available to assist with at risk marking during and after the grade award meeting, and
for EUR re-marking (July September/Jan-March).

5
The senior examiner pool (approval management)
Examiners who do not demonstrate the appropriate marking standard during the first qualification
set or deviate from the standard during marking are suspended by RM Assessor and can only be
restarted by a team leader, provided that the team leader is confident that the examiners
misunderstanding has been addressed.
Suspended examiners are accessible, via the senior examiner pool, to all team leaders because
members of examining teams are likely to be working in different time zones, and an examiners
own team leader may be asleep when an examiner is suspended from marking. It is preferable for
an examiners own team leader to offer feedback if s/he is available. This helps on occasions when
an examiner is suspended for a second time. At this point, IB assessment centre staff will look at the
examiners marking, and may decide that they can be given a further opportunity. The examiners
own team leader is then asked to provide further feedback, and it is helpful if that team leader is
aware of any other feedback already provided.
If an examiner has been suspended from marking for more than 12 hours then another team leader
should step in and support the examiner in question, so that they can get back to marking promptly.
In this way, the team leaders can work together as a single team to support all examiners.
For technical guidance on the senior examiner pool and other aspects of approval management
please refer to the RM Assessor Principal Examiner and Team Leader Guide.

Exemplar communications
Example 1 Introduction to the session communication from the TL
Dear colleagues,

I would like to introduce myself as your team leader for the session. I look forward to working closely
with you over the coming weeks. Please be reminded that the examination paper is sat on May 12th,
standardization will be taking place during the 17th 20th and marking is planned to start on the 21st.
Please be aware that this date is contingent on many factors included scripts being available on RM
Assessor at the appropriate time and so there is a possibility of further delay.

Some of you will now be very experienced with using RM Assessor but for others this will be a new
experience, please take some time to familiarize yourself with the system before marking begins.
You can do this by logging in to RM Assessor over the next few weeks. After the 16th, scripts should
be available for you to practise marking in simulation mode. Before this time, RM Assessor will be
available in familiarization mode only.

Once standardization is complete the markscheme and the practice scripts will become available to
you. Please contact me if there is anything in the markscheme or practice scripts that needs
clarification. It is important that you study these carefully and only start qualification when you are
sure that you understand the agreed marking standard.

If while attempting to qualify, one of your scripts is found to have been marked out of tolerance, you
will receive an automatic message from RM Assessor. Please do not be too alarmed by this message;
I or another team leader will be in contact shortly after to help identify and address the differences
between your marking and that of the principal examiner. There may be further opportunities to
discuss the marking standard if seeds are marked out of tolerance, in both these scenarios please
6
note that the more contextual information you can give about your interpretation of the
markscheme, the better the quality of the feedback will be that we can offer you.

Finally, please remember that you should aim to qualify within the first week of marking and have
completed half of your marking target two weeks after that. The final date for submission of marks is
another week later (approximately 28 days after the exam date).

Please reply to this email to show that you have received and understand the information I have
provided. I wish you all the best of luck for the session!

Best wishes,

Team Leader

Example 2 Introduction to the session communication from the PE

Dear Colleagues,
Hello and welcome to the M16 eMarking session. The markscheme for Paper 1 has now been
released and we are ready to begin the process of e-marking.
The following documents are available on IBIS for your information and reference:
RM Assessor examiner guide
The examination paper and markscheme
Please ensure you read the markscheme fully including the General marking instructions and
subject details located at the start of the markscheme and pay careful attention to the markscheme
advice (notes in italics).
The practice scripts should also be carefully studied and any concerns you have about the marking
shown in these scripts should be discussed with your team leader. Only when you feel familiar with
the paper, the markscheme and the marking standard as displayed in the practice scripts should you
attempt qualification.
Please note that in both the markscheme and the practice scripts we have highlighted our new
agreed approach for dealing with questions where the candidates response only partially matches
the question set. Please see the notes in the markscheme and the practice scripts for questions 11
(b) and 12 (c) for examples.
Please contact your team leader in the first instance if you need any support. For technical support
and help with using RM Assessor please refer to the guides listed at the top of this email and then
the eMarking help team (emarking@ibo.org).
Good luck!
PE