Implementing e-Assessment and Building an e-Assessment Centre at

Edinburghs Telford College

Summary
This case study details how Edinburghs Telford College have embedded the use of
e-Assessment across the college through the development of a dedicated e-
Assessment Centre, staff development and use of e-Assessment authoring tools and
delivery platforms.

About the Institution
Edinburghs Telford College serves the education and training needs of 20,000
students, industry and the wider community and work in partnership with a range of
local, national and international organisations. Gavin Lang is the Qualifications and
Standards Manager and has a lead role in establishing and promoting the use of
online learning resources and e-Assessment across the College.

The Challenge
The College conducted a college wide review and evaluation of learning and
teaching practice through the Learning and Teaching @ETC project and asked all
curriculum teams if they would like to incorporate some element of e-Assessment or
new technology into the delivery of their course. Based on this, a review of action
plans from curriculum teams and the Colleges Self-Assessment Report and Quality
Improvement Plan, which highlighted key areas for improvement, a plan was formed
to embed e-Assessment in the curriculum and create a purpose built centre for
delivery.

Working closely with interested lecturers, the e-Assessment team helped them to
embed the use of e-Assessment in their courses in a variety of formats, including
pre-course, diagnostic, formative and summative testing. However, the team
encountered anxiety amongst some lecturers in moving into new ways of teaching
and assessing and security concerns about assessing online.
It was clear from requests from staff, that they were interested in more than the
summative assessments available through external awarding bodies, so the team
considered purchasing an e-Assessment authoring tool and delivery platform.

The Activity
Based on the evidence gathered from
the Learning and Teaching @ETC
project, the decision was taken to
build a dedicated e-Assessment
Centre to provide a specific space for
e-Assessment to be undertaken and
to help address security fears. The e-
Assessment team also reviewed a
number of e-Assessment authoring
tools and delivery engines and
selected Questionmark Perception,
as the tool that most met their needs
at the time.

The e-Assessment Centre was designed and built to BSI standards and the team
used guidance from external awarding bodies, finding the SQA and City and Guilds
specifications particularly useful. Considerations included optimal room layouts,
forms to be used and the rules and regulations to be applied. The team worked
through external awarding body regulations, such as those from the SQA, C&G,
Microsoft, Association of Accounting Technicians and VTCT. They also worked
closely with the SQA during the implementation to ensure they were comfortable
with the college moving to online tests. No significant issues were raised and the
SQA gave approval to change from paper to online assessment as long as there were
no differences between the tests.

Software had to be installed on the
computers in the e-Assessment Centre to
enable access to assessment engines such
as GOLA, Pearson Vue, SAGE, AAT, VTCT,
Tribal. Web based systems such as CISCO
and TEST VQ could also be accessed
online. To ensure the security and
integrity of e-Assessments hosted online,
the team installed secure browsers for
Questionmark Perception, AAT and SQA,
as well as browse control and iTALC to give additional functionality to control the
computers in the centre e.g. blocking applications, CD drives, USB drives, use of
calculators and limiting available websites to prevent unauthorised use of the
internet. The team did encounter some issues initially when installing multiple
pieces of software on each computer, as there were clashes between the software
and secure browsers, however, the relevant companies fixed the issues.

The process for creating e-Assessments involves staff giving the e-Assessment team
questions and the team creating them using Questionmark Perception or other
relevant authoring tool. Staff can direct students to Questionmark Perception to
prepare for summative assessments, NQ Higher NAB revision, and, given that it has
been integrated with Moodle, for accessing grades for summative assessments.
Turnitin can also be accessed via Moodle and Moodle has become part of the
provision that enables 24/7 access to e-Assessment opportunities from both inside
and outwith the college. The e-Assessment team plan to create an area in Moodle
to provide support information to staff and will include case studies to showcase
examples of best practice.

When students use the centre, they
enter a holding room where they are
registered and receive a number
which links to the computer they will
sit at and a storage box where they
can store their personal possessions,
such as mobile phones, which they
are not allowed to take in to
summative exams. A map in the
holding room shows the number of
the computers so students know
where to go. The e-Assessment team
provides invigilation duties but there is also a room checklist for staff using the
centre for assessment, which provides tips such as checking that all the mice are
working or that it is not too hot or cold and there are also issue sheets where tutors
can document any problems.

To encourage the highest possible use of e-Assessments , the e-Assessment team
has undertaken various promotional activities such as drop ins, fliers, direct
marketing, managers meetings and SQA SOLAR updates.
The Outcomes
Over 10,000 learners have passed through the e-Assessment centre since November
2009. 4500 questions have been authored for over 100 assessments, which have
been used for 402 sessions of pre course / diagnostic testing, 173 formative sessions
and 326 summative exams. The average utilization of the centre for session
2010/11 was 70%.

Anecdotal evidence from students is that they enjoy using the centre and e-
Assessment. Some have commented that they find e-Assessments easier than a
paper version of the same test, they like getting instant feedback and, in some
cases, immediate results and also believe this is a more environmentally friendly
option as it saves paper. One interesting comment related to an emotional
response to e-Assessment as opposed to paper based exams: Didnt panic as much
as you are not aware of other people writing down things and turning pages, if Im
aware of this it makes me think that Im not going fast enough and therefore I panic.
Dont get that feeling in the e-assessment centre.

Staff have generally reported positive reactions to the use of e-Assessment and the
centre, which include:
Dont have to read handwriting
Better presentation of questions
Improved accessibility
Instant scores and feedback
Coaching report generated by Questionmark Perception which provides
information on student progress and allows them to identify at an early stage
key areas to work on in class
Students can complete tests multiple times (formative only) which is an
effective revision tool
Confident the tests are undertaken in a controlled environment and that they
are moving with the times
Reduction in marking and invigilation time

Staff have found that the upfront workload involved in writing questions and
feedback is time consuming, however, and no controlled studies have been carried
out yet to test whether this initial investment of time, is balanced out by less input
in future years due to the opportunity to use the same tests multiple times, for self-
testing as revision without the need to re-distribute to students, etc. Despite some
concerns about the upfront time involved, many staff have reported that they would
like to write e-assessments for every course learning outcome.

The staff and the institution have been pleased by the accessibility benefits of e-
Assessment, as the tests can be presented to meet individual requirements. The
institution has also realized reductions in the use of paper, printing costs and offsite
storage. These savings are balanced against the cost of subscriptions to 3
rd
party e-
Assessment tools and delivery engines. The College acknowledges that they were
fortunate to have access to capital when they set up the centre.

Lessons Learnt
The e-Assessment team have received
numerous requests to use the e-
Assessment centre for non-on-line / e-
assessment testing (so that staff can
access the 31 available computers). The
decision was taken to not allow this and
although this felt difficult to justify, the
team believe it was the right decision.
Given that statistics show the average
use for session 2010/11 was 70%, there
is little downtime.

Some things the team might have changed, however, include the use of screen
filters as well as separation boards and a different design and layout of the centre.
The centre currently has a very linear layout which does not lend itself to more
informal assessment which might include group work, for example. They would also
like to have lockable storage units although this would have increased costs.

Useful Links

Edinburghs Telford College: http://www.ed-coll.ac.uk/

JISC RSC Scotland e-Assessment resources:
http://www.rsc-scotland.ac.uk/eAssessment/eAssessment.htm#resources

SQA guide to effective practice: http://www.sqa.org.uk/sqa/23369.2254.html

SQA e-Assessment Resource: http://www.sqa.org.uk/ear

The e-Assessment team would welcome visits from other FE institutions interested
in having a tour of the centre or to discuss in more detail how they have embedded
the use of e-Assessment in the curriculum.

Disclaimer - The Regional Support Centre Scotland supports the development of
educational e-Learning. We may refer to specific products, processes or services.
Such references are examples and are not endorsements or recommendations and
should not be used for product endorsement purposes.