Evaluation

Of
Training and Development
DEFINITION
Evaluation is a process to determine the
relevance ,effectiveness, and impact of activities
in light of their objectives.
Training evaluation is the process of deep
examination and analysis of
Selecting measures
Gathering information based on those
measures
Comparing what participant learn to some
standard ,goal or expectation

DEFINITION
Training evaluation is a systematic process of
collecting information for and about a training
activity which can then be used for guiding decision
making and for assessing the relevance and
effectiveness of various training components.
TYPES OF EVALUATION

Formative Evaluation
Summative Evaluation
FORMATIVE E VALUATION
It involves the collection of relevant and useful data
while the training and development is being
conducted.
This information can identify drawbacks and
unintended outcomes and is helpful in revising the
plan and structure of training and development
programmes to suit needs of the situation.
SUMMATIVE EVALUATION
It is done at the end of the programme and makes
an overall assessment of its effectiveness in
relation to achieving the objectives and goals.
TYPES OF EVALUATION
Evaluation for Planning provides information with
planning decision are made. Methods ,Materials
,Instructional aids, strategies are planned.
Process Evaluation Conducted to detect defects
in the procedural design of training activity during
the implementation phase. Periodically conducted
TYPES OF EVALUATION
Terminal Evaluation Conducted to find out the
effectiveness of training and development after it is
completed.
Impact Evaluation Assesses changes in on-the-
job behaviour as a result of training and
development efforts.
Evaluation
Stakeholders
Training Organiser
Trainers
Participants
Curriculum developers
Training and
Development
Increased skill

Increased Knowledge

Increased awareness

Improved work
Performance

Improved
quality of
Service

Increased
Revenues
Leads to
Leads to
Reaction
Learning
Evaluation
Application
Cost benefit
Evaluation
OBJECTIVES OF EVALUATION
To validate training and development as a business
tool.
To justify the cost incurred in training and
development.
To help improve the design of training and
development programme
To help in selecting training and development
methods
OBJECTIVES OF EVALUATION
To determine strength and weakness of training and
development.
To determine success in accomplishing programme
objectives.
To test clarity and validity of methods used.
To reinforce major points.
To decide who should participate in future.

OBJECTIVES OF EVALUATION
To determine whether the programme was the appropriate
solution for the specified training.
To establish a database that can assist management in
making decisions.
PURPOSE OF EVALUATION
At Macro level

At Micro Level
AT MACRO LEVEL
To assess training and development programme's
business impact as a whole.
To facilitate selection of training and development
programme's performance matrices related to
organizations goals and objectives.
To get at the business impact measure through
direct correlation to organisational goals and
objectives.
AT MICRO LEVEL
To measure a specific training and development
programmes business impact
To combine this to overall programme business
impact.
STRATEGIES TO EVALUATION
Evaluation design must be formed from strategic
perspective.
Designing a good evaluation effort involves when to
collect evaluation measures and which group to
collect them from.

STRATEGIES TO EVALUATION
Compare the trainees after the training with the way
they were before the training.
Compare the learning ,behaviour or result of the
trained group with that has not been trained.
EVALUATION DESIGN
One-Shot Posttest Only Design
One-Group Pretest - Posttest Design
Multiple Baseline Design
Pretest-Posttest Control Group Design
ONE-SHOT POSTTEST ONLY DESIGN

Training evaluation measures are collected only
from the trained group
There is no pertaining measure and no untrained
group for the purpose of comparison.
Train
Measure
ONE-GROUP PRETEST - POSTTEST
DESIGN
Training group is assessed both before and after
the training programme.
Helps to assess change from one time period to
another.
Train
Measure Measure
MULTIPLE BASELINE DESIGN

Trainer measures the group several times before
and after the training.
It helps to observe trends in performance and
change in trends
The design is the best one to use if all employees
are to be trained simultaneously.
Measure Measure Train Measure Measure Measure Measure
PRETEST-POSTTEST CONTROL GROUP
DESIGN

A control group of employees who are very similar
to the training group except that they do not receive
the training for last one year.
Both group are measured at least once before and
after the training.
It helps to analyse whether any change had
occurred and whether the change had resulted from
the training.
Train
Measure Measure
Measure Measure
No
Training
Group 1
Group 2
Models of Training Evaluation
MODELS
Kirkpatrick Four Level Approach
Kaufmans Five Level Approach
CIRO Approach
Philips Five Level ROI Framework
KIRKPATRICK FOUR LEVEL
APPROACH

Donald Kirkpatrick developed in 1959.
Four Levels
Level 1: Reaction - a measure of satisfaction
Level 2 : Learning - a measure of learning
Level 3 : a measure of behavioural change
Level 4 : Results a measure of results
REACTION
Simply assess the participants reaction to training and
programme.
Did they like the programme, the trainer and course
material.
Did they feel that they had learned something
Approach helps assesses the perceived value of
participants
Survey ,simple evaluation sheet (smile sheets, happy
sheets)
LEARNING
Learning objectives are met or not
Measuring the extend which principles ,facts,
techniques ,skills have been acquired
Pretest and posttest and score compilation
BEHAVIOUR
New knowledge and skill are retained
Skills and Knowledge are applied in job for
improved behaviour
Direct Observation at specific intervals
Behavioural scorecards

RESULTS
Business Impact
Pretest-Posttest Control Group Design

Level Measure Key
question
Indicators/Metho
dologies
Limitations
Reaction Satisfaction What was
the
participants
reaction to
the
programme
?
Programme
evaluation
sheets,interview
s,questionnaire
and general
participants
comments
throughout the
programme
Cant measure
what is
learned,gurantee
behavioural
change, nor
ensure quality
result from
learning
Learning Knowledge What did the
participant
learn
Pre-post
testing,obseravti
ons,focus
groups,interview
s,survey people
who observe the
performer,
questionnaires
Will not measure
the likeness of
programme,if
they will behave
differently,and if
results will be
achieved
Level Measure Key
question
Indicators/Metho
dologies
Limitations
Behaviour Transfer of
learning and
achievement
performance
objectives
Did the
participants
learning
affect their
behaviour
Pre-post
testing,observati
ons by
trainers,manager
sand peers,team
and self
assessment,inter
views and
surveys
Cant determine if
participants liked
the training and if
the behaviour
accomplished
results
Results Transfer or
impact on
workplace
Did
participants
behaviour
changes
affect the
organisation
Productivity
,profitability,
sales and profit,
decreased
employee
turnover and
cost
Cannot
determine if
participants like
the training
,understood it or
if it affected their
previous
behaviour
SOME EXAMPLES
Sales Training measure change in sales volume,
customer retention, length of sales cycle,
profitability on each sale after training
Technical training reduction in calls to help desk,
reduced time to complete report, forms, or tasks or
improved use of software or systems
SOME EXAMPLES
Quality training reduction in number of defect
items
Safety training reduction in number or severity of
accidents

Level Evaluation Focus
1b.Reaction Methods, means and
process acceptability
and efficiency
1a.Enabling Availability and quality of
human, financial and
physical resource input
2.Acqusition Individual and small
mastery and
competency
KAUFMAN'S FIVE -LEVEL
Level Evaluation Focus
5.Societal outcomes Societal and client
responsiveness,
consequences and
payoffs.
4.Organisational output Organisational
contributions and
payoffs
3.Application Individual and small
group (products)
utilisation within the
organisation
KAUFMAN'S FIVE -LEVEL
CIRO APPROACH
Developed by Warr , Bird, Racham
C- Context evaluation
I Input evaluation
R Reaction Evaluation
O Outcome Evaluation
CONTEXT EVALUATION

involves obtaining and using information about the
current operational situation or context to determine
training needs and objectives
Ultimate objectives ( particular deficiency in the
organisation will eliminate or overcome)
Intermediate objectives ( the changes in employees
work behaviour that will be necessary for the ultimate
objectives to be attained)
Immediate objective ( the new skill ,knowledge or
attitude that employees must acquire to change their
behaviour and reach the immediate objective)
INPUT EVALUATION
Obtaining and using information about possible
training resources to choose between alternative
inputs to training.
Assessing the resource available
REACTION EVALUATION
Participants reaction to improve the HRD process
OUTCOME EVALUATION
Four stages for outcome evaluation
Defining trend objectives
Selecting or constructing some measures of
those objectives
Make the measurement at appropriate time
Assessing the results and using them to improve
later.
PHILIPS FIVE LEVEL ROI
FRAMEWORK
Reaction measuring the participants reaction to
the programme and outlines specific plans for
implementation
Learning Measure skills, Knowledge ,attitude
changes
Job Applications Measuring the change in
behaviour on the job and specific application of the
training material
PHILIPS FIVE LEVEL ROI
FRAMEWORK
Business Results Output, quality ,costs, time and
customer satisfaction
ROI Comparing the monetary benefits from the
programme with its costs.
ROI
Return on investment is the order of the day.
ROI is expressed as a percent of cost benefit ratio
% ROI = Benefits/Costs * 100
Another approach is pay back calculation how
much time it will take before the benefits of the
training match the costs and the training pay for
itself.
ROI
Pay back Period = Costs/monthly benefits
TRAINING COSTS
Design and Development Costs
Promotional costs
Administration costs
Faculty costs
Material costs
Cost of facilities
Participation costs
Evaluation Costs
TRAINING BENEFITS
Labour Savings
Productivity Increases
Other cost savings
Other income generations
Duration of Training
33hrs
Estimated number of
Participants
750
Period over which benefits are
calculated
12 months
Costs
Design and development
Rs 40,930
Promotion
Rs 4,744
Administration
Rs 12,713
Faculty
Rs 86,250
Materials

15,000
Facilities
40,500
Participation
553,156
Evaluation
872
Total Costs 754,165
Benefits
Labour Savings
241,071
Productivity Increases
675,000
Other cost savings
161,250
Other income generation
0
Total benefits 1,077,321
Return on Investment 143%
Payback period 8 months
Input
Resources
Immediate
Outcomes
Participants Skill
And
Knowledge
Increase
Intermediate
Outcomes
Participants
Use their skills
And knowledge
In their
Workplace
Context
Ultimate
Outcomes
Participants are
More effective
Or efficient
In their
job
Activities
Deliver Training
Output
Training Sessions
Effectiveness and Economy of Training
TRAINING DELIVERABILITY
Economy
Quality
Convenience
ECONOMY

Lower upfront implementation costs
Reduced content development costs
Low managing costs
Reduce time away from job
Reduce travel costs

QUALITY
Consistency
Blended Learning
Employee empowerment
Qualification and standard based
Enhanced Loyalty and commitment.
CONVENIENCE
Automatic generation of Training records
Flexibility, Stability
Learning fit with personal goals
Personal support
Tailored Training resources
CRITERIA OF SUCCESS
Numbers
Direct costs
Indirect costs
Efficiency
Performance to schedule
Income received
The extend to which trainees mix

CRITERIA OF SUCCESS
Reactions
Learning
Behavioural change
Performance change