GROUP 4

TRAINING Principles and Techniques in

EVALUATION
Human Resource Development
INTRODUCTION

Training effectiveness - the benefits that

the company and the trainees receive from
training.
Training outcomes or criteria - measures
that the trainer and the company use to
evaluate training programs.
Training evaluation - the process of
collecting the outcomes needed to
determine if training is effective.
Evaluation design - collection of
information, including whom, what, when,
and how, for determining the effectiveness
of the training program.

WOODGROVE
TRAINING
2
BANK
EVALUATION
REASONS FOR EVALUATING TRAINING

• Companies make large investments in

training and education and view them as
a strategy to be successful; they expect
the outcomes of training to be
measurable.
• Training evaluation provides the data
needed to demonstrate that training does
provide benefits to the company.
• It involves formative and summative
evaluation.

WOODGROVE
TRAINING
3
BANK
EVALUATION
TRAINING EVALUATION INVOLVES:

Formative evaluation - takes place Summative evaluation - determine the

during program design and extent to which trainees have changed
development. as a result of participating in the
training program.
 It helps ensure that the training  It may include measuring the monetary
program is well organized and runs benefits that the company receives from
smoothly, and trainees learn and are the program.
satisfied with the program.  It involves collecting quantitative data.
 It provides information about how to
make the program better; it involves
collecting qualitative data about the
program.
 Pilot testing - process of previewing
the training program with potential
trainees and managers or with other
customers.
WOODGROVE
TRAINING
4
BANK
EVALUATION
WHY SHOULD A TRAINING PROGRAM
BE EVALUATED?
 To identify the program’s strengths and weaknesses.
 To assess whether content, organization, and
administration of the program contribute to learning
and the use of training content on the job.
 To identify which trainees benefited most or least
from the program.
 To gather data to assist in marketing training
programs.
 To determine the financial benefits and costs of the
program.
 To compare the costs and benefits of:
 training versus non-training investments.
 different training programs to choose the best
program.

WOODGROVE
TRAINING
5
BANK
EVALUATION
THE EVALUATION PROCESS

Conduct a Needs Analysis

Develop Measurable Learning Objectives and

Analyze Transfer of Training

Develop Outcome Measures

Choose an Evaluation Strategy

Plan and Execute the Evaluation

WOODGROVE
TRAINING
6
BANK
EVALUATION
TRAINING OUTCOMES: KIRKPATRICK’S
FOUR-LEVEL FRAMEWORK OF
EVALUATION CRITERIA

Level Criteria Focus

4 Results Business results achieved by trainees
3 Behavior Improvement of behavior on the job
2 Learning Acquisition of knowledge, skills, attitudes, behavior
1 Reactions Trainee satisfaction

The hierarchical nature of Kirkpatrick’s framework suggests

that higher level outcomes should not be measured unless
positive changes occur in lower level outcomes.

The framework implies that changes at a higher level are more

beneficial than changes at a lower level.
WOODGROVE
TRAINING
7
BANK
EVALUATION
OUTCOMES USED IN EVALUATING
TRAINING PROGRAMS:
COGNITIVE OUTCOMES

SKILL-BASED OUTCOMES

AFFECTIVE OUTCOMES

RESULTS

RETURN ON INVESTMENT

WOODGROVE
TRAINING
8
BANK
EVALUATION
COGNITIVE OUTCOMES
 Determine the degree to which trainees are familiar
with principles, facts, techniques, procedures, or
processes emphasized in the training program
 Measure what knowledge trainees learned in the
program

SKILL-BASED OUTCOMES
 Assess the level of technical or motor skills
 Include acquisition or learning of skills and use of
skills on the job

WOODGROVE
TRAINING
9
BANK
EVALUATION
AFFECTIVE OUTCOMES
 Include attitudes and motivation
 Trainees’ perceptions of the program including the
facilities, trainers, and content

RESULTS
 Determine the training program’s payoff for the
company

RETURN ON INVESTMENT
 Comparing the training’s monetary benefits with the cost
of the training
 Direct costs - salaries and benefits for all employees involved
in training; program material and supplies; equipment or
classroom rentals or purchases; and travel costs.
 Indirect costs - not related directly to the design,
development, or delivery of the training program.
 Benefits - value that the company gains from the training
program.
WOODGROVE
TRAINING
10
BANK
EVALUATION
DETERMINING WHETHER OUTCOMES
ARE APPROPRIATE
Relevance The extent to which training outcomes are related to the learned
capabilities emphasized in the training program.
Criterion contamination - the extent that training outcomes
measure inappropriate capabilities or are affected by extraneous
conditions.
Criterion deficiency - the failure to measure training outcomes
that were emphasized in the training objectives.
Reliability The degree to which outcomes can be measured consistently over
time.
Discrimination The degree to which trainees’ performance on the outcome
actually reflects true differences in performance.
Practicality The ease with which the outcome measures can be collected.

Good outcomes are Relevant, Reliable,

Discriminative, and Practical
WOODGROVE
TRAINING
11
BANK
EVALUATION
CRITERION DEFICIENCY,
RELEVANCE, AND CONTAMINATION

Outcomes
Outcomes Outcomes Identified by
Measured Related to Needs
in Training Assessment
and Included
Evaluation Objectives
in Training
Objectives

CONTAMINATION RELEVANCE DEFICIENCY

WOODGROVE
TRAINING
12
BANK
EVALUATION
TRAINING EVALUATION PRACTICES

WOODGROVE
TRAINING
13
BANK
EVALUATION
TRAINING PROGRAM OBJECTIVES AND
THEIR IMPLICATIONS FOR EVALUATION

WOODGROVE
TRAINING
14
BANK
EVALUATION
EVALUATION DESIGNS:
THREATS TO VALIDITY
Threats to validity - factors that will lead Methods to Control for Threats to
an evaluator to question either the: Validity:

Internal validity - the believability of the • Pretests and Posttests

study results.
External validity - the extent to which the
evaluation results are generalizable to
other groups of trainees and situations.
• Use of Comparison Groups
• Random Assignment - assigning
employees to the training or comparison
group on the basis of chance.

THREATS TO INTERNAL VALIDITY THREATS TO EXTERNAL VALIDITY

Company Reaction to Pretest

Persons Reaction to Evaluation
Outcome Measures Interaction of Selection and Training
Interaction of Methods
WOODGROVE
TRAINING
15
BANK
EVALUATION
TYPES OF EVALUATION DESIGNS:

• Posttest only
• Pretest/Posttest
• Posttest only with Comparison Group
• Pretest/Posttest with Comparison
Group

• Time Series
• Time Series with Comparison Group
and Reversal
• Solomon Four-group

WOODGROVE
TRAINING
16
BANK
EVALUATION
COMPARISON OF EVALUATION
DESIGNS:

WOODGROVE
TRAINING
17
BANK
EVALUATION
FACTORS THAT INFLUENCE THE
TYPE OF EVALUATION DESIGN:

WOODGROVE
TRAINING
18
BANK
EVALUATION
CONSIDERATIONS IN CHOOSING AN
EVALUATION DESIGN:

1. The evaluation results can be used to change the program.

2. The training program is ongoing and has the potential to have an important
influence on (employees or customers).
3. The training program involves multiple classes and a large number of trainees.
4. Cost justification for training is based on numerical indicators. (Here the
company has a strong orientation toward evaluation.)
5. Trainers or others in the company have the expertise (or the budget to purchase
expertise from outside the company) to design and evaluate the data collected
from an evaluation study.
6. The cost of the training creates a need to show that it works.
7. There is sufficient time for conducting an evaluation. Here, information
regarding training effectiveness is not needed immediately.
8. There is interest in measuring change (in knowledge, behavior, skill, etc.) from
pretraining levels or in comparing two or more different programs.

WOODGROVE
TRAINING
19
BANK
EVALUATION
IMPORTANCE OF TRAINING COST
INFORMATION:

1. To understand total expenditures for training, including direct

and indirect costs.
2. To compare the costs of alternative training programs.
3. To evaluate the proportion of money spent on training
development, administration, and evaluation as well as to
compare monies spent on training for different groups of
employees.
4. To control costs.

WOODGROVE
TRAINING
20
BANK
EVALUATION
DETERMINING RETURN ON
INVESTMENT
• Cost-benefit analysis - process of determining the economic
benefits of a training program using accounting methods that
look at training costs and benefits.
• ROI should be limited only to certain training programs, because
it can be costly.
• Determining costs
• Methods for comparing costs of alternative training programs
include the resource requirements model and accounting.
• Determining benefits – methods include:
• technical, academic, and practitioner literature.
• pilot training programs and observance of successful job
performers.
• Estimates by trainees and their managers.
• To calculate ROI, divide benefits by costs. The ROI gives an
estimate of the dollar return expected from each dollar invested
in training.

WOODGROVE
TRAINING
21
BANK
EVALUATION