EVALUATION AND MEASUREMENT OF TRAINING

What is evaluation? It is the systematic collection and assessment of information for deciding how best to utilize available training resources in order to achieve organizational objectives. Types of Training Evaluation: One of the most common ways of evaluating training is named CIRO Framework of Evaluation. CIRO stands for Context, Input, Reaction and Outcome.

a)

b)

c)

Context evaluation: Obtaining and using information about the current operational context that is, about individual difficulties, organizational deficiencies, and so on. In practice, this mainly implies the assessment of training needs as a basis for decision. Input Evaluation: Obtaining and using information about possible training resources in order to choose between alternative inputs to learning. Outcome evaluation: Monitoring the consequences of training.

Three levels of outcome evaluation may be distinguished: Immediate Outcomes. Changes in trainees knowledge, skills and attitudes which can be identified immediately after the completion of training. Assessment involves some measures of how people have changed during a training programme. Intermediate outcomes. The changes in trainees actual work behaviour which result from training. Assessment involves monitoring performance on the job.

Long-term outcomes. The changes in the functioning of part or all of the oranization which have resulted from changes in work behaviour originating in training. Assessment is usually in terms of output or financial measures. d) Reaction Evaluation. Obtaining and using information about trainees on expressed current or subsequent reactions in order to improve training.

This framework is based on three simple and fundamental questions, which the trainer must constantly ask. These questions are as follows: 1) What needs to be changed? 2) What procedures are most likely to bring about this change? 3) What evidence is there that change has occurred? The first two questions must be settled before any training can begin. The third question however, can only be answered fully after the training has been completed.

KIRKPATRICKS MODEL Another popular framework for training evaluation is known as Kirkpatricks Model . This model has 4 steps as described below: Step 1. REACTION : How well did the trainees like the programme? Step 2. LEARNING : What principles facts & techniques were learned? Step 3. BEHAVIOUR : What changes in job behaviour resulted? Step 4. RESULTS : What were the tangible results?

For the evaluation exercise to be effective, it is very important that training objectives are carefully thought and framed. WHY EVALUATION ? Training is done with specific objectives. Hence, evaluation of training is a must. It is necessary in order to determine:

a)
b) c)

if the developmental objectives were achieved. the effectiveness of methods of instruction. if the best and most economic training activities were conducted. The importance of training evaluation is shown in the following diagram which shows the sequence of training activities.

THE SEQUENCE OF TRAINING ACTIVITIES

The Evaluation / Assessment Design It follows the Kirkpatrick model and seeks to capture information at the four levels. It also seeks to collect data from multiple sources: the course reactionaries, interviews with participants after the training event; interviews with business heads; other relevant results. In addition, there is an expressed commitment to devise measures of evaluation and to tackle the systems issues -- a recognition that training is only one of the operating systems in the organization and should not be viewed in isolation.

This design is intended as an aim or objective rather than a reflection of standard operating practice. It indicates how things should be done rather than they are usually done. For example, participants bosses are not always aware of the purposes of the training event and these interviews can produce limited results. However, an appropriate design assists in putting evaluation activity in its context.

1.

2.

DETERMINIG RETURN ON IVESTMENT IN TRAINING Return on investment (ROI) is an important training outcome. Cost-benefit in this situation is the process of determining the economic benefits of a training programme using accounting methods that look at training costs and benefits. Training cost information is important for several reasons: To understand total expenditures for training, including direct and indirect costs. To compare the costs of alternative training

3.

4.

programmes. To evaluate the proportion of money spent on training development, administration, and evaluation as well as to compare monies spent on training for different groups of employees. To control costs. The process of determining ROI begins with an understanding of the objectives of the training programmes. Plans are developed for collecting data related to measuring these objectives.

The next step is to isolate , if possible, the effects of training from other factors that might influence the data. Last, the data is converted to monetary value and ROI is calculated. Four Steps of Cost Benefit Analysis Comparing costs and benefits can be done in the following four simple steps: 1. Calculate the cost of training. This will include training costs such as: Facilitator fees Training design

Course materials Videos and workbooks Facilities rental Equipment rentals (such as overhead projectors) Production downtime (including employee time off the job) Videoconferencing facilities Specialized computer equipment Administration (such as registration procedures or confirmation notices)

All the relevant costs, divided by the anticipated number of participants, gives the cost per participant. 2. Determine the potential savings generated. These savings might include: Fewer errors Reduced customer turnover Less equipment downtime Increased revenue collection Faster equipment start-up time Reduced employee turnover, when turnover is attributable to poor supervision

Proper implementation of new customer strategies Higher workplace morale through more effective management practices Less time lost to grievance hearings and work stoppages because of ineffective supervision Reduced recruitment costs (because training can create more job-ready candidates for promotion) Maximized productivity of new employees through efficient orientation training

3.

Calculate the potential savings. To calculate potential savings, set goals for post-training achievements by identifying and quantifying the changes a training initiative will produce if all other factors are constant. The factors in the formula include the following: Current level of performance (for example, 200 error rates per month; six lost customer accounts per month; five days lost to work stoppages per year) Translate the current level of performance into a Rupee figure (for example: 200 error

rates x five minutes correction time x Rs.15 salary per hour = Rs. 250 per month). Identify the change that training can produce (for example, reduce errors to 50 per month). Calculate the savings that the target criteria will generate (for example; 200 errors 50 errors = decrease of 150 errors per month savings = 150 x five minutes / 60 x Rs.15 = Rs. 87.50) Identify a meaningful time line for realizing savings, based on your best business predictions about factors contributing to

errors remaining unchanged. Identify the number of employees in the target training group. Divide the total anticipated savings by the number of participants to identify the savings per participant. 4. Compare the costs to savings. Multiply the cost per participant by the total number of participants. Multiply the savings per participant by the total number of participants. Compare your figures to establish your

business case for training.

This exercise not only identifies actual costs and realistic savings but also ensures that your training expenditures are reasonable and targeted to measure business outcomes.

1. 2. 3.

In Actual Practice How to Calculate ROI ? Once the costs and benefits of the programme are determined, ROI is calculated by dividing return or benefits by costs. To calculate return on investment, follow these steps: Identify outcomes (e.g., quality, accidents). Place a value on the outcomes. Determine the change in performance after eliminating other potential influences on training results.

4.

5.

6.

7.

Obtain an annual amount of benefits (operational results) from training by comparing results after training to results before training (in Rupees). Determine the training costs (direct costs + indirect costs + development costs + overhead costs + compensation for trainees). Calculate the total savings by subtracting the training costs from benefits (operational results). Calculate the ROI by dividing benefits (operational results) by costs. The ROI gives

an estimate of the Rupees return expected from each Rupee invested in training.