Evaluating training.

[check] This checklist provides ideas and key points for line managers to build into their employees' training programmes and to help assess the effectiveness of training. It should be read in conjunction with the Checklist on Training Needs Analysis. It is not written for trainers themselves.

Virtually all training events end with the participant completing a 'happy sheet' on which they record their feelings about the course or the instructor. These may fail to indicate, however, whether anything useful was learned or how that knowledge will be transferred to the workplace.

The training evaluation process is a continuous cycle of defining the objectives of training, identifying training needs, delivering training to meet those needs and objectives, assessing trainees' reactions to the training (happy sheets), seeking evidence of skills or knowledge learned and of their implementation in the workplace, and measuring the effects of training on bottom-line results.

It is not always possible to carry out such in-depth evaluation, but that is not to say that nothing should be done at all. The key is to have a training objective and have some indicator(s) in place to see if that objective has been met. In this way it should be possible to get some idea of the return on investment.

Definition

Evaluation is an analytical process of assessing the value of something. In the case of training, it focuses on whether the time and money spent on training have achieved the required results.

Advantages of evaluating training

* Broadly, it can tell you whether what you have done has worked.

* allows you to check that resources were targeted at identified priorities.

* The evaluation process confirms--or denies--that the improvements in individual performance which you sought have been achieved.

* If training has not achieved its objectives, the evaluation gives you information that should help you to improve it next time.

* The information gained feeds back into the appraisal process and helps managers discuss progress with individuals.

* Individuals and teams know what results are expected from training before they start, raising their commitment to, and involvement in, the training itself.

* Having an effective evaluation process in place enhances the organisation's progress towards Investors in People, as it is one of the key standards

* It helps you to see clearly where the organisation stands in terms of employee development, and provides information about performance on which to base appropriate future training plans and processes

Requirements for the evaluating of training

* A commitment to training as an important and central business function rather than an optional or non-essential activity.

* A disciplined and active planning approach rather than a reactive management style that concentrates on fire fighting.

* Valuable management time to be spent on careful consideration of what is to be achieved and measured, and how to measure it, before allocating any training.

* Commitment of time and resources to the detailed analysis afterwards.

Action checklist

1. Define what you want training to achieve Remember the evaluation process starts as soon as you begin constructing a training plan Having identified needs, quantify as specifically as possible what results and outcomes you expect. In many processes this can be relatively easy to define, for example:

* operate a machine safely

* use a graphics package

* set up a World Wide Web site

* construct widgets using new technology.

In many cases, these outcomes can be specified and measured by occupational, organisational or national standards.

It is harder to set measurable targets when it comes to events designed to contribute to continuing learning and changes in behaviour,. Building up knowledge and experience in a specific area is fundamental to development but difficult to quantify. Work with the trainee to specify expected outcomes--for example, a more effective selling behaviour.

2. Turn targets into objectives

Objectives tell you what is to be achieved, by when. They should be SMART:

Specific--Measurable--Achievable--Realistic--Time limited

A training objective specifies what you realistically can expect the trainee to be able to do or to know as a result of the training.

If it is a skill that is to be achieved, the measurability aspect could be, for example, that within six weeks of the end of training, the trainee will be able to type a ten-page report with no more than six mistakes within an hour.

In the case of knowledge, avoid the word 'understand' because it is not measurable. Replace it with something like 'state', 'explain' or 'describe', because they are checkable and the trainee will need to have absorbed the knowledge in order to meet the objective.

3. Make sure everyone knows the objectives from the start

These include:

* the trainees, in the information they receive in advance--both via personal briefings from their manager and from any materials they get as joining instructions

* their managers (if you are arranging the training on behalf of other departments), so they know what their staff should be able to do as a result of the training

* the trainers--this may sound obvious, but they need to design the training based on what it should achieve, rather than the areas in which they can train others (which may be different). Where training is provided by an external organisation, check that the provider can meet the objectives you specify.

4. Design methods for comparing results with objectives

The best way to do this is to get people together to come up with one agreed and consistent approach. It may involve a post-training action plan, a debriefing session on return to the workplace, forms, questionnaires, observation checklists, feedback meetings or statistical data, but the key point is that you must design the assessment procedures early on.

Immediate feedback is important, but do assess performance improvements over a realistic time span, often weeks and sometimes months. This allows time for the training to be applied and practised, leading to the actual outcomes you want to evaluate.

5. Evaluate the input

Remind trainees to keep their objectives in mind throughout the training and to raise the matter with the trainer if their needs are not being met. If the training is provided by an external organisation, ask the trainees to give you a summary of their reactions to the course at a debriefing session on their return. Encourage them to be honest in their opinion of the worth of the training.

6. Assess the training in the workplace

The process of evaluation is a matter of comparing results with expectations. Encourage trainees to produce a realistic action plan to implement what they have learned now that they are back at work. In the longer term, perhaps three months, ask the trainees what the training has helped them to achieve within a particular period, for example, that the knowledge gained has contributed to a successful business plan.

7. Use the results

The information gained from evaluation is critical in starting the training cycle again, and planning what needs to be tackled next year, and how. Evaluation sets out key facts and measures of progress more clearly than any sort of gut reaction or guesswork.

Dos and don'ts for evaluating training

Do

* Specify the outcomes and results required.

* Design the assessment and evaluation procedures early on.

* Involve other managers with a stake in the training outcomes.

* Involve trainees themselves.

* Review with an open mind what the evaluation tells you--mistakes and failures can be more helpful in making continuous improvements than convincing yourself it really was all right, when it wasn't.

Don't 

* Avoid setting measures for the hardest-to-measure activities; if the best available is rough and ready, it remains the best available and it is much better than doing nothing.

* Try to justify poor results with excuses--if there is a lesson to be learned, value it.

* Rely on 'happy sheets'.

* Give up--evaluating training is widely regarded as the most difficult aspect of the training function.

Useful reading

Evaluating training: from personal insight to organisational performance, 2nd ed Peter Bramley 

Chartered Institute of Personnel and Development London, 2003

Evaluating training, Sharon Bartram and Brenda Gibson

Aldershot: Gower, 1999

How to measure training effectiveness, 3rd ed, Leslie Rae Aldershot: Gower, 1997

Thought starters

* How do you find out now whether training is achieving the right results?

* Wouldn't you like to know which training activities were effective and which could be improved?

* Aren't there some training activities that you already believe either don't work or could be improved, but you haven't got any evidence on which to base a case for improvement? Updated July 2005