Measuring training & development.
Training and development continues to be a valued component in the modern management of human resources. Emerging research shows that investments in human capital, including training, are positively related to organisational performance (Kraiger, MeLinden & Casper 2004). The scope of training and development activities vary from one organisation to another. The activities include employee, associate, technical, operator, supervisor and management training and development programmes. Training and development professionals manning the training departments carry out the entire process of training and development that includes preparation of budgets, needs assessment and analysis, programme design, delivery, and evaluation of training programmes.
As the companies spend huge amounts on training and development, they expect results from the programmes. The results can be measured through evaluation of training programmes. Kirkpatrick's (1994) four level hierarchy is a classical model of evaluation for guiding evaluation practices. The first level is participants' reactions, the feedback of the participants that are collected during or at the end of the training programme. The second level measures learning that has taken place based on the inputs gained during the training programme. The third level measures changes in the behaviour because of the transfer of learning to back on the job situation. The fourth level measures the business results gained from the training programme. Phillips (1997) added the fifth level, return on investment that compares monetary value of the benefits to the costs of the training programme. But organisations still do not routinely measure the impact of training. Among organisations participating in ASTD's Benchmarking Services, 75% measured training reactions and 41% measured learning, but only 11 % measured business results (Sugrue 2003). Bersin (2006) has identified that 81% of the organisations routinely measure reactions; 35% measure learning of the participants; 14% measure job impact; 10% measure business impact; and 5% measure return on investment.
Training & Development Measures
There are some attempts to identify measures of training and development that are considered to assess the effectiveness of training and development as an integral function of human resource management in organisations. As a part of HR practice measures, Saratoga Institute (1994) has identified the following training and development measures: 1) number of training days and programs held per year, 2) cost per trainee per hour, 3) percent of employees involved in training, 4) number of courses taught by subject, 5) percent of employees with development plans, 6) number of courses taught by subject, 7) percent of payroll spent on training, 8) payroll expense per employee, 9) comparison: who did and who did not attend training, 10) Ratio of advanced to remedial education, 11) time for new programme design, 12) percent of new programme material each year, and 13) efficiency of training registration.
Considering levels of training evaluation, Phillips (1999) has suggested training measures which relate to productivity, quality, cost, time, and soft data. Productivity measures are of two types. One is performance measures that are related directly to participant output in work units produced or services provided and another one is process measures that include the total number of employees trained, percentage of employees receiving a particular training, and total hours of training per employee. Quality measures include reject rates, scrap rates, rework, error rates, defects, and downtime. Phillips classified cost measures also in two categories such as performance measures and process measures. Performance measures include cost reductions achieved by participants, and application of cost control methods learned in the training and process measures cover cost of training. Time measures also include both performance measures such as improvements in time brought by some training programmes and process measures that cover the time needed for training. Finally soft data includes participants' feedback, their learning, measures that include skill application.
Bersin (2006) has conducted a study among senior professionals associated with training and development in North America. It is focused on finding the measures of training and development that are extremely valuable as perceived by the training professionals and the measures that are really used routinely in the respective organisations. Based on his research study, Bersin has identified the following training measures: 1) enrolment to the programmes, 2) compliance, 3) course completion, 4) student hours, 5) students' satisfaction, 6) managers' satisfaction, 7) learning scores, 8) cost to develop, 9) cost to deliver, 10) cost per student, 11) total cost, 12) job impact, 13) business impact, 14) actual business impact, and 15) return on investment. The study has indicated that most organisations focus on measuring standard course operations: enrolment, and completion of the course. Reaction level evaluation surveys are very widely used. A very small number of organisations routinely measure return on investment, business impact or job impact. Another interesting finding of the study is that only one-third organisations measure cost. The study has concluded that the current state of measurement is almost inverse to the perceptions of value.
The Present Study
Organisations in India are committing more resources, in the forms of both time and money, towards training and development of employees aiming at improving their competencies. Training and development has evolved and matured to a substantial degree in India (Rao, Rao & Yadav 2001). Organisations are increasing training budgets year after year (Srimannarayana 2006). But to what extent measurement of training and development has evolved and matured in India? Prompted by the study undertaken by Bersin, the present study is carried out to find out value of measures of training and development as perceived by the HR/ training professionals in India and the extent of using them routinely in their respective organisations to assess training and development function.
A questionnaire has been developed incorporating the following measures of training and development after considering various possible measures of training and development function that have been identified in literature and earlier research survey to assess this function:
1) Number of employees trained in training programmes
2) Number of training days
3) Training costs
4) Percent of amount spent on training in payroll
5) Feedback of participants
6) Learning of the participants during training
7) Transfer of learning on-the-j ob
8) Performance improvements made because of training received
9) Cost and benefit analysis of training
10) Satisfaction of line managers on training
These measures may be classified into two categories for the purpose of this study. One is the traditional measures that are easy to develop and measure and another is the impact measures that add value to employees and organisations. Traditional measures include number of employees trained per year, number of training days, training costs, percent of amount spent on training, feedback of participants, and satisfaction of line managers. Impact measures include learning of the participants during training, transfer of learning on the job, performance improvements made because of training received, and cost and benefit analysis of training.
The questionnaire has been administered among 105 HR/training professionals working in different sectors of employment in India during May-June 2009. The respondents work in manufacturing, IT/ITES and service sectors with a minimum experience of three years in their respective organisations in the functional area of training and development. They have represented private, joint venture and public sector organisations across India.
The respondent HR/training professionals were asked two questions.
1. What do you consider extremely valuable measures of training and development among the measures mentioned in the list?
2. What do you actually measure in training and development among the measures mentioned in the list?
Their responses have formed the basis for this study.
Results & Findings
The Measures that are considered extremely valuable: Fig. 1 presents the percentage of HR/training professionals who have perceived the measures of training and development as extremely valuable. Interestingly, about threefourths of the respondents have considered that performance improvement made because of training is the extremely valuable measure of training. This is followed by transfer of learning inputs on the job (70.48%). More than half (53.33%) of the respondents have viewed cost and benefit analysis as extremely valuable. It is significant to note that nearly same percentage (51.43%) of the respondents have considered training costs as extremely valuable. Nearly the same number of respondents (49. 52%) has considered line managers' satisfaction as extremely valuable a measure. Learning of participants during training and number of training days are extremely valuable to 47. 62% of the respondents. 45.7% of the respondents have considered percent of amount spent on training in payroll is extremely valuable. Interestingly, only 42. 86 % of the respondents have considered feedback of the participants as an extremely valuable measure and only 40% of them have viewed number of employees trained in training programmes as an extremely valuable measure.
The Measures that are actually used: As shown in fig. 2, feedback of the participants (95.24%) is a very widely used measure of training and development programmes. Another interesting finding is that number of employees trained in various training programmes (84.76%) is very popular among the training and development measures. This is followed by training costs ((79. 05%) and number of training days (77.14%). More than half of the respondents have mentioned that they measure percent of amount on training in payroll. Less than half (42.86%) of the respondents have mentioned that they routinely measure learning of participants during training. Only 35.24% of the respondents have stated that they measure transfer of training on the job. In case of measuring performance improvements made by the participants, nearly one-fourth of the respondents mentioned that they measure it. Another significant finding is that though training costs are measured on a routine basis, cost and benefit analysis of the training programmes does not take place in a great majority of the organisations. Surprisingly, satisfaction of line managers who nominate employees for training and development programmes is also not measured in these organisations.
Comparison: More than three-fourth of respondents have believed that performance improvements made because of training is an extremely valuable measure. But actually less than one-fourth of respondents use it routinely to measure training and development. A great majority (70.48%) have believed that transfer of learning is extremely important. But only 35.24% of them use this measure routinely in their respective organisations in measuring training and development. Almost all (95.24%) respondents have mentioned that they collect feedback of the participants after completion of training programme routinely. But surprisingly, less than half (42.86%) of the respondents have considered it as an extremely important measure. An overwhelming majority (84. 76%) of the respondents routinely measure number of employees trained in training programmes. But only 40.1% of the respondents have believed that this is an extremely important measure of training. While 53.33% of the respondents have believed that cost and benefit analysis is extremely valuable, only 8. 57% respondents have mentioned that they actually measure it. Only 47.62% of the respondents have considered number of training days as a valuable measure, but 77. 14% of the respondents have mentioned that they use this measure routinely in measuring training efforts.
Traditional and Impact Measures: As indicated in fig. 3, traditional measures such as feedback of participants, number of employees trained, number of training days, training costs, and percent of amount spent on training in payroll appear to be more popular measures in use than the impact measures such as learning, transfer of learning, performance improvements, and cost and benefit analysis of training and development programmes.
The findings indicate that there is a gap between the measures that are valued and the measures that are used actually in assessing training and development. HR/training professionals are aware of the valuable measures of training such as performance improvements, training transfer, and cost and benefit analysis, which really indicate the impact of training. But they do not make attempts to measure these aspects. What would be the reason behind this current state of measurement? This situation might be because of a combination of factors that crept into the entire training process. Inadequate needs assessment and improper nominations make training professionals frustrated to go ahead with the use of impact measures. If the nominations are not proper, impact of the training cannot be seen on the job. Subsequently cost and benefit analysis would become a futile exercise. Another combination of factors might be training design and delivery. If the training is not designed based on the training needs of the participants and delivery is not carried out with appropriate methodology incorporating identical elements or at least general principles, transfer of training would not be possible, even though the participants are happy with the programme. Having known this fact, the HR/ Training professionals might not venture to measure training impact.
Assume that the needs assessment is adequate; nominations are proper; and training design and delivery are appropriate to participants and their needs. HR/training professionals are aware of what are the valuable measures of training. Even then they do not use the valuable measures to assess training impact because of a different combination of factors. One cannot say that they do not believe in impact measures as a majority of the respondent HR/ training professionals could identify the impact measures as valuable for the organisations in this study. Therefore, there might be different factors associated with this situation. One would be HR/training professionals might lack the knowledge, skills, and abilities, and the orientation required for assessing training and development using impact measures though various models and methods are available on training measurement for impact. Second would be that the HR/training professionals might be more concerned with the negative consequences of the results of their training measurement.Third would be the belief that the measuring impact of training is time consuming and expensive. Fourth would be the limitations and difficulties in the available measurement models. They might be looking for the models that are simple and easy to measure. Finally, would be the cooperation required from participants and line managers. HR/training professionals have to get inputs required for measuring training and development using impact measures from participants and their managers. HR/ training professionals do not have control over data that is required for measuring training and development using impact measures. If the HR/training professionals believe that they do not get enough cooperation from the line managers, they might not go for assessing training using impact measures. Subsequent to all these factors, training and development measurement now in India is confined to traditional measures such as feedback, number of employees trained, training costs, and training days that are simple and easy to measure. HR/ training professionals have most control over the information required for using these measures to assess training function.
Conclusions & Recommendations
The study concludes that measuring training and development has not matured to a substantial level in India. It confines to traditional measures such as collecting feedback from the participants, measuring number of employees trained per year, training costs, training days, and percent of amount spent on training, but not impact measures. HR/ training professionals are aware of the impact measures. They consider them as valuable measures of training, but do not venture to use these measures in training evaluation. There are multiple factors responsible for this situation. Having achieved satisfactory progress with respect to traditional measures, it is now imperative on the part of HR/training professional to focus on what actually they wanted to focus in measuring training and development. That is, impact measures. Adequate needs assessment, proper nominations, appropriate training design, and delivery are prerequisites for measuring impact of training and development. This should be supplemented with adequate knowledge, skills, and orientation of HR/ training professionals on measuring training and development using impact measures. If training is expected to show its impact, HR/ training professionals have to master the training measurement methodologies using impact measures.
Bersin, J. (2008), The Training Measurement Book: Best Practices, Proven Methodologies, and Practical Approaches, San Francisco: John Wiley & Sons.
Fitz-Enz, J. (1994), SHRM/ Saratoga Institute Human Resource Effectiveness Survey, CA: Saratoga Institute
Kirkpatrick, D. L. (1994), Evaluating Training Programs: The Four levels. San Francisco: Berret-Koehler
Kraiger, K., McLinden, D & Casper, W. J. (2004), "Collaborative Planning for Training Impact", Human Resource Management, 43(4): 337-51.
Phillips, J. J. (1997;, Handbook of Training Evaluation and Measurement Methods. Houston, TX: Gulf Publishing.
Phillips J. J. (1999), Accountability in Human Resource Management, Houston, TX: Butterworth-Heinemann
Rao, T.V, Rao, Raju & Yadav, Taru (2001), "A Study of HRD Concepts, Structure of HRD Departments, and HRD practices in India", Vikalpa, 26(1): 49-60
Srimannarayana, M. (2006) "Training Trends in India", Indian Journal of Training and Development, XXXVI (2): 41-57.
Sugrue, B. (2003), State of Industry: ASTD's Annual Review of U.S. and international Trends in Workplace Learning and Performance, Alexandria, VA: ASTD.
Yeung, A. K. (1997), "Measuring Human resource Effectiveness and Impact", Human Resource Management, 36 (3): 299.
M. Srimannarayana is Professor, XLRI, Jamshedpur 8310 001, Email: email@example.com
Fig. 1: What Training & Development Measures are Considered Extremely Valuable Number of employees trained 40.1% Number of training days 47.82% Training costs 51.43% Percent of amount spent on training 45.7% Feedback 42.86% Learning of the participants 47.62% Transfer of learning 70.48% Performance improvements made 76.19% Cost and benefit analysis 53.33% Satisfaction of line managers 49.52% Note: Table made from bar graph. Fig. 2: What do you Measure Routinely in Training & Development Number of employees trained 84.76% Number of training days 77.14% Training costs 79.05% Percent of amount spent on training 56.19% Feedback 95.24% Learning of the participants 42.86% Transfer of learning 35.24% Performance improvements made 22.86% Cost and benefit analysis 8.57% Satisfaction of line managers 16.19% Note: Table made from bar graph. Fig. 3: Traditional and Impact Measures Impact Measures Number of employees trained 84.76% Number of training days 77.14% Training costs 79.05% Percent of amount spent on training 56.19% Feedback 95.24% Satisfaction of line managers 42.86% Impact Measures Learning of the participants 22.86% Transfer of learning 8.57% Performance improvements made 16.19% Cost and benefit analysis 8.57% Note: Table made from bar graph.
|Printer friendly Cite/link Email Feedback|
|Publication:||Indian Journal of Industrial Relations|
|Date:||Jul 1, 2011|
|Previous Article:||Growth in human motivation: beyond Maslow.|
|Next Article:||Burnout components as predictors of job & life satisfaction of university employees.|