Evaluating the Training Effort
Evaluating the Training Effort. With today’s emphasis on measuring results, it is crucial that the manager evaluate
the training program. There are several things you can measure: participants’ reactions to the program, what (if anything) the trainees learned from the program, and to what extent their on-the-job behavior or results changed as a result of the program.
In one survey of about 500 U.S. organizations, 77% evaluated their training programs by eliciting reactions, 36% evaluated learning, and about 10% to 15% assessed the program’s behavior and/or results.167 Computerization facilitates evaluation.
For example, Bovis Lend Lease uses learning management system software to monitor which employees are taking which courses, and the extent to which they’re improving their skills.
There are two basic issues to address when evaluating training programs. One is the design of the evaluation study and, in particular, whether to use controlled experimentation. The second is, “What should we measure?”
Designing the Study
In deciding how to design the evaluation study, the basic concern is this: How can we be sure that the training (rather than, say, a company-wide wage increase) caused the results that we’re trying to measure? The time series design is one option. Here, you take a series of performance measures before and after the training program.
This can provide some insight into the program’s effectiveness. However, you can’t be sure that the training (rather than, say, the raise) caused any change.
Formal methods for testing the effectiveness of a training program, preferably with before-and-after tests and a control group.
Controlled experimentation is therefore the gold standard. A controlled experiment uses a training group and a control group that receives no training. Data (for instance, on quantity of sales or quality of service) are obtained both before and after one group is exposed to training and before and after a corresponding period in
the control group.
This makes it easier to determine the extent to which any change in the training group’s performance resulted from the training, rather than from some organizationwide change like a raise in pay. (The pay raise should have affected
employees in both groups equally.)
Training Effects to Measure
The manager can measure four basic categories of training outcomes or effects:
1. Reaction. Evaluate trainees’ reactions to the program. Did they like the program? Did they think it worthwhile?
2. Learning. Test the trainees to determine whether they learned the principles, skills, and facts they were supposed to learn.
3. Behavior. Ask whether the trainees’ on-the-job behavior changed because of the training program. For example, are employees in the store’s complaint department more courteous toward disgruntled customers?
4. Results. Most important, ask, “What results did we achieve, in terms of the training objectives previously set?” For example, did the number of customer complaints diminish? Reactions, learning, and behavior are important. But if
the training program doesn’t produce measurable performance-related results, then it probably hasn’t achieved its goals.
Evaluating these is straightforward. presents one page from a sample evaluation questionnaire for assessing reactions. Or, you might assess trainees’ learning by testing their new knowledge. For behavioral change, perhaps assess the effectiveness of a supervisory performance appraisal training program by asking that person’s subordinates, “Did your supervisor provide you with examples of good and bad performance when he or she appraised your performance most recently?”
Finally, directly assess a training program’s results by measuring, say, the percentage of phone caller questions that call center trainees subsequently answered correctly.
A careful comparison of the training program’s costs and benefits can enable the human resource team to compute the program’s return on investment. Online calculators are available to facilitate such analyses.
A program at MGM Resorts illustrates training evaluation. In the hospitality industry, how likely guests are to return is a crucial metric, and is measured at MGM by “Net Promoter Scores” (NPS). With MGM’s NPS scores not up to par, its training team concluded “guest facing” employees weren’t sufficiently engaged. It created an Essentials of Hotel Management Program for front desk and assistant managers.
The program emphasized skills like collaboration and communications. At the end of the approximately 1-year program, NPS scores had risen about 2% (which is considered a notable accomplishment).