Trainers Worry about Measuring the Impact of Training

You and your organization invested time and money to design, implement and conduct training for a group of employees. Now comes the hard part: Determining whether the program had any impact on the business.

At this point, many training managers break into a cold sweat. A business, not to mention the world, is complex. Any number of variables can affect outcomes. A training program is just one of them. But there's no reason to panic as there are many tools to help the training manager and stakeholders determine whether a program was worth the investment.

One key challenge, and the focus of this post, is isolating the impact of the learning. This involves understanding the goals of a program and what other factors may be influencing the result. For example, a training session on sales techniques may happen to correlate with a 36% increase in sales. The training, of course, may have been the reason. But for a complete analysis, you'll want to investigate what was happening with the economy, business rivals, your own firm's product offerings and even the weather.

For the sake of argument, let's say the economy remained stagnant, your product mix was unchanged and the weather was not a factor for the period. You may also find no change in business at the rival firm. If this is the case, you can examine the impact of your training with more confidence.

Even if you identify changes in other variables, you can isolate these, too. The job is generally easier if you have quantitative data available, such as the percentage decline in overall business activity.

There are a number of analysis techniques for isolating the impact of training, some of which require advance planning. My three favorites are:

1. Plotting a trend line that illustrates how sales changed before and after the training. You can plot other lines as well, including economic stagnation or even the competitor's sales collapse.

2. Using a control group. This takes some advance planning and involves not training a subset of employees. After the program is finished, you can compare results.

3. Using matched pairs. This involves comparing the performance of each individual before and after the training. If there are significant differences, you can say that the training had measurable impact.

Of course, you will likely encounter training programs where isolating business impact is impossible or not appropriate. In one of my books on training evaluation, I used the example of machine operators who were trained on quality-improvement techniques. They not only learned about working as teams but also how to use a document-management system. Their incentive program was revised as well. All of these components form a single, integrated change program.

Another example: Training the sales team on a new customer relationship management (CRM) system. In this case, it makes no sense to measure the effect of training versus the impact of the new CRM system. Why? -precisely because employees can't apply the training without the new CRM being installed.

If you're having no luck finding quantitative numbers to measure the impact of training, you can always ask stakeholders to estimate the effect of the program. This is not ideal and introduces a significant subjective element, but the information may be all that's available to do any analysis at all.