Taking leaders of any level away from their everyday roles is a costly proposition. If business leaders and stakeholders are truly going to be convinced and buy in to leadership training, they’re going to want tangible evidence of bottom-line business impact. So, as training evaluators, we always have to ask the question: “Do the revenue-generating or cost-saving benefits of the leader training outweigh the costs of the entire training experience?” In other words, is the leadership training really worth it?

Another question training evaluators should be able to answer for their stakeholders is, “How much does it benefit the business to send participants through the training sooner rather than later?” For instance, what’s the benefit of training new leaders in the first three to six months of their new role, as opposed to postponing or putting off the training? Many leaders and their managers think they can’t afford to leave the business for one or two days to go through training. As an evaluator, I wanted to conduct a case study to prove that leaders can’t afford not to go through training. The following case study is based on a two-day leadership training program for new leaders that I evaluated at Verizon.

What We Measured

To provide evidence of the impact of training, we implemented a comprehensive evaluation strategy that measured success at every level and milestone of the trainee’s experience. That is, were they engaged and satisfied with the experience? Did they learn anything new? Did they apply the training and improve any key leadership behaviors back on the job? Did those behaviors impact the business? Which performance metrics did their teams and direct reports improve? What were those improvements worth to the organization in dollar value? Did the dollar benefits outweigh the costs of training, and what helped or hurt the transfer of learning back to the job. Why would some participants get a bigger impact and return on investment (ROI) than others?

This approach is based on the five-level Phillips/Kirkpatrick model of training evaluation, with the addition of a level six added to understand what transfer climate factors were at work to improve impact and ROI.

• Level 1: Did they like it? Were participants engaged with content, facilitators and overall delivery of the training experience?

• Level 2: Did they learn anything? What new knowledge and skills did participants gain from the training?

• Level 3: Are they doing anything differently or better? What improvements have they had in key behaviors back on the job after attending the training?

• Level 4: Are they impacting the business? Are these improved behaviors increasing their key performance metrics that can increase revenue or create cost-saving efficiencies?

• Level 5: Was it worth the investment? Did the benefits of training outweigh its costs?

• Level 6: What maximizes training impact? What factors back on the job can help or hinder the application and transfer of learning?

This is powerful data to show stakeholders because it describes how successive rollouts can be optimized and underscores the importance of things like manager endorsement of the training and manager support back on the job.

Using this model, we can essentially tell the extent of the training’s impact — from learners’ participation in the program to business results to ROI to and how to improve the training impact in the future. Additionally, by creating this comprehensive, connected and cohesive picture of impact, we would be able to show how valuable it was (in quantifiable terms) for business leaders and stakeholders to send their new leaders through the training early in their role, as opposed to later, which typically happens when the everyday demands of the business seem to take priority.

The Training

The leadership training was delivered to over 2,000 first-time managers to help them transition from individual contributors to people leaders, and to help them ramp up their leadership skills as early as possible in their new roles. These skills include coaching their frontline employees, aligning their teams to organizational priorities and recognizing the right behaviors. These managers could be in their role anywhere from just a few months to more than a year before they attended the training, which is why we wanted to prove the value of early attendance. Ultimately, the objective is to develop leaders who inspire people to perform at a higher level and, therefore, increase organizational productivity and profits.

How We Measured Impact

Level 1: A survey was administered to participants immediately after training, asking participants to rate various factors like instructor effectiveness, course content and relevance to their job.

Level 2: We added several learning questions to the same post-training survey. Here, we asked participants whether they gained new knowledge and skills that were relevant for their roles.

Level 3: We used multi-rater feedback. First, we administered a survey three months post-training that asked participants to rate the improvement (if any) they observed in very specific leadership behaviors. We then gave these same core questions to both the participants’ manager and direct reports. This was to compare and corroborate that these leadership behaviors did, in fact, improve over the post-training period.

Level 4: To define and isolate the impact training had on the business, we used a control group design. That is, we compared the performance improvements of a trained group of leaders who had been through the training (within their first three to six months as a new leader) and a twin sample or control group of leaders who had not yet attended the training (also within their first three to six months in their role). For each group, we tracked their performance three months before the training and three months after training to see if there was any improvement. The control group of untrained leaders was tracked during the same timeframe, and all other factors were controlled for (e.g. tenure, location, team size).

Level 5: To measure ROI, we turned the performance benefits we found at Level 4 into a dollar value (only the metrics we could monetize), and then compared them to the fully loaded cost of training.

Level 6: To understand which factors impacted ROI, we added several questions to the original survey asking participants about their immediate work environments, such as “how supportive was your immediate manager when you tried to apply all your learning?” Answers to these questions were then analyzed and correlated to the amount of impact the training had on their behavioral improvements and performance.

The Results

Level 1: Satisfaction with training: The average rating was 4.73 out 5.00 stars.

Level 2: Learning new knowledge and skills: 97% of participants gained new and valuable knowledge or skills for their job.

Level 3: Behavioral change on the job: 94% of participants showed “some” to “exceptional” improvement in the seven key leadership behaviors included in the multi-rater survey. There was consistent corroboration from both direct reports and managers that these improvement had indeed taken place.

Level 4: Business impact: Results within the retail business showed the overall increase in performance for the trained group of managers was significantly higher than the performance increase of the control group. Using the five metrics we tracked, there was a 2.1% average improvement for the trained group above and beyond the control group. This became the tangible “benefit” of training to the business.

Level 5: ROI:

Benefits of training: Once we partnered with the finance team to ascertain what each metric was worth to the business, we were able to identify and monetize an impressive benefit of $2,138 per participant.

Costs of training: To calculate the cost of training, we used a fully loaded cost calculator to account for every expense incurred as a result of training. These included development costs, attendance costs, participant travel and time away from job, materials, instructors, etc. The total fully loaded cost for each participant was $1,668.

Once both benefits and costs were calculated, we then plugged these dollar values into the ROI formula. We were able to confidently and conservatively say that the business had an ROI of 29% within just the first three months post-training and a final annualized ROI of 415%. This means that not only did the business recoup the fully loaded costs of training for the participants in the study, but they also made $4.15 for every $1 spent.

Level 6: Factors that maximized training impact:

The top three factors that were most effective in maximizing the impact and ROI of the training were:

• Having an immediate manager who discussed the training with them and encouraged them to apply new skills
• Being given the opportunity and extra time to have coaching conversations with each of their direct reports
• Quickly identifying and addressing the resistors to change within their teams

Conclusion

All too often when L&D offers training to new leaders, many employees will procrastinate or postpone attendance, rationalizing that they can’t afford to leave their roles to participate in training or they need to stay focused on their metrics. What we wanted to prove with this case study was that leaders can’t afford not to attend leadership training and that, the sooner they go, the sooner they can add even more value to their organization’s bottom line.

This is exactly what our case study proved: New leaders were adding more value to the business the sooner they went through the training. In fact, by using this measurement approach and looking at participants’ business benefits three, six and 12 months post-training, we were able to show exactly how much performance advantage was created by attending the training and, at the same time, how much could be lost by not attending the training.

By duplicating the same rigorous trained versus control methodologies and designs other disciplined researchers might use to test whether a medicine or drug “works,” we sought to prove how much training “worked” on improving job performance. Medical researchers would call this the “experimental” group (the ones who get the new medicine) and the “control” or “placebo” group (the ones that don’t get the new medicine).

Further, we were able to see how much would be gained or lost by putting off taking the “medicine.” Results told us that if managers put off training for three months, they would lose $2,138 in improved performance. If they put off going through training for six months, they would be losing $4,276 and if they put off training for a full year after they assume leadership roles, they could be losing $8,552 in extra performance.

At a first glance, this might not seem like much for a company that generates significant annual revenue — but it’s a very significant finding. When you consider how many employees are stepping into new leadership roles across the enterprise and how many employees will be influenced every day by these leaders, the numbers add up. Add to this the fact that this significant benefit comes at the relatively small cost of $1,662 per participant, and it creates a compelling, and even irrefutable, case for all leaders to take their prescribed training. To put it another way, for the benefit of their organizations, leaders should hurry up and take their medicine.