Many happy returns

Study: ROI of training program for chief engineers tops 500%

BY ROBERT STERLING

Just how valuable are corporate learning and training programs? Recent evaluations of a Boeing leadership program for chief engineers may surprise even the most fervent supporters.

The evaluations showed that thanks to the participants' improvements in on-the-job performance and quality, the average training return on investment (ROI) results for the first two Engineering Leadership Program (ELP) courses studied was more than 500 percent. The study identified advances in areas such as efficiency, quality and general problem avoidance. (The study was a qualitative assessment, where ELP participants helped quantify the findings—and was not representative of an in-depth financial analysis.)

In other words, participants are making better decisions in a more timely fashion thanks to the principles they learned in ELP—and are saving Boeing money totaling five times the overall cost of the program.

For Ron Johnson, vice president of Engineering and Operations for Air Force Systems, the results are tangible and transparent on the job, and may even be conservative.

"The graduates of this program are using the tools from class on a daily basis," Johnson said. "All it takes is an adjustment in a critical program at a critical milestone, and savings in the tens of millions can be realized. In that sense, I really believe the ROI is a lot higher."

The studies were completed by Boeing's Learning, Training and Development organization, in partnership with the Enterprise Engineering Process Council and The Learning Alliance, a company that specializes in training evaluation. The group designed a process to assess whether ELP participants had applied new skills they learned and whether those skills added value in the workplace. ELP participants learn to lead engineering initiatives and manage performance in organizations that rely on engineering excellence.

According to the study, one participant said the job-simulation portion of ELP helped him learn how to deal with customers and suppliers better. That's led to repeat business at the low-speed wind tunnel where he works. Another participant credited the program with helping him develop a better understanding of team dynamics and subteams, allowing him to reprioritize projects and understand the impact of the goals he sets. With this knowledge, he was able to win executive support for his virtual office plan—which led to a substantial reduction in internal billing rates.

"What makes me most proud of this program is that we used a unique approach to learning," said Kim Armstrong, ELP program manager. "The Enterprise Engineering Process Council came to us with a need, and we were able to design a program that was different and cutting-edge, much like engineers do when they develop our high-tech products. To top that off, we were able to show them what impact their investment had made."

"We know the [ELP] design is good, and Engineering leadership is validating the results we have presented," said Mark Dana of Learning, Training and Development's Strategy, Integration and Metrics group, which conducted the evaluation. Dana added that four classes have already been approved for 2006.

The training ROI evaluation process began last year when the Learning, Training and Development leadership team made accountability one of their organization's business strategies. "We wanted a way to measure how employees' on-the-job performance had changed," said Bonnie Stoufer, Learning, Training and Development vice president. Stoufer said this analysis not only helps business units "clearly identify the benefits of the time and money they had invested in their employees' learning," but it helps Learning, Training and Development align more closely with its customers' needs.

The training ROI evaluation team relied on an initial Web-based survey, 10 interviews and an extensive phone survey to gather results. Interviews occurred immediately after the ELP and then six months later to capture on-the-job experience.

The team then followed up with 30-minute random phone interviews of 10 participants from each ELP class. Through the interview process, the team calculated a dollar figure for positive business impact. The participants were asked to identify an amount of money saved on a particular project or process because of what they had learned in the Engineering Leadership Program.

Because of the success of the ROI evaluation with the engineering program, it will be used to evaluate seven other initiatives, including other leadership development programs. Said Jim Eckels, evaluation focal for Leadership Development and Functional Excellence: "Quantifying behavior and performance changes from leadership development programs isn't easy, but it's the direction we'll need to go to measure the impact of training at Boeing." 

robert.sterling2@boeing.com