By Scott Knutson, Manager, Learning and Leadership, Freeport McMoRan

ROI is the gold standard for measuring the effectiveness of training. To be able to show your company received $4 for every $1 you invested in a particular training program is what every training manager strives for. But should it be? Here are some interesting facts concerning ROI:

  • Out of 96 company CEOs surveyed in a study conducted by the ROI Institute in 2009 and reported by Hansen and Lee, only 4% of those companies measured ROI
  • 58% of training managers are not required to report on the effectiveness of their work (IBID)
  • 69% are not required to report on productivity (IBID)
  • Even ROI proponents agree accounting methodologies make it difficult to measure ROI.

Additionally, even CEOs are not sold on ROI. In a 2005 study by ASTD and IBM of CXOs, researchers reported out of 52 interviews of CXOs in 26 organizations spanning 11 industries, they had two very interesting findings: 1) CXOs are less concerned with quantitative metrics that show learning’s value contribution to business outcomes, and 2) Perception of stakeholders are a key indicator of learning’s value. Researchers also found word-of-mouth support of a particular training event was the most common measure of training effectiveness that trainers felt was most important for their customers (Sugrue, O’Driscoll, & Vona, Mary Kay).

So what does this mean for ROI. We like to say that ROI is DOA. Not dead on arrival, but demanded by older authorities. We see ROI as an old technology. This doesn’t make it useless. Kerosene lamps are an old technology, but they are far from useless. What this means is that a newer technology needs to replace it. And for us, that newer technology is Return on Expectations (ROE).

While we are biased, we firmly believe that ROE is the best measuring tool available to discover if a training event has been effective. Donald Kirkpatrick, the father of ROI, agrees. In a recent article, he lists some highlights of ROE:

  • It’s a proactive, business partnership approach that unifies teams
  • It defines training as a contributor to key business results
  • Its value is defined by business stakeholders in cooperation with training
  • It focuses on comprehensive evidence and a compelling story of value
  • It’s easy to understand, flexible, and cost-effective

Even Jack Phillips, the founder of the ROI Institute, and the father of ROI measurement, says that ROI should only be used nominally. In an article on ROI best practices, Phillips states that ROI should be used minimally–evaluating no more than 5-10% of training programs.

So, if you are a trainer and you truly want to show your customer the value of your training (even quantitatively, if you’d like) work with them up front to define their expectations for said training. You’ll be happy you did. More importantly, your customer will be happy you did as well.

To learn more on this topic from Scott, register today for THE Performance Improvement Conference at www.ispi.org/ac2012.

References
The Cultural Context of Human Resource Development. Eds. Carol D. Hansen and Yih-teen Lee. Palgrave Macmillan, 2009.

Sugrue, Brenda; O’Driscoll, Tony; Vona, Mary Kay, C-Level Perceptions of the Strategic Value of Learning, ASTD and IBM Research Report, January, 2006

Kirkpatrick, Donald, ROE’s Rising Star, T&D Magazine, August, 2010

Phillips, Jack (2003). ROI Best Practices. Retrieved from http://media.roiinstitute.net/articles/pdf/2007/10/07/ROI_Best_Practices_-_Business_Intelligence.pdf

About the Author
Scott Knutson, MBA, CPLP, is Corporate Manager of Learning and Leadership Development for Freeport-McMoRan Copper and Gold in Phoenix, Arizona. He has more than 20 years of experience in the design and delivery of various leadership, management, and continuous improvement topics as well as managing training departments in the semiconductor, pharmaceutical, and mining industries.