A competent evaluator can encapsulate Philips’ fifth level of ROI based upon cost-benefit analysis into the fourth level of Kirkpatrick if Kirkpatrick’s Results level is broadened to include bottom-line figures and accountability. That is, to take the measurement of training away from the trainers and put it squarely back in the hands of the managers responsible for P&L. To do this, the first action of the manager must be to insist on management input being central to the training content. It is at this point that a new factor enters the equation: we meet expectation as well as investment.
ROE, not just ROI!
If included as an element of the final assessment of learning, Return on Expectations (ROE) inserts qualitative into the more-usual quantitative models of accounting. Kirkpatrick and his organization, initially through the efforts of his son, Jim, and daughter-in-law, Wendy, broadened their understanding of the need for the business objectives and goals to form an integral part of the learning outcomes. This meant meeting with on-the-job supervisors and considering the wider responsibility of stakeholder expectations.
Jack Philips mirrors this to some extent in his model wherein he supports isolating the effects of learning. He also broadens Kirkpatrick’s level 3 and level 4 definitions, and, in tandem with the work of Dr Patti P Philips, Jack’s wife, partner, co-author and author in her own right, Jack Philips supports quantitative assessment using advanced mathematical tools and external comparisons. However, the stakeholder chooses to measure the amount of learning, though, it is always useful for those involved to share a common language, be it a qualitative or quantitative (or mixed!) one.
Measurement – Bloom’s Taxonomy
Possibly the most authoritative source of measurement terminology is that contained in the works of Bloom et al (1956) in what is usually referred to as “Bloom’s Taxonomy.”
It is hard to envision a more relevant source of terms to describe organizational goals, and the fact that many formal schools and colleges still use this source in their curriculum design is testament to its longevity. Satisfying the head, the heart, and the hands must still remain the purpose of the trainer, and of the learner.
“Allo?
I am reminded of the government department official who, when discussing how far their customer service training had progressed, responded to my challenge to ‘prove it’ by agreeing to allow us (I was facilitating a group of about 12 senior officials on management behavior) to put a colleague’s phone on speaker as he called the department reception.
I argued that the response would be a single-word salutation, probably something like “allo?” The official was adamant that his people had passed that stage. The call connected and, “allo?” rang out across the room. The official insisted it must be the new girl, and we should try a different office. We dialed the new number: “allo?” rang out once more and, while I had not made a friend of that official (the laughter of his colleagues assured me of that!), I had made the point that it is a change in behavior not attendance at a workshop that showed whether learning had taken place. And measurement of a change in behavior had taken no more than a single phone call.
The pleasant corollary to this story was that, at the next refreshment break, the official did approach me, somewhat ruefully, and assured me that he would be looking into the training content, and the training provider, of the recent ‘customer service’ programme.
Dr Barry J Cummings ChartFCIPD, Chairman and Managing Partner of Action in Business International (ABI: www.axnbiz.com), is an executive coach and learning & development consultant with over 40 years of experience worldwide in designing and delivering individual and group learning solutions.
Based for almost 20 years in Dubai and serving the entire MENA region, Dr Barry can be reached
ROI Feature