BY JOSHUA MATHEWS-AILSWORTH
EVALUATING YOUR TEAM
Nearly sixty years ago, Donald Kirkpatrick developed the now ubiquitous four-level training evaluation model. The four levels were: reaction, learning, behavior, and results. Each level increased in complexity from a measurement standpoint, while providing even richer information about the value and impact of training programs. Other theorists have since suggested adding a fifth level, such as ROI, but Kirkpatrick’s original four levels are a mainstay in training evaluation to this day.
The first level of Kirkpatrick’s model, reaction, is the most widely used level. Reaction refers, in its essence, to a satisfaction evaluation. Years ago, these were often referred to as ‘smile sheets’. Since the reaction evaluation is easy to design, deploy, and analyze, most training programs today utilize a version of a reaction evaluation. However, many training programs fail to design their reaction evaluation in ways that provide meaningful, useable data about the learner experience. In addition, the evaluations fail to at least touch, albeit briefly, on other levels of evaluation to provide a richer set of data.
Reaction level evaluations usually include three sections: instructions for learners, a set of Likert scale, fixed-choice response questions and a set of open-ended, free text response questions. Instructions are the simplest section of the evaluation to design. Instructions should tell learners several key elements: what is the evaluation about, why is the evaluation important, how long will it take to complete the evaluation, and how will their information be analyzed and shared. Likert scale questions are popular because you can receive easily quantifiable data to quickly understand the reaction of learners. Open-ended questions provide the opposite: qualitative data that must be read and searched for trends, but which can provide richer information than a single, fixed-response question. However, designing meaningful Likert scale and open-ended questions can be difficult.
The biggest mistake most survey designers make when writing reaction evaluation questions is focusing questions on the facilitator, program, materials, or facilities rather than the learner. Common Likert scale reaction questions include:
• The learning objectives were clearly defined
• The pace of the course was appropriate
• The facilitator used various instructional methods
DESIGNING EFFECTIVE
REACTION LEVEL
SURVEYS