Kirkpatrick's 4 Levels of Evaluation

Captain Kirkpatrick, like Captain Kirk, went where nobody had gone before. He defined the way the training universe thinks about measuring outcomes.

This is number 8 on my list of Learning Theories That Actually Work.

You have a course, a workshop, a program and you need to measure the results in some way. You are probably using Kirkpatrick, maybe without knowing it. Donald Kirkpatrick has had such a heavy influence that in many training circles his levels are used as shorthand, almost a code. "We did a level one, but the level two isn't til next week."

The four levels are:

1. Reaction. Did they like it?
2. Learning. Did they learn it?
3. Behavior. Did they use it?
4. Results. Did it change anything?

The levels are usually diagramed as steps or as a pyramid. The idea is that each step is more valuable and harder to measure than the previous. The Level 1 "smile sheet" therefore is often regarded as an inferior set of data, almost a necessary evil, and its value downplayed.

But like "Captain Kirkpatrick" above, I think this is a mistake. The emotional component, the gut reaction, matters. In learning, motivation is everything. If learners simply don't like what you're providing, for whatever reason, all your other efforts may be for naught.

Note: people don't dislike training or education just because it's hard. This is a myth. Why do people sign up for difficult professions, play difficult games, solve difficult puzzles? People like difficulty if it is rewarding. So measuring Level 1 is important. If you get that wrong you probably are missing on some far deeper level than you think (see Social Learning Theory and Maslow with a twist for potential clues).

from Human Resource Managment https://goo.gl/aPuAt1

Here's a diagram I like far better, because it puts the learning in the middle where it belongs. You want to measure what you controlled most directly: did they learn it? Then the other three facets give you a handle on other outcomes that are not quite so directly in your control.

So here's Kirkpatrick, stacked against my criteria:

• It makes sense on the face of it

Mr. Spock likes it. It's logical.

• It has a solid history in research and practice

This has been around since 1959 and, as mentioned, is just assumed to be bedrock in many quarters. More on Kirkpatrick here. Are there other categories of assessment? Certainly. By all means get more sophisticated. CI 484 Learning Technologies offers a few more levels here, including Anthony Hamblin and others.

• It's easy to implement

The steps do tend to get more difficult to measure as you go up the presumptive ladder. However, the model itself just calls for you to address each of the different facets. In most cases it takes the extra effort of asking appropriate questions, checking appropriate data, and/or following up a few months down the road. It may be a pain. But it's not hard.

• I've tried it and it works

The critical thing in implementing Kirkpatrick is to plan ahead. When you define your learning objectives (see Bloom's), consider all four of Kirkpatrick's levels. Consider student's immediate response to be one of your objectives. Then measure direct learning outcomes, related behavior change, and longer-term business or other results. Figure out ahead of time what you will accept as measurements, as evidence, and make that work.

In higher education, we had weekly course-level "smile sheet" surveys (1), end-of-course grades (2), application assignments in which learning had to be applied and the results reported back (3), and we used course-to-course retention as a standard measure of business results (4). In sales training, we tracked engagement in training contests and activities week to week, month after month (1). We had quizzes (2), mystery-shopped our people (3), and of course measured improvement in sales (4).    Were these arbitrary measurements? Probably. Were they the best possible measures? Probably not. But we picked our measures and stuck with them, and each of them drove us to improve. It's easy to forget that at the end of the day, you're really measuring your own work.

Click here to go to back to the first learning theory that actually works

Or...
Click in any order:

1. Gagne's 9 Events (Learning Model)
2. Felder-Silverman Learning Styles Model (Global/Sequential, Visual/Verbal)
3. Social Learning Theory (Role Models)
4. Maslow's Hierarchy (Identity-Level Outcomes)
5. Bloom's Taxonomy (Critical Thinking)
6. Active Learning (Discovery, Flipping the Classroom)
7. Metacognition (Self-Awareness)
8. Kirkpatrick's 4 Levels of Evaluation (Outcome Measurement)

Click here to return to the original post