This is number 8 on my list of Learning Theories That Actually Work.
The four levels are:
- Reaction. Did they like it?
- Learning. Did they learn it?
- Behavior. Did they use it?
- Results. Did it change anything?
The levels are usually diagramed as steps or as a pyramid. The idea is that each step is more valuable and harder to measure than the previous. The Level 1 "smile sheet" therefore is often regarded as an inferior set of data, almost a necessary evil, and its value downplayed.
But like "Captain Kirkpatrick" above, I think this is a mistake. The emotional component, the gut reaction, matters. In learning, motivation is everything. If learners simply don't like what you're providing, for whatever reason, all your other efforts may be for naught.
Note: people don't dislike training or education just because it's hard. This is a myth. Why do people sign up for difficult professions, play difficult games, solve difficult puzzles? People like difficulty if it is rewarding. So measuring Level 1 is important. If you get that wrong you probably are missing on some far deeper level than you think (see Social Learning Theory and Maslow with a twist for potential clues).
Here's a diagram I like far better, because it puts the learning in the middle where it belongs. You want to measure what you controlled most directly: did they learn it? Then the other three facets give you a handle on other outcomes that are not quite so directly in your control.
So here's Kirkpatrick, stacked against my criteria:
Mr. Spock likes it. It's logical.
This has been around since 1959 and, as mentioned, is just assumed to be bedrock in many quarters. More on Kirkpatrick here. Are there other categories of assessment? Certainly. By all means get more sophisticated. CI 484 Learning Technologies offers a few more levels here, including Anthony Hamblin and others.
The steps do tend to get more difficult to measure as you go up the presumptive ladder. However, the model itself just calls for you to address each of the different facets. In most cases it takes the extra effort of asking appropriate questions, checking appropriate data, and/or following up a few months down the road. It may be a pain. But it's not hard.
The critical thing in implementing Kirkpatrick is to plan ahead. When you define your learning objectives (see Bloom's), consider all four of Kirkpatrick's levels. Consider student's immediate response to be one of your objectives. Then measure direct learning outcomes, related behavior change, and longer-term business or other results. Figure out ahead of time what you will accept as measurements, as evidence, and make that work.
In higher education, we had weekly course-level "smile sheet" surveys (1), end-of-course grades (2), application assignments in which learning had to be applied and the results reported back (3), and we used course-to-course retention as a standard measure of business results (4). In sales training, we tracked engagement in training contests and activities week to week, month after month (1). We had quizzes (2), mystery-shopped our people (3), and of course measured improvement in sales (4). Were these arbitrary measurements? Probably. Were they the best possible measures? Probably not. But we picked our measures and stuck with them, and each of them drove us to improve. It's easy to forget that at the end of the day, you're really measuring your own work.
But like "Captain Kirkpatrick" above, I think this is a mistake. The emotional component, the gut reaction, matters. In learning, motivation is everything. If learners simply don't like what you're providing, for whatever reason, all your other efforts may be for naught.
Note: people don't dislike training or education just because it's hard. This is a myth. Why do people sign up for difficult professions, play difficult games, solve difficult puzzles? People like difficulty if it is rewarding. So measuring Level 1 is important. If you get that wrong you probably are missing on some far deeper level than you think (see Social Learning Theory and Maslow with a twist for potential clues).
from Human Resource Managment https://goo.gl/aPuAt1 |
So here's Kirkpatrick, stacked against my criteria:
- It makes sense on the face of it
Mr. Spock likes it. It's logical.
- It has a solid history in research and practice
This has been around since 1959 and, as mentioned, is just assumed to be bedrock in many quarters. More on Kirkpatrick here. Are there other categories of assessment? Certainly. By all means get more sophisticated. CI 484 Learning Technologies offers a few more levels here, including Anthony Hamblin and others.
- It's easy to implement
The steps do tend to get more difficult to measure as you go up the presumptive ladder. However, the model itself just calls for you to address each of the different facets. In most cases it takes the extra effort of asking appropriate questions, checking appropriate data, and/or following up a few months down the road. It may be a pain. But it's not hard.
- I've tried it and it works
The critical thing in implementing Kirkpatrick is to plan ahead. When you define your learning objectives (see Bloom's), consider all four of Kirkpatrick's levels. Consider student's immediate response to be one of your objectives. Then measure direct learning outcomes, related behavior change, and longer-term business or other results. Figure out ahead of time what you will accept as measurements, as evidence, and make that work.
In higher education, we had weekly course-level "smile sheet" surveys (1), end-of-course grades (2), application assignments in which learning had to be applied and the results reported back (3), and we used course-to-course retention as a standard measure of business results (4). In sales training, we tracked engagement in training contests and activities week to week, month after month (1). We had quizzes (2), mystery-shopped our people (3), and of course measured improvement in sales (4). Were these arbitrary measurements? Probably. Were they the best possible measures? Probably not. But we picked our measures and stuck with them, and each of them drove us to improve. It's easy to forget that at the end of the day, you're really measuring your own work.
Click here to go to back to the first learning theory that actually works
Or...
Click in any order:
1. Gagne's 9 Events (Learning Model)
2. Felder-Silverman Learning Styles Model (Global/Sequential, Visual/Verbal)
3. Social Learning Theory (Role Models)
4. Maslow's Hierarchy (Identity-Level Outcomes)
5. Bloom's Taxonomy (Critical Thinking)
6. Active Learning (Discovery, Flipping the Classroom)
7. Metacognition (Self-Awareness)
8. Kirkpatrick's 4 Levels of Evaluation (Outcome Measurement)
Click here to return to the original post
Click in any order:
1. Gagne's 9 Events (Learning Model)
2. Felder-Silverman Learning Styles Model (Global/Sequential, Visual/Verbal)
3. Social Learning Theory (Role Models)
4. Maslow's Hierarchy (Identity-Level Outcomes)
5. Bloom's Taxonomy (Critical Thinking)
6. Active Learning (Discovery, Flipping the Classroom)
7. Metacognition (Self-Awareness)
8. Kirkpatrick's 4 Levels of Evaluation (Outcome Measurement)
Click here to return to the original post
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.