Tuesday, March 19, 2019

Is online education effective?

Updated: May 2021

First let’s define what we mean by “effective.” If we mean, is it possible to learn a subject thoroughly in an online class, then yes. If we mean, is it possible to teach a subject well in an online class, then yes again. If we mean, is online education a worthy replacement for bricks and mortar classrooms in every instance, then the answer is a resounding no.

To think about why, let’s not discuss any of those factors that make teaching and learning ineffective anywhere… like poorly constructed lesson plans, lazy teaching, lack of effort on the part of students… and focus on how the online classroom measures up to the physical classroom.

  • The positives: Online can reach geographically diverse students with much more convenience than classroom learning. It levels the playing field for students by removing the back row, limiting the effect of know-it-alls and quick responders, and it gives all students an opportunity to draft considered responses.
  • The neutrals: Data tend to show no significant difference in student achievement and educational outcomes, in general, between online learning and classroom learning.
  • The negatives: Online takes away social learning opportunities, eliminates most hands-on learning, makes group work and group projects difficult, appeals to a few learning styles, and isolates students to the point that the drop-out rate is very high.

Let’s look at those negatives a little more closely. Without social learning opportunities, you can often lose the key driver that results in, “I want to be like that.” Your favorite teacher, the one who inspired you in middle school or high school or college? That’s much less unlikely to happen online. And without hands-on, in-the-room-with-you guidance, some content can’t be taught very well and some not at all. I wouldn’t recommend a surgeon who only had an online degree. Group work and group projects are the cornerstone of applied learning. Sone group work can be done effectively online, but it's those late-night, pizza-delivery, caffeine-powered foxhole-like projects that produce career-launching experiences.

Covid has changed a lot, but it cannot change some simple facts. Learning is not simply transactional, a binary “you get it or you don’t.” Learning comes in layers, or waves, as understanding deepens. Peers help this process immensely, particularly in project work. Auditory and to some extent visual learners may do well online, but other learning styles can suffer. And then there’s the isolation. This is what most students who are attracted to the the lower cost and higher convenience of online seriously underestimate... That dream of working on your degree at the beach is way too often a mirage. They end up doing hours upon hours of work sitting alone, in the dark, in front of a glowing screen.

This is where blended learning comes in. Coursework online, project work in person. This is also where the flipped classroom comes in. Do the learning of facts on your own, then come together to experiment, discuss, test theories, engage with one another. Opening up classrooms on a limited basis should take all this into account. Wise faculty will take full advantage of every face to face opportunity. Go online for transaction. Come together for transformation.

The bottom line is that online education is not for everything, and it’s not for everyone. But where it works, for whom it works, it works very well.

Tuesday, July 17, 2018

What is an academic LMS, really?

As an avid member (mostly reader) of Quora, I occasionally get an opportunity to answer a question that truly interests me. This one I could not stay away from... "What is an academic LMS?"

Here's my answer, with apologies for its slightly subversive point of view...
__________________

An academic LMS is a software system heavily criticized by teachers for requiring them to relearn some of the most fundamental aspects of teaching while taking away much of the personality and spontaneity of the classroom, and grudgingly accepted by students who are accustomed to a much a better user experience in all other areas of their lives but who are willing to put up with this because after all, it’s just school and how can you expect teachers to know about technology?

Efforts to improve on the original model developed by Blackboard and others twenty years ago have been slow and painful because even though the original model is fundamentally flawed (a low-bandwidth online classroom primarily navigated by posting written content and then writing messages back and forth), it is also heavily adopted in a field where change of any kind is always slow and painful.

But bandwidth is not generally such a challenge in the US any more, so change—both incremental and disruptive—is happening. And inevitable. Stay tuned.
__________________

If you like this definition, you can upvote it here!

https://www.quora.com/What-is-academic-LMS/answer/Bryan-Polivka-1

Friday, December 30, 2016

The Seven-Minute Rule

Google "seven-minute rule" and you'll find a number of references. Not all are associated with learning or training, but a number of them are. It's amazing how this has caught on/stuck around/been discovered by others over the years. It's amazing to me, because I established the seven-minute rule for video-based education in the early 1990s. I think I was 18 at the time (Did I do that math right?).

The rule was created for use by a group of content and continuing education developers, and it went something like this:

Seven minutes is as long as you want a segment to run. Period. 

It may seem like an oddly ironclad rule for a relatively arbitrary use, but it actually came from arduous research through mountains of data. Well, it wasn't exactly scientific research, and it wasn't exactly data, but there was a mountain of it and it was arduous. I was hired to produce a daily training program, delivered via satellite, that was formatted like a morning show. We had co-hosts, guests, sets with couches, and regular training segments. One of my first tasks was to wade through an enormous library of 25-minute motivational videos and turn them into short videos that could be played inside our 15-minute segments.

There were hundreds and hundreds of these videos, all by top motivational speakers of the day, from Zig Ziglar to Tony Robbins. What I found amazed me. All of these speakers, and I mean all of them, spoke in easily-editable segments of 4 to 7 minutes in length. This was not a requirement imposed on them, it was their natural flow. They would start a story or a thought, build it up, make a point, wind it down, and transition to the next thought... in 4 to 7 minutes. Every time.

So I got to thinking about that, about this natural cadence of the world's best speakers, and decided it should apply to all our other segments as well. After all, if Steven Covey and Les Brown don't drag a story out, why should we? Thus, the seven-minute rule.

I've used it ever since, in many different educational products and services, and it has held up well. And now you know (as another well-known speaker of yesteryear liked to say) ...the rest of the story.  


Thursday, December 10, 2015

Kirkpatrick's 4 Levels of Evaluation

Captain Kirkpatrick, like Captain Kirk, went where nobody had gone before. He defined the way the training universe thinks about measuring outcomes.

This is number 8 on my list of Learning Theories That Actually Work.

You have a course, a workshop, a program and you need to measure the results in some way. You are probably using Kirkpatrick, maybe without knowing it. Donald Kirkpatrick has had such a heavy influence that in many training circles his levels are used as shorthand, almost a code. "We did a level one, but the level two isn't til next week."

The four levels are:
  1. Reaction. Did they like it?
  2. Learning. Did they learn it?
  3. Behavior. Did they use it?
  4. Results. Did it change anything?

The levels are usually diagramed as steps or as a pyramid. The idea is that each step is more valuable and harder to measure than the previous. The Level 1 "smile sheet" therefore is often regarded as an inferior set of data, almost a necessary evil, and its value downplayed.

But like "Captain Kirkpatrick" above, I think this is a mistake. The emotional component, the gut reaction, matters. In learning, motivation is everything. If learners simply don't like what you're providing, for whatever reason, all your other efforts may be for naught.

Note: people don't dislike training or education just because it's hard. This is a myth. Why do people sign up for difficult professions, play difficult games, solve difficult puzzles? People like difficulty if it is rewarding. So measuring Level 1 is important. If you get that wrong you probably are missing on some far deeper level than you think (see Social Learning Theory and Maslow with a twist for potential clues).

from Human Resource Managment https://goo.gl/aPuAt1
Here's a diagram I like far better, because it puts the learning in the middle where it belongs. You want to measure what you controlled most directly: did they learn it? Then the other three facets give you a handle on other outcomes that are not quite so directly in your control.

So here's Kirkpatrick, stacked against my criteria:

  • It makes sense on the face of it

Mr. Spock likes it. It's logical.

  • It has a solid history in research and practice

This has been around since 1959 and, as mentioned, is just assumed to be bedrock in many quarters. More on Kirkpatrick here. Are there other categories of assessment? Certainly. By all means get more sophisticated. CI 484 Learning Technologies offers a few more levels here, including Anthony Hamblin and others.

  • It's easy to implement

The steps do tend to get more difficult to measure as you go up the presumptive ladder. However, the model itself just calls for you to address each of the different facets. In most cases it takes the extra effort of asking appropriate questions, checking appropriate data, and/or following up a few months down the road. It may be a pain. But it's not hard.

  • I've tried it and it works

The critical thing in implementing Kirkpatrick is to plan ahead. When you define your learning objectives (see Bloom's), consider all four of Kirkpatrick's levels. Consider student's immediate response to be one of your objectives. Then measure direct learning outcomes, related behavior change, and longer-term business or other results. Figure out ahead of time what you will accept as measurements, as evidence, and make that work.

In higher education, we had weekly course-level "smile sheet" surveys (1), end-of-course grades (2), application assignments in which learning had to be applied and the results reported back (3), and we used course-to-course retention as a standard measure of business results (4). In sales training, we tracked engagement in training contests and activities week to week, month after month (1). We had quizzes (2), mystery-shopped our people (3), and of course measured improvement in sales (4).    Were these arbitrary measurements? Probably. Were they the best possible measures? Probably not. But we picked our measures and stuck with them, and each of them drove us to improve. It's easy to forget that at the end of the day, you're really measuring your own work.

Click here to go to back to the first learning theory that actually works

Or...
Click in any order:

1. Gagne's 9 Events (Learning Model)
2. Felder-Silverman Learning Styles Model (Global/Sequential, Visual/Verbal)
3. Social Learning Theory (Role Models)
4. Maslow's Hierarchy (Identity-Level Outcomes)
5. Bloom's Taxonomy (Critical Thinking)
6. Active Learning (Discovery, Flipping the Classroom)
7. Metacognition (Self-Awareness)
8. Kirkpatrick's 4 Levels of Evaluation (Outcome Measurement)

Click here to return to the original post

Wednesday, December 9, 2015

Metacognition

Metacognition is an extremely powerful ally in learning, as necessary a tool for instructional designers as a hammer is for a carpenter. Building it into a learning experience is not hard. Leaving it out is not wise. 

This is number 7 on my list of Learning Theories That Actually Work.


Metacognition means thinking about thinking, and in learning environments that means getting students past thinking about what they are learning and into  thinking about how they are learning--what that learning means to them, how they can use it. .

The following is taken verbatim from the Teaching Excellence in Adult Literacy Fact Sheet (I added the italics):

____________________________________

Metacognitive knowledge refers to what individuals know about themselves as cognitive processors, about different approaches that can be used for learning and problem solving, and about the demands of a particular learning task. Metacognitive regulation refers to adjustments individuals make to their processes to help control their learning, such as planning, information management strategies, comprehension monitoring, de-bugging strategies, and evaluation of progress and goals. [John] Flavell (1979) further divides metacognitive knowledge into three categories:
  • Person variables: What one recognizes about his or her strengths and weaknesses in learning and processing information.
  • Task variables: What one knows or can figure out about the nature of a task and the processing demands required to complete the task—for example, knowledge that it will take more time to read, comprehend, and remember a technical article than it will a similar-length passage from a novel.
  • Strategy variables: The strategies a person has “at the ready” to apply in a flexible way to successfully accomplish a task; for example, knowing how to activate prior knowledge before reading a technical article, using a glossary to look up unfamiliar words, or recognizing that sometimes one has to reread a paragraph several times before it makes sense.
[Jennifer A.] Livingston (1997) provides an example of all three variables: “I know that I (person variable) have difficulty with word problems (task variable), so I will answer the computational problems first and save the word problems for last (strategy variable).”

____________________________________

Here's how metacognition stacks up against my four criteria for making the list:
  • It makes sense on the face of it
This is about taking responsibility my own learning. If I'm doing that, if I'm thinking about how I learn best, modifying my strategies, figuring out how to figure it out, I will learn better. So as a developer of learning, if you build in opportunities for this, or even require it, then you'll increase learning and retention. It's a no-brainer (so to speak).
  • It has a solid history in research and practice
As the excerpt above demonstrates, this has been around a while and as far as I can tell, nobody is debunking anything about it. It's a very real phenomenon, and putting into practice yields good results.
  • It's easy to implement
It is actually far simpler to put this into practice than you might guess. Add discussion questions that require it, assignments that require it, and review sessions that emphasize it... these are simple strategies that take just a little time and thought. One caution: never grade a metacognitive assignment! Explanation below.
  • I've tried it, and it works
The standard higher-education online course has a weekly structure that goes something like this:
  • Start a new week on Monday. 
  • Read/listen/absorb new material by Wednesday. 
  • Post in the discussion forum by Friday. 
  • Turn in a paper by Sunday. 
  • Lather, rinse, repeat.
This can be very a poor model if it's pretty much the whole model, but it can be very good if these Learning Theories That Work are incorporated. My teams have always made metacognition a significant component in higher education... and we call it, the Reflection Paper. (Quick aside: credit here to Barbara Schadlow, who really opened my eyes to the power of this.) We didn't require a Reflection Paper every week, but rather at the beginning, middle, and end of the course, just a few paragraphs describing what you're learning, how it's going, how you plan to use what you're learning, what you need to do differently in order to learn more.

It was never graded, other than a check mark for having completed it. This is important, because you want honesty. The value is in students actually reflecting on their own learning. We also made it private, between the student and the faculty only, which is also important. You don't want people comparing themselves to one another. It should always positioned for the benefit of the learner, which it is, in more than one way. In addition to helping in its own right, it gives teachers the opportunity to help learners with new learning strategies and approaches.

Adding metacognitive activities isn't hard. The concept is research-based. And it works. In my experience, students consistently write Reflection Papers about rededicating themselves to their own efforts, without any prompting, simply because they are taking the time to think about how things are going and how they might go better.

Click here to go to the next learning theory that actually works 

Or...
Click in any order:

1. Gagne's 9 Events (Learning Model)
2. Felder-Silverman Learning Styles Model (Global/Sequential, Visual/Verbal)
3. Social Learning Theory (Role Models)
4. Maslow's Hierarchy (Identity-Level Outcomes)
5. Bloom's Taxonomy (Critical Thinking)
6. Active Learning (Discovery, Flipping the Classroom)
7. Metacognition (Self-Awareness)
8. Kirkpatrick's 4 Levels of Evaluation (Outcome Measurement)

Click here to return to the original post

Tuesday, December 8, 2015

Active Learning

Active learning at root is about learners doing things and thinking about the things they are doing.

This is number 6 on my list of Learning Theories That Actually Work.

The following is verbatim from "Active Learning: Creating Excitement in the Classroom" by Charles Bonwell (Italics below are added by me).
_______________________________

Some of the major characteristics associated with active learning strategies include:
Doing and thinking about doing
  1. Students are involved in more than passive listening
  2. Students are engaged in activities (e.g., reading, discussing, writing)
  3. There is less emphasis placed on information transmission and greater emphasis placed on developing student skills
  4. There is greater emphasis placed on the exploration of attitudes and values
  5. Student motivation is increased (especially for adult learners)
  6. Students can receive immediate feedback from their instructor
  7. Students are involved in higher order thinking (analysis, synthesis, evaluation)
In summary, in the context of the college classroom, active learning involves students in doing things and thinking about the things they are doing
_______________________________

Bonwell wrote the above in the 1990s. Today, readers might find this a bit of a compendium of other learning theories. Just looking at my own list, there are clear parallels to Learning Styles (2 and 4), Maslow's Hierarchy (4), Gagne's 9 Events (6),  Bloom's Taxonomy (7), and Metacognition (thinking about the things they are doing).

But that's all to the good. Because Active Learning does include those things, plus it covers other very effective theories and models. Discovery Learning, in which the learners start with the problem and learn by solving it, is a type of Active Learning. Individualized Instruction is based on Active Learning; it's the logical byproduct of an emphasis on activities, skills, exploration of student's attitudes and values, immediate feedback, and higher-order thinking, which is by definition must be individualized. Take Active Learning to the next level and you get Personalized Instruction, which is the basis of Adaptive Learning. A lot of these methods and models make my list because they are just not that easy to implement yet. But all of them are types of Active Learning, and so all of them are worth exploring.

But let's take Active Learning through my criteria:
  • It makes sense on the face of it
"Doing things and thinking about the things they are doing." How is that not better than sitting and listening?
  • It has a solid history in research and practice
This has the longest history of practice of any learning model. Period. Call it apprenticeship for trades, call it discipleship for religious studies, the ancients wouldn't think of lecturing and then going home to dinner, thinking their work was done. Students did what the master did, that came first. If they could succeed at doing, then the explanation and nuance would come later. You learned this from this movie, which you will remember if you do something: click--> here (language warning)
  • It's easy to implement
Get them to do things. Get them to talk about the things they do. Any questions?
  • I've tried it, and it works
Flipping the classroom--doing the homework first--is active learning. Most online learning courses in higher education do this as part of the weekly strategy. We had a mantra when I had teams building many online courses simultaneously: "It's all about the assignments."

We had lots of pretty pictures, videos, great-looking user interfaces, and truly the best content we could find, sourced directly from the best experts. But our goal was learning, and none of those things mattered if the students didn't do something with them. So our "secret sauce" was in the assignments. We paid very close attention to what we asked the students to do, every week.

When they opened up their online classroom on a Monday, they were very likely already feeling the pressures of the coming week, feeling anything but motivated, maybe even dreading what they had to do this week for their degree program. That was the critical moment, the moment we wanted them to read the objectives and the activities and the assignment and think, "Hey, that's actually interesting. I can see how that will help me. I really want to do that."

So it's not what you tell them or show them, it's what you ask them to DO that matters. That's active learning in a nutshell.

Click here to go to the next learning theory that actually works:

Or...
Click in any order:

1. Gagne's 9 Events (Learning Model)
2. Felder-Silverman Learning Styles Model (Global/Sequential, Visual/Verbal)
3. Social Learning Theory (Role Models)
4. Maslow's Hierarchy (Identity-Level Outcomes)
5. Bloom's Taxonomy (Critical Thinking)
6. Active Learning (Discovery, Flipping the Classroom)
7. Metacognition (Self-Awareness)
8. Kirkpatrick's 4 Levels of Evaluation (Outcome Measurement)

Click here to return to the original post

Monday, December 7, 2015

Bloom's Taxonomy

This is number 5 on my list of Learning Theories That Actually Work

Bloom's original pyramid
Bloom's does a lot of things, but mostly it is a very practical tool for writing learning objectives. If you want to create content and assignments and assessments that achieve the desired end, you need to carefully define the desired end. Enter Bloom's.

The image above is the original pyramid of critical thinking. From bottom to top:

  • Knowledge: I can recite it 
  • Comprehension: I can put in my own words
  • Application: I can apply it to a new situation
  • Analysis: I can break it down into component parts
  • Synthesis: I can reconstruct those parts and combine them with other concepts
  • Evaluation: I can make an informed critical judgment about it.

The graphic below illustrates the modified version by Anderson and Krathwohl. Click on the image to see the whole thing.

Click image to enlarge

This diagram is not nearly so complex as it may at first seem. The idea is to write your learning objectives starting with your content type on the left ("Knowledge Dimension") and then finding the appropriate level of critical thinking on the right ("Cognitive Process Dimension"). The Cognitive side is the old Bloom's pyramid, but with Synthesis dropped (actually folded into Analysis) and "Create" added as the highest level.

By my criteria:
  • It makes sense on the face of it
Your course, lesson, elearning product, and/or assessment will naturally benefit from a little careful planning around learning objectives. Bloom's is a tool to help you get the right objective on a logical scale, from simple to complex.
  • It has a solid history in research and practice
Nothing on my list is more tried and true. Benjamin Bloom chaired the committee that first developed the model, which was published in 1956. Pretty much everybody's been using it since. The updated version by Anderson and Krathwohl was created in 2000. The update is an improvement, and that is not controversial. Krathwohl actually worked with Bloom back in the 1960's.
  • It's easy to implement
I think of Bloom's as a place to shop for active verbs. "At the end of the lesson, the learner will be able to VERB something." So in my practical way of thinking, Bloom's is a thesaurus for writing objectives. The lesson might require little or no critical thinking ("recite"), or it might require a lot ("analyze"). Get the right verb in the objective, use that verb when you design activities and assessments, and you can prove, and improve, learning outcomes.

Click image to enlarge
Above is one list of verbs, from the same site as the previous image. There are many other lists directly linked to a search engine near you. Find a verb, use it, done.

  • I've tried it, and it works
I've found verbs, used them, and included Bloom's in development processes for courses, programs, seminars, workshops, videos, workbooks, you name it. Constructing learning objectives is actually a very small part of learning-product development, because it's so easily done and takes so very little time. But it's arguably the most critical part, because it's the seed from which learning success ultimately grows. So you need to do it well. And Bloom's is a critical asset for doing it well.

Click here to go to the next learning theory that actually works

Or...
Click in any order:

1. Gagne's 9 Events (Learning Model)
2. Felder-Silverman Learning Styles Model (Global/Sequential, Visual/Verbal)
3. Social Learning Theory (Role Models)
4. Maslow's Hierarchy (Identity-Level Outcomes)
5. Bloom's Taxonomy (Critical Thinking)
6. Active Learning (Discovery, Flipping the Classroom)
7. Metacognition (Self-Awareness)
8. Kirkpatrick's 4 Levels of Evaluation (Outcome Measurement)

Click here to return to the original post