Khan Academy and the Effectiveness of Science Videos

I want to talk about Con Academy. If you haven't heard of it, you should definitely check it out. One guy, KH, has made thousands of videos, over 2,200 at the moment, on everything from math to history and also quite a few videos about science. There are a lot of things to love about KH Academy.

First of all, everything's freely available through YouTube. Khan's goal is to improve the education for all students, and he is an exceptional teacher, an absolutely fantastic speaker. He also emphasizes mastery, which I think is very important. So, students in the program can only move on after completing 10 questions in a row correctly.

He's also developed some activities and software, and he's working with schools to try to humanize the classroom, to try to help improve the quality of student-teacher time. He's pushing the learning well beyond the video area. But I want to put that work aside for a moment and just focus on the millions of viewers who watch his science videos online.

You see, I'm skeptical that they actually promote some meaningful learning, and that's not just a comment about Khan's videos, but a comment on science videos in general. So let me explain that skepticism. I wrote my PhD thesis on how to create films to teach science, specifically physics.

In a typical study, students accessed a website where they took a multiple-choice pre-test. The questions were something like: "Consider a basketball player shooting from the free throw line. After the ball leaves his hand, the force on the ball is A: upwards and constant, B: upwards and decreasing, C: downwards and constant, D: downwards and decreasing, or E: tangent to the path of the ball." Take a minute to think about which answer you would pick and what you think most other people would pick.

After the pre-test, students were randomly assigned to see one of several online videos. The videos contained correct answers to many of the pre-test questions. For example, here is a clip: "Now we'll consider a case where gravitational force is the only force acting on an object." While a juggling ball is in the air, we'll ignore air resistance because it's so small. Only one force acts on the ball throughout its flight; this is the force of gravity, which is constant and downwards.

Gravity accelerates the ball in the downward direction. After being thrown up, a ball travels slower and slower upwards. Its velocity goes through zero, and then it speeds up in the downward direction. The whole time, the ball is accelerating downwards. Then it meets the juggler's hand. Immediately after watching the roughly 10-minute video, the students took exactly the same test.

I also interviewed some students to see what they thought of the video. The most common comments were that it was clear, concise, and easy to understand. The students also increased their confidence in the correctness of their answers compared to the pre-test. So what about how much they learned on the pre-test? The average score was 6.0 out of 26, and after the video, the average was 6.3.

Well, what was going on? I dug deep in interviews to find out. It turned out students did not even correctly remember what was presented in the video that they had seen a few minutes earlier. One recalled, "In the video, it said the ball is slowly decreasing in force, so therefore it stops at one point and then comes down." What was worse, another told me, "It wasn't that hard to pay attention to because I knew already what she was talking about." I hadn't told them about this six out of 26 yet, so I was listening, but I wasn't really paying utmost attention.

What can be done about this? Perhaps videos are just too passive a medium to attract attention for 10 minutes, even when you know you're being interviewed after. But I see the problem a different way. Typically, we think of education as informing students about things they are not aware of, like the French Revolution, for example.

But science presents a different challenge. It is not that students know nothing about it, but that they already have plenty of ideas, most of which are unfortunately wrong. Scientifically speaking, they don't pay attention because they think they know it. Then, when asked what they saw, they falsely remember their own ideas as what was presented.

Is there a way to overcome this? Well, I thought students might pay more attention and be able to understand if their ideas were presented in the video. So one of the other videos involved an actor pretending to be a student with the most common misconceptions, which were illustrated.

"Can you tell me what happens when a single ball goes around once?" "Well, Luke's hand gives the ball a force that drives it upwards against gravity, but as it goes up, force gradually dies away until at the very top it perfectly balances gravity, and then gravity wins and the ball falls down." He then discussed with the other dialogue participant why the misconception didn't work and how the scientific idea differed.

In interviews with students who watched this video, no one used the words clear, concise, or easy to understand. Most often, they said it was confusing. But on the post-test, the average score nearly doubled to 11 out of 26. When asked to rate how much mental effort they invested in watching the videos, students who saw the dialogue with misconceptions averaged a whole point higher than those who saw the explanation without misconceptions.

It seems like it worked. That increased mental effort translated into more learning. So with the Veritasium films, I always try to start with the misconceptions. "How long does it take for the Earth to go around the Sun?" "Well, die obviously a die, yes. This is H representing the Earth, okay? And this represents what do you think?" "Yes, how far apart are they? Like roughly, like roughly that much?"

Is the following statement true or false: humans lived at the same time as dinosaurs? "I'll say true." We really have to tackle these misconceptions somehow. It seems if you just present the correct information, five things happen:

1. Students think they know it.
2. They don't pay their utmost attention.
3. They don't recognize that what is presented is different from what they're already thinking.
4. They don't learn a thing.
5. Finally, perhaps most troublingly, they get more confident in the ideas they were thinking before.

This is a vicious cycle that we have to find a way to break in order to have effective science education. So the Khan videos are amazing and definitely a valuable resource for science educators and perhaps people looking for a bit of review. But for those who are just starting to learn science, I question whether it's going to be that valuable because it doesn't really question their misconceptions.

He: "The Earth doesn't take one day to get around the sun. It's actually..."