Khan Academy Ed Talks with Adam Green, PhD - Wednesday, August 18

Hello and welcome to Ed Talks with Khan Academy, where we talk education with a variety of experts in the field. Today, I am excited to talk to my own teammate Dr. Adam Green about new content that we have just released on Khan Academy for the start of the new school year. Before we get going, I want to remind folks that we rely on donations from folks like you to keep our work going, and you can find a spot to donate right at the top of our Khan Academy page or by going to khanacademy.org/donate. We appreciate as much or as little as you can give, so any amount helps us continue to do this work.

I also want to recognize a few companies that have really helped us during the pandemic to make sure that we can continue to support education, and that includes Bank of America, AT&T, google.org, Novartis, Fastly, and General Motors. As I said, today I am happy to welcome Adam Green to talk about our new content. If you want to catch up on anything you've missed, you can catch us on our podcast where you're able to catch an audio version of our conversation later. So with that, welcome Adam.

Thanks, Kristen. Super happy to be here and to talk about all the work the content team's been doing over the last year. So I can just jump right into it and yeah, get inside!

Let's do it! Tell me what you've got new.

Awesome! Well, I'll focus on some of the U.S. content first, specifically science courses and new math courses. I'll start off by talking about science—super exciting there! We have brand new, totally redesigned versions of several courses, including high school biology, high school physics, and AP Physics 1. In addition to that, we've added a lot more new content to our new AP Chemistry course and I’m very excited about a brand new course that we just launched: AP Environmental Science. Like I said, I'm excited about all of those things, but I want to talk about two of them in a little bit more detail: high school biology and AP Environmental Science.

It's our high school biology course; it's important to know that since we had a high school biology course on Khan Academy for the last couple of years, this new course is redesigned from the ground up. It's aligned with the Next Generation Science Standards, and it contains two types of practice exercises that are clearly labeled for the first time ever on Khan Academy. We have exercises that are called "Understand Exercises," and then we have other practice exercises called "Apply Exercises." Very exciting to be able to really delineate those two types of expectations that students and teachers can have as they move through the course.

I also want to talk about how this course—actually both high school biology and AP Environmental Science—showcase a fun new aspect of our courses called lesson teasers. These are fun, short little descriptions of the lessons that you can see on the unit pages for the course; they're kind of the Khan Academy equivalent of an Easter egg. When it comes to AP Environmental Science, this is our first-ever AP Environmental Science course, even though College Board has offered that for more than 20 years now. There was a really big redesign in 2020, and this course takes that into account.

So our course was designed by a long-time AP Environmental Science teacher—an incredibly skilled teacher and also a long-time exam reader, so she knows her stuff when she was building this course. It's designed to make sure that students are prepared for that exam and, to help with that, it's clearly aligned to the most up-to-date College Board AP Environmental Science standards. And like I said, it also has those lesson teasers!

So, also talk a little bit about math courses. We always have a lot of new work happening in our math courses. At this back-to-school, so just about a month ago—in July—we had major additions to our high school and AP statistics courses, in addition to additions in our pre-calculus courses, arithmetic, basic geometry, and pre-algebra, as well as some brand new "Get Ready" courses. Those have been incredibly popular, so we added new Get Ready courses for statistics and for AP Statistics and AP Calculus.

I'm really excited about the high school statistics course, so I'll give like a little extra blurb there. That course is now better aligned to Common Core high school statistics and probability standards, and there's new instructional and practice content for standards in new and interesting topics like study design, probability distributions, and expected value. And then when it comes to the videos and exercises, I'm happy to announce that now they go beyond those sort of rote procedures and context-free data that sometimes can make statistics less interesting.

So in this course now, learners kind of both apply concepts and procedures in real-world contexts, can answer questions like, "Should you buy that extended warranty that the salesperson is trying to offer you?" So I'll leave it at that—altogether, there's just a ton of new content. Very excited about it, and the content team has been working really hard to provide this content for these new courses for students all over the U.S. and eventually around the world.

That's awesome! And I know I have seen you and the whole team hard at work for a year creating these courses—basically, so congratulations on all of their launches. Who do you think about when you're designing these courses? How do you think about the audience for them and who they're built for?

Yeah, that's a good question. Historically, Khan Academy really looks at the independent learner, and we're still very focused on making sure that everything we produce is great for that just-in-time learning for anybody out there who just needs a little boost when it comes to biology or pre-calculus or anything else. But these courses in particular, we're designing really with the classroom in mind—so really thinking about the teachers and students in a classroom context and what we can do to help support that learning as best as possible, while still making sure that these courses provide exceptional value for anybody who's just coming in to learn a concept inside or outside of the classroom.

So what does that look like then in application for—to be really also thinking about teachers and what my teachers see or may be responding to in the new courses?

Yeah, really, I think that comes down to standards alignment. I mentioned with the science courses that these are built with the Next Generation Science Standards in mind; all of our AP courses are built directly with the AP standards in mind. They're built by and reviewed by teachers in classrooms teaching these subjects. So all of those components together, we really hope that it's going to supercharge what's going on in classrooms.

So how can people give us feedback on whether these new courses are hitting the mark or not?

I'm really glad you asked that question. Please, everybody out there, give us feedback—especially on these science courses. This is the first time we're designing these courses in this particular way, and you'll notice long-time Khan Academy users will notice that these are a little bit shorter courses than normal. We did that on purpose because we want feedback from teachers and learners out there to tell us what is it that this course could add that would be really helpful for your learning journey in the sciences.

So you'll notice at the top of each course in the actual Khan Academy library, you'll see a call-to-action button that says "Give Feedback Here." Click here for feedback, and you can click on that and fill out a little form about any of our new science courses. All of that information comes right to me; even right now on my second screen here, I'm looking at the latest feedback that we've gotten through this button. You can see right there at the bottom: "Read me a little feedback." Here's from an AP Environmental Science student who says, "The questions and information are amazing! I just find the course a little short. I completed it and wish I would have gotten more mastery points."

So yes, I hear you! They are a little short. I'm so excited that somebody has already gone in and completed the entire AP Environmental Science course, so that's great to hear!

Awesome! I love hearing feedback about how people like our courses. Also, would love feedback about what would you like to see. So yes, these are a little bit on the short side. What would help you have an even better experience in your course? Let us know, and we'll put that on the list of things to add as we continue to work on these courses and release more and more updates in the future months and year or so.

I'm so impressed that someone's actually gone through the whole course already—good for them! So as I've poked into some of the courses, I know it was by style, but there's some new voices here. Tell me a little more about them.

Yeah, there are a lot of new voices in our videos! So yeah, we really had an intentional effort, especially in these science videos, to bring a more diverse set of voices into our video creation process. And no fear out there! Sal is still making videos—he's made videos in our new science content—but we've also brought in more than a dozen new video creators. This is mostly focused on the high school and AP science so far. We're also hot in the middle of creating middle school science for the first time, so that's a little sneak preview for folks on Ed Talks. That middle school science content is coming soon, and we'll be using those same new video creators and even more new video creators in our middle school content.

But like I said, Sal is still in the mix and is still creating great videos for us.

Great! Well, I know when I came on, I was fascinated by the process of how these videos get made and how involved it is. Can you share a little bit of that with folks?

Sure! Yeah, it is a pretty long process, and we do that because we care so much about the content. We want to make sure that we're providing the best, most accurate, most engaging content possible. So, when we're talking about videos, what really happens first is we have our content experts that are working with Khan Academy write up a request for a new video, and inside that request, it'll have things like the standards that that video needs to be aligned to, it'll have any kind of misconceptions that we want to make sure we address, and it’ll have any kind of language that we really want to make sure that students hear.

It may include language that we want to make sure that students don't hear quite yet because the sequencing of the video is such that we want to introduce content and terms at certain times during the course, so it might not quite be ready yet. So we take that exercise request, we hand it over to a new video creator; they create a script. That script comes back to Khan Academy content experts for review; there's back and forth in that scripting process.

Then the video creator creates a first draft of a video, which then comes back again to the Khan Academy experts. Even I get into the mix a little bit here and review that video, make notes, and send that information back to the video content creator. Then that process sort of continues. It usually takes two to three drafts of the actual video after the scripting process before we really align that this video is ready to be published, and that can take a couple of weeks sometimes per video.

Wow! Yeah, I've seen this going back and forth and all of the detailed comments and the deep discussions about the content itself and what's the right way to present it. So congratulations to you and the team; it's really a lot of work I know that's gone into these. Now when I put on my learning science hat, I know that it's not just the videos, but it's that active application and exercises too. So tell us a little bit about the exercises and how you create those.

That's right! Yeah, learning by doing—that's where you really, you know, where the rubber hits the road. So I've been a teacher in the classroom too, and watching videos is great for instruction, but really to solidify that understanding you have to get into the exercises, like you said. So if you thought that the video process was a little bit lengthy and detailed, the exercise writing process is even more detailed because we really, really care about accuracy, scientific accuracy, but also just making sure that the content is aligned to standards and that it's meeting students and teachers where they need to be or we're meeting expectations around the quality of what happens in an exercise for Khan Academy.

So that process has around 15 or 16 steps per every practice exercise. It starts off pretty much the same with one of those requests, and then the editing and the reviewing just keeps on going back and forth. Then if there's images or any math in the question, then that goes through a separate review process, and we just keep on turning back and forth, grinding on it, polishing it, making it the best it can possibly be before we release it to all of our teachers and learners. And then again, feedback—there are feedback buttons even within the exercises, let alone these feedback forms or going through the forums to customer—to our support, our community support staff.

And the content team has an ear to the ground anytime anything comes up, anything confusing, anything that anybody's concerned about. We take that feedback and immediately see if we need to make changes to those exercises or add additional exercises, remove things. So we're always looking for feedback and always looking to make our courses even better.

Great! We had a question from YouTube from Unkeep Handy, who says, "Thanks for the content! I've checked all of the new courses. Great! So please add exercises as well. Do all of our new courses have exercises, or are there some to come, or how can we make sure everyone's seeing all the content that's available?"

Thanks! Yeah, all of our new courses that I've been talking about do have exercises, but not all of the courses on Khan Academy have exercises—especially some of our courses that have been up there for a little while are very heavy on the instructional side. So videos in some cases, articles, and not all of them have exercises, but that is our goal with our small but mighty content team is to eventually make sure that all of our courses that we're showcasing here, and you can see a lot of the ones with the little new buttons next to them—those are all the new things from last month—all of those have exercises, and most of our other courses also have exercises.

But occasionally, you'll be able to pop around and find one that's just videos and articles, and we hope that those also will be useful. But like we were just talking about, really learning by doing is important, so we're hoping to eventually have exercises across all of our courses.

Great! Just to let you know, I'm seeing compliments coming in, so thank you to Squid Nugget Gnaw on YouTube, Yajjaju on Facebook, and Rocio Rosado on Facebook who've all said that they really like the new courses, and Russia specifically says, "Thank you very much for the AP Chemistry! Much improvement!"

So glad to see folks are diving in already and enjoying them!

Yes, Lauren, our content creator for Chemistry will be very happy to hear that—thank you! Yes, she has worked hard on that. So we have lots of viewers outside the U.S. too. What new content do we have that's specific to countries outside the U.S. or languages other than English?

Sure! Yeah, we have a lot of teams working around the world—in some cases creating their own content in their own countries, and in some cases using the U.S. content and localizing it—so translating it and making it available for teachers and students in other countries. So I'll talk a little bit about the first case of some of the new content that's been made in several countries.

For example, India—there's tons of fresh new math and science content for grade 12. And then on the localization side, there's new localizations in India for Hindi, Punjabi, and Marathi coming next year. Hindi and Punjabi are there right now; Marathi will be coming next year. Peru saw updates across math and some science courses last month, and it's notable to say that this is the first time that we're bringing science courses to our Peru site for Khan Academy, so very excited about that as a former scientist especially.

And look for a lot more content coming just in March of next year to Peru. In Brazil, we have new math and Portuguese language arts content in addition to new science courses—especially at the elementary and middle school level. There'll be new high school science courses coming early next year hopefully. And also notable in Brazil is that they're working on the new BNCC high school guidelines and creating math and science courses aligned to those guidelines.

Finally, I'll say that I'm very excited that we just launched our first site in Vietnam earlier this year. In general, for folks thinking about Khan Academy in just the U.S. context, it is global! This Khan Academy experiment that we have going is used in over 190 countries right now, and content is available in 50 languages across the globe. So I'm very excited to be able to talk about those types of statistics.

That is exciting! Yeah, and I have a question for you, Kristen. So we're talking about all these new courses, and I have to keep my content hat on here. I'm curious: What's most exciting to you?

Well, so I always have a soft spot for statistics. It's a course that I've taught in the past at different levels, and I kind of think that we might be doing better just as a global community if we all had a little bit better understanding of statistics. So I'm always excited; I'm happy to see we now have a high school statistics course. So that's really exciting to me, but I also find it difficult to pick one course.

I know there's so much of the work that the team does that brings really good learning science—our understanding of learning—into the courses. So one of the things I like across particularly these new science videos is how much the videos themselves indicate and kind of bring, "Wow! Isn’t the world amazing? Isn't it amazing how just things in science work? How sustainability and adaptability and the whole global ecosystem works together?" The videos really manage to communicate that just excitement about the world, and I love that!

I think it's also really important from a motivation point of view to help keep students excited about what they're learning and where things are. So I always appreciate that. But let me ask you what you like about the new courses.

Wow! Yeah, hard to follow that answer—that's so great! You know, as I mentioned before, I'm a scientist by training. I used to be a research scientist, so I have a very big soft spot in my heart for anything science. But what's really getting me excited here is these new science courses definitely represent a shift in how we're approaching science—I think for the better here. I love the breadth of our math content and how that keeps on growing and keeps on getting better and better.

One thing that I'm looking forward to is really as those things start to come together. So, as you mentioned, like in our statistics courses, we have some stimuli that talk about science; it's in our science courses—we have a lot of math in some of our environmental science and obviously with AP Physics especially, but even in chemistry and some of our other courses, we're really getting into math. So I like to look at everything in a more holistic sort of way in that there's not one particular course that I love more than any other course. But really as a whole, all of our offerings together, I hope come together to serve our mission and really give a world-class education to anyone, anywhere, who can have access to Khan Academy. That's what keeps me and the rest of the content team moving.

Great! We still have a little bit of time for a little more behind-the-curtain peaks at these things, so I thought it might be fun to just talk a little bit about—unsurprisingly—learning science and how we fit those in. So to help folks who are out there understand a little bit more about how these things work, Adam and I over the past year have started a project to start thinking about how we make sure things like our hints and our rationales are aligned to what education research says they should be and how we can write hints in the best way to help students really get the support they need to answer questions.

But can you just talk a little bit about hints and rationales and why those are important in the exercises that we create?

Sure! Yeah, absolutely. I could talk about that for much longer than we actually have time for, so I'll try to keep it short. But overall, the thing that I really love about hints and rationales—these are things that come up when, right on the screen, instant feedback when a student answers a question. If they answer the question correctly, they can still see all of these rationales and hints that come up. Or if they answer a question incorrectly, they can get a hint that starts to stepwise work them through each part of the question.

Or a rationale, which is a different form of feedback that is tagged exactly to each question answer choice. So usually, if there's a question that's more procedural, where you need to work through step-by-step, we'll have a hint to help you work through step-by-step. But if the question isn't that kind of procedural question and it's more conceptual or more applied, then we'll give more exact feedback at the item answer choice level, and that's called a rationale.

But either way that you get that feedback, I think the most important thing is that students can go through questions at their own pace. As they get things correct and incorrect, they can see the rationale for the correct answer or the hint—so the way to get the correct answer if they're having trouble. And then even more powerfully, they can just go through the exercise again; they'll see slightly different questions each time they go through and then they'll get a chance to apply that knowledge that maybe they learned in a rationale or in a hint to another shot and just to see if they can solidify that concept in their mind.

So I think between getting that just-in-time information about what's going on in that question and then the opportunity to have that growth mindset to persevere and try again makes the practice exercises on Khan very powerful.

Absolutely! So if you're a student out there, or if you're a teacher or a parent coaching a student, and you're thinking, "Should I ask for a hint or not?" here's a little tip to walk you through. If you're looking at the question and you think, "I have no idea how to start to answer this," take a hint. If you're thinking, "I might have an idea," go ahead and try that—go ahead and work through it on your own.

Actually, in the research, it shows that if you're trying to answer a question and you give it a try, that's a good approach to take before taking a hint. And then if you don't get it right, go ahead and look at the hint or the rationale, depending on what kind of question it is—if it's that procedural kind of question or not. But then keep asking yourself those questions, and then take a minute, as Adam said, to really think about that hint or that rationale and maybe explain it to yourself in your own words.

Like, "Why, oh yeah, that's the next step because I needed to get the variables on one side," or "Oh, that is the correct answer because..." Say it in your own words; that actually really will help you remember it later. So a little tip for those that are out there learning and thinking about new questions and how to work through some of our content and where things are.

And I'll do a little shout-out here to some teacher tools because teachers might be listening. The one thing that I said earlier is you get different questions every time you go through—it's incredibly powerful to be able to have that second chance. Also, for teachers though, if you want to have all of your students answer exactly the same questions so that you can go over those in class and know exactly that every student has seen each one of those questions, you have that option with our teacher tools.

When you make an assignment to all of your students, you can check a little box that says "Show all of my students different questions" or "Show all my students the same questions," and that can be really useful when you're doing a group discussion in classrooms so that everybody has seen the same things and you can go through those hints and rationales as a group as a different way to kind of get that learning experience.

Great! We've got a question coming in from Facebook. David Clavercamp wants to know, "How might Khan Academy be used to facilitate support of English Language Learners in the future or teachers of English as a second language?"

Yeah! Would you like me to take that, Kristen?

Go for it, please chime in!

So we have our English Language Arts courses up right now, which is a little bit different of a question than I think you're asking. We're working on trying to figure out the best ways to be able to support English Language Learners on the site. At the moment, we're trying to make sure that all the content that's been up for a little while—this is not true for the new science courses that just came out quite yet—but to make sure that all of those are also available in Spanish to at least help English Language Learners that are starting from a Spanish base.

So that you can see all of our content both in Spanish and in English quite easily—that’s our goal. We're not quite there yet for some of these new courses, as I mentioned, but hopefully, that will help with some of that transition, along with looking at our ELA or English Language Arts content. Hopefully, at least in the short term, that'll help learners like that. We're also very careful to make sure that our reading levels for each of the grades are matched, to make sure that we're keeping reading levels where they should be for sixth graders, seventh graders, ninth graders, tenth graders, etc.

So we're also hoping that that will help folks that are learning English while also learning science and math on Khan Academy.

Great! Thank you! One more question. I'll start, and then you can follow up. Mr. New Blocks on YouTube has a question: "Any tips on what to do if you get bored during class?" This is obviously a question that we get a lot from both students who are looking for help and then teachers who are looking to help students of how to stay motivated and engaged.

There are a couple of things that I'll say that I think are important. One is think about how might this apply to something that you're interested in. So think about, is there something I can take away here that's gonna help me in a hobby that I'm interested in? Is there something that I can take away here that's going to help me in a future job that I might like? Or is there something here that just is really interesting about how the world works and I can take and understand better how something works—how my bicycle tires and the knee pedaling help move those tires? There's a whole bunch of interesting physics behind that.

But think about—like I said—thinking about how the things you're learning are applying to the real world and are kind of exciting. One of the things that Sal always talks about that's amazing is we have so much collected knowledge in our textbooks or in the videos that we've created and the exercises that we're doing, and what a privilege it is for us to have all of this just readily available.

Think of all of the people who've contributed to all of this knowledge and where that is can help kind of think about how exciting it is to have the opportunity to work through some of it. So that's some of the big pieces!

All right, here's a really specific though, kind of more behavioral. If you're familiar with behavioral psychology, have something where you can check in every so often. You might set it for five minutes; you might set it for ten minutes. It just gives you a reminder, and whenever you see it, you think, "Am I paying attention? Is my mind wandering?" That can actually help train you to make sure you're staying in attention and being able to focus better and help teach you that way.

So lots of different things that might help you stay focused. Do you have any thoughts for your stay on what happens if you get bored in class?

Yeah, and I've been there—maybe not so long ago—in a boring class! I really like the idea of applying—like I'm thinking about what's going on up on the board, in the book, in the video, in class, or even on Khan Academy. It can relate to what's going on in your life and be more relevant. But the spin that I'll put on that is sometimes it's for me—it's really interesting to think about connections between courses.

So if I'm in a chemistry course or listening to a chemistry lecture, I'm a biologist at heart, so I love biology, and I'm thinking about how this chemistry that I'm learning relates to the biology that I love. Or how does this statistical analysis that I find really interesting—how is that being applied—could be applied to a physics problem? Even not a specific physics problem, but a general concept in physics. How could these two things relate? You can kind of play those games in your mind a little bit—taking a subject or a topic that you're interested in, whether it's an academic topic or something around sports or reading, something that you've read or some kind of hobby like Kristin mentioned—and apply that to the topic at hand.

That definitely has kept me interested in the past.

Great! So one more plug: we would love to hear from you about how you find these new courses! Please give us feedback. Adam is sitting ready to read it. So please do, as you're going through the courses, let us know how you find them and what else we can do to make them better in particular.

So thank you, Adam, for joining us today! A reminder for folks that tomorrow we're going to have Sophie Bossmany, who works with our Khan Academy Kids app, talk about content for kids who are ages two through eight and what's new there. So looking forward to that! Thank you all for joining today. Have a good day!