AI in Education: Opportunities + Pitfalls

All right, welcome everyone! This is Jeremy Schiefling with Khan Academy. I am so thrilled to welcome you back for round two of our AI and education webinar series this summer. I know that this summer time is your time, and so I apologize for intruding upon it; but I'm so thrilled that you decided to join us because we have a rock star panel here.

We have a couple of incredible educators from across the universe. We’ve got Emily and Kelly coming to you from the Khan World School in partnership with ASU. They're sweating it out right now down in Phoenix, as you probably heard about. We have Madeline coming from the School City of Hobart in Indiana, one of our amazing Khan Academy district partners and one of the pioneers in using Conmigo in the classroom. Then we have Malia from our OG Khan lab school right here in Mountain View, California.

Um, so we have lots of different perspectives represented. And then, of course, a man who needs no introduction, Sal Khan himself, is on the line, ladies and gentlemen, here to tell you about Conmigo and what the future looks like in this space. Now that being said, before we dive in, we want to get to know you a little bit better. So what I'm going to do is launch a poll right now, and I’d love to hear from you: where are you coming from occupationally? Are you a classroom teacher, a counselor, an instructional coach? Let us know right in the chat.

Number two, let us know the kinds of students you serve. Are you on the older end of the spectrum, like so many of the educators here, or maybe you’re Early Childhood? I was a former kindergarten teacher; I would love to hear from that perspective as well. Finally, let us know your primary subject area. We definitely have a lot of really strong humanities folks, but Sal himself has volunteered to speak directly to the math and science angles if you're curious as well.

So with about five seconds left—five, four, three, two, and one—we'll end the poll and let's share these results out. So, as you can see, a ton of classroom teachers, which is awesome! But also some admins, instructional coaches, librarians, and curriculum developers. Love to see that! We have a pretty even mix across elementary, middle, and high, with a little sort of preference for the latter, but even some folks all the way to college and adult education, which is fantastic.

Finally, as you might expect, we've got a strong showing from the math educators out there but lots of great folks in science, ELA, social studies, and beyond. Now, with that said, quick ground rules before we dive in: this session is going to be recorded. That’s the number one question that everyone has asked in every webinar ever since the pandemic, and yes, you will absolutely get a copy sent directly to your email afterwards. However, what’s more, because you joined live and shared some of your summertime with us, you’re going to receive a free professional development certificate mailed to you right after the session and be entered into a drawing for 12 free months of Conmigo, our brand new AI guide.

To make the most of the session, because you are here live, what we would love for you to do, more than anything else, is come to the Q&A and do two things. Number one, whatever questions you have about AI and education, no matter how big, no matter how small, no matter how spicy, feel free to put those in right now and throughout the session. Number two, just as importantly, support your fellow educators by upvoting the ones that you’re excited about. For anyone who came to our last session, you know that we took those primarily in the order of upvotes. So if you’ve got big, juicy questions, feel free to give that that big plus sign; we will be sure to take those.

With all of that said, let’s start with a question that so many people asked in our pre-survey, which is: how do you believe AI is going to actually impact education? I know this is a little bit crystal ball-like in the sense that we’re thinking about where things are headed versus think where things are here in the summer of 2023. For our panelists, if you want to take that question, if you’re excited to dive into it, just go ahead and raise your hand and I will call on you to take it.

Sal, it sounds like you might have something you want to share right off the top.

Oh sure! And you know, obviously none of us have a crystal ball or can predict the future, but in some ways I view, you know, my answer to that question is also the answer to how I thought about edtech even pre-generative AI. It’s never about somehow—it’s never about the technology; it’s really about what the pedagogical goals that we all want to do. I think for time immemorial, well before AI, well before the internet and computers, all educators have always recognized that the gold standard is some form of differentiation.

There’s a lot of literature and honestly common sense behind mastery learning. If a student hasn’t fully mastered a concept, they should always have the opportunity to get their, you know, their understanding of decimals from an 80 to a 95. Because if they don’t, they’re really going to struggle with algebra when those equations involve decimals. This is true in pretty much every subject area.

One of the ways that would have been addressed, say, 50 years ago is by having better student-teacher ratios, having teaching assistants in the room. Middle-class, upper-middle-class affluent families can afford tutors, or maybe sometimes the parents will do the tutoring themselves to provide that level of personalization. Obviously, you know, I'm a fan of the more resources we can throw at the students to enable them to reach their potential, the better; but there are some in our control and there are some that aren’t in our control. We can’t control the student-teacher ratio; we can’t control whether or not a student has tutoring at home or whether there’s a teaching assistant in the class. If there is all of those things—awesome!

So, our view at Khan Academy has always been, how can we use technology to at least start to approximate some of those things? That’s where classic Khan, let’s call it, has been. Let’s give people practice at their own time and space, initially in math and science, and then we’ve also started moving the humanities. ELA is not that far away where someone can learn at their own time pace—very deep item banks so that if they haven’t learned it, they can keep practicing it; spaced repetition, spiral review to make sure that they’ve really gotten to proficiency and mastering things.

There's on-demand video and hints so that people can get unstuck, hopefully; and all of that gets reported to the teacher so that they can get better information on how they can work with their students. They don’t have to give a one-size-fits-all for every student. That hey, those 10 students have got it; they can keep working. These 10 students could use a more focused intervention. Then after I’m done with those 10, these five over here could use a different focus intervention. So that’s kind of been the vision.

I think generative AI just takes that to another level. Where now if a student has, you know, the videos aren’t doing enough for the student, the exercise hints aren’t doing it, the articles, they can ask questions. Sometimes questions they would have been afraid to ask teachers; I get that answer. But in many cases, they might have been willing to ask the teacher, but the teacher can’t go to 30 kids who are all asking different questions.

The generative AI can act more like a teaching assistant. You know, the best that edtech could do pre-generative AI is give you all these dashboards that look like spreadsheets and you have to kind of like, “Okay, which kids are struggling?” And now, you know, we’re investing in modalities where you can have a conversation with the data, so to speak. We think that’ll make it much, much easier for teachers.

Then there’s a whole series of workflows which we hadn’t thought about before because we didn’t think it was possible, but now you can where teachers spend a ton of time doing things like writing lesson plans, grading papers, creating rubrics, progress reports, and those are all areas where we think generative AI could, you know, substantially save teachers time.

So long answer, all my answers aren’t going to be this long, but I hope in five years we’re in a world where students are feeling much more supported, teachers are feeling much more supported, they have much more time and energy for themselves so they don’t burn out, and more time and energy to do kind of the things why I think most teachers went into the profession—kind of more time for those really amazing, engaging moments with students.

I'll say one last thing. I also think the AI being able to have that loop between assignment, rubric, lesson creation, supporting students that say they work on an essay and then being able to report back to a teacher—not only is that going to be really supportive of all of the parties in that mix, but I actually think it can address the chat GPT cheating issue. Where if you assign an essay and a student—and a student, let’s see—the ring on Conmigo, and they’re doing brainstorming with the AI, the AI’s not doing it for them, but it’s riffing with them; it’s acting as a guide.

Then the AI can report back to the teacher saying, “Yes, we spent this much time, we did this, we changed our thesis, but eventually got there and based on the rubric, I’d give it an initial 85. What do you think, teacher?” You could feel confident that that student didn’t cheat. But if a student goes to ChatGPT, has that essay generated, and just copies and pastes that into Conmigo, then Conmigo itself will say, “Hey, this looks fishy; there’s no process here.”

I think, you know, honestly, because of bandwidth issues, it was already very expensive and time-consuming to grade 30 papers on the same topic and for middle school students, so you definitely never had time to really go into the process. But now we can hopefully streamline that.

Love that! Thank you for starting us off, Sal! Let’s go next to Emily over at the Khan World School. Emily, what’s your take on where this whole education and AI world is headed?

Sal offered a lot of those, but I definitely see it as a tutoring option to help get at students more individually, on an individual level. I'm an English teacher by trade, and so I'm just always thinking about, well, if they want to use AI as a way to generate a first draft, I think the power of writing comes in revision anyway. So if they're going to learn how to revise, how to make ideas better, how to get rid of very formulaic-type writing, that can be very helpful at the high school level.

If they're using it as a tool, and not as a method of “this is doing my work for me,” but teaching them those subtle distinctions about how we use tools to help us, but we are the ones that can then flesh that out and create something better and stronger, and so I think in that way it’s very helpful.

In the same way, I’ll also mention that it becomes a way to create time. There are shortcuts that are helpful for teachers. You don’t always have to write the same email or you’ve created a tool that can also save you time. But I also think it can be like that tutor, but the power of a teacher really comes in the relationship. AI will never circumvent a relationship, right? That will always be there for a teacher. That will always be something that in those moments when you can get at that individual student, which maybe the AI will help you to have more of those individual moments because then those students can be working at a computer, and then you can use that time to pull kids aside and have those one-on-one interactions that are especially valuable, and that will create those really strong teachable moments.

So that’s my hope. I am taking a very positive and optimistic view of what AI can do for educators. I certainly hope that it doesn’t remove educators from the field because education is such a powerful tool. But then again, we have a teacher crisis, at least in Arizona, where I live, and so maybe there is more need for that if we’re not able to recruit teachers, but that kind of breaks my heart at the same time. So those are some of my thoughts as I think about the power and future of AI.

Wow! And then, um, let’s go to Madeline as for the last sort of answer on this one; we’ll come to Malia for the next question because that’s juicy as well. Madeline, any thoughts here?

Absolutely! So for just a little bit of context, I am an English teacher and the English department chair at Hobart High School, and we do a lot of partnerships with Early College programs. So we have a lot of students who are juniors, sophomores, who are taking college-level classes. That’s amazing for them and wonderful for them, but we’re in a very blue-collar town. I have a lot of first-generation college students and a lot of kids who have to work to help their parents pay the rent or the mortgage, and they have practice for their multiple sports on top of that because the competition among students to be able to get into colleges, as the cost is getting higher and higher, it’s just getting all the more expensive for them. There's so much fear about not being able to be enough.

So one of the areas that I think hasn’t been touched on a lot in this conversation so far has been on how is it going to help our students. It kind of—I guess I am building off of what Sal said a little bit in terms of leveling the playing field.

I was looking over my students' Conmigo chat history, and I was watching the different time frames in which they were logging in and using it. Many of them were using it after I would have gone to bed or when I would not be available for my students. I used to feel this paranoia about needing to check my email all the time in my off-hours just to make sure that if my kids had a question, you know, God forbid it was the night before the essay was due. But let's all be real; we know our kids. We know that they’re going to do that. But when I'm not reachable, I want to know that they have the ability to get their needs met, so this is a great way to be able to do that.

Yeah, I think that's a great call of that. I think even the world's greatest educator can't be by their student’s side at every single moment, nor should we expect them to. To have that resource there in that moment when they need it—that's pretty powerful.

Okay, speaking of the power of AI and then also a little bit of the dark side, I’ve got to tell you—it is overwhelming in the Q&A. The top two questions are from Wendy: how do students use AI without cheating? and from Erica: how best can we detect students using AI inappropriately and then handle those situations?

For our panel, I know we all have different experiences with cheating or with trying to prevent that. Thoughts about how students can, or teachers can grapple with this reality? Let’s go to Madeline; I know you’ve seen this quite a bit, you probably have some strong thoughts here.

I have, yeah. So when we were doing our beta test with Conmigo, actually, I did run into this issue a little bit where we’d had conversations about responsible use with our students, and they didn't always want to follow those parameters. I think the reality that we have to face with AI is it’s a crisis that is not entirely unfamiliar to us, right? We went through this same thing when Google came out in the field of education. People just concluded, like, “Oh God, no kid is ever going to have to memorize anything anymore.” What did we have to do? We had to change the way that we assigned content, right? To focus less on rote memorization. I think we’re going to see that same kind of shift here.

So one thing that my district has done, just for context, I’ve been working with my superintendent a lot and kind of been spearheading our AI initiative in our district. I’m going to share my screen really quick. Uh, let’s see; I want to share this.

One thing that our district built to help with this issue is we built a rubric. It was based off of Matt Miller and his AI for Educators book. Our mascot is the Bricky; I know that won’t make sense to a lot of you, but just go with me on this. So we have a rubric that we’ve made where we have human-generated at the bottom and AI-generated at the top.

We have descriptors in here for the level to which AI would be allowed to be used. So whenever an assignment would be given, we would be able to say, “Okay kids, this is the level of comfort we will have with you using AI versus human-generated content.”

So, you know, for instance here, the one that I’m on right now talks about using AI to help with brainstorming but then having to go find sources on their own that address that same subject. Through having access to chat history with Conmigo, that will help a lot in terms of mitigating those concerns.

Awesome! I love that and now that I will follow up with you afterward and see if we can borrow that to share with others. I saw everyone taking screenshots as you were sharing that; that was so cool!

Let’s go next to Kelly. Kelly, what’s your take on this question?

Yeah, hi everyone! You know, I think this question always comes up, and I think personally, you know, we keep doing the same things we always do to prevent cheating, regardless of the source. You know, kids are going to cheat if they’re going to cheat; they’re going to find a way.

You know, I think, you know— and I love what Madeline just shared—part of what I think is also educating everyone around the student too. Making sure everyone on, you know, your teaching team knows how to use these tools and knows about these options. Parents—making sure parents know, you know, perhaps having some informative nights and materials that can go out to families so they know how to work with their children on some of these platforms.

So, you know, I think it’s just new, and I think you do the same thing we always do, like designing assignments to circumvent cheating as well. You know, making sure there are steps in the process that they have to go through that in the end result in their own thinking. There too.

I also love, you know, the thing about something like Conmigo that I like is that back-end data too. So it’s not if you’re promoting the use of it, teaching them how to use it, including it in the assignment; perhaps we can be more likely to not control it necessarily, but be able to see later in that data how students are using it too. So I think it’s all those things combined with what we’re already doing and right now the biggest part of it is just making sure educating all of our staff and families, you know, and parents and people working with students on how to use it properly.

That’s awesome! Then finally, let’s go to Sal. Sal, I’d love for you to speak to this, and then I would love to dive into math next because I know that’s a hot topic as well.

Yeah, I’ll just say a little bit from the technology side of things. You know, anyone who’s telling you—and I know there’s a lot of folks out there who are claiming that they’re creating generative AI writing detectors, the same way that you have plagiarism detectors—anyone who says that to you is selling you snake oil. So be very suspicious of anything else that comes out of that person’s mouth.

You know, there are ways that you might be able to do it in very basic ways that might detect it in some cases, but we know our students. Your students, it’s going to be very easy to circumvent that, and I think we should all operate from the assumption that it’s essentially undetectable.

I think that’s the only safe assumption. So the option that we all have is, you know, the non-tech options are, you know, hopefully some type of an honor code could be used in certain cases. In certain cases, I’m sure many of y'all would probably roll your eyes and say, “Nope, I think it's too tempting when you have something that can write an essay so easily.”

There’s a lot of gray area around like how much help is too much help. The other option is obviously just do more in class. You’re not obviously going to be able to do like very ambitious projects in class, but you’ll be able to maybe do some more of that. The other extreme is to have more ambitious projects and say, “Yeah, use whatever you’re going to use.” I don’t think that’s appropriate for like kids who are learning to write; that might be inappropriate at late high school or college where they already know how to write and you’re making a business or something like that.

So I don’t think that’s a viable option. I think the reality is, if we want—which I want, and I think most of y'all want—if we want kids to continue to do the types of research papers, term papers that all of us grew up with because it did help us think, it did help us learn to write, and even just to project manage ourselves and to regulate ourselves and plan something over, you know, on our own, I think it’s going to have to be in a safe environment where you can keep track of the process. This is what we’re trying to do at Conmigo where the teacher can assign the assignment through the AI, and then the teacher could decide how much help or no help the AI should provide.

But then the student has essentially has to do the paper in that type of an environment so that basically there’s an audit trail of their work and we can see, “Oh, it took them this many hours, and this was V1, and after for one hour this is V2 after two hours.” I think if students know that there’s that audit trail, it’ll be much harder for them to just go to ChatGPT and generate something and copy and paste it.

Very cool! Okay, so now to switch from this idea of how do we deal with the realities of students in the classroom to the realities of math education. I think this is the perfect question for Sal. Peter says, “My interest is math. I watched the last Conmigo presentation and was disappointed that uses and math was not covered, and that’s totally on me, Peter. Definitely, I have a little bias towards the humanities, but I’m really excited to remediate that today because Sal, of course, got to start with Khan Academy as a math tutor to his own cousin. So what Peter wants to know is, Sal, what are the use cases for applying Conmigo as a math tutor? We know that these AI tools are so great with verbal skills, so great with writing—what about the math context? What should Peter know?

Yeah, and in all fairness, you know, this is still early days where we're trying to put a lot of stuff out there and learn what’s working and what’s not. You know, the core idea behind Khan Academy that—and now we have actually a ton of efficacy data behind it—you’re going to hear more about it, but we have 50-plus efficacy studies on Khan Academy before generative AI and it’s all around if students are able to put in, you know, the threshold disability studies 18 hours a year, it amounts to about 30 to 60 minutes a week, and they're able to do personalized practice.

Different teachers have implemented different ways, but the bottom line is if they’re able to get those two to three 20-minute sessions a week for most of the school year, these kids are dramatically accelerating. They’re accelerating, depending on the study and the grade level, 30, 40, 50, 60%. So the key question for us is, “Okay, we have these studies; how do we get more kids to engage and to feel progress?”

I think every all educators know regardless, motivation and engagement is often the key. Even 30 years ago, if you were sufficiently motivated, you could power through a textbook if you really had to, but most people weren’t that motivated. And so, you know, on the Khan Academy side, that’s why we’ve always wanted to provide supports like videos, etc. I think generative AI and what we’re trying to do with Conmigo we see as another powerful support.

I’m curious, well, at places like KWS, we do have a math teacher; I know they’re not here today, who’ve been using it. I’ll tell you what I've been hearing: that it has been able to unblock students where they might have been blocked before. Sometimes it’s a block of literally they’re confused; they’ve watched the video, it still doesn’t make sense to them, they’ve read the article, they’ve looked at the hints, and it still doesn’t make sense to them, so that helps to have that generative AI.

Just so you all know, you know, we’ve gone through a lot above and beyond the core GPT-4 model, and you know, ChatGPT uses GPT-3.5, or at least the free version does. But even GPT-4, which is dramatically better, we’ve done a lot of work to make it much, much better in math. And so we’ve definitely already heard feedback in the last three months of last school year when we were piloting it, that it is helping students get their questions answered in the moment, oftentimes on exactly that question that they’re struggling with, or that concept. Sometimes it’s a motivational block like, “Why do I need to learn this?”

The AI can help out with things like that. Those are, I think, the two clear math use cases. Beyond that, we do have a general “tutor-me” in STEM or math type of subjects where if a student just has any broad question, I could imagine—and this is kind of something we threw out—there’s a new scope for Khan Academy, which is, you know, if they're working on a particular exercise on Khan Academy or watching a video and they're confused about something, Conmigo can show up during those sessions and it has the context of the video.

So, you could say, “I don’t understand what Sal meant at this moment,” and it can answer that or, “Wait, this question, I don’t understand what this question is asking for.” It can do that. We obviously haven’t had time to do rich efficacy studies yet; we have done some pretty robust user research to at least make sure that it’s not doing harm, so far the initial indications are it’s not doing harm.

I think there are a little bit of classroom management issues sometimes because there’s so much that you can do with the AI that, you know, “Okay, don’t talk to Abraham Lincoln right now; we have to work on our math.” So we’re thinking through with educators on how we can control that a little bit better. We have some early indications that it is indeed helping kids be more engaged in their at least in their Khan Academy core work and be less unblocked, but that’s still very preliminary news.

Very cool! Then let’s go next to Malia. Malia, I wanted to focus on something Sal said right there about students being able to kind of like ask in the moment. Like even if they’re listening to a lecture or even in like a live class session, just to kind of share an anecdote from one of my classes. This past term we were talking about the Founding Fathers, and we were—the students were reading a book that was talking aloud about Ben Franklin and the book was also talking a lot about his sister Jane, right? The book was talking about how Ben kind of went from rags to riches and Jane went from brags to rags, right?

One of my students had the question: “Is this author biased?” Right? Okay, so I want them to discuss that. That’s a good one! Right? I don’t want to show my hand, so I said, “Why don’t we ask Conmigo?” Right? And so we sat around and we talked about, “All right, how would we prompt this? Right? How are we going to get the information that we want?” And Khan Migo was able to play with it, and it knew exactly the passage we were talking about, and then it gave an answer about, like, “This is the difference between bias and perspective, and all historians have a perspective, and like what do you think?”

That was an opportunity for me to decenter myself and let them have the conversation that they were organically having. So I think that’s like a good kind of in the moment—and then again, that modeling of how do we get the types of responses that we want out of this fun tool?

Very cool! And then one last thought here—Madeline, give us your take, and then we're going to go to a live caller if you will, so get ready! I saw someone in the chat also asking, like, when is science going to come into the equation as well? So I wanted to bring something up that I know had been helpful for a science teacher who piloted Conmigo in my building.

She and her students really appreciated the fact that Khan Migo is great at giving analogies for particular concepts involving the sciences and involving, you know, history. So, you know, one of the things that immediately comes to mind for me is the cell wall, right? I was really bad at biology when I was in high school, and I didn't have a resource like this. So I went into Conmigo and asked for some analogies for the actual, you know, term—make it make sense to me. It was able to give me like five or six different analogies that made that concept make more sense, and I think the final one that it came up with was like Captain America’s shield! And I love that!

So, you know, I was immediately picturing an activity where you could take your vocab list of scientific terms and have Conmigo give a bunch of analogies that would help it stick with your kids better. That’s a great way that it would be able to be more than just an answer machine. It’s a way to cement knowledge for them.

For those who don’t know my process, when I make, say, science videos, some of them I have the intuition, but a lot of it, I spend a lot of time researching, and I don’t even want to make a video until I feel like I really have a conceptual understanding of this thing. I’ve been using Conmigo myself. You know, a couple of months ago, I was always curious about, you know, “Why does a supernova—if you’re out of fuel in the star, why doesn’t it just collapse? Why does it eject material?” Conceptually, why does it do that?

I went to Conmigo, tutor me in STEM, and I mean, it was beautiful! It said, “Well, do you have any theories why it does that?” And I was like, “Oh, you’re asking me?” And then I was like, “Well, huh!” You know what? It made me think. I’m like, “Well, I don’t know. Is it like a rebound? Is it like, you know?” And then they were like, “Yeah, you’re on the right track. Because the collapse, there’s a lot of pressure, and it’s so dramatic.” Like, you know, “Have you ever seen something dropped from a high height that doesn’t bounce?” And I’m like, “Oh, good point!”

So yes, it’s that, then the bounce ejects some of the stuff on the app. I never understood that. This is something I’ve always been interested in. I’ve even asked experts, and they never quite gave it to me. Or something—they gave me an explanation without making me think it through myself. So yeah, I actually think science is one of the areas because so much of science, if we want—and I think we all want—the kids to go beyond memorizing the formulas or memorizing the terms. So much of science is making sure you understand it conceptually. You know, we've gone to great pains over the years at Khan Academy to hopefully do that through videos and articles, but you know, different metaphors, different visualizations are going to connect for different folks.

I think this is where it’s really powerful.

Love that! Definitely so much creativity baked in here! Um, let’s do something a little crazy we’ve never actually tried here at Khan Academy before—let’s do some live call-in! I know live talk radio and EdTech don’t always go together; let’s give it a shot in the interest of innovation. So, Sandia, I know you’ve had your hand up for a while! Thank you for your patience! I’ll give you the chance to unmute your line, feel free to come on, Sandy, if you can ask your question. I’m so excited! I’ve never done this before either!

Thank you for giving me this opportunity. So, I’m a math teacher; I teach seventh-grade math, and I went back this year after 13 years. The math teacher quit in the middle of the year at my daughter’s school, and they asked me to fill in. So my learning curve is extremely steep coming back after 13 years and then with all these tools, and I've spent the whole summer studying AI. I was looking at the question that we had asked in the Conmigo about the fractions—help with multiplying two fractions—and the response that came back was basically, “Multiply the numerator and the denominator.”

Is AI or Conmigo able to go into more “I think,” Sal, you were talking about conceptual understanding. “Why are we multiplying the numerator and denominator?” Are they just going to get this the steps? Great question! Simple answer is it should be able to do that. We could even demo it right now live. I would say like, “Why do you multiply the numerator and denominator?” Can you explain that?

I don’t know!

Yeah, absolutely! You want to see what it does, Jeremy? Like for example, this year, I was teaching the kids like, “Why do we, when we divide decimals, why do we flip it and multiply?” And the reality is you don’t have to, right? All the kids were like, “Seriously, you don’t have to.” I’m like, “It’s to make your life simpler; that’s why you do it!” You know, and I just don’t—I want, as we're building thinking classrooms, that’s really what we need to do—teach the children how to think! If they don’t understand why they’re doing the steps, even with AI, it’s not going to help them become thinkers.

Yeah, I like the way to sort of Conmigo again reframed it through the kids’ lens of not—we just do it because that's the way we’ve always done it—but it sort of puts it in this sort of pizza frame, if you will.

Okay! Yeah, I think of being more effective! Yeah, and you know, the one you just gave, Jeremy, like we didn’t give it the context of the month.

That’s right! Why do we multiply fractions? Why do we multiply numerators and denominators?

Let’s clear the chat! And then ask it; it’ll probably give it! When we multiply—and this is a tip because when people are asking like what’s the best way to use AI—one thing that’s for both for your students and for your own use, whether you use Conmigo or ChatGPT or whatever, always think about that sort of classic idea of garbage in, garbage out. If you just ask something really generic like “How do you do this thing?” it may not have enough context to go on!

But if you explain what’s going on, what you’re trying to achieve, it can be a lot more effective! So when you say, “When we multiply fractions, why do we multiply the numerators and denominators?” Listen! Imagine you have half a pizza and you want to share it with a friend! How much does each pizza get?

Yeah, you could—let’s try it forth, and this is great! This is live TV! Everyone, this is not—this has not been planned! Sandia, can you confirm that we’re meeting for the first time right now?

Yeah, we are meeting for the first time, and I’m loving this because I love how it’s going back and saying how much like asking the question back to students!

Yeah, so I mean, you know, we could keep going with this, but yes, it is actually, I’ve actually—the places where generative AI historically struggles the most is actually on just doing the math. We’ve gone through great pains to make that a lot better than you would get beforehand. But on the conceptual, it’s quite strong, which is still kind of amazing to me.

So how do we use Conmigo? We have to get a permission, I’m assuming, through our IT department or at school, or can I just use it on my own, or how does that generally work?

Jeremy and I are working very hard to make that answer simpler over time! So unfortunately, it’s a little bit complicated right now. There are a couple of ways. There’s a group of districts, so there’s ways to go through the district that we’re talking to school districts for this coming back to school already.

There’s also going to be ways—or there is a way that a parent or teacher can sign up for an individual account, and Jeremy knows more about that. I will say, and this is, you know, our mission as a non-profit is free world-class education for anyone, anywhere. Obviously it always costs money to build Khan Academy; our budget is about $70 million a year. But the reason why we’ve been able to, you know, it’s funded with philanthropy primarily, and then the marginal cost, the incremental cost of serving up the pages or the videos, it’s low enough that we can do it for free for the most part.

Generative AI, like every one of those interactions that we just did, was literally going to like hundreds, if not thousands, of like state-of-the-art NVIDIA chips. We get charged a lot right now! So we are essentially just charging folks for that cost, so we’re passing through us to essentially go to folks like OpenAI and Microsoft and NVIDIA so that we can do it.

Currently, it costs about nine dollars a month, is where we are. There are different permutations, but that’s where it is today. We’re working pretty hard to see if we can get the cost down and if we can make it easier for people to just sign up and sign up their students.

Cool! Great question, Sandia! You have restored our faith! What would you recommend? Because Jeremy, beyond running awesome webinars like this, is also in charge of trying to get what we call grassroots users, like people who aren’t doing it through the district to how they use Khan Academy and Conmigo in particular. What is the easiest way that you would say if a teacher wants to use it, Jeremy?

Oh, sorry! My apologies! Yeah, so the link that I shared in the chat is the easiest way to get access. Similar to like Netflix or something like that, you know, you just sort of sign up. That being said, as Sal mentioned, we’re always trying to find ways to make it even more affordable, more accessible, so look for developments with that in the near future.

I will just say I didn’t know if that was a question for Sandia. Sandia, if you have ideas for how we can get into more teachers’ hands, more parents’ hands, please let me know when I send you that PD certificate at the end, and I'll be putting a gold star on it because you rock this on!

Yes! We'll do! We’ll do! I'm wondering just—sorry, I don’t want to monopolize—but one more question. Because my daughter and my school have required kids to do Khan Academy all summer to close the gap, and my daughter had 133 assignments to do. Is there—do you see the future of like when they're watching the videos or doing their practice questions—ending up being like a collaboration with Conmigo where if they're stuck on that practice question, Conmigo could be more interactive with whatever they're learning in their skill-based learning videos and practice?

Yeah, that already exists! I mean, does the larger group—that already exists! When Conmigo gets turned on, any time someone is doing anything on Khan Academy, Conmigo can assist them, answer questions, quiz them about it, connect it to other things.

Oh, that’s awesome! Thank you!

Yeah! Conmigo will always be that sort of guide on the side, as we say. Yeah! Watch that video on intro to fractions, and yeah, and ask a question. Oh, you’re in teacher mode right now. That's why it's saying write a lesson plan! Things like that! I mean you could say, you know, write a poem about this video! Right?

This is a—it’s actually a pretty good summary when it does a poem about a square, a rectangle, and pie all cut up.

Oh! Oh my!

Wait, I missed that as you scroll up a little bit!

Oh, whoops! All right! So yeah, definitely lots of use cases! Lots of use cases! From the sort of teacher dashboard all the way to the student experience!

Sonia, you’ve been amazing! Thank you so much! Thank you! Lots more of these, but for now, let me actually go to a question that’s been waiting very patiently in the wings in the Q&A and actually speaks to sort of two different variants here. First, Scott says the internet enabled data retrieval; online tools enabled self-learning; AI enhances both and brings new capabilities. Given all this, how much class time should teachers spend demonstrating the best and most ethical practices when using these resources? So in other words, should we be really diving deep here or just providing some general guidelines?

Then Laura asks a complimentary question, which is how can I teach my ninth graders how to use AI effectively? So we’ve kind of gone past that plagiarism and cheating issue to “Okay! This is the reality of 2023! How do we make the most of it?” Any thoughts from our amazing panelists? Let’s go to Malia!

The number one thing that I’ve stressed in the classroom is that this is an opportunity for us to all learn together, right? Like, this just came out; it's new to all of us! It’s new to you; it’s new to me. So being curious and being a role model for them saying like, “I’m trying to figure out how to use it ethically. I’m trying to figure out how to use it creatively.” Like it’s a tool to like, do the thing with our students, which I think is powerful.

And to say to them, you know, as I have before, sometimes I’ll say like, “Well this is how I’m prompting it; how are you prompting it?” Right? And to let their voices feel empowered.

Let’s go to Madeline.

So, kind of two things in response to that on the responsible use front: this is exactly what I was just talking about. Earlier today, I was with our students who are getting trained as Writing Center tutors, and I had a lot of them as students last year, and one of them got pretty vulnerable and she said, “Miss March, I was so scared to use AI because I didn’t want to be doing something wrong.”

I think what we really need to internalize is because the primary discourse right now is, “ChatGPT is an answer machine, ChatGPT is, you know, causing a cheating outbreak.” There are a lot of kids who are genuinely fearful that by using it, they’re doing something wrong. So by giving them that kind of rubric, like the one that I showed you, where we’re setting the tone of, “Hey, this is what we’re permitting,” and giving them the chance to play with it—that’s how you switch the narrative is through giving them that exposure and through giving them that opportunity to try.

And then that’s how they’re going to get more comfortable and also get better at asking it the right questions. The other thing that I brought up to them is we would be doing a disservice to them if we didn’t give them that exposure, right? Because the world that they’re going into is going to be increasingly AI-based or AI-integrated.

If we continued to shield them entirely from it, we would be causing a problem! I think having those structured activities where they can play, but play responsibly, is very, very critical.

So, for instance, for an essay that I was doing with my students, I let them brainstorm, make outlines using Conmigo. A lot of my students struggle with counterarguments. I said if you want to debate with Conmigo and find a counterargument, go for it! In the last webinar, I actually shared a story about how one of my students wanted to write a paper on the likelihood of future nuclear warfare, which I know nothing about. I—what was I going to be able to do to help Jimmy with that paper, right?

But he was—I literally have read through hours of chat history of his this summer, just amazed at the kinds of questions that he was able to ask because he was given permission to do so.

So, they do need that freedom a little bit.

I love that, Madeline! So much more—so much better to face the world that’s coming than to stick our heads in the sand! I gotta tell you—listening to speak Madeline, I’m getting ready to move my family to Hobart, Indiana, because I want you teaching my kids!

Okay! I think we’re going to roll the dice again a little bit here and do another caller. So, Sheree Nelson, if you are a game for it, we’re going to give you the opportunity to turn on your microphone. Feel free to join us on “Chat with Sal.”

Hey, Dr. Sheree Nelson! I teach economics at Purdue Global University! Cool! I’m the course lead for microeconomics and macroeconomics. I designed the courses, and unlike some universities, what course I design, every macro and micro teacher uses, so it’s not just for me! So that being said, what’s been really interesting is that I have integrated Khan Academy videos into my courses before, and now I’m just doing it even more! It’s just great!

But I’ll be honest; I don’t know much about Conmigo, but I have been using ChatGPT to even generate my own discussion board prompts. And I tell my students, “I’m using it; it’s okay.” In fact, Purdue Global had a school-wide meeting with all of the faculty encouraging them to use AI— but do it responsibly, so that we can learn to do it and teach our students to take advantage of it.

So, what I was wondering is—and I know I took all that time explaining everything—but I hope I gave you a pretty good message!

Yeah! Well, and also, I’m doing things now that other course leads haven’t done yet that I want to teach them how to do because we’re a huge university and we have a lot of opportunity.

So what I do in my classes, I actually do approach with the Sal Khan approach, and that is I tell my students, “If you don’t have the foundation—” because I've watched an interview with Sal where, you know, you talk to Sal about, “If you don’t have the foundation and you’re moving forward,” basically, the analogy I give to my students is if you don’t have a good base in your house, the rest that’s built upon it might fall down. That’s I learned from Sal, and so I share that.

So, I was wondering, within…here’s my question! I took me forever to get to that! Just be glad I’m not your professor! So I was wondering, is there a way—or is it already done—that something is built in with Conmigo where when you type something in, it gives you the information you look for, but it also directs you or asks you, “By the way, did you know…?” and then ask a question about a concept that is before that, like a foundational thing?

Yeah!

Yeah! Absolutely! I’ll try to do a little bit of show and tell. Let me see if I can share a window!

This is an exercise—this is a video on Khan Academy on price elasticity of supply, and then I could go to Conmigo and say, “Can you quiz me to make sure I am ready to understand what is in this video?” That’s what you were asking, aren’t you? Like the prerequisite knowledge?

I love that!

Yeah, yeah! So you’re right! So the, um, so I think there are two layers. What you’re describing can already be done if someone knows to do it. Which, as you know, most people now, it could become through—the teacher could—and we’re all doing this together. I like the previous answer around, like, you know, we’re all learning how AI is going to be.

Absolutely, yes! But I think it could be a cool practice when teachers are having students watch a Khan Academy video or Khan Academy exercise. You know, if the student immediately jumps into it and they’re able to engage, great!

But maybe if there’s already a sense that they might struggle, like, “Hey, why don’t you start with the quiz to make sure you're refreshed on the prerequisite knowledge?” That’s one way. If that seems to be something that works, then yeah, we would think about productizing that.

Let me clear this chat. Let me make this a little bit bigger; I think my—you know, we have these choices of things that we think a lot of people might ask, and obviously you can type whatever you want. But maybe that’s something we can add to this. You know, help me prep with the quiz right now. That’ll just quiz you on the content of this. There could be another one just like that we add that says, “You know, make sure I'm ready for this video,” or “Make sure I’m ready for this exercise,” or something like that.

What made me think of that is I just currently am just finishing up the revision of my microeconomics course and I’m using the new—and I’m not into like promoting a publisher at this time but so I won’t mention which one—but I am with the new publisher who has so many, you know, third-party tools for my students, and they have it so that when you’re in the book and you’re reading a concept, it will—it has the icon that says, you know, would you need a little more practice with this? Would you like more background? You know, things like that. Because sometimes, if they’re just looking at one concept, they don’t know what concepts they’re missing, right? Like the suggestions there!

Oh, I’m sorry, Sal!

No, no, I agree. I actually love this! We’re going to take this to the brain! Like, would you like more background? Obviously, Khan Academy—we’ve always had the exercise piece. So like, in theory, you could practice what you learned, but this idea of “Would you like more background for it?” I think it’s a really powerful idea.

Is it okay if we borrow that idea, Sheree?

Well, you know what? I have no problem with that. In fact, the new publisher that I’m working with has also looked through this process and has said, “We hire you for to be a Content controller.”

There you go! Because I have all seen online, I’ve been with Purdue Global, I’ll have my 20-year anniversary in October, so I have a lot of online experience. I’m always looking to see how we can do it better!

Oh! Cool! Because there’s—you can never stop improving!

That’s right! Thank you so much, Sheree! You were fantastic! Thank you so much!

All right, so I think this is the great thing about where Conmigo is headed and where AI is headed. We all get a chance to write this story, just like our awesome panel has been talking about. Let’s get our educators involved; let’s get our students involved! We should all be driving this forward if this is going to be a big thing for our future.

Now that being said, we’re coming to our final 10 minutes. I want to end with those proverbial fireworks. A chance for the panel to show off some of their best AI skills. I want to tackle these two twin questions: Maria says, “How do I best use AI?” Period. Scott says, “How can we make this a powerful tool?”

I’m curious from Emily, Kelly, Sal, Madeline, and Malia—are there any amazing pro tips that you pioneers at the vanguard of this movement can share with educators across the country?

Let’s go over to Madeline!

Sorry if I’m doing some monopolizing! I love all of these questions! I—you know, like I said, I was just talking about this yesterday with the staff at my building. I think one of the most powerful ways that Conmigo has helped me as an educator is through—it was something that Sal touched on earlier—expediting certain tasks so I can use my time more effectively to work on relationships with my students.

One of the things that all English teachers hate making more than anything in the world is a rubric! Right? Mostly because, I think we are some of the most critical of our own writing; and even in those, like, nitty-gritty tasks to make a really good rubric that takes a lot of time! I’ve got some slides here; let me scroll to them super quick and then I’ll share my screen.

I had first told Conmigo, “Hey, I’m trying to work on making a rubric that assesses a summative for my English 9 honors students.” It gave its own suggestions for categories, but I said I wanted to make my own that were based off of my curriculum map.

So then I made my own criteria and kind of explained what I thought those would look like. Conmigo asked how many levels I would want for those criteria. I changed the number of levels that it suggested and then after 15 minutes of that back and forth and then some formatting time, this is the rubric I was able to make! I did some tweaking of the phrasing. A rubric like that just to give you an idea would normally take me about 50 minutes, which is almost that entire plan period to do! And I was able to get so many other activities done for my classroom in that extra time because I didn’t have to worry so hard about finding that exact phrasing for something that really doesn’t—I don't want to say it doesn’t deserve that amount of my attention and energy, but there are so many other areas that I would rather be spending that energy in!

You know, I want to be giving my students quality feedback on their own writing, not spending that time drafting a rubric myself.

I love that, Madeline! Calling out, “Hey! If you’ve got tedious parts of your job—and don’t we all?—there’s probably a good chance that AI can take on some of that!” And then B, don’t treat, you know, AI like Google.

Like we have for the last 20 years; we just put in a question and settle for the answer! Continue to push it! Think of it as a dialogue the way that Madeline did and get better and better outputs!

Thanks for sharing that, Madeline! Let’s go to Kelly!

So, my experiences so far have been similar to Madeline’s as far as proficiencies! I’ll talk a little bit about how some of my students have used it too!

But, you know, I wrote in the chat earlier when our Purdue professor was talking about using it to design some work, I’ve also used it to design an online art history class that was brand new—kind of under a time crunch, and I was like, “Okay, we gotta get some creative assignments here!”

It had been a little while since I put together some new things like that on my own, so I used Conmigo to help create some projects. At Khan World School, we do some cross-disciplinary work, so I even asked some questions like, “I’d love to get some novel suggestions that would also work with this art history unit, you know, some of this African art unit I’m working on.”

I kind of went back and forth with it a little while and came up with a good interdisciplinary project within English where they read “Things Fall Apart,” and it goes along with some of the art history unit or something like that! So that was just one example where it saved me a ton of time! Where I think if I was just kind of sitting in front of my screen pondering that myself, I just would have gone back and forth and maybe even walked away before finishing what I was trying to do.

I had a few handful of assignments that really saved me a ton of time! And then with our students, we have a seminar at Khan World School where we go over different topics every week, and sometimes we do debates or arguments too, and I had a student that used it to flesh out her argument!

She kind of asked it to poke holes in it or went back and forth with different perspectives, and she said it really prepared her, you know, for the following day where she could have done that with me, but with all your students, you have time to do that with all of your students, and you probably not! For us, she was also in a time zone that was nine hours ahead of me, so the chance for us to get together before our next session probably wouldn’t have happened!

So she was able to do that online, and I loved that! So she felt so much more prepared the next day!

Oh cool! So clearly it’s not just the tedious stuff; it’s also the most important stuff. Whether it’s designing a brand new curriculum or helping students sort of grapple with complex ideas—that’s pretty powerful!

Let’s go next to Emily!

I was just going to add I also work at Khan World School, and I think the power in helping students to ask questions is really at the crux of what you can do with Khan Academy. I want my students to be curious! I want my students to ask questions! And I can’t be there all the time to be at the sounding board to go back and forth, but Conmigo can be that.

Whether they’re talking about an important world topic or a history topic or they’re working on an English essay, they have that as a constant sounding board. Now, I would love to be that starter to like kind of work them through modeling asking questions because I don’t necessarily want questions that are instant recall kinds of responses.

But if they want to ask questions about a novel, and if they want to ask questions— I know Sal has included our student Sandy in a number of talks where she discussed “The Great Gatsby” and looking at the green light and had a conversation with Conmigo about that—those are powerful interactions! The simulation of Jay Gatsby!

Oh yes, yes! Elation!

And that curiosity! Now Sandy, she is a very unique student; she is incredibly curious! I wish all of my students were as curious as she is!

I love that! I’ll just show a little demo as we switch to our final panelist, Malia. Take us home! What tips do you have for educators out there?

The most practical thing that all of my colleagues—the stuff that the AI is good at, it is good at—let it do that. And spend your time with your students.

Then I also just—I’ll stop there, Jeremy.

Okay, no, I love that! I love that! And it actually gives us two minutes, which is perfect! I want to have Sal close us off. Sal, you came to me in the fall last year and said, “Jeremy, there is this thing that’s coming out that could change everything!” And I said, “Yeah, yeah, yeah, Sal, who knows?” You know, people in Silicon Valley say all sorts of crazy stuff!

But now, it is a reality! Now it’s part of our lives! What words of wisdom do you have for educators across the country as they finish up this beautiful summer and then get ready to take on the first fall with AI intact?

You know, I’ll say kind of what we’ve been trying to tell ourselves at Khan Academy—because some of the stuff is very exciting, but it can also be intimidating. It’s a little bit scary; there are a lot of edge cases that none of us know what to deal with, and the solution isn't to hide from it. The solution is to try to engage with it!

Yeah, you know, there are all these debates about whether AI is going to be good or bad for humanity, and what I’ve been telling people is, I don’t think it’s a flip of a coin. I actually think it’s going to depend on whether good people with good intent are jumping in and trying to make good use of it—because we know the bad people with bad intent are already jumping in and trying to make bad use of it.

So I think it’s actually really important for all of us—hopefully everyone here is, you know, good people with good use cases who want to do good by humanity—to jump in, learn it ourselves! Don’t—you know, this is one of those moments where no one is an expert and everyone can be an expert! If you’ve played with Conmigo or ChatGPT for an hour, you’re already in like the top 10% of humanity, right? Like you’re already an expert!

So this is a moment for people to jump in, but also don’t judge all generative AI based on ChatGPT, which is not designed for this, and it’s obvious in any way, shape, or form. But give tools like Conmigo a shot! If we can hopefully get you access, start playing with it, seeing how you use it with your students! Give us feedback because we are very serious about—we didn’t talk about all the guardrails about keeping it safe; we talked a little bit about the transcripting.

We have a second AI that actively notifies parents and teachers if something is amiss. Obviously, we’re thinking a lot about cheating and things like that. But yeah, jump into it! I think number one, if you just, as was said, if you start using it to start saving you from some of your work that’s not your favorite thing—the writing rubrics or whatever—you’re going to start being an expert! If you start having your students use it for even light cases, you’re going to start learning a lot.

Where else you’re going to learn a lot of—we're all relatively new to this space, but it’s all about leaning into it and thinking about it boldly but responsibly. A lot of that!

And I’ll just end with the words that I always told my kindergarten students, which is: if you want to learn something, the best way is don’t be afraid to get your hands dirty!

So I think this is a chance for us all to build something really awesome together. Here’s wishing you all a beautiful rest of the summer, a great start to the new year, and we’d love to be part of that journey as well. Here’s wishing you many exciting adventures! Here’s everyone!