Brave New Words - Bill Gates & Sal Khan

Hi everyone, it's here from Khan Academy, and as some of you all know, I have released my second book, Brave New Words, about the future of AI, education, and work. It's available wherever you might buy your books. But as part of the research for that book, I did some interviews with some fascinating people, which you are about to watch. I'm very excited to introduce you to our guest who probably needs no introduction: Bill Gates. Bill, thanks for joining us. Great to see you!

So, I know AI is very front of mind for you. Obviously, all of us at Khan Academy have been thinking a lot about AI in education, and you know the focus of this is AI in education and work. I'll start with a question that I'm getting from a lot of teenagers and folks in college and their parents: given what you know about what's happening in artificial intelligence and maybe in other technologies, well, like, what advice would you give to someone who’s trying to figure out what skills should they focus on or not focus on? What should they major in? What are the jobs going to look like, or not look like?

Well, the AI will make people more productive, and you know, in some cases, like writing software, there's a lot of unmet demand for good software. You can make it higher quality software. Exactly what the job market looks like is very hard to predict, but the deeper skill set, you know, whether it's medical consultation or scientific thinking or customer support, you know, the more value that's going to have even in a world where your productivity will be enhanced by having this sort of almost free worker, you know, is looking over your shoulder, giving you advice, giving you lots of feedback on what it is that you're doing. So, I don't think there's anything like, you know, don’t become a barber or a radiologist that we have in mind because the new capabilities are very horizontal in nature. It's not a topic-specific set of advances.

So, it's kind of like everyone's going to have an intern or a research assistant or an analyst, so maybe the answer is don't be the research assistant or the intern or the analyst; learn to manage them. And of course, jobs like, you know, helping handicapped elderly, helping to motivate students, you know, there's such an infinite demand for those things even in the rich world, but even more dramatically so in developing countries that, you know, jobs that require deep human connection and empathy, you know, we'd say it's a long ways off before there's a substitute for those things.

You know, we have two revolutions kind of in parallel. We have this chat GPT-enabled sort of reading and writing white-collar enhancement, but in the meantime, people shouldn't underestimate that the blue-collar sort of robotics, eventually leading to a humanoid robot that's quite general purpose in terms of the physical skills—arms, legs, the things that it can do—that's also happening. And so the good news is that we're going to have more leisure time. A lot of the shortages that we have on medical advice, or when we get confused, being able to be straightened out, you know, we have this supply of these new capabilities.

The unique roles that humans take on or how they'll take their extra spare time and make that fulfilling are our challenges. You know, what you just touched on is interesting because I think people have always theorized that, hey, we're going to get more productive, we're going to be able to do more with less time, and so we'll have more leisure, which is, you know, I think utopian for a lot of folks. But it looks like the patterns of the past of productivity improvements are, if anything, that the hyper-productive are working harder. They’re, they’re, because they get so much leverage from every minute, that it's not necessarily leading to that.

Do you think something's different about this wave, where maybe things like the four-day or maybe even eventually a three-day work week will become common, or do you think people are just going to be able to do more with less and hire fewer people than they do today? The analogies from the past are worth understanding, but the changes in the past were generational in nature, you know. So, the mechanization of the primary occupation, which was growing food, that happens gradually. And so, you know, Grandpa was a farmer, and your parents less so. And then, you know, now I don’t know many rich world farmers. I know a few, but not many. And so you didn't have to change mid-career.

It is true that demand for things that we wouldn't have expected emerged, and so it’s comforting to look at that history. But the total level of capability here and the speed at which it's going to suffuse into the economy and change job markets, you know, in five years, the effect will be very substantial. I'd say in ten years, it's going to be quite dramatic, and so I don't think we can completely dismiss the idea of this change will finally get us to the point where the tradeoff between labor and leisure actually becomes important.

Instead of taxing labor, like with payroll taxes, you know, I called it a robot tax when I talked about it, and people thought it was strange. But we'll shift to where we essentially subsidize labor and tax, you know, completely on the capital and profit side, which is that alone is a pretty big shift that government will need to make. It's a fascinating idea; it makes a ton of sense because obviously, if we have a more abundant society, a more productive society...

But, you know, let's take an investment bank or consulting firm. A lot of the traditional work that people are building and the size of the organization are analysts doing spreadsheets and making PowerPoint presentations, and a lot of that stuff's going to get automated very fast. And so this is a world where you subsidize these firms to say, "No, keep hiring interns, keep hiring these junior analysts." We'll subsidize to have a sense of purpose, as opposed to these young people not having something to do, not building their skill set, and never being able to become the manager.

Yeah, if you get to the extreme where, you know, the purpose that jobs provide isn't the central organizing principle in society, then you get into, you know, some deep philosophical questions about, you know, what should priorities and what is the purpose in an abstruse way. You know, the Boston Book Superintelligence touched on that. I don't really think it's been written about all that well. You know, Life 3.0 did a good job of saying, "Okay, what is the role of intelligence in these futuristic scenarios?" A lot of new books coming out right now—I haven't been able to catch up to them all—probably some of those will be helpful. I'm interviewing you right now for one of them.

Perfect! Just going back to, I guess, more basics, so to speak. Obviously, a lot of folks... We talk a little bit about work, but now going back to the core of education, which I know you care deeply about. We've chatted many times about education. Right now, there's a bit of an emergency going on in education. People—like kids—are going to cheat using this technology. How… what do you have an answer for?

And we're hearing this from every, you know, faculty across grade levels across... what would you do if you were a history professor or a writing professor at a high school or a college right now? Well, to have a pure test of somebody's ability to write on their own or calculate on their own, you'll need to create an environment where they don't have access to the software tools. Our efforts to detect whether software has helped you write something, you know, will catch the sloppiest, you know, sort of the equivalent of cotton paste from Wikipedia. But in the end, we won't be able to tell because there'd be such a variety of tools, and the variety of prompts will make that quite difficult.

So, the good news here is that for the first time, if you write on a topic, the AI can give you feedback. It can help you with the first draft. It can help compare what you've done to something else. It can create a program of study of things that you might try to write about. And so the level of engagement where it can help you whenever you're stuck really is something new. It goes even beyond the fact that there are great YouTube videos about all the sciences and history. This gets into the realm of custom tutors and feedback on reading and writing.

So the paradox is that now we have this tool that, if you want to learn, it makes it easier. But testing somebody's capability, we're going to have to use new techniques. Sadly, in some cases, it might make people feel like, "Wow, wow, if it's so good, do I still need to push myself as hard?" We're, you know, you and I are seeing both for personal fulfillment and to play a role in contribution in a job market. The higher level your skills are, you know, the more it will retain a substantial value.

So, helping kids learn how to get feedback on their writing, I think that’s imperative because we do a very poor job of that today. So, what I’m hearing is, is that not only is there not less of a reason—there’s more of a reason to accelerate part the earlier part of our conversation—which is those intern-level, entry-level things are quickly going to expect you to be more of the architect, more of the editor, more of the manager. And if you can't make that leap sooner than later, the good news is these same tools can be used to do that, which I agree with as well, and obviously, we're working on a lot of that.

What would you—so if there was a profession, I mean we get emails from folks, maybe you do too, of like, "What do I do this fall?" And it sounds like you would do more in-class proctor things for assessment. Would you still do the traditional term paper, or would you change the rules of engagement on that? Well, yes, in the workplace, people are going to be encouraged to come up with the best product they can, where it's them plus the AI. And so there's no reason not to have that be one of the skills that you practice.

Is, “Hey, you know, let's do a business plan and sure, you know, learn how to use the very best AI tools. But, you know, I will view the, if I just have the AI do it without any of your value added, you know, that's going to be a C minus or worse. You have to get in there and, you know, bring in a little more accuracy, different sources of knowledge, and, you know, have unique value added. I mean, you understand the audience that it's going to. It's even hard still for the computer to understand all those different contexts.

So yes, you're going to have two kinds of writing: writing you do completely without the help of the computer, and writing you do where you're encouraged to make it be the best assistant possible. No, that makes sense. You know, we talked to Kevin Roose, who writes for the New York Times, also famously wrote about his interactions with Sydney and all of that. And his point is there's writing, and there's journalism. The writing, the AIs are getting very good at, but the journalism—making the contacts, being able to interview people, form bonds, etc.—and that there's an analog to everything.

If you're an entrepreneur doing user testing, you know, although the AI might be able to do pieces of that related to this whole practice feedback. You know, you touched on the word assessment. I'm curious where you think assessment is going. Obviously for many years, scalable assessment was fairly limited—Scantron, multiple choice—that's what we did because we couldn't do richer assessment. It started to drive what does or does not happen in the classroom. Do you think that's about to change?

Well, there's a controversy over assessment, independent of the AI. You know, should SAT scores be part of college admissions? You know, I do think proving that your math skills and your basic understanding of, you know, history and science is strong, you know, that's pretty important. And that when you want to consolidate knowledge, the fact that you'll be asked very broad questions about it is very motivational to making it accessible—not just, "Okay, tomorrow I'll know it in a simplistic form," rather "I know how to access it and combine it with my other knowledge."

So I'm still somebody who thinks that tests can help, you know, you know where you are in a subject area. And, you know, it's a meritocratic thing that you really want to be able to prove that you have some capability. Certainly, when I go to a doctor, I'm glad they've been subject to some examination and a test of their competence. Now dealing with the fact that if you have a disadvantaged background, even if you're on a slope to get really good, that looking at you just at that moment in time, you know, may not—maybe shouldn't be directly comparable. That's an interesting challenge.

Partly, that's why I love the idea of the computer tutor because it can be encouraging, it can come, it can understand what your misconceptions are, and it will, you know, if we can make it available very broadly, it means that the sophistication that, say, my parents, you know, provided to me as college graduates is less unique in a world where the AI tutor is super helpful in a huge variety of subjects. But, you know, I'll stick up in many cases—you know, having an objective assessment of your skills—I think once you move away from that, you're in a very difficult place, even in terms of your own discipline of—do I understand this topic or not?

Yeah, no, I agree. And, you know, whenever someone says, "Oh, I'm against standardized assessment," I'm like, "Which part are you against? The standardized part or the assessment part?" And it's usually hard for them to be against either of those, but I think what they are implicitly saying is they feel like it's maybe not fully capturing where the student is. There's only certain things you can measure; maybe it's not capturing their slope to, your point, it's a snapshot in time.

Do you think that AI in the next five years, ten years might solve this by being able to give richer assessment? And, you know, assessment could be where you are in math; it could be where you are in writing; it could even be things like job interviews—that's a form of assessment—career progression. Do you think it's going to play a big role there?

Well, I think given that we still have certain imperfections in the AI, people can still be skeptical when we say that an interactive assessment with the AI discussing a topic like your understanding of history or economics—that we can make that equivalent to the very best sort of professor who would engage you deeply. And I think we will get to that level, and that is superior to just a set of standardized questions that, you know, have to have sort of a straightforward answer. Your richness of understanding the topic, and there's—you know, there are people who are better at multiple choice type questions and test things.

You know, I think you and I probably would have done well educationally, but I think partly we are good at tests, and I felt bad for kids who, you know, the test—I could tell they were intimidated. You know, when that, you know, multiple choice, you have to answer all of these in an hour would come out. So by accommodating a more flexible dialogue, I think different styles of learning will be appreciated. Different a broader set of skills will be appreciated.

But we still have to prove that to people. We're not at that level of matching a great professor discussing a topic with you yet and then being able to actually give a good assessment of it. Exactly! We've seen we can emulate the actual discussion part quite good already, but that the strong assessment—I think there's… but probably going to happen, my guess. Yes, it's going to happen. I mean, when you have experts who are good at something that's linguistic in nature, and as you gather those data sets and constantly give feedback to them, you know, even with today's mechanisms, which can be improved, but even just with this, we can develop something like a human expert.

And I mean with that it’s funny—I mean this conversation, as you know, even a year ago would have all seemed like science fiction. And this is all changing so quickly. I know I feel this way, and I feel like I'm in the middle of it, but still, things are changing so fast. You also are probably seeing even more.

I want to ask you a long-term question because who knows what the world's going to be like in even 20 years? But let’s say five years, which is I think what most people should be kind of planning for in their lives—whether they run a company, whether they're a young person for their career. What do you think the world of education and work is going to look like? Like, paint a picture. Are we going to be able to have a video conference with—I mean, I don't want to cloud your—what would it look like? How's it going to be dramatically different or not than the world we're in right now?

Well, there'll be a certain set of jobs where you're just moving documents around and looking at documents—say, you know, accounting or medical claims—where the depth of understanding in those jobs is pretty straightforward, and some of those jobs will just be purely automated. Then there'll be a ton of jobs where they're still being done by humans, you know, like making greeting cards.

I went back and watched the movie Her because it raises—in a pretty—I thought greeting cards were definitely going to be automated because now you can have mass customization of greeting cards. Well, anyway, yeah, you probably can do it by yourself to, you know, take, you know, DALL·E 10 and say, "Hey, you know, here's how I want to make fun of my sister, remind her about this thing from the past, and here's some photos to stick in."

So, yeah, you might eliminate the professional there because it's very—you and the AI can do it in a very bespoke way. But there'll be a lot of jobs left that are just more productive and that, you know, where you have this kind of co-pilot that's taking the, you know, okay, convert the PowerPoint to a Word document, and boom, you know, that draft is available for you to have as the starting point. And, you know, so that extra productivity will create, you know, new opportunity to take things like medical care and make it better, education and make it better, care of the elderly and make it better.

You know, productivity is a good thing, and people have been bemoaning—and there's many books about, oh, you know, we had electricity, and then since then, everything hasn't been quite, at least the way it reflects into economic figures, as dramatic as what happened there. This you will see in the economic figures in a big way—both the robotic side of it and the LLM reading and writing part of it as well. So in five years, we will be in a different space.

You know, hopefully, we won’t—you know, elections will still be uncontested, and deep fakes won't make the problems we already have there even more difficult. It is interesting that government at a time when, you know, it’s not as trusted as we’d like will have to step forward and change tax policies and have retraining policies and, you know, be part of the dialogue of setting rules. You know, the entire industry is saying, you know, there’s a role for government here, but, you know, we have, at a time where trust in that and the divisiveness is very challenging.

So I hope that we haven't suffered that much from the negatives in the five-year time period. And by then, you know, society will have reoriented itself and say, okay, how do we avoid this being used as an attack vehicle? How do we make sure it's used in an equitable way? You know, certainly the dreams I have about, you know, health and education in poor countries, you know, I think it'll take us five years to kind of get it out and get into widespread use. Five years is a very short period of time, but it lays a foundation where I hope by being more productive as a society, many of the challenges of aging or climate or, you know, people expecting government to do more than it's capable of doing—that will say, "Wow, you know, we're making obesity drugs better, we're making Alzheimer's drugs better."

And you'll get this sort of positive view of the current system, making it work better in an incremental way versus the divisiveness or, you know, even considering radical alternatives that I think would be pretty scary. So, innovation here, because it will surprise people, it's not expected, does have a lot of positive upside.

Yeah, that leads to the last question, which is how are you feeling? You know, my view on AI changes almost daily as I read more and I think more and have conversations like this. Are you feeling net positive? Are you really worried about certain things within your positivity? Are you net negative? How are you feeling about it, and what are the things you're thinking most about?

You know, I'm thrilled that when I, you know, sit with the malaria team and we talk about how we're going to have, you know, better ways of killing mosquitoes, that, you know, these software things can help with. Or, you know, we sit and talk about kids who've decided they're not good at math—that we haven't made much progress, you know, for those kids. Just having all the ability to try out their knowledge, you know, that wasn't enough to draw them in.

And so that issue of motivation and the fact the AI tutor—one of its excellent things will be understanding the right level, the right encouragement, to make somebody feel like, yeah, this mathematical understanding of the world is something that can work for me and make my life better even beyond, you know, what sort of job opportunities it enables. So, I'm very thrilled about that.

The world that we're delivering it into and its ability to have, you know, complex factual debates and, you know, different countries trying out different regulations—I would say that the political challenge, that where everyone says government has this big role to play to take advantage of the positives—for example, changing the tax structure and to mitigate negatives—deep fakes, bias, AI-enabled cyber attacks—are our governance structures mature enough, agile enough, and can they consensus enough that we'll manage this opportunity in a good way?

And there are definitely days that I worry we won't—the government side of this thing won't step up to what's necessary. Yeah, well, I could talk to you for several more hours, maybe days about this, but Bill, thanks for taking your time out for this. This is super valuable.

No, it's fun to be on this journey together, and, you know, we'll be a great partner and help you realize your vision. Great, well, we appreciate that so much. It is exciting times; let's see what happens. All right, thanks! All right, thanks, Bill.