AI: The Beast or Jerusalem? | Jonathan Pageau & Jim Keller | EP 308

So the Hebrews created history as we know it. You don't get away with anything. And so you might think you can bend the fabric of reality, and that you can treat people instrumentally, and that you can bow to the Tyrant and violate your conscience without cost. You will pay the piper; it's going to call you out of that slavery into Freedom, even if that pulls you into the desert. And we're going to see that there's something else going on here that is far more cosmic and deeper than what you can imagine. The highest ethical spirit to which we're beholden is presented precisely as that spirit that allies itself with the cause of Freedom against tyranny.

I want villains to get punished. But do you want the villains to learn before they have to pay the ultimate price? That's such a Christian question. That has to do with attention, by the way. It has to do with a subsidiary hierarchy—a hierarchy of attention which is set up in a way in which all the levels can have room to exist, let's say. And so you know these new systems, the new way, let's say, the new urbanist movement, similar to what you're talking about—that's what they've understood. It's like we need places of intimacy in terms of the house. We need places of communion in terms of parks and alleyways and buildings where we meet, and a church—all these places that kind of manifest our community together.

Yeah, so those existed coherently for long periods of time. And then the abundance post World War II and some ideas about what life could be like caused this big change. That change satisfied some needs; people got houses but broke community. Community needs, and then new sets of ideas about what's the synthesis—what's the possibility of having your own home but also having community? Not having to drive 15 minutes for every single thing. Some people live in those worlds, and some people don't.

Do you think we'll be smart? So one of the problems—why were we smart enough to solve some of those? Because we had 20 years. But now, because one of the things that's happening now, as you pointed out earlier, is we're going to be producing equally revolutionary transformations but at a much smaller scale of time. What's natural to our children is so different than what was natural to us. But what was natural to us was very different from our parents. So some changes get accepted generationally, really.

So what's made you so optimistic?

[Music]

Hello everyone watching on YouTube or listening on associated platforms. I'm very excited today to be bringing you two of the people I admire most intellectually, I would say, and morally for that matter: Jonathan Pajo and Jim Keller. Very different thinkers. Jonathan Pajo is a French Canadian liturgical artist and icon carver known for his work featured in museums across the world. He carves Eastern Orthodox, among other traditional images, and teaches an online carving class. He also runs a YouTube channel, This Symbolic World, dedicated to the exploration of symbolism across history and religion. Jonathan is one of the deepest religious thinkers I've ever met.

Jim Keller is a microprocessor engineer known very well in the relevant communities and beyond them for his work at Apple and AMD among other corporations. He served in the role of architect for numerous game-changing processors, has co-authored multiple instruction sets for highly complicated designs, and is credited for being the key player behind AMD's renewed ability to compete with Intel in the high-end CPU market. In 2016, Keller joined Tesla, becoming vice president of autopilot hardware engineering. In 2018, he became a senior vice president for Intel. In 2020, he resigned due to disagreements over outsourcing production but quickly found a new position at Tens Torrent as Chief Technical Officer.

We're going to sit today and discuss the perils and promise of artificial intelligence, and it's a conversation I'm very much looking forward to. So welcome to all of you watching and listening. I thought it would be interesting to have a three-way conversation. Jonathan and I have been talking a lot lately, especially with John Vervaki and some other people as well, about the fact that we seem—it seems necessary for us to view, for human beings to view the world through a story. In fact, when we describe the structure that governs our action and our perception, that is a story.

And so we've been trying to puzzle out, I would say, to some degree on the religious front, what might be the deepest stories. And I'm very curious about the fact that we perceive the world through a story—human beings do—and that seems to be a fundamental part of our cognitive architecture and of cognitive architecture in general. According to some of the world's top neuroscientists, and I'm curious, and I know Jim is interested in cognitive processing and in building systems that, in some sense, seem to run in a manner analogous to the manner in which our brains run.

And so I'm curious about the overlap between the notion that we have to view the world through a story and what's happening on the AI front. There's all sorts of other places that we can take the conversation. So maybe I'll start with you, Jim. Do you want to tell people what you've been working on and maybe give a bit of a background to everyone about how you conceptualize artificial intelligence?

Yeah, sure. So first I'll say technically I'm not an artificial intelligence researcher; I'm a computer architect. And I'd say my skill set goes from, you know, somewhere around the atom up to the program. So we make transistors at atoms; we make logical gates out of transistors; we make computers out of logical gates; we run programs on those. And recently, we've been able to run programs fast enough to do something called an artificial intelligence model or neural network, depending on how we say it.

And then we're building chips now that run artificial intelligence models fast, and we have a novel way to do it at a company I work at, but lots of people are working on it. And I think we were sort of taken by surprise with what's happened in the last five years—how quickly models started to do interesting and intelligent-seeming things.

There's been an estimate that human brains do about 10 to the 18th operations a second, which sounds like a lot. It's a billion billion operations a second, and a little computer, you know, the processor in your phone probably does 10 billion operations a second, you know, ish. And then, if you use the GPU, maybe 100 billion, something like that. And big modern AI computers—like OpenAI, use this—or Google or somebody, they're doing like 10 to the 16th, maybe slightly more operations a second. So they're within a factor of a hundred of a human brain's raw computational ability.

And by the way, that could be completely wrong. Our understanding of how the human brain does computation could be wrong. But lots of people have estimated based on the number of neurons, number of connections, how fast neurons fire, how many operations a neuron firing seems to involve. I mean, the estimates range by a couple of orders of magnitude. But when our computers got fast enough, we started to build things called language models and image models that do fairly remarkable things.

So what have you seen in the last few years that's been indicative of this, of the change that you described as revolutionary? What are the com—what are computers doing now that you found surprising because of this increase in speed?

Yeah, you can have a language model read a 200,000-word book and summarize it fairly accurately, so it can extract out the gist of it. Can it do that with fiction?

Yeah, yeah! And I'm going to introduce you to a friend who took a language model and changed it and fine-tuned it with Shakespeare and used it to write screenplays that are pretty good, and these kinds of things are really interesting.

And then we were talking about this a little bit earlier. So when computers do computations, you know, a program will say, "Add A equal B plus C." The computer does those operations on representations of information—ones and zeros. It doesn't understand them at all; the computer has no understanding of it. But what we call a language model translates information like words and images and ideas into a space where the program, the ideas and the operation it does on them are all essentially the same thing.

We'll be right back with Jonathan Pajo and Jim Keller. First, we wanted to give you a sneak peek at Jordan's new documentary, Logos and Literacy. I was very much struck by how the translation of the biblical writings jump-started the development of literacy across the entire world. Illiteracy was the norm. The pastor's home was the first school, and every morning it would begin with singing. The Christian faith is a singing religion. Probably 80 percent of scripture memorization today exists only because of what is sung. This is amazing. Here we have a Gutenberg Bible printed on the press of Johann Gutenberg. Science and religion are opposing forces in the world, but historically that has not been the case. Now the book is available to everyone—from Shakespeare to modern education and medicine and science to civilization itself. It is the most influential book in all history, and hopefully people can walk away with at least a sense of that.

Right, so a language model can produce words and then use those words as inputs, and it seems to have an understanding of what those words are, which is very different from our computer.

I'm curious—operates on data about the language models. I mean, my sense at least in part how we understand the story is that maybe we're watching a movie, let's say, and we get some sense of the character's goals, and then we see the manner in which that character perceives the world, and we, in some sense, adopt his goals, which is to identify with the character. And then we play out a panoply of emotions and motivations on our body because we now inhabit that goal space, and we understand the character as a consequence of mimicking the character with our own physiology.

And you have computers that can summarize the gist of a story, but they don't have that underlying physiology. First of all, that's a theory that your physiology has anything to do with it. You could understand the character's goals and then get involved in the details of the story, and then you're predicting the path of the story and also having expectations and hopes for this story.

Yeah, and a good story kind of takes you on a ride because it teases you with doing some of the things you expect, but also doing things that are unexpected, and possibly that creates emotional—that could—yeah, it does, it does.

So in an AI model, so you can easily have a set of goals, so you have your personal goals. And then when you watch the story, you have those goals. Yeah, you put those together like how many goals is that? Like the story's goals and your goals—hundreds? Thousands? Those are small numbers, right? Then you have the story—the AI model can predict the story too just as well as you can.

Howdy! And that's the thing that I find mysterious. As the story progresses, it can look at the error between what it predicted and what actually happened, and then iterate on that, right? So you would call that emotional excitement, disappointment, anxiety?

Yeah, definitely. Well, a big part of what anxious discrepancy—in fact, some of those states are manifesting in your body because you trigger hormone cascades, a bunch of stuff, but you also can just scan your brain and see that stuff move around, right?

Right, and you know the AI model can have an error function and look at the difference between what it expected and not, and you could call that the emotional state.

Yeah, yeah. Well, if you want it, I just talked with it. But not speculation. But no, no, I think that's actually—it’s... but you know we can make an AI model that could predict a result of a story probably better than the average person.

So one of some people are really good at—you know they’re rarely well-educated about stories or they know the genre or something. But, yeah—but you know these things, and what they see today is the capacity of the models is if you say, "Start describing a lot," it'll make sense for a while, but it'll slowly stop making sense. But that's possible—that's simply the capacity of the model right now, and the model is not grounded enough in the subtlety to say goals in reality or something to make sure.

So what do you think would happen, Jonathan? This is, I think, associated with the kind of things that we've talked through to some degree. So one of my hypotheses, let's say, about deep stories is that they're meta-gists in some sense. So you could imagine a hundred people telling you a tragic story, and then you could reduce each of those tragic stories to the gist of the tragic story, and then you could aggregate the gists, and then you'd have something like a meta-tragedy. And I would say the deeper the gist, the more religious light the story gets. And that's part of—it's that idea is part of the reason that I wanted to bring you guys together.

I mean one of the things that what you just said makes me wonder is: imagine that you took Shakespeare and you took Dante and you took like the canonical Western writers, and you trained an AI system to understand the structure of each of them. And now you have—you can pull out the summaries of those structures, the gists, and then couldn't you pull out another gist out of that? So it would be like the essential element of Dante and Shakespeare.

And I want to hear what John thinks.

So here's one funny thing to think about: you use the word "pull out." So when you train a model to know something, you can't just look in it and say, "What is it?" No, you have to quirk it.

Right, what's the next sentence in this paragraph? What's the answer to this question? There's a thing on the internet now called prompt engineering, and it's the same way. I can't look in your brain to see what you think.

Yeah, I have to ask you what you think because if I killed you and scanned your brain and got the current state of all the synapses and stuff, A, you'd be dead—which should be sad—and B, I wouldn't know anything about your thoughts. Your thoughts are embedded in this model that your brain carries around, and you can express it in a lot of ways.

And so that—so you could—how do you train? So this is my big question. Because the way that I've been seeing it until now is that artificial intelligence is—it’s based on us. It doesn't exist independently from humans, and it doesn't have care. The question would be: why does the computer care?

Yeah, that's—yeah, that's not true.

Well, what does a computer care to get the gist of the story?

Well, yeah, so I think you're asking kind of the wrong question. So you can train an AI model on like the physics and reality and images in the world just with images, and there are people who are figuring out how to train a model with just images. But the model itself still conceptualizes things like tree and dog and action and run because those all exist in the world, right?

So you can actually train—so when you train a model with all the language and words—so all information has structure, and I know you're a structure guy from your video. So if you look around you at any image, every single point you see makes sense, right? It's a teleological structure; it's like a purpose-in-laden structure, right?

So this is something—so it turns out all the words that have ever been spoken by human beings also have structure, right? And so physics has structure, and then it turns out that some of the deep structure of images and actions and words and sentences are related. Like there's actually a common core of—like you imagine there's like a knowledge space, and sure there's details of humanity where they prefer this accent versus that. Those are kind of details, but they're coherent in the language model, but the language models themselves are coherent with our world ideas, and humans are trained in the world just the way the models are trained in the world.

They look at a baby as it's learning, looking around—it's training on everything it sees when it's very young, and then its training rate goes down, and it starts interacting with what it's learning, interacting with the people around. But it's trying to survive. It's trying to live. It has a—the infant or the child has—kids aren't trying; the weights and the neurons aren't trying to live. What they're trying to do is reduce the error.

So neural networks generally are predictive things—like what's coming next, what makes sense, you know, how does this work? And when you train them—when you train an AI model, you're training it to reduce the error in the model.

And if you're modeling, okay, let me ask you about that. So first of all—and so babies are doing the same thing; like they're looking at stuff going around, and in the beginning, their neurons are just randomly firing, but as it starts to get object permanence, it's looking at stuff. It starts predicting what will make sense for that thing to do, and when it doesn't make sense, it'll—it'll update as well, basically it compares its prediction to the events, and then it will adjust its prediction. So in a story prediction model, the AI would predict the story, then compare it to its prediction, and then fine-tune itself slowly as it trains itself.

Okay, reverse—you could ask it to say, given the set of things, tell the rest of the story, and it could do that, right? And that's what—and the state of it right now is there are people having conversations with this that are pretty good. So I talked to Carl Friston about this prediction idea in some detail, and for those of you who are watching and listening, he's one of the world's top neuroscientists, and he's developed an entropy enclosure model of conceptualization, which is analogous to one that I was working on, I suppose across approximately the same time frame.

So the first issue—and this has been well established in the neuropsychological literature for quite a long time—is that anxiety is an indicator of discrepancy between prediction and actuality, and then positive emotion also looks like a discrepancy reduction indicator. So imagine that you're moving towards a goal, and then you evaluate what happens as you move towards the goal, and if you're moving in the right direction, what happens is what you might say, what you’ll expect to happen, and that produces positive emotion, and it's actually an indicator of reduction in entropy. That's one way of looking at it.

Then the point is, yeah, yeah—you have a bunch of words in there that are psychological definitions of states, but you could say there’s a prediction in there—a prediction, yes, you're reducing error—but what’s simple—but what I’m trying to make a case for is that your emotions directly map, that both positive and negative emotion looks like they’re signifiers of discrepancy reduction.

Well, on the positive and negative emotion side, but then there's a complexity that I think is germane to part of Jonathan's query, which is that—the neuropsychologists and the cognitive scientists have talked a lot about expectation, prediction, and discrepancy reduction, but one of the things they haven't talked about is: it isn't exactly that you expect things. It's that you desire them; you want them to happen.

Like because you could imagine that there's, in some sense, a literally infinite number of things you could expect, and we don’t strive only to match prediction; we strive to bring about what it is that we want. And so we have these preset systems that are teleological, that are motivational systems.

Well, I mean it depends; like if you're sitting idly on the beach, like—and a bird flies by, you expect it to fly along in a regular path, right? But you don't really want that to happen.

Yeah, but you don't want it to turn into something that could peck out your eyes either, like there's a disaster.

Yeah, but you're kind of following it with your expectation to look for discrepancy.

Yes, now you'll also have a, you know, depends on the person, somewhere between 10 and a million desires, right? And you also have fears and avoidance, and those are contexts.

So if you’re sitting on the beach with some anxiety that the birds are going to swerve at you and knock your eyes out, so then you might be watching it much more attentively than somebody who doesn't have that worry, for example.

Well, but both of you can predict where it's going to fly, and you will both notice a discrepancy, right? The motivations—one way of conceptualizing fundamental motivation is they're like don’t—they're like a primary prediction domains, right? And so it helps us narrow our attentional focus.

Because I know when you're sitting and you're not motivated in any sense, you can be doing just in some sense trivial expectation computations. But often we're in a highly motivated state, sure, and what we're expecting is bounded by what we desire, and what we desire is oriented, as Jonathan pointed out, towards the fact that we want to exist.

And one of the things I don't understand and wanted to talk about today is how the computer models, the AI models can generate intelligible sense without this—without mimicking that sense of motivation. Because you've said, for example, they can just derive the patterns from observations of the objective world, but...

So let's—so again, I don’t want to do all the talking, but so AI, generally speaking, like when I first learned about it, had two behaviors they call it inference and training. So inference is you have a trained model, so you say—you give it a picture and say, "Is there a cat in it?" and it tells you where the cat is. That's inference—the model has been trained to know where a cat is.

And training is the process of giving it an input and an expected output, and when you first start training the model, it gives you garbage out like an untrained brain. But then you take the difference between the garbage output and the expected output and call that the error, and then the big revelation was something called backpropagation with gradient descent. That means it takes the error and divides it up across the layers and corrects those calculations so that when you put a new thing in, it gives you a better answer.

And then—to somewhat my astonishment—if you have a model of sufficient capacity and you train it with a hundred million images, if you give it a novel image and say, "Tell me where the cat is," it can do it, right? That's called inference.

That's called—inference is the behavior of putting something in and getting it out.

Yeah, I think I—I'm really pulling. But there's a third piece, which is what the new models do, which is called a generative model. So for example, say you put in a sentence and you say, "Predict the next word." This is the simplest thing, so it predicts the next word. So you add that word to the input, and I'll say, "Predict the next word." So it contains the original sentence and the word you generated, and it keeps generating words that make sense in the context of the original word, in addition, right?

Right, right, this is the simplest basis, and then it turns out you can train this to do lots of things. You can change it to summarize a sentence; you can train it to answer a question. There's a big thing about, you know, like Google, every day has hundreds of millions of people asking it questions, giving answers, and then rating the results. You can train a model with that information so you can ask it a question, it gives you a sensible answer.

But I think in what you said, I actually have—the issue that has been going through my mind so much is when you said, you know, people put in the question and then they rate the answer, my intuition is that the intelligence still comes from humans in the sense that it seems like in order to train whatever AI you have to be able to give it a lot of power and then say at the beginning, "This is good, this is bad," "This is good, this is bad," like reject certain things, accept certain things in order to then reach a point when then you train the AI.

And so that's what I mean about the care. So the care will come from humans because the care is the one giving it the value—saying this is what is valuable, this is what is not valuable in your calculation.

So when they first—so there's the program called AlphaGo that learned how to play Go better than a human. So there's two ways to train the model. One is they have a huge database of lots of Go games with good winning moves, so they trained the model with that, and that worked pretty good. And they also took two simulations of Go and they did random moves. And all that happened was these two simulators played one Go game, and they just recorded whichever moves happened to win, and it started out really horrible, and just started training the model.

This is called adversarial learning—it's a particular adversarial—it’s like, you know, you make your moves randomly and you train a model, and so they train multiple models, and over time, those models got very good, and they actually got better than human players because the humans have limitations about what they know, whereas the models could experiment in a really random space and go very far.

Yeah, but experiment towards the brothers watching the game.

Yes, well, but you can experiment towards all kinds of things. It turns out, and humans are also training that way—like when you were learning, you were reading, you were saying, "This is a good book, this is a bad book, this is good sentence construction, it's good spelling." So you've gotten so many error signals over your life.

Well, that's what culture does in large partners. Culture does that; religion does that. Your everyday experience does that; your family.

So we embody that, right? And we're all—and everything that happens to us, we process it on the inference pass, which generates outputs, and then sometimes we look at that and say, "Hey, that's unexpected," or "That got a bad result," or "That got bad feedback," and then we backpropagate that and update our models.

So could very well-trained models can then train other models. So the difference right now is—are the smartest people in the world? So the biggest question that comes now based on what you said is because my main point is to try to show how it seems like artificial intelligence is always an extension of human intelligence. Like it remains an extension of human intelligence, and maybe the way down can't be true at all.

So do you think that at some point the artificial intelligence will be able to—because the goals recognizing cats, writing plays, all these goals are goals which are based on embodied human existence—could you train—what could an AI, at some point, develop a goal which would be uncomprehensible to humans because of its own existence?

Yeah, I mean like for example, there's a small population of humans that enjoy math, right? And they are pursuing, you know, adventures in math space that are incomprehensible to 99.99% of humans, but they're interested in it. And you can imagine an AI program working with those mathematicians and coming up with very novel math ideas and then interacting with them, but they could also—you know, if some AIs were elaborating out really interesting and detailed stories, they could come up with stories that are really interesting. We're going to see it pretty soon; like could there be everything—a story that is interesting only to the AI and not interesting to us? That's possible.

So stories are like, I think, some high-level information space. So the computing age of Big Data—there's all this data running on computers where nobody—only humans understood it, right? The computers don't. So AI programs are now at the state where the information, the processing, and the feedback loops are all kind of in the same space. They're still, you know, relatively rudimentary to humans like in some AI programs, and certain things are better than humans already, but for the most part, they're not. But it's moving really fast.

And so you could imagine—I think in five or ten years, most people's best friends will be AIs, and you know, they'll know you really well and they'll be interested in you. And you know, it's kind of like your real friends.

Yeah, real friends are problematic. They're only interested in you when you're interested.

Yeah, real friends are—the AI systems will love you even when you're dull and miserable.

Well, there's so much idea space to explore, and humans have a wide range. Some humans like to go through their everyday life doing their everyday things, and some people spend a lot of time like you, a lot of time reading and thinking and talking and arguing and debating, you know, and you know, that's—there's going to be, like, say a diversity of possibilities with what's—what a thinking thing can do when the thinking is fairly unlimited.

So I'm curious about—I'm still curious in pursuing this issue that Jonathan has been developing.

So there's a—there's literally an infinite number of ways, virtually an infinite number of ways, that we could take images of this room right now. If a human being is taking images of this room, they're going to sample a very small space of that infinite range of possibilities because if I was taking pictures in this room, in all likelihood, I would take pictures of identifiable objects that are functional to human beings at a level of focus that makes those objects clear.

And so then you could imagine that the set of all images on the internet has that implicit structure of perception built into it, and that's a function of what human beings find useful. You know, I mean, I could take a photo of you that was—the focal depth was here and here and here and here and here and two inches past you.

And now I suppose you—because there's a technology for that called light fields.

Okay, so then you could—if you had that picture properly done, then you could move around in an image and see.

But, yeah, fair enough, I get your point. Like the human recorded data has—here's our biology built into it—has our biology built into it.

But also, unbelievably detailed encoding of how physical reality works, right? So every single pixel in those pictures—even though you kind of selected the view, the focus, the frame, right—it still encoded a lot more information than you're processing, right?

And if you take a large—it turns out if you take a large number of images of things in general, so you've seen these things where you take a 2D image and turn it into a 3D image, yeah?

Right, the reason that works is even in the 2D image, the 3D's in the room actually got embedded in that. And then if you have the right understanding of how physics and reality works, you can reconstruct a 3D model. You know, an AI scientist may cruise around the world with infrared and radio wave cameras, and they might take pictures of all different kinds of things, and every once in a while they'd show up and go, "Hey, the sun, you know, I've been staring at the sun in ultraviolet and radio, yes, for the last month, and it's way different than anybody's thought," because humans tend to look at light in the visible spectrum.

And, you know, there could be some really novel things coming out of that.

Well, so—but humans also—we live in the spectrum we live in because it's a pretty good one for planet Earth.

Like it wouldn't be obvious that AI would start some different place; like visible spectrum is interesting for a whole bunch of reasons, right?

So in a set of images that are human-derived, you're saying that there's—the way I would conceptualize that is that there are two kinds of logos embedded in that. One would be that you could extract out from that set of images what was relevant to human beings, but you're saying that the fine structure of the objective world outside of human concern is also embedded in the set of images, and that an AI system could extract out a representation of the world but also a representation of what's motivating to human beings.

Yes, so—and then some human scientists already do look at the sun in radio waves and other things because they're trying to, you know, get different angles on how things work.

Yeah, well, I guess it's a curious thing—it's like the same with, like, buildings and architecture. Mostly fit people—well, the other—you know, there’s a reason for that.

The reason that I keep coming back hammering the same point is that even in terms of the development of the AI that is developing, AI requires immense amount of money, energy, you know, and time. And so that's a transient thing; in 30 years it won't cost anything. So that's going to change so fast—it's amazing.

So that's a like super computers used to cost millions of dollars, and now your phone is the supercomputer. So it's—the time between millions of dollars and ten dollars is about 30 years.

So I'm just saying, the time and effort isn't a thing in technology; it's moving pretty fast.

It's just— that just says the date.

Yeah, but even making—let's say even— I mean, I guess maybe the—the nightmare question—like could you imagine an AI system which becomes completely autonomous, which is creating itself even physically through automated factories, which is, you know, programming itself, which is creating its own goals, which is not at all connected to human endeavor?

Yeah, I mean, individual researchers can— you know, I have a friend, who I’m going to introduce you to him tomorrow. He wrote a program that scraped all of the internet and trained an AI model to be a language model on a relatively small computer. And in 10 years, the computer he could easily afford would be as smart as a human.

So he could train that pretty easily, and that model could go on Amazon and buy a hundred more of those computers and copy itself. So yeah, we’re—we're 10 years away from that.

And then—but then why? Like why would it do that? I mean, what does it—does it—is it possible?

It's all about the motivational question. I think that’s what even Jordan and I both have been coming at from the outset is like, so you have an image, right? You have an image of Skynet or of the Matrix, you know, in which the sentient AI is actually fighting for its survival. So it has a survival instinct which is pushing it to self-perpetuate—to replicate itself and to create variation on itself in order to survive and identify as humans as an obstacle to that, you know?

Yeah, yeah. So you have a whole bunch of implicit assumptions there. So humans, last I checked, are unbelievably competitive, and when you let people get into power with no checks on them, they typically run amok. It's been historical experience. And then humans are, you know, self-regulating to some extent, obviously with some serious outliers because they self-regulate with each other.

And humans and AI models at some point will have to find their own calculation of self-regulation and trade-offs about that.

Yeah, because AI doesn't feel pain—at least as we—we don’t know that it feels.

Well, lots of humans don't feel pain either.

So I mean, that's—I mean, humans feeling pain or not didn't, you know, it doesn't stop a whole bunch of activity. I mean, that's—I mean, it doesn't—the fact that we feel pain doesn't stop many people, right?

Right, I mean, there's definitely people, like, you know, children—if you threaten them with, you know, "Go to your room," and stuff, you can regulate them that way, but some kids ignore that completely—and adults, and it's often counterproductive.

Yeah, so right. You know, culture and societies and organizations—we regulate each other, you know, sometimes in competition and cooperation.

Yeah. Do you think that—we’ve talked about this to some degree for decades? I mean, when you look at how fast things are moving now and as you push that along, and what—when you look out ten years and you see the relationship between the AI systems that are being built and human beings, what do you envision? Or can you envision it?

Okay, well, can I—yeah, so like I said, I'm a computer guy, and I'm watching this with, let's say, some fascination as well. I mean the last—so Ray Kurzweil said you know prior progress accelerates, right? So we have this idea that 20 years of progress is 20 years, but you know the last 20 years of progress was 20 years, and the next 20 years would probably be, you know, 5 to 10, right?

Right, right.

And you can really feel that happening. To some level, that causes social stress, independent of whether it's AI or Amazon deliveries, you know?

There's so many things that are going into the stress of it all, but there's progress which is an extension of human capacity and then there's this progress which I'm hearing about the way that you're describing it, which seems to be an inevitable progress towards creating something which is more powerful than you, right?

And so what is that? I don't even understand that drive; like what is that drive to create something which can supplant you?

Look at the average person in the world, right? So the average person already exists in this world because the average person is halfway up the human hierarchy. There's already many people more powerful than any of us. They could be smarter; they could be richer; they could be better connected. We already live in a world like that. Very few people are at the top of anything, right?

So that's already a thing. So basically the drive to make someone a superstar, let's say, or the drive to elevate someone above you—that would be the same drive that is bringing us to creating these ultra-powerful machines, because we have that. Like we have a drive to elevate.

Like, you know, when we see a rock star that we like, people want to submit themselves to that. They want to dress like them. They want to raise them up above them as an example—something to follow, right? Something to say—to subject themselves to. You see that with leaders. You see that in the political world, ah, and in teams. You see that in sports teams—the same thing.

So do you think we've always tried to build things that are beyond us? You know, I mean, I mean, it's about—is we building a god? Is that what people—that is the drive that is pushing someone towards? Because when I hear what you're describing, Jim, I hear something that is extremely dangerous, right?

It sounds extremely dangerous to the very existence of humans, yet I see humans acting and moving in that direction almost without being able to stop it, as if there's no one.

Now, I think it is unstoppable. That's one of the things we've also talked about, is because I've asked Jim straight out, you know, because of the hypothetical danger associated with this—why not stop doing it?

And, well, part of his answer is the ambivalence about the outcome, but also that it isn't obvious at all that, in some sense, it's stoppable. I mean, it's the cumulative action of many, many people that are driving us along, and even if you took out one player— even a key player—the probability that you do anything but slowing infinitesimally is quite happy because there’s also a massive payoff for those that will succeed.

It's also set up that way. People know that at least until the AI take over, whatever that is, whoever is on the line towards increasing the power of the AI will rake in major rewards.

Right, well, so there's a cognitive acceleration, right? Yeah, I can recommend Ian Banks as an author—an English author. I think he wrote a series of books on what he called the culture novels, and it was a world where there were humans and then there were AIs—the smartest humans and AIs that were dumber than humans. But there were some AIs that were much, much smarter, and they lived in harmony because they mostly all pursued what they wanted to pursue. Humans pursued human goals, and super smart AIs pursued super smart AI goals, and, you know, they communicated and worked with each other.

But they mostly—you know, they were different enough that that was problematic; their goals were different enough that they didn't overlap.

Because, one of the things that would be my guess is, like, these ideas were these super AIs get smart, and the first thing they do is stomp out to humans.

It's like you don't do that.

Like, you like—you don't wake up in the morning and think, "I have to stomp out all the cats."

No, the cats do cat things, and the ants do ant things, and the birds do bird things, and, you know, super smart mathematicians do smart mathematician things, and you know, guys who like to build houses do building house things, and you know, everybody—there's so much space in the intellectual zone that people—people tend to go pursue the—in a good society, like you tend to pursue the stuff that you do.

And then the people in your zone, you self-regulate, and you also— even in the social strategies, we self-regulate.

I mean, the recent political events of the last 10 years—the weird thing to me has been why, you know, people with power have been overreaching to take too much from people with less. Like that's bad regulation.

But one of the things—go ahead.

Of the aspects of increase in power is that increase in power is always mediated at least in one aspect by the military—by military, by, let's say, physical power on others, you know? And we can see that technology is linked and has been linked always to military power. And so the idea that there could be some AIs that will be our friends or whatever is maybe possible; but the idea that there will be some AIs which will be weaponized seems absolutely inevitable because increase in power is always to increase in technological power always moves towards military.

So we've lived with atomic bombs since the 40s, right? So the—I mean the solution to this has been mostly, you know, some form of mutual assured destruction or attacking me. Like the response to attacking me is so much worse than the—

Yeah, but it's also because we rest—we have reciprocity. We recognize each other as the same. So if I look into the face of another human, there's a limit of how—how different I think that person is from me. But if I'm hearing something described as the possibility of super intelligences that have their own goals, their own cares, their own structures, then how much mirror is there between these two groups of people, these two groups?

Well, the objection seems to be something like, we're making—we may be making—when we're doomsaying, let's say, and I'm not saying there's no place for that, we're making the presumption of something like a zero-sum competitive landscape, right?

Is that the idea? And the idea behind movies like the, like, the Terminator, is that there is only so much resources, and the machines and the human beings would have to fight over it, and you can see that that that could easily be a preposterous assumption.

Now, I think that one of the fundamental points you're making, though, is also, there will definitely be people that will weaponize AI, and those weaponized AI systems will have as their goals something like the destruction of human beings, at least under some circumstances, and then there's the possibility that that will get out of control because the most effective systems out destroying human beings might be the ones that win, let's say.

And that could happen independently of whether or not it is a true zero-sum competition.

Yeah, and also, the—to the extent that we self-regulated through the nuclear crisis, it’s interesting. I don't know if it's because we thought that the Russians were like us. I kind of suspect the problem was that we thought they weren't like us, and but we still managed to make some calculation to say that any kind of attack would be mutually devastating.

Well, when you look at, you know, the destructive power of the military we already have so far exceeds the planet, I'm not sure, like, adding intelligence to it is the tipping point.

Like, that's—and I think the more likely thing is things that are truly smart in different ways will be interested in different things, and then the possibility for, let's say, mutual flourishing is really interesting. And I know artists using AI already to do really amazing things, and that's already happening.

Well, when you're working on the frontiers of AI development and you see the development of increasingly intelligent machines, I mean, I know that part of what drives you is—I don't want to put words in your mouth—but what drives intelligent engineers in general, which is to take something that works and make it better, and maybe to make it radically better and radically cheaper.

So there's this tech drive toward technological improvement. And I know that you like to solve complex problems, and you do that extraordinarily well. But do you—is there also a vision of a more abundant form of human flourishing emerging from the development?

So what do you see happening? Years ago, it's like we’re going to run out of energy. What's next? We're going to run out of matter, right? Like our ability to do what we want in ways that are interesting and funny— for some people, beautiful is limited by a whole bunch of things, because we're, you know, partly it’s technological and partly what's—you know we're stupidly divisive.

But there is—there's a possibility that there’s also a reality which is one of the things that technology has been is, of course, an increase in power towards desire—towards human desire. And that is represented in mythological stories where let's say technology is used to accomplish impossible desire, right?

We have, you know, the story of the story of building the mechanic—the bull around the king of Mino—the wife of the king of Minos, you know in order to be inseminated by a bull. We have the story of the—the we have the story of the—I swear, Frankenstein, etc. The story of the Golem where we put our desire into this increased power, and then what happens is that we don't know our desires.

That's one of the things that I've also been worried about in terms of AI is that we act—we have secret desires that enter into what we do that people aren't totally aware of. And as we increase empower these systems, those desires—let’s say the idea, for example, of the possibility of having an AI friend—and the idea that an AI friend would be the best friend you've ever had because that friend would be the nicest to you, would care the most about you, would do all those things—that would be an exact example of what I'm talking about—which is it's really the story of the genie, right?

It's the story of the genie and the lamp where the genie says, "What do you wish?" And the person—and I have unlimited power to give it to you—and so I give him my wish. But that wish has all these underlying implications that I don't understand—all these underlying possibilities.

The cool thing—almost all those stories is having unlimited wishes will lead to your downfall. And so humans—like if you give, you know, a young person an unlimited amount of stuff to drink for six months, they're going to be falling down drunk and they're going to get over it, right?

Having a friend that's always your friend no matter what is probably going to get boring pretty well.

The literature on marital stability indicates that—so there's a sweet spot with regards to marital stability in terms of the ratio of negative to positive communication. So if on average you receive five positive communications and one negative communication from your spouse—that's on the low threshold for stability.

If it's four positive to one negative, you're headed for divorce. But interestingly enough on the other end, there's a threshold as well which is that if it exceeds eleven positive to one negative, you're also moving towards divorce.

So there might be self-regulating mechanisms that would incense take care of that. You might find a Yes Man AI friend extraordinarily boring very, very rapidly; but as opposed to an AI friend that was interested in what you're interested in—it was actually interesting.

Like, you know, we go through friends in the course of our lives—like different friends are interesting at different times. And some friends we grow with, and that continues to be really interesting for years and years, and other friends—some people get stuck in their thing, and then you've moved on or they've moved on or something.

So yeah, I—so we—I tend to think of a world where there was more abundance and more possibilities and more interesting things to do is is an interesting.

Okay, okay. So as modern society has let the human population—and some people think this is a bad thing, but I don't know, I'm a fan of it; you know, modern population has gone from tens of—100 million to billions of people. That's generally being a good thing.

We're not running out of space. I've been in—some of your audience has probably been in an airplane. If you look out the window, the country is actually mostly empty. The oceans are mostly empty. Like we're weirdly good at polluting large areas, but as soon as we decide not to, we don't have to.

Like technology—most—most of our energy pollution problems are technical, like we can stop polluting. Like electric cars are great.

So here's so many things that we could do technically.

I forget the guy's name; he said the Earth could easily support a population of petroleum people, and a trillion people would be a lot more people doing, you know, random stuff.

And he didn't imagine that the future population would be a trillion humans and a trillion AIs, but it probably will be. So we'll probably exist on multiple planets, which would be good the next time an asteroid shows up.

So what do you think about—so one of the things that seems to be happening—tell me if you think I'm wrong here, and I think it’s germane to—and Jordan, I just want to make the point: you know, where we are compared to living in the Middle Ages, our lives are longer. Our families are healthier. Our children are more likely to survive.

Like many, many good things happen. Like setting the clock back would be good, you know, if we have some care and people who actually care about how culture interacts with technology for the next 50 years, you know, we'll get through this hopefully more successful than we did the atomic bomb and the Cold War.

But it's okay, so it's a major change. I mean this is like—like your worries are, you know, I mean, they're relevant, but that, you know—also, you're Jonathan—your stories about how humans have faced abundance and faced evil kings and evil overlords—like we have thousands of years of history of facing the challenge of the future and the challenge of things that cause radical change.

Yeah, but it's just that—that's very valuable information, you know? Information—but for the most part, nobody succeeded by stopping change. They've succeeded by bringing to bear on the change our capability to self-regulate the balance.

Like a good life isn't having as much gold as possible; it's a boring life. A good life is, you know, having some quality friends and doing what you want and having some insight in life and some optimal challenge.

And, you know, and then in a world where a larger percentage of people can have wealth, live in relative abundance, and have tools and opportunities, I think is a good thing.

Yeah, and I don't want to pull back abundance, but what I have noticed is that our abundance brings a kind of nihilism to people.

And I don't like—I said, I don't want to go back. I'm happy to live here and to have these tech things, but I think it's something that I've also noticed that increase of the capacity to get your desires—when that increases to a certain extent, also leads to a kind of nihilism where—exactly that.

Well, yeah, I wonder, Jonathan said—I wonder if that's part, partly a consequence of the erroneous maximization of short-term desire. I mean, one of the things that you might think about that could be dangerous on the AI front is that we optimize the manner in which we interact with our electronic gadgets to capture short-term attention, right?

Because there's a difference between getting what you want right now and getting what you need in some more mature sense across a reasonable span of time. One of the things that does seem to be happening online, and I think it is driven by the development of AI systems, is that we're assaulted by systems that parasitize our short-term attention at the expense of longer-term attention.

And I—if the AI systems emerge to optimize attentional grip, it isn't obvious to me that they're going to optimize for the attention that works over the medium to long run, right? They're gonna—they could conceivably maximize something like whim-centered.

Exactly, because all the virality is based on that—all the social media, yeah—they're all based on this reduction—this reduction of attention, this reduction of desire to reaching your rest, let's say, in that desire, right?

Now, yeah—exactly. So—but that's something that, you know, for reasons that are somewhat puzzling, but maybe not, you know, the business models around a lot of those interfaces are around, you know, the part—the users—the product, and you know that the advertisers are trying to get your attention, yeah.

But that's something culture could regulate. We could decide that, no, we don't—we don't want tech platforms to be driven by advertising money like that would be a smart decision probably.

And that could be a big change, and also if you see it as an older—see, well, the problem is markets drive that in some sense, right?

And yeah, and I know they're driving that way; we can take steps—like, you know, at various times, you know, alcohol's been illegal. Like you can—society can decide to regulate all kinds of things, and you know, sometimes some things need to be regulated, and some things don't.

Like, when you buy a hammer, you don't fight with your hammer for its attention.

Right? It’s a tool. You buy one when you need one. Nobody’s marketing hammers to you.

Like that—that has a relationship that’s transactional to your purpose, right?

Yeah, well, our technology has become a thing where, I mean, but there's a relationship, I would say, between high human goals—something like attention and status—and what we talked about, which is the idea of elevating something higher in order to see it as a model.

See, these are where intelligence exists in the human person, and so when we notice that in the systems—in the platforms, these are the aspects of intelligence which are being weaponized in some ways—not against us—but are just kind of being weaponized because they're the most beneficial at the short term to be able to generate our constant attention.

And so what I mean is that is what the—as AIs are made of, right? They're made of attention, prioritization, you know, good, bad. What is it that is worth putting energy into in order to predict towards a telos?

And so, yeah, I’m seeing that it's the idea that we could disconnect them suddenly seems very difficult to me.

Yeah, so I'll give it to—first I want to give an old example. So after World War II, America went through this amazing building boom of building suburbs, and the American dream was you could have your own house, your own yard in the suburb with a good school.

Right, so in the '50s, '60s, early '70s, they were building that like crazy. By the time I grew up, I lived in this, you know, suburban dystopia, right? And we found that that as a goal wasn't a good thing because people ended up in houses separated from social studies instructors. And then new towns are built around like a hub with, you know, places to go and eat, you know?

So there was a good that was viewed in terms of opportunity and abundance but it actually was a fail culturally. And then some places modified and continued; in some places it’s dystopian.

You know, suburban areas and some places people simply learned to live. By the way, it has to do with a subsidiary hierarchy—a hierarchy of attention which is set up in a way in which all the levels can have room to exist, let's say.

And so, you know, these new—the new systems, the new way, let's say, the new urbanist movement, similar to what you're talking about—that's what they've understood. It’s like we need places of intimacy in terms of the house.

We need places of communion in terms of, you know, parks and alleyways and buildings where we meet, and a church—all these places that kind of manifest our community together. Yeah, so those existed coherently for long periods of time, and then the abundance post World War II and some ideas about like what life could be like caused this big change.

And that change satisfied some needs; people got houses but broke community. Community needs, and then new sets of ideas about what's the synthesis—what's the possibility of having your own home but also having community? Not having to drive 15 minutes for every single thing, and some people live in those worlds and some people don't. Do you think we'll be smart?

So one of the problems—why were we smart enough to solve some of those? Because we had 20 years, but now—because one of the things that's happening now is we're, as you pointed out earlier, we're going to be producing equally revolutionary transformations but at a much smaller scale of time.

So, Mike, one of the things I wonder about, I think it's driving some of the concerns in the conversation is are we going to be intelligent enough to direct with regulation the transformations of technology as they start to accelerate?

I mean we've already—look what's happened online. I mean we've inadvertently, for example, radically magnified the voices of narcissists, psychopaths, and Machiavellians, and we've done that so intensely partly—and I would say partly as a conservation consequence of AI mediation that I think it's destabilizing the entire body.

It's destabilizing part of it—like Scott Adams points out, you just block everybody that acts like that. I don't pay attention to people that talk like that.

Yeah, but they seem—they seem to be real. There's still places that are sensitive to it. Like 10,000 people here can make a storm in some corporate—you know, person, you know, fire somebody.

But I think that's, like, we're five years from that being over; corporations will go 10,000 people out of 10 billion—not a big deal.

Okay, so you think—yeah, that's a learning moment that will re-regulate.

Well, what's natural to our children is so different than it was natural to us, but what was natural to us was very different from our parents, so some changes get accepted generationally, really.

So what's made you so optimistic?

What's means—what do you mean optimistic?

Well, most of the things that you have said today—and maybe it's also because we're pushing you. I mean, you really, you know, my nephew Kyle is a really smart clever guy; he calls me a cynical optimist, like I believe in people.

Like I like people, but also people are complicated, they all got all kinds of nefarious goals. Like I worry a lot more about people burning down the world than I do about artificial intelligence just because you know people, well, you know people, they're difficult, right?

And—but the interesting thing is in aggregate, we mostly self-regulate, and when things change, you have these dislocations, and then it's up to people who talk and think—while we're having this conversation, I suppose to talk about how do we re-regulate this stuff.

Well, because one of the things that the increase in power has done in terms of AI—and you can see it with Google and you can see it online—is that there are certain people who hold the keys, let's say, and then who hold the keys to what you see and what you don't see.

So you see that on Google, right, and you know it if you know what searches to make, where you realize that this is—not—this is actually being directed by someone who now has a huge amount of power in order to direct my attention towards their ideological purpose and so, yeah.

Yeah, yeah, so that's why, like I think that—to me, I personally think it would—I always tend to see AI as an extension of human power even though there is this idea that it could somehow become totally independent. I still tend to see it as an increase of the human care, and whoever will be able to hold the keys to that will have an increase in power, and that can be like—and I think we're already seeing it.

Well, that's—that's not really any different, though, is it, Jonathan, the—the situation that's always confronted us in the past. I mean we've always had to deal with the evil uncle of the king, and we've always had to deal with the fact that an increase in ability could also produce a commensurate increase in tyrannical power, right?

I mean, so that might be magnified now, and maybe the danger in some sense is more acute, but—possibly the possibility is more present as well because you get randomly to find hate speech, right?

You can train an AI to find hate speech and then to act on that hate speech immediately. And now, it’s not only we’re talking about social media, but we’ve seen, we've seen it happen, obviously, in the media recently.

But it—so does decentralization win over centralization? How is that even possible? It seems— I mean, and it's also interesting, like when Amazon became a platform, suddenly any mom-and-pop business could have a—you know, Amazon eBay.

There are a bunch of platforms which had an amazing impact because any business could get to anybody, but then the platform itself started to control the information flow.

Yeah, right, but at some point, that’ll turn into people, like, “Well why am I letting somebody control my information flow when Amazon objectively doesn’t really have any capability?” Right?

So like you point out, though, the waves are getting bigger, but they're real waves. It’s the same with information—the information is all online; it’s also on a billion hard drives, right?

So somebody says, “I’m going to erase objective fact.” The distributed information system would say, “Yeah, go ahead and erase it anywhere you want; here's another thousand copies of it.”

Yeah, and that's what—but again, this is where thinking people have to say “Yeah, this is a serious problem.” Like if humans don't have anything to fight for, they get lazy and, you know, a little bit dopey, in my view.

Like we do have something to fight for, and you know that—that's worth talking about. Like what would a great world with, you know, distributed, you know, human intelligence and artificial intelligence working together in a collaborative way to create abundance and fairness and, you know, like some—some better way at arriving at good decisions than what the truth is—that would be a good thing.

But you know, it's not well— "We'll leave it to the experts, and then the experts will tell us what to do." That's a bad thing.

Yeah, well, so do you see the more likely—certainly the more desirable future is something like a set of distributed AIs, many of which are under personal—in personal relationship, in some sense, the same way that we’re in personal relationship with our phones and our computers, and that that would give people the illustrations back, so to speak, against this?

And there’s lots of people really interested in distributed platforms, and one of the interesting things about the AI world is, you know, there’s a company called Open AI that opened up—open-sourced a lot of it.

The AI research is amazingly open. It's all done in public; people publish the new models all the time; you can try them out. People—there's a lot of startups doing AI in all different kinds of places.

You know, it's a very curious phenomenon, yeah, and it's kind of like a big, huge wave. It's not like a—you can't stop a wave with your hands.

Yeah, what do you think is happening?

Waves—there are two—actually, in the Book of Revelation—which describes the end or describes the finality of all things or the totality of all things—is baby away for people who are more secular to kind of understand it.

And in that—in that book, there are two images—interesting images about technology. One is that there's a dragon that falls from the heavens, and that dragon makes a beast—and then that beast makes an image of the beast.

And then the image speaks, and when the image speaks, then people are so mesmerized by the speaking image that they worship the beast, ultimately. So that is one image of, let's say, making and technology in scripture—in Revelation.

But there’s a lad—another image which is the image of the Heavenly Jerusalem. And that image is more an image of balance. It’s an image of the city which comes down from heaven with a garden in the center and then becomes this glorious city.

And it says the glory of all the kings is gathered into the city, like so the glory of all the nations is gathered into this city.

So now you see a technology which is at the service of human flourishing and takes the best of humans and brings it into itself in order to kind of manifest.

And it also has hierarchy, which means it has the natural at the center and then has the artificial as serving the natural, you could say.

So those two images seem to reflect these two waves that we see. And this kind of idea of an artificial intelligence will be which will be ruling over us or speaking over us, but there’s a—a secret person controlling it. Even in the—in Revelation, it’s like there’s a beast controlling it and making it speak.

So now we're mesmerized by it, and then this other image—so I don’t—I don’t know, Jordan, if you ever thought about those two images in real relation as being related to technology, let’s say.

I don’t think I’ve thought about those two images, right? But I would say that the work that I've been doing—and I think the work you've been doing too in the public front reflects the dichotomy between those images, and it's relevant to the points that Jim has been making.

I mean, we are definitely increasing our technological power, and you can imagine that that'll increase our capacity for tyranny and also our capacity for abundance. And then the question becomes what do we need to do in order to increase the probability that we tilt the future towards Jerusalem and away from the Beast.

And the reason that I've been concentrating on helping people bolster their individual morality to the degree that I've managed that is because I think that whether the outcome is the positive outcome that in some sense Jim is being outlining or the negative outcomes that we've been querying him about, I think that's going to be dependent on the individual ethical choices of people.

Well, at the individual level, but then cumulatively, right? So if we decide that we're going to worship the image of the Beast, so to speak, because we're mesmerized by our own reflection, that's another way of thinking about it, and we want to be the victim of our own dark desires, then the AI revolution is going to go very, very badly.

But if we decide that we're going to aim up in some positive way and we make the right micro decisions, well then maybe we can harness this technology to produce a time of abundance in the manner that Jim is hopeful about.

Yeah, and let me make two funny points. So one is I think there's going to be continuum. Like the word artificial intelligence won't actually make any sense, right?

So humans collectively—like individuals know stuff but collectively we know a lot more, right? And the thing that's really good is in a diverse society with lots of people pursuing individual interesting, you know, ideas, worlds, like we have a lot of things.

And more people, more independence generates more diversity, and that's a good thing. Where, you know, totalitarian society where everybody's told to wear the same shirt and like it’s inherently boring.

Like the Beast speaking through the monsters is inherently dull, right? Like but in an intelligent world where not only can we have more intelligent things but in some places go far beyond what most humans are capable of in pursuit of interesting variety and, you know, like I believe the information—and, well, let’s say intelligence is essentially unlimited, right?

Like—and the unlimited intelligence won't be the shiny thing that tells everybody what to do. That's—that's sort of the opposite of interesting. Intelligence will be more diverse, not less diverse—like that's a—that's a good future.

And your second description—that seems like a future was working for and also was fighting for. And that means concrete things today. And also, you know, it's a—it's a good conceptualization.

Like I see the messages my kids are taught, you know, don’t have children, and the world’s going to end, we’re going to run out of everything, you’re a bad person, why do you even exist?

It's like these messages are terrible. It's the opposite is true; more people would be better. We live in a world of potential abundance, right?

It's right in front of us. Like there’s so much energy available, it’s just amazing. It's possible to build technology without, you know, pollution consequences.

That's called externalizing costs. Like we know how to—we know how to do that. We can have very good, clean technology; we can do lots of interesting things. So if the goal is maximum diversity, then the line between human intelligence and artificial intelligence that we draw will—I think you'll see all these kind of really interesting partnerships and all kinds of things with more people doing what they want, which is the world I want to live in.

Yeah, but to me, it seems like the question is going to be related to attention ultimately. That is, what are humans attending to at their highest? What is it that humans care for in the highest?

You know, in some ways, you could say what do humans—what are humans worshiping? And, like, depending on what humans worship, then their actions will play out in the technology that they're creating and the increase in power that they're creating.

Well, that's—well, and if we're guided by the negative vision, the sort of thing that Jim laid out, that is being taught to his children, you can imagine that we're in for a pretty damn dismal future, right?

Human beings are a cancer on the face of the planet. There's too many of us; we have to accept top-down compelled limits to growth; there's not enough for everybody—a bunch of us have to go because there's too many people on the planet.

We have to raise up the price of energy so that we don't what—burn the planet up with carbon dioxide pollution, etc. It’s a pretty damn dismal view of the potential that's in front of us.

And so, you know, the world should be exciting and the future should be exciting.

Well, we've been sitting here for about 90 minutes, banding back and forth both visions of abundance and visions of apocalypse.

And, I mean, it's—I’ve been heartened, I would say, over the decades talking to Jim about what he's doing on the technological front.

And I think part of the reason I've been heartened is because I do think that his vision is guided primarily by a desire to help bring about something approximating life more abundant.

And I would rather see people on the AI front who were guided by that vision working on this technology. But I also think it’s useful to do what you and I have been doing in this conversation, Jonathan, and acting, in some senses, friendly critics and hopefully learning something in the interim.

Do you have anything you want to say in conclusion?

I mean, I just think that the question is linked very directly to what we've been talking about now for several years, which is the question of attention—the question of what is the highest attention.

And I think the reason why I have more alarm, let's say, than Jim is that I've noticed that in some ways human beings have come to worship their own desires.

They've come to worship—and that even the strange thing of worshiping their own desires is that it actually led to an anti-human narrative. You know this weird idea—it's almost suicidal desire that humans have.

And so I think that seeing all of that together in the increase of power, I worry that the image of the Beast is closer to what will manifest itself.

And I feel like during COVID that sense in me was accelerated tenfold in noticing to what extent technology was used, especially in Canada, how technology was used to instigate something which looked like authoritarian systems.

And so I am worried about it, but I think, like Jim, honestly, although I say that, I do believe that in the end, truth wins. I do believe that in the end, you know, these things will level themselves out.

But I think that because I see people rushing towards AI almost, you know, like lemmings going off a cliff, I feel like it is important to sound the alarm once in a while and say, you know, we need to orient our desire before we go towards these this extreme power.

So I think that that's mostly the thing that worries me the most and that preoccupies me the most. But I think that ultimately, in the end, I do share Jim's positive vision, and I do believe that—I do believe the story has a happy ending. It’s just you might have to go through hell before we get there.

I hope not.

So Jim, how about you? What have you got to say in closing?

A couple years ago, a friend who’s you know my age said, “Oh, kids coming out of college, they don’t know anything anymore; they’re lazy.” And I thought—I work at Tesla. I was working at Tesla at the time, and we hired kids out of college, and they couldn’t wait to make things. They were like, “It’s a hands-on place; it’s a great place.”

And I’ve told people, like, if you’re not at a place where you’re doing stuff, it’s growing, it’s making things—you need to go somewhere else. Like—and also, I think you’re right—the mindset of if people are feeling this is a productive, creative technology that’s really cool, they’re going to go build cool stuff.

And if they think it’s a shitty job, and they’re just tuning the algorithm so they can get more clicks, they’re going to make—they're going to make something beastly, you know, perhaps.

And the stories, you know, our cultural tradition is super useful, both cautionary and, you know, explanatory about something good.

Like—and I think it's up to us to go do something about this, and I know people are working really hard to make the internet a more open place that makes your information be distributed, to make sure AI isn’t a winner-take-all thing.

Like—and these are real things and people should be talking about them. And then they should be worrying, but the upside's really high.

And we’ve faced these kind of technological, like this is a big change. Like the AI is bigger than the internet.

Like I’ve said this publicly, like the internet was pretty big. And, you know, this is bigger.

It’s true, but the possibilities are amazing.

With some sense, we could achieve it.

Yeah, and the world is interesting.

Like I think it'll be a more interesting place.

Well, that's an extraordinarily cynically optimistic place to end.

I'd like to thank everybody who is watching and listening, and thank you, Jonathan, for participating in the conversation; it's much appreciated.

I'm going to talk to Jim Keller for another half an hour on the Daily Wire Plus platform. I use that extra half an hour to usually walk people through their biography. I’m very interested in how people develop successful careers and lives and how their destiny unfolded in front of them.

And so for all those of you who are watching and listening who might be interested in that, consider heading over to the Daily Wire Plus platform and partaking in that.

And otherwise, Jonathan, we'll see you