Zeroes and Ones: Into the Depths of Computation | Jim Keller | EP 272

I knew this wasn't going to work, so you're in a space. The direction I'm going is not going to work. Do you know how mosquitoes work? Mosquitoes are fun! So they detect two things: they detect water vapor and carbon dioxide. Mosquitoes will fly along in a direction as long as the water vapor and carbon dioxide are staying the same or going up. But as soon as it starts to go down, they change direction in a random direction, right? And within a couple of turns, they're aiming right for a mammal. They can fight. It's gotten colorful. So if you know you're going in the wrong direction, a change in direction can maybe be just as bad, but there's some chance it's good, especially if you're somewhat smart and you have some experience, right?

So there was a whole bunch. Even a random move is better than no move if the outcome is certain failure. And so that is some justification for taking a risk [Music].

Hello everyone! I'm pleased to welcome Jim Keller to my YouTube channel and podcast today. Jim is a microprocessor engineer known for his work at Digital Equipment, AMD, Apple, Tesla, and Intel. He was co-architect for what were among the earliest 64-bit microprocessors, the EV5 and EV6 digital alpha processors designed in the 90s. In the later 90s, he served as lead architect for the AMD K8 microarchitecture, including the original Athlon 64, and was involved in designing the Athlon K7 and Apple A4 through A7 processors. He was also the co-author of the specifications for the x86 64 instruction set and HyperTransport interconnect. From 2012 to 2015, he returned to AMD to work on the AMD K12 and Zen microarchitectures. At Tesla, he worked on automotive autopilot hardware and software, designing the Hardware 3 autopilot chip. He then served as senior VP of silicon engineering, heading a team of 10,000 people. At Intel, he is presently president and CTO at Tenstorrent, building AI computers. Uh, he's also my brother-in-law, and we've talked a lot over the last 20 years. He was a friend of mine before he married my sister, and we've known each other for a very long time since we both lived in Boston.

So I'm very happy to have you here to talk to you today, Jim. I'm really looking forward to it. Um, so thanks for agreeing to do this. Your thing. So let's walk through your career first. It takes some unpacking. Your resume takes some unpacking to be comprehensible, I would say. So let's start with Digital Equipment. You were working on very early stage sophisticated microprocessors. So tell me about that.

Well, it's the long story. I graduated college as an electrical engineer, you know, with a bachelor's, and I took a job in Florida because I wanted to live on the beach. And it turned out to be a really interesting job. I was at Harris, and I spent like two years working in labs, fixing up electrical equipment and doing some networking stuff and some digital design. At some point, a friend told me I should work at Digital. So I read about Digital, and I literally read the computer architecture manual for the VAX 7800 on the plane on the way to the interview. And then I interviewed them with a whole bunch of questions because I had just read this architecture spec, which I didn't know that much about, to be honest, but I was kind of a wiseass as a kid. And, um, they hired me because they thought it was funny. And uh, so I got my, you know, architectural education working on the VAX 8800, working for a guy named Bob Stewart along with some other really great architects. I spent about seven years in that group, you know, learning to be a processor architect. Then I spent a little time at Digital Research in California for six months and then went back and joined the Hudson team where they were building Alpha processors and became co-architect with Pete Bannon on EV5 and then Dirk Meyer.

And even Segue, now, those chips, from what I remember, were remarkably ahead of their time, but that didn't seem to save Digital Equipment Corporation, yeah? Is that fair to say?

That's definitely fair to say. Digital literally made the world's fastest computer in the years they were going out of business. And there was a complicated market dynamic. Digital was very successful, building minicomputers in the minicomputer age, which replaced the mainframes to a large extent. But they missed the boat on the PC revolution and would say workstations, and they had a big, expensive computer mindset right when computer prices were falling much. So we were building really fast processors for server and high-end workstations, and the market had kind of moved on. And let's say a lot of crazy things were going on inside the company at the time.

Yeah, well, we're going to return to the topic of crazy things going on inside computers. But it's interesting to note right there. So that, that's a situation where a company has a great product but doesn't know how to launch it into the marketplace or is blinded by its own preconceptions. You can't even see necessarily more than that. Gordon Bell was CTO, and he was a really brilliant computer architect, but he also had really good, let's say, observational skills. And like during midway through the VAX 8800, he decided the technology we were using was too little too late and redirected the program. And it became a very successful product because he knew what was going on, and he made decisions like that.

When he left, I think Digital became an argument between business unit managers, not about technology. And Alpha was a great technology, but it went into business units that were aiming at high prices and high margins, not market penetration and not basically keeping up with the software revolution that was happening. So it was, uh, Ken Olsen was a great manager, but he wasn't a technical leader. And without Gordon Bell, the company kind of lost its way. And when companies lose their way, they fail, and they fail fast too. You know, they went from record profits to losing a billion a year, and they kind of rectified that for a little while, and then sales dropped off again, and over.

Yeah, well, one of the things I've been struck by watching your career and talking to you over the years is exactly that. The rate at which a company that appears dominant can disintegrate and disappear is really quite stunning. I think Fortune 500 companies tend to last no more than 30 years. That's approximately the span, and that's not a tremendously long period of time. So there's always dominant companies and always a handful of dominant companies, but the company that's dominant tends to shift quite quickly.

So we should return to this a couple of times is the, you know, the classic S curve in economics. You start out low, you solve a problem, you ramp up, you plateau, and then you fail. And this dominates business and dominates humanity at some level, and it plays out over and over. Now it sounds like you didn't have, and so how is it that you managed to do this job? You intimated when we were talking that you weren't really trained for it. And so you were trained as an engineer, you had a, is it a bachelor's degree in engineering? And how prepared were you as a consequence of your degree for any of the jobs that you undertook?

Training is highly overrated. So a good engineering degree is math, physics, basic understanding of the science, and some smattering of communication skills. You can probably do a great engineering degree in two or three years if you're dedicated to it. You know, the things that stretched my brains the most when I went to college was with math and mechanical engineering. I went to Penn State, and they used, um, mechanical engineering and masses through the weed-out courses to find out if you had the chops or the gumption to get through engineering. So there was a fairly high failure rate there. But mechanical engineering is a really interesting discipline because you have to think about solving mass problems spatially, like, you know, do things like how do you calculate the force on a rotating accelerating object? Like it's somewhat complicated, and it makes you really think.

So you have to learn to think. In engineering school, you never answer a multiple choice question; you learn stuff and thermos and stuff. But then you calculate and understand what the result is, and the problem sets are like little design exercises. And so how much of it do you think is pure screening? Let's say what wouldn't be pure screening but fundamental screening for conscientiousness and IQ, and how much of it is learning to think? How much of the education process is that?

Like if you're going to hire an engineer, are you hiring fundamentally on the basis of IQ (and you get smarter people from the top schools), or do you think that the engineering training actually does prepare people for a technical career?

Um, well, it depends on the engineer and depends on the school and depends on their approach. My IQ isn't super high compared to really smart people. I mean it's high enough. When I went to college, it took me about a year and a half to learn how to think properly, and I found for me personally I had to do the work on a regular basis early. I didn't study for finals; I wasn't the kind of person that could pick up a book, understand and get an A the next day, and then forget about it. I'm not. I can't do that. So I had to learn how to do the work, go through the mechanisms, automatize some of the basics so I didn't have to think about them so hard. But literally let my brain work on this stuff so then I could use them to go problem solved.

And especially in engineering, there's lots of different kinds of engineering. There's like highly technical stuff where you turn the crank like a skilled lawyer might, but there's other stuff where you have to be really creative. If you have a problem that nobody solved before, as an engineer, you have a skill set, right? But you have to apply it creatively. There's lots of high-IQ people who aren't creative, and there's low-IQ people that are creative.

And you find in a big engineering team there's a real diversity of personality types. There's open-minded people, conscientious people, you know, gregarious people, and it takes many different kinds of people working together to do something sophisticated. I'd say, you know, like some of my senior classes in engineering were just going a little deeper on stuff I already knew. I could have left it after three years and been just fine. But I do think the work I did helped me, you know, be an engineer. But then the problems I saw, you know, I worked on after I graduated college, like in school, most of the problems you're given there's a known answer because they're in a book, you know, and you're in a room with 20 people, and they're doing the same stuff.

Um, when you're an engineer working in a company, they never have two people doing the same thing to do because that's a waste of money, right? And when you start engineering, you're given relatively small tasks by, you know, your manager or supervisor. But as you go along, at some point, it depends again on who you are. You're working on stuff that you don't even know how to deal with. You know, there's no answer in the book.

So it's but it's not like physics, right? Like physicists are a funny bunch. I realized this the other day, that physicists, they're supposed to work on stuff that's unsolved, whereas engineers, you know, there's a big repertoire of engineering and then it's reduction in practice and then the world is complicated. So you know, go build a new bridge that's never been built before. It's not like bridges are unsolved problems. This particular bridge hasn't been solved before. There may be unique challenges to it, but it's not like physics where you're looking for an unknown particle or, you know, it's you know, there's a pretty big dividing line between engineering and pure science.

Engineers typically work in the names where there's many, many knowns and the unknowns are problems of the combination of, you know, reality, complexity, whereas physics is physics in principle. They're working on stuff that's fundamentally unknown. As soon as it's known, they have to move on because then it's engineering.

Like the physicists, you know, translate the unknown into engineering, and engineering applies known concepts to unknown problems.

Okay, so you, okay. So now you went from Digital to AMD and you learned how to design microprocessors. So at AMD, you worked on the K8, yeah? And at that point, AMD was losing ground to Intel, yes? And so how did you fix that?

So basically, Dirk Meyer and I were co-architects of EV6, the third Alpha chip. He left about a year before I did. He started the K7 project when I joined. I started the K8 project and then held and then worked with him significantly on K7 as well. Um, and how did we do it? Well, yeah, so those were 64-bit chips that you guys designed to compete with the, the fun, the Intel chips that have dominated the comp, the home computer market at that point.

Well, so there's a funny thing, which is like at some level, building fast computers isn't that far off, right? So you have to have a goal. Like so a lot of designers, they have a design and then the easiest thing for the next one is to go look at that design and make it like 10% better, 20% better, right? But every one of those designs has limitations built into it. Like it's sort of like if you have, if you buy a two-bedroom house, you can add one bedroom. You can't add eight bedrooms. Like if you want an eight-bedroom house, you have to build a different kind of house, right? So every design build has a kind of a, you know, a range that it can play. And, you know, you build the first one and, you know, you can make some improvements, but at some point, the improvements don't really help that much, right?

So AMD, they had a design called K5, which for complicated reasons didn't work out that well, and they lost ground to Intel. Before that, they literally, the 386 and 46, AMD copied Intel's designs; they were a clone manufacturer. The K5 was their first design and didn't work out that good, and then they bought a company called NextGen, which had K6, which is an okay design, but it wasn't competitive against Intel. And then K7, Dirk was the chief architect of, and he designed a computer that was competitive in the head of Intel. From that, some of that came from our work at Digital on EV5 and 86 turquoise as well.

And some of it was just saying in this day, you know, we have this many transistors because you get more transistors every generation. So you can basically imagine you're building a house. Suddenly you have way more bricks and way bigger steel beams. So your idea about what the build has to scale with that. And then K7 was a 32-bit chip, and then K8 was a 64-bit chip, you know, somewhat related to that, as it turned out. Um, but also, it was built to be bigger, and what I did is I wrote the performance model. I came up with the basic architecture and I started organizing around building it.

And while we were doing that, we also wrote the thing called HyperTransport spec, which became the basis of essentially all modern server computers, or what's called two-socket servers. We wrote that in ’98, and in 2022, they're still building them that way.

And when you say you wrote it, what does that mean? What does the process of writing that entail? What is it that you're writing and how do you do that?

I'm dyslexic, so I wrote complete, you know, protocol spec about how two computer chips talk to each other in 18 pages, right? Which is relatively terse. And there's a couple of pictures. Computer protocols are pretty straightforward. There's a command, there's the address you're talking to, there's the data you're moving, there's some protocol bits that tell you how to exchange commands, right? And then Dirk took the spec and said, you mind if I flesh this out a little bit? And three days later, he sent me a 50-page version of it, which clarified all the little bullets. And then that specification we literally used to build the interface between K8s, right?

So there's a couple levels designed. What sort of impact did that have? What sort of impact did that have on the broader world? What's the significance?

It's very difficult for non-engineers to understand any of this, but AMD's market share in servers went from zero to 35%, which was a huge impact for the business, and it became essentially the standard because apparently Intel had a version of that, but it didn't go to market. But after Opteron came out to market, Intel built a similar version, some more protocol about how to connect a small number of processors together with that kind of interconnect. And then that, let's say, design framework became standard in the industry.

So if you go into like a Google data center and you pull it out, there'll be two sockets with an interconnect between the two of them and each socket will have memory attached to it, and they call it the two P server or two-processor server. And it had a really big impact. We didn't do it because we thought it was going to have a big impact. We did it because we thought it was a better way to build computers.

And at AMD, we were somewhat resource constrained, so we couldn't build a thing that looked like a big IBM server. So what we built was basically a small server with the minimal amount of interconnect between it. So it was a little bit of creativity by constraint. The Steve Jobs line.

And what function do those servers have in, again, in the broader world? What are they doing now for people?

Well, it's basically the entire cloud. It's all Google, Amazon, or Facebook and all Microsoft servers. But here's the interesting thing. When we built them, the big server guys – servers used to be backlights like this big with multiple CPU slots, multiple memory cards, multiple I/O slots, and the server manufacturers thought the server was oriented around the back. So IBM, HP, Dell, they all turned this down. But all the little startups at the time like Google were using PCs as low-cost servers.

And we made this, basically, you take a PC board instead of putting one computer on, you could put two, which radically saved the money. So when AMD made those kinds of servers, it was a way lower entry point for server-class technology, and the little startups that deployed used it, and then over 15 years, disrupted all the big server manufacturers.

So it's, you know, it's one of those, I couldn't say we planned it. Like there, the constraints that we had a target market, we didn't know that it was going to become essentially how servers were built for 20-odd years.

But it happens. After AMD, you went to Apple. You worked on the A4 through A7 processors.

Well, I worked at two startups that did processors for networking Sidebite and TA Semi, and that was probably about five or six years. And then I joined Apple in 2008, so I guess I was, no, it must have been eight years. I was AMD ’98, ’99, 2000, and then I worked at startups for about eight years, and then I went to Apple.

Um, yeah, and worked on mobile processors. And so tell me about those chips and what you did at Apple.

I had some friends that were working in Apple, and they wouldn't tell me what I was going to work on. So when I interviewed there, they said, oh, you should just come here. It'll be fun. And I didn't actually know what I was going to work in. They had a group called platform architecture run by a guy named Mike Colbert, who was like the unofficial CPO of Apple. He worked for Steve Jobs, and he had a group of architects that, you know, looked at what Apple was doing and figured out what they should do next.

And I worked on a MacBook Air definition, like I wrote the power management spec and just did some other architectural work, which ultimately was an NVIDIA chip called MCP89. And then I was one of the chief architects of, you know, four generations of SoCs, what's called A4, A5, A6, A7. And we did a lot of stuff there because, but the division was, you know, SoCs are, what?

Yeah, system on a chip.

Oh, yes. You pack a computer into a phone; you have a piece of silicon about that big, and all the components, the CPU, the GPUs, I/O, are all on the same chip. And when they first started building phone chips, they were considered to be very slow, low-cost, you know, very integrated chips. And we thought if you looked ahead, because technology shrinks about every two years, in about six or eight years, we'd have enough transistors on a phone chip that would be more powerful than the PC at the time.

So we started architecting computers interconnects and other functions so that when we had enough transistors, we could literally have, you know, a high-end desktop on a phone. And Apple's DNA is, you create the product that kills your current product. So you create the product that killed. So every company has a great product, and they worry about competitors coming in to kill it, and Apple wanted to be the first to kill their own products.

So Steve Jobs thought phones and tablets would replace PCs, and he wanted to be the first to do it. He didn't want somebody else to do it to him. And did you know Jobs?

No, I've seen him a couple times. I said hi to him twice. I felt like I knew him pretty well. Everybody at Apple did, like when Steve wanted something done, everybody knew the next day.

Um, my boss, Mike, talked to him every single day multiple times. Sometimes it’s a joke; like we’d walk in Mike’s office, and he’d be holding the phone out like this. He goes, “Steve, he’s pissed.” We’re like, “Yes!” So what? But Mike could translate what Steve wanted into engineering stuff, and he trusted Mike a lot.

And like he could translate division into engineering, and Steve's judgment on stuff like this is spectacular. And did you have any sense, do you have any sense of why that is? I mean Jobs was famously, obviously originated Apple and then was famously brought back in to save them when they were in danger of extinction and then in fact did seem to save them.

And you never know when you hear about these things from the outside how much of that is sort of a mythology, mythologization of a person and how much of it is, you know, this person was really singular and how unique?

Singular and unique. I mean, your psychological parlance, as we talked about, he would be considered high in openness and disagreeableness, right? And I think negative emotionality. Like he was a very difficult person. But the solution to, you know, things could go really bad and being disagreeable was “I’m going to make it as great as possible.” And he was willing to take the risk for that.

You know, his public persona was very well practiced. Mike used to say the worse the practice for the, you know, Apple keynotes, uh, the better they would go off. Like, like he was throwing iPhones, you know, at one of the iPhone, you know, pre-launch practices because nothing was right. But then when he showed up and, you know, his persona of, uh, you know, technical explainer, let’s say, you know, that was very real. That’s what he wanted to portray, and he believed every single bit of it. You could tell.

So, you know, any engineers, when I joined Apple, I watched some of his early keynotes when he came back to Apple and changed the Macs. You know, it’s inspiring, but you know, it’s also super tough, right? Because he went into a company that was very dysfunctional, had a whole bunch of, you know, engineering groups doing basically random stuff, let's say senior managers who felt like they owned their product lines and knew what they were doing, and you know, Steve wanted him to do what he wanted them to do, and they didn't want to do it.

I'm pretty sure he cleaned house early. And he famously reduced the product line from, you know, who knows how many products, like four, you know, it was consumer professional.

So do you think it was that disagreeableness? I mean, we hear all the time now in the modern world about the necessity for empathy and so forth and, um, that's the agreeableness dimension, and you're making the claim that Jobs was low in agreeableness and that he was able to kill off malfunctioning projects, and that's not exactly a nice thing to do.

And so imagine you go to a room full of people dedicated their life in the last five years of their work, five years of their careers, building products that you can’t sell, and you say, we have to do something completely different, and everybody's, you know, every day as an engineer, you're working something you embrace it, you love it, care about it like engineers are very emotional people somewhere in their pointy little souls.

Right? So, but, so, but if it’s not working out for whatever reason, you have to do something different, and if you listen to everybody, you'll never change anything.

Right. It's difficult to get people reoriented. Now, another line you gave me, I don't know where it came from, is you run fastest when you're running towards something and away from something.

Yeah, that was from animal experimental literature. If you threaten a rat and re and offered a reward simultaneously, it will run faster towards the reward than if you just reward it because you get all your motivational systems on board that way.

Yeah, yeah. So Steve was very good at division. We are going to build this beautiful computer, right? And you better goddamn build it now or you're going to die. So that was his.

Okay, so the openness, so that’s the creativity dimension that gives him the vision. He's extroverted. Can he communicate enthusiastically?

He can certainly put on the actor. I have no idea who's an expert or not. I never saw them be extroverted in any natural setting. I've seen him walk around, like I said, I used to see him in the cafeteria; he was visible on the Apple campus even until his last days.

He didn't like to be bothered, like he didn’t go up to say to Steve and say, “Hey, Steve, how's it going?” right?

Well, that would be reflective of basic disagreeableness too, right? You know, it’s hard with Steve. Sometimes people can communicate very effectively communicate a vision because they are high in openness. Extroverted people are enthusiastic and assertive, so they tend to be verbally dominant and can inspire people because they generate a lot of positive emotion, but that can be mimicked by openness.

So yeah, so you have to remember, like Steve was part of Pixar and very much part of Hollywood and, you know, creating movies and creating personas and characters and archetypes, though he was super well grounded in how that stuff works and what works and doesn't work about it, right? And he had an unerring eye for beauty and elegance and remarked about it, and he would fight for that.

And that's hard. It's very hard to fight for beauty and elegance, and I suspect it’s particularly hard, perhaps—maybe I'm wrong about this—but I would think that would be a hard sell to, at least, a subset of engineers.

Yeah, so another addition to explain to my boss at Tesla was I worked both for Elon and for a guy named Doug Field. Doug said there's this productivity graph versus order. So the origin is zero productivity and chaos, right? And then as you add order to your design methods, your productivity will go up. And what happens with engineers is they understand as they get better processes, they get better trained, they get better at working together, every single thing that makes the whole organization more orderly improves productivity.

Unfortunately, that peaks at some point, and then too much order, productivity goes down. And so then, the, you know, as I say, any idiot can see you should be at the peak: note enough order to really be effective, but not so much ordering grind to a halt.

But why can’t you stay there? And the reason is once order takes over the organization, it’s unstoppable, right? It feels good. You get even better at doing what you’re doing. You get even more organized, you micromanage your time even better, you close out all the creativity, you're not open to change. A whole bunch of bad things happen; you shut out the disorderly people who actually know how to make a change and do something creative, and the organization dies.

So successful—I think that Jobs was conscientious as well, you know? Like, was he in early working 18-hour days? I know he was up in the middle of the night because he called Michael on that stuff. Well, the point is, both Steve and Elon were counterforces to order, right? You have to be really strong to avoid your organization getting captured by order.

Well, order also has its remarkable error of moral virtue, right? Because it’s pure and it's efficient; it feels good, you know? But it's, you know, it’s like, it’s like alcohol. The first drink feels right, the second feels okay, the third one, not that good, but you know, you keep remembering what the first one did, so you drink too much, right? It’s a—there’s lots and lots of processes, you know, where some is good. Too much is bad, but the counterforce, the more is weak.

And that’s the thing that, you know, so Steve was interesting because he was simultaneously super creative and had visions which could inspire people, but he also prevented the company from being over-organized and preventing him from doing what he was doing. And that's hard. Because people, like I said, they get committed to what they're doing.

Yeah, well, it's an open question. Like imagine that the creative process has a productive component and then a culling component, and the productive component looks like it's associated with openness, but what the culling component is is open to question. It does seem to me that at least upon occasion, it's low agreeableness. It's the ability to say, "No, we're going to dispense with that,” and to not let anything stand in the face of that decision, which would also include often human compassion.

Yeah, and people have different approaches to it. Like Jobs would call things so they weren't beautiful or they weren’t great. You know, Elon Musk was famous for getting the first principles and really understanding it fundamentally and culling from like a standpoint of knowledge.

Um, yeah, and you’ve asked me like what makes an engineer great? Like, so you have to have the will to creativity. Like, now there are lots of engineering jobs that aren't creative, like you need a skill set; you can exercise the skill set. But if you're going to build new things, you need to be creative, but you also have to have a filter good enough to figure out what's actually good and bad.

Like I know a lot of really creative engineers, and they find a new thing exciting and go down the rabbit hole on it, and they can work on it for six months and have nothing to show for it. So you have to have that conscientiousness.

I don't know if it's conscientious disagreeableness, you know, that taste on how well the conscientiousness would keep you working in the direction that you've chosen and doing that diligently and orderly—the low agreeableness? Well, that's the open question because agreeableness is such a complicated dimension. There are obvious disadvantages and advantages at every point on the distribution. I mean, disagreeable people are often harder to work with because they don't care much about your feelings.

But one thing I've noted about working with disagreeable people is you always know what they're thinking. And if you want someone to tell you what's stupid and wrong, they're perfectly willing to do that.

Yeah, you saw, I used to watch—I used to wonder, so Dirk Meyer was a disagreeable manager, but he could tell you what was wrong with what you were doing in the way you would go, okay. Like, he was very unemotional about it, like he goes, "Jim, I really like this and this, but this isn't working for like what are we going to do about it?" And you'd just be like, "All right."

As a matter of fact, um, I would say when I was younger, I was a lot less disagreeable. You know, I'm fairly open-minded, and I like to create new stuff, kind of stuff, things. But then I saw enough things fail over the years because we didn't make that, you know, let's say the hard choices about something.

And then, you know, you hate to work on something for two years, and have it go away because at some point you realize you're doing a couple wrong things, and you didn't do something about it when you could. And so as a manager and a senior leader, I'm somewhat famously disagreeable, and part of it's an act to get people to move, and part of it’s my beliefs that I can't have people dedicate themselves to doing bad things for very long because it'll, it'll bite us.

Yeah, well we've talked a little bit about this too, about the moral dilemma between agreeableness and conscientiousness. They're both virtues. Agreeableness seems to me to govern short-term intimate relationships, like that between a mother and a child, and it involves very careful attention to the emotional reactions of another person and, and, and the optimization of those in the short term.

But conscientiousness looks like a longer-term virtue, and they come into conflict at some points because—sometimes because in the midterm? Yes, right? Yeah, that's, you know, could just be, you know, how our brains see the future, but it's like, you know, if you're managing the group and you have to fire somebody, it's hard, right? But do you want to fire five people now or everybody later?

Like, once you’ve internalized that and taken responsibility for that decision, then making, you know, management leadership position choices is always hard. But it's so much better to make them and then succeed than it is to fail because you couldn't make the hard calls.

Yeah, well, it isn't obvious at all who's got the upper hand. You know, someone who fires early out of necessity, but is accurate in looking carefully, or someone who, you know, is willing to let people just drag on...

I'll give you two counterexamples of that. So Jack Walsh, in his book Straight from the Gut, the weird thing he said, you know, once you have a doubt on somebody, you never act fast enough, which, you know, it took me years to really believe that. And then the other weird one is people say, I have this organization of 100 people, and there's five, five people aren't working out, but I'm not sure who they are. So I'm going to be really careful because I don't want to accidentally fire a good person.

Right, that makes sense, right? You got five bad people, you know, maybe you figure out who two or three of them are but there's this other group of five or ten you're not sure which ones are the wrong ones.

Here's the sad truth: there are a lot of people in the world you're better off firing too many than too few.

And how did you come to terms with that emotionally? I mean, look, we have a mutual friend who fires people with quite great regularity, and I’ve talked to him, and he scores very high in disagreeableness. And I talked to him about firing, which he's done a lot of, and he was actually quite positive about it.

He said, I don't fire anyone who I don't think is causing more trouble than preventing. And so by firing the person that I'm firing, I'm actually doing a very large number of people, including potentially that person, a favor. It didn't bother him, but he was temperamentally wired that way.

I would say, you know, Digital Equipment went bankrupt because they had bad people who didn't fire. I've seen many groups fail because they couldn't clean the house.

Right, and the impact on, you know, the greater good equation is super easy. You want to save 90 people or lose a hundred? So that's true. The thing that took me a while to realize was that the world needs shaking up all over the place and individuals do, right? A lot of people who are not doing too good, they need a wake-up call. You give them a bad review, and they kind of shrug, they're like, what are you gonna do about it? You know, it's like a spoiled kid, nothing, right?

But when they actually get fired, they really have to do some soul searching. And then the fact that if you’re doing something good, there's always a queue outside the door; more people.

Now here’s another way to think about it: take a group of a hundred people and rank them from top to bottom. Human beings, by the way, are really good at this; you have four managers in the group, except for the manager’s individual friends. They'll tend to rank a hundred people the same way. I’ve done this experiment many times. But we’re really good at ranking, and there's a little bit of, what are you ranking for? You're ranking for creativity, productivity, conscientiousness, but if you set the criteria right, people rank pretty well.

If you have a hundred people in your group and there’s 50 people outside the distribution of those 50 people is around the average of the team, right? So there’s this idea that you fire the bottom 10% of its team because the random people you hired will be better on average than the bottom 10% of your team, right? It’s just math, unfriendly statistics, right?

But yes, I get the argument, right? The problem is that every company that does that first gets gamed because managers hire bottom 10%ers, so when they get the fire, the 10 they don't have to fire their friends. Right? And it also really is hard on morale, like people bond, and there may be people in the bottom 10 of your group that are the social glue of the organization, so you may be inadvertently taking out the stuff that makes the team work, right?

Well, that’s a measurement error too, right? It means that your criteria for competence aren't broad enough.

Yeah, that's tough. Good point. Maybe you’re ranking a little wrong, but, uh, impact on morale is high. Teams, generally speaking, uh, Rory Reed was CEO of AMD when I joined, and we had a big layoff, which we had to do because we were running out of money. We were broke. And when all the dust settled, we landed on just the right amount of people for the money we had.

And he basically registered, actually, said, "Guys, teams have to grow. When you cut, you always cut further. And then you grow. People aren't happy unless they're growing." Right? It’s like when you prune bushes and stuff, you don’t prune the bush to where you want the bush, you prune the bush past that point so it grows out and looks nice, right? Things have to grow.

It's really an amazing dynamic.

Well, yeah, well, and it's never clear how much death there is involved in growth, and the pruning analogy is exactly that. And this is harsh stuff, obviously, but yeah, you look like one collapse or another.

It’s right; that's the thing. It’s not harsh because it’s beautiful when you prune your bush and growth back beautifully, it’s great when you rebuild an organization that's really strong and powerful because you made the right calls. Right? Like, this isn’t just negative stuff. It’s hard stuff to do that creates something really great.

Like, you know, when I joined AMD, and what was it? 2013 or something, like they had two product lines, you know, bulldozer and jaguar, and they were both failing. And I had to cancel both products. Like, okay, and so what was the human cost of that? I would say both to you and also to the people that were involved.

Well, I did the math on it. It's like, you know, we needed to be building eight-billion-bedroom houses, and we're trying to add six bedrooms to a two-bedroom house. In one case, it was never going to work.

Yeah, so you saw that as doomed to failure.

And the other one was structurally screwed up, it seemed to be the right ballpark for the performance we should get, but the way it was engineered and built was sort of like, you know, you let the plumber do the architecting, and the house looks like [exhales] and it was difficult, you know, for complicated reasons.

Technical reasons there was no path out of where they were, and when I realized I had to cancel them, um, yeah, it was sleepless nights. Here we are; we had revenue on that, we had people committed to it, people really liked it. When I canceled it, um, especially on the Jaguar team, a significant number of people quit because they were angry about it.

Um, there were some pretty big new organizations. There was some management we had to let go—the best architect at the time. Well, one of the best architects at AMD was he really was my way or the highway, and he could not communicate what he was doing, so I let him go.

Um, which is a strange thing to do. So if he ranked the organization, you were ranked near the top, and I let one of those guys go because he was ineffective working with the team.

And how did you justify that to yourself? And how did you check yourself against stupidity and ignorance and, you know, self-interest and how did you know that what you were doing was right?

Well, I’m a little lower in conscientiousness than I should be for a senior leader, first of all, so I knew this wasn't going to work. So, so you're in a space—the direction I'm going is not going to work.

Do you know how mosquitoes work? Mosquitoes are fun! So they detect two things: they detect water vapor and carbon dioxide. So mosquitoes will fly along in a direction as long as the water vapor and carbon dioxide are staying the same or going up. But as soon as it starts to go down, they change direction in a random direction, right?

And within a couple of turns, they're aiming right for a mammal. They can fight. It's gotten colorful. So if you know you're going in the wrong direction, a change in direction can maybe be just as bad, but there's some chance it's good, especially if you're somewhat smart and you have some experience, right?

So there was a whole bunch. So even a random move is better than no move if the outcome is certain failure. And so that is some justification for taking a risk.

Now, there's an incident number of failed directions, but you're somewhat informed, right? And then the other problem is, like, when you build a house, it has a foundation. Once the foundation is built, it's very difficult to change the top of the house. A lot like if you have a foundation for a two-story house, it’s hard to make it into an eight-story house.

So when we cancel those projects, we consciously reset some design methodologies, you know, some team organizations and leadership—lets say, you know, we said we're going to have the best in-class leadership design methodology, and some of the architectural tools we're just going to—we're going to take those as givens.

Now that the land has been cleared, we had the opportunity to go back and do that. And it was interesting, and in the design teams, it turns out there were some very good pieces, you know, in the two processors they had, but they weren't working together organically like they should. And then, say, the framework of the design wasn't big enough. And then the tools over the years have evolved into lots of little local improvements, but it wasn't really the right tool set.

Now AMD leapt forward when you did this, and they were the only competitor to Intel in a realistic sense, and so these actions on your part were part of what made that company thrive and kept competition within the microprocessor world, so these didn't have trivial outcomes.

Oh no, it had really great outcomes. And there was the really cool thing was, you know, when we did that, we didn't really bring in outsiders. Like that Zen design was entirely based on people who worked at AMD at the time, right? So what we needed was to clear the plate a little bit to re-establish some, you know, first principles about how we were doing things to have a better goal.

Um, there was a little bit of head-knocking on getting the methodologies straightened out, you know, I was, uh, let's say fairly disagreeable about how we were going to get through that because people kept saying, "Oh, it's too hard to do this."

Well, is it any good? No? Well, if it's no good, it doesn't matter how hard it is. You have to do it, right? If you're going to drown, you don't go, "Well, a mile is too far to swim, so I'm just not going to swim." You're going to drown, right? If you're a mile offshore and you're drowning, swimming a mile is the requirement.

Right? And I've explained that like a million different ways. When something is pretty good, you know, the—the world is—you don't divide it into three things; some things are good, so you're happy, and some things are bad, you fix it, and then there's the middle ground where it's, it's not great but it's not killing.

Those are the ones where human beings have a really hard time improving, right? So by canceling the projects and declaring everything bad, everything could be improved.

All right, what was it that bugged me about it? It was like, "Jim, this wasn't that bad." Well, was it great? Was it going to win? No? All right, it's bad. I defined everything that’s not great back. If I’ve moved, you know, a little dis on the continuum, everything five percent or more away from great is bad, period.

And how did you, like how did you decide that was a good criteria?

Well, we were competing with Intel on a whole bunch of metrics. Like, like literally, their CPU had twice the frequency, twice the performance per clock, a whole bunch of metrics were so good. So we plotted them all like, you know, I had a whole bunch of data on this stuff; but also as a mindset like if it wasn’t best in class—like a computer has a whole bunch of things in it.

There’s something called a branch predictor, which predicts which way branches are going. Theirs was way better than ours, we could measure. So we did; there's a thing called, you know, memory system. Their memory system was way better, it was twice as fast. So it’s like, well we need to be within 10% of them on everything or we’re going to get our assets kicked and it’ll be really nice if we’re better on some things.

So we measured all their stuff. Like if you're going to compete, you know, like in basketball, you don't just play yourself; you know, you try to see what the other teams are doing. What are they good at? What are they bad at? You know, every coach is great at doing analysis of all the competition, and then, then you know you win two ways: you meet the competition with what they’re good at and then you have some secret plays that you're better off than surprising.

Given recent SCOTUS wins, it feels like the pendulum may be swinging back to a time when the nuclear family was situated at the center of American life, where real conversation, learning, and growth began at home. President Ronald Reagan said in his farewell address that all great change in America begins around the dinner table.

Well, all great meals in America begin with Good Ranchers. Good Ranchers cares deeply about providing families with steakhouse quality beef, chicken, and seafood meat at a reasonable price. Their mission is to bring people to the table, making those shared moments with your loved ones easy, accessible, and delicious.

Good Ranchers ships 100% American meat, born, raised, and harvested in the U.S. right to your door. Plus, when you subscribe, your price is locked in for the life of your subscription. Great food creates great conversation, and great conversation makes great change.

So start bringing people back to the table with high-quality American meat. Go to GoodRanchers.com/Peterson to get $30 off your first order plus free shipping. That’s GoodRanchers.com/Peterson.

So you went from AMD to Tesla. So, and we talked a little bit about Elon Musk. I want to ask you about him and your experiences at Tesla, um, but also, and what you did there.

Sure, so let's talk about Elon Musk to begin with. Sure. Well, he's a pretty public guy, so I don't know that I can add that much.

Well, kind of what you said about Steve Jobs: was it true?

Yeah, it's mostly true. Like, like Elon's the real deal. He's like, he's a really good engineer. He has his belief that he can learn deeply about anything fast, and he's practicing. He does it. I've seen him do it. Like, he gets into lots of details.

Now, he has done five impossible things.

Yeah, yeah, it’s pretty respectable, and he likes the details, and he has a good eye for it.

Um, usually before Tesla, when you do a technical presentation, usually dumb, you have some kind of methodology for presenting the problem to an executive. What’s the problem? What’s, you know, the background of that? Elon is a solution person. Like, if I don't like your solution, I don't care about the stupid problem and your data; you know, just forget it.

No, what's the solution? There's the solution. Great. Great. Then everything else is backup. You know, what's the data behind that thing? Oh, we've got the data. What was the problem? Like, how did you figure this out? Oh, here’s the original problem we found.

Like, he was like, what the—how can you have a solution?

Well, yeah, we do. We have the software that does this.

Well, page 12? Why is it on page 12? Page one would have been a really good place. Here’s the old image, here’s the new image, here’s how we did it! Right?

That’s what he wanted! Now, partly it's high back.

Did that—would I—so what I'm wondering is what was effective about that? Was it that there was an ethos, then, that the most important thing that you had to bring forward wasn't a problem, but it was a solution? And so that people were striving constantly to generate and communicate solutions, which seems like a good strategy?

Well, no, I think he's worried. So engineer sees a problem and they start investigating it, and then as you're investigating, you get to develop understanding and you're understanding the problem as a good—like, you would think that, right?

You'd think that.

Lots of people think that. Elon doesn’t think that. Elon wants a solution, right? And if you're falling in love with your understanding and you're falling in love with your little details, well, that also feels like work. You know, you mentioned earlier that just because it's hard doesn’t mean it’s useful, right? And both focus, you are focused like crazy on the solution.

The best way to do that, so then you have two states, right? If there's a problem, there's two states: you have a solution, and you don't have a solution. If you have a solution, that’s on page one; we're all happy.

Yeah, well, I’ve seen in the software projects and other projects that I’ve been involved in too that focusing on a solution, I think this is along the same lines as what you’re discussing is, well, then you get a— you get to product a lot faster. It’s like, this thing has to exist and work. Maybe it won’t solve—the problem with the problem is that you can indefinitely investigate the problem and expand it.

And also that that feels like work, but it's not saleable.

Yeah, yeah. It puts in the forefront of your mind, you know, that constant, you know, you need to be creative pursuing the problem but also make sure you're really on track of the solution, and you’re just not falling in love with the problem. People call it “admiring the problem.”

Some engineers are great at admiring problems. Like I’ve worked with lots of people who come in with a 12-page presentation, and when I’m done, I’m like, did you guys just give me a whole bunch more data on the problem?

You said, yeah, we’re really getting to the bottom of it. It’s like, no, you’re not. You’re not getting anywhere.

Well, the bottom of a problem is a solution because why would you just investigate the problem, right?

I mean, that's your destination, no?

No, they just keep getting deeper and deeper into problem admiration, and nothing happens. Okay, three-quarters of engineers would be perfectly happy to do that their whole life because some people find they're really good at it. They’re a problem—they're analysts. They generate lots of data, but if you’re in charge of solving problems, you know, that period needs to be focused short and concise, and you need to move on to solutions.

Right? I think that's probably why I'm not so temperamentally fond of activists.

Are they problem admirers?

Well, that's what it looks like to me. It's like, well, this is one of the reasons I admire Elon Musk. It's like, well, you're comparing, you're concerned about the environment. Well, why don't you build an electric car then? Right?

Well, so a large number of people—this is my favorite thing I learned working with mechanical engineers at Tesla—they think the world's made out of silly putty, right? They used to design. When we were building Model 3, they designed the part and joked about how they’re going to make it. Are they going to, you know, CNC it, like mill it, or they can injection mold it, 3D print it, stamp it, make it with a hammer, you know, cut it out scissors, you know, carve it out of a block made.

This cool machine that could carve 3D models out of clay, like, it was funny. Like, so they could design things in their heads on computers and then go build any damn thing they want. If you ever look at them, complicated mechanical assembly would be some screwed aluminum thing, and it would be built somewhere and then drilled, and there’s screws going through it, maybe some little tabs sticking off of it that holds another thing, like they can make stuff, right? They think they can make anything, right?

And there’s a whole bunch of people in the world that don’t think they can make him. They think the world is what it is. I had a friend; he had a rattle on his dashboard, and he didn’t know what to do about it. And I was asking him where the rattle was, and you know, and I was thinking that I was talking to him about, like, how the dashboard is made, and he goes, "Oh, I get it. You think the dashboard's made of a whole bunch of parts that are put together somewhere."

I thought the dashboard was just the dashboard. Like he could, he didn’t conceptualize it as there’s his outer piece and there’s inner brackets and there’s radio, and there’s just these things because mentally I can’t help but see the whole thing in 3D and then I’m wondering which which piece is loose and where it is, right?

And then how to fix it? Like, like I’m not mechanical engineering creative, but I’m visual like that. Like, and for people who get stuck on activism as problem descriptions, they don’t think the world can change, which at some level makes sense, you know, for human evolution. Like, you know, it was pretty much the same for a million years. Like, it’s weird how good we are at change.

Um, and my best theory on it is from zero to 20. Like, your brain is going through radical change because you're going from, you know, silly putty and not knowing much to being pretty smart. So you have to change and adapt really fast. And then humans are adapted to deal with each other, and the humans are fairly crafty, you know, you have to deal with that. But, but, you know, the lifestyle of most people from, you know, you know, say 30 deaths, it’s fairly static.

And, you know, so we have this funny capacity for learning rapidly, exponentially, and then dealing with slowly changing environments. But we’re not naturally adopted the rapids in the modern world is, well, especially in engineering, is rapid change.

And, um, it was just so funny, like you never knew what they were up to. Like one day, like they were working on the interior for the car and they made this crazy-looking model which kind of looked like a car, but it turned out it was a thing you could move around and have the attachment points for all the interior parts.

So you could basically, it looked like a weird skeleton, but it had the attachment points that you could adjust and you could build all the interior parts and put a Tesla interior together right in the middle of where the engineering desks were. It was really cool.

Um, and let them go build it and think about it, and then within the CAD model and the computer, you could see it, but you didn’t always work out in real life. You know, we have a scale problem when you look at something that's small. Even if you scale it up perfectly, when it’s big, sometimes that's just what you thought, and sometimes it doesn't work.

And so you, you want to do, you know, computers like to change things fast, but like real scale models that you sit in and live it and, you know, get a human experience for it. And it was really fun for them to just show up and be like, holy cats, if I did a similar thing, we took all the electrical subsystems and motors and laid them out on two tables, two big tables covered with all the electrical parts of a Model 3.

We stared at it, and once you see them all together, it’s crystal clear that could be a lot better because, you know, there are your three motors that look almost the same. Why isn’t that one motor?

There are these two parts that are completely separate, separate some release assemblies, but if you build it together, you could have one thing do both things really much more naturally, a lighter weight, right? And by laying that all out in front of you, you didn't have to do the mental work of representing that you could do the mental work of seeing how all the parts interconnected and what might be.

Yeah, yeah, Doug Clark is an architect I worked with when I was a kid at Digital called it the intraocular traumatic test. When you look at it, does it bug you? And a lot of things, when you really lay them out like that, you go, oh, we're not doing this right.

And uh, Elon liked that kind of stuff. Like, you know, you're almost afraid to show them like when you laid it all out, you looked at it and go, this is crazy! Like you built this car? Who did this?

Gave us—so I wanted to talk to you too. I want to talk to you like I'm someone very stupid, and in this particular regard, I am. I really don't understand how computation works. And you're a microprocessor architect, and you build computers.

And so I'm—and I listened to a discussion you had earlier this month, and there was a lot of it I couldn't follow.

I thought it might be helpful and interesting for you just to walk through for me and for my audience how a computer actually works, what it does and how you build it, and then what it would be like to design and to architect a microprocessor.

Yeah, well, it's somewhat hard to describe, but there's a couple simple things. So let's start with the easiest thing: the computers have three components: really memory, programs, and input and output, right? Those are the three basic things we always build, right?

So memory is like the DRAM or the disk drive, a place where you store data, and it’s just stored, and it can have different representations. We currently use ones and zeros. Like, so you can take any bit of information and describe it as a sequence of ones and zeros. And it's stored in silicon in either static and dynamic memories or on disk drives, which, you know, there are a couple technologies for that.

Memory make sense to you?

It does, although I have some difficulty in understanding exactly how the transformation is undertaken to represent things in zeros and ones.

I mean, I’ll give you a simple example. So if you shine a light on a photo detector, right? So the light comes in, and it's a stream of photons, right? And the photo detector counts the photons literally. So every photon that hits it or a couple photons hit it, they cause some electric charge to move and that causes the circuit to wake up and say, I saw some photons.

So say you're trying to evaluate how strong that light is, so it can be anywhere from nothing to super intense, right? And then you might say, well, let’s, let’s put that in a range of numbers from zero to a thousand, right?

So—and then you're trying to lay on it and so you count the photons for, say, you know, microseconds, and then you translate how many photons you counted into the number, right? So—and so just imagine as the light varies up and down, the count, the number coming out of your photo detector is varying between zero and a thousand, right?

And we use base 10, but you can translate that to binary, which is base two and now you have ones and zeros. You’ve basically now translated a light ray, optical information to a count.

So virtually everything can be represented by a count, apparently.

So just think of your computer; it has a camera, which is essentially doing just that when you get different colors. But first you go through color filters, so you have a green filter and a red filter and a blue filter. That's enough to represent the color spectrum, and then you have a little photon counter underneath that, and it counts for a little while, and then you ship out the number and you reset it and count again.

So the light varies; you're getting that there's a pretty big grid. Like a modern camera’s 12 million, you know, photo detectors in it. Actually, 36 million because it’s got different colors. But, well, it depends on how they build it.

Like, they might have different pixels be different colors and interpolate the colors. Like a keyboard is really simple.

So the fundamental issue to begin with is that you reduce everything to a count and then you represent that count in in base two, base two, right? Zero and one, and you can do the same thing with sound. You can have a sound detector that basically counts, you know, intensity of sound waves, and the keyboard, well, you have a little grid of keys. When you push a key, it sends what, which key was that? That was key number 27.

So now you can, it’s either on or off, on or off. You know, so D might be count number 26, and F is 27, and G is 28. So everything gets encoded into a number, and then computer is like binary numbers, you know, for technical reasons, but that’s not a big deal.

So memory, so input and output is the first thing you see. So the computer's built around the memory, so the input and output system. So the inputs are all your photo detectors, sound detectors, keyboard detectors, and there's amazing numbers of sensors. These days, you can detect gravitational waves, maybe you can detect, you know, photons, you can detect electrical waves, sound waves, you can take temperature, you know, it’s very varied. You turn all that stuff into a number.

And then your input device writes that into memory, and memory stores information. You know, memory used to be small and expensive and now it's big and cheap, but it’s still just memories, and nothing happens to it in the memory; it just gets stored.

If you look inside the computer, it's a big memory, and it’s full of numbers. Now, the person who's writing the program, you know, is telling like the input-output, like here comes the input video stream, put the video stream address one million. So all the memory is addressed, and the address typically starts at zero, and modern computer goes up to billions.

Okay, so walk through that again, the addressing. So what what's exactly the function of that?

Well, you want to know where the memory is.

Okay, right, you need to—okay, fine.

So basically your phone probably—I don't know, has eight or sixteen gigabytes of memory in it.

Yeah, maybe.

Yeah, four or eight, I don't know. So billion, you know, eight billion bytes of information in there. So—and when you're designing your programs, you kind of lay out, well here's our—the operating system is going, here's what the input-output buffers are, here’s memory we're going to use to run some program.

So all that stuff is addressed. So you can think of the addresses. It’s just like a post office, right? So you know, every house has a postal address, you know, it’s a street address and then your house address, and so you can find every person, and the address corresponds to the physical location.

In some sense, to the physical location of the—with the chip—memories—

How are the zero and ones represented in the memory?

It's literally a voltage that's either high or low, and zero is using ground, which is zero volts; modern, you know, DRAM cells probably stored at 1.1 volts, and you know, in a DRAM cell, it’s a capacitor that’s holding electrons.

So basically when you store the cell, either you drain all the electrons out, so it’s zero volts, or you put a bunch of electrons in so that it holds a one volt. So it’s literally a number of electrons in there; there's a couple ways to make memory cells.

There's another way, which is called a bistable element where you have what's called cross-coupled inverters, but that's too complicated to explain.

And then the memories are usually built in arrays, so there's an XY, so you take the number and you say I'll take the bottom half of the number and figure out which row it’s in, and the top has a number of which column it's in. And where the column and the row overlap, then I'll write my new data, a one and zero, in that spot.

It's literally that simple. So if you look at a memory chip, you'll see this array of bits with little blocks on two edges. Usually, you know, one side's the row, one side's the column, and then the bottom is what they call a sense when you read it back out again.

So, so memory processes, you activate the wrong column to a spot which gives you the address of that bit and then you drive the bid in and charge up or discharge that cell, and then it holds them. And it's super simple. You can build a memory with a pegboard. You could build a memory; people literally did, you know, way back when there was something called clear memory where they had essentially the XY grid.

And at each little place, there was a little magnetic bead, which when you put the currency, you could make it be north to south and the opposite direction, south and north.

So the you know, magnet, you basically re-magnetize the little beads. So there's lots of ways to make memory, but currently, the really dense memories are called dynamic memories where you literally put charge in there.

And then there's fun stuff that happens, like flash cells; the cells got so small that the electrons, you know, from the quantum effects tunnel out occasionally. So you put about where the electron actually is, yes, and sometimes it could literally jump out of the cell. Once it jumps out and it doesn't come back, so they got down to, like, 25 electrons in the cell, and they would wander off over a couple hours. You have to refresh them.

So periodically, you go back, and you read the data before too much of it escapes and you write it back in, you know, so it's called refreshing the memory. But DRAMs hold more charge than that, and then the flash guys figured out how to stack the cell.

So modern flash chips, there’s an XY grid, but there’s also a C dimension. They’re like 256 layers thick now, so it’s like a three-dimensional memory. But the simple thing still is it’s a linear range of addresses where you put some data.

Okay, so that's memory. So the next component is programs. This is the compute part. So it took a simple program. Simple program is A equal B plus C, right? So the data at address A. So when you write the program, you tend to use what you call variable names A, B, and C.

But there's a tool called a compiler which will sign A and say address 100 and B the address 101 and C the address 102, right?

And then the pro—the computer when it’s running says do what I told you to do. So you see this, this program A equal B plus C. So you get B, you get C, you add them together, you put it in A. And typically what happens is you have a what's called a local memory or a registered file.

So you get the data from memory into the register file; you do whatever operation you're supposed to be told to do, like ADS, and then you put C back in them. And what are the range of operations, or is that too broader question? What are the fundamental operations?

Apparently, they're arithmetic. I've done this, like the number of operations that a computer does, like the construction sets can have a hundred or five hundred or a thousand different instructions. But the most common ones are load, data from memory to the processor, or the program ones, store memory-backed.

Those are your first two instructions. And then add, subtract, multiply, divide, you know, clear, you know, very simple, it's written, you know?

Then there’s something called logical operators, AND, OR, NOT. It’s stunning to me. Conceptually thinking through this that computers, which can produce whole worlds in some sense, can do that as a consequence of zeros and ones and arithmetic operators.

Sure. Well, your brain's, it's doing something interesting like that. There’s no magic to it. So the key to programs is abstraction layers, right? So at some low level, you know, like I understand computers from atoms up to operators, which is a fairly broad range.

But there’s lots of people who can do that, yet I understand them like the surface of the keyboard. Yes, yes, monkey with military helicopter basically.

Pete Bannon had an interview question. So, you know, computer scientists would say, “Tell me what happens when I move when I type a key?” Right? Because you can talk all day. I could talk all day then, you know, because the key is that the position which encoded the number which got sent into the memory.

There’s an interrupt delivered to a processor to say there's new data. Remember to go take a look at it. But you can describe that in many, many levels, right? So it's a good place to start people; it's not bad.

All right, an interview question; it was an interview.

Yeah, right. Some people, by the way, are stumped. They go to college and they can't tell you what happens, you know, the key is clicked, which is weird.

So, back to the computer. So, yep, with the basic operations, that subtract, multiply, divide, you know, clear, set the one, and or not XOR, you know, you can take a number and you can shift it around, you can mask it.

And so different architectures, how are those operators discovered, Jim? I mean, I know there’re arithmetic operators, and is that just the question of how was their arithmetic discovered? But I mean after, way after math. So computers, you know, at some level, they're doing arithmetic.

Like, it’s not very sophisticated, and I’ll get to a little more complicated version of this. And by the time we invented computers, people had pretty good idea of number theory; they’d figured out that base 10 was just one of the bases. You could have two, three, four, five, six, seven, eight.

People had—the philosophers had worked out what logic is, you know, if this is true and this is true then this is true, this is true or this is true. Like the logical operators are real. Like there’s a whole bunch—there was a whole bunch of—it’s the real—it's the realism of them that’s stuck. That’s stunning to me.

Yes! So there’s the basic operator set, and then there’s something called control flow. So computers typically, they put a program like add, you know, A equal B plus C, you know, D equals E plus F, F equals E plus A.

You typically put that in what's called program memory, but it’s just part of the memory of the computer. And you have a program counter, which you know I know it's called counter, but the thing that points at the next instruction to execute, and its default thing is do this instruction and then do the one right after it.

That became like the way computers are built; it’s an arbitrary choice, by the way. You can have every instruction tell you where to get the next instruction. Here’s a bunch of things you can do, but for simplicity, people said this piece of memory has programs in it. Start at the first instruction and then do the next one and the next one, the next. The program counters, but that’s not good enough.

Because then you would just start the first one, and you go to the end of memory and be done. So there's something called control flow.

So a program called, sorry called control, control.

So imagine you wanted to add up a list of ten numbers, so you—so your first instruction says I'm on the first instruction. Then you say that the sum equals the current sum plus the next number, increment the counter of how many instructions I've had, increment count by one, and then test is the counter equal to ten? If yes, keep going straight. If no, go back to get the next number, right?

So you created a little loop, right? So—and there’s, it turns out there’s computer scientists and invite invented a whole bunch of kind of loop—what they call control flow constraints.

Do this while X is true. Do this until the counter gets to a number, right? So there’s, so you can create little, you know, basically—subprograms in the program, right?

And then there’s a couple—you can—you can test, like, hey, I need to decide if this is a dog or a cat. So if it’s a one, go look it here. If it’s a zero, go look at that, right? So that’s a conditional branch with group branches.

And