Why Can't We Find the Theory of Everything? Einstein, Rogue Genius, String Theory | Eric Weinstein

In some ways we’ve been making amazing progress for 40 years—in my opinion—in the mathematics of field theory, which is the underlying geometric structure that undergirds both particle theory and general relativity. So this has been an incredibly exciting time because this dictionary has opened up which ports all of the best insights from physics into differential geometry and from differential geometry back into physics. So you’d be hard-pressed to say that nothing is happening.

The problem is that we really wanted to quantize the geometry of general relativity but, in fact, what we ended up doing was geometrizing the quantum. And so it’s been a bit of a disappointment for theoretical physicists who hoped that they would be living through a golden age of theoretical physics rather than the mathematics of theoretical physics. So the field of particle theory has in some ways seemed to be advancing in terms of its mathematical underpinnings. But the elaborations on the standard model, which is our specific understanding of the world in which we live, has been all but stalled from the theory side since around 1973-1974.

So it’s a bit of a paradoxical situation and I think that, in part, we’ve never really been here before. There was a period between about 1928 until the late 40s when theoretical physics had found quantum electrodynamics, the theory of electrons and photons, where most of the calculations we wanted to do gave infinite answers. The underlying theory seems sound. We just didn’t know how to get real contact with experiment. And it took a long time for us to realize that we had a technical problem rather than a need for an absolutely fundamental revolution of the kind that brought us general relativity and quantum theory.

So I think that we’re a bit stuck and we don’t really know how long this very strange period is going to go on for. This period has been dominated by the sort of quixotic hopes that one of a number of theories—whether it be super-symmetry theory, grand unified theory, technicolor, or even noncommunicative geometry—might be our way out. But the problem is that all of these highly speculative theories have remained in limbo, and many of them have gotten rolled into this very strange complex of ideas that we call either string theory or M theory or some variant thereof.

And it is a question as to whether this is more of a physics-inspired theory or whether it’s really an economic and sociological phenomenon, which is that you have a generation of physicists in the baby boom who seem to be absolutely astounding geometers but appear to be wanting in terms of their ability to make contact with the natural world by the standards of previous generations. Naturally, that’s going to elicit some very strong feelings because the idea that we would have had perhaps two generations, let’s say, in 40 years of physicists who can’t make contact with experimental reality with their theories is completely unprecedented in the modern era. This is very interesting and rather disturbing.

So I was quite inspired by a talk or two that I’ve seen of the distinguished physicist Nima Arkani-Hamed, where in essence he points out that the three main equations that give us all of theoretical physics—the Dirac equation for matter and then the force equations, the Yang-Mills equation, and the Einstein field equations—are all in some sense provably the best possible equations in their category of equations. And so what happened was that we had a question: is there any way to go about finding even better equations?

We can essentially prove that these equations cannot be beaten in any simple way. So the possible elaborations I would say are now obvious, and we’ve tried all of them and none of them have seemed to yield to anything that clearly advances our picture beyond where we are now. So the question is, do we need a radical rethinking? Is there something wrong with the fundamentals? Is Einstein, in fact, wrong to slip in space-time on the ground floor of the theoretical physics which is shared by both quantum field theory and general relativity?

Or are we simply in that situation where you think you’ve searched your apartment everywhere for your missing spectacles or keys but, in fact, it was hiding in plain sight the whole time—you just didn’t think to look in the right place? And I would say the jury is really out, and the problem is that this is in some sense—and I say this not as an insider in physics but really as an outsider since I wasn’t trained in that subject per se—but this is the world’s most accomplished intellectual community, whether you find them easy to deal with or sometimes rather unpleasant, as I occasionally do.

There is no question in my mind that no other group has ever achieved anything like the theoretical physics community. But the question is, why are they stumped? And if they do need help, where can it come from? It doesn’t seem that any of the chemists or the biologists would have enough to contribute even though physics has contributed to both of those fields. And so the real hope is that it’s either going to come from theoretical physicists themselves, from mathematicians who struggle to make any kind of contact because the pedagogy in physics is quite forbidding (and I would say it’s not quite as good as the pedagogy in mathematics, generally speaking), or it is going to come from some completely strange source, maybe somebody who is a self-teacher, off the grid, that we’ve never heard of.

But we’ve heard from all of the leading lights, and I would have to say that almost no one from the traditional community really has any kind of a great idea as to how to make the next progress. Well, I think that if you think about Einstein’s vision abstractly, properly, in all probability, I think he’ll be proved right in the end in the abstract. But the key question is, did he get some of the particulars wrong? He has a beautiful quote where he says that his equation can be viewed as a mansion with two wings, one of which is made from fine marble and the other is made from cheap wood (being the two signs of the equality).

Now most people have looked at the cheap wood and said, well, our theory of matter and the stress-energy tensor, as it’s known technically, is probably what needs to be upgraded so that the equation is pure marble on both sides. There’s a rather more disturbing possibility which is that the marble is, in fact, a premature codification of the geometry and that, in fact, it is not impossible that we have been so beguiled by the beauty and elegance of the marble side of Einstein’s equation that we haven’t put nearly the time or the energy into figuring out whether that’s where the problem is.

But the problem for us if we do go down that route is that Einstein’s theory is so locked in at this point through path dependence. We’ve built everything upon his insights that it’s not really clear how you could make a modification to the foundations of physics without having the whole thing collapse around you. And so even if you have an idea that you’re going to do something very heterodox, which is to question the bedrock or the marble of the geometry, the question is can you even get to it given the incredible skyscraper that has been built on his solid geometric foundations?

So this is in some sense the route that I’ve gone down, which is to try to think about novel approaches. If you are going to break with the community, it’s very difficult to keep up with that level of neural horsepower if you have any other commitments on your time. So in some sense, if you choose the path of the dissident or the heterodox or the crank, you will find that your only hope and chance is to have a really novel idea about how this game goes so that you have some time and some breathing room for everyone else.

And, of course, nobody’s very optimistic about that prospect because it’s very difficult to do work on one’s own as Einstein did in the patent office. In fact, we haven’t seen a second version of his story since his famous emergence from the patent office. However, the fact is that the traditional community is also stalled out.

So you have two horses, neither of which seems to be capable of finishing the race, and the question at this moment is should we be looking more to the heterodox—running the risk of craziness and cranks—or should we be looking more to the traditional community which seems to have gotten itself in a cul-de-sac that we call string theory, M-theory, and super-symmetry? The jury is out, but I think it’s become a much more interesting question because traditionally we would have bet on the experts.

But the experts have taken more time researching this theory than any group I think has ever taken to research a theory. And the fact that they have been unable to find anything, in fact, means that perhaps the odds have changed in that race.