Pessimism Appears to Be the Intellectually Serious Position
If you're an academic of some kind, then being able to explain all of the problems that are out there and how dangerous these problems are, and why you need funding in order to look at these problems in more depth, that appears to be the intellectually serious position. Someone who claims that we can solve this sounds a little bit kumbaya. Even though it's quite right that, in fact, collaboration, cooperation, and resource exploitation will actually be the thing that's going to drive this knowledge economy forward so that we can solve these problems.
It always seems more intellectually serious if you can stand up there with a frown on your face in front of a TED Talk audience and say, "These are all the ways in which we're going to fail and which we're going to come to ruin." I'm guilty of having recorded one of these doomsayer podcasts about enders blowing up the earth, but that was the one podcast that I regretted the most. We had a great conversation, but I don't fundamentally agree with any of the conclusions that might come out of that, which say the world is going to end, so we should slow down. The only way out is through progress.
Subsequently, I haven't promoted it as much as I promoted my other podcast, and upon reading Deutsch, I realize why. It's because it's easy to be a pessimist; it's an easy trap to fall into, but it implies that humans are not creative. It doesn't acknowledge all the ways that we have innovated our way out of previous traps. Fundamentally, entrepreneurs are inherently optimistic because they get rewarded for being optimistic. As you're saying, intellectuals get rewarded for being pessimistic.
So, there is always a lot of incentive bias here. As an academic, you may be incented to be pessimistic; as an entrepreneur, you may be incented to be optimistic. If you're a pessimist, you get your feedback from other people. It's a social act; you're convincing other people of your pessimism. So far, most of their pessimistic predictions have turned out to be false. If you look at any timelines on which the world was supposed to end or environmental catastrophes were supposed to happen, they've been quite wrong.
But if you look at the optimistic entrepreneurs, they are rated by feedback from nature and free markets, which I believe are much more realistic feedback mechanisms.