Intellectual dark matter: What is it, and why is it meaningful? | Samo Burja | Big Think

Knowledge rests at the foundation of our society. Knowledge of how to build things like cars, airplanes, nuclear reactors, computers, but also knowledge how to build the companies and governments that allow for things such as cars, airplanes, and nuclear reactors. A failure of these social technologies ultimately results in a failure of the material technologies at hand. A lot of this knowledge, both the physical and the social knowledge, is, however, not readily apparent and not easily describable in words.

We live in a society that’s dominated by bureaucracies. Bureaucracies that have evolved and where people follow rules and procedures that are written down. The vast universe of tacit knowledge, however, shows that there exists an intellectual dark matter all around us that we can’t put into writing, can’t put into words. Intellectual dark matter is perhaps very similar to physical dark matter. Most of the mass in our universe is invisible, at least to telescopes, but we know it’s there. We know it’s there almost certainly because the galaxies are spinning too fast. The stars would fly apart were there not this 85 percent of invisible mass.

Looking at society today, if we only saw what was written down, what was explicitly laid out, what was explicitly documented on Wikipedia, for example, this could never hold together. This is why we know that intellectual dark matter is out there. To understand intellectual dark matter, you have to understand tacit knowledge, and you have to understand technical debt. Technical debt is when someone else solves a very difficult problem, be it in code or a manufacturing process or perhaps a philosophical puzzle. You’re relying on their solution without even understanding their solution. You’re taking these facts and already built pieces as a given. And on top of them, you build something else.

It’s, in fact, possible to lose this original knowledge. You then end up simply relying on this black box technology that’s no longer around. A classical example of this is perhaps that to manufacture a Saturn V engine in say the Apollo rocket would today require a major reengineering effort by NASA. The thousands of subcontractors that built the individual pieces of the engine, even if the engine blueprints are around, those companies are no longer around and the components would not be built to the same tolerance.

So the technical debt that exists in an ecosystem of companies and other organizations, and especially very skilled individuals, can be very difficult to replicate, can be very difficult to document properly. The second type of knowledge is this knowledge debt can’t really be documented. How do you describe in words the exact right way to spin during a dance or the exact right way to, you know, throw a basketball into a hoop? These things matter immensely.

A world class surgeon is notably better than a mediocre surgeon for reasons that are almost impossible to put into words. A lot of our great scientific advances arrived out of us formalizing, using mathematics, the implicit and unstated understanding of how the physical world around us behaves. Say how an object falls to something that could be made visible, transparent. In a very real sense, every step you might take in transforming the intellectual dark matter of our society into visible intellectual matter—stuff that’s written down, that is formalized, that is perhaps recorded in video—is a step that reduces the fragility of our society.