A short history of knowledge, from feudalism to the Internet | Alice Dreger

Peer review is not a simple thing because humans do it. And so whenever there are humans, there will be bias introduced; relationships get complicated with regard to peer review. We have a fantasy that peer review will be totally anonymous on both ends – you won’t know who the person you’re reviewing is, and they won’t know who you are, and it will all be blind. But in reality, people figure out in peer review who is who, and biases do get introduced.

But that said, the idea of peer review is a really important one, and we can sort of approach it. That’s the idea that we have people who are qualified judging each other’s work. It allows us to essentially crowd source knowledge and to have an opportunity where blind spots are picked off, where errors are picked up, and where we can make work better. So it’s essentially a way to crowd source knowledge.

I want to point out, by the way, that peer review is historically really interesting. It came out of the Enlightenment period. So this was the period when thinkers were beginning to really appreciate the idea that humans together could know more. And what’s fascinating is that democracy and science grew up together, and they both use peer review.

So science uses peer review because scientific ideas are put forth, and then scientists are qualified to do so, to judge that work. And in democracy, peer review is used in things like voting systems. So when we do voting, that’s a peer review form. When we do judging of criminal or non-criminal acts in courts, that’s a form of peer review when we have a jury.

And so it’s not a coincidence because what was happening was the thinkers of the Enlightenment were beginning to figure out that more people looking at a problem could get you better knowledge. Before the Enlightenment, the concept was knowledge came from above; it came from the church, from the state, from God, it came from an external higher authority.

But the real revelation of the Enlightenment was the idea that people could do this themselves; they didn’t have to rely on the church, the state, God, or an external authority. They could do it themselves. And so they began to have the idea that they would reject the king, and they would essentially reject the teachings of the church, and they would reject the state being run by the king; that they would take back control of knowledge.

And that was true in democracy and in science. So, it’s no coincidence that a lot of the founding fathers were science geeks. They were thinking about crowdsourcing. It is what we call crowdsourcing. There are more checks and balances on bad knowledge going forward, so there’s accountability at some level.

When you’re doing peer review, the editor at least knows who you are. When you’re doing voting, theoretically, you’re not allowed to vote more than once. When you’re on a jury, you have got a judge keeping track of whether or not information should be admissible in court, whether it’s fair to admit it in court.

The internet is crowdsourcing gone wild. It has no limits on it, and so you can have things like bots – like things until it actually is noticed by real human beings. You can have situations where something looks incredibly real, but it’s not real, and it will take off much faster than we can stop it.

So the internet really is a beautiful thing in many ways. It allows people to find each other who never could have found each other before. For example, people with very unusual medical conditions can find each other, people with very unusual interests can find each other. The problem is that you have a situation where there are no checks and balances, and so you get a phenomenon whereby things that are not real can go forward.

But there are some places on the Internet where there are checks and balances. So Wikipedia is a great example, actually. Wikipedia actually has people who function as editors, and they will talk to each other, fight with each other, and all the discussions get externalized. That allows a level of accountability that much of the Internet doesn’t have.

It’s also the case that Wikipedia paid editors can actually stop people from editing in some circumstances or stop people from messing with pages. So there are places on the Internet that have been born of crowdsourcing but that do have some checks and balances built in.