The security mirage - Bruce Schneier

[Music] [Applause] So security is two different things, right? It's a feeling and it's a reality, and they're different. You could feel secure even if you're not, and you can be secure even if you don't feel it. I mean, really, we have two separate concepts mapped onto the same word. What I want to do in this talk is to split them apart, figuring out when they diverge and how they converge.

Language is actually a problem here; there aren't a lot of good words for the concepts we're going to talk about. If you look at security from economic terms, it's a trade-off. Every time you get some security, you're always trading off something. Whether this is a personal decision—whether you're going to install a burglar alarm in your home—or a national decision when you're going to invade some foreign country, you're going to trade off something—either money or time, convenience, capabilities, maybe fundamental liberties.

The question to ask when you look at security in anything is not whether this makes us safer but whether it's worth the trade-off. You've heard in the past several years, "The world is safer because Saddam Hussein is not in power." That might be true, but it's not terribly relevant. The question is, was it worth it? And you know, you can make your own decision, and then you'll decide whether the invasion was worth it. That's how you think about security in terms of the trade-off.

Now, there's often no right or wrong here. Some of us have a burglar alarm system at home, and some of us don't. It'll depend on where we live, whether we live alone or have a family, how much cool stuff we have, and how much we're willing to accept the risk of theft. Right? In politics also, there are different opinions. A lot of times, these trade-offs are about more than just security. I think it's really important. Now, people have a natural intuition about these trade-offs. We make them every day. Last night in my hotel room, when I decided to double-lock the door, or you, in your car when you drove here, you know, when we go eat lunch and decide the food's not poison and we'll eat it, right? We make these trade-offs again and again, multiple times a day. We often don't even notice them; they're just part of being alive. We all do it; every species does it.

Right? Imagine a rabbit in a field eating grass. The rabbit's going to see a fox. That rabbit will make a security trade-off: should I stay or should I flee? If you think about it, the rabbits that are good at making that trade-off will tend to live and reproduce, and the rabbits that are bad at it will get eaten or starve. So you'd think that us, as a successful species on the planet—right, you, me, everybody—would be really good at making these trade-offs. Yet it seems again and again that we're hopelessly bad at it, right? And I think that's a fundamentally interesting question.

I'll give you the short answer: the answer is we respond to the feeling of security and not the reality. Now, most of the time that works, right? Most of the time, feeling and reality are the same. Certainly, that's true for most of human prehistory, right? We've developed this ability because it makes evolutionary sense. I mean, one way to think of it is that we're highly optimized for risk decisions that are endemic to living in small family groups in the East African Highlands in 100,000 BC. 2010 New York, you know—not so much.

Now, there are several biases in risk. There are a lot of good experiments in this, and you can see certain biases that come up again and again. I'll give you four: we tend to exaggerate spectacular and rare risks and downplay common risks—so flying versus driving. The unknown is perceived to be riskier than the familiar. One example would be people fearing kidnapping by strangers, when the data supports kidnapping by relatives being much more common. This is for children.

Third, personified risks are perceived to be greater than anonymous risks—so Bin Laden is scarier because he has a name. And fourth, people underestimate security risks in situations they do control and overestimate them in situations they don't control. So once you take up skydiving or smoking, you downplay the risks. If a risk is thrust upon you, terrorism is a good example; you'll overplay it because you don't feel like it's in your control.

There are a bunch of other biases—these cognitive biases—that affect our risk decisions. There's the availability heuristic, which basically means we estimate the probability of something by how easy it is to bring instances of it to mind. So you could imagine how that works: if you hear a lot about tiger attacks, there must be a lot of tigers around. You don't hear about lion attacks; there aren't a lot of lions around. This works until you invent newspapers because what newspapers do is they repeat again and again rare risks. I tell people if it's in the news, don't worry about it because by definition, news is something that almost never happens, right? When something is so common, it's no longer news. Where car crashes, domestic violence, those are the risks you worry about.

We're also a species of storytellers; we respond to stories more than data. And there's some basic numeracy going on. I mean, the joke "one, two, three, many" is kind of right. We're really good at small numbers—one mango, two mangoes, three mangoes, 10,000 mangoes, 100,000 mangoes. It's still more mangoes than you can eat before they rot. So one half, one quarter, one fifth, we're good at that. One in a million, one in a billion, they're both almost never, so we have trouble with the risks that aren't very common.

What these cognitive biases do is they act as filters between feeling and reality, right? And the result is that feeling and reality get out of whack; they get different. Now we either have, you know, feeling— you feel more secure than you are—there's a false sense of security—or the other way, and there's, you know, a false sense of insecurity.

I write a lot about security theater, which are products that make people feel secure but don't actually do anything. There's no real word for stuff that makes us secure but doesn't make us feel secure. Maybe it's what the CIA is supposed to do for us. So back to economics: if economics, if the market drives security, and if people make trade-offs based on the feeling of security, then the smart thing for companies to do, for the economic incentives, is to make people feel secure, right?

And there are two ways to do this. One, you can make people actually secure and hope they notice; or two, you can make people just feel secure and hope they don't notice, right? So what makes people notice? Well, a couple of things: understanding, right? The security of the risks, the threats, the countermeasures, how they work. If you know stuff, you're more likely to have your feelings match reality. Enough real-world examples help. Now we all know the crime rate in our neighborhood because we live there, and we get a feeling about it that basically matches reality, right? Security theater is exposed when it's obvious that it's not working properly.

Okay, so what makes people not notice? Well, a poor understanding, right? If you don't understand the risks, you don't understand the costs, you're likely to get the trade-off wrong, and your feeling doesn't match reality, right? Not enough examples. There's an inherent problem with low-probability events. If, for example, terrorism almost never happens, it's really hard to judge the efficacy of counterterrorism measures. Right? I mean, this is why you keep sacrificing virgins and why your unicorn defenses are working just great.

There aren't enough examples of failures. Also, feelings that are clouding the issues, right? The cognitive biases I talked about earlier, fears, folk beliefs—right? Basically an inadequate model of reality. So let me complicate things, right? I have feeling and reality. I want to add a third element—I want to add model, right? Feeling and model in our head; reality is the outside world. It doesn't change; it's real, right? So feeling is based on our intuition; model is based on reason— that's basically the difference.

In a primitive and simple world, there's really no reason for a model, right? Because feeling is close to reality; you don't need a model. But in a modern and complex world, you need models to understand a lot of the risks we face. You know, there's no feeling about germs; you need a model to understand them. So this model is an intelligent representation of reality. It's, of course, limited by science, by technology, right? We couldn't have a germ theory of disease before we invented the microscope to see them.

It's limited by our cognitive biases, but it has the ability to override our feelings. Where do we get these models? We get them from others. We get them from religion, from culture, teachers, elders. A couple of years ago, I was in South Africa on safari. The tracker I was with grew up in Kruger National Park. He had some very complex models on how to survive, and it depended if you were attacked by a lion or a leopard or a rhino and an elephant, and when you had to run away and when you couldn't run away and when you had to climb a tree and when you could never climb a tree.

I would have, you know, died in a day, right? But he was born there and he understood how to survive. I was born in New York City; I could have taken him to New York, and he would have died in a day, right? Because we had different models based on our different experiences. Models can come from the media, from our elected officials, right? Think of models of terrorism, child kidnapping, airline safety, car safety. Models can come from industry, and too, I'm following our surveillance cameras, ID cards. Quite a lot of our computer security models come from there. A lot of models come from science; health models are a great example.

Think of cancer, bird flu, swine flu, SARS. I mean all of our feelings of security about those diseases come from models given to us, really by science, filtered through the media, right? So models can change, right? Models are not static. As we become more comfortable in our environments, our model can move closer to our feelings. So an example might be, if you go back 100 years ago when electricity was first becoming common, there were a lot of fears about it. I mean, there were people who were afraid to push doorbells because there was electricity in there, and that was dangerous, right? For us, we're very fossil around electricity; we change light bulbs without even thinking about it, right?

Our model of security around electricity is something we were born into; it hasn't changed as we were growing up, and we're good at it, right? Or think of the risks on the Internet across generations—how your parents approach internet security versus how you do versus how our kids will. You know, models eventually fade into the background. You know, intuitive is just another word for familiar, right? So as your model is close to reality and it converges with feelings, you often don't even know it's there.

So a nice example of this came from last year and swine flu. When swine flu first appeared, the initial news caused a lot of overreaction. Now, it had a name, which made it scarier than the regular flu, even though it was more deadly, and people thought doctors should be able to deal with it. So there was that feeling of lack of control, and those two things made the risk more than it was. As the novelty wore off, the months went by, there was some amount of tolerance; people got used to it, right? There was no new data, but there was less fear.

By autumn, people thought, "The doctors should have solved this already," and there's kind of a bifurcation. People had to choose between sort of fear and acceptance—actually fear and indifference—that kind of choice: suspicion. When the vaccine appeared last winter, there were a lot of people, a surprising number, who refused to get it, and it's a nice example of how people's feelings of security change—how their model changes sort of wildly with no new information, with no new input. This kind of thing happens a lot.

I'm going to give one more complication: we have feeling, model, reality. I have a very relativistic view of security. I think it depends on the observer, and most security decisions have a variety of people involved, and stakeholders with specific trade-offs will try to influence the decision, and I call that their agenda. You see agendas; this is marketing, this is politics—trying to convince you to have one model versus another, trying to convince you to ignore a model, and you trust your feelings by marginalizing people with models you don't like, right? This is not uncommon.

A great example is the risk of smoking and the history of the past 50 years. The smoking risk shows how a model changes and also shows how an industry fights against a model it doesn't like. Right? Compare that to the secondhand smoke debate; you know, probably about 20 years behind. Think about seat belts: when I was a kid, no one wore a seat belt. Nowadays, no kid will let you drive if you're not wearing a seat belt. I compare that to the airbag debate; probably about 30 years behind. I mean, all examples of models changing.

What we learn is that changing models is hard, right? Models are hard to dislodge. If they equal your feelings, you don't even know you have a model. And there's another cognitive bias called confirmation bias, where we tend to accept data that confirms our beliefs and reject data that contradicts our beliefs. I mean, so evidence against our model, we're likely to ignore, even if it's compelling. It has to get very compelling before we'll pay attention.

New models that extend long periods of time are hard. Global warming is a great example. We're terrible at models that span 80 years. We can do to the next harvest; we can often do till our kids grow up, but 80 years, we're just not good at, so it's a very hard model to accept. You know, we can have both models in our head simultaneously, or that kind of, you know, that kind of problem where we have holding both beliefs together by cognitive dissonance. Eventually, the new model will replace the old model. Strong feelings can create a model, right? September 11th created a social security model in a lot of people's heads.

Also, personal experiences with crime can do it. Personal health scare—health scares in the news; you'll see these called flashbulb events by psychiatrists, right? They can create a model instantaneously because they're very emotive. So in the technological world, we don't have experience to judge models, and we rely on others; we rely on proxies. I mean, this works as long as it's the correct others, right? We rely on government agencies to tell us what pharmaceuticals are safe. I flew here yesterday; I didn't check the airplane. You know, I relied on some other group to determine whether my plane was safe to fly.

We're here, and none of us fear the roof is going to collapse on us—not because we checked, but because we're pretty sure the building codes are good, right? I mean, we—it's a model we just accept pretty much by faith, and that's okay. Now what we want is people to get familiar enough with better models, right? Have it reflected in their feelings to allow them to make security trade-offs. Now when these go out of whack, you sort of have—you have two options. One, you can fix people's feelings, right? Directly appeal to feelings. It's manipulation, but it can work.

The second, more honest way is to actually fix the model, right? Change happens slowly. The smoking debate took 40 years, and that was an easy one, right? Some of this stuff is hard. I mean, really, though information seems like our best hope, and I lied. Remember I said feeling, re-feeling, model, reality? I said reality doesn't change; it actually does. We live in a technological world; reality changes all the time.

So we might have, for the first time in our species, feeling chases model, model chases reality. Reality is moving; they might never catch up, right? We don't know, right? But in the long term, both feeling and reality are important. I want to close with two quick stories to illustrate this. In 1982, I know people remember this: there was a short epidemic of Tylenol poisonings in the United States. It's a horrific story. Someone took a bottle of Tylenol, put poison in it, closed it up, and put it back on the shelf. Someone else bought it and died; this terrified people.

There were a couple of copycat attacks. There wasn't any real risk, but people were scared, and this is how the tamper-proof drug industry was invented. Those T-proof caps that came from this—it's complete security theater. You know, as a homework assignment, think of 10 ways to get around it. I'll give you one: a syringe, right? But it made people feel better; it made their feeling of security more match the reality.

Last story: a few years ago, a friend of mine gave birth, and a visitor in the hospital turns out—when a baby’s born now, they put an RFID bracelet on the baby, put a corresponding one on the mother. So if anyone other than the mother takes the baby out of the maternity ward, an alarm goes off. I said, "Well, that's kind of neat; I wonder how rampant baby snatching is out of hospitals?" I go home; I look it up. It basically never happens. But if you think about it, if you are a hospital and you need to take a baby away from its mother, out of the room to run some tests, you better have some good security theater, or she's going to rip your arm off.

So it's important for us—those of us who design security, who look at security policy, or even look at public policy in ways that affect security, right? It's not just reality; it's feeling and reality. What's important is that they be about the same. It's important that if our feelings match reality, we make better security trade-offs. Thank you.