Your Face Makes the First Impression—What Does It Say? Bias, Evolution, Trust | Alexander Todorov

So in the early days, including from the time of Aristotle and later in the 16th and 17th century, most of physiognomy consisted of whimsical comparisons between the physiognomy of humans and animals. So, for example, you would have a drawing of a human who presumably looks like a cow, and from there you will make all kinds of inferences that perhaps the character of the person matches the character of the cow—whatever that might be.

In fact, if you look at the history of European history, most of the 19th century's novels had a standard feature of this analysis: you have physiognomic descriptions of the characters. So it was very, very popular and influenced not only marginal writers but big names like Balzac, Stendhal, and many others. Now, interestingly enough, in the early 20th century, people no longer talk about physiognomy; they talk about character analysis.

And in fact, a lot of the references are no longer to Lavater's whimsical ideas, but they're to evolutionary ideas. The so-called character analysts were quite influential and were involved heavily in business and recruitment of employees, but this is exactly the time when the new science of psychology arises. Then psychologists are kind of skeptical about the claims of the character analysts—or really the new physiognomists. The fact that we agree on this impression was discovered over 100 years ago in psychology, but psychologists at the time were really focused on the accuracy of the impressions and paid very little attention to the extremely interesting psychological fact that we actually agree on these impressions.

Very often in psychology, and generally in the social sciences lately, if you observe that there's a pervasive bias, that is, it's something that feels fairly automatic, and we can all do it. There is kind of almost immediate assumption that this might be actually wired; that it's something that we are born to be able to do. Of course, you can easily think of counterexamples like driving, which essentially becomes automatic and there's nothing evolutionary about driving—or reading. But nevertheless, things that seem to have been present always in our environment, like faces, that seem like natural assumptions.

It's interesting actually, we've done some studies—there are many, many different inputs to impressions. One is emotional expressions; there's stereotypes about gender, there's cues about age and facial maturity—all of this goes into our impressions. Another one that is very interesting is typicality. So as it turns out, we tend to like faces that are typical; that means faces that are closer to what we perceive as typical in our social environment.

Now there's an interesting wrinkle because typicality is also culturally specific, especially if the different cultures are linked to different ethnicities and there's distinctive physiognomies, and that makes it worse. We've done a study where we created morphs of a typical Japanese face and a typical Israeli face, and then we can interpolate the morph. So, we can imagine like a typical Japanese face gradually turning into a typical Israeli face.

Now, if you ask Israeli and Japanese participants to evaluate the faces, what happens is as the face becomes more Israeli-looking, the Israelis believe the face is becoming more trustworthy, and the other way around for the Japanese. So, in a sense, to a large extent, what we perceive as typical is shaped by our natural environment. And it's something that we very rapidly extract; we are incredibly good learners about faces.

Most likely, people who live in New York City, with a hugely diverse face, will have a different notion of typicality if you live in a small rural town where there's not so much diversity. In this case, this can lead to different kinds of suboptimal outcomes because naturally we wouldn't trust people that do not look like us, not having any other information. The reason why we will never be able to get rid of first impressions is because they serve important psychological functions.

That is, in the absence of any other information, we're trying the best we can to figure out what the other people are thinking. That doesn't mean that we wouldn't change our minds; on the contrary, when you have good diagnostic evidence about the person or when you know about past behavior, that would change inferences based on appearance. But most of the time, if you don't have any other information, people will act on this, and that may not be in their best interest.

Starting back to Cesare Lombroso, who wrote books like The Criminal Man and The Criminal Woman, he claimed that he could identify these “inferior types” based on their facial features. To Francis Galton, who invented composite photography, and in fact all of today's morphing methods are based on this method of composite photography. The first application of the method was to identify the criminal type, so it has a very long history.

I think a very reasonable argument could be that we are kind of hardwired to figure out the intentions of other people, of the people around us, because what is the most important thing in our social life? It's other people! In interactions with strangers, you're always trying to figure out what are their intentions? Are they good? Are they bad? What are they going to do? Can they hurt me, whether that’s physical or in a non-physical way? So, these are things that have always been a concern for us.

But let's think in terms of evolutionary history. Well, for most of our evolutionary history, we've basically lived in extended families, typically between five to eight individuals. All of these changes in the last 50,000—in fact, even less—maybe in the last 20,000 years, when you have large societies—that is, if you imagine human evolution compressed within 24 hours, we have been living with strangers surrounded by strangers in the last five minutes. So it's not obvious at all that we are kind of endowed with reading the characters of other faces.

I think there's good comparative evidence that in fact we are very good at picking up on social cues in the immediate situations. So if you look, for example, at comparative studies, you look across all primates, it turns out that we are the only primate which has whites of the eyes. That is, our iris is dark and then you have the white sclera, and then you have darker skin. There are no other primates with this kind of coloration.

So why is that interesting or important? Well, the fact that you have the white of the sclera makes it super easy to detect eye gaze. Eye gaze is very important for sharing social attention; we can communicate from a long distance. Similar emotional expressions are very important; the fact that we are kind of the “naked ape” and the fact that our faces are not completely covered by hair makes it very easy to detect changes in skin colorations, which is often an indication of different kinds of emotional or mental states.

So we are very sensitive to changes—momentary changes—in what we are observing because these momentary changes are indicative of what is important in this situation, what is happening right now. But it's hard to make the argument that somehow we are endowed also to read into faces the character of others. I think we have the natural propensity trying to figure out what these other people are thinking or feeling right now.

And I think the problem with physiognomy in the modern version is this assumption that because you can make these rapid inferences, that's also informative about these people across time and situations.