What Jumping Spiders Teach Us About Color

You are not looking at a yellow ball. Your brain might think you're looking at a yellow ball, but look closer. The screen you're watching this on displays color using only red, green, and blue subpixels. The yellow your brain thinks it's seeing is actually a mix of red and green light. The camera I'm talking to right now has a sensor composed of red, green, and blue-sensitive photosites. Again, no yellow. Of course, I have the ball here in my hand, so I am looking at a yellow ball, or am I? (light mysterious music) After all, my eyes aren't so different from that camera. The human retina only has cone cells sensitive to red, green, or blue wavelengths of light. To perceive other colors, we have to integrate the inputs from those three cone types. When yellow light enters my eye, it stimulates my red and green-sensitive cone cells, although not as much as pure red or green light would. With red and green-sensitive cones equally excited, my brain tells me I'm looking at yellow. This is how our technology, our cameras, and screens, and projectors can trick our brains into seeing a whole rainbow of colors using just three wavelengths of light by triggering our three different types of cone cells in different proportions. So is the ball really yellow? What does yellow even mean? A lot of people would say that color is the wavelengths of light that an object reflects. In other words, like Aristotle thought, color is a property of the object. But looking at the same ball on a screen, your eyes only sensed red and green light, yet your brain still perceived it as yellow. So it's also possible, like Galileo believed, that color isn't a property of an object at all, but a phenomenon of the mind instead. But whose mind? Because we aren't the only animals that can see the world in color.

We oftentimes don't really think about how other animals see color. So for example, we buy our dogs bright red or orange toys that are only bright red or orange for us and not for them because they can't see orange or red as being distinct from green. - [Derek] So maybe we should start taking in the world from more than just the human perspective. Doing that might just teach us why color vision evolved in the first place. - We're actually towards the lower end of the spectrum, honestly. We're a step up from our household pets, maybe, if they're cats or dogs, but not nearly as good as many animal groups out there, butterflies, birds, fish, lizards, jumping spiders. Jumping spiders are harmless creatures, actually. They don't ever really get big enough to pose much of a threat to humans. Of course, if you were a small insect, yes, you would absolutely need to be afraid of a jumping spider. - [Lisa] They'll take down prey that's sometimes like two or three times their own body size. - The Chinese word for jumping spider translates literally to "fly tiger," and that's the way I like to think about them, as the kind of small cats of the undergrowth. - Jumping spiders are everywhere. They're in your backyard. They're probably in your kitchen. - There are about 6,000 species of jumping spiders known. Some are sort of furry, some are sort of shiny, striped, spotted, red, green, blue. Pretty much anything you can imagine. It's like every one is a little work of art.

As a group, spiders aren't known for their vision. I mean, most species are nocturnal, and for many, their webs act as a sort of extra sense organ, so they just don't need to see that well. But jumping spiders, as active daytime hunters, well, they're different. Not only do they have great eyesight, but different species have different forms of color vision. Look at those eyes. - Jumping spider eyes are fascinating, and when I say "eyes," I mean eight eyes. Jumping spiders split up things like motion detection and light sensitivity to some eyes, and then color vision and fine detail vision into others. The pair of eyes that are perhaps the most fascinating or most unusual are what we call the principal eyes, and those are the really big eyes in the front of the face that make jumping spiders look a little cute if you're willing to say a spider looks cute ever, and those are built unlike any other eye in the animal kingdom. - [Derek] It turns out the way jumping spiders perceive color has everything to do with the anatomy of those principal eyes. - [Nathan] They're really actually built a lot like a Galilean telescope or binoculars. That big lens that you see from the outside of the animal is one of two lenses in these eyes, and in between those two lenses is a long fluid-filled tube. At the end of that fluid-filled tube is a second lens, and what that lens does is it magnifies the image that that first lens projects down that long tube, and in that way, it increases the ability to see detail by the retina that sits right below it. - [Derek] And when it comes to seeing detail, it's hard to beat a jumping spider.

For most animals, the bigger the eye, the better it functions. Jumping spiders absolutely break this rule. The secondary eyes can see the world about as well as the absolute best insect eyes out there, better than the world's biggest dragonflies, whose entire head is consumed by an eye. The principal eyes, they can actually see pattern in the world better than a lap dog, a house cat, an elephant, and nearly as good as the sharp-sighted pigeon, but it's a very narrow slice of the world that they can see. It's about like your thumb held at arm's length. - [Derek] And it's only in that narrow slice of the world that jumping spiders can see fine detail and color. - So the jumping spider's secondary eyes give them a full 360-degree view of the world. Now, imagine most of that is in black and white. When you see something move, you can swivel to look at it. And now, anything that's really of curiosity to you, you can add to this world of black and white vision fine detail and color. But you can only do it moment by moment. So you're really kind of painting additional details about color and pattern that you couldn't see otherwise. It's a wild world to try to put yourself into. (bee buzzing)

As they sweep their principal eyes across a scene, some species of jumping spiders are adding a lot more color information to their world than others. Most jumping spiders, including this one, are dichromats, meaning they have two types of color-sensitive cone cells in their retinas, just like dogs and most other mammals. - [Nathan] And by comparing those two kinds of cells and how they respond to light in the environment, they get a coarse understanding of color. They can tell the difference between UV, violet, blue, and green. - But some types of jumping spiders are trichromats with three types of cones, like humans, and others are tetrachromats, like birds. (light music) The weird thing is all these species with expanded color vision aren't necessarily close relatives. - In jumping spiders, we have huge variation, even from closely related groups, in how well they're able to see color in the world. Jumping spiders are reinventing, in some ways, the ability to see color over and over again in different ways. - That makes jumping spiders pretty special. I mean, consider primates. Old World monkeys, apes, and humans all have trichromatic color vision, but we also share a common ancestor. So our color vision probably evolved only once, and then it stuck around. This is where jumping spiders really stand out. The ability to see red, for example, has evolved several times in jumping spiders. Researchers know that because they've figured out how different groups are related, and by they, I mostly mean jumping spider fanatic Wayne Maddison. - Oh my gosh, Havaika. Fantastic! Beautiful male. Ah, it's been, it's been 30 years since I've seen a live Havaika. - [Megan] He is absolutely Mr. Jumping Spider. His expertise really is in jumping spider taxonomy.

I work on the evolutionary tree of jumping spiders. The evolutionary tree of life is basically the pathway of genetic descent that links all of us. - [Derek] The position of different species on this evolutionary tree can tell us how they ended up with the traits they have. Like if most jumping spiders don't see red, but two species on two very different parts of the tree do, chances are that those two species evolved those abilities independently. - In which case, then we can start to ask questions like what's driving that evolution? Do they have similar ecologies? Do they hunt similar prey? And try to really understand what selective forces are leading to these expanded color vision systems. - It might help them find food, or discriminate tasty prey from prey that can harm them. - [Nathan] Because, of course, lots of small insects are brightly colored, and some of them are using those bright colors to advertise that they're toxic. - [Derek] Another possibility is that seeing a richer world of color might help animals, from lizards to spiders, choose better mates. To test these ideas, the researchers needed to know which of the 6,000 species of jumping spiders had expanded color vision and which didn't, and outside of a few well-studied species, no one really knew. So the team set out to collect spiders from every major branch of the jumping spider family tree. - One of our first things is just prioritizing where to go and what to look for. And so, it's a lot of sampling in a lot of places. - Let's see who lives here. - [Derek] It's kind of like "Pokemon GO," except the Pokemon are real, they're smaller than your pinky fingernail, and they're really good at hiding.

Some jumping spiders have evolved to be really fine-tuned to a particular situation. So for example, there are termite-eating specialist jumping spiders that you're only gonna find around termites. There was this one species that we only found in piles of bones in South Africa. Who knows what it was doing there, but we quickly learned that that was the only place in the environment we were gonna find it. And so, that's part of the fun of it. I feel like it's a bit of a treasure hunt, really. (light music) - With no proverbial stone left unturned, the team returns to their labs with hundreds of spiders representing many different species. They want to figure out which species have expanded color vision, how each species does it, and why. It's actually a hard question to tell how animals can see color. We can't just connect our brains to see what they see. So how do we do it? - [Nathan] We begin by using a technique called microspectrophotometry. It's a really long word. What it simply means is a microscope paired with a device that measures different wavelengths of light, a spectrophotometer. - [Derek] The researchers take ultra-thin slices of jumping spider retinas, and then they measure the wavelengths of light absorbed by individual cone cells. With enough of these measurements, they can tell if a species is a dichromat, trichromat, or tetrachromat, and what wavelengths of light its cells are best at detecting. But that's not the whole story.

Having that knowledge of what's in the retina tells us what is or isn't possible for these animals to see, but it doesn't actually tell us what they do see or how they might use that. And so, the gold standard for establishing that an animal can see color is to do so behaviorally. - In other words, we somehow need to ask the spiders what they can see and then understand their answers. Figuring out what's going on inside a spider's mind is difficult. It's no surprise that it takes a group of expert zoologists to do so. But our own minds can be just as complicated, yet we rarely talk with mental health experts to help us interpret our thoughts and emotions. Whether you're feeling depressed, anxious, or are just stressed and want someone to talk to, a therapist lets you see things from a different perspective, and that's where today's sponsor, BetterHelp, comes in. I know finding a therapist you're comfortable around can be intimidating, especially if you're only limited to the options in your city, but BetterHelp lets you get around this because it's an online platform, and they make it really easy to connect with a professional therapist who can help you work through whatever you're facing. It's easy to sign up. There's a link in the description, betterhelp.com/veritasium, and after answering a few questions, BetterHelp will match you with one of their more than 30,000 therapists. Each therapist is licensed, has a master's or a PhD, and has spent over three years and over 1,000 hours with people. And if you don't click with your first therapist, you can simply switch to a new one for free without things getting awkward. If you feel like you'd benefit from talking to someone, getting advice, feedback, and help, then visit betterhelp.com/veritasium to get started. Clicking that link both helps support this channel, but it also gets you 10% off your first month. So I wanna thank BetterHelp for sponsoring this video, and now, back to jumping spiders and how they see color.

These animals are particularly motivated to investigate things that move, and these responses can be guided by color. - [Derek] The theory is simple. You show the spider a screen with a moving shape that differs from the background in color, but not in brightness, and you see if the spider tries to follow. - [Nathan] The problem with letting the jumping spider actually turn and respond is that they'll absolutely do so, but it changes some of what they see. So what we want is to really have some control over what the spiders can see at any given moment. - So the researchers hold them in place with tiny magnets attached to their heads. - [Nathan] What we do is we give them a ball to stand on. They actually hold it with their feet, and we can monitor how that ball moves around in their feet to know where they would want to go. - If the spider turns the ball to the left, it's probably trying to look to the right to follow the moving shape, and that's evidence the spider can discriminate between the colors of the shape and the background. (playful music) Once the team knows which species can see which colors, the next question is how do they do it? What's different in the DNA of these spiders that can see and discriminate more colors? If you ask Megan Porter, a lot of it comes down to genes that encode proteins called opsins.

The way that animals achieve color vision is to have different copies of this opsin gene, and those variations then are what produce proteins that are sensitive to different colors of light. The first technique that we generally go to with a new species is called transcriptome sequencing, and this is where we can take the entire head of a jumping spider, and we can get the sequences for every single gene that is expressed in that tissue. - [Derek] This method gives the researchers an inventory of all the genes being expressed, in other words, all the genes that are copied out of the DNA and sent to make a protein. Then the team can figure out where each of these genes is expressed, in which eyes, and in which parts of the eye. - And we do that using a fancy technique called immunohistochemistry. - [Derek] The researchers basically create glowing molecular tags specific to each protein they're interested in. - [Megan] And then looking for which parts glow in the right color, we can figure out where each opsin is being expressed, where the protein is located. - The team is especially interested in genes that are expressed in the retinas of the principal eyes. These are the genes most likely to be related to changes in color vision.

Already, this process of asking which species have expanded color vision and how they accomplish it has led to some surprising discoveries. The researchers already knew that the ability to see and discriminate more colors had evolved more than once among jumping spiders. But they hadn't realized just how widespread this ability would be. After measuring 45 species across the evolutionary tree, the team has already found as many as 12 independent changes in color vision. In evolutionary terms, jumping spiders seem to be evolving new expanded forms of color vision all the time, and different species have acquired their new visual capabilities in very different ways. Take, for example, the ability to see red. Most jumping spiders only have green-sensitive and UV-sensitive photo pigments in their retinas. But some species became sensitive to red when their green-sensitive opsin gene was accidentally duplicated in the genome, and the new copy started to evolve, shifting its sensitivity to longer wavelengths. - So in our eyes, that's exactly what happened. The opsin gene for our green-sensitive visual pigment was duplicated, and the second version evolved to be more red-sensitive, and we see this happen over and over and over again in jumping spiders. - But other jumping spiders see red in a totally different way. Rather than evolving new photo pigments, they added an internal filter to some of their green-sensitive cone cells, which cuts out green light and forces those cells to respond only to longer wavelengths, like red. - They can basically create two kinds of cells from the same type of photoreceptor simply by using a filter in front of some of them and not in front of others.

All this evolutionary innovation makes the original question even more intriguing. Why evolve expanded color vision in the first place? That's the question Lisa Taylor is trying to answer. For a visual predator, like a jumping spider, better color vision could mean finding more prey. It could also mean avoiding prey that might be harmful. - [Lisa] And so, a lot of prey in the environment advertise their toxicity with bright colors, and particularly with long wavelength colors, such as red and orange. So we're testing the idea that the ability to use color vision will help these spiders learn to avoid and continue to avoid red prey that taste bad. - [Derek] In this experiment, all the prey are termites. Some have a dab of red paint on their backs, and others have a dab of gray, - And this doesn't affect their behavior in any way. They still move around naturally, and the spiders really like to eat termites. - [Derek] The red termites also get treated with a compound called Bitrex, which is actually the most bitter substance known. - And yeah, it turns out that the spiders also think it tastes disgusting. So we can simultaneously and independently manipulate color and palatability.

In other words, the researchers can make the termites red and bitter, gray and tasty, or if they want to mess with the spiders, gray and bitter, or red and tasty. The first part of the experiment is the training phase. Basically, the spiders get to choose from a tiny buffet of termite prey, each one in its own little Petri dish. - [Lisa] In three of the Petri dishes, they get a red-painted, bitter-tasting termite, and then the other three Petri dishes, they get a gray-painted, very tasty termite. As they interact with this prey, they are constantly learning, and it's constantly being reinforced that whenever they attack something red, they get a mouthful of bitter-tasting termite, and whenever they attack something gray, they get a mouthful of tasty termite. The first spiders to go through this experiment are Habronattus pyrrithrix, and we started with them because we know that they have good color vision that extends into the long wavelengths. - [Derek] Habronattus pyrrithrix is one of the species that can see red using a red filter in front of some of its green-sensitive cone cells. - [Lisa] Our data so far suggests that the spiders are really good at learning the rules.

And once they learn the rules, then the real experiment begins. The spiders hunt for all their food in a setup just like the termite buffet where they were trained, except that for half of the spiders there's a big difference. Some of the termites are still bitter, but they're all gray. The color cues are gone. Now, the question becomes do the spiders that have color cues available, in other words, the ones for which bitter termites are still red, do they do better? - So our data so far show that they do fare better when they have access to those color cues, they lay eggs sooner, and that they're also heavier at the end of the experiment if they're in the treatment group where they have access to color cues. - One hypothesis for why primates evolved expanded color vision is to distinguish ripe from unripe fruit, or tender new leaves from older, tougher ones, in other words, telling good food apart from bad food, kind of like what these spiders are doing. - Here, we've got this kind of evidence in a jumping spider, and we're gonna repeatedly test that in other jumping spider species that have different forms of color vision.

The team predicts that spiders with expanded color vision will use color cues to their advantage. So they'll do better when color can tell them which prey items taste bad. Species that can't see red won't get any benefit from the warning colors or from the training. If the data support these predictions, these will be some of the first experiments in any species to reveal an evolutionary advantage to seeing and discriminating more colors. But feeding behavior can't be the whole story, because the spiders had some more surprises in store. - For example, there's a genus of jumping spiders in Central America called Mexigonus, where males and only males sport incredibly bright red colors on parts of their body that they use during courtship. - We thought for sure the female has got to be paying attention to red, distinguishing it from other colors. They've got to have red color vision in some special way. - And it turns out that at least by our measurements, they don't have the ability to see red. They just have UV and green-sensitive cells in those principal eye retinas. - I don't mind being proved wrong at all. It usually means something more exciting because it means that, oh my God, there's something cool and new in the world, right? And you've learned something new.

So what's going on here? To help answer that question and maybe understand why some spiders are displaying to one another with colors they can't see, it's time to revisit the jumping spider retina. - Instead of just one retina like we have, they have a stack of translucent retinas right on top of each other. - One thing that we think that this layering does is to correct for a problem that the optics present to the retina. It's called chromatic aberration. - Most optical materials, like these glass prisms, refract or bend short wavelength light, like blue and UV, more strongly than long wavelength light, like red. That's chromatic aberration. Lenses do this, too. In photos taken with vintage camera lenses, you can often see a fringe of color around high contrast edges. The sensor in a camera is a single flat layer of photosites. So getting the different colors of light to focus in the same plane is critical. Modern camera lenses correct for chromatic aberration using complicated optical designs with lots of lens elements. - But the other solution is to put different color-sensitive cells at the right depths behind the lenses so that the colors that they're sensitive to are in proper focus.

That's exactly what jumping spider retinas do, and this gets us one step closer to understanding what red might mean to a spider that can't actually see red. In the jumping spider eye, the cells sensitive to shorter wavelengths are generally closer to the lens, and those sensitive to longer wavelengths are farther away. But most jumping spiders are dichromats. They only have two cone cell types. So why have four layers in their retinas? - [Nathan] Typically, the bottom or farthest away from the lens two tiers, we call those tiers 1 and 2. Those are both typically sensitive just to green light. And with a retina like that, an object in that world might appear in different focus in tier 2 than in tier 1. - [Derek] Researchers in Japan have shown that jumping spiders can actually use this discrepancy in focus to perceive depth and distance in their environment. - But there is a liability with this system. It only works if you're just using one color of light, like green. If you start to mix in other colors of light, for example, red, then the system creates errors. Essentially, colors like red might create this perception of being close or being looming towards the receiver, and that would provide a totally different perceptual experience for the viewer.

So a jumping spider's red coloration might not look red to another jumping spider, but instead create a sort of depth illusion. But why would a male spider benefit from displaying an optical illusion? - One thing about jumping spiders is that females often are quite aggressive towards prospective mates. In fact, they can often eat the male rather than allowing him to mate with them. So these males, when they're dancing for females, are really actually dancing for their lives in many instances. - [Derek] If a female thinks a male is closer than he really is, that could throw off her attack, or maybe confusing the female pays off in other ways. - If she can't quite figure out the male's display, she might stick around paying attention to it for longer, and this might result in better outcomes for the male at the end of courtship. - [Derek] And amorous male spiders might not be the only ones exploiting these depth delusions.

So imagine a red prey item. We might look at it and say, "That's probably toxic, and it's warning birds that it's toxic." But another possibility is that it's red simply to look like it's closer to a jumping spider so that it has a better chance of escaping. We also see small insects with red and blue patterns on them, which would create a really complicated visual illusion that might simply baffle it, and require it a longer period of time before it judges this distance. Even a split second can really matter. - But in this tiny game of cat and mouse, a spider that could see and discriminate red from green would be a lot harder to fool, and this could be another surprising benefit of color vision, one that isn't really about color at all. - And what we really need to ask whether or not this hypothesis is even plausible is really good high-resolution measurements of the distances of things in their eyes, including the retina and the lenses, from live animals. - [Derek] This information you can't just get from preserved specimens on microscope slides, but there is another way. (light music) (air whooshing) By using a particle accelerator called the Advanced Photon Source, the researchers have started to collect high resolution X-ray videos through the spider's exoskeletons. - This has never been done before. It's in X-ray, so we can see through their eyes, and we can see how these eye tubes are moving around. - [Derek] If the spider's retinal movements change the shape or length of their eye tubes, that'll affect what they're capable of perceiving. - It would change how they experience depth. It would change how they experience color. Previously, this information has only been collected from dissections. So we're very excited to get super high-resolution videos of the inside of the spider's head as it's performing complicated visual motions.

Unfortunately, a few months after their initial tests, the Advanced Photon Source shut down for upgrades that'll take over a year to complete. So it looks like we'll have to wait a little longer for some of the answers the team has been looking for. - We know that the retinas can be moved around and that they maybe have between a 50 and 60-degree travel. Not only can they be moved in the horizontal plane, but in the vertical plane, and they can actually be twisted to change the orientation of their field of view. - The question is how do these movements affect what the spiders can focus on, or how they sense depth, or even how they perceive color? It's this connection between focus, depth, and color that makes these spiders so intriguing. - It opens up all sorts of questions about what color is in the first place. (light music)

It's already clear that these spiders have a lot to teach us about color vision, how and why it evolves, and how many forms it can take even within a single group of animals. (gentle music) If our understanding of their visual system is correct, the experience of color for jumping spiders might even be three-dimensional in a way that's totally different from how we see the world. And we haven't even talked about their other senses, like their ability to communicate through vibration. When you think about it, you realize that the universe we humans perceive, even with all our technology, is just a sliver of what's out there. (light music)

If we owe anything to the world, it's to allow the world to be experienced in the fullness of itself. I think this is one of the tragedies of extinction, is the loss of oftentimes a totally unique way of experiencing our world, a way of experiencing our world that we probably couldn't even imagine. - So color, what is it? Is it an intrinsic property of an object, like Aristotle thought, or something that exists only in the mind perceiving it, like Galileo believed? Maybe it's not an either/or question. - My belief is that color is something that emerges through the evolution of the eyes that see the world and the world that the eyes see. Color as a thing emerges through this dance, this evolutionary dance between what can be sensed about the world and those that are sensing it. - It's that dance playing out over millions of generations that created the colorful world we inhabit and shaped the countless ways that we and our fellow life forms experience it. (light music) (graphics beeping) Come to me. Okay, not that far. Ah! (Derek laughing) They're not called jumping spiders for nothing. Yeah, come on.