Aposematic coloration, often called “warning coloration”, is the presence of bright or conspicuous colors or patterns in animals that are toxic, noxious, dangerous, or poisonous to predators. Here’s an example from Wikipedia, the granular poison frog (Oophaga granulifera). Like many dendrobatid frogs, this has a number of poison alkaloids in its skin, and they have been used in Central and South America to tip arrows or darts, which can kill mammals. Any predator that tried to eat one of these would probably be dead, or at least very ill.
My own frog, Atelopus coynei, looks conspicuous too [but see Lou Jost’s comment below], and may be toxic, but I don’t think people know anything about that:
But of course far more animals than amphibians are aposematic. The skunk advertises its toxicity with a pair of conspicuous stripes. Many insects, like ladybugs and some leipidopterans, are also aposematic and toxic, including at least one bird species: see here for a Google image search of aposematic animals.
The colors and patterns, as the name implies, gives their bearers an evolutionary advantage over their presumably camouflaged ancestors, for predators will deliberately avoid the pattern, usually because they’ve learned to recognize and stay away from it because of previous unpleasant experiences. (The avoidance can also be evolved rather than learned, as you’ll see if you think about it. Even if eating one of these kills you, individual predators having less of a propensity to attack the pattern would be favored.) Usually, however, learning is involved.
But to get that advantage, the aposematic species has to be sufficiently numerous to afford predators a chance to learn and then avoid the next aposematic animal. And this creates an evolutionary problem.
We are pretty sure that aposematic species evolved from camouflaged ones. To get the warning coloration started, there have to be mutations in the camouflaged population that produce individuals with bright colors and patterns, at least in incipient form.
And that’s the rub: the first mutant individual stands a higher chance of being attacked and killed than do cryptic individuals. Even if it’s toxic, it may still get killed or injured by being attacked for being a novel, conspicuous creature. So how does the adaptation ever spread through the population from a rare initial state?
Previously, as described in the excellent Nature News & Views summary by Tim Caro below (click to read), we had a few answers:
1.) The trait could evolved by kin selection in gregarious animals. While the first mutant individual might be attacked, it might be part of a group of relatives that share that aposematic mutation. Assuming the predator learns to avoid the pattern after killing or hurting the first individual, it would avoid its similarly-colored kin, and that is a form of kin selection on the color/pattern genes that could make them spread.
2.) The trait could have evolved from a state that was conspicuous but not as conspicuous as the animals above. But this runs into the same problem as #1!
3.) The attacked aposematic mutant could avoid being killed by the predator because it smells or tastes bad, or is injured only slightly. If the predator learns from one experience (and some do), then that individual would henceforth be protected from predation, perhaps giving the mutant color/pattern gene an advantage. This seems somewhat likely, and could be tested by exposing naive predators to aposematic prey.
4.) Predators might avoid novel colors or patterns in general since the hunters have a search image for edible species. As Caro says, there’s some evidence for this, too.
But now, in his summary of the original paper, Caro describes a fifth hypothesis that is described in the Science paper below that. The authors test this interesting hypothesis using phylogenetic data, and it seems to be supported.
Click the original Science paper below to read about the novel hypothesis for the evolution of aposematism. The authors test it in amphibians, but may hold for other creatures as well. You can also find the pdf here , and the reference is at the bottom. 
Again, I’ll try to be brief, but may not succeed. The authors’ hypothesis, which is very clever, is that full aposematic coloration may have evolved, at least in amphibians from an earlier state where it wasn’t clearly visible to predators. This could involve the colors/patterns starting their evolution on the BOTTOM (ventral) side of the animal, which wouldn’t draw attention until the animal was attacked, at which point it could flash its pattern and possibly startle the predator (the predator could also learn from a brief encounter that the prey was toxic). And the bottom-colored state could itself be of two types: small patches on the ventral surface (PV) or a fully colored ventral surface (FV). This is in contrast to an animal that is fully colored all over its body.
Once the predator started learning what the color/pattern means from the animals that had it on their belly, then the color could evolve to cover the animal, making it fully aposematic.
But how do you test this hypothesis? Well, you could see if predators learn to avoid toxic amphibians that had color patches painted on their belly, but there are few amphibians that are toxic and lack aposematic coloration. No, the authors tested their hypothesis by doing phylogenetic reconstruction: they used living species and their known family tree to deduce what the color/pattern of the ancestors were. This kind of reconstruction, which makes sense if you have enough data, is increasingly used to study evolution.
And so Loeffler-Henry et al. did a big reconstruction of the evolutionary history of amphibians, many of whom were aposematically colored. They used 1106 species, putting each in one of five evolutionary categories:
species cryptic (camouflaged; “cry” in photo below)
species PV (ventral side partly aposematic)
species FV (ventral side fully aposematic)
species fully aposematic all over its body (“conspicuous” or “con” in photo below)
species polymorphic (some individuals are aposematic, others not). There aren’t many of these, and I won’t go into why they are supposed to exist.
Here’s a photo from the paper showing four of the five states (a polymorphic species isn’t shown):
And here’s the reconstruction of the phylogeny showing the position in the family tree of each of the five states. Click to enlarge:
There’s a pie diagram at each node of the tree showing the probability that that ancestor had one of the five states scored. I won’t go into the methods for deriving probabilities (in truth, I don’t understand them); but her are the salient points:
1.) Ancestors tend to be cryptic (camouflaged; gray dots), with the possible exception of some salamanders. This comports with the evolutionary view that aposematic coloration was not an ancestral condition but evolved as a defensive adaptation to deter predators.
2.) Full aposematism—the orange state—didn’t appear until later in amphibians, and
3.) . . . it did so generally going through an intermediate state of aposematic coloration on the belly (purple and red species)
4.) The preponderance of purple circles earlier than red ones suggests that the condition of full ventral coloration was preceded in time by the evolution of partial ventral coloration: patches of color that could be flashed but are still less conspicuous to predators than fully belly coloration. This suggestion is supported by statistical analysis of the likelihood of the models, but I’ll skip that.
Now this is an analysis of amphibians, but could apply equally well to other species. In fact, many butterflies that have warning coloration have it on their rear wings, which are covered up when they’re resting. It’s only when they fly, or when a predator startles them, that the aposematic coloration is revealed. Here’s an example: an aposematic butterfly from Ray Cannon’s Nature Notes. It’s the common birdwing (Troides helena), known to be very poisonous since the larvae feed on plants containing toxic aristolochic acids.
And here’s a fully aposematic butterfly:
For a long time the evolution of aposematic coloration posed the problem of what evolutionists call an “adaptive valley”: how do you get from one adaptive state (toxic but camouflaged) to a presumably more adapted state (toxic and brightly colored), when the intermediate evolutionary stage (the first mutant individual) was at a disadvantage: mired in an adaptive valley? This could not occur by natural selecction since selection cannot favor the less adapted (here, “less avoided”) individuals.
The authors propose a solution to this: an adaptive valley wasn’t crossed because the intermediate state—ventral coloration—did confer a selective advantage on the first mutant individuals.
The authors end the paper by suggesting that their scenario could apply to many species; and it well could:
. . . macroevolutionary studies on animal coloration should take into account these underappreciated hidden signals, which are both common and widespread across the animal kingdom, to advance our understanding of the evolution of antipredator defenses. Indeed, many animal taxa such as snakes, fishes, and a variety of arthropods (see fig. S12 for example groups) include species that are cryptic, are aposematic, and have hidden conspicuous signals. We therefore encourage follow-up studies in other taxa to evaluate the generality of the stepping-stone hypothesis as a route to aposematism.
_________________
Loeffler, K., C. Kang, and T. N. Sherratt. 2023. Evolutionary transitions from camouflage to aposematism: Hidden signals play a pivotal role. Science 379:1136-1140. DOI: 10.1126/science.ade5156
Fascinating! Real science tends to be.
Really interesting. It might be worth noting that even today, most of our local (Ecuadorian) amphibians are well camouflaged, but many have hidden markings; fewer have fully colored undersides, and fewer still are brightly colored on top.
Bright hidden colors are common in our herps, whether a species is toxic or not, but these are usually species-specific colors believed to be used in mating. I think that warning colors are exaptations of existing species-recognition cues aimed at the opposite sex. Because they are usually hidden, they incur no loss in fitness. This fits into the authors’ hypothesis, but gives an alternative take on the origin of the colors.
Jerry, in nature your frog lives among leafy bryophytes that match its color very well. At least in its resting poses in habitat, it is quite cryptic. Nevertheless it is in a genus famous for its bright colors and toxicity, so it might be toxic.
Lou,
Lick it and see!
Isn’t that what graduate students are for?
Being (relatively) young, and (relatively) fit after months of frog-hunting through the jungle, they’re probably in a better condition to stand a few days of bidirectional purging than the poor Prof(Emeritus) chained to his desk swamped under administrative duties.
There was an “Attenborough” on frogs on the telly last night, describing the widespread “chitrid fungus” problems of amphibians in the wild. Which prompts me to wonder, on the subject of “My own frog, Atelopus coynei,“, have you considered what investment (time, effort, facilities) would be required to maintain a (breeding) colony – say in the facilities of the Botany Dept’s “hot greenhouse” as a refugium against habitat loss? If, indeed “Jerrybufo” is threatened in the wild. It being named relatively recently, I infer that it’s not the commonest frog in it’s environment.
That is cool and clever! On both the evolving animals and the creativity of the researchers here.
Lepidopterans would offer another way to test this. And an interesting wrinkle is that apparently butterflies evolved from moths. To put ones’ impressions crudely, moths tend to be either entirely cryptic, which would represent the primitive state, or they have bright hind wings that they flash, which would represent how they could have crossed the adaptive valley. Some moths are colorful all over, and that trait could be more derived. The tiger moths (Arctiidae) really stand out here, and they are reliably toxic. But butterflies came from moths, and of course as a group they also tend to be colorful all over.
I imagine the evolution of diurnal activity must be a big factor in the evolution of bright colors, which can’t be seen at night. Many diurnal moths are as colorful as butterflies.
I was wondering about this recently. Thanks.
I didn’t see anything on how these amphibians acquired the enzymatic machinery to make the pigments to begin with.
Were genes or casettes of genes somehow acquired at some point in their evolution, or did various transferases, phosphorylases, and such already extant in non-pigmented ancestors undergo mutations that enabled them to act on substrates that led to pigmented products? Are enough genomes now known to be able to address that?
Such a great piece to read, appreciated.
Interesting stuff! The paper is paywalled so I can’t read the full text — out of interest/curiosity as this is not a subject I’m very familiar with. But two questions come to mind while reading this post, that are related to the fact that there are two traits that are tightly linked in aposematic species, the colour pattern and “toxicity”, and maybe is not so easy to disassociate the two in these analyses:
1) Because it is highly unlikely the two evolved simultaneously, one must have evolved first. But which one? From my uneducated perspective on this subject, I would guess toxicity evolved first. This seems to be a fair assumption if there are toxic cryptic species.
2) How would toxicity fit into the ancestral state reconstruction of Fig 2? For example, are there toxic cryptic species that are sister groups to aposematic species? If there are, predators could have “learned” to avoid the toxic species before colouration evolved, and the bright colouring of aposematic species could just be the result of some kind of “runaway” process, not necessarily constrained by natural selection to avoid predators.
I think the paper talks about this (I’ve already forgotten!) and says that phylogenetic reconstruction strongly suggests that the toxicity evolved first. I think somebody’s already done that analysis. I sent you a pdf of the paper so you can check for yourself.
Very interesting piece!
Suppose that a gene for toxicity evolved first (before aposematic coloration) in a population and that this gene also affected some recognizable physical feature of the organism—morphology, coloration, odor. (This would be a pleiotropic gene.) If a predator could distinguish non-poisonous from poisonous prey by one of those features, toxicity would be selected through predator avoidance even if toxicity weren’t advertised specifically by coloration.
Now suppose that enhanced coloration appears as a trait in an already-toxic animal. Since predators already avoid toxic animals for reasons other than aposematic coloration, a more colorful toxic animal might not be more vulnerable to predation than a less colorful one (so long as predators continue to recognize the traits they avoid). In small populations, such aposematic color might drift to fixation, or it might serve to allow conspecifics to recognize each other and reach fixation by selection. Once fixed, aposematic coloration would offer an even stronger signal that the prey item is toxic and should be avoided. A stronger signal might be more effective against a wider range of predators. The stronger signal would have a selective advantage and would achieve fixation.
In other words, starting with toxicity, a population might be able to add aposematic coloration without having to pass through a non-adaptive valley (or, if non-adaptive, might pass through it in a small population). Mine could be another “just-so” story conceived by an armchair biologist, but I’ll throw it out there for consideration (or savagery). It’s probably not even original, as many biologist have addressed this question in the past.
The toxicity does seem to evolve first, as being colorful but not toxic would definitely be selected against continually. As for your second point, I suppose one could say that if the species is already toxic, a new color wouldn’t turn away the predator, whose avoidance image doesn’t include that new color, and so the mutant would be selected against. This is the supposition that led people to posit these new scenarios.
I’m thinking that the new color—not being part of the predator’s avoidance image—is not even noticed (it’s not part of the predator’s predation image either) and doesn’t elicit an enhanced predatory response. In other words, under this scenario there’s no selection against the new mutant. This would allow it to drift to fixation in a small population or be selected to fixation if the new color is beneficial for other reasons. Once fixed in the population, the new color can then be co-opted to become a aposematic cue to protect against predation.
This may be a naive question, but how did “being toxic to avoid being eaten” evolve in the first place? Presumably, producing the toxins comes with a price, and the individual that is eaten by the predator to teach it a lesson does, by definition, not benefit. Is this a case where kin selection is the relevant mechanism?
Many animals don’t produce the toxins themselves, but get them by eating (sometimes early in life) toxic plants. Even if there is a small price, predator avoidance may more than compensate for that.
Quite a few animals, including amphibians, harbor bacteria in their skin that produce tetrodotoxin (which is scary poisonous). Why bacteria have that, and how the symbiotic relationship got started I don’t know, but there would be simple paths to it.
Presumably, it would require some adaption on the part of the animal so they can tolerate the toxins themselves?
Thanks for the post. For the record, I like your science posts
Very interesting indeed. I love these chicken/egg mysteries that are solved with clever observation and analysis. Thanks for the introduction to phylogenic reconstruction; it seems very complicated.
And just think, this fascinating theory was presumably established without once going into the field with a net. So much to be minded from the massive amount of DNA data being accumulated.
oops – mined
I don’t often comment on science posts, due to not being qualified to do so, but I do appreciate them all, and I’m so grateful to our host for taking the trouble to bring the issues to our attention, and to explain what they’re all about. Thank you.
I love your evolution articles, thank you very much for your effort!
POSTING TO LET YOU KNOW–SCIENCE POSTS ARE THE BOMB!
Errr, is that good or bad?
There is a well-known example of coloration in females employed to lure males to their doom. I am speaking of red-haired women of course.
Interesting and well written. Thank you.
Tour de force.
Finally got to read this. Very interesting!