“Reinforcement” and the origin of species

December 8, 2010 • 8:13 am

The conventional definition of “a species” amongst evolutionary biologists is “a group of organisms whose members interbreed among themselves, but are separated from other groups by genetically-based barriers to gene flow.”  Under this view, the origin of a new species is the origin of those reproductive isolating barriers that keep a population distinct from other groups.

Genetic barriers aren’t thought to arise for the purpose of keeping species distinct.  Rather, they are usually thought to be evolutionary accidents:  geographically isolated populations diverge genetically under natural selection or other evolutionary forces like genetic drift, and that divergence leads to the evolution of genetic barriers (mate discrimination, the sterility of hybrids, ecological differences, etc.) as byproducts of evolutionary change.  For example, populations could adapt to different environments (one dry, one wet, for example), leading to them becoming genetically different.  When these populations meet each other again, this genetic divergence could result in hybrids that don’t develop properly because the parental genomes are sufficiently diverged that they can’t cooperate in building a single individual.

Under some conditions, however, natural selection might directly favor increasing the genetic barriers between newly-forming species. One of these processes is called “reinforcement,” and it works like this.  Suppose two populations have begun to differentiate when they are geographically isolated.  They differentiate to the point that there are some problems with the hybrids: hybrids might be partly sterile, for example, or only partly viable. Because these problems might not completely block gene flow (say, only 50% of the hybrids are sterile), the populations aren’t yet regarded as having become completely different species.

But suppose these isolated populations come back into contact with one another.  Individuals who mate with members of the “wrong” population produce some maladaptive hybrids.  Any individual that could discriminate, and mate only with members of its own population, would leave more copies of its genes than individuals who mate wrongly.

Under these conditions, natural selection could favor the evolution of mate discrimination, promoting those adaptations that allow you to selectively mate only with others of your type.  In this way genetic barriers could arise as the direct object of natural selection, and speciation might be completed.  This process—the evolution of reproductive barriers to prevent maldaptive hybridization between two populations that attained secondary contact—is called reinforcement.

Reinforcement was once a popular idea in evolution, for it gave natural selection a way to finish off the speciation process.  But does it work?  One problem is that if two populations come back together again, and can interbreed to some extent, then the evolution of high genetic barriers will be countered by the fact that hybrids keep forming, driving the populations to fuse at the same time selection “wants” them to separate.  Which will win?  This depends on the balance between selection, hybridization, and also migration of individuals into the “contact zone” from outside.  While there is some evidence of reinforcement in nature—seen in patterns of higher mate discrimination between species in areas where their ranges overlap than elsewhere—there hasn’t been much evidence from the lab that natural selection can increase barriers between “incipient” species that are allowed to hybridize.

In a new paper in Current Biology, my hotshot student Daniel Matute modeled the evolution of reinforcement in two species of Drosophila, and found that reproductive barriers could indeed arise—and arise very quickly—when populations were forced to coexist and hybridize, even if migration were allowed from the outside.

He used two of the groups we work on: Drosophila santomea and D. yakuba, two closely related species that coexist on the African island of São Tomé.  While these are designated as different species, they can still hybridize in the lab, and half of the hybrids (the females) are fertile, so gene exchange is possible between them.  (They also hybridize a bit in the wild where their ranges overlap on the island.)  But because there is a penalty associated with hybridization (half of hybrid offspring—all the males—are sterile), natural selection might be able to increase their genetic isolation if the species were forced to coexist.

Daniel produced this coexistence in the laboratory, forcing the species to cohabit in bottles where they could mate either with their own kind or with the other.  Further, he allowed different amounts of this hybridization by removing different numbers of the hybrids (easily distinguished by their intermediate pigmentation) from the bottles each generation.  Finally, he allowed different amounts of “migration” from the outside by introducing different numbers of flies into the “mixed” bottles.  This corresponds to individuals moving into an area of geographic overlap from the outside, a factor that works to overcome reinforcement.

What he found is that, under many conditions, reinforcement did work: the species became more genetically isolated when forced to coexist.   And this evolutionary change happened quickly: within five to ten generations (a generation in the lab is about two weeks).  Two types of barriers were strengthened:  sexual isolation (the species forced to coexist became less willing to mate with each other) and gametic isolation (females who mated with the “wrong” males evolved the ability to get rid of the foreign sperm more quickly, giving them a chance to mate with the “right” males again).  Predictably, when hybridization was too strong, or migration from the outside too pervasive, these forms of reinforcement did not evolve.  But what is surprising is that under “reasonable” levels of hybridization—meaning conditions likely to be met in wild populations—reinforcement evolved fairly quickly.

The upshot is that these experiments establish reinforcement as a viable process that can “polish off” speciation in the wild.  And I should add that in populations of these species on São Tomé, reproductive isolation is indeed higher between populations taken from areas where they coexist than from areas where the species live separately, so perhaps reinforcement in nature explains this.

Curiously, though, the “reinforcement” seen in the wild applies to gametic isolation but not sexual isolation.  While sexual isolation (mate discrimination) quickly became stronger in forcibly-coexisting lab populations, it’s no stronger in nature in areas where the species coexist than elsewhere.  It’s a mystery to us why both forms of isolation evolve so quickly in the lab but only one is seen in co-occurring populations in nature.


Matute, D. R. 2010. Reinforcement can overcome gene flow during speciation in Drosophila. Curr. Biol. 20:doi:10.1016/j.cub.2010.11.036.

40 thoughts on ““Reinforcement” and the origin of species

  1. It’s a mystery to us why both forms of isolation evolve so quickly in the lab but only one is seen in coexisting populations in nature.

    I smell a dissertation….



    P.S. You’ve got a minor typo (missing space) in the first line…. b&

    1. No, because you would need a non-trivial amount of hybrid sterility or inviability, which does not appear to occur between any pair of geographic populations of humans. There is little or nothing to “reinforce.” One could speculate that racism is in part an extension of a general sort of xenophobia that may be adaptive in a social primate lineage that occur in stable kin groups, and might have evolved to show greater altruistic behavior toward like phenotypes. Hard to say.

        1. I’d say it’s common knowledge that we are all members of the same subspecies, Homo sapiens sapiens.

          So what OTHER reason do you suspect that the same factors that constitute fuzzy traits are also genetic barriers to breeding?

        2. Yes–conception rates are the same. WASP mother, married to a Japanese guy, we had three beautiful children in four years. Absolutely *no* problems conceiving whatsoever–and I didn’t start ’til I was 32. More like a “Honey–you go sleep over *there*” situation… ;-)) You’ll have to wait a few years, though, to see whether or not they’re sterile. Of course, I sincerely hope they’re not- I can’t wait to be a Grandma. Only form of justice in the world ;-))

  2. It occurs to me that while lab populations presumably have plenty of opportunities to mate, and therefore can spurn the advances of the “wrong” suitor while “knowing” a better one will present himself soon, in the wild such an embarrasment of choice may not be present.

    OK, I’m writing long this morning. I’ll try again: in the lab, females evolve strong behavioural mate preference in a situation with many opportunities to exercise that choice. In the wild, females do not evolve strong behavioural mate preference because the opportunities to choose between two competing males, of any species or species combination, is much more rare.

    The population density is orders of magnitude higher in the lab than in the field, right?

    1. It’s not the density per se but the relative numbers of individuals of each type of species, for that determines the probability of encounter. And in the “hybrid zone” in nature, both the density of individuals are high and there are plenty of individuals of both types.

      There certainly was enough opportunity for choice in the wild to allow for the evolution of gametic isolation, presumably by reinforcement, and this should have been sufficient opportunity to also allow the evolution of increased sexual isolation. But it didn’t, and we don’t know why.

      1. It still seems possible that, regardless of relative densities, flies in a natural environment have a more complex set of other stimuli that preclude a fine-tuned level of mate discrimination. I don’t know what would constitute a semi-natural enclosure for a drosophilid, but it would be interesting to see if mating (behavioral) isolation indeed evolves more slowly than gametic isolation in such an environment.

      2. Then what criteria or method do Drosophila typically employ for mate selection?

        If it’s scent, a wild guess, might that be artificially heightened in small confines?

  3. I (in my layperson ignorance) always thought that species had to remain isolated in order to diverge. Live and learn. Thanks.

    1. Darwin discussed this, when he proposed how natural selection leads to “divergence of character”.

      Origin of Species, Chapter 4:

      …it should be remembered that the competition will generally be most severe between those forms which are most nearly related to each other in habits, constitution, and structure. Hence all the intermediate forms between the earlier and later states, that is between the less and more improved state of a species, as well as the original parent-species itself, will generally tend to become extinct. [my emphasis]

      1. I’m still reading that as the parent species will become extinct, also. Not living side by side with the improved version as two separate species. I’ll allow that I could be wrong.

        1. True, he does apply the principle to explain the extinction of the parent species. But when you read that concluding sentence like this:

          “…all the intermediate forms … between the less and more improved state of a species… will generally tend to become extinct”

          He’s saying that natural selection kills off in-between varieties too. That covers the hybrid individuals that Dr. Coyne’s talking about.

          If you look at the context of that quote you’ll see how Darwin’s proposing that a single species would tend to diverge without undergoing isolation — because in any environment greater divergence means more opportunity to find resources to survive.

  4. the “reinforcement” seen in the wild applies to gametic isolation but not sexual isolation.

    How is that known? Someone has done a random survey of the sex habits of these flies in the wild? That doesn’t sound easy.

    BTW the reinforcement concept is new to me. Cool stuff, thanks 🙂

  5. As I understand it, a perfect example of partial sterility of hybrids is the liger, which is the offspring of a male lion and a female tiger. Male ligers are believed to be sterile but female ligers are not and can mate with either a male tiger (producing a ti-liger)or a male lion (producing a li-liger).

  6. What about mechanical barriers to reproduction?

    I admit it has been some time since my undergrad bio and ecology classes, and as my graduate training was in micro and immuno I’m not as up on the current status of the evo community as I perhaps should be; but the definition of species (at least for multicellular sexual species) that I had always been taught included both genetic and mechanical barriers to reproduction.

    True, mechanical barriers may and often do precede genetic barriers but they still have a tendency to block gene flow.

    I just remember being specifically told in both high school and college bio classes that different species didn’t necessarily have to be genetically incompatible.

    Those are both over 10 years ago however and I know times do change.

      1. Yeah something like that. (hilarious mental picture) Though with chihuahuas and mastiffs you can still get gene flow through intermediates. But if say chihuahuas and mastiffs were the only two canis breeds (including eliminating wolves from this fantasy world) would they qualify as separate species? You could produce a mix by artificially inseminating the mastiff (I think the size of the pups might kill a female chihuahua), but without human intervention such a pairing would seem to be nigh impossible.

        Situations where the mommy parts and daddy parts don’t fit together, or perhaps due to shifts in mating periods or behavior (though maybe the latter two don’t quite fit into mechanical).

        Though after reading it over again it says “genetically-based barriers” which in my mind was parsed into “genetic barriers”. I suppose even mechanical barriers would be “genetically-based”, and that it only rules out environmental barriers.

          1. You mean that a species with such varied sizes as we see in the various breeds of domesticated dogs, from toy to huge, and which has the problems of mating from the disparity in sizes, is a ring species? Dogs are a ring species because of mechanical restraints if not genetic restraints? Or am I totally misunderstanding what you said?

  7. This concerns a questions I have had. At what point are two animals considered different species (such as lions and tigers) and when are they just different breeds (such as in mastiffs and chihuahuas)?

    1. Species boundaries are much fuzzier than our Latin names would lead us to believe. The biological species concept places primary emphasis on degree of reproductive isolation, but THERE IS NO ONE “POINT” where two animals are considered different species vs. different breeds, and that is what a gradual speciation process would predict. Even Darwin, with no understanding of the genetics underlying speciation, emphasized that there is no line of demarcation in the near continuum between reproductively isolated species and what he called “well-marked varieties.”

      1. Are there any hybrids that are as successful or more successful than the two parent species, which I suppose would be a new species in its own right? Either way, it would make a good sci-fi story 🙂
        I had gotten the impression that the lines between species was fuzzy. The layman’s idea of a hybrid is that they are all totally sterile (mules), but that is apparently not true.

        1. There are hybrids that form from something called allopolyploidy and become reproductively isolated from both of its “parent” species. I don’t know if you would call the hybrids “more successful” but there are cases of wild sunflowers, for example, where the new hybrid species can occupy certain habitats better than either parent species. This is due to the new gene combinations (by something called transgressive segregation) in the hybrid not found in either parent. But this is a wholly different phenomenon than reinforcement following maladaptive hybridization.

      2. Right–Wilson talks about that, too, in Diversity of Life.

        Step by step, the appearance of even one such mechanism will cleave them into two distinct species. Distinct, that is, if you accept the biological-species concept, which you must do if we are to avoid chaos in general discussions of evolution. By all means let us avoid chaos.

        The biological-species concept as chaos-avoidance-mechanism. :-))

  8. My guess is that gametic isolation is more complex and slower to evolve, while sexual isolation is faster but more expensive (mate discrimination isn’t free).

  9. Excellent and novel (to me) look at reinforcement in a manipulable experimental setting, and with two species that show a pretty straightforward and clear type of maladaptive hybridization. Will use in future class discussions of secondary reinforcement.

  10. In “The Ancestor’s tale”, Dawkins described an interesting version of the sexual isolation in two species (Eastern and Western) of the North American narrowmouth frog.

    Originally one species, some time in the past they became separated, but now overlap again. In the overlap zone, the mating calls differ: one higher pitched than the other (by nearly an octave), one about 2 times longer than the other, etc. But outside the overlap zone, the mating calls of the two species are essentially identical, with characteristics approximately midway between the extremes in the overlap zone.

  11. An uneducated guess re gametic vs. behavioral isolation in lab and in nature: Perhaps gametic isolation is just “cheaper” energetically for the parents in the wild than not having sex?

    I assume that in the wild some cost is connected with finding a mate to mate with. As a fly, you need to buzz around to find your partner. If you avoid having sex with the wrong partner, you’ll need to spend more time on the wing to find the next (hopefully right) one. In a reaction tube, that distance is much smaller.

    In the wild, indiscriminate sex, gametic isolation (removing sperm after a “wrong” tryst), and a renewed flight are sufficient to get isolation, and in fifty percent of meetings (or whatever the mix of species) the match is right, removing the need for another search. (The potential cost would be tied to the composition of the fly population.)

    In the lab, the cost of searching for a new mate is practically nil (at least I assume much lower than in the wild) and perhaps the cost of mating becomes a controlling factor. It’s cheaper to figure out that your potential mate isn’t the right one and move on to the (very close-by) next than to go through courtship, sex, and removing the sperm before picking the next one.

    Could this be tested in the lab? Sorry but I have almost no experience in designing experiments.

  12. I don’t understand why mate discrimination not being stronger in species coexisting in the wild is a mystery. Reinforcement works with gametic isolation because half the hybrids are sterile (thus hybrids are selected against). How many of the hybrids are sterile in the case of mate discrimination? If none: there, problem solved.

  13. Re: your comment, “…the evolution of REPRODUCTIVE BARRIERS TO prevent maladaptive hybridization…” how about ‘the evolution of CHARACTERISTICS WHICH prevent maladaptive hybridization’? “Oh Jerry … don’t be such a selectionist.” I enjoy your postings tremendously, thank you.

Leave a Reply