Defining species: a new but problematic species concept

March 10, 2017 • 9:15 am

A few days ago I was interviewed by , a science writer for the Christian Science Monitor. She wanted to discuss a new paper on speciation in birds, a commentary published in The Auk by Geoffrey E. Hill of Auburn University: “The mitonuclear compatibility species concept” (free download, reference at bottom). She also interviewed several other evolutionary biologists and ornithologists.

Hill’s idea is that speciation in birds proceeds largely through the mitochondria of one isolated population evolving divergently from the nuclear genomes of another population, so when the populations encounter each other after a long period of isolation, the mitochondrial genes of one species are mismatched with some nuclear genes from the other, and the hybrids become either sterile or inviable. That would make them different species if the hybrid problems are severe as gene flow between the populations would be very low. The strongest evidence Hill has for his hypothesis is that for the two bird “species,” the blue-winged warbler and golden-winged warbler have very low divergence in the nuclear genes (0.03% to be exact), but the mitochondria differ much more strongly—3%.  They are considered species because they have different markings and maintain their marking distinctness when they meet.

Here they are:

Blue-winged warbler (Vermivora cyanoptera):

Golden-winged warbler (V. chrysoptera):

Hill’s “mitonuclear compatibility species concept” proposes that when a certain degree of genetic difference between mitochondria of different groups is seen, that is indicative of mitonuclear incompatibility, and the groups should be called different species:

As a result, once populations diverge in coadapted mitonuclear genotypes, the reduced fitness of offspring due to mitonuclear incompatibilities prohibits exchange of mt and N-mt genes and effectively isolates individuals with shared coadapted N and mt genotypes. Given these considerations, I propose that avian species can be objectively diagnosed by uniquely coadapted mt and N genotypes that are incompatible with the coadapted mt and N genotype of any other population. According to this mitonuclear compatibility species concept, mitochondrial genotype is the best current method for diagnosing species.

But he doesn’t say how much difference between mitochondrial DNA would mandate a diagnosis that two populations are different species.

To Hill, the warbler mtDNA divergence suggested that the big divergence of mitochondria played the major role in preventing gene flow between these species. But this is problematic for several reasons.

First, for a deleterious mitochondrial mt/nuclear DNA interaction, the nuclear DNA would have to have diverged as well in some places. We know that there are six regions in these species that have marked divergence in nuclear DNA, and these include the genes for body color (there are also ecological differences). But why couldn’t the speciation of these groups have involved sexual selection, so that they’ve diverged in both male color and female preference, rather than hybrid inviability due to mt/nuclear DNA divergence?

Or maybe ecological differentiation plays a role. One problem with Hill’s theory, pointed out by evolutionists Darren Irwin and Brian Sidlauskas in the Monitor piece, is that it assumes that nuclear/mtDNA incompatibility is the cause of speciation, when it could (if it even exists; see below) might have followed speciation that had already occurred through sexual selection or other processes.

But if that’s the case, why are the mitochondria so diverged compared to the nuclear DNA? I have my own theory on that, which I imparted to Ms. Botkin-Kowacki, but which she didn’t mention in her piece. In birds, females are the heterogametic sex (the sex with unlike sex chromosomes), having ZW females and ZZ males. This is the reverse of the situation in mammals and insects, in which males are heterogametic, with XX chromosomes in females and XY chromosomes in males (Lepidoptera are like birds in this respect.)

We also know, in a phenomenon called “Haldane’s Rule” (after biologist J.B.S. Haldane), that in hybrids between species and populations, if only one of the two sexes is sterile or inviable, it is almost invariably the heterogametic sex. So in species hybrids in birds and butterflies, the females are often sterile or inviable, while in mammals and insects it’s usually the males. I spent much of my career working on this phenomenon, and my work and that of others have suggested some explanations, which I won’t go into here.

If Haldane’s rule applies in these warblers, then the female hybrids could be more sterile or inviable than males. Since females but not males pass on mitochondrial DNA, this would—because female hybrids couldn’t mate with males of either parental species—prevent the mitochondrial DNA from moving between the two species. This wouldn’t apply to nuclear DNA, which could move between species when male hybrids mated with either parental species. But those male hybrids wouldn’t pass on their mitochondrial DNA, which is transmitted only by the female parent. This phenomenon alone would account for the disparity in mitochondrial versus nuclear DNA divergence, without having to invoke any bad interactions between mitochondrial and nuclear genes. It’s simply a phenomenon of genetics–if part of the DNA can’t move between species, then that part will diverge faster. And that would knock down Hill’s strongest evidence. (His evidence for his theory isn’t very strong anyway, though we have seen the phenomenon in some copepods).

There are other problems with Hill’s theory:

  • It’s subjective: how much divergence between mitochondrial DNA of two groups would make them count as different species? While the Biological Species Concept (BSC), which counts species as different if gene flow between them is severely impeded, is subjective in some cases of incomplete gene flow, in many others it’s objective: humans can’t exchange genes with chimps, or Drosophila simulans with D. melanogaster (hybrids are completely sterile or inviable).
  • Hill’s species concept is merely a subset of the BSC: it’s just one of many ways that gene flow can be interrupted, and we now of many species, like ducks, that maintain distinctness through other reproductive barriers not involving mt/nuclear DNA problems.
  • Hill’s species concept isn’t general: we know of many species in other groups for which gene flow is prevented not by hybrid sterility or inviability, but things like differences in ecology, mating preference, or time of mating, or use of different pollinators.
  • As Hill admits, there is no direct evidence in any bird species for hybrid problems being caused by deleterious interactions between the mitochondrial DNA of one species and the nuclear DNA of another. It’s a a purely speculative theory based on observations, like the warbler data, that have other and better explanations.
  • In other groups like mammals and flies, it is the mitochondria that move easily between species while the nuclear DNA is more divergent. This is explained by Haldane’s Rule (in those groups fertile hybrid females can move mtDNA between species), but not by Hill’s idea that mitochondria are genetically incompatible with nuclear DNA.

As I told Ms. Botkin-Kowacki, I thought Hill’s theory was somewhat interesting, but was surprised that she was writing an article on it since it wasn’t that earth-shaking. She replied that the idea of “species” is always being revised, and now there was yet another species concept.  I then told her why I thought the BSC was the most useful one in understanding the “species problem” that intrigued evolutionary biologists during the Modern Synthesis, and my explanation is below:

Jerry Coyne, a biologist at the University of Chicago and co-author of the book “Speciation,” agrees that Hill’s hypothesis could only be one aspect of what is going on as part of the classic biological species concept. “He hasn’t established that this is a better criterion for a species concept than the one that is traditionally used,” Dr. Coyne says in a phone interview with the Monitor. “It always comes down to reproductive isolation.”

“If you ask why nature is lumpy,” he says referring to the groupings that scientists call species, “you can hardly arrive at any other conclusion other than that the things that would make these lumps a continuum instead of a lumpiness are reproductive barriers.”

Trying to find a one-size-fits-all species concept might not be the best approach for biologists, Irwin says. “It may be that different concepts work better in different groups of animals or plants and it may be that different processes are sort of occurring in different cases,” he says. “There may not be a perfect species concept.”

I don’t think there’s one species concept that covers both asexual and sexual groups (you can’t have reproductive isolation in a group that doesn’t reproduce!), but that the BSC has proven the most useful one in understanding nature’s discontinuity. I also told her that every paper of which I’m aware that discusses the process of speciation uses the BSC, so biologists implicitly realize that reproductive isolation is crucial in maintaining the distinct groups in nature we call species.


Hill, G. E. 2007. The mitonuclear compatibility species concept. The Auk 134: 393-409. doi:

45 thoughts on “Defining species: a new but problematic species concept

  1. Fascinating analysis. Incompatibility or speciation; which came first – a chicken and egg problem. 🙂

    1. I was once told by a baptist that there were no chicken egg problems, because god made animals, he didn’t make eggs. So that, apparently, was that.

      1. Your – is a baptist a sort of hairdresser? Sprays water about to little effect? – thing is wrong about chicken-egg problems : the question is valid, and very clearly resolved in the egg’s favour. The amniotic egg with a waterproof shell and membranes predates modern chickens – and indeed the rest of the dinosaurs – by something like 50 million years.
        With a moderately fine pen, you should be able to write that on the business end of a shillelagh, all the better for getting the idea past the skull bones encrusted with religion. Don’t forget to sterilise your shillelagh before education!

  2. This seems to be a case of finding one magic bit of genome, preferably one that behaves oxotically, and attributing all speciation to it. The late Lynn Margulis used to argue that all speciation involved new endosymbioses. Without producing evidence that most routine cases of speciation involved any new endosymbiosis events.

    It’s startling, it’s dramatic, good fodder for an uncritical popular science press. It’s also implausible, as you note.

    1. The New York Times asked me to review Magulis’s book on speciation (with her son) that proposed the endosymbiont hypothesis. Much as I liked reviewing for the NYT, the book was so absolutely dreadful that I couldn’t bring myself to write about it, and so I declined.

  3. It’s an interesting idea and I could sort of see how this might work, but then you are speculating that the key speciation changes are in genes that are involved at the interface between mitochondria and their hosts. Not my field but I’ve read a little on this. Over their history mitochondria have farmed out a lot of the functions that, as free-living organisms, they performed for themselves to their hosts – if the host makes the protein you need and can provide it to you then it makes energetic sense to lose that gene from the mitochondrial genome (essentially that’s why we need vitamins in our diet and why these requirements are different between species). One would think that to support this there would have to be some data showing that the changes that occur are specifically in these interacting regions.

    At this point I figured I’d better glance at the paper, no time to read properly right now, but they cover the comments above. However they don’t provide the sequencing and functional details that I’d like to see to really support the hypothesis “Unfortunately, there have been no direct tests for
    mitonuclear compatibility between avian species”. I guess this becomes a complex, but achievable exercise in functional genomics. If anyone can come up with the money and enthusiasm.

  4. The problem that really stood out for me was declaring a whole new species concept, at least for birds, based on just one putative example. That seems a bit premature.

    1. That seems a bit premature.

      I’m trying to remember the Shakespearean line about either Caesar or Brutus being (something messy).
      Sorry, MacBeth, Act 5 Scene 8, Line 15, “Tell thee, MacDuff was from his mother’s womb Untimely ripped.”

      1. It’s interesting that you originally thought it was Caesar, since the usual interpretation of the line is that MacDuff was born by caesarean section.

        1. Yeah. Mixing memories. I was listening to the news with the wife on our way to the swimming pool earlier, and the term “mogul” came up in respect of some USian politician, which she needed explaining. I haven’t checked this, but I thought it was probably related to “Mongol” as in Genghis Khan and friend, maybe via “Mughal” in N.India. Then I tried drawing an analogy to “Caesar”, and “Kaiser” … and brain-faded on the other popular member of that clade. Which was odd as the wife is Russian, and I really should have cottoned on to “Tsar”.

  5. I agree with those skeptical of the “The mitonuclear compatibility species concept”.

    I don’t know much about bird genomes so I refer to humans.

    In humans, the mitochondreal genome is only about 16,000 base pairs. I have a file of my full mtDNA, which I have traced to western Ireland.

    Using a bioinformatics program, I compared this file to that of a Yoruba (Nigerian) woman. The two genomes are almost identical except in places where the DNA is not part of a gene (non-coding DNA.)

    With so few base pairs and so few genes most mutations in the mitochondrial genome would be lethal, unless they occur in the part of the genome that does not code for genes.

    And since those mutations would not be expressed (because not part of genes) it is not plausible that there would be any interaction with nuclear genes.

    1. With so few base pairs and so few genes most mutations in the mitochondrial genome would be lethal, unless they occur in the part of the genome that does not code for genes.

      That is a hypothesis that is amenable to a literature search – or whatever the “bioinformatics” equivalent is – to see what proportion of mutations (SNIPs? For simplicity?) in mtDNA are lethal (or cause serious disease) compared to mutations in nuclear DNA.
      P>I get the impression that mitochondrial diseases tend to be relatively serious compared to the number of nuclear mutations which need to gang up en masse to cause for example, a cancer. But that’s pretty weak even by the undemanding standards of anecdotal evidence.

  6. Maybe this is well-known to everybody else, but the bit that jumped out at me was the almost throwaway remark “it is the mitochondria that move easily between species”. Never really thought about mitochondrial migration, but now I’m going to have to find out more…..

    1. It’s one of the elephant footprints in the butter of the “three-parent embryo” proposals. Just how well can mitochondria be transferred between bloodlines, even within one species.

  7. Interesting analysis. I am ashamed to say that I did not know that female birds are the digametic sex. Love learning new stuff.

    Could the molecular clock determine whether the mitochondrial DNA divergence is due to drift resulting from sterile female hybrids?

  8. Great science post – I do find it amazing the way sex determination is different between different species. XX = Female, XY = Male vs. ZW = Female, ZZ = males.

    It was great to learn that in these cases mitochondria is still passed on by the female.

    Is this down to our definition of female = egg producer and the mitochondria is passed on via the egg? While male = sperm producer without mitochondria?

    Are there any species where the male transfers mitochondrial dna?

    It is these strange exceptions to the norm, which help me understand more about evolution and reading about Haldane’s Rule and how it might explain the results in this case was a brilliant learning moment. Thanks.

    1. yes, that’s how it works. the mitochondria the offspring has are the ones in the egg cell. I’m not aware of any male transferred mitochondria and i’d say it’s unlikely they exist.

      1. The sperm cell has a few dozens of mitochondria (at best) vs. many thousands in the egg (at worst), so sperm mitochondria would make little change even if they survived. However, there are reports that after egg penetration, sperm mitochondria are encircled by egg lysosomes and other membrane organelles, included in digestive vacuoles and destroyed. No idea how such a mechanism has evolved, but this is the right thing to do, because DNA of sperm mitochondria is surely damaged by the hard work they have done. Poor mitochondria.

    2. There is at least one species with regular paternal transmission of mtDNA, the Atlantic mussel Mytilus edulis, first described by Hoeh et al. in 1991.

  9. Non-expert alert, beware of ignorance.

    It really does seem improbable that anything other than reproductive barriers could be the cause of speciation in sexually reproducing organisms. The ways that reproductive barriers can arise are surely myriad and can be subtle and difficult to suss out. But it all comes down to gene frequencies in populations and if genes can’t move between populations then changes in one sub-population can not be distributed into the other sub-population to the same degree, or at all.

  10. Interesting use of words by Jerry, highlighting where our ideas come from:

    “I don’t think there’s one species concept that covers both asexual and sexual groups (you can’t have reproductive isolation in a group that doesn’t reproduce!)”

    The term “reproduction” was first used in the 1730s to describe asexual species, or rather, parthenogenetic, asexual reproduction in aphids (see this morning’s Readers’ photographs!).

    Previously everyone talked about ‘generation’, which was both how eggs turned into fully-fledged organisms and how the eggs got there in the first place. This term has now disappeared, apart from when we talk about “spontaneous generation”.

    Reproduction initially meant literally that – copying, or reproducing a female without the involvement of a male, as in aphids. If you think about it, you do not literally reproduce yourself when you have a child!

    Eventually, the term got used to describe the way that organisms… reproduce (there aren’t many synonyms, even for an inaccurate word!), whether they are sexual or not.

    So, if you were speaking to someone in the first part of the 18th century, the ‘group that didn’t reproduce’ would be the sexually-reproducing organisms!

    Here endeth today’s history lesson.

    – Matthew Cobb

    1. And you restrained yourself from plugging “The Egg and Sperm Race“!
      Actually, I thought of the distinction between “reproducing” and “breeding” too. I know Jerry (and the regulars here) get the difference, but I’ve got tripped up over the difference before when trying to explain biology to the interested but uninformed. It make me think of this particular cave. Ummm, run carefully!

  11. Nick Lane discussed mitonuclear incompatibility in “The Vital Question”. I don’t remember any specifics (rarely do), but he made a good argument that it could be important in some groups and could be one, amongst many, mechanisms contributing to speciation.

  12. Ever since I found out in college Biology that Species wasn’t a hard and fast concept, I’ve been fascinated by it. Thanks.

  13. Am I correct in saying that the other reason why base-pair differences don’t help is that in one pair of organisms the difference would be X and be one species and yet others X would be enough for species difference?

    1. If you mean that different environments would affect a certain genome difference (X number of SNPs, say) differently, that is not clear to me. When you want to test a BSC species you would have to introduce population members in the same environment.

  14. “Hill’s species concept is merely a subset of the BSC: it’s just one of many ways that gene flow can be interrupted, and we now of many species, like ducks, that maintain distinctness through other reproductive barriers not involving mt/nuclear DNA problems.” That’s what I was going to say.

  15. Two articles on species this week. Yes! I have nothing intelligent to add, but I am fascinated by speciation. I read both through twice and understood more than I usually do. Now I’m off to comment about the silly lady who won’t read male authors and who benefits the world by making the rest of us look well educated and well informed

  16. Somewhat off-topic, but if in birds it’s the females that are heterogametic, and in mammals it’s the males, how did this state of affairs come about? How was sex determined in the common ancestor of birds and mammals? Did something happen to flip the sense of the sex-determining chromosomes in one lineage or the other? Or was heterogameticity independently invented in different lineages?

    1. This was exactly my question.

      I did a quick Google search and came up with this interesting article related to the topic: “Sex Determination” by Gamble and Zarkower in Current Biology:

      The article is free, and among the interesting tidbits is the fact that the Japanese frog Rana rugosa has different populations that are either XX/XY or ZZ/ZW!

      For most clades sex determination is fairly hardwired, but not all!

  17. Genetic distance can be a heuristic for community genomics analyses, but as a serious species concept it will never work. Surely there is incomplete lineage sorting even in animal mitochondrial lineages? Wish people would give up on that idea, and while they are at it, perhaps also give up on the idea that there will ever be one species concept for everything.

    The reference at the end of the OP should probably say 2017.

  18. Thanks for an excellent analysis! There was some material on the latest mitochondrial results in a recent course that discussed incompatibilities, I now have reason to go back to it.

  19. I’m also cautious that this concept might not be general enough and it might be too soon. But I’m convinced that mitonuclear co-evolution is more relevant for divergence processes that sometimes granted.

    In this preprint we show clear evidence for both the mitonuclear compatibility idea and Haldane’s Rule.

    We show that mitonuclear interactions maintain a deep mitochondrial divergence in the face of nuclear gene flow between two lineages of the songbird Eastern Yellow Robin occupying contrasting climatic habitats. We found a replicated pattern of low genome-wide differentiation (while mtDNA differentiation is 6%!!). This contrast with two prominent regions of high nDNA differentiation (genomic islands of divergence). The largest island of divergence (~15.4 Mb) showed a significant excess of nuclear-encoded genes with mitochondrial functions (N-mt genes), low genetic diversity and high levels of linkage disequilibrium. These findings resemble at a great extent previous findings of non-neutral mitochondrial evolution in the EYR.

    The second island of divergence mapped to the sex Z-chromosome, suggesting that nuclear gene flow occurs primarily via male hybrids, in accordance with Haldane’s Rule.

    Besides mtDNA and these genomic islands containing N-mt genes, the two EYR mitogroups are visually identical (we are working on this at the moment). Granted we do not talk about speciation (yet) and we are unclear if divergence occurred during gene flow and before. But, for us, the evidence of mitonuclear interactions/incompatibilities is very strong.

    Happy to hear your comments!

Leave a Reply