The two articles I want to discuss today are fascinating, for they raise a problem that’s now vexing many scientists (especially physicists)—the problem of testability. (Thanks to reader Mark H. for calling my attention to them.)

It all goes back to the philosopher Karl Popper (1902-1994). Popper’s views about what made a theory “scientific” were immensely influential. They’re summed up in the Wikipedia piece on him:

A theory in the empirical sciences can never be proven, but it can be falsified, meaning that it can and should be scrutinized by decisive experiments. If the outcome of an experiment contradicts the theory, one should refrain from ad hoc manoeuvres that evade the contradiction merely by making it less falsifiable.

In other words, a theory that can’t in principle be shown to be wrong isn’t a scientific theory. But I disagree with that characterization, and the one from Wikipedia, in two ways.  First, a theory might appear at present to not be “falsifiable”, but it can still be considered “scientific” in that it explains a variety of phenomena and, more important, someday we might find a way to test it. Without supercolliders, we had no way to test the prediction of the existence of the Higgs boson, for existence. When we managed to build that machine, we could look for its existence, and we found it. That, more or less, is the current position of string theory and multiverse theory in physics. They are elegant, can explain some phenomena (the unification of the four forces for the former; the vaunted “fine tuning” of the constants of physics in our universe for the latter),  but neither can yet be tested. But I still see them as scientific theories.

Second, a theory can be “patched up” and still retain its integrity, so some ad hoc-ing is acceptable. For example, we now know that some environmentally induced changes in the DNA can be inherited for one or a few generations, which appears to contradict an important tenet of neo-Darwinism. But all we need to do is realize that these changes are temporary and haven’t contributed to organismal adaptation in the long term. So “epigenetics” in this sense simply broadens a theory that initially maintained that no environmental modification could be passed on. But it doesn’t destroy that theory. It’s only when the patches on a theory become seriously pervasive that the theory must be abandoned. That is what happened to cold fusion, or the theory that vaccines caused autism. Both of those went away because they were falsified by data.

The way I construe falsifiability is like this: A theory for which there are no conceivable observations that could show it to be wrong is not a theory in which you can place much confidence (i.e., regard it as provisionally “true”). 

I know I’m treading on dangerous ground here, as philosophers will circle that sentence like orcas around a wounded seal, but in fact that is the way that science really works. String theory is elegant, a lot of people are working on it, and someday we may find a way to “test” it, but for the present it’s not regarded as a “true” theory—not in the way that the Standard Model of physics is, a theory that has predictions that could be have been falsified, but haven’t been.

Nevertheless, there’s a lot of Popper-dissing going around. I’ve already taken issue with two of his points, but I firmly adhere to the way I’ve construed falsifiability (in italics) above. Scientists still behave as if they don’t accept something as “true” unless it’s passed tests that could have shown it to be false.

As expected, most of the Popper-dissing comes from physics, which now has theories that are so hard to test—as they involve things occurring on scales too small to observe—like the “strings” of string theory—that they are resorting to other ways to confirm such theories. One is “beauty”: a theory which is explanatory and beautiful, like string theory, can provisionally be seen to be correct. I disagree. Einstein’s general theory of relativity was a beautiful theory, and Einstein seemed to regard it as correct on that count alone, but the physics community didn’t take it as true until it made predictions that could be tested, and were verified. These included the bending of light by celestial bodies (as in Eddington’s experiment) and the precise quantification of the advance of Mercury’s perihelion. Since then, other tests have amply confirmed both the general and special theories.

Other ways to “confirm” theories include Bayesian approaches that are said to give greater or lesser confirmation to a theory. I’m not familiar with how Bayes’ Theorem is used in this way, but I know that many physicists think that this isn’t a proper way to test something like string theory.

In 2014, a mathematician and physicist, George Ellis and Joe Silk, went after the increasing tendency of physicists to validate theories without any empirical observation. In their article, “Defend the integrity of physics,” they said this, concentrating on string theory and multiverse theory (all bolding in this post is mine):

This year, debates in physics circles took a worrying turn. Faced with difficulties in applying fundamental theories to the observed Universe, some researchers called for a change in how theoretical physics is done. They began to argue — explicitly — that if a theory is sufficiently elegant and explanatory, it need not be tested experimentally, breaking with centuries of philosophical tradition of defining scientific knowledge as empirical. We disagree. As the philosopher of science Karl Popper argued: a theory must be falsifiable to be scientific.

Chief among the ‘elegance will suffice’ advocates are some string theorists. Because string theory is supposedly the ‘only game in town’ capable of unifying the four fundamental forces, they believe that it must contain a grain of truth even though it relies on extra dimensions that we can never observe. Some cosmologists, too, are seeking to abandon experimental verification of grand hypotheses that invoke imperceptible domains such as the kaleidoscopic multiverse (comprising myriad universes), the ‘many worlds’ version of quantum reality (in which observations spawn parallel branches of reality) and pre-Big Bang concepts.

These unprovable hypotheses are quite different from those that relate directly to the real world and that are testable through observations — such as the standard model of particle physics and the existence of dark matter and dark energy. As we see it, theoretical physics risks becoming a no-man’s-land between mathematics, physics and philosophy that does not truly meet the requirements of any.

Ellis and Silk name Richard Dawid and our Official Website Physicist™ Sean Carroll among those asking to weaken the “testability” criterion for theories of physics. Ellis and Silk’s piece, which is easily readable by non-specialists, ends like this:

What to do about it? Physicists, philosophers and other scientists should hammer out a new narrative for the scientific method that can deal with the scope of modern physics. In our view, the issue boils down to clarifying one question: what potential observational or experimental evidence is there that would persuade you that the theory is wrong and lead you to abandoning it? If there is none, it is not a scientific theory.

Such a case must be made in formal philosophical terms. A conference should be convened next year to take the first steps. People from both sides of the testability debate must be involved.

In the meantime, journal editors and publishers could assign speculative work to other research categories — such as mathematical rather than physical cosmology — according to its potential testability. And the domination of some physics departments and institutes by such activities could be rethought.

The imprimatur of science should be awarded only to a theory that is testable. Only then can we defend science from attack.

I’m in nearly complete agreement with this, except that I’d still award the imprimatur of “science” to string and multiverse theories. They are scientific theories—just not ones that we can have any confidence in. They’re not even close to being as “true” as, say, Einstein’s theories of relativity or the theory of evolution.

At any rate, the conference called for by Ellis and Silk has taken place this month—at Ludwig Maximilian University in Munich. It’s described in Natalie Wolchover’s article in Quanta, “A fight for the soul of science“:

Once again there was the customary Popper-bashing:

But, as many in Munich were surprised to learn, falsificationism is no longer the reigning philosophy of science. Massimo Pigliucci, a philosopher at the Graduate Center of the City University of New York, pointed out that falsifiability is woefully inadequate as a separator of science and nonscience, as Popper himself recognized. Astrology, for instance, is falsifiable — indeed, it has been falsified ad nauseam — and yet it isn’t science. Physicists’ preoccupation with Popper “is really something that needs to stop,” Pigliucci said. “We need to talk about current philosophy of science. We don’t talk about something that was current 50 years ago.”

I have to disagree with Pigliucci here; I haven’t yet seen a theory accepted as true that hasn’t survived a test that could show it to be wrong. And yes, I do think that at one time astrology was a scientific theory: a theory claiming that one’s personality could be affected by one’s time of birth, connected with the configuration of stars and planets at that time. Astrology, like creationism, was once a scientific theory, but now it’s a falsified scientific theory, and can’t be accepted as true. It isn’t science any more, but it once was. To say that the falsification of astrology completely refutes the value of falsificationism or Popperianism seems to me a circular argument. At any rate, I will agree with Massimo that if a reasonable theory can’t yet be falsified, it can still be seen as “scientific”; but as the years pass and one can’t find a way to test that theory, it eventually passes into the hinterlands of “nonscientific.” What Pigliucci appears to be doing here (and I’ll grant that I haven’t seen his paper) is conflating what we regard as “scientific theories” with what regard as “true scientific theories.”

I won’t recount the to-and-fros that occurred at this meeting, as you can read Wolchover’s piece for yourself, but it does give three alternatives to falsification, at least for string theory. Here are the first two: 1. no other theory that explains “everything” has yet been found; and 2. string theory came from the Standard Model, which for a long time itself had no alternatives, buttressing the possibility that a “no-alternatives” theory could be right on that ground alone.

I find neither of these arguments convincing, if for no other reason than a good alternative theory might some day surface. How can we have so much hubris that we think that no human will ever devise an alternative to string theory? And, of course, by now the Standard Model has been tested many times, and passed the tests. It wasn’t taken to be true until these confirmations. Why, then, do we behave differently with string theory? Only because it’s much harder to test.

Finally, string theory is said to have provided explanations for previously inexplicable phenomena, like the entropy of black holes. To me that is something worth considering, but not enough to confirm the theory. Explaining phenomena is one way to validate a theory (after all, that’s what Darwin did when using evolution to explain the peculiar distribution of species on oceanic islands), but you must be able to devise tests that could show a theory is wrong before you accept it as correct. To disprove neo-Darwinism, for instance, you might find a lack of genetic variation in species, fossils in the wrong places, one species with adaptations that increase the fitness only of a second species and not itself, and so on. These are potential falsifiers, but none of them have been seen. We have no similar falsifiers for string theory.

As for Bayesian ways of getting greater confidence in theories, I haven’t read anything about them, and so can’t weigh in here, but I have to say that I’m dubious.

If you have any interest in the history and philosophy of science, I’d recommend reading both the Nature and Quanta articles, as the debate is not only fascinating, but goes to the very heart of science: how we decide what is provisionally true, and how much confidence to apportion to our beliefs. I would add that we should be wary of those like Pigliucci who, on weak grounds, claim that “Popperism is dead.” It’s telling that the Quanta piece describes the outcome of the Munich conference like this (my emphasis):

The Munich proceedings will be compiled and published, probably as a book, in 2017. As for what was accomplished, one important outcome, according to Ellis, was an acknowledgment by participating string theorists that the theory is not “confirmed” in the sense of being verified. “David Gross made his position clear: Dawid’s criteria are good for justifying working on the theory, not for saying the theory is validated in a non-empirical way,” Ellis wrote in an email. “That seems to me a good position — and explicitly stating that is progress.”

In considering how theorists should proceed, many attendees expressed the view that work on string theory and other as-yet-untestable ideas should continue. “Keep speculating,” Achinstein [Peter Achenstein, a historian and philosopher of science] wrote in an email after the workshop, but “give your motivation for speculating, give your explanations, but admit that they are only possible explanations.”

“Maybe someday things will change,” Achinstein added, “and the speculations will become testable; and maybe not, maybe never.” We may never know for sure the way the universe works at all distances and all times, “but perhaps you can narrow the live possibilities to just a few,” he said. “I think that would be some progress.”

What I’ve put in bold is a tacit admission that string theory has not been verified in the way that the Standard Model, or the theory of evolution, has. Yes, string theory is still a scientific theory, and maybe we’ll find a way to test it. I certainly don’t think physicists should stop working on it just because they haven’t yet found a way to falsify it. That would be premature. But until they do find a way to test it, I don’t see it as a scientific theory in which we should place a lot of confidence; i.e., I don’t see it as “true.” Neither, apparently, do the participants in that conference! Karl Popper’s ideas are regularly declared dead, but they refuse to lie down.

I feel the pain of these physicists, for their theories are now so elegant and abstruse that it may be impossible to test them properly. They involve strings that are impossibly tiny, and theories with so many parameters that the notion of testability is elusive. In other words, their success has painted them into a corner. By changing the rules of science, they’re trying to make a virtue of necessity. But the way out of their corner isn’t to change the rules—the way we establish things as true. If physicists can’t find a way to falsify their theories, in that corner they shall stay. After all, every scientist admits that there are some things about the Universe that we simply will never know.