Correction: Colin told me this: “One minor correction: Theory and Society isn’t a new journal. What’s new is the newly approved ‘Peer Review’ paper category they offer. We had been working with the Springer Nature people for close to 6 months back and forth, and finally they approved it and implemented it on the journal website’s backend so it appears in the drop-down menu when people submit articles.
Luana sent me this tweet, which I’d missed, announcing the founding of a new section of a scientific journal that exists to critique articles in other journals (with the original author given the right of reply). Click on the screenshot if you want to go to the original tweet:
To see the Wall Street Journal op-ed by Kevin McCaffree and Colin explaining the journal, click on the screenshot below—or you can find the article archived here.
Some excerpts:
We’re often told that science is “self-correcting.” But science isn’t like a thermostat regulating your home’s temperature. It’s a human institution run by fallible human beings. Scientists and scholars are susceptible to career incentives, moral fads, groupthink and fear. When those pressures capture journals or entire fields, peer review can become less a filter for error than a credentialing system for fashionable nonsense.
. . . Decades of studies on publication bias, replication failures and political bias in the social sciences have shown that peer-reviewed papers are often less reliable than the public assumes. John Ioannidis’s famous 2005 paper, “Why Most Published Research Findings Are False,” remains disturbing because its basic insight about the fallibility of medical research remains true. In fields that rely heavily on narrative or qualitative methods, or that touch on politicized topics (as much social science does), ideology influences which questions are asked and which conclusions are professionally acceptable.
The authors then mention the Sokal hoax as well as the “Grievance Studies affair” involving submission of bogus papers to social-science and humanities journals by Helen Pluckrose, James Lindsay, and Peter Boghassian. (There were no submissions to STEM journals in either “affair” unless you consider gender studies journals as being in STEM.
A bit more:
This problem is growing more serious. Across swaths of the humanities, social sciences, medicine and biology, some narratives have become taboo. Papers presenting contrary evidence or dissenting viewpoints are rejected without comment. Letters to the editor, which are supposed to provide a quick way to respond to flawed work, are ignored or unavailable. The result is an ideologically biased literature that’s presented as an expert consensus and cited by journalists, courts, school boards, medical associations, government agencies and lawmakers to justify policies that affect millions of people.
The most obvious answer is better peer review. But ideologically captured fields have little incentive to correct themselves. As a result, objections to progressive orthodoxy are relegated to social-media threads, blog posts and newspaper opinion sections.
This is where the myth of “self-correcting” science becomes a problem. People assume the system will fix itself, but first someone has to notice the problem and create a mechanism for correction.
That is what we have done. As an editor-in-chief and a member of the editorial advisory board of Theory and Society, an interdisciplinary journal published by Springer Nature, we are proud to announce a first-of-its-kind article type called “Peer Review.” The purpose is to avoid procedural traps that can prevent legitimate criticism from being published and to recover what peer review was supposed to be: serious, good-faith analysis by experts seeking clarity and truth.
As in postpublication peer review, a Peer Review article may address a paper from any scholarly journal so long as it raises concerns about methods, evidence, logic, definitions or theory. The focus must be on claims, arguments and scholarly standards, not the author’s character or motives.
Submissions, limited to 2,500 words, will undergo a simple merit review rather than endless rounds of gatekeeping. An editor or subject-matter expert will ask a straightforward question: Is this critique coherent, serious and reasonable enough to deserve scholarly attention? If so, it will be accepted.
This is a good idea, and I can easily see myself writing a short response to some pieces that I find deficient. (Some of my website critiques of “sex-is-a-spectrum” posts might have been appropriate.
The only problem is what to do with papers (not just critiques) that try to air subjects that are forbidden or inflammatory. Those might be suitable for The Journal of Controversial Ideas, but I’ve never seen a straight science/data paper there. (Granted, I haven’t looked at every issue.)
Anyway, pass this news along to those who might be interested.
Wow. I feel the same kind of happiness as when https://journalofcontroversialideas.org/ came out under the tutelage of Peter Singer et al a few years ago.
What a great idea, particularly in light of the (now old) replication crisis.
There are currently so few “signals” like price is in a marketplace, in academia.
How can people know where the train has run off the track and landed in a ditch?
Kudos!
D.A.
NYC 🗽
Since the range of topics is to include scholarly articles from the Humanities, Peer Review should be an entire journal.
I think it will be.
About time. Congratulations to Colin Wright and associates.
Call me a skeptic. Another bad idea from the science police. Comments on scientific articles, positive or negative, need to be published in the specialist journals read by experts in the area. Otherwise, what is the point? According to a Google AI overview: “Most major journals (like those from the American Physical Society or Springer Nature) publish structured comments. If you submit a scientific critique, the original authors are usually invited to submit a formal ‘Reply’. Both undergo peer review.” What does a separate review journal add to that? Seems doubtful that everyone across the myriad areas of science will spend time monitoring yet another journal for relevant reviews. And somehow, the editor will manage to find reviewers with requisite expertise across the spectrum of science (and humanities?) and then be able to evaluate the quality of the resulting reviews. As for the targets of critical reviews in an esoteric journal, is responding there the best scientific use of their time and energy as opposed to responding in journals read by their peers? Despite the positive (if misguided?) intentions of the promoters, I suspect that SpringerNature might be more interested in its financial implications than advancing science … another Google AI review: “In full-year 2025, Springer Nature reported a revenue of €1,926.4 million ($2.05 billion USD) and an adjusted operating profit of €543.6 million. This marked a strong underlying revenue growth of 6.2%.”
Some journals will not publish critiques or rebuttals, or will have different standards from this new journal. Remember, all papers in this new journal, and presumably the replies, will be peer-reviewed. I am familiar with at least one recent paper that was so bad that there was a drastic need for a rebuttal. One was written, I reviewed it and approved it, and it has not been published. It would have been published, I think–and maybe will be–in this new journal.
Agreed. Most journals will not publish criticism of articles appearing in that journal. So this sounds like a great idea to me. Even as an author, I would love to see scientific critiques of my work, as long as I have the chance of rebuttal.
It seems that it’s not actually a new journal, it’s a new concept and category of paper. Colin Wright has got agreement from the editors of one journal (“Theory and Society”) to accept such papers. Whether other journals will accept such papers is unclear, though I hope that they do.
I have had the same experience as Jerry. I have seen flawed articles get published because of the reputations of the authors, and I have seen rejection of articles (and peer reviews) critical of some “sacred cow” research programs in biology. A journal and its editors often have vested interests in justifying their decisions about what to publish; public criticism of those decisions can reflect badly on them..
Google AI is naive. Both the American Physical Society (APS) and Springer Nature are woke as hell. Good luck getting them to publish a critique of one of their hopelessly progressive-tainted articles. To understand what really happens when you criticize a published paper, check out the article “Resistance to Critiques in the Academic Literature: An Example from Physics Education Research,” which details the refusal of Physical Review – Physics Education Research, an APS journal, to publish a critique of the ridiculous article, “Observing Whiteness in Introductory Physics: A Case Study,” published in the same journal. The critique ultimately had to be published in European Reviews, not only a generalist journal, but one on a different continent.
When I co-authored a systematic review of experimental psychology papers published in Science that found that every human-subject paper we reviewed showed evidence of p-hacking, the review was desk-rejected by Science. We eventually published (after a fight) in PLOS One, a generalist journal.
The fact is, it is often very difficult to get critiques of published papers published, and the critiques often have to be published in generalist journals, because the specialist fields do not welcome critiques of their work. McCaffree and Wright’s initiative is not “misguided”; it is a necessary corrective to a systemically broken scientific publishing enterprise.
I see that one of the journal creators is Colin Wright who I respect and like a lot. That said, I am not optimistic about this idea meeting its proposed mission. The issue is overwhelmingly in the social sciences and studies areas…areas which are heavily infected with anti-science, post modernist epistemologies such as driven by Foucolt, Derrida, Fanon, Judith Butler, and the like. If scientific method is simply a colonialist, power-driven, socially created process and thus rejected by these people and their flock, we should not expect their writings to be anything other than pure crap…no matter how many post-facto essays or critiques might be written.
Plus springer, in my experience always seemed to be a money grubbing journal publisher, many of its journals not affordable by small schools and dept libraries. Maybe if PLOS would take on a similar task…..?
You kids get off my lawn!
For me, the puzzle is why anybody in academia took seriously the now conventional doctrine that “scientific method is simply a colonialist, power-driven, socially created process”. How did this gain currency, while its intellectual equals homeopathy, manuscript analysis, voodoo, dianetics, and the flat earth were
all banished from the ivory towers? What was its secret sauce? [Oh wait, some of it was mistranslated from French. ]
Even if this was confined to the hard sciences and not the humanities, there will still be plenty of opportunities to publish much-needed critiques. Not just critiquing the low-hanging fruit papers like those that claim that this or that species of animal has multiple sexes. Journals regularly publish peer-reviewed papers where the authors have made a career out of claiming rather bogus things like most of our DNA having function. Or that slime molds are somehow intelligent.
Even if one thinks that most DNA has function — even though it absorbs mutations and deletions without loss of fitness. Or that a slime mold is able to think because it can solve a maze by the shortest route (this too is baloney, most likely), the give and take of having the opportunity to rebut those critiques is what this is all about.
This could be a good idea, but I am amused to see it mentioned in the Opinion section of the WSJ.
I’ve grown to like the WSJ dedicated news and science reporting, but am often unimpressed with the WSJ Opinion section. It often publishes controversial perspectives on climate change, public health, and scientific consensus. The board and opinion contributors have frequently argued that scientific consensus is an oxymoron and that skepticism (i.e. the WSJ Opinion Page’s opinion) should override the prevailing scientific consensus. The Opinion page often conflates political policy disputes with deficiencies in the scientific method. WSJ Opinion pieces historically downplay climate change, favor the Covid pandemic lab leak hypothesis and argue for herd immunity. Around 2020, there was a letter from the WSJ own staff (> 280 journalists, editors and other employees) to their publisher expressing concerns about misinformation in the paper’s opinion section (lack of fact-checking and disregard for evidence), citing pandemic & racial issues.
So, this proposal may be good, but the WSJ is the pot calling the kettle black.
Agree with you about the WSJ editorial board.
I figure they’ll come around on global warming once it becomes impossible to deny or explain what bogus reasons.
There’s “woke” craziness and there’s right wing craziness (in their case, probably influenced by the oil and gas industry).
Peer review, the past ten years, has become very weak in the Humanities. In my field, a small group of us wrote our MA and PhD theses on a specialist archive-driven subject. Now we rarely are asked to peer review anything. Those so-called experts asked to peer review are not experts and don’t know the archives. But they know about lived experience and other lightweight nonsense.
Regarding prevailing ideological biases infecting journals, a classic example from way back when was Garcia and Koelling’s seminal research on flavor-aversion learning. Most journals would not accept their work because the then-prevailing behaviorist paradigm claimed that conditioning could only occur with very short intervals between stimuli, and only after multiple stimulus pairings, whereas in flavor-aversion learning the interval between tasting a given flavor and nausea/vomiting could be many hours and only one trial was needed to establish an aversion to the flavor. (Some of us have even experienced this phenomenon ourselves, perhaps most commonly with alcoholic drinks.)