Over at the Guardian, Ananyo Bhattacharya, the chief online editor of Nature, answers some common criticisms that scientists have of science journalism. His piece, called “Nine ways scientists demonstrate that they don’t understand journalism,” is pretty tame, though, and I think a lot of us would agree that science journalists must write their stories using certain conventions. Bhattacharya defends the following conventions that, he says, are criticized by scientists (go to his piece to see some others):
- Starting a story with the important results
- Using limited space because of readers’ limited attention spans
- Using headlines that will draw attention to the study
- Quoting scientists who disagree with the highlighted research
The story didn’t contain this or that “essential” caveat.
Was the caveat really essential to someone’s understanding of the story? Are you sure? In my experience, it’s rare that it is. Research papers contain all the caveats that are essential for a complete understanding of the science. They are also seldom read. Even by scientists.
Yes, journalists don’t need to put in every caveat that we’re required to add in the discussion, but some of them are important. Take the use of limited sample sizes to demonstrate the existence of “gay genes” or “depression genes” for example, or the fact that early reports of these genes (later found to be bogus) were limited to single lineages, or used associated markers that were reported by the press to be the genes themselves. These are important problems, not trivial caveats. And the caveats weren’t seen in most of the breathless news stories about “genes for gayness” of “genes for depression.”
Second, highlighting potential problems brings home to the reader that science is an ongoing enterprise, that no study is perfect, and, most important, all scientific truths are provisional. Too many journalists accepted the “arsenic bacteria” story, or the existence of the Darwinius masillae fossil as a missing link between the two major groups of primates. A finding can be wrong, or can be revised.
Why aren’t such caveats, or such dissent, presented more often? Well, yes, they could bog down a story, but often I think that journalists aren’t sufficiently trained in science to recognize when a problem is serious. Also, though Bhattacharya rightly emphasizes the need for science journalists to summon dissenting voices in their stories, many journalists are either too lazy to do this or don’t know who to call. There are some notable counterexamples. Carl Zimmer does a good job of this at The New York Times, and Faye Flam at The Philadelphia Inquirer. When reporting a new discovery, scientists should routinely search for dissent, and should know enough to determine whether that dissent is significant.
So my main complaint about science journalists is fourfold. First, they often aren’t trained sufficiently to write about science in a meaningful way. It would be nice if the journalist had a degree in the subject described, preferably an advanced degree. A journalist should be able to read the paper under consideration and understand it well.
Second, lazy science journlists often just reproduce press releases produced by universities instead of reading a paper and dissecting it themselves. Press releases are not journalism, but puffery.
Third, science journalists are often too lazy to do a proper job of vetting a story (this is related to the preceding beef).
Fourth, journalists often don’t seek out dissent, or make do with a token and meaningless dissent.
Readers: what are your complaints about science journalism? Who, in particular, doing you think is doing a really good job or a really crappy job?
h/t: Dom
A lot of the complete rubbish in science stories I see in the popular press was present in the original university press release. What can be done about these?
I agree. And the only remedy, it seems to me, would be to have the science faculty themselves complain/work with the publicity office to correct what JAC rightly calls puffery.
Probably a hard slog, though; the PR people no doubt have stats on just how much alumni donations increase for every time the old alma mater is mentioned in the news…
My experience in the UK is that those journalists who specialise in science reporting tend to do a good job. The problems come when the story is written by a standard news journalist who will not have the experience to look at things such as sample size and add an caveat if appropriate.
I don’t think either side can be said to be doing a particularly bad or particularly good job – there are good and bad journalists, good and bad scientists, good and bad science journalists in fact.
I saw this on Not Exactly Rocket Science the other day “Every scientists-vs-journalists debate ever” in one diagram (although I think perhaps “vice versa” should be added in the title, and as a commenter points out as well, wouldn’t necessarily agree with only bad scientists / journalists ending up in the pub):
http://blogs.discovermagazine.com/notrocketscience/2012/01/17/every-scientist-versus-journalist-debate-ever-in-one-diagram/
One problem is that the journalist is often not a science journalist as such at all – hence reliance on the university press release, and Wikipedia for background if they’re particularly diligent. (Another reason for roving gangs of scientists to fix up the Wikipedia articles on their subject areas. Yes, I know there are a lot of idiots on Wikipedia. [God, do I know it.])
They especially tend not to be science journalists when the story can make a sensationalist headline.
Here in UK you often find a sensationalist story written by a news journalist on the front page, with a more considered piece deeper inside written by a specialist science journalist.
We need a few more public sceptics like Ben Goldacre. Cute, friendly, endearing and merciless.
My take: science is hard and understanding the result requires nuanced reasoning. Example: when some study produces a result such as “we found a statistically significant correlation (p = .04) between consumption of coffee and the rate of skin rash (r = .78), N = 25” few know what that means; instead we see headlines such as “coffee is bad for your skin” or even “coffee is bad for you”.
The scientist/journalist rift is like a clash of cultures, as I say here:
http://www.yaleclimatemediaforum.org/2012/01/the-rift-between-journalism-and-science/
As many people have noted, the main problems seem to be
many/most science journalists are non-specialist
many/most science journalists don’t have enough time to do a “proper” job
many/most editors think their readers are dumb
In case you’ve not heard of it, the best discussion I’ve seen of why so much journalism is tat is in the book Flat Earth News, by Nick Davies (who also broke the NOTW phone hacking story).
I do a lot of press for Wikipedia. In my experience, most journalists are perfectly decent people of good will – ones of ill-will are rare – and the subjects will still think they’ve made a hash of it. But it’s not malice or unusual carelessness. So a useful approach would be to make their job easier and raise general expectations. Wikipedia does help, pretty much every journalist relies on it.
Indeed, the system means that most journos just don’t have the time or the background knowledge to do what most scientists view as a proper job of it.
Here in Australia, “Science” and “Journalism” don’t go together in the Murdoch press: See http://scienceblogs.com/deltoid/the_war_on_science/ for all too numerous and depressing examples.
My favourite are Nicholas Wade, who is now, unfortunately, retiring from the New York Times and Ann Gibbons.
I am not sure if Matt Ridley counts as a science “journalist,” but if he does, then he is up there with Wade and Gibbons.
My main quibble would be that science reporting is by no means representative of what goes on in science. Journalists have a penchant for cuddly animals, and so-called medical breakthroughs. Of many exciting advances, e.g. in cell biology or genomics, you never hear a thing. This leads in the public eye to a skewed perception of science of either a kind of curiosity cabinet, or even worse excessive expectations.
Of course, the arguments for why this should be so are repeated over and over, but that doesn’t make them true. The deplorable lack of public knowledge is also owing to journalists taking people for dumber than they probably are….
Subscribing
Nice post. I agree with your four concerns. I personally get bothered when science journalists
1) don’t seem to recognize or be aware of the limitations of a study, thereby making it out to be far more important and more definitive than it really is.
2) recklessly sensationalize studies to to make the story more appealing to a wider readership.
3) say things in their articles that are quite simply wrong. And not just wrong in some trivial way, but wrong in some utterly appalling, this-is-basic-science kind of way.
Not all science journalism looks like this, of course; there’s lots of good science writing out there no doubt. I really like Nick Wade at the NYT, for example, and it’s interesting to note that Carl Zimmer is apparently the only journalist who didn’t immediately buy into the Arsenic Life paper. But a lot of science journalism is not very good, and I doubt the public is learning much (if anything) about science from it. I guess it depends to some extent, however, on what you believe science journalism is for — i.e. what purpose does it serve?
The Beauty Of Settled Science notes the problem – the public get the wrong impression about science because journalism is about stories of controversy, and they forget just how much we really do actually know after 400 years of this stuff.
The suggestion that journalists ought to have an advanced degree in the area he covers is preposterous. Most newspapers have only a few reporters. Each one would have to have a dozen advanced degrees in various disciplines.
Also, why would anyone with an advanced degree in anything work for a newspaper? Even big, prestigious papers are going to start you at $20K. After a dozen years, you might be making $30K Then you die of stress and alcoholism.
I’ve worked for newspapers all my life, as did my father before me. These are not exaggerations.
Precisely. This is why getting your area’s Wikipedia articles in order is actually vitally important outreach work.
(If you as an individual get annoyed with idiots – Wikipedia can’t keep idiots out of people’s faces – getting a group of academics together to improve articles in a field is a very useful thing to do, and is the sort of thing the Wikimedia Foundation publicises. Works very well.)
They need not have an advanced degree, but if you want to cover science how about an introductory course in a physical science (Chemistry, Geology, Biology) + a stats course? That way the journalist atleast knows the difference between causation and correlation, strong correlation, and weak correlation.
there is no way for a layman to keep up a few courses won’t do it
I think having that gives you the foundation of what science is and how strong a conclusion really is. The specific topical knowledge is secondary. Many science journalists don’t understand the scientific method and don’t know enough about statistics to know the difference between an r^2 of .5 and an r^2 of .9 much less how sample size impacts the certainty of a result. I think it would be a great start for them to understand the basics.
in fact, there is no unified scientific method nor a unified definition of “science” independent of a specific study and set of results — there are similarities but these are also always changing…no, the specific content is critical, for example, talking about biological topics with a, fairly deep, understanding of evolutionary processes is a mistake..
At the end of the day journalists need to write articles about these topics. Realistically no paper can afford to hire enough scientists to cover science that way (I suppose they could outsource the article writing to others in the field, but that’s besides the point). To do that they need to know that hypotheses must be written such that they are disprovable, or else they are useless… Not every journalist even realizes that.
I’m not trying to argue that journalists who take these “basics” will be able to make good conclusions based on really complex papers, what I am saying is that to be good scientific journalists and to ask the experts the right questions they ought to understand what sample size means, what a level of certainty is, and how one should approach experimentation.
I may not be a biologist (in fact I’m a geologist), but when a biologist tells me he looked at 10 schnauzers and concludes something about dogs in general, I’m going to call him on it because I know enough about sample selection and sample size to realize it’s not a representative sample.
I’m not expecting these journalists to write things that are publishable in nature, what I look at as competent to publish in a news paper and what you’re looking for is probably different. What I’m looking for is someone who is capable of asking questions about the topic, and recognizing a crackpot study when they see one. Something like the “gay gene” study should stand out to anyone as crap… It doesn’t take much to recognize that.
the motives of journalists and scientists run at cross purposes…facts never sell newspapers and real information is way too complex for a lay audience…
it’s a pop myth that any advanced knowledge will ever be accessible to the lay person…it is also a myth that sells..
we opropse that pop science actually creates more blowback than understanding
you’re wrong on sample size in physiological studies as a geologist would be…
sleeprunning: I probably should have been more specific in coming up with an example, what I meant by that was something breed specific like “all dogs have short tails because 10 schnauzers have short tails”. Sure, if someone made an argument about allometry based on 10 dogs, provided the traits they were measuring weren’t breed specific I wouldn’t argue about it. And you know what else? I’d know to ask the other scientist, “Why did you only use 10 dogs?” and “What makes this selection criteria valid?”.
You’re taking for granted the notion that newspapers have to be structured the way they currently are. In reality, the way newspapers are currently structured does not work and they are mostly hemorrhaging money and waiting to be eaten by the internet.
Don’t you think we can find a better way to do journalism than what you describe, especially now that distribution costs and delays are essentially zero?
That was my reaction, though I wouldn’t have expressed it so strongly. 😀
Journalism simply doesn’t pay enough (at most venues).
Oh, yeah. One more thing: The journalist who writes the story doesn’t write the headline. The headline is written by a copy editor, because only the copy desk knows how many columns wide the head will be, and how large the font will be. Writing heads to fit a particulaar space, while remaining reasonably true to the article, is a considerable art in itself. Most people can’t do it.
Isn’t there a dissonance between Bhattacharya’s responses to “The story didn’t contain this or that ‘essential’ caveat” and “How could you quote that person who disagrees with me? He’s wrong!”? Surely the best kind of dissenter is one who points out the significant caveats… ?
/@
PS. Bhattacharya didn’t address the specific question of why articles are frequently cast as a question — to which the answer is “no”.
*articles’ titles
My biggest complaint is always failure to include information about sample size. That’s something that’s 1 sentence that would allow me to vet the study on the spot. If I see that an article that claims that there’s a gene that controls whether you like pizza, but it’s based on a study of 50 men, I know the study is crap.
Of course there’s always the causation/correlation problem as well… That’s also annoying. That’s where the journalist decides that a study linking something like blue eyes to increased cancer rates means “Blue eyes cause cancer” instead of “Blue eyed people have a higher rate of cancer”.
Agreed, I always teach my students to distrust anything that doesn’t quote sample size or error bars.
I think it’s usually not that they are lazy, but that they don’t have much time to finish the story. I talked to the science reporter at my local newspaper (medium-sized university town) and they really don’t have time to sit down and really go through things carefully. (Yes, this sucks.)
I have to think about this…
Brian Switek has some views here on the WIRED science blogs
http://www.wired.com/wiredscience/2012/01/the-way-things-are/
Quote: “The chief characteristic of the US reporter used to be an appalling level of incompetence. This has been superseded in recent years by a stunning level of laziness”.
And the TV ‘reporters’ are infinitely worse on every subject. They all went to the Titanic sinking as a comparison to the sinking of the Costa Concordia whereas those of us with brains (almost all bloggers) went to the much more recent sinking of the Oceanos.
Most of these seem to be bordering on strawmen. For example: I’ve never seen a scientist who didn’t understand that headlines should grab the reader’s interest, and that that means they’ll often be hyperbolic. What I have seen, and what this article doesn’t address at all, is that scientists quite regularly complain that the headline often contradicts what the research says. On top of that, you’ll often have to read till the end to find that out (see Ben Goldacre’s piece The caveat in paragraph number 19). But I suppose that’s just because scientists don’t understand how the “inverted pyramid model” works…
If I remember right, it’s often the newspaper editor who gets to choose the headlines, not the article writer. That can’t help the situation much.
” When reporting a new discovery, scientists should routinely search for dissent, and should know enough to determine whether that dissent is significant.”
Did I misread this, or did you mean to say journalists here instead of scientists?
we are contrarians on the belief in the ability of the general public to comprehend science…it’s a likely as understanding medicine.
in fact, medical matters have an advantage in that everyone is interested in medical matters.
we find that popular science and science journalism are oxymorons.
science is for policy makers and decision makers, not the general public…they don’t want to know…”…people are persuaded by fiction just as much as fact.”
My big complaint is when journalists throw a dissenting opinion in for “balance”, and give it essentially equal weight, when it is an extreme minority opinion in scientific circles. Or conversely, do a pseudoscience story like crop circles, and only give a little space to the dissenting view.
I think we also have to extend the discussion to the internet, since many papers are going online, and of course articles are being shared an emailed.
Someone needs to create a “useful virus” that spreads around the internet cleaning things up by deleting garbage that demonstratably wrong. If I get another stupid email telling me that it’ll be 800 years until we get the same calendar, or that Mars will be as big as the full moon, I’ll start sneaking around and switching every computer keyboard to Dvorak that I can! (Which would be useful)
Coyne: “So my main complaint about science journalists is fourfold. First, they often aren’t trained sufficiently to write about science in a meaningful way. It would be nice if the journalist had a degree in the subject described, preferably an advanced degree. A journalist should be able to read the paper under consideration and understand it well.”
This is not practical and to the extent it does take place science journalism is not like business journalism. An MS in biology (with a dissertation on Indonesian Megapodes) will qualify you to cover cosmology or even medicine no more than a MFA in science journalism will. The only advantage is a PhD, and it’s only a superficial form of respect for the title, because an advanced degree, again, doesn’t prepare one to discuss things outside their previous field of research.
Readers: what are your complaints about science journalism?
That most of the time the journalists in question do not seem to have even the foggiest understanding of the topic they are reporting on, and seem to do only the most perfunctory research before starting to write or record. Seriously, a German journalist reporting on bird flu a few years ago took the cake for me when he essentially said “specialists are studying the disease with a recently invented super-complicated method, PCR”. Argh. Argh. Argh.
But in my experience, there is nothing special about science journalism in this regard. Being lazy and a complete idiot seems to be a prerequisite for most jobs in journalism, be it reporting on science, politics or whatever. Apparently you get bonus points if you cannot spell nor use grammar correctly. Even in Germany’s most prestigious journal (Der Spiegel), whenever I read a story about a topic on which I have any first-hand knowledge whatsoever (biology, the political party I used to be active in, a specific country I have traveled though), I realize that they are just writing up their own preconceptions, misunderstandings and prejudices after talking to one or two people carefully selected for confirming them – tops. And those are the cream of the crop; don’t get me started on Bildzeitung or TV news…
I’m a bit amazed about many of the reactions to science journalism on this blog. Do scientists really think that composers of music write music only for the orchestra? I organize a Science Cafe and my experience is quite different.
that’s a false analogy…music is by definition to communicate with the public, science is to effect physical reality regardless of communications to non-specialists…but even an orchestra is a very small and specialized limited market and definitely not for lay people…it almost requires specialty knowledge…
btw, i organize a brain research meet up group and find the material not only completely inaccessible to the members but also triggering mainly pushback….the members self-select and are also well educated folks and not the general public..
they just do not have the IQ, emotional commitment to knowledge nor capabilities to understand complex biological information….plus, when the science challenges their beliefs and preferences they pushback…human nature….
Some people think that science should be locked in the ivory tower and that the world would be better off if we didn’t bother to communicate with lay men. I for one enjoy helping others understand my field, the research I’ve done, and the research others in my field do. The less willing we are to engage the general public the more easily they can be deceived by pseudoscience about important topics like climate change.
It’s really unfortunate to see people who have such little faith in others’ ability to ask questions and learn. Obviously not every topic is going to be accessible to the general public, but many topics are surprisingly accessible. We shouldn’t let some outlying topic like “The Effect of Water Pressure in Weibe-type Magma Chambers on Feldspar Crystal Development” define every other discussion as out of reach…Science isn’t just some broad universally mysterious field, sure there are complex topics, but not everything works that way.
the road to blowback is paved with good intentions…wanting something to happen is different from whether is is possible…
lay folks just can’t understand…
You will be surprised. Why does Scientific American have a million readers?
BTW, I used to write about research for the science pages of a big Dutch newspaper (NRC Handelsblad), and a physicists told me: great coverage, I read it to keep up what happens elsewhere in science.
we need data, 1mm is nothing…best to be skeptical given the blowback and political attacks that have accompained any attempt to bring science to the public ……
“Reporting from Washington— A flash point has emerged in American science education that echoes the battle over evolution, as scientists and educators report mounting resistance to the study of man-made climate change in middle and high schools.
Although scientific evidence increasingly shows that fossil fuel consumption has caused the climate to change rapidly, the issue has grown so politicized that skepticism of the broad scientific consensus has seeped into classrooms.
Texas and Louisiana have introduced education standards that require educators to teach climate change denial as a valid scientific position. South Dakota and Utah passed resolutions denying climate change. Tennessee and Oklahoma also have introduced legislation to give climate change skeptics a place in the classroom.”
So it looks that you have to blame the absence of science reporting, not the science reporting itself. I know there are so-called journalists reporting the nonsense you mention, but you cannot blame science writers for this. O’Reilly and the other flacks at Fox new are not science writers.
blaming is too easy and a cop out, the problem is more systemic…science is getting more and more complex and simply is intractable to everyday understanding…who can work on their car anymore?
the idea that journalists can exp0lain science is just a scam to “sell newspapers”
To Sleeprunning, who says:
“who can work on their car anymore?”
This is technology, not science. Your great-great grandfather could just as well have said, “who can fix his own pocket watch anymore?”
Current science consists of ideas, and there are a lot of people around who are interested in them. Evolution and climate change are just two examples.
no, science consists of data and very precise methodologies/technologies…the analogy stands because the human brain is really limited and getting more so everyday…
the underlying motivation for denial of science complexity and inaccessibility is fundamentally dishonest, manipulative and a sales one — “Sure, anyone can understand anything!!” it’s just dum
@ Harlan Sanders: Hear, hear.
It is far more common that science journalists don’t understand science than scientists don’t understand science journalism. Some scientists (way too many) play the hype game and get all the press they can – not to inform the public but to say “hey, we’re so important we were in the papers!” when they get around to writing grant proposals.
http://www.smbc-comics.com/index.php?db=comics&id=1623#comic
Until recently my biggest complaint was they don’t link to the studies. This is Mickey Mouse stuff. Direct us to the abstract!
Now BBC, Wired, NY Times and even Guardian seem to be better about this, but others not so much. Now ideally, they would, at the end of every article, tell the reader to PLEASE READ THE STUDY for details and TELL the reader that there may be points missed in the article, which is more of a summary than a complete look at the research conducted.
Lastly I would just be glad if they didn’t get the study blatantly wrong. An article on the effects of local honey on allergies got tons of attention.
http://www.annallergy.org/article/S1081-1206(10)61996-5/abstract
From the Conclusion of the article:
“This study does not confirm the widely held belief that honey relieves the symptoms of allergic rhinoconjunctivitis.”
Naturally, journalists then reported that it the local honey remedy “does not work according to a new study”.
Agh!
Natalie Angier. (Pretty much goes without saying!)
In response to point one, I feel I must point out that Carl Zimmer doesn’t have a science degree. At least I’m pretty sure. By all means correct me if I’m wrong.
the exception that proves the rule
What rule?
individuals are too dum to understand the first thing about science, of course….
Too “dum?”
OK…
the evolutionary and inclusive fitness benefits to Baylor seem pretty clear, pander to big money contributors and alum who mouth evangelical nonsense to further themselves and their own wealth and — bingo — more money goes to Baylor.
garden variety marketing scam….
I was fortunate that I either wrote my press releases or got to correct and edit them before they went out.
Among the popular science magazines, I liked Natural History the best, because the articles were written by the scientists doing the work.
It’s true that some of these complaints are invalid. For instance, I remember PZ once whining about the fact that journalism comes in 2-4 sentence paragraphs rather than larger sections. He only displayed his ignorance.
That said, your complaint here is quite valid, Jerry. Science journalists need to be aware of when ‘caveats’ are important or dissent is bogus/needed.
My biggest complaint with almost all science journalism (which has yet to be mentioned – I’m in agreement with most of the rest above) is a failure to explain how science gets some of its strength by interrelating and connection of hypotheses (what Wilson and Whewell called “consilience”). This is poorly understood even by practicing scientists (and is a good topic for philosophers in my view to address) so it is not surprising. However understanding it seems to me to be absolutely vital, as it would help understand why sometimes scientists are accused of being closed minded or dogmatic, when really they are claiming something to be massively unlikely to impossible based on everything else we know (e.g. perpetual motion machines, homeopathic cures, etc. to take extreme cases).