When I started writing books for the general public (“trade books”, as they call them), I was surprised to find out that I alone was responsible for the accuracy of their content. While publishers do have their legal departments vet books that may violate laws against libel, or cause other legal troubles, publishers have neither the time nor the money to have books thoroughly fact-checked. This is one reason why authors put references for many factual statements at the end of the book, as I did with my two.
Even so, errors slip in, and sometimes those errors are numerous. A new piece in the New York Times cites some serious or numerous mistakes in popular books. Here are four that are highlighted:
Accusations of sloppiness and journalistic malpractice now quickly explode on social media. Ms. [Jill] Abramson was pilloried on Twitter by sources and other journalists this year for mistakes in her book “Merchants of Truth” and for failing to cite source material from other writers. She made corrections and credited sources in the digital version and future print editions.
. . . scrutiny of such books is growing. After [Michael] Wolff in June published “Siege,” his account of volatility inside the Trump administration, journalists highlighted numerous inaccuracies in the book.
. . . In May, The New York Times Book Review published a blistering review of [Jared] Diamond’s book “Upheaval.” The reviewer, the author Anand Giridharadas, cited mangled facts and what he described as misleading generalizations, and argued that the flaws were emblematic of a systemic lack of fact-checking in publishing.
. . . When publishers do conduct a factual review, it’s often in response to a crisis. In June, Houghton Mifflin Harcourt postponed the release of [Naomi] Wolf’s “Outrages,” which explores how 19th-century British courts criminalized same-sex relationships, and commissioned independent evaluations from several scholars after questions were raised about the accuracy of her research. The publisher took the unusual and costly step of recalling copies from retailers and pulping them. Ms. Wolf has said that she disagrees with the delay, and that only a small number of errors must be corrected.
To be fair, some of the authors contest these accusations, while others are correcting the errors. But if you’re writing nonfiction, rigorous scrutiny should be given to all assertions of fact.
When we write scientific papers, all factual statements are followed by a citation like (Schlemiel, 2015), with the reference given at the end of the paper. For you know that captious reviewers are going to scrutinize your paper before a decision is made on whether it should be published. If trade publishers or authors could follow such a practice (my habit was to send bits of the book to experts in the field for their reactions), it would dramatically improve the accuracy of books.
Which brings us to Malcolm Gladwell.
I haven’t read any of Gladwell’s trade books, but of course they’re immensely popular and have made him a multimillionaire. Some of my friends who are savvy and have read them dismiss them as pop sociology, lessons in how to market the obvious by pretending it’s profound. I can neither agree nor disagree, as I plead ignorance.
But the verdict of “superficial” is rendered by Andrew Ferguson in this new Atlantic article (click on screenshot), which claims that with his newest book, Talking to Strangers, Gladwell has finally exhausted his formula—to the point where a central thesis is barely discernible. If you’ve read this book, or Gladwell’s other ones, you can be the judge.
But though I haven’t read the book, I was pretty offended by one of Gladwell’s assertions (also mentioned in the NYT article), a falsehood that bespeaks poor scholarship, and would make me wonder how sound the rest of his assertions are. Here’s the fact in dispute (Ferguson’s prose):
I don’t know whether default to truth [Gladwell’s catch phrase for the obvious fact that we tend to believe things other people tell us] will enter the Gladwell lexicon with tipping point and stickiness. But his appropriation of the phrase does show that his attitude to social science remains unquestioning. When he encounters a study published in a journal with a complicated name, he defaults to swallowing it whole. At times he approaches self-parody. Just follow the footnotes.
“Poets die young,” he writes, in a section on Sylvia Plath. “And of every occupational category, [poets] have far and away the highest suicide rates—as much as five times higher than the general population.”
Interesting, sort of, if true! But how would such a calculation be made? Poet is a strange “occupational category.” Hardly anybody makes a living as a professional poet. The Bureau of Labor Statistics, in its Standard Occupational Classification System, lists 867 occupations. The closest it gets to “poet” is “Writers and Authors,” a category so baggy it includes bloggers and advertising copywriters. What kind of poet wants to be confused with Mad Men? Wallace Stevens wrote sublime poetry, but I think the BLS would still prefer to classify him as a vice president of an insurance company.
Gladwell’s footnote shows he has drawn this curious statistic from a paper titled “Suicide and Creativity,” by a college professor named Mark Runco, published in 1998 in the journal Death Studies. Runco in turn cites a book, Touched With Fire, by Kay Redfield Jamison, a clinical psychologist.
To get her “five times” figure, Jamison explains in her book, she studied the lives of “all major British and Irish poets born between 1705 and 1805.” She determined their “major” status by consulting old poetry anthologies. She decided there were 36—not 35, not 37, but 36—major poets, ranging from the well-known and era-defining (William Wordsworth) to the obscure and improbably named (John Bampfylde). Of the 36 poets, two committed suicide. (It’s not clear that these two can even be classified as poets, however: One was a physician by trade, and the other died at 17, probably too young to qualify for an occupational category.) Jamison reckoned that two out of 36, proportionally, is five times the suicide rate for the general population.
Voilà! A statistic is born.
This is thin soup. One wonders whether Gladwell bothered to trace the statistic back to its source. Jamison’s sample is clearly too small and peculiar to yield a reliable understanding of the suicide rate among poets, even 18th-century poets in the British Isles. Many people who spend a lot of time writing poetry are eccentric; the elevated suicide rate feels true, intuitively. But for Gladwell, as for so many consumers of social science, the intuition becomes real only if it’s quantified, even when any kind of useful quantification is implausible on its face.
I read Jamison’s book years ago, but don’t even remember that assertion. But what Gladwell did with it is pretty bad—turning a sketchy hypothesis based on scanty data into a general claim about suicide and professions. This is not simply a misreading, but a distortion bespeaking, at the least, sloppy research.
You may say that this is trivial, but as a scientist it bothers me. A claim like that, were it to appear in a scientific paper, would be pounced on by reviewers demanding that it be fixed or clarified. We should expect no less from factual assertions in popular books.
At any rate, if you don’t like Gladwell, you’ll enjoy a good bout of Schadenfreude with Ferguson’s article. To wit: its ending:
Rather than offering made-up rules and biases and effects, Gladwell has chosen to issue a plea, asking that we recognize how difficult it is for us to understand one another.
Of course, if Malcolm Gladwell had practiced epistemological humility for the past 20 years, he would have sold millions fewer books. But let’s pass over the irony. When you’re talking to millions of strangers, as Gladwell does, saying nothing in particular is better than telling them things that aren’t so. He may have embarked on an exciting new career.
I find it mind boggling that anyone understands the full analysis behind scientific papers, and I wonder if some of the people summarizing in their books them share this trait. Generally I look for statements like “After correcting for (age, weight, socioeconomic status, etc.)”; “significant differences” and so on. But fairly often you read studies that look very impressive and conclusive, only for someone to later come out and say “When a meta analysis was conducted, effects were weak or insignificant”. Clearly the authors of the earlier paper got by with putting those statements in peer reviewed journals, but even then, there can still be a lot of error or room for interpretation.
Yes, and then these papers go on to influence entire fields for decades because they provide useful ideas, like the poorly executed experiments on implicit bias, which you see taken as a given truth all over the po-mo studies literature and discussion.
I feel like I see this a lot regarding health issues. Fish oil, red wine, doing crossword puzzles to stave off dementia – first they’re great, then they do nothing, then finally it’s “Maybe they do something but evidence is not conclusive”.
Haha every year I hear on the news that a new study shows that coffee/red wine is either great for you or bad for you. It seems to change on an annual cycle. My mom and I always joke about it.
This is mostly an indictment of journalism, not science. Mostly.
I don’t know – I’d have to go back and look to be certain, but I feel like it’s really not that uncommon for meta analyses to conclude that the methodology in a number of studies was weak or the results insignificant, overturning previous conclusions. Obviously if you read the original studies themselves, they don’t say “We conclude that we conducted an experiment with weak methodology and poor statistical analysis”, so that comes from the original studies, not news summaries.
Meta-analysis is also hard to do well. John Ioannidis (King of meta-analysis!) had a paper entitled:
“The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta‐analyses”
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5020151/
Interesting – I suppose I should have inferred that meta-analysis itself being difficult to conduct well would be the next logical step!
It’s hard work, and requires a good memory, or a habit of making detailed notes. But it also generates a “different attitude”, I suspect.
If there’s any truth to Ferguson’s description of Gladwell’s work as follows (I don’t think I’ve ever read any of his stuff, nor am I likely to buy any unless direly short of toilet paper) :
Amongst people in the habit of reading scientific papers, or even having to write them (or technical reports under commercial confidentiality) almost the opposite would be the case – you default to scepticism. You read something, and you do follow up an appreciable number of the references – as time and library facilities allow (heil Sci-Hub!) ; you do work through the arguments ; you follow the maths, in part because that shows you how strongly or weakly a datum is going to affect the outcome of the analysis. It’s hard work.
I also find that it makes reading fiction pretty hard work, because you actually have to suspend your disbelief in order to get into the author’s world. Which is much easier (for me) in a fantasy world than in something that even vaguely resembles a world where I need to apply critical thinking.
I think you need a fair amount of training in statistics as well.
Training in statistics is certainly useful. It may even be obligatory in science faculties in tertiary education, though it wasn’t when I was a student. But properly presented data – e.g. graphs of results with error bars – should cover the main points of your statistical analysis. If the statistics are the bulk of the paper, then it’s pretty likely that the results are at best weakly supported by the data and default to scepticism applies.
Sub
It’s a small number of words, but very clearly articulated. If it was a genuine error I think the only reasonable way is if it was lost in the volume of material being edited. The big question is if it was deliberate – is Gladwell putting this down and expecting the audience to like it?
Since I can imagine looking at my own writing and being puzzled how I could have written such a thing, I’m not going to judge. Sometimes, I have to mash out an idea real rough to get it down.
All I know is I read Blink several years ago and it was full of assumptions based on scant evidence and outright assertions based on no evidence. I found his “formula” very underwhelming, indeed.
I read The Tipping Point when it came out. IIRC, it was shallow at best: a bunch of mildly interesting anecdotes about various sudden, remarkable successes, but not a single word on how you might effect such a success. Underwhelming is an overstatement.
Yes, it was like a sugar rush – immediately seemed interesting & plausible, but ultimately left one hungry… Other people are better at this sort of thing, for example Bruce Hood, https://en.wikipedia.org/wiki/Bruce_Hood_%28psychologist%29
or Richard Wiseman
https://en.wikipedia.org/wiki/Richard_Wiseman
I have read a couple of Gladwell’s books and I liked them very much. I will admit to not doing any fact-checking on it. Still, up until now I have not heard many complaints, unlike with Pinker’s books. (I’m not saying I agree with the complaints, just that they exist and are widely acknowledged.)
I haven’t read Gladwell’s latest book but it sounds doubtful he hung much of its thesis on the suicide rate of poets. And it also sounds like he was completely transparent on how he arrived at his statistic. Is it possible it was offered partly as humor?
By the way, Malcolm Gladwell was on last Sunday’s Fareed Zakaria GPS. It’s probably available online. Fareed acknowledged criticism of Gladwell. Perhaps Ferguson’s article was what he was referring to. Zakaria mentioned Gladwell’s formula and supported it. I don’t see much wrong with the formula either.
https://archives.cjr.org/the_observatory/the_gladwellian_debate.php
Gladwell is not well liked among people knowledgeable about what he writes about. He is a hack of the highest caliber. That you do not know that makes me doubt you have any knowledge yourself of the subjects he covers except the farces he calls serious works.
These complaints like Pinker’s and the one you link to sound like pretty thin stuff. He’s a journalist, not a scientific researcher. That he gets dislike from researchers in the field is only natural. He has much more fame and money than they do while reporting on their work to a lower standard of scrutiny.
That you have doubts about me is neither here nor there.
No credit for Schlemiel’s co-authors Schlimazel and Schmendrik?
Their dearly beloved mentor, Schmuck, sadly departed late last year…
Schorry to hear that! :'(
The veracity of Malcolm Gladwell’s writing was first debunked years ago and has been continually debunked but that hasn’t stopped him from adding to his extremely flawed oeuvre. I fail to understand how anybody can be taken in by Gladwell’s assertions and descriptions when there is ample evidence to prove them wrong — factually false, distorted, oversimplified, etc. — yet many are. A few weeks ago he was fawned over when a guest on the insufferably twee “Radiolab.”
An article from the Columbia Journalism Review (2013) https://archives.cjr.org/the_observatory/the_gladwellian_debate.php, whose subtitle reads: “Why are we still listening to Malcolm Gladwell’s cherry-picked gospel?” “Still” is the operative word here, and that was 2013.
But that isn’t the half of it. I’d pretty much forgotten about the questionable ethical situations he’s been involved in. https://shameproject.com/profile/malcolm-gladwell-2/. The caption below his photo reads: Shills for Pharma, Big Tobacco, Wall Street, Health Insurance Industry.” As far as I’m concerned he’s thoroughly disreputable and that judgment extends to his writing because one can’t tell fact from fiction.
The “reception” section of his Wiki, beginning with the third paragraph, notes his numerous factual and conceptual transgressions https://en.wikipedia.org/wiki/Malcolm_Gladwell#Reception.
Pinker’s complaint also comes with quite a bit of praise. His review can be fairly summarized to say that Gladwell is weak on statistics and that his essays are better than his books. This is not surprising coming from Pinker who is a scientist, not a journalist like Gladwell. Pinker’s approach is much more convincing but Gladwell is very entertaining and, so far, I haven’t seen any case where he goes completely off the rails.
BTW, Pinker’s review of Gladwell is here: https://www.nytimes.com/2009/11/15/books/review/Pinker-t.html
Thanks for the link. I’ll read it.
I recall him arguing for allowing doping in sports, without bothering to so much as mention side effects.
The’re now lining up for the 100 metre dash Off your Face final, we will report back in 30 mins for the results… 🙃 a delayed side effect?
“[poets] have far and away the highest suicide rates—as much as five times higher than the general population.”
Even accepting the source data as valid, it could also be written as:
“Most poets (94%) do not commit suicide.”
It is all about how you want to spin it. What sort of claim about poets is he supporting with his observation?
I’m pretty sure I would not rely on Gladwell to guide my conversations with poets.
Righto.
https://www.bookdepository.com/Beautiful-Poetry-Donald-Trump-Robert-Sears/9781786892270?ref=grid-view&qid=1569286257316&sr=1-1
Follow the link but be warned you, suicide may follow… 😜😳
“Hardly anybody makes a living as a professional poet.”
I have, but I’ll give Ferguson that one: it’s generally true that there’s no money in poetry. Then again, there’s no poetry in money.
That was poetic!
You can take the poet out of the <a href="https://en.wikipedia.org/wiki/Antimetabole"antimetabole, but you can’t take the antimetabole out of the poet.
“Antimetabole”? Really? Ken, who may be the only person ever to use “litotical” in a sentence, was apparently frightened at a young age by a trope. Needless to say, the trope was also frightened by Ken. 😊
What can I say, Gary? I picked up a copy of Mr. Quinn’s slender volume Figures of Speech at an impressionable age and — from anaphora to zeugma — never quite got over it. 🙂
We’re glad to see you’re still among the 17 out of 18 poets who haven’t snuffed it, Gary! 🙂
The statistic is based on a selection of 19th century poets. A proper comparison would then take 19th century farmers, doctors, shop assistants, industrial works, lay-about aristocrats, & see how they compared.
Also the sample size is pretty feeble. I would take every possible published 19th century anthology & work out all of the poets cause of death.
I admire Dr. Coyne’s forbearance in not feeling adequately informed to pass judgment on Gladwell because he hadn’t read him. I, on the other hand, have not read him because of the judgment I have passed on him based solely on my gut reaction to what I have read and heard about his writings, and that he has been embraced by the looneys.
A sensible way to conserve one’s time and energy. I can’t begin to count how many highly-touted authors I did not read, and learned later, from unimpeachable sources, that I hadn’t missed anything.
I believe he could do better if he wanted to but perhaps he writes that way because it sells. So is he in it for the money or does it satisfy an intense craving for attention? Or both? The articles he did for The New Yorker were better.
I never got much out of his books but I like his voice so I listened to them instead of reading them. I’m not bothering with this one though.
Diamond’s book should not be on this list. The Anand Giridharadas review of it was horrific. I don’t recall any citations of factual errors at all. At any rate the book is largely Diamond’s account of personal experiences living in various countries, and he goes to lengths to caution against generalizations. The review mainly objected to “big picture” books ala Diamond (and Pinker would also fit his target) for no good reason that I could discern.
It’s not just published authors who play ‘fast and loose’ with the facts. I stumbled across a review of a horror film ‘Inside’ released in 2016 where the reviewer takes the film-makers to task for this:
“The intro also offers the entirely made-up claim that there are 300 cases of fetal abduction per year in the USA. In fact (according to Wikipedia), the total number of fetal abduction cases has been a mere eighteen between 1983 and 2015 and the so-called figure of 300 per year is rather more modestly a total of only 302 cases of infants being abducted in the entire thirty year period studied.”
http://www.moriareviews.com/horror/inside-2016.htm
Numbers like this tend to escape the sensationalist context in which they are used and enter the general conversation. Eventually they may become the basis of policy.
Everyone knows half of all statistics are made up. 🙂
But the film in question was a work of fiction. Why would you expect anything in it to necessarily be true?
I think the reviewer was more complaining about the fact that the statement was a spoiler for the rest of the film/
Isn’t the point that people take information that is presented as fact in films at face value? And thus misinformation enters the public domain — or it’s presence there is reinforced (e.g., the “the ten percent of the brain myth” in Lucy).
🐜
Do they? More fool them.
“citation like (Schlemiel, 2015)”
I hate this form of citation; it makes the text hard to read to keep skipping over such noise.
🐜
I haven’t read any of Gladwell’s books, but it does show the difference between peer reviewed and otherwise.
I do have a fondness for big picture synthesis books (even if popularizations) but I have learned to be careful. (Pinker and Diamond for example have to be read with that in mind- and as we have seen I agree sometimes, not always.)
As for what should be done about errors, I dare say these days it is easy enough to put up a web page for errata, as the academic and industry (e.g., computing) press does – but that would require the author or publisher to maintain something – and give a damn.
Of course even the latter doesn’t always work – I remember posting to a forum about an O’Reilly book and not getting even an acknowledgement. (A shame since they are generally quite good.)
I woke to Gladwell when he suggested that the solution to doping in sport was to allow any and all doping and not to bother testing for anything.
I’ve heard this from other respected sources too. If I remember correctly, the Economist suggested this long ago. The idea is to basically acknowledge that there’s an unwinnable arms race between athletes and drug testers that basically gives the advantage to the best cheater, and that this arms race would only get worse due to technology.
What “give up the unwinnable arms race” doesn’t account for is that the rules are also there to protect athlete health. In an open market the advantage will go to the athlete most willing to give up their long term health. Anyone unwilling to do that would be priced out of the market.
In the early 00s cyclists were wearing their heart rate monitors to bed with a low limit alarm. If their heart slowed too much the alarm would wake them and they’d do jumping jacks.
(I realize that my anecdote is a bit ironic in the current context- I got it from Roland Green who turned down a spot on US Postal for this reason. There are several athlete biographies that relate similar risk taking)