Apropos of yesterday’s post on the unproven efficacy of ivermectin for Covid-19, I talked to my GP, Dr. Alex Lickerman, about the drug (he’s read the studies). I learned not only that there is no publication of high-quality controlled double-blind tests needed to show that ivermectin is effective against Covid-19 (there’s a big one that should be published by the end of the year), but also, surprisingly, nearly half of the medical drugs and procedures we use have not been subject to these tests. Very often the outcomes of clinical practice are just assumed to be efficacious without any rigorous tests with placebos and so on. Sometimes some people improve, but there is no randomized control group to compare them to. This is also true for some operations, in which “sham operations”—procedures that mimic real operations without the real surgical manipulation—have shown to be no better than the placebo procedures.
This is all summarized in a 2015 book shown below (click screenshot to go to Amazon link), and in a 2011 paper below that written by the same two physicians (Vinay Prasad and Adam Cifu, the latter from University of Chicago Medicine). What they mean by “medical reversal” is that later and better tests often show that drugs or procedures are either not helpful or could be harmful, so there’s a reversal of opinion and—if doctors are aware of this!—the procedure is abandoned or modified. We will soon know whether ivermectin is such a case.
If you want a shorter read on medical reversal (I haven’t read the book), see the paper below by Prasad and Cifu from the Yale Journal of Biological Medicine. It gives lots of examples, including both drugs and surgery, and describes why medical reversal is important. It’s not detected as often as it should be because double-blind randomized tests with controls are time-consuming, expensive, and hard to do for surgery. Neverthetless, I was surprised to find out that roughly 40% of procedures or drugs prescribed by doctors have been shown to be either unnecessary or harmful. Now I’m not a doctor, but I recommend you at least scan the paper below (click on screenshot) or listen to the audio link below that.
If you prefer listening to reading, you can find an hourlong conversation with Dr. Cifu on econtalk in which he summarizes “medical reversal” and gives examples.
Here are a couple of excerpts from the paper:
The second phenomenon is reversal: A medical practice falls out of favor not by being surpassed, but when we discover that it did not work all along, either failing to achieve its intended goal or carrying harms that outweighed the benefits. Although this phenomenon should be rare in the age of evidence-based medicine, it is ubiquitous. Common use of avandia [3], ezetimibe [4], atenolol [5], hormone replacement therapy [6], and the class 1C antiarrhythmic agents [7] all stopped when trials showed they were either ineffective or harmful. Reversal not only affects medications. Previously accepted indications for surgical and medical procedures also have been abandoned. In 2009, stenting for renal artery stenosis was shown to be ineffective for many patients by the Angioplasty and Stenting for Renal Artery Lesions (ASTRAL) trial [8], and in 2007, the Clinical Outcomes Utilizing Revascularization and Aggressive Drug Evaluation (COURAGE) [9] trial found no benefit to support percutaneous coronary intervention (PCI) (versus optimal medical therapy) in most patients with stable coronary artery disease. In these cases, reversal does not mean that for every indication and purpose the therapy in question was shown not to work, but simply that it was contradicted for key indications.
. . . Reversal differs from replacement in that it produces three perils. First, reversal implies mistake or harm to patients cared for under the old model. The abandoned practices were ineffective or harmful. The cases of CAST and Avandia demonstrate harms, while COURAGE and Atenolol suggest only the harm of misplaced financial and social resources.
. . . Second, removing a once-commonplace practice can be more difficult than imagined. Adherence to the contradicted claim furthers malfeasance. The idea that beta-carotene could diminish cancer gained popularity in the early 1980s [25]. By the mid-1990s, however, three randomized controlled trials overturned the claim [26,27,28]. However, nearly a decade passed before counterarguments were uncommon in the literature [29].
. . . Third, reversal undermines trust in the medical system. In the case of hormone replacement therapy (HRT) — once thought to be beneficial for reducing a woman’s risk of heart disease while treating menopausal symptoms and contradicted by the Womens’ Health Initiative — patients report feeling “furious” with doctors who “pushed” therapy upon them [34].
I asked my doc about hormone replacement therapy (HRT), as of course it’s still being widely used (check the internet). However, it was long thought, without any real tests showing it, that HRT, among other benefits, would also help postmenopausal women prevent the development of heart disease. A controlled test of that claim showed it was wrong: if anything, HRT increases the risk of heart disease. The problem may have been, as Cifu mentions in the podcast, that women seeking HRT could have been younger, thinner, and healthier (conditions that help prevent heart disease) than those who didn’t seek HRT. The “control” group was the latter, but it wasn’t a randomized trial: the “controls” may have been a nonrandom sample more likely to develop health disease. The upshot is that HRT, which used to be given to all symptomatic post-menopausal women, is not given to women with heart conditions, and patients are (or should be) informed about the slightly increased risk of heart disease. (See the Mayo Clinic’s advice here.)
Dr. Lickerman added this (quoted with permission):
If you’re a post-menopausal woman with post-menopausal symptoms and known heart disease, you probably shouldn’t get it. If your risk of heart disease is otherwise average and your post-menopausal symptoms are severe, it’s a tool that can be used. Think of it like using Advil to treat arthritis. There are definitely risks, but we judge them against the benefits in each individual case. What we no longer do is give HRT to all post-menopausal women because our original thinking was it would benefit them all as a preventative. Now we know better. We no longer use it for prevention; only for treatment when benefits outweigh risks.
Most of what we do in medicine is done based on observational studies. Prospective, placebo-controlled randomized trials are very expensive and time-consuming. My colleague, Adam Cifu, co-wrote a book called Ending Medical Reversal in which he did a survey of the literature and estimated that ultimately 35-40% of medical practices, when finally prospective studies are conducted, are found to be useless or even harmful. It’s quite shocking. This is why I focus so much on evidence.
The lesson, as Cifu says in the podcast, is to interact with your doctor, and ask for evidence if you’re dubious or unclear, for a patient doctor relationship is just that—a relationship. A doctor who is imperious or who won’t even talk about evidence isn’t worth having.
Finally, a couple of quotes from the New York Times‘s 2015 review of Ending Medical Reversal:
The incremental progress of ordinary science is one thing, as individual treatments are progressively replaced by better variants. We all happily accept that kind of revision. But medical reversal, the authors’ sober term for sudden flip-flops in standards of care, unnerves and demoralizes everyone, doctors no less than their patients.
Dr. Vinayak K. Prasad and Dr. Adam S. Cifu, of Oregon Health & Science University and the University of Chicago, have set themselves the task of figuring out how often modern medicine reverses itself, analyzing why it happens, and suggesting ways to make it stop. If this short list of objectives explodes into a breathless and somewhat unwieldy critique of all of Western medicine, you still have to appreciate both their ambition and their argument.
An old saw has long held that 50 percent of everything a student learns in medical school is wrong. Actual calculations suggest that number is not too far off base — Dr. Prasad and Dr. Cifu extrapolate from past reversals to conclude that about 40 percent of what we consider state-of-the-art health care is likely to turn out to be unhelpful or actually harmful.
Recent official flip-flops include habits of treating everything from lead poisoning to blood clots, from kidney stones to heart attacks. One reversal concerned an extremely common orthopedic procedure, the surgical repair of the meniscus in the knee, which turns out to be no more effective than physical therapy alone. The interested reader can plow through almost 150 disproved treatments in the book’s appendix.
. . . What could make more sense, after all, than finding some cancers early, fixing a piece of torn cartilage, closing a hole in the heart, and propping open blood vessels that have become perilously narrow? And yet not one of these helpful interventions has been shown to make a difference in the health or survival of patients who obediently line up to have them done.
. . . Dr. Prasad and Dr. Cifu offer a five-step plan, including pointers for determining if a given treatment is really able to do what you want it to do, and advice on finding a like-minded doctor who won’t object to a certain amount of back-seat driving. Of course, there are no guarantees that their tips will endure forever, but they probably have a longer shelf life than most medical advice.
h/t: Alex Lickerman
sub
I offer another example, perhaps relevant :
taking aspirin … in some “dose” … to do … something … to blood … that is supposed to be “good”. There is all sorts of interesting biochemistry.
With regard to the medical health benefit notion, I claim ignorance – I merely point to it as some example that comes to mind. But, as I recall, the advice has changed over the years on aspirin. That is the point / relevance to this topic.
^^^ it is hard to write a simple idea down.
I’d be interested if you have more detail since I take a daily dose to prevent a repeating MCI (myocardal infarction.)
You can do a simple search yourself in the brilliant Pubmed. Just remember your boolean operators when you search.
https://pubmed.ncbi.nlm.nih.gov/?term=dose+aspirin&sort=date&size=50
A wonderful resource…
Yes indeed, but this is really about doctor-patient guidance. In my anecdote / “lived experience”, aspirin was once a great idea as guided by doctor-patient in-office dialogue, but recently, for me (of course), the patient-doctor in-office dialogue suggested not to get into that.
Say what one will, the above will not be in PubMed but is critical for medicine. I think is really depends critically on what records the patient has as discussed with the doctor. That also is not in PubMed.
BTW : I’d like to know any solid guidance on aspirin as well!
I don’t have heart problems, but I asked my doc whether I should take a daily aspirin, which I had for years before I met him. He said “no.”
What you are talking about is so called “primary prevention”. You do not have problems. And indeed, the evidence does not show any real advantage of taking aspirin to prevent the occurence of a myocardial infarction or “heart infarct” in people without a history of heart infarct. But the evidence that aspirin is usefull in secondary prevention (for people who have had a heart infarct), to prevent new ones, is strong.
Yes, I understand.
Low dose aspirin (75mg a day is as effective as150 mg/d) has benefits in some patients at risk for vascular events, such as DM2 or previous thrombosis.
Giving it as prevention in otherwise healthy patients has always been controversial since it was proposed, so either way I would not call it a ‘reversal’.
“… I would not call it a ‘reversal’.”
Thanks to everyone’s discussion here. Things makes sense now – it seems to me low-dose aspirin was a fad – for those who never experienced cardio problems. I suppose those specific patients might still worry about this if their ancestors _have_ experienced cardio problems.
So as they say, “talk to your doctor”.
May interest -https://www.ajpmonline.org/retrieve/pii/S0749379721000179
Also
https://www.ahajournals.org/doi/10.1161/JAHA.115.002320?url_ver=Z39.88-2003&rfr_id=ori%3Arid%3Acrossref.org&rfr_dat=cr_pub++0pubmed&
I appreciate it, but I’m not really qualified to read much more than abstracts and conclusions to see what the researchers report as the meaning of their findings. I’m aware that the advice has changed to make sure that in the case of aspirin being taken for thinning blood and preventing clots, patients are taking a high enough dosage to account for obesity and my cardiologist assured me that I am well under the weight for this particular concern. My request from ThyroidPlanet was to ask if there is more detail specific to what he or she posted.
Maurice above has it right. The final verdict on aspirin as primary prevention—that is, to prevent heart attacks in people without known pre-existing coronary artery disease—is that benefits do not outweigh risks, so we no longer recommend it. In patients with known coronary disease—especially if you’ve actually had a heart attack—the benefits of taking aspirin clearly outweigh the risks and we do recommend it (secondary prevention).
Michael, if you already had a heart attack, then taking an aspirin probably is evidence-based. (I’m not a healthcare professional.) In Paul Offit’s book “Overkill: When Modern Medicine Goes Too Far” (2020), there is a chapter entitled “Baby aspirin doesn’t prevent first strokes or first heart attacks”. I don’t have a copy of this book with me now, but to me this chapter title suggests that the effects of aspirin on heart heath depend on whether it is for primary prevention (never had a heart attack before) or secondary prevention of a heart attack.
See also here: https://en.wikipedia.org/wiki/Aspirin#Heart_attacks_and_strokes
Thanks, Peter!
I think you got the information you need – indeed, _I_ got the information _I_ needed! I learned more in less than 24 hours here than I ever did about low-dose aspirin by osmosis of random conversations and such over the years.
The study hits home! I take flecainide for PVCs, but have not had a prior heart attack. For all you cardiologists who follow this blog, should I be concerned?
Very interesting, thanks! I had no idea about the prevalence of “medical reversal” or what brings it about.
A review led by NHS England’s chief pharmaceutical officer was published earlier this week. It found that 10% of drugs currently being prescribed in the UK are either ineffective or potentially harmful, although the issue is related to overprescribing by doctors and pharmacists rather than the efficacy of the various medications more generally.
https://www.theguardian.com/society/2021/sep/22/government-review-finds-10-of-drugs-dispensed-in-england-are-pointless
Part of what brings it about is that pharmaceuticals are huge business. Companies have a strong monetary disincentive to undertake tests not required for FDA approval (after all, if your drug would be approved for sale without a test, why risk the test and the chance it comes out negative?)
The federal government really needs to set more stringent standards here. For example, having companies register tests before they are done (with any registered but unreported result counting against approval), and requiring more on the efficacy side.
Forensics have a somewhat similar problem, IMO. Having prosecutors and police decide what tests to do and how they are to be done creates the same strong pressure not test techniques “too much” when it comes to accuracy or reproducibility.
I’ve no idea if it is still the case, but certainly in the 1980s my parents (in the UK) were friends with several general practitioners (GPs) who were regularly “wined and dined” and given various freebies (such as branded stationery etc.) by sales reps from pharmaceutical companies. I only know this because the doctors had so much of the stuff that they offloaded it to their friends and I was a recipient fairly often.
I think it would be easier if the public in general were more literate about the nature of the “scientific method” and medical science in particular. All scientific conclusions are provisional (at least in principle) and in medicine more so than in many other fields. We’re attempting to manipulate the most complicated individual things (if you count the nervous system) in the known universe using fairly blunt instruments such as drugs and, well, sharp instruments. It’s remarkable that we get it “right” as often as we do, but we should all always hold the conclusions of medicine as more provisional than those of some other sciences. The problems arise in part because it deals with human health, well-being, and mortality. And everyone wants to do SOMETHING, and do the best we can in any situation. And, of course, sometimes doctors have to be pushy even to get people to use tried and confirmed things like vaccines, but that can backfire when they’re being pushy about something they have reason to think is probably beneficial, but turns out not to be.
+1K^K
I suspect there is a lot of difference in the medical world concerning medicines that doctors might put you on, just as there are differences among people getting the medicines. I have always kind of been a minimalist on taking things. The least the better. Nearly all I take is due to type II diabetes and that is enough for anyone. You better have good insurance because otherwise you better be rich. That is the name of medication in this country (money). Kind of strange but I take a pill for cholesterol and have for many years and it cost nothing. Procedure-wise I have had Heart catheterizations and echocardiograms, however those are pretty standard for some things.
Iatrogenic illnesses- I do not trust GPs. They are such generalists. One hears horror stories – on the radio yesterday, how a GP missed that a boy was diabetic as the surgery refused an appointment in person.
Specialists, & surgeons, yes, I trust them.
I recall when I worked at the Institute of Ophthalmology Library at Moorfields (another part of the UCL empire) doing a literature search for why a particular eye drop mixture was used. There was no evidence base for it – it was a historical convention from somewhere in the 20th century, that just was passed on with no one questioning it for decades! I wish I could remember the actual substance used.
Paul Offit’s Overkill is a fairly quick read along the same lines.
“The incremental progress of ordinary science is one thing, as individual treatments are progressively replaced by better variants. We all happily accept that kind of revision.“
Not true. There is huge pressure in the industry, the FDA, and payers like insurance companies against “me too” drugs, drugs that are of the same class and mechanism but have slight improvements in their efficacy or safety profile. Why do insurance companies get to second guess your doctor’s expert authority to prescribe certain medications by insisting they’re too expensive and the generic, cheaper 1st gen drug is just as good as the pricier 2nd gen drug? To save money. In fact, clinical trials showing “superiority” to existing drugs are often a requirement for them getting approved. Also, some drugs in the same class work better for some patients than others. Tylenol is only effective in about 70% of people with headaches (a very high response rate). Others respond better to aspirin or ibuprofen. Human medicine is so much more complex than most everyone realizes.
I have a question for doctors/medical researchers in the house: what’s the best way to do a double-blind study on serious surgical procedures? How do you set up such a study for a group of people who need heart surgery? My uncle recently had triple-bypass surgery and, at least according to his doctors, he would have died pretty soon without it. He certainly is a lot better since having the surgery, as has everyone else I’ve ever known to have such surgery, but that is of course anecdotal.
So, how would you conduct a proper study on the efficacy of this kind of surgery? Surgery that is, in essence, to save the patient’s life? Surely you can’t give a group of such patients a “placebo” operation and see if they die sooner. If a double-blind study of such procedures isn’t possible, what is the best way to study whether a new procedure would be better, beyond trying that procedure on many patients over many years and simply measuring the outcomes of those patients versus the patients that received the older method of surgery?
Good question, one that I wondered about. You might just have a control group of comparable people that don’t get the bypass. You can conrol for age and other factors by choosing judiciously. I suspect there would be a huge difference in mortality, and that would be enough for me.
Yes, Jerry is correct here. One can try to match groups across many demographics (age, etc.) but also stage of disease using objective diagnostic standards, risk factors including genetics and so on, and then longitudinal studies comparing endpoints such as death or disease progression. If the outcome is highly statistically significant, e.g. increased life expectancy for the treated group, then the evidence shows the surgical intervention worked. Even then, there are caveats.
Which endpoints are chosen is a matter of much debate with the FDA and longitudinal studies often take a very long time and are extremely expensive, requiring large cohorts or big effects of the surgical intervention. So rather than time-to-death as an outcome, how about objective “quality-of-life” standards and/or cost-saving criteria? This could be “% reduction in hospitalizations/ complications due to that disease” or “significant extension of time from disease stage 3 to disease stage 4.” Not as definitive as mortality but significant “benefit” to the patient nonetheless if the consequences of disease stage 4 are well-established.
A good example would be opening blocked arteries with wire mesh stents that are placed by a surgeon using an arterial catheter and expanding the stent like an umbrella to force open the blockage. They definitely worked but it was later found after years that bare-naked metal stents actually allowed reblockage to occur as fast or possibly faster than untreated due to more rapid clotting caused by the metal itself. Then came metal stents that were coated with slowly released drugs that prevented reblockage. Better. Then came results for stents without drugs integrated into them but taking oral anticoagulants like Plavix – better still.
I think it would be easier to make a case against alternative medicine if science based medicine didn’t have so many skeletons in its closets. It’s hard for me to argue effectively with relatives who buy into all of the CAM nonsense when they rightfully point out that sbm has much entrenched fraud, greed and incompetence. How do you respond when it is pointed out that tens of thousands of needless surgeries are performed each year in the United States? That ineffective drugs, often with serious side effects, get to market due to fraudulent manipulation of clinical trials? That thousands of deaths each year are due to medical errors? It would be much easier to argue against CAM if sbm cleaned up its act.
A good source for an unbiased evaluation of the efficacy of many medicines and surgeries is https://www.thennt.com/home-nnt/ (they give lots of literature sources)
This is operated by Oxford University as the UK Center for Evidence Based Medicine. Both benefits and harmful side effects are presented in a simple numerical format. Their evaluation of Statins for people who have not had a heart attack is particularly interesting. In agreement with their evaluation, and after reading the literature, I refused Statin treatment.
Excellent
… how many people get prescriptions for “conditions” which are dubious, I don’t know….
It makes it easy to remember, as with average 60 % efficacy and 80 % diagnosis correctness [an astoundingly high number IMHO], likelihood is 50 % to get help with a putatively medical condition. A coin throw, really.
[But much better than before evidence based medicine, of course!]
I should perhaps add that I’ve used this estimate for years, and so I’m pleased [in a way] that it still holds up.
This comment suggested to me to share these two excellent general audience treatments of Bayes’ theorem :
1. https://opinionator.blogs.nytimes.com/2009/05/22/what-are-the-odds/
Great general demo / discussion of Bayes’ theorem in medicine etc.
2. Grant Sanderson on Bayes’ factor : https://youtu.be/lG4VkPoG3ko