Last night on the NBC News (and also on the same station the night before) I heard a report on a new cancer drug touted as being almost miraculous. The drug was called daraxonrasib, was described as working by blocking a mutated promoter of tumor growth in people with metastatic pancreatic cancer—a notoriously fatal disease (the median survival period after diagnosis of this stage is about 3-6 months, and the five-year survival rate is 3.2%). But the news confused survival time with survival rate, saying something like “the drug doubles the survival rate. . . .from 6 months to 13.2 months”. (I may have gotten the figures wrong as I’m working from memory.) I knew that something was wrong, as metastatic pancreatic cancer is almost always fatal, so the survival rate, which the percentage of people still alive after a specified period of time (often five years), cannot be expressed in months.
Sure enough, this mistake, expressing the effects as a doubling of survival rate, was not only misleading, but widespread. It’s easy to find similar errors in the press; just google the drug name and “survival rate”:
From CBS News (click all screenshots to read):
An excerpt (all excerpts are indented). I’ve put the confusing bits in bold:
A new, experimental medication nearly doubled overall survival rates for patients with advanced pancreatic cancer, according to the results of a study published Sunday.
Researchers say the findings are a significant marker of progress toward treating a notoriously deadly type of cancer, for which there have historically been limited effective options for therapies.
The drug is called daraxonrasib and it blocks a mutated protein that fuels tumor growth in more than 90% of pancreatic cancer cases — a target that had eluded treatment for decades.
“While not curing the cancer, it is a very large step forward,” said Dr. Zev Wainberg, of the University of California, Los Angeles, who helped lead the study.
The research team found that taking the medication, as a daily pill, reduced the risk of death by 60% for patients with metastatic, or spreading, pancreatic cancer who had previously received treatment. That was compared with survival rates of patients receiving standard chemotherapy, according to UCLA Health.
It randomly assigned the experimental drug or more chemotherapy to 500 patients whose metastatic cancer had quit responding to prior treatment. The findings were published in the New England Journal of Medicine and presented Sunday at the American Society for Clinical Oncology meeting in Chicago.
Those taking daraxonrasib lived for a median of 13.2 months compared with 6.7 months for chemotherapy recipients. While that may seem like a small improvement, Wainberg said it marked the first drug to show a substantial advantage over chemotherapy.
Note that while CBS says that it reduces the risk of death by 60%, there are NO DATA showing that. The risk of death is again nearly 100%, though survival time increases by a bit more than two. Also, “survival rates” have not been doubled. There are no data on that, at least not in the article.
From USA Today:
Excerpt:
An experimental drug nearly doubled the overall survival rates of pancreatic cancer patients, according to the results of its latest clinical trial.
The drug, daraxonrasib, targets the gene mutation behind most pancreatic cancer diagnoses.
In the phase 3, randomized trial, published in the New England Journal of Medicine on May 31, researchers found patients who received the drug lived a median of 13.2 months compared with 6.7 months for those who received chemotherapy.
They use “rate” but give “times.”
It’s easy to find similar conflations. This one, less excusable because of the venue, is from The Clinical Trial Vanguard:
They give the results correctly but characterize them as showing “death risk”:
A 60% reduction in the risk of death—HR 0.40—in previously treated metastatic pancreatic cancer is not a number the oncology community has seen before, in any phase 3 trial, in any line of therapy. That is the threshold RASolute 302 crossed. Revolution Medicines enrolled 500 patients, randomized them between once-daily oral daraxonrasib and investigator’s choice of standard cytotoxic chemotherapy, and watched median overall survival reach 13.2 months on the experimental arm versus 6.6 months on chemotherapy in the RAS G12 mutant population. Doubling median OS in second-line pancreatic ductal adenocarcinoma, a disease where incremental gains have defined ambition for decades, reframes what the endpoint space for this indication even looks like.
Nope; the chance of dying within a year or two remains about the same, I’d guess.
. . . and a post from someone on Facebook (I won’t give a name), touting a “60% reduction in the risk of death”. That’s wrong: the risk of death is probably still about 100%
The BBC gets it right, however:
This is correct:
A pill has been found to almost double the survival time for advanced pancreatic cancer patients, with experts describing the trial as a game changer.
The drug, called daraxonrasib, appears to be a breakthrough in managing a disease that has the highest mortality rate of all major cancers.
It helps prevent the spread of cancer by locking onto and shutting off the mutated KRAS gene, which is in more than 90% of pancreatic tumours and spurs cancer growth.
The trial, which included 500 patients in North America, Europe, and Asia, found the average survival time for patients on chemotherapy was 6.6 months, compared with 13.2 months for patients on daraxonrasib. It also caused fewer side-effects.
One other point: if “death risk” is meant to say “death risk over the course of the study,” then that might be accurate. But then the journalists must clarify it.
There are two points to be made, and they’re obvious. First, more than a few science/medicine journalists, including some writing on medical websites, don’t understand statistics, mistaking “rate” for “time”. I asked a science-friendly doctor if this mistake is common, and he replied, “All the time. Sometimes, I’m not sure it’s an unintentional mistake.”
Which leads us to the second point: this kind of conflation could provide false hope for cancer patients and their families. Knowing that you will live, on average, 6½ months longer if you take the new drug is a very different thing from knowing that you will still die with near certainty. It’s easy for one to think—and this is what I thought when I heard the teaser on television—that the drug will reduce the chance of dying by half. Seriously, journalists, please brush up on your statistics, for this one is not rocket science!
Good post on the use of survival rate vs survival time terms
The BBC got part of it right, but not this part: “locking onto and shutting off the mutated KRAS gene”. It does not “shut off a gene”. It (daraxonrasib) interferes with the activated RAS protein in a novel way. It does not affect gene expression, but it does affect protein function in cell signaling.
Pancreatic cancer is so difficult to treat that a significant increase in survival time is good news for some patients. It’s certainly too early to learn if appropriate treatment with this drug will lead to extended remission. A lot of us have personal experience involving friends or relatives who have succumbed to or are afflicted with pancreatic tumors.
Pancreatic cancer is a horrible disease, my wife was diagnosed in October of 2022 and lost her battle at the end of November 2022. Often by the time you have symptoms it is already stage 4. Even with these misleading statistics, I wish I had the chance to spend an extra 6 months with her.
Sending a big virtual hug, Lee.
Some people do survive it, musician Chris Rea for example. (He is now dead, but of a different cause.)
I see this kind of thing all the time, journalists saying stuff that doesn’t even make sense when you read it carefully. I often wonder if they even know what they are saying. What I do hope is that the researchers and the people charged with drug approvals know how to interpret data.
We had a dear friend suffer from the disease for three years, before finally dying peacefully. My condolences go out to Lee Beringsmith (above), who had to endure tragedy at the hands of pancreatic cancer. The reporters may have made a mess of things, but if medical researchers are applauding the research, maybe there’s hope.
I’m sure the ASCO audience, who were enthused, knew exactly what they were seeing. As you say, it’s journalists (possibly willfully) misinterpreting the data (to generate more clicks or hits?) that are the issue.
Overall though, there is slow movement on targeting some of these oncogene pathways, such as RAS in this case, that had long been considered undraggable. Closer to my own field progress is being made targeting the Mac oncogene in prostate cancer.
This is how we make progress, one small painful step at a time. There are few “magic bullets – HPV vaccine and the HPV-driven tumors such as cervical, perhaps being one exception). Eventually (hopefully) you can look back and see that what was a death sentence in the 1970s is now survivable, although still serious (a number of childhood cancers are in this group). It might be several decades yet before we can do this with pancreatic cancer.
Noticed the same thing. Personally I would not get that excited about an extra six months, much less an extra six months of likely ill health waiting to die. These stories drive me crazy. It’s why I pay literally no heed to any health stories about great new drugs to either cure cancer or make us live to 100 or whatever.
I agree with your doctor friend that the mistakes aren’t always done accidentally. Sometimes they’ll get done on purpose for clicks. The truth is nobody would care about this story if they understood it immediately. “New cancer drug barely makes any difference” is not catchy. However most media sites have editors too overworked and/or badly educated and inexperienced to catch it.
And another part of the problem is university and drug company PR releases that radically oversell the accomplishments, too.
Glad you mentioned the university/drug company PR releases.
They are often intentionally misleading to improve chances of publication in higher impact journals.
Where you stand depends on where you sit. The drug won’t be marketed to people who don’t have pancreatic cancer and who don’t think six months extra survival is worth anything. It will be marketed to patients who do have pancreatic cancer and who do value the extra time, as Mr. Beringsmith’s wife probably would have with him. (Walking down the aisle with one’s daughter is the cliché.) They always hope/imagine that they will beat the odds and live much longer than average, and half of course will. Follow-up stopped at 18 months which was long enough to prove biological effectiveness. The actual survival of the patients whose disease responded will be longer than 13 months. Remember average survival includes patients who dropped dead immediately after starting treatment and those where disease progression occurred relentlessly in the first month. A patient who was showing a response at, say, three months, will almost certainly go longer than average because she’s had a trial of life, as we say.
The point of a cancer clinical trial is to test if a new drug works in some meaningful way, such as clearly prolonging survival, not to gauge a precise estimate of how much extra survival the average patient gets. That can’t be determined until all the patients in the trial have been followed to death. (The FDA will probably want to see that before licensing.) Remember with standard treatment there are 3% or so still alive at 5 years. We don’t know, can’t know, from this study that the 5-year survival in the good responders won’t be 5% or 10%. Not the way to bet, but some people do hit 17 at blackjack or draw to an inside straight. And if all you’re betting is the insurance company’s money and the drug doesn’t make you (too) sick, why not go for it?
For those who enjoyed a good response — median or better — their personal benefit is not diminished by the non-responses that other patients got on the left side of the median who were going to die quickly anyway. I guarantee you everyone with newly diagnosed pancreas cancer will clamour for it, except those who opt to go straight to hospice after receiving the diagnosis (which is probably what I would do. I don’t want to burden or impoverish my wife.). They might not want (or be able) to pay for that time out of their pockets but they will demand the insurance companies do. Blockbusters aren’t cheap.
The confusion about reporting survival rates and risk of death is well-taken. But prolonging the median survival from 6.7 to 13.2 months (with follow-up censored at 18 months) is the real deal. Worth a standing ovation.
You would be excited if you got six more months after being diagnosed. Plus, this is the first drug that shows any meaningful effect on pancreatic cancer and is likely a window into future treatments. Cancer treatments get refined constantly and one would expect this result to lead to further improvements in mean survival times.
Unfortunately, news outlets could care less when a random person without scientific credentials (like me for example) points out to them that they made a factual error. Especially in matters they think I have no business knowing about. I once pointed out to “United Nations News” that a change of 10 degrees Celsius is NOT equivalent to 50 degrees Fahrenheit, but 18. Crickets. Why spoil the clickbait value of the article?
Jerry, I read press releases and press accounts like the ones that set you off as saying “send money.”
In this case, I wonder whether buying people who have pancreatic cancer 6.6 months is worth the expense and the pain (if any, and my friends who were killed by pancreatic certainly had pain).
I’ve lost friends to invariably highly lethal cancers. Most of them wanted every plausible treatment, and some implausible ones, available. None died by their own hands or by assisted death. The one exception was my mother, who received, at the age of 94, a diagnosis of a rapidly advancing and fatal cancer. She refused treatment.
While it is confusing, I don’t think what some of these reports say is completely wrong. A hazard ratio of 0.4 can be interpreted as the “risk” of death decreasing by 60%. This risk is “instantaneous risk”, and not a 5-year (or 1 or 2-year) risk of death. You may say that someone might care less about the instantaneous risk and care more about survival probability over a certain amount of time, but technically this interpretation is not wrong.
As other have said, it also matters how miserable one is during that extended time of life, which is not captured by either of the two measures (survival probability or hazard).
P.S. Hazard is technically not a probability. So, there can be an argument against using the word “risk”. If I were reporting this, I would just say hazard or “instantaneous risk” to avoid confusion.
Jerry writes: “They give the results correctly but characterize them as showing ‘death risk’:
The correct interpretation of the hazard ratio, HR=.40, is that the rate of death was reduced by 60% in the treatment group compared with control group. The error was calling it “risk.” Rate is risk per unit time.
Jerry: “Nope; the chance of dying within a year or two remains about the same, I’d guess.”
Since the median survival time was 13.2 months for the treatment group, 1-year survival must have been greater than 50%, which we would expect to be much greater than 1-year survival in the control group, whose median survival was only 6.6 months.
In fact, the paper shows 1-year survival of 53.3% of the treatment group compared with only 18.7% of the control group, indicating that survival at one year was near tripled by treatment relative to control. Two-year survival isn’t given, as maximum follow-up in the study was only 18 months.
This is the right way to report it. In cancer clinical trials, there are numerous ways to report such data using measures such as Kaplan-Meier plots or disease-free progression meant to show statistically meaningful differences. There is a lot of miscommunication around these statistical methodologies and even nuance that isn’t correctly communicated to the public. 60% reduction of death for daraxonrasib is a huge effect for a single agent in highly fatal pancreatic cancer and is highly statistically significant – never seen before. Exciting too since this drug is a new drug modality called a “molecular glue” which opens the door for even more efficacious versions in the future and synergistic drug combinations tailored to specific cancers with specific RAS mutations. Side effects of this treatment aka “tolerability” is also important to consider and discontinuations due to them during the trials. Then there is real world performance of drugs in the general population, not as carefully controlled as in clinical trials, with far greater statistical numbers to reflect true performance with respect to average survival and risk-benefit determination.
Unfortunately, there is a disease that is very common in society, and that is innumeracy.
(Please do not think that I am writing with disrespect or making light about any diseases mentioned in this thread: I have dedicated much of my career to developing treatments for diseases that have no present treatments, and I am sorry for the losses caused by those and other diseases.)
Innumeracy is a major cause of the misunderstandings of the general public about disease, prognoses, and general survival. And one major cause is the health reporting in the popular press. Looking at the curriculum of journalism schools, how many include requirements for basic mathematics, or a course in statistics and probability? How many reporters have those tools? One would think that reporters in the health and science fields have a basic understanding of those topics. Some few, especially those reporting on physics advances, have at least a basic proficiency, but most do not.
Often, on this and other sites, we discuss the fact that much of the media prefers advancing their agendas to honest reporting. Understanding mathematics is an important tool of honest reporting. Innumeracy can be a tool of those who do not care about facts, and lying about numbers and statistics (as opposed to simple misunderstanding), i.e. using innumeracy of the audience, is the tool of dishonest journalists with an agenda.
I’ve lost friends and family members to this disease. Years ago a colleague was dying of it and came to his office one last time. He died a few days later but was in a good mood this day, probably due to opioids. He told me he was “facing death”, to which I replied “we all are, you just have a timeline”. He laughed heartily at that.
Noticed the same thing. Personally I would not get that excited about an extra six months, much less an extra six months of likely ill health waiting to die. These stories drive me crazy. It’s why I pay literally no heed to any health stories about great new drugs to either cure cancer or make us live to 100 or whatever.
I agree with your doctor friend that the mistakes aren’t always done accidentally. Sometimes they’ll get done on purpose for clicks. The truth is nobody would care about this story if they understood it immediately. “New cancer drug barely makes any difference” is not catchy. However most media sites have editors too overworked and/or badly educated and inexperienced to catch it.
And another part of the problem is university and drug company PR releases that radically oversell the accomplishments, too.
I’m glad you wrote this. I viewed the first commercial & shouted “no” at the screen, as I heard the error immediately. I’ve written on the misuse of science myself in a blog post & in a journal. Neither, I expect, will be seen by the general public, who are the ones who need the information.
I haven’t gone back and checked your sources, but it sounds like this is not merely an extra 6 months, but an extra 6 months without chemotherapy. In fact, the whole 13-month survival period is without chemotherapy. That makes huge difference to quality of life.
So yeah, bad reporting, no surprise, but there is a kernel of real good news here.
Since the drug hasn’t been licensed for sale yet, we don’t know how much it might sell for. This industry watcher
Jerry is correct. So are just about all the commenters.
Unless I am missing something, the patients in the daraxonrasib studies were all pretreated. As doctors learn the best way to use daraxonrasib, we may or may not see some cured patients. One hopes it will be like the early childhood leukemia advances which at first just gave the doomed children a little more time, but eventually led to durable responses. Maybe some of the doctors who cheered were thinking of that.
My point was that the reporting about what the drug did was at best confusing, and at worst misleading. At least journalists should report what was found. I would be delighted if this drug led to read, permanent cures but that was not my point.
Not directly related (but also involving cancer and statistics): Stephen Jay Gould’s essay (and great pun) “The median isn’t the message”, on another often misunderstood cancer-related statistic. Whatever one thinks of Gould’s evolutionary biology, or Marxist politics, or NOMA, he was really good at explaining common misconceptions regarding statistics.
https://people.umass.edu/biep540w/pdf/Stephen%20Jay%20Gould.pdf