The most popular mathematical fallacies sport fruity names such as the ever-popular cherry-picking of data which means selecting numerical information that supports your point but ignoring data and statistics that might weaken your argument. This is the fallacy the New York Times charges President Trump of committing repeatedly at a recent press conference.
Another produce-related “crime” arguers commit when using numbers and statistics is comparing apples to oranges, which means making an analogy between unlike things. While this phrase doesn’t apply only to numerical arguments, this opinion piece from Scientific American (reproduced at the end of this post) makes the case that a different Trump statement – which compared COVID deaths to annual deaths due to seasonal flu – represented inaccurate comparison of an apple to an orange.
During an early week in the COVID outbreak in the US, Trump commented that he was shocked to learn that between 25,000 and 69,000 people die every year due to the flu, a statistic he obtained from the same Centers for Disease Control (CDC) that has been so central to tracking Coronavirus statistics.
Given that, at the time, it was estimated that COVID might kill close to 70,000 Americans, Trump was trying to reassure the public that this epidemic was no worse than a disease which kills a comparable number of people each year (the flu) which does not require shutting down the nation.
The author of the Scientific American blog, Dr. Jeremy Faust, was perplexed by this comparison, not because he was unfamiliar with the 25,000-69,000 flu death statistic the president was using, but because this huge number did not comport with the reality he experienced as a physician during which he had only seen one flu death during all his years in medicine.
Because his own time on the front lines was limited (less than eight years) he reached out to other physicians to confirm that they too had rarely seen a flu death, compared not just to Coronavirus deaths that were visible (and countable) by everyone, but also to other types of deaths from opioid overdoses and gun violence (which CDC reports as each close to or topping 40,000 in 2018) which he and other physicians see all the time.
This caused him to look more closely at how the CDC compiles its flu statistics where he discovered that the 25,000-69,000 per year death rate was not the result of a physical count of victims, but rather a statistical estimate based on a number of assumptions that may have caused CDC to inflate the number of annual flu deaths by more than 500%.
During normal times, there was social benefit to public belief that the flu was more deadly than it actually was, given that fear of such a lethal virus might cause people to take prevention measures like personal hygiene more seriously. But during the time of COVID, this misinformation led to a minimizing of the seriousness of the present pandemic since it creates a misconception not just for Trump but also for other leaders and citizens that today’s apple (COVID-19) could be compared to a flu orange that allegedly killed comparable numbers without requiring serious preventative measures such as social distancing and lockdowns.
Given this, Dr. Faust recommends that CDC revise its flu reporting methodology to better reflect reality, rather than continuing to rely on flawed statistical models that inflate flu death stats, especially now that we know erring on the high side does in fact have a major public downside.
There are several things I love about this argument, which is why I have given it the coveted five-dumbbell rating for a news LogicChecker:
(1) The argument makes a compelling case that current methods for reporting flu statistics are flawed, starting with his own experience and that of other doctors, then diving into how a stat so at odds with that experience came about.
(2) His discovery is an excellent illustration of what I described here and here regarding how "perfect" numbers become very imperfect indeed when applied to messy and complex reality.
(3) The argument’s conclusion, that CDC should modify its approach to generating flu statistics, is a perfect example of something we can do to improve things in the future, making this a classical deliberative argument.
(4) While the writer uses Trump’s misconception as a starting point, he does not accuse the President of intentionally misleading the public, but rather points out that Trump’s misunderstanding was based on figures many people (including Dr. Faust himself) assumed were correct until he looked at them more closely.
Faust’s non-partisan approach allows his argument to appeal to people across the political spectrum since he is not using his discovery to condemn Trump (or anyone else, including CDC) but instead suggesting a way to ensure all of us do not labor under misperceptions that can cloud important decision-making if and when the next major health threat occurs.
So Bravo Dr. Faust! 5/5 dumbbells for you!
Original Article
Comparing COVID-19 Deaths to Flu Deaths Is like Comparing Apples to Oranges
Dr. Jeremy Samuel Faust, Scientific American (April 28, 2020)
In late February, when the stock market was beginning to fall over coronavirus fears, President Donald Trump held a briefing at the White House to reassure people that there was little chance of the virus causing significant disruption in the United States.
“I want you to understand something that shocked me when I saw it,” he said. “The flu, in our country, kills from 25,000 people to 69,000 people a year. That was shocking to me.”
His point was to suggest that the coronavirus was no worse than the flu, whose toll of deaths most of us apparently barely noticed.
In early April, as social distancing measures began to succeed in flattening the curve in some parts of the country, an influential forecasting model revised the number of American deaths from coronavirus that it was projecting by summer downward to 60,400, and some people again began making comparisons to the flu, arguing that, if this will ultimately be no worse than a bad flu season, we should open the country up for business again. (On April 22, the model’s forecast rose to 67,641 deaths.)
But these arguments, like the president’s comments, are based on a flawed understanding of how flu deaths are counted, which may leave us with a distorted view of how coronavirus compares with it.
When reports about the novel coronavirus SARS-CoV-2 began circulating earlier this year and questions were being raised about how the illness it causes, COVID-19, compared to the flu, it occurred to me that, in four years of emergency medicine residency and over three and a half years as an attending physician, I had almost never seen anyone die of the flu. I could only remember one tragic pediatric case.
Based on the CDC numbers though, I should have seen many, many more. In 2018, over 46,000 Americans died from opioid overdoses. Over 36,500 died in traffic accidents. Nearly 40,000 died from gun violence. I see those deaths all the time. Was I alone in noticing this discrepancy?
I decided to call colleagues around the country who work in other emergency departments and in intensive care units to ask a simple question: how many patients could they remember dying from the flu? Most of the physicians I surveyed couldn’t remember a single one over their careers. Some said they recalled a few. All of them seemed to be having the same light bulb moment I had already experienced: For too long, we have blindly accepted a statistic that does not match our clinical experience.
The 25,000 to 69,000 numbers that Trump cited do not represent counted flu deaths per year; they are estimates that the CDC produces by multiplying the number of flu death counts reported by various coefficients produced through complicated algorithms. These coefficients are based on assumptions of how many cases, hospitalizations, and deaths they believe went unreported. In the last six flu seasons, the CDC’s reported number of actual confirmed flu deaths—that is, counting flu deaths the way we are currently counting deaths from the coronavirus—has ranged from 3,448 to 15,620, which far lower than the numbers commonly repeated by public officials and even public health experts.
There is some logic behind the CDC’s methods. There are, of course, some flu deaths that are missed, because not everyone who contracts the flu gets a flu test. But there are little data to support the CDC’s assumption that the number of people who die of flu each year is on average six times greater than the number of flu deaths that are actually confirmed. In fact, in the fine print, the CDC’s flu numbers also include pneumonia deaths.
The CDC should immediately change how it reports flu deaths. While in the past it was justifiable to err on the side of substantially overestimating flu deaths, in order to encourage vaccination and good hygiene, at this point the CDC’s reporting about flu deaths is dangerously misleading the public and even public officials about the comparison between these two viruses. If we incorrectly conclude that COVID-19 is “just another flu,” we may retreat from strategies that appear to be working in minimizing the speed of spread of the virus.
The question remains. Can we accurately compare the toll of the flu to the toll of the coronavirus pandemic?
To do this, we have to compare counted deaths to counted deaths, not counted deaths to wildly inflated statistical estimates. If we compare, for instance, the number of people who died in the United States from COVID-19 in the second full week of April to the number of people who died from influenza during the worst week of the past seven flu seasons (as reported to the CDC), we find that the novel coronavirus killed between 9.5 and 44 times more people than seasonal flu. In other words, the coronavirus is not anything like the flu: It is much, much worse.
From this perspective, the data on coronavirus and flu actually match—rather than flying in the face of—our lived reality in the coronavirus pandemic: hospitals in hot spots stretched to their limits and, in New York City in particular, so many dead that the bodies are stacked in refrigerator trucks. We have never seen such conditions.
In that briefing in late February, Trump downplayed the likelihood that the virus would spread significantly in the United States and that extreme measures like closing schools would need to be taken, saying that “we have it so well under control” and returning again to the flu.
“Sixty-nine thousand people die every year—from 26 to 69—every year from the flu,” he said. “Now, think of that. It’s incredible.”
We now know that Trump was disastrously wrong about the threat that the coronavirus posed to the United States. But his take that the cited numbers of flu deaths were incredible? On that, he was spot-on.