The gap I mentioned last time between perceptions of perfection people invest in numerical information and the grittier reality numbers possess on planet earth is particularly prominent when dealing with issues that hinge on rates and percentages.
For example, back when I worked in business, people routinely used measures of success such as “our sales percentage growth is up 20%!” “Impressive,” I thought, assuming that meant sales had grown by twenty points from 10% to 30%. Imagine my disappointment, then, to discover sales had only grown from 10% to 12%.
How did this “triumph” go from tripling to measly growth of just 2%? Well that 20% growth rate was applied to the original 10% (20% of 10% is 2%), in other words it used ambiguity over what number that 20% rate was being applied to in order to make modest progress look like spectacular success.
Today, rates and ratios are playing a far more central role for all of us as decisions regarding how we get to live during the age of COVID-19 are based on statistics, including totals and rates of infection. Daily reports of these values, as well as other stats such as those that measure hospitalizations and deaths, appear in newspapers, government alerts, even The Weather Channel. And they are telling us something. But what?
For example, infection rates seem like a pretty straightforward fraction we can all understand, one that has the number of positive COVID test results in the numerator and the total number of tests given in the denominator. What’s there to debate?
Plenty, it turns out. For it only makes sense to compare infection rates between populations, or between the same population at different times, if that denominator (number of tests) are comparable. For example, if all state infection rate measures were based on testing the entire population of a state at the same time (or at least the same day), that would give us a genuine (and ideal) basis to make comparisons.
But here on earth, COVID testing is not universal anywhere for all kinds of financial and logistical reasons. This can lead to infection rates that are too low (in fact, this article talks about how low infection rate numbers often signal insufficient testing is taking place). But they can also lead to rates that are too high if, for instance, tests are only given to people showing symptoms of the disease.
That same article talks about other real-world issues that derive from “ideal” numbers meeting grubby reality. For example, daily infection rates can drop considerably on weekends simply because those are days that labs analyzing test results are often closed.
Now government officials and journalists covering the COVID beat understand and take into account such anomalies by, for example, communicating average rates over the previous seven days on a daily basis (rather than just totals for a particular day), which means every weekly average will take into account one weekend dip.
While statisticians and scientists try to control for as many confounding factors as they can, complex measures – like infection rates – are always going to carry some degree of ambiguity until we start testing 100% of the population instantly and on a regular basis, which is likely to happen soon after vaccines start spontaneously growing on trees.
Even straightforward counts, like the number of people who die each day of COVID, may tell us something other than how we are doing today since death tends to follow infection by several weeks (meaning outbreaks may have gotten better or worse since a person who died today contracted the disease).
None of these situations should be treated as invitations to cynicism about vital data we need to fight the pandemic. But they should provide you examples of how numerical data about real-world things inherit all of the messiness of the real world.
So how should we deal with the gap between numbers as ideals vs. numbers as real things when trying to think critically during a pandemic? Next time, we’ll give it a whirl by logic-checking an argument that claims its conclusion is built on sturdy numerical foundation.