Out of How Many?

Before I believe any health statistic, I dig into how many people it counted, like a bouncer checking IDs.

A report might list 500 deaths in the city, or 248 in a small town. Both tally events. They show us how many happened, but not yet how often. At first glance, the city looks like the bigger problem and the town the smaller one. A city of 100,000 with 500 deaths has a death rate of 5 per 1,000 people. A town of 10,000 with 248 deaths has a death rate of 24.8 per 1,000. Suddenly the "smaller" problem looks much larger.

The top of a fraction is the numerator. Underneath that is the denominator, the population that could have had the event. A rate combines them over a stretch of time, usually scaled to something readable like per 1,000 or per 100,000 people per year. A count missing its bottom half can only confirm that something happened, which is about as informative as a group-chat rumor.

I wrote about the numerator side of things in Every New Code Is a New Count. A new diagnosis code counts a condition, but turning them into rates means you need a population to divide by. Billing records often can't supply those by themselves. Epidemiologists spend most of their sweat hunting for a defensible denominator.

Measures of disease occurrence come in many flavors, and people mix up incidence and prevalence all the time. Incidence is new cases arising in a population over a period of time. Prevalence is all the people living with the condition at a given moment, whether they developed it last week or twenty years ago.

Let’s use a bathtub analogy so we can tell them apart. The tap's flow is the incidence, and the water already in the tub is prevalence. The drain gets rid of the people who have either recovered or died.

This matters because the two can go off in opposite directions. Say there’s a better treatment for a chronic disease, keeping people alive longer. Prevalence goes up even while incidence is flat. A short illness like the flu can post high incidence during a bad season and low prevalence on any single day, because people get through the illness quickly. When someone says a condition keeps "rising," ask which measure they used.

Even a correct rate can mislead when two populations differ in age, because that’s the biggest risk factor for major chronic diseases such as cancer, heart disease and dementia. Mortality climbs with it too, so comparing an older population with a younger one often says more about age structure than health risk.

Let’s use a pair of hypothetical towns, each one home to 10,000 residents. In both, anyone under 65 dies at 2 per 1,000 per year, and those 65 and older die at 40 per 1,000 per year. Risk at any given age matches exactly across the two.

Town A is more young, with 8,000 residents under 65 and 2,000 at 65 or older. It records 16 deaths among the younger crowd and 80 among the older one, 96 in total. Divide all deaths by all residents, and its crude death rate comes to 9.6 per 1,000.

Now consider Town B, a retirement community. It flips the mix, with 4,000 residents under 65 and 6,000 aged 65 or older. The same age-specific rates produce 8 deaths in the younger group and 240 in the older one, 248 altogether, for a crude rate of 24.8 per 1,000.

If you judge these by crude numbers alone, Town B looks more than two and a half times as deadly. Moving to either doesn’t raise anyone’s risk; it all depends on who lives there.

As a way to fix this, age adjustment asks what each town's rate would be with the same age mix. Analysts use a standard population and run the age-specific rates through it. Suppose the standard splits evenly, half under 65 and half older. Either town's adjusted rate then works out to half of 2 plus half of 40, or 21 per 1,000. In effect, we've forced both towns to have the same age structure before comparing them.

For planning, the crude rate still wins, since Town B really needs more hospice beds and more funeral directors than Town A. Each number is right for its own question, and the trouble is using one for the other's.

Real reports are more detailed, with more age bands and an official standard population. National agencies comparing states or years usually publish age-adjusted rates for this reason, while local planning documents are more likely to use crude ones. News stories are full of both, often with no label.

The math is all about division, which most of us should have mastered around the time teachers confiscated Pokémon cards at recess. The harder part is checking under the fraction bar before reacting to the figure on top.

Before accepting any health statistic, ask four things: what's being counted, out of how many, over what period of time, and compared to whom. Check whether it's a count or a rate, whether it tracks new cases (incidence) or all the people currently living with the condition (prevalence), whether anyone adjusted for age, and whether a new code or case definition changed the counting overnight. Most misunderstandings begin when one of those answers goes missing.

Next Thursday, just in time for trick-or-treating, I'll put these tools to work on a very common Halloween fear that has nothing to do with zombies or vampires.

Sources: Every New Code Is a New Count; Niccoli T, Partridge L. Ageing as a risk factor for disease. Curr Biol. 2012;22(17):R741-52. The two-town example is hypothetical and constructed for illustration.