Covid response hampered by population data glitches

In less than a year, almost 80 countries worldwide have managed to offer all eligible citizens a Covid-19 vaccine and are on the verge of hailing completion of their primary vaccination campaigns. But all is not as it seems.

The reliability of the population data on which public health decisions are based has recently been thrown into sharp relief, with experts warning that anomalies in how vaccination rates are calculated could lead to policy missteps and provide fodder for vaccine sceptics.

It turns out many countries do not know how many residents they have, what proportion of eligible people have come forward for the jab and how many remain unvaccinated.

“The average person would be surprised that governments don’t know how many people are actually in the country,” said Stian Westlake, chief executive of the UK’s Royal Statistical Society. “But this great unknown can cause a whole host of data glitches, especially when responding to a health emergency.”

What are the anomalies in the data?

The fundamental problem is that countries’ denominator data — the estimate of their total population from which they calculate vaccination rates — are imperfect and inconsistent.

Misleading data from the UK’s Health Security Agency, first published in September, showed infection rates among fully vaccinated people aged 40 and above were higher than among unvaccinated counterparts.

The report did warn the data sets “should be interpreted with extra caution” because the chosen population denominator, based on the National Immunisation Management Service (Nims) records — which rely on lists of patients registered with family doctors — could skew the vaccination rates.

Nims was “never actually designed as a population estimate”, said David Martin, co-director of the Economic and Social Research Council’s UK Data Service.

Using it could lead to people being missed or double-counted because they were either not registered with a doctor or were listed at more than one, he said, adding: “University students, recent migrants and people not registered with primary care are obvious examples.”

When the same Covid-19 caseloads are measured against Office for National Statistics population estimates — which are based on census data readjusted for recent births, deaths and migration — infection rates are higher for unvaccinated than fully vaccinated people in all age groups except the over-80s. The figures for that age group are themselves likely to be skewed because very few in the cohort are unvaccinated and vaccinated people are more likely to get tested, say experts.

Elsewhere in Europe, some countries have recorded vaccine uptake of more than 100 per cent of their official population in certain cohorts.

The number of first doses administered exceeds the reported population size among people over 60 in Ireland and Portugal and among over-80s in Austria, Denmark, Spain, Iceland and Malta, according to data from the European Centre for Disease Prevention and Control (ECDC).

Edouard Mathieu, head of data at research project Our World in Data, told the FT it was “frustrating” that rich countries could “mess this up”.

“For some countries, this could be explained by people going abroad, visiting their children and getting vaccinated [while there], but . . . that can’t possibly apply to all countries,” he said, adding that the denominator was “wrong” but choosing the right one was “a very thorny question”.

Conversely, the Robert Koch Institute, Germany’s main public health authority, said last week it had probably undercounted vaccine uptake. New survey data suggested some inoculations had not been correctly recorded and those vaccinated abroad had been overlooked. The revised figures increased Germany’s full vaccination rate from 80 per cent to 84 per cent of those eligible, it said.

In Florida, unreliable vaccination rates have “become a theatre of the absurd”, according to Dan Gelber, mayor of Miami Beach.

In one Florida zip-code, vaccine uptake rates reached 3,512 per cent of the local adult population. Another 50 zip-codes in Miami-Dade county, more than half of the total, show uptake of more than 100 per cent among over-65s.

Gelber speculated that the faulty data could be a consequence of vaccine tourism or the so-called “snowbird effect”, in which retirees move to Florida for the winter, meaning large numbers of non-residents appeared in the records.

What problems is this causing?

Public health experts and statisticians warn imprecise vaccination data have hampered pandemic management and fanned the flames of misinformation.

“Imperfect data at the very least means imperfect decision-making,” said Westlake.

In England, the misleading UKHSA data were seized on by anti-vax groups online.

In Florida, the high number of people in the state assumed to be protected against infection by the jabs “created a false sense of security” among the unvaccinated who believed they did not have to worry because others had done their part, said Gelber, adding: “Complacency was definitely a factor [in Florida’s Delta wave].”

Jennifer Nuzzo, senior scholar at the Johns Hopkins Center for Health Security, said US healthcare and population data suffered from “major blind spots” caused by states not sharing data effectively.

“The reality is that we don’t actually have as high a level of coverage as we think and that could undermine confidence and further hinder leaders’ abilities to encourage people to get vaccinated,” said Nuzzo.

“Worst of all, anti-vaxxers and Covid deniers feed on the daylight between reality and the incomplete data we currently have as evidence of a grand conspiracy or bureaucratic incompetence.”

How are health authorities responding?

An ECDC spokesperson told the FT they were “very concerned” about the quality of information displayed in the organisation’s vaccine tracker, which is leading to vaccination rates in excess of 100 per cent.

They said the ECDC was considering changing the calculation behind the uptake rates by using “customised denominators from national sources on a country by country basis”.

German health minister Jens Spahn said last week his country’s revised vaccination figures meant the nation was better prepared for the winter and could “do without” mask-wearing outdoors.

Anthony Masters, a member of the UK’s Royal Statistical Society, said the increased scrutiny on population data could revive calls for identity cards in some countries.

Some nations, such as Slovenia and South Korea, have benefited from compulsory ID card schemes linked to healthcare data, giving them more accurate insights into population dynamics and vaccination rates.

“There are some countries that have proper, full-on population registers. In the UK, for various political reasons, there is no such register,” said Masters. “If you are a number in a state computer, that makes it a whole lot easier to produce more accurate representations of the population and consequently work out vaccination rates and [make] a load of other data-based policy decisions.”