Commentary: Measuring excess mortality due to the COVID-19 pandemic: progress and persistent challenges

Since the beginning of the COVID-19 pandemic, health authorities worldwide periodically report COVID-19 cases and deaths. These ‘official’ counts are aggregated primarily from data compiled by laboratories and hospitals,¹ and they document, imperfectly, the effects of COVID-19 on population health.^2–4 Case counts are partial due to limited testing⁵ and death counts might miss COVID deaths that occur outside of health facilities.⁶ These figures also exclude non-COVID deaths which are indirectly a result of the pandemic (e.g. because of disrupted health services).⁷

The total effects of the pandemic are more reliably captured by the concept of ‘excess mortality’. This is the difference between the number of deaths (from any cause) that occur during the pandemic and the number of deaths that would have occurred in the absence of the pandemic. Many analysts consider excess mortality as ‘the most objective possible indicator of the COVID-19 death toll’.⁸

Four recent papers in the International Journal of Epidemiology used this counterfactual approach to paint a nuanced picture of the impact of COVID-19.^9–12 In 2020, countries like Japan¹² and Denmark¹⁰ experienced lower than expected mortality, whereas deaths have increased substantially in the USA, UK and other countries.^9–11 Excess mortality has affected older adults disproportionately, but increases in mortality among people of working age were observed in the USA and elsewhere.⁹ Gender differences in excess mortality have varied across countries. Excess deaths were also concentrated in socioeconomically disadvantaged groups.¹¹

Data from systems that continuously record all deaths, irrespective of their cause(s), testing status and the place where they occurred, are essential in investigations of excess mortality. The four recent International Journal of Epidemiology papers relied on civil registration—the administrative systems that aim to record all vital events. In Ecuador, excess deaths estimated from civil registration data were five times larger than the number of reported COVID-19 deaths.¹¹ Similar discrepancies between excess mortality estimates and official counts of COVID-19 deaths have been documented elsewhere.¹³

Measuring excess mortality requires comparing weekly or monthly counts of deaths with the expected number of deaths in that same week or month. In some studies, this baseline value is computed as the average number of deaths in pre-pandemic years (e.g. 2015–19).^10,11 More refined models also account for the seasonality of deaths¹⁰ and long-term trends in population size and mortality,¹⁴ as well as other potential confounders. Onozuka et al. thus controlled for weather patterns and influenza activity that might have prompted short-term mortality fluctuations in Japan, independently of the COVID-19 pandemic.¹² Several research groups have used an ensemble approach that combines estimates from a large number of statistical models with different covariates and specifications.^15,16

International comparisons of excess mortality might be complicated by cross-country differences in population characteristics. Analysts frequently use P-scores (i.e. the percentage by which observed deaths exceed expected deaths) to account for differences in baseline mortality.¹⁴ Aburto and colleagues also elegantly controlled for differences in population age structures.⁹ They translated age patterns of excess mortality into changes in life expectancy at birth—the average lifespan a fictitious cohort would achieve if its members experienced the age-specific death rates in a given year through their entire (hypothetical) life course. Using this metric, they compared the magnitude of mortality shocks prompted by COVID-19 between disparate countries, and across historical periods.

Analyses of excess mortality have great potential to help understand what works in mitigating the effects of health crises. In Ecuador, public health and social measures (e.g. social distancing) preceded sharp declines in excess mortality.¹¹ By comparison, these interventions were weakly associated with trends in official counts of COVID-19 deaths, possibly because increased testing led to improved case detection. Achilleos et al. found high excess mortality in countries that delayed COVID-19 control measures.¹⁰ Conversely, countries with lower than expected mortality might provide lessons in COVID-19 control and in the prevention of conditions indirectly affected by the pandemic (e.g. respiratory diseases such as influenza).

Data limitations might affect excess mortality measures. Some deaths are only registered after delays, prompting repeated updates of excess mortality figures. Civil registration systems also miss some vital events.¹⁷ In high-income countries, unrecorded deaths are few and do not affect most estimates. Elsewhere, this issue is more pervasive. In Ecuador, for example, only two-thirds of deaths might be registered.¹⁸ Analysts often assume that excess mortality estimates are accurate despite such data gaps if the completeness of death registration stays constant over time.¹¹ This reasoning only applies to the ratio of recorded to expected mortality, not to the number of excess deaths. If civil registers are incomplete, the true number of excess deaths will be larger than observed by a factor equal to the inverse of the proportion of registered deaths. Accurate excess mortality estimates require careful assessments of the completeness of civil registration.^19,20

Data challenges are compounded when the completeness of registration varies between groups and/or over time. In Ecuador, death registration rates prior to COVID-19 varied from less than 30% to more than 90% across provinces.¹⁸ During the pandemic, some countries suspended registration services for extended periods.²¹ Where registration continued,²² mobility restrictions and fear of infection might have prevented visits to registration offices. Yet in other places, registration rates might have recently increased due to new reporting systems or greater public attention on death reporting.²³ In many settings, fluctuations in the number of recorded deaths might thus reflect changes in the completeness of death registration, as well as excess mortality. In some cases, drops in registration rates might even hide the occurrence of excess deaths.

Several methods exist to adjust incomplete civil registration data. Unfortunately, commonly used death-distribution methods²⁴ require data that may not have been collected recently in many countries (e.g. census data). They also make unrealistic assumptions about migration and other recent population dynamics. The recent Adair-Lopez method uses more widely available data inputs, and relaxes several implausible assumptions.²⁵ However, this method relies largely on child mortality estimates to infer the completeness of death registration. Its assumptions might thus be less robust during a pandemic like COVID-19 that disproportionately affects mortality among older people. Sensitivity analyses and robustness tests should accompany estimates of excess mortality obtained from incomplete death records.

Some high-income and upper-middle-income countries are now accelerating the release of provisional data on all-cause mortality,²⁶ but data on excess mortality are lacking for large parts of the world. Among 39 countries investigated in the recent International Journal of Epidemiology papers,^2–4 more than 30 were in Europe versus four in Latin America, one in Africa and none in the Middle East or South Asia. Similar gaps exist in comprehensive excess mortality databases^8,27 because civil registration systems in many low- and lower-middle-income countries (LLMICs) are too incomplete to produce vital statistics.¹⁷ Instead, outdated snapshots of mortality in LLMICs are obtained every few years, when retrospective data are collected during household surveys or decennial censuses. This constrains the ability of LLMICs to document how health crises affect their populations and might sustain beliefs that they have been ‘spared’ by the COVID-19 pandemic, and thus require fewer resources (e.g. vaccines) than wealthier countries.²⁸

Improving the recording of deaths in civil registration systems is one of the indicators of progress towards the sustainable development goals. Obstacles to death registration in LLMICs include, for example, inadequate legal frameworks, insufficient operating budgets, difficulties in accessing registration offices, imperfect coordination between governmental agencies, and limited knowledge about death registration among families.¹⁷ Recently, interventions that engage health workers and other community-based agents and equip them with improved digital tools have been successful in rapidly increasing death reporting in several LLMIC communities.²⁹ Policies that jointly promote the availability of registration services and awareness about civil registration might also be well suited to further accelerating progress towards universal death registration.³⁰ Future global pandemic preparedness plans should include large investments to support and expand similar efforts to strengthen civil registration systems in LLMICs.^21,28

Funding

This work was supported by grants from the US National Institute of Aging (R03AG070660), the Eunice Kennedy Shriver National Institute of Child Health and Human Development (R01HD088516) and the Brazilian National Council for Scientific and Technological Development (CNPq 303341/2018-1).

Author contributions

S.H. and B.L.Q. conceived of the paper, drafted and subsequently revised the text. Both authors approved the final version.

Conflict of interest

None declared.