Estimates of excess mortality for the five Nordic countries during the COVID-19 pandemic 2020−2021

Abstract

Background

Excess mortality during the COVID-19 pandemic is of major scientific and political interest.

Methods

We critically reviewed different estimates of all-cause excess mortality for the five Nordic countries (Denmark, Finland, Iceland, Norway and Sweden), which have been much studied during the COVID-19 pandemic, using the latest register data to discuss uncertainties and implications.

Results

We show using back-calculation of expected deaths from Nordic all-cause deaths that the Institute for Health Metrics and Evaluation model is a clear outlier in the compared estimates and likely substantially overestimates excess mortality of Finland and Denmark, and probably Sweden. Our review suggests a range of total Nordic excess deaths of perhaps 15 000–20 000, but results are sensitive to assumptions in the models as shown.

Conclusions

We document substantial heterogeneity and uncertainty in estimates of excess mortality. All estimates should be taken with caution in their interpretation as they miss detailed account of demographics, such as changes in the age group populations over the study period.

COVID-19, excess mortality, review, Nordic countries, register-based modelling

Key Messages

• Excess mortality during the COVID-19 pandemic is of major scientific and political importance, yet various methods produce heterogeneous estimates and implications.

• Our critical analysis of methods using the final register data for 2020 and 2021 for the Nordic countries quantifies major heterogeneity and uncertainties, and shows that some results appear inconsistent with the register data.

• Most models suggest total Nordic excess deaths for 2020–2021 of perhaps 15 000–20 000, and more homogeneous reporting ratios and infection fatality consistent with expectations from pandemic management.

• Our study illustrates the importance of systematic data-based critical analysis of excess mortality models in epidemiological research.

Introduction

Excess mortalities (differences between observed and expected number of deaths) during the Coronavirus Disease 2019 (COVID-19) pandemic are of major scientific and political interest, as they estimate the pandemic burden without effects of different testing procedures and registration criteria of COVID-19 deaths between and within countries.^1–5 All-cause excess mortality is of interest for a total evaluation of pandemic impact as it includes not only deaths due to COVID-19 after infection by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), affected by e.g. vaccination, healthcare system competency and population age and health status, but also indirect effects of the pandemic response such as delayed cancer treatment⁶ and other behavioural changes in society. Because of their major importance, detailed and critical review of methods to obtain excess mortality and their implications should be of high priority. Here, we provide such a systematic review of several methods to estimate the all-cause excess mortality for 2020 and 2021 for the five Nordic countries (Denmark, Finland, Iceland, Norway and Sweden).

We studied the Nordic countries because (i) they form a historically and culturally related entity that is highly comparable and often compared;⁷ (ii) they have been much studied during the pandemic, with both criticism (e.g. of Sweden) and claims of successes (e.g. Norway and Denmark);^8–10 (iii) all five countries have high-quality population and healthcare data and final annual all-cause deaths available for 2020 and 2021. For many countries it is not possible to validate model results due to insufficient or incommensurable data.¹¹ Accordingly, our study represents a critical assessment using best-possible complete data for a comparable selection of countries, rather than a broad assessment using less complete and comparable data for less comparable countries.

A paper in The Lancet (Wang et al.) by the Institute for Health Metrics and Evaluation (IHME)¹² concluded that excess mortalities of Sweden, Denmark and Finland were much larger¹² than previously estimated,^10,13 with excess deaths per capita of Denmark (and almost Finland) similar to Sweden, and very large differences in the five countries’ ability to identify COVID-19 deaths, with ratios of excess deaths to official COVID-19 deaths of 3.2 and 5.0 for Denmark and Finland but –8.5 and 0.6 for Iceland and Norway.¹² Although registration criteria are never perfect,¹⁴ these differences are surprisingly large and invite further analysis. Another surprising consequence of the IHME model,¹² when combined with infection estimates, is six to seven times higher infection fatality ratios (IFRs) in Finland and Denmark than in Norway, and almost double that of Sweden.¹⁵ Due to the topic's importance, these major differences warrant scrutiny.

To better understand the different model results, we use the latest official administrative register data to examine the death estimates via linear extrapolation, and we use annual all-cause Nordic death data to back-calculate the expected deaths required (but not reported) for stated excess deaths to be accurate. Additionally, we review the different model estimates and discuss limitations and sensitivities to unusual years and time periods used for estimating expected deaths.

Methods

Data used

We collected the final all-cause deaths for 2010–2021 from the relevant statistics authorities, divided into years to avoid seasonal effects (Supplementary Table S1, available as Supplementary data at IJE online), as well as mean population data per year. The links to the sources of the data can be found in the ‘Data availability’ statement. The data were confirmed again in late July 2022 and had not changed, as expected because the data are defined as final by the statistics departments.

Back-calculating expected deaths

Excess deaths are defined as observed real deaths minus the expected deaths. To test the validity of the IHME model¹² where expected deaths were not directly reported, we used reported excess deaths and the actual, final deaths for back-calculating the expected deaths implied via Equation (1):

$$\mathit{Expected} \mathit{deaths} \mathit{from} \mathit{model} = \mathit{observed} \mathit{deaths} – \mathit{excess} \mathit{deaths} \mathit{from} \mathit{model}$$

(1)

We then compared these expected deaths, which are modelled and thus always subject to potential uncertainties, with observed final Nordic annualized death data to test the reasonability of the model’s extrapolation to 2020–2021, and conversely the stated excess deaths.

Challenges estimating excess mortality

Challenges when estimating excess mortality include: (i) trends in population structure, notably changes in ageing and demographic shifts, should be accounted for; (ii) data quality, and effects of seasonality and week/year overlap [International Organization for Standardization (ISO) week] if using weekly vs annual or monthly data;¹⁶ (iii) the time period and other uncertainties related to estimating the expected deaths;^16,17 (iv) mortality displacement, with mortality in one time period correlating with the next period,^18–20 can affect the attribution of excess deaths to specific years but possibly also the baselines of expected deaths; and (v) unusual recent events such as a severe influenza season or heatwave could distort baselines, by artificially raising or lowering expected deaths. If analysing individual causes of death, additional assumptions emerge as true causes of death are frequently multifactorial and difficult to establish, and thus beyond the scope of this study.

No method handles all issues perfectly. Linear extrapolation on full-year data solves some of the issues as it averages out seasons and can handle population structure on short timescales but is sensitive to recent unusual events, as analysed below. Yet this approach, as applied e.g. by Karlinsky and Kobak,¹³ does not include any assumptions beyond linearity, and mortality displacement is partly accounted for by averaging low and high death years. Application of distinct models is important as sensitivity tests for understanding and comparison of the data.¹⁶ Methods that use fixed functional (e.g. sinusoidal) forms to estimate the baseline and reduce the impact of unusual influenza seasons or heat waves also exist.^21,22

Sensitivity analysis and comparisons

To understand ranges and uncertainties, we calculated 5-year and 10-year linear trends in all-cause deaths (2015–2019 and 2010–2019). The removal of single unusual years as part of a sensitivity analysis provides an estimate of maximum baseline impact. This was done for the recent years 2018 and 2019. We used linear extrapolation because it is increasingly considered an important alternative to gradient-containing splines that may overemphasize recent year trends,²³ includes population trends partly and enables transparent sensitivity analysis. Our sensitivity analysis applies directly to the World Mortality Dataset (WMD) estimates,¹³ which are based on such linear extrapolation, but are only indicative for other models.

Changes of the population age impact expected deaths (and thus deduced excess deaths),²⁴ with death rates steeply increasing with age,²⁵ and can be accounted for using death rates based on age-group-specific populations from the Nordic Council's aggregate data (see https://pxweb.nordicstatistics.org/). However, all estimates reviewed here only reported total all-cause mortality, so comparison of age-specific mortality was not possible. Accordingly, except for this illustration of age effects, we did not consider any other covariates in our analysis.

The IHME’s excess mortality estimates for 2020 and 2021 by Wang et al.¹² were compiled directly from the main table of their paper. The IHME model estimates expected deaths via a six-model ensemble that tries to correct for missing data due to late registration and leaves out heatwaves of 3 weeks inside the time series. It also applies a global statistical model to countries that do not have data available. For more details on this method, we refer to the Appendix of Wang et al.¹²

We also compiled estimates from the WMD (Karlinsky and Kobak), available at GitHub.¹³ These models run weekly from 30 December 2019 to 2 January 2022, giving 4 days of difference in total deaths relative to the yearly time series in Supplementary Table S1 (available as Supplementary data at IJE online), and use linear trends to estimate expected deaths.

We also reviewed estimates from the method used by the Economist (via Sondre Solstad) in two different versions:²⁶ one that includes the January and February 2020 death data in fitting the expected death trends, and one that does not. Their machine-learning (gradient-boosting) model instead uses actual death data when available at high quality as for the Nordics, based on the direct (not log-linear) deaths, and estimates expected deaths in a similar way to the WMD approach.¹³

We also included estimates of the 2020 and 2021 excess mortality by the World Health Organization (WHO) Technical Advisory Group for COVID-19 Mortality Assessment, released on 5 May 2022 (note: revisions may occur).²⁷ These estimates are based on a statistical Poisson-type model that, as with the WMD and Economist models, emphasizes direct death data for countries where these are available, which includes the Nordics, and predictions for those where they are unavailable.²⁸ The WHO model uses log-linear fitting with time variation modelled via splines where full death data are available. More details on these models are available elsewhere.²⁸

In addition, we included estimates from an ensemble of Bayesian methods,^29,30 referred to below as the ‘Bayesian model ensemble’ (BME). This method uses weekly deaths and populations from Eurostat and an ensemble of Bayesian probabilistic models to estimate the expected number of deaths in the absence of the pandemic.²⁹ The models were designed to account for medium- to long-term secular trends in mortality, the potential dependency of death rates in each week on those in preceding week(s) and in each year on those in preceding year(s), and factors that affect mortality, including seasonality, temperature and public holidays. The models were fitted to up-to-date data from 2010 until the last week of December 2019 and then used to generate predictions of expected deaths in 2020–2021.

We did not include EuroMOMO,²¹ a European mortality monitoring activity to measure excess deaths, in our analysis because its all-cause excess mortalities for 2020 and 2021 were unavailable. Also, since EuroMOMO removes winter periods when fitting expected mortality, its baselines will be lower than corresponding all-cause baselines and the model tends to produce excess mortality every year.

Results

Overview of mortality estimates

Table 1 shows an overview of the analysed data for the five Nordic countries. Although most methods are in some relative agreement, one of the methods produces very different results from other models. This has major implications, as seen from the excess deaths per 100 000 people in Table 2, with IHME estimates for Denmark, Finland and Sweden being much higher and similar to each other than estimated by other methods.

Table 1

Summary of data for the five Nordic countries

	Denmark	Finland	Iceland	Norway	Sweden	Total
Actual deaths 2020 and 2021	111 797	113 147	4640	82 613	190 082	502 279
Excess deaths 2020 + 2021, IHME12	10 400	8780	–314	742	18 100	37 708
95% confidence interval, IHME12	8900 to 11 700	7190 to 10 200	–703 to 11	–4 to 1630	16 900 to 19 500
Expected deaths required to be true	101 397	104 367	4954	81 871	171 982	464 571
Average expected deaths required/year	50 699	52 184	2477	40 936	85 991	232 287
Excess mortality estimates 2020 + 2021
From annual linear trend 2015–2019	1015a	3239	33	1095	10 313b	15 695
From annual linear trend 2010–2019	4007	2756	–98	1907	8475	17 047
Trend 2010–2019 without 2018	5587	3168	–115	2104	9836	20 580
Trend 2010–2019 without 2019	4 354	1994	–144	2116	5863	14 183
World Mortality Dataset (WMD)c,13	962	2661	53	1126	9936	14 738
Economistc,26	2168	4039	–14	1822	11 566	19 581
Economistc,d	1020	2706	43	1151	9859	14 779
World Health Organization (WHO)27	3716 (2601 to 4796)	2857 (2025 to 3690)	–11 (–64 to 46)	–101 (–685 to 541)	11 255 (9870 to 12 667)e	17 716
Bayesian model ensemblec,29	4799 (51 to 9397)	3029 (282 to 5907)	–215 (–588 to 146)	3018 (162 to 5837)	10 050 (3798 to 16 574)	20 681

a

2814 if removing 2018 as sensitivity test (influenza 2017–2018 season).

b

3200 if removing 2019 from the trend.

c

Weekly data.

d

Economist model when excluding January and February 2020 from trend calculation.

e

A revision 18.5% higher (∼13 330) has been reported but comparability to other estimates is unclear.²³ IHME, Institute for Health Metrics and Evaluation.

Table 2

Excess mortality estimates 2020 + 2021 per 100 000 people (using population 1 January 2021)

	Denmark	Finland	Iceland	Norway	Sweden
Population	5 840 045	5 533 793	368 792	5 391 369	10 379 295
IHMEa,12	178	159	–85	14	174
Trend 2015–2019	17	59	9	20	99
Trend 2010–2019	69	50	–27	35	82
Trend 2010–2019 without 2018	96	57	–31	39	95
Trend 2010–2019 without 2019	75	36	–39	39	56
World Mortality Dataset (WMD)13	16	48	14	21	96
Economist26	37	73	–4	34	111
Economistb	17	49	12	21	95
World Health Organization (WHO)27	64	52	–3	–2	108
Bayesian model ensemble29	82	55	–58	56	97

a

Institute for Health Metrics and Evaluation.

b

Economist model excluding January and February 2020 from trend calculation.

To understand these differences in more detail, we estimated what the expected deaths would have been if they followed a trend in the actual annual death data and compared these to the final Nordic annual deaths for 2020 and 2021 to estimate what the excess mortality would correspondingly be via Equation (1) and subject these extrapolations to sensitivity tests of time period and unusual years.

Figure 1 shows the actual all-cause annual deaths of the five Nordic countries for the years 2010–2021, updated as of 27 April 2022. We added a red line for each country indicating the average expected deaths of 2020 and 2021 required for the excess mortality estimated by Wang et al.¹² to be true, back-calculated using Equation (1). As seen from Figure 1, the implied expected deaths (red lines) were inconsistent with the actual data for the years prior to 2020 for Denmark, Finland and Sweden. For all three countries, the expected all-cause deaths are substantially underestimated relative to both 5-year and 10-year trends of the data. For Denmark and Sweden, the implied expected deaths are lower than any observed deaths in the previous 10 years despite a recent increasing trend. A similar result was seen for mortality rates that account for changing population size (Supplementary Figure S1, available as Supplementary data at IJE online) (calculated as in Supplementary Table S2, available as Supplementary data at IJE online). Thus, we conclude that the estimates are unlikely to be realistic.

Figure 1

All-cause deaths of the Nordic countries 2010–2021 (squares). (a) Denmark; (b) Finland; (c) Iceland; (d) Norway; (e) Sweden. The lines show the back-calculated expected deaths (average of 2020 and 2021) implied by the excess deaths in Wang et al.,¹² using Equation (1) (similar analysis for mortality rates in Supplementary Figure S1, available as Supplementary data at IJE online)

Estimates of sensitivities

We used the annual Nordic all-cause death data to compute simple excess death estimates with 5- or 10-year linear trends as sensitivity estimates of the impact of time period and tested the sensitivity to leaving out recent years with large potential impact (Supplementary Figure S1, available as Supplementary data at IJE online). Two special years are notable: (i) some countries had a particularly deadly 2017–2018 influenza season³¹ as is visible in Figure 1 for Denmark (Nordic influenza deaths typically cluster in January to March even if the season starts earlier); (ii) Sweden had unusually low mortality in 2019. The extrapolations without 2018 or 2019 show relatively little impact on Finland’s, Iceland’s and Norway's deaths, but a large effect for Denmark and Sweden, indicating that the excess deaths of the two latter countries are more difficult to estimate. Methods that do not account for these unusual years may suffer uncertainties as implied in Table 1.

In principle, special periods of unusual low or high mortality could be smoothed out, but such removals could also produce errors due to mortality displacement.^20,32 For example, if one discards 2019 completely, Sweden's excess mortality would be substantially lower. Although Sweden experienced less mortality in 2019 and more in 2020, other Nordic countries had lower mortality in 2020, as noted previously,^10,13,29 but relatively more in 2021 (Figure 1). This could suggest mortality displacement³³ or e.g. immunity effects, although this needs further study. Another noteworthy finding is the sensitivity of the excess mortality estimates for Denmark to the time period used for the baseline (5 or 10 years), suggesting trend changes that may affect the baseline.

For Iceland, estimates also differed substantially partly due to the small numbers involved and to fluctuations, but the IHME estimate was still far from any other estimate in Table 1. Figure 1 suggests that the implied baseline is high. In total, the excess reported by Wang et al.¹² for the five Nordic countries was more than a factor of 2 of that deduced from the 5-year or 10-year trends, and this difference was not reduced by leaving out the most impactful special years.

Comparison of models

The total excess mortality estimates for 2020 and 2021 from the WMD¹³ were compiled as in Table 1. This method uses linear extrapolations and thus carries the types of uncertainties analysed above in Table 1. The WMD estimates agree well with the annual data trends as expected due to their similar methodology, with variations far from the estimates by Wang et al. In total, the numbers of the IHME model are 2.5-fold those of the WMD—an enormous difference if both models apply similar data for the Nordic countries (Table 1).

We also reviewed the estimates of two Economist models. Wang et al. provide a double-logarithmic plot of absolute excess deaths (their Supplementary Figure S5) to suggest agreement with the Economist, but such a plot is dominated by large countries, making discrepancies less clear. Table 1 lists the Economist estimates both with and without the first 2 months of 2020 included when estimating baselines, which has a notable impact. Still, these estimates are far from those of the IHME: e.g. the Economist estimate for Denmark is less than a quarter of the 10 400 suggested by the IHME.

The WHO estimates for 2020 and 2021 from May 2022²⁷ also show good agreement with the ranges of other methods, except having a somewhat lower excess mortality for Norway. We find that this method also gives total excess mortality for the five Nordic countries combined of approximately half that of the IHME.¹² Despite the variations in Table 1, the IHME estimates are outside the ranges of all other methods for Sweden, Finland and Denmark. For example, for Denmark, the IHME estimate is 8900–11 700, i.e. even the smallest number is much larger than other ranges in Table 1. In this light, the narrow IHME confidence intervals are concerning. Although Wang et al. did not separate years, their excess deaths also seem high vs other estimates for earlier parts of the pandemic listing a few thousand excess deaths for Denmark and Finland.³⁴

The BME^29,30 also has its central estimates relatively similar to those of the other methods, with the exception that it gives the highest central estimate for Norway, although the confidence interval spans from 162 to 5837, i.e. still within the range of all other methods (Table 1). For the other countries, the BME is in relatively good agreement with the other methods except the IHME, as is the sum of the median estimates, 20 681. We also note that, although not directly comparable, the lower IHME estimates for Finland and Sweden are well above the upper end of the 95% uncertainty interval for both the WHO and the BME. The average total for the two Economist models, WMD, WHO, and BME is 17 499 with a standard deviation of 2717. The 95% CI for all nine non-IHME estimates is 17 222 (15 248–19 197).

Islam et al.¹⁰ performed a detailed age-specific study of the excess deaths of 2020, which cannot be directly compared with the IHME’s combined estimates for 2020 and 2021, but they reported central estimates of –160 deaths for Denmark, –70 for Norway, 1000 for Finland and 9300 for Sweden for 2020. Although mortality patterns may have reversed after 2020 for Sweden and its neighbours according to several models (e.g. WHO and WMD, and the linear estimates) this result by Islam et al. is also distinct from the IHME’s estimate and relatively more in line with the other model estimates examined here.

The Nordic countries' capacity to identify COVID-19 deaths

The results by Wang et al.¹² suggest that Nordic countries had enormous differences in their ability to identify deaths due to COVID-19, with a ratio between estimated excess and official COVID-19 deaths of 3.2 and 5.0 for Denmark and Finland, but only 0.6 for Norway and 1.2 for Sweden.¹² Although we expect differences due to different reporting strategies in the reporting ratio over 2020 and 2021 as testing intensified, knowing the Nordic healthcare systems and responses, the many-fold under-registration appears implausible to us, as does the major heterogeneity in this capacity, inviting further analysis.

We calculated this ratio as shown in Figure 2 (raw ratios are summarized in Supplementary Table S3, available as Supplementary data at IJE online), using the official COVID-19 deaths until 31 December 2021 (these numbers are very similar to those reported by Wang et al., i.e. not a source of uncertainty). We find that the Nordic countries' ability to identify COVID-19 deaths (assuming most excess deaths are COVID-19) is much more homogeneous with the other estimates than with the IHME model.¹² It is the only model that estimates that Sweden had more excess deaths than official COVID-19 deaths, and the apparent ability of Finland and Denmark to identify their COVID-19 deaths is much more similar to other countries for the other studied estimates. The ratios for Iceland are highly fluctuating and uncertain, due to the relatively large spread in absolute estimates relative to the overall small numbers markedly affecting the ratio, and thus are not shown in Figure 2.

Figure 2

Estimated excess deaths divided by official COVID-19 deaths for 2020 and 2021. Economist (a) and (b) refer to the model with and without January and February 2020 included when estimating expected deaths (Iceland is not shown due to large uncertainty in the small crude death numbers). IHME, Institute for Health Metrics and Evaluation; WMD, World Mortality Dataset; WHO, World Health Organization; BME, Bayesian Model Ensemble

Considerations of the impact of population structure

The Nordic countries differ somewhat in age structures (Supplementary Table S4, available as Supplementary data at IJE online). Figure 3 shows the death rates of the 5-year groups based on total deaths and the mean population of each age group (numbers in Supplementary Table S5, available as Supplementary data at IJE online). By far the most excess mortality is observed in the 70+ age groups, consistent with the exponential impact of age on (COVID-19) mortality.²⁵ Different changes of the populations of the age groups from 2010 to 2019 (Supplementary Figure S2, available as Supplementary data at IJE online) may affect excess death estimates. As such an analysis was not done by any of the reviewed models, the numbers summarized in this work should be interpreted with caution given the possible impact of such variations in demographic development.

Figure 3

Nordic age-specific mortality rates per 1000 people within 5-year age groups for 70+ years. Based on final official deaths until 2021 per age group and mean annual population of each age group. (a) Denmark; (b) Finland; (c) Iceland; (d) Norway; (e) Sweden

Discussion

The results above indicate major uncertainties in published excess death estimates and particular caution with regard to the IHME estimates. Although estimates of infection are uncertain and heterogeneous over time, we also note that the very different mortality estimates have implications for the fatality of infections (i.e. the IFR, the number of COVID-19 deaths divided by the best estimate of all infections), given that most excess deaths are due to COVID-19. The IHME estimates until 14 November 2021¹⁵ suggest that the average infections in Denmark and Finland were six to seven times more deadly than in Norway and almost twice as deadly as in Sweden; since the other models had more similar death estimates, they would also give more similar IFR values, reasonably assuming that most excess deaths are due to COVID-19.¹⁵ The IFR estimates by O'Driscoll et al.²⁵ for the first part of 2020 were also substantially more homogeneous, at ∼0.5–0.7% for Denmark, Norway, Finland and Sweden. We find it anomalous that Finland and Denmark would be worse at identifying their COVID-19 deaths by factors of 2–4 vs Sweden or 4–8 vs Norway (Figure 2) and simultaneously have much more lethal SARS-CoV-2 infections.¹⁵ The parsimonious explanation to these anomalies is that the IHME mortality model may not be accurate for these countries, and these IFR estimates invite further analysis.

The total Nordic excess death estimate from the IHME is consistently approximately twice that of the other models analysed here (Supplementary Table S6, available as Supplementary data at IJE online). It is challenging to track the sources of this discrepancy due to the model’s complexity. Upon personal communication with the corresponding author (Prof. Wang) we propose that the discrepancy can be isolated to lower modelled expected deaths for 2020 and 2021, rather than to other parts of their modelling or data use. We suspect that some of the splines used in four of the six IHME sub-models overemphasize recent declines in deaths. The sixth model that simply assumes that expected deaths for 2020 and 2021 equal those of 2019 (a low mortality year) will underestimate expected deaths for 2020 and 2021 for countries with an increasing trend such as Denmark and Finland and for Sweden with its particularly low 2019 mortality, but the IHME’s expected deaths are even lower than 2019, indicating some additional effect, plausibly of splines (Figure 1). However, these potential causes for the disagreement will require IHME confirmation, ideally by separating expected and used total deaths for 2020 and 2021 and rerunning with the final data and sensitivity tests leaving out each of the six models in the ensemble.

Given the scientific and political importance of excess mortalities, we consider systematic and open critical comparison of different methods to estimate these of very high priority. Scientific consensus on excess mortality would enable a detailed analysis of the performance of specific countries under the impact of crises such as a pandemic. We note that the total excess numbers as reviewed here cannot directly inform performance comparisons or policy implications, both due to the sensitivities identified and due to missing context on demographics such as changes in population age structure over time.

Conclusion

We reviewed estimates of the excess mortality during the pandemic 2020 and 2021 for the Nordic countries Denmark, Finland, Iceland, Norway and Sweden, which have been of much interest as both possible successes and failures, as an ideal study case due to their high-quality data and similarities. Our purpose was not to provide new advanced estimates, but to critically review methods and estimate uncertainties, limitations and implications of the numbers, especially due to recent debate on per capita deaths and registration differences, and our study should only be seen in this specific context.

As one of the methods (IHME, Wang et al.¹²) produces very distinct results from all other studied estimates, additional analysis of these results was performed. By back-calculation we show that the IHME’s expected deaths¹² seem inconsistent with actual data and, accordingly, excess mortalities seem substantially overestimated relative to reasonable variations in the data for Finland, Denmark and Sweden. We find that the main uncertainties in determining the excess deaths are the time period used for determining the baseline of expected deaths (e.g. for Denmark, Table 1) and the fluctuations (and potential mortality displacement) caused by unusual mortality events such as the 2018 influenza season in Denmark and the low Swedish 2019 mortality. Although differences between the IHME and other health metrics have been noted before,^35,36 our study represents a systematic assessment of both numbers and their implications, which we hope will set precedence.

Our review of methods and sensitivity tests suggest that the overall excess mortality for 2020–2021 in the Nordic countries most likely ranged between 15 000 and 20 000. The recent WHO estimates (May 2022) are in the middle of this range (17 716).²⁷ The BME²⁹ gives a result of 20 681 and a variety of linear regressions produce relatively similar results. These numbers are approximately half that suggested by the IHME model and imply much more similar capability of identifying COVID-19 deaths and fatalities of infections probably more consistent with the similarities between the countries. Policymakers and others interpreting excess death data are encouraged to consider multiple models and appreciate relevant sensitivities and uncertainties. As a hallmark example of this, the IHME predicts similar excess deaths per capita for Sweden, Denmark and Finland, quite distinct from other models (Table 2), i.e. relative country estimates are particularly sensitive to the uncertainties described here and could lead to very different conclusions even before adding population demographics.

The heterogeneity in estimates even for countries with essentially complete data as reviewed here raises concerns regarding estimates for countries without good data. It points to the importance of careful and public evaluation and comparison of methods to calculate excess mortality and a resolute need for data and method transparency, in particular metrics from training and external validation, and critical discussion of results. We hope that our study will set such a precedence for the future.

Furthermore, our study illustrates the general need for data-focused quality control of complex models whose uncertainties and assumptions may be difficult to interpret. Clear messaging is important for policy and the wider public, but high-quality data should not be subordinate to complex models. We warmly invite further studies that account in more detail for these topics and uncertainties.

Ethics approval

The study used anonymous, public total annual death and population data from the statistics departments of the Nordic countries, and thus did not require any ethics approval.

Data availability

All data required for the calculations in this work are available at the web pages of Statistics Denmark, Statistics Norway, Statistics Sweden, Statistics Finland and Statistics Iceland, and the method estimates are available at public sites: Statistics Finland: https://pxweb2.stat.fi/PxWeb/pxweb/en/StatFin/StatFin__kuol/statfin_kuol_pxt_12ak.px/; Statistics Iceland: https://px.hagstofa.is/pxen/pxweb/en/Ibuar/Ibuar__Faeddirdanir__danir__danir/MAN05210.px/table/tableViewLayout1/?rxid=247e4620-6490-4f04-b60a-58a68a3afbd9; Statistics Denmark: https://www.statistikbanken.dk/20014; Statistics Norway: https://www.ssb.no/en/statbank/table/08425/; Statistics Sweden: https://www.statistikdatabasen.scb.se/pxweb/en/ssd/START__BE__BE0101__BE0101G/ManadFoddDod/; comparative Nordic data: mean population sizes, mortality rates: https://pxweb.nordicstatistics.org/pxweb/en/Nordic%20Statistics/Nordic%20Statistics__Demography__Population%20change/; WMD: https://github.com/dkobak/excess-mortality/blob/main/excess-mortality-timeseries.csv; Economist estimates: https://www.economist.com/graphic-detail/coronavirus-excess-deaths-estimates; WHO estimates: https://www.who.int/data/sets/global-excess-deaths-associated-with-covid-19-modelled-estimates; code and data used for the BME: https://github.com/vkontis/excess_mortality/tree/pub2; weekly deaths and population data (Eurostat): https://ec.europa.eu/eurostat/data/database (tables demo_r_mwk_05 and demo_pjangroup); temperature (ERA5) and gridded population: https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era5https://sedac.ciesin.columbia.edu/data/collection/gpw-v4.

Supplementary data

Supplementary data are available at IJE online.

Author contributions

K.P.K. designed the study, contributed to the calculations and analysis, and wrote the first draft. J.B. contributed to the analysis, interpretation of data and writing of the paper. V.K. and R.M.P. contributed the estimates of the BME and to the analysis and writing of the paper. K.T.B. contributed to the data and analysis of death rates. L.E. contributed to the analysis, interpretation and writing of the paper. T.L. contributed to the data and analysis, interpretation and co-wrote the paper.

Funding

We did not have funding specific for this study. J.B. was supported by grants for research on COVID-19 and pandemic preparedness from the Swedish Research Council (VR; grant number 2021–04665) and Sweden’s Innovation Agency (Vinnova; grant number 2021–02648).

Acknowledgements

The authors thank Tauno Tyllinen from Statistics Finland, Tomas Johansson from Statistics Sweden, Bergný Tryggvadóttir from Statistics Iceland and Dorthe Larsen from Statistics Denmark for communication regarding country data; Sondre Solstad for discussions on the Economist model and the estimates in Table 1; Charles Tallack (The Health Foundation), Jens Nielsen (Statens Seruminstitut), James Wood and David Muscatello (School of Population Health, UNSW) and Theis Lange and Terese Jørgensen (Copenhagen University) for stimulating feedback; and Dmitry Kobak (Tübingen University) and Ariel Karlinsky (Hebrew University) for helpful discussion and confirmation of the WMD data.

Conflict of interest

Dr Vasilis Kontis and Dr Robbie M. Parks were involved in developing one of the reviewed methods (the BME). No other conflicts of interest were declared.