Abstract
Death within a specified time window following a positive SARS-CoV-2 test is used by some agencies for attributing death to COVID-19. With Omicron variants, widespread immunity, and asymptomatic screening, there is cause to re-evaluate COVID-19 death attribution methods and develop tools to improve case ascertainment.
All patients who died following microbiologically confirmed SARS-CoV-2 in the Veterans Health Administration (VA) and at Tufts Medical Center (TMC) were identified. Records of selected vaccinated VA patients with positive tests in 2022, and of all TMC patients with positive tests in 2021–2022, were manually reviewed to classify deaths as COVID-19–related (either directly caused by or contributed to), focused on deaths within 30 days. Logistic regression was used to develop and validate a surveillance model for identifying deaths in which COVID-19 was causal or contributory.
Among vaccinated VA patients who died ≤30 days after a positive test in January–February 2022, death was COVID-19–related in 103/150 cases (69%) (55% causal, 14% contributory). In June–August 2022, death was COVID-19–related in 70/150 cases (47%) (22% causal, 25% contributory). Similar results were seen among the 71 patients who died at TMC. A model including hypoxemia, remdesivir, and anti-inflammatory drugs had positive and negative predictive values of 0.82–0.95 and 0.64–0.83, respectively.
By mid-2022, “death within 30 days” did not provide an accurate estimate of COVID-19–related death in 2 US healthcare systems with routine admission screening. Hypoxemia and use of antiviral and anti-inflammatory drugs—variables feasible for reporting to public health agencies—would improve classification of death as COVID-19–related.
Accurate metrics are essential for guiding and adapting the public health response to the ongoing coronavirus disease 2019 (COVID-19) pandemic based on changing conditions and contexts. The primary measures of pandemic burden have been case rates, hospitalization rates, and death rates. Methods for ascertaining these metrics also require ongoing reassessment, given major therapeutic and preventative milestones that have fundamentally altered COVID-19 disease severity [1]. We previously demonstrated that, among vaccinated US veteran patients hospitalized contemporaneously with a positive severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) test, receipt of dexamethasone was an indicator of hospitalization for COVID-19 with significant respiratory involvement, and that this simple metric was feasible for statewide implementation [2–4]. Given changing conditions, an assessment of methods for attribution of deaths where COVID-19 is either the primary cause or a contributor is also needed.
Manual case review of all deaths by trained reviewers within healthcare systems or public health professionals is not feasible, and all other methods for attributing death to COVID-19 are inferior. Measurement of COVID-19–related deaths from death certificate data alone has high face validity, and prior investigations suggest high positive predictive value (PPV) early in the pandemic [5]. For these reasons, the US National Center for Health Statistics (NCHS) defines a COVID-19 death as one listed as the primary cause or a contributing cause of death on the death certificate [5, 6]. The NCHS system is considered the gold standard for measurement, but the information is typically delayed 1–8 weeks due to the time required for death certificate review and coding prior to public release of data. In addition, there are concerns about possible misclassification due to a variety of factors, including physicians with insufficient experience or information (underreporting) and policies for testing asymptomatic patients [7] or reimbursing funeral expenses (overreporting) [8]. The PPV of death certificates has not been re-assessed as the pandemic has evolved, and to our knowledge, negative predictive value (NPV) has not been estimated.
Due to inherent delays in processing and reporting of death certificate data coupled with a need for near-real-time data to guide policy adaptations, other US federal and state agencies and healthcare systems developed alternative COVID-19 mortality surveillance methods. The methods range from complex modeling to the simple metric of considering death within a specific time period after a positive test for SARS-CoV-2 to be caused by COVID-19, which had good predictive utility during early phases of the pandemic [9] but has not been reassessed after advancements in medical countermeasures (Supplementary Table 1). The US Centers for Disease Control and Prevention (CDC) populates a daily tracker, which reports COVID-19 deaths collected from state and territorial health departments [10]. Guided by the Council of State and Territorial Epidemiologists (CSTE), states and territories are permitted to use their own definitions of COVID-19–related death, which, for expediency, are often designed to be more feasible for data extraction and application by non-clinicians [10, 11]. In late 2021, the CSTE changed the surveillance definition of a COVID-19 death from one that occurs 60 or 90 days after a positive test to one that is within 30 days (and due to natural causes) [12]. In November 2022, the CSTE revised its recommendations to focus almost entirely on death certificates, but implementation has been variable [13].
With substantial immunity in the population and medical countermeasures, pandemic severity has changed substantially. Agencies have revised definitions of COVID-19 mortality, but data to support these changes remain limited. Thus, the aims of this retrospective cohort study conducted in 2 different settings were as follows: (1) to use clinical review to assess cause of death in patients who had recently tested positive for SARS-CoV-2 and (2) to develop and validate a simple multivariable model with high PPV for classifying a death as COVID-19–related. Secondarily, we hypothesized that the percentages of deaths that were COVID-19–related would be lower in mid-2022 than in early 2022, and we could test this hypothesis in one of these healthcare systems. An assessment of the PPV and NPV of death certificates was feasible in the other healthcare system for patients who died as inpatients.
METHODS
Cohorts
Veterans Health Administration
A cohort of patients was created with a documented SARS-CoV-2 infection between 1 January 2020 and 30 September 2022 who died at any time after the positive test. Analysis was limited to the first positive test after vaccination among vaccinated patients. The time period for assessing completion of an initial series of SARS-CoV-2 vaccination (2 doses of an mRNA vaccine or 1 dose of Ad26.COV2.S) was 15 December 2020 through 30 November 2021. Detailed chart review was confined to vaccinated patients with first positive tests between 1 January 2022 and 28 February 2022 (Omicron-BA.1 period) or between 1 June 2022 and 31 August 2022 (Omicron-BA.5 period), subdivided further by death at 1–30 days or 31–90 days after infection. Data from unvaccinated patients were used only to plot times between a positive test and death for a qualitative comparison to vaccinated patients.
Data were obtained from the Veterans Health Administration (VA) COVID-19 shared data resource [14] and Corporate Data Warehouse (CDW; last accessed 6 October 2022). Supplementary Table 2 gives a complete list of definitions. Review of electronic health records (EHRs) from individual patients was performed using the VA's Joint Legacy Viewer interface.
Tufts Medical Center
Throughout the pandemic, the Department of Emergency Management at Tufts Medical Center (TMC), a 415-bed urban tertiary care academic hospital, maintained a record of “COVID deaths”: patients admitted to the hospital with a diagnosis of COVID-19 while still requiring isolation precautions who died during their inpatient admission at TMC. These patients could have had a positive polymerase chain reaction (PCR) test at TMC or a PCR or antigen test done at a transferring facility. All such patients, either vaccinated or unvaccinated, who died between 28 January 2021 and 13 October 2022 were included in the cohort, and cases were manually reviewed for causality.
Outcomes
The primary outcome was attribution of cause of death to COVID-19. Each case was classified as having SARS-CoV-2 infection as the clear cause or one of multiple essential causes of the patient's death even if not via a respiratory illness (“caused”) or not caused by or significantly influenced by infection (“noncontributory”), or contributing to demise either by accelerating decline from a chronic and irreversible underlying process (such as dementia or metastatic cancer or a combination of causes with failure to thrive), or by initiating a decline after which death occurred due to a defined cause other than COVID-19 (“contributory”). Receipt of medications to treat COVID-19 was not used to make determinations about cause of death. “Caused” and “contributory” were combined as “COVID-19–related deaths” for statistical analyses and model building.
Additional data obtained electronically for VA cases are described in Supplementary File 1. COVID-19–related variables included evidence of hypoxemia (minimum SpO2 <90% or maximum supplementary oxygen >2 L/min at any time from days −1 to +14 relative to the first positive test), mechanical ventilation, and use of remdesivir and/or systemic anti-inflammatory drugs (dexamethasone or methylprednisolone, baricitinib, or tocilizumab) between the positive test and death. For the TMC cohort, data were obtained via manual chart review. Variables included age, sex, prespecified comorbidities, number of SARS-CoV-2 vaccine doses received, and use of immunosuppressive drugs (including chemotherapy) within 6 months prior to infection. Information about the decedents’ clinical course and management was also collected, including use of COVID-19–directed therapies (eg, remdesivir, dexamethasone or methylprednisolone, baricitinib, tocilizumab), evidence of hypoxemia (defined as a minimum SpO2 <90% or requiring supplementary oxygen >2 L/min during hospitalization), and receipt of mechanical ventilation, at any time between the positive test and death.
When available in the TMC cohort, death certificates were reviewed to determine if COVID-19 was listed as either a primary or a secondary cause of death. Death certificates are infrequently scanned into the VA EHR or scanned in with significant delay and were therefore not available for review as part of this study.
Chart Review Process
Veterans Health Administration
For each subset defined by calendar period and time interval from infection to death, 100–150 patients were selected randomly for detailed review by a clinician (P. A. M. or L. L. L.). Cases regarded as having uncertain attribution by the 1 reviewer were discussed with the other reviewer. Reliability of this approach was tested by simultaneous, independent review of 20 cases by both reviewers. Fourteen cases were not deemed by either reviewer to require discussion, and there was agreement on 13 of 14 cases; the 6 other cases were flagged by 1 reviewer as uncertain, with consensus reached by discussion. Since it was essential to know dates of death and infection, reviewers were not blinded.
Tufts Medical Center
Each case was manually adjudicated separately by 2 physicians (C. T. and M. A.). Any discrepancies in adjudication were discussed between the 2 reviewers. In 2 cases in which a consensus could not be reached, a third adjudicator reviewed the case to determine attribution (S. D.).
Statistical Analysis
The VA groups were defined by time since positive test (1–30 days or 31–90 days) and calendar period of positive test (January–February 2022 or June–August 2022), and the percentages of patients in each of the 3 categories were plotted as moving averages in 10-day increments of time since the positive test. Parametric 95% confidence intervals were calculated. Proportions of patients infected in January–February 2022 or June–August 2022 were compared by Fisher's exact test, with “contributory” and “caused” combined to capture all “COVID-19–related” deaths.
The association of demographic and clinical variables with classification as COVID-19–related was estimated in the VA dataset using logistic regression, with and without calendar period and time since positive test (in days). The first set of regressions determined adjusted odds ratios (aORs) for a broad set of demographic and treatment variables, timing of death, and comorbidities [15]. This set of variables was then reduced to identify sets of variables that are feasible for electronic measurement. Model performance was assessed using the areas under the receiver operating characteristic curves (AUCs), without attempting to formally compare AUCs for significant differences.
Multivariable models were then validated using the TMC cohort of patients who died within 30 days of a positive test. In a second step, 3 or 4 variables were used additively to create simple indices (0–3 or 0–4 for each patient). Logistic regression was used with the index as the single, ordinal independent variable. The PPVs and NPVs for COVID-19–related death were calculated using different cutoff values for the indices. All analyses were performed in R version 4.0.3 (R Foundation for Statistical Computing) and SAS version 9.4 (SAS Institute).
Ethical Considerations
This study was approved as an exempt study by the TMC Institutional Review Board (IRB) and the VA Boston Healthcare System IRB and Research and Development Committee prior to data collection and analysis. No identifiable data were shared between the 2 institutions.
RESULTS
Among 66 575 vaccinated patients in the VA cohort with first positive breakthrough tests between 1 January 2022 and 28 February 2022, 745 (1.12%) died within 30 days and 1396 (2.10%) within 90 days. Among 46 623 vaccinated patients with first positive breakthrough tests between 1 June 2022 and 31 August 2022, 398 (0.85%) died within 30 days and 692 (1.48%) within 90 days, although patients diagnosed in July or August did not have a full 90-day follow-up period. The great majority of apparent excess deaths among vaccinated patients in the VA cohort occurred within 40 days throughout the pandemic, and the magnitude above baseline declined steadily but was still detectable through day 90, with a qualitatively similar pattern to that seen historically in unvaccinated patients (Supplementary Figure 1). From 28 January 2021 to 13 October 2022, 86 “COVID deaths” occurred at TMC; 71 (82.6%) of the deaths occurred within 30 days.
Demographic and clinical data for the 371 chart-reviewed cases of death within 30 days in both cohorts are shown in Table 1, stratified by calendar period for the VA cohort. Additional data are shown in Supplementary Tables 3 and 4, which also include data for VA patients who died 31–90 days after the positive test and demonstrate the representativeness of the manually reviewed VA sample. The characteristics for patients who died within 30 days (TMC) or 90 days (VA) of a SARS-CoV-2 test, stratified by adjudicated cause of death, are shown in Supplementary Tables 4 and 5.
Demographic and Clinical Characteristics of Patients Whose Cases Underwent Chart Review to Determine Cause of Death, Stratified by Medical System and (for VA) Calendar Period of Positive SARS-CoV-2 Test Preceding Death
| VA | TMC | ||
|---|---|---|---|
| January–February 2022 | June–August 2022 | January 2021–October 2022 | |
| No. | 150 | 150 | 71 |
| Sex, n (%) | |||
| Female | 2 (1.3) | 6 (4.0) | 29 (40.8) |
| Male | 148 (98.7) | 144 (96.0) | 42 (59.2) |
| Age ranges,a n (%) | |||
| <40 y | 0 (0.0) | 0 (0.0) | 7 (9.9) |
| 40–45 y | 0 (0.0) | 0 (0.0) | 0 (0.0) |
| 45–50 y | 0 (0.0) | 0 (0.0) | 4 (5.6) |
| 50–55 y | 1 (0.7) | 1 (0.7) | 4 (5.6) |
| 55–60 y | 1 (0.7) | 0 (0.0) | 9 (12.7) |
| 60–65 y | 4 (2.7) | 6 (4.0) | 22 (31.0) |
| 65–70 y | 25 (16.7) | 17 (11.3) | 7 (9.9) |
| 70–75 y | 30 (20.0) | 23 (15.3) | 3 (4.2) |
| 75–80 y | 31 (20.7) | 29 (19.3) | 3 (4.2) |
| ≥80 y | 58 (38.7) | 74 (49.3) | 15 (21.1) |
| Age, median [IQR], y | 76.16 [71.38, 85.97] | 79.94 [72.82, 89.34] | 66.59 [56.88, 74.52] |
| Vaccination status, n (%) | |||
| Initial vaccination series received | 150 (100) | 150 (100) | 21 (29.6) |
| Boosted | 54 (36.0) | 104 (69.3) | 11 (15.5) |
| BMI, median [IQR], kg/m2 | 29.04 [25.09, 33.54] | 26.93 [22.59, 31.57] | 28.45 [24.68, 34.03] |
| Immunosuppressive medications before breakthrough, n (%) | |||
| Chemotherapy | 8 (5.3) | 6 (4.0) | 6 (8.5) |
| Cytokine-blocking | 3 (2.0) | 2 (1.3) | 0 (0.0) |
| Glucocorticoids | 31 (20.7) | 22 (14.7) | 1 (1.4) |
| Leukocyte-inhibitory | 10 (6.7) | 5 (3.3) | 0 (0.0) |
| Lymphocyte-depleting | 4 (2.7) | 3 (2.0) | 1 (1.4) |
| Comorbidities from algorithms, n (%) | |||
| Alzheimer's/dementia | 19 (12.7) | 26 (17.3) | 4 (5.6) |
| Chronic kidney disease | 52 (34.7) | 43 (28.7) | 14 (19.7) |
| COPD and bronchiectasis | 14 (9.3) | 22 (14.7) | 10 (14.1) |
| Heart failure | 33 (22.0) | 29 (19.3) | 16 (22.5) |
| Liver disease/cirrhosis | 12 (8.0) | 9 (6.0) | 4 (5.6) |
| Schizophrenia and other psychoses | 7 (4.7) | 3 (2.0) | 4 (5.6) |
| Advanced cancer | 29 (19.3) | 31 (20.7) | 10 (14.1) |
| Solid-organ transplant | 6 (4.0) | 1 (0.7) | 4 (5.6) |
| Severity variables, n (%) | |||
| Mechanical ventilation | 23 (15.3) | 20 (13.3) | 53 (74.6) |
| Hypoxemia | 63 (42.0) | 47 (31.3) | 61 (85.9) |
| Medications for COVID-19, n (%) | |||
| Baricitinib | 15 (10.0) | 8 (5.3) | 1 (1.7) |
| Dexamethasoneb | 91 (60.7) | 69 (46.0) | 59 (83.1) |
| Remdesivir | 71 (47.3) | 68 (45.3) | 51 (71.8) |
| Tocilizumab | 11 (7.3) | 2 (1.3) | 15 (21.1) |
| COVID-19–related death from chart review, n (%) | |||
| Yes | 82 (54.7) | 33 (22.0) | 53 (74.6) |
| Contributory | 21 (14.0) | 37 (24.7) | 15 (21.1) |
| No | 47 (31.3) | 80 (53.3) | 13 (18.3) |
| VA | TMC | ||
|---|---|---|---|
| January–February 2022 | June–August 2022 | January 2021–October 2022 | |
| No. | 150 | 150 | 71 |
| Sex, n (%) | |||
| Female | 2 (1.3) | 6 (4.0) | 29 (40.8) |
| Male | 148 (98.7) | 144 (96.0) | 42 (59.2) |
| Age ranges,a n (%) | |||
| <40 y | 0 (0.0) | 0 (0.0) | 7 (9.9) |
| 40–45 y | 0 (0.0) | 0 (0.0) | 0 (0.0) |
| 45–50 y | 0 (0.0) | 0 (0.0) | 4 (5.6) |
| 50–55 y | 1 (0.7) | 1 (0.7) | 4 (5.6) |
| 55–60 y | 1 (0.7) | 0 (0.0) | 9 (12.7) |
| 60–65 y | 4 (2.7) | 6 (4.0) | 22 (31.0) |
| 65–70 y | 25 (16.7) | 17 (11.3) | 7 (9.9) |
| 70–75 y | 30 (20.0) | 23 (15.3) | 3 (4.2) |
| 75–80 y | 31 (20.7) | 29 (19.3) | 3 (4.2) |
| ≥80 y | 58 (38.7) | 74 (49.3) | 15 (21.1) |
| Age, median [IQR], y | 76.16 [71.38, 85.97] | 79.94 [72.82, 89.34] | 66.59 [56.88, 74.52] |
| Vaccination status, n (%) | |||
| Initial vaccination series received | 150 (100) | 150 (100) | 21 (29.6) |
| Boosted | 54 (36.0) | 104 (69.3) | 11 (15.5) |
| BMI, median [IQR], kg/m2 | 29.04 [25.09, 33.54] | 26.93 [22.59, 31.57] | 28.45 [24.68, 34.03] |
| Immunosuppressive medications before breakthrough, n (%) | |||
| Chemotherapy | 8 (5.3) | 6 (4.0) | 6 (8.5) |
| Cytokine-blocking | 3 (2.0) | 2 (1.3) | 0 (0.0) |
| Glucocorticoids | 31 (20.7) | 22 (14.7) | 1 (1.4) |
| Leukocyte-inhibitory | 10 (6.7) | 5 (3.3) | 0 (0.0) |
| Lymphocyte-depleting | 4 (2.7) | 3 (2.0) | 1 (1.4) |
| Comorbidities from algorithms, n (%) | |||
| Alzheimer's/dementia | 19 (12.7) | 26 (17.3) | 4 (5.6) |
| Chronic kidney disease | 52 (34.7) | 43 (28.7) | 14 (19.7) |
| COPD and bronchiectasis | 14 (9.3) | 22 (14.7) | 10 (14.1) |
| Heart failure | 33 (22.0) | 29 (19.3) | 16 (22.5) |
| Liver disease/cirrhosis | 12 (8.0) | 9 (6.0) | 4 (5.6) |
| Schizophrenia and other psychoses | 7 (4.7) | 3 (2.0) | 4 (5.6) |
| Advanced cancer | 29 (19.3) | 31 (20.7) | 10 (14.1) |
| Solid-organ transplant | 6 (4.0) | 1 (0.7) | 4 (5.6) |
| Severity variables, n (%) | |||
| Mechanical ventilation | 23 (15.3) | 20 (13.3) | 53 (74.6) |
| Hypoxemia | 63 (42.0) | 47 (31.3) | 61 (85.9) |
| Medications for COVID-19, n (%) | |||
| Baricitinib | 15 (10.0) | 8 (5.3) | 1 (1.7) |
| Dexamethasoneb | 91 (60.7) | 69 (46.0) | 59 (83.1) |
| Remdesivir | 71 (47.3) | 68 (45.3) | 51 (71.8) |
| Tocilizumab | 11 (7.3) | 2 (1.3) | 15 (21.1) |
| COVID-19–related death from chart review, n (%) | |||
| Yes | 82 (54.7) | 33 (22.0) | 53 (74.6) |
| Contributory | 21 (14.0) | 37 (24.7) | 15 (21.1) |
| No | 47 (31.3) | 80 (53.3) | 13 (18.3) |
Numbers of cases determined to be caused by COVID-19, or with infection as an important contributor to demise, or unrelated, are shown at the bottom of the table. Additional variables, stratification by adjudicated cause of death, and (for VA) comparison to the cohorts from which these patients were chosen for review are provided in Supplementary Tables 3–5.
Abbreviations: BMI, body mass index; COPD, chronic obstructive pulmonary disease; COVID-19, coronavirus disease 2019; IQR, interquartile range; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; TMC, Tufts Medical Center; VA, Veterans Health Administration.
Age ranges are inclusive of the first number and exclusive of the second, eg, “40–45” indicates “≥40 and <45”.
See Supplementary File 1 regarding the decision to exclude use of methylprednisolone from analysis.
Demographic and Clinical Characteristics of Patients Whose Cases Underwent Chart Review to Determine Cause of Death, Stratified by Medical System and (for VA) Calendar Period of Positive SARS-CoV-2 Test Preceding Death
| VA | TMC | ||
|---|---|---|---|
| January–February 2022 | June–August 2022 | January 2021–October 2022 | |
| No. | 150 | 150 | 71 |
| Sex, n (%) | |||
| Female | 2 (1.3) | 6 (4.0) | 29 (40.8) |
| Male | 148 (98.7) | 144 (96.0) | 42 (59.2) |
| Age ranges,a n (%) | |||
| <40 y | 0 (0.0) | 0 (0.0) | 7 (9.9) |
| 40–45 y | 0 (0.0) | 0 (0.0) | 0 (0.0) |
| 45–50 y | 0 (0.0) | 0 (0.0) | 4 (5.6) |
| 50–55 y | 1 (0.7) | 1 (0.7) | 4 (5.6) |
| 55–60 y | 1 (0.7) | 0 (0.0) | 9 (12.7) |
| 60–65 y | 4 (2.7) | 6 (4.0) | 22 (31.0) |
| 65–70 y | 25 (16.7) | 17 (11.3) | 7 (9.9) |
| 70–75 y | 30 (20.0) | 23 (15.3) | 3 (4.2) |
| 75–80 y | 31 (20.7) | 29 (19.3) | 3 (4.2) |
| ≥80 y | 58 (38.7) | 74 (49.3) | 15 (21.1) |
| Age, median [IQR], y | 76.16 [71.38, 85.97] | 79.94 [72.82, 89.34] | 66.59 [56.88, 74.52] |
| Vaccination status, n (%) | |||
| Initial vaccination series received | 150 (100) | 150 (100) | 21 (29.6) |
| Boosted | 54 (36.0) | 104 (69.3) | 11 (15.5) |
| BMI, median [IQR], kg/m2 | 29.04 [25.09, 33.54] | 26.93 [22.59, 31.57] | 28.45 [24.68, 34.03] |
| Immunosuppressive medications before breakthrough, n (%) | |||
| Chemotherapy | 8 (5.3) | 6 (4.0) | 6 (8.5) |
| Cytokine-blocking | 3 (2.0) | 2 (1.3) | 0 (0.0) |
| Glucocorticoids | 31 (20.7) | 22 (14.7) | 1 (1.4) |
| Leukocyte-inhibitory | 10 (6.7) | 5 (3.3) | 0 (0.0) |
| Lymphocyte-depleting | 4 (2.7) | 3 (2.0) | 1 (1.4) |
| Comorbidities from algorithms, n (%) | |||
| Alzheimer's/dementia | 19 (12.7) | 26 (17.3) | 4 (5.6) |
| Chronic kidney disease | 52 (34.7) | 43 (28.7) | 14 (19.7) |
| COPD and bronchiectasis | 14 (9.3) | 22 (14.7) | 10 (14.1) |
| Heart failure | 33 (22.0) | 29 (19.3) | 16 (22.5) |
| Liver disease/cirrhosis | 12 (8.0) | 9 (6.0) | 4 (5.6) |
| Schizophrenia and other psychoses | 7 (4.7) | 3 (2.0) | 4 (5.6) |
| Advanced cancer | 29 (19.3) | 31 (20.7) | 10 (14.1) |
| Solid-organ transplant | 6 (4.0) | 1 (0.7) | 4 (5.6) |
| Severity variables, n (%) | |||
| Mechanical ventilation | 23 (15.3) | 20 (13.3) | 53 (74.6) |
| Hypoxemia | 63 (42.0) | 47 (31.3) | 61 (85.9) |
| Medications for COVID-19, n (%) | |||
| Baricitinib | 15 (10.0) | 8 (5.3) | 1 (1.7) |
| Dexamethasoneb | 91 (60.7) | 69 (46.0) | 59 (83.1) |
| Remdesivir | 71 (47.3) | 68 (45.3) | 51 (71.8) |
| Tocilizumab | 11 (7.3) | 2 (1.3) | 15 (21.1) |
| COVID-19–related death from chart review, n (%) | |||
| Yes | 82 (54.7) | 33 (22.0) | 53 (74.6) |
| Contributory | 21 (14.0) | 37 (24.7) | 15 (21.1) |
| No | 47 (31.3) | 80 (53.3) | 13 (18.3) |
| VA | TMC | ||
|---|---|---|---|
| January–February 2022 | June–August 2022 | January 2021–October 2022 | |
| No. | 150 | 150 | 71 |
| Sex, n (%) | |||
| Female | 2 (1.3) | 6 (4.0) | 29 (40.8) |
| Male | 148 (98.7) | 144 (96.0) | 42 (59.2) |
| Age ranges,a n (%) | |||
| <40 y | 0 (0.0) | 0 (0.0) | 7 (9.9) |
| 40–45 y | 0 (0.0) | 0 (0.0) | 0 (0.0) |
| 45–50 y | 0 (0.0) | 0 (0.0) | 4 (5.6) |
| 50–55 y | 1 (0.7) | 1 (0.7) | 4 (5.6) |
| 55–60 y | 1 (0.7) | 0 (0.0) | 9 (12.7) |
| 60–65 y | 4 (2.7) | 6 (4.0) | 22 (31.0) |
| 65–70 y | 25 (16.7) | 17 (11.3) | 7 (9.9) |
| 70–75 y | 30 (20.0) | 23 (15.3) | 3 (4.2) |
| 75–80 y | 31 (20.7) | 29 (19.3) | 3 (4.2) |
| ≥80 y | 58 (38.7) | 74 (49.3) | 15 (21.1) |
| Age, median [IQR], y | 76.16 [71.38, 85.97] | 79.94 [72.82, 89.34] | 66.59 [56.88, 74.52] |
| Vaccination status, n (%) | |||
| Initial vaccination series received | 150 (100) | 150 (100) | 21 (29.6) |
| Boosted | 54 (36.0) | 104 (69.3) | 11 (15.5) |
| BMI, median [IQR], kg/m2 | 29.04 [25.09, 33.54] | 26.93 [22.59, 31.57] | 28.45 [24.68, 34.03] |
| Immunosuppressive medications before breakthrough, n (%) | |||
| Chemotherapy | 8 (5.3) | 6 (4.0) | 6 (8.5) |
| Cytokine-blocking | 3 (2.0) | 2 (1.3) | 0 (0.0) |
| Glucocorticoids | 31 (20.7) | 22 (14.7) | 1 (1.4) |
| Leukocyte-inhibitory | 10 (6.7) | 5 (3.3) | 0 (0.0) |
| Lymphocyte-depleting | 4 (2.7) | 3 (2.0) | 1 (1.4) |
| Comorbidities from algorithms, n (%) | |||
| Alzheimer's/dementia | 19 (12.7) | 26 (17.3) | 4 (5.6) |
| Chronic kidney disease | 52 (34.7) | 43 (28.7) | 14 (19.7) |
| COPD and bronchiectasis | 14 (9.3) | 22 (14.7) | 10 (14.1) |
| Heart failure | 33 (22.0) | 29 (19.3) | 16 (22.5) |
| Liver disease/cirrhosis | 12 (8.0) | 9 (6.0) | 4 (5.6) |
| Schizophrenia and other psychoses | 7 (4.7) | 3 (2.0) | 4 (5.6) |
| Advanced cancer | 29 (19.3) | 31 (20.7) | 10 (14.1) |
| Solid-organ transplant | 6 (4.0) | 1 (0.7) | 4 (5.6) |
| Severity variables, n (%) | |||
| Mechanical ventilation | 23 (15.3) | 20 (13.3) | 53 (74.6) |
| Hypoxemia | 63 (42.0) | 47 (31.3) | 61 (85.9) |
| Medications for COVID-19, n (%) | |||
| Baricitinib | 15 (10.0) | 8 (5.3) | 1 (1.7) |
| Dexamethasoneb | 91 (60.7) | 69 (46.0) | 59 (83.1) |
| Remdesivir | 71 (47.3) | 68 (45.3) | 51 (71.8) |
| Tocilizumab | 11 (7.3) | 2 (1.3) | 15 (21.1) |
| COVID-19–related death from chart review, n (%) | |||
| Yes | 82 (54.7) | 33 (22.0) | 53 (74.6) |
| Contributory | 21 (14.0) | 37 (24.7) | 15 (21.1) |
| No | 47 (31.3) | 80 (53.3) | 13 (18.3) |
Numbers of cases determined to be caused by COVID-19, or with infection as an important contributor to demise, or unrelated, are shown at the bottom of the table. Additional variables, stratification by adjudicated cause of death, and (for VA) comparison to the cohorts from which these patients were chosen for review are provided in Supplementary Tables 3–5.
Abbreviations: BMI, body mass index; COPD, chronic obstructive pulmonary disease; COVID-19, coronavirus disease 2019; IQR, interquartile range; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; TMC, Tufts Medical Center; VA, Veterans Health Administration.
Age ranges are inclusive of the first number and exclusive of the second, eg, “40–45” indicates “≥40 and <45”.
See Supplementary File 1 regarding the decision to exclude use of methylprednisolone from analysis.
Causes of Death: Descriptive Analysis
In the VA cohort, in January–February 2022, death was COVID-19–related in 103 of 150 (68.7%) vaccinated patients who died within 30 days (caused by COVID-19 in 54.7% and contributory in 14.0%). In June–August 2022, death was COVID-19–related in 70 of 150 (46.7%) vaccinated patients who died within 30 days (caused by COVID-19 in 22.0% and contributory in 24.7%). Proportions of VA cases classified as caused, contributory, or noncontributory are presented longitudinally in Figure 1. COVID-19–related cases (ie, caused by and contributory combined) represented a lower proportion of cases during June–August 2022 than during January–February 2022 (P = .0002). Proportions classified as COVID-19–related at 31–90 days after the positive test did not differ between the 2 periods (27/100 [27.0%] in January–February vs 24/100 [24.0%] in June–August; P = .75) (Supplementary Figure 2).
Adjudication of cause of death by chart review, among vaccinated VA patients who died within 90 days of a first positive SARS-CoV-2 test in January–February 2022 (left panel) or June–August 2022 (right panel). Deaths were classified as being caused by COVID-19, or with COVID-19 being contributory, or with infection being noncontributory . A total of 150 cases were reviewed for deaths 1–30 days after documented infection and 100 cases 31–90 days after infection. Data are plotted as moving averages, with each point representing data from 10 days centered on that day. Abbreviations: COVID-19, coronavirus disease 2019; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.
In the TMC cohort, the percentages of COVID-19–related deaths may also have dropped by mid-2022, although numbers were too low to draw independent conclusions. Deaths within 30 days adjudicated as COVID-19–related were 19 of 24 (79.2%) between January 1 and 31 May 2021 (a period prior to predominance of the highly transmissible Delta and Omicron variants), 6 of 7 (85.7%) between July 1 and 30 November 2021 (Delta variant predominance), 30 of 33 (90.9%) between 1 December 2021 and 31 May 2022 (Omicron BA.1 and BA.2 predominance), and 3 of 7 deaths (42.9%) between June 1 and 13 October 2022 (Omicron BA.4 and BA.5 predominance). The causes of death for the 13 cases of death within 30 days of a positive test at TMC in which COVID-19 was determined to be noncontributory are summarized in Supplementary Table 6.
Death Certificates: TMC Cohort
Eighty-five of 86 (98.8%) patients in the TMC cohort had death certificates available for review; COVID-19 was listed as a cause of death in 52 of 85 patients (61.2%). The reviewers concurred that all 52 cases were COVID-19–related (PPV = 1.0). Among the 33 cases in which COVID-19 was not listed as a cause of death, the reviewers regarded the deaths as COVID-19–related in 19 cases (NPV = 0.42). A summary of the cases that adjudicators attributed to COVID-19 but that did not have COVID-19 listed as a cause of death on their death certificates is shown in Supplementary Table 7.
Causes of Death: Multivariable Modeling in the VA Cohort
In initial multivariable models using the full VA cohort of 500 manually reviewed cases of patients who died within 90 days, the time between the positive test and death (aOR: 0.80 per 5-d increase) and the calendar period June–August 2022 (aOR: 0.56 compared to January–February 2022) were associated with lower odds (Supplementary Table 8).
Hypoxemia and/or mechanical ventilation, use of drugs to treat COVID-19 during hospitalization, and use of immunosuppressive drugs before diagnosis of SARS-CoV-2 infection were associated with the outcome of COVID-19–related death, whereas recent chemotherapy was inversely associated. Similarly, advanced cancer was associated with the outcome of COVID-19 being noncontributory, although most other comorbidities had no significant association. Age was associated with increased odds of death being COVID-19–related (aOR: 1.31 per 5-y increase). As the models were simplified to remove most comorbidities and consolidate groups of medications, performance was only slightly lower (AUC: 0.90 vs 0.85–0.87) (Supplementary Table 8). Modeling then proceeded with the goal of developing and validating a parsimonious model using a small number of easily measured variables.
In the VA dataset of all 500 patients who died within 90 days, death within 30 days without additional variables had limited predictive value for measuring a COVID-19–related death (AUC: 0.656); use of a 40-day cutoff instead did not improve performance (AUC: 0.651) (Figure 2, Supplementary Figure 3); thus, 30 days was retained as the time-related variable for further analyses. The predictive utility improved with the addition of COVID-19 treatments (dexamethasone, baricitinib, tocilizumab, and/or remdesivir given after the positive test) and/or hypoxemia, but not with the addition of age (AUCs: 0.799 and 0.810, respectively). AUCs were higher if treatment was modeled with remdesivir and anti-inflammatory drugs (dexamethasone, baricitinib, and/or tocilizumab) as 2 separate variables (J.L. and P.A.M., unpublished data, 2023). Although predictive value was slightly improved (AUC: 0.869 vs 0.852) with the addition of variables that require complex coding (cancer, receipt of immunosuppressive drugs before infection), given the importance of feasibility for future implementation, these were not considered for inclusion in a final, pragmatic model.
Association of groups of clinical and demographic variables with COVID-19–related death: model development in the VA cohort. In the left panel, all 500 cases with death 1–90 days after a breakthrough positive SARS-CoV-2 test were included. The predictive values of death within 30 days (light green) or death within 40 days (dark green) were limited and similar (AUCs: 0.656 and 0.651, respectively). The addition of treatment variables to death within 30 days (pink; AUC: 0.784) had more benefit than adding hypoxemia (magenta; AUC: 0.745), with possible additional benefit of including all (orange; 0.799). In the middle panel, the dataset was limited to the 300 cases of death within 30 days. Hypoxemia alone (light green) had limited value (AUC: 0.640) but was better than mechanical ventilation alone (dark green; AUC: 0.542). Treatment variables plus hypoxemia (pink; AUC: 0.787) had slightly better performance than treatment alone (magenta; AUC: 0.767). In the right panel, in the dataset of 300 cases of death within 30 days, indices were made by adding either 3 variables (Index2 = hypoxemia, remdesivir, and anti-inflammatory drugs) or 4 variables (Index1 = also ventilation). Performance was similar with the 2 indices (light and dark green; AUCs: 0.770, 0.772 for Index1 and Index2, respectively), and slightly better when applied to the subcohort of patients diagnosed in January–February 2022 (magenta and pink; AUCs: 0.794, 0.786) than to the subcohort diagnosed in June–August 2022 (orange and brown; AUCs: 0.749, 0.755). Abbreviations: AUC, area under the receiver operating characteristic curve; COVID-19, coronavirus disease 2019; D/B/T, dexamethasone, baricitinib, or tocilizumab; H, hypoxemia; R, remdesivir; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; V, mechanical ventilation; VA, Veterans Health Administration.
The VA dataset was then limited to the 300 patients who died within 30 days, with remdesivir, anti-inflammatory drugs, and hypoxemia as the independent variables associated with COVID-19–related death. The AUC was 0.787, without any benefit from adding age or including mechanical ventilation instead of or in addition to hypoxemia (Figure 2, Supplementary Figure 4). Mechanical ventilation was not highly predictive, in part due to the high number of patients in the VA sample who were never intubated (Supplementary Table 5) per their expressed goals of care, whether their terminal illness was COVID-19–related or not. Two indices were made by adding the numbers of the independent variables for each patient, including remdesivir, anti-inflammatory drugs, and hypoxemia (0–3), without or with the addition of mechanical ventilation (0–4); AUCs were 0.770 and 0.772, respectively (Figure 2). Using the 0–3 index, a score of 2–3 had a PPV of 0.82 for COVID-19–related death and a score of 0–1 had an NPV of 0.64.
TMC Validation Cohort
In the validation cohort, the outcome of COVID-19–related death was strongly associated with a combination of the 3 variables remdesivir, anti-inflammatory treatment (which included dexamethasone in all cases), and hypoxemia (AUC: 0.885) (Figure 3, Supplementary Figure 5). The inclusion of age did not change the PPV. Although the addition of mechanical ventilation improved model performance slightly (AUC: 0.898), this variable was associated with reduced odds of adjudication as being COVID-19–related in the TMC cohort, due to a combination of collinearity with the other variables and ventilation for reasons other than hypoxemic respiratory failure (Supplementary Table 6).
Association of groups of clinical and demographic variables with COVID-19–related death: model validation in the Tufts Medical Center cohort. In the left panel, all 71 cases who died within 30 days after a positive SARS-CoV-2 test were included. Hypoxemia alone (light green) had limited value (AUC: 0.696) but was better than mechanical ventilation alone (dark green; AUC: 0.627). Treatment variables plus hypoxemia (pink; AUC: 0.885) had similar performance to treatment alone (magenta; AUC: 0.881). In the right panel, in the dataset of 71 cases of death within 30 days, indices were made by adding either 3 variables (Index2 = hypoxemia, remdesivir, and anti-inflammatory drugs) or 4 variables (Index1 = also ventilation). Performance was slightly better with Index2 (dark green; AUC: 0.883) than with Index1 (AUC: 0.859). Abbreviations: AUC, area under the receiver operating characteristic curve; COVID-19, coronavirus disease 2019; D/B/T, dexamethasone, baricitinib, or tocilizumab; H, hypoxemia; R, remdesivir; SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; V, mechanical ventilation.
The simple additive indices incorporating 3–4 easily ascertained variables (remdesivir, anti-inflammatory drugs, and hypoxemia, with or with addition of mechanical ventilation) were then tested for association with COVID-19–related death. The index 0–3 had better performance (AUC: 0.884) than the index 0–4 (AUC: 0.860) (Figure 3). Use of aORs from multivariable regression to weight the variables to calculate an index led to reduced performance (AUC: 0.866) compared with weighting each variable equally. Using the 0–3 index, a score of 2–3 had a PPV of 0.95 (56/59 cases) for a case being COVID-19–related and a score of 0–1 had an NPV of 0.83 (10/12 cases).
DISCUSSION
Among a longitudinal cohort of US veterans previously vaccinated against SARS-CoV-2, death within 30 days of a positive test for SARS-CoV-2 infection, a metric commonly used by states and territories to measure daily COVID-19 deaths, was COVID-19–related in 69% of cases in January–February 2022, whereas in June–August 2022, only 47% were COVID-19–related, a statistically significant decline. In the smaller TMC cohort, trends were similar, but the number of deaths in mid-2022 was too low for statistical analysis. The VA patients for whom COVID-19 was a contributor to death often had systemic or neurologic decline rather than hypoxemic respiratory illness, suggesting that nonrespiratory features of SARS-CoV-2 infection, although recognized since the beginning of the pandemic, have become a progressively more important feature of severe disease in susceptible patients. Patients who died either 1–30 or 31–90 days after infection despite vaccination typically had severe chronic illnesses regardless of whether COVID-19 was regarded as the cause of or a contributor to death. These findings have implications for the methods used to attribute deaths to COVID-19 and for institutional screening strategies, and therefore public health policy. We are unaware of other studies that used chart review as a gold standard, but our finding of a rising percentage of “contributory” cases in 2022 is consistent with CDC data [16].
Our review of inpatient deaths at TMC suggested that COVID-19 deaths may remain underreported by death certificates alone (PPV: 1.0; NPV: 0.42), even in an era where COVID-19–directed therapies and vaccines are readily available and testing is widespread. This finding is similar to that of a study conducted in Texas during the pre-vaccination era, which found that 75% of cases were reported as COVID-19–attributable deaths on death certificates versus 85% that underwent manual review [5]. Death certificate–based surveillance inherently relies on physicians’ individual assessments, which may underestimate the role of COVID-19 in hastening demise, particularly when the physician's only role is inpatient care. Death certificates may also be less reliable for identifying COVID-19–related deaths that occur in an outpatient setting, due to incomplete testing and/or lack of knowledge about the case, although this could not be explored in our study. Because few patients in our VA cohort had death certificates available, we were not able to assess their accuracy in that population. Additional study of the PPV and NPV of death certificates is needed in other healthcare systems.
Accurate data are needed to guide the ongoing policy response. Our results from 1 tertiary care hospital raise the concern that current NCHS metrics based on death certificates may be underreporting deaths attributable, at least in part, to COVID-19. At the same time, the simple method of counting deaths by using a positive test within 30 days of death overestimates COVID-19–related deaths.
Due to the inherent challenges with currently available surveillance methodologies, a predictive model using electronically available data may improve measurement accuracy. A simple index including receipt of remdesivir, anti-inflammatory medications, and hypoxemia performed well in both healthcare systems, with a slightly lower PPV (0.82%, 0.95%) but a substantially higher NPV (0.64%, 0.83%) compared with the use of death certificates alone. This index, in which a score of 2 or higher indicates COVID-19–related death, is likely generalizable to other settings, given its grounding in clinical care management pathways. Although highly correlated, the inclusion of remdesivir and anti-inflammatory drugs as separate variables led to improved model performance. We predict that antiviral treatment will continue to be an important and independent predictor as indications expand [17], severe nonrespiratory presentations predominate, and circumstances of use are likely to exceed guidelines. It is difficult to predict the extent to which the use of antiviral treatment, such as 3 days of remdesivir or nirmatrelvir-ritonavir given to inpatients, will be a surrogate marker for clinically significant, nonrespiratory disease versus clinically insignificant disease in high-risk patients. Thus, as clinical management and disease severity continue to evolve, these metrics will need updates and re-evaluation.
Notably, hypoxemia but not mechanical ventilation was an important predictor of COVID-19–related death. We suspect that this finding is driven by unique factors within the 2 healthcare systems, but both are important for generalizability. At TMC, other indications for ventilation, such as substance overdose, catastrophic neurologic events, and shock, contributed to an inverse association between mechanical ventilation and COVID-19–related death when added to other variables. In contrast, only 17.9% of the patients in the VA cohort whose deaths within 30 days were COVID-19–related received mechanical ventilation, largely due to patients and families opting for “do not intubate” (DNI) status based on chronic medical problems or advanced age. Additionally, the use of noninvasive mechanical ventilation, such as high-flow nasal cannula, has led to decreased utilization of mechanical ventilation, potentially further limiting the utility of this intuitively useful variable [18].
Each of the settings evaluated in the study offers advantages and disadvantages. The TMC cohort had the benefit of death certificate data, which enabled us to study their predictive value in determining COVID-19–related death. The main disadvantage of this dataset is that it only captured patients who died while hospitalized at TMC. Outpatients, patients who were transferred to another institution, or who otherwise died after discharge were not captured. Additionally, the single-center nature of this analysis limits generalizability. These data were complemented by the national, longitudinal nature of the VA EHR with near-complete linkage of vital status, which allowed evaluation of all deaths within 90 days of a known SARS-CoV-2 diagnosis, regardless of whether these deaths occurred on an inpatient or outpatient basis or in a VA long-term care facility. However, death certificates were not available for review from the VA cohort. Additionally, differences in care delivery in a government healthcare system, specifically with regard to mandatory compliance with the Food and Drug Administration’s Emergency Use Authorization policy for use of antivirals, may limit the generalizability of the predictive model to settings without these restrictions.
Several additional limitations should be noted. The reviewers may have been biased in classifying cases, particularly once trends became evident; since review of events at or before date of death and review of medical status before, during, and after the positive SARS-CoV-2 test were all essential, blinding was not feasible. For healthcare systems using this index through entirely electronic means, cases of dexamethasone use for other reasons could lead to overcounting, but such use is uncommon (eg, for multiple myeloma, but rarely for inflammatory diseases), and the goal of developing the index was to estimate case counts rather than to interpret individual cases. Similarly, the inclusion of other systemic glucocorticoids would improve sensitivity but, we suspect, at the cost of a greater loss in specificity unless cases are adjudicated individually (Supplementary File 1). In considering generalizability to the US population, the VA population is predominantly male and older, with underrepresentation of Hispanic/Latino and Asian patients, and the TMC cohort had a low vaccination rate. Although all cases at TMC were reviewed by 2 physicians, only a minority of VA cases received dual review after a limited estimate of interuser reliability. The number of cases in the study was small, but sufficiently large to feel confident about the validity of the differences that were observed, even if the estimates are imprecise.
Conclusions
Over time, the proportion of deaths within a 30–90-day time period that are attributable to COVID-19 has fallen substantially across 2 US healthcare systems serving very different populations. A simple predictive tool based on easily defined and extracted variables may improve the accuracy of COVID-19 death attribution. Surveillance definitions for measuring COVID-19 pandemic burden require ongoing adaption in order to maintain their predictive utility, and to achieve a real-time, learning healthcare system.
Supplementary Data
Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.
Notes
Author Contributions. Study conception and design: P. A. M., W. B.-E., S. D. Data acquisition: C. T., P. A. M., L. L. L., J. L., A. D. V. Data analysis: J. L., P. A. M., C. T., N. R. F. Funding and supervision: N. R. F. Drafting of the manuscript: P. A. M., W. B.-E., S. D., C. T. Approval of the final manuscript: all authors.
Acknowledgments. This research involved detailed review of the medical records of 586 patients after their deaths, and the research question could not have been answered otherwise. The authors express their condolences to their families and friends and acknowledge the veterans’ service in the US Armed Forces. They thank Sarah Abdella-el Kallassy and Allison Schulman at the VA National Artificial Intelligence Institute for consultation on design of the graphical abstract, Ms. Dipandita Basnet Thapa for her assistance with manuscript preparation and referencing, and Jonathan Morely and Alexa Zilberfarb of Tufts Medical Center for their assistance with this project. Study data were collected and managed using REDCap (Research Electronic Data Capture) electronic data capture tools hosted at Tufts Medical Center (www.projectredcap.org). REDCap is a secure, web-based software platform designed to support data capture for research studies, providing (1) an intuitive interface for validated data capture, (2) audit trails for tracking data manipulation and export procedures, (3) automated export procedures for seamless data downloads to common statistical packages, and (4) procedures for data integration and interoperability with external sources.
Disclaimer. The opinions discussed are those of the authors and not those of the Department of Veterans Affairs or other parts of the US government, which is the employer of most of the authors.
Financial support. This work was supported by the VA Cooperative Studies Program. C. T. was supported by National Institutes of Health/National Center for Advancing Translational Sciences (NIH/NCATS) training grant TL1TR002546.
References
Author notes
C. T., J. L., and L. L. L. contributed equally to this manuscript.
S. D. and P. A. M. contributed equally to this manuscript.
Potential conflicts of interest. W. B.-E. and P. A. M. were co-investigators on a study funded by Gilead Sciences (funds to their institution). All other authors report no potential conflicts.
All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.
