Durability analysis of the highly effective mRNA-1273 vaccine against COVID-19

Abstract COVID-19 vaccines are effective, but breakthrough infections have been increasingly reported. We conducted a test-negative case-control study to assess the durability of protection against symptomatic infection after vaccination with mRNA-1273. We fit conditional logistic regression (CLR) models stratified on residential county and calendar date of SARS-CoV-2 PCR testing to assess the association between the time elapsed since vaccination and the odds of symptomatic infection, adjusted for several covariates. There were 2,364 symptomatic individuals who had a positive SARS-CoV-2 PCR test after full vaccination with mRNA-1273 (“cases”) and 12,949 symptomatic individuals who contributed 15,087 negative tests after full vaccination (“controls”). The odds of symptomatic infection were significantly higher 250 days after full vaccination compared to the date of full vaccination (Odds Ratio [OR]: 2.47, 95% confidence interval [CI]: 1.19–5.13). The odds of non-COVID-19 associated hospitalization and non-COVID-19 pneumonia (negative control outcomes) remained relatively stable over the same time interval (Day 250 ORNon-COVID Hospitalization: 0.68, 95% CI: 0.47–1.0; Day 250 ORNon-COVID Pneumonia: 1.11, 95% CI: 0.24–5.2). The odds of symptomatic infection remained significantly lower almost 300 days after the first mRNA-1273 dose as compared to 4 days after the first dose, when immune protection approximates the unvaccinated state (OR: 0.26, 95% CI: 0.17–0.39). Low rates of COVID-19 associated hospitalization or death in this cohort precluded analyses of these severe outcomes. In summary, mRNA-1273 robustly protected against symptomatic SARS-CoV-2 infection at least 8 months after full vaccination, but the degree of protection waned over this time period.

Increasing reports of breakthrough infections (i.e. infections in fully vaccinated individuals) amidst the surge of the Delta variant starting in July 2021 raised questions about the durability of these vaccines, and these questions have only been augmented with the most recent global surge of the Omicron variant. After several studies demonstrated reductions in neutralizing antibody titers and protection against symptomatic SARS-CoV-2 infection several months after vaccination, the Centers for Disease Control and Prevention (CDC) recommended that individuals at high risk for infection or severe disease should receive an additional (booster) vaccine dose (19)(20)(21)(22)(23)(24)(25)(26)(27)(28)(29). However, relatively few studies have specifically analyzed the durability of mRNA-1273 (30) or even included individuals receiving this vaccine (20)(21)(22), while more studies have analyzed the durability of BNT162b2, perhaps due its wider use both in the United States and worldwide (31,32). It is thus critical to gather additional data to better understand the durability of protection against infection conferred by vaccination with mRNA-1273.

Results
Of 122,710 individuals who received two doses of mRNA-1273, 25 to 35 days apart, with no evidence of SARS-CoV-2 infection before reaching their date of full vaccination (14 days after the second dose), 20,165 subsequently underwent symptomatic testing and were eligible for inclusion in the test-negative case-control analysis ( Figure 1; Figure S1 and Table S1, Supplementary Material).
In the primary analysis, the conditional logistic regression (CLR) model was stratified on residential county and the calendar date of testing in order to control for geographic and temporal differences in community exposure levels, SARS-CoV-2 variant prevalence, and implementation of nonpharmaceutical interventions (NPIs) such as masking and social distancing. There were 2,364 cases and 15,087 controls that contributed to analyzable strata for this analysis (i.e. strata with at least one case and at least one control; Table S1, Supplementary Material). Demographic and clinical characteristics of cases and controls were generally similar to each other and to the underlying population of fully vaccinated individuals (  Figure 2A and Table 1). Among the other covariates, Asian race and the prior diagnosis of either cardiovascular disease or chronic pulmonary disease were associated with slightly lower odds of symptomatic infection after full vaccination (Table S2, Supplementary Material). The odds of experiencing non-COVID-19 hospitalization or pneumonia (negative controls) remained relatively stable over time ( Figure 2A and Table 1).
By stratifying on the calendar time of testing, individuals with small or large gaps between vaccination and testing within any given stratum were inherently vaccinated during different phases of the rollout. This would result in the comparison of individuals who likely differed in comorbidities, occupational risks of exposure to SARS-CoV-2, willingness to be vaccinated, and/or likelihood of engaging with the healthcare system. To address this, we modified the CLR to instead stratify on the date of vaccination and the county-level COVID-19 incidence at the time of PCR testing (see Methods). This yielded 2,388 positive symptomatic tests (cases) and 20,978 negative symptomatic tests (controls) contributing to analyzable strata (Table S1, Supplementary Material). With this modified approach, the adjusted odds of symptomatic infection at 150, 200, and 250 days after full vaccination were 8.78 (95% CI: 3.54-21.73), 13.2 (95% CI: 5.30-32.7), and 23.0 (95% CI: 9.11-58.1) times higher than at the date of full vaccination, respectively ( Figure S2A, Supplementary Material; Table 1). Asian race and the prior diagnosis of cardiovascular disease were associated with modestly lower odds of symptomatic infection, while male sex was associated with slightly higher odds (Table S3, Supplementary Material).
To contextualize any changes in the odds of infection after full vaccination, it is important to understand how they compare to the odds during a period before the onset of vaccinemediated protection. Thus, we modified the CLR to assess the odds of symptomatic infection over time starting 4 days after the first dose ("post-first dose CLR"), a reference point intended to approximate the unvaccinated state (see Methods). Of 143,443 individuals who received at least one dose of mRNA-1273 with no positive SARS-CoV-2 tests prior to vaccination, 22,864 underwent symptomatic testing after their first dose (  Figure 2B and Table 1). Increasing odds at later time points indicated a reduction in protection against symptomatic infection compared to the expected full vaccination date (e.g. OR Day 192 : 0.16; 95% CI: 0.11-0.24; OR Day 242 : 0.20; 95% CI: 0.14-0.30; OR Day 292 : 0.26; 95% CI: 0.17-0.39; Figure 2B and Table 1). A similar pattern was observed among individuals who were at least 65 years old ( Figure S3A and  Figure 2B and Table 1). A similar trend was observed Figure 1. Schematic representation of study design. From top to bottom, (i) inclusion and exclusion criteria to define the population eligible for this test-negative analysis, (ii) definition of the clinical outcomes of interest, (iii) framework for statistical analysis, and (iv) schematic representation of data interpretation. (i) Individuals were included if they were at least 18 years old, had a record of at least one encounter at the Mayo Clinic in the 3 years prior to the study start date, and were fully vaccinated per-protocol with mRNA-1273 (with the first dose administered on or after 2020 December 15), and underwent at least one symptomatic SARS-CoV-2 PCR test after the date of full vaccination. Individuals were excluded if they received a positive PCR test prior to their date of full vaccination. (ii) The outcome was defined as symptomatic SARS-CoV-2 infection, and cases and controls were defined accordingly. (iii) Conditional logistic regression was used to assess the potential relationship between the odds of experiencing symptomatic infection and time since vaccination, while accounting for other clinical and demographic covariates. (iv) The odds of symptomatic infection were assessed over time after full vaccination (modeled as a linear spline) relative to the odds at the date of full vaccination, which is expected to correspond to maximal vaccine-mediated protection. An increase in the odds ratio with time since vaccination would be interpreted as evidence for waning protection, while a consistent odds ratio over (relative) time would be interpreted as durable protection. among individuals who were at least 65 years old ( Figure S3B and Table S6, Supplementary Material).

Discussion
Taken together, these data suggest that the strong protection against symptomatic SARS-CoV-2 infection which is conferred by vaccination with mRNA-1273 wanes over time. This trend is consistent with updated results from the phase 3 clinical trial of mRNA-1273, in which it was found that that individuals vaccinated by October 2020 (median follow-up: 13 months) were less protected against breakthrough infection during July and August 2021 than those vaccinated after December 1 2020 (median follow-up: 8 months) (33). This is also consistent with a recent large real-world analysis of a veteran population in the United States, which demonstrated waning effectiveness of mRNA-1273 over time through November 2021 (34). The current study provides additional value by analyzing a longer period which extends Odds ratios correspond to the odds of experiencing the outcome at a given time t divided by the odds of experiencing that outcome at the reference date (t = 0 days since full vaccination), adjusted for several demographic and clinical covariates via CLR. (B) Adjusted odds ratios of outcomes over time relative to 4 days after the first mRNA-1273 dose, which is intended to approximate the unvaccinated status. Odds ratios correspond to the odds of experiencing the outcome at a given time t divided by the odds of experiencing that outcome at the reference date (t = 4 days since first vaccine dose), again adjusted for demographic and clinical covariates via CLR. The outcomes are symptomatic SARS-CoV-2 infection (shown in orange), non-COVID-19 associated hospitalization, and non-COVID-19 pneumonia (negative control outcomes shown in gray). For both panels, the CLR was stratified on residential county and the date of SARS-CoV-2 PCR testing.
through the era of the Omicron variant, which marked the largest surge to date in the United States among both vaccinated and unvaccinated individuals. Given that approximately 100 million Americans have now received booster doses, we highlight the importance of continuing to monitor the durability of protection afforded by both primary and booster vaccination with mRNA-1273.
Both approaches that we considered-namely, stratifying the CLR on the date of testing or the date of vaccination-indicate that the effectiveness of mRNA-1273 against symptomatic infection wanes over time. However, the point estimates were quite different between these approaches, with stratification by vaccination date suggesting considerably stronger waning. It is likely that the true trend lies between their estimates. Unfortunately, because our primary variable of interest itself is mathematically defined by these two dates (i.e. Time Since Vaccination = Date of Testing-Date of Vaccination), it is not feasible to control for both of these variables in a single method. It is thus important to recognize the strengths and weaknesses of both approaches.
Stratification on the time of testing controls for community level exposure, NPI practices in place at the time of symptom onset, and (perhaps most importantly) the current local variant landscape, but it ignores differences in the underlying characteristics of individuals vaccinated during different phases of the rollout. Such characteristics certainly include age, comorbidities, and occupation, and may include other factors such as willingness to be vaccinated, likelihood of interaction with the healthcare system, and the symptomaticity threshold required to seek out a COVID-19 test. While we do adjust for age and several COVID-19 associated comorbidities by including them as covariates in the model, we are not able to control for these other characteristics. Table 1. Adjusted odds of experiencing each outcome of interest at defined dates after full vaccination or the first vaccine dose. The outcomes of interest are symptomatic SARS-CoV-2 infection, non-COVID-19 associated hospitalization (negative control), and non-COVID-19 pneumonia (negative control). The top panel shows adjusted odds of experiencing each outcome in 50-day intervals after the date of full vaccination (i.e. 14 days after the second dose). The bottom panel shows adjusted odds of experiencing each outcome at relevant time points after the first vaccine dose; the expected date of second dose administration is 28 days after the first dose, and the expected date of full vaccination is 42 days after the first dose. On the left side, results are shown for the CLR models stratified on residential county and the calendar date of SARS-CoV-2 testing. On the right side, results are shown for the CLR models stratified on residential county, the date of full vaccination, and the 7-day trailing county-level COVID-19 incidence on the date of the SARS-CoV-2 PCR test. On the other hand, stratification on the time of vaccination intentionally controls for these characteristics, but makes it more difficult to account for the effects of variants and NPIs. We do try to address this shortcoming by adding the dominant variant and county-level COVID-19 incidence as covariates to the model. However, because Omicron is both the latest and most immunoevasive variant to date, it is possible that this method would not adequately discriminate between the established reduction in vaccine effectiveness against Omicron and a natural decline in vaccine effectiveness over time (35)(36)(37)(38)(39)(40)(41). That said, the more pronounced waning signal suggested by this approach is still quite relevant, as it provides an alternative view of "real world waning" that occurs as a function of both time and the introduction of newly evolved SARS-CoV-2 variants. In addition to assessing waning protection after full vaccination, it is important to contextualize this analysis by estimating protection compared to the unvaccinated state. Because the study cohort only included vaccinated individuals, it was not possible to establish this true baseline rate of symptomatic infection at any given time in the absence of vaccination. However, to approximate this, we leveraged the fact that the 1 to 2 weeks after the first mRNA vaccine dose can serve as a proxy for the unprotected state (3,4). The odds of symptomatic infection at the expected date of full vaccination (42 days after the first dose) compared to this unprotected state were 0.05 when the CLR was stratified on the date of testing or the date of vaccination, respectively. This corresponds to an estimated effectiveness of about 95% by both methods, consistent with the expected protection based on results of the phase 3 randomized trial and several real world effectiveness studies (3,5,9,12,13,42). By about 240 days after the first dose, these relative odds increased to 0.20 and 0.58 when stratifying on the date of testing or vaccination, respectively, suggesting a decline in effectiveness to approximately 40% to 80% over the 200 days following full vaccination. This estimate is quite consistent with the previously reported decline in mRNA-1273 effectiveness from 89% to 58% in a large United States veteran population over a similar period (34).
This study has limitations. First, the cohort is over 90% Caucasian and, thus does not reflect the broader population of vaccinated individuals in the United States or globally. Second, there are individual-level SARS-CoV-2 exposure risks that could not be directly accounted for in our analyses, such as adherence to recommendations regarding masking, travel, and social distancing. As a retrospective observational study, there also may be additional unconsidered confounding factors that were unequally distributed within the study population. Third, it is challenging to select and interpret an appropriate negative control outcome. Ideally, such an outcome would be related to the symptomatic presentation to the clinic but unaffected by vaccination status (or time elapsed since vaccination). For example, prior studies of influenza vaccine effectiveness have considered the lab testbased diagnosis of another respiratory infection as a negative control outcome (43). However, the impact of the COVID-19 pandemic on the incidence and diagnosis of other respiratory illnesses (e.g. low rates of influenza until the end of 2021) made it difficult to consider such outcomes as robust negative controls (44)(45)(46). Fourth, because we censored individuals after receiving a third dose of mRNA-1723 (if applicable), any increase in effectiveness or durability attributable to booster doses is not reflected in this study. Finally, due to the low absolute number of COVID-19 associated hospital admissions, ICU admissions, and deaths after full vaccination with mRNA-1273 in this study, there was not adequate power to assess vaccine durability against severe illness. mRNA-1273 demonstrated strong protection against symptomatic and severe disease in clinical trials and the real world setting during early phases of the vaccine rollout internationally (3,5,9,13,27,42). This study suggests that mRNA-1273 strongly protects against symptomatic infection for at least 8 months after full vaccination, but its effectiveness declines over time. Moreover, the reduced effectiveness at later time points was amplified by the introduction of the highly immunoevasive Omicron variant. It is important to continue delivering first and second vaccine doses to as many people as possible, while also continuing to administer boosters to eligible and at-risk populations. Further, it will be important to continuously evaluate the durability of protection afforded by mRNA-1273 and other vaccines both over time and in the context of any new SARS-CoV-2 variants following Omicron.

Study population
This is a retrospective study of individuals who were vaccinated with mRNA-1273 between 2020 December 15 and 2022 January 27, and who subsequently underwent PCR testing for suspected symptomatic SARS-CoV-2 infection at the Mayo Clinic. According to the CDC, full vaccination with mRNA-1273 is defined as beginning 14 days after the second dose (47). This study was reviewed and deemed exempt by the Mayo Clinic institutional review board. Those who had specifically opted out of inclusion of electronic medical records in research were excluded. Inclusion and exclusion criteria were defined as follows:

Exclusion criteria
1. Any positive SARS-CoV-2 PCR test prior to the date of full vaccination.
The derivation of this study population is illustrated in Figure  S1 (Supplementary Material), and the demographic and clinical characteristics of the cohort along with the underlying fully vaccinated population is shown in Table 1.

Study design
We performed a test-negative case-control analysis to assess whether the protection conferred by mRNA-1273 wanes over time, similar to a study design described previously to analyze intraseason waning effectiveness of influenza vaccination (43). To do so, we used CLR to assess the odds of symptomatic SARS-CoV-2 infection and two negative control outcomes (non-COVID-19 hospitalization and non-COVID-19 pneumonia) over time after full vaccination, while adjusting for relevant covariates. Symptomatic infection was defined as a positive result from a SARS-CoV-2 PCR test that was not designated as "asymptomatic" by the ordering provider (subsequently referred to as "symptomatic tests"). Because we expect the date of full vaccination to approximate the time of maximal protection, we assess vaccine durability by estimating the odds of symptomatic infection at 50, 100, 150, 200, and 250 days after this time point.

Definitions of cases, controls, and at-risk time
Cases were defined as the first positive symptomatic test for a given individual; if an individual contributed multiple positive tests, only their first test was included as a case. Controls were defined as negative symptomatic tests, which did not occur after any prior positive SARS-CoV-2 PCR tests (asymptomatic or symptomatic). Individuals who met the inclusion and exclusion criteria outlined above were eligible to contribute cases and controls from their date of full vaccination until they (i) had any positive test result (symptomatic or asymptomatic), (ii) received a third dose of any COVID-19 vaccine (mRNA-1273, BNT162b2, or Ad26.COV2.S), (iii) died, or (iv) reached the end of the study period. If an individual contributed a negative symptomatic test 15 or fewer days before a positive test, that negative test was excluded as a possible false negative. If an individual contributed multiple negative symptomatic tests within 15 days of each other, then one of those tests was randomly selected as a control while the others were dropped; this step was taken to avoid counting multiple controls from a potential single symptomatic illness. Further, if an individual contributed more than three negative symptomatic tests over the study duration, then three tests were randomly selected as controls while the others were dropped, as was recently described in a test-negative case-control study of COVID-19 vaccine effectiveness (8).
As a negative control analysis, we assessed protection against non-COVID-19 hospitalization and non-COVID-19 pneumonia, outcomes which we do not expect to be impacted by time since vaccination. In other test-negative designs of influenza vaccine effectiveness, the diagnosis of other respiratory infections was considered as the negative control (43). Because of the myriad impacts of the pandemic and NPIs on other respiratory infections (e.g. very low rates of influenza in 2020 and through most of 2021), such an approach may not be valid here (48). Although non-COVID-19 related hospitalization could be impacted by factors such as changes in healthcare-seeking behavior (including elective procedures) after vaccination, it appeared to be a reasonable negative control to evaluate. For non-COVID-19 hospitalization, cases were defined as instances in which an individual experienced a negative symptomatic test (i.e. ruled out for COVID-19 diagnosis) and was subsequently admitted to the hospital within 14 days. Controls were defined as instances in which an individual experienced a negative symptomatic test and was not subsequently admitted to the hospital within 14 days. Individuals who met the inclusion and exclusion criteria outlined above were eligible to contribute cases and controls from their date of full vaccination until they (i) were hospitalized within 14 days of a negative symptomatic test, (ii) received a third dose of any COVID-19 vaccine (mRNA-1273, BNT162b2, or Ad26.COV2.S), (iii) died, or (iv) reached the end of the study period. The same rules were applied as described above for cases in which an individual contributed (i) a negative test shortly before contributing a positive test, (ii) multiple negative symptomatic tests within 15 days of each other, or (iii) more than three negative symptomatic tests over the duration of the study. Because 14 days of follow-up were required after a positive symptomatic test to observe this outcome, cases and controls were only considered from tests that were performed on or before 2022 January 17 (14 days before the last date of data collection). A similar process was followed for non-COVID-19 pneumonia, except that cases were defined by the presence of at least one corresponding ICD-10 code (J12-J18, with the exceptions of J12.81, J12.82, and J12.89) within 14 days of a negative symptomatic test.

Primary exposure, covariates, and stratification factors
Variables that are potentially associated with the likelihood of eligibility for vaccination at a given time, seeking out vaccination, testing positive for SARS-CoV-2, or experiencing severe COVID-19 were included as covariates or stratification variables in the regression models. The primary exposure of interest and each such other variable, denoted as X 1 to X 15 in the regression equation listed in the Statistical Analysis section, is described below.
Primary exposure r X 1 : time since full vaccination, defined as the number of days between the symptomatic PCR test and the date of full vaccination. This variable was modeled as a linear spline with knots at 50-day intervals since the full vaccination date. As described above, the date of full vaccination is expected to correspond to a time of maximal protection and, thus was considered as the reference. Results are presented as the odds of symptomatic infection at each knot relative to this reference.
Covariates r X 2 : age in years as of the study start date (2020 December 15), modeled as a linear spline with knots at 25, 35, 45, 55, 65, 75, and 85 years. The minimum age (18 years old) was considered the reference. Results are presented as the odds of symptomatic infection at each subsequent knot relative to this reference. r X 3 to X 10 : individual comorbidity categories, binarized based on whether the individual had at least one instance of a corresponding ICD-9 or ICD-10 code in the 5 years prior to the study period. The comorbidity categories included cardiovascular disease, pulmonary disease, diabetes, kidney disease, liver disease, HIV/AIDS, cancer, and obesity. r X 11 : race, categorized into seven groups (listed alphabetically: Asian, Black/African American, Native American, Native Hawaiian/Pacific Islander, other, White, and unknown). White was considered the reference category because it comprised the majority of individuals in the study. r X 12 : ethnicity, categorized into three groups (listed alphabetically: Hispanic/Latino, not Hispanic/Latino, and unknown). "Not Hispanic/Latino" was considered the reference category because it comprised the majority of individuals in the study.

Determination of comorbidities
We used the comorbidity package (version 0.5.3) in R (version 4.1.0, www.r-project.org, Vienna, Austria) to identify ICD-9 and ICD-10 codes that correspond to the comorbidity categories listed above. For each individual, we extracted all such diagnosis codes in the Mayo Clinic electronic health record from the 5 years preceding this study (i.e. between 2015 December 15 and 2020 December 15). The eight comorbidity categories were defined as one or more diseases from the Elixhauser score as follows:

Statistical analysis
Briefly, for each outcome (i.e. symptomatic SARS-CoV-2 infection, non-COVID-19 hospitalization, and non-COVID-19 pneumonia), we fit a CLR model to estimate the odds of experiencing the outcome of interest each day after the date of full vaccination compared to the odds of experiencing that outcome on the date of full vaccination, while adjusting for the covariates described above.
Models were fit using the clogit function from the survival package (version 3.2.11) in R (Version, 4.1.0, www.r-project.org, Vienna, Austria). Confidence intervals and tests for individual covariates were based upon the Wald method, and the Efron method was used to approximate the conditional likelihood. Odds ratios were considered statistically significant if the confidence interval did not include 1. In addition, Nagelkerke R-squared values are reported for each model in Table S8 (Supplementary Material).

Secondary analysis: odds of infection relative to the first vaccine dose rather than full vaccination
The primary analysis estimates the change in odds of infection over time relative to the time of maximal vaccine protection without providing the important context of the baseline risk of infection in the absence of vaccination. In our study population, which consists entirely of vaccinated individuals with no prior history of COVID-19 diagnosis, it is not possible to directly measure this risk of infection in the unvaccinated state. However, the protective effect of mRNA-1273 sets in after approximately 2 weeks, and we reasoned that the risk of infection in the unvaccinated state could be approximated by the risk of infection shortly after initial vaccination (e.g. within Days 4 to 10 after the first dose) (3,5). We, thus modified the inclusion and exclusion criteria from our primary analysis as follows.

Funding
This study was funded by nference.