https://orcid.org/0009-0007-2256-1983
Abstract
During the coronavirus disease 2019 (COVID-19) pandemic, Pfizer/BioNTech BNT162b2, and Moderna mRNA-1273 vaccines were central to the global pandemic control measures.
Here, we conducted a systematic review and meta-analysis to evaluate their real-world vaccine effectiveness (VE). Our study focussed on those that reported the efficacy of these vaccines against COVID-19 hospitalization. Hospitalization was chosen as the primary outcome as it directly reflects the ability of the vaccine to prevent severe disease. A literature search was undertaken using Medline and Embase on 25 February 2024. From this, 50 studies out of 18,347 articles were included for further analysis.
High VE against hospitalization was reported for both the BNT162b2 and mRNA-1273 COVID-19 vaccines when used either as a primary vaccination series (2-dose) or following an additional booster dose (3-dose). Meta-analysis indicated that the pooled VE estimates for each of these vaccination protocols ranged from 84% to 86%, suggesting strong protectiveness. Our data also imply that booster doses can restore waning effectiveness, with no significant differences observed in VE between the 2-dose and 3-dose protocols. However, subgroup analysis revealed an association between the presence of the Omicron variant and a drop in VE, indicating that future emerging SARS-CoV-2 virus variants could similarly affect VE.
Our review underscores the importance of ongoing research to ensure vaccine strategies remain effective against evolving variants. Our study also identified the need for expanding data collection to include underrepresented populations.
Introduction
Coronavirus disease 2019 (COVID-19) is caused by infection with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), a positive-sense, single-stranded RNA virus of the genus Betacoronavirus. The SARS-CoV-2 coronavirus shares a lineage with SARS-CoV-1 and MERS-CoV, both responsible for previous severe respiratory syndromes [1]. The virus’s spike proteins, essential for infection and the primary target of the host immune response, facilitate its high transmissibility [2]. Early in the pandemic, 81% of reported COVID-19 cases exhibited mild symptoms, such as cough and fever, while 14% developed severe symptoms including dyspnoea, with 5% progressing to critical conditions such as respiratory failure and multiple organ dysfunction [3, 4]. The reproductive number (R0) for the original strain of SARS-CoV-2 was estimated at 3.14, significantly higher than that of seasonal influenza (R0 = 1.28) [5, 6]. This high transmissibility contributed to the rapid global spread of the SARS-CoV-2 coronavirus. On 30 January 2020, the World Health Organization declared COVID-19 a public health emergency. As of March 2024, the pandemic had resulted in over 7 million deaths and 774 million cases worldwide [7].
Despite the development and deployment of various COVID-19 vaccines during the pandemic, significant gaps remain in our understanding of their real-world effectiveness. The Pfizer/BioNTech BNT162b2 and Moderna mRNA-1273 COVID-19 vaccines, which were rapidly developed using mRNA technology, demonstrated high efficacy in clinical trials, with reported vaccine efficacy (VE) rates against COVID-19 infection of 95% and 94.1%, respectively [8, 9]. However, translating these efficacy rates into real-world effectiveness is complex. Vaccine effectiveness (VE) in practical settings can be influenced by demographic variations, storage conditions, as well as vaccine uptake levels [10]. In addition, the emergence of SARS-CoV-2 variants of concern (VOCs), such as Alpha, Beta, Gamma, Delta, and Omicron, which possess mutations affecting viral transmissibility and immune escape, further complicate this assessment [11, 12]. Studies have indicated that these variants, particularly Delta and Omicron, reduce VE, underscoring the necessity for continuous evaluation of vaccine performance in response to evolving viral strains [13].
In our study, we systematically reviewed and meta-analysed the real-world effectiveness of the Pfizer/BioNTech BNT162b2 and Moderna mRNA-1273 vaccines against COVID-19 hospitalization. Specifically, the objectives were to: (1) summarize the existing research undertaken on the VE of the BNT162b2 and mRNA-1273 COVID-19 mRNA vaccines against hospitalization; (2) assess the VE of primary series and booster doses throughout the pandemic; and (3) investigate potential differences in VE during the emergence of different VOCs. By focussing on hospitalization as the primary endpoint, our study aimed to provide insights into each vaccine’s effectiveness in preventing severe disease, which is critical for public health planning and resource allocation. This analysis will aid our understanding of mRNA COVID-19 vaccine performance in an evolving pandemic landscape.
Materials and methods
Search strategy
A literature search was conducted on 25 February 2024 using two online databases: Embase (https://www.embase.com/) and Medline (https://www.nlm.nih.gov/medline/). The concept words and search terms used in the search are presented in Table 1. The same search strategy was applied to both databases. The search strategy was designed to cover all the literature reporting the effectiveness or efficacy of any vaccine format against COVID-19 independent of vaccination protocols, vaccine design or effectiveness endpoints.
A table showing the search strategy designed for the literature search. The concept words and search terms used in the literature search were shown.
| Concept words | Search terms |
|---|---|
| COVID-19 | ‘SARS-CoV-2’ OR ‘COVID-19’ OR ‘COVID’ |
| AND | |
| Vaccine | ‘vaccine’ OR ‘vaccination’ OR ‘booster*’ OR ‘third dose’ OR ‘fourth dose’ OR ‘fifth dose’ |
| AND | |
| Effectiveness | ‘effectiveness’ OR ‘efficacy’ |
| Concept words | Search terms |
|---|---|
| COVID-19 | ‘SARS-CoV-2’ OR ‘COVID-19’ OR ‘COVID’ |
| AND | |
| Vaccine | ‘vaccine’ OR ‘vaccination’ OR ‘booster*’ OR ‘third dose’ OR ‘fourth dose’ OR ‘fifth dose’ |
| AND | |
| Effectiveness | ‘effectiveness’ OR ‘efficacy’ |
A table showing the search strategy designed for the literature search. The concept words and search terms used in the literature search were shown.
| Concept words | Search terms |
|---|---|
| COVID-19 | ‘SARS-CoV-2’ OR ‘COVID-19’ OR ‘COVID’ |
| AND | |
| Vaccine | ‘vaccine’ OR ‘vaccination’ OR ‘booster*’ OR ‘third dose’ OR ‘fourth dose’ OR ‘fifth dose’ |
| AND | |
| Effectiveness | ‘effectiveness’ OR ‘efficacy’ |
| Concept words | Search terms |
|---|---|
| COVID-19 | ‘SARS-CoV-2’ OR ‘COVID-19’ OR ‘COVID’ |
| AND | |
| Vaccine | ‘vaccine’ OR ‘vaccination’ OR ‘booster*’ OR ‘third dose’ OR ‘fourth dose’ OR ‘fifth dose’ |
| AND | |
| Effectiveness | ‘effectiveness’ OR ‘efficacy’ |
Study eligibility criteria
The eligibility criteria were designed in alignment with the PICOS framework, which outlines the scope of a review based on population, intervention, comparison, outcome, and study type [14]. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist is provided in the Supplementary information. A comprehensive list of the inclusion and exclusion criteria used in this study is provided in Table 2. Studies were limited to those that reported VE for the Pfizer/BioNTech BNT162b2 and Moderna mRNA-1273 COVID-19 vaccines. Hospitalization was selected as the sole VE endpoint in this review as it directly reflects the ability of the vaccine to prevent severe disease. Thus, studies not reporting VE for the two vaccines of interest or reporting VE for outcomes other than hospitalization were excluded (Table 2). Studies reporting VE in an at-risk population subgroup such as health care workers or elderly care home residents were also excluded, as these subgroups may not provide a representative assessment of VE in the broader community [15, 16]. The review included studies reporting absolute VE and excluded those reporting relative VE. Absolute VE was defined as the reduction in COVID-19 hospitalization in the vaccinated group relative to an unvaccinated group. Relative VE, in contrast, compares the reduction of disease between different vaccination protocols [17].
Inclusion and exclusion criteria were designed based on the PICOS framework. Only studies that met all the inclusion criteria were included in the analysis.
| PICOS | Inclusion criteria | Exclusion criteria |
|---|---|---|
| Population | Studies reporting findings on adults (>18) | Animal studies; Studies focussing on demographics with comorbidities; Studies focussing on at-risk populations, for example, healthcare workers, elder care home residents; Studies reporting findings on children or adolescents (<18) only; Studies reporting findings on the elderly (>65) only |
| Intervention | Any dosage of monovalent Pfizer-BioNTech BNT162b2 vaccine; Any dosage of monovalent Moderna mRNA-1273 vaccine | Any other COVID-19 vaccines; Any bivalent COVID-19 vaccine |
| Comparison | Comparing VE between vaccinated group and unvaccinated population group | Comparing VE between different vaccination protocols, for example, comparing VE of 2-dose BNT162b2 against 3-dose BNT162b2 |
| Outcome | Absolute VE (%) against COVID-19 hospitalization | Studies reporting VE on outcomes other than hospitalization, for example, infection, severe disease, death, and psychological impact; Studies with incomplete data; Studies reporting VE in effect sizes (e.g. risk ratio, odds ratio, or hazard ratio) instead of percentages |
| Study type | Cohort studies; Case-control studies; Test-negative studies; Registry-based studies | Studies not reporting on primary data, for example, reviews, systematic reviews; Case reports or case series; Opinion pieces; Correspondence; Studies that were not peer-reviewed; Studies not written in English |
| PICOS | Inclusion criteria | Exclusion criteria |
|---|---|---|
| Population | Studies reporting findings on adults (>18) | Animal studies; Studies focussing on demographics with comorbidities; Studies focussing on at-risk populations, for example, healthcare workers, elder care home residents; Studies reporting findings on children or adolescents (<18) only; Studies reporting findings on the elderly (>65) only |
| Intervention | Any dosage of monovalent Pfizer-BioNTech BNT162b2 vaccine; Any dosage of monovalent Moderna mRNA-1273 vaccine | Any other COVID-19 vaccines; Any bivalent COVID-19 vaccine |
| Comparison | Comparing VE between vaccinated group and unvaccinated population group | Comparing VE between different vaccination protocols, for example, comparing VE of 2-dose BNT162b2 against 3-dose BNT162b2 |
| Outcome | Absolute VE (%) against COVID-19 hospitalization | Studies reporting VE on outcomes other than hospitalization, for example, infection, severe disease, death, and psychological impact; Studies with incomplete data; Studies reporting VE in effect sizes (e.g. risk ratio, odds ratio, or hazard ratio) instead of percentages |
| Study type | Cohort studies; Case-control studies; Test-negative studies; Registry-based studies | Studies not reporting on primary data, for example, reviews, systematic reviews; Case reports or case series; Opinion pieces; Correspondence; Studies that were not peer-reviewed; Studies not written in English |
Inclusion and exclusion criteria were designed based on the PICOS framework. Only studies that met all the inclusion criteria were included in the analysis.
| PICOS | Inclusion criteria | Exclusion criteria |
|---|---|---|
| Population | Studies reporting findings on adults (>18) | Animal studies; Studies focussing on demographics with comorbidities; Studies focussing on at-risk populations, for example, healthcare workers, elder care home residents; Studies reporting findings on children or adolescents (<18) only; Studies reporting findings on the elderly (>65) only |
| Intervention | Any dosage of monovalent Pfizer-BioNTech BNT162b2 vaccine; Any dosage of monovalent Moderna mRNA-1273 vaccine | Any other COVID-19 vaccines; Any bivalent COVID-19 vaccine |
| Comparison | Comparing VE between vaccinated group and unvaccinated population group | Comparing VE between different vaccination protocols, for example, comparing VE of 2-dose BNT162b2 against 3-dose BNT162b2 |
| Outcome | Absolute VE (%) against COVID-19 hospitalization | Studies reporting VE on outcomes other than hospitalization, for example, infection, severe disease, death, and psychological impact; Studies with incomplete data; Studies reporting VE in effect sizes (e.g. risk ratio, odds ratio, or hazard ratio) instead of percentages |
| Study type | Cohort studies; Case-control studies; Test-negative studies; Registry-based studies | Studies not reporting on primary data, for example, reviews, systematic reviews; Case reports or case series; Opinion pieces; Correspondence; Studies that were not peer-reviewed; Studies not written in English |
| PICOS | Inclusion criteria | Exclusion criteria |
|---|---|---|
| Population | Studies reporting findings on adults (>18) | Animal studies; Studies focussing on demographics with comorbidities; Studies focussing on at-risk populations, for example, healthcare workers, elder care home residents; Studies reporting findings on children or adolescents (<18) only; Studies reporting findings on the elderly (>65) only |
| Intervention | Any dosage of monovalent Pfizer-BioNTech BNT162b2 vaccine; Any dosage of monovalent Moderna mRNA-1273 vaccine | Any other COVID-19 vaccines; Any bivalent COVID-19 vaccine |
| Comparison | Comparing VE between vaccinated group and unvaccinated population group | Comparing VE between different vaccination protocols, for example, comparing VE of 2-dose BNT162b2 against 3-dose BNT162b2 |
| Outcome | Absolute VE (%) against COVID-19 hospitalization | Studies reporting VE on outcomes other than hospitalization, for example, infection, severe disease, death, and psychological impact; Studies with incomplete data; Studies reporting VE in effect sizes (e.g. risk ratio, odds ratio, or hazard ratio) instead of percentages |
| Study type | Cohort studies; Case-control studies; Test-negative studies; Registry-based studies | Studies not reporting on primary data, for example, reviews, systematic reviews; Case reports or case series; Opinion pieces; Correspondence; Studies that were not peer-reviewed; Studies not written in English |
The Covidence App for duplicate removal, title, and abstract screening
The literature search results were uploaded to the Covidence App (https://www.covidence.org/), a web-based software platform for the automatic identification and removal of duplicates. Additional duplicates identified in the screening process were removed manually. After removing duplicates, the titles and abstracts were screened in the remaining unique articles based on the eligibility criteria presented in Table 2.
Data extraction
Data extraction from the full texts of the eligible studies was conducted and archived using Microsoft Excel (Version 2403). Data extracted from eligible studies included the study characteristics, VE% against COVID-19 hospitalization and the vaccination protocols. The entire review process, including study identification, screening, and data extraction, was performed by BKFW.
Statistical analysis
Data processing and preliminary analyses were primarily performed in Microsoft Excel. Advanced statistical analyses and data visualization were undertaken using RStudio (version 4.2.2). The forest plots were generated using the R packages ‘ggplot2’, ‘dplyr’, and ‘patchwork’. Given the high heterogeneity observed among the included studies, a meta-analysis was conducted using a random effects model using the metafor package. This approach accounts for variability across studies and is suitable when there is significant heterogeneity [18]. The I2 statistic, which measures the proportion of total variability due to heterogeneity, was calculated to evaluate the consistency of results among the studies [19].
Results
Characteristics of the included studies
The literature search results, structured according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Supplementary information) are presented in Fig. 1 [20]. Initially, 16,492 records were identified from Embase and 11,282 records from Medline. Upon removal of 9,427 duplicates, of which 11 records were identified manually, 18,347 unique studies underwent title and abstract screening. Subsequently, 890 studies were eligible for full-text screening, during which 839 studies were excluded as they did not match the inclusion criteria for the study, such as VE measures other than hospitalization (Fig. 1). Ultimately, 50 studies met the inclusion criteria and were included in this review. The details of each included study are provided in Supplementary Table 1.
PRISMA flow diagram depicting the number of studies at each stage of the review process. The flowchart was generated using the Covidence App.
Of the 50 studies that met the eligibility criteria, the majority (30 out of 50) were conducted between the years 2021 and 2022. More than half of the studies reported multiple VE readings, with a total of 92 records extracted. Approximately three-quarters (37/50) of the studies had sample sizes ranging from 10,000 to 10 million participants. The term “sample size” in Table 3 refers to the total number of participants or data included in each study. This encompasses the initial cohort size for cohort studies, and the combined number of cases and controls in case-control studies but does not indicate the number of subjects per vaccination regimen. Regarding the predominant coronavirus VOCs during the study period, the majority of the studies reported VE during the Delta-predominant period (n = 22), followed by Omicron (n = 14) and Alpha (n = 13). Only nine studies made no mention of the circulating coronavirus variants at the time the study was conducted.
Summary of characteristics of the included studies. The study period, vaccination protocol, sample size, and prevalent variant of concern of the included studies were reported.
| Study characteristics | Number of studies |
|---|---|
| Study period | |
| 2020–2021 | 16 |
| 2021 only | 19 |
| 2021–2022 | 11 |
| 2022 only | 4 |
| Vaccination protocol | |
| 2-dose BNT162b2 | 44 |
| 2-dose mRNA-1273 | 26 |
| 3-dose BNT162b2 | 15 |
| 3-dose mRNA-1273 | 7 |
| Sample size | |
| Under 10 thousand | 10 |
| 10 thousand—1 million | 19 |
| 1–10 million | 18 |
| Over 10 million | 3 |
| Prevalent variant of concern (VOC) | |
| Alpha | 13 |
| Gamma | 2 |
| Delta | 22 |
| Omicron | 14 |
| Did not specified | 9 |
| Study characteristics | Number of studies |
|---|---|
| Study period | |
| 2020–2021 | 16 |
| 2021 only | 19 |
| 2021–2022 | 11 |
| 2022 only | 4 |
| Vaccination protocol | |
| 2-dose BNT162b2 | 44 |
| 2-dose mRNA-1273 | 26 |
| 3-dose BNT162b2 | 15 |
| 3-dose mRNA-1273 | 7 |
| Sample size | |
| Under 10 thousand | 10 |
| 10 thousand—1 million | 19 |
| 1–10 million | 18 |
| Over 10 million | 3 |
| Prevalent variant of concern (VOC) | |
| Alpha | 13 |
| Gamma | 2 |
| Delta | 22 |
| Omicron | 14 |
| Did not specified | 9 |
Summary of characteristics of the included studies. The study period, vaccination protocol, sample size, and prevalent variant of concern of the included studies were reported.
| Study characteristics | Number of studies |
|---|---|
| Study period | |
| 2020–2021 | 16 |
| 2021 only | 19 |
| 2021–2022 | 11 |
| 2022 only | 4 |
| Vaccination protocol | |
| 2-dose BNT162b2 | 44 |
| 2-dose mRNA-1273 | 26 |
| 3-dose BNT162b2 | 15 |
| 3-dose mRNA-1273 | 7 |
| Sample size | |
| Under 10 thousand | 10 |
| 10 thousand—1 million | 19 |
| 1–10 million | 18 |
| Over 10 million | 3 |
| Prevalent variant of concern (VOC) | |
| Alpha | 13 |
| Gamma | 2 |
| Delta | 22 |
| Omicron | 14 |
| Did not specified | 9 |
| Study characteristics | Number of studies |
|---|---|
| Study period | |
| 2020–2021 | 16 |
| 2021 only | 19 |
| 2021–2022 | 11 |
| 2022 only | 4 |
| Vaccination protocol | |
| 2-dose BNT162b2 | 44 |
| 2-dose mRNA-1273 | 26 |
| 3-dose BNT162b2 | 15 |
| 3-dose mRNA-1273 | 7 |
| Sample size | |
| Under 10 thousand | 10 |
| 10 thousand—1 million | 19 |
| 1–10 million | 18 |
| Over 10 million | 3 |
| Prevalent variant of concern (VOC) | |
| Alpha | 13 |
| Gamma | 2 |
| Delta | 22 |
| Omicron | 14 |
| Did not specified | 9 |
Geographical distribution of the included studies
The largest proportion of studies included in the current review reported VE data against hospitalization from the United States, representing 17 out of the 50 included studies (Fig. 2). Six studies reported data from Israel, with five studies were included from the United Kingdom. The geographical distribution of the included studies is presented in Fig. 2.
Map of the world showing the geological distribution of the included studies. The colour scale depicts the number of included studies from each region. No data were collected from the countries within the grey-shaded regions.
Europe and South America also contributed to the body of research analysed, though to a lesser extent, indicating a more dispersed research presence across these continents. Notably, no studies were included from Africa, highlighting an important data gap for this geographic region. In Asia, only data from two studies from Hong Kong and one study from Taiwan contributed to this study, indicating limited representation of VE data from regions outside of Europe, North America, and South America.
High vaccine effectiveness for the BNT162b2 and mRNA-1273 vaccines after a primary series and one booster dose
A total of 92 records of VE were extracted from the 50 studies. Among these, 44 records included data for VE after a primary series (2-dose) protocol using the BNT162b2 mRNA COVID-19 vaccine, and 26 records described VE after the primary series protocol using the mRNA-1273 COVID-19 vaccine. In comparison, there were fewer records of VE after one booster dose (3-dose) for either of the mRNA vaccines (BNT162b2, 15 records; mRNA-1273, 7 records).
The VE records were extracted from the included studies and presented in a forest plot grouped by each of the four vaccination protocols (vaccine type, no. doses; Fig. 3). Despite some heterogeneity between the studies, irrespective of the vaccine or vaccination protocol used, the reported VE data for most studies was positioned to the right-hand side of (above) the 80% VE reference line (Fig. 3). This suggested that both the BNT162b2 and mRNA-1273 COVID-19 vaccines provided strong VE against COVID-19 hospitalization.
Forest plot showing reported VE% in each of the eligible studies. The plot shows the VE% against hospitalization for the BNT162b2 and mRNA-1273 COVID-19 vaccines after both a primary series (2-dose) and booster (3-dose) protocol. The vertical dotted reference line denotes 80% VE against hospitalization. The plot is colour-coded to represent the geographical region of each study.
Pooled vaccine effectiveness estimates for each vaccination protocol
To better understand the overall VE of the four vaccination protocols, a meta-analysis was undertaken to calculate the pooled estimates of VE. By using pooled estimates, this enabled VE extracted from studies conducted throughout the pandemic to be considered. A random effects model was used to enable the heterogeneity observed in Fig. 3 to be taken into account. This meta-analysis suggested the pooled VE estimates against hospitalization across the four vaccination protocols ranged from 84.34% to 86.71% (Table 4), indicating high overall VE for both the BNT162b2 and mRNA-1273 COVID-19 vaccines.
Pooled estimates of VE% of the four regimens. The pooled estimates were provided along with the I2 statistics, which measure the degree of heterogeneity among the study results.
| Vaccination protocol | Pooled VE estimates (%) | I2 | P value |
|---|---|---|---|
| 2-dose BNT162b2 | 84.34 (80.27–88.40) | 99.81% | <.0001 |
| 2-dose mRNA-1273 | 86.02 (80.76–91.32) | 99.44% | <.0001 |
| 3 dose BNT162b2 | 86.71 (80.56–92.86) | 99.35% | <.0001 |
| 3 dose mRNA-1273 | 85.17 (76.78–93.55) | 99.07% | <.0001 |
| Vaccination protocol | Pooled VE estimates (%) | I2 | P value |
|---|---|---|---|
| 2-dose BNT162b2 | 84.34 (80.27–88.40) | 99.81% | <.0001 |
| 2-dose mRNA-1273 | 86.02 (80.76–91.32) | 99.44% | <.0001 |
| 3 dose BNT162b2 | 86.71 (80.56–92.86) | 99.35% | <.0001 |
| 3 dose mRNA-1273 | 85.17 (76.78–93.55) | 99.07% | <.0001 |
Pooled estimates of VE% of the four regimens. The pooled estimates were provided along with the I2 statistics, which measure the degree of heterogeneity among the study results.
| Vaccination protocol | Pooled VE estimates (%) | I2 | P value |
|---|---|---|---|
| 2-dose BNT162b2 | 84.34 (80.27–88.40) | 99.81% | <.0001 |
| 2-dose mRNA-1273 | 86.02 (80.76–91.32) | 99.44% | <.0001 |
| 3 dose BNT162b2 | 86.71 (80.56–92.86) | 99.35% | <.0001 |
| 3 dose mRNA-1273 | 85.17 (76.78–93.55) | 99.07% | <.0001 |
| Vaccination protocol | Pooled VE estimates (%) | I2 | P value |
|---|---|---|---|
| 2-dose BNT162b2 | 84.34 (80.27–88.40) | 99.81% | <.0001 |
| 2-dose mRNA-1273 | 86.02 (80.76–91.32) | 99.44% | <.0001 |
| 3 dose BNT162b2 | 86.71 (80.56–92.86) | 99.35% | <.0001 |
| 3 dose mRNA-1273 | 85.17 (76.78–93.55) | 99.07% | <.0001 |
In addition, there were no significant differences in the pooled VE estimates between the primary series and one booster dose for both BNT162b2 and mRNA-1273 COVID-19 vaccines, as their confidence intervals largely overlap (Table 4). This implies that for each of the COVID-19 mRNA vaccines analysed, one homologous booster dose was sufficient to restore the VE of the primary series.
No significant differences in VE were also identified between the primary series and one booster dose protocol for either vaccine. This suggests that the BNT162b2 and mRNA-1273 COVID-19 vaccines were similarly effective throughout the pandemic, with little if any indication of either mRNA vaccine being more effective than the other.
However, the meta-analysis also revealed substantial heterogeneity among the studies, with I2 values exceeding 99% and high significance for each of the VE estimates (Table 4). This high degree of heterogeneity suggests that much of the variability among these studies was due to intrinsic differences in study designs rather than sampling error. Thus, the pooled VE estimates should be interpreted with caution.
Reduced vaccine effectiveness in presence of the Omicron coronavirus variant
To address the significant heterogeneity in VE, a subgroup analysis of VE was undertaken based on the predominant coronavirus VOC that was prevalent at the time the studies were undertaken. As described in Table 3, most included studies reported VE under the prevalence of the Alpha, Delta, and Omicron coronavirus variants. The VE for primary and one booster dose series for both BNT162b2 and mRNA-1273 COVID-19 vaccines grouped by the VOC are shown in Figs 4 and 5. During the prevalence of Alpha and Delta coronavirus variants, the VE for each vaccine typically exceeded 80%, indicating robust protection against hospitalization after infection with these variants (Fig. 4). However, a notable reduction in VE was observed with the emergence of the Omicron coronavirus variant. Our analysis showed that VE was typically lower during periods when Omicron was prevalent, with most estimates falling below the 80% reference line regardless of the vaccination protocol (Fig. 5).
Forest plots showing VE of both BNT162b2 and mRNA-1273 COVID-19, for both the primary (2 dose) and one booster (3 dose) series against the Alpha and Delta coronavirus VOC. (A–C), BNT162b2. (D–F), mRNA-1273. The vertical dotted line represents 80% VE for reference.
Forest plots showing VE of both BNT162b2 and mRNA-1273 COVID-19, for both the primary (2 dose) and one booster (3 dose) series against the Omicron coronavirus VOC. (A and B), BNT162b2. (C and D), mRNA-1273. The vertical dotted line represents 80% VE for reference.
Discussion
Our systematic review summarized the results of 50 distinct studies published between 2021 and 2024, which reported VE against COVID-19 hospitalization for patients immunized with a primary (2-dose) or one booster (3-dose) protocol using either the BNT162b2 (Pfizer/BioNTech) or mRNA-1273 (Moderna) mRNA COVID-19 vaccines. Across the four vaccination protocols, the majority of the included studies reported VE above 80%, suggesting that these mRNA vaccines were associated with robust protection against COVID-19 hospitalization. Meta-analysis of the included studies reported consistent results, with the pooled estimates of VE of the four vaccination protocols ranging between 84.34% and 86.71%. However, subgroup analysis of VE by the prevalence of VOC revealed that the predominance of the Omicron variant in the circulating strains was associated with a decrease in VE.
Our study aimed to assess the real-world VE of the BNT162b2 and mRNA-1273 mRNA COVID-19 vaccines. We selected COVID-19 hospitalization as the sole VE endpoint for multiple reasons. Firstly, it reflects vaccine protectiveness against severe disease directly, compared to infection where mild disease is also recorded [21]. Secondly, the selection of hospitalization allows for higher consistency across different studies as definitions for COVID-19 hospitalization are less likely to be influenced by variations in testing methods, providing a more objective and reliable basis for comparing VE across independent studies [22]. In addition, hospitalization has a direct impact on healthcare systems, influencing hospital capacity, resource allocation, and overall healthcare burden [23]. We included studies that reported VE against COVID-19 hospitalization in percentages, whereas those that reported VE in effect sizes such as risk ratios or odds ratios were excluded. Including VE in percentages only, ensured direct comparability between the included studies and simplified data interpretation.
Comparison of our pooled VE estimates against hospitalization for a primary series of the BNT162b2 and mRNA-1273 mRNA COVID-19 vaccines with those for other vaccine formats shows they have a similar or higher level of VE. For example, the VE against hospitalization for a primary series of the BBIBP-CorV (Sinopharm) inactivated COVID-19 vaccine was estimated to be 79.6% (77.7–81.3) [24], the inactivated CoronaVac inactivated vaccine has a VE of 87.5% (86.7–88.2%) [25] and the Ad26.COV2.S viral vector COVID-19 vaccine has a VE of 72% (64–77%) [26].
Our subgroup analysis revealed an observable drop in VE during the period when Omicron was the predominant VOC. This association between the presence of the Omicron coronavirus variant and a reduction in VE is likely due to the presence of many inherent mutations within the Omicron strain that confer immune escape. Relative to the ancestral strain of SARS-CoV-2, the Omicron strain carries 60 mutations, 32 of which are located within the spike protein region [27]. As the mRNA vaccines were designed to deliver genetic information to produce coronavirus spike protein-specific antibodies, mutations in this region are likely to reduce the binding specificity of the vaccine-induced antibodies, leading to decreased neutralization and lower VE. A study by Cele et al. [28] provided supporting evidence by testing the ability of plasma from 19 individuals who had completed the primary series of BNT162b2 to neutralize the Omicron variant. They found a 22-fold reduction in vaccine-elicited neutralization against the Omicron variant. This decline in VE has also been observed in VE studies targeting symptomatic infection. Florentino et al. [29] reported 14.0% and 10.2% lower VE of a 2-dose course of the BNT162b2 vaccine against symptomatic infection in adolescents in Brazil and Scotland, respectively, when compared to the periods when the Delta and Omicron variants were predominant.
To address the reduced VE associated with the emergence of the Omicron variant, bivalent mRNA vaccines containing the mRNA encoding the spike proteins of both the ancestral Wuhan-1 strain and the Omicron strain were developed [30]. Tartof et al. [31] reported a 50% (23%–68%) higher VE against critical illness and a 39% (28%–49%) higher VE against hospital admission for the BNT162b2 BA.4/5 bivalent vaccine compared to 2 doses of BNT162b2 or mRNA-1273. The subsequent emergence of sub-lineages of the Omicron variant that also exhibit lower susceptibility to the monovalent mRNA COVID-19 vaccines underscores the need to regularly update the COVID-19 vaccines to maintain protectiveness against emerging and circulating coronavirus VOCs [32].
This review provides indirect evidence for the effect of a booster dose in rescuing the waning VE of a primary immunization series against hospitalization. Although the included studies reported different lengths of follow-up periods, the pooled VE estimates for the 2-dose and 3-dose protocols for each of the mRNA COVID-19 vaccines analysed overlapped. This suggested that the VE conferred by these two immunization protocols were similar. This alignment in VE between the 2-dose and 3-dose protocols implies that administering a subsequent booster dose can effectively restore VE. Indeed, among the included studies that provided data for both a primary series and one booster dose of BNT162b2 or mRNA-1273, the VE of the booster dose was always higher than primary series [33, 34]. Bar-on et al. [35] has similarly reported that compared with the primary series of immunization with BNT162b2 a booster dose resulted in a lower rate of COVID-19 infection by a factor of 11.3.
Notably, our analysis identified outliers in VE, specifically the studies by Adams et al. [36] and Chatzilena et al. [37], which reported significantly lower VE for the 3-dose regimens of BNT162b2 and mRNA-1273, respectively, compared to other studies. These discrepancies may be attributed to the majority of the sample populations in these studies receiving the ancestral strain vaccine rather than the updated bivalent vaccine. In contrast, the studies reporting higher VE for the 3-dose regimens may include a mix of both ancestral and bivalent vaccines, potentially leading to enhanced effectiveness against variants. Although this review provides insights into the VE of two mRNA vaccines against COVID-19 hospitalization, we acknowledge some potential limitations. While the current study did not impose any geographical constraints in the selection of studies, we nonetheless identified an uneven distribution of the studies, with the majority reporting data from the Americas and Europe, with limited representation from Asia, and no studies from Africa. Some COVID-19 mRNA VE studies have been published in Singapore [38] and Japan [39]. However, these were excluded during the screening process since they did not report VE against COVID-19 hospitalization or study effects of the bivalent mRNA vaccines. Whether the VE estimates reported in this review are applicable to distinct populations across the world is uncertain, as many factors unique to these regions may have an important influence. For example, helminth parasite infections are prevalent in people in low- and middle-income countries [40] and the immune responses to them can affect the efficacy of some vaccines [41, 42]. It is, therefore, essential that studies are also conducted in low- and middle-income countries to enable the impact of factors such as co-infection with other pathogens have on VE [43].
In addition, this review did not account for the impact of follow-up duration on VE, as the reported follow-up periods varied significantly among the included studies. Some studies reported follow-up durations of over 7 days, with others reporting durations extending beyond 210 days. This variability in reporting indicates a lack of standardization in how follow-up durations are defined and documented across different studies. Such inconsistencies can complicate the interpretation of VE results, as the timing of follow-up can influence the observed effectiveness of vaccines. Short follow-up periods may not capture the full extent of waning immunity, while longer follow-up durations provide insights into the durability of the immune response over time. Furthermore, very few studies specified follow-up durations for each mRNA vaccine, which limits our ability to assess how differences in vaccine type and administration timing may affect VE. This gap in data underscores the need for future research to adopt standardized protocols for reporting follow-up durations, enabling more accurate comparisons and a clearer understanding of the long-term effectiveness of mRNA vaccines against COVID-19.
Stringent transportation and storage requirements impact the distribution of the COVID-19 mRNA vaccines since these require cold-chain storage, in which the Pfizer vaccine has to be kept at −60°C [44]. This greatly limits mRNA vaccine access in regions that lack access to reliable storage and transportation facilities. As such, among the different brands of vaccine supplied to Africa under the COVID-19 Vaccines Global Access (COVAX) initiative, the Oxford/AstraZeneca vaccine was the most widely distributed since it can be safely stored at 2–8°C and does not require the same logistical requirements as the mRNA vaccines [45]. These conditions may contribute to the lack of COVID-19 mRNA VE studies from African and South-East Asian regions.
Conclusion
In conclusion, this systematic review of 50 independent studies suggests that the Pfizer/BioNTech BNT162b2 and Moderna mRNA-1273 mRNA vaccines confer robust protection against COVID-19 hospitalization after a primary or booster immunization series. The study also highlighted how different VOCs such as the Omicron variant could impact on VE, underscoring the importance of ongoing research to ensure vaccine strategies remain effective against evolving variants. Our study also identified the need for expanding data collection to include underrepresented populations.
Abbreviations
- COVID-19
coronavirus disease 2019
- PRISMA
preferred reporting items for systematic reviews and meta-analyses
- SARS-CoV-2
severe acute respiratory syndrome coronavirus 2
- VE
vaccine efficacy
- VOC
variant of concern
- WHO
World Health Organization
Acknowledgements
The Editor-in-Chief and handling editor, Tim Elliott, would like to thank the following reviewers, Ester Gea-Mallorquí and an anonymous reviewer, for their contribution to the publication of this article. The authors would also like to thank the reviewers for their contributions during the peer-review of this article.
Author contributions
Bill K.-F. Wong (Data curation, Formal analysis, Investigation, Writing—original draft, Writing—review & editing), and Neil A. Mabbot (Conceptualization, Funding acquisition, Project administration, Supervision, Writing—original draft, Writing—review & editing)
Conflict of interest
None declared.
Funding
The study was supported by Institute Strategic Programme grant funding from the UK Biotechnology and Biological Sciences Research Council (grant number BBS/E/RL/230002B).
Data availability
The data underlying this article are available in the article and in its online supplementary material.
Clinical trial registration
This review was not registered and the protocol was not prepared.
