The impact of pausing the Oxford-AstraZeneca COVID-19 vaccine on uptake in Europe: a difference-in-differences analysis

Abstract Background Several countries paused their rollouts of the Oxford-AstraZeneca coronavirus disease-19 (COVID-19) vaccine in mid-March 2021 due to concerns about vaccine-induced thrombosis and thrombocytopenia. Many warned that this risked damaging public confidence during a critical period of pandemic response. This study investigated whether the pause in the use of the Oxford-AstraZeneca vaccine had an impact on subsequent vaccine uptake in European countries. Methods We used a difference-in-differences approach capitalizing on the fact that some countries halted their rollouts whilst others did not. A longitudinal panel was constructed for European Economic Area countries spanning 15 weeks in early 2021. Media reports were used to identify countries that paused the Oxford-AstraZeneca vaccine and the timing of this. Data on vaccine uptake were available through the European Centre for Disease Control and Prevention COVID-19 Vaccine Tracker. Difference-in-differences linear regression models controlled for key confounders that could influence vaccine uptake, and country and week fixed effects. Further models and robustness checks were performed. Results The panel included 28 countries, with 19 in the intervention group and 9 in the control group. Pausing the Oxford-AstraZeneca vaccine was associated with a 0.52% decrease in uptake for the first dose of a COVID-19 vaccine and a 1.49% decrease in the uptake for both doses, comparing countries that paused to those that did not. These estimates are not statistically significant (P = 0.86 and 0.39, respectively). For the Oxford-AstraZeneca vaccine only, the pause was associated with a 0.56% increase in uptake for the first dose and a 0.07% decrease in uptake for both doses. These estimates are also not statistically significant (P = 0.56 and 0.51, respectively). All our findings are robust to sensitivity analyses. Conclusions As new COVID-19 vaccines emerge, regulators should be cautious to deviate from usual protocols if further investigation on clinical or epidemiological grounds is warranted.


Introduction
A number of European countries paused their national rollouts of the Oxford-AstraZeneca coronavirus disease-19 (COVID- 19) vaccine in mid-March 2021, [1][2][3] due to concerns about a rare but serious syndrome known as vaccine-induced thrombosis and thrombocytopenia (VITT). According to the European Medicines Agency (EMA), 30 cases of thromboembolic events (predominantly venous) had been reported by 10 March 2021, among the approximately 5 million recipients of the Oxford-AstraZeneca COVID-19 vaccine in the European Economic Area (EEA). The EMA subsequently stated, 'The number of thromboembolic events in vaccinated people is no higher than the number seen in the general population'. 4 The World Health Organization (WHO) also recommended continuing vaccination whilst further data were awaited and reviewed. 5 Given the uncertainty around a causal relationship, many criticized regulatory intervention to halt vaccination as unnecessarily damaging to public confidence, during a critical period of epidemic response. 6 A YouGov poll covering around 8000 people in seven European countries from 12 to 18 March found that in France, Germany, Spain and Italy, people were more likely to see the AstraZeneca vaccine as unsafe than as safe, 7 unlike in previous surveys. 8 After a short pause of approximately 1 week, most EEA nations resumed their use of the vaccine, 9,10 although the Scandinavian countries did not. Sweden, Finland and Iceland moved towards limiting the use of the Oxford-AstraZeneca vaccine for their elderly population, while Denmark and Norway prolonged their suspension of the AstraZeneca vaccine and shifted to providing alternative vaccines. 11 This study seeks to understand whether the pause in the use of the Oxford-AstraZeneca vaccine had a significant impact on uptake for both the Oxford-AstraZeneca vaccine specifically, or on COVID-19 vaccine uptake more broadly. We were able to use quasi-experimental difference-in-differences methods as the vaccination pause occurred in some European countries but not others, at approximately the same time and for a similar duration across countries. With another generation of COVID-19 vaccines on the horizon, this study will help to inform future regulatory processes in the context of novel technologies used in pandemic response.

Study design
We used a difference-in-differences approach to assess the impact of pausing the Oxford-AstraZeneca vaccine on uptake. This is a quasiexperimental technique that can be used to assess the causal impacts of policies by comparing a group that received the intervention with a similar group that did not. It was suitable here because some European countries temporarily paused their vaccine rollouts in early 2021, whilst others did not. A pause was defined as a temporary halt in the rollout of the Oxford-AstraZeneca vaccine within a country, affecting the population eligible for vaccines. All included countries were part of the EEA and therefore to some extent comparable, and our statistical analyses accounted for important differences across groups of countries where present.
We explored this causal link due to concerns about the impact policy changes could have on public confidence in novel vaccines (produced at record speed) and in turn, on vaccine uptake. The intervention itself occurred at approximately the same time across countries, with a direct and immediate impact on both public information and national vaccine programme rollouts, further supporting our approach. Outcome data were available from a single and reliable source [the European Centre for Disease Prevention and Control (ECDC)] meaning our dataset was largely complete, of a high quality and comparable across countries. Through investigating the impact on uptake for all vaccines as well as the Oxford-AstraZeneca vaccine specifically, our study was well designed to understand the various mechanisms through which the pause could have affected vaccine uptake.
We assumed that decisions by national authorities to continue or pause vaccination had a greater impact on public confidence within a country than general media coverage.

Data sources
A longitudinal panel was constructed for 28 EEA countries with vaccine uptake data at three-week intervals from Weeks 3-18 of the COVID-19 vaccination rollout in 2021. This did not include the United Kingdom (UK), Denmark or Norway. The UK is no longer in the EEA and had different plans and systems for vaccine rollout compared to the rest of the EEA. Denmark and Norway did not restart their use of the Oxford-AstraZeneca vaccine after the pause, unlike other countries. Media reports were used to identify countries that paused the Oxford-AstraZeneca vaccine and the timing of this. 1-3 Data on overall COVID-19 vaccine uptake and only Oxford-AstraZeneca uptake, for first and both doses, were obtained directly from the European Centre for Disease Control and Prevention COVID-19 Vaccine Tracker and Open Data. 12,13 Uptake was provided as a cumulative percentage of the adult population (18þ) for all vaccines, and due to data limitations, calculated as a cumulative percentage of the entire population for the Oxford-AstraZeneca vaccine.
Data on numbers of vaccines supplied, for both the Oxford-AstraZeneca vaccine alone and for all vaccines, were also obtained from the same dataset and adjusted for the population size of each country, prior to analysis. Data on other potential confounders, including demographic, economic and health system indicators, were from 2019 and collected from the World Bank Open Database. 14 Two indicators (universal health coverage service index and health expenditure per capita) were from 2017 and 2018, respectively. Given the role of public confidence and trust in vaccine uptake, the state of democracy in included countries was ascertained through the Economist Intelligence Unit's 2020 Democracy Index. 15 Data on weekly COVID-19 cases and cumulative deaths at baseline (Week 3 of 2021) were obtained from the open-access Our World in Data. 16

Exposures and outcomes
Exposure to the intervention was defined as those EEA countries that paused the rollout of the Oxford-AstraZeneca vaccine. This action was taken over the week of 8 March. Those that did not pause the vaccine at all were treated as the control group. A dummy variable was generated to indicate whether the vaccine rollout was paused in a given country at a given week of the vaccination programme. Our primary outcome was cumulative uptake for first and both doses for all vaccines, defined as the percentage of the adult (18þ) population that had received (i) one dose and (ii) both doses of a COVID-19 vaccine, from Week 3 of 2021 to Week 18. The secondary outcome was cumulative uptake for first and both doses of the Oxford-AstraZeneca vaccine from Weeks 6-18. Due to data availability, this was defined as the percentage of the whole population that had received (i) one dose and (ii) both doses of the Oxford-AstraZeneca vaccine.

Statistical analysis
All analyses were conducted using STATA 14. 17 We first compared both groups of countries on a range of non-normally distributed baseline characteristics, using the Wilcoxon Rank Sum test to identify statistically significant differences in mean values between groups. P-values of <0.05 were considered statistically significant. Although we adjusted for time and week-fixed effects, we also performed a random effects model to adjust for any of these factors that were found to be statistically significantly different between the groups. We then fit multivariate linear regression models in which the key variable of interest was a difference-in-differences estimator, interpreted as the average effect of pausing the Oxford-AstraZeneca vaccine on overall vaccine uptake, in the countries that paused it. Our main model that investigated uptake of all vaccines as the primary outcome controlled for weekly supplies of all vaccines per population, and the model investigating the impact on uptake of the Oxford-AstraZeneca vaccine only controlled for weekly supplies per population of this specific vaccine. This was deemed to be a critical and observed time-varying confounder, which also varied across countries despite joint procurement plans, therefore being potentially correlated with differences in vaccine uptake between the countries that paused the vaccine rollout and those that did not. We included both country fixed effects to control for underlying time-invariant differences between countries, and week fixed effects to account for changes over time common to all countries.
We ran several further checks to assess whether our results were robust and to probe the assumptions that support the methodology, including the parallel trends assumption, which states that in the absence of the pause in vaccine rollout, included countries would have had the same trends in vaccine uptake as control countries. We first constructed graphs to examine the trends in average vaccine uptake before the pause. We then ran a statistical test for parallel trends using the 'estat ptrends' command in STATA 17. 18 We ran a second model also adjusting for country and week fixed effects, to exclude countries that imposed later age restrictions on the use of the Oxford AstraZeneca vaccine from early April 19-23 : Belgium, Estonia, Finland, France, Germany Iceland, Ireland, Italy, the Netherlands, Portugal, Sweden and Spain. We also ran a third multivariable model using the random-effects estimator and adding timeinvariant covariates. In all models, similar results were obtained (Supplementary File).
Data were collected at the country-level and all countries were part of the EEA trading bloc, part of a joint vaccine procurement and distribution strategy. 24 The difference-in-differences panel estimator could therefore be subject to serial autocorrelation, within-cluster correlation and cross-sectional dependence, potentially leading to standard errors that are not robust. We used clustered standard errors in all models to account for this.

Results
A total of 30 EEA countries were initially selected but two were excluded, leaving 28 for the primary analysis. The intervention group consisted of 19 countries and the remaining 9 countries that did not pause the vaccine rollout formed the control group. Linear trends are parallel in both groups, as assessed visually on a graph as well as statistically, with no evidence against the null hypothesis (P ¼ 0.56 for overall uptake of first dose, P ¼ 0.67 for overall uptake of both doses; P ¼ 0.53 for Oxford-AstraZeneca first dose and P ¼ 0.79 for Oxford-AstraZeneca both doses). The general trend for vaccination uptake across all countries from Weeks 3-18 of 2021 was upward, although the gradient of the increase was steeper when looking at all vaccines, compared to Oxford-AstraZeneca alone, and for first doses compared to both doses (figure 1). Table 1 shows the demographic, socioeconomic and health system characteristics, as well as data on COVID-19 epidemics at baseline, presented as means (95% confidence interval), for each group of countries. On average, countries that paused the vaccine had higher GDP per capita and stronger democratic performance, compared to those that did not. By controlling for these time-invariant covariates we adjusted for these differences between countries in the differencein-differences analysis. Population size, age, gender, life expectancy at birth and COVID-19 epidemic characteristics at baseline (measured as a 7-day average COVID-19 case rate and cumulative death rate per million in the week up to 21 January) were not significantly different between both groups of countries. Figures 2 and 3 show line graphs for first and both dose vaccine uptake over time comparing countries that paused with those that did not. For overall COVID-19 vaccination, the trend in both groups is similar for the first dose, with overlapping confidence intervals at all time points and no deviation after the pause. For both doses, there appears to be some deviation after the pause, with countries that paused having marginally lower rates of uptake, but confidence intervals suggested that these differences were nonsignificant. For the Oxford-AstraZeneca vaccine, uptake was similar in both groups over time with minimal deviation after the pause. Although the population denominators for vaccine uptake were not directly comparable, the Oxford-AstraZeneca vaccine contributed little to overall vaccine uptake. This was particularly pronounced when looking at both doses.
Our difference-in-differences estimates found that pausing the rollout of the Oxford-AstraZeneca vaccine was associated with a subsequent 0.52% decrease in the overall population uptake for the first dose of a COVID-19 vaccine and a 1.49% decrease in the uptake for both doses, comparing countries that paused to those that did not. These estimates are not statistically significant (P ¼ 0.86 and 0.39, respectively). When looking at the uptake of the Oxford-AstraZeneca vaccine only, the pause was associated with a 0.56% increase in uptake for the first dose and a 0.07% decrease in uptake for both doses. These estimates are also not statistically significant (P ¼ 0.56 and 0.51, respectively). These findings are robust to sensitivity analyses including the exclusion of some countries and the use of the random-effects estimator (Supplementary File) suggesting that the pause had no significant impact on subsequent vaccine uptake.

Key findings
There was no significant difference in subsequent vaccine uptake in countries that paused the Oxford-AstraZeneca vaccine in mid-March compared with those that did not. This was true for both one and two doses of the vaccine, and when looking at overall vaccine uptake (including all vaccines) and the Oxford-AstraZeneca vaccine alone. This suggests that over time, public confidence in vaccination was not significantly more damaged in those countries that paused the vaccine compared to those that did not. It appears there has been a shift away from using the Oxford-AstraZeneca vaccine in the EEA due to relatively low rates of uptake as a proportion of all vaccine uptake and a shallower increasing gradient in uptake over time compared to all vaccines, but this trend was similar across all included countries. This analysis suggests that as new COVID-19 vaccines emerge, regulators should be cautious to deviate from usual pharmacovigilance protocols on the sole basis of potential impacts on public confidence, if further investigation on clinical or epidemiological grounds is warranted.

Explaining findings
Many experts were concerned about the impact of pausing the Oxford-AstraZeneca vaccine on public confidence and consequent demand. 6 Regulators that paused the rollout highlighted that VITT was serious but rare. Media and scientific communications emphasized the fact that investigating safety signals in this manner is an established phamacovigilance process done out of an abundance of caution, reiterating the well-established safety and effectiveness of vaccine. 25 For instance, after pausing the vaccine Spain's health minister Carolina Dias said 'We take this decision today in the interest of caution', referring to 'infrequent cases which are very few but very significant and have prompted Spain to join the other countries that have opted for this precautionary suspension'. 26 Around the same time, the UK Medicines and Healthcare products Regulatory Agency (MHRA) reported that VITT was seen in <1 per million vaccinated people in the UK, which had administered more Oxford-AstraZeneca doses than other European countries, and that a causal association had not been established. 27 Most countries restarted using that vaccine approximately 1 week later, serving to further reassure the public in countries that initially paused the vaccine. It is likely that a short pause, variation in regulatory practice across countries, and reassuring public communications all helped to mitigate any potential impact on vaccine uptake in countries that did pause their rollouts. Vaccine uptake may also have been similar across both groups of countries because of later scientific and policy developments on the Oxford-AstraZeneca vaccine. Although the majority of countries that had initially paused the vaccine in mid-March restarted it shortly after, 12 out of 28 included countries restricted its use in younger people from the second week of April. [20][21][22]28 This was in response to concern about the balance of risks and benefits in the young. In these countries, the later impact of the pause on vaccine uptake (at Weeks 15 and 18 in our study) may have been blurred by the impact of these restrictions. We did however control for vaccine supply over time across countries, which can be considered a proxy for the impact of such additional regulations since countries would be unlikely to procure vaccines they were not planning to use. A sensitivity analysis (Supplementary File) excluding the countries that imposed additional later restrictions on the use of the Oxford-AstraZeneca vaccine showed that the results did not change. The relatively timid increase over time in uptake for both doses of the Oxford-AstraZeneca vaccine is likely a product of timing (with the second dose administered at 4-12 weeks after the first 29 ) as well as these additional restrictions. Importantly, the same trend was seen in both countries that paused the vaccine and those that did not, suggesting that countries that paused the vaccine were not at a later disadvantage compared to others.

Limitations
Our study had several limitations. First, although we were able to allocate countries to one of two groups, it is possible that there was contamination from the intervention to the control group. The European medicines regulatory system is based on a network of around 50 regulatory authorities from across EEA countries. 30 Citizens may place more trust in their own national regulators compared with others. But populations in countries that did not pause the Oxford-AstraZeneca vaccine may have also been affected by knowing that similar countries (operating under a common European regulatory framework) decided to pause their rollouts of the vaccine, despite it being restarted soon after. While this could theoretically minimize any observed differences in vaccine uptake between countries, this is an important corollary of the intervention in a complex global regulatory environment during a pandemic, which should not be artificially adjusted for but considered part of the intervention itself. Second, although we were able to include data spanning 15 weeks, the Oxford-AstraZeneca vaccine was approved for a dosing interval of 4-12 weeks, meaning that we may have underestimated the impact of the pause on second doses in some cases. But the fact that uptake for both doses was low throughout the study period, and across both country groups, suggests that there was a true shift away from using the Oxford-AstraZeneca vaccine. Third, the difference-in-differences methodology relies on the assumption that there are no factors beyond the pausing of the Oxford-AstraZeneca vaccine that differentially affect vaccine uptake and thereby confound the observed effect. We accounted for these through adjusting for fixed country and week effects, adding time-variant fluctuations in supplies of vaccines across countries. We also controlled for other potential timeinvariant confounders through a random-effects model. Nevertheless, the omission of other important, unobserved, timevarying characteristics could bias our findings if they affected the intervention and control group differentially. For instance, although we adjusted for vaccine supplies we were not able to identify whether these were delayed, or less than countries expected. If this was the case countries may have re-designed already running vaccination programmes, consequently reducing uptake due to dose unavailability. Finally, although difference-in-differences is an often used quasi-experimental method, 31 inferences about causality can be less definitive than those obtained through a well-designed randomized controlled trial. But it would not be possible to evaluate the impact of pausing the rollout of a vaccine during a pandemic, making quasi-experimental methods such as those used here the gold standard for policy evaluation.

Implications for research and policy
Although we found small non-significant decreases in overall vaccine uptake in countries that paused the Oxford-AstraZeneca vaccine compared to those that did not, we found a small non-significant increase in uptake for the first dose of the Oxford-AstraZeneca vaccine itself. Further research is required to understand whether populations feel more reassured after they know regulators have closely scrutinized the data on a particular vaccine and whether this paradoxically puts it at an advantage compared to others in terms of public perception and attitudes.
Despite arguments made against pausing the Oxford-AstraZeneca vaccine rollout, we found no evidence that it significantly impacted vaccine uptake. The findings from this study cannot however be generalized to future scenarios where vaccine risks, the stage of the pandemic, the mechanism and duration of pausing rollout, public perception of the regulators that pause and those that do not, and public communications could all be different. Future qualitative research will be needed to understand how these factors affect the impact of a nationwide pause in the rollout of a novel vaccine, and which policies and plans can be put in place to pause vaccines in as safe a way as possible, allowing further investigation whilst preserving public confidence. This analysis also highlights the general lack of alignment between European decision-making bodies, despite the fact that the public health impact of a regionally uncoordinated approach to stopping, restarting, and later restricting vaccine use, was largely unknown. Policymakers and regulatory authorities in Europe must collaborate through regional organizations like the EMA and global organizations like the WHO, to ensure that in the context of an international health threat, national policies and processes on vaccines are harmonized as far as possible. In the longterm, this will be necessary to promote public trust in COVID-19 vaccines and in the institutional processes related to their selection, procurement, distribution and monitoring.

Conclusions
In this quasi-experimental study, we employed a difference-indifferences design to estimate the causal effect of pausing the Oxford-AstraZeneca vaccine on later vaccine uptake. We found that there was no significant difference in overall COVID-19 vaccine uptake in countries that paused the Oxford-AstraZeneca vaccine in mid-March compared with those that did not. There was a shift away from using the Oxford-AstraZeneca vaccine itself, but uptake was not significantly different across countries. Further research is required to understand the precise mechanism underlying our findings, and the factors associated with how the public within and across countries perceive a pause in the national rollout of a novel vaccine. As new COVID-19 vaccines emerge, regulators should be cautious to deviate from usual pharmacovigilance protocols if further investigation on clinical or epidemiological grounds is warranted.