How effective are digital interventions in increasing flu vaccination among pregnant women? A systematic review and meta-analysis

ABSTRACT Background Flu can have serious consequences for pregnant woman and unborn babies. Vaccination provides safe and effective protection, yet uptake among pregnant women is below national targets. Digital interventions are effective at increasing adherence to health interventions. Aims This review aimed to establish whether digital interventions are effective at increasing rates of flu vaccination among pregnant women, and to determine the overall effect size. Method Systematic searches identified digital intervention trials, aiming to increase rate of flu vaccination among pregnant women. Random-effects meta-analysis provided a combined effect size and examined which mode of digital interventions had the largest effects on flu vaccination. Results Ten studies were included in the review. The majority of digital interventions were more effective at increasing rates of flu vaccination (7–81.3% uptake) than usual care or non-digital interventions (7.3–47.1% uptake). When meta-analysed, digital interventions had a small, non-significant effect (odds ratio [OR] = 1.29, 95% confidence interval [CI]: 0.71, 2.31), P = 0.40. Text messages (OR = 1.25, 95% CI: 0.58, 2.67), P = 0.57 appeared less effective than other digital interventions (OR = 1.58, 95% CI: 1.02, 2.46), P = 0.04. Conclusions Overall, there is a lack of high-quality studies reporting the effectiveness of digital interventions at increasing flu vaccination during pregnancy. Future interventions may benefit from using video or social media to communicate messages for maximum success in targeting an increase in rates of flu vaccination in pregnancy.


Introduction
Pregnant women and their unborn babies are at increased risk of complications from flu, due to physiological and immunological changes occurring during pregnancy. Pregnant women are approximately four times more likely to be hospitalized with flu than non-pregnant women, and risk of death from flu is higher among pregnant women. [1][2][3] Furthermore, there is an increased risk of premature birth, stillbirth and low birthweight for unborn babies, resulting from maternal flu. 1 The flu vaccination has been shown to be safe and effective, 4-6 yet uptake among pregnant women in England is annually below the 75% national target, with only 45.2% of pregnant women receiving the vaccination in 2018/19. 7 Pregnant women with lower education, living at or below the poverty line, non-Hispanic or black ethnicity [8][9][10] and smokers 11 are less likely to have the flu vaccination during pregnancy. Pregnant women have been shown to underestimate their susceptibility to and the seriousness of flu while pregnant, which may influence their vaccination decisions. 12,13 Internet use has increased rapidly over recent years, with 96% of households in Great Britain having internet access in 2020, compared to 56% in 2006. 14 The popularity of internet use is expected to further increase, with nearly 54 million people estimated to use internet-enabled smartphones in the UK by 2022, 15 making digital health interventions accessible to many people.
Digital interventions have proven effectiveness in increasing health behaviours such as smoking cessation, 16 physical activity, 17 physical activity in cardiovascular disease 18 and asthma self-management. 19 If digital interventions are an effective approach to increase flu vaccination among pregnant women, it suggests an accessible mechanism for primary care services to improve health of pregnant women and unborn babies, in turn reducing associated healthcare costs resulting from maternal flu. To date, effectiveness of digital interventions for increasing flu vaccination rates among pregnant women is yet to be determined.
This review aimed to establish whether digital interventions are effective at increasing flu vaccination rates among pregnant women, and to determine the size of the effect.
Review objectives: (i) To examine the effectiveness of digital interventions for increasing rate of flu vaccination among pregnant women. (ii) To compare the effectiveness of different types of digital interventions for increasing rate of flu vaccination among pregnant women.

Method
This study was conducted in line with a pre-defined protocol 20 and is reported in line with PRISMA guidelines. 21

Eligibility criteria
Studies testing effectiveness of digital interventions for increasing flu vaccination rate among pregnant women were eligible for inclusion. For the purposes of this review, the term 'digital intervention' is defined as an intervention that attempts to change pregnant women's vaccination behaviour, delivered via digital or mobile devices directly to participants. This includes text messages (including text, video or audiobased messages), internet-delivered interventions (including websites, mobile applications (apps) or social media sites) and other digital strategies. 22 Any comparison group was acceptable, including usual care, wait-list comparators, historical control groups (without digital intervention), digital interventions unrelated to flu vaccination or non-digital interventions. Only original research studies were eligible for inclusion, with systematic reviews, protocols, commentaries and conference abstracts excluded.
Studies were required to be randomized or non-randomized controlled trials, quasi-randomized controlled trials or other quantitative designs reporting rate of flu vaccination (e.g. before and after trials) following implementation of a digital intervention, which also contained a comparator. Case series and case reports were excluded. No date or country restrictions were included, but studies were required to be published in English. Full inclusion and exclusion criteria can be found in Table 1.

Outcome measures
The primary outcome was rate of flu vaccination among pregnant women after receiving targeted digital interventions, compared to a comparator group. This could be either selfreported vaccination status or status obtained from electronic patient records. The secondary outcome was the size of the effect of digital interventions (using odds ratio [OR]).

Information sources
The following electronic bibliography databases were searched: MEDLINE, Embase, Web of Science, Scopus, Cochrane database, PsycINFO and Cochrane Central Register of Controlled Trials (CENTRAL). In-progress trials were searched for on the clinical trials register. Searches were conducted in April 2020.

Search strategy
Search terms included all possible terms relating to 'vaccination', 'influenza', 'pregnancy' and variations of 'digital interventions' to include interventions containing significant influence from text messages, video, Internet, or mobile phone apps. 22,23 Reference sections of studies meeting inclusion criteria and papers citing studies meeting inclusion criteria were screened to identify other eligible studies. The full search strategy can be found in Supplemental 1.

Data management and screening process
Results from database searches were combined and duplicates removed. Endnote X9 and Covidence software were used to organize data. Titles and abstracts of all search results were first screened to assess eligibility for inclusion in the review. Any studies that appeared to be eligible were subjected to the next stage of screening. Full text of studies were then obtained and screened against the predefined inclusion criteria. Screening was conducted by two researchers independently, and discrepancies were discussed until consensus was reached. This resulted in a full and final set of studies for inclusion in the review.
Data were then extracted from included studies. This step was conducted by two researchers independently, using a predefined extraction form. The following information was Any discrepancies in data extraction were discussed until a consensus was reached. Eligibility for inclusion in the metaanalysis was also determined for each study.

Quality assessment
For randomized controlled trials, risk of bias was assessed using the Cochrane Risk of Bias Tool. 24 Each study was rated as low, medium or high risk of bias on each domain. For nonrandomized controlled trials, risk of bias was assessed using the Cochrane Risk of Bias in Non-Randomised Studies of Interventions. 25 Each study was rated as low, moderate, serious or critical risk of bias, or categorized as no information to make a judgement for each domain.
Overall ratings of risk of bias were calculated by totalling numbers of domains for each paper rated as low risk, some concerns (or moderate risk) or high risk. Randomized controlled trials were deemed to be as follows: 'low risk', if all domains were rated low risk; 'some concerns', if at least one domain was rated some concerns but no domains rated high risk and 'high risk', if at least one domain was rated high risk or multiple domains were rated as some concerns. 24 Nonrandomized controlled trials were deemed to be as follows: 'low risk', if all domains were rated low risk; 'moderate risk', if all domains are rated low or moderate risks; 'serious risk', if there was at least one domain rated serious risk of bias but no ratings of critical risk and 'critical risk', if at least one domain was rated critical risk. 25 Quality was assessed by two authors independently. Any discrepancies were discussed until consensus was reached.

Data synthesis
Key information extracted from included studies was synthesized, including descriptive information about type and content of intervention and control conditions for each study. Rates of flu vaccination were extracted and synthesized to determine the effectiveness of digital interventions at increasing flu vaccination among pregnant women. Summaries of risk of bias of included studies were reported.

Data analysis
Heterogeneity was assessed using meta-analysis software.
RevMan software version 5.4.1 was used to calculate OR for each digital intervention, using a random-effects model. Where studies included more than one digital intervention, the most digitally intensive intervention was included in the meta-analysis.
A sensitivity analysis was conducted to examine whether risk of bias of included studies affected the overall effect size of digital interventions on flu vaccination rate. A moderator analysis was conducted to examine differences in effects between types of digital interventions and to determine which mode of delivery is more effective in increasing the rate of flu vaccination among pregnant women.

Main characteristics of included studies
A total of 479 results (after duplicates were removed) were subjected to title and abstract screening. Of these, 33 full-text papers were obtained and screened against the eligibility criteria. Ten studies met all inclusion criteria and were included in the review. The number and reasons for exclusion can be seen in the PRISMA flowchart found in Supplemental 2. 'Reasons for exclusions include the study having the wrong population, i.e. not pregnant women, or a study design not meeting the inclusion criteria for the review.
The majority of included studies were set in hospital or clinic settings, [26][27][28][30][31][32][33][34] while one involved current enrolees of the Text4baby Service (a free national mobile health service in the USA), 29 and one involved a national internet survey. 35 Five studies employed objective measures of vaccination uptake, verified by patient records or monitoring uptake on the day of the study, 28,[30][31][32][33] and four used selfreported measures. 27,29,34,35 One study used three methods of obtaining rate of vaccination (self-report, reviewing of electronic records and verification via a local vaccination register). 26 Full characteristics of studies can be found in Table 2.

Digital interventions
The most common mode of digital intervention used in the included studies was text messages. 29,30,[32][33][34][35] Other methods of intervention included videos, 27,28 website or social media 31 and an iBook-based app. 26 Interventions in three studies were delivered face-to-face in study conditions, 27,28 and in examination rooms while waiting to be seen by a physician. 26 The remaining seven studies involved interventions being delivered remotely, consisting of text messages or links being sent from the study team to participants at home. [29][30][31][32][33][34][35] Comparators used in included studies included no intervention or usual care, 26,29,[31][32][33][34][35] non-digital interventions 27 and non-vaccination-related interventions. 28,30 Details of intervention and comparator conditions are available in Table 2.

Quality assessment
Five studies were given an overall rating of high risk of bias. 26,27,29,31,34 Three were given an overall rating of some concerns or moderate risk of bias, 32,33,35 and two studies were given an overall rating of low risk of bias. 28,30 The domain with the most occurrences of potential bias was 'Risk of bias arising from the randomisation process'. 24 Individual domain ratings and overall risk of bias ratings for each study can be seen in Table 3.

Effectiveness of digital interventions
The rate of vaccination (reported as percentage of pregnant women within the sample receiving the flu vaccination) in included studies ranged between 7% (reported by the iBook condition in Frew et al .'s study) 27 and 81.3% 35 in intervention conditions and between 7.3 26 and 47.1% 35 in control conditions. Full flu vaccination uptake rates can be found in Table 4. Overall rates of vaccination suggest that the majority of intervention conditions were more effective than control conditions [26][27][28]31,[33][34][35] at increasing flu vaccination uptake among pregnant women. This shows that digital interventions are often a more effective approach than non-digital or no intervention.

Additional analyses
A sensitivity analysis examined whether the effect of digital interventions was increased when studies rated as high risk of bias were removed from the meta-analysis. Removing the five high risk of bias studies resulted in a larger effect of digital interventions on the rate of flu vaccination. However, this effect was still non-significant (OR = 1.47, 95% CI: 0.65, 3.34), P = 0.35, I 2 = 95%. See Supplemental 4 for sensitivity analysis forest plot. Self-report Table 3 Risk of bias ratings A moderator analysis was conducted to examine whether there was a difference in effectiveness depending on the type of digital intervention used. Six studies used text messagebased interventions, and these had a smaller, non-significant effect on flu vaccination uptake (OR = 1.25, 95% CI: 0.58, 2.67), P = 0.57, I 2 = 97%, than all other modes (video, social media and iBook) of digital interventions (OR = 1.58, 95% CI: 1.02, 2.46), P = 0.04, I 2 = 2%. See Supplementa l 5 for moderator analysis forest plots.

Heterogeneity
A very high level of heterogeneity (I 2 > 75% 36 ) was present in the effect of digital interventions for flu vaccination (I 2 = 96%). As heterogeneity was above 75%, a randomeffects model was used.

Publication bias
Examination of the funnel plot (see Supplemental 6) suggests the presence of asymmetry across studies, possibly indicating some publication bias, and some missing unpublished studies with negative effects. The analysis of funnel plots however can be subjective and difficult to interpret. 37

Main findings of the study
The majority of individual digital interventions were more effective at increasing flu vaccination among pregnant women than usual care or non-digital interventions. However, when the studies were pooled and weighted in the meta-analysis, there was a small non-significant effect. There was considerable heterogeneity in the results (particularly in those using text message interventions), and these findings are likely to be attributable to the small sample sizes found in more than half of the included studies and differences in interventions. This highlights the need for further, well-conducted studies with larger sample sizes.
A moderator analysis examining the effectiveness of different types of digital interventions showed that text messages were less effective than other modes of intervention, although there was significant heterogeneity present. This is particularly interesting as more than half of the digital interventions in this study used text messages to convey the digital message; the use of text messages is generally a popular approach for  Table 2 Yudin 2017 Overall influenza vaccination rate in the whole sample was 29%, with no significant difference in rates between intervention group: 40/129 = 31% and control group: 41/152 = 27%.
Not reported digital health interventions, yet in this review they were less effective than videos, social media and iBooks. This finding differs to findings of previous meta-analyses, which found that text message-based interventions were more effective at changing health behaviours than other modes of digital interventions. 16,38 What is already known on this topic The susceptibility of pregnant women to flu and the effectiveness of digital interventions for some health behaviours are well known, yet little is known about the effectiveness of digital interventions in increasing flu vaccination uptake among this population. Differences between findings of the current review and previous reviews in the effectiveness of text message interventions in changing behaviour may be explained by the type of behaviour being examined. Previous research has suggested that health-related interventions conveying risk are more effective when engaging and visual information is used. 39,40 This may explain why visual interventions (such as video, social media and iBooks) for flu vaccination are more effective than text messages, which are limited to the presentation of facts and statistics. Visual and engaging interventions are not easily communicated using text messages alone.

What this study adds
This study increases knowledge around appropriate approaches to increase flu vaccination among this population, potentially influencing clinical practice and service improvement for this under-researched area. This can ultimately have a positive impact on the rate of flu vaccination uptake, improving health and reducing mortality of pregnant women and unborn babies.
The majority of studies included in this review showed that digital interventions were more effective at increasing the rate of flu vaccination, when compared to non-digital interventions or usual care. This suggests that campaigns and interventions aiming to increase flu vaccination for this population may benefit from including digital components: specifically, videos, social media and iBooks, rather than text messages. This has practical implications for recommended content of new interventions in development, both for routine vaccinations during pregnancy and for the development of interventions for new diseases, such as for the new COVID-19 vaccination.
Although the majority of included studies showed that digital interventions were more effective at increasing flu vaccination among pregnant women, when the studies were pooled and weighted for the meta-analysis, there was no effect compared to non-digital interventions or usual care. This contradicts previous research showing digital interventions improve health-related behaviours. [16][17][18][19] This may be due in part to the small number of included studies, highlighting the need for more research examining the effectiveness of digital interventions for flu vaccination in pregnancy.

Limitations of this study
Many of the studies included in this review have small sample sizes, which may contribute to the non-significant effect of digital interventions in increasing flu vaccination in this study. There are likely to be differences between studies that provide interventions in study or clinical settings compared to those delivered remotely. The presence of experimental settings or researchers may impact uptake of vaccination.
Comparators or level of usual care also varied significantly between studies. Some involved no information or general health information, whereas others provided information about flu, which may have more impact on intention to vaccinate. The majority of studies were conducted in the USA. There is the potential that this country has different levels of usual care or better access to Internet than other countries. More research is needed in other countries to see if digital interventions are effective there (e.g. in remote populations where usual care may be considerably more limited). Additional research in the UK would be beneficial to support NHS maternity care for mothers and babies, as this is currently lacking.

Conclusion
While digital interventions had proven efficacy for some health behaviours, effectiveness over other interventions for increasing flu vaccination in pregnancy had not previously been established. This review showed that digital interventions taken individually were generally more effective at increasing flu vaccinations among pregnant women, but the overall pooled and weighted effect was small and nonsignificant. Text messages appeared to be less effective than other digital methods at increasing flu vaccination among this population, providing valuable insight for future digital interventions.

Supplementary data
Supplementary data are available at the Journal of Public Health online.