Cost-effectiveness of health systems strengthening interventions in improving maternal and child health in low- and middle-income countries: a systematic review

Abstract

Health systems strengthening (HSS) interventions are increasingly being implemented to improve maternal and child health (MCH) services in low- and middle-income countries (LMICs). This study reviews global literature on cost-effectiveness of HSS interventions in improving MCH. A systematic review was conducted. Keywords, based on World Health Organization framework on health systems and prior studies, were applied to search in bibliographic databases and on the web. Articles that estimated cost-effectiveness of HSS interventions in LMICs were included in the analysis. Each of the 24 selected studies from 15 countries was assessed in terms of quality and biases using Cochrane’s criteria. Review Manager and an Excel template were used to extract data and synthesize findings. HSS interventions concentrated on the components of service delivery, health financing, human resources and quality improvement. Within each component, there existed diverse strategies to strengthen health systems. Among the 24 studies, 15 were rated as high quality, 5 as medium and 4 as low quality. A majority of studies reported cost per disability-adjusted life year (DALY) averted or cost per quality-adjusted life year (QALY) gained; other studies reported cost per life saved or life year gained. However, studies used mixed perspectives of analyses. Compared with gross domestic product per capita, interventions in studies reporting cost per DALY averted or QALY gained were all cost-effective, including performance-based financing, health insurance and quality improvement. This review shows the diversity of HSS interventions in improving MCH, and their potential cost-effectiveness. However, the different perspectives employed in the studies, costing components included in the analyses, and heterogeneous measures of effectiveness and outputs, made it challenging to compare cost-effectiveness across all studies, calling for more and standardized cost-effectiveness studies. For policy making, it is critical to examine long-term cost-effectiveness of programs and cost-effectiveness of synergistic demand- and supply-side interventions.

Cost-effectiveness, health system strengthening, maternal and child services, systematic review

Key Messages

• Various health system strategies have been taken to address maternal and child health in low- and middle-income countries. However, the evidence on cost-effectiveness of the strategies remains limited, and more studies on value for money of health system strengthening interventions are needed.

• Available economic evaluation studies on health system interventions vary substantially in terms of perspectives of analyses, length of impact under evaluation and measures of impact and effectiveness.

• In spite of challenges to compare results across studies due to heterogeneous methodologies, findings from relatively homogenous studies show that interventions, such as performance-based financing and quality improvement prove to be cost-effective in improving maternal and child health.

Introduction

Maternal and child health (MCH) remains a major health concern in low- and middle-income countries (LMICs). During the Millennium Development Goals (MDGs) era, strong international commitment was directed to MCH; MCH services expanded substantially to poor and vulnerable populations and equity of reproductive and maternal health services continuously improved (Alkenbrack et al. 2015). Globally, the under-5 mortality rate (U5MR) was reduced from 90.6 in 1990 to 42.5 per 1000 live births in 2000 (53% reduction) (You et al. 2015), and the maternal mortality rate (MMR) from 282 in 1990 to 196 per 100 000 live births in 2015 (30% reduction) (GBD 2015 Maternal Mortality Collaborators 2016). These declines are impressive. However, only a few countries achieved all of the MDGs health-related goals. For example, only 10 countries achieved MDG 5 on reduction of MMR (GBD 2015 Maternal Mortality Collaborators 2016). To maintain the momentum of reducing maternal and child mortality rates, international communities developed the Sustainable Development Goals (SDGs) aimed to sustain health gains in combating MCH-related illnesses.

The reduction of MMR and U5MR significantly benefited from increased financial commitments from donors though official development aid (ODA). ODA has tripled from $54.8 billion in 2001 to $167 billion in 2013 (Valentine et al. 2015). ODA funding for health during the same period increased more than five times, rising from $4.4 billion to $22.8 billion (Valentine et al. 2015). A substantial amount of donor support was allocated to combating human immunodeficiency virus/acquired immune deficiency syndrome, malaria and tuberculosis, as well as for providing MCH services. In spite of the increase in resources, there is a great need to continue investing in health programs to sustain health gains. In the last decades, the international community has increasingly realized that treatment expansion efforts have been slowed by insufficient health infrastructure and inefficient health systems. To accelerate the pace of delivering effective and available prevention and treatment to populations in need, donors are paying increasing attention to health systems strengthening (HSS), which is defined as ‘initiating activities in the six internationally accepted core health system functions—human resources for health; health finance; health governance; health information; medical products, vaccines and technologies; and service delivery’ (U.S. Agency for International Development (USAID) 2015) and calling for interventions for building a more efficient and effective health system (American Public Health Association 2008). Taking projects funded by USAID as an example, there have been multiple global flagship projects on HSS awarded to implementation partners, including a $209 million health finance and governance project (2012–17) (Health Finance & Governance Project) for strengthening health finance and governance systems, a health policy plus (HP+) project of $185 million for generating evidence for policy making (Palladium 2015), and DELIVERY Project for enhancing supply chains. Similarly, The Global Fund has gradually switched from funding disease-specific interventions to HSS. Thirty-seven percent ($362 million) of the Global Fund Round 8 funding was allocated for HSS (Warren et al. 2013). HSS becomes a critical element to catalyse efforts to expand service coverage.

Along with improving MCH services, many HSS programs, with the support from donors and governments, have been designed. For example, performance-based financing (PBF) programs provide financial incentives to health facilities for delivering MCH services. With support from the Health Results Innovation Trust Fund (HRITF), 35 PBF programs have been implemented in 29 countries since 2007, with a total of commitment of $385.6 million as of September 2016 (RBFHEALTH 2017). PBF aims to improve MCH through improving the management and financial situation of health facilities. Voucher programs incentivize pregnant women to seek essential maternal care, and have been implemented widely (Hurst et al. 2015). The initial impact evaluation shows a positive impact of these programs in improving the use of MCH services (Basinga et al. 2011; Hurst et al. 2015). However, there is little evidence concerning cost-effectiveness of these programs (Witter and Somanathan 2012; Turcotte-Tremblay et al. 2016).

Many such programs have been supported by donors initially, with the expectation that they will be transferred to governments as programs mature. This raises an important question as to whether the governments can afford to take on and sustain the programs. Each government has to operate programs within its budget, and when multiple programs are available, governments must wisely choose among them. Additionally, with substantial investment in HSS, both donors and governments call for value for money to maximize the impact of available resources (Department for International Development 2011). A study in the USA showed that using cost-effectiveness information to allocate resources had the potential to improve a population’s health status (Chambers et al. 2013). Thus, it is critical to include costs in assessing the range of available programs and in evaluating programs’ impact, to understand better economy, efficiency and effectiveness of programs (Department for International Development 2011).

Recognizing the critical role of health systems in improving MCH, this study aims to provide a systematic review of cost-effectiveness of HSS interventions in addressing MCH. The term cost-effectiveness in this review is used in a more generic way, and it includes any studies linking costs to the impact of programs, encapsulating cost-effectiveness analysis (CEA), cost-utility analysis and cost-benefit analysis. Through this review, we hope to provide a more complete picture of HSS interventions in order for countries to make informed decisions and to identify gaps in existing economic evaluations of HSS interventions.

Methods

This review concentrated on the cost-effectiveness of HSS interventions. World Health Organization (WHO) developed a framework with six building blocks (Figure 1), providing a common understanding of what a health system is and what constitutes HSS (World Health Organization 2007). Building on this framework, HSS interventions, in this review, are defined as activities aiming to improve the six building blocks of a health system (leadership/governance, health financing, human resources, medical products and technologies, health information and service delivery). This definition is consistent with what Warren et al. (2013) used to analyse Global Fund funding for HSS.

Figure 1.

Six building blocks of a health system (source: WHO, 2007).

Service delivery, among the six building blocks, is slightly different from the other five blocks. Service delivery is regarded as an immediate goal of a health system, while the remaining building blocks serve as inputs for effective, safe, quality service provision. Given that service delivery is often measured by access, coverage and quality of care, and that one of WHO’s priorities in responding to health system challenges on service delivery is to develop effective service delivery models (World Health Organization 2007), HSS interventions on service delivery are limited to activities for improving quality of care, enhancing engagement of the demand-side and developing innovative service delivery models (e.g. community engagement, public-private partnerships and social marketing). Activities directly targeted to patients or potential patients for preventive and curative purposes are regarded as clinical interventions (e.g. screening, testing and treatment), rather than HSS interventions, and thus excluded from the review.

Desk search strategy

To identify articles for review, we used the combination of terms in the following three areas to conduct the desk research: (1) cost-effectiveness, (2) MCH and (3) the six building blocks. For cost-effectiveness, we used the terms of ‘cost-effectiveness’, or ‘cost-benefit’, or ‘economic evaluation’. To capture MCH, the search terms were ‘reproductive’, or ‘maternal’, or ‘neonatal’, or ‘child’ or ‘motherhood program’ or ‘prenatal care’. As to key words for the six building blocks, given the wide variation of health system interventions for MCH, we first reviewed an article that synthesized key innovative interventions addressing MCH (Lunze et al. 2015), and developed key words for searching articles. Table 1 show the terms that we used for each of the six building blocks to conduct the search. In general, the key words started with terms or synonyms of the building blocks, and proceeded with more specific interventions within the block.

Table 1.

Key search terms on the six building blocks of the health system

	Service delivery	Health workforce	Information	Medical products	Financing	Leadership/governance
Search words	Quality improvement	Human resources	Information	Supply chain	Financing	Governance
	Community	Human workforce		Technology	Cash transfer	Leadership
	Public-private partnership	Training			Incentives	Health policy
		Education			Voucher	Regulation
					Health insurance
					User fees
					Performance-based financing
					Results-based financing
					Pay-for-performance

We conducted searches in four major electronic bibliographic databases on public health and economics: PubMed, EconLit, Academic Search Premier and Web of Science, on January 12, 2017 and updated the search on February 10, 2017. We also conducted a search for grey literature through Popline database. All searches were conducted in English. With an additional three articles written by authors, the initial search identified 4197 non-duplicate publications that were eligible for title and abstract screening (Figure 2). To include more grey literature for review, we also searched through Google Scholar using the same key words as those applied to bibliographic databases. The research team reviewed the first 100 records from the search and compared them with those obtained from electronic databases. We further checked their eligibility and found no additional articles that could be included in the review from Google Scholar.

Figure 2.

Flow diagram for study identification.

Exclusion criteria

All the search records were first uploaded in Endnote X8 and independently screened by two reviewers. We used Endnote to eliminate duplicated records (1236 records). The remaining records (4197 records) were first reviewed through titles and abstracts to assess their relevance: (1) studies focused on MCH, (2) interventions concerned with HSS, (3) studies that reported on cost-effectiveness measures and (4) studies conducted in LMIC. Articles that met these inclusion criteria and those that possibly met the criteria were included in the full-text review. As a result, 38 articles were selected for full-text review. Articles were excluded if they met one of the following exclusion criteria:

• Studies not related to MCH;

• Studies not conducted in LMICs;

• Studies where interventions did not fall in any of the six building blocks of a health system;

• Studies where interventions were direct preventive and curative services;

• Qualitative studies;

• Studies synthesizing prior studies;

• Studies evaluating programs that had not been implemented;

• Studies where the final effectiveness in the economic evaluation was not measured in terms of health outcome [e.g. life years saved, disability-adjusted life years (DALYs) averted, quality-adjusted life years (QALYs) gained, lives saved or deaths averted];

• Studies reporting average cost-effectiveness ratio without any comparison group, rather than incremental cost-effectiveness ratio; and

• Studies published prior to 1990.

The full text review was conducted independently by WZ and HA for all 38 articles, and 18 articles were excluded based on exclusion criteria. We further reviewed references from the remaining 20 articles, and included 4 more articles. In the end, 24 articles were selected in the final review for synthesizing findings. Table 2 show the reason for exclusion for the 18 articles.

Table 2.

Reasons for exclusion of 18 out of 38 articles

Number of articles	Reasons for exclusion
10	Cost-effectiveness measures not in terms of cost per outcome measures
2	No empirical data, CEA purely based on assumptions and secondary data
2	Reporting average cost-effectiveness ratio
1	Duplication
1	Not related to MCH
1	Not conducted in LMIC
1	Synthesis of prior studies

CEA, cost-effectiveness analysis; MCH, maternal and child health; LMIC, low- and mid-income countries.

Study selection and data extraction

An Excel template was developed to extract data from the 24 articles. The collected information included: (1) characteristics of studies: publication year, interventions under examination, research design for impact evaluation, outcome measures of impact evaluation, along with other relevant study characteristics and (2) components of economic evaluation: perspective of cost analyses, cost components, length of assessment, year in which costs were assessed and expressed, cost-effectiveness measures, value of cost-effectiveness, whether sensitivity analyses were conducted, and the like. For each article, corresponding information was extracted. Data extraction was primarily carried out by one researcher, while another researcher reviewed and checked collected data. If information (e.g. year in which costs were assessed and expressed) was not available in the main text, two researchers discussed and made best guesses.

Quality assessment or risk of bias in individual studies

The final 24 studies were then imported into Review Manager software 5.3 (RevMan) (Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration) and subsequently evaluated for quality of evidence or risk of bias using the Cochrane Effective Practice and Organisation of Care (EPOC) risk of bias criteria and study quality guide (Higgins and Green 2011; Cochrane Consumer & Communication Review Group 2013).

The quality of evidence was assessed through examining the risk of bias of each study. There were seven major biases assessed using the checklist based on the Cochrane criteria, and we assigned scores to each study on the following seven categories: (1) random sequence generation (selection bias), (2) allocation concealment (selection bias), (3) blinding of participants and personnel (performance bias), (4) blinding of outcome assessment (detection bias), (5) incomplete outcome data (attrition bias), (6) selective reporting (reporting bias) and (7) other risks of bias, including publication bias.

For each category, a study received a rating of low, high or unclear risk. Numerical rating codes were assigned to each of the three ratings: low risk was assigned a numerical code of 1, high risk a numerical code of 0 and unclear risk a numerical code of 0.5. A composite quality score for each study was calculated by averaging the seven numerical rating codes. We then rated each study based on the overall quality score: low (<60%), medium (an average 60–80%) or high (≥80%).

Data synthesis

Consistent with the Excel template, the data analyses focused on two dimensions. The first dimension concerned characteristics of overall study and impact evaluation, which examined HSS strategies, building blocks to which interventions/strategies belong, study design of impact evaluation, primary measures for impact evaluation, and impact of interventions/strategies if this was reported. It should be noted that sometimes, there were overlaps when categorizing a particular intervention/strategy into building blocks. For example, comprehensive quality improvement programs (block of service delivery) sometimes incorporated training of health providers (block of human workforce). When encountering such a circumstance, core research members (WZ and HA) further examined the overall intervention and categorized it based on the key components and purposes of the intervention. For example, if a training program was a single intervention for an existing and conventional delivery approach, we categorized it into the block of human workforce. If the training was part of a quality improvement package, or the training of human resources was for implementing a new service delivery model, then it was categorized into the block of service delivery.

The second dimension of the analysis was for characteristics of economic evaluation/cost-effectiveness for each article. The major characteristics included perspectives of cost-effectiveness, cost components, measures and value of cost-effectiveness and use of sensitivity analysis, as well as length of cost-effectiveness assessment [long-term (≥4 years) or short-term assessment (<4 years)]. The cost-effectiveness was reported as US Dollars (USD) per effectiveness measure in the year when costs of programs were expressed and evaluated in the article. To standardize cost-effectiveness measures, we also listed gross domestic product (GDP) per capita in the same year for comparison. For cost-effectiveness of strategies that reported as cost per DALY averted or cost per QALY gained, the WHO’s Commission on Macroeconomics and Health considers interventions/strategies to be highly cost effective if they are less than one times GDP per capita, and being cost-effective if they are less than three times GDP per capita (Hutubessy et al. 2003; World Health Organization 2017).

Results

Overview of studies

Of the 24 selected articles, 21 were obtained from peer review articles and 3 written by the authors; 23 were published since the year 2000 and 1 published a few years prior. The 24 studies were conducted in 15 countries, with four in Zambia, three each in Uganda and India, two each in Bangladesh and Malawi and one in each of remaining countries (Argentina, Gambia, Kenya, Myanmar, Nepal, Niger, Nigeria, Papua and New Guinea, Ukraine and Zimbabwe). Geographically, there were 14 studies conducted in Africa, 7 in Asia, 1 in Oceania, 1 in Eastern Europe and 1 in Central America. Table 3 provides detailed information of the selected studies.

Table 3.

Interventions, study design and impact evaluation of included studies

Last name of the first author	Publication year	Building Block	Country	Region	Interventions	Study design for impact evaluation	Key measures of the impact evaluation	Impact of intervention
Shepard	2017	Financing	Zimbabwe	Africa	Provide financial incentives to health providers for maternal and child health services	Pre-post controlled design	Quality indictors; institutional delivery, post-partum care	RBF increased the share of institutional deliveries by 13.4%, and post-partum tetanus vaccinations by 20.0%.
Zeng	2017	Financing	Zambia	Africa	Provide financial incentives to health providers for maternal and child health services	cRCT	Institutional delivery, family planning, and quality of care	Compared to IFG, RBF improved utilization of Hib vaccination (15.0%) and family planning (21.8%), and resulted in quality of care changes ranging from −0.8% to + 4.9%. Compared with PCG, RBF improved quality of care, ranging from 2.3% to 9.7%, and significantly increased utilization of postnatal care (7.8%), institutional delivery (12.2%), Hib (19.1%) and family planning (19.5%)
Wang	2016	Financing	Zambia	Africa	Provide ‘Mama kit’ incentives to mothers conditional on delivering baby in facilities	cRCT	Use of institutional delivery	The odds of delivering at a facility were increased by 63% (29–106%), or an increase of 9.9% points
Bishai	2015	Service delivery	Myanmar	Asia	Add ORS-Z as an additional product line in an existing social franchise program	Pre-post controlled design	Use of ORS-Z	7.6% increase in zinc and ORS use, which would translate to 2.85 (SD = 0.29) incremental deaths averted in a total community population of 1 million
Colbourn	2015	Service delivery	Malawi	Africa	Mobilize community around maternal and neonate health through women’s group (CM) Improve quality of health facilities through training staff, implement change packages focused on obstetric and newborn care, conduct death reviews, leadership training and protocol-based clinical trainings (FI)	cRCT	Neonatal mortality rate; maternal mortality rate	The neonatal mortality rate was 22% lower in CM+FI than control clusters (OR = 0.78, 95% CI 0.60–1.01), and the perinatal mortality rate was 16% lower in CI clusters (OR = 0.84, 95% CI 0.72–0.97). No intervention effects on maternal mortality
Gomez	2015	Financing	Nigeria	Africa	Implement a health insurance program, which provided access to comprehensive health care	Cohort study	Use of antenatal care and institutional delivery	Access to antenatal care increased from o.65 to 0.85; institutional delivery from 0.50 to 0.675, with an estimated 47 deaths averted per 10 000 deliveries
Saya	2015	Financing	Uganda	Africa	Implement community health insurance schemes covering immunizations and curative services, as well as transport for pregnant women to and from contracted facilities	Health insurance: Assumption based on prior study	Use of facility delivery	1% insurance increase in the health insurance enrollment rate of the entire population would raise the proportion of facility deliveries by 0.9% from its initial value
Gerler	2014	Financing	Argentina	Central America	Provide financial incentives to health providers for maternal and child health services	Pre-post controlled design	Neonatal mortality rate	Beneficiaries’ probability of low birth-weight is estimated to be reduced by 19%. Beneficiaries have a 74% lower chance of in-hospital neonatal mortality in larger facilities
Alfonso	2013	Financing	Uganda	Africa	Provide eligible pregnant women with a health vouchers for a subsidized price of US$1.40 covering four ANC visits, delivery care, referral and treatment of eventual complications, and a postnatal care visit	Pre-post controlled design	Use of institutional delivery	The demand for births at HFs enrolled in the voucher scheme increased by 52.3 percentage points. Out of this value, conservative estimates indicate that at least 9.4 percentage points are new health facility users. This 9.4% bump in institutional delivery coverage implies 20 deaths averted
Broughton	2013	Service delivery	Niger	Africa	Implement quality improvement program through clinical and improvement capacity-building sessions for participants; coaching visits to participating sites; learning sessions; office personnel and administrative support; and other resources used to coordinate these activities	Pre-post design	Postpartum haemorrhage; adherence to newborn care standards; maternal mortality ratio	Probability of postpartum haemorrhage decreased from 0.0202 to 0.00216; probability of adherence to newborn care standards increased from 0.185 to 0.975; maternal mortality ratio decreased from 7.11 to 0.98 per 10 000 births
Fottrell	2013	Service delivery	Bangladesh	Asia	Convene women’s groups for participatory learning and action cycle in which they prioritize issues that affect maternal and neonatal health, and design and implement strategies to address these issues	cRCT	Neonatal mortality rate	The neonatal mortality rate was significantly lower in the intervention arm (21.3 neonatal deaths per 1000 live births vs 30.1 per 1000 in control areas), a reduction in neonatal mortality of 38% [risk ratio, 0.62 (95% CI, 0.43–0.89)] when adjusted for socioeconomic factors.
LeFevre	2013	Service delivery	Bangladesh	Asia	Recruit and train community health workers (CHWs) to conduct home visits during pregnancy and post-natal period Conduct other health system strengthening activities: train facility-level providers in maternal and neonatal health care, distribute drugs & supplies, and establish a system for tracking neonatal care	cRCT	Neonatal mortality rate	Neonatal mortality was reduced in the home-care arm by 34% (adjusted relative risk 0.66; 95% CI 0.47–0.93) during the last 6 months vs that in the comparison arm. No mortality reduction was noted in the community-care arm (0.95; 95% CI 0.69–1.31)
Lewycka	2013	Service delivery	Malawi	Africa	Recruit and train facilitators to convene women’s groups. Women’s groups encouraged to identify and adopt local strategies to improve maternal and child health Volunteer peer counsellors made home visits during pregnancy and post-birth to support breastfeeding and infant care	Factorial cluster randomized trial	Neonatal mortality rate	After adjustment for parity, socioeconomic quintile and baseline measures, effects were larger for NMR (0.85, 95% CI 0.59–1.22) and MMR (0.48, 95% CI 0.26–0.91). Because of the interaction between the two interventions, a stratified analysis was done. For women’s groups, in adjusted analyses, MMR fell by 74% (0.26, 95% CI 0.10–0.70), and NMR by 41% (0.59, 95% CI 0.40–0.86) in areas with no peer counsellors, but there was no effect in areas with counsellors (1.09, 95% CI 0.40–2.98 and 1.38, 95% CI 0.75–2.54). Factorial analysis for the peer counselling intervention for years 1–3 showed a fall in IMR of 18% (0.82, 0.67–1.00) and an improvement in EBF rates (2.42, 1.48–3.96)
Barasa	2012	Service delivery	Kenya	Africa	Implement quality improvement through employing guidelines, training, supervision, feedback and facilitation, called the Emergency Triage and Treatment Plus (ETAT+) strategy	cRCT	14 process measures (e.g. child’s weight documented)	It was assumed that the impact of the intervention produced a 1–10% relative reduction of mortality rate of 7%
Sabin	2012	Health work-force	Zambia	Africa	Conduct 4-day sessions of training to traditional birth attendants (TBAs) to perform interventions targeting birth asphyxia, hypothermia, and neonatal sepsis, followed by 1–2 day refresher trainings approximately every 3–4 months for the duration of the trial	cRCT	Neonatal mortality rate	Neonatal mortality was 45% lower among live born infants delivered by intervention birth attendants than control birth attendants (rate ratio 0.55, 95% CI 0.33–0.90). Deaths due to birth asphyxia were reduced by 63% among infants delivered by intervention birth attendants (0.37, 0.17–.81) and by 81% within the first 2 days after birth (0.19, 0.07–0.52)
Manasyan	2011	Health work-force	Zambia	Africa	Offer the essential newborn care course, which included universal precautions and cleanliness; routine neonatal care; initiation of breathing and resuscitation; prevention of hypothermia; early and exclusive breastfeeding; kangaroo (skin-to-skin) care; small infant care; counselling on infant care; and danger signs, recognition, and initial management of illnesses	Pre-post design	Neonatal mortality rate	All-cause 7-day (early) neonatal mortality decreased from 11.5 per 1000 to 6.8 per 1000 live births after essential newborn care training of the clinic midwives (relative risk: 0.59; 95% confidence interval: 0.48–0.77; 40 615 births)
Somigliana	2011	Service delivery	Uganda	Africa	Use an ambulance within a hospital-/community-based reproductive health service	Cohort study	Referrals	Ninety-two obstetrical referrals were recorded. Eleven (12%) were considered effective, corresponding to 611.7 years saved
Nizalova	2010	Service delivery	Ukraine	Eastern Europe	Implement a comprehensive mother and infant health project (MIHP). The MIHP promoted new evidence-based medicine (EBM) standards: partner deliveries; avoidance of unnecessary C-sections, amniotomies and episiotomies; use of free position during delivery; immediate skin-to-skin contact; early breastfeeding; and the rooming-in of mothers and newborns	Pre-post controlled design	Maternal and infant deaths	The number of C-sections in the MIHP participating Rayons decreased by 4.71% or by 132.17 deliveries on average per year. The estimates suggest that the MIHP participation on average translates into 1.69 fewer maternal deaths per maternity per year and 5.63 fewer infant deaths resulted from deviations in perinatal period
Tripathy	2010	Service delivery	India	Asia	Recruit and train facilitators to convene monthly women’s group meetings, and encourage women’s groups to adopt strategies to improve maternal and neonatal health Implement other health system strengthening activities (including training in newborn care, equipment and supplies)	cRCT	Neonatal mortality rate	NMR was 32% lower in intervention clusters adjusted for clustering, stratification, and baseline differences (odds ratio 0.68, 95% CI 0.59–0.78) during the 3 years and 45% lower in years 2 and 3 (0.55, 0.46–0.66). Although we did not note a significant effect on maternal depression overall, reduction in moderate depression was 57% in year 3 (0.43, 0.23–0.80)
Bang	2005	Service delivery	India	Asia	1. Train female village health workers (VHWs) to diagnose and manage birth asphyxia (when supported by TBAs at delivery) in comparison with current practice with TBAs trained to manage birth asphyxia	Pre-post design	Incidence of mild birth asphyxia; case fatality of neonates with severe asphyxia	The incidence of mild birth asphyxia decreased by 60%, from 14% in the observation year (1995–96) to 6% in the intervention years. The incidence of severe asphyxia did not change significantly, but the CF in neonates with severe asphyxia decreased by 47.5%, from 39% to 20% and ASMR by 65%, from 11% to 4%. Mouth-to-mouth resuscitation reduced the ASMR by 12%, tube-mask further reduced the CF by 27% and the ASMR by 67%
Bang	2005	Service delivery	India	Asia	Train village health workers to provide neonatal care	Pre-post controlled design	Neonatal mortality rate	The NMR in the intervention area decreased from 62 to 25. The reduction in comparison to the control area was 70%. The reduction in the NMR was created by the reduction in both the early NMR (24 points) and the late NMR (20 points). The SBR decreased by 49%; the PMR decreased by 56%
Borghi	2005	Service delivery	Nepal	Asia	Training community facilitators to work with women’s groups to develop strategies for improvement of maternal and neonatal health	cRCT	Neonatal mortality rate and maternal mortality	Intervention group achieved a 29% reduction in neonatal mortality and a substantial reduction in maternal mortality during 33 months
Duke	2000	Service delivery	Papua and New Guinea	Oceania	Introduce minimal standards of neonatal care in 10 areas	Pre-post design	Neonatal mortality rate	The in-hospital neonatal mortality in the 30-month period after the interventions began was 44% lower (relative risk (RR) 0.56). After adjustment for a higher number of neonates <1.5 kg in the pre-intervention period, the relative risk was 0.59. The mortality in the intervention phase for very low birthweight babies was 56% lower (RR 0.44) and for moderate low birthweight (1.5–2 kg) 50% lower (RR 0.50)
Fox-Rushby	1996	Service delivery	Gambia	Africa	Conduct mobile outreach services, with two midwives providing antenatal and family planning care for 22 villages, and visiting villages regularly.	Pre-post controlled design	Neonatal mortality rate and maternal mortality rate	Neonatal mortality rate was reduced from 32.2 to 16 per 1000 live births, and maternal mortality rate from 7 to 3.1 per 1000 live births

cRCT: cluster randomized control trial; RBF: results-based financing; IFG: input financing group; PCG: pure control group; Hib: haemophilus influenzae type B vaccine; SD: standard deviation; CI: confidence interval; ANC: antenatal care; CHW: community health worker; NMR: neonatal mortality rate; MMR: maternal mortality rate; IMR: infant mortality rate; SBR: stillbirth rate; PMR: perinatal mortality rate; EBF: exclusive breastfeeding; TBA: traditional birth attendant; ASMR: asphyxia-specific mortality rate; CF: case fatality; ORS-Z: oral rehydration solution plus zinc.

Strategy of HSS

Using the six building blocks to categorize the studies, 15 were on enhancing service delivery (Fox-Rushby and Foord 1996; Duke et al. 2000; Borghi et al. 2005; Bang et al. 2005a,b; Tripathy et al. 2010; Somigliana et al. 2011; Barasa et al. 2012; Broughton et al. 2013; Fottrell et al. 2013; LeFevre et al. 2013; Lewycka et al. 2013; Bishai et al. 2015; Colbourn et al. 2015; Nizalova and Vyshnya 2010), 7 on financing (Gertler et al. 2014; Alfonso et al. 2015; Gomez et al. 2015; Saya et al. 2015; Wang et al. 2016; Shepard et al. 2017; Zeng et al. 2017), and 2 studies on workforce (Manasyan et al. 2011; Sabin et al. 2012). However, we found no studies on cost-effectiveness of the building blocks of leadership/governance, information and medical supplies.

For service delivery, strategies were taken at both community- and health-facility levels. At the community level, five studies examined community mobilization through participatory women’s groups or peer counsellors to develop strategies to improve maternal and neonatal mortality rate (Borghi et al. 2005; Tripathy et al. 2010; Fottrell et al. 2013; Lewycka et al. 2013; Colbourn et al. 2015); three studies investigated training to community health workers for providing MCH services (e.g. conducting home visits) (Bang et al. 2005a; Bang et al. 2005b; LeFevre et al. 2013); and one study used a social marketing approach to distribute oral rehydration salts and zinc (Bishai et al. 2015).

At the health-facility level, strategies undertaken included quality improvement (Duke et al. 2000; Nizalova and Vyshnya 2010; Barasa et al. 2012; Broughton et al. 2013; Colbourn et al. 2015) through training health providers, coaching visits, supporting administrative management and strengthening adherence to treatment guidelines and standards. In one study, conducted in The Gambia, health providers were also encouraged to conduct more outreach activities to expand services to pregnant women and infants who would otherwise be neglected (Fox-Rushby and Foord 1996). Strategies to provide transportation were undertaken (Somigliana et al. 2011) to remove a barrier for pregnant women to access MCH services.

Similarly, various approaches on financing strategies were also implemented. From the supply-side, there were three articles on PBF (Gertler et al. 2014; Shepard et al. 2017; Zeng et al. 2017) conducted in Argentina, Zimbabwe and Zambia, respectively, where health providers were offered financial incentives for providing MCH services. From the demand-side, two articles concerned health insurance (Gomez et al. 2015; Saya et al. 2015); one examined voucher schemes that offered financial incentives to pregnant women for seeking MCH services (Alfonso et al. 2015) and one from Zambia reported offering ‘Mama kit’, a non-financial incentive conditional on giving birth in health facilities.

On strategies concerning workforce, the major intervention was providing training to health personnel. As mentioned earlier, there existed overlap of training health providers between the block of human resources and service delivery. The two studies categorized in the block of workforce were for capacity building only and did not have other complementary activities. Sabin et al. (2012) examined the cost-effectiveness of training traditional birth attendants in Zambia, while Manasyan et al. (2011) focused on training related to health providers on essential newborn care.

Study design and assessment of study quality

Of the 24 articles, 10 applied cluster randomized control trial design, 7 pre-post controlled design and 4 pre-post design without a control group. There were two studies using cohort study design, following research subjects over a certain period and observing health outputs or outcomes. The last study used parameters from another country to estimate effectiveness.

Based on the grading process described in the Methods section, the average quality score for the 24 articles was 82%, with 15 articles rated as high quality, 5 as medium quality and 4 as low quality. The primary reasons for low quality grade were defective study design in assessing the program’s impact, inappropriate approaches to convert health outputs to outcomes or missing some cost components.

Depending on perspectives of the studies, cost components of the 24 articles varied substantially. Four studies examined costs from the project/program’s perspective (Borghi et al. 2005; Nizalova and Vyshnya 2010; Tripathy et al. 2010; Manasyan et al. 2011), which did not account for costs occurred in health facilities, either in public or private settings. Nor did it account for costs borne by households. As most interventions intended to improve utilization of MCH services, without considering potential increased costs due to increased use of services, cost-effectiveness of interventions may be overestimated. One study only costed equipment and suppliers (Duke et al. 2000), which might substantially inflate the cost-effectiveness of the intervention.

Measures of cost and effectiveness

Table 4 provides detailed information on cost-effectiveness from the 24 articles. In total, 38 cost-effectiveness values were obtained. Only five values provide the long-term cost-effectiveness of the program (length of assessment ≥ 4 years). As to study perspectives, 22 values (57.89%) were from the health provider’s perspective, eight (21.05%) from the program/project’s perspective, and eight (21.05%) from the societal perspective.

Table 4.

Characteristics of cost-effectiveness analysis of included studies

Last name of the first author	Perspective	Year of dollar expressed	Length of assessment (months)	Intervention	Comparator/ scenarios	CE value	CE measure	Sensitivity analysis	GDP/capita (USD)Δ
Colbourn	Health provider	2013	27	Community mobilization	CM vs SQ	79	DALY averted	Yes	317
Colbourn	Health provider	2013	27	Quality improvement	FI vs SQ	281	DALY averted	Yes	317
Colbourn	Health provider	2013	27	CM+FI	CM+FI vs SQ	146	DALY averted	Yes	317
Gomez	Health provider	2012	144	Health insurance	Int vs SQ	46	DALY averted	Yes	2798
Saya	Health provider	2013a	Not clear	Health insurance	Int vs SQ	298	DALY averted	No	681
Gerler	Health provider	2005	60	PBF	Int vs SQ	814	DALY averted	Yes	5164
Bang	Health provider	2003a	84	VHWs training	Int vs SQ	6.8	DALY averted	No	572
Broughton	Health provider	2008	30	Quality improvement	Int vs SQ	147	DALY averted	No	382
Barasa	Health provider	2011	18	Quality improvement	Int vs SQ (Best case)	40	DALY averted	Yes	1062
Barasa	Health provider	2011	18	Quality improvement	Int vs SQ (Worst case)	398	DALY averted	Yes	1062
LeFevre	Program	2010	30	CHWs training	Int vs SQ	103	DALY averted	Yes	808
Alfonso	Program	2010	37	Voucher scheme	Int vs SQ	338	DALY averted	Yes	594
Manasyan	Program	2005	24	Midwife training	Int vs SQ	5.2	DALY averted	No	692
Alfonso	Societal	2010	37	Voucher scheme	Int vs SQ	302	DALY averted	Yes	594
LeFevre	Societal	2010	30	CHWs training	Int vs SQ	105	DALY averted	Yes	808
Bishai	Societal	2010	10	Social marketing	Int vs SQ	214	DALY averted	Yes	997
Sabin	Societal	2011	120	TBAs training	Int vs SQ	74	DALY averted	Yes	1636
Shepard	Health provider	2013	27	PBF	Int vs SQ	662	QALY gained	Yes	1005
Zeng	Health provider	2013	27	PBF	Int vs input financing	1350	QALY gained	Yes	1840
Zeng	Health provider	2013	27	PBF	Int vs pure control	809	QALY gained	Yes	1840
Lewycka	Health provider	2010	72	Community mobilization	Int vs SQ	33–114	LYS	No	443
Somigliana	Health provider	2009	3	Ambulance service	Int vs SQ	16	LYS	Yes	565
Tripathy	Program	2007	36	Community mobilization	Women’s group vs SQ	33	LYS	Yes	1081
Tripathy	Program	2008	36	Community mobilization	Women’s group + HSS vs SQ	48	LYS	Yes	1081
Borghi	Program	2003	33	Community mobilization	Int vs SQ	211	LYS	Yes	254
Fottrell	Health provider	2011	24	Community mobilization	Int vs SQ	330	LYS	No	857
Fox-Rushby	Societal	1991	24	Mobile outreach	Int vs SQ (best - worst case)	42.9–459.0	Discounted LYS	Yes	486
Nizalova	Program	2005	72	Quality improvement	Int vs SQ	0.01	Dollar gained	No	1910
Wang	Health provider	2013a	11	Mama kit	Int vs SQ	5183	Life saved	No	1840
Bang	Health provider	2003a	84	VHWs training	Int vs SQ	13	Life saved	No	572
Bang	Health provider	2003a	84	VHWs training	Int vs SQ	150.5	Life saved	No	572
Duke	Health provider	1998a	30	Quality improvement	Int vs SQ	445	Life saved	No	1158
Alfonso	Health provider	2010	37	Voucher scheme	Int vs SQ	22 933	Life saved	Yes	594
Fottrell	Health provider	2011	24	Community mobilization	Int vs SQ	10 053	life saved	No	857
Manasyan	Program	2005	24	Midwife training	Int vs SQ	208	Life saved	No	692
Alfonso	Societal	2010	37	Voucher scheme	Int vs SQ	20 575	Life saved	Yes	594
Bishai	Societal	2010	10	Social marketing	Int vs SQ	5955	Life saved	Yes	997
Fox-Rushby	Societal	1991	24	Mobile outreach	Int vs SQ (Best - Worst case)	206.3–2134.0	Life saved	Yes	486

a

Best guess; Δ source: international monetary fund, available at http://www.imf.org/external/pubs/ft/weo/2016/02/weodata/index.aspx.

CE: cost-effectiveness; GDP: gross domestic product; Int: intervention; SQ: status quo; HSS: health system strengthening; CM: community mobilization; FI: facility intervention; TBAs: traditional birth attendants; VHWs: village health workers; CHW: community health workers; PBF: performance-based financing; LYS: life year saved; DALY: disability-adjusted life year; QALY: quality-adjusted life year; USD: United States Dollar.

Of the 38 values of cost-effectiveness, 17 were measured with DALYs averted, followed by 10 using lives saved, 6 using life years saved, 3 using QALYs gained, 1 using discounted life years saved and 1 using dollars gained.

If cost-effectiveness were examined using the same perspective for the studies reporting DALY or QALYs, the values could be compared. Compared with the benchmark of GDP per capita, all 13 cost-effectiveness values reported as cost per DALY averted or QALY gained from the health provider’s perspective were less than one times GDP per capita (Table 5), suggesting that associated interventions were highly cost-effective. Among them, the training of village health workers in India had the lowest cost-effectiveness ratio and the lowest relative cost-effectiveness to its GDP/capita. The three PBF programs had cost-effectiveness ratios ranging from $662 to $1350/DALY averted or QALY gained. Their relative values to GDP per capita spanned from 0.158 to 0.734, indicating that PBF was among the highly cost-effective interventions in addressing MCH.

Table 5.

Cost-effectiveness reported as cost/DALY averted or QALY gained from health provider’s perspective

Last name of the first author	Intervention	Comparator/scenarios	Cost($)DALY averted or cost($)QALY gained	Proportion of GDP/capita
Bang	VHWs training	Int vs SQ	6.8	0.012
Gomez	Health insurance	Int vs SQ	46.4	0.017
Barasa	Quality improvement	Int vs SQ (Best case)	39.8	0.037
Gerler	PBF	Int vs SQ	814.0	0.158
Colbourn	Community mobilization	CM vs SQ	79.0	0.249
Barasa	Quality improvement	Int vs SQ (Worst case)	398.3	0.375
Broughton	Quality improvement	Int vs SQ	147.0	0.385
Saya	Health insurance	Int vs SQ	298.0	0.438
Zeng	PBF	Int vs pure control	809.0	0.440
Colbourn	CM+FI	CM+FI vs SQ	146.0	0.461
Shepard	PBF	Int vs SQ	662.0	0.659
Zeng	PBF	Int vs input financing	1350.0	0.734
Colbourn	Quality improvement	FI vs SQ	281.0	0.886

DALY: disability-adjusted life year; QALY: quality-adjusted life year; GDP: gross domestic product; VHWs: village health workers; Int: intervention; SQ: status quo; CM: community mobilization; FI: facility intervention; PBF: performance-based financing.

Discussion

This systematic review identified 24 articles on cost-effectiveness of HSS interventions for improving MCH. The major HSS interventions concern service delivery, health financing and human workforce. None of the articles reported on governance/leadership, supply chain or information systems.

Consistent to what had been synthesized before on cost-effectiveness of overall strategies in improving MCH (Mangham-Jefferies et al. 2014), this review of HSS strategies has also found that countries take diverse approaches to address their health system gaps in responding to MCH concerns. For example, in Bangladesh, where awareness of MCH services is low and there exists a dynamic non-governmental organization sector (Ahmed et al. 2013), community mobilization programs are piloted and tested (Fottrell et al. 2013; LeFevre et al. 2013). In contrast, in countries, such as Kenya and Nigeria, where quality of health care is a major concern, quality improvement programs through training personnel and strengthening adherence to protocol are implemented (Barasa et al. 2012; Broughton et al. 2013). To address financial barriers for pregnant women, strategies such as health insurance and voucher schemes, as well as providing ambulance services, are carried out to improve MCH service coverage and outcomes (Somigliana et al. 2011; Alfonso et al. 2015; Gomez et al. 2015; Saya et al. 2015). Each country has its own health system concerns. To design cost-effective health system interventions, it is important to conduct health system diagnosis to identify its malfunctions (Balabanova et al. 2010), in order to design more targeted and effective interventions.

We also found that the diversity of HSS strategies lies in both the supply- and demand-side interventions. From the supply side, HSS strategies targeting health service providers (e.g. hospitals, health clinics and medical personnel) take the form of, e.g. training health personnel, providing equipment, and incentivizing health providers, and these strategies are instrumental in ensuring quality of care and reaching out to targeted populations to deliver services. Donors play an important role in supporting supply interventions (Valentine et al. 2015). On the other hand, demand-side strategies are directly targeted to users of health care services, such as pregnant women and children. Community engagement and providing financial or non-financial incentives to service users are the most common demand-side interventions (Hurst et al. 2015). As demand for health services increases, it is expected that the use of needed services would increase. A review shows that, overall, demand-side interventions increase the use of health services, but do not necessarily improve health outcomes (Hurst et al. 2015). To address MCH more effectively, it is critical to leverage strengths of both supply- and demand-side strategies. In Cambodia, it was reported that the effects of PBF on MCH services quadrupled when it was implemented simultaneously with a voucher scheme (Van de Poel et al. 2016). Similarly, Colbourn et al. (2015) examined cost-effectiveness of combined demand- and supply-side interventions and found a lower cost-effectiveness ratio for the combined approach than supply-side intervention alone, suggesting strong synergy between demand- and supply-side interventions. When designing comprehensive programs, policy makers should take a holistic approach that considers synergies among programs in order to achieve better outcomes with lower costs. The complementarities among HSS interventions may preclude a league table approach to report their cost-effectiveness.

We extracted cost-effectiveness values from the 24 articles in the hope of making a comparison among them. However, the studies used different cost-effectiveness measures, took different perspectives of analysis, and applied different assumptions for modelling, which significantly limited the comparability among studies. Even though some studies under the review used the same cost-effectiveness measures and took the same perspective, some results were still not comparable, due to, e.g. cost components included in the analysis, as mentioned in the Results section. In addition, cost effectiveness comparisons were difficult because of the inconsistency in approaches used in measuring effectiveness. Some studies used measures of changes of mortality rate, from which DALYs averted or QALYs gained could be derived directly. Other studies collected changes of utilization of health services as impact measures. Thus, modelling was needed to convert utilization of services to health outcomes for cost-effectiveness analyses. Often modelling relies on international literature, which may not provide accurate parameters for the country where the program was implemented. Given the heterogeneity of the approaches employed in the CEA for HSS interventions, to allow for better comparison among studies, standardization of CEA of health system intervention should be given a priority. The recommendations from the Second Panel of CEA in Health and Medicine (Sanders et al. 2016) provide a solid foundation to accomplish the needed standardization. Additionally, with only 24 articles found on economic evaluation of HSS interventions for MCH, it is critical to generate more relevant evidence by conducting cost-effectiveness/benefit studies on this topic. Future reviews may also use a broader range of search terms to capture such evidence. This evidence will help inform donors and governments about HSS investments.

Specifically for PBF, one of the major HSS interventions applied in LMICs, impact evaluation of PBF programs generally demonstrates a positive impact of financial incentives on quality and coverage of MCH services (Basinga et al. 2011; Soeters et al. 2011; Zeng et al. 2013; Gertler et al. 2014; Shepard et al. 2017; Zeng et al. 2017), such as prenatal care, institutional deliveries and postnatal care. According to the recommendation from the WHO’s Commission on Macroeconomics and Health (World Health Organization 2009), PBF programs, although costly, prove to be highly cost-effective, whether they are modelled through health outcomes (Shepard et al. 2017; Zeng et al. 2017) or through direct examination of maternal and neonatal mortality rates (Gertler et al. 2014). As the coverage of services increases through PBF, PBF programs may need to switch their focus to the improvement of quality of care in the future. Given the challenge in modelling the health impact of quality of care, direct examination of changes of mortality rates attributable to PBF would be more appropriate when conducting CEA or the impact evaluation of future PBF programs (Makate and Makate 2016). Table 5 shows that there are three HSS interventions, such as training of village health workers (Bang et al. 2005b), health insurance (Gomez et al. 2015), quality improvement (Barasa et al. 2012), having a lower relative cost-effectiveness ratio than the PBF program with the lowest relative cost-effectiveness ratio (Gertler et al. 2014). Those three studies were conducted on a relatively smaller scale (e.g. coverage population) than was the RBF program. It is likely that as those programs scale up, their cost-effectiveness ratio may increase and they might be less cost-effective, given diminishing returns to the investment.

For HSS interventions carried out on a large scale, as the interventions are generally costly and involve in system-wide changes, factors beyond cost-effectiveness of interventions, such as the availability of financial resources, may play an important role in deciding whether to implement or scale up the interventions. To better inform decision making for policy makers, health system constraints, such as governance and political constraints, should also be considered alongside assessments of the cost-effectiveness of the interventions (Hauck et al. 2016). For example, in the case of severe governance constraints that undermine the capacity to implement or scale up programs, implementing CEA may need to consider these constraints by restricting the number of decisions proposed in the analysis (Hauck et al. 2016).

It should also be noted that among the 24 articles, only a few studies examined long-term cost-effectiveness of interventions. Perhaps due to time and budget constraints, most cost-effectiveness studies included in this review were for short-term assessment, <4 years. Given that some start-up costs could be shared for a longer period and that program management and implementation skills improve over time, cost-effectiveness of a mature program with a long implementation period tends to be more favorable. Bang et al. (2005a) estimated that over the 7 years of their study, the cost-effectiveness ratio for training village health workers in India was only 0.12 times of GDP per capita, one of the lowest cost-effectiveness ratios among all the interventions. More studies should be conducted to examine long-term cost-effectiveness of a program for informed policy-making. At the same time, policy makers should also be aware of the length of cost-effectiveness assessments, and gauge the program’s long-term cost-effectiveness when making decisions.

Several limitations of this review should be acknowledged. First, although we endeavoured to obtain as many studies as possible for screening, we were not able to review all records from the large amount of search results from Google Scholar. Additionally, the number of articles for inclusion in this study depended on the choice of key search words. We used only few choice key words for the building block of information, realizing that this would result in missing some relevant studies. Second, most interventions were cost-effective, and some studies had a very low cost-effectiveness ratio, which may suggest under reporting of negative results. Third, given the broad definition of health systems, we had to limit the scope of interventions to some domains of the health system, particularly around service delivery, where we limited the search to quality improvement and innovative delivery models. In spite of these limitations, this review assembled evidence on HSS interventions systematically, contributing to a better understanding of HSS in addressing MCH and evidence-based decision making. This review found that all of the HSS strategies reporting cost/QALY or cost/DALY proved cost-effective. As the strategies operate in different ways, they are often complementary rather than competitive. The strategies with multiple evaluations (quality improvement, PBF, and health insurance) had substantially overlapping ranges of cost/QALY or cost/DALY. This finding suggests that policy makers should endeavor to support multiple approaches, with choices focusing less on broad strategy and more on details of the design and implementation.

Acknowledgements

The authors are indebted to Clare Hurley at Brandeis for editorial assistance. Views expressed in this article are those of the authors and do not necessarily reflect the views of the authors’ institutions or the funder of this study.

Funding

This study was funded by the World Bank (contract number: 7181627).

Conflict of interest statement. None declared.

Author notes

Wu Zeng and Guohong Li authors contributed equally to this work.