Modeling Human Papillomavirus Vaccine Effectiveness : Quantifying the Impact of Parameter Uncertainty

The development of models is based on assumptions, which inevitably embed a level of uncertainty. Quantifying such uncertainty is particularly important when modeling human papillomavirus (HPV) vaccine effectiveness; the natural history of infection and disease is complex, and ageand type-specific data remain scarce and incomplete. The aim of this study was to predict the impact of HPV-6/11/16/18 vaccination, using a cohort model andmeasuring parameter uncertainty. An extensive fitting procedure was conducted, which identified 164 posterior parameter combinations (out of 200,000 prior parameter sets) that fit simultaneously HPV type-specific incidence and prevalence data for infection, cervical intraepithelial neoplasia (CIN), and squamous cell carcinoma (SCC). Results based on these posterior parameter sets suggest that vaccinating girls aged 12 years (vaccine efficacy 1⁄4 95%, no waning) would reduce their lifetime risk of HPV infection, CIN1, CIN2/3, and SCC by 21% (80% credibility interval: 17, 29), 24% (80% credibility interval: 17, 31), 49% (80% credibility interval: 36, 60), and 61% (80% credibility interval: 47, 73), respectively. If vaccine efficacy is reduced or vaccine protection is assumed to wane, uncertainty surrounding predictions widens considerably. Important priorities for future research are to understand the role of natural immunity and to measure the duration of vaccine protection because results were most sensitive to these parameters.

Policy makers will require further questions to be answered, beyond clinical efficacy and safety, before decisions are made regarding HPV vaccination programs (36)(37)(38).These questions include determining the optimal vaccine strategy for reducing HPV-related disease, age(s) at vaccination, and whether booster vaccinations will be necessary.Mathematical models have proven extremely useful to examine these types of questions.By combining clinical trial results with information from various sources (biologic, epidemiologic), models are able to address questions that cannot feasibly or ethically be answered in a trial setting (39).The development of models is based on assumptions, which inevitably embed a level of uncertainty.It is important to quantify the impact of such uncertainty on model predictions to provide policy makers with the necessary information to make appropriate decisions.In the case of HPV, it is particularly important to quantify uncertainty for two main reasons.First, the model structure is complex because the natural history of HPV encompasses numerous stages of disease progression that may vary depending on the genotype, the screening policy, and the treatment.Second, the model parameters are difficult to estimate because the scientific literature on age-and type-specific HPV natural history and transmission remains scarce.Moreover, for some parameters, such as progression rates to cancer, empirical studies cannot be carried out for ethical reasons.
Despite considerable uncertainty surrounding typespecific natural history parameters, previous modeling studies have derived predictions regarding the impact of HPV vaccination by using only a base-case parameter set (40)(41)(42)(43)(44)(45)(46)(47).In these studies, one-way sensitivity analyses were performed, which are limited in scope because they do not take into account the conjoint uncertainty of all natural history parameters.
On the basis of these considerations, the aim of our study was to 1) develop and parameterize an HPV natural history model, 2) estimate the potential effectiveness of HPV vaccination, and 3) assess the sensitivity of model predictions to parameter uncertainty.

Model structure
We developed a model of the natural history of HPV infection and squamous cell carcinoma (SCC) in a cohort of women followed over their lifetimes.The natural history of HPV is represented by six mutually exclusive disease states: susceptible, immune, infected, CIN1, CIN2/3, and SCC.Women can transition between these states, as described in figure 1, left.Transition probabilities between disease states are allowed to vary according to HPV genotype and age.We do not model adenocarcinoma because its natural history is different from that of SCC (48).
The model includes four classes of HPV genotypes: HPV-16, HPV-18, other high-oncogenic risk types (HR), and lowrisk oncogenic types (49)(50)(51).The model incorporates the following dual coinfections-16-18, 16-HR, 16-low oncogenic risk, 18-HR, and 18-low oncogenic risk (figure 1, right)-because they represent the large majority of multiple concurrent coinfections (3,(49)(50)(51)(52)(53)(54)(55)(56).Coinfected women will progress and regress through the HPV disease states according to the transition rates of the most aggressive type.Following clearance from HPV-16 or -18, women have a probability of developing type-specific lifelong natural immunity (figure 1, right).However, we assume that infection from HR or low oncogenic risk types does not result in lifelong natural immunity since these groups comprise many genotypes and have a small probability of reexposure to the same genotype.
The model also accounts for screening and treatment outcomes.Women have an age-specific probability of being screened and a lesion-specific test sensitivity of being detected.Following treatment, women can remain infected with the genotype, clear the infection, or stop being susceptible to HPV infection and disease because of hysterectomy or ablative treatment (57)(58)(59)(60)(61)(62)(63)(64)(65)(66).Women also face age-specific mortality and hysterectomy for conditions unrelated to HPV.

Parameter values and model fit
Parameter values and references are summarized in tables 1 and 2. All natural history parameters are type and age specific.Parameterization and model validation were performed by using the following five-step process: Step 1. Setting prior distributions from prospective studies: A comprehensive search of prospective HPV studies published between 1995 and 2005 was conducted.Each type-specific natural history parameter was assigned a prior uniform distribution between the minimum and maximum estimates found in the literature review.With the exception of the type-specific force of HPV infection and progression rate from CIN2/3 to SCC (appendix 1), the prior distributions are not age specific because the important heterogeneity or the lack of information in the literature did not enable us to make any assumptions regarding age-specific patterns.Refer to table 2 for prior parameter ranges.Step 2. Sampling parameter sets: A total of 200,000 parameter sets were drawn from the uniform prior parameter distributions by using Latin Hypercube Sampling (67).
Step 3. Identifying epidemiologic data for parameter fit and validation: North American-specific epidemiologic data were identified, from a comprehensive review of articles and reports, for model fit and cross-validation.The literature review included studies published between 1995 and 2005.Studies among populations at ''high risk'' of HPV infection were excluded.Whenever possible, data specific to Canada were used.Appendix 2 lists epidemiologic outcomes and references.
Step 4. Fitting the model and identifying posterior parameter sets: Parameter sets from step 2 were judged to produce ''acceptable'' fit if the associated model predictions were simultaneously within prespecified targets (ranges) defined by using the epidemiologic data from step 3. Refer to appendix 3 for a description of how targets were chosen.
Step 5. Cross-validating model fit: To assess the predictive value of the model by using the posterior (''acceptable'') parameter sets, we compared model predictions for the cross-validation outcomes (those not used in step 4) with observed epidemiologic data from step 3 (appendix 2).

Model simulations
Model predictions of the natural history of HPV and the impact of HPV vaccination are based on the posterior parameter sets identified during model fitting.The uncertainty  of model predictions is presented as the median, 10th and 90th percentiles of results from the posterior parameter sets, which we call the 80 percent credibility intervals.Impact of vaccination was assessed under current screening algorithms in Canada (68).

Vaccination strategies
The base-case vaccine strategy assumes vaccination of girls aged 12 years.Sensitivity analysis is performed to evaluate the impact of age at vaccination and booster scenarios.

Vaccine characteristics
Base-case vaccine characteristics are assumed to be as follows: 1) The proportion of females protected following vaccination (take) is 100 percent, 2) vaccine duration is lifelong, and 3) reduction in susceptibility to HPV-6/11/16/18 (vaccine efficacy) is 95 percent.A sensitivity analysis is performed to explore the impact of vaccine parameters on model predictions.Vaccine duration is varied by assuming a constant waning of vaccine protection, resulting in an exponential decay of efficacy for the protected population.

Model fit and validation
Of 200,000 different combinations of parameters sampled from the prior parameter distributions, 164 parameter sets produced model results within the prespecified targets.The model predictions based on the 164 parameter sets produced good visual fit to the age-and type-specific epidemiologic data listed in appendix 2, which include overall HR HPV prevalence, overall SCC incidence, and the proportion of SCC attributable to HPV-16 and -18 (figure 2, upper left, upper right, lower left).In addition, model predictions for outcomes not used in the fitting procedure (cross-validation outcomes) were also consistent with their corresponding epidemiologic data (figure 2, lower right).

Parameter uncertainty
The posterior parameter sets that best fit the observed epidemiologic data cover a wide range of values.Figure 3 illustrates the age-specific prior and posterior ranges for HPV-16 force of infection.Here, and for the other age-specific parameters, the range of values of the posterior parameter sets is only 40-50 percent narrower than the wide ranges given to the prior distributions (table 2), suggesting that the natural history of HPV and SCC can be fitted by using a broad range of parameter combinations that seem equally probable.The parameter with the least amount of information gained from the fitting procedure and with the greatest uncertainty is natural immunity (percentage of individuals who acquire natural lifelong immunity) because its posterior range covers most of the prior distribution.

Base-case vaccine scenario
Under our base-case vaccine assumptions, the model predicts that vaccinating girls aged 12 years against HPV-6/11/ 16/18 would reduce their overall lifetime risk of HPV infection, CIN1, CIN2/3, and SCC by 21 percent (80 percent credibility interval: 17, 29), 24 percent (80 percent credibility interval: 17, 31), 49 percent (80 percent credibility interval: 36, 60), and 61 percent (80 percent credibility interval: 47, 73), respectively (figure 4, left).The proportions of these lifetime risk reductions achieved before 30 years of age for HPV infection, CIN1, CIN2/3, and SCC are, respectively, 40 percent, 44 percent, 55 percent, and 20 percent.As expected, the HPV-16 component of the vaccine contributes most to the overall reduction in the burden of HPV disease (figure 4, right).Although the HPV-6/11 component of the vaccine is primarily thought of for its potential impact on genital warts, these genotypes also contribute significantly to the reduction in HPV infection and CIN1 (figure 4, right).Of note, the model predicts an increase in the number of CINs and SCCs attributable to the HR genotypes not included in the vaccine.This increase occurs mainly because, in the model, we assume that women with coinfection progress and regress according to the transition rates of the most aggressive HPV genotype, and disease is attributed to this type.Hence, by preventing disease due to HPV-6/11/16/18, vaccination allows greater opportunity for HR genotypes to cause CIN and SCC even if a woman's risk of infection remains unchanged (figure 4, right).Since multiple coinfections are allowed and cross-protection is not modeled, the model predicts that vaccination will have very little impact on the incidence of HPV infection from HR (figure 4, right).

Impact of vaccine characteristics and vaccination scenarios
The model shows that small increases in vaccine efficacy can result in a disproportionately higher reduction in disease (figure 5, upper left).The source of this amplification occurs at the level of genotype-specific infection.For example, a 5 percent increase (from 95 percent to 100 percent) in HPV-16 vaccine efficacy increases the lifetime reduction in HPV-16related infection by 14 percent.This effect occurs only if clearance of infection confers natural immunity, resulting in depletion of susceptible women who would otherwise become infected following successive exposure to the same genotype.In a model that assumes no natural immunity, vaccine efficacy will be equal to effectiveness.
Model predictions are particularly sensitive to the duration of vaccine protection (figure 5, upper right).A vaccine with an average duration of 30 years and 95 percent efficacy given to girls aged 12 years would reduce their lifetime risk of SCC by 6 percent compared with 61 percent, if lifelong protection is assumed.This effect is a consequence of shifts in age at infection, and the magnitude of the difference is mainly due to the level of natural immunity and the agespecific progression/regression of disease.A vaccine with waning immunity prevents natural infection at early stages of sexual activity and, if natural immunity occurs, moves the pool of susceptibles toward older ages.If the force of HPV infection remains high among older women and/or progression rates toward SCC are higher in older women, then waning of vaccine protection can have an important impact on vaccine effectiveness.On the basis of our best-fit parameter sets, the force of infection remains high, the progression from CIN2/3 to SCC is higher in age groups older than 30 years, and lifelong natural immunity occurs.As a consequence, with an average vaccine duration of 30 years, some of the SCC that would have occurred in younger age groups is temporarily prevented and delayed to when women reach age 45 years or older (figure 5, upper right).If the HPV vaccine has significant waning, booster shots are likely to be needed to maintain and lengthen vaccineinduced immunity.The impact of additional booster doses to vaccination of girls aged 12 years is shown in figure 5, upper right.
Under base-case vaccine characteristics, results suggest that vaccination of older age groups can still be highly effective in reducing HPV infection, CIN, and SCC (figure 5, lower left).In fact, vaccinating women aged 30 years can reduce SCC by 49 percent (80 percent credibility interval: 31, 65) because of the large fraction of women still susceptible at this age (e.g., 54 percent are estimated to be susceptible to HPV-16).
Figure 5 shows that the variability in the predicted lifetime reduction in HPV infection and disease depends on vaccine characteristics and the vaccination scenarios investigated.The uncertainty is smaller when vaccine characteristics such as vaccine efficacy (figure 5, lower right) and duration of protection are high; when vaccine efficacy is high and lifelong protection is assumed, age-specific associations between natural history parameter values have less opportunity to affect results.In such cases, the variability around predictions mostly corresponds to the variability of the epidemiologic data used for model fit.

Impact of natural immunity
The most uncertain natural history model parameter is natural immunity (table 2). Figure 6 shows that vaccine effectiveness increases as the probability of developing natural immunity decreases.Most of the difference is explained by women over 50 years of age who benefit from natural protection acquired at younger ages, when the risk of infection is high but clearance of infection is more likely.

DISCUSSION
Model results suggest that, for a wide range of ages at vaccination, the overall incidence of HPV infection, CIN, and SCC, and, by association, repeat Papanicolaou smears, treatment, and mortality, are likely to be significantly reduced by vaccination against HPV.The overall effectiveness of HPV immunization will depend on the characteristics of the vaccine, the type of vaccination strategy, and the agespecific natural history of SCC.
Results from large, randomized controlled trials have shown that the efficacy of HPV vaccines is close to 100 percent at preventing type-specific CIN (30)(31)(32)(33)(34).When 100 percent vaccine efficacy against HPV-6/11/16/18 and lifelong vaccine protection are assumed, the model estimates that vaccinating girls aged 12 years would reduce their overall lifetime risk of SCC by 72 percent (80 percent credibility interval: 58, 82).This reduction is in line with previously published estimates from Goldie et al. (42) and Taira et al. (46), who reported that cervical cancer would be reduced by 62 percent and 66 percent, respectively.Models that assume HPV vaccine to have close to perfect characteristics (or close to 100 percent vaccine type-specific effectiveness) are expected to produce similar results because they are insensitive to natural history assumptions.In such cases, model results are mostly dependent on the incidence and type distribution to which the model has been calibrated.However, when vaccine efficacy is reduced or vaccine protection is assumed to wane over time, then predictions regarding the impact of HPV vaccination become highly sensitive to natural history assumptions, and the uncertainty surrounding vaccine effectiveness predictions widens considerably.It is therefore important when examining the potential impact of vaccine characteristics on effectiveness to do so by varying the natural history of HPV to illustrate the level of confidence one can have in model predictions.
Of the different vaccine characteristics, waning immunity has the greatest impact on effectiveness and generates high levels of uncertainty surrounding model predictions, mostly as a result of three factors.First, vaccine effectiveness is reduced when natural immunity is longer than vaccineinduced immunity.Currently, little is known about the level of natural immunity following clearance of infection.Only a few of the posterior parameter combinations included low probabilities of natural immunity following HPV infection, which suggests that some degree of type-specific immunity occurs.Second, vaccine effectiveness is reduced if progression from CIN2/3 to cancer increases with age.Again, no epidemiologic evidence exists regarding this factor, mostly because of ethical reasons.Third, the effectiveness is lower if the force of infection remains high in women over 30 years of age.Previous studies have shown that the mean rate of new partner acquisition (refer to appendix 1), which should correlate with the force of infection, in women aged 35-39 years is only slightly lower than for those aged 20-24 years (69).
The main strength of the model is that it involves an extensive fitting procedure that identifies multiple parameter sets, which fit simultaneously different epidemiologic data.The parameter sets identified can loosely be considered as different models that enable thorough investigation of the impact of natural history assumptions and parameter uncertainty on predictions of vaccine effectiveness.This strength contrasts with previous modeling approaches, which were centered mainly on a base-case scenario and simple univariate sensitivity analyses.Quantifying parameter uncertainty is particularly important in the context of HPV vaccination because empirical data are scarce and incomplete (especially in older age groups), and the course of disease from infection to development of SCC may never entirely be measured.
There are several methodological advantages to the fitting procedure used.First, we fit HPV type-specific incidence and prevalence data for infection, CIN, and cervical cancer.It is important to fit the model to epidemiologic data from the different stages of disease in order to accurately represent the entire natural history of HPV and SCC.When we fit the model to only observed SCC incidence and genotype distribution, we identify 4,798 posterior parameter sets compared with 164 when HPV infection and CIN data are included in the fitting procedure.These 4,798 parameter sets produce effectiveness results that are much wider than those for the 164 parameter sets we used.Hence, models that use only a base case and fit few epidemiologic outputs have a significant chance of using a parameter set that lacks validity.Second, we do not assume, a priori, age dependencies in our progression and regression rates because no conclusive evidence exists as to these associations.This advantage enables us to explore the impact of age-specific natural history assumptions.Other strengths of our model are that it is, to our knowledge, the first to incorporate genotype-specific susceptibility, natural immunity, and coinfection.
Tangible implications of the modeling approach presented here are that it can help identify the most important data to collect, and it provides decision makers with a measure of uncertainty surrounding predictions.In terms of value of information, model results suggest that efforts should be made to better characterize natural immunity following HPV infection and the epidemiology of HPV in older age groups, because they are important causes of uncertainty in vaccine effectiveness.Similarly, more studies should be focused on quantifying the rate of waning protection following vaccination by measuring antibody decay in clinical trials and/or duration of vaccine protection from surveillance data.
The main limitation of our modeling approach is that it does not take into account the change in the transmission dynamics of infection following vaccination, and not all HPV-related diseases have been factored in.Because model predictions are based on a static model, the effectiveness of vaccination is underestimated because it does not account for herd immunity effects.The rationale behind this choice is that we first need to understand thoroughly the role of parameter uncertainty and the impact of natural history assumptions within a static structure before superimposing the greater uncertainty inherent to dynamic modeling.This model focuses on the role of HPV in the development of SCC because it is currently the most important HPVassociated disease in terms of mortality and health resource use.However, to capture the full potential of the HPV-6/11/ 16/18 vaccine, one should ideally account for the potential to prevent cases of adenocarcinoma, other anogenital cancers, head and neck cancers, and genital warts.
In summary, our modeling approach, incorporating the extensive parameterization procedure, provides a flexible framework to evaluate vaccination strategies that takes into account uncertainty in parameter assumptions.Model predictions suggest that an HPV-6/11/16/18 prophylactic vaccine would significantly reduce the incidence of HPV infection, CIN, and SCC.Important priorities for future research include understanding the role of natural immunity and modeling the duration of vaccine-induced protection.

FIGURE 1 .
FIGURE 1. Model flow diagram.Left: natural history of squamous cell carcinoma (SCC).The different compartments represent individuals in each mutually exclusive state of human papillomavirus (HPV) infection and disease.The arrows represent the flow between these states (solid lines represent progression, dashed lines clearance and regression, dotted lines flow due to screening and treatment, rates of death, and hysterectomy due to reasons other than HPV infection).Right: infection and natural immunity.The different compartments represent the underlying pattern of infection within each disease state.Solid lines represent acquisition of another HPV type, dashed lines clearance of an HPV type, and dotted lines clearance with a probability of developing natural immunity.Women have a certain probability of developing lifelong immunity following natural infection with HPV-16 and HPV-18 (but not other high oncogenic risk types (HR) or low oncogenic risk (LR)).The model allows for dual coinfections with HPV-16 and HPV-18.CIN, cervical intraepithelial neoplasia. d

FIGURE 2 .
FIGURE 2. Model fit.Model fit to observed overall other high oncogenic risk types (HR) human papillomavirus (HPV) prevalence (including HPV-16 and -18, upper left)(73,78,82,86,87); overall incidence of squamous cell carcinoma (SCC) (upper right) (88-92); and distribution of HPV types among cervical cancers (lower left)(29,54).Cross-validation of model predictions to overall prevalence of cervical intraepithelial neoplasia (CIN)2/ 3 (lower right)(82,86, 89,.Solid and dotted lines (black squares and T-shaped bars in the lower left) represent the medians and the 10th and 90th percentiles of the predictions generated by the 164 posterior parameter sets, respectively.BC, British Columbia Annual Report; PHAC, Public Health Agency of Canada; NBCCEDP, National Breast and Cervical Cancer Early Detection Program.

FIGURE 3 .
FIGURE 3. Prior and posterior parameter ranges of human papillomavirus (HPV)-16 force of infection.The shaded area represents the prior parameter space.The white area within the prior parameter space (shaded area) represents the 80% credibility interval of the posterior parameter ranges.The black line in the middle represents the median of the posterior ranges.

FIGURE 4 .
FIGURE 4. Impact of human papillomavirus (HPV)-16/18/6/11 vaccine.Predicted reduction in the overall lifetime risk of HPV infection, cervical intraepithelial neoplasia (CIN)1, CIN2/3, and squamous cell carcinoma (SCC) in vaccinated women by age categories (left) and by genotype (right) (vaccine efficacy ¼ 95%, average duration of protection ¼ life, age at vaccination ¼ 12 years).Bars and T-shaped bars represent the medians and the 10th and 90th percentiles of the predictions generated by the posterior parameter sets, respectively.Note that the reductions in infection and disease due to low oncogenic risk (LR) HPV are the result of the 6/11 component of the HPV-6/11/16/18 vaccine.HR, other high oncogenic risk types.

FIGURE 5 .
FIGURE 5. Impact of vaccine characteristics and vaccination scenarios.Upper left: impact of vaccine efficacy on predicted reductions in the overall lifetime risk of human papillomavirus (HPV) infection, cervical intraepithelial neoplasia (CIN)1, CIN2/3, and squamous cell carcinoma (SCC) (average duration of protection ¼ life, age at vaccination ¼ 12 years); upper right: impact of vaccine duration on the predicted cumulative reduction in the overall lifetime risk of SCC (vaccine efficacy ¼ 95%, age at vaccination ¼ 12 years, medians of the predictions generated by the posterior parameter sets); lower left: impact of age at vaccination on predicted reductions in the overall lifetime risk of HPV infection, CIN1, CIN2/3, and SCC (vaccine efficacy ¼ 95%, average duration of protection ¼ life); lower right: distribution of model predictions on the overall lifetime risk reduction of SCC generated by the posterior parameter sets (average duration of protection ¼ life, age at vaccination ¼ 12 years).Bars and T-shaped bars represent the medians and the 10th and 90th percentiles of the predictions generated by the posterior parameter sets, respectively.

FIGURE 6 .
FIGURE 6. Impact of natural immunity assumptions on vaccine effectiveness.Predicted reductions in the overall lifetime risk of squamous cell carcinoma (SCC) in vaccinated women by level of natural immunity and age categories (vaccine efficacy ¼ 95%, average duration of protection ¼ life, age at vaccination ¼ 12 years).Bars and T-shaped bars represent the medians and the 10th and 90th percentiles of the predictions generated by the posterior parameter sets, respectively.

TABLE 1 .
Demographic, screening, and treatment parameters * HPV, human papillomavirus; CIN, cervical intraepithelial neoplasia.y Assumption because no prior distribution was found in the literature.764 Van de Velde et al.Am J Epidemiol 2007;165:762-775

TABLE 2 .
Natural history parameters