A Systematic Review of Human Challenge Trials, Designs, and Safety

Abstract Background Few studies have assessed participant safety in human challenge trials (HCTs). Key questions regarding HCTs include how risky such trials have been, how often adverse events (AEs) and serious adverse events (SAEs) occur, and whether risk mitigation measures have been effective. Methods A systematic search of PubMed and PubMed Central for articles reporting on results of HCTs published between 1980 and 2021 was performed and completed by 7 October 2021. Results Of 2838 articles screened, 276 were reviewed in full. A total of 15 046 challenged participants were described in 308 studies that met inclusion criteria; 286 (92.9%) of these studies reported mitigation measures used to minimize risk to the challenge population. Among 187 studies that reported on SAEs, 0.2% of participants experienced at least 1 challenge-related SAE. Among 94 studies that graded AEs by severity, challenge-related AEs graded “severe” were reported by between 5.6% and 15.8% of participants. AE data were provided as a range to account for unclear reporting. Eighty percent of studies published after 2010 were registered in a trials database. Conclusions HCTs are increasingly common and used for an expanding list of diseases. Although AEs occur, severe AEs and SAEs are rare. Reporting has improved over time, though not all papers provide a comprehensive report of relevant health impacts. We found very few severe symptoms or SAEs in studies that reported them, but many HCTs did not report relevant safety data. This study was preregistered on PROSPERO as CRD42021247218.

Human challenge trials (HCTs) are a clinical research method in which volunteers are exposed to a pathogen to derive scientifically useful information about the pathogen and/or an intervention [1]. Such trials have been conducted with ethical oversight since the development of the modern institutional review system of clinical trials in the 1970s. More recently, there has been renewed discussion about the ethical and practical aspects of conducting HCTs, largely fueled by interest in conducting HCTs for severe acute respiratory syndrome coronavirus 2. Past reviews of HCTs focused on reporting methods [2] and safety for single pathogens [3][4][5][6], but these did not explicitly evaluate the safety of HCTs by assessing reported adverse events (AEs) and serious adverse events (SAEs) across a range of pathogens. Furthermore, many additional HCTs have been performed since the publication of these reviews. To better inform discussions about future uses of HCTs, including during pandemic response, this article presents a systematic review of challenge trials since 1980 and reports on their clinical outcomes, with particular focus on risk of AEs and risk mitigation strategies.
HCTs are often used to support development of therapies and vaccines more efficiently than conventional clinical trials [6,7] and have recently been discussed as particularly valuable in the context of novel disease pandemics such as coronavirus disease 2019, Zika virus, or a future disease X [8,9]. The benefits of such trials include defining and evaluating correlates of protection [10]; the first Food and Drug Administration (FDA)-approved cholera vaccine, Vaxchora, which proved its efficacy using a small HCT [7]; a contribution to the development of the FDA-approved therapeutic oseltamivir for influenza [11]; the Vi-tetanus toxoid conjugate vaccine for Salmonella typhi [12]; and dosing schedules and adjuvant selection for the RTS,S/AS01 malaria vaccine [13,14].
Arguments against the use of HCTs have centered around ethics of participant compensation and the populations represented, and whether the risks and lack of personal benefit can be compatible with the principle of primum non nocere [15,16] because of the potential risks they may inflict on a study population. Despite the debate, there is a long-standing consensus that infecting healthy volunteers is ethically justifiable as long as the risk of harm is acceptably low [15]. HCTs can therefore be ethical, based on a case-by-case assessment of risk as part of wider research ethics oversight mechanisms.
AEs related to challenge are 1 measure of health risk in HCTs. AEs refer to "any untoward medical occurrence associated with the use of a drug in humans" [17]. The FDA considers challenge agents as investigational new drugs [18], such that AEs in HCTs refer to any untoward medical occurrence associated with the challenge. AEs that result in death, hospitalization, disability, permanent damage, or other important medical events are reported as SAEs [17]. AEs graded "severe" by studies are distinct from SAEs in most cases, usually because they are not life-threatening or do not require hospitalization.
A systematic review was performed to characterize the frequency and nature of AEs and SAEs in HCTs related to the challenge and the risk mitigation measures used. The review also investigated the pathogens studied, the clinical outcomes in participants, study registration in databases, the number and uses of HCTs over time, and the quality of data reporting.

Search Strategy
A systematic review of records from 1980 to 2021 indexed in the PubMed and PubMed Central databases was performed to identify published articles describing HCTs. Articles published before 1980 were not assessed because the modern institutional review system was not in place until after the 1979 Belmont report. The initial search was preregistered on PROSPERO as CRD42021247218 [19], but it identified few studies published before 2010. Additional searches were performed to appropriately discover studies for each decade of interest, as detailed in the amended preregistration [19] and the Supplementary Methods. The database search strategy is presented in Table 1. Further manual searches of references lists and reviews were performed to identify additional articles describing HCTs that were missed.

Screening Process
Titles and abstracts of search results were manually screened by 3 authors working independently to identify articles that were eligible for full-text review. Case reports, reviews, articles not available in English, studies that did not meet the criteria for an HCT, and articles published before 1980 were excluded. Secondary reviews of 2 past reviews [2,20] were also performed to identify more articles that were missed by the searches. Articles that described studies that performed secondary analysis of results from previously conducted HCTs were excluded, but their reference lists were reviewed to identify the original publication of these results.

Full-text Review Process
The unit of analysis is the individual study, as described within a published article detailing results. Individual studies were identified by trial registration. If trial registration was not reported, studies were counted per the article description, or as a single study if participants were challenged with a single pathogen. If multiple articles were published discussing the same study, the earliest published article was included. In some cases, multiple articles were combined (see Supplementary Methods).
There is an ongoing discussion on the precise definition of an HCT [21]. In general, studies that had been completed and involved intentional exposure of human volunteers to a pathogen were included. Challenges with candidate vaccine viruses were also included, as were studies in which previously challenged participants were challenged again with the same pathogen. Consistent with Kalil et al, studies involving live, attenuated vaccines that were not followed by intentional infection, as well as data from phases of studies involving immunization or vaccination with live, attenuated vaccines or other methods that could have potentially resulted in infection, but that are not generally referred to as HCTs, were excluded [22].

Data Collection Process
At least 2 reviewers independently examined each publication selected for full-text review and any discrepancies were either reconciled or resolved by the senior author. Data collection was performed manually and results were input into a spreadsheet.

Data Extraction
The following numerical data were extracted from each study: year of article publication, size of cohort, sex breakdowns; mean or median age, standard deviation, and age range; number of participants challenged, number of challenged participants infected with pathogen, number of participants in control group (those who did not undergo a challenge), number of control participants infected with pathogen, number of control participants with at least 1 AE, and number of challenged participants with: (1) at least 1 AE, (b) at least 1 "severe" or "very severe" (grade 3 or higher) AE, (3) at least 1 SAE.
In addition, the following nonnumerical data were extracted from each study: clinical trial registration, pathogen assessed, definition of infection, definition of AEs, treatments administered to participants, risk mitigations taken, ethics committee and review board approvals reported, and a brief description of the study design.
For articles that reported separate study arms that were all exposed to a pathogen within a single pathogen category, data were summed across all arms to be treated as a single study. Data from rechallenges were extracted separately and treated as individual studies. No treatment effect measures were extracted.
AEs among challenged participants that were not related to challenge (such as AEs related to vaccination or drug treatment) were not extracted (see Supplementary Methods). For studies that did not define and/or report AEs, reported symptom data were extracted instead. For studies that did not define and/or report SAEs, reported symptom data that met the 2016 definition of SAEs provided by the FDA [17] based on reviewer judgment were extracted as SAEs. Figure 1 shows a Preferred Reporting Items for Systematic Reviews and Meta-Analyses flowchart of study selection. Searches yielded a total of 2654 results; 183 additional results were added by citation searching the reference lists of 2 past reviews [2,20] and articles identified among search results that used data from prior HCTs. One article [23] provided updated data for another [24]. Eleven results were not retrieved (5 with no full text available and 6 with unpublished data) and 47 duplicates were removed. No further efforts were made to identify unpublished or unidentified work. A total of 276 articles were included, describing 308 studies from which data were extracted. Excluded results were primarily reviews and articles discussing non-HCT clinical trials. See the Supplementary references for the complete reference list of included articles.

Results of Individual Studies
Data from 284 studies, with 14 628 challenged participants, were extracted (Table 2). Additional data were extracted from 24 rechallenge studies (Supplementary Tables 3, 4

Reported AEs and Unreported Data
Among 284 studies, 94 and 97 did not report any AE or SAE data, respectively (Table 3, Figure 2). The precise number of participants experiencing at least 1 SAE could not be extracted from 2 studies: 1 lost challenged subjects' records in a flooded storage facility [27] and the other did not provide any detail on the AEs observed [28]. ((((("human challenge") OR ("controlled human infection")) AND (trial OR vaccine OR model)) AND ((("adverse events") OR (medical* AND "significant event" OR "significant events")))) AND ("1980 Articles before 1990 PubMed 6 January 2021 (("human challenge") OR ("controlled human infection") OR ("experimental" AND "infection" AND "human*") OR ("wild-type virus" AND infection)) AND (trial OR vaccine OR model OR inoculat*) AND (("adverse events") OR (medical* AND "significant event" OR "significant events") OR (illness)) AND ( Among 10 325 challenged participants in studies that reported AEs, between 4317 (41.8%) and 5730 (55.5%) experienced at least 1 AE (Table 4). Among 5083 challenged participants in studies that graded severity of AEs, between 285 (5.6%) and 801 (15.8%) experienced at least 1 severe or very severe (grade 3 or higher) AE ( Table 5). The range in possible AE values is greater in more recent decades as a result of more studies reporting AEs by individual or symptom, rather than reporting the total number of participants with at least 1 AE. Nineteen studies included control (nonchallenged) participants (n = 433); only 2 of these studies reported AE data for control participants (n = 69). Between 7 (10.1%) and 12 (17.4%) control participants experienced at least 1 AE.
Among 10 016 challenged participants in studies that reported SAEs, 23 (0.2%) experienced at least 1 SAE (Table 6). Among 146 rechallenged participants in studies that reported SAEs, 1 additional participant (0.7%) experienced at least 1 SAE (Supplementary Table 6). No fatalities were reported. SAEs are described in more detail in Table 7, and some SAEs deemed not related to challenge are discussed further in Supplementary  Table 7.

Studies by Pathogen
The numbers of studies and participants challenged within each category of pathogen are presented in Table 8, and Figure 3A illustrates studies of different pathogens have occurred over time. There were 28 pathogen categories, with the most commonly studied being Plasmodium spp (73 studies, 1689 participants), influenza viruses (45 studies, 3536 participants), and rhinovirus (43 studies, 4332 participants). Studies investigating Plasmodium spp had the greatest number of challenged participants with SAEs, with 7 SAEs (of 23 in all nonrechallenge studies) occurring among 1129 participants in 52 studies. Studies investigating norovirus had the greatest proportion of SAEs to number challenged, with 4 SAEs occurring among 163 participants in 3 studies.

Reporting AEs and Use of Trial Registries Over Time
Overall, the number of challenge studies has been increasing each decade ( Figure 3B). Before the 2000s, many studies did not report AEs, but instead reported comparable symptom data. These were extracted as AEs. Of the 283 included studies, 123 explicitly mentioned or defined AEs, but not all reported them for the challenge phase specifically. The proportion of studies with definitions has increased over time, from only 19.4%, 23.9%, and 21.1% in the 1980s, 1990s, and 2000s,   [29]. However,  publication year lags year of registration, so it is unclear how much of the lack of registration is noncompliance and how much is delayed publication. Still, only 5.3% of included studies published in the 2000s were registered in at least 1 registry; 76.4% of included studies published in the 2010s were registered in at least 1 registry (Figure 2). Every included study published so far this decade was registered ( Figure 2).

Risk Mitigation
Text describing specific risk mitigation measures was found in 286 of the 308 studies, which is included in the dataset [26], and a descriptive summary follows. The qualitative nature of these mitigation descriptions precluded meaningful quantitative analysis. Risk mitigation measures typically include evaluating participants' risk of disease if exposed to a challenge agent by using medical screening and assessing participants' medical histories. In some cases, checking for previous exposure to the pathogen was a risk mitigation strategy, but it could also be done for other reasons. Demographic criteria, pregnancy screening, assessment of cardiac risk, and assessment of weight and/or body mass index were often used to evaluate risk.
Some studies reported mitigation strategies for risks to nonparticipants, such as isolation throughout the duration of the study, requiring birth control, or excluding participants with employment posing risk of spread (for example, excluding food handlers in HCTs investigating Escherichia coli, norovirus, and Salmonella spp). Validity of informed consent was sometimes assessed by testing participants' understanding of the study protocol.

DISCUSSION
The present review found a total of 24 (23 reported in traditional challenges, 1 in a rechallenge) SAEs and 0 reported deaths or cases of permanent damage among 15 046 participants in 308 studies spanning 1980 to 2021. It is unlikely that any SAEs captured in this review (Table 7) were life-threatening because the events were primarily brief hospitalizations for observation or supportive care requiring noninvasive interventions or falling under the broad category of "other serious (important medical events)" in the FDA definition of SAEs. The proportions of studies that define AEs and mention SAEs have increased over time, although inconsistent definitions make it challenging to compare reported data, particularly across studies investigating different pathogens. Unfortunately, the proportions of studies that do not report AE and SAE data related to challenges remained unacceptably high in the 2010s at 24.5% and 30.2%, respectively (Table 3). Although a high rate of failing to report SAEs may be indicative of their rarity in the HCT setting, clearer reporting would allow for better understanding of the risks and benefits of HCTs.
Issues surrounding AE reporting in clinical trials are not exclusive to HCTs [30]. However, confusion related to reporting challenge-related AEs is an issue specific to HCTs. For example,   One additional SAE from a rechallenge is described in Table 7 but not included in this total.  [17], but many studies, especially those published before 2000, did not define or report SAEs. Guidelines  for HCT reporting have been suggested [22] but have not yet been adopted. Accordingly, a major conclusion of this review is that in addition to a greater effort to standardize AE reporting in general, which others have postulated [30], these standardization efforts are particularly valuable to HCTs. The number of new HCTs has been increasing; however, it is unclear whether this increase is proportional to the general growth trend in the number of new (non-HCT) clinical trials. Since 2010, pathogens such as Bordetella pertussis, Schistosoma mansoni, and Streptococcus pneumoniae have been studied in HCTs for the first time. Figure 3A shows that the number of influenza and rhinovirus HCTs has declined somewhat over time, following the discontinuation of several research programs focused on common cold, whereas the number of Plasmodium spp HCTs sharply increased in the 2010s. These trends demonstrate that HCTs are an increasingly ubiquitous tool and that their relative speed allows researchers to investigate new pathogens of interest more rapidly than in traditional clinical trials.
Limitations of this review are primarily related to uncertainties around the accuracy of AE reporting. This includes potential bias in AE reporting, inconsistent reporting, and difficulty in precisely estimating the rates of events based on provided data. Many studies reported either no or unclear AE and/or SAE data, and issues of censoring and misclassification are common with respect to AE reporting in general [31]. To partially address issues with different standards for reporting over time, we extracted symptom data as AE and/or SAE data from studies that did not mention or define AEs/SAEs, but this means that AEs for decades in which these studies occurred are not fully comparable. The review is further limited by our inability to locate some results, including published HCTs that were not on PubMed [32] and HCTs whose results have only been published as case reports [33]. These limitations further highlight the need for improvements in the field of HCTs with respect to AE reporting and availability of results. Future work building off of this review includes policy recommendations around the issues of standardization and AE reporting, investigating the registration of HCTs in databases, and further qualitative analysis of risk mitigation measures in published articles.

CONCLUSIONS
The recent literature contains hundreds of HCTs involving more than 10 000 participants and only 24 SAEs. With the qualification that systematic AE reporting in many studies has been incomplete, reports of severe symptoms and SAEs related to infectious challenge in HCTs are notably infrequent. Specifically, participation in an HCT has not been associated with permanent impairment or death. HCTs are now routinely used to understand infectious dose, disease progression, clinical efficacy of novel interventions, and immune response for a wide variety of pathogens. As evidenced by recent HCTs for coronavirus disease 2019, they may be conducted for novel as well as familiar diseases. This review can help support public discussion and expert deliberation regarding the safety of HCTs. It may also inform future discussions among HCT researchers and members of ethics review committees regarding the planning, conduct, and reporting of future HCTs.
Preregistration, Protocol, and Conflict of Interest Disclosures The review was preregistered on PROSPERO as CRD42021247218, risk outcomes and risk mitigation measures in human challenge trials: a systematic review. As mentioned previously, the preregistration was amended to include additional searches and data. The review protocol is available online as Supplementary Material-Protocol.