Abstract

Background

Phase 2 trials are instrumental for designing definitive efficacy trials or attaining accelerated approval. However, high attrition of drug candidates in phase 2 trials raises questions about their supporting evidence.

Methods

We developed a typology of supporting evidence for phase 2 cancer trials. We also devised a scheme for capturing elements that enable an assessment of the strength of such evidence. Using this framework, we content analyzed supporting evidence provided in protocols of 50 randomly sampled phase 2 cancer monotherapy trials between January 2014 and January 2019, available on ClinicalTrials.gov.

Results

Of the 50 protocols in our sample, 52% were industry funded. Most invoked supporting evidence deriving from trials against different cancers (n = 28, 56%) or preclinical studies (n = 48, 96%) but not from clinical studies involving the target drug-indication pairing (n = 23, 46%). When presenting evidence from models, only 1 (2%) protocol explained its translational relevance. Instead, protocols implied translatability by describing molecular (86%) and pathophysiological (84%) processes shared by model and target systems. Protocols often provided information for assessing the magnitude, precision, and risk of bias for supporting trials (n = 43; 93%, 91%, 47%, respectively). However, such information was often unavailable for preclinical studies (n = 49; 53%, 22%, 59%, respectively).

Conclusions

Supporting evidence is key to justifying the commitment of scientific resources and patients to a clinical hypothesis. Protocols often omit elements that would enable critical assessment of supporting evidence for phase 2 monotherapy cancer trials. These gaps suggest the promise of more structured approaches for presenting supporting evidence.

Introduction

New drugs are typically developed through a series of clinical trials, beginning with safety studies in phase 1 and advancing to regulatory approval through phase 3 randomized trials. In between, phase 2 trials are often used to gather evidence of efficacy using surrogate endpoints. Phase 2 trials thus bridge a drug’s promise established in preclinical studies to definitive trials establishing efficacy.

Phase 2 trials account for the largest proportion of trials testing anticancer interventions.1 However, 90% of drugs reaching this stage of testing never advance to approval, and only 30% of oncology drugs advance from phase 2 to phase 3.2-4 Phase 2 trials absorb substantial scientific resources, and in the aggregate, they expose large numbers of patients to interventions that will not prove safe and effective. Evidence supporting phase 2 clinical hypotheses is key to justifying this commitment of scientific resources and patient welfare.

Major human protection policies urge an evidence-based, comprehensive, and systematic evaluation of risk and benefit.5,6 However, neither drug regulators nor human protection policies provide guidance on accomplishing this.7,8 This places the onus of assessing risk and benefit on ethics and/or scientific review committees like those that are mandated by the National Cancer Institute.9,10 Surveys suggest that ethics committees find this task daunting.11,12

The challenges of reviewing supporting evidence in early phase trials are compounded by several factors. First, much of the evidence backing clinical hypotheses in early phase trials derives from a variety of sources, including in vitro studies, preclinical experiments, and trials. There are no obvious ways of aggregating such evidence. Second, cancer reproducibility studies13,14 and other reports15,16 suggest that many basic and preclinical studies do not withstand replication. Third, some types of supporting evidence, like case reports or phase 1 trials, involve small samples and high risk of bias.

Regardless of how reviewers interpret or synthesize studies, protocols should summarize supporting evidence so that reviewers can critically assess the strength of a clinical hypothesis. We report the results of a content analysis of evidence presented in phase 2 monotherapy cancer trial protocols.

Methods

Objectives

We performed a deductive content analysis of phase 2 prelicense cancer trial protocols using coding categories and norms articulated in evidence-based medicine.17-19 Our aim was not to assess substantively the strength of evidence but rather to assess the information available for evaluating the strength of evidence for the trial’s clinical hypothesis. Our primary objective was to categorize and quantify the types of supporting evidence presented in protocols.

Sample creation

We identified protocols of all adult solid tumor phase 2 trials registered on ClinicalTrials.gov with a start date from January 1, 2014, to December 31, 2018, inclusive (search conducted March 10, 2023; Supplementary Methods). The 2018 cutoff was selected to allow 5 years for sponsors to deposit protocols, based on the median times required for trial completion.20 We used simple randomization to select 50 protocols, which we considered sufficient for the descriptive aims of the present study. We included trials (1) evaluating a regulated drug or biologic as systemic monotherapy; (2) with at least 1 site in the United States or Canada; (3) only enrolling participants aged older than 18 years; and (4) had an actual enrollment of more than 10. We excluded trials (1) without an efficacy primary endpoint; (2) evaluating multiple indications; and (3) evaluating a drug-indication pairing already approved by the US Food and Drug Administration or tested in a completed phase 3 trial. Finally, we excluded trials if the supporting evidence within their protocols was redacted or entirely omitted.

Protocols were assessed for eligibility and partitioned into statements by 1 author (SB). Protocol sections titled “Introduction,” “Background,” and “Rationale” were selected for partitioning into coding units (statements). Trial information was downloaded from ClinicalTrials.gov. Drug class, indication, and biomarker eligibility were independently double extracted by 3 authors (SB, KC, AN) using Numbat Systematic Review Manager.21

Coding framework

At the outset of our study, we used several sources18,22-24 to derive a typology of the information needed for assessing the strength of evidence supporting a study drug’s ultimate aim, which we took to be extending the survival of patients like those in the proposed trial (ie, the target population). First, we reasoned that, by definition, evidence directly measuring the study drug’s effect in the target population (ie, evidence from the target system) would be limited. Instead, claims about an intervention’s promise would be supported with evidence from 3 models: trials of the same drug in different indications, preclinical and in vitro studies, and clinical or preclinical studies involving different but related drugs. Second, we reasoned that using such model evidence to support a trial would require evidence for the translatability to the target system (eg, why outcomes of a trial in a different disease might predict outcomes in the target system). Third, we reasoned that evidence supporting a trial—whether from other trials or preclinical studies—might address 1 of 3 mechanistic steps: molecular target engagement, physiological responses (eg, tumor shrinkage), and clinical responses (eg, survival). Ultimately, a reviewer would be tasked with assessing the prospects of a drug’s action through these mechanistic steps to a clinical benefit. Last, we reasoned that assessing evidence would require information on its strength. For this, we sought the following 3 attributes: the magnitude of effect sizes, the precision of estimates, and the risk of bias for supporting studies. For a complete list of terms, refer to the glossary in Table 1.25

Table 1.

Glossary of concepts and terms included in the content analysis

TermDefinitionExample(s)
Drug activity claimStatement describing the results of a study or group of studies suggesting the study drug engages its target or has anticancer or clinical activity“[Drug] significantly reduced tumor growth compared with the vehicle control (87%, P < .01) and achieved a 56% response rate.”
Translational claimStatement supporting the generalizability of models and outcomes to target system“Abnormalities of genes involving [target] signaling pathways have been described in [indication]”
Target populationGroup of individuals with the same cancer as enrolled in the proposed trialPatients with malignant glioma
Target systemThe ultimate clinical application of the drug should the phase 2 and subsequent phase 3 trial vindicate it (evidence in protocols, like a phase 1 trial of the same drug in the target population, might sample the target system)[Drug] used in the setting of [target population] to extend survival
Model systemAny other experimental model used to support efficacy in the target population, including patients with other cancer indications or animal modelsPatients with colorectal cancer; immunodeficient mice implanted with glioma xenografts
Molecular outcomesOutcomes measuring biochemical changes at the level of biomarkers or molecular targetsTarget engagement, enzyme inhibition, gene expression, activation of signal transduction
Physiological outcomesOutcomes measuring changes that occur at the level of groups of cells, tissues, and/or organsProgression-free survival, tumor response, antitumor activity, antiproliferative activity
Clinical outcomesOutcomes measuring changes at the level of the entire organismOverall survival, clinical improvement, symptom improvement
TermDefinitionExample(s)
Drug activity claimStatement describing the results of a study or group of studies suggesting the study drug engages its target or has anticancer or clinical activity“[Drug] significantly reduced tumor growth compared with the vehicle control (87%, P < .01) and achieved a 56% response rate.”
Translational claimStatement supporting the generalizability of models and outcomes to target system“Abnormalities of genes involving [target] signaling pathways have been described in [indication]”
Target populationGroup of individuals with the same cancer as enrolled in the proposed trialPatients with malignant glioma
Target systemThe ultimate clinical application of the drug should the phase 2 and subsequent phase 3 trial vindicate it (evidence in protocols, like a phase 1 trial of the same drug in the target population, might sample the target system)[Drug] used in the setting of [target population] to extend survival
Model systemAny other experimental model used to support efficacy in the target population, including patients with other cancer indications or animal modelsPatients with colorectal cancer; immunodeficient mice implanted with glioma xenografts
Molecular outcomesOutcomes measuring biochemical changes at the level of biomarkers or molecular targetsTarget engagement, enzyme inhibition, gene expression, activation of signal transduction
Physiological outcomesOutcomes measuring changes that occur at the level of groups of cells, tissues, and/or organsProgression-free survival, tumor response, antitumor activity, antiproliferative activity
Clinical outcomesOutcomes measuring changes at the level of the entire organismOverall survival, clinical improvement, symptom improvement
Table 1.

Glossary of concepts and terms included in the content analysis

TermDefinitionExample(s)
Drug activity claimStatement describing the results of a study or group of studies suggesting the study drug engages its target or has anticancer or clinical activity“[Drug] significantly reduced tumor growth compared with the vehicle control (87%, P < .01) and achieved a 56% response rate.”
Translational claimStatement supporting the generalizability of models and outcomes to target system“Abnormalities of genes involving [target] signaling pathways have been described in [indication]”
Target populationGroup of individuals with the same cancer as enrolled in the proposed trialPatients with malignant glioma
Target systemThe ultimate clinical application of the drug should the phase 2 and subsequent phase 3 trial vindicate it (evidence in protocols, like a phase 1 trial of the same drug in the target population, might sample the target system)[Drug] used in the setting of [target population] to extend survival
Model systemAny other experimental model used to support efficacy in the target population, including patients with other cancer indications or animal modelsPatients with colorectal cancer; immunodeficient mice implanted with glioma xenografts
Molecular outcomesOutcomes measuring biochemical changes at the level of biomarkers or molecular targetsTarget engagement, enzyme inhibition, gene expression, activation of signal transduction
Physiological outcomesOutcomes measuring changes that occur at the level of groups of cells, tissues, and/or organsProgression-free survival, tumor response, antitumor activity, antiproliferative activity
Clinical outcomesOutcomes measuring changes at the level of the entire organismOverall survival, clinical improvement, symptom improvement
TermDefinitionExample(s)
Drug activity claimStatement describing the results of a study or group of studies suggesting the study drug engages its target or has anticancer or clinical activity“[Drug] significantly reduced tumor growth compared with the vehicle control (87%, P < .01) and achieved a 56% response rate.”
Translational claimStatement supporting the generalizability of models and outcomes to target system“Abnormalities of genes involving [target] signaling pathways have been described in [indication]”
Target populationGroup of individuals with the same cancer as enrolled in the proposed trialPatients with malignant glioma
Target systemThe ultimate clinical application of the drug should the phase 2 and subsequent phase 3 trial vindicate it (evidence in protocols, like a phase 1 trial of the same drug in the target population, might sample the target system)[Drug] used in the setting of [target population] to extend survival
Model systemAny other experimental model used to support efficacy in the target population, including patients with other cancer indications or animal modelsPatients with colorectal cancer; immunodeficient mice implanted with glioma xenografts
Molecular outcomesOutcomes measuring biochemical changes at the level of biomarkers or molecular targetsTarget engagement, enzyme inhibition, gene expression, activation of signal transduction
Physiological outcomesOutcomes measuring changes that occur at the level of groups of cells, tissues, and/or organsProgression-free survival, tumor response, antitumor activity, antiproliferative activity
Clinical outcomesOutcomes measuring changes at the level of the entire organismOverall survival, clinical improvement, symptom improvement

Extraction

We developed an a priori extraction framework (Supplementary Methods) from the above typology. All statements were coded for the presence of attributes described above. We used lenient coding criteria. For example, a protocol was classified as including magnitude information about a drug’s effect if it included at least 1 statement that reported magnitude, even if most other such statements lacked magnitude information. Similarly, we accepted any reporting of sample size, range, confidence intervals, or statistical significance as providing information about precision. Each statement was independently double coded by 3 authors (SB, KC, AN) using Numbat. Disagreements were resolved by discussions with co-authors.

Analysis

Our primary outcome was descriptive and included the proportion of protocols reporting different types of supporting evidence. Model statements included all statements that used a different drug and/or evaluated response in a nontarget population.

Our secondary outcomes estimated the proportion of protocols and statements that are presented with information important for assessing the strength of evidence for claims. We preregistered the study protocol on Open Science Framework (see https://osf.io/6zye8). We also performed post hoc analyses relating supporting evidence to trial outcomes; these are reported in our Supplementary Materials.

Results

Characteristics of protocol sample

Of our sample of 50 trial protocols, 26 (52%) were industry funded, and 24 (48%) were funded by a nonindustry organization. Most (n = 35; 70%) trials involved a targeted agent, 12 (24%) tested an immunotherapy, and 3 (6%) tested a cytotoxic therapy (Table 2). The search process is summarized in the Preferred Reporting Items for Systematic Review and Meta-Analyses flow diagram (Figure S1). Individual study characteristics are presented in Table S1.

Table 2.

Characteristics of solid tumor monotherapy phase 2 trials

Sample characteristicsNo. (%) of trials
Biomarker enriched9 (18)
Drug US Food and Drug Administration approved in a different indication16 (32)
Funder type
 Nonindustry24 (48)
 Industry26 (52)
Drug class
 Targeted therapy35 (70)
 Immunotherapy12 (24)
 Cytotoxic therapy3 (6)
Primary outcome measure(s)
 Response rate33 (66)
 Progression-free survival14 (28)
 Overall survival1 (2)
 Other3 (6)
Indication
 Neuroendocrine and adrenal tumors6 (12)
 Non–small cell lung cancer6 (12)
 Bladder cancer4 (8)
 Hepatobiliary cancer4 (8)
 Prostate cancer4 (8)
 Othera26 (52)
Status
 Completed30 (60)
 Active, not recruiting12 (24)
 Terminated8 (16)
Median enrollment (IQR)34 (18-76)
Sample characteristicsNo. (%) of trials
Biomarker enriched9 (18)
Drug US Food and Drug Administration approved in a different indication16 (32)
Funder type
 Nonindustry24 (48)
 Industry26 (52)
Drug class
 Targeted therapy35 (70)
 Immunotherapy12 (24)
 Cytotoxic therapy3 (6)
Primary outcome measure(s)
 Response rate33 (66)
 Progression-free survival14 (28)
 Overall survival1 (2)
 Other3 (6)
Indication
 Neuroendocrine and adrenal tumors6 (12)
 Non–small cell lung cancer6 (12)
 Bladder cancer4 (8)
 Hepatobiliary cancer4 (8)
 Prostate cancer4 (8)
 Othera26 (52)
Status
 Completed30 (60)
 Active, not recruiting12 (24)
 Terminated8 (16)
Median enrollment (IQR)34 (18-76)
a

Includes breast cancer, colorectal cancer, esophageal cancers, ovarian cancer, central nervous system cancer, pancreatic adenocarcinoma, soft tissue sarcoma, basal cell skin cancer, cervical cancer, kidney cancer, pleural mesothelioma, small cell lung cancer, thymomas and thymic carcinomas, uterine neoplasms, and uveal melanoma.

Table 2.

Characteristics of solid tumor monotherapy phase 2 trials

Sample characteristicsNo. (%) of trials
Biomarker enriched9 (18)
Drug US Food and Drug Administration approved in a different indication16 (32)
Funder type
 Nonindustry24 (48)
 Industry26 (52)
Drug class
 Targeted therapy35 (70)
 Immunotherapy12 (24)
 Cytotoxic therapy3 (6)
Primary outcome measure(s)
 Response rate33 (66)
 Progression-free survival14 (28)
 Overall survival1 (2)
 Other3 (6)
Indication
 Neuroendocrine and adrenal tumors6 (12)
 Non–small cell lung cancer6 (12)
 Bladder cancer4 (8)
 Hepatobiliary cancer4 (8)
 Prostate cancer4 (8)
 Othera26 (52)
Status
 Completed30 (60)
 Active, not recruiting12 (24)
 Terminated8 (16)
Median enrollment (IQR)34 (18-76)
Sample characteristicsNo. (%) of trials
Biomarker enriched9 (18)
Drug US Food and Drug Administration approved in a different indication16 (32)
Funder type
 Nonindustry24 (48)
 Industry26 (52)
Drug class
 Targeted therapy35 (70)
 Immunotherapy12 (24)
 Cytotoxic therapy3 (6)
Primary outcome measure(s)
 Response rate33 (66)
 Progression-free survival14 (28)
 Overall survival1 (2)
 Other3 (6)
Indication
 Neuroendocrine and adrenal tumors6 (12)
 Non–small cell lung cancer6 (12)
 Bladder cancer4 (8)
 Hepatobiliary cancer4 (8)
 Prostate cancer4 (8)
 Othera26 (52)
Status
 Completed30 (60)
 Active, not recruiting12 (24)
 Terminated8 (16)
Median enrollment (IQR)34 (18-76)
a

Includes breast cancer, colorectal cancer, esophageal cancers, ovarian cancer, central nervous system cancer, pancreatic adenocarcinoma, soft tissue sarcoma, basal cell skin cancer, cervical cancer, kidney cancer, pleural mesothelioma, small cell lung cancer, thymomas and thymic carcinomas, uterine neoplasms, and uveal melanoma.

Evidence suggesting drug activity

A total of 904 supporting evidence statements were identified in our protocol sample. The median number of supporting statements in protocols was 17 (IQR = 11-23). A total of 832 (92%) statements had sufficient information for extraction using our coding framework. Of these, 50 (6%) provided direct evidence of the drug’s effect in the target system (eg, phase 1 trial results), and 529 (64%) described the drug’s effect in a model system; 253 (30%) statements addressed the translatability of model system findings.

The most common model systems were in vitro experiments (160 statements, 30%) and animal studies (122 statements, 23%) followed by trials of the study drug in patients with nontarget indications (111 statements, 21%). Other model evidence included trials of different drugs in the same class in the target population (n = 51; 10%), trials of a same class drug in nontarget populations (n = 16; 3%), and preclinical studies of same class drugs (n = 69; 13%). The most common outcomes provided were physiological (374 statements, 65%) and molecular target (157; 27%) outcomes. A total of 48 (8%) statements addressed clinical outcomes.

Table 3 shows the frequency with which trial protocols contained at least 1 statement involving various types of supporting evidence. A total of 23 (46%) protocols presented direct evidence from the target systems; 49 (98%) presented evidence from model systems. For the latter, 28 (56%) protocols offered supporting evidence deriving from clinical trials of the same drug in nontarget indications, and 48 (96%) protocols offered supporting evidence from preclinical studies. Nearly all protocols offered supporting evidence with molecular (n = 48; 96%) and physiological (n = 50; 100%) outcomes, and half (n = 25; 50%) offered supporting evidence for a clinical outcome. Only 1 (2%) protocol explained the relevance of a model system, and 48 (96%) protocols included at least some description of processes implied to be present in both model and target systems.

Table 3.

Supporting evidence provided in phase 2 cancer trial protocols

Evidence typeCausal stepNo. (%) protocols with at least 1 statementNo. (%) protocols with at least 1 statement
Mean percentage of statements in protocols
MagnitudePrecisionRisk of biasMagnitudePrecisionRisk of bias
Target system evidenceMolecular2 (4)0 (0)0 (0)0 (0)000
Physiological23 (46)22 (96)21 (91)5 (22)868213
Clinical5 (10)2 (40)3 (60)2 (40)406040
Total23 (46)22 (96)21 (91)6 (26)787515
Model system evidence
 ClinicalMolecular8 (16)6 (75)2 (25)3 (38)622533
Physiological35 (70)30 (86)29 (83)15 (43)665927
Clinical17 (34)14 (82)14 (82)12 (71)687757
Total35 (70)31 (89)29 (83)17 (49)655928
 PreclinicalMolecular47 (94)17 (36)5 (11)15 (32)17413
Physiological43 (86)18 (42)10 (23)22 (51)20827
Clinical6 (12)2 (33)2 (33)3 (50)252542
Total49 (98)26 (53)11 (22)29 (59)19720
Evidence typeCausal stepNo. (%) protocols with at least 1 statementNo. (%) protocols with at least 1 statement
Mean percentage of statements in protocols
MagnitudePrecisionRisk of biasMagnitudePrecisionRisk of bias
Target system evidenceMolecular2 (4)0 (0)0 (0)0 (0)000
Physiological23 (46)22 (96)21 (91)5 (22)868213
Clinical5 (10)2 (40)3 (60)2 (40)406040
Total23 (46)22 (96)21 (91)6 (26)787515
Model system evidence
 ClinicalMolecular8 (16)6 (75)2 (25)3 (38)622533
Physiological35 (70)30 (86)29 (83)15 (43)665927
Clinical17 (34)14 (82)14 (82)12 (71)687757
Total35 (70)31 (89)29 (83)17 (49)655928
 PreclinicalMolecular47 (94)17 (36)5 (11)15 (32)17413
Physiological43 (86)18 (42)10 (23)22 (51)20827
Clinical6 (12)2 (33)2 (33)3 (50)252542
Total49 (98)26 (53)11 (22)29 (59)19720
Table 3.

Supporting evidence provided in phase 2 cancer trial protocols

Evidence typeCausal stepNo. (%) protocols with at least 1 statementNo. (%) protocols with at least 1 statement
Mean percentage of statements in protocols
MagnitudePrecisionRisk of biasMagnitudePrecisionRisk of bias
Target system evidenceMolecular2 (4)0 (0)0 (0)0 (0)000
Physiological23 (46)22 (96)21 (91)5 (22)868213
Clinical5 (10)2 (40)3 (60)2 (40)406040
Total23 (46)22 (96)21 (91)6 (26)787515
Model system evidence
 ClinicalMolecular8 (16)6 (75)2 (25)3 (38)622533
Physiological35 (70)30 (86)29 (83)15 (43)665927
Clinical17 (34)14 (82)14 (82)12 (71)687757
Total35 (70)31 (89)29 (83)17 (49)655928
 PreclinicalMolecular47 (94)17 (36)5 (11)15 (32)17413
Physiological43 (86)18 (42)10 (23)22 (51)20827
Clinical6 (12)2 (33)2 (33)3 (50)252542
Total49 (98)26 (53)11 (22)29 (59)19720
Evidence typeCausal stepNo. (%) protocols with at least 1 statementNo. (%) protocols with at least 1 statement
Mean percentage of statements in protocols
MagnitudePrecisionRisk of biasMagnitudePrecisionRisk of bias
Target system evidenceMolecular2 (4)0 (0)0 (0)0 (0)000
Physiological23 (46)22 (96)21 (91)5 (22)868213
Clinical5 (10)2 (40)3 (60)2 (40)406040
Total23 (46)22 (96)21 (91)6 (26)787515
Model system evidence
 ClinicalMolecular8 (16)6 (75)2 (25)3 (38)622533
Physiological35 (70)30 (86)29 (83)15 (43)665927
Clinical17 (34)14 (82)14 (82)12 (71)687757
Total35 (70)31 (89)29 (83)17 (49)655928
 PreclinicalMolecular47 (94)17 (36)5 (11)15 (32)17413
Physiological43 (86)18 (42)10 (23)22 (51)20827
Clinical6 (12)2 (33)2 (33)3 (50)252542
Total49 (98)26 (53)11 (22)29 (59)19720

Information on strength of supporting evidence

Supporting evidence statements in protocols often lacked information for assessing the strength of claims. When protocols invoked trial evidence, on average, 71% of statements provided some information on magnitude, 67% provided some information about the precision of treatment effect estimates, and 25% provided some information about risk of bias. When protocols invoked preclinical evidence, on average 19% of statements provided information on magnitude, 7% on precision, and 20% on risk of bias.

At the level of protocols, most providing support from trials (n = 43) included at least 1 statement where information was provided about the magnitude of anticancer effects (n = 40; 93%) or precision around that estimate (n = 39; 91%). However, risk of bias information was included at least once in only 47% of such protocols. Protocols invoking evidence from preclinical studies less frequently provided any information on magnitude (n = 26; 53%) or precision (n = 11; 22%). A total of 29 (59%) protocols contained at least 1 statement that enabled an assessment of risk of bias for preclinical studies. In a post hoc exploratory analysis, we selected 7 key supporting evidence characteristics to assess whether positive trials were more likely to exhibit them (Table S2). None of the 7 characteristics showed a statistically significant association with positive trial results. We also did not observe statistically significant differences in evidence presentation for industry- vs nonindustry-supported trials.

Information on the translatability of model systems

Only 1 (2%) protocol explained why outcomes of administering the study drug to patients with different indications might predict efficacy in the target population. No protocols explained why particular animal models were chosen or how well such models predict clinical response. Instead, protocols implied translatability with descriptions of molecular (86%) or pathophysiological (84%) processes that might be present in both target and model systems.

Protocols rarely explained the relationships between surrogate outcomes used in animal models or nontarget indication trials and clinical outcomes. Only 14 (28%) protocols provided evidence that response biomarkers (eg, microvessel density) predicted clinical outcomes like survival. Only 1 (2%) protocol presented evidence of correlation between surrogate outcomes (eg, progression-free survival) and survival within the target population.

Protocols generally provided at least some information about the robustness of supporting evidence (ie, external validity). Specifically, 77% of the protocols reporting clinical studies described at least 1 replication in different patients. Likewise, 86% of the protocols referencing preclinical evidence described at least 1 replication. Robustness was also often conveyed by describing anticancer activity in clinical and preclinical systems. Thus, 42 (84%) protocols described at least 1 clinical and 1 preclinical efficacy study each. Only 1 (2%) protocol relied solely on clinical evidence, and another 7 (14%) relied solely on preclinical studies.

A reference to at least 1 published trial was present for 33 of 43 (77%) protocols invoking support from clinical trials and for 35 of 49 (71%) protocols invoking support from preclinical studies. Most protocols (n = 44; 92%) providing translational evidence also contained at least 1 citation to the reported study.

Discussion

Phase 2 trial protocols often lack information that physicians, scientific review committees, or research ethics boards would need to assess support for a clinical hypothesis. First, protocols often lacked information about the strength of evidence for certain claims. On average, only one-fifth of the preclinical evidence in protocols described effect sizes, and fewer (7%) provided information about precision. Instead, studies are often described in qualitative terms like “potent antitumor activity.”26 Second, only 2% of protocols described evidence connecting the impact of a surrogate (eg, tumor shrinkage) on a clinical endpoint. Numerous studies relating surrogate to clinical outcomes suggest that such relationships cannot always be assumed.27-32 Third, protocols rarely explained the translational relevance of model studies. The National Institutes of Health identified justifying the choice of models as a key recommendation for improving translatability in preclinical research; this was largely absent in our sample.24 Only 1 protocol provided information explicitly identifying shared features between target and model populations. The absence of such evidence is particularly problematic for the protocols (more than half of our sample) that relied solely on model systems to substantiate clinical hypotheses. Consequently, reviewers assessing such protocols must draw on outside knowledge about the predictive validity of models. Unless reviewers have deep content knowledge about the subject of a protocol, this may represent a big ask.

Our study aimed at describing the types of information available to reviewers, not weaknesses in evidence per se. Nevertheless, our analysis suggests variability in the strength of evidence supporting phase 2 trials. For example, 14% of protocols did not describe any replications of preclinical studies, leaving such trials vulnerable to reproducibility threats. Four phase 2 trials (8% of our sample) did not cite basic science studies establishing the mechanism of the drug, leaving reviewers to use their own knowledge about the pathophysiological premises of a trial. Though all drugs in our sample had reached phase 2 testing, more than half (54%) of protocols did not present any phase 1 or case report evidence suggesting promise in target populations.

Human protections policies require that trials present a favorable balance of risks against benefits. Supporting evidence is key to establishing this. If there is insufficient evidence to support a clinical hypothesis, a trial is unlikely to deliver important scientific knowledge. Evidence supporting late phase trials can be synthesized using systematic review and/or meta-analysis. With phase 2 trials, however, synthesis of supporting evidence presents major challenges, given the variety of study methods. This greatly complicates scientific and ethical review of phase 2 trials.

Our analysis builds on studies suggesting gaps in supporting evidence for trials. One analysis of phase 2 trial publications had found that only 46% of the 179 publications cited a clinical study in the same indication as the trial.33 In our study, we observed a similar gap, with only 46% of trial protocols citing prior clinical studies for the same drug-indication pairing. Similar limitations have been identified in investigator brochures of early phase trials.34,35 More broadly, numerous studies have found that trial protocols often omit mention of prior trials of the same drug-indication pairing.36,37

Current guidelines for writing clinical trial protocols, like Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT), International Conference on Harmonization-Good Clinical Practice (ICH-GCP) guidelines, and the National Institutes of Health protocol template, primarily focus on the design elements of protocols, such as study objectives, rather than background sections.38-40 Their guidance on the background section asks for little more than a summary of previous findings. The Council for International Organizations of Medical Sciences (CIOMS) offers more explicit guidance.6 It urges sponsors to provide a “comprehensive and balanced overview of the available evidence that is relevant for evaluating the risks and potential individual benefits of the research.” Sponsors are asked to include “the nature, extent and relevance of animal studies and other preclinical and clinical studies” for the proposed trial and to clearly describe their results.

Our study has limitations. First, we assessed supporting evidence in protocols on the assumption that physicians, ethics committees, and scientific review bodies use such documents to evaluate trials. Investigator brochures may address some of the gaps reported here. Nevertheless, the authors’ own experience is that most review activities center on protocols, with investigator brochures primarily focusing on pharmacology and safety. Moreover, not all trials are accompanied by an investigator brochure. Second, for reasons of feasibility, our sample relied on protocols of trials from a recent 5-year interval available in ClinicalTrials.gov. Protocols in registration records may reflect revisions made after review committees approved a study. Third, our findings are sensitive to our coding categories and criteria. To the former, we created a priori categories of evidentiary statements that captured 3 causal steps in the pharmacology of a drug. The conceptual framework behind our categories underwent peer review before coding and is published.25 Regarding the latter, we erred on the side of coding information as present in protocols. A restrictive coding approach would offer a less sanguine picture of supporting evidence in protocols. Lastly, our coding scheme excluded statements about regulatory approvals, which might reasonably be interpreted as proxies for scientific support.

Future work will aim at extending these findings by probing whether supporting evidence correlates with positivity in phase 2 trials or whether supporting evidence reflects morally appropriate patterns (eg, stronger and more transparent evidence where interventions present greater risk). The results from the present study indicate that current practices for writing phase 2 cancer trial protocols omit information needed by reviewers to assess merit and ethics. Guidelines tailored to trial protocols could facilitate review by encouraging sponsors to disclose critical information like effect sizes and to state why they think outcomes in model systems have predictive value. As a starting point to this goal, we offer recommendations for writing protocols in Table 4. These recommendations are intended not as additional steps but rather as a replacement for existing procedures. Implementing such recommendations would better justify the commitment of patients and/or animals in studies supporting clinical trials. It would also promote a more efficient use of research resources by rewarding sponsors who present trials with a strong scientific rationale. Finally, the more explicit approach favored by our suggestions would better safeguard the interests of patients who participate in phase 2 trials.

Table 4.

Recommendations for describing supporting evidence in phase 2 cancer trial protocols

ChallengeEthical issueRecommendationExamples or approaches for implementation
Unstructured evidence in protocolsLack of structure makes identifying missing or weak information difficult.Protocols should structure supporting evidence based on the mechanistic step a study supports (eg, molecular, physiological, or clinical activity).Separate sections describing evidence that (1) the drug can access and engage its target, (2) target engagement produces impacts on efficacy surrogates, and (3) surrogate efficacy predicts clinical efficacy.
Absence of information on strength of evidenceAssessing support for a trial requires knowing something about the strength of evidence for supporting studies. Such information is often absent, especially for preclinical studies.All supporting studies invoked in protocols should provide information about effect size, precision, and measures for limiting risk of bias.“[Drug] inhibited tumor growth (P < .05) in the [target]-expressing cell line for all doses (2, 20, or 40 mg/kg) when compared with vehicle-treated tumors. The 40 mg/kg dose resulted in significant tumor regression (82%, P < .01) when compared with pre-dose tumor weights.”
Absence of information about the predictive value of modelsAssessing support from studies involving animal models or patients with other diseases requires the reviewer to understand their predictive value for the target system.Protocols should contain explicit statements addressing the predictive value of supporting evidence from animal models or trials of other indications.A systematic review demonstrating that a similar drug’s effect on selected models was predictive of an effect in humans, and/or a statement like “MB49 syngeneic murine model was selected for preclinical studies because their intact immune systems are critical for recapitulating the clinical properties of immune checkpoint inhibitors.”
Absence of information about a drug’s expected impact on clinical outcomesDrugs are ultimately developed not to affect surrogate measures but rather to impact clinical outcomes. Few protocols provide any evidence suggesting a drug can impact clinical outcomes.Protocols should include evidence suggesting a drug impacts clinical endpoints or that surrogate outcomes measured in animal models or trials predict clinical outcomes.“In a pooled landmark analysis of [target patients] in phase 3 trials, Williams et al. demonstrated that the estimated correlation between progression-free survival and overall survival was 0.85 (standard error 0.01).”
Absence of guidelines for reviewing supporting evidenceThe density, variety, and complexity of information presented in protocols presents a challenge for critical evaluation by review committees.Funding agencies and reviewers should consider developing guidelines and structured processes for evaluating supporting evidence in protocols.Such guidelines should capitalize on what is known about decision making41 by offering a structure that enables reviewers to parse and integrate evidence about discrete mechanistic steps and different experimental systems into an overall judgment about the promise of a treatment strategy.
ChallengeEthical issueRecommendationExamples or approaches for implementation
Unstructured evidence in protocolsLack of structure makes identifying missing or weak information difficult.Protocols should structure supporting evidence based on the mechanistic step a study supports (eg, molecular, physiological, or clinical activity).Separate sections describing evidence that (1) the drug can access and engage its target, (2) target engagement produces impacts on efficacy surrogates, and (3) surrogate efficacy predicts clinical efficacy.
Absence of information on strength of evidenceAssessing support for a trial requires knowing something about the strength of evidence for supporting studies. Such information is often absent, especially for preclinical studies.All supporting studies invoked in protocols should provide information about effect size, precision, and measures for limiting risk of bias.“[Drug] inhibited tumor growth (P < .05) in the [target]-expressing cell line for all doses (2, 20, or 40 mg/kg) when compared with vehicle-treated tumors. The 40 mg/kg dose resulted in significant tumor regression (82%, P < .01) when compared with pre-dose tumor weights.”
Absence of information about the predictive value of modelsAssessing support from studies involving animal models or patients with other diseases requires the reviewer to understand their predictive value for the target system.Protocols should contain explicit statements addressing the predictive value of supporting evidence from animal models or trials of other indications.A systematic review demonstrating that a similar drug’s effect on selected models was predictive of an effect in humans, and/or a statement like “MB49 syngeneic murine model was selected for preclinical studies because their intact immune systems are critical for recapitulating the clinical properties of immune checkpoint inhibitors.”
Absence of information about a drug’s expected impact on clinical outcomesDrugs are ultimately developed not to affect surrogate measures but rather to impact clinical outcomes. Few protocols provide any evidence suggesting a drug can impact clinical outcomes.Protocols should include evidence suggesting a drug impacts clinical endpoints or that surrogate outcomes measured in animal models or trials predict clinical outcomes.“In a pooled landmark analysis of [target patients] in phase 3 trials, Williams et al. demonstrated that the estimated correlation between progression-free survival and overall survival was 0.85 (standard error 0.01).”
Absence of guidelines for reviewing supporting evidenceThe density, variety, and complexity of information presented in protocols presents a challenge for critical evaluation by review committees.Funding agencies and reviewers should consider developing guidelines and structured processes for evaluating supporting evidence in protocols.Such guidelines should capitalize on what is known about decision making41 by offering a structure that enables reviewers to parse and integrate evidence about discrete mechanistic steps and different experimental systems into an overall judgment about the promise of a treatment strategy.
Table 4.

Recommendations for describing supporting evidence in phase 2 cancer trial protocols

ChallengeEthical issueRecommendationExamples or approaches for implementation
Unstructured evidence in protocolsLack of structure makes identifying missing or weak information difficult.Protocols should structure supporting evidence based on the mechanistic step a study supports (eg, molecular, physiological, or clinical activity).Separate sections describing evidence that (1) the drug can access and engage its target, (2) target engagement produces impacts on efficacy surrogates, and (3) surrogate efficacy predicts clinical efficacy.
Absence of information on strength of evidenceAssessing support for a trial requires knowing something about the strength of evidence for supporting studies. Such information is often absent, especially for preclinical studies.All supporting studies invoked in protocols should provide information about effect size, precision, and measures for limiting risk of bias.“[Drug] inhibited tumor growth (P < .05) in the [target]-expressing cell line for all doses (2, 20, or 40 mg/kg) when compared with vehicle-treated tumors. The 40 mg/kg dose resulted in significant tumor regression (82%, P < .01) when compared with pre-dose tumor weights.”
Absence of information about the predictive value of modelsAssessing support from studies involving animal models or patients with other diseases requires the reviewer to understand their predictive value for the target system.Protocols should contain explicit statements addressing the predictive value of supporting evidence from animal models or trials of other indications.A systematic review demonstrating that a similar drug’s effect on selected models was predictive of an effect in humans, and/or a statement like “MB49 syngeneic murine model was selected for preclinical studies because their intact immune systems are critical for recapitulating the clinical properties of immune checkpoint inhibitors.”
Absence of information about a drug’s expected impact on clinical outcomesDrugs are ultimately developed not to affect surrogate measures but rather to impact clinical outcomes. Few protocols provide any evidence suggesting a drug can impact clinical outcomes.Protocols should include evidence suggesting a drug impacts clinical endpoints or that surrogate outcomes measured in animal models or trials predict clinical outcomes.“In a pooled landmark analysis of [target patients] in phase 3 trials, Williams et al. demonstrated that the estimated correlation between progression-free survival and overall survival was 0.85 (standard error 0.01).”
Absence of guidelines for reviewing supporting evidenceThe density, variety, and complexity of information presented in protocols presents a challenge for critical evaluation by review committees.Funding agencies and reviewers should consider developing guidelines and structured processes for evaluating supporting evidence in protocols.Such guidelines should capitalize on what is known about decision making41 by offering a structure that enables reviewers to parse and integrate evidence about discrete mechanistic steps and different experimental systems into an overall judgment about the promise of a treatment strategy.
ChallengeEthical issueRecommendationExamples or approaches for implementation
Unstructured evidence in protocolsLack of structure makes identifying missing or weak information difficult.Protocols should structure supporting evidence based on the mechanistic step a study supports (eg, molecular, physiological, or clinical activity).Separate sections describing evidence that (1) the drug can access and engage its target, (2) target engagement produces impacts on efficacy surrogates, and (3) surrogate efficacy predicts clinical efficacy.
Absence of information on strength of evidenceAssessing support for a trial requires knowing something about the strength of evidence for supporting studies. Such information is often absent, especially for preclinical studies.All supporting studies invoked in protocols should provide information about effect size, precision, and measures for limiting risk of bias.“[Drug] inhibited tumor growth (P < .05) in the [target]-expressing cell line for all doses (2, 20, or 40 mg/kg) when compared with vehicle-treated tumors. The 40 mg/kg dose resulted in significant tumor regression (82%, P < .01) when compared with pre-dose tumor weights.”
Absence of information about the predictive value of modelsAssessing support from studies involving animal models or patients with other diseases requires the reviewer to understand their predictive value for the target system.Protocols should contain explicit statements addressing the predictive value of supporting evidence from animal models or trials of other indications.A systematic review demonstrating that a similar drug’s effect on selected models was predictive of an effect in humans, and/or a statement like “MB49 syngeneic murine model was selected for preclinical studies because their intact immune systems are critical for recapitulating the clinical properties of immune checkpoint inhibitors.”
Absence of information about a drug’s expected impact on clinical outcomesDrugs are ultimately developed not to affect surrogate measures but rather to impact clinical outcomes. Few protocols provide any evidence suggesting a drug can impact clinical outcomes.Protocols should include evidence suggesting a drug impacts clinical endpoints or that surrogate outcomes measured in animal models or trials predict clinical outcomes.“In a pooled landmark analysis of [target patients] in phase 3 trials, Williams et al. demonstrated that the estimated correlation between progression-free survival and overall survival was 0.85 (standard error 0.01).”
Absence of guidelines for reviewing supporting evidenceThe density, variety, and complexity of information presented in protocols presents a challenge for critical evaluation by review committees.Funding agencies and reviewers should consider developing guidelines and structured processes for evaluating supporting evidence in protocols.Such guidelines should capitalize on what is known about decision making41 by offering a structure that enables reviewers to parse and integrate evidence about discrete mechanistic steps and different experimental systems into an overall judgment about the promise of a treatment strategy.

Acknowledgments

The authors thank the Canadian Institutes of Health Research for funding this research. The funder had no role in the study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author contributions

Selin Bicer, MSc (Conceptualization; Data curation; Formal analysis; Investigation; Methodology; Writing – original draft; Writing – review & editing), Angela Nelson, BSc (Investigation; Writing – review & editing), Katerina Carayannis, BSc (Investigation; Writing – review & editing), and Jonathan Kimmelman, PhD (Conceptualization; Funding acquisition; Methodology; Project administration; Supervision; Writing – original draft; Writing – review & editing).

Supplementary material

Supplementary material is available at JNCI: Journal of the National Cancer Institute online.

Funding

This work was supported by the Canadian Institutes of Health Research (CIHR) (grant number PJT-175217).

Conflicts of interest

The authors have no conflicts of interest relevant to this article to disclose.

Data availability

The datasets generated and analyzed during the current study are available from the corresponding author upon reasonable request.

References

1

Gumnit
E
,
Feustel
AC
,
Wong
S
,
Bosan
R
,
Hutchinson
N
,
Kimmelman
J.
The proportion of North American cancer trials that evaluate novel targets
.
Invest New Drugs
.
2021
;
39
:
256
-
259
.

2

Wong
CH
,
Siah
KW
,
Lo
AW.
Estimation of clinical trial success rates and related parameters
.
Biostatistics
.
2019
;
20
:
273
-
286
.

3

Hay
M
,
Thomas
DW
,
Craighead
JL
,
Economides
C
,
Rosenthal
J.
Clinical development success rates for investigational drugs
.
Nat Biotechnol
.
2014
;
32
:
40
-
51
.

4

Dimasi
JA
,
Feldman
L
,
Seckler
A
,
Wilson
A.
Trends in risks associated with new drug development: success rates for investigational drugs
.
Clin Pharmacol Ther
.
2010
;
87
:
272
-
277
.

5

National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research
.
The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research
. National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research;
1979
. Accessed December 2, 2024. http://timetravel.mementoweb.org/memento/1978/http://www.hhs.gov/ohrp/archive/nationalcommission.html

6

Council for International Organizations of Medical Sciences (CIOMS)
. International Ethical Guidelines for Health-related Research Involving Humans, 4th ed. World Health Organization, WHO Press. Accessed December 4, 2023. https://cioms.ch/wp-content/uploads/2017/01/WEB-CIOMS-EthicalGuidelines.pdf

7

Kimmelman
J.
A theoretical framework for early human studies: uncertainty, intervention ensembles, and boundaries
.
Trials
.
2012
;
13
:
173
.

8

Kimmelman
J
,
London
AJ.
The structure of clinical translation: efficiency, information, and ethics
.
Hastings Cent Rep
.
2015
;
45
:
27
-
39
.

9

Ning
N
,
Yan
J
,
Xie
X-J
,
Gerber
DE.
Impact of NCI-mandated scientific review on protocol development and content
.
J Natl Compr Cancer Netw
.
2015
;
13
:
409
-
416
.

10

U.S. Department of Health and Human Services
. PAR-21-321: Cancer Center Support Grants (CCSGS) for NCI-designated cancer centers (P30 clinical trial optional). Accessed October 9, 2024. https://grants.nih.gov/grants/guide/pa-files/PAR-21-321.html

11

Van Luijn
HE
,
Musschenga
AW
,
Keus
RB
,
Aaronson
NK.
Evaluating the risks and benefits of phase II and III cancer clinical trials: a look at Institutional Review Board members in the Netherlands
.
IRB
.
2007
;
29
:
13
-
18
.

12

Van Luijn
HEM
,
Musschenga
AW
,
Keus
RB
,
Robinson
WM
,
Aaronson
NK.
Assessment of the risk/benefit ratio of phase II cancer clinical trials by Institutional Review Board (IRB) members
.
Ann Oncol
.
2002
;
13
:
1307
-
1313
.

13

Errington
TM
,
Mathur
M
,
Soderberg
CK
, et al.
Investigating the replicability of preclinical cancer biology
.
eLife
.
2021
;
10
:
e71601
.

14

Kane
PB
,
Kimmelman
J.
Is preclinical research in cancer biology reproducible enough?
eLife
.
2021
;
10
:
e67527
.

15

Prinz
F
,
Schlange
T
,
Asadullah
K.
Believe it or not: how much can we rely on published data on potential drug targets?
Nat Rev Drug Discov
.
2011
;
10
:
712
.

16

Begley
CG
,
Ellis
LM.
Raise standards for preclinical cancer research
.
Nature
.
2012
;
483
:
531
-
533
.

17

Begley
CG.
Six red flags for suspect work
.
Nature
.
2013
;
497
:
433
-
434
.

18

Henderson
VC
,
Kimmelman
J
,
Fergusson
D
,
Grimshaw
JM
,
Hackam
DG.
Threats to validity in the design and conduct of preclinical efficacy studies: a systematic review of guidelines for in vivo animal experiments
.
PLoS Med
.
2013
;
10
:
e1001489
.

19

Stroke Therapy Academic Industry Roundtable (STAIR)
.
Recommendations for standards regarding preclinical neuroprotective and restorative drug development
.
Stroke
.
1999
;
30
:
2752
-
2758
.

20

Gresham
G
,
Meinert
JL
,
Gresham
AG
,
Meinert
CL.
Assessment of trends in the design, accrual, and completion of trials registered in ClinicalTrials.gov by sponsor type, 2000-2019
.
JAMA Netw Open
.
2020
;
3
:
e2014682
.

21

Carlisle
BG.
Numbat systematic review manager. Accessed April 1, 2021. https://numbat.bgcarlisle.com

22

Shadish
WR
,
Cook
TD
,
Campbell
DT.
Experimental and Quasi-Experimental Designs for Generalized Causal Inference
.
Houghton Mifflin
;
2002
.

23

Kimmelman
J
,
London
AJ.
Predicting harms and benefits in translational trials: ethics, evidence, and uncertainty
.
PLoS Med
.
2011
;
8
:
e1001010
.

24

Wold
B
,
Tabak
L.
ACD Working Group on Enhancing Rigor, Transparency, and Translatability in Animal Research: Final Report
.
NIH Advisory Committee to the Director
;
2021
.

25

Kimmelman
J
,
Bodilly Kane
P
,
Bicer
S
,
Carlisle
BG.
Preclinical assessment for translation to humans: the PATH approach for assessing supporting evidence for early-phase trials and innovative care
.
Med
.
2024
;
5
:
1227
-
1236
.

26

Yu
M
,
Montroy
J
,
Fergusson
D
,
Lalu
MM
,
Kimmelman
J.
Systematic review and narrative review lead experts to different cancer trial predictions: a randomized trial
.
J Clin Epidemiol
.
2021
;
132
:
116
-
124
.

27

Savina
M
,
Gourgou
S
,
Italiano
A
, et al.
Meta-analyses evaluating surrogate endpoints for overall survival in cancer randomized trials: a critical review
.
Crit Rev Oncol/Hematol
.
2018
;
123
:
21
-
41
.

28

Cooper
K
,
Tappenden
P
,
Cantrell
A
,
Ennis
K.
A systematic review of meta-analyses assessing the validity of tumour response endpoints as surrogates for progression-free or overall survival in cancer
.
Br J Cancer
.
2020
;
123
:
1686
-
1696
.

29

Hua
T
,
Gao
Y
,
Zhang
R
,
Wei
Y
,
Chen
F.
Validating ORR and PFS as surrogate endpoints in phase II and III clinical trials for NSCLC patients: difference exists in the strength of surrogacy in various trial settings
.
BMC Cancer
.
2022
;
22
:
1022
.

30

Pasalic
D
,
McGinnis
GJ
,
Fuller
CD
, et al.
Progression-free survival is a suboptimal predictor for overall survival among metastatic solid tumour clinical trials
.
Eur J Cancer
.
2020
;
136
:
176
-
185
.

31

Prasad
V
,
Kim
C
,
Burotto
M
,
Vandross
A.
The strength of association between surrogate end points and survival in oncology: a systematic review of trial-level meta-analyses
.
JAMA Intern Med
.
2015
;
175
:
1389
-
1398
.

32

Hwang
TJ
,
Gyawali
B.
Association between progression‐free survival and patients’ quality of life in cancer clinical trials
.
Int J Cancer
.
2019
;
144
:
1746
-
1751
.

33

Pratte
M
,
Ganeshamoorthy
S
,
Carlisle
B
,
Kimmelman
J.
How well are phase 2 cancer trial publications supported by preclinical efficacy evidence?
Int J Cancer
.
2019
;
145
:
3370
-
3375
.

34

Sievers
S
,
Wieschowski
S
,
Strech
D.
Investigator brochures for phase I/II trials lack information on the robustness of preclinical safety studies
.
Br J Clin Pharmacol
.
2021
;
87
:
2723
-
2731
.

35

Wieschowski
S
,
Chin
WWL
,
Federico
C
,
Sievers
S
,
Kimmelman
J
,
Strech
D.
Preclinical efficacy studies in investigator brochures: do they enable risk–benefit assessment?
PLoS Biol
.
2018
;
16
:
e2004879
.

36

Sheng
J
,
Feldhake
E
,
Zarin
DA
,
Kimmelman
J.
Completeness of clinical evidence citation in trial protocols: a cross-sectional analysis
.
Med
.
2022
;
3
:
335
-
343.e6
.

37

Paludan-Müller
AS
,
Ogden
MC
,
Marquardsen
M
,
Vive
J
,
Jørgensen
KJ
,
Gøtzsche
PC.
Do protocols for new randomised trials take previous similar trials into account? Cohort study of contemporary trial protocols
.
BMJ Open
.
2019
;
9
:
e026661
.

38

Chan
A-W
,
Tetzlaff
JM
,
Altman
DG
, et al.
SPIRIT 2013 statement: defining standard protocol items for clinical trials
.
Ann Intern Med
.
2013
;
158
:
200
-
207
.

39

U.S. Food and Drug Administration
. E6(R2) good clinical practice: integrated addendum to ICH E6(R1). Accessed December 4, 2023. https://www.fda.gov/media/93884/download

40

National Institutes of Health (NIH)
. NIH-FDA phase 2 and 3 IND/IDE clinical trial protocol template. Accessed December 4,
2023
. https://osp.od.nih.gov/wp-content/uploads/2022/11/Protocol-Template-Version-1.0-040717.docx

41

Macgregor
D
,
Lichtenstein
S
,
Slovic
P.
Structuring knowledge retrieval—an analysis of decomposed quantitative judgments
.
Organ Behav Hum Decis Process
.
1988
;
42
:
303
-
323
.

This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial License (https://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact [email protected] for reprints and translation rights for reprints. All other permissions can be obtained through our RightsLink service via the Permissions link on the article page on our site—for further information please contact [email protected].

Supplementary data