Abstract

The intracluster correlation coefficient (ICC) measures the correlation of observations within clusters and is a key parameter for power and sample size calculations for cluster randomized trials (CRTs). To facilitate the design of future CRTs within the National Cancer Institute Community Oncology Research Program (NCORP), all studies from the NCORP website were reviewed to identify completed CRTs. ICCs for primary and secondary outcomes (when available) were ascertained from these trials and summarized in this article as a resource for future trial development. Although ICCs are relatively small for many outcome cluster combinations, that is not always the case, so consideration should always be given to the specific outcome of interest, trial design, and type of cluster when estimating an ICC to facilitate trial development.

Introduction

In traditional randomized controlled trials, individuals are randomized independently to a given intervention or comparator group. Randomization may be stratified by individual or disease characteristics as well as by participating sites to counter imbalance in randomization. However, in cluster randomized trials (CRTs), entire groups or clusters are randomized to a given intervention or comparator instead.1,2 Although randomization occurs at the group or cluster level, interventions can be directed to the cluster level (eg, clinic or provider) or to an individual within that cluster. Examples of cluster-level interventions include incorporation of a decision-making tool into a clinic’s electronic health record (EHR) or education of a health-care provider. The provider is considered a cluster in this example because all participants seen by that provider will be affected by the education given to the provider. Individual-level interventions could include using a decision aid with an individual participant or delivering a behavioral intervention such as an exercise program to all participants within the same health-care clinic.3 Regardless of the level of the intervention, if randomization happens at the cluster level and the unit of analysis is at the individual level (eg, clinics are randomized but outcomes are collected on individual participants), there could be correlation of the individual-level outcomes within the clusters. This means that the individual observations cannot be considered independent and must be accounted for in both the design and analysis.4

The most straightforward way to calculate the sample size for a CRT is to calculate the sample size required for an individually randomized trial with the same effect size and power, and then inflate it using a design effect.5 The design effect accounts for the lack of independence of observations within clusters by incorporating an intracluster correlation coefficient (ICC). The ICC measures the correlation of observations within clusters and generally ranges from 0 (no correlation or independence) to 1 (perfect correlation or all outcomes in the cluster are the same). The particular design effect used in a sample size calculation depends on the specific design (eg, parallel, cross-over, stepped-wedge) and the study outcome (eg, continuous, binary) being used, but all design effects rely on an estimate of the ICC, making it a key parameter in sample size determination.3,6 In addition, the ICC is dependent on the particular outcome being considered, as well as the type of cluster. For example, the ICC may be lower for a patient-reported outcome such as quality of life than for an outcome that is more directly tied to the intervention such as number of discussions between the provider and participant. Furthermore, the ICC may be higher when the cluster is a family compared to when the cluster is a clinic. Mis-specification of the ICC is problematic because sample size calculations are sensitive to the particular ICC used and can therefore lead to wasted resources and potential ethical concerns. Specifically, an ICC estimate that is too small leads to an underpowered study, whereas an ICC estimate that is too large leads to a trial with more clusters and/or participants than necessary.7 Therefore, reporting ICCs in the literature is critical for the development of future CRTs, and reporting of ICCs for primary outcomes is included in the Consolidated Standards of Reporting Trials (CONSORT) extension for CRTs.8

Despite the larger sample size and added complexity of CRTs, utilization of these trial designs has been increasing.9 CRTs are necessary when the intervention is at the cluster level, and may also be considered when the intervention is at the individual level, but there is concern about contamination.3 CRTs have been used within the National Cancer Institute’s Community Oncology Research Program (NCORP) in multiple focus areas such as cancer prevention, supportive care, and symptom management, but they have become particularly popular within cancer care delivery research (CCDR), because care delivery interventions are often at the cluster level such as clinic or provider.10 To facilitate the design of future CRTs within NCORP, this article reports ICCs for primary and secondary (when available) outcomes from CRTs conducted within the NCORP network with data available to date.

Methods

All studies listed on the NCORP website (ncorp.cancer.gov/find-a-study/) as of September 1, 2023 were reviewed. Studies that were not affiliated with one of the seven current NCORP Research Bases (RB) were removed from further consideration because we focused on trials conducted within the current NCORP structure. The seven RBs include Alliance, Children’s Oncology Group (COG), ECOG-ACRIN, NRG Oncology, SWOG, University of Rochester, and Wake Forest. ClinicalTrials.gov records were then reviewed by a single reviewer (AS) for all of the remaining studies to determine whether the study was cluster randomized, individually randomized, or had some other design. The current trial status was then reviewed for each of the identified CRTs. If the CRT was complete or had passed the primary completion date, the study statistician was contacted to provide ICC estimates for the primary outcome and any available secondary outcomes. In addition, ClinicalTrials.gov records and any available publications were reviewed to provide basic details on the study design in order to provide appropriate context for each included study.

Using a pragmatic approach, the method used for ICC estimation was at the discretion of the study statistician. However, ICCs for continuous outcomes were all estimated using a linear mixed-model framework. Estimation of ICCs for binary outcomes was more variable, which was anticipated given that there are multiple ways to approach the calculation of binary ICCs.5,6,11 The particular method used by each study is documented in footnotes in the tables reporting the ICC estimates to facilitate interpretation. All ICC estimates were adjusted for intervention arm because this is a standard covariate included in an analysis of a CRT. Any other adjustments are noted in the footnotes for transparency, because additional covariate adjustment could affect the ICC estimate in either direction.

Results

Of the 159 studies reviewed that were affiliated with one of the seven current NCORP Research Bases, 16 (10%) were cluster randomized. The majority of these CRTs are classified as CCDR (n = 11; 69%), and just over half of the trials (n = 9; 56%) were complete or had passed the primary completion date. We were able to report on ICCs (n = 53) for all nine of these completed studies. Figure 1 provides a flow diagram for all studies reviewed.

Study flow diagram.
Figure 1.

Study flow diagram.

Study information for each of the nine trials with ICC data available is summarized in Table 1 to provide appropriate context for ICC interpretation. In addition, Table 1 includes references for all publications available to date associated with each trial. Table 2 presents ICCs for the two trials that report ICCs at two levels (site and provider), and Table 3 presents ICCs for the remaining trials that report a single site-level ICC. Tables 2 and 3 also include the ICC assumed at the time of sample size calculation for the primary outcome(s). ICC estimates were highly variable, ranging from -0.0005 to 0.50, demonstrating the importance of the considering the particular outcome and cluster type when estimating the ICC. In addition, mis-specification of ICCs was common at the design stage, with all trials being off by at least 0.01 in the ICC estimate, but with estimates being off by as much as 0.17. The average absolute deviation was 0.08; approximately half of the estimates assumed at sample size calculation were underestimates, and half were overestimates.

Table 1.

Summary of the identified cluster randomized trials with ICC data available.

Trial TitleResearch BaseTrial TypeSample SizeSummaryDesignPrimary Outcome(s)
A231701CD: Increasing Socioeconomically Disadvantaged Patients’ Engagement in Breast Cancer Surgery Decision Making Through a Shared Decision Making Intervention (NCT03766009)17AllianceCancer Care Delivery Research576 patients were clustered within 25 surgeons who were clustered within 10 sitesThis trial studies how well a breast cancer surgery decision aid works in increasing patient engagement in decision making in clinics serving a high proportion of socioeconomically disadvantaged patients.Cluster randomized trial using a continuous recruitment stepped wedge design. Clinics were randomized to the timing with which they cross over to the decision aid intervention, with new clinics crossing over every 10 weeks.(1) Self-efficacy in patient physician interactions as measured by the Patient’s Self-Efficacy in Patient-Physician Interactions (PEPPI)-5 point scale within 3 weeks of post-surgical consultation and (2) active patient participation measured by the Active Patient Participation Behaviors during the surgeon consultation.
A231601CD: Improving Surgical Care and Outcomes in Older Cancer Patients Through Implementation of an Efficient Pre-Surgical Toolkit (OPTI-Surg; NCT03857620)18AllianceCancer Care Delivery Research325 patients were clustered within 29 sitesThis trial studies how well the use of a pre-surgical toolkit (OPTI-Surg) works in improving surgical care and outcomes in older participants with cancer.1:1:1 parallel cluster-randomized trial with sites stratified by practice type (thoracic, major abdominal, urologic). Sites were randomized to Usual Care, OPTI-Surg, or OPTI-Surg + Coach.Patient function per Community Healthy Activities Model Program for Seniors (CHAMPS) questionnaire at 8 weeks post-surgery.
A191402CD: Testing Decision Aids to Improve Prostate Cancer Decisions for Minority Men (NCT03103321)16,19AllianceCancer Care Delivery Research158 patients were clustered within 15 sitesThis trial studies how well decision aids work in improving knowledge in patients with newly diagnosed prostate cancer.Cluster randomized trial using a 2 × 2 factorial design. Sites were randomized to pre-visit decision aid, within-visit decision aid, pre- and within-visit decision aid, or usual care.Patient knowledge of prostate cancer risks, features, and implications for treatment assessed immediately after the index specialist consultation (proportion correct on 12 items).
E1Q11: EROS: Engendering Reproductive Health Within Oncologic Survivorship (NCT01806129)20ECOG-ACRINSupportive Care/Symptom Management/PROs420 patients were clustered within 17 sitesThis trial studies whether a reproductive health intervention (including education and navigation components) may improve reproductive health goal concordant management (ie, patient’s receipt of appropriate referrals to achieve her reproductive health goals) for patients with a new diagnosis of any type of cancer.1:1 parallel cluster-randomized trial in NCORP and Minority Underserved (MU) NCORP sites. Sites were randomized to ARM A (standard practice related to reproductive health) or ARM B (reproductive health intervention).Reproductive health goal concordant management within 3 months from study entry.
S1415CD: A Pragmatic Trial to Evaluate a Guideline-Based Colony Stimulating Factor Standing Order Intervention and to Determine the Effectiveness of Colony Stimulating Factor Use as a Prophylaxis for Patients Receiving Chemotherapy with Intermediate Risk for Febrile Neutropenia - Trial Assessing CSF Prescribing Effectiveness and Risk (TrACER; NCT02728596)12,21,22SWOGCancer Care Delivery Research2946 patients were clustered within 32 sitesThis trial studies prophylactic colony stimulating factor (CSF) management in patients with breast, colorectal or non-small cell lung cancer receiving chemotherapy and with risk of developing febrile neutropenia.3:1 parallel cluster-randomized trial comparing intervention (standing order for high risk and alert not to prescribe for low risk) to usual care. Clinics were stratified on clinic size and whether the clinic was classified as serving a predominantly underserved or minority population. There was a second clinic-level 1:1 randomization for patients at intervention clinics at intermediate risk for febrile neutropenia comparing a standard order to an alert not to prescribe.(1) CSF prescribed as primary prophylaxis, defined as the initiation of granulocyte CSFs during the first cycle of myelosuppressive systemic therapy, given 24 to 72 hours after cessation of systemic therapy, and (2) incidence of febrile neutropenia within 6 months.
URCC-13059: A Geriatric Assessment Intervention for Patients Aged 70 and Over Receiving Chemotherapy or Similar Agents for Advanced Cancer: Reducing Toxicity in Older Adults (GAP70+; NCT02054741)14,23-25University of RochesterOther718 patients were clustered within 40 sitesThis trial compares a geriatric assessment intervention with usual care for reducing cancer treatment toxicity in older patients with cancer that has spread to other places in the body.Parallel cluster-randomized trial with sites randomized to intervention (patients and physicians are provided with the geriatric assessment information and recommendations) or usual care (only clinically significant cognitive impairment and depression is provided to the oncology teams). Sites were stratified by previous accrual (high vs low).Any grade 3–5 toxicity (measured by NCI-CTCAE v4) over 3 months.
URCC-13070: Improving Communication for Cancer Treatment: Addressing Concerns of Older Cancer Patients and Caregivers (COACH; NCT02107443)26-34University of RochesterOther541 patients were clustered within 31 sitesThis trial examines whether providing a web-generated geriatric assessment (GA) summary with targeted recommendations to older patients with advanced cancer, their caregivers, and their oncologists can improve communication about age-related concerns that could affect efficacy and tolerance of cancer treatment.Parallel cluster-randomized trial with sites randomized to intervention (GA summary and targeted recommendations provided) or usual care. Sites were stratified by previous accrual (high vs low).Patient satisfaction with communication about age-related concerns as measured by the modified Health Care Climate Questionnaire (HCCQ) within 1-7 days of the baseline clinic consultation.
WF-20817CD: Implementation of Smoking Cessation Services Within NCI NCORP Community Sites with Organized Lung Cancer Screening Programs (OaSiS; NCT03291587)15,35Wake ForestCancer Care Delivery Research1094 patients were clustered within 26 sitesThis trial evaluates a multifaceted training program to improve smoking cessation among smokers who present for lung cancer screening in community-based practices.Hybrid Type II effectiveness-implementation parallel cluster-randomized trial with sites randomized to intervention (training of site personnel on implementation of the US Public Health Service Guidelines for Smoking Cessation and Performance Coaching) or usual care. Sites were matched based on lung screening volume and racial/ethnic diversity, and then assigned to the intervention or usual care group.Effectiveness as measured by 7-day sustained smoking abstinence at 6 months as reported on the patient survey.
WF-1804CD: Assessing Effectiveness and Implementation of an EHR Tool to Assess Heart Health Among Survivors (AH-HA; NCT03935282)13,36Wake ForestCancer Care Delivery Research645 patients were clustered within 9 sitesThis trial examines the effects of an EHR-based cardiovascular health (CVH) assessment tool (AH-HA) on (1) CVH discussions among survivors and oncology providers, (2) referrals and visits to primary care, and (3) cardiovascular risk reduction and health promotion activities among cancer survivors.Hybrid effectiveness-implementation parallel cluster-randomized trial with sites randomized receive the AH-HA tool or continue with usual care. Sites were randomized in pairs.Discussion of non-ideal CVH factors, defined as patient-reported discussions with their provider for any of the 7 non-ideal CVH conditions identified for that patient. Conditions include CVH factors (cholesterol, blood pressure, glucose/hemoglobin A1c) and CVH behaviors (body mass index, smoking, diet, and physical activity).
Trial TitleResearch BaseTrial TypeSample SizeSummaryDesignPrimary Outcome(s)
A231701CD: Increasing Socioeconomically Disadvantaged Patients’ Engagement in Breast Cancer Surgery Decision Making Through a Shared Decision Making Intervention (NCT03766009)17AllianceCancer Care Delivery Research576 patients were clustered within 25 surgeons who were clustered within 10 sitesThis trial studies how well a breast cancer surgery decision aid works in increasing patient engagement in decision making in clinics serving a high proportion of socioeconomically disadvantaged patients.Cluster randomized trial using a continuous recruitment stepped wedge design. Clinics were randomized to the timing with which they cross over to the decision aid intervention, with new clinics crossing over every 10 weeks.(1) Self-efficacy in patient physician interactions as measured by the Patient’s Self-Efficacy in Patient-Physician Interactions (PEPPI)-5 point scale within 3 weeks of post-surgical consultation and (2) active patient participation measured by the Active Patient Participation Behaviors during the surgeon consultation.
A231601CD: Improving Surgical Care and Outcomes in Older Cancer Patients Through Implementation of an Efficient Pre-Surgical Toolkit (OPTI-Surg; NCT03857620)18AllianceCancer Care Delivery Research325 patients were clustered within 29 sitesThis trial studies how well the use of a pre-surgical toolkit (OPTI-Surg) works in improving surgical care and outcomes in older participants with cancer.1:1:1 parallel cluster-randomized trial with sites stratified by practice type (thoracic, major abdominal, urologic). Sites were randomized to Usual Care, OPTI-Surg, or OPTI-Surg + Coach.Patient function per Community Healthy Activities Model Program for Seniors (CHAMPS) questionnaire at 8 weeks post-surgery.
A191402CD: Testing Decision Aids to Improve Prostate Cancer Decisions for Minority Men (NCT03103321)16,19AllianceCancer Care Delivery Research158 patients were clustered within 15 sitesThis trial studies how well decision aids work in improving knowledge in patients with newly diagnosed prostate cancer.Cluster randomized trial using a 2 × 2 factorial design. Sites were randomized to pre-visit decision aid, within-visit decision aid, pre- and within-visit decision aid, or usual care.Patient knowledge of prostate cancer risks, features, and implications for treatment assessed immediately after the index specialist consultation (proportion correct on 12 items).
E1Q11: EROS: Engendering Reproductive Health Within Oncologic Survivorship (NCT01806129)20ECOG-ACRINSupportive Care/Symptom Management/PROs420 patients were clustered within 17 sitesThis trial studies whether a reproductive health intervention (including education and navigation components) may improve reproductive health goal concordant management (ie, patient’s receipt of appropriate referrals to achieve her reproductive health goals) for patients with a new diagnosis of any type of cancer.1:1 parallel cluster-randomized trial in NCORP and Minority Underserved (MU) NCORP sites. Sites were randomized to ARM A (standard practice related to reproductive health) or ARM B (reproductive health intervention).Reproductive health goal concordant management within 3 months from study entry.
S1415CD: A Pragmatic Trial to Evaluate a Guideline-Based Colony Stimulating Factor Standing Order Intervention and to Determine the Effectiveness of Colony Stimulating Factor Use as a Prophylaxis for Patients Receiving Chemotherapy with Intermediate Risk for Febrile Neutropenia - Trial Assessing CSF Prescribing Effectiveness and Risk (TrACER; NCT02728596)12,21,22SWOGCancer Care Delivery Research2946 patients were clustered within 32 sitesThis trial studies prophylactic colony stimulating factor (CSF) management in patients with breast, colorectal or non-small cell lung cancer receiving chemotherapy and with risk of developing febrile neutropenia.3:1 parallel cluster-randomized trial comparing intervention (standing order for high risk and alert not to prescribe for low risk) to usual care. Clinics were stratified on clinic size and whether the clinic was classified as serving a predominantly underserved or minority population. There was a second clinic-level 1:1 randomization for patients at intervention clinics at intermediate risk for febrile neutropenia comparing a standard order to an alert not to prescribe.(1) CSF prescribed as primary prophylaxis, defined as the initiation of granulocyte CSFs during the first cycle of myelosuppressive systemic therapy, given 24 to 72 hours after cessation of systemic therapy, and (2) incidence of febrile neutropenia within 6 months.
URCC-13059: A Geriatric Assessment Intervention for Patients Aged 70 and Over Receiving Chemotherapy or Similar Agents for Advanced Cancer: Reducing Toxicity in Older Adults (GAP70+; NCT02054741)14,23-25University of RochesterOther718 patients were clustered within 40 sitesThis trial compares a geriatric assessment intervention with usual care for reducing cancer treatment toxicity in older patients with cancer that has spread to other places in the body.Parallel cluster-randomized trial with sites randomized to intervention (patients and physicians are provided with the geriatric assessment information and recommendations) or usual care (only clinically significant cognitive impairment and depression is provided to the oncology teams). Sites were stratified by previous accrual (high vs low).Any grade 3–5 toxicity (measured by NCI-CTCAE v4) over 3 months.
URCC-13070: Improving Communication for Cancer Treatment: Addressing Concerns of Older Cancer Patients and Caregivers (COACH; NCT02107443)26-34University of RochesterOther541 patients were clustered within 31 sitesThis trial examines whether providing a web-generated geriatric assessment (GA) summary with targeted recommendations to older patients with advanced cancer, their caregivers, and their oncologists can improve communication about age-related concerns that could affect efficacy and tolerance of cancer treatment.Parallel cluster-randomized trial with sites randomized to intervention (GA summary and targeted recommendations provided) or usual care. Sites were stratified by previous accrual (high vs low).Patient satisfaction with communication about age-related concerns as measured by the modified Health Care Climate Questionnaire (HCCQ) within 1-7 days of the baseline clinic consultation.
WF-20817CD: Implementation of Smoking Cessation Services Within NCI NCORP Community Sites with Organized Lung Cancer Screening Programs (OaSiS; NCT03291587)15,35Wake ForestCancer Care Delivery Research1094 patients were clustered within 26 sitesThis trial evaluates a multifaceted training program to improve smoking cessation among smokers who present for lung cancer screening in community-based practices.Hybrid Type II effectiveness-implementation parallel cluster-randomized trial with sites randomized to intervention (training of site personnel on implementation of the US Public Health Service Guidelines for Smoking Cessation and Performance Coaching) or usual care. Sites were matched based on lung screening volume and racial/ethnic diversity, and then assigned to the intervention or usual care group.Effectiveness as measured by 7-day sustained smoking abstinence at 6 months as reported on the patient survey.
WF-1804CD: Assessing Effectiveness and Implementation of an EHR Tool to Assess Heart Health Among Survivors (AH-HA; NCT03935282)13,36Wake ForestCancer Care Delivery Research645 patients were clustered within 9 sitesThis trial examines the effects of an EHR-based cardiovascular health (CVH) assessment tool (AH-HA) on (1) CVH discussions among survivors and oncology providers, (2) referrals and visits to primary care, and (3) cardiovascular risk reduction and health promotion activities among cancer survivors.Hybrid effectiveness-implementation parallel cluster-randomized trial with sites randomized receive the AH-HA tool or continue with usual care. Sites were randomized in pairs.Discussion of non-ideal CVH factors, defined as patient-reported discussions with their provider for any of the 7 non-ideal CVH conditions identified for that patient. Conditions include CVH factors (cholesterol, blood pressure, glucose/hemoglobin A1c) and CVH behaviors (body mass index, smoking, diet, and physical activity).
Table 1.

Summary of the identified cluster randomized trials with ICC data available.

Trial TitleResearch BaseTrial TypeSample SizeSummaryDesignPrimary Outcome(s)
A231701CD: Increasing Socioeconomically Disadvantaged Patients’ Engagement in Breast Cancer Surgery Decision Making Through a Shared Decision Making Intervention (NCT03766009)17AllianceCancer Care Delivery Research576 patients were clustered within 25 surgeons who were clustered within 10 sitesThis trial studies how well a breast cancer surgery decision aid works in increasing patient engagement in decision making in clinics serving a high proportion of socioeconomically disadvantaged patients.Cluster randomized trial using a continuous recruitment stepped wedge design. Clinics were randomized to the timing with which they cross over to the decision aid intervention, with new clinics crossing over every 10 weeks.(1) Self-efficacy in patient physician interactions as measured by the Patient’s Self-Efficacy in Patient-Physician Interactions (PEPPI)-5 point scale within 3 weeks of post-surgical consultation and (2) active patient participation measured by the Active Patient Participation Behaviors during the surgeon consultation.
A231601CD: Improving Surgical Care and Outcomes in Older Cancer Patients Through Implementation of an Efficient Pre-Surgical Toolkit (OPTI-Surg; NCT03857620)18AllianceCancer Care Delivery Research325 patients were clustered within 29 sitesThis trial studies how well the use of a pre-surgical toolkit (OPTI-Surg) works in improving surgical care and outcomes in older participants with cancer.1:1:1 parallel cluster-randomized trial with sites stratified by practice type (thoracic, major abdominal, urologic). Sites were randomized to Usual Care, OPTI-Surg, or OPTI-Surg + Coach.Patient function per Community Healthy Activities Model Program for Seniors (CHAMPS) questionnaire at 8 weeks post-surgery.
A191402CD: Testing Decision Aids to Improve Prostate Cancer Decisions for Minority Men (NCT03103321)16,19AllianceCancer Care Delivery Research158 patients were clustered within 15 sitesThis trial studies how well decision aids work in improving knowledge in patients with newly diagnosed prostate cancer.Cluster randomized trial using a 2 × 2 factorial design. Sites were randomized to pre-visit decision aid, within-visit decision aid, pre- and within-visit decision aid, or usual care.Patient knowledge of prostate cancer risks, features, and implications for treatment assessed immediately after the index specialist consultation (proportion correct on 12 items).
E1Q11: EROS: Engendering Reproductive Health Within Oncologic Survivorship (NCT01806129)20ECOG-ACRINSupportive Care/Symptom Management/PROs420 patients were clustered within 17 sitesThis trial studies whether a reproductive health intervention (including education and navigation components) may improve reproductive health goal concordant management (ie, patient’s receipt of appropriate referrals to achieve her reproductive health goals) for patients with a new diagnosis of any type of cancer.1:1 parallel cluster-randomized trial in NCORP and Minority Underserved (MU) NCORP sites. Sites were randomized to ARM A (standard practice related to reproductive health) or ARM B (reproductive health intervention).Reproductive health goal concordant management within 3 months from study entry.
S1415CD: A Pragmatic Trial to Evaluate a Guideline-Based Colony Stimulating Factor Standing Order Intervention and to Determine the Effectiveness of Colony Stimulating Factor Use as a Prophylaxis for Patients Receiving Chemotherapy with Intermediate Risk for Febrile Neutropenia - Trial Assessing CSF Prescribing Effectiveness and Risk (TrACER; NCT02728596)12,21,22SWOGCancer Care Delivery Research2946 patients were clustered within 32 sitesThis trial studies prophylactic colony stimulating factor (CSF) management in patients with breast, colorectal or non-small cell lung cancer receiving chemotherapy and with risk of developing febrile neutropenia.3:1 parallel cluster-randomized trial comparing intervention (standing order for high risk and alert not to prescribe for low risk) to usual care. Clinics were stratified on clinic size and whether the clinic was classified as serving a predominantly underserved or minority population. There was a second clinic-level 1:1 randomization for patients at intervention clinics at intermediate risk for febrile neutropenia comparing a standard order to an alert not to prescribe.(1) CSF prescribed as primary prophylaxis, defined as the initiation of granulocyte CSFs during the first cycle of myelosuppressive systemic therapy, given 24 to 72 hours after cessation of systemic therapy, and (2) incidence of febrile neutropenia within 6 months.
URCC-13059: A Geriatric Assessment Intervention for Patients Aged 70 and Over Receiving Chemotherapy or Similar Agents for Advanced Cancer: Reducing Toxicity in Older Adults (GAP70+; NCT02054741)14,23-25University of RochesterOther718 patients were clustered within 40 sitesThis trial compares a geriatric assessment intervention with usual care for reducing cancer treatment toxicity in older patients with cancer that has spread to other places in the body.Parallel cluster-randomized trial with sites randomized to intervention (patients and physicians are provided with the geriatric assessment information and recommendations) or usual care (only clinically significant cognitive impairment and depression is provided to the oncology teams). Sites were stratified by previous accrual (high vs low).Any grade 3–5 toxicity (measured by NCI-CTCAE v4) over 3 months.
URCC-13070: Improving Communication for Cancer Treatment: Addressing Concerns of Older Cancer Patients and Caregivers (COACH; NCT02107443)26-34University of RochesterOther541 patients were clustered within 31 sitesThis trial examines whether providing a web-generated geriatric assessment (GA) summary with targeted recommendations to older patients with advanced cancer, their caregivers, and their oncologists can improve communication about age-related concerns that could affect efficacy and tolerance of cancer treatment.Parallel cluster-randomized trial with sites randomized to intervention (GA summary and targeted recommendations provided) or usual care. Sites were stratified by previous accrual (high vs low).Patient satisfaction with communication about age-related concerns as measured by the modified Health Care Climate Questionnaire (HCCQ) within 1-7 days of the baseline clinic consultation.
WF-20817CD: Implementation of Smoking Cessation Services Within NCI NCORP Community Sites with Organized Lung Cancer Screening Programs (OaSiS; NCT03291587)15,35Wake ForestCancer Care Delivery Research1094 patients were clustered within 26 sitesThis trial evaluates a multifaceted training program to improve smoking cessation among smokers who present for lung cancer screening in community-based practices.Hybrid Type II effectiveness-implementation parallel cluster-randomized trial with sites randomized to intervention (training of site personnel on implementation of the US Public Health Service Guidelines for Smoking Cessation and Performance Coaching) or usual care. Sites were matched based on lung screening volume and racial/ethnic diversity, and then assigned to the intervention or usual care group.Effectiveness as measured by 7-day sustained smoking abstinence at 6 months as reported on the patient survey.
WF-1804CD: Assessing Effectiveness and Implementation of an EHR Tool to Assess Heart Health Among Survivors (AH-HA; NCT03935282)13,36Wake ForestCancer Care Delivery Research645 patients were clustered within 9 sitesThis trial examines the effects of an EHR-based cardiovascular health (CVH) assessment tool (AH-HA) on (1) CVH discussions among survivors and oncology providers, (2) referrals and visits to primary care, and (3) cardiovascular risk reduction and health promotion activities among cancer survivors.Hybrid effectiveness-implementation parallel cluster-randomized trial with sites randomized receive the AH-HA tool or continue with usual care. Sites were randomized in pairs.Discussion of non-ideal CVH factors, defined as patient-reported discussions with their provider for any of the 7 non-ideal CVH conditions identified for that patient. Conditions include CVH factors (cholesterol, blood pressure, glucose/hemoglobin A1c) and CVH behaviors (body mass index, smoking, diet, and physical activity).
Trial TitleResearch BaseTrial TypeSample SizeSummaryDesignPrimary Outcome(s)
A231701CD: Increasing Socioeconomically Disadvantaged Patients’ Engagement in Breast Cancer Surgery Decision Making Through a Shared Decision Making Intervention (NCT03766009)17AllianceCancer Care Delivery Research576 patients were clustered within 25 surgeons who were clustered within 10 sitesThis trial studies how well a breast cancer surgery decision aid works in increasing patient engagement in decision making in clinics serving a high proportion of socioeconomically disadvantaged patients.Cluster randomized trial using a continuous recruitment stepped wedge design. Clinics were randomized to the timing with which they cross over to the decision aid intervention, with new clinics crossing over every 10 weeks.(1) Self-efficacy in patient physician interactions as measured by the Patient’s Self-Efficacy in Patient-Physician Interactions (PEPPI)-5 point scale within 3 weeks of post-surgical consultation and (2) active patient participation measured by the Active Patient Participation Behaviors during the surgeon consultation.
A231601CD: Improving Surgical Care and Outcomes in Older Cancer Patients Through Implementation of an Efficient Pre-Surgical Toolkit (OPTI-Surg; NCT03857620)18AllianceCancer Care Delivery Research325 patients were clustered within 29 sitesThis trial studies how well the use of a pre-surgical toolkit (OPTI-Surg) works in improving surgical care and outcomes in older participants with cancer.1:1:1 parallel cluster-randomized trial with sites stratified by practice type (thoracic, major abdominal, urologic). Sites were randomized to Usual Care, OPTI-Surg, or OPTI-Surg + Coach.Patient function per Community Healthy Activities Model Program for Seniors (CHAMPS) questionnaire at 8 weeks post-surgery.
A191402CD: Testing Decision Aids to Improve Prostate Cancer Decisions for Minority Men (NCT03103321)16,19AllianceCancer Care Delivery Research158 patients were clustered within 15 sitesThis trial studies how well decision aids work in improving knowledge in patients with newly diagnosed prostate cancer.Cluster randomized trial using a 2 × 2 factorial design. Sites were randomized to pre-visit decision aid, within-visit decision aid, pre- and within-visit decision aid, or usual care.Patient knowledge of prostate cancer risks, features, and implications for treatment assessed immediately after the index specialist consultation (proportion correct on 12 items).
E1Q11: EROS: Engendering Reproductive Health Within Oncologic Survivorship (NCT01806129)20ECOG-ACRINSupportive Care/Symptom Management/PROs420 patients were clustered within 17 sitesThis trial studies whether a reproductive health intervention (including education and navigation components) may improve reproductive health goal concordant management (ie, patient’s receipt of appropriate referrals to achieve her reproductive health goals) for patients with a new diagnosis of any type of cancer.1:1 parallel cluster-randomized trial in NCORP and Minority Underserved (MU) NCORP sites. Sites were randomized to ARM A (standard practice related to reproductive health) or ARM B (reproductive health intervention).Reproductive health goal concordant management within 3 months from study entry.
S1415CD: A Pragmatic Trial to Evaluate a Guideline-Based Colony Stimulating Factor Standing Order Intervention and to Determine the Effectiveness of Colony Stimulating Factor Use as a Prophylaxis for Patients Receiving Chemotherapy with Intermediate Risk for Febrile Neutropenia - Trial Assessing CSF Prescribing Effectiveness and Risk (TrACER; NCT02728596)12,21,22SWOGCancer Care Delivery Research2946 patients were clustered within 32 sitesThis trial studies prophylactic colony stimulating factor (CSF) management in patients with breast, colorectal or non-small cell lung cancer receiving chemotherapy and with risk of developing febrile neutropenia.3:1 parallel cluster-randomized trial comparing intervention (standing order for high risk and alert not to prescribe for low risk) to usual care. Clinics were stratified on clinic size and whether the clinic was classified as serving a predominantly underserved or minority population. There was a second clinic-level 1:1 randomization for patients at intervention clinics at intermediate risk for febrile neutropenia comparing a standard order to an alert not to prescribe.(1) CSF prescribed as primary prophylaxis, defined as the initiation of granulocyte CSFs during the first cycle of myelosuppressive systemic therapy, given 24 to 72 hours after cessation of systemic therapy, and (2) incidence of febrile neutropenia within 6 months.
URCC-13059: A Geriatric Assessment Intervention for Patients Aged 70 and Over Receiving Chemotherapy or Similar Agents for Advanced Cancer: Reducing Toxicity in Older Adults (GAP70+; NCT02054741)14,23-25University of RochesterOther718 patients were clustered within 40 sitesThis trial compares a geriatric assessment intervention with usual care for reducing cancer treatment toxicity in older patients with cancer that has spread to other places in the body.Parallel cluster-randomized trial with sites randomized to intervention (patients and physicians are provided with the geriatric assessment information and recommendations) or usual care (only clinically significant cognitive impairment and depression is provided to the oncology teams). Sites were stratified by previous accrual (high vs low).Any grade 3–5 toxicity (measured by NCI-CTCAE v4) over 3 months.
URCC-13070: Improving Communication for Cancer Treatment: Addressing Concerns of Older Cancer Patients and Caregivers (COACH; NCT02107443)26-34University of RochesterOther541 patients were clustered within 31 sitesThis trial examines whether providing a web-generated geriatric assessment (GA) summary with targeted recommendations to older patients with advanced cancer, their caregivers, and their oncologists can improve communication about age-related concerns that could affect efficacy and tolerance of cancer treatment.Parallel cluster-randomized trial with sites randomized to intervention (GA summary and targeted recommendations provided) or usual care. Sites were stratified by previous accrual (high vs low).Patient satisfaction with communication about age-related concerns as measured by the modified Health Care Climate Questionnaire (HCCQ) within 1-7 days of the baseline clinic consultation.
WF-20817CD: Implementation of Smoking Cessation Services Within NCI NCORP Community Sites with Organized Lung Cancer Screening Programs (OaSiS; NCT03291587)15,35Wake ForestCancer Care Delivery Research1094 patients were clustered within 26 sitesThis trial evaluates a multifaceted training program to improve smoking cessation among smokers who present for lung cancer screening in community-based practices.Hybrid Type II effectiveness-implementation parallel cluster-randomized trial with sites randomized to intervention (training of site personnel on implementation of the US Public Health Service Guidelines for Smoking Cessation and Performance Coaching) or usual care. Sites were matched based on lung screening volume and racial/ethnic diversity, and then assigned to the intervention or usual care group.Effectiveness as measured by 7-day sustained smoking abstinence at 6 months as reported on the patient survey.
WF-1804CD: Assessing Effectiveness and Implementation of an EHR Tool to Assess Heart Health Among Survivors (AH-HA; NCT03935282)13,36Wake ForestCancer Care Delivery Research645 patients were clustered within 9 sitesThis trial examines the effects of an EHR-based cardiovascular health (CVH) assessment tool (AH-HA) on (1) CVH discussions among survivors and oncology providers, (2) referrals and visits to primary care, and (3) cardiovascular risk reduction and health promotion activities among cancer survivors.Hybrid effectiveness-implementation parallel cluster-randomized trial with sites randomized receive the AH-HA tool or continue with usual care. Sites were randomized in pairs.Discussion of non-ideal CVH factors, defined as patient-reported discussions with their provider for any of the 7 non-ideal CVH conditions identified for that patient. Conditions include CVH factors (cholesterol, blood pressure, glucose/hemoglobin A1c) and CVH behaviors (body mass index, smoking, diet, and physical activity).
Table 2.

ICCs for trials with 2 levels of correlation.

Outcome TypeOutcome DescriptionAssumed ICCEstimated ICCAssumed  ICCEstimated ICC
Site LevelSite LevelProvider LevelProvider Level
A231701CDa
 Primary Outcome #1 (continuous)Self-efficacy in patient physician interactions (PEPPI-5)0.070.0360.070.005
 Primary Outcome #2 (continuous)Active Patient Participation Behaviors0.00010.0460.010.108
 Secondary Outcome (continuous)Decision Quality Instrument-Breast Surgery knowledge0.0030.020
S1415CD (TrACER)b
 Primary Outcome #1 (binary)Colony Stimulating Factor prescribed to patients at high risk for febrile neutropenia0.020.1390.235
(0.127)(0.215)
 Primary Outcome #1 (binary)Colony Stimulating Factor prescribed to patients at intermediate risk for febrile neutropenia0.020.1860.381
(0.148)(0.319)
 Primary Outcome #1 (binary)Colony Stimulating Factor prescribed to patients at low risk for febrile neutropenia0.020.0370.114
(0.036)(0.106)
 Primary Outcome #2 (binary)Febrile neutropenia0.020.005−0.001
(0.004)(0.001)
Outcome TypeOutcome DescriptionAssumed ICCEstimated ICCAssumed  ICCEstimated ICC
Site LevelSite LevelProvider LevelProvider Level
A231701CDa
 Primary Outcome #1 (continuous)Self-efficacy in patient physician interactions (PEPPI-5)0.070.0360.070.005
 Primary Outcome #2 (continuous)Active Patient Participation Behaviors0.00010.0460.010.108
 Secondary Outcome (continuous)Decision Quality Instrument-Breast Surgery knowledge0.0030.020
S1415CD (TrACER)b
 Primary Outcome #1 (binary)Colony Stimulating Factor prescribed to patients at high risk for febrile neutropenia0.020.1390.235
(0.127)(0.215)
 Primary Outcome #1 (binary)Colony Stimulating Factor prescribed to patients at intermediate risk for febrile neutropenia0.020.1860.381
(0.148)(0.319)
 Primary Outcome #1 (binary)Colony Stimulating Factor prescribed to patients at low risk for febrile neutropenia0.020.0370.114
(0.036)(0.106)
 Primary Outcome #2 (binary)Febrile neutropenia0.020.005−0.001
(0.004)(0.001)
a

For A231701, the relevant provider is the surgeon; linear mixed effects models also included time effects in addition to intervention arm since a stepped wedge design was used; ICC was not assumed for the secondary outcome.

b

For S1415CD, the design did not incorporate an ICC at the physician level, but ICCs are included here as additional information; ICCs are calculated using a GEE with exchangeable correlation; the first ICC reported is on the proportion scale (identity link), and the second ICC in parentheses is on the logistic scale; for primary outcome #2, models also included risk group.

Table 2.

ICCs for trials with 2 levels of correlation.

Outcome TypeOutcome DescriptionAssumed ICCEstimated ICCAssumed  ICCEstimated ICC
Site LevelSite LevelProvider LevelProvider Level
A231701CDa
 Primary Outcome #1 (continuous)Self-efficacy in patient physician interactions (PEPPI-5)0.070.0360.070.005
 Primary Outcome #2 (continuous)Active Patient Participation Behaviors0.00010.0460.010.108
 Secondary Outcome (continuous)Decision Quality Instrument-Breast Surgery knowledge0.0030.020
S1415CD (TrACER)b
 Primary Outcome #1 (binary)Colony Stimulating Factor prescribed to patients at high risk for febrile neutropenia0.020.1390.235
(0.127)(0.215)
 Primary Outcome #1 (binary)Colony Stimulating Factor prescribed to patients at intermediate risk for febrile neutropenia0.020.1860.381
(0.148)(0.319)
 Primary Outcome #1 (binary)Colony Stimulating Factor prescribed to patients at low risk for febrile neutropenia0.020.0370.114
(0.036)(0.106)
 Primary Outcome #2 (binary)Febrile neutropenia0.020.005−0.001
(0.004)(0.001)
Outcome TypeOutcome DescriptionAssumed ICCEstimated ICCAssumed  ICCEstimated ICC
Site LevelSite LevelProvider LevelProvider Level
A231701CDa
 Primary Outcome #1 (continuous)Self-efficacy in patient physician interactions (PEPPI-5)0.070.0360.070.005
 Primary Outcome #2 (continuous)Active Patient Participation Behaviors0.00010.0460.010.108
 Secondary Outcome (continuous)Decision Quality Instrument-Breast Surgery knowledge0.0030.020
S1415CD (TrACER)b
 Primary Outcome #1 (binary)Colony Stimulating Factor prescribed to patients at high risk for febrile neutropenia0.020.1390.235
(0.127)(0.215)
 Primary Outcome #1 (binary)Colony Stimulating Factor prescribed to patients at intermediate risk for febrile neutropenia0.020.1860.381
(0.148)(0.319)
 Primary Outcome #1 (binary)Colony Stimulating Factor prescribed to patients at low risk for febrile neutropenia0.020.0370.114
(0.036)(0.106)
 Primary Outcome #2 (binary)Febrile neutropenia0.020.005−0.001
(0.004)(0.001)
a

For A231701, the relevant provider is the surgeon; linear mixed effects models also included time effects in addition to intervention arm since a stepped wedge design was used; ICC was not assumed for the secondary outcome.

b

For S1415CD, the design did not incorporate an ICC at the physician level, but ICCs are included here as additional information; ICCs are calculated using a GEE with exchangeable correlation; the first ICC reported is on the proportion scale (identity link), and the second ICC in parentheses is on the logistic scale; for primary outcome #2, models also included risk group.

Table 3.

ICCs for trials with a single level of correlation

Outcome TypeOutcome DescriptionAssumed
ICCh
Estimated ICC
A231601CD (OPTI-Surg)a
Primary Outcome (continuous)Patient function as measured by CHAMPS total score (28 items) 8 weeks post-surgery0.0050.027
Secondary Outcome (binary)Postoperative complications 12 weeks post-surgery: grade 0 versus grade 1-50.216
Secondary Outcome (continuous)Quality of life as measured by EQ-5D-5L index (5 items) at 8 weeks post-surgery0.015
Secondary Outcome (continuous)Quality of life as measured by EQ-5D-5L Visual Analog Scale at 8 weeks post-surgery0.055
A191402CDb
Primary Outcome (continuous)Patient knowledge (proportion correct on 12 items)0.100.230
E1Q11c
Primary Outcome (binary)Reproductive health goal concordant management within 3 months from study entry0.100.024
Secondary Outcome (binary)Long-term contraception uptake at 6 months from study entry0.002
URCC-13059 (GAP70+)d
Primary Outcome (binary)Any grade 3–5 toxicity (measured by NCI-CTCAE v4) over 3 months0.100.017
Secondary Outcome (binary)Reduced dose intensity in Cycle 10.025
Secondary Outcome (binary)Dose modified at 3 months related to toxicity0.039
URCC-13070 (COACH)e
Primary Outcome (continuous)Patients’ satisfaction with communication about aging-related concerns (modified HCCQ)0.140.016
Secondary Outcome (continuous)Number of aging-related concerns discussed during the visit (from content analysis of audio recordings)0.138
Secondary Outcome (continuous)Patients’ quality of life (measured with the Functional Assessment of Cancer Therapy scale)0.041
Secondary Outcome (continuous)Caregivers’ quality of life (measured with 12-Item Short Form Health Survey)0.042
Secondary Outcome (continuous)Caregivers’ satisfaction with communication about aging-related patients’ concerns0.013
WF-20817CD (OaSiS)f
Primary Outcome (binary)Effectiveness: 7-day sustained smoking abstinence at 6 months0.030.018
Secondary Outcome (binary)Effectiveness: Short term smoking abstinence (attempt for 1 day) at 6 months0.064
Secondary Outcome (continuous)Implementation (Fidelity to the Intervention): Patients asked if they received 19 cessation services at 14 days; outcome is the sum of services received0.500
WF-1804CD (AH-HA)g
Primary Outcome (binary)Discussion of any non-ideal CVH factor (cholesterol, blood pressure, glucose/hemoglobin A1c, body mass index, smoking, diet, and physical activity) by patient report0.040.016
Secondary Outcomes (binary)Discussion of the 7 individual non-ideal CVH factors by patient report (each factor considered separately)-0.008 to 0.036
Secondary Outcomes (binary)EHR documentation of CVH factor discussions (each factor considered separately)0.150 to 0.310
Secondary Outcome (binary)Patient referred to primary care (patient reported)0.023
Secondary Outcome (binary)Patient referred to cardiology (patient reported)0.002
Secondary Outcome (binary)Patient recommended to primary care (EHR documented)0.279
Secondary Outcome (binary)Patient recommended to cardiology (EHR documented)0.036
Secondary Outcome (binary)EHR documentation of being prescribed a new cholesterol, diabetes, and/or blood pressure medication-0.005
Secondary Outcome (binary)EHR report of cholesterol, glucose, and/or HbA1c test ordered0.104
Outcome TypeOutcome DescriptionAssumed
ICCh
Estimated ICC
A231601CD (OPTI-Surg)a
Primary Outcome (continuous)Patient function as measured by CHAMPS total score (28 items) 8 weeks post-surgery0.0050.027
Secondary Outcome (binary)Postoperative complications 12 weeks post-surgery: grade 0 versus grade 1-50.216
Secondary Outcome (continuous)Quality of life as measured by EQ-5D-5L index (5 items) at 8 weeks post-surgery0.015
Secondary Outcome (continuous)Quality of life as measured by EQ-5D-5L Visual Analog Scale at 8 weeks post-surgery0.055
A191402CDb
Primary Outcome (continuous)Patient knowledge (proportion correct on 12 items)0.100.230
E1Q11c
Primary Outcome (binary)Reproductive health goal concordant management within 3 months from study entry0.100.024
Secondary Outcome (binary)Long-term contraception uptake at 6 months from study entry0.002
URCC-13059 (GAP70+)d
Primary Outcome (binary)Any grade 3–5 toxicity (measured by NCI-CTCAE v4) over 3 months0.100.017
Secondary Outcome (binary)Reduced dose intensity in Cycle 10.025
Secondary Outcome (binary)Dose modified at 3 months related to toxicity0.039
URCC-13070 (COACH)e
Primary Outcome (continuous)Patients’ satisfaction with communication about aging-related concerns (modified HCCQ)0.140.016
Secondary Outcome (continuous)Number of aging-related concerns discussed during the visit (from content analysis of audio recordings)0.138
Secondary Outcome (continuous)Patients’ quality of life (measured with the Functional Assessment of Cancer Therapy scale)0.041
Secondary Outcome (continuous)Caregivers’ quality of life (measured with 12-Item Short Form Health Survey)0.042
Secondary Outcome (continuous)Caregivers’ satisfaction with communication about aging-related patients’ concerns0.013
WF-20817CD (OaSiS)f
Primary Outcome (binary)Effectiveness: 7-day sustained smoking abstinence at 6 months0.030.018
Secondary Outcome (binary)Effectiveness: Short term smoking abstinence (attempt for 1 day) at 6 months0.064
Secondary Outcome (continuous)Implementation (Fidelity to the Intervention): Patients asked if they received 19 cessation services at 14 days; outcome is the sum of services received0.500
WF-1804CD (AH-HA)g
Primary Outcome (binary)Discussion of any non-ideal CVH factor (cholesterol, blood pressure, glucose/hemoglobin A1c, body mass index, smoking, diet, and physical activity) by patient report0.040.016
Secondary Outcomes (binary)Discussion of the 7 individual non-ideal CVH factors by patient report (each factor considered separately)-0.008 to 0.036
Secondary Outcomes (binary)EHR documentation of CVH factor discussions (each factor considered separately)0.150 to 0.310
Secondary Outcome (binary)Patient referred to primary care (patient reported)0.023
Secondary Outcome (binary)Patient referred to cardiology (patient reported)0.002
Secondary Outcome (binary)Patient recommended to primary care (EHR documented)0.279
Secondary Outcome (binary)Patient recommended to cardiology (EHR documented)0.036
Secondary Outcome (binary)EHR documentation of being prescribed a new cholesterol, diabetes, and/or blood pressure medication-0.005
Secondary Outcome (binary)EHR report of cholesterol, glucose, and/or HbA1c test ordered0.104
a

For A231601, ICCs for continuous outcomes were estimated using a linear mixed model, and the binary ICC was calculated using a random intercept logistic model.12

b

For A191402, the ICC estimate comes from the primary publication,17 and the linear mixed model also included baseline patient characteristics.

c

For E1Q11, ICCs were estimated using a GEE with exchangeable correlation and a logit link.37

d

For URCC-13059, ICCs were estimated using a linear mixed model (ie, on the proportion scale).

e

For URCC-13070, ICCs were estimated using a linear mixed model; for longitudinal outcomes, a constant ICC over time was assumed.

f

For WF-20817CD, binary ICCs were calculated using a random intercept logistic model,12 and the continuous ICC was estimated using a linear mixed model.

g

For WF-1804CD, ICCs were estimated using a GEE with exchangeable correlation and a logit link.

h

There were no ICCs assumed for secondary outcomes.

Table 3.

ICCs for trials with a single level of correlation

Outcome TypeOutcome DescriptionAssumed
ICCh
Estimated ICC
A231601CD (OPTI-Surg)a
Primary Outcome (continuous)Patient function as measured by CHAMPS total score (28 items) 8 weeks post-surgery0.0050.027
Secondary Outcome (binary)Postoperative complications 12 weeks post-surgery: grade 0 versus grade 1-50.216
Secondary Outcome (continuous)Quality of life as measured by EQ-5D-5L index (5 items) at 8 weeks post-surgery0.015
Secondary Outcome (continuous)Quality of life as measured by EQ-5D-5L Visual Analog Scale at 8 weeks post-surgery0.055
A191402CDb
Primary Outcome (continuous)Patient knowledge (proportion correct on 12 items)0.100.230
E1Q11c
Primary Outcome (binary)Reproductive health goal concordant management within 3 months from study entry0.100.024
Secondary Outcome (binary)Long-term contraception uptake at 6 months from study entry0.002
URCC-13059 (GAP70+)d
Primary Outcome (binary)Any grade 3–5 toxicity (measured by NCI-CTCAE v4) over 3 months0.100.017
Secondary Outcome (binary)Reduced dose intensity in Cycle 10.025
Secondary Outcome (binary)Dose modified at 3 months related to toxicity0.039
URCC-13070 (COACH)e
Primary Outcome (continuous)Patients’ satisfaction with communication about aging-related concerns (modified HCCQ)0.140.016
Secondary Outcome (continuous)Number of aging-related concerns discussed during the visit (from content analysis of audio recordings)0.138
Secondary Outcome (continuous)Patients’ quality of life (measured with the Functional Assessment of Cancer Therapy scale)0.041
Secondary Outcome (continuous)Caregivers’ quality of life (measured with 12-Item Short Form Health Survey)0.042
Secondary Outcome (continuous)Caregivers’ satisfaction with communication about aging-related patients’ concerns0.013
WF-20817CD (OaSiS)f
Primary Outcome (binary)Effectiveness: 7-day sustained smoking abstinence at 6 months0.030.018
Secondary Outcome (binary)Effectiveness: Short term smoking abstinence (attempt for 1 day) at 6 months0.064
Secondary Outcome (continuous)Implementation (Fidelity to the Intervention): Patients asked if they received 19 cessation services at 14 days; outcome is the sum of services received0.500
WF-1804CD (AH-HA)g
Primary Outcome (binary)Discussion of any non-ideal CVH factor (cholesterol, blood pressure, glucose/hemoglobin A1c, body mass index, smoking, diet, and physical activity) by patient report0.040.016
Secondary Outcomes (binary)Discussion of the 7 individual non-ideal CVH factors by patient report (each factor considered separately)-0.008 to 0.036
Secondary Outcomes (binary)EHR documentation of CVH factor discussions (each factor considered separately)0.150 to 0.310
Secondary Outcome (binary)Patient referred to primary care (patient reported)0.023
Secondary Outcome (binary)Patient referred to cardiology (patient reported)0.002
Secondary Outcome (binary)Patient recommended to primary care (EHR documented)0.279
Secondary Outcome (binary)Patient recommended to cardiology (EHR documented)0.036
Secondary Outcome (binary)EHR documentation of being prescribed a new cholesterol, diabetes, and/or blood pressure medication-0.005
Secondary Outcome (binary)EHR report of cholesterol, glucose, and/or HbA1c test ordered0.104
Outcome TypeOutcome DescriptionAssumed
ICCh
Estimated ICC
A231601CD (OPTI-Surg)a
Primary Outcome (continuous)Patient function as measured by CHAMPS total score (28 items) 8 weeks post-surgery0.0050.027
Secondary Outcome (binary)Postoperative complications 12 weeks post-surgery: grade 0 versus grade 1-50.216
Secondary Outcome (continuous)Quality of life as measured by EQ-5D-5L index (5 items) at 8 weeks post-surgery0.015
Secondary Outcome (continuous)Quality of life as measured by EQ-5D-5L Visual Analog Scale at 8 weeks post-surgery0.055
A191402CDb
Primary Outcome (continuous)Patient knowledge (proportion correct on 12 items)0.100.230
E1Q11c
Primary Outcome (binary)Reproductive health goal concordant management within 3 months from study entry0.100.024
Secondary Outcome (binary)Long-term contraception uptake at 6 months from study entry0.002
URCC-13059 (GAP70+)d
Primary Outcome (binary)Any grade 3–5 toxicity (measured by NCI-CTCAE v4) over 3 months0.100.017
Secondary Outcome (binary)Reduced dose intensity in Cycle 10.025
Secondary Outcome (binary)Dose modified at 3 months related to toxicity0.039
URCC-13070 (COACH)e
Primary Outcome (continuous)Patients’ satisfaction with communication about aging-related concerns (modified HCCQ)0.140.016
Secondary Outcome (continuous)Number of aging-related concerns discussed during the visit (from content analysis of audio recordings)0.138
Secondary Outcome (continuous)Patients’ quality of life (measured with the Functional Assessment of Cancer Therapy scale)0.041
Secondary Outcome (continuous)Caregivers’ quality of life (measured with 12-Item Short Form Health Survey)0.042
Secondary Outcome (continuous)Caregivers’ satisfaction with communication about aging-related patients’ concerns0.013
WF-20817CD (OaSiS)f
Primary Outcome (binary)Effectiveness: 7-day sustained smoking abstinence at 6 months0.030.018
Secondary Outcome (binary)Effectiveness: Short term smoking abstinence (attempt for 1 day) at 6 months0.064
Secondary Outcome (continuous)Implementation (Fidelity to the Intervention): Patients asked if they received 19 cessation services at 14 days; outcome is the sum of services received0.500
WF-1804CD (AH-HA)g
Primary Outcome (binary)Discussion of any non-ideal CVH factor (cholesterol, blood pressure, glucose/hemoglobin A1c, body mass index, smoking, diet, and physical activity) by patient report0.040.016
Secondary Outcomes (binary)Discussion of the 7 individual non-ideal CVH factors by patient report (each factor considered separately)-0.008 to 0.036
Secondary Outcomes (binary)EHR documentation of CVH factor discussions (each factor considered separately)0.150 to 0.310
Secondary Outcome (binary)Patient referred to primary care (patient reported)0.023
Secondary Outcome (binary)Patient referred to cardiology (patient reported)0.002
Secondary Outcome (binary)Patient recommended to primary care (EHR documented)0.279
Secondary Outcome (binary)Patient recommended to cardiology (EHR documented)0.036
Secondary Outcome (binary)EHR documentation of being prescribed a new cholesterol, diabetes, and/or blood pressure medication-0.005
Secondary Outcome (binary)EHR report of cholesterol, glucose, and/or HbA1c test ordered0.104
a

For A231601, ICCs for continuous outcomes were estimated using a linear mixed model, and the binary ICC was calculated using a random intercept logistic model.12

b

For A191402, the ICC estimate comes from the primary publication,17 and the linear mixed model also included baseline patient characteristics.

c

For E1Q11, ICCs were estimated using a GEE with exchangeable correlation and a logit link.37

d

For URCC-13059, ICCs were estimated using a linear mixed model (ie, on the proportion scale).

e

For URCC-13070, ICCs were estimated using a linear mixed model; for longitudinal outcomes, a constant ICC over time was assumed.

f

For WF-20817CD, binary ICCs were calculated using a random intercept logistic model,12 and the continuous ICC was estimated using a linear mixed model.

g

For WF-1804CD, ICCs were estimated using a GEE with exchangeable correlation and a logit link.

h

There were no ICCs assumed for secondary outcomes.

Discussion

When designing a CRT, having a good estimate of the ICC(s) is critical because even small changes in the ICC can have a large impact on the required sample size.7 Mis-specification of the ICC is common and can lead to underpowered trials or trials that are unnecessarily large. The literature can be a valuable resource for deciding on an appropriate ICC estimate, but not all trials report the ICC for the primary outcome (even though it is recommended8), and very few report ICCs for secondary outcomes. Therefore, the purpose of this paper was to fill that gap for CRTs conducted within the NCORP network, providing a resource for future trial development.

Historically, many trials assume an ICC of less than or equal to 0.10 for power and sample size calculations to account for correlation, particularly at the site level, and this was true for 8 of the 9 trials described in the current article. Although this is appropriate for many outcomes (as seen in Tables 2 and 3), this is not true across the board. This highlights the importance of thinking carefully about the particular outcome being used in the trial, and not relying on the assumption that an ICC greater than 0.10 is unlikely. Although not always true, there is a pattern suggesting that ICCs are higher at the provider level than for the same outcome at the site level. For example, in the TrACER trial,12 when looking at whether colony-stimulating factor was prescribed to patients, the ICC was higher at the provider level than the site level for all 3 risk groups (0.235 vs 0.139, 0.381 vs 0.186 and 0.114 vs 0.037). This should be a consideration for future trials relying on a provider-level ICC. Similarly, ICCs at the site level seem to be higher if the particular outcome relies in some way on the provider rather than just on the individual patient. For example, in the AH-HA trial,13 when discussions about cardiovascular health factors are patient-reported, the ICCs ranged from -0.008 to 0.036. However, when looking at whether the discussion was documented in the EHR (which relies on the provider), the ICCs ranged from 0.150 to 0.310. In addition, in the COACH study,14 where oncologists were provided with a geriatric assessment summary and recommendations, the ICC for the number of aging-related concerns discussed during the visit was 0.138. These higher ICCs are likely related to individual physician practice patterns, so this needs to be considered for any outcomes that rely on a specific action taken by a provider nested within sites.

Another particularly high ICC was seen in the OaSiS trial,15 when fidelity to the intervention was considered by summing the number of cessation services received as reported by patients. This outcome had an ICC of 0.500, which is exceptionally high, but also makes sense because it is a fidelity measure and would be expected to have higher correlation within clusters by design. However, this is critical to keep in mind when designing implementation trials where fidelity measures are common. Finally, a high ICC was seen in A191402CD16 for the primary outcome of patient knowledge. In the primary analysis, the ICC was 0.23, but in a post hoc analysis with a single site removed, the ICC went down to 0.08. This demonstrates how sensitive the ICC estimate can be to individual clusters and outliers. Although this is hard to anticipate at the design phase, it is worth noting.

Although this article includes all available ICC data from CRTs conducted within NCORP to date, there are some limitations. We used a pragmatic approach to data collection, meaning that no individual data were collected centrally for this study, and we relied on study statisticians to report summary statistics. The ICC estimation strategy was therefore at the discretion of the study statistician, so ICCs were not all calculated in the same way, measures of uncertainty were not reported, and ICCs for some binary outcomes are reported on the logistic scale rather than the proportions scale. However, information on the method used by each study is documented in footnotes in the tables reporting the ICC estimates to facilitate interpretation. In addition, cluster randomization versus individual randomization was determined by review of ClinicalTrials.gov records, so it is possible that some misclassification occurred and a cluster randomized trial was missed. Finally, all trials included in this article were conducted within the NCORP network, so these results may not be generalizable to CRTs in general.

Conclusion

Having an accurate ICC estimate is an important component for designing a CRT, so this paper can serve as a reference for future trial development within NCORP. Although ICCs are relatively small for many outcome cluster combinations, that is not always the case, so consideration should always be given to the specific outcome of interest, trial design, and type of cluster. Particular attention should be given to intervention fidelity measures, ICCs at the provider level, and outcomes that rely on the provider even if the ICC is at the site level to ensure that the assumed ICC is large enough to ensure an adequately powered trial.

Funding

This work was supported in part by the National Institutes of Health grant numbers National Cancer Institute (NCI) UG1CA189824 (Wake Forest NCORP RB); U10CA180822 and UG1CA189867 (NRG Oncology NCORP RB); U10CA180882 and UG1CA189823 (Alliance NCORP RB); UG1CA189974 and UG1CA180819 (SWOG NCORP RB); UG1CA189961 (University of Rochester NCORP RB); U10CA180820 and U10CA180794 (ECOG-ACRIN NCORP RB).

Monograph sponsorship

This article appears as part of the monograph “Statistical and Practical Considerations in Design and Analysis of Clinical Trials with Patient-Centered Outcomes,” sponsored by the National Cancer Institute (NCI) and the following NCI Community Oncology Research Program (NCORP) Research bases: NRG Oncology (UG1CA189867), Alliance (UG1CA189823), SWOG (UG1CA189974), ECOG-ACRIN (UG1CA189828), Wake Forest University (UG1CA189824), University of Rochester (UG1CA189961), and Children’s Oncology Group (5UG1CA189955-11).

Conflicts of interest

There are no conflicts of interest to report.

Data availability

No individual data were collected centrally as a part of this study, so there are no data to share.

References

1

Murray
DM.
 
The Design and Analysis of Group Randomized Trials
.
Oxford University Press
;
1998
.

2

Donner
A
,
Klar
N.
 
Design and Analysis of Cluster Randomization Trials in Health Research
.
Arnold
;
2000
.

3

Hemming
K
,
Taljaard
M.
 
Key considerations for designing, conducting and analysing a cluster randomized trial
.
Int J Epidemiol
.
2023
;
52
:
1648
-
1658
.

4

Murray
DM
,
Taljaard
M
,
Turner
EL
,
George
SM.
 
Essential ingredients and innovations in the design and analysis of group-randomized trials
.
Annu Rev Public Health
.
2020
;
41
:
1
-
19
.

5

Eldridge
SM
,
Ukoumunne
OC
,
Carlin
JB.
 
The intra‐cluster correlation coefficient in cluster randomized trials: a review of definitions
.
Int Statistical Rev
.
2009
;
77
:
378
-
394
.

6

Turner
EL
,
Li
F
,
Gallis
JA
,
Prague
M
,
Murray
DM.
 
Review of recent methodological developments in group-randomized trials: part 1—design
.
Am J Public Health
.
2017
;
107
:
907
-
915
.

7

Tishkovskaya
SV
,
Sutton
CJ
,
Thomas
LH
,
Watkins
CL.
 
Determining the sample size for a cluster-randomised trial using knowledge elicitation: Bayesian hierarchical modelling of the intracluster correlation coefficient
.
Clin Trials Lond Engl
.
2023
;
20
:
293
-
306
.

8

Campbell
MK
,
Piaggio
G
,
Elbourne
DR
,
Altman
DG
,
for the CONSORT Group
.
Consort 2010 statement: extension to cluster randomised trials
.
BMJ
.
2012
;
345
:
e5661-e5661
.

9

Murray
DM.
 
Influential methods reports for group-randomized trials and related designs
.
Clin Trials
.
2022
;
19
:
353
-
362
.

10

Geiger
AM
,
O’Mara
AM
,
McCaskill-Stevens
WJ
,
Adjei
B
,
Tuovenin
P
,
Castro
KM.
 
Evolution of cancer care delivery research in the NCI community oncology research program
.
JNCI J Natl Cancer Inst
.
2020
;
112
:
557
-
561
.

11

Wu
S
,
Crespi
CM
,
Wong
WK.
 
Comparison of methods for estimating the intraclass correlation coefficient for binary responses in cancer prevention cluster randomized trials
.
Contemp Clin Trials
.
2012
;
33
:
869
-
880
.

12

Ramsey
SD
,
Bansal
A
,
Sullivan
SD
, et al.  
Effects of a guideline-informed clinical decision support system intervention to improve colony-stimulating factor prescribing: a cluster randomized clinical trial
.
JAMA Netw Open
.
2022
;
5
:
e2238191
.

13

Foraker
RE
,
Davidson
EC
,
Dressler
EV
, et al.  
Addressing cancer survivors’ cardiovascular health using the automated heart health assessment (AH-HA) EHR tool: initial protocol and modifications to address COVID-19 challenges
.
Contemp Clin Trials Commun
.
2021
;
22
:
100808
.

14

Mohile
SG
,
Mohamed
MR
,
Xu
H
, et al.  
Evaluation of geriatric assessment and management on the toxic effects of cancer treatment (GAP70+): a cluster-randomised study
.
Lancet
 
2021
;
398
:
1894
-
1904
.

15

Foley
KL
,
Miller
DP
,
Weaver
K
, et al.  
The OaSiS trial: a hybrid type II, national cluster randomized trial to implement smoking cessation during CT screening for lung cancer
.
Contemp Clin Trials
.
2020
;
91
:
105963
.

16

Tilburt
JC
,
Zahrieh
D
,
Pacyna
JE
, et al.  
Decision aids for localized prostate cancer in diverse minority men: primary outcome results from a multicenter cancer care delivery trial (Alliance A191402CD)
.
Cancer
.
2022
;
128
:
1242
-
1251
.

17

Schumacher
JR
,
Zahrieh
D
,
Chow
S
, et al.  
Increasing socioeconomically disadvantaged patients’ engagement in breast cancer surgery decision-making through a shared decision-making intervention (A231701CD): protocol for a cluster randomised clinical trial
.
BMJ Open
.
2022
;
12
:
e063895
.

18

Chang
GJ
,
Gunn
H
,
Barber
A
, et al.  
Surgical care and outcomes in older patients with cancer through implementation of a pre-surgical toolkit (OPTI-Surg): interim results of a phase III cluster randomized trial (Alliance A231601CD)
.
J Clin Oncol
 
2023
;
41
:
1541
-
1541
.

19

Pacyna
JE
,
Kim
S
,
Yost
K
, et al.  
The comparative effectiveness of decision aids in diverse populations with early stage prostate cancer: a study protocol for a cluster-randomized controlled trial in the NCI Community Oncology Research Program (NCORP), Alliance A191402CD
.
BMC Cancer
.
2018
;
18
:
788
.

20

Patel
A
,
Lee
JW
,
Zaren
HA
, et al.  
Cluster-randomized trial to evaluate the implementation of reproductive health care in cancer care delivery in community oncology practices: results from ECOG-ACRIN E1Q11
.
J Clin Oncol
.
2022
;
40
:
1519
-
1519
.

21

Hershman
DL
,
Bansal
A
,
Sullivan
SD
, et al.  
A pragmatic cluster-randomized trial of a standing order entry intervention for colony-stimulating factor use among patients at intermediate risk for febrile neutropenia
.
J Clin Oncol
.
2023
;
41
:
590
-
598
.

22

Watabayashi
KK
,
Bell-Brown
A
,
Kreizenbeck
K
, et al.  
Successes and challenges of implementing a cancer care delivery intervention in community oncology practices: lessons learned from SWOG S1415CD
.
BMC Health Serv Res
.
2022
;
22
:
432
.

23

Mohamed
MR
,
Kyi
K
,
Mohile
SG
, et al.  
Prevalence of and factors associated with treatment modification at first cycle in older adults with advanced cancer receiving palliative treatment
.
J Geriatr Oncol
.
2021
;
12
:
1208
-
1213
.

24

Presley
CJ
,
Mohamed
MR
,
Culakova
E
, et al.  
A geriatric assessment intervention to reduce treatment toxicity among older adults with advanced lung cancer: a subgroup analysis from a cluster randomized controlled trial
.
Front Oncol
.
2022
;
12
:
835582
.

25

Culakova
E
,
Mohile
SG
,
Peppone
L
, et al.  
Effects of a geriatric assessment intervention on patient-reported symptomatic toxicity in older adults with advanced cancer
.
J Clin Oncol
 
2023
;
41
:
835
-
846
.

26

Mohile
SG
,
Epstein
RM
,
Hurria
A
, et al.  
Communication with older patients with cancer using geriatric assessment: a cluster-randomized clinical trial from the national cancer institute community oncology research program
.
JAMA Oncol
.
2020
;
6
:
196
-
204
.

27

Kehoe
LA
,
Xu
H
,
Duberstein
P
, et al.  
Quality of life of caregivers of older patients with advanced cancer
.
J Am Geriatr Soc
.
2019
;
67
:
969
-
977
.

28

Xu
H
,
Kadambi
S
,
Mohile
SG
, et al.  
Caregiving burden of informal caregivers of older adults with advanced cancer: the effects of rurality and education
.
J Geriatr Oncol
.
2021
;
12
:
1015
-
1021
.

29

Kleckner
AS
,
Wells
M
,
Kehoe
LA
, et al.  
Using geriatric assessment to guide conversations regarding comorbidities among older patients with advanced cancer
.
J Clin Oncol Oncol Pract
.
2022
;
18
:
e9
-
e19
.

30

Flannery
MA
,
Mohile
S
,
Culakova
E
, et al.  
Completion of patient-reported outcome questionnaires among older adults with advanced cancer
.
J Pain Symptom Manage
.
2022
;
63
:
301
-
310
.

31

Gilmore
N
,
Xu
H
,
Kehoe
L
, et al.  
Evaluating the association of frailty with communication about aging‐related concerns between older patients with advanced cancer and their oncologists
.
Cancer
.
2022
;
128
:
1101
-
1109
.

32

Loh
KP
,
Seplaki
CL
,
Sanapala
C
, et al.  
Association of prognostic understanding with health care use among older adults with advanced cancer: a secondary analysis of a cluster randomized clinical trial
.
JAMA Netw Open
.
2022
;
5
:
e220018
.

33

Jensen-Battaglia
M
,
Lei
L
,
Xu
H
, et al.  
Association of oncologist-patient communication with functional status and physical performance in older adults: a secondary analysis of a cluster randomized clinical trial
.
JAMA Netw Open
.
2022
;
5
:
e223039
.

34

Jensen-Battaglia
M
,
Lei
L
,
Xu
H
, et al.  
Communication about fall risk in community oncology practice: the role of geriatric assessment
.
J Clin Oncol Oncol Pract
.
2022
;
18
:
e1630
-
e1640
.

35

Foley
KL
,
Dressler
EV
,
Weaver
KE
, et al. ;
Optimizing Lung Screening Trial Writing Team
.
The Optimizing Lung Screening Trial (WF-20817CD)
.
CHEST
.
2023
;
164
:
531
-
543
.

36

Weaver
KE
,
Dressler
EV
,
Smith
S
, et al.  
Cardiovascular health assessment in routine cancer follow-up in community settings: survivor risk awareness and perspectives
.
BMC Cancer
.
2024
;
24
:
158
.

37

Westgate
PM.
 
A readily available improvement over method of moments for intra-cluster correlation estimation in the context of cluster randomized trials and fitting a GEE-type marginal model for binary outcomes
.
Clin Trials
.
2019
;
16
:
41
-
51
.

This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/pages/standard-publication-reuse-rights)