Quality of reporting of chemotherapy compliance in randomized controlled trials of breast cancer treatment

Objective: The Consolidated Standards of Reporting Trials statement requires detailed reporting of interventions for randomized controlled trials. We hypothesized that there was variable reporting of chemotherapy compliance in published randomized controlled trials in breast cancer, and therefore surveyed the literature to assess this parameter and determine the study characteristics associated with reporting quality. Methods: Published Phase III randomized controlled trials (January 2005 – December 2011; English language) evaluating chemotherapy in breast cancer were identi ﬁ ed through a systematic literature search. Articles scored 1 point each for reporting of the four measures: number of chemotherapy cycles, dose modi ﬁ cation, early treatment discontinuation and relative dose intensity. Logistic regression identi ﬁ ed study characteristics associated with reporting quality score of ≥ 2. Results: Of the 115 eligible randomized controlled trials, 79 (69%) were published in high-impact journals, 66 (57%) were published since 2008, 43 (37%) reported advanced-stage disease and 37 (32%) were industry sponsored. Relative dose intensity, number of cycles, dose modi ﬁ cation and early treatment discontinuation were reported in 70 (61%), 53 (46%), 65 (57%) and 81 (70%) articles, respectively. Eighty-two (71%) articles showed a quality score of ≥ 2; 25 (22%) articles reported all four compliance measures. Articles published since 2008 ( P = 0.035) and those reporting advanced-stage disease ( P < 0.001) showed signi ﬁ cantly higher quality of compliance. Conclusions: Our results demonstrate variable reporting of chemotherapy compliance in published randomized controlled trials with a modest improvement noted in recent years. Incorporating standards for reporting chemotherapy compliance in scienti ﬁ c guidelines or the journal peer review process may decrease the variability and improve the quality of reporting.


Introduction
For cytotoxic agents, a direct relationship exists between the dose of antitumor drugs and the corresponding antitumor effects. A lack of specificity of the pharmacodynamic effects on tumor cells when compared with normal tissue limits dose escalation, and thus the recommended dose of a specific agent represents a balance between antitumor effects and host tissue toxicity (1).
Dose intensity (DI) is a term that represents the amount of drug administered per unit time (usually expressed in milligram per meter squared per week) and is a commonly used measure of drug exposure.
A related measure, the relative dose intensity (RDI) indicates the actual dose exposure compared with the ideal exposure based on standard guidelines and/or approved indications for drug doses (1,2). RDI is the ratio of the dose administered to the dose planned per the treatment protocol (expressed as a percentage) and is a commonly used measure of compliance with a particular drug/drug regimen. There is wellestablished evidence supporting the significance of maintaining DI and RDI to optimize tumor control and patient outcomes in patients with breast cancer (3)(4)(5)(6). The importance of DI and RDI has also been demonstrated for patients with B-cell lymphomas, non-small cell lung cancer, ovarian cancer and chronic lymphocytic leukemia (2,(7)(8)(9)(10)(11).
The randomized controlled trials (RCTs) are one of the most powerful tools of clinical research and are considered the gold standard for testing new therapies and therapeutic strategies (12,13). The results of RCTs are the basis upon which treatment standards and recommendations are built, and thus detailed knowledge regarding the administration and compliance associated with specific therapeutic interventions is necessary for the adoption of new therapies into clinical practice (14). Although Item 5 of the 2010 Consolidated Standards of Reporting Trials (CONSORT) statement checklist states that RCT reporting should include 'interventions for each group with sufficient details to allow replication, including how and when they were actually administered,' (15) a recent study has shown that RCTs published in major oncology journals do not consistently report essential therapeutic details necessary for the clinical translation of trial findings (16). This might be attributed to the complexity of chemotherapy regimens and a lack of guidance on the quality of reporting chemotherapy compliance. This is of concern, as there has been a substantial increase in the number of oncology RCTs published over the last three decades (17).
Breast cancer is one of the most common malignancies and the second leading cause of cancer-related deaths in women (18). In the Cancer and Leukemia Group B 8541 study, an RCT recruiting patients with Stage II breast cancer receiving adjuvant chemotherapy, moderate and high DI of cyclophosphamide, doxorubicin and 5-fluorouracil was associated with a significantly longer disease-free survival and overall survival compared with the corresponding survival measures for patients receiving low DI at median follow-ups of 3.4 and 9 years (3,6). Reduction of dose below the currently accepted optimal conventional range in patients with breast cancer leads to an inferior outcome and should thus be avoided (3,6).
Given the lack of guidance on the quality of reporting chemotherapy compliance and the heterogeneity in reporting this measure, (19,20) accurate implementation of RCT results can be a major challenge. Similar challenges pertaining to the interpretation, replication and application of RCT results appear to exist in the radiotherapy literature (21). In light of these considerations, we undertook this study to assess the quality of reporting chemotherapy compliance in RCTs involving patients with breast cancer, with a focus on key measures of chemotherapy compliance including RDI. We also sought to identify study characteristics associated with the quality of drug compliance reporting.

Search strategy
A structured literature search was performed to identify breast cancer RCTs published from January 2005 to December 2011 and verified by a library sciences consultant. For the MEDLINE search, we used the keywords 'breast neoplasm' and 'chemotherapy' with the search limited to English language papers. Similar searches were conducted in EMBASE and the Cochrane Central Register of Controlled Trials (CENTRAL). The titles and abstracts of studies shortlisted from the initial search were screened based on predetermined inclusion and exclusion criteria.
Articles eligible for inclusion were Phase III RCTs assessing the efficacy of chemotherapy in the experimental, control or both arms in studies of patients with breast cancer. Chemotherapy could be administered with or without other systemic therapies such as targeted and/ or endocrine therapy. Studies were excluded if they were (i) not Phase III RCTs; (ii) RCTs primarily reporting safety, toxicity or quality of life; (iii) RCTs involving radiation (with or without chemotherapy) and/or surgical intervention; (iv) reviews or meta-analyses; (v) those presenting subgroup analyses; (vi) prognostic studies and (vii) duplicate reports (the first final report in a journal was included).

Data collection
The first author (A.K.A.) extracted data using a standardized data collection sheet that had been previously pilot tested and revised. To assess the quality of reporting chemotherapy compliance, each article was examined for information on the following four measures: number of chemotherapy cycles received (median and/or total), dose modifications (reduction, omission or delay), early treatment discontinuation rate and RDI. Each of the four elements was rated as yes (1 point) or no (0 points) and summed to provide an overall compliance reporting quality rating score (range, 0-4 points). To assess inter-observer variability, the second author (A.H.A.) independently extracted these four elements from a random sample of 60 articles (52% of the total) to ensure reliability of the data extraction process. The inter-observer agreement was >0.9. Any disagreement was resolved through consensus. Further, the intra-class correlation for the final score, calculated using strict definition of absolute agreement rather than consistency, was 0.931.
Each article was also assessed for several clinical, trial and publication characteristics. These included details of the region where the trial was conducted (North America, Europe or others), year of publication, journal (name and impact factor dichotomized at an impact factor of 10), disease stage (early or advanced), interventions (chemotherapy only or mixed), involvement of a cooperative study group and sponsorship. RCTs were classified as relating to early-stage disease if they evaluated neoadjuvant/adjuvant therapy and to advanced-stage disease if they evaluated therapy in the advanced disease setting. Industry support/involvement was determined on the basis of explicit statements in the article along with affiliations of the study authors. Studies that reported the involvement of pharmaceutical companies were classified as 'industry supported,' while those supported solely by non-profit organizations (i.e. governments or foundations) were classified as 'non-industry supported.' Studies for which funding sources were not identified were classified as 'not known.' Although we were primarily interested in cytotoxic chemotherapy compliance, we also evaluated trials that included molecular-targeted or hormonal therapies in the therapeutic regimens. Thus, trials without any molecular-targeted or hormonal therapy were classified as 'chemotherapy only.' If there was a targeted and/or hormonal agent involved in at least one study arm, the trial was classified as 'mixed.' We dichotomized the papers at a quality score of 2 and evaluated the relationship between the study characteristics and an overall quality score of ≥2.

Statistical analysis
Data were collected and imported into SPSS (Version 20.0 for Windows; IBM) for statistical analysis. The data were first analyzed descriptively using frequencies and percentages for categorical data and medians for sample size data. Inter-observer reliability was assessed with the intra-class correlation for absolute agreement. Study characteristics were compared with respect to RDI reporting (yes/no) and quality score (<2 versus ≥2) using Pearson's χ 2 tests. Univariate and multivariate logistic regression modeling was used to generate odds ratios and to determine the association of the study characteristics with better reporting of chemotherapy compliance (quality score ≥2).

Results
Of the 681 articles initially identified, 142 full-text articles were retrieved and analyzed; 115 of the 142 articles satisfied the inclusion and exclusion criteria. These 115 articles comprised our study cohort. The RCT selection process is outlined in Fig. 1.

Description of studies
The 115 RCTs included 170 220 patients with a median sample size of 549 patients; 79 (69%) RCTs were published in high-impact factor journals (Journal of Clinical Oncology, Journal of the National Cancer Institute, New England Journal of Medicine and The Lancet). Sixty-seven (58%) originated in Europe. Seventy-two trials (63%) were conducted in an early-stage setting. Industry funding was reported in 37 (32%) articles; 74 (64%) articles were non-industry funded; the source of funding could not be determined for 4 (3%) articles. Chemotherapy was the sole intervention evaluated in 84 (73%) trials, while 31 (27%) trials evaluated a targeted and/or hormonal agent in one or more of the treatment arms. Significant differences in efficacy were demonstrated between treatment arms in 53 (46%) articles with respect to the primary endpoint. Additional characteristics of the study cohort are listed in Table 1.

Reporting of chemotherapy compliance
Inter-observer agreement for all items pertaining to chemotherapy compliance was >0.90. The intra-class correlation was 0.931 for the final score calculated using the strict definition of absolute agreement rather than consistency. RDI was reported in 70 (61%) articles, while 53 (46%) articles mentioned the total and/or median number of chemotherapy cycles administered (Fig. 2). Dose modifications (including reduction, omission and/or delay of doses) were reported in 65 (57%) articles, while the reasons for dose modifications were described in approximately half of these [33 of the 65 (51%)]. Finally, early chemotherapy discontinuation rates were described in 81 (70%) articles (Fig. 2). The reasons for discontinuation were described in 95% (77 of the 81) of these articles and included adverse events [73 (95%)], progression/relapse [52 (68%)], and other reasons, including death, consent withdrawal, patient wish or protocol violation [59 (77%)].
Reporting of RDI according to study characteristics Table 2 summarizes the number of studies reporting RDI according to the study characteristics. Neither journal impact factor nor year of publication appeared to be important differentiators for reporting RDI (P = 0.20 and P = 0.65, respectively). RDI was reported in a significantly greater number of articles on advanced-stage disease (74%) than early-stage disease (53%; P = 0.021). RDI reporting did not differ substantially by characteristics such as study origin, participation type, involvement of a cooperative study group, industry sponsorship and the type of intervention.

Quality of reporting chemotherapy compliance according to study characteristics
Only 25 (22%) of the 115 RCTs reported all four compliance measures (quality score, 4), whereas 5 (4%) reported none (quality score, 0). The quality score was ≥2 in 82 (71%) articles. When classified by study characteristics, the percentage of reports with a quality score of ≥2 ranged from 58 to 93%. Articles published since 2008 and trials conducted in the advanced-stage setting were associated with higher quality scores (P = 0.035 and P < 0.001, respectively). Journal impact factor, region where the trials were conducted, participation type, involvement of a cooperative group, industry sponsorship and type of intervention were not associated with a higher quality score. Table 3 provides additional details about the quality of reporting chemotherapy compliance in the study cohort and the results of regression analyses performed on eligible reports.

Discussion
To the best of our knowledge, there are no previous studies evaluating the quality of reporting of chemotherapy compliance measures in breast cancer RCTs. A total of 115 published articles of Phase III RCTs investigating various chemotherapeutic agents in patients with breast cancer were analyzed. While there is a definite increase in the reporting of chemotherapy compliance measures in recent years, the overall quality of reporting remains highly variable and inadequate based on our scoring system. The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) defined treatment adherence as being synonymous with compliance, i.e. 'the extent to which a patient acts in accordance with the prescribed interval and dose of a dosing regimen.' (22) Non- Percentages may not add up to 100% owing to rounding up to the nearest integer.
a Trials without any targeted or hormonal therapy were classified as chemotherapy only. If there was a targeted agent involved in at least one study arm, the trial was classified as mixed.  compliance is a problem affecting many therapeutic areas and has been cited as a reason for suboptimal clinical benefit, increased healthcare costs and the development of resistance (23). One of the key reasons cited for the differences between clinical trial efficacy results and the effectiveness of pharmacological interventions in the real-world setting is the highly controlled environment provided by a clinical trial, including structured guidance for treatment administration to a selective patient population. Based on our findings, we postulate that a lack of adequate reporting measures of treatment delivery and compliance may be a contributing factor to this phenomenon.
Characterization and measurement of treatment compliance are challenging to summarize. In this study, we assessed four measures that we considered to be important for the adequate reporting of chemotherapy compliance. Chemotherapy DI and RDI have been shown to be important measures of compliance that are associated with outcome in patients with breast cancer (3,6,24). Optimizing RDI >85% appears to improve the long-term outcome of patients with breast cancer receiving chemotherapy (4). In a retrospective database analysis of 793 patients with a median follow-up period of almost 10 years, patients who received a reduced RDI (<85%) had a significantly lower probability of disease-free survival and overall survival when compared with patients receiving ≥85% RDI (4). The importance of DI has been demonstrated in other malignancies. The 3-year overall survival in patients with diffuse large B-cell lymphomas receiving rituximab combined with cyclophosphamide, doxorubicin, vincristine and prednisone was 92% in the group with RDI above the median and 74% in the group with RDI below the median (7).
Another measure of compliance-the number of cycles administered-may also impact outcome. In a retrospective analysis (1996-2001) of the Henry Ford Health System tumor registry, of the 472 patients receiving adjuvant chemotherapy for early breast cancer, 344 completed all cycles. Completion of chemotherapy cycles was associated with a 5-year survival of 89% when compared with 74% in patients who did not complete all cycles (P = 0.03) (25).
Others studies have noted the suboptimal quality of reporting in oncology RCT publications (26)(27)(28)(29)(30). A recent study by Dodd et al. (31) showed non-adherence to treatment protocols in RCTs published during 2008 in the British Medical Journal, New England Journal of Medicine, Journal of the American Medical Association and The Lancet. The elements reviewed by them included (i) randomization; (ii) initiation, completion and persistence to randomized treatment; (iii) method used for judging compliance, and its justification and analysis and (iv) whether a statistical method addressing non-adherence to treatment protocol was reported in the study. Of the 100 trials reviewed, 98 reported non-adherence to treatment, but the reporting was vague or incomplete. They concluded that 'non-adherence to treatment protocol is widespread among trials and is recognized by some trialists as potentially obscuring treatment efficacy, but statistical methods of analysis to handle such non-adherence typically exclude or censor participants who deviate from the treatment protocol without discussion of the potential bias introduced in such an analysis.' Inadequate reporting of necessary therapeutic details in RCTs published in major oncology journals from 2005 to 2008 has been reported by Duff et al. (16). They assessed the reporting of certain details related to therapy administration, patient monitoring and supportive care provisions in trials. For this purpose, they developed a list of 10 essential elements, including drug name, dose, route of administration, cycle length, maximum number of cycles, premedication, growth factor support, patient monitoring parameters, dosage adjustments for hematologic toxicity and dosage adjustments for organ-specific toxicity. Reporting of these details was assessed in 262 published articles, and only 11% of the articles were found to have reported all the 10 elements.
A previous study reported an improvement in the quality of reporting of 18 of the 22 items detailed in the 2001 CONSORT statement in oncology trials from 2005 to 2009 (29). Our results are in agreement with this finding and demonstrate an improved quality of reporting in oncology RCTs published since 2008, which may be attributed to an increased awareness among researchers and reviewers of publication guidelines.
We found that higher reporting quality was associated with advanced disease studies. This is not surprising since agents/regimens are usually first tested in the metastatic disease setting prior to evaluation in the adjuvant setting. Furthermore, the impact of treatment compliance on toxicity and quality of life are very important in the advanced disease setting, which may lead to better reporting of these measures.
Our study had limitations. We reviewed only published Phase III RCTs and restricted our review to English language publications. Although this may be a potential source for the introduction of bias, removing the English language limit did not yield a significantly greater number of studies. Another limitation relates to the lack of validation of the instrument used to assess the quality of chemotherapy compliance reporting. The robust characteristics of the instrument, which includes only four factors that were easily defined and identified as commonly reported measures of compliance in a pilot review of 10 trials by the authors overcomes this limitation to a certain extent. The cut of quality score of ≥2 reflects better (rather than higher) reporting of chemotherapy compliance. Finally, the reliability of data retrieval was determined to be very high based on the level of agreement between the two reviewers on the overall quality scores.
Inadequate reporting of chemotherapy compliance may result from a lack of relevant standardized guidance. Although we were unable to measure the impact of such a deficiency on patient outcomes in the community setting, it is entirely plausible that patient outcomes have the potential to be affected by inadequate implementation of therapeutic interventions shown to be efficacious in the clinical trial setting. A Cochrane review of 53 publications assessing 16 604 RCTs has advocated the use of the CONSORT statement, which beneficially improves the reporting quality of RCTs (32). Given the results of our study and the findings of others, we advocate the development of guidelines for reporting chemotherapy compliance in the medical literature and adherence to those guidelines in the peer review process.