Adherence to cancer treatment guidelines: influence of general and cancer-specific guideline characteristics

Abstract

Background

Guideline adherence remains a challenge in clinical practice, despite guidelines’ ascribed potential to improve patient outcomes. We studied the level of adherence to recommendations from Dutch national cancer treatment guidelines, and the influence of general and cancer-specific guideline characteristics on adherence.

Methods

Based on data from a national cancer registry, adherence was evaluated for 15 treatment recommendations for breast, colorectal, prostate and lung cancer, and melanoma. Recommendations were selected by representatives of the medical specialist associations responsible for developing and implementing the guidelines. We used multivariable multilevel analysis to calculate mean adherence and variation between individual hospitals.

Results

Mean adherence to the different treatment recommendations ranged from 40 to 99%. Adherence differed only slightly between older and newer guidelines and between recommendations with low, moderate or high levels of evidence (range 79–84% and 77–91%, respectively), while adherence differed more between recommendations for different cancer types (range 54–99%), different treatment modalities (adherence ranged from 40 to 92%) or recommendations that advised against or recommended in favour of particular treatment (adherence ranged from 75 to 98%).

Conclusion

We found significant variation in adherence between different cancer treatment guidelines. While some guideline characteristics that seem to explain this variation may be considered difficult to modify, the potential for variance across cancer types and treatment modalities suggests that adherence could be further improved. At the same time, these results warrant tailored strategies for the improvement of adherence to clinical practice guidelines.

Introduction

Ideally, clinical practice guidelines provide scientifically informed guidance to both practitioners and patients for making decisions about appropriate health care.¹ As such, they have the potential to improve patient outcomes, for instance through promoting effective treatments and discouraging ineffective ones. They thereby enhance consistency of care practices across clinicians and hospitals, and support quality improvement activities.²

Because of the sheer amount of studies, scientific evidence in oncology is rapidly evolving and treatment recommendations in guidelines are frequently adjusted to keep them up to date. However, despite of guidelines’ ascribed potential to improve the quality of care, several studies investigating care practices have reported low adherence rates, ranging from 33 to 61%.^3–7 Moreover, considerable variation in adherence to treatment recommendations has been found between hospitals, some of which may not be explained by differences in disease characteristics, patient risk factors or patient preferences.^8,9 Since adherence to treatment recommendations was found to be associated with improved patient survival in several cancer types (e.g. breast, colon and pancreatic cancer)^4,10–13 non-adherence warrants further scrutiny.

Numerous factors may drive the actual adherence to recommendations in the clinic.¹⁴ These may in the first place be related to characteristics of patients and professionals involved in the clinical decision-making process; in case of patients exhibiting comorbidity or poor general health, for instance, physicians are less likely to follow prevailing treatment recommendations.^15–17 Physicians are, not surprisingly, more likely to follow guidelines with which they are more familiar.¹⁸ In addition, contextual factors may impact adherence, like work pressure, or time and personnel resources experienced by physicians.^13,18–20

Furthermore, adherence may depend on features of the guideline itself, such as, for example, complexity.^17,19,20 There are also indications that implementation is better for recommendations that are based on what is considered a high level of scientific evidence, i.e. supported by results from randomized controlled trials.^19,21–23 The time since the introduction of the recommendation may also prove relevant: although recommendations are continuously at risk of becoming out-of-date in light of scientific progress, adherence may well increase over time.³ Interestingly, adherence may also differ based on whether recommendations advise against or recommend a certain treatment.²⁴ In those that advise against a particular intervention (proscriptive), the reasons for non-adherence may be related to physicians’ practice of their professional responsibility, i.e. their exercise of professional judgement regarding the costs and benefits of a specific treatment for individual patients. In some cases, physicians may choose to go with a patient’s request for a particular therapy that is not recommended, thus preserving the doctor–patient relationship. In those that recommend an intervention (prescriptive), non-adherence may be due to practical constraints, e.g. limited availability or high costs of treatment, as well as to difficulties in adapting recommendations to the individual patient, e.g. because of comorbidity.²⁴

With respect to cancer treatment recommendations, it could be hypothesized that some characteristics specifically related to cancer, for instance which cancer type and treatment modality is involved, may greatly affect their adoption in the clinic. Adherence may be lower, for instance, for cancers with a generally poor prognosis and for treatments that are likely to cause serious side-effects.²⁵ Ongoing study of the impact of these tumour-specific factors on adherence should guide future quality improvement strategies and health services research in oncology.

We therefore aim to study the level of adherence to treatment recommendations made in Dutch national cancer treatment guidelines, and factors affecting this adherence, using data from the Netherlands Cancer Registry for information on treatment provision to patients with cancer of the breast, lung (specifically non-small cell lung cancer, NSCLC) or prostate, with intestinal cancer, or with melanoma.

Methods

Selection of guidelines and recommendations

For the Netherlands, cancer treatment guidelines are included in a national database made available at: www.oncoline.nl. We selected guidelines concerning the most common cancer types²⁶ in force during the years 2007–12. An expert panel meeting was organized with seven representatives of the medical specialist associations that were responsible for developing and implementing the guidelines; the panel consisted of a clinical oncologist, radiotherapist,² urologist, pulmonologist, gastroenterologist and an oncology surgeon. The panel members were asked to identify the most relevant patient profiles for each cancer type, i.e. patient subgroups in terms of, e.g. age and tumour stage for which the guidelines recommend a specific therapeutic management. Relevance was determined on the basis of the expected amount of variation in guideline adherence between hospitals. Larger variations were sought for since these could account for better contrast for distinguishing factors affecting adherence on this level. For the present analyses, we selected those patient profiles with clear-cut recommendations for or against a specific treatment.

The Netherlands Cancer Registry

The evaluation of guideline adherence was carried out using data from the Netherlands Cancer Registry,²⁷ which is hosted by the Netherlands Comprehensive Cancer Organisation (IKNL). For this registry, data managers of IKNL routinely collect information on all Dutch patients diagnosed with cancer from the hospital medical files, including data on patient (age, sex) and tumour characteristics (cancer type, tumour site, stage at diagnosis), first-line treatment (modality, date of provision) and all hospitals involved (as location of diagnosis or treatment). In addition, specific tumour characteristics are recorded, such as hormone receptor status in breast cancer and Gleason score in prostate cancer. The study design, data abstraction process and storage protocols were approved by the review board of the Netherlands Cancer Registry.

Patient selection

Based on the information from the Netherlands Cancer Registry, we identified all patients aged ≥18 who were diagnosed with either one of the aforementioned five cancer types between January 2007 and December 2012. For breast cancer, we only included women. Patients’ postal codes were used to approximate socioeconomic status.

Treatment

First-line cancer treatments were analysed according to the main therapeutic modalities of surgery, chemotherapy, radiotherapy and hormone therapy. Chemotherapy included all classical chemotherapeutic agents, while radiotherapy comprised all forms of external beam as well as internal radiation (also known as brachytherapy). Considering chemoradiation, provision was considered concomitant whenever the radiotherapy and chemotherapy were administered within 1 month (30 days) following the other, and in all other cases, chemoradiation was deemed sequential.

Statistical analyses

For each guideline recommendation that was selected for evaluation, we identified those cases from the cancer registry database that matched the corresponding patient profiles. For guideline recommendations that were issued in the course of the study period, evaluation started in the first calendar year following the year of issue and, conversely, for recommendations that became obsolete, evaluation ended the year prior to their removal or alteration in the guideline text.

To account for clustering of adherence within hospitals, we applied multivariable multilevel analyses to estimate the mean percentage of patients who were managed in accordance with the guideline. In the multilevel model, we discerned two levels: the hospital and the individual patient. Overall adherence to the guidelines was calculated with 15 dummy (0/1) variables for each recommendation, both at the patient and the hospital level. We adjusted estimates for differences in hospital size and case-mix, i.e. differences between hospitals in patient and tumour characteristics. To avoid overcorrection, potential confounders were first tested in separate multilevel models for each recommendation, and those resulting in a P values <0.05 were entered into the final multilevel model. The mean percentage of adherence for each recommendation was calculated based on the estimate of the dummy variables at the patient level. To estimate the variation in adherence we calculated a 95% confidence interval (95% CI) using the estimates of the dummy variables at the hospital level.

We then estimated the effect of several general guideline characteristics (recommendation for or against a treatment, older/newer guidelines, level of evidence) as well as cancer-specific characteristics (cancer type, treatment modality) by adding them to the multilevel model one by one, and calculating the mean percentage of adherence and 95% CI interval for these groups. In testing differences in mean adherence for variables with more than two categories, we set the middle category as reference. All multilevel analyses were performed with MLwiN 2.30 in Stata/SE 13.1^®.

Results

For 15 of the 31 patient profiles the panel members selected, clear-cut recommendations for or against a specific treatment were provided in the guidelines. These recommendations were selected for the present analyses. They included five recommendations for breast cancer, four for lung cancer, three for prostate cancer, two for colorectal cancer and one for melanoma (see table 1). Six recommendations involved radiotherapy, five involved chemotherapy, four involved hormonal therapy and two involved surgery (including sentinel node biopsy).

Table 1

Characteristics of selected treatment recommendations

#	Type	Profile	Treatment	Recommendation	Start	Level of evidence†	No patients	No hospitals
1	Melanoma	pT1a	Sentinel node biopsy	Proscriptive	2005	2	12 839	91
2	Colon	Stage III	Chemotherapy, adjuvant	Prescriptive	2008	1	8787	91
3	Rectal	T2	Radiotherapy, neoadjuvant	Prescriptive	2008 (–2014)	Moderate	1941	85
4	Breast	Stage II/III, age>70, ER/PR+ and surgery	Hormonal therapy	Prescriptive	2005	1	7697	90
5	Breast	Stage II/III, age>70, ER/PR+, no surgery	Hormonal therapy	Prescriptive	2005	1	2842	84
6	Breast	Stage II/III, age>70, ER/PR+, and surgery	Chemotherapy	Proscriptive	2005	No	7697	90
7	Breast	Postmenopausal, N0, tumour size 1, 1–2 cm, grade II, HER2-, ER/PR+	Hormonal therapy	Prescriptive	2008	1	4064	90
8	Breast	T2N0–2 age <70	Chemotherapy	Prescriptive	2005	1	15 622	91
9	Prostate	Locally advanced high-risk, Gleason >7	Radiotherapy	Prescriptive	2007	4	3027	82
10	Prostate	Locally advanced high-risk, Gleason >7	Hormonal therapy after radiotherapy	Prescriptive	2007	2	1685	82
11	Prostate	Locally advanced high-risk, Gleason ≤7	Radiotherapy	Prescriptive	2007(–2014)	No	4063	84
12	Lung‡‡	Stage III	Chemotherapy and radiotherapy	Prescriptive	2004	1	13 747	89
13	Lung‡‡	Stage III	Chemotherapy and radiotherapy given subsequently	Prescriptive	2004(–2011)	1	1168	54
14	Lung‡‡	Stage I/II	Surgery	Prescriptive	2004	No	12 728	89
15	Lung‡‡	Stage I/II, no surgery	Radiotherapy	Prescriptive	2004	2	4365	88

‡

Level of evidence indicates the amount of scientific evidence behind a recommendation made in a guideline. It ranges from 1 (strong evidence) to 4 (weak evidence). For some profiles the level of evidence was not specified, or was described in words, e.g. ‘moderate’.

‡‡

Non-small cell lung carcinoma.

Mean adherence to the treatment recommendations ranged from 40 to 99% (see figure 1). When looking at the effect of general guideline characteristics, the two recommendations that advised against a treatment had considerably higher rates of adherence than recommendations that recommended a specific treatment (98 vs. 75%, P < 0.001) (see figure 2). The impact of guidelines’ age was smaller. Adherence to older guidelines issued in 2004/05 was 84 vs. 79% for newer guidelines issued in 2006/07 (P < 0.001). The effect of the guidelines’ levels of evidence was also moderate, ranging from 77% in recommendations based on high levels of evidence (Level 1) and 80% in recommendations based on low levels of evidence, to 91% in those based on medium levels of evidence (level 2, P < 0.001).

Figure 1

Guideline adherence and 95% confidence interval for inter-hospital variation in adherence for 15 recommendations; Legend: # Recommendation number (see Table 1)

Figure 2

Guideline adherence and 95% confidence interval for inter-hospital variation in adherence by guideline characteristics; Legend: Prescriptive: recommend a certain treatment. Proscriptive: advise against a certain treatment. 2004/05 and 2007/08: year of issue of guideline

When looking at guidelines’ cancer-specific characteristics, the effect of cancer type was large, ranging from a mean adherence of 53% in lung cancer to 99% in melanoma. Type of treatment also affected adherence, ranging from a mean adherence of 40% for recommendations involving chemoradiation to 92% in those involving hormonal therapy (figure 2).

Discussion

In this study, overall adherence to recommendations in cancer treatment guidelines was associated with both general and cancer-specific guideline characteristics. With respect to the general characteristics, adherence was significantly higher for recommendations that advise against a treatment. The effects of the guidelines’ age and the levels of evidence underlying recommendations were modest. The cancer-specific attributes, cancer type and treatment modality, showed much larger effects; while adherence was relatively high in the selected recommendations for breast cancer and melanoma, rates were relatively low in the selected recommendations for lung cancer or those involving (chemo)radiotherapy.

Using a nationwide cancer registry, we were able to examine the actual adherence to several cancer treatment guidelines over a number of years. The study results were obtained in unselected samples of both patients and hospitals, and did not depend on self-reported adherence measures, which are likely to be biased.²⁸ Although we lacked information on individual physicians, it could be assumed that most treatment decisions are nowadays made within a multidisciplinary team, and this should be largely adjusted for by the hospital level. Since data were routinely collected for first-line therapy, information on follow-up or postponed treatments were missing for our analyses. This could have affected outcomes especially in prostate cancer, where treatments may be postponed for several years or treatment is provided outside of the hospital, as is generally the case with hormone therapy. Despite of this, adherence was generally high for treatment recommendations regarding hormonal therapy.

As the available data did not permit consideration of all potential exceptions to the guideline recommendations, complete adherence was not be expected. For instance, although chemoradiation is generally recommended for Stage III NSCLC patients (Profile 12), patients in a poor condition and those who suffer from cancer spread in the pleural cavity are explicitly exempt from this advice. We could not exclude these patients in our data. A previous study showed that almost 60% of patients did not qualify for concurrent chemoradiation.²⁹ Besides, in some cases physicians may deliberately choose not to follow a treatment recommendation because of the individual situation of the patient.

Our findings do largely correspond with prior research results. Regarding hormonal therapy in elderly patients with hormone-receptor positive breast cancer (Profiles 4 and 5), Weggelaar et al.³⁰ found that 78–81% of Dutch patients aged over 70 years received hormone therapy during the time period 2001–06, which was comparable to the 78–87% we observed. As to neoadjuvant radiotherapy in patients with T2 rectal cancer (Profile 3), van Leersum et al.³¹ reported rates between 80 and 90% for 2009–11, which corresponds to the 90% in our analysis. Considering chemotherapy in Stage III colon cancer (Profile 2), results obtained by several studies were likewise in line with the 64% we found,^32–37 although the rates reported by US-based studies were considerably higher, ranging from 72% to almost 100%.^3,38,39 While the rate for surgery in Stage I/II NSCLC (Profile 14) was similar to the estimate presented by Landrum et al.³⁸ (67% vs. 65%), there were substantial differences for other treatment modalities in lung cancer. For the USA, 77% of patients with NSCLC (mostly Stage III) received concurrent chemoradiation (Profiles 12 and 13) in 2006–07,⁴⁰ which was considerably higher than the 16% (47% of 40% receiving chemotherapy and radiation) in the Dutch setting. However, this difference may largely be explained by the fact that US guidelines at that time recommended concurrent chemoradiation, while in the Dutch guidelines sequential provision was recommended up to 2011. In Spanish patients with Stage I/II NSCLC without surgery, 30% received radiotherapy (Profile 9),⁶ which was substantially lower than the rate we found (64%). This may be explained by the relatively swift adoption of stereotactic (precision) radiotherapy in the Dutch oncology setting.⁴¹

Caution is warranted in extrapolating our findings to other guidelines. Significant effects were obtained for 15 treatment recommendations that were selected because of the anticipated amount of variation in adherence. So, the adherence and variation we found may not be representative for general treatment recommendations in oncology guidelines or for the cancer types and treatments studied.

In conclusion, we found considerable variation in adherence to different recommendations made in cancer treatment guidelines. The observed variance across cancer types and treatment modalities at least suggests that adherence could be further improved. Our analysis identified potential determinants of adherence which may help to improve the actual use of guidelines in clinical practice. At the same time, these results warrant tailored strategies for the improvement of adherence to clinical practice guidelines.

Key points

• We found that mean adherence to different recommendations made in cancer treatment guidelines ranged from 40 to 99%.

• Adherence differed substantially across cancer types and treatment modalities.

• This warrants tailored strategies for improvement of adherence to clinical practice guidelines.

Acknowledgements

This study was funded by the National Health Care Institute (Zorginstituut Nederland). They had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Funding

This study was funded by Zorginstituut Nederland.

Conflicts of interest: None declared.