The influence of patient case mix on public health area statistics for cancer stage at diagnosis: a cross-sectional study

Abstract Background Summary statistics comparing the stage at diagnosis of geographically defined populations of cancer patients are increasingly used in public reporting to monitor geographical inequalities but may be confounded by patient case mix. We explore the impact of case-mix adjustment on a publicly reported measure of early stage at diagnosis in England. Methods We analyzed data used for publicly reported statistics about the stage of patients diagnosed with 1 of 11 solid tumours in 2015 in England, including information on cancer site (bladder, breast, colon, rectum, kidney, lung, melanoma, non-Hodgkin lymphoma, ovarian, prostate, endometrial), age, gender, income deprivation and population-based commissioning organization. We investigated how cancer site and other patient characteristics influence organizational comparisons and attainment of early-stage targets (≥60% of all cases diagnosed in TNM stages I–II). Results Adjusting for patient case mix reduced between-organization variance by more than 50%, resulting in appreciable discordance in organizational ranks (Kendall’s tau = 0.53), with 18% (37/207) of organizations being reclassified as meeting/failing the early-stage target due to case mix. Conclusion Summary statistics on stage of cancer diagnosis for geographical populations currently used as public health surveillance tools to monitor organizational inequalities need to account for patient sociodemographic characteristics and cancer site case mix.


Introduction
A cross the world, the increasing disease burden associated with cancer is motivating new approaches to public health surveillance to support cancer prevention and control. Monitoring of stage at diagnosis for cancer patients of organizations responsible for the planning and administration of local healthcare systems (including screening programmes and diagnostic services) may be a promising approach. In England, substantial recent improvements in cancer registration have enabled the publication of annual statistics for such geographically defined populations corresponding to commissioning organizations (known as Clinical Commissioning Groups or CCGs). These summarize, for 11 cancer sites combined, the proportion of patients who are diagnosed at early stage. 1 CCGs typically have a population of about 250 000 residents and a varied sociodemographic profile. As cancer incidence varies by sociodemographic group, and certain cancer sites (e.g. lung and breast cancer) have both contrasting sociodemographic predictors and stage distribution, 2 variation in patient case mix may confound such statistics. Previous evidence raises concerns about chance variation and potential bias from missing data for early-stage indicators, but the potential influence of patient case mix poses additional concerns. 3 We therefore examined the degree by which patient characteristics and cancer site affect observed variation in cancer stage at diagnosis between English CCG populations.
Stage data were highly complete (91%, see Results), and given empirical evidence supporting the appropriateness of complete case analysis when comparing stage at diagnosis between CCGs, 3 we included only patients with valid stage in all analyses.

Statistical analysis
Initially, we assessed how case-mix adjustment (CMA) affected the distribution of early stage at diagnosis between CCGs using multilevel logistic models, including CCG as a random effect. We compared the size of the CCG random effect between models with and without fixed effect case-mix variables. We used the user-written programme meresc to rescale results for fair comparisons between these models. [5][6][7] We subsequently considered five possible specifications of an 'early stage at diagnosis' indicator. Starting from a specification without adjustment for case mix (i.e. emulating the convention in current use), we incrementally considered case-mix factors in order of their impact on between-CCG variance when used as single adjustors. Cancer site had the largest impact when used as a single adjustor (decreasing the degree of variance), followed by deprivation quintile, while gender had minimal impact; adjusting for age at diagnosis increased the between-CCG variance. Therefore, the five possible specifications of the indicators comprised: (1) No CMA (2) Adjustment for cancer site only (3) Adjustment for cancer site and income deprivation quintile (4) Adjustment for cancer site, income deprivation quintile and gender (5) Adjustment for cancer site, income deprivation quintile, gender and age group We calculated the case-mix-adjusted CCG percentage of tumours diagnosed at early stage (TNM stages I-II). We fit fixed-effects logistic models to the whole dataset and recycled the model estimates to predict the stage distribution for each CCG that would have been seen if the case mix in each CCG was the same as the whole national population of those diagnosed with these cancers in that year. 8 We assessed the impact of CMA on observed (crude) CCG percentage of early stage at diagnosis by investigating: (1) Change in the 5th-95th percentile of the CCG range of earlystage proportion. We summarised results graphically by producing paired coordinate plots showing adjusted CCG proportion early stage and adjusted rank on the early-stage indicator under each of the five CMA specifications we considered.

Results
Our dataset included 208 586 patients diagnosed in 2015, distributed across 207 geographically defined CCG populations in England (table 1), of whom 189 632 (91%) with known stage were included in our analysis (see Supplementary table A1 for breakdown of stage categories by cancer site). There were substantial between-area differences in proportions of patients diagnosed with specific cancer sites. For example, in the 5% of CCGs with the fewest lung cancer patients less than one in eight patients (<12%) had lung cancer, compared with more than a quarter of patients (>26%) in the 5% of areas where lung cancer was most common (table 1). There was also substantial between-area variation in the age and deprivation distribution of people diagnosed with cancer (table 1).
Adjusting for cancer site alone (i.e. without any other case-mix adjustor) reduced the size of between-area variance by 76.9% (table 2). Additionally adjusting for deprivation quintile and gender made little further difference, with cumulative reductions of 78.2 and 78.3%. Additionally adjusting for age marginally increased the between-area variance, so that full adjustment (i.e. for cancer site, deprivation quintile, gender and age) reduced the between-area variance by 77.8% compared with the unadjusted model. Although all case-mix variables were associated with stage at diagnosis (P < 0.001), adjusting for the three-patient characteristic variables (deprivation, age, gender) had minimal additional impact on between-area variance beyond adjusting for cancer site alone (Supplementary table A2).
CMA affected the absolute and relative performance of individual CCG ( figure 1). The CCG with the highest unadjusted early-stage proportion appears to have had average performance given its case mix, with between one in five and one in three CCGs having better performance in each of the case-mix-adjusted results ( figure 1B). Adjustment for cancer site, and to a lesser extent deprivation, caused noticeable reordering of CCGs (Supplementary figure A2). There was only moderate agreement between CCG ranks on the crude and the (cancer site-deprivation-gender-age) adjusted indicator (Kendall's tau = 0.53).
CMA changed classification on pay-for-performance targets (60% or more of all patients with monitored cancers being diagnosed in stages I-II) for a number of CCGs (figure 2). After CMA, 37 (18%) of all CCGs were reclassified as either meeting or failing the target of at least 60% of their patients being diagnosed at early stage (table 2), 27 (13%) moving from above to below target and 10 (5%) in the opposite direction ( figure 2).

Discussion
Patient case mix, particularly by cancer site and deprivation, has a substantial impact on summary statistics of early stage at diagnosis for geographically defined populations of cancer patients. Adjusting for case mix substantially reduces between-CCG variance and reclassifies about one in six organizations regarding their target attainment status-with several of them moving above or below target.
Guidelines for the production of indicators recommend CMA when the distribution of patient-level factors varies geographically and these factors are strongly associated with the outcome-both conditions being met in our study context. [13][14][15] Some public reporting schemes, such as for the Cancer Patient Experience Survey in England, now use synchronous reporting of both crude and case-mix-adjusted scores. 16 The crude scores show users the actual performance; interpreting them jointly with the adjusted scores helps judge whether an organization attains higher or lower scores than what could be expected by their patient case mix.
Although our study relates to a specific country setting, it has paradigmatic implications for any country with a developed cancer registration system who would consider public health surveillance of stage at diagnosis of cancer, particularly when such indicators combine information from multiple cancer sites.
We used the same national, population-based data as used in early-stage indicators. The large sample size allows for adequately precise estimation of the impact of CMA. Additional adjustors could be considered, including ethnicity, comorbidity and tumour morphology, but their contribution above and beyond that of the four adjustors we examined is likely to be small.
There is debate about whether adjustment for measures of socioeconomic status is justifiable. Such adjustment can be viewed as 'accepting' of worse outcomes in patients from more deprived communities, yet lack of adjustment produces performance indicators that reflect broader socioeconomic factors than organizational performance. [17][18][19] In the context of our study, this debate is largely redundant, as adjustment for cancer site greatly dominates the degree of changes that result from additional adjustments (including for deprivation, see Results, table 2 and Supplementary Appendices 1 and 2).
The motivating principle for our inquiry is the need for better understanding, and better description to users, of the technical and statistical properties of indicators in current use. The early-stage indicator examined in our study is a summary statistic considering  multiple cancer sites. Summary indicators generally give more precise estimates but can mask variation between sub-populations. For example, one CCG may have a lower than average proportion of advanced stage for patients with lung and a higher than average proportion for breast cancer, and another CCG a higher than average proportion for lung and lower than average for breast, but these genuine differences may average out so that the two CCGs have the same performance on the early-stage indicator. Summary indicators may alert users to broad failings in certain organizations, but have the potential to mask heterogeneity and failings in specific domains. Identifying the full nature of such potential problems and guiding improvement actions requires further detailed enquiry. Our results demonstrate that the case mix of diagnosed patients explains much of the observed variance in area statistics for stage at diagnosis. Informative reporting of such early-stage indicators for geographically defined populations must account for patient case mix, particularly regarding cancer site. Taken together with prior evidence, the findings indicate that early-stage indicators need to account for potential bias from missing data, chance variation and patient case mix. 3 Direct standardization for cancer site alone (e.g. using the distribution of cancer sites among the nationwide cohort of incident patients) would be an easily implementable solution to reduce bias due to both random and systematic differences in case mix in the early-stage indicator, and would provide for a more appropriate public health and health system performance measure. We recommend the introduction of synchronous reporting of both observed (crude) and adjusted proportions of early stage at diagnosis to support the interpretation of such indicators and enable fair organizational comparisons. . The thick black dashed line highlights the performance and ranking of the CCG with the highest observed proportion of early stage at diagnosis without adjusting for case mix: after adjustment for cancer site around 20% of CCGs have a higher proportion early stage than that CCG; after adjustment for other case-mix variables around 30% of the compared organizations have a higher proportion early stage than that CCG. The thick black line highlights the performance and ranking of the organizations with the lowest proportion early stage without adjusting for case mix Figure 2 Observed crude proportion of patients diagnosed at an early stage by CCG against case-mix-adjusted proportion of the same organizational indicator, adjusted for cancer site, deprivation, gender and age group at diagnosis. Grey circles represent CCGs (either above or below target proportion of early stage) which remain similarly classified after CMA, whereas black squares represent re-classified CCGs