Value of individual surgeon performance metrics as quality assurance measures in oesophagogastric cancer surgery

Background Surgeon‐level operative mortality is widely seen as a measure of quality after gastric and oesophageal resection. This study aimed to evaluate this alongside a compound‐level outcome analysis. Methods Consecutive patients who underwent treatment including surgery delivered by a multidisciplinary team, which included seven specialist surgeons, were studied. The primary outcome was death within 30 days of surgery; secondary outcomes were anastomotic leak, Clavien–Dindo morbidity score, lymph node harvest, circumferential resection margin (CRM) status, disease‐free (DFS), and overall (OS) survival. Results The median number of annual resections per surgeon was 10 (range 5–25), compared with 14 (5–25) for joint consultant teams (P = 0·855). The median annual surgeon‐level mortality rate was 0 (0–9) per cent versus an overall network annual operative mortality rate of 1·8 (0–3·7) per cent. Joint consultant team procedures were associated with fewer operative deaths (0·5 per cent versus 3·4 per cent at surgeon level; P = 0·027). The median surgeon anastomotic leak rate was 12·4 (range 9–20) per cent (P = 0·625 versus the whole surgical range), overall morbidity 46·5 (31–60) per cent (P = 0·066), lymph node harvest 16 (9–29) (P < 0·001), CRM positivity 32·0 (16–46) per cent (P = 0·003), 5‐year DFS rate 44·8 (29–60) per cent and OS rate 46·5 (35–53) per cent. No designated metrics were independently associated with DFS or OS in multivariable analysis. Conclusion Annual surgeon‐level metrics demonstrated wide variations (fivefold), but these performance metrics were not associated with survival.


Introduction
There is considerable evidence that there is a relationship between increasing volume and decreasing surgical mortality following oesophagogastric resection for cancer. The evidence is stronger for institutional outcomes than for individual surgeons 1 -3 . Specialist multidisciplinary team (MDT) expertise has been reported to improve patient outcomes 4 -6 , but remains untested by a randomized trial.
Quality assurance metrics in surgery have traditionally included operative mortality within 30 days of an operation. The UK National Oesophago-Gastric Cancer Audit includes other variables as indicators of surgical quality related to lymph node harvest, circumferential resection margin (CRM) involvement and duration of hospital stay, although not disease-free (DFS) or overall (OS) survival.
The aim of this study was to evaluate all of the compound metrics of surgical quality assurance at surgeon and unit level, using time frames of 1-, 3-and 5-year survival as end-points.   Diagnosis and staging was undertaken locally, coordinated via three local weekly MDT meetings. All patients deemed suitable for curative treatment were discussed at a weekly regional network MDT meeting. Integral to the new surgical model was the establishment of an enhanced recovery programme 7 based on established principles in colorectal surgery 8 .

30-day mortality
Data regarding the oesophageal and gastric cancer workload were collected using a combination of a prospectively developed database in combination with MDT records and review of hospital records. Pathological variables were recorded from histopathology reports issued at the time of surgery. CRM status was defined using the Royal College of Pathologists guidelines 9,10 . Measures of outcome included postoperative morbidity and mortality, length of hospital stay and survival at 1, 3 and 5 years from diagnosis. Patients were followed up at regular intervals of 3 months for the first year and 6 months thereafter. In the event that patients developed symptoms suggestive of recurrent disease, investigations were undertaken sooner. Follow-up surveillance was conducted for 5 years or until death, whichever was sooner. Dates and causes of death were obtained from the Wales Cancer Intelligence and Surveillance Unit and from the Office for National Statistics. Regional ethical approval was sought, but deemed unnecessary because the study was considered to be service evaluation.
All patients had management plans individually tailored according to factors relating to both patient and stage of disease 11 . Staging included CT, endoscopic ultrasonography, CT-PET and staging laparoscopy as appropriate. The South East Wales MDT treatment algorithms for oesophageal and gastric cancer have been described previously 12,13 . Operative morbidity was graded in accordance with the Clavien-Dindo classification 14,15 . Particular emphasis was placed on the incidence of morbidity of grade III or higher. Definitive chemoradiotherapy was offered to patients with localized squamous cell carcinoma and to patients with adenocarcinoma deemed unsuitable for surgery because of disease extent and/or medical co-morbidity 16,17 .

Statistical analysis
Grouped data were expressed as median (i.q.r.) values, and non-parametric statistical methods were used. Continuous data were compared using the Mann-Whitney U test, and categorical data using the χ 2 test or Fisher's exact test when the number of events was low. A non-parametric two-sample test on the equality of medians was carried out. Differences were deemed to be statistically significant when the P value was less than 0⋅050.    DFS for all patients was calculated by measuring the interval from a landmark time of 6 months after diagnosis to the date of recurrence. This approach has been adopted in previous randomized trials 18 , to allow for the variable interval to surgery following diagnosis, depending on whether neoadjuvant therapy was prescribed. Events resulting in a failure to complete curative treatment, such as not proceeding to surgery, open and close laparotomy, palliative resection, in-hospital mortality and disease progression during neoadjuvant chemotherapy, were assumed to have occurred at this landmark time, to maintain the intention-to-treat analysis. Overall survival was measured from the date of diagnosis. Cumulative survival was calculated according to the Kaplan-Meier method; differences between groups were analysed with the log rank test. Proportional hazard plots were created and Schoenfeld residuals were calculated to confirm that the proportional hazard assumption was appropriate for overall survival. Univariable analyses involving potential factors influencing survival were examined initially by the life-table method  of Kaplan and Meier, and those with associations where P < 0⋅010 were retained in a Cox proportional hazards model using forward conditional methodology to assess the prognostic value of individual variables. All statistical analysis was performed using IBM ® SPSS ® statistics v25.0.0.0 (IBM, Armonk, New York, USA) with extension R.

Table 5 Univariable and multivariable analysis of clinicopathological factors and complication markers, overall and disease-free survival in patients with oesophageal cancer
During follow-up, 122 patients (23⋅2 per cent) developed cancer recurrence, and 213 (40⋅6 per cent) died.

Collective, annual and 3-year measures of surgical quality assurance
Complete characteristics related to quality assurance and outcome measures are shown in Table 3.  Table 3). The median annual surgeon-level mortality rate was 0 (0-9) per cent versus an overall network annual rate of 1⋅8 (0-3⋅7) per cent.  Table 3).

Survival analysis
Univariable and multivariable survival analyses relating to all patients are shown in Tables 4-6. There was no relationship between OS or DFS and operating surgeon (Fig. 1).

Discussion
This study examined compound-level clinical outcome metrics across an UGI cancer network. The principal finding was that surgeon-level annual data varied markedly. Operative mortality varied fivefold, anastomotic leak and overall morbidity twofold, lymph node harvest threefold and CRM positivity threefold. These annual variations resulted in a 5-year cumulative OS rate that varied by nearly 50 per cent, and a 5-year DFS rate that varied by about 30 per cent. Three-year outcome measures demonstrated less variation than yearly measures and may be a superior reporting metric for surgical performance.
A consultant team-focused operative approach to patients with a high-risk profile was sevenfold safer in terms of operative mortality within 30 days. Over a 3-year period, the operative mortality rate varied by 1⋅5 per cent, anastomotic leak by 2⋅2 per cent, overall morbidity by 13⋅9 per cent, lymph node harvest by 29 per cent, CRM positivity by 4⋅2 per cent, and 5-year cumulative OS by 3⋅4 per cent. The hypothesis that there was no significant intersurgeon variation related to operative mortality was supported. Three-year time frames provided a more balanced and uniform measure of performance than annual snapshots.
Clinical performance and patient outcome measures can be used to improve patient safety and clinical effectiveness. Public reporting of these metrics, such as individual surgeon outcome data, is designed to demonstrate transparency to consumers, allowing comparison and a sense of competition. For those involved in the organization and delivery of healthcare, these metrics can be used to set performance targets that may be associated with financial rewards or penalties at local, regional or national level.
In the context of UGI cancer surgery, operative mortality has been the outcome measure made publicly available in the UK 3 . The publication of mortality data as an indicator of quality of clinical care, however, may make some surgeons reluctant to operate on high-risk patients 19 . Because surgical mortality rates are extremely low (about 2 per cent), one extra death has a notable impact on a surgeon's performance in a year, and risk-adjustment methods cannot resolve such problems. The findings of this study suggest that 3-yearly measures of quality including operative (margin status and lymph node yield) and postoperative information (Clavien-Dindo grade above II and postoperative death), together with 5-year OS and DFS rates are necessary to measure surgical performance and outcome objectively.
Other UK centres have reported their experience after centralizing oesophagogastric cancer surgery 20 -22 . In terms of compound-level metrics, the reported rates for anastomotic leak were 7⋅3 per cent 20 to 10⋅0 per cent 21 , margin involvement 46⋅0 per cent 20 , LOS 14 days 20 and postoperative mortality between 0 and 3⋅6 per cent 20 -22 . These figures are similar to those reported here and support the notion that higher patient volumes result in improved outcomes 23 . The centralization effect may in part reflect technical performance of the surgeon, but also includes performance of all team members contributing to perioperative care, the recognition and management of complications, and longer-term nutritional support following discharge. Collectively, these features suggest that institutional data may be more useful to healthcare planners than those relating to individual surgeons.
This study has a number of limitations. Data were obtained from a single UK regional cancer network, so it is unclear to what extent the conclusions may apply elsewhere. Relatively few patients underwent a minimally invasive approach for either oesophageal or gastric cancer, as this was introduced in 2017. Conversely, data were collected in a contemporaneous way at all local and regional MDT meetings over a period of over 8 years; survival data were particularly robust because no patients were lost to follow-up and death certification was obtained from the Office for National Statistics.
Improvements in any arena demand the measurement of results. Teams and their performance advance by tracing progress over time and relating performance to rivals both inside and outside their group. Rigorous value measurements (clinical outcomes and costs) are vital steps in refining healthcare. In the current arena of UGI cancer, where operative mortality is now low, this measure alone is no longer a robust predictor of long-term survival nor a reliable measure of surgical performance when examined on an annual basis.