Abstract
A number of studies have been conducted with the onset of secondary progressive multiple sclerosis as an inclusion criterion or an outcome of interest. However, a standardized objective definition of secondary progressive multiple sclerosis has been lacking. The aim of this work was to evaluate the accuracy and feasibility of an objective definition for secondary progressive multiple sclerosis, to enable comparability of future research studies. Using MSBase, a large, prospectively acquired, global cohort study, we analysed the accuracy of 576 data-derived onset definitions for secondary progressive multiple sclerosis and first compared these to a consensus opinion of three neurologists. All definitions were then evaluated against 5-year disease outcomes post-assignment of secondary progressive multiple sclerosis: sustained disability, subsequent sustained progression, positive disability trajectory, and accumulation of severe disability. The five best performing definitions were further investigated for their timeliness and overall disability burden. A total of 17 356 patients were analysed. The best definition included a 3-strata progression magnitude in the absence of a relapse, confirmed after 3 months within the leading Functional System and required an Expanded Disability Status Scale step ≥4 and pyramidal score ≥2. It reached an accuracy of 87% compared to the consensus diagnosis. Seventy-eight per cent of the identified patients showed a positive disability trajectory and 70% reached significant disability after 5 years. The time until half of all patients were diagnosed was 32.6 years (95% confidence interval 32–33.6) after disease onset compared with the physicians’ diagnosis at 36 (35–39) years. The identified patients experienced a greater disease burden [median annualized area under the disability-time curve 4.7 (quartiles 3.6, 6.0)] versus non-progressive patients [1.8 (1.2, 1.9)]. This objective definition of secondary progressive multiple sclerosis based on the Expanded Disability Status Scale and information about preceding relapses provides a tool for a reproducible, accurate and timely diagnosis that requires a very short confirmation period. If applied broadly, the definition has the potential to strengthen the design and improve comparability of clinical trials and observational studies in secondary progressive multiple sclerosis.
To date, no definition of secondary progressive multiple sclerosis (SPMS) has been universally accepted. Using MSBase, a large multiple sclerosis cohort study, Lorscheider et al. identify an objective SPMS definition based on disability progression and preceding relapses, which provides a reproducible, accurate and timely diagnosis suitable for clinical studies.
Introduction
Conversion to secondary progressive multiple sclerosis (SPMS) is associated with a relatively poor prognosis, which is in part due to the limited effect of the available disease-modifying therapies in progressive disease (Scalfari et al., 2013; Ontaneda et al., 2015). To date, no definition for SPMS has been universally accepted; SPMS is mostly diagnosed in retrospect, based on a history of gradual worsening after an initial relapsing disease course (Lublin et al., 2014). This is partly caused by the fact that relapsing-remitting multiple sclerosis and SPMS form a continuum with the boundary between them being somewhat indistinct, which is also reflected by the latest revision of the disease phenotypes (Katz Sand et al., 2014; Lublin et al., 2014).
The lack of a consistent definition for SPMS has resulted in heterogeneity in inclusion criteria for clinical trials and has limited the ability to study the biomarkers that may characterize this disease course (Koch et al., 2013; Lublin et al., 2014; Ontaneda et al., 2015) For the purpose of clinical trials and studies and to improve clinical care, a validated definition of SPMS is needed.
In general, conversion to SPMS is characterized by irreversible disability progression that is independent of a relapse, although patients with SPMS can still experience relapses. In addition, in most physicians’ opinion, a patient needs to accumulate a minimum level of disability before the diagnosis can be made (Ebers et al., 2008). A pilot study using MSBase, a global observational multiple sclerosis cohort, suggested that identification of the secondary progressive phase by physicians occurs late in the disease course and tends to be more specific but less sensitive than definitions purely based on the Expanded Disability Status Scale (EDSS) (Spelman et al., 2013). However, no systematic validation of objective diagnostic criteria for SPMS has been conducted so far.
In this study, we used MSBase to evaluate a matrix of 576 definitions of SPMS based on different EDSS progression magnitudes, minimum EDSS and Functional System (FS) scores, confirmed progression after various time periods, requirement for confirmation within the FS leading to progression, and correction for relapses preceding and following the diagnosis. Our aim was to establish an objective SPMS definition that is timely and reflective of long-term disability outcomes. The purpose of this definition is to unify inclusion criteria and criteria for SPMS as an endpoint in order to improve comparability among clinical trials and observational studies.
Patients and methods
Ethics statement
The MSBase cohort study (Butzkueven et al., 2006) (registered with WHO ICTRP, ID ACTRN12605000455662) was approved by the Melbourne Health Human Research Ethics Committee, and by the local ethics committees in all participating centres (or exemptions granted, according to applicable local laws and regulations). If required, written informed consent was obtained from enrolled patients, in accordance with the Declaration of Helsinki.
Patients and follow-up
Longitudinal clinical data from 34 154 patients from 113 multiple sclerosis centres in 34 countries were extracted from the MSBase registry in June 2015. The inclusion criteria consisted of the diagnosis of multiple sclerosis or clinically isolated syndrome based on the 2005 or 2010 revised McDonald criteria (Polman et al., 2005, 2011) and availability of the minimum dataset (i.e. patient sex, year of birth, year of the first clinical presentation, multiple sclerosis course, treating centre, a minimum follow-up time of 12 months and at least three clinical visits with recorded EDSS scores and complete information regarding FS scores). We allowed patients already diagnosed with SPMS to enter the study, but excluded patients with primary progressive multiple sclerosis comprising both active and non-active phenotypes as defined by Lublin et al. (2014). The data quality assessment was conducted using a series of procedures to identify any invalid or inconsistent entries, as described elsewhere (Kalincik et al., 2013a, b); only information from centres contributing at least 10 patient records was included as defined a priori in the study protocol, and a date of onset was required for all recorded events. The analysed data were recorded as part of quality clinical practice, mostly at large tertiary multiple sclerosis centres. The usual data entry practice was real-time or near-real time data entry (at the time of clinical visits). The MSBase protocol stipulates a required annual update of the minimum dataset, but patients with less frequent visits were not excluded from the analysis. Data entry portal was either the iMed patient record system or the MSBase online data entry system. The on-study follow-up was defined as the time between the first and the last available EDSS entry. Disability was scored by accredited scorers (Neurostatus certification was required at each centre) using the EDSS, calculated based on FS and ambulation scores. Only EDSS scores recorded more than 30 days after the onset of a preceding relapse were used to identify and confirm progression events.
Definitions of secondary progressive multiple sclerosis
We developed 576 definitions of SPMS, which represent all possible combinations of the following seven criteria:
(i) Based on the suggested use of half-step progression above EDSS step 5.5 and on our previous work (Weinshenker et al., 1996; Kalincik et al., 2015a), we defined the required magnitude of EDSS change as:
(a) three strata: an increase in EDSS by 1.5 points if the last EDSS before conversion to SPMS was 0, an increase by 1 point if the EDSS was between 1 and 5.5, or an increase by 0.5 points if the EDSS was above 5.5,
(b) one stratum: increase by 1 EDSS point, or
(c) one stratum: increase by 2 EDSS points
(ii) A minimum EDSS score at the time of progression:
(a) 0
(b) 3
(c) 4
(iii) A minimum pyramidal FS score at the time of progression:
(a) 0
(b) 2
(iv) Confirmation of disability progression at two or more consecutive visits separated in time by the minimum of:
(a) 3 months
(b) 6 months
(c) 12 months
(d) 24 months
For confirmation of progression, all EDSS scores recorded during the required confirmation period and the first EDSS score recorded after this period were used.
(v) Confirmation within the FS leading to the progression event:
(a) not required, or
(b) required
(vi) Relapse activity prior to the diagnosis of SPMS:
(a) no relapses within 30 days prior to diagnosis or
(b) level (a) and exclusion of any disability progression attributable to relapses
(vii) Relapse activity after the diagnosis of SPMS:
(a) not specified, or
(b) did not exceed 2 relapses over the 24 months following the diagnosis SPMS.
Study design
We used a stepwise approach to identify the best objective definition for SPMS (Fig. 1). In a first step, we selected a sample of 200 patients from the MSBase cohort, who had a complete minimum dataset, a visit density of at least one visit per year and the longest available follow-up time. In this subset, a consensus diagnosis date of SPMS was derived from the independent opinion of three multiple sclerosis neurologists (J.L., K.B. and T.K.) by inspection of the recorded EDSS trajectories and relapse information. Each rater was blinded to the other raters’ assessment and to the patients’ demographic, clinical or paraclinical information (apart from their disability trajectory and incidence of relapses). Consensus was assumed if at least two of the three raters agreed on the diagnosis of SPMS according to the 2013 revisions of the clinical disease course descriptions (Lublin et al., 2014). The date of conversion to SPMS was recorded as the date on which the rater considered the clinical diagnostic criteria of SPMS were first met. In addition, a date on which the rater had sufficient clinical information to confirm the diagnosis was noted. Only patients with a confirmed diagnosis were considered to be secondary progressive and the difference between the two dates was used to determine the period of diagnostic uncertainty. Sensitivity and specificity for all 576 operational definitions and a diagnosis by the treating physician were then tested against the consensus diagnosis.
Study design.
In a second step, we aimed to assess the entire MSBase cohort by applying all definitions of SPMS to the dataset and validating the accuracy of the diagnosis in a proportion of patients with minimum 5-year follow-up after SPMS assignation. We decided to use four post-assignation metrics as follows. The proportion of patients with: (i) sustained disability for the remainder of the follow-up, i.e. no disability regression confirmed at two or more consecutive visits separated in time by the minimum of 3 months; (ii) sustained disability and a second progression event according to the respective SPMS definition; (iii) a positive disability trajectory (i.e. the regression line projected over the EDSS/time-points having a slope >0); and (iv) an EDSS score of 6 or above at censoring.
In a third step, we selected the five definitions with the best overall sensitivity and specificity in steps one and two. These were further characterized for their ability to provide a timely diagnosis, disease activity and disability outcomes. We used Kaplan-Meier survival statistics to evaluate the proportion of patients converting to SPMS over time. The individual annualized relapse rate was calculated as the number of recorded relapses per year between inclusion and the end of follow-up or between the diagnosis of SPMS and the end of follow-up.
Where available, the proportion of patients with gadolinium-enhancing lesions on brain MRI 12 months before and after the diagnosis of SPMS was determined.
The area under the curve (AUC) was previously validated as a sensitive summative metric of all disability (transient as well as permanent) experienced by a patient during a follow-up period, with an effective use of serial data (Liu et al., 1998; Liu and Blumhardt, 1999; Kalincik et al., 2015b). We calculated the annualized AUC between the first and last documented EDSS within the observational period using the trapezium rule (Liu and Blumhardt, 1999). To determine the AUC during the secondary progressive phase, we calculated the annualized AUC between the EDSS score immediately preceding the diagnosis of SPMS and the last documented EDSS.
Statistical analysis
Statistical analyses were carried out using R, version 3.1.2. (R Development Core Team, 2011). The point and interval estimates of variable distributions were expressed as mean with 95% confidence intervals (CI), or median with interquartile range (IQR), as appropriate. Time from multiple sclerosis onset to diagnosis of SPMS was visualized with Kaplan-Meier curves and time until half of all patients had converted to SPMS was calculated for each definition.
Results
A total of 17 356 patients were included in the analysis (Fig. 2 and Supplementary Table 1). Their characteristics are shown in Table 1. Of the included patients, 2360 (14%) were diagnosed with SPMS by the treating physician, of which 1482 (9%) converted during the study and 878 (5%) had already entered the progressive phase of the disease before inclusion.
CONSORT flowchart of included patients.
Patient characteristics
| Count | Mean (SD) | Median (quartiles) | |
|---|---|---|---|
| Patients (female) | 17 356 (72%) | ||
| Age at inclusion, years | 37.6 (11.1) | 36.6 (29.3, 45.1) | |
| Disease duration at inclusion, years | 6.9 (8.0) | 3.8 (0.9, 10.3) | |
| Disability at inclusion, EDSS step | 2.4 (1.8) | 2 (1, 3.5) | |
| Disability at censoring, EDSS step | 3.1 (2.2) | 2.5 (1.5, 4.5) | |
| Annualized EDSS changea | 0.08 (0.36) | 0.04 (0, 0.22) | |
| Annualized relapse ratea | 0.28 (0.37) | 0.17 (0, 0.44) | |
| Patients ever treated with DMT | 14 437 (83%) | ||
| Proportion of time on DMT per patienta | 0.58 (0.41) | 0.75 (0.04, 0.98) | |
| Follow-up (years)a | 6.8 (4.7) | 5.8 (3.4, 9.6) | |
| Number of visitsa | 11.4 (8.9) | 9 (5, 14) |
| Count | Mean (SD) | Median (quartiles) | |
|---|---|---|---|
| Patients (female) | 17 356 (72%) | ||
| Age at inclusion, years | 37.6 (11.1) | 36.6 (29.3, 45.1) | |
| Disease duration at inclusion, years | 6.9 (8.0) | 3.8 (0.9, 10.3) | |
| Disability at inclusion, EDSS step | 2.4 (1.8) | 2 (1, 3.5) | |
| Disability at censoring, EDSS step | 3.1 (2.2) | 2.5 (1.5, 4.5) | |
| Annualized EDSS changea | 0.08 (0.36) | 0.04 (0, 0.22) | |
| Annualized relapse ratea | 0.28 (0.37) | 0.17 (0, 0.44) | |
| Patients ever treated with DMT | 14 437 (83%) | ||
| Proportion of time on DMT per patienta | 0.58 (0.41) | 0.75 (0.04, 0.98) | |
| Follow-up (years)a | 6.8 (4.7) | 5.8 (3.4, 9.6) | |
| Number of visitsa | 11.4 (8.9) | 9 (5, 14) |
aRecorded between the first and the last recorded EDSS visit.
DMT = disease-modifying therapy.
Patient characteristics
| Count | Mean (SD) | Median (quartiles) | |
|---|---|---|---|
| Patients (female) | 17 356 (72%) | ||
| Age at inclusion, years | 37.6 (11.1) | 36.6 (29.3, 45.1) | |
| Disease duration at inclusion, years | 6.9 (8.0) | 3.8 (0.9, 10.3) | |
| Disability at inclusion, EDSS step | 2.4 (1.8) | 2 (1, 3.5) | |
| Disability at censoring, EDSS step | 3.1 (2.2) | 2.5 (1.5, 4.5) | |
| Annualized EDSS changea | 0.08 (0.36) | 0.04 (0, 0.22) | |
| Annualized relapse ratea | 0.28 (0.37) | 0.17 (0, 0.44) | |
| Patients ever treated with DMT | 14 437 (83%) | ||
| Proportion of time on DMT per patienta | 0.58 (0.41) | 0.75 (0.04, 0.98) | |
| Follow-up (years)a | 6.8 (4.7) | 5.8 (3.4, 9.6) | |
| Number of visitsa | 11.4 (8.9) | 9 (5, 14) |
| Count | Mean (SD) | Median (quartiles) | |
|---|---|---|---|
| Patients (female) | 17 356 (72%) | ||
| Age at inclusion, years | 37.6 (11.1) | 36.6 (29.3, 45.1) | |
| Disease duration at inclusion, years | 6.9 (8.0) | 3.8 (0.9, 10.3) | |
| Disability at inclusion, EDSS step | 2.4 (1.8) | 2 (1, 3.5) | |
| Disability at censoring, EDSS step | 3.1 (2.2) | 2.5 (1.5, 4.5) | |
| Annualized EDSS changea | 0.08 (0.36) | 0.04 (0, 0.22) | |
| Annualized relapse ratea | 0.28 (0.37) | 0.17 (0, 0.44) | |
| Patients ever treated with DMT | 14 437 (83%) | ||
| Proportion of time on DMT per patienta | 0.58 (0.41) | 0.75 (0.04, 0.98) | |
| Follow-up (years)a | 6.8 (4.7) | 5.8 (3.4, 9.6) | |
| Number of visitsa | 11.4 (8.9) | 9 (5, 14) |
aRecorded between the first and the last recorded EDSS visit.
DMT = disease-modifying therapy.
Validation against a consensus diagnosis
The 200 sample patients selected in the first step of the analysis had a median follow-up of 17.3 years (quartiles 15.7, 18.5) and a median visit density of 1.8 (quartiles 1.5, 2.0) per year. Of these patients, 41 (20.5%) were labelled as secondary progressive by their treating physician during the follow-up. Further characteristics of these patients are shown in Table 2. Compared with the entire cohort, patients in this subset were younger at disease onset (Cohen’s d0.39) and inclusion (Cohen’s d 0.72), had a higher annualized relapse rate (Cohen’s d 0.30) and showed a greater annualized EDSS change (Cohen’s d 0.11). After independent review of the patient data, 57 were diagnosed with SPMS (48 of them by all three neurologists, and an additional nine subjects by at least two neurologists). Agreement between the raters about the diagnosis was good (Cohen’s kappa 0.71–0.84). However, only in 10 patients the conversion dates assigned by the three physicians lay within a period of 6 months. The median time difference between the dates of diagnosis was 1.4 years (quartiles 0.5, 2.3). The median duration from the date when secondary progression was first suspected until the date when the assessing physician had sufficient information to confirm the retrospective diagnosis of SPMS was 2 years (quartiles 1.1, 3.2). Using the consensus SPMS diagnosis as a comparator, sensitivity of the operational definitions varied from 70% to 100% and specificity ranged from 17% to 97%. The definition with the highest accuracy (91%, sensitivity of 88%, specificity of 92%) consisted of an initial progression by ≥2 EDSS steps, a minimum EDSS score of 4 and pyramidal FS score of 2, confirmation of progression after 6 months within the leading FS, and excluded worsening of disability attributable to recent relapse activity. Hence, we selected this definition for our shortlist. In addition, we also chose the definition with the highest accuracy among those definitions using the 3-strata progression paradigm and a shorter confirmation period of 3 months for the potentially improved timeliness of the diagnosis. The criteria of 3-strata progression and 3-month confirmation period resulted in an improved sensitivity (89%) and lower specificity (86%) relative to the above definition. As information about FS scores may not always be available in clinical practice, we omitted the minimal required pyramidal FS of ≥2 and confirmation within the leading FS for another candidate definition, which did not improve sensitivity (89%) but further reduced specificity (84%). This drop in accuracy was not overcome by extending the confirmation period to 12 or 24 months. Compared with all operational definitions, diagnosis by the treating physicians as recorded in the MSBase had the highest specificity but lowest sensitivity.
Characteristics of patient sample
| Count | Mean (SD) | Median (quartiles) | |
|---|---|---|---|
| Patients (female) | 200 (74%) | ||
| Age at inclusion | 30.4 (8.7) | 30.2 (24.0, 35.3) | |
| Disease duration at inclusion, years | 3.5 (4.4) | 1.7 (0.6, 4.7) | |
| EDSS at inclusion | 1.9 (1.3) | 2.0 (1.0, 2.5) | |
| EDSS at censoring | 3.9 (2.0) | 3.5 (2.0, 5.5) | |
| Annualized EDSS changea | 0.11 (0.12) | 0.1 (0.02, 0.19) | |
| Annualized relapse ratea | 0.39 (0.27) | 0.35 (0.18, 0.55) | |
| Follow-up, yearsa | 17.5 (2.2) | 17.3 (15.7, 18.5) | |
| Number of visitsa | 30.6 (6.1) | 31 (26.0, 34.3) | |
| Interval between visits, years | 0.6 (0.1) | 0.6 (0.5, 0.7) |
| Count | Mean (SD) | Median (quartiles) | |
|---|---|---|---|
| Patients (female) | 200 (74%) | ||
| Age at inclusion | 30.4 (8.7) | 30.2 (24.0, 35.3) | |
| Disease duration at inclusion, years | 3.5 (4.4) | 1.7 (0.6, 4.7) | |
| EDSS at inclusion | 1.9 (1.3) | 2.0 (1.0, 2.5) | |
| EDSS at censoring | 3.9 (2.0) | 3.5 (2.0, 5.5) | |
| Annualized EDSS changea | 0.11 (0.12) | 0.1 (0.02, 0.19) | |
| Annualized relapse ratea | 0.39 (0.27) | 0.35 (0.18, 0.55) | |
| Follow-up, yearsa | 17.5 (2.2) | 17.3 (15.7, 18.5) | |
| Number of visitsa | 30.6 (6.1) | 31 (26.0, 34.3) | |
| Interval between visits, years | 0.6 (0.1) | 0.6 (0.5, 0.7) |
aRecorded between the first and the last recorded EDSS visit.
Characteristics of patient sample
| Count | Mean (SD) | Median (quartiles) | |
|---|---|---|---|
| Patients (female) | 200 (74%) | ||
| Age at inclusion | 30.4 (8.7) | 30.2 (24.0, 35.3) | |
| Disease duration at inclusion, years | 3.5 (4.4) | 1.7 (0.6, 4.7) | |
| EDSS at inclusion | 1.9 (1.3) | 2.0 (1.0, 2.5) | |
| EDSS at censoring | 3.9 (2.0) | 3.5 (2.0, 5.5) | |
| Annualized EDSS changea | 0.11 (0.12) | 0.1 (0.02, 0.19) | |
| Annualized relapse ratea | 0.39 (0.27) | 0.35 (0.18, 0.55) | |
| Follow-up, yearsa | 17.5 (2.2) | 17.3 (15.7, 18.5) | |
| Number of visitsa | 30.6 (6.1) | 31 (26.0, 34.3) | |
| Interval between visits, years | 0.6 (0.1) | 0.6 (0.5, 0.7) |
| Count | Mean (SD) | Median (quartiles) | |
|---|---|---|---|
| Patients (female) | 200 (74%) | ||
| Age at inclusion | 30.4 (8.7) | 30.2 (24.0, 35.3) | |
| Disease duration at inclusion, years | 3.5 (4.4) | 1.7 (0.6, 4.7) | |
| EDSS at inclusion | 1.9 (1.3) | 2.0 (1.0, 2.5) | |
| EDSS at censoring | 3.9 (2.0) | 3.5 (2.0, 5.5) | |
| Annualized EDSS changea | 0.11 (0.12) | 0.1 (0.02, 0.19) | |
| Annualized relapse ratea | 0.39 (0.27) | 0.35 (0.18, 0.55) | |
| Follow-up, yearsa | 17.5 (2.2) | 17.3 (15.7, 18.5) | |
| Number of visitsa | 30.6 (6.1) | 31 (26.0, 34.3) | |
| Interval between visits, years | 0.6 (0.1) | 0.6 (0.5, 0.7) |
aRecorded between the first and the last recorded EDSS visit.
Long-term disability outcomes
In step two we applied the 576 definitions to all eligible patients. The number of patients identified as secondary progressive varied from 1092 to 7930 (6–46%). The most inclusive definition consisted of a disability progression by ≥1 EDSS step, confirmation of progression after 3 months and no other requirements. In contrast, the most conservative definition required an initial progression by ≥2 EDSS steps, a minimum EDSS score of 4, a pyramidal FS score of ≥2, a 24-month confirmation of the initial progression event, including confirmation within the FS leading to progression, no relapses preceding the progression event since the previous recorded EDSS and ≤2 documented relapses in the 24 months following the initial progression event. Figure 3 shows the comparison of all definitions with respect to the proportion of patients diagnosed with SPMS and the four specified long-term outcomes. Of note, while the proportion of patients diagnosed with SPMS tended to be lower when diagnosed based on the physicians’ diagnosis, these patients were more likely to reach the disability endpoints after 5 years compared to all operational definitions. We found a positive disability trajectory after conversion to SPMS in 85% of patients and 81% had an EDSS of ≥6 at censoring.
Long-term disability outcomes.X-axes: Proportion of patients with at least 5-year follow-up and (A) sustained disability, (B) second progression event with sustained disability, (C) positive disability trajectory, and (D) EDSS of 6 or above at censoring. Y-axes: Proportion of patients diagnosed with SPMS by the respective definition. Grey circles represent the definitions. The five best performing definitions and the physicians’ diagnosis are highlighted. Error bars represent 95% CI.
The definition selected in step one for its highest accuracy compared with the consensus diagnosis identified fewer patients than the physicians’ diagnosis (10% versus 14%; Fig. 3). However, the definition using the 3-strata progression magnitude with a minimum EDSS score of 4 and confirmation after ≥3 months resulted in a larger proportion of patients diagnosed with SPMS (18%). Among the patients with ≥5-year follow-up after the diagnosis of SPMS according to the latter definition, 78% showed an ongoing positive disability trajectory and 70% reached an EDSS of ≥6.
A further three definitions with the best performance relative to the four long-term outcomes were based on the 3-strata progression magnitude criterion, a minimum EDSS score of 3, a minimum pyramidal FS score of 2, and confirmation in the FS leading to progression. All of them excluded progression attributable to relapse activity, but varied in the required confirmation period (6, 12 or 24 months). Seventy-eight to 84% of the patients diagnosed as secondary progressive by these definitions had a sustained level of disability over the 5 years following the diagnosis, 44–54% had sustained disability with further progression, 79–82% showed a positive disability trajectory and 62–66% reached an EDSS of ≥ 6. The proportion of identified patients varied from 13–19% (Fig. 3). The accuracy of the five shortlisted definitions and the recorded physicians’ diagnosis in comparison to the consensus diagnosis is shown in Table 3. It is worth noting that the three definitions identified in step two tended to have a higher sensitivity and lower specificity than the definitions identified in step one.
Sensitivity and specificity of the shortlisted definitions
| Definition | Sensitivity | Specificity | Accuracy | ||
|---|---|---|---|---|---|
| Progression magnitude | Minimal EDSS | Confirmation | |||
| 1-stratum | 4 | 6 months | 0.88 | 0.92 | 0.91 |
| 3-strata | 4 | 3 months | 0.89 | 0.86 | 0.87 |
| 3 | 24 monthsa | 0.95 | 0.84 | 0.87 | |
| 12 monthsa | 0.97 | 0.80 | 0.85 | ||
| 6 monthsa | 1.0 | 0.77 | 0.84 | ||
| Physicians’ diagnosis | 0.61 | 0.95 | 0.86 | ||
| Definition | Sensitivity | Specificity | Accuracy | ||
|---|---|---|---|---|---|
| Progression magnitude | Minimal EDSS | Confirmation | |||
| 1-stratum | 4 | 6 months | 0.88 | 0.92 | 0.91 |
| 3-strata | 4 | 3 months | 0.89 | 0.86 | 0.87 |
| 3 | 24 monthsa | 0.95 | 0.84 | 0.87 | |
| 12 monthsa | 0.97 | 0.80 | 0.85 | ||
| 6 monthsa | 1.0 | 0.77 | 0.84 | ||
| Physicians’ diagnosis | 0.61 | 0.95 | 0.86 | ||
aDefinitions identified in step two of the selection process.
Sensitivity and specificity of the shortlisted definitions
| Definition | Sensitivity | Specificity | Accuracy | ||
|---|---|---|---|---|---|
| Progression magnitude | Minimal EDSS | Confirmation | |||
| 1-stratum | 4 | 6 months | 0.88 | 0.92 | 0.91 |
| 3-strata | 4 | 3 months | 0.89 | 0.86 | 0.87 |
| 3 | 24 monthsa | 0.95 | 0.84 | 0.87 | |
| 12 monthsa | 0.97 | 0.80 | 0.85 | ||
| 6 monthsa | 1.0 | 0.77 | 0.84 | ||
| Physicians’ diagnosis | 0.61 | 0.95 | 0.86 | ||
| Definition | Sensitivity | Specificity | Accuracy | ||
|---|---|---|---|---|---|
| Progression magnitude | Minimal EDSS | Confirmation | |||
| 1-stratum | 4 | 6 months | 0.88 | 0.92 | 0.91 |
| 3-strata | 4 | 3 months | 0.89 | 0.86 | 0.87 |
| 3 | 24 monthsa | 0.95 | 0.84 | 0.87 | |
| 12 monthsa | 0.97 | 0.80 | 0.85 | ||
| 6 monthsa | 1.0 | 0.77 | 0.84 | ||
| Physicians’ diagnosis | 0.61 | 0.95 | 0.86 | ||
aDefinitions identified in step two of the selection process.
Timeliness of the diagnosis, disease activity and disability burden
In step three, timeliness of the diagnosis as well as relapse incidence and disability outcomes among the patients diagnosed with SPMS based on the selected definitions were evaluated. Characteristics of the patients identified by each of these definitions are shown in Table 4. At the time of SPMS diagnosis, 75–79% of the patients identified by the shortlisted definitions were receiving disease-modifying treatment. The mean proportion of follow-up time spent on disease modifying therapy ranged from 58% to 61% [standard deviation (SD) 41–42%] for the time before SPMS, and from 57% to 59% (SD 43–44%) for the time after the diagnosis of SPMS. Compared to that, 67% of patients diagnosed by their physicians were on disease modifying therapy when they converted to SPMS and the mean proportion of time on treatment was 57% (SD 43%) before SPMS, and 43% (SD 63%) after the diagnosis of SPMS (Supplementary Table 2).
Characteristics of patients diagnosed with SPMS based on the shortlisted definitions
| Definition | Patients (female) | Age at SPMS, years | Disease duration at SPMS, years | EDSS before SPMS | EDSS at SPMS | EDSS at censoring | ||
|---|---|---|---|---|---|---|---|---|
| Progression magnitude | Minimal EDSS | Confirmation | ||||||
| 1-stratum | 4 | 6 months | 1652 (67%) | 46.4 (10.2) | 14.8 (8.8) | 3.8 (1.4) | 5.3 (1.2) | 5.9 (1.7) |
| 3-strata | 4 | 3 months | 3066 (69%) | 46.5 (10.5) | 14.8 (9.3) | 4.1 (1.6) | 5.4 (1.3) | 5.9 (1.5) |
| 3 | 24 months | 2281 (69%) | 45.3 (10.4) | 13.9 (9.0) | 3.7 (1.7) | 4.9 (1.5) | 5.8 (1.6) | |
| 12 months | 2940 (69%) | 45.7 (10.5) | 14.1 (9.2) | 3.7 (1.7) | 5.0 (1.5) | 5.7 (1.7) | ||
| 6 months | 3361 (69%) | 45.8 (10.5) | 14.2 (9.2) | 3.7 (1.7) | 5.0 (1.5) | 5.6 (1.7) | ||
| Physicians’ diagnosis | 2360 (68%) | 45.4 (10.1) | 14.3 (8.8) | NA | 4.5 (2.0) | 6.1 (1.5) | ||
| Definition | Patients (female) | Age at SPMS, years | Disease duration at SPMS, years | EDSS before SPMS | EDSS at SPMS | EDSS at censoring | ||
|---|---|---|---|---|---|---|---|---|
| Progression magnitude | Minimal EDSS | Confirmation | ||||||
| 1-stratum | 4 | 6 months | 1652 (67%) | 46.4 (10.2) | 14.8 (8.8) | 3.8 (1.4) | 5.3 (1.2) | 5.9 (1.7) |
| 3-strata | 4 | 3 months | 3066 (69%) | 46.5 (10.5) | 14.8 (9.3) | 4.1 (1.6) | 5.4 (1.3) | 5.9 (1.5) |
| 3 | 24 months | 2281 (69%) | 45.3 (10.4) | 13.9 (9.0) | 3.7 (1.7) | 4.9 (1.5) | 5.8 (1.6) | |
| 12 months | 2940 (69%) | 45.7 (10.5) | 14.1 (9.2) | 3.7 (1.7) | 5.0 (1.5) | 5.7 (1.7) | ||
| 6 months | 3361 (69%) | 45.8 (10.5) | 14.2 (9.2) | 3.7 (1.7) | 5.0 (1.5) | 5.6 (1.7) | ||
| Physicians’ diagnosis | 2360 (68%) | 45.4 (10.1) | 14.3 (8.8) | NA | 4.5 (2.0) | 6.1 (1.5) | ||
Mean and (SD) are given. NA = not available for all subjects.
Characteristics of patients diagnosed with SPMS based on the shortlisted definitions
| Definition | Patients (female) | Age at SPMS, years | Disease duration at SPMS, years | EDSS before SPMS | EDSS at SPMS | EDSS at censoring | ||
|---|---|---|---|---|---|---|---|---|
| Progression magnitude | Minimal EDSS | Confirmation | ||||||
| 1-stratum | 4 | 6 months | 1652 (67%) | 46.4 (10.2) | 14.8 (8.8) | 3.8 (1.4) | 5.3 (1.2) | 5.9 (1.7) |
| 3-strata | 4 | 3 months | 3066 (69%) | 46.5 (10.5) | 14.8 (9.3) | 4.1 (1.6) | 5.4 (1.3) | 5.9 (1.5) |
| 3 | 24 months | 2281 (69%) | 45.3 (10.4) | 13.9 (9.0) | 3.7 (1.7) | 4.9 (1.5) | 5.8 (1.6) | |
| 12 months | 2940 (69%) | 45.7 (10.5) | 14.1 (9.2) | 3.7 (1.7) | 5.0 (1.5) | 5.7 (1.7) | ||
| 6 months | 3361 (69%) | 45.8 (10.5) | 14.2 (9.2) | 3.7 (1.7) | 5.0 (1.5) | 5.6 (1.7) | ||
| Physicians’ diagnosis | 2360 (68%) | 45.4 (10.1) | 14.3 (8.8) | NA | 4.5 (2.0) | 6.1 (1.5) | ||
| Definition | Patients (female) | Age at SPMS, years | Disease duration at SPMS, years | EDSS before SPMS | EDSS at SPMS | EDSS at censoring | ||
|---|---|---|---|---|---|---|---|---|
| Progression magnitude | Minimal EDSS | Confirmation | ||||||
| 1-stratum | 4 | 6 months | 1652 (67%) | 46.4 (10.2) | 14.8 (8.8) | 3.8 (1.4) | 5.3 (1.2) | 5.9 (1.7) |
| 3-strata | 4 | 3 months | 3066 (69%) | 46.5 (10.5) | 14.8 (9.3) | 4.1 (1.6) | 5.4 (1.3) | 5.9 (1.5) |
| 3 | 24 months | 2281 (69%) | 45.3 (10.4) | 13.9 (9.0) | 3.7 (1.7) | 4.9 (1.5) | 5.8 (1.6) | |
| 12 months | 2940 (69%) | 45.7 (10.5) | 14.1 (9.2) | 3.7 (1.7) | 5.0 (1.5) | 5.7 (1.7) | ||
| 6 months | 3361 (69%) | 45.8 (10.5) | 14.2 (9.2) | 3.7 (1.7) | 5.0 (1.5) | 5.6 (1.7) | ||
| Physicians’ diagnosis | 2360 (68%) | 45.4 (10.1) | 14.3 (8.8) | NA | 4.5 (2.0) | 6.1 (1.5) | ||
Mean and (SD) are given. NA = not available for all subjects.
The two definitions that led to the earliest diagnosis of SPMS were both based on the 3-strata criterion of progression magnitude (Fig. 4A). In addition, they used a minimum EDSS score of 3 combined with a confirmation period of 6 months or a minimum EDSS score of 4 in combination with a confirmation period of 3 months. According to these definitions half of the patients were diagnosed with SPMS within 32.0–32.6 years from multiple sclerosis onset (95% CI 31.0–33.6), which was markedly earlier than the physicians’ diagnosis [36 years from multiple sclerosis onset (95% CI 35–39)].
Timeliness of diagnosis and disability burden.(A) Time from disease onset to secondary progression (Kaplan-Meier estimate). (B) Median annualized area under the EDSS-time curve (AUC) for patients diagnosed (filled squares) and not diagnosed (triangles) with SPMS based on the compared definitions. In addition, AUC during the progressive phase is shown (open squares). Error bars represent the IQR.
The annualized relapse rates were 0.26–0.28 (0.28–0.31) [mean (SD)] in patients diagnosed by the five shortlisted operational definitions versus 0.28–0.29 (0.37–0.38) in patients not diagnosed with SPMS. When analysing the progressive disease phase exclusively, mean relapse rates varied between 0.23 and 0.26 (0.35–0.44). In comparison, the patients diagnosed versus not diagnosed by the treating physician showed an annualized relapse rate of 0.2 (0.29) versus 0.29 (0.38), respectively [0.17 (0.33) after the diagnosis of SPMS].
In subjects for whom sufficient MRI data were available (22–25%), the proportion of patients diagnosed with SPMS who showed one or more gadolinium-enhancing lesions within the 12 months prior to the diagnosis was 13–15%, and 20–22% at any time after the SPMS diagnosis. In comparison, 38–40% of the undiagnosed patients showed contrast enhancement at any time point during the recorded follow-up. In the group with physician-assigned diagnosis of SPMS, only 15% of patients had an MRI scan with gadolinium application within the year prior to conversion to SPMS. In 12% of these, at least one enhancing lesion was revealed during this period and 21% showed gadolinium enhancement following the diagnosis of SPMS. Patients with progressive disease experienced a higher overall burden of disability, expressed as the annualized area under the EDSS-time curve compared with non-progressive patients (Fig. 4B). Also, the median increase in the overall disability burden (the annualized AUC change) ranged from 1.13 to 1.68 EDSS steps (95% CI 0.54–1.06 to 1.93–2.38) versus 0.77 EDSS steps (0.16 to 1.83) in subjects with versus without the diagnosis of SPMS, respectively [median (quartiles)]. Patients with ongoing relapses or contrast-enhancing MRI lesions after the diagnosis of SPMS did not tend to accumulate more disability compared with patients without episodic inflammatory activity (Supplementary Table 3).
Discussion
In this analysis from the prospective observational MSBase cohort study, we have identified an objective definition of SPMS with the best performance from a pool of 576 candidate definitions. This definition consists of a disability progression by 1 EDSS step in patients with EDSS ≤5.5 or 0.5 EDSS steps in patients with EDSS ≥6 in the absence of a relapse, a minimum EDSS score of 4 and pyramidal FS score of 2 and confirmed progression over ≥3 months, including confirmation within the leading FS. The accuracy of this definition compared to a consensus diagnosis by three independent raters was 87%. The definition identified 18% of the eligible MSBase cohort as converting to SPMS. Among the patients with ≥5 years follow-up after the SPMS diagnosis, 78% showed a positive disability trajectory and 70% reached an EDSS of ≥6 at censoring. The objective definition was more sensitive but less specific than the retrospective physicians’ diagnosis. Importantly, it enabled the diagnosis of SPMS more than 3 years earlier than the diagnosis date assigned by the treating physicians.
Determinants of diagnostic accuracy
Currently, there is no gold-standard objective definition of SPMS. We therefore used a consensus diagnosis of SPMS as one of the comparators for the 576 evaluated definitions. In keeping with previous studies, we observed good agreement among neurologists when determining the conversion to SPMS in selected patients with extended follow-up and high visit density (Amato et al., 2004; Katz Sand et al., 2014). However, consensus on the date of conversion to SPMS was only moderate and the variability in the time of the diagnosed conversion was remarkable even among the three neurologists with a largely similar view of the SPMS diagnosis. We found a period of uncertainty of ∼2 years between the visit when disease progression was first suspected and the date when SPMS was finally diagnosed. Such a period has been described previously and may be required to distinguish fluctuations from true disability progression (Katz Sand et al., 2014). Our proposed objective definition of SPMS minimises this period of uncertainty, which could reduce an important source of bias in studies where SPMS is an inclusion criterion or an outcome.
As expected, specificity and sensitivity of the operational definitions varied broadly when compared with the consensus diagnosis. It is not surprising that the definitions with the best accuracy defined minimal EDSS and pyramidal FS scores and also required confirmation within the FS leading to progression. Omitting a minimal pyramidal FS score and confirmation within the leading FS resulted in a lower diagnostic accuracy, which could not be easily overcome by using more stringent requirements for other criteria, such as the required confirmation time. While using a definition without the FS information represents a pragmatic approach where this information is not available, we strongly advocate utilization of FS scores. This not only increases diagnostic accuracy of the definition but also enables review of the internal validity of EDSS scores.
While correction for disability progression attributable to preceding relapse activity improved accuracy, excluding patients with a higher number of relapses after the diagnosis had no relevant impact.
For the analysis in the complete dataset, we used the proportion of patients diagnosed with SPMS and a set of unfavourable long-term outcomes as to assess sensitivities and specificities. Noting that no ‘post-diagnosis’ measure of accuracy currently exists, we aimed to capture different aspects of the progressive disease phase, such as sustained disability, further disability progression (reflected by a positive disability trajectory or a second sustained progression event) and reaching significant walking impairment at EDSS 6 during follow-up.
It has been previously demonstrated that the most important determinant of progression stability is the required confirmation time and that disability metrics based on short-term confirmed progression overestimated the long-term accumulation of irreversible disability (Kalincik et al., 2015a). However, patients with progressive multiple sclerosis and higher EDSS score were less likely to regress. This finding can be explained by the properties of the EDSS, which is known to have less variability at higher stages of disability and also by the fact that there is less fluctuation once a patient has reached a certain level of disability (Weinshenker et al., 1996; Hohol et al., 1999; Ravnborg et al., 2005; Kalincik et al., 2015a). This allowed us to use relatively shorter periods for EDSS confirmation where a minimum disability was defined as EDSS step ≥3 or ≥4. We also found that the exclusion of progression events that were related to preceding relapse activity was a critical factor of progression stability. In addition, the requirement of a minimal pyramidal FS of ≥2 and confirmation in the leading FS further enhanced the stability. On the other hand, we observed that replacement of the 3-strata progression paradigm by the more stringent criterion of a progression magnitude of 2 EDSS steps led to fewer identified patients without increasing the proportion of patients reaching the predefined disability outcomes.
Characteristics of the diagnosed patients
The median time from disease onset until conversion to SPMS has been reported as 15 years in the London, Ontario cohort and 19 years in the Lyon and Gothenburg cohorts, whereas contemporary cohorts have shown a shift towards a longer time to conversion (Confavreux et al., 2003; Eriksson et al., 2003; Tremlett et al., 2006; Tedeholm et al., 2013; Scalfari et al., 2014; Ribbons et al., 2015). Similarly, we also found a substantially longer time from disease onset to SPMS compared to the mentioned natural history studies. It has been previously suggested that the trend towards slower disability progression may represent a flux in the patient population seen in multiple sclerosis clinics, and might also be driven by an increased recognition of the disease, changing diagnostic criteria, availability of more potent immunomodulatory drugs, as well as improving management of chronic disease (Tremlett et al., 2010). In addition, MSBase is not a natural history cohort, and the population seen in the tertiary multiple sclerosis centres contributing to MSBase is enriched for patients with active relapsing-remitting disease and treated with immunomodulatory agents. In fact, the patients identified as secondary progressive by the selected objective definitions had spent 58–68% of their previous follow-up time on disease-modifying treatment. Finally, there is well known heterogeneity among observational studies in terms of design, data collection, definition of outcomes, and analysis (Tremlett et al., 2010). For example, we found that patients in the London, Ontario cohort were diagnosed with SPMS at markedly lower disability levels (median Disability Status Scale score 2.9) compared to our study (median EDSS 4.9–5.4 for the objective definitions and 4.5 for the physicians’ diagnosis) (Scalfari et al., 2014). This implies a systematic difference in the use of diagnostic criteria, which again proves the need for a unified, validated SPMS definition.
None of the shortlisted definitions excluded patients with ongoing relapses, which may raise concerns about an increased proportion of patients with disease activity in the selected populations. However, compared with the data available from treatment trials in SPMS, the observed annualised relapse rates during the progressive phase (0.23–0.26) were within the lower range of the spectrum for all chosen definitions (Ontaneda et al., 2015). In our population, sufficient MRI data were available for 22–25% of the patients. Of these, 20–22% showed contrast enhancement during the progressive phase of the disease. While previous studies reported a decrease in the frequency of gadolinium-enhancing lesions in progressive multiple sclerosis compared with relapsing-remitting multiple sclerosis, studies utilizing frequent MRI scans still documented contrast-enhancing lesions in 29–70% of SPMS patients (Filippi et al., 1997; Wolinsky et al., 2000; Zhao et al., 2010). This suggests that our proposed definition of SPMS does not result in an overrepresentation of patients with inflammatory activity.
In combination with the substantially greater disability burden among the patients identified by the shortlisted definitions, this implies that the identified patients continue to accumulate further disability that is not directly associated with episodic CNS inflammation.
Best objective definition versus physicians’ diagnosis
We conclude that the definition using a 3-strata progression paradigm, with a minimum EDSS of 4 and a confirmation time of 3 months had the best overall performance out of the candidate definitions. In comparison with the physicians’ diagnosis it was less specific (86% versus 95%) but more sensitive (89% versus 61%). The objective definition identified a larger number of eligible patients out of the MSBase dataset than the physicians’ diagnosis (18% versus 14%). The true difference is even more pronounced when considering only the patients with physician-diagnosed SPMS after the start of prospective follow-up in the MSBase study (9%). On the other hand, a greater proportion of physician-diagnosed SPMS patients fulfilled both the endpoints of a positive disability trajectory and an EDSS of ≥6 when followed-up over ≥5 years (81% versus 70%, respectively). This difference reflects the retrospective nature of the physicians’ diagnosis, as it is often only assigned after these endpoints have been reached.
Importantly, our objective definition identified patients with SPMS more than 3 years earlier in the disease than the physicians’ diagnosis. Relapse rate and accumulation of new disability during the progressive multiple sclerosis were greater in the patients diagnosed by the objective definition than those with an SPMS diagnosis by the treating physicians. These findings confirm that neurologists wait until substantial disability has been accumulated and episodic disease activity has ceased before the SPMS diagnosis is confidently assigned, one motivation probably being their desire for patients to remain on disease-modifying therapy, which in some jurisdictions is not allowed after diagnosis of SPMS. While the effect of a diagnostic delay invoked by this conservative approach is negligible in clinical practice, it can introduce a significant bias in clinical trial settings.
Limitations
The main limitation of our study overlaps with the limitations of the EDSS. The score is weighted towards motor and lower limb functions, which leads to a decreased sensitivity for detecting more subtle forms of disease progression. Our proposed definition of SPMS is therefore likely to be rather conservative. To adjust for the well-known non-linearity of the scale, we stratified the required magnitude of the progression by the level of pre-existing disability (Amato and Ponziani, 1999). The EDSS is burdened with a relatively low intra- and inter-rater reliability, especially at the lower end of the scale (Amato et al., 1988; Goodkin et al., 1992). As our study involved 81 centres over long follow-up periods, this probably led to inflation of the EDSS variance. On the other hand, we aimed to mitigate variance through the requirement of Neurostatus certification and its impact through the size of the study population. Furthermore, all of the shortlisted definitions required minimal EDSS levels, thus restricting evaluation of progression to the higher EDSS steps, which are known to be relatively more stable (Hohol et al., 1999; Ravnborg et al., 2005). In addition, confirmation after defined time periods and within FS scores were used to further improve EDSS stability.
In clinical practice, SPMS is typically diagnosed retrospectively. As the time-point when the diagnosis is assigned is not recorded in MSBase, we were unable to analyse the period of uncertainty in the complete dataset. Hence, we used an estimate from 200 sample patients, which probably still underestimated the lag to the diagnosis, as the sample was selected with the intention of maximising follow-up duration and visit density. While this sample cannot be regarded as being representative of the full MSBase cohort, it provides an optimal setting for the comparison of the tested definitions of SPMS to clinical judgement.
The relatively poor availability of MRI data hindered conclusive assessment of disease activity in the progressive patients. On the other hand, it forced us to focus our analysis on clinical data, which facilitated development of an accessible and easy-to-use definition for SPMS.
Conclusion
Our objective definition of SPMS based on the EDSS and information about preceding relapses provides a tool for a reproducible, accurate and timely SPMS diagnosis that requires a very short confirmation period and can also be easily applied retrospectively to longitudinal data. If applied broadly, the definition has the potential to strengthen the design and improve comparability of clinical trials and observational studies in SPMS.
Acknowledgements
We would like to acknowledge Murat Terzi, MD (Samsun, Turkey), Ricardo Fernandez Bolaños, MD (Seville, Spain), Maria Edite Rio, MD (Porto, Portugal), José Antonio Cabrera Gómez, MD (Havanna, Cuba), Shlomo Flechter, MD (Beer-Yaakov, Israel) and Norma Deri, MD (Buenos Aires, Argentina) for their contribution to data acquisition.
Funding
This study was financially supported by the National Health and Medical Research Council (practitioner fellowship 1080518, project grants 1083539 and 1032484 and centre for research excellence 1001216), the University of Melbourne (Faculty of Medicine, Dentistry and Health Sciences research fellowship) and Biogen (Fellowship in MS Registries Research). The MSBase Foundation is a not-for-profit organization that receives support from Merck, Biogen, Novartis, Bayer-Schering, Genzyme, Teva and Sanofi-Aventis. The study was conducted separately and apart from the guidance of the sponsors.
Supplementary material
Supplementary material is available at Brain online.
Abbreviations
- AUC
area under the EDSS-time curve
- EDSS
Expanded Disability Status Scale
- FS
Functional System
- SPMS
secondary progressive multiple sclerosis
References
Author notes
*These authors contributed equally to this work.
