Abstract

Background. Major depressive disorder often remains unrecognized in primary care.

Objective. Development of a clinical prediction rule using easily obtainable predictors for major depressive disorder in primary care patients.

Methods. A total of 1046 subjects, aged 18–65 years, were included from seven large general practices in the center of The Netherlands. All subjects were recruited in the general practice waiting room, irrespective of their presenting complaint. Major depressive disorder according to Diagnostic and Statistical Manual of Mental Disorders, Fourth Text Revision edition criteria was assessed with the Composite International Diagnostic Interview. Candidate predictors were gender, age, educational level, being single, number of presented complaints, presence of non-somatic complaints, whether a diagnosis was assigned, consultation rate in past 12 months, presentation of depressive complaints or prescription of antidepressants in past 12 months, number of life events in past 6 months and any history of depression.

Results. The first multivariable logistic regression model including only predictors that require no confronting depression-related questions had a reasonable degree of discrimination (area under the receiver operating characteristic curve or concordance-statistic (c-statistic) = 0.71; 95% Confidence Interval (CI): 0.67–0.76). Addition of three simple though more depression-related predictors, number of life events and history of depression, significantly increased the c-statistic to 0.80 (95% CI: 0.76–0.83). After transforming this second model to an easily to use risk score, the lowest risk category (sum score < 5) showed a 1% risk of depression, which increased to 49% in the highest category (sum score ≥ 30).

Conclusion. A clinical prediction rule allows GPs to identify patients—irrespective of their complaints—in whom diagnostic workup for major depressive disorder is indicated.

Introduction

Major depressive disorder is a serious health problem. Estimations by the World Health Organization suggest that it will be the second ranking cause of disability by 2020, after cardiovascular disease.1 Lifetime prevalence for this mood disorder is estimated up to 25%.2,3 If not treated, major depressive disorder has a marked impact on quality of life and use of health care services.4–6 Hence, detection of major depressive disorder is important to improve the patient's prognosis and reduce health care consumption.7,8

Recent studies showed that a significant proportion of major depressive disorders remain unrecognized by clinicians. Estimates of undetected and therefore untreated depressive disorder are reported >50%, depending on the studied patient population.5,9–12 Many instruments are available, designed to screen for major depressive disorder in individual patients.13–16 However, the majority of these instruments are questionnaires that have to be filled in by the patients themselves, thus requiring an active and relative time-consuming strategy to detect primary care patients at high risk of depression.

We aimed to develop a clinical prediction rule to enable primary care physicians to better identify (adult) patients at high risk of major depressive disorder (who may need a more comprehensive diagnostic workup), with minimal involvement of the patients. Hence, we required that the tool can easily be incorporated into the daily routine of the GP. Consequently, the predictors in the clinical prediction rule had to be comprised of patient characteristics or information that will be directly available to the GP.

Methods

Design

This study is part of the international PREDICT study that is set in The Netherlands.17 In brief, PREDICT is a large prospective cohort study in six European countries, the aim of which is to develop a multifactor risk algorithm for the onset of major depression over 12 months in primary care.18 This study uses the Dutch patients who were included in PREDICT (PREDICT-NL). In The Netherlands, patients were recruited from seven general practices in the city of Utrecht and surrounding areas. Patients aged ≥18 who visited the GP were asked to participate in the study while waiting to see their doctor, irrespective of their reasons for consulting the GP. Patients willing to participate were asked to complete the baseline questionnaire and sign the informed consent form within 2 weeks. If necessary, a first reminder was sent after 2 weeks, and a second one after 4 weeks. Participants who did not respond to the second reminder were considered non-responders. The study was approved by the Medical Ethics committee of the universities of participating countries and for The Netherlands part by the University Medical Center Utrecht.

The baseline questionnaire was primarily used for the main PREDICT study to collect information on candidate predictors of the patients’ prognosis and included questions about demographics, health, lifestyle and several psychological measurements. After the informed consent and baseline questionnaire were returned, an appointment was made to conduct the Composite International Diagnostic Interview (CIDI). The CIDI was administered separately from the questionnaire. Information regarding health and health care consumption (e.g. consultation rate), however, was extracted from patients medical database, as this was considered a more reliable source.

In total, 3089 patients were asked to take part in the PREDICT-NL study, 75 were excluded because of problems understanding the Dutch language, 5 because of dementia, 2 because of psychosis and 1 because of mental retardation. Of the 3006 eligible patients, 1338 (44.5%) participated in the PREDICT-NL study. Reasons for not participating were mostly lack of time and no interest in the study. Since in the elderly, different factors (e.g. cognitive dysfunctioning and functional limitations) may predict the presence or absence of depression as compared to patients at younger age, we included patients aged 18–65 years (n = 1046) in the present analysis.

Diagnosis of major depressive disorder

The diagnosis of major depressive disorder at the baseline visit was assessed in all patients according to Diagnostic and Statistical Manual of Mental Disorders, Fourth Text Revision edition (DSM-IV) criteria19 using the depression section of the CIDI.20 The CIDI is a structured interview that is administered by trained researchers; it was conducted at the general practice. If the participant was unable to schedule the interview at the general practice, the interview was done by telephone (26% of the interviews) to obtain as complete as possible outcome information. Previous studies showed that telephone interviews are valid for clinical assessment of depression.3,21

The depression section of the CIDI was used to establish whether the patient had suffered a major depressive disorder over the past 6 months.

The interviewers were unaware of the values of the diagnostic predictors under study (see below) and the GPs were unaware of the diagnosis according to the CIDI.

Predictors for the presence of major depressive disorder

We a priori selected candidate predictors for the presence of major depressive disorder based on the literature and clinical reasoning. These predictors were collected for each patient with the baseline questionnaire and the medical records of the GPs. Dutch GPs register all contacts, diagnoses and interventions in an automated database using the International Classification of Primary Care (ICPC).22,23

The predictors were divided into two main categories: the first category included easily obtainable predictors that require no sensitive, depression-related questions during the consultation. These predictors were mainly obtained from the GP's automated database. The second category included predictors that are known risk factors for, and therefore more explicitly related to, major depressive disorder. The first category of predictors included the following: (i) gender; (ii) age; (iii) educational level; (iv) being single;24,25 (v) number of presenting complaints at the consult when recruitment took place (inclusion consult);26,27 (vi) assignment of a complaint (ICPC code levels <70) versus a diagnosis (ICPC code levels ≥70) at inclusion consult;4 (vii) assignment of non-somatic complaint or diagnosis at inclusion consult [ICPC code in chapter A (general), Z (social) or P (psychological) versus any other ICPC chapters, with exclusion of the depression codes P03 and P76]4; (viii) consultation rate (number of consults in the previous 12 months)26; (ix) assignment of an ICPC code for depression or depressive complaints (i.e. P03 or P76) in the previous 12 months and (x) prescription of antidepressants in the previous 12 months.

The second category of predictors included the number of life events in the previous 6 months28,29 and lifetime history of depression beyond the previous 6 months assessed with the two lifetime questions of the CIDI, i.e. depressed mood and loss of interest for a ≥2-week period, ever.28

Age and consultation rate were analysed as continuous variables. We verified whether these predictors were linearly associated with the outcome, using restricted cubic splines.30 Educational level was dichotomized into no or primary education only versus secondary and higher education. The number of complaints presented to the GP at the inclusion consult was categorized into 1, 2 and ≥3 health complaints as only 5.5% of the patients reported three or more complaints. The number of life events was categorized into 0, 1, 2 and ≥3 events since only a small number of patients (8%) reported more than three life events.

Data analysis

The overall percentage of missing values was 5.9%. Missing data rarely occur at random and a complete case analysis (deletion of all patients with one or more missing values) leads to loss of statistical power and to biased results. We therefore used multiple imputation to address the missing values, including missing values of the outcome.31–33

Univariable associations between the candidate predictors and the presence of major depressive disorder (yes/no) were estimated with logistic regression analysis. No selection was made based on these estimations since selection of predictors based on univariable statistics may result in unstable prediction models.30,34

Selection of predictors was performed in two steps with backward stepwise selection in multivariable logistic regression models. First, the most important predictors of the easily obtainable candidate predictors were selected with age and gender always retained in the model (Model 1). Second, the three known risk factors (number of life events in the previous 6 months, lifetime depressed mood and lifetime loss of interest) were added to Model 1 to quantify the added diagnostic value (Model 2). Backwards selection was based on Akaike's Information Criterium,35 which is similar to a selection based on a P-value of 0.157 if the predictor is modelled with one regression coefficient.

The ability to discriminate between patients with and without a major depressive disorder was studied with the concordance-statistic (c-statistic), i.e. the area under the Receiver Operating Characteristic curve. Calibration, which is the agreement between the observed proportions of major depressive disorder and the predicted risks, was studied with a calibration plot.30,36

Prediction models derived with multivariable regression analysis are known for overestimated regression coefficients, which results in too extreme predictions when applied in new patients.30,37 Therefore, we (internally) validated our models with bootstrapping techniques where in each bootstrap sample the entire modelling process was repeated. This yielded a shrinkage factor for the regression coefficients.30 The bootstrap procedure was also used to estimate a value of the c-statistic that was corrected for optimism. The corrected c-statistic may be considered as an estimate of discriminative ability that is expected in future patients.

To construct an easy to use clinical prediction rule, the shrunken regression coefficients of the predictors in Model 1 and Model 2 were transformed into points by multiplying by 10. Coefficients for categorized predictors were then rounded. Coefficients for continuous variables were first multiplied with the variable value and then rounded. The total scores were linked to the risk of major depressive disorder. The analyses were performed with SPSS 14 (SPSS Inc., Chicago, IL) and S-plus 6.2 (Insightful Corp., Seattle, WA).

Results

The mean age of the 1046 patients was 45 years (SD = 13, range 18–65 years) and 673 (64%) were female (Table 1). The majority of patients (n = 829, 79%) consulted the GP for one complaint. In 378 patients (36%), the GP did not assign a diagnosis (ICPC coding below level 70). The median consultation rate in the past 12 months was 8 (interquartile range: 5–15). One hundred and eleven (11%) patients had a non-somatic diagnosis or complaint (ICPC chapters P, Z or A). Major depressive disorder according to DSM-IV criteria was diagnosed in 157 patients (prevalence or a priori risk of 15%).

TABLE 1

Characteristics of 1046 primary care patients

Characteristics n (%) 
Candidate predictors  
    Female gender 673 (64) 
    Age, yearsa 44.7 (12.8) 
    Educational level, none/primary only 211 (20) 
    Being single 237 (23) 
Number of presented complaints  
    1 829 (79) 
    2 162 (16) 
    ≥3 55 (5) 
GP did not assign a diagnosis at inclusion consultb 378 (36) 
Non-somatic diagnosis/complaint at inclusion consultc 111 (11) 
Consultation rate (number of consults in past 12 months)d 8 (5–15) 
Received depression code in past 12 monthse 58 (6) 
Prescription of antidepressants in past 12 months 90 (9) 
Number of life events in past 6 months  
    0 408 (39) 
    1 285 (27) 
    2 188 (18) 
    ≥3 165 (16) 
Any depressed feelings, lifetime 514 (49) 
Any loss of interest, lifetime 421 (40) 
Outcome  
    Major depressive disorder 157 (15) 
Characteristics n (%) 
Candidate predictors  
    Female gender 673 (64) 
    Age, yearsa 44.7 (12.8) 
    Educational level, none/primary only 211 (20) 
    Being single 237 (23) 
Number of presented complaints  
    1 829 (79) 
    2 162 (16) 
    ≥3 55 (5) 
GP did not assign a diagnosis at inclusion consultb 378 (36) 
Non-somatic diagnosis/complaint at inclusion consultc 111 (11) 
Consultation rate (number of consults in past 12 months)d 8 (5–15) 
Received depression code in past 12 monthse 58 (6) 
Prescription of antidepressants in past 12 months 90 (9) 
Number of life events in past 6 months  
    0 408 (39) 
    1 285 (27) 
    2 188 (18) 
    ≥3 165 (16) 
Any depressed feelings, lifetime 514 (49) 
Any loss of interest, lifetime 421 (40) 
Outcome  
    Major depressive disorder 157 (15) 

Values are n (%) unless stated otherwise.

a

Mean (SD).

b

ICPC coding below level 70.

c

ICPC chapters general (A), social (Z) or psychological (P) excluding codes P03 and P76.

d

Median (interquartile range).

e

ICPC codes P03 or P76.

Female gender was univariably associated with a higher risk of having major depressive disorder (Table 2). Other variables that were univariably associated with a high risk of major depressive disorder were younger age, low educational level, being single, more than one presenting complaints, non-somatic diagnosis or complaint, higher consultation rate, depression code (P03 or P76) in past 12 months, prescription of antidepressants in past 12 months, one or more life events in the preceding 6 months and lifetime history of depression (Table 2).

TABLE 2

Univariable associations between candidate predictors and major depressive disorder

Predictor Major depressive disorder
 
 
 Yes No Odds ratio (95% CI) 
Female gender 115 (73) 558 (63) 1.62 (1.11–2.37) 
Age, yearsa 43.1 (12) 45.0 (13) 0.99 (0.98–1.00) 
Educational level, none/primary only 44 (28) 167 (19) 1.54 (0.97–2.46) 
Being single 55 (35) 182 (21) 2.10 (1.45–3.03) 
Number of presented complaints    
    1 107 (68) 722 (81) 
    2 34 (22) 128 (14) 1.65 (1.07–2.56) 
    ≥3 16 (10) 39 (4) 2.75 (1.48–5.10) 
GP did not assign a diagnosis at inclusion consultc 50 (32) 328 (37) 0.89 (0.59–1.34) 
Non-somatic diagnosis/complaint at inclusion consultd 29 (19) 82 (9) 2.21 (1.39–3.51) 
Consultation rate (number of consults in past 12 months)e 11 (7–20) 8 (5–14) 1.06 (1.04–1.09) 
Received depression codes in past 12 monthsf 33 (21) 25 (3) 8.94 (5.13–15.6) 
Prescription of antidepressants in past 12 months 41 (26) 49 (6) 6.03 (3.81–9.54) 
Number of life events in past 6 months    
    0 36 (23) 372 (42) 
    1 33 (21) 252 (28) 1.36 (0.83–2.25) 
    2 27 (17) 161 (18) 1.71 (1.00–2.93) 
    ≥3 61 (39) 104 (12) 6.09 (3.82–9.70) 
Any depressed feelings, lifetime 120 (76) 394 (44) 4.07 (2.75–6.02) 
Any loss of interest, lifetime 108 (69) 313 (35) 4.04 (2.80–5.83) 
Predictor Major depressive disorder
 
 
 Yes No Odds ratio (95% CI) 
Female gender 115 (73) 558 (63) 1.62 (1.11–2.37) 
Age, yearsa 43.1 (12) 45.0 (13) 0.99 (0.98–1.00) 
Educational level, none/primary only 44 (28) 167 (19) 1.54 (0.97–2.46) 
Being single 55 (35) 182 (21) 2.10 (1.45–3.03) 
Number of presented complaints    
    1 107 (68) 722 (81) 
    2 34 (22) 128 (14) 1.65 (1.07–2.56) 
    ≥3 16 (10) 39 (4) 2.75 (1.48–5.10) 
GP did not assign a diagnosis at inclusion consultc 50 (32) 328 (37) 0.89 (0.59–1.34) 
Non-somatic diagnosis/complaint at inclusion consultd 29 (19) 82 (9) 2.21 (1.39–3.51) 
Consultation rate (number of consults in past 12 months)e 11 (7–20) 8 (5–14) 1.06 (1.04–1.09) 
Received depression codes in past 12 monthsf 33 (21) 25 (3) 8.94 (5.13–15.6) 
Prescription of antidepressants in past 12 months 41 (26) 49 (6) 6.03 (3.81–9.54) 
Number of life events in past 6 months    
    0 36 (23) 372 (42) 
    1 33 (21) 252 (28) 1.36 (0.83–2.25) 
    2 27 (17) 161 (18) 1.71 (1.00–2.93) 
    ≥3 61 (39) 104 (12) 6.09 (3.82–9.70) 
Any depressed feelings, lifetime 120 (76) 394 (44) 4.07 (2.75–6.02) 
Any loss of interest, lifetime 108 (69) 313 (35) 4.04 (2.80–5.83) 

Values are n (%) unless stated otherwise.

a

Mean (SD).

b

Reference category.

c

ICPC coding below level 70.

d

ICPC chapters general (A), social (Z) or psychological (P) excluding codes P03 and P76.

e

Median (interquartile range).

f

ICPC codes P03 or P76.

Multivariable regression analysis showed that all easily obtainable predictors—except complaint versus diagnosis—remained in the model (Model 1, Table 3). The c-statistic of the model was 0.71 (95% CI: 0.67–0.76). Model extension with the three additional predictors—number of life events in the previous 6 months and the two lifetime questions on history of depression—increased the discriminative ability of the model to a c-statistic of 0.80 (95% CI: 0.76–0.83) (Model 2, Table 3). The effect of gender was retained in the model, even though the odds ratio was reduced to nearly 1 (Model 2, Table 3).

TABLE 3

Multivariable logistic regression models for the diagnosis of major depressive disorder

Predictor Model 1
 
Model 2
 
 Odds ratio (95% CI) Betaa (P value) Odds ratio (95% CI) Betaa (P value) 
Female gender 1.20 (0.79–1.82) 0.18 (0.40) 1.01 (0.64–1.57) 0.01 (0.98) 
Age, per year increase 0.99 (0.97–1.03) −0.01 (0.08) 0.99 (0.97–1.00) −0.01 (0.07) 
Educational level, none/primary only 1.50 (0.97–2.33) 0.41 (0.07) 1.42 (0.90–2.26) 0.35 (0.14) 
Being single 1.52 (1.00–2.29) 0.42 (0.05) 1.27 (0.82–1.96) 0.23 (0.29) 
Number of presented complaints     
    1 
    2 1.46 (0.91–2.37) 0.38 (0.12) 1.37 (0.83–2.26) 0.31 (0.23) 
    ≥3 2.03 (1.02–4.03) 0.71 (0.04) 2.09 (1.01–4.33) 0.74 (0.05) 
Non-somatic diagnosis/complaint at inclusion consultc 1.63 (0.95–2.79) 0.49 (0.07) 1.62 (0.89–2.95) 0.48 (0.11) 
Consultation rate (number of consults in past 12 months) 1.03 (1.00–1.06) 0.03 (0.03) 1.02 (0.99–1.05) 0.02 (0.27) 
Received depression code in past 12 monthsd 3.52 (1.77–6.99) 1.26 (0.00) 3.26 (1.59–6.70) 1.18 (0.00) 
Prescription of antidepressants in past 12 months 2.33 (1.29–4.20) 0.85 (0.00) 2.00 (1.07–3.76) 0.70 (0.03) 
Number of life events – –   
    0   
    1   1.21 (0.70–2.08) 0.19 (0.49) 
    2   1.41 (0.79–2.53) 0.34 (0.25) 
    ≥3   3.72 (2.20–6.29) 1.31 (0.00) 
Any depressed feelings, lifetime – – 1.71 (0.99–2.97) 0.54 (0.06) 
Any loss of interest, lifetime – – 1.70 (1.01–2.86) 0.53 (0.04) 
Predictor Model 1
 
Model 2
 
 Odds ratio (95% CI) Betaa (P value) Odds ratio (95% CI) Betaa (P value) 
Female gender 1.20 (0.79–1.82) 0.18 (0.40) 1.01 (0.64–1.57) 0.01 (0.98) 
Age, per year increase 0.99 (0.97–1.03) −0.01 (0.08) 0.99 (0.97–1.00) −0.01 (0.07) 
Educational level, none/primary only 1.50 (0.97–2.33) 0.41 (0.07) 1.42 (0.90–2.26) 0.35 (0.14) 
Being single 1.52 (1.00–2.29) 0.42 (0.05) 1.27 (0.82–1.96) 0.23 (0.29) 
Number of presented complaints     
    1 
    2 1.46 (0.91–2.37) 0.38 (0.12) 1.37 (0.83–2.26) 0.31 (0.23) 
    ≥3 2.03 (1.02–4.03) 0.71 (0.04) 2.09 (1.01–4.33) 0.74 (0.05) 
Non-somatic diagnosis/complaint at inclusion consultc 1.63 (0.95–2.79) 0.49 (0.07) 1.62 (0.89–2.95) 0.48 (0.11) 
Consultation rate (number of consults in past 12 months) 1.03 (1.00–1.06) 0.03 (0.03) 1.02 (0.99–1.05) 0.02 (0.27) 
Received depression code in past 12 monthsd 3.52 (1.77–6.99) 1.26 (0.00) 3.26 (1.59–6.70) 1.18 (0.00) 
Prescription of antidepressants in past 12 months 2.33 (1.29–4.20) 0.85 (0.00) 2.00 (1.07–3.76) 0.70 (0.03) 
Number of life events – –   
    0   
    1   1.21 (0.70–2.08) 0.19 (0.49) 
    2   1.41 (0.79–2.53) 0.34 (0.25) 
    ≥3   3.72 (2.20–6.29) 1.31 (0.00) 
Any depressed feelings, lifetime – – 1.71 (0.99–2.97) 0.54 (0.06) 
Any loss of interest, lifetime – – 1.70 (1.01–2.86) 0.53 (0.04) 

–; not selected in the model.

a

Beta: logistic regression coefficient, which was shrunk to improve predictions for future patients.

b

ICPC chapters general (A), social (Z) or psychological (P), excluding codes P03 and P76.

c

ICPC codes P03 or P76.

d

Reference category.

Figures 1 and 2 show the agreement between the predicted risks estimated with Model 1 and 2, respectively, and the observed proportions of major depressive disorder. Predicted risks around 25% of Model 1 were underestimated where predicted risks of ≥55% were overestimated. Model 2 clearly showed better calibration, with some discrepancy between predicted and observed risk in the high range of predicted risks (>50%). This is largely due to the low number of patients in these groups. Figures 3 and 4 show the score chart derived from Models 1 and 2, respectively (Table 3), that can be used as a clinical prediction rules. The regression coefficient for gender in Model 2 was close to zero (0.01). Therefore, gender was not included in the score chart. The lower part of Figures 3 and 4 show predicted risks and observed proportions for ranges of total scores. As with the calibration plots, some discrepancies between predicted risks and observed proportions were observed, especially in higher predicted risk categories, where the number of patients is low. An example of the use of the clinical prediction rules is given in the legend.

FIGURE 1

Agreement between the predicted risks of major depressive disorder according to Model 1 and the observed proportions. The solid line indicates the agreement between predicted risks of major depressive disorder and observed proportions. The dotted line indicates ideal calibration. The triangles indicate the observed proportions of major depressive disorder in patients with similar predicted risks grouped in quintiles. The vertical lines just above the horizontal axis show the distribution of the predicted risks

FIGURE 1

Agreement between the predicted risks of major depressive disorder according to Model 1 and the observed proportions. The solid line indicates the agreement between predicted risks of major depressive disorder and observed proportions. The dotted line indicates ideal calibration. The triangles indicate the observed proportions of major depressive disorder in patients with similar predicted risks grouped in quintiles. The vertical lines just above the horizontal axis show the distribution of the predicted risks

FIGURE 2

Agreement between the predicted risks of major depressive disorder according to Model 2 and the observed proportions. The solid line indicates the agreement between predicted risks of major depressive disorder and observed proportions. The dotted line indicates ideal calibration. The triangles indicate the observed proportions of major depressive disorder in patients with similar predicted risks grouped in quintiles. The vertical lines just above the horizontal axis show the distribution of the predicted risks

FIGURE 2

Agreement between the predicted risks of major depressive disorder according to Model 2 and the observed proportions. The solid line indicates the agreement between predicted risks of major depressive disorder and observed proportions. The dotted line indicates ideal calibration. The triangles indicate the observed proportions of major depressive disorder in patients with similar predicted risks grouped in quintiles. The vertical lines just above the horizontal axis show the distribution of the predicted risks

FIGURE 3

Score chart based on Model 1 to calculate the predicted risk of major depressive disorder for an individual primary care patient. The upper part shows the points corresponding to each predictor value. For the continuous predictors not all values are given. The correct number of points for age can be found by rounding downwards to a value in the chart. The correct number of points for consultation rate can be found by rounding upwards to a value in the chart. The points are summed up into a score. The corresponding risk for major depressive disorder can be found for ranges of scores in the lower part of Figure 2 in the column ‘predicted risk’. For comparison, the observed percentage of patients with major depressive disorder is shown in the column ‘observed proportions’

FIGURE 3

Score chart based on Model 1 to calculate the predicted risk of major depressive disorder for an individual primary care patient. The upper part shows the points corresponding to each predictor value. For the continuous predictors not all values are given. The correct number of points for age can be found by rounding downwards to a value in the chart. The correct number of points for consultation rate can be found by rounding upwards to a value in the chart. The points are summed up into a score. The corresponding risk for major depressive disorder can be found for ranges of scores in the lower part of Figure 2 in the column ‘predicted risk’. For comparison, the observed percentage of patients with major depressive disorder is shown in the column ‘observed proportions’

FIGURE 4

Score chart based on Model 2 to calculate the predicted risk of major depressive disorder for an individual primary care patient. The upper part shows the points corresponding to each predictor value. For the continuous predictors not all values are given. The correct number of points for age can be found by rounding downwards to a value in the chart. The correct number of points for consultation rate can be found by rounding upwards to a value in the chart. The points are summed up into a score. The corresponding risk for major depressive disorder can be found for ranges of scores in the lower part of Figure 3 in the column ‘predicted risk’. For comparison, the observed percentage of patients with major depressive disorder is shown in the column ‘observed proportions’. To illustrate the use of the score chart: a patient, aged 27 (four points), with a high education (zero points) and single (two points) consults the GP. The patient presents three separate health complaints (seven points), all somatic (zero points). This consultation is the 3rd in the past 12 months (two points). The medical database shows that the patient did not receive a depression code in the prior consultations (zero points) and no antidepressive medication was prescribed in the previous 12 months (zero points). The GP further enquires about recent life events (three life events, 13 points) and asks whether the patient has ever had depressed feelings for >14 days (yes, five points) or loss of interest (no, zero points). The score is 33, which relates to a predicted risk of major depressive disorder of 52% (Figure 4, lower part)

FIGURE 4

Score chart based on Model 2 to calculate the predicted risk of major depressive disorder for an individual primary care patient. The upper part shows the points corresponding to each predictor value. For the continuous predictors not all values are given. The correct number of points for age can be found by rounding downwards to a value in the chart. The correct number of points for consultation rate can be found by rounding upwards to a value in the chart. The points are summed up into a score. The corresponding risk for major depressive disorder can be found for ranges of scores in the lower part of Figure 3 in the column ‘predicted risk’. For comparison, the observed percentage of patients with major depressive disorder is shown in the column ‘observed proportions’. To illustrate the use of the score chart: a patient, aged 27 (four points), with a high education (zero points) and single (two points) consults the GP. The patient presents three separate health complaints (seven points), all somatic (zero points). This consultation is the 3rd in the past 12 months (two points). The medical database shows that the patient did not receive a depression code in the prior consultations (zero points) and no antidepressive medication was prescribed in the previous 12 months (zero points). The GP further enquires about recent life events (three life events, 13 points) and asks whether the patient has ever had depressed feelings for >14 days (yes, five points) or loss of interest (no, zero points). The score is 33, which relates to a predicted risk of major depressive disorder of 52% (Figure 4, lower part)

Discussion

We developed a two-step clinical prediction rule to predict the likelihood of presence of major depressive disorder in primary care patients. The first step was to develop a model with easily obtainable predictors (Model 1, Table 3) that could be used by the GP without asking for specific, sensitive questions with reference to major depressive disorder. This was done because GPs may be reluctant to ask specific depression-related questions, particularly if patients present with somatic complaints. In addition, patients may be reluctant to answer such questions. This first model was then extended by inclusion of three known risk factors that are more explicitly related to major depressive disorder (Model 2). This extension clearly improved the discriminative ability and calibration of the model. The use of both models is facilitated by a simple to use score chart (Figs 3 and 4). Although the model requires use of >10 parameters, the majority of predictors can simply be derived from the (often electronic) medical files in primary care practices, without having to question the patient. The applicability of the two-step clinical prediction rule can obviously be improved by incorporating it (notably the first model) as an automatic tool (calculator) in the electronic patient medical record. During the consultation, the programme could ‘warn’ GPs if a patient is at an elevated risk of major depressive disorder.

Since adding the three extra predictors in Model 2 substantially improved the discrimination, we advocate a two-step approach to select patients in primary care practice who can benefit from diagnostic workup. Model 1 can be used as a first selection tool to distinguish patients with lower predicted risk from patients with higher predicted risk of major depressive disorder. The use of Model 1 can be facilitated by a computer programme (using data from the most recent consultation for the ‘inclusion consult’) to calculate the predicted risk and alert the GP to patients at relatively high risk during the consult. The doctor might only ask questions related to major depressive disorder that are included in Model 2 when the predicted risk of Model 1 is sufficiently high. For example, a female patient (two points), aged 25 (five points), with primary education only (four points) and single (four points) consults the GP. The patient presents with three health complaints (seven points), all somatic (zero points). This consultation is the second in the past 12 months (one point) and the visits did not result in an ICPC code for depression (zero points) or in prescriptions for antidepressants (zero points). As a consequence, this patient receives a total score of 23 that relates to a predicted risk of 30% (Fig. 3). For this patient, it seems reasonable to ask the additional questions of Model 2.

We need to define threshold values for ‘high risk of major depressive disorder’, in order to apply the two models. Ideally, such threshold values are assessed in formal decision analyses that weigh benefits and harms of further diagnostic workup. Since these analyses are currently lacking, we propose a low threshold of ≥11 for Model 1. Using this threshold, 66% of the patients with major depressive disorder will be asked the additional questions of Model 2. For Model 2, we would recommend a threshold of ≥21 as an indicator for a further diagnostic workup. This would result in a diagnostic workup for most of the patients with major depressive disorder (n = 107; 68%). We report the risk for several total score categories of both models to enable GPs to choose their desired threshold of risk of major depressive disorder.

Strong predictors (variables with high odds ratios) in Model 1 were the assignment of an ICPC code for depression or depressive complaint in the past 12 months and prescription of antidepressants in the past 12 months. The number of presented health complaints and assignment of a non-somatic diagnosis or complaint at the inclusion consult were also strong predictors. These results are consistent with other studies.25,26,28

It may seem odd to include the predictors ‘depression or depressive complaint in the last 12 months’ and ‘prescription of antidepressants in the last 12 months’ for the detection of major depressive disorder. However, a diagnosis of major depressive disorder by GPs is usually not assessed with reference tests such as the CIDI and therefore not the same as the diagnosis of major depressive disorder assessed in our study. Further, antidepressants may be used for other conditions (i.e. anxiety, obsessive–compulsive disorder and dysthymia) than major depressive disorder.

Three or more life events increased the risk for major depressive disorder dramatically. Life events are a known risk factor for major depressive disorder.11,24,25 The clear distinction in risk between one or two versus three or more life events was unexpected, though it has been reported that a high number of life events is associated with persistence of major depressive disorder.38 Gender was predictive in Model 1 but had no predictive value anymore in Model 2. This is due to the association between the gender and the extra three predictors that were included in Model 2, which in fact took over the predictive ability of gender. Women had experienced life events more often and responded positively more often on the CIDI lifetime questions.

The question may arise whether our models were similar if patients who were already diagnosed by their GP for depression were excluded from the analysis. An additional analysis excluding patients with ICPC depression codes P03 or P76 showed similar results (analysis not shown). The selected predictors were the same—except for the ICPC depression codes that were excluded in this analysis—and yielded similar regression coefficients. We specifically chose to include patients with recognized or treated depression since the degree of recognition might vary across GPs.

To our knowledge, this is the first attempt to develop and internally validate a simple rule for GPs to determine the risk of major depressive disorder in individual patients, with a minimal need for depression-related questions. A strength of this study is the inclusion of consecutive primary care patients, irrespective of their presented symptoms or signs. No bias was introduced due to selection, as the reference test was applied in all patients.

Our study has some limitations. First, the non-response rate for our study was relatively high. Compared to responders, however, we found very minor differences in distributions of gender and age. Since other predictors could not be collected in the non-response groups, we can only assume that both groups were largely similar with respect to the other characteristics. Furthermore, the prevalence of major depressive disorder in this study was relatively high. This may suggest that patients with mood problems were more willing to participate and may have resulted in an overestimation of the predicted risk of major depressive disorder. The low response rate and high prevalence could indeed be indications of some bias in the inclusion of patients. Consequently, we stress that external validation of our prediction rule, as is always the case with developed rules, is needed to study the generalizability of our models.

Second, some predictors in Model 1 may currently not be available on GP records. For example in the UK, the information on single status and education is not available. GPs should first have this information, before the model can be automatically applied. Third, the clinical prediction rule was explicitly developed on data of adult patients aged 18–65. Older patients were excluded; the prediction rule may not be valid for these patients.

In conclusion, we have developed a clinical prediction rule to detect major depressive disorder in adult primary care patients. The two score charts were internally validated with a bootstrapping technique. External validation, including prospective testing, is required before the models can be applied in other general practices. If proved to be generalizable, the prediction rules can be used to detect primary care patients for a workup to diagnose major depressive disorder. Early detection of major depressive disorder, provided it is followed by adequate treatment and follow-up, may improve the long-term prognoses for these patients.

Declaration

Funding: The European Commission (ref PREDICT-QL4-CT2002-00683); The Netherlands Organization for Scientific Research (ref ZonMw 016.046.360).

Ethical approval: This study was approved by the Medical Ethics committee of the University Medical Center Utrecht.

Conflict of interest: none.

We thank participating patients for their time and effort. We thank the University Medical Center Utrecht primary care network and its participating GPs for their participation in this study.

References

1
Murray
CJ
Lopez
AD
Global mortality, disability, and the contribution of risk factors: Global Burden of Disease Study
Lancet
 , 
1997
, vol. 
349
 (pg. 
1436
-
42
)
2
Kruijshaar
ME
Barendregt
J
Vos
T
, et al.  . 
Lifetime prevalence estimates of major depression: an indirect estimation method and a quantification of recall bias
Eur J Epidemiol
 , 
2005
, vol. 
20
 (pg. 
103
-
11
)
3
Kessler
RC
Berglund
P
Demler
O
, et al.  . 
The epidemiology of major depressive disorder: results from the National Comorbidity Survey Replication (NCS-R)
J Am Med Assoc
 , 
2003
, vol. 
289
 (pg. 
3095
-
105
)
4
Goldman
LS
Nielsen
NH
Champion
HC
Awareness, diagnosis, and treatment of depression
J Gen Intern Med
 , 
1999
, vol. 
14
 (pg. 
569
-
80
)
5
McQuaid
JR
Stein
MB
Laffaye
C
McCahill
ME
Depression in a primary care clinic: the prevalence and impact of an unrecognized disorder
J Affect Disord
 , 
1999
, vol. 
55
 (pg. 
1
-
10
)
6
Rost
K
Zhang
M
Fortney
J
, et al.  . 
Persistently poor outcomes of undetected major depression in primary care
Gen Hosp Psychiatry
 , 
1998
, vol. 
20
 (pg. 
12
-
20
)
7
Pignone
MP
Gaynes
BN
Rushton
JL
, et al.  . 
Screening for depression in adults: a summary of the evidence for the U.S. Preventive Services Task Force
Ann Intern Med
 , 
2002
, vol. 
136
 (pg. 
765
-
76
)
8
Bijl
D
van Marwijk
HW
de Haan
M
van Tilburg
W
Beekman
AJ
Effectiveness of disease management programmes for recognition, diagnosis and treatment of depression in primary care
Eur J Gen Pract
 , 
2004
, vol. 
10
 (pg. 
6
-
12
)
9
Barkow
K
Maier
W
Ustun
TB
, et al.  . 
Risk factors for depression at 12-month follow-up in adult primary health care patients with major depression: an international prospective study
J Affect Disord
 , 
2003
, vol. 
76
 (pg. 
157
-
69
)
10
Nuyen
J
Volkers
AC
Verhaak
PF
, et al.  . 
Accuracy of diagnosing depression in primary care: the impact of chronic somatic and psychiatric co-morbidity
Psychol Med
 , 
2005
, vol. 
35
 (pg. 
1185
-
95
)
11
Olsen
LR
Mortensen
EL
Bech
P
Prevalence of major depression and stress indicators in the Danish general population
Acta Psychiatr Scand
 , 
2004
, vol. 
109
 (pg. 
96
-
103
)
12
Cepoiu
M
McCusker
J
Cole
MG
, et al.  . 
Recognition of depression by non-psychiatric physicians—a systematic literature review and meta-analysis
J Gen Intern Med
 , 
2008
, vol. 
23
 (pg. 
25
-
36
)
13
Kroenke
K
Spitzer
RL
Williams
JB
The PHQ-9: validity of a brief depression severity measure
J Gen Intern Med
 , 
2001
, vol. 
16
 (pg. 
606
-
13
)
14
Williams
JW
Jr
Pignone
M
Ramirez
G
Perez
SC
Identifying depression in primary care: a literature synthesis of case-finding instruments
Gen Hosp Psychiatry
 , 
2002
, vol. 
24
 (pg. 
225
-
37
)
15
Williams
JW
Jr
Noel
PH
Cordes
JA
Ramirez
G
Pignone
M
Is this patient clinically depressed?
J Am Med Assoc
 , 
2002
, vol. 
287
 (pg. 
1160
-
70
)
16
Zigmond
AS
Snaith
RP
The hospital anxiety and depression scale
Acta Psychiatr Scand
 , 
1983
, vol. 
67
 (pg. 
361
-
70
)
17
King
M
Weich
S
Torres
F
, et al.  . 
Prediction of depression in European general practice attendees: the PREDICT study
BMC Public Health
 , 
2006
, vol. 
6
 pg. 
6
 
18
King
M
Walker
C
Levy
G
, et al.  . 
Development and validation of an international risk prediction algorithm for episodes of major depression in general practice attendees: the PredictD study
Arch Gen Psychiatry
 , 
2008
, vol. 
65
 (pg. 
1368
-
76
)
19
American Psychiatric Association
Diagnostic and Statistical Manual of Mental Disorders 4th Text Revision Edition
 , 
2000
Washington, DC
American Psychiatric Association
20
Ter Smitten
RH
Smeets
RMW
Van den Brink
W
Composite International Diagnostic Interview—Computerized Version 2.1: Dutch Translation and Adaptation. WHO-CIDI Training en Referentie centrum
 , 
2007
Amsterdam, The Netherlands
Psychiatrisch Centrum AMC
21
Sobin
C
Weissman
MM
Goldstein
RB
, et al.  . 
Diagnostic interviewing for family studies: comparing telephone and face-to-face methods for the diagnosis of lifetime psychiatric disorders
Psychiatr Genet
 , 
1993
, vol. 
3
 (pg. 
227
-
34
)
22
Classification Committee of WONCA
International Classification of Primary Care
 , 
1998
Oxford
Oxford University Press
23
Okkes
I
Jamoulle
M
Lamberts
H
Bentzen
N
ICPC-2-E: the electronic version of ICPC-2. Differences from the printed version and the consequences
Fam Pract
 , 
2000
, vol. 
17
 (pg. 
101
-
7
)
24
De Graaf
R
Bijl
RV
Ravelli
A
Smit
F
Vollebergh
WA
Predictors of first incidence of DSM-III-R psychiatric disorders in the general population: findings from the Netherlands Mental Health Survey and Incidence Study
Acta Psychiatr Scand
 , 
2002
, vol. 
106
 (pg. 
303
-
13
)
25
Salokangas
RK
Poutanen
O
Risk factors for depression in primary care. Findings of the TADEP project
J Affect Disord
 , 
1998
, vol. 
48
 (pg. 
171
-
80
)
26
Katon
W
Berg
AO
Robins
AJ
Risse
S
Depression—medical utilization and somatization
West J Med
 , 
1986
, vol. 
144
 (pg. 
564
-
8
)
27
Ormel
J
Bartel
M
Nolen
WA
[Undertreatment of depression; causes and recommendations]
Ned Tijdschr Geneeskd
 , 
2003
, vol. 
147
 (pg. 
1005
-
9
)
28
Barkow
K
Maier
W
Ustun
TB
, et al.  . 
Risk factors for new depressive episodes in primary health care: an international prospective 12-month follow-up study
Psychol Med
 , 
2002
, vol. 
32
 (pg. 
595
-
607
)
29
Brugha
TS
Cragg
D
The list of threatening experiences: the reliability and validity of a brief life events questionnaire
Acta Psychiatr Scand
 , 
1990
, vol. 
82
 (pg. 
77
-
81
)
30
Harrell
FE
Jr
Lee
KL
Mark
DB
Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors
Stat Med
 , 
1996
, vol. 
15
 (pg. 
361
-
87
)
31
Donders
AR
van der Heijden
GJ
Stijnen
T
Moons
KG
Review: a gentle introduction to imputation of missing values
J Clin Epidemiol
 , 
2006
, vol. 
59
 (pg. 
1087
-
91
)
32
Little
RJA
Rubin
DB
Statistical Analysis with Missing Data
 , 
1987
New York
Wiley
33
van der Heijden
GJ
Donders
AR
Stijnen
T
Moons
KG
Imputation of missing values is superior to complete case analysis and the missing-indicator method in multivariable diagnostic research: a clinical example
J Clin Epidemiol
 , 
2006
, vol. 
59
 (pg. 
1102
-
9
)
34
Sun
GW
Shook
TL
Kay
GL
Inappropriate use of bivariable analysis to screen risk factors for use in multivariable analysis
J Clin Epidemiol
 , 
1996
, vol. 
49
 (pg. 
907
-
16
)
35
Atkinson
A
A note on the generalized information criterion for choice of a model
Biometrika
 , 
1980
, vol. 
67
 (pg. 
413
-
8
)
36
Miller
ME
Hui
SL
Tierney
WM
Validation techniques for logistic regression models
Stat Med
 , 
1991
, vol. 
10
 (pg. 
1213
-
26
)
37
Spiegelhalter
DJ
Probabilistic prediction in patient management and clinical trials
Stat Med
 , 
1986
, vol. 
5
 (pg. 
421
-
33
)
38
Spijker
J
De Graaf
R
Bijl
RV
, et al.  . 
Determinants of persistence of major depressive episodes in the general population. Results from the Netherlands Mental Health Survey and Incidence Study (NEMESIS)
J Affect Disord
 , 
2004
, vol. 
81
 (pg. 
231
-
40
)

Author notes

Zuithoff NPA, Vergouwe Y, King M, Nazareth I, Hak E, Moons KGM and Geerlings MI. A clinical prediction rule for detecting major depressive disorder in primary care: the PREDICT-NL study. Family Practice 2009; 26: 241–250.