Abstract

This article discusses the effects of sociodemographics and the presence of psychiatric disorders diagnosed in the 12 months before the first interview by using the Diagnostic and Statistical Manual of Mental Disorders: DSM-III-R, third edition, revised, on three types of attrition (failure to locate, refusal to participate, morbidity/mortality) in the second wave (1997–1998) of the Netherlands Mental Health Survey and Incidence Study, a longitudinal, general population survey of psychopathology among 7,076 subjects aged 18–64 years. Compared with those reinterviewed successfully, persons not located at the 1-year follow-up (n = 219) were more often younger, poorly educated, urban, not cohabiting with a steady partner, and born outside the Netherlands. Refusers (n = 923) had a lower educational level. Morbidity/mortality (n = 72) was associated with higher age, lower educational level, not being employed, and somatic disorders. After adjustment for sociodemographics, none of the disorders was positively associated with refusal. Failure to locate was linked to agoraphobia, alcohol abuse, and the categories of mood, substance use, and eating disorders. Morbidity/mortality was linked to dysthymia, agoraphobia, simple phobia, obsessive-compulsive disorder, and the category of anxiety disorders. Overall attrition was only slightly higher among respondents with one or more disorders (odds ratio = 1.20, 95% confidence interval: 1.04, 1.38). Thus, psychopathology has only weak-to-moderate effects on attrition and is mainly related to failure to locate and morbidity/mortality but not to refusal.

Loss of respondents in the second or subsequent waves of longitudinal data collection can be selective. This selection bias can reduce the internal or external validity of the research findings (1, 2). Methodological research of the determinants of attrition makes it possible to identify which categories of subjects are most likely to be lost to follow-up in longitudinal studies. Such categories can then be specially targeted in order to curb such losses in later waves of a study (1, 3). Attrition may be associated with certain demographic characteristics of the participants but also with the dependent variable itself (46).

Factors associated with attrition in longitudinal surveys have been investigated in a number of studies (3, 7), but little data are available on the loss of respondents in general population studies on mental health. Only a few such large-scale longitudinal studies exist, the best known of which are the Epidemiologic Catchment Area (ECA) Program in the United States (8), the Stirling County study in Canada (9), and the Lundby Study in Sweden (10). Only the ECA study extensively reported on the relation between psychopathology and attrition (8). Since mental health research is now on the increase, there is a growing need to identify its characteristic sources of sample attrition.

To understand as clearly as possible the differences between respondents who continue to participate and those lost to follow-up, we need to distinguish between different types of attrition. The most prominent groups of respondents lost to follow-up are those who refuse further participation, those who cannot be located at follow-up, and those who become incapacitated by illness or have died. Psychopathology may be associated with all three types of attrition. In studies of clinical samples, psychopathology is associated with higher rates of mortality; attitudes toward the survey process and social interaction in general may be associated with some forms of psychopathology, which may affect the rate of refusal to participate; and psychopathology may be associated with residential mobility (8).

In the United States, the Epidemiologic Follow-Up Study of the National Health and Nutrition Examination Survey I (8-year follow-up among 2,981 subjects) has found distinctly different predictors for the three types of attrition (3). Psychiatric diagnoses were not a focus of the study, but a Center for Epidemiologic Studies Depression Scale was administered. Multivariate analysis revealed that participants with a high score, as well as those who were young, single, and smoked tobacco, were less likely to be located for follow-up. Educational level was the only factor associated with refusal; those at the lowest level were more likely to refuse participation. Males, elderly people, single people, the unemployed, people with high blood pressure, and smokers were more likely to die during the follow-up interval. Thus, depression was linked to failure to locate but not to refusal or mortality.

The Journal has published findings from the ECA, a 1-year follow-up study among 10,167 subjects, on the effects of psychopathology on attrition (8). Respondents not located at follow-up were more likely to be male, young, moderately educated, and unmarried than those who were reinterviewed successfully. Refusal to participate was associated with higher age and lower educational level. Attrition due to morbidity or mortality was not assessed. After adjustment for demographics, psychopathology was found to have a weak-to-moderate effect on attrition, with a stronger effect on failure to locate than on refusal. Failure to locate was associated with the presence, measured at baseline, of 12-month panic disorder, major depression, drug abuse/dependence, alcohol abuse/dependence, and antisocial personality disorder. Major depression was the only disorder associated with refusal but in the reverse direction than the one expected. One of the five ECA data collection sites, Baltimore, Maryland, also investigated attrition in a 15-year follow-up (11). Failure to locate and mortality were strongly influenced by psychopathology, but refusal was not.

Similar to the paper by Eaton et al. (8), the present article describes the sociodemographic and psychiatric determinants of attrition in the second wave (1-year follow-up) of the Netherlands Mental Health Survey and Incidence Study (NEMESIS), a longitudinal survey investigating psychopathology in the general population. In addition to failure to locate and refusal to participate, we investigated morbidity and mortality as a source of attrition. Item nonresponse was not dealt with, since it was negligible as a result of the computer-assisted interviewing.

MATERIALS AND METHODS

First wave

NEMESIS is based on a multistage, stratified, random sampling procedure (12). Our first step was to draw a sample of 90 Dutch municipalities. The stratification criteria were urbanization (five categories as classified by the Netherlands Central Bureau of Statistics) and adequate dispersion over the 12 provinces. The second step was to draw a sample of private households (addresses) from post office registers. The number of households selected in each municipality was determined by the size of its population. The third step was to choose which persons to interview. The selected households were sent a letter of introduction signed by the national government Minister of Health, Welfare and Sport requesting them to take part. Shortly thereafter, interviewers contacted these households by telephone. Households with no telephones or with unlisted numbers (18 percent) were visited in person. One respondent was randomly selected from each household, the member with the most recent birthday, on the condition that he or she was aged 18–64 years and sufficiently fluent in Dutch to be interviewed. Persons who were not immediately available (because of such circumstances as hospitalization, travel, or imprisonment) were contacted later in the year. To establish contact, the interviewers made a minimum of 10 telephone calls or visits to a given address at different times of the day and week, if necessary. Respondents received a token of appreciation at the end of the interview.

To optimize the response rate and to compensate for possible seasonal influences, we spread the initial data collection phase over the entire period from February through December 1996. No adjustments had to be made to the procedure in the course of the fieldwork; hence, no additional respondents were drawn from specific groups.

A total of 7,076 persons were interviewed in the first wave. Depending on the method of calculation, the response rate was 64.2 percent (of the households eligible for interviewing) or 69.7 percent (of the adults eligible for interviewing) (12). No proxy information was collected. Persons who declined to take part in the full interview (despite our offer of a 50-guilder (20-dollar) gift certificate to these subjects only) were asked to furnish several key data (age, gender) and to complete the General Health Questionnaire-12, a screener for current mental health problems (13). Some 43.6 percent of the refusers agreed to do so. These nonrespondents were found to have a slightly lower average questionnaire score (that is, to be in better mental health) than the respondents (1.16 vs. 1.22); they also had a lower average age, and there was a higher percentage of women. Compared with the Dutch population (according to the Netherlands Central Bureau of Statistics), survey participants are fairly representative of the population in terms of gender, civil status, and degree of urbanization of place of residence (12). Only the group aged 18–24 years was significantly underrepresented.

Second wave

All persons who had taken part in the first interview were approached for the follow-up. As in the first wave, the interviewers made a minimum of 10 telephone calls or visits at various times of the day and week to establish contact. A tracing process involving mail, telephone calls, field tracing, and municipality records was used to locate the original sample. At the end of the first interview, a change of address card was left behind that could be mailed to us in case a respondent moved. Interviewers recorded the reasons that respondents failed to continue participation.

The fieldwork in the second wave took place from February 1997 through January 1998. The mean interval between the first and second interviews was 379 days (standard deviation, 35), slightly longer than the intended 365 days.

Variables

The dependent variables were the three types of attrition: failure to locate respondents, refusal to participate, and morbidity or mortality. We used two types of prognostic variables, measured at baseline, that might potentially predict attrition: sociodemographic characteristics and the presence of psychopathology at baseline or at any time during the 12 months before the first interview. The demographic variables were gender, age, educational level, degree of urbanization (rural, municipalities with fewer than 500 addresses per square kilometer; urban, municipalities with 500 or more addresses per square kilometer), cohabitation status (irrespective of children), employment status, country of birth, and presence of one or more conditions from a list of 31 somatic disorders treated or monitored by a physician in the 12 months prior to baseline (for example, asthma, lung emphysema, arthritis, rheumatism, heart disease, heart attack, stomach or intestinal ulcer, diabetes).

The diagnoses of psychiatric disorders were based on the Diagnostic and Statistical Manual of Mental Disorders: DSM-III-R, third edition, revised (DSM-III-R) (14). The instrument used to determine the diagnoses was the computerized version of the Composite International Diagnostic Interview (CIDI), version 1.1 (15). The CIDI is a structured interview developed by the World Health Organization (16, 17) on the basis of the Diagnostic Interview Schedule and the Present State Examination. It was designed for use by trained interviewers who are not clinicians. CIDI version 1.1 has two diagnostic programs to compute diagnoses according to the criteria and definitions of either DSM-III-R or the International Statistical Classification of Diseases and Related Health Problems, Tenth Revision. The CIDI is now being used worldwide, and World Health Organization research has found high interrater reliability (18, 19) and high test-retest reliability (2022). The following DSM-III-R diagnoses were recorded in the NEMESIS data set: mood disorders (depression, dysthymia, bipolar disorder), anxiety disorders (panic disorder, agoraphobia, social phobia, simple phobia, obsessive-compulsive disorder, generalized anxiety disorder), psychoactive substance use disorders (alcohol or drug abuse and dependence, including sedatives, hypnotics, and anxiolytics), eating disorders, schizophrenia, and other nonaffective psychotic disorders.

Statistical analysis

We first carried out bivariate and multivariate logistic regression analyses to obtain the odds ratios and their 95 percent confidence intervals that reflected the associations between the demographic characteristics and the three types of attrition. Respondents who were reinterviewed successfully served as the comparison category.

We then used logistic regression to investigate to what extent psychopathology in the 12 months prior to the first interview predicted the three types of attrition. Because the prevalence of psychopathology varies strongly by demographic characteristics (23, 24), these analyses were adjusted for demographics. The odds ratios were calculated in a series of models into which the presence of each disorder or each category of disorders was introduced, along with the demographic variables. In addition, these analyses also were carried out by using lifetime disorders measured at baseline. Since we focused on associations, we made no use of sample weights that generalize to the general population, except when reporting prevalences of disorders.

RESULTS

Attrition

Of the 7,076 persons who participated in the first interview, 5,618 were reinterviewed in the second wave. Thus, the follow-up response rate was 79.4 percent, considerably higher than the initial response rate of 64.2 or 69.7 percent (depending on the method of calculation). Table 1 shows the attrition broken down according to its apparent sources. The predominant source was refusal.

TABLE 1.

Attrition in the second wave of the Netherlands Mental Health Survey and Incidence Study (NEMESIS), 1997–1998

 No. 
Total respondents in the first wave (1996–1997) 7,076 100.0 
Attrition in the second wave (1997–1998)* 1,458 20.6 
 Sources of attrition   
  Respondents not located   
   Not able to reach 112 1.6 
   Moved to unknown address, not traceable 81 1.1 
   Interview not possible within required period 26 0.4 
  Refusal of further participation   
   Unpleasant experience with previous interview 163 2.3 
   Too busy; no time 325 4.6 
   No purpose; no interest; no desire 435 6.1 
  Physical reasons   
   Illness; death 72 1.0 
  Other causes   
   Reason unknown 55 0.8 
   Technical problems transmitting data files from interviewers to researchers 122 1.7 
   Inadvertent failure to contact 67 0.9 
 No. 
Total respondents in the first wave (1996–1997) 7,076 100.0 
Attrition in the second wave (1997–1998)* 1,458 20.6 
 Sources of attrition   
  Respondents not located   
   Not able to reach 112 1.6 
   Moved to unknown address, not traceable 81 1.1 
   Interview not possible within required period 26 0.4 
  Refusal of further participation   
   Unpleasant experience with previous interview 163 2.3 
   Too busy; no time 325 4.6 
   No purpose; no interest; no desire 435 6.1 
  Physical reasons   
   Illness; death 72 1.0 
  Other causes   
   Reason unknown 55 0.8 
   Technical problems transmitting data files from interviewers to researchers 122 1.7 
   Inadvertent failure to contact 67 0.9 
*

Percentages do not total 20.6 because of rounding.

As discussed in further detail below, the types of attrition we included were respondent refusal, failure to locate respondents, and morbidity/mortality. Sources not included in these analyses were errors in carrying out the study (technical problems in transmitting data files from the interviewers to the researchers, inadvertent failure to contact respondents) and unknown causes.

Sociodemographic characteristics and attrition

We first focused on the relation between follow-up status and certain sociodemographic characteristics. Table 2 presents the demographic characteristics of the respondents who stayed in the sample and of those in the three groups lost to follow-up. We used this distribution to calculate the crude odds ratios for attrition, comparing each nonresponse group with the response group (table 3). Table 3 also shows the multivariate odds ratios for the three types of attrition relative to the demographic variables.

TABLE 2.

Sociodemographic characteristics (%) of respondents and nonrespondents in the second wave of the Netherlands Mental Health Survey and Incidence Study (NEMESIS), 1997–1998*

 Respondents (n = 5,618) Nonrespondents
 
Failure to locate (n = 219) Refusal (n = 923) Morbidity/mortality (n = 72) 
Gender     
 Male 46.5 50.7 46.4 48.6 
 Female 53.5 49.3 53.6 51.4 
Age (years)     
 18–24 7.6 24.7 10.0 1.4 
 25–34 26.1 33.3 20.4 5.6 
 35–44 27.6 21.5 24.3 18.1 
 45–54 22.0 13.7 21.6 27.8 
 55–64 16.7 6.8 23.8 47.2 
Education     
 Primary, basic vocational 26.0 28.0 40.8 53.5 
 Lower secondary 37.1 38.3 36.2 28.2 
 Higher secondary 7.6 6.1 5.7 5.6 
 Higher professional, university 29.3 27.6 17.3 12.7 
Urbanization     
 Rural 17.4 6.8 15.8 13.9 
 Urban 82.6 93.2 84.2 86.1 
Cohabitation status     
 Not living with a partner 29.5 59.4 29.8 30.6 
 Living with a partner 70.5 40.6 70.2 69.4 
Employment status     
 No paid employment 36.3 40.2 44.6 63.9 
 Paid employment 63.7 59.8 55.4 36.1 
Country of birth     
 The Netherlands 93.8 81.3 92.4 90.3 
 Elsewhere 6.2 18.7 7.6 9.7 
One or more somatic disorders     
 No 59.4 61.6 58.7 30.6 
 Yes 40.6 38.4 41.3 69.4 
 Respondents (n = 5,618) Nonrespondents
 
Failure to locate (n = 219) Refusal (n = 923) Morbidity/mortality (n = 72) 
Gender     
 Male 46.5 50.7 46.4 48.6 
 Female 53.5 49.3 53.6 51.4 
Age (years)     
 18–24 7.6 24.7 10.0 1.4 
 25–34 26.1 33.3 20.4 5.6 
 35–44 27.6 21.5 24.3 18.1 
 45–54 22.0 13.7 21.6 27.8 
 55–64 16.7 6.8 23.8 47.2 
Education     
 Primary, basic vocational 26.0 28.0 40.8 53.5 
 Lower secondary 37.1 38.3 36.2 28.2 
 Higher secondary 7.6 6.1 5.7 5.6 
 Higher professional, university 29.3 27.6 17.3 12.7 
Urbanization     
 Rural 17.4 6.8 15.8 13.9 
 Urban 82.6 93.2 84.2 86.1 
Cohabitation status     
 Not living with a partner 29.5 59.4 29.8 30.6 
 Living with a partner 70.5 40.6 70.2 69.4 
Employment status     
 No paid employment 36.3 40.2 44.6 63.9 
 Paid employment 63.7 59.8 55.4 36.1 
Country of birth     
 The Netherlands 93.8 81.3 92.4 90.3 
 Elsewhere 6.2 18.7 7.6 9.7 
One or more somatic disorders     
 No 59.4 61.6 58.7 30.6 
 Yes 40.6 38.4 41.3 69.4 
*

Some percentages do not total 100 because of rounding.

TABLE 3.

Crude odds ratios (OR) and adjusted* odds ratios and 95% confidence intervals (CI) of three types of attrition in the second wave, by demographic characteristics, the Netherlands Mental Health Survey and Incidence Study (NEMESIS), 1997–1998

 Failure to locate
 
Refusal
 
Morbidity/mortality
 
Crude OR Adjusted OR 95% CI Crude OR Adjusted OR 95% CI Crude OR Adjusted OR 95% CI 
Gender          
 Male 1.00 1.00 Reference 1.00 1.00 Reference 1.00 1.00 Reference 
 Female 0.85 0.82 0.61, 1.09 1.01 0.93 0.80, 1.08 0.92 0.63 0.38, 1.03 
Age (years)          
 18–24 1.00 1.00 Reference 1.00 1.00 Reference 1.00 1.00 Reference 
 25–34 0.39 0.56 0.37, 0.84 0.59 0.58 0.44, 0.78 1.16 1.46 0.16, 13.35 
 35–44 0.24 0.38 0.24, 0.59 0.67 0.62 0.47, 0.83 3.56 4.07 0.52, 32.06 
 45–54 0.19 0.27 0.17, 0.45 0.75 0.65 0.49, 0.88 6.88 6.46 0.84, 49.54 
 55–64 0.13 0.17 0.09, 0.32 1.09 0.87 0.65, 1.17 15.40 9.38 1.24, 70.81 
p for trend <0.0001 <0.0001  0.0006   <0.0001 <0.0001  
Education          
 Primary, basic vocational 1.00 1.00 Reference 1.00 1.00 Reference 1.00 1.00 Reference 
 Lower secondary 0.96 0.80 0.56, 1.15 0.62 0.65 0.55, 0.76 0.37 0.54 0.31, 0.94 
 Higher secondary 0.74 0.52 0.28, 0.97 0.47 0.48 0.35, 0.65 0.36 0.55 0.19, 1.59 
 Higher professional, university 0.87 0.63 0.43, 0.93 0.38 0.38 0.31, 0.47 0.21 0.34 0.16, 0.72 
p for trend  0.01  <0.0001 <0.0001  <0.0001 0.003  
Urbanization          
 Rural 1.00 1.00 Reference 1.00 1.00 Reference 1.00 1.00 Reference 
 Urban 2.86 2.37 1.38, 4.06 1.12 1.20 0.99, 1.46 1.31 1.27 0.64, 2.52 
Cohabitation status          
 Not living with a partner 1.00 1.00 Reference 1.00 1.00 Reference 1.00 1.00 Reference 
 Living with a partner 0.29 0.39 0.29, 0.53 0.99 1.01 0.86, 1.20 0.95 0.87 0.51, 1.47 
Employment status          
 No paid employment 1.00 1.00 Reference 1.00 1.00 Reference 1.00 1.00 Reference 
 Paid employment 0.85 0.95 0.69, 1.31 0.71 0.88 0.75, 1.03 0.32 0.55 0.31, 0.97 
Country of birth          
 The Netherlands 1.00 1.00 Reference 1.00 1.00 Reference 1.00 1.00 Reference 
 Elsewhere 3.46 2.86 1.96, 4.18 1.23 1.28 0.98, 1.69 1.62 1.64 0.73, 3.67 
One or more somatic disorders          
 No 1.00 1.00 Reference 1.00 1.00 Reference 1.00 1.00 Reference 
 Yes 0.91 1.10 0.82, 1.48 1.03 0.89 0.77, 1.04 3.33 2.20 1.30, 3.71 
 Failure to locate
 
Refusal
 
Morbidity/mortality
 
Crude OR Adjusted OR 95% CI Crude OR Adjusted OR 95% CI Crude OR Adjusted OR 95% CI 
Gender          
 Male 1.00 1.00 Reference 1.00 1.00 Reference 1.00 1.00 Reference 
 Female 0.85 0.82 0.61, 1.09 1.01 0.93 0.80, 1.08 0.92 0.63 0.38, 1.03 
Age (years)          
 18–24 1.00 1.00 Reference 1.00 1.00 Reference 1.00 1.00 Reference 
 25–34 0.39 0.56 0.37, 0.84 0.59 0.58 0.44, 0.78 1.16 1.46 0.16, 13.35 
 35–44 0.24 0.38 0.24, 0.59 0.67 0.62 0.47, 0.83 3.56 4.07 0.52, 32.06 
 45–54 0.19 0.27 0.17, 0.45 0.75 0.65 0.49, 0.88 6.88 6.46 0.84, 49.54 
 55–64 0.13 0.17 0.09, 0.32 1.09 0.87 0.65, 1.17 15.40 9.38 1.24, 70.81 
p for trend <0.0001 <0.0001  0.0006   <0.0001 <0.0001  
Education          
 Primary, basic vocational 1.00 1.00 Reference 1.00 1.00 Reference 1.00 1.00 Reference 
 Lower secondary 0.96 0.80 0.56, 1.15 0.62 0.65 0.55, 0.76 0.37 0.54 0.31, 0.94 
 Higher secondary 0.74 0.52 0.28, 0.97 0.47 0.48 0.35, 0.65 0.36 0.55 0.19, 1.59 
 Higher professional, university 0.87 0.63 0.43, 0.93 0.38 0.38 0.31, 0.47 0.21 0.34 0.16, 0.72 
p for trend  0.01  <0.0001 <0.0001  <0.0001 0.003  
Urbanization          
 Rural 1.00 1.00 Reference 1.00 1.00 Reference 1.00 1.00 Reference 
 Urban 2.86 2.37 1.38, 4.06 1.12 1.20 0.99, 1.46 1.31 1.27 0.64, 2.52 
Cohabitation status          
 Not living with a partner 1.00 1.00 Reference 1.00 1.00 Reference 1.00 1.00 Reference 
 Living with a partner 0.29 0.39 0.29, 0.53 0.99 1.01 0.86, 1.20 0.95 0.87 0.51, 1.47 
Employment status          
 No paid employment 1.00 1.00 Reference 1.00 1.00 Reference 1.00 1.00 Reference 
 Paid employment 0.85 0.95 0.69, 1.31 0.71 0.88 0.75, 1.03 0.32 0.55 0.31, 0.97 
Country of birth          
 The Netherlands 1.00 1.00 Reference 1.00 1.00 Reference 1.00 1.00 Reference 
 Elsewhere 3.46 2.86 1.96, 4.18 1.23 1.28 0.98, 1.69 1.62 1.64 0.73, 3.67 
One or more somatic disorders          
 No 1.00 1.00 Reference 1.00 1.00 Reference 1.00 1.00 Reference 
 Yes 0.91 1.10 0.82, 1.48 1.03 0.89 0.77, 1.04 3.33 2.20 1.30, 3.71 
*

Odds ratios adjusted for the other demographic variables in the table.

Significant at p < 0.05.

According to both the bivariate and multivariate analyses, there was a clear trend toward a reduced risk for attrition due to failure to locate with increasing age. The multivariate analysis also showed a trend toward a reduced risk for this type of attrition with increasing educational level, although subjects with the highest educational level had somewhat higher odds than subjects with the second highest educational level. Furthermore, failure to locate was more likely to occur among urbanized subjects, those in the noncohabiting category, and those not born in the Netherlands. Failure to locate was not related to gender, employment status, or the presence of somatic disorders.

There was a trend toward an increased risk for attrition due to refusal with increasing age (although the odds for those in the oldest age category were similar to the odds for those in the youngest age category) and lower educational level, but the first trend was significant only in the bivariate analysis. In the bivariate analysis, attrition due to refusal was also significantly related to employment status; people not in paid employment were more likely to refuse to cooperate. In the multivariate analysis, respondents not born in the Netherlands had a 28 percent greater odds of attrition due to refusal than respondents born in the Netherlands, and subjects living in urban areas had a 20 percent greater odds than subjects living in rural areas; however, these associations were not statistically significant at the conventional alpha = 0.05 level. Gender, cohabitation status, and presence of somatic disorders were not related to refusal.

Age, education, employment status, and presence of somatic disorders predicted attrition due to morbidity or mortality in both the bivariate and multivariate analyses. Older subjects and subjects with a lower educational level had an increased risk for attrition due to illness or death. Subjects not in paid employment had higher odds than subjects in paid employment. Subjects with one or more somatic disorders showed a greater likelihood of attrition due to illness or death than those without. In the multivariate analysis, female respondents had a 37 percent lower odds of attrition due to morbidity or mortality than males, but the 95 percent confidence interval overlapped the null hypothesis. Urbanization, cohabitation status, and country of birth were not associated significantly with this type of attrition.

Psychopathology and attrition

Because prevalence rates of psychiatric disorders are known to vary by demographics, we adjusted for gender, age, education, degree of urbanization, cohabitation status, employment status, country of birth, and presence of somatic disorders while investigating the relation between mental disorders and attrition. The odds ratios were calculated in a series of models; each model contained the presence of a separate disorder or a category of disorders and the demographic variables.

For nearly every disorder diagnosed in the 12 months preceding the first interview, we found that the odds ratio of attrition due to failure to locate was greater than 1.0 when compared with absence of the disorder (table 4). Exceptions were panic disorder and, surprisingly, drug abuse. The associations of failure to locate with agoraphobia and with alcohol abuse, and the categories of mood disorders, substance use disorders, and eating disorders, were found to be statistically significant. In addition, the presence of one or more DSM-III-R diagnoses was associated with failure to locate.

TABLE 4.

Adjusted* odds ratios (OR) and 95% confidence intervals (CI) of three types of attrition, by psychopathology ascertained at baseline (12-month prevalences), the Netherlands Mental Health Survey and Incidence Study (NEMESIS), 1997–1998

 12-Month prevalence ascertained at baseline (%) Failure to locate
 
Refusal
 
Morbidity/mortality
 
OR 95% CI OR 95% CI OR 95% CI 
Mood disorders 7.6 1.79 1.19, 2.68 0.94 0.72, 1.24 1.61 0.77, 3.34 
 Major depression 5.8 1.45 0.90, 2.34 0.87 0.63, 1.19 1.38 0.58, 3.30 
 Dysthymia 2.3 1.74 0.87, 3.49 0.92 0.58, 1.45 2.56 1.05, 6.23 
 Bipolar disorder 1.1 1.65 0.64, 4.26 0.74 0.33, 1.65 1.48 0.19, 11.26 
Anxiety disorders 12.4 1.24 0.84, 1.83 0.87 0.70, 1.09 2.75 1.60, 4.73 
 Panic disorder 2.2 0.65 0.23, 1.81 0.67 0.39, 1.14 1.98 0.68, 5.75 
 Agoraphobia (without panic) 1.6 3.97 2.05, 7.69 1.33 0.78, 2.26 3.37 1.27, 8.95 
 Simple phobia 7.1 1.15 0.70, 1.91 0.74 0.55, 0.99 3.03 1.65, 5.59 
 Social phobia 4.8 1.52 0.89, 2.58 1.08 0.78, 1.48 1.90 0.84, 4.28 
 Generalized anxiety disorder 1.2   1.45 0.81, 2.59 2.06 0.46, 9.27 
 Obsessive-compulsive disorder 0.5   0.67 0.20, 2.24 6.44 1.38, 30.05 
Substance use disorders total 8.9 1.56 1.04, 2.34 0.85 0.63, 1.15 1.31 0.50, 3.42 
 Alcohol abuse 4.9 2.04 1.25, 3.34 0.85 0.56, 1.29 1.25 0.29, 5.37 
 Alcohol dependence 3.7 0.99 0.51, 1.91 0.92 0.60, 1.41 1.67 0.50, 5.61 
 Drug abuse 0.5 0.47 0.06, 3.61 0.88 0.26, 3.03   
 Drug dependence 0.8 1.82 0.71, 4.68 0.52 0.18, 1.50 2.32 0.29, 18.58 
Schizophrenia 0.2 1.57 0.19, 12.87 0.57 0.07, 4.49   
Eating disorders 0.4 7.32 2.39, 22.42 1.17 0.33, 4.13   
One or more CIDI§/DSM-III-R§ diagnoses 23.2 1.85 1.38, 2.49 0.93 0.78, 1.11 2.06 1.24, 3.42 
 12-Month prevalence ascertained at baseline (%) Failure to locate
 
Refusal
 
Morbidity/mortality
 
OR 95% CI OR 95% CI OR 95% CI 
Mood disorders 7.6 1.79 1.19, 2.68 0.94 0.72, 1.24 1.61 0.77, 3.34 
 Major depression 5.8 1.45 0.90, 2.34 0.87 0.63, 1.19 1.38 0.58, 3.30 
 Dysthymia 2.3 1.74 0.87, 3.49 0.92 0.58, 1.45 2.56 1.05, 6.23 
 Bipolar disorder 1.1 1.65 0.64, 4.26 0.74 0.33, 1.65 1.48 0.19, 11.26 
Anxiety disorders 12.4 1.24 0.84, 1.83 0.87 0.70, 1.09 2.75 1.60, 4.73 
 Panic disorder 2.2 0.65 0.23, 1.81 0.67 0.39, 1.14 1.98 0.68, 5.75 
 Agoraphobia (without panic) 1.6 3.97 2.05, 7.69 1.33 0.78, 2.26 3.37 1.27, 8.95 
 Simple phobia 7.1 1.15 0.70, 1.91 0.74 0.55, 0.99 3.03 1.65, 5.59 
 Social phobia 4.8 1.52 0.89, 2.58 1.08 0.78, 1.48 1.90 0.84, 4.28 
 Generalized anxiety disorder 1.2   1.45 0.81, 2.59 2.06 0.46, 9.27 
 Obsessive-compulsive disorder 0.5   0.67 0.20, 2.24 6.44 1.38, 30.05 
Substance use disorders total 8.9 1.56 1.04, 2.34 0.85 0.63, 1.15 1.31 0.50, 3.42 
 Alcohol abuse 4.9 2.04 1.25, 3.34 0.85 0.56, 1.29 1.25 0.29, 5.37 
 Alcohol dependence 3.7 0.99 0.51, 1.91 0.92 0.60, 1.41 1.67 0.50, 5.61 
 Drug abuse 0.5 0.47 0.06, 3.61 0.88 0.26, 3.03   
 Drug dependence 0.8 1.82 0.71, 4.68 0.52 0.18, 1.50 2.32 0.29, 18.58 
Schizophrenia 0.2 1.57 0.19, 12.87 0.57 0.07, 4.49   
Eating disorders 0.4 7.32 2.39, 22.42 1.17 0.33, 4.13   
One or more CIDI§/DSM-III-R§ diagnoses 23.2 1.85 1.38, 2.49 0.93 0.78, 1.11 2.06 1.24, 3.42 
*

Odds ratios adjusted for gender, age, education, urbanization, cohabitation status, employment status, country of birth, and presence of somatic disorders.

Significant at p < 0.05.

Odds ratio could not be calculated in the attrition group because the prevalence approximated zero.

§

CIDI, Composite International Diagnostic Interview; DSM-III-R, Diagnostic and Statistical Manual of Mental Disorders, third edition, revised.

Contrary to failure to locate, the odds ratio of attrition due to refusal was less than 1.0 for most disorders. Refusal was found less often among subjects diagnosed with a disorder in the 12 months before the first interview than among those without a disorder, with the exception of agoraphobia, social phobia, generalized anxiety disorder, and the category of eating disorders. The odds ratios were the lowest for drug dependence and schizophrenia. Refusal was associated significantly with the presence of just one disorder, simple phobia, and with none of the categories of disorders or with having one or more diagnoses.

For all disorders, we found that morbidity/mortality was evident more often among subjects diagnosed with a disorder than among subjects not diagnosed with that disorder. Morbidity/mortality was significantly associated with the presence of dysthymia, agoraphobia, simple phobia, and obsessive-compulsive disorder. Some of the confidence intervals for this type of attrition were wide because of the small number of respondents lost to illness or death (n = 72) and the low prevalences of psychiatric disorders. The general category of anxiety disorders and the presence of one or more CIDI/DSM-III-R disorders also predicted attrition due to morbidity or mortality.

These analyses also were carried out by using the presence of lifetime disorders as predictors of attrition (not shown in table). Failure to locate was predicted significantly only by agoraphobia (odds ratio (OR) = 1.94, 95 percent confidence interval (CI): 1.06, 3.54), the category of eating disorders (OR = 4.63, 95 percent CI: 2.00, 10.72), and the presence of one or more disorders (OR = 1.40, 95 percent CI: 1.05, 1.87). Simple phobia still predicted a significantly smaller likelihood of refusal (OR = 0.70, 95 percent CI: 0.54, 0.90), but refusal also was seen less often in subjects with lifetime major depression (OR = 0.77, 95 percent CI: 0.62, 0.95), dysthymia (OR = 0.72, 95 percent CI: 0.53, 0.97), drug abuse (OR = 0.42, 95 percent CI: 0.18, 0.98), and the main category of mood disorders (OR = 0.76, 95 percent CI: 0.62, 0.92). For morbidity/mortality, the only significant predictors were simple phobia (OR = 2.73, 95 percent CI: 1.55, 4.82) and the category of anxiety disorders (OR = 1.97, 95 percent CI: 1.18, 3.28).

Number of psychiatric disorders and attrition

We next investigated whether attrition was related to the number of psychiatric disorders a participant had had in the 12 months before the first interview. Again, we adjusted for demographic variables. Five comorbidity categories were used: none, one, two, three, and four or more disorders. Having no disorder was the reference category (table 5). The odds for respondents with one or with four or more disorders were high (OR = 2.09 for both categories), while respondents with two or with three disorders had relatively low odds (OR = 1.27 and OR = 1.29, respectively). No consistent pattern for refusal was seen in relation to number of disorders. Attrition due to morbidity or mortality was more likely for those with two or more disorders, and a trend was suggested toward an increased odds of this type of attrition with a higher number of disorders. When we used lifetime disorders in the analyses, similar results were found (not shown in table), with the exception of morbidity/mortality, which was more likely for only those with four or more disorders (OR = 2.80, 95 percent CI: 1.28, 6.12).

TABLE 5.

Adjusted* odds ratios (OR) and 95% confidence intervals (CI) of three types of attrition, by number of psychiatric disorders ascertained at baseline (12-month prevalences), the Netherlands Mental Health Survey and Incidence Study (NEMESIS), 1997–1998

 12-Month prevalence ascertained at baseline (%) Failure to locate
 
Refusal
 
Morbidity/mortality
 
 OR 95% CI OR 95% CI OR 95% CI 
No. of psychiatric disorders        
 0 76.5 1.00 Reference 1.00 Reference 1.00 Reference 
 1 15.3 2.09 1.51, 2.91 0.97 0.79, 1.19 1.64 0.85, 3.16 
 2 4.4 1.27 0.68, 2.37 0.91 0.64, 1.31 3.09 1.33, 7.18 
 3 1.9 1.29 0.54, 3.07 0.68 0.38, 1.20 4.10 1.51, 11.10 
 ≥4 1.9 2.09 0.99, 4.39 0.82 0.47, 1.43 3.50 1.18, 10.37 
p for trend  0.005    0.0001  
 12-Month prevalence ascertained at baseline (%) Failure to locate
 
Refusal
 
Morbidity/mortality
 
 OR 95% CI OR 95% CI OR 95% CI 
No. of psychiatric disorders        
 0 76.5 1.00 Reference 1.00 Reference 1.00 Reference 
 1 15.3 2.09 1.51, 2.91 0.97 0.79, 1.19 1.64 0.85, 3.16 
 2 4.4 1.27 0.68, 2.37 0.91 0.64, 1.31 3.09 1.33, 7.18 
 3 1.9 1.29 0.54, 3.07 0.68 0.38, 1.20 4.10 1.51, 11.10 
 ≥4 1.9 2.09 0.99, 4.39 0.82 0.47, 1.43 3.50 1.18, 10.37 
p for trend  0.005    0.0001  
*

Odds ratios adjusted for gender, age, education, urbanization, cohabitation status, employment status, country of birth, and presence of somatic disorders.

Significant at p < 0.05.

Overall attrition

Since NEMESIS is a study of mental health, it was crucial to know whether the total attrition in the second wave (n = 1,458) was a function of the presence of psychiatric disorders. Again, odds ratios were calculated in a series of models in which each disorder or each category of disorders occurring within 12 months prior to the baseline interview was entered into separate models along with the demographic variables (not shown in a table). All demographic characteristics were associated with total attrition, except for gender and somatic disorders. Compared with persons reinterviewed successfully, those lost to follow-up were more often younger, poorly educated, urban, not cohabiting with a steady partner, not in paid employment, and born outside the Netherlands. Adjustment for these demographic variables showed that only agoraphobia (OR = 1.96, 95 percent CI: 1.33, 2.90), social phobia (OR = 1.37, 95 percent CI: 1.07, 1.75), and the category of eating disorders (OR = 2.96, 95 percent CI: 1.35, 6.49) were associated with an increased likelihood of being lost to follow-up. The presence of at least one disorder was also related to attrition; however, this association was weak (OR = 1.20, 95 percent CI: 1.04, 1.38). We concluded that psychopathology has only a limited added effect on overall attrition, after adjustment for demographic influences.

When lifetime disorders were used in these analyses, social phobia (OR = 1.28, 95 percent CI: 1.04, 1.57) as well as drug abuse (OR = 0.51, 95 percent CI: 0.28, 0.93) were associated with attrition. None of the categories of disorders, nor the presence of at least one disorder, was associated with attrition.

DISCUSSION

We investigated whether certain psychopathologic and sociodemographic characteristics of respondents are related to loss to follow-up in a longitudinal study on mental health. The follow-up response rate of 79.4 percent in the second wave of the NEMESIS study was very near that in the ECA (80.2 percent) (25). The three sources of attrition that were distinguished–failure to locate respondents, respondents' refusal to participate further, and morbidity/mortality–showed a number of links to demographic factors. Notably, however, no statistically significant association was found between gender and any type of attrition, although gender was associated with nonresponse in the first wave of the survey (12). The group aged 18–24 years–which, as in many other studies (1), was also underrepresented in the first wave of NEMESIS–was prone to further shrinkage in the second wave caused by refusal and failure to locate. Nonresponse and loss to follow-up among young adults can be a particular source of bias in relation to problems such as substance use disorder, which are more common in this age group (23). In future research, special efforts should be made to recruit and track young adults.

In agreement with Eaton et al. (8), we found that psychopathology, adjusted for demographic factors, has no more than a weak-to-moderate effect on attrition and that psychopathology is more strongly related to failure to locate respondents than to refusal. Only one disorder in the 12 months before baseline was found to be related significantly to refusal in either study–simple phobia in NEMESIS and depression in the ECA (8)–but, in both cases, the disorder was linked to a reduced likelihood of refusal. In neither study did the number of disorders influence refusal at follow-up. In addition, Badawi et al. (11) and Farmer et al. (3) found that psychopathology and that depressive symptomatology, respectively, were linked to failure to locate but not to refusal. In an overview of many studies on a range of issues, Ribisl et al. (1) similarly concluded that participants who are not located at follow-up introduce more systematic bias in the dependent variable measured at the second wave compared with participants who refuse to be reinterviewed. We showed that subjects with a lifetime history of major depression, dysthymia, simple phobia, or drug abuse were even more willing to stay in this study on psychopathology than subjects without such a history.

A few differences were found between the ECA (8) and NEMESIS in relation to the specific 12-month disorders associated with not-located respondents. The former study found that panic disorder and major depression were linked to this source of attrition. The latter found that agoraphobia and the category of mood disorders (although not major depression by itself) were implicated. These differences may have to do with different samples and different sample sizes of the two surveys. Eaton et al. (8) also found an association between failure to locate and drug abuse/dependence, alcohol abuse/dependence, and antisocial personality disorder. Our study examined (alcohol and drug) abuse and dependence separately, and only alcohol abuse was found to be related. Antisocial personality was not included in our study.

In NEMESIS, we were also able to study attrition due to morbidity/mortality, albeit among limited numbers of participants. It was significantly linked to dysthymia, agoraphobia, simple phobia, and obsessive-compulsive disorder and to the category of anxiety disorders.

The results of our study should be interpreted within the context of its limitations. First, in analyses such as these, the determinants of attrition are identified in terms of baseline data, while the dependent variable attrition is measured in the second wave of the study. Demographic data are unlikely to change much between the two assessments, but psychopathology may be subject to greater variation (even though many disorders are chronic). Second, we used DSM-III-R psychopathology as a predictor of attrition. Mental health problems below the clinical level might be associated with attrition as well. Third, because we analyzed numerous independent variables and three outcome variables, some associations might be significant by chance.

How much bias causes the attrition associated with psychopathology to affect the measurements at the second wave, such as prevalence and incidence rates? These prevalences and incidences differ only slightly from those controlled for attrition due to psychopathology at baseline. For example, in the general population, the 12-month prevalence of one or more diagnoses at the second wave, weighted for demographics, was 15.3 percent. When we also weighted for the presence of at least one disorder in the 12 months before the first wave, this prevalence was 15.5 percent.

Methodological research is important to minimize attrition in future longitudinal studies (26). As the present analysis has shown, epidemiologic research on mental health should pay extra attention to participants likely to be more difficult to trace in follow-up waves. If researchers keep in touch with the groups in question, they may succeed in tracking down more of the difficult-to-reach respondents, which will enhance the validity of the survey outcomes.

Correspondence to Ron de Graaf, Netherlands Institute of Mental Health and Addiction, Da Costakade 45, 3521 VS Utrecht, the Netherlands (e-mail: rgraaf@trimbos.nl).

NEMESIS is being conducted by the Netherlands Institute of Mental Health and Addiction (Trimbos Institute) in Utrecht. Financial support has been received from the Netherlands Ministry of Health, Welfare and Sport (VWS), the Medical Sciences Department of the Netherlands Organization for Scientific Research (NWO), and the National Institute for Public Health and Environment (RIVM).

The authors thank Dr. J. Ormel (University of Groningen), Dr. W. van den Brink (University of Amsterdam), Dr. H. Verkleij (RIVM), and S. Laitinen-Krispijn (Trimbos Institute) for their comments on previous versions of this manuscript.

REFERENCES

1.
Ribisl KM, Walton MA, Mowbray CT, et al. Minimizing participant attrition in panel studies through the use of effective retention and tracking strategies: review and recommendations.
Eval Program Plan
 
1996
;
19
:
1
–25.
2.
Miller RB, Wright DW. Detecting and correcting attrition bias in longitudinal family research.
J Marriage Fam
 
1995
;
57
:
921
–9.
3.
Farmer ME, Locke BZ, Liu IY, et al. Depressive symptoms and attrition: the NHANES I epidemiologic follow-up study.
Int J Methods Psychiatr Res
 
1994
;
4
:
19
–27.
4.
Kessler RC, Little RJA, Groves RM. Advances in strategies for minimizing and adjusting for survey nonresponse.
Epidemiol Rev
 
1995
;
17
:
192
–204.
5.
Bongers IMB, Lemmens PHHM, Van Oers HAM, et al. Methode ter correctie voor vertekening van surveyresultaten ten gevolge van indirecte selectieve non-respons. (In Dutch).
T Soc Gezondheidsz
 
1997
;
75
:
122
–8.
6.
Coen AS, Patrick DC, Shern DL. Minimizing attrition in longitudinal studies of special populations: an integrated management approach.
Eval Program Plan
 
1996
;
19
:
309
–19.
7.
Cottler LB, Zipp JF, Robins LN, et al. Difficult-to-recruit respondents and their effect on prevalence estimates in an epidemiologic survey.
Am J Epidemiol
 
1987
;
125
:
329
–39.
8.
Eaton WW, Anthony JC, Tepper S, et al. Psychopathology and attrition in the Epidemiologic Catchment Area Surveys.
Am J Epidemiol
 
1992
;
135
:
1051
–9.
9.
Murphy JM, Monson RR, Olivier DC, et al. Affective disorders and mortality: a general population study.
Arch Gen Psychiatry
 
1987
;
44
:
473
–80.
10.
Rorsman B, Grasbeck A, Hagnell O, et al. Premorbid personality traits and psychosomatic background factors in depression: the Lundby Study 1957–1972.
Neuropsychobiology
 
1993
;
27
:
72
–9.
11.
Badawi MA, Eaton WW, Myllyluoma J, et al. Psychopathology and attrition in the Baltimore ECA 15-year follow-up 1981–1996.
Soc Psychiatry Psychiatr Epidemiol
 
1994
;
34
:
91
–8.
12.
Bijl RV, Van Zessen G, Ravelli A. The Netherlands Mental Health Survey and Incidence Study (NEMESIS): objectives and design.
Soc Psychiatry Psychiatr Epidemiol
 
1998
;
33
:
581
–6.
13.
Goldberg DP, Williams P. A users guide to the General Health Questionnaire. Windsor, United Kingdom: Nelson, 1988.
14.
American Psychiatric Association. Diagnostic and statistical manual of mental disorders: DSM-III-R. 3rd ed, rev. Washington, DC: American Psychiatric Association, 1987.
15.
Smeets RMW, Dingemans PMAJ. Composite International Diagnostic Interview (CIDI), version 1.1. Amsterdam, the Netherlands: World Health Organization, 1993.
16.
World Health Organization. Composite International Diagnostic Interview (CIDI), version 1.0. Geneva, Switzerland: World Health Organization, 1990.
17.
Robins LN, Wing J, Wittchen HU, et al. The Composite International Diagnostic Interview: an epidemiologic instrument suitable for use in conjunction with different diagnostic systems and in different cultures.
Arch Gen Psychiatry
 
1988
;
45
:
1069
–77.
18.
Wittchen HU, Robins LN, Cottler LB, et al. Cross-cultural feasibility, reliability and sources of variance in the Composite International Diagnostic Interview (CIDI).
Br J Psychiatry
 
1991
;
159
:
645
–53.
19.
Cottler LB, Robins LN, Grant BF, et al. The CIDI-core substance abuse and dependence questions: cross-cultural and nosological issues.
Br J Psychiatry
 
1991
;
159
:
653
–8.
20.
Semler G, von Cranach M, Wittchen HU. Comparison between the Composite International Diagnostic Interview and the Present State Examination. Report to the WHO/ADAMHA Task Force on Instrument Development. Geneva, Switzerland: World Health Organization, 1987.
21.
Wacker HR, Battegay R, Mullejans R, et al. Using the CIDI-C in the general population. In: Stefanis CN, Rabavilas AD, Soldatos CR, eds. Psychiatry: a world perspective. Amsterdam, the Netherlands: Elsevier Science Publishers, 1990:138–43.
22.
Wittchen HU, Burke JD, Semler G, et al. Recall and dating of psychiatric symptoms: test-retest reliability of time-related symptom questions in a standardized psychiatric interview.
Arch Gen Psychiatry
 
1989
;
46
:
437
–43.
23.
Bijl RV, Ravelli A, van Zessen G. Prevalence of psychiatric disorder in the general population: results of The Netherlands Mental Health Survey and Incidence Study (NEMESIS).
Soc Psychiatry Psychiatr Epidemiol
 
1998
;
33
:
587
–95.
24.
Kessler RC, McGonagle KA, Zhao S, et al. Lifetime and 12-month prevalence of DSM-III-R psychiatric disorders in the United States: results from the National Comorbidity Survey.
Arch Gen Psychiatry
 
1994
;
51
:
8
–19.
25.
Eaton WW, Kramer M, Anthony JC, et al. The incidence of specific DIS/DSM-III mental disorders: data from the NIMH Epidemiologic Catchment Area Program.
Acta Psychiatr Scand
 
1989
;
79
:
163
–78.
26.
Farrington DP. Longitudinal research strategies: advantages, problems, and prospects.
J Am Acad Child Adolesc Psychiatry
 
1991
;
30
:
369
–74.